2023-07-21T18:19:06.370 INFO:root:teuthology version: 0.0.1.dev153+g79491cb 2023-07-21T18:19:06.371 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} --verbose --name rishabh-2023-07-21_17:58:19-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi --owner scheduled_rishabh@teuthology --archive /home/teuthworker/archive/rishabh-2023-07-21_17:58:19-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7346865 -- /home/teuthworker/archive/rishabh-2023-07-21_17:58:19-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7346865/orig.config.yaml 2023-07-21T18:19:06.431 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-21T18:19:06.554 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2023-07-21_17:58:19-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7346865 branch: wip-rishabh-2023Jul13-m-quota description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} email: null first_in_suite: false job_id: '7346865' kernel: client: sha1: distro flavor: default kdb: true sha1: c685da6f56cc77f2cfb9c4221be3960522616caf ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2023-07-21_17:58:19-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-rishabh-2023Jul13-m-quota ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request sha1: 81cf785aa7781c4e08381c540f890cb9bc987ef9 subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: 81cf785aa7781c4e08381c540f890cb9bc987ef9 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - nowsync syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-2023Jul13-m-quota sha1: 81cf785aa7781c4e08381c540f890cb9bc987ef9 owner: scheduled_rishabh@teuthology priority: 80 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 sha1: 81cf785aa7781c4e08381c540f890cb9bc987ef9 sleep_before_teardown: 0 suite: fs suite_branch: wip-rishabh-2023Jul13-m-quota suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_81cf785aa7781c4e08381c540f890cb9bc987ef9/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 81cf785aa7781c4e08381c540f890cb9bc987ef9 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi039.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= smithi155.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi193.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1M - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6M3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 79491cb69dd37a1eafb93b3471998174bdfa2fd9 timestamp: 2023-07-21_17:58:19 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.23459 2023-07-21T18:19:06.555 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_81cf785aa7781c4e08381c540f890cb9bc987ef9/qa; will attempt to use it 2023-07-21T18:19:06.556 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_81cf785aa7781c4e08381c540f890cb9bc987ef9/qa/tasks 2023-07-21T18:19:06.556 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-07-21T18:19:06.558 INFO:teuthology.task.internal:Checking packages... 2023-07-21T18:19:06.587 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '81cf785aa7781c4e08381c540f890cb9bc987ef9' 2023-07-21T18:19:06.588 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-07-21T18:19:06.588 INFO:teuthology.packaging:ref: None 2023-07-21T18:19:06.588 INFO:teuthology.packaging:tag: None 2023-07-21T18:19:06.589 INFO:teuthology.packaging:branch: wip-rishabh-2023Jul13-m-quota 2023-07-21T18:19:06.589 INFO:teuthology.packaging:sha1: 81cf785aa7781c4e08381c540f890cb9bc987ef9 2023-07-21T18:19:06.589 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-rishabh-2023Jul13-m-quota 2023-07-21T18:19:06.814 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-4746.g81cf785a 2023-07-21T18:19:06.814 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-07-21T18:19:06.820 INFO:teuthology.task.internal:no buildpackages task found 2023-07-21T18:19:06.820 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-07-21T18:19:06.848 INFO:teuthology.task.internal:Saving configuration 2023-07-21T18:19:06.877 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-07-21T18:19:06.883 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-21T18:19:06.913 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi039.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-21_17:58:19-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7346865', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-21 18:12:12.109513', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-21T18:19:06.941 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi155.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-21_17:58:19-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7346865', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-21 18:12:12.107384', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-21T18:19:06.967 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi193.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-21_17:58:19-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7346865', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-21 18:12:12.110764', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-21T18:19:06.967 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-07-21T18:19:06.973 INFO:teuthology.task.internal:roles: ubuntu@smithi039.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2023-07-21T18:19:06.974 INFO:teuthology.task.internal:roles: ubuntu@smithi155.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2023-07-21T18:19:06.974 INFO:teuthology.task.internal:roles: ubuntu@smithi193.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2023-07-21T18:19:06.974 INFO:teuthology.run_tasks:Running task console_log... 2023-07-21T18:19:07.121 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fd90bc9e5e0>, signals=[15]) 2023-07-21T18:19:07.123 INFO:teuthology.run_tasks:Running task internal.connect... 2023-07-21T18:19:07.131 INFO:teuthology.task.internal:Opening connections... 2023-07-21T18:19:07.132 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi039.front.sepia.ceph.com 2023-07-21T18:19:07.134 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T18:19:07.207 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi155.front.sepia.ceph.com 2023-07-21T18:19:07.208 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T18:19:07.275 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi193.front.sepia.ceph.com 2023-07-21T18:19:07.276 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T18:19:07.387 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-07-21T18:19:07.395 DEBUG:teuthology.orchestra.run.smithi039:> uname -m 2023-07-21T18:19:07.416 INFO:teuthology.orchestra.run.smithi039.stdout:x86_64 2023-07-21T18:19:07.417 DEBUG:teuthology.orchestra.run.smithi039:> cat /etc/os-release 2023-07-21T18:19:07.476 INFO:teuthology.orchestra.run.smithi039.stdout:NAME="Red Hat Enterprise Linux" 2023-07-21T18:19:07.477 INFO:teuthology.orchestra.run.smithi039.stdout:VERSION="8.6 (Ootpa)" 2023-07-21T18:19:07.477 INFO:teuthology.orchestra.run.smithi039.stdout:ID="rhel" 2023-07-21T18:19:07.477 INFO:teuthology.orchestra.run.smithi039.stdout:ID_LIKE="fedora" 2023-07-21T18:19:07.477 INFO:teuthology.orchestra.run.smithi039.stdout:VERSION_ID="8.6" 2023-07-21T18:19:07.477 INFO:teuthology.orchestra.run.smithi039.stdout:PLATFORM_ID="platform:el8" 2023-07-21T18:19:07.478 INFO:teuthology.orchestra.run.smithi039.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-07-21T18:19:07.478 INFO:teuthology.orchestra.run.smithi039.stdout:ANSI_COLOR="0;31" 2023-07-21T18:19:07.478 INFO:teuthology.orchestra.run.smithi039.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-07-21T18:19:07.478 INFO:teuthology.orchestra.run.smithi039.stdout:HOME_URL="https://www.redhat.com/" 2023-07-21T18:19:07.478 INFO:teuthology.orchestra.run.smithi039.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-21T18:19:07.478 INFO:teuthology.orchestra.run.smithi039.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-21T18:19:07.478 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:19:07.479 INFO:teuthology.orchestra.run.smithi039.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-21T18:19:07.479 INFO:teuthology.orchestra.run.smithi039.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-07-21T18:19:07.479 INFO:teuthology.orchestra.run.smithi039.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-21T18:19:07.479 INFO:teuthology.orchestra.run.smithi039.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-07-21T18:19:07.480 INFO:teuthology.lock.ops:Updating smithi039.front.sepia.ceph.com on lock server 2023-07-21T18:19:07.515 DEBUG:teuthology.orchestra.run.smithi155:> uname -m 2023-07-21T18:19:07.537 INFO:teuthology.orchestra.run.smithi155.stdout:x86_64 2023-07-21T18:19:07.538 DEBUG:teuthology.orchestra.run.smithi155:> cat /etc/os-release 2023-07-21T18:19:07.597 INFO:teuthology.orchestra.run.smithi155.stdout:NAME="Red Hat Enterprise Linux" 2023-07-21T18:19:07.597 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION="8.6 (Ootpa)" 2023-07-21T18:19:07.598 INFO:teuthology.orchestra.run.smithi155.stdout:ID="rhel" 2023-07-21T18:19:07.598 INFO:teuthology.orchestra.run.smithi155.stdout:ID_LIKE="fedora" 2023-07-21T18:19:07.598 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION_ID="8.6" 2023-07-21T18:19:07.598 INFO:teuthology.orchestra.run.smithi155.stdout:PLATFORM_ID="platform:el8" 2023-07-21T18:19:07.598 INFO:teuthology.orchestra.run.smithi155.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-07-21T18:19:07.598 INFO:teuthology.orchestra.run.smithi155.stdout:ANSI_COLOR="0;31" 2023-07-21T18:19:07.598 INFO:teuthology.orchestra.run.smithi155.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-07-21T18:19:07.599 INFO:teuthology.orchestra.run.smithi155.stdout:HOME_URL="https://www.redhat.com/" 2023-07-21T18:19:07.599 INFO:teuthology.orchestra.run.smithi155.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-21T18:19:07.599 INFO:teuthology.orchestra.run.smithi155.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-21T18:19:07.599 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-07-21T18:19:07.599 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-21T18:19:07.599 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-07-21T18:19:07.600 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-21T18:19:07.600 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-07-21T18:19:07.600 INFO:teuthology.lock.ops:Updating smithi155.front.sepia.ceph.com on lock server 2023-07-21T18:19:07.624 DEBUG:teuthology.orchestra.run.smithi193:> uname -m 2023-07-21T18:19:07.644 INFO:teuthology.orchestra.run.smithi193.stdout:x86_64 2023-07-21T18:19:07.645 DEBUG:teuthology.orchestra.run.smithi193:> cat /etc/os-release 2023-07-21T18:19:07.703 INFO:teuthology.orchestra.run.smithi193.stdout:NAME="Red Hat Enterprise Linux" 2023-07-21T18:19:07.703 INFO:teuthology.orchestra.run.smithi193.stdout:VERSION="8.6 (Ootpa)" 2023-07-21T18:19:07.703 INFO:teuthology.orchestra.run.smithi193.stdout:ID="rhel" 2023-07-21T18:19:07.703 INFO:teuthology.orchestra.run.smithi193.stdout:ID_LIKE="fedora" 2023-07-21T18:19:07.703 INFO:teuthology.orchestra.run.smithi193.stdout:VERSION_ID="8.6" 2023-07-21T18:19:07.704 INFO:teuthology.orchestra.run.smithi193.stdout:PLATFORM_ID="platform:el8" 2023-07-21T18:19:07.704 INFO:teuthology.orchestra.run.smithi193.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-07-21T18:19:07.704 INFO:teuthology.orchestra.run.smithi193.stdout:ANSI_COLOR="0;31" 2023-07-21T18:19:07.704 INFO:teuthology.orchestra.run.smithi193.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-07-21T18:19:07.704 INFO:teuthology.orchestra.run.smithi193.stdout:HOME_URL="https://www.redhat.com/" 2023-07-21T18:19:07.704 INFO:teuthology.orchestra.run.smithi193.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-21T18:19:07.704 INFO:teuthology.orchestra.run.smithi193.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-21T18:19:07.705 INFO:teuthology.orchestra.run.smithi193.stdout: 2023-07-21T18:19:07.705 INFO:teuthology.orchestra.run.smithi193.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-21T18:19:07.705 INFO:teuthology.orchestra.run.smithi193.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-07-21T18:19:07.705 INFO:teuthology.orchestra.run.smithi193.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-21T18:19:07.705 INFO:teuthology.orchestra.run.smithi193.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-07-21T18:19:07.705 INFO:teuthology.lock.ops:Updating smithi193.front.sepia.ceph.com on lock server 2023-07-21T18:19:07.740 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-07-21T18:19:07.751 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-07-21T18:19:07.773 INFO:teuthology.task.internal:Checking for old test directory... 2023-07-21T18:19:07.773 DEBUG:teuthology.orchestra.run.smithi039:> test '!' -e /home/ubuntu/cephtest 2023-07-21T18:19:07.776 DEBUG:teuthology.orchestra.run.smithi155:> test '!' -e /home/ubuntu/cephtest 2023-07-21T18:19:07.778 DEBUG:teuthology.orchestra.run.smithi193:> test '!' -e /home/ubuntu/cephtest 2023-07-21T18:19:07.796 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-07-21T18:19:07.802 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-07-21T18:19:07.802 DEBUG:teuthology.orchestra.run.smithi039:> test -z $(ls -A /var/lib/ceph) 2023-07-21T18:19:07.835 DEBUG:teuthology.orchestra.run.smithi155:> test -z $(ls -A /var/lib/ceph) 2023-07-21T18:19:07.838 DEBUG:teuthology.orchestra.run.smithi193:> test -z $(ls -A /var/lib/ceph) 2023-07-21T18:19:07.864 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-07-21T18:19:07.959 INFO:teuthology.run_tasks:Running task kernel... 2023-07-21T18:19:07.973 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': 'c685da6f56cc77f2cfb9c4221be3960522616caf'} 2023-07-21T18:19:07.974 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2023-07-21T18:19:07.974 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2023-07-21T18:19:07.975 DEBUG:teuthology.orchestra.run.smithi039:> test -f /run/.containerenv -o -f /.dockerenv 2023-07-21T18:19:07.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T18:19:07.995 DEBUG:teuthology.orchestra.run.smithi039:> uname -r 2023-07-21T18:19:08.054 INFO:teuthology.orchestra.run.smithi039.stdout:4.18.0-372.9.1.el8.x86_64 2023-07-21T18:19:08.055 INFO:teuthology.task.kernel:Running kernel on smithi039: 4.18.0-372.9.1.el8.x86_64 2023-07-21T18:19:08.055 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum install -y kernel 2023-07-21T18:19:09.246 INFO:teuthology.orchestra.run.smithi039.stdout:Updating Subscription Management repositories. 2023-07-21T18:19:09.247 INFO:teuthology.orchestra.run.smithi039.stdout:Unable to read consumer identity 2023-07-21T18:19:10.693 INFO:teuthology.orchestra.run.smithi039.stdout:Copr repo for python3-asyncssh owned by ceph 15 kB/s | 3.5 kB 00:00 2023-07-21T18:19:15.135 INFO:teuthology.orchestra.run.smithi039.stdout:Extra Packages for Enterprise Linux 3.5 MB/s | 16 MB 00:04 2023-07-21T18:19:18.852 INFO:teuthology.orchestra.run.smithi039.stdout:lab-extras 363 kB/s | 24 kB 00:00 2023-07-21T18:19:19.671 INFO:teuthology.orchestra.run.smithi039.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-07-21T18:19:19.732 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2023-07-21T18:19:19.733 INFO:teuthology.orchestra.run.smithi039.stdout:Nothing to do. 2023-07-21T18:19:19.734 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2023-07-21T18:19:19.861 DEBUG:teuthology.orchestra.run.smithi039:> echo no | sudo yum reinstall kernel || true 2023-07-21T18:19:20.210 INFO:teuthology.orchestra.run.smithi039.stdout:Updating Subscription Management repositories. 2023-07-21T18:19:20.211 INFO:teuthology.orchestra.run.smithi039.stdout:Unable to read consumer identity 2023-07-21T18:19:20.484 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:00:02 ago on Fri 21 Jul 2023 06:19:18 PM UTC. 2023-07-21T18:19:20.530 INFO:teuthology.orchestra.run.smithi039.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-07-21T18:19:20.569 INFO:teuthology.orchestra.run.smithi039.stderr:Error: No packages marked for reinstall. 2023-07-21T18:19:20.616 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum reinstall -y kernel || true 2023-07-21T18:19:20.966 INFO:teuthology.orchestra.run.smithi039.stdout:Updating Subscription Management repositories. 2023-07-21T18:19:20.966 INFO:teuthology.orchestra.run.smithi039.stdout:Unable to read consumer identity 2023-07-21T18:19:21.235 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:00:03 ago on Fri 21 Jul 2023 06:19:18 PM UTC. 2023-07-21T18:19:21.276 INFO:teuthology.orchestra.run.smithi039.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-07-21T18:19:21.421 INFO:teuthology.orchestra.run.smithi039.stderr:Error: No packages marked for reinstall. 2023-07-21T18:19:21.468 DEBUG:teuthology.orchestra.run.smithi039:> rpm -q kernel | sort -rV | head -n 1 2023-07-21T18:19:21.551 INFO:teuthology.orchestra.run.smithi039.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-07-21T18:19:21.551 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-07-21T18:19:21.552 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-07-21T18:19:21.552 DEBUG:teuthology.parallel:result is None 2023-07-21T18:19:21.552 INFO:teuthology.run_tasks:Running task internal.base... 2023-07-21T18:19:21.560 INFO:teuthology.task.internal:Creating test directory... 2023-07-21T18:19:21.560 DEBUG:teuthology.orchestra.run.smithi039:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-07-21T18:19:21.562 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-07-21T18:19:21.565 DEBUG:teuthology.orchestra.run.smithi193:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-07-21T18:19:21.585 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-07-21T18:19:21.598 INFO:teuthology.run_tasks:Running task internal.archive... 2023-07-21T18:19:21.623 INFO:teuthology.task.internal:Creating archive directory... 2023-07-21T18:19:21.623 DEBUG:teuthology.orchestra.run.smithi039:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-07-21T18:19:21.626 DEBUG:teuthology.orchestra.run.smithi155:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-07-21T18:19:21.628 DEBUG:teuthology.orchestra.run.smithi193:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-07-21T18:19:21.665 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-07-21T18:19:21.672 INFO:teuthology.task.internal:Enabling coredump saving... 2023-07-21T18:19:21.672 DEBUG:teuthology.orchestra.run.smithi155:> test -f /run/.containerenv -o -f /.dockerenv 2023-07-21T18:19:21.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T18:19:21.712 DEBUG:teuthology.orchestra.run.smithi193:> test -f /run/.containerenv -o -f /.dockerenv 2023-07-21T18:19:21.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T18:19:21.729 DEBUG:teuthology.orchestra.run.smithi039:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-07-21T18:19:21.731 DEBUG:teuthology.orchestra.run.smithi155:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-07-21T18:19:21.754 DEBUG:teuthology.orchestra.run.smithi193:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-07-21T18:19:21.761 INFO:teuthology.orchestra.run.smithi039.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-21T18:19:21.772 INFO:teuthology.orchestra.run.smithi039.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-21T18:19:21.799 INFO:teuthology.orchestra.run.smithi155.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-21T18:19:21.812 INFO:teuthology.orchestra.run.smithi193.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-21T18:19:21.831 INFO:teuthology.orchestra.run.smithi193.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-21T18:19:21.848 INFO:teuthology.orchestra.run.smithi155.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-21T18:19:21.849 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-07-21T18:19:21.856 INFO:teuthology.task.internal:Configuring sudo... 2023-07-21T18:19:21.857 DEBUG:teuthology.orchestra.run.smithi039:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-07-21T18:19:21.859 DEBUG:teuthology.orchestra.run.smithi155:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-07-21T18:19:21.861 DEBUG:teuthology.orchestra.run.smithi193:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-07-21T18:19:21.901 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-07-21T18:19:21.910 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-07-21T18:19:21.911 DEBUG:teuthology.orchestra.run.smithi039:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-07-21T18:19:21.926 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-07-21T18:19:21.930 DEBUG:teuthology.orchestra.run.smithi193:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-07-21T18:19:21.958 DEBUG:teuthology.orchestra.run.smithi039:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-21T18:19:27.031 DEBUG:teuthology.orchestra.run.smithi039:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-21T18:19:27.068 DEBUG:teuthology.orchestra.run.smithi039:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-21T18:19:35.290 DEBUG:teuthology.orchestra.run.smithi039:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-21T18:19:35.322 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-07-21T18:19:35.323 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-07-21T18:19:35.395 DEBUG:teuthology.orchestra.run.smithi155:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-21T18:19:35.444 DEBUG:teuthology.orchestra.run.smithi155:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-21T18:19:35.500 DEBUG:teuthology.orchestra.run.smithi155:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-21T18:19:35.583 DEBUG:teuthology.orchestra.run.smithi155:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-21T18:19:35.630 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-07-21T18:19:35.631 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-07-21T18:19:35.718 DEBUG:teuthology.orchestra.run.smithi193:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-21T18:19:35.763 DEBUG:teuthology.orchestra.run.smithi193:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-21T18:19:35.822 DEBUG:teuthology.orchestra.run.smithi193:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-21T18:19:35.908 DEBUG:teuthology.orchestra.run.smithi193:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-21T18:19:35.953 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2023-07-21T18:19:35.954 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-07-21T18:19:36.034 DEBUG:teuthology.orchestra.run.smithi039:> sudo service rsyslog restart 2023-07-21T18:19:36.036 DEBUG:teuthology.orchestra.run.smithi155:> sudo service rsyslog restart 2023-07-21T18:19:36.039 DEBUG:teuthology.orchestra.run.smithi193:> sudo service rsyslog restart 2023-07-21T18:19:36.110 INFO:teuthology.orchestra.run.smithi155.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-21T18:19:36.130 INFO:teuthology.orchestra.run.smithi039.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-21T18:19:36.131 INFO:teuthology.orchestra.run.smithi193.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-21T18:19:36.453 INFO:teuthology.run_tasks:Running task internal.timer... 2023-07-21T18:19:36.461 INFO:teuthology.task.internal:Starting timer... 2023-07-21T18:19:36.461 INFO:teuthology.run_tasks:Running task pcp... 2023-07-21T18:19:36.501 INFO:teuthology.run_tasks:Running task selinux... 2023-07-21T18:19:36.526 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']} 2023-07-21T18:19:36.527 DEBUG:teuthology.orchestra.run.smithi039:> sudo service auditd rotate 2023-07-21T18:19:36.596 INFO:teuthology.orchestra.run.smithi039.stdout:Rotating logs: 2023-07-21T18:19:36.598 DEBUG:teuthology.orchestra.run.smithi155:> sudo service auditd rotate 2023-07-21T18:19:36.664 INFO:teuthology.orchestra.run.smithi155.stdout:Rotating logs: 2023-07-21T18:19:36.666 DEBUG:teuthology.orchestra.run.smithi193:> sudo service auditd rotate 2023-07-21T18:19:36.726 INFO:teuthology.orchestra.run.smithi193.stdout:Rotating logs: 2023-07-21T18:19:36.727 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-07-21T18:19:36.728 DEBUG:teuthology.orchestra.run.smithi039:> /usr/sbin/getenforce 2023-07-21T18:19:36.756 INFO:teuthology.orchestra.run.smithi039.stdout:Permissive 2023-07-21T18:19:36.756 DEBUG:teuthology.orchestra.run.smithi155:> /usr/sbin/getenforce 2023-07-21T18:19:36.785 INFO:teuthology.orchestra.run.smithi155.stdout:Permissive 2023-07-21T18:19:36.785 DEBUG:teuthology.orchestra.run.smithi193:> /usr/sbin/getenforce 2023-07-21T18:19:36.804 INFO:teuthology.orchestra.run.smithi193.stdout:Permissive 2023-07-21T18:19:36.804 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi039.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi155.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi193.front.sepia.ceph.com': 'permissive'} 2023-07-21T18:19:36.805 DEBUG:teuthology.orchestra.run.smithi039:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-21T18:19:36.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T18:19:36.833 DEBUG:teuthology.orchestra.run.smithi155:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-21T18:19:36.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T18:19:36.862 DEBUG:teuthology.orchestra.run.smithi193:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-21T18:19:36.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T18:19:36.890 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-07-21T18:19:36.890 DEBUG:teuthology.orchestra.run.smithi039:> sudo /usr/sbin/setenforce permissive 2023-07-21T18:19:36.923 DEBUG:teuthology.orchestra.run.smithi155:> sudo /usr/sbin/setenforce permissive 2023-07-21T18:19:36.963 DEBUG:teuthology.orchestra.run.smithi193:> sudo /usr/sbin/setenforce permissive 2023-07-21T18:19:36.989 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-07-21T18:19:37.000 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-07-21T18:19:37.007 INFO:teuthology.repo_utils:Fetching main from origin 2023-07-21T18:19:37.138 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-07-21T18:19:37.159 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-07-21T18:19:37.161 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi039.front.sepia.ceph.com,smithi155.front.sepia.ceph.com,smithi193.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-07-21T18:29:00.339 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi039.front.sepia.ceph.com'), Remote(name='ubuntu@smithi155.front.sepia.ceph.com'), Remote(name='ubuntu@smithi193.front.sepia.ceph.com')] 2023-07-21T18:29:00.341 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi039.front.sepia.ceph.com' 2023-07-21T18:29:00.342 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T18:29:05.416 DEBUG:teuthology.orchestra.run.smithi039:> true 2023-07-21T18:29:05.488 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi039.front.sepia.ceph.com' 2023-07-21T18:29:05.488 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2023-07-21T18:29:05.489 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T18:29:10.561 DEBUG:teuthology.orchestra.run.smithi155:> true 2023-07-21T18:29:10.589 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi155.front.sepia.ceph.com' 2023-07-21T18:29:10.590 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi193.front.sepia.ceph.com' 2023-07-21T18:29:10.590 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T18:29:10.660 DEBUG:teuthology.orchestra.run.smithi193:> true 2023-07-21T18:29:10.731 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi193.front.sepia.ceph.com' 2023-07-21T18:29:10.732 INFO:teuthology.run_tasks:Running task clock... 2023-07-21T18:29:10.743 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-07-21T18:29:10.743 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-07-21T18:29:10.744 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-21T18:29:10.747 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-07-21T18:29:10.747 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-21T18:29:10.749 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-07-21T18:29:10.749 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-21T18:29:10.782 INFO:teuthology.orchestra.run.smithi155.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-07-21T18:29:10.784 INFO:teuthology.orchestra.run.smithi039.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-07-21T18:29:10.797 INFO:teuthology.orchestra.run.smithi155.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-07-21T18:29:10.801 INFO:teuthology.orchestra.run.smithi039.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-07-21T18:29:10.806 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-07-21T18:29:10.821 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-07-21T18:29:10.875 INFO:teuthology.orchestra.run.smithi155.stderr:sudo: ntpd: command not found 2023-07-21T18:29:10.882 INFO:teuthology.orchestra.run.smithi039.stderr:sudo: ntpd: command not found 2023-07-21T18:29:10.885 INFO:teuthology.orchestra.run.smithi193.stderr:sudo: ntpd: command not found 2023-07-21T18:29:10.887 INFO:teuthology.orchestra.run.smithi155.stdout:506 Cannot talk to daemon 2023-07-21T18:29:10.895 INFO:teuthology.orchestra.run.smithi039.stdout:506 Cannot talk to daemon 2023-07-21T18:29:10.897 INFO:teuthology.orchestra.run.smithi193.stdout:506 Cannot talk to daemon 2023-07-21T18:29:10.902 INFO:teuthology.orchestra.run.smithi155.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-07-21T18:29:10.910 INFO:teuthology.orchestra.run.smithi039.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-07-21T18:29:10.912 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-07-21T18:29:10.916 INFO:teuthology.orchestra.run.smithi155.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-07-21T18:29:10.925 INFO:teuthology.orchestra.run.smithi039.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-07-21T18:29:10.926 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-07-21T18:29:10.991 INFO:teuthology.orchestra.run.smithi155.stderr:bash: ntpq: command not found 2023-07-21T18:29:10.994 INFO:teuthology.orchestra.run.smithi155.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-21T18:29:10.994 INFO:teuthology.orchestra.run.smithi155.stdout:=============================================================================== 2023-07-21T18:29:10.995 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T18:29:10.995 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T18:29:10.995 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T18:29:10.995 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T18:29:11.001 INFO:teuthology.orchestra.run.smithi193.stderr:bash: ntpq: command not found 2023-07-21T18:29:11.005 INFO:teuthology.orchestra.run.smithi193.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-21T18:29:11.005 INFO:teuthology.orchestra.run.smithi193.stdout:=============================================================================== 2023-07-21T18:29:11.005 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T18:29:11.005 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T18:29:11.005 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T18:29:11.006 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T18:29:11.008 INFO:teuthology.orchestra.run.smithi039.stderr:bash: ntpq: command not found 2023-07-21T18:29:11.012 INFO:teuthology.orchestra.run.smithi039.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-21T18:29:11.012 INFO:teuthology.orchestra.run.smithi039.stdout:=============================================================================== 2023-07-21T18:29:11.012 INFO:teuthology.orchestra.run.smithi039.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T18:29:11.013 INFO:teuthology.orchestra.run.smithi039.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T18:29:11.013 INFO:teuthology.orchestra.run.smithi039.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T18:29:11.013 INFO:teuthology.orchestra.run.smithi039.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T18:29:11.013 INFO:teuthology.run_tasks:Running task pexec... 2023-07-21T18:29:11.022 INFO:teuthology.task.pexec:Executing custom commands... 2023-07-21T18:29:11.023 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi039.front.sepia.ceph.com 2023-07-21T18:29:11.023 DEBUG:teuthology.orchestra.run.smithi039:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-07-21T18:29:11.024 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi155.front.sepia.ceph.com 2023-07-21T18:29:11.024 DEBUG:teuthology.orchestra.run.smithi155:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-07-21T18:29:11.024 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi193.front.sepia.ceph.com 2023-07-21T18:29:11.025 DEBUG:teuthology.orchestra.run.smithi193:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-07-21T18:29:11.406 INFO:teuthology.orchestra.run.smithi155.stdout:Updating Subscription Management repositories. 2023-07-21T18:29:11.417 INFO:teuthology.orchestra.run.smithi193.stdout:Updating Subscription Management repositories. 2023-07-21T18:29:11.445 INFO:teuthology.orchestra.run.smithi039.stdout:Updating Subscription Management repositories. 2023-07-21T18:29:12.303 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:35 ago on Fri 21 Jul 2023 06:27:36 PM UTC. 2023-07-21T18:29:12.307 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:01:52 ago on Fri 21 Jul 2023 06:27:19 PM UTC. 2023-07-21T18:29:12.347 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:01:40 ago on Fri 21 Jul 2023 06:27:31 PM UTC. 2023-07-21T18:29:14.033 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2023-07-21T18:29:14.034 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2023-07-21T18:29:14.035 INFO:teuthology.orchestra.run.smithi193.stdout: Package Architecture Version Repository Size 2023-07-21T18:29:14.035 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2023-07-21T18:29:14.035 INFO:teuthology.orchestra.run.smithi193.stdout:Resetting modules: 2023-07-21T18:29:14.035 INFO:teuthology.orchestra.run.smithi193.stdout: container-tools 2023-07-21T18:29:14.035 INFO:teuthology.orchestra.run.smithi193.stdout: 2023-07-21T18:29:14.035 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2023-07-21T18:29:14.035 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2023-07-21T18:29:14.036 INFO:teuthology.orchestra.run.smithi193.stdout: 2023-07-21T18:29:14.168 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2023-07-21T18:29:14.169 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2023-07-21T18:29:14.169 INFO:teuthology.orchestra.run.smithi155.stdout: Package Architecture Version Repository Size 2023-07-21T18:29:14.169 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2023-07-21T18:29:14.169 INFO:teuthology.orchestra.run.smithi155.stdout:Resetting modules: 2023-07-21T18:29:14.170 INFO:teuthology.orchestra.run.smithi155.stdout: container-tools 2023-07-21T18:29:14.170 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-07-21T18:29:14.170 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2023-07-21T18:29:14.170 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2023-07-21T18:29:14.170 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-07-21T18:29:14.343 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2023-07-21T18:29:14.345 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2023-07-21T18:29:14.345 INFO:teuthology.orchestra.run.smithi039.stdout: Package Architecture Version Repository Size 2023-07-21T18:29:14.345 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2023-07-21T18:29:14.345 INFO:teuthology.orchestra.run.smithi039.stdout:Resetting modules: 2023-07-21T18:29:14.345 INFO:teuthology.orchestra.run.smithi039.stdout: container-tools 2023-07-21T18:29:14.345 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:29:14.345 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction Summary 2023-07-21T18:29:14.346 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2023-07-21T18:29:14.346 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:29:14.997 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:01:55 ago on Fri 21 Jul 2023 06:27:19 PM UTC. 2023-07-21T18:29:15.142 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:38 ago on Fri 21 Jul 2023 06:27:36 PM UTC. 2023-07-21T18:29:15.339 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:01:43 ago on Fri 21 Jul 2023 06:27:31 PM UTC. 2023-07-21T18:29:17.663 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2023-07-21T18:29:17.891 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2023-07-21T18:29:18.135 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2023-07-21T18:29:18.148 INFO:teuthology.orchestra.run.smithi193.stdout:Updating Subscription Management repositories. 2023-07-21T18:29:18.357 INFO:teuthology.orchestra.run.smithi155.stdout:Updating Subscription Management repositories. 2023-07-21T18:29:18.618 INFO:teuthology.orchestra.run.smithi039.stdout:Updating Subscription Management repositories. 2023-07-21T18:29:18.954 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:01:59 ago on Fri 21 Jul 2023 06:27:19 PM UTC. 2023-07-21T18:29:19.170 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:42 ago on Fri 21 Jul 2023 06:27:36 PM UTC. 2023-07-21T18:29:19.414 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:01:47 ago on Fri 21 Jul 2023 06:27:31 PM UTC. 2023-07-21T18:29:20.670 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2023-07-21T18:29:20.765 INFO:teuthology.orchestra.run.smithi193.stdout:============================================================================================================ 2023-07-21T18:29:20.765 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repository Size 2023-07-21T18:29:20.766 INFO:teuthology.orchestra.run.smithi193.stdout:============================================================================================================ 2023-07-21T18:29:20.766 INFO:teuthology.orchestra.run.smithi193.stdout:Installing group/module packages: 2023-07-21T18:29:20.766 INFO:teuthology.orchestra.run.smithi193.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-07-21T18:29:20.766 INFO:teuthology.orchestra.run.smithi193.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-07-21T18:29:20.767 INFO:teuthology.orchestra.run.smithi193.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-07-21T18:29:20.767 INFO:teuthology.orchestra.run.smithi193.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-07-21T18:29:20.767 INFO:teuthology.orchestra.run.smithi193.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-07-21T18:29:20.767 INFO:teuthology.orchestra.run.smithi193.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-07-21T18:29:20.768 INFO:teuthology.orchestra.run.smithi193.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-07-21T18:29:20.768 INFO:teuthology.orchestra.run.smithi193.stdout:Installing dependencies: 2023-07-21T18:29:20.768 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-07-21T18:29:20.768 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-07-21T18:29:20.769 INFO:teuthology.orchestra.run.smithi193.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-07-21T18:29:20.769 INFO:teuthology.orchestra.run.smithi193.stdout:Installing module profiles: 2023-07-21T18:29:20.769 INFO:teuthology.orchestra.run.smithi193.stdout: container-tools/common 2023-07-21T18:29:20.769 INFO:teuthology.orchestra.run.smithi193.stdout: 2023-07-21T18:29:20.770 INFO:teuthology.orchestra.run.smithi193.stdout:Enabling module streams: 2023-07-21T18:29:20.770 INFO:teuthology.orchestra.run.smithi193.stdout: container-tools rhel8 2023-07-21T18:29:20.770 INFO:teuthology.orchestra.run.smithi193.stdout: 2023-07-21T18:29:20.771 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2023-07-21T18:29:20.771 INFO:teuthology.orchestra.run.smithi193.stdout:============================================================================================================ 2023-07-21T18:29:20.771 INFO:teuthology.orchestra.run.smithi193.stdout:Install 10 Packages 2023-07-21T18:29:20.771 INFO:teuthology.orchestra.run.smithi193.stdout: 2023-07-21T18:29:20.772 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 18 M 2023-07-21T18:29:20.772 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 65 M 2023-07-21T18:29:20.773 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2023-07-21T18:29:20.826 INFO:teuthology.orchestra.run.smithi193.stdout:(1/10): yajl-2.1.0-10.el8.x86_64.rpm 692 kB/s | 41 kB 00:00 2023-07-21T18:29:20.852 INFO:teuthology.orchestra.run.smithi193.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 295 kB/s | 25 kB 00:00 2023-07-21T18:29:20.873 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2023-07-21T18:29:20.877 INFO:teuthology.orchestra.run.smithi193.stdout:(3/10): udica-0.2.6-4.module+el8.6.0+16771+28df 984 kB/s | 49 kB 00:00 2023-07-21T18:29:20.902 INFO:teuthology.orchestra.run.smithi193.stdout:(4/10): python3-pyxdg-0.25-16.el8.noarch.rpm 697 kB/s | 94 kB 00:00 2023-07-21T18:29:20.944 INFO:teuthology.orchestra.run.smithi193.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 5.6 MB/s | 524 kB 00:00 2023-07-21T18:29:20.961 INFO:teuthology.orchestra.run.smithi193.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.5 MB/s | 149 kB 00:00 2023-07-21T18:29:20.967 INFO:teuthology.orchestra.run.smithi155.stdout:============================================================================================================ 2023-07-21T18:29:20.967 INFO:teuthology.orchestra.run.smithi155.stdout: Package Arch Version Repository Size 2023-07-21T18:29:20.967 INFO:teuthology.orchestra.run.smithi155.stdout:============================================================================================================ 2023-07-21T18:29:20.967 INFO:teuthology.orchestra.run.smithi155.stdout:Installing group/module packages: 2023-07-21T18:29:20.968 INFO:teuthology.orchestra.run.smithi155.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-07-21T18:29:20.968 INFO:teuthology.orchestra.run.smithi155.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-07-21T18:29:20.968 INFO:teuthology.orchestra.run.smithi155.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-07-21T18:29:20.968 INFO:teuthology.orchestra.run.smithi155.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-07-21T18:29:20.968 INFO:teuthology.orchestra.run.smithi155.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-07-21T18:29:20.969 INFO:teuthology.orchestra.run.smithi155.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-07-21T18:29:20.969 INFO:teuthology.orchestra.run.smithi155.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-07-21T18:29:20.969 INFO:teuthology.orchestra.run.smithi155.stdout:Installing dependencies: 2023-07-21T18:29:20.969 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-07-21T18:29:20.970 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-07-21T18:29:20.970 INFO:teuthology.orchestra.run.smithi155.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-07-21T18:29:20.970 INFO:teuthology.orchestra.run.smithi155.stdout:Installing module profiles: 2023-07-21T18:29:20.970 INFO:teuthology.orchestra.run.smithi155.stdout: container-tools/common 2023-07-21T18:29:20.971 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-07-21T18:29:20.971 INFO:teuthology.orchestra.run.smithi155.stdout:Enabling module streams: 2023-07-21T18:29:20.971 INFO:teuthology.orchestra.run.smithi155.stdout: container-tools rhel8 2023-07-21T18:29:20.971 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-07-21T18:29:20.972 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2023-07-21T18:29:20.972 INFO:teuthology.orchestra.run.smithi155.stdout:============================================================================================================ 2023-07-21T18:29:20.972 INFO:teuthology.orchestra.run.smithi155.stdout:Install 10 Packages 2023-07-21T18:29:20.973 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-07-21T18:29:20.973 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 18 M 2023-07-21T18:29:20.974 INFO:teuthology.orchestra.run.smithi155.stdout:Installed size: 65 M 2023-07-21T18:29:20.974 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2023-07-21T18:29:20.986 INFO:teuthology.orchestra.run.smithi193.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 5.0 MB/s | 213 kB 00:00 2023-07-21T18:29:21.057 INFO:teuthology.orchestra.run.smithi155.stdout:(1/10): yajl-2.1.0-10.el8.x86_64.rpm 464 kB/s | 41 kB 00:00 2023-07-21T18:29:21.085 INFO:teuthology.orchestra.run.smithi155.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 215 kB/s | 25 kB 00:00 2023-07-21T18:29:21.086 INFO:teuthology.orchestra.run.smithi193.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 22 MB/s | 2.2 MB 00:00 2023-07-21T18:29:21.110 INFO:teuthology.orchestra.run.smithi155.stdout:(3/10): udica-0.2.6-4.module+el8.6.0+16771+28df 938 kB/s | 49 kB 00:00 2023-07-21T18:29:21.135 INFO:teuthology.orchestra.run.smithi155.stdout:(4/10): python3-pyxdg-0.25-16.el8.noarch.rpm 565 kB/s | 94 kB 00:00 2023-07-21T18:29:21.169 INFO:teuthology.orchestra.run.smithi155.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 6.1 MB/s | 524 kB 00:00 2023-07-21T18:29:21.186 INFO:teuthology.orchestra.run.smithi155.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.9 MB/s | 149 kB 00:00 2023-07-21T18:29:21.195 INFO:teuthology.orchestra.run.smithi193.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 30 MB/s | 7.0 MB 00:00 2023-07-21T18:29:21.211 INFO:teuthology.orchestra.run.smithi155.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 5.0 MB/s | 213 kB 00:00 2023-07-21T18:29:21.278 INFO:teuthology.orchestra.run.smithi193.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 20 MB/s | 8.0 MB 00:00 2023-07-21T18:29:21.279 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2023-07-21T18:29:21.279 INFO:teuthology.orchestra.run.smithi193.stdout:Total 36 MB/s | 18 MB 00:00 2023-07-21T18:29:21.303 INFO:teuthology.orchestra.run.smithi155.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 24 MB/s | 2.2 MB 00:00 2023-07-21T18:29:21.376 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2023-07-21T18:29:21.420 INFO:teuthology.orchestra.run.smithi155.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 30 MB/s | 7.0 MB 00:00 2023-07-21T18:29:21.476 INFO:teuthology.orchestra.run.smithi039.stdout:============================================================================================================ 2023-07-21T18:29:21.477 INFO:teuthology.orchestra.run.smithi039.stdout: Package Arch Version Repository Size 2023-07-21T18:29:21.477 INFO:teuthology.orchestra.run.smithi039.stdout:============================================================================================================ 2023-07-21T18:29:21.477 INFO:teuthology.orchestra.run.smithi039.stdout:Installing group/module packages: 2023-07-21T18:29:21.477 INFO:teuthology.orchestra.run.smithi039.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-07-21T18:29:21.477 INFO:teuthology.orchestra.run.smithi039.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-07-21T18:29:21.477 INFO:teuthology.orchestra.run.smithi039.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-07-21T18:29:21.477 INFO:teuthology.orchestra.run.smithi039.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-07-21T18:29:21.478 INFO:teuthology.orchestra.run.smithi039.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-07-21T18:29:21.478 INFO:teuthology.orchestra.run.smithi039.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-07-21T18:29:21.478 INFO:teuthology.orchestra.run.smithi039.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-07-21T18:29:21.478 INFO:teuthology.orchestra.run.smithi039.stdout:Installing dependencies: 2023-07-21T18:29:21.478 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-07-21T18:29:21.478 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-07-21T18:29:21.478 INFO:teuthology.orchestra.run.smithi039.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-07-21T18:29:21.479 INFO:teuthology.orchestra.run.smithi039.stdout:Installing module profiles: 2023-07-21T18:29:21.479 INFO:teuthology.orchestra.run.smithi039.stdout: container-tools/common 2023-07-21T18:29:21.479 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:29:21.479 INFO:teuthology.orchestra.run.smithi039.stdout:Enabling module streams: 2023-07-21T18:29:21.479 INFO:teuthology.orchestra.run.smithi039.stdout: container-tools rhel8 2023-07-21T18:29:21.479 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:29:21.479 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction Summary 2023-07-21T18:29:21.480 INFO:teuthology.orchestra.run.smithi039.stdout:============================================================================================================ 2023-07-21T18:29:21.480 INFO:teuthology.orchestra.run.smithi039.stdout:Install 10 Packages 2023-07-21T18:29:21.480 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:29:21.480 INFO:teuthology.orchestra.run.smithi039.stdout:Total download size: 18 M 2023-07-21T18:29:21.481 INFO:teuthology.orchestra.run.smithi039.stdout:Installed size: 65 M 2023-07-21T18:29:21.481 INFO:teuthology.orchestra.run.smithi039.stdout:Downloading Packages: 2023-07-21T18:29:21.482 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2023-07-21T18:29:21.500 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2023-07-21T18:29:21.501 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2023-07-21T18:29:21.553 INFO:teuthology.orchestra.run.smithi039.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 1.2 MB/s | 94 kB 00:00 2023-07-21T18:29:21.554 INFO:teuthology.orchestra.run.smithi155.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 18 MB/s | 8.0 MB 00:00 2023-07-21T18:29:21.554 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2023-07-21T18:29:21.554 INFO:teuthology.orchestra.run.smithi155.stdout:Total 31 MB/s | 18 MB 00:00 2023-07-21T18:29:21.579 INFO:teuthology.orchestra.run.smithi039.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 249 kB/s | 25 kB 00:00 2023-07-21T18:29:21.604 INFO:teuthology.orchestra.run.smithi039.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 327 kB/s | 41 kB 00:00 2023-07-21T18:29:21.629 INFO:teuthology.orchestra.run.smithi039.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 653 kB/s | 49 kB 00:00 2023-07-21T18:29:21.671 INFO:teuthology.orchestra.run.smithi039.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 5.6 MB/s | 524 kB 00:00 2023-07-21T18:29:21.682 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2023-07-21T18:29:21.686 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2023-07-21T18:29:21.696 INFO:teuthology.orchestra.run.smithi039.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.2 MB/s | 149 kB 00:00 2023-07-21T18:29:21.722 INFO:teuthology.orchestra.run.smithi039.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 4.1 MB/s | 213 kB 00:00 2023-07-21T18:29:21.755 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2023-07-21T18:29:21.774 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2023-07-21T18:29:21.774 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2023-07-21T18:29:21.822 INFO:teuthology.orchestra.run.smithi039.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 22 MB/s | 2.2 MB 00:00 2023-07-21T18:29:21.947 INFO:teuthology.orchestra.run.smithi039.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 28 MB/s | 7.0 MB 00:00 2023-07-21T18:29:21.955 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2023-07-21T18:29:21.959 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2023-07-21T18:29:22.056 INFO:teuthology.orchestra.run.smithi039.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 18 MB/s | 8.0 MB 00:00 2023-07-21T18:29:22.056 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2023-07-21T18:29:22.057 INFO:teuthology.orchestra.run.smithi039.stdout:Total 32 MB/s | 18 MB 00:00 2023-07-21T18:29:22.196 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2023-07-21T18:29:22.265 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction check 2023-07-21T18:29:22.285 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction check succeeded. 2023-07-21T18:29:22.286 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction test 2023-07-21T18:29:22.401 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-07-21T18:29:22.466 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2023-07-21T18:29:22.480 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction test succeeded. 2023-07-21T18:29:22.484 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction 2023-07-21T18:29:22.574 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-21T18:29:22.649 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-07-21T18:29:22.776 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-07-21T18:29:22.818 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-21T18:29:22.996 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-07-21T18:29:23.011 INFO:teuthology.orchestra.run.smithi039.stdout: Preparing : 1/1 2023-07-21T18:29:23.054 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-07-21T18:29:23.191 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-07-21T18:29:23.262 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-07-21T18:29:23.369 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-07-21T18:29:23.396 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-21T18:29:23.584 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-07-21T18:29:23.630 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-07-21T18:29:23.788 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-07-21T18:29:24.142 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-07-21T18:29:24.170 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-07-21T18:29:24.428 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-07-21T18:29:24.976 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-07-21T18:29:25.006 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-07-21T18:29:25.243 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-07-21T18:29:25.285 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-07-21T18:29:25.398 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-07-21T18:29:25.492 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-07-21T18:29:25.566 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-21T18:29:25.701 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-07-21T18:29:25.824 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-21T18:29:25.824 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-07-21T18:29:25.824 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-21T18:29:25.825 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-07-21T18:29:25.825 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-07-21T18:29:25.825 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-07-21T18:29:25.825 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-07-21T18:29:25.825 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-07-21T18:29:25.825 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-07-21T18:29:25.825 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-07-21T18:29:25.877 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-21T18:29:25.882 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-07-21T18:29:26.100 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-07-21T18:29:26.151 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-21T18:29:26.151 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-07-21T18:29:26.151 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-21T18:29:26.151 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-07-21T18:29:26.152 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-07-21T18:29:26.152 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-07-21T18:29:26.152 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-07-21T18:29:26.152 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-07-21T18:29:26.152 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-07-21T18:29:26.152 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-07-21T18:29:26.270 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-07-21T18:29:26.488 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-21T18:29:26.753 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-21T18:29:26.754 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-07-21T18:29:26.754 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-21T18:29:26.754 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-07-21T18:29:26.754 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-07-21T18:29:26.754 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-07-21T18:29:26.754 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-07-21T18:29:26.755 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-07-21T18:29:26.755 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-07-21T18:29:26.755 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-07-21T18:29:26.996 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-07-21T18:29:26.997 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:02:07 ago on Fri 21 Jul 2023 06:27:19 PM UTC. 2023-07-21T18:29:27.258 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-07-21T18:29:27.258 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:50 ago on Fri 21 Jul 2023 06:27:36 PM UTC. 2023-07-21T18:29:27.907 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-07-21T18:29:27.908 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:01:56 ago on Fri 21 Jul 2023 06:27:31 PM UTC. 2023-07-21T18:29:29.742 INFO:teuthology.orchestra.run.smithi193.stdout:Installed products updated. 2023-07-21T18:29:29.932 INFO:teuthology.orchestra.run.smithi193.stdout: 2023-07-21T18:29:29.933 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2023-07-21T18:29:29.933 INFO:teuthology.orchestra.run.smithi193.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-21T18:29:29.933 INFO:teuthology.orchestra.run.smithi193.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-07-21T18:29:29.933 INFO:teuthology.orchestra.run.smithi193.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-21T18:29:29.933 INFO:teuthology.orchestra.run.smithi193.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-07-21T18:29:29.933 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-07-21T18:29:29.934 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-07-21T18:29:29.934 INFO:teuthology.orchestra.run.smithi193.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-21T18:29:29.934 INFO:teuthology.orchestra.run.smithi193.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-07-21T18:29:29.934 INFO:teuthology.orchestra.run.smithi193.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-07-21T18:29:29.934 INFO:teuthology.orchestra.run.smithi193.stdout: yajl-2.1.0-10.el8.x86_64 2023-07-21T18:29:29.934 INFO:teuthology.orchestra.run.smithi193.stdout: 2023-07-21T18:29:29.934 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2023-07-21T18:29:29.996 INFO:teuthology.orchestra.run.smithi155.stdout:Installed products updated. 2023-07-21T18:29:30.084 DEBUG:teuthology.parallel:result is None 2023-07-21T18:29:30.195 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-07-21T18:29:30.196 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2023-07-21T18:29:30.196 INFO:teuthology.orchestra.run.smithi155.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-21T18:29:30.196 INFO:teuthology.orchestra.run.smithi155.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-07-21T18:29:30.197 INFO:teuthology.orchestra.run.smithi155.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-21T18:29:30.197 INFO:teuthology.orchestra.run.smithi155.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-07-21T18:29:30.197 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-07-21T18:29:30.198 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-07-21T18:29:30.198 INFO:teuthology.orchestra.run.smithi155.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-21T18:29:30.198 INFO:teuthology.orchestra.run.smithi155.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-07-21T18:29:30.198 INFO:teuthology.orchestra.run.smithi155.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-07-21T18:29:30.199 INFO:teuthology.orchestra.run.smithi155.stdout: yajl-2.1.0-10.el8.x86_64 2023-07-21T18:29:30.199 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-07-21T18:29:30.199 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2023-07-21T18:29:30.339 DEBUG:teuthology.parallel:result is None 2023-07-21T18:29:30.618 INFO:teuthology.orchestra.run.smithi039.stdout:Installed products updated. 2023-07-21T18:29:30.824 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:29:30.825 INFO:teuthology.orchestra.run.smithi039.stdout:Installed: 2023-07-21T18:29:30.825 INFO:teuthology.orchestra.run.smithi039.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-21T18:29:30.825 INFO:teuthology.orchestra.run.smithi039.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-07-21T18:29:30.825 INFO:teuthology.orchestra.run.smithi039.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-21T18:29:30.825 INFO:teuthology.orchestra.run.smithi039.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-07-21T18:29:30.825 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-07-21T18:29:30.825 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-07-21T18:29:30.826 INFO:teuthology.orchestra.run.smithi039.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-21T18:29:30.826 INFO:teuthology.orchestra.run.smithi039.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-07-21T18:29:30.826 INFO:teuthology.orchestra.run.smithi039.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-07-21T18:29:30.826 INFO:teuthology.orchestra.run.smithi039.stdout: yajl-2.1.0-10.el8.x86_64 2023-07-21T18:29:30.826 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:29:30.826 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2023-07-21T18:29:30.970 DEBUG:teuthology.parallel:result is None 2023-07-21T18:29:30.971 INFO:teuthology.run_tasks:Running task install... 2023-07-21T18:29:30.979 DEBUG:teuthology.task.install:project ceph 2023-07-21T18:29:30.979 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '81cf785aa7781c4e08381c540f890cb9bc987ef9'}} 2023-07-21T18:29:30.980 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '81cf785aa7781c4e08381c540f890cb9bc987ef9'} 2023-07-21T18:29:30.980 INFO:teuthology.task.install:Using flavor: default 2023-07-21T18:29:30.985 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-07-21T18:29:30.986 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-07-21T18:29:30.986 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '81cf785aa7781c4e08381c540f890cb9bc987ef9', 'tag': None, 'wait_for_package': False} 2023-07-21T18:29:30.987 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=81cf785aa7781c4e08381c540f890cb9bc987ef9 2023-07-21T18:29:30.989 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '81cf785aa7781c4e08381c540f890cb9bc987ef9', 'tag': None, 'wait_for_package': False} 2023-07-21T18:29:30.989 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=81cf785aa7781c4e08381c540f890cb9bc987ef9 2023-07-21T18:29:30.990 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '81cf785aa7781c4e08381c540f890cb9bc987ef9', 'tag': None, 'wait_for_package': False} 2023-07-21T18:29:30.991 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=81cf785aa7781c4e08381c540f890cb9bc987ef9 2023-07-21T18:29:36.292 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/flavors/default/ 2023-07-21T18:29:36.292 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4746.g81cf785a 2023-07-21T18:29:36.307 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/flavors/default/ 2023-07-21T18:29:36.307 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4746.g81cf785a 2023-07-21T18:29:36.311 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/flavors/default/ 2023-07-21T18:29:36.311 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4746.g81cf785a 2023-07-21T18:29:36.525 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-07-21T18:29:36.525 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-07-21T18:29:36.525 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-07-21T18:29:36.527 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-07-21T18:29:36.528 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-07-21T18:29:36.528 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-07-21T18:29:36.540 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-07-21T18:29:36.540 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2023-07-21T18:29:36.540 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-07-21T18:29:36.560 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-07-21T18:29:36.560 DEBUG:teuthology.orchestra.run.smithi155:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/81cf785aa7781c4e08381c540f890cb9bc987ef9/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-07-21T18:29:36.561 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-07-21T18:29:36.562 DEBUG:teuthology.orchestra.run.smithi039:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/81cf785aa7781c4e08381c540f890cb9bc987ef9/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-07-21T18:29:36.573 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-07-21T18:29:36.573 DEBUG:teuthology.orchestra.run.smithi193:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/81cf785aa7781c4e08381c540f890cb9bc987ef9/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-07-21T18:29:36.631 DEBUG:teuthology.orchestra.run.smithi155:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-07-21T18:29:36.633 DEBUG:teuthology.orchestra.run.smithi039:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-07-21T18:29:36.643 DEBUG:teuthology.orchestra.run.smithi193:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-07-21T18:29:36.712 DEBUG:teuthology.orchestra.run.smithi155:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-07-21T18:29:36.715 DEBUG:teuthology.orchestra.run.smithi039:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-07-21T18:29:36.722 DEBUG:teuthology.orchestra.run.smithi193:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-07-21T18:29:36.780 INFO:teuthology.orchestra.run.smithi155.stdout:check_obsoletes = 1 2023-07-21T18:29:36.782 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum clean all 2023-07-21T18:29:36.783 INFO:teuthology.orchestra.run.smithi039.stdout:check_obsoletes = 1 2023-07-21T18:29:36.785 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum clean all 2023-07-21T18:29:36.789 INFO:teuthology.orchestra.run.smithi193.stdout:check_obsoletes = 1 2023-07-21T18:29:36.791 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum clean all 2023-07-21T18:29:37.157 INFO:teuthology.orchestra.run.smithi155.stdout:Updating Subscription Management repositories. 2023-07-21T18:29:37.162 INFO:teuthology.orchestra.run.smithi193.stdout:Updating Subscription Management repositories. 2023-07-21T18:29:37.180 INFO:teuthology.orchestra.run.smithi039.stdout:Updating Subscription Management repositories. 2023-07-21T18:29:37.475 INFO:teuthology.orchestra.run.smithi193.stdout:44 files removed 2023-07-21T18:29:37.484 INFO:teuthology.orchestra.run.smithi155.stdout:44 files removed 2023-07-21T18:29:37.495 INFO:teuthology.orchestra.run.smithi039.stdout:44 files removed 2023-07-21T18:29:37.522 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-07-21T18:29:37.530 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-07-21T18:29:37.543 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-07-21T18:29:37.853 INFO:teuthology.orchestra.run.smithi193.stdout:Updating Subscription Management repositories. 2023-07-21T18:29:37.861 INFO:teuthology.orchestra.run.smithi155.stdout:Updating Subscription Management repositories. 2023-07-21T18:29:37.894 INFO:teuthology.orchestra.run.smithi039.stdout:Updating Subscription Management repositories. 2023-07-21T18:29:38.913 INFO:teuthology.orchestra.run.smithi155.stdout:ceph packages for x86_64 126 kB/s | 81 kB 00:00 2023-07-21T18:29:38.927 INFO:teuthology.orchestra.run.smithi193.stdout:ceph packages for x86_64 122 kB/s | 81 kB 00:00 2023-07-21T18:29:38.959 INFO:teuthology.orchestra.run.smithi039.stdout:ceph packages for x86_64 129 kB/s | 81 kB 00:00 2023-07-21T18:29:39.464 INFO:teuthology.orchestra.run.smithi155.stdout:ceph noarch packages 18 kB/s | 9.4 kB 00:00 2023-07-21T18:29:39.527 INFO:teuthology.orchestra.run.smithi039.stdout:ceph noarch packages 18 kB/s | 9.4 kB 00:00 2023-07-21T18:29:39.530 INFO:teuthology.orchestra.run.smithi193.stdout:ceph noarch packages 17 kB/s | 9.4 kB 00:00 2023-07-21T18:29:40.056 INFO:teuthology.orchestra.run.smithi155.stdout:ceph source packages 2.6 kB/s | 1.5 kB 00:00 2023-07-21T18:29:40.103 INFO:teuthology.orchestra.run.smithi039.stdout:ceph source packages 2.7 kB/s | 1.5 kB 00:00 2023-07-21T18:29:40.122 INFO:teuthology.orchestra.run.smithi193.stdout:ceph source packages 2.6 kB/s | 1.5 kB 00:00 2023-07-21T18:29:40.365 INFO:teuthology.orchestra.run.smithi155.stdout:Copr repo for python3-asyncssh owned by ceph 12 kB/s | 3.5 kB 00:00 2023-07-21T18:29:40.403 INFO:teuthology.orchestra.run.smithi039.stdout:Copr repo for python3-asyncssh owned by ceph 12 kB/s | 3.5 kB 00:00 2023-07-21T18:29:40.423 INFO:teuthology.orchestra.run.smithi193.stdout:Copr repo for python3-asyncssh owned by ceph 12 kB/s | 3.5 kB 00:00 2023-07-21T18:29:43.032 INFO:teuthology.orchestra.run.smithi155.stdout:Extra Packages for Enterprise Linux 5.9 MB/s | 16 MB 00:02 2023-07-21T18:29:46.466 INFO:teuthology.orchestra.run.smithi155.stdout:lab-extras 298 kB/s | 24 kB 00:00 2023-07-21T18:29:47.367 INFO:teuthology.orchestra.run.smithi155.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 57 MB/s | 47 MB 00:00 2023-07-21T18:29:50.791 INFO:teuthology.orchestra.run.smithi193.stdout:Extra Packages for Enterprise Linux 1.5 MB/s | 16 MB 00:10 2023-07-21T18:29:51.380 INFO:teuthology.orchestra.run.smithi039.stdout:Extra Packages for Enterprise Linux 1.4 MB/s | 16 MB 00:10 2023-07-21T18:29:54.264 INFO:teuthology.orchestra.run.smithi193.stdout:lab-extras 566 kB/s | 24 kB 00:00 2023-07-21T18:29:55.153 INFO:teuthology.orchestra.run.smithi039.stdout:lab-extras 387 kB/s | 24 kB 00:00 2023-07-21T18:29:55.190 INFO:teuthology.orchestra.run.smithi193.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 62 MB/s | 53 MB 00:00 2023-07-21T18:29:55.260 INFO:teuthology.orchestra.run.smithi155.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 61 MB/s | 53 MB 00:00 2023-07-21T18:29:56.221 INFO:teuthology.orchestra.run.smithi039.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 54 MB/s | 53 MB 00:00 2023-07-21T18:30:03.375 INFO:teuthology.orchestra.run.smithi193.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 53 MB/s | 47 MB 00:00 2023-07-21T18:30:04.934 INFO:teuthology.orchestra.run.smithi039.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 53 MB/s | 47 MB 00:00 2023-07-21T18:30:06.325 INFO:teuthology.orchestra.run.smithi155.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-21T18:30:06.326 INFO:teuthology.orchestra.run.smithi155.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-21T18:30:06.340 INFO:teuthology.orchestra.run.smithi155.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-07-21T18:30:06.341 INFO:teuthology.orchestra.run.smithi155.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-07-21T18:30:06.342 INFO:teuthology.orchestra.run.smithi155.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-07-21T18:30:06.343 INFO:teuthology.orchestra.run.smithi155.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-07-21T18:30:06.345 INFO:teuthology.orchestra.run.smithi155.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-07-21T18:30:06.348 INFO:teuthology.orchestra.run.smithi155.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-07-21T18:30:06.349 INFO:teuthology.orchestra.run.smithi155.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-07-21T18:30:06.482 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2023-07-21T18:30:06.494 INFO:teuthology.orchestra.run.smithi155.stdout:=========================================================================================================================== 2023-07-21T18:30:06.494 INFO:teuthology.orchestra.run.smithi155.stdout: Package Arch Version Repository Size 2023-07-21T18:30:06.494 INFO:teuthology.orchestra.run.smithi155.stdout:=========================================================================================================================== 2023-07-21T18:30:06.495 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2023-07-21T18:30:06.495 INFO:teuthology.orchestra.run.smithi155.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-07-21T18:30:06.495 INFO:teuthology.orchestra.run.smithi155.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-07-21T18:30:06.495 INFO:teuthology.orchestra.run.smithi155.stdout: ceph x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 6.4 k 2023-07-21T18:30:06.495 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-base x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 5.4 M 2023-07-21T18:30:06.495 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-fuse x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 848 k 2023-07-21T18:30:06.495 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 165 k 2023-07-21T18:30:06.496 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 1.5 M 2023-07-21T18:30:06.496 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 129 k 2023-07-21T18:30:06.496 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 1.7 M 2023-07-21T18:30:06.496 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 7.4 M 2023-07-21T18:30:06.496 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-rook noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 47 k 2023-07-21T18:30:06.496 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-radosgw x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 13 M 2023-07-21T18:30:06.496 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-test x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 48 M 2023-07-21T18:30:06.496 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-volume noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 258 k 2023-07-21T18:30:06.497 INFO:teuthology.orchestra.run.smithi155.stdout: cephadm noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 166 k 2023-07-21T18:30:06.497 INFO:teuthology.orchestra.run.smithi155.stdout: cephfs-mirror x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 258 k 2023-07-21T18:30:06.497 INFO:teuthology.orchestra.run.smithi155.stdout: cephfs-top noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 18 k 2023-07-21T18:30:06.497 INFO:teuthology.orchestra.run.smithi155.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-07-21T18:30:06.497 INFO:teuthology.orchestra.run.smithi155.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-07-21T18:30:06.497 INFO:teuthology.orchestra.run.smithi155.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-07-21T18:30:06.497 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs-devel x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 31 k 2023-07-21T18:30:06.497 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs2 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 772 k 2023-07-21T18:30:06.498 INFO:teuthology.orchestra.run.smithi155.stdout: librados-devel x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 139 k 2023-07-21T18:30:06.498 INFO:teuthology.orchestra.run.smithi155.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-07-21T18:30:06.498 INFO:teuthology.orchestra.run.smithi155.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-07-21T18:30:06.498 INFO:teuthology.orchestra.run.smithi155.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-07-21T18:30:06.498 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-07-21T18:30:06.498 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-07-21T18:30:06.498 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-07-21T18:30:06.499 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cephfs x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 197 k 2023-07-21T18:30:06.499 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rados x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 370 k 2023-07-21T18:30:06.499 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rbd x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 364 k 2023-07-21T18:30:06.499 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rgw x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 107 k 2023-07-21T18:30:06.499 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-fuse x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 94 k 2023-07-21T18:30:06.499 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-mirror x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 3.4 M 2023-07-21T18:30:06.499 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-nbd x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 179 k 2023-07-21T18:30:06.499 INFO:teuthology.orchestra.run.smithi155.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-07-21T18:30:06.500 INFO:teuthology.orchestra.run.smithi155.stdout:Upgrading: 2023-07-21T18:30:06.500 INFO:teuthology.orchestra.run.smithi155.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-07-21T18:30:06.500 INFO:teuthology.orchestra.run.smithi155.stdout: librados2 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 3.7 M 2023-07-21T18:30:06.500 INFO:teuthology.orchestra.run.smithi155.stdout: librbd1 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 3.6 M 2023-07-21T18:30:06.500 INFO:teuthology.orchestra.run.smithi155.stdout:Installing dependencies: 2023-07-21T18:30:06.500 INFO:teuthology.orchestra.run.smithi155.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-07-21T18:30:06.500 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-common x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 22 M 2023-07-21T18:30:06.500 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 23 k 2023-07-21T18:30:06.501 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mds x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 2.3 M 2023-07-21T18:30:06.501 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 240 k 2023-07-21T18:30:06.501 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mon x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 4.6 M 2023-07-21T18:30:06.501 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-osd x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 18 M 2023-07-21T18:30:06.501 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 15 k 2023-07-21T18:30:06.501 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-selinux x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 24 k 2023-07-21T18:30:06.501 INFO:teuthology.orchestra.run.smithi155.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-07-21T18:30:06.502 INFO:teuthology.orchestra.run.smithi155.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-07-21T18:30:06.502 INFO:teuthology.orchestra.run.smithi155.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-07-21T18:30:06.502 INFO:teuthology.orchestra.run.smithi155.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-07-21T18:30:06.502 INFO:teuthology.orchestra.run.smithi155.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-07-21T18:30:06.502 INFO:teuthology.orchestra.run.smithi155.stdout: libcephsqlite x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 137 k 2023-07-21T18:30:06.502 INFO:teuthology.orchestra.run.smithi155.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-07-21T18:30:06.502 INFO:teuthology.orchestra.run.smithi155.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-07-21T18:30:06.502 INFO:teuthology.orchestra.run.smithi155.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-07-21T18:30:06.503 INFO:teuthology.orchestra.run.smithi155.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-07-21T18:30:06.503 INFO:teuthology.orchestra.run.smithi155.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-07-21T18:30:06.503 INFO:teuthology.orchestra.run.smithi155.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-07-21T18:30:06.503 INFO:teuthology.orchestra.run.smithi155.stdout: libradosstriper1 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 473 k 2023-07-21T18:30:06.503 INFO:teuthology.orchestra.run.smithi155.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-07-21T18:30:06.503 INFO:teuthology.orchestra.run.smithi155.stdout: librgw2 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 6.7 M 2023-07-21T18:30:06.503 INFO:teuthology.orchestra.run.smithi155.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-07-21T18:30:06.504 INFO:teuthology.orchestra.run.smithi155.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-07-21T18:30:06.504 INFO:teuthology.orchestra.run.smithi155.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-07-21T18:30:06.504 INFO:teuthology.orchestra.run.smithi155.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-07-21T18:30:06.504 INFO:teuthology.orchestra.run.smithi155.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-07-21T18:30:06.504 INFO:teuthology.orchestra.run.smithi155.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-07-21T18:30:06.504 INFO:teuthology.orchestra.run.smithi155.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-07-21T18:30:06.504 INFO:teuthology.orchestra.run.smithi155.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-07-21T18:30:06.504 INFO:teuthology.orchestra.run.smithi155.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-07-21T18:30:06.505 INFO:teuthology.orchestra.run.smithi155.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-07-21T18:30:06.505 INFO:teuthology.orchestra.run.smithi155.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-07-21T18:30:06.505 INFO:teuthology.orchestra.run.smithi155.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-07-21T18:30:06.505 INFO:teuthology.orchestra.run.smithi155.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-07-21T18:30:06.505 INFO:teuthology.orchestra.run.smithi155.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-07-21T18:30:06.505 INFO:teuthology.orchestra.run.smithi155.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-07-21T18:30:06.505 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-07-21T18:30:06.505 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-argparse x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 45 k 2023-07-21T18:30:06.506 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-common x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 119 k 2023-07-21T18:30:06.506 INFO:teuthology.orchestra.run.smithi155.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-07-21T18:30:06.506 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-07-21T18:30:06.506 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-07-21T18:30:06.506 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-07-21T18:30:06.506 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-07-21T18:30:06.506 INFO:teuthology.orchestra.run.smithi155.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-07-21T18:30:06.506 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-07-21T18:30:06.507 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-07-21T18:30:06.507 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-07-21T18:30:06.507 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-07-21T18:30:06.507 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-07-21T18:30:06.507 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-21T18:30:06.507 INFO:teuthology.orchestra.run.smithi155.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-07-21T18:30:06.507 INFO:teuthology.orchestra.run.smithi155.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-07-21T18:30:06.508 INFO:teuthology.orchestra.run.smithi155.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-07-21T18:30:06.508 INFO:teuthology.orchestra.run.smithi155.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-07-21T18:30:06.508 INFO:teuthology.orchestra.run.smithi155.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-07-21T18:30:06.508 INFO:teuthology.orchestra.run.smithi155.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-07-21T18:30:06.508 INFO:teuthology.orchestra.run.smithi155.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-07-21T18:30:06.508 INFO:teuthology.orchestra.run.smithi155.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-07-21T18:30:06.508 INFO:teuthology.orchestra.run.smithi155.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-07-21T18:30:06.508 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-07-21T18:30:06.509 INFO:teuthology.orchestra.run.smithi155.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-07-21T18:30:06.509 INFO:teuthology.orchestra.run.smithi155.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-07-21T18:30:06.509 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-07-21T18:30:06.509 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-07-21T18:30:06.509 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-07-21T18:30:06.509 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-07-21T18:30:06.509 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-07-21T18:30:06.509 INFO:teuthology.orchestra.run.smithi155.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-07-21T18:30:06.510 INFO:teuthology.orchestra.run.smithi155.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-21T18:30:06.510 INFO:teuthology.orchestra.run.smithi155.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-07-21T18:30:06.510 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-07-21T18:30:06.510 INFO:teuthology.orchestra.run.smithi155.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-07-21T18:30:06.510 INFO:teuthology.orchestra.run.smithi155.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-07-21T18:30:06.510 INFO:teuthology.orchestra.run.smithi155.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-07-21T18:30:06.510 INFO:teuthology.orchestra.run.smithi155.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-07-21T18:30:06.511 INFO:teuthology.orchestra.run.smithi155.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-07-21T18:30:06.511 INFO:teuthology.orchestra.run.smithi155.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-07-21T18:30:06.511 INFO:teuthology.orchestra.run.smithi155.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-07-21T18:30:06.511 INFO:teuthology.orchestra.run.smithi155.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-07-21T18:30:06.511 INFO:teuthology.orchestra.run.smithi155.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-07-21T18:30:06.511 INFO:teuthology.orchestra.run.smithi155.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-07-21T18:30:06.511 INFO:teuthology.orchestra.run.smithi155.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-07-21T18:30:06.511 INFO:teuthology.orchestra.run.smithi155.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-07-21T18:30:06.512 INFO:teuthology.orchestra.run.smithi155.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-07-21T18:30:06.512 INFO:teuthology.orchestra.run.smithi155.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-07-21T18:30:06.512 INFO:teuthology.orchestra.run.smithi155.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-07-21T18:30:06.512 INFO:teuthology.orchestra.run.smithi155.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-07-21T18:30:06.512 INFO:teuthology.orchestra.run.smithi155.stdout:Enabling module streams: 2023-07-21T18:30:06.512 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql 10 2023-07-21T18:30:06.512 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-07-21T18:30:06.512 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2023-07-21T18:30:06.513 INFO:teuthology.orchestra.run.smithi155.stdout:=========================================================================================================================== 2023-07-21T18:30:06.513 INFO:teuthology.orchestra.run.smithi155.stdout:Install 128 Packages 2023-07-21T18:30:06.513 INFO:teuthology.orchestra.run.smithi155.stdout:Upgrade 3 Packages 2023-07-21T18:30:06.513 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-07-21T18:30:06.513 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 205 M 2023-07-21T18:30:06.514 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2023-07-21T18:30:07.952 INFO:teuthology.orchestra.run.smithi155.stdout:(1/131): ceph-18.0.0-4746.g81cf785a.el8.x86_64. 20 kB/s | 6.4 kB 00:00 2023-07-21T18:30:08.569 INFO:teuthology.orchestra.run.smithi155.stdout:(2/131): ceph-fuse-18.0.0-4746.g81cf785a.el8.x8 1.3 MB/s | 848 kB 00:00 2023-07-21T18:30:08.695 INFO:teuthology.orchestra.run.smithi155.stdout:(3/131): ceph-immutable-object-cache-18.0.0-474 1.3 MB/s | 165 kB 00:00 2023-07-21T18:30:09.428 INFO:teuthology.orchestra.run.smithi155.stdout:(4/131): ceph-mds-18.0.0-4746.g81cf785a.el8.x86 3.1 MB/s | 2.3 MB 00:00 2023-07-21T18:30:09.853 INFO:teuthology.orchestra.run.smithi155.stdout:(5/131): ceph-mgr-18.0.0-4746.g81cf785a.el8.x86 3.6 MB/s | 1.5 MB 00:00 2023-07-21T18:30:10.328 INFO:teuthology.orchestra.run.smithi155.stdout:(6/131): ceph-base-18.0.0-4746.g81cf785a.el8.x8 2.0 MB/s | 5.4 MB 00:02 2023-07-21T18:30:10.737 INFO:teuthology.orchestra.run.smithi155.stdout:(7/131): ceph-mon-18.0.0-4746.g81cf785a.el8.x86 5.2 MB/s | 4.6 MB 00:00 2023-07-21T18:30:12.037 INFO:teuthology.orchestra.run.smithi155.stdout:(8/131): ceph-radosgw-18.0.0-4746.g81cf785a.el8 10 MB/s | 13 MB 00:01 2023-07-21T18:30:12.121 INFO:teuthology.orchestra.run.smithi155.stdout:(9/131): ceph-selinux-18.0.0-4746.g81cf785a.el8 292 kB/s | 24 kB 00:00 2023-07-21T18:30:14.289 INFO:teuthology.orchestra.run.smithi193.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-21T18:30:14.290 INFO:teuthology.orchestra.run.smithi193.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-21T18:30:14.304 INFO:teuthology.orchestra.run.smithi193.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-07-21T18:30:14.305 INFO:teuthology.orchestra.run.smithi193.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-07-21T18:30:14.306 INFO:teuthology.orchestra.run.smithi193.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-07-21T18:30:14.307 INFO:teuthology.orchestra.run.smithi193.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-07-21T18:30:14.309 INFO:teuthology.orchestra.run.smithi193.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-07-21T18:30:14.312 INFO:teuthology.orchestra.run.smithi193.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-07-21T18:30:14.313 INFO:teuthology.orchestra.run.smithi193.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-07-21T18:30:14.444 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2023-07-21T18:30:14.456 INFO:teuthology.orchestra.run.smithi193.stdout:=========================================================================================================================== 2023-07-21T18:30:14.457 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repository Size 2023-07-21T18:30:14.457 INFO:teuthology.orchestra.run.smithi193.stdout:=========================================================================================================================== 2023-07-21T18:30:14.457 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2023-07-21T18:30:14.457 INFO:teuthology.orchestra.run.smithi193.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-07-21T18:30:14.457 INFO:teuthology.orchestra.run.smithi193.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-07-21T18:30:14.457 INFO:teuthology.orchestra.run.smithi193.stdout: ceph x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 6.4 k 2023-07-21T18:30:14.458 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-base x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 5.4 M 2023-07-21T18:30:14.458 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-fuse x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 848 k 2023-07-21T18:30:14.458 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 165 k 2023-07-21T18:30:14.458 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 1.5 M 2023-07-21T18:30:14.458 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 129 k 2023-07-21T18:30:14.458 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 1.7 M 2023-07-21T18:30:14.459 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 7.4 M 2023-07-21T18:30:14.459 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-rook noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 47 k 2023-07-21T18:30:14.459 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-radosgw x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 13 M 2023-07-21T18:30:14.459 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-test x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 48 M 2023-07-21T18:30:14.459 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-volume noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 258 k 2023-07-21T18:30:14.459 INFO:teuthology.orchestra.run.smithi193.stdout: cephadm noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 166 k 2023-07-21T18:30:14.459 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-mirror x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 258 k 2023-07-21T18:30:14.460 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-top noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 18 k 2023-07-21T18:30:14.460 INFO:teuthology.orchestra.run.smithi193.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-07-21T18:30:14.460 INFO:teuthology.orchestra.run.smithi193.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-07-21T18:30:14.460 INFO:teuthology.orchestra.run.smithi193.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-07-21T18:30:14.460 INFO:teuthology.orchestra.run.smithi193.stdout: libcephfs-devel x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 31 k 2023-07-21T18:30:14.460 INFO:teuthology.orchestra.run.smithi193.stdout: libcephfs2 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 772 k 2023-07-21T18:30:14.460 INFO:teuthology.orchestra.run.smithi193.stdout: librados-devel x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 139 k 2023-07-21T18:30:14.461 INFO:teuthology.orchestra.run.smithi193.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-07-21T18:30:14.461 INFO:teuthology.orchestra.run.smithi193.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-07-21T18:30:14.461 INFO:teuthology.orchestra.run.smithi193.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-07-21T18:30:14.461 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-07-21T18:30:14.461 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-07-21T18:30:14.461 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-07-21T18:30:14.461 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cephfs x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 197 k 2023-07-21T18:30:14.462 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rados x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 370 k 2023-07-21T18:30:14.462 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rbd x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 364 k 2023-07-21T18:30:14.462 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rgw x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 107 k 2023-07-21T18:30:14.462 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-fuse x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 94 k 2023-07-21T18:30:14.462 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-mirror x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 3.4 M 2023-07-21T18:30:14.462 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-nbd x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 179 k 2023-07-21T18:30:14.462 INFO:teuthology.orchestra.run.smithi193.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-07-21T18:30:14.463 INFO:teuthology.orchestra.run.smithi193.stdout:Upgrading: 2023-07-21T18:30:14.463 INFO:teuthology.orchestra.run.smithi193.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-07-21T18:30:14.463 INFO:teuthology.orchestra.run.smithi193.stdout: librados2 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 3.7 M 2023-07-21T18:30:14.463 INFO:teuthology.orchestra.run.smithi193.stdout: librbd1 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 3.6 M 2023-07-21T18:30:14.463 INFO:teuthology.orchestra.run.smithi193.stdout:Installing dependencies: 2023-07-21T18:30:14.463 INFO:teuthology.orchestra.run.smithi193.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-07-21T18:30:14.463 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-common x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 22 M 2023-07-21T18:30:14.464 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 23 k 2023-07-21T18:30:14.464 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mds x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 2.3 M 2023-07-21T18:30:14.464 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 240 k 2023-07-21T18:30:14.464 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mon x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 4.6 M 2023-07-21T18:30:14.464 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-osd x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 18 M 2023-07-21T18:30:14.464 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 15 k 2023-07-21T18:30:14.464 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-selinux x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 24 k 2023-07-21T18:30:14.465 INFO:teuthology.orchestra.run.smithi193.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-07-21T18:30:14.465 INFO:teuthology.orchestra.run.smithi193.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-07-21T18:30:14.465 INFO:teuthology.orchestra.run.smithi193.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-07-21T18:30:14.465 INFO:teuthology.orchestra.run.smithi193.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-07-21T18:30:14.465 INFO:teuthology.orchestra.run.smithi193.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-07-21T18:30:14.465 INFO:teuthology.orchestra.run.smithi193.stdout: libcephsqlite x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 137 k 2023-07-21T18:30:14.465 INFO:teuthology.orchestra.run.smithi193.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-07-21T18:30:14.466 INFO:teuthology.orchestra.run.smithi193.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-07-21T18:30:14.466 INFO:teuthology.orchestra.run.smithi193.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-07-21T18:30:14.466 INFO:teuthology.orchestra.run.smithi193.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-07-21T18:30:14.466 INFO:teuthology.orchestra.run.smithi193.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-07-21T18:30:14.466 INFO:teuthology.orchestra.run.smithi193.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-07-21T18:30:14.466 INFO:teuthology.orchestra.run.smithi193.stdout: libradosstriper1 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 473 k 2023-07-21T18:30:14.467 INFO:teuthology.orchestra.run.smithi193.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-07-21T18:30:14.467 INFO:teuthology.orchestra.run.smithi193.stdout: librgw2 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 6.7 M 2023-07-21T18:30:14.467 INFO:teuthology.orchestra.run.smithi193.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-07-21T18:30:14.467 INFO:teuthology.orchestra.run.smithi193.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-07-21T18:30:14.467 INFO:teuthology.orchestra.run.smithi193.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-07-21T18:30:14.467 INFO:teuthology.orchestra.run.smithi193.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-07-21T18:30:14.467 INFO:teuthology.orchestra.run.smithi193.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-07-21T18:30:14.468 INFO:teuthology.orchestra.run.smithi193.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-07-21T18:30:14.468 INFO:teuthology.orchestra.run.smithi193.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-07-21T18:30:14.468 INFO:teuthology.orchestra.run.smithi193.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-07-21T18:30:14.468 INFO:teuthology.orchestra.run.smithi193.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-07-21T18:30:14.468 INFO:teuthology.orchestra.run.smithi193.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-07-21T18:30:14.468 INFO:teuthology.orchestra.run.smithi193.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-07-21T18:30:14.468 INFO:teuthology.orchestra.run.smithi193.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-07-21T18:30:14.469 INFO:teuthology.orchestra.run.smithi193.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-07-21T18:30:14.469 INFO:teuthology.orchestra.run.smithi193.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-07-21T18:30:14.469 INFO:teuthology.orchestra.run.smithi193.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-07-21T18:30:14.469 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-07-21T18:30:14.470 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ceph-argparse x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 45 k 2023-07-21T18:30:14.470 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ceph-common x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 119 k 2023-07-21T18:30:14.470 INFO:teuthology.orchestra.run.smithi193.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-07-21T18:30:14.470 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-07-21T18:30:14.470 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-07-21T18:30:14.471 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-07-21T18:30:14.471 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-07-21T18:30:14.471 INFO:teuthology.orchestra.run.smithi193.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-07-21T18:30:14.471 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-07-21T18:30:14.471 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-07-21T18:30:14.471 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-07-21T18:30:14.471 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-07-21T18:30:14.472 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-07-21T18:30:14.472 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-21T18:30:14.472 INFO:teuthology.orchestra.run.smithi193.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-07-21T18:30:14.472 INFO:teuthology.orchestra.run.smithi193.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-07-21T18:30:14.472 INFO:teuthology.orchestra.run.smithi193.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-07-21T18:30:14.472 INFO:teuthology.orchestra.run.smithi193.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-07-21T18:30:14.473 INFO:teuthology.orchestra.run.smithi193.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-07-21T18:30:14.473 INFO:teuthology.orchestra.run.smithi193.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-07-21T18:30:14.473 INFO:teuthology.orchestra.run.smithi193.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-07-21T18:30:14.473 INFO:teuthology.orchestra.run.smithi193.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-07-21T18:30:14.473 INFO:teuthology.orchestra.run.smithi193.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-07-21T18:30:14.473 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-07-21T18:30:14.473 INFO:teuthology.orchestra.run.smithi193.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-07-21T18:30:14.474 INFO:teuthology.orchestra.run.smithi193.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-07-21T18:30:14.474 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-07-21T18:30:14.474 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-07-21T18:30:14.474 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-07-21T18:30:14.474 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-07-21T18:30:14.474 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-07-21T18:30:14.474 INFO:teuthology.orchestra.run.smithi193.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-07-21T18:30:14.475 INFO:teuthology.orchestra.run.smithi193.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-21T18:30:14.475 INFO:teuthology.orchestra.run.smithi193.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-07-21T18:30:14.475 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-07-21T18:30:14.475 INFO:teuthology.orchestra.run.smithi193.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-07-21T18:30:14.475 INFO:teuthology.orchestra.run.smithi193.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-07-21T18:30:14.475 INFO:teuthology.orchestra.run.smithi193.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-07-21T18:30:14.475 INFO:teuthology.orchestra.run.smithi193.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-07-21T18:30:14.475 INFO:teuthology.orchestra.run.smithi193.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-07-21T18:30:14.476 INFO:teuthology.orchestra.run.smithi193.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-07-21T18:30:14.476 INFO:teuthology.orchestra.run.smithi193.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-07-21T18:30:14.476 INFO:teuthology.orchestra.run.smithi193.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-07-21T18:30:14.476 INFO:teuthology.orchestra.run.smithi193.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-07-21T18:30:14.476 INFO:teuthology.orchestra.run.smithi193.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-07-21T18:30:14.476 INFO:teuthology.orchestra.run.smithi193.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-07-21T18:30:14.477 INFO:teuthology.orchestra.run.smithi193.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-07-21T18:30:14.477 INFO:teuthology.orchestra.run.smithi193.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-07-21T18:30:14.477 INFO:teuthology.orchestra.run.smithi193.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-07-21T18:30:14.477 INFO:teuthology.orchestra.run.smithi193.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-07-21T18:30:14.477 INFO:teuthology.orchestra.run.smithi193.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-07-21T18:30:14.477 INFO:teuthology.orchestra.run.smithi193.stdout:Enabling module streams: 2023-07-21T18:30:14.477 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql 10 2023-07-21T18:30:14.478 INFO:teuthology.orchestra.run.smithi193.stdout: 2023-07-21T18:30:14.478 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2023-07-21T18:30:14.478 INFO:teuthology.orchestra.run.smithi193.stdout:=========================================================================================================================== 2023-07-21T18:30:14.478 INFO:teuthology.orchestra.run.smithi193.stdout:Install 128 Packages 2023-07-21T18:30:14.478 INFO:teuthology.orchestra.run.smithi193.stdout:Upgrade 3 Packages 2023-07-21T18:30:14.478 INFO:teuthology.orchestra.run.smithi193.stdout: 2023-07-21T18:30:14.479 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 205 M 2023-07-21T18:30:14.479 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2023-07-21T18:30:14.792 INFO:teuthology.orchestra.run.smithi193.stdout:(1/131): ceph-18.0.0-4746.g81cf785a.el8.x86_64. 23 kB/s | 6.4 kB 00:00 2023-07-21T18:30:14.921 INFO:teuthology.orchestra.run.smithi155.stdout:(10/131): ceph-test-18.0.0-4746.g81cf785a.el8.x 17 MB/s | 48 MB 00:02 2023-07-21T18:30:15.005 INFO:teuthology.orchestra.run.smithi155.stdout:(11/131): cephfs-mirror-18.0.0-4746.g81cf785a.e 3.0 MB/s | 258 kB 00:00 2023-07-21T18:30:15.080 INFO:teuthology.orchestra.run.smithi155.stdout:(12/131): libcephfs-devel-18.0.0-4746.g81cf785a 414 kB/s | 31 kB 00:00 2023-07-21T18:30:15.247 INFO:teuthology.orchestra.run.smithi155.stdout:(13/131): libcephfs2-18.0.0-4746.g81cf785a.el8. 4.5 MB/s | 772 kB 00:00 2023-07-21T18:30:15.334 INFO:teuthology.orchestra.run.smithi193.stdout:(2/131): ceph-fuse-18.0.0-4746.g81cf785a.el8.x8 1.5 MB/s | 848 kB 00:00 2023-07-21T18:30:15.352 INFO:teuthology.orchestra.run.smithi155.stdout:(14/131): libcephsqlite-18.0.0-4746.g81cf785a.e 1.3 MB/s | 137 kB 00:00 2023-07-21T18:30:15.447 INFO:teuthology.orchestra.run.smithi155.stdout:(15/131): librados-devel-18.0.0-4746.g81cf785a. 1.4 MB/s | 139 kB 00:00 2023-07-21T18:30:15.484 INFO:teuthology.orchestra.run.smithi193.stdout:(3/131): ceph-immutable-object-cache-18.0.0-474 1.1 MB/s | 165 kB 00:00 2023-07-21T18:30:15.564 INFO:teuthology.orchestra.run.smithi155.stdout:(16/131): libradosstriper1-18.0.0-4746.g81cf785 4.0 MB/s | 473 kB 00:00 2023-07-21T18:30:16.131 INFO:teuthology.orchestra.run.smithi155.stdout:(17/131): librgw2-18.0.0-4746.g81cf785a.el8.x86 12 MB/s | 6.7 MB 00:00 2023-07-21T18:30:16.215 INFO:teuthology.orchestra.run.smithi155.stdout:(18/131): python3-ceph-argparse-18.0.0-4746.g81 539 kB/s | 45 kB 00:00 2023-07-21T18:30:16.293 INFO:teuthology.orchestra.run.smithi193.stdout:(4/131): ceph-base-18.0.0-4746.g81cf785a.el8.x8 3.0 MB/s | 5.4 MB 00:01 2023-07-21T18:30:16.298 INFO:teuthology.orchestra.run.smithi155.stdout:(19/131): python3-ceph-common-18.0.0-4746.g81cf 1.4 MB/s | 119 kB 00:00 2023-07-21T18:30:16.351 INFO:teuthology.orchestra.run.smithi193.stdout:(5/131): ceph-mds-18.0.0-4746.g81cf785a.el8.x86 2.6 MB/s | 2.3 MB 00:00 2023-07-21T18:30:16.382 INFO:teuthology.orchestra.run.smithi155.stdout:(20/131): python3-cephfs-18.0.0-4746.g81cf785a. 2.3 MB/s | 197 kB 00:00 2023-07-21T18:30:16.491 INFO:teuthology.orchestra.run.smithi155.stdout:(21/131): python3-rados-18.0.0-4746.g81cf785a.e 3.3 MB/s | 370 kB 00:00 2023-07-21T18:30:16.624 INFO:teuthology.orchestra.run.smithi155.stdout:(22/131): python3-rbd-18.0.0-4746.g81cf785a.el8 2.7 MB/s | 364 kB 00:00 2023-07-21T18:30:16.626 INFO:teuthology.orchestra.run.smithi193.stdout:(6/131): ceph-mgr-18.0.0-4746.g81cf785a.el8.x86 4.6 MB/s | 1.5 MB 00:00 2023-07-21T18:30:16.632 INFO:teuthology.orchestra.run.smithi039.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-21T18:30:16.633 INFO:teuthology.orchestra.run.smithi039.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-21T18:30:16.647 INFO:teuthology.orchestra.run.smithi039.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-07-21T18:30:16.648 INFO:teuthology.orchestra.run.smithi039.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-07-21T18:30:16.649 INFO:teuthology.orchestra.run.smithi039.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-07-21T18:30:16.651 INFO:teuthology.orchestra.run.smithi039.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-07-21T18:30:16.653 INFO:teuthology.orchestra.run.smithi039.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-07-21T18:30:16.656 INFO:teuthology.orchestra.run.smithi039.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-07-21T18:30:16.657 INFO:teuthology.orchestra.run.smithi039.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-07-21T18:30:16.750 INFO:teuthology.orchestra.run.smithi155.stdout:(23/131): python3-rgw-18.0.0-4746.g81cf785a.el8 854 kB/s | 107 kB 00:00 2023-07-21T18:30:16.797 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2023-07-21T18:30:16.810 INFO:teuthology.orchestra.run.smithi039.stdout:=========================================================================================================================== 2023-07-21T18:30:16.810 INFO:teuthology.orchestra.run.smithi039.stdout: Package Arch Version Repository Size 2023-07-21T18:30:16.810 INFO:teuthology.orchestra.run.smithi039.stdout:=========================================================================================================================== 2023-07-21T18:30:16.810 INFO:teuthology.orchestra.run.smithi039.stdout:Installing: 2023-07-21T18:30:16.811 INFO:teuthology.orchestra.run.smithi039.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-07-21T18:30:16.811 INFO:teuthology.orchestra.run.smithi039.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-07-21T18:30:16.811 INFO:teuthology.orchestra.run.smithi039.stdout: ceph x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 6.4 k 2023-07-21T18:30:16.811 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-base x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 5.4 M 2023-07-21T18:30:16.811 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-fuse x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 848 k 2023-07-21T18:30:16.811 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 165 k 2023-07-21T18:30:16.811 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 1.5 M 2023-07-21T18:30:16.812 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 129 k 2023-07-21T18:30:16.812 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 1.7 M 2023-07-21T18:30:16.812 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 7.4 M 2023-07-21T18:30:16.812 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-rook noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 47 k 2023-07-21T18:30:16.812 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-radosgw x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 13 M 2023-07-21T18:30:16.812 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-test x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 48 M 2023-07-21T18:30:16.812 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-volume noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 258 k 2023-07-21T18:30:16.812 INFO:teuthology.orchestra.run.smithi039.stdout: cephadm noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 166 k 2023-07-21T18:30:16.813 INFO:teuthology.orchestra.run.smithi039.stdout: cephfs-mirror x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 258 k 2023-07-21T18:30:16.813 INFO:teuthology.orchestra.run.smithi039.stdout: cephfs-top noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 18 k 2023-07-21T18:30:16.813 INFO:teuthology.orchestra.run.smithi039.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-07-21T18:30:16.813 INFO:teuthology.orchestra.run.smithi039.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-07-21T18:30:16.813 INFO:teuthology.orchestra.run.smithi039.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-07-21T18:30:16.813 INFO:teuthology.orchestra.run.smithi039.stdout: libcephfs-devel x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 31 k 2023-07-21T18:30:16.813 INFO:teuthology.orchestra.run.smithi039.stdout: libcephfs2 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 772 k 2023-07-21T18:30:16.814 INFO:teuthology.orchestra.run.smithi039.stdout: librados-devel x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 139 k 2023-07-21T18:30:16.814 INFO:teuthology.orchestra.run.smithi039.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-07-21T18:30:16.814 INFO:teuthology.orchestra.run.smithi039.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-07-21T18:30:16.814 INFO:teuthology.orchestra.run.smithi039.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-07-21T18:30:16.814 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-07-21T18:30:16.814 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-07-21T18:30:16.814 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-07-21T18:30:16.814 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cephfs x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 197 k 2023-07-21T18:30:16.815 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rados x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 370 k 2023-07-21T18:30:16.815 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rbd x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 364 k 2023-07-21T18:30:16.815 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rgw x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 107 k 2023-07-21T18:30:16.815 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-fuse x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 94 k 2023-07-21T18:30:16.815 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-mirror x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 3.4 M 2023-07-21T18:30:16.815 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-nbd x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 179 k 2023-07-21T18:30:16.815 INFO:teuthology.orchestra.run.smithi039.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-07-21T18:30:16.815 INFO:teuthology.orchestra.run.smithi039.stdout:Upgrading: 2023-07-21T18:30:16.816 INFO:teuthology.orchestra.run.smithi039.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-07-21T18:30:16.816 INFO:teuthology.orchestra.run.smithi039.stdout: librados2 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 3.7 M 2023-07-21T18:30:16.816 INFO:teuthology.orchestra.run.smithi039.stdout: librbd1 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 3.6 M 2023-07-21T18:30:16.816 INFO:teuthology.orchestra.run.smithi039.stdout:Installing dependencies: 2023-07-21T18:30:16.816 INFO:teuthology.orchestra.run.smithi039.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-07-21T18:30:16.816 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-common x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 22 M 2023-07-21T18:30:16.816 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 23 k 2023-07-21T18:30:16.816 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mds x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 2.3 M 2023-07-21T18:30:16.817 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 240 k 2023-07-21T18:30:16.817 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mon x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 4.6 M 2023-07-21T18:30:16.817 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-osd x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 18 M 2023-07-21T18:30:16.817 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 15 k 2023-07-21T18:30:16.817 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-selinux x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 24 k 2023-07-21T18:30:16.817 INFO:teuthology.orchestra.run.smithi039.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-07-21T18:30:16.817 INFO:teuthology.orchestra.run.smithi039.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-07-21T18:30:16.818 INFO:teuthology.orchestra.run.smithi039.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-07-21T18:30:16.818 INFO:teuthology.orchestra.run.smithi039.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-07-21T18:30:16.818 INFO:teuthology.orchestra.run.smithi039.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-07-21T18:30:16.818 INFO:teuthology.orchestra.run.smithi039.stdout: libcephsqlite x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 137 k 2023-07-21T18:30:16.818 INFO:teuthology.orchestra.run.smithi039.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-07-21T18:30:16.818 INFO:teuthology.orchestra.run.smithi039.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-07-21T18:30:16.818 INFO:teuthology.orchestra.run.smithi039.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-07-21T18:30:16.818 INFO:teuthology.orchestra.run.smithi039.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-07-21T18:30:16.819 INFO:teuthology.orchestra.run.smithi039.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-07-21T18:30:16.819 INFO:teuthology.orchestra.run.smithi039.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-07-21T18:30:16.819 INFO:teuthology.orchestra.run.smithi039.stdout: libradosstriper1 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 473 k 2023-07-21T18:30:16.819 INFO:teuthology.orchestra.run.smithi039.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-07-21T18:30:16.819 INFO:teuthology.orchestra.run.smithi039.stdout: librgw2 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 6.7 M 2023-07-21T18:30:16.819 INFO:teuthology.orchestra.run.smithi039.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-07-21T18:30:16.819 INFO:teuthology.orchestra.run.smithi039.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-07-21T18:30:16.819 INFO:teuthology.orchestra.run.smithi039.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-07-21T18:30:16.820 INFO:teuthology.orchestra.run.smithi039.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-07-21T18:30:16.820 INFO:teuthology.orchestra.run.smithi039.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-07-21T18:30:16.820 INFO:teuthology.orchestra.run.smithi039.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-07-21T18:30:16.820 INFO:teuthology.orchestra.run.smithi039.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-07-21T18:30:16.820 INFO:teuthology.orchestra.run.smithi039.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-07-21T18:30:16.820 INFO:teuthology.orchestra.run.smithi039.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-07-21T18:30:16.820 INFO:teuthology.orchestra.run.smithi039.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-07-21T18:30:16.820 INFO:teuthology.orchestra.run.smithi039.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-07-21T18:30:16.821 INFO:teuthology.orchestra.run.smithi039.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-07-21T18:30:16.821 INFO:teuthology.orchestra.run.smithi039.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-07-21T18:30:16.821 INFO:teuthology.orchestra.run.smithi039.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-07-21T18:30:16.821 INFO:teuthology.orchestra.run.smithi039.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-07-21T18:30:16.821 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-07-21T18:30:16.821 INFO:teuthology.orchestra.run.smithi039.stdout: python3-ceph-argparse x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 45 k 2023-07-21T18:30:16.821 INFO:teuthology.orchestra.run.smithi039.stdout: python3-ceph-common x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 119 k 2023-07-21T18:30:16.822 INFO:teuthology.orchestra.run.smithi039.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-07-21T18:30:16.822 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-07-21T18:30:16.822 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-07-21T18:30:16.822 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-07-21T18:30:16.822 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-07-21T18:30:16.822 INFO:teuthology.orchestra.run.smithi039.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-07-21T18:30:16.822 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-07-21T18:30:16.822 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-07-21T18:30:16.823 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-07-21T18:30:16.823 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-07-21T18:30:16.823 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-07-21T18:30:16.823 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-21T18:30:16.823 INFO:teuthology.orchestra.run.smithi039.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-07-21T18:30:16.823 INFO:teuthology.orchestra.run.smithi039.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-07-21T18:30:16.823 INFO:teuthology.orchestra.run.smithi039.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-07-21T18:30:16.823 INFO:teuthology.orchestra.run.smithi039.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-07-21T18:30:16.824 INFO:teuthology.orchestra.run.smithi039.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-07-21T18:30:16.824 INFO:teuthology.orchestra.run.smithi039.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-07-21T18:30:16.824 INFO:teuthology.orchestra.run.smithi039.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-07-21T18:30:16.824 INFO:teuthology.orchestra.run.smithi039.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-07-21T18:30:16.824 INFO:teuthology.orchestra.run.smithi039.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-07-21T18:30:16.824 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-07-21T18:30:16.824 INFO:teuthology.orchestra.run.smithi039.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-07-21T18:30:16.824 INFO:teuthology.orchestra.run.smithi039.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-07-21T18:30:16.825 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-07-21T18:30:16.825 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-07-21T18:30:16.825 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-07-21T18:30:16.825 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-07-21T18:30:16.825 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-07-21T18:30:16.825 INFO:teuthology.orchestra.run.smithi039.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-07-21T18:30:16.825 INFO:teuthology.orchestra.run.smithi039.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-21T18:30:16.825 INFO:teuthology.orchestra.run.smithi039.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-07-21T18:30:16.826 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-07-21T18:30:16.826 INFO:teuthology.orchestra.run.smithi039.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-07-21T18:30:16.826 INFO:teuthology.orchestra.run.smithi039.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-07-21T18:30:16.826 INFO:teuthology.orchestra.run.smithi039.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-07-21T18:30:16.826 INFO:teuthology.orchestra.run.smithi039.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-07-21T18:30:16.826 INFO:teuthology.orchestra.run.smithi039.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-07-21T18:30:16.826 INFO:teuthology.orchestra.run.smithi039.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-07-21T18:30:16.827 INFO:teuthology.orchestra.run.smithi039.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-07-21T18:30:16.827 INFO:teuthology.orchestra.run.smithi039.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-07-21T18:30:16.827 INFO:teuthology.orchestra.run.smithi039.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-07-21T18:30:16.827 INFO:teuthology.orchestra.run.smithi039.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-07-21T18:30:16.827 INFO:teuthology.orchestra.run.smithi039.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-07-21T18:30:16.827 INFO:teuthology.orchestra.run.smithi039.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-07-21T18:30:16.827 INFO:teuthology.orchestra.run.smithi039.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-07-21T18:30:16.827 INFO:teuthology.orchestra.run.smithi039.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-07-21T18:30:16.828 INFO:teuthology.orchestra.run.smithi039.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-07-21T18:30:16.828 INFO:teuthology.orchestra.run.smithi039.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-07-21T18:30:16.828 INFO:teuthology.orchestra.run.smithi039.stdout:Enabling module streams: 2023-07-21T18:30:16.828 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql 10 2023-07-21T18:30:16.828 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:30:16.828 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction Summary 2023-07-21T18:30:16.828 INFO:teuthology.orchestra.run.smithi039.stdout:=========================================================================================================================== 2023-07-21T18:30:16.828 INFO:teuthology.orchestra.run.smithi039.stdout:Install 128 Packages 2023-07-21T18:30:16.829 INFO:teuthology.orchestra.run.smithi039.stdout:Upgrade 3 Packages 2023-07-21T18:30:16.829 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:30:16.829 INFO:teuthology.orchestra.run.smithi039.stdout:Total download size: 205 M 2023-07-21T18:30:16.830 INFO:teuthology.orchestra.run.smithi039.stdout:Downloading Packages: 2023-07-21T18:30:16.842 INFO:teuthology.orchestra.run.smithi155.stdout:(24/131): rbd-fuse-18.0.0-4746.g81cf785a.el8.x8 1.0 MB/s | 94 kB 00:00 2023-07-21T18:30:17.292 INFO:teuthology.orchestra.run.smithi155.stdout:(25/131): ceph-osd-18.0.0-4746.g81cf785a.el8.x8 2.5 MB/s | 18 MB 00:06 2023-07-21T18:30:17.334 INFO:teuthology.orchestra.run.smithi039.stdout:(1/131): ceph-18.0.0-4746.g81cf785a.el8.x86_64. 24 kB/s | 6.4 kB 00:00 2023-07-21T18:30:17.485 INFO:teuthology.orchestra.run.smithi193.stdout:(7/131): ceph-mon-18.0.0-4746.g81cf785a.el8.x86 4.1 MB/s | 4.6 MB 00:01 2023-07-21T18:30:17.492 INFO:teuthology.orchestra.run.smithi155.stdout:(26/131): ceph-common-18.0.0-4746.g81cf785a.el8 2.2 MB/s | 22 MB 00:09 2023-07-21T18:30:17.517 INFO:teuthology.orchestra.run.smithi155.stdout:(27/131): rbd-nbd-18.0.0-4746.g81cf785a.el8.x86 798 kB/s | 179 kB 00:00 2023-07-21T18:30:17.598 INFO:teuthology.orchestra.run.smithi155.stdout:(28/131): rbd-mirror-18.0.0-4746.g81cf785a.el8. 4.6 MB/s | 3.4 MB 00:00 2023-07-21T18:30:17.626 INFO:teuthology.orchestra.run.smithi155.stdout:(29/131): ceph-grafana-dashboards-18.0.0-4746.g 173 kB/s | 23 kB 00:00 2023-07-21T18:30:17.652 INFO:teuthology.orchestra.run.smithi155.stdout:(30/131): ceph-mgr-cephadm-18.0.0-4746.g81cf785 963 kB/s | 129 kB 00:00 2023-07-21T18:30:17.902 INFO:teuthology.orchestra.run.smithi155.stdout:(31/131): ceph-mgr-modules-core-18.0.0-4746.g81 962 kB/s | 240 kB 00:00 2023-07-21T18:30:17.977 INFO:teuthology.orchestra.run.smithi155.stdout:(32/131): ceph-mgr-dashboard-18.0.0-4746.g81cf7 4.5 MB/s | 1.7 MB 00:00 2023-07-21T18:30:18.002 INFO:teuthology.orchestra.run.smithi155.stdout:(33/131): ceph-mgr-rook-18.0.0-4746.g81cf785a.e 473 kB/s | 47 kB 00:00 2023-07-21T18:30:18.042 INFO:teuthology.orchestra.run.smithi039.stdout:(2/131): ceph-fuse-18.0.0-4746.g81cf785a.el8.x8 1.2 MB/s | 848 kB 00:00 2023-07-21T18:30:18.078 INFO:teuthology.orchestra.run.smithi155.stdout:(34/131): ceph-prometheus-alerts-18.0.0-4746.g8 145 kB/s | 15 kB 00:00 2023-07-21T18:30:18.186 INFO:teuthology.orchestra.run.smithi155.stdout:(35/131): ceph-volume-18.0.0-4746.g81cf785a.el8 1.4 MB/s | 258 kB 00:00 2023-07-21T18:30:18.212 INFO:teuthology.orchestra.run.smithi155.stdout:(36/131): cephadm-18.0.0-4746.g81cf785a.el8.noa 1.2 MB/s | 166 kB 00:00 2023-07-21T18:30:18.226 INFO:teuthology.orchestra.run.smithi039.stdout:(3/131): ceph-immutable-object-cache-18.0.0-474 902 kB/s | 165 kB 00:00 2023-07-21T18:30:18.278 INFO:teuthology.orchestra.run.smithi155.stdout:(37/131): cephfs-top-18.0.0-4746.g81cf785a.el8. 195 kB/s | 18 kB 00:00 2023-07-21T18:30:18.587 INFO:teuthology.orchestra.run.smithi155.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 191 kB/s | 59 kB 00:00 2023-07-21T18:30:18.677 INFO:teuthology.orchestra.run.smithi193.stdout:(8/131): ceph-osd-18.0.0-4746.g81cf785a.el8.x86 8.6 MB/s | 18 MB 00:02 2023-07-21T18:30:18.760 INFO:teuthology.orchestra.run.smithi193.stdout:(9/131): ceph-selinux-18.0.0-4746.g81cf785a.el8 292 kB/s | 24 kB 00:00 2023-07-21T18:30:18.762 INFO:teuthology.orchestra.run.smithi155.stdout:(39/131): gperftools-libs-2.7-9.el8.x86_64.rpm 556 kB/s | 306 kB 00:00 2023-07-21T18:30:18.788 INFO:teuthology.orchestra.run.smithi155.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 375 kB/s | 75 kB 00:00 2023-07-21T18:30:18.904 INFO:teuthology.orchestra.run.smithi155.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 375 kB/s | 44 kB 00:00 2023-07-21T18:30:19.060 INFO:teuthology.orchestra.run.smithi039.stdout:(4/131): ceph-base-18.0.0-4746.g81cf785a.el8.x8 2.7 MB/s | 5.4 MB 00:01 2023-07-21T18:30:19.080 INFO:teuthology.orchestra.run.smithi155.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.0 MB/s | 185 kB 00:00 2023-07-21T18:30:19.118 INFO:teuthology.orchestra.run.smithi039.stdout:(5/131): ceph-mds-18.0.0-4746.g81cf785a.el8.x86 2.6 MB/s | 2.3 MB 00:00 2023-07-21T18:30:19.147 INFO:teuthology.orchestra.run.smithi155.stdout:(43/131): python3-asyncssh-2.7.0-2.el8.noarch.r 1.2 MB/s | 476 kB 00:00 2023-07-21T18:30:19.172 INFO:teuthology.orchestra.run.smithi155.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 359 kB/s | 33 kB 00:00 2023-07-21T18:30:19.239 INFO:teuthology.orchestra.run.smithi155.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 177 kB/s | 16 kB 00:00 2023-07-21T18:30:19.339 INFO:teuthology.orchestra.run.smithi155.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.0 MB/s | 173 kB 00:00 2023-07-21T18:30:19.439 INFO:teuthology.orchestra.run.smithi155.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 813 kB/s | 81 kB 00:00 2023-07-21T18:30:19.477 INFO:teuthology.orchestra.run.smithi039.stdout:(6/131): ceph-mgr-18.0.0-4746.g81cf785a.el8.x86 3.7 MB/s | 1.5 MB 00:00 2023-07-21T18:30:19.481 INFO:teuthology.orchestra.run.smithi155.stdout:(48/131): python3-cherrypy-18.4.0-1.el8.noarch. 1.6 MB/s | 384 kB 00:00 2023-07-21T18:30:19.531 INFO:teuthology.orchestra.run.smithi155.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 121 kB/s | 11 kB 00:00 2023-07-21T18:30:19.573 INFO:teuthology.orchestra.run.smithi155.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 192 kB/s | 18 kB 00:00 2023-07-21T18:30:19.674 INFO:teuthology.orchestra.run.smithi155.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 485 kB/s | 49 kB 00:00 2023-07-21T18:30:19.766 INFO:teuthology.orchestra.run.smithi155.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 640 kB/s | 59 kB 00:00 2023-07-21T18:30:19.866 INFO:teuthology.orchestra.run.smithi155.stdout:(53/131): python3-natsort-7.1.1-2.el8.noarch.rp 604 kB/s | 60 kB 00:00 2023-07-21T18:30:20.051 INFO:teuthology.orchestra.run.smithi155.stdout:(54/131): python3-pecan-1.3.2-9.el8.noarch.rpm 1.5 MB/s | 283 kB 00:00 2023-07-21T18:30:20.142 INFO:teuthology.orchestra.run.smithi155.stdout:(55/131): python3-portend-2.6-1.el8.noarch.rpm 170 kB/s | 16 kB 00:00 2023-07-21T18:30:20.242 INFO:teuthology.orchestra.run.smithi155.stdout:(56/131): python3-kubernetes-11.0.0-6.el8.noarc 1.6 MB/s | 1.2 MB 00:00 2023-07-21T18:30:20.289 INFO:teuthology.orchestra.run.smithi155.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 230 kB/s | 34 kB 00:00 2023-07-21T18:30:20.384 INFO:teuthology.orchestra.run.smithi155.stdout:(58/131): python3-rsa-4.8-1.el8.noarch.rpm 646 kB/s | 61 kB 00:00 2023-07-21T18:30:20.410 INFO:teuthology.orchestra.run.smithi155.stdout:(59/131): python3-routes-2.4.1-12.el8.noarch.rp 1.1 MB/s | 196 kB 00:00 2023-07-21T18:30:20.468 INFO:teuthology.orchestra.run.smithi155.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 224 kB/s | 19 kB 00:00 2023-07-21T18:30:20.510 INFO:teuthology.orchestra.run.smithi155.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 239 kB/s | 24 kB 00:00 2023-07-21T18:30:20.560 INFO:teuthology.orchestra.run.smithi039.stdout:(7/131): ceph-mon-18.0.0-4746.g81cf785a.el8.x86 3.2 MB/s | 4.6 MB 00:01 2023-07-21T18:30:20.561 INFO:teuthology.orchestra.run.smithi155.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 319 kB/s | 29 kB 00:00 2023-07-21T18:30:20.602 INFO:teuthology.orchestra.run.smithi155.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 299 kB/s | 27 kB 00:00 2023-07-21T18:30:20.761 INFO:teuthology.orchestra.run.smithi155.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 1.2 MB/s | 255 kB 00:00 2023-07-21T18:30:20.795 INFO:teuthology.orchestra.run.smithi155.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 1.3 MB/s | 251 kB 00:00 2023-07-21T18:30:20.861 INFO:teuthology.orchestra.run.smithi155.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 610 kB/s | 61 kB 00:00 2023-07-21T18:30:20.895 INFO:teuthology.orchestra.run.smithi155.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 853 kB/s | 85 kB 00:00 2023-07-21T18:30:20.970 INFO:teuthology.orchestra.run.smithi155.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 211 kB/s | 23 kB 00:00 2023-07-21T18:30:20.995 INFO:teuthology.orchestra.run.smithi155.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.8 MB/s | 70 kB 00:00 2023-07-21T18:30:21.304 INFO:teuthology.orchestra.run.smithi155.stdout:(70/131): python3-scipy-1.0.0-21.module+el8.5.0 44 MB/s | 14 MB 00:00 2023-07-21T18:30:21.346 INFO:teuthology.orchestra.run.smithi155.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.6 MB/s | 110 kB 00:00 2023-07-21T18:30:21.388 INFO:teuthology.orchestra.run.smithi155.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.3 MB/s | 54 kB 00:00 2023-07-21T18:30:21.430 INFO:teuthology.orchestra.run.smithi155.stdout:(73/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.1 MB/s | 259 kB 00:00 2023-07-21T18:30:21.472 INFO:teuthology.orchestra.run.smithi155.stdout:(74/131): socat-1.7.4.1-1.el8.x86_64.rpm 7.6 MB/s | 323 kB 00:00 2023-07-21T18:30:21.514 INFO:teuthology.orchestra.run.smithi155.stdout:(75/131): python3-markupsafe-0.23-19.el8.x86_64 930 kB/s | 39 kB 00:00 2023-07-21T18:30:21.518 INFO:teuthology.orchestra.run.smithi193.stdout:(10/131): ceph-common-18.0.0-4746.g81cf785a.el8 3.2 MB/s | 22 MB 00:06 2023-07-21T18:30:21.564 INFO:teuthology.orchestra.run.smithi155.stdout:(76/131): bison-3.0.4-10.el8.x86_64.rpm 13 MB/s | 688 kB 00:00 2023-07-21T18:30:21.645 INFO:teuthology.orchestra.run.smithi155.stdout:(77/131): thrift-0.13.0-2.el8.x86_64.rpm 2.3 MB/s | 1.7 MB 00:00 2023-07-21T18:30:21.652 INFO:teuthology.orchestra.run.smithi193.stdout:(11/131): ceph-radosgw-18.0.0-4746.g81cf785a.el 3.2 MB/s | 13 MB 00:04 2023-07-21T18:30:21.690 INFO:teuthology.orchestra.run.smithi155.stdout:(78/131): python3-jinja2-2.10.1-3.el8.noarch.rp 4.2 MB/s | 538 kB 00:00 2023-07-21T18:30:21.782 INFO:teuthology.orchestra.run.smithi155.stdout:(79/131): ceph-mgr-diskprediction-local-18.0.0- 1.8 MB/s | 7.4 MB 00:04 2023-07-21T18:30:21.804 INFO:teuthology.orchestra.run.smithi193.stdout:(12/131): cephfs-mirror-18.0.0-4746.g81cf785a.e 905 kB/s | 258 kB 00:00 2023-07-21T18:30:21.807 INFO:teuthology.orchestra.run.smithi155.stdout:(80/131): python3-mako-1.0.6-13.el8.noarch.rpm 971 kB/s | 157 kB 00:00 2023-07-21T18:30:21.836 INFO:teuthology.orchestra.run.smithi193.stdout:(13/131): libcephfs-devel-18.0.0-4746.g81cf785a 169 kB/s | 31 kB 00:00 2023-07-21T18:30:21.841 INFO:teuthology.orchestra.run.smithi155.stdout:(81/131): oniguruma-6.8.2-2.el8.x86_64.rpm 1.3 MB/s | 187 kB 00:00 2023-07-21T18:30:21.874 INFO:teuthology.orchestra.run.smithi155.stdout:(82/131): automake-1.16.1-7.el8.noarch.rpm 7.6 MB/s | 713 kB 00:00 2023-07-21T18:30:21.900 INFO:teuthology.orchestra.run.smithi155.stdout:(83/131): python3-jsonpatch-1.21-2.el8.noarch.r 455 kB/s | 27 kB 00:00 2023-07-21T18:30:21.933 INFO:teuthology.orchestra.run.smithi155.stdout:(84/131): flex-2.6.1-9.el8.x86_64.rpm 5.3 MB/s | 320 kB 00:00 2023-07-21T18:30:21.959 INFO:teuthology.orchestra.run.smithi155.stdout:(85/131): python3-prettytable-0.7.2-14.el8.noar 750 kB/s | 44 kB 00:00 2023-07-21T18:30:21.961 INFO:teuthology.orchestra.run.smithi193.stdout:(14/131): libcephsqlite-18.0.0-4746.g81cf785a.e 1.1 MB/s | 137 kB 00:00 2023-07-21T18:30:22.051 INFO:teuthology.orchestra.run.smithi155.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 4.9 MB/s | 457 kB 00:00 2023-07-21T18:30:22.062 INFO:teuthology.orchestra.run.smithi193.stdout:(15/131): librados-devel-18.0.0-4746.g81cf785a. 1.4 MB/s | 139 kB 00:00 2023-07-21T18:30:22.109 INFO:teuthology.orchestra.run.smithi155.stdout:(87/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 3.8 MB/s | 225 kB 00:00 2023-07-21T18:30:22.110 INFO:teuthology.orchestra.run.smithi039.stdout:(8/131): ceph-common-18.0.0-4746.g81cf785a.el8. 4.4 MB/s | 22 MB 00:05 2023-07-21T18:30:22.111 INFO:teuthology.orchestra.run.smithi193.stdout:(16/131): libcephfs2-18.0.0-4746.g81cf785a.el8. 2.5 MB/s | 772 kB 00:00 2023-07-21T18:30:22.184 INFO:teuthology.orchestra.run.smithi155.stdout:(88/131): python3-babel-2.5.1-7.el8.noarch.rpm 13 MB/s | 4.8 MB 00:00 2023-07-21T18:30:22.227 INFO:teuthology.orchestra.run.smithi039.stdout:(9/131): ceph-selinux-18.0.0-4746.g81cf785a.el8 209 kB/s | 24 kB 00:00 2023-07-21T18:30:22.243 INFO:teuthology.orchestra.run.smithi155.stdout:(89/131): python3-numpy-1.14.3-10.el8.x86_64.rp 12 MB/s | 3.7 MB 00:00 2023-07-21T18:30:22.260 INFO:teuthology.orchestra.run.smithi155.stdout:(90/131): uuid-1.6.2-43.el8.x86_64.rpm 426 kB/s | 64 kB 00:00 2023-07-21T18:30:22.262 INFO:teuthology.orchestra.run.smithi193.stdout:(17/131): libradosstriper1-18.0.0-4746.g81cf785 2.3 MB/s | 473 kB 00:00 2023-07-21T18:30:22.277 INFO:teuthology.orchestra.run.smithi155.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 600 kB/s | 20 kB 00:00 2023-07-21T18:30:22.302 INFO:teuthology.orchestra.run.smithi155.stdout:(92/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.1 MB/s | 126 kB 00:00 2023-07-21T18:30:22.319 INFO:teuthology.orchestra.run.smithi155.stdout:(93/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 1.7 MB/s | 103 kB 00:00 2023-07-21T18:30:22.361 INFO:teuthology.orchestra.run.smithi155.stdout:(94/131): libtool-2.4.6-25.el8.x86_64.rpm 8.3 MB/s | 709 kB 00:00 2023-07-21T18:30:22.386 INFO:teuthology.orchestra.run.smithi155.stdout:(95/131): libpq-13.5-1.el8.x86_64.rpm 2.9 MB/s | 198 kB 00:00 2023-07-21T18:30:22.387 INFO:teuthology.orchestra.run.smithi193.stdout:(18/131): python3-ceph-argparse-18.0.0-4746.g81 358 kB/s | 45 kB 00:00 2023-07-21T18:30:22.428 INFO:teuthology.orchestra.run.smithi155.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-07-21T18:30:22.478 INFO:teuthology.orchestra.run.smithi155.stdout:(97/131): librdkafka-0.11.4-3.el8.x86_64.rpm 7.0 MB/s | 354 kB 00:00 2023-07-21T18:30:22.496 INFO:teuthology.orchestra.run.smithi193.stdout:(19/131): python3-ceph-common-18.0.0-4746.g81cf 1.1 MB/s | 119 kB 00:00 2023-07-21T18:30:22.511 INFO:teuthology.orchestra.run.smithi039.stdout:(10/131): ceph-radosgw-18.0.0-4746.g81cf785a.el 6.9 MB/s | 13 MB 00:01 2023-07-21T18:30:22.579 INFO:teuthology.orchestra.run.smithi155.stdout:(98/131): openblas-threads-0.3.15-3.el8.x86_64. 18 MB/s | 4.9 MB 00:00 2023-07-21T18:30:22.613 INFO:teuthology.orchestra.run.smithi193.stdout:(20/131): python3-cephfs-18.0.0-4746.g81cf785a. 1.7 MB/s | 197 kB 00:00 2023-07-21T18:30:22.636 INFO:teuthology.orchestra.run.smithi039.stdout:(11/131): cephfs-mirror-18.0.0-4746.g81cf785a.e 2.0 MB/s | 258 kB 00:00 2023-07-21T18:30:22.679 INFO:teuthology.orchestra.run.smithi155.stdout:(99/131): openblas-0.3.15-3.el8.x86_64.rpm 15 MB/s | 4.7 MB 00:00 2023-07-21T18:30:22.721 INFO:teuthology.orchestra.run.smithi155.stdout:(100/131): postgresql-contrib-10.21-2.module+el 5.6 MB/s | 808 kB 00:00 2023-07-21T18:30:22.728 INFO:teuthology.orchestra.run.smithi039.stdout:(12/131): libcephfs-devel-18.0.0-4746.g81cf785a 338 kB/s | 31 kB 00:00 2023-07-21T18:30:22.763 INFO:teuthology.orchestra.run.smithi155.stdout:(101/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2023-07-21T18:30:22.804 INFO:teuthology.orchestra.run.smithi155.stdout:(102/131): postgresql-10.21-2.module+el8.6.0+15 12 MB/s | 1.5 MB 00:00 2023-07-21T18:30:22.805 INFO:teuthology.orchestra.run.smithi193.stdout:(21/131): python3-rados-18.0.0-4746.g81cf785a.e 1.9 MB/s | 370 kB 00:00 2023-07-21T18:30:22.830 INFO:teuthology.orchestra.run.smithi155.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 2.9 MB/s | 200 kB 00:00 2023-07-21T18:30:22.855 INFO:teuthology.orchestra.run.smithi155.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 847 kB/s | 43 kB 00:00 2023-07-21T18:30:22.880 INFO:teuthology.orchestra.run.smithi155.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-07-21T18:30:22.906 INFO:teuthology.orchestra.run.smithi155.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 958 kB/s | 48 kB 00:00 2023-07-21T18:30:22.931 INFO:teuthology.orchestra.run.smithi155.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-07-21T18:30:22.945 INFO:teuthology.orchestra.run.smithi039.stdout:(13/131): ceph-osd-18.0.0-4746.g81cf785a.el8.x8 5.1 MB/s | 18 MB 00:03 2023-07-21T18:30:22.956 INFO:teuthology.orchestra.run.smithi155.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 852 kB/s | 43 kB 00:00 2023-07-21T18:30:22.978 INFO:teuthology.orchestra.run.smithi039.stdout:(14/131): libcephfs2-18.0.0-4746.g81cf785a.el8. 3.0 MB/s | 772 kB 00:00 2023-07-21T18:30:22.988 INFO:teuthology.orchestra.run.smithi193.stdout:(22/131): python3-rbd-18.0.0-4746.g81cf785a.el8 1.9 MB/s | 364 kB 00:00 2023-07-21T18:30:23.015 INFO:teuthology.orchestra.run.smithi155.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 1.3 MB/s | 109 kB 00:00 2023-07-21T18:30:23.040 INFO:teuthology.orchestra.run.smithi155.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 2.7 MB/s | 229 kB 00:00 2023-07-21T18:30:23.045 INFO:teuthology.orchestra.run.smithi039.stdout:(15/131): libcephsqlite-18.0.0-4746.g81cf785a.e 1.3 MB/s | 137 kB 00:00 2023-07-21T18:30:23.071 INFO:teuthology.orchestra.run.smithi039.stdout:(16/131): librados-devel-18.0.0-4746.g81cf785a. 1.5 MB/s | 139 kB 00:00 2023-07-21T18:30:23.089 INFO:teuthology.orchestra.run.smithi193.stdout:(23/131): python3-rgw-18.0.0-4746.g81cf785a.el8 1.0 MB/s | 107 kB 00:00 2023-07-21T18:30:23.091 INFO:teuthology.orchestra.run.smithi155.stdout:(111/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 7.3 MB/s | 560 kB 00:00 2023-07-21T18:30:23.116 INFO:teuthology.orchestra.run.smithi155.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 248 kB/s | 19 kB 00:00 2023-07-21T18:30:23.141 INFO:teuthology.orchestra.run.smithi155.stdout:(113/131): python3-oauthlib-2.1.0-1.el8.noarch. 3.0 MB/s | 155 kB 00:00 2023-07-21T18:30:23.217 INFO:teuthology.orchestra.run.smithi155.stdout:(114/131): postgresql-server-10.21-2.module+el8 6.9 MB/s | 5.1 MB 00:00 2023-07-21T18:30:23.222 INFO:teuthology.orchestra.run.smithi193.stdout:(24/131): rbd-fuse-18.0.0-4746.g81cf785a.el8.x8 708 kB/s | 94 kB 00:00 2023-07-21T18:30:23.242 INFO:teuthology.orchestra.run.smithi155.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 179 kB/s | 18 kB 00:00 2023-07-21T18:30:23.254 INFO:teuthology.orchestra.run.smithi039.stdout:(17/131): libradosstriper1-18.0.0-4746.g81cf785 2.2 MB/s | 473 kB 00:00 2023-07-21T18:30:23.284 INFO:teuthology.orchestra.run.smithi155.stdout:(116/131): python3-cryptography-3.2.1-5.el8.x86 3.3 MB/s | 559 kB 00:00 2023-07-21T18:30:23.325 INFO:teuthology.orchestra.run.smithi155.stdout:(117/131): pcre2-devel-10.32-2.el8.x86_64.rpm 5.4 MB/s | 605 kB 00:00 2023-07-21T18:30:23.351 INFO:teuthology.orchestra.run.smithi155.stdout:(118/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 2.1 MB/s | 238 kB 00:00 2023-07-21T18:30:23.376 INFO:teuthology.orchestra.run.smithi155.stdout:(119/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 1.4 MB/s | 72 kB 00:00 2023-07-21T18:30:23.393 INFO:teuthology.orchestra.run.smithi155.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 1.9 MB/s | 83 kB 00:00 2023-07-21T18:30:23.396 INFO:teuthology.orchestra.run.smithi039.stdout:(18/131): python3-ceph-argparse-18.0.0-4746.g81 316 kB/s | 45 kB 00:00 2023-07-21T18:30:23.437 INFO:teuthology.orchestra.run.smithi155.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 988 kB/s | 60 kB 00:00 2023-07-21T18:30:23.468 INFO:teuthology.orchestra.run.smithi155.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 1.3 MB/s | 98 kB 00:00 2023-07-21T18:30:23.494 INFO:teuthology.orchestra.run.smithi155.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.0 MB/s | 58 kB 00:00 2023-07-21T18:30:23.519 INFO:teuthology.orchestra.run.smithi155.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 769 kB/s | 39 kB 00:00 2023-07-21T18:30:23.553 INFO:teuthology.orchestra.run.smithi155.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 11 MB/s | 644 kB 00:00 2023-07-21T18:30:23.578 INFO:teuthology.orchestra.run.smithi155.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.9 MB/s | 171 kB 00:00 2023-07-21T18:30:23.603 INFO:teuthology.orchestra.run.smithi155.stdout:(127/131): NetworkManager-initscripts-updown-1. 2.7 MB/s | 138 kB 00:00 2023-07-21T18:30:23.662 INFO:teuthology.orchestra.run.smithi155.stdout:(128/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 6.2 MB/s | 2.3 MB 00:00 2023-07-21T18:30:23.663 INFO:teuthology.orchestra.run.smithi039.stdout:(19/131): python3-ceph-common-18.0.0-4746.g81cf 445 kB/s | 119 kB 00:00 2023-07-21T18:30:23.712 INFO:teuthology.orchestra.run.smithi155.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2023-07-21T18:30:23.788 INFO:teuthology.orchestra.run.smithi039.stdout:(20/131): python3-cephfs-18.0.0-4746.g81cf785a. 1.5 MB/s | 197 kB 00:00 2023-07-21T18:30:23.881 INFO:teuthology.orchestra.run.smithi039.stdout:(21/131): librgw2-18.0.0-4746.g81cf785a.el8.x86 8.3 MB/s | 6.7 MB 00:00 2023-07-21T18:30:23.972 INFO:teuthology.orchestra.run.smithi039.stdout:(22/131): python3-rados-18.0.0-4746.g81cf785a.e 2.0 MB/s | 370 kB 00:00 2023-07-21T18:30:23.998 INFO:teuthology.orchestra.run.smithi039.stdout:(23/131): python3-rbd-18.0.0-4746.g81cf785a.el8 3.0 MB/s | 364 kB 00:00 2023-07-21T18:30:24.065 INFO:teuthology.orchestra.run.smithi039.stdout:(24/131): python3-rgw-18.0.0-4746.g81cf785a.el8 1.1 MB/s | 107 kB 00:00 2023-07-21T18:30:24.404 INFO:teuthology.orchestra.run.smithi155.stdout:(130/131): librados2-18.0.0-4746.g81cf785a.el8. 4.4 MB/s | 3.7 MB 00:00 2023-07-21T18:30:24.456 INFO:teuthology.orchestra.run.smithi193.stdout:(25/131): librgw2-18.0.0-4746.g81cf785a.el8.x86 2.9 MB/s | 6.7 MB 00:02 2023-07-21T18:30:24.507 INFO:teuthology.orchestra.run.smithi039.stdout:(25/131): rbd-fuse-18.0.0-4746.g81cf785a.el8.x8 186 kB/s | 94 kB 00:00 2023-07-21T18:30:24.564 INFO:teuthology.orchestra.run.smithi193.stdout:(26/131): rbd-mirror-18.0.0-4746.g81cf785a.el8. 2.6 MB/s | 3.4 MB 00:01 2023-07-21T18:30:24.574 INFO:teuthology.orchestra.run.smithi039.stdout:(26/131): rbd-nbd-18.0.0-4746.g81cf785a.el8.x86 2.6 MB/s | 179 kB 00:00 2023-07-21T18:30:24.615 INFO:teuthology.orchestra.run.smithi193.stdout:(27/131): rbd-nbd-18.0.0-4746.g81cf785a.el8.x86 1.1 MB/s | 179 kB 00:00 2023-07-21T18:30:24.632 INFO:teuthology.orchestra.run.smithi193.stdout:(28/131): ceph-grafana-dashboards-18.0.0-4746.g 345 kB/s | 23 kB 00:00 2023-07-21T18:30:24.640 INFO:teuthology.orchestra.run.smithi039.stdout:(27/131): ceph-grafana-dashboards-18.0.0-4746.g 347 kB/s | 23 kB 00:00 2023-07-21T18:30:24.699 INFO:teuthology.orchestra.run.smithi193.stdout:(29/131): ceph-mgr-cephadm-18.0.0-4746.g81cf785 1.5 MB/s | 129 kB 00:00 2023-07-21T18:30:24.724 INFO:teuthology.orchestra.run.smithi039.stdout:(28/131): ceph-mgr-cephadm-18.0.0-4746.g81cf785 1.5 MB/s | 129 kB 00:00 2023-07-21T18:30:24.966 INFO:teuthology.orchestra.run.smithi039.stdout:(29/131): rbd-mirror-18.0.0-4746.g81cf785a.el8. 3.8 MB/s | 3.4 MB 00:00 2023-07-21T18:30:24.987 INFO:teuthology.orchestra.run.smithi155.stdout:(131/131): librbd1-18.0.0-4746.g81cf785a.el8.x8 2.6 MB/s | 3.6 MB 00:01 2023-07-21T18:30:24.990 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2023-07-21T18:30:24.990 INFO:teuthology.orchestra.run.smithi155.stdout:Total 11 MB/s | 205 MB 00:18 2023-07-21T18:30:25.008 INFO:teuthology.orchestra.run.smithi039.stdout:(30/131): ceph-mgr-dashboard-18.0.0-4746.g81cf7 6.0 MB/s | 1.7 MB 00:00 2023-07-21T18:30:25.083 INFO:teuthology.orchestra.run.smithi039.stdout:(31/131): ceph-mgr-modules-core-18.0.0-4746.g81 3.1 MB/s | 240 kB 00:00 2023-07-21T18:30:25.150 INFO:teuthology.orchestra.run.smithi039.stdout:(32/131): ceph-mgr-rook-18.0.0-4746.g81cf785a.e 712 kB/s | 47 kB 00:00 2023-07-21T18:30:25.217 INFO:teuthology.orchestra.run.smithi039.stdout:(33/131): ceph-prometheus-alerts-18.0.0-4746.g8 218 kB/s | 15 kB 00:00 2023-07-21T18:30:25.274 INFO:teuthology.orchestra.run.smithi193.stdout:(30/131): ceph-mgr-dashboard-18.0.0-4746.g81cf7 2.7 MB/s | 1.7 MB 00:00 2023-07-21T18:30:25.301 INFO:teuthology.orchestra.run.smithi039.stdout:(34/131): ceph-volume-18.0.0-4746.g81cf785a.el8 3.0 MB/s | 258 kB 00:00 2023-07-21T18:30:25.376 INFO:teuthology.orchestra.run.smithi039.stdout:(35/131): cephadm-18.0.0-4746.g81cf785a.el8.noa 2.2 MB/s | 166 kB 00:00 2023-07-21T18:30:25.391 INFO:teuthology.orchestra.run.smithi193.stdout:(31/131): ceph-mgr-modules-core-18.0.0-4746.g81 2.0 MB/s | 240 kB 00:00 2023-07-21T18:30:25.435 INFO:teuthology.orchestra.run.smithi039.stdout:(36/131): cephfs-top-18.0.0-4746.g81cf785a.el8. 308 kB/s | 18 kB 00:00 2023-07-21T18:30:25.458 INFO:teuthology.orchestra.run.smithi193.stdout:(32/131): ceph-mgr-rook-18.0.0-4746.g81cf785a.e 713 kB/s | 47 kB 00:00 2023-07-21T18:30:25.516 INFO:teuthology.orchestra.run.smithi193.stdout:(33/131): ceph-prometheus-alerts-18.0.0-4746.g8 250 kB/s | 15 kB 00:00 2023-07-21T18:30:25.633 INFO:teuthology.orchestra.run.smithi193.stdout:(34/131): ceph-volume-18.0.0-4746.g81cf785a.el8 2.2 MB/s | 258 kB 00:00 2023-07-21T18:30:25.693 INFO:teuthology.orchestra.run.smithi039.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 1.2 MB/s | 306 kB 00:00 2023-07-21T18:30:25.717 INFO:teuthology.orchestra.run.smithi193.stdout:(35/131): cephadm-18.0.0-4746.g81cf785a.el8.noa 2.0 MB/s | 166 kB 00:00 2023-07-21T18:30:25.761 INFO:teuthology.orchestra.run.smithi039.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 883 kB/s | 59 kB 00:00 2023-07-21T18:30:25.801 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2023-07-21T18:30:25.829 INFO:teuthology.orchestra.run.smithi193.stdout:(36/131): cephfs-top-18.0.0-4746.g81cf785a.el8. 161 kB/s | 18 kB 00:00 2023-07-21T18:30:25.836 INFO:teuthology.orchestra.run.smithi039.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 1.0 MB/s | 75 kB 00:00 2023-07-21T18:30:25.921 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2023-07-21T18:30:25.921 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2023-07-21T18:30:25.969 INFO:teuthology.orchestra.run.smithi039.stdout:(40/131): python3-asyncssh-2.7.0-2.el8.noarch.r 3.5 MB/s | 476 kB 00:00 2023-07-21T18:30:26.028 INFO:teuthology.orchestra.run.smithi039.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 751 kB/s | 44 kB 00:00 2023-07-21T18:30:26.130 INFO:teuthology.orchestra.run.smithi039.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.8 MB/s | 185 kB 00:00 2023-07-21T18:30:26.195 INFO:teuthology.orchestra.run.smithi039.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 512 kB/s | 33 kB 00:00 2023-07-21T18:30:26.254 INFO:teuthology.orchestra.run.smithi039.stdout:(44/131): python3-certifi-2018.10.15-7.el8.noar 279 kB/s | 16 kB 00:00 2023-07-21T18:30:26.337 INFO:teuthology.orchestra.run.smithi039.stdout:(45/131): python3-cheroot-8.5.2-1.el8.noarch.rp 2.0 MB/s | 173 kB 00:00 2023-07-21T18:30:26.421 INFO:teuthology.orchestra.run.smithi039.stdout:(46/131): ceph-mgr-diskprediction-local-18.0.0- 5.1 MB/s | 7.4 MB 00:01 2023-07-21T18:30:26.442 INFO:teuthology.orchestra.run.smithi193.stdout:(37/131): ceph-test-18.0.0-4746.g81cf785a.el8.x 6.2 MB/s | 48 MB 00:07 2023-07-21T18:30:26.446 INFO:teuthology.orchestra.run.smithi039.stdout:(47/131): python3-cherrypy-18.4.0-1.el8.noarch. 3.5 MB/s | 384 kB 00:00 2023-07-21T18:30:26.513 INFO:teuthology.orchestra.run.smithi039.stdout:(48/131): python3-jaraco-6.2-6.el8.noarch.rpm 166 kB/s | 11 kB 00:00 2023-07-21T18:30:26.580 INFO:teuthology.orchestra.run.smithi039.stdout:(49/131): python3-jaraco-functools-2.0-4.el8.no 263 kB/s | 18 kB 00:00 2023-07-21T18:30:26.656 INFO:teuthology.orchestra.run.smithi039.stdout:(50/131): python3-google-auth-1.1.1-10.el8.noar 347 kB/s | 81 kB 00:00 2023-07-21T18:30:26.775 INFO:teuthology.orchestra.run.smithi039.stdout:(51/131): python3-kubernetes-11.0.0-6.el8.noarc 6.0 MB/s | 1.2 MB 00:00 2023-07-21T18:30:26.806 INFO:teuthology.orchestra.run.smithi039.stdout:(52/131): python3-logutils-0.3.5-11.el8.noarch. 325 kB/s | 49 kB 00:00 2023-07-21T18:30:26.848 INFO:teuthology.orchestra.run.smithi039.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 800 kB/s | 59 kB 00:00 2023-07-21T18:30:26.898 INFO:teuthology.orchestra.run.smithi039.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 656 kB/s | 60 kB 00:00 2023-07-21T18:30:26.924 INFO:teuthology.orchestra.run.smithi039.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 3.7 MB/s | 283 kB 00:00 2023-07-21T18:30:26.942 INFO:teuthology.orchestra.run.smithi193.stdout:(38/131): gperftools-libs-2.7-9.el8.x86_64.rpm 275 kB/s | 306 kB 00:01 2023-07-21T18:30:26.966 INFO:teuthology.orchestra.run.smithi039.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 233 kB/s | 16 kB 00:00 2023-07-21T18:30:26.968 INFO:teuthology.orchestra.run.smithi193.stdout:(39/131): liboath-2.6.2-3.el8.x86_64.rpm 112 kB/s | 59 kB 00:00 2023-07-21T18:30:26.991 INFO:teuthology.orchestra.run.smithi039.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 504 kB/s | 34 kB 00:00 2023-07-21T18:30:27.040 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2023-07-21T18:30:27.043 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2023-07-21T18:30:27.058 INFO:teuthology.orchestra.run.smithi039.stdout:(58/131): python3-rsa-4.8-1.el8.noarch.rpm 924 kB/s | 61 kB 00:00 2023-07-21T18:30:27.068 INFO:teuthology.orchestra.run.smithi193.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 599 kB/s | 75 kB 00:00 2023-07-21T18:30:27.100 INFO:teuthology.orchestra.run.smithi039.stdout:(59/131): python3-routes-2.4.1-12.el8.noarch.rp 1.4 MB/s | 196 kB 00:00 2023-07-21T18:30:27.125 INFO:teuthology.orchestra.run.smithi039.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 280 kB/s | 19 kB 00:00 2023-07-21T18:30:27.160 INFO:teuthology.orchestra.run.smithi193.stdout:(41/131): ceph-mgr-diskprediction-local-18.0.0- 3.0 MB/s | 7.4 MB 00:02 2023-07-21T18:30:27.167 INFO:teuthology.orchestra.run.smithi039.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 357 kB/s | 24 kB 00:00 2023-07-21T18:30:27.185 INFO:teuthology.orchestra.run.smithi193.stdout:(42/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 373 kB/s | 44 kB 00:00 2023-07-21T18:30:27.192 INFO:teuthology.orchestra.run.smithi039.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 436 kB/s | 29 kB 00:00 2023-07-21T18:30:27.234 INFO:teuthology.orchestra.run.smithi039.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 412 kB/s | 27 kB 00:00 2023-07-21T18:30:27.260 INFO:teuthology.orchestra.run.smithi039.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 3.7 MB/s | 255 kB 00:00 2023-07-21T18:30:27.319 INFO:teuthology.orchestra.run.smithi193.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 248 kB/s | 33 kB 00:00 2023-07-21T18:30:27.327 INFO:teuthology.orchestra.run.smithi039.stdout:(65/131): python3-websocket-client-0.56.0-5.el8 919 kB/s | 61 kB 00:00 2023-07-21T18:30:27.352 INFO:teuthology.orchestra.run.smithi039.stdout:(66/131): python3-webob-1.8.5-1.el8.1.noarch.rp 2.1 MB/s | 251 kB 00:00 2023-07-21T18:30:27.369 INFO:teuthology.orchestra.run.smithi193.stdout:(44/131): python3-asyncssh-2.7.0-2.el8.noarch.r 1.2 MB/s | 476 kB 00:00 2023-07-21T18:30:27.419 INFO:teuthology.orchestra.run.smithi039.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 931 kB/s | 85 kB 00:00 2023-07-21T18:30:27.444 INFO:teuthology.orchestra.run.smithi039.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 250 kB/s | 23 kB 00:00 2023-07-21T18:30:27.469 INFO:teuthology.orchestra.run.smithi039.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-07-21T18:30:27.570 INFO:teuthology.orchestra.run.smithi039.stdout:(70/131): thrift-0.13.0-2.el8.x86_64.rpm 12 MB/s | 1.7 MB 00:00 2023-07-21T18:30:27.595 INFO:teuthology.orchestra.run.smithi039.stdout:(71/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 374 kB/s | 47 kB 00:00 2023-07-21T18:30:27.622 INFO:teuthology.orchestra.run.smithi193.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 54 kB/s | 16 kB 00:00 2023-07-21T18:30:27.653 INFO:teuthology.orchestra.run.smithi193.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 611 kB/s | 173 kB 00:00 2023-07-21T18:30:27.653 INFO:teuthology.orchestra.run.smithi039.stdout:(72/131): python3-jwt-1.6.1-2.el8.noarch.rpm 856 kB/s | 43 kB 00:00 2023-07-21T18:30:27.679 INFO:teuthology.orchestra.run.smithi039.stdout:(73/131): libselinux-devel-2.9-5.el8.x86_64.rpm 1.8 MB/s | 200 kB 00:00 2023-07-21T18:30:27.696 INFO:teuthology.orchestra.run.smithi039.stdout:(74/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 5.2 MB/s | 220 kB 00:00 2023-07-21T18:30:27.721 INFO:teuthology.orchestra.run.smithi039.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 1.1 MB/s | 48 kB 00:00 2023-07-21T18:30:27.738 INFO:teuthology.orchestra.run.smithi039.stdout:(76/131): libsepol-devel-2.9-3.el8.x86_64.rpm 2.0 MB/s | 87 kB 00:00 2023-07-21T18:30:27.763 INFO:teuthology.orchestra.run.smithi039.stdout:(77/131): python3-requests-oauthlib-1.0.0-1.el8 1.0 MB/s | 43 kB 00:00 2023-07-21T18:30:27.778 INFO:teuthology.orchestra.run.smithi193.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 650 kB/s | 81 kB 00:00 2023-07-21T18:30:27.789 INFO:teuthology.orchestra.run.smithi039.stdout:(78/131): python3-pycparser-2.14-14.el8.noarch. 2.1 MB/s | 109 kB 00:00 2023-07-21T18:30:27.814 INFO:teuthology.orchestra.run.smithi039.stdout:(79/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.4 MB/s | 229 kB 00:00 2023-07-21T18:30:27.829 INFO:teuthology.orchestra.run.smithi193.stdout:(48/131): python3-beautifulsoup4-4.6.3-2.el8.1. 277 kB/s | 185 kB 00:00 2023-07-21T18:30:27.848 INFO:teuthology.orchestra.run.smithi039.stdout:(80/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 9.3 MB/s | 560 kB 00:00 2023-07-21T18:30:27.854 INFO:teuthology.orchestra.run.smithi193.stdout:(49/131): python3-cherrypy-18.4.0-1.el8.noarch. 1.6 MB/s | 384 kB 00:00 2023-07-21T18:30:27.864 INFO:teuthology.orchestra.run.smithi039.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 373 kB/s | 19 kB 00:00 2023-07-21T18:30:27.890 INFO:teuthology.orchestra.run.smithi039.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 3.6 MB/s | 155 kB 00:00 2023-07-21T18:30:27.913 INFO:teuthology.orchestra.run.smithi193.stdout:(50/131): python3-jaraco-6.2-6.el8.noarch.rpm 83 kB/s | 11 kB 00:00 2023-07-21T18:30:27.923 INFO:teuthology.orchestra.run.smithi039.stdout:(83/131): python3-cryptography-3.2.1-5.el8.x86_ 9.3 MB/s | 559 kB 00:00 2023-07-21T18:30:27.949 INFO:teuthology.orchestra.run.smithi039.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 306 kB/s | 18 kB 00:00 2023-07-21T18:30:27.954 INFO:teuthology.orchestra.run.smithi193.stdout:(51/131): python3-jaraco-functools-2.0-4.el8.no 140 kB/s | 18 kB 00:00 2023-07-21T18:30:27.983 INFO:teuthology.orchestra.run.smithi039.stdout:(85/131): pcre2-devel-10.32-2.el8.x86_64.rpm 10 MB/s | 605 kB 00:00 2023-07-21T18:30:28.038 INFO:teuthology.orchestra.run.smithi193.stdout:(52/131): python3-logutils-0.3.5-11.el8.noarch. 389 kB/s | 49 kB 00:00 2023-07-21T18:30:28.046 INFO:teuthology.orchestra.run.smithi039.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 2.4 MB/s | 238 kB 00:00 2023-07-21T18:30:28.091 INFO:teuthology.orchestra.run.smithi039.stdout:(87/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.6 MB/s | 72 kB 00:00 2023-07-21T18:30:28.097 INFO:teuthology.orchestra.run.smithi193.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 414 kB/s | 59 kB 00:00 2023-07-21T18:30:28.125 INFO:teuthology.orchestra.run.smithi039.stdout:(88/131): fuse-2.9.7-15.el8.x86_64.rpm 2.4 MB/s | 83 kB 00:00 2023-07-21T18:30:28.167 INFO:teuthology.orchestra.run.smithi039.stdout:(89/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 13 MB/s | 2.3 MB 00:00 2023-07-21T18:30:28.184 INFO:teuthology.orchestra.run.smithi193.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 414 kB/s | 60 kB 00:00 2023-07-21T18:30:28.192 INFO:teuthology.orchestra.run.smithi039.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 897 kB/s | 60 kB 00:00 2023-07-21T18:30:28.217 INFO:teuthology.orchestra.run.smithi039.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.9 MB/s | 98 kB 00:00 2023-07-21T18:30:28.222 INFO:teuthology.orchestra.run.smithi193.stdout:(55/131): python3-kubernetes-11.0.0-6.el8.noarc 3.2 MB/s | 1.2 MB 00:00 2023-07-21T18:30:28.243 INFO:teuthology.orchestra.run.smithi039.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2023-07-21T18:30:28.268 INFO:teuthology.orchestra.run.smithi039.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 768 kB/s | 39 kB 00:00 2023-07-21T18:30:28.302 INFO:teuthology.orchestra.run.smithi039.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 11 MB/s | 644 kB 00:00 2023-07-21T18:30:28.306 INFO:teuthology.orchestra.run.smithi193.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 128 kB/s | 16 kB 00:00 2023-07-21T18:30:28.327 INFO:teuthology.orchestra.run.smithi039.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.9 MB/s | 171 kB 00:00 2023-07-21T18:30:28.331 INFO:teuthology.orchestra.run.smithi193.stdout:(57/131): python3-pecan-1.3.2-9.el8.noarch.rpm 1.2 MB/s | 283 kB 00:00 2023-07-21T18:30:28.352 INFO:teuthology.orchestra.run.smithi039.stdout:(96/131): NetworkManager-initscripts-updown-1.3 2.7 MB/s | 138 kB 00:00 2023-07-21T18:30:28.356 INFO:teuthology.orchestra.run.smithi193.stdout:(58/131): python3-repoze-lru-0.7-6.el8.noarch.r 251 kB/s | 34 kB 00:00 2023-07-21T18:30:28.394 INFO:teuthology.orchestra.run.smithi039.stdout:(97/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.6 MB/s | 110 kB 00:00 2023-07-21T18:30:28.428 INFO:teuthology.orchestra.run.smithi039.stdout:(98/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.6 MB/s | 54 kB 00:00 2023-07-21T18:30:28.482 INFO:teuthology.orchestra.run.smithi193.stdout:(59/131): python3-simplegeneric-0.8.1-17.el8.no 150 kB/s | 19 kB 00:00 2023-07-21T18:30:28.553 INFO:teuthology.orchestra.run.smithi039.stdout:(99/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.0 MB/s | 259 kB 00:00 2023-07-21T18:30:28.574 INFO:teuthology.orchestra.run.smithi193.stdout:(60/131): python3-routes-2.4.1-12.el8.noarch.rp 735 kB/s | 196 kB 00:00 2023-07-21T18:30:28.599 INFO:teuthology.orchestra.run.smithi193.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 205 kB/s | 24 kB 00:00 2023-07-21T18:30:28.624 INFO:teuthology.orchestra.run.smithi193.stdout:(62/131): python3-rsa-4.8-1.el8.noarch.rpm 210 kB/s | 61 kB 00:00 2023-07-21T18:30:28.779 INFO:teuthology.orchestra.run.smithi193.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 153 kB/s | 27 kB 00:00 2023-07-21T18:30:28.781 INFO:teuthology.orchestra.run.smithi039.stdout:(100/131): socat-1.7.4.1-1.el8.x86_64.rpm 1.6 MB/s | 323 kB 00:00 2023-07-21T18:30:28.808 INFO:teuthology.orchestra.run.smithi193.stdout:(64/131): python3-tempora-1.14.1-5.el8.noarch.r 125 kB/s | 29 kB 00:00 2023-07-21T18:30:28.829 INFO:teuthology.orchestra.run.smithi039.stdout:(101/131): python3-markupsafe-0.23-19.el8.x86_6 829 kB/s | 39 kB 00:00 2023-07-21T18:30:28.833 INFO:teuthology.orchestra.run.smithi193.stdout:(65/131): python3-waitress-1.4.3-1.el8.noarch.r 1.2 MB/s | 255 kB 00:00 2023-07-21T18:30:28.879 INFO:teuthology.orchestra.run.smithi039.stdout:(102/131): bison-3.0.4-10.el8.x86_64.rpm 14 MB/s | 688 kB 00:00 2023-07-21T18:30:28.908 INFO:teuthology.orchestra.run.smithi193.stdout:(66/131): python3-webob-1.8.5-1.el8.1.noarch.rp 1.9 MB/s | 251 kB 00:00 2023-07-21T18:30:28.929 INFO:teuthology.orchestra.run.smithi039.stdout:(103/131): python3-jinja2-2.10.1-3.el8.noarch.r 11 MB/s | 538 kB 00:00 2023-07-21T18:30:28.934 INFO:teuthology.orchestra.run.smithi193.stdout:(67/131): python3-websocket-client-0.56.0-5.el8 488 kB/s | 61 kB 00:00 2023-07-21T18:30:28.959 INFO:teuthology.orchestra.run.smithi193.stdout:(68/131): python3-webtest-2.0.33-1.el8.noarch.r 680 kB/s | 85 kB 00:00 2023-07-21T18:30:28.971 INFO:teuthology.orchestra.run.smithi039.stdout:(104/131): python3-mako-1.0.6-13.el8.noarch.rpm 3.7 MB/s | 157 kB 00:00 2023-07-21T18:30:28.984 INFO:teuthology.orchestra.run.smithi193.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-07-21T18:30:29.013 INFO:teuthology.orchestra.run.smithi039.stdout:(105/131): oniguruma-6.8.2-2.el8.x86_64.rpm 4.4 MB/s | 187 kB 00:00 2023-07-21T18:30:29.026 INFO:teuthology.orchestra.run.smithi193.stdout:(70/131): python3-zc-lockfile-2.0-2.el8.noarch. 195 kB/s | 23 kB 00:00 2023-07-21T18:30:29.051 INFO:teuthology.orchestra.run.smithi193.stdout:(71/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 700 kB/s | 47 kB 00:00 2023-07-21T18:30:29.072 INFO:teuthology.orchestra.run.smithi039.stdout:(106/131): automake-1.16.1-7.el8.noarch.rpm 12 MB/s | 713 kB 00:00 2023-07-21T18:30:29.093 INFO:teuthology.orchestra.run.smithi193.stdout:(72/131): python3-jwt-1.6.1-2.el8.noarch.rpm 1.1 MB/s | 43 kB 00:00 2023-07-21T18:30:29.119 INFO:teuthology.orchestra.run.smithi193.stdout:(73/131): libselinux-devel-2.9-5.el8.x86_64.rpm 2.1 MB/s | 200 kB 00:00 2023-07-21T18:30:29.144 INFO:teuthology.orchestra.run.smithi193.stdout:(74/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-07-21T18:30:29.161 INFO:teuthology.orchestra.run.smithi193.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 1.1 MB/s | 48 kB 00:00 2023-07-21T18:30:29.186 INFO:teuthology.orchestra.run.smithi193.stdout:(76/131): libsepol-devel-2.9-3.el8.x86_64.rpm 2.0 MB/s | 87 kB 00:00 2023-07-21T18:30:29.203 INFO:teuthology.orchestra.run.smithi193.stdout:(77/131): python3-requests-oauthlib-1.0.0-1.el8 1.0 MB/s | 43 kB 00:00 2023-07-21T18:30:29.220 INFO:teuthology.orchestra.run.smithi193.stdout:(78/131): python3-pycparser-2.14-14.el8.noarch. 3.2 MB/s | 109 kB 00:00 2023-07-21T18:30:29.239 INFO:teuthology.orchestra.run.smithi039.stdout:(107/131): python3-scipy-1.0.0-21.module+el8.5. 15 MB/s | 14 MB 00:00 2023-07-21T18:30:29.254 INFO:teuthology.orchestra.run.smithi193.stdout:(79/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.4 MB/s | 229 kB 00:00 2023-07-21T18:30:29.273 INFO:teuthology.orchestra.run.smithi039.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 802 kB/s | 27 kB 00:00 2023-07-21T18:30:29.296 INFO:teuthology.orchestra.run.smithi193.stdout:(80/131): thrift-0.13.0-2.el8.x86_64.rpm 4.8 MB/s | 1.7 MB 00:00 2023-07-21T18:30:29.315 INFO:teuthology.orchestra.run.smithi039.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-07-21T18:30:29.346 INFO:teuthology.orchestra.run.smithi193.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 204 kB/s | 19 kB 00:00 2023-07-21T18:30:29.348 INFO:teuthology.orchestra.run.smithi039.stdout:(110/131): python3-prettytable-0.7.2-14.el8.noa 1.3 MB/s | 44 kB 00:00 2023-07-21T18:30:29.388 INFO:teuthology.orchestra.run.smithi193.stdout:(82/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 3.3 MB/s | 560 kB 00:00 2023-07-21T18:30:29.422 INFO:teuthology.orchestra.run.smithi193.stdout:(83/131): python3-oauthlib-2.1.0-1.el8.noarch.r 1.2 MB/s | 155 kB 00:00 2023-07-21T18:30:29.447 INFO:teuthology.orchestra.run.smithi193.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 306 kB/s | 18 kB 00:00 2023-07-21T18:30:29.448 INFO:teuthology.orchestra.run.smithi039.stdout:(111/131): python3-babel-2.5.1-7.el8.noarch.rpm 13 MB/s | 4.8 MB 00:00 2023-07-21T18:30:29.481 INFO:teuthology.orchestra.run.smithi193.stdout:(85/131): python3-cryptography-3.2.1-5.el8.x86_ 4.2 MB/s | 559 kB 00:00 2023-07-21T18:30:29.490 INFO:teuthology.orchestra.run.smithi039.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 11 MB/s | 457 kB 00:00 2023-07-21T18:30:29.514 INFO:teuthology.orchestra.run.smithi193.stdout:(86/131): pcre2-devel-10.32-2.el8.x86_64.rpm 6.4 MB/s | 605 kB 00:00 2023-07-21T18:30:29.532 INFO:teuthology.orchestra.run.smithi039.stdout:(113/131): python3-numpy-f2py-1.14.3-10.el8.x86 5.3 MB/s | 225 kB 00:00 2023-07-21T18:30:29.539 INFO:teuthology.orchestra.run.smithi193.stdout:(87/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 2.5 MB/s | 238 kB 00:00 2023-07-21T18:30:29.556 INFO:teuthology.orchestra.run.smithi193.stdout:(88/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.7 MB/s | 72 kB 00:00 2023-07-21T18:30:29.573 INFO:teuthology.orchestra.run.smithi193.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 2.4 MB/s | 83 kB 00:00 2023-07-21T18:30:29.591 INFO:teuthology.orchestra.run.smithi039.stdout:(114/131): python3-numpy-1.14.3-10.el8.x86_64.r 15 MB/s | 3.7 MB 00:00 2023-07-21T18:30:29.598 INFO:teuthology.orchestra.run.smithi193.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 1.4 MB/s | 60 kB 00:00 2023-07-21T18:30:29.608 INFO:teuthology.orchestra.run.smithi039.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 853 kB/s | 64 kB 00:00 2023-07-21T18:30:29.624 INFO:teuthology.orchestra.run.smithi193.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.9 MB/s | 98 kB 00:00 2023-07-21T18:30:29.633 INFO:teuthology.orchestra.run.smithi039.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.9 MB/s | 126 kB 00:00 2023-07-21T18:30:29.649 INFO:teuthology.orchestra.run.smithi193.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2023-07-21T18:30:29.650 INFO:teuthology.orchestra.run.smithi039.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 477 kB/s | 20 kB 00:00 2023-07-21T18:30:29.674 INFO:teuthology.orchestra.run.smithi193.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 771 kB/s | 39 kB 00:00 2023-07-21T18:30:29.675 INFO:teuthology.orchestra.run.smithi039.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.4 MB/s | 103 kB 00:00 2023-07-21T18:30:29.708 INFO:teuthology.orchestra.run.smithi193.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 11 MB/s | 644 kB 00:00 2023-07-21T18:30:29.709 INFO:teuthology.orchestra.run.smithi039.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 12 MB/s | 709 kB 00:00 2023-07-21T18:30:29.733 INFO:teuthology.orchestra.run.smithi193.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.9 MB/s | 171 kB 00:00 2023-07-21T18:30:29.750 INFO:teuthology.orchestra.run.smithi193.stdout:(96/131): NetworkManager-initscripts-updown-1.3 3.2 MB/s | 138 kB 00:00 2023-07-21T18:30:29.751 INFO:teuthology.orchestra.run.smithi039.stdout:(120/131): libpq-13.5-1.el8.x86_64.rpm 4.6 MB/s | 198 kB 00:00 2023-07-21T18:30:29.809 INFO:teuthology.orchestra.run.smithi193.stdout:(97/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 7.1 MB/s | 2.3 MB 00:00 2023-07-21T18:30:29.834 INFO:teuthology.orchestra.run.smithi193.stdout:(98/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.3 MB/s | 110 kB 00:00 2023-07-21T18:30:29.859 INFO:teuthology.orchestra.run.smithi193.stdout:(99/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.1 MB/s | 54 kB 00:00 2023-07-21T18:30:29.860 INFO:teuthology.orchestra.run.smithi039.stdout:(121/131): openblas-threads-0.3.15-3.el8.x86_64 26 MB/s | 4.9 MB 00:00 2023-07-21T18:30:29.885 INFO:teuthology.orchestra.run.smithi193.stdout:(100/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 5.0 MB/s | 259 kB 00:00 2023-07-21T18:30:29.901 INFO:teuthology.orchestra.run.smithi039.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-07-21T18:30:29.943 INFO:teuthology.orchestra.run.smithi193.stdout:(101/131): socat-1.7.4.1-1.el8.x86_64.rpm 3.8 MB/s | 323 kB 00:00 2023-07-21T18:30:29.968 INFO:teuthology.orchestra.run.smithi193.stdout:(102/131): python3-markupsafe-0.23-19.el8.x86_6 465 kB/s | 39 kB 00:00 2023-07-21T18:30:29.969 INFO:teuthology.orchestra.run.smithi039.stdout:(123/131): openblas-0.3.15-3.el8.x86_64.rpm 22 MB/s | 4.7 MB 00:00 2023-07-21T18:30:30.002 INFO:teuthology.orchestra.run.smithi193.stdout:(103/131): bison-3.0.4-10.el8.x86_64.rpm 12 MB/s | 688 kB 00:00 2023-07-21T18:30:30.035 INFO:teuthology.orchestra.run.smithi039.stdout:(124/131): librdkafka-0.11.4-3.el8.x86_64.rpm 2.6 MB/s | 354 kB 00:00 2023-07-21T18:30:30.036 INFO:teuthology.orchestra.run.smithi193.stdout:(104/131): python3-jinja2-2.10.1-3.el8.noarch.r 7.9 MB/s | 538 kB 00:00 2023-07-21T18:30:30.061 INFO:teuthology.orchestra.run.smithi193.stdout:(105/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-07-21T18:30:30.086 INFO:teuthology.orchestra.run.smithi039.stdout:(125/131): postgresql-contrib-10.21-2.module+el 16 MB/s | 808 kB 00:00 2023-07-21T18:30:30.086 INFO:teuthology.orchestra.run.smithi193.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.6 MB/s | 187 kB 00:00 2023-07-21T18:30:30.120 INFO:teuthology.orchestra.run.smithi193.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 12 MB/s | 713 kB 00:00 2023-07-21T18:30:30.153 INFO:teuthology.orchestra.run.smithi039.stdout:(126/131): postgresql-10.21-2.module+el8.6.0+15 23 MB/s | 1.5 MB 00:00 2023-07-21T18:30:30.153 INFO:teuthology.orchestra.run.smithi193.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 795 kB/s | 27 kB 00:00 2023-07-21T18:30:30.196 INFO:teuthology.orchestra.run.smithi193.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-07-21T18:30:30.220 INFO:teuthology.orchestra.run.smithi039.stdout:(127/131): postgresql-server-10.21-2.module+el8 20 MB/s | 5.1 MB 00:00 2023-07-21T18:30:30.229 INFO:teuthology.orchestra.run.smithi193.stdout:(110/131): python3-prettytable-0.7.2-14.el8.noa 1.3 MB/s | 44 kB 00:00 2023-07-21T18:30:30.321 INFO:teuthology.orchestra.run.smithi193.stdout:(111/131): python3-babel-2.5.1-7.el8.noarch.rpm 20 MB/s | 4.8 MB 00:00 2023-07-21T18:30:30.363 INFO:teuthology.orchestra.run.smithi193.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 11 MB/s | 457 kB 00:00 2023-07-21T18:30:30.405 INFO:teuthology.orchestra.run.smithi193.stdout:(113/131): python3-numpy-f2py-1.14.3-10.el8.x86 5.3 MB/s | 225 kB 00:00 2023-07-21T18:30:30.464 INFO:teuthology.orchestra.run.smithi193.stdout:(114/131): python3-numpy-1.14.3-10.el8.x86_64.r 16 MB/s | 3.7 MB 00:00 2023-07-21T18:30:30.514 INFO:teuthology.orchestra.run.smithi193.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 590 kB/s | 64 kB 00:00 2023-07-21T18:30:30.539 INFO:teuthology.orchestra.run.smithi193.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 1.6 MB/s | 126 kB 00:00 2023-07-21T18:30:30.564 INFO:teuthology.orchestra.run.smithi193.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 398 kB/s | 20 kB 00:00 2023-07-21T18:30:30.590 INFO:teuthology.orchestra.run.smithi193.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.0 MB/s | 103 kB 00:00 2023-07-21T18:30:30.632 INFO:teuthology.orchestra.run.smithi193.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 10 MB/s | 709 kB 00:00 2023-07-21T18:30:30.790 INFO:teuthology.orchestra.run.smithi193.stdout:(120/131): python3-scipy-1.0.0-21.module+el8.5. 13 MB/s | 14 MB 00:01 2023-07-21T18:30:30.811 INFO:teuthology.orchestra.run.smithi039.stdout:(128/131): librados2-18.0.0-4746.g81cf785a.el8. 5.6 MB/s | 3.7 MB 00:00 2023-07-21T18:30:30.815 INFO:teuthology.orchestra.run.smithi193.stdout:(121/131): libpq-13.5-1.el8.x86_64.rpm 1.1 MB/s | 198 kB 00:00 2023-07-21T18:30:30.859 INFO:teuthology.orchestra.run.smithi193.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.7 MB/s | 202 kB 00:00 2023-07-21T18:30:30.861 INFO:teuthology.orchestra.run.smithi039.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2023-07-21T18:30:30.899 INFO:teuthology.orchestra.run.smithi193.stdout:(123/131): librdkafka-0.11.4-3.el8.x86_64.rpm 8.3 MB/s | 354 kB 00:00 2023-07-21T18:30:30.983 INFO:teuthology.orchestra.run.smithi193.stdout:(124/131): openblas-threads-0.3.15-3.el8.x86_64 12 MB/s | 4.9 MB 00:00 2023-07-21T18:30:31.082 INFO:teuthology.orchestra.run.smithi193.stdout:(125/131): openblas-0.3.15-3.el8.x86_64.rpm 16 MB/s | 4.7 MB 00:00 2023-07-21T18:30:31.125 INFO:teuthology.orchestra.run.smithi193.stdout:(126/131): postgresql-contrib-10.21-2.module+el 5.6 MB/s | 808 kB 00:00 2023-07-21T18:30:31.184 INFO:teuthology.orchestra.run.smithi193.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 15 MB/s | 1.5 MB 00:00 2023-07-21T18:30:31.195 INFO:teuthology.orchestra.run.smithi039.stdout:(130/131): librbd1-18.0.0-4746.g81cf785a.el8.x8 3.6 MB/s | 3.6 MB 00:00 2023-07-21T18:30:31.267 INFO:teuthology.orchestra.run.smithi193.stdout:(128/131): postgresql-server-10.21-2.module+el8 14 MB/s | 5.1 MB 00:00 2023-07-21T18:30:31.318 INFO:teuthology.orchestra.run.smithi193.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2023-07-21T18:30:32.204 INFO:teuthology.orchestra.run.smithi039.stdout:(131/131): ceph-test-18.0.0-4746.g81cf785a.el8. 4.8 MB/s | 48 MB 00:09 2023-07-21T18:30:32.206 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2023-07-21T18:30:32.207 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2023-07-21T18:30:32.207 INFO:teuthology.orchestra.run.smithi039.stdout:Total 13 MB/s | 205 MB 00:15 2023-07-21T18:30:32.251 INFO:teuthology.orchestra.run.smithi193.stdout:(130/131): librbd1-18.0.0-4746.g81cf785a.el8.x8 3.3 MB/s | 3.6 MB 00:01 2023-07-21T18:30:32.377 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-07-21T18:30:32.575 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-07-21T18:30:32.874 INFO:teuthology.orchestra.run.smithi193.stdout:(131/131): librados2-18.0.0-4746.g81cf785a.el8. 2.1 MB/s | 3.7 MB 00:01 2023-07-21T18:30:32.879 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2023-07-21T18:30:32.879 INFO:teuthology.orchestra.run.smithi193.stdout:Total 11 MB/s | 205 MB 00:18 2023-07-21T18:30:33.047 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction check 2023-07-21T18:30:33.088 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-07-21T18:30:33.179 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction check succeeded. 2023-07-21T18:30:33.179 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction test 2023-07-21T18:30:33.271 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-07-21T18:30:33.382 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-07-21T18:30:33.663 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-07-21T18:30:33.716 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2023-07-21T18:30:33.835 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2023-07-21T18:30:33.836 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2023-07-21T18:30:34.071 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-07-21T18:30:34.254 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : librados2-2:18.0.0-4746.g81cf785a.el8.x86_64 6/134 2023-07-21T18:30:34.450 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction test succeeded. 2023-07-21T18:30:34.454 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction 2023-07-21T18:30:34.631 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librados2-2:18.0.0-4746.g81cf785a.el8.x86_64 6/134 2023-07-21T18:30:34.879 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : librbd1-2:18.0.0-4746.g81cf785a.el8.x86_64 7/134 2023-07-21T18:30:34.955 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2023-07-21T18:30:34.959 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2023-07-21T18:30:35.004 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librbd1-2:18.0.0-4746.g81cf785a.el8.x86_64 7/134 2023-07-21T18:30:35.204 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libcephfs2-2:18.0.0-4746.g81cf785a.el8.x86_64 8/134 2023-07-21T18:30:35.317 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libcephfs2-2:18.0.0-4746.g81cf785a.el8.x86_64 8/134 2023-07-21T18:30:35.530 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-rados-2:18.0.0-4746.g81cf785a.el8.x86_64 9/134 2023-07-21T18:30:35.799 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-07-21T18:30:35.998 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-07-21T18:30:36.422 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-07-21T18:30:36.674 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-07-21T18:30:36.820 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libcephsqlite-2:18.0.0-4746.g81cf785a.el8.x86_64 14/134 2023-07-21T18:30:36.905 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4746.g81cf785a.el8.x86_64 14/134 2023-07-21T18:30:37.128 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libradosstriper1-2:18.0.0-4746.g81cf785a.el8.x86 15/134 2023-07-21T18:30:37.203 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4746.g81cf785a.el8.x86 15/134 2023-07-21T18:30:37.345 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-07-21T18:30:37.489 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-07-21T18:30:37.687 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-07-21T18:30:38.306 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-07-21T18:30:38.787 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-07-21T18:30:39.370 INFO:teuthology.orchestra.run.smithi039.stdout: Preparing : 1/1 2023-07-21T18:30:39.388 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-07-21T18:30:39.553 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-07-21T18:30:39.578 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-07-21T18:30:39.738 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-07-21T18:30:39.873 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2023-07-21T18:30:40.005 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-07-21T18:30:40.030 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-07-21T18:30:40.117 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-07-21T18:30:40.222 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-07-21T18:30:40.288 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-07-21T18:30:40.322 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-07-21T18:30:40.407 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-07-21T18:30:40.522 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-07-21T18:30:40.576 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-07-21T18:30:40.724 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-07-21T18:30:40.772 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-07-21T18:30:40.846 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-07-21T18:30:40.971 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : librados2-2:18.0.0-4746.g81cf785a.el8.x86_64 5/134 2023-07-21T18:30:41.082 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-07-21T18:30:41.121 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-07-21T18:30:41.307 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : librados2-2:18.0.0-4746.g81cf785a.el8.x86_64 5/134 2023-07-21T18:30:41.325 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-07-21T18:30:41.416 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: librados2-2:18.0.0-4746.g81cf785a.el8.x86_64 5/134 2023-07-21T18:30:41.550 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-07-21T18:30:41.580 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : librbd1-2:18.0.0-4746.g81cf785a.el8.x86_64 6/134 2023-07-21T18:30:41.699 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librados2-2:18.0.0-4746.g81cf785a.el8.x86_64 5/134 2023-07-21T18:30:41.718 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: librbd1-2:18.0.0-4746.g81cf785a.el8.x86_64 6/134 2023-07-21T18:30:41.812 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-07-21T18:30:41.857 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : librbd1-2:18.0.0-4746.g81cf785a.el8.x86_64 6/134 2023-07-21T18:30:41.913 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libcephfs2-2:18.0.0-4746.g81cf785a.el8.x86_64 7/134 2023-07-21T18:30:41.967 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librbd1-2:18.0.0-4746.g81cf785a.el8.x86_64 6/134 2023-07-21T18:30:41.976 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-ceph-common-2:18.0.0-4746.g81cf785a.el8. 26/134 2023-07-21T18:30:42.009 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: libcephfs2-2:18.0.0-4746.g81cf785a.el8.x86_64 7/134 2023-07-21T18:30:42.147 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-ceph-argparse-2:18.0.0-4746.g81cf785a.el 27/134 2023-07-21T18:30:42.157 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libcephfs2-2:18.0.0-4746.g81cf785a.el8.x86_64 7/134 2023-07-21T18:30:42.176 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-rados-2:18.0.0-4746.g81cf785a.el8.x86_64 8/134 2023-07-21T18:30:42.255 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: libcephfs2-2:18.0.0-4746.g81cf785a.el8.x86_64 7/134 2023-07-21T18:30:42.364 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-07-21T18:30:42.420 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-rados-2:18.0.0-4746.g81cf785a.el8.x86_64 8/134 2023-07-21T18:30:42.571 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-07-21T18:30:42.588 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-07-21T18:30:42.792 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-07-21T18:30:42.818 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-07-21T18:30:43.018 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-07-21T18:30:43.069 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cephfs-2:18.0.0-4746.g81cf785a.el8.x86_6 28/134 2023-07-21T18:30:43.219 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-07-21T18:30:43.392 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-07-21T18:30:43.434 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-07-21T18:30:43.453 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-07-21T18:30:43.563 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libcephsqlite-2:18.0.0-4746.g81cf785a.el8.x86_64 14/134 2023-07-21T18:30:43.674 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4746.g81cf785a.el8.x86_64 14/134 2023-07-21T18:30:43.722 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-07-21T18:30:43.863 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libradosstriper1-2:18.0.0-4746.g81cf785a.el8.x86 15/134 2023-07-21T18:30:43.935 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-07-21T18:30:44.017 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-07-21T18:30:44.019 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4746.g81cf785a.el8.x86 15/134 2023-07-21T18:30:44.057 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libcephsqlite-2:18.0.0-4746.g81cf785a.el8.x86_64 14/134 2023-07-21T18:30:44.160 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4746.g81cf785a.el8.x86_64 14/134 2023-07-21T18:30:44.236 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-07-21T18:30:44.259 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-07-21T18:30:44.356 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libradosstriper1-2:18.0.0-4746.g81cf785a.el8.x86 15/134 2023-07-21T18:30:44.445 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-07-21T18:30:44.503 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4746.g81cf785a.el8.x86 15/134 2023-07-21T18:30:44.626 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-07-21T18:30:44.702 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-07-21T18:30:44.754 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-07-21T18:30:44.881 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-07-21T18:30:44.898 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-07-21T18:30:45.079 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-07-21T18:30:45.190 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-07-21T18:30:45.296 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-07-21T18:30:45.326 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-07-21T18:30:45.506 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-07-21T18:30:45.954 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-07-21T18:30:45.994 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-07-21T18:30:46.112 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-07-21T18:30:46.118 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-07-21T18:30:46.290 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-07-21T18:30:46.354 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-07-21T18:30:46.565 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-07-21T18:30:46.754 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-07-21T18:30:46.767 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-07-21T18:30:46.884 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-07-21T18:30:46.950 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-rbd-2:18.0.0-4746.g81cf785a.el8.x86_64 36/134 2023-07-21T18:30:46.954 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-07-21T18:30:47.039 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-07-21T18:30:47.147 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : librados-devel-2:18.0.0-4746.g81cf785a.el8.x86_6 37/134 2023-07-21T18:30:47.318 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-07-21T18:30:47.511 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-07-21T18:30:47.696 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-07-21T18:30:47.699 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-07-21T18:30:47.728 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-07-21T18:30:47.830 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-07-21T18:30:47.919 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-07-21T18:30:47.996 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-07-21T18:30:48.084 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-07-21T18:30:48.140 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-07-21T18:30:48.170 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-07-21T18:30:48.276 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-07-21T18:30:48.359 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-07-21T18:30:48.415 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-07-21T18:30:48.536 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-ceph-common-2:18.0.0-4746.g81cf785a.el8. 26/134 2023-07-21T18:30:48.554 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-ceph-common-2:18.0.0-4746.g81cf785a.el8. 26/134 2023-07-21T18:30:48.658 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-ceph-argparse-2:18.0.0-4746.g81cf785a.el 27/134 2023-07-21T18:30:48.683 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-ceph-argparse-2:18.0.0-4746.g81cf785a.el 27/134 2023-07-21T18:30:48.697 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-07-21T18:30:48.901 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cephfs-2:18.0.0-4746.g81cf785a.el8.x86_6 28/134 2023-07-21T18:30:48.932 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cephfs-2:18.0.0-4746.g81cf785a.el8.x86_6 28/134 2023-07-21T18:30:48.989 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-07-21T18:30:49.106 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-07-21T18:30:49.112 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-07-21T18:30:49.195 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-07-21T18:30:49.439 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-07-21T18:30:49.715 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-07-21T18:30:50.061 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-07-21T18:30:50.310 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-07-21T18:30:50.513 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-07-21T18:30:50.765 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-07-21T18:30:50.858 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-07-21T18:30:50.930 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-07-21T18:30:50.978 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-07-21T18:30:51.097 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-07-21T18:30:51.428 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-07-21T18:30:51.736 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-07-21T18:30:51.859 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-07-21T18:30:51.990 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-07-21T18:30:52.067 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-07-21T18:30:52.170 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-07-21T18:30:52.250 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-07-21T18:30:52.393 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-07-21T18:30:52.425 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-07-21T18:30:52.457 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-07-21T18:30:52.611 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-07-21T18:30:52.685 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-07-21T18:30:52.702 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-07-21T18:30:52.793 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-07-21T18:30:52.851 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-07-21T18:30:52.883 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-07-21T18:30:52.908 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-07-21T18:30:53.085 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-07-21T18:30:53.117 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-07-21T18:30:53.126 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-07-21T18:30:53.236 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-rbd-2:18.0.0-4746.g81cf785a.el8.x86_64 36/134 2023-07-21T18:30:53.330 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-07-21T18:30:53.345 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-07-21T18:30:53.386 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : librados-devel-2:18.0.0-4746.g81cf785a.el8.x86_6 37/134 2023-07-21T18:30:53.510 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-07-21T18:30:53.518 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-rbd-2:18.0.0-4746.g81cf785a.el8.x86_64 36/134 2023-07-21T18:30:53.537 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-07-21T18:30:53.588 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-07-21T18:30:53.676 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : librados-devel-2:18.0.0-4746.g81cf785a.el8.x86_6 37/134 2023-07-21T18:30:53.705 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-07-21T18:30:53.815 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-07-21T18:30:53.860 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-07-21T18:30:53.863 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-07-21T18:30:53.918 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-07-21T18:30:54.023 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-07-21T18:30:54.035 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-07-21T18:30:54.142 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-07-21T18:30:54.192 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-07-21T18:30:54.203 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-07-21T18:30:54.271 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-07-21T18:30:54.391 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-07-21T18:30:54.404 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-07-21T18:30:54.507 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-07-21T18:30:54.538 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-07-21T18:30:54.613 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-07-21T18:30:54.644 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-07-21T18:30:54.725 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-07-21T18:30:54.834 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-07-21T18:30:54.891 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-07-21T18:30:54.894 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-07-21T18:30:54.959 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-07-21T18:30:54.984 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-07-21T18:30:55.047 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-07-21T18:30:55.136 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-07-21T18:30:55.186 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-07-21T18:30:55.203 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-07-21T18:30:55.339 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-07-21T18:30:55.357 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-07-21T18:30:55.402 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-07-21T18:30:55.508 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-07-21T18:30:55.519 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-07-21T18:30:55.620 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-07-21T18:30:55.641 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-07-21T18:30:55.681 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-07-21T18:30:55.826 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-07-21T18:30:55.846 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-07-21T18:30:55.872 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-07-21T18:30:56.033 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-07-21T18:30:56.037 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-07-21T18:30:56.050 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-07-21T18:30:56.216 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-07-21T18:30:56.246 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-07-21T18:30:56.386 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-07-21T18:30:56.471 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-07-21T18:30:56.475 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-07-21T18:30:56.681 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-07-21T18:30:56.756 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-07-21T18:30:56.849 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-07-21T18:30:56.891 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-07-21T18:30:56.932 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-07-21T18:30:57.126 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-07-21T18:30:57.189 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-07-21T18:30:57.215 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-07-21T18:30:57.359 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-07-21T18:30:57.382 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-07-21T18:30:57.425 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-07-21T18:30:57.587 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-07-21T18:30:57.634 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-07-21T18:30:57.669 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-07-21T18:30:57.852 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-07-21T18:30:57.862 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-07-21T18:30:57.868 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-07-21T18:30:58.020 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-07-21T18:30:58.171 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-07-21T18:30:58.259 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-07-21T18:30:58.402 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-07-21T18:30:58.416 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-07-21T18:30:58.558 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-07-21T18:30:58.579 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-07-21T18:30:58.800 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-07-21T18:30:58.848 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-07-21T18:30:58.991 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-07-21T18:30:58.997 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-07-21T18:30:59.092 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-07-21T18:30:59.174 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-07-21T18:30:59.280 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-07-21T18:30:59.326 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-07-21T18:30:59.488 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-07-21T18:30:59.572 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-07-21T18:30:59.592 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-07-21T18:30:59.712 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-07-21T18:30:59.756 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-07-21T18:30:59.865 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-07-21T18:30:59.907 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-07-21T18:30:59.978 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-07-21T18:31:00.124 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-07-21T18:31:00.203 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-07-21T18:31:00.204 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-07-21T18:31:00.319 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-07-21T18:31:00.394 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-07-21T18:31:00.430 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-07-21T18:31:00.450 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-07-21T18:31:00.564 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-07-21T18:31:00.631 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-07-21T18:31:00.815 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-07-21T18:31:01.026 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-07-21T18:31:01.028 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-07-21T18:31:01.225 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-07-21T18:31:01.226 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-07-21T18:31:01.276 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-07-21T18:31:01.382 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-07-21T18:31:01.408 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : librgw2-2:18.0.0-4746.g81cf785a.el8.x86_64 92/134 2023-07-21T18:31:01.423 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-07-21T18:31:01.464 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librgw2-2:18.0.0-4746.g81cf785a.el8.x86_64 92/134 2023-07-21T18:31:01.519 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-07-21T18:31:01.597 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-07-21T18:31:01.659 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-rgw-2:18.0.0-4746.g81cf785a.el8.x86_64 93/134 2023-07-21T18:31:01.782 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-07-21T18:31:01.790 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-07-21T18:31:01.993 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-07-21T18:31:02.008 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-07-21T18:31:02.185 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-07-21T18:31:02.199 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-07-21T18:31:02.364 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-07-21T18:31:02.451 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-07-21T18:31:02.509 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-07-21T18:31:02.653 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-07-21T18:31:02.750 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-07-21T18:31:02.795 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-07-21T18:31:02.999 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-07-21T18:31:03.037 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-07-21T18:31:03.192 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-07-21T18:31:03.431 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-07-21T18:31:03.694 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-07-21T18:31:03.862 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-07-21T18:31:04.072 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-07-21T18:31:04.145 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-07-21T18:31:04.473 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-07-21T18:31:04.736 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-07-21T18:31:05.020 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-07-21T18:31:05.220 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-07-21T18:31:05.287 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-07-21T18:31:05.427 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 94/134 2023-07-21T18:31:05.472 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-07-21T18:31:05.541 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-07-21T18:31:05.692 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 94/134 2023-07-21T18:31:05.794 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-07-21T18:31:06.085 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-07-21T18:31:06.227 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-07-21T18:31:06.272 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 94/134 2023-07-21T18:31:06.339 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-07-21T18:31:06.420 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : librgw2-2:18.0.0-4746.g81cf785a.el8.x86_64 92/134 2023-07-21T18:31:06.478 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librgw2-2:18.0.0-4746.g81cf785a.el8.x86_64 92/134 2023-07-21T18:31:06.578 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-07-21T18:31:06.602 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-base-2:18.0.0-4746.g81cf785a.el8.x86_64 95/134 2023-07-21T18:31:06.628 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-rgw-2:18.0.0-4746.g81cf785a.el8.x86_64 93/134 2023-07-21T18:31:06.690 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-base-2:18.0.0-4746.g81cf785a.el8.x86_64 95/134 2023-07-21T18:31:06.824 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-selinux-2:18.0.0-4746.g81cf785a.el8.x86_64 96/134 2023-07-21T18:31:07.427 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-07-21T18:31:07.584 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : librgw2-2:18.0.0-4746.g81cf785a.el8.x86_64 92/134 2023-07-21T18:31:07.663 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: librgw2-2:18.0.0-4746.g81cf785a.el8.x86_64 92/134 2023-07-21T18:31:07.843 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-rgw-2:18.0.0-4746.g81cf785a.el8.x86_64 93/134 2023-07-21T18:31:10.203 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 94/134 2023-07-21T18:31:10.453 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 94/134 2023-07-21T18:31:11.028 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 94/134 2023-07-21T18:31:11.281 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-base-2:18.0.0-4746.g81cf785a.el8.x86_64 95/134 2023-07-21T18:31:11.361 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-base-2:18.0.0-4746.g81cf785a.el8.x86_64 95/134 2023-07-21T18:31:11.519 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-selinux-2:18.0.0-4746.g81cf785a.el8.x86_64 96/134 2023-07-21T18:31:11.656 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 94/134 2023-07-21T18:31:12.084 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 94/134 2023-07-21T18:31:12.711 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 94/134 2023-07-21T18:31:12.961 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-base-2:18.0.0-4746.g81cf785a.el8.x86_64 95/134 2023-07-21T18:31:13.048 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-base-2:18.0.0-4746.g81cf785a.el8.x86_64 95/134 2023-07-21T18:31:13.217 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-selinux-2:18.0.0-4746.g81cf785a.el8.x86_64 96/134 2023-07-21T18:31:26.335 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4746.g81cf785a.el8.x86_64 96/134 2023-07-21T18:31:26.335 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /sys 2023-07-21T18:31:26.335 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /proc 2023-07-21T18:31:26.335 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /dev 2023-07-21T18:31:26.336 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /run 2023-07-21T18:31:26.336 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /mnt 2023-07-21T18:31:26.336 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /var/tmp 2023-07-21T18:31:26.336 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /home 2023-07-21T18:31:26.336 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /tmp 2023-07-21T18:31:26.336 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /dev 2023-07-21T18:31:26.336 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-07-21T18:31:26.672 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-volume-2:18.0.0-4746.g81cf785a.el8.noarch 97/134 2023-07-21T18:31:28.250 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-volume-2:18.0.0-4746.g81cf785a.el8.noarch 97/134 2023-07-21T18:31:28.432 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-osd-2:18.0.0-4746.g81cf785a.el8.x86_64 98/134 2023-07-21T18:31:28.652 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-osd-2:18.0.0-4746.g81cf785a.el8.x86_64 98/134 2023-07-21T18:31:28.838 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mds-2:18.0.0-4746.g81cf785a.el8.x86_64 99/134 2023-07-21T18:31:29.295 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mds-2:18.0.0-4746.g81cf785a.el8.x86_64 99/134 2023-07-21T18:31:29.497 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mon-2:18.0.0-4746.g81cf785a.el8.x86_64 100/134 2023-07-21T18:31:29.513 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mon-2:18.0.0-4746.g81cf785a.el8.x86_64 100/134 2023-07-21T18:31:30.269 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4746.g81cf785a.el8.x86_64 96/134 2023-07-21T18:31:30.270 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /sys 2023-07-21T18:31:30.270 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /proc 2023-07-21T18:31:30.270 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /dev 2023-07-21T18:31:30.270 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /run 2023-07-21T18:31:30.271 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /mnt 2023-07-21T18:31:30.271 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /var/tmp 2023-07-21T18:31:30.271 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /home 2023-07-21T18:31:30.271 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /tmp 2023-07-21T18:31:30.271 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /dev 2023-07-21T18:31:30.271 INFO:teuthology.orchestra.run.smithi193.stdout: 2023-07-21T18:31:30.650 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-volume-2:18.0.0-4746.g81cf785a.el8.noarch 97/134 2023-07-21T18:31:31.213 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 101/134 2023-07-21T18:31:31.418 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 101/134 2023-07-21T18:31:31.527 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4746.g81cf785a.e 102/134 2023-07-21T18:31:31.771 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4746.g81cf785a. 103/134 2023-07-21T18:31:31.946 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4746.g81cf785a.el8.noa 104/134 2023-07-21T18:31:32.163 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4746.g81cf785a.el8.noa 104/134 2023-07-21T18:31:32.225 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-volume-2:18.0.0-4746.g81cf785a.el8.noarch 97/134 2023-07-21T18:31:32.403 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-osd-2:18.0.0-4746.g81cf785a.el8.x86_64 98/134 2023-07-21T18:31:32.490 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4746.g81cf785a.el8.n 105/134 2023-07-21T18:31:32.622 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-osd-2:18.0.0-4746.g81cf785a.el8.x86_64 98/134 2023-07-21T18:31:32.791 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mds-2:18.0.0-4746.g81cf785a.el8.x86_64 99/134 2023-07-21T18:31:33.246 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mds-2:18.0.0-4746.g81cf785a.el8.x86_64 99/134 2023-07-21T18:31:33.360 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4746.g81cf785a.el8.x86_64 96/134 2023-07-21T18:31:33.361 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /sys 2023-07-21T18:31:33.361 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /proc 2023-07-21T18:31:33.361 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /dev 2023-07-21T18:31:33.361 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /run 2023-07-21T18:31:33.361 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /mnt 2023-07-21T18:31:33.361 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /var/tmp 2023-07-21T18:31:33.362 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /home 2023-07-21T18:31:33.362 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /tmp 2023-07-21T18:31:33.362 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /dev 2023-07-21T18:31:33.362 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:31:33.374 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mon-2:18.0.0-4746.g81cf785a.el8.x86_64 100/134 2023-07-21T18:31:33.391 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mon-2:18.0.0-4746.g81cf785a.el8.x86_64 100/134 2023-07-21T18:31:33.455 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4746.g81cf785a.el8.n 105/134 2023-07-21T18:31:33.629 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4746.g81c 106/134 2023-07-21T18:31:33.720 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4746.g81c 106/134 2023-07-21T18:31:33.731 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-volume-2:18.0.0-4746.g81cf785a.el8.noarch 97/134 2023-07-21T18:31:34.118 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4746.g81cf785a.el 107/134 2023-07-21T18:31:34.339 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-2:18.0.0-4746.g81cf785a.el8.x86_64 108/134 2023-07-21T18:31:34.375 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4746.g81cf785a.el8.x86_64 108/134 2023-07-21T18:31:34.546 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-rook-2:18.0.0-4746.g81cf785a.el8.noarch 109/134 2023-07-21T18:31:34.557 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4746.g81cf785a.el8.noarch 109/134 2023-07-21T18:31:34.991 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 101/134 2023-07-21T18:31:35.172 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 101/134 2023-07-21T18:31:35.313 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4746.g81cf785a.e 102/134 2023-07-21T18:31:35.409 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-volume-2:18.0.0-4746.g81cf785a.el8.noarch 97/134 2023-07-21T18:31:35.521 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4746.g81cf785a. 103/134 2023-07-21T18:31:35.607 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-osd-2:18.0.0-4746.g81cf785a.el8.x86_64 98/134 2023-07-21T18:31:35.707 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4746.g81cf785a.el8.noa 104/134 2023-07-21T18:31:35.839 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-osd-2:18.0.0-4746.g81cf785a.el8.x86_64 98/134 2023-07-21T18:31:35.869 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-2:18.0.0-4746.g81cf785a.el8.x86_64 110/134 2023-07-21T18:31:35.922 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4746.g81cf785a.el8.noa 104/134 2023-07-21T18:31:36.023 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mds-2:18.0.0-4746.g81cf785a.el8.x86_64 99/134 2023-07-21T18:31:36.062 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-radosgw-2:18.0.0-4746.g81cf785a.el8.x86_64 111/134 2023-07-21T18:31:36.098 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4746.g81cf785a.el8.x86_64 111/134 2023-07-21T18:31:36.210 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4746.g81cf785a.el8.n 105/134 2023-07-21T18:31:36.304 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4746.g81cf7 112/134 2023-07-21T18:31:36.345 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4746.g81cf7 112/134 2023-07-21T18:31:36.504 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : cephfs-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 113/134 2023-07-21T18:31:36.513 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mds-2:18.0.0-4746.g81cf785a.el8.x86_64 99/134 2023-07-21T18:31:36.648 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mon-2:18.0.0-4746.g81cf785a.el8.x86_64 100/134 2023-07-21T18:31:36.665 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mon-2:18.0.0-4746.g81cf785a.el8.x86_64 100/134 2023-07-21T18:31:36.850 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 113/134 2023-07-21T18:31:37.020 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : rbd-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 114/134 2023-07-21T18:31:37.168 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4746.g81cf785a.el8.n 105/134 2023-07-21T18:31:37.315 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4746.g81c 106/134 2023-07-21T18:31:37.404 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4746.g81c 106/134 2023-07-21T18:31:37.771 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4746.g81cf785a.el 107/134 2023-07-21T18:31:37.971 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-2:18.0.0-4746.g81cf785a.el8.x86_64 108/134 2023-07-21T18:31:38.006 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4746.g81cf785a.el8.x86_64 108/134 2023-07-21T18:31:38.182 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-rook-2:18.0.0-4746.g81cf785a.el8.noarch 109/134 2023-07-21T18:31:38.193 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4746.g81cf785a.el8.noarch 109/134 2023-07-21T18:31:38.478 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 101/134 2023-07-21T18:31:38.701 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 101/134 2023-07-21T18:31:38.837 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4746.g81cf785a.e 102/134 2023-07-21T18:31:39.014 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4746.g81cf785a. 103/134 2023-07-21T18:31:39.197 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4746.g81cf785a.el8.noa 104/134 2023-07-21T18:31:39.418 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4746.g81cf785a.el8.noa 104/134 2023-07-21T18:31:39.477 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-2:18.0.0-4746.g81cf785a.el8.x86_64 110/134 2023-07-21T18:31:39.649 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-radosgw-2:18.0.0-4746.g81cf785a.el8.x86_64 111/134 2023-07-21T18:31:39.691 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4746.g81cf785a.el8.x86_64 111/134 2023-07-21T18:31:39.715 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4746.g81cf785a.el8.n 105/134 2023-07-21T18:31:39.857 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4746.g81cf7 112/134 2023-07-21T18:31:39.898 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4746.g81cf7 112/134 2023-07-21T18:31:40.031 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : cephfs-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 113/134 2023-07-21T18:31:40.377 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 113/134 2023-07-21T18:31:40.523 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : rbd-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 114/134 2023-07-21T18:31:40.737 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4746.g81cf785a.el8.n 105/134 2023-07-21T18:31:40.913 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4746.g81c 106/134 2023-07-21T18:31:41.009 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4746.g81c 106/134 2023-07-21T18:31:41.419 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4746.g81cf785a.el 107/134 2023-07-21T18:31:41.561 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 114/134 2023-07-21T18:31:41.614 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mgr-2:18.0.0-4746.g81cf785a.el8.x86_64 108/134 2023-07-21T18:31:41.651 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4746.g81cf785a.el8.x86_64 108/134 2023-07-21T18:31:41.797 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mgr-rook-2:18.0.0-4746.g81cf785a.el8.noarch 109/134 2023-07-21T18:31:41.808 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4746.g81cf785a.el8.noarch 109/134 2023-07-21T18:31:41.895 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-test-2:18.0.0-4746.g81cf785a.el8.x86_64 115/134 2023-07-21T18:31:42.078 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-07-21T18:31:42.269 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-07-21T18:31:43.133 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-07-21T18:31:43.229 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-2:18.0.0-4746.g81cf785a.el8.x86_64 110/134 2023-07-21T18:31:43.387 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-radosgw-2:18.0.0-4746.g81cf785a.el8.x86_64 111/134 2023-07-21T18:31:43.426 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4746.g81cf785a.el8.x86_64 111/134 2023-07-21T18:31:43.509 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-07-21T18:31:43.555 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4746.g81cf7 112/134 2023-07-21T18:31:43.597 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4746.g81cf7 112/134 2023-07-21T18:31:43.689 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 119/134 2023-07-21T18:31:43.763 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : cephfs-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 113/134 2023-07-21T18:31:43.882 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-07-21T18:31:44.056 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libcephfs-devel-2:18.0.0-4746.g81cf785a.el8.x86_ 121/134 2023-07-21T18:31:44.129 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 113/134 2023-07-21T18:31:44.229 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-21T18:31:44.288 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : rbd-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 114/134 2023-07-21T18:31:45.051 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 114/134 2023-07-21T18:31:45.903 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-test-2:18.0.0-4746.g81cf785a.el8.x86_64 115/134 2023-07-21T18:31:46.237 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-07-21T18:31:46.391 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 117/134 2023-07-21T18:31:46.402 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-21T18:31:46.621 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-07-21T18:31:46.669 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-21T18:31:46.686 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-21T18:31:46.780 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-07-21T18:31:46.852 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : cephfs-top-2:18.0.0-4746.g81cf785a.el8.noarch 124/134 2023-07-21T18:31:46.969 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-07-21T18:31:47.193 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-07-21T18:31:47.193 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : rbd-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 125/134 2023-07-21T18:31:47.409 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libcephfs-devel-2:18.0.0-4746.g81cf785a.el8.x86_ 121/134 2023-07-21T18:31:47.507 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : rbd-nbd-2:18.0.0-4746.g81cf785a.el8.x86_64 126/134 2023-07-21T18:31:47.565 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-21T18:31:47.871 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-07-21T18:31:48.027 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-21T18:31:48.035 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-21T18:31:48.215 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-07-21T18:31:48.427 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-07-21T18:31:48.516 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-07-21T18:31:48.735 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-07-21T18:31:48.748 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-07-21T18:31:48.748 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-21T18:31:48.972 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-21T18:31:48.972 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-21T18:31:49.119 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 114/134 2023-07-21T18:31:49.208 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-21T18:31:49.476 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-21T18:31:49.671 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-21T18:31:49.947 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-21T18:31:49.965 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-21T18:31:50.007 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-07-21T18:31:50.007 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-2:18.0.0-4746.g81cf785a.el8.x86_64 1/134 2023-07-21T18:31:50.007 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-base-2:18.0.0-4746.g81cf785a.el8.x86_64 2/134 2023-07-21T18:31:50.008 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 3/134 2023-07-21T18:31:50.008 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 4/134 2023-07-21T18:31:50.008 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4746.g81cf7 5/134 2023-07-21T18:31:50.008 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mds-2:18.0.0-4746.g81cf785a.el8.x86_64 6/134 2023-07-21T18:31:50.008 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-2:18.0.0-4746.g81cf785a.el8.x86_64 7/134 2023-07-21T18:31:50.009 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mon-2:18.0.0-4746.g81cf785a.el8.x86_64 8/134 2023-07-21T18:31:50.009 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-osd-2:18.0.0-4746.g81cf785a.el8.x86_64 9/134 2023-07-21T18:31:50.009 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-radosgw-2:18.0.0-4746.g81cf785a.el8.x86_64 10/134 2023-07-21T18:31:50.009 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-selinux-2:18.0.0-4746.g81cf785a.el8.x86_64 11/134 2023-07-21T18:31:50.009 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-test-2:18.0.0-4746.g81cf785a.el8.x86_64 12/134 2023-07-21T18:31:50.009 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : cephfs-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 13/134 2023-07-21T18:31:50.009 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libcephfs-devel-2:18.0.0-4746.g81cf785a.el8.x86_ 14/134 2023-07-21T18:31:50.009 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libcephfs2-2:18.0.0-4746.g81cf785a.el8.x86_64 15/134 2023-07-21T18:31:50.010 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libcephsqlite-2:18.0.0-4746.g81cf785a.el8.x86_64 16/134 2023-07-21T18:31:50.010 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librados-devel-2:18.0.0-4746.g81cf785a.el8.x86_6 17/134 2023-07-21T18:31:50.010 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libradosstriper1-2:18.0.0-4746.g81cf785a.el8.x86 18/134 2023-07-21T18:31:50.010 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librgw2-2:18.0.0-4746.g81cf785a.el8.x86_64 19/134 2023-07-21T18:31:50.010 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4746.g81cf785a.el 20/134 2023-07-21T18:31:50.010 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-ceph-common-2:18.0.0-4746.g81cf785a.el8. 21/134 2023-07-21T18:31:50.010 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cephfs-2:18.0.0-4746.g81cf785a.el8.x86_6 22/134 2023-07-21T18:31:50.011 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rados-2:18.0.0-4746.g81cf785a.el8.x86_64 23/134 2023-07-21T18:31:50.011 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rbd-2:18.0.0-4746.g81cf785a.el8.x86_64 24/134 2023-07-21T18:31:50.011 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rgw-2:18.0.0-4746.g81cf785a.el8.x86_64 25/134 2023-07-21T18:31:50.012 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : rbd-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 26/134 2023-07-21T18:31:50.012 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : rbd-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 27/134 2023-07-21T18:31:50.012 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : rbd-nbd-2:18.0.0-4746.g81cf785a.el8.x86_64 28/134 2023-07-21T18:31:50.013 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4746.g81cf785a. 29/134 2023-07-21T18:31:50.013 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4746.g81cf785a.el8.noa 30/134 2023-07-21T18:31:50.013 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4746.g81cf785a.el8.n 31/134 2023-07-21T18:31:50.013 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4746.g81c 32/134 2023-07-21T18:31:50.014 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4746.g81cf785a.el 33/134 2023-07-21T18:31:50.014 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4746.g81cf785a.el8.noarch 34/134 2023-07-21T18:31:50.014 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4746.g81cf785a.e 35/134 2023-07-21T18:31:50.014 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-volume-2:18.0.0-4746.g81cf785a.el8.noarch 36/134 2023-07-21T18:31:50.015 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 37/134 2023-07-21T18:31:50.015 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : cephfs-top-2:18.0.0-4746.g81cf785a.el8.noarch 38/134 2023-07-21T18:31:50.015 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-07-21T18:31:50.016 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-07-21T18:31:50.016 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-07-21T18:31:50.016 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-07-21T18:31:50.016 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-07-21T18:31:50.017 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-07-21T18:31:50.017 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-07-21T18:31:50.017 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-07-21T18:31:50.017 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-07-21T18:31:50.017 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-07-21T18:31:50.017 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-07-21T18:31:50.017 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-07-21T18:31:50.017 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-07-21T18:31:50.018 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-07-21T18:31:50.018 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-07-21T18:31:50.018 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-07-21T18:31:50.018 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-07-21T18:31:50.018 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-07-21T18:31:50.018 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-07-21T18:31:50.019 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-07-21T18:31:50.019 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-07-21T18:31:50.019 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-07-21T18:31:50.019 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-07-21T18:31:50.019 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-07-21T18:31:50.019 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-07-21T18:31:50.019 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-07-21T18:31:50.019 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-07-21T18:31:50.020 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-07-21T18:31:50.020 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-07-21T18:31:50.020 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-07-21T18:31:50.020 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-07-21T18:31:50.020 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-07-21T18:31:50.020 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-07-21T18:31:50.020 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-07-21T18:31:50.021 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-07-21T18:31:50.021 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-07-21T18:31:50.021 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-07-21T18:31:50.021 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-07-21T18:31:50.021 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-07-21T18:31:50.021 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-07-21T18:31:50.021 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-07-21T18:31:50.022 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-07-21T18:31:50.022 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-07-21T18:31:50.022 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-07-21T18:31:50.022 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-07-21T18:31:50.022 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-07-21T18:31:50.022 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-07-21T18:31:50.022 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-07-21T18:31:50.022 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-07-21T18:31:50.023 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-07-21T18:31:50.023 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-07-21T18:31:50.023 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-07-21T18:31:50.023 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-07-21T18:31:50.023 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-07-21T18:31:50.023 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-07-21T18:31:50.023 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-07-21T18:31:50.024 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-07-21T18:31:50.024 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-07-21T18:31:50.024 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-07-21T18:31:50.024 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-07-21T18:31:50.024 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-07-21T18:31:50.024 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-07-21T18:31:50.024 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-07-21T18:31:50.025 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-07-21T18:31:50.025 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-07-21T18:31:50.025 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-07-21T18:31:50.025 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-07-21T18:31:50.025 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-07-21T18:31:50.025 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-07-21T18:31:50.025 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-07-21T18:31:50.025 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-07-21T18:31:50.026 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-07-21T18:31:50.026 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-07-21T18:31:50.026 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-07-21T18:31:50.026 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-07-21T18:31:50.026 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-07-21T18:31:50.026 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-07-21T18:31:50.026 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-07-21T18:31:50.026 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-07-21T18:31:50.027 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-07-21T18:31:50.027 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-07-21T18:31:50.027 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-07-21T18:31:50.028 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-07-21T18:31:50.028 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-21T18:31:50.028 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-07-21T18:31:50.028 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-07-21T18:31:50.029 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-07-21T18:31:50.029 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-07-21T18:31:50.029 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-07-21T18:31:50.029 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-21T18:31:50.030 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librados2-2:18.0.0-4746.g81cf785a.el8.x86_64 129/134 2023-07-21T18:31:50.030 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-07-21T18:31:50.030 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librbd1-2:18.0.0-4746.g81cf785a.el8.x86_64 131/134 2023-07-21T18:31:50.031 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-21T18:31:50.031 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-07-21T18:31:50.032 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-test-2:18.0.0-4746.g81cf785a.el8.x86_64 115/134 2023-07-21T18:31:50.105 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : cephfs-top-2:18.0.0-4746.g81cf785a.el8.noarch 124/134 2023-07-21T18:31:50.357 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-07-21T18:31:50.421 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : rbd-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 125/134 2023-07-21T18:31:50.533 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 117/134 2023-07-21T18:31:50.731 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : rbd-nbd-2:18.0.0-4746.g81cf785a.el8.x86_64 126/134 2023-07-21T18:31:50.801 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-07-21T18:31:50.987 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-07-21T18:31:51.096 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-07-21T18:31:51.192 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-07-21T18:31:51.254 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-07-21T18:31:51.346 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-07-21T18:31:51.425 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-07-21T18:31:51.529 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-07-21T18:31:51.542 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-07-21T18:31:51.633 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libcephfs-devel-2:18.0.0-4746.g81cf785a.el8.x86_ 121/134 2023-07-21T18:31:51.704 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-07-21T18:31:51.712 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-07-21T18:31:51.796 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-21T18:31:51.840 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-07-21T18:31:51.840 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-21T18:31:52.057 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-21T18:31:52.057 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-21T18:31:52.277 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-21T18:31:52.495 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-21T18:31:52.539 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-21T18:31:52.539 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:58 ago on Fri 21 Jul 2023 06:29:54 PM UTC. 2023-07-21T18:31:53.003 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-07-21T18:31:53.004 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-2:18.0.0-4746.g81cf785a.el8.x86_64 1/134 2023-07-21T18:31:53.004 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-base-2:18.0.0-4746.g81cf785a.el8.x86_64 2/134 2023-07-21T18:31:53.004 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 3/134 2023-07-21T18:31:53.004 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 4/134 2023-07-21T18:31:53.005 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4746.g81cf7 5/134 2023-07-21T18:31:53.005 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mds-2:18.0.0-4746.g81cf785a.el8.x86_64 6/134 2023-07-21T18:31:53.005 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-2:18.0.0-4746.g81cf785a.el8.x86_64 7/134 2023-07-21T18:31:53.005 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mon-2:18.0.0-4746.g81cf785a.el8.x86_64 8/134 2023-07-21T18:31:53.005 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-osd-2:18.0.0-4746.g81cf785a.el8.x86_64 9/134 2023-07-21T18:31:53.005 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-radosgw-2:18.0.0-4746.g81cf785a.el8.x86_64 10/134 2023-07-21T18:31:53.005 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-selinux-2:18.0.0-4746.g81cf785a.el8.x86_64 11/134 2023-07-21T18:31:53.006 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-test-2:18.0.0-4746.g81cf785a.el8.x86_64 12/134 2023-07-21T18:31:53.006 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : cephfs-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 13/134 2023-07-21T18:31:53.006 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libcephfs-devel-2:18.0.0-4746.g81cf785a.el8.x86_ 14/134 2023-07-21T18:31:53.006 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libcephfs2-2:18.0.0-4746.g81cf785a.el8.x86_64 15/134 2023-07-21T18:31:53.006 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libcephsqlite-2:18.0.0-4746.g81cf785a.el8.x86_64 16/134 2023-07-21T18:31:53.006 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librados-devel-2:18.0.0-4746.g81cf785a.el8.x86_6 17/134 2023-07-21T18:31:53.007 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libradosstriper1-2:18.0.0-4746.g81cf785a.el8.x86 18/134 2023-07-21T18:31:53.007 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librgw2-2:18.0.0-4746.g81cf785a.el8.x86_64 19/134 2023-07-21T18:31:53.007 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4746.g81cf785a.el 20/134 2023-07-21T18:31:53.007 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-ceph-common-2:18.0.0-4746.g81cf785a.el8. 21/134 2023-07-21T18:31:53.007 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cephfs-2:18.0.0-4746.g81cf785a.el8.x86_6 22/134 2023-07-21T18:31:53.008 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-rados-2:18.0.0-4746.g81cf785a.el8.x86_64 23/134 2023-07-21T18:31:53.008 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-rbd-2:18.0.0-4746.g81cf785a.el8.x86_64 24/134 2023-07-21T18:31:53.008 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-rgw-2:18.0.0-4746.g81cf785a.el8.x86_64 25/134 2023-07-21T18:31:53.008 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : rbd-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 26/134 2023-07-21T18:31:53.008 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : rbd-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 27/134 2023-07-21T18:31:53.009 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : rbd-nbd-2:18.0.0-4746.g81cf785a.el8.x86_64 28/134 2023-07-21T18:31:53.009 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4746.g81cf785a. 29/134 2023-07-21T18:31:53.009 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4746.g81cf785a.el8.noa 30/134 2023-07-21T18:31:53.018 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4746.g81cf785a.el8.n 31/134 2023-07-21T18:31:53.019 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4746.g81c 32/134 2023-07-21T18:31:53.019 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4746.g81cf785a.el 33/134 2023-07-21T18:31:53.019 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4746.g81cf785a.el8.noarch 34/134 2023-07-21T18:31:53.019 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4746.g81cf785a.e 35/134 2023-07-21T18:31:53.019 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-volume-2:18.0.0-4746.g81cf785a.el8.noarch 36/134 2023-07-21T18:31:53.019 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 37/134 2023-07-21T18:31:53.020 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : cephfs-top-2:18.0.0-4746.g81cf785a.el8.noarch 38/134 2023-07-21T18:31:53.020 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-07-21T18:31:53.020 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-07-21T18:31:53.020 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-07-21T18:31:53.020 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-07-21T18:31:53.020 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-07-21T18:31:53.020 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-07-21T18:31:53.021 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-07-21T18:31:53.021 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-07-21T18:31:53.021 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-07-21T18:31:53.021 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-07-21T18:31:53.021 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-07-21T18:31:53.021 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-07-21T18:31:53.021 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-07-21T18:31:53.022 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-07-21T18:31:53.022 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-07-21T18:31:53.022 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-07-21T18:31:53.022 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-07-21T18:31:53.022 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-07-21T18:31:53.022 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-07-21T18:31:53.023 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-07-21T18:31:53.023 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-07-21T18:31:53.023 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-07-21T18:31:53.023 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-07-21T18:31:53.023 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-07-21T18:31:53.024 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-07-21T18:31:53.024 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-07-21T18:31:53.024 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-07-21T18:31:53.024 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-07-21T18:31:53.024 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-07-21T18:31:53.024 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-07-21T18:31:53.024 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-07-21T18:31:53.025 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-07-21T18:31:53.025 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-07-21T18:31:53.025 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-07-21T18:31:53.025 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-07-21T18:31:53.025 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-07-21T18:31:53.025 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-07-21T18:31:53.025 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-07-21T18:31:53.026 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-07-21T18:31:53.026 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-07-21T18:31:53.026 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-07-21T18:31:53.026 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-07-21T18:31:53.026 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-07-21T18:31:53.026 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-07-21T18:31:53.026 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-07-21T18:31:53.026 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-07-21T18:31:53.027 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-07-21T18:31:53.027 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-07-21T18:31:53.027 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-07-21T18:31:53.027 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-07-21T18:31:53.027 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-07-21T18:31:53.027 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-07-21T18:31:53.027 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-07-21T18:31:53.028 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-07-21T18:31:53.028 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-07-21T18:31:53.028 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-07-21T18:31:53.028 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-07-21T18:31:53.028 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-07-21T18:31:53.029 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-07-21T18:31:53.029 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-07-21T18:31:53.029 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-07-21T18:31:53.029 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-07-21T18:31:53.029 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-07-21T18:31:53.029 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-07-21T18:31:53.029 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-07-21T18:31:53.030 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-07-21T18:31:53.030 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-07-21T18:31:53.030 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-07-21T18:31:53.030 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-07-21T18:31:53.030 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-07-21T18:31:53.030 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-07-21T18:31:53.030 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-07-21T18:31:53.030 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-07-21T18:31:53.031 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-07-21T18:31:53.031 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-07-21T18:31:53.031 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-07-21T18:31:53.031 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-07-21T18:31:53.031 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-07-21T18:31:53.031 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-07-21T18:31:53.032 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-07-21T18:31:53.032 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-07-21T18:31:53.032 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-07-21T18:31:53.032 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-07-21T18:31:53.032 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-07-21T18:31:53.032 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-07-21T18:31:53.032 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-07-21T18:31:53.033 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-07-21T18:31:53.033 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-07-21T18:31:53.033 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-07-21T18:31:53.033 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-07-21T18:31:53.033 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librados2-2:18.0.0-4746.g81cf785a.el8.x86_64 129/134 2023-07-21T18:31:53.033 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-07-21T18:31:53.033 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librbd1-2:18.0.0-4746.g81cf785a.el8.x86_64 131/134 2023-07-21T18:31:53.033 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-21T18:31:53.034 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-07-21T18:31:54.020 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-21T18:31:54.270 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-21T18:31:54.289 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-21T18:31:54.454 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : cephfs-top-2:18.0.0-4746.g81cf785a.el8.noarch 124/134 2023-07-21T18:31:54.771 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : rbd-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 125/134 2023-07-21T18:31:55.061 INFO:teuthology.orchestra.run.smithi155.stdout:Installed products updated. 2023-07-21T18:31:55.091 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : rbd-nbd-2:18.0.0-4746.g81cf785a.el8.x86_64 126/134 2023-07-21T18:31:55.273 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-07-21T18:31:55.273 INFO:teuthology.orchestra.run.smithi155.stdout:Upgraded: 2023-07-21T18:31:55.273 INFO:teuthology.orchestra.run.smithi155.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-07-21T18:31:55.274 INFO:teuthology.orchestra.run.smithi155.stdout: librados2-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:31:55.274 INFO:teuthology.orchestra.run.smithi155.stdout: librbd1-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:31:55.274 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2023-07-21T18:31:55.274 INFO:teuthology.orchestra.run.smithi155.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-07-21T18:31:55.274 INFO:teuthology.orchestra.run.smithi155.stdout: automake-1.16.1-7.el8.noarch 2023-07-21T18:31:55.274 INFO:teuthology.orchestra.run.smithi155.stdout: bison-3.0.4-10.el8.x86_64 2023-07-21T18:31:55.274 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:31:55.275 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-base-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:31:55.275 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:31:55.275 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:31:55.275 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-grafana-dashboards-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-21T18:31:55.275 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-immutable-object-cache-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:31:55.275 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mds-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:31:55.275 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:31:55.275 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-21T18:31:55.276 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-dashboard-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-21T18:31:55.276 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-21T18:31:55.276 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-modules-core-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-21T18:31:55.276 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-rook-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-21T18:31:55.276 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mon-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:31:55.276 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-osd-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:31:55.276 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-prometheus-alerts-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-21T18:31:55.277 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-radosgw-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:31:55.277 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-selinux-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:31:55.277 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-test-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:31:55.277 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-volume-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-21T18:31:55.277 INFO:teuthology.orchestra.run.smithi155.stdout: cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-21T18:31:55.277 INFO:teuthology.orchestra.run.smithi155.stdout: cephfs-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:31:55.277 INFO:teuthology.orchestra.run.smithi155.stdout: cephfs-top-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-21T18:31:55.277 INFO:teuthology.orchestra.run.smithi155.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-07-21T18:31:55.278 INFO:teuthology.orchestra.run.smithi155.stdout: flex-2.6.1-9.el8.x86_64 2023-07-21T18:31:55.278 INFO:teuthology.orchestra.run.smithi155.stdout: fuse-2.9.7-15.el8.x86_64 2023-07-21T18:31:55.278 INFO:teuthology.orchestra.run.smithi155.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-07-21T18:31:55.278 INFO:teuthology.orchestra.run.smithi155.stdout: jq-1.6-3.el8.x86_64 2023-07-21T18:31:55.278 INFO:teuthology.orchestra.run.smithi155.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-07-21T18:31:55.278 INFO:teuthology.orchestra.run.smithi155.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-07-21T18:31:55.278 INFO:teuthology.orchestra.run.smithi155.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-07-21T18:31:55.278 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs-devel-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:31:55.279 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs2-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:31:55.279 INFO:teuthology.orchestra.run.smithi155.stdout: libcephsqlite-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:31:55.279 INFO:teuthology.orchestra.run.smithi155.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-07-21T18:31:55.279 INFO:teuthology.orchestra.run.smithi155.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-07-21T18:31:55.279 INFO:teuthology.orchestra.run.smithi155.stdout: liboath-2.6.2-3.el8.x86_64 2023-07-21T18:31:55.279 INFO:teuthology.orchestra.run.smithi155.stdout: libpq-13.5-1.el8.x86_64 2023-07-21T18:31:55.279 INFO:teuthology.orchestra.run.smithi155.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-07-21T18:31:55.280 INFO:teuthology.orchestra.run.smithi155.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-07-21T18:31:55.280 INFO:teuthology.orchestra.run.smithi155.stdout: librados-devel-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:31:55.280 INFO:teuthology.orchestra.run.smithi155.stdout: libradosstriper1-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:31:55.280 INFO:teuthology.orchestra.run.smithi155.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-07-21T18:31:55.280 INFO:teuthology.orchestra.run.smithi155.stdout: librgw2-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:31:55.280 INFO:teuthology.orchestra.run.smithi155.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-07-21T18:31:55.280 INFO:teuthology.orchestra.run.smithi155.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-07-21T18:31:55.280 INFO:teuthology.orchestra.run.smithi155.stdout: libtool-2.4.6-25.el8.x86_64 2023-07-21T18:31:55.281 INFO:teuthology.orchestra.run.smithi155.stdout: libunwind-1.3.1-3.el8.x86_64 2023-07-21T18:31:55.281 INFO:teuthology.orchestra.run.smithi155.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-07-21T18:31:55.281 INFO:teuthology.orchestra.run.smithi155.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-07-21T18:31:55.281 INFO:teuthology.orchestra.run.smithi155.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-07-21T18:31:55.281 INFO:teuthology.orchestra.run.smithi155.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-07-21T18:31:55.281 INFO:teuthology.orchestra.run.smithi155.stdout: openblas-0.3.15-3.el8.x86_64 2023-07-21T18:31:55.281 INFO:teuthology.orchestra.run.smithi155.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-07-21T18:31:55.281 INFO:teuthology.orchestra.run.smithi155.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-07-21T18:31:55.282 INFO:teuthology.orchestra.run.smithi155.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-07-21T18:31:55.282 INFO:teuthology.orchestra.run.smithi155.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-07-21T18:31:55.282 INFO:teuthology.orchestra.run.smithi155.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-07-21T18:31:55.282 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-21T18:31:55.282 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-21T18:31:55.282 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-21T18:31:55.282 INFO:teuthology.orchestra.run.smithi155.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-07-21T18:31:55.283 INFO:teuthology.orchestra.run.smithi155.stdout: python3-babel-2.5.1-7.el8.noarch 2023-07-21T18:31:55.283 INFO:teuthology.orchestra.run.smithi155.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-07-21T18:31:55.283 INFO:teuthology.orchestra.run.smithi155.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-07-21T18:31:55.283 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-07-21T18:31:55.283 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-argparse-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:31:55.283 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:31:55.283 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cephfs-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:31:55.283 INFO:teuthology.orchestra.run.smithi155.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-07-21T18:31:55.284 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-07-21T18:31:55.284 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-07-21T18:31:55.284 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-07-21T18:31:55.284 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-07-21T18:31:55.284 INFO:teuthology.orchestra.run.smithi155.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-07-21T18:31:55.284 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-6.2-6.el8.noarch 2023-07-21T18:31:55.284 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-07-21T18:31:55.284 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-07-21T18:31:55.285 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-07-21T18:31:55.285 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-07-21T18:31:55.285 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-07-21T18:31:55.285 INFO:teuthology.orchestra.run.smithi155.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-07-21T18:31:55.285 INFO:teuthology.orchestra.run.smithi155.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-07-21T18:31:55.285 INFO:teuthology.orchestra.run.smithi155.stdout: python3-mako-1.0.6-13.el8.noarch 2023-07-21T18:31:55.285 INFO:teuthology.orchestra.run.smithi155.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-07-21T18:31:55.285 INFO:teuthology.orchestra.run.smithi155.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-07-21T18:31:55.286 INFO:teuthology.orchestra.run.smithi155.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-07-21T18:31:55.286 INFO:teuthology.orchestra.run.smithi155.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-07-21T18:31:55.286 INFO:teuthology.orchestra.run.smithi155.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-07-21T18:31:55.286 INFO:teuthology.orchestra.run.smithi155.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-07-21T18:31:55.286 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-07-21T18:31:55.286 INFO:teuthology.orchestra.run.smithi155.stdout: python3-portend-2.6-1.el8.noarch 2023-07-21T18:31:55.286 INFO:teuthology.orchestra.run.smithi155.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-07-21T18:31:55.287 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-07-21T18:31:55.287 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-07-21T18:31:55.287 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-07-21T18:31:55.287 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pycparser-2.14-14.el8.noarch 2023-07-21T18:31:55.287 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pytz-2017.2-9.el8.noarch 2023-07-21T18:31:55.287 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rados-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:31:55.287 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rbd-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:31:55.287 INFO:teuthology.orchestra.run.smithi155.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-07-21T18:31:55.288 INFO:teuthology.orchestra.run.smithi155.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-07-21T18:31:55.288 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rgw-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:31:55.288 INFO:teuthology.orchestra.run.smithi155.stdout: python3-routes-2.4.1-12.el8.noarch 2023-07-21T18:31:55.288 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rsa-4.8-1.el8.noarch 2023-07-21T18:31:55.288 INFO:teuthology.orchestra.run.smithi155.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-07-21T18:31:55.288 INFO:teuthology.orchestra.run.smithi155.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-07-21T18:31:55.288 INFO:teuthology.orchestra.run.smithi155.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-07-21T18:31:55.289 INFO:teuthology.orchestra.run.smithi155.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-07-21T18:31:55.289 INFO:teuthology.orchestra.run.smithi155.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-07-21T18:31:55.289 INFO:teuthology.orchestra.run.smithi155.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-07-21T18:31:55.289 INFO:teuthology.orchestra.run.smithi155.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-07-21T18:31:55.289 INFO:teuthology.orchestra.run.smithi155.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-07-21T18:31:55.289 INFO:teuthology.orchestra.run.smithi155.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-07-21T18:31:55.289 INFO:teuthology.orchestra.run.smithi155.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-07-21T18:31:55.289 INFO:teuthology.orchestra.run.smithi155.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-07-21T18:31:55.290 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:31:55.290 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:31:55.290 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-nbd-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:31:55.290 INFO:teuthology.orchestra.run.smithi155.stdout: socat-1.7.4.1-1.el8.x86_64 2023-07-21T18:31:55.290 INFO:teuthology.orchestra.run.smithi155.stdout: thrift-0.13.0-2.el8.x86_64 2023-07-21T18:31:55.290 INFO:teuthology.orchestra.run.smithi155.stdout: uuid-1.6.2-43.el8.x86_64 2023-07-21T18:31:55.290 INFO:teuthology.orchestra.run.smithi155.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-07-21T18:31:55.290 INFO:teuthology.orchestra.run.smithi155.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-07-21T18:31:55.291 INFO:teuthology.orchestra.run.smithi155.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-07-21T18:31:55.291 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-07-21T18:31:55.291 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2023-07-21T18:31:55.435 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-07-21T18:31:55.454 DEBUG:teuthology.parallel:result is None 2023-07-21T18:31:55.478 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-21T18:31:55.478 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:01:53 ago on Fri 21 Jul 2023 06:30:02 PM UTC. 2023-07-21T18:31:55.594 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-07-21T18:31:55.681 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-07-21T18:31:55.878 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-07-21T18:31:55.891 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-07-21T18:31:56.094 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-07-21T18:31:56.103 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-07-21T18:31:56.230 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-07-21T18:31:56.230 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-21T18:31:56.415 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-21T18:31:56.415 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-21T18:31:56.659 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-21T18:31:56.853 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-21T18:31:57.387 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-07-21T18:31:57.387 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-2:18.0.0-4746.g81cf785a.el8.x86_64 1/134 2023-07-21T18:31:57.387 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-base-2:18.0.0-4746.g81cf785a.el8.x86_64 2/134 2023-07-21T18:31:57.387 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 3/134 2023-07-21T18:31:57.387 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 4/134 2023-07-21T18:31:57.388 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4746.g81cf7 5/134 2023-07-21T18:31:57.388 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mds-2:18.0.0-4746.g81cf785a.el8.x86_64 6/134 2023-07-21T18:31:57.388 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mgr-2:18.0.0-4746.g81cf785a.el8.x86_64 7/134 2023-07-21T18:31:57.388 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mon-2:18.0.0-4746.g81cf785a.el8.x86_64 8/134 2023-07-21T18:31:57.388 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-osd-2:18.0.0-4746.g81cf785a.el8.x86_64 9/134 2023-07-21T18:31:57.388 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-radosgw-2:18.0.0-4746.g81cf785a.el8.x86_64 10/134 2023-07-21T18:31:57.388 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-selinux-2:18.0.0-4746.g81cf785a.el8.x86_64 11/134 2023-07-21T18:31:57.388 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-test-2:18.0.0-4746.g81cf785a.el8.x86_64 12/134 2023-07-21T18:31:57.389 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : cephfs-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 13/134 2023-07-21T18:31:57.389 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libcephfs-devel-2:18.0.0-4746.g81cf785a.el8.x86_ 14/134 2023-07-21T18:31:57.389 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libcephfs2-2:18.0.0-4746.g81cf785a.el8.x86_64 15/134 2023-07-21T18:31:57.389 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libcephsqlite-2:18.0.0-4746.g81cf785a.el8.x86_64 16/134 2023-07-21T18:31:57.389 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librados-devel-2:18.0.0-4746.g81cf785a.el8.x86_6 17/134 2023-07-21T18:31:57.389 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libradosstriper1-2:18.0.0-4746.g81cf785a.el8.x86 18/134 2023-07-21T18:31:57.389 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librgw2-2:18.0.0-4746.g81cf785a.el8.x86_64 19/134 2023-07-21T18:31:57.389 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4746.g81cf785a.el 20/134 2023-07-21T18:31:57.390 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-ceph-common-2:18.0.0-4746.g81cf785a.el8. 21/134 2023-07-21T18:31:57.390 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cephfs-2:18.0.0-4746.g81cf785a.el8.x86_6 22/134 2023-07-21T18:31:57.390 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-rados-2:18.0.0-4746.g81cf785a.el8.x86_64 23/134 2023-07-21T18:31:57.390 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-rbd-2:18.0.0-4746.g81cf785a.el8.x86_64 24/134 2023-07-21T18:31:57.390 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-rgw-2:18.0.0-4746.g81cf785a.el8.x86_64 25/134 2023-07-21T18:31:57.390 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : rbd-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 26/134 2023-07-21T18:31:57.390 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : rbd-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 27/134 2023-07-21T18:31:57.390 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : rbd-nbd-2:18.0.0-4746.g81cf785a.el8.x86_64 28/134 2023-07-21T18:31:57.391 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4746.g81cf785a. 29/134 2023-07-21T18:31:57.391 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4746.g81cf785a.el8.noa 30/134 2023-07-21T18:31:57.391 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4746.g81cf785a.el8.n 31/134 2023-07-21T18:31:57.391 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4746.g81c 32/134 2023-07-21T18:31:57.391 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4746.g81cf785a.el 33/134 2023-07-21T18:31:57.391 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4746.g81cf785a.el8.noarch 34/134 2023-07-21T18:31:57.391 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4746.g81cf785a.e 35/134 2023-07-21T18:31:57.392 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-volume-2:18.0.0-4746.g81cf785a.el8.noarch 36/134 2023-07-21T18:31:57.392 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 37/134 2023-07-21T18:31:57.392 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : cephfs-top-2:18.0.0-4746.g81cf785a.el8.noarch 38/134 2023-07-21T18:31:57.392 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-07-21T18:31:57.392 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-07-21T18:31:57.392 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-07-21T18:31:57.392 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-07-21T18:31:57.392 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-07-21T18:31:57.393 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-07-21T18:31:57.393 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-07-21T18:31:57.393 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-07-21T18:31:57.393 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-07-21T18:31:57.393 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-07-21T18:31:57.393 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-07-21T18:31:57.393 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-07-21T18:31:57.393 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-07-21T18:31:57.394 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-07-21T18:31:57.394 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-07-21T18:31:57.394 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-07-21T18:31:57.394 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-07-21T18:31:57.394 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-07-21T18:31:57.394 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-07-21T18:31:57.394 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-07-21T18:31:57.394 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-07-21T18:31:57.395 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-07-21T18:31:57.395 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-07-21T18:31:57.395 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-07-21T18:31:57.395 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-07-21T18:31:57.395 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-07-21T18:31:57.395 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-07-21T18:31:57.395 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-07-21T18:31:57.395 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-07-21T18:31:57.396 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-07-21T18:31:57.396 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-07-21T18:31:57.396 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-07-21T18:31:57.396 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-07-21T18:31:57.396 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-07-21T18:31:57.396 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-07-21T18:31:57.396 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-07-21T18:31:57.396 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-07-21T18:31:57.397 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-07-21T18:31:57.397 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-07-21T18:31:57.397 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-07-21T18:31:57.397 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-07-21T18:31:57.397 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-07-21T18:31:57.397 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-07-21T18:31:57.397 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-07-21T18:31:57.397 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-07-21T18:31:57.398 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-07-21T18:31:57.398 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-07-21T18:31:57.398 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-07-21T18:31:57.398 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-07-21T18:31:57.398 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-07-21T18:31:57.398 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-07-21T18:31:57.398 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-07-21T18:31:57.399 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-07-21T18:31:57.399 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-07-21T18:31:57.399 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-07-21T18:31:57.399 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-07-21T18:31:57.399 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-07-21T18:31:57.399 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-07-21T18:31:57.399 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-07-21T18:31:57.399 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-07-21T18:31:57.400 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-07-21T18:31:57.400 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-07-21T18:31:57.400 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-07-21T18:31:57.400 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-07-21T18:31:57.400 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-07-21T18:31:57.400 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-07-21T18:31:57.400 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-07-21T18:31:57.400 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-07-21T18:31:57.401 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-07-21T18:31:57.401 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-07-21T18:31:57.401 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-07-21T18:31:57.401 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-07-21T18:31:57.401 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-07-21T18:31:57.401 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-07-21T18:31:57.401 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-07-21T18:31:57.401 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-07-21T18:31:57.402 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-07-21T18:31:57.402 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-07-21T18:31:57.402 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-07-21T18:31:57.402 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-07-21T18:31:57.402 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-07-21T18:31:57.402 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-07-21T18:31:57.402 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-07-21T18:31:57.402 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-07-21T18:31:57.403 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-07-21T18:31:57.403 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-07-21T18:31:57.403 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-07-21T18:31:57.403 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-07-21T18:31:57.403 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-07-21T18:31:57.403 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-07-21T18:31:57.403 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librados2-2:18.0.0-4746.g81cf785a.el8.x86_64 129/134 2023-07-21T18:31:57.403 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-07-21T18:31:57.404 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librbd1-2:18.0.0-4746.g81cf785a.el8.x86_64 131/134 2023-07-21T18:31:57.404 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-21T18:31:57.404 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-07-21T18:31:58.127 INFO:teuthology.orchestra.run.smithi193.stdout:Installed products updated. 2023-07-21T18:31:58.337 INFO:teuthology.orchestra.run.smithi193.stdout: 2023-07-21T18:31:58.338 INFO:teuthology.orchestra.run.smithi193.stdout:Upgraded: 2023-07-21T18:31:58.338 INFO:teuthology.orchestra.run.smithi193.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-07-21T18:31:58.338 INFO:teuthology.orchestra.run.smithi193.stdout: librados2-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:31:58.338 INFO:teuthology.orchestra.run.smithi193.stdout: librbd1-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:31:58.338 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2023-07-21T18:31:58.338 INFO:teuthology.orchestra.run.smithi193.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-07-21T18:31:58.339 INFO:teuthology.orchestra.run.smithi193.stdout: automake-1.16.1-7.el8.noarch 2023-07-21T18:31:58.339 INFO:teuthology.orchestra.run.smithi193.stdout: bison-3.0.4-10.el8.x86_64 2023-07-21T18:31:58.339 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:31:58.339 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-base-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:31:58.339 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:31:58.339 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:31:58.339 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-grafana-dashboards-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-21T18:31:58.339 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-immutable-object-cache-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:31:58.340 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mds-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:31:58.340 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:31:58.340 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-21T18:31:58.340 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-dashboard-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-21T18:31:58.340 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-21T18:31:58.340 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-modules-core-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-21T18:31:58.340 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-rook-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-21T18:31:58.341 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mon-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:31:58.341 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-osd-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:31:58.341 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-prometheus-alerts-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-21T18:31:58.341 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-radosgw-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:31:58.341 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-selinux-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:31:58.341 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-test-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:31:58.342 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-volume-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-21T18:31:58.342 INFO:teuthology.orchestra.run.smithi193.stdout: cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-21T18:31:58.342 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:31:58.342 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-top-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-21T18:31:58.342 INFO:teuthology.orchestra.run.smithi193.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-07-21T18:31:58.342 INFO:teuthology.orchestra.run.smithi193.stdout: flex-2.6.1-9.el8.x86_64 2023-07-21T18:31:58.342 INFO:teuthology.orchestra.run.smithi193.stdout: fuse-2.9.7-15.el8.x86_64 2023-07-21T18:31:58.343 INFO:teuthology.orchestra.run.smithi193.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-07-21T18:31:58.343 INFO:teuthology.orchestra.run.smithi193.stdout: jq-1.6-3.el8.x86_64 2023-07-21T18:31:58.343 INFO:teuthology.orchestra.run.smithi193.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-07-21T18:31:58.343 INFO:teuthology.orchestra.run.smithi193.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-07-21T18:31:58.343 INFO:teuthology.orchestra.run.smithi193.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-07-21T18:31:58.343 INFO:teuthology.orchestra.run.smithi193.stdout: libcephfs-devel-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:31:58.343 INFO:teuthology.orchestra.run.smithi193.stdout: libcephfs2-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:31:58.343 INFO:teuthology.orchestra.run.smithi193.stdout: libcephsqlite-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:31:58.344 INFO:teuthology.orchestra.run.smithi193.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-07-21T18:31:58.344 INFO:teuthology.orchestra.run.smithi193.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-07-21T18:31:58.344 INFO:teuthology.orchestra.run.smithi193.stdout: liboath-2.6.2-3.el8.x86_64 2023-07-21T18:31:58.344 INFO:teuthology.orchestra.run.smithi193.stdout: libpq-13.5-1.el8.x86_64 2023-07-21T18:31:58.344 INFO:teuthology.orchestra.run.smithi193.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-07-21T18:31:58.344 INFO:teuthology.orchestra.run.smithi193.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-07-21T18:31:58.344 INFO:teuthology.orchestra.run.smithi193.stdout: librados-devel-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:31:58.345 INFO:teuthology.orchestra.run.smithi193.stdout: libradosstriper1-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:31:58.345 INFO:teuthology.orchestra.run.smithi193.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-07-21T18:31:58.345 INFO:teuthology.orchestra.run.smithi193.stdout: librgw2-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:31:58.345 INFO:teuthology.orchestra.run.smithi193.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-07-21T18:31:58.345 INFO:teuthology.orchestra.run.smithi193.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-07-21T18:31:58.345 INFO:teuthology.orchestra.run.smithi193.stdout: libtool-2.4.6-25.el8.x86_64 2023-07-21T18:31:58.345 INFO:teuthology.orchestra.run.smithi193.stdout: libunwind-1.3.1-3.el8.x86_64 2023-07-21T18:31:58.345 INFO:teuthology.orchestra.run.smithi193.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-07-21T18:31:58.346 INFO:teuthology.orchestra.run.smithi193.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-07-21T18:31:58.346 INFO:teuthology.orchestra.run.smithi193.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-07-21T18:31:58.346 INFO:teuthology.orchestra.run.smithi193.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-07-21T18:31:58.346 INFO:teuthology.orchestra.run.smithi193.stdout: openblas-0.3.15-3.el8.x86_64 2023-07-21T18:31:58.346 INFO:teuthology.orchestra.run.smithi193.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-07-21T18:31:58.346 INFO:teuthology.orchestra.run.smithi193.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-07-21T18:31:58.346 INFO:teuthology.orchestra.run.smithi193.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-07-21T18:31:58.347 INFO:teuthology.orchestra.run.smithi193.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-07-21T18:31:58.348 INFO:teuthology.orchestra.run.smithi193.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-07-21T18:31:58.348 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-21T18:31:58.348 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-21T18:31:58.348 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-21T18:31:58.348 INFO:teuthology.orchestra.run.smithi193.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-07-21T18:31:58.348 INFO:teuthology.orchestra.run.smithi193.stdout: python3-babel-2.5.1-7.el8.noarch 2023-07-21T18:31:58.348 INFO:teuthology.orchestra.run.smithi193.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-07-21T18:31:58.349 INFO:teuthology.orchestra.run.smithi193.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-07-21T18:31:58.349 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-07-21T18:31:58.349 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ceph-argparse-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:31:58.349 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:31:58.349 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cephfs-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:31:58.349 INFO:teuthology.orchestra.run.smithi193.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-07-21T18:31:58.350 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-07-21T18:31:58.350 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-07-21T18:31:58.350 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-07-21T18:31:58.350 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-07-21T18:31:58.351 INFO:teuthology.orchestra.run.smithi193.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-07-21T18:31:58.351 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-6.2-6.el8.noarch 2023-07-21T18:31:58.351 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-07-21T18:31:58.351 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-07-21T18:31:58.351 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-07-21T18:31:58.351 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-07-21T18:31:58.351 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-07-21T18:31:58.352 INFO:teuthology.orchestra.run.smithi193.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-07-21T18:31:58.352 INFO:teuthology.orchestra.run.smithi193.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-07-21T18:31:58.352 INFO:teuthology.orchestra.run.smithi193.stdout: python3-mako-1.0.6-13.el8.noarch 2023-07-21T18:31:58.352 INFO:teuthology.orchestra.run.smithi193.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-07-21T18:31:58.352 INFO:teuthology.orchestra.run.smithi193.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-07-21T18:31:58.352 INFO:teuthology.orchestra.run.smithi193.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-07-21T18:31:58.352 INFO:teuthology.orchestra.run.smithi193.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-07-21T18:31:58.353 INFO:teuthology.orchestra.run.smithi193.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-07-21T18:31:58.353 INFO:teuthology.orchestra.run.smithi193.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-07-21T18:31:58.353 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-07-21T18:31:58.353 INFO:teuthology.orchestra.run.smithi193.stdout: python3-portend-2.6-1.el8.noarch 2023-07-21T18:31:58.353 INFO:teuthology.orchestra.run.smithi193.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-07-21T18:31:58.353 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-07-21T18:31:58.353 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-07-21T18:31:58.353 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-07-21T18:31:58.354 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pycparser-2.14-14.el8.noarch 2023-07-21T18:31:58.354 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pytz-2017.2-9.el8.noarch 2023-07-21T18:31:58.354 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rados-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:31:58.354 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rbd-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:31:58.354 INFO:teuthology.orchestra.run.smithi193.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-07-21T18:31:58.354 INFO:teuthology.orchestra.run.smithi193.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-07-21T18:31:58.354 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rgw-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:31:58.355 INFO:teuthology.orchestra.run.smithi193.stdout: python3-routes-2.4.1-12.el8.noarch 2023-07-21T18:31:58.355 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rsa-4.8-1.el8.noarch 2023-07-21T18:31:58.355 INFO:teuthology.orchestra.run.smithi193.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-07-21T18:31:58.355 INFO:teuthology.orchestra.run.smithi193.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-07-21T18:31:58.355 INFO:teuthology.orchestra.run.smithi193.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-07-21T18:31:58.356 INFO:teuthology.orchestra.run.smithi193.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-07-21T18:31:58.356 INFO:teuthology.orchestra.run.smithi193.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-07-21T18:31:58.356 INFO:teuthology.orchestra.run.smithi193.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-07-21T18:31:58.356 INFO:teuthology.orchestra.run.smithi193.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-07-21T18:31:58.356 INFO:teuthology.orchestra.run.smithi193.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-07-21T18:31:58.357 INFO:teuthology.orchestra.run.smithi193.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-07-21T18:31:58.357 INFO:teuthology.orchestra.run.smithi193.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-07-21T18:31:58.357 INFO:teuthology.orchestra.run.smithi193.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-07-21T18:31:58.357 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:31:58.357 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:31:58.357 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-nbd-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:31:58.357 INFO:teuthology.orchestra.run.smithi193.stdout: socat-1.7.4.1-1.el8.x86_64 2023-07-21T18:31:58.358 INFO:teuthology.orchestra.run.smithi193.stdout: thrift-0.13.0-2.el8.x86_64 2023-07-21T18:31:58.358 INFO:teuthology.orchestra.run.smithi193.stdout: uuid-1.6.2-43.el8.x86_64 2023-07-21T18:31:58.358 INFO:teuthology.orchestra.run.smithi193.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-07-21T18:31:58.358 INFO:teuthology.orchestra.run.smithi193.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-07-21T18:31:58.358 INFO:teuthology.orchestra.run.smithi193.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-07-21T18:31:58.359 INFO:teuthology.orchestra.run.smithi193.stdout: 2023-07-21T18:31:58.359 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2023-07-21T18:31:58.527 DEBUG:teuthology.parallel:result is None 2023-07-21T18:31:59.837 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-21T18:31:59.838 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:01:55 ago on Fri 21 Jul 2023 06:30:04 PM UTC. 2023-07-21T18:32:02.653 INFO:teuthology.orchestra.run.smithi039.stdout:Installed products updated. 2023-07-21T18:32:02.886 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:32:02.886 INFO:teuthology.orchestra.run.smithi039.stdout:Upgraded: 2023-07-21T18:32:02.886 INFO:teuthology.orchestra.run.smithi039.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-07-21T18:32:02.887 INFO:teuthology.orchestra.run.smithi039.stdout: librados2-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:32:02.887 INFO:teuthology.orchestra.run.smithi039.stdout: librbd1-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:32:02.887 INFO:teuthology.orchestra.run.smithi039.stdout:Installed: 2023-07-21T18:32:02.888 INFO:teuthology.orchestra.run.smithi039.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-07-21T18:32:02.888 INFO:teuthology.orchestra.run.smithi039.stdout: automake-1.16.1-7.el8.noarch 2023-07-21T18:32:02.888 INFO:teuthology.orchestra.run.smithi039.stdout: bison-3.0.4-10.el8.x86_64 2023-07-21T18:32:02.888 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:32:02.889 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-base-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:32:02.889 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:32:02.889 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:32:02.889 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-grafana-dashboards-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-21T18:32:02.890 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-immutable-object-cache-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:32:02.890 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mds-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:32:02.890 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:32:02.890 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-21T18:32:02.891 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-dashboard-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-21T18:32:02.891 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-21T18:32:02.891 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-modules-core-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-21T18:32:02.892 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-rook-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-21T18:32:02.892 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mon-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:32:02.892 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-osd-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:32:02.892 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-prometheus-alerts-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-21T18:32:02.893 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-radosgw-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:32:02.893 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-selinux-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:32:02.893 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-test-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:32:02.893 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-volume-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-21T18:32:02.894 INFO:teuthology.orchestra.run.smithi039.stdout: cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-21T18:32:02.894 INFO:teuthology.orchestra.run.smithi039.stdout: cephfs-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:32:02.894 INFO:teuthology.orchestra.run.smithi039.stdout: cephfs-top-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-21T18:32:02.894 INFO:teuthology.orchestra.run.smithi039.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-07-21T18:32:02.895 INFO:teuthology.orchestra.run.smithi039.stdout: flex-2.6.1-9.el8.x86_64 2023-07-21T18:32:02.895 INFO:teuthology.orchestra.run.smithi039.stdout: fuse-2.9.7-15.el8.x86_64 2023-07-21T18:32:02.895 INFO:teuthology.orchestra.run.smithi039.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-07-21T18:32:02.895 INFO:teuthology.orchestra.run.smithi039.stdout: jq-1.6-3.el8.x86_64 2023-07-21T18:32:02.896 INFO:teuthology.orchestra.run.smithi039.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-07-21T18:32:02.896 INFO:teuthology.orchestra.run.smithi039.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-07-21T18:32:02.896 INFO:teuthology.orchestra.run.smithi039.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-07-21T18:32:02.896 INFO:teuthology.orchestra.run.smithi039.stdout: libcephfs-devel-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:32:02.897 INFO:teuthology.orchestra.run.smithi039.stdout: libcephfs2-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:32:02.897 INFO:teuthology.orchestra.run.smithi039.stdout: libcephsqlite-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:32:02.897 INFO:teuthology.orchestra.run.smithi039.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-07-21T18:32:02.897 INFO:teuthology.orchestra.run.smithi039.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-07-21T18:32:02.898 INFO:teuthology.orchestra.run.smithi039.stdout: liboath-2.6.2-3.el8.x86_64 2023-07-21T18:32:02.898 INFO:teuthology.orchestra.run.smithi039.stdout: libpq-13.5-1.el8.x86_64 2023-07-21T18:32:02.898 INFO:teuthology.orchestra.run.smithi039.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-07-21T18:32:02.898 INFO:teuthology.orchestra.run.smithi039.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-07-21T18:32:02.899 INFO:teuthology.orchestra.run.smithi039.stdout: librados-devel-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:32:02.899 INFO:teuthology.orchestra.run.smithi039.stdout: libradosstriper1-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:32:02.899 INFO:teuthology.orchestra.run.smithi039.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-07-21T18:32:02.899 INFO:teuthology.orchestra.run.smithi039.stdout: librgw2-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:32:02.900 INFO:teuthology.orchestra.run.smithi039.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-07-21T18:32:02.900 INFO:teuthology.orchestra.run.smithi039.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-07-21T18:32:02.900 INFO:teuthology.orchestra.run.smithi039.stdout: libtool-2.4.6-25.el8.x86_64 2023-07-21T18:32:02.901 INFO:teuthology.orchestra.run.smithi039.stdout: libunwind-1.3.1-3.el8.x86_64 2023-07-21T18:32:02.901 INFO:teuthology.orchestra.run.smithi039.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-07-21T18:32:02.901 INFO:teuthology.orchestra.run.smithi039.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-07-21T18:32:02.901 INFO:teuthology.orchestra.run.smithi039.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-07-21T18:32:02.902 INFO:teuthology.orchestra.run.smithi039.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-07-21T18:32:02.902 INFO:teuthology.orchestra.run.smithi039.stdout: openblas-0.3.15-3.el8.x86_64 2023-07-21T18:32:02.902 INFO:teuthology.orchestra.run.smithi039.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-07-21T18:32:02.902 INFO:teuthology.orchestra.run.smithi039.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-07-21T18:32:02.903 INFO:teuthology.orchestra.run.smithi039.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-07-21T18:32:02.903 INFO:teuthology.orchestra.run.smithi039.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-07-21T18:32:02.903 INFO:teuthology.orchestra.run.smithi039.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-07-21T18:32:02.903 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-21T18:32:02.904 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-21T18:32:02.904 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-21T18:32:02.904 INFO:teuthology.orchestra.run.smithi039.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-07-21T18:32:02.905 INFO:teuthology.orchestra.run.smithi039.stdout: python3-babel-2.5.1-7.el8.noarch 2023-07-21T18:32:02.905 INFO:teuthology.orchestra.run.smithi039.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-07-21T18:32:02.905 INFO:teuthology.orchestra.run.smithi039.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-07-21T18:32:02.905 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-07-21T18:32:02.906 INFO:teuthology.orchestra.run.smithi039.stdout: python3-ceph-argparse-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:32:02.906 INFO:teuthology.orchestra.run.smithi039.stdout: python3-ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:32:02.906 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cephfs-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:32:02.906 INFO:teuthology.orchestra.run.smithi039.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-07-21T18:32:02.907 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-07-21T18:32:02.907 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-07-21T18:32:02.907 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-07-21T18:32:02.907 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-07-21T18:32:02.908 INFO:teuthology.orchestra.run.smithi039.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-07-21T18:32:02.908 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-6.2-6.el8.noarch 2023-07-21T18:32:02.908 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-07-21T18:32:02.909 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-07-21T18:32:02.909 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-07-21T18:32:02.909 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-07-21T18:32:02.909 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-07-21T18:32:02.910 INFO:teuthology.orchestra.run.smithi039.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-07-21T18:32:02.910 INFO:teuthology.orchestra.run.smithi039.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-07-21T18:32:02.910 INFO:teuthology.orchestra.run.smithi039.stdout: python3-mako-1.0.6-13.el8.noarch 2023-07-21T18:32:02.910 INFO:teuthology.orchestra.run.smithi039.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-07-21T18:32:02.911 INFO:teuthology.orchestra.run.smithi039.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-07-21T18:32:02.911 INFO:teuthology.orchestra.run.smithi039.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-07-21T18:32:02.911 INFO:teuthology.orchestra.run.smithi039.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-07-21T18:32:02.911 INFO:teuthology.orchestra.run.smithi039.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-07-21T18:32:02.912 INFO:teuthology.orchestra.run.smithi039.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-07-21T18:32:02.912 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-07-21T18:32:02.912 INFO:teuthology.orchestra.run.smithi039.stdout: python3-portend-2.6-1.el8.noarch 2023-07-21T18:32:02.912 INFO:teuthology.orchestra.run.smithi039.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-07-21T18:32:02.913 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-07-21T18:32:02.913 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-07-21T18:32:02.913 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-07-21T18:32:02.914 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pycparser-2.14-14.el8.noarch 2023-07-21T18:32:02.914 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pytz-2017.2-9.el8.noarch 2023-07-21T18:32:02.914 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rados-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:32:02.914 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rbd-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:32:02.914 INFO:teuthology.orchestra.run.smithi039.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-07-21T18:32:02.915 INFO:teuthology.orchestra.run.smithi039.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-07-21T18:32:02.915 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rgw-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:32:02.915 INFO:teuthology.orchestra.run.smithi039.stdout: python3-routes-2.4.1-12.el8.noarch 2023-07-21T18:32:02.915 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rsa-4.8-1.el8.noarch 2023-07-21T18:32:02.916 INFO:teuthology.orchestra.run.smithi039.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-07-21T18:32:02.916 INFO:teuthology.orchestra.run.smithi039.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-07-21T18:32:02.916 INFO:teuthology.orchestra.run.smithi039.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-07-21T18:32:02.917 INFO:teuthology.orchestra.run.smithi039.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-07-21T18:32:02.917 INFO:teuthology.orchestra.run.smithi039.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-07-21T18:32:02.917 INFO:teuthology.orchestra.run.smithi039.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-07-21T18:32:02.917 INFO:teuthology.orchestra.run.smithi039.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-07-21T18:32:02.918 INFO:teuthology.orchestra.run.smithi039.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-07-21T18:32:02.918 INFO:teuthology.orchestra.run.smithi039.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-07-21T18:32:02.918 INFO:teuthology.orchestra.run.smithi039.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-07-21T18:32:02.919 INFO:teuthology.orchestra.run.smithi039.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-07-21T18:32:02.919 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:32:02.919 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:32:02.919 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-nbd-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:32:02.920 INFO:teuthology.orchestra.run.smithi039.stdout: socat-1.7.4.1-1.el8.x86_64 2023-07-21T18:32:02.920 INFO:teuthology.orchestra.run.smithi039.stdout: thrift-0.13.0-2.el8.x86_64 2023-07-21T18:32:02.920 INFO:teuthology.orchestra.run.smithi039.stdout: uuid-1.6.2-43.el8.x86_64 2023-07-21T18:32:02.921 INFO:teuthology.orchestra.run.smithi039.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-07-21T18:32:02.921 INFO:teuthology.orchestra.run.smithi039.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-07-21T18:32:02.922 INFO:teuthology.orchestra.run.smithi039.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-07-21T18:32:02.922 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:32:02.922 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2023-07-21T18:32:03.122 DEBUG:teuthology.parallel:result is None 2023-07-21T18:32:03.122 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=81cf785aa7781c4e08381c540f890cb9bc987ef9 2023-07-21T18:32:08.434 DEBUG:teuthology.orchestra.run.smithi039:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-07-21T18:32:08.464 INFO:teuthology.orchestra.run.smithi039.stdout:18.0.0-4746.g81cf785a.el8 2023-07-21T18:32:08.465 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4746.g81cf785a.el8 2023-07-21T18:32:08.466 INFO:teuthology.task.install:The correct ceph version 18.0.0-4746.g81cf785a is installed. 2023-07-21T18:32:08.466 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=81cf785aa7781c4e08381c540f890cb9bc987ef9 2023-07-21T18:32:13.703 DEBUG:teuthology.orchestra.run.smithi155:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-07-21T18:32:13.731 INFO:teuthology.orchestra.run.smithi155.stdout:18.0.0-4746.g81cf785a.el8 2023-07-21T18:32:13.732 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4746.g81cf785a.el8 2023-07-21T18:32:13.732 INFO:teuthology.task.install:The correct ceph version 18.0.0-4746.g81cf785a is installed. 2023-07-21T18:32:13.732 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=81cf785aa7781c4e08381c540f890cb9bc987ef9 2023-07-21T18:32:15.066 DEBUG:teuthology.orchestra.run.smithi193:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-07-21T18:32:15.096 INFO:teuthology.orchestra.run.smithi193.stdout:18.0.0-4746.g81cf785a.el8 2023-07-21T18:32:15.096 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4746.g81cf785a.el8 2023-07-21T18:32:15.096 INFO:teuthology.task.install:The correct ceph version 18.0.0-4746.g81cf785a is installed. 2023-07-21T18:32:15.097 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-07-21T18:32:15.097 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-07-21T18:32:15.098 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-07-21T18:32:15.147 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-07-21T18:32:15.148 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-07-21T18:32:15.197 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2023-07-21T18:32:15.197 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-07-21T18:32:15.246 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-07-21T18:32:15.247 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-07-21T18:32:15.247 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/usr/bin/daemon-helper 2023-07-21T18:32:15.281 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-07-21T18:32:15.351 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-07-21T18:32:15.352 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/usr/bin/daemon-helper 2023-07-21T18:32:15.384 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-07-21T18:32:15.453 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2023-07-21T18:32:15.453 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/usr/bin/daemon-helper 2023-07-21T18:32:15.486 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-07-21T18:32:15.558 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-07-21T18:32:15.558 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-07-21T18:32:15.559 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/usr/bin/adjust-ulimits 2023-07-21T18:32:15.592 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-07-21T18:32:15.664 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-07-21T18:32:15.665 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/usr/bin/adjust-ulimits 2023-07-21T18:32:15.695 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-07-21T18:32:15.763 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2023-07-21T18:32:15.763 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/usr/bin/adjust-ulimits 2023-07-21T18:32:15.795 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-07-21T18:32:15.867 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-07-21T18:32:15.867 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-07-21T18:32:15.867 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/usr/bin/stdin-killer 2023-07-21T18:32:15.900 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-07-21T18:32:15.970 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-07-21T18:32:15.971 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/usr/bin/stdin-killer 2023-07-21T18:32:16.000 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-07-21T18:32:16.069 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2023-07-21T18:32:16.069 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/usr/bin/stdin-killer 2023-07-21T18:32:16.101 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-07-21T18:32:16.171 INFO:teuthology.run_tasks:Running task cephadm... 2023-07-21T18:32:16.264 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '81cf785aa7781c4e08381c540f890cb9bc987ef9', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2023-07-21T18:32:16.265 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 2023-07-21T18:32:16.274 INFO:tasks.cephadm:Cluster fsid is eaf8e25a-27f4-11ee-9b34-001a4aab830c 2023-07-21T18:32:16.275 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2023-07-21T18:32:16.275 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.39', 'mon.b': '172.21.15.155', 'mon.c': '172.21.15.193'} 2023-07-21T18:32:16.276 INFO:tasks.cephadm:First mon is mon.a on smithi039 2023-07-21T18:32:16.276 INFO:tasks.cephadm:First mgr is x 2023-07-21T18:32:16.276 INFO:tasks.cephadm:Normalizing hostnames... 2023-07-21T18:32:16.277 DEBUG:teuthology.orchestra.run.smithi039:> sudo hostname $(hostname -s) 2023-07-21T18:32:16.311 DEBUG:teuthology.orchestra.run.smithi155:> sudo hostname $(hostname -s) 2023-07-21T18:32:16.341 DEBUG:teuthology.orchestra.run.smithi193:> sudo hostname $(hostname -s) 2023-07-21T18:32:16.371 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2023-07-21T18:32:16.372 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=81cf785aa7781c4e08381c540f890cb9bc987ef9 2023-07-21T18:32:16.613 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '81cf785aa7781c4e08381c540f890cb9bc987ef9', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/71509/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.0.0-4746-g81cf785a', 'node_name': '172.21.2.5+braggi05', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.0.0-4746.g81cf785a'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2023-07-18 00:38:12.274423', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-2023Jul13-m-quota', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2023-07-21T18:32:16.843 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-rishabh-2023Jul13-m-quota, sha1 81cf785aa7781c4e08381c540f890cb9bc987ef9 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=81cf785aa7781c4e08381c540f890cb9bc987ef9 2023-07-21T18:32:16.843 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/x86_64/flavors/default/cephadm 2023-07-21T18:32:16.844 DEBUG:teuthology.orchestra.run.smithi039:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-07-21T18:32:17.582 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-rw-r--. 1 ubuntu ubuntu 663791 Jul 21 18:32 /home/ubuntu/cephtest/cephadm 2023-07-21T18:32:17.583 DEBUG:teuthology.orchestra.run.smithi155:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-07-21T18:32:18.182 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-rw-r--. 1 ubuntu ubuntu 663791 Jul 21 18:32 /home/ubuntu/cephtest/cephadm 2023-07-21T18:32:18.183 DEBUG:teuthology.orchestra.run.smithi193:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-07-21T18:32:18.772 INFO:teuthology.orchestra.run.smithi193.stdout:-rw-rw-r--. 1 ubuntu ubuntu 663791 Jul 21 18:32 /home/ubuntu/cephtest/cephadm 2023-07-21T18:32:18.773 DEBUG:teuthology.orchestra.run.smithi039:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-07-21T18:32:18.795 DEBUG:teuthology.orchestra.run.smithi155:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-07-21T18:32:18.817 DEBUG:teuthology.orchestra.run.smithi193:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-07-21T18:32:18.857 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 on all hosts... 2023-07-21T18:32:18.858 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 pull 2023-07-21T18:32:18.860 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 pull 2023-07-21T18:32:18.862 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 pull 2023-07-21T18:32:21.013 INFO:teuthology.orchestra.run.smithi155.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9... 2023-07-21T18:32:21.422 INFO:teuthology.orchestra.run.smithi193.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9... 2023-07-21T18:32:21.719 INFO:teuthology.orchestra.run.smithi039.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9... 2023-07-21T18:32:47.413 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2023-07-21T18:32:47.414 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_version": "ceph version 18.0.0-4746-g81cf785a (81cf785aa7781c4e08381c540f890cb9bc987ef9) reef (dev)", 2023-07-21T18:32:47.414 INFO:teuthology.orchestra.run.smithi039.stdout: "image_id": "66a4f7b015b53bd7c8d67b561c2d8f6525dccea2a3d3e0d20f2a1cd17a765421", 2023-07-21T18:32:47.414 INFO:teuthology.orchestra.run.smithi039.stdout: "repo_digests": [ 2023-07-21T18:32:47.414 INFO:teuthology.orchestra.run.smithi039.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b4b7dd677a0f5b8e56a57eadd762824d684980cb80d319543e86b72e549ef9c" 2023-07-21T18:32:47.414 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2023-07-21T18:32:47.415 INFO:teuthology.orchestra.run.smithi039.stdout:} 2023-07-21T18:32:50.139 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2023-07-21T18:32:50.139 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_version": "ceph version 18.0.0-4746-g81cf785a (81cf785aa7781c4e08381c540f890cb9bc987ef9) reef (dev)", 2023-07-21T18:32:50.139 INFO:teuthology.orchestra.run.smithi193.stdout: "image_id": "66a4f7b015b53bd7c8d67b561c2d8f6525dccea2a3d3e0d20f2a1cd17a765421", 2023-07-21T18:32:50.140 INFO:teuthology.orchestra.run.smithi193.stdout: "repo_digests": [ 2023-07-21T18:32:50.140 INFO:teuthology.orchestra.run.smithi193.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b4b7dd677a0f5b8e56a57eadd762824d684980cb80d319543e86b72e549ef9c" 2023-07-21T18:32:50.140 INFO:teuthology.orchestra.run.smithi193.stdout: ] 2023-07-21T18:32:50.141 INFO:teuthology.orchestra.run.smithi193.stdout:} 2023-07-21T18:32:56.957 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2023-07-21T18:32:56.958 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_version": "ceph version 18.0.0-4746-g81cf785a (81cf785aa7781c4e08381c540f890cb9bc987ef9) reef (dev)", 2023-07-21T18:32:56.958 INFO:teuthology.orchestra.run.smithi155.stdout: "image_id": "66a4f7b015b53bd7c8d67b561c2d8f6525dccea2a3d3e0d20f2a1cd17a765421", 2023-07-21T18:32:56.958 INFO:teuthology.orchestra.run.smithi155.stdout: "repo_digests": [ 2023-07-21T18:32:56.958 INFO:teuthology.orchestra.run.smithi155.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b4b7dd677a0f5b8e56a57eadd762824d684980cb80d319543e86b72e549ef9c" 2023-07-21T18:32:56.958 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2023-07-21T18:32:56.959 INFO:teuthology.orchestra.run.smithi155.stdout:} 2023-07-21T18:32:56.977 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /etc/ceph 2023-07-21T18:32:57.016 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /etc/ceph 2023-07-21T18:32:57.046 DEBUG:teuthology.orchestra.run.smithi193:> sudo mkdir -p /etc/ceph 2023-07-21T18:32:57.083 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 777 /etc/ceph 2023-07-21T18:32:57.115 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 777 /etc/ceph 2023-07-21T18:32:57.142 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod 777 /etc/ceph 2023-07-21T18:32:57.172 INFO:tasks.cephadm:Writing seed config... 2023-07-21T18:32:57.174 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2023-07-21T18:32:57.175 INFO:tasks.cephadm: override: [client] debug client = 20 2023-07-21T18:32:57.175 INFO:tasks.cephadm: override: [client] debug ms = 1 2023-07-21T18:32:57.175 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2023-07-21T18:32:57.175 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2023-07-21T18:32:57.175 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2023-07-21T18:32:57.176 INFO:tasks.cephadm: override: [mds] debug mds = 20 2023-07-21T18:32:57.176 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2023-07-21T18:32:57.176 INFO:tasks.cephadm: override: [mds] debug ms = 1 2023-07-21T18:32:57.176 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2023-07-21T18:32:57.177 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2023-07-21T18:32:57.177 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2023-07-21T18:32:57.177 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2023-07-21T18:32:57.177 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2023-07-21T18:32:57.177 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2023-07-21T18:32:57.178 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2023-07-21T18:32:57.178 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2023-07-21T18:32:57.178 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2023-07-21T18:32:57.178 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2023-07-21T18:32:57.178 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2023-07-21T18:32:57.179 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2023-07-21T18:32:57.179 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2023-07-21T18:32:57.179 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2023-07-21T18:32:57.179 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2023-07-21T18:32:57.179 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2023-07-21T18:32:57.179 INFO:tasks.cephadm: override: [mon] debug mon = 20 2023-07-21T18:32:57.180 INFO:tasks.cephadm: override: [mon] debug ms = 1 2023-07-21T18:32:57.180 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2023-07-21T18:32:57.180 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2023-07-21T18:32:57.180 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2023-07-21T18:32:57.181 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2023-07-21T18:32:57.181 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2023-07-21T18:32:57.181 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2023-07-21T18:32:57.181 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2023-07-21T18:32:57.182 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2023-07-21T18:32:57.182 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2023-07-21T18:32:57.182 INFO:tasks.cephadm: override: [osd] debug ms = 1 2023-07-21T18:32:57.182 INFO:tasks.cephadm: override: [osd] debug osd = 20 2023-07-21T18:32:57.182 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2023-07-21T18:32:57.183 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2023-07-21T18:32:57.183 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2023-07-21T18:32:57.183 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2023-07-21T18:32:57.183 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2023-07-21T18:32:57.183 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2023-07-21T18:32:57.183 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2023-07-21T18:32:57.184 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2023-07-21T18:32:57.184 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2023-07-21T18:32:57.185 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-07-21T18:32:57.185 DEBUG:teuthology.orchestra.run.smithi039:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2023-07-21T18:32:57.205 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = eaf8e25a-27f4-11ee-9b34-001a4aab830c ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2023-07-21T18:32:57.206 DEBUG:teuthology.orchestra.run.smithi039:mon.a> sudo journalctl -f -n 0 -u ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@mon.a.service 2023-07-21T18:32:57.248 DEBUG:teuthology.orchestra.run.smithi039:mgr.x> sudo journalctl -f -n 0 -u ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@mgr.x.service 2023-07-21T18:32:57.281 INFO:journalctl@ceph.mon.a.smithi039.stdout:-- Logs begin at Fri 2023-07-21 18:18:00 UTC. -- 2023-07-21T18:32:57.281 INFO:tasks.cephadm:Bootstrapping... 2023-07-21T18:32:57.282 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 -v bootstrap --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.39 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2023-07-21T18:32:57.311 INFO:journalctl@ceph.mgr.x.smithi039.stdout:-- Logs begin at Fri 2023-07-21 18:18:00 UTC. -- 2023-07-21T18:32:57.454 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2023-07-21T18:32:57.455 INFO:teuthology.orchestra.run.smithi039.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9', '-v', 'bootstrap', '--fsid', 'eaf8e25a-27f4-11ee-9b34-001a4aab830c', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.39', '--skip-admin-label'] 2023-07-21T18:32:57.608 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stdout 4.1.1 2023-07-21T18:32:57.609 INFO:teuthology.orchestra.run.smithi039.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2023-07-21T18:32:57.609 INFO:teuthology.orchestra.run.smithi039.stdout:Verifying podman|docker is present... 2023-07-21T18:32:57.750 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stdout 4.1.1 2023-07-21T18:32:57.750 INFO:teuthology.orchestra.run.smithi039.stdout:Verifying lvm2 is present... 2023-07-21T18:32:57.751 INFO:teuthology.orchestra.run.smithi039.stdout:Verifying time synchronization is in place... 2023-07-21T18:32:57.757 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-07-21T18:32:57.758 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-07-21T18:32:57.764 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-07-21T18:32:57.764 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout inactive 2023-07-21T18:32:57.770 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout enabled 2023-07-21T18:32:57.776 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout active 2023-07-21T18:32:57.776 INFO:teuthology.orchestra.run.smithi039.stdout:Unit chronyd.service is enabled and running 2023-07-21T18:32:57.776 INFO:teuthology.orchestra.run.smithi039.stdout:Repeating the final host check... 2023-07-21T18:32:57.929 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stdout 4.1.1 2023-07-21T18:32:57.929 INFO:teuthology.orchestra.run.smithi039.stdout:podman (/bin/podman) version 4.1.1 is present 2023-07-21T18:32:57.930 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl is present 2023-07-21T18:32:57.930 INFO:teuthology.orchestra.run.smithi039.stdout:lvcreate is present 2023-07-21T18:32:57.937 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-07-21T18:32:57.937 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-07-21T18:32:57.943 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-07-21T18:32:57.943 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout inactive 2023-07-21T18:32:57.949 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout enabled 2023-07-21T18:32:57.955 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout active 2023-07-21T18:32:57.955 INFO:teuthology.orchestra.run.smithi039.stdout:Unit chronyd.service is enabled and running 2023-07-21T18:32:57.956 INFO:teuthology.orchestra.run.smithi039.stdout:Host looks OK 2023-07-21T18:32:57.956 INFO:teuthology.orchestra.run.smithi039.stdout:Cluster fsid: eaf8e25a-27f4-11ee-9b34-001a4aab830c 2023-07-21T18:32:57.956 INFO:teuthology.orchestra.run.smithi039.stdout:Acquiring lock 139725220256232 on /run/cephadm/eaf8e25a-27f4-11ee-9b34-001a4aab830c.lock 2023-07-21T18:32:57.956 INFO:teuthology.orchestra.run.smithi039.stdout:Lock 139725220256232 acquired on /run/cephadm/eaf8e25a-27f4-11ee-9b34-001a4aab830c.lock 2023-07-21T18:32:57.957 INFO:teuthology.orchestra.run.smithi039.stdout:Verifying IP 172.21.15.39 port 3300 ... 2023-07-21T18:32:57.957 INFO:teuthology.orchestra.run.smithi039.stdout:Verifying IP 172.21.15.39 port 6789 ... 2023-07-21T18:32:57.957 INFO:teuthology.orchestra.run.smithi039.stdout:Base mon IP(s) is [172.21.15.39:3300, 172.21.15.39:6789], mon addrv is [v2:172.21.15.39:3300,v1:172.21.15.39:6789] 2023-07-21T18:32:57.960 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp metric 100 2023-07-21T18:32:57.960 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.39 metric 100 2023-07-21T18:32:57.963 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2023-07-21T18:32:57.963 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2023-07-21T18:32:57.963 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1792sec hoplimit 64 pref medium 2023-07-21T18:32:57.965 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2023-07-21T18:32:57.965 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2023-07-21T18:32:57.966 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-07-21T18:32:57.966 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2023-07-21T18:32:57.966 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1370/64 scope link 2023-07-21T18:32:57.966 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-07-21T18:32:57.967 INFO:teuthology.orchestra.run.smithi039.stdout:Mon IP `172.21.15.39` is in CIDR network `172.21.0.0/20` 2023-07-21T18:32:57.967 INFO:teuthology.orchestra.run.smithi039.stdout:Mon IP `172.21.15.39` is in CIDR network `172.21.0.0/20` 2023-07-21T18:32:57.967 INFO:teuthology.orchestra.run.smithi039.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2023-07-21T18:32:57.967 INFO:teuthology.orchestra.run.smithi039.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2023-07-21T18:32:57.968 INFO:teuthology.orchestra.run.smithi039.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9... 2023-07-21T18:32:58.413 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stdout 66a4f7b015b53bd7c8d67b561c2d8f6525dccea2a3d3e0d20f2a1cd17a765421 2023-07-21T18:32:58.413 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9... 2023-07-21T18:32:58.414 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stderr Getting image source signatures 2023-07-21T18:32:58.414 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stderr Copying blob sha256:711183bdaeabd1b922e81a2784cd69d11b05a304985585795e1ae029bf69712a 2023-07-21T18:32:58.414 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stderr Copying blob sha256:6022c611d474f2b306ddd1edfc8360123ece5266119096f25fde843d7967e636 2023-07-21T18:32:58.414 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stderr Copying config sha256:66a4f7b015b53bd7c8d67b561c2d8f6525dccea2a3d3e0d20f2a1cd17a765421 2023-07-21T18:32:58.414 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stderr Writing manifest to image destination 2023-07-21T18:32:58.414 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stderr Storing signatures 2023-07-21T18:32:59.261 INFO:teuthology.orchestra.run.smithi039.stdout:ceph: stdout ceph version 18.0.0-4746-g81cf785a (81cf785aa7781c4e08381c540f890cb9bc987ef9) reef (dev) 2023-07-21T18:32:59.261 INFO:teuthology.orchestra.run.smithi039.stdout:Ceph version: ceph version 18.0.0-4746-g81cf785a (81cf785aa7781c4e08381c540f890cb9bc987ef9) reef (dev) 2023-07-21T18:32:59.261 INFO:teuthology.orchestra.run.smithi039.stdout:Extracting ceph user uid/gid from container image... 2023-07-21T18:32:59.910 INFO:teuthology.orchestra.run.smithi039.stdout:stat: stdout 167 167 2023-07-21T18:32:59.911 INFO:teuthology.orchestra.run.smithi039.stdout:Creating initial keys... 2023-07-21T18:33:00.657 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph-authtool: stdout AQBcz7pkiCYADhAAjV6sDLtW6PrvKOFwqiPViQ== 2023-07-21T18:33:01.302 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph-authtool: stdout AQBcz7pkiaYAOhAAATSWmeuanqKwtIjStHbf6Q== 2023-07-21T18:33:02.051 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph-authtool: stdout AQBdz7pkt8jlKRAAWbuPq2VBOQ2EqqR7agx/pw== 2023-07-21T18:33:02.051 INFO:teuthology.orchestra.run.smithi039.stdout:Creating initial monmap... 2023-07-21T18:33:02.799 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2023-07-21T18:33:02.799 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2023-07-21T18:33:02.799 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to eaf8e25a-27f4-11ee-9b34-001a4aab830c 2023-07-21T18:33:02.799 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-07-21T18:33:02.800 INFO:teuthology.orchestra.run.smithi039.stdout:monmaptool for a [v2:172.21.15.39:3300,v1:172.21.15.39:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2023-07-21T18:33:02.800 INFO:teuthology.orchestra.run.smithi039.stdout:setting min_mon_release = pacific 2023-07-21T18:33:02.800 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/monmaptool: set fsid to eaf8e25a-27f4-11ee-9b34-001a4aab830c 2023-07-21T18:33:02.800 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-07-21T18:33:02.800 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:33:02.800 INFO:teuthology.orchestra.run.smithi039.stdout:Creating mon... 2023-07-21T18:33:04.850 INFO:teuthology.orchestra.run.smithi039.stdout:create mon.a on 2023-07-21T18:33:05.173 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2023-07-21T18:33:05.272 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2023-07-21T18:33:05.651 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c.target → /etc/systemd/system/ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c.target. 2023-07-21T18:33:05.651 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c.target → /etc/systemd/system/ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c.target. 2023-07-21T18:33:05.894 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@mon.a 2023-07-21T18:33:05.894 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Failed to reset failed state of unit ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@mon.a.service: Unit ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@mon.a.service not loaded. 2023-07-21T18:33:05.987 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c.target.wants/ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@mon.a.service → /etc/systemd/system/ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@.service. 2023-07-21T18:33:06.199 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:05 smithi039 systemd[1]: Starting Ceph mon.a for eaf8e25a-27f4-11ee-9b34-001a4aab830c... 2023-07-21T18:33:06.557 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:06 smithi039 podman[95935]: 2023-07-21T18:33:06.557 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:06 smithi039 bash[95935]: 0efc6952161331662cb397ede644c317c91f741beb00e442d0a7210b0962370a 2023-07-21T18:33:06.595 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-07-21T18:33:06.595 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout disabled 2023-07-21T18:33:06.602 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-07-21T18:33:06.603 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout inactive 2023-07-21T18:33:06.603 INFO:teuthology.orchestra.run.smithi039.stdout:firewalld.service is not enabled 2023-07-21T18:33:06.603 INFO:teuthology.orchestra.run.smithi039.stdout:Not possible to enable service . firewalld.service is not available 2023-07-21T18:33:06.604 INFO:teuthology.orchestra.run.smithi039.stdout:Waiting for mon to start... 2023-07-21T18:33:06.604 INFO:teuthology.orchestra.run.smithi039.stdout:Waiting for mon... 2023-07-21T18:33:06.825 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:06 smithi039 ceph-mon[95975]: mkfs eaf8e25a-27f4-11ee-9b34-001a4aab830c 2023-07-21T18:33:06.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:06 smithi039 ceph-mon[95975]: mon.a is new leader, mons a in quorum (ranks 0) 2023-07-21T18:33:06.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:06 smithi039 systemd[1]: Started Ceph mon.a for eaf8e25a-27f4-11ee-9b34-001a4aab830c. 2023-07-21T18:33:07.554 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout cluster: 2023-07-21T18:33:07.555 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout id: eaf8e25a-27f4-11ee-9b34-001a4aab830c 2023-07-21T18:33:07.555 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2023-07-21T18:33:07.555 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2023-07-21T18:33:07.555 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout services: 2023-07-21T18:33:07.555 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.690183s) 2023-07-21T18:33:07.556 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2023-07-21T18:33:07.556 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2023-07-21T18:33:07.556 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2023-07-21T18:33:07.556 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout data: 2023-07-21T18:33:07.556 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2023-07-21T18:33:07.556 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2023-07-21T18:33:07.556 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2023-07-21T18:33:07.556 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout pgs: 2023-07-21T18:33:07.557 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2023-07-21T18:33:07.557 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:07.053+0000 7f39f2408700 1 Processor -- start 2023-07-21T18:33:07.557 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:07.053+0000 7f39f2408700 1 -- start start 2023-07-21T18:33:07.557 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:07.053+0000 7f39f2408700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39ec1054f0 0x7f39ec105900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:33:07.557 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:07.053+0000 7f39f2408700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39ec105e40 con 0x7f39ec1054f0 2023-07-21T18:33:07.557 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:07.054+0000 7f39ebfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39ec1054f0 0x7f39ec105900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:33:07.557 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:07.054+0000 7f39ebfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39ec1054f0 0x7f39ec105900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53674/0 (socket says 172.21.15.39:53674) 2023-07-21T18:33:07.558 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:07.054+0000 7f39ebfff700 1 -- 172.21.15.39:0/1036981902 learned_addr learned my addr 172.21.15.39:0/1036981902 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:33:07.558 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:07.054+0000 7f39ebfff700 1 -- 172.21.15.39:0/1036981902 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39ec105f80 con 0x7f39ec1054f0 2023-07-21T18:33:07.558 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:07.054+0000 7f39ebfff700 1 --2- 172.21.15.39:0/1036981902 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39ec1054f0 0x7f39ec105900 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f39dc0098a0 tx=0x7f39dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9453cedfc343207 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:33:07.558 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:07.054+0000 7f39eaffd700 1 -- 172.21.15.39:0/1036981902 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f39dc004160 con 0x7f39ec1054f0 2023-07-21T18:33:07.558 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:07.054+0000 7f39eaffd700 1 -- 172.21.15.39:0/1036981902 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39dc0042c0 con 0x7f39ec1054f0 2023-07-21T18:33:07.558 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:07.054+0000 7f39eaffd700 1 -- 172.21.15.39:0/1036981902 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f39dc0045a0 con 0x7f39ec1054f0 2023-07-21T18:33:07.558 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:07.055+0000 7f39f2408700 1 -- 172.21.15.39:0/1036981902 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39ec1054f0 msgr2=0x7f39ec105900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:33:07.558 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:07.055+0000 7f39f2408700 1 --2- 172.21.15.39:0/1036981902 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39ec1054f0 0x7f39ec105900 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f39dc0098a0 tx=0x7f39dc009bb0 comp rx=0 tx=0).stop 2023-07-21T18:33:07.559 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:07.055+0000 7f39f2408700 1 -- 172.21.15.39:0/1036981902 shutdown_connections 2023-07-21T18:33:07.559 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:07.055+0000 7f39f2408700 1 --2- 172.21.15.39:0/1036981902 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39ec1054f0 0x7f39ec105900 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:33:07.559 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:07.055+0000 7f39f2408700 1 -- 172.21.15.39:0/1036981902 >> 172.21.15.39:0/1036981902 conn(0x7f39ec100b40 msgr2=0x7f39ec102f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:33:07.559 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:07.055+0000 7f39f2408700 1 -- 172.21.15.39:0/1036981902 shutdown_connections 2023-07-21T18:33:07.559 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:07.055+0000 7f39f2408700 1 -- 172.21.15.39:0/1036981902 wait complete. 2023-07-21T18:33:07.559 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:07.056+0000 7f39f2408700 1 Processor -- start 2023-07-21T18:33:07.559 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:07.056+0000 7f39f2408700 1 -- start start 2023-07-21T18:33:07.560 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:07.056+0000 7f39f2408700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39ec1054f0 0x7f39ec1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:33:07.560 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:07.056+0000 7f39f2408700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39ec198f10 con 0x7f39ec1054f0 2023-07-21T18:33:07.560 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:07.056+0000 7f39ebfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39ec1054f0 0x7f39ec1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:33:07.560 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:07.056+0000 7f39ebfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39ec1054f0 0x7f39ec1989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53676/0 (socket says 172.21.15.39:53676) 2023-07-21T18:33:07.560 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:07.056+0000 7f39ebfff700 1 -- 172.21.15.39:0/2895364553 learned_addr learned my addr 172.21.15.39:0/2895364553 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:33:07.560 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:07.057+0000 7f39ebfff700 1 -- 172.21.15.39:0/2895364553 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39dc009550 con 0x7f39ec1054f0 2023-07-21T18:33:07.560 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:07.057+0000 7f39ebfff700 1 --2- 172.21.15.39:0/2895364553 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39ec1054f0 0x7f39ec1989d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f39ec106220 tx=0x7f39dc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:33:07.560 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:07.057+0000 7f39e97fa700 1 -- 172.21.15.39:0/2895364553 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f39dc011630 con 0x7f39ec1054f0 2023-07-21T18:33:07.561 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:07.057+0000 7f39e97fa700 1 -- 172.21.15.39:0/2895364553 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39dc011790 con 0x7f39ec1054f0 2023-07-21T18:33:07.561 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:07.057+0000 7f39e97fa700 1 -- 172.21.15.39:0/2895364553 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f39dc011a30 con 0x7f39ec1054f0 2023-07-21T18:33:07.561 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:07.057+0000 7f39f2408700 1 -- 172.21.15.39:0/2895364553 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39ec199110 con 0x7f39ec1054f0 2023-07-21T18:33:07.561 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:07.057+0000 7f39f2408700 1 -- 172.21.15.39:0/2895364553 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39ec199530 con 0x7f39ec1054f0 2023-07-21T18:33:07.561 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:07.057+0000 7f39e97fa700 1 -- 172.21.15.39:0/2895364553 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f39dc022070 con 0x7f39ec1054f0 2023-07-21T18:33:07.561 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:07.057+0000 7f39e97fa700 1 -- 172.21.15.39:0/2895364553 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f39dc01c450 con 0x7f39ec1054f0 2023-07-21T18:33:07.561 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:07.058+0000 7f39f2408700 1 -- 172.21.15.39:0/2895364553 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39ec192230 con 0x7f39ec1054f0 2023-07-21T18:33:07.562 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:07.061+0000 7f39e97fa700 1 -- 172.21.15.39:0/2895364553 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f39dc01b450 con 0x7f39ec1054f0 2023-07-21T18:33:07.562 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:07.193+0000 7f39f2408700 1 -- 172.21.15.39:0/2895364553 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f39ec02c850 con 0x7f39ec1054f0 2023-07-21T18:33:07.562 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:07.193+0000 7f39e97fa700 1 -- 172.21.15.39:0/2895364553 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7f39dc01b5f0 con 0x7f39ec1054f0 2023-07-21T18:33:07.562 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:07.194+0000 7f39f2408700 1 -- 172.21.15.39:0/2895364553 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39ec1054f0 msgr2=0x7f39ec1989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:33:07.562 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:07.194+0000 7f39f2408700 1 --2- 172.21.15.39:0/2895364553 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39ec1054f0 0x7f39ec1989d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f39ec106220 tx=0x7f39dc004750 comp rx=0 tx=0).stop 2023-07-21T18:33:07.562 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:07.194+0000 7f39f2408700 1 -- 172.21.15.39:0/2895364553 shutdown_connections 2023-07-21T18:33:07.562 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:07.194+0000 7f39f2408700 1 --2- 172.21.15.39:0/2895364553 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39ec1054f0 0x7f39ec1989d0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:33:07.562 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:07.194+0000 7f39f2408700 1 -- 172.21.15.39:0/2895364553 >> 172.21.15.39:0/2895364553 conn(0x7f39ec100b40 msgr2=0x7f39ec18cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:33:07.563 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:07.194+0000 7f39f2408700 1 -- 172.21.15.39:0/2895364553 shutdown_connections 2023-07-21T18:33:07.563 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:07.194+0000 7f39f2408700 1 -- 172.21.15.39:0/2895364553 wait complete. 2023-07-21T18:33:07.563 INFO:teuthology.orchestra.run.smithi039.stdout:mon is available 2023-07-21T18:33:07.563 INFO:teuthology.orchestra.run.smithi039.stdout:Assimilating anything we can from ceph.conf... 2023-07-21T18:33:07.813 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:07 smithi039 ceph-mon[95975]: mon.a is new leader, mons a in quorum (ranks 0) 2023-07-21T18:33:07.813 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:07 smithi039 ceph-mon[95975]: monmap e1: 1 mons at {a=[v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0]} removed_ranks: {} 2023-07-21T18:33:07.813 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:07 smithi039 ceph-mon[95975]: fsmap 2023-07-21T18:33:07.814 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:07 smithi039 ceph-mon[95975]: osdmap e1: 0 total, 0 up, 0 in 2023-07-21T18:33:07.814 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:07 smithi039 ceph-mon[95975]: mgrmap e1: no daemons active 2023-07-21T18:33:07.814 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:07 smithi039 ceph-mon[95975]: from='client.? 172.21.15.39:0/2895364553' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-07-21T18:33:08.498 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2023-07-21T18:33:08.506 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout [global] 2023-07-21T18:33:08.506 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout fsid = eaf8e25a-27f4-11ee-9b34-001a4aab830c 2023-07-21T18:33:08.506 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.39:3300,v1:172.21.15.39:6789] 2023-07-21T18:33:08.507 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-07-21T18:33:08.507 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-07-21T18:33:08.507 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-07-21T18:33:08.507 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-07-21T18:33:08.507 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2023-07-21T18:33:08.507 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout [mgr] 2023-07-21T18:33:08.507 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-07-21T18:33:08.508 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2023-07-21T18:33:08.508 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout [osd] 2023-07-21T18:33:08.508 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-07-21T18:33:08.508 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-07-21T18:33:08.508 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-07-21T18:33:08.508 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-07-21T18:33:08.508 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-07-21T18:33:08.508 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:07.996+0000 7f729120c700 1 Processor -- start 2023-07-21T18:33:08.509 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:07.996+0000 7f729120c700 1 -- start start 2023-07-21T18:33:08.509 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:07.997+0000 7f729120c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f728c104400 0x7f728c104810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:33:08.509 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:07.997+0000 7f729120c700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f728c104d50 con 0x7f728c104400 2023-07-21T18:33:08.509 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:07.997+0000 7f728ad9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f728c104400 0x7f728c104810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:33:08.509 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:07.997+0000 7f728ad9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f728c104400 0x7f728c104810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53678/0 (socket says 172.21.15.39:53678) 2023-07-21T18:33:08.509 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:07.997+0000 7f728ad9d700 1 -- 172.21.15.39:0/1594344960 learned_addr learned my addr 172.21.15.39:0/1594344960 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:33:08.509 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:07.997+0000 7f728ad9d700 1 -- 172.21.15.39:0/1594344960 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f728c104e90 con 0x7f728c104400 2023-07-21T18:33:08.510 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:07.998+0000 7f728ad9d700 1 --2- 172.21.15.39:0/1594344960 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f728c104400 0x7f728c104810 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f727c0098a0 tx=0x7f727c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f30b081ce67ea00 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:33:08.510 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:07.998+0000 7f7289d9b700 1 -- 172.21.15.39:0/1594344960 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f727c004160 con 0x7f728c104400 2023-07-21T18:33:08.510 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:07.998+0000 7f7289d9b700 1 -- 172.21.15.39:0/1594344960 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f727c0042c0 con 0x7f728c104400 2023-07-21T18:33:08.510 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:07.998+0000 7f7289d9b700 1 -- 172.21.15.39:0/1594344960 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f727c0045a0 con 0x7f728c104400 2023-07-21T18:33:08.510 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:07.998+0000 7f729120c700 1 -- 172.21.15.39:0/1594344960 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f728c104400 msgr2=0x7f728c104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:33:08.510 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:07.998+0000 7f729120c700 1 --2- 172.21.15.39:0/1594344960 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f728c104400 0x7f728c104810 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f727c0098a0 tx=0x7f727c009bb0 comp rx=0 tx=0).stop 2023-07-21T18:33:08.510 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:07.998+0000 7f729120c700 1 -- 172.21.15.39:0/1594344960 shutdown_connections 2023-07-21T18:33:08.511 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:07.998+0000 7f729120c700 1 --2- 172.21.15.39:0/1594344960 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f728c104400 0x7f728c104810 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:33:08.511 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:07.998+0000 7f729120c700 1 -- 172.21.15.39:0/1594344960 >> 172.21.15.39:0/1594344960 conn(0x7f728c0ffa50 msgr2=0x7f728c101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:33:08.511 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:07.998+0000 7f729120c700 1 -- 172.21.15.39:0/1594344960 shutdown_connections 2023-07-21T18:33:08.511 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:07.998+0000 7f729120c700 1 -- 172.21.15.39:0/1594344960 wait complete. 2023-07-21T18:33:08.511 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:07.999+0000 7f729120c700 1 Processor -- start 2023-07-21T18:33:08.511 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:07.999+0000 7f729120c700 1 -- start start 2023-07-21T18:33:08.511 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:08.000+0000 7f729120c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f728c104400 0x7f728c0739d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:33:08.511 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:08.000+0000 7f729120c700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f728c19b790 con 0x7f728c104400 2023-07-21T18:33:08.512 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:08.000+0000 7f728ad9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f728c104400 0x7f728c0739d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:33:08.512 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:08.000+0000 7f728ad9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f728c104400 0x7f728c0739d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53680/0 (socket says 172.21.15.39:53680) 2023-07-21T18:33:08.512 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:08.000+0000 7f728ad9d700 1 -- 172.21.15.39:0/1839851324 learned_addr learned my addr 172.21.15.39:0/1839851324 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:33:08.512 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:08.000+0000 7f728ad9d700 1 -- 172.21.15.39:0/1839851324 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f727c009550 con 0x7f728c104400 2023-07-21T18:33:08.512 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:08.000+0000 7f728ad9d700 1 --2- 172.21.15.39:0/1839851324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f728c104400 0x7f728c0739d0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f728c105130 tx=0x7f727c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:33:08.512 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:08.000+0000 7f7273fff700 1 -- 172.21.15.39:0/1839851324 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f727c011630 con 0x7f728c104400 2023-07-21T18:33:08.512 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:08.000+0000 7f7273fff700 1 -- 172.21.15.39:0/1839851324 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f727c011790 con 0x7f728c104400 2023-07-21T18:33:08.513 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:08.000+0000 7f729120c700 1 -- 172.21.15.39:0/1839851324 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f728c19b8d0 con 0x7f728c104400 2023-07-21T18:33:08.513 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:08.001+0000 7f7273fff700 1 -- 172.21.15.39:0/1839851324 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f727c011a30 con 0x7f728c104400 2023-07-21T18:33:08.513 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:08.001+0000 7f729120c700 1 -- 172.21.15.39:0/1839851324 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f728c19bcf0 con 0x7f728c104400 2023-07-21T18:33:08.513 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:08.001+0000 7f7273fff700 1 -- 172.21.15.39:0/1839851324 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f727c022070 con 0x7f728c104400 2023-07-21T18:33:08.513 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:08.001+0000 7f7273fff700 1 -- 172.21.15.39:0/1839851324 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f727c01c450 con 0x7f728c104400 2023-07-21T18:33:08.513 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:08.002+0000 7f729120c700 1 -- 172.21.15.39:0/1839851324 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f728c191120 con 0x7f728c104400 2023-07-21T18:33:08.513 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:08.004+0000 7f7273fff700 1 -- 172.21.15.39:0/1839851324 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f727c01b450 con 0x7f728c104400 2023-07-21T18:33:08.514 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:08.131+0000 7f729120c700 1 -- 172.21.15.39:0/1839851324 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f728c02bea0 con 0x7f728c104400 2023-07-21T18:33:08.514 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:08.134+0000 7f7273fff700 1 -- 172.21.15.39:0/1839851324 <== mon.0 v2:172.21.15.39:3300/0 7 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f727c01b5f0 con 0x7f728c104400 2023-07-21T18:33:08.514 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:08.134+0000 7f7273fff700 1 -- 172.21.15.39:0/1839851324 <== mon.0 v2:172.21.15.39:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7f727c024630 con 0x7f728c104400 2023-07-21T18:33:08.514 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:08.135+0000 7f729120c700 1 -- 172.21.15.39:0/1839851324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f728c104400 msgr2=0x7f728c0739d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:33:08.514 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:08.135+0000 7f729120c700 1 --2- 172.21.15.39:0/1839851324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f728c104400 0x7f728c0739d0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f728c105130 tx=0x7f727c004750 comp rx=0 tx=0).stop 2023-07-21T18:33:08.514 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:08.135+0000 7f729120c700 1 -- 172.21.15.39:0/1839851324 shutdown_connections 2023-07-21T18:33:08.514 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:08.135+0000 7f729120c700 1 --2- 172.21.15.39:0/1839851324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f728c104400 0x7f728c0739d0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:33:08.514 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:08.135+0000 7f729120c700 1 -- 172.21.15.39:0/1839851324 >> 172.21.15.39:0/1839851324 conn(0x7f728c0ffa50 msgr2=0x7f728c18be60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:33:08.515 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:08.135+0000 7f729120c700 1 -- 172.21.15.39:0/1839851324 shutdown_connections 2023-07-21T18:33:08.515 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:08.135+0000 7f729120c700 1 -- 172.21.15.39:0/1839851324 wait complete. 2023-07-21T18:33:08.515 INFO:teuthology.orchestra.run.smithi039.stdout:Generating new minimal ceph.conf... 2023-07-21T18:33:08.750 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:08 smithi039 ceph-mon[95975]: from='client.? 172.21.15.39:0/1839851324' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-07-21T18:33:08.751 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:08 smithi039 ceph-mon[95975]: from='client.? 172.21.15.39:0/1839851324' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2023-07-21T18:33:09.445 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:08.929+0000 7f76155d5700 1 Processor -- start 2023-07-21T18:33:09.446 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:08.929+0000 7f76155d5700 1 -- start start 2023-07-21T18:33:09.446 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:08.929+0000 7f76155d5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7610073ba0 0x7f761010a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:33:09.446 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:08.929+0000 7f76155d5700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7610105e80 con 0x7f7610073ba0 2023-07-21T18:33:09.446 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:08.929+0000 7f760effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7610073ba0 0x7f761010a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:33:09.446 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:08.929+0000 7f760effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7610073ba0 0x7f761010a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53682/0 (socket says 172.21.15.39:53682) 2023-07-21T18:33:09.446 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:08.929+0000 7f760effd700 1 -- 172.21.15.39:0/2067829639 learned_addr learned my addr 172.21.15.39:0/2067829639 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:33:09.447 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:08.930+0000 7f760effd700 1 -- 172.21.15.39:0/2067829639 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f761010a9d0 con 0x7f7610073ba0 2023-07-21T18:33:09.447 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:08.930+0000 7f760effd700 1 --2- 172.21.15.39:0/2067829639 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7610073ba0 0x7f761010a490 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f76040098a0 tx=0x7f7604009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db07c7f04b934e2e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:33:09.447 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:08.930+0000 7f760dffb700 1 -- 172.21.15.39:0/2067829639 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7604004160 con 0x7f7610073ba0 2023-07-21T18:33:09.447 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:08.930+0000 7f760dffb700 1 -- 172.21.15.39:0/2067829639 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f76040042c0 con 0x7f7610073ba0 2023-07-21T18:33:09.447 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:08.930+0000 7f760dffb700 1 -- 172.21.15.39:0/2067829639 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7604011a00 con 0x7f7610073ba0 2023-07-21T18:33:09.447 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:08.931+0000 7f76155d5700 1 -- 172.21.15.39:0/2067829639 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7610073ba0 msgr2=0x7f761010a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:33:09.447 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:08.931+0000 7f76155d5700 1 --2- 172.21.15.39:0/2067829639 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7610073ba0 0x7f761010a490 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f76040098a0 tx=0x7f7604009bb0 comp rx=0 tx=0).stop 2023-07-21T18:33:09.448 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:08.931+0000 7f76155d5700 1 -- 172.21.15.39:0/2067829639 shutdown_connections 2023-07-21T18:33:09.448 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:08.931+0000 7f76155d5700 1 --2- 172.21.15.39:0/2067829639 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7610073ba0 0x7f761010a490 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:33:09.448 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:08.931+0000 7f76155d5700 1 -- 172.21.15.39:0/2067829639 >> 172.21.15.39:0/2067829639 conn(0x7f76100ffa40 msgr2=0x7f7610101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:33:09.448 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:08.931+0000 7f76155d5700 1 -- 172.21.15.39:0/2067829639 shutdown_connections 2023-07-21T18:33:09.448 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:08.931+0000 7f76155d5700 1 -- 172.21.15.39:0/2067829639 wait complete. 2023-07-21T18:33:09.448 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:08.931+0000 7f76155d5700 1 Processor -- start 2023-07-21T18:33:09.448 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:08.931+0000 7f76155d5700 1 -- start start 2023-07-21T18:33:09.449 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:08.932+0000 7f76155d5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f761019b470 0x7f761019b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:33:09.449 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:08.932+0000 7f76155d5700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f761019bdc0 con 0x7f761019b470 2023-07-21T18:33:09.449 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:08.932+0000 7f760effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f761019b470 0x7f761019b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:33:09.449 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:08.932+0000 7f760effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f761019b470 0x7f761019b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53684/0 (socket says 172.21.15.39:53684) 2023-07-21T18:33:09.449 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:08.932+0000 7f760effd700 1 -- 172.21.15.39:0/2383328051 learned_addr learned my addr 172.21.15.39:0/2383328051 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:33:09.449 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:08.932+0000 7f760effd700 1 -- 172.21.15.39:0/2383328051 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7604009550 con 0x7f761019b470 2023-07-21T18:33:09.449 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:08.932+0000 7f760effd700 1 --2- 172.21.15.39:0/2383328051 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f761019b470 0x7f761019b880 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7604009fb0 tx=0x7f7604011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:33:09.449 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:08.932+0000 7f75f7fff700 1 -- 172.21.15.39:0/2383328051 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7604003a40 con 0x7f761019b470 2023-07-21T18:33:09.450 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:08.932+0000 7f76155d5700 1 -- 172.21.15.39:0/2383328051 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f761019bfc0 con 0x7f761019b470 2023-07-21T18:33:09.450 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:08.932+0000 7f75f7fff700 1 -- 172.21.15.39:0/2383328051 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7604003ba0 con 0x7f761019b470 2023-07-21T18:33:09.450 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:08.932+0000 7f75f7fff700 1 -- 172.21.15.39:0/2383328051 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f760401a750 con 0x7f761019b470 2023-07-21T18:33:09.450 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:08.932+0000 7f76155d5700 1 -- 172.21.15.39:0/2383328051 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f761019ec20 con 0x7f761019b470 2023-07-21T18:33:09.450 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:08.933+0000 7f75f7fff700 1 -- 172.21.15.39:0/2383328051 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f760401acd0 con 0x7f761019b470 2023-07-21T18:33:09.450 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:08.933+0000 7f75f7fff700 1 -- 172.21.15.39:0/2383328051 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f76040193c0 con 0x7f761019b470 2023-07-21T18:33:09.450 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:08.934+0000 7f76155d5700 1 -- 172.21.15.39:0/2383328051 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7610061b40 con 0x7f761019b470 2023-07-21T18:33:09.450 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:08.936+0000 7f75f7fff700 1 -- 172.21.15.39:0/2383328051 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f760402de40 con 0x7f761019b470 2023-07-21T18:33:09.451 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:09.062+0000 7f76155d5700 1 -- 172.21.15.39:0/2383328051 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f7610073ba0 con 0x7f761019b470 2023-07-21T18:33:09.451 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:09.062+0000 7f75f7fff700 1 -- 172.21.15.39:0/2383328051 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f7604024b60 con 0x7f761019b470 2023-07-21T18:33:09.451 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:09.063+0000 7f76155d5700 1 -- 172.21.15.39:0/2383328051 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f761019b470 msgr2=0x7f761019b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:33:09.451 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:09.063+0000 7f76155d5700 1 --2- 172.21.15.39:0/2383328051 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f761019b470 0x7f761019b880 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7604009fb0 tx=0x7f7604011ad0 comp rx=0 tx=0).stop 2023-07-21T18:33:09.451 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:09.063+0000 7f76155d5700 1 -- 172.21.15.39:0/2383328051 shutdown_connections 2023-07-21T18:33:09.451 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:09.063+0000 7f76155d5700 1 --2- 172.21.15.39:0/2383328051 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f761019b470 0x7f761019b880 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:33:09.451 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:09.063+0000 7f76155d5700 1 -- 172.21.15.39:0/2383328051 >> 172.21.15.39:0/2383328051 conn(0x7f76100ffa40 msgr2=0x7f7610101110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:33:09.452 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:09.063+0000 7f76155d5700 1 -- 172.21.15.39:0/2383328051 shutdown_connections 2023-07-21T18:33:09.452 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:09.063+0000 7f76155d5700 1 -- 172.21.15.39:0/2383328051 wait complete. 2023-07-21T18:33:09.452 INFO:teuthology.orchestra.run.smithi039.stdout:Restarting the monitor... 2023-07-21T18:33:09.700 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:09 smithi039 systemd[1]: Stopping Ceph mon.a for eaf8e25a-27f4-11ee-9b34-001a4aab830c... 2023-07-21T18:33:09.700 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:09 smithi039 ceph-mon[95975]: from='client.? 172.21.15.39:0/2383328051' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:33:10.050 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:09 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mon-a[95951]: 2023-07-21T18:33:09.734+0000 7fc27cfb3700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T18:33:10.050 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:09 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mon-a[95951]: 2023-07-21T18:33:09.734+0000 7fc27cfb3700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-07-21T18:33:10.370 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 bash[96315]: ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mon-a 2023-07-21T18:33:10.658 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 systemd[1]: ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@mon.a.service: Succeeded. 2023-07-21T18:33:10.658 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 systemd[1]: Stopped Ceph mon.a for eaf8e25a-27f4-11ee-9b34-001a4aab830c. 2023-07-21T18:33:10.658 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 systemd[1]: Starting Ceph mon.a for eaf8e25a-27f4-11ee-9b34-001a4aab830c... 2023-07-21T18:33:10.931 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 podman[96449]: 2023-07-21T18:33:10.931 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: set uid:gid to 167:167 (ceph:ceph) 2023-07-21T18:33:10.931 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: ceph version 18.0.0-4746-g81cf785a (81cf785aa7781c4e08381c540f890cb9bc987ef9) reef (dev), process ceph-mon, pid 7 2023-07-21T18:33:10.931 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: pidfile_write: ignore empty --pid-file 2023-07-21T18:33:10.931 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: load: jerasure load: lrc 2023-07-21T18:33:10.932 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: RocksDB version: 7.9.2 2023-07-21T18:33:10.932 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Git sha 0 2023-07-21T18:33:10.932 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Compile date 2023-07-17 23:00:14 2023-07-21T18:33:10.932 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: DB SUMMARY 2023-07-21T18:33:10.932 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: DB Session ID: M8DZ9URL3KNLTACT6C46 2023-07-21T18:33:10.932 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: CURRENT file: CURRENT 2023-07-21T18:33:10.932 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: IDENTITY file: IDENTITY 2023-07-21T18:33:10.933 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2023-07-21T18:33:10.933 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2023-07-21T18:33:10.933 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 100055 ; 2023-07-21T18:33:10.933 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.error_if_exists: 0 2023-07-21T18:33:10.933 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.create_if_missing: 0 2023-07-21T18:33:10.933 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.paranoid_checks: 1 2023-07-21T18:33:10.933 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.flush_verify_memtable_count: 1 2023-07-21T18:33:10.933 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-07-21T18:33:10.934 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-07-21T18:33:10.934 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.env: 0x5619df784720 2023-07-21T18:33:10.934 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.fs: PosixFileSystem 2023-07-21T18:33:10.934 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.info_log: 0x5619e09e5280 2023-07-21T18:33:10.934 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.max_file_opening_threads: 16 2023-07-21T18:33:10.934 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.statistics: (nil) 2023-07-21T18:33:10.934 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.use_fsync: 0 2023-07-21T18:33:10.935 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.max_log_file_size: 0 2023-07-21T18:33:10.935 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-07-21T18:33:10.935 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.log_file_time_to_roll: 0 2023-07-21T18:33:10.935 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.keep_log_file_num: 1000 2023-07-21T18:33:10.935 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.recycle_log_file_num: 0 2023-07-21T18:33:10.935 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.allow_fallocate: 1 2023-07-21T18:33:10.935 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.allow_mmap_reads: 0 2023-07-21T18:33:10.935 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.allow_mmap_writes: 0 2023-07-21T18:33:10.936 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.use_direct_reads: 0 2023-07-21T18:33:10.936 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-07-21T18:33:10.936 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.create_missing_column_families: 0 2023-07-21T18:33:10.936 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.db_log_dir: 2023-07-21T18:33:10.936 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.wal_dir: 2023-07-21T18:33:10.936 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.table_cache_numshardbits: 6 2023-07-21T18:33:10.936 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.WAL_ttl_seconds: 0 2023-07-21T18:33:10.937 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.WAL_size_limit_MB: 0 2023-07-21T18:33:10.937 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-07-21T18:33:10.937 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-07-21T18:33:10.937 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.is_fd_close_on_exec: 1 2023-07-21T18:33:10.937 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.advise_random_on_open: 1 2023-07-21T18:33:10.937 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.db_write_buffer_size: 0 2023-07-21T18:33:10.937 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.write_buffer_manager: 0x5619e09063c0 2023-07-21T18:33:10.937 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-07-21T18:33:10.938 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-07-21T18:33:10.938 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.use_adaptive_mutex: 0 2023-07-21T18:33:10.938 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.rate_limiter: (nil) 2023-07-21T18:33:10.938 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-07-21T18:33:10.938 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.wal_recovery_mode: 2 2023-07-21T18:33:10.938 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.enable_thread_tracking: 0 2023-07-21T18:33:10.938 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.enable_pipelined_write: 0 2023-07-21T18:33:10.939 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.unordered_write: 0 2023-07-21T18:33:10.939 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-07-21T18:33:10.939 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-07-21T18:33:10.939 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-07-21T18:33:10.939 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-07-21T18:33:10.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.row_cache: None 2023-07-21T18:33:10.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.wal_filter: None 2023-07-21T18:33:10.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-07-21T18:33:10.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.allow_ingest_behind: 0 2023-07-21T18:33:10.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.two_write_queues: 0 2023-07-21T18:33:10.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.manual_wal_flush: 0 2023-07-21T18:33:10.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.wal_compression: 0 2023-07-21T18:33:10.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.atomic_flush: 0 2023-07-21T18:33:10.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-07-21T18:33:10.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.persist_stats_to_disk: 0 2023-07-21T18:33:10.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.write_dbid_to_manifest: 0 2023-07-21T18:33:10.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.log_readahead_size: 0 2023-07-21T18:33:10.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-07-21T18:33:10.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.best_efforts_recovery: 0 2023-07-21T18:33:10.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-07-21T18:33:10.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-07-21T18:33:10.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.allow_data_in_errors: 0 2023-07-21T18:33:10.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.db_host_id: __hostname__ 2023-07-21T18:33:10.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.enforce_single_del_contracts: true 2023-07-21T18:33:10.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.max_background_jobs: 2 2023-07-21T18:33:10.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.max_background_compactions: -1 2023-07-21T18:33:10.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.max_subcompactions: 1 2023-07-21T18:33:10.943 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-07-21T18:33:10.943 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-07-21T18:33:10.943 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.delayed_write_rate : 16777216 2023-07-21T18:33:10.943 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.max_total_wal_size: 0 2023-07-21T18:33:10.943 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-07-21T18:33:10.943 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.stats_dump_period_sec: 600 2023-07-21T18:33:10.943 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.stats_persist_period_sec: 600 2023-07-21T18:33:10.943 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-07-21T18:33:10.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.max_open_files: -1 2023-07-21T18:33:10.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.bytes_per_sync: 0 2023-07-21T18:33:10.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.wal_bytes_per_sync: 0 2023-07-21T18:33:10.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.strict_bytes_per_sync: 0 2023-07-21T18:33:10.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.compaction_readahead_size: 0 2023-07-21T18:33:10.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.max_background_flushes: -1 2023-07-21T18:33:10.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Compression algorithms supported: 2023-07-21T18:33:10.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-07-21T18:33:10.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: kZSTD supported: 0 2023-07-21T18:33:10.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: kXpressCompression supported: 0 2023-07-21T18:33:10.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: kLZ4HCCompression supported: 1 2023-07-21T18:33:10.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: kZlibCompression supported: 1 2023-07-21T18:33:10.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: kSnappyCompression supported: 1 2023-07-21T18:33:10.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: kLZ4Compression supported: 1 2023-07-21T18:33:10.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: kBZip2Compression supported: 0 2023-07-21T18:33:10.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Fast CRC32 supported: Supported on x86 2023-07-21T18:33:10.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: DMutex implementation: pthread_mutex_t 2023-07-21T18:33:10.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-07-21T18:33:10.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-07-21T18:33:10.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-07-21T18:33:10.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.merge_operator: 2023-07-21T18:33:10.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.compaction_filter: None 2023-07-21T18:33:10.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.compaction_filter_factory: None 2023-07-21T18:33:10.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.sst_partitioner_factory: None 2023-07-21T18:33:10.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.memtable_factory: SkipListFactory 2023-07-21T18:33:10.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.table_factory: BlockBasedTable 2023-07-21T18:33:10.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5619e09e5380) 2023-07-21T18:33:10.947 INFO:journalctl@ceph.mon.a.smithi039.stdout: cache_index_and_filter_blocks: 1 2023-07-21T18:33:10.947 INFO:journalctl@ceph.mon.a.smithi039.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-07-21T18:33:10.947 INFO:journalctl@ceph.mon.a.smithi039.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-07-21T18:33:10.947 INFO:journalctl@ceph.mon.a.smithi039.stdout: pin_top_level_index_and_filter: 1 2023-07-21T18:33:10.948 INFO:journalctl@ceph.mon.a.smithi039.stdout: index_type: 0 2023-07-21T18:33:10.948 INFO:journalctl@ceph.mon.a.smithi039.stdout: data_block_index_type: 0 2023-07-21T18:33:10.948 INFO:journalctl@ceph.mon.a.smithi039.stdout: index_shortening: 1 2023-07-21T18:33:10.948 INFO:journalctl@ceph.mon.a.smithi039.stdout: data_block_hash_table_util_ratio: 0.750000 2023-07-21T18:33:10.948 INFO:journalctl@ceph.mon.a.smithi039.stdout: checksum: 4 2023-07-21T18:33:10.948 INFO:journalctl@ceph.mon.a.smithi039.stdout: no_block_cache: 0 2023-07-21T18:33:10.948 INFO:journalctl@ceph.mon.a.smithi039.stdout: block_cache: 0x5619e09a51f0 2023-07-21T18:33:10.948 INFO:journalctl@ceph.mon.a.smithi039.stdout: block_cache_name: BinnedLRUCache 2023-07-21T18:33:10.949 INFO:journalctl@ceph.mon.a.smithi039.stdout: block_cache_options: 2023-07-21T18:33:10.949 INFO:journalctl@ceph.mon.a.smithi039.stdout: capacity : 536870912 2023-07-21T18:33:10.949 INFO:journalctl@ceph.mon.a.smithi039.stdout: num_shard_bits : 4 2023-07-21T18:33:10.949 INFO:journalctl@ceph.mon.a.smithi039.stdout: strict_capacity_limit : 0 2023-07-21T18:33:10.949 INFO:journalctl@ceph.mon.a.smithi039.stdout: high_pri_pool_ratio: 0.000 2023-07-21T18:33:10.949 INFO:journalctl@ceph.mon.a.smithi039.stdout: block_cache_compressed: (nil) 2023-07-21T18:33:10.949 INFO:journalctl@ceph.mon.a.smithi039.stdout: persistent_cache: (nil) 2023-07-21T18:33:10.950 INFO:journalctl@ceph.mon.a.smithi039.stdout: block_size: 4096 2023-07-21T18:33:10.950 INFO:journalctl@ceph.mon.a.smithi039.stdout: block_size_deviation: 10 2023-07-21T18:33:10.950 INFO:journalctl@ceph.mon.a.smithi039.stdout: block_restart_interval: 16 2023-07-21T18:33:10.950 INFO:journalctl@ceph.mon.a.smithi039.stdout: index_block_restart_interval: 1 2023-07-21T18:33:10.950 INFO:journalctl@ceph.mon.a.smithi039.stdout: metadata_block_size: 4096 2023-07-21T18:33:10.951 INFO:journalctl@ceph.mon.a.smithi039.stdout: partition_filters: 0 2023-07-21T18:33:10.951 INFO:journalctl@ceph.mon.a.smithi039.stdout: use_delta_encoding: 1 2023-07-21T18:33:10.951 INFO:journalctl@ceph.mon.a.smithi039.stdout: filter_policy: bloomfilter 2023-07-21T18:33:10.951 INFO:journalctl@ceph.mon.a.smithi039.stdout: whole_key_filtering: 1 2023-07-21T18:33:10.952 INFO:journalctl@ceph.mon.a.smithi039.stdout: verify_compression: 0 2023-07-21T18:33:10.952 INFO:journalctl@ceph.mon.a.smithi039.stdout: read_amp_bytes_per_bit: 0 2023-07-21T18:33:10.952 INFO:journalctl@ceph.mon.a.smithi039.stdout: format_version: 5 2023-07-21T18:33:10.953 INFO:journalctl@ceph.mon.a.smithi039.stdout: enable_index_compression: 1 2023-07-21T18:33:10.953 INFO:journalctl@ceph.mon.a.smithi039.stdout: block_align: 0 2023-07-21T18:33:10.953 INFO:journalctl@ceph.mon.a.smithi039.stdout: max_auto_readahead_size: 262144 2023-07-21T18:33:10.953 INFO:journalctl@ceph.mon.a.smithi039.stdout: prepopulate_block_cache: 0 2023-07-21T18:33:10.954 INFO:journalctl@ceph.mon.a.smithi039.stdout: initial_auto_readahead_size: 8192 2023-07-21T18:33:10.954 INFO:journalctl@ceph.mon.a.smithi039.stdout: num_file_reads_for_auto_readahead: 2 2023-07-21T18:33:10.954 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.write_buffer_size: 33554432 2023-07-21T18:33:10.955 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.max_write_buffer_number: 2 2023-07-21T18:33:10.955 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.compression: NoCompression 2023-07-21T18:33:10.955 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.bottommost_compression: Disabled 2023-07-21T18:33:10.955 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.prefix_extractor: nullptr 2023-07-21T18:33:10.956 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-07-21T18:33:10.956 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.num_levels: 7 2023-07-21T18:33:10.956 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-07-21T18:33:10.957 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-07-21T18:33:10.957 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-07-21T18:33:10.957 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-07-21T18:33:10.957 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-07-21T18:33:10.958 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-07-21T18:33:10.958 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-07-21T18:33:10.958 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-07-21T18:33:10.959 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-07-21T18:33:10.960 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-07-21T18:33:10.960 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-07-21T18:33:10.960 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-07-21T18:33:10.961 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.compression_opts.window_bits: -14 2023-07-21T18:33:10.961 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.compression_opts.level: 32767 2023-07-21T18:33:10.961 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.compression_opts.strategy: 0 2023-07-21T18:33:10.962 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-07-21T18:33:10.962 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-07-21T18:33:10.962 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-07-21T18:33:10.962 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-07-21T18:33:10.962 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.compression_opts.enabled: false 2023-07-21T18:33:10.962 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-07-21T18:33:10.962 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-07-21T18:33:10.963 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-07-21T18:33:10.963 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-07-21T18:33:10.963 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.target_file_size_base: 67108864 2023-07-21T18:33:10.963 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.target_file_size_multiplier: 1 2023-07-21T18:33:10.963 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-07-21T18:33:10.963 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-07-21T18:33:10.963 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-07-21T18:33:10.964 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-07-21T18:33:10.964 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-07-21T18:33:10.964 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-07-21T18:33:10.964 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-07-21T18:33:10.964 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-07-21T18:33:10.964 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-07-21T18:33:10.964 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-07-21T18:33:10.965 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-07-21T18:33:10.965 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-07-21T18:33:10.965 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-07-21T18:33:10.965 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.arena_block_size: 1048576 2023-07-21T18:33:10.965 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-07-21T18:33:10.965 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-07-21T18:33:10.965 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.disable_auto_compactions: 0 2023-07-21T18:33:10.965 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-07-21T18:33:10.966 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-07-21T18:33:10.966 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-07-21T18:33:10.966 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-07-21T18:33:10.966 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-07-21T18:33:10.966 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-07-21T18:33:10.966 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-07-21T18:33:10.966 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-07-21T18:33:10.966 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-07-21T18:33:10.967 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-07-21T18:33:10.967 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.table_properties_collectors: 2023-07-21T18:33:10.967 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.inplace_update_support: 0 2023-07-21T18:33:10.967 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.inplace_update_num_locks: 10000 2023-07-21T18:33:10.967 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-07-21T18:33:10.967 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-07-21T18:33:10.967 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.memtable_huge_page_size: 0 2023-07-21T18:33:10.968 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.bloom_locality: 0 2023-07-21T18:33:10.968 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.max_successive_merges: 0 2023-07-21T18:33:10.968 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.optimize_filters_for_hits: 0 2023-07-21T18:33:10.968 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.paranoid_file_checks: 0 2023-07-21T18:33:10.968 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.force_consistency_checks: 1 2023-07-21T18:33:10.968 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.report_bg_io_stats: 0 2023-07-21T18:33:10.968 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.ttl: 2592000 2023-07-21T18:33:10.968 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.periodic_compaction_seconds: 0 2023-07-21T18:33:10.969 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-07-21T18:33:10.969 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-07-21T18:33:10.969 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.enable_blob_files: false 2023-07-21T18:33:10.969 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.min_blob_size: 0 2023-07-21T18:33:10.969 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.blob_file_size: 268435456 2023-07-21T18:33:10.969 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.blob_compression_type: NoCompression 2023-07-21T18:33:10.969 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.enable_blob_garbage_collection: false 2023-07-21T18:33:10.970 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-07-21T18:33:10.970 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-07-21T18:33:10.970 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-07-21T18:33:10.970 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.blob_file_starting_level: 0 2023-07-21T18:33:10.970 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-07-21T18:33:10.970 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2023-07-21T18:33:10.970 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2023-07-21T18:33:10.970 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: edab150c-d9af-4da4-a38f-7ddb0357754a 2023-07-21T18:33:10.971 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689964390820778, "job": 1, "event": "recovery_started", "wal_files": [9]} 2023-07-21T18:33:10.971 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2023-07-21T18:33:10.971 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689964390822447, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 95167, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 320, "table_properties": {"data_size": 93155, "index_size": 294, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13991, "raw_average_key_size": 50, "raw_value_size": 85545, "raw_average_value_size": 306, "num_data_blocks": 11, "num_entries": 279, "num_filter_entries": 279, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1689964390, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "edab150c-d9af-4da4-a38f-7ddb0357754a", "db_session_id": "M8DZ9URL3KNLTACT6C46", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2023-07-21T18:33:10.971 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689964390822550, "job": 1, "event": "recovery_finished"} 2023-07-21T18:33:10.971 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2023-07-21T18:33:10.971 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-07-21T18:33:10.972 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5619e0a3e000 2023-07-21T18:33:10.972 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: DB pointer 0x5619e0a2a000 2023-07-21T18:33:10.972 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-07-21T18:33:10.972 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-07-21T18:33:10.972 INFO:journalctl@ceph.mon.a.smithi039.stdout: ** DB Stats ** 2023-07-21T18:33:10.973 INFO:journalctl@ceph.mon.a.smithi039.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-21T18:33:10.973 INFO:journalctl@ceph.mon.a.smithi039.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-07-21T18:33:10.973 INFO:journalctl@ceph.mon.a.smithi039.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-21T18:33:10.973 INFO:journalctl@ceph.mon.a.smithi039.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-21T18:33:10.973 INFO:journalctl@ceph.mon.a.smithi039.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-07-21T18:33:10.973 INFO:journalctl@ceph.mon.a.smithi039.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-21T18:33:10.973 INFO:journalctl@ceph.mon.a.smithi039.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-21T18:33:10.973 INFO:journalctl@ceph.mon.a.smithi039.stdout: 2023-07-21T18:33:10.974 INFO:journalctl@ceph.mon.a.smithi039.stdout: ** Compaction Stats [default] ** 2023-07-21T18:33:10.974 INFO:journalctl@ceph.mon.a.smithi039.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-21T18:33:10.974 INFO:journalctl@ceph.mon.a.smithi039.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-07-21T18:33:10.974 INFO:journalctl@ceph.mon.a.smithi039.stdout: L0 2/0 94.77 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 90.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-21T18:33:10.974 INFO:journalctl@ceph.mon.a.smithi039.stdout: Sum 2/0 94.77 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 90.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-21T18:33:10.974 INFO:journalctl@ceph.mon.a.smithi039.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 90.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-21T18:33:10.974 INFO:journalctl@ceph.mon.a.smithi039.stdout: 2023-07-21T18:33:10.975 INFO:journalctl@ceph.mon.a.smithi039.stdout: ** Compaction Stats [default] ** 2023-07-21T18:33:10.975 INFO:journalctl@ceph.mon.a.smithi039.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-21T18:33:10.975 INFO:journalctl@ceph.mon.a.smithi039.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-07-21T18:33:10.975 INFO:journalctl@ceph.mon.a.smithi039.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 90.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-21T18:33:10.975 INFO:journalctl@ceph.mon.a.smithi039.stdout: 2023-07-21T18:33:10.975 INFO:journalctl@ceph.mon.a.smithi039.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-07-21T18:33:10.975 INFO:journalctl@ceph.mon.a.smithi039.stdout: 2023-07-21T18:33:10.975 INFO:journalctl@ceph.mon.a.smithi039.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-21T18:33:10.976 INFO:journalctl@ceph.mon.a.smithi039.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-07-21T18:33:10.976 INFO:journalctl@ceph.mon.a.smithi039.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-07-21T18:33:10.976 INFO:journalctl@ceph.mon.a.smithi039.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-07-21T18:33:10.976 INFO:journalctl@ceph.mon.a.smithi039.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-07-21T18:33:10.976 INFO:journalctl@ceph.mon.a.smithi039.stdout: AddFile(Keys): cumulative 0, interval 0 2023-07-21T18:33:10.976 INFO:journalctl@ceph.mon.a.smithi039.stdout: Cumulative compaction: 0.00 GB write, 18.29 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-21T18:33:10.976 INFO:journalctl@ceph.mon.a.smithi039.stdout: Interval compaction: 0.00 GB write, 18.29 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-21T18:33:10.976 INFO:journalctl@ceph.mon.a.smithi039.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-07-21T18:33:10.977 INFO:journalctl@ceph.mon.a.smithi039.stdout: Block cache BinnedLRUCache@0x5619e09a51f0#7 capacity: 512.00 MB usage: 1.39 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2023-07-21T18:33:10.977 INFO:journalctl@ceph.mon.a.smithi039.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2023-07-21T18:33:10.977 INFO:journalctl@ceph.mon.a.smithi039.stdout: 2023-07-21T18:33:10.977 INFO:journalctl@ceph.mon.a.smithi039.stdout: ** File Read Latency Histogram By Level [default] ** 2023-07-21T18:33:10.977 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: starting mon.a rank 0 at public addrs [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] at bind addrs [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c 2023-07-21T18:33:10.977 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: mon.a@-1(???) e1 preinit fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c 2023-07-21T18:33:10.977 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: mon.a@-1(???).mds e1 new map 2023-07-21T18:33:10.978 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: mon.a@-1(???).mds e1 print_map 2023-07-21T18:33:10.978 INFO:journalctl@ceph.mon.a.smithi039.stdout: e1 2023-07-21T18:33:10.978 INFO:journalctl@ceph.mon.a.smithi039.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-07-21T18:33:10.978 INFO:journalctl@ceph.mon.a.smithi039.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-21T18:33:10.978 INFO:journalctl@ceph.mon.a.smithi039.stdout: legacy client fscid: -1 2023-07-21T18:33:10.978 INFO:journalctl@ceph.mon.a.smithi039.stdout: 2023-07-21T18:33:10.978 INFO:journalctl@ceph.mon.a.smithi039.stdout: No filesystems configured 2023-07-21T18:33:10.979 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2023-07-21T18:33:10.979 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-07-21T18:33:10.979 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-07-21T18:33:10.979 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-07-21T18:33:10.979 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2023-07-21T18:33:10.979 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: expand_channel_meta expand map: {default=false} 2023-07-21T18:33:10.979 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: expand_channel_meta from 'false' to 'false' 2023-07-21T18:33:10.979 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: expand_channel_meta expanded map: {default=false} 2023-07-21T18:33:10.980 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: expand_channel_meta expand map: {default=info} 2023-07-21T18:33:10.980 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: expand_channel_meta from 'info' to 'info' 2023-07-21T18:33:10.980 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: expand_channel_meta expanded map: {default=info} 2023-07-21T18:33:10.980 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: expand_channel_meta expand map: {default=daemon} 2023-07-21T18:33:10.980 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: expand_channel_meta from 'daemon' to 'daemon' 2023-07-21T18:33:10.980 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: expand_channel_meta expanded map: {default=daemon} 2023-07-21T18:33:10.980 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: expand_channel_meta expand map: {default=debug} 2023-07-21T18:33:10.981 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: expand_channel_meta from 'debug' to 'debug' 2023-07-21T18:33:10.981 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: expand_channel_meta expanded map: {default=debug} 2023-07-21T18:33:10.981 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: mon.a is new leader, mons a in quorum (ranks 0) 2023-07-21T18:33:10.981 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: monmap e1: 1 mons at {a=[v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0]} removed_ranks: {} 2023-07-21T18:33:10.981 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: fsmap 2023-07-21T18:33:10.981 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: osdmap e1: 0 total, 0 up, 0 in 2023-07-21T18:33:10.981 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:10 smithi039 ceph-mon[96489]: mgrmap e1: no daemons active 2023-07-21T18:33:11.157 INFO:teuthology.orchestra.run.smithi039.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2023-07-21T18:33:11.199 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:11 smithi039 bash[96449]: a136b64d7de779c7f2d8da09c5ab406cd70678e195fdcbfbfabefbda2a597c96 2023-07-21T18:33:11.200 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:11 smithi039 systemd[1]: Started Ceph mon.a for eaf8e25a-27f4-11ee-9b34-001a4aab830c. 2023-07-21T18:33:14.949 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:14 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/2723926707' entity='client.admin' 2023-07-21T18:33:15.920 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:13.341+0000 7f6322296700 1 Processor -- start 2023-07-21T18:33:15.920 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:13.341+0000 7f6322296700 1 -- start start 2023-07-21T18:33:15.920 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:13.341+0000 7f6322296700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f631c105500 0x7f631c105910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:33:15.921 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:13.341+0000 7f6322296700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f631c105e50 con 0x7f631c105500 2023-07-21T18:33:15.921 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:13.342+0000 7f631bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f631c105500 0x7f631c105910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:33:15.921 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:13.342+0000 7f631bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f631c105500 0x7f631c105910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53686/0 (socket says 172.21.15.39:53686) 2023-07-21T18:33:15.921 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:13.342+0000 7f631bfff700 1 -- 172.21.15.39:0/2700994910 learned_addr learned my addr 172.21.15.39:0/2700994910 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:33:15.921 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:13.342+0000 7f631bfff700 1 -- 172.21.15.39:0/2700994910 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f631c105f90 con 0x7f631c105500 2023-07-21T18:33:15.921 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:13.342+0000 7f631bfff700 1 --2- 172.21.15.39:0/2700994910 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f631c105500 0x7f631c105910 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f630c0098a0 tx=0x7f630c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c238b2d827eb9226 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:33:15.921 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:13.342+0000 7f631affd700 1 -- 172.21.15.39:0/2700994910 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f630c004160 con 0x7f631c105500 2023-07-21T18:33:15.922 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:13.343+0000 7f631affd700 1 -- 172.21.15.39:0/2700994910 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f630c0042c0 con 0x7f631c105500 2023-07-21T18:33:15.922 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:13.343+0000 7f631affd700 1 -- 172.21.15.39:0/2700994910 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f630c011a00 con 0x7f631c105500 2023-07-21T18:33:15.922 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:13.343+0000 7f6322296700 1 -- 172.21.15.39:0/2700994910 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f631c105500 msgr2=0x7f631c105910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:33:15.922 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:13.343+0000 7f6322296700 1 --2- 172.21.15.39:0/2700994910 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f631c105500 0x7f631c105910 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f630c0098a0 tx=0x7f630c009bb0 comp rx=0 tx=0).stop 2023-07-21T18:33:15.922 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:13.343+0000 7f6322296700 1 -- 172.21.15.39:0/2700994910 shutdown_connections 2023-07-21T18:33:15.922 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:13.343+0000 7f6322296700 1 --2- 172.21.15.39:0/2700994910 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f631c105500 0x7f631c105910 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:33:15.922 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:13.343+0000 7f6322296700 1 -- 172.21.15.39:0/2700994910 >> 172.21.15.39:0/2700994910 conn(0x7f631c100b70 msgr2=0x7f631c102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:33:15.922 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:13.343+0000 7f6322296700 1 -- 172.21.15.39:0/2700994910 shutdown_connections 2023-07-21T18:33:15.923 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:13.343+0000 7f6322296700 1 -- 172.21.15.39:0/2700994910 wait complete. 2023-07-21T18:33:15.923 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:13.344+0000 7f6322296700 1 Processor -- start 2023-07-21T18:33:15.923 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:13.344+0000 7f6322296700 1 -- start start 2023-07-21T18:33:15.923 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:13.345+0000 7f6322296700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f631c105500 0x7f631c199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:33:15.923 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:13.345+0000 7f6322296700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f631c1997d0 con 0x7f631c105500 2023-07-21T18:33:15.923 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:13.345+0000 7f631bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f631c105500 0x7f631c199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:33:15.923 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:13.345+0000 7f631bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f631c105500 0x7f631c199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53688/0 (socket says 172.21.15.39:53688) 2023-07-21T18:33:15.923 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:13.345+0000 7f631bfff700 1 -- 172.21.15.39:0/2723926707 learned_addr learned my addr 172.21.15.39:0/2723926707 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:33:15.924 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:13.345+0000 7f631bfff700 1 -- 172.21.15.39:0/2723926707 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f630c009550 con 0x7f631c105500 2023-07-21T18:33:15.924 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:13.345+0000 7f631bfff700 1 --2- 172.21.15.39:0/2723926707 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f631c105500 0x7f631c199290 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f630c009fb0 tx=0x7f630c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:33:15.924 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:13.345+0000 7f63197fa700 1 -- 172.21.15.39:0/2723926707 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f630c003a40 con 0x7f631c105500 2023-07-21T18:33:15.924 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:13.345+0000 7f63197fa700 1 -- 172.21.15.39:0/2723926707 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f630c003ba0 con 0x7f631c105500 2023-07-21T18:33:15.924 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:13.345+0000 7f6322296700 1 -- 172.21.15.39:0/2723926707 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f631c1999d0 con 0x7f631c105500 2023-07-21T18:33:15.924 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:13.345+0000 7f63197fa700 1 -- 172.21.15.39:0/2723926707 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f630c01a750 con 0x7f631c105500 2023-07-21T18:33:15.924 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:13.346+0000 7f6322296700 1 -- 172.21.15.39:0/2723926707 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f631c199df0 con 0x7f631c105500 2023-07-21T18:33:15.925 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:13.346+0000 7f63197fa700 1 -- 172.21.15.39:0/2723926707 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f630c01a9d0 con 0x7f631c105500 2023-07-21T18:33:15.925 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:13.346+0000 7f63197fa700 1 -- 172.21.15.39:0/2723926707 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f630c02d9e0 con 0x7f631c105500 2023-07-21T18:33:15.925 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:13.347+0000 7f6322296700 1 -- 172.21.15.39:0/2723926707 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f631c061b40 con 0x7f631c105500 2023-07-21T18:33:15.925 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:13.349+0000 7f63197fa700 1 -- 172.21.15.39:0/2723926707 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f630c01adc0 con 0x7f631c105500 2023-07-21T18:33:15.925 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:13.477+0000 7f6322296700 1 -- 172.21.15.39:0/2723926707 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f631c1099a0 con 0x7f631c105500 2023-07-21T18:33:15.925 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:13.484+0000 7f63197fa700 1 -- 172.21.15.39:0/2723926707 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f630c0248d0 con 0x7f631c105500 2023-07-21T18:33:15.925 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:13.485+0000 7f6322296700 1 -- 172.21.15.39:0/2723926707 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f631c105500 msgr2=0x7f631c199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:33:15.925 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:13.485+0000 7f6322296700 1 --2- 172.21.15.39:0/2723926707 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f631c105500 0x7f631c199290 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f630c009fb0 tx=0x7f630c011ad0 comp rx=0 tx=0).stop 2023-07-21T18:33:15.926 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:13.485+0000 7f6322296700 1 -- 172.21.15.39:0/2723926707 shutdown_connections 2023-07-21T18:33:15.926 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:13.485+0000 7f6322296700 1 --2- 172.21.15.39:0/2723926707 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f631c105500 0x7f631c199290 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:33:15.926 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:13.485+0000 7f6322296700 1 -- 172.21.15.39:0/2723926707 >> 172.21.15.39:0/2723926707 conn(0x7f631c100b70 msgr2=0x7f631c102240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:33:15.926 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:13.485+0000 7f6322296700 1 -- 172.21.15.39:0/2723926707 shutdown_connections 2023-07-21T18:33:15.926 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:13.485+0000 7f6322296700 1 -- 172.21.15.39:0/2723926707 wait complete. 2023-07-21T18:33:15.927 INFO:teuthology.orchestra.run.smithi039.stdout:Wrote config to /etc/ceph/ceph.conf 2023-07-21T18:33:16.242 INFO:teuthology.orchestra.run.smithi039.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2023-07-21T18:33:16.242 INFO:teuthology.orchestra.run.smithi039.stdout:Creating mgr... 2023-07-21T18:33:16.243 INFO:teuthology.orchestra.run.smithi039.stdout:Verifying port 9283 ... 2023-07-21T18:33:16.243 INFO:teuthology.orchestra.run.smithi039.stdout:Verifying port 8765 ... 2023-07-21T18:33:16.957 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@mgr.x 2023-07-21T18:33:16.957 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Failed to reset failed state of unit ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@mgr.x.service: Unit ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@mgr.x.service not loaded. 2023-07-21T18:33:17.050 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c.target.wants/ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@mgr.x.service → /etc/systemd/system/ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@.service. 2023-07-21T18:33:17.199 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:17 smithi039 systemd[1]: Starting Ceph mgr.x for eaf8e25a-27f4-11ee-9b34-001a4aab830c... 2023-07-21T18:33:18.950 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:18 smithi039 podman[96719]: 2023-07-21T18:33:19.869 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-07-21T18:33:19.869 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout disabled 2023-07-21T18:33:19.877 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-07-21T18:33:19.877 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout inactive 2023-07-21T18:33:19.877 INFO:teuthology.orchestra.run.smithi039.stdout:firewalld.service is not enabled 2023-07-21T18:33:19.877 INFO:teuthology.orchestra.run.smithi039.stdout:Not possible to enable service . firewalld.service is not available 2023-07-21T18:33:19.886 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-07-21T18:33:19.886 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout disabled 2023-07-21T18:33:19.894 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-07-21T18:33:19.894 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout inactive 2023-07-21T18:33:19.894 INFO:teuthology.orchestra.run.smithi039.stdout:firewalld.service is not enabled 2023-07-21T18:33:19.894 INFO:teuthology.orchestra.run.smithi039.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2023-07-21T18:33:19.895 INFO:teuthology.orchestra.run.smithi039.stdout:Waiting for mgr to start... 2023-07-21T18:33:19.895 INFO:teuthology.orchestra.run.smithi039.stdout:Waiting for mgr... 2023-07-21T18:33:20.117 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:19 smithi039 bash[96719]: 9506db3d7e2883334ac3a53638ff2b3f17dd41c414fcb613e21e4d305d10496c 2023-07-21T18:33:20.117 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:19 smithi039 systemd[1]: Started Ceph mgr.x for eaf8e25a-27f4-11ee-9b34-001a4aab830c. 2023-07-21T18:33:20.117 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:19 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:33:19.907+0000 7ff7773f7200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-21T18:33:20.117 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:20 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:33:20.002+0000 7ff7773f7200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-21T18:33:20.449 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:20 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:33:20.117+0000 7ff7773f7200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-21T18:33:20.449 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:20 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:33:20.231+0000 7ff7773f7200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-21T18:33:20.844 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2023-07-21T18:33:20.845 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout { 2023-07-21T18:33:20.845 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "fsid": "eaf8e25a-27f4-11ee-9b34-001a4aab830c", 2023-07-21T18:33:20.846 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:20 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/2309480695' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-07-21T18:33:20.847 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "health": { 2023-07-21T18:33:20.847 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-07-21T18:33:20.847 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-07-21T18:33:20.847 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-07-21T18:33:20.847 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2023-07-21T18:33:20.847 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-07-21T18:33:20.848 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-07-21T18:33:20.848 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 0 2023-07-21T18:33:20.848 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2023-07-21T18:33:20.848 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-07-21T18:33:20.848 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:20 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:33:20.442+0000 7ff7773f7200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-21T18:33:20.849 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "a" 2023-07-21T18:33:20.850 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2023-07-21T18:33:20.850 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2023-07-21T18:33:20.850 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "monmap": { 2023-07-21T18:33:20.850 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T18:33:20.850 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-07-21T18:33:20.850 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-07-21T18:33:20.851 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2023-07-21T18:33:20.851 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-07-21T18:33:20.851 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T18:33:20.851 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-07-21T18:33:20.851 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-07-21T18:33:20.856 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-07-21T18:33:20.857 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-07-21T18:33:20.857 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-07-21T18:33:20.857 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-07-21T18:33:20.857 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2023-07-21T18:33:20.857 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-07-21T18:33:20.857 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-07-21T18:33:20.857 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-07-21T18:33:20.858 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-07-21T18:33:20.858 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-07-21T18:33:20.858 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-07-21T18:33:20.858 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-07-21T18:33:20.858 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-07-21T18:33:20.858 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-07-21T18:33:20.858 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2023-07-21T18:33:20.858 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-07-21T18:33:20.859 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T18:33:20.859 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-07-21T18:33:20.859 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-07-21T18:33:20.859 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2023-07-21T18:33:20.859 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-07-21T18:33:20.859 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "available": false, 2023-07-21T18:33:20.859 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-07-21T18:33:20.859 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "modules": [ 2023-07-21T18:33:20.860 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "iostat", 2023-07-21T18:33:20.860 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "nfs", 2023-07-21T18:33:20.860 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "restful" 2023-07-21T18:33:20.860 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2023-07-21T18:33:20.860 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-21T18:33:20.860 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2023-07-21T18:33:20.860 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-07-21T18:33:20.861 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T18:33:20.861 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "modified": "2023-07-21T18:33:06.505474+0000", 2023-07-21T18:33:20.861 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-21T18:33:20.861 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2023-07-21T18:33:20.861 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-07-21T18:33:20.861 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout } 2023-07-21T18:33:20.861 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:20.372+0000 7f5724117700 1 Processor -- start 2023-07-21T18:33:20.861 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:20.373+0000 7f5724117700 1 -- start start 2023-07-21T18:33:20.862 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:20.373+0000 7f5724117700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f571c104400 0x7f571c104810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:33:20.862 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:20.373+0000 7f5724117700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f571c104d50 con 0x7f571c104400 2023-07-21T18:33:20.862 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:20.373+0000 7f5721eb3700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f571c104400 0x7f571c104810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:33:20.862 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:20.373+0000 7f5721eb3700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f571c104400 0x7f571c104810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53694/0 (socket says 172.21.15.39:53694) 2023-07-21T18:33:20.862 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:20.373+0000 7f5721eb3700 1 -- 172.21.15.39:0/748159551 learned_addr learned my addr 172.21.15.39:0/748159551 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:33:20.862 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:20.374+0000 7f5721eb3700 1 -- 172.21.15.39:0/748159551 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f571c104e90 con 0x7f571c104400 2023-07-21T18:33:20.862 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:20.374+0000 7f5721eb3700 1 --2- 172.21.15.39:0/748159551 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f571c104400 0x7f571c104810 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f57180098a0 tx=0x7f5718009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1ebac89076cbe4b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:33:20.862 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:20.374+0000 7f5720eb1700 1 -- 172.21.15.39:0/748159551 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5718004160 con 0x7f571c104400 2023-07-21T18:33:20.863 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:20.374+0000 7f5720eb1700 1 -- 172.21.15.39:0/748159551 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f57180042c0 con 0x7f571c104400 2023-07-21T18:33:20.863 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:20.374+0000 7f5720eb1700 1 -- 172.21.15.39:0/748159551 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5718011a00 con 0x7f571c104400 2023-07-21T18:33:20.863 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:20.375+0000 7f5724117700 1 -- 172.21.15.39:0/748159551 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f571c104400 msgr2=0x7f571c104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:33:20.863 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:20.375+0000 7f5724117700 1 --2- 172.21.15.39:0/748159551 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f571c104400 0x7f571c104810 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f57180098a0 tx=0x7f5718009bb0 comp rx=0 tx=0).stop 2023-07-21T18:33:20.863 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:20.375+0000 7f5724117700 1 -- 172.21.15.39:0/748159551 shutdown_connections 2023-07-21T18:33:20.863 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:20.375+0000 7f5724117700 1 --2- 172.21.15.39:0/748159551 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f571c104400 0x7f571c104810 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:33:20.863 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:20.375+0000 7f5724117700 1 -- 172.21.15.39:0/748159551 >> 172.21.15.39:0/748159551 conn(0x7f571c0ffa50 msgr2=0x7f571c101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:33:20.863 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:20.375+0000 7f5724117700 1 -- 172.21.15.39:0/748159551 shutdown_connections 2023-07-21T18:33:20.864 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:20.375+0000 7f5724117700 1 -- 172.21.15.39:0/748159551 wait complete. 2023-07-21T18:33:20.864 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:20.375+0000 7f5724117700 1 Processor -- start 2023-07-21T18:33:20.864 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:20.376+0000 7f5724117700 1 -- start start 2023-07-21T18:33:20.864 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:20.376+0000 7f5724117700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f571c104400 0x7f571c18f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:33:20.864 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:20.376+0000 7f5724117700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f571c18fac0 con 0x7f571c104400 2023-07-21T18:33:20.864 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:20.376+0000 7f5721eb3700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f571c104400 0x7f571c18f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:33:20.864 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:20.376+0000 7f5721eb3700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f571c104400 0x7f571c18f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53696/0 (socket says 172.21.15.39:53696) 2023-07-21T18:33:20.865 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:20.376+0000 7f5721eb3700 1 -- 172.21.15.39:0/2309480695 learned_addr learned my addr 172.21.15.39:0/2309480695 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:33:20.865 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:20.376+0000 7f5721eb3700 1 -- 172.21.15.39:0/2309480695 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5718009550 con 0x7f571c104400 2023-07-21T18:33:20.865 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:20.376+0000 7f5721eb3700 1 --2- 172.21.15.39:0/2309480695 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f571c104400 0x7f571c18f580 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5718009130 tx=0x7f5718011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:33:20.865 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:20.376+0000 7f5712ffd700 1 -- 172.21.15.39:0/2309480695 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5718003b70 con 0x7f571c104400 2023-07-21T18:33:20.865 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:20.376+0000 7f5724117700 1 -- 172.21.15.39:0/2309480695 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f571c18dbd0 con 0x7f571c104400 2023-07-21T18:33:20.865 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:20.377+0000 7f5712ffd700 1 -- 172.21.15.39:0/2309480695 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5718003cd0 con 0x7f571c104400 2023-07-21T18:33:20.865 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:20.377+0000 7f5712ffd700 1 -- 172.21.15.39:0/2309480695 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f57180197d0 con 0x7f571c104400 2023-07-21T18:33:20.865 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:20.377+0000 7f5712ffd700 1 -- 172.21.15.39:0/2309480695 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f5718019a30 con 0x7f571c104400 2023-07-21T18:33:20.866 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:20.377+0000 7f5724117700 1 -- 172.21.15.39:0/2309480695 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f571c18dff0 con 0x7f571c104400 2023-07-21T18:33:20.866 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:20.377+0000 7f5712ffd700 1 -- 172.21.15.39:0/2309480695 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f57180257a0 con 0x7f571c104400 2023-07-21T18:33:20.866 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:20.378+0000 7f5724117700 1 -- 172.21.15.39:0/2309480695 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f571c061b40 con 0x7f571c104400 2023-07-21T18:33:20.866 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:20.382+0000 7f5712ffd700 1 -- 172.21.15.39:0/2309480695 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f571802a060 con 0x7f571c104400 2023-07-21T18:33:20.866 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:20.516+0000 7f5724117700 1 -- 172.21.15.39:0/2309480695 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f571c1062c0 con 0x7f571c104400 2023-07-21T18:33:20.866 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:20.516+0000 7f5712ffd700 1 -- 172.21.15.39:0/2309480695 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f5718025960 con 0x7f571c104400 2023-07-21T18:33:20.866 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:20.517+0000 7f5724117700 1 -- 172.21.15.39:0/2309480695 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f571c104400 msgr2=0x7f571c18f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:33:20.867 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:20.517+0000 7f5724117700 1 --2- 172.21.15.39:0/2309480695 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f571c104400 0x7f571c18f580 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5718009130 tx=0x7f5718011ad0 comp rx=0 tx=0).stop 2023-07-21T18:33:20.867 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:20.518+0000 7f5724117700 1 -- 172.21.15.39:0/2309480695 shutdown_connections 2023-07-21T18:33:20.867 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:20.518+0000 7f5724117700 1 --2- 172.21.15.39:0/2309480695 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f571c104400 0x7f571c18f580 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:33:20.867 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:20.518+0000 7f5724117700 1 -- 172.21.15.39:0/2309480695 >> 172.21.15.39:0/2309480695 conn(0x7f571c0ffa50 msgr2=0x7f571c072f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:33:20.867 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:20.518+0000 7f5724117700 1 -- 172.21.15.39:0/2309480695 shutdown_connections 2023-07-21T18:33:20.867 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:20.518+0000 7f5724117700 1 -- 172.21.15.39:0/2309480695 wait complete. 2023-07-21T18:33:20.867 INFO:teuthology.orchestra.run.smithi039.stdout:mgr not available, waiting (1/15)... 2023-07-21T18:33:21.143 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:20 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:33:20.850+0000 7ff7773f7200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-21T18:33:21.143 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:20 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:33:20.959+0000 7ff7773f7200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-21T18:33:21.449 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:21 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:33:21.109+0000 7ff7773f7200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-21T18:33:21.450 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:21 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:33:21.223+0000 7ff7773f7200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-21T18:33:22.199 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:21 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:33:21.733+0000 7ff7773f7200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-21T18:33:22.949 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:22 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:33:22.427+0000 7ff7773f7200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-21T18:33:22.949 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:22 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:33:22.619+0000 7ff7773f7200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-21T18:33:23.897 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2023-07-21T18:33:23.897 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout { 2023-07-21T18:33:23.898 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "fsid": "eaf8e25a-27f4-11ee-9b34-001a4aab830c", 2023-07-21T18:33:23.898 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "health": { 2023-07-21T18:33:23.898 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-07-21T18:33:23.898 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-07-21T18:33:23.898 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-07-21T18:33:23.898 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2023-07-21T18:33:23.898 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-07-21T18:33:23.898 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-07-21T18:33:23.899 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 0 2023-07-21T18:33:23.899 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2023-07-21T18:33:23.899 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-07-21T18:33:23.899 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "a" 2023-07-21T18:33:23.899 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2023-07-21T18:33:23.899 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2023-07-21T18:33:23.899 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "monmap": { 2023-07-21T18:33:23.899 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T18:33:23.900 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-07-21T18:33:23.900 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-07-21T18:33:23.900 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2023-07-21T18:33:23.900 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-07-21T18:33:23.900 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T18:33:23.900 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-07-21T18:33:23.900 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-07-21T18:33:23.901 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-07-21T18:33:23.906 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-07-21T18:33:23.907 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-07-21T18:33:23.907 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-07-21T18:33:23.907 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2023-07-21T18:33:23.907 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-07-21T18:33:23.907 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-07-21T18:33:23.907 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-07-21T18:33:23.907 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-07-21T18:33:23.908 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-07-21T18:33:23.908 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-07-21T18:33:23.908 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-07-21T18:33:23.908 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-07-21T18:33:23.908 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-07-21T18:33:23.908 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2023-07-21T18:33:23.908 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-07-21T18:33:23.909 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T18:33:23.909 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-07-21T18:33:23.909 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-07-21T18:33:23.909 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2023-07-21T18:33:23.909 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-07-21T18:33:23.909 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "available": false, 2023-07-21T18:33:23.909 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-07-21T18:33:23.909 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "modules": [ 2023-07-21T18:33:23.910 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "iostat", 2023-07-21T18:33:23.910 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "nfs", 2023-07-21T18:33:23.910 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "restful" 2023-07-21T18:33:23.910 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2023-07-21T18:33:23.910 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-21T18:33:23.910 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2023-07-21T18:33:23.910 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-07-21T18:33:23.910 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T18:33:23.911 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "modified": "2023-07-21T18:33:06.505474+0000", 2023-07-21T18:33:23.911 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-21T18:33:23.911 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2023-07-21T18:33:23.911 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-07-21T18:33:23.911 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout } 2023-07-21T18:33:23.911 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:23.318+0000 7fae408b6700 1 Processor -- start 2023-07-21T18:33:23.911 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:23.319+0000 7fae408b6700 1 -- start start 2023-07-21T18:33:23.912 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:23.319+0000 7fae408b6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae3c106010 0x7fae3c1083f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:33:23.912 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:23.319+0000 7fae408b6700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3c073c20 con 0x7fae3c106010 2023-07-21T18:33:23.912 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:23.319+0000 7fae3a59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae3c106010 0x7fae3c1083f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:33:23.912 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:23.319+0000 7fae3a59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae3c106010 0x7fae3c1083f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53698/0 (socket says 172.21.15.39:53698) 2023-07-21T18:33:23.912 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:23.319+0000 7fae3a59c700 1 -- 172.21.15.39:0/4076500888 learned_addr learned my addr 172.21.15.39:0/4076500888 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:33:23.912 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:23.319+0000 7fae3a59c700 1 -- 172.21.15.39:0/4076500888 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae3c073d60 con 0x7fae3c106010 2023-07-21T18:33:23.912 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:23.320+0000 7fae3a59c700 1 --2- 172.21.15.39:0/4076500888 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae3c106010 0x7fae3c1083f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fae2c0098a0 tx=0x7fae2c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55524db0478d24a3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:33:23.912 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:23.320+0000 7fae3959a700 1 -- 172.21.15.39:0/4076500888 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fae2c004160 con 0x7fae3c106010 2023-07-21T18:33:23.913 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:23.320+0000 7fae3959a700 1 -- 172.21.15.39:0/4076500888 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fae2c0042c0 con 0x7fae3c106010 2023-07-21T18:33:23.913 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:23.320+0000 7fae3959a700 1 -- 172.21.15.39:0/4076500888 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fae2c011a00 con 0x7fae3c106010 2023-07-21T18:33:23.913 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:23.320+0000 7fae408b6700 1 -- 172.21.15.39:0/4076500888 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae3c106010 msgr2=0x7fae3c1083f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:33:23.913 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:23.320+0000 7fae408b6700 1 --2- 172.21.15.39:0/4076500888 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae3c106010 0x7fae3c1083f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fae2c0098a0 tx=0x7fae2c009bb0 comp rx=0 tx=0).stop 2023-07-21T18:33:23.913 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:23.320+0000 7fae408b6700 1 -- 172.21.15.39:0/4076500888 shutdown_connections 2023-07-21T18:33:23.913 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:23.320+0000 7fae408b6700 1 --2- 172.21.15.39:0/4076500888 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae3c106010 0x7fae3c1083f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:33:23.913 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:23.321+0000 7fae408b6700 1 -- 172.21.15.39:0/4076500888 >> 172.21.15.39:0/4076500888 conn(0x7fae3c0ffa50 msgr2=0x7fae3c101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:33:23.914 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:23.321+0000 7fae408b6700 1 -- 172.21.15.39:0/4076500888 shutdown_connections 2023-07-21T18:33:23.914 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:23.321+0000 7fae408b6700 1 -- 172.21.15.39:0/4076500888 wait complete. 2023-07-21T18:33:23.914 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:23.321+0000 7fae408b6700 1 Processor -- start 2023-07-21T18:33:23.914 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:23.321+0000 7fae408b6700 1 -- start start 2023-07-21T18:33:23.914 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:23.322+0000 7fae408b6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae3c106010 0x7fae3c105970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:33:23.914 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:23.322+0000 7fae408b6700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3c105eb0 con 0x7fae3c106010 2023-07-21T18:33:23.914 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:23.322+0000 7fae3a59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae3c106010 0x7fae3c105970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:33:23.914 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:23.322+0000 7fae3a59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae3c106010 0x7fae3c105970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53700/0 (socket says 172.21.15.39:53700) 2023-07-21T18:33:23.915 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:23.322+0000 7fae3a59c700 1 -- 172.21.15.39:0/1800268174 learned_addr learned my addr 172.21.15.39:0/1800268174 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:33:23.915 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:23.322+0000 7fae3a59c700 1 -- 172.21.15.39:0/1800268174 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae2c009550 con 0x7fae3c106010 2023-07-21T18:33:23.915 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:23.322+0000 7fae3a59c700 1 --2- 172.21.15.39:0/1800268174 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae3c106010 0x7fae3c105970 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fae2c009fb0 tx=0x7fae2c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:33:23.915 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:23.322+0000 7fae337fe700 1 -- 172.21.15.39:0/1800268174 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fae2c003a40 con 0x7fae3c106010 2023-07-21T18:33:23.915 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:23.322+0000 7fae337fe700 1 -- 172.21.15.39:0/1800268174 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fae2c003ba0 con 0x7fae3c106010 2023-07-21T18:33:23.915 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:23.322+0000 7fae337fe700 1 -- 172.21.15.39:0/1800268174 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fae2c01a750 con 0x7fae3c106010 2023-07-21T18:33:23.915 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:23.322+0000 7fae408b6700 1 -- 172.21.15.39:0/1800268174 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae3c103fc0 con 0x7fae3c106010 2023-07-21T18:33:23.915 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:23.322+0000 7fae408b6700 1 -- 172.21.15.39:0/1800268174 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae3c1043e0 con 0x7fae3c106010 2023-07-21T18:33:23.916 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:23.323+0000 7fae337fe700 1 -- 172.21.15.39:0/1800268174 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fae2c01acd0 con 0x7fae3c106010 2023-07-21T18:33:23.916 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:23.323+0000 7fae337fe700 1 -- 172.21.15.39:0/1800268174 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fae2c0193c0 con 0x7fae3c106010 2023-07-21T18:33:23.916 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:23.324+0000 7fae408b6700 1 -- 172.21.15.39:0/1800268174 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae3c109600 con 0x7fae3c106010 2023-07-21T18:33:23.916 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:23.326+0000 7fae337fe700 1 -- 172.21.15.39:0/1800268174 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7fae2c02de40 con 0x7fae3c106010 2023-07-21T18:33:23.916 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:23.460+0000 7fae408b6700 1 -- 172.21.15.39:0/1800268174 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fae3c1097f0 con 0x7fae3c106010 2023-07-21T18:33:23.916 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:23.460+0000 7fae337fe700 1 -- 172.21.15.39:0/1800268174 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7fae2c024c10 con 0x7fae3c106010 2023-07-21T18:33:23.916 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:23.461+0000 7fae408b6700 1 -- 172.21.15.39:0/1800268174 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae3c106010 msgr2=0x7fae3c105970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:33:23.917 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:23.461+0000 7fae408b6700 1 --2- 172.21.15.39:0/1800268174 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae3c106010 0x7fae3c105970 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fae2c009fb0 tx=0x7fae2c011ad0 comp rx=0 tx=0).stop 2023-07-21T18:33:23.917 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:23.461+0000 7fae408b6700 1 -- 172.21.15.39:0/1800268174 shutdown_connections 2023-07-21T18:33:23.917 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:23.461+0000 7fae408b6700 1 --2- 172.21.15.39:0/1800268174 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae3c106010 0x7fae3c105970 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:33:23.917 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:23.461+0000 7fae408b6700 1 -- 172.21.15.39:0/1800268174 >> 172.21.15.39:0/1800268174 conn(0x7fae3c0ffa50 msgr2=0x7fae3c101140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:33:23.917 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:23.461+0000 7fae408b6700 1 -- 172.21.15.39:0/1800268174 shutdown_connections 2023-07-21T18:33:23.917 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:23.461+0000 7fae408b6700 1 -- 172.21.15.39:0/1800268174 wait complete. 2023-07-21T18:33:23.917 INFO:teuthology.orchestra.run.smithi039.stdout:mgr not available, waiting (2/15)... 2023-07-21T18:33:23.939 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:23 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/1800268174' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-07-21T18:33:23.939 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:23 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:33:23.597+0000 7ff7773f7200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-21T18:33:23.939 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:23 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:33:23.708+0000 7ff7773f7200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-21T18:33:24.199 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:23 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:33:23.928+0000 7ff7773f7200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-21T18:33:24.561 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:24 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:33:24.275+0000 7ff7773f7200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-21T18:33:24.949 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:24 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:33:24.527+0000 7ff7773f7200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-21T18:33:24.949 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:24 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:33:24.708+0000 7ff7773f7200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-21T18:33:25.449 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:25 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:33:25.035+0000 7ff7773f7200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-21T18:33:26.153 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:25 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:33:25.863+0000 7ff7773f7200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-21T18:33:26.154 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:26 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:33:26.025+0000 7ff7773f7200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-21T18:33:26.449 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:26 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:33:26.282+0000 7ff7773f7200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-21T18:33:26.736 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:26 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:33:26.423+0000 7ff7773f7200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-21T18:33:26.736 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:26 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:33:26.702+0000 7ff7773f7200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-21T18:33:26.945 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2023-07-21T18:33:26.945 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout { 2023-07-21T18:33:26.945 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "fsid": "eaf8e25a-27f4-11ee-9b34-001a4aab830c", 2023-07-21T18:33:26.945 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "health": { 2023-07-21T18:33:26.946 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-07-21T18:33:26.946 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-07-21T18:33:26.946 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-07-21T18:33:26.946 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2023-07-21T18:33:26.946 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-07-21T18:33:26.946 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-07-21T18:33:26.946 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 0 2023-07-21T18:33:26.946 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2023-07-21T18:33:26.947 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-07-21T18:33:26.947 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "a" 2023-07-21T18:33:26.947 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2023-07-21T18:33:26.947 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum_age": 15, 2023-07-21T18:33:26.947 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "monmap": { 2023-07-21T18:33:26.947 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T18:33:26.947 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-07-21T18:33:26.948 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-07-21T18:33:26.948 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2023-07-21T18:33:26.948 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-07-21T18:33:26.948 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T18:33:26.948 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-07-21T18:33:26.949 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-07-21T18:33:26.956 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-07-21T18:33:26.956 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-07-21T18:33:26.957 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-07-21T18:33:26.957 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-07-21T18:33:26.957 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2023-07-21T18:33:26.957 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-07-21T18:33:26.957 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-07-21T18:33:26.957 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-07-21T18:33:26.958 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-07-21T18:33:26.958 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-07-21T18:33:26.958 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-07-21T18:33:26.958 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-07-21T18:33:26.958 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-07-21T18:33:26.958 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-07-21T18:33:26.959 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2023-07-21T18:33:26.959 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-07-21T18:33:26.959 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T18:33:26.959 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-07-21T18:33:26.959 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-07-21T18:33:26.959 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2023-07-21T18:33:26.960 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-07-21T18:33:26.960 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "available": false, 2023-07-21T18:33:26.960 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-07-21T18:33:26.960 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "modules": [ 2023-07-21T18:33:26.960 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "iostat", 2023-07-21T18:33:26.960 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "nfs", 2023-07-21T18:33:26.960 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "restful" 2023-07-21T18:33:26.961 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2023-07-21T18:33:26.961 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-21T18:33:26.961 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2023-07-21T18:33:26.961 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-07-21T18:33:26.961 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T18:33:26.962 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "modified": "2023-07-21T18:33:06.505474+0000", 2023-07-21T18:33:26.962 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-21T18:33:26.962 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2023-07-21T18:33:26.962 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-07-21T18:33:26.962 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout } 2023-07-21T18:33:26.962 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:26.386+0000 7f9a9302f700 1 Processor -- start 2023-07-21T18:33:26.962 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:26.386+0000 7f9a9302f700 1 -- start start 2023-07-21T18:33:26.963 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:26.386+0000 7f9a9302f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a8c106010 0x7f9a8c1083f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:33:26.963 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:26.386+0000 7f9a9302f700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8c073c20 con 0x7f9a8c106010 2023-07-21T18:33:26.963 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:26.387+0000 7f9a90dcb700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a8c106010 0x7f9a8c1083f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:33:26.963 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:26.387+0000 7f9a90dcb700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a8c106010 0x7f9a8c1083f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53702/0 (socket says 172.21.15.39:53702) 2023-07-21T18:33:26.963 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:26.387+0000 7f9a90dcb700 1 -- 172.21.15.39:0/1804410650 learned_addr learned my addr 172.21.15.39:0/1804410650 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:33:26.963 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:26.387+0000 7f9a90dcb700 1 -- 172.21.15.39:0/1804410650 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a8c073d60 con 0x7f9a8c106010 2023-07-21T18:33:26.963 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:26.387+0000 7f9a90dcb700 1 --2- 172.21.15.39:0/1804410650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a8c106010 0x7f9a8c1083f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9a800098a0 tx=0x7f9a80009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15667b2aa0a6b163 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:33:26.964 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:26.387+0000 7f9a8b7fe700 1 -- 172.21.15.39:0/1804410650 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9a80004160 con 0x7f9a8c106010 2023-07-21T18:33:26.964 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:26.387+0000 7f9a8b7fe700 1 -- 172.21.15.39:0/1804410650 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9a800042c0 con 0x7f9a8c106010 2023-07-21T18:33:26.964 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:26.387+0000 7f9a8b7fe700 1 -- 172.21.15.39:0/1804410650 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9a80011a00 con 0x7f9a8c106010 2023-07-21T18:33:26.964 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:26.388+0000 7f9a9302f700 1 -- 172.21.15.39:0/1804410650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a8c106010 msgr2=0x7f9a8c1083f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:33:26.964 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:26.388+0000 7f9a9302f700 1 --2- 172.21.15.39:0/1804410650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a8c106010 0x7f9a8c1083f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9a800098a0 tx=0x7f9a80009bb0 comp rx=0 tx=0).stop 2023-07-21T18:33:26.964 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:26.388+0000 7f9a9302f700 1 -- 172.21.15.39:0/1804410650 shutdown_connections 2023-07-21T18:33:26.965 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:26.388+0000 7f9a9302f700 1 --2- 172.21.15.39:0/1804410650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a8c106010 0x7f9a8c1083f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:33:26.965 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:26.388+0000 7f9a9302f700 1 -- 172.21.15.39:0/1804410650 >> 172.21.15.39:0/1804410650 conn(0x7f9a8c0ffa50 msgr2=0x7f9a8c101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:33:26.965 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:26.388+0000 7f9a9302f700 1 -- 172.21.15.39:0/1804410650 shutdown_connections 2023-07-21T18:33:26.965 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:26.388+0000 7f9a9302f700 1 -- 172.21.15.39:0/1804410650 wait complete. 2023-07-21T18:33:26.965 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:26.389+0000 7f9a9302f700 1 Processor -- start 2023-07-21T18:33:26.966 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:26.389+0000 7f9a9302f700 1 -- start start 2023-07-21T18:33:26.966 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:26.389+0000 7f9a9302f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a8c106010 0x7f9a8c105970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:33:26.966 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:26.389+0000 7f9a9302f700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8c105eb0 con 0x7f9a8c106010 2023-07-21T18:33:26.966 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:26.389+0000 7f9a90dcb700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a8c106010 0x7f9a8c105970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:33:26.966 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:26.389+0000 7f9a90dcb700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a8c106010 0x7f9a8c105970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53704/0 (socket says 172.21.15.39:53704) 2023-07-21T18:33:26.966 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:26.389+0000 7f9a90dcb700 1 -- 172.21.15.39:0/20070621 learned_addr learned my addr 172.21.15.39:0/20070621 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:33:26.966 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:26.389+0000 7f9a90dcb700 1 -- 172.21.15.39:0/20070621 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a80009550 con 0x7f9a8c106010 2023-07-21T18:33:26.967 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:26.390+0000 7f9a90dcb700 1 --2- 172.21.15.39:0/20070621 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a8c106010 0x7f9a8c105970 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9a80009130 tx=0x7f9a80011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:33:26.967 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:26.390+0000 7f9a89ffb700 1 -- 172.21.15.39:0/20070621 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9a80003ab0 con 0x7f9a8c106010 2023-07-21T18:33:26.967 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:26.390+0000 7f9a89ffb700 1 -- 172.21.15.39:0/20070621 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9a80003c10 con 0x7f9a8c106010 2023-07-21T18:33:26.967 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:26.390+0000 7f9a89ffb700 1 -- 172.21.15.39:0/20070621 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9a800247d0 con 0x7f9a8c106010 2023-07-21T18:33:26.968 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:26.390+0000 7f9a9302f700 1 -- 172.21.15.39:0/20070621 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a8c103fc0 con 0x7f9a8c106010 2023-07-21T18:33:26.968 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:26.390+0000 7f9a9302f700 1 -- 172.21.15.39:0/20070621 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a8c1043e0 con 0x7f9a8c106010 2023-07-21T18:33:26.968 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:26.390+0000 7f9a89ffb700 1 -- 172.21.15.39:0/20070621 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f9a80024d00 con 0x7f9a8c106010 2023-07-21T18:33:26.968 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:26.390+0000 7f9a89ffb700 1 -- 172.21.15.39:0/20070621 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f9a8002db10 con 0x7f9a8c106010 2023-07-21T18:33:26.968 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:26.391+0000 7f9a9302f700 1 -- 172.21.15.39:0/20070621 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a8c109600 con 0x7f9a8c106010 2023-07-21T18:33:26.968 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:26.393+0000 7f9a89ffb700 1 -- 172.21.15.39:0/20070621 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f9a8002dd70 con 0x7f9a8c106010 2023-07-21T18:33:26.969 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:26.526+0000 7f9a9302f700 1 -- 172.21.15.39:0/20070621 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f9a8c109820 con 0x7f9a8c106010 2023-07-21T18:33:26.969 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:26.526+0000 7f9a89ffb700 1 -- 172.21.15.39:0/20070621 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f9a8001abb0 con 0x7f9a8c106010 2023-07-21T18:33:26.969 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:26.527+0000 7f9a9302f700 1 -- 172.21.15.39:0/20070621 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a8c106010 msgr2=0x7f9a8c105970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:33:26.969 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:26.527+0000 7f9a9302f700 1 --2- 172.21.15.39:0/20070621 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a8c106010 0x7f9a8c105970 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9a80009130 tx=0x7f9a80011ad0 comp rx=0 tx=0).stop 2023-07-21T18:33:26.969 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:26.527+0000 7f9a9302f700 1 -- 172.21.15.39:0/20070621 shutdown_connections 2023-07-21T18:33:26.970 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:26.527+0000 7f9a9302f700 1 --2- 172.21.15.39:0/20070621 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a8c106010 0x7f9a8c105970 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:33:26.970 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:26.527+0000 7f9a9302f700 1 -- 172.21.15.39:0/20070621 >> 172.21.15.39:0/20070621 conn(0x7f9a8c0ffa50 msgr2=0x7f9a8c101140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:33:26.970 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:26.528+0000 7f9a9302f700 1 -- 172.21.15.39:0/20070621 shutdown_connections 2023-07-21T18:33:26.970 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:26.528+0000 7f9a9302f700 1 -- 172.21.15.39:0/20070621 wait complete. 2023-07-21T18:33:26.970 INFO:teuthology.orchestra.run.smithi039.stdout:mgr not available, waiting (3/15)... 2023-07-21T18:33:27.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:26 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/20070621' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-07-21T18:33:28.199 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:27 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:33:27.749+0000 7ff7773f7200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-21T18:33:28.200 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:27 smithi039 ceph-mon[96489]: Activating manager daemon x 2023-07-21T18:33:28.200 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:27 smithi039 ceph-mon[96489]: mgrmap e2: x(active, starting, since 0.00754245s) 2023-07-21T18:33:28.200 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:27 smithi039 ceph-mon[96489]: from='mgr.14100 172.21.15.39:0/1900447294' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-07-21T18:33:28.200 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:27 smithi039 ceph-mon[96489]: from='mgr.14100 172.21.15.39:0/1900447294' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-07-21T18:33:28.200 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:27 smithi039 ceph-mon[96489]: from='mgr.14100 172.21.15.39:0/1900447294' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-07-21T18:33:28.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:27 smithi039 ceph-mon[96489]: from='mgr.14100 172.21.15.39:0/1900447294' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T18:33:28.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:27 smithi039 ceph-mon[96489]: from='mgr.14100 172.21.15.39:0/1900447294' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-07-21T18:33:28.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:27 smithi039 ceph-mon[96489]: Manager daemon x is now available 2023-07-21T18:33:28.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:27 smithi039 ceph-mon[96489]: from='mgr.14100 172.21.15.39:0/1900447294' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-21T18:33:28.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:27 smithi039 ceph-mon[96489]: from='mgr.14100 172.21.15.39:0/1900447294' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-21T18:33:28.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:27 smithi039 ceph-mon[96489]: from='mgr.14100 172.21.15.39:0/1900447294' entity='mgr.x' 2023-07-21T18:33:28.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:27 smithi039 ceph-mon[96489]: from='mgr.14100 172.21.15.39:0/1900447294' entity='mgr.x' 2023-07-21T18:33:28.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:27 smithi039 ceph-mon[96489]: from='mgr.14100 172.21.15.39:0/1900447294' entity='mgr.x' 2023-07-21T18:33:30.107 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:29 smithi039 ceph-mon[96489]: mgrmap e3: x(active, since 1.01572s) 2023-07-21T18:33:30.309 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2023-07-21T18:33:30.310 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout { 2023-07-21T18:33:30.310 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "fsid": "eaf8e25a-27f4-11ee-9b34-001a4aab830c", 2023-07-21T18:33:30.310 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "health": { 2023-07-21T18:33:30.310 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-07-21T18:33:30.310 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-07-21T18:33:30.310 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-07-21T18:33:30.310 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2023-07-21T18:33:30.311 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-07-21T18:33:30.311 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-07-21T18:33:30.311 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 0 2023-07-21T18:33:30.311 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2023-07-21T18:33:30.311 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-07-21T18:33:30.311 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "a" 2023-07-21T18:33:30.311 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2023-07-21T18:33:30.311 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum_age": 19, 2023-07-21T18:33:30.312 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "monmap": { 2023-07-21T18:33:30.312 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T18:33:30.312 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-07-21T18:33:30.312 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-07-21T18:33:30.312 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2023-07-21T18:33:30.312 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-07-21T18:33:30.313 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T18:33:30.313 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-07-21T18:33:30.313 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-07-21T18:33:30.313 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-07-21T18:33:30.314 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-07-21T18:33:30.314 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-07-21T18:33:30.314 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-07-21T18:33:30.314 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2023-07-21T18:33:30.315 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-07-21T18:33:30.315 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-07-21T18:33:30.315 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-07-21T18:33:30.315 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-07-21T18:33:30.316 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-07-21T18:33:30.316 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-07-21T18:33:30.316 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-07-21T18:33:30.317 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-07-21T18:33:30.317 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-07-21T18:33:30.317 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2023-07-21T18:33:30.317 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-07-21T18:33:30.318 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T18:33:30.318 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-07-21T18:33:30.318 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-07-21T18:33:30.318 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2023-07-21T18:33:30.318 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-07-21T18:33:30.319 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "available": true, 2023-07-21T18:33:30.319 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-07-21T18:33:30.319 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "modules": [ 2023-07-21T18:33:30.319 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "iostat", 2023-07-21T18:33:30.319 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "nfs", 2023-07-21T18:33:30.319 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "restful" 2023-07-21T18:33:30.319 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2023-07-21T18:33:30.319 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-21T18:33:30.320 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2023-07-21T18:33:30.320 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-07-21T18:33:30.320 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T18:33:30.320 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "modified": "2023-07-21T18:33:06.505474+0000", 2023-07-21T18:33:30.320 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-21T18:33:30.320 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2023-07-21T18:33:30.320 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-07-21T18:33:30.321 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout } 2023-07-21T18:33:30.321 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:29.408+0000 7f5641038700 1 Processor -- start 2023-07-21T18:33:30.321 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:29.408+0000 7f5641038700 1 -- start start 2023-07-21T18:33:30.321 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:29.408+0000 7f5641038700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f563c1043f0 0x7f563c104800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:33:30.321 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:29.408+0000 7f5641038700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f563c104d40 con 0x7f563c1043f0 2023-07-21T18:33:30.321 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:29.408+0000 7f563ad9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f563c1043f0 0x7f563c104800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:33:30.321 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:29.408+0000 7f563ad9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f563c1043f0 0x7f563c104800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53732/0 (socket says 172.21.15.39:53732) 2023-07-21T18:33:30.322 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:29.408+0000 7f563ad9d700 1 -- 172.21.15.39:0/2374752121 learned_addr learned my addr 172.21.15.39:0/2374752121 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:33:30.322 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:29.409+0000 7f563ad9d700 1 -- 172.21.15.39:0/2374752121 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f563c104e80 con 0x7f563c1043f0 2023-07-21T18:33:30.322 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:29.409+0000 7f563ad9d700 1 --2- 172.21.15.39:0/2374752121 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f563c1043f0 0x7f563c104800 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f562c0098a0 tx=0x7f562c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8f1848c8ca8079a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:33:30.322 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:29.409+0000 7f5639d9b700 1 -- 172.21.15.39:0/2374752121 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f562c004160 con 0x7f563c1043f0 2023-07-21T18:33:30.322 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:29.409+0000 7f5639d9b700 1 -- 172.21.15.39:0/2374752121 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f562c0042c0 con 0x7f563c1043f0 2023-07-21T18:33:30.322 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:29.409+0000 7f5639d9b700 1 -- 172.21.15.39:0/2374752121 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f562c011a00 con 0x7f563c1043f0 2023-07-21T18:33:30.322 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:29.410+0000 7f5641038700 1 -- 172.21.15.39:0/2374752121 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f563c1043f0 msgr2=0x7f563c104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:33:30.323 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:29.410+0000 7f5641038700 1 --2- 172.21.15.39:0/2374752121 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f563c1043f0 0x7f563c104800 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f562c0098a0 tx=0x7f562c009bb0 comp rx=0 tx=0).stop 2023-07-21T18:33:30.323 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:29.410+0000 7f5641038700 1 -- 172.21.15.39:0/2374752121 shutdown_connections 2023-07-21T18:33:30.323 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:29.410+0000 7f5641038700 1 --2- 172.21.15.39:0/2374752121 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f563c1043f0 0x7f563c104800 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:33:30.323 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:29.410+0000 7f5641038700 1 -- 172.21.15.39:0/2374752121 >> 172.21.15.39:0/2374752121 conn(0x7f563c0ffa40 msgr2=0x7f563c101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:33:30.323 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:29.410+0000 7f5641038700 1 -- 172.21.15.39:0/2374752121 shutdown_connections 2023-07-21T18:33:30.323 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:29.410+0000 7f5641038700 1 -- 172.21.15.39:0/2374752121 wait complete. 2023-07-21T18:33:30.323 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:29.410+0000 7f5641038700 1 Processor -- start 2023-07-21T18:33:30.323 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:29.411+0000 7f5641038700 1 -- start start 2023-07-21T18:33:30.324 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:29.411+0000 7f5641038700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f563c1043f0 0x7f563c0739f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:33:30.324 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:29.411+0000 7f5641038700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f563c19bf70 con 0x7f563c1043f0 2023-07-21T18:33:30.324 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:29.411+0000 7f563ad9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f563c1043f0 0x7f563c0739f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:33:30.324 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:29.411+0000 7f563ad9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f563c1043f0 0x7f563c0739f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53734/0 (socket says 172.21.15.39:53734) 2023-07-21T18:33:30.324 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:29.411+0000 7f563ad9d700 1 -- 172.21.15.39:0/2598208228 learned_addr learned my addr 172.21.15.39:0/2598208228 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:33:30.324 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:29.411+0000 7f563ad9d700 1 -- 172.21.15.39:0/2598208228 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f562c009550 con 0x7f563c1043f0 2023-07-21T18:33:30.324 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:29.411+0000 7f563ad9d700 1 --2- 172.21.15.39:0/2598208228 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f563c1043f0 0x7f563c0739f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f562c009fb0 tx=0x7f562c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:33:30.325 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:29.411+0000 7f5623fff700 1 -- 172.21.15.39:0/2598208228 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f562c003930 con 0x7f563c1043f0 2023-07-21T18:33:30.325 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:29.412+0000 7f5641038700 1 -- 172.21.15.39:0/2598208228 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f563c19c0b0 con 0x7f563c1043f0 2023-07-21T18:33:30.325 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:29.412+0000 7f5623fff700 1 -- 172.21.15.39:0/2598208228 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f562c003a90 con 0x7f563c1043f0 2023-07-21T18:33:30.325 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:29.412+0000 7f5623fff700 1 -- 172.21.15.39:0/2598208228 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f562c019610 con 0x7f563c1043f0 2023-07-21T18:33:30.325 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:29.412+0000 7f5641038700 1 -- 172.21.15.39:0/2598208228 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f563c19c4d0 con 0x7f563c1043f0 2023-07-21T18:33:30.325 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:29.412+0000 7f5623fff700 1 -- 172.21.15.39:0/2598208228 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 3) v1 ==== 45366+0+0 (secure 0 0 0) 0x7f562c0198f0 con 0x7f563c1043f0 2023-07-21T18:33:30.326 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:29.413+0000 7f5623fff700 1 --2- 172.21.15.39:0/2598208228 >> [v2:172.21.15.39:6800/889516631,v1:172.21.15.39:6801/889516631] conn(0x7f5624038550 0x7f562403aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:33:30.326 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:29.413+0000 7f5623fff700 1 -- 172.21.15.39:0/2598208228 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f562c04c390 con 0x7f563c1043f0 2023-07-21T18:33:30.326 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:29.413+0000 7f5641038700 1 -- 172.21.15.39:0/2598208228 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f563c061b40 con 0x7f563c1043f0 2023-07-21T18:33:30.326 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:29.413+0000 7f563a59c700 1 --2- 172.21.15.39:0/2598208228 >> [v2:172.21.15.39:6800/889516631,v1:172.21.15.39:6801/889516631] conn(0x7f5624038550 0x7f562403aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:33:30.326 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:29.413+0000 7f563a59c700 1 --2- 172.21.15.39:0/2598208228 >> [v2:172.21.15.39:6800/889516631,v1:172.21.15.39:6801/889516631] conn(0x7f5624038550 0x7f562403aa00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5630006fd0 tx=0x7f5630006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:33:30.326 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:29.418+0000 7f5623fff700 1 -- 172.21.15.39:0/2598208228 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f562c019ba0 con 0x7f563c1043f0 2023-07-21T18:33:30.326 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:29.825+0000 7f5623fff700 1 -- 172.21.15.39:0/2598208228 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mgrmap(e 4) v1 ==== 45472+0+0 (secure 0 0 0) 0x7f562c02da50 con 0x7f563c1043f0 2023-07-21T18:33:30.326 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:29.931+0000 7f5641038700 1 -- 172.21.15.39:0/2598208228 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f563c19cd10 con 0x7f563c1043f0 2023-07-21T18:33:30.327 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:29.932+0000 7f5623fff700 1 -- 172.21.15.39:0/2598208228 <== mon.0 v2:172.21.15.39:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f562c003c00 con 0x7f563c1043f0 2023-07-21T18:33:30.327 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:29.934+0000 7f5641038700 1 -- 172.21.15.39:0/2598208228 >> [v2:172.21.15.39:6800/889516631,v1:172.21.15.39:6801/889516631] conn(0x7f5624038550 msgr2=0x7f562403aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:33:30.327 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:29.934+0000 7f5641038700 1 --2- 172.21.15.39:0/2598208228 >> [v2:172.21.15.39:6800/889516631,v1:172.21.15.39:6801/889516631] conn(0x7f5624038550 0x7f562403aa00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5630006fd0 tx=0x7f5630006e40 comp rx=0 tx=0).stop 2023-07-21T18:33:30.327 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:29.934+0000 7f5641038700 1 -- 172.21.15.39:0/2598208228 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f563c1043f0 msgr2=0x7f563c0739f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:33:30.327 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:29.934+0000 7f5641038700 1 --2- 172.21.15.39:0/2598208228 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f563c1043f0 0x7f563c0739f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f562c009fb0 tx=0x7f562c011ad0 comp rx=0 tx=0).stop 2023-07-21T18:33:30.327 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:29.934+0000 7f5641038700 1 -- 172.21.15.39:0/2598208228 shutdown_connections 2023-07-21T18:33:30.327 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:29.934+0000 7f5641038700 1 --2- 172.21.15.39:0/2598208228 >> [v2:172.21.15.39:6800/889516631,v1:172.21.15.39:6801/889516631] conn(0x7f5624038550 0x7f562403aa00 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:33:30.328 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:29.934+0000 7f5641038700 1 --2- 172.21.15.39:0/2598208228 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f563c1043f0 0x7f563c0739f0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:33:30.328 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:29.934+0000 7f5641038700 1 -- 172.21.15.39:0/2598208228 >> 172.21.15.39:0/2598208228 conn(0x7f563c0ffa40 msgr2=0x7f563c1010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:33:30.328 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:29.934+0000 7f5641038700 1 -- 172.21.15.39:0/2598208228 shutdown_connections 2023-07-21T18:33:30.328 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:29.934+0000 7f5641038700 1 -- 172.21.15.39:0/2598208228 wait complete. 2023-07-21T18:33:30.328 INFO:teuthology.orchestra.run.smithi039.stdout:mgr is available 2023-07-21T18:33:31.165 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:30 smithi039 ceph-mon[96489]: mgrmap e4: x(active, since 2s) 2023-07-21T18:33:31.166 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:30 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/2598208228' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-07-21T18:33:31.454 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2023-07-21T18:33:31.454 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout [global] 2023-07-21T18:33:31.454 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout fsid = eaf8e25a-27f4-11ee-9b34-001a4aab830c 2023-07-21T18:33:31.454 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-07-21T18:33:31.454 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-07-21T18:33:31.455 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-07-21T18:33:31.455 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-07-21T18:33:31.455 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2023-07-21T18:33:31.455 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout [mgr] 2023-07-21T18:33:31.455 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-07-21T18:33:31.455 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2023-07-21T18:33:31.455 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout [osd] 2023-07-21T18:33:31.455 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-07-21T18:33:31.456 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-07-21T18:33:31.456 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-07-21T18:33:31.456 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-07-21T18:33:31.456 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-07-21T18:33:31.456 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:30.743+0000 7f59d1176700 1 Processor -- start 2023-07-21T18:33:31.456 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:30.743+0000 7f59d1176700 1 -- start start 2023-07-21T18:33:31.456 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:30.743+0000 7f59d1176700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59cc106050 0x7f59cc108430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:33:31.457 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:30.743+0000 7f59d1176700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59cc073be0 con 0x7f59cc106050 2023-07-21T18:33:31.457 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:30.743+0000 7f59cad9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59cc106050 0x7f59cc108430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:33:31.457 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:30.743+0000 7f59cad9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59cc106050 0x7f59cc108430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53738/0 (socket says 172.21.15.39:53738) 2023-07-21T18:33:31.457 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:30.743+0000 7f59cad9d700 1 -- 172.21.15.39:0/876923426 learned_addr learned my addr 172.21.15.39:0/876923426 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:33:31.457 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:30.743+0000 7f59cad9d700 1 -- 172.21.15.39:0/876923426 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59cc073d20 con 0x7f59cc106050 2023-07-21T18:33:31.458 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:30.744+0000 7f59cad9d700 1 --2- 172.21.15.39:0/876923426 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59cc106050 0x7f59cc108430 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f59bc0098a0 tx=0x7f59bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b664211bf5fb38e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:33:31.458 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:30.744+0000 7f59c9d9b700 1 -- 172.21.15.39:0/876923426 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f59bc004160 con 0x7f59cc106050 2023-07-21T18:33:31.458 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:30.744+0000 7f59c9d9b700 1 -- 172.21.15.39:0/876923426 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f59bc0042c0 con 0x7f59cc106050 2023-07-21T18:33:31.458 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:30.744+0000 7f59c9d9b700 1 -- 172.21.15.39:0/876923426 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f59bc011a00 con 0x7f59cc106050 2023-07-21T18:33:31.458 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:30.744+0000 7f59d1176700 1 -- 172.21.15.39:0/876923426 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59cc106050 msgr2=0x7f59cc108430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:33:31.458 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:30.744+0000 7f59d1176700 1 --2- 172.21.15.39:0/876923426 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59cc106050 0x7f59cc108430 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f59bc0098a0 tx=0x7f59bc009bb0 comp rx=0 tx=0).stop 2023-07-21T18:33:31.458 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:30.744+0000 7f59d1176700 1 -- 172.21.15.39:0/876923426 shutdown_connections 2023-07-21T18:33:31.459 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:30.744+0000 7f59d1176700 1 --2- 172.21.15.39:0/876923426 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59cc106050 0x7f59cc108430 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:33:31.459 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:30.744+0000 7f59d1176700 1 -- 172.21.15.39:0/876923426 >> 172.21.15.39:0/876923426 conn(0x7f59cc0ffa50 msgr2=0x7f59cc101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:33:31.459 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:30.744+0000 7f59d1176700 1 -- 172.21.15.39:0/876923426 shutdown_connections 2023-07-21T18:33:31.459 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:30.745+0000 7f59d1176700 1 -- 172.21.15.39:0/876923426 wait complete. 2023-07-21T18:33:31.459 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:30.745+0000 7f59d1176700 1 Processor -- start 2023-07-21T18:33:31.459 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:30.746+0000 7f59d1176700 1 -- start start 2023-07-21T18:33:31.459 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:30.746+0000 7f59d1176700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59cc106050 0x7f59cc199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:33:31.460 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:30.746+0000 7f59d1176700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59cc199790 con 0x7f59cc106050 2023-07-21T18:33:31.460 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:30.746+0000 7f59cad9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59cc106050 0x7f59cc199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:33:31.460 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:30.746+0000 7f59cad9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59cc106050 0x7f59cc199250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53740/0 (socket says 172.21.15.39:53740) 2023-07-21T18:33:31.461 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:30.746+0000 7f59cad9d700 1 -- 172.21.15.39:0/2860215723 learned_addr learned my addr 172.21.15.39:0/2860215723 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:33:31.461 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:30.746+0000 7f59cad9d700 1 -- 172.21.15.39:0/2860215723 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59bc009550 con 0x7f59cc106050 2023-07-21T18:33:31.461 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:30.746+0000 7f59cad9d700 1 --2- 172.21.15.39:0/2860215723 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59cc106050 0x7f59cc199250 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f59bc009fb0 tx=0x7f59bc011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:33:31.462 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:30.747+0000 7f59b3fff700 1 -- 172.21.15.39:0/2860215723 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f59bc003a40 con 0x7f59cc106050 2023-07-21T18:33:31.462 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:30.747+0000 7f59b3fff700 1 -- 172.21.15.39:0/2860215723 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f59bc003ba0 con 0x7f59cc106050 2023-07-21T18:33:31.462 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:30.747+0000 7f59d1176700 1 -- 172.21.15.39:0/2860215723 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59cc1999f0 con 0x7f59cc106050 2023-07-21T18:33:31.463 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:30.747+0000 7f59b3fff700 1 -- 172.21.15.39:0/2860215723 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f59bc01a750 con 0x7f59cc106050 2023-07-21T18:33:31.463 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:30.747+0000 7f59d1176700 1 -- 172.21.15.39:0/2860215723 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59cc199e70 con 0x7f59cc106050 2023-07-21T18:33:31.463 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:30.748+0000 7f59b3fff700 1 -- 172.21.15.39:0/2860215723 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 4) v1 ==== 45472+0+0 (secure 0 0 0) 0x7f59bc01a980 con 0x7f59cc106050 2023-07-21T18:33:31.463 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:30.748+0000 7f59b3fff700 1 --2- 172.21.15.39:0/2860215723 >> [v2:172.21.15.39:6800/889516631,v1:172.21.15.39:6801/889516631] conn(0x7f59b40386c0 0x7f59b403ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:33:31.464 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:30.748+0000 7f59b3fff700 1 -- 172.21.15.39:0/2860215723 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f59bc04c290 con 0x7f59cc106050 2023-07-21T18:33:31.464 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:30.748+0000 7f59d1176700 1 -- 172.21.15.39:0/2860215723 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59cc109570 con 0x7f59cc106050 2023-07-21T18:33:31.464 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:30.748+0000 7f59ca59c700 1 --2- 172.21.15.39:0/2860215723 >> [v2:172.21.15.39:6800/889516631,v1:172.21.15.39:6801/889516631] conn(0x7f59b40386c0 0x7f59b403ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:33:31.465 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:30.748+0000 7f59ca59c700 1 --2- 172.21.15.39:0/2860215723 >> [v2:172.21.15.39:6800/889516631,v1:172.21.15.39:6801/889516631] conn(0x7f59b40386c0 0x7f59b403ab70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f59c0006fd0 tx=0x7f59c0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:33:31.465 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:30.753+0000 7f59b3fff700 1 -- 172.21.15.39:0/2860215723 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f59bc018db0 con 0x7f59cc106050 2023-07-21T18:33:31.465 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:31.106+0000 7f59d1176700 1 -- 172.21.15.39:0/2860215723 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f59cc19cc10 con 0x7f59cc106050 2023-07-21T18:33:31.465 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:31.107+0000 7f59b3fff700 1 -- 172.21.15.39:0/2860215723 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f59bc018350 con 0x7f59cc106050 2023-07-21T18:33:31.465 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:31.109+0000 7f59d1176700 1 -- 172.21.15.39:0/2860215723 >> [v2:172.21.15.39:6800/889516631,v1:172.21.15.39:6801/889516631] conn(0x7f59b40386c0 msgr2=0x7f59b403ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:33:31.466 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:31.109+0000 7f59d1176700 1 --2- 172.21.15.39:0/2860215723 >> [v2:172.21.15.39:6800/889516631,v1:172.21.15.39:6801/889516631] conn(0x7f59b40386c0 0x7f59b403ab70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f59c0006fd0 tx=0x7f59c0006e40 comp rx=0 tx=0).stop 2023-07-21T18:33:31.466 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:31.109+0000 7f59d1176700 1 -- 172.21.15.39:0/2860215723 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59cc106050 msgr2=0x7f59cc199250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:33:31.466 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:31.109+0000 7f59d1176700 1 --2- 172.21.15.39:0/2860215723 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59cc106050 0x7f59cc199250 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f59bc009fb0 tx=0x7f59bc011ad0 comp rx=0 tx=0).stop 2023-07-21T18:33:31.466 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:31.110+0000 7f59d1176700 1 -- 172.21.15.39:0/2860215723 shutdown_connections 2023-07-21T18:33:31.466 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:31.110+0000 7f59d1176700 1 --2- 172.21.15.39:0/2860215723 >> [v2:172.21.15.39:6800/889516631,v1:172.21.15.39:6801/889516631] conn(0x7f59b40386c0 0x7f59b403ab70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:33:31.466 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:31.110+0000 7f59d1176700 1 --2- 172.21.15.39:0/2860215723 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59cc106050 0x7f59cc199250 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:33:31.466 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:31.110+0000 7f59d1176700 1 -- 172.21.15.39:0/2860215723 >> 172.21.15.39:0/2860215723 conn(0x7f59cc0ffa50 msgr2=0x7f59cc101120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:33:31.466 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:31.110+0000 7f59d1176700 1 -- 172.21.15.39:0/2860215723 shutdown_connections 2023-07-21T18:33:31.467 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:31.110+0000 7f59d1176700 1 -- 172.21.15.39:0/2860215723 wait complete. 2023-07-21T18:33:31.467 INFO:teuthology.orchestra.run.smithi039.stdout:Enabling cephadm module... 2023-07-21T18:33:32.199 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:31 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/2860215723' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-07-21T18:33:33.186 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:32 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/2885530486' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2023-07-21T18:33:33.186 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:32 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: ignoring --setuser ceph since I am not root 2023-07-21T18:33:33.187 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:32 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: ignoring --setgroup ceph since I am not root 2023-07-21T18:33:33.187 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:33 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:33:33.055+0000 7ff841b51200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-21T18:33:33.294 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:31.849+0000 7f8ead5b0700 1 Processor -- start 2023-07-21T18:33:33.294 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:31.849+0000 7f8ead5b0700 1 -- start start 2023-07-21T18:33:33.294 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:31.849+0000 7f8ead5b0700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ea8073ba0 0x7f8ea810a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:33:33.295 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:31.849+0000 7f8ead5b0700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea8105e80 con 0x7f8ea8073ba0 2023-07-21T18:33:33.295 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:31.849+0000 7f8ea6ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ea8073ba0 0x7f8ea810a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:33:33.295 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:31.849+0000 7f8ea6ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ea8073ba0 0x7f8ea810a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53744/0 (socket says 172.21.15.39:53744) 2023-07-21T18:33:33.295 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:31.849+0000 7f8ea6ffd700 1 -- 172.21.15.39:0/1260929231 learned_addr learned my addr 172.21.15.39:0/1260929231 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:33:33.295 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:31.850+0000 7f8ea6ffd700 1 -- 172.21.15.39:0/1260929231 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ea810a9d0 con 0x7f8ea8073ba0 2023-07-21T18:33:33.298 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:31.850+0000 7f8ea6ffd700 1 --2- 172.21.15.39:0/1260929231 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ea8073ba0 0x7f8ea810a490 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c0098a0 tx=0x7f8e9c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccc23ee29657b93 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:33:33.299 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:31.850+0000 7f8ea5ffb700 1 -- 172.21.15.39:0/1260929231 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e9c004160 con 0x7f8ea8073ba0 2023-07-21T18:33:33.299 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:31.850+0000 7f8ea5ffb700 1 -- 172.21.15.39:0/1260929231 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8e9c0042c0 con 0x7f8ea8073ba0 2023-07-21T18:33:33.299 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:31.850+0000 7f8ea5ffb700 1 -- 172.21.15.39:0/1260929231 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e9c011a00 con 0x7f8ea8073ba0 2023-07-21T18:33:33.299 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:31.851+0000 7f8ead5b0700 1 -- 172.21.15.39:0/1260929231 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ea8073ba0 msgr2=0x7f8ea810a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:33:33.299 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:31.851+0000 7f8ead5b0700 1 --2- 172.21.15.39:0/1260929231 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ea8073ba0 0x7f8ea810a490 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c0098a0 tx=0x7f8e9c009bb0 comp rx=0 tx=0).stop 2023-07-21T18:33:33.299 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:31.851+0000 7f8ead5b0700 1 -- 172.21.15.39:0/1260929231 shutdown_connections 2023-07-21T18:33:33.299 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:31.851+0000 7f8ead5b0700 1 --2- 172.21.15.39:0/1260929231 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ea8073ba0 0x7f8ea810a490 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:33:33.300 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:31.851+0000 7f8ead5b0700 1 -- 172.21.15.39:0/1260929231 >> 172.21.15.39:0/1260929231 conn(0x7f8ea80ffa40 msgr2=0x7f8ea8101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:33:33.300 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:31.851+0000 7f8ead5b0700 1 -- 172.21.15.39:0/1260929231 shutdown_connections 2023-07-21T18:33:33.300 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:31.851+0000 7f8ead5b0700 1 -- 172.21.15.39:0/1260929231 wait complete. 2023-07-21T18:33:33.300 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:31.851+0000 7f8ead5b0700 1 Processor -- start 2023-07-21T18:33:33.300 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:31.852+0000 7f8ead5b0700 1 -- start start 2023-07-21T18:33:33.300 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:31.852+0000 7f8ead5b0700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ea819b400 0x7f8ea819b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:33:33.301 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:31.852+0000 7f8ead5b0700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea819bd50 con 0x7f8ea819b400 2023-07-21T18:33:33.301 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:31.852+0000 7f8ea6ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ea819b400 0x7f8ea819b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:33:33.301 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:31.852+0000 7f8ea6ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ea819b400 0x7f8ea819b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53746/0 (socket says 172.21.15.39:53746) 2023-07-21T18:33:33.301 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:31.852+0000 7f8ea6ffd700 1 -- 172.21.15.39:0/2885530486 learned_addr learned my addr 172.21.15.39:0/2885530486 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:33:33.301 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:31.852+0000 7f8ea6ffd700 1 -- 172.21.15.39:0/2885530486 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e9c009550 con 0x7f8ea819b400 2023-07-21T18:33:33.301 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:31.852+0000 7f8ea6ffd700 1 --2- 172.21.15.39:0/2885530486 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ea819b400 0x7f8ea819b810 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c006af0 tx=0x7f8e9c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:33:33.301 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:31.852+0000 7f8e8ffff700 1 -- 172.21.15.39:0/2885530486 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e9c003a40 con 0x7f8ea819b400 2023-07-21T18:33:33.302 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:31.852+0000 7f8e8ffff700 1 -- 172.21.15.39:0/2885530486 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8e9c003ba0 con 0x7f8ea819b400 2023-07-21T18:33:33.302 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:31.852+0000 7f8ead5b0700 1 -- 172.21.15.39:0/2885530486 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ea819bf50 con 0x7f8ea819b400 2023-07-21T18:33:33.302 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:31.852+0000 7f8e8ffff700 1 -- 172.21.15.39:0/2885530486 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e9c01a700 con 0x7f8ea819b400 2023-07-21T18:33:33.302 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:31.852+0000 7f8ead5b0700 1 -- 172.21.15.39:0/2885530486 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ea819ebb0 con 0x7f8ea819b400 2023-07-21T18:33:33.302 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:31.853+0000 7f8e8ffff700 1 -- 172.21.15.39:0/2885530486 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 4) v1 ==== 45472+0+0 (secure 0 0 0) 0x7f8e9c024ad0 con 0x7f8ea819b400 2023-07-21T18:33:33.302 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:31.854+0000 7f8e8ffff700 1 --2- 172.21.15.39:0/2885530486 >> [v2:172.21.15.39:6800/889516631,v1:172.21.15.39:6801/889516631] conn(0x7f8e90038710 0x7f8e9003abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:33:33.302 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:31.854+0000 7f8ead5b0700 1 -- 172.21.15.39:0/2885530486 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ea8073ba0 con 0x7f8ea819b400 2023-07-21T18:33:33.302 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:31.854+0000 7f8e8ffff700 1 -- 172.21.15.39:0/2885530486 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f8e9c04c600 con 0x7f8ea819b400 2023-07-21T18:33:33.303 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:31.854+0000 7f8ea67fc700 1 --2- 172.21.15.39:0/2885530486 >> [v2:172.21.15.39:6800/889516631,v1:172.21.15.39:6801/889516631] conn(0x7f8e90038710 0x7f8e9003abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:33:33.303 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:31.854+0000 7f8ea67fc700 1 --2- 172.21.15.39:0/2885530486 >> [v2:172.21.15.39:6800/889516631,v1:172.21.15.39:6801/889516631] conn(0x7f8e90038710 0x7f8e9003abc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8e98006fd0 tx=0x7f8e98006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:33:33.303 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:31.858+0000 7f8e8ffff700 1 -- 172.21.15.39:0/2885530486 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f8e9c02d3f0 con 0x7f8ea819b400 2023-07-21T18:33:33.303 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:32.323+0000 7f8ead5b0700 1 -- 172.21.15.39:0/2885530486 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f8ea8073d90 con 0x7f8ea819b400 2023-07-21T18:33:33.303 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:32.858+0000 7f8e8ffff700 1 -- 172.21.15.39:0/2885530486 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mgrmap(e 5) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f8e9c02d770 con 0x7f8ea819b400 2023-07-21T18:33:33.303 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:32.860+0000 7f8e8ffff700 1 -- 172.21.15.39:0/2885530486 <== mon.0 v2:172.21.15.39:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f8e9c018dd0 con 0x7f8ea819b400 2023-07-21T18:33:33.303 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:32.863+0000 7f8ead5b0700 1 -- 172.21.15.39:0/2885530486 >> [v2:172.21.15.39:6800/889516631,v1:172.21.15.39:6801/889516631] conn(0x7f8e90038710 msgr2=0x7f8e9003abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:33:33.304 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:32.863+0000 7f8ead5b0700 1 --2- 172.21.15.39:0/2885530486 >> [v2:172.21.15.39:6800/889516631,v1:172.21.15.39:6801/889516631] conn(0x7f8e90038710 0x7f8e9003abc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8e98006fd0 tx=0x7f8e98006e40 comp rx=0 tx=0).stop 2023-07-21T18:33:33.304 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:32.863+0000 7f8ead5b0700 1 -- 172.21.15.39:0/2885530486 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ea819b400 msgr2=0x7f8ea819b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:33:33.304 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:32.863+0000 7f8ead5b0700 1 --2- 172.21.15.39:0/2885530486 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ea819b400 0x7f8ea819b810 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c006af0 tx=0x7f8e9c011ad0 comp rx=0 tx=0).stop 2023-07-21T18:33:33.304 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:32.863+0000 7f8ead5b0700 1 -- 172.21.15.39:0/2885530486 shutdown_connections 2023-07-21T18:33:33.304 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:32.863+0000 7f8ead5b0700 1 --2- 172.21.15.39:0/2885530486 >> [v2:172.21.15.39:6800/889516631,v1:172.21.15.39:6801/889516631] conn(0x7f8e90038710 0x7f8e9003abc0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:33:33.304 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:32.863+0000 7f8ead5b0700 1 --2- 172.21.15.39:0/2885530486 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ea819b400 0x7f8ea819b810 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:33:33.304 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:32.863+0000 7f8ead5b0700 1 -- 172.21.15.39:0/2885530486 >> 172.21.15.39:0/2885530486 conn(0x7f8ea80ffa40 msgr2=0x7f8ea81010c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:33:33.304 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:32.863+0000 7f8ead5b0700 1 -- 172.21.15.39:0/2885530486 shutdown_connections 2023-07-21T18:33:33.305 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:32.863+0000 7f8ead5b0700 1 -- 172.21.15.39:0/2885530486 wait complete. 2023-07-21T18:33:33.449 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:33 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:33:33.151+0000 7ff841b51200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-21T18:33:33.449 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:33 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:33:33.261+0000 7ff841b51200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-21T18:33:33.450 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:33 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:33:33.388+0000 7ff841b51200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-21T18:33:33.896 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:33 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:33:33.599+0000 7ff841b51200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-21T18:33:34.199 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:33 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/2885530486' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2023-07-21T18:33:34.199 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:33 smithi039 ceph-mon[96489]: mgrmap e5: x(active, since 5s) 2023-07-21T18:33:34.200 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:33 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:33:33.997+0000 7ff841b51200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-21T18:33:34.200 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:34 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:33:34.106+0000 7ff841b51200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-21T18:33:34.659 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout { 2023-07-21T18:33:34.659 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 5, 2023-07-21T18:33:34.660 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "available": true, 2023-07-21T18:33:34.660 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-07-21T18:33:34.660 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-07-21T18:33:34.660 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout } 2023-07-21T18:33:34.660 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:33.774+0000 7f7b803ac700 1 Processor -- start 2023-07-21T18:33:34.660 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:33.774+0000 7f7b803ac700 1 -- start start 2023-07-21T18:33:34.661 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:33.775+0000 7f7b803ac700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b78106010 0x7f7b781083f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:33:34.661 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:33.775+0000 7f7b803ac700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b78073c20 con 0x7f7b78106010 2023-07-21T18:33:34.661 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:33.775+0000 7f7b7e148700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b78106010 0x7f7b781083f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:33:34.661 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:33.775+0000 7f7b7e148700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b78106010 0x7f7b781083f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53760/0 (socket says 172.21.15.39:53760) 2023-07-21T18:33:34.661 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:33.775+0000 7f7b7e148700 1 -- 172.21.15.39:0/2996195889 learned_addr learned my addr 172.21.15.39:0/2996195889 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:33:34.661 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:33.775+0000 7f7b7e148700 1 -- 172.21.15.39:0/2996195889 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b78073d60 con 0x7f7b78106010 2023-07-21T18:33:34.662 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:33.775+0000 7f7b7e148700 1 --2- 172.21.15.39:0/2996195889 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b78106010 0x7f7b781083f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7b740098a0 tx=0x7f7b74009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c922be40eb69a098 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:33:34.662 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:33.775+0000 7f7b7d146700 1 -- 172.21.15.39:0/2996195889 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7b74004160 con 0x7f7b78106010 2023-07-21T18:33:34.662 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:33.775+0000 7f7b7d146700 1 -- 172.21.15.39:0/2996195889 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7b740042c0 con 0x7f7b78106010 2023-07-21T18:33:34.662 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:33.775+0000 7f7b7d146700 1 -- 172.21.15.39:0/2996195889 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7b74011a00 con 0x7f7b78106010 2023-07-21T18:33:34.662 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:33.776+0000 7f7b803ac700 1 -- 172.21.15.39:0/2996195889 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b78106010 msgr2=0x7f7b781083f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:33:34.663 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:33.776+0000 7f7b803ac700 1 --2- 172.21.15.39:0/2996195889 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b78106010 0x7f7b781083f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7b740098a0 tx=0x7f7b74009bb0 comp rx=0 tx=0).stop 2023-07-21T18:33:34.663 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:33.776+0000 7f7b803ac700 1 -- 172.21.15.39:0/2996195889 shutdown_connections 2023-07-21T18:33:34.663 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:33.776+0000 7f7b803ac700 1 --2- 172.21.15.39:0/2996195889 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b78106010 0x7f7b781083f0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:33:34.663 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:33.776+0000 7f7b803ac700 1 -- 172.21.15.39:0/2996195889 >> 172.21.15.39:0/2996195889 conn(0x7f7b780ffa50 msgr2=0x7f7b78101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:33:34.663 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:33.776+0000 7f7b803ac700 1 -- 172.21.15.39:0/2996195889 shutdown_connections 2023-07-21T18:33:34.663 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:33.776+0000 7f7b803ac700 1 -- 172.21.15.39:0/2996195889 wait complete. 2023-07-21T18:33:34.664 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:33.777+0000 7f7b803ac700 1 Processor -- start 2023-07-21T18:33:34.664 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:33.777+0000 7f7b803ac700 1 -- start start 2023-07-21T18:33:34.664 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:33.777+0000 7f7b803ac700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b78106010 0x7f7b78199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:33:34.664 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:33.777+0000 7f7b803ac700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b781997c0 con 0x7f7b78106010 2023-07-21T18:33:34.664 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:33.777+0000 7f7b7e148700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b78106010 0x7f7b78199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:33:34.665 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:33.777+0000 7f7b7e148700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b78106010 0x7f7b78199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53762/0 (socket says 172.21.15.39:53762) 2023-07-21T18:33:34.665 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:33.777+0000 7f7b7e148700 1 -- 172.21.15.39:0/4144486050 learned_addr learned my addr 172.21.15.39:0/4144486050 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:33:34.665 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:33.777+0000 7f7b7e148700 1 -- 172.21.15.39:0/4144486050 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b74009550 con 0x7f7b78106010 2023-07-21T18:33:34.665 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:33.778+0000 7f7b7e148700 1 --2- 172.21.15.39:0/4144486050 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b78106010 0x7f7b78199280 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7b74009130 tx=0x7f7b74011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:33:34.665 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:33.778+0000 7f7b6f7fe700 1 -- 172.21.15.39:0/4144486050 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7b74003ab0 con 0x7f7b78106010 2023-07-21T18:33:34.665 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:33.778+0000 7f7b6f7fe700 1 -- 172.21.15.39:0/4144486050 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7b74003c10 con 0x7f7b78106010 2023-07-21T18:33:34.665 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:33.778+0000 7f7b6f7fe700 1 -- 172.21.15.39:0/4144486050 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7b740247d0 con 0x7f7b78106010 2023-07-21T18:33:34.666 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:33.778+0000 7f7b803ac700 1 -- 172.21.15.39:0/4144486050 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b781999c0 con 0x7f7b78106010 2023-07-21T18:33:34.666 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:33.778+0000 7f7b803ac700 1 -- 172.21.15.39:0/4144486050 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b78199de0 con 0x7f7b78106010 2023-07-21T18:33:34.666 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:33.779+0000 7f7b6f7fe700 1 -- 172.21.15.39:0/4144486050 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f7b7402d3f0 con 0x7f7b78106010 2023-07-21T18:33:34.666 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:33.779+0000 7f7b6f7fe700 1 --2- 172.21.15.39:0/4144486050 >> [v2:172.21.15.39:6800/889516631,v1:172.21.15.39:6801/889516631] conn(0x7f7b64038760 0x7f7b6403ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:33:34.666 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:33.779+0000 7f7b6f7fe700 1 -- 172.21.15.39:0/4144486050 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f7b7404c470 con 0x7f7b78106010 2023-07-21T18:33:34.666 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:33.779+0000 7f7b7d947700 1 -- 172.21.15.39:0/4144486050 >> [v2:172.21.15.39:6800/889516631,v1:172.21.15.39:6801/889516631] conn(0x7f7b64038760 msgr2=0x7f7b6403ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/889516631 2023-07-21T18:33:34.666 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:33.779+0000 7f7b7d947700 1 --2- 172.21.15.39:0/4144486050 >> [v2:172.21.15.39:6800/889516631,v1:172.21.15.39:6801/889516631] conn(0x7f7b64038760 0x7f7b6403ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-21T18:33:34.667 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:33.779+0000 7f7b803ac700 1 -- 172.21.15.39:0/4144486050 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b78061b40 con 0x7f7b78106010 2023-07-21T18:33:34.667 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:33.784+0000 7f7b6f7fe700 1 -- 172.21.15.39:0/4144486050 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f7b74010970 con 0x7f7b78106010 2023-07-21T18:33:34.667 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:33.979+0000 7f7b7d947700 1 -- 172.21.15.39:0/4144486050 >> [v2:172.21.15.39:6800/889516631,v1:172.21.15.39:6801/889516631] conn(0x7f7b64038760 msgr2=0x7f7b6403ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/889516631 2023-07-21T18:33:34.667 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:33.979+0000 7f7b7d947700 1 --2- 172.21.15.39:0/4144486050 >> [v2:172.21.15.39:6800/889516631,v1:172.21.15.39:6801/889516631] conn(0x7f7b64038760 0x7f7b6403ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-21T18:33:34.667 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:34.244+0000 7f7b803ac700 1 -- 172.21.15.39:0/4144486050 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f7b7819c910 con 0x7f7b78106010 2023-07-21T18:33:34.667 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:34.245+0000 7f7b6f7fe700 1 -- 172.21.15.39:0/4144486050 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f7b74020220 con 0x7f7b78106010 2023-07-21T18:33:34.667 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:34.249+0000 7f7b803ac700 1 -- 172.21.15.39:0/4144486050 >> [v2:172.21.15.39:6800/889516631,v1:172.21.15.39:6801/889516631] conn(0x7f7b64038760 msgr2=0x7f7b6403ac10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-21T18:33:34.668 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:34.249+0000 7f7b803ac700 1 --2- 172.21.15.39:0/4144486050 >> [v2:172.21.15.39:6800/889516631,v1:172.21.15.39:6801/889516631] conn(0x7f7b64038760 0x7f7b6403ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:33:34.668 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:34.249+0000 7f7b803ac700 1 -- 172.21.15.39:0/4144486050 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b78106010 msgr2=0x7f7b78199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:33:34.668 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:34.249+0000 7f7b803ac700 1 --2- 172.21.15.39:0/4144486050 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b78106010 0x7f7b78199280 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7b74009130 tx=0x7f7b74011ad0 comp rx=0 tx=0).stop 2023-07-21T18:33:34.668 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:34.249+0000 7f7b803ac700 1 -- 172.21.15.39:0/4144486050 shutdown_connections 2023-07-21T18:33:34.668 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:34.249+0000 7f7b803ac700 1 --2- 172.21.15.39:0/4144486050 >> [v2:172.21.15.39:6800/889516631,v1:172.21.15.39:6801/889516631] conn(0x7f7b64038760 0x7f7b6403ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:33:34.668 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:34.249+0000 7f7b803ac700 1 --2- 172.21.15.39:0/4144486050 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b78106010 0x7f7b78199280 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:33:34.668 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:34.249+0000 7f7b803ac700 1 -- 172.21.15.39:0/4144486050 >> 172.21.15.39:0/4144486050 conn(0x7f7b780ffa50 msgr2=0x7f7b78101140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:33:34.669 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:34.249+0000 7f7b803ac700 1 -- 172.21.15.39:0/4144486050 shutdown_connections 2023-07-21T18:33:34.669 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:34.249+0000 7f7b803ac700 1 -- 172.21.15.39:0/4144486050 wait complete. 2023-07-21T18:33:34.669 INFO:teuthology.orchestra.run.smithi039.stdout:Waiting for the mgr to restart... 2023-07-21T18:33:34.669 INFO:teuthology.orchestra.run.smithi039.stdout:Waiting for mgr epoch 5... 2023-07-21T18:33:34.670 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:34 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:33:34.262+0000 7ff841b51200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-21T18:33:34.670 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:34 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:33:34.386+0000 7ff841b51200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-21T18:33:34.922 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:34 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/4144486050' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-07-21T18:33:35.199 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:34 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:33:34.911+0000 7ff841b51200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-21T18:33:35.949 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:35 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:33:35.595+0000 7ff841b51200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-21T18:33:35.950 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:35 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:33:35.787+0000 7ff841b51200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-21T18:33:37.107 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:36 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:33:36.744+0000 7ff841b51200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-21T18:33:37.107 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:36 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:33:36.854+0000 7ff841b51200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-21T18:33:37.449 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:37 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:33:37.071+0000 7ff841b51200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-21T18:33:37.702 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:37 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:33:37.416+0000 7ff841b51200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-21T18:33:38.199 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:37 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:33:37.667+0000 7ff841b51200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-21T18:33:38.199 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:37 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:33:37.848+0000 7ff841b51200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-21T18:33:38.699 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:38 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:33:38.174+0000 7ff841b51200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-21T18:33:39.424 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:38 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:33:38.989+0000 7ff841b51200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-21T18:33:39.425 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:39 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:33:39.148+0000 7ff841b51200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-21T18:33:39.699 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:39 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:33:39.389+0000 7ff841b51200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-21T18:33:39.699 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:39 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:33:39.529+0000 7ff841b51200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-21T18:33:40.199 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:39 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:33:39.799+0000 7ff841b51200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-21T18:33:41.199 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:40 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:33:40.837+0000 7ff841b51200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-21T18:33:42.200 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:41 smithi039 ceph-mon[96489]: Active manager daemon x restarted 2023-07-21T18:33:42.200 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:41 smithi039 ceph-mon[96489]: Activating manager daemon x 2023-07-21T18:33:42.200 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:41 smithi039 ceph-mon[96489]: osdmap e2: 0 total, 0 up, 0 in 2023-07-21T18:33:42.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:41 smithi039 ceph-mon[96489]: mgrmap e6: x(active, starting, since 0.00670994s) 2023-07-21T18:33:42.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:41 smithi039 ceph-mon[96489]: from='mgr.14120 172.21.15.39:0/2168989012' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T18:33:42.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:41 smithi039 ceph-mon[96489]: from='mgr.14120 172.21.15.39:0/2168989012' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-07-21T18:33:42.202 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:41 smithi039 ceph-mon[96489]: from='mgr.14120 172.21.15.39:0/2168989012' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-07-21T18:33:42.202 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:41 smithi039 ceph-mon[96489]: from='mgr.14120 172.21.15.39:0/2168989012' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-07-21T18:33:42.202 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:41 smithi039 ceph-mon[96489]: from='mgr.14120 172.21.15.39:0/2168989012' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-07-21T18:33:42.202 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:41 smithi039 ceph-mon[96489]: Manager daemon x is now available 2023-07-21T18:33:42.203 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:41 smithi039 ceph-mon[96489]: Found migration_current of "None". Setting to last migration. 2023-07-21T18:33:42.203 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:41 smithi039 ceph-mon[96489]: from='mgr.14120 172.21.15.39:0/2168989012' entity='mgr.x' 2023-07-21T18:33:42.203 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:41 smithi039 ceph-mon[96489]: from='mgr.14120 172.21.15.39:0/2168989012' entity='mgr.x' 2023-07-21T18:33:42.204 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:41 smithi039 ceph-mon[96489]: from='mgr.14120 172.21.15.39:0/2168989012' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:33:42.204 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:41 smithi039 ceph-mon[96489]: from='mgr.14120 172.21.15.39:0/2168989012' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:33:42.204 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:41 smithi039 ceph-mon[96489]: from='mgr.14120 172.21.15.39:0/2168989012' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:33:42.204 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:41 smithi039 ceph-mon[96489]: from='mgr.14120 172.21.15.39:0/2168989012' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-21T18:33:42.204 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:41 smithi039 ceph-mon[96489]: from='mgr.14120 172.21.15.39:0/2168989012' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-21T18:33:42.320 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout { 2023-07-21T18:33:42.321 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2023-07-21T18:33:42.321 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "initialized": true 2023-07-21T18:33:42.321 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout } 2023-07-21T18:33:42.321 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:35.125+0000 7f744c446700 1 Processor -- start 2023-07-21T18:33:42.321 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:35.125+0000 7f744c446700 1 -- start start 2023-07-21T18:33:42.321 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:35.126+0000 7f744c446700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7444073ba0 0x7f74441080a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:33:42.322 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:35.126+0000 7f744c446700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74441085e0 con 0x7f7444073ba0 2023-07-21T18:33:42.322 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:35.126+0000 7f744a1e2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7444073ba0 0x7f74441080a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:33:42.322 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:35.126+0000 7f744a1e2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7444073ba0 0x7f74441080a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53770/0 (socket says 172.21.15.39:53770) 2023-07-21T18:33:42.322 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:35.126+0000 7f744a1e2700 1 -- 172.21.15.39:0/2774796587 learned_addr learned my addr 172.21.15.39:0/2774796587 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:33:42.322 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:35.126+0000 7f744a1e2700 1 -- 172.21.15.39:0/2774796587 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7444108720 con 0x7f7444073ba0 2023-07-21T18:33:42.322 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:35.127+0000 7f744a1e2700 1 --2- 172.21.15.39:0/2774796587 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7444073ba0 0x7f74441080a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f74400098a0 tx=0x7f7440009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f19fb0408f5dbe20 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:33:42.322 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:35.127+0000 7f74491e0700 1 -- 172.21.15.39:0/2774796587 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7440004160 con 0x7f7444073ba0 2023-07-21T18:33:42.323 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:35.127+0000 7f74491e0700 1 -- 172.21.15.39:0/2774796587 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f74400042c0 con 0x7f7444073ba0 2023-07-21T18:33:42.323 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:35.127+0000 7f74491e0700 1 -- 172.21.15.39:0/2774796587 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7440011a00 con 0x7f7444073ba0 2023-07-21T18:33:42.323 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:35.128+0000 7f744c446700 1 -- 172.21.15.39:0/2774796587 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7444073ba0 msgr2=0x7f74441080a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:33:42.323 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:35.128+0000 7f744c446700 1 --2- 172.21.15.39:0/2774796587 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7444073ba0 0x7f74441080a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f74400098a0 tx=0x7f7440009bb0 comp rx=0 tx=0).stop 2023-07-21T18:33:42.323 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:35.128+0000 7f744c446700 1 -- 172.21.15.39:0/2774796587 shutdown_connections 2023-07-21T18:33:42.323 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:35.128+0000 7f744c446700 1 --2- 172.21.15.39:0/2774796587 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7444073ba0 0x7f74441080a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:33:42.323 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:35.128+0000 7f744c446700 1 -- 172.21.15.39:0/2774796587 >> 172.21.15.39:0/2774796587 conn(0x7f74440ffa50 msgr2=0x7f7444101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:33:42.323 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:35.128+0000 7f744c446700 1 -- 172.21.15.39:0/2774796587 shutdown_connections 2023-07-21T18:33:42.324 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:35.128+0000 7f744c446700 1 -- 172.21.15.39:0/2774796587 wait complete. 2023-07-21T18:33:42.324 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:35.129+0000 7f744c446700 1 Processor -- start 2023-07-21T18:33:42.324 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:35.129+0000 7f744c446700 1 -- start start 2023-07-21T18:33:42.324 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:35.129+0000 7f744c446700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74441059b0 0x7f7444104040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:33:42.325 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:35.129+0000 7f744c446700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7444105dc0 con 0x7f74441059b0 2023-07-21T18:33:42.325 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:35.129+0000 7f744a1e2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74441059b0 0x7f7444104040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:33:42.325 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:35.129+0000 7f744a1e2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74441059b0 0x7f7444104040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53772/0 (socket says 172.21.15.39:53772) 2023-07-21T18:33:42.325 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:35.129+0000 7f744a1e2700 1 -- 172.21.15.39:0/4035116593 learned_addr learned my addr 172.21.15.39:0/4035116593 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:33:42.326 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:35.130+0000 7f744a1e2700 1 -- 172.21.15.39:0/4035116593 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7440009550 con 0x7f74441059b0 2023-07-21T18:33:42.326 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:35.130+0000 7f744a1e2700 1 --2- 172.21.15.39:0/4035116593 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74441059b0 0x7f7444104040 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7440009130 tx=0x7f7440011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:33:42.326 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:35.130+0000 7f743b7fe700 1 -- 172.21.15.39:0/4035116593 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7440003ab0 con 0x7f74441059b0 2023-07-21T18:33:42.326 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:35.130+0000 7f743b7fe700 1 -- 172.21.15.39:0/4035116593 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7440003c10 con 0x7f74441059b0 2023-07-21T18:33:42.327 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:35.130+0000 7f743b7fe700 1 -- 172.21.15.39:0/4035116593 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f74400247d0 con 0x7f74441059b0 2023-07-21T18:33:42.327 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:35.130+0000 7f744c446700 1 -- 172.21.15.39:0/4035116593 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7444104580 con 0x7f74441059b0 2023-07-21T18:33:42.327 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:35.130+0000 7f744c446700 1 -- 172.21.15.39:0/4035116593 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74441049a0 con 0x7f74441059b0 2023-07-21T18:33:42.327 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:35.131+0000 7f743b7fe700 1 -- 172.21.15.39:0/4035116593 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f744001a430 con 0x7f74441059b0 2023-07-21T18:33:42.327 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:35.131+0000 7f743b7fe700 1 --2- 172.21.15.39:0/4035116593 >> [v2:172.21.15.39:6800/889516631,v1:172.21.15.39:6801/889516631] conn(0x7f7430038710 0x7f743003abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:33:42.327 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:35.131+0000 7f743b7fe700 1 -- 172.21.15.39:0/4035116593 --> [v2:172.21.15.39:6800/889516631,v1:172.21.15.39:6801/889516631] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f743003b2d0 con 0x7f7430038710 2023-07-21T18:33:42.328 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:35.131+0000 7f743b7fe700 1 -- 172.21.15.39:0/4035116593 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f744004c4b0 con 0x7f74441059b0 2023-07-21T18:33:42.328 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:35.131+0000 7f74499e1700 1 -- 172.21.15.39:0/4035116593 >> [v2:172.21.15.39:6800/889516631,v1:172.21.15.39:6801/889516631] conn(0x7f7430038710 msgr2=0x7f743003abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/889516631 2023-07-21T18:33:42.328 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:35.131+0000 7f74499e1700 1 --2- 172.21.15.39:0/4035116593 >> [v2:172.21.15.39:6800/889516631,v1:172.21.15.39:6801/889516631] conn(0x7f7430038710 0x7f743003abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-21T18:33:42.328 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:35.332+0000 7f74499e1700 1 -- 172.21.15.39:0/4035116593 >> [v2:172.21.15.39:6800/889516631,v1:172.21.15.39:6801/889516631] conn(0x7f7430038710 msgr2=0x7f743003abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/889516631 2023-07-21T18:33:42.328 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:35.332+0000 7f74499e1700 1 --2- 172.21.15.39:0/4035116593 >> [v2:172.21.15.39:6800/889516631,v1:172.21.15.39:6801/889516631] conn(0x7f7430038710 0x7f743003abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-21T18:33:42.329 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:35.733+0000 7f74499e1700 1 -- 172.21.15.39:0/4035116593 >> [v2:172.21.15.39:6800/889516631,v1:172.21.15.39:6801/889516631] conn(0x7f7430038710 msgr2=0x7f743003abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/889516631 2023-07-21T18:33:42.329 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:35.733+0000 7f74499e1700 1 --2- 172.21.15.39:0/4035116593 >> [v2:172.21.15.39:6800/889516631,v1:172.21.15.39:6801/889516631] conn(0x7f7430038710 0x7f743003abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-07-21T18:33:42.329 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:36.534+0000 7f74499e1700 1 -- 172.21.15.39:0/4035116593 >> [v2:172.21.15.39:6800/889516631,v1:172.21.15.39:6801/889516631] conn(0x7f7430038710 msgr2=0x7f743003abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/889516631 2023-07-21T18:33:42.329 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:36.534+0000 7f74499e1700 1 --2- 172.21.15.39:0/4035116593 >> [v2:172.21.15.39:6800/889516631,v1:172.21.15.39:6801/889516631] conn(0x7f7430038710 0x7f743003abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-07-21T18:33:42.329 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:38.135+0000 7f74499e1700 1 -- 172.21.15.39:0/4035116593 >> [v2:172.21.15.39:6800/889516631,v1:172.21.15.39:6801/889516631] conn(0x7f7430038710 msgr2=0x7f743003abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/889516631 2023-07-21T18:33:42.329 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:38.135+0000 7f74499e1700 1 --2- 172.21.15.39:0/4035116593 >> [v2:172.21.15.39:6800/889516631,v1:172.21.15.39:6801/889516631] conn(0x7f7430038710 0x7f743003abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-07-21T18:33:42.329 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:40.856+0000 7f743b7fe700 1 -- 172.21.15.39:0/4035116593 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mgrmap(e 6) v1 ==== 45250+0+0 (secure 0 0 0) 0x7f744002d4e0 con 0x7f74441059b0 2023-07-21T18:33:42.330 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:40.856+0000 7f743b7fe700 1 -- 172.21.15.39:0/4035116593 >> [v2:172.21.15.39:6800/889516631,v1:172.21.15.39:6801/889516631] conn(0x7f7430038710 msgr2=0x7f743003abc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-21T18:33:42.330 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:40.856+0000 7f743b7fe700 1 --2- 172.21.15.39:0/4035116593 >> [v2:172.21.15.39:6800/889516631,v1:172.21.15.39:6801/889516631] conn(0x7f7430038710 0x7f743003abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:33:42.330 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:41.856+0000 7f743b7fe700 1 -- 172.21.15.39:0/4035116593 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mgrmap(e 7) v1 ==== 45377+0+0 (secure 0 0 0) 0x7f744004e0f0 con 0x7f74441059b0 2023-07-21T18:33:42.330 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:41.856+0000 7f743b7fe700 1 --2- 172.21.15.39:0/4035116593 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7f743003bfe0 0x7f743003e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:33:42.330 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:41.856+0000 7f743b7fe700 1 -- 172.21.15.39:0/4035116593 --> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f743003b2d0 con 0x7f743003bfe0 2023-07-21T18:33:42.331 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:41.856+0000 7f74499e1700 1 --2- 172.21.15.39:0/4035116593 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7f743003bfe0 0x7f743003e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:33:42.331 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:41.857+0000 7f74499e1700 1 --2- 172.21.15.39:0/4035116593 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7f743003bfe0 0x7f743003e3c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7434003a10 tx=0x7f7434009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:33:42.331 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:41.858+0000 7f743b7fe700 1 -- 172.21.15.39:0/4035116593 <== mgr.14120 v2:172.21.15.39:6800/2893866425 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f743003b2d0 con 0x7f743003bfe0 2023-07-21T18:33:42.331 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:41.868+0000 7f744c446700 1 -- 172.21.15.39:0/4035116593 --> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f7444105590 con 0x7f743003bfe0 2023-07-21T18:33:42.331 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:41.869+0000 7f743b7fe700 1 -- 172.21.15.39:0/4035116593 <== mgr.14120 v2:172.21.15.39:6800/2893866425 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f7444105590 con 0x7f743003bfe0 2023-07-21T18:33:42.332 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:41.869+0000 7f744c446700 1 -- 172.21.15.39:0/4035116593 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7f743003bfe0 msgr2=0x7f743003e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:33:42.332 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:41.869+0000 7f744c446700 1 --2- 172.21.15.39:0/4035116593 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7f743003bfe0 0x7f743003e3c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7434003a10 tx=0x7f7434009290 comp rx=0 tx=0).stop 2023-07-21T18:33:42.332 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:41.869+0000 7f744c446700 1 -- 172.21.15.39:0/4035116593 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74441059b0 msgr2=0x7f7444104040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:33:42.332 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:41.869+0000 7f744c446700 1 --2- 172.21.15.39:0/4035116593 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74441059b0 0x7f7444104040 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7440009130 tx=0x7f7440011ad0 comp rx=0 tx=0).stop 2023-07-21T18:33:42.332 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:41.869+0000 7f744c446700 1 -- 172.21.15.39:0/4035116593 shutdown_connections 2023-07-21T18:33:42.333 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:41.869+0000 7f744c446700 1 --2- 172.21.15.39:0/4035116593 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7f743003bfe0 0x7f743003e3c0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:33:42.333 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:41.869+0000 7f744c446700 1 --2- 172.21.15.39:0/4035116593 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74441059b0 0x7f7444104040 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:33:42.333 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:41.869+0000 7f744c446700 1 --2- 172.21.15.39:0/4035116593 >> [v2:172.21.15.39:6800/889516631,v1:172.21.15.39:6801/889516631] conn(0x7f7430038710 0x7f743003abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:33:42.333 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:41.869+0000 7f744c446700 1 -- 172.21.15.39:0/4035116593 >> 172.21.15.39:0/4035116593 conn(0x7f74440ffa50 msgr2=0x7f7444101120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:33:42.333 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:41.869+0000 7f744c446700 1 -- 172.21.15.39:0/4035116593 shutdown_connections 2023-07-21T18:33:42.333 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:41.869+0000 7f744c446700 1 -- 172.21.15.39:0/4035116593 wait complete. 2023-07-21T18:33:42.334 INFO:teuthology.orchestra.run.smithi039.stdout:mgr epoch 5 is available 2023-07-21T18:33:42.334 INFO:teuthology.orchestra.run.smithi039.stdout:Setting orchestrator backend to cephadm... 2023-07-21T18:33:43.192 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:42 smithi039 ceph-mon[96489]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-07-21T18:33:43.193 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:42 smithi039 ceph-mon[96489]: mgrmap e7: x(active, since 1.00729s) 2023-07-21T18:33:43.193 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:42 smithi039 ceph-mon[96489]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-07-21T18:33:43.193 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:42 smithi039 ceph-mon[96489]: from='mgr.14120 172.21.15.39:0/2168989012' entity='mgr.x' 2023-07-21T18:33:43.193 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:42 smithi039 ceph-mon[96489]: from='mgr.14120 172.21.15.39:0/2168989012' entity='mgr.x' 2023-07-21T18:33:43.193 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:42 smithi039 ceph-mon[96489]: from='mgr.14120 172.21.15.39:0/2168989012' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:33:43.563 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:42.722+0000 7f06d4746700 1 Processor -- start 2023-07-21T18:33:43.569 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:42.722+0000 7f06d4746700 1 -- start start 2023-07-21T18:33:43.570 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:42.723+0000 7f06d4746700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06cc073ba0 0x7f06cc1080a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:33:43.570 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:42.723+0000 7f06d4746700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06cc1085e0 con 0x7f06cc073ba0 2023-07-21T18:33:43.570 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:42.723+0000 7f06d24e2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06cc073ba0 0x7f06cc1080a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:33:43.571 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:42.723+0000 7f06d24e2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06cc073ba0 0x7f06cc1080a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53824/0 (socket says 172.21.15.39:53824) 2023-07-21T18:33:43.571 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:42.723+0000 7f06d24e2700 1 -- 172.21.15.39:0/3793940783 learned_addr learned my addr 172.21.15.39:0/3793940783 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:33:43.571 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:42.723+0000 7f06d24e2700 1 -- 172.21.15.39:0/3793940783 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06cc108720 con 0x7f06cc073ba0 2023-07-21T18:33:43.572 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:42.723+0000 7f06d24e2700 1 --2- 172.21.15.39:0/3793940783 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06cc073ba0 0x7f06cc1080a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f06c80098a0 tx=0x7f06c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a470c43d3d168cc server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:33:43.572 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:42.724+0000 7f06d14e0700 1 -- 172.21.15.39:0/3793940783 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f06c8004160 con 0x7f06cc073ba0 2023-07-21T18:33:43.572 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:42.724+0000 7f06d14e0700 1 -- 172.21.15.39:0/3793940783 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f06c80042c0 con 0x7f06cc073ba0 2023-07-21T18:33:43.572 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:42.724+0000 7f06d14e0700 1 -- 172.21.15.39:0/3793940783 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f06c8011a00 con 0x7f06cc073ba0 2023-07-21T18:33:43.573 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:42.724+0000 7f06d4746700 1 -- 172.21.15.39:0/3793940783 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06cc073ba0 msgr2=0x7f06cc1080a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:33:43.573 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:42.724+0000 7f06d4746700 1 --2- 172.21.15.39:0/3793940783 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06cc073ba0 0x7f06cc1080a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f06c80098a0 tx=0x7f06c8009bb0 comp rx=0 tx=0).stop 2023-07-21T18:33:43.573 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:42.724+0000 7f06d4746700 1 -- 172.21.15.39:0/3793940783 shutdown_connections 2023-07-21T18:33:43.573 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:42.724+0000 7f06d4746700 1 --2- 172.21.15.39:0/3793940783 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06cc073ba0 0x7f06cc1080a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:33:43.573 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:42.724+0000 7f06d4746700 1 -- 172.21.15.39:0/3793940783 >> 172.21.15.39:0/3793940783 conn(0x7f06cc0ffa50 msgr2=0x7f06cc101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:33:43.574 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:42.724+0000 7f06d4746700 1 -- 172.21.15.39:0/3793940783 shutdown_connections 2023-07-21T18:33:43.574 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:42.724+0000 7f06d4746700 1 -- 172.21.15.39:0/3793940783 wait complete. 2023-07-21T18:33:43.574 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:42.725+0000 7f06d4746700 1 Processor -- start 2023-07-21T18:33:43.575 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:42.725+0000 7f06d4746700 1 -- start start 2023-07-21T18:33:43.575 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:42.725+0000 7f06d4746700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06cc199280 0x7f06cc199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:33:43.575 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:42.725+0000 7f06d4746700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06cc199bd0 con 0x7f06cc199280 2023-07-21T18:33:43.575 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:42.725+0000 7f06d24e2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06cc199280 0x7f06cc199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:33:43.575 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:42.725+0000 7f06d24e2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06cc199280 0x7f06cc199690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53826/0 (socket says 172.21.15.39:53826) 2023-07-21T18:33:43.576 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:42.725+0000 7f06d24e2700 1 -- 172.21.15.39:0/1908132693 learned_addr learned my addr 172.21.15.39:0/1908132693 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:33:43.576 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:42.726+0000 7f06d24e2700 1 -- 172.21.15.39:0/1908132693 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06c8009550 con 0x7f06cc199280 2023-07-21T18:33:43.576 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:42.726+0000 7f06d24e2700 1 --2- 172.21.15.39:0/1908132693 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06cc199280 0x7f06cc199690 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f06c8009130 tx=0x7f06c8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:33:43.576 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:42.726+0000 7f06c37fe700 1 -- 172.21.15.39:0/1908132693 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f06c8003ab0 con 0x7f06cc199280 2023-07-21T18:33:43.576 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:42.726+0000 7f06c37fe700 1 -- 172.21.15.39:0/1908132693 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f06c8003c10 con 0x7f06cc199280 2023-07-21T18:33:43.576 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:42.726+0000 7f06c37fe700 1 -- 172.21.15.39:0/1908132693 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f06c80247d0 con 0x7f06cc199280 2023-07-21T18:33:43.576 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:42.726+0000 7f06d4746700 1 -- 172.21.15.39:0/1908132693 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06cc199dd0 con 0x7f06cc199280 2023-07-21T18:33:43.577 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:42.726+0000 7f06d4746700 1 -- 172.21.15.39:0/1908132693 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06cc19ca30 con 0x7f06cc199280 2023-07-21T18:33:43.577 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:42.727+0000 7f06d4746700 1 -- 172.21.15.39:0/1908132693 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06cc073ba0 con 0x7f06cc199280 2023-07-21T18:33:43.577 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:42.728+0000 7f06c37fe700 1 -- 172.21.15.39:0/1908132693 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 7) v1 ==== 45377+0+0 (secure 0 0 0) 0x7f06c801abb0 con 0x7f06cc199280 2023-07-21T18:33:43.577 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:42.728+0000 7f06c37fe700 1 --2- 172.21.15.39:0/1908132693 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7f06b8038630 0x7f06b803aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:33:43.577 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:42.728+0000 7f06c37fe700 1 -- 172.21.15.39:0/1908132693 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f06c804c8e0 con 0x7f06cc199280 2023-07-21T18:33:43.577 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:42.728+0000 7f06d1ce1700 1 --2- 172.21.15.39:0/1908132693 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7f06b8038630 0x7f06b803aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:33:43.577 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:42.729+0000 7f06d1ce1700 1 --2- 172.21.15.39:0/1908132693 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7f06b8038630 0x7f06b803aae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f06bc006fd0 tx=0x7f06bc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:33:43.578 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:42.732+0000 7f06c37fe700 1 -- 172.21.15.39:0/1908132693 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f06c8018760 con 0x7f06cc199280 2023-07-21T18:33:43.578 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:43.119+0000 7f06d4746700 1 -- 172.21.15.39:0/1908132693 --> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f06cc101d80 con 0x7f06b8038630 2023-07-21T18:33:43.578 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:43.127+0000 7f06c37fe700 1 -- 172.21.15.39:0/1908132693 <== mgr.14120 v2:172.21.15.39:6800/2893866425 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f06cc101d80 con 0x7f06b8038630 2023-07-21T18:33:43.578 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:43.129+0000 7f06d4746700 1 -- 172.21.15.39:0/1908132693 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7f06b8038630 msgr2=0x7f06b803aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:33:43.578 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:43.129+0000 7f06d4746700 1 --2- 172.21.15.39:0/1908132693 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7f06b8038630 0x7f06b803aae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f06bc006fd0 tx=0x7f06bc006e40 comp rx=0 tx=0).stop 2023-07-21T18:33:43.578 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:43.129+0000 7f06d4746700 1 -- 172.21.15.39:0/1908132693 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06cc199280 msgr2=0x7f06cc199690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:33:43.578 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:43.129+0000 7f06d4746700 1 --2- 172.21.15.39:0/1908132693 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06cc199280 0x7f06cc199690 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f06c8009130 tx=0x7f06c8011ad0 comp rx=0 tx=0).stop 2023-07-21T18:33:43.578 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:43.130+0000 7f06d4746700 1 -- 172.21.15.39:0/1908132693 shutdown_connections 2023-07-21T18:33:43.579 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:43.130+0000 7f06d4746700 1 --2- 172.21.15.39:0/1908132693 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7f06b8038630 0x7f06b803aae0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:33:43.579 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:43.130+0000 7f06d4746700 1 --2- 172.21.15.39:0/1908132693 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06cc199280 0x7f06cc199690 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:33:43.579 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:43.130+0000 7f06d4746700 1 -- 172.21.15.39:0/1908132693 >> 172.21.15.39:0/1908132693 conn(0x7f06cc0ffa50 msgr2=0x7f06cc101120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:33:43.579 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:43.130+0000 7f06d4746700 1 -- 172.21.15.39:0/1908132693 shutdown_connections 2023-07-21T18:33:43.579 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:43.130+0000 7f06d4746700 1 -- 172.21.15.39:0/1908132693 wait complete. 2023-07-21T18:33:44.199 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:43 smithi039 ceph-mon[96489]: [21/Jul/2023:18:33:42] ENGINE Bus STARTING 2023-07-21T18:33:44.200 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:43 smithi039 ceph-mon[96489]: [21/Jul/2023:18:33:42] ENGINE Serving on https://172.21.15.39:7150 2023-07-21T18:33:44.200 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:43 smithi039 ceph-mon[96489]: [21/Jul/2023:18:33:42] ENGINE Serving on http://172.21.15.39:8765 2023-07-21T18:33:44.200 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:43 smithi039 ceph-mon[96489]: [21/Jul/2023:18:33:42] ENGINE Bus STARTED 2023-07-21T18:33:44.200 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:43 smithi039 ceph-mon[96489]: from='mgr.14120 172.21.15.39:0/2168989012' entity='mgr.x' 2023-07-21T18:33:44.200 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:43 smithi039 ceph-mon[96489]: from='mgr.14120 172.21.15.39:0/2168989012' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:33:44.813 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout value unchanged 2023-07-21T18:33:44.814 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:43.991+0000 7f0ca078f700 1 Processor -- start 2023-07-21T18:33:44.814 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:43.991+0000 7f0ca078f700 1 -- start start 2023-07-21T18:33:44.814 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:43.992+0000 7f0ca078f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c98104400 0x7f0c98104810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:33:44.814 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:43.992+0000 7f0ca078f700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c98104d50 con 0x7f0c98104400 2023-07-21T18:33:44.814 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:43.992+0000 7f0c9e52b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c98104400 0x7f0c98104810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:33:44.815 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:43.992+0000 7f0c9e52b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c98104400 0x7f0c98104810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53830/0 (socket says 172.21.15.39:53830) 2023-07-21T18:33:44.815 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:43.992+0000 7f0c9e52b700 1 -- 172.21.15.39:0/199776817 learned_addr learned my addr 172.21.15.39:0/199776817 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:33:44.815 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:43.992+0000 7f0c9e52b700 1 -- 172.21.15.39:0/199776817 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c98104e90 con 0x7f0c98104400 2023-07-21T18:33:44.815 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:43.992+0000 7f0c9e52b700 1 --2- 172.21.15.39:0/199776817 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c98104400 0x7f0c98104810 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0c880098a0 tx=0x7f0c88009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1acb62df5263503e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:33:44.815 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:43.992+0000 7f0c9d529700 1 -- 172.21.15.39:0/199776817 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0c88004160 con 0x7f0c98104400 2023-07-21T18:33:44.815 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:43.992+0000 7f0c9d529700 1 -- 172.21.15.39:0/199776817 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0c880042c0 con 0x7f0c98104400 2023-07-21T18:33:44.815 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:43.992+0000 7f0c9d529700 1 -- 172.21.15.39:0/199776817 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0c88011a00 con 0x7f0c98104400 2023-07-21T18:33:44.816 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:43.993+0000 7f0ca078f700 1 -- 172.21.15.39:0/199776817 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c98104400 msgr2=0x7f0c98104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:33:44.816 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:43.993+0000 7f0ca078f700 1 --2- 172.21.15.39:0/199776817 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c98104400 0x7f0c98104810 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0c880098a0 tx=0x7f0c88009bb0 comp rx=0 tx=0).stop 2023-07-21T18:33:44.816 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:43.993+0000 7f0ca078f700 1 -- 172.21.15.39:0/199776817 shutdown_connections 2023-07-21T18:33:44.816 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:43.993+0000 7f0ca078f700 1 --2- 172.21.15.39:0/199776817 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c98104400 0x7f0c98104810 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:33:44.816 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:43.993+0000 7f0ca078f700 1 -- 172.21.15.39:0/199776817 >> 172.21.15.39:0/199776817 conn(0x7f0c980ffa50 msgr2=0x7f0c98101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:33:44.816 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:43.993+0000 7f0ca078f700 1 -- 172.21.15.39:0/199776817 shutdown_connections 2023-07-21T18:33:44.816 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:43.993+0000 7f0ca078f700 1 -- 172.21.15.39:0/199776817 wait complete. 2023-07-21T18:33:44.817 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:43.994+0000 7f0ca078f700 1 Processor -- start 2023-07-21T18:33:44.817 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:43.994+0000 7f0ca078f700 1 -- start start 2023-07-21T18:33:44.817 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:43.994+0000 7f0ca078f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c98104400 0x7f0c98198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:33:44.817 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:43.994+0000 7f0ca078f700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c981986d0 con 0x7f0c98104400 2023-07-21T18:33:44.817 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:43.994+0000 7f0c9e52b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c98104400 0x7f0c98198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:33:44.817 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:43.994+0000 7f0c9e52b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c98104400 0x7f0c98198190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53832/0 (socket says 172.21.15.39:53832) 2023-07-21T18:33:44.817 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:43.994+0000 7f0c9e52b700 1 -- 172.21.15.39:0/1212070082 learned_addr learned my addr 172.21.15.39:0/1212070082 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:33:44.818 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:43.994+0000 7f0c9e52b700 1 -- 172.21.15.39:0/1212070082 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c88009550 con 0x7f0c98104400 2023-07-21T18:33:44.818 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:43.995+0000 7f0c9e52b700 1 --2- 172.21.15.39:0/1212070082 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c98104400 0x7f0c98198190 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0c88009130 tx=0x7f0c88011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:33:44.818 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:43.995+0000 7f0c8f7fe700 1 -- 172.21.15.39:0/1212070082 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0c88003b70 con 0x7f0c98104400 2023-07-21T18:33:44.818 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:43.995+0000 7f0c8f7fe700 1 -- 172.21.15.39:0/1212070082 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0c88003cd0 con 0x7f0c98104400 2023-07-21T18:33:44.818 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:43.995+0000 7f0ca078f700 1 -- 172.21.15.39:0/1212070082 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c981988d0 con 0x7f0c98104400 2023-07-21T18:33:44.818 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:43.995+0000 7f0c8f7fe700 1 -- 172.21.15.39:0/1212070082 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0c880197d0 con 0x7f0c98104400 2023-07-21T18:33:44.818 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:43.995+0000 7f0ca078f700 1 -- 172.21.15.39:0/1212070082 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c98198cf0 con 0x7f0c98104400 2023-07-21T18:33:44.818 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:43.996+0000 7f0c8f7fe700 1 -- 172.21.15.39:0/1212070082 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f0c8801abb0 con 0x7f0c98104400 2023-07-21T18:33:44.819 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:43.996+0000 7f0ca078f700 1 -- 172.21.15.39:0/1212070082 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c98061b40 con 0x7f0c98104400 2023-07-21T18:33:44.819 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:43.996+0000 7f0c8f7fe700 1 --2- 172.21.15.39:0/1212070082 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7f0c84038760 0x7f0c8403ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:33:44.819 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:43.996+0000 7f0c8f7fe700 1 -- 172.21.15.39:0/1212070082 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f0c8804c8f0 con 0x7f0c98104400 2023-07-21T18:33:44.819 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:43.996+0000 7f0c9dd2a700 1 --2- 172.21.15.39:0/1212070082 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7f0c84038760 0x7f0c8403ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:33:44.819 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:43.997+0000 7f0c9dd2a700 1 --2- 172.21.15.39:0/1212070082 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7f0c84038760 0x7f0c8403ac10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0c94006fd0 tx=0x7f0c94006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:33:44.819 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:44.001+0000 7f0c8f7fe700 1 -- 172.21.15.39:0/1212070082 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f0c8801a430 con 0x7f0c98104400 2023-07-21T18:33:44.819 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:44.362+0000 7f0ca078f700 1 -- 172.21.15.39:0/1212070082 --> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f0c98101d60 con 0x7f0c84038760 2023-07-21T18:33:44.819 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:44.363+0000 7f0c8f7fe700 1 -- 172.21.15.39:0/1212070082 <== mgr.14120 v2:172.21.15.39:6800/2893866425 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f0c98101d60 con 0x7f0c84038760 2023-07-21T18:33:44.820 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:44.365+0000 7f0ca078f700 1 -- 172.21.15.39:0/1212070082 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7f0c84038760 msgr2=0x7f0c8403ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:33:44.820 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:44.365+0000 7f0ca078f700 1 --2- 172.21.15.39:0/1212070082 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7f0c84038760 0x7f0c8403ac10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0c94006fd0 tx=0x7f0c94006e40 comp rx=0 tx=0).stop 2023-07-21T18:33:44.820 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:44.365+0000 7f0ca078f700 1 -- 172.21.15.39:0/1212070082 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c98104400 msgr2=0x7f0c98198190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:33:44.820 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:44.365+0000 7f0ca078f700 1 --2- 172.21.15.39:0/1212070082 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c98104400 0x7f0c98198190 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0c88009130 tx=0x7f0c88011ad0 comp rx=0 tx=0).stop 2023-07-21T18:33:44.820 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:44.365+0000 7f0ca078f700 1 -- 172.21.15.39:0/1212070082 shutdown_connections 2023-07-21T18:33:44.820 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:44.365+0000 7f0ca078f700 1 --2- 172.21.15.39:0/1212070082 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7f0c84038760 0x7f0c8403ac10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:33:44.821 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:44.365+0000 7f0ca078f700 1 --2- 172.21.15.39:0/1212070082 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c98104400 0x7f0c98198190 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:33:44.821 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:44.365+0000 7f0ca078f700 1 -- 172.21.15.39:0/1212070082 >> 172.21.15.39:0/1212070082 conn(0x7f0c980ffa50 msgr2=0x7f0c98101100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:33:44.821 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:44.365+0000 7f0ca078f700 1 -- 172.21.15.39:0/1212070082 shutdown_connections 2023-07-21T18:33:44.821 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:44.366+0000 7f0ca078f700 1 -- 172.21.15.39:0/1212070082 wait complete. 2023-07-21T18:33:44.821 INFO:teuthology.orchestra.run.smithi039.stdout:Generating ssh key... 2023-07-21T18:33:45.199 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:44 smithi039 ceph-mon[96489]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:33:45.200 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:44 smithi039 ceph-mon[96489]: mgrmap e8: x(active, since 3s) 2023-07-21T18:33:46.199 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:45 smithi039 ceph-mon[96489]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:33:46.601 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:46 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: Generating public/private rsa key pair. 2023-07-21T18:33:46.601 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:46 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: Your identification has been saved in /tmp/tmp83oyf82_/key. 2023-07-21T18:33:46.601 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:46 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: Your public key has been saved in /tmp/tmp83oyf82_/key.pub. 2023-07-21T18:33:46.601 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:46 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: The key fingerprint is: 2023-07-21T18:33:46.602 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:46 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: SHA256:qqqOXqJ+7ncwmlg99BNsxWi2lGl+B7EOr8cegmiriBU ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c 2023-07-21T18:33:46.602 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:46 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: The key's randomart image is: 2023-07-21T18:33:46.602 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:46 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: +---[RSA 3072]----+ 2023-07-21T18:33:46.602 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:46 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: | =.. | 2023-07-21T18:33:46.602 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:46 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: | O +. | 2023-07-21T18:33:46.602 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:46 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: | B.o.. | 2023-07-21T18:33:46.602 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:46 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: | . *+. . | 2023-07-21T18:33:46.602 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:46 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: | E o o So. | 2023-07-21T18:33:46.603 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:46 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: | o.=.+o | 2023-07-21T18:33:46.603 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:46 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: | .++o.=o.+ | 2023-07-21T18:33:46.603 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:46 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: |=o++.o .+ . | 2023-07-21T18:33:46.603 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:46 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: |X**=o . . | 2023-07-21T18:33:46.603 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:46 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: +----[SHA256]-----+ 2023-07-21T18:33:46.659 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:45.256+0000 7f2e3c398700 1 Processor -- start 2023-07-21T18:33:46.659 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:45.256+0000 7f2e3c398700 1 -- start start 2023-07-21T18:33:46.660 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:45.256+0000 7f2e3c398700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e341043f0 0x7f2e34104800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:33:46.660 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:45.256+0000 7f2e3c398700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e34104d40 con 0x7f2e341043f0 2023-07-21T18:33:46.660 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:45.256+0000 7f2e3a134700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e341043f0 0x7f2e34104800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:33:46.660 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:45.256+0000 7f2e3a134700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e341043f0 0x7f2e34104800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53836/0 (socket says 172.21.15.39:53836) 2023-07-21T18:33:46.660 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:45.256+0000 7f2e3a134700 1 -- 172.21.15.39:0/3201831510 learned_addr learned my addr 172.21.15.39:0/3201831510 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:33:46.660 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:45.257+0000 7f2e3a134700 1 -- 172.21.15.39:0/3201831510 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e34104e80 con 0x7f2e341043f0 2023-07-21T18:33:46.660 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:45.257+0000 7f2e3a134700 1 --2- 172.21.15.39:0/3201831510 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e341043f0 0x7f2e34104800 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2e3000bb40 tx=0x7f2e3000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=948b3b5aafa11d38 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:33:46.661 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:45.257+0000 7f2e39132700 1 -- 172.21.15.39:0/3201831510 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2e3000dbe0 con 0x7f2e341043f0 2023-07-21T18:33:46.661 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:45.257+0000 7f2e39132700 1 -- 172.21.15.39:0/3201831510 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2e300044d0 con 0x7f2e341043f0 2023-07-21T18:33:46.661 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:45.257+0000 7f2e39132700 1 -- 172.21.15.39:0/3201831510 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2e30010470 con 0x7f2e341043f0 2023-07-21T18:33:46.661 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:45.258+0000 7f2e3c398700 1 -- 172.21.15.39:0/3201831510 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e341043f0 msgr2=0x7f2e34104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:33:46.661 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:45.258+0000 7f2e3c398700 1 --2- 172.21.15.39:0/3201831510 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e341043f0 0x7f2e34104800 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2e3000bb40 tx=0x7f2e3000be50 comp rx=0 tx=0).stop 2023-07-21T18:33:46.661 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:45.258+0000 7f2e3c398700 1 -- 172.21.15.39:0/3201831510 shutdown_connections 2023-07-21T18:33:46.661 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:45.258+0000 7f2e3c398700 1 --2- 172.21.15.39:0/3201831510 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e341043f0 0x7f2e34104800 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:33:46.661 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:45.258+0000 7f2e3c398700 1 -- 172.21.15.39:0/3201831510 >> 172.21.15.39:0/3201831510 conn(0x7f2e340ffa40 msgr2=0x7f2e34101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:33:46.662 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:45.258+0000 7f2e3c398700 1 -- 172.21.15.39:0/3201831510 shutdown_connections 2023-07-21T18:33:46.662 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:45.258+0000 7f2e3c398700 1 -- 172.21.15.39:0/3201831510 wait complete. 2023-07-21T18:33:46.662 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:45.258+0000 7f2e3c398700 1 Processor -- start 2023-07-21T18:33:46.662 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:45.259+0000 7f2e3c398700 1 -- start start 2023-07-21T18:33:46.662 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:45.259+0000 7f2e3c398700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e341043f0 0x7f2e3419b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:33:46.662 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:45.259+0000 7f2e3c398700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3419b920 con 0x7f2e341043f0 2023-07-21T18:33:46.662 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:45.259+0000 7f2e3a134700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e341043f0 0x7f2e3419b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:33:46.662 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:45.259+0000 7f2e3a134700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e341043f0 0x7f2e3419b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53838/0 (socket says 172.21.15.39:53838) 2023-07-21T18:33:46.663 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:45.259+0000 7f2e3a134700 1 -- 172.21.15.39:0/941719902 learned_addr learned my addr 172.21.15.39:0/941719902 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:33:46.663 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:45.259+0000 7f2e3a134700 1 -- 172.21.15.39:0/941719902 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e3000b7f0 con 0x7f2e341043f0 2023-07-21T18:33:46.663 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:45.259+0000 7f2e3a134700 1 --2- 172.21.15.39:0/941719902 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e341043f0 0x7f2e3419b3e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2e3000b340 tx=0x7f2e30003df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:33:46.663 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:45.259+0000 7f2e2b7fe700 1 -- 172.21.15.39:0/941719902 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2e30004340 con 0x7f2e341043f0 2023-07-21T18:33:46.663 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:45.259+0000 7f2e2b7fe700 1 -- 172.21.15.39:0/941719902 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2e30010bd0 con 0x7f2e341043f0 2023-07-21T18:33:46.663 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:45.259+0000 7f2e2b7fe700 1 -- 172.21.15.39:0/941719902 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2e30021c60 con 0x7f2e341043f0 2023-07-21T18:33:46.663 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:45.259+0000 7f2e3c398700 1 -- 172.21.15.39:0/941719902 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e3419bb20 con 0x7f2e341043f0 2023-07-21T18:33:46.663 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:45.260+0000 7f2e3c398700 1 -- 172.21.15.39:0/941719902 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e3419bee0 con 0x7f2e341043f0 2023-07-21T18:33:46.664 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:45.261+0000 7f2e2b7fe700 1 -- 172.21.15.39:0/941719902 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f2e30010730 con 0x7f2e341043f0 2023-07-21T18:33:46.664 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:45.261+0000 7f2e3c398700 1 -- 172.21.15.39:0/941719902 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e34190d20 con 0x7f2e341043f0 2023-07-21T18:33:46.664 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:45.261+0000 7f2e2b7fe700 1 --2- 172.21.15.39:0/941719902 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7f2e20038710 0x7f2e2003abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:33:46.664 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:45.261+0000 7f2e2b7fe700 1 -- 172.21.15.39:0/941719902 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f2e3004cb20 con 0x7f2e341043f0 2023-07-21T18:33:46.664 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:45.261+0000 7f2e39933700 1 --2- 172.21.15.39:0/941719902 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7f2e20038710 0x7f2e2003abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:33:46.664 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:45.261+0000 7f2e39933700 1 --2- 172.21.15.39:0/941719902 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7f2e20038710 0x7f2e2003abc0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2e24006fd0 tx=0x7f2e24006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:33:46.664 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:45.266+0000 7f2e2b7fe700 1 -- 172.21.15.39:0/941719902 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f2e3000f5b0 con 0x7f2e341043f0 2023-07-21T18:33:46.665 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:45.624+0000 7f2e3c398700 1 -- 172.21.15.39:0/941719902 --> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f2e34060950 con 0x7f2e20038710 2023-07-21T18:33:46.665 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:46.244+0000 7f2e2b7fe700 1 -- 172.21.15.39:0/941719902 <== mgr.14120 v2:172.21.15.39:6800/2893866425 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f2e34060950 con 0x7f2e20038710 2023-07-21T18:33:46.665 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:46.246+0000 7f2e3c398700 1 -- 172.21.15.39:0/941719902 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7f2e20038710 msgr2=0x7f2e2003abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:33:46.665 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:46.246+0000 7f2e3c398700 1 --2- 172.21.15.39:0/941719902 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7f2e20038710 0x7f2e2003abc0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2e24006fd0 tx=0x7f2e24006e40 comp rx=0 tx=0).stop 2023-07-21T18:33:46.665 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:46.247+0000 7f2e3c398700 1 -- 172.21.15.39:0/941719902 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e341043f0 msgr2=0x7f2e3419b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:33:46.665 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:46.247+0000 7f2e3c398700 1 --2- 172.21.15.39:0/941719902 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e341043f0 0x7f2e3419b3e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2e3000b340 tx=0x7f2e30003df0 comp rx=0 tx=0).stop 2023-07-21T18:33:46.665 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:46.247+0000 7f2e3c398700 1 -- 172.21.15.39:0/941719902 shutdown_connections 2023-07-21T18:33:46.665 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:46.247+0000 7f2e3c398700 1 --2- 172.21.15.39:0/941719902 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7f2e20038710 0x7f2e2003abc0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:33:46.666 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:46.247+0000 7f2e3c398700 1 --2- 172.21.15.39:0/941719902 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e341043f0 0x7f2e3419b3e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:33:46.666 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:46.247+0000 7f2e3c398700 1 -- 172.21.15.39:0/941719902 >> 172.21.15.39:0/941719902 conn(0x7f2e340ffa40 msgr2=0x7f2e3418de60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:33:46.666 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:46.247+0000 7f2e3c398700 1 -- 172.21.15.39:0/941719902 shutdown_connections 2023-07-21T18:33:46.666 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:46.247+0000 7f2e3c398700 1 -- 172.21.15.39:0/941719902 wait complete. 2023-07-21T18:33:47.199 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:46 smithi039 ceph-mon[96489]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:33:47.200 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:46 smithi039 ceph-mon[96489]: Generating ssh key... 2023-07-21T18:33:47.200 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:46 smithi039 ceph-mon[96489]: from='mgr.14120 172.21.15.39:0/2168989012' entity='mgr.x' 2023-07-21T18:33:47.200 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:46 smithi039 ceph-mon[96489]: from='mgr.14120 172.21.15.39:0/2168989012' entity='mgr.x' 2023-07-21T18:33:47.804 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCebQXoWW5pGKf3B9BRIRTx6M1njcb9Jo/HOWcgbq3rW5l42Nngvd8fmR4NAIAB8AsjjyziK3NchU0DMbb9OKFNXOoPDqTZ1CUSw7uTedVTFqaco7MMOScBWxtSReh9AkRBD0K6hXFIrzVFfMjkCLP1/t356gXWFToEIfYCO0yBfTVuV08PzVp7Nxh7S+PzsAD/+CzXVLgrmOEPorNOcwT532jfCvepP7pm8wlefbfMgq2FJnj5dJmh7ldabie+FBvAXujxLbjzVr1BKrIyL9UYFRi0zP8tBRV7ar1jJ3ZISz5UKXKEKg9DFgGi381qrlj6oKL9UtsaO5SOByaM75pR2j+splllPFUabZUJC6oY4rpLRgSzQ4CoBax+vFjkD/d2eVQuemQ9SDLqViV1QLN2C/0NEpTO3R+vhUyE0cifUYWbWMfTXrg7N84sg0rNYO7sV0GWLgFOj9X9kizFbcC3a61A5axIQ1YzV2/83aynpJCw7mXgianKv9NBlueFyLk= ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c 2023-07-21T18:33:47.804 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:47.056+0000 7f8ca6b03700 1 Processor -- start 2023-07-21T18:33:47.804 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:47.056+0000 7f8ca6b03700 1 -- start start 2023-07-21T18:33:47.804 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:47.056+0000 7f8ca6b03700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ca01037e0 0x7f8ca0103bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:33:47.805 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:47.056+0000 7f8ca6b03700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca0104130 con 0x7f8ca01037e0 2023-07-21T18:33:47.805 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:47.056+0000 7f8ca489f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ca01037e0 0x7f8ca0103bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:33:47.805 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:47.056+0000 7f8ca489f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ca01037e0 0x7f8ca0103bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53842/0 (socket says 172.21.15.39:53842) 2023-07-21T18:33:47.805 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:47.056+0000 7f8ca489f700 1 -- 172.21.15.39:0/2194844329 learned_addr learned my addr 172.21.15.39:0/2194844329 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:33:47.806 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:47.057+0000 7f8ca489f700 1 -- 172.21.15.39:0/2194844329 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ca0104270 con 0x7f8ca01037e0 2023-07-21T18:33:47.806 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:47.057+0000 7f8ca489f700 1 --2- 172.21.15.39:0/2194844329 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ca01037e0 0x7f8ca0103bf0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8c940098a0 tx=0x7f8c94009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6609e6f9100fdfe3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:33:47.806 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:47.057+0000 7f8c9f7fe700 1 -- 172.21.15.39:0/2194844329 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8c94004160 con 0x7f8ca01037e0 2023-07-21T18:33:47.806 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:47.057+0000 7f8c9f7fe700 1 -- 172.21.15.39:0/2194844329 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8c940042c0 con 0x7f8ca01037e0 2023-07-21T18:33:47.806 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:47.057+0000 7f8c9f7fe700 1 -- 172.21.15.39:0/2194844329 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8c94011a00 con 0x7f8ca01037e0 2023-07-21T18:33:47.806 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:47.058+0000 7f8ca6b03700 1 -- 172.21.15.39:0/2194844329 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ca01037e0 msgr2=0x7f8ca0103bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:33:47.807 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:47.058+0000 7f8ca6b03700 1 --2- 172.21.15.39:0/2194844329 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ca01037e0 0x7f8ca0103bf0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8c940098a0 tx=0x7f8c94009bb0 comp rx=0 tx=0).stop 2023-07-21T18:33:47.807 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:47.058+0000 7f8ca6b03700 1 -- 172.21.15.39:0/2194844329 shutdown_connections 2023-07-21T18:33:47.807 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:47.058+0000 7f8ca6b03700 1 --2- 172.21.15.39:0/2194844329 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ca01037e0 0x7f8ca0103bf0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:33:47.807 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:47.058+0000 7f8ca6b03700 1 -- 172.21.15.39:0/2194844329 >> 172.21.15.39:0/2194844329 conn(0x7f8ca00ff1c0 msgr2=0x7f8ca01015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:33:47.808 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:47.058+0000 7f8ca6b03700 1 -- 172.21.15.39:0/2194844329 shutdown_connections 2023-07-21T18:33:47.808 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:47.058+0000 7f8ca6b03700 1 -- 172.21.15.39:0/2194844329 wait complete. 2023-07-21T18:33:47.808 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:47.058+0000 7f8ca6b03700 1 Processor -- start 2023-07-21T18:33:47.809 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:47.059+0000 7f8ca6b03700 1 -- start start 2023-07-21T18:33:47.809 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:47.059+0000 7f8ca6b03700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ca01037e0 0x7f8ca01992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:33:47.809 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:47.059+0000 7f8ca6b03700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca01997f0 con 0x7f8ca01037e0 2023-07-21T18:33:47.809 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:47.059+0000 7f8ca489f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ca01037e0 0x7f8ca01992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:33:47.809 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:47.059+0000 7f8ca489f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ca01037e0 0x7f8ca01992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53844/0 (socket says 172.21.15.39:53844) 2023-07-21T18:33:47.810 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:47.059+0000 7f8ca489f700 1 -- 172.21.15.39:0/2346661163 learned_addr learned my addr 172.21.15.39:0/2346661163 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:33:47.810 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:47.059+0000 7f8ca489f700 1 -- 172.21.15.39:0/2346661163 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c94009550 con 0x7f8ca01037e0 2023-07-21T18:33:47.810 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:47.059+0000 7f8ca489f700 1 --2- 172.21.15.39:0/2346661163 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ca01037e0 0x7f8ca01992b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8c94009fb0 tx=0x7f8c94011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:33:47.810 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:47.059+0000 7f8c9dffb700 1 -- 172.21.15.39:0/2346661163 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8c94003a40 con 0x7f8ca01037e0 2023-07-21T18:33:47.810 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:47.059+0000 7f8c9dffb700 1 -- 172.21.15.39:0/2346661163 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8c94003ba0 con 0x7f8ca01037e0 2023-07-21T18:33:47.811 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:47.059+0000 7f8c9dffb700 1 -- 172.21.15.39:0/2346661163 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8c9401a750 con 0x7f8ca01037e0 2023-07-21T18:33:47.811 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:47.059+0000 7f8ca6b03700 1 -- 172.21.15.39:0/2346661163 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ca01999f0 con 0x7f8ca01037e0 2023-07-21T18:33:47.811 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:47.060+0000 7f8ca6b03700 1 -- 172.21.15.39:0/2346661163 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ca0199e10 con 0x7f8ca01037e0 2023-07-21T18:33:47.811 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:47.060+0000 7f8c9dffb700 1 -- 172.21.15.39:0/2346661163 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f8c94024b60 con 0x7f8ca01037e0 2023-07-21T18:33:47.811 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:47.061+0000 7f8c9dffb700 1 --2- 172.21.15.39:0/2346661163 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7f8c90038710 0x7f8c9003abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:33:47.811 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:47.061+0000 7f8ca6b03700 1 -- 172.21.15.39:0/2346661163 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ca0061b40 con 0x7f8ca01037e0 2023-07-21T18:33:47.812 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:47.061+0000 7f8c9dffb700 1 -- 172.21.15.39:0/2346661163 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f8c9404c640 con 0x7f8ca01037e0 2023-07-21T18:33:47.812 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:47.061+0000 7f8c9ffff700 1 --2- 172.21.15.39:0/2346661163 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7f8c90038710 0x7f8c9003abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:33:47.812 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:47.061+0000 7f8c9ffff700 1 --2- 172.21.15.39:0/2346661163 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7f8c90038710 0x7f8c9003abc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c006fd0 tx=0x7f8c8c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:33:47.812 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:47.066+0000 7f8c9dffb700 1 -- 172.21.15.39:0/2346661163 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f8c94024430 con 0x7f8ca01037e0 2023-07-21T18:33:47.812 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:47.425+0000 7f8ca6b03700 1 -- 172.21.15.39:0/2346661163 --> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f8ca0101510 con 0x7f8c90038710 2023-07-21T18:33:47.812 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:47.426+0000 7f8c9dffb700 1 -- 172.21.15.39:0/2346661163 <== mgr.14120 v2:172.21.15.39:6800/2893866425 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f8ca0101510 con 0x7f8c90038710 2023-07-21T18:33:47.812 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:47.429+0000 7f8ca6b03700 1 -- 172.21.15.39:0/2346661163 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7f8c90038710 msgr2=0x7f8c9003abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:33:47.813 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:47.429+0000 7f8ca6b03700 1 --2- 172.21.15.39:0/2346661163 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7f8c90038710 0x7f8c9003abc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c006fd0 tx=0x7f8c8c006e40 comp rx=0 tx=0).stop 2023-07-21T18:33:47.813 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:47.429+0000 7f8ca6b03700 1 -- 172.21.15.39:0/2346661163 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ca01037e0 msgr2=0x7f8ca01992b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:33:47.813 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:47.429+0000 7f8ca6b03700 1 --2- 172.21.15.39:0/2346661163 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ca01037e0 0x7f8ca01992b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8c94009fb0 tx=0x7f8c94011ad0 comp rx=0 tx=0).stop 2023-07-21T18:33:47.813 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:47.429+0000 7f8ca6b03700 1 -- 172.21.15.39:0/2346661163 shutdown_connections 2023-07-21T18:33:47.813 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:47.429+0000 7f8ca6b03700 1 --2- 172.21.15.39:0/2346661163 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7f8c90038710 0x7f8c9003abc0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:33:47.813 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:47.429+0000 7f8ca6b03700 1 --2- 172.21.15.39:0/2346661163 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ca01037e0 0x7f8ca01992b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:33:47.813 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:47.429+0000 7f8ca6b03700 1 -- 172.21.15.39:0/2346661163 >> 172.21.15.39:0/2346661163 conn(0x7f8ca00ff1c0 msgr2=0x7f8ca01008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:33:47.814 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:47.429+0000 7f8ca6b03700 1 -- 172.21.15.39:0/2346661163 shutdown_connections 2023-07-21T18:33:47.814 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:47.429+0000 7f8ca6b03700 1 -- 172.21.15.39:0/2346661163 wait complete. 2023-07-21T18:33:47.815 INFO:teuthology.orchestra.run.smithi039.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2023-07-21T18:33:47.815 INFO:teuthology.orchestra.run.smithi039.stdout:Adding key to root@localhost authorized_keys... 2023-07-21T18:33:47.815 INFO:teuthology.orchestra.run.smithi039.stdout:Adding host smithi039... 2023-07-21T18:33:48.069 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:47 smithi039 ceph-mon[96489]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:33:49.231 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:48 smithi039 ceph-mon[96489]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi039", "addr": "172.21.15.39", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:33:50.449 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:49 smithi039 ceph-mon[96489]: Deploying cephadm binary to smithi039 2023-07-21T18:33:52.057 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout Added host 'smithi039' with addr '172.21.15.39' 2023-07-21T18:33:52.058 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:48.277+0000 7f135ed53700 1 Processor -- start 2023-07-21T18:33:52.058 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:48.277+0000 7f135ed53700 1 -- start start 2023-07-21T18:33:52.058 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:48.277+0000 7f135ed53700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1358073ba0 0x7f135810a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:33:52.058 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:48.277+0000 7f135ed53700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1358105e80 con 0x7f1358073ba0 2023-07-21T18:33:52.058 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:48.277+0000 7f135caef700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1358073ba0 0x7f135810a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:33:52.058 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:48.277+0000 7f135caef700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1358073ba0 0x7f135810a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53848/0 (socket says 172.21.15.39:53848) 2023-07-21T18:33:52.058 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:48.277+0000 7f135caef700 1 -- 172.21.15.39:0/1911600107 learned_addr learned my addr 172.21.15.39:0/1911600107 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:33:52.059 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:48.277+0000 7f135caef700 1 -- 172.21.15.39:0/1911600107 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f135810a9d0 con 0x7f1358073ba0 2023-07-21T18:33:52.060 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:48.278+0000 7f135caef700 1 --2- 172.21.15.39:0/1911600107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1358073ba0 0x7f135810a490 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f134c0098a0 tx=0x7f134c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e492a1861b0dff54 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:33:52.060 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:48.278+0000 7f13577fe700 1 -- 172.21.15.39:0/1911600107 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f134c004160 con 0x7f1358073ba0 2023-07-21T18:33:52.060 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:48.278+0000 7f13577fe700 1 -- 172.21.15.39:0/1911600107 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f134c0042c0 con 0x7f1358073ba0 2023-07-21T18:33:52.060 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:48.278+0000 7f13577fe700 1 -- 172.21.15.39:0/1911600107 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f134c011a00 con 0x7f1358073ba0 2023-07-21T18:33:52.061 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:48.278+0000 7f135ed53700 1 -- 172.21.15.39:0/1911600107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1358073ba0 msgr2=0x7f135810a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:33:52.061 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:48.278+0000 7f135ed53700 1 --2- 172.21.15.39:0/1911600107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1358073ba0 0x7f135810a490 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f134c0098a0 tx=0x7f134c009bb0 comp rx=0 tx=0).stop 2023-07-21T18:33:52.061 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:48.279+0000 7f135ed53700 1 -- 172.21.15.39:0/1911600107 shutdown_connections 2023-07-21T18:33:52.061 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:48.279+0000 7f135ed53700 1 --2- 172.21.15.39:0/1911600107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1358073ba0 0x7f135810a490 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:33:52.061 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:48.279+0000 7f135ed53700 1 -- 172.21.15.39:0/1911600107 >> 172.21.15.39:0/1911600107 conn(0x7f13580ffa40 msgr2=0x7f1358101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:33:52.061 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:48.279+0000 7f135ed53700 1 -- 172.21.15.39:0/1911600107 shutdown_connections 2023-07-21T18:33:52.061 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:48.279+0000 7f135ed53700 1 -- 172.21.15.39:0/1911600107 wait complete. 2023-07-21T18:33:52.061 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:48.279+0000 7f135ed53700 1 Processor -- start 2023-07-21T18:33:52.062 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:48.279+0000 7f135ed53700 1 -- start start 2023-07-21T18:33:52.062 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:48.279+0000 7f135ed53700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f135819b4a0 0x7f135819b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:33:52.062 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:48.279+0000 7f135ed53700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f135819bdf0 con 0x7f135819b4a0 2023-07-21T18:33:52.062 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:48.280+0000 7f135caef700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f135819b4a0 0x7f135819b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:33:52.062 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:48.280+0000 7f135caef700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f135819b4a0 0x7f135819b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53850/0 (socket says 172.21.15.39:53850) 2023-07-21T18:33:52.062 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:48.280+0000 7f135caef700 1 -- 172.21.15.39:0/3402592851 learned_addr learned my addr 172.21.15.39:0/3402592851 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:33:52.062 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:48.280+0000 7f135caef700 1 -- 172.21.15.39:0/3402592851 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f134c009550 con 0x7f135819b4a0 2023-07-21T18:33:52.063 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:48.280+0000 7f135caef700 1 --2- 172.21.15.39:0/3402592851 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f135819b4a0 0x7f135819b8b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f134c009130 tx=0x7f134c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:33:52.063 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:48.280+0000 7f1355ffb700 1 -- 172.21.15.39:0/3402592851 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f134c003ab0 con 0x7f135819b4a0 2023-07-21T18:33:52.063 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:48.280+0000 7f1355ffb700 1 -- 172.21.15.39:0/3402592851 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f134c003c10 con 0x7f135819b4a0 2023-07-21T18:33:52.063 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:48.280+0000 7f135ed53700 1 -- 172.21.15.39:0/3402592851 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f135819bff0 con 0x7f135819b4a0 2023-07-21T18:33:52.063 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:48.280+0000 7f1355ffb700 1 -- 172.21.15.39:0/3402592851 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f134c0247d0 con 0x7f135819b4a0 2023-07-21T18:33:52.063 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:48.280+0000 7f135ed53700 1 -- 172.21.15.39:0/3402592851 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f135819ec50 con 0x7f135819b4a0 2023-07-21T18:33:52.063 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:48.281+0000 7f1355ffb700 1 -- 172.21.15.39:0/3402592851 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f134c02d3f0 con 0x7f135819b4a0 2023-07-21T18:33:52.063 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:48.281+0000 7f1355ffb700 1 --2- 172.21.15.39:0/3402592851 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7f1348038710 0x7f134803abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:33:52.064 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:48.281+0000 7f1355ffb700 1 -- 172.21.15.39:0/3402592851 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f134c04c9d0 con 0x7f135819b4a0 2023-07-21T18:33:52.064 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:48.281+0000 7f135ed53700 1 -- 172.21.15.39:0/3402592851 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f135819c180 con 0x7f135819b4a0 2023-07-21T18:33:52.064 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:48.282+0000 7f1357fff700 1 --2- 172.21.15.39:0/3402592851 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7f1348038710 0x7f134803abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:33:52.064 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:48.282+0000 7f1357fff700 1 --2- 172.21.15.39:0/3402592851 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7f1348038710 0x7f134803abc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1344006fd0 tx=0x7f1344006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:33:52.064 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:48.286+0000 7f1355ffb700 1 -- 172.21.15.39:0/3402592851 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f134c010560 con 0x7f135819b4a0 2023-07-21T18:33:52.064 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:48.650+0000 7f135ed53700 1 -- 172.21.15.39:0/3402592851 --> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi039", "addr": "172.21.15.39", "target": ["mon-mgr", ""]}) v1 -- 0x7f1358101da0 con 0x7f1348038710 2023-07-21T18:33:52.065 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:51.400+0000 7f1355ffb700 1 -- 172.21.15.39:0/3402592851 <== mgr.14120 v2:172.21.15.39:6800/2893866425 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f1358101da0 con 0x7f1348038710 2023-07-21T18:33:52.065 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:51.403+0000 7f135ed53700 1 -- 172.21.15.39:0/3402592851 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7f1348038710 msgr2=0x7f134803abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:33:52.065 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:51.403+0000 7f135ed53700 1 --2- 172.21.15.39:0/3402592851 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7f1348038710 0x7f134803abc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1344006fd0 tx=0x7f1344006e40 comp rx=0 tx=0).stop 2023-07-21T18:33:52.065 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:51.403+0000 7f135ed53700 1 -- 172.21.15.39:0/3402592851 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f135819b4a0 msgr2=0x7f135819b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:33:52.065 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:51.403+0000 7f135ed53700 1 --2- 172.21.15.39:0/3402592851 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f135819b4a0 0x7f135819b8b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f134c009130 tx=0x7f134c011ad0 comp rx=0 tx=0).stop 2023-07-21T18:33:52.065 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:51.403+0000 7f135ed53700 1 -- 172.21.15.39:0/3402592851 shutdown_connections 2023-07-21T18:33:52.065 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:51.403+0000 7f135ed53700 1 --2- 172.21.15.39:0/3402592851 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7f1348038710 0x7f134803abc0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:33:52.065 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:51.403+0000 7f135ed53700 1 --2- 172.21.15.39:0/3402592851 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f135819b4a0 0x7f135819b8b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:33:52.066 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:51.403+0000 7f135ed53700 1 -- 172.21.15.39:0/3402592851 >> 172.21.15.39:0/3402592851 conn(0x7f13580ffa40 msgr2=0x7f1358101140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:33:52.066 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:51.403+0000 7f135ed53700 1 -- 172.21.15.39:0/3402592851 shutdown_connections 2023-07-21T18:33:52.067 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:51.403+0000 7f135ed53700 1 -- 172.21.15.39:0/3402592851 wait complete. 2023-07-21T18:33:52.067 INFO:teuthology.orchestra.run.smithi039.stdout:Deploying unmanaged mon service... 2023-07-21T18:33:52.700 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:52 smithi039 ceph-mon[96489]: from='mgr.14120 172.21.15.39:0/2168989012' entity='mgr.x' 2023-07-21T18:33:52.701 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:52 smithi039 ceph-mon[96489]: Added host smithi039 2023-07-21T18:33:52.701 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:52 smithi039 ceph-mon[96489]: from='mgr.14120 172.21.15.39:0/2168989012' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:33:53.402 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2023-07-21T18:33:53.407 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:52.570+0000 7fe98fb15700 1 Processor -- start 2023-07-21T18:33:53.407 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:52.570+0000 7fe98fb15700 1 -- start start 2023-07-21T18:33:53.407 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:52.570+0000 7fe98fb15700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe988073ba0 0x7fe98810a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:33:53.407 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:52.570+0000 7fe98fb15700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe988105e80 con 0x7fe988073ba0 2023-07-21T18:33:53.407 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:52.571+0000 7fe98d8b1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe988073ba0 0x7fe98810a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:33:53.407 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:52.571+0000 7fe98d8b1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe988073ba0 0x7fe98810a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53884/0 (socket says 172.21.15.39:53884) 2023-07-21T18:33:53.408 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:52.571+0000 7fe98d8b1700 1 -- 172.21.15.39:0/3487207538 learned_addr learned my addr 172.21.15.39:0/3487207538 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:33:53.408 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:52.571+0000 7fe98d8b1700 1 -- 172.21.15.39:0/3487207538 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe98810a9d0 con 0x7fe988073ba0 2023-07-21T18:33:53.408 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:52.571+0000 7fe98d8b1700 1 --2- 172.21.15.39:0/3487207538 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe988073ba0 0x7fe98810a490 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe980009e80 tx=0x7fe98000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ae661da6aabd860 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:33:53.408 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:52.571+0000 7fe98c8af700 1 -- 172.21.15.39:0/3487207538 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe98000e070 con 0x7fe988073ba0 2023-07-21T18:33:53.408 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:52.571+0000 7fe98c8af700 1 -- 172.21.15.39:0/3487207538 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe98000b8c0 con 0x7fe988073ba0 2023-07-21T18:33:53.408 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:52.571+0000 7fe98c8af700 1 -- 172.21.15.39:0/3487207538 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe9800044c0 con 0x7fe988073ba0 2023-07-21T18:33:53.409 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:52.572+0000 7fe98fb15700 1 -- 172.21.15.39:0/3487207538 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe988073ba0 msgr2=0x7fe98810a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:33:53.409 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:52.572+0000 7fe98fb15700 1 --2- 172.21.15.39:0/3487207538 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe988073ba0 0x7fe98810a490 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe980009e80 tx=0x7fe98000b330 comp rx=0 tx=0).stop 2023-07-21T18:33:53.409 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:52.572+0000 7fe98fb15700 1 -- 172.21.15.39:0/3487207538 shutdown_connections 2023-07-21T18:33:53.409 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:52.572+0000 7fe98fb15700 1 --2- 172.21.15.39:0/3487207538 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe988073ba0 0x7fe98810a490 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:33:53.409 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:52.572+0000 7fe98fb15700 1 -- 172.21.15.39:0/3487207538 >> 172.21.15.39:0/3487207538 conn(0x7fe9880ffa40 msgr2=0x7fe988101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:33:53.409 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:52.572+0000 7fe98fb15700 1 -- 172.21.15.39:0/3487207538 shutdown_connections 2023-07-21T18:33:53.409 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:52.572+0000 7fe98fb15700 1 -- 172.21.15.39:0/3487207538 wait complete. 2023-07-21T18:33:53.410 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:52.573+0000 7fe98fb15700 1 Processor -- start 2023-07-21T18:33:53.410 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:52.573+0000 7fe98fb15700 1 -- start start 2023-07-21T18:33:53.410 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:52.573+0000 7fe98fb15700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe98819b440 0x7fe98819b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:33:53.410 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:52.573+0000 7fe98fb15700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe98819bd90 con 0x7fe98819b440 2023-07-21T18:33:53.410 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:52.573+0000 7fe98d8b1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe98819b440 0x7fe98819b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:33:53.410 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:52.573+0000 7fe98d8b1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe98819b440 0x7fe98819b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53886/0 (socket says 172.21.15.39:53886) 2023-07-21T18:33:53.410 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:52.573+0000 7fe98d8b1700 1 -- 172.21.15.39:0/47676982 learned_addr learned my addr 172.21.15.39:0/47676982 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:33:53.411 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:52.574+0000 7fe98d8b1700 1 -- 172.21.15.39:0/47676982 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe980009b30 con 0x7fe98819b440 2023-07-21T18:33:53.411 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:52.574+0000 7fe98d8b1700 1 --2- 172.21.15.39:0/47676982 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe98819b440 0x7fe98819b850 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe9800072a0 tx=0x7fe9800045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:33:53.411 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:52.574+0000 7fe986ffd700 1 -- 172.21.15.39:0/47676982 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe98000e040 con 0x7fe98819b440 2023-07-21T18:33:53.411 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:52.574+0000 7fe986ffd700 1 -- 172.21.15.39:0/47676982 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe98000bd70 con 0x7fe98819b440 2023-07-21T18:33:53.411 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:52.574+0000 7fe98fb15700 1 -- 172.21.15.39:0/47676982 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe98819bf90 con 0x7fe98819b440 2023-07-21T18:33:53.412 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:52.574+0000 7fe986ffd700 1 -- 172.21.15.39:0/47676982 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe98001c6c0 con 0x7fe98819b440 2023-07-21T18:33:53.412 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:52.574+0000 7fe98fb15700 1 -- 172.21.15.39:0/47676982 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe98819ebf0 con 0x7fe98819b440 2023-07-21T18:33:53.412 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:52.575+0000 7fe986ffd700 1 -- 172.21.15.39:0/47676982 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fe98001c980 con 0x7fe98819b440 2023-07-21T18:33:53.412 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:52.575+0000 7fe986ffd700 1 --2- 172.21.15.39:0/47676982 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7fe9700386c0 0x7fe97003ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:33:53.412 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:52.575+0000 7fe986ffd700 1 -- 172.21.15.39:0/47676982 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fe98004c7d0 con 0x7fe98819b440 2023-07-21T18:33:53.412 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:52.575+0000 7fe98d0b0700 1 --2- 172.21.15.39:0/47676982 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7fe9700386c0 0x7fe97003ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:33:53.412 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:52.575+0000 7fe98fb15700 1 -- 172.21.15.39:0/47676982 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe988061b40 con 0x7fe98819b440 2023-07-21T18:33:53.413 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:52.576+0000 7fe98d0b0700 1 --2- 172.21.15.39:0/47676982 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7fe9700386c0 0x7fe97003ab70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe97c006fd0 tx=0x7fe97c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:33:53.413 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:52.581+0000 7fe986ffd700 1 -- 172.21.15.39:0/47676982 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fe980020080 con 0x7fe98819b440 2023-07-21T18:33:53.413 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:52.969+0000 7fe98fb15700 1 -- 172.21.15.39:0/47676982 --> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fe988073e10 con 0x7fe9700386c0 2023-07-21T18:33:53.413 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:52.979+0000 7fe986ffd700 1 -- 172.21.15.39:0/47676982 <== mgr.14120 v2:172.21.15.39:6800/2893866425 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fe988073e10 con 0x7fe9700386c0 2023-07-21T18:33:53.413 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:52.981+0000 7fe98fb15700 1 -- 172.21.15.39:0/47676982 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7fe9700386c0 msgr2=0x7fe97003ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:33:53.413 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:52.981+0000 7fe98fb15700 1 --2- 172.21.15.39:0/47676982 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7fe9700386c0 0x7fe97003ab70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe97c006fd0 tx=0x7fe97c006e40 comp rx=0 tx=0).stop 2023-07-21T18:33:53.413 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:52.981+0000 7fe98fb15700 1 -- 172.21.15.39:0/47676982 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe98819b440 msgr2=0x7fe98819b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:33:53.414 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:52.981+0000 7fe98fb15700 1 --2- 172.21.15.39:0/47676982 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe98819b440 0x7fe98819b850 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe9800072a0 tx=0x7fe9800045f0 comp rx=0 tx=0).stop 2023-07-21T18:33:53.414 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:52.981+0000 7fe98fb15700 1 -- 172.21.15.39:0/47676982 shutdown_connections 2023-07-21T18:33:53.414 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:52.981+0000 7fe98fb15700 1 --2- 172.21.15.39:0/47676982 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7fe9700386c0 0x7fe97003ab70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:33:53.414 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:52.981+0000 7fe98fb15700 1 --2- 172.21.15.39:0/47676982 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe98819b440 0x7fe98819b850 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:33:53.414 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:52.981+0000 7fe98fb15700 1 -- 172.21.15.39:0/47676982 >> 172.21.15.39:0/47676982 conn(0x7fe9880ffa40 msgr2=0x7fe988101140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:33:53.414 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:52.981+0000 7fe98fb15700 1 -- 172.21.15.39:0/47676982 shutdown_connections 2023-07-21T18:33:53.414 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:52.981+0000 7fe98fb15700 1 -- 172.21.15.39:0/47676982 wait complete. 2023-07-21T18:33:53.415 INFO:teuthology.orchestra.run.smithi039.stdout:Deploying unmanaged mgr service... 2023-07-21T18:33:54.359 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:53 smithi039 ceph-mon[96489]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:33:54.359 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:53 smithi039 ceph-mon[96489]: Saving service mon spec with placement count:5 2023-07-21T18:33:54.359 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:53 smithi039 ceph-mon[96489]: from='mgr.14120 172.21.15.39:0/2168989012' entity='mgr.x' 2023-07-21T18:33:54.702 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2023-07-21T18:33:54.703 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:53.888+0000 7f02c073e700 1 Processor -- start 2023-07-21T18:33:54.703 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:53.888+0000 7f02c073e700 1 -- start start 2023-07-21T18:33:54.703 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:53.888+0000 7f02c073e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f02b8106000 0x7f02b81083e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:33:54.703 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:53.888+0000 7f02c073e700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b8073c20 con 0x7f02b8106000 2023-07-21T18:33:54.703 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:53.888+0000 7f02be4da700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f02b8106000 0x7f02b81083e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:33:54.703 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:53.888+0000 7f02be4da700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f02b8106000 0x7f02b81083e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53890/0 (socket says 172.21.15.39:53890) 2023-07-21T18:33:54.703 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:53.888+0000 7f02be4da700 1 -- 172.21.15.39:0/1094311776 learned_addr learned my addr 172.21.15.39:0/1094311776 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:33:54.704 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:53.889+0000 7f02be4da700 1 -- 172.21.15.39:0/1094311776 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02b8073d60 con 0x7f02b8106000 2023-07-21T18:33:54.704 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:53.889+0000 7f02be4da700 1 --2- 172.21.15.39:0/1094311776 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f02b8106000 0x7f02b81083e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f02b40098a0 tx=0x7f02b4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86b06bab361abb2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:33:54.704 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:53.889+0000 7f02bd4d8700 1 -- 172.21.15.39:0/1094311776 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f02b4004160 con 0x7f02b8106000 2023-07-21T18:33:54.704 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:53.889+0000 7f02bd4d8700 1 -- 172.21.15.39:0/1094311776 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f02b40042c0 con 0x7f02b8106000 2023-07-21T18:33:54.704 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:53.889+0000 7f02bd4d8700 1 -- 172.21.15.39:0/1094311776 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f02b4011a00 con 0x7f02b8106000 2023-07-21T18:33:54.704 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:53.890+0000 7f02c073e700 1 -- 172.21.15.39:0/1094311776 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f02b8106000 msgr2=0x7f02b81083e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:33:54.704 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:53.890+0000 7f02c073e700 1 --2- 172.21.15.39:0/1094311776 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f02b8106000 0x7f02b81083e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f02b40098a0 tx=0x7f02b4009bb0 comp rx=0 tx=0).stop 2023-07-21T18:33:54.705 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:53.890+0000 7f02c073e700 1 -- 172.21.15.39:0/1094311776 shutdown_connections 2023-07-21T18:33:54.705 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:53.890+0000 7f02c073e700 1 --2- 172.21.15.39:0/1094311776 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f02b8106000 0x7f02b81083e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:33:54.705 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:53.890+0000 7f02c073e700 1 -- 172.21.15.39:0/1094311776 >> 172.21.15.39:0/1094311776 conn(0x7f02b80ffa40 msgr2=0x7f02b8101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:33:54.705 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:53.890+0000 7f02c073e700 1 -- 172.21.15.39:0/1094311776 shutdown_connections 2023-07-21T18:33:54.705 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:53.890+0000 7f02c073e700 1 -- 172.21.15.39:0/1094311776 wait complete. 2023-07-21T18:33:54.705 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:53.890+0000 7f02c073e700 1 Processor -- start 2023-07-21T18:33:54.705 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:53.890+0000 7f02c073e700 1 -- start start 2023-07-21T18:33:54.705 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:53.891+0000 7f02c073e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f02b8106000 0x7f02b8199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:33:54.706 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:53.891+0000 7f02c073e700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b8199770 con 0x7f02b8106000 2023-07-21T18:33:54.706 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:53.891+0000 7f02be4da700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f02b8106000 0x7f02b8199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:33:54.706 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:53.891+0000 7f02be4da700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f02b8106000 0x7f02b8199230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53892/0 (socket says 172.21.15.39:53892) 2023-07-21T18:33:54.706 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:53.891+0000 7f02be4da700 1 -- 172.21.15.39:0/3382476557 learned_addr learned my addr 172.21.15.39:0/3382476557 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:33:54.706 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:53.891+0000 7f02be4da700 1 -- 172.21.15.39:0/3382476557 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02b4009550 con 0x7f02b8106000 2023-07-21T18:33:54.706 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:53.891+0000 7f02be4da700 1 --2- 172.21.15.39:0/3382476557 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f02b8106000 0x7f02b8199230 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f02b4006af0 tx=0x7f02b4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:33:54.706 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:53.891+0000 7f02af7fe700 1 -- 172.21.15.39:0/3382476557 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f02b4003a40 con 0x7f02b8106000 2023-07-21T18:33:54.707 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:53.891+0000 7f02c073e700 1 -- 172.21.15.39:0/3382476557 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02b8199970 con 0x7f02b8106000 2023-07-21T18:33:54.707 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:53.891+0000 7f02af7fe700 1 -- 172.21.15.39:0/3382476557 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f02b4003ba0 con 0x7f02b8106000 2023-07-21T18:33:54.707 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:53.891+0000 7f02af7fe700 1 -- 172.21.15.39:0/3382476557 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f02b401a700 con 0x7f02b8106000 2023-07-21T18:33:54.707 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:53.891+0000 7f02c073e700 1 -- 172.21.15.39:0/3382476557 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02b8199d90 con 0x7f02b8106000 2023-07-21T18:33:54.707 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:53.892+0000 7f02af7fe700 1 -- 172.21.15.39:0/3382476557 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f02b4024430 con 0x7f02b8106000 2023-07-21T18:33:54.707 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:53.892+0000 7f02c073e700 1 -- 172.21.15.39:0/3382476557 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02b8061b40 con 0x7f02b8106000 2023-07-21T18:33:54.707 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:53.893+0000 7f02af7fe700 1 --2- 172.21.15.39:0/3382476557 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7f02a4038760 0x7f02a403ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:33:54.707 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:53.893+0000 7f02af7fe700 1 -- 172.21.15.39:0/3382476557 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f02b404c750 con 0x7f02b8106000 2023-07-21T18:33:54.708 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:53.893+0000 7f02bdcd9700 1 --2- 172.21.15.39:0/3382476557 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7f02a4038760 0x7f02a403ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:33:54.708 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:53.893+0000 7f02bdcd9700 1 --2- 172.21.15.39:0/3382476557 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7f02a4038760 0x7f02a403ac10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f02a8006fd0 tx=0x7f02a8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:33:54.708 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:53.897+0000 7f02af7fe700 1 -- 172.21.15.39:0/3382476557 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f02b40246e0 con 0x7f02b8106000 2023-07-21T18:33:54.708 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:54.022+0000 7f02af7fe700 1 -- 172.21.15.39:0/3382476557 <== mon.0 v2:172.21.15.39:3300/0 7 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f02b40246e0 con 0x7f02b8106000 2023-07-21T18:33:54.708 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:54.292+0000 7f02c073e700 1 -- 172.21.15.39:0/3382476557 --> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f02a0000d90 con 0x7f02a4038760 2023-07-21T18:33:54.708 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:54.299+0000 7f02af7fe700 1 -- 172.21.15.39:0/3382476557 <== mgr.14120 v2:172.21.15.39:6800/2893866425 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f02a0000d90 con 0x7f02a4038760 2023-07-21T18:33:54.708 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:54.301+0000 7f02c073e700 1 -- 172.21.15.39:0/3382476557 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7f02a4038760 msgr2=0x7f02a403ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:33:54.709 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:54.301+0000 7f02c073e700 1 --2- 172.21.15.39:0/3382476557 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7f02a4038760 0x7f02a403ac10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f02a8006fd0 tx=0x7f02a8006e40 comp rx=0 tx=0).stop 2023-07-21T18:33:54.709 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:54.301+0000 7f02c073e700 1 -- 172.21.15.39:0/3382476557 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f02b8106000 msgr2=0x7f02b8199230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:33:54.709 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:54.301+0000 7f02c073e700 1 --2- 172.21.15.39:0/3382476557 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f02b8106000 0x7f02b8199230 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f02b4006af0 tx=0x7f02b4011ad0 comp rx=0 tx=0).stop 2023-07-21T18:33:54.709 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:54.301+0000 7f02c073e700 1 -- 172.21.15.39:0/3382476557 shutdown_connections 2023-07-21T18:33:54.709 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:54.301+0000 7f02c073e700 1 --2- 172.21.15.39:0/3382476557 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7f02a4038760 0x7f02a403ac10 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:33:54.709 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:54.301+0000 7f02c073e700 1 --2- 172.21.15.39:0/3382476557 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f02b8106000 0x7f02b8199230 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:33:54.709 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:54.301+0000 7f02c073e700 1 -- 172.21.15.39:0/3382476557 >> 172.21.15.39:0/3382476557 conn(0x7f02b80ffa40 msgr2=0x7f02b81010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:33:54.709 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:54.301+0000 7f02c073e700 1 -- 172.21.15.39:0/3382476557 shutdown_connections 2023-07-21T18:33:54.710 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:54.301+0000 7f02c073e700 1 -- 172.21.15.39:0/3382476557 wait complete. 2023-07-21T18:33:55.450 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:55 smithi039 ceph-mon[96489]: from='mgr.14120 172.21.15.39:0/2168989012' entity='mgr.x' 2023-07-21T18:33:55.450 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:55 smithi039 ceph-mon[96489]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:33:55.450 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:55 smithi039 ceph-mon[96489]: Saving service mgr spec with placement count:2 2023-07-21T18:33:55.450 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:55 smithi039 ceph-mon[96489]: from='mgr.14120 172.21.15.39:0/2168989012' entity='mgr.x' 2023-07-21T18:33:55.450 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:55 smithi039 ceph-mon[96489]: from='mgr.14120 172.21.15.39:0/2168989012' entity='mgr.x' 2023-07-21T18:33:56.148 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:55.139+0000 7f7ecb620700 1 Processor -- start 2023-07-21T18:33:56.149 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:55.139+0000 7f7ecb620700 1 -- start start 2023-07-21T18:33:56.149 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:55.139+0000 7f7ecb620700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ec4073b80 0x7f7ec410a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:33:56.149 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:55.139+0000 7f7ecb620700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec4105ea0 con 0x7f7ec4073b80 2023-07-21T18:33:56.149 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:55.139+0000 7f7ec93bc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ec4073b80 0x7f7ec410a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:33:56.150 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:55.139+0000 7f7ec93bc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ec4073b80 0x7f7ec410a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53896/0 (socket says 172.21.15.39:53896) 2023-07-21T18:33:56.150 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:55.139+0000 7f7ec93bc700 1 -- 172.21.15.39:0/2798771705 learned_addr learned my addr 172.21.15.39:0/2798771705 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:33:56.150 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:55.140+0000 7f7ec93bc700 1 -- 172.21.15.39:0/2798771705 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ec410a9f0 con 0x7f7ec4073b80 2023-07-21T18:33:56.151 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:55.140+0000 7f7ec93bc700 1 --2- 172.21.15.39:0/2798771705 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ec4073b80 0x7f7ec410a4b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7ec00098a0 tx=0x7f7ec0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67f7353e0db5d0cf server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:33:56.151 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:55.140+0000 7f7ebbfff700 1 -- 172.21.15.39:0/2798771705 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7ec0004160 con 0x7f7ec4073b80 2023-07-21T18:33:56.151 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:55.140+0000 7f7ebbfff700 1 -- 172.21.15.39:0/2798771705 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ec00042c0 con 0x7f7ec4073b80 2023-07-21T18:33:56.151 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:55.140+0000 7f7ebbfff700 1 -- 172.21.15.39:0/2798771705 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7ec00119b0 con 0x7f7ec4073b80 2023-07-21T18:33:56.152 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:55.140+0000 7f7ecb620700 1 -- 172.21.15.39:0/2798771705 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ec4073b80 msgr2=0x7f7ec410a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:33:56.152 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:55.140+0000 7f7ecb620700 1 --2- 172.21.15.39:0/2798771705 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ec4073b80 0x7f7ec410a4b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7ec00098a0 tx=0x7f7ec0009bb0 comp rx=0 tx=0).stop 2023-07-21T18:33:56.152 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:55.141+0000 7f7ecb620700 1 -- 172.21.15.39:0/2798771705 shutdown_connections 2023-07-21T18:33:56.152 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:55.141+0000 7f7ecb620700 1 --2- 172.21.15.39:0/2798771705 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ec4073b80 0x7f7ec410a4b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:33:56.153 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:55.141+0000 7f7ecb620700 1 -- 172.21.15.39:0/2798771705 >> 172.21.15.39:0/2798771705 conn(0x7f7ec40ffa40 msgr2=0x7f7ec4101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:33:56.153 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:55.141+0000 7f7ecb620700 1 -- 172.21.15.39:0/2798771705 shutdown_connections 2023-07-21T18:33:56.153 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:55.141+0000 7f7ecb620700 1 -- 172.21.15.39:0/2798771705 wait complete. 2023-07-21T18:33:56.154 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:55.141+0000 7f7ecb620700 1 Processor -- start 2023-07-21T18:33:56.154 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:55.141+0000 7f7ecb620700 1 -- start start 2023-07-21T18:33:56.154 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:55.142+0000 7f7ecb620700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ec4073b80 0x7f7ec419b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:33:56.154 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:55.142+0000 7f7ecb620700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec419b9a0 con 0x7f7ec4073b80 2023-07-21T18:33:56.155 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:55.142+0000 7f7ec93bc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ec4073b80 0x7f7ec419b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:33:56.155 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:55.142+0000 7f7ec93bc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ec4073b80 0x7f7ec419b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53898/0 (socket says 172.21.15.39:53898) 2023-07-21T18:33:56.155 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:55.142+0000 7f7ec93bc700 1 -- 172.21.15.39:0/1851184408 learned_addr learned my addr 172.21.15.39:0/1851184408 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:33:56.156 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:55.142+0000 7f7ec93bc700 1 -- 172.21.15.39:0/1851184408 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ec0009550 con 0x7f7ec4073b80 2023-07-21T18:33:56.156 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:55.142+0000 7f7ec93bc700 1 --2- 172.21.15.39:0/1851184408 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ec4073b80 0x7f7ec419b460 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0003e60 tx=0x7f7ec0011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:33:56.156 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:55.142+0000 7f7eba7fc700 1 -- 172.21.15.39:0/1851184408 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7ec0003930 con 0x7f7ec4073b80 2023-07-21T18:33:56.157 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:55.142+0000 7f7eba7fc700 1 -- 172.21.15.39:0/1851184408 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ec0003a90 con 0x7f7ec4073b80 2023-07-21T18:33:56.157 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:55.142+0000 7f7ecb620700 1 -- 172.21.15.39:0/1851184408 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ec419bba0 con 0x7f7ec4073b80 2023-07-21T18:33:56.157 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:55.142+0000 7f7eba7fc700 1 -- 172.21.15.39:0/1851184408 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7ec0019550 con 0x7f7ec4073b80 2023-07-21T18:33:56.158 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:55.142+0000 7f7ecb620700 1 -- 172.21.15.39:0/1851184408 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ec419bfc0 con 0x7f7ec4073b80 2023-07-21T18:33:56.158 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:55.143+0000 7f7eba7fc700 1 -- 172.21.15.39:0/1851184408 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f7ec001a430 con 0x7f7ec4073b80 2023-07-21T18:33:56.158 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:55.143+0000 7f7eba7fc700 1 --2- 172.21.15.39:0/1851184408 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7f7eb00386f0 0x7f7eb003aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:33:56.158 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:55.143+0000 7f7eba7fc700 1 -- 172.21.15.39:0/1851184408 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f7ec004c7b0 con 0x7f7ec4073b80 2023-07-21T18:33:56.159 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:55.143+0000 7f7ecb620700 1 -- 172.21.15.39:0/1851184408 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ec4061b40 con 0x7f7ec4073b80 2023-07-21T18:33:56.159 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:55.144+0000 7f7ec8bbb700 1 --2- 172.21.15.39:0/1851184408 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7f7eb00386f0 0x7f7eb003aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:33:56.159 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:55.144+0000 7f7ec8bbb700 1 --2- 172.21.15.39:0/1851184408 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7f7eb00386f0 0x7f7eb003aba0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7eb4006fd0 tx=0x7f7eb4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:33:56.160 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:55.148+0000 7f7eba7fc700 1 -- 172.21.15.39:0/1851184408 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f7ec00197c0 con 0x7f7ec4073b80 2023-07-21T18:33:56.160 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:55.510+0000 7f7ecb620700 1 -- 172.21.15.39:0/1851184408 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f7ec419ea00 con 0x7f7ec4073b80 2023-07-21T18:33:56.160 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:55.514+0000 7f7eba7fc700 1 -- 172.21.15.39:0/1851184408 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f7ec0018350 con 0x7f7ec4073b80 2023-07-21T18:33:56.160 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:55.516+0000 7f7ecb620700 1 -- 172.21.15.39:0/1851184408 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7f7eb00386f0 msgr2=0x7f7eb003aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:33:56.161 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:55.516+0000 7f7ecb620700 1 --2- 172.21.15.39:0/1851184408 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7f7eb00386f0 0x7f7eb003aba0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7eb4006fd0 tx=0x7f7eb4006e40 comp rx=0 tx=0).stop 2023-07-21T18:33:56.161 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:55.516+0000 7f7ecb620700 1 -- 172.21.15.39:0/1851184408 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ec4073b80 msgr2=0x7f7ec419b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:33:56.161 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:55.516+0000 7f7ecb620700 1 --2- 172.21.15.39:0/1851184408 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ec4073b80 0x7f7ec419b460 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0003e60 tx=0x7f7ec0011a30 comp rx=0 tx=0).stop 2023-07-21T18:33:56.161 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:55.517+0000 7f7ecb620700 1 -- 172.21.15.39:0/1851184408 shutdown_connections 2023-07-21T18:33:56.161 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:55.517+0000 7f7ecb620700 1 --2- 172.21.15.39:0/1851184408 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7f7eb00386f0 0x7f7eb003aba0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:33:56.162 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:55.517+0000 7f7ecb620700 1 --2- 172.21.15.39:0/1851184408 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ec4073b80 0x7f7ec419b460 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:33:56.162 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:55.517+0000 7f7ecb620700 1 -- 172.21.15.39:0/1851184408 >> 172.21.15.39:0/1851184408 conn(0x7f7ec40ffa40 msgr2=0x7f7ec4101120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:33:56.162 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:55.517+0000 7f7ecb620700 1 -- 172.21.15.39:0/1851184408 shutdown_connections 2023-07-21T18:33:56.163 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:55.517+0000 7f7ecb620700 1 -- 172.21.15.39:0/1851184408 wait complete. 2023-07-21T18:33:56.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:56 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/1851184408' entity='client.admin' 2023-07-21T18:33:56.678 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:56 smithi039 ceph-mon[96489]: from='mgr.14120 172.21.15.39:0/2168989012' entity='mgr.x' 2023-07-21T18:33:57.390 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:56.634+0000 7fe34d530700 1 Processor -- start 2023-07-21T18:33:57.390 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:56.634+0000 7fe34d530700 1 -- start start 2023-07-21T18:33:57.391 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:56.634+0000 7fe34d530700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe348073ba0 0x7fe34810a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:33:57.391 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:56.634+0000 7fe34d530700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe348105e80 con 0x7fe348073ba0 2023-07-21T18:33:57.391 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:56.634+0000 7fe346ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe348073ba0 0x7fe34810a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:33:57.391 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:56.634+0000 7fe346ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe348073ba0 0x7fe34810a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53902/0 (socket says 172.21.15.39:53902) 2023-07-21T18:33:57.391 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:56.634+0000 7fe346ffd700 1 -- 172.21.15.39:0/3056951561 learned_addr learned my addr 172.21.15.39:0/3056951561 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:33:57.391 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:56.635+0000 7fe346ffd700 1 -- 172.21.15.39:0/3056951561 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe34810a9d0 con 0x7fe348073ba0 2023-07-21T18:33:57.391 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:56.635+0000 7fe346ffd700 1 --2- 172.21.15.39:0/3056951561 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe348073ba0 0x7fe34810a490 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe33c0098a0 tx=0x7fe33c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87d2a71d8e6abde4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:33:57.392 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:56.635+0000 7fe345ffb700 1 -- 172.21.15.39:0/3056951561 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe33c004160 con 0x7fe348073ba0 2023-07-21T18:33:57.392 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:56.635+0000 7fe345ffb700 1 -- 172.21.15.39:0/3056951561 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe33c0042c0 con 0x7fe348073ba0 2023-07-21T18:33:57.392 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:56.635+0000 7fe345ffb700 1 -- 172.21.15.39:0/3056951561 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe33c0119b0 con 0x7fe348073ba0 2023-07-21T18:33:57.392 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:56.635+0000 7fe34d530700 1 -- 172.21.15.39:0/3056951561 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe348073ba0 msgr2=0x7fe34810a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:33:57.392 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:56.635+0000 7fe34d530700 1 --2- 172.21.15.39:0/3056951561 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe348073ba0 0x7fe34810a490 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe33c0098a0 tx=0x7fe33c009bb0 comp rx=0 tx=0).stop 2023-07-21T18:33:57.392 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:56.636+0000 7fe34d530700 1 -- 172.21.15.39:0/3056951561 shutdown_connections 2023-07-21T18:33:57.392 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:56.636+0000 7fe34d530700 1 --2- 172.21.15.39:0/3056951561 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe348073ba0 0x7fe34810a490 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:33:57.393 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:56.636+0000 7fe34d530700 1 -- 172.21.15.39:0/3056951561 >> 172.21.15.39:0/3056951561 conn(0x7fe3480ffa40 msgr2=0x7fe348101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:33:57.393 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:56.636+0000 7fe34d530700 1 -- 172.21.15.39:0/3056951561 shutdown_connections 2023-07-21T18:33:57.393 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:56.636+0000 7fe34d530700 1 -- 172.21.15.39:0/3056951561 wait complete. 2023-07-21T18:33:57.393 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:56.636+0000 7fe34d530700 1 Processor -- start 2023-07-21T18:33:57.393 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:56.637+0000 7fe34d530700 1 -- start start 2023-07-21T18:33:57.393 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:56.637+0000 7fe34d530700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe34819b420 0x7fe34819b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:33:57.393 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:56.637+0000 7fe34d530700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe34819bd70 con 0x7fe34819b420 2023-07-21T18:33:57.393 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:56.637+0000 7fe346ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe34819b420 0x7fe34819b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:33:57.394 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:56.637+0000 7fe346ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe34819b420 0x7fe34819b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53904/0 (socket says 172.21.15.39:53904) 2023-07-21T18:33:57.394 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:56.637+0000 7fe346ffd700 1 -- 172.21.15.39:0/2350054956 learned_addr learned my addr 172.21.15.39:0/2350054956 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:33:57.394 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:56.637+0000 7fe346ffd700 1 -- 172.21.15.39:0/2350054956 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe33c009550 con 0x7fe34819b420 2023-07-21T18:33:57.394 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:56.637+0000 7fe346ffd700 1 --2- 172.21.15.39:0/2350054956 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe34819b420 0x7fe34819b830 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe33c000c00 tx=0x7fe33c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:33:57.394 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:56.638+0000 7fe32ffff700 1 -- 172.21.15.39:0/2350054956 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe33c0039f0 con 0x7fe34819b420 2023-07-21T18:33:57.394 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:56.638+0000 7fe34d530700 1 -- 172.21.15.39:0/2350054956 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe34819bf70 con 0x7fe34819b420 2023-07-21T18:33:57.394 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:56.638+0000 7fe32ffff700 1 -- 172.21.15.39:0/2350054956 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe33c003b50 con 0x7fe34819b420 2023-07-21T18:33:57.394 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:56.638+0000 7fe32ffff700 1 -- 172.21.15.39:0/2350054956 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe33c01a780 con 0x7fe34819b420 2023-07-21T18:33:57.395 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:56.638+0000 7fe34d530700 1 -- 172.21.15.39:0/2350054956 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe34819ebd0 con 0x7fe34819b420 2023-07-21T18:33:57.395 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:56.638+0000 7fe32ffff700 1 -- 172.21.15.39:0/2350054956 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fe33c01aa40 con 0x7fe34819b420 2023-07-21T18:33:57.395 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:56.639+0000 7fe32ffff700 1 --2- 172.21.15.39:0/2350054956 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7fe330038710 0x7fe33003abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:33:57.395 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:56.639+0000 7fe32ffff700 1 -- 172.21.15.39:0/2350054956 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fe33c024850 con 0x7fe34819b420 2023-07-21T18:33:57.395 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:56.639+0000 7fe3467fc700 1 --2- 172.21.15.39:0/2350054956 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7fe330038710 0x7fe33003abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:33:57.395 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:56.639+0000 7fe34d530700 1 -- 172.21.15.39:0/2350054956 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe348061b40 con 0x7fe34819b420 2023-07-21T18:33:57.395 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:56.639+0000 7fe3467fc700 1 --2- 172.21.15.39:0/2350054956 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7fe330038710 0x7fe33003abc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe338006fd0 tx=0x7fe338006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:33:57.396 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:56.644+0000 7fe32ffff700 1 -- 172.21.15.39:0/2350054956 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fe33c01acf0 con 0x7fe34819b420 2023-07-21T18:33:57.396 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:57.004+0000 7fe34d530700 1 -- 172.21.15.39:0/2350054956 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7fe34819ef50 con 0x7fe34819b420 2023-07-21T18:33:57.396 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:57.010+0000 7fe32ffff700 1 -- 172.21.15.39:0/2350054956 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7fe33c018350 con 0x7fe34819b420 2023-07-21T18:33:57.396 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:57.012+0000 7fe34d530700 1 -- 172.21.15.39:0/2350054956 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7fe330038710 msgr2=0x7fe33003abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:33:57.396 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:57.012+0000 7fe34d530700 1 --2- 172.21.15.39:0/2350054956 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7fe330038710 0x7fe33003abc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe338006fd0 tx=0x7fe338006e40 comp rx=0 tx=0).stop 2023-07-21T18:33:57.396 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:57.012+0000 7fe34d530700 1 -- 172.21.15.39:0/2350054956 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe34819b420 msgr2=0x7fe34819b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:33:57.396 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:57.012+0000 7fe34d530700 1 --2- 172.21.15.39:0/2350054956 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe34819b420 0x7fe34819b830 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe33c000c00 tx=0x7fe33c011a30 comp rx=0 tx=0).stop 2023-07-21T18:33:57.396 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:57.012+0000 7fe34d530700 1 -- 172.21.15.39:0/2350054956 shutdown_connections 2023-07-21T18:33:57.397 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:57.012+0000 7fe34d530700 1 --2- 172.21.15.39:0/2350054956 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7fe330038710 0x7fe33003abc0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:33:57.397 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:57.012+0000 7fe34d530700 1 --2- 172.21.15.39:0/2350054956 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe34819b420 0x7fe34819b830 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:33:57.397 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:57.012+0000 7fe34d530700 1 -- 172.21.15.39:0/2350054956 >> 172.21.15.39:0/2350054956 conn(0x7fe3480ffa40 msgr2=0x7fe3481010c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:33:57.397 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:57.012+0000 7fe34d530700 1 -- 172.21.15.39:0/2350054956 shutdown_connections 2023-07-21T18:33:57.397 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:57.012+0000 7fe34d530700 1 -- 172.21.15.39:0/2350054956 wait complete. 2023-07-21T18:33:57.397 INFO:teuthology.orchestra.run.smithi039.stdout:Enabling the dashboard module... 2023-07-21T18:33:58.373 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:58 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/2350054956' entity='client.admin' 2023-07-21T18:33:58.373 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:58 smithi039 ceph-mon[96489]: from='mgr.14120 172.21.15.39:0/2168989012' entity='mgr.x' 2023-07-21T18:33:59.354 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:33:59 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/3932469396' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2023-07-21T18:33:59.355 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:59 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: ignoring --setuser ceph since I am not root 2023-07-21T18:33:59.355 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:59 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: ignoring --setgroup ceph since I am not root 2023-07-21T18:33:59.355 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:59 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:33:59.223+0000 7fe4ca152200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-21T18:33:59.355 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:59 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:33:59.320+0000 7fe4ca152200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-21T18:33:59.449 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:57.850+0000 7faccf63a700 1 Processor -- start 2023-07-21T18:33:59.449 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:57.850+0000 7faccf63a700 1 -- start start 2023-07-21T18:33:59.449 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:57.850+0000 7faccf63a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7facc8073ba0 0x7facc810a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:33:59.449 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:57.850+0000 7faccf63a700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc8105e80 con 0x7facc8073ba0 2023-07-21T18:33:59.449 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:57.850+0000 7faccd3d6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7facc8073ba0 0x7facc810a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:33:59.449 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:57.850+0000 7faccd3d6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7facc8073ba0 0x7facc810a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53908/0 (socket says 172.21.15.39:53908) 2023-07-21T18:33:59.450 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:57.850+0000 7faccd3d6700 1 -- 172.21.15.39:0/1897808319 learned_addr learned my addr 172.21.15.39:0/1897808319 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:33:59.450 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:57.851+0000 7faccd3d6700 1 -- 172.21.15.39:0/1897808319 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc810a9d0 con 0x7facc8073ba0 2023-07-21T18:33:59.450 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:57.851+0000 7faccd3d6700 1 --2- 172.21.15.39:0/1897808319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7facc8073ba0 0x7facc810a490 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7facc40098a0 tx=0x7facc4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e14a1ce48c3900c7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:33:59.450 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:57.851+0000 7facbffff700 1 -- 172.21.15.39:0/1897808319 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7facc4004160 con 0x7facc8073ba0 2023-07-21T18:33:59.450 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:57.851+0000 7facbffff700 1 -- 172.21.15.39:0/1897808319 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7facc40042c0 con 0x7facc8073ba0 2023-07-21T18:33:59.450 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:57.851+0000 7facbffff700 1 -- 172.21.15.39:0/1897808319 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7facc40119b0 con 0x7facc8073ba0 2023-07-21T18:33:59.450 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:57.852+0000 7faccf63a700 1 -- 172.21.15.39:0/1897808319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7facc8073ba0 msgr2=0x7facc810a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:33:59.451 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:57.852+0000 7faccf63a700 1 --2- 172.21.15.39:0/1897808319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7facc8073ba0 0x7facc810a490 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7facc40098a0 tx=0x7facc4009bb0 comp rx=0 tx=0).stop 2023-07-21T18:33:59.451 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:57.852+0000 7faccf63a700 1 -- 172.21.15.39:0/1897808319 shutdown_connections 2023-07-21T18:33:59.451 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:57.852+0000 7faccf63a700 1 --2- 172.21.15.39:0/1897808319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7facc8073ba0 0x7facc810a490 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:33:59.454 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:57.852+0000 7faccf63a700 1 -- 172.21.15.39:0/1897808319 >> 172.21.15.39:0/1897808319 conn(0x7facc80ffa40 msgr2=0x7facc8101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:33:59.454 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:57.852+0000 7faccf63a700 1 -- 172.21.15.39:0/1897808319 shutdown_connections 2023-07-21T18:33:59.454 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:57.852+0000 7faccf63a700 1 -- 172.21.15.39:0/1897808319 wait complete. 2023-07-21T18:33:59.454 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:57.853+0000 7faccf63a700 1 Processor -- start 2023-07-21T18:33:59.454 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:57.853+0000 7faccf63a700 1 -- start start 2023-07-21T18:33:59.454 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:57.853+0000 7faccf63a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7facc819b4c0 0x7facc819b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:33:59.454 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:57.853+0000 7faccf63a700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc819be10 con 0x7facc819b4c0 2023-07-21T18:33:59.455 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:57.853+0000 7faccd3d6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7facc819b4c0 0x7facc819b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:33:59.455 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:57.853+0000 7faccd3d6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7facc819b4c0 0x7facc819b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53910/0 (socket says 172.21.15.39:53910) 2023-07-21T18:33:59.455 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:57.853+0000 7faccd3d6700 1 -- 172.21.15.39:0/3932469396 learned_addr learned my addr 172.21.15.39:0/3932469396 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:33:59.455 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:57.854+0000 7faccd3d6700 1 -- 172.21.15.39:0/3932469396 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc4009550 con 0x7facc819b4c0 2023-07-21T18:33:59.455 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:57.854+0000 7faccd3d6700 1 --2- 172.21.15.39:0/3932469396 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7facc819b4c0 0x7facc819b8d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7facc4006af0 tx=0x7facc4011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:33:59.455 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:57.854+0000 7facbe7fc700 1 -- 172.21.15.39:0/3932469396 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7facc4003a40 con 0x7facc819b4c0 2023-07-21T18:33:59.456 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:57.854+0000 7faccf63a700 1 -- 172.21.15.39:0/3932469396 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facc819c010 con 0x7facc819b4c0 2023-07-21T18:33:59.456 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:57.854+0000 7facbe7fc700 1 -- 172.21.15.39:0/3932469396 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7facc4003ba0 con 0x7facc819b4c0 2023-07-21T18:33:59.456 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:57.854+0000 7facbe7fc700 1 -- 172.21.15.39:0/3932469396 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7facc4019690 con 0x7facc819b4c0 2023-07-21T18:33:59.456 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:57.854+0000 7faccf63a700 1 -- 172.21.15.39:0/3932469396 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facc819ec70 con 0x7facc819b4c0 2023-07-21T18:33:59.456 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:57.855+0000 7facbe7fc700 1 -- 172.21.15.39:0/3932469396 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7facc4019900 con 0x7facc819b4c0 2023-07-21T18:33:59.457 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:57.855+0000 7facbe7fc700 1 --2- 172.21.15.39:0/3932469396 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7facb4038710 0x7facb403abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:33:59.457 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:57.855+0000 7facbe7fc700 1 -- 172.21.15.39:0/3932469396 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7facc404c900 con 0x7facc819b4c0 2023-07-21T18:33:59.457 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:57.855+0000 7facccbd5700 1 --2- 172.21.15.39:0/3932469396 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7facb4038710 0x7facb403abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:33:59.457 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:57.855+0000 7faccf63a700 1 -- 172.21.15.39:0/3932469396 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facc8061b40 con 0x7facc819b4c0 2023-07-21T18:33:59.457 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:57.856+0000 7facccbd5700 1 --2- 172.21.15.39:0/3932469396 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7facb4038710 0x7facb403abc0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7facb8006fd0 tx=0x7facb8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:33:59.457 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:57.861+0000 7facbe7fc700 1 -- 172.21.15.39:0/3932469396 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7facc4019e30 con 0x7facc819b4c0 2023-07-21T18:33:59.457 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:58.348+0000 7faccf63a700 1 -- 172.21.15.39:0/3932469396 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7facc8073ba0 con 0x7facc819b4c0 2023-07-21T18:33:59.457 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:59.020+0000 7facbe7fc700 1 -- 172.21.15.39:0/3932469396 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7facc402d9a0 con 0x7facc819b4c0 2023-07-21T18:33:59.458 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:59.021+0000 7facbe7fc700 1 -- 172.21.15.39:0/3932469396 <== mon.0 v2:172.21.15.39:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7facc4024e10 con 0x7facc819b4c0 2023-07-21T18:33:59.458 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:59.023+0000 7faccf63a700 1 -- 172.21.15.39:0/3932469396 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7facb4038710 msgr2=0x7facb403abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:33:59.458 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:59.023+0000 7faccf63a700 1 --2- 172.21.15.39:0/3932469396 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7facb4038710 0x7facb403abc0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7facb8006fd0 tx=0x7facb8006e40 comp rx=0 tx=0).stop 2023-07-21T18:33:59.458 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:59.023+0000 7faccf63a700 1 -- 172.21.15.39:0/3932469396 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7facc819b4c0 msgr2=0x7facc819b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:33:59.459 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:59.023+0000 7faccf63a700 1 --2- 172.21.15.39:0/3932469396 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7facc819b4c0 0x7facc819b8d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7facc4006af0 tx=0x7facc4011a10 comp rx=0 tx=0).stop 2023-07-21T18:33:59.459 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:59.024+0000 7faccf63a700 1 -- 172.21.15.39:0/3932469396 shutdown_connections 2023-07-21T18:33:59.459 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:59.024+0000 7faccf63a700 1 --2- 172.21.15.39:0/3932469396 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7facb4038710 0x7facb403abc0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:33:59.459 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:59.024+0000 7faccf63a700 1 --2- 172.21.15.39:0/3932469396 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7facc819b4c0 0x7facc819b8d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:33:59.459 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:59.024+0000 7faccf63a700 1 -- 172.21.15.39:0/3932469396 >> 172.21.15.39:0/3932469396 conn(0x7facc80ffa40 msgr2=0x7facc8101140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:33:59.459 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:59.024+0000 7faccf63a700 1 -- 172.21.15.39:0/3932469396 shutdown_connections 2023-07-21T18:33:59.459 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:59.024+0000 7faccf63a700 1 -- 172.21.15.39:0/3932469396 wait complete. 2023-07-21T18:33:59.610 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:59 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:33:59.431+0000 7fe4ca152200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-21T18:33:59.610 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:59 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:33:59.548+0000 7fe4ca152200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-21T18:33:59.949 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:33:59 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:33:59.764+0000 7fe4ca152200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-21T18:34:00.307 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:34:00 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:34:00.164+0000 7fe4ca152200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-21T18:34:00.308 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:00 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/3932469396' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2023-07-21T18:34:00.308 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:00 smithi039 ceph-mon[96489]: mgrmap e9: x(active, since 18s) 2023-07-21T18:34:00.583 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:34:00 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:34:00.273+0000 7fe4ca152200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-21T18:34:00.583 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:34:00 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:34:00.429+0000 7fe4ca152200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-21T18:34:00.583 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:34:00 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:34:00.548+0000 7fe4ca152200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-21T18:34:00.695 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout { 2023-07-21T18:34:00.701 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 9, 2023-07-21T18:34:00.701 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "available": true, 2023-07-21T18:34:00.701 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-07-21T18:34:00.701 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-07-21T18:34:00.702 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout } 2023-07-21T18:34:00.702 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:59.876+0000 7fdb8cf67700 1 Processor -- start 2023-07-21T18:34:00.702 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:59.876+0000 7fdb8cf67700 1 -- start start 2023-07-21T18:34:00.702 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:59.876+0000 7fdb8cf67700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdb88106070 0x7fdb88108450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:34:00.702 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:59.876+0000 7fdb8cf67700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb88073bc0 con 0x7fdb88106070 2023-07-21T18:34:00.702 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:59.877+0000 7fdb8659c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdb88106070 0x7fdb88108450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:34:00.702 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:59.877+0000 7fdb8659c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdb88106070 0x7fdb88108450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53924/0 (socket says 172.21.15.39:53924) 2023-07-21T18:34:00.703 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:59.877+0000 7fdb8659c700 1 -- 172.21.15.39:0/2185892799 learned_addr learned my addr 172.21.15.39:0/2185892799 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:34:00.703 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:59.877+0000 7fdb8659c700 1 -- 172.21.15.39:0/2185892799 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb88073d00 con 0x7fdb88106070 2023-07-21T18:34:00.703 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:59.877+0000 7fdb8659c700 1 --2- 172.21.15.39:0/2185892799 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdb88106070 0x7fdb88108450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdb780098a0 tx=0x7fdb78009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13fcd2f4a7086e54 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:34:00.703 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:59.877+0000 7fdb8559a700 1 -- 172.21.15.39:0/2185892799 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdb78004160 con 0x7fdb88106070 2023-07-21T18:34:00.703 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:59.877+0000 7fdb8559a700 1 -- 172.21.15.39:0/2185892799 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb780042c0 con 0x7fdb88106070 2023-07-21T18:34:00.704 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:59.877+0000 7fdb8559a700 1 -- 172.21.15.39:0/2185892799 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdb780119b0 con 0x7fdb88106070 2023-07-21T18:34:00.704 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:59.878+0000 7fdb8cf67700 1 -- 172.21.15.39:0/2185892799 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdb88106070 msgr2=0x7fdb88108450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:34:00.704 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:59.878+0000 7fdb8cf67700 1 --2- 172.21.15.39:0/2185892799 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdb88106070 0x7fdb88108450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdb780098a0 tx=0x7fdb78009bb0 comp rx=0 tx=0).stop 2023-07-21T18:34:00.704 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:59.878+0000 7fdb8cf67700 1 -- 172.21.15.39:0/2185892799 shutdown_connections 2023-07-21T18:34:00.704 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:59.878+0000 7fdb8cf67700 1 --2- 172.21.15.39:0/2185892799 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdb88106070 0x7fdb88108450 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:34:00.704 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:59.878+0000 7fdb8cf67700 1 -- 172.21.15.39:0/2185892799 >> 172.21.15.39:0/2185892799 conn(0x7fdb880ffa50 msgr2=0x7fdb88101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:34:00.704 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:59.878+0000 7fdb8cf67700 1 -- 172.21.15.39:0/2185892799 shutdown_connections 2023-07-21T18:34:00.705 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:59.878+0000 7fdb8cf67700 1 -- 172.21.15.39:0/2185892799 wait complete. 2023-07-21T18:34:00.705 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:59.879+0000 7fdb8cf67700 1 Processor -- start 2023-07-21T18:34:00.705 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:59.879+0000 7fdb8cf67700 1 -- start start 2023-07-21T18:34:00.705 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:59.879+0000 7fdb8cf67700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdb88106070 0x7fdb881992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:34:00.705 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:59.879+0000 7fdb8cf67700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb88199810 con 0x7fdb88106070 2023-07-21T18:34:00.705 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:59.879+0000 7fdb8659c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdb88106070 0x7fdb881992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:34:00.705 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:59.880+0000 7fdb8659c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdb88106070 0x7fdb881992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53926/0 (socket says 172.21.15.39:53926) 2023-07-21T18:34:00.705 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:59.880+0000 7fdb8659c700 1 -- 172.21.15.39:0/3594522597 learned_addr learned my addr 172.21.15.39:0/3594522597 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:34:00.706 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:59.880+0000 7fdb8659c700 1 -- 172.21.15.39:0/3594522597 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb78009550 con 0x7fdb88106070 2023-07-21T18:34:00.706 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:59.880+0000 7fdb8659c700 1 --2- 172.21.15.39:0/3594522597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdb88106070 0x7fdb881992d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdb78003e60 tx=0x7fdb78011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:34:00.706 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:59.880+0000 7fdb777fe700 1 -- 172.21.15.39:0/3594522597 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdb78003a60 con 0x7fdb88106070 2023-07-21T18:34:00.706 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:59.880+0000 7fdb8cf67700 1 -- 172.21.15.39:0/3594522597 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb88199a10 con 0x7fdb88106070 2023-07-21T18:34:00.706 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:59.880+0000 7fdb777fe700 1 -- 172.21.15.39:0/3594522597 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb78003bc0 con 0x7fdb88106070 2023-07-21T18:34:00.706 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:59.880+0000 7fdb777fe700 1 -- 172.21.15.39:0/3594522597 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdb780196c0 con 0x7fdb88106070 2023-07-21T18:34:00.706 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:59.880+0000 7fdb8cf67700 1 -- 172.21.15.39:0/3594522597 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb88199e30 con 0x7fdb88106070 2023-07-21T18:34:00.707 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:59.881+0000 7fdb777fe700 1 -- 172.21.15.39:0/3594522597 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7fdb78019930 con 0x7fdb88106070 2023-07-21T18:34:00.707 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:59.881+0000 7fdb777fe700 1 --2- 172.21.15.39:0/3594522597 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7fdb70038790 0x7fdb7003ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:34:00.707 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:59.881+0000 7fdb777fe700 1 -- 172.21.15.39:0/3594522597 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fdb7804d8d0 con 0x7fdb88106070 2023-07-21T18:34:00.707 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:59.881+0000 7fdb85d9b700 1 -- 172.21.15.39:0/3594522597 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7fdb70038790 msgr2=0x7fdb7003ac40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/2893866425 2023-07-21T18:34:00.707 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:59.881+0000 7fdb85d9b700 1 --2- 172.21.15.39:0/3594522597 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7fdb70038790 0x7fdb7003ac40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-21T18:34:00.707 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:59.882+0000 7fdb8cf67700 1 -- 172.21.15.39:0/3594522597 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb88061b40 con 0x7fdb88106070 2023-07-21T18:34:00.707 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:33:59.886+0000 7fdb777fe700 1 -- 172.21.15.39:0/3594522597 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fdb78019e60 con 0x7fdb88106070 2023-07-21T18:34:00.708 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:00.082+0000 7fdb85d9b700 1 -- 172.21.15.39:0/3594522597 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7fdb70038790 msgr2=0x7fdb7003ac40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/2893866425 2023-07-21T18:34:00.708 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:00.082+0000 7fdb85d9b700 1 --2- 172.21.15.39:0/3594522597 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7fdb70038790 0x7fdb7003ac40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-21T18:34:00.708 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:00.351+0000 7fdb8cf67700 1 -- 172.21.15.39:0/3594522597 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fdb8819c870 con 0x7fdb88106070 2023-07-21T18:34:00.708 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:00.352+0000 7fdb777fe700 1 -- 172.21.15.39:0/3594522597 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7fdb78010b70 con 0x7fdb88106070 2023-07-21T18:34:00.708 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:00.354+0000 7fdb8cf67700 1 -- 172.21.15.39:0/3594522597 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7fdb70038790 msgr2=0x7fdb7003ac40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-21T18:34:00.708 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:00.354+0000 7fdb8cf67700 1 --2- 172.21.15.39:0/3594522597 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7fdb70038790 0x7fdb7003ac40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:34:00.708 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:00.354+0000 7fdb8cf67700 1 -- 172.21.15.39:0/3594522597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdb88106070 msgr2=0x7fdb881992d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:34:00.709 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:00.354+0000 7fdb8cf67700 1 --2- 172.21.15.39:0/3594522597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdb88106070 0x7fdb881992d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdb78003e60 tx=0x7fdb78011a30 comp rx=0 tx=0).stop 2023-07-21T18:34:00.709 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:00.354+0000 7fdb8cf67700 1 -- 172.21.15.39:0/3594522597 shutdown_connections 2023-07-21T18:34:00.709 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:00.354+0000 7fdb8cf67700 1 --2- 172.21.15.39:0/3594522597 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7fdb70038790 0x7fdb7003ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:34:00.709 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:00.354+0000 7fdb8cf67700 1 --2- 172.21.15.39:0/3594522597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdb88106070 0x7fdb881992d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:34:00.709 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:00.354+0000 7fdb8cf67700 1 -- 172.21.15.39:0/3594522597 >> 172.21.15.39:0/3594522597 conn(0x7fdb880ffa50 msgr2=0x7fdb88101120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:34:00.709 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:00.354+0000 7fdb8cf67700 1 -- 172.21.15.39:0/3594522597 shutdown_connections 2023-07-21T18:34:00.709 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:00.354+0000 7fdb8cf67700 1 -- 172.21.15.39:0/3594522597 wait complete. 2023-07-21T18:34:00.709 INFO:teuthology.orchestra.run.smithi039.stdout:Waiting for the mgr to restart... 2023-07-21T18:34:00.710 INFO:teuthology.orchestra.run.smithi039.stdout:Waiting for mgr epoch 9... 2023-07-21T18:34:01.200 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:01 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/3594522597' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-07-21T18:34:01.200 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:34:01 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:34:01.071+0000 7fe4ca152200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-21T18:34:02.199 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:34:01 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:34:01.759+0000 7fe4ca152200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-21T18:34:02.200 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:34:01 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:34:01.952+0000 7fe4ca152200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-21T18:34:03.199 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:34:02 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:34:02.913+0000 7fe4ca152200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-21T18:34:03.200 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:34:03 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:34:03.024+0000 7fe4ca152200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-21T18:34:03.625 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:34:03 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:34:03.243+0000 7fe4ca152200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-21T18:34:03.878 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:34:03 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:34:03.591+0000 7fe4ca152200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-21T18:34:04.200 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:34:03 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:34:03.844+0000 7fe4ca152200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-21T18:34:04.200 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:34:04 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:34:04.027+0000 7fe4ca152200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-21T18:34:04.700 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:34:04 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:34:04.355+0000 7fe4ca152200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-21T18:34:05.615 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:34:05 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:34:05.180+0000 7fe4ca152200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-21T18:34:05.615 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:34:05 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:34:05.337+0000 7fe4ca152200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-21T18:34:05.949 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:34:05 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:34:05.580+0000 7fe4ca152200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-21T18:34:05.950 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:34:05 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:34:05.718+0000 7fe4ca152200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-21T18:34:06.449 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:34:05 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:34:05.989+0000 7fe4ca152200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-21T18:34:07.450 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:34:07 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:34:07.026+0000 7fe4ca152200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-21T18:34:07.450 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:07 smithi039 ceph-mon[96489]: Active manager daemon x restarted 2023-07-21T18:34:07.450 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:07 smithi039 ceph-mon[96489]: Activating manager daemon x 2023-07-21T18:34:07.450 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:07 smithi039 ceph-mon[96489]: osdmap e3: 0 total, 0 up, 0 in 2023-07-21T18:34:07.450 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:07 smithi039 ceph-mon[96489]: mgrmap e10: x(active, starting, since 0.00804447s) 2023-07-21T18:34:07.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:07 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T18:34:07.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:07 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-07-21T18:34:07.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:07 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-07-21T18:34:07.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:07 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-07-21T18:34:07.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:07 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-07-21T18:34:07.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:07 smithi039 ceph-mon[96489]: Manager daemon x is now available 2023-07-21T18:34:07.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:07 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:34:08.357 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout { 2023-07-21T18:34:08.362 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2023-07-21T18:34:08.363 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "initialized": true 2023-07-21T18:34:08.363 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout } 2023-07-21T18:34:08.363 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:01.181+0000 7ff31f205700 1 Processor -- start 2023-07-21T18:34:08.363 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:01.181+0000 7ff31f205700 1 -- start start 2023-07-21T18:34:08.363 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:01.181+0000 7ff31f205700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff318103e30 0x7ff318104240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:34:08.363 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:01.181+0000 7ff31f205700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff318073be0 con 0x7ff318103e30 2023-07-21T18:34:08.363 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:01.182+0000 7ff31cfa1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff318103e30 0x7ff318104240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:34:08.364 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:01.182+0000 7ff31cfa1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff318103e30 0x7ff318104240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53934/0 (socket says 172.21.15.39:53934) 2023-07-21T18:34:08.364 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:01.182+0000 7ff31cfa1700 1 -- 172.21.15.39:0/2878685611 learned_addr learned my addr 172.21.15.39:0/2878685611 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:34:08.364 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:01.182+0000 7ff31cfa1700 1 -- 172.21.15.39:0/2878685611 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff318073d20 con 0x7ff318103e30 2023-07-21T18:34:08.364 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:01.182+0000 7ff31cfa1700 1 --2- 172.21.15.39:0/2878685611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff318103e30 0x7ff318104240 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff30c0098a0 tx=0x7ff30c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7206415229709d0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:34:08.364 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:01.182+0000 7ff3177fe700 1 -- 172.21.15.39:0/2878685611 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff30c004160 con 0x7ff318103e30 2023-07-21T18:34:08.364 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:01.182+0000 7ff3177fe700 1 -- 172.21.15.39:0/2878685611 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff30c0042c0 con 0x7ff318103e30 2023-07-21T18:34:08.364 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:01.182+0000 7ff3177fe700 1 -- 172.21.15.39:0/2878685611 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff30c0119b0 con 0x7ff318103e30 2023-07-21T18:34:08.365 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:01.183+0000 7ff31f205700 1 -- 172.21.15.39:0/2878685611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff318103e30 msgr2=0x7ff318104240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:34:08.365 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:01.183+0000 7ff31f205700 1 --2- 172.21.15.39:0/2878685611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff318103e30 0x7ff318104240 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff30c0098a0 tx=0x7ff30c009bb0 comp rx=0 tx=0).stop 2023-07-21T18:34:08.365 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:01.183+0000 7ff31f205700 1 -- 172.21.15.39:0/2878685611 shutdown_connections 2023-07-21T18:34:08.365 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:01.183+0000 7ff31f205700 1 --2- 172.21.15.39:0/2878685611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff318103e30 0x7ff318104240 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:34:08.365 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:01.183+0000 7ff31f205700 1 -- 172.21.15.39:0/2878685611 >> 172.21.15.39:0/2878685611 conn(0x7ff3180ffa40 msgr2=0x7ff318101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:34:08.365 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:01.183+0000 7ff31f205700 1 -- 172.21.15.39:0/2878685611 shutdown_connections 2023-07-21T18:34:08.365 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:01.183+0000 7ff31f205700 1 -- 172.21.15.39:0/2878685611 wait complete. 2023-07-21T18:34:08.365 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:01.184+0000 7ff31f205700 1 Processor -- start 2023-07-21T18:34:08.366 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:01.184+0000 7ff31f205700 1 -- start start 2023-07-21T18:34:08.366 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:01.184+0000 7ff31f205700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff318103e30 0x7ff31819b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:34:08.366 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:01.184+0000 7ff31f205700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff31819b970 con 0x7ff318103e30 2023-07-21T18:34:08.366 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:01.184+0000 7ff31cfa1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff318103e30 0x7ff31819b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:34:08.366 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:01.184+0000 7ff31cfa1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff318103e30 0x7ff31819b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53936/0 (socket says 172.21.15.39:53936) 2023-07-21T18:34:08.366 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:01.184+0000 7ff31cfa1700 1 -- 172.21.15.39:0/614615285 learned_addr learned my addr 172.21.15.39:0/614615285 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:34:08.366 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:01.184+0000 7ff31cfa1700 1 -- 172.21.15.39:0/614615285 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff30c009550 con 0x7ff318103e30 2023-07-21T18:34:08.366 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:01.185+0000 7ff31cfa1700 1 --2- 172.21.15.39:0/614615285 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff318103e30 0x7ff31819b430 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff30c006af0 tx=0x7ff30c011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:34:08.367 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:01.185+0000 7ff315ffb700 1 -- 172.21.15.39:0/614615285 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff30c003a40 con 0x7ff318103e30 2023-07-21T18:34:08.367 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:01.185+0000 7ff315ffb700 1 -- 172.21.15.39:0/614615285 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff30c003ba0 con 0x7ff318103e30 2023-07-21T18:34:08.367 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:01.185+0000 7ff31f205700 1 -- 172.21.15.39:0/614615285 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff31819bbd0 con 0x7ff318103e30 2023-07-21T18:34:08.367 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:01.185+0000 7ff315ffb700 1 -- 172.21.15.39:0/614615285 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff30c019690 con 0x7ff318103e30 2023-07-21T18:34:08.367 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:01.185+0000 7ff31f205700 1 -- 172.21.15.39:0/614615285 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff31819c050 con 0x7ff318103e30 2023-07-21T18:34:08.367 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:01.186+0000 7ff315ffb700 1 -- 172.21.15.39:0/614615285 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7ff30c019900 con 0x7ff318103e30 2023-07-21T18:34:08.367 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:01.186+0000 7ff315ffb700 1 --2- 172.21.15.39:0/614615285 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7ff308038760 0x7ff30803ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:34:08.367 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:01.186+0000 7ff315ffb700 1 -- 172.21.15.39:0/614615285 --> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff30803b320 con 0x7ff308038760 2023-07-21T18:34:08.368 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:01.186+0000 7ff315ffb700 1 -- 172.21.15.39:0/614615285 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ff30c04c860 con 0x7ff318103e30 2023-07-21T18:34:08.368 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:01.186+0000 7ff317fff700 1 -- 172.21.15.39:0/614615285 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7ff308038760 msgr2=0x7ff30803ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/2893866425 2023-07-21T18:34:08.368 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:01.186+0000 7ff317fff700 1 --2- 172.21.15.39:0/614615285 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7ff308038760 0x7ff30803ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-21T18:34:08.368 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:01.387+0000 7ff317fff700 1 -- 172.21.15.39:0/614615285 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7ff308038760 msgr2=0x7ff30803ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/2893866425 2023-07-21T18:34:08.368 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:01.387+0000 7ff317fff700 1 --2- 172.21.15.39:0/614615285 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7ff308038760 0x7ff30803ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-21T18:34:08.368 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:01.788+0000 7ff317fff700 1 -- 172.21.15.39:0/614615285 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7ff308038760 msgr2=0x7ff30803ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/2893866425 2023-07-21T18:34:08.368 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:01.788+0000 7ff317fff700 1 --2- 172.21.15.39:0/614615285 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7ff308038760 0x7ff30803ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-07-21T18:34:08.369 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:02.589+0000 7ff317fff700 1 -- 172.21.15.39:0/614615285 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7ff308038760 msgr2=0x7ff30803ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/2893866425 2023-07-21T18:34:08.369 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:02.589+0000 7ff317fff700 1 --2- 172.21.15.39:0/614615285 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7ff308038760 0x7ff30803ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-07-21T18:34:08.369 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:04.190+0000 7ff317fff700 1 -- 172.21.15.39:0/614615285 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7ff308038760 msgr2=0x7ff30803ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/2893866425 2023-07-21T18:34:08.369 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:04.190+0000 7ff317fff700 1 --2- 172.21.15.39:0/614615285 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7ff308038760 0x7ff30803ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-07-21T18:34:08.369 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:07.046+0000 7ff315ffb700 1 -- 172.21.15.39:0/614615285 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mgrmap(e 10) v1 ==== 45263+0+0 (secure 0 0 0) 0x7ff30c02d960 con 0x7ff318103e30 2023-07-21T18:34:08.369 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:07.046+0000 7ff315ffb700 1 -- 172.21.15.39:0/614615285 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7ff308038760 msgr2=0x7ff30803ac10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-21T18:34:08.369 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:07.046+0000 7ff315ffb700 1 --2- 172.21.15.39:0/614615285 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7ff308038760 0x7ff30803ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:34:08.369 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:08.043+0000 7ff315ffb700 1 -- 172.21.15.39:0/614615285 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mgrmap(e 11) v1 ==== 45390+0+0 (secure 0 0 0) 0x7ff30c04e160 con 0x7ff318103e30 2023-07-21T18:34:08.370 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:08.043+0000 7ff315ffb700 1 --2- 172.21.15.39:0/614615285 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7ff30803bff0 0x7ff30803e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:34:08.370 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:08.043+0000 7ff315ffb700 1 -- 172.21.15.39:0/614615285 --> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff30803b320 con 0x7ff30803bff0 2023-07-21T18:34:08.370 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:08.044+0000 7ff317fff700 1 --2- 172.21.15.39:0/614615285 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7ff30803bff0 0x7ff30803e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:34:08.370 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:08.044+0000 7ff317fff700 1 --2- 172.21.15.39:0/614615285 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7ff30803bff0 0x7ff30803e3d0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7ff304003a10 tx=0x7ff304009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:34:08.370 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:08.045+0000 7ff315ffb700 1 -- 172.21.15.39:0/614615285 <== mgr.14152 v2:172.21.15.39:6800/2116291712 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7ff30803b320 con 0x7ff30803bff0 2023-07-21T18:34:08.370 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:08.055+0000 7ff31f205700 1 -- 172.21.15.39:0/614615285 --> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7ff31819c770 con 0x7ff30803bff0 2023-07-21T18:34:08.370 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:08.056+0000 7ff315ffb700 1 -- 172.21.15.39:0/614615285 <== mgr.14152 v2:172.21.15.39:6800/2116291712 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7ff31819c770 con 0x7ff30803bff0 2023-07-21T18:34:08.371 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:08.056+0000 7ff31f205700 1 -- 172.21.15.39:0/614615285 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7ff30803bff0 msgr2=0x7ff30803e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:34:08.371 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:08.056+0000 7ff31f205700 1 --2- 172.21.15.39:0/614615285 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7ff30803bff0 0x7ff30803e3d0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7ff304003a10 tx=0x7ff304009270 comp rx=0 tx=0).stop 2023-07-21T18:34:08.371 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:08.056+0000 7ff31f205700 1 -- 172.21.15.39:0/614615285 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff318103e30 msgr2=0x7ff31819b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:34:08.371 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:08.056+0000 7ff31f205700 1 --2- 172.21.15.39:0/614615285 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff318103e30 0x7ff31819b430 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff30c006af0 tx=0x7ff30c011a10 comp rx=0 tx=0).stop 2023-07-21T18:34:08.371 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:08.056+0000 7ff31f205700 1 -- 172.21.15.39:0/614615285 shutdown_connections 2023-07-21T18:34:08.371 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:08.056+0000 7ff31f205700 1 --2- 172.21.15.39:0/614615285 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7ff30803bff0 0x7ff30803e3d0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:34:08.371 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:08.056+0000 7ff31f205700 1 --2- 172.21.15.39:0/614615285 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff318103e30 0x7ff31819b430 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:34:08.371 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:08.056+0000 7ff31f205700 1 --2- 172.21.15.39:0/614615285 >> [v2:172.21.15.39:6800/2893866425,v1:172.21.15.39:6801/2893866425] conn(0x7ff308038760 0x7ff30803ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:34:08.372 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:08.056+0000 7ff31f205700 1 -- 172.21.15.39:0/614615285 >> 172.21.15.39:0/614615285 conn(0x7ff3180ffa40 msgr2=0x7ff31818f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:34:08.372 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:08.056+0000 7ff31f205700 1 -- 172.21.15.39:0/614615285 shutdown_connections 2023-07-21T18:34:08.372 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:08.056+0000 7ff31f205700 1 -- 172.21.15.39:0/614615285 wait complete. 2023-07-21T18:34:08.372 INFO:teuthology.orchestra.run.smithi039.stdout:mgr epoch 9 is available 2023-07-21T18:34:08.372 INFO:teuthology.orchestra.run.smithi039.stdout:Generating a dashboard self-signed certificate... 2023-07-21T18:34:08.373 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:08 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:34:08.373 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:08 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-21T18:34:08.373 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:08 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-21T18:34:08.373 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:08 smithi039 ceph-mon[96489]: [21/Jul/2023:18:34:07] ENGINE Bus STARTING 2023-07-21T18:34:08.374 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:08 smithi039 ceph-mon[96489]: [21/Jul/2023:18:34:07] ENGINE Serving on http://172.21.15.39:8765 2023-07-21T18:34:08.374 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:08 smithi039 ceph-mon[96489]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-07-21T18:34:08.374 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:08 smithi039 ceph-mon[96489]: mgrmap e11: x(active, since 1.00617s) 2023-07-21T18:34:08.374 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:08 smithi039 ceph-mon[96489]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-07-21T18:34:08.374 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:08 smithi039 ceph-mon[96489]: [21/Jul/2023:18:34:08] ENGINE Serving on https://172.21.15.39:7150 2023-07-21T18:34:08.374 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:08 smithi039 ceph-mon[96489]: [21/Jul/2023:18:34:08] ENGINE Bus STARTED 2023-07-21T18:34:09.908 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2023-07-21T18:34:09.909 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:08.936+0000 7fa15ff09700 1 Processor -- start 2023-07-21T18:34:09.909 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:08.936+0000 7fa15ff09700 1 -- start start 2023-07-21T18:34:09.909 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:08.937+0000 7fa15ff09700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa158073ba0 0x7fa15810a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:34:09.909 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:08.937+0000 7fa15ff09700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa158105e00 con 0x7fa158073ba0 2023-07-21T18:34:09.909 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:08.937+0000 7fa15dca5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa158073ba0 0x7fa15810a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:34:09.909 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:08.937+0000 7fa15dca5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa158073ba0 0x7fa15810a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53990/0 (socket says 172.21.15.39:53990) 2023-07-21T18:34:09.910 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:08.937+0000 7fa15dca5700 1 -- 172.21.15.39:0/2560096018 learned_addr learned my addr 172.21.15.39:0/2560096018 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:34:09.910 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:08.937+0000 7fa15dca5700 1 -- 172.21.15.39:0/2560096018 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa15810a950 con 0x7fa158073ba0 2023-07-21T18:34:09.910 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:08.937+0000 7fa15dca5700 1 --2- 172.21.15.39:0/2560096018 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa158073ba0 0x7fa15810a410 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa1480098a0 tx=0x7fa148009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34624c6ff1882168 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:34:09.910 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:08.937+0000 7fa15cca3700 1 -- 172.21.15.39:0/2560096018 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa148004160 con 0x7fa158073ba0 2023-07-21T18:34:09.910 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:08.937+0000 7fa15cca3700 1 -- 172.21.15.39:0/2560096018 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa1480042c0 con 0x7fa158073ba0 2023-07-21T18:34:09.910 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:08.937+0000 7fa15cca3700 1 -- 172.21.15.39:0/2560096018 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa1480119b0 con 0x7fa158073ba0 2023-07-21T18:34:09.910 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:08.938+0000 7fa15ff09700 1 -- 172.21.15.39:0/2560096018 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa158073ba0 msgr2=0x7fa15810a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:34:09.910 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:08.938+0000 7fa15ff09700 1 --2- 172.21.15.39:0/2560096018 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa158073ba0 0x7fa15810a410 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa1480098a0 tx=0x7fa148009bb0 comp rx=0 tx=0).stop 2023-07-21T18:34:09.911 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:08.938+0000 7fa15ff09700 1 -- 172.21.15.39:0/2560096018 shutdown_connections 2023-07-21T18:34:09.911 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:08.938+0000 7fa15ff09700 1 --2- 172.21.15.39:0/2560096018 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa158073ba0 0x7fa15810a410 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:34:09.911 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:08.938+0000 7fa15ff09700 1 -- 172.21.15.39:0/2560096018 >> 172.21.15.39:0/2560096018 conn(0x7fa1580ff9e0 msgr2=0x7fa158101df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:34:09.911 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:08.938+0000 7fa15ff09700 1 -- 172.21.15.39:0/2560096018 shutdown_connections 2023-07-21T18:34:09.911 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:08.938+0000 7fa15ff09700 1 -- 172.21.15.39:0/2560096018 wait complete. 2023-07-21T18:34:09.911 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:08.939+0000 7fa15ff09700 1 Processor -- start 2023-07-21T18:34:09.911 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:08.939+0000 7fa15ff09700 1 -- start start 2023-07-21T18:34:09.911 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:08.939+0000 7fa15ff09700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa15819b380 0x7fa15819b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:34:09.912 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:08.939+0000 7fa15ff09700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa15819bcd0 con 0x7fa15819b380 2023-07-21T18:34:09.912 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:08.939+0000 7fa15dca5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa15819b380 0x7fa15819b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:34:09.912 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:08.939+0000 7fa15dca5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa15819b380 0x7fa15819b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53992/0 (socket says 172.21.15.39:53992) 2023-07-21T18:34:09.912 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:08.939+0000 7fa15dca5700 1 -- 172.21.15.39:0/2618755469 learned_addr learned my addr 172.21.15.39:0/2618755469 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:34:09.912 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:08.939+0000 7fa15dca5700 1 -- 172.21.15.39:0/2618755469 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa148009550 con 0x7fa15819b380 2023-07-21T18:34:09.912 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:08.940+0000 7fa15dca5700 1 --2- 172.21.15.39:0/2618755469 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa15819b380 0x7fa15819b790 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa148006af0 tx=0x7fa148011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:34:09.912 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:08.940+0000 7fa14effd700 1 -- 172.21.15.39:0/2618755469 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa148003a40 con 0x7fa15819b380 2023-07-21T18:34:09.913 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:08.940+0000 7fa14effd700 1 -- 172.21.15.39:0/2618755469 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa148003ba0 con 0x7fa15819b380 2023-07-21T18:34:09.913 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:08.940+0000 7fa15ff09700 1 -- 172.21.15.39:0/2618755469 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa15819bed0 con 0x7fa15819b380 2023-07-21T18:34:09.913 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:08.940+0000 7fa14effd700 1 -- 172.21.15.39:0/2618755469 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa148019690 con 0x7fa15819b380 2023-07-21T18:34:09.913 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:08.940+0000 7fa15ff09700 1 -- 172.21.15.39:0/2618755469 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa15819eb30 con 0x7fa15819b380 2023-07-21T18:34:09.913 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:08.941+0000 7fa14effd700 1 -- 172.21.15.39:0/2618755469 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 11) v1 ==== 45390+0+0 (secure 0 0 0) 0x7fa14801a430 con 0x7fa15819b380 2023-07-21T18:34:09.913 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:08.941+0000 7fa14effd700 1 --2- 172.21.15.39:0/2618755469 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fa144038640 0x7fa14403aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:34:09.913 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:08.941+0000 7fa15ff09700 1 -- 172.21.15.39:0/2618755469 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa158073ba0 con 0x7fa15819b380 2023-07-21T18:34:09.914 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:08.941+0000 7fa14effd700 1 -- 172.21.15.39:0/2618755469 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa14804c980 con 0x7fa15819b380 2023-07-21T18:34:09.914 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:08.941+0000 7fa15d4a4700 1 --2- 172.21.15.39:0/2618755469 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fa144038640 0x7fa14403aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:34:09.914 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:08.942+0000 7fa15d4a4700 1 --2- 172.21.15.39:0/2618755469 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fa144038640 0x7fa14403aaf0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa154006fd0 tx=0x7fa154006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:34:09.914 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:08.946+0000 7fa14effd700 1 -- 172.21.15.39:0/2618755469 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fa14802d430 con 0x7fa15819b380 2023-07-21T18:34:09.914 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:09.329+0000 7fa15ff09700 1 -- 172.21.15.39:0/2618755469 --> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fa158101c80 con 0x7fa144038640 2023-07-21T18:34:09.914 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:09.522+0000 7fa14effd700 1 -- 172.21.15.39:0/2618755469 <== mgr.14152 v2:172.21.15.39:6800/2116291712 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fa158101c80 con 0x7fa144038640 2023-07-21T18:34:09.914 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:09.524+0000 7fa15ff09700 1 -- 172.21.15.39:0/2618755469 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fa144038640 msgr2=0x7fa14403aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:34:09.914 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:09.524+0000 7fa15ff09700 1 --2- 172.21.15.39:0/2618755469 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fa144038640 0x7fa14403aaf0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa154006fd0 tx=0x7fa154006e40 comp rx=0 tx=0).stop 2023-07-21T18:34:09.915 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:09.524+0000 7fa15ff09700 1 -- 172.21.15.39:0/2618755469 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa15819b380 msgr2=0x7fa15819b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:34:09.915 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:09.524+0000 7fa15ff09700 1 --2- 172.21.15.39:0/2618755469 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa15819b380 0x7fa15819b790 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa148006af0 tx=0x7fa148011a10 comp rx=0 tx=0).stop 2023-07-21T18:34:09.915 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:09.524+0000 7fa15ff09700 1 -- 172.21.15.39:0/2618755469 shutdown_connections 2023-07-21T18:34:09.915 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:09.524+0000 7fa15ff09700 1 --2- 172.21.15.39:0/2618755469 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fa144038640 0x7fa14403aaf0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:34:09.915 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:09.524+0000 7fa15ff09700 1 --2- 172.21.15.39:0/2618755469 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa15819b380 0x7fa15819b790 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:34:09.915 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:09.524+0000 7fa15ff09700 1 -- 172.21.15.39:0/2618755469 >> 172.21.15.39:0/2618755469 conn(0x7fa1580ff9e0 msgr2=0x7fa158101020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:34:09.915 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:09.524+0000 7fa15ff09700 1 -- 172.21.15.39:0/2618755469 shutdown_connections 2023-07-21T18:34:09.916 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:09.524+0000 7fa15ff09700 1 -- 172.21.15.39:0/2618755469 wait complete. 2023-07-21T18:34:09.916 INFO:teuthology.orchestra.run.smithi039.stdout:Creating initial admin user... 2023-07-21T18:34:10.152 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:10 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:10.153 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:10 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:10.153 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:10 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:10.153 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:10 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:11.369 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:11 smithi039 ceph-mon[96489]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:34:11.370 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:11 smithi039 ceph-mon[96489]: mgrmap e12: x(active, since 3s) 2023-07-21T18:34:11.370 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:11 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:11.555 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$QM2sRta/5NKtgiHnOC6yo.LNpFsnmxJA.vqAXQyOfUmvrjUn8dVN2", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1689964451, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2023-07-21T18:34:11.556 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:10.372+0000 7f15321b7700 1 Processor -- start 2023-07-21T18:34:11.556 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:10.372+0000 7f15321b7700 1 -- start start 2023-07-21T18:34:11.556 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:10.372+0000 7f15321b7700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f152c073ba0 0x7f152c1080a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:34:11.556 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:10.372+0000 7f15321b7700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152c1085e0 con 0x7f152c073ba0 2023-07-21T18:34:11.556 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:10.372+0000 7f152b7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f152c073ba0 0x7f152c1080a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:34:11.556 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:10.372+0000 7f152b7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f152c073ba0 0x7f152c1080a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53996/0 (socket says 172.21.15.39:53996) 2023-07-21T18:34:11.557 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:10.372+0000 7f152b7fe700 1 -- 172.21.15.39:0/153192191 learned_addr learned my addr 172.21.15.39:0/153192191 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:34:11.557 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:10.373+0000 7f152b7fe700 1 -- 172.21.15.39:0/153192191 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f152c108720 con 0x7f152c073ba0 2023-07-21T18:34:11.557 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:10.373+0000 7f152b7fe700 1 --2- 172.21.15.39:0/153192191 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f152c073ba0 0x7f152c1080a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f151c0098a0 tx=0x7f151c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a3f97b521051e23 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:34:11.557 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:10.373+0000 7f152a7fc700 1 -- 172.21.15.39:0/153192191 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f151c004160 con 0x7f152c073ba0 2023-07-21T18:34:11.557 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:10.373+0000 7f152a7fc700 1 -- 172.21.15.39:0/153192191 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f151c0042c0 con 0x7f152c073ba0 2023-07-21T18:34:11.557 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:10.373+0000 7f152a7fc700 1 -- 172.21.15.39:0/153192191 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f151c0119b0 con 0x7f152c073ba0 2023-07-21T18:34:11.557 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:10.373+0000 7f15321b7700 1 -- 172.21.15.39:0/153192191 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f152c073ba0 msgr2=0x7f152c1080a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:34:11.558 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:10.373+0000 7f15321b7700 1 --2- 172.21.15.39:0/153192191 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f152c073ba0 0x7f152c1080a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f151c0098a0 tx=0x7f151c009bb0 comp rx=0 tx=0).stop 2023-07-21T18:34:11.558 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:10.373+0000 7f15321b7700 1 -- 172.21.15.39:0/153192191 shutdown_connections 2023-07-21T18:34:11.558 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:10.373+0000 7f15321b7700 1 --2- 172.21.15.39:0/153192191 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f152c073ba0 0x7f152c1080a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:34:11.558 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:10.373+0000 7f15321b7700 1 -- 172.21.15.39:0/153192191 >> 172.21.15.39:0/153192191 conn(0x7f152c0ffa50 msgr2=0x7f152c101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:34:11.558 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:10.374+0000 7f15321b7700 1 -- 172.21.15.39:0/153192191 shutdown_connections 2023-07-21T18:34:11.558 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:10.374+0000 7f15321b7700 1 -- 172.21.15.39:0/153192191 wait complete. 2023-07-21T18:34:11.558 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:10.374+0000 7f15321b7700 1 Processor -- start 2023-07-21T18:34:11.559 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:10.375+0000 7f15321b7700 1 -- start start 2023-07-21T18:34:11.559 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:10.375+0000 7f15321b7700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f152c1059b0 0x7f152c104040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:34:11.559 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:10.375+0000 7f15321b7700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152c105dc0 con 0x7f152c1059b0 2023-07-21T18:34:11.559 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:10.375+0000 7f152b7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f152c1059b0 0x7f152c104040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:34:11.559 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:10.375+0000 7f152b7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f152c1059b0 0x7f152c104040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53998/0 (socket says 172.21.15.39:53998) 2023-07-21T18:34:11.559 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:10.375+0000 7f152b7fe700 1 -- 172.21.15.39:0/2849941507 learned_addr learned my addr 172.21.15.39:0/2849941507 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:34:11.559 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:10.375+0000 7f152b7fe700 1 -- 172.21.15.39:0/2849941507 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f151c009550 con 0x7f152c1059b0 2023-07-21T18:34:11.560 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:10.375+0000 7f152b7fe700 1 --2- 172.21.15.39:0/2849941507 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f152c1059b0 0x7f152c104040 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f151c003e60 tx=0x7f151c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:34:11.560 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:10.375+0000 7f1528ff9700 1 -- 172.21.15.39:0/2849941507 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f151c003a60 con 0x7f152c1059b0 2023-07-21T18:34:11.560 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:10.376+0000 7f1528ff9700 1 -- 172.21.15.39:0/2849941507 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f151c003bc0 con 0x7f152c1059b0 2023-07-21T18:34:11.560 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:10.376+0000 7f1528ff9700 1 -- 172.21.15.39:0/2849941507 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f151c0196c0 con 0x7f152c1059b0 2023-07-21T18:34:11.560 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:10.376+0000 7f15321b7700 1 -- 172.21.15.39:0/2849941507 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f152c104580 con 0x7f152c1059b0 2023-07-21T18:34:11.560 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:10.376+0000 7f15321b7700 1 -- 172.21.15.39:0/2849941507 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f152c1049a0 con 0x7f152c1059b0 2023-07-21T18:34:11.560 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:10.376+0000 7f1528ff9700 1 -- 172.21.15.39:0/2849941507 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f151c01ab60 con 0x7f152c1059b0 2023-07-21T18:34:11.561 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:10.377+0000 7f1528ff9700 1 --2- 172.21.15.39:0/2849941507 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f1514038790 0x7f151403ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:34:11.561 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:10.377+0000 7f1528ff9700 1 -- 172.21.15.39:0/2849941507 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f151c04ca20 con 0x7f152c1059b0 2023-07-21T18:34:11.561 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:10.377+0000 7f152affd700 1 --2- 172.21.15.39:0/2849941507 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f1514038790 0x7f151403ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:34:11.561 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:10.377+0000 7f15321b7700 1 -- 172.21.15.39:0/2849941507 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f152c061b40 con 0x7f152c1059b0 2023-07-21T18:34:11.561 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:10.377+0000 7f152affd700 1 --2- 172.21.15.39:0/2849941507 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f1514038790 0x7f151403ac40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1520006fd0 tx=0x7f1520006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:34:11.561 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:10.382+0000 7f1528ff9700 1 -- 172.21.15.39:0/2849941507 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f151c01a430 con 0x7f152c1059b0 2023-07-21T18:34:11.561 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:10.781+0000 7f15321b7700 1 -- 172.21.15.39:0/2849941507 --> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f152c061960 con 0x7f1514038790 2023-07-21T18:34:11.562 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:11.034+0000 7f1528ff9700 1 -- 172.21.15.39:0/2849941507 <== mgr.14152 v2:172.21.15.39:6800/2116291712 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f152c061960 con 0x7f1514038790 2023-07-21T18:34:11.562 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:11.037+0000 7f15321b7700 1 -- 172.21.15.39:0/2849941507 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f1514038790 msgr2=0x7f151403ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:34:11.562 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:11.037+0000 7f15321b7700 1 --2- 172.21.15.39:0/2849941507 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f1514038790 0x7f151403ac40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1520006fd0 tx=0x7f1520006e40 comp rx=0 tx=0).stop 2023-07-21T18:34:11.562 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:11.037+0000 7f15321b7700 1 -- 172.21.15.39:0/2849941507 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f152c1059b0 msgr2=0x7f152c104040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:34:11.562 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:11.037+0000 7f15321b7700 1 --2- 172.21.15.39:0/2849941507 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f152c1059b0 0x7f152c104040 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f151c003e60 tx=0x7f151c011a30 comp rx=0 tx=0).stop 2023-07-21T18:34:11.562 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:11.037+0000 7f15321b7700 1 -- 172.21.15.39:0/2849941507 shutdown_connections 2023-07-21T18:34:11.562 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:11.037+0000 7f15321b7700 1 --2- 172.21.15.39:0/2849941507 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f1514038790 0x7f151403ac40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:34:11.563 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:11.037+0000 7f15321b7700 1 --2- 172.21.15.39:0/2849941507 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f152c1059b0 0x7f152c104040 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:34:11.563 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:11.037+0000 7f15321b7700 1 -- 172.21.15.39:0/2849941507 >> 172.21.15.39:0/2849941507 conn(0x7f152c0ffa50 msgr2=0x7f152c101130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:34:11.563 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:11.037+0000 7f15321b7700 1 -- 172.21.15.39:0/2849941507 shutdown_connections 2023-07-21T18:34:11.563 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:11.037+0000 7f15321b7700 1 -- 172.21.15.39:0/2849941507 wait complete. 2023-07-21T18:34:11.563 INFO:teuthology.orchestra.run.smithi039.stdout:Fetching dashboard port number... 2023-07-21T18:34:12.392 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:12 smithi039 ceph-mon[96489]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:34:12.704 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 8443 2023-07-21T18:34:12.704 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:11.960+0000 7f024bfc2700 1 Processor -- start 2023-07-21T18:34:12.704 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:11.960+0000 7f024bfc2700 1 -- start start 2023-07-21T18:34:12.704 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:11.961+0000 7f024bfc2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0244103b70 0x7f0244103f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:34:12.704 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:11.961+0000 7f024bfc2700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02441044c0 con 0x7f0244103b70 2023-07-21T18:34:12.705 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:11.961+0000 7f0249d5e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0244103b70 0x7f0244103f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:34:12.705 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:11.961+0000 7f0249d5e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0244103b70 0x7f0244103f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54002/0 (socket says 172.21.15.39:54002) 2023-07-21T18:34:12.705 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:11.961+0000 7f0249d5e700 1 -- 172.21.15.39:0/2097877633 learned_addr learned my addr 172.21.15.39:0/2097877633 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:34:12.705 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:11.961+0000 7f0249d5e700 1 -- 172.21.15.39:0/2097877633 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0244104600 con 0x7f0244103b70 2023-07-21T18:34:12.705 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:11.961+0000 7f0249d5e700 1 --2- 172.21.15.39:0/2097877633 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0244103b70 0x7f0244103f80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f02400098a0 tx=0x7f0240009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f7ce8c5053ac8bb server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:34:12.705 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:11.961+0000 7f0248d5c700 1 -- 172.21.15.39:0/2097877633 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0240004160 con 0x7f0244103b70 2023-07-21T18:34:12.705 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:11.962+0000 7f0248d5c700 1 -- 172.21.15.39:0/2097877633 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f02400042c0 con 0x7f0244103b70 2023-07-21T18:34:12.706 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:11.962+0000 7f0248d5c700 1 -- 172.21.15.39:0/2097877633 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f02400119b0 con 0x7f0244103b70 2023-07-21T18:34:12.706 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:11.962+0000 7f024bfc2700 1 -- 172.21.15.39:0/2097877633 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0244103b70 msgr2=0x7f0244103f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:34:12.706 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:11.962+0000 7f024bfc2700 1 --2- 172.21.15.39:0/2097877633 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0244103b70 0x7f0244103f80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f02400098a0 tx=0x7f0240009bb0 comp rx=0 tx=0).stop 2023-07-21T18:34:12.706 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:11.962+0000 7f024bfc2700 1 -- 172.21.15.39:0/2097877633 shutdown_connections 2023-07-21T18:34:12.706 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:11.962+0000 7f024bfc2700 1 --2- 172.21.15.39:0/2097877633 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0244103b70 0x7f0244103f80 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:34:12.706 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:11.962+0000 7f024bfc2700 1 -- 172.21.15.39:0/2097877633 >> 172.21.15.39:0/2097877633 conn(0x7f02440ff1c0 msgr2=0x7f02441015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:34:12.706 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:11.962+0000 7f024bfc2700 1 -- 172.21.15.39:0/2097877633 shutdown_connections 2023-07-21T18:34:12.706 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:11.962+0000 7f024bfc2700 1 -- 172.21.15.39:0/2097877633 wait complete. 2023-07-21T18:34:12.707 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:11.963+0000 7f024bfc2700 1 Processor -- start 2023-07-21T18:34:12.707 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:11.963+0000 7f024bfc2700 1 -- start start 2023-07-21T18:34:12.707 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:11.963+0000 7f024bfc2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0244103b70 0x7f0244199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:34:12.707 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:11.963+0000 7f024bfc2700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02441997d0 con 0x7f0244103b70 2023-07-21T18:34:12.707 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:11.963+0000 7f0249d5e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0244103b70 0x7f0244199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:34:12.707 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:11.963+0000 7f0249d5e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0244103b70 0x7f0244199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54004/0 (socket says 172.21.15.39:54004) 2023-07-21T18:34:12.707 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:11.963+0000 7f0249d5e700 1 -- 172.21.15.39:0/1067428496 learned_addr learned my addr 172.21.15.39:0/1067428496 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:34:12.708 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:11.963+0000 7f0249d5e700 1 -- 172.21.15.39:0/1067428496 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0240009550 con 0x7f0244103b70 2023-07-21T18:34:12.708 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:11.964+0000 7f0249d5e700 1 --2- 172.21.15.39:0/1067428496 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0244103b70 0x7f0244199290 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0240006af0 tx=0x7f0240011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:34:12.708 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:11.964+0000 7f023affd700 1 -- 172.21.15.39:0/1067428496 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0240003a40 con 0x7f0244103b70 2023-07-21T18:34:12.708 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:11.964+0000 7f024bfc2700 1 -- 172.21.15.39:0/1067428496 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02441999d0 con 0x7f0244103b70 2023-07-21T18:34:12.708 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:11.964+0000 7f023affd700 1 -- 172.21.15.39:0/1067428496 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0240003ba0 con 0x7f0244103b70 2023-07-21T18:34:12.708 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:11.964+0000 7f023affd700 1 -- 172.21.15.39:0/1067428496 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f02400196c0 con 0x7f0244103b70 2023-07-21T18:34:12.708 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:11.964+0000 7f024bfc2700 1 -- 172.21.15.39:0/1067428496 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0244199df0 con 0x7f0244103b70 2023-07-21T18:34:12.709 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:11.965+0000 7f023affd700 1 -- 172.21.15.39:0/1067428496 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f0240019980 con 0x7f0244103b70 2023-07-21T18:34:12.709 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:11.965+0000 7f023affd700 1 --2- 172.21.15.39:0/1067428496 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f0230038700 0x7f023003abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:34:12.709 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:11.965+0000 7f023affd700 1 -- 172.21.15.39:0/1067428496 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f024004cc30 con 0x7f0244103b70 2023-07-21T18:34:12.709 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:11.965+0000 7f024bfc2700 1 -- 172.21.15.39:0/1067428496 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0244192a60 con 0x7f0244103b70 2023-07-21T18:34:12.709 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:11.965+0000 7f024955d700 1 --2- 172.21.15.39:0/1067428496 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f0230038700 0x7f023003abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:34:12.709 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:11.965+0000 7f024955d700 1 --2- 172.21.15.39:0/1067428496 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f0230038700 0x7f023003abb0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0234006fd0 tx=0x7f0234006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:34:12.710 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:11.970+0000 7f023affd700 1 -- 172.21.15.39:0/1067428496 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f024002c3a0 con 0x7f0244103b70 2023-07-21T18:34:12.710 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:12.325+0000 7f024bfc2700 1 -- 172.21.15.39:0/1067428496 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f024402c850 con 0x7f0244103b70 2023-07-21T18:34:12.710 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:12.326+0000 7f023affd700 1 -- 172.21.15.39:0/1067428496 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f0240018350 con 0x7f0244103b70 2023-07-21T18:34:12.710 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:12.328+0000 7f024bfc2700 1 -- 172.21.15.39:0/1067428496 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f0230038700 msgr2=0x7f023003abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:34:12.710 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:12.328+0000 7f024bfc2700 1 --2- 172.21.15.39:0/1067428496 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f0230038700 0x7f023003abb0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0234006fd0 tx=0x7f0234006e40 comp rx=0 tx=0).stop 2023-07-21T18:34:12.710 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:12.328+0000 7f024bfc2700 1 -- 172.21.15.39:0/1067428496 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0244103b70 msgr2=0x7f0244199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:34:12.710 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:12.328+0000 7f024bfc2700 1 --2- 172.21.15.39:0/1067428496 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0244103b70 0x7f0244199290 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0240006af0 tx=0x7f0240011a10 comp rx=0 tx=0).stop 2023-07-21T18:34:12.710 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:12.328+0000 7f024bfc2700 1 -- 172.21.15.39:0/1067428496 shutdown_connections 2023-07-21T18:34:12.711 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:12.328+0000 7f024bfc2700 1 --2- 172.21.15.39:0/1067428496 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f0230038700 0x7f023003abb0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:34:12.711 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:12.328+0000 7f024bfc2700 1 --2- 172.21.15.39:0/1067428496 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0244103b70 0x7f0244199290 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:34:12.711 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:12.328+0000 7f024bfc2700 1 -- 172.21.15.39:0/1067428496 >> 172.21.15.39:0/1067428496 conn(0x7f02440ff1c0 msgr2=0x7f024418d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:34:12.711 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:12.328+0000 7f024bfc2700 1 -- 172.21.15.39:0/1067428496 shutdown_connections 2023-07-21T18:34:12.711 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:12.328+0000 7f024bfc2700 1 -- 172.21.15.39:0/1067428496 wait complete. 2023-07-21T18:34:12.713 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-07-21T18:34:12.714 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout disabled 2023-07-21T18:34:12.721 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-07-21T18:34:12.721 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout inactive 2023-07-21T18:34:12.721 INFO:teuthology.orchestra.run.smithi039.stdout:firewalld.service is not enabled 2023-07-21T18:34:12.721 INFO:teuthology.orchestra.run.smithi039.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2023-07-21T18:34:12.723 INFO:teuthology.orchestra.run.smithi039.stdout:Ceph Dashboard is now available at: 2023-07-21T18:34:12.723 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:34:12.723 INFO:teuthology.orchestra.run.smithi039.stdout: URL: https://smithi039:8443/ 2023-07-21T18:34:12.723 INFO:teuthology.orchestra.run.smithi039.stdout: User: admin 2023-07-21T18:34:12.723 INFO:teuthology.orchestra.run.smithi039.stdout: Password: xbv8x4hu07 2023-07-21T18:34:12.723 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:34:12.724 INFO:teuthology.orchestra.run.smithi039.stdout:Saving cluster configuration to /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/config directory 2023-07-21T18:34:12.724 INFO:teuthology.orchestra.run.smithi039.stdout:Enabling autotune for osd_memory_target 2023-07-21T18:34:13.450 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:13 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/1067428496' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2023-07-21T18:34:13.869 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:13.164+0000 7fd2f3b98700 1 Processor -- start 2023-07-21T18:34:13.869 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:13.165+0000 7fd2f3b98700 1 -- start start 2023-07-21T18:34:13.869 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:13.165+0000 7fd2f3b98700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd2ec106af0 0x7fd2ec106f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:34:13.869 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:13.165+0000 7fd2f3b98700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ec107440 con 0x7fd2ec106af0 2023-07-21T18:34:13.869 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:13.165+0000 7fd2f1934700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd2ec106af0 0x7fd2ec106f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:34:13.870 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:13.165+0000 7fd2f1934700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd2ec106af0 0x7fd2ec106f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54012/0 (socket says 172.21.15.39:54012) 2023-07-21T18:34:13.870 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:13.165+0000 7fd2f1934700 1 -- 172.21.15.39:0/2705323345 learned_addr learned my addr 172.21.15.39:0/2705323345 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:34:13.870 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:13.165+0000 7fd2f1934700 1 -- 172.21.15.39:0/2705323345 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2ec107580 con 0x7fd2ec106af0 2023-07-21T18:34:13.870 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:13.165+0000 7fd2f1934700 1 --2- 172.21.15.39:0/2705323345 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd2ec106af0 0x7fd2ec106f00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd2e80098a0 tx=0x7fd2e8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6eb3c9bc0072882 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:34:13.870 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:13.166+0000 7fd2f0932700 1 -- 172.21.15.39:0/2705323345 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd2e8004160 con 0x7fd2ec106af0 2023-07-21T18:34:13.870 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:13.166+0000 7fd2f0932700 1 -- 172.21.15.39:0/2705323345 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2e80042c0 con 0x7fd2ec106af0 2023-07-21T18:34:13.870 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:13.166+0000 7fd2f0932700 1 -- 172.21.15.39:0/2705323345 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd2e80119b0 con 0x7fd2ec106af0 2023-07-21T18:34:13.871 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:13.166+0000 7fd2f3b98700 1 -- 172.21.15.39:0/2705323345 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd2ec106af0 msgr2=0x7fd2ec106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:34:13.871 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:13.166+0000 7fd2f3b98700 1 --2- 172.21.15.39:0/2705323345 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd2ec106af0 0x7fd2ec106f00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd2e80098a0 tx=0x7fd2e8009bb0 comp rx=0 tx=0).stop 2023-07-21T18:34:13.871 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:13.166+0000 7fd2f3b98700 1 -- 172.21.15.39:0/2705323345 shutdown_connections 2023-07-21T18:34:13.871 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:13.166+0000 7fd2f3b98700 1 --2- 172.21.15.39:0/2705323345 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd2ec106af0 0x7fd2ec106f00 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:34:13.871 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:13.166+0000 7fd2f3b98700 1 -- 172.21.15.39:0/2705323345 >> 172.21.15.39:0/2705323345 conn(0x7fd2ec0ff1c0 msgr2=0x7fd2ec1015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:34:13.871 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:13.166+0000 7fd2f3b98700 1 -- 172.21.15.39:0/2705323345 shutdown_connections 2023-07-21T18:34:13.871 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:13.166+0000 7fd2f3b98700 1 -- 172.21.15.39:0/2705323345 wait complete. 2023-07-21T18:34:13.871 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:13.167+0000 7fd2f3b98700 1 Processor -- start 2023-07-21T18:34:13.872 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:13.167+0000 7fd2f3b98700 1 -- start start 2023-07-21T18:34:13.872 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:13.167+0000 7fd2f3b98700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd2ec106af0 0x7fd2ec1061f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:34:13.872 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:13.167+0000 7fd2f3b98700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ec106730 con 0x7fd2ec106af0 2023-07-21T18:34:13.872 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:13.167+0000 7fd2f1934700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd2ec106af0 0x7fd2ec1061f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:34:13.872 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:13.167+0000 7fd2f1934700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd2ec106af0 0x7fd2ec1061f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54014/0 (socket says 172.21.15.39:54014) 2023-07-21T18:34:13.872 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:13.167+0000 7fd2f1934700 1 -- 172.21.15.39:0/920865669 learned_addr learned my addr 172.21.15.39:0/920865669 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:34:13.872 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:13.168+0000 7fd2f1934700 1 -- 172.21.15.39:0/920865669 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2e8009550 con 0x7fd2ec106af0 2023-07-21T18:34:13.873 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:13.168+0000 7fd2f1934700 1 --2- 172.21.15.39:0/920865669 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd2ec106af0 0x7fd2ec1061f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8006af0 tx=0x7fd2e8011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:34:13.873 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:13.168+0000 7fd2deffd700 1 -- 172.21.15.39:0/920865669 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd2e8003a40 con 0x7fd2ec106af0 2023-07-21T18:34:13.873 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:13.168+0000 7fd2deffd700 1 -- 172.21.15.39:0/920865669 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2e8003ba0 con 0x7fd2ec106af0 2023-07-21T18:34:13.873 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:13.168+0000 7fd2f3b98700 1 -- 172.21.15.39:0/920865669 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2ec104840 con 0x7fd2ec106af0 2023-07-21T18:34:13.873 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:13.168+0000 7fd2deffd700 1 -- 172.21.15.39:0/920865669 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd2e8019690 con 0x7fd2ec106af0 2023-07-21T18:34:13.873 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:13.168+0000 7fd2f3b98700 1 -- 172.21.15.39:0/920865669 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2ec104c60 con 0x7fd2ec106af0 2023-07-21T18:34:13.873 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:13.169+0000 7fd2deffd700 1 -- 172.21.15.39:0/920865669 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7fd2e801aa60 con 0x7fd2ec106af0 2023-07-21T18:34:13.873 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:13.169+0000 7fd2deffd700 1 --2- 172.21.15.39:0/920865669 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fd2d8038710 0x7fd2d803abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:34:13.874 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:13.169+0000 7fd2deffd700 1 -- 172.21.15.39:0/920865669 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd2e804cc30 con 0x7fd2ec106af0 2023-07-21T18:34:13.874 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:13.169+0000 7fd2f3b98700 1 -- 172.21.15.39:0/920865669 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2ec061b40 con 0x7fd2ec106af0 2023-07-21T18:34:13.874 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:13.170+0000 7fd2f1133700 1 --2- 172.21.15.39:0/920865669 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fd2d8038710 0x7fd2d803abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:34:13.874 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:13.170+0000 7fd2f1133700 1 --2- 172.21.15.39:0/920865669 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fd2d8038710 0x7fd2d803abc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd2e0006fd0 tx=0x7fd2e0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:34:13.874 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:13.174+0000 7fd2deffd700 1 -- 172.21.15.39:0/920865669 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fd2e8023980 con 0x7fd2ec106af0 2023-07-21T18:34:13.874 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:13.529+0000 7fd2f3b98700 1 -- 172.21.15.39:0/920865669 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7fd2ec105560 con 0x7fd2ec106af0 2023-07-21T18:34:13.874 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:13.530+0000 7fd2deffd700 1 -- 172.21.15.39:0/920865669 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7fd2e8023390 con 0x7fd2ec106af0 2023-07-21T18:34:13.875 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:13.532+0000 7fd2f3b98700 1 -- 172.21.15.39:0/920865669 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fd2d8038710 msgr2=0x7fd2d803abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:34:13.875 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:13.532+0000 7fd2f3b98700 1 --2- 172.21.15.39:0/920865669 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fd2d8038710 0x7fd2d803abc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd2e0006fd0 tx=0x7fd2e0006e40 comp rx=0 tx=0).stop 2023-07-21T18:34:13.875 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:13.532+0000 7fd2f3b98700 1 -- 172.21.15.39:0/920865669 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd2ec106af0 msgr2=0x7fd2ec1061f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:34:13.875 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:13.532+0000 7fd2f3b98700 1 --2- 172.21.15.39:0/920865669 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd2ec106af0 0x7fd2ec1061f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8006af0 tx=0x7fd2e8011a10 comp rx=0 tx=0).stop 2023-07-21T18:34:13.875 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:13.532+0000 7fd2f3b98700 1 -- 172.21.15.39:0/920865669 shutdown_connections 2023-07-21T18:34:13.875 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:13.532+0000 7fd2f3b98700 1 --2- 172.21.15.39:0/920865669 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fd2d8038710 0x7fd2d803abc0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:34:13.875 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:13.532+0000 7fd2f3b98700 1 --2- 172.21.15.39:0/920865669 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd2ec106af0 0x7fd2ec1061f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:34:13.875 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:13.532+0000 7fd2f3b98700 1 -- 172.21.15.39:0/920865669 >> 172.21.15.39:0/920865669 conn(0x7fd2ec0ff1c0 msgr2=0x7fd2ec1008c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:34:13.876 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:13.533+0000 7fd2f3b98700 1 -- 172.21.15.39:0/920865669 shutdown_connections 2023-07-21T18:34:13.876 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:13.533+0000 7fd2f3b98700 1 -- 172.21.15.39:0/920865669 wait complete. 2023-07-21T18:34:15.193 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:15 smithi039 ceph-mon[96489]: mgrmap e13: x(active, since 7s) 2023-07-21T18:34:15.193 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:15 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/1222888566' entity='client.admin' 2023-07-21T18:34:15.215 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:14.281+0000 7fe7f19c9700 1 Processor -- start 2023-07-21T18:34:15.216 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:14.281+0000 7fe7f19c9700 1 -- start start 2023-07-21T18:34:15.216 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:14.281+0000 7fe7f19c9700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7ec07acc0 0x7fe7ec0791c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:34:15.216 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:14.281+0000 7fe7f19c9700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ec079700 con 0x7fe7ec07acc0 2023-07-21T18:34:15.216 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:14.281+0000 7fe7eaffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7ec07acc0 0x7fe7ec0791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:34:15.216 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:14.282+0000 7fe7eaffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7ec07acc0 0x7fe7ec0791c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54018/0 (socket says 172.21.15.39:54018) 2023-07-21T18:34:15.217 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:14.282+0000 7fe7eaffd700 1 -- 172.21.15.39:0/1179207516 learned_addr learned my addr 172.21.15.39:0/1179207516 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:34:15.217 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:14.282+0000 7fe7eaffd700 1 -- 172.21.15.39:0/1179207516 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7ec079840 con 0x7fe7ec07acc0 2023-07-21T18:34:15.217 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:14.282+0000 7fe7eaffd700 1 --2- 172.21.15.39:0/1179207516 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7ec07acc0 0x7fe7ec0791c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc009e80 tx=0x7fe7dc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=762ba2467ca8bd56 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:34:15.217 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:14.282+0000 7fe7e9ffb700 1 -- 172.21.15.39:0/1179207516 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe7dc00e070 con 0x7fe7ec07acc0 2023-07-21T18:34:15.217 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:14.282+0000 7fe7e9ffb700 1 -- 172.21.15.39:0/1179207516 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe7dc00b8c0 con 0x7fe7ec07acc0 2023-07-21T18:34:15.217 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:14.282+0000 7fe7e9ffb700 1 -- 172.21.15.39:0/1179207516 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe7dc0043e0 con 0x7fe7ec07acc0 2023-07-21T18:34:15.217 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:14.283+0000 7fe7f19c9700 1 -- 172.21.15.39:0/1179207516 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7ec07acc0 msgr2=0x7fe7ec0791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:34:15.218 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:14.283+0000 7fe7f19c9700 1 --2- 172.21.15.39:0/1179207516 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7ec07acc0 0x7fe7ec0791c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc009e80 tx=0x7fe7dc00b330 comp rx=0 tx=0).stop 2023-07-21T18:34:15.218 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:14.283+0000 7fe7f19c9700 1 -- 172.21.15.39:0/1179207516 shutdown_connections 2023-07-21T18:34:15.218 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:14.283+0000 7fe7f19c9700 1 --2- 172.21.15.39:0/1179207516 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7ec07acc0 0x7fe7ec0791c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:34:15.218 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:14.283+0000 7fe7f19c9700 1 -- 172.21.15.39:0/1179207516 >> 172.21.15.39:0/1179207516 conn(0x7fe7ec100b40 msgr2=0x7fe7ec102f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:34:15.218 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:14.283+0000 7fe7f19c9700 1 -- 172.21.15.39:0/1179207516 shutdown_connections 2023-07-21T18:34:15.219 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:14.283+0000 7fe7f19c9700 1 -- 172.21.15.39:0/1179207516 wait complete. 2023-07-21T18:34:15.219 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:14.284+0000 7fe7f19c9700 1 Processor -- start 2023-07-21T18:34:15.219 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:14.284+0000 7fe7f19c9700 1 -- start start 2023-07-21T18:34:15.219 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:14.284+0000 7fe7f19c9700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7ec07acc0 0x7fe7ec19b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:34:15.219 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:14.284+0000 7fe7f19c9700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ec19b9a0 con 0x7fe7ec07acc0 2023-07-21T18:34:15.219 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:14.284+0000 7fe7eaffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7ec07acc0 0x7fe7ec19b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:34:15.219 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:14.284+0000 7fe7eaffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7ec07acc0 0x7fe7ec19b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54020/0 (socket says 172.21.15.39:54020) 2023-07-21T18:34:15.220 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:14.284+0000 7fe7eaffd700 1 -- 172.21.15.39:0/1222888566 learned_addr learned my addr 172.21.15.39:0/1222888566 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:34:15.220 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:14.285+0000 7fe7eaffd700 1 -- 172.21.15.39:0/1222888566 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7dc009b30 con 0x7fe7ec07acc0 2023-07-21T18:34:15.220 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:14.285+0000 7fe7eaffd700 1 --2- 172.21.15.39:0/1222888566 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7ec07acc0 0x7fe7ec19b460 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc009e50 tx=0x7fe7dc003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:34:15.220 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:14.285+0000 7fe7f09c7700 1 -- 172.21.15.39:0/1222888566 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe7dc00e040 con 0x7fe7ec07acc0 2023-07-21T18:34:15.220 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:14.285+0000 7fe7f19c9700 1 -- 172.21.15.39:0/1222888566 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7ec19bba0 con 0x7fe7ec07acc0 2023-07-21T18:34:15.220 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:14.285+0000 7fe7f09c7700 1 -- 172.21.15.39:0/1222888566 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe7dc00be30 con 0x7fe7ec07acc0 2023-07-21T18:34:15.220 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:14.285+0000 7fe7f09c7700 1 -- 172.21.15.39:0/1222888566 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe7dc01c710 con 0x7fe7ec07acc0 2023-07-21T18:34:15.221 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:14.285+0000 7fe7f19c9700 1 -- 172.21.15.39:0/1222888566 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7ec19c020 con 0x7fe7ec07acc0 2023-07-21T18:34:15.221 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:14.286+0000 7fe7f09c7700 1 -- 172.21.15.39:0/1222888566 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fe7dc01c9d0 con 0x7fe7ec07acc0 2023-07-21T18:34:15.221 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:14.286+0000 7fe7f09c7700 1 --2- 172.21.15.39:0/1222888566 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fe7d80387b0 0x7fe7d803ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:34:15.221 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:14.286+0000 7fe7f09c7700 1 -- 172.21.15.39:0/1222888566 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe7dc04c950 con 0x7fe7ec07acc0 2023-07-21T18:34:15.221 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:14.286+0000 7fe7f19c9700 1 -- 172.21.15.39:0/1222888566 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7ec061b40 con 0x7fe7ec07acc0 2023-07-21T18:34:15.221 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:14.287+0000 7fe7ea7fc700 1 --2- 172.21.15.39:0/1222888566 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fe7d80387b0 0x7fe7d803ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:34:15.221 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:14.287+0000 7fe7ea7fc700 1 --2- 172.21.15.39:0/1222888566 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fe7d80387b0 0x7fe7d803ac60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4006fd0 tx=0x7fe7d4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:34:15.222 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:14.291+0000 7fe7f09c7700 1 -- 172.21.15.39:0/1222888566 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fe7dc017020 con 0x7fe7ec07acc0 2023-07-21T18:34:15.222 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:14.860+0000 7fe7f19c9700 1 -- 172.21.15.39:0/1222888566 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7fe7ec19eaf0 con 0x7fe7ec07acc0 2023-07-21T18:34:15.222 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:14.863+0000 7fe7f09c7700 1 -- 172.21.15.39:0/1222888566 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7fe7dc011750 con 0x7fe7ec07acc0 2023-07-21T18:34:15.222 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:14.866+0000 7fe7f19c9700 1 -- 172.21.15.39:0/1222888566 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fe7d80387b0 msgr2=0x7fe7d803ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:34:15.222 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:14.866+0000 7fe7f19c9700 1 --2- 172.21.15.39:0/1222888566 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fe7d80387b0 0x7fe7d803ac60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4006fd0 tx=0x7fe7d4006e40 comp rx=0 tx=0).stop 2023-07-21T18:34:15.222 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:14.866+0000 7fe7f19c9700 1 -- 172.21.15.39:0/1222888566 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7ec07acc0 msgr2=0x7fe7ec19b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:34:15.222 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:14.866+0000 7fe7f19c9700 1 --2- 172.21.15.39:0/1222888566 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7ec07acc0 0x7fe7ec19b460 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc009e50 tx=0x7fe7dc003ce0 comp rx=0 tx=0).stop 2023-07-21T18:34:15.223 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:14.866+0000 7fe7f19c9700 1 -- 172.21.15.39:0/1222888566 shutdown_connections 2023-07-21T18:34:15.223 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:14.866+0000 7fe7f19c9700 1 --2- 172.21.15.39:0/1222888566 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fe7d80387b0 0x7fe7d803ac60 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:34:15.223 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:14.866+0000 7fe7f19c9700 1 --2- 172.21.15.39:0/1222888566 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7ec07acc0 0x7fe7ec19b460 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:34:15.223 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:14.866+0000 7fe7f19c9700 1 -- 172.21.15.39:0/1222888566 >> 172.21.15.39:0/1222888566 conn(0x7fe7ec100b40 msgr2=0x7fe7ec102230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:34:15.223 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:14.866+0000 7fe7f19c9700 1 -- 172.21.15.39:0/1222888566 shutdown_connections 2023-07-21T18:34:15.223 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-07-21T18:34:14.866+0000 7fe7f19c9700 1 -- 172.21.15.39:0/1222888566 wait complete. 2023-07-21T18:34:15.223 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2023-07-21T18:34:15.224 INFO:teuthology.orchestra.run.smithi039.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2023-07-21T18:34:15.224 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:34:15.224 INFO:teuthology.orchestra.run.smithi039.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2023-07-21T18:34:15.224 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:34:15.224 INFO:teuthology.orchestra.run.smithi039.stdout:Or, if you are only running a single cluster on this host: 2023-07-21T18:34:15.224 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:34:15.224 INFO:teuthology.orchestra.run.smithi039.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2023-07-21T18:34:15.225 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:34:15.225 INFO:teuthology.orchestra.run.smithi039.stdout:Please consider enabling telemetry to help improve Ceph: 2023-07-21T18:34:15.225 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:34:15.225 INFO:teuthology.orchestra.run.smithi039.stdout: ceph telemetry on 2023-07-21T18:34:15.225 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:34:15.225 INFO:teuthology.orchestra.run.smithi039.stdout:For more information see: 2023-07-21T18:34:15.225 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:34:15.226 INFO:teuthology.orchestra.run.smithi039.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2023-07-21T18:34:15.226 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:34:15.226 INFO:teuthology.orchestra.run.smithi039.stdout:Bootstrap complete. 2023-07-21T18:34:15.251 INFO:tasks.cephadm:Fetching config... 2023-07-21T18:34:15.252 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-07-21T18:34:15.252 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2023-07-21T18:34:15.310 INFO:tasks.cephadm:Fetching client.admin keyring... 2023-07-21T18:34:15.311 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-07-21T18:34:15.311 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2023-07-21T18:34:15.367 INFO:tasks.cephadm:Fetching mon keyring... 2023-07-21T18:34:15.368 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-07-21T18:34:15.368 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.a/keyring of=/dev/stdout 2023-07-21T18:34:15.437 INFO:tasks.cephadm:Fetching pub ssh key... 2023-07-21T18:34:15.438 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-07-21T18:34:15.438 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2023-07-21T18:34:15.494 INFO:tasks.cephadm:Installing pub ssh key for root users... 2023-07-21T18:34:15.495 DEBUG:teuthology.orchestra.run.smithi039:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCebQXoWW5pGKf3B9BRIRTx6M1njcb9Jo/HOWcgbq3rW5l42Nngvd8fmR4NAIAB8AsjjyziK3NchU0DMbb9OKFNXOoPDqTZ1CUSw7uTedVTFqaco7MMOScBWxtSReh9AkRBD0K6hXFIrzVFfMjkCLP1/t356gXWFToEIfYCO0yBfTVuV08PzVp7Nxh7S+PzsAD/+CzXVLgrmOEPorNOcwT532jfCvepP7pm8wlefbfMgq2FJnj5dJmh7ldabie+FBvAXujxLbjzVr1BKrIyL9UYFRi0zP8tBRV7ar1jJ3ZISz5UKXKEKg9DFgGi381qrlj6oKL9UtsaO5SOByaM75pR2j+splllPFUabZUJC6oY4rpLRgSzQ4CoBax+vFjkD/d2eVQuemQ9SDLqViV1QLN2C/0NEpTO3R+vhUyE0cifUYWbWMfTXrg7N84sg0rNYO7sV0GWLgFOj9X9kizFbcC3a61A5axIQ1YzV2/83aynpJCw7mXgianKv9NBlueFyLk= ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-07-21T18:34:15.575 INFO:teuthology.orchestra.run.smithi039.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCebQXoWW5pGKf3B9BRIRTx6M1njcb9Jo/HOWcgbq3rW5l42Nngvd8fmR4NAIAB8AsjjyziK3NchU0DMbb9OKFNXOoPDqTZ1CUSw7uTedVTFqaco7MMOScBWxtSReh9AkRBD0K6hXFIrzVFfMjkCLP1/t356gXWFToEIfYCO0yBfTVuV08PzVp7Nxh7S+PzsAD/+CzXVLgrmOEPorNOcwT532jfCvepP7pm8wlefbfMgq2FJnj5dJmh7ldabie+FBvAXujxLbjzVr1BKrIyL9UYFRi0zP8tBRV7ar1jJ3ZISz5UKXKEKg9DFgGi381qrlj6oKL9UtsaO5SOByaM75pR2j+splllPFUabZUJC6oY4rpLRgSzQ4CoBax+vFjkD/d2eVQuemQ9SDLqViV1QLN2C/0NEpTO3R+vhUyE0cifUYWbWMfTXrg7N84sg0rNYO7sV0GWLgFOj9X9kizFbcC3a61A5axIQ1YzV2/83aynpJCw7mXgianKv9NBlueFyLk= ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c 2023-07-21T18:34:15.591 DEBUG:teuthology.orchestra.run.smithi155:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCebQXoWW5pGKf3B9BRIRTx6M1njcb9Jo/HOWcgbq3rW5l42Nngvd8fmR4NAIAB8AsjjyziK3NchU0DMbb9OKFNXOoPDqTZ1CUSw7uTedVTFqaco7MMOScBWxtSReh9AkRBD0K6hXFIrzVFfMjkCLP1/t356gXWFToEIfYCO0yBfTVuV08PzVp7Nxh7S+PzsAD/+CzXVLgrmOEPorNOcwT532jfCvepP7pm8wlefbfMgq2FJnj5dJmh7ldabie+FBvAXujxLbjzVr1BKrIyL9UYFRi0zP8tBRV7ar1jJ3ZISz5UKXKEKg9DFgGi381qrlj6oKL9UtsaO5SOByaM75pR2j+splllPFUabZUJC6oY4rpLRgSzQ4CoBax+vFjkD/d2eVQuemQ9SDLqViV1QLN2C/0NEpTO3R+vhUyE0cifUYWbWMfTXrg7N84sg0rNYO7sV0GWLgFOj9X9kizFbcC3a61A5axIQ1YzV2/83aynpJCw7mXgianKv9NBlueFyLk= ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-07-21T18:34:15.637 INFO:teuthology.orchestra.run.smithi155.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCebQXoWW5pGKf3B9BRIRTx6M1njcb9Jo/HOWcgbq3rW5l42Nngvd8fmR4NAIAB8AsjjyziK3NchU0DMbb9OKFNXOoPDqTZ1CUSw7uTedVTFqaco7MMOScBWxtSReh9AkRBD0K6hXFIrzVFfMjkCLP1/t356gXWFToEIfYCO0yBfTVuV08PzVp7Nxh7S+PzsAD/+CzXVLgrmOEPorNOcwT532jfCvepP7pm8wlefbfMgq2FJnj5dJmh7ldabie+FBvAXujxLbjzVr1BKrIyL9UYFRi0zP8tBRV7ar1jJ3ZISz5UKXKEKg9DFgGi381qrlj6oKL9UtsaO5SOByaM75pR2j+splllPFUabZUJC6oY4rpLRgSzQ4CoBax+vFjkD/d2eVQuemQ9SDLqViV1QLN2C/0NEpTO3R+vhUyE0cifUYWbWMfTXrg7N84sg0rNYO7sV0GWLgFOj9X9kizFbcC3a61A5axIQ1YzV2/83aynpJCw7mXgianKv9NBlueFyLk= ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c 2023-07-21T18:34:15.650 DEBUG:teuthology.orchestra.run.smithi193:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCebQXoWW5pGKf3B9BRIRTx6M1njcb9Jo/HOWcgbq3rW5l42Nngvd8fmR4NAIAB8AsjjyziK3NchU0DMbb9OKFNXOoPDqTZ1CUSw7uTedVTFqaco7MMOScBWxtSReh9AkRBD0K6hXFIrzVFfMjkCLP1/t356gXWFToEIfYCO0yBfTVuV08PzVp7Nxh7S+PzsAD/+CzXVLgrmOEPorNOcwT532jfCvepP7pm8wlefbfMgq2FJnj5dJmh7ldabie+FBvAXujxLbjzVr1BKrIyL9UYFRi0zP8tBRV7ar1jJ3ZISz5UKXKEKg9DFgGi381qrlj6oKL9UtsaO5SOByaM75pR2j+splllPFUabZUJC6oY4rpLRgSzQ4CoBax+vFjkD/d2eVQuemQ9SDLqViV1QLN2C/0NEpTO3R+vhUyE0cifUYWbWMfTXrg7N84sg0rNYO7sV0GWLgFOj9X9kizFbcC3a61A5axIQ1YzV2/83aynpJCw7mXgianKv9NBlueFyLk= ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-07-21T18:34:15.696 INFO:teuthology.orchestra.run.smithi193.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCebQXoWW5pGKf3B9BRIRTx6M1njcb9Jo/HOWcgbq3rW5l42Nngvd8fmR4NAIAB8AsjjyziK3NchU0DMbb9OKFNXOoPDqTZ1CUSw7uTedVTFqaco7MMOScBWxtSReh9AkRBD0K6hXFIrzVFfMjkCLP1/t356gXWFToEIfYCO0yBfTVuV08PzVp7Nxh7S+PzsAD/+CzXVLgrmOEPorNOcwT532jfCvepP7pm8wlefbfMgq2FJnj5dJmh7ldabie+FBvAXujxLbjzVr1BKrIyL9UYFRi0zP8tBRV7ar1jJ3ZISz5UKXKEKg9DFgGi381qrlj6oKL9UtsaO5SOByaM75pR2j+splllPFUabZUJC6oY4rpLRgSzQ4CoBax+vFjkD/d2eVQuemQ9SDLqViV1QLN2C/0NEpTO3R+vhUyE0cifUYWbWMfTXrg7N84sg0rNYO7sV0GWLgFOj9X9kizFbcC3a61A5axIQ1YzV2/83aynpJCw7mXgianKv9NBlueFyLk= ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c 2023-07-21T18:34:15.708 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph config set mgr mgr/cephadm/allow_ptrace true 2023-07-21T18:34:16.520 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:34:17.837 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:17.802+0000 7fbbf82c0700 1 -- 172.21.15.39:0/1207619026 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbf00fe4f0 msgr2=0x7fbbf00fe900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:34:17.837 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:17.802+0000 7fbbf82c0700 1 --2- 172.21.15.39:0/1207619026 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbf00fe4f0 0x7fbbf00fe900 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbbec009910 tx=0x7fbbec009c20 comp rx=0 tx=0).stop 2023-07-21T18:34:17.838 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:17.803+0000 7fbbf82c0700 1 -- 172.21.15.39:0/1207619026 shutdown_connections 2023-07-21T18:34:17.838 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:17.803+0000 7fbbf82c0700 1 --2- 172.21.15.39:0/1207619026 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbf00fe4f0 0x7fbbf00fe900 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:34:17.838 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:17.803+0000 7fbbf82c0700 1 -- 172.21.15.39:0/1207619026 >> 172.21.15.39:0/1207619026 conn(0x7fbbf00f9a80 msgr2=0x7fbbf00fbed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:34:17.838 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:17.803+0000 7fbbf82c0700 1 -- 172.21.15.39:0/1207619026 shutdown_connections 2023-07-21T18:34:17.838 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:17.803+0000 7fbbf82c0700 1 -- 172.21.15.39:0/1207619026 wait complete. 2023-07-21T18:34:17.840 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:17.803+0000 7fbbf82c0700 1 Processor -- start 2023-07-21T18:34:17.840 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:17.803+0000 7fbbf82c0700 1 -- start start 2023-07-21T18:34:17.840 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:17.803+0000 7fbbf82c0700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbf00fe4f0 0x7fbbf0193660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:34:17.840 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:17.804+0000 7fbbf82c0700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf0193ba0 con 0x7fbbf00fe4f0 2023-07-21T18:34:17.840 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:17.804+0000 7fbbf605c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbf00fe4f0 0x7fbbf0193660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:34:17.840 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:17.804+0000 7fbbf605c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbf00fe4f0 0x7fbbf0193660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54026/0 (socket says 172.21.15.39:54026) 2023-07-21T18:34:17.841 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:17.804+0000 7fbbf605c700 1 -- 172.21.15.39:0/3642025488 learned_addr learned my addr 172.21.15.39:0/3642025488 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:34:17.841 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:17.804+0000 7fbbf605c700 1 -- 172.21.15.39:0/3642025488 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbec0095f0 con 0x7fbbf00fe4f0 2023-07-21T18:34:17.841 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:17.804+0000 7fbbf605c700 1 --2- 172.21.15.39:0/3642025488 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbf00fe4f0 0x7fbbf0193660 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbbec006010 tx=0x7fbbec0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:34:17.841 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:17.804+0000 7fbbe37fe700 1 -- 172.21.15.39:0/3642025488 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbbec01d070 con 0x7fbbf00fe4f0 2023-07-21T18:34:17.841 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:17.804+0000 7fbbe37fe700 1 -- 172.21.15.39:0/3642025488 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbbec0044e0 con 0x7fbbf00fe4f0 2023-07-21T18:34:17.841 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:17.804+0000 7fbbf82c0700 1 -- 172.21.15.39:0/3642025488 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbf0193da0 con 0x7fbbf00fe4f0 2023-07-21T18:34:17.841 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:17.804+0000 7fbbe37fe700 1 -- 172.21.15.39:0/3642025488 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbbec0039b0 con 0x7fbbf00fe4f0 2023-07-21T18:34:17.842 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:17.804+0000 7fbbf82c0700 1 -- 172.21.15.39:0/3642025488 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbf01941e0 con 0x7fbbf00fe4f0 2023-07-21T18:34:17.842 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:17.805+0000 7fbbe37fe700 1 -- 172.21.15.39:0/3642025488 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fbbec003c20 con 0x7fbbf00fe4f0 2023-07-21T18:34:17.842 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:17.805+0000 7fbbf82c0700 1 -- 172.21.15.39:0/3642025488 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbf018d0c0 con 0x7fbbf00fe4f0 2023-07-21T18:34:17.842 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:17.805+0000 7fbbe37fe700 1 --2- 172.21.15.39:0/3642025488 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fbbdc038800 0x7fbbdc03acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:34:17.842 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:17.805+0000 7fbbe37fe700 1 -- 172.21.15.39:0/3642025488 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fbbec04cd40 con 0x7fbbf00fe4f0 2023-07-21T18:34:17.842 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:17.806+0000 7fbbf585b700 1 --2- 172.21.15.39:0/3642025488 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fbbdc038800 0x7fbbdc03acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:34:17.843 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:17.806+0000 7fbbf585b700 1 --2- 172.21.15.39:0/3642025488 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fbbdc038800 0x7fbbdc03acb0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbbe4006fd0 tx=0x7fbbe4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:34:17.844 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:17.810+0000 7fbbe37fe700 1 -- 172.21.15.39:0/3642025488 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fbbec02a950 con 0x7fbbf00fe4f0 2023-07-21T18:34:18.042 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:18.007+0000 7fbbf82c0700 1 -- 172.21.15.39:0/3642025488 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7fbbf002c850 con 0x7fbbf00fe4f0 2023-07-21T18:34:18.045 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:18.011+0000 7fbbe37fe700 1 -- 172.21.15.39:0/3642025488 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7fbbec02a360 con 0x7fbbf00fe4f0 2023-07-21T18:34:18.048 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:18.013+0000 7fbbf82c0700 1 -- 172.21.15.39:0/3642025488 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fbbdc038800 msgr2=0x7fbbdc03acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:34:18.048 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:18.013+0000 7fbbf82c0700 1 --2- 172.21.15.39:0/3642025488 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fbbdc038800 0x7fbbdc03acb0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbbe4006fd0 tx=0x7fbbe4006e40 comp rx=0 tx=0).stop 2023-07-21T18:34:18.048 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:18.013+0000 7fbbf82c0700 1 -- 172.21.15.39:0/3642025488 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbf00fe4f0 msgr2=0x7fbbf0193660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:34:18.048 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:18.013+0000 7fbbf82c0700 1 --2- 172.21.15.39:0/3642025488 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbf00fe4f0 0x7fbbf0193660 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbbec006010 tx=0x7fbbec0057c0 comp rx=0 tx=0).stop 2023-07-21T18:34:18.048 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:18.013+0000 7fbbf82c0700 1 -- 172.21.15.39:0/3642025488 shutdown_connections 2023-07-21T18:34:18.048 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:18.013+0000 7fbbf82c0700 1 --2- 172.21.15.39:0/3642025488 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fbbdc038800 0x7fbbdc03acb0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:34:18.049 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:18.013+0000 7fbbf82c0700 1 --2- 172.21.15.39:0/3642025488 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbf00fe4f0 0x7fbbf0193660 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:34:18.049 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:18.013+0000 7fbbf82c0700 1 -- 172.21.15.39:0/3642025488 >> 172.21.15.39:0/3642025488 conn(0x7fbbf00f9a80 msgr2=0x7fbbf00fa7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:34:18.049 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:18.013+0000 7fbbf82c0700 1 -- 172.21.15.39:0/3642025488 shutdown_connections 2023-07-21T18:34:18.049 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:18.013+0000 7fbbf82c0700 1 -- 172.21.15.39:0/3642025488 wait complete. 2023-07-21T18:34:18.382 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2023-07-21T18:34:18.383 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph orch client-keyring set client.admin '*' --mode 0755 2023-07-21T18:34:18.945 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:34:19.417 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:19 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/3642025488' entity='client.admin' 2023-07-21T18:34:19.417 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:19 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:19.417 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:19 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:19.418 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:19 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi039", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:34:19.418 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:19 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:19.418 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:19 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:34:19.418 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:19 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:20.031 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:19.996+0000 7f24baea3700 1 -- 172.21.15.39:0/163988973 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24b40fe4e0 msgr2=0x7f24b40fe8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:34:20.032 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:19.996+0000 7f24baea3700 1 --2- 172.21.15.39:0/163988973 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24b40fe4e0 0x7f24b40fe8f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f24a8009910 tx=0x7f24a8009c20 comp rx=0 tx=0).stop 2023-07-21T18:34:20.032 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:19.996+0000 7f24baea3700 1 -- 172.21.15.39:0/163988973 shutdown_connections 2023-07-21T18:34:20.032 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:19.996+0000 7f24baea3700 1 --2- 172.21.15.39:0/163988973 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24b40fe4e0 0x7f24b40fe8f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:34:20.032 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:19.996+0000 7f24baea3700 1 -- 172.21.15.39:0/163988973 >> 172.21.15.39:0/163988973 conn(0x7f24b40f9a70 msgr2=0x7f24b40fbec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:34:20.032 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:19.996+0000 7f24baea3700 1 -- 172.21.15.39:0/163988973 shutdown_connections 2023-07-21T18:34:20.032 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:19.996+0000 7f24baea3700 1 -- 172.21.15.39:0/163988973 wait complete. 2023-07-21T18:34:20.032 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:19.997+0000 7f24baea3700 1 Processor -- start 2023-07-21T18:34:20.033 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:19.997+0000 7f24baea3700 1 -- start start 2023-07-21T18:34:20.033 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:19.997+0000 7f24baea3700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24b40fe4e0 0x7f24b4194750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:34:20.033 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:19.997+0000 7f24baea3700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b4194c90 con 0x7f24b40fe4e0 2023-07-21T18:34:20.034 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:19.997+0000 7f24b8c3f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24b40fe4e0 0x7f24b4194750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:34:20.034 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:19.997+0000 7f24b8c3f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24b40fe4e0 0x7f24b4194750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54032/0 (socket says 172.21.15.39:54032) 2023-07-21T18:34:20.034 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:19.997+0000 7f24b8c3f700 1 -- 172.21.15.39:0/2676764653 learned_addr learned my addr 172.21.15.39:0/2676764653 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:34:20.035 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:19.997+0000 7f24b8c3f700 1 -- 172.21.15.39:0/2676764653 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24a80095f0 con 0x7f24b40fe4e0 2023-07-21T18:34:20.035 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:19.998+0000 7f24b8c3f700 1 --2- 172.21.15.39:0/2676764653 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24b40fe4e0 0x7f24b4194750 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f24a8004f40 tx=0x7f24a80057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:34:20.035 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:19.998+0000 7f24b1ffb700 1 -- 172.21.15.39:0/2676764653 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f24a801d070 con 0x7f24b40fe4e0 2023-07-21T18:34:20.035 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:19.998+0000 7f24b1ffb700 1 -- 172.21.15.39:0/2676764653 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24a800bc10 con 0x7f24b40fe4e0 2023-07-21T18:34:20.035 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:19.998+0000 7f24b1ffb700 1 -- 172.21.15.39:0/2676764653 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f24a8003a70 con 0x7f24b40fe4e0 2023-07-21T18:34:20.035 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:19.998+0000 7f24baea3700 1 -- 172.21.15.39:0/2676764653 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24b4194e90 con 0x7f24b40fe4e0 2023-07-21T18:34:20.035 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:19.998+0000 7f24baea3700 1 -- 172.21.15.39:0/2676764653 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24b4195330 con 0x7f24b40fe4e0 2023-07-21T18:34:20.035 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:19.999+0000 7f24b1ffb700 1 -- 172.21.15.39:0/2676764653 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f24a8003ce0 con 0x7f24b40fe4e0 2023-07-21T18:34:20.036 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:19.999+0000 7f24b1ffb700 1 --2- 172.21.15.39:0/2676764653 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f24a4038800 0x7f24a403acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:34:20.036 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:19.999+0000 7f24baea3700 1 -- 172.21.15.39:0/2676764653 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24b418e0c0 con 0x7f24b40fe4e0 2023-07-21T18:34:20.036 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:19.999+0000 7f24b1ffb700 1 -- 172.21.15.39:0/2676764653 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f24a804c470 con 0x7f24b40fe4e0 2023-07-21T18:34:20.036 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:19.999+0000 7f24b3fff700 1 --2- 172.21.15.39:0/2676764653 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f24a4038800 0x7f24a403acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:34:20.036 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:20.000+0000 7f24b3fff700 1 --2- 172.21.15.39:0/2676764653 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f24a4038800 0x7f24a403acb0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f24a0006fd0 tx=0x7f24a0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:34:20.038 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:20.004+0000 7f24b1ffb700 1 -- 172.21.15.39:0/2676764653 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f24a800bd80 con 0x7f24b40fe4e0 2023-07-21T18:34:20.240 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:20.205+0000 7f24baea3700 1 -- 172.21.15.39:0/2676764653 --> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f24b40609e0 con 0x7f24a4038800 2023-07-21T18:34:20.245 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:20.211+0000 7f24b1ffb700 1 -- 172.21.15.39:0/2676764653 <== mgr.14152 v2:172.21.15.39:6800/2116291712 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f24b40609e0 con 0x7f24a4038800 2023-07-21T18:34:20.247 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:20.213+0000 7f24baea3700 1 -- 172.21.15.39:0/2676764653 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f24a4038800 msgr2=0x7f24a403acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:34:20.248 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:20.213+0000 7f24baea3700 1 --2- 172.21.15.39:0/2676764653 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f24a4038800 0x7f24a403acb0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f24a0006fd0 tx=0x7f24a0006e40 comp rx=0 tx=0).stop 2023-07-21T18:34:20.248 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:20.213+0000 7f24baea3700 1 -- 172.21.15.39:0/2676764653 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24b40fe4e0 msgr2=0x7f24b4194750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:34:20.248 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:20.213+0000 7f24baea3700 1 --2- 172.21.15.39:0/2676764653 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24b40fe4e0 0x7f24b4194750 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f24a8004f40 tx=0x7f24a80057c0 comp rx=0 tx=0).stop 2023-07-21T18:34:20.248 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:20.213+0000 7f24baea3700 1 -- 172.21.15.39:0/2676764653 shutdown_connections 2023-07-21T18:34:20.248 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:20.213+0000 7f24baea3700 1 --2- 172.21.15.39:0/2676764653 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f24a4038800 0x7f24a403acb0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:34:20.249 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:20.213+0000 7f24baea3700 1 --2- 172.21.15.39:0/2676764653 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24b40fe4e0 0x7f24b4194750 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:34:20.249 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:20.213+0000 7f24baea3700 1 -- 172.21.15.39:0/2676764653 >> 172.21.15.39:0/2676764653 conn(0x7f24b40f9a70 msgr2=0x7f24b4067fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:34:20.249 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:20.213+0000 7f24baea3700 1 -- 172.21.15.39:0/2676764653 shutdown_connections 2023-07-21T18:34:20.249 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:20.213+0000 7f24baea3700 1 -- 172.21.15.39:0/2676764653 wait complete. 2023-07-21T18:34:20.697 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi155 2023-07-21T18:34:20.697 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-07-21T18:34:20.697 DEBUG:teuthology.orchestra.run.smithi155:> dd of=/etc/ceph/ceph.conf 2023-07-21T18:34:20.718 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-07-21T18:34:20.718 DEBUG:teuthology.orchestra.run.smithi155:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-21T18:34:20.777 INFO:tasks.cephadm:Adding host smithi155 to orchestrator... 2023-07-21T18:34:20.777 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph orch host add smithi155 2023-07-21T18:34:21.352 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:34:21.539 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:21 smithi039 ceph-mon[96489]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:34:21.539 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:21 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:21.540 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:21 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:34:21.540 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:21 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:34:21.540 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:21 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:34:21.540 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:21 smithi039 ceph-mon[96489]: Updating smithi039:/etc/ceph/ceph.conf 2023-07-21T18:34:21.540 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:21 smithi039 ceph-mon[96489]: Updating smithi039:/var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T18:34:21.540 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:21 smithi039 ceph-mon[96489]: Updating smithi039:/etc/ceph/ceph.client.admin.keyring 2023-07-21T18:34:21.541 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:21 smithi039 ceph-mon[96489]: Updating smithi039:/var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-21T18:34:22.382 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:22.347+0000 7fc617789700 1 -- 172.21.15.39:0/2320957382 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6101044f0 msgr2=0x7fc6101048c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:34:22.383 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:22.347+0000 7fc617789700 1 --2- 172.21.15.39:0/2320957382 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6101044f0 0x7fc6101048c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc60c009910 tx=0x7fc60c009c20 comp rx=0 tx=0).stop 2023-07-21T18:34:22.383 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:22.347+0000 7fc617789700 1 -- 172.21.15.39:0/2320957382 shutdown_connections 2023-07-21T18:34:22.383 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:22.347+0000 7fc617789700 1 --2- 172.21.15.39:0/2320957382 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6101044f0 0x7fc6101048c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:34:22.383 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:22.347+0000 7fc617789700 1 -- 172.21.15.39:0/2320957382 >> 172.21.15.39:0/2320957382 conn(0x7fc6100742b0 msgr2=0x7fc6100746b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:34:22.383 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:22.348+0000 7fc617789700 1 -- 172.21.15.39:0/2320957382 shutdown_connections 2023-07-21T18:34:22.383 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:22.348+0000 7fc617789700 1 -- 172.21.15.39:0/2320957382 wait complete. 2023-07-21T18:34:22.383 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:22.348+0000 7fc617789700 1 Processor -- start 2023-07-21T18:34:22.384 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:22.348+0000 7fc617789700 1 -- start start 2023-07-21T18:34:22.385 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:22.348+0000 7fc617789700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6101044f0 0x7fc610195960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:34:22.385 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:22.348+0000 7fc617789700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc610195ea0 con 0x7fc6101044f0 2023-07-21T18:34:22.385 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:22.349+0000 7fc615525700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6101044f0 0x7fc610195960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:34:22.386 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:22.349+0000 7fc615525700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6101044f0 0x7fc610195960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54038/0 (socket says 172.21.15.39:54038) 2023-07-21T18:34:22.386 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:22.349+0000 7fc615525700 1 -- 172.21.15.39:0/1822303746 learned_addr learned my addr 172.21.15.39:0/1822303746 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:34:22.386 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:22.349+0000 7fc615525700 1 -- 172.21.15.39:0/1822303746 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc60c0095f0 con 0x7fc6101044f0 2023-07-21T18:34:22.386 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:22.349+0000 7fc615525700 1 --2- 172.21.15.39:0/1822303746 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6101044f0 0x7fc610195960 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc60c004f40 tx=0x7fc60c005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:34:22.386 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:22.349+0000 7fc6027fc700 1 -- 172.21.15.39:0/1822303746 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc60c01d070 con 0x7fc6101044f0 2023-07-21T18:34:22.386 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:22.349+0000 7fc6027fc700 1 -- 172.21.15.39:0/1822303746 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc60c004370 con 0x7fc6101044f0 2023-07-21T18:34:22.386 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:22.349+0000 7fc617789700 1 -- 172.21.15.39:0/1822303746 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6101960a0 con 0x7fc6101044f0 2023-07-21T18:34:22.387 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:22.349+0000 7fc6027fc700 1 -- 172.21.15.39:0/1822303746 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc60c0039b0 con 0x7fc6101044f0 2023-07-21T18:34:22.387 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:22.349+0000 7fc617789700 1 -- 172.21.15.39:0/1822303746 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6101964c0 con 0x7fc6101044f0 2023-07-21T18:34:22.387 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:22.350+0000 7fc6027fc700 1 -- 172.21.15.39:0/1822303746 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fc60c003c20 con 0x7fc6101044f0 2023-07-21T18:34:22.387 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:22.350+0000 7fc6027fc700 1 --2- 172.21.15.39:0/1822303746 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fc5fc038800 0x7fc5fc03acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:34:22.387 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:22.350+0000 7fc6027fc700 1 -- 172.21.15.39:0/1822303746 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc60c04c510 con 0x7fc6101044f0 2023-07-21T18:34:22.387 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:22.350+0000 7fc617789700 1 -- 172.21.15.39:0/1822303746 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6101027a0 con 0x7fc6101044f0 2023-07-21T18:34:22.387 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:22.350+0000 7fc614d24700 1 --2- 172.21.15.39:0/1822303746 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fc5fc038800 0x7fc5fc03acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:34:22.387 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:22.351+0000 7fc614d24700 1 --2- 172.21.15.39:0/1822303746 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fc5fc038800 0x7fc5fc03acb0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc604006fd0 tx=0x7fc604006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:34:22.389 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:22.355+0000 7fc6027fc700 1 -- 172.21.15.39:0/1822303746 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fc60c026070 con 0x7fc6101044f0 2023-07-21T18:34:22.450 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:22 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:22.450 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:22 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:22.450 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:22 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:22.591 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:22.557+0000 7fc617789700 1 -- 172.21.15.39:0/1822303746 --> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi155", "target": ["mon-mgr", ""]}) v1 -- 0x7fc610067d00 con 0x7fc5fc038800 2023-07-21T18:34:23.700 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:23 smithi039 ceph-mon[96489]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi155", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:34:23.700 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:23 smithi039 ceph-mon[96489]: Deploying cephadm binary to smithi155 2023-07-21T18:34:25.338 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:25.303+0000 7fc6027fc700 1 -- 172.21.15.39:0/1822303746 <== mgr.14152 v2:172.21.15.39:6800/2116291712 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fc610067d00 con 0x7fc5fc038800 2023-07-21T18:34:25.338 INFO:teuthology.orchestra.run.smithi039.stdout:Added host 'smithi155' with addr '172.21.15.155' 2023-07-21T18:34:25.340 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:25.306+0000 7fc617789700 1 -- 172.21.15.39:0/1822303746 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fc5fc038800 msgr2=0x7fc5fc03acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:34:25.340 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:25.306+0000 7fc617789700 1 --2- 172.21.15.39:0/1822303746 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fc5fc038800 0x7fc5fc03acb0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc604006fd0 tx=0x7fc604006e40 comp rx=0 tx=0).stop 2023-07-21T18:34:25.340 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:25.306+0000 7fc617789700 1 -- 172.21.15.39:0/1822303746 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6101044f0 msgr2=0x7fc610195960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:34:25.341 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:25.306+0000 7fc617789700 1 --2- 172.21.15.39:0/1822303746 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6101044f0 0x7fc610195960 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc60c004f40 tx=0x7fc60c005020 comp rx=0 tx=0).stop 2023-07-21T18:34:25.341 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:25.306+0000 7fc617789700 1 -- 172.21.15.39:0/1822303746 shutdown_connections 2023-07-21T18:34:25.341 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:25.306+0000 7fc617789700 1 --2- 172.21.15.39:0/1822303746 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fc5fc038800 0x7fc5fc03acb0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:34:25.341 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:25.306+0000 7fc617789700 1 --2- 172.21.15.39:0/1822303746 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6101044f0 0x7fc610195960 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:34:25.341 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:25.306+0000 7fc617789700 1 -- 172.21.15.39:0/1822303746 >> 172.21.15.39:0/1822303746 conn(0x7fc6100742b0 msgr2=0x7fc6100fcce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:34:25.341 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:25.306+0000 7fc617789700 1 -- 172.21.15.39:0/1822303746 shutdown_connections 2023-07-21T18:34:25.341 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:25.306+0000 7fc617789700 1 -- 172.21.15.39:0/1822303746 wait complete. 2023-07-21T18:34:25.709 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph orch host ls --format=json 2023-07-21T18:34:26.268 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:34:26.678 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:26 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:26.679 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:26 smithi039 ceph-mon[96489]: Added host smithi155 2023-07-21T18:34:26.679 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:26 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:34:26.679 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:26 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:27.522 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:27.486+0000 7ffa19b26700 1 -- 172.21.15.39:0/2899803614 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa14105630 msgr2=0x7ffa14105a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:34:27.522 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:27.486+0000 7ffa19b26700 1 --2- 172.21.15.39:0/2899803614 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa14105630 0x7ffa14105a00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ffa08009910 tx=0x7ffa08009c20 comp rx=0 tx=0).stop 2023-07-21T18:34:27.523 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:27.486+0000 7ffa19b26700 1 -- 172.21.15.39:0/2899803614 shutdown_connections 2023-07-21T18:34:27.523 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:27.486+0000 7ffa19b26700 1 --2- 172.21.15.39:0/2899803614 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa14105630 0x7ffa14105a00 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:34:27.523 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:27.486+0000 7ffa19b26700 1 -- 172.21.15.39:0/2899803614 >> 172.21.15.39:0/2899803614 conn(0x7ffa140fb130 msgr2=0x7ffa140fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:34:27.523 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:27.486+0000 7ffa19b26700 1 -- 172.21.15.39:0/2899803614 shutdown_connections 2023-07-21T18:34:27.523 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:27.486+0000 7ffa19b26700 1 -- 172.21.15.39:0/2899803614 wait complete. 2023-07-21T18:34:27.525 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:27.487+0000 7ffa19b26700 1 Processor -- start 2023-07-21T18:34:27.525 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:27.487+0000 7ffa19b26700 1 -- start start 2023-07-21T18:34:27.525 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:27.487+0000 7ffa19b26700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa14105630 0x7ffa14194970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:34:27.525 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:27.487+0000 7ffa19b26700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa14194eb0 con 0x7ffa14105630 2023-07-21T18:34:27.525 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:27.488+0000 7ffa137fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa14105630 0x7ffa14194970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:34:27.525 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:27.488+0000 7ffa137fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa14105630 0x7ffa14194970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54064/0 (socket says 172.21.15.39:54064) 2023-07-21T18:34:27.526 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:27.488+0000 7ffa137fe700 1 -- 172.21.15.39:0/2872335336 learned_addr learned my addr 172.21.15.39:0/2872335336 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:34:27.526 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:27.488+0000 7ffa137fe700 1 -- 172.21.15.39:0/2872335336 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa080095f0 con 0x7ffa14105630 2023-07-21T18:34:27.526 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:27.488+0000 7ffa137fe700 1 --2- 172.21.15.39:0/2872335336 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa14105630 0x7ffa14194970 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ffa08004f40 tx=0x7ffa08005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:34:27.526 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:27.488+0000 7ffa10ff9700 1 -- 172.21.15.39:0/2872335336 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffa0801d070 con 0x7ffa14105630 2023-07-21T18:34:27.526 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:27.488+0000 7ffa19b26700 1 -- 172.21.15.39:0/2872335336 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa14195110 con 0x7ffa14105630 2023-07-21T18:34:27.526 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:27.488+0000 7ffa10ff9700 1 -- 172.21.15.39:0/2872335336 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa08004370 con 0x7ffa14105630 2023-07-21T18:34:27.526 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:27.488+0000 7ffa10ff9700 1 -- 172.21.15.39:0/2872335336 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffa080039b0 con 0x7ffa14105630 2023-07-21T18:34:27.527 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:27.488+0000 7ffa19b26700 1 -- 172.21.15.39:0/2872335336 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa14195610 con 0x7ffa14105630 2023-07-21T18:34:27.527 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:27.489+0000 7ffa10ff9700 1 -- 172.21.15.39:0/2872335336 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7ffa08003c70 con 0x7ffa14105630 2023-07-21T18:34:27.527 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:27.489+0000 7ffa10ff9700 1 --2- 172.21.15.39:0/2872335336 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7ff9fc0387b0 0x7ff9fc03ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:34:27.527 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:27.489+0000 7ffa19b26700 1 -- 172.21.15.39:0/2872335336 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa14061b40 con 0x7ffa14105630 2023-07-21T18:34:27.527 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:27.489+0000 7ffa10ff9700 1 -- 172.21.15.39:0/2872335336 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ffa0804c500 con 0x7ffa14105630 2023-07-21T18:34:27.527 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:27.489+0000 7ffa12ffd700 1 --2- 172.21.15.39:0/2872335336 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7ff9fc0387b0 0x7ff9fc03ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:34:27.528 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:27.490+0000 7ffa12ffd700 1 --2- 172.21.15.39:0/2872335336 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7ff9fc0387b0 0x7ff9fc03ac60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ffa04006fd0 tx=0x7ffa04006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:34:27.529 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:27.494+0000 7ffa10ff9700 1 -- 172.21.15.39:0/2872335336 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7ffa0802aa00 con 0x7ffa14105630 2023-07-21T18:34:27.700 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:27 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:27.700 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:27 smithi039 ceph-mon[96489]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:34:27.731 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:27.695+0000 7ffa19b26700 1 -- 172.21.15.39:0/2872335336 --> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ffa14104320 con 0x7ff9fc0387b0 2023-07-21T18:34:27.731 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:27.696+0000 7ffa10ff9700 1 -- 172.21.15.39:0/2872335336 <== mgr.14152 v2:172.21.15.39:6800/2116291712 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+160 (secure 0 0 0) 0x7ffa14104320 con 0x7ff9fc0387b0 2023-07-21T18:34:27.732 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:34:27.732 INFO:teuthology.orchestra.run.smithi039.stdout:[{"addr": "172.21.15.39", "hostname": "smithi039", "labels": [], "status": ""}, {"addr": "172.21.15.155", "hostname": "smithi155", "labels": [], "status": ""}] 2023-07-21T18:34:27.734 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:27.698+0000 7ffa19b26700 1 -- 172.21.15.39:0/2872335336 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7ff9fc0387b0 msgr2=0x7ff9fc03ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:34:27.734 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:27.698+0000 7ffa19b26700 1 --2- 172.21.15.39:0/2872335336 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7ff9fc0387b0 0x7ff9fc03ac60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ffa04006fd0 tx=0x7ffa04006e40 comp rx=0 tx=0).stop 2023-07-21T18:34:27.734 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:27.698+0000 7ffa19b26700 1 -- 172.21.15.39:0/2872335336 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa14105630 msgr2=0x7ffa14194970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:34:27.734 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:27.698+0000 7ffa19b26700 1 --2- 172.21.15.39:0/2872335336 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa14105630 0x7ffa14194970 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ffa08004f40 tx=0x7ffa08005020 comp rx=0 tx=0).stop 2023-07-21T18:34:27.734 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:27.698+0000 7ffa19b26700 1 -- 172.21.15.39:0/2872335336 shutdown_connections 2023-07-21T18:34:27.735 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:27.698+0000 7ffa19b26700 1 --2- 172.21.15.39:0/2872335336 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7ff9fc0387b0 0x7ff9fc03ac60 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:34:27.735 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:27.698+0000 7ffa19b26700 1 --2- 172.21.15.39:0/2872335336 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa14105630 0x7ffa14194970 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:34:27.735 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:27.698+0000 7ffa19b26700 1 -- 172.21.15.39:0/2872335336 >> 172.21.15.39:0/2872335336 conn(0x7ffa140fb130 msgr2=0x7ffa140fbd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:34:27.735 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:27.698+0000 7ffa19b26700 1 -- 172.21.15.39:0/2872335336 shutdown_connections 2023-07-21T18:34:27.735 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:27.698+0000 7ffa19b26700 1 -- 172.21.15.39:0/2872335336 wait complete. 2023-07-21T18:34:28.127 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi193 2023-07-21T18:34:28.127 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2023-07-21T18:34:28.127 DEBUG:teuthology.orchestra.run.smithi193:> dd of=/etc/ceph/ceph.conf 2023-07-21T18:34:28.148 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2023-07-21T18:34:28.148 DEBUG:teuthology.orchestra.run.smithi193:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-21T18:34:28.207 INFO:tasks.cephadm:Adding host smithi193 to orchestrator... 2023-07-21T18:34:28.208 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph orch host add smithi193 2023-07-21T18:34:28.766 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:34:28.971 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:28 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:28.972 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:28 smithi039 ceph-mon[96489]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T18:34:29.913 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:29.874+0000 7fd9aef55700 1 -- 172.21.15.39:0/1487742282 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9a81044f0 msgr2=0x7fd9a81048c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:34:29.913 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:29.874+0000 7fd9aef55700 1 --2- 172.21.15.39:0/1487742282 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9a81044f0 0x7fd9a81048c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd99c009910 tx=0x7fd99c009c20 comp rx=0 tx=0).stop 2023-07-21T18:34:29.913 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:29.874+0000 7fd9aef55700 1 -- 172.21.15.39:0/1487742282 shutdown_connections 2023-07-21T18:34:29.914 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:29.874+0000 7fd9aef55700 1 --2- 172.21.15.39:0/1487742282 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9a81044f0 0x7fd9a81048c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:34:29.914 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:29.874+0000 7fd9aef55700 1 -- 172.21.15.39:0/1487742282 >> 172.21.15.39:0/1487742282 conn(0x7fd9a80fa010 msgr2=0x7fd9a80fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:34:29.914 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:29.874+0000 7fd9aef55700 1 -- 172.21.15.39:0/1487742282 shutdown_connections 2023-07-21T18:34:29.915 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:29.874+0000 7fd9aef55700 1 -- 172.21.15.39:0/1487742282 wait complete. 2023-07-21T18:34:29.915 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:29.875+0000 7fd9aef55700 1 Processor -- start 2023-07-21T18:34:29.915 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:29.875+0000 7fd9aef55700 1 -- start start 2023-07-21T18:34:29.916 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:29.875+0000 7fd9aef55700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9a81044f0 0x7fd9a80fa4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:34:29.916 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:29.875+0000 7fd9aef55700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a80fc430 con 0x7fd9a81044f0 2023-07-21T18:34:29.917 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:29.875+0000 7fd9accf1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9a81044f0 0x7fd9a80fa4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:34:29.917 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:29.875+0000 7fd9accf1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9a81044f0 0x7fd9a80fa4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54070/0 (socket says 172.21.15.39:54070) 2023-07-21T18:34:29.917 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:29.875+0000 7fd9accf1700 1 -- 172.21.15.39:0/95663369 learned_addr learned my addr 172.21.15.39:0/95663369 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:34:29.918 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:29.875+0000 7fd9accf1700 1 -- 172.21.15.39:0/95663369 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd99c0095f0 con 0x7fd9a81044f0 2023-07-21T18:34:29.918 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:29.875+0000 7fd9accf1700 1 --2- 172.21.15.39:0/95663369 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9a81044f0 0x7fd9a80fa4b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd99c004f40 tx=0x7fd99c005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:34:29.918 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:29.875+0000 7fd9a5ffb700 1 -- 172.21.15.39:0/95663369 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd99c01d070 con 0x7fd9a81044f0 2023-07-21T18:34:29.918 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:29.875+0000 7fd9a5ffb700 1 -- 172.21.15.39:0/95663369 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd99c004370 con 0x7fd9a81044f0 2023-07-21T18:34:29.919 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:29.875+0000 7fd9aef55700 1 -- 172.21.15.39:0/95663369 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9a80fa9f0 con 0x7fd9a81044f0 2023-07-21T18:34:29.919 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:29.875+0000 7fd9a5ffb700 1 -- 172.21.15.39:0/95663369 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd99c0039b0 con 0x7fd9a81044f0 2023-07-21T18:34:29.919 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:29.876+0000 7fd9aef55700 1 -- 172.21.15.39:0/95663369 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9a80fae70 con 0x7fd9a81044f0 2023-07-21T18:34:29.919 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:29.876+0000 7fd9a5ffb700 1 -- 172.21.15.39:0/95663369 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fd99c003c70 con 0x7fd9a81044f0 2023-07-21T18:34:29.920 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:29.877+0000 7fd9a5ffb700 1 --2- 172.21.15.39:0/95663369 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fd9980387b0 0x7fd99803ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:34:29.920 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:29.877+0000 7fd9a5ffb700 1 -- 172.21.15.39:0/95663369 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd99c04c560 con 0x7fd9a81044f0 2023-07-21T18:34:29.920 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:29.877+0000 7fd9aef55700 1 -- 172.21.15.39:0/95663369 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9a8061b40 con 0x7fd9a81044f0 2023-07-21T18:34:29.920 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:29.877+0000 7fd9a7fff700 1 --2- 172.21.15.39:0/95663369 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fd9980387b0 0x7fd99803ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:34:29.921 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:29.877+0000 7fd9a7fff700 1 --2- 172.21.15.39:0/95663369 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fd9980387b0 0x7fd99803ac60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd994006fd0 tx=0x7fd994006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:34:29.922 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:29.881+0000 7fd9a5ffb700 1 -- 172.21.15.39:0/95663369 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fd99c026020 con 0x7fd9a81044f0 2023-07-21T18:34:30.124 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:30.088+0000 7fd9aef55700 1 -- 172.21.15.39:0/95663369 --> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi193", "target": ["mon-mgr", ""]}) v1 -- 0x7fd9a80fb150 con 0x7fd9980387b0 2023-07-21T18:34:30.200 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:29 smithi039 ceph-mon[96489]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:34:31.450 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:31 smithi039 ceph-mon[96489]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi193", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:34:31.450 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:31 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:31.450 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:31 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:31.450 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:31 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:31.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:31 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:31.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:31 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi155", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:34:31.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:31 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:34:31.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:31 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:34:31.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:31 smithi039 ceph-mon[96489]: Updating smithi155:/etc/ceph/ceph.conf 2023-07-21T18:34:31.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:31 smithi039 ceph-mon[96489]: Updating smithi155:/var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T18:34:31.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:31 smithi039 ceph-mon[96489]: Deploying cephadm binary to smithi193 2023-07-21T18:34:31.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:31 smithi039 ceph-mon[96489]: Updating smithi155:/etc/ceph/ceph.client.admin.keyring 2023-07-21T18:34:31.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:31 smithi039 ceph-mon[96489]: Updating smithi155:/var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-21T18:34:31.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:31 smithi039 ceph-mon[96489]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:34:32.879 INFO:teuthology.orchestra.run.smithi039.stdout:Added host 'smithi193' with addr '172.21.15.193' 2023-07-21T18:34:32.880 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:32.843+0000 7fd9a5ffb700 1 -- 172.21.15.39:0/95663369 <== mgr.14152 v2:172.21.15.39:6800/2116291712 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fd9a80fb150 con 0x7fd9980387b0 2023-07-21T18:34:32.881 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:32.845+0000 7fd9aef55700 1 -- 172.21.15.39:0/95663369 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fd9980387b0 msgr2=0x7fd99803ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:34:32.881 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:32.845+0000 7fd9aef55700 1 --2- 172.21.15.39:0/95663369 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fd9980387b0 0x7fd99803ac60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd994006fd0 tx=0x7fd994006e40 comp rx=0 tx=0).stop 2023-07-21T18:34:32.881 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:32.845+0000 7fd9aef55700 1 -- 172.21.15.39:0/95663369 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9a81044f0 msgr2=0x7fd9a80fa4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:34:32.881 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:32.845+0000 7fd9aef55700 1 --2- 172.21.15.39:0/95663369 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9a81044f0 0x7fd9a80fa4b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd99c004f40 tx=0x7fd99c005020 comp rx=0 tx=0).stop 2023-07-21T18:34:32.881 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:32.845+0000 7fd9aef55700 1 -- 172.21.15.39:0/95663369 shutdown_connections 2023-07-21T18:34:32.882 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:32.845+0000 7fd9aef55700 1 --2- 172.21.15.39:0/95663369 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fd9980387b0 0x7fd99803ac60 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:34:32.882 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:32.845+0000 7fd9aef55700 1 --2- 172.21.15.39:0/95663369 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9a81044f0 0x7fd9a80fa4b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:34:32.882 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:32.845+0000 7fd9aef55700 1 -- 172.21.15.39:0/95663369 >> 172.21.15.39:0/95663369 conn(0x7fd9a80fa010 msgr2=0x7fd9a81068a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:34:32.882 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:32.845+0000 7fd9aef55700 1 -- 172.21.15.39:0/95663369 shutdown_connections 2023-07-21T18:34:32.882 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:32.845+0000 7fd9aef55700 1 -- 172.21.15.39:0/95663369 wait complete. 2023-07-21T18:34:33.217 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph orch host ls --format=json 2023-07-21T18:34:33.700 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:33 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:33.700 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:33 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:33.701 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:33 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:33.701 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:33 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:33.701 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:33 smithi039 ceph-mon[96489]: Added host smithi193 2023-07-21T18:34:33.701 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:33 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:34:33.701 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:33 smithi039 ceph-mon[96489]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:34:33.701 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:33 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:33.780 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:34:34.866 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:34.830+0000 7f5efb98f700 1 -- 172.21.15.39:0/419309558 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ef41044f0 msgr2=0x7f5ef41048c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:34:34.867 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:34.830+0000 7f5efb98f700 1 --2- 172.21.15.39:0/419309558 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ef41044f0 0x7f5ef41048c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8009910 tx=0x7f5ee8009c20 comp rx=0 tx=0).stop 2023-07-21T18:34:34.867 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:34.830+0000 7f5efb98f700 1 -- 172.21.15.39:0/419309558 shutdown_connections 2023-07-21T18:34:34.867 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:34.830+0000 7f5efb98f700 1 --2- 172.21.15.39:0/419309558 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ef41044f0 0x7f5ef41048c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:34:34.867 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:34.830+0000 7f5efb98f700 1 -- 172.21.15.39:0/419309558 >> 172.21.15.39:0/419309558 conn(0x7f5ef40fa010 msgr2=0x7f5ef40fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:34:34.867 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:34.830+0000 7f5efb98f700 1 -- 172.21.15.39:0/419309558 shutdown_connections 2023-07-21T18:34:34.868 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:34.830+0000 7f5efb98f700 1 -- 172.21.15.39:0/419309558 wait complete. 2023-07-21T18:34:34.868 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:34.831+0000 7f5efb98f700 1 Processor -- start 2023-07-21T18:34:34.868 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:34.831+0000 7f5efb98f700 1 -- start start 2023-07-21T18:34:34.869 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:34.831+0000 7f5efb98f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ef41044f0 0x7f5ef4195940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:34:34.869 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:34.831+0000 7f5efb98f700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef4195e80 con 0x7f5ef41044f0 2023-07-21T18:34:34.870 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:34.831+0000 7f5ef972b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ef41044f0 0x7f5ef4195940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:34:34.870 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:34.831+0000 7f5ef972b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ef41044f0 0x7f5ef4195940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54096/0 (socket says 172.21.15.39:54096) 2023-07-21T18:34:34.870 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:34.831+0000 7f5ef972b700 1 -- 172.21.15.39:0/3971079061 learned_addr learned my addr 172.21.15.39:0/3971079061 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:34:34.870 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:34.831+0000 7f5ef972b700 1 -- 172.21.15.39:0/3971079061 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ee80095f0 con 0x7f5ef41044f0 2023-07-21T18:34:34.870 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:34.831+0000 7f5ef972b700 1 --2- 172.21.15.39:0/3971079061 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ef41044f0 0x7f5ef4195940 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8004f40 tx=0x7f5ee8005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:34:34.870 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:34.832+0000 7f5ee67fc700 1 -- 172.21.15.39:0/3971079061 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5ee801d070 con 0x7f5ef41044f0 2023-07-21T18:34:34.870 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:34.832+0000 7f5efb98f700 1 -- 172.21.15.39:0/3971079061 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ef4196080 con 0x7f5ef41044f0 2023-07-21T18:34:34.871 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:34.832+0000 7f5ee67fc700 1 -- 172.21.15.39:0/3971079061 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ee8004370 con 0x7f5ef41044f0 2023-07-21T18:34:34.871 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:34.832+0000 7f5ee67fc700 1 -- 172.21.15.39:0/3971079061 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5ee80039b0 con 0x7f5ef41044f0 2023-07-21T18:34:34.871 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:34.832+0000 7f5efb98f700 1 -- 172.21.15.39:0/3971079061 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ef41964a0 con 0x7f5ef41044f0 2023-07-21T18:34:34.871 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:34.833+0000 7f5ee67fc700 1 -- 172.21.15.39:0/3971079061 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f5ee8003c70 con 0x7f5ef41044f0 2023-07-21T18:34:34.871 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:34.833+0000 7f5efb98f700 1 -- 172.21.15.39:0/3971079061 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ef41027a0 con 0x7f5ef41044f0 2023-07-21T18:34:34.871 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:34.833+0000 7f5ee67fc700 1 --2- 172.21.15.39:0/3971079061 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f5ee00387b0 0x7f5ee003ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:34:34.871 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:34.833+0000 7f5ee67fc700 1 -- 172.21.15.39:0/3971079061 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f5ee804c520 con 0x7f5ef41044f0 2023-07-21T18:34:34.871 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:34.833+0000 7f5ef8f2a700 1 --2- 172.21.15.39:0/3971079061 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f5ee00387b0 0x7f5ee003ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:34:34.872 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:34.833+0000 7f5ef8f2a700 1 --2- 172.21.15.39:0/3971079061 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f5ee00387b0 0x7f5ee003ac60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0006fd0 tx=0x7f5ef0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:34:34.873 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:34.838+0000 7f5ee67fc700 1 -- 172.21.15.39:0/3971079061 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f5ee802a980 con 0x7f5ef41044f0 2023-07-21T18:34:34.950 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:34 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:34.950 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:34 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:35.076 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:35.041+0000 7f5efb98f700 1 -- 172.21.15.39:0/3971079061 --> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5ef40685b0 con 0x7f5ee00387b0 2023-07-21T18:34:35.077 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:35.042+0000 7f5ee67fc700 1 -- 172.21.15.39:0/3971079061 <== mgr.14152 v2:172.21.15.39:6800/2116291712 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+240 (secure 0 0 0) 0x7f5ef40685b0 con 0x7f5ee00387b0 2023-07-21T18:34:35.077 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:34:35.078 INFO:teuthology.orchestra.run.smithi039.stdout:[{"addr": "172.21.15.39", "hostname": "smithi039", "labels": [], "status": ""}, {"addr": "172.21.15.155", "hostname": "smithi155", "labels": [], "status": ""}, {"addr": "172.21.15.193", "hostname": "smithi193", "labels": [], "status": ""}] 2023-07-21T18:34:35.080 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:35.044+0000 7f5efb98f700 1 -- 172.21.15.39:0/3971079061 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f5ee00387b0 msgr2=0x7f5ee003ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:34:35.080 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:35.044+0000 7f5efb98f700 1 --2- 172.21.15.39:0/3971079061 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f5ee00387b0 0x7f5ee003ac60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0006fd0 tx=0x7f5ef0006e40 comp rx=0 tx=0).stop 2023-07-21T18:34:35.080 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:35.044+0000 7f5efb98f700 1 -- 172.21.15.39:0/3971079061 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ef41044f0 msgr2=0x7f5ef4195940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:34:35.080 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:35.044+0000 7f5efb98f700 1 --2- 172.21.15.39:0/3971079061 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ef41044f0 0x7f5ef4195940 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8004f40 tx=0x7f5ee8005020 comp rx=0 tx=0).stop 2023-07-21T18:34:35.080 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:35.044+0000 7f5efb98f700 1 -- 172.21.15.39:0/3971079061 shutdown_connections 2023-07-21T18:34:35.080 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:35.044+0000 7f5efb98f700 1 --2- 172.21.15.39:0/3971079061 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f5ee00387b0 0x7f5ee003ac60 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:34:35.081 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:35.044+0000 7f5efb98f700 1 --2- 172.21.15.39:0/3971079061 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ef41044f0 0x7f5ef4195940 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:34:35.081 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:35.044+0000 7f5efb98f700 1 -- 172.21.15.39:0/3971079061 >> 172.21.15.39:0/3971079061 conn(0x7f5ef40fa010 msgr2=0x7f5ef4106850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:34:35.081 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:35.044+0000 7f5efb98f700 1 -- 172.21.15.39:0/3971079061 shutdown_connections 2023-07-21T18:34:35.081 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:35.044+0000 7f5efb98f700 1 -- 172.21.15.39:0/3971079061 wait complete. 2023-07-21T18:34:35.441 INFO:tasks.cephadm:Setting crush tunables to default 2023-07-21T18:34:35.442 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph osd crush tunables default 2023-07-21T18:34:36.000 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:34:36.200 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:35 smithi039 ceph-mon[96489]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T18:34:36.200 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:35 smithi039 ceph-mon[96489]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:34:37.168 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:37.132+0000 7f61ec5ff700 1 -- 172.21.15.39:0/3696968860 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61e4072870 msgr2=0x7f61e4072c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:34:37.169 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:37.132+0000 7f61ec5ff700 1 --2- 172.21.15.39:0/3696968860 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61e4072870 0x7f61e4072c40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f61e0009910 tx=0x7f61e0009c20 comp rx=0 tx=0).stop 2023-07-21T18:34:37.169 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:37.133+0000 7f61ec5ff700 1 -- 172.21.15.39:0/3696968860 shutdown_connections 2023-07-21T18:34:37.169 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:37.133+0000 7f61ec5ff700 1 --2- 172.21.15.39:0/3696968860 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61e4072870 0x7f61e4072c40 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:34:37.169 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:37.133+0000 7f61ec5ff700 1 -- 172.21.15.39:0/3696968860 >> 172.21.15.39:0/3696968860 conn(0x7f61e40fb0f0 msgr2=0x7f61e40fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:34:37.169 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:37.133+0000 7f61ec5ff700 1 -- 172.21.15.39:0/3696968860 shutdown_connections 2023-07-21T18:34:37.170 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:37.133+0000 7f61ec5ff700 1 -- 172.21.15.39:0/3696968860 wait complete. 2023-07-21T18:34:37.171 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:37.133+0000 7f61ec5ff700 1 Processor -- start 2023-07-21T18:34:37.171 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:37.133+0000 7f61ec5ff700 1 -- start start 2023-07-21T18:34:37.171 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:37.133+0000 7f61ec5ff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61e4072870 0x7f61e4196ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:34:37.172 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:37.134+0000 7f61ec5ff700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e41971e0 con 0x7f61e4072870 2023-07-21T18:34:37.172 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:37.134+0000 7f61ea39b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61e4072870 0x7f61e4196ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:34:37.172 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:37.134+0000 7f61ea39b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61e4072870 0x7f61e4196ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54102/0 (socket says 172.21.15.39:54102) 2023-07-21T18:34:37.172 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:37.134+0000 7f61ea39b700 1 -- 172.21.15.39:0/1251732569 learned_addr learned my addr 172.21.15.39:0/1251732569 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:34:37.172 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:37.134+0000 7f61ea39b700 1 -- 172.21.15.39:0/1251732569 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61e00095f0 con 0x7f61e4072870 2023-07-21T18:34:37.172 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:37.134+0000 7f61ea39b700 1 --2- 172.21.15.39:0/1251732569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61e4072870 0x7f61e4196ca0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f61e0006010 tx=0x7f61e0004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:34:37.172 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:37.134+0000 7f61db7fe700 1 -- 172.21.15.39:0/1251732569 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f61e001d070 con 0x7f61e4072870 2023-07-21T18:34:37.173 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:37.134+0000 7f61db7fe700 1 -- 172.21.15.39:0/1251732569 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f61e00044a0 con 0x7f61e4072870 2023-07-21T18:34:37.173 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:37.134+0000 7f61ec5ff700 1 -- 172.21.15.39:0/1251732569 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61e41973e0 con 0x7f61e4072870 2023-07-21T18:34:37.173 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:37.134+0000 7f61db7fe700 1 -- 172.21.15.39:0/1251732569 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f61e0003a70 con 0x7f61e4072870 2023-07-21T18:34:37.173 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:37.134+0000 7f61ec5ff700 1 -- 172.21.15.39:0/1251732569 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61e4197800 con 0x7f61e4072870 2023-07-21T18:34:37.173 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:37.135+0000 7f61db7fe700 1 -- 172.21.15.39:0/1251732569 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f61e0003cb0 con 0x7f61e4072870 2023-07-21T18:34:37.173 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:37.135+0000 7f61db7fe700 1 --2- 172.21.15.39:0/1251732569 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f61d00387b0 0x7f61d003ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:34:37.173 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:37.135+0000 7f61db7fe700 1 -- 172.21.15.39:0/1251732569 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f61e004c5d0 con 0x7f61e4072870 2023-07-21T18:34:37.173 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:37.135+0000 7f61ec5ff700 1 -- 172.21.15.39:0/1251732569 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61e4061b40 con 0x7f61e4072870 2023-07-21T18:34:37.174 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:37.135+0000 7f61e9b9a700 1 --2- 172.21.15.39:0/1251732569 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f61d00387b0 0x7f61d003ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:34:37.174 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:37.136+0000 7f61e9b9a700 1 --2- 172.21.15.39:0/1251732569 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f61d00387b0 0x7f61d003ac60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f61d4006fd0 tx=0x7f61d4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:34:37.176 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:37.140+0000 7f61db7fe700 1 -- 172.21.15.39:0/1251732569 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f61e0026020 con 0x7f61e4072870 2023-07-21T18:34:37.381 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:37.346+0000 7f61ec5ff700 1 -- 172.21.15.39:0/1251732569 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f61e4108a70 con 0x7f61e4072870 2023-07-21T18:34:37.450 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:37 smithi039 ceph-mon[96489]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:34:38.255 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:38.219+0000 7f61db7fe700 1 -- 172.21.15.39:0/1251732569 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f61e000fd00 con 0x7f61e4072870 2023-07-21T18:34:38.257 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:38.221+0000 7f61ec5ff700 1 -- 172.21.15.39:0/1251732569 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f61d00387b0 msgr2=0x7f61d003ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:34:38.257 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:38.221+0000 7f61ec5ff700 1 --2- 172.21.15.39:0/1251732569 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f61d00387b0 0x7f61d003ac60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f61d4006fd0 tx=0x7f61d4006e40 comp rx=0 tx=0).stop 2023-07-21T18:34:38.257 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:38.221+0000 7f61ec5ff700 1 -- 172.21.15.39:0/1251732569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61e4072870 msgr2=0x7f61e4196ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:34:38.257 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:38.221+0000 7f61ec5ff700 1 --2- 172.21.15.39:0/1251732569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61e4072870 0x7f61e4196ca0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f61e0006010 tx=0x7f61e0004ff0 comp rx=0 tx=0).stop 2023-07-21T18:34:38.258 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:38.221+0000 7f61ec5ff700 1 -- 172.21.15.39:0/1251732569 shutdown_connections 2023-07-21T18:34:38.258 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:38.221+0000 7f61ec5ff700 1 --2- 172.21.15.39:0/1251732569 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f61d00387b0 0x7f61d003ac60 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:34:38.258 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:38.221+0000 7f61ec5ff700 1 --2- 172.21.15.39:0/1251732569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61e4072870 0x7f61e4196ca0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:34:38.258 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:38.221+0000 7f61ec5ff700 1 -- 172.21.15.39:0/1251732569 >> 172.21.15.39:0/1251732569 conn(0x7f61e40fb0f0 msgr2=0x7f61e4106170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:34:38.258 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:38.221+0000 7f61ec5ff700 1 -- 172.21.15.39:0/1251732569 shutdown_connections 2023-07-21T18:34:38.258 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:34:38.221+0000 7f61ec5ff700 1 -- 172.21.15.39:0/1251732569 wait complete. 2023-07-21T18:34:38.259 INFO:teuthology.orchestra.run.smithi039.stderr:adjusted tunables profile to default 2023-07-21T18:34:38.649 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:38 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:38.649 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:38 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:38.650 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:38 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:38.650 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:38 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:38.650 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:38 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi193", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:34:38.651 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:38 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:34:38.651 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:38 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:34:38.651 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:38 smithi039 ceph-mon[96489]: Updating smithi193:/etc/ceph/ceph.conf 2023-07-21T18:34:38.652 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:38 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/1251732569' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-07-21T18:34:38.652 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:38 smithi039 ceph-mon[96489]: Updating smithi193:/var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T18:34:38.653 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:38 smithi039 ceph-mon[96489]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2023-07-21T18:34:38.653 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:38 smithi039 ceph-mon[96489]: Updating smithi193:/var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-21T18:34:38.653 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:38 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:38.654 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:38 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:38.654 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:38 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:38.668 INFO:tasks.cephadm:Adding mon.a on smithi039 2023-07-21T18:34:38.669 INFO:tasks.cephadm:Adding mon.b on smithi155 2023-07-21T18:34:38.669 INFO:tasks.cephadm:Adding mon.c on smithi193 2023-07-21T18:34:38.670 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph orch apply mon '3;smithi039:172.21.15.39=a;smithi155:172.21.15.155=b;smithi193:172.21.15.193=c' 2023-07-21T18:34:39.255 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T18:34:39.700 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:39 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/1251732569' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-07-21T18:34:39.700 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:39 smithi039 ceph-mon[96489]: osdmap e4: 0 total, 0 up, 0 in 2023-07-21T18:34:39.701 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:39 smithi039 ceph-mon[96489]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:34:40.550 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:40.513+0000 7fa30988f700 1 -- 172.21.15.193:0/1538906781 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3041044f0 msgr2=0x7fa3041048c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:34:40.551 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:40.513+0000 7fa30988f700 1 --2- 172.21.15.193:0/1538906781 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3041044f0 0x7fa3041048c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa2f8009910 tx=0x7fa2f8009c20 comp rx=0 tx=0).stop 2023-07-21T18:34:40.551 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:40.513+0000 7fa30988f700 1 -- 172.21.15.193:0/1538906781 shutdown_connections 2023-07-21T18:34:40.551 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:40.513+0000 7fa30988f700 1 --2- 172.21.15.193:0/1538906781 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3041044f0 0x7fa3041048c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:34:40.552 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:40.513+0000 7fa30988f700 1 -- 172.21.15.193:0/1538906781 >> 172.21.15.193:0/1538906781 conn(0x7fa3040fa010 msgr2=0x7fa3040fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:34:40.552 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:40.513+0000 7fa30988f700 1 -- 172.21.15.193:0/1538906781 shutdown_connections 2023-07-21T18:34:40.552 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:40.513+0000 7fa30988f700 1 -- 172.21.15.193:0/1538906781 wait complete. 2023-07-21T18:34:40.553 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:40.514+0000 7fa30988f700 1 Processor -- start 2023-07-21T18:34:40.553 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:40.514+0000 7fa30988f700 1 -- start start 2023-07-21T18:34:40.553 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:40.514+0000 7fa30988f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3041044f0 0x7fa3041959b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:34:40.554 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:40.514+0000 7fa30988f700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa304195ef0 con 0x7fa3041044f0 2023-07-21T18:34:40.554 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:40.514+0000 7fa302ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3041044f0 0x7fa3041959b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:34:40.554 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:40.514+0000 7fa302ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3041044f0 0x7fa3041959b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.193:60194/0 (socket says 172.21.15.193:60194) 2023-07-21T18:34:40.554 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:40.514+0000 7fa302ffd700 1 -- 172.21.15.193:0/1758382107 learned_addr learned my addr 172.21.15.193:0/1758382107 (peer_addr_for_me v2:172.21.15.193:0/0) 2023-07-21T18:34:40.555 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:40.515+0000 7fa302ffd700 1 -- 172.21.15.193:0/1758382107 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2f80095f0 con 0x7fa3041044f0 2023-07-21T18:34:40.555 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:40.515+0000 7fa302ffd700 1 --2- 172.21.15.193:0/1758382107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3041044f0 0x7fa3041959b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa2f8004f40 tx=0x7fa2f8005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:34:40.555 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:40.515+0000 7fa30888d700 1 -- 172.21.15.193:0/1758382107 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa2f801d070 con 0x7fa3041044f0 2023-07-21T18:34:40.556 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:40.515+0000 7fa30988f700 1 -- 172.21.15.193:0/1758382107 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa304196090 con 0x7fa3041044f0 2023-07-21T18:34:40.556 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:40.515+0000 7fa30888d700 1 -- 172.21.15.193:0/1758382107 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa2f8004370 con 0x7fa3041044f0 2023-07-21T18:34:40.556 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:40.515+0000 7fa30888d700 1 -- 172.21.15.193:0/1758382107 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa2f80039b0 con 0x7fa3041044f0 2023-07-21T18:34:40.556 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:40.515+0000 7fa30988f700 1 -- 172.21.15.193:0/1758382107 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3041964b0 con 0x7fa3041044f0 2023-07-21T18:34:40.557 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:40.516+0000 7fa30888d700 1 -- 172.21.15.193:0/1758382107 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fa2f8003c70 con 0x7fa3041044f0 2023-07-21T18:34:40.557 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:40.516+0000 7fa30988f700 1 -- 172.21.15.193:0/1758382107 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3041027a0 con 0x7fa3041044f0 2023-07-21T18:34:40.558 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:40.516+0000 7fa30888d700 1 --2- 172.21.15.193:0/1758382107 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fa2ec0387b0 0x7fa2ec03ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:34:40.558 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:40.516+0000 7fa30888d700 1 -- 172.21.15.193:0/1758382107 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa2f804c830 con 0x7fa3041044f0 2023-07-21T18:34:40.558 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:40.516+0000 7fa3027fc700 1 --2- 172.21.15.193:0/1758382107 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fa2ec0387b0 0x7fa2ec03ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:34:40.558 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:40.517+0000 7fa3027fc700 1 --2- 172.21.15.193:0/1758382107 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fa2ec0387b0 0x7fa2ec03ac60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4006fd0 tx=0x7fa2f4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:34:40.558 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:40.522+0000 7fa30888d700 1 -- 172.21.15.193:0/1758382107 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fa2f8029770 con 0x7fa3041044f0 2023-07-21T18:34:40.747 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:40.712+0000 7fa30988f700 1 -- 172.21.15.193:0/1758382107 --> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi039:172.21.15.39=a;smithi155:172.21.15.155=b;smithi193:172.21.15.193=c", "target": ["mon-mgr", ""]}) v1 -- 0x7fa304196760 con 0x7fa2ec0387b0 2023-07-21T18:34:40.760 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:40.725+0000 7fa30888d700 1 -- 172.21.15.193:0/1758382107 <== mgr.14152 v2:172.21.15.39:6800/2116291712 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fa304196760 con 0x7fa2ec0387b0 2023-07-21T18:34:40.760 INFO:teuthology.orchestra.run.smithi193.stdout:Scheduled mon update... 2023-07-21T18:34:40.761 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:40.726+0000 7fa30988f700 1 -- 172.21.15.193:0/1758382107 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fa2ec0387b0 msgr2=0x7fa2ec03ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:34:40.762 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:40.726+0000 7fa30988f700 1 --2- 172.21.15.193:0/1758382107 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fa2ec0387b0 0x7fa2ec03ac60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4006fd0 tx=0x7fa2f4006e40 comp rx=0 tx=0).stop 2023-07-21T18:34:40.762 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:40.726+0000 7fa30988f700 1 -- 172.21.15.193:0/1758382107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3041044f0 msgr2=0x7fa3041959b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:34:40.762 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:40.726+0000 7fa30988f700 1 --2- 172.21.15.193:0/1758382107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3041044f0 0x7fa3041959b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa2f8004f40 tx=0x7fa2f8005020 comp rx=0 tx=0).stop 2023-07-21T18:34:40.763 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:40.726+0000 7fa30988f700 1 -- 172.21.15.193:0/1758382107 shutdown_connections 2023-07-21T18:34:40.763 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:40.726+0000 7fa30988f700 1 --2- 172.21.15.193:0/1758382107 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fa2ec0387b0 0x7fa2ec03ac60 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:34:40.763 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:40.726+0000 7fa30988f700 1 --2- 172.21.15.193:0/1758382107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3041044f0 0x7fa3041959b0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:34:40.763 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:40.726+0000 7fa30988f700 1 -- 172.21.15.193:0/1758382107 >> 172.21.15.193:0/1758382107 conn(0x7fa3040fa010 msgr2=0x7fa304106900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:34:40.763 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:40.727+0000 7fa30988f700 1 -- 172.21.15.193:0/1758382107 shutdown_connections 2023-07-21T18:34:40.763 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:40.727+0000 7fa30988f700 1 -- 172.21.15.193:0/1758382107 wait complete. 2023-07-21T18:34:41.136 DEBUG:teuthology.orchestra.run.smithi155:mon.b> sudo journalctl -f -n 0 -u ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@mon.b.service 2023-07-21T18:34:41.138 DEBUG:teuthology.orchestra.run.smithi193:mon.c> sudo journalctl -f -n 0 -u ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@mon.c.service 2023-07-21T18:34:41.140 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-07-21T18:34:41.141 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph mon dump -f json 2023-07-21T18:34:41.169 INFO:journalctl@ceph.mon.c.smithi193.stdout:-- Logs begin at Fri 2023-07-21 18:17:09 UTC. -- 2023-07-21T18:34:41.174 INFO:journalctl@ceph.mon.b.smithi155.stdout:-- Logs begin at Fri 2023-07-21 18:17:11 UTC. -- 2023-07-21T18:34:41.770 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T18:34:42.200 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:41 smithi039 ceph-mon[96489]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi039:172.21.15.39=a;smithi155:172.21.15.155=b;smithi193:172.21.15.193=c", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:34:42.200 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:41 smithi039 ceph-mon[96489]: Saving service mon spec with placement smithi039:172.21.15.39=a;smithi155:172.21.15.155=b;smithi193:172.21.15.193=c;count:3 2023-07-21T18:34:42.200 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:41 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:42.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:41 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:34:42.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:41 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:34:42.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:41 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:34:42.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:41 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:42.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:41 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T18:34:42.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:41 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:34:42.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:41 smithi039 ceph-mon[96489]: Deploying daemon mon.c on smithi193 2023-07-21T18:34:42.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:41 smithi039 ceph-mon[96489]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:34:43.450 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:43 smithi039 ceph-mon[96489]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:34:43.688 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:43.652+0000 7f15bd043700 1 -- 172.21.15.193:0/618589258 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15b81044f0 msgr2=0x7f15b81048c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:34:43.689 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:43.652+0000 7f15bd043700 1 --2- 172.21.15.193:0/618589258 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15b81044f0 0x7f15b81048c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f15a8009910 tx=0x7f15a8009c20 comp rx=0 tx=0).stop 2023-07-21T18:34:43.689 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:43.652+0000 7f15bd043700 1 -- 172.21.15.193:0/618589258 shutdown_connections 2023-07-21T18:34:43.689 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:43.652+0000 7f15bd043700 1 --2- 172.21.15.193:0/618589258 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15b81044f0 0x7f15b81048c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:34:43.689 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:43.652+0000 7f15bd043700 1 -- 172.21.15.193:0/618589258 >> 172.21.15.193:0/618589258 conn(0x7f15b80fa010 msgr2=0x7f15b80fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:34:43.689 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:43.652+0000 7f15bd043700 1 -- 172.21.15.193:0/618589258 shutdown_connections 2023-07-21T18:34:43.689 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:43.652+0000 7f15bd043700 1 -- 172.21.15.193:0/618589258 wait complete. 2023-07-21T18:34:43.690 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:43.653+0000 7f15bd043700 1 Processor -- start 2023-07-21T18:34:43.690 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:43.653+0000 7f15bd043700 1 -- start start 2023-07-21T18:34:43.690 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:43.653+0000 7f15bd043700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15b81044f0 0x7f15b81959d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:34:43.690 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:43.653+0000 7f15bd043700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b8195f10 con 0x7f15b81044f0 2023-07-21T18:34:43.691 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:43.654+0000 7f15b6d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15b81044f0 0x7f15b81959d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:34:43.692 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:43.654+0000 7f15b6d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15b81044f0 0x7f15b81959d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.193:60200/0 (socket says 172.21.15.193:60200) 2023-07-21T18:34:43.692 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:43.654+0000 7f15b6d9d700 1 -- 172.21.15.193:0/2809422467 learned_addr learned my addr 172.21.15.193:0/2809422467 (peer_addr_for_me v2:172.21.15.193:0/0) 2023-07-21T18:34:43.692 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:43.654+0000 7f15b6d9d700 1 -- 172.21.15.193:0/2809422467 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a80095f0 con 0x7f15b81044f0 2023-07-21T18:34:43.692 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:43.654+0000 7f15b6d9d700 1 --2- 172.21.15.193:0/2809422467 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15b81044f0 0x7f15b81959d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f15a8004f40 tx=0x7f15a8005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:34:43.692 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:43.654+0000 7f159ffff700 1 -- 172.21.15.193:0/2809422467 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f15a801d070 con 0x7f15b81044f0 2023-07-21T18:34:43.692 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:43.654+0000 7f15bd043700 1 -- 172.21.15.193:0/2809422467 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15b8196110 con 0x7f15b81044f0 2023-07-21T18:34:43.693 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:43.654+0000 7f159ffff700 1 -- 172.21.15.193:0/2809422467 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15a8004370 con 0x7f15b81044f0 2023-07-21T18:34:43.693 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:43.654+0000 7f159ffff700 1 -- 172.21.15.193:0/2809422467 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f15a80039b0 con 0x7f15b81044f0 2023-07-21T18:34:43.693 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:43.654+0000 7f15bd043700 1 -- 172.21.15.193:0/2809422467 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15b8196530 con 0x7f15b81044f0 2023-07-21T18:34:43.693 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:43.655+0000 7f159ffff700 1 -- 172.21.15.193:0/2809422467 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f15a8003c70 con 0x7f15b81044f0 2023-07-21T18:34:43.693 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:43.655+0000 7f15bd043700 1 -- 172.21.15.193:0/2809422467 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15b81027a0 con 0x7f15b81044f0 2023-07-21T18:34:43.693 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:43.656+0000 7f159ffff700 1 --2- 172.21.15.193:0/2809422467 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f15a00387b0 0x7f15a003ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:34:43.693 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:43.656+0000 7f159ffff700 1 -- 172.21.15.193:0/2809422467 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f15a804d510 con 0x7f15b81044f0 2023-07-21T18:34:43.694 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:43.656+0000 7f15b659c700 1 --2- 172.21.15.193:0/2809422467 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f15a00387b0 0x7f15a003ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:34:43.694 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:43.656+0000 7f15b659c700 1 --2- 172.21.15.193:0/2809422467 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f15a00387b0 0x7f15a003ac60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f15ac006fd0 tx=0x7f15ac006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:34:43.696 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:43.661+0000 7f159ffff700 1 -- 172.21.15.193:0/2809422467 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f15a802b440 con 0x7f15b81044f0 2023-07-21T18:34:43.823 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 systemd[1]: Starting Ceph mon.c for eaf8e25a-27f4-11ee-9b34-001a4aab830c... 2023-07-21T18:34:43.968 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:43.932+0000 7f15bd043700 1 -- 172.21.15.193:0/2809422467 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f15b8061b40 con 0x7f15b81044f0 2023-07-21T18:34:43.968 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:43.933+0000 7f159ffff700 1 -- 172.21.15.193:0/2809422467 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+736 (secure 0 0 0) 0x7f15a8026070 con 0x7f15b81044f0 2023-07-21T18:34:43.968 INFO:teuthology.orchestra.run.smithi193.stdout: 2023-07-21T18:34:43.969 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":1,"fsid":"eaf8e25a-27f4-11ee-9b34-001a4aab830c","modified":"2023-07-21T18:33:02.398821Z","created":"2023-07-21T18:33:02.398821Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:3300","nonce":0},{"type":"v1","addr":"172.21.15.39:6789","nonce":0}]},"addr":"172.21.15.39:6789/0","public_addr":"172.21.15.39:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-07-21T18:34:43.970 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:43.935+0000 7f15bd043700 1 -- 172.21.15.193:0/2809422467 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f15a00387b0 msgr2=0x7f15a003ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:34:43.970 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:43.935+0000 7f15bd043700 1 --2- 172.21.15.193:0/2809422467 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f15a00387b0 0x7f15a003ac60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f15ac006fd0 tx=0x7f15ac006e40 comp rx=0 tx=0).stop 2023-07-21T18:34:43.971 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:43.935+0000 7f15bd043700 1 -- 172.21.15.193:0/2809422467 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15b81044f0 msgr2=0x7f15b81959d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:34:43.971 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:43.935+0000 7f15bd043700 1 --2- 172.21.15.193:0/2809422467 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15b81044f0 0x7f15b81959d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f15a8004f40 tx=0x7f15a8005020 comp rx=0 tx=0).stop 2023-07-21T18:34:43.971 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:43.935+0000 7f15bd043700 1 -- 172.21.15.193:0/2809422467 shutdown_connections 2023-07-21T18:34:43.971 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:43.935+0000 7f15bd043700 1 --2- 172.21.15.193:0/2809422467 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f15a00387b0 0x7f15a003ac60 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:34:43.971 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:43.935+0000 7f15bd043700 1 --2- 172.21.15.193:0/2809422467 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15b81044f0 0x7f15b81959d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:34:43.971 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:43.935+0000 7f15bd043700 1 -- 172.21.15.193:0/2809422467 >> 172.21.15.193:0/2809422467 conn(0x7f15b80fa010 msgr2=0x7f15b8106900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:34:43.971 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:43.935+0000 7f15bd043700 1 -- 172.21.15.193:0/2809422467 shutdown_connections 2023-07-21T18:34:43.972 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:43.935+0000 7f15bd043700 1 -- 172.21.15.193:0/2809422467 wait complete. 2023-07-21T18:34:43.972 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 1 2023-07-21T18:34:44.085 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 podman[98145]: 2023-07-21T18:34:44.085 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 bash[98145]: 5a7b2b6ffbc0d8c33ccbb28c24fb11c5c0deb6a8d041317426fc4058d142904b 2023-07-21T18:34:44.085 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: set uid:gid to 167:167 (ceph:ceph) 2023-07-21T18:34:44.086 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: ceph version 18.0.0-4746-g81cf785a (81cf785aa7781c4e08381c540f890cb9bc987ef9) reef (dev), process ceph-mon, pid 7 2023-07-21T18:34:44.086 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: pidfile_write: ignore empty --pid-file 2023-07-21T18:34:44.086 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: load: jerasure load: lrc 2023-07-21T18:34:44.086 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: RocksDB version: 7.9.2 2023-07-21T18:34:44.086 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Git sha 0 2023-07-21T18:34:44.086 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Compile date 2023-07-17 23:00:14 2023-07-21T18:34:44.086 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: DB SUMMARY 2023-07-21T18:34:44.087 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: DB Session ID: F9PY7B4JJTNIPX55S6GE 2023-07-21T18:34:44.087 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: CURRENT file: CURRENT 2023-07-21T18:34:44.087 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: IDENTITY file: IDENTITY 2023-07-21T18:34:44.087 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-07-21T18:34:44.087 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2023-07-21T18:34:44.087 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2023-07-21T18:34:44.088 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.error_if_exists: 0 2023-07-21T18:34:44.088 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.create_if_missing: 0 2023-07-21T18:34:44.088 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.paranoid_checks: 1 2023-07-21T18:34:44.089 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.flush_verify_memtable_count: 1 2023-07-21T18:34:44.089 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-07-21T18:34:44.089 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-07-21T18:34:44.090 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.env: 0x5597df7ec720 2023-07-21T18:34:44.090 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.fs: PosixFileSystem 2023-07-21T18:34:44.090 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.info_log: 0x5597e2075280 2023-07-21T18:34:44.090 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.max_file_opening_threads: 16 2023-07-21T18:34:44.091 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.statistics: (nil) 2023-07-21T18:34:44.091 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.use_fsync: 0 2023-07-21T18:34:44.091 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.max_log_file_size: 0 2023-07-21T18:34:44.092 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-07-21T18:34:44.092 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.log_file_time_to_roll: 0 2023-07-21T18:34:44.092 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.keep_log_file_num: 1000 2023-07-21T18:34:44.092 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.recycle_log_file_num: 0 2023-07-21T18:34:44.093 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.allow_fallocate: 1 2023-07-21T18:34:44.093 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.allow_mmap_reads: 0 2023-07-21T18:34:44.093 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.allow_mmap_writes: 0 2023-07-21T18:34:44.094 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.use_direct_reads: 0 2023-07-21T18:34:44.094 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-07-21T18:34:44.094 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.create_missing_column_families: 0 2023-07-21T18:34:44.094 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.db_log_dir: 2023-07-21T18:34:44.095 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.wal_dir: 2023-07-21T18:34:44.095 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.table_cache_numshardbits: 6 2023-07-21T18:34:44.095 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.WAL_ttl_seconds: 0 2023-07-21T18:34:44.096 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.WAL_size_limit_MB: 0 2023-07-21T18:34:44.096 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-07-21T18:34:44.096 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-07-21T18:34:44.096 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.is_fd_close_on_exec: 1 2023-07-21T18:34:44.096 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.advise_random_on_open: 1 2023-07-21T18:34:44.097 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.db_write_buffer_size: 0 2023-07-21T18:34:44.097 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.write_buffer_manager: 0x5597e1f963c0 2023-07-21T18:34:44.097 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-07-21T18:34:44.097 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-07-21T18:34:44.097 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.use_adaptive_mutex: 0 2023-07-21T18:34:44.097 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.rate_limiter: (nil) 2023-07-21T18:34:44.097 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-07-21T18:34:44.098 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.wal_recovery_mode: 2 2023-07-21T18:34:44.098 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.enable_thread_tracking: 0 2023-07-21T18:34:44.098 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.enable_pipelined_write: 0 2023-07-21T18:34:44.098 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.unordered_write: 0 2023-07-21T18:34:44.098 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-07-21T18:34:44.098 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-07-21T18:34:44.098 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-07-21T18:34:44.099 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-07-21T18:34:44.099 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.row_cache: None 2023-07-21T18:34:44.099 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.wal_filter: None 2023-07-21T18:34:44.099 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-07-21T18:34:44.099 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.allow_ingest_behind: 0 2023-07-21T18:34:44.099 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.two_write_queues: 0 2023-07-21T18:34:44.099 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.manual_wal_flush: 0 2023-07-21T18:34:44.100 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.wal_compression: 0 2023-07-21T18:34:44.100 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.atomic_flush: 0 2023-07-21T18:34:44.100 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-07-21T18:34:44.100 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.persist_stats_to_disk: 0 2023-07-21T18:34:44.100 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.write_dbid_to_manifest: 0 2023-07-21T18:34:44.100 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.log_readahead_size: 0 2023-07-21T18:34:44.101 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-07-21T18:34:44.101 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.best_efforts_recovery: 0 2023-07-21T18:34:44.101 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-07-21T18:34:44.101 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-07-21T18:34:44.101 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.allow_data_in_errors: 0 2023-07-21T18:34:44.101 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.db_host_id: __hostname__ 2023-07-21T18:34:44.101 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.enforce_single_del_contracts: true 2023-07-21T18:34:44.102 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.max_background_jobs: 2 2023-07-21T18:34:44.102 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.max_background_compactions: -1 2023-07-21T18:34:44.102 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.max_subcompactions: 1 2023-07-21T18:34:44.102 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-07-21T18:34:44.102 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-07-21T18:34:44.102 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.delayed_write_rate : 16777216 2023-07-21T18:34:44.102 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.max_total_wal_size: 0 2023-07-21T18:34:44.103 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-07-21T18:34:44.103 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.stats_dump_period_sec: 600 2023-07-21T18:34:44.103 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.stats_persist_period_sec: 600 2023-07-21T18:34:44.103 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-07-21T18:34:44.103 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.max_open_files: -1 2023-07-21T18:34:44.104 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.bytes_per_sync: 0 2023-07-21T18:34:44.104 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.wal_bytes_per_sync: 0 2023-07-21T18:34:44.104 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.strict_bytes_per_sync: 0 2023-07-21T18:34:44.104 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.compaction_readahead_size: 0 2023-07-21T18:34:44.105 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.max_background_flushes: -1 2023-07-21T18:34:44.105 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Compression algorithms supported: 2023-07-21T18:34:44.105 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-07-21T18:34:44.106 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: kZSTD supported: 0 2023-07-21T18:34:44.106 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: kXpressCompression supported: 0 2023-07-21T18:34:44.106 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: kLZ4HCCompression supported: 1 2023-07-21T18:34:44.106 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: kZlibCompression supported: 1 2023-07-21T18:34:44.107 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: kSnappyCompression supported: 1 2023-07-21T18:34:44.107 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: kLZ4Compression supported: 1 2023-07-21T18:34:44.107 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: kBZip2Compression supported: 0 2023-07-21T18:34:44.108 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Fast CRC32 supported: Supported on x86 2023-07-21T18:34:44.108 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: DMutex implementation: pthread_mutex_t 2023-07-21T18:34:44.108 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2023-07-21T18:34:44.109 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-07-21T18:34:44.109 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-07-21T18:34:44.109 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.merge_operator: 2023-07-21T18:34:44.110 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.compaction_filter: None 2023-07-21T18:34:44.110 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.compaction_filter_factory: None 2023-07-21T18:34:44.110 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.sst_partitioner_factory: None 2023-07-21T18:34:44.110 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.memtable_factory: SkipListFactory 2023-07-21T18:34:44.111 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.table_factory: BlockBasedTable 2023-07-21T18:34:44.111 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5597e2075380) 2023-07-21T18:34:44.111 INFO:journalctl@ceph.mon.c.smithi193.stdout: cache_index_and_filter_blocks: 1 2023-07-21T18:34:44.112 INFO:journalctl@ceph.mon.c.smithi193.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-07-21T18:34:44.112 INFO:journalctl@ceph.mon.c.smithi193.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-07-21T18:34:44.112 INFO:journalctl@ceph.mon.c.smithi193.stdout: pin_top_level_index_and_filter: 1 2023-07-21T18:34:44.112 INFO:journalctl@ceph.mon.c.smithi193.stdout: index_type: 0 2023-07-21T18:34:44.112 INFO:journalctl@ceph.mon.c.smithi193.stdout: data_block_index_type: 0 2023-07-21T18:34:44.113 INFO:journalctl@ceph.mon.c.smithi193.stdout: index_shortening: 1 2023-07-21T18:34:44.113 INFO:journalctl@ceph.mon.c.smithi193.stdout: data_block_hash_table_util_ratio: 0.750000 2023-07-21T18:34:44.113 INFO:journalctl@ceph.mon.c.smithi193.stdout: checksum: 4 2023-07-21T18:34:44.114 INFO:journalctl@ceph.mon.c.smithi193.stdout: no_block_cache: 0 2023-07-21T18:34:44.114 INFO:journalctl@ceph.mon.c.smithi193.stdout: block_cache: 0x5597e20351f0 2023-07-21T18:34:44.114 INFO:journalctl@ceph.mon.c.smithi193.stdout: block_cache_name: BinnedLRUCache 2023-07-21T18:34:44.114 INFO:journalctl@ceph.mon.c.smithi193.stdout: block_cache_options: 2023-07-21T18:34:44.115 INFO:journalctl@ceph.mon.c.smithi193.stdout: capacity : 536870912 2023-07-21T18:34:44.115 INFO:journalctl@ceph.mon.c.smithi193.stdout: num_shard_bits : 4 2023-07-21T18:34:44.115 INFO:journalctl@ceph.mon.c.smithi193.stdout: strict_capacity_limit : 0 2023-07-21T18:34:44.115 INFO:journalctl@ceph.mon.c.smithi193.stdout: high_pri_pool_ratio: 0.000 2023-07-21T18:34:44.115 INFO:journalctl@ceph.mon.c.smithi193.stdout: block_cache_compressed: (nil) 2023-07-21T18:34:44.115 INFO:journalctl@ceph.mon.c.smithi193.stdout: persistent_cache: (nil) 2023-07-21T18:34:44.116 INFO:journalctl@ceph.mon.c.smithi193.stdout: block_size: 4096 2023-07-21T18:34:44.116 INFO:journalctl@ceph.mon.c.smithi193.stdout: block_size_deviation: 10 2023-07-21T18:34:44.116 INFO:journalctl@ceph.mon.c.smithi193.stdout: block_restart_interval: 16 2023-07-21T18:34:44.117 INFO:journalctl@ceph.mon.c.smithi193.stdout: index_block_restart_interval: 1 2023-07-21T18:34:44.117 INFO:journalctl@ceph.mon.c.smithi193.stdout: metadata_block_size: 4096 2023-07-21T18:34:44.117 INFO:journalctl@ceph.mon.c.smithi193.stdout: partition_filters: 0 2023-07-21T18:34:44.117 INFO:journalctl@ceph.mon.c.smithi193.stdout: use_delta_encoding: 1 2023-07-21T18:34:44.118 INFO:journalctl@ceph.mon.c.smithi193.stdout: filter_policy: bloomfilter 2023-07-21T18:34:44.118 INFO:journalctl@ceph.mon.c.smithi193.stdout: whole_key_filtering: 1 2023-07-21T18:34:44.118 INFO:journalctl@ceph.mon.c.smithi193.stdout: verify_compression: 0 2023-07-21T18:34:44.119 INFO:journalctl@ceph.mon.c.smithi193.stdout: read_amp_bytes_per_bit: 0 2023-07-21T18:34:44.119 INFO:journalctl@ceph.mon.c.smithi193.stdout: format_version: 5 2023-07-21T18:34:44.119 INFO:journalctl@ceph.mon.c.smithi193.stdout: enable_index_compression: 1 2023-07-21T18:34:44.119 INFO:journalctl@ceph.mon.c.smithi193.stdout: block_align: 0 2023-07-21T18:34:44.120 INFO:journalctl@ceph.mon.c.smithi193.stdout: max_auto_readahead_size: 262144 2023-07-21T18:34:44.120 INFO:journalctl@ceph.mon.c.smithi193.stdout: prepopulate_block_cache: 0 2023-07-21T18:34:44.120 INFO:journalctl@ceph.mon.c.smithi193.stdout: initial_auto_readahead_size: 8192 2023-07-21T18:34:44.120 INFO:journalctl@ceph.mon.c.smithi193.stdout: num_file_reads_for_auto_readahead: 2 2023-07-21T18:34:44.121 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.write_buffer_size: 33554432 2023-07-21T18:34:44.121 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.max_write_buffer_number: 2 2023-07-21T18:34:44.121 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.compression: NoCompression 2023-07-21T18:34:44.122 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.bottommost_compression: Disabled 2023-07-21T18:34:44.122 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.prefix_extractor: nullptr 2023-07-21T18:34:44.122 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-07-21T18:34:44.122 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.num_levels: 7 2023-07-21T18:34:44.122 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-07-21T18:34:44.123 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-07-21T18:34:44.123 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-07-21T18:34:44.123 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-07-21T18:34:44.123 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-07-21T18:34:44.123 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-07-21T18:34:44.123 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-07-21T18:34:44.125 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-07-21T18:34:44.125 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-07-21T18:34:44.126 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-07-21T18:34:44.126 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-07-21T18:34:44.126 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-07-21T18:34:44.126 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.compression_opts.window_bits: -14 2023-07-21T18:34:44.126 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.compression_opts.level: 32767 2023-07-21T18:34:44.126 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.compression_opts.strategy: 0 2023-07-21T18:34:44.127 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-07-21T18:34:44.127 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-07-21T18:34:44.127 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-07-21T18:34:44.127 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-07-21T18:34:44.127 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.compression_opts.enabled: false 2023-07-21T18:34:44.128 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-07-21T18:34:44.128 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-07-21T18:34:44.129 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-07-21T18:34:44.129 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-07-21T18:34:44.129 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.target_file_size_base: 67108864 2023-07-21T18:34:44.129 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.target_file_size_multiplier: 1 2023-07-21T18:34:44.129 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-07-21T18:34:44.129 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-07-21T18:34:44.130 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-07-21T18:34:44.130 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-07-21T18:34:44.130 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-07-21T18:34:44.130 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-07-21T18:34:44.130 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-07-21T18:34:44.131 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-07-21T18:34:44.131 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-07-21T18:34:44.131 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-07-21T18:34:44.131 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-07-21T18:34:44.131 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-07-21T18:34:44.131 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-07-21T18:34:44.131 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.arena_block_size: 1048576 2023-07-21T18:34:44.132 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-07-21T18:34:44.132 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-07-21T18:34:44.132 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.disable_auto_compactions: 0 2023-07-21T18:34:44.132 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-07-21T18:34:44.132 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-07-21T18:34:44.133 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-07-21T18:34:44.133 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-07-21T18:34:44.133 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-07-21T18:34:44.133 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-07-21T18:34:44.133 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-07-21T18:34:44.134 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-07-21T18:34:44.134 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-07-21T18:34:44.134 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-07-21T18:34:44.134 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.table_properties_collectors: 2023-07-21T18:34:44.134 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.inplace_update_support: 0 2023-07-21T18:34:44.134 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.inplace_update_num_locks: 10000 2023-07-21T18:34:44.134 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-07-21T18:34:44.135 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-07-21T18:34:44.135 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.memtable_huge_page_size: 0 2023-07-21T18:34:44.135 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.bloom_locality: 0 2023-07-21T18:34:44.135 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.max_successive_merges: 0 2023-07-21T18:34:44.135 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.optimize_filters_for_hits: 0 2023-07-21T18:34:44.135 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.paranoid_file_checks: 0 2023-07-21T18:34:44.136 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.force_consistency_checks: 1 2023-07-21T18:34:44.136 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.report_bg_io_stats: 0 2023-07-21T18:34:44.136 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.ttl: 2592000 2023-07-21T18:34:44.136 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.periodic_compaction_seconds: 0 2023-07-21T18:34:44.136 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-07-21T18:34:44.137 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-07-21T18:34:44.137 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.enable_blob_files: false 2023-07-21T18:34:44.137 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.min_blob_size: 0 2023-07-21T18:34:44.137 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.blob_file_size: 268435456 2023-07-21T18:34:44.137 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.blob_compression_type: NoCompression 2023-07-21T18:34:44.137 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.enable_blob_garbage_collection: false 2023-07-21T18:34:44.137 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-07-21T18:34:44.138 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-07-21T18:34:44.138 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-07-21T18:34:44.138 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.blob_file_starting_level: 0 2023-07-21T18:34:44.138 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-07-21T18:34:44.138 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-07-21T18:34:44.138 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-07-21T18:34:44.138 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: fa25d0c8-260a-4ba6-8d05-138ca4d8b4a8 2023-07-21T18:34:44.139 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689964483952114, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-07-21T18:34:44.139 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-07-21T18:34:44.139 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689964483952680, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1689964483, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "fa25d0c8-260a-4ba6-8d05-138ca4d8b4a8", "db_session_id": "F9PY7B4JJTNIPX55S6GE", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-07-21T18:34:44.139 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689964483952770, "job": 1, "event": "recovery_finished"} 2023-07-21T18:34:44.140 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-07-21T18:34:44.140 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-07-21T18:34:44.140 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5597e20ce000 2023-07-21T18:34:44.140 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: DB pointer 0x5597e20ba000 2023-07-21T18:34:44.141 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-07-21T18:34:44.141 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-07-21T18:34:44.141 INFO:journalctl@ceph.mon.c.smithi193.stdout: ** DB Stats ** 2023-07-21T18:34:44.141 INFO:journalctl@ceph.mon.c.smithi193.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-21T18:34:44.141 INFO:journalctl@ceph.mon.c.smithi193.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-07-21T18:34:44.142 INFO:journalctl@ceph.mon.c.smithi193.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-21T18:34:44.142 INFO:journalctl@ceph.mon.c.smithi193.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-21T18:34:44.142 INFO:journalctl@ceph.mon.c.smithi193.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-07-21T18:34:44.142 INFO:journalctl@ceph.mon.c.smithi193.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-21T18:34:44.142 INFO:journalctl@ceph.mon.c.smithi193.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-21T18:34:44.142 INFO:journalctl@ceph.mon.c.smithi193.stdout: 2023-07-21T18:34:44.143 INFO:journalctl@ceph.mon.c.smithi193.stdout: ** Compaction Stats [default] ** 2023-07-21T18:34:44.143 INFO:journalctl@ceph.mon.c.smithi193.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-21T18:34:44.143 INFO:journalctl@ceph.mon.c.smithi193.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-07-21T18:34:44.143 INFO:journalctl@ceph.mon.c.smithi193.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-21T18:34:44.143 INFO:journalctl@ceph.mon.c.smithi193.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-21T18:34:44.143 INFO:journalctl@ceph.mon.c.smithi193.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-21T18:34:44.143 INFO:journalctl@ceph.mon.c.smithi193.stdout: 2023-07-21T18:34:44.144 INFO:journalctl@ceph.mon.c.smithi193.stdout: ** Compaction Stats [default] ** 2023-07-21T18:34:44.144 INFO:journalctl@ceph.mon.c.smithi193.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-21T18:34:44.144 INFO:journalctl@ceph.mon.c.smithi193.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-07-21T18:34:44.144 INFO:journalctl@ceph.mon.c.smithi193.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-21T18:34:44.144 INFO:journalctl@ceph.mon.c.smithi193.stdout: 2023-07-21T18:34:44.144 INFO:journalctl@ceph.mon.c.smithi193.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-07-21T18:34:44.145 INFO:journalctl@ceph.mon.c.smithi193.stdout: 2023-07-21T18:34:44.145 INFO:journalctl@ceph.mon.c.smithi193.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-21T18:34:44.145 INFO:journalctl@ceph.mon.c.smithi193.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-07-21T18:34:44.145 INFO:journalctl@ceph.mon.c.smithi193.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-07-21T18:34:44.145 INFO:journalctl@ceph.mon.c.smithi193.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-07-21T18:34:44.145 INFO:journalctl@ceph.mon.c.smithi193.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-07-21T18:34:44.145 INFO:journalctl@ceph.mon.c.smithi193.stdout: AddFile(Keys): cumulative 0, interval 0 2023-07-21T18:34:44.146 INFO:journalctl@ceph.mon.c.smithi193.stdout: Cumulative compaction: 0.00 GB write, 0.46 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-21T18:34:44.146 INFO:journalctl@ceph.mon.c.smithi193.stdout: Interval compaction: 0.00 GB write, 0.46 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-21T18:34:44.146 INFO:journalctl@ceph.mon.c.smithi193.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-07-21T18:34:44.146 INFO:journalctl@ceph.mon.c.smithi193.stdout: Block cache BinnedLRUCache@0x5597e20351f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2023-07-21T18:34:44.146 INFO:journalctl@ceph.mon.c.smithi193.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-07-21T18:34:44.146 INFO:journalctl@ceph.mon.c.smithi193.stdout: 2023-07-21T18:34:44.146 INFO:journalctl@ceph.mon.c.smithi193.stdout: ** File Read Latency Histogram By Level [default] ** 2023-07-21T18:34:44.147 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: mon.c does not exist in monmap, will attempt to join an existing cluster 2023-07-21T18:34:44.147 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: using public_addr v2:172.21.15.193:0/0 -> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] 2023-07-21T18:34:44.147 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: starting mon.c rank -1 at public addrs [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] at bind addrs [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c 2023-07-21T18:34:44.147 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: mon.c@-1(???) e0 preinit fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c 2023-07-21T18:34:44.147 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 systemd[1]: Started Ceph mon.c for eaf8e25a-27f4-11ee-9b34-001a4aab830c. 2023-07-21T18:34:44.148 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: mon.c@-1(synchronizing).mds e1 new map 2023-07-21T18:34:44.148 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: mon.c@-1(synchronizing).mds e1 print_map 2023-07-21T18:34:44.148 INFO:journalctl@ceph.mon.c.smithi193.stdout: e1 2023-07-21T18:34:44.148 INFO:journalctl@ceph.mon.c.smithi193.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-07-21T18:34:44.148 INFO:journalctl@ceph.mon.c.smithi193.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-21T18:34:44.148 INFO:journalctl@ceph.mon.c.smithi193.stdout: legacy client fscid: -1 2023-07-21T18:34:44.148 INFO:journalctl@ceph.mon.c.smithi193.stdout: 2023-07-21T18:34:44.149 INFO:journalctl@ceph.mon.c.smithi193.stdout: No filesystems configured 2023-07-21T18:34:44.149 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-07-21T18:34:44.149 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-07-21T18:34:44.149 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-07-21T18:34:44.149 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-07-21T18:34:44.149 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-07-21T18:34:44.149 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-07-21T18:34:44.150 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-07-21T18:34:44.150 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-21T18:34:44.150 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-21T18:34:44.151 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-21T18:34:44.151 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:44.152 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:44.152 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T18:34:44.152 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:34:44.152 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:34:44.152 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:44.153 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:44.153 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:44.153 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:44.153 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi193", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:34:44.153 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:34:44.153 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:34:44.153 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: Updating smithi193:/etc/ceph/ceph.conf 2023-07-21T18:34:44.154 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/1251732569' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-07-21T18:34:44.154 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: Updating smithi193:/var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T18:34:44.154 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2023-07-21T18:34:44.154 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: Updating smithi193:/var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-21T18:34:44.154 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:44.154 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:44.155 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:44.155 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/1251732569' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-07-21T18:34:44.155 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: osdmap e4: 0 total, 0 up, 0 in 2023-07-21T18:34:44.155 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:34:44.156 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi039:172.21.15.39=a;smithi155:172.21.15.155=b;smithi193:172.21.15.193=c", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:34:44.156 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: Saving service mon spec with placement smithi039:172.21.15.39=a;smithi155:172.21.15.155=b;smithi193:172.21.15.193=c;count:3 2023-07-21T18:34:44.156 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:44.156 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:34:44.156 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:34:44.157 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:34:44.157 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:44.157 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T18:34:44.157 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:34:44.157 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: Deploying daemon mon.c on smithi193 2023-07-21T18:34:44.158 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:34:44.158 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:34:44.158 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-07-21T18:34:44.158 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:43 smithi193 ceph-mon[98204]: mon.c@-1(synchronizing) e1 handle_conf_change mon_allow_pool_delete 2023-07-21T18:34:45.403 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-07-21T18:34:45.403 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph mon dump -f json 2023-07-21T18:34:45.966 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T18:34:49.382 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:49 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T18:34:49.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:49 smithi193 ceph-mon[98204]: mon.a calling monitor election 2023-07-21T18:34:49.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:49 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T18:34:49.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:49 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T18:34:49.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:49 smithi193 ceph-mon[98204]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:34:49.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:49 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T18:34:49.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:49 smithi193 ceph-mon[98204]: mon.c calling monitor election 2023-07-21T18:34:49.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:49 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T18:34:49.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:49 smithi193 ceph-mon[98204]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:34:49.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:49 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T18:34:49.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:49 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T18:34:49.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:49 smithi193 ceph-mon[98204]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-21T18:34:49.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:49 smithi193 ceph-mon[98204]: monmap e2: 2 mons at {a=[v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0],c=[v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0]} removed_ranks: {} 2023-07-21T18:34:49.385 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:49 smithi193 ceph-mon[98204]: fsmap 2023-07-21T18:34:49.385 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:49 smithi193 ceph-mon[98204]: osdmap e4: 0 total, 0 up, 0 in 2023-07-21T18:34:49.385 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:49 smithi193 ceph-mon[98204]: mgrmap e13: x(active, since 41s) 2023-07-21T18:34:49.385 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:49 smithi193 ceph-mon[98204]: overall HEALTH_OK 2023-07-21T18:34:49.385 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:49 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:49.385 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:49 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:49.450 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:49 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T18:34:49.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:49 smithi039 ceph-mon[96489]: mon.a calling monitor election 2023-07-21T18:34:49.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:49 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T18:34:49.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:49 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T18:34:49.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:49 smithi039 ceph-mon[96489]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:34:49.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:49 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T18:34:49.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:49 smithi039 ceph-mon[96489]: mon.c calling monitor election 2023-07-21T18:34:49.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:49 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T18:34:49.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:49 smithi039 ceph-mon[96489]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:34:49.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:49 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T18:34:49.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:49 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T18:34:49.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:49 smithi039 ceph-mon[96489]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-21T18:34:49.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:49 smithi039 ceph-mon[96489]: monmap e2: 2 mons at {a=[v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0],c=[v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0]} removed_ranks: {} 2023-07-21T18:34:49.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:49 smithi039 ceph-mon[96489]: fsmap 2023-07-21T18:34:49.453 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:49 smithi039 ceph-mon[96489]: osdmap e4: 0 total, 0 up, 0 in 2023-07-21T18:34:49.453 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:49 smithi039 ceph-mon[96489]: mgrmap e13: x(active, since 41s) 2023-07-21T18:34:49.453 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:49 smithi039 ceph-mon[96489]: overall HEALTH_OK 2023-07-21T18:34:49.453 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:49 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:49.453 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:49 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:50.023 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:49.987+0000 7fd49f837700 1 -- 172.21.15.193:0/1075504894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4840035c0 msgr2=0x7fd484005a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:34:50.023 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:49.987+0000 7fd49f837700 1 --2- 172.21.15.193:0/1075504894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4840035c0 0x7fd484005a40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd498067c90 tx=0x7fd49400feb0 comp rx=0 tx=0).stop 2023-07-21T18:34:50.023 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:49.987+0000 7fd49f837700 1 -- 172.21.15.193:0/1075504894 shutdown_connections 2023-07-21T18:34:50.023 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:49.987+0000 7fd49f837700 1 --2- 172.21.15.193:0/1075504894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4840035c0 0x7fd484005a40 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:34:50.024 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:49.987+0000 7fd49f837700 1 -- 172.21.15.193:0/1075504894 >> 172.21.15.193:0/1075504894 conn(0x7fd4980743d0 msgr2=0x7fd4980747d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:34:50.024 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:49.987+0000 7fd49f837700 1 -- 172.21.15.193:0/1075504894 shutdown_connections 2023-07-21T18:34:50.024 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:49.987+0000 7fd49f837700 1 -- 172.21.15.193:0/1075504894 wait complete. 2023-07-21T18:34:50.026 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:49.988+0000 7fd49f837700 1 Processor -- start 2023-07-21T18:34:50.026 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:49.988+0000 7fd49f837700 1 -- start start 2023-07-21T18:34:50.027 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:49.988+0000 7fd49f837700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4840035c0 0x7fd49819aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:34:50.027 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:49.988+0000 7fd49f837700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd49819af50 0x7fd498194a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:34:50.027 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:49.988+0000 7fd49f837700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd49819b480 con 0x7fd4840035c0 2023-07-21T18:34:50.027 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:49.988+0000 7fd49f837700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd49819b5f0 con 0x7fd49819af50 2023-07-21T18:34:50.028 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:49.989+0000 7fd49cdd2700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd49819af50 0x7fd498194a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:34:50.028 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:49.989+0000 7fd49cdd2700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd49819af50 0x7fd498194a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:50500/0 (socket says 172.21.15.193:50500) 2023-07-21T18:34:50.028 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:49.989+0000 7fd49cdd2700 1 -- 172.21.15.193:0/852877077 learned_addr learned my addr 172.21.15.193:0/852877077 (peer_addr_for_me v2:172.21.15.193:0/0) 2023-07-21T18:34:50.029 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:49.989+0000 7fd49d5d3700 1 --2- 172.21.15.193:0/852877077 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4840035c0 0x7fd49819aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:34:50.029 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:49.989+0000 7fd49cdd2700 1 -- 172.21.15.193:0/852877077 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd49819af50 msgr2=0x7fd498194a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2023-07-21T18:34:50.029 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:49.989+0000 7fd49cdd2700 1 -- 172.21.15.193:0/852877077 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd49819af50 msgr2=0x7fd498194a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-07-21T18:34:50.029 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:49.989+0000 7fd49cdd2700 1 --2- 172.21.15.193:0/852877077 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd49819af50 0x7fd498194a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-21T18:34:50.030 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:49.989+0000 7fd49cdd2700 1 --2- 172.21.15.193:0/852877077 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd49819af50 0x7fd498194a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-21T18:34:50.030 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:49.989+0000 7fd49d5d3700 1 -- 172.21.15.193:0/852877077 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd49819af50 msgr2=0x7fd498194a90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-21T18:34:50.030 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:49.989+0000 7fd49d5d3700 1 --2- 172.21.15.193:0/852877077 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd49819af50 0x7fd498194a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:34:50.030 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:49.989+0000 7fd49d5d3700 1 -- 172.21.15.193:0/852877077 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd49400f900 con 0x7fd4840035c0 2023-07-21T18:34:50.031 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:49.989+0000 7fd49d5d3700 1 --2- 172.21.15.193:0/852877077 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4840035c0 0x7fd49819aa10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd498067c90 tx=0x7fd4940047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:34:50.031 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:49.989+0000 7fd48e7fc700 1 -- 172.21.15.193:0/852877077 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fd494013b00 con 0x7fd4840035c0 2023-07-21T18:34:50.031 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:49.990+0000 7fd48e7fc700 1 -- 172.21.15.193:0/852877077 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd4940044f0 con 0x7fd4840035c0 2023-07-21T18:34:50.031 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:49.990+0000 7fd49f837700 1 -- 172.21.15.193:0/852877077 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd498195090 con 0x7fd4840035c0 2023-07-21T18:34:50.032 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:49.990+0000 7fd48e7fc700 1 -- 172.21.15.193:0/852877077 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fd494003be0 con 0x7fd4840035c0 2023-07-21T18:34:50.032 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:49.990+0000 7fd49f837700 1 -- 172.21.15.193:0/852877077 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4981955e0 con 0x7fd4840035c0 2023-07-21T18:34:50.033 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:49.990+0000 7fd48e7fc700 1 -- 172.21.15.193:0/852877077 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fd494004030 con 0x7fd4840035c0 2023-07-21T18:34:50.033 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:49.991+0000 7fd49f837700 1 -- 172.21.15.193:0/852877077 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd498061b40 con 0x7fd4840035c0 2023-07-21T18:34:50.033 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:49.991+0000 7fd48e7fc700 1 --2- 172.21.15.193:0/852877077 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fd47c0385f0 0x7fd47c03aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:34:50.033 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:49.991+0000 7fd48e7fc700 1 -- 172.21.15.193:0/852877077 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd49404dca0 con 0x7fd4840035c0 2023-07-21T18:34:50.034 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:49.991+0000 7fd49cdd2700 1 --2- 172.21.15.193:0/852877077 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fd47c0385f0 0x7fd47c03aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:34:50.035 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:49.992+0000 7fd49cdd2700 1 --2- 172.21.15.193:0/852877077 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fd47c0385f0 0x7fd47c03aaa0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd488009180 tx=0x7fd488008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:34:50.035 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:49.997+0000 7fd48e7fc700 1 -- 172.21.15.193:0/852877077 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fd4940347c0 con 0x7fd4840035c0 2023-07-21T18:34:50.293 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:50.257+0000 7fd49f837700 1 -- 172.21.15.193:0/852877077 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fd49819f820 con 0x7fd4840035c0 2023-07-21T18:34:50.293 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:50.258+0000 7fd48e7fc700 1 -- 172.21.15.193:0/852877077 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1006 (secure 0 0 0) 0x7fd49404de60 con 0x7fd4840035c0 2023-07-21T18:34:50.293 INFO:teuthology.orchestra.run.smithi193.stdout: 2023-07-21T18:34:50.294 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":2,"fsid":"eaf8e25a-27f4-11ee-9b34-001a4aab830c","modified":"2023-07-21T18:34:43.985374Z","created":"2023-07-21T18:33:02.398821Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:3300","nonce":0},{"type":"v1","addr":"172.21.15.39:6789","nonce":0}]},"addr":"172.21.15.39:6789/0","public_addr":"172.21.15.39:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:3300","nonce":0},{"type":"v1","addr":"172.21.15.193:6789","nonce":0}]},"addr":"172.21.15.193:6789/0","public_addr":"172.21.15.193:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-07-21T18:34:50.295 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:50.260+0000 7fd49f837700 1 -- 172.21.15.193:0/852877077 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fd47c0385f0 msgr2=0x7fd47c03aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:34:50.295 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:50.260+0000 7fd49f837700 1 --2- 172.21.15.193:0/852877077 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fd47c0385f0 0x7fd47c03aaa0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd488009180 tx=0x7fd488008040 comp rx=0 tx=0).stop 2023-07-21T18:34:50.296 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:50.260+0000 7fd49f837700 1 -- 172.21.15.193:0/852877077 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4840035c0 msgr2=0x7fd49819aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:34:50.296 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:50.260+0000 7fd49f837700 1 --2- 172.21.15.193:0/852877077 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4840035c0 0x7fd49819aa10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd498067c90 tx=0x7fd4940047a0 comp rx=0 tx=0).stop 2023-07-21T18:34:50.296 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:50.260+0000 7fd49f837700 1 -- 172.21.15.193:0/852877077 shutdown_connections 2023-07-21T18:34:50.296 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:50.260+0000 7fd49f837700 1 --2- 172.21.15.193:0/852877077 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fd47c0385f0 0x7fd47c03aaa0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:34:50.296 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:50.260+0000 7fd49f837700 1 --2- 172.21.15.193:0/852877077 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4840035c0 0x7fd49819aa10 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:34:50.296 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:50.260+0000 7fd49f837700 1 --2- 172.21.15.193:0/852877077 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd49819af50 0x7fd498194a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:34:50.296 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:50.260+0000 7fd49f837700 1 -- 172.21.15.193:0/852877077 >> 172.21.15.193:0/852877077 conn(0x7fd4980743d0 msgr2=0x7fd498101830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:34:50.297 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:50.260+0000 7fd49f837700 1 -- 172.21.15.193:0/852877077 shutdown_connections 2023-07-21T18:34:50.297 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:50.260+0000 7fd49f837700 1 -- 172.21.15.193:0/852877077 wait complete. 2023-07-21T18:34:50.297 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 2 2023-07-21T18:34:50.382 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:50 smithi193 ceph-mon[98204]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:34:50.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:50 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:50.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:50 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T18:34:50.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:50 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:34:50.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:50 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T18:34:50.450 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:50 smithi039 ceph-mon[96489]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:34:50.450 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:50 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:50.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:50 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T18:34:50.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:50 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:34:50.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:50 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T18:34:51.382 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:51 smithi193 ceph-mon[98204]: Deploying daemon mon.b on smithi155 2023-07-21T18:34:51.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:51 smithi193 ceph-mon[98204]: from='client.? 172.21.15.193:0/852877077' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-21T18:34:51.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:51 smithi193 ceph-mon[98204]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:34:51.450 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:51 smithi039 ceph-mon[96489]: Deploying daemon mon.b on smithi155 2023-07-21T18:34:51.450 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:51 smithi039 ceph-mon[96489]: from='client.? 172.21.15.193:0/852877077' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-21T18:34:51.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:51 smithi039 ceph-mon[96489]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:34:51.719 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-07-21T18:34:51.719 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph mon dump -f json 2023-07-21T18:34:51.879 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:51 smithi155 systemd[1]: Starting Ceph mon.b for eaf8e25a-27f4-11ee-9b34-001a4aab830c... 2023-07-21T18:34:52.152 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:51 smithi155 podman[97894]: 2023-07-21T18:34:52.152 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: set uid:gid to 167:167 (ceph:ceph) 2023-07-21T18:34:52.152 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: ceph version 18.0.0-4746-g81cf785a (81cf785aa7781c4e08381c540f890cb9bc987ef9) reef (dev), process ceph-mon, pid 7 2023-07-21T18:34:52.152 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: pidfile_write: ignore empty --pid-file 2023-07-21T18:34:52.153 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 bash[97894]: b4501ab044c18a77d8026bbf35a32a17941df47ec48fc8699d44c661a0490814 2023-07-21T18:34:52.153 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: load: jerasure load: lrc 2023-07-21T18:34:52.153 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: RocksDB version: 7.9.2 2023-07-21T18:34:52.153 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Git sha 0 2023-07-21T18:34:52.153 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Compile date 2023-07-17 23:00:14 2023-07-21T18:34:52.153 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: DB SUMMARY 2023-07-21T18:34:52.153 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: DB Session ID: 35XE1202OM1N15MCVUDG 2023-07-21T18:34:52.154 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: CURRENT file: CURRENT 2023-07-21T18:34:52.154 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: IDENTITY file: IDENTITY 2023-07-21T18:34:52.154 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-07-21T18:34:52.154 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2023-07-21T18:34:52.154 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2023-07-21T18:34:52.154 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.error_if_exists: 0 2023-07-21T18:34:52.154 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.create_if_missing: 0 2023-07-21T18:34:52.154 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.paranoid_checks: 1 2023-07-21T18:34:52.155 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.flush_verify_memtable_count: 1 2023-07-21T18:34:52.155 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-07-21T18:34:52.155 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-07-21T18:34:52.155 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.env: 0x562beca6e720 2023-07-21T18:34:52.155 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.fs: PosixFileSystem 2023-07-21T18:34:52.155 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.info_log: 0x562bed493280 2023-07-21T18:34:52.155 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.max_file_opening_threads: 16 2023-07-21T18:34:52.155 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.statistics: (nil) 2023-07-21T18:34:52.156 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.use_fsync: 0 2023-07-21T18:34:52.156 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.max_log_file_size: 0 2023-07-21T18:34:52.156 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-07-21T18:34:52.156 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.log_file_time_to_roll: 0 2023-07-21T18:34:52.156 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.keep_log_file_num: 1000 2023-07-21T18:34:52.156 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.recycle_log_file_num: 0 2023-07-21T18:34:52.156 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.allow_fallocate: 1 2023-07-21T18:34:52.157 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.allow_mmap_reads: 0 2023-07-21T18:34:52.157 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.allow_mmap_writes: 0 2023-07-21T18:34:52.157 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.use_direct_reads: 0 2023-07-21T18:34:52.157 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-07-21T18:34:52.157 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.create_missing_column_families: 0 2023-07-21T18:34:52.157 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.db_log_dir: 2023-07-21T18:34:52.157 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.wal_dir: 2023-07-21T18:34:52.157 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.table_cache_numshardbits: 6 2023-07-21T18:34:52.158 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.WAL_ttl_seconds: 0 2023-07-21T18:34:52.158 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.WAL_size_limit_MB: 0 2023-07-21T18:34:52.158 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-07-21T18:34:52.158 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-07-21T18:34:52.158 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.is_fd_close_on_exec: 1 2023-07-21T18:34:52.158 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.advise_random_on_open: 1 2023-07-21T18:34:52.158 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.db_write_buffer_size: 0 2023-07-21T18:34:52.158 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.write_buffer_manager: 0x562bed3b43c0 2023-07-21T18:34:52.159 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-07-21T18:34:52.159 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-07-21T18:34:52.159 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.use_adaptive_mutex: 0 2023-07-21T18:34:52.159 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.rate_limiter: (nil) 2023-07-21T18:34:52.159 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-07-21T18:34:52.159 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.wal_recovery_mode: 2 2023-07-21T18:34:52.159 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.enable_thread_tracking: 0 2023-07-21T18:34:52.160 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.enable_pipelined_write: 0 2023-07-21T18:34:52.160 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.unordered_write: 0 2023-07-21T18:34:52.160 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-07-21T18:34:52.160 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-07-21T18:34:52.160 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-07-21T18:34:52.160 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-07-21T18:34:52.160 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.row_cache: None 2023-07-21T18:34:52.160 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.wal_filter: None 2023-07-21T18:34:52.161 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-07-21T18:34:52.161 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.allow_ingest_behind: 0 2023-07-21T18:34:52.161 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.two_write_queues: 0 2023-07-21T18:34:52.161 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.manual_wal_flush: 0 2023-07-21T18:34:52.161 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.wal_compression: 0 2023-07-21T18:34:52.161 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.atomic_flush: 0 2023-07-21T18:34:52.161 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-07-21T18:34:52.161 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.persist_stats_to_disk: 0 2023-07-21T18:34:52.162 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.write_dbid_to_manifest: 0 2023-07-21T18:34:52.162 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.log_readahead_size: 0 2023-07-21T18:34:52.162 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-07-21T18:34:52.162 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.best_efforts_recovery: 0 2023-07-21T18:34:52.162 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-07-21T18:34:52.162 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-07-21T18:34:52.162 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.allow_data_in_errors: 0 2023-07-21T18:34:52.164 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.db_host_id: __hostname__ 2023-07-21T18:34:52.164 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.enforce_single_del_contracts: true 2023-07-21T18:34:52.164 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.max_background_jobs: 2 2023-07-21T18:34:52.164 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.max_background_compactions: -1 2023-07-21T18:34:52.164 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.max_subcompactions: 1 2023-07-21T18:34:52.165 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-07-21T18:34:52.165 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-07-21T18:34:52.165 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.delayed_write_rate : 16777216 2023-07-21T18:34:52.165 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.max_total_wal_size: 0 2023-07-21T18:34:52.165 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-07-21T18:34:52.165 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.stats_dump_period_sec: 600 2023-07-21T18:34:52.165 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.stats_persist_period_sec: 600 2023-07-21T18:34:52.166 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-07-21T18:34:52.166 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.max_open_files: -1 2023-07-21T18:34:52.166 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.bytes_per_sync: 0 2023-07-21T18:34:52.166 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.wal_bytes_per_sync: 0 2023-07-21T18:34:52.166 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.strict_bytes_per_sync: 0 2023-07-21T18:34:52.167 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.compaction_readahead_size: 0 2023-07-21T18:34:52.167 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.max_background_flushes: -1 2023-07-21T18:34:52.167 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Compression algorithms supported: 2023-07-21T18:34:52.167 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-07-21T18:34:52.167 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: kZSTD supported: 0 2023-07-21T18:34:52.167 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: kXpressCompression supported: 0 2023-07-21T18:34:52.167 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: kLZ4HCCompression supported: 1 2023-07-21T18:34:52.167 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: kZlibCompression supported: 1 2023-07-21T18:34:52.168 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: kSnappyCompression supported: 1 2023-07-21T18:34:52.168 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: kLZ4Compression supported: 1 2023-07-21T18:34:52.168 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: kBZip2Compression supported: 0 2023-07-21T18:34:52.168 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Fast CRC32 supported: Supported on x86 2023-07-21T18:34:52.168 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: DMutex implementation: pthread_mutex_t 2023-07-21T18:34:52.168 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2023-07-21T18:34:52.168 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-07-21T18:34:52.168 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-07-21T18:34:52.169 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.merge_operator: 2023-07-21T18:34:52.169 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.compaction_filter: None 2023-07-21T18:34:52.169 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.compaction_filter_factory: None 2023-07-21T18:34:52.169 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.sst_partitioner_factory: None 2023-07-21T18:34:52.169 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.memtable_factory: SkipListFactory 2023-07-21T18:34:52.169 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.table_factory: BlockBasedTable 2023-07-21T18:34:52.170 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x562bed493380) 2023-07-21T18:34:52.170 INFO:journalctl@ceph.mon.b.smithi155.stdout: cache_index_and_filter_blocks: 1 2023-07-21T18:34:52.170 INFO:journalctl@ceph.mon.b.smithi155.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-07-21T18:34:52.170 INFO:journalctl@ceph.mon.b.smithi155.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-07-21T18:34:52.170 INFO:journalctl@ceph.mon.b.smithi155.stdout: pin_top_level_index_and_filter: 1 2023-07-21T18:34:52.170 INFO:journalctl@ceph.mon.b.smithi155.stdout: index_type: 0 2023-07-21T18:34:52.170 INFO:journalctl@ceph.mon.b.smithi155.stdout: data_block_index_type: 0 2023-07-21T18:34:52.171 INFO:journalctl@ceph.mon.b.smithi155.stdout: index_shortening: 1 2023-07-21T18:34:52.171 INFO:journalctl@ceph.mon.b.smithi155.stdout: data_block_hash_table_util_ratio: 0.750000 2023-07-21T18:34:52.171 INFO:journalctl@ceph.mon.b.smithi155.stdout: checksum: 4 2023-07-21T18:34:52.171 INFO:journalctl@ceph.mon.b.smithi155.stdout: no_block_cache: 0 2023-07-21T18:34:52.171 INFO:journalctl@ceph.mon.b.smithi155.stdout: block_cache: 0x562bed4531f0 2023-07-21T18:34:52.171 INFO:journalctl@ceph.mon.b.smithi155.stdout: block_cache_name: BinnedLRUCache 2023-07-21T18:34:52.171 INFO:journalctl@ceph.mon.b.smithi155.stdout: block_cache_options: 2023-07-21T18:34:52.171 INFO:journalctl@ceph.mon.b.smithi155.stdout: capacity : 536870912 2023-07-21T18:34:52.172 INFO:journalctl@ceph.mon.b.smithi155.stdout: num_shard_bits : 4 2023-07-21T18:34:52.172 INFO:journalctl@ceph.mon.b.smithi155.stdout: strict_capacity_limit : 0 2023-07-21T18:34:52.172 INFO:journalctl@ceph.mon.b.smithi155.stdout: high_pri_pool_ratio: 0.000 2023-07-21T18:34:52.172 INFO:journalctl@ceph.mon.b.smithi155.stdout: block_cache_compressed: (nil) 2023-07-21T18:34:52.172 INFO:journalctl@ceph.mon.b.smithi155.stdout: persistent_cache: (nil) 2023-07-21T18:34:52.172 INFO:journalctl@ceph.mon.b.smithi155.stdout: block_size: 4096 2023-07-21T18:34:52.172 INFO:journalctl@ceph.mon.b.smithi155.stdout: block_size_deviation: 10 2023-07-21T18:34:52.172 INFO:journalctl@ceph.mon.b.smithi155.stdout: block_restart_interval: 16 2023-07-21T18:34:52.173 INFO:journalctl@ceph.mon.b.smithi155.stdout: index_block_restart_interval: 1 2023-07-21T18:34:52.173 INFO:journalctl@ceph.mon.b.smithi155.stdout: metadata_block_size: 4096 2023-07-21T18:34:52.173 INFO:journalctl@ceph.mon.b.smithi155.stdout: partition_filters: 0 2023-07-21T18:34:52.173 INFO:journalctl@ceph.mon.b.smithi155.stdout: use_delta_encoding: 1 2023-07-21T18:34:52.173 INFO:journalctl@ceph.mon.b.smithi155.stdout: filter_policy: bloomfilter 2023-07-21T18:34:52.173 INFO:journalctl@ceph.mon.b.smithi155.stdout: whole_key_filtering: 1 2023-07-21T18:34:52.173 INFO:journalctl@ceph.mon.b.smithi155.stdout: verify_compression: 0 2023-07-21T18:34:52.173 INFO:journalctl@ceph.mon.b.smithi155.stdout: read_amp_bytes_per_bit: 0 2023-07-21T18:34:52.174 INFO:journalctl@ceph.mon.b.smithi155.stdout: format_version: 5 2023-07-21T18:34:52.174 INFO:journalctl@ceph.mon.b.smithi155.stdout: enable_index_compression: 1 2023-07-21T18:34:52.174 INFO:journalctl@ceph.mon.b.smithi155.stdout: block_align: 0 2023-07-21T18:34:52.174 INFO:journalctl@ceph.mon.b.smithi155.stdout: max_auto_readahead_size: 262144 2023-07-21T18:34:52.174 INFO:journalctl@ceph.mon.b.smithi155.stdout: prepopulate_block_cache: 0 2023-07-21T18:34:52.174 INFO:journalctl@ceph.mon.b.smithi155.stdout: initial_auto_readahead_size: 8192 2023-07-21T18:34:52.174 INFO:journalctl@ceph.mon.b.smithi155.stdout: num_file_reads_for_auto_readahead: 2 2023-07-21T18:34:52.174 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.write_buffer_size: 33554432 2023-07-21T18:34:52.175 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.max_write_buffer_number: 2 2023-07-21T18:34:52.175 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.compression: NoCompression 2023-07-21T18:34:52.175 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.bottommost_compression: Disabled 2023-07-21T18:34:52.175 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.prefix_extractor: nullptr 2023-07-21T18:34:52.175 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-07-21T18:34:52.175 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.num_levels: 7 2023-07-21T18:34:52.175 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-07-21T18:34:52.176 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-07-21T18:34:52.176 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-07-21T18:34:52.176 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-07-21T18:34:52.176 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-07-21T18:34:52.176 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-07-21T18:34:52.176 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-07-21T18:34:52.176 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-07-21T18:34:52.176 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-07-21T18:34:52.177 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-07-21T18:34:52.177 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-07-21T18:34:52.177 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-07-21T18:34:52.177 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.compression_opts.window_bits: -14 2023-07-21T18:34:52.177 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.compression_opts.level: 32767 2023-07-21T18:34:52.177 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.compression_opts.strategy: 0 2023-07-21T18:34:52.177 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-07-21T18:34:52.177 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-07-21T18:34:52.178 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-07-21T18:34:52.178 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-07-21T18:34:52.178 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.compression_opts.enabled: false 2023-07-21T18:34:52.178 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-07-21T18:34:52.178 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-07-21T18:34:52.178 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-07-21T18:34:52.178 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-07-21T18:34:52.178 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.target_file_size_base: 67108864 2023-07-21T18:34:52.179 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.target_file_size_multiplier: 1 2023-07-21T18:34:52.179 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-07-21T18:34:52.179 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-07-21T18:34:52.179 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-07-21T18:34:52.179 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-07-21T18:34:52.179 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-07-21T18:34:52.179 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-07-21T18:34:52.180 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-07-21T18:34:52.180 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-07-21T18:34:52.180 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-07-21T18:34:52.180 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-07-21T18:34:52.180 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-07-21T18:34:52.180 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-07-21T18:34:52.180 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-07-21T18:34:52.180 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.arena_block_size: 1048576 2023-07-21T18:34:52.181 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-07-21T18:34:52.181 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-07-21T18:34:52.181 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.disable_auto_compactions: 0 2023-07-21T18:34:52.181 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-07-21T18:34:52.181 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-07-21T18:34:52.181 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-07-21T18:34:52.181 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-07-21T18:34:52.181 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-07-21T18:34:52.182 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-07-21T18:34:52.182 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-07-21T18:34:52.182 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-07-21T18:34:52.182 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-07-21T18:34:52.182 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-07-21T18:34:52.182 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.table_properties_collectors: 2023-07-21T18:34:52.182 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.inplace_update_support: 0 2023-07-21T18:34:52.182 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.inplace_update_num_locks: 10000 2023-07-21T18:34:52.183 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-07-21T18:34:52.183 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-07-21T18:34:52.183 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.memtable_huge_page_size: 0 2023-07-21T18:34:52.183 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.bloom_locality: 0 2023-07-21T18:34:52.183 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.max_successive_merges: 0 2023-07-21T18:34:52.184 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.optimize_filters_for_hits: 0 2023-07-21T18:34:52.184 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.paranoid_file_checks: 0 2023-07-21T18:34:52.184 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.force_consistency_checks: 1 2023-07-21T18:34:52.184 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.report_bg_io_stats: 0 2023-07-21T18:34:52.184 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.ttl: 2592000 2023-07-21T18:34:52.184 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.periodic_compaction_seconds: 0 2023-07-21T18:34:52.184 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-07-21T18:34:52.184 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-07-21T18:34:52.185 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.enable_blob_files: false 2023-07-21T18:34:52.185 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.min_blob_size: 0 2023-07-21T18:34:52.185 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.blob_file_size: 268435456 2023-07-21T18:34:52.185 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.blob_compression_type: NoCompression 2023-07-21T18:34:52.185 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.enable_blob_garbage_collection: false 2023-07-21T18:34:52.185 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-07-21T18:34:52.185 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-07-21T18:34:52.185 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-07-21T18:34:52.186 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.blob_file_starting_level: 0 2023-07-21T18:34:52.186 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-07-21T18:34:52.186 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-07-21T18:34:52.186 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-07-21T18:34:52.186 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 27830648-09ae-492c-a405-3e37554d0770 2023-07-21T18:34:52.186 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689964492024272, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-07-21T18:34:52.186 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-07-21T18:34:52.187 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689964492024826, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1689964492, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "27830648-09ae-492c-a405-3e37554d0770", "db_session_id": "35XE1202OM1N15MCVUDG", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-07-21T18:34:52.187 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689964492024915, "job": 1, "event": "recovery_finished"} 2023-07-21T18:34:52.187 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-07-21T18:34:52.187 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-07-21T18:34:52.187 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x562bed4ec000 2023-07-21T18:34:52.187 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: DB pointer 0x562bed4d8000 2023-07-21T18:34:52.187 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-07-21T18:34:52.187 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-07-21T18:34:52.188 INFO:journalctl@ceph.mon.b.smithi155.stdout: ** DB Stats ** 2023-07-21T18:34:52.188 INFO:journalctl@ceph.mon.b.smithi155.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-21T18:34:52.188 INFO:journalctl@ceph.mon.b.smithi155.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-07-21T18:34:52.188 INFO:journalctl@ceph.mon.b.smithi155.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-21T18:34:52.188 INFO:journalctl@ceph.mon.b.smithi155.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-21T18:34:52.188 INFO:journalctl@ceph.mon.b.smithi155.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-07-21T18:34:52.188 INFO:journalctl@ceph.mon.b.smithi155.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-21T18:34:52.189 INFO:journalctl@ceph.mon.b.smithi155.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-21T18:34:52.189 INFO:journalctl@ceph.mon.b.smithi155.stdout: 2023-07-21T18:34:52.189 INFO:journalctl@ceph.mon.b.smithi155.stdout: ** Compaction Stats [default] ** 2023-07-21T18:34:52.189 INFO:journalctl@ceph.mon.b.smithi155.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-21T18:34:52.189 INFO:journalctl@ceph.mon.b.smithi155.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-07-21T18:34:52.189 INFO:journalctl@ceph.mon.b.smithi155.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-21T18:34:52.189 INFO:journalctl@ceph.mon.b.smithi155.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-21T18:34:52.189 INFO:journalctl@ceph.mon.b.smithi155.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-21T18:34:52.190 INFO:journalctl@ceph.mon.b.smithi155.stdout: 2023-07-21T18:34:52.190 INFO:journalctl@ceph.mon.b.smithi155.stdout: ** Compaction Stats [default] ** 2023-07-21T18:34:52.190 INFO:journalctl@ceph.mon.b.smithi155.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-21T18:34:52.190 INFO:journalctl@ceph.mon.b.smithi155.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-07-21T18:34:52.190 INFO:journalctl@ceph.mon.b.smithi155.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-21T18:34:52.190 INFO:journalctl@ceph.mon.b.smithi155.stdout: 2023-07-21T18:34:52.190 INFO:journalctl@ceph.mon.b.smithi155.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-07-21T18:34:52.190 INFO:journalctl@ceph.mon.b.smithi155.stdout: 2023-07-21T18:34:52.191 INFO:journalctl@ceph.mon.b.smithi155.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-21T18:34:52.191 INFO:journalctl@ceph.mon.b.smithi155.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-07-21T18:34:52.191 INFO:journalctl@ceph.mon.b.smithi155.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-07-21T18:34:52.191 INFO:journalctl@ceph.mon.b.smithi155.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-07-21T18:34:52.191 INFO:journalctl@ceph.mon.b.smithi155.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-07-21T18:34:52.191 INFO:journalctl@ceph.mon.b.smithi155.stdout: AddFile(Keys): cumulative 0, interval 0 2023-07-21T18:34:52.191 INFO:journalctl@ceph.mon.b.smithi155.stdout: Cumulative compaction: 0.00 GB write, 0.50 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-21T18:34:52.191 INFO:journalctl@ceph.mon.b.smithi155.stdout: Interval compaction: 0.00 GB write, 0.50 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-21T18:34:52.192 INFO:journalctl@ceph.mon.b.smithi155.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-07-21T18:34:52.192 INFO:journalctl@ceph.mon.b.smithi155.stdout: Block cache BinnedLRUCache@0x562bed4531f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2023-07-21T18:34:52.192 INFO:journalctl@ceph.mon.b.smithi155.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-07-21T18:34:52.192 INFO:journalctl@ceph.mon.b.smithi155.stdout: 2023-07-21T18:34:52.192 INFO:journalctl@ceph.mon.b.smithi155.stdout: ** File Read Latency Histogram By Level [default] ** 2023-07-21T18:34:52.192 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: mon.b does not exist in monmap, will attempt to join an existing cluster 2023-07-21T18:34:52.192 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: using public_addr v2:172.21.15.155:0/0 -> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] 2023-07-21T18:34:52.192 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: starting mon.b rank -1 at public addrs [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] at bind addrs [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c 2023-07-21T18:34:52.193 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: mon.b@-1(???) e0 preinit fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c 2023-07-21T18:34:52.193 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 systemd[1]: Started Ceph mon.b for eaf8e25a-27f4-11ee-9b34-001a4aab830c. 2023-07-21T18:34:52.193 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: mon.b@-1(synchronizing).mds e1 new map 2023-07-21T18:34:52.193 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: mon.b@-1(synchronizing).mds e1 print_map 2023-07-21T18:34:52.193 INFO:journalctl@ceph.mon.b.smithi155.stdout: e1 2023-07-21T18:34:52.193 INFO:journalctl@ceph.mon.b.smithi155.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-07-21T18:34:52.193 INFO:journalctl@ceph.mon.b.smithi155.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-21T18:34:52.194 INFO:journalctl@ceph.mon.b.smithi155.stdout: legacy client fscid: -1 2023-07-21T18:34:52.194 INFO:journalctl@ceph.mon.b.smithi155.stdout: 2023-07-21T18:34:52.194 INFO:journalctl@ceph.mon.b.smithi155.stdout: No filesystems configured 2023-07-21T18:34:52.194 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-07-21T18:34:52.194 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-07-21T18:34:52.194 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-07-21T18:34:52.194 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-07-21T18:34:52.194 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-07-21T18:34:52.195 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-07-21T18:34:52.195 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-07-21T18:34:52.195 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-21T18:34:52.195 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-21T18:34:52.195 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-21T18:34:52.195 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:52.195 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:52.195 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T18:34:52.196 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:34:52.196 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:34:52.196 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:52.196 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:52.196 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:52.196 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:52.196 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi193", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:34:52.196 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:34:52.197 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:34:52.197 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: Updating smithi193:/etc/ceph/ceph.conf 2023-07-21T18:34:52.197 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/1251732569' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-07-21T18:34:52.197 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: Updating smithi193:/var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T18:34:52.197 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2023-07-21T18:34:52.197 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: Updating smithi193:/var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-21T18:34:52.197 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:52.198 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:52.198 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:52.198 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/1251732569' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-07-21T18:34:52.198 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: osdmap e4: 0 total, 0 up, 0 in 2023-07-21T18:34:52.198 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:34:52.198 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi039:172.21.15.39=a;smithi155:172.21.15.155=b;smithi193:172.21.15.193=c", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:34:52.198 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: Saving service mon spec with placement smithi039:172.21.15.39=a;smithi155:172.21.15.155=b;smithi193:172.21.15.193=c;count:3 2023-07-21T18:34:52.198 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:52.199 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:34:52.199 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:34:52.199 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:34:52.199 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:52.199 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T18:34:52.199 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:34:52.199 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: Deploying daemon mon.c on smithi193 2023-07-21T18:34:52.199 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:34:52.200 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:34:52.200 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T18:34:52.200 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: mon.a calling monitor election 2023-07-21T18:34:52.200 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T18:34:52.200 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T18:34:52.200 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:34:52.200 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T18:34:52.200 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: mon.c calling monitor election 2023-07-21T18:34:52.201 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T18:34:52.201 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:34:52.201 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T18:34:52.201 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T18:34:52.201 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-21T18:34:52.201 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: monmap e2: 2 mons at {a=[v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0],c=[v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0]} removed_ranks: {} 2023-07-21T18:34:52.201 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: fsmap 2023-07-21T18:34:52.202 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: osdmap e4: 0 total, 0 up, 0 in 2023-07-21T18:34:52.202 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: mgrmap e13: x(active, since 41s) 2023-07-21T18:34:52.202 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: overall HEALTH_OK 2023-07-21T18:34:52.202 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:52.202 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:52.202 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:34:52.202 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:52.202 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T18:34:52.203 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:34:52.203 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T18:34:52.203 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: Deploying daemon mon.b on smithi155 2023-07-21T18:34:52.203 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: from='client.? 172.21.15.193:0/852877077' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-21T18:34:52.203 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:34:52.203 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-07-21T18:34:52.203 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: mon.b@-1(synchronizing) e2 handle_conf_change mon_allow_pool_delete 2023-07-21T18:34:52.203 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: expand_channel_meta expand map: {default=false} 2023-07-21T18:34:52.204 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: expand_channel_meta from 'false' to 'false' 2023-07-21T18:34:52.204 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: expand_channel_meta expanded map: {default=false} 2023-07-21T18:34:52.204 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: expand_channel_meta expand map: {default=info} 2023-07-21T18:34:52.204 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: expand_channel_meta from 'info' to 'info' 2023-07-21T18:34:52.204 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: expand_channel_meta expanded map: {default=info} 2023-07-21T18:34:52.204 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: expand_channel_meta expand map: {default=daemon} 2023-07-21T18:34:52.204 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: expand_channel_meta from 'daemon' to 'daemon' 2023-07-21T18:34:52.204 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: expand_channel_meta expanded map: {default=daemon} 2023-07-21T18:34:52.205 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: expand_channel_meta expand map: {default=debug} 2023-07-21T18:34:52.205 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: expand_channel_meta from 'debug' to 'debug' 2023-07-21T18:34:52.205 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:52 smithi155 ceph-mon[97934]: expand_channel_meta expanded map: {default=debug} 2023-07-21T18:34:52.276 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T18:34:57.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:57 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T18:34:57.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:57 smithi039 ceph-mon[96489]: mon.c calling monitor election 2023-07-21T18:34:57.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:57 smithi039 ceph-mon[96489]: mon.a calling monitor election 2023-07-21T18:34:57.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:57 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T18:34:57.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:57 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T18:34:57.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:57 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T18:34:57.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:57 smithi039 ceph-mon[96489]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:34:57.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:57 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T18:34:57.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:57 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T18:34:57.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:57 smithi039 ceph-mon[96489]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:34:57.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:57 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T18:34:57.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:57 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T18:34:57.453 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:57 smithi039 ceph-mon[96489]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:34:57.453 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:57 smithi039 ceph-mon[96489]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-21T18:34:57.453 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:57 smithi039 ceph-mon[96489]: monmap e3: 3 mons at {a=[v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0],b=[v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0],c=[v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0]} removed_ranks: {} 2023-07-21T18:34:57.453 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:57 smithi039 ceph-mon[96489]: fsmap 2023-07-21T18:34:57.453 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:57 smithi039 ceph-mon[96489]: osdmap e4: 0 total, 0 up, 0 in 2023-07-21T18:34:57.453 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:57 smithi039 ceph-mon[96489]: mgrmap e13: x(active, since 50s) 2023-07-21T18:34:57.453 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:57 smithi039 ceph-mon[96489]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-07-21T18:34:57.454 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:57 smithi039 ceph-mon[96489]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-07-21T18:34:57.454 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:57 smithi039 ceph-mon[96489]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-07-21T18:34:57.454 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:57 smithi039 ceph-mon[96489]: mon.b (rank 2) addr [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] is down (out of quorum) 2023-07-21T18:34:57.454 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:57 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:57.454 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:57 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:57.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:57 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T18:34:57.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:57 smithi193 ceph-mon[98204]: mon.c calling monitor election 2023-07-21T18:34:57.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:57 smithi193 ceph-mon[98204]: mon.a calling monitor election 2023-07-21T18:34:57.634 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:57 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T18:34:57.634 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:57 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T18:34:57.634 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:57 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T18:34:57.634 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:57 smithi193 ceph-mon[98204]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:34:57.634 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:57 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T18:34:57.634 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:57 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T18:34:57.634 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:57 smithi193 ceph-mon[98204]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:34:57.634 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:57 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T18:34:57.635 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:57 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T18:34:57.635 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:57 smithi193 ceph-mon[98204]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:34:57.635 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:57 smithi193 ceph-mon[98204]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-21T18:34:57.635 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:57 smithi193 ceph-mon[98204]: monmap e3: 3 mons at {a=[v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0],b=[v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0],c=[v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0]} removed_ranks: {} 2023-07-21T18:34:57.635 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:57 smithi193 ceph-mon[98204]: fsmap 2023-07-21T18:34:57.635 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:57 smithi193 ceph-mon[98204]: osdmap e4: 0 total, 0 up, 0 in 2023-07-21T18:34:57.635 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:57 smithi193 ceph-mon[98204]: mgrmap e13: x(active, since 50s) 2023-07-21T18:34:57.636 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:57 smithi193 ceph-mon[98204]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-07-21T18:34:57.636 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:57 smithi193 ceph-mon[98204]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-07-21T18:34:57.636 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:57 smithi193 ceph-mon[98204]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-07-21T18:34:57.636 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:57 smithi193 ceph-mon[98204]: mon.b (rank 2) addr [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] is down (out of quorum) 2023-07-21T18:34:57.636 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:57 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:57.636 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:57 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:57.830 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:57.794+0000 7ffa1a559700 1 -- 172.21.15.193:0/3715891016 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa14105730 msgr2=0x7ff9fc005610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:34:57.830 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:57.794+0000 7ffa1a559700 1 --2- 172.21.15.193:0/3715891016 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa14105730 0x7ff9fc005610 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ffa0400fc30 tx=0x7ffa0400ff40 comp rx=0 tx=0).stop 2023-07-21T18:34:57.830 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:57.794+0000 7ffa1a559700 1 -- 172.21.15.193:0/3715891016 shutdown_connections 2023-07-21T18:34:57.830 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:57.794+0000 7ffa1a559700 1 --2- 172.21.15.193:0/3715891016 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa14105730 0x7ff9fc005610 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:34:57.830 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:57.794+0000 7ffa1a559700 1 -- 172.21.15.193:0/3715891016 >> 172.21.15.193:0/3715891016 conn(0x7ffa140fb210 msgr2=0x7ffa140fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:34:57.831 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:57.794+0000 7ffa1a559700 1 -- 172.21.15.193:0/3715891016 shutdown_connections 2023-07-21T18:34:57.831 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:57.794+0000 7ffa1a559700 1 -- 172.21.15.193:0/3715891016 wait complete. 2023-07-21T18:34:57.833 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:57.795+0000 7ffa1a559700 1 Processor -- start 2023-07-21T18:34:57.833 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:57.795+0000 7ffa1a559700 1 -- start start 2023-07-21T18:34:57.833 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:57.796+0000 7ffa1a559700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa14105730 0x7ffa14194dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:34:57.833 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:57.796+0000 7ffa1a559700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa14195310 0x7ffa14199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:34:57.833 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:57.796+0000 7ffa1a559700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa14199cc0 0x7ffa1419a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:34:57.833 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:57.796+0000 7ffa1a559700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa141959e0 con 0x7ffa14199cc0 2023-07-21T18:34:57.834 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:57.796+0000 7ffa1a559700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa141a1a30 con 0x7ffa14105730 2023-07-21T18:34:57.834 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:57.796+0000 7ffa1a559700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa141a1b70 con 0x7ffa14195310 2023-07-21T18:34:57.834 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:57.796+0000 7ffa137fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa14195310 0x7ffa14199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:34:57.834 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:57.796+0000 7ffa137fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa14195310 0x7ffa14199780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:50526/0 (socket says 172.21.15.193:50526) 2023-07-21T18:34:57.834 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:57.796+0000 7ffa137fe700 1 -- 172.21.15.193:0/685145005 learned_addr learned my addr 172.21.15.193:0/685145005 (peer_addr_for_me v2:172.21.15.193:0/0) 2023-07-21T18:34:57.834 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:57.796+0000 7ffa137fe700 1 -- 172.21.15.193:0/685145005 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa14195310 msgr2=0x7ffa14199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2023-07-21T18:34:57.834 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:57.796+0000 7ffa137fe700 1 -- 172.21.15.193:0/685145005 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa14195310 msgr2=0x7ffa14199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-07-21T18:34:57.835 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:57.796+0000 7ffa137fe700 1 --2- 172.21.15.193:0/685145005 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa14195310 0x7ffa14199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-21T18:34:57.835 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:57.796+0000 7ffa137fe700 1 --2- 172.21.15.193:0/685145005 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa14195310 0x7ffa14199780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-21T18:34:57.835 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:57.796+0000 7ffa18af6700 1 --2- 172.21.15.193:0/685145005 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa14199cc0 0x7ffa1419a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:34:57.835 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:57.796+0000 7ffa13fff700 1 --2- 172.21.15.193:0/685145005 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa14105730 0x7ffa14194dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:34:57.836 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:57.796+0000 7ffa13fff700 1 -- 172.21.15.193:0/685145005 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa14105730 msgr2=0x7ffa14194dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2023-07-21T18:34:57.836 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:57.796+0000 7ffa13fff700 1 -- 172.21.15.193:0/685145005 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa14105730 msgr2=0x7ffa14194dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-07-21T18:34:57.836 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:57.796+0000 7ffa13fff700 1 --2- 172.21.15.193:0/685145005 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa14105730 0x7ffa14194dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-21T18:34:57.837 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:57.796+0000 7ffa13fff700 1 --2- 172.21.15.193:0/685145005 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa14105730 0x7ffa14194dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-21T18:34:57.837 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:57.796+0000 7ffa18af6700 1 -- 172.21.15.193:0/685145005 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa14195310 msgr2=0x7ffa14199780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-21T18:34:57.837 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:57.796+0000 7ffa18af6700 1 --2- 172.21.15.193:0/685145005 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa14195310 0x7ffa14199780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:34:57.837 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:57.796+0000 7ffa18af6700 1 -- 172.21.15.193:0/685145005 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa14105730 msgr2=0x7ffa14194dd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-21T18:34:57.837 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:57.796+0000 7ffa18af6700 1 --2- 172.21.15.193:0/685145005 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa14105730 0x7ffa14194dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:34:57.837 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:57.796+0000 7ffa18af6700 1 -- 172.21.15.193:0/685145005 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa0400f910 con 0x7ffa14199cc0 2023-07-21T18:34:57.837 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:57.797+0000 7ffa18af6700 1 --2- 172.21.15.193:0/685145005 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa14199cc0 0x7ffa1419a160 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c00bd80 tx=0x7ffa0c00be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:34:57.837 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:57.797+0000 7ffa117fa700 1 -- 172.21.15.193:0/685145005 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c00cd10 con 0x7ffa14199cc0 2023-07-21T18:34:57.838 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:57.797+0000 7ffa117fa700 1 -- 172.21.15.193:0/685145005 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa0c00ce70 con 0x7ffa14199cc0 2023-07-21T18:34:57.838 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:57.797+0000 7ffa1a559700 1 -- 172.21.15.193:0/685145005 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa141a1e20 con 0x7ffa14199cc0 2023-07-21T18:34:57.838 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:57.797+0000 7ffa117fa700 1 -- 172.21.15.193:0/685145005 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c012880 con 0x7ffa14199cc0 2023-07-21T18:34:57.838 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:57.797+0000 7ffa1a559700 1 -- 172.21.15.193:0/685145005 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa141a22f0 con 0x7ffa14199cc0 2023-07-21T18:34:57.838 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:57.798+0000 7ffa117fa700 1 -- 172.21.15.193:0/685145005 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7ffa0c012ac0 con 0x7ffa14199cc0 2023-07-21T18:34:57.839 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:57.798+0000 7ffa117fa700 1 --2- 172.21.15.193:0/685145005 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7ffa00038c30 0x7ffa0003b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:34:57.839 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:57.798+0000 7ffa117fa700 1 -- 172.21.15.193:0/685145005 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ffa0c04ad40 con 0x7ffa14199cc0 2023-07-21T18:34:57.839 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:57.798+0000 7ffa1a559700 1 -- 172.21.15.193:0/685145005 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa14061b40 con 0x7ffa14199cc0 2023-07-21T18:34:57.839 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:57.798+0000 7ffa13fff700 1 --2- 172.21.15.193:0/685145005 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7ffa00038c30 0x7ffa0003b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:34:57.840 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:57.799+0000 7ffa13fff700 1 --2- 172.21.15.193:0/685145005 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7ffa00038c30 0x7ffa0003b0e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ffa04000c00 tx=0x7ffa04014040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:34:57.840 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:57.804+0000 7ffa117fa700 1 -- 172.21.15.193:0/685145005 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7ffa0c019070 con 0x7ffa14199cc0 2023-07-21T18:34:58.110 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:58.074+0000 7ffa1a559700 1 -- 172.21.15.193:0/685145005 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ffa141a2610 con 0x7ffa14199cc0 2023-07-21T18:34:58.110 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:58.075+0000 7ffa117fa700 1 -- 172.21.15.193:0/685145005 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1274 (secure 0 0 0) 0x7ffa0c01d3b0 con 0x7ffa14199cc0 2023-07-21T18:34:58.111 INFO:teuthology.orchestra.run.smithi193.stdout: 2023-07-21T18:34:58.111 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":3,"fsid":"eaf8e25a-27f4-11ee-9b34-001a4aab830c","modified":"2023-07-21T18:34:52.055826Z","created":"2023-07-21T18:33:02.398821Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:3300","nonce":0},{"type":"v1","addr":"172.21.15.39:6789","nonce":0}]},"addr":"172.21.15.39:6789/0","public_addr":"172.21.15.39:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:3300","nonce":0},{"type":"v1","addr":"172.21.15.193:6789","nonce":0}]},"addr":"172.21.15.193:6789/0","public_addr":"172.21.15.193:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:3300","nonce":0},{"type":"v1","addr":"172.21.15.155:6789","nonce":0}]},"addr":"172.21.15.155:6789/0","public_addr":"172.21.15.155:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-07-21T18:34:58.113 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:58.078+0000 7ffa1a559700 1 -- 172.21.15.193:0/685145005 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7ffa00038c30 msgr2=0x7ffa0003b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:34:58.113 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:58.078+0000 7ffa1a559700 1 --2- 172.21.15.193:0/685145005 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7ffa00038c30 0x7ffa0003b0e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ffa04000c00 tx=0x7ffa04014040 comp rx=0 tx=0).stop 2023-07-21T18:34:58.113 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:58.078+0000 7ffa1a559700 1 -- 172.21.15.193:0/685145005 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa14199cc0 msgr2=0x7ffa1419a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:34:58.114 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:58.078+0000 7ffa1a559700 1 --2- 172.21.15.193:0/685145005 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa14199cc0 0x7ffa1419a160 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c00bd80 tx=0x7ffa0c00be60 comp rx=0 tx=0).stop 2023-07-21T18:34:58.114 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:58.078+0000 7ffa1a559700 1 -- 172.21.15.193:0/685145005 shutdown_connections 2023-07-21T18:34:58.114 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:58.078+0000 7ffa1a559700 1 --2- 172.21.15.193:0/685145005 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7ffa00038c30 0x7ffa0003b0e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:34:58.114 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:58.078+0000 7ffa1a559700 1 --2- 172.21.15.193:0/685145005 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa14105730 0x7ffa14194dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:34:58.114 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:58.078+0000 7ffa1a559700 1 --2- 172.21.15.193:0/685145005 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa14199cc0 0x7ffa1419a160 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:34:58.114 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:58.078+0000 7ffa1a559700 1 --2- 172.21.15.193:0/685145005 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa14195310 0x7ffa14199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:34:58.114 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:58.078+0000 7ffa1a559700 1 -- 172.21.15.193:0/685145005 >> 172.21.15.193:0/685145005 conn(0x7ffa140fb210 msgr2=0x7ffa1418db90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:34:58.114 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:58.078+0000 7ffa1a559700 1 -- 172.21.15.193:0/685145005 shutdown_connections 2023-07-21T18:34:58.115 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:34:58.078+0000 7ffa1a559700 1 -- 172.21.15.193:0/685145005 wait complete. 2023-07-21T18:34:58.115 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 3 2023-07-21T18:34:58.438 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:58 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:58.438 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:58 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:58.439 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:58 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:34:58.439 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:58 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T18:34:58.439 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:58 smithi193 ceph-mon[98204]: from='client.? 172.21.15.193:0/685145005' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-21T18:34:58.440 INFO:tasks.cephadm:Generating final ceph.conf file... 2023-07-21T18:34:58.440 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph config generate-minimal-conf 2023-07-21T18:34:58.450 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:58 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:58.450 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:58 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:58.450 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:58 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:34:58.450 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:58 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T18:34:58.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:58 smithi039 ceph-mon[96489]: from='client.? 172.21.15.193:0/685145005' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-21T18:34:59.015 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:34:59.401 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:59 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T18:34:59.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:59 smithi155 ceph-mon[97934]: mon.c calling monitor election 2023-07-21T18:34:59.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:59 smithi155 ceph-mon[97934]: mon.a calling monitor election 2023-07-21T18:34:59.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:59 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T18:34:59.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:59 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T18:34:59.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:59 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T18:34:59.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:59 smithi155 ceph-mon[97934]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:34:59.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:59 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T18:34:59.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:59 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T18:34:59.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:59 smithi155 ceph-mon[97934]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:34:59.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:59 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T18:34:59.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:59 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T18:34:59.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:59 smithi155 ceph-mon[97934]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:34:59.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:59 smithi155 ceph-mon[97934]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-21T18:34:59.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:59 smithi155 ceph-mon[97934]: monmap e3: 3 mons at {a=[v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0],b=[v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0],c=[v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0]} removed_ranks: {} 2023-07-21T18:34:59.404 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:59 smithi155 ceph-mon[97934]: fsmap 2023-07-21T18:34:59.404 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:59 smithi155 ceph-mon[97934]: osdmap e4: 0 total, 0 up, 0 in 2023-07-21T18:34:59.404 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:59 smithi155 ceph-mon[97934]: mgrmap e13: x(active, since 50s) 2023-07-21T18:34:59.404 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:59 smithi155 ceph-mon[97934]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-07-21T18:34:59.404 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:59 smithi155 ceph-mon[97934]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-07-21T18:34:59.404 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:59 smithi155 ceph-mon[97934]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-07-21T18:34:59.404 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:59 smithi155 ceph-mon[97934]: mon.b (rank 2) addr [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] is down (out of quorum) 2023-07-21T18:34:59.405 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:59 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:59.405 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:59 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:59.405 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:59 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:59.405 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:59 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:34:59.405 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:59 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:34:59.405 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:59 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T18:34:59.405 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:59 smithi155 ceph-mon[97934]: from='client.? 172.21.15.193:0/685145005' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-21T18:35:00.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:59 smithi039 ceph-mon[96489]: mon.b calling monitor election 2023-07-21T18:35:00.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:59 smithi039 ceph-mon[96489]: Updating smithi039:/etc/ceph/ceph.conf 2023-07-21T18:35:00.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:59 smithi039 ceph-mon[96489]: Updating smithi155:/etc/ceph/ceph.conf 2023-07-21T18:35:00.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:59 smithi039 ceph-mon[96489]: Updating smithi193:/etc/ceph/ceph.conf 2023-07-21T18:35:00.202 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:59 smithi039 ceph-mon[96489]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:35:00.202 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:59 smithi039 ceph-mon[96489]: mon.b calling monitor election 2023-07-21T18:35:00.202 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:59 smithi039 ceph-mon[96489]: mon.a calling monitor election 2023-07-21T18:35:00.202 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:59 smithi039 ceph-mon[96489]: mon.c calling monitor election 2023-07-21T18:35:00.202 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:59 smithi039 ceph-mon[96489]: Updating smithi155:/var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T18:35:00.202 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:59 smithi039 ceph-mon[96489]: Updating smithi039:/var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T18:35:00.202 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:59 smithi039 ceph-mon[96489]: Updating smithi193:/var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T18:35:00.203 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:59 smithi039 ceph-mon[96489]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-07-21T18:35:00.203 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:59 smithi039 ceph-mon[96489]: monmap e3: 3 mons at {a=[v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0],b=[v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0],c=[v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0]} removed_ranks: {} 2023-07-21T18:35:00.203 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:59 smithi039 ceph-mon[96489]: fsmap 2023-07-21T18:35:00.203 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:59 smithi039 ceph-mon[96489]: osdmap e4: 0 total, 0 up, 0 in 2023-07-21T18:35:00.203 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:59 smithi039 ceph-mon[96489]: mgrmap e13: x(active, since 52s) 2023-07-21T18:35:00.203 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:59 smithi039 ceph-mon[96489]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-07-21T18:35:00.203 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:59 smithi039 ceph-mon[96489]: Cluster is now healthy 2023-07-21T18:35:00.203 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:59 smithi039 ceph-mon[96489]: overall HEALTH_OK 2023-07-21T18:35:00.204 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:59 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:00.204 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:59 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:00.204 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:59 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:00.204 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:59 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:00.204 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:59 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:00.204 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:59 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:00.204 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:59 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:00.205 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:59 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T18:35:00.205 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:59 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-21T18:35:00.205 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:34:59 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:35:00.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:59 smithi193 ceph-mon[98204]: mon.b calling monitor election 2023-07-21T18:35:00.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:59 smithi193 ceph-mon[98204]: Updating smithi039:/etc/ceph/ceph.conf 2023-07-21T18:35:00.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:59 smithi193 ceph-mon[98204]: Updating smithi155:/etc/ceph/ceph.conf 2023-07-21T18:35:00.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:59 smithi193 ceph-mon[98204]: Updating smithi193:/etc/ceph/ceph.conf 2023-07-21T18:35:00.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:59 smithi193 ceph-mon[98204]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:35:00.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:59 smithi193 ceph-mon[98204]: mon.b calling monitor election 2023-07-21T18:35:00.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:59 smithi193 ceph-mon[98204]: mon.a calling monitor election 2023-07-21T18:35:00.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:59 smithi193 ceph-mon[98204]: mon.c calling monitor election 2023-07-21T18:35:00.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:59 smithi193 ceph-mon[98204]: Updating smithi155:/var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T18:35:00.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:59 smithi193 ceph-mon[98204]: Updating smithi039:/var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T18:35:00.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:59 smithi193 ceph-mon[98204]: Updating smithi193:/var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T18:35:00.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:59 smithi193 ceph-mon[98204]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-07-21T18:35:00.385 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:59 smithi193 ceph-mon[98204]: monmap e3: 3 mons at {a=[v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0],b=[v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0],c=[v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0]} removed_ranks: {} 2023-07-21T18:35:00.385 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:59 smithi193 ceph-mon[98204]: fsmap 2023-07-21T18:35:00.385 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:59 smithi193 ceph-mon[98204]: osdmap e4: 0 total, 0 up, 0 in 2023-07-21T18:35:00.385 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:59 smithi193 ceph-mon[98204]: mgrmap e13: x(active, since 52s) 2023-07-21T18:35:00.385 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:59 smithi193 ceph-mon[98204]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-07-21T18:35:00.385 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:59 smithi193 ceph-mon[98204]: Cluster is now healthy 2023-07-21T18:35:00.385 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:59 smithi193 ceph-mon[98204]: overall HEALTH_OK 2023-07-21T18:35:00.385 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:59 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:00.386 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:59 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:00.386 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:59 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:00.386 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:59 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:00.386 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:59 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:00.386 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:59 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:00.386 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:59 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:00.386 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:59 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T18:35:00.386 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:59 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-21T18:35:00.387 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:34:59 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:35:00.401 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:59 smithi155 ceph-mon[97934]: mon.b calling monitor election 2023-07-21T18:35:00.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:59 smithi155 ceph-mon[97934]: Updating smithi039:/etc/ceph/ceph.conf 2023-07-21T18:35:00.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:59 smithi155 ceph-mon[97934]: Updating smithi155:/etc/ceph/ceph.conf 2023-07-21T18:35:00.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:59 smithi155 ceph-mon[97934]: Updating smithi193:/etc/ceph/ceph.conf 2023-07-21T18:35:00.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:59 smithi155 ceph-mon[97934]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:35:00.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:59 smithi155 ceph-mon[97934]: mon.b calling monitor election 2023-07-21T18:35:00.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:59 smithi155 ceph-mon[97934]: mon.a calling monitor election 2023-07-21T18:35:00.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:59 smithi155 ceph-mon[97934]: mon.c calling monitor election 2023-07-21T18:35:00.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:59 smithi155 ceph-mon[97934]: Updating smithi155:/var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T18:35:00.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:59 smithi155 ceph-mon[97934]: Updating smithi039:/var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T18:35:00.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:59 smithi155 ceph-mon[97934]: Updating smithi193:/var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T18:35:00.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:59 smithi155 ceph-mon[97934]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-07-21T18:35:00.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:59 smithi155 ceph-mon[97934]: monmap e3: 3 mons at {a=[v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0],b=[v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0],c=[v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0]} removed_ranks: {} 2023-07-21T18:35:00.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:59 smithi155 ceph-mon[97934]: fsmap 2023-07-21T18:35:00.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:59 smithi155 ceph-mon[97934]: osdmap e4: 0 total, 0 up, 0 in 2023-07-21T18:35:00.404 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:59 smithi155 ceph-mon[97934]: mgrmap e13: x(active, since 52s) 2023-07-21T18:35:00.404 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:59 smithi155 ceph-mon[97934]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-07-21T18:35:00.404 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:59 smithi155 ceph-mon[97934]: Cluster is now healthy 2023-07-21T18:35:00.404 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:59 smithi155 ceph-mon[97934]: overall HEALTH_OK 2023-07-21T18:35:00.404 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:59 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:00.404 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:59 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:00.404 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:59 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:00.405 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:59 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:00.405 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:59 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:00.405 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:59 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:00.405 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:59 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:00.405 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:59 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T18:35:00.405 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:59 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-21T18:35:00.405 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:34:59 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:35:00.438 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:00.399+0000 7f6a92991700 1 -- 172.21.15.39:0/3312502684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a8c105620 msgr2=0x7f6a8c1059f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:35:00.439 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:00.399+0000 7f6a92991700 1 --2- 172.21.15.39:0/3312502684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a8c105620 0x7f6a8c1059f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c009910 tx=0x7f6a7c009c20 comp rx=0 tx=0).stop 2023-07-21T18:35:00.439 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:00.399+0000 7f6a92991700 1 -- 172.21.15.39:0/3312502684 shutdown_connections 2023-07-21T18:35:00.439 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:00.399+0000 7f6a92991700 1 --2- 172.21.15.39:0/3312502684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a8c105620 0x7f6a8c1059f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:35:00.439 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:00.399+0000 7f6a92991700 1 -- 172.21.15.39:0/3312502684 >> 172.21.15.39:0/3312502684 conn(0x7f6a8c0fb100 msgr2=0x7f6a8c0fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:35:00.440 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:00.399+0000 7f6a92991700 1 -- 172.21.15.39:0/3312502684 shutdown_connections 2023-07-21T18:35:00.441 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:00.400+0000 7f6a92991700 1 -- 172.21.15.39:0/3312502684 wait complete. 2023-07-21T18:35:00.441 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:00.400+0000 7f6a92991700 1 Processor -- start 2023-07-21T18:35:00.441 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:00.400+0000 7f6a92991700 1 -- start start 2023-07-21T18:35:00.442 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:00.400+0000 7f6a92991700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a8c105620 0x7f6a8c19a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:35:00.442 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:00.400+0000 7f6a92991700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a8c19ae70 0x7f6a8c1949b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:35:00.442 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:00.400+0000 7f6a92991700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a8c194ef0 0x7f6a8c195390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:35:00.442 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:00.401+0000 7f6a92991700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8c19b480 con 0x7f6a8c194ef0 2023-07-21T18:35:00.443 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:00.401+0000 7f6a92991700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8c1a1750 con 0x7f6a8c19ae70 2023-07-21T18:35:00.443 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:00.401+0000 7f6a92991700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8c1a1890 con 0x7f6a8c105620 2023-07-21T18:35:00.443 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:00.401+0000 7f6a90f2e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a8c194ef0 0x7f6a8c195390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:35:00.443 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:00.401+0000 7f6a90f2e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a8c194ef0 0x7f6a8c195390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54112/0 (socket says 172.21.15.39:54112) 2023-07-21T18:35:00.443 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:00.401+0000 7f6a90f2e700 1 -- 172.21.15.39:0/1607323814 learned_addr learned my addr 172.21.15.39:0/1607323814 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:35:00.443 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:00.401+0000 7f6a8bfff700 1 --2- 172.21.15.39:0/1607323814 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a8c105620 0x7f6a8c19a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:35:00.443 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:00.401+0000 7f6a90f2e700 1 -- 172.21.15.39:0/1607323814 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a8c105620 msgr2=0x7f6a8c19a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:35:00.444 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:00.401+0000 7f6a90f2e700 1 --2- 172.21.15.39:0/1607323814 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a8c105620 0x7f6a8c19a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:35:00.444 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:00.401+0000 7f6a90f2e700 1 -- 172.21.15.39:0/1607323814 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a8c19ae70 msgr2=0x7f6a8c1949b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:35:00.444 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:00.401+0000 7f6a90f2e700 1 --2- 172.21.15.39:0/1607323814 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a8c19ae70 0x7f6a8c1949b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:35:00.444 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:00.401+0000 7f6a90f2e700 1 -- 172.21.15.39:0/1607323814 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a7c0095f0 con 0x7f6a8c194ef0 2023-07-21T18:35:00.444 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:00.401+0000 7f6a90f2e700 1 --2- 172.21.15.39:0/1607323814 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a8c194ef0 0x7f6a8c195390 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6a8400ba50 tx=0x7f6a8400be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:35:00.445 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:00.401+0000 7f6a897fa700 1 -- 172.21.15.39:0/1607323814 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8400cab0 con 0x7f6a8c194ef0 2023-07-21T18:35:00.445 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:00.401+0000 7f6a897fa700 1 -- 172.21.15.39:0/1607323814 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a8400cc10 con 0x7f6a8c194ef0 2023-07-21T18:35:00.445 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:00.401+0000 7f6a92991700 1 -- 172.21.15.39:0/1607323814 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a8c1a1b70 con 0x7f6a8c194ef0 2023-07-21T18:35:00.445 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:00.401+0000 7f6a897fa700 1 -- 172.21.15.39:0/1607323814 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a840075b0 con 0x7f6a8c194ef0 2023-07-21T18:35:00.446 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:00.401+0000 7f6a92991700 1 -- 172.21.15.39:0/1607323814 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a8c1a2020 con 0x7f6a8c194ef0 2023-07-21T18:35:00.446 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:00.402+0000 7f6a897fa700 1 -- 172.21.15.39:0/1607323814 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f6a84007710 con 0x7f6a8c194ef0 2023-07-21T18:35:00.446 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:00.402+0000 7f6a92991700 1 -- 172.21.15.39:0/1607323814 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a8c061b40 con 0x7f6a8c194ef0 2023-07-21T18:35:00.447 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:00.402+0000 7f6a897fa700 1 --2- 172.21.15.39:0/1607323814 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f6a74038c80 0x7f6a7403b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:35:00.447 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:00.402+0000 7f6a897fa700 1 -- 172.21.15.39:0/1607323814 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6a8404ab70 con 0x7f6a8c194ef0 2023-07-21T18:35:00.447 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:00.403+0000 7f6a8bfff700 1 --2- 172.21.15.39:0/1607323814 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f6a74038c80 0x7f6a7403b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:35:00.447 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:00.403+0000 7f6a8bfff700 1 --2- 172.21.15.39:0/1607323814 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f6a74038c80 0x7f6a7403b130 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c006010 tx=0x7f6a7c00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:35:00.448 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:00.407+0000 7f6a897fa700 1 -- 172.21.15.39:0/1607323814 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6a8401b8b0 con 0x7f6a8c194ef0 2023-07-21T18:35:00.652 INFO:teuthology.orchestra.run.smithi039.stdout:# minimal ceph.conf for eaf8e25a-27f4-11ee-9b34-001a4aab830c 2023-07-21T18:35:00.652 INFO:teuthology.orchestra.run.smithi039.stdout:[global] 2023-07-21T18:35:00.652 INFO:teuthology.orchestra.run.smithi039.stdout: fsid = eaf8e25a-27f4-11ee-9b34-001a4aab830c 2023-07-21T18:35:00.652 INFO:teuthology.orchestra.run.smithi039.stdout: mon_host = [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] 2023-07-21T18:35:00.653 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:00.610+0000 7f6a92991700 1 -- 172.21.15.39:0/1607323814 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f6a8c195ec0 con 0x7f6a8c194ef0 2023-07-21T18:35:00.653 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:00.611+0000 7f6a897fa700 1 -- 172.21.15.39:0/1607323814 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+275 (secure 0 0 0) 0x7f6a8401b2c0 con 0x7f6a8c194ef0 2023-07-21T18:35:00.653 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:00.613+0000 7f6a92991700 1 -- 172.21.15.39:0/1607323814 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f6a74038c80 msgr2=0x7f6a7403b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:35:00.653 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:00.613+0000 7f6a92991700 1 --2- 172.21.15.39:0/1607323814 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f6a74038c80 0x7f6a7403b130 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c006010 tx=0x7f6a7c00b540 comp rx=0 tx=0).stop 2023-07-21T18:35:00.653 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:00.613+0000 7f6a92991700 1 -- 172.21.15.39:0/1607323814 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a8c194ef0 msgr2=0x7f6a8c195390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:35:00.653 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:00.613+0000 7f6a92991700 1 --2- 172.21.15.39:0/1607323814 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a8c194ef0 0x7f6a8c195390 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6a8400ba50 tx=0x7f6a8400be10 comp rx=0 tx=0).stop 2023-07-21T18:35:00.654 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:00.613+0000 7f6a92991700 1 -- 172.21.15.39:0/1607323814 shutdown_connections 2023-07-21T18:35:00.654 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:00.613+0000 7f6a92991700 1 --2- 172.21.15.39:0/1607323814 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f6a74038c80 0x7f6a7403b130 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:35:00.654 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:00.613+0000 7f6a92991700 1 --2- 172.21.15.39:0/1607323814 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a8c194ef0 0x7f6a8c195390 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:35:00.654 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:00.613+0000 7f6a92991700 1 --2- 172.21.15.39:0/1607323814 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a8c105620 0x7f6a8c19a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:35:00.654 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:00.613+0000 7f6a92991700 1 --2- 172.21.15.39:0/1607323814 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a8c19ae70 0x7f6a8c1949b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:35:00.654 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:00.613+0000 7f6a92991700 1 -- 172.21.15.39:0/1607323814 >> 172.21.15.39:0/1607323814 conn(0x7f6a8c0fb100 msgr2=0x7f6a8c0fb4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:35:00.654 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:00.613+0000 7f6a92991700 1 -- 172.21.15.39:0/1607323814 shutdown_connections 2023-07-21T18:35:00.655 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:00.614+0000 7f6a92991700 1 -- 172.21.15.39:0/1607323814 wait complete. 2023-07-21T18:35:01.175 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:00 smithi039 ceph-mon[96489]: Reconfiguring mon.a (unknown last config time)... 2023-07-21T18:35:01.175 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:00 smithi039 ceph-mon[96489]: Reconfiguring daemon mon.a on smithi039 2023-07-21T18:35:01.175 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:00 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T18:35:01.175 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:00 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/1607323814' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:35:01.274 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2023-07-21T18:35:01.274 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-07-21T18:35:01.275 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/etc/ceph/ceph.conf 2023-07-21T18:35:01.344 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-07-21T18:35:01.344 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-21T18:35:01.382 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:00 smithi193 ceph-mon[98204]: Reconfiguring mon.a (unknown last config time)... 2023-07-21T18:35:01.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:00 smithi193 ceph-mon[98204]: Reconfiguring daemon mon.a on smithi039 2023-07-21T18:35:01.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:00 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T18:35:01.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:00 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/1607323814' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:35:01.401 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:00 smithi155 ceph-mon[97934]: Reconfiguring mon.a (unknown last config time)... 2023-07-21T18:35:01.401 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:00 smithi155 ceph-mon[97934]: Reconfiguring daemon mon.a on smithi039 2023-07-21T18:35:01.401 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:00 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T18:35:01.401 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:00 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/1607323814' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:35:01.412 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-07-21T18:35:01.412 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/ceph/ceph.conf 2023-07-21T18:35:01.446 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-07-21T18:35:01.447 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-21T18:35:01.516 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2023-07-21T18:35:01.517 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/ceph/ceph.conf 2023-07-21T18:35:01.553 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2023-07-21T18:35:01.553 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-21T18:35:01.625 INFO:tasks.cephadm:Adding mgr.x on smithi039 2023-07-21T18:35:01.625 INFO:tasks.cephadm:Adding mgr.y on smithi155 2023-07-21T18:35:01.626 INFO:tasks.cephadm:Adding mgr.z on smithi193 2023-07-21T18:35:01.626 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph orch apply mgr '3;smithi039=x;smithi155=y;smithi193=z' 2023-07-21T18:35:02.146 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:01 smithi155 ceph-mon[97934]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:35:02.146 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:01 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:02.146 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:01 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:02.146 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:01 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T18:35:02.147 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:01 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-21T18:35:02.147 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:01 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:35:02.218 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T18:35:02.379 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:01 smithi193 ceph-mon[98204]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:35:02.379 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:01 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:02.379 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:01 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:02.379 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:01 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T18:35:02.379 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:01 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-21T18:35:02.380 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:01 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:35:02.450 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:01 smithi039 ceph-mon[96489]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:35:02.450 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:01 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:02.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:01 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:02.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:01 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T18:35:02.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:01 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-21T18:35:02.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:01 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:35:03.204 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:02 smithi193 ceph-mon[98204]: Reconfiguring mon.b (monmap changed)... 2023-07-21T18:35:03.204 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:02 smithi193 ceph-mon[98204]: Reconfiguring daemon mon.b on smithi155 2023-07-21T18:35:03.204 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:02 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:03.205 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:02 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:03.205 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:02 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:03.205 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:02 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T18:35:03.205 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:02 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-21T18:35:03.205 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:02 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:35:03.401 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:02 smithi155 ceph-mon[97934]: Reconfiguring mon.b (monmap changed)... 2023-07-21T18:35:03.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:02 smithi155 ceph-mon[97934]: Reconfiguring daemon mon.b on smithi155 2023-07-21T18:35:03.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:02 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:03.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:02 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:03.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:02 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:03.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:02 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T18:35:03.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:02 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-21T18:35:03.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:02 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:35:03.411 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:03.375+0000 7fca498aa700 1 -- 172.21.15.193:0/3556875225 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca440fe600 msgr2=0x7fca440fea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:35:03.411 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:03.375+0000 7fca498aa700 1 --2- 172.21.15.193:0/3556875225 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca440fe600 0x7fca440fea10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fca38009910 tx=0x7fca38009c20 comp rx=0 tx=0).stop 2023-07-21T18:35:03.412 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:03.375+0000 7fca498aa700 1 -- 172.21.15.193:0/3556875225 shutdown_connections 2023-07-21T18:35:03.412 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:03.375+0000 7fca498aa700 1 --2- 172.21.15.193:0/3556875225 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca440fe600 0x7fca440fea10 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:35:03.412 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:03.375+0000 7fca498aa700 1 -- 172.21.15.193:0/3556875225 >> 172.21.15.193:0/3556875225 conn(0x7fca440f9bb0 msgr2=0x7fca440fbfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:35:03.412 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:03.375+0000 7fca498aa700 1 -- 172.21.15.193:0/3556875225 shutdown_connections 2023-07-21T18:35:03.412 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:03.376+0000 7fca498aa700 1 -- 172.21.15.193:0/3556875225 wait complete. 2023-07-21T18:35:03.412 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:03.376+0000 7fca498aa700 1 Processor -- start 2023-07-21T18:35:03.414 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:03.376+0000 7fca498aa700 1 -- start start 2023-07-21T18:35:03.414 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:03.376+0000 7fca498aa700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca440fe600 0x7fca441938e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:35:03.414 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:03.376+0000 7fca498aa700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca44193e20 0x7fca44198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:35:03.414 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:03.376+0000 7fca498aa700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fca44194290 0x7fca4419b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:35:03.414 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:03.376+0000 7fca498aa700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4419bba0 con 0x7fca440fe600 2023-07-21T18:35:03.415 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:03.376+0000 7fca498aa700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4419bd10 con 0x7fca44193e20 2023-07-21T18:35:03.415 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:03.376+0000 7fca498aa700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4419bed0 con 0x7fca44194290 2023-07-21T18:35:03.415 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:03.377+0000 7fca42ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca440fe600 0x7fca441938e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:35:03.415 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:03.377+0000 7fca42ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca440fe600 0x7fca441938e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.193:60268/0 (socket says 172.21.15.193:60268) 2023-07-21T18:35:03.415 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:03.377+0000 7fca42ffd700 1 -- 172.21.15.193:0/700922436 learned_addr learned my addr 172.21.15.193:0/700922436 (peer_addr_for_me v2:172.21.15.193:0/0) 2023-07-21T18:35:03.415 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:03.377+0000 7fca427fc700 1 --2- 172.21.15.193:0/700922436 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca44193e20 0x7fca44198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:35:03.415 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:03.377+0000 7fca437fe700 1 --2- 172.21.15.193:0/700922436 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fca44194290 0x7fca4419b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:35:03.416 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:03.377+0000 7fca42ffd700 1 -- 172.21.15.193:0/700922436 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fca44194290 msgr2=0x7fca4419b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:35:03.416 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:03.377+0000 7fca42ffd700 1 --2- 172.21.15.193:0/700922436 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fca44194290 0x7fca4419b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:35:03.416 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:03.377+0000 7fca42ffd700 1 -- 172.21.15.193:0/700922436 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca44193e20 msgr2=0x7fca44198e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:35:03.416 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:03.377+0000 7fca42ffd700 1 --2- 172.21.15.193:0/700922436 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca44193e20 0x7fca44198e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:35:03.416 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:03.377+0000 7fca42ffd700 1 -- 172.21.15.193:0/700922436 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca380095f0 con 0x7fca440fe600 2023-07-21T18:35:03.416 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:03.377+0000 7fca42ffd700 1 --2- 172.21.15.193:0/700922436 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca440fe600 0x7fca441938e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fca38006010 tx=0x7fca38004cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:35:03.416 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:03.377+0000 7fca488a8700 1 -- 172.21.15.193:0/700922436 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3801d070 con 0x7fca440fe600 2023-07-21T18:35:03.417 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:03.378+0000 7fca488a8700 1 -- 172.21.15.193:0/700922436 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca3800be20 con 0x7fca440fe600 2023-07-21T18:35:03.417 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:03.378+0000 7fca498aa700 1 -- 172.21.15.193:0/700922436 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca4419c150 con 0x7fca440fe600 2023-07-21T18:35:03.417 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:03.378+0000 7fca488a8700 1 -- 172.21.15.193:0/700922436 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca38022e40 con 0x7fca440fe600 2023-07-21T18:35:03.417 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:03.378+0000 7fca498aa700 1 -- 172.21.15.193:0/700922436 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca4419c670 con 0x7fca440fe600 2023-07-21T18:35:03.417 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:03.379+0000 7fca488a8700 1 -- 172.21.15.193:0/700922436 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fca3800f460 con 0x7fca440fe600 2023-07-21T18:35:03.417 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:03.379+0000 7fca498aa700 1 -- 172.21.15.193:0/700922436 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca4418d1d0 con 0x7fca440fe600 2023-07-21T18:35:03.417 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:03.379+0000 7fca488a8700 1 --2- 172.21.15.193:0/700922436 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fca2c038c80 0x7fca2c03b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:35:03.417 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:03.379+0000 7fca488a8700 1 -- 172.21.15.193:0/700922436 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fca3804c9a0 con 0x7fca440fe600 2023-07-21T18:35:03.418 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:03.379+0000 7fca427fc700 1 --2- 172.21.15.193:0/700922436 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fca2c038c80 0x7fca2c03b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:35:03.418 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:03.380+0000 7fca427fc700 1 --2- 172.21.15.193:0/700922436 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fca2c038c80 0x7fca2c03b130 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fca34009ad0 tx=0x7fca34009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:35:03.419 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:03.384+0000 7fca488a8700 1 -- 172.21.15.193:0/700922436 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fca38020960 con 0x7fca440fe600 2023-07-21T18:35:03.450 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:02 smithi039 ceph-mon[96489]: Reconfiguring mon.b (monmap changed)... 2023-07-21T18:35:03.450 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:02 smithi039 ceph-mon[96489]: Reconfiguring daemon mon.b on smithi155 2023-07-21T18:35:03.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:02 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:03.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:02 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:03.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:02 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:03.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:02 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T18:35:03.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:02 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-21T18:35:03.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:02 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:35:03.618 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:03.583+0000 7fca498aa700 1 -- 172.21.15.193:0/700922436 --> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi039=x;smithi155=y;smithi193=z", "target": ["mon-mgr", ""]}) v1 -- 0x7fca440609e0 con 0x7fca2c038c80 2023-07-21T18:35:03.627 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:03.592+0000 7fca488a8700 1 -- 172.21.15.193:0/700922436 <== mgr.14152 v2:172.21.15.39:6800/2116291712 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fca440609e0 con 0x7fca2c038c80 2023-07-21T18:35:03.627 INFO:teuthology.orchestra.run.smithi193.stdout:Scheduled mgr update... 2023-07-21T18:35:03.629 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:03.594+0000 7fca498aa700 1 -- 172.21.15.193:0/700922436 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fca2c038c80 msgr2=0x7fca2c03b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:35:03.629 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:03.594+0000 7fca498aa700 1 --2- 172.21.15.193:0/700922436 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fca2c038c80 0x7fca2c03b130 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fca34009ad0 tx=0x7fca34009210 comp rx=0 tx=0).stop 2023-07-21T18:35:03.629 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:03.594+0000 7fca498aa700 1 -- 172.21.15.193:0/700922436 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca440fe600 msgr2=0x7fca441938e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:35:03.629 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:03.594+0000 7fca498aa700 1 --2- 172.21.15.193:0/700922436 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca440fe600 0x7fca441938e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fca38006010 tx=0x7fca38004cb0 comp rx=0 tx=0).stop 2023-07-21T18:35:03.630 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:03.594+0000 7fca498aa700 1 -- 172.21.15.193:0/700922436 shutdown_connections 2023-07-21T18:35:03.630 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:03.594+0000 7fca498aa700 1 --2- 172.21.15.193:0/700922436 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fca2c038c80 0x7fca2c03b130 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:35:03.630 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:03.594+0000 7fca498aa700 1 --2- 172.21.15.193:0/700922436 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fca44194290 0x7fca4419b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:35:03.630 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:03.594+0000 7fca498aa700 1 --2- 172.21.15.193:0/700922436 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca440fe600 0x7fca441938e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:35:03.630 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:03.594+0000 7fca498aa700 1 --2- 172.21.15.193:0/700922436 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca44193e20 0x7fca44198e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:35:03.630 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:03.594+0000 7fca498aa700 1 -- 172.21.15.193:0/700922436 >> 172.21.15.193:0/700922436 conn(0x7fca440f9bb0 msgr2=0x7fca440fa880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:35:03.630 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:03.594+0000 7fca498aa700 1 -- 172.21.15.193:0/700922436 shutdown_connections 2023-07-21T18:35:03.631 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:03.594+0000 7fca498aa700 1 -- 172.21.15.193:0/700922436 wait complete. 2023-07-21T18:35:04.073 DEBUG:teuthology.orchestra.run.smithi155:mgr.y> sudo journalctl -f -n 0 -u ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@mgr.y.service 2023-07-21T18:35:04.076 DEBUG:teuthology.orchestra.run.smithi193:mgr.z> sudo journalctl -f -n 0 -u ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@mgr.z.service 2023-07-21T18:35:04.078 INFO:tasks.cephadm:Deploying OSDs... 2023-07-21T18:35:04.078 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-07-21T18:35:04.078 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/scratch_devs of=/dev/stdout 2023-07-21T18:35:04.100 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-07-21T18:35:04.100 DEBUG:teuthology.orchestra.run.smithi039:> stat /dev/vg_nvme/lv_1 2023-07-21T18:35:04.106 INFO:journalctl@ceph.mgr.z.smithi193.stdout:-- Logs begin at Fri 2023-07-21 18:17:09 UTC. -- 2023-07-21T18:35:04.111 INFO:journalctl@ceph.mgr.y.smithi155.stdout:-- Logs begin at Fri 2023-07-21 18:17:11 UTC. -- 2023-07-21T18:35:04.160 INFO:teuthology.orchestra.run.smithi039.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-07-21T18:35:04.161 INFO:teuthology.orchestra.run.smithi039.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-21T18:35:04.161 INFO:teuthology.orchestra.run.smithi039.stdout:Device: 6h/6d Inode: 382089 Links: 1 2023-07-21T18:35:04.161 INFO:teuthology.orchestra.run.smithi039.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-21T18:35:04.161 INFO:teuthology.orchestra.run.smithi039.stdout:Context: system_u:object_r:device_t:s0 2023-07-21T18:35:04.162 INFO:teuthology.orchestra.run.smithi039.stdout:Access: 2023-07-21 18:34:17.539163480 +0000 2023-07-21T18:35:04.162 INFO:teuthology.orchestra.run.smithi039.stdout:Modify: 2023-07-21 18:28:27.110150107 +0000 2023-07-21T18:35:04.162 INFO:teuthology.orchestra.run.smithi039.stdout:Change: 2023-07-21 18:28:27.110150107 +0000 2023-07-21T18:35:04.162 INFO:teuthology.orchestra.run.smithi039.stdout: Birth: - 2023-07-21T18:35:04.162 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-07-21T18:35:04.231 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records in 2023-07-21T18:35:04.231 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records out 2023-07-21T18:35:04.232 INFO:teuthology.orchestra.run.smithi039.stderr:512 bytes copied, 0.000475621 s, 1.1 MB/s 2023-07-21T18:35:04.233 DEBUG:teuthology.orchestra.run.smithi039:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-07-21T18:35:04.292 DEBUG:teuthology.orchestra.run.smithi039:> stat /dev/vg_nvme/lv_2 2023-07-21T18:35:04.351 INFO:teuthology.orchestra.run.smithi039.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-07-21T18:35:04.352 INFO:teuthology.orchestra.run.smithi039.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-21T18:35:04.352 INFO:teuthology.orchestra.run.smithi039.stdout:Device: 6h/6d Inode: 380536 Links: 1 2023-07-21T18:35:04.352 INFO:teuthology.orchestra.run.smithi039.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-21T18:35:04.352 INFO:teuthology.orchestra.run.smithi039.stdout:Context: system_u:object_r:device_t:s0 2023-07-21T18:35:04.352 INFO:teuthology.orchestra.run.smithi039.stdout:Access: 2023-07-21 18:34:17.539163480 +0000 2023-07-21T18:35:04.353 INFO:teuthology.orchestra.run.smithi039.stdout:Modify: 2023-07-21 18:28:27.649139956 +0000 2023-07-21T18:35:04.353 INFO:teuthology.orchestra.run.smithi039.stdout:Change: 2023-07-21 18:28:27.649139956 +0000 2023-07-21T18:35:04.353 INFO:teuthology.orchestra.run.smithi039.stdout: Birth: - 2023-07-21T18:35:04.353 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-07-21T18:35:04.382 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:03 smithi193 ceph-mon[98204]: Reconfiguring mon.c (monmap changed)... 2023-07-21T18:35:04.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:03 smithi193 ceph-mon[98204]: Reconfiguring daemon mon.c on smithi193 2023-07-21T18:35:04.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:03 smithi193 ceph-mon[98204]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:35:04.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:03 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:04.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:03 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:04.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:03 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:04.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:03 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:35:04.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:03 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:35:04.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:03 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:35:04.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:03 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:04.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:03 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-21T18:35:04.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:03 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-21T18:35:04.385 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:03 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-21T18:35:04.385 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:03 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:35:04.401 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:03 smithi155 ceph-mon[97934]: Reconfiguring mon.c (monmap changed)... 2023-07-21T18:35:04.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:03 smithi155 ceph-mon[97934]: Reconfiguring daemon mon.c on smithi193 2023-07-21T18:35:04.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:03 smithi155 ceph-mon[97934]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:35:04.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:03 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:04.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:03 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:04.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:03 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:04.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:03 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:35:04.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:03 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:35:04.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:03 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:35:04.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:03 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:04.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:03 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-21T18:35:04.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:03 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-21T18:35:04.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:03 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-21T18:35:04.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:03 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:35:04.417 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:03 smithi039 ceph-mon[96489]: Reconfiguring mon.c (monmap changed)... 2023-07-21T18:35:04.417 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:03 smithi039 ceph-mon[96489]: Reconfiguring daemon mon.c on smithi193 2023-07-21T18:35:04.417 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:03 smithi039 ceph-mon[96489]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:35:04.418 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:03 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:04.418 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:03 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:04.418 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:03 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:04.418 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:03 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:35:04.418 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:03 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:35:04.418 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:03 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:35:04.418 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:03 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:04.418 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:03 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-21T18:35:04.419 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:03 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-21T18:35:04.419 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:03 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-21T18:35:04.419 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:03 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:35:04.419 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records in 2023-07-21T18:35:04.419 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records out 2023-07-21T18:35:04.420 INFO:teuthology.orchestra.run.smithi039.stderr:512 bytes copied, 0.0002946 s, 1.7 MB/s 2023-07-21T18:35:04.420 DEBUG:teuthology.orchestra.run.smithi039:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-07-21T18:35:04.480 DEBUG:teuthology.orchestra.run.smithi039:> stat /dev/vg_nvme/lv_3 2023-07-21T18:35:04.538 INFO:teuthology.orchestra.run.smithi039.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-07-21T18:35:04.539 INFO:teuthology.orchestra.run.smithi039.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-21T18:35:04.539 INFO:teuthology.orchestra.run.smithi039.stdout:Device: 6h/6d Inode: 378370 Links: 1 2023-07-21T18:35:04.539 INFO:teuthology.orchestra.run.smithi039.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-21T18:35:04.539 INFO:teuthology.orchestra.run.smithi039.stdout:Context: system_u:object_r:device_t:s0 2023-07-21T18:35:04.539 INFO:teuthology.orchestra.run.smithi039.stdout:Access: 2023-07-21 18:34:17.539163480 +0000 2023-07-21T18:35:04.539 INFO:teuthology.orchestra.run.smithi039.stdout:Modify: 2023-07-21 18:28:28.158130370 +0000 2023-07-21T18:35:04.539 INFO:teuthology.orchestra.run.smithi039.stdout:Change: 2023-07-21 18:28:28.158130370 +0000 2023-07-21T18:35:04.540 INFO:teuthology.orchestra.run.smithi039.stdout: Birth: - 2023-07-21T18:35:04.540 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-07-21T18:35:04.605 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records in 2023-07-21T18:35:04.606 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records out 2023-07-21T18:35:04.606 INFO:teuthology.orchestra.run.smithi039.stderr:512 bytes copied, 0.000296502 s, 1.7 MB/s 2023-07-21T18:35:04.607 DEBUG:teuthology.orchestra.run.smithi039:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-07-21T18:35:04.665 DEBUG:teuthology.orchestra.run.smithi039:> stat /dev/vg_nvme/lv_4 2023-07-21T18:35:04.723 INFO:teuthology.orchestra.run.smithi039.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-07-21T18:35:04.723 INFO:teuthology.orchestra.run.smithi039.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-21T18:35:04.724 INFO:teuthology.orchestra.run.smithi039.stdout:Device: 6h/6d Inode: 381170 Links: 1 2023-07-21T18:35:04.724 INFO:teuthology.orchestra.run.smithi039.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-21T18:35:04.724 INFO:teuthology.orchestra.run.smithi039.stdout:Context: system_u:object_r:device_t:s0 2023-07-21T18:35:04.724 INFO:teuthology.orchestra.run.smithi039.stdout:Access: 2023-07-21 18:34:17.539163480 +0000 2023-07-21T18:35:04.724 INFO:teuthology.orchestra.run.smithi039.stdout:Modify: 2023-07-21 18:28:28.695120258 +0000 2023-07-21T18:35:04.724 INFO:teuthology.orchestra.run.smithi039.stdout:Change: 2023-07-21 18:28:28.695120258 +0000 2023-07-21T18:35:04.724 INFO:teuthology.orchestra.run.smithi039.stdout: Birth: - 2023-07-21T18:35:04.725 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-07-21T18:35:04.790 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records in 2023-07-21T18:35:04.790 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records out 2023-07-21T18:35:04.790 INFO:teuthology.orchestra.run.smithi039.stderr:512 bytes copied, 0.000370193 s, 1.4 MB/s 2023-07-21T18:35:04.791 DEBUG:teuthology.orchestra.run.smithi039:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-07-21T18:35:04.849 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-07-21T18:35:04.850 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/scratch_devs of=/dev/stdout 2023-07-21T18:35:04.870 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-07-21T18:35:04.871 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_1 2023-07-21T18:35:04.930 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-07-21T18:35:04.931 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-21T18:35:04.931 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 6h/6d Inode: 374745 Links: 1 2023-07-21T18:35:04.931 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-21T18:35:04.931 INFO:teuthology.orchestra.run.smithi155.stdout:Context: system_u:object_r:device_t:s0 2023-07-21T18:35:04.931 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2023-07-21 18:34:51.507394459 +0000 2023-07-21T18:35:04.931 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2023-07-21 18:28:30.676142058 +0000 2023-07-21T18:35:04.932 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2023-07-21 18:28:30.676142058 +0000 2023-07-21T18:35:04.932 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: - 2023-07-21T18:35:04.932 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-07-21T18:35:05.001 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2023-07-21T18:35:05.001 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2023-07-21T18:35:05.001 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000265115 s, 1.9 MB/s 2023-07-21T18:35:05.003 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-07-21T18:35:05.063 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_2 2023-07-21T18:35:05.122 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-07-21T18:35:05.122 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-21T18:35:05.122 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 6h/6d Inode: 380936 Links: 1 2023-07-21T18:35:05.123 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-21T18:35:05.123 INFO:teuthology.orchestra.run.smithi155.stdout:Context: system_u:object_r:device_t:s0 2023-07-21T18:35:05.123 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2023-07-21 18:34:51.507394459 +0000 2023-07-21T18:35:05.123 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2023-07-21 18:28:31.185131627 +0000 2023-07-21T18:35:05.123 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2023-07-21 18:28:31.185131627 +0000 2023-07-21T18:35:05.123 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: - 2023-07-21T18:35:05.123 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-07-21T18:35:05.189 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2023-07-21T18:35:05.190 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2023-07-21T18:35:05.190 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000226005 s, 2.3 MB/s 2023-07-21T18:35:05.191 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-07-21T18:35:05.200 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:04 smithi039 ceph-mon[96489]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi039=x;smithi155=y;smithi193=z", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:35:05.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:04 smithi039 ceph-mon[96489]: Saving service mgr spec with placement smithi039=x;smithi155=y;smithi193=z;count:3 2023-07-21T18:35:05.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:04 smithi039 ceph-mon[96489]: Deploying daemon mgr.z on smithi193 2023-07-21T18:35:05.250 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_3 2023-07-21T18:35:05.308 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-07-21T18:35:05.309 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-21T18:35:05.309 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 6h/6d Inode: 380984 Links: 1 2023-07-21T18:35:05.309 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-21T18:35:05.309 INFO:teuthology.orchestra.run.smithi155.stdout:Context: system_u:object_r:device_t:s0 2023-07-21T18:35:05.309 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2023-07-21 18:34:51.507394459 +0000 2023-07-21T18:35:05.310 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2023-07-21 18:28:31.724120581 +0000 2023-07-21T18:35:05.310 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2023-07-21 18:28:31.724120581 +0000 2023-07-21T18:35:05.310 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: - 2023-07-21T18:35:05.310 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-07-21T18:35:05.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:04 smithi155 ceph-mon[97934]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi039=x;smithi155=y;smithi193=z", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:35:05.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:04 smithi155 ceph-mon[97934]: Saving service mgr spec with placement smithi039=x;smithi155=y;smithi193=z;count:3 2023-07-21T18:35:05.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:04 smithi155 ceph-mon[97934]: Deploying daemon mgr.z on smithi193 2023-07-21T18:35:05.374 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2023-07-21T18:35:05.375 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2023-07-21T18:35:05.375 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000244047 s, 2.1 MB/s 2023-07-21T18:35:05.376 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-07-21T18:35:05.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:04 smithi193 ceph-mon[98204]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi039=x;smithi155=y;smithi193=z", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:35:05.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:04 smithi193 ceph-mon[98204]: Saving service mgr spec with placement smithi039=x;smithi155=y;smithi193=z;count:3 2023-07-21T18:35:05.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:04 smithi193 ceph-mon[98204]: Deploying daemon mgr.z on smithi193 2023-07-21T18:35:05.383 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Jul 21 18:35:05 smithi193 systemd[1]: Starting Ceph mgr.z for eaf8e25a-27f4-11ee-9b34-001a4aab830c... 2023-07-21T18:35:05.435 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_4 2023-07-21T18:35:05.493 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-07-21T18:35:05.494 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-21T18:35:05.494 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 6h/6d Inode: 384016 Links: 1 2023-07-21T18:35:05.494 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-21T18:35:05.494 INFO:teuthology.orchestra.run.smithi155.stdout:Context: system_u:object_r:device_t:s0 2023-07-21T18:35:05.495 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2023-07-21 18:34:51.507394459 +0000 2023-07-21T18:35:05.495 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2023-07-21 18:28:32.236110088 +0000 2023-07-21T18:35:05.495 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2023-07-21 18:28:32.236110088 +0000 2023-07-21T18:35:05.496 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: - 2023-07-21T18:35:05.496 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-07-21T18:35:05.559 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2023-07-21T18:35:05.560 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2023-07-21T18:35:05.560 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000277572 s, 1.8 MB/s 2023-07-21T18:35:05.561 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-07-21T18:35:05.618 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2023-07-21T18:35:05.619 DEBUG:teuthology.orchestra.run.smithi193:> dd if=/scratch_devs of=/dev/stdout 2023-07-21T18:35:05.636 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-07-21T18:35:05.637 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_1 2023-07-21T18:35:05.692 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-07-21T18:35:05.693 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-21T18:35:05.693 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 6h/6d Inode: 371455 Links: 1 2023-07-21T18:35:05.693 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-21T18:35:05.693 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2023-07-21T18:35:05.693 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2023-07-21 18:35:05.103133970 +0000 2023-07-21T18:35:05.694 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2023-07-21 18:28:12.991704513 +0000 2023-07-21T18:35:05.694 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2023-07-21 18:28:12.991704513 +0000 2023-07-21T18:35:05.694 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: - 2023-07-21T18:35:05.694 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-07-21T18:35:05.761 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Jul 21 18:35:05 smithi193 podman[100056]: 2023-07-21T18:35:05.763 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2023-07-21T18:35:05.763 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2023-07-21T18:35:05.763 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.000243129 s, 2.1 MB/s 2023-07-21T18:35:05.765 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-07-21T18:35:05.823 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_2 2023-07-21T18:35:05.880 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-07-21T18:35:05.881 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-21T18:35:05.881 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 6h/6d Inode: 380699 Links: 1 2023-07-21T18:35:05.881 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-21T18:35:05.881 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2023-07-21T18:35:05.881 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2023-07-21 18:35:05.103133970 +0000 2023-07-21T18:35:05.882 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2023-07-21 18:28:13.532706268 +0000 2023-07-21T18:35:05.882 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2023-07-21 18:28:13.532706268 +0000 2023-07-21T18:35:05.882 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: - 2023-07-21T18:35:05.882 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-07-21T18:35:05.948 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2023-07-21T18:35:05.949 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2023-07-21T18:35:05.949 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.000231085 s, 2.2 MB/s 2023-07-21T18:35:05.950 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-07-21T18:35:06.008 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_3 2023-07-21T18:35:06.034 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Jul 21 18:35:05 smithi193 bash[100056]: 1d812b344ad6006a12cd15e793b89044ba1ddfbf81d7345739588a4acc9c6609 2023-07-21T18:35:06.034 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Jul 21 18:35:05 smithi193 systemd[1]: Started Ceph mgr.z for eaf8e25a-27f4-11ee-9b34-001a4aab830c. 2023-07-21T18:35:06.035 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Jul 21 18:35:05 smithi193 ceph-mgr[100153]: -- 172.21.15.193:0/2591783211 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5625d2b911e0 con 0x5625d2b66800 2023-07-21T18:35:06.035 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Jul 21 18:35:05 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-z[100072]: 2023-07-21T18:35:05.866+0000 7f7321f6b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-21T18:35:06.035 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Jul 21 18:35:05 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-z[100072]: 2023-07-21T18:35:05.949+0000 7f7321f6b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-21T18:35:06.052 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-07-21T18:35:06.052 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-21T18:35:06.052 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 6h/6d Inode: 378435 Links: 1 2023-07-21T18:35:06.053 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-21T18:35:06.053 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2023-07-21T18:35:06.053 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2023-07-21 18:35:05.103133970 +0000 2023-07-21T18:35:06.053 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2023-07-21 18:28:14.039707913 +0000 2023-07-21T18:35:06.053 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2023-07-21 18:28:14.039707913 +0000 2023-07-21T18:35:06.053 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: - 2023-07-21T18:35:06.054 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-07-21T18:35:06.118 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2023-07-21T18:35:06.118 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2023-07-21T18:35:06.119 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.00020173 s, 2.5 MB/s 2023-07-21T18:35:06.119 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-07-21T18:35:06.177 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_4 2023-07-21T18:35:06.235 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-07-21T18:35:06.235 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-21T18:35:06.236 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 6h/6d Inode: 382021 Links: 1 2023-07-21T18:35:06.236 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-21T18:35:06.237 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2023-07-21T18:35:06.237 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2023-07-21 18:35:05.103133970 +0000 2023-07-21T18:35:06.238 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2023-07-21 18:28:14.558709597 +0000 2023-07-21T18:35:06.238 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2023-07-21 18:28:14.558709597 +0000 2023-07-21T18:35:06.238 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: - 2023-07-21T18:35:06.239 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-07-21T18:35:06.299 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Jul 21 18:35:06 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-z[100072]: 2023-07-21T18:35:06.043+0000 7f7321f6b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-21T18:35:06.299 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Jul 21 18:35:06 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-z[100072]: 2023-07-21T18:35:06.141+0000 7f7321f6b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-21T18:35:06.300 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:06 smithi193 ceph-mon[98204]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:35:06.300 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:06 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:06.300 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:06 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:06.300 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:06 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:06.300 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:06 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-21T18:35:06.300 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:06 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-21T18:35:06.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:06 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-21T18:35:06.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:06 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:35:06.301 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2023-07-21T18:35:06.302 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2023-07-21T18:35:06.302 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.000191587 s, 2.7 MB/s 2023-07-21T18:35:06.303 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-07-21T18:35:06.361 INFO:tasks.cephadm:Deploying osd.0 on smithi039 with /dev/vg_nvme/lv_4... 2023-07-21T18:35:06.361 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-07-21T18:35:06.391 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:05 smithi039 ceph-mon[96489]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:35:06.391 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:05 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:06.391 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:05 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:06.391 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:05 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:06.391 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:05 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-21T18:35:06.392 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:05 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-21T18:35:06.392 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:05 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-21T18:35:06.392 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:05 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:35:06.401 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:06 smithi155 ceph-mon[97934]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:35:06.401 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:06 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:06.401 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:06 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:06.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:06 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:06.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:06 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-21T18:35:06.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:06 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-21T18:35:06.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:06 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-21T18:35:06.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:06 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:35:06.632 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Jul 21 18:35:06 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-z[100072]: 2023-07-21T18:35:06.315+0000 7f7321f6b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-21T18:35:06.929 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:35:06.968 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Jul 21 18:35:06 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-z[100072]: 2023-07-21T18:35:06.623+0000 7f7321f6b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-21T18:35:06.968 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Jul 21 18:35:06 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-z[100072]: 2023-07-21T18:35:06.712+0000 7f7321f6b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-21T18:35:06.968 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Jul 21 18:35:06 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-z[100072]: 2023-07-21T18:35:06.839+0000 7f7321f6b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-21T18:35:07.365 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:07 smithi039 ceph-mon[96489]: Deploying daemon mgr.y on smithi155 2023-07-21T18:35:07.382 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:07 smithi193 ceph-mon[98204]: Deploying daemon mgr.y on smithi155 2023-07-21T18:35:07.383 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Jul 21 18:35:06 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-z[100072]: 2023-07-21T18:35:06.932+0000 7f7321f6b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-21T18:35:07.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:07 smithi155 ceph-mon[97934]: Deploying daemon mgr.y on smithi155 2023-07-21T18:35:07.402 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 21 18:35:07 smithi155 systemd[1]: Starting Ceph mgr.y for eaf8e25a-27f4-11ee-9b34-001a4aab830c... 2023-07-21T18:35:07.823 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 21 18:35:07 smithi155 podman[99373]: 2023-07-21T18:35:07.824 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 21 18:35:07 smithi155 bash[99373]: 50bbf712a79309ba62e8b382f180cdb409dc77ecb5cd7421ff41dde937b59964 2023-07-21T18:35:07.824 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 21 18:35:07 smithi155 ceph-mgr[99413]: -- 172.21.15.155:0/1074057564 <== mon.2 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564b726491e0 con 0x564b7261f000 2023-07-21T18:35:07.824 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 21 18:35:07 smithi155 systemd[1]: Started Ceph mgr.y for eaf8e25a-27f4-11ee-9b34-001a4aab830c. 2023-07-21T18:35:07.882 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Jul 21 18:35:07 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-z[100072]: 2023-07-21T18:35:07.348+0000 7f7321f6b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-21T18:35:08.094 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 21 18:35:07 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-y[99389]: 2023-07-21T18:35:07.788+0000 7f07796ae200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-21T18:35:08.094 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 21 18:35:07 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-y[99389]: 2023-07-21T18:35:07.867+0000 7f07796ae200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-21T18:35:08.095 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 21 18:35:07 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-y[99389]: 2023-07-21T18:35:07.960+0000 7f07796ae200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-21T18:35:08.095 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:08 smithi155 ceph-mon[97934]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:35:08.095 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:08 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:08.095 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:08 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:08.095 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:08 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:08.095 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:08 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:08.096 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:08 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:35:08.383 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Jul 21 18:35:07 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-z[100072]: 2023-07-21T18:35:07.920+0000 7f7321f6b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-21T18:35:08.383 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Jul 21 18:35:08 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-z[100072]: 2023-07-21T18:35:08.100+0000 7f7321f6b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-21T18:35:08.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:08 smithi193 ceph-mon[98204]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:35:08.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:08 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:08.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:08 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:08.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:08 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:08.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:08 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:08.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:08 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:35:08.401 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 21 18:35:08 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-y[99389]: 2023-07-21T18:35:08.059+0000 7f07796ae200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-21T18:35:08.401 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 21 18:35:08 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-y[99389]: 2023-07-21T18:35:08.235+0000 7f07796ae200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-21T18:35:08.450 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:08 smithi039 ceph-mon[96489]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:35:08.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:08 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:08.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:08 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:08.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:08 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:08.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:08 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:08.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:08 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:35:08.512 INFO:teuthology.orchestra.run.smithi039.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-07-21T18:35:08.512 INFO:teuthology.orchestra.run.smithi039.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-07-21T18:35:08.513 INFO:teuthology.orchestra.run.smithi039.stderr: stderr: 10+0 records in 2023-07-21T18:35:08.513 INFO:teuthology.orchestra.run.smithi039.stderr:10+0 records out 2023-07-21T18:35:08.513 INFO:teuthology.orchestra.run.smithi039.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0142928 s, 734 MB/s 2023-07-21T18:35:08.513 INFO:teuthology.orchestra.run.smithi039.stderr:--> Zapping successful for: 2023-07-21T18:35:08.888 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi039:vg_nvme/lv_4 2023-07-21T18:35:08.901 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 21 18:35:08 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-y[99389]: 2023-07-21T18:35:08.570+0000 7f07796ae200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-21T18:35:08.901 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 21 18:35:08 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-y[99389]: 2023-07-21T18:35:08.665+0000 7f07796ae200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-21T18:35:08.902 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 21 18:35:08 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-y[99389]: 2023-07-21T18:35:08.802+0000 7f07796ae200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-21T18:35:09.219 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:09 smithi193 ceph-mon[98204]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:35:09.219 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Jul 21 18:35:08 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-z[100072]: 2023-07-21T18:35:08.903+0000 7f7321f6b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-21T18:35:09.219 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Jul 21 18:35:09 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-z[100072]: 2023-07-21T18:35:08.999+0000 7f7321f6b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-21T18:35:09.382 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:09 smithi155 ceph-mon[97934]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:35:09.383 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 21 18:35:08 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-y[99389]: 2023-07-21T18:35:08.901+0000 7f07796ae200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-21T18:35:09.450 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:09 smithi039 ceph-mon[96489]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:35:09.490 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:35:09.513 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Jul 21 18:35:09 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-z[100072]: 2023-07-21T18:35:09.183+0000 7f7321f6b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-21T18:35:09.651 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 21 18:35:09 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-y[99389]: 2023-07-21T18:35:09.347+0000 7f07796ae200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-21T18:35:09.882 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Jul 21 18:35:09 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-z[100072]: 2023-07-21T18:35:09.477+0000 7f7321f6b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-21T18:35:09.883 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Jul 21 18:35:09 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-z[100072]: 2023-07-21T18:35:09.706+0000 7f7321f6b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-21T18:35:10.150 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Jul 21 18:35:09 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-z[100072]: 2023-07-21T18:35:09.851+0000 7f7321f6b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-21T18:35:10.396 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 21 18:35:09 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-y[99389]: 2023-07-21T18:35:09.929+0000 7f07796ae200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-21T18:35:10.396 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 21 18:35:10 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-y[99389]: 2023-07-21T18:35:10.099+0000 7f07796ae200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-21T18:35:10.651 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:10 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:10.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:10 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:10.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:10 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:10.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:10 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:10.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:10 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:35:10.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:10 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:35:10.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:10 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:10.653 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:10 smithi155 ceph-mon[97934]: Reconfiguring mgr.x (unknown last config time)... 2023-07-21T18:35:10.653 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:10 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-21T18:35:10.653 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:10 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-21T18:35:10.653 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:10 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:35:10.653 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:10 smithi155 ceph-mon[97934]: Reconfiguring daemon mgr.x on smithi039 2023-07-21T18:35:10.663 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:10 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:10.664 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:10 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:10.664 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:10 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:10.665 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:10 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:10.665 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:10 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:35:10.665 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:10 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:35:10.666 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:10 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:10.666 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:10 smithi039 ceph-mon[96489]: Reconfiguring mgr.x (unknown last config time)... 2023-07-21T18:35:10.666 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:10 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-21T18:35:10.666 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:10 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-21T18:35:10.667 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:10 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:35:10.667 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:10 smithi039 ceph-mon[96489]: Reconfiguring daemon mgr.x on smithi039 2023-07-21T18:35:10.775 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:10.740+0000 7f14119f5700 1 -- 172.21.15.39:0/3331734517 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f140c105890 msgr2=0x7f140c107cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:35:10.776 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:10.740+0000 7f14119f5700 1 --2- 172.21.15.39:0/3331734517 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f140c105890 0x7f140c107cb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f140400b0a0 tx=0x7f140400b3b0 comp rx=0 tx=0).stop 2023-07-21T18:35:10.776 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:10.740+0000 7f14119f5700 1 -- 172.21.15.39:0/3331734517 shutdown_connections 2023-07-21T18:35:10.776 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:10.740+0000 7f14119f5700 1 --2- 172.21.15.39:0/3331734517 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f140c105890 0x7f140c107cb0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:35:10.777 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:10.740+0000 7f14119f5700 1 --2- 172.21.15.39:0/3331734517 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f140c100650 0x7f140c102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:35:10.777 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:10.740+0000 7f14119f5700 1 --2- 172.21.15.39:0/3331734517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f140c102f70 0x7f140c105350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:35:10.777 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:10.740+0000 7f14119f5700 1 -- 172.21.15.39:0/3331734517 >> 172.21.15.39:0/3331734517 conn(0x7f140c0fa0b0 msgr2=0x7f140c0fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:35:10.777 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:10.740+0000 7f14119f5700 1 -- 172.21.15.39:0/3331734517 shutdown_connections 2023-07-21T18:35:10.777 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:10.740+0000 7f14119f5700 1 -- 172.21.15.39:0/3331734517 wait complete. 2023-07-21T18:35:10.778 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:10.741+0000 7f14119f5700 1 Processor -- start 2023-07-21T18:35:10.781 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:10.741+0000 7f14119f5700 1 -- start start 2023-07-21T18:35:10.781 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:10.741+0000 7f14119f5700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f140c100650 0x7f140c0fffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:35:10.781 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:10.741+0000 7f14119f5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f140c102f70 0x7f140c0fe600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:35:10.781 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:10.741+0000 7f14119f5700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f140c105890 0x7f140c0feb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:35:10.782 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:10.741+0000 7f14119f5700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f140c0ff2c0 con 0x7f140c102f70 2023-07-21T18:35:10.782 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:10.741+0000 7f14119f5700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f140c0ff430 con 0x7f140c100650 2023-07-21T18:35:10.782 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:10.741+0000 7f14119f5700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f140c19fa20 con 0x7f140c105890 2023-07-21T18:35:10.782 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:10.741+0000 7f140a7fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f140c102f70 0x7f140c0fe600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:35:10.782 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:10.741+0000 7f140a7fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f140c102f70 0x7f140c0fe600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54122/0 (socket says 172.21.15.39:54122) 2023-07-21T18:35:10.783 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:10.741+0000 7f140a7fc700 1 -- 172.21.15.39:0/1353011747 learned_addr learned my addr 172.21.15.39:0/1353011747 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:35:10.783 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:10.741+0000 7f140affd700 1 --2- 172.21.15.39:0/1353011747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f140c100650 0x7f140c0fffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:35:10.783 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:10.741+0000 7f140b7fe700 1 --2- 172.21.15.39:0/1353011747 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f140c105890 0x7f140c0feb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:35:10.783 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:10.742+0000 7f140a7fc700 1 -- 172.21.15.39:0/1353011747 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f140c105890 msgr2=0x7f140c0feb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:35:10.784 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:10.742+0000 7f140a7fc700 1 --2- 172.21.15.39:0/1353011747 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f140c105890 0x7f140c0feb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:35:10.784 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:10.742+0000 7f140a7fc700 1 -- 172.21.15.39:0/1353011747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f140c100650 msgr2=0x7f140c0fffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:35:10.784 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:10.742+0000 7f140a7fc700 1 --2- 172.21.15.39:0/1353011747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f140c100650 0x7f140c0fffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:35:10.784 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:10.742+0000 7f140a7fc700 1 -- 172.21.15.39:0/1353011747 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1404009d00 con 0x7f140c102f70 2023-07-21T18:35:10.784 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:10.742+0000 7f140affd700 1 --2- 172.21.15.39:0/1353011747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f140c100650 0x7f140c0fffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:35:10.784 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:10.742+0000 7f140a7fc700 1 --2- 172.21.15.39:0/1353011747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f140c102f70 0x7f140c0fe600 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f140000e9a0 tx=0x7f140000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:35:10.785 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:10.742+0000 7f14109f3700 1 -- 172.21.15.39:0/1353011747 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1400009790 con 0x7f140c102f70 2023-07-21T18:35:10.785 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:10.742+0000 7f14109f3700 1 -- 172.21.15.39:0/1353011747 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1400010430 con 0x7f140c102f70 2023-07-21T18:35:10.785 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:10.742+0000 7f14119f5700 1 -- 172.21.15.39:0/1353011747 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f140c19fd00 con 0x7f140c102f70 2023-07-21T18:35:10.785 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:10.742+0000 7f14109f3700 1 -- 172.21.15.39:0/1353011747 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1400018680 con 0x7f140c102f70 2023-07-21T18:35:10.785 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:10.742+0000 7f14119f5700 1 -- 172.21.15.39:0/1353011747 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f140c1a01b0 con 0x7f140c102f70 2023-07-21T18:35:10.785 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:10.743+0000 7f14109f3700 1 -- 172.21.15.39:0/1353011747 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f14000187e0 con 0x7f140c102f70 2023-07-21T18:35:10.785 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:10.743+0000 7f14119f5700 1 -- 172.21.15.39:0/1353011747 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f140c0fbc20 con 0x7f140c102f70 2023-07-21T18:35:10.786 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:10.743+0000 7f14109f3700 1 --2- 172.21.15.39:0/1353011747 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f13f40389e0 0x7f13f403ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:35:10.786 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:10.743+0000 7f14109f3700 1 -- 172.21.15.39:0/1353011747 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f1400014070 con 0x7f140c102f70 2023-07-21T18:35:10.786 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:10.743+0000 7f140affd700 1 --2- 172.21.15.39:0/1353011747 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f13f40389e0 0x7f13f403ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:35:10.786 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:10.744+0000 7f140affd700 1 --2- 172.21.15.39:0/1353011747 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f13f40389e0 0x7f13f403ae90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f13fc005950 tx=0x7f13fc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:35:10.787 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:10.748+0000 7f14109f3700 1 -- 172.21.15.39:0/1353011747 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f14000152d0 con 0x7f140c102f70 2023-07-21T18:35:10.840 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Jul 21 18:35:10 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-z[100072]: 2023-07-21T18:35:10.109+0000 7f7321f6b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-21T18:35:10.841 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:10 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:10.841 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:10 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:10.841 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:10 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:10.841 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:10 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:10.841 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:10 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:35:10.842 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:10 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:35:10.842 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:10 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:10.842 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:10 smithi193 ceph-mon[98204]: Reconfiguring mgr.x (unknown last config time)... 2023-07-21T18:35:10.842 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:10 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-21T18:35:10.842 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:10 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-21T18:35:10.842 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:10 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:35:10.842 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:10 smithi193 ceph-mon[98204]: Reconfiguring daemon mgr.x on smithi039 2023-07-21T18:35:10.995 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:10.960+0000 7f14119f5700 1 -- 172.21.15.39:0/1353011747 --> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f140c060950 con 0x7f13f40389e0 2023-07-21T18:35:11.132 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Jul 21 18:35:10 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-z[100072]: 2023-07-21T18:35:10.805+0000 7f7321f6b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-21T18:35:11.133 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Jul 21 18:35:10 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-z[100072]: 2023-07-21T18:35:10.935+0000 7f7321f6b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-21T18:35:11.149 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 21 18:35:10 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-y[99389]: 2023-07-21T18:35:10.856+0000 7f07796ae200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-21T18:35:11.149 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 21 18:35:10 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-y[99389]: 2023-07-21T18:35:10.946+0000 7f07796ae200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-21T18:35:11.486 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Jul 21 18:35:11 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-z[100072]: 2023-07-21T18:35:11.124+0000 7f7321f6b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-21T18:35:11.486 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Jul 21 18:35:11 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-z[100072]: 2023-07-21T18:35:11.238+0000 7f7321f6b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-21T18:35:11.797 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 21 18:35:11 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-y[99389]: 2023-07-21T18:35:11.114+0000 7f07796ae200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-21T18:35:11.797 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 21 18:35:11 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-y[99389]: 2023-07-21T18:35:11.385+0000 7f07796ae200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-21T18:35:11.797 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 21 18:35:11 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-y[99389]: 2023-07-21T18:35:11.614+0000 7f07796ae200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-21T18:35:11.882 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Jul 21 18:35:11 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-z[100072]: 2023-07-21T18:35:11.450+0000 7f7321f6b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-21T18:35:12.017 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:11 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:12.018 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:11 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:12.018 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:11 smithi039 ceph-mon[96489]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:35:12.018 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:11 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T18:35:12.018 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:11 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T18:35:12.018 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:11 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:35:12.019 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:11 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:35:12.019 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:11 smithi039 ceph-mon[96489]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:35:12.019 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:11 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:35:12.019 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:11 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:35:12.019 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:11 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:12.054 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 21 18:35:11 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-y[99389]: 2023-07-21T18:35:11.761+0000 7f07796ae200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-21T18:35:12.055 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:11 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:12.055 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:11 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:12.055 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:11 smithi155 ceph-mon[97934]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:35:12.056 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:11 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T18:35:12.056 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:11 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T18:35:12.056 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:11 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:35:12.056 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:11 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:35:12.056 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:11 smithi155 ceph-mon[97934]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:35:12.056 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:11 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:35:12.056 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:11 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:35:12.057 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:11 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:12.353 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:11 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:12.354 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:11 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:12.354 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:11 smithi193 ceph-mon[98204]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:35:12.354 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:11 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T18:35:12.354 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:11 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T18:35:12.354 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:11 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:35:12.354 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:11 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:35:12.354 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:11 smithi193 ceph-mon[98204]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:35:12.355 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:11 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:35:12.355 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:11 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:35:12.355 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:11 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:12.401 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 21 18:35:12 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-y[99389]: 2023-07-21T18:35:12.018+0000 7f07796ae200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-21T18:35:12.633 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Jul 21 18:35:12 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-z[100072]: 2023-07-21T18:35:12.318+0000 7f7321f6b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-21T18:35:13.070 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 21 18:35:12 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-y[99389]: 2023-07-21T18:35:12.714+0000 7f07796ae200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-21T18:35:13.070 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 21 18:35:12 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-y[99389]: 2023-07-21T18:35:12.844+0000 7f07796ae200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-21T18:35:13.166 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:13.130+0000 7f14109f3700 1 -- 172.21.15.39:0/1353011747 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mgrmap(e 14) v1 ==== 90599+0+0 (secure 0 0 0) 0x7f1400010880 con 0x7f140c102f70 2023-07-21T18:35:13.395 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:13 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:13.396 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:13 smithi155 ceph-mon[97934]: Standby manager daemon z started 2023-07-21T18:35:13.396 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:13 smithi155 ceph-mon[97934]: from='mgr.? 172.21.15.193:0/1875296443' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-07-21T18:35:13.396 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:13 smithi155 ceph-mon[97934]: from='mgr.? 172.21.15.193:0/1875296443' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-21T18:35:13.396 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:13 smithi155 ceph-mon[97934]: from='mgr.? 172.21.15.193:0/1875296443' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-07-21T18:35:13.396 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:13 smithi155 ceph-mon[97934]: from='mgr.? 172.21.15.193:0/1875296443' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-21T18:35:13.397 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:13 smithi155 ceph-mon[97934]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:35:13.397 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 21 18:35:13 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-y[99389]: 2023-07-21T18:35:13.034+0000 7f07796ae200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-21T18:35:13.397 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 21 18:35:13 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-y[99389]: 2023-07-21T18:35:13.148+0000 7f07796ae200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-21T18:35:13.397 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 21 18:35:13 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-y[99389]: 2023-07-21T18:35:13.360+0000 7f07796ae200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-21T18:35:13.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:13 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:13.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:13 smithi039 ceph-mon[96489]: Standby manager daemon z started 2023-07-21T18:35:13.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:13 smithi039 ceph-mon[96489]: from='mgr.? 172.21.15.193:0/1875296443' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-07-21T18:35:13.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:13 smithi039 ceph-mon[96489]: from='mgr.? 172.21.15.193:0/1875296443' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-21T18:35:13.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:13 smithi039 ceph-mon[96489]: from='mgr.? 172.21.15.193:0/1875296443' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-07-21T18:35:13.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:13 smithi039 ceph-mon[96489]: from='mgr.? 172.21.15.193:0/1875296443' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-21T18:35:13.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:13 smithi039 ceph-mon[96489]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:35:13.634 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:13 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:13.635 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:13 smithi193 ceph-mon[98204]: Standby manager daemon z started 2023-07-21T18:35:13.635 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:13 smithi193 ceph-mon[98204]: from='mgr.? 172.21.15.193:0/1875296443' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-07-21T18:35:13.635 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:13 smithi193 ceph-mon[98204]: from='mgr.? 172.21.15.193:0/1875296443' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-21T18:35:13.635 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:13 smithi193 ceph-mon[98204]: from='mgr.? 172.21.15.193:0/1875296443' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-07-21T18:35:13.635 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:13 smithi193 ceph-mon[98204]: from='mgr.? 172.21.15.193:0/1875296443' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-21T18:35:13.635 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:13 smithi193 ceph-mon[98204]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:35:14.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:14 smithi193 ceph-mon[98204]: mgrmap e14: x(active, since 66s), standbys: z 2023-07-21T18:35:14.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:14 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-07-21T18:35:14.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:14 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/1054531555' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "03502c07-2d0f-4723-8d0b-d34c9f5b91bf"}]: dispatch 2023-07-21T18:35:14.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:14 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/1054531555' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "03502c07-2d0f-4723-8d0b-d34c9f5b91bf"}]': finished 2023-07-21T18:35:14.634 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:14 smithi193 ceph-mon[98204]: osdmap e5: 1 total, 0 up, 1 in 2023-07-21T18:35:14.634 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:14 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T18:35:14.634 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:14 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/2295264353' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T18:35:14.651 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 21 18:35:14 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-y[99389]: 2023-07-21T18:35:14.205+0000 7f07796ae200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-21T18:35:14.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:14 smithi155 ceph-mon[97934]: mgrmap e14: x(active, since 66s), standbys: z 2023-07-21T18:35:14.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:14 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-07-21T18:35:14.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:14 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/1054531555' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "03502c07-2d0f-4723-8d0b-d34c9f5b91bf"}]: dispatch 2023-07-21T18:35:14.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:14 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/1054531555' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "03502c07-2d0f-4723-8d0b-d34c9f5b91bf"}]': finished 2023-07-21T18:35:14.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:14 smithi155 ceph-mon[97934]: osdmap e5: 1 total, 0 up, 1 in 2023-07-21T18:35:14.653 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:14 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T18:35:14.653 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:14 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/2295264353' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T18:35:14.663 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:14.628+0000 7f14109f3700 1 -- 172.21.15.39:0/1353011747 <== mon.0 v2:172.21.15.39:3300/0 8 ==== mgrmap(e 15) v1 ==== 135659+0+0 (secure 0 0 0) 0x7f14000173d0 con 0x7f140c102f70 2023-07-21T18:35:14.700 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:14 smithi039 ceph-mon[96489]: mgrmap e14: x(active, since 66s), standbys: z 2023-07-21T18:35:14.701 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:14 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-07-21T18:35:14.701 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:14 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/1054531555' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "03502c07-2d0f-4723-8d0b-d34c9f5b91bf"}]: dispatch 2023-07-21T18:35:14.701 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:14 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/1054531555' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "03502c07-2d0f-4723-8d0b-d34c9f5b91bf"}]': finished 2023-07-21T18:35:14.701 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:14 smithi039 ceph-mon[96489]: osdmap e5: 1 total, 0 up, 1 in 2023-07-21T18:35:14.701 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:14 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T18:35:14.701 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:14 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/2295264353' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T18:35:15.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:15 smithi193 ceph-mon[98204]: from='mgr.? 172.21.15.155:0/313917668' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-07-21T18:35:15.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:15 smithi193 ceph-mon[98204]: from='mgr.? 172.21.15.155:0/313917668' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-21T18:35:15.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:15 smithi193 ceph-mon[98204]: from='mgr.? 172.21.15.155:0/313917668' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-07-21T18:35:15.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:15 smithi193 ceph-mon[98204]: from='mgr.? 172.21.15.155:0/313917668' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-21T18:35:15.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:15 smithi193 ceph-mon[98204]: Standby manager daemon y started 2023-07-21T18:35:15.634 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:15 smithi193 ceph-mon[98204]: mgrmap e15: x(active, since 67s), standbys: z, y 2023-07-21T18:35:15.634 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:15 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-07-21T18:35:15.634 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:15 smithi193 ceph-mon[98204]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:35:15.651 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:15 smithi155 ceph-mon[97934]: from='mgr.? 172.21.15.155:0/313917668' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-07-21T18:35:15.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:15 smithi155 ceph-mon[97934]: from='mgr.? 172.21.15.155:0/313917668' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-21T18:35:15.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:15 smithi155 ceph-mon[97934]: from='mgr.? 172.21.15.155:0/313917668' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-07-21T18:35:15.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:15 smithi155 ceph-mon[97934]: from='mgr.? 172.21.15.155:0/313917668' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-21T18:35:15.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:15 smithi155 ceph-mon[97934]: Standby manager daemon y started 2023-07-21T18:35:15.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:15 smithi155 ceph-mon[97934]: mgrmap e15: x(active, since 67s), standbys: z, y 2023-07-21T18:35:15.653 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:15 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-07-21T18:35:15.653 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:15 smithi155 ceph-mon[97934]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:35:15.701 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:15 smithi039 ceph-mon[96489]: from='mgr.? 172.21.15.155:0/313917668' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-07-21T18:35:15.701 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:15 smithi039 ceph-mon[96489]: from='mgr.? 172.21.15.155:0/313917668' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-21T18:35:15.701 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:15 smithi039 ceph-mon[96489]: from='mgr.? 172.21.15.155:0/313917668' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-07-21T18:35:15.701 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:15 smithi039 ceph-mon[96489]: from='mgr.? 172.21.15.155:0/313917668' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-21T18:35:15.701 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:15 smithi039 ceph-mon[96489]: Standby manager daemon y started 2023-07-21T18:35:15.702 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:15 smithi039 ceph-mon[96489]: mgrmap e15: x(active, since 67s), standbys: z, y 2023-07-21T18:35:15.702 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:15 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-07-21T18:35:15.702 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:15 smithi039 ceph-mon[96489]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:35:17.450 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:17 smithi039 ceph-mon[96489]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:35:17.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:17 smithi193 ceph-mon[98204]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:35:17.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:17 smithi155 ceph-mon[97934]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:35:19.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:19 smithi039 ceph-mon[96489]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:35:19.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:19 smithi193 ceph-mon[98204]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:35:19.653 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:19 smithi155 ceph-mon[97934]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:35:22.200 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:21 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-07-21T18:35:22.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:21 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:35:22.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:21 smithi039 ceph-mon[96489]: Deploying daemon osd.0 on smithi039 2023-07-21T18:35:22.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:21 smithi039 ceph-mon[96489]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:35:22.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:21 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-07-21T18:35:22.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:21 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:35:22.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:21 smithi193 ceph-mon[98204]: Deploying daemon osd.0 on smithi039 2023-07-21T18:35:22.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:21 smithi193 ceph-mon[98204]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:35:22.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:21 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-07-21T18:35:22.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:21 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:35:22.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:21 smithi155 ceph-mon[97934]: Deploying daemon osd.0 on smithi039 2023-07-21T18:35:22.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:21 smithi155 ceph-mon[97934]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:35:23.282 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:23 smithi039 ceph-mon[96489]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:35:23.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:23 smithi193 ceph-mon[98204]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:35:23.651 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:23 smithi155 ceph-mon[97934]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:35:25.200 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:25 smithi039 ceph-mon[96489]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:35:25.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:25 smithi193 ceph-mon[98204]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:35:25.651 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:25 smithi155 ceph-mon[97934]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:35:26.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:26 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:26.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:26 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:26.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:26 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:35:26.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:26 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:26.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:26 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:26.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:26 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:35:26.651 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:26 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:26.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:26 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:26.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:26 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:35:27.450 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:27 smithi039 ceph-mon[96489]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:35:27.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:27 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:27.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:27 smithi193 ceph-mon[98204]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:35:27.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:27 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:27.651 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:27 smithi155 ceph-mon[97934]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:35:27.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:27 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:27.903 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:27.867+0000 7f14109f3700 1 -- 172.21.15.39:0/1353011747 <== mgr.14152 v2:172.21.15.39:6800/2116291712 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f140c060950 con 0x7f13f40389e0 2023-07-21T18:35:27.904 INFO:teuthology.orchestra.run.smithi039.stdout:Created osd(s) 0 on host 'smithi039' 2023-07-21T18:35:27.905 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:27.869+0000 7f14119f5700 1 -- 172.21.15.39:0/1353011747 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f13f40389e0 msgr2=0x7f13f403ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:35:27.905 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:27.869+0000 7f14119f5700 1 --2- 172.21.15.39:0/1353011747 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f13f40389e0 0x7f13f403ae90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f13fc005950 tx=0x7f13fc00b410 comp rx=0 tx=0).stop 2023-07-21T18:35:27.906 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:27.869+0000 7f14119f5700 1 -- 172.21.15.39:0/1353011747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f140c102f70 msgr2=0x7f140c0fe600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:35:27.906 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:27.869+0000 7f14119f5700 1 --2- 172.21.15.39:0/1353011747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f140c102f70 0x7f140c0fe600 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f140000e9a0 tx=0x7f140000ecb0 comp rx=0 tx=0).stop 2023-07-21T18:35:27.906 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:27.869+0000 7f14119f5700 1 -- 172.21.15.39:0/1353011747 shutdown_connections 2023-07-21T18:35:27.906 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:27.869+0000 7f14119f5700 1 --2- 172.21.15.39:0/1353011747 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f13f40389e0 0x7f13f403ae90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:35:27.906 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:27.869+0000 7f14119f5700 1 --2- 172.21.15.39:0/1353011747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f140c100650 0x7f140c0fffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:35:27.906 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:27.869+0000 7f14119f5700 1 --2- 172.21.15.39:0/1353011747 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f140c105890 0x7f140c0feb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:35:27.906 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:27.869+0000 7f14119f5700 1 --2- 172.21.15.39:0/1353011747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f140c102f70 0x7f140c0fe600 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:35:27.906 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:27.869+0000 7f14119f5700 1 -- 172.21.15.39:0/1353011747 >> 172.21.15.39:0/1353011747 conn(0x7f140c0fa0b0 msgr2=0x7f140c1012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:35:27.907 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:27.869+0000 7f14119f5700 1 -- 172.21.15.39:0/1353011747 shutdown_connections 2023-07-21T18:35:27.907 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:35:27.869+0000 7f14119f5700 1 -- 172.21.15.39:0/1353011747 wait complete. 2023-07-21T18:35:28.295 DEBUG:teuthology.orchestra.run.smithi039:osd.0> sudo journalctl -f -n 0 -u ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@osd.0.service 2023-07-21T18:35:28.297 INFO:tasks.cephadm:Deploying osd.1 on smithi155 with /dev/vg_nvme/lv_4... 2023-07-21T18:35:28.298 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-07-21T18:35:28.329 INFO:journalctl@ceph.osd.0.smithi039.stdout:-- Logs begin at Fri 2023-07-21 18:18:00 UTC. -- 2023-07-21T18:35:28.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:28 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:28.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:28 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:35:28.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:28 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:35:28.634 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:28 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:28.634 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:28 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:28.634 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:28 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:28.634 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:28 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:35:28.651 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:28 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:28.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:28 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:35:28.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:28 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:35:28.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:28 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:28.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:28 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:28.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:28 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:28.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:28 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:35:28.701 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:28 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:28.701 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:28 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:35:28.701 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:28 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:35:28.701 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:28 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:28.701 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:28 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:28.702 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:28 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:28.702 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:28 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:35:28.848 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.b/config 2023-07-21T18:35:29.350 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:29 smithi155 ceph-mon[97934]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:35:29.471 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:29 smithi039 ceph-mon[96489]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:35:29.472 INFO:journalctl@ceph.osd.0.smithi039.stdout:Jul 21 18:35:29 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-0[109474]: 2023-07-21T18:35:29.169+0000 7f6699e99640 -1 osd.0 0 log_to_monitors true 2023-07-21T18:35:29.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:29 smithi193 ceph-mon[98204]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:35:30.317 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-07-21T18:35:30.317 INFO:teuthology.orchestra.run.smithi155.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-07-21T18:35:30.317 INFO:teuthology.orchestra.run.smithi155.stderr: stderr: 10+0 records in 2023-07-21T18:35:30.317 INFO:teuthology.orchestra.run.smithi155.stderr:10+0 records out 2023-07-21T18:35:30.317 INFO:teuthology.orchestra.run.smithi155.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0110211 s, 951 MB/s 2023-07-21T18:35:30.318 INFO:teuthology.orchestra.run.smithi155.stderr: stderr: 2023-07-21T18:35:30.318 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping successful for: 2023-07-21T18:35:30.626 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:30 smithi039 ceph-mon[96489]: from='osd.0 [v2:172.21.15.39:6802/860193857,v1:172.21.15.39:6803/860193857]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-07-21T18:35:30.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:30 smithi193 ceph-mon[98204]: from='osd.0 [v2:172.21.15.39:6802/860193857,v1:172.21.15.39:6803/860193857]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-07-21T18:35:30.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:30 smithi155 ceph-mon[97934]: from='osd.0 [v2:172.21.15.39:6802/860193857,v1:172.21.15.39:6803/860193857]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-07-21T18:35:30.704 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi155:vg_nvme/lv_4 2023-07-21T18:35:31.260 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.b/config 2023-07-21T18:35:31.481 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:31 smithi155 ceph-mon[97934]: from='osd.0 [v2:172.21.15.39:6802/860193857,v1:172.21.15.39:6803/860193857]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-07-21T18:35:31.481 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:31 smithi155 ceph-mon[97934]: osdmap e6: 1 total, 0 up, 1 in 2023-07-21T18:35:31.481 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:31 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T18:35:31.481 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:31 smithi155 ceph-mon[97934]: from='osd.0 [v2:172.21.15.39:6802/860193857,v1:172.21.15.39:6803/860193857]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2023-07-21T18:35:31.482 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:31 smithi155 ceph-mon[97934]: Detected new or changed devices on smithi039 2023-07-21T18:35:31.482 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:31 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:31.482 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:31 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:31.482 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:31 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:35:31.482 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:31 smithi155 ceph-mon[97934]: Adjusting osd_memory_target on smithi039 to 17168M 2023-07-21T18:35:31.482 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:31 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:31.482 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:31 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:35:31.483 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:31 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:35:31.483 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:31 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:31.483 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:31 smithi155 ceph-mon[97934]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:35:31.483 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:31 smithi155 ceph-mon[97934]: from='osd.0 [v2:172.21.15.39:6802/860193857,v1:172.21.15.39:6803/860193857]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2023-07-21T18:35:31.483 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:31 smithi155 ceph-mon[97934]: osdmap e7: 1 total, 0 up, 1 in 2023-07-21T18:35:31.483 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:31 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T18:35:31.483 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:31 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T18:35:31.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:31 smithi193 ceph-mon[98204]: from='osd.0 [v2:172.21.15.39:6802/860193857,v1:172.21.15.39:6803/860193857]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-07-21T18:35:31.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:31 smithi193 ceph-mon[98204]: osdmap e6: 1 total, 0 up, 1 in 2023-07-21T18:35:31.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:31 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T18:35:31.634 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:31 smithi193 ceph-mon[98204]: from='osd.0 [v2:172.21.15.39:6802/860193857,v1:172.21.15.39:6803/860193857]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2023-07-21T18:35:31.634 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:31 smithi193 ceph-mon[98204]: Detected new or changed devices on smithi039 2023-07-21T18:35:31.634 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:31 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:31.634 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:31 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:31.634 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:31 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:35:31.634 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:31 smithi193 ceph-mon[98204]: Adjusting osd_memory_target on smithi039 to 17168M 2023-07-21T18:35:31.634 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:31 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:31.634 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:31 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:35:31.635 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:31 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:35:31.635 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:31 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:31.635 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:31 smithi193 ceph-mon[98204]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:35:31.635 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:31 smithi193 ceph-mon[98204]: from='osd.0 [v2:172.21.15.39:6802/860193857,v1:172.21.15.39:6803/860193857]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2023-07-21T18:35:31.635 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:31 smithi193 ceph-mon[98204]: osdmap e7: 1 total, 0 up, 1 in 2023-07-21T18:35:31.635 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:31 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T18:35:31.635 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:31 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T18:35:31.700 INFO:journalctl@ceph.osd.0.smithi039.stdout:Jul 21 18:35:31 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-0[109474]: 2023-07-21T18:35:31.185+0000 7f668ed10700 -1 osd.0 0 waiting for initial osdmap 2023-07-21T18:35:31.701 INFO:journalctl@ceph.osd.0.smithi039.stdout:Jul 21 18:35:31 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-0[109474]: 2023-07-21T18:35:31.190+0000 7f6687aff700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-21T18:35:31.701 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:31 smithi039 ceph-mon[96489]: from='osd.0 [v2:172.21.15.39:6802/860193857,v1:172.21.15.39:6803/860193857]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-07-21T18:35:31.701 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:31 smithi039 ceph-mon[96489]: osdmap e6: 1 total, 0 up, 1 in 2023-07-21T18:35:31.701 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:31 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T18:35:31.702 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:31 smithi039 ceph-mon[96489]: from='osd.0 [v2:172.21.15.39:6802/860193857,v1:172.21.15.39:6803/860193857]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2023-07-21T18:35:31.702 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:31 smithi039 ceph-mon[96489]: Detected new or changed devices on smithi039 2023-07-21T18:35:31.702 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:31 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:31.702 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:31 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:31.702 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:31 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:35:31.702 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:31 smithi039 ceph-mon[96489]: Adjusting osd_memory_target on smithi039 to 17168M 2023-07-21T18:35:31.702 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:31 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:31.703 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:31 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:35:31.703 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:31 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:35:31.703 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:31 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:31.703 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:31 smithi039 ceph-mon[96489]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:35:31.703 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:31 smithi039 ceph-mon[96489]: from='osd.0 [v2:172.21.15.39:6802/860193857,v1:172.21.15.39:6803/860193857]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2023-07-21T18:35:31.703 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:31 smithi039 ceph-mon[96489]: osdmap e7: 1 total, 0 up, 1 in 2023-07-21T18:35:31.703 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:31 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T18:35:31.703 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:31 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T18:35:32.362 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:35:32.326+0000 7f6d44383700 1 -- 172.21.15.155:0/695652610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d3c0ffc80 msgr2=0x7f6d3c1000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:35:32.362 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:35:32.326+0000 7f6d44383700 1 --2- 172.21.15.155:0/695652610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d3c0ffc80 0x7f6d3c1000d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6d38009990 tx=0x7f6d38009ca0 comp rx=0 tx=0).stop 2023-07-21T18:35:32.362 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:35:32.326+0000 7f6d44383700 1 -- 172.21.15.155:0/695652610 shutdown_connections 2023-07-21T18:35:32.362 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:35:32.326+0000 7f6d44383700 1 --2- 172.21.15.155:0/695652610 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d3c100610 0x7f6d3c106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:35:32.362 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:35:32.326+0000 7f6d44383700 1 --2- 172.21.15.155:0/695652610 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d3c0fea80 0x7f6d3c0fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:35:32.363 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:35:32.326+0000 7f6d44383700 1 --2- 172.21.15.155:0/695652610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d3c0ffc80 0x7f6d3c1000d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:35:32.363 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:35:32.326+0000 7f6d44383700 1 -- 172.21.15.155:0/695652610 >> 172.21.15.155:0/695652610 conn(0x7f6d3c0fa0b0 msgr2=0x7f6d3c0fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:35:32.363 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:35:32.326+0000 7f6d44383700 1 -- 172.21.15.155:0/695652610 shutdown_connections 2023-07-21T18:35:32.363 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:35:32.326+0000 7f6d44383700 1 -- 172.21.15.155:0/695652610 wait complete. 2023-07-21T18:35:32.364 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:35:32.326+0000 7f6d44383700 1 Processor -- start 2023-07-21T18:35:32.365 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:35:32.326+0000 7f6d44383700 1 -- start start 2023-07-21T18:35:32.365 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:35:32.327+0000 7f6d44383700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d3c0fea80 0x7f6d3c102a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:35:32.365 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:35:32.327+0000 7f6d44383700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d3c0ffc80 0x7f6d3c101060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:35:32.365 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:35:32.327+0000 7f6d44383700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d3c100610 0x7f6d3c1015a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:35:32.365 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:35:32.327+0000 7f6d44383700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3c101d20 con 0x7f6d3c0fea80 2023-07-21T18:35:32.365 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:35:32.327+0000 7f6d44383700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3c101e90 con 0x7f6d3c0ffc80 2023-07-21T18:35:32.366 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:35:32.327+0000 7f6d44383700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3c19f9e0 con 0x7f6d3c100610 2023-07-21T18:35:32.366 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:35:32.327+0000 7f6d4191e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d3c0ffc80 0x7f6d3c101060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:35:32.366 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:35:32.327+0000 7f6d4191e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d3c0ffc80 0x7f6d3c101060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:57184/0 (socket says 172.21.15.155:57184) 2023-07-21T18:35:32.366 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:35:32.327+0000 7f6d4191e700 1 -- 172.21.15.155:0/3218851440 learned_addr learned my addr 172.21.15.155:0/3218851440 (peer_addr_for_me v2:172.21.15.155:0/0) 2023-07-21T18:35:32.366 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:35:32.327+0000 7f6d4211f700 1 --2- 172.21.15.155:0/3218851440 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d3c0fea80 0x7f6d3c102a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:35:32.366 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:35:32.327+0000 7f6d4191e700 1 -- 172.21.15.155:0/3218851440 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d3c100610 msgr2=0x7f6d3c1015a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:35:32.366 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:35:32.327+0000 7f6d42920700 1 --2- 172.21.15.155:0/3218851440 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d3c100610 0x7f6d3c1015a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:35:32.366 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:35:32.327+0000 7f6d4191e700 1 --2- 172.21.15.155:0/3218851440 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d3c100610 0x7f6d3c1015a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:35:32.367 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:35:32.327+0000 7f6d4191e700 1 -- 172.21.15.155:0/3218851440 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d3c0fea80 msgr2=0x7f6d3c102a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:35:32.367 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:35:32.327+0000 7f6d4191e700 1 --2- 172.21.15.155:0/3218851440 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d3c0fea80 0x7f6d3c102a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:35:32.367 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:35:32.327+0000 7f6d4191e700 1 -- 172.21.15.155:0/3218851440 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d380095f0 con 0x7f6d3c0ffc80 2023-07-21T18:35:32.367 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:35:32.327+0000 7f6d4211f700 1 --2- 172.21.15.155:0/3218851440 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d3c0fea80 0x7f6d3c102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:35:32.367 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:35:32.327+0000 7f6d4191e700 1 --2- 172.21.15.155:0/3218851440 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d3c0ffc80 0x7f6d3c101060 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6d38005910 tx=0x7f6d3800f740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:35:32.367 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:35:32.327+0000 7f6d337fe700 1 -- 172.21.15.155:0/3218851440 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d3801d070 con 0x7f6d3c0ffc80 2023-07-21T18:35:32.367 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:35:32.327+0000 7f6d337fe700 1 -- 172.21.15.155:0/3218851440 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d3800fc30 con 0x7f6d3c0ffc80 2023-07-21T18:35:32.368 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:35:32.327+0000 7f6d44383700 1 -- 172.21.15.155:0/3218851440 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d3c19fc60 con 0x7f6d3c0ffc80 2023-07-21T18:35:32.368 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:35:32.327+0000 7f6d337fe700 1 -- 172.21.15.155:0/3218851440 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d38017650 con 0x7f6d3c0ffc80 2023-07-21T18:35:32.368 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:35:32.327+0000 7f6d44383700 1 -- 172.21.15.155:0/3218851440 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d3c1a0070 con 0x7f6d3c0ffc80 2023-07-21T18:35:32.368 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:35:32.328+0000 7f6d44383700 1 -- 172.21.15.155:0/3218851440 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d3c10a3a0 con 0x7f6d3c0ffc80 2023-07-21T18:35:32.369 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:35:32.330+0000 7f6d337fe700 1 -- 172.21.15.155:0/3218851440 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 15) v1 ==== 135659+0+0 (secure 0 0 0) 0x7f6d38022470 con 0x7f6d3c0ffc80 2023-07-21T18:35:32.369 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:35:32.330+0000 7f6d337fe700 1 --2- 172.21.15.155:0/3218851440 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f6d280a10c0 0x7f6d280a3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:35:32.369 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:35:32.330+0000 7f6d337fe700 1 -- 172.21.15.155:0/3218851440 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1767+0+0 (secure 0 0 0) 0x7f6d380cc350 con 0x7f6d3c0ffc80 2023-07-21T18:35:32.369 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:35:32.330+0000 7f6d4211f700 1 --2- 172.21.15.155:0/3218851440 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f6d280a10c0 0x7f6d280a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:35:32.369 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:35:32.331+0000 7f6d4211f700 1 --2- 172.21.15.155:0/3218851440 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f6d280a10c0 0x7f6d280a3570 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c0ffae0 tx=0x7f6d2c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:35:32.370 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:35:32.335+0000 7f6d337fe700 1 -- 172.21.15.155:0/3218851440 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6d38097630 con 0x7f6d3c0ffc80 2023-07-21T18:35:32.563 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:35:32.527+0000 7f6d44383700 1 -- 172.21.15.155:0/3218851440 --> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f6d3c069070 con 0x7f6d280a10c0 2023-07-21T18:35:33.152 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:32 smithi155 ceph-mon[97934]: purged_snaps scrub starts 2023-07-21T18:35:33.152 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:32 smithi155 ceph-mon[97934]: purged_snaps scrub ok 2023-07-21T18:35:33.152 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:32 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T18:35:33.152 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:32 smithi155 ceph-mon[97934]: osd.0 [v2:172.21.15.39:6802/860193857,v1:172.21.15.39:6803/860193857] boot 2023-07-21T18:35:33.152 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:32 smithi155 ceph-mon[97934]: osdmap e8: 1 total, 1 up, 1 in 2023-07-21T18:35:33.152 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:32 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T18:35:33.153 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:32 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T18:35:33.153 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:32 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T18:35:33.153 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:32 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:35:33.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:32 smithi039 ceph-mon[96489]: purged_snaps scrub starts 2023-07-21T18:35:33.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:32 smithi039 ceph-mon[96489]: purged_snaps scrub ok 2023-07-21T18:35:33.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:32 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T18:35:33.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:32 smithi039 ceph-mon[96489]: osd.0 [v2:172.21.15.39:6802/860193857,v1:172.21.15.39:6803/860193857] boot 2023-07-21T18:35:33.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:32 smithi039 ceph-mon[96489]: osdmap e8: 1 total, 1 up, 1 in 2023-07-21T18:35:33.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:32 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T18:35:33.202 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:32 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T18:35:33.202 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:32 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T18:35:33.202 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:32 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:35:33.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:32 smithi193 ceph-mon[98204]: purged_snaps scrub starts 2023-07-21T18:35:33.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:32 smithi193 ceph-mon[98204]: purged_snaps scrub ok 2023-07-21T18:35:33.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:32 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T18:35:33.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:32 smithi193 ceph-mon[98204]: osd.0 [v2:172.21.15.39:6802/860193857,v1:172.21.15.39:6803/860193857] boot 2023-07-21T18:35:33.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:32 smithi193 ceph-mon[98204]: osdmap e8: 1 total, 1 up, 1 in 2023-07-21T18:35:33.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:32 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T18:35:33.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:32 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T18:35:33.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:32 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T18:35:33.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:32 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:35:34.182 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:33 smithi155 ceph-mon[97934]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:35:34.182 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:33 smithi155 ceph-mon[97934]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:35:34.182 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:33 smithi155 ceph-mon[97934]: osdmap e9: 1 total, 1 up, 1 in 2023-07-21T18:35:34.200 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:33 smithi039 ceph-mon[96489]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:35:34.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:33 smithi039 ceph-mon[96489]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:35:34.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:33 smithi039 ceph-mon[96489]: osdmap e9: 1 total, 1 up, 1 in 2023-07-21T18:35:34.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:33 smithi193 ceph-mon[98204]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:35:34.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:33 smithi193 ceph-mon[98204]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:35:34.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:33 smithi193 ceph-mon[98204]: osdmap e9: 1 total, 1 up, 1 in 2023-07-21T18:35:35.200 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:34 smithi039 ceph-mon[96489]: from='client.? 172.21.15.155:0/184493012' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0831a0c0-a1fa-4d30-9a28-513223aa69eb"}]: dispatch 2023-07-21T18:35:35.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:34 smithi039 ceph-mon[96489]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0831a0c0-a1fa-4d30-9a28-513223aa69eb"}]: dispatch 2023-07-21T18:35:35.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:34 smithi039 ceph-mon[96489]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0831a0c0-a1fa-4d30-9a28-513223aa69eb"}]': finished 2023-07-21T18:35:35.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:34 smithi039 ceph-mon[96489]: osdmap e10: 2 total, 1 up, 2 in 2023-07-21T18:35:35.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:34 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T18:35:35.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:34 smithi193 ceph-mon[98204]: from='client.? 172.21.15.155:0/184493012' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0831a0c0-a1fa-4d30-9a28-513223aa69eb"}]: dispatch 2023-07-21T18:35:35.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:34 smithi193 ceph-mon[98204]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0831a0c0-a1fa-4d30-9a28-513223aa69eb"}]: dispatch 2023-07-21T18:35:35.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:34 smithi193 ceph-mon[98204]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0831a0c0-a1fa-4d30-9a28-513223aa69eb"}]': finished 2023-07-21T18:35:35.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:34 smithi193 ceph-mon[98204]: osdmap e10: 2 total, 1 up, 2 in 2023-07-21T18:35:35.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:34 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T18:35:35.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:34 smithi155 ceph-mon[97934]: from='client.? 172.21.15.155:0/184493012' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0831a0c0-a1fa-4d30-9a28-513223aa69eb"}]: dispatch 2023-07-21T18:35:35.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:34 smithi155 ceph-mon[97934]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0831a0c0-a1fa-4d30-9a28-513223aa69eb"}]: dispatch 2023-07-21T18:35:35.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:34 smithi155 ceph-mon[97934]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0831a0c0-a1fa-4d30-9a28-513223aa69eb"}]': finished 2023-07-21T18:35:35.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:34 smithi155 ceph-mon[97934]: osdmap e10: 2 total, 1 up, 2 in 2023-07-21T18:35:35.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:34 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T18:35:36.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:35 smithi039 ceph-mon[96489]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T18:35:36.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:35 smithi039 ceph-mon[96489]: from='client.? 172.21.15.155:0/1402475296' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T18:35:36.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:35 smithi193 ceph-mon[98204]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T18:35:36.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:35 smithi193 ceph-mon[98204]: from='client.? 172.21.15.155:0/1402475296' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T18:35:36.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:35 smithi155 ceph-mon[97934]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T18:35:36.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:35 smithi155 ceph-mon[97934]: from='client.? 172.21.15.155:0/1402475296' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T18:35:37.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:37 smithi039 ceph-mon[96489]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T18:35:37.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:37 smithi193 ceph-mon[98204]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T18:35:37.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:37 smithi155 ceph-mon[97934]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T18:35:39.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:39 smithi039 ceph-mon[96489]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T18:35:39.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:39 smithi193 ceph-mon[98204]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T18:35:39.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:39 smithi155 ceph-mon[97934]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T18:35:42.168 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:41 smithi155 ceph-mon[97934]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T18:35:42.169 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:41 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-07-21T18:35:42.169 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:41 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:35:42.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:41 smithi039 ceph-mon[96489]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T18:35:42.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:41 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-07-21T18:35:42.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:41 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:35:42.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:41 smithi193 ceph-mon[98204]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T18:35:42.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:41 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-07-21T18:35:42.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:41 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:35:43.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:42 smithi039 ceph-mon[96489]: Deploying daemon osd.1 on smithi155 2023-07-21T18:35:43.370 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:42 smithi155 ceph-mon[97934]: Deploying daemon osd.1 on smithi155 2023-07-21T18:35:43.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:42 smithi193 ceph-mon[98204]: Deploying daemon osd.1 on smithi155 2023-07-21T18:35:44.040 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:43 smithi155 ceph-mon[97934]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T18:35:44.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:43 smithi039 ceph-mon[96489]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T18:35:44.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:43 smithi193 ceph-mon[98204]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T18:35:45.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:45 smithi155 ceph-mon[97934]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T18:35:45.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:45 smithi039 ceph-mon[96489]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T18:35:45.634 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:45 smithi193 ceph-mon[98204]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T18:35:47.132 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:47 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:47.132 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:47 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:35:47.133 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:47 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:47.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:47 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:47.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:47 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:35:47.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:47 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:47.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:47 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:47.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:47 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:35:47.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:47 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:48.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:48 smithi193 ceph-mon[98204]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T18:35:48.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:48 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:48.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:48 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:48.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:48 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:35:48.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:48 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:35:48.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:48 smithi155 ceph-mon[97934]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T18:35:48.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:48 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:48.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:48 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:48.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:48 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:35:48.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:48 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:35:48.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:48 smithi039 ceph-mon[96489]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T18:35:48.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:48 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:48.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:48 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:48.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:48 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:35:48.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:48 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:35:48.579 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:35:48.544+0000 7f6d337fe700 1 -- 172.21.15.155:0/3218851440 <== mgr.14152 v2:172.21.15.39:6800/2116291712 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f6d3c069070 con 0x7f6d280a10c0 2023-07-21T18:35:48.580 INFO:teuthology.orchestra.run.smithi155.stdout:Created osd(s) 1 on host 'smithi155' 2023-07-21T18:35:48.581 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:35:48.546+0000 7f6d44383700 1 -- 172.21.15.155:0/3218851440 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f6d280a10c0 msgr2=0x7f6d280a3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:35:48.581 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:35:48.546+0000 7f6d44383700 1 --2- 172.21.15.155:0/3218851440 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f6d280a10c0 0x7f6d280a3570 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c0ffae0 tx=0x7f6d2c008040 comp rx=0 tx=0).stop 2023-07-21T18:35:48.582 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:35:48.546+0000 7f6d44383700 1 -- 172.21.15.155:0/3218851440 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d3c0ffc80 msgr2=0x7f6d3c101060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:35:48.582 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:35:48.546+0000 7f6d44383700 1 --2- 172.21.15.155:0/3218851440 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d3c0ffc80 0x7f6d3c101060 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6d38005910 tx=0x7f6d3800f740 comp rx=0 tx=0).stop 2023-07-21T18:35:48.582 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:35:48.546+0000 7f6d44383700 1 -- 172.21.15.155:0/3218851440 shutdown_connections 2023-07-21T18:35:48.582 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:35:48.546+0000 7f6d44383700 1 --2- 172.21.15.155:0/3218851440 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f6d280a10c0 0x7f6d280a3570 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:35:48.582 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:35:48.546+0000 7f6d44383700 1 --2- 172.21.15.155:0/3218851440 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d3c100610 0x7f6d3c1015a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:35:48.582 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:35:48.546+0000 7f6d44383700 1 --2- 172.21.15.155:0/3218851440 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d3c0fea80 0x7f6d3c102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:35:48.582 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:35:48.546+0000 7f6d44383700 1 --2- 172.21.15.155:0/3218851440 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d3c0ffc80 0x7f6d3c101060 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:35:48.583 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:35:48.546+0000 7f6d44383700 1 -- 172.21.15.155:0/3218851440 >> 172.21.15.155:0/3218851440 conn(0x7f6d3c0fa0b0 msgr2=0x7f6d3c068960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:35:48.583 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:35:48.546+0000 7f6d44383700 1 -- 172.21.15.155:0/3218851440 shutdown_connections 2023-07-21T18:35:48.583 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:35:48.546+0000 7f6d44383700 1 -- 172.21.15.155:0/3218851440 wait complete. 2023-07-21T18:35:49.045 DEBUG:teuthology.orchestra.run.smithi155:osd.1> sudo journalctl -f -n 0 -u ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@osd.1.service 2023-07-21T18:35:49.047 INFO:tasks.cephadm:Deploying osd.2 on smithi193 with /dev/vg_nvme/lv_4... 2023-07-21T18:35:49.047 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-07-21T18:35:49.076 INFO:journalctl@ceph.osd.1.smithi155.stdout:-- Logs begin at Fri 2023-07-21 18:17:11 UTC. -- 2023-07-21T18:35:49.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:49 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:49.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:49 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:35:49.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:49 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:49.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:49 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:49.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:49 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:49.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:49 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:35:49.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:49 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:49.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:49 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:49.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:49 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:49.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:49 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:35:49.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:49 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:49.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:49 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:49.606 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T18:35:50.071 INFO:journalctl@ceph.osd.1.smithi155.stdout:Jul 21 18:35:49 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-1[103027]: 2023-07-21T18:35:49.887+0000 7f5509d1d640 -1 osd.1 0 log_to_monitors true 2023-07-21T18:35:50.336 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:50 smithi155 ceph-mon[97934]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T18:35:50.336 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:50 smithi155 ceph-mon[97934]: from='osd.1 [v2:172.21.15.155:6800/3246721764,v1:172.21.15.155:6801/3246721764]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-21T18:35:50.336 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:50 smithi155 ceph-mon[97934]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-21T18:35:50.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:50 smithi193 ceph-mon[98204]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T18:35:50.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:50 smithi193 ceph-mon[98204]: from='osd.1 [v2:172.21.15.155:6800/3246721764,v1:172.21.15.155:6801/3246721764]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-21T18:35:50.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:50 smithi193 ceph-mon[98204]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-21T18:35:50.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:50 smithi039 ceph-mon[96489]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T18:35:50.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:50 smithi039 ceph-mon[96489]: from='osd.1 [v2:172.21.15.155:6800/3246721764,v1:172.21.15.155:6801/3246721764]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-21T18:35:50.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:50 smithi039 ceph-mon[96489]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-21T18:35:51.000 INFO:teuthology.orchestra.run.smithi193.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-07-21T18:35:51.000 INFO:teuthology.orchestra.run.smithi193.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-07-21T18:35:51.000 INFO:teuthology.orchestra.run.smithi193.stderr: stderr: 10+0 records in 2023-07-21T18:35:51.001 INFO:teuthology.orchestra.run.smithi193.stderr:10+0 records out 2023-07-21T18:35:51.001 INFO:teuthology.orchestra.run.smithi193.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110527 s, 949 MB/s 2023-07-21T18:35:51.001 INFO:teuthology.orchestra.run.smithi193.stderr:--> Zapping successful for: 2023-07-21T18:35:51.152 INFO:journalctl@ceph.osd.1.smithi155.stdout:Jul 21 18:35:51 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-1[103027]: 2023-07-21T18:35:51.049+0000 7f54feb94700 -1 osd.1 0 waiting for initial osdmap 2023-07-21T18:35:51.152 INFO:journalctl@ceph.osd.1.smithi155.stdout:Jul 21 18:35:51 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-1[103027]: 2023-07-21T18:35:51.053+0000 7f54fb18a700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-21T18:35:51.152 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:51 smithi155 ceph-mon[97934]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-07-21T18:35:51.153 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:51 smithi155 ceph-mon[97934]: osdmap e11: 2 total, 1 up, 2 in 2023-07-21T18:35:51.153 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:51 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T18:35:51.153 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:51 smithi155 ceph-mon[97934]: from='osd.1 [v2:172.21.15.155:6800/3246721764,v1:172.21.15.155:6801/3246721764]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2023-07-21T18:35:51.153 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:51 smithi155 ceph-mon[97934]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2023-07-21T18:35:51.330 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:51 smithi193 ceph-mon[98204]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-07-21T18:35:51.330 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:51 smithi193 ceph-mon[98204]: osdmap e11: 2 total, 1 up, 2 in 2023-07-21T18:35:51.330 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:51 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T18:35:51.330 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:51 smithi193 ceph-mon[98204]: from='osd.1 [v2:172.21.15.155:6800/3246721764,v1:172.21.15.155:6801/3246721764]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2023-07-21T18:35:51.330 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:51 smithi193 ceph-mon[98204]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2023-07-21T18:35:51.358 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi193:vg_nvme/lv_4 2023-07-21T18:35:51.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:51 smithi039 ceph-mon[96489]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-07-21T18:35:51.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:51 smithi039 ceph-mon[96489]: osdmap e11: 2 total, 1 up, 2 in 2023-07-21T18:35:51.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:51 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T18:35:51.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:51 smithi039 ceph-mon[96489]: from='osd.1 [v2:172.21.15.155:6800/3246721764,v1:172.21.15.155:6801/3246721764]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2023-07-21T18:35:51.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:51 smithi039 ceph-mon[96489]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2023-07-21T18:35:51.907 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T18:35:52.381 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:52 smithi193 ceph-mon[98204]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2023-07-21T18:35:52.382 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:52 smithi193 ceph-mon[98204]: osdmap e12: 2 total, 1 up, 2 in 2023-07-21T18:35:52.382 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:52 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T18:35:52.382 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:52 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T18:35:52.382 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:52 smithi193 ceph-mon[98204]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T18:35:52.382 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:52 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:52.382 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:52 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:52.382 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:52 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:35:52.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:52 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:52.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:52 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:35:52.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:52 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:35:52.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:52 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:52.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:52 smithi155 ceph-mon[97934]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2023-07-21T18:35:52.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:52 smithi155 ceph-mon[97934]: osdmap e12: 2 total, 1 up, 2 in 2023-07-21T18:35:52.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:52 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T18:35:52.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:52 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T18:35:52.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:52 smithi155 ceph-mon[97934]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T18:35:52.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:52 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:52.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:52 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:52.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:52 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:35:52.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:52 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:52.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:52 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:35:52.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:52 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:35:52.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:52 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:52.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:52 smithi039 ceph-mon[96489]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2023-07-21T18:35:52.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:52 smithi039 ceph-mon[96489]: osdmap e12: 2 total, 1 up, 2 in 2023-07-21T18:35:52.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:52 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T18:35:52.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:52 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T18:35:52.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:52 smithi039 ceph-mon[96489]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T18:35:52.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:52 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:52.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:52 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:52.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:52 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:35:52.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:52 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:52.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:52 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:35:52.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:52 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:35:52.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:52 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:35:53.003 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:52.966+0000 7efd17619700 1 -- 172.21.15.193:0/2353627132 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efd100ffd10 msgr2=0x7efd10100160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:35:53.003 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:52.966+0000 7efd17619700 1 --2- 172.21.15.193:0/2353627132 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efd100ffd10 0x7efd10100160 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7efd00009990 tx=0x7efd00009ca0 comp rx=0 tx=0).stop 2023-07-21T18:35:53.003 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:52.966+0000 7efd17619700 1 -- 172.21.15.193:0/2353627132 shutdown_connections 2023-07-21T18:35:53.004 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:52.966+0000 7efd17619700 1 --2- 172.21.15.193:0/2353627132 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd100feb10 0x7efd100fef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:35:53.004 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:52.966+0000 7efd17619700 1 --2- 172.21.15.193:0/2353627132 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd101006a0 0x7efd10106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:35:53.004 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:52.966+0000 7efd17619700 1 --2- 172.21.15.193:0/2353627132 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efd100ffd10 0x7efd10100160 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:35:53.004 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:52.966+0000 7efd17619700 1 -- 172.21.15.193:0/2353627132 >> 172.21.15.193:0/2353627132 conn(0x7efd100fa160 msgr2=0x7efd100fc590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:35:53.004 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:52.966+0000 7efd17619700 1 -- 172.21.15.193:0/2353627132 shutdown_connections 2023-07-21T18:35:53.004 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:52.966+0000 7efd17619700 1 -- 172.21.15.193:0/2353627132 wait complete. 2023-07-21T18:35:53.004 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:52.966+0000 7efd17619700 1 Processor -- start 2023-07-21T18:35:53.005 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:52.966+0000 7efd17619700 1 -- start start 2023-07-21T18:35:53.005 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:52.966+0000 7efd17619700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd100feb10 0x7efd10195860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:35:53.005 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:52.967+0000 7efd17619700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd100ffd10 0x7efd10195da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:35:53.005 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:52.967+0000 7efd17619700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efd101006a0 0x7efd1019ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:35:53.005 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:52.967+0000 7efd17619700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd1019d3f0 con 0x7efd100ffd10 2023-07-21T18:35:53.005 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:52.967+0000 7efd17619700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd1019d530 con 0x7efd100feb10 2023-07-21T18:35:53.005 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:52.967+0000 7efd17619700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd1019d6c0 con 0x7efd101006a0 2023-07-21T18:35:53.006 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:52.967+0000 7efd15bb6700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efd101006a0 0x7efd1019ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:35:53.006 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:52.967+0000 7efd15bb6700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efd101006a0 0x7efd1019ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:50572/0 (socket says 172.21.15.193:50572) 2023-07-21T18:35:53.006 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:52.967+0000 7efd15bb6700 1 -- 172.21.15.193:0/1864445459 learned_addr learned my addr 172.21.15.193:0/1864445459 (peer_addr_for_me v2:172.21.15.193:0/0) 2023-07-21T18:35:53.006 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:52.967+0000 7efd153b5700 1 --2- 172.21.15.193:0/1864445459 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd100feb10 0x7efd10195860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:35:53.006 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:52.967+0000 7efd14bb4700 1 --2- 172.21.15.193:0/1864445459 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd100ffd10 0x7efd10195da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:35:53.006 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:52.967+0000 7efd15bb6700 1 -- 172.21.15.193:0/1864445459 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd100feb10 msgr2=0x7efd10195860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:35:53.006 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:52.967+0000 7efd15bb6700 1 --2- 172.21.15.193:0/1864445459 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd100feb10 0x7efd10195860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:35:53.006 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:52.967+0000 7efd15bb6700 1 -- 172.21.15.193:0/1864445459 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd100ffd10 msgr2=0x7efd10195da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:35:53.007 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:52.967+0000 7efd15bb6700 1 --2- 172.21.15.193:0/1864445459 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd100ffd10 0x7efd10195da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:35:53.007 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:52.967+0000 7efd15bb6700 1 -- 172.21.15.193:0/1864445459 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd000095f0 con 0x7efd101006a0 2023-07-21T18:35:53.007 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:52.967+0000 7efd15bb6700 1 --2- 172.21.15.193:0/1864445459 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efd101006a0 0x7efd1019ce20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7efd0800ef60 tx=0x7efd0800ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:35:53.007 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:52.967+0000 7efd067fc700 1 -- 172.21.15.193:0/1864445459 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd08018070 con 0x7efd101006a0 2023-07-21T18:35:53.007 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:52.967+0000 7efd067fc700 1 -- 172.21.15.193:0/1864445459 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd0800f040 con 0x7efd101006a0 2023-07-21T18:35:53.007 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:52.967+0000 7efd17619700 1 -- 172.21.15.193:0/1864445459 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd1019d940 con 0x7efd101006a0 2023-07-21T18:35:53.007 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:52.967+0000 7efd067fc700 1 -- 172.21.15.193:0/1864445459 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd0800ccc0 con 0x7efd101006a0 2023-07-21T18:35:53.007 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:52.967+0000 7efd17619700 1 -- 172.21.15.193:0/1864445459 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd1019ddf0 con 0x7efd101006a0 2023-07-21T18:35:53.008 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:52.968+0000 7efd17619700 1 -- 172.21.15.193:0/1864445459 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd1010a2d0 con 0x7efd101006a0 2023-07-21T18:35:53.009 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:52.969+0000 7efd067fc700 1 -- 172.21.15.193:0/1864445459 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 15) v1 ==== 135659+0+0 (secure 0 0 0) 0x7efd08003a00 con 0x7efd101006a0 2023-07-21T18:35:53.009 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:52.970+0000 7efd067fc700 1 --2- 172.21.15.193:0/1864445459 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7efcfc0a10c0 0x7efcfc0a3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:35:53.009 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:52.970+0000 7efd067fc700 1 -- 172.21.15.193:0/1864445459 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2309+0+0 (secure 0 0 0) 0x7efd080cb530 con 0x7efd101006a0 2023-07-21T18:35:53.009 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:52.970+0000 7efd153b5700 1 --2- 172.21.15.193:0/1864445459 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7efcfc0a10c0 0x7efcfc0a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:35:53.009 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:52.971+0000 7efd153b5700 1 --2- 172.21.15.193:0/1864445459 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7efcfc0a10c0 0x7efcfc0a3570 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7efd100ffb70 tx=0x7efd0c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:35:53.010 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:52.973+0000 7efd067fc700 1 -- 172.21.15.193:0/1864445459 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7efd08096560 con 0x7efd101006a0 2023-07-21T18:35:53.206 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:35:53.170+0000 7efd17619700 1 -- 172.21.15.193:0/1864445459 --> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7efd10069040 con 0x7efcfc0a10c0 2023-07-21T18:35:53.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:53 smithi193 ceph-mon[98204]: purged_snaps scrub starts 2023-07-21T18:35:53.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:53 smithi193 ceph-mon[98204]: purged_snaps scrub ok 2023-07-21T18:35:53.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:53 smithi193 ceph-mon[98204]: Detected new or changed devices on smithi155 2023-07-21T18:35:53.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:53 smithi193 ceph-mon[98204]: Adjusting osd_memory_target on smithi155 to 17168M 2023-07-21T18:35:53.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:53 smithi193 ceph-mon[98204]: osd.1 [v2:172.21.15.155:6800/3246721764,v1:172.21.15.155:6801/3246721764] boot 2023-07-21T18:35:53.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:53 smithi193 ceph-mon[98204]: osdmap e13: 2 total, 2 up, 2 in 2023-07-21T18:35:53.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:53 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T18:35:53.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:53 smithi155 ceph-mon[97934]: purged_snaps scrub starts 2023-07-21T18:35:53.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:53 smithi155 ceph-mon[97934]: purged_snaps scrub ok 2023-07-21T18:35:53.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:53 smithi155 ceph-mon[97934]: Detected new or changed devices on smithi155 2023-07-21T18:35:53.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:53 smithi155 ceph-mon[97934]: Adjusting osd_memory_target on smithi155 to 17168M 2023-07-21T18:35:53.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:53 smithi155 ceph-mon[97934]: osd.1 [v2:172.21.15.155:6800/3246721764,v1:172.21.15.155:6801/3246721764] boot 2023-07-21T18:35:53.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:53 smithi155 ceph-mon[97934]: osdmap e13: 2 total, 2 up, 2 in 2023-07-21T18:35:53.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:53 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T18:35:53.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:53 smithi039 ceph-mon[96489]: purged_snaps scrub starts 2023-07-21T18:35:53.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:53 smithi039 ceph-mon[96489]: purged_snaps scrub ok 2023-07-21T18:35:53.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:53 smithi039 ceph-mon[96489]: Detected new or changed devices on smithi155 2023-07-21T18:35:53.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:53 smithi039 ceph-mon[96489]: Adjusting osd_memory_target on smithi155 to 17168M 2023-07-21T18:35:53.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:53 smithi039 ceph-mon[96489]: osd.1 [v2:172.21.15.155:6800/3246721764,v1:172.21.15.155:6801/3246721764] boot 2023-07-21T18:35:53.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:53 smithi039 ceph-mon[96489]: osdmap e13: 2 total, 2 up, 2 in 2023-07-21T18:35:53.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:53 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T18:35:54.264 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:54 smithi193 ceph-mon[98204]: pgmap v57: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T18:35:54.264 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:54 smithi193 ceph-mon[98204]: osdmap e14: 2 total, 2 up, 2 in 2023-07-21T18:35:54.265 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:54 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T18:35:54.265 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:54 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T18:35:54.265 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:54 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:35:54.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:54 smithi155 ceph-mon[97934]: pgmap v57: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T18:35:54.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:54 smithi155 ceph-mon[97934]: osdmap e14: 2 total, 2 up, 2 in 2023-07-21T18:35:54.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:54 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T18:35:54.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:54 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T18:35:54.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:54 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:35:54.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:54 smithi039 ceph-mon[96489]: pgmap v57: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T18:35:54.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:54 smithi039 ceph-mon[96489]: osdmap e14: 2 total, 2 up, 2 in 2023-07-21T18:35:54.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:54 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T18:35:54.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:54 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T18:35:54.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:54 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:35:55.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:55 smithi193 ceph-mon[98204]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:35:55.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:55 smithi155 ceph-mon[97934]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:35:55.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:55 smithi039 ceph-mon[96489]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:35:56.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:56 smithi193 ceph-mon[98204]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T18:35:56.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:56 smithi193 ceph-mon[98204]: from='client.? 172.21.15.193:0/830485092' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22df0003-bde9-48e4-b47a-cb8eafbab0db"}]: dispatch 2023-07-21T18:35:56.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:56 smithi193 ceph-mon[98204]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22df0003-bde9-48e4-b47a-cb8eafbab0db"}]: dispatch 2023-07-21T18:35:56.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:56 smithi193 ceph-mon[98204]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "22df0003-bde9-48e4-b47a-cb8eafbab0db"}]': finished 2023-07-21T18:35:56.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:56 smithi193 ceph-mon[98204]: osdmap e15: 3 total, 2 up, 3 in 2023-07-21T18:35:56.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:56 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T18:35:56.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:56 smithi193 ceph-mon[98204]: from='client.? 172.21.15.193:0/2560231481' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T18:35:56.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:56 smithi155 ceph-mon[97934]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T18:35:56.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:56 smithi155 ceph-mon[97934]: from='client.? 172.21.15.193:0/830485092' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22df0003-bde9-48e4-b47a-cb8eafbab0db"}]: dispatch 2023-07-21T18:35:56.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:56 smithi155 ceph-mon[97934]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22df0003-bde9-48e4-b47a-cb8eafbab0db"}]: dispatch 2023-07-21T18:35:56.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:56 smithi155 ceph-mon[97934]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "22df0003-bde9-48e4-b47a-cb8eafbab0db"}]': finished 2023-07-21T18:35:56.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:56 smithi155 ceph-mon[97934]: osdmap e15: 3 total, 2 up, 3 in 2023-07-21T18:35:56.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:56 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T18:35:56.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:56 smithi155 ceph-mon[97934]: from='client.? 172.21.15.193:0/2560231481' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T18:35:56.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:56 smithi039 ceph-mon[96489]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T18:35:56.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:56 smithi039 ceph-mon[96489]: from='client.? 172.21.15.193:0/830485092' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22df0003-bde9-48e4-b47a-cb8eafbab0db"}]: dispatch 2023-07-21T18:35:56.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:56 smithi039 ceph-mon[96489]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22df0003-bde9-48e4-b47a-cb8eafbab0db"}]: dispatch 2023-07-21T18:35:56.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:56 smithi039 ceph-mon[96489]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "22df0003-bde9-48e4-b47a-cb8eafbab0db"}]': finished 2023-07-21T18:35:56.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:56 smithi039 ceph-mon[96489]: osdmap e15: 3 total, 2 up, 3 in 2023-07-21T18:35:56.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:56 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T18:35:56.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:56 smithi039 ceph-mon[96489]: from='client.? 172.21.15.193:0/2560231481' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T18:35:58.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:58 smithi155 ceph-mon[97934]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T18:35:58.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:58 smithi039 ceph-mon[96489]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T18:35:58.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:58 smithi193 ceph-mon[98204]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T18:35:59.447 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:35:59 smithi193 ceph-mon[98204]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T18:35:59.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:35:59 smithi039 ceph-mon[96489]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T18:35:59.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:35:59 smithi155 ceph-mon[97934]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T18:36:02.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:01 smithi039 ceph-mon[96489]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T18:36:02.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:01 smithi193 ceph-mon[98204]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T18:36:02.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:01 smithi155 ceph-mon[97934]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T18:36:03.133 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:02 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-07-21T18:36:03.133 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:02 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:36:03.134 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:02 smithi193 ceph-mon[98204]: Deploying daemon osd.2 on smithi193 2023-07-21T18:36:03.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:02 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-07-21T18:36:03.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:02 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:36:03.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:02 smithi039 ceph-mon[96489]: Deploying daemon osd.2 on smithi193 2023-07-21T18:36:03.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:02 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-07-21T18:36:03.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:02 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:36:03.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:02 smithi155 ceph-mon[97934]: Deploying daemon osd.2 on smithi193 2023-07-21T18:36:04.019 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:03 smithi193 ceph-mon[98204]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T18:36:04.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:03 smithi039 ceph-mon[96489]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T18:36:04.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:03 smithi155 ceph-mon[97934]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T18:36:05.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:05 smithi039 ceph-mon[96489]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T18:36:05.488 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:05 smithi193 ceph-mon[98204]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T18:36:05.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:05 smithi155 ceph-mon[97934]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T18:36:07.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:07 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:07.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:07 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:07.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:07 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:36:07.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:07 smithi193 ceph-mon[98204]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T18:36:07.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:07 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:07.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:07 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:07.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:07 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:36:07.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:07 smithi039 ceph-mon[96489]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T18:36:07.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:07 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:07.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:07 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:07.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:07 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:36:07.653 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:07 smithi155 ceph-mon[97934]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T18:36:08.917 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:36:08.881+0000 7efd067fc700 1 -- 172.21.15.193:0/1864445459 <== mgr.14152 v2:172.21.15.39:6800/2116291712 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7efd10069040 con 0x7efcfc0a10c0 2023-07-21T18:36:08.917 INFO:teuthology.orchestra.run.smithi193.stdout:Created osd(s) 2 on host 'smithi193' 2023-07-21T18:36:08.919 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:36:08.883+0000 7efd17619700 1 -- 172.21.15.193:0/1864445459 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7efcfc0a10c0 msgr2=0x7efcfc0a3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:36:08.920 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:36:08.883+0000 7efd17619700 1 --2- 172.21.15.193:0/1864445459 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7efcfc0a10c0 0x7efcfc0a3570 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7efd100ffb70 tx=0x7efd0c009260 comp rx=0 tx=0).stop 2023-07-21T18:36:08.920 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:36:08.883+0000 7efd17619700 1 -- 172.21.15.193:0/1864445459 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efd101006a0 msgr2=0x7efd1019ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:36:08.920 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:36:08.883+0000 7efd17619700 1 --2- 172.21.15.193:0/1864445459 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efd101006a0 0x7efd1019ce20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7efd0800ef60 tx=0x7efd0800ef90 comp rx=0 tx=0).stop 2023-07-21T18:36:08.920 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:36:08.883+0000 7efd17619700 1 -- 172.21.15.193:0/1864445459 shutdown_connections 2023-07-21T18:36:08.920 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:36:08.883+0000 7efd17619700 1 --2- 172.21.15.193:0/1864445459 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7efcfc0a10c0 0x7efcfc0a3570 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:36:08.920 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:36:08.883+0000 7efd17619700 1 --2- 172.21.15.193:0/1864445459 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd100feb10 0x7efd10195860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:36:08.920 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:36:08.883+0000 7efd17619700 1 --2- 172.21.15.193:0/1864445459 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efd101006a0 0x7efd1019ce20 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:36:08.921 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:36:08.883+0000 7efd17619700 1 --2- 172.21.15.193:0/1864445459 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd100ffd10 0x7efd10195da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:36:08.921 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:36:08.883+0000 7efd17619700 1 -- 172.21.15.193:0/1864445459 >> 172.21.15.193:0/1864445459 conn(0x7efd100fa160 msgr2=0x7efd10068930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:36:08.921 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:36:08.883+0000 7efd17619700 1 -- 172.21.15.193:0/1864445459 shutdown_connections 2023-07-21T18:36:08.921 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:36:08.883+0000 7efd17619700 1 -- 172.21.15.193:0/1864445459 wait complete. 2023-07-21T18:36:09.133 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:08 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:09.134 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:08 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:09.134 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:08 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:36:09.134 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:08 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:36:09.134 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:08 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:09.134 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:08 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:09.134 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:08 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:09.134 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:08 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:36:09.298 DEBUG:teuthology.orchestra.run.smithi193:osd.2> sudo journalctl -f -n 0 -u ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@osd.2.service 2023-07-21T18:36:09.300 INFO:tasks.cephadm:Deploying osd.3 on smithi039 with /dev/vg_nvme/lv_3... 2023-07-21T18:36:09.300 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-07-21T18:36:09.332 INFO:journalctl@ceph.osd.2.smithi193.stdout:-- Logs begin at Fri 2023-07-21 18:17:09 UTC. -- 2023-07-21T18:36:09.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:08 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:09.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:08 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:09.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:08 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:36:09.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:08 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:36:09.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:08 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:09.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:08 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:09.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:08 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:09.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:08 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:36:09.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:08 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:09.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:08 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:09.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:08 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:36:09.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:08 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:36:09.404 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:08 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:09.404 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:08 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:09.405 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:08 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:09.405 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:08 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:36:09.876 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:36:10.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:09 smithi039 ceph-mon[96489]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T18:36:10.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:09 smithi193 ceph-mon[98204]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T18:36:10.384 INFO:journalctl@ceph.osd.2.smithi193.stdout:Jul 21 18:36:10 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-2[104065]: 2023-07-21T18:36:10.150+0000 7f67a22f0640 -1 osd.2 0 log_to_monitors true 2023-07-21T18:36:10.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:09 smithi155 ceph-mon[97934]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T18:36:11.036 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:10 smithi039 ceph-mon[96489]: from='osd.2 [v2:172.21.15.193:6800/2563594449,v1:172.21.15.193:6801/2563594449]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-21T18:36:11.037 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:10 smithi039 ceph-mon[96489]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-21T18:36:11.293 INFO:teuthology.orchestra.run.smithi039.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-07-21T18:36:11.293 INFO:teuthology.orchestra.run.smithi039.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-07-21T18:36:11.294 INFO:teuthology.orchestra.run.smithi039.stderr: stderr: 10+0 records in 2023-07-21T18:36:11.294 INFO:teuthology.orchestra.run.smithi039.stderr:10+0 records out 2023-07-21T18:36:11.294 INFO:teuthology.orchestra.run.smithi039.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0138411 s, 758 MB/s 2023-07-21T18:36:11.294 INFO:teuthology.orchestra.run.smithi039.stderr: stderr: 2023-07-21T18:36:11.294 INFO:teuthology.orchestra.run.smithi039.stderr:--> Zapping successful for: 2023-07-21T18:36:11.319 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:10 smithi193 ceph-mon[98204]: from='osd.2 [v2:172.21.15.193:6800/2563594449,v1:172.21.15.193:6801/2563594449]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-21T18:36:11.319 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:10 smithi193 ceph-mon[98204]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-21T18:36:11.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:10 smithi155 ceph-mon[97934]: from='osd.2 [v2:172.21.15.193:6800/2563594449,v1:172.21.15.193:6801/2563594449]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-21T18:36:11.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:10 smithi155 ceph-mon[97934]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-21T18:36:11.737 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi039:vg_nvme/lv_3 2023-07-21T18:36:12.335 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:36:12.383 INFO:journalctl@ceph.osd.2.smithi193.stdout:Jul 21 18:36:12 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-2[104065]: 2023-07-21T18:36:11.997+0000 7f6797167700 -1 osd.2 0 waiting for initial osdmap 2023-07-21T18:36:12.384 INFO:journalctl@ceph.osd.2.smithi193.stdout:Jul 21 18:36:12 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-2[104065]: 2023-07-21T18:36:12.002+0000 7f678ff56700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-21T18:36:12.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:11 smithi193 ceph-mon[98204]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-07-21T18:36:12.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:11 smithi193 ceph-mon[98204]: osdmap e16: 3 total, 2 up, 3 in 2023-07-21T18:36:12.385 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:11 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T18:36:12.385 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:11 smithi193 ceph-mon[98204]: from='osd.2 [v2:172.21.15.193:6800/2563594449,v1:172.21.15.193:6801/2563594449]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2023-07-21T18:36:12.385 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:11 smithi193 ceph-mon[98204]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2023-07-21T18:36:12.385 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:11 smithi193 ceph-mon[98204]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T18:36:12.385 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:11 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:12.385 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:11 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:12.385 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:11 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:36:12.386 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:11 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:12.386 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:11 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:36:12.386 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:11 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:36:12.386 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:11 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:12.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:11 smithi155 ceph-mon[97934]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-07-21T18:36:12.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:11 smithi155 ceph-mon[97934]: osdmap e16: 3 total, 2 up, 3 in 2023-07-21T18:36:12.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:11 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T18:36:12.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:11 smithi155 ceph-mon[97934]: from='osd.2 [v2:172.21.15.193:6800/2563594449,v1:172.21.15.193:6801/2563594449]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2023-07-21T18:36:12.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:11 smithi155 ceph-mon[97934]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2023-07-21T18:36:12.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:11 smithi155 ceph-mon[97934]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T18:36:12.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:11 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:12.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:11 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:12.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:11 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:36:12.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:11 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:12.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:11 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:36:12.404 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:11 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:36:12.404 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:11 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:12.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:11 smithi039 ceph-mon[96489]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-07-21T18:36:12.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:11 smithi039 ceph-mon[96489]: osdmap e16: 3 total, 2 up, 3 in 2023-07-21T18:36:12.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:11 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T18:36:12.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:11 smithi039 ceph-mon[96489]: from='osd.2 [v2:172.21.15.193:6800/2563594449,v1:172.21.15.193:6801/2563594449]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2023-07-21T18:36:12.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:11 smithi039 ceph-mon[96489]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2023-07-21T18:36:12.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:11 smithi039 ceph-mon[96489]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T18:36:12.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:11 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:12.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:11 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:12.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:11 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:36:12.453 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:11 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:12.453 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:11 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:36:12.453 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:11 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:36:12.453 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:11 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:13.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:13 smithi039 ceph-mon[96489]: Detected new or changed devices on smithi193 2023-07-21T18:36:13.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:13 smithi039 ceph-mon[96489]: Adjusting osd_memory_target on smithi193 to 17168M 2023-07-21T18:36:13.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:13 smithi039 ceph-mon[96489]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2023-07-21T18:36:13.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:13 smithi039 ceph-mon[96489]: osdmap e17: 3 total, 2 up, 3 in 2023-07-21T18:36:13.202 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:13 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T18:36:13.202 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:13 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T18:36:13.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:13 smithi193 ceph-mon[98204]: Detected new or changed devices on smithi193 2023-07-21T18:36:13.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:13 smithi193 ceph-mon[98204]: Adjusting osd_memory_target on smithi193 to 17168M 2023-07-21T18:36:13.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:13 smithi193 ceph-mon[98204]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2023-07-21T18:36:13.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:13 smithi193 ceph-mon[98204]: osdmap e17: 3 total, 2 up, 3 in 2023-07-21T18:36:13.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:13 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T18:36:13.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:13 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T18:36:13.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:13 smithi155 ceph-mon[97934]: Detected new or changed devices on smithi193 2023-07-21T18:36:13.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:13 smithi155 ceph-mon[97934]: Adjusting osd_memory_target on smithi193 to 17168M 2023-07-21T18:36:13.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:13 smithi155 ceph-mon[97934]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2023-07-21T18:36:13.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:13 smithi155 ceph-mon[97934]: osdmap e17: 3 total, 2 up, 3 in 2023-07-21T18:36:13.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:13 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T18:36:13.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:13 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T18:36:13.608 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:36:13.572+0000 7f41453ac700 1 -- 172.21.15.39:0/3308770013 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f41400fea80 msgr2=0x7f41400fee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:36:13.609 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:36:13.572+0000 7f41453ac700 1 --2- 172.21.15.39:0/3308770013 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f41400fea80 0x7f41400fee90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4130009990 tx=0x7f4130009ca0 comp rx=0 tx=0).stop 2023-07-21T18:36:13.609 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:36:13.572+0000 7f41453ac700 1 -- 172.21.15.39:0/3308770013 shutdown_connections 2023-07-21T18:36:13.609 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:36:13.572+0000 7f41453ac700 1 --2- 172.21.15.39:0/3308770013 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4140100610 0x7f4140106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:36:13.610 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:36:13.572+0000 7f41453ac700 1 --2- 172.21.15.39:0/3308770013 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f41400fea80 0x7f41400fee90 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:36:13.610 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:36:13.572+0000 7f41453ac700 1 --2- 172.21.15.39:0/3308770013 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41400ffc80 0x7f41401000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:36:13.610 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:36:13.572+0000 7f41453ac700 1 -- 172.21.15.39:0/3308770013 >> 172.21.15.39:0/3308770013 conn(0x7f41400fa0b0 msgr2=0x7f41400fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:36:13.610 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:36:13.572+0000 7f41453ac700 1 -- 172.21.15.39:0/3308770013 shutdown_connections 2023-07-21T18:36:13.610 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:36:13.572+0000 7f41453ac700 1 -- 172.21.15.39:0/3308770013 wait complete. 2023-07-21T18:36:13.612 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:36:13.572+0000 7f41453ac700 1 Processor -- start 2023-07-21T18:36:13.612 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:36:13.573+0000 7f41453ac700 1 -- start start 2023-07-21T18:36:13.612 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:36:13.573+0000 7f41453ac700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41400fea80 0x7f4140109330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:36:13.612 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:36:13.573+0000 7f41453ac700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41400ffc80 0x7f4140107980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:36:13.613 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:36:13.573+0000 7f41453ac700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4140100610 0x7f4140107ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:36:13.613 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:36:13.573+0000 7f41453ac700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4140108640 con 0x7f4140100610 2023-07-21T18:36:13.613 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:36:13.573+0000 7f41453ac700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41401087b0 con 0x7f41400ffc80 2023-07-21T18:36:13.613 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:36:13.573+0000 7f41453ac700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f414019f910 con 0x7f41400fea80 2023-07-21T18:36:13.613 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:36:13.573+0000 7f413f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4140100610 0x7f4140107ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:36:13.613 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:36:13.573+0000 7f413f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4140100610 0x7f4140107ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54198/0 (socket says 172.21.15.39:54198) 2023-07-21T18:36:13.613 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:36:13.573+0000 7f413f7fe700 1 -- 172.21.15.39:0/924428875 learned_addr learned my addr 172.21.15.39:0/924428875 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:36:13.613 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:36:13.573+0000 7f413e7fc700 1 --2- 172.21.15.39:0/924428875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41400ffc80 0x7f4140107980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:36:13.614 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:36:13.573+0000 7f413effd700 1 --2- 172.21.15.39:0/924428875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41400fea80 0x7f4140109330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:36:13.614 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:36:13.573+0000 7f413f7fe700 1 -- 172.21.15.39:0/924428875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41400fea80 msgr2=0x7f4140109330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:36:13.614 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:36:13.573+0000 7f413f7fe700 1 --2- 172.21.15.39:0/924428875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41400fea80 0x7f4140109330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:36:13.614 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:36:13.573+0000 7f413f7fe700 1 -- 172.21.15.39:0/924428875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41400ffc80 msgr2=0x7f4140107980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:36:13.614 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:36:13.573+0000 7f413f7fe700 1 --2- 172.21.15.39:0/924428875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41400ffc80 0x7f4140107980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:36:13.614 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:36:13.573+0000 7f413f7fe700 1 -- 172.21.15.39:0/924428875 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4134009520 con 0x7f4140100610 2023-07-21T18:36:13.614 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:36:13.574+0000 7f413f7fe700 1 --2- 172.21.15.39:0/924428875 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4140100610 0x7f4140107ec0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f413800c370 tx=0x7f413800c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:36:13.615 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:36:13.574+0000 7f4127fff700 1 -- 172.21.15.39:0/924428875 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4138017070 con 0x7f4140100610 2023-07-21T18:36:13.615 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:36:13.574+0000 7f41453ac700 1 -- 172.21.15.39:0/924428875 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4138009c40 con 0x7f4140100610 2023-07-21T18:36:13.615 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:36:13.574+0000 7f4127fff700 1 -- 172.21.15.39:0/924428875 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f413800f040 con 0x7f4140100610 2023-07-21T18:36:13.615 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:36:13.574+0000 7f4127fff700 1 -- 172.21.15.39:0/924428875 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f413800e050 con 0x7f4140100610 2023-07-21T18:36:13.615 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:36:13.574+0000 7f41453ac700 1 -- 172.21.15.39:0/924428875 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41300095f0 con 0x7f4140100610 2023-07-21T18:36:13.615 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:36:13.575+0000 7f41453ac700 1 -- 172.21.15.39:0/924428875 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f414010b3a0 con 0x7f4140100610 2023-07-21T18:36:13.616 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:36:13.577+0000 7f4127fff700 1 -- 172.21.15.39:0/924428875 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 15) v1 ==== 135659+0+0 (secure 0 0 0) 0x7f4138004520 con 0x7f4140100610 2023-07-21T18:36:13.616 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:36:13.577+0000 7f4127fff700 1 --2- 172.21.15.39:0/924428875 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f41280a10c0 0x7f41280a3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:36:13.616 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:36:13.578+0000 7f413effd700 1 --2- 172.21.15.39:0/924428875 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f41280a10c0 0x7f41280a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:36:13.616 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:36:13.578+0000 7f4127fff700 1 -- 172.21.15.39:0/924428875 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 2851+0+0 (secure 0 0 0) 0x7f41380cb310 con 0x7f4140100610 2023-07-21T18:36:13.617 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:36:13.578+0000 7f413effd700 1 --2- 172.21.15.39:0/924428875 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f41280a10c0 0x7f41280a3570 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4130006010 tx=0x7f413000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:36:13.617 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:36:13.581+0000 7f4127fff700 1 -- 172.21.15.39:0/924428875 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f4138096090 con 0x7f4140100610 2023-07-21T18:36:13.825 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:36:13.789+0000 7f41453ac700 1 -- 172.21.15.39:0/924428875 --> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f41401a03b0 con 0x7f41280a10c0 2023-07-21T18:36:14.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:14 smithi193 ceph-mon[98204]: purged_snaps scrub starts 2023-07-21T18:36:14.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:14 smithi193 ceph-mon[98204]: purged_snaps scrub ok 2023-07-21T18:36:14.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:14 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T18:36:14.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:14 smithi193 ceph-mon[98204]: osd.2 [v2:172.21.15.193:6800/2563594449,v1:172.21.15.193:6801/2563594449] boot 2023-07-21T18:36:14.385 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:14 smithi193 ceph-mon[98204]: osdmap e18: 3 total, 3 up, 3 in 2023-07-21T18:36:14.385 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:14 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T18:36:14.385 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:14 smithi193 ceph-mon[98204]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T18:36:14.385 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:14 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-07-21T18:36:14.385 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:14 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T18:36:14.385 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:14 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T18:36:14.385 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:14 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:36:14.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:14 smithi155 ceph-mon[97934]: purged_snaps scrub starts 2023-07-21T18:36:14.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:14 smithi155 ceph-mon[97934]: purged_snaps scrub ok 2023-07-21T18:36:14.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:14 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T18:36:14.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:14 smithi155 ceph-mon[97934]: osd.2 [v2:172.21.15.193:6800/2563594449,v1:172.21.15.193:6801/2563594449] boot 2023-07-21T18:36:14.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:14 smithi155 ceph-mon[97934]: osdmap e18: 3 total, 3 up, 3 in 2023-07-21T18:36:14.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:14 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T18:36:14.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:14 smithi155 ceph-mon[97934]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T18:36:14.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:14 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-07-21T18:36:14.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:14 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T18:36:14.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:14 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T18:36:14.404 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:14 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:36:14.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:14 smithi039 ceph-mon[96489]: purged_snaps scrub starts 2023-07-21T18:36:14.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:14 smithi039 ceph-mon[96489]: purged_snaps scrub ok 2023-07-21T18:36:14.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:14 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T18:36:14.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:14 smithi039 ceph-mon[96489]: osd.2 [v2:172.21.15.193:6800/2563594449,v1:172.21.15.193:6801/2563594449] boot 2023-07-21T18:36:14.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:14 smithi039 ceph-mon[96489]: osdmap e18: 3 total, 3 up, 3 in 2023-07-21T18:36:14.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:14 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T18:36:14.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:14 smithi039 ceph-mon[96489]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T18:36:14.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:14 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-07-21T18:36:14.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:14 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T18:36:14.453 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:14 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T18:36:14.453 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:14 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:36:15.282 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:15 smithi039 ceph-mon[96489]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:36:15.283 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:15 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-07-21T18:36:15.283 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:15 smithi039 ceph-mon[96489]: osdmap e19: 3 total, 3 up, 3 in 2023-07-21T18:36:15.283 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:15 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-07-21T18:36:15.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:15 smithi193 ceph-mon[98204]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:36:15.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:15 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-07-21T18:36:15.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:15 smithi193 ceph-mon[98204]: osdmap e19: 3 total, 3 up, 3 in 2023-07-21T18:36:15.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:15 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-07-21T18:36:15.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:15 smithi155 ceph-mon[97934]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:36:15.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:15 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-07-21T18:36:15.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:15 smithi155 ceph-mon[97934]: osdmap e19: 3 total, 3 up, 3 in 2023-07-21T18:36:15.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:15 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-07-21T18:36:15.951 INFO:journalctl@ceph.osd.0.smithi039.stdout:Jul 21 18:36:15 smithi039 sudo[112556]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-21T18:36:15.951 INFO:journalctl@ceph.osd.0.smithi039.stdout:Jul 21 18:36:15 smithi039 sudo[112556]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-21T18:36:15.952 INFO:journalctl@ceph.osd.0.smithi039.stdout:Jul 21 18:36:15 smithi039 sudo[112556]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-21T18:36:15.952 INFO:journalctl@ceph.osd.0.smithi039.stdout:Jul 21 18:36:15 smithi039 sudo[112556]: pam_unix(sudo:session): session closed for user root 2023-07-21T18:36:16.076 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:36:16.040+0000 7f4127fff700 1 -- 172.21.15.39:0/924428875 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f4138096460 con 0x7f4140100610 2023-07-21T18:36:16.348 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:16 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-07-21T18:36:16.348 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:16 smithi155 ceph-mon[97934]: osdmap e20: 3 total, 3 up, 3 in 2023-07-21T18:36:16.348 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:16 smithi155 ceph-mon[97934]: pgmap v74: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-07-21T18:36:16.348 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:16 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/1166571045' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "54562558-a2e8-45dd-bc95-561569b9ae6b"}]: dispatch 2023-07-21T18:36:16.348 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:16 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/1166571045' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "54562558-a2e8-45dd-bc95-561569b9ae6b"}]': finished 2023-07-21T18:36:16.349 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:16 smithi155 ceph-mon[97934]: osdmap e21: 4 total, 3 up, 4 in 2023-07-21T18:36:16.349 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:16 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T18:36:16.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:16 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-07-21T18:36:16.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:16 smithi193 ceph-mon[98204]: osdmap e20: 3 total, 3 up, 3 in 2023-07-21T18:36:16.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:16 smithi193 ceph-mon[98204]: pgmap v74: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-07-21T18:36:16.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:16 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/1166571045' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "54562558-a2e8-45dd-bc95-561569b9ae6b"}]: dispatch 2023-07-21T18:36:16.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:16 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/1166571045' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "54562558-a2e8-45dd-bc95-561569b9ae6b"}]': finished 2023-07-21T18:36:16.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:16 smithi193 ceph-mon[98204]: osdmap e21: 4 total, 3 up, 4 in 2023-07-21T18:36:16.385 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:16 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T18:36:16.451 INFO:journalctl@ceph.osd.0.smithi039.stdout:Jul 21 18:36:16 smithi039 sudo[112593]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-21T18:36:16.451 INFO:journalctl@ceph.osd.0.smithi039.stdout:Jul 21 18:36:16 smithi039 sudo[112593]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-21T18:36:16.452 INFO:journalctl@ceph.osd.0.smithi039.stdout:Jul 21 18:36:16 smithi039 sudo[112593]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-21T18:36:16.452 INFO:journalctl@ceph.osd.0.smithi039.stdout:Jul 21 18:36:16 smithi039 sudo[112593]: pam_unix(sudo:session): session closed for user root 2023-07-21T18:36:16.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:16 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-07-21T18:36:16.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:16 smithi039 ceph-mon[96489]: osdmap e20: 3 total, 3 up, 3 in 2023-07-21T18:36:16.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:16 smithi039 ceph-mon[96489]: pgmap v74: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-07-21T18:36:16.453 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:16 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/1166571045' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "54562558-a2e8-45dd-bc95-561569b9ae6b"}]: dispatch 2023-07-21T18:36:16.453 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:16 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/1166571045' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "54562558-a2e8-45dd-bc95-561569b9ae6b"}]': finished 2023-07-21T18:36:16.453 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:16 smithi039 ceph-mon[96489]: osdmap e21: 4 total, 3 up, 4 in 2023-07-21T18:36:16.453 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:16 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T18:36:16.603 INFO:journalctl@ceph.osd.1.smithi155.stdout:Jul 21 18:36:16 smithi155 sudo[105234]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-21T18:36:16.603 INFO:journalctl@ceph.osd.1.smithi155.stdout:Jul 21 18:36:16 smithi155 sudo[105234]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-21T18:36:16.603 INFO:journalctl@ceph.osd.1.smithi155.stdout:Jul 21 18:36:16 smithi155 sudo[105234]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-21T18:36:16.603 INFO:journalctl@ceph.osd.1.smithi155.stdout:Jul 21 18:36:16 smithi155 sudo[105234]: pam_unix(sudo:session): session closed for user root 2023-07-21T18:36:16.902 INFO:journalctl@ceph.osd.1.smithi155.stdout:Jul 21 18:36:16 smithi155 sudo[105237]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-21T18:36:16.902 INFO:journalctl@ceph.osd.1.smithi155.stdout:Jul 21 18:36:16 smithi155 sudo[105237]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-21T18:36:16.902 INFO:journalctl@ceph.osd.1.smithi155.stdout:Jul 21 18:36:16 smithi155 sudo[105237]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-21T18:36:16.902 INFO:journalctl@ceph.osd.1.smithi155.stdout:Jul 21 18:36:16 smithi155 sudo[105237]: pam_unix(sudo:session): session closed for user root 2023-07-21T18:36:17.126 INFO:journalctl@ceph.osd.2.smithi193.stdout:Jul 21 18:36:16 smithi193 sudo[106341]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-21T18:36:17.127 INFO:journalctl@ceph.osd.2.smithi193.stdout:Jul 21 18:36:16 smithi193 sudo[106341]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-21T18:36:17.127 INFO:journalctl@ceph.osd.2.smithi193.stdout:Jul 21 18:36:16 smithi193 sudo[106341]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-21T18:36:17.127 INFO:journalctl@ceph.osd.2.smithi193.stdout:Jul 21 18:36:16 smithi193 sudo[106341]: pam_unix(sudo:session): session closed for user root 2023-07-21T18:36:17.383 INFO:journalctl@ceph.osd.2.smithi193.stdout:Jul 21 18:36:17 smithi193 sudo[106344]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-21T18:36:17.383 INFO:journalctl@ceph.osd.2.smithi193.stdout:Jul 21 18:36:17 smithi193 sudo[106344]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-21T18:36:17.384 INFO:journalctl@ceph.osd.2.smithi193.stdout:Jul 21 18:36:17 smithi193 sudo[106344]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-21T18:36:17.384 INFO:journalctl@ceph.osd.2.smithi193.stdout:Jul 21 18:36:17 smithi193 sudo[106344]: pam_unix(sudo:session): session closed for user root 2023-07-21T18:36:17.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:17 smithi193 ceph-mon[98204]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-07-21T18:36:17.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:17 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:17.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:17 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:17.385 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:17 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/1153002028' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T18:36:17.653 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:17 smithi155 ceph-mon[97934]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-07-21T18:36:17.653 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:17 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:17.653 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:17 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:17.653 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:17 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/1153002028' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T18:36:17.701 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:17 smithi039 ceph-mon[96489]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-07-21T18:36:17.701 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:17 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:17.701 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:17 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:17.702 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:17 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/1153002028' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T18:36:18.951 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:18 smithi039 sudo[112682]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-21T18:36:18.951 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:18 smithi039 sudo[112682]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-21T18:36:18.951 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:18 smithi039 sudo[112682]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-21T18:36:18.952 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:18 smithi039 sudo[112682]: pam_unix(sudo:session): session closed for user root 2023-07-21T18:36:19.239 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:18 smithi193 ceph-mon[98204]: pgmap v76: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-07-21T18:36:19.239 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:18 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:19.239 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:18 smithi193 ceph-mon[98204]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-21T18:36:19.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:19 smithi193 sudo[106347]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-21T18:36:19.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:19 smithi193 sudo[106347]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-21T18:36:19.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:19 smithi193 sudo[106347]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-21T18:36:19.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:19 smithi193 sudo[106347]: pam_unix(sudo:session): session closed for user root 2023-07-21T18:36:19.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:18 smithi155 ceph-mon[97934]: pgmap v76: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-07-21T18:36:19.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:18 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:19.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:18 smithi155 ceph-mon[97934]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-21T18:36:19.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:18 smithi039 sudo[113056]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-21T18:36:19.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:18 smithi039 sudo[113056]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-21T18:36:19.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:18 smithi039 sudo[113056]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-21T18:36:19.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:18 smithi039 sudo[113056]: pam_unix(sudo:session): session closed for user root 2023-07-21T18:36:19.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:18 smithi039 ceph-mon[96489]: pgmap v76: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-07-21T18:36:19.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:18 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:19.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:18 smithi039 ceph-mon[96489]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-21T18:36:19.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:19 smithi193 sudo[106350]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-21T18:36:19.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:19 smithi193 sudo[106350]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-21T18:36:19.634 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:19 smithi193 sudo[106350]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-21T18:36:19.634 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:19 smithi193 sudo[106350]: pam_unix(sudo:session): session closed for user root 2023-07-21T18:36:19.972 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:19 smithi155 sudo[105240]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-21T18:36:19.972 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:19 smithi155 sudo[105240]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-21T18:36:19.973 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:19 smithi155 sudo[105240]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-21T18:36:19.973 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:19 smithi155 sudo[105240]: pam_unix(sudo:session): session closed for user root 2023-07-21T18:36:20.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:19 smithi193 ceph-mon[98204]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-21T18:36:20.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:19 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T18:36:20.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:19 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T18:36:20.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:19 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T18:36:20.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:19 smithi193 ceph-mon[98204]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-21T18:36:20.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:19 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T18:36:20.385 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:19 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T18:36:20.385 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:19 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T18:36:20.385 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:19 smithi193 ceph-mon[98204]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2023-07-21T18:36:20.385 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:19 smithi193 ceph-mon[98204]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-21T18:36:20.385 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:19 smithi193 ceph-mon[98204]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-21T18:36:20.385 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:19 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T18:36:20.385 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:19 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T18:36:20.386 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:19 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T18:36:20.386 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:19 smithi193 ceph-mon[98204]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-21T18:36:20.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:19 smithi155 sudo[105250]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-21T18:36:20.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:19 smithi155 sudo[105250]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-21T18:36:20.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:19 smithi155 sudo[105250]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-21T18:36:20.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:19 smithi155 sudo[105250]: pam_unix(sudo:session): session closed for user root 2023-07-21T18:36:20.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:19 smithi155 ceph-mon[97934]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-21T18:36:20.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:19 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T18:36:20.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:19 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T18:36:20.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:19 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T18:36:20.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:19 smithi155 ceph-mon[97934]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-21T18:36:20.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:19 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T18:36:20.404 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:19 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T18:36:20.404 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:19 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T18:36:20.404 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:19 smithi155 ceph-mon[97934]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2023-07-21T18:36:20.404 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:19 smithi155 ceph-mon[97934]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-21T18:36:20.404 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:19 smithi155 ceph-mon[97934]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-21T18:36:20.404 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:19 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T18:36:20.404 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:19 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T18:36:20.405 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:19 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T18:36:20.405 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:19 smithi155 ceph-mon[97934]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-21T18:36:20.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:19 smithi039 ceph-mon[96489]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-21T18:36:20.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:19 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T18:36:20.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:19 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T18:36:20.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:19 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T18:36:20.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:19 smithi039 ceph-mon[96489]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-21T18:36:20.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:19 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T18:36:20.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:19 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T18:36:20.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:19 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T18:36:20.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:19 smithi039 ceph-mon[96489]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2023-07-21T18:36:20.453 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:19 smithi039 ceph-mon[96489]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-21T18:36:20.453 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:19 smithi039 ceph-mon[96489]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-21T18:36:20.453 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:19 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T18:36:20.453 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:19 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T18:36:20.453 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:19 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T18:36:20.453 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:19 smithi039 ceph-mon[96489]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-21T18:36:22.173 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:21 smithi039 ceph-mon[96489]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2023-07-21T18:36:22.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:21 smithi193 ceph-mon[98204]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2023-07-21T18:36:22.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:21 smithi155 ceph-mon[97934]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2023-07-21T18:36:23.361 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:23 smithi039 ceph-mon[96489]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2023-07-21T18:36:23.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:23 smithi193 ceph-mon[98204]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2023-07-21T18:36:23.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:23 smithi155 ceph-mon[97934]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2023-07-21T18:36:24.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:24 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-07-21T18:36:24.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:24 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:36:24.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:24 smithi039 ceph-mon[96489]: Deploying daemon osd.3 on smithi039 2023-07-21T18:36:24.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:24 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-07-21T18:36:24.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:24 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:36:24.634 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:24 smithi193 ceph-mon[98204]: Deploying daemon osd.3 on smithi039 2023-07-21T18:36:24.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:24 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-07-21T18:36:24.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:24 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:36:24.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:24 smithi155 ceph-mon[97934]: Deploying daemon osd.3 on smithi039 2023-07-21T18:36:25.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:25 smithi039 ceph-mon[96489]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2023-07-21T18:36:25.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:25 smithi193 ceph-mon[98204]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2023-07-21T18:36:25.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:25 smithi155 ceph-mon[97934]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2023-07-21T18:36:27.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:27 smithi039 ceph-mon[96489]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2023-07-21T18:36:27.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:27 smithi193 ceph-mon[98204]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2023-07-21T18:36:27.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:27 smithi155 ceph-mon[97934]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2023-07-21T18:36:29.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:29 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:29.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:29 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:29.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:29 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:36:29.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:29 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:29.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:29 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:29.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:29 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:36:29.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:29 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:29.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:29 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:29.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:29 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:36:30.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:30 smithi193 ceph-mon[98204]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2023-07-21T18:36:30.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:30 smithi155 ceph-mon[97934]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2023-07-21T18:36:30.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:30 smithi039 ceph-mon[96489]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2023-07-21T18:36:30.884 INFO:teuthology.orchestra.run.smithi039.stdout:Created osd(s) 3 on host 'smithi039' 2023-07-21T18:36:30.884 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:36:30.847+0000 7f4127fff700 1 -- 172.21.15.39:0/924428875 <== mgr.14152 v2:172.21.15.39:6800/2116291712 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f41401a03b0 con 0x7f41280a10c0 2023-07-21T18:36:30.885 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:36:30.849+0000 7f41453ac700 1 -- 172.21.15.39:0/924428875 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f41280a10c0 msgr2=0x7f41280a3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:36:30.886 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:36:30.849+0000 7f41453ac700 1 --2- 172.21.15.39:0/924428875 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f41280a10c0 0x7f41280a3570 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4130006010 tx=0x7f413000b260 comp rx=0 tx=0).stop 2023-07-21T18:36:30.886 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:36:30.849+0000 7f41453ac700 1 -- 172.21.15.39:0/924428875 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4140100610 msgr2=0x7f4140107ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:36:30.886 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:36:30.849+0000 7f41453ac700 1 --2- 172.21.15.39:0/924428875 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4140100610 0x7f4140107ec0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f413800c370 tx=0x7f413800c680 comp rx=0 tx=0).stop 2023-07-21T18:36:30.886 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:36:30.849+0000 7f41453ac700 1 -- 172.21.15.39:0/924428875 shutdown_connections 2023-07-21T18:36:30.886 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:36:30.849+0000 7f41453ac700 1 --2- 172.21.15.39:0/924428875 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f41280a10c0 0x7f41280a3570 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:36:30.886 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:36:30.849+0000 7f41453ac700 1 --2- 172.21.15.39:0/924428875 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4140100610 0x7f4140107ec0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:36:30.886 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:36:30.849+0000 7f41453ac700 1 --2- 172.21.15.39:0/924428875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41400fea80 0x7f4140109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:36:30.887 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:36:30.849+0000 7f41453ac700 1 --2- 172.21.15.39:0/924428875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41400ffc80 0x7f4140107980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:36:30.887 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:36:30.849+0000 7f41453ac700 1 -- 172.21.15.39:0/924428875 >> 172.21.15.39:0/924428875 conn(0x7f41400fa0b0 msgr2=0x7f41400fc360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:36:30.887 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:36:30.849+0000 7f41453ac700 1 -- 172.21.15.39:0/924428875 shutdown_connections 2023-07-21T18:36:30.887 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:36:30.849+0000 7f41453ac700 1 -- 172.21.15.39:0/924428875 wait complete. 2023-07-21T18:36:31.220 DEBUG:teuthology.orchestra.run.smithi039:osd.3> sudo journalctl -f -n 0 -u ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@osd.3.service 2023-07-21T18:36:31.223 INFO:tasks.cephadm:Deploying osd.4 on smithi155 with /dev/vg_nvme/lv_3... 2023-07-21T18:36:31.223 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-07-21T18:36:31.252 INFO:journalctl@ceph.osd.3.smithi039.stdout:-- Logs begin at Fri 2023-07-21 18:18:00 UTC. -- 2023-07-21T18:36:31.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:31 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:31.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:31 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:31.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:31 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:36:31.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:31 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:36:31.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:31 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:31.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:31 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:31.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:31 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:31.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:31 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:36:31.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:31 smithi039 ceph-mon[96489]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-07-21T18:36:31.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:31 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:31.634 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:31 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:31.634 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:31 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:36:31.634 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:31 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:36:31.634 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:31 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:31.634 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:31 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:31.634 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:31 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:31.635 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:31 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:36:31.635 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:31 smithi193 ceph-mon[98204]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-07-21T18:36:31.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:31 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:31.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:31 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:31.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:31 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:36:31.653 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:31 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:36:31.653 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:31 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:31.653 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:31 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:31.653 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:31 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:31.653 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:31 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:36:31.653 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:31 smithi155 ceph-mon[97934]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-07-21T18:36:31.797 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.b/config 2023-07-21T18:36:32.115 INFO:journalctl@ceph.osd.3.smithi039.stdout:Jul 21 18:36:31 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-3[114860]: 2023-07-21T18:36:31.747+0000 7f79e7973640 -1 osd.3 0 log_to_monitors true 2023-07-21T18:36:32.341 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:32 smithi155 ceph-mon[97934]: from='osd.3 [v2:172.21.15.39:6810/863708386,v1:172.21.15.39:6811/863708386]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-07-21T18:36:32.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:32 smithi039 ceph-mon[96489]: from='osd.3 [v2:172.21.15.39:6810/863708386,v1:172.21.15.39:6811/863708386]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-07-21T18:36:32.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:32 smithi193 ceph-mon[98204]: from='osd.3 [v2:172.21.15.39:6810/863708386,v1:172.21.15.39:6811/863708386]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-07-21T18:36:33.401 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-07-21T18:36:33.402 INFO:teuthology.orchestra.run.smithi155.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-07-21T18:36:33.402 INFO:teuthology.orchestra.run.smithi155.stderr: stderr: 10+0 records in 2023-07-21T18:36:33.402 INFO:teuthology.orchestra.run.smithi155.stderr:10+0 records out 2023-07-21T18:36:33.402 INFO:teuthology.orchestra.run.smithi155.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.012613 s, 831 MB/s 2023-07-21T18:36:33.402 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping successful for: 2023-07-21T18:36:33.447 INFO:journalctl@ceph.osd.3.smithi039.stdout:Jul 21 18:36:33 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-3[114860]: 2023-07-21T18:36:33.181+0000 7f79dc7ea700 -1 osd.3 0 waiting for initial osdmap 2023-07-21T18:36:33.448 INFO:journalctl@ceph.osd.3.smithi039.stdout:Jul 21 18:36:33 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-3[114860]: 2023-07-21T18:36:33.190+0000 7f79d85df700 -1 osd.3 23 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-21T18:36:33.448 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:33 smithi039 ceph-mon[96489]: from='osd.3 [v2:172.21.15.39:6810/863708386,v1:172.21.15.39:6811/863708386]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-07-21T18:36:33.448 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:33 smithi039 ceph-mon[96489]: osdmap e22: 4 total, 3 up, 4 in 2023-07-21T18:36:33.448 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:33 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T18:36:33.448 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:33 smithi039 ceph-mon[96489]: from='osd.3 [v2:172.21.15.39:6810/863708386,v1:172.21.15.39:6811/863708386]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2023-07-21T18:36:33.449 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:33 smithi039 ceph-mon[96489]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-07-21T18:36:33.533 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:33 smithi155 ceph-mon[97934]: from='osd.3 [v2:172.21.15.39:6810/863708386,v1:172.21.15.39:6811/863708386]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-07-21T18:36:33.533 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:33 smithi155 ceph-mon[97934]: osdmap e22: 4 total, 3 up, 4 in 2023-07-21T18:36:33.533 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:33 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T18:36:33.533 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:33 smithi155 ceph-mon[97934]: from='osd.3 [v2:172.21.15.39:6810/863708386,v1:172.21.15.39:6811/863708386]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2023-07-21T18:36:33.534 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:33 smithi155 ceph-mon[97934]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-07-21T18:36:33.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:33 smithi193 ceph-mon[98204]: from='osd.3 [v2:172.21.15.39:6810/863708386,v1:172.21.15.39:6811/863708386]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-07-21T18:36:33.634 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:33 smithi193 ceph-mon[98204]: osdmap e22: 4 total, 3 up, 4 in 2023-07-21T18:36:33.634 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:33 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T18:36:33.634 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:33 smithi193 ceph-mon[98204]: from='osd.3 [v2:172.21.15.39:6810/863708386,v1:172.21.15.39:6811/863708386]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2023-07-21T18:36:33.634 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:33 smithi193 ceph-mon[98204]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-07-21T18:36:33.749 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi155:vg_nvme/lv_3 2023-07-21T18:36:34.301 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.b/config 2023-07-21T18:36:34.506 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:34 smithi155 ceph-mon[97934]: from='osd.3 [v2:172.21.15.39:6810/863708386,v1:172.21.15.39:6811/863708386]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2023-07-21T18:36:34.506 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:34 smithi155 ceph-mon[97934]: osdmap e23: 4 total, 3 up, 4 in 2023-07-21T18:36:34.506 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:34 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T18:36:34.506 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:34 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T18:36:34.506 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:34 smithi155 ceph-mon[97934]: Detected new or changed devices on smithi039 2023-07-21T18:36:34.507 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:34 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:34.507 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:34 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:34.507 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:34 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:36:34.507 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:34 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:36:34.507 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:34 smithi155 ceph-mon[97934]: Adjusting osd_memory_target on smithi039 to 8584M 2023-07-21T18:36:34.507 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:34 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:34.507 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:34 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:36:34.508 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:34 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:36:34.508 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:34 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:34.508 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:34 smithi155 ceph-mon[97934]: osd.3 [v2:172.21.15.39:6810/863708386,v1:172.21.15.39:6811/863708386] boot 2023-07-21T18:36:34.508 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:34 smithi155 ceph-mon[97934]: osdmap e24: 4 total, 4 up, 4 in 2023-07-21T18:36:34.508 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:34 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T18:36:34.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:34 smithi193 ceph-mon[98204]: from='osd.3 [v2:172.21.15.39:6810/863708386,v1:172.21.15.39:6811/863708386]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2023-07-21T18:36:34.634 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:34 smithi193 ceph-mon[98204]: osdmap e23: 4 total, 3 up, 4 in 2023-07-21T18:36:34.634 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:34 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T18:36:34.634 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:34 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T18:36:34.634 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:34 smithi193 ceph-mon[98204]: Detected new or changed devices on smithi039 2023-07-21T18:36:34.634 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:34 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:34.634 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:34 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:34.635 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:34 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:36:34.635 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:34 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:36:34.635 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:34 smithi193 ceph-mon[98204]: Adjusting osd_memory_target on smithi039 to 8584M 2023-07-21T18:36:34.635 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:34 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:34.635 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:34 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:36:34.635 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:34 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:36:34.635 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:34 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:34.636 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:34 smithi193 ceph-mon[98204]: osd.3 [v2:172.21.15.39:6810/863708386,v1:172.21.15.39:6811/863708386] boot 2023-07-21T18:36:34.636 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:34 smithi193 ceph-mon[98204]: osdmap e24: 4 total, 4 up, 4 in 2023-07-21T18:36:34.636 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:34 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T18:36:34.701 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:34 smithi039 ceph-mon[96489]: from='osd.3 [v2:172.21.15.39:6810/863708386,v1:172.21.15.39:6811/863708386]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2023-07-21T18:36:34.701 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:34 smithi039 ceph-mon[96489]: osdmap e23: 4 total, 3 up, 4 in 2023-07-21T18:36:34.702 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:34 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T18:36:34.702 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:34 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T18:36:34.702 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:34 smithi039 ceph-mon[96489]: Detected new or changed devices on smithi039 2023-07-21T18:36:34.702 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:34 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:34.702 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:34 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:34.702 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:34 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:36:34.702 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:34 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:36:34.703 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:34 smithi039 ceph-mon[96489]: Adjusting osd_memory_target on smithi039 to 8584M 2023-07-21T18:36:34.703 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:34 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:34.703 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:34 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:36:34.703 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:34 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:36:34.703 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:34 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:34.703 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:34 smithi039 ceph-mon[96489]: osd.3 [v2:172.21.15.39:6810/863708386,v1:172.21.15.39:6811/863708386] boot 2023-07-21T18:36:34.703 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:34 smithi039 ceph-mon[96489]: osdmap e24: 4 total, 4 up, 4 in 2023-07-21T18:36:34.703 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:34 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T18:36:35.504 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:36:35.468+0000 7f1895534700 1 -- 172.21.15.155:0/412343438 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18900ffc10 msgr2=0x7f1890100020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:36:35.504 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:36:35.468+0000 7f1895534700 1 --2- 172.21.15.155:0/412343438 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18900ffc10 0x7f1890100020 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1880009990 tx=0x7f1880009ca0 comp rx=0 tx=0).stop 2023-07-21T18:36:35.504 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:36:35.468+0000 7f1895534700 1 -- 172.21.15.155:0/412343438 shutdown_connections 2023-07-21T18:36:35.504 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:36:35.468+0000 7f1895534700 1 --2- 172.21.15.155:0/412343438 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18900ffc10 0x7f1890100020 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:36:35.505 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:36:35.468+0000 7f1895534700 1 --2- 172.21.15.155:0/412343438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f18901017a0 0x7f1890108060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:36:35.505 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:36:35.468+0000 7f1895534700 1 --2- 172.21.15.155:0/412343438 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1890100e10 0x7f1890101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:36:35.505 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:36:35.468+0000 7f1895534700 1 -- 172.21.15.155:0/412343438 >> 172.21.15.155:0/412343438 conn(0x7f18900fb220 msgr2=0x7f18900fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:36:35.505 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:36:35.469+0000 7f1895534700 1 -- 172.21.15.155:0/412343438 shutdown_connections 2023-07-21T18:36:35.505 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:36:35.469+0000 7f1895534700 1 -- 172.21.15.155:0/412343438 wait complete. 2023-07-21T18:36:35.507 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:36:35.469+0000 7f1895534700 1 Processor -- start 2023-07-21T18:36:35.507 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:36:35.469+0000 7f1895534700 1 -- start start 2023-07-21T18:36:35.507 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:36:35.469+0000 7f1895534700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18900ffc10 0x7f18901958c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:36:35.507 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:36:35.469+0000 7f1895534700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1890100e10 0x7f1890195e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:36:35.507 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:36:35.469+0000 7f1895534700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f18901017a0 0x7f189019ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:36:35.507 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:36:35.469+0000 7f1895534700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189019d450 con 0x7f18900ffc10 2023-07-21T18:36:35.508 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:36:35.469+0000 7f1895534700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189019d590 con 0x7f1890100e10 2023-07-21T18:36:35.508 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:36:35.469+0000 7f1895534700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189019d720 con 0x7f18901017a0 2023-07-21T18:36:35.508 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:36:35.470+0000 7f188e7fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1890100e10 0x7f1890195e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:36:35.508 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:36:35.470+0000 7f188e7fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1890100e10 0x7f1890195e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:57262/0 (socket says 172.21.15.155:57262) 2023-07-21T18:36:35.508 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:36:35.470+0000 7f188e7fc700 1 -- 172.21.15.155:0/3846928716 learned_addr learned my addr 172.21.15.155:0/3846928716 (peer_addr_for_me v2:172.21.15.155:0/0) 2023-07-21T18:36:35.508 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:36:35.470+0000 7f188f7fe700 1 --2- 172.21.15.155:0/3846928716 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f18901017a0 0x7f189019ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:36:35.508 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:36:35.470+0000 7f188effd700 1 --2- 172.21.15.155:0/3846928716 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18900ffc10 0x7f18901958c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:36:35.509 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:36:35.470+0000 7f188e7fc700 1 -- 172.21.15.155:0/3846928716 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f18901017a0 msgr2=0x7f189019ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:36:35.509 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:36:35.470+0000 7f188e7fc700 1 --2- 172.21.15.155:0/3846928716 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f18901017a0 0x7f189019ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:36:35.509 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:36:35.470+0000 7f188e7fc700 1 -- 172.21.15.155:0/3846928716 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18900ffc10 msgr2=0x7f18901958c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:36:35.509 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:36:35.470+0000 7f188e7fc700 1 --2- 172.21.15.155:0/3846928716 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18900ffc10 0x7f18901958c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:36:35.509 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:36:35.470+0000 7f188e7fc700 1 -- 172.21.15.155:0/3846928716 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18800095f0 con 0x7f1890100e10 2023-07-21T18:36:35.509 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:36:35.470+0000 7f188e7fc700 1 --2- 172.21.15.155:0/3846928716 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1890100e10 0x7f1890195e00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f188400d950 tx=0x7f188400dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:36:35.509 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:36:35.470+0000 7f1877fff700 1 -- 172.21.15.155:0/3846928716 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1884009940 con 0x7f1890100e10 2023-07-21T18:36:35.509 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:36:35.470+0000 7f1877fff700 1 -- 172.21.15.155:0/3846928716 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1884010460 con 0x7f1890100e10 2023-07-21T18:36:35.510 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:36:35.470+0000 7f1895534700 1 -- 172.21.15.155:0/3846928716 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f189019da00 con 0x7f1890100e10 2023-07-21T18:36:35.510 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:36:35.470+0000 7f1877fff700 1 -- 172.21.15.155:0/3846928716 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188400f600 con 0x7f1890100e10 2023-07-21T18:36:35.510 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:36:35.470+0000 7f1895534700 1 -- 172.21.15.155:0/3846928716 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f189019df00 con 0x7f1890100e10 2023-07-21T18:36:35.510 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:36:35.471+0000 7f1895534700 1 -- 172.21.15.155:0/3846928716 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f189010b3d0 con 0x7f1890100e10 2023-07-21T18:36:35.511 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:36:35.473+0000 7f1877fff700 1 -- 172.21.15.155:0/3846928716 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f18840105d0 con 0x7f1890100e10 2023-07-21T18:36:35.511 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:36:35.473+0000 7f1877fff700 1 --2- 172.21.15.155:0/3846928716 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f18780a1150 0x7f18780a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:36:35.511 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:36:35.473+0000 7f1877fff700 1 -- 172.21.15.155:0/3846928716 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 3698+0+0 (secure 0 0 0) 0x7f18840cb610 con 0x7f1890100e10 2023-07-21T18:36:35.511 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:36:35.474+0000 7f188effd700 1 --2- 172.21.15.155:0/3846928716 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f18780a1150 0x7f18780a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:36:35.511 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:36:35.474+0000 7f188effd700 1 --2- 172.21.15.155:0/3846928716 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f18780a1150 0x7f18780a3600 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1880006010 tx=0x7f188000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:36:35.512 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:36:35.476+0000 7f1877fff700 1 -- 172.21.15.155:0/3846928716 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f1884095ec0 con 0x7f1890100e10 2023-07-21T18:36:35.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:35 smithi193 ceph-mon[98204]: purged_snaps scrub starts 2023-07-21T18:36:35.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:35 smithi193 ceph-mon[98204]: purged_snaps scrub ok 2023-07-21T18:36:35.634 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:35 smithi193 ceph-mon[98204]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-21T18:36:35.634 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:35 smithi193 ceph-mon[98204]: osdmap e25: 4 total, 4 up, 4 in 2023-07-21T18:36:35.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:35 smithi155 ceph-mon[97934]: purged_snaps scrub starts 2023-07-21T18:36:35.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:35 smithi155 ceph-mon[97934]: purged_snaps scrub ok 2023-07-21T18:36:35.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:35 smithi155 ceph-mon[97934]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-21T18:36:35.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:35 smithi155 ceph-mon[97934]: osdmap e25: 4 total, 4 up, 4 in 2023-07-21T18:36:35.701 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:35 smithi039 ceph-mon[96489]: purged_snaps scrub starts 2023-07-21T18:36:35.701 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:35 smithi039 ceph-mon[96489]: purged_snaps scrub ok 2023-07-21T18:36:35.701 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:35 smithi039 ceph-mon[96489]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-21T18:36:35.701 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:35 smithi039 ceph-mon[96489]: osdmap e25: 4 total, 4 up, 4 in 2023-07-21T18:36:35.705 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:36:35.669+0000 7f1895534700 1 -- 172.21.15.155:0/3846928716 --> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f1890060950 con 0x7f18780a1150 2023-07-21T18:36:36.513 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:36 smithi155 ceph-mon[97934]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:36:36.514 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:36 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T18:36:36.514 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:36 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T18:36:36.514 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:36 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:36:36.514 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:36 smithi155 ceph-mon[97934]: osdmap e26: 4 total, 4 up, 4 in 2023-07-21T18:36:36.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:36 smithi193 ceph-mon[98204]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:36:36.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:36 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T18:36:36.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:36 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T18:36:36.634 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:36 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:36:36.634 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:36 smithi193 ceph-mon[98204]: osdmap e26: 4 total, 4 up, 4 in 2023-07-21T18:36:36.701 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:36 smithi039 ceph-mon[96489]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:36:36.701 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:36 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T18:36:36.702 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:36 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T18:36:36.702 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:36 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:36:36.702 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:36 smithi039 ceph-mon[96489]: osdmap e26: 4 total, 4 up, 4 in 2023-07-21T18:36:37.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:37 smithi193 ceph-mon[98204]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-21T18:36:37.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:37 smithi155 ceph-mon[97934]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-21T18:36:37.701 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:37 smithi039 ceph-mon[96489]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-21T18:36:38.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:38 smithi193 ceph-mon[98204]: osdmap e27: 4 total, 4 up, 4 in 2023-07-21T18:36:38.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:38 smithi193 ceph-mon[98204]: from='client.? 172.21.15.155:0/2941729645' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "491118e7-ae1e-495c-9eac-ef1cdcae9600"}]: dispatch 2023-07-21T18:36:38.634 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:38 smithi193 ceph-mon[98204]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "491118e7-ae1e-495c-9eac-ef1cdcae9600"}]: dispatch 2023-07-21T18:36:38.634 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:38 smithi193 ceph-mon[98204]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "491118e7-ae1e-495c-9eac-ef1cdcae9600"}]': finished 2023-07-21T18:36:38.635 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:38 smithi193 ceph-mon[98204]: osdmap e28: 5 total, 4 up, 5 in 2023-07-21T18:36:38.635 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:38 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T18:36:38.651 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:38 smithi155 ceph-mon[97934]: osdmap e27: 4 total, 4 up, 4 in 2023-07-21T18:36:38.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:38 smithi155 ceph-mon[97934]: from='client.? 172.21.15.155:0/2941729645' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "491118e7-ae1e-495c-9eac-ef1cdcae9600"}]: dispatch 2023-07-21T18:36:38.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:38 smithi155 ceph-mon[97934]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "491118e7-ae1e-495c-9eac-ef1cdcae9600"}]: dispatch 2023-07-21T18:36:38.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:38 smithi155 ceph-mon[97934]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "491118e7-ae1e-495c-9eac-ef1cdcae9600"}]': finished 2023-07-21T18:36:38.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:38 smithi155 ceph-mon[97934]: osdmap e28: 5 total, 4 up, 5 in 2023-07-21T18:36:38.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:38 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T18:36:38.701 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:38 smithi039 ceph-mon[96489]: osdmap e27: 4 total, 4 up, 4 in 2023-07-21T18:36:38.701 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:38 smithi039 ceph-mon[96489]: from='client.? 172.21.15.155:0/2941729645' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "491118e7-ae1e-495c-9eac-ef1cdcae9600"}]: dispatch 2023-07-21T18:36:38.702 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:38 smithi039 ceph-mon[96489]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "491118e7-ae1e-495c-9eac-ef1cdcae9600"}]: dispatch 2023-07-21T18:36:38.702 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:38 smithi039 ceph-mon[96489]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "491118e7-ae1e-495c-9eac-ef1cdcae9600"}]': finished 2023-07-21T18:36:38.702 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:38 smithi039 ceph-mon[96489]: osdmap e28: 5 total, 4 up, 5 in 2023-07-21T18:36:38.702 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:38 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T18:36:39.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:39 smithi193 ceph-mon[98204]: from='client.? 172.21.15.155:0/4168128323' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T18:36:39.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:39 smithi193 ceph-mon[98204]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-21T18:36:39.651 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:39 smithi155 ceph-mon[97934]: from='client.? 172.21.15.155:0/4168128323' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T18:36:39.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:39 smithi155 ceph-mon[97934]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-21T18:36:39.701 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:39 smithi039 ceph-mon[96489]: from='client.? 172.21.15.155:0/4168128323' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T18:36:39.701 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:39 smithi039 ceph-mon[96489]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-21T18:36:42.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:41 smithi039 ceph-mon[96489]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 98 KiB/s, 0 objects/s recovering 2023-07-21T18:36:42.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:41 smithi193 ceph-mon[98204]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 98 KiB/s, 0 objects/s recovering 2023-07-21T18:36:42.401 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:41 smithi155 ceph-mon[97934]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 98 KiB/s, 0 objects/s recovering 2023-07-21T18:36:43.450 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:43 smithi039 ceph-mon[96489]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 84 KiB/s, 0 objects/s recovering 2023-07-21T18:36:43.479 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:43 smithi155 ceph-mon[97934]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 84 KiB/s, 0 objects/s recovering 2023-07-21T18:36:43.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:43 smithi193 ceph-mon[98204]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 84 KiB/s, 0 objects/s recovering 2023-07-21T18:36:45.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:45 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-07-21T18:36:45.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:45 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:36:45.401 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:45 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-07-21T18:36:45.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:45 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:36:45.450 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:45 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-07-21T18:36:45.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:45 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:36:46.292 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:46 smithi155 ceph-mon[97934]: Deploying daemon osd.4 on smithi155 2023-07-21T18:36:46.292 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:46 smithi155 ceph-mon[97934]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-07-21T18:36:46.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:46 smithi193 ceph-mon[98204]: Deploying daemon osd.4 on smithi155 2023-07-21T18:36:46.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:46 smithi193 ceph-mon[98204]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-07-21T18:36:46.450 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:46 smithi039 ceph-mon[96489]: Deploying daemon osd.4 on smithi155 2023-07-21T18:36:46.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:46 smithi039 ceph-mon[96489]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-07-21T18:36:47.450 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:47 smithi039 ceph-mon[96489]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-07-21T18:36:47.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:47 smithi155 ceph-mon[97934]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-07-21T18:36:47.632 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:47 smithi193 ceph-mon[98204]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-07-21T18:36:49.453 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:49 smithi155 ceph-mon[97934]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2023-07-21T18:36:49.632 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:49 smithi193 ceph-mon[98204]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2023-07-21T18:36:49.700 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:49 smithi039 ceph-mon[96489]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2023-07-21T18:36:50.651 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:50 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:50.651 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:50 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:50.651 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:50 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:36:50.882 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:50 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:50.883 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:50 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:50.883 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:50 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:36:50.950 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:50 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:50.950 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:50 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:50.950 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:50 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:36:52.200 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:51 smithi039 ceph-mon[96489]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-21T18:36:52.200 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:51 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:52.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:51 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:52.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:51 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:36:52.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:51 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:36:52.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:51 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:52.248 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:51 smithi155 ceph-mon[97934]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-21T18:36:52.248 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:51 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:52.248 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:51 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:52.248 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:51 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:36:52.248 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:51 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:36:52.249 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:51 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:52.323 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:36:52.287+0000 7f1877fff700 1 -- 172.21.15.155:0/3846928716 <== mgr.14152 v2:172.21.15.39:6800/2116291712 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f1890060950 con 0x7f18780a1150 2023-07-21T18:36:52.323 INFO:teuthology.orchestra.run.smithi155.stdout:Created osd(s) 4 on host 'smithi155' 2023-07-21T18:36:52.325 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:36:52.289+0000 7f1895534700 1 -- 172.21.15.155:0/3846928716 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f18780a1150 msgr2=0x7f18780a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:36:52.326 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:36:52.289+0000 7f1895534700 1 --2- 172.21.15.155:0/3846928716 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f18780a1150 0x7f18780a3600 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1880006010 tx=0x7f188000b540 comp rx=0 tx=0).stop 2023-07-21T18:36:52.326 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:36:52.290+0000 7f1895534700 1 -- 172.21.15.155:0/3846928716 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1890100e10 msgr2=0x7f1890195e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:36:52.326 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:36:52.290+0000 7f1895534700 1 --2- 172.21.15.155:0/3846928716 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1890100e10 0x7f1890195e00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f188400d950 tx=0x7f188400dc60 comp rx=0 tx=0).stop 2023-07-21T18:36:52.327 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:36:52.290+0000 7f1895534700 1 -- 172.21.15.155:0/3846928716 shutdown_connections 2023-07-21T18:36:52.329 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:36:52.290+0000 7f1895534700 1 --2- 172.21.15.155:0/3846928716 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f18780a1150 0x7f18780a3600 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:36:52.329 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:36:52.290+0000 7f1895534700 1 --2- 172.21.15.155:0/3846928716 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f18901017a0 0x7f189019ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:36:52.329 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:36:52.290+0000 7f1895534700 1 --2- 172.21.15.155:0/3846928716 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18900ffc10 0x7f18901958c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:36:52.329 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:36:52.290+0000 7f1895534700 1 --2- 172.21.15.155:0/3846928716 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1890100e10 0x7f1890195e00 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:36:52.329 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:36:52.290+0000 7f1895534700 1 -- 172.21.15.155:0/3846928716 >> 172.21.15.155:0/3846928716 conn(0x7f18900fb220 msgr2=0x7f18900fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:36:52.330 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:36:52.290+0000 7f1895534700 1 -- 172.21.15.155:0/3846928716 shutdown_connections 2023-07-21T18:36:52.330 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:36:52.290+0000 7f1895534700 1 -- 172.21.15.155:0/3846928716 wait complete. 2023-07-21T18:36:52.382 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:51 smithi193 ceph-mon[98204]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-21T18:36:52.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:51 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:52.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:51 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:52.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:51 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:36:52.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:51 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:36:52.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:51 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:52.696 DEBUG:teuthology.orchestra.run.smithi155:osd.4> sudo journalctl -f -n 0 -u ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@osd.4.service 2023-07-21T18:36:52.699 INFO:tasks.cephadm:Deploying osd.5 on smithi193 with /dev/vg_nvme/lv_3... 2023-07-21T18:36:52.699 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-07-21T18:36:52.732 INFO:journalctl@ceph.osd.4.smithi155.stdout:-- Logs begin at Fri 2023-07-21 18:17:11 UTC. -- 2023-07-21T18:36:53.259 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T18:36:53.537 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:53 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:53.537 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:53 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:53.537 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:53 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:36:53.537 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:53 smithi155 ceph-mon[97934]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-21T18:36:53.537 INFO:journalctl@ceph.osd.4.smithi155.stdout:Jul 21 18:36:53 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-4[108462]: 2023-07-21T18:36:53.397+0000 7f62dbb4b640 -1 osd.4 0 log_to_monitors true 2023-07-21T18:36:53.632 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:53 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:53.632 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:53 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:53.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:53 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:36:53.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:53 smithi193 ceph-mon[98204]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-21T18:36:53.700 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:53 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:53.700 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:53 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:53.700 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:53 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:36:53.700 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:53 smithi039 ceph-mon[96489]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-21T18:36:54.560 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:54 smithi155 ceph-mon[97934]: from='osd.4 [v2:172.21.15.155:6808/3361780673,v1:172.21.15.155:6809/3361780673]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-21T18:36:54.561 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:54 smithi155 ceph-mon[97934]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-21T18:36:54.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:54 smithi193 ceph-mon[98204]: from='osd.4 [v2:172.21.15.155:6808/3361780673,v1:172.21.15.155:6809/3361780673]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-21T18:36:54.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:54 smithi193 ceph-mon[98204]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-21T18:36:54.700 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:54 smithi039 ceph-mon[96489]: from='osd.4 [v2:172.21.15.155:6808/3361780673,v1:172.21.15.155:6809/3361780673]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-21T18:36:54.700 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:54 smithi039 ceph-mon[96489]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-21T18:36:54.712 INFO:teuthology.orchestra.run.smithi193.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-07-21T18:36:54.713 INFO:teuthology.orchestra.run.smithi193.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-07-21T18:36:54.713 INFO:teuthology.orchestra.run.smithi193.stderr: stderr: 10+0 records in 2023-07-21T18:36:54.713 INFO:teuthology.orchestra.run.smithi193.stderr:10+0 records out 2023-07-21T18:36:54.713 INFO:teuthology.orchestra.run.smithi193.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0147993 s, 709 MB/s 2023-07-21T18:36:54.713 INFO:teuthology.orchestra.run.smithi193.stderr:--> Zapping successful for: 2023-07-21T18:36:55.098 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi193:vg_nvme/lv_3 2023-07-21T18:36:55.382 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:55 smithi193 ceph-mon[98204]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-07-21T18:36:55.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:55 smithi193 ceph-mon[98204]: osdmap e29: 5 total, 4 up, 5 in 2023-07-21T18:36:55.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:55 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T18:36:55.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:55 smithi193 ceph-mon[98204]: from='osd.4 [v2:172.21.15.155:6808/3361780673,v1:172.21.15.155:6809/3361780673]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2023-07-21T18:36:55.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:55 smithi193 ceph-mon[98204]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2023-07-21T18:36:55.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:55 smithi193 ceph-mon[98204]: Detected new or changed devices on smithi155 2023-07-21T18:36:55.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:55 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:55.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:55 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:55.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:55 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:36:55.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:55 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:36:55.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:55 smithi193 ceph-mon[98204]: Adjusting osd_memory_target on smithi155 to 8584M 2023-07-21T18:36:55.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:55 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:55.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:55 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:36:55.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:55 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:36:55.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:55 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:55.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:55 smithi193 ceph-mon[98204]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-21T18:36:55.650 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T18:36:55.652 INFO:journalctl@ceph.osd.4.smithi155.stdout:Jul 21 18:36:55 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-4[108462]: 2023-07-21T18:36:55.272+0000 7f62d09c2700 -1 osd.4 0 waiting for initial osdmap 2023-07-21T18:36:55.652 INFO:journalctl@ceph.osd.4.smithi155.stdout:Jul 21 18:36:55 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-4[108462]: 2023-07-21T18:36:55.277+0000 7f62ca7b3700 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-21T18:36:55.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:55 smithi155 ceph-mon[97934]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-07-21T18:36:55.653 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:55 smithi155 ceph-mon[97934]: osdmap e29: 5 total, 4 up, 5 in 2023-07-21T18:36:55.653 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:55 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T18:36:55.653 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:55 smithi155 ceph-mon[97934]: from='osd.4 [v2:172.21.15.155:6808/3361780673,v1:172.21.15.155:6809/3361780673]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2023-07-21T18:36:55.654 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:55 smithi155 ceph-mon[97934]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2023-07-21T18:36:55.654 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:55 smithi155 ceph-mon[97934]: Detected new or changed devices on smithi155 2023-07-21T18:36:55.654 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:55 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:55.654 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:55 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:55.655 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:55 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:36:55.655 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:55 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:36:55.655 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:55 smithi155 ceph-mon[97934]: Adjusting osd_memory_target on smithi155 to 8584M 2023-07-21T18:36:55.655 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:55 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:55.656 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:55 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:36:55.656 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:55 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:36:55.656 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:55 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:55.657 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:55 smithi155 ceph-mon[97934]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-21T18:36:55.700 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:55 smithi039 ceph-mon[96489]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-07-21T18:36:55.700 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:55 smithi039 ceph-mon[96489]: osdmap e29: 5 total, 4 up, 5 in 2023-07-21T18:36:55.701 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:55 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T18:36:55.701 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:55 smithi039 ceph-mon[96489]: from='osd.4 [v2:172.21.15.155:6808/3361780673,v1:172.21.15.155:6809/3361780673]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2023-07-21T18:36:55.701 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:55 smithi039 ceph-mon[96489]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2023-07-21T18:36:55.701 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:55 smithi039 ceph-mon[96489]: Detected new or changed devices on smithi155 2023-07-21T18:36:55.701 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:55 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:55.701 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:55 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:55.702 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:55 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:36:55.702 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:55 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:36:55.702 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:55 smithi039 ceph-mon[96489]: Adjusting osd_memory_target on smithi155 to 8584M 2023-07-21T18:36:55.702 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:55 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:55.702 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:55 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:36:55.702 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:55 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:36:55.702 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:55 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:36:55.702 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:55 smithi039 ceph-mon[96489]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-21T18:36:56.601 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:56 smithi193 ceph-mon[98204]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2023-07-21T18:36:56.601 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:56 smithi193 ceph-mon[98204]: osdmap e30: 5 total, 4 up, 5 in 2023-07-21T18:36:56.602 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:56 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T18:36:56.602 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:56 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T18:36:56.602 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:56 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T18:36:56.651 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:56 smithi155 ceph-mon[97934]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2023-07-21T18:36:56.651 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:56 smithi155 ceph-mon[97934]: osdmap e30: 5 total, 4 up, 5 in 2023-07-21T18:36:56.651 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:56 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T18:36:56.651 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:56 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T18:36:56.651 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:56 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T18:36:56.700 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:56 smithi039 ceph-mon[96489]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2023-07-21T18:36:56.700 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:56 smithi039 ceph-mon[96489]: osdmap e30: 5 total, 4 up, 5 in 2023-07-21T18:36:56.700 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:56 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T18:36:56.700 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:56 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T18:36:56.700 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:56 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T18:36:56.790 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:36:56.755+0000 7f8a2b9ee700 1 -- 172.21.15.193:0/2850204076 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a24100610 msgr2=0x7f8a24106ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:36:56.790 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:36:56.755+0000 7f8a2b9ee700 1 --2- 172.21.15.193:0/2850204076 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a24100610 0x7f8a24106ed0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c00b0a0 tx=0x7f8a1c00b3b0 comp rx=0 tx=0).stop 2023-07-21T18:36:56.790 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:36:56.755+0000 7f8a2b9ee700 1 -- 172.21.15.193:0/2850204076 shutdown_connections 2023-07-21T18:36:56.790 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:36:56.755+0000 7f8a2b9ee700 1 --2- 172.21.15.193:0/2850204076 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a24100610 0x7f8a24106ed0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:36:56.790 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:36:56.755+0000 7f8a2b9ee700 1 --2- 172.21.15.193:0/2850204076 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a240fea80 0x7f8a240fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:36:56.790 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:36:56.755+0000 7f8a2b9ee700 1 --2- 172.21.15.193:0/2850204076 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8a240ffc80 0x7f8a241000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:36:56.791 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:36:56.755+0000 7f8a2b9ee700 1 -- 172.21.15.193:0/2850204076 >> 172.21.15.193:0/2850204076 conn(0x7f8a240fa0b0 msgr2=0x7f8a240fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:36:56.791 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:36:56.755+0000 7f8a2b9ee700 1 -- 172.21.15.193:0/2850204076 shutdown_connections 2023-07-21T18:36:56.791 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:36:56.755+0000 7f8a2b9ee700 1 -- 172.21.15.193:0/2850204076 wait complete. 2023-07-21T18:36:56.793 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:36:56.755+0000 7f8a2b9ee700 1 Processor -- start 2023-07-21T18:36:56.793 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:36:56.755+0000 7f8a2b9ee700 1 -- start start 2023-07-21T18:36:56.793 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:36:56.755+0000 7f8a2b9ee700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a240fea80 0x7f8a24194880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:36:56.793 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:36:56.756+0000 7f8a2b9ee700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a240ffc80 0x7f8a24194dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:36:56.793 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:36:56.756+0000 7f8a2b9ee700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8a24100610 0x7f8a2419be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:36:56.793 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:36:56.756+0000 7f8a2b9ee700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2419c410 con 0x7f8a240ffc80 2023-07-21T18:36:56.794 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:36:56.756+0000 7f8a2b9ee700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2419c550 con 0x7f8a24100610 2023-07-21T18:36:56.794 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:36:56.756+0000 7f8a2b9ee700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2419c6e0 con 0x7f8a240fea80 2023-07-21T18:36:56.794 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:36:56.756+0000 7f8a2978a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a240fea80 0x7f8a24194880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:36:56.794 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:36:56.756+0000 7f8a2978a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a240fea80 0x7f8a24194880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:50648/0 (socket says 172.21.15.193:50648) 2023-07-21T18:36:56.794 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:36:56.756+0000 7f8a2978a700 1 -- 172.21.15.193:0/1313237262 learned_addr learned my addr 172.21.15.193:0/1313237262 (peer_addr_for_me v2:172.21.15.193:0/0) 2023-07-21T18:36:56.794 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:36:56.756+0000 7f8a29f8b700 1 --2- 172.21.15.193:0/1313237262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8a24100610 0x7f8a2419be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:36:56.794 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:36:56.756+0000 7f8a2978a700 1 -- 172.21.15.193:0/1313237262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8a24100610 msgr2=0x7f8a2419be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:36:56.794 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:36:56.756+0000 7f8a2978a700 1 --2- 172.21.15.193:0/1313237262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8a24100610 0x7f8a2419be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:36:56.795 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:36:56.756+0000 7f8a2978a700 1 -- 172.21.15.193:0/1313237262 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a240ffc80 msgr2=0x7f8a24194dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:36:56.795 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:36:56.756+0000 7f8a2978a700 1 --2- 172.21.15.193:0/1313237262 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a240ffc80 0x7f8a24194dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:36:56.795 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:36:56.756+0000 7f8a2978a700 1 -- 172.21.15.193:0/1313237262 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a1c009d00 con 0x7f8a240fea80 2023-07-21T18:36:56.795 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:36:56.756+0000 7f8a29f8b700 1 --2- 172.21.15.193:0/1313237262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8a24100610 0x7f8a2419be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:36:56.795 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:36:56.756+0000 7f8a2978a700 1 --2- 172.21.15.193:0/1313237262 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a240fea80 0x7f8a24194880 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8a2000e9a0 tx=0x7f8a2000ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:36:56.795 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:36:56.756+0000 7f8a1a7fc700 1 -- 172.21.15.193:0/1313237262 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a20009790 con 0x7f8a240fea80 2023-07-21T18:36:56.795 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:36:56.756+0000 7f8a1a7fc700 1 -- 172.21.15.193:0/1313237262 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a200044e0 con 0x7f8a240fea80 2023-07-21T18:36:56.796 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:36:56.756+0000 7f8a1a7fc700 1 -- 172.21.15.193:0/1313237262 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a20003d70 con 0x7f8a240fea80 2023-07-21T18:36:56.796 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:36:56.756+0000 7f8a2b9ee700 1 -- 172.21.15.193:0/1313237262 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a2419c9c0 con 0x7f8a240fea80 2023-07-21T18:36:56.796 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:36:56.756+0000 7f8a2b9ee700 1 -- 172.21.15.193:0/1313237262 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a2419ce70 con 0x7f8a240fea80 2023-07-21T18:36:56.796 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:36:56.757+0000 7f8a2b9ee700 1 -- 172.21.15.193:0/1313237262 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a24106020 con 0x7f8a240fea80 2023-07-21T18:36:56.796 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:36:56.758+0000 7f8a1a7fc700 1 -- 172.21.15.193:0/1313237262 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f8a20003f10 con 0x7f8a240fea80 2023-07-21T18:36:56.797 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:36:56.759+0000 7f8a1a7fc700 1 --2- 172.21.15.193:0/1313237262 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f8a100a11d0 0x7f8a100a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:36:56.797 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:36:56.759+0000 7f8a1a7fc700 1 -- 172.21.15.193:0/1313237262 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4101+0+0 (secure 0 0 0) 0x7f8a20014070 con 0x7f8a240fea80 2023-07-21T18:36:56.797 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:36:56.759+0000 7f8a28f89700 1 --2- 172.21.15.193:0/1313237262 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f8a100a11d0 0x7f8a100a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:36:56.797 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:36:56.760+0000 7f8a28f89700 1 --2- 172.21.15.193:0/1313237262 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f8a100a11d0 0x7f8a100a3680 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8a14005950 tx=0x7f8a140058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:36:56.798 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:36:56.763+0000 7f8a1a7fc700 1 -- 172.21.15.193:0/1313237262 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f8a200962d0 con 0x7f8a240fea80 2023-07-21T18:36:56.990 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:36:56.955+0000 7f8a2b9ee700 1 -- 172.21.15.193:0/1313237262 --> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f8a24060950 con 0x7f8a100a11d0 2023-07-21T18:36:57.632 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:57 smithi193 ceph-mon[98204]: purged_snaps scrub starts 2023-07-21T18:36:57.632 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:57 smithi193 ceph-mon[98204]: purged_snaps scrub ok 2023-07-21T18:36:57.632 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:57 smithi193 ceph-mon[98204]: osd.4 [v2:172.21.15.155:6808/3361780673,v1:172.21.15.155:6809/3361780673] boot 2023-07-21T18:36:57.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:57 smithi193 ceph-mon[98204]: osdmap e31: 5 total, 5 up, 5 in 2023-07-21T18:36:57.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:57 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T18:36:57.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:57 smithi193 ceph-mon[98204]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:36:57.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:57 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T18:36:57.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:57 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T18:36:57.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:57 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:36:57.634 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:57 smithi193 ceph-mon[98204]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-21T18:36:57.651 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:57 smithi155 ceph-mon[97934]: purged_snaps scrub starts 2023-07-21T18:36:57.651 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:57 smithi155 ceph-mon[97934]: purged_snaps scrub ok 2023-07-21T18:36:57.651 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:57 smithi155 ceph-mon[97934]: osd.4 [v2:172.21.15.155:6808/3361780673,v1:172.21.15.155:6809/3361780673] boot 2023-07-21T18:36:57.651 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:57 smithi155 ceph-mon[97934]: osdmap e31: 5 total, 5 up, 5 in 2023-07-21T18:36:57.651 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:57 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T18:36:57.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:57 smithi155 ceph-mon[97934]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:36:57.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:57 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T18:36:57.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:57 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T18:36:57.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:57 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:36:57.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:57 smithi155 ceph-mon[97934]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-21T18:36:57.700 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:57 smithi039 ceph-mon[96489]: purged_snaps scrub starts 2023-07-21T18:36:57.700 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:57 smithi039 ceph-mon[96489]: purged_snaps scrub ok 2023-07-21T18:36:57.700 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:57 smithi039 ceph-mon[96489]: osd.4 [v2:172.21.15.155:6808/3361780673,v1:172.21.15.155:6809/3361780673] boot 2023-07-21T18:36:57.700 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:57 smithi039 ceph-mon[96489]: osdmap e31: 5 total, 5 up, 5 in 2023-07-21T18:36:57.701 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:57 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T18:36:57.701 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:57 smithi039 ceph-mon[96489]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:36:57.701 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:57 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T18:36:57.701 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:57 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T18:36:57.701 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:57 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:36:57.701 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:57 smithi039 ceph-mon[96489]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-21T18:36:58.632 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:58 smithi193 ceph-mon[98204]: osdmap e32: 5 total, 5 up, 5 in 2023-07-21T18:36:58.651 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:58 smithi155 ceph-mon[97934]: osdmap e32: 5 total, 5 up, 5 in 2023-07-21T18:36:58.700 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:58 smithi039 ceph-mon[96489]: osdmap e32: 5 total, 5 up, 5 in 2023-07-21T18:36:59.632 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:59 smithi193 ceph-mon[98204]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T18:36:59.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:59 smithi193 ceph-mon[98204]: from='client.? 172.21.15.193:0/1974731537' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d267bc2c-42b3-4d96-b2d1-b186dc7e6b18"}]: dispatch 2023-07-21T18:36:59.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:59 smithi193 ceph-mon[98204]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d267bc2c-42b3-4d96-b2d1-b186dc7e6b18"}]: dispatch 2023-07-21T18:36:59.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:59 smithi193 ceph-mon[98204]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d267bc2c-42b3-4d96-b2d1-b186dc7e6b18"}]': finished 2023-07-21T18:36:59.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:59 smithi193 ceph-mon[98204]: osdmap e33: 6 total, 5 up, 6 in 2023-07-21T18:36:59.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:36:59 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T18:36:59.651 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:59 smithi155 ceph-mon[97934]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T18:36:59.651 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:59 smithi155 ceph-mon[97934]: from='client.? 172.21.15.193:0/1974731537' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d267bc2c-42b3-4d96-b2d1-b186dc7e6b18"}]: dispatch 2023-07-21T18:36:59.651 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:59 smithi155 ceph-mon[97934]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d267bc2c-42b3-4d96-b2d1-b186dc7e6b18"}]: dispatch 2023-07-21T18:36:59.651 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:59 smithi155 ceph-mon[97934]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d267bc2c-42b3-4d96-b2d1-b186dc7e6b18"}]': finished 2023-07-21T18:36:59.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:59 smithi155 ceph-mon[97934]: osdmap e33: 6 total, 5 up, 6 in 2023-07-21T18:36:59.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:36:59 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T18:36:59.700 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:59 smithi039 ceph-mon[96489]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T18:36:59.700 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:59 smithi039 ceph-mon[96489]: from='client.? 172.21.15.193:0/1974731537' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d267bc2c-42b3-4d96-b2d1-b186dc7e6b18"}]: dispatch 2023-07-21T18:36:59.700 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:59 smithi039 ceph-mon[96489]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d267bc2c-42b3-4d96-b2d1-b186dc7e6b18"}]: dispatch 2023-07-21T18:36:59.701 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:59 smithi039 ceph-mon[96489]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d267bc2c-42b3-4d96-b2d1-b186dc7e6b18"}]': finished 2023-07-21T18:36:59.701 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:59 smithi039 ceph-mon[96489]: osdmap e33: 6 total, 5 up, 6 in 2023-07-21T18:36:59.701 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:36:59 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T18:37:00.632 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:00 smithi193 ceph-mon[98204]: from='client.? 172.21.15.193:0/2230803007' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T18:37:00.650 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:00 smithi155 ceph-mon[97934]: from='client.? 172.21.15.193:0/2230803007' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T18:37:00.699 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:00 smithi039 ceph-mon[96489]: from='client.? 172.21.15.193:0/2230803007' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T18:37:02.199 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:01 smithi039 ceph-mon[96489]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T18:37:02.382 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:01 smithi193 ceph-mon[98204]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T18:37:02.400 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:01 smithi155 ceph-mon[97934]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T18:37:03.381 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:03 smithi193 ceph-mon[98204]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T18:37:03.449 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:03 smithi039 ceph-mon[96489]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T18:37:03.650 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:03 smithi155 ceph-mon[97934]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T18:37:05.449 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:05 smithi039 ceph-mon[96489]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T18:37:05.534 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:05 smithi193 ceph-mon[98204]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T18:37:05.650 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:05 smithi155 ceph-mon[97934]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T18:37:06.453 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:06 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-07-21T18:37:06.454 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:06 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:37:06.454 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:06 smithi193 ceph-mon[98204]: Deploying daemon osd.5 on smithi193 2023-07-21T18:37:06.650 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:06 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-07-21T18:37:06.650 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:06 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:37:06.651 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:06 smithi155 ceph-mon[97934]: Deploying daemon osd.5 on smithi193 2023-07-21T18:37:06.699 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:06 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-07-21T18:37:06.699 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:06 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:37:06.700 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:06 smithi039 ceph-mon[96489]: Deploying daemon osd.5 on smithi193 2023-07-21T18:37:07.493 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:07 smithi193 ceph-mon[98204]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T18:37:07.650 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:07 smithi155 ceph-mon[97934]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T18:37:07.699 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:07 smithi039 ceph-mon[96489]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T18:37:09.382 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:09 smithi193 ceph-mon[98204]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T18:37:09.650 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:09 smithi155 ceph-mon[97934]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T18:37:09.699 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:09 smithi039 ceph-mon[96489]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T18:37:10.950 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:10 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:10.950 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:10 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:10.951 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:10 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:37:11.150 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:10 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:11.150 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:10 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:11.150 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:10 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:37:11.199 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:10 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:11.199 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:10 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:11.199 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:10 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:37:12.199 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:11 smithi039 ceph-mon[96489]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T18:37:12.236 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:11 smithi193 ceph-mon[98204]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T18:37:12.249 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:37:12.214+0000 7f8a1a7fc700 1 -- 172.21.15.193:0/1313237262 <== mgr.14152 v2:172.21.15.39:6800/2116291712 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f8a24060950 con 0x7f8a100a11d0 2023-07-21T18:37:12.249 INFO:teuthology.orchestra.run.smithi193.stdout:Created osd(s) 5 on host 'smithi193' 2023-07-21T18:37:12.251 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:37:12.217+0000 7f8a2b9ee700 1 -- 172.21.15.193:0/1313237262 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f8a100a11d0 msgr2=0x7f8a100a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:37:12.251 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:37:12.217+0000 7f8a2b9ee700 1 --2- 172.21.15.193:0/1313237262 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f8a100a11d0 0x7f8a100a3680 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8a14005950 tx=0x7f8a140058e0 comp rx=0 tx=0).stop 2023-07-21T18:37:12.252 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:37:12.217+0000 7f8a2b9ee700 1 -- 172.21.15.193:0/1313237262 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a240fea80 msgr2=0x7f8a24194880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:37:12.252 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:37:12.217+0000 7f8a2b9ee700 1 --2- 172.21.15.193:0/1313237262 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a240fea80 0x7f8a24194880 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8a2000e9a0 tx=0x7f8a2000ed60 comp rx=0 tx=0).stop 2023-07-21T18:37:12.252 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:37:12.217+0000 7f8a2b9ee700 1 -- 172.21.15.193:0/1313237262 shutdown_connections 2023-07-21T18:37:12.252 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:37:12.217+0000 7f8a2b9ee700 1 --2- 172.21.15.193:0/1313237262 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f8a100a11d0 0x7f8a100a3680 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:37:12.252 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:37:12.217+0000 7f8a2b9ee700 1 --2- 172.21.15.193:0/1313237262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8a24100610 0x7f8a2419be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:37:12.252 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:37:12.217+0000 7f8a2b9ee700 1 --2- 172.21.15.193:0/1313237262 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a240fea80 0x7f8a24194880 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:37:12.252 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:37:12.217+0000 7f8a2b9ee700 1 --2- 172.21.15.193:0/1313237262 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a240ffc80 0x7f8a24194dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:37:12.253 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:37:12.217+0000 7f8a2b9ee700 1 -- 172.21.15.193:0/1313237262 >> 172.21.15.193:0/1313237262 conn(0x7f8a240fa0b0 msgr2=0x7f8a24103160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:37:12.253 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:37:12.217+0000 7f8a2b9ee700 1 -- 172.21.15.193:0/1313237262 shutdown_connections 2023-07-21T18:37:12.253 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:37:12.217+0000 7f8a2b9ee700 1 -- 172.21.15.193:0/1313237262 wait complete. 2023-07-21T18:37:12.400 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:11 smithi155 ceph-mon[97934]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T18:37:12.638 DEBUG:teuthology.orchestra.run.smithi193:osd.5> sudo journalctl -f -n 0 -u ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@osd.5.service 2023-07-21T18:37:12.640 INFO:tasks.cephadm:Deploying osd.6 on smithi039 with /dev/vg_nvme/lv_2... 2023-07-21T18:37:12.640 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-07-21T18:37:12.672 INFO:journalctl@ceph.osd.5.smithi193.stdout:-- Logs begin at Fri 2023-07-21 18:17:09 UTC. -- 2023-07-21T18:37:13.219 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:37:13.358 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:13 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:13.358 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:13 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:13.358 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:13 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:37:13.358 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:13 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:37:13.358 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:13 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:13.359 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:13 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:13.359 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:13 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:13.359 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:13 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:37:13.400 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:13 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:13.400 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:13 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:13.400 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:13 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:37:13.400 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:13 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:37:13.401 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:13 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:13.401 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:13 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:13.401 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:13 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:13.401 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:13 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:37:13.405 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:13 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:13.405 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:13 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:13.406 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:13 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:37:13.406 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:13 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:37:13.406 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:13 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:13.406 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:13 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:13.406 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:13 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:13.406 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:13 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:37:14.056 INFO:journalctl@ceph.osd.5.smithi193.stdout:Jul 21 18:37:13 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-5[109570]: 2023-07-21T18:37:13.866+0000 7fdc724f6640 -1 osd.5 0 log_to_monitors true 2023-07-21T18:37:14.371 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:14 smithi193 ceph-mon[98204]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T18:37:14.371 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:14 smithi193 ceph-mon[98204]: from='osd.5 [v2:172.21.15.193:6808/2376867721,v1:172.21.15.193:6809/2376867721]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-21T18:37:14.371 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:14 smithi193 ceph-mon[98204]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-21T18:37:14.400 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:14 smithi155 ceph-mon[97934]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T18:37:14.400 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:14 smithi155 ceph-mon[97934]: from='osd.5 [v2:172.21.15.193:6808/2376867721,v1:172.21.15.193:6809/2376867721]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-21T18:37:14.401 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:14 smithi155 ceph-mon[97934]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-21T18:37:14.449 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:14 smithi039 ceph-mon[96489]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T18:37:14.449 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:14 smithi039 ceph-mon[96489]: from='osd.5 [v2:172.21.15.193:6808/2376867721,v1:172.21.15.193:6809/2376867721]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-21T18:37:14.449 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:14 smithi039 ceph-mon[96489]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-21T18:37:14.637 INFO:teuthology.orchestra.run.smithi039.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-07-21T18:37:14.637 INFO:teuthology.orchestra.run.smithi039.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-07-21T18:37:14.638 INFO:teuthology.orchestra.run.smithi039.stderr: stderr: 10+0 records in 2023-07-21T18:37:14.638 INFO:teuthology.orchestra.run.smithi039.stderr:10+0 records out 2023-07-21T18:37:14.638 INFO:teuthology.orchestra.run.smithi039.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0147435 s, 711 MB/s 2023-07-21T18:37:14.638 INFO:teuthology.orchestra.run.smithi039.stderr: stderr: 2023-07-21T18:37:14.639 INFO:teuthology.orchestra.run.smithi039.stderr:--> Zapping successful for: 2023-07-21T18:37:15.070 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi039:vg_nvme/lv_2 2023-07-21T18:37:15.381 INFO:journalctl@ceph.osd.5.smithi193.stdout:Jul 21 18:37:15 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-5[109570]: 2023-07-21T18:37:15.034+0000 7fdc6736d700 -1 osd.5 0 waiting for initial osdmap 2023-07-21T18:37:15.381 INFO:journalctl@ceph.osd.5.smithi193.stdout:Jul 21 18:37:15 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-5[109570]: 2023-07-21T18:37:15.041+0000 7fdc6195f700 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-21T18:37:15.382 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:15 smithi193 ceph-mon[98204]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-07-21T18:37:15.382 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:15 smithi193 ceph-mon[98204]: osdmap e34: 6 total, 5 up, 6 in 2023-07-21T18:37:15.382 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:15 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T18:37:15.382 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:15 smithi193 ceph-mon[98204]: from='osd.5 [v2:172.21.15.193:6808/2376867721,v1:172.21.15.193:6809/2376867721]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2023-07-21T18:37:15.382 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:15 smithi193 ceph-mon[98204]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2023-07-21T18:37:15.382 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:15 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:15.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:15 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:15.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:15 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:37:15.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:15 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:37:15.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:15 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:15.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:15 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:37:15.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:15 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:37:15.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:15 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:15.400 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:15 smithi155 ceph-mon[97934]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-07-21T18:37:15.400 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:15 smithi155 ceph-mon[97934]: osdmap e34: 6 total, 5 up, 6 in 2023-07-21T18:37:15.400 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:15 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T18:37:15.401 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:15 smithi155 ceph-mon[97934]: from='osd.5 [v2:172.21.15.193:6808/2376867721,v1:172.21.15.193:6809/2376867721]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2023-07-21T18:37:15.401 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:15 smithi155 ceph-mon[97934]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2023-07-21T18:37:15.401 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:15 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:15.401 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:15 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:15.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:15 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:37:15.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:15 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:37:15.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:15 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:15.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:15 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:37:15.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:15 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:37:15.403 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:15 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:15.449 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:15 smithi039 ceph-mon[96489]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-07-21T18:37:15.449 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:15 smithi039 ceph-mon[96489]: osdmap e34: 6 total, 5 up, 6 in 2023-07-21T18:37:15.449 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:15 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T18:37:15.449 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:15 smithi039 ceph-mon[96489]: from='osd.5 [v2:172.21.15.193:6808/2376867721,v1:172.21.15.193:6809/2376867721]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2023-07-21T18:37:15.450 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:15 smithi039 ceph-mon[96489]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2023-07-21T18:37:15.450 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:15 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:15.450 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:15 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:15.450 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:15 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:37:15.450 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:15 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:37:15.450 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:15 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:15.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:15 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:37:15.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:15 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:37:15.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:15 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:15.630 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:37:16.382 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:16 smithi193 ceph-mon[98204]: Detected new or changed devices on smithi193 2023-07-21T18:37:16.382 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:16 smithi193 ceph-mon[98204]: Adjusting osd_memory_target on smithi193 to 8584M 2023-07-21T18:37:16.382 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:16 smithi193 ceph-mon[98204]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2023-07-21T18:37:16.382 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:16 smithi193 ceph-mon[98204]: osdmap e35: 6 total, 5 up, 6 in 2023-07-21T18:37:16.382 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:16 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T18:37:16.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:16 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T18:37:16.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:16 smithi193 ceph-mon[98204]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T18:37:16.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:16 smithi193 ceph-mon[98204]: osd.5 [v2:172.21.15.193:6808/2376867721,v1:172.21.15.193:6809/2376867721] boot 2023-07-21T18:37:16.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:16 smithi193 ceph-mon[98204]: osdmap e36: 6 total, 6 up, 6 in 2023-07-21T18:37:16.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:16 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T18:37:16.400 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:16 smithi155 ceph-mon[97934]: Detected new or changed devices on smithi193 2023-07-21T18:37:16.400 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:16 smithi155 ceph-mon[97934]: Adjusting osd_memory_target on smithi193 to 8584M 2023-07-21T18:37:16.401 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:16 smithi155 ceph-mon[97934]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2023-07-21T18:37:16.401 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:16 smithi155 ceph-mon[97934]: osdmap e35: 6 total, 5 up, 6 in 2023-07-21T18:37:16.401 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:16 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T18:37:16.401 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:16 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T18:37:16.401 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:16 smithi155 ceph-mon[97934]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T18:37:16.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:16 smithi155 ceph-mon[97934]: osd.5 [v2:172.21.15.193:6808/2376867721,v1:172.21.15.193:6809/2376867721] boot 2023-07-21T18:37:16.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:16 smithi155 ceph-mon[97934]: osdmap e36: 6 total, 6 up, 6 in 2023-07-21T18:37:16.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:16 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T18:37:16.449 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:16 smithi039 ceph-mon[96489]: Detected new or changed devices on smithi193 2023-07-21T18:37:16.449 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:16 smithi039 ceph-mon[96489]: Adjusting osd_memory_target on smithi193 to 8584M 2023-07-21T18:37:16.450 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:16 smithi039 ceph-mon[96489]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2023-07-21T18:37:16.450 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:16 smithi039 ceph-mon[96489]: osdmap e35: 6 total, 5 up, 6 in 2023-07-21T18:37:16.450 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:16 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T18:37:16.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:16 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T18:37:16.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:16 smithi039 ceph-mon[96489]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T18:37:16.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:16 smithi039 ceph-mon[96489]: osd.5 [v2:172.21.15.193:6808/2376867721,v1:172.21.15.193:6809/2376867721] boot 2023-07-21T18:37:16.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:16 smithi039 ceph-mon[96489]: osdmap e36: 6 total, 6 up, 6 in 2023-07-21T18:37:16.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:16 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T18:37:16.692 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:37:16.654+0000 7fe23e4c0700 1 -- 172.21.15.39:0/1981828824 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2380fece0 msgr2=0x7fe2380ff150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:37:16.692 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:37:16.654+0000 7fe23e4c0700 1 --2- 172.21.15.39:0/1981828824 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2380fece0 0x7fe2380ff150 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe22c009990 tx=0x7fe22c009ca0 comp rx=0 tx=0).stop 2023-07-21T18:37:16.692 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:37:16.654+0000 7fe23e4c0700 1 -- 172.21.15.39:0/1981828824 shutdown_connections 2023-07-21T18:37:16.692 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:37:16.654+0000 7fe23e4c0700 1 --2- 172.21.15.39:0/1981828824 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2380ff690 0x7fe238106f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:37:16.693 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:37:16.654+0000 7fe23e4c0700 1 --2- 172.21.15.39:0/1981828824 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe238100110 0x7fe238100520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:37:16.693 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:37:16.654+0000 7fe23e4c0700 1 --2- 172.21.15.39:0/1981828824 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2380fece0 0x7fe2380ff150 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:37:16.693 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:37:16.654+0000 7fe23e4c0700 1 -- 172.21.15.39:0/1981828824 >> 172.21.15.39:0/1981828824 conn(0x7fe2380f9860 msgr2=0x7fe2380fbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:37:16.693 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:37:16.655+0000 7fe23e4c0700 1 -- 172.21.15.39:0/1981828824 shutdown_connections 2023-07-21T18:37:16.693 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:37:16.655+0000 7fe23e4c0700 1 -- 172.21.15.39:0/1981828824 wait complete. 2023-07-21T18:37:16.693 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:37:16.655+0000 7fe23e4c0700 1 Processor -- start 2023-07-21T18:37:16.693 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:37:16.655+0000 7fe23e4c0700 1 -- start start 2023-07-21T18:37:16.694 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:37:16.655+0000 7fe23e4c0700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2380fece0 0x7fe2380fe730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:37:16.694 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:37:16.655+0000 7fe23e4c0700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2380ff690 0x7fe23819a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:37:16.694 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:37:16.655+0000 7fe23e4c0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe238100110 0x7fe23819cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:37:16.694 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:37:16.655+0000 7fe23e4c0700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe23819d3b0 con 0x7fe2380fece0 2023-07-21T18:37:16.694 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:37:16.655+0000 7fe23e4c0700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe23819d520 con 0x7fe238100110 2023-07-21T18:37:16.695 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:37:16.655+0000 7fe23e4c0700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe23819d6e0 con 0x7fe2380ff690 2023-07-21T18:37:16.695 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:37:16.656+0000 7fe237fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2380fece0 0x7fe2380fe730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:37:16.695 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:37:16.656+0000 7fe237fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2380fece0 0x7fe2380fe730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54326/0 (socket says 172.21.15.39:54326) 2023-07-21T18:37:16.695 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:37:16.656+0000 7fe237fff700 1 -- 172.21.15.39:0/1558747838 learned_addr learned my addr 172.21.15.39:0/1558747838 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:37:16.695 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:37:16.656+0000 7fe2377fe700 1 --2- 172.21.15.39:0/1558747838 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2380ff690 0x7fe23819a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:37:16.695 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:37:16.656+0000 7fe23ca5d700 1 --2- 172.21.15.39:0/1558747838 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe238100110 0x7fe23819cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:37:16.695 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:37:16.656+0000 7fe237fff700 1 -- 172.21.15.39:0/1558747838 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2380ff690 msgr2=0x7fe23819a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:37:16.696 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:37:16.656+0000 7fe237fff700 1 --2- 172.21.15.39:0/1558747838 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2380ff690 0x7fe23819a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:37:16.696 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:37:16.656+0000 7fe237fff700 1 -- 172.21.15.39:0/1558747838 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe238100110 msgr2=0x7fe23819cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:37:16.696 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:37:16.656+0000 7fe237fff700 1 --2- 172.21.15.39:0/1558747838 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe238100110 0x7fe23819cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:37:16.696 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:37:16.656+0000 7fe237fff700 1 -- 172.21.15.39:0/1558747838 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe22c0095f0 con 0x7fe2380fece0 2023-07-21T18:37:16.696 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:37:16.656+0000 7fe237fff700 1 --2- 172.21.15.39:0/1558747838 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2380fece0 0x7fe2380fe730 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe22800c920 tx=0x7fe22800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:37:16.696 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:37:16.656+0000 7fe2357fa700 1 -- 172.21.15.39:0/1558747838 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe228007b00 con 0x7fe2380fece0 2023-07-21T18:37:16.697 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:37:16.656+0000 7fe2357fa700 1 -- 172.21.15.39:0/1558747838 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe22800ce70 con 0x7fe2380fece0 2023-07-21T18:37:16.697 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:37:16.656+0000 7fe2357fa700 1 -- 172.21.15.39:0/1558747838 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2280054e0 con 0x7fe2380fece0 2023-07-21T18:37:16.697 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:37:16.656+0000 7fe23e4c0700 1 -- 172.21.15.39:0/1558747838 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2381944c0 con 0x7fe2380fece0 2023-07-21T18:37:16.698 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:37:16.656+0000 7fe23e4c0700 1 -- 172.21.15.39:0/1558747838 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2381949c0 con 0x7fe2380fece0 2023-07-21T18:37:16.698 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:37:16.657+0000 7fe23e4c0700 1 -- 172.21.15.39:0/1558747838 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2381060e0 con 0x7fe2380fece0 2023-07-21T18:37:16.699 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:37:16.659+0000 7fe2357fa700 1 -- 172.21.15.39:0/1558747838 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fe228004750 con 0x7fe2380fece0 2023-07-21T18:37:16.700 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:37:16.659+0000 7fe2357fa700 1 --2- 172.21.15.39:0/1558747838 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fe2200a11d0 0x7fe2200a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:37:16.700 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:37:16.659+0000 7fe2357fa700 1 -- 172.21.15.39:0/1558747838 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4566+0+0 (secure 0 0 0) 0x7fe2280cb980 con 0x7fe2380fece0 2023-07-21T18:37:16.700 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:37:16.659+0000 7fe2377fe700 1 --2- 172.21.15.39:0/1558747838 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fe2200a11d0 0x7fe2200a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:37:16.700 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:37:16.660+0000 7fe2377fe700 1 --2- 172.21.15.39:0/1558747838 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fe2200a11d0 0x7fe2200a3680 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe2380fecb0 tx=0x7fe22c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:37:16.701 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:37:16.663+0000 7fe2357fa700 1 -- 172.21.15.39:0/1558747838 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fe228095ff0 con 0x7fe2380fece0 2023-07-21T18:37:16.904 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:37:16.870+0000 7fe23e4c0700 1 -- 172.21.15.39:0/1558747838 --> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fe238195ed0 con 0x7fe2200a11d0 2023-07-21T18:37:17.381 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:17 smithi193 ceph-mon[98204]: purged_snaps scrub starts 2023-07-21T18:37:17.381 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:17 smithi193 ceph-mon[98204]: purged_snaps scrub ok 2023-07-21T18:37:17.382 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:17 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T18:37:17.382 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:17 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T18:37:17.382 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:17 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:37:17.382 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:17 smithi193 ceph-mon[98204]: osdmap e37: 6 total, 6 up, 6 in 2023-07-21T18:37:17.400 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:17 smithi155 ceph-mon[97934]: purged_snaps scrub starts 2023-07-21T18:37:17.400 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:17 smithi155 ceph-mon[97934]: purged_snaps scrub ok 2023-07-21T18:37:17.400 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:17 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T18:37:17.400 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:17 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T18:37:17.400 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:17 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:37:17.401 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:17 smithi155 ceph-mon[97934]: osdmap e37: 6 total, 6 up, 6 in 2023-07-21T18:37:17.449 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:17 smithi039 ceph-mon[96489]: purged_snaps scrub starts 2023-07-21T18:37:17.449 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:17 smithi039 ceph-mon[96489]: purged_snaps scrub ok 2023-07-21T18:37:17.450 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:17 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T18:37:17.450 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:17 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T18:37:17.450 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:17 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:37:17.450 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:17 smithi039 ceph-mon[96489]: osdmap e37: 6 total, 6 up, 6 in 2023-07-21T18:37:18.381 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:18 smithi193 ceph-mon[98204]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:37:18.381 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:18 smithi193 ceph-mon[98204]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-07-21T18:37:18.381 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:18 smithi193 ceph-mon[98204]: osdmap e38: 6 total, 6 up, 6 in 2023-07-21T18:37:18.400 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:18 smithi155 ceph-mon[97934]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:37:18.400 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:18 smithi155 ceph-mon[97934]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-07-21T18:37:18.400 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:18 smithi155 ceph-mon[97934]: osdmap e38: 6 total, 6 up, 6 in 2023-07-21T18:37:18.448 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:18 smithi039 ceph-mon[96489]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:37:18.449 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:18 smithi039 ceph-mon[96489]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-07-21T18:37:18.449 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:18 smithi039 ceph-mon[96489]: osdmap e38: 6 total, 6 up, 6 in 2023-07-21T18:37:20.632 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:20 smithi193 ceph-mon[98204]: osdmap e39: 6 total, 6 up, 6 in 2023-07-21T18:37:20.632 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:20 smithi193 ceph-mon[98204]: pgmap v125: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-21T18:37:20.632 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:20 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/509980606' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "beec8088-970a-473e-bd67-6eee7cbdef24"}]: dispatch 2023-07-21T18:37:20.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:20 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/509980606' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "beec8088-970a-473e-bd67-6eee7cbdef24"}]': finished 2023-07-21T18:37:20.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:20 smithi193 ceph-mon[98204]: osdmap e40: 7 total, 6 up, 7 in 2023-07-21T18:37:20.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:20 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T18:37:20.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:20 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/881357768' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T18:37:20.650 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:20 smithi155 ceph-mon[97934]: osdmap e39: 6 total, 6 up, 6 in 2023-07-21T18:37:20.651 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:20 smithi155 ceph-mon[97934]: pgmap v125: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-21T18:37:20.651 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:20 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/509980606' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "beec8088-970a-473e-bd67-6eee7cbdef24"}]: dispatch 2023-07-21T18:37:20.651 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:20 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/509980606' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "beec8088-970a-473e-bd67-6eee7cbdef24"}]': finished 2023-07-21T18:37:20.651 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:20 smithi155 ceph-mon[97934]: osdmap e40: 7 total, 6 up, 7 in 2023-07-21T18:37:20.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:20 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T18:37:20.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:20 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/881357768' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T18:37:20.699 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:20 smithi039 ceph-mon[96489]: osdmap e39: 6 total, 6 up, 6 in 2023-07-21T18:37:20.699 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:20 smithi039 ceph-mon[96489]: pgmap v125: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-21T18:37:20.700 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:20 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/509980606' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "beec8088-970a-473e-bd67-6eee7cbdef24"}]: dispatch 2023-07-21T18:37:20.700 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:20 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/509980606' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "beec8088-970a-473e-bd67-6eee7cbdef24"}]': finished 2023-07-21T18:37:20.700 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:20 smithi039 ceph-mon[96489]: osdmap e40: 7 total, 6 up, 7 in 2023-07-21T18:37:20.700 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:20 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T18:37:20.700 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:20 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/881357768' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T18:37:22.380 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:22 smithi193 ceph-mon[98204]: pgmap v127: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 138 KiB/s, 0 objects/s recovering 2023-07-21T18:37:22.399 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:22 smithi155 ceph-mon[97934]: pgmap v127: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 138 KiB/s, 0 objects/s recovering 2023-07-21T18:37:22.448 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:22 smithi039 ceph-mon[96489]: pgmap v127: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 138 KiB/s, 0 objects/s recovering 2023-07-21T18:37:23.630 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:23 smithi193 ceph-mon[98204]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-21T18:37:23.649 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:23 smithi155 ceph-mon[97934]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-21T18:37:23.698 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:23 smithi039 ceph-mon[96489]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-21T18:37:25.560 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:25 smithi039 ceph-mon[96489]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 80 KiB/s, 0 objects/s recovering 2023-07-21T18:37:25.630 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:25 smithi193 ceph-mon[98204]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 80 KiB/s, 0 objects/s recovering 2023-07-21T18:37:25.649 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:25 smithi155 ceph-mon[97934]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 80 KiB/s, 0 objects/s recovering 2023-07-21T18:37:27.448 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:27 smithi039 ceph-mon[96489]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-21T18:37:27.630 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:27 smithi193 ceph-mon[98204]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-21T18:37:27.649 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:27 smithi155 ceph-mon[97934]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-21T18:37:28.448 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:28 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-07-21T18:37:28.448 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:28 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:37:28.448 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:28 smithi039 ceph-mon[96489]: Deploying daemon osd.6 on smithi039 2023-07-21T18:37:28.630 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:28 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-07-21T18:37:28.630 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:28 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:37:28.631 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:28 smithi193 ceph-mon[98204]: Deploying daemon osd.6 on smithi039 2023-07-21T18:37:28.649 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:28 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-07-21T18:37:28.649 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:28 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:37:28.649 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:28 smithi155 ceph-mon[97934]: Deploying daemon osd.6 on smithi039 2023-07-21T18:37:29.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:29 smithi039 ceph-mon[96489]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-21T18:37:29.630 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:29 smithi193 ceph-mon[98204]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-21T18:37:29.649 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:29 smithi155 ceph-mon[97934]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-21T18:37:32.380 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:31 smithi193 ceph-mon[98204]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-21T18:37:32.380 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:31 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:32.381 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:31 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:37:32.381 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:31 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:32.399 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:31 smithi155 ceph-mon[97934]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-21T18:37:32.399 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:31 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:32.399 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:31 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:37:32.400 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:31 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:32.448 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:31 smithi039 ceph-mon[96489]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-21T18:37:32.448 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:31 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:32.448 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:31 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:37:32.448 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:31 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:33.538 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:33 smithi039 ceph-mon[96489]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-21T18:37:33.630 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:33 smithi193 ceph-mon[98204]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-21T18:37:33.649 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:33 smithi155 ceph-mon[97934]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-21T18:37:34.572 INFO:teuthology.orchestra.run.smithi039.stdout:Created osd(s) 6 on host 'smithi039' 2023-07-21T18:37:34.572 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:37:34.539+0000 7fe2357fa700 1 -- 172.21.15.39:0/1558747838 <== mgr.14152 v2:172.21.15.39:6800/2116291712 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fe238195ed0 con 0x7fe2200a11d0 2023-07-21T18:37:34.574 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:37:34.541+0000 7fe23e4c0700 1 -- 172.21.15.39:0/1558747838 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fe2200a11d0 msgr2=0x7fe2200a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:37:34.574 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:37:34.541+0000 7fe23e4c0700 1 --2- 172.21.15.39:0/1558747838 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fe2200a11d0 0x7fe2200a3680 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe2380fecb0 tx=0x7fe22c005fd0 comp rx=0 tx=0).stop 2023-07-21T18:37:34.574 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:37:34.541+0000 7fe23e4c0700 1 -- 172.21.15.39:0/1558747838 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2380fece0 msgr2=0x7fe2380fe730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:37:34.574 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:37:34.541+0000 7fe23e4c0700 1 --2- 172.21.15.39:0/1558747838 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2380fece0 0x7fe2380fe730 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe22800c920 tx=0x7fe22800cc30 comp rx=0 tx=0).stop 2023-07-21T18:37:34.574 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:37:34.542+0000 7fe23e4c0700 1 -- 172.21.15.39:0/1558747838 shutdown_connections 2023-07-21T18:37:34.575 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:37:34.542+0000 7fe23e4c0700 1 --2- 172.21.15.39:0/1558747838 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fe2200a11d0 0x7fe2200a3680 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:37:34.575 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:37:34.542+0000 7fe23e4c0700 1 --2- 172.21.15.39:0/1558747838 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe238100110 0x7fe23819cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:37:34.575 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:37:34.542+0000 7fe23e4c0700 1 --2- 172.21.15.39:0/1558747838 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2380fece0 0x7fe2380fe730 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:37:34.575 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:37:34.542+0000 7fe23e4c0700 1 --2- 172.21.15.39:0/1558747838 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2380ff690 0x7fe23819a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:37:34.575 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:37:34.542+0000 7fe23e4c0700 1 -- 172.21.15.39:0/1558747838 >> 172.21.15.39:0/1558747838 conn(0x7fe2380f9860 msgr2=0x7fe238103220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:37:34.575 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:37:34.542+0000 7fe23e4c0700 1 -- 172.21.15.39:0/1558747838 shutdown_connections 2023-07-21T18:37:34.575 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:37:34.542+0000 7fe23e4c0700 1 -- 172.21.15.39:0/1558747838 wait complete. 2023-07-21T18:37:34.929 DEBUG:teuthology.orchestra.run.smithi039:osd.6> sudo journalctl -f -n 0 -u ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@osd.6.service 2023-07-21T18:37:34.931 INFO:tasks.cephadm:Deploying osd.7 on smithi155 with /dev/vg_nvme/lv_2... 2023-07-21T18:37:34.931 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-07-21T18:37:34.973 INFO:journalctl@ceph.osd.6.smithi039.stdout:-- Logs begin at Fri 2023-07-21 18:18:00 UTC. -- 2023-07-21T18:37:35.198 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:34 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:35.198 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:34 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:35.198 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:34 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:37:35.198 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:34 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:37:35.199 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:34 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:35.199 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:34 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:35.199 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:34 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:35.199 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:34 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:37:35.380 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:34 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:35.380 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:34 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:35.380 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:34 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:37:35.381 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:34 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:37:35.381 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:34 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:35.381 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:34 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:35.381 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:34 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:35.381 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:34 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:37:35.399 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:34 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:35.399 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:34 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:35.399 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:34 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:37:35.399 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:34 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:37:35.399 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:34 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:35.399 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:34 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:35.400 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:34 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:35.400 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:34 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:37:35.504 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.b/config 2023-07-21T18:37:35.727 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 21 18:37:35 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-6[120362]: 2023-07-21T18:37:35.556+0000 7fe23a989640 -1 osd.6 0 log_to_monitors true 2023-07-21T18:37:36.149 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:35 smithi155 ceph-mon[97934]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-21T18:37:36.149 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:35 smithi155 ceph-mon[97934]: from='osd.6 [v2:172.21.15.39:6818/2417770529,v1:172.21.15.39:6819/2417770529]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-07-21T18:37:36.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:35 smithi039 ceph-mon[96489]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-21T18:37:36.257 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:35 smithi039 ceph-mon[96489]: from='osd.6 [v2:172.21.15.39:6818/2417770529,v1:172.21.15.39:6819/2417770529]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-07-21T18:37:36.380 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:35 smithi193 ceph-mon[98204]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-21T18:37:36.380 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:35 smithi193 ceph-mon[98204]: from='osd.6 [v2:172.21.15.39:6818/2417770529,v1:172.21.15.39:6819/2417770529]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-07-21T18:37:37.079 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 21 18:37:36 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-6[120362]: 2023-07-21T18:37:36.966+0000 7fe22f800700 -1 osd.6 0 waiting for initial osdmap 2023-07-21T18:37:37.079 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 21 18:37:36 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-6[120362]: 2023-07-21T18:37:36.976+0000 7fe229df2700 -1 osd.6 42 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-21T18:37:37.080 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:36 smithi039 ceph-mon[96489]: from='osd.6 [v2:172.21.15.39:6818/2417770529,v1:172.21.15.39:6819/2417770529]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-07-21T18:37:37.080 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:36 smithi039 ceph-mon[96489]: osdmap e41: 7 total, 6 up, 7 in 2023-07-21T18:37:37.080 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:36 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T18:37:37.080 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:36 smithi039 ceph-mon[96489]: from='osd.6 [v2:172.21.15.39:6818/2417770529,v1:172.21.15.39:6819/2417770529]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2023-07-21T18:37:37.270 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-07-21T18:37:37.270 INFO:teuthology.orchestra.run.smithi155.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-07-21T18:37:37.271 INFO:teuthology.orchestra.run.smithi155.stderr: stderr: 10+0 records in 2023-07-21T18:37:37.271 INFO:teuthology.orchestra.run.smithi155.stderr:10+0 records out 2023-07-21T18:37:37.271 INFO:teuthology.orchestra.run.smithi155.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0108398 s, 967 MB/s 2023-07-21T18:37:37.271 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping successful for: 2023-07-21T18:37:37.306 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:36 smithi155 ceph-mon[97934]: from='osd.6 [v2:172.21.15.39:6818/2417770529,v1:172.21.15.39:6819/2417770529]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-07-21T18:37:37.306 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:36 smithi155 ceph-mon[97934]: osdmap e41: 7 total, 6 up, 7 in 2023-07-21T18:37:37.307 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:36 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T18:37:37.307 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:36 smithi155 ceph-mon[97934]: from='osd.6 [v2:172.21.15.39:6818/2417770529,v1:172.21.15.39:6819/2417770529]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2023-07-21T18:37:37.380 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:36 smithi193 ceph-mon[98204]: from='osd.6 [v2:172.21.15.39:6818/2417770529,v1:172.21.15.39:6819/2417770529]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-07-21T18:37:37.380 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:36 smithi193 ceph-mon[98204]: osdmap e41: 7 total, 6 up, 7 in 2023-07-21T18:37:37.380 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:36 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T18:37:37.380 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:36 smithi193 ceph-mon[98204]: from='osd.6 [v2:172.21.15.39:6818/2417770529,v1:172.21.15.39:6819/2417770529]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2023-07-21T18:37:37.657 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi155:vg_nvme/lv_2 2023-07-21T18:37:38.208 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.b/config 2023-07-21T18:37:38.380 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:37 smithi193 ceph-mon[98204]: from='osd.6 [v2:172.21.15.39:6818/2417770529,v1:172.21.15.39:6819/2417770529]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2023-07-21T18:37:38.380 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:37 smithi193 ceph-mon[98204]: osdmap e42: 7 total, 6 up, 7 in 2023-07-21T18:37:38.380 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:37 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T18:37:38.380 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:37 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T18:37:38.381 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:37 smithi193 ceph-mon[98204]: Detected new or changed devices on smithi039 2023-07-21T18:37:38.381 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:37 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:38.381 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:37 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:38.381 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:37 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:37:38.382 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:37 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:37:38.382 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:37 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:37:38.382 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:37 smithi193 ceph-mon[98204]: Adjusting osd_memory_target on smithi039 to 5722M 2023-07-21T18:37:38.382 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:37 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:38.382 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:37 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:37:38.382 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:37 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:37:38.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:37 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:38.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:37 smithi193 ceph-mon[98204]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-21T18:37:38.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:37 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T18:37:38.392 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:37 smithi155 ceph-mon[97934]: from='osd.6 [v2:172.21.15.39:6818/2417770529,v1:172.21.15.39:6819/2417770529]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2023-07-21T18:37:38.393 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:37 smithi155 ceph-mon[97934]: osdmap e42: 7 total, 6 up, 7 in 2023-07-21T18:37:38.393 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:37 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T18:37:38.393 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:37 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T18:37:38.394 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:37 smithi155 ceph-mon[97934]: Detected new or changed devices on smithi039 2023-07-21T18:37:38.394 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:37 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:38.394 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:37 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:38.394 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:37 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:37:38.395 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:37 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:37:38.395 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:37 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:37:38.395 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:37 smithi155 ceph-mon[97934]: Adjusting osd_memory_target on smithi039 to 5722M 2023-07-21T18:37:38.395 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:37 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:38.396 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:37 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:37:38.396 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:37 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:37:38.396 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:37 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:38.396 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:37 smithi155 ceph-mon[97934]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-21T18:37:38.396 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:37 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T18:37:38.448 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:37 smithi039 ceph-mon[96489]: from='osd.6 [v2:172.21.15.39:6818/2417770529,v1:172.21.15.39:6819/2417770529]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2023-07-21T18:37:38.448 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:37 smithi039 ceph-mon[96489]: osdmap e42: 7 total, 6 up, 7 in 2023-07-21T18:37:38.448 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:37 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T18:37:38.448 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:37 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T18:37:38.449 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:37 smithi039 ceph-mon[96489]: Detected new or changed devices on smithi039 2023-07-21T18:37:38.449 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:37 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:38.449 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:37 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:38.449 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:37 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:37:38.449 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:37 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:37:38.449 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:37 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:37:38.449 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:37 smithi039 ceph-mon[96489]: Adjusting osd_memory_target on smithi039 to 5722M 2023-07-21T18:37:38.449 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:37 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:38.450 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:37 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:37:38.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:37 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:37:38.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:37 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:38.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:37 smithi039 ceph-mon[96489]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-21T18:37:38.453 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:37 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T18:37:39.234 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:38 smithi155 ceph-mon[97934]: purged_snaps scrub starts 2023-07-21T18:37:39.234 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:38 smithi155 ceph-mon[97934]: purged_snaps scrub ok 2023-07-21T18:37:39.234 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:38 smithi155 ceph-mon[97934]: osd.6 [v2:172.21.15.39:6818/2417770529,v1:172.21.15.39:6819/2417770529] boot 2023-07-21T18:37:39.234 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:38 smithi155 ceph-mon[97934]: osdmap e43: 7 total, 7 up, 7 in 2023-07-21T18:37:39.234 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:38 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T18:37:39.365 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:37:39.333+0000 7f0b6942c700 1 -- 172.21.15.155:0/1333808050 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b640ffbd0 msgr2=0x7f0b640fffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:37:39.366 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:37:39.333+0000 7f0b6942c700 1 --2- 172.21.15.155:0/1333808050 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b640ffbd0 0x7f0b640fffe0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0b58009990 tx=0x7f0b58009ca0 comp rx=0 tx=0).stop 2023-07-21T18:37:39.366 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:37:39.333+0000 7f0b6942c700 1 -- 172.21.15.155:0/1333808050 shutdown_connections 2023-07-21T18:37:39.366 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:37:39.334+0000 7f0b6942c700 1 --2- 172.21.15.155:0/1333808050 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b640ffbd0 0x7f0b640fffe0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:37:39.366 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:37:39.334+0000 7f0b6942c700 1 --2- 172.21.15.155:0/1333808050 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0b64101760 0x7f0b64108020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:37:39.366 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:37:39.334+0000 7f0b6942c700 1 --2- 172.21.15.155:0/1333808050 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b64100dd0 0x7f0b64101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:37:39.367 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:37:39.334+0000 7f0b6942c700 1 -- 172.21.15.155:0/1333808050 >> 172.21.15.155:0/1333808050 conn(0x7f0b640fb220 msgr2=0x7f0b640fd650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:37:39.367 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:37:39.334+0000 7f0b6942c700 1 -- 172.21.15.155:0/1333808050 shutdown_connections 2023-07-21T18:37:39.367 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:37:39.334+0000 7f0b6942c700 1 -- 172.21.15.155:0/1333808050 wait complete. 2023-07-21T18:37:39.369 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:37:39.334+0000 7f0b6942c700 1 Processor -- start 2023-07-21T18:37:39.369 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:37:39.334+0000 7f0b6942c700 1 -- start start 2023-07-21T18:37:39.369 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:37:39.334+0000 7f0b6942c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b640ffbd0 0x7f0b641958a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:37:39.369 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:37:39.334+0000 7f0b6942c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0b64100dd0 0x7f0b64195de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:37:39.369 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:37:39.335+0000 7f0b6942c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b64101760 0x7f0b6419ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:37:39.369 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:37:39.335+0000 7f0b6942c700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b6419d430 con 0x7f0b64100dd0 2023-07-21T18:37:39.370 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:37:39.335+0000 7f0b6942c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b6419d570 con 0x7f0b640ffbd0 2023-07-21T18:37:39.370 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:37:39.335+0000 7f0b6942c700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b6419d700 con 0x7f0b64101760 2023-07-21T18:37:39.370 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:37:39.335+0000 7f0b62ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b640ffbd0 0x7f0b641958a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:37:39.370 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:37:39.335+0000 7f0b62ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b640ffbd0 0x7f0b641958a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:57370/0 (socket says 172.21.15.155:57370) 2023-07-21T18:37:39.370 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:37:39.335+0000 7f0b62ffd700 1 -- 172.21.15.155:0/941265199 learned_addr learned my addr 172.21.15.155:0/941265199 (peer_addr_for_me v2:172.21.15.155:0/0) 2023-07-21T18:37:39.370 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:37:39.335+0000 7f0b627fc700 1 --2- 172.21.15.155:0/941265199 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0b64100dd0 0x7f0b64195de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:37:39.371 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:37:39.335+0000 7f0b62ffd700 1 -- 172.21.15.155:0/941265199 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b64101760 msgr2=0x7f0b6419ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:37:39.371 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:37:39.335+0000 7f0b637fe700 1 --2- 172.21.15.155:0/941265199 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b64101760 0x7f0b6419ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:37:39.371 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:37:39.335+0000 7f0b62ffd700 1 --2- 172.21.15.155:0/941265199 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b64101760 0x7f0b6419ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:37:39.371 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:37:39.335+0000 7f0b62ffd700 1 -- 172.21.15.155:0/941265199 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0b64100dd0 msgr2=0x7f0b64195de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:37:39.371 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:37:39.335+0000 7f0b62ffd700 1 --2- 172.21.15.155:0/941265199 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0b64100dd0 0x7f0b64195de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:37:39.371 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:37:39.335+0000 7f0b62ffd700 1 -- 172.21.15.155:0/941265199 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b580095f0 con 0x7f0b640ffbd0 2023-07-21T18:37:39.372 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:37:39.335+0000 7f0b627fc700 1 --2- 172.21.15.155:0/941265199 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0b64100dd0 0x7f0b64195de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:37:39.372 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:37:39.335+0000 7f0b637fe700 1 --2- 172.21.15.155:0/941265199 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b64101760 0x7f0b6419ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:37:39.372 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:37:39.335+0000 7f0b62ffd700 1 --2- 172.21.15.155:0/941265199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b640ffbd0 0x7f0b641958a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0b580095c0 tx=0x7f0b5800f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:37:39.372 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:37:39.335+0000 7f0b4bfff700 1 -- 172.21.15.155:0/941265199 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b5801d070 con 0x7f0b640ffbd0 2023-07-21T18:37:39.372 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:37:39.335+0000 7f0b4bfff700 1 -- 172.21.15.155:0/941265199 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b5800fb80 con 0x7f0b640ffbd0 2023-07-21T18:37:39.372 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:37:39.335+0000 7f0b6942c700 1 -- 172.21.15.155:0/941265199 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b64072800 con 0x7f0b640ffbd0 2023-07-21T18:37:39.372 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:37:39.335+0000 7f0b4bfff700 1 -- 172.21.15.155:0/941265199 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b58017650 con 0x7f0b640ffbd0 2023-07-21T18:37:39.373 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:37:39.335+0000 7f0b6942c700 1 -- 172.21.15.155:0/941265199 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b64072ca0 con 0x7f0b640ffbd0 2023-07-21T18:37:39.373 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:37:39.336+0000 7f0b6942c700 1 -- 172.21.15.155:0/941265199 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b64107170 con 0x7f0b640ffbd0 2023-07-21T18:37:39.373 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:37:39.337+0000 7f0b4bfff700 1 -- 172.21.15.155:0/941265199 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f0b58022470 con 0x7f0b640ffbd0 2023-07-21T18:37:39.374 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:37:39.338+0000 7f0b4bfff700 1 --2- 172.21.15.155:0/941265199 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f0b4c0a1150 0x7f0b4c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:37:39.374 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:37:39.338+0000 7f0b4bfff700 1 -- 172.21.15.155:0/941265199 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 4965+0+0 (secure 0 0 0) 0x7f0b580cd160 con 0x7f0b640ffbd0 2023-07-21T18:37:39.374 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:37:39.338+0000 7f0b627fc700 1 --2- 172.21.15.155:0/941265199 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f0b4c0a1150 0x7f0b4c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:37:39.374 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:37:39.339+0000 7f0b627fc700 1 --2- 172.21.15.155:0/941265199 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f0b4c0a1150 0x7f0b4c0a3600 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0b54005950 tx=0x7f0b540058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:37:39.374 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:37:39.342+0000 7f0b4bfff700 1 -- 172.21.15.155:0/941265199 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f0b58097650 con 0x7f0b640ffbd0 2023-07-21T18:37:39.380 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:38 smithi193 ceph-mon[98204]: purged_snaps scrub starts 2023-07-21T18:37:39.380 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:38 smithi193 ceph-mon[98204]: purged_snaps scrub ok 2023-07-21T18:37:39.380 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:38 smithi193 ceph-mon[98204]: osd.6 [v2:172.21.15.39:6818/2417770529,v1:172.21.15.39:6819/2417770529] boot 2023-07-21T18:37:39.380 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:38 smithi193 ceph-mon[98204]: osdmap e43: 7 total, 7 up, 7 in 2023-07-21T18:37:39.380 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:38 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T18:37:39.448 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:38 smithi039 ceph-mon[96489]: purged_snaps scrub starts 2023-07-21T18:37:39.448 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:38 smithi039 ceph-mon[96489]: purged_snaps scrub ok 2023-07-21T18:37:39.448 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:38 smithi039 ceph-mon[96489]: osd.6 [v2:172.21.15.39:6818/2417770529,v1:172.21.15.39:6819/2417770529] boot 2023-07-21T18:37:39.449 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:38 smithi039 ceph-mon[96489]: osdmap e43: 7 total, 7 up, 7 in 2023-07-21T18:37:39.449 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:38 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T18:37:39.567 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:37:39.535+0000 7f0b6942c700 1 -- 172.21.15.155:0/941265199 --> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f0b64060950 con 0x7f0b4c0a1150 2023-07-21T18:37:40.380 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:40 smithi193 ceph-mon[98204]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-21T18:37:40.380 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:40 smithi193 ceph-mon[98204]: osdmap e44: 7 total, 7 up, 7 in 2023-07-21T18:37:40.380 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:40 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T18:37:40.380 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:40 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T18:37:40.381 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:40 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:37:40.381 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:40 smithi155 ceph-mon[97934]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-21T18:37:40.382 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:40 smithi155 ceph-mon[97934]: osdmap e44: 7 total, 7 up, 7 in 2023-07-21T18:37:40.382 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:40 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T18:37:40.382 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:40 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T18:37:40.383 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:40 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:37:40.448 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:40 smithi039 ceph-mon[96489]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-21T18:37:40.448 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:40 smithi039 ceph-mon[96489]: osdmap e44: 7 total, 7 up, 7 in 2023-07-21T18:37:40.448 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:40 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T18:37:40.449 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:40 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T18:37:40.449 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:40 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:37:41.380 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:41 smithi193 ceph-mon[98204]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:37:41.399 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:41 smithi155 ceph-mon[97934]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:37:41.447 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:41 smithi039 ceph-mon[96489]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:37:42.380 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:42 smithi193 ceph-mon[98204]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T18:37:42.380 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:42 smithi193 ceph-mon[98204]: from='client.? 172.21.15.155:0/2024810933' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a88ce30-1835-4254-90cc-804ed5cf1066"}]: dispatch 2023-07-21T18:37:42.380 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:42 smithi193 ceph-mon[98204]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a88ce30-1835-4254-90cc-804ed5cf1066"}]: dispatch 2023-07-21T18:37:42.380 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:42 smithi193 ceph-mon[98204]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5a88ce30-1835-4254-90cc-804ed5cf1066"}]': finished 2023-07-21T18:37:42.381 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:42 smithi193 ceph-mon[98204]: osdmap e45: 8 total, 7 up, 8 in 2023-07-21T18:37:42.381 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:42 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T18:37:42.398 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:42 smithi155 ceph-mon[97934]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T18:37:42.398 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:42 smithi155 ceph-mon[97934]: from='client.? 172.21.15.155:0/2024810933' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a88ce30-1835-4254-90cc-804ed5cf1066"}]: dispatch 2023-07-21T18:37:42.399 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:42 smithi155 ceph-mon[97934]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a88ce30-1835-4254-90cc-804ed5cf1066"}]: dispatch 2023-07-21T18:37:42.399 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:42 smithi155 ceph-mon[97934]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5a88ce30-1835-4254-90cc-804ed5cf1066"}]': finished 2023-07-21T18:37:42.399 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:42 smithi155 ceph-mon[97934]: osdmap e45: 8 total, 7 up, 8 in 2023-07-21T18:37:42.399 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:42 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T18:37:42.447 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:42 smithi039 ceph-mon[96489]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T18:37:42.448 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:42 smithi039 ceph-mon[96489]: from='client.? 172.21.15.155:0/2024810933' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a88ce30-1835-4254-90cc-804ed5cf1066"}]: dispatch 2023-07-21T18:37:42.448 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:42 smithi039 ceph-mon[96489]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a88ce30-1835-4254-90cc-804ed5cf1066"}]: dispatch 2023-07-21T18:37:42.448 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:42 smithi039 ceph-mon[96489]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5a88ce30-1835-4254-90cc-804ed5cf1066"}]': finished 2023-07-21T18:37:42.448 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:42 smithi039 ceph-mon[96489]: osdmap e45: 8 total, 7 up, 8 in 2023-07-21T18:37:42.448 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:42 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T18:37:43.398 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:43 smithi155 ceph-mon[97934]: from='client.? 172.21.15.155:0/1586446439' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T18:37:43.447 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:43 smithi039 ceph-mon[96489]: from='client.? 172.21.15.155:0/1586446439' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T18:37:43.630 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:43 smithi193 ceph-mon[98204]: from='client.? 172.21.15.155:0/1586446439' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T18:37:44.398 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:44 smithi155 ceph-mon[97934]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T18:37:44.447 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:44 smithi039 ceph-mon[96489]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T18:37:44.630 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:44 smithi193 ceph-mon[98204]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T18:37:46.398 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:46 smithi155 ceph-mon[97934]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T18:37:46.447 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:46 smithi039 ceph-mon[96489]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T18:37:46.629 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:46 smithi193 ceph-mon[98204]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T18:37:47.317 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:47 smithi155 ceph-mon[97934]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T18:37:47.629 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:47 smithi193 ceph-mon[98204]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T18:37:47.697 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:47 smithi039 ceph-mon[96489]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T18:37:48.879 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:48 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-07-21T18:37:48.880 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:48 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:37:48.898 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:48 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-07-21T18:37:48.898 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:48 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:37:48.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:48 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-07-21T18:37:48.948 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:48 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:37:49.795 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:49 smithi155 ceph-mon[97934]: Deploying daemon osd.7 on smithi155 2023-07-21T18:37:49.795 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:49 smithi155 ceph-mon[97934]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T18:37:49.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:49 smithi039 ceph-mon[96489]: Deploying daemon osd.7 on smithi155 2023-07-21T18:37:49.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:49 smithi039 ceph-mon[96489]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T18:37:50.129 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:49 smithi193 ceph-mon[98204]: Deploying daemon osd.7 on smithi155 2023-07-21T18:37:50.129 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:49 smithi193 ceph-mon[98204]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T18:37:52.379 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:51 smithi193 ceph-mon[98204]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T18:37:52.398 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:51 smithi155 ceph-mon[97934]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T18:37:52.447 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:51 smithi039 ceph-mon[96489]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T18:37:53.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:52 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:37:53.379 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:52 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:37:53.447 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:52 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:37:54.124 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:53 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:54.125 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:53 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:54.125 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:53 smithi155 ceph-mon[97934]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T18:37:54.379 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:53 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:54.379 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:53 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:54.379 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:53 smithi193 ceph-mon[98204]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T18:37:54.447 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:53 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:54.447 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:53 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:54.447 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:53 smithi039 ceph-mon[96489]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T18:37:55.499 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:55 smithi155 ceph-mon[97934]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T18:37:55.629 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:55 smithi193 ceph-mon[98204]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T18:37:55.697 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:55 smithi039 ceph-mon[96489]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T18:37:55.838 INFO:teuthology.orchestra.run.smithi155.stdout:Created osd(s) 7 on host 'smithi155' 2023-07-21T18:37:55.838 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:37:55.806+0000 7f0b4bfff700 1 -- 172.21.15.155:0/941265199 <== mgr.14152 v2:172.21.15.39:6800/2116291712 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f0b64060950 con 0x7f0b4c0a1150 2023-07-21T18:37:55.840 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:37:55.808+0000 7f0b6942c700 1 -- 172.21.15.155:0/941265199 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f0b4c0a1150 msgr2=0x7f0b4c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:37:55.840 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:37:55.808+0000 7f0b6942c700 1 --2- 172.21.15.155:0/941265199 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f0b4c0a1150 0x7f0b4c0a3600 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0b54005950 tx=0x7f0b540058e0 comp rx=0 tx=0).stop 2023-07-21T18:37:55.840 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:37:55.808+0000 7f0b6942c700 1 -- 172.21.15.155:0/941265199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b640ffbd0 msgr2=0x7f0b641958a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:37:55.840 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:37:55.808+0000 7f0b6942c700 1 --2- 172.21.15.155:0/941265199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b640ffbd0 0x7f0b641958a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0b580095c0 tx=0x7f0b5800f690 comp rx=0 tx=0).stop 2023-07-21T18:37:55.841 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:37:55.809+0000 7f0b6942c700 1 -- 172.21.15.155:0/941265199 shutdown_connections 2023-07-21T18:37:55.841 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:37:55.809+0000 7f0b6942c700 1 --2- 172.21.15.155:0/941265199 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f0b4c0a1150 0x7f0b4c0a3600 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:37:55.841 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:37:55.809+0000 7f0b6942c700 1 --2- 172.21.15.155:0/941265199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b640ffbd0 0x7f0b641958a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:37:55.841 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:37:55.809+0000 7f0b6942c700 1 --2- 172.21.15.155:0/941265199 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b64101760 0x7f0b6419ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:37:55.841 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:37:55.809+0000 7f0b6942c700 1 --2- 172.21.15.155:0/941265199 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0b64100dd0 0x7f0b64195de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:37:55.842 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:37:55.809+0000 7f0b6942c700 1 -- 172.21.15.155:0/941265199 >> 172.21.15.155:0/941265199 conn(0x7f0b640fb220 msgr2=0x7f0b641042b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:37:55.842 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:37:55.809+0000 7f0b6942c700 1 -- 172.21.15.155:0/941265199 shutdown_connections 2023-07-21T18:37:55.842 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:37:55.809+0000 7f0b6942c700 1 -- 172.21.15.155:0/941265199 wait complete. 2023-07-21T18:37:56.211 DEBUG:teuthology.orchestra.run.smithi155:osd.7> sudo journalctl -f -n 0 -u ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@osd.7.service 2023-07-21T18:37:56.213 INFO:tasks.cephadm:Deploying osd.8 on smithi193 with /dev/vg_nvme/lv_2... 2023-07-21T18:37:56.214 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-07-21T18:37:56.247 INFO:journalctl@ceph.osd.7.smithi155.stdout:-- Logs begin at Fri 2023-07-21 18:17:11 UTC. -- 2023-07-21T18:37:56.629 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:56 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:56.629 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:56 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:56.629 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:56 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:37:56.629 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:56 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:37:56.630 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:56 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:56.630 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:56 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:56.630 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:56 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:56.630 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:56 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:37:56.757 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:56 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:56.758 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:56 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:56.758 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:56 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:37:56.758 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:56 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:37:56.758 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:56 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:56.758 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:56 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:56.758 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:56 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:56.759 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:56 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:37:56.785 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T18:37:56.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:56 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:56.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:56 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:56.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:56 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:37:56.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:56 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:37:56.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:56 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:56.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:56 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:56.948 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:56 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:56.948 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:56 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:37:57.148 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 21 18:37:56 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-7[114007]: 2023-07-21T18:37:56.863+0000 7fceee0bd640 -1 osd.7 0 log_to_monitors true 2023-07-21T18:37:57.629 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:57 smithi193 ceph-mon[98204]: from='osd.7 [v2:172.21.15.155:6816/2542002111,v1:172.21.15.155:6817/2542002111]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-21T18:37:57.630 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:57 smithi193 ceph-mon[98204]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-21T18:37:57.630 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:57 smithi193 ceph-mon[98204]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T18:37:57.898 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:57 smithi155 ceph-mon[97934]: from='osd.7 [v2:172.21.15.155:6816/2542002111,v1:172.21.15.155:6817/2542002111]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-21T18:37:57.898 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:57 smithi155 ceph-mon[97934]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-21T18:37:57.899 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:57 smithi155 ceph-mon[97934]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T18:37:57.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:57 smithi039 ceph-mon[96489]: from='osd.7 [v2:172.21.15.155:6816/2542002111,v1:172.21.15.155:6817/2542002111]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-21T18:37:57.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:57 smithi039 ceph-mon[96489]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-21T18:37:57.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:57 smithi039 ceph-mon[96489]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T18:37:58.233 INFO:teuthology.orchestra.run.smithi193.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-07-21T18:37:58.234 INFO:teuthology.orchestra.run.smithi193.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-07-21T18:37:58.234 INFO:teuthology.orchestra.run.smithi193.stderr: stderr: 10+0 records in 2023-07-21T18:37:58.234 INFO:teuthology.orchestra.run.smithi193.stderr:10+0 records out 2023-07-21T18:37:58.234 INFO:teuthology.orchestra.run.smithi193.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0127968 s, 819 MB/s 2023-07-21T18:37:58.234 INFO:teuthology.orchestra.run.smithi193.stderr:--> Zapping successful for: 2023-07-21T18:37:58.587 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:58 smithi193 ceph-mon[98204]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-07-21T18:37:58.587 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:58 smithi193 ceph-mon[98204]: osdmap e46: 8 total, 7 up, 8 in 2023-07-21T18:37:58.587 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:58 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T18:37:58.587 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:58 smithi193 ceph-mon[98204]: from='osd.7 [v2:172.21.15.155:6816/2542002111,v1:172.21.15.155:6817/2542002111]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2023-07-21T18:37:58.588 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:58 smithi193 ceph-mon[98204]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2023-07-21T18:37:58.588 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:58 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:58.588 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:58 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:58.588 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:58 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:37:58.588 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:58 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:37:58.588 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:58 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:37:58.588 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:58 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:58.589 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:58 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:37:58.589 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:58 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:37:58.589 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:58 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:58.624 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi193:vg_nvme/lv_2 2023-07-21T18:37:58.648 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 21 18:37:58 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-7[114007]: 2023-07-21T18:37:58.479+0000 7fcee2f34700 -1 osd.7 0 waiting for initial osdmap 2023-07-21T18:37:58.648 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 21 18:37:58 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-7[114007]: 2023-07-21T18:37:58.491+0000 7fcedd526700 -1 osd.7 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-21T18:37:58.649 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:58 smithi155 ceph-mon[97934]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-07-21T18:37:58.649 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:58 smithi155 ceph-mon[97934]: osdmap e46: 8 total, 7 up, 8 in 2023-07-21T18:37:58.649 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:58 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T18:37:58.649 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:58 smithi155 ceph-mon[97934]: from='osd.7 [v2:172.21.15.155:6816/2542002111,v1:172.21.15.155:6817/2542002111]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2023-07-21T18:37:58.650 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:58 smithi155 ceph-mon[97934]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2023-07-21T18:37:58.650 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:58 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:58.650 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:58 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:58.650 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:58 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:37:58.651 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:58 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:37:58.651 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:58 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:37:58.651 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:58 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:58.651 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:58 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:37:58.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:58 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:37:58.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:58 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:58.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:58 smithi039 ceph-mon[96489]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-07-21T18:37:58.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:58 smithi039 ceph-mon[96489]: osdmap e46: 8 total, 7 up, 8 in 2023-07-21T18:37:58.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:58 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T18:37:58.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:58 smithi039 ceph-mon[96489]: from='osd.7 [v2:172.21.15.155:6816/2542002111,v1:172.21.15.155:6817/2542002111]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2023-07-21T18:37:58.948 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:58 smithi039 ceph-mon[96489]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2023-07-21T18:37:58.948 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:58 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:58.948 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:58 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:58.948 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:58 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:37:58.948 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:58 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:37:58.948 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:58 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:37:58.948 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:58 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:58.949 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:58 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:37:58.949 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:58 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:37:58.949 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:58 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:37:59.175 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T18:37:59.629 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:59 smithi193 ceph-mon[98204]: Detected new or changed devices on smithi155 2023-07-21T18:37:59.629 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:59 smithi193 ceph-mon[98204]: Adjusting osd_memory_target on smithi155 to 5722M 2023-07-21T18:37:59.629 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:59 smithi193 ceph-mon[98204]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2023-07-21T18:37:59.629 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:59 smithi193 ceph-mon[98204]: osdmap e47: 8 total, 7 up, 8 in 2023-07-21T18:37:59.630 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:59 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T18:37:59.630 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:59 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T18:37:59.630 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:37:59 smithi193 ceph-mon[98204]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T18:37:59.898 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:59 smithi155 ceph-mon[97934]: Detected new or changed devices on smithi155 2023-07-21T18:37:59.898 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:59 smithi155 ceph-mon[97934]: Adjusting osd_memory_target on smithi155 to 5722M 2023-07-21T18:37:59.898 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:59 smithi155 ceph-mon[97934]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2023-07-21T18:37:59.898 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:59 smithi155 ceph-mon[97934]: osdmap e47: 8 total, 7 up, 8 in 2023-07-21T18:37:59.898 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:59 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T18:37:59.899 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:59 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T18:37:59.899 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:37:59 smithi155 ceph-mon[97934]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T18:37:59.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:59 smithi039 ceph-mon[96489]: Detected new or changed devices on smithi155 2023-07-21T18:37:59.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:59 smithi039 ceph-mon[96489]: Adjusting osd_memory_target on smithi155 to 5722M 2023-07-21T18:37:59.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:59 smithi039 ceph-mon[96489]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2023-07-21T18:37:59.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:59 smithi039 ceph-mon[96489]: osdmap e47: 8 total, 7 up, 8 in 2023-07-21T18:37:59.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:59 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T18:37:59.948 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:59 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T18:37:59.948 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:37:59 smithi039 ceph-mon[96489]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T18:38:00.388 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:38:00.356+0000 7f9c0f393700 1 -- 172.21.15.193:0/723135971 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c080fed20 msgr2=0x7f9c080ff170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:38:00.388 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:38:00.356+0000 7f9c0f393700 1 --2- 172.21.15.193:0/723135971 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c080fed20 0x7f9c080ff170 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9c04009990 tx=0x7f9c04009ca0 comp rx=0 tx=0).stop 2023-07-21T18:38:00.388 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:38:00.356+0000 7f9c0f393700 1 -- 172.21.15.193:0/723135971 shutdown_connections 2023-07-21T18:38:00.388 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:38:00.356+0000 7f9c0f393700 1 --2- 172.21.15.193:0/723135971 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c08100070 0x7f9c08100480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:38:00.388 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:38:00.356+0000 7f9c0f393700 1 --2- 172.21.15.193:0/723135971 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c080ff6b0 0x7f9c08106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:38:00.389 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:38:00.356+0000 7f9c0f393700 1 --2- 172.21.15.193:0/723135971 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c080fed20 0x7f9c080ff170 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:38:00.389 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:38:00.356+0000 7f9c0f393700 1 -- 172.21.15.193:0/723135971 >> 172.21.15.193:0/723135971 conn(0x7f9c080f9860 msgr2=0x7f9c080fbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:38:00.389 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:38:00.356+0000 7f9c0f393700 1 -- 172.21.15.193:0/723135971 shutdown_connections 2023-07-21T18:38:00.389 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:38:00.356+0000 7f9c0f393700 1 -- 172.21.15.193:0/723135971 wait complete. 2023-07-21T18:38:00.391 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:38:00.357+0000 7f9c0f393700 1 Processor -- start 2023-07-21T18:38:00.391 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:38:00.357+0000 7f9c0f393700 1 -- start start 2023-07-21T18:38:00.391 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:38:00.357+0000 7f9c0f393700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c080fed20 0x7f9c08194790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:38:00.391 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:38:00.357+0000 7f9c0f393700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c080ff6b0 0x7f9c08194cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:38:00.391 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:38:00.357+0000 7f9c0f393700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c08100070 0x7f9c0819bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:38:00.391 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:38:00.357+0000 7f9c0f393700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0819c320 con 0x7f9c08100070 2023-07-21T18:38:00.391 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:38:00.357+0000 7f9c0f393700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0819c460 con 0x7f9c080fed20 2023-07-21T18:38:00.392 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:38:00.357+0000 7f9c0f393700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0819c5f0 con 0x7f9c080ff6b0 2023-07-21T18:38:00.392 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:38:00.357+0000 7f9c0c92e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c080ff6b0 0x7f9c08194cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:38:00.392 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:38:00.357+0000 7f9c0c92e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c080ff6b0 0x7f9c08194cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:50762/0 (socket says 172.21.15.193:50762) 2023-07-21T18:38:00.392 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:38:00.357+0000 7f9c0c92e700 1 -- 172.21.15.193:0/3870237955 learned_addr learned my addr 172.21.15.193:0/3870237955 (peer_addr_for_me v2:172.21.15.193:0/0) 2023-07-21T18:38:00.392 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:38:00.357+0000 7f9c0d930700 1 --2- 172.21.15.193:0/3870237955 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c08100070 0x7f9c0819bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:38:00.392 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:38:00.357+0000 7f9c0d12f700 1 --2- 172.21.15.193:0/3870237955 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c080fed20 0x7f9c08194790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:38:00.392 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:38:00.357+0000 7f9c0c92e700 1 -- 172.21.15.193:0/3870237955 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c080fed20 msgr2=0x7f9c08194790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:38:00.393 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:38:00.357+0000 7f9c0c92e700 1 --2- 172.21.15.193:0/3870237955 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c080fed20 0x7f9c08194790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:38:00.393 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:38:00.357+0000 7f9c0c92e700 1 -- 172.21.15.193:0/3870237955 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c08100070 msgr2=0x7f9c0819bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:38:00.393 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:38:00.357+0000 7f9c0c92e700 1 --2- 172.21.15.193:0/3870237955 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c08100070 0x7f9c0819bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:38:00.393 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:38:00.357+0000 7f9c0c92e700 1 -- 172.21.15.193:0/3870237955 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c040095f0 con 0x7f9c080ff6b0 2023-07-21T18:38:00.393 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:38:00.358+0000 7f9c0c92e700 1 --2- 172.21.15.193:0/3870237955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c080ff6b0 0x7f9c08194cd0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9c04005910 tx=0x7f9c0400f710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:38:00.393 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:38:00.358+0000 7f9bfe7fc700 1 -- 172.21.15.193:0/3870237955 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0401d070 con 0x7f9c080ff6b0 2023-07-21T18:38:00.393 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:38:00.358+0000 7f9bfe7fc700 1 -- 172.21.15.193:0/3870237955 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c04022470 con 0x7f9c080ff6b0 2023-07-21T18:38:00.393 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:38:00.358+0000 7f9c0f393700 1 -- 172.21.15.193:0/3870237955 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c0819c870 con 0x7f9c080ff6b0 2023-07-21T18:38:00.394 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:38:00.358+0000 7f9bfe7fc700 1 -- 172.21.15.193:0/3870237955 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c04017650 con 0x7f9c080ff6b0 2023-07-21T18:38:00.394 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:38:00.358+0000 7f9c0f393700 1 -- 172.21.15.193:0/3870237955 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c0819cd10 con 0x7f9c080ff6b0 2023-07-21T18:38:00.394 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:38:00.359+0000 7f9c0f393700 1 -- 172.21.15.193:0/3870237955 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c0810a250 con 0x7f9c080ff6b0 2023-07-21T18:38:00.395 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:38:00.360+0000 7f9bfe7fc700 1 -- 172.21.15.193:0/3870237955 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f9c040225e0 con 0x7f9c080ff6b0 2023-07-21T18:38:00.395 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:38:00.360+0000 7f9bfe7fc700 1 --2- 172.21.15.193:0/3870237955 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f9bf40a1150 0x7f9bf40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:38:00.395 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:38:00.360+0000 7f9bfe7fc700 1 -- 172.21.15.193:0/3870237955 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5430+0+0 (secure 0 0 0) 0x7f9c040cd3c0 con 0x7f9c080ff6b0 2023-07-21T18:38:00.395 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:38:00.361+0000 7f9c0d12f700 1 --2- 172.21.15.193:0/3870237955 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f9bf40a1150 0x7f9bf40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:38:00.395 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:38:00.361+0000 7f9c0d12f700 1 --2- 172.21.15.193:0/3870237955 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f9bf40a1150 0x7f9bf40a3600 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9c08067d90 tx=0x7f9bf800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:38:00.396 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:38:00.364+0000 7f9bfe7fc700 1 -- 172.21.15.193:0/3870237955 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f9c040976e0 con 0x7f9c080ff6b0 2023-07-21T18:38:00.591 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:38:00.560+0000 7f9c0f393700 1 -- 172.21.15.193:0/3870237955 --> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f9c080fe580 con 0x7f9bf40a1150 2023-07-21T18:38:00.879 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:00 smithi193 ceph-mon[98204]: purged_snaps scrub starts 2023-07-21T18:38:00.880 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:00 smithi193 ceph-mon[98204]: purged_snaps scrub ok 2023-07-21T18:38:00.880 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:00 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T18:38:00.880 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:00 smithi193 ceph-mon[98204]: osd.7 [v2:172.21.15.155:6816/2542002111,v1:172.21.15.155:6817/2542002111] boot 2023-07-21T18:38:00.880 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:00 smithi193 ceph-mon[98204]: osdmap e48: 8 total, 8 up, 8 in 2023-07-21T18:38:00.880 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:00 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T18:38:00.897 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:00 smithi155 ceph-mon[97934]: purged_snaps scrub starts 2023-07-21T18:38:00.898 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:00 smithi155 ceph-mon[97934]: purged_snaps scrub ok 2023-07-21T18:38:00.898 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:00 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T18:38:00.898 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:00 smithi155 ceph-mon[97934]: osd.7 [v2:172.21.15.155:6816/2542002111,v1:172.21.15.155:6817/2542002111] boot 2023-07-21T18:38:00.898 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:00 smithi155 ceph-mon[97934]: osdmap e48: 8 total, 8 up, 8 in 2023-07-21T18:38:00.898 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:00 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T18:38:00.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:00 smithi039 ceph-mon[96489]: purged_snaps scrub starts 2023-07-21T18:38:00.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:00 smithi039 ceph-mon[96489]: purged_snaps scrub ok 2023-07-21T18:38:00.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:00 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T18:38:00.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:00 smithi039 ceph-mon[96489]: osd.7 [v2:172.21.15.155:6816/2542002111,v1:172.21.15.155:6817/2542002111] boot 2023-07-21T18:38:00.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:00 smithi039 ceph-mon[96489]: osdmap e48: 8 total, 8 up, 8 in 2023-07-21T18:38:00.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:00 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T18:38:01.629 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:01 smithi193 ceph-mon[98204]: osdmap e49: 8 total, 8 up, 8 in 2023-07-21T18:38:01.630 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:01 smithi193 ceph-mon[98204]: from='client.24289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:38:01.630 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:01 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T18:38:01.630 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:01 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T18:38:01.630 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:01 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:38:01.630 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:01 smithi193 ceph-mon[98204]: pgmap v156: 1 pgs: 1 remapped+peering; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T18:38:01.631 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:01 smithi193 ceph-mon[98204]: osdmap e50: 8 total, 8 up, 8 in 2023-07-21T18:38:01.897 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:01 smithi155 ceph-mon[97934]: osdmap e49: 8 total, 8 up, 8 in 2023-07-21T18:38:01.898 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:01 smithi155 ceph-mon[97934]: from='client.24289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:38:01.898 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:01 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T18:38:01.898 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:01 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T18:38:01.898 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:01 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:38:01.898 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:01 smithi155 ceph-mon[97934]: pgmap v156: 1 pgs: 1 remapped+peering; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T18:38:01.898 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:01 smithi155 ceph-mon[97934]: osdmap e50: 8 total, 8 up, 8 in 2023-07-21T18:38:01.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:01 smithi039 ceph-mon[96489]: osdmap e49: 8 total, 8 up, 8 in 2023-07-21T18:38:01.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:01 smithi039 ceph-mon[96489]: from='client.24289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:38:01.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:01 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T18:38:01.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:01 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T18:38:01.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:01 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:38:01.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:01 smithi039 ceph-mon[96489]: pgmap v156: 1 pgs: 1 remapped+peering; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T18:38:01.948 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:01 smithi039 ceph-mon[96489]: osdmap e50: 8 total, 8 up, 8 in 2023-07-21T18:38:03.878 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:03 smithi193 ceph-mon[98204]: osdmap e51: 8 total, 8 up, 8 in 2023-07-21T18:38:03.879 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:03 smithi193 ceph-mon[98204]: from='client.? 172.21.15.193:0/1155005778' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d964a183-3021-4ace-9a0c-aad00f2570e4"}]: dispatch 2023-07-21T18:38:03.879 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:03 smithi193 ceph-mon[98204]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d964a183-3021-4ace-9a0c-aad00f2570e4"}]: dispatch 2023-07-21T18:38:03.879 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:03 smithi193 ceph-mon[98204]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d964a183-3021-4ace-9a0c-aad00f2570e4"}]': finished 2023-07-21T18:38:03.879 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:03 smithi193 ceph-mon[98204]: osdmap e52: 9 total, 8 up, 9 in 2023-07-21T18:38:03.879 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:03 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T18:38:03.880 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:03 smithi193 ceph-mon[98204]: pgmap v160: 1 pgs: 1 remapped+peering; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T18:38:03.880 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:03 smithi193 ceph-mon[98204]: from='client.? 172.21.15.193:0/250134091' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T18:38:03.897 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:03 smithi155 ceph-mon[97934]: osdmap e51: 8 total, 8 up, 8 in 2023-07-21T18:38:03.897 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:03 smithi155 ceph-mon[97934]: from='client.? 172.21.15.193:0/1155005778' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d964a183-3021-4ace-9a0c-aad00f2570e4"}]: dispatch 2023-07-21T18:38:03.898 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:03 smithi155 ceph-mon[97934]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d964a183-3021-4ace-9a0c-aad00f2570e4"}]: dispatch 2023-07-21T18:38:03.898 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:03 smithi155 ceph-mon[97934]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d964a183-3021-4ace-9a0c-aad00f2570e4"}]': finished 2023-07-21T18:38:03.898 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:03 smithi155 ceph-mon[97934]: osdmap e52: 9 total, 8 up, 9 in 2023-07-21T18:38:03.898 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:03 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T18:38:03.898 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:03 smithi155 ceph-mon[97934]: pgmap v160: 1 pgs: 1 remapped+peering; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T18:38:03.898 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:03 smithi155 ceph-mon[97934]: from='client.? 172.21.15.193:0/250134091' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T18:38:03.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:03 smithi039 ceph-mon[96489]: osdmap e51: 8 total, 8 up, 8 in 2023-07-21T18:38:03.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:03 smithi039 ceph-mon[96489]: from='client.? 172.21.15.193:0/1155005778' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d964a183-3021-4ace-9a0c-aad00f2570e4"}]: dispatch 2023-07-21T18:38:03.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:03 smithi039 ceph-mon[96489]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d964a183-3021-4ace-9a0c-aad00f2570e4"}]: dispatch 2023-07-21T18:38:03.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:03 smithi039 ceph-mon[96489]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d964a183-3021-4ace-9a0c-aad00f2570e4"}]': finished 2023-07-21T18:38:03.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:03 smithi039 ceph-mon[96489]: osdmap e52: 9 total, 8 up, 9 in 2023-07-21T18:38:03.948 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:03 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T18:38:03.948 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:03 smithi039 ceph-mon[96489]: pgmap v160: 1 pgs: 1 remapped+peering; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T18:38:03.948 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:03 smithi039 ceph-mon[96489]: from='client.? 172.21.15.193:0/250134091' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T18:38:05.628 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:05 smithi193 ceph-mon[98204]: pgmap v161: 1 pgs: 1 remapped+peering; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T18:38:05.647 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:05 smithi155 ceph-mon[97934]: pgmap v161: 1 pgs: 1 remapped+peering; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T18:38:05.696 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:05 smithi039 ceph-mon[96489]: pgmap v161: 1 pgs: 1 remapped+peering; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T18:38:07.628 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:07 smithi193 ceph-mon[98204]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-07-21T18:38:07.647 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:07 smithi155 ceph-mon[97934]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-07-21T18:38:07.696 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:07 smithi039 ceph-mon[96489]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-07-21T18:38:09.479 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:09 smithi193 ceph-mon[98204]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T18:38:09.647 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:09 smithi155 ceph-mon[97934]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T18:38:09.696 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:09 smithi039 ceph-mon[96489]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T18:38:10.362 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:10 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-07-21T18:38:10.362 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:10 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:38:10.362 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:10 smithi193 ceph-mon[98204]: Deploying daemon osd.8 on smithi193 2023-07-21T18:38:10.647 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:10 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-07-21T18:38:10.682 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:10 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:38:10.682 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:10 smithi155 ceph-mon[97934]: Deploying daemon osd.8 on smithi193 2023-07-21T18:38:10.696 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:10 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-07-21T18:38:10.696 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:10 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:38:10.696 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:10 smithi039 ceph-mon[96489]: Deploying daemon osd.8 on smithi193 2023-07-21T18:38:12.128 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:11 smithi193 ceph-mon[98204]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T18:38:12.397 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:11 smithi155 ceph-mon[97934]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T18:38:12.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:11 smithi039 ceph-mon[96489]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T18:38:13.553 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:13 smithi193 ceph-mon[98204]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T18:38:13.647 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:13 smithi155 ceph-mon[97934]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T18:38:13.696 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:13 smithi039 ceph-mon[96489]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T18:38:14.737 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:14 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:14.737 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:14 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:38:14.737 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:14 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:14.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:14 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:14.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:14 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:38:14.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:14 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:15.147 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:14 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:15.147 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:14 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:38:15.147 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:14 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:15.878 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:15 smithi193 ceph-mon[98204]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T18:38:15.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:15 smithi039 ceph-mon[96489]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T18:38:16.147 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:15 smithi155 ceph-mon[97934]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T18:38:16.404 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:38:16.372+0000 7f9bfe7fc700 1 -- 172.21.15.193:0/3870237955 <== mgr.14152 v2:172.21.15.39:6800/2116291712 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f9c080fe580 con 0x7f9bf40a1150 2023-07-21T18:38:16.405 INFO:teuthology.orchestra.run.smithi193.stdout:Created osd(s) 8 on host 'smithi193' 2023-07-21T18:38:16.406 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:38:16.375+0000 7f9c0f393700 1 -- 172.21.15.193:0/3870237955 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f9bf40a1150 msgr2=0x7f9bf40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:38:16.406 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:38:16.375+0000 7f9c0f393700 1 --2- 172.21.15.193:0/3870237955 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f9bf40a1150 0x7f9bf40a3600 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9c08067d90 tx=0x7f9bf800b410 comp rx=0 tx=0).stop 2023-07-21T18:38:16.406 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:38:16.375+0000 7f9c0f393700 1 -- 172.21.15.193:0/3870237955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c080ff6b0 msgr2=0x7f9c08194cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:38:16.406 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:38:16.375+0000 7f9c0f393700 1 --2- 172.21.15.193:0/3870237955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c080ff6b0 0x7f9c08194cd0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9c04005910 tx=0x7f9c0400f710 comp rx=0 tx=0).stop 2023-07-21T18:38:16.406 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:38:16.375+0000 7f9c0f393700 1 -- 172.21.15.193:0/3870237955 shutdown_connections 2023-07-21T18:38:16.407 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:38:16.375+0000 7f9c0f393700 1 --2- 172.21.15.193:0/3870237955 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f9bf40a1150 0x7f9bf40a3600 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:38:16.407 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:38:16.375+0000 7f9c0f393700 1 --2- 172.21.15.193:0/3870237955 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c080fed20 0x7f9c08194790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:38:16.407 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:38:16.375+0000 7f9c0f393700 1 --2- 172.21.15.193:0/3870237955 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c08100070 0x7f9c0819bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:38:16.407 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:38:16.375+0000 7f9c0f393700 1 --2- 172.21.15.193:0/3870237955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c080ff6b0 0x7f9c08194cd0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:38:16.407 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:38:16.375+0000 7f9c0f393700 1 -- 172.21.15.193:0/3870237955 >> 172.21.15.193:0/3870237955 conn(0x7f9c080f9860 msgr2=0x7f9c080fbac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:38:16.407 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:38:16.375+0000 7f9c0f393700 1 -- 172.21.15.193:0/3870237955 shutdown_connections 2023-07-21T18:38:16.408 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:38:16.375+0000 7f9c0f393700 1 -- 172.21.15.193:0/3870237955 wait complete. 2023-07-21T18:38:16.746 DEBUG:teuthology.orchestra.run.smithi193:osd.8> sudo journalctl -f -n 0 -u ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@osd.8.service 2023-07-21T18:38:16.748 INFO:tasks.cephadm:Deploying osd.9 on smithi039 with /dev/vg_nvme/lv_1... 2023-07-21T18:38:16.749 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-07-21T18:38:16.779 INFO:journalctl@ceph.osd.8.smithi193.stdout:-- Logs begin at Fri 2023-07-21 18:17:09 UTC. -- 2023-07-21T18:38:17.327 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:38:17.378 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:17 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:17.378 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:17 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:17.378 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:17 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:38:17.378 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:17 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:38:17.379 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:17 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:17.379 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:17 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:17.379 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:17 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:17.379 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:17 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:38:17.397 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:17 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:17.397 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:17 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:17.397 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:17 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:38:17.398 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:17 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:38:17.398 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:17 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:17.398 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:17 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:17.398 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:17 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:17.398 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:17 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:38:17.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:17 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:17.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:17 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:17.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:17 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:38:17.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:17 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:38:17.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:17 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:17.447 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:17 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:17.447 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:17 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:17.447 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:17 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:38:18.060 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 18:38:17 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T18:38:17.679+0000 7feac5663640 -1 osd.8 0 log_to_monitors true 2023-07-21T18:38:18.378 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:18 smithi193 ceph-mon[98204]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T18:38:18.378 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:18 smithi193 ceph-mon[98204]: from='osd.8 [v2:172.21.15.193:6816/2985835085,v1:172.21.15.193:6817/2985835085]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-21T18:38:18.397 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:18 smithi155 ceph-mon[97934]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T18:38:18.397 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:18 smithi155 ceph-mon[97934]: from='osd.8 [v2:172.21.15.193:6816/2985835085,v1:172.21.15.193:6817/2985835085]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-21T18:38:18.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:18 smithi039 ceph-mon[96489]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T18:38:18.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:18 smithi039 ceph-mon[96489]: from='osd.8 [v2:172.21.15.193:6816/2985835085,v1:172.21.15.193:6817/2985835085]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-21T18:38:18.714 INFO:teuthology.orchestra.run.smithi039.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-07-21T18:38:18.715 INFO:teuthology.orchestra.run.smithi039.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-07-21T18:38:18.715 INFO:teuthology.orchestra.run.smithi039.stderr: stderr: 10+0 records in 2023-07-21T18:38:18.715 INFO:teuthology.orchestra.run.smithi039.stderr:10+0 records out 2023-07-21T18:38:18.715 INFO:teuthology.orchestra.run.smithi039.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00971801 s, 1.1 GB/s 2023-07-21T18:38:18.715 INFO:teuthology.orchestra.run.smithi039.stderr: stderr: 2023-07-21T18:38:18.715 INFO:teuthology.orchestra.run.smithi039.stderr:--> Zapping successful for: 2023-07-21T18:38:19.089 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi039:vg_nvme/lv_1 2023-07-21T18:38:19.378 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 18:38:19 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T18:38:19.043+0000 7feaba4da700 -1 osd.8 0 waiting for initial osdmap 2023-07-21T18:38:19.378 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 18:38:19 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T18:38:19.056+0000 7feab5ace700 -1 osd.8 54 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-21T18:38:19.378 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:19 smithi193 ceph-mon[98204]: from='osd.8 [v2:172.21.15.193:6816/2985835085,v1:172.21.15.193:6817/2985835085]' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-07-21T18:38:19.379 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:19 smithi193 ceph-mon[98204]: osdmap e53: 9 total, 8 up, 9 in 2023-07-21T18:38:19.379 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:19 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T18:38:19.379 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:19 smithi193 ceph-mon[98204]: from='osd.8 [v2:172.21.15.193:6816/2985835085,v1:172.21.15.193:6817/2985835085]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2023-07-21T18:38:19.379 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:19 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:19.379 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:19 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:19.379 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:19 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:38:19.379 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:19 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:38:19.379 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:19 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:38:19.380 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:19 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:19.380 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:19 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:38:19.380 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:19 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:38:19.380 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:19 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:19.397 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:19 smithi155 ceph-mon[97934]: from='osd.8 [v2:172.21.15.193:6816/2985835085,v1:172.21.15.193:6817/2985835085]' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-07-21T18:38:19.397 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:19 smithi155 ceph-mon[97934]: osdmap e53: 9 total, 8 up, 9 in 2023-07-21T18:38:19.397 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:19 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T18:38:19.397 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:19 smithi155 ceph-mon[97934]: from='osd.8 [v2:172.21.15.193:6816/2985835085,v1:172.21.15.193:6817/2985835085]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2023-07-21T18:38:19.397 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:19 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:19.398 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:19 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:19.398 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:19 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:38:19.398 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:19 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:38:19.398 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:19 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:38:19.398 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:19 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:19.398 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:19 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:38:19.398 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:19 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:38:19.399 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:19 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:19.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:19 smithi039 ceph-mon[96489]: from='osd.8 [v2:172.21.15.193:6816/2985835085,v1:172.21.15.193:6817/2985835085]' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-07-21T18:38:19.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:19 smithi039 ceph-mon[96489]: osdmap e53: 9 total, 8 up, 9 in 2023-07-21T18:38:19.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:19 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T18:38:19.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:19 smithi039 ceph-mon[96489]: from='osd.8 [v2:172.21.15.193:6816/2985835085,v1:172.21.15.193:6817/2985835085]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2023-07-21T18:38:19.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:19 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:19.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:19 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:19.447 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:19 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:38:19.447 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:19 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:38:19.447 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:19 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:38:19.447 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:19 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:19.447 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:19 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:38:19.448 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:19 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:38:19.448 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:19 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:19.647 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:38:20.110 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:20 smithi039 ceph-mon[96489]: Detected new or changed devices on smithi193 2023-07-21T18:38:20.110 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:20 smithi039 ceph-mon[96489]: Adjusting osd_memory_target on smithi193 to 5722M 2023-07-21T18:38:20.110 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:20 smithi039 ceph-mon[96489]: from='osd.8 [v2:172.21.15.193:6816/2985835085,v1:172.21.15.193:6817/2985835085]' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2023-07-21T18:38:20.110 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:20 smithi039 ceph-mon[96489]: osdmap e54: 9 total, 8 up, 9 in 2023-07-21T18:38:20.111 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:20 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T18:38:20.111 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:20 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T18:38:20.111 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:20 smithi039 ceph-mon[96489]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T18:38:20.378 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:20 smithi193 ceph-mon[98204]: Detected new or changed devices on smithi193 2023-07-21T18:38:20.378 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:20 smithi193 ceph-mon[98204]: Adjusting osd_memory_target on smithi193 to 5722M 2023-07-21T18:38:20.378 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:20 smithi193 ceph-mon[98204]: from='osd.8 [v2:172.21.15.193:6816/2985835085,v1:172.21.15.193:6817/2985835085]' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2023-07-21T18:38:20.379 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:20 smithi193 ceph-mon[98204]: osdmap e54: 9 total, 8 up, 9 in 2023-07-21T18:38:20.379 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:20 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T18:38:20.379 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:20 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T18:38:20.379 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:20 smithi193 ceph-mon[98204]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T18:38:20.396 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:20 smithi155 ceph-mon[97934]: Detected new or changed devices on smithi193 2023-07-21T18:38:20.397 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:20 smithi155 ceph-mon[97934]: Adjusting osd_memory_target on smithi193 to 5722M 2023-07-21T18:38:20.397 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:20 smithi155 ceph-mon[97934]: from='osd.8 [v2:172.21.15.193:6816/2985835085,v1:172.21.15.193:6817/2985835085]' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2023-07-21T18:38:20.397 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:20 smithi155 ceph-mon[97934]: osdmap e54: 9 total, 8 up, 9 in 2023-07-21T18:38:20.397 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:20 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T18:38:20.397 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:20 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T18:38:20.397 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:20 smithi155 ceph-mon[97934]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T18:38:20.821 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:38:20.788+0000 7f38e90f5700 1 -- 172.21.15.39:0/4094676393 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38e4100610 msgr2=0x7f38e4106ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:38:20.821 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:38:20.788+0000 7f38e90f5700 1 --2- 172.21.15.39:0/4094676393 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38e4100610 0x7f38e4106ed0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f38dc00b0a0 tx=0x7f38dc00b3b0 comp rx=0 tx=0).stop 2023-07-21T18:38:20.822 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:38:20.788+0000 7f38e90f5700 1 -- 172.21.15.39:0/4094676393 shutdown_connections 2023-07-21T18:38:20.822 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:38:20.788+0000 7f38e90f5700 1 --2- 172.21.15.39:0/4094676393 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38e4100610 0x7f38e4106ed0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:38:20.822 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:38:20.788+0000 7f38e90f5700 1 --2- 172.21.15.39:0/4094676393 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f38e40fea80 0x7f38e40fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:38:20.822 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:38:20.788+0000 7f38e90f5700 1 --2- 172.21.15.39:0/4094676393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38e40ffc80 0x7f38e41000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:38:20.822 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:38:20.788+0000 7f38e90f5700 1 -- 172.21.15.39:0/4094676393 >> 172.21.15.39:0/4094676393 conn(0x7f38e40fa0b0 msgr2=0x7f38e40fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:38:20.822 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:38:20.788+0000 7f38e90f5700 1 -- 172.21.15.39:0/4094676393 shutdown_connections 2023-07-21T18:38:20.823 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:38:20.788+0000 7f38e90f5700 1 -- 172.21.15.39:0/4094676393 wait complete. 2023-07-21T18:38:20.823 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:38:20.789+0000 7f38e90f5700 1 Processor -- start 2023-07-21T18:38:20.823 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:38:20.789+0000 7f38e90f5700 1 -- start start 2023-07-21T18:38:20.823 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:38:20.790+0000 7f38e90f5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38e40fea80 0x7f38e4102a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:38:20.823 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:38:20.790+0000 7f38e90f5700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f38e40ffc80 0x7f38e4101060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:38:20.823 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:38:20.790+0000 7f38e90f5700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38e4100610 0x7f38e41015a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:38:20.823 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:38:20.790+0000 7f38e90f5700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e4101d20 con 0x7f38e40fea80 2023-07-21T18:38:20.824 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:38:20.790+0000 7f38e90f5700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e4101e90 con 0x7f38e4100610 2023-07-21T18:38:20.824 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:38:20.790+0000 7f38e90f5700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e419f9f0 con 0x7f38e40ffc80 2023-07-21T18:38:20.824 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:38:20.790+0000 7f38e2d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38e40fea80 0x7f38e4102a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:38:20.824 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:38:20.790+0000 7f38e2d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38e40fea80 0x7f38e4102a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54456/0 (socket says 172.21.15.39:54456) 2023-07-21T18:38:20.824 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:38:20.790+0000 7f38e2d9d700 1 -- 172.21.15.39:0/370052068 learned_addr learned my addr 172.21.15.39:0/370052068 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:38:20.824 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:38:20.790+0000 7f38e359e700 1 --2- 172.21.15.39:0/370052068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38e4100610 0x7f38e41015a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:38:20.824 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:38:20.790+0000 7f38e259c700 1 --2- 172.21.15.39:0/370052068 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f38e40ffc80 0x7f38e4101060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:38:20.825 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:38:20.790+0000 7f38e2d9d700 1 -- 172.21.15.39:0/370052068 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f38e40ffc80 msgr2=0x7f38e4101060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:38:20.825 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:38:20.790+0000 7f38e2d9d700 1 --2- 172.21.15.39:0/370052068 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f38e40ffc80 0x7f38e4101060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:38:20.825 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:38:20.790+0000 7f38e2d9d700 1 -- 172.21.15.39:0/370052068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38e4100610 msgr2=0x7f38e41015a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:38:20.825 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:38:20.790+0000 7f38e2d9d700 1 --2- 172.21.15.39:0/370052068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38e4100610 0x7f38e41015a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:38:20.825 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:38:20.790+0000 7f38e2d9d700 1 -- 172.21.15.39:0/370052068 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38dc009d00 con 0x7f38e40fea80 2023-07-21T18:38:20.825 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:38:20.790+0000 7f38e259c700 1 --2- 172.21.15.39:0/370052068 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f38e40ffc80 0x7f38e4101060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:38:20.825 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:38:20.790+0000 7f38e359e700 1 --2- 172.21.15.39:0/370052068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38e4100610 0x7f38e41015a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:38:20.826 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:38:20.790+0000 7f38e2d9d700 1 --2- 172.21.15.39:0/370052068 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38e40fea80 0x7f38e4102a10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f38d400c970 tx=0x7f38d400cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:38:20.826 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:38:20.790+0000 7f38cbfff700 1 -- 172.21.15.39:0/370052068 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d4004220 con 0x7f38e40fea80 2023-07-21T18:38:20.826 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:38:20.790+0000 7f38e90f5700 1 -- 172.21.15.39:0/370052068 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38e419fcd0 con 0x7f38e40fea80 2023-07-21T18:38:20.826 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:38:20.790+0000 7f38cbfff700 1 -- 172.21.15.39:0/370052068 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38d4007d60 con 0x7f38e40fea80 2023-07-21T18:38:20.826 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:38:20.791+0000 7f38cbfff700 1 -- 172.21.15.39:0/370052068 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d4003d00 con 0x7f38e40fea80 2023-07-21T18:38:20.826 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:38:20.791+0000 7f38e90f5700 1 -- 172.21.15.39:0/370052068 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38e41a0180 con 0x7f38e40fea80 2023-07-21T18:38:20.827 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:38:20.791+0000 7f38e90f5700 1 -- 172.21.15.39:0/370052068 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38e410a250 con 0x7f38e40fea80 2023-07-21T18:38:20.827 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:38:20.793+0000 7f38cbfff700 1 -- 172.21.15.39:0/370052068 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f38d4004380 con 0x7f38e40fea80 2023-07-21T18:38:20.827 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:38:20.793+0000 7f38cbfff700 1 --2- 172.21.15.39:0/370052068 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f38cc0a1150 0x7f38cc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:38:20.828 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:38:20.793+0000 7f38e259c700 1 --2- 172.21.15.39:0/370052068 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f38cc0a1150 0x7f38cc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:38:20.828 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:38:20.793+0000 7f38cbfff700 1 -- 172.21.15.39:0/370052068 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5862+0+0 (secure 0 0 0) 0x7f38d40cbf70 con 0x7f38e40fea80 2023-07-21T18:38:20.828 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:38:20.794+0000 7f38e259c700 1 --2- 172.21.15.39:0/370052068 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f38cc0a1150 0x7f38cc0a3600 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f38d8005fd0 tx=0x7f38d8005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:38:20.828 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:38:20.797+0000 7f38cbfff700 1 -- 172.21.15.39:0/370052068 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f38d4096050 con 0x7f38e40fea80 2023-07-21T18:38:21.034 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:38:21.004+0000 7f38e90f5700 1 -- 172.21.15.39:0/370052068 --> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f38e4069030 con 0x7f38cc0a1150 2023-07-21T18:38:21.378 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:21 smithi193 ceph-mon[98204]: purged_snaps scrub starts 2023-07-21T18:38:21.378 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:21 smithi193 ceph-mon[98204]: purged_snaps scrub ok 2023-07-21T18:38:21.378 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:21 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T18:38:21.379 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:21 smithi193 ceph-mon[98204]: osd.8 [v2:172.21.15.193:6816/2985835085,v1:172.21.15.193:6817/2985835085] boot 2023-07-21T18:38:21.379 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:21 smithi193 ceph-mon[98204]: osdmap e55: 9 total, 9 up, 9 in 2023-07-21T18:38:21.379 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:21 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T18:38:21.379 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:21 smithi193 ceph-mon[98204]: osdmap e56: 9 total, 9 up, 9 in 2023-07-21T18:38:21.379 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:21 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T18:38:21.380 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:21 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T18:38:21.380 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:21 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:38:21.396 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:21 smithi155 ceph-mon[97934]: purged_snaps scrub starts 2023-07-21T18:38:21.397 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:21 smithi155 ceph-mon[97934]: purged_snaps scrub ok 2023-07-21T18:38:21.397 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:21 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T18:38:21.397 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:21 smithi155 ceph-mon[97934]: osd.8 [v2:172.21.15.193:6816/2985835085,v1:172.21.15.193:6817/2985835085] boot 2023-07-21T18:38:21.397 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:21 smithi155 ceph-mon[97934]: osdmap e55: 9 total, 9 up, 9 in 2023-07-21T18:38:21.397 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:21 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T18:38:21.397 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:21 smithi155 ceph-mon[97934]: osdmap e56: 9 total, 9 up, 9 in 2023-07-21T18:38:21.398 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:21 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T18:38:21.398 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:21 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T18:38:21.398 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:21 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:38:21.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:21 smithi039 ceph-mon[96489]: purged_snaps scrub starts 2023-07-21T18:38:21.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:21 smithi039 ceph-mon[96489]: purged_snaps scrub ok 2023-07-21T18:38:21.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:21 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T18:38:21.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:21 smithi039 ceph-mon[96489]: osd.8 [v2:172.21.15.193:6816/2985835085,v1:172.21.15.193:6817/2985835085] boot 2023-07-21T18:38:21.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:21 smithi039 ceph-mon[96489]: osdmap e55: 9 total, 9 up, 9 in 2023-07-21T18:38:21.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:21 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T18:38:21.447 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:21 smithi039 ceph-mon[96489]: osdmap e56: 9 total, 9 up, 9 in 2023-07-21T18:38:21.447 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:21 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T18:38:21.447 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:21 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T18:38:21.447 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:21 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:38:22.378 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:22 smithi193 ceph-mon[98204]: from='client.14409 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:38:22.378 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:22 smithi193 ceph-mon[98204]: pgmap v173: 1 pgs: 1 remapped+peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-21T18:38:22.378 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:22 smithi193 ceph-mon[98204]: osdmap e57: 9 total, 9 up, 9 in 2023-07-21T18:38:22.397 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:22 smithi155 ceph-mon[97934]: from='client.14409 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:38:22.398 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:22 smithi155 ceph-mon[97934]: pgmap v173: 1 pgs: 1 remapped+peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-21T18:38:22.398 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:22 smithi155 ceph-mon[97934]: osdmap e57: 9 total, 9 up, 9 in 2023-07-21T18:38:22.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:22 smithi039 ceph-mon[96489]: from='client.14409 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:38:22.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:22 smithi039 ceph-mon[96489]: pgmap v173: 1 pgs: 1 remapped+peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-21T18:38:22.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:22 smithi039 ceph-mon[96489]: osdmap e57: 9 total, 9 up, 9 in 2023-07-21T18:38:24.878 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:24 smithi193 ceph-mon[98204]: osdmap e58: 9 total, 9 up, 9 in 2023-07-21T18:38:24.878 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:24 smithi193 ceph-mon[98204]: pgmap v176: 1 pgs: 1 remapped+peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-21T18:38:24.878 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:24 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/3155770805' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9725ea5c-49a3-4c99-bac4-e95069100b1e"}]: dispatch 2023-07-21T18:38:24.878 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:24 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/3155770805' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9725ea5c-49a3-4c99-bac4-e95069100b1e"}]': finished 2023-07-21T18:38:24.878 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:24 smithi193 ceph-mon[98204]: osdmap e59: 10 total, 9 up, 10 in 2023-07-21T18:38:24.878 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:24 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T18:38:24.879 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:24 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/4122464273' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T18:38:24.896 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:24 smithi155 ceph-mon[97934]: osdmap e58: 9 total, 9 up, 9 in 2023-07-21T18:38:24.897 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:24 smithi155 ceph-mon[97934]: pgmap v176: 1 pgs: 1 remapped+peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-21T18:38:24.897 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:24 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/3155770805' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9725ea5c-49a3-4c99-bac4-e95069100b1e"}]: dispatch 2023-07-21T18:38:24.897 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:24 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/3155770805' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9725ea5c-49a3-4c99-bac4-e95069100b1e"}]': finished 2023-07-21T18:38:24.897 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:24 smithi155 ceph-mon[97934]: osdmap e59: 10 total, 9 up, 10 in 2023-07-21T18:38:24.897 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:24 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T18:38:24.897 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:24 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/4122464273' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T18:38:24.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:24 smithi039 ceph-mon[96489]: osdmap e58: 9 total, 9 up, 9 in 2023-07-21T18:38:24.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:24 smithi039 ceph-mon[96489]: pgmap v176: 1 pgs: 1 remapped+peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-21T18:38:24.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:24 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/3155770805' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9725ea5c-49a3-4c99-bac4-e95069100b1e"}]: dispatch 2023-07-21T18:38:24.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:24 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/3155770805' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9725ea5c-49a3-4c99-bac4-e95069100b1e"}]': finished 2023-07-21T18:38:24.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:24 smithi039 ceph-mon[96489]: osdmap e59: 10 total, 9 up, 10 in 2023-07-21T18:38:24.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:24 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T18:38:24.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:24 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/4122464273' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T18:38:25.878 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:25 smithi193 ceph-mon[98204]: pgmap v178: 1 pgs: 1 remapped+peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-21T18:38:25.896 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:25 smithi155 ceph-mon[97934]: pgmap v178: 1 pgs: 1 remapped+peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-21T18:38:25.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:25 smithi039 ceph-mon[96489]: pgmap v178: 1 pgs: 1 remapped+peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-21T18:38:27.627 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:27 smithi193 ceph-mon[98204]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-21T18:38:27.646 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:27 smithi155 ceph-mon[97934]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-21T18:38:27.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:27 smithi039 ceph-mon[96489]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-21T18:38:29.545 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:29 smithi039 ceph-mon[96489]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 81 KiB/s, 0 objects/s recovering 2023-07-21T18:38:29.627 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:29 smithi193 ceph-mon[98204]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 81 KiB/s, 0 objects/s recovering 2023-07-21T18:38:29.646 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:29 smithi155 ceph-mon[97934]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 81 KiB/s, 0 objects/s recovering 2023-07-21T18:38:31.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:30 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-07-21T18:38:31.196 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:30 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:38:31.377 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:30 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-07-21T18:38:31.378 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:30 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:38:31.396 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:30 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-07-21T18:38:31.396 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:30 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:38:32.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:31 smithi039 ceph-mon[96489]: Deploying daemon osd.9 on smithi039 2023-07-21T18:38:32.196 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:31 smithi039 ceph-mon[96489]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-07-21T18:38:32.377 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:31 smithi193 ceph-mon[98204]: Deploying daemon osd.9 on smithi039 2023-07-21T18:38:32.378 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:31 smithi193 ceph-mon[98204]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-07-21T18:38:32.396 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:31 smithi155 ceph-mon[97934]: Deploying daemon osd.9 on smithi039 2023-07-21T18:38:32.397 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:31 smithi155 ceph-mon[97934]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-07-21T18:38:33.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:33 smithi039 ceph-mon[96489]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-07-21T18:38:33.627 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:33 smithi193 ceph-mon[98204]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-07-21T18:38:33.646 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:33 smithi155 ceph-mon[97934]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-07-21T18:38:35.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:35 smithi039 ceph-mon[96489]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 49 KiB/s, 0 objects/s recovering 2023-07-21T18:38:35.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:35 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:35.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:35 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:35.627 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:35 smithi193 ceph-mon[98204]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 49 KiB/s, 0 objects/s recovering 2023-07-21T18:38:35.628 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:35 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:35.628 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:35 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:35.646 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:35 smithi155 ceph-mon[97934]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 49 KiB/s, 0 objects/s recovering 2023-07-21T18:38:35.646 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:35 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:35.647 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:35 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:36.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:36 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:38:36.627 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:36 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:38:36.646 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:36 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:38:37.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:37 smithi039 ceph-mon[96489]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-21T18:38:37.627 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:37 smithi193 ceph-mon[98204]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-21T18:38:37.646 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:37 smithi155 ceph-mon[97934]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-21T18:38:37.888 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:38:37.858+0000 7f38cbfff700 1 -- 172.21.15.39:0/370052068 <== mgr.14152 v2:172.21.15.39:6800/2116291712 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f38e4069030 con 0x7f38cc0a1150 2023-07-21T18:38:37.888 INFO:teuthology.orchestra.run.smithi039.stdout:Created osd(s) 9 on host 'smithi039' 2023-07-21T18:38:37.890 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:38:37.860+0000 7f38e90f5700 1 -- 172.21.15.39:0/370052068 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f38cc0a1150 msgr2=0x7f38cc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:38:37.890 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:38:37.860+0000 7f38e90f5700 1 --2- 172.21.15.39:0/370052068 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f38cc0a1150 0x7f38cc0a3600 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f38d8005fd0 tx=0x7f38d8005ee0 comp rx=0 tx=0).stop 2023-07-21T18:38:37.890 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:38:37.860+0000 7f38e90f5700 1 -- 172.21.15.39:0/370052068 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38e40fea80 msgr2=0x7f38e4102a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:38:37.890 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:38:37.860+0000 7f38e90f5700 1 --2- 172.21.15.39:0/370052068 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38e40fea80 0x7f38e4102a10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f38d400c970 tx=0x7f38d400cd30 comp rx=0 tx=0).stop 2023-07-21T18:38:37.890 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:38:37.860+0000 7f38e90f5700 1 -- 172.21.15.39:0/370052068 shutdown_connections 2023-07-21T18:38:37.890 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:38:37.860+0000 7f38e90f5700 1 --2- 172.21.15.39:0/370052068 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f38cc0a1150 0x7f38cc0a3600 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:38:37.891 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:38:37.860+0000 7f38e90f5700 1 --2- 172.21.15.39:0/370052068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38e4100610 0x7f38e41015a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:38:37.891 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:38:37.860+0000 7f38e90f5700 1 --2- 172.21.15.39:0/370052068 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38e40fea80 0x7f38e4102a10 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:38:37.891 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:38:37.860+0000 7f38e90f5700 1 --2- 172.21.15.39:0/370052068 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f38e40ffc80 0x7f38e4101060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:38:37.891 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:38:37.860+0000 7f38e90f5700 1 -- 172.21.15.39:0/370052068 >> 172.21.15.39:0/370052068 conn(0x7f38e40fa0b0 msgr2=0x7f38e4068920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:38:37.891 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:38:37.861+0000 7f38e90f5700 1 -- 172.21.15.39:0/370052068 shutdown_connections 2023-07-21T18:38:37.891 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:38:37.861+0000 7f38e90f5700 1 -- 172.21.15.39:0/370052068 wait complete. 2023-07-21T18:38:38.235 DEBUG:teuthology.orchestra.run.smithi039:osd.9> sudo journalctl -f -n 0 -u ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@osd.9.service 2023-07-21T18:38:38.239 INFO:tasks.cephadm:Deploying osd.10 on smithi155 with /dev/vg_nvme/lv_1... 2023-07-21T18:38:38.239 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-07-21T18:38:38.270 INFO:journalctl@ceph.osd.9.smithi039.stdout:-- Logs begin at Fri 2023-07-21 18:18:00 UTC. -- 2023-07-21T18:38:38.805 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.b/config 2023-07-21T18:38:38.896 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:38 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:38.896 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:38 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:38.896 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:38 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:38.896 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:38 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:38.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:38 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:38.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:38 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:38.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:38 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:38.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:38 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:39.127 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:38 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:39.128 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:38 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:39.128 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:38 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:39.128 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:38 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:39.257 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 18:38:39 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T18:38:39.101+0000 7f1c11c1d640 -1 osd.9 0 log_to_monitors true 2023-07-21T18:38:39.893 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:39 smithi155 ceph-mon[97934]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-21T18:38:39.893 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:39 smithi155 ceph-mon[97934]: from='osd.9 [v2:172.21.15.39:6826/1946493651,v1:172.21.15.39:6827/1946493651]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-07-21T18:38:39.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:39 smithi039 ceph-mon[96489]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-21T18:38:39.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:39 smithi039 ceph-mon[96489]: from='osd.9 [v2:172.21.15.39:6826/1946493651,v1:172.21.15.39:6827/1946493651]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-07-21T18:38:40.127 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:39 smithi193 ceph-mon[98204]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-21T18:38:40.127 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:39 smithi193 ceph-mon[98204]: from='osd.9 [v2:172.21.15.39:6826/1946493651,v1:172.21.15.39:6827/1946493651]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-07-21T18:38:40.343 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-07-21T18:38:40.343 INFO:teuthology.orchestra.run.smithi155.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-07-21T18:38:40.343 INFO:teuthology.orchestra.run.smithi155.stderr: stderr: 10+0 records in 2023-07-21T18:38:40.344 INFO:teuthology.orchestra.run.smithi155.stderr:10+0 records out 2023-07-21T18:38:40.344 INFO:teuthology.orchestra.run.smithi155.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0107629 s, 974 MB/s 2023-07-21T18:38:40.344 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping successful for: 2023-07-21T18:38:40.767 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi155:vg_nvme/lv_1 2023-07-21T18:38:40.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:40 smithi039 ceph-mon[96489]: from='osd.9 [v2:172.21.15.39:6826/1946493651,v1:172.21.15.39:6827/1946493651]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-07-21T18:38:40.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:40 smithi039 ceph-mon[96489]: osdmap e60: 10 total, 9 up, 10 in 2023-07-21T18:38:40.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:40 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T18:38:40.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:40 smithi039 ceph-mon[96489]: from='osd.9 [v2:172.21.15.39:6826/1946493651,v1:172.21.15.39:6827/1946493651]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2023-07-21T18:38:40.946 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 18:38:40 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T18:38:40.640+0000 7f1c08297700 -1 osd.9 0 waiting for initial osdmap 2023-07-21T18:38:40.946 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 18:38:40 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T18:38:40.652+0000 7f1c00885700 -1 osd.9 61 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-21T18:38:41.127 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:40 smithi193 ceph-mon[98204]: from='osd.9 [v2:172.21.15.39:6826/1946493651,v1:172.21.15.39:6827/1946493651]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-07-21T18:38:41.127 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:40 smithi193 ceph-mon[98204]: osdmap e60: 10 total, 9 up, 10 in 2023-07-21T18:38:41.128 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:40 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T18:38:41.128 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:40 smithi193 ceph-mon[98204]: from='osd.9 [v2:172.21.15.39:6826/1946493651,v1:172.21.15.39:6827/1946493651]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2023-07-21T18:38:41.146 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:40 smithi155 ceph-mon[97934]: from='osd.9 [v2:172.21.15.39:6826/1946493651,v1:172.21.15.39:6827/1946493651]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-07-21T18:38:41.146 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:40 smithi155 ceph-mon[97934]: osdmap e60: 10 total, 9 up, 10 in 2023-07-21T18:38:41.146 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:40 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T18:38:41.146 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:40 smithi155 ceph-mon[97934]: from='osd.9 [v2:172.21.15.39:6826/1946493651,v1:172.21.15.39:6827/1946493651]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2023-07-21T18:38:41.323 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.b/config 2023-07-21T18:38:41.815 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:41 smithi155 ceph-mon[97934]: from='osd.9 [v2:172.21.15.39:6826/1946493651,v1:172.21.15.39:6827/1946493651]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2023-07-21T18:38:41.816 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:41 smithi155 ceph-mon[97934]: osdmap e61: 10 total, 9 up, 10 in 2023-07-21T18:38:41.816 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:41 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T18:38:41.816 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:41 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T18:38:41.816 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:41 smithi155 ceph-mon[97934]: Detected new or changed devices on smithi039 2023-07-21T18:38:41.816 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:41 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:41.817 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:41 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:41.817 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:41 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:38:41.817 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:41 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:38:41.817 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:41 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:38:41.817 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:41 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:38:41.818 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:41 smithi155 ceph-mon[97934]: Adjusting osd_memory_target on smithi039 to 4292M 2023-07-21T18:38:41.818 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:41 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:41.818 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:41 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:38:41.818 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:41 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:38:41.819 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:41 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:41.819 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:41 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:38:41.819 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:41 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:38:41.819 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:41 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:38:41.820 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:41 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:41.820 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:41 smithi155 ceph-mon[97934]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-21T18:38:41.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:41 smithi039 ceph-mon[96489]: from='osd.9 [v2:172.21.15.39:6826/1946493651,v1:172.21.15.39:6827/1946493651]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2023-07-21T18:38:41.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:41 smithi039 ceph-mon[96489]: osdmap e61: 10 total, 9 up, 10 in 2023-07-21T18:38:41.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:41 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T18:38:41.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:41 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T18:38:41.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:41 smithi039 ceph-mon[96489]: Detected new or changed devices on smithi039 2023-07-21T18:38:41.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:41 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:41.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:41 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:41.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:41 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:38:41.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:41 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:38:41.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:41 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:38:41.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:41 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:38:41.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:41 smithi039 ceph-mon[96489]: Adjusting osd_memory_target on smithi039 to 4292M 2023-07-21T18:38:41.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:41 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:41.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:41 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:38:41.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:41 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:38:41.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:41 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:41.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:41 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:38:41.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:41 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:38:41.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:41 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:38:41.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:41 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:41.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:41 smithi039 ceph-mon[96489]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-21T18:38:42.127 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:41 smithi193 ceph-mon[98204]: from='osd.9 [v2:172.21.15.39:6826/1946493651,v1:172.21.15.39:6827/1946493651]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2023-07-21T18:38:42.128 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:41 smithi193 ceph-mon[98204]: osdmap e61: 10 total, 9 up, 10 in 2023-07-21T18:38:42.128 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:41 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T18:38:42.128 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:41 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T18:38:42.129 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:41 smithi193 ceph-mon[98204]: Detected new or changed devices on smithi039 2023-07-21T18:38:42.129 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:41 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:42.129 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:41 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:42.129 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:41 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:38:42.130 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:41 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:38:42.130 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:41 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:38:42.131 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:41 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:38:42.131 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:41 smithi193 ceph-mon[98204]: Adjusting osd_memory_target on smithi039 to 4292M 2023-07-21T18:38:42.131 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:41 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:42.131 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:41 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:38:42.132 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:41 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:38:42.132 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:41 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:42.132 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:41 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:38:42.132 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:41 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:38:42.132 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:41 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:38:42.132 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:41 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:42.133 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:41 smithi193 ceph-mon[98204]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-21T18:38:42.517 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:38:42.485+0000 7fc2f55f6700 1 -- 172.21.15.155:0/39224539 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2f0100670 msgr2=0x7fc2f0106f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:38:42.517 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:38:42.485+0000 7fc2f55f6700 1 --2- 172.21.15.155:0/39224539 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2f0100670 0x7fc2f0106f30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc2e800b0a0 tx=0x7fc2e800b3b0 comp rx=0 tx=0).stop 2023-07-21T18:38:42.517 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:38:42.485+0000 7fc2f55f6700 1 -- 172.21.15.155:0/39224539 shutdown_connections 2023-07-21T18:38:42.517 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:38:42.485+0000 7fc2f55f6700 1 --2- 172.21.15.155:0/39224539 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2f0100670 0x7fc2f0106f30 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:38:42.517 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:38:42.485+0000 7fc2f55f6700 1 --2- 172.21.15.155:0/39224539 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc2f00feae0 0x7fc2f00feef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:38:42.518 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:38:42.485+0000 7fc2f55f6700 1 --2- 172.21.15.155:0/39224539 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2f00ffce0 0x7fc2f0100130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:38:42.518 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:38:42.485+0000 7fc2f55f6700 1 -- 172.21.15.155:0/39224539 >> 172.21.15.155:0/39224539 conn(0x7fc2f00fa130 msgr2=0x7fc2f00fc560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:38:42.518 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:38:42.485+0000 7fc2f55f6700 1 -- 172.21.15.155:0/39224539 shutdown_connections 2023-07-21T18:38:42.518 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:38:42.485+0000 7fc2f55f6700 1 -- 172.21.15.155:0/39224539 wait complete. 2023-07-21T18:38:42.518 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:38:42.486+0000 7fc2f55f6700 1 Processor -- start 2023-07-21T18:38:42.518 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:38:42.486+0000 7fc2f55f6700 1 -- start start 2023-07-21T18:38:42.518 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:38:42.486+0000 7fc2f55f6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2f00feae0 0x7fc2f0195a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:38:42.519 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:38:42.486+0000 7fc2f55f6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2f00ffce0 0x7fc2f0195f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:38:42.519 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:38:42.486+0000 7fc2f55f6700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc2f0100670 0x7fc2f019cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:38:42.519 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:38:42.486+0000 7fc2f55f6700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f019d5c0 con 0x7fc2f00feae0 2023-07-21T18:38:42.519 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:38:42.486+0000 7fc2f55f6700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f019d700 con 0x7fc2f00ffce0 2023-07-21T18:38:42.519 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:38:42.486+0000 7fc2f55f6700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f019d890 con 0x7fc2f0100670 2023-07-21T18:38:42.519 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:38:42.486+0000 7fc2ee7fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2f00ffce0 0x7fc2f0195f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:38:42.519 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:38:42.486+0000 7fc2ee7fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2f00ffce0 0x7fc2f0195f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:57516/0 (socket says 172.21.15.155:57516) 2023-07-21T18:38:42.520 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:38:42.486+0000 7fc2ee7fc700 1 -- 172.21.15.155:0/1922888464 learned_addr learned my addr 172.21.15.155:0/1922888464 (peer_addr_for_me v2:172.21.15.155:0/0) 2023-07-21T18:38:42.520 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:38:42.486+0000 7fc2ef7fe700 1 --2- 172.21.15.155:0/1922888464 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc2f0100670 0x7fc2f019cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:38:42.520 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:38:42.487+0000 7fc2eeffd700 1 --2- 172.21.15.155:0/1922888464 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2f00feae0 0x7fc2f0195a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:38:42.520 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:38:42.487+0000 7fc2ee7fc700 1 -- 172.21.15.155:0/1922888464 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc2f0100670 msgr2=0x7fc2f019cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:38:42.520 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:38:42.487+0000 7fc2ee7fc700 1 --2- 172.21.15.155:0/1922888464 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc2f0100670 0x7fc2f019cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:38:42.520 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:38:42.487+0000 7fc2ee7fc700 1 -- 172.21.15.155:0/1922888464 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2f00feae0 msgr2=0x7fc2f0195a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:38:42.521 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:38:42.487+0000 7fc2ee7fc700 1 --2- 172.21.15.155:0/1922888464 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2f00feae0 0x7fc2f0195a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:38:42.521 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:38:42.487+0000 7fc2ee7fc700 1 -- 172.21.15.155:0/1922888464 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2e8009d00 con 0x7fc2f00ffce0 2023-07-21T18:38:42.521 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:38:42.487+0000 7fc2ee7fc700 1 --2- 172.21.15.155:0/1922888464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2f00ffce0 0x7fc2f0195f70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc2e000d980 tx=0x7fc2e000dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:38:42.521 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:38:42.487+0000 7fc2e7fff700 1 -- 172.21.15.155:0/1922888464 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e0004220 con 0x7fc2f00ffce0 2023-07-21T18:38:42.521 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:38:42.487+0000 7fc2e7fff700 1 -- 172.21.15.155:0/1922888464 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc2e0007640 con 0x7fc2f00ffce0 2023-07-21T18:38:42.521 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:38:42.487+0000 7fc2e7fff700 1 -- 172.21.15.155:0/1922888464 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e0005680 con 0x7fc2f00ffce0 2023-07-21T18:38:42.522 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:38:42.487+0000 7fc2f55f6700 1 -- 172.21.15.155:0/1922888464 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2f019db70 con 0x7fc2f00ffce0 2023-07-21T18:38:42.522 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:38:42.487+0000 7fc2f55f6700 1 -- 172.21.15.155:0/1922888464 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2f019e020 con 0x7fc2f00ffce0 2023-07-21T18:38:42.523 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:38:42.488+0000 7fc2f55f6700 1 -- 172.21.15.155:0/1922888464 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2f010a340 con 0x7fc2f00ffce0 2023-07-21T18:38:42.523 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:38:42.489+0000 7fc2e7fff700 1 -- 172.21.15.155:0/1922888464 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fc2e0004380 con 0x7fc2f00ffce0 2023-07-21T18:38:42.523 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:38:42.490+0000 7fc2e7fff700 1 --2- 172.21.15.155:0/1922888464 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fc2dc0a1150 0x7fc2dc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:38:42.523 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:38:42.490+0000 7fc2e7fff700 1 -- 172.21.15.155:0/1922888464 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6261+0+0 (secure 0 0 0) 0x7fc2e00cbf60 con 0x7fc2f00ffce0 2023-07-21T18:38:42.523 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:38:42.490+0000 7fc2eeffd700 1 --2- 172.21.15.155:0/1922888464 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fc2dc0a1150 0x7fc2dc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:38:42.523 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:38:42.490+0000 7fc2eeffd700 1 --2- 172.21.15.155:0/1922888464 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fc2dc0a1150 0x7fc2dc0a3600 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc2d8006fd0 tx=0x7fc2d8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:38:42.524 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:38:42.494+0000 7fc2e7fff700 1 -- 172.21.15.155:0/1922888464 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fc2e0095eb0 con 0x7fc2f00ffce0 2023-07-21T18:38:42.719 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:38:42.689+0000 7fc2f55f6700 1 -- 172.21.15.155:0/1922888464 --> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fc2f0069060 con 0x7fc2dc0a1150 2023-07-21T18:38:42.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:42 smithi039 ceph-mon[96489]: purged_snaps scrub starts 2023-07-21T18:38:42.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:42 smithi039 ceph-mon[96489]: purged_snaps scrub ok 2023-07-21T18:38:42.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:42 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T18:38:42.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:42 smithi039 ceph-mon[96489]: osd.9 [v2:172.21.15.39:6826/1946493651,v1:172.21.15.39:6827/1946493651] boot 2023-07-21T18:38:42.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:42 smithi039 ceph-mon[96489]: osdmap e62: 10 total, 10 up, 10 in 2023-07-21T18:38:42.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:42 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T18:38:43.127 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:42 smithi193 ceph-mon[98204]: purged_snaps scrub starts 2023-07-21T18:38:43.127 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:42 smithi193 ceph-mon[98204]: purged_snaps scrub ok 2023-07-21T18:38:43.127 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:42 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T18:38:43.127 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:42 smithi193 ceph-mon[98204]: osd.9 [v2:172.21.15.39:6826/1946493651,v1:172.21.15.39:6827/1946493651] boot 2023-07-21T18:38:43.127 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:42 smithi193 ceph-mon[98204]: osdmap e62: 10 total, 10 up, 10 in 2023-07-21T18:38:43.128 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:42 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T18:38:43.145 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:42 smithi155 ceph-mon[97934]: purged_snaps scrub starts 2023-07-21T18:38:43.146 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:42 smithi155 ceph-mon[97934]: purged_snaps scrub ok 2023-07-21T18:38:43.146 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:42 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T18:38:43.146 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:42 smithi155 ceph-mon[97934]: osd.9 [v2:172.21.15.39:6826/1946493651,v1:172.21.15.39:6827/1946493651] boot 2023-07-21T18:38:43.146 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:42 smithi155 ceph-mon[97934]: osdmap e62: 10 total, 10 up, 10 in 2023-07-21T18:38:43.146 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:42 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T18:38:43.848 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:43 smithi155 ceph-mon[97934]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:38:43.848 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:43 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T18:38:43.849 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:43 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T18:38:43.849 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:43 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:38:43.849 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:43 smithi155 ceph-mon[97934]: osdmap e63: 10 total, 10 up, 10 in 2023-07-21T18:38:43.849 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:43 smithi155 ceph-mon[97934]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T18:38:43.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:43 smithi039 ceph-mon[96489]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:38:43.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:43 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T18:38:43.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:43 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T18:38:43.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:43 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:38:43.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:43 smithi039 ceph-mon[96489]: osdmap e63: 10 total, 10 up, 10 in 2023-07-21T18:38:43.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:43 smithi039 ceph-mon[96489]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T18:38:44.127 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:43 smithi193 ceph-mon[98204]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:38:44.127 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:43 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T18:38:44.127 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:43 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T18:38:44.127 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:43 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:38:44.127 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:43 smithi193 ceph-mon[98204]: osdmap e63: 10 total, 10 up, 10 in 2023-07-21T18:38:44.127 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:43 smithi193 ceph-mon[98204]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T18:38:45.627 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:45 smithi193 ceph-mon[98204]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T18:38:45.627 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:45 smithi193 ceph-mon[98204]: from='client.? 172.21.15.155:0/398297653' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fd60ccb-92db-45f0-a580-2003db0976e4"}]: dispatch 2023-07-21T18:38:45.627 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:45 smithi193 ceph-mon[98204]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fd60ccb-92db-45f0-a580-2003db0976e4"}]: dispatch 2023-07-21T18:38:45.627 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:45 smithi193 ceph-mon[98204]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1fd60ccb-92db-45f0-a580-2003db0976e4"}]': finished 2023-07-21T18:38:45.627 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:45 smithi193 ceph-mon[98204]: osdmap e64: 11 total, 10 up, 11 in 2023-07-21T18:38:45.628 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:45 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T18:38:45.645 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:45 smithi155 ceph-mon[97934]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T18:38:45.646 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:45 smithi155 ceph-mon[97934]: from='client.? 172.21.15.155:0/398297653' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fd60ccb-92db-45f0-a580-2003db0976e4"}]: dispatch 2023-07-21T18:38:45.646 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:45 smithi155 ceph-mon[97934]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fd60ccb-92db-45f0-a580-2003db0976e4"}]: dispatch 2023-07-21T18:38:45.647 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:45 smithi155 ceph-mon[97934]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1fd60ccb-92db-45f0-a580-2003db0976e4"}]': finished 2023-07-21T18:38:45.647 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:45 smithi155 ceph-mon[97934]: osdmap e64: 11 total, 10 up, 11 in 2023-07-21T18:38:45.647 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:45 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T18:38:45.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:45 smithi039 ceph-mon[96489]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T18:38:45.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:45 smithi039 ceph-mon[96489]: from='client.? 172.21.15.155:0/398297653' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fd60ccb-92db-45f0-a580-2003db0976e4"}]: dispatch 2023-07-21T18:38:45.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:45 smithi039 ceph-mon[96489]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fd60ccb-92db-45f0-a580-2003db0976e4"}]: dispatch 2023-07-21T18:38:45.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:45 smithi039 ceph-mon[96489]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1fd60ccb-92db-45f0-a580-2003db0976e4"}]': finished 2023-07-21T18:38:45.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:45 smithi039 ceph-mon[96489]: osdmap e64: 11 total, 10 up, 11 in 2023-07-21T18:38:45.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:45 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T18:38:46.627 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:46 smithi193 ceph-mon[98204]: from='client.? 172.21.15.155:0/2122238906' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T18:38:46.645 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:46 smithi155 ceph-mon[97934]: from='client.? 172.21.15.155:0/2122238906' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T18:38:46.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:46 smithi039 ceph-mon[96489]: from='client.? 172.21.15.155:0/2122238906' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T18:38:47.627 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:47 smithi193 ceph-mon[98204]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T18:38:47.645 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:47 smithi155 ceph-mon[97934]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T18:38:47.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:47 smithi039 ceph-mon[96489]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T18:38:49.395 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:49 smithi155 ceph-mon[97934]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T18:38:49.626 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:49 smithi193 ceph-mon[98204]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T18:38:49.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:49 smithi039 ceph-mon[96489]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T18:38:52.145 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:51 smithi155 ceph-mon[97934]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T18:38:52.146 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:51 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-07-21T18:38:52.146 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:51 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:38:52.376 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:51 smithi193 ceph-mon[98204]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T18:38:52.377 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:51 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-07-21T18:38:52.377 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:51 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:38:52.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:51 smithi039 ceph-mon[96489]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T18:38:52.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:51 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-07-21T18:38:52.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:51 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:38:53.145 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:52 smithi155 ceph-mon[97934]: Deploying daemon osd.10 on smithi155 2023-07-21T18:38:53.376 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:52 smithi193 ceph-mon[98204]: Deploying daemon osd.10 on smithi155 2023-07-21T18:38:53.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:52 smithi039 ceph-mon[96489]: Deploying daemon osd.10 on smithi155 2023-07-21T18:38:54.355 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:53 smithi155 ceph-mon[97934]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T18:38:54.376 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:53 smithi193 ceph-mon[98204]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T18:38:54.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:53 smithi039 ceph-mon[96489]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T18:38:55.488 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:55 smithi155 ceph-mon[97934]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T18:38:55.626 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:55 smithi193 ceph-mon[98204]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T18:38:55.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:55 smithi039 ceph-mon[96489]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T18:38:57.626 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:57 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:57.627 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:57 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:38:57.627 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:57 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:57.627 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:57 smithi193 ceph-mon[98204]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T18:38:57.637 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:57 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:57.637 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:57 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:38:57.638 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:57 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:57.638 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:57 smithi155 ceph-mon[97934]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T18:38:57.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:57 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:57.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:57 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:38:57.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:57 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:38:57.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:57 smithi039 ceph-mon[96489]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T18:38:59.235 INFO:teuthology.orchestra.run.smithi155.stdout:Created osd(s) 10 on host 'smithi155' 2023-07-21T18:38:59.235 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:38:59.205+0000 7fc2e7fff700 1 -- 172.21.15.155:0/1922888464 <== mgr.14152 v2:172.21.15.39:6800/2116291712 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fc2f0069060 con 0x7fc2dc0a1150 2023-07-21T18:38:59.237 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:38:59.207+0000 7fc2f55f6700 1 -- 172.21.15.155:0/1922888464 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fc2dc0a1150 msgr2=0x7fc2dc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:38:59.237 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:38:59.208+0000 7fc2f55f6700 1 --2- 172.21.15.155:0/1922888464 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fc2dc0a1150 0x7fc2dc0a3600 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc2d8006fd0 tx=0x7fc2d8008040 comp rx=0 tx=0).stop 2023-07-21T18:38:59.237 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:38:59.208+0000 7fc2f55f6700 1 -- 172.21.15.155:0/1922888464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2f00ffce0 msgr2=0x7fc2f0195f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:38:59.237 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:38:59.208+0000 7fc2f55f6700 1 --2- 172.21.15.155:0/1922888464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2f00ffce0 0x7fc2f0195f70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc2e000d980 tx=0x7fc2e000dc90 comp rx=0 tx=0).stop 2023-07-21T18:38:59.238 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:38:59.208+0000 7fc2f55f6700 1 -- 172.21.15.155:0/1922888464 shutdown_connections 2023-07-21T18:38:59.238 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:38:59.208+0000 7fc2f55f6700 1 --2- 172.21.15.155:0/1922888464 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fc2dc0a1150 0x7fc2dc0a3600 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:38:59.238 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:38:59.208+0000 7fc2f55f6700 1 --2- 172.21.15.155:0/1922888464 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc2f0100670 0x7fc2f019cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:38:59.238 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:38:59.208+0000 7fc2f55f6700 1 --2- 172.21.15.155:0/1922888464 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2f00feae0 0x7fc2f0195a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:38:59.238 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:38:59.208+0000 7fc2f55f6700 1 --2- 172.21.15.155:0/1922888464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2f00ffce0 0x7fc2f0195f70 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:38:59.238 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:38:59.208+0000 7fc2f55f6700 1 -- 172.21.15.155:0/1922888464 >> 172.21.15.155:0/1922888464 conn(0x7fc2f00fa130 msgr2=0x7fc2f0068950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:38:59.238 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:38:59.208+0000 7fc2f55f6700 1 -- 172.21.15.155:0/1922888464 shutdown_connections 2023-07-21T18:38:59.239 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-21T18:38:59.208+0000 7fc2f55f6700 1 -- 172.21.15.155:0/1922888464 wait complete. 2023-07-21T18:38:59.631 DEBUG:teuthology.orchestra.run.smithi155:osd.10> sudo journalctl -f -n 0 -u ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@osd.10.service 2023-07-21T18:38:59.635 INFO:tasks.cephadm:Deploying osd.11 on smithi193 with /dev/vg_nvme/lv_1... 2023-07-21T18:38:59.636 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-07-21T18:38:59.668 INFO:journalctl@ceph.osd.10.smithi155.stdout:-- Logs begin at Fri 2023-07-21 18:17:11 UTC. -- 2023-07-21T18:39:00.126 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:59 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:00.126 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:59 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:00.127 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:59 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:39:00.127 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:59 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:39:00.127 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:59 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:00.127 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:59 smithi193 ceph-mon[98204]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T18:39:00.127 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:59 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:00.127 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:59 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:00.127 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:38:59 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:39:00.159 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:59 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:00.160 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:59 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:00.160 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:59 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:39:00.160 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:59 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:39:00.160 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:59 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:00.160 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:59 smithi155 ceph-mon[97934]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T18:39:00.160 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:59 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:00.161 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:59 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:00.161 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:38:59 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:39:00.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:59 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:00.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:59 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:00.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:59 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:39:00.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:59 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:39:00.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:59 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:00.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:59 smithi039 ceph-mon[96489]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T18:39:00.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:59 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:00.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:59 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:00.196 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:38:59 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:39:00.200 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T18:39:00.645 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 18:39:00 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T18:39:00.315+0000 7f80c905e640 -1 osd.10 0 log_to_monitors true 2023-07-21T18:39:01.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:00 smithi039 ceph-mon[96489]: from='osd.10 [v2:172.21.15.155:6824/1234615022,v1:172.21.15.155:6825/1234615022]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-21T18:39:01.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:00 smithi039 ceph-mon[96489]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-21T18:39:01.377 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:00 smithi193 ceph-mon[98204]: from='osd.10 [v2:172.21.15.155:6824/1234615022,v1:172.21.15.155:6825/1234615022]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-21T18:39:01.377 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:00 smithi193 ceph-mon[98204]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-21T18:39:01.395 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:00 smithi155 ceph-mon[97934]: from='osd.10 [v2:172.21.15.155:6824/1234615022,v1:172.21.15.155:6825/1234615022]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-21T18:39:01.395 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:00 smithi155 ceph-mon[97934]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-21T18:39:01.560 INFO:teuthology.orchestra.run.smithi193.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-07-21T18:39:01.561 INFO:teuthology.orchestra.run.smithi193.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-07-21T18:39:01.561 INFO:teuthology.orchestra.run.smithi193.stderr: stderr: 10+0 records in 2023-07-21T18:39:01.561 INFO:teuthology.orchestra.run.smithi193.stderr:10+0 records out 2023-07-21T18:39:01.562 INFO:teuthology.orchestra.run.smithi193.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.011503 s, 912 MB/s 2023-07-21T18:39:01.562 INFO:teuthology.orchestra.run.smithi193.stderr:--> Zapping successful for: 2023-07-21T18:39:01.955 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi193:vg_nvme/lv_1 2023-07-21T18:39:02.126 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:01 smithi193 ceph-mon[98204]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-07-21T18:39:02.127 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:01 smithi193 ceph-mon[98204]: osdmap e65: 11 total, 10 up, 11 in 2023-07-21T18:39:02.127 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:01 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T18:39:02.127 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:01 smithi193 ceph-mon[98204]: from='osd.10 [v2:172.21.15.155:6824/1234615022,v1:172.21.15.155:6825/1234615022]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2023-07-21T18:39:02.128 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:01 smithi193 ceph-mon[98204]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2023-07-21T18:39:02.128 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:01 smithi193 ceph-mon[98204]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T18:39:02.128 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:01 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:02.128 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:01 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:02.128 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:01 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:39:02.129 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:01 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:39:02.129 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:01 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:39:02.129 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:01 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:39:02.129 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:01 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:02.129 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:01 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:39:02.129 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:01 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:39:02.129 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:01 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:02.145 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 18:39:01 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T18:39:01.911+0000 7f80bded5700 -1 osd.10 0 waiting for initial osdmap 2023-07-21T18:39:02.145 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 18:39:01 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T18:39:01.926+0000 7f80b8cc8700 -1 osd.10 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-21T18:39:02.145 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:01 smithi155 ceph-mon[97934]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-07-21T18:39:02.145 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:01 smithi155 ceph-mon[97934]: osdmap e65: 11 total, 10 up, 11 in 2023-07-21T18:39:02.146 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:01 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T18:39:02.146 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:01 smithi155 ceph-mon[97934]: from='osd.10 [v2:172.21.15.155:6824/1234615022,v1:172.21.15.155:6825/1234615022]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2023-07-21T18:39:02.146 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:01 smithi155 ceph-mon[97934]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2023-07-21T18:39:02.146 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:01 smithi155 ceph-mon[97934]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T18:39:02.146 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:01 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:02.146 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:01 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:02.147 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:01 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:39:02.147 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:01 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:39:02.147 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:01 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:39:02.147 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:01 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:39:02.147 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:01 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:02.147 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:01 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:39:02.147 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:01 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:39:02.148 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:01 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:02.197 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:01 smithi039 ceph-mon[96489]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-07-21T18:39:02.197 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:01 smithi039 ceph-mon[96489]: osdmap e65: 11 total, 10 up, 11 in 2023-07-21T18:39:02.198 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:01 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T18:39:02.198 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:01 smithi039 ceph-mon[96489]: from='osd.10 [v2:172.21.15.155:6824/1234615022,v1:172.21.15.155:6825/1234615022]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2023-07-21T18:39:02.198 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:01 smithi039 ceph-mon[96489]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2023-07-21T18:39:02.199 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:01 smithi039 ceph-mon[96489]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T18:39:02.199 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:01 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:02.199 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:01 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:02.199 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:01 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:39:02.200 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:01 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:39:02.200 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:01 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:39:02.200 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:01 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:39:02.200 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:01 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:02.200 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:01 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:39:02.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:01 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:39:02.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:01 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:02.505 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T18:39:03.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:02 smithi039 ceph-mon[96489]: Detected new or changed devices on smithi155 2023-07-21T18:39:03.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:02 smithi039 ceph-mon[96489]: Adjusting osd_memory_target on smithi155 to 4292M 2023-07-21T18:39:03.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:02 smithi039 ceph-mon[96489]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2023-07-21T18:39:03.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:02 smithi039 ceph-mon[96489]: osdmap e66: 11 total, 10 up, 11 in 2023-07-21T18:39:03.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:02 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T18:39:03.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:02 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T18:39:03.376 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:02 smithi193 ceph-mon[98204]: Detected new or changed devices on smithi155 2023-07-21T18:39:03.376 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:02 smithi193 ceph-mon[98204]: Adjusting osd_memory_target on smithi155 to 4292M 2023-07-21T18:39:03.377 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:02 smithi193 ceph-mon[98204]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2023-07-21T18:39:03.377 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:02 smithi193 ceph-mon[98204]: osdmap e66: 11 total, 10 up, 11 in 2023-07-21T18:39:03.377 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:02 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T18:39:03.377 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:02 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T18:39:03.395 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:02 smithi155 ceph-mon[97934]: Detected new or changed devices on smithi155 2023-07-21T18:39:03.395 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:02 smithi155 ceph-mon[97934]: Adjusting osd_memory_target on smithi155 to 4292M 2023-07-21T18:39:03.395 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:02 smithi155 ceph-mon[97934]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2023-07-21T18:39:03.395 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:02 smithi155 ceph-mon[97934]: osdmap e66: 11 total, 10 up, 11 in 2023-07-21T18:39:03.395 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:02 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T18:39:03.395 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:02 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T18:39:03.579 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:03.550+0000 7f39ff4b5700 1 -- 172.21.15.193:0/201260556 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f39f81000a0 msgr2=0x7f39f81004b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:03.580 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:03.550+0000 7f39ff4b5700 1 --2- 172.21.15.193:0/201260556 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f39f81000a0 0x7f39f81004b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f39e8009990 tx=0x7f39e8009ca0 comp rx=0 tx=0).stop 2023-07-21T18:39:03.580 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:03.550+0000 7f39ff4b5700 1 -- 172.21.15.193:0/201260556 shutdown_connections 2023-07-21T18:39:03.580 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:03.550+0000 7f39ff4b5700 1 --2- 172.21.15.193:0/201260556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f39f80ff6b0 0x7f39f8106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:03.581 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:03.550+0000 7f39ff4b5700 1 --2- 172.21.15.193:0/201260556 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f39f81000a0 0x7f39f81004b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:03.581 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:03.550+0000 7f39ff4b5700 1 --2- 172.21.15.193:0/201260556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39f80fed00 0x7f39f80ff170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:03.581 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:03.550+0000 7f39ff4b5700 1 -- 172.21.15.193:0/201260556 >> 172.21.15.193:0/201260556 conn(0x7f39f80f9860 msgr2=0x7f39f80fbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:39:03.581 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:03.551+0000 7f39ff4b5700 1 -- 172.21.15.193:0/201260556 shutdown_connections 2023-07-21T18:39:03.582 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:03.551+0000 7f39ff4b5700 1 -- 172.21.15.193:0/201260556 wait complete. 2023-07-21T18:39:03.584 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:03.551+0000 7f39ff4b5700 1 Processor -- start 2023-07-21T18:39:03.585 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:03.551+0000 7f39ff4b5700 1 -- start start 2023-07-21T18:39:03.585 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:03.552+0000 7f39ff4b5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39f80fed00 0x7f39f80fe770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:03.585 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:03.552+0000 7f39ff4b5700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f39f80ff6b0 0x7f39f819a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:03.586 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:03.552+0000 7f39ff4b5700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f39f81000a0 0x7f39f819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:03.586 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:03.552+0000 7f39ff4b5700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f819d480 con 0x7f39f80fed00 2023-07-21T18:39:03.586 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:03.552+0000 7f39ff4b5700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f819d5f0 con 0x7f39f80ff6b0 2023-07-21T18:39:03.586 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:03.552+0000 7f39ff4b5700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f819d7b0 con 0x7f39f81000a0 2023-07-21T18:39:03.587 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:03.552+0000 7f39fda52700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f39f81000a0 0x7f39f819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:03.587 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:03.552+0000 7f39fda52700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f39f81000a0 0x7f39f819cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:50916/0 (socket says 172.21.15.193:50916) 2023-07-21T18:39:03.587 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:03.552+0000 7f39fda52700 1 -- 172.21.15.193:0/1563152479 learned_addr learned my addr 172.21.15.193:0/1563152479 (peer_addr_for_me v2:172.21.15.193:0/0) 2023-07-21T18:39:03.587 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:03.552+0000 7f39fd251700 1 --2- 172.21.15.193:0/1563152479 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39f80fed00 0x7f39f80fe770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:03.588 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:03.552+0000 7f39fca50700 1 --2- 172.21.15.193:0/1563152479 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f39f80ff6b0 0x7f39f819a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:03.588 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:03.552+0000 7f39fda52700 1 -- 172.21.15.193:0/1563152479 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f39f80ff6b0 msgr2=0x7f39f819a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:03.588 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:03.552+0000 7f39fda52700 1 --2- 172.21.15.193:0/1563152479 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f39f80ff6b0 0x7f39f819a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:03.589 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:03.552+0000 7f39fda52700 1 -- 172.21.15.193:0/1563152479 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39f80fed00 msgr2=0x7f39f80fe770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:03.589 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:03.552+0000 7f39fda52700 1 --2- 172.21.15.193:0/1563152479 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39f80fed00 0x7f39f80fe770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:03.589 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:03.552+0000 7f39fda52700 1 -- 172.21.15.193:0/1563152479 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39e80095f0 con 0x7f39f81000a0 2023-07-21T18:39:03.589 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:03.552+0000 7f39fda52700 1 --2- 172.21.15.193:0/1563152479 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f39f81000a0 0x7f39f819cce0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f39f0007ee0 tx=0x7f39f000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:39:03.590 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:03.552+0000 7f39ee7fc700 1 -- 172.21.15.193:0/1563152479 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f0017070 con 0x7f39f81000a0 2023-07-21T18:39:03.590 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:03.552+0000 7f39ee7fc700 1 -- 172.21.15.193:0/1563152479 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39f000f040 con 0x7f39f81000a0 2023-07-21T18:39:03.590 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:03.552+0000 7f39ee7fc700 1 -- 172.21.15.193:0/1563152479 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f0005740 con 0x7f39f81000a0 2023-07-21T18:39:03.590 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:03.552+0000 7f39ff4b5700 1 -- 172.21.15.193:0/1563152479 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39f819da90 con 0x7f39f81000a0 2023-07-21T18:39:03.591 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:03.552+0000 7f39ff4b5700 1 -- 172.21.15.193:0/1563152479 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39f819df40 con 0x7f39f81000a0 2023-07-21T18:39:03.591 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:03.553+0000 7f39ff4b5700 1 -- 172.21.15.193:0/1563152479 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39f810a2d0 con 0x7f39f81000a0 2023-07-21T18:39:03.591 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:03.554+0000 7f39ee7fc700 1 -- 172.21.15.193:0/1563152479 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f39f000dc60 con 0x7f39f81000a0 2023-07-21T18:39:03.591 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:03.555+0000 7f39ee7fc700 1 --2- 172.21.15.193:0/1563152479 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f39e40a1150 0x7f39e40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:03.591 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:03.555+0000 7f39ee7fc700 1 -- 172.21.15.193:0/1563152479 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6727+0+0 (secure 0 0 0) 0x7f39f00cbb20 con 0x7f39f81000a0 2023-07-21T18:39:03.592 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:03.555+0000 7f39fd251700 1 --2- 172.21.15.193:0/1563152479 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f39e40a1150 0x7f39e40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:03.592 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:03.556+0000 7f39fd251700 1 --2- 172.21.15.193:0/1563152479 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f39e40a1150 0x7f39e40a3600 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f39e800b5c0 tx=0x7f39e8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:39:03.592 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:03.559+0000 7f39ee7fc700 1 -- 172.21.15.193:0/1563152479 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f39f00958a0 con 0x7f39f81000a0 2023-07-21T18:39:03.785 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:03.756+0000 7f39ff4b5700 1 -- 172.21.15.193:0/1563152479 --> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f39f8060950 con 0x7f39e40a1150 2023-07-21T18:39:04.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:03 smithi039 ceph-mon[96489]: purged_snaps scrub starts 2023-07-21T18:39:04.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:03 smithi039 ceph-mon[96489]: purged_snaps scrub ok 2023-07-21T18:39:04.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:03 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T18:39:04.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:03 smithi039 ceph-mon[96489]: osd.10 [v2:172.21.15.155:6824/1234615022,v1:172.21.15.155:6825/1234615022] boot 2023-07-21T18:39:04.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:03 smithi039 ceph-mon[96489]: osdmap e67: 11 total, 11 up, 11 in 2023-07-21T18:39:04.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:03 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T18:39:04.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:03 smithi039 ceph-mon[96489]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T18:39:04.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:03 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T18:39:04.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:03 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T18:39:04.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:03 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:39:04.376 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:03 smithi193 ceph-mon[98204]: purged_snaps scrub starts 2023-07-21T18:39:04.376 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:03 smithi193 ceph-mon[98204]: purged_snaps scrub ok 2023-07-21T18:39:04.376 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:03 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T18:39:04.376 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:03 smithi193 ceph-mon[98204]: osd.10 [v2:172.21.15.155:6824/1234615022,v1:172.21.15.155:6825/1234615022] boot 2023-07-21T18:39:04.377 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:03 smithi193 ceph-mon[98204]: osdmap e67: 11 total, 11 up, 11 in 2023-07-21T18:39:04.377 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:03 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T18:39:04.377 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:03 smithi193 ceph-mon[98204]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T18:39:04.377 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:03 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T18:39:04.377 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:03 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T18:39:04.377 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:03 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:39:04.394 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:03 smithi155 ceph-mon[97934]: purged_snaps scrub starts 2023-07-21T18:39:04.395 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:03 smithi155 ceph-mon[97934]: purged_snaps scrub ok 2023-07-21T18:39:04.395 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:03 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T18:39:04.395 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:03 smithi155 ceph-mon[97934]: osd.10 [v2:172.21.15.155:6824/1234615022,v1:172.21.15.155:6825/1234615022] boot 2023-07-21T18:39:04.395 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:03 smithi155 ceph-mon[97934]: osdmap e67: 11 total, 11 up, 11 in 2023-07-21T18:39:04.395 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:03 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T18:39:04.396 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:03 smithi155 ceph-mon[97934]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T18:39:04.396 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:03 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T18:39:04.396 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:03 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T18:39:04.396 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:03 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:39:05.307 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:04 smithi193 ceph-mon[98204]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:39:05.307 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:04 smithi193 ceph-mon[98204]: osdmap e68: 11 total, 11 up, 11 in 2023-07-21T18:39:05.394 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:04 smithi155 ceph-mon[97934]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:39:05.395 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:04 smithi155 ceph-mon[97934]: osdmap e68: 11 total, 11 up, 11 in 2023-07-21T18:39:05.443 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:04 smithi039 ceph-mon[96489]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:39:05.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:04 smithi039 ceph-mon[96489]: osdmap e68: 11 total, 11 up, 11 in 2023-07-21T18:39:06.376 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:05 smithi193 ceph-mon[98204]: osdmap e69: 11 total, 11 up, 11 in 2023-07-21T18:39:06.376 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:05 smithi193 ceph-mon[98204]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-21T18:39:06.394 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:05 smithi155 ceph-mon[97934]: osdmap e69: 11 total, 11 up, 11 in 2023-07-21T18:39:06.395 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:05 smithi155 ceph-mon[97934]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-21T18:39:06.443 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:05 smithi039 ceph-mon[96489]: osdmap e69: 11 total, 11 up, 11 in 2023-07-21T18:39:06.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:05 smithi039 ceph-mon[96489]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-21T18:39:07.376 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:06 smithi193 ceph-mon[98204]: osdmap e70: 11 total, 11 up, 11 in 2023-07-21T18:39:07.376 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:06 smithi193 ceph-mon[98204]: from='client.? 172.21.15.193:0/1826078747' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "efb31ea1-4775-4e5f-8740-728d6f4aeea6"}]: dispatch 2023-07-21T18:39:07.376 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:06 smithi193 ceph-mon[98204]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "efb31ea1-4775-4e5f-8740-728d6f4aeea6"}]: dispatch 2023-07-21T18:39:07.376 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:06 smithi193 ceph-mon[98204]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "efb31ea1-4775-4e5f-8740-728d6f4aeea6"}]': finished 2023-07-21T18:39:07.376 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:06 smithi193 ceph-mon[98204]: osdmap e71: 12 total, 11 up, 12 in 2023-07-21T18:39:07.377 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:06 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T18:39:07.378 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:06 smithi193 ceph-mon[98204]: from='client.? 172.21.15.193:0/2816544315' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T18:39:07.394 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:06 smithi155 ceph-mon[97934]: osdmap e70: 11 total, 11 up, 11 in 2023-07-21T18:39:07.395 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:06 smithi155 ceph-mon[97934]: from='client.? 172.21.15.193:0/1826078747' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "efb31ea1-4775-4e5f-8740-728d6f4aeea6"}]: dispatch 2023-07-21T18:39:07.395 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:06 smithi155 ceph-mon[97934]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "efb31ea1-4775-4e5f-8740-728d6f4aeea6"}]: dispatch 2023-07-21T18:39:07.395 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:06 smithi155 ceph-mon[97934]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "efb31ea1-4775-4e5f-8740-728d6f4aeea6"}]': finished 2023-07-21T18:39:07.395 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:06 smithi155 ceph-mon[97934]: osdmap e71: 12 total, 11 up, 12 in 2023-07-21T18:39:07.395 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:06 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T18:39:07.395 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:06 smithi155 ceph-mon[97934]: from='client.? 172.21.15.193:0/2816544315' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T18:39:07.443 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:39:07 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:39:07.077+0000 7fe45a530700 -1 calc_pg_upmaps abort due to max <= 0 2023-07-21T18:39:07.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:06 smithi039 ceph-mon[96489]: osdmap e70: 11 total, 11 up, 11 in 2023-07-21T18:39:07.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:06 smithi039 ceph-mon[96489]: from='client.? 172.21.15.193:0/1826078747' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "efb31ea1-4775-4e5f-8740-728d6f4aeea6"}]: dispatch 2023-07-21T18:39:07.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:06 smithi039 ceph-mon[96489]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "efb31ea1-4775-4e5f-8740-728d6f4aeea6"}]: dispatch 2023-07-21T18:39:07.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:06 smithi039 ceph-mon[96489]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "efb31ea1-4775-4e5f-8740-728d6f4aeea6"}]': finished 2023-07-21T18:39:07.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:06 smithi039 ceph-mon[96489]: osdmap e71: 12 total, 11 up, 12 in 2023-07-21T18:39:07.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:06 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T18:39:07.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:06 smithi039 ceph-mon[96489]: from='client.? 172.21.15.193:0/2816544315' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T18:39:08.375 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:07 smithi193 ceph-mon[98204]: pgmap v211: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-21T18:39:08.394 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:07 smithi155 ceph-mon[97934]: pgmap v211: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-21T18:39:08.443 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:07 smithi039 ceph-mon[96489]: pgmap v211: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-21T18:39:09.625 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:09 smithi193 ceph-mon[98204]: pgmap v212: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 111 KiB/s, 0 objects/s recovering 2023-07-21T18:39:09.644 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:09 smithi155 ceph-mon[97934]: pgmap v212: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 111 KiB/s, 0 objects/s recovering 2023-07-21T18:39:09.693 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:09 smithi039 ceph-mon[96489]: pgmap v212: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 111 KiB/s, 0 objects/s recovering 2023-07-21T18:39:12.125 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:11 smithi193 ceph-mon[98204]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 93 KiB/s, 0 objects/s recovering 2023-07-21T18:39:12.394 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:11 smithi155 ceph-mon[97934]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 93 KiB/s, 0 objects/s recovering 2023-07-21T18:39:12.443 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:11 smithi039 ceph-mon[96489]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 93 KiB/s, 0 objects/s recovering 2023-07-21T18:39:13.294 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:12 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-07-21T18:39:13.295 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:12 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:39:13.394 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:12 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-07-21T18:39:13.395 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:12 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:39:13.443 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:12 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-07-21T18:39:13.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:12 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:39:14.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:13 smithi193 ceph-mon[98204]: Deploying daemon osd.11 on smithi193 2023-07-21T18:39:14.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:13 smithi193 ceph-mon[98204]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-07-21T18:39:14.394 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:13 smithi155 ceph-mon[97934]: Deploying daemon osd.11 on smithi193 2023-07-21T18:39:14.394 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:13 smithi155 ceph-mon[97934]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-07-21T18:39:14.443 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:13 smithi039 ceph-mon[96489]: Deploying daemon osd.11 on smithi193 2023-07-21T18:39:14.443 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:13 smithi039 ceph-mon[96489]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-07-21T18:39:15.376 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:15 smithi193 ceph-mon[98204]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-21T18:39:15.644 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:15 smithi155 ceph-mon[97934]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-21T18:39:15.693 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:15 smithi039 ceph-mon[96489]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-21T18:39:17.125 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:16 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:39:17.126 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:16 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:17.126 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:16 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:17.394 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:16 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:39:17.394 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:16 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:17.394 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:16 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:17.443 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:16 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:39:17.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:16 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:17.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:16 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:18.235 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:17 smithi193 ceph-mon[98204]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-21T18:39:18.394 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:17 smithi155 ceph-mon[97934]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-21T18:39:18.443 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:17 smithi039 ceph-mon[96489]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-21T18:39:19.488 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:19 smithi193 ceph-mon[98204]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-21T18:39:19.596 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:19.568+0000 7f39ee7fc700 1 -- 172.21.15.193:0/1563152479 <== mgr.14152 v2:172.21.15.39:6800/2116291712 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f39f8060950 con 0x7f39e40a1150 2023-07-21T18:39:19.597 INFO:teuthology.orchestra.run.smithi193.stdout:Created osd(s) 11 on host 'smithi193' 2023-07-21T18:39:19.598 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:19.570+0000 7f39ff4b5700 1 -- 172.21.15.193:0/1563152479 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f39e40a1150 msgr2=0x7f39e40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:19.599 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:19.570+0000 7f39ff4b5700 1 --2- 172.21.15.193:0/1563152479 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f39e40a1150 0x7f39e40a3600 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f39e800b5c0 tx=0x7f39e8005fd0 comp rx=0 tx=0).stop 2023-07-21T18:39:19.599 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:19.571+0000 7f39ff4b5700 1 -- 172.21.15.193:0/1563152479 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f39f81000a0 msgr2=0x7f39f819cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:19.599 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:19.571+0000 7f39ff4b5700 1 --2- 172.21.15.193:0/1563152479 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f39f81000a0 0x7f39f819cce0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f39f0007ee0 tx=0x7f39f000d3b0 comp rx=0 tx=0).stop 2023-07-21T18:39:19.599 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:19.571+0000 7f39ff4b5700 1 -- 172.21.15.193:0/1563152479 shutdown_connections 2023-07-21T18:39:19.599 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:19.571+0000 7f39ff4b5700 1 --2- 172.21.15.193:0/1563152479 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f39e40a1150 0x7f39e40a3600 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:19.599 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:19.571+0000 7f39ff4b5700 1 --2- 172.21.15.193:0/1563152479 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f39f81000a0 0x7f39f819cce0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:19.599 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:19.571+0000 7f39ff4b5700 1 --2- 172.21.15.193:0/1563152479 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39f80fed00 0x7f39f80fe770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:19.600 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:19.571+0000 7f39ff4b5700 1 --2- 172.21.15.193:0/1563152479 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f39f80ff6b0 0x7f39f819a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:19.600 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:19.571+0000 7f39ff4b5700 1 -- 172.21.15.193:0/1563152479 >> 172.21.15.193:0/1563152479 conn(0x7f39f80f9860 msgr2=0x7f39f80fbac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:39:19.600 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:19.571+0000 7f39ff4b5700 1 -- 172.21.15.193:0/1563152479 shutdown_connections 2023-07-21T18:39:19.600 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:19.571+0000 7f39ff4b5700 1 -- 172.21.15.193:0/1563152479 wait complete. 2023-07-21T18:39:19.644 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:19 smithi155 ceph-mon[97934]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-21T18:39:19.693 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:19 smithi039 ceph-mon[96489]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-21T18:39:19.895 DEBUG:teuthology.orchestra.run.smithi193:osd.11> sudo journalctl -f -n 0 -u ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@osd.11.service 2023-07-21T18:39:19.899 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2023-07-21T18:39:19.899 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph osd stat -f json 2023-07-21T18:39:19.927 INFO:journalctl@ceph.osd.11.smithi193.stdout:-- Logs begin at Fri 2023-07-21 18:17:09 UTC. -- 2023-07-21T18:39:20.476 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:39:20.747 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:20 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:20.747 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:20 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:20.748 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:20 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:20.748 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:20 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:20.862 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:20 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:20.862 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:20 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:20.862 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:20 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:20.863 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:20 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:20.894 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:20 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:20.894 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:20 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:20.895 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:20 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:20.895 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:20 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:21.125 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 18:39:20 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T18:39:20.792+0000 7fc9653d2640 -1 osd.11 0 log_to_monitors true 2023-07-21T18:39:21.651 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:21.622+0000 7f12c0234700 1 -- 172.21.15.39:0/3074778331 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12b80ffc80 msgr2=0x7f12b81000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:21.652 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:21.622+0000 7f12c0234700 1 --2- 172.21.15.39:0/3074778331 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12b80ffc80 0x7f12b81000d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f12a8009990 tx=0x7f12a8009ca0 comp rx=0 tx=0).stop 2023-07-21T18:39:21.652 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:21.623+0000 7f12c0234700 1 -- 172.21.15.39:0/3074778331 shutdown_connections 2023-07-21T18:39:21.652 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:21.623+0000 7f12c0234700 1 --2- 172.21.15.39:0/3074778331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12b80fea80 0x7f12b80fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:21.652 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:21.623+0000 7f12c0234700 1 --2- 172.21.15.39:0/3074778331 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12b8100610 0x7f12b8106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:21.652 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:21.623+0000 7f12c0234700 1 --2- 172.21.15.39:0/3074778331 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12b80ffc80 0x7f12b81000d0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:21.652 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:21.623+0000 7f12c0234700 1 -- 172.21.15.39:0/3074778331 >> 172.21.15.39:0/3074778331 conn(0x7f12b80fa0b0 msgr2=0x7f12b80fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:39:21.652 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:21.623+0000 7f12c0234700 1 -- 172.21.15.39:0/3074778331 shutdown_connections 2023-07-21T18:39:21.653 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:21.623+0000 7f12c0234700 1 -- 172.21.15.39:0/3074778331 wait complete. 2023-07-21T18:39:21.653 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:21.623+0000 7f12c0234700 1 Processor -- start 2023-07-21T18:39:21.653 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:21.623+0000 7f12c0234700 1 -- start start 2023-07-21T18:39:21.653 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:21.623+0000 7f12c0234700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12b80fea80 0x7f12b8068b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:21.653 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:21.623+0000 7f12c0234700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12b80ffc80 0x7f12b819c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:21.653 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:21.623+0000 7f12c0234700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12b8100610 0x7f12b819ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:21.653 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:21.624+0000 7f12c0234700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b819f5d0 con 0x7f12b80ffc80 2023-07-21T18:39:21.654 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:21.624+0000 7f12c0234700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b819f740 con 0x7f12b80fea80 2023-07-21T18:39:21.654 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:21.624+0000 7f12c0234700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b819f900 con 0x7f12b8100610 2023-07-21T18:39:21.654 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:21.624+0000 7f12bd7cf700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12b80ffc80 0x7f12b819c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:21.654 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:21.624+0000 7f12bd7cf700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12b80ffc80 0x7f12b819c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54620/0 (socket says 172.21.15.39:54620) 2023-07-21T18:39:21.654 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:21.624+0000 7f12bd7cf700 1 -- 172.21.15.39:0/2447018310 learned_addr learned my addr 172.21.15.39:0/2447018310 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:39:21.655 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:21.624+0000 7f12bdfd0700 1 --2- 172.21.15.39:0/2447018310 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12b80fea80 0x7f12b8068b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:21.655 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:21.624+0000 7f12be7d1700 1 --2- 172.21.15.39:0/2447018310 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12b8100610 0x7f12b819ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:21.655 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:21.624+0000 7f12bd7cf700 1 -- 172.21.15.39:0/2447018310 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12b8100610 msgr2=0x7f12b819ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:21.655 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:21.624+0000 7f12bd7cf700 1 --2- 172.21.15.39:0/2447018310 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12b8100610 0x7f12b819ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:21.655 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:21.624+0000 7f12bd7cf700 1 -- 172.21.15.39:0/2447018310 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12b80fea80 msgr2=0x7f12b8068b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:21.655 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:21.624+0000 7f12bd7cf700 1 --2- 172.21.15.39:0/2447018310 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12b80fea80 0x7f12b8068b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:21.655 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:21.624+0000 7f12bd7cf700 1 -- 172.21.15.39:0/2447018310 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a80095f0 con 0x7f12b80ffc80 2023-07-21T18:39:21.655 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:21.624+0000 7f12bd7cf700 1 --2- 172.21.15.39:0/2447018310 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12b80ffc80 0x7f12b819c9d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f12a8000c00 tx=0x7f12a800f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:39:21.656 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:21.624+0000 7f12aeffd700 1 -- 172.21.15.39:0/2447018310 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a801d070 con 0x7f12b80ffc80 2023-07-21T18:39:21.657 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:21.624+0000 7f12aeffd700 1 -- 172.21.15.39:0/2447018310 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f12a800fb50 con 0x7f12b80ffc80 2023-07-21T18:39:21.657 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:21.624+0000 7f12aeffd700 1 -- 172.21.15.39:0/2447018310 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a80173f0 con 0x7f12b80ffc80 2023-07-21T18:39:21.657 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:21.624+0000 7f12c0234700 1 -- 172.21.15.39:0/2447018310 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12b819fb80 con 0x7f12b80ffc80 2023-07-21T18:39:21.657 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:21.624+0000 7f12c0234700 1 -- 172.21.15.39:0/2447018310 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12b819ffd0 con 0x7f12b80ffc80 2023-07-21T18:39:21.657 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:21.625+0000 7f12c0234700 1 -- 172.21.15.39:0/2447018310 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12b8106080 con 0x7f12b80ffc80 2023-07-21T18:39:21.657 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:21.626+0000 7f12aeffd700 1 -- 172.21.15.39:0/2447018310 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f12a800fcc0 con 0x7f12b80ffc80 2023-07-21T18:39:21.658 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:21.627+0000 7f12aeffd700 1 --2- 172.21.15.39:0/2447018310 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f12a40a1150 0x7f12a40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:21.658 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:21.627+0000 7f12bdfd0700 1 --2- 172.21.15.39:0/2447018310 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f12a40a1150 0x7f12a40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:21.658 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:21.627+0000 7f12aeffd700 1 -- 172.21.15.39:0/2447018310 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6831+0+0 (secure 0 0 0) 0x7f12a80cd9b0 con 0x7f12b80ffc80 2023-07-21T18:39:21.658 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:21.627+0000 7f12bdfd0700 1 --2- 172.21.15.39:0/2447018310 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f12a40a1150 0x7f12a40a3600 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f12b80ffae0 tx=0x7f12b400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:39:21.658 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:21.630+0000 7f12aeffd700 1 -- 172.21.15.39:0/2447018310 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f12a8097760 con 0x7f12b80ffc80 2023-07-21T18:39:21.811 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:21 smithi193 ceph-mon[98204]: from='osd.11 [v2:172.21.15.193:6824/215104663,v1:172.21.15.193:6825/215104663]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-21T18:39:21.812 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:21 smithi193 ceph-mon[98204]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-21T18:39:21.812 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:21 smithi193 ceph-mon[98204]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-21T18:39:21.855 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:21.827+0000 7f12c0234700 1 -- 172.21.15.39:0/2447018310 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f12b802caa0 con 0x7f12b80ffc80 2023-07-21T18:39:21.855 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:21.827+0000 7f12aeffd700 1 -- 172.21.15.39:0/2447018310 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v72) v1 ==== 74+0+133 (secure 0 0 0) 0x7f12a809ad80 con 0x7f12b80ffc80 2023-07-21T18:39:21.856 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:39:21.858 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:21.829+0000 7f12c0234700 1 -- 172.21.15.39:0/2447018310 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f12a40a1150 msgr2=0x7f12a40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:21.858 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:21.829+0000 7f12c0234700 1 --2- 172.21.15.39:0/2447018310 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f12a40a1150 0x7f12a40a3600 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f12b80ffae0 tx=0x7f12b400b410 comp rx=0 tx=0).stop 2023-07-21T18:39:21.858 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:21.829+0000 7f12c0234700 1 -- 172.21.15.39:0/2447018310 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12b80ffc80 msgr2=0x7f12b819c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:21.859 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:21.829+0000 7f12c0234700 1 --2- 172.21.15.39:0/2447018310 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12b80ffc80 0x7f12b819c9d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f12a8000c00 tx=0x7f12a800f710 comp rx=0 tx=0).stop 2023-07-21T18:39:21.859 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:21.829+0000 7f12c0234700 1 -- 172.21.15.39:0/2447018310 shutdown_connections 2023-07-21T18:39:21.859 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:21.829+0000 7f12c0234700 1 --2- 172.21.15.39:0/2447018310 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f12a40a1150 0x7f12a40a3600 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:21.859 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:21.829+0000 7f12c0234700 1 --2- 172.21.15.39:0/2447018310 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12b80fea80 0x7f12b8068b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:21.860 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:21.829+0000 7f12c0234700 1 --2- 172.21.15.39:0/2447018310 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12b8100610 0x7f12b819ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:21.860 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:21.829+0000 7f12c0234700 1 --2- 172.21.15.39:0/2447018310 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12b80ffc80 0x7f12b819c9d0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:21.860 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:21.829+0000 7f12c0234700 1 -- 172.21.15.39:0/2447018310 >> 172.21.15.39:0/2447018310 conn(0x7f12b80fa0b0 msgr2=0x7f12b81031c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:39:21.861 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:21.830+0000 7f12c0234700 1 -- 172.21.15.39:0/2447018310 shutdown_connections 2023-07-21T18:39:21.861 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:21.830+0000 7f12c0234700 1 -- 172.21.15.39:0/2447018310 wait complete. 2023-07-21T18:39:21.881 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:21 smithi039 ceph-mon[96489]: from='osd.11 [v2:172.21.15.193:6824/215104663,v1:172.21.15.193:6825/215104663]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-21T18:39:21.882 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:21 smithi039 ceph-mon[96489]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-21T18:39:21.882 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:21 smithi039 ceph-mon[96489]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-21T18:39:21.894 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:21 smithi155 ceph-mon[97934]: from='osd.11 [v2:172.21.15.193:6824/215104663,v1:172.21.15.193:6825/215104663]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-21T18:39:21.894 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:21 smithi155 ceph-mon[97934]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-21T18:39:21.894 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:21 smithi155 ceph-mon[97934]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-21T18:39:22.222 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":72,"num_osds":12,"num_up_osds":11,"osd_up_since":1689964742,"num_in_osds":12,"osd_in_since":1689964745,"num_remapped_pgs":0} 2023-07-21T18:39:22.875 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 18:39:22 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T18:39:22.536+0000 7fc95a249700 -1 osd.11 0 waiting for initial osdmap 2023-07-21T18:39:22.875 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 18:39:22 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T18:39:22.551+0000 7fc95483b700 -1 osd.11 73 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-21T18:39:22.876 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:22 smithi193 ceph-mon[98204]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-07-21T18:39:22.876 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:22 smithi193 ceph-mon[98204]: osdmap e72: 12 total, 11 up, 12 in 2023-07-21T18:39:22.876 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:22 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T18:39:22.876 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:22 smithi193 ceph-mon[98204]: from='osd.11 [v2:172.21.15.193:6824/215104663,v1:172.21.15.193:6825/215104663]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2023-07-21T18:39:22.876 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:22 smithi193 ceph-mon[98204]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2023-07-21T18:39:22.877 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:22 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/2447018310' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-21T18:39:22.877 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:22 smithi193 ceph-mon[98204]: Detected new or changed devices on smithi193 2023-07-21T18:39:22.877 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:22 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:22.877 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:22 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:22.877 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:22 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:39:22.877 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:22 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:39:22.877 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:22 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:39:22.877 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:22 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:39:22.878 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:22 smithi193 ceph-mon[98204]: Adjusting osd_memory_target on smithi193 to 4292M 2023-07-21T18:39:22.878 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:22 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:22.878 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:22 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:39:22.878 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:22 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:39:22.878 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:22 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:22.878 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:22 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:39:22.878 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:22 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:39:22.879 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:22 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:39:22.879 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:22 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:22.894 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:22 smithi155 ceph-mon[97934]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-07-21T18:39:22.894 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:22 smithi155 ceph-mon[97934]: osdmap e72: 12 total, 11 up, 12 in 2023-07-21T18:39:22.894 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:22 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T18:39:22.895 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:22 smithi155 ceph-mon[97934]: from='osd.11 [v2:172.21.15.193:6824/215104663,v1:172.21.15.193:6825/215104663]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2023-07-21T18:39:22.895 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:22 smithi155 ceph-mon[97934]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2023-07-21T18:39:22.895 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:22 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/2447018310' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-21T18:39:22.895 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:22 smithi155 ceph-mon[97934]: Detected new or changed devices on smithi193 2023-07-21T18:39:22.895 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:22 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:22.895 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:22 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:22.895 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:22 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:39:22.896 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:22 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:39:22.896 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:22 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:39:22.896 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:22 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:39:22.896 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:22 smithi155 ceph-mon[97934]: Adjusting osd_memory_target on smithi193 to 4292M 2023-07-21T18:39:22.896 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:22 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:22.896 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:22 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:39:22.896 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:22 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:39:22.897 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:22 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:22.897 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:22 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:39:22.897 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:22 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:39:22.897 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:22 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:39:22.897 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:22 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:22.943 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:22 smithi039 ceph-mon[96489]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-07-21T18:39:22.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:22 smithi039 ceph-mon[96489]: osdmap e72: 12 total, 11 up, 12 in 2023-07-21T18:39:22.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:22 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T18:39:22.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:22 smithi039 ceph-mon[96489]: from='osd.11 [v2:172.21.15.193:6824/215104663,v1:172.21.15.193:6825/215104663]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2023-07-21T18:39:22.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:22 smithi039 ceph-mon[96489]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2023-07-21T18:39:22.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:22 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/2447018310' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-21T18:39:22.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:22 smithi039 ceph-mon[96489]: Detected new or changed devices on smithi193 2023-07-21T18:39:22.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:22 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:22.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:22 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:22.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:22 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:39:22.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:22 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:39:22.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:22 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:39:22.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:22 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:39:22.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:22 smithi039 ceph-mon[96489]: Adjusting osd_memory_target on smithi193 to 4292M 2023-07-21T18:39:22.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:22 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:22.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:22 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:39:22.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:22 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:39:22.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:22 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:22.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:22 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:39:22.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:22 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:39:22.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:22 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:39:22.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:22 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:23.224 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph osd stat -f json 2023-07-21T18:39:23.804 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:39:23.875 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:23 smithi193 ceph-mon[98204]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2023-07-21T18:39:23.876 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:23 smithi193 ceph-mon[98204]: osdmap e73: 12 total, 11 up, 12 in 2023-07-21T18:39:23.876 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:23 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T18:39:23.876 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:23 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T18:39:23.876 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:23 smithi193 ceph-mon[98204]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-21T18:39:23.894 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:23 smithi155 ceph-mon[97934]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2023-07-21T18:39:23.894 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:23 smithi155 ceph-mon[97934]: osdmap e73: 12 total, 11 up, 12 in 2023-07-21T18:39:23.895 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:23 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T18:39:23.895 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:23 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T18:39:23.895 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:23 smithi155 ceph-mon[97934]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-21T18:39:23.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:23 smithi039 ceph-mon[96489]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2023-07-21T18:39:23.943 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:23 smithi039 ceph-mon[96489]: osdmap e73: 12 total, 11 up, 12 in 2023-07-21T18:39:23.943 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:23 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T18:39:23.943 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:23 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T18:39:23.943 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:23 smithi039 ceph-mon[96489]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-21T18:39:24.863 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:24.834+0000 7f5663170700 1 -- 172.21.15.39:0/2075448078 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f565c100dc0 msgr2=0x7f565c101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:24.863 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:24.834+0000 7f5663170700 1 --2- 172.21.15.39:0/2075448078 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f565c100dc0 0x7f565c101210 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5648009990 tx=0x7f5648009ca0 comp rx=0 tx=0).stop 2023-07-21T18:39:24.863 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:24.834+0000 7f5663170700 1 -- 172.21.15.39:0/2075448078 shutdown_connections 2023-07-21T18:39:24.864 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:24.834+0000 7f5663170700 1 --2- 172.21.15.39:0/2075448078 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f565c0ffbc0 0x7f565c0fffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:24.864 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:24.834+0000 7f5663170700 1 --2- 172.21.15.39:0/2075448078 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f565c101750 0x7f565c108010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:24.864 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:24.834+0000 7f5663170700 1 --2- 172.21.15.39:0/2075448078 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f565c100dc0 0x7f565c101210 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:24.864 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:24.834+0000 7f5663170700 1 -- 172.21.15.39:0/2075448078 >> 172.21.15.39:0/2075448078 conn(0x7f565c0fb230 msgr2=0x7f565c0fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:39:24.864 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:24.834+0000 7f5663170700 1 -- 172.21.15.39:0/2075448078 shutdown_connections 2023-07-21T18:39:24.864 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:24.834+0000 7f5663170700 1 -- 172.21.15.39:0/2075448078 wait complete. 2023-07-21T18:39:24.865 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:24.835+0000 7f5663170700 1 Processor -- start 2023-07-21T18:39:24.865 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:24.835+0000 7f5663170700 1 -- start start 2023-07-21T18:39:24.865 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:24.835+0000 7f5663170700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f565c0ffbc0 0x7f565c195870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:24.865 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:24.835+0000 7f5663170700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f565c100dc0 0x7f565c195db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:24.865 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:24.835+0000 7f5663170700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f565c101750 0x7f565c19ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:24.865 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:24.835+0000 7f5663170700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f565c19d400 con 0x7f565c100dc0 2023-07-21T18:39:24.865 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:24.835+0000 7f5663170700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f565c19d540 con 0x7f565c0ffbc0 2023-07-21T18:39:24.865 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:24.835+0000 7f5663170700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f565c19d6d0 con 0x7f565c101750 2023-07-21T18:39:24.866 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:24.835+0000 7f565bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f565c100dc0 0x7f565c195db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:24.866 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:24.835+0000 7f565bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f565c100dc0 0x7f565c195db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54646/0 (socket says 172.21.15.39:54646) 2023-07-21T18:39:24.866 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:24.835+0000 7f565bfff700 1 -- 172.21.15.39:0/2501908238 learned_addr learned my addr 172.21.15.39:0/2501908238 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:39:24.867 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:24.835+0000 7f566170d700 1 --2- 172.21.15.39:0/2501908238 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f565c101750 0x7f565c19ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:24.867 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:24.835+0000 7f565bfff700 1 -- 172.21.15.39:0/2501908238 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f565c101750 msgr2=0x7f565c19ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:24.867 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:24.835+0000 7f565bfff700 1 --2- 172.21.15.39:0/2501908238 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f565c101750 0x7f565c19ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:24.867 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:24.835+0000 7f5660f0c700 1 --2- 172.21.15.39:0/2501908238 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f565c0ffbc0 0x7f565c195870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:24.867 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:24.835+0000 7f565bfff700 1 -- 172.21.15.39:0/2501908238 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f565c0ffbc0 msgr2=0x7f565c195870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:24.867 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:24.835+0000 7f565bfff700 1 --2- 172.21.15.39:0/2501908238 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f565c0ffbc0 0x7f565c195870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:24.867 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:24.835+0000 7f565bfff700 1 -- 172.21.15.39:0/2501908238 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56480095f0 con 0x7f565c100dc0 2023-07-21T18:39:24.867 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:24.835+0000 7f566170d700 1 --2- 172.21.15.39:0/2501908238 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f565c101750 0x7f565c19ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:39:24.868 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:24.835+0000 7f565bfff700 1 --2- 172.21.15.39:0/2501908238 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f565c100dc0 0x7f565c195db0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5648005910 tx=0x7f564800f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:39:24.868 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:24.836+0000 7f5659ffb700 1 -- 172.21.15.39:0/2501908238 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564801d070 con 0x7f565c100dc0 2023-07-21T18:39:24.868 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:24.836+0000 7f5659ffb700 1 -- 172.21.15.39:0/2501908238 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f564800fbc0 con 0x7f565c100dc0 2023-07-21T18:39:24.868 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:24.836+0000 7f5663170700 1 -- 172.21.15.39:0/2501908238 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f565c19d950 con 0x7f565c100dc0 2023-07-21T18:39:24.868 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:24.836+0000 7f5659ffb700 1 -- 172.21.15.39:0/2501908238 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5648017650 con 0x7f565c100dc0 2023-07-21T18:39:24.868 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:24.836+0000 7f5663170700 1 -- 172.21.15.39:0/2501908238 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f565c19de40 con 0x7f565c100dc0 2023-07-21T18:39:24.869 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:24.837+0000 7f5663170700 1 -- 172.21.15.39:0/2501908238 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f565c10b380 con 0x7f565c100dc0 2023-07-21T18:39:24.869 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:24.838+0000 7f5659ffb700 1 -- 172.21.15.39:0/2501908238 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5648022470 con 0x7f565c100dc0 2023-07-21T18:39:24.870 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:24.838+0000 7f5659ffb700 1 --2- 172.21.15.39:0/2501908238 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f564c0a11d0 0x7f564c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:24.870 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:24.839+0000 7f5660f0c700 1 --2- 172.21.15.39:0/2501908238 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f564c0a11d0 0x7f564c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:24.870 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:24.839+0000 7f5659ffb700 1 -- 172.21.15.39:0/2501908238 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f56480cdaf0 con 0x7f565c100dc0 2023-07-21T18:39:24.870 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:24.839+0000 7f5660f0c700 1 --2- 172.21.15.39:0/2501908238 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f564c0a11d0 0x7f564c0a3680 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5650005950 tx=0x7f56500058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:39:24.870 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:24.842+0000 7f5659ffb700 1 -- 172.21.15.39:0/2501908238 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f56480976d0 con 0x7f565c100dc0 2023-07-21T18:39:24.875 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:24 smithi193 ceph-mon[98204]: purged_snaps scrub starts 2023-07-21T18:39:24.875 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:24 smithi193 ceph-mon[98204]: purged_snaps scrub ok 2023-07-21T18:39:24.875 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:24 smithi193 ceph-mon[98204]: osd.11 [v2:172.21.15.193:6824/215104663,v1:172.21.15.193:6825/215104663] boot 2023-07-21T18:39:24.875 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:24 smithi193 ceph-mon[98204]: osdmap e74: 12 total, 12 up, 12 in 2023-07-21T18:39:24.876 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:24 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T18:39:24.894 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:24 smithi155 ceph-mon[97934]: purged_snaps scrub starts 2023-07-21T18:39:24.894 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:24 smithi155 ceph-mon[97934]: purged_snaps scrub ok 2023-07-21T18:39:24.894 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:24 smithi155 ceph-mon[97934]: osd.11 [v2:172.21.15.193:6824/215104663,v1:172.21.15.193:6825/215104663] boot 2023-07-21T18:39:24.894 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:24 smithi155 ceph-mon[97934]: osdmap e74: 12 total, 12 up, 12 in 2023-07-21T18:39:24.895 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:24 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T18:39:24.943 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:24 smithi039 ceph-mon[96489]: purged_snaps scrub starts 2023-07-21T18:39:24.943 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:24 smithi039 ceph-mon[96489]: purged_snaps scrub ok 2023-07-21T18:39:24.943 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:24 smithi039 ceph-mon[96489]: osd.11 [v2:172.21.15.193:6824/215104663,v1:172.21.15.193:6825/215104663] boot 2023-07-21T18:39:24.943 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:24 smithi039 ceph-mon[96489]: osdmap e74: 12 total, 12 up, 12 in 2023-07-21T18:39:24.943 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:24 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T18:39:25.068 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:39:25.069 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:25.040+0000 7f5663170700 1 -- 172.21.15.39:0/2501908238 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f565c10b570 con 0x7f565c100dc0 2023-07-21T18:39:25.069 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:25.041+0000 7f5659ffb700 1 -- 172.21.15.39:0/2501908238 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v75) v1 ==== 74+0+133 (secure 0 0 0) 0x7f564809acf0 con 0x7f565c100dc0 2023-07-21T18:39:25.070 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:25.043+0000 7f5663170700 1 -- 172.21.15.39:0/2501908238 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f564c0a11d0 msgr2=0x7f564c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:25.071 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:25.043+0000 7f5663170700 1 --2- 172.21.15.39:0/2501908238 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f564c0a11d0 0x7f564c0a3680 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5650005950 tx=0x7f56500058e0 comp rx=0 tx=0).stop 2023-07-21T18:39:25.071 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:25.043+0000 7f5663170700 1 -- 172.21.15.39:0/2501908238 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f565c100dc0 msgr2=0x7f565c195db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:25.071 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:25.043+0000 7f5663170700 1 --2- 172.21.15.39:0/2501908238 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f565c100dc0 0x7f565c195db0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5648005910 tx=0x7f564800f710 comp rx=0 tx=0).stop 2023-07-21T18:39:25.071 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:25.043+0000 7f5663170700 1 -- 172.21.15.39:0/2501908238 shutdown_connections 2023-07-21T18:39:25.071 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:25.043+0000 7f5663170700 1 --2- 172.21.15.39:0/2501908238 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f564c0a11d0 0x7f564c0a3680 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:25.071 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:25.043+0000 7f5663170700 1 --2- 172.21.15.39:0/2501908238 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f565c0ffbc0 0x7f565c195870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:25.072 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:25.043+0000 7f5663170700 1 --2- 172.21.15.39:0/2501908238 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f565c101750 0x7f565c19ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:25.072 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:25.043+0000 7f5663170700 1 --2- 172.21.15.39:0/2501908238 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f565c100dc0 0x7f565c195db0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:25.072 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:25.043+0000 7f5663170700 1 -- 172.21.15.39:0/2501908238 >> 172.21.15.39:0/2501908238 conn(0x7f565c0fb230 msgr2=0x7f565c0fd460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:39:25.072 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:25.043+0000 7f5663170700 1 -- 172.21.15.39:0/2501908238 shutdown_connections 2023-07-21T18:39:25.072 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:25.043+0000 7f5663170700 1 -- 172.21.15.39:0/2501908238 wait complete. 2023-07-21T18:39:25.451 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":75,"num_osds":12,"num_up_osds":12,"osd_up_since":1689964763,"num_in_osds":12,"osd_in_since":1689964745,"num_remapped_pgs":0} 2023-07-21T18:39:25.452 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph osd dump --format=json 2023-07-21T18:39:25.693 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:25 smithi039 ceph-mon[96489]: osdmap e75: 12 total, 12 up, 12 in 2023-07-21T18:39:25.693 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:25 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/2501908238' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-21T18:39:25.693 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:25 smithi039 ceph-mon[96489]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:39:25.875 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:25 smithi193 ceph-mon[98204]: osdmap e75: 12 total, 12 up, 12 in 2023-07-21T18:39:25.875 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:25 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/2501908238' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-21T18:39:25.875 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:25 smithi193 ceph-mon[98204]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:39:25.894 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:25 smithi155 ceph-mon[97934]: osdmap e75: 12 total, 12 up, 12 in 2023-07-21T18:39:25.894 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:25 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/2501908238' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-21T18:39:25.894 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:25 smithi155 ceph-mon[97934]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:39:26.012 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:39:27.479 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:27.451+0000 7fa77fe4e700 1 -- 172.21.15.39:0/64811960 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7780fee80 msgr2=0x7fa7780ff250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:27.479 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:27.451+0000 7fa77fe4e700 1 --2- 172.21.15.39:0/64811960 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7780fee80 0x7fa7780ff250 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa76c0099c0 tx=0x7fa76c009cd0 comp rx=0 tx=0).stop 2023-07-21T18:39:27.479 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:27.451+0000 7fa77fe4e700 1 -- 172.21.15.39:0/64811960 shutdown_connections 2023-07-21T18:39:27.480 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:27.451+0000 7fa77fe4e700 1 --2- 172.21.15.39:0/64811960 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa778100f80 0x7fa77810ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:27.480 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:27.451+0000 7fa77fe4e700 1 --2- 172.21.15.39:0/64811960 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7780fee80 0x7fa7780ff250 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:27.480 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:27.451+0000 7fa77fe4e700 1 --2- 172.21.15.39:0/64811960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa7781005d0 0x7fa778100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:27.480 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:27.451+0000 7fa77fe4e700 1 -- 172.21.15.39:0/64811960 >> 172.21.15.39:0/64811960 conn(0x7fa778075b60 msgr2=0x7fa778075f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:39:27.485 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:27.451+0000 7fa77fe4e700 1 -- 172.21.15.39:0/64811960 shutdown_connections 2023-07-21T18:39:27.485 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:27.451+0000 7fa77fe4e700 1 -- 172.21.15.39:0/64811960 wait complete. 2023-07-21T18:39:27.486 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:27.452+0000 7fa77fe4e700 1 Processor -- start 2023-07-21T18:39:27.486 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:27.452+0000 7fa77fe4e700 1 -- start start 2023-07-21T18:39:27.486 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:27.452+0000 7fa77fe4e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7780fee80 0x7fa77819bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:27.486 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:27.452+0000 7fa77fe4e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7781005d0 0x7fa77819c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:27.487 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:27.452+0000 7fa77fe4e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa778100f80 0x7fa778195b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:27.487 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:27.452+0000 7fa77fe4e700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7781961b0 con 0x7fa7781005d0 2023-07-21T18:39:27.487 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:27.452+0000 7fa77fe4e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa778196320 con 0x7fa778100f80 2023-07-21T18:39:27.487 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:27.452+0000 7fa77fe4e700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7781964e0 con 0x7fa7780fee80 2023-07-21T18:39:27.488 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:27.453+0000 7fa77d3e9700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7781005d0 0x7fa77819c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:27.488 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:27.453+0000 7fa77d3e9700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7781005d0 0x7fa77819c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54660/0 (socket says 172.21.15.39:54660) 2023-07-21T18:39:27.488 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:27.453+0000 7fa77d3e9700 1 -- 172.21.15.39:0/2821498158 learned_addr learned my addr 172.21.15.39:0/2821498158 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:39:27.489 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:27.453+0000 7fa77e3eb700 1 --2- 172.21.15.39:0/2821498158 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa778100f80 0x7fa778195b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:27.489 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:27.453+0000 7fa77dbea700 1 --2- 172.21.15.39:0/2821498158 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7780fee80 0x7fa77819bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:27.489 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:27.453+0000 7fa77d3e9700 1 -- 172.21.15.39:0/2821498158 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7780fee80 msgr2=0x7fa77819bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:27.489 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:27.453+0000 7fa77d3e9700 1 --2- 172.21.15.39:0/2821498158 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7780fee80 0x7fa77819bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:27.489 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:27.453+0000 7fa77d3e9700 1 -- 172.21.15.39:0/2821498158 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa778100f80 msgr2=0x7fa778195b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:27.489 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:27.453+0000 7fa77d3e9700 1 --2- 172.21.15.39:0/2821498158 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa778100f80 0x7fa778195b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:27.489 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:27.453+0000 7fa77d3e9700 1 -- 172.21.15.39:0/2821498158 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa76c0095f0 con 0x7fa7781005d0 2023-07-21T18:39:27.490 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:27.453+0000 7fa77e3eb700 1 --2- 172.21.15.39:0/2821498158 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa778100f80 0x7fa778195b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:39:27.490 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:27.453+0000 7fa77d3e9700 1 --2- 172.21.15.39:0/2821498158 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7781005d0 0x7fa77819c000 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa77400c920 tx=0x7fa77400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:39:27.490 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:27.453+0000 7fa76affd700 1 -- 172.21.15.39:0/2821498158 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa774007b00 con 0x7fa7781005d0 2023-07-21T18:39:27.490 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:27.453+0000 7fa76affd700 1 -- 172.21.15.39:0/2821498158 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa774004cf0 con 0x7fa7781005d0 2023-07-21T18:39:27.490 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:27.453+0000 7fa76affd700 1 -- 172.21.15.39:0/2821498158 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa774005610 con 0x7fa7781005d0 2023-07-21T18:39:27.490 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:27.453+0000 7fa77fe4e700 1 -- 172.21.15.39:0/2821498158 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7781967c0 con 0x7fa7781005d0 2023-07-21T18:39:27.490 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:27.453+0000 7fa77fe4e700 1 -- 172.21.15.39:0/2821498158 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7781a2ab0 con 0x7fa7781005d0 2023-07-21T18:39:27.491 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:27.454+0000 7fa77fe4e700 1 -- 172.21.15.39:0/2821498158 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa77804e210 con 0x7fa7781005d0 2023-07-21T18:39:27.491 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:27.455+0000 7fa76affd700 1 -- 172.21.15.39:0/2821498158 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fa774004750 con 0x7fa7781005d0 2023-07-21T18:39:27.491 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:27.456+0000 7fa76affd700 1 --2- 172.21.15.39:0/2821498158 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fa7640a11d0 0x7fa7640a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:27.491 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:27.456+0000 7fa77dbea700 1 --2- 172.21.15.39:0/2821498158 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fa7640a11d0 0x7fa7640a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:27.491 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:27.456+0000 7fa76affd700 1 -- 172.21.15.39:0/2821498158 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa7740cc3a0 con 0x7fa7781005d0 2023-07-21T18:39:27.492 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:27.456+0000 7fa77dbea700 1 --2- 172.21.15.39:0/2821498158 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fa7640a11d0 0x7fa7640a3680 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa76c000c00 tx=0x7fa76c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:39:27.492 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:27.460+0000 7fa76affd700 1 -- 172.21.15.39:0/2821498158 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fa774095f90 con 0x7fa7781005d0 2023-07-21T18:39:27.625 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:27 smithi193 ceph-mon[98204]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:39:27.644 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:27 smithi155 ceph-mon[97934]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:39:27.683 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:27.656+0000 7fa77fe4e700 1 -- 172.21.15.39:0/2821498158 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa778196fe0 con 0x7fa7781005d0 2023-07-21T18:39:27.684 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:27.657+0000 7fa76affd700 1 -- 172.21.15.39:0/2821498158 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+18420 (secure 0 0 0) 0x7fa7740995b0 con 0x7fa7781005d0 2023-07-21T18:39:27.684 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:39:27.685 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":75,"fsid":"eaf8e25a-27f4-11ee-9b34-001a4aab830c","created":"2023-07-21T18:33:06.504979+0000","modified":"2023-07-21T18:39:24.539865+0000","last_up_change":"2023-07-21T18:39:23.531009+0000","last_in_change":"2023-07-21T18:39:05.993024+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T18:36:13.572583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03502c07-2d0f-4723-8d0b-d34c9f5b91bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6803","nonce":860193857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6805","nonce":860193857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6809","nonce":860193857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6807","nonce":860193857}]},"public_addr":"172.21.15.39:6803/860193857","cluster_addr":"172.21.15.39:6805/860193857","heartbeat_back_addr":"172.21.15.39:6809/860193857","heartbeat_front_addr":"172.21.15.39:6807/860193857","state":["exists","up"]},{"osd":1,"uuid":"0831a0c0-a1fa-4d30-9a28-513223aa69eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6801","nonce":3246721764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6803","nonce":3246721764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6807","nonce":3246721764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6805","nonce":3246721764}]},"public_addr":"172.21.15.155:6801/3246721764","cluster_addr":"172.21.15.155:6803/3246721764","heartbeat_back_addr":"172.21.15.155:6807/3246721764","heartbeat_front_addr":"172.21.15.155:6805/3246721764","state":["exists","up"]},{"osd":2,"uuid":"22df0003-bde9-48e4-b47a-cb8eafbab0db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6801","nonce":2563594449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6803","nonce":2563594449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6807","nonce":2563594449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6805","nonce":2563594449}]},"public_addr":"172.21.15.193:6801/2563594449","cluster_addr":"172.21.15.193:6803/2563594449","heartbeat_back_addr":"172.21.15.193:6807/2563594449","heartbeat_front_addr":"172.21.15.193:6805/2563594449","state":["exists","up"]},{"osd":3,"uuid":"54562558-a2e8-45dd-bc95-561569b9ae6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6811","nonce":863708386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6813","nonce":863708386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6817","nonce":863708386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6815","nonce":863708386}]},"public_addr":"172.21.15.39:6811/863708386","cluster_addr":"172.21.15.39:6813/863708386","heartbeat_back_addr":"172.21.15.39:6817/863708386","heartbeat_front_addr":"172.21.15.39:6815/863708386","state":["exists","up"]},{"osd":4,"uuid":"491118e7-ae1e-495c-9eac-ef1cdcae9600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6809","nonce":3361780673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6811","nonce":3361780673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6815","nonce":3361780673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6813","nonce":3361780673}]},"public_addr":"172.21.15.155:6809/3361780673","cluster_addr":"172.21.15.155:6811/3361780673","heartbeat_back_addr":"172.21.15.155:6815/3361780673","heartbeat_front_addr":"172.21.15.155:6813/3361780673","state":["exists","up"]},{"osd":5,"uuid":"d267bc2c-42b3-4d96-b2d1-b186dc7e6b18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6809","nonce":2376867721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6811","nonce":2376867721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6815","nonce":2376867721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6813","nonce":2376867721}]},"public_addr":"172.21.15.193:6809/2376867721","cluster_addr":"172.21.15.193:6811/2376867721","heartbeat_back_addr":"172.21.15.193:6815/2376867721","heartbeat_front_addr":"172.21.15.193:6813/2376867721","state":["exists","up"]},{"osd":6,"uuid":"beec8088-970a-473e-bd67-6eee7cbdef24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6819","nonce":2417770529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6821","nonce":2417770529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6825","nonce":2417770529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6823","nonce":2417770529}]},"public_addr":"172.21.15.39:6819/2417770529","cluster_addr":"172.21.15.39:6821/2417770529","heartbeat_back_addr":"172.21.15.39:6825/2417770529","heartbeat_front_addr":"172.21.15.39:6823/2417770529","state":["exists","up"]},{"osd":7,"uuid":"5a88ce30-1835-4254-90cc-804ed5cf1066","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6817","nonce":2542002111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6819","nonce":2542002111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6823","nonce":2542002111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6821","nonce":2542002111}]},"public_addr":"172.21.15.155:6817/2542002111","cluster_addr":"172.21.15.155:6819/2542002111","heartbeat_back_addr":"172.21.15.155:6823/2542002111","heartbeat_front_addr":"172.21.15.155:6821/2542002111","state":["exists","up"]},{"osd":8,"uuid":"d964a183-3021-4ace-9a0c-aad00f2570e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6817","nonce":2985835085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6819","nonce":2985835085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6823","nonce":2985835085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6821","nonce":2985835085}]},"public_addr":"172.21.15.193:6817/2985835085","cluster_addr":"172.21.15.193:6819/2985835085","heartbeat_back_addr":"172.21.15.193:6823/2985835085","heartbeat_front_addr":"172.21.15.193:6821/2985835085","state":["exists","up"]},{"osd":9,"uuid":"9725ea5c-49a3-4c99-bac4-e95069100b1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6827","nonce":1946493651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6829","nonce":1946493651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6833","nonce":1946493651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6831","nonce":1946493651}]},"public_addr":"172.21.15.39:6827/1946493651","cluster_addr":"172.21.15.39:6829/1946493651","heartbeat_back_addr":"172.21.15.39:6833/1946493651","heartbeat_front_addr":"172.21.15.39:6831/1946493651","state":["exists","up"]},{"osd":10,"uuid":"1fd60ccb-92db-45f0-a580-2003db0976e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6825","nonce":1234615022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6827","nonce":1234615022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6831","nonce":1234615022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6829","nonce":1234615022}]},"public_addr":"172.21.15.155:6825/1234615022","cluster_addr":"172.21.15.155:6827/1234615022","heartbeat_back_addr":"172.21.15.155:6831/1234615022","heartbeat_front_addr":"172.21.15.155:6829/1234615022","state":["exists","up"]},{"osd":11,"uuid":"efb31ea1-4775-4e5f-8740-728d6f4aeea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6825","nonce":215104663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6827","nonce":215104663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6831","nonce":215104663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6829","nonce":215104663}]},"public_addr":"172.21.15.193:6825/215104663","cluster_addr":"172.21.15.193:6827/215104663","heartbeat_back_addr":"172.21.15.193:6831/215104663","heartbeat_front_addr":"172.21.15.193:6829/215104663","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:35:30.211698+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:35:50.883846+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:36:11.124168+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:36:32.711905+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:36:54.442007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:37:14.843526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:37:36.587354+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:37:57.882374+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:38:18.729407+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:38:40.139231+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:39:01.290862+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:39:21.767537+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:0/2902600414":"2023-07-22T18:34:07.039823+0000","172.21.15.39:6801/2893866425":"2023-07-22T18:34:07.039823+0000","172.21.15.39:0/19717002":"2023-07-22T18:34:07.039823+0000","172.21.15.39:6800/2893866425":"2023-07-22T18:34:07.039823+0000","172.21.15.39:6800/889516631":"2023-07-22T18:33:40.852706+0000","172.21.15.39:6801/889516631":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/3636510379":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/1275748831":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/3819152602":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/1127767082":"2023-07-22T18:34:07.039823+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T18:39:27.687 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:27.659+0000 7fa77fe4e700 1 -- 172.21.15.39:0/2821498158 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fa7640a11d0 msgr2=0x7fa7640a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:27.687 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:27.659+0000 7fa77fe4e700 1 --2- 172.21.15.39:0/2821498158 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fa7640a11d0 0x7fa7640a3680 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa76c000c00 tx=0x7fa76c005fb0 comp rx=0 tx=0).stop 2023-07-21T18:39:27.687 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:27.659+0000 7fa77fe4e700 1 -- 172.21.15.39:0/2821498158 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7781005d0 msgr2=0x7fa77819c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:27.687 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:27.659+0000 7fa77fe4e700 1 --2- 172.21.15.39:0/2821498158 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7781005d0 0x7fa77819c000 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa77400c920 tx=0x7fa77400cce0 comp rx=0 tx=0).stop 2023-07-21T18:39:27.687 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:27.659+0000 7fa77fe4e700 1 -- 172.21.15.39:0/2821498158 shutdown_connections 2023-07-21T18:39:27.687 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:27.659+0000 7fa77fe4e700 1 --2- 172.21.15.39:0/2821498158 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fa7640a11d0 0x7fa7640a3680 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:27.688 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:27.659+0000 7fa77fe4e700 1 --2- 172.21.15.39:0/2821498158 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa778100f80 0x7fa778195b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:27.688 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:27.659+0000 7fa77fe4e700 1 --2- 172.21.15.39:0/2821498158 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7780fee80 0x7fa77819bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:27.688 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:27.659+0000 7fa77fe4e700 1 --2- 172.21.15.39:0/2821498158 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7781005d0 0x7fa77819c000 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:27.688 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:27.659+0000 7fa77fe4e700 1 -- 172.21.15.39:0/2821498158 >> 172.21.15.39:0/2821498158 conn(0x7fa778075b60 msgr2=0x7fa7780fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:39:27.688 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:27.660+0000 7fa77fe4e700 1 -- 172.21.15.39:0/2821498158 shutdown_connections 2023-07-21T18:39:27.688 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:27.660+0000 7fa77fe4e700 1 -- 172.21.15.39:0/2821498158 wait complete. 2023-07-21T18:39:27.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:27 smithi039 ceph-mon[96489]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:39:28.171 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-21T18:36:13.572583+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-21T18:39:28.172 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph osd pool get .mgr pg_num 2023-07-21T18:39:28.443 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:28 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/2821498158' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T18:39:28.625 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:28 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/2821498158' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T18:39:28.644 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:28 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/2821498158' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T18:39:28.728 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:39:29.625 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:29 smithi193 ceph-mon[98204]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:39:29.643 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:29 smithi155 ceph-mon[97934]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:39:29.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:29 smithi039 ceph-mon[96489]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:39:29.896 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:29.869+0000 7f9eb31d4700 1 -- 172.21.15.39:0/2302938578 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9eac0ff460 msgr2=0x7f9eac10b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:29.897 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:29.869+0000 7f9eb31d4700 1 --2- 172.21.15.39:0/2302938578 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9eac0ff460 0x7f9eac10b8f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9ea400b0a0 tx=0x7f9ea400b3b0 comp rx=0 tx=0).stop 2023-07-21T18:39:29.897 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:29.869+0000 7f9eb31d4700 1 -- 172.21.15.39:0/2302938578 shutdown_connections 2023-07-21T18:39:29.897 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:29.869+0000 7f9eb31d4700 1 --2- 172.21.15.39:0/2302938578 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9eac0ff460 0x7f9eac10b8f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:29.897 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:29.869+0000 7f9eb31d4700 1 --2- 172.21.15.39:0/2302938578 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9eac104ab0 0x7f9eac104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:29.897 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:29.869+0000 7f9eb31d4700 1 --2- 172.21.15.39:0/2302938578 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9eac0feab0 0x7f9eac0fef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:29.897 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:29.869+0000 7f9eb31d4700 1 -- 172.21.15.39:0/2302938578 >> 172.21.15.39:0/2302938578 conn(0x7f9eac075b70 msgr2=0x7f9eac075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:39:29.898 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:29.869+0000 7f9eb31d4700 1 -- 172.21.15.39:0/2302938578 shutdown_connections 2023-07-21T18:39:29.898 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:29.869+0000 7f9eb31d4700 1 -- 172.21.15.39:0/2302938578 wait complete. 2023-07-21T18:39:29.899 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:29.870+0000 7f9eb31d4700 1 Processor -- start 2023-07-21T18:39:29.899 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:29.870+0000 7f9eb31d4700 1 -- start start 2023-07-21T18:39:29.900 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:29.870+0000 7f9eb31d4700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9eac0feab0 0x7f9eac196a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:29.900 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:29.870+0000 7f9eb31d4700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9eac0ff460 0x7f9eac196fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:29.900 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:29.870+0000 7f9eb31d4700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9eac104ab0 0x7f9eac1a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:29.900 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:29.870+0000 7f9eb31d4700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eac1978f0 con 0x7f9eac0ff460 2023-07-21T18:39:29.900 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:29.870+0000 7f9eb31d4700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eac1a2030 con 0x7f9eac0feab0 2023-07-21T18:39:29.900 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:29.870+0000 7f9eb31d4700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eac1a21f0 con 0x7f9eac104ab0 2023-07-21T18:39:29.900 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:29.870+0000 7f9eabfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9eac0ff460 0x7f9eac196fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:29.901 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:29.870+0000 7f9eabfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9eac0ff460 0x7f9eac196fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54674/0 (socket says 172.21.15.39:54674) 2023-07-21T18:39:29.901 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:29.870+0000 7f9eabfff700 1 -- 172.21.15.39:0/862426704 learned_addr learned my addr 172.21.15.39:0/862426704 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:39:29.901 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:29.871+0000 7f9eabfff700 1 -- 172.21.15.39:0/862426704 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9eac104ab0 msgr2=0x7f9eac1a1af0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-21T18:39:29.901 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:29.871+0000 7f9eabfff700 1 --2- 172.21.15.39:0/862426704 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9eac104ab0 0x7f9eac1a1af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:29.901 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:29.871+0000 7f9eabfff700 1 -- 172.21.15.39:0/862426704 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9eac0feab0 msgr2=0x7f9eac196a70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-21T18:39:29.901 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:29.871+0000 7f9eabfff700 1 --2- 172.21.15.39:0/862426704 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9eac0feab0 0x7f9eac196a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:29.901 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:29.871+0000 7f9eabfff700 1 -- 172.21.15.39:0/862426704 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea4009d00 con 0x7f9eac0ff460 2023-07-21T18:39:29.902 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:29.871+0000 7f9eabfff700 1 --2- 172.21.15.39:0/862426704 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9eac0ff460 0x7f9eac196fb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9e9800c9b0 tx=0x7f9e9800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:39:29.902 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:29.871+0000 7f9ea9ffb700 1 -- 172.21.15.39:0/862426704 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e98007b00 con 0x7f9eac0ff460 2023-07-21T18:39:29.902 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:29.871+0000 7f9ea9ffb700 1 -- 172.21.15.39:0/862426704 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e98007c60 con 0x7f9eac0ff460 2023-07-21T18:39:29.902 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:29.871+0000 7f9eb31d4700 1 -- 172.21.15.39:0/862426704 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9eac1a24d0 con 0x7f9eac0ff460 2023-07-21T18:39:29.902 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:29.871+0000 7f9ea9ffb700 1 -- 172.21.15.39:0/862426704 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e98018660 con 0x7f9eac0ff460 2023-07-21T18:39:29.902 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:29.871+0000 7f9eb31d4700 1 -- 172.21.15.39:0/862426704 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9eac1a2a20 con 0x7f9eac0ff460 2023-07-21T18:39:29.902 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:29.872+0000 7f9eb31d4700 1 -- 172.21.15.39:0/862426704 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9eac04e210 con 0x7f9eac0ff460 2023-07-21T18:39:29.903 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:29.873+0000 7f9ea9ffb700 1 -- 172.21.15.39:0/862426704 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f9e98018880 con 0x7f9eac0ff460 2023-07-21T18:39:29.903 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:29.874+0000 7f9ea9ffb700 1 --2- 172.21.15.39:0/862426704 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f9e9c0a11d0 0x7f9e9c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:29.906 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:29.874+0000 7f9eb0f70700 1 --2- 172.21.15.39:0/862426704 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f9e9c0a11d0 0x7f9e9c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:29.906 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:29.874+0000 7f9ea9ffb700 1 -- 172.21.15.39:0/862426704 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f9e980cc550 con 0x7f9eac0ff460 2023-07-21T18:39:29.906 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:29.874+0000 7f9eb0f70700 1 --2- 172.21.15.39:0/862426704 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f9e9c0a11d0 0x7f9e9c0a3680 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0005950 tx=0x7f9ea00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:39:29.907 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:29.878+0000 7f9ea9ffb700 1 -- 172.21.15.39:0/862426704 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f9e980960a0 con 0x7f9eac0ff460 2023-07-21T18:39:30.100 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:30.073+0000 7f9eb31d4700 1 -- 172.21.15.39:0/862426704 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9eac1a2e30 con 0x7f9eac0ff460 2023-07-21T18:39:30.101 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:30.073+0000 7f9ea9ffb700 1 -- 172.21.15.39:0/862426704 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9e980996c0 con 0x7f9eac0ff460 2023-07-21T18:39:30.101 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2023-07-21T18:39:30.102 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:30.075+0000 7f9eb31d4700 1 -- 172.21.15.39:0/862426704 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f9e9c0a11d0 msgr2=0x7f9e9c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:30.102 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:30.075+0000 7f9eb31d4700 1 --2- 172.21.15.39:0/862426704 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f9e9c0a11d0 0x7f9e9c0a3680 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0005950 tx=0x7f9ea00058e0 comp rx=0 tx=0).stop 2023-07-21T18:39:30.102 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:30.075+0000 7f9eb31d4700 1 -- 172.21.15.39:0/862426704 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9eac0ff460 msgr2=0x7f9eac196fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:30.103 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:30.075+0000 7f9eb31d4700 1 --2- 172.21.15.39:0/862426704 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9eac0ff460 0x7f9eac196fb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9e9800c9b0 tx=0x7f9e9800cd70 comp rx=0 tx=0).stop 2023-07-21T18:39:30.103 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:30.075+0000 7f9eb31d4700 1 -- 172.21.15.39:0/862426704 shutdown_connections 2023-07-21T18:39:30.103 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:30.075+0000 7f9eb31d4700 1 --2- 172.21.15.39:0/862426704 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f9e9c0a11d0 0x7f9e9c0a3680 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:30.103 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:30.075+0000 7f9eb31d4700 1 --2- 172.21.15.39:0/862426704 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9eac0feab0 0x7f9eac196a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:30.103 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:30.075+0000 7f9eb31d4700 1 --2- 172.21.15.39:0/862426704 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9eac104ab0 0x7f9eac1a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:30.103 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:30.075+0000 7f9eb31d4700 1 --2- 172.21.15.39:0/862426704 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9eac0ff460 0x7f9eac196fb0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:30.103 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:30.075+0000 7f9eb31d4700 1 -- 172.21.15.39:0/862426704 >> 172.21.15.39:0/862426704 conn(0x7f9eac075b70 msgr2=0x7f9eac102600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:39:30.104 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:30.075+0000 7f9eb31d4700 1 -- 172.21.15.39:0/862426704 shutdown_connections 2023-07-21T18:39:30.104 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:30.075+0000 7f9eb31d4700 1 -- 172.21.15.39:0/862426704 wait complete. 2023-07-21T18:39:30.388 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:30 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/862426704' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T18:39:30.477 INFO:tasks.cephadm:Adding mds.a on smithi039 2023-07-21T18:39:30.478 INFO:tasks.cephadm:Adding mds.d on smithi039 2023-07-21T18:39:30.478 INFO:tasks.cephadm:Adding mds.g on smithi039 2023-07-21T18:39:30.478 INFO:tasks.cephadm:Adding mds.j on smithi039 2023-07-21T18:39:30.478 INFO:tasks.cephadm:Adding mds.b on smithi155 2023-07-21T18:39:30.478 INFO:tasks.cephadm:Adding mds.e on smithi155 2023-07-21T18:39:30.479 INFO:tasks.cephadm:Adding mds.h on smithi155 2023-07-21T18:39:30.479 INFO:tasks.cephadm:Adding mds.k on smithi155 2023-07-21T18:39:30.479 INFO:tasks.cephadm:Adding mds.c on smithi193 2023-07-21T18:39:30.479 INFO:tasks.cephadm:Adding mds.f on smithi193 2023-07-21T18:39:30.479 INFO:tasks.cephadm:Adding mds.i on smithi193 2023-07-21T18:39:30.480 INFO:tasks.cephadm:Adding mds.l on smithi193 2023-07-21T18:39:30.480 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph orch apply mds all '12;smithi039=a;smithi039=d;smithi039=g;smithi039=j;smithi155=b;smithi155=e;smithi155=h;smithi155=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l' 2023-07-21T18:39:30.513 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:30 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/862426704' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T18:39:30.643 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:30 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/862426704' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T18:39:31.044 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T18:39:32.125 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:31 smithi193 ceph-mon[98204]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:39:32.202 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:32.175+0000 7fc1d3339700 1 -- 172.21.15.193:0/1324990794 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1cc1017c0 msgr2=0x7fc1cc108080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:32.203 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:32.175+0000 7fc1d3339700 1 --2- 172.21.15.193:0/1324990794 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1cc1017c0 0x7fc1cc108080 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc00b0a0 tx=0x7fc1bc00b3b0 comp rx=0 tx=0).stop 2023-07-21T18:39:32.203 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:32.175+0000 7fc1d3339700 1 -- 172.21.15.193:0/1324990794 shutdown_connections 2023-07-21T18:39:32.203 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:32.175+0000 7fc1d3339700 1 --2- 172.21.15.193:0/1324990794 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1cc1017c0 0x7fc1cc108080 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:32.203 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:32.175+0000 7fc1d3339700 1 --2- 172.21.15.193:0/1324990794 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1cc0ffc30 0x7fc1cc100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:32.203 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:32.175+0000 7fc1d3339700 1 --2- 172.21.15.193:0/1324990794 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1cc100e30 0x7fc1cc101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:32.204 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:32.175+0000 7fc1d3339700 1 -- 172.21.15.193:0/1324990794 >> 172.21.15.193:0/1324990794 conn(0x7fc1cc0fb2a0 msgr2=0x7fc1cc0fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:39:32.204 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:32.175+0000 7fc1d3339700 1 -- 172.21.15.193:0/1324990794 shutdown_connections 2023-07-21T18:39:32.204 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:32.175+0000 7fc1d3339700 1 -- 172.21.15.193:0/1324990794 wait complete. 2023-07-21T18:39:32.205 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:32.176+0000 7fc1d3339700 1 Processor -- start 2023-07-21T18:39:32.205 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:32.176+0000 7fc1d3339700 1 -- start start 2023-07-21T18:39:32.206 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:32.176+0000 7fc1d3339700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1cc0ffc30 0x7fc1cc195a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:32.206 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:32.176+0000 7fc1d3339700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1cc100e30 0x7fc1cc195f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:32.206 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:32.176+0000 7fc1d3339700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1cc1017c0 0x7fc1cc19cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:32.206 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:32.176+0000 7fc1d3339700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1cc19d590 con 0x7fc1cc100e30 2023-07-21T18:39:32.206 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:32.176+0000 7fc1d3339700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1cc19d6d0 con 0x7fc1cc1017c0 2023-07-21T18:39:32.206 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:32.176+0000 7fc1d3339700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1cc19d860 con 0x7fc1cc0ffc30 2023-07-21T18:39:32.207 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:32.176+0000 7fc1d10d5700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1cc0ffc30 0x7fc1cc195a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:32.207 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:32.176+0000 7fc1d10d5700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1cc0ffc30 0x7fc1cc195a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:51046/0 (socket says 172.21.15.193:51046) 2023-07-21T18:39:32.207 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:32.176+0000 7fc1d10d5700 1 -- 172.21.15.193:0/2635710603 learned_addr learned my addr 172.21.15.193:0/2635710603 (peer_addr_for_me v2:172.21.15.193:0/0) 2023-07-21T18:39:32.207 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:32.176+0000 7fc1d18d6700 1 --2- 172.21.15.193:0/2635710603 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1cc1017c0 0x7fc1cc19cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:32.207 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:32.176+0000 7fc1d08d4700 1 --2- 172.21.15.193:0/2635710603 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1cc100e30 0x7fc1cc195f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:32.207 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:32.176+0000 7fc1d10d5700 1 -- 172.21.15.193:0/2635710603 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1cc1017c0 msgr2=0x7fc1cc19cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:32.207 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:32.176+0000 7fc1d10d5700 1 --2- 172.21.15.193:0/2635710603 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1cc1017c0 0x7fc1cc19cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:32.208 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:32.176+0000 7fc1d10d5700 1 -- 172.21.15.193:0/2635710603 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1cc100e30 msgr2=0x7fc1cc195f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:32.208 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:32.176+0000 7fc1d10d5700 1 --2- 172.21.15.193:0/2635710603 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1cc100e30 0x7fc1cc195f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:32.208 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:32.176+0000 7fc1d10d5700 1 -- 172.21.15.193:0/2635710603 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1bc009d00 con 0x7fc1cc0ffc30 2023-07-21T18:39:32.208 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:32.176+0000 7fc1d10d5700 1 --2- 172.21.15.193:0/2635710603 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1cc0ffc30 0x7fc1cc195a00 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc1c400c9a0 tx=0x7fc1c400cd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:39:32.208 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:32.176+0000 7fc1ca7fc700 1 -- 172.21.15.193:0/2635710603 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c4004220 con 0x7fc1cc0ffc30 2023-07-21T18:39:32.208 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:32.177+0000 7fc1ca7fc700 1 -- 172.21.15.193:0/2635710603 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1c4007d20 con 0x7fc1cc0ffc30 2023-07-21T18:39:32.208 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:32.177+0000 7fc1d3339700 1 -- 172.21.15.193:0/2635710603 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1cc19db40 con 0x7fc1cc0ffc30 2023-07-21T18:39:32.209 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:32.177+0000 7fc1ca7fc700 1 -- 172.21.15.193:0/2635710603 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c4003d00 con 0x7fc1cc0ffc30 2023-07-21T18:39:32.209 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:32.177+0000 7fc1d3339700 1 -- 172.21.15.193:0/2635710603 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1cc19e090 con 0x7fc1cc0ffc30 2023-07-21T18:39:32.209 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:32.178+0000 7fc1d3339700 1 -- 172.21.15.193:0/2635710603 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1cc1071e0 con 0x7fc1cc0ffc30 2023-07-21T18:39:32.209 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:32.178+0000 7fc1ca7fc700 1 -- 172.21.15.193:0/2635710603 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fc1c4004380 con 0x7fc1cc0ffc30 2023-07-21T18:39:32.210 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:32.179+0000 7fc1ca7fc700 1 --2- 172.21.15.193:0/2635710603 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fc1b40a11d0 0x7fc1b40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:32.210 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:32.179+0000 7fc1ca7fc700 1 -- 172.21.15.193:0/2635710603 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc1c40cc030 con 0x7fc1cc0ffc30 2023-07-21T18:39:32.210 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:32.179+0000 7fc1d08d4700 1 --2- 172.21.15.193:0/2635710603 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fc1b40a11d0 0x7fc1b40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:32.210 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:32.180+0000 7fc1d08d4700 1 --2- 172.21.15.193:0/2635710603 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fc1b40a11d0 0x7fc1b40a3680 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0006fd0 tx=0x7fc1c0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:39:32.211 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:32.183+0000 7fc1ca7fc700 1 -- 172.21.15.193:0/2635710603 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fc1c4095c20 con 0x7fc1cc0ffc30 2023-07-21T18:39:32.393 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:31 smithi155 ceph-mon[97934]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:39:32.404 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:32.377+0000 7fc1d3339700 1 -- 172.21.15.193:0/2635710603 --> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi039=a;smithi039=d;smithi039=g;smithi039=j;smithi155=b;smithi155=e;smithi155=h;smithi155=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l", "target": ["mon-mgr", ""]}) v1 -- 0x7fc1cc060c10 con 0x7fc1b40a11d0 2023-07-21T18:39:32.414 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:32.386+0000 7fc1ca7fc700 1 -- 172.21.15.193:0/2635710603 <== mgr.14152 v2:172.21.15.39:6800/2116291712 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7fc1cc060c10 con 0x7fc1b40a11d0 2023-07-21T18:39:32.414 INFO:teuthology.orchestra.run.smithi193.stdout:Scheduled mds.all update... 2023-07-21T18:39:32.416 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:32.388+0000 7fc1d3339700 1 -- 172.21.15.193:0/2635710603 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fc1b40a11d0 msgr2=0x7fc1b40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:32.416 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:32.388+0000 7fc1d3339700 1 --2- 172.21.15.193:0/2635710603 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fc1b40a11d0 0x7fc1b40a3680 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0006fd0 tx=0x7fc1c0008040 comp rx=0 tx=0).stop 2023-07-21T18:39:32.417 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:32.388+0000 7fc1d3339700 1 -- 172.21.15.193:0/2635710603 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1cc0ffc30 msgr2=0x7fc1cc195a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:32.417 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:32.388+0000 7fc1d3339700 1 --2- 172.21.15.193:0/2635710603 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1cc0ffc30 0x7fc1cc195a00 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc1c400c9a0 tx=0x7fc1c400cd60 comp rx=0 tx=0).stop 2023-07-21T18:39:32.417 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:32.388+0000 7fc1d3339700 1 -- 172.21.15.193:0/2635710603 shutdown_connections 2023-07-21T18:39:32.417 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:32.388+0000 7fc1d3339700 1 --2- 172.21.15.193:0/2635710603 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fc1b40a11d0 0x7fc1b40a3680 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:32.418 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:32.388+0000 7fc1d3339700 1 --2- 172.21.15.193:0/2635710603 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1cc1017c0 0x7fc1cc19cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:32.418 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:32.388+0000 7fc1d3339700 1 --2- 172.21.15.193:0/2635710603 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1cc0ffc30 0x7fc1cc195a00 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:32.418 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:32.388+0000 7fc1d3339700 1 --2- 172.21.15.193:0/2635710603 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1cc100e30 0x7fc1cc195f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:32.418 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:32.388+0000 7fc1d3339700 1 -- 172.21.15.193:0/2635710603 >> 172.21.15.193:0/2635710603 conn(0x7fc1cc0fb2a0 msgr2=0x7fc1cc104320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:39:32.419 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:32.388+0000 7fc1d3339700 1 -- 172.21.15.193:0/2635710603 shutdown_connections 2023-07-21T18:39:32.419 INFO:teuthology.orchestra.run.smithi193.stderr:2023-07-21T18:39:32.389+0000 7fc1d3339700 1 -- 172.21.15.193:0/2635710603 wait complete. 2023-07-21T18:39:32.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:31 smithi039 ceph-mon[96489]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:39:32.801 DEBUG:teuthology.orchestra.run.smithi039:mds.a> sudo journalctl -f -n 0 -u ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@mds.a.service 2023-07-21T18:39:32.803 DEBUG:teuthology.orchestra.run.smithi039:mds.d> sudo journalctl -f -n 0 -u ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@mds.d.service 2023-07-21T18:39:32.841 INFO:journalctl@ceph.mds.a.smithi039.stdout:-- Logs begin at Fri 2023-07-21 18:18:00 UTC. -- 2023-07-21T18:39:32.842 DEBUG:teuthology.orchestra.run.smithi039:mds.g> sudo journalctl -f -n 0 -u ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@mds.g.service 2023-07-21T18:39:32.872 INFO:journalctl@ceph.mds.d.smithi039.stdout:-- Logs begin at Fri 2023-07-21 18:18:00 UTC. -- 2023-07-21T18:39:32.873 DEBUG:teuthology.orchestra.run.smithi039:mds.j> sudo journalctl -f -n 0 -u ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@mds.j.service 2023-07-21T18:39:32.901 INFO:journalctl@ceph.mds.g.smithi039.stdout:-- Logs begin at Fri 2023-07-21 18:18:00 UTC. -- 2023-07-21T18:39:32.903 DEBUG:teuthology.orchestra.run.smithi155:mds.b> sudo journalctl -f -n 0 -u ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@mds.b.service 2023-07-21T18:39:32.905 DEBUG:teuthology.orchestra.run.smithi155:mds.e> sudo journalctl -f -n 0 -u ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@mds.e.service 2023-07-21T18:39:32.930 INFO:journalctl@ceph.mds.j.smithi039.stdout:-- Logs begin at Fri 2023-07-21 18:18:00 UTC. -- 2023-07-21T18:39:32.943 INFO:journalctl@ceph.mds.b.smithi155.stdout:-- Logs begin at Fri 2023-07-21 18:17:11 UTC. -- 2023-07-21T18:39:32.944 DEBUG:teuthology.orchestra.run.smithi155:mds.h> sudo journalctl -f -n 0 -u ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@mds.h.service 2023-07-21T18:39:32.974 INFO:journalctl@ceph.mds.e.smithi155.stdout:-- Logs begin at Fri 2023-07-21 18:17:11 UTC. -- 2023-07-21T18:39:32.975 DEBUG:teuthology.orchestra.run.smithi155:mds.k> sudo journalctl -f -n 0 -u ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@mds.k.service 2023-07-21T18:39:33.002 INFO:journalctl@ceph.mds.h.smithi155.stdout:-- Logs begin at Fri 2023-07-21 18:17:11 UTC. -- 2023-07-21T18:39:33.003 DEBUG:teuthology.orchestra.run.smithi193:mds.c> sudo journalctl -f -n 0 -u ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@mds.c.service 2023-07-21T18:39:33.005 DEBUG:teuthology.orchestra.run.smithi193:mds.f> sudo journalctl -f -n 0 -u ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@mds.f.service 2023-07-21T18:39:33.030 INFO:journalctl@ceph.mds.k.smithi155.stdout:-- Logs begin at Fri 2023-07-21 18:17:11 UTC. -- 2023-07-21T18:39:33.035 INFO:journalctl@ceph.mds.c.smithi193.stdout:-- Logs begin at Fri 2023-07-21 18:17:09 UTC. -- 2023-07-21T18:39:33.036 DEBUG:teuthology.orchestra.run.smithi193:mds.i> sudo journalctl -f -n 0 -u ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@mds.i.service 2023-07-21T18:39:33.061 INFO:journalctl@ceph.mds.f.smithi193.stdout:-- Logs begin at Fri 2023-07-21 18:17:09 UTC. -- 2023-07-21T18:39:33.062 DEBUG:teuthology.orchestra.run.smithi193:mds.l> sudo journalctl -f -n 0 -u ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@mds.l.service 2023-07-21T18:39:33.087 INFO:journalctl@ceph.mds.i.smithi193.stdout:-- Logs begin at Fri 2023-07-21 18:17:09 UTC. -- 2023-07-21T18:39:33.088 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2023-07-21T18:39:33.089 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T18:39:33.113 INFO:journalctl@ceph.mds.l.smithi193.stdout:-- Logs begin at Fri 2023-07-21 18:17:09 UTC. -- 2023-07-21T18:39:33.235 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.208+0000 7fbed63cf700 1 -- 172.21.15.39:0/1114401311 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbed00ffb00 msgr2=0x7fbed00fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:33.235 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.208+0000 7fbed63cf700 1 --2- 172.21.15.39:0/1114401311 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbed00ffb00 0x7fbed00fff10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbec0009990 tx=0x7fbec0009ca0 comp rx=0 tx=0).stop 2023-07-21T18:39:33.235 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.208+0000 7fbed63cf700 1 -- 172.21.15.39:0/1114401311 shutdown_connections 2023-07-21T18:39:33.236 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.208+0000 7fbed63cf700 1 --2- 172.21.15.39:0/1114401311 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbed0101680 0x7fbed0107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:33.236 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.208+0000 7fbed63cf700 1 --2- 172.21.15.39:0/1114401311 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbed00ffb00 0x7fbed00fff10 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:33.236 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.208+0000 7fbed63cf700 1 --2- 172.21.15.39:0/1114401311 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbed0100cf0 0x7fbed0101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:33.236 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.208+0000 7fbed63cf700 1 -- 172.21.15.39:0/1114401311 >> 172.21.15.39:0/1114401311 conn(0x7fbed00fb0e0 msgr2=0x7fbed00fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:39:33.236 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.208+0000 7fbed63cf700 1 -- 172.21.15.39:0/1114401311 shutdown_connections 2023-07-21T18:39:33.236 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.208+0000 7fbed63cf700 1 -- 172.21.15.39:0/1114401311 wait complete. 2023-07-21T18:39:33.237 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.208+0000 7fbed63cf700 1 Processor -- start 2023-07-21T18:39:33.237 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.208+0000 7fbed63cf700 1 -- start start 2023-07-21T18:39:33.238 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.209+0000 7fbed63cf700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbed00ffb00 0x7fbed0197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:33.239 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.209+0000 7fbed63cf700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbed0100cf0 0x7fbed0197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:33.239 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.209+0000 7fbed63cf700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbed0101680 0x7fbed019efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:33.239 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.209+0000 7fbed63cf700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed019f590 con 0x7fbed00ffb00 2023-07-21T18:39:33.239 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.209+0000 7fbed63cf700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed019f6d0 con 0x7fbed0100cf0 2023-07-21T18:39:33.239 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.209+0000 7fbed63cf700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed019f860 con 0x7fbed0101680 2023-07-21T18:39:33.239 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.209+0000 7fbecffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbed00ffb00 0x7fbed0197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:33.240 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.209+0000 7fbecffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbed00ffb00 0x7fbed0197a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54686/0 (socket says 172.21.15.39:54686) 2023-07-21T18:39:33.240 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.209+0000 7fbecffff700 1 -- 172.21.15.39:0/2451377886 learned_addr learned my addr 172.21.15.39:0/2451377886 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:39:33.240 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.209+0000 7fbed496c700 1 --2- 172.21.15.39:0/2451377886 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbed0101680 0x7fbed019efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:33.240 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.209+0000 7fbec7fff700 1 --2- 172.21.15.39:0/2451377886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbed0100cf0 0x7fbed0197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:33.240 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.209+0000 7fbecffff700 1 -- 172.21.15.39:0/2451377886 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbed0101680 msgr2=0x7fbed019efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:33.241 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.209+0000 7fbecffff700 1 --2- 172.21.15.39:0/2451377886 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbed0101680 0x7fbed019efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:33.241 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.209+0000 7fbecffff700 1 -- 172.21.15.39:0/2451377886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbed0100cf0 msgr2=0x7fbed0197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:33.241 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.209+0000 7fbecffff700 1 --2- 172.21.15.39:0/2451377886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbed0100cf0 0x7fbed0197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:33.241 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.209+0000 7fbecffff700 1 -- 172.21.15.39:0/2451377886 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbec00095f0 con 0x7fbed00ffb00 2023-07-21T18:39:33.241 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.209+0000 7fbed496c700 1 --2- 172.21.15.39:0/2451377886 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbed0101680 0x7fbed019efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:39:33.241 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.209+0000 7fbec7fff700 1 --2- 172.21.15.39:0/2451377886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbed0100cf0 0x7fbed0197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:39:33.242 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.209+0000 7fbecffff700 1 --2- 172.21.15.39:0/2451377886 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbed00ffb00 0x7fbed0197a10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbec0004730 tx=0x7fbec0009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:39:33.242 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.209+0000 7fbecdffb700 1 -- 172.21.15.39:0/2451377886 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec001d070 con 0x7fbed00ffb00 2023-07-21T18:39:33.242 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.209+0000 7fbecdffb700 1 -- 172.21.15.39:0/2451377886 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbec0004cd0 con 0x7fbed00ffb00 2023-07-21T18:39:33.243 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.209+0000 7fbed63cf700 1 -- 172.21.15.39:0/2451377886 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbed019fae0 con 0x7fbed00ffb00 2023-07-21T18:39:33.243 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.209+0000 7fbecdffb700 1 -- 172.21.15.39:0/2451377886 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec0005530 con 0x7fbed00ffb00 2023-07-21T18:39:33.243 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.209+0000 7fbed63cf700 1 -- 172.21.15.39:0/2451377886 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbed019ff00 con 0x7fbed00ffb00 2023-07-21T18:39:33.243 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.210+0000 7fbed63cf700 1 -- 172.21.15.39:0/2451377886 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbed010b320 con 0x7fbed00ffb00 2023-07-21T18:39:33.243 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.212+0000 7fbecdffb700 1 -- 172.21.15.39:0/2451377886 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fbec000fbc0 con 0x7fbed00ffb00 2023-07-21T18:39:33.244 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.212+0000 7fbecdffb700 1 --2- 172.21.15.39:0/2451377886 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fbebc0a1220 0x7fbebc0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:33.244 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.213+0000 7fbec7fff700 1 --2- 172.21.15.39:0/2451377886 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fbebc0a1220 0x7fbebc0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:33.244 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.213+0000 7fbecdffb700 1 -- 172.21.15.39:0/2451377886 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fbec00cdb20 con 0x7fbed00ffb00 2023-07-21T18:39:33.244 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.213+0000 7fbec7fff700 1 --2- 172.21.15.39:0/2451377886 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fbebc0a1220 0x7fbebc0a36d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8009cb0 tx=0x7fbeb8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:39:33.244 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.216+0000 7fbecdffb700 1 -- 172.21.15.39:0/2451377886 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fbec00977a0 con 0x7fbed00ffb00 2023-07-21T18:39:33.442 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.415+0000 7fbed63cf700 1 -- 172.21.15.39:0/2451377886 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbed01a0510 con 0x7fbed00ffb00 2023-07-21T18:39:33.443 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.416+0000 7fbecdffb700 1 -- 172.21.15.39:0/2451377886 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+18420 (secure 0 0 0) 0x7fbec009adc0 con 0x7fbed00ffb00 2023-07-21T18:39:33.444 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:39:33.444 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":75,"fsid":"eaf8e25a-27f4-11ee-9b34-001a4aab830c","created":"2023-07-21T18:33:06.504979+0000","modified":"2023-07-21T18:39:24.539865+0000","last_up_change":"2023-07-21T18:39:23.531009+0000","last_in_change":"2023-07-21T18:39:05.993024+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T18:36:13.572583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03502c07-2d0f-4723-8d0b-d34c9f5b91bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6803","nonce":860193857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6805","nonce":860193857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6809","nonce":860193857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6807","nonce":860193857}]},"public_addr":"172.21.15.39:6803/860193857","cluster_addr":"172.21.15.39:6805/860193857","heartbeat_back_addr":"172.21.15.39:6809/860193857","heartbeat_front_addr":"172.21.15.39:6807/860193857","state":["exists","up"]},{"osd":1,"uuid":"0831a0c0-a1fa-4d30-9a28-513223aa69eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6801","nonce":3246721764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6803","nonce":3246721764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6807","nonce":3246721764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6805","nonce":3246721764}]},"public_addr":"172.21.15.155:6801/3246721764","cluster_addr":"172.21.15.155:6803/3246721764","heartbeat_back_addr":"172.21.15.155:6807/3246721764","heartbeat_front_addr":"172.21.15.155:6805/3246721764","state":["exists","up"]},{"osd":2,"uuid":"22df0003-bde9-48e4-b47a-cb8eafbab0db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6801","nonce":2563594449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6803","nonce":2563594449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6807","nonce":2563594449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6805","nonce":2563594449}]},"public_addr":"172.21.15.193:6801/2563594449","cluster_addr":"172.21.15.193:6803/2563594449","heartbeat_back_addr":"172.21.15.193:6807/2563594449","heartbeat_front_addr":"172.21.15.193:6805/2563594449","state":["exists","up"]},{"osd":3,"uuid":"54562558-a2e8-45dd-bc95-561569b9ae6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6811","nonce":863708386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6813","nonce":863708386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6817","nonce":863708386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6815","nonce":863708386}]},"public_addr":"172.21.15.39:6811/863708386","cluster_addr":"172.21.15.39:6813/863708386","heartbeat_back_addr":"172.21.15.39:6817/863708386","heartbeat_front_addr":"172.21.15.39:6815/863708386","state":["exists","up"]},{"osd":4,"uuid":"491118e7-ae1e-495c-9eac-ef1cdcae9600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6809","nonce":3361780673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6811","nonce":3361780673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6815","nonce":3361780673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6813","nonce":3361780673}]},"public_addr":"172.21.15.155:6809/3361780673","cluster_addr":"172.21.15.155:6811/3361780673","heartbeat_back_addr":"172.21.15.155:6815/3361780673","heartbeat_front_addr":"172.21.15.155:6813/3361780673","state":["exists","up"]},{"osd":5,"uuid":"d267bc2c-42b3-4d96-b2d1-b186dc7e6b18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6809","nonce":2376867721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6811","nonce":2376867721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6815","nonce":2376867721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6813","nonce":2376867721}]},"public_addr":"172.21.15.193:6809/2376867721","cluster_addr":"172.21.15.193:6811/2376867721","heartbeat_back_addr":"172.21.15.193:6815/2376867721","heartbeat_front_addr":"172.21.15.193:6813/2376867721","state":["exists","up"]},{"osd":6,"uuid":"beec8088-970a-473e-bd67-6eee7cbdef24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6819","nonce":2417770529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6821","nonce":2417770529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6825","nonce":2417770529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6823","nonce":2417770529}]},"public_addr":"172.21.15.39:6819/2417770529","cluster_addr":"172.21.15.39:6821/2417770529","heartbeat_back_addr":"172.21.15.39:6825/2417770529","heartbeat_front_addr":"172.21.15.39:6823/2417770529","state":["exists","up"]},{"osd":7,"uuid":"5a88ce30-1835-4254-90cc-804ed5cf1066","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6817","nonce":2542002111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6819","nonce":2542002111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6823","nonce":2542002111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6821","nonce":2542002111}]},"public_addr":"172.21.15.155:6817/2542002111","cluster_addr":"172.21.15.155:6819/2542002111","heartbeat_back_addr":"172.21.15.155:6823/2542002111","heartbeat_front_addr":"172.21.15.155:6821/2542002111","state":["exists","up"]},{"osd":8,"uuid":"d964a183-3021-4ace-9a0c-aad00f2570e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6817","nonce":2985835085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6819","nonce":2985835085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6823","nonce":2985835085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6821","nonce":2985835085}]},"public_addr":"172.21.15.193:6817/2985835085","cluster_addr":"172.21.15.193:6819/2985835085","heartbeat_back_addr":"172.21.15.193:6823/2985835085","heartbeat_front_addr":"172.21.15.193:6821/2985835085","state":["exists","up"]},{"osd":9,"uuid":"9725ea5c-49a3-4c99-bac4-e95069100b1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6827","nonce":1946493651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6829","nonce":1946493651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6833","nonce":1946493651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6831","nonce":1946493651}]},"public_addr":"172.21.15.39:6827/1946493651","cluster_addr":"172.21.15.39:6829/1946493651","heartbeat_back_addr":"172.21.15.39:6833/1946493651","heartbeat_front_addr":"172.21.15.39:6831/1946493651","state":["exists","up"]},{"osd":10,"uuid":"1fd60ccb-92db-45f0-a580-2003db0976e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6825","nonce":1234615022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6827","nonce":1234615022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6831","nonce":1234615022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6829","nonce":1234615022}]},"public_addr":"172.21.15.155:6825/1234615022","cluster_addr":"172.21.15.155:6827/1234615022","heartbeat_back_addr":"172.21.15.155:6831/1234615022","heartbeat_front_addr":"172.21.15.155:6829/1234615022","state":["exists","up"]},{"osd":11,"uuid":"efb31ea1-4775-4e5f-8740-728d6f4aeea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6825","nonce":215104663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6827","nonce":215104663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6831","nonce":215104663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6829","nonce":215104663}]},"public_addr":"172.21.15.193:6825/215104663","cluster_addr":"172.21.15.193:6827/215104663","heartbeat_back_addr":"172.21.15.193:6831/215104663","heartbeat_front_addr":"172.21.15.193:6829/215104663","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:35:30.211698+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:35:50.883846+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:36:11.124168+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:36:32.711905+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:36:54.442007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:37:14.843526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:37:36.587354+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:37:57.882374+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:38:18.729407+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:38:40.139231+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:39:01.290862+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:39:21.767537+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:0/2902600414":"2023-07-22T18:34:07.039823+0000","172.21.15.39:6801/2893866425":"2023-07-22T18:34:07.039823+0000","172.21.15.39:0/19717002":"2023-07-22T18:34:07.039823+0000","172.21.15.39:6800/2893866425":"2023-07-22T18:34:07.039823+0000","172.21.15.39:6800/889516631":"2023-07-22T18:33:40.852706+0000","172.21.15.39:6801/889516631":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/3636510379":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/1275748831":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/3819152602":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/1127767082":"2023-07-22T18:34:07.039823+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T18:39:33.446 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.419+0000 7fbed63cf700 1 -- 172.21.15.39:0/2451377886 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fbebc0a1220 msgr2=0x7fbebc0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:33.446 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.419+0000 7fbed63cf700 1 --2- 172.21.15.39:0/2451377886 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fbebc0a1220 0x7fbebc0a36d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8009cb0 tx=0x7fbeb8009260 comp rx=0 tx=0).stop 2023-07-21T18:39:33.446 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.419+0000 7fbed63cf700 1 -- 172.21.15.39:0/2451377886 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbed00ffb00 msgr2=0x7fbed0197a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:33.446 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.419+0000 7fbed63cf700 1 --2- 172.21.15.39:0/2451377886 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbed00ffb00 0x7fbed0197a10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbec0004730 tx=0x7fbec0009510 comp rx=0 tx=0).stop 2023-07-21T18:39:33.446 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.419+0000 7fbed63cf700 1 -- 172.21.15.39:0/2451377886 shutdown_connections 2023-07-21T18:39:33.446 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.419+0000 7fbed63cf700 1 --2- 172.21.15.39:0/2451377886 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fbebc0a1220 0x7fbebc0a36d0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:33.447 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.419+0000 7fbed63cf700 1 --2- 172.21.15.39:0/2451377886 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbed0101680 0x7fbed019efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:33.447 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.419+0000 7fbed63cf700 1 --2- 172.21.15.39:0/2451377886 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbed00ffb00 0x7fbed0197a10 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:33.447 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.419+0000 7fbed63cf700 1 --2- 172.21.15.39:0/2451377886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbed0100cf0 0x7fbed0197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:33.447 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.419+0000 7fbed63cf700 1 -- 172.21.15.39:0/2451377886 >> 172.21.15.39:0/2451377886 conn(0x7fbed00fb0e0 msgr2=0x7fbed0106530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:39:33.447 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.419+0000 7fbed63cf700 1 -- 172.21.15.39:0/2451377886 shutdown_connections 2023-07-21T18:39:33.447 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.419+0000 7fbed63cf700 1 -- 172.21.15.39:0/2451377886 wait complete. 2023-07-21T18:39:33.459 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-21T18:36:13.572583+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-21T18:39:33.460 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-21T18:39:33.592 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.561+0000 7fa5cd96a700 1 -- 172.21.15.39:0/3371439147 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5c81005a0 msgr2=0x7fa5c8102980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:33.592 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.561+0000 7fa5cd96a700 1 --2- 172.21.15.39:0/3371439147 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5c81005a0 0x7fa5c8102980 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8009990 tx=0x7fa5b8009ca0 comp rx=0 tx=0).stop 2023-07-21T18:39:33.592 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.561+0000 7fa5cd96a700 1 -- 172.21.15.39:0/3371439147 shutdown_connections 2023-07-21T18:39:33.592 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.561+0000 7fa5cd96a700 1 --2- 172.21.15.39:0/3371439147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5c81057e0 0x7fa5c8107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:33.593 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.561+0000 7fa5cd96a700 1 --2- 172.21.15.39:0/3371439147 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5c81005a0 0x7fa5c8102980 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:33.593 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.561+0000 7fa5cd96a700 1 --2- 172.21.15.39:0/3371439147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5c8102ec0 0x7fa5c81052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:33.593 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.561+0000 7fa5cd96a700 1 -- 172.21.15.39:0/3371439147 >> 172.21.15.39:0/3371439147 conn(0x7fa5c80fa000 msgr2=0x7fa5c80fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:39:33.593 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.561+0000 7fa5cd96a700 1 -- 172.21.15.39:0/3371439147 shutdown_connections 2023-07-21T18:39:33.593 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.561+0000 7fa5cd96a700 1 -- 172.21.15.39:0/3371439147 wait complete. 2023-07-21T18:39:33.593 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.562+0000 7fa5cd96a700 1 Processor -- start 2023-07-21T18:39:33.593 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.562+0000 7fa5cd96a700 1 -- start start 2023-07-21T18:39:33.594 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.562+0000 7fa5cd96a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5c81005a0 0x7fa5c80fff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:33.594 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.562+0000 7fa5cd96a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5c8102ec0 0x7fa5c80fe550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:33.594 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.562+0000 7fa5cd96a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5c81057e0 0x7fa5c80fea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:33.594 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.562+0000 7fa5cd96a700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c80ff230 con 0x7fa5c81057e0 2023-07-21T18:39:33.594 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.562+0000 7fa5cd96a700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c80ff3a0 con 0x7fa5c8102ec0 2023-07-21T18:39:33.594 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.562+0000 7fa5cd96a700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c819f880 con 0x7fa5c81005a0 2023-07-21T18:39:33.594 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.562+0000 7fa5c77fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5c81057e0 0x7fa5c80fea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:33.595 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.562+0000 7fa5c6ffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5c81005a0 0x7fa5c80fff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:33.595 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.562+0000 7fa5c77fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5c81057e0 0x7fa5c80fea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54704/0 (socket says 172.21.15.39:54704) 2023-07-21T18:39:33.595 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.562+0000 7fa5c77fe700 1 -- 172.21.15.39:0/422139633 learned_addr learned my addr 172.21.15.39:0/422139633 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:39:33.595 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.562+0000 7fa5c67fc700 1 --2- 172.21.15.39:0/422139633 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5c8102ec0 0x7fa5c80fe550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:33.595 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.562+0000 7fa5c77fe700 1 -- 172.21.15.39:0/422139633 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5c81005a0 msgr2=0x7fa5c80fff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:33.595 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.562+0000 7fa5c77fe700 1 --2- 172.21.15.39:0/422139633 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5c81005a0 0x7fa5c80fff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:33.595 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.562+0000 7fa5c77fe700 1 -- 172.21.15.39:0/422139633 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5c8102ec0 msgr2=0x7fa5c80fe550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:33.596 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.562+0000 7fa5c77fe700 1 --2- 172.21.15.39:0/422139633 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5c8102ec0 0x7fa5c80fe550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:33.596 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.562+0000 7fa5c77fe700 1 -- 172.21.15.39:0/422139633 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b80095f0 con 0x7fa5c81057e0 2023-07-21T18:39:33.596 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.563+0000 7fa5c77fe700 1 --2- 172.21.15.39:0/422139633 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5c81057e0 0x7fa5c80fea90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0007ee0 tx=0x7fa5c000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:39:33.596 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.563+0000 7fa5cc968700 1 -- 172.21.15.39:0/422139633 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c0017070 con 0x7fa5c81057e0 2023-07-21T18:39:33.596 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.563+0000 7fa5cc968700 1 -- 172.21.15.39:0/422139633 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa5c000f040 con 0x7fa5c81057e0 2023-07-21T18:39:33.597 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.563+0000 7fa5cd96a700 1 -- 172.21.15.39:0/422139633 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5c819fb60 con 0x7fa5c81057e0 2023-07-21T18:39:33.597 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.563+0000 7fa5cc968700 1 -- 172.21.15.39:0/422139633 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c0005740 con 0x7fa5c81057e0 2023-07-21T18:39:33.597 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.563+0000 7fa5cd96a700 1 -- 172.21.15.39:0/422139633 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5c81a0070 con 0x7fa5c81057e0 2023-07-21T18:39:33.597 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.564+0000 7fa5cd96a700 1 -- 172.21.15.39:0/422139633 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5c80fbb70 con 0x7fa5c81057e0 2023-07-21T18:39:33.597 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.565+0000 7fa5cc968700 1 -- 172.21.15.39:0/422139633 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fa5c000dc60 con 0x7fa5c81057e0 2023-07-21T18:39:33.598 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.565+0000 7fa5cc968700 1 --2- 172.21.15.39:0/422139633 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fa5b00a1150 0x7fa5b00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:33.598 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.566+0000 7fa5c6ffd700 1 --2- 172.21.15.39:0/422139633 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fa5b00a1150 0x7fa5b00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:33.598 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.566+0000 7fa5cc968700 1 -- 172.21.15.39:0/422139633 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa5c00cbcc0 con 0x7fa5c81057e0 2023-07-21T18:39:33.598 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.566+0000 7fa5c6ffd700 1 --2- 172.21.15.39:0/422139633 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fa5b00a1150 0x7fa5b00a3600 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa5b800b5c0 tx=0x7fa5b8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:39:33.599 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.570+0000 7fa5cc968700 1 -- 172.21.15.39:0/422139633 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fa5c00d2080 con 0x7fa5c81057e0 2023-07-21T18:39:33.625 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:33 smithi193 ceph-mon[98204]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi039=a;smithi039=d;smithi039=g;smithi039=j;smithi155=b;smithi155=e;smithi155=h;smithi155=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:39:33.625 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:33 smithi193 ceph-mon[98204]: Saving service mds.all spec with placement smithi039=a;smithi039=d;smithi039=g;smithi039=j;smithi155=b;smithi155=e;smithi155=h;smithi155=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l;count:12 2023-07-21T18:39:33.625 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:33 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:33.625 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:33 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:39:33.625 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:33 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:39:33.626 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:33 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:39:33.626 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:33 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:33.626 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:33 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T18:39:33.626 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:33 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T18:39:33.626 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:33 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:39:33.626 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:33 smithi193 ceph-mon[98204]: Deploying daemon mds.l on smithi193 2023-07-21T18:39:33.626 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:33 smithi193 ceph-mon[98204]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:39:33.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:33 smithi039 ceph-mon[96489]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi039=a;smithi039=d;smithi039=g;smithi039=j;smithi155=b;smithi155=e;smithi155=h;smithi155=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:39:33.693 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:33 smithi039 ceph-mon[96489]: Saving service mds.all spec with placement smithi039=a;smithi039=d;smithi039=g;smithi039=j;smithi155=b;smithi155=e;smithi155=h;smithi155=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l;count:12 2023-07-21T18:39:33.693 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:33 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:33.693 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:33 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:39:33.693 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:33 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:39:33.693 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:33 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:39:33.693 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:33 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:33.693 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:33 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T18:39:33.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:33 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T18:39:33.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:33 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:39:33.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:33 smithi039 ceph-mon[96489]: Deploying daemon mds.l on smithi193 2023-07-21T18:39:33.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:33 smithi039 ceph-mon[96489]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:39:33.793 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.766+0000 7fa5cd96a700 1 -- 172.21.15.39:0/422139633 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa5c80611c0 con 0x7fa5c81057e0 2023-07-21T18:39:33.794 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.766+0000 7fa5cc968700 1 -- 172.21.15.39:0/422139633 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa5c0095810 con 0x7fa5c81057e0 2023-07-21T18:39:33.794 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2023-07-21T18:39:33.796 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.768+0000 7fa5cd96a700 1 -- 172.21.15.39:0/422139633 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fa5b00a1150 msgr2=0x7fa5b00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:33.796 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.768+0000 7fa5cd96a700 1 --2- 172.21.15.39:0/422139633 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fa5b00a1150 0x7fa5b00a3600 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa5b800b5c0 tx=0x7fa5b8005fd0 comp rx=0 tx=0).stop 2023-07-21T18:39:33.796 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.768+0000 7fa5cd96a700 1 -- 172.21.15.39:0/422139633 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5c81057e0 msgr2=0x7fa5c80fea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:33.797 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.768+0000 7fa5cd96a700 1 --2- 172.21.15.39:0/422139633 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5c81057e0 0x7fa5c80fea90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0007ee0 tx=0x7fa5c000d3b0 comp rx=0 tx=0).stop 2023-07-21T18:39:33.797 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.768+0000 7fa5cd96a700 1 -- 172.21.15.39:0/422139633 shutdown_connections 2023-07-21T18:39:33.797 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.768+0000 7fa5cd96a700 1 --2- 172.21.15.39:0/422139633 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fa5b00a1150 0x7fa5b00a3600 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:33.798 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.768+0000 7fa5cd96a700 1 --2- 172.21.15.39:0/422139633 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5c81057e0 0x7fa5c80fea90 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:33.798 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.768+0000 7fa5cd96a700 1 --2- 172.21.15.39:0/422139633 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5c81005a0 0x7fa5c80fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:33.798 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.768+0000 7fa5cd96a700 1 --2- 172.21.15.39:0/422139633 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5c8102ec0 0x7fa5c80fe550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:33.798 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.768+0000 7fa5cd96a700 1 -- 172.21.15.39:0/422139633 >> 172.21.15.39:0/422139633 conn(0x7fa5c80fa000 msgr2=0x7fa5c8101200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:39:33.798 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.768+0000 7fa5cd96a700 1 -- 172.21.15.39:0/422139633 shutdown_connections 2023-07-21T18:39:33.798 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.768+0000 7fa5cd96a700 1 -- 172.21.15.39:0/422139633 wait complete. 2023-07-21T18:39:33.808 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T18:39:33.893 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:33 smithi155 ceph-mon[97934]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi039=a;smithi039=d;smithi039=g;smithi039=j;smithi155=b;smithi155=e;smithi155=h;smithi155=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:39:33.894 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:33 smithi155 ceph-mon[97934]: Saving service mds.all spec with placement smithi039=a;smithi039=d;smithi039=g;smithi039=j;smithi155=b;smithi155=e;smithi155=h;smithi155=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l;count:12 2023-07-21T18:39:33.894 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:33 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:33.894 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:33 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:39:33.894 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:33 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:39:33.895 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:33 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:39:33.895 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:33 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:33.895 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:33 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T18:39:33.895 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:33 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T18:39:33.895 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:33 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:39:33.895 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:33 smithi155 ceph-mon[97934]: Deploying daemon mds.l on smithi193 2023-07-21T18:39:33.896 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:33 smithi155 ceph-mon[97934]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:39:33.936 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.907+0000 7f10d3a81700 1 -- 172.21.15.39:0/3315724416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10cc0ffbf0 msgr2=0x7f10cc100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:33.937 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.907+0000 7f10d3a81700 1 --2- 172.21.15.39:0/3315724416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10cc0ffbf0 0x7f10cc100040 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f10bc0098f0 tx=0x7f10bc009c00 comp rx=0 tx=0).stop 2023-07-21T18:39:33.937 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.907+0000 7f10d3a81700 1 -- 172.21.15.39:0/3315724416 shutdown_connections 2023-07-21T18:39:33.937 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.907+0000 7f10d3a81700 1 --2- 172.21.15.39:0/3315724416 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10cc0fea00 0x7f10cc0fee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:33.937 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.907+0000 7f10d3a81700 1 --2- 172.21.15.39:0/3315724416 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10cc100580 0x7f10cc106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:33.938 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.907+0000 7f10d3a81700 1 --2- 172.21.15.39:0/3315724416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10cc0ffbf0 0x7f10cc100040 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:33.938 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.907+0000 7f10d3a81700 1 -- 172.21.15.39:0/3315724416 >> 172.21.15.39:0/3315724416 conn(0x7f10cc0f9fe0 msgr2=0x7f10cc0fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:39:33.938 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.907+0000 7f10d3a81700 1 -- 172.21.15.39:0/3315724416 shutdown_connections 2023-07-21T18:39:33.938 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.907+0000 7f10d3a81700 1 -- 172.21.15.39:0/3315724416 wait complete. 2023-07-21T18:39:33.938 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.908+0000 7f10d3a81700 1 Processor -- start 2023-07-21T18:39:33.939 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.908+0000 7f10d3a81700 1 -- start start 2023-07-21T18:39:33.939 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.908+0000 7f10d3a81700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10cc0fea00 0x7f10cc1946d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:33.939 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.908+0000 7f10d3a81700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10cc0ffbf0 0x7f10cc194c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:33.939 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.908+0000 7f10d3a81700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10cc100580 0x7f10cc19bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:33.940 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.908+0000 7f10d3a81700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10cc19c250 con 0x7f10cc0fea00 2023-07-21T18:39:33.940 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.908+0000 7f10d3a81700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10cc19c390 con 0x7f10cc100580 2023-07-21T18:39:33.940 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.908+0000 7f10d3a81700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10cc19c520 con 0x7f10cc0ffbf0 2023-07-21T18:39:33.940 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.908+0000 7f10d181d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10cc0fea00 0x7f10cc1946d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:33.940 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.908+0000 7f10d181d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10cc0fea00 0x7f10cc1946d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54714/0 (socket says 172.21.15.39:54714) 2023-07-21T18:39:33.941 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.908+0000 7f10d181d700 1 -- 172.21.15.39:0/1146963355 learned_addr learned my addr 172.21.15.39:0/1146963355 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:39:33.941 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.908+0000 7f10d101c700 1 --2- 172.21.15.39:0/1146963355 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10cc0ffbf0 0x7f10cc194c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:33.941 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.908+0000 7f10d201e700 1 --2- 172.21.15.39:0/1146963355 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10cc100580 0x7f10cc19bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:33.941 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.908+0000 7f10d181d700 1 -- 172.21.15.39:0/1146963355 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10cc0ffbf0 msgr2=0x7f10cc194c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:33.942 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.908+0000 7f10d181d700 1 --2- 172.21.15.39:0/1146963355 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10cc0ffbf0 0x7f10cc194c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:33.942 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.908+0000 7f10d181d700 1 -- 172.21.15.39:0/1146963355 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10cc100580 msgr2=0x7f10cc19bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:33.942 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.908+0000 7f10d181d700 1 --2- 172.21.15.39:0/1146963355 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10cc100580 0x7f10cc19bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:33.942 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.908+0000 7f10d181d700 1 -- 172.21.15.39:0/1146963355 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10c4009d00 con 0x7f10cc0fea00 2023-07-21T18:39:33.942 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.908+0000 7f10d101c700 1 --2- 172.21.15.39:0/1146963355 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10cc0ffbf0 0x7f10cc194c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:39:33.943 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.908+0000 7f10d181d700 1 --2- 172.21.15.39:0/1146963355 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10cc0fea00 0x7f10cc1946d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f10c800d980 tx=0x7f10c800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:39:33.943 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.909+0000 7f10c2ffd700 1 -- 172.21.15.39:0/1146963355 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c8004020 con 0x7f10cc0fea00 2023-07-21T18:39:33.943 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.909+0000 7f10d3a81700 1 -- 172.21.15.39:0/1146963355 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10bc009520 con 0x7f10cc0fea00 2023-07-21T18:39:33.943 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.909+0000 7f10c2ffd700 1 -- 172.21.15.39:0/1146963355 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f10c800f690 con 0x7f10cc0fea00 2023-07-21T18:39:33.944 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.909+0000 7f10c2ffd700 1 -- 172.21.15.39:0/1146963355 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c8003a60 con 0x7f10cc0fea00 2023-07-21T18:39:33.944 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.909+0000 7f10d3a81700 1 -- 172.21.15.39:0/1146963355 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10cc19cb80 con 0x7f10cc0fea00 2023-07-21T18:39:33.944 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.909+0000 7f10d3a81700 1 -- 172.21.15.39:0/1146963355 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10cc10a170 con 0x7f10cc0fea00 2023-07-21T18:39:33.945 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.911+0000 7f10c2ffd700 1 -- 172.21.15.39:0/1146963355 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f10c800fb00 con 0x7f10cc0fea00 2023-07-21T18:39:33.945 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.911+0000 7f10c2ffd700 1 --2- 172.21.15.39:0/1146963355 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f10b80a11a0 0x7f10b80a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:33.946 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.911+0000 7f10d101c700 1 --2- 172.21.15.39:0/1146963355 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f10b80a11a0 0x7f10b80a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:33.946 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.911+0000 7f10c2ffd700 1 -- 172.21.15.39:0/1146963355 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f10c80cc3a0 con 0x7f10cc0fea00 2023-07-21T18:39:33.946 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.912+0000 7f10d101c700 1 --2- 172.21.15.39:0/1146963355 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f10b80a11a0 0x7f10b80a3650 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f10bc00fa40 tx=0x7f10bc019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:39:33.946 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:33.915+0000 7f10c2ffd700 1 -- 172.21.15.39:0/1146963355 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f10c8095f90 con 0x7f10cc0fea00 2023-07-21T18:39:34.186 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:34.159+0000 7f10d3a81700 1 -- 172.21.15.39:0/1146963355 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f10cc105920 con 0x7f10cc0fea00 2023-07-21T18:39:34.186 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:34.159+0000 7f10c2ffd700 1 -- 172.21.15.39:0/1146963355 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+501 (secure 0 0 0) 0x7f10c80995b0 con 0x7f10cc0fea00 2023-07-21T18:39:34.187 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:39:34.187 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2023-07-21T18:39:34.188 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:34.161+0000 7f10d3a81700 1 -- 172.21.15.39:0/1146963355 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f10b80a11a0 msgr2=0x7f10b80a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:34.189 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:34.161+0000 7f10d3a81700 1 --2- 172.21.15.39:0/1146963355 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f10b80a11a0 0x7f10b80a3650 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f10bc00fa40 tx=0x7f10bc019040 comp rx=0 tx=0).stop 2023-07-21T18:39:34.189 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:34.161+0000 7f10d3a81700 1 -- 172.21.15.39:0/1146963355 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10cc0fea00 msgr2=0x7f10cc1946d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:34.189 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:34.161+0000 7f10d3a81700 1 --2- 172.21.15.39:0/1146963355 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10cc0fea00 0x7f10cc1946d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f10c800d980 tx=0x7f10c800dc90 comp rx=0 tx=0).stop 2023-07-21T18:39:34.189 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:34.161+0000 7f10d3a81700 1 -- 172.21.15.39:0/1146963355 shutdown_connections 2023-07-21T18:39:34.189 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:34.161+0000 7f10d3a81700 1 --2- 172.21.15.39:0/1146963355 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f10b80a11a0 0x7f10b80a3650 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:34.189 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:34.161+0000 7f10d3a81700 1 --2- 172.21.15.39:0/1146963355 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10cc100580 0x7f10cc19bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:34.189 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:34.161+0000 7f10d3a81700 1 --2- 172.21.15.39:0/1146963355 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10cc0fea00 0x7f10cc1946d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:34.190 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:34.161+0000 7f10d3a81700 1 --2- 172.21.15.39:0/1146963355 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10cc0ffbf0 0x7f10cc194c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:34.190 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:34.161+0000 7f10d3a81700 1 -- 172.21.15.39:0/1146963355 >> 172.21.15.39:0/1146963355 conn(0x7f10cc0f9fe0 msgr2=0x7f10cc0fc270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:39:34.190 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:34.162+0000 7f10d3a81700 1 -- 172.21.15.39:0/1146963355 shutdown_connections 2023-07-21T18:39:34.190 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:34.162+0000 7f10d3a81700 1 -- 172.21.15.39:0/1146963355 wait complete. 2023-07-21T18:39:34.190 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 1 2023-07-21T18:39:34.292 INFO:journalctl@ceph.mds.l.smithi193.stdout:Jul 21 18:39:34 smithi193 systemd[1]: Starting Ceph mds.l for eaf8e25a-27f4-11ee-9b34-001a4aab830c... 2023-07-21T18:39:34.624 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:34 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/2451377886' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T18:39:34.625 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:34 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/422139633' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T18:39:34.625 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:34 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/1146963355' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:39:34.625 INFO:journalctl@ceph.mds.l.smithi193.stdout:Jul 21 18:39:34 smithi193 podman[123548]: 2023-07-21T18:39:34.625 INFO:journalctl@ceph.mds.l.smithi193.stdout:Jul 21 18:39:34 smithi193 bash[123548]: 7c8836bfc045c5f94d54ce0897fbb2fe06522089204c8f588c1fdaad4145cc09 2023-07-21T18:39:34.626 INFO:journalctl@ceph.mds.l.smithi193.stdout:Jul 21 18:39:34 smithi193 ceph-mds[123588]: -- 172.21.15.193:0/2984785777 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55cde6e471e0 con 0x55cde6e1d000 2023-07-21T18:39:34.626 INFO:journalctl@ceph.mds.l.smithi193.stdout:Jul 21 18:39:34 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-l[123564]: starting mds.l at 2023-07-21T18:39:34.626 INFO:journalctl@ceph.mds.l.smithi193.stdout:Jul 21 18:39:34 smithi193 systemd[1]: Started Ceph mds.l for eaf8e25a-27f4-11ee-9b34-001a4aab830c. 2023-07-21T18:39:34.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:34 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/2451377886' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T18:39:34.693 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:34 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/422139633' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T18:39:34.693 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:34 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/1146963355' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:39:34.893 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:34 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/2451377886' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T18:39:34.894 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:34 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/422139633' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T18:39:34.894 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:34 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/1146963355' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:39:35.774 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:35 smithi155 ceph-mon[97934]: mds.? [v2:172.21.15.193:6832/3106166804,v1:172.21.15.193:6833/3106166804] up:boot 2023-07-21T18:39:35.774 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:35 smithi155 ceph-mon[97934]: fsmap 1 up:standby 2023-07-21T18:39:35.774 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:35 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-21T18:39:35.775 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:35 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:35.775 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:35 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:35.775 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:35 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:35.775 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:35 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T18:39:35.775 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:35 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T18:39:35.775 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:35 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:39:35.775 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:35 smithi155 ceph-mon[97934]: Deploying daemon mds.k on smithi155 2023-07-21T18:39:35.776 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:35 smithi155 ceph-mon[97934]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:39:35.875 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:35 smithi193 ceph-mon[98204]: mds.? [v2:172.21.15.193:6832/3106166804,v1:172.21.15.193:6833/3106166804] up:boot 2023-07-21T18:39:35.875 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:35 smithi193 ceph-mon[98204]: fsmap 1 up:standby 2023-07-21T18:39:35.875 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:35 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-21T18:39:35.875 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:35 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:35.875 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:35 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:35.876 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:35 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:35.876 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:35 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T18:39:35.876 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:35 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T18:39:35.876 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:35 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:39:35.876 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:35 smithi193 ceph-mon[98204]: Deploying daemon mds.k on smithi155 2023-07-21T18:39:35.876 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:35 smithi193 ceph-mon[98204]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:39:35.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:35 smithi039 ceph-mon[96489]: mds.? [v2:172.21.15.193:6832/3106166804,v1:172.21.15.193:6833/3106166804] up:boot 2023-07-21T18:39:35.943 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:35 smithi039 ceph-mon[96489]: fsmap 1 up:standby 2023-07-21T18:39:35.943 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:35 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-21T18:39:35.943 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:35 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:35.943 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:35 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:35.943 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:35 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:35.943 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:35 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T18:39:35.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:35 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T18:39:35.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:35 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:39:35.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:35 smithi039 ceph-mon[96489]: Deploying daemon mds.k on smithi155 2023-07-21T18:39:35.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:35 smithi039 ceph-mon[96489]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:39:36.059 INFO:journalctl@ceph.mds.k.smithi155.stdout:Jul 21 18:39:36 smithi155 systemd[1]: Starting Ceph mds.k for eaf8e25a-27f4-11ee-9b34-001a4aab830c... 2023-07-21T18:39:36.201 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T18:39:36.342 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:36.315+0000 7fa751062700 1 -- 172.21.15.39:0/3254884733 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa74c100550 msgr2=0x7fa74c106e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:36.342 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:36.315+0000 7fa751062700 1 --2- 172.21.15.39:0/3254884733 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa74c100550 0x7fa74c106e10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa74400b0a0 tx=0x7fa74400b3b0 comp rx=0 tx=0).stop 2023-07-21T18:39:36.342 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:36.315+0000 7fa751062700 1 -- 172.21.15.39:0/3254884733 shutdown_connections 2023-07-21T18:39:36.342 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:36.315+0000 7fa751062700 1 --2- 172.21.15.39:0/3254884733 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa74c0fe9d0 0x7fa74c0fede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:36.342 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:36.315+0000 7fa751062700 1 --2- 172.21.15.39:0/3254884733 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa74c100550 0x7fa74c106e10 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:36.343 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:36.315+0000 7fa751062700 1 --2- 172.21.15.39:0/3254884733 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa74c0ffbc0 0x7fa74c100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:36.343 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:36.315+0000 7fa751062700 1 -- 172.21.15.39:0/3254884733 >> 172.21.15.39:0/3254884733 conn(0x7fa74c0f9fd0 msgr2=0x7fa74c0fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:39:36.343 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:36.315+0000 7fa751062700 1 -- 172.21.15.39:0/3254884733 shutdown_connections 2023-07-21T18:39:36.343 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:36.315+0000 7fa751062700 1 -- 172.21.15.39:0/3254884733 wait complete. 2023-07-21T18:39:36.345 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:36.315+0000 7fa751062700 1 Processor -- start 2023-07-21T18:39:36.345 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:36.316+0000 7fa751062700 1 -- start start 2023-07-21T18:39:36.345 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:36.316+0000 7fa751062700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa74c0fe9d0 0x7fa74c102960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:36.345 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:36.316+0000 7fa751062700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa74c0ffbc0 0x7fa74c100fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:36.345 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:36.316+0000 7fa751062700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa74c100550 0x7fa74c1014e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:36.346 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:36.316+0000 7fa751062700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa74c101c80 con 0x7fa74c0ffbc0 2023-07-21T18:39:36.346 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:36.316+0000 7fa751062700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa74c101df0 con 0x7fa74c0fe9d0 2023-07-21T18:39:36.346 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:36.316+0000 7fa751062700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa74c19f960 con 0x7fa74c100550 2023-07-21T18:39:36.346 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:36.316+0000 7fa74a59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa74c0ffbc0 0x7fa74c100fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:36.346 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:36.316+0000 7fa74a59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa74c0ffbc0 0x7fa74c100fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54730/0 (socket says 172.21.15.39:54730) 2023-07-21T18:39:36.346 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:36.316+0000 7fa74a59c700 1 -- 172.21.15.39:0/3978273095 learned_addr learned my addr 172.21.15.39:0/3978273095 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:39:36.346 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:36.316+0000 7fa74ad9d700 1 --2- 172.21.15.39:0/3978273095 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa74c0fe9d0 0x7fa74c102960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:36.347 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:36.316+0000 7fa74b59e700 1 --2- 172.21.15.39:0/3978273095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa74c100550 0x7fa74c1014e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:36.347 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:36.316+0000 7fa74a59c700 1 -- 172.21.15.39:0/3978273095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa74c100550 msgr2=0x7fa74c1014e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:36.347 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:36.316+0000 7fa74a59c700 1 --2- 172.21.15.39:0/3978273095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa74c100550 0x7fa74c1014e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:36.347 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:36.316+0000 7fa74a59c700 1 -- 172.21.15.39:0/3978273095 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa74c0fe9d0 msgr2=0x7fa74c102960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:36.347 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:36.316+0000 7fa74a59c700 1 --2- 172.21.15.39:0/3978273095 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa74c0fe9d0 0x7fa74c102960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:36.347 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:36.316+0000 7fa74a59c700 1 -- 172.21.15.39:0/3978273095 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa744009d00 con 0x7fa74c0ffbc0 2023-07-21T18:39:36.347 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:36.317+0000 7fa74a59c700 1 --2- 172.21.15.39:0/3978273095 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa74c0ffbc0 0x7fa74c100fa0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa73400c920 tx=0x7fa73400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:39:36.348 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:36.317+0000 7fa743fff700 1 -- 172.21.15.39:0/3978273095 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa734007b00 con 0x7fa74c0ffbc0 2023-07-21T18:39:36.348 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:36.317+0000 7fa743fff700 1 -- 172.21.15.39:0/3978273095 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa73400ce70 con 0x7fa74c0ffbc0 2023-07-21T18:39:36.348 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:36.317+0000 7fa743fff700 1 -- 172.21.15.39:0/3978273095 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7340054e0 con 0x7fa74c0ffbc0 2023-07-21T18:39:36.348 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:36.317+0000 7fa751062700 1 -- 172.21.15.39:0/3978273095 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa74c19fc40 con 0x7fa74c0ffbc0 2023-07-21T18:39:36.348 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:36.317+0000 7fa751062700 1 -- 172.21.15.39:0/3978273095 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa74c1a0100 con 0x7fa74c0ffbc0 2023-07-21T18:39:36.348 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:36.318+0000 7fa751062700 1 -- 172.21.15.39:0/3978273095 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa74c10a200 con 0x7fa74c0ffbc0 2023-07-21T18:39:36.349 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:36.319+0000 7fa743fff700 1 -- 172.21.15.39:0/3978273095 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fa734004750 con 0x7fa74c0ffbc0 2023-07-21T18:39:36.349 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:36.319+0000 7fa743fff700 1 --2- 172.21.15.39:0/3978273095 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fa7380a11a0 0x7fa7380a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:36.349 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:36.320+0000 7fa74ad9d700 1 --2- 172.21.15.39:0/3978273095 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fa7380a11a0 0x7fa7380a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:36.349 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:36.320+0000 7fa743fff700 1 -- 172.21.15.39:0/3978273095 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa7340cc410 con 0x7fa74c0ffbc0 2023-07-21T18:39:36.350 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:36.320+0000 7fa74ad9d700 1 --2- 172.21.15.39:0/3978273095 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fa7380a11a0 0x7fa7380a3650 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa74c0ffa20 tx=0x7fa73c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:39:36.350 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:36.323+0000 7fa743fff700 1 -- 172.21.15.39:0/3978273095 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fa734096000 con 0x7fa74c0ffbc0 2023-07-21T18:39:36.595 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:36.568+0000 7fa751062700 1 -- 172.21.15.39:0/3978273095 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa74c1a0710 con 0x7fa74c0ffbc0 2023-07-21T18:39:36.596 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:36.569+0000 7fa743fff700 1 -- 172.21.15.39:0/3978273095 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1260 (secure 0 0 0) 0x7fa73401d500 con 0x7fa74c0ffbc0 2023-07-21T18:39:36.596 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:39:36.596 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/3106166804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3106166804},{"type":"v1","addr":"172.21.15.193:6833","nonce":3106166804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-07-21T18:39:36.597 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:36.570+0000 7fa751062700 1 -- 172.21.15.39:0/3978273095 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fa7380a11a0 msgr2=0x7fa7380a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:36.598 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:36.570+0000 7fa751062700 1 --2- 172.21.15.39:0/3978273095 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fa7380a11a0 0x7fa7380a3650 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa74c0ffa20 tx=0x7fa73c00a380 comp rx=0 tx=0).stop 2023-07-21T18:39:36.598 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:36.570+0000 7fa751062700 1 -- 172.21.15.39:0/3978273095 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa74c0ffbc0 msgr2=0x7fa74c100fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:36.598 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:36.570+0000 7fa751062700 1 --2- 172.21.15.39:0/3978273095 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa74c0ffbc0 0x7fa74c100fa0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa73400c920 tx=0x7fa73400cc30 comp rx=0 tx=0).stop 2023-07-21T18:39:36.598 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:36.570+0000 7fa751062700 1 -- 172.21.15.39:0/3978273095 shutdown_connections 2023-07-21T18:39:36.598 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:36.570+0000 7fa751062700 1 --2- 172.21.15.39:0/3978273095 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fa7380a11a0 0x7fa7380a3650 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:36.598 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:36.570+0000 7fa751062700 1 --2- 172.21.15.39:0/3978273095 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa74c0fe9d0 0x7fa74c102960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:36.598 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:36.570+0000 7fa751062700 1 --2- 172.21.15.39:0/3978273095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa74c100550 0x7fa74c1014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:36.599 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:36.570+0000 7fa751062700 1 --2- 172.21.15.39:0/3978273095 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa74c0ffbc0 0x7fa74c100fa0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:36.599 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:36.571+0000 7fa751062700 1 -- 172.21.15.39:0/3978273095 >> 172.21.15.39:0/3978273095 conn(0x7fa74c0f9fd0 msgr2=0x7fa74c067730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:39:36.599 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:36.571+0000 7fa751062700 1 -- 172.21.15.39:0/3978273095 shutdown_connections 2023-07-21T18:39:36.599 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:36.571+0000 7fa751062700 1 -- 172.21.15.39:0/3978273095 wait complete. 2023-07-21T18:39:36.599 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 2 2023-07-21T18:39:36.603 INFO:journalctl@ceph.mds.k.smithi155.stdout:Jul 21 18:39:36 smithi155 podman[122232]: 2023-07-21T18:39:36.893 INFO:journalctl@ceph.mds.k.smithi155.stdout:Jul 21 18:39:36 smithi155 bash[122232]: 5d4b3ec0067ad747babeb463b53cd748abed2e15f67588b00fae4147eb50e013 2023-07-21T18:39:36.893 INFO:journalctl@ceph.mds.k.smithi155.stdout:Jul 21 18:39:36 smithi155 ceph-mds[122272]: -- 172.21.15.155:0/2137373669 <== mon.2 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5596d82691e0 con 0x5596d823f000 2023-07-21T18:39:36.894 INFO:journalctl@ceph.mds.k.smithi155.stdout:Jul 21 18:39:36 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-k[122248]: starting mds.k at 2023-07-21T18:39:36.894 INFO:journalctl@ceph.mds.k.smithi155.stdout:Jul 21 18:39:36 smithi155 systemd[1]: Started Ceph mds.k for eaf8e25a-27f4-11ee-9b34-001a4aab830c. 2023-07-21T18:39:37.311 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:36 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/3978273095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:39:37.311 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:36 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:37.312 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:36 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:37.312 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:36 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:37.312 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:36 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T18:39:37.312 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:36 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T18:39:37.313 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:36 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:39:37.375 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:36 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/3978273095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:39:37.375 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:36 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:37.375 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:36 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:37.375 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:36 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:37.375 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:36 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T18:39:37.375 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:36 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T18:39:37.376 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:36 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:39:37.393 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:36 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/3978273095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:39:37.393 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:36 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:37.394 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:36 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:37.394 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:36 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:37.394 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:36 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T18:39:37.394 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:36 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T18:39:37.394 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:36 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:39:38.193 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:37 smithi039 ceph-mon[96489]: Deploying daemon mds.j on smithi039 2023-07-21T18:39:38.193 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:37 smithi039 ceph-mon[96489]: mds.? [v2:172.21.15.155:6832/1034755277,v1:172.21.15.155:6833/1034755277] up:boot 2023-07-21T18:39:38.193 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:37 smithi039 ceph-mon[96489]: fsmap 2 up:standby 2023-07-21T18:39:38.193 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:37 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-21T18:39:38.193 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:37 smithi039 ceph-mon[96489]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:39:38.194 INFO:journalctl@ceph.mds.j.smithi039.stdout:Jul 21 18:39:38 smithi039 systemd[1]: Starting Ceph mds.j for eaf8e25a-27f4-11ee-9b34-001a4aab830c... 2023-07-21T18:39:38.374 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:37 smithi193 ceph-mon[98204]: Deploying daemon mds.j on smithi039 2023-07-21T18:39:38.375 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:37 smithi193 ceph-mon[98204]: mds.? [v2:172.21.15.155:6832/1034755277,v1:172.21.15.155:6833/1034755277] up:boot 2023-07-21T18:39:38.375 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:37 smithi193 ceph-mon[98204]: fsmap 2 up:standby 2023-07-21T18:39:38.375 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:37 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-21T18:39:38.375 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:37 smithi193 ceph-mon[98204]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:39:38.393 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:37 smithi155 ceph-mon[97934]: Deploying daemon mds.j on smithi039 2023-07-21T18:39:38.393 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:37 smithi155 ceph-mon[97934]: mds.? [v2:172.21.15.155:6832/1034755277,v1:172.21.15.155:6833/1034755277] up:boot 2023-07-21T18:39:38.394 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:37 smithi155 ceph-mon[97934]: fsmap 2 up:standby 2023-07-21T18:39:38.394 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:37 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-21T18:39:38.394 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:37 smithi155 ceph-mon[97934]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:39:38.611 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T18:39:38.641 INFO:journalctl@ceph.mds.j.smithi039.stdout:Jul 21 18:39:38 smithi039 podman[129783]: 2023-07-21T18:39:38.751 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:38.724+0000 7fcec7b26700 1 -- 172.21.15.39:0/674467521 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcec00fea10 msgr2=0x7fcec00fee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:38.751 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:38.724+0000 7fcec7b26700 1 --2- 172.21.15.39:0/674467521 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcec00fea10 0x7fcec00fee20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fceb0009990 tx=0x7fceb0009ca0 comp rx=0 tx=0).stop 2023-07-21T18:39:38.751 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:38.724+0000 7fcec7b26700 1 -- 172.21.15.39:0/674467521 shutdown_connections 2023-07-21T18:39:38.752 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:38.724+0000 7fcec7b26700 1 --2- 172.21.15.39:0/674467521 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcec0100590 0x7fcec0106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:38.752 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:38.724+0000 7fcec7b26700 1 --2- 172.21.15.39:0/674467521 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcec00fea10 0x7fcec00fee20 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:38.752 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:38.724+0000 7fcec7b26700 1 --2- 172.21.15.39:0/674467521 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcec00ffc00 0x7fcec0100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:38.752 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:38.724+0000 7fcec7b26700 1 -- 172.21.15.39:0/674467521 >> 172.21.15.39:0/674467521 conn(0x7fcec00fa030 msgr2=0x7fcec00fc460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:39:38.752 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:38.724+0000 7fcec7b26700 1 -- 172.21.15.39:0/674467521 shutdown_connections 2023-07-21T18:39:38.752 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:38.724+0000 7fcec7b26700 1 -- 172.21.15.39:0/674467521 wait complete. 2023-07-21T18:39:38.754 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:38.725+0000 7fcec7b26700 1 Processor -- start 2023-07-21T18:39:38.754 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:38.725+0000 7fcec7b26700 1 -- start start 2023-07-21T18:39:38.754 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:38.725+0000 7fcec7b26700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcec00fea10 0x7fcec00691d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:38.755 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:38.725+0000 7fcec7b26700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcec00ffc00 0x7fcec019a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:38.755 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:38.725+0000 7fcec7b26700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcec0100590 0x7fcec019cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:38.755 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:38.725+0000 7fcec7b26700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec019d360 con 0x7fcec00fea10 2023-07-21T18:39:38.755 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:38.725+0000 7fcec7b26700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec019d4d0 con 0x7fcec0100590 2023-07-21T18:39:38.755 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:38.725+0000 7fcec7b26700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec019d690 con 0x7fcec00ffc00 2023-07-21T18:39:38.755 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:38.725+0000 7fcec58c2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcec00fea10 0x7fcec00691d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:38.755 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:38.725+0000 7fcec58c2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcec00fea10 0x7fcec00691d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54750/0 (socket says 172.21.15.39:54750) 2023-07-21T18:39:38.756 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:38.725+0000 7fcec58c2700 1 -- 172.21.15.39:0/3778894814 learned_addr learned my addr 172.21.15.39:0/3778894814 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:39:38.756 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:38.725+0000 7fcec50c1700 1 --2- 172.21.15.39:0/3778894814 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcec00ffc00 0x7fcec019a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:38.756 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:38.725+0000 7fcec60c3700 1 --2- 172.21.15.39:0/3778894814 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcec0100590 0x7fcec019cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:38.756 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:38.725+0000 7fcec58c2700 1 -- 172.21.15.39:0/3778894814 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcec00ffc00 msgr2=0x7fcec019a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:38.756 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:38.725+0000 7fcec58c2700 1 --2- 172.21.15.39:0/3778894814 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcec00ffc00 0x7fcec019a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:38.756 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:38.725+0000 7fcec58c2700 1 -- 172.21.15.39:0/3778894814 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcec0100590 msgr2=0x7fcec019cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:38.756 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:38.725+0000 7fcec58c2700 1 --2- 172.21.15.39:0/3778894814 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcec0100590 0x7fcec019cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:38.756 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:38.725+0000 7fcec58c2700 1 -- 172.21.15.39:0/3778894814 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceb00095f0 con 0x7fcec00fea10 2023-07-21T18:39:38.757 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:38.725+0000 7fcec50c1700 1 --2- 172.21.15.39:0/3778894814 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcec00ffc00 0x7fcec019a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:39:38.757 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:38.725+0000 7fcec58c2700 1 --2- 172.21.15.39:0/3778894814 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcec00fea10 0x7fcec00691d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fceb0004760 tx=0x7fceb0009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:39:38.757 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:38.726+0000 7fceb6ffd700 1 -- 172.21.15.39:0/3778894814 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb001d070 con 0x7fcec00fea10 2023-07-21T18:39:38.757 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:38.726+0000 7fcec7b26700 1 -- 172.21.15.39:0/3778894814 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcec019d910 con 0x7fcec00fea10 2023-07-21T18:39:38.757 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:38.726+0000 7fceb6ffd700 1 -- 172.21.15.39:0/3778894814 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fceb0004120 con 0x7fcec00fea10 2023-07-21T18:39:38.757 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:38.726+0000 7fceb6ffd700 1 -- 172.21.15.39:0/3778894814 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb0005290 con 0x7fcec00fea10 2023-07-21T18:39:38.757 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:38.726+0000 7fcec7b26700 1 -- 172.21.15.39:0/3778894814 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcec019dd30 con 0x7fcec00fea10 2023-07-21T18:39:38.757 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:38.727+0000 7fcec7b26700 1 -- 172.21.15.39:0/3778894814 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcec010a230 con 0x7fcec00fea10 2023-07-21T18:39:38.758 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:38.728+0000 7fceb6ffd700 1 -- 172.21.15.39:0/3778894814 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fceb0004810 con 0x7fcec00fea10 2023-07-21T18:39:38.758 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:38.728+0000 7fceb6ffd700 1 --2- 172.21.15.39:0/3778894814 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fceac0a11a0 0x7fceac0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:38.758 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:38.728+0000 7fceb6ffd700 1 -- 172.21.15.39:0/3778894814 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fceb00cd920 con 0x7fcec00fea10 2023-07-21T18:39:38.759 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:38.728+0000 7fcec50c1700 1 --2- 172.21.15.39:0/3778894814 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fceac0a11a0 0x7fceac0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:38.759 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:38.729+0000 7fcec50c1700 1 --2- 172.21.15.39:0/3778894814 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fceac0a11a0 0x7fceac0a3650 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcec0068f40 tx=0x7fcebc005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:39:38.759 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:38.732+0000 7fceb6ffd700 1 -- 172.21.15.39:0/3778894814 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fceb00975a0 con 0x7fcec00fea10 2023-07-21T18:39:38.942 INFO:journalctl@ceph.mds.j.smithi039.stdout:Jul 21 18:39:38 smithi039 bash[129783]: 6c7a67820ddedba98cfb53f94eac093a98d42dc2bb9f543b03ebcdf044ba87d6 2023-07-21T18:39:38.942 INFO:journalctl@ceph.mds.j.smithi039.stdout:Jul 21 18:39:38 smithi039 ceph-mds[129832]: -- 172.21.15.39:0/2180387487 <== mon.0 v2:172.21.15.39:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5579e763b1e0 con 0x5579e7611000 2023-07-21T18:39:38.943 INFO:journalctl@ceph.mds.j.smithi039.stdout:Jul 21 18:39:38 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-j[129799]: starting mds.j at 2023-07-21T18:39:38.943 INFO:journalctl@ceph.mds.j.smithi039.stdout:Jul 21 18:39:38 smithi039 systemd[1]: Started Ceph mds.j for eaf8e25a-27f4-11ee-9b34-001a4aab830c. 2023-07-21T18:39:39.020 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:38.993+0000 7fcec7b26700 1 -- 172.21.15.39:0/3778894814 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcec0105860 con 0x7fcec00fea10 2023-07-21T18:39:39.021 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:38.994+0000 7fceb6ffd700 1 -- 172.21.15.39:0/3778894814 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2774 (secure 0 0 0) 0x7fceb009abc0 con 0x7fcec00fea10 2023-07-21T18:39:39.021 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:39:39.021 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/147318681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":147318681},{"type":"v1","addr":"172.21.15.39:6835","nonce":147318681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/3106166804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3106166804},{"type":"v1","addr":"172.21.15.193:6833","nonce":3106166804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1034755277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1034755277},{"type":"v1","addr":"172.21.15.155:6833","nonce":1034755277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-07-21T18:39:39.023 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:38.996+0000 7fcec7b26700 1 -- 172.21.15.39:0/3778894814 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fceac0a11a0 msgr2=0x7fceac0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:39.023 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:38.996+0000 7fcec7b26700 1 --2- 172.21.15.39:0/3778894814 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fceac0a11a0 0x7fceac0a3650 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcec0068f40 tx=0x7fcebc005f50 comp rx=0 tx=0).stop 2023-07-21T18:39:39.023 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:38.996+0000 7fcec7b26700 1 -- 172.21.15.39:0/3778894814 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcec00fea10 msgr2=0x7fcec00691d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:39.023 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:38.996+0000 7fcec7b26700 1 --2- 172.21.15.39:0/3778894814 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcec00fea10 0x7fcec00691d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fceb0004760 tx=0x7fceb0009580 comp rx=0 tx=0).stop 2023-07-21T18:39:39.023 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:38.996+0000 7fcec7b26700 1 -- 172.21.15.39:0/3778894814 shutdown_connections 2023-07-21T18:39:39.024 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:38.996+0000 7fcec7b26700 1 --2- 172.21.15.39:0/3778894814 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fceac0a11a0 0x7fceac0a3650 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:39.024 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:38.996+0000 7fcec7b26700 1 --2- 172.21.15.39:0/3778894814 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcec0100590 0x7fcec019cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:39.024 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:38.996+0000 7fcec7b26700 1 --2- 172.21.15.39:0/3778894814 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcec00fea10 0x7fcec00691d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:39.024 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:38.996+0000 7fcec7b26700 1 --2- 172.21.15.39:0/3778894814 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcec00ffc00 0x7fcec019a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:39.024 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:38.996+0000 7fcec7b26700 1 -- 172.21.15.39:0/3778894814 >> 172.21.15.39:0/3778894814 conn(0x7fcec00fa030 msgr2=0x7fcec00fc210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:39:39.024 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:38.996+0000 7fcec7b26700 1 -- 172.21.15.39:0/3778894814 shutdown_connections 2023-07-21T18:39:39.024 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:38.996+0000 7fcec7b26700 1 -- 172.21.15.39:0/3778894814 wait complete. 2023-07-21T18:39:39.025 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 4 2023-07-21T18:39:39.926 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:39 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:39.926 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:39 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:39.927 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:39 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:39.927 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:39 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T18:39:39.927 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:39 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T18:39:39.927 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:39 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:39:39.928 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:39 smithi039 ceph-mon[96489]: Deploying daemon mds.d on smithi039 2023-07-21T18:39:39.928 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:39 smithi039 ceph-mon[96489]: mds.? [v2:172.21.15.39:6834/147318681,v1:172.21.15.39:6835/147318681] up:boot 2023-07-21T18:39:39.928 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:39 smithi039 ceph-mon[96489]: fsmap 3 up:standby 2023-07-21T18:39:39.928 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:39 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-07-21T18:39:39.928 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:39 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/3778894814' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:39:39.928 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:39 smithi039 ceph-mon[96489]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:39:40.124 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:39 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:40.125 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:39 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:40.125 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:39 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:40.125 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:39 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T18:39:40.125 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:39 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T18:39:40.125 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:39 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:39:40.125 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:39 smithi193 ceph-mon[98204]: Deploying daemon mds.d on smithi039 2023-07-21T18:39:40.126 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:39 smithi193 ceph-mon[98204]: mds.? [v2:172.21.15.39:6834/147318681,v1:172.21.15.39:6835/147318681] up:boot 2023-07-21T18:39:40.126 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:39 smithi193 ceph-mon[98204]: fsmap 3 up:standby 2023-07-21T18:39:40.126 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:39 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-07-21T18:39:40.126 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:39 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/3778894814' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:39:40.126 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:39 smithi193 ceph-mon[98204]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:39:40.143 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:39 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:40.143 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:39 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:40.144 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:39 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:40.144 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:39 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T18:39:40.144 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:39 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T18:39:40.144 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:39 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:39:40.144 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:39 smithi155 ceph-mon[97934]: Deploying daemon mds.d on smithi039 2023-07-21T18:39:40.144 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:39 smithi155 ceph-mon[97934]: mds.? [v2:172.21.15.39:6834/147318681,v1:172.21.15.39:6835/147318681] up:boot 2023-07-21T18:39:40.144 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:39 smithi155 ceph-mon[97934]: fsmap 3 up:standby 2023-07-21T18:39:40.145 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:39 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-07-21T18:39:40.145 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:39 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/3778894814' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:39:40.145 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:39 smithi155 ceph-mon[97934]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:39:40.178 INFO:journalctl@ceph.mds.d.smithi039.stdout:Jul 21 18:39:40 smithi039 systemd[1]: Starting Ceph mds.d for eaf8e25a-27f4-11ee-9b34-001a4aab830c... 2023-07-21T18:39:40.722 INFO:journalctl@ceph.mds.d.smithi039.stdout:Jul 21 18:39:40 smithi039 podman[130171]: 2023-07-21T18:39:40.722 INFO:journalctl@ceph.mds.d.smithi039.stdout:Jul 21 18:39:40 smithi039 bash[130171]: 6a19ff935597d405f95068f42e80184bf15de91b3a686b19976dc6c7a5cd8c62 2023-07-21T18:39:40.723 INFO:journalctl@ceph.mds.d.smithi039.stdout:Jul 21 18:39:40 smithi039 ceph-mds[130211]: -- 172.21.15.39:0/1576678824 <== mon.0 v2:172.21.15.39:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e0251511e0 con 0x55e025126c00 2023-07-21T18:39:41.036 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T18:39:41.064 INFO:journalctl@ceph.mds.d.smithi039.stdout:Jul 21 18:39:40 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-d[130187]: starting mds.d at 2023-07-21T18:39:41.065 INFO:journalctl@ceph.mds.d.smithi039.stdout:Jul 21 18:39:40 smithi039 systemd[1]: Started Ceph mds.d for eaf8e25a-27f4-11ee-9b34-001a4aab830c. 2023-07-21T18:39:41.168 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:41.141+0000 7fc08b3c7700 1 -- 172.21.15.39:0/123353360 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc084100550 msgr2=0x7fc084106e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:41.169 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:41.141+0000 7fc08b3c7700 1 --2- 172.21.15.39:0/123353360 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc084100550 0x7fc084106e10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc07c00b0a0 tx=0x7fc07c00b3b0 comp rx=0 tx=0).stop 2023-07-21T18:39:41.169 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:41.141+0000 7fc08b3c7700 1 -- 172.21.15.39:0/123353360 shutdown_connections 2023-07-21T18:39:41.169 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:41.141+0000 7fc08b3c7700 1 --2- 172.21.15.39:0/123353360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0840fe9d0 0x7fc0840fede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:41.169 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:41.141+0000 7fc08b3c7700 1 --2- 172.21.15.39:0/123353360 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc084100550 0x7fc084106e10 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:41.169 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:41.141+0000 7fc08b3c7700 1 --2- 172.21.15.39:0/123353360 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0840ffbc0 0x7fc084100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:41.169 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:41.141+0000 7fc08b3c7700 1 -- 172.21.15.39:0/123353360 >> 172.21.15.39:0/123353360 conn(0x7fc0840f9fd0 msgr2=0x7fc0840fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:39:41.169 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:41.142+0000 7fc08b3c7700 1 -- 172.21.15.39:0/123353360 shutdown_connections 2023-07-21T18:39:41.170 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:41.142+0000 7fc08b3c7700 1 -- 172.21.15.39:0/123353360 wait complete. 2023-07-21T18:39:41.171 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:41.142+0000 7fc08b3c7700 1 Processor -- start 2023-07-21T18:39:41.171 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:41.142+0000 7fc08b3c7700 1 -- start start 2023-07-21T18:39:41.172 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:41.142+0000 7fc08b3c7700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0840fe9d0 0x7fc084195900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:41.172 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:41.142+0000 7fc08b3c7700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0840ffbc0 0x7fc084195e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:41.172 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:41.142+0000 7fc08b3c7700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc084100550 0x7fc08419ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:41.172 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:41.142+0000 7fc08b3c7700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08419d480 con 0x7fc084100550 2023-07-21T18:39:41.172 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:41.142+0000 7fc08b3c7700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08419d5c0 con 0x7fc0840ffbc0 2023-07-21T18:39:41.172 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:41.142+0000 7fc08b3c7700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08419d750 con 0x7fc0840fe9d0 2023-07-21T18:39:41.172 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:41.142+0000 7fc089964700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc084100550 0x7fc08419ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:41.173 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:41.143+0000 7fc089964700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc084100550 0x7fc08419ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54782/0 (socket says 172.21.15.39:54782) 2023-07-21T18:39:41.173 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:41.143+0000 7fc089964700 1 -- 172.21.15.39:0/3898604702 learned_addr learned my addr 172.21.15.39:0/3898604702 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:39:41.173 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:41.143+0000 7fc088962700 1 --2- 172.21.15.39:0/3898604702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0840ffbc0 0x7fc084195e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:41.173 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:41.143+0000 7fc089163700 1 --2- 172.21.15.39:0/3898604702 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0840fe9d0 0x7fc084195900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:41.173 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:41.143+0000 7fc089964700 1 -- 172.21.15.39:0/3898604702 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0840fe9d0 msgr2=0x7fc084195900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:41.173 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:41.143+0000 7fc089964700 1 --2- 172.21.15.39:0/3898604702 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0840fe9d0 0x7fc084195900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:41.173 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:41.143+0000 7fc089964700 1 -- 172.21.15.39:0/3898604702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0840ffbc0 msgr2=0x7fc084195e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:41.173 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:41.143+0000 7fc089964700 1 --2- 172.21.15.39:0/3898604702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0840ffbc0 0x7fc084195e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:41.174 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:41.143+0000 7fc089964700 1 -- 172.21.15.39:0/3898604702 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc07c009d00 con 0x7fc084100550 2023-07-21T18:39:41.174 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:41.143+0000 7fc089964700 1 --2- 172.21.15.39:0/3898604702 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc084100550 0x7fc08419ceb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc07c00d010 tx=0x7fc07c004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:39:41.174 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:41.143+0000 7fc0767fc700 1 -- 172.21.15.39:0/3898604702 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07c00e040 con 0x7fc084100550 2023-07-21T18:39:41.174 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:41.143+0000 7fc0767fc700 1 -- 172.21.15.39:0/3898604702 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc07c009440 con 0x7fc084100550 2023-07-21T18:39:41.174 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:41.143+0000 7fc0767fc700 1 -- 172.21.15.39:0/3898604702 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07c004020 con 0x7fc084100550 2023-07-21T18:39:41.174 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:41.143+0000 7fc08b3c7700 1 -- 172.21.15.39:0/3898604702 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc08419d9d0 con 0x7fc084100550 2023-07-21T18:39:41.174 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:41.143+0000 7fc08b3c7700 1 -- 172.21.15.39:0/3898604702 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc08419de00 con 0x7fc084100550 2023-07-21T18:39:41.175 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:41.144+0000 7fc08b3c7700 1 -- 172.21.15.39:0/3898604702 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc084106a10 con 0x7fc084100550 2023-07-21T18:39:41.175 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:41.145+0000 7fc0767fc700 1 -- 172.21.15.39:0/3898604702 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fc07c01d690 con 0x7fc084100550 2023-07-21T18:39:41.175 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:41.146+0000 7fc0767fc700 1 --2- 172.21.15.39:0/3898604702 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fc0700a11a0 0x7fc0700a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:41.176 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:41.146+0000 7fc0767fc700 1 -- 172.21.15.39:0/3898604702 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc07c0ce330 con 0x7fc084100550 2023-07-21T18:39:41.176 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:41.146+0000 7fc089163700 1 --2- 172.21.15.39:0/3898604702 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fc0700a11a0 0x7fc0700a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:41.176 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:41.146+0000 7fc089163700 1 --2- 172.21.15.39:0/3898604702 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fc0700a11a0 0x7fc0700a3650 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc078005950 tx=0x7fc0780058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:39:41.176 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:41.149+0000 7fc0767fc700 1 -- 172.21.15.39:0/3898604702 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fc07c097f10 con 0x7fc084100550 2023-07-21T18:39:41.421 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:41.393+0000 7fc08b3c7700 1 -- 172.21.15.39:0/3898604702 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc08402c850 con 0x7fc084100550 2023-07-21T18:39:41.421 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:41.393+0000 7fc0767fc700 1 -- 172.21.15.39:0/3898604702 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3531 (secure 0 0 0) 0x7fc07c021de0 con 0x7fc084100550 2023-07-21T18:39:41.421 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:39:41.421 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/147318681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":147318681},{"type":"v1","addr":"172.21.15.39:6835","nonce":147318681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/1062891760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1062891760},{"type":"v1","addr":"172.21.15.39:6837","nonce":1062891760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/3106166804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3106166804},{"type":"v1","addr":"172.21.15.193:6833","nonce":3106166804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1034755277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1034755277},{"type":"v1","addr":"172.21.15.155:6833","nonce":1034755277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-07-21T18:39:41.423 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:41.395+0000 7fc08b3c7700 1 -- 172.21.15.39:0/3898604702 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fc0700a11a0 msgr2=0x7fc0700a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:41.423 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:41.395+0000 7fc08b3c7700 1 --2- 172.21.15.39:0/3898604702 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fc0700a11a0 0x7fc0700a3650 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc078005950 tx=0x7fc0780058e0 comp rx=0 tx=0).stop 2023-07-21T18:39:41.423 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:41.395+0000 7fc08b3c7700 1 -- 172.21.15.39:0/3898604702 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc084100550 msgr2=0x7fc08419ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:41.423 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:41.395+0000 7fc08b3c7700 1 --2- 172.21.15.39:0/3898604702 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc084100550 0x7fc08419ceb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc07c00d010 tx=0x7fc07c004c80 comp rx=0 tx=0).stop 2023-07-21T18:39:41.423 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:41.395+0000 7fc08b3c7700 1 -- 172.21.15.39:0/3898604702 shutdown_connections 2023-07-21T18:39:41.423 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:41.395+0000 7fc08b3c7700 1 --2- 172.21.15.39:0/3898604702 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fc0700a11a0 0x7fc0700a3650 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:41.423 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:41.395+0000 7fc08b3c7700 1 --2- 172.21.15.39:0/3898604702 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc084100550 0x7fc08419ceb0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:41.424 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:41.395+0000 7fc08b3c7700 1 --2- 172.21.15.39:0/3898604702 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0840fe9d0 0x7fc084195900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:41.424 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:41.395+0000 7fc08b3c7700 1 --2- 172.21.15.39:0/3898604702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0840ffbc0 0x7fc084195e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:41.424 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:41.395+0000 7fc08b3c7700 1 -- 172.21.15.39:0/3898604702 >> 172.21.15.39:0/3898604702 conn(0x7fc0840f9fd0 msgr2=0x7fc0840fc140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:39:41.424 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:41.395+0000 7fc08b3c7700 1 -- 172.21.15.39:0/3898604702 shutdown_connections 2023-07-21T18:39:41.424 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:41.395+0000 7fc08b3c7700 1 -- 172.21.15.39:0/3898604702 wait complete. 2023-07-21T18:39:41.424 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 5 2023-07-21T18:39:42.090 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:41 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:42.090 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:41 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:42.090 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:41 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:42.090 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:41 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T18:39:42.090 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:41 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T18:39:42.091 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:41 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:39:42.091 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:41 smithi155 ceph-mon[97934]: Deploying daemon mds.e on smithi155 2023-07-21T18:39:42.091 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:41 smithi155 ceph-mon[97934]: mds.? [v2:172.21.15.39:6836/1062891760,v1:172.21.15.39:6837/1062891760] up:boot 2023-07-21T18:39:42.091 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:41 smithi155 ceph-mon[97934]: fsmap 4 up:standby 2023-07-21T18:39:42.091 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:41 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-07-21T18:39:42.091 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:41 smithi155 ceph-mon[97934]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:39:42.091 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:41 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/3898604702' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:39:42.124 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:41 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:42.125 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:41 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:42.125 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:41 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:42.125 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:41 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T18:39:42.125 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:41 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T18:39:42.126 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:41 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:39:42.126 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:41 smithi193 ceph-mon[98204]: Deploying daemon mds.e on smithi155 2023-07-21T18:39:42.126 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:41 smithi193 ceph-mon[98204]: mds.? [v2:172.21.15.39:6836/1062891760,v1:172.21.15.39:6837/1062891760] up:boot 2023-07-21T18:39:42.126 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:41 smithi193 ceph-mon[98204]: fsmap 4 up:standby 2023-07-21T18:39:42.126 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:41 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-07-21T18:39:42.126 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:41 smithi193 ceph-mon[98204]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:39:42.127 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:41 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/3898604702' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:39:42.192 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:41 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:42.193 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:41 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:42.193 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:41 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:42.193 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:41 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T18:39:42.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:41 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T18:39:42.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:41 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:39:42.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:41 smithi039 ceph-mon[96489]: Deploying daemon mds.e on smithi155 2023-07-21T18:39:42.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:41 smithi039 ceph-mon[96489]: mds.? [v2:172.21.15.39:6836/1062891760,v1:172.21.15.39:6837/1062891760] up:boot 2023-07-21T18:39:42.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:41 smithi039 ceph-mon[96489]: fsmap 4 up:standby 2023-07-21T18:39:42.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:41 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-07-21T18:39:42.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:41 smithi039 ceph-mon[96489]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:39:42.196 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:41 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/3898604702' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:39:42.393 INFO:journalctl@ceph.mds.e.smithi155.stdout:Jul 21 18:39:42 smithi155 systemd[1]: Starting Ceph mds.e for eaf8e25a-27f4-11ee-9b34-001a4aab830c... 2023-07-21T18:39:42.877 INFO:journalctl@ceph.mds.e.smithi155.stdout:Jul 21 18:39:42 smithi155 podman[122575]: 2023-07-21T18:39:43.143 INFO:journalctl@ceph.mds.e.smithi155.stdout:Jul 21 18:39:42 smithi155 ceph-mds[122615]: -- 172.21.15.155:0/1706945693 <== mon.2 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5565a7ce51e0 con 0x5565a7cba800 2023-07-21T18:39:43.144 INFO:journalctl@ceph.mds.e.smithi155.stdout:Jul 21 18:39:42 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-e[122591]: starting mds.e at 2023-07-21T18:39:43.144 INFO:journalctl@ceph.mds.e.smithi155.stdout:Jul 21 18:39:42 smithi155 bash[122575]: 665fc908fce1f5516731a90edd016b280871a328eb2e64f20f31617490777dcb 2023-07-21T18:39:43.144 INFO:journalctl@ceph.mds.e.smithi155.stdout:Jul 21 18:39:42 smithi155 systemd[1]: Started Ceph mds.e for eaf8e25a-27f4-11ee-9b34-001a4aab830c. 2023-07-21T18:39:43.435 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T18:39:43.577 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:43.550+0000 7f647c305700 1 -- 172.21.15.39:0/1621458753 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64740ff340 msgr2=0x7f64740ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:43.577 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:43.550+0000 7f647c305700 1 --2- 172.21.15.39:0/1621458753 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64740ff340 0x7f64740ff790 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6470009990 tx=0x7f6470009ca0 comp rx=0 tx=0).stop 2023-07-21T18:39:43.577 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:43.550+0000 7f647c305700 1 -- 172.21.15.39:0/1621458753 shutdown_connections 2023-07-21T18:39:43.577 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:43.550+0000 7f647c305700 1 --2- 172.21.15.39:0/1621458753 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64740fe150 0x7f64740fe560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:43.577 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:43.550+0000 7f647c305700 1 --2- 172.21.15.39:0/1621458753 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f64740ffcd0 0x7f6474106590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:43.578 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:43.550+0000 7f647c305700 1 --2- 172.21.15.39:0/1621458753 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64740ff340 0x7f64740ff790 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:43.578 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:43.550+0000 7f647c305700 1 -- 172.21.15.39:0/1621458753 >> 172.21.15.39:0/1621458753 conn(0x7f64740f9770 msgr2=0x7f64740fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:39:43.578 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:43.550+0000 7f647c305700 1 -- 172.21.15.39:0/1621458753 shutdown_connections 2023-07-21T18:39:43.578 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:43.550+0000 7f647c305700 1 -- 172.21.15.39:0/1621458753 wait complete. 2023-07-21T18:39:43.580 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:43.551+0000 7f647c305700 1 Processor -- start 2023-07-21T18:39:43.580 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:43.551+0000 7f647c305700 1 -- start start 2023-07-21T18:39:43.580 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:43.551+0000 7f647c305700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64740fe150 0x7f6474195770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:43.580 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:43.551+0000 7f647c305700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f64740ff340 0x7f6474195cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:43.580 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:43.551+0000 7f647c305700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64740ffcd0 0x7f647419cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:43.580 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:43.551+0000 7f647c305700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f647419d2f0 con 0x7f64740ffcd0 2023-07-21T18:39:43.581 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:43.551+0000 7f647c305700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f647419d430 con 0x7f64740fe150 2023-07-21T18:39:43.581 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:43.551+0000 7f647c305700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f647419d5c0 con 0x7f64740ff340 2023-07-21T18:39:43.581 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:43.551+0000 7f64798a0700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f64740ff340 0x7f6474195cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:43.581 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:43.551+0000 7f647a8a2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64740ffcd0 0x7f647419cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:43.581 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:43.551+0000 7f64798a0700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f64740ff340 0x7f6474195cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:45112/0 (socket says 172.21.15.39:45112) 2023-07-21T18:39:43.581 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:43.551+0000 7f64798a0700 1 -- 172.21.15.39:0/2046315407 learned_addr learned my addr 172.21.15.39:0/2046315407 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:39:43.581 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:43.551+0000 7f647a8a2700 1 --2- 172.21.15.39:0/2046315407 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64740ffcd0 0x7f647419cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54796/0 (socket says 172.21.15.39:54796) 2023-07-21T18:39:43.582 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:43.551+0000 7f647a0a1700 1 --2- 172.21.15.39:0/2046315407 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64740fe150 0x7f6474195770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:43.582 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:43.551+0000 7f64798a0700 1 -- 172.21.15.39:0/2046315407 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64740fe150 msgr2=0x7f6474195770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:43.582 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:43.551+0000 7f64798a0700 1 --2- 172.21.15.39:0/2046315407 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64740fe150 0x7f6474195770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:43.582 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:43.551+0000 7f64798a0700 1 -- 172.21.15.39:0/2046315407 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64740ffcd0 msgr2=0x7f647419cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:43.582 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:43.551+0000 7f64798a0700 1 --2- 172.21.15.39:0/2046315407 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64740ffcd0 0x7f647419cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:43.582 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:43.551+0000 7f64798a0700 1 -- 172.21.15.39:0/2046315407 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64700095f0 con 0x7f64740ff340 2023-07-21T18:39:43.582 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:43.551+0000 7f647a8a2700 1 --2- 172.21.15.39:0/2046315407 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64740ffcd0 0x7f647419cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:39:43.582 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:43.552+0000 7f64798a0700 1 --2- 172.21.15.39:0/2046315407 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f64740ff340 0x7f6474195cb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f647000b620 tx=0x7f6470004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:39:43.583 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:43.552+0000 7f646b7fe700 1 -- 172.21.15.39:0/2046315407 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647001d070 con 0x7f64740ff340 2023-07-21T18:39:43.583 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:43.553+0000 7f646b7fe700 1 -- 172.21.15.39:0/2046315407 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6470005030 con 0x7f64740ff340 2023-07-21T18:39:43.583 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:43.553+0000 7f647c305700 1 -- 172.21.15.39:0/2046315407 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f647419d840 con 0x7f64740ff340 2023-07-21T18:39:43.583 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:43.553+0000 7f646b7fe700 1 -- 172.21.15.39:0/2046315407 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6470004b80 con 0x7f64740ff340 2023-07-21T18:39:43.583 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:43.553+0000 7f647c305700 1 -- 172.21.15.39:0/2046315407 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f647419dcc0 con 0x7f64740ff340 2023-07-21T18:39:43.583 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:43.553+0000 7f647c305700 1 -- 172.21.15.39:0/2046315407 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6474105690 con 0x7f64740ff340 2023-07-21T18:39:43.584 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:43.555+0000 7f646b7fe700 1 -- 172.21.15.39:0/2046315407 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f64700051a0 con 0x7f64740ff340 2023-07-21T18:39:43.584 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:43.555+0000 7f646b7fe700 1 --2- 172.21.15.39:0/2046315407 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f64600a1220 0x7f64600a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:43.585 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:43.556+0000 7f647a0a1700 1 --2- 172.21.15.39:0/2046315407 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f64600a1220 0x7f64600a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:43.585 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:43.556+0000 7f646b7fe700 1 -- 172.21.15.39:0/2046315407 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f64700cd7f0 con 0x7f64740ff340 2023-07-21T18:39:43.585 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:43.556+0000 7f647a0a1700 1 --2- 172.21.15.39:0/2046315407 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f64600a1220 0x7f64600a36d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f64740ff1a0 tx=0x7f6464010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:39:43.586 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:43.559+0000 7f646b7fe700 1 -- 172.21.15.39:0/2046315407 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f64700973d0 con 0x7f64740ff340 2023-07-21T18:39:43.829 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:43.803+0000 7f647c305700 1 -- 172.21.15.39:0/2046315407 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f647402c850 con 0x7f64740ff340 2023-07-21T18:39:43.830 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:43.804+0000 7f646b7fe700 1 -- 172.21.15.39:0/2046315407 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4291 (secure 0 0 0) 0x7f6470004ce0 con 0x7f64740ff340 2023-07-21T18:39:43.831 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:39:43.831 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/147318681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":147318681},{"type":"v1","addr":"172.21.15.39:6835","nonce":147318681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/1062891760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1062891760},{"type":"v1","addr":"172.21.15.39:6837","nonce":1062891760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/3106166804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3106166804},{"type":"v1","addr":"172.21.15.193:6833","nonce":3106166804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1034755277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1034755277},{"type":"v1","addr":"172.21.15.155:6833","nonce":1034755277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3854856286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3854856286},{"type":"v1","addr":"172.21.15.155:6835","nonce":3854856286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[]} 2023-07-21T18:39:43.833 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:43.806+0000 7f647c305700 1 -- 172.21.15.39:0/2046315407 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f64600a1220 msgr2=0x7f64600a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:43.833 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:43.806+0000 7f647c305700 1 --2- 172.21.15.39:0/2046315407 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f64600a1220 0x7f64600a36d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f64740ff1a0 tx=0x7f6464010040 comp rx=0 tx=0).stop 2023-07-21T18:39:43.834 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:43.806+0000 7f647c305700 1 -- 172.21.15.39:0/2046315407 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f64740ff340 msgr2=0x7f6474195cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:43.834 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:43.806+0000 7f647c305700 1 --2- 172.21.15.39:0/2046315407 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f64740ff340 0x7f6474195cb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f647000b620 tx=0x7f6470004730 comp rx=0 tx=0).stop 2023-07-21T18:39:43.834 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:43.806+0000 7f647c305700 1 -- 172.21.15.39:0/2046315407 shutdown_connections 2023-07-21T18:39:43.834 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:43.806+0000 7f647c305700 1 --2- 172.21.15.39:0/2046315407 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f64600a1220 0x7f64600a36d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:43.835 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:43.806+0000 7f647c305700 1 --2- 172.21.15.39:0/2046315407 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64740fe150 0x7f6474195770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:43.835 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:43.806+0000 7f647c305700 1 --2- 172.21.15.39:0/2046315407 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64740ffcd0 0x7f647419cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:43.835 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:43.806+0000 7f647c305700 1 --2- 172.21.15.39:0/2046315407 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f64740ff340 0x7f6474195cb0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:43.835 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:43.806+0000 7f647c305700 1 -- 172.21.15.39:0/2046315407 >> 172.21.15.39:0/2046315407 conn(0x7f64740f9770 msgr2=0x7f64741027d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:39:43.835 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:43.806+0000 7f647c305700 1 -- 172.21.15.39:0/2046315407 shutdown_connections 2023-07-21T18:39:43.835 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:43.806+0000 7f647c305700 1 -- 172.21.15.39:0/2046315407 wait complete. 2023-07-21T18:39:43.836 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 6 2023-07-21T18:39:44.152 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:43 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:44.152 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:43 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:44.152 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:43 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:44.152 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:43 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T18:39:44.153 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:43 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T18:39:44.153 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:43 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:39:44.153 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:43 smithi193 ceph-mon[98204]: Deploying daemon mds.f on smithi193 2023-07-21T18:39:44.153 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:43 smithi193 ceph-mon[98204]: mds.? [v2:172.21.15.155:6834/3854856286,v1:172.21.15.155:6835/3854856286] up:boot 2023-07-21T18:39:44.153 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:43 smithi193 ceph-mon[98204]: fsmap 5 up:standby 2023-07-21T18:39:44.153 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:43 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-21T18:39:44.154 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:43 smithi193 ceph-mon[98204]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:39:44.154 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:43 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/2046315407' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:39:44.192 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:43 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:44.192 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:43 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:44.193 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:43 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:44.193 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:43 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T18:39:44.193 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:43 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T18:39:44.193 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:43 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:39:44.193 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:43 smithi039 ceph-mon[96489]: Deploying daemon mds.f on smithi193 2023-07-21T18:39:44.193 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:43 smithi039 ceph-mon[96489]: mds.? [v2:172.21.15.155:6834/3854856286,v1:172.21.15.155:6835/3854856286] up:boot 2023-07-21T18:39:44.193 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:43 smithi039 ceph-mon[96489]: fsmap 5 up:standby 2023-07-21T18:39:44.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:43 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-21T18:39:44.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:43 smithi039 ceph-mon[96489]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:39:44.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:43 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/2046315407' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:39:44.393 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:43 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:44.393 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:43 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:44.394 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:43 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:44.394 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:43 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T18:39:44.394 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:43 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T18:39:44.394 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:43 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:39:44.394 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:43 smithi155 ceph-mon[97934]: Deploying daemon mds.f on smithi193 2023-07-21T18:39:44.394 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:43 smithi155 ceph-mon[97934]: mds.? [v2:172.21.15.155:6834/3854856286,v1:172.21.15.155:6835/3854856286] up:boot 2023-07-21T18:39:44.394 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:43 smithi155 ceph-mon[97934]: fsmap 5 up:standby 2023-07-21T18:39:44.395 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:43 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-21T18:39:44.395 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:43 smithi155 ceph-mon[97934]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:39:44.395 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:43 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/2046315407' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:39:44.624 INFO:journalctl@ceph.mds.f.smithi193.stdout:Jul 21 18:39:44 smithi193 systemd[1]: Starting Ceph mds.f for eaf8e25a-27f4-11ee-9b34-001a4aab830c... 2023-07-21T18:39:44.949 INFO:journalctl@ceph.mds.f.smithi193.stdout:Jul 21 18:39:44 smithi193 podman[123888]: 2023-07-21T18:39:44.949 INFO:journalctl@ceph.mds.f.smithi193.stdout:Jul 21 18:39:44 smithi193 ceph-mds[123929]: -- 172.21.15.193:0/3287620470 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562a410a91e0 con 0x562a4107f000 2023-07-21T18:39:44.950 INFO:journalctl@ceph.mds.f.smithi193.stdout:Jul 21 18:39:44 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-f[123905]: starting mds.f at 2023-07-21T18:39:45.374 INFO:journalctl@ceph.mds.f.smithi193.stdout:Jul 21 18:39:44 smithi193 bash[123888]: d7af23d66669a5954ff9694fcb711e7a3053540f9efb1533dabd3134c9fe101b 2023-07-21T18:39:45.375 INFO:journalctl@ceph.mds.f.smithi193.stdout:Jul 21 18:39:44 smithi193 systemd[1]: Started Ceph mds.f for eaf8e25a-27f4-11ee-9b34-001a4aab830c. 2023-07-21T18:39:45.845 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T18:39:45.986 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:45.960+0000 7fdd05141700 1 -- 172.21.15.39:0/1138651385 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd000fe9d0 msgr2=0x7fdd000fede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:45.987 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:45.960+0000 7fdd05141700 1 --2- 172.21.15.39:0/1138651385 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd000fe9d0 0x7fdd000fede0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fdce8009990 tx=0x7fdce8009ca0 comp rx=0 tx=0).stop 2023-07-21T18:39:45.987 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:45.960+0000 7fdd05141700 1 -- 172.21.15.39:0/1138651385 shutdown_connections 2023-07-21T18:39:45.987 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:45.960+0000 7fdd05141700 1 --2- 172.21.15.39:0/1138651385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd00100550 0x7fdd00106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:45.987 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:45.960+0000 7fdd05141700 1 --2- 172.21.15.39:0/1138651385 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd000fe9d0 0x7fdd000fede0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:45.987 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:45.960+0000 7fdd05141700 1 --2- 172.21.15.39:0/1138651385 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd000ffbc0 0x7fdd00100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:45.987 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:45.960+0000 7fdd05141700 1 -- 172.21.15.39:0/1138651385 >> 172.21.15.39:0/1138651385 conn(0x7fdd000f9fd0 msgr2=0x7fdd000fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:39:45.987 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:45.960+0000 7fdd05141700 1 -- 172.21.15.39:0/1138651385 shutdown_connections 2023-07-21T18:39:45.988 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:45.960+0000 7fdd05141700 1 -- 172.21.15.39:0/1138651385 wait complete. 2023-07-21T18:39:45.989 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:45.961+0000 7fdd05141700 1 Processor -- start 2023-07-21T18:39:45.989 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:45.961+0000 7fdd05141700 1 -- start start 2023-07-21T18:39:45.990 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:45.961+0000 7fdd05141700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd000fe9d0 0x7fdd00195780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:45.990 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:45.961+0000 7fdd05141700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd000ffbc0 0x7fdd00195cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:45.990 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:45.961+0000 7fdd05141700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd00100550 0x7fdd0019cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:45.990 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:45.961+0000 7fdd05141700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0019d300 con 0x7fdd00100550 2023-07-21T18:39:45.990 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:45.961+0000 7fdd05141700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0019d440 con 0x7fdd000fe9d0 2023-07-21T18:39:45.990 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:45.961+0000 7fdd05141700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0019d5d0 con 0x7fdd000ffbc0 2023-07-21T18:39:45.990 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:45.961+0000 7fdcff59e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd00100550 0x7fdd0019cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:45.991 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:45.961+0000 7fdcff59e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd00100550 0x7fdd0019cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54810/0 (socket says 172.21.15.39:54810) 2023-07-21T18:39:45.991 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:45.961+0000 7fdcff59e700 1 -- 172.21.15.39:0/3080174291 learned_addr learned my addr 172.21.15.39:0/3080174291 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:39:45.991 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:45.962+0000 7fdcfed9d700 1 --2- 172.21.15.39:0/3080174291 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd000fe9d0 0x7fdd00195780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:45.991 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:45.962+0000 7fdcfe59c700 1 --2- 172.21.15.39:0/3080174291 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd000ffbc0 0x7fdd00195cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:45.991 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:45.962+0000 7fdcff59e700 1 -- 172.21.15.39:0/3080174291 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd000ffbc0 msgr2=0x7fdd00195cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:45.991 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:45.962+0000 7fdcff59e700 1 --2- 172.21.15.39:0/3080174291 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd000ffbc0 0x7fdd00195cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:45.991 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:45.962+0000 7fdcff59e700 1 -- 172.21.15.39:0/3080174291 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd000fe9d0 msgr2=0x7fdd00195780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:45.991 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:45.962+0000 7fdcff59e700 1 --2- 172.21.15.39:0/3080174291 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd000fe9d0 0x7fdd00195780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:45.992 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:45.962+0000 7fdcff59e700 1 -- 172.21.15.39:0/3080174291 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdce80095f0 con 0x7fdd00100550 2023-07-21T18:39:45.992 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:45.962+0000 7fdcff59e700 1 --2- 172.21.15.39:0/3080174291 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd00100550 0x7fdd0019cd30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdcf8007ee0 tx=0x7fdcf800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:39:45.992 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:45.962+0000 7fdcf7fff700 1 -- 172.21.15.39:0/3080174291 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf8017070 con 0x7fdd00100550 2023-07-21T18:39:45.992 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:45.962+0000 7fdcf7fff700 1 -- 172.21.15.39:0/3080174291 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdcf800f040 con 0x7fdd00100550 2023-07-21T18:39:45.992 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:45.962+0000 7fdd05141700 1 -- 172.21.15.39:0/3080174291 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd0019d8b0 con 0x7fdd00100550 2023-07-21T18:39:45.992 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:45.962+0000 7fdcf7fff700 1 -- 172.21.15.39:0/3080174291 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf8005580 con 0x7fdd00100550 2023-07-21T18:39:45.992 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:45.962+0000 7fdd05141700 1 -- 172.21.15.39:0/3080174291 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd0019dd70 con 0x7fdd00100550 2023-07-21T18:39:45.993 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:45.963+0000 7fdd05141700 1 -- 172.21.15.39:0/3080174291 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd0010b320 con 0x7fdd00100550 2023-07-21T18:39:45.993 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:45.965+0000 7fdcf7fff700 1 -- 172.21.15.39:0/3080174291 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fdcf800db20 con 0x7fdd00100550 2023-07-21T18:39:45.993 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:45.965+0000 7fdcf7fff700 1 --2- 172.21.15.39:0/3080174291 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fdcec0a11a0 0x7fdcec0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:45.994 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:45.966+0000 7fdcfed9d700 1 --2- 172.21.15.39:0/3080174291 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fdcec0a11a0 0x7fdcec0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:45.994 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:45.966+0000 7fdcf7fff700 1 -- 172.21.15.39:0/3080174291 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fdcf80cc250 con 0x7fdd00100550 2023-07-21T18:39:45.994 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:45.966+0000 7fdcfed9d700 1 --2- 172.21.15.39:0/3080174291 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fdcec0a11a0 0x7fdcec0a3650 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdce800b5c0 tx=0x7fdce8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:39:45.994 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:45.968+0000 7fdcf7fff700 1 -- 172.21.15.39:0/3080174291 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fdcf8095e40 con 0x7fdd00100550 2023-07-21T18:39:46.192 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:45 smithi039 ceph-mon[96489]: mds.? [v2:172.21.15.193:6834/381560657,v1:172.21.15.193:6835/381560657] up:boot 2023-07-21T18:39:46.192 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:45 smithi039 ceph-mon[96489]: fsmap 6 up:standby 2023-07-21T18:39:46.192 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:45 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-21T18:39:46.192 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:45 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:46.193 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:45 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:46.193 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:45 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:46.193 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:45 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T18:39:46.193 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:45 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T18:39:46.193 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:45 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:39:46.193 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:45 smithi039 ceph-mon[96489]: Deploying daemon mds.c on smithi193 2023-07-21T18:39:46.193 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:45 smithi039 ceph-mon[96489]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:39:46.238 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:46.212+0000 7fdd05141700 1 -- 172.21.15.39:0/3080174291 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdd00105810 con 0x7fdd00100550 2023-07-21T18:39:46.239 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:46.212+0000 7fdcf7fff700 1 -- 172.21.15.39:0/3080174291 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5048 (secure 0 0 0) 0x7fdcf8099460 con 0x7fdd00100550 2023-07-21T18:39:46.239 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:39:46.239 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/147318681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":147318681},{"type":"v1","addr":"172.21.15.39:6835","nonce":147318681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/1062891760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1062891760},{"type":"v1","addr":"172.21.15.39:6837","nonce":1062891760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/3106166804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3106166804},{"type":"v1","addr":"172.21.15.193:6833","nonce":3106166804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1034755277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1034755277},{"type":"v1","addr":"172.21.15.155:6833","nonce":1034755277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/381560657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":381560657},{"type":"v1","addr":"172.21.15.193:6835","nonce":381560657}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3854856286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3854856286},{"type":"v1","addr":"172.21.15.155:6835","nonce":3854856286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[]} 2023-07-21T18:39:46.242 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:46.215+0000 7fdd05141700 1 -- 172.21.15.39:0/3080174291 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fdcec0a11a0 msgr2=0x7fdcec0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:46.242 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:46.215+0000 7fdd05141700 1 --2- 172.21.15.39:0/3080174291 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fdcec0a11a0 0x7fdcec0a3650 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdce800b5c0 tx=0x7fdce8005960 comp rx=0 tx=0).stop 2023-07-21T18:39:46.242 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:46.215+0000 7fdd05141700 1 -- 172.21.15.39:0/3080174291 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd00100550 msgr2=0x7fdd0019cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:46.242 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:46.215+0000 7fdd05141700 1 --2- 172.21.15.39:0/3080174291 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd00100550 0x7fdd0019cd30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdcf8007ee0 tx=0x7fdcf800d3b0 comp rx=0 tx=0).stop 2023-07-21T18:39:46.242 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:46.215+0000 7fdd05141700 1 -- 172.21.15.39:0/3080174291 shutdown_connections 2023-07-21T18:39:46.243 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:46.215+0000 7fdd05141700 1 --2- 172.21.15.39:0/3080174291 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fdcec0a11a0 0x7fdcec0a3650 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:46.243 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:46.215+0000 7fdd05141700 1 --2- 172.21.15.39:0/3080174291 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd000fe9d0 0x7fdd00195780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:46.243 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:46.215+0000 7fdd05141700 1 --2- 172.21.15.39:0/3080174291 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd00100550 0x7fdd0019cd30 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:46.243 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:46.215+0000 7fdd05141700 1 --2- 172.21.15.39:0/3080174291 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd000ffbc0 0x7fdd00195cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:46.243 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:46.215+0000 7fdd05141700 1 -- 172.21.15.39:0/3080174291 >> 172.21.15.39:0/3080174291 conn(0x7fdd000f9fd0 msgr2=0x7fdd000fc1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:39:46.243 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:46.215+0000 7fdd05141700 1 -- 172.21.15.39:0/3080174291 shutdown_connections 2023-07-21T18:39:46.243 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:46.215+0000 7fdd05141700 1 -- 172.21.15.39:0/3080174291 wait complete. 2023-07-21T18:39:46.244 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:45 smithi193 ceph-mon[98204]: mds.? [v2:172.21.15.193:6834/381560657,v1:172.21.15.193:6835/381560657] up:boot 2023-07-21T18:39:46.245 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:45 smithi193 ceph-mon[98204]: fsmap 6 up:standby 2023-07-21T18:39:46.245 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:45 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-21T18:39:46.245 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:45 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:46.245 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:45 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:46.246 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:45 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:46.246 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:45 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T18:39:46.246 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:45 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T18:39:46.247 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:45 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:39:46.247 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:45 smithi193 ceph-mon[98204]: Deploying daemon mds.c on smithi193 2023-07-21T18:39:46.247 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:45 smithi193 ceph-mon[98204]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:39:46.247 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 7 2023-07-21T18:39:46.393 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:45 smithi155 ceph-mon[97934]: mds.? [v2:172.21.15.193:6834/381560657,v1:172.21.15.193:6835/381560657] up:boot 2023-07-21T18:39:46.394 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:45 smithi155 ceph-mon[97934]: fsmap 6 up:standby 2023-07-21T18:39:46.394 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:45 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-21T18:39:46.394 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:45 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:46.395 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:45 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:46.395 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:45 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:46.395 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:45 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T18:39:46.396 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:45 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T18:39:46.396 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:45 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:39:46.396 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:45 smithi155 ceph-mon[97934]: Deploying daemon mds.c on smithi193 2023-07-21T18:39:46.396 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:45 smithi155 ceph-mon[97934]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:39:46.822 INFO:journalctl@ceph.mds.c.smithi193.stdout:Jul 21 18:39:46 smithi193 systemd[1]: Starting Ceph mds.c for eaf8e25a-27f4-11ee-9b34-001a4aab830c... 2023-07-21T18:39:47.073 INFO:journalctl@ceph.mds.c.smithi193.stdout:Jul 21 18:39:46 smithi193 podman[124236]: 2023-07-21T18:39:47.374 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:47 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/3080174291' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:39:47.375 INFO:journalctl@ceph.mds.c.smithi193.stdout:Jul 21 18:39:47 smithi193 ceph-mds[124276]: -- 172.21.15.193:0/3879335483 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e0de8eb1e0 con 0x55e0de8c1000 2023-07-21T18:39:47.375 INFO:journalctl@ceph.mds.c.smithi193.stdout:Jul 21 18:39:47 smithi193 bash[124236]: 24117b1517243b619df64db939148bbec987735735896f83734c2906cfa22d48 2023-07-21T18:39:47.375 INFO:journalctl@ceph.mds.c.smithi193.stdout:Jul 21 18:39:47 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-c[124252]: starting mds.c at 2023-07-21T18:39:47.375 INFO:journalctl@ceph.mds.c.smithi193.stdout:Jul 21 18:39:47 smithi193 systemd[1]: Started Ceph mds.c for eaf8e25a-27f4-11ee-9b34-001a4aab830c. 2023-07-21T18:39:47.393 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:47 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/3080174291' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:39:47.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:47 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/3080174291' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:39:48.143 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:48 smithi155 ceph-mon[97934]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:39:48.143 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:48 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:48.143 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:48 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:48.144 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:48 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:48.144 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:48 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T18:39:48.144 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:48 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T18:39:48.144 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:48 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:39:48.254 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T18:39:48.374 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:48 smithi193 ceph-mon[98204]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:39:48.374 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:48 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:48.375 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:48 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:48.375 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:48 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:48.375 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:48 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T18:39:48.375 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:48 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T18:39:48.375 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:48 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:39:48.395 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:48.367+0000 7f60d981d700 1 -- 172.21.15.39:0/508251540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f60d40ffbc0 msgr2=0x7f60d4100010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:48.395 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:48.367+0000 7f60d981d700 1 --2- 172.21.15.39:0/508251540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f60d40ffbc0 0x7f60d4100010 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f60c8009990 tx=0x7f60c8009ca0 comp rx=0 tx=0).stop 2023-07-21T18:39:48.396 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:48.367+0000 7f60d981d700 1 -- 172.21.15.39:0/508251540 shutdown_connections 2023-07-21T18:39:48.396 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:48.367+0000 7f60d981d700 1 --2- 172.21.15.39:0/508251540 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60d40fe9d0 0x7f60d40fede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:48.396 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:48.367+0000 7f60d981d700 1 --2- 172.21.15.39:0/508251540 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60d4100550 0x7f60d4106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:48.396 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:48.367+0000 7f60d981d700 1 --2- 172.21.15.39:0/508251540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f60d40ffbc0 0x7f60d4100010 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:48.396 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:48.367+0000 7f60d981d700 1 -- 172.21.15.39:0/508251540 >> 172.21.15.39:0/508251540 conn(0x7f60d40f9fd0 msgr2=0x7f60d40fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:39:48.396 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:48.367+0000 7f60d981d700 1 -- 172.21.15.39:0/508251540 shutdown_connections 2023-07-21T18:39:48.396 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:48.368+0000 7f60d981d700 1 -- 172.21.15.39:0/508251540 wait complete. 2023-07-21T18:39:48.397 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:48.368+0000 7f60d981d700 1 Processor -- start 2023-07-21T18:39:48.398 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:48.368+0000 7f60d981d700 1 -- start start 2023-07-21T18:39:48.399 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:48.368+0000 7f60d981d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60d40fe9d0 0x7f60d41946a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:48.399 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:48.368+0000 7f60d981d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60d40ffbc0 0x7f60d4194be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:48.399 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:48.369+0000 7f60d981d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f60d4100550 0x7f60d419bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:48.399 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:48.369+0000 7f60d981d700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d419c220 con 0x7f60d4100550 2023-07-21T18:39:48.399 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:48.369+0000 7f60d981d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d419c360 con 0x7f60d40ffbc0 2023-07-21T18:39:48.399 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:48.369+0000 7f60d981d700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d419c4f0 con 0x7f60d40fe9d0 2023-07-21T18:39:48.399 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:48.369+0000 7f60d37fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f60d4100550 0x7f60d419bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:48.400 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:48.369+0000 7f60d37fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f60d4100550 0x7f60d419bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54824/0 (socket says 172.21.15.39:54824) 2023-07-21T18:39:48.400 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:48.369+0000 7f60d37fe700 1 -- 172.21.15.39:0/2510999777 learned_addr learned my addr 172.21.15.39:0/2510999777 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:39:48.400 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:48.369+0000 7f60d2ffd700 1 --2- 172.21.15.39:0/2510999777 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60d40fe9d0 0x7f60d41946a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:48.400 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:48.369+0000 7f60d27fc700 1 --2- 172.21.15.39:0/2510999777 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60d40ffbc0 0x7f60d4194be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:48.400 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:48.369+0000 7f60d37fe700 1 -- 172.21.15.39:0/2510999777 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60d40fe9d0 msgr2=0x7f60d41946a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:48.400 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:48.369+0000 7f60d37fe700 1 --2- 172.21.15.39:0/2510999777 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60d40fe9d0 0x7f60d41946a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:48.400 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:48.369+0000 7f60d37fe700 1 -- 172.21.15.39:0/2510999777 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60d40ffbc0 msgr2=0x7f60d4194be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:48.400 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:48.369+0000 7f60d37fe700 1 --2- 172.21.15.39:0/2510999777 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60d40ffbc0 0x7f60d4194be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:48.401 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:48.369+0000 7f60d37fe700 1 -- 172.21.15.39:0/2510999777 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60c80095f0 con 0x7f60d4100550 2023-07-21T18:39:48.401 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:48.369+0000 7f60d37fe700 1 --2- 172.21.15.39:0/2510999777 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f60d4100550 0x7f60d419bc50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f60cc007f10 tx=0x7f60cc00f5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:39:48.401 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:48.369+0000 7f60d881b700 1 -- 172.21.15.39:0/2510999777 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60cc00fd30 con 0x7f60d4100550 2023-07-21T18:39:48.401 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:48.369+0000 7f60d981d700 1 -- 172.21.15.39:0/2510999777 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60d4068990 con 0x7f60d4100550 2023-07-21T18:39:48.401 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:48.369+0000 7f60d881b700 1 -- 172.21.15.39:0/2510999777 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60cc00fe90 con 0x7f60d4100550 2023-07-21T18:39:48.401 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:48.369+0000 7f60d881b700 1 -- 172.21.15.39:0/2510999777 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60cc005020 con 0x7f60d4100550 2023-07-21T18:39:48.401 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:48.369+0000 7f60d981d700 1 -- 172.21.15.39:0/2510999777 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60d4068e50 con 0x7f60d4100550 2023-07-21T18:39:48.402 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:48.370+0000 7f60d981d700 1 -- 172.21.15.39:0/2510999777 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60d4067750 con 0x7f60d4100550 2023-07-21T18:39:48.402 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:48.371+0000 7f60d881b700 1 -- 172.21.15.39:0/2510999777 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f60cc012460 con 0x7f60d4100550 2023-07-21T18:39:48.402 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:48.372+0000 7f60d881b700 1 --2- 172.21.15.39:0/2510999777 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f60bc0a11a0 0x7f60bc0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:48.403 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:48.372+0000 7f60d2ffd700 1 --2- 172.21.15.39:0/2510999777 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f60bc0a11a0 0x7f60bc0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:48.403 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:48.372+0000 7f60d881b700 1 -- 172.21.15.39:0/2510999777 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f60cc0cea20 con 0x7f60d4100550 2023-07-21T18:39:48.403 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:48.373+0000 7f60d2ffd700 1 --2- 172.21.15.39:0/2510999777 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f60bc0a11a0 0x7f60bc0a3650 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f60d40ffa20 tx=0x7f60c4007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:39:48.403 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:48.375+0000 7f60d881b700 1 -- 172.21.15.39:0/2510999777 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f60cc098610 con 0x7f60d4100550 2023-07-21T18:39:48.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:48 smithi039 ceph-mon[96489]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:39:48.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:48 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:48.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:48 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:48.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:48 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:48.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:48 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T18:39:48.443 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:48 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T18:39:48.443 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:48 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:39:48.648 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:48.621+0000 7f60d981d700 1 -- 172.21.15.39:0/2510999777 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f60d41058f0 con 0x7f60d4100550 2023-07-21T18:39:48.649 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:48.622+0000 7f60d881b700 1 -- 172.21.15.39:0/2510999777 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+5808 (secure 0 0 0) 0x7f60cc09bc30 con 0x7f60d4100550 2023-07-21T18:39:48.649 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:39:48.650 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/147318681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":147318681},{"type":"v1","addr":"172.21.15.39:6835","nonce":147318681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/1062891760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1062891760},{"type":"v1","addr":"172.21.15.39:6837","nonce":1062891760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/3106166804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3106166804},{"type":"v1","addr":"172.21.15.193:6833","nonce":3106166804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1034755277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1034755277},{"type":"v1","addr":"172.21.15.155:6833","nonce":1034755277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/381560657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":381560657},{"type":"v1","addr":"172.21.15.193:6835","nonce":381560657}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/2626761843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2626761843},{"type":"v1","addr":"172.21.15.193:6837","nonce":2626761843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3854856286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3854856286},{"type":"v1","addr":"172.21.15.155:6835","nonce":3854856286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[]} 2023-07-21T18:39:48.651 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:48.624+0000 7f60d981d700 1 -- 172.21.15.39:0/2510999777 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f60bc0a11a0 msgr2=0x7f60bc0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:48.651 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:48.624+0000 7f60d981d700 1 --2- 172.21.15.39:0/2510999777 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f60bc0a11a0 0x7f60bc0a3650 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f60d40ffa20 tx=0x7f60c4007400 comp rx=0 tx=0).stop 2023-07-21T18:39:48.652 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:48.624+0000 7f60d981d700 1 -- 172.21.15.39:0/2510999777 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f60d4100550 msgr2=0x7f60d419bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:48.652 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:48.624+0000 7f60d981d700 1 --2- 172.21.15.39:0/2510999777 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f60d4100550 0x7f60d419bc50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f60cc007f10 tx=0x7f60cc00f5c0 comp rx=0 tx=0).stop 2023-07-21T18:39:48.652 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:48.624+0000 7f60d981d700 1 -- 172.21.15.39:0/2510999777 shutdown_connections 2023-07-21T18:39:48.652 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:48.624+0000 7f60d981d700 1 --2- 172.21.15.39:0/2510999777 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f60bc0a11a0 0x7f60bc0a3650 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:48.652 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:48.624+0000 7f60d981d700 1 --2- 172.21.15.39:0/2510999777 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f60d4100550 0x7f60d419bc50 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:48.652 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:48.624+0000 7f60d981d700 1 --2- 172.21.15.39:0/2510999777 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60d40fe9d0 0x7f60d41946a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:48.653 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:48.624+0000 7f60d981d700 1 --2- 172.21.15.39:0/2510999777 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60d40ffbc0 0x7f60d4194be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:48.653 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:48.624+0000 7f60d981d700 1 -- 172.21.15.39:0/2510999777 >> 172.21.15.39:0/2510999777 conn(0x7f60d40f9fd0 msgr2=0x7f60d4103050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:39:48.653 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:48.624+0000 7f60d981d700 1 -- 172.21.15.39:0/2510999777 shutdown_connections 2023-07-21T18:39:48.653 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:48.624+0000 7f60d981d700 1 -- 172.21.15.39:0/2510999777 wait complete. 2023-07-21T18:39:48.653 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 8 2023-07-21T18:39:48.730 INFO:journalctl@ceph.mds.h.smithi155.stdout:Jul 21 18:39:48 smithi155 systemd[1]: Starting Ceph mds.h for eaf8e25a-27f4-11ee-9b34-001a4aab830c... 2023-07-21T18:39:49.374 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:49 smithi193 ceph-mon[98204]: Deploying daemon mds.h on smithi155 2023-07-21T18:39:49.374 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:49 smithi193 ceph-mon[98204]: mds.? [v2:172.21.15.193:6836/2626761843,v1:172.21.15.193:6837/2626761843] up:boot 2023-07-21T18:39:49.375 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:49 smithi193 ceph-mon[98204]: fsmap 7 up:standby 2023-07-21T18:39:49.375 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:49 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-21T18:39:49.375 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:49 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/2510999777' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:39:49.393 INFO:journalctl@ceph.mds.h.smithi155.stdout:Jul 21 18:39:49 smithi155 podman[122915]: 2023-07-21T18:39:49.393 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:49 smithi155 ceph-mon[97934]: Deploying daemon mds.h on smithi155 2023-07-21T18:39:49.393 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:49 smithi155 ceph-mon[97934]: mds.? [v2:172.21.15.193:6836/2626761843,v1:172.21.15.193:6837/2626761843] up:boot 2023-07-21T18:39:49.393 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:49 smithi155 ceph-mon[97934]: fsmap 7 up:standby 2023-07-21T18:39:49.393 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:49 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-21T18:39:49.393 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:49 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/2510999777' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:39:49.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:49 smithi039 ceph-mon[96489]: Deploying daemon mds.h on smithi155 2023-07-21T18:39:49.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:49 smithi039 ceph-mon[96489]: mds.? [v2:172.21.15.193:6836/2626761843,v1:172.21.15.193:6837/2626761843] up:boot 2023-07-21T18:39:49.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:49 smithi039 ceph-mon[96489]: fsmap 7 up:standby 2023-07-21T18:39:49.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:49 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-21T18:39:49.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:49 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/2510999777' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:39:49.893 INFO:journalctl@ceph.mds.h.smithi155.stdout:Jul 21 18:39:49 smithi155 ceph-mds[122955]: -- 172.21.15.155:0/1579375663 <== mon.2 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562376c051e0 con 0x562376bdb000 2023-07-21T18:39:49.893 INFO:journalctl@ceph.mds.h.smithi155.stdout:Jul 21 18:39:49 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-h[122931]: starting mds.h at 2023-07-21T18:39:49.893 INFO:journalctl@ceph.mds.h.smithi155.stdout:Jul 21 18:39:49 smithi155 bash[122915]: a4a7b89025235e9e7f0814ecd9ee47881146e654574d539afd170ae9dd3126ba 2023-07-21T18:39:49.893 INFO:journalctl@ceph.mds.h.smithi155.stdout:Jul 21 18:39:49 smithi155 systemd[1]: Started Ceph mds.h for eaf8e25a-27f4-11ee-9b34-001a4aab830c. 2023-07-21T18:39:50.343 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:50 smithi193 ceph-mon[98204]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:39:50.344 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:50 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:50.344 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:50 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:50.344 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:50 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:50.344 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:50 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T18:39:50.344 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:50 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T18:39:50.345 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:50 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:39:50.393 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:50 smithi155 ceph-mon[97934]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:39:50.393 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:50 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:50.393 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:50 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:50.394 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:50 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:50.394 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:50 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T18:39:50.394 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:50 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T18:39:50.394 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:50 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:39:50.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:50 smithi039 ceph-mon[96489]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:39:50.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:50 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:50.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:50 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:50.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:50 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:50.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:50 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T18:39:50.443 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:50 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T18:39:50.443 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:50 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:39:50.664 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T18:39:50.805 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:50.777+0000 7f858b082700 1 -- 172.21.15.39:0/365801949 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8584105790 msgr2=0x7f8584107bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:50.805 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:50.777+0000 7f858b082700 1 --2- 172.21.15.39:0/365801949 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8584105790 0x7f8584107bf0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f857c00b0a0 tx=0x7f857c00b3b0 comp rx=0 tx=0).stop 2023-07-21T18:39:50.805 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:50.777+0000 7f858b082700 1 -- 172.21.15.39:0/365801949 shutdown_connections 2023-07-21T18:39:50.805 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:50.777+0000 7f858b082700 1 --2- 172.21.15.39:0/365801949 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8584105790 0x7f8584107bf0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:50.805 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:50.777+0000 7f858b082700 1 --2- 172.21.15.39:0/365801949 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8584100550 0x7f8584102930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:50.806 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:50.777+0000 7f858b082700 1 --2- 172.21.15.39:0/365801949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8584102e70 0x7f8584105250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:50.806 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:50.777+0000 7f858b082700 1 -- 172.21.15.39:0/365801949 >> 172.21.15.39:0/365801949 conn(0x7f85840f9fd0 msgr2=0x7f85840fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:39:50.806 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:50.777+0000 7f858b082700 1 -- 172.21.15.39:0/365801949 shutdown_connections 2023-07-21T18:39:50.806 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:50.777+0000 7f858b082700 1 -- 172.21.15.39:0/365801949 wait complete. 2023-07-21T18:39:50.808 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:50.778+0000 7f858b082700 1 Processor -- start 2023-07-21T18:39:50.808 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:50.778+0000 7f858b082700 1 -- start start 2023-07-21T18:39:50.808 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:50.778+0000 7f858b082700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8584100550 0x7f8584197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:50.808 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:50.778+0000 7f858b082700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8584102e70 0x7f8584197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:50.809 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:50.778+0000 7f858b082700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8584105790 0x7f858419f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:50.809 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:50.778+0000 7f858b082700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f858419f620 con 0x7f8584100550 2023-07-21T18:39:50.809 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:50.778+0000 7f858b082700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f858419f760 con 0x7f8584102e70 2023-07-21T18:39:50.809 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:50.778+0000 7f858b082700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f858419f8f0 con 0x7f8584105790 2023-07-21T18:39:50.809 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:50.778+0000 7f8588e1e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8584100550 0x7f8584197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:50.809 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:50.778+0000 7f8588e1e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8584100550 0x7f8584197aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54834/0 (socket says 172.21.15.39:54834) 2023-07-21T18:39:50.809 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:50.778+0000 7f8588e1e700 1 -- 172.21.15.39:0/2147664424 learned_addr learned my addr 172.21.15.39:0/2147664424 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:39:50.810 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:50.778+0000 7f8583fff700 1 --2- 172.21.15.39:0/2147664424 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8584102e70 0x7f8584197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:50.810 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:50.779+0000 7f858961f700 1 --2- 172.21.15.39:0/2147664424 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8584105790 0x7f858419f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:50.810 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:50.779+0000 7f8588e1e700 1 -- 172.21.15.39:0/2147664424 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8584105790 msgr2=0x7f858419f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:50.810 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:50.779+0000 7f8588e1e700 1 --2- 172.21.15.39:0/2147664424 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8584105790 0x7f858419f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:50.810 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:50.779+0000 7f8588e1e700 1 -- 172.21.15.39:0/2147664424 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8584102e70 msgr2=0x7f8584197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:50.810 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:50.779+0000 7f8588e1e700 1 --2- 172.21.15.39:0/2147664424 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8584102e70 0x7f8584197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:50.810 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:50.779+0000 7f8588e1e700 1 -- 172.21.15.39:0/2147664424 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f857c009d00 con 0x7f8584100550 2023-07-21T18:39:50.810 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:50.779+0000 7f8583fff700 1 --2- 172.21.15.39:0/2147664424 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8584102e70 0x7f8584197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:39:50.811 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:50.779+0000 7f8588e1e700 1 --2- 172.21.15.39:0/2147664424 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8584100550 0x7f8584197aa0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f857800d950 tx=0x7f857800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:39:50.811 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:50.779+0000 7f8581ffb700 1 -- 172.21.15.39:0/2147664424 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8578009b40 con 0x7f8584100550 2023-07-21T18:39:50.811 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:50.779+0000 7f858b082700 1 -- 172.21.15.39:0/2147664424 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f858419fbd0 con 0x7f8584100550 2023-07-21T18:39:50.811 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:50.779+0000 7f8581ffb700 1 -- 172.21.15.39:0/2147664424 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8578009ca0 con 0x7f8584100550 2023-07-21T18:39:50.811 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:50.779+0000 7f8581ffb700 1 -- 172.21.15.39:0/2147664424 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f857800f620 con 0x7f8584100550 2023-07-21T18:39:50.811 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:50.779+0000 7f858b082700 1 -- 172.21.15.39:0/2147664424 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85841a0130 con 0x7f8584100550 2023-07-21T18:39:50.811 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:50.780+0000 7f858b082700 1 -- 172.21.15.39:0/2147664424 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85840fbb40 con 0x7f8584100550 2023-07-21T18:39:50.812 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:50.781+0000 7f8581ffb700 1 -- 172.21.15.39:0/2147664424 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f8578010460 con 0x7f8584100550 2023-07-21T18:39:50.812 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:50.782+0000 7f8581ffb700 1 --2- 172.21.15.39:0/2147664424 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f85740a11a0 0x7f85740a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:50.812 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:50.782+0000 7f8583fff700 1 --2- 172.21.15.39:0/2147664424 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f85740a11a0 0x7f85740a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:50.813 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:50.782+0000 7f8581ffb700 1 -- 172.21.15.39:0/2147664424 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f85780cc890 con 0x7f8584100550 2023-07-21T18:39:50.813 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:50.782+0000 7f8583fff700 1 --2- 172.21.15.39:0/2147664424 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f85740a11a0 0x7f85740a3650 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8570005950 tx=0x7f85700058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:39:50.813 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:50.786+0000 7f8581ffb700 1 -- 172.21.15.39:0/2147664424 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f85780963e0 con 0x7f8584100550 2023-07-21T18:39:51.059 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:51.032+0000 7f858b082700 1 -- 172.21.15.39:0/2147664424 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f858402c850 con 0x7f8584100550 2023-07-21T18:39:51.060 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:51.032+0000 7f8581ffb700 1 -- 172.21.15.39:0/2147664424 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6565 (secure 0 0 0) 0x7f8578099a00 con 0x7f8584100550 2023-07-21T18:39:51.060 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:39:51.060 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/147318681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":147318681},{"type":"v1","addr":"172.21.15.39:6835","nonce":147318681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/1062891760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1062891760},{"type":"v1","addr":"172.21.15.39:6837","nonce":1062891760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/3106166804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3106166804},{"type":"v1","addr":"172.21.15.193:6833","nonce":3106166804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1034755277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1034755277},{"type":"v1","addr":"172.21.15.155:6833","nonce":1034755277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/381560657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":381560657},{"type":"v1","addr":"172.21.15.193:6835","nonce":381560657}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/2626761843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2626761843},{"type":"v1","addr":"172.21.15.193:6837","nonce":2626761843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3854856286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3854856286},{"type":"v1","addr":"172.21.15.155:6835","nonce":3854856286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24500,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6837/947670056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":947670056},{"type":"v1","addr":"172.21.15.155:6837","nonce":947670056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[]} 2023-07-21T18:39:51.062 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:51.035+0000 7f858b082700 1 -- 172.21.15.39:0/2147664424 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f85740a11a0 msgr2=0x7f85740a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:51.062 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:51.035+0000 7f858b082700 1 --2- 172.21.15.39:0/2147664424 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f85740a11a0 0x7f85740a3650 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8570005950 tx=0x7f85700058e0 comp rx=0 tx=0).stop 2023-07-21T18:39:51.063 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:51.035+0000 7f858b082700 1 -- 172.21.15.39:0/2147664424 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8584100550 msgr2=0x7f8584197aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:51.063 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:51.035+0000 7f858b082700 1 --2- 172.21.15.39:0/2147664424 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8584100550 0x7f8584197aa0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f857800d950 tx=0x7f857800dd10 comp rx=0 tx=0).stop 2023-07-21T18:39:51.063 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:51.035+0000 7f858b082700 1 -- 172.21.15.39:0/2147664424 shutdown_connections 2023-07-21T18:39:51.063 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:51.035+0000 7f858b082700 1 --2- 172.21.15.39:0/2147664424 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f85740a11a0 0x7f85740a3650 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:51.063 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:51.035+0000 7f858b082700 1 --2- 172.21.15.39:0/2147664424 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8584105790 0x7f858419f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:51.063 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:51.035+0000 7f858b082700 1 --2- 172.21.15.39:0/2147664424 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8584100550 0x7f8584197aa0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:51.063 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:51.035+0000 7f858b082700 1 --2- 172.21.15.39:0/2147664424 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8584102e70 0x7f8584197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:51.063 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:51.035+0000 7f858b082700 1 -- 172.21.15.39:0/2147664424 >> 172.21.15.39:0/2147664424 conn(0x7f85840f9fd0 msgr2=0x7f85841011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:39:51.064 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:51.035+0000 7f858b082700 1 -- 172.21.15.39:0/2147664424 shutdown_connections 2023-07-21T18:39:51.064 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:51.035+0000 7f858b082700 1 -- 172.21.15.39:0/2147664424 wait complete. 2023-07-21T18:39:51.064 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 9 2023-07-21T18:39:51.124 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:51 smithi193 ceph-mon[98204]: Deploying daemon mds.i on smithi193 2023-07-21T18:39:51.125 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:51 smithi193 ceph-mon[98204]: mds.? [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] up:boot 2023-07-21T18:39:51.125 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:51 smithi193 ceph-mon[98204]: fsmap 8 up:standby 2023-07-21T18:39:51.125 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:51 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-21T18:39:51.125 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:51 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/2147664424' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:39:51.393 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:51 smithi155 ceph-mon[97934]: Deploying daemon mds.i on smithi193 2023-07-21T18:39:51.393 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:51 smithi155 ceph-mon[97934]: mds.? [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] up:boot 2023-07-21T18:39:51.393 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:51 smithi155 ceph-mon[97934]: fsmap 8 up:standby 2023-07-21T18:39:51.393 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:51 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-21T18:39:51.393 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:51 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/2147664424' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:39:51.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:51 smithi039 ceph-mon[96489]: Deploying daemon mds.i on smithi193 2023-07-21T18:39:51.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:51 smithi039 ceph-mon[96489]: mds.? [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] up:boot 2023-07-21T18:39:51.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:51 smithi039 ceph-mon[96489]: fsmap 8 up:standby 2023-07-21T18:39:51.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:51 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-21T18:39:51.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:51 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/2147664424' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:39:51.455 INFO:journalctl@ceph.mds.i.smithi193.stdout:Jul 21 18:39:51 smithi193 systemd[1]: Starting Ceph mds.i for eaf8e25a-27f4-11ee-9b34-001a4aab830c... 2023-07-21T18:39:51.706 INFO:journalctl@ceph.mds.i.smithi193.stdout:Jul 21 18:39:51 smithi193 podman[124572]: 2023-07-21T18:39:51.706 INFO:journalctl@ceph.mds.i.smithi193.stdout:Jul 21 18:39:51 smithi193 ceph-mds[124612]: -- 172.21.15.193:0/1556027645 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b5644bb1e0 con 0x55b564490c00 2023-07-21T18:39:51.706 INFO:journalctl@ceph.mds.i.smithi193.stdout:Jul 21 18:39:51 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-i[124588]: starting mds.i at 2023-07-21T18:39:52.112 INFO:journalctl@ceph.mds.i.smithi193.stdout:Jul 21 18:39:51 smithi193 bash[124572]: 226e6661067a3c6d79edaf4277b76920c4c67bd12d0928ebc39f14c7f5ef8f9b 2023-07-21T18:39:52.112 INFO:journalctl@ceph.mds.i.smithi193.stdout:Jul 21 18:39:51 smithi193 systemd[1]: Started Ceph mds.i for eaf8e25a-27f4-11ee-9b34-001a4aab830c. 2023-07-21T18:39:52.374 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:52 smithi193 ceph-mon[98204]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:39:52.374 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:52 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:52.374 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:52 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:52.374 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:52 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:52.375 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:52 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T18:39:52.375 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:52 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T18:39:52.375 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:52 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:39:52.393 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:52 smithi155 ceph-mon[97934]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:39:52.393 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:52 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:52.394 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:52 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:52.394 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:52 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:52.394 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:52 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T18:39:52.394 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:52 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T18:39:52.395 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:52 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:39:52.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:52 smithi039 ceph-mon[96489]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:39:52.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:52 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:52.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:52 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:52.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:52 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:52.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:52 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T18:39:52.443 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:52 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T18:39:52.443 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:52 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:39:53.076 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T18:39:53.281 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:53.254+0000 7f068d233700 1 -- 172.21.15.39:0/450566758 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0688102f30 msgr2=0x7f0688105310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:53.282 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:53.254+0000 7f068d233700 1 --2- 172.21.15.39:0/450566758 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0688102f30 0x7f0688105310 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f067c009990 tx=0x7f067c009ca0 comp rx=0 tx=0).stop 2023-07-21T18:39:53.282 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:53.254+0000 7f068d233700 1 -- 172.21.15.39:0/450566758 shutdown_connections 2023-07-21T18:39:53.282 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:53.254+0000 7f068d233700 1 --2- 172.21.15.39:0/450566758 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0688100610 0x7f06881029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:53.283 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:53.254+0000 7f068d233700 1 --2- 172.21.15.39:0/450566758 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0688105850 0x7f0688107c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:53.283 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:53.254+0000 7f068d233700 1 --2- 172.21.15.39:0/450566758 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0688102f30 0x7f0688105310 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:53.283 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:53.254+0000 7f068d233700 1 -- 172.21.15.39:0/450566758 >> 172.21.15.39:0/450566758 conn(0x7f06880f9fd0 msgr2=0x7f06880fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:39:53.284 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:53.254+0000 7f068d233700 1 -- 172.21.15.39:0/450566758 shutdown_connections 2023-07-21T18:39:53.284 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:53.254+0000 7f068d233700 1 -- 172.21.15.39:0/450566758 wait complete. 2023-07-21T18:39:53.287 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:53.254+0000 7f068d233700 1 Processor -- start 2023-07-21T18:39:53.288 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:53.255+0000 7f068d233700 1 -- start start 2023-07-21T18:39:53.288 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:53.255+0000 7f068d233700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0688100610 0x7f0688195740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:53.289 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:53.255+0000 7f068d233700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0688102f30 0x7f0688195c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:53.289 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:53.255+0000 7f068d233700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0688105850 0x7f068819ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:53.289 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:53.255+0000 7f068d233700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f068819d2c0 con 0x7f0688100610 2023-07-21T18:39:53.289 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:53.255+0000 7f068d233700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f068819d400 con 0x7f0688102f30 2023-07-21T18:39:53.290 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:53.255+0000 7f068d233700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f068819d590 con 0x7f0688105850 2023-07-21T18:39:53.290 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:53.255+0000 7f0686d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0688100610 0x7f0688195740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:53.290 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:53.255+0000 7f0686d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0688100610 0x7f0688195740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54848/0 (socket says 172.21.15.39:54848) 2023-07-21T18:39:53.291 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:53.255+0000 7f0686d9d700 1 -- 172.21.15.39:0/3277258437 learned_addr learned my addr 172.21.15.39:0/3277258437 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:39:53.291 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:53.255+0000 7f068759e700 1 --2- 172.21.15.39:0/3277258437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0688105850 0x7f068819ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:53.291 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:53.255+0000 7f068659c700 1 --2- 172.21.15.39:0/3277258437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0688102f30 0x7f0688195c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:53.291 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:53.255+0000 7f0686d9d700 1 -- 172.21.15.39:0/3277258437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0688105850 msgr2=0x7f068819ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:53.291 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:53.255+0000 7f0686d9d700 1 --2- 172.21.15.39:0/3277258437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0688105850 0x7f068819ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:53.291 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:53.255+0000 7f0686d9d700 1 -- 172.21.15.39:0/3277258437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0688102f30 msgr2=0x7f0688195c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:53.291 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:53.255+0000 7f0686d9d700 1 --2- 172.21.15.39:0/3277258437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0688102f30 0x7f0688195c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:53.292 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:53.255+0000 7f0686d9d700 1 -- 172.21.15.39:0/3277258437 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f067c0095f0 con 0x7f0688100610 2023-07-21T18:39:53.292 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:53.255+0000 7f068759e700 1 --2- 172.21.15.39:0/3277258437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0688105850 0x7f068819ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:39:53.292 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:53.256+0000 7f0686d9d700 1 --2- 172.21.15.39:0/3277258437 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0688100610 0x7f0688195740 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f067800c9b0 tx=0x7f067800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:39:53.292 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:53.256+0000 7f066ffff700 1 -- 172.21.15.39:0/3277258437 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0678007b00 con 0x7f0688100610 2023-07-21T18:39:53.292 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:53.256+0000 7f066ffff700 1 -- 172.21.15.39:0/3277258437 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f067800ce50 con 0x7f0688100610 2023-07-21T18:39:53.292 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:53.256+0000 7f066ffff700 1 -- 172.21.15.39:0/3277258437 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0678018660 con 0x7f0688100610 2023-07-21T18:39:53.292 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:53.256+0000 7f068d233700 1 -- 172.21.15.39:0/3277258437 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f068819d870 con 0x7f0688100610 2023-07-21T18:39:53.293 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:53.256+0000 7f068d233700 1 -- 172.21.15.39:0/3277258437 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f068819dd80 con 0x7f0688100610 2023-07-21T18:39:53.293 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:53.257+0000 7f068d233700 1 -- 172.21.15.39:0/3277258437 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06880fbb40 con 0x7f0688100610 2023-07-21T18:39:53.294 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:53.259+0000 7f066ffff700 1 -- 172.21.15.39:0/3277258437 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f067800f450 con 0x7f0688100610 2023-07-21T18:39:53.294 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:53.260+0000 7f066ffff700 1 --2- 172.21.15.39:0/3277258437 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f06700a1220 0x7f06700a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:53.294 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:53.260+0000 7f068659c700 1 --2- 172.21.15.39:0/3277258437 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f06700a1220 0x7f06700a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:53.294 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:53.260+0000 7f066ffff700 1 -- 172.21.15.39:0/3277258437 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f06780cc610 con 0x7f0688100610 2023-07-21T18:39:53.294 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:53.260+0000 7f068659c700 1 --2- 172.21.15.39:0/3277258437 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f06700a1220 0x7f06700a36d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f067c00b2e0 tx=0x7f067c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:39:53.294 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:53.263+0000 7f066ffff700 1 -- 172.21.15.39:0/3277258437 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f0678096270 con 0x7f0688100610 2023-07-21T18:39:53.316 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:53 smithi039 ceph-mon[96489]: Deploying daemon mds.g on smithi039 2023-07-21T18:39:53.316 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:53 smithi039 ceph-mon[96489]: mds.? [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] up:boot 2023-07-21T18:39:53.316 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:53 smithi039 ceph-mon[96489]: fsmap 9 up:standby 2023-07-21T18:39:53.318 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:53 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-21T18:39:53.374 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:53 smithi193 ceph-mon[98204]: Deploying daemon mds.g on smithi039 2023-07-21T18:39:53.375 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:53 smithi193 ceph-mon[98204]: mds.? [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] up:boot 2023-07-21T18:39:53.375 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:53 smithi193 ceph-mon[98204]: fsmap 9 up:standby 2023-07-21T18:39:53.375 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:53 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-21T18:39:53.393 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:53 smithi155 ceph-mon[97934]: Deploying daemon mds.g on smithi039 2023-07-21T18:39:53.394 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:53 smithi155 ceph-mon[97934]: mds.? [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] up:boot 2023-07-21T18:39:53.394 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:53 smithi155 ceph-mon[97934]: fsmap 9 up:standby 2023-07-21T18:39:53.394 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:53 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-21T18:39:53.553 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:53.526+0000 7f068d233700 1 -- 172.21.15.39:0/3277258437 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f068802c850 con 0x7f0688100610 2023-07-21T18:39:53.554 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:53.526+0000 7f066ffff700 1 -- 172.21.15.39:0/3277258437 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7327 (secure 0 0 0) 0x7f0678007e60 con 0x7f0688100610 2023-07-21T18:39:53.554 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:39:53.554 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/147318681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":147318681},{"type":"v1","addr":"172.21.15.39:6835","nonce":147318681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/1062891760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1062891760},{"type":"v1","addr":"172.21.15.39:6837","nonce":1062891760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/3106166804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3106166804},{"type":"v1","addr":"172.21.15.193:6833","nonce":3106166804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1034755277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1034755277},{"type":"v1","addr":"172.21.15.155:6833","nonce":1034755277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/381560657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":381560657},{"type":"v1","addr":"172.21.15.193:6835","nonce":381560657}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/2626761843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2626761843},{"type":"v1","addr":"172.21.15.193:6837","nonce":2626761843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3854856286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3854856286},{"type":"v1","addr":"172.21.15.155:6835","nonce":3854856286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/3148488639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3148488639},{"type":"v1","addr":"172.21.15.193:6839","nonce":3148488639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24500,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6837/947670056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":947670056},{"type":"v1","addr":"172.21.15.155:6837","nonce":947670056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[]} 2023-07-21T18:39:53.556 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:53.528+0000 7f068d233700 1 -- 172.21.15.39:0/3277258437 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f06700a1220 msgr2=0x7f06700a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:53.556 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:53.528+0000 7f068d233700 1 --2- 172.21.15.39:0/3277258437 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f06700a1220 0x7f06700a36d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f067c00b2e0 tx=0x7f067c005fb0 comp rx=0 tx=0).stop 2023-07-21T18:39:53.556 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:53.528+0000 7f068d233700 1 -- 172.21.15.39:0/3277258437 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0688100610 msgr2=0x7f0688195740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:53.557 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:53.528+0000 7f068d233700 1 --2- 172.21.15.39:0/3277258437 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0688100610 0x7f0688195740 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f067800c9b0 tx=0x7f067800ccc0 comp rx=0 tx=0).stop 2023-07-21T18:39:53.557 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:53.528+0000 7f068d233700 1 -- 172.21.15.39:0/3277258437 shutdown_connections 2023-07-21T18:39:53.557 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:53.528+0000 7f068d233700 1 --2- 172.21.15.39:0/3277258437 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f06700a1220 0x7f06700a36d0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:53.557 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:53.528+0000 7f068d233700 1 --2- 172.21.15.39:0/3277258437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0688105850 0x7f068819ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:53.557 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:53.528+0000 7f068d233700 1 --2- 172.21.15.39:0/3277258437 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0688100610 0x7f0688195740 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:53.557 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:53.528+0000 7f068d233700 1 --2- 172.21.15.39:0/3277258437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0688102f30 0x7f0688195c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:53.557 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:53.528+0000 7f068d233700 1 -- 172.21.15.39:0/3277258437 >> 172.21.15.39:0/3277258437 conn(0x7f06880f9fd0 msgr2=0x7f0688101270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:39:53.558 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:53.529+0000 7f068d233700 1 -- 172.21.15.39:0/3277258437 shutdown_connections 2023-07-21T18:39:53.558 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:53.529+0000 7f068d233700 1 -- 172.21.15.39:0/3277258437 wait complete. 2023-07-21T18:39:53.558 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 10 2023-07-21T18:39:53.568 INFO:journalctl@ceph.mds.g.smithi039.stdout:Jul 21 18:39:53 smithi039 systemd[1]: Starting Ceph mds.g for eaf8e25a-27f4-11ee-9b34-001a4aab830c... 2023-07-21T18:39:53.852 INFO:journalctl@ceph.mds.g.smithi039.stdout:Jul 21 18:39:53 smithi039 podman[130774]: 2023-07-21T18:39:54.121 INFO:journalctl@ceph.mds.g.smithi039.stdout:Jul 21 18:39:53 smithi039 ceph-mds[130817]: -- 172.21.15.39:0/3694940060 <== mon.0 v2:172.21.15.39:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55fdea86b1e0 con 0x55fdea840800 2023-07-21T18:39:54.121 INFO:journalctl@ceph.mds.g.smithi039.stdout:Jul 21 18:39:53 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-g[130792]: starting mds.g at 2023-07-21T18:39:54.121 INFO:journalctl@ceph.mds.g.smithi039.stdout:Jul 21 18:39:53 smithi039 bash[130774]: 032128f34a00a48b71eb17e343dd64231aac3d78ca6e692874062b663a7fd074 2023-07-21T18:39:54.121 INFO:journalctl@ceph.mds.g.smithi039.stdout:Jul 21 18:39:53 smithi039 systemd[1]: Started Ceph mds.g for eaf8e25a-27f4-11ee-9b34-001a4aab830c. 2023-07-21T18:39:54.122 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:54 smithi039 ceph-mon[96489]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:39:54.374 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:54 smithi193 ceph-mon[98204]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:39:54.374 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:54 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/3277258437' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:39:54.374 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:54 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:54.374 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:54 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:54.375 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:54 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:54.375 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:54 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T18:39:54.375 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:54 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T18:39:54.375 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:54 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:39:54.393 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:54 smithi155 ceph-mon[97934]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:39:54.393 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:54 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/3277258437' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:39:54.393 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:54 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:54.393 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:54 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:54.393 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:54 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:54.393 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:54 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T18:39:54.394 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:54 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T18:39:54.394 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:54 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:39:54.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:54 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/3277258437' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:39:54.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:54 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:54.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:54 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:54.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:54 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:54.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:54 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T18:39:54.443 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:54 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T18:39:54.443 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:54 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:39:55.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:55 smithi039 ceph-mon[96489]: Deploying daemon mds.a on smithi039 2023-07-21T18:39:55.192 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:55 smithi039 ceph-mon[96489]: mds.? [v2:172.21.15.39:6838/3184273349,v1:172.21.15.39:6839/3184273349] up:boot 2023-07-21T18:39:55.192 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:55 smithi039 ceph-mon[96489]: fsmap 10 up:standby 2023-07-21T18:39:55.192 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:55 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-21T18:39:55.374 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:55 smithi193 ceph-mon[98204]: Deploying daemon mds.a on smithi039 2023-07-21T18:39:55.374 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:55 smithi193 ceph-mon[98204]: mds.? [v2:172.21.15.39:6838/3184273349,v1:172.21.15.39:6839/3184273349] up:boot 2023-07-21T18:39:55.374 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:55 smithi193 ceph-mon[98204]: fsmap 10 up:standby 2023-07-21T18:39:55.375 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:55 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-21T18:39:55.393 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:55 smithi155 ceph-mon[97934]: Deploying daemon mds.a on smithi039 2023-07-21T18:39:55.393 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:55 smithi155 ceph-mon[97934]: mds.? [v2:172.21.15.39:6838/3184273349,v1:172.21.15.39:6839/3184273349] up:boot 2023-07-21T18:39:55.393 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:55 smithi155 ceph-mon[97934]: fsmap 10 up:standby 2023-07-21T18:39:55.393 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:55 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-21T18:39:55.569 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T18:39:55.709 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:55.682+0000 7eff90c26700 1 -- 172.21.15.39:0/1001521212 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff8c0ffbc0 msgr2=0x7eff8c100010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:55.710 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:55.682+0000 7eff90c26700 1 --2- 172.21.15.39:0/1001521212 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff8c0ffbc0 0x7eff8c100010 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7eff80009990 tx=0x7eff80009ca0 comp rx=0 tx=0).stop 2023-07-21T18:39:55.710 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:55.682+0000 7eff90c26700 1 -- 172.21.15.39:0/1001521212 shutdown_connections 2023-07-21T18:39:55.710 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:55.682+0000 7eff90c26700 1 --2- 172.21.15.39:0/1001521212 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff8c100550 0x7eff8c106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:55.710 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:55.682+0000 7eff90c26700 1 --2- 172.21.15.39:0/1001521212 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff8c0fe9d0 0x7eff8c0fede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:55.710 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:55.682+0000 7eff90c26700 1 --2- 172.21.15.39:0/1001521212 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff8c0ffbc0 0x7eff8c100010 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:55.711 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:55.682+0000 7eff90c26700 1 -- 172.21.15.39:0/1001521212 >> 172.21.15.39:0/1001521212 conn(0x7eff8c0f9fd0 msgr2=0x7eff8c0fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:39:55.711 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:55.682+0000 7eff90c26700 1 -- 172.21.15.39:0/1001521212 shutdown_connections 2023-07-21T18:39:55.711 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:55.682+0000 7eff90c26700 1 -- 172.21.15.39:0/1001521212 wait complete. 2023-07-21T18:39:55.713 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:55.683+0000 7eff90c26700 1 Processor -- start 2023-07-21T18:39:55.713 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:55.683+0000 7eff90c26700 1 -- start start 2023-07-21T18:39:55.713 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:55.683+0000 7eff90c26700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff8c0fe9d0 0x7eff8c102bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:55.714 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:55.683+0000 7eff90c26700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff8c0ffbc0 0x7eff8c101200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:55.714 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:55.683+0000 7eff90c26700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff8c100550 0x7eff8c101740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:55.714 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:55.683+0000 7eff90c26700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff8c101ee0 con 0x7eff8c100550 2023-07-21T18:39:55.714 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:55.683+0000 7eff90c26700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff8c102050 con 0x7eff8c0ffbc0 2023-07-21T18:39:55.714 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:55.683+0000 7eff90c26700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff8c19f7f0 con 0x7eff8c0fe9d0 2023-07-21T18:39:55.714 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:55.683+0000 7eff8ad9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff8c100550 0x7eff8c101740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:55.714 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:55.683+0000 7eff8ad9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff8c100550 0x7eff8c101740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54880/0 (socket says 172.21.15.39:54880) 2023-07-21T18:39:55.715 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:55.683+0000 7eff8ad9d700 1 -- 172.21.15.39:0/3571461327 learned_addr learned my addr 172.21.15.39:0/3571461327 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:39:55.715 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:55.683+0000 7eff8a59c700 1 --2- 172.21.15.39:0/3571461327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff8c0fe9d0 0x7eff8c102bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:55.715 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:55.684+0000 7eff89d9b700 1 --2- 172.21.15.39:0/3571461327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff8c0ffbc0 0x7eff8c101200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:55.715 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:55.684+0000 7eff8ad9d700 1 -- 172.21.15.39:0/3571461327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff8c0fe9d0 msgr2=0x7eff8c102bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:55.715 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:55.684+0000 7eff8ad9d700 1 --2- 172.21.15.39:0/3571461327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff8c0fe9d0 0x7eff8c102bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:55.715 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:55.684+0000 7eff8ad9d700 1 -- 172.21.15.39:0/3571461327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff8c0ffbc0 msgr2=0x7eff8c101200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:55.715 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:55.684+0000 7eff8ad9d700 1 --2- 172.21.15.39:0/3571461327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff8c0ffbc0 0x7eff8c101200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:55.715 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:55.684+0000 7eff8ad9d700 1 -- 172.21.15.39:0/3571461327 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff800095f0 con 0x7eff8c100550 2023-07-21T18:39:55.716 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:55.684+0000 7eff8a59c700 1 --2- 172.21.15.39:0/3571461327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff8c0fe9d0 0x7eff8c102bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:39:55.716 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:55.684+0000 7eff8ad9d700 1 --2- 172.21.15.39:0/3571461327 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff8c100550 0x7eff8c101740 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7eff8400ef30 tx=0x7eff8400c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:39:55.716 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:55.684+0000 7eff7b7fe700 1 -- 172.21.15.39:0/3571461327 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff84018070 con 0x7eff8c100550 2023-07-21T18:39:55.716 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:55.684+0000 7eff7b7fe700 1 -- 172.21.15.39:0/3571461327 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff8400f040 con 0x7eff8c100550 2023-07-21T18:39:55.716 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:55.684+0000 7eff90c26700 1 -- 172.21.15.39:0/3571461327 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff8c19fa70 con 0x7eff8c100550 2023-07-21T18:39:55.716 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:55.684+0000 7eff7b7fe700 1 -- 172.21.15.39:0/3571461327 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff84003dc0 con 0x7eff8c100550 2023-07-21T18:39:55.716 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:55.684+0000 7eff90c26700 1 -- 172.21.15.39:0/3571461327 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff8c19ff80 con 0x7eff8c100550 2023-07-21T18:39:55.717 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:55.685+0000 7eff90c26700 1 -- 172.21.15.39:0/3571461327 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff8c106a00 con 0x7eff8c100550 2023-07-21T18:39:55.717 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:55.686+0000 7eff7b7fe700 1 -- 172.21.15.39:0/3571461327 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7eff8400caa0 con 0x7eff8c100550 2023-07-21T18:39:55.717 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:55.687+0000 7eff7b7fe700 1 --2- 172.21.15.39:0/3571461327 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7eff740a11a0 0x7eff740a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:55.718 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:55.687+0000 7eff8a59c700 1 --2- 172.21.15.39:0/3571461327 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7eff740a11a0 0x7eff740a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:55.718 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:55.687+0000 7eff7b7fe700 1 -- 172.21.15.39:0/3571461327 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7eff840cc0c0 con 0x7eff8c100550 2023-07-21T18:39:55.718 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:55.687+0000 7eff8a59c700 1 --2- 172.21.15.39:0/3571461327 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7eff740a11a0 0x7eff740a3650 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7eff8c0ffa20 tx=0x7eff7c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:39:55.718 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:55.691+0000 7eff7b7fe700 1 -- 172.21.15.39:0/3571461327 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7eff84095d40 con 0x7eff8c100550 2023-07-21T18:39:55.802 INFO:journalctl@ceph.mds.a.smithi039.stdout:Jul 21 18:39:55 smithi039 systemd[1]: Starting Ceph mds.a for eaf8e25a-27f4-11ee-9b34-001a4aab830c... 2023-07-21T18:39:55.968 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:55.941+0000 7eff90c26700 1 -- 172.21.15.39:0/3571461327 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7eff8c02ca60 con 0x7eff8c100550 2023-07-21T18:39:55.969 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:55.942+0000 7eff7b7fe700 1 -- 172.21.15.39:0/3571461327 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8085 (secure 0 0 0) 0x7eff84099360 con 0x7eff8c100550 2023-07-21T18:39:55.969 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:39:55.969 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/147318681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":147318681},{"type":"v1","addr":"172.21.15.39:6835","nonce":147318681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/1062891760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1062891760},{"type":"v1","addr":"172.21.15.39:6837","nonce":1062891760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/3184273349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":3184273349},{"type":"v1","addr":"172.21.15.39:6839","nonce":3184273349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/3106166804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3106166804},{"type":"v1","addr":"172.21.15.193:6833","nonce":3106166804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1034755277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1034755277},{"type":"v1","addr":"172.21.15.155:6833","nonce":1034755277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/381560657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":381560657},{"type":"v1","addr":"172.21.15.193:6835","nonce":381560657}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/2626761843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2626761843},{"type":"v1","addr":"172.21.15.193:6837","nonce":2626761843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3854856286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3854856286},{"type":"v1","addr":"172.21.15.155:6835","nonce":3854856286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/3148488639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3148488639},{"type":"v1","addr":"172.21.15.193:6839","nonce":3148488639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24500,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6837/947670056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":947670056},{"type":"v1","addr":"172.21.15.155:6837","nonce":947670056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[]} 2023-07-21T18:39:55.971 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:55.944+0000 7eff90c26700 1 -- 172.21.15.39:0/3571461327 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7eff740a11a0 msgr2=0x7eff740a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:55.971 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:55.944+0000 7eff90c26700 1 --2- 172.21.15.39:0/3571461327 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7eff740a11a0 0x7eff740a3650 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7eff8c0ffa20 tx=0x7eff7c009260 comp rx=0 tx=0).stop 2023-07-21T18:39:55.971 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:55.944+0000 7eff90c26700 1 -- 172.21.15.39:0/3571461327 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff8c100550 msgr2=0x7eff8c101740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:55.972 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:55.944+0000 7eff90c26700 1 --2- 172.21.15.39:0/3571461327 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff8c100550 0x7eff8c101740 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7eff8400ef30 tx=0x7eff8400c450 comp rx=0 tx=0).stop 2023-07-21T18:39:55.972 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:55.944+0000 7eff90c26700 1 -- 172.21.15.39:0/3571461327 shutdown_connections 2023-07-21T18:39:55.972 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:55.944+0000 7eff90c26700 1 --2- 172.21.15.39:0/3571461327 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7eff740a11a0 0x7eff740a3650 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:55.973 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:55.944+0000 7eff90c26700 1 --2- 172.21.15.39:0/3571461327 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff8c100550 0x7eff8c101740 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:55.973 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:55.944+0000 7eff90c26700 1 --2- 172.21.15.39:0/3571461327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff8c0fe9d0 0x7eff8c102bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:55.973 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:55.944+0000 7eff90c26700 1 --2- 172.21.15.39:0/3571461327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff8c0ffbc0 0x7eff8c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:55.973 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:55.944+0000 7eff90c26700 1 -- 172.21.15.39:0/3571461327 >> 172.21.15.39:0/3571461327 conn(0x7eff8c0f9fd0 msgr2=0x7eff8c0fc1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:39:55.973 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:55.944+0000 7eff90c26700 1 -- 172.21.15.39:0/3571461327 shutdown_connections 2023-07-21T18:39:55.973 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:55.944+0000 7eff90c26700 1 -- 172.21.15.39:0/3571461327 wait complete. 2023-07-21T18:39:55.974 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 11 2023-07-21T18:39:56.057 INFO:journalctl@ceph.mds.a.smithi039.stdout:Jul 21 18:39:55 smithi039 podman[131139]: 2023-07-21T18:39:56.393 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:56 smithi155 ceph-mon[97934]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:39:56.393 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:56 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/3571461327' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:39:56.394 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:56 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:56.394 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:56 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:56.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:56 smithi039 ceph-mon[96489]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:39:56.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:56 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/3571461327' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:39:56.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:56 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:56.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:56 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:56.442 INFO:journalctl@ceph.mds.a.smithi039.stdout:Jul 21 18:39:56 smithi039 bash[131139]: a5de16bb3f5dd880f6d7769a8c0b31f29e7b7cbea40224eaffe40459ef21a252 2023-07-21T18:39:56.443 INFO:journalctl@ceph.mds.a.smithi039.stdout:Jul 21 18:39:56 smithi039 ceph-mds[131201]: -- 172.21.15.39:0/333466291 <== mon.0 v2:172.21.15.39:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562a0d7311e0 con 0x562a0d706800 2023-07-21T18:39:56.443 INFO:journalctl@ceph.mds.a.smithi039.stdout:Jul 21 18:39:56 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-a[131175]: starting mds.a at 2023-07-21T18:39:56.443 INFO:journalctl@ceph.mds.a.smithi039.stdout:Jul 21 18:39:56 smithi039 systemd[1]: Started Ceph mds.a for eaf8e25a-27f4-11ee-9b34-001a4aab830c. 2023-07-21T18:39:56.624 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:56 smithi193 ceph-mon[98204]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:39:56.624 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:56 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/3571461327' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:39:56.624 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:56 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:56.624 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:56 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:57.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:57 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:57.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:57 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T18:39:57.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:57 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T18:39:57.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:57 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:39:57.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:57 smithi039 ceph-mon[96489]: mds.? [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] up:boot 2023-07-21T18:39:57.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:57 smithi039 ceph-mon[96489]: fsmap 11 up:standby 2023-07-21T18:39:57.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:57 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-21T18:39:57.513 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:57 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:57.513 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:57 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T18:39:57.514 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:57 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T18:39:57.514 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:57 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:39:57.514 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:57 smithi155 ceph-mon[97934]: mds.? [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] up:boot 2023-07-21T18:39:57.514 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:57 smithi155 ceph-mon[97934]: fsmap 11 up:standby 2023-07-21T18:39:57.514 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:57 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-21T18:39:57.624 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:57 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:57.624 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:57 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T18:39:57.624 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:57 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T18:39:57.625 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:57 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:39:57.625 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:57 smithi193 ceph-mon[98204]: mds.? [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] up:boot 2023-07-21T18:39:57.625 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:57 smithi193 ceph-mon[98204]: fsmap 11 up:standby 2023-07-21T18:39:57.625 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:57 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-21T18:39:57.893 INFO:journalctl@ceph.mds.b.smithi155.stdout:Jul 21 18:39:57 smithi155 systemd[1]: Starting Ceph mds.b for eaf8e25a-27f4-11ee-9b34-001a4aab830c... 2023-07-21T18:39:57.986 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T18:39:58.128 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:58.101+0000 7f952a290700 1 -- 172.21.15.39:0/2451713669 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95240ffc10 msgr2=0x7f9524100060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:58.128 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:58.101+0000 7f952a290700 1 --2- 172.21.15.39:0/2451713669 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95240ffc10 0x7f9524100060 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9518009990 tx=0x7f9518009ca0 comp rx=0 tx=0).stop 2023-07-21T18:39:58.128 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:58.101+0000 7f952a290700 1 -- 172.21.15.39:0/2451713669 shutdown_connections 2023-07-21T18:39:58.129 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:58.101+0000 7f952a290700 1 --2- 172.21.15.39:0/2451713669 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95241005a0 0x7f9524106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:58.129 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:58.101+0000 7f952a290700 1 --2- 172.21.15.39:0/2451713669 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95240fea20 0x7f95240fee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:58.129 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:58.101+0000 7f952a290700 1 --2- 172.21.15.39:0/2451713669 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95240ffc10 0x7f9524100060 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:58.129 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:58.101+0000 7f952a290700 1 -- 172.21.15.39:0/2451713669 >> 172.21.15.39:0/2451713669 conn(0x7f95240fa020 msgr2=0x7f95240fc470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:39:58.129 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:58.101+0000 7f952a290700 1 -- 172.21.15.39:0/2451713669 shutdown_connections 2023-07-21T18:39:58.129 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:58.101+0000 7f952a290700 1 -- 172.21.15.39:0/2451713669 wait complete. 2023-07-21T18:39:58.132 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:58.101+0000 7f952a290700 1 Processor -- start 2023-07-21T18:39:58.132 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:58.102+0000 7f952a290700 1 -- start start 2023-07-21T18:39:58.132 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:58.102+0000 7f952a290700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95240fea20 0x7f9524069190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:58.132 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:58.102+0000 7f952a290700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95240ffc10 0x7f952419a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:58.133 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:58.102+0000 7f952a290700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95241005a0 0x7f952419cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:58.133 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:58.102+0000 7f952a290700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952419d320 con 0x7f95240ffc10 2023-07-21T18:39:58.133 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:58.102+0000 7f952a290700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952419d490 con 0x7f95240fea20 2023-07-21T18:39:58.133 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:58.102+0000 7f952a290700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952419d650 con 0x7f95241005a0 2023-07-21T18:39:58.133 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:58.102+0000 7f95237fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95240ffc10 0x7f952419a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:58.133 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:58.102+0000 7f9523fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95240fea20 0x7f9524069190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:58.133 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:58.102+0000 7f9523fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95240fea20 0x7f9524069190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:39268/0 (socket says 172.21.15.39:39268) 2023-07-21T18:39:58.134 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:58.102+0000 7f95237fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95240ffc10 0x7f952419a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54906/0 (socket says 172.21.15.39:54906) 2023-07-21T18:39:58.134 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:58.102+0000 7f9523fff700 1 -- 172.21.15.39:0/1085498854 learned_addr learned my addr 172.21.15.39:0/1085498854 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:39:58.134 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:58.102+0000 7f952882d700 1 --2- 172.21.15.39:0/1085498854 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95241005a0 0x7f952419cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:58.134 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:58.102+0000 7f95237fe700 1 -- 172.21.15.39:0/1085498854 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95241005a0 msgr2=0x7f952419cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:58.134 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:58.102+0000 7f95237fe700 1 --2- 172.21.15.39:0/1085498854 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95241005a0 0x7f952419cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:58.134 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:58.102+0000 7f95237fe700 1 -- 172.21.15.39:0/1085498854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95240fea20 msgr2=0x7f9524069190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:58.134 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:58.102+0000 7f95237fe700 1 --2- 172.21.15.39:0/1085498854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95240fea20 0x7f9524069190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:58.134 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:58.102+0000 7f95237fe700 1 -- 172.21.15.39:0/1085498854 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95180095f0 con 0x7f95240ffc10 2023-07-21T18:39:58.135 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:58.102+0000 7f9523fff700 1 --2- 172.21.15.39:0/1085498854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95240fea20 0x7f9524069190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:39:58.135 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:58.103+0000 7f95237fe700 1 --2- 172.21.15.39:0/1085498854 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95240ffc10 0x7f952419a500 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f95180059d0 tx=0x7f9518004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:39:58.135 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:58.103+0000 7f95217fa700 1 -- 172.21.15.39:0/1085498854 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951801d070 con 0x7f95240ffc10 2023-07-21T18:39:58.135 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:58.103+0000 7f95217fa700 1 -- 172.21.15.39:0/1085498854 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f951800fbc0 con 0x7f95240ffc10 2023-07-21T18:39:58.135 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:58.103+0000 7f952a290700 1 -- 172.21.15.39:0/1085498854 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f952419d8d0 con 0x7f95240ffc10 2023-07-21T18:39:58.135 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:58.103+0000 7f95217fa700 1 -- 172.21.15.39:0/1085498854 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9518017650 con 0x7f95240ffc10 2023-07-21T18:39:58.135 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:58.103+0000 7f952a290700 1 -- 172.21.15.39:0/1085498854 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f952419dcf0 con 0x7f95240ffc10 2023-07-21T18:39:58.135 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:58.104+0000 7f952a290700 1 -- 172.21.15.39:0/1085498854 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f952410a1f0 con 0x7f95240ffc10 2023-07-21T18:39:58.136 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:58.105+0000 7f95217fa700 1 -- 172.21.15.39:0/1085498854 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f9518022470 con 0x7f95240ffc10 2023-07-21T18:39:58.136 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:58.105+0000 7f95217fa700 1 --2- 172.21.15.39:0/1085498854 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f950c0a11a0 0x7f950c0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:39:58.137 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:58.106+0000 7f9523fff700 1 --2- 172.21.15.39:0/1085498854 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f950c0a11a0 0x7f950c0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:39:58.137 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:58.106+0000 7f95217fa700 1 -- 172.21.15.39:0/1085498854 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f95180ce380 con 0x7f95240ffc10 2023-07-21T18:39:58.137 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:58.106+0000 7f9523fff700 1 --2- 172.21.15.39:0/1085498854 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f950c0a11a0 0x7f950c0a3650 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f95140105c0 tx=0x7f951400b4a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:39:58.137 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:58.109+0000 7f95217fa700 1 -- 172.21.15.39:0/1085498854 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f9518097f60 con 0x7f95240ffc10 2023-07-21T18:39:58.331 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:58 smithi155 ceph-mon[97934]: Deploying daemon mds.b on smithi155 2023-07-21T18:39:58.332 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:58 smithi155 ceph-mon[97934]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:39:58.332 INFO:journalctl@ceph.mds.b.smithi155.stdout:Jul 21 18:39:58 smithi155 podman[123254]: 2023-07-21T18:39:58.384 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:58.357+0000 7f952a290700 1 -- 172.21.15.39:0/1085498854 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9524105880 con 0x7f95240ffc10 2023-07-21T18:39:58.384 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:58.357+0000 7f95217fa700 1 -- 172.21.15.39:0/1085498854 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+8840 (secure 0 0 0) 0x7f951809b580 con 0x7f95240ffc10 2023-07-21T18:39:58.385 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:39:58.385 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/147318681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":147318681},{"type":"v1","addr":"172.21.15.39:6835","nonce":147318681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/1062891760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1062891760},{"type":"v1","addr":"172.21.15.39:6837","nonce":1062891760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/3184273349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":3184273349},{"type":"v1","addr":"172.21.15.39:6839","nonce":3184273349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6841/640981220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":640981220},{"type":"v1","addr":"172.21.15.39:6841","nonce":640981220}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/3106166804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3106166804},{"type":"v1","addr":"172.21.15.193:6833","nonce":3106166804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1034755277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1034755277},{"type":"v1","addr":"172.21.15.155:6833","nonce":1034755277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/381560657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":381560657},{"type":"v1","addr":"172.21.15.193:6835","nonce":381560657}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/2626761843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2626761843},{"type":"v1","addr":"172.21.15.193:6837","nonce":2626761843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3854856286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3854856286},{"type":"v1","addr":"172.21.15.155:6835","nonce":3854856286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/3148488639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3148488639},{"type":"v1","addr":"172.21.15.193:6839","nonce":3148488639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24500,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6837/947670056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":947670056},{"type":"v1","addr":"172.21.15.155:6837","nonce":947670056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[]} 2023-07-21T18:39:58.388 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:58.360+0000 7f952a290700 1 -- 172.21.15.39:0/1085498854 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f950c0a11a0 msgr2=0x7f950c0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:58.388 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:58.361+0000 7f952a290700 1 --2- 172.21.15.39:0/1085498854 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f950c0a11a0 0x7f950c0a3650 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f95140105c0 tx=0x7f951400b4a0 comp rx=0 tx=0).stop 2023-07-21T18:39:58.388 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:58.361+0000 7f952a290700 1 -- 172.21.15.39:0/1085498854 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95240ffc10 msgr2=0x7f952419a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:39:58.388 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:58.361+0000 7f952a290700 1 --2- 172.21.15.39:0/1085498854 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95240ffc10 0x7f952419a500 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f95180059d0 tx=0x7f9518004730 comp rx=0 tx=0).stop 2023-07-21T18:39:58.388 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:58.361+0000 7f952a290700 1 -- 172.21.15.39:0/1085498854 shutdown_connections 2023-07-21T18:39:58.389 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:58.361+0000 7f952a290700 1 --2- 172.21.15.39:0/1085498854 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f950c0a11a0 0x7f950c0a3650 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:58.389 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:58.361+0000 7f952a290700 1 --2- 172.21.15.39:0/1085498854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95240fea20 0x7f9524069190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:58.389 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:58.361+0000 7f952a290700 1 --2- 172.21.15.39:0/1085498854 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95241005a0 0x7f952419cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:58.389 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:58.361+0000 7f952a290700 1 --2- 172.21.15.39:0/1085498854 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95240ffc10 0x7f952419a500 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:39:58.389 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:58.361+0000 7f952a290700 1 -- 172.21.15.39:0/1085498854 >> 172.21.15.39:0/1085498854 conn(0x7f95240fa020 msgr2=0x7f95240fc1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:39:58.390 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:58.361+0000 7f952a290700 1 -- 172.21.15.39:0/1085498854 shutdown_connections 2023-07-21T18:39:58.390 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:39:58.361+0000 7f952a290700 1 -- 172.21.15.39:0/1085498854 wait complete. 2023-07-21T18:39:58.390 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 12 2023-07-21T18:39:58.400 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:58 smithi039 ceph-mon[96489]: Deploying daemon mds.b on smithi155 2023-07-21T18:39:58.400 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:58 smithi039 ceph-mon[96489]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:39:58.624 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:58 smithi193 ceph-mon[98204]: Deploying daemon mds.b on smithi155 2023-07-21T18:39:58.624 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:58 smithi193 ceph-mon[98204]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:39:58.646 INFO:journalctl@ceph.mds.b.smithi155.stdout:Jul 21 18:39:58 smithi155 ceph-mds[123294]: -- 172.21.15.155:0/1281661987 <== mon.2 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ae26b171e0 con 0x55ae26aed000 2023-07-21T18:39:58.646 INFO:journalctl@ceph.mds.b.smithi155.stdout:Jul 21 18:39:58 smithi155 bash[123254]: 009e64f168b0da081c0c23cff84cd0ab8106447f6bc36fc237622aaf7c9c4f7c 2023-07-21T18:39:58.646 INFO:journalctl@ceph.mds.b.smithi155.stdout:Jul 21 18:39:58 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-b[123270]: starting mds.b at 2023-07-21T18:39:58.646 INFO:journalctl@ceph.mds.b.smithi155.stdout:Jul 21 18:39:58 smithi155 systemd[1]: Started Ceph mds.b for eaf8e25a-27f4-11ee-9b34-001a4aab830c. 2023-07-21T18:39:59.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:59 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/1085498854' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:39:59.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:59 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:59.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:59 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:59.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:59 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:59.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:59 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:59.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:59 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:59.443 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:59 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:39:59.443 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:39:59 smithi039 ceph-mon[96489]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:39:59.624 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:59 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/1085498854' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:39:59.625 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:59 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:59.625 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:59 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:59.625 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:59 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:59.625 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:59 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:59.625 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:59 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:59.625 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:59 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:39:59.626 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:39:59 smithi193 ceph-mon[98204]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:39:59.642 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:59 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/1085498854' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:39:59.643 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:59 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:59.643 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:59 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:59.643 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:59 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:59.643 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:59 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:59.643 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:59 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:39:59.644 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:59 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:39:59.644 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:39:59 smithi155 ceph-mon[97934]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:40:00.373 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:00 smithi193 ceph-mon[98204]: mds.? [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] up:boot 2023-07-21T18:40:00.374 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:00 smithi193 ceph-mon[98204]: fsmap 12 up:standby 2023-07-21T18:40:00.374 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:00 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-21T18:40:00.374 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:00 smithi193 ceph-mon[98204]: overall HEALTH_OK 2023-07-21T18:40:00.392 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:00 smithi155 ceph-mon[97934]: mds.? [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] up:boot 2023-07-21T18:40:00.393 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:00 smithi155 ceph-mon[97934]: fsmap 12 up:standby 2023-07-21T18:40:00.393 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:00 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-21T18:40:00.393 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:00 smithi155 ceph-mon[97934]: overall HEALTH_OK 2023-07-21T18:40:00.401 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T18:40:00.429 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:00 smithi039 ceph-mon[96489]: mds.? [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] up:boot 2023-07-21T18:40:00.429 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:00 smithi039 ceph-mon[96489]: fsmap 12 up:standby 2023-07-21T18:40:00.429 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:00 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-21T18:40:00.429 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:00 smithi039 ceph-mon[96489]: overall HEALTH_OK 2023-07-21T18:40:00.536 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.509+0000 7ff65fe34700 1 -- 172.21.15.39:0/1673613322 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff658102e70 msgr2=0x7ff658105250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:00.536 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.509+0000 7ff65fe34700 1 --2- 172.21.15.39:0/1673613322 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff658102e70 0x7ff658105250 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff648009990 tx=0x7ff648009ca0 comp rx=0 tx=0).stop 2023-07-21T18:40:00.537 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.509+0000 7ff65fe34700 1 -- 172.21.15.39:0/1673613322 shutdown_connections 2023-07-21T18:40:00.537 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.509+0000 7ff65fe34700 1 --2- 172.21.15.39:0/1673613322 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff658105790 0x7ff658107bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:00.537 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.509+0000 7ff65fe34700 1 --2- 172.21.15.39:0/1673613322 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff658100550 0x7ff658102930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:00.537 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.509+0000 7ff65fe34700 1 --2- 172.21.15.39:0/1673613322 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff658102e70 0x7ff658105250 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:00.537 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.509+0000 7ff65fe34700 1 -- 172.21.15.39:0/1673613322 >> 172.21.15.39:0/1673613322 conn(0x7ff6580f9fd0 msgr2=0x7ff6580fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:00.537 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.509+0000 7ff65fe34700 1 -- 172.21.15.39:0/1673613322 shutdown_connections 2023-07-21T18:40:00.537 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.509+0000 7ff65fe34700 1 -- 172.21.15.39:0/1673613322 wait complete. 2023-07-21T18:40:00.538 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.510+0000 7ff65fe34700 1 Processor -- start 2023-07-21T18:40:00.539 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.510+0000 7ff65fe34700 1 -- start start 2023-07-21T18:40:00.540 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.510+0000 7ff65fe34700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff658100550 0x7ff6580ffec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:00.540 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.510+0000 7ff65fe34700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff658102e70 0x7ff6580fe500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:00.540 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.510+0000 7ff65fe34700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff658105790 0x7ff6580fea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:00.540 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.510+0000 7ff65fe34700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6580ff1e0 con 0x7ff658100550 2023-07-21T18:40:00.540 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.510+0000 7ff65fe34700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6580ff350 con 0x7ff658105790 2023-07-21T18:40:00.540 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.510+0000 7ff65fe34700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff65819f7f0 con 0x7ff658102e70 2023-07-21T18:40:00.540 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.510+0000 7ff65dbd0700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff658100550 0x7ff6580ffec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:00.541 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.510+0000 7ff65dbd0700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff658100550 0x7ff6580ffec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54916/0 (socket says 172.21.15.39:54916) 2023-07-21T18:40:00.541 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.510+0000 7ff65dbd0700 1 -- 172.21.15.39:0/1908037618 learned_addr learned my addr 172.21.15.39:0/1908037618 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:40:00.541 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.510+0000 7ff65d3cf700 1 --2- 172.21.15.39:0/1908037618 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff658102e70 0x7ff6580fe500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:00.541 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.510+0000 7ff65e3d1700 1 --2- 172.21.15.39:0/1908037618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff658105790 0x7ff6580fea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:00.541 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.510+0000 7ff65dbd0700 1 -- 172.21.15.39:0/1908037618 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff658102e70 msgr2=0x7ff6580fe500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:00.541 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.510+0000 7ff65dbd0700 1 --2- 172.21.15.39:0/1908037618 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff658102e70 0x7ff6580fe500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:00.541 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.510+0000 7ff65dbd0700 1 -- 172.21.15.39:0/1908037618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff658105790 msgr2=0x7ff6580fea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:00.541 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.510+0000 7ff65dbd0700 1 --2- 172.21.15.39:0/1908037618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff658105790 0x7ff6580fea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:00.542 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.510+0000 7ff65dbd0700 1 -- 172.21.15.39:0/1908037618 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6480095f0 con 0x7ff658100550 2023-07-21T18:40:00.542 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.510+0000 7ff65e3d1700 1 --2- 172.21.15.39:0/1908037618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff658105790 0x7ff6580fea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:40:00.542 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.510+0000 7ff65dbd0700 1 --2- 172.21.15.39:0/1908037618 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff658100550 0x7ff6580ffec0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff65400d980 tx=0x7ff65400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:00.542 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.511+0000 7ff64effd700 1 -- 172.21.15.39:0/1908037618 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff654004220 con 0x7ff658100550 2023-07-21T18:40:00.542 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.511+0000 7ff64effd700 1 -- 172.21.15.39:0/1908037618 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff654007640 con 0x7ff658100550 2023-07-21T18:40:00.542 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.511+0000 7ff65fe34700 1 -- 172.21.15.39:0/1908037618 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff65819fad0 con 0x7ff658100550 2023-07-21T18:40:00.542 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.511+0000 7ff64effd700 1 -- 172.21.15.39:0/1908037618 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff654005610 con 0x7ff658100550 2023-07-21T18:40:00.542 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.511+0000 7ff65fe34700 1 -- 172.21.15.39:0/1908037618 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff65819ffe0 con 0x7ff658100550 2023-07-21T18:40:00.543 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.512+0000 7ff65fe34700 1 -- 172.21.15.39:0/1908037618 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6580fbb40 con 0x7ff658100550 2023-07-21T18:40:00.543 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.513+0000 7ff64effd700 1 -- 172.21.15.39:0/1908037618 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff654004750 con 0x7ff658100550 2023-07-21T18:40:00.543 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.513+0000 7ff64effd700 1 --2- 172.21.15.39:0/1908037618 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7ff6440a11a0 0x7ff6440a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:00.544 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.513+0000 7ff65d3cf700 1 --2- 172.21.15.39:0/1908037618 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7ff6440a11a0 0x7ff6440a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:00.544 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.513+0000 7ff64effd700 1 -- 172.21.15.39:0/1908037618 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff6540cc810 con 0x7ff658100550 2023-07-21T18:40:00.544 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.514+0000 7ff65d3cf700 1 --2- 172.21.15.39:0/1908037618 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7ff6440a11a0 0x7ff6440a3650 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff64800b5c0 tx=0x7ff648005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:00.544 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.517+0000 7ff64effd700 1 -- 172.21.15.39:0/1908037618 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7ff654096400 con 0x7ff658100550 2023-07-21T18:40:00.799 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.772+0000 7ff65fe34700 1 -- 172.21.15.39:0/1908037618 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff65802c850 con 0x7ff658100550 2023-07-21T18:40:00.799 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.772+0000 7ff64effd700 1 -- 172.21.15.39:0/1908037618 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+9598 (secure 0 0 0) 0x7ff654099a20 con 0x7ff658100550 2023-07-21T18:40:00.800 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:40:00.800 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/147318681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":147318681},{"type":"v1","addr":"172.21.15.39:6835","nonce":147318681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/1062891760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1062891760},{"type":"v1","addr":"172.21.15.39:6837","nonce":1062891760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/3184273349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":3184273349},{"type":"v1","addr":"172.21.15.39:6839","nonce":3184273349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6841/640981220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":640981220},{"type":"v1","addr":"172.21.15.39:6841","nonce":640981220}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/3106166804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3106166804},{"type":"v1","addr":"172.21.15.193:6833","nonce":3106166804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1034755277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1034755277},{"type":"v1","addr":"172.21.15.155:6833","nonce":1034755277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/381560657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":381560657},{"type":"v1","addr":"172.21.15.193:6835","nonce":381560657}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/2626761843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2626761843},{"type":"v1","addr":"172.21.15.193:6837","nonce":2626761843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3854856286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3854856286},{"type":"v1","addr":"172.21.15.155:6835","nonce":3854856286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/3148488639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3148488639},{"type":"v1","addr":"172.21.15.193:6839","nonce":3148488639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24500,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6837/947670056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":947670056},{"type":"v1","addr":"172.21.15.155:6837","nonce":947670056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24536,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6839/431690342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":431690342},{"type":"v1","addr":"172.21.15.155:6839","nonce":431690342}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[]} 2023-07-21T18:40:00.801 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.774+0000 7ff65fe34700 1 -- 172.21.15.39:0/1908037618 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7ff6440a11a0 msgr2=0x7ff6440a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:00.801 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.774+0000 7ff65fe34700 1 --2- 172.21.15.39:0/1908037618 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7ff6440a11a0 0x7ff6440a3650 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff64800b5c0 tx=0x7ff648005fb0 comp rx=0 tx=0).stop 2023-07-21T18:40:00.802 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.774+0000 7ff65fe34700 1 -- 172.21.15.39:0/1908037618 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff658100550 msgr2=0x7ff6580ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:00.802 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.774+0000 7ff65fe34700 1 --2- 172.21.15.39:0/1908037618 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff658100550 0x7ff6580ffec0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff65400d980 tx=0x7ff65400dc90 comp rx=0 tx=0).stop 2023-07-21T18:40:00.802 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.775+0000 7ff65fe34700 1 -- 172.21.15.39:0/1908037618 shutdown_connections 2023-07-21T18:40:00.802 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.775+0000 7ff65fe34700 1 --2- 172.21.15.39:0/1908037618 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7ff6440a11a0 0x7ff6440a3650 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:00.802 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.775+0000 7ff65fe34700 1 --2- 172.21.15.39:0/1908037618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff658105790 0x7ff6580fea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:00.802 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.775+0000 7ff65fe34700 1 --2- 172.21.15.39:0/1908037618 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff658100550 0x7ff6580ffec0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:00.802 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.775+0000 7ff65fe34700 1 --2- 172.21.15.39:0/1908037618 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff658102e70 0x7ff6580fe500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:00.803 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.775+0000 7ff65fe34700 1 -- 172.21.15.39:0/1908037618 >> 172.21.15.39:0/1908037618 conn(0x7ff6580f9fd0 msgr2=0x7ff6581011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:00.803 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.775+0000 7ff65fe34700 1 -- 172.21.15.39:0/1908037618 shutdown_connections 2023-07-21T18:40:00.803 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.775+0000 7ff65fe34700 1 -- 172.21.15.39:0/1908037618 wait complete. 2023-07-21T18:40:00.803 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 13 2023-07-21T18:40:00.814 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T18:40:00.992 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.965+0000 7f4aeb7d5700 1 -- 172.21.15.39:0/1033266415 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ae40fe9d0 msgr2=0x7f4ae40fede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:00.992 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.965+0000 7f4aeb7d5700 1 --2- 172.21.15.39:0/1033266415 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ae40fe9d0 0x7f4ae40fede0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4ad8009990 tx=0x7f4ad8009ca0 comp rx=0 tx=0).stop 2023-07-21T18:40:00.992 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.965+0000 7f4aeb7d5700 1 -- 172.21.15.39:0/1033266415 shutdown_connections 2023-07-21T18:40:00.992 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.965+0000 7f4aeb7d5700 1 --2- 172.21.15.39:0/1033266415 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4ae4100550 0x7f4ae4106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:00.993 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.965+0000 7f4aeb7d5700 1 --2- 172.21.15.39:0/1033266415 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ae40fe9d0 0x7f4ae40fede0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:00.993 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.965+0000 7f4aeb7d5700 1 --2- 172.21.15.39:0/1033266415 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ae40ffbc0 0x7f4ae4100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:00.993 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.965+0000 7f4aeb7d5700 1 -- 172.21.15.39:0/1033266415 >> 172.21.15.39:0/1033266415 conn(0x7f4ae40f9fd0 msgr2=0x7f4ae40fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:00.993 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.965+0000 7f4aeb7d5700 1 -- 172.21.15.39:0/1033266415 shutdown_connections 2023-07-21T18:40:00.993 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.965+0000 7f4aeb7d5700 1 -- 172.21.15.39:0/1033266415 wait complete. 2023-07-21T18:40:00.996 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.966+0000 7f4aeb7d5700 1 Processor -- start 2023-07-21T18:40:00.996 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.966+0000 7f4aeb7d5700 1 -- start start 2023-07-21T18:40:00.996 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.966+0000 7f4aeb7d5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ae40fe9d0 0x7f4ae4068200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:00.997 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.966+0000 7f4aeb7d5700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ae40ffbc0 0x7f4ae419a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:00.997 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.966+0000 7f4aeb7d5700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4ae4100550 0x7f4ae419cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:00.998 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.966+0000 7f4aeb7d5700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae419d280 con 0x7f4ae40fe9d0 2023-07-21T18:40:00.998 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.966+0000 7f4aeb7d5700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae419d3f0 con 0x7f4ae40ffbc0 2023-07-21T18:40:00.998 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.966+0000 7f4aeb7d5700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae419d5b0 con 0x7f4ae4100550 2023-07-21T18:40:00.998 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.966+0000 7f4ae9571700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ae40fe9d0 0x7f4ae4068200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:00.998 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.966+0000 7f4ae9571700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ae40fe9d0 0x7f4ae4068200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54928/0 (socket says 172.21.15.39:54928) 2023-07-21T18:40:00.999 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.966+0000 7f4ae9571700 1 -- 172.21.15.39:0/1377300033 learned_addr learned my addr 172.21.15.39:0/1377300033 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:40:00.999 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.966+0000 7f4ae9d72700 1 --2- 172.21.15.39:0/1377300033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4ae4100550 0x7f4ae419cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:00.999 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.966+0000 7f4ae9571700 1 -- 172.21.15.39:0/1377300033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4ae4100550 msgr2=0x7f4ae419cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:00.999 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.966+0000 7f4ae9571700 1 --2- 172.21.15.39:0/1377300033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4ae4100550 0x7f4ae419cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:00.999 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.966+0000 7f4ae9571700 1 -- 172.21.15.39:0/1377300033 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ae40ffbc0 msgr2=0x7f4ae419a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:01.000 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.966+0000 7f4ae8d70700 1 --2- 172.21.15.39:0/1377300033 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ae40ffbc0 0x7f4ae419a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:01.000 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.966+0000 7f4ae9571700 1 --2- 172.21.15.39:0/1377300033 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ae40ffbc0 0x7f4ae419a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:01.000 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.966+0000 7f4ae9571700 1 -- 172.21.15.39:0/1377300033 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ad80095f0 con 0x7f4ae40fe9d0 2023-07-21T18:40:01.000 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.966+0000 7f4ae8d70700 1 --2- 172.21.15.39:0/1377300033 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ae40ffbc0 0x7f4ae419a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:40:01.001 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.967+0000 7f4ae9571700 1 --2- 172.21.15.39:0/1377300033 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ae40fe9d0 0x7f4ae4068200 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4ad80059d0 tx=0x7f4ad8005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:01.001 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.967+0000 7f4ad67fc700 1 -- 172.21.15.39:0/1377300033 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad801d070 con 0x7f4ae40fe9d0 2023-07-21T18:40:01.001 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.967+0000 7f4ad67fc700 1 -- 172.21.15.39:0/1377300033 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ad80040e0 con 0x7f4ae40fe9d0 2023-07-21T18:40:01.001 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.967+0000 7f4aeb7d5700 1 -- 172.21.15.39:0/1377300033 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ae419d830 con 0x7f4ae40fe9d0 2023-07-21T18:40:01.001 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.967+0000 7f4ad67fc700 1 -- 172.21.15.39:0/1377300033 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad800f460 con 0x7f4ae40fe9d0 2023-07-21T18:40:01.001 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.967+0000 7f4aeb7d5700 1 -- 172.21.15.39:0/1377300033 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ae419dcb0 con 0x7f4ae40fe9d0 2023-07-21T18:40:01.001 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.968+0000 7f4aeb7d5700 1 -- 172.21.15.39:0/1377300033 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ae4105860 con 0x7f4ae40fe9d0 2023-07-21T18:40:01.002 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.969+0000 7f4ad67fc700 1 -- 172.21.15.39:0/1377300033 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f4ad8003c20 con 0x7f4ae40fe9d0 2023-07-21T18:40:01.002 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.969+0000 7f4ad67fc700 1 --2- 172.21.15.39:0/1377300033 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f4ad00a1220 0x7f4ad00a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:01.003 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.969+0000 7f4ad67fc700 1 -- 172.21.15.39:0/1377300033 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4ad80cdff0 con 0x7f4ae40fe9d0 2023-07-21T18:40:01.004 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.969+0000 7f4ae8d70700 1 --2- 172.21.15.39:0/1377300033 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f4ad00a1220 0x7f4ad00a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:01.004 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.970+0000 7f4ae8d70700 1 --2- 172.21.15.39:0/1377300033 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f4ad00a1220 0x7f4ad00a36d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0005fd0 tx=0x7f4ae0005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:01.004 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:00.973+0000 7f4ad67fc700 1 -- 172.21.15.39:0/1377300033 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f4ad8097c70 con 0x7f4ae40fe9d0 2023-07-21T18:40:01.203 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.176+0000 7f4aeb7d5700 1 -- 172.21.15.39:0/1377300033 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ae402caa0 con 0x7f4ae40fe9d0 2023-07-21T18:40:01.204 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.178+0000 7f4ad67fc700 1 -- 172.21.15.39:0/1377300033 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+18420 (secure 0 0 0) 0x7f4ad809b290 con 0x7f4ae40fe9d0 2023-07-21T18:40:01.205 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:40:01.205 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":75,"fsid":"eaf8e25a-27f4-11ee-9b34-001a4aab830c","created":"2023-07-21T18:33:06.504979+0000","modified":"2023-07-21T18:39:24.539865+0000","last_up_change":"2023-07-21T18:39:23.531009+0000","last_in_change":"2023-07-21T18:39:05.993024+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T18:36:13.572583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03502c07-2d0f-4723-8d0b-d34c9f5b91bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6803","nonce":860193857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6805","nonce":860193857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6809","nonce":860193857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6807","nonce":860193857}]},"public_addr":"172.21.15.39:6803/860193857","cluster_addr":"172.21.15.39:6805/860193857","heartbeat_back_addr":"172.21.15.39:6809/860193857","heartbeat_front_addr":"172.21.15.39:6807/860193857","state":["exists","up"]},{"osd":1,"uuid":"0831a0c0-a1fa-4d30-9a28-513223aa69eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6801","nonce":3246721764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6803","nonce":3246721764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6807","nonce":3246721764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6805","nonce":3246721764}]},"public_addr":"172.21.15.155:6801/3246721764","cluster_addr":"172.21.15.155:6803/3246721764","heartbeat_back_addr":"172.21.15.155:6807/3246721764","heartbeat_front_addr":"172.21.15.155:6805/3246721764","state":["exists","up"]},{"osd":2,"uuid":"22df0003-bde9-48e4-b47a-cb8eafbab0db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6801","nonce":2563594449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6803","nonce":2563594449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6807","nonce":2563594449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6805","nonce":2563594449}]},"public_addr":"172.21.15.193:6801/2563594449","cluster_addr":"172.21.15.193:6803/2563594449","heartbeat_back_addr":"172.21.15.193:6807/2563594449","heartbeat_front_addr":"172.21.15.193:6805/2563594449","state":["exists","up"]},{"osd":3,"uuid":"54562558-a2e8-45dd-bc95-561569b9ae6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6811","nonce":863708386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6813","nonce":863708386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6817","nonce":863708386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6815","nonce":863708386}]},"public_addr":"172.21.15.39:6811/863708386","cluster_addr":"172.21.15.39:6813/863708386","heartbeat_back_addr":"172.21.15.39:6817/863708386","heartbeat_front_addr":"172.21.15.39:6815/863708386","state":["exists","up"]},{"osd":4,"uuid":"491118e7-ae1e-495c-9eac-ef1cdcae9600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6809","nonce":3361780673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6811","nonce":3361780673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6815","nonce":3361780673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6813","nonce":3361780673}]},"public_addr":"172.21.15.155:6809/3361780673","cluster_addr":"172.21.15.155:6811/3361780673","heartbeat_back_addr":"172.21.15.155:6815/3361780673","heartbeat_front_addr":"172.21.15.155:6813/3361780673","state":["exists","up"]},{"osd":5,"uuid":"d267bc2c-42b3-4d96-b2d1-b186dc7e6b18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6809","nonce":2376867721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6811","nonce":2376867721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6815","nonce":2376867721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6813","nonce":2376867721}]},"public_addr":"172.21.15.193:6809/2376867721","cluster_addr":"172.21.15.193:6811/2376867721","heartbeat_back_addr":"172.21.15.193:6815/2376867721","heartbeat_front_addr":"172.21.15.193:6813/2376867721","state":["exists","up"]},{"osd":6,"uuid":"beec8088-970a-473e-bd67-6eee7cbdef24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6819","nonce":2417770529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6821","nonce":2417770529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6825","nonce":2417770529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6823","nonce":2417770529}]},"public_addr":"172.21.15.39:6819/2417770529","cluster_addr":"172.21.15.39:6821/2417770529","heartbeat_back_addr":"172.21.15.39:6825/2417770529","heartbeat_front_addr":"172.21.15.39:6823/2417770529","state":["exists","up"]},{"osd":7,"uuid":"5a88ce30-1835-4254-90cc-804ed5cf1066","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6817","nonce":2542002111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6819","nonce":2542002111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6823","nonce":2542002111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6821","nonce":2542002111}]},"public_addr":"172.21.15.155:6817/2542002111","cluster_addr":"172.21.15.155:6819/2542002111","heartbeat_back_addr":"172.21.15.155:6823/2542002111","heartbeat_front_addr":"172.21.15.155:6821/2542002111","state":["exists","up"]},{"osd":8,"uuid":"d964a183-3021-4ace-9a0c-aad00f2570e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6817","nonce":2985835085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6819","nonce":2985835085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6823","nonce":2985835085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6821","nonce":2985835085}]},"public_addr":"172.21.15.193:6817/2985835085","cluster_addr":"172.21.15.193:6819/2985835085","heartbeat_back_addr":"172.21.15.193:6823/2985835085","heartbeat_front_addr":"172.21.15.193:6821/2985835085","state":["exists","up"]},{"osd":9,"uuid":"9725ea5c-49a3-4c99-bac4-e95069100b1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6827","nonce":1946493651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6829","nonce":1946493651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6833","nonce":1946493651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6831","nonce":1946493651}]},"public_addr":"172.21.15.39:6827/1946493651","cluster_addr":"172.21.15.39:6829/1946493651","heartbeat_back_addr":"172.21.15.39:6833/1946493651","heartbeat_front_addr":"172.21.15.39:6831/1946493651","state":["exists","up"]},{"osd":10,"uuid":"1fd60ccb-92db-45f0-a580-2003db0976e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6825","nonce":1234615022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6827","nonce":1234615022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6831","nonce":1234615022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6829","nonce":1234615022}]},"public_addr":"172.21.15.155:6825/1234615022","cluster_addr":"172.21.15.155:6827/1234615022","heartbeat_back_addr":"172.21.15.155:6831/1234615022","heartbeat_front_addr":"172.21.15.155:6829/1234615022","state":["exists","up"]},{"osd":11,"uuid":"efb31ea1-4775-4e5f-8740-728d6f4aeea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6825","nonce":215104663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6827","nonce":215104663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6831","nonce":215104663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6829","nonce":215104663}]},"public_addr":"172.21.15.193:6825/215104663","cluster_addr":"172.21.15.193:6827/215104663","heartbeat_back_addr":"172.21.15.193:6831/215104663","heartbeat_front_addr":"172.21.15.193:6829/215104663","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:35:30.211698+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:35:50.883846+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:36:11.124168+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:36:32.711905+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:36:54.442007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:37:14.843526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:37:36.587354+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:37:57.882374+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:38:18.729407+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:38:40.139231+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:39:01.290862+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:39:21.767537+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:0/2902600414":"2023-07-22T18:34:07.039823+0000","172.21.15.39:6801/2893866425":"2023-07-22T18:34:07.039823+0000","172.21.15.39:0/19717002":"2023-07-22T18:34:07.039823+0000","172.21.15.39:6800/2893866425":"2023-07-22T18:34:07.039823+0000","172.21.15.39:6800/889516631":"2023-07-22T18:33:40.852706+0000","172.21.15.39:6801/889516631":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/3636510379":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/1275748831":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/3819152602":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/1127767082":"2023-07-22T18:34:07.039823+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T18:40:01.208 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.181+0000 7f4aeb7d5700 1 -- 172.21.15.39:0/1377300033 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f4ad00a1220 msgr2=0x7f4ad00a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:01.208 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.181+0000 7f4aeb7d5700 1 --2- 172.21.15.39:0/1377300033 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f4ad00a1220 0x7f4ad00a36d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0005fd0 tx=0x7f4ae0005dc0 comp rx=0 tx=0).stop 2023-07-21T18:40:01.208 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.181+0000 7f4aeb7d5700 1 -- 172.21.15.39:0/1377300033 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ae40fe9d0 msgr2=0x7f4ae4068200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:01.208 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.181+0000 7f4aeb7d5700 1 --2- 172.21.15.39:0/1377300033 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ae40fe9d0 0x7f4ae4068200 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4ad80059d0 tx=0x7f4ad8005740 comp rx=0 tx=0).stop 2023-07-21T18:40:01.209 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.181+0000 7f4aeb7d5700 1 -- 172.21.15.39:0/1377300033 shutdown_connections 2023-07-21T18:40:01.209 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.181+0000 7f4aeb7d5700 1 --2- 172.21.15.39:0/1377300033 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f4ad00a1220 0x7f4ad00a36d0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:01.209 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.181+0000 7f4aeb7d5700 1 --2- 172.21.15.39:0/1377300033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4ae4100550 0x7f4ae419cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:01.209 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.181+0000 7f4aeb7d5700 1 --2- 172.21.15.39:0/1377300033 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ae40fe9d0 0x7f4ae4068200 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:01.209 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.181+0000 7f4aeb7d5700 1 --2- 172.21.15.39:0/1377300033 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ae40ffbc0 0x7f4ae419a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:01.209 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.181+0000 7f4aeb7d5700 1 -- 172.21.15.39:0/1377300033 >> 172.21.15.39:0/1377300033 conn(0x7f4ae40f9fd0 msgr2=0x7f4ae40fc190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:01.209 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.181+0000 7f4aeb7d5700 1 -- 172.21.15.39:0/1377300033 shutdown_connections 2023-07-21T18:40:01.210 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.181+0000 7f4aeb7d5700 1 -- 172.21.15.39:0/1377300033 wait complete. 2023-07-21T18:40:01.222 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-21T18:36:13.572583+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-21T18:40:01.223 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-21T18:40:01.360 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.331+0000 7f8078c38700 1 -- 172.21.15.39:0/4098601133 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80740fe210 msgr2=0x7f80740fe620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:01.360 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.331+0000 7f8078c38700 1 --2- 172.21.15.39:0/4098601133 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80740fe210 0x7f80740fe620 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f806400b0a0 tx=0x7f806400b3b0 comp rx=0 tx=0).stop 2023-07-21T18:40:01.360 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.331+0000 7f8078c38700 1 -- 172.21.15.39:0/4098601133 shutdown_connections 2023-07-21T18:40:01.361 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.331+0000 7f8078c38700 1 --2- 172.21.15.39:0/4098601133 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80740ffd90 0x7f8074106650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:01.361 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.331+0000 7f8078c38700 1 --2- 172.21.15.39:0/4098601133 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80740fe210 0x7f80740fe620 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:01.361 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.331+0000 7f8078c38700 1 --2- 172.21.15.39:0/4098601133 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f80740ff400 0x7f80740ff850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:01.362 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.331+0000 7f8078c38700 1 -- 172.21.15.39:0/4098601133 >> 172.21.15.39:0/4098601133 conn(0x7f80740f9850 msgr2=0x7f80740fbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:01.362 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.331+0000 7f8078c38700 1 -- 172.21.15.39:0/4098601133 shutdown_connections 2023-07-21T18:40:01.362 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.331+0000 7f8078c38700 1 -- 172.21.15.39:0/4098601133 wait complete. 2023-07-21T18:40:01.362 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.332+0000 7f8078c38700 1 Processor -- start 2023-07-21T18:40:01.363 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.332+0000 7f8078c38700 1 -- start start 2023-07-21T18:40:01.363 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.332+0000 7f8078c38700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80740fe210 0x7f8074107af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:01.363 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.332+0000 7f8078c38700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f80740ff400 0x7f807419cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:01.364 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.332+0000 7f8078c38700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80740ffd90 0x7f807419f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:01.364 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.332+0000 7f8078c38700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f807419f740 con 0x7f80740fe210 2023-07-21T18:40:01.364 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.332+0000 7f8078c38700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f807419f8b0 con 0x7f80740ffd90 2023-07-21T18:40:01.365 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.332+0000 7f8078c38700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f807419fa70 con 0x7f80740ff400 2023-07-21T18:40:01.365 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.332+0000 7f807259c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80740fe210 0x7f8074107af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:01.365 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.332+0000 7f8071d9b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f80740ff400 0x7f807419cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:01.366 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.332+0000 7f807259c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80740fe210 0x7f8074107af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54942/0 (socket says 172.21.15.39:54942) 2023-07-21T18:40:01.366 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.332+0000 7f807259c700 1 -- 172.21.15.39:0/3493895650 learned_addr learned my addr 172.21.15.39:0/3493895650 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:40:01.366 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.332+0000 7f8072d9d700 1 --2- 172.21.15.39:0/3493895650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80740ffd90 0x7f807419f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:01.367 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.332+0000 7f8071d9b700 1 -- 172.21.15.39:0/3493895650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80740ffd90 msgr2=0x7f807419f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:01.367 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.332+0000 7f8071d9b700 1 --2- 172.21.15.39:0/3493895650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80740ffd90 0x7f807419f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:01.367 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.332+0000 7f8071d9b700 1 -- 172.21.15.39:0/3493895650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80740fe210 msgr2=0x7f8074107af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:01.368 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.332+0000 7f8071d9b700 1 --2- 172.21.15.39:0/3493895650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80740fe210 0x7f8074107af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:01.368 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.332+0000 7f8071d9b700 1 -- 172.21.15.39:0/3493895650 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8064009d00 con 0x7f80740ff400 2023-07-21T18:40:01.368 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.332+0000 7f8072d9d700 1 --2- 172.21.15.39:0/3493895650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80740ffd90 0x7f807419f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:40:01.368 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.332+0000 7f807259c700 1 --2- 172.21.15.39:0/3493895650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80740fe210 0x7f8074107af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:40:01.369 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.332+0000 7f8071d9b700 1 --2- 172.21.15.39:0/3493895650 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f80740ff400 0x7f807419cb40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f805c00e9f0 tx=0x7f805c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:01.369 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.333+0000 7f806b7fe700 1 -- 172.21.15.39:0/3493895650 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f805c00eee0 con 0x7f80740ff400 2023-07-21T18:40:01.369 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.333+0000 7f8078c38700 1 -- 172.21.15.39:0/3493895650 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f807419fd50 con 0x7f80740ff400 2023-07-21T18:40:01.369 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.333+0000 7f806b7fe700 1 -- 172.21.15.39:0/3493895650 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f805c00cbd0 con 0x7f80740ff400 2023-07-21T18:40:01.370 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.333+0000 7f806b7fe700 1 -- 172.21.15.39:0/3493895650 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f805c018680 con 0x7f80740ff400 2023-07-21T18:40:01.370 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.333+0000 7f8078c38700 1 -- 172.21.15.39:0/3493895650 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80741a0200 con 0x7f80740ff400 2023-07-21T18:40:01.371 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.334+0000 7f8078c38700 1 -- 172.21.15.39:0/3493895650 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f807410a2e0 con 0x7f80740ff400 2023-07-21T18:40:01.372 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.335+0000 7f806b7fe700 1 -- 172.21.15.39:0/3493895650 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f805c00cd40 con 0x7f80740ff400 2023-07-21T18:40:01.372 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.336+0000 7f806b7fe700 1 --2- 172.21.15.39:0/3493895650 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f80600a1150 0x7f80600a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:01.373 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.336+0000 7f806b7fe700 1 -- 172.21.15.39:0/3493895650 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f805c014070 con 0x7f80740ff400 2023-07-21T18:40:01.373 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.336+0000 7f807259c700 1 --2- 172.21.15.39:0/3493895650 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f80600a1150 0x7f80600a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:01.373 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.336+0000 7f807259c700 1 --2- 172.21.15.39:0/3493895650 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f80600a1150 0x7f80600a3600 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f80640079f0 tx=0x7f806400b700 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:01.373 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.339+0000 7f806b7fe700 1 -- 172.21.15.39:0/3493895650 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f805c096280 con 0x7f80740ff400 2023-07-21T18:40:01.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:01 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/1908037618' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:40:01.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:01 smithi039 ceph-mon[96489]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:40:01.567 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.540+0000 7f8078c38700 1 -- 172.21.15.39:0/3493895650 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8074105160 con 0x7f80740ff400 2023-07-21T18:40:01.568 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.541+0000 7f806b7fe700 1 -- 172.21.15.39:0/3493895650 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) v1 ==== 93+0+10 (secure 0 0 0) 0x7f805c0998a0 con 0x7f80740ff400 2023-07-21T18:40:01.568 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2023-07-21T18:40:01.570 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.543+0000 7f8078c38700 1 -- 172.21.15.39:0/3493895650 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f80600a1150 msgr2=0x7f80600a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:01.570 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.543+0000 7f8078c38700 1 --2- 172.21.15.39:0/3493895650 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f80600a1150 0x7f80600a3600 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f80640079f0 tx=0x7f806400b700 comp rx=0 tx=0).stop 2023-07-21T18:40:01.570 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.543+0000 7f8078c38700 1 -- 172.21.15.39:0/3493895650 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f80740ff400 msgr2=0x7f807419cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:01.571 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.543+0000 7f8078c38700 1 --2- 172.21.15.39:0/3493895650 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f80740ff400 0x7f807419cb40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f805c00e9f0 tx=0x7f805c00ed00 comp rx=0 tx=0).stop 2023-07-21T18:40:01.571 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.543+0000 7f8078c38700 1 -- 172.21.15.39:0/3493895650 shutdown_connections 2023-07-21T18:40:01.571 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.543+0000 7f8078c38700 1 --2- 172.21.15.39:0/3493895650 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f80600a1150 0x7f80600a3600 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:01.571 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.543+0000 7f8078c38700 1 --2- 172.21.15.39:0/3493895650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80740ffd90 0x7f807419f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:01.571 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.543+0000 7f8078c38700 1 --2- 172.21.15.39:0/3493895650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80740fe210 0x7f8074107af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:01.571 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.543+0000 7f8078c38700 1 --2- 172.21.15.39:0/3493895650 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f80740ff400 0x7f807419cb40 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:01.571 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.543+0000 7f8078c38700 1 -- 172.21.15.39:0/3493895650 >> 172.21.15.39:0/3493895650 conn(0x7f80740f9850 msgr2=0x7f80740fbb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:01.572 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.543+0000 7f8078c38700 1 -- 172.21.15.39:0/3493895650 shutdown_connections 2023-07-21T18:40:01.572 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.543+0000 7f8078c38700 1 -- 172.21.15.39:0/3493895650 wait complete. 2023-07-21T18:40:01.583 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-07-21T18:40:01.623 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:01 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/1908037618' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:40:01.624 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:01 smithi193 ceph-mon[98204]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:40:01.642 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:01 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/1908037618' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:40:01.642 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:01 smithi155 ceph-mon[97934]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:40:01.715 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.685+0000 7f3f38d8c700 1 -- 172.21.15.39:0/3979944257 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f340fed10 msgr2=0x7f3f340ff180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:01.715 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.685+0000 7f3f38d8c700 1 --2- 172.21.15.39:0/3979944257 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f340fed10 0x7f3f340ff180 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3f28009990 tx=0x7f3f28009ca0 comp rx=0 tx=0).stop 2023-07-21T18:40:01.716 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.685+0000 7f3f38d8c700 1 -- 172.21.15.39:0/3979944257 shutdown_connections 2023-07-21T18:40:01.716 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.685+0000 7f3f38d8c700 1 --2- 172.21.15.39:0/3979944257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f340fe3c0 0x7f3f340fe7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:01.716 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.685+0000 7f3f38d8c700 1 --2- 172.21.15.39:0/3979944257 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f340ff6c0 0x7f3f340ffb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:01.716 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.685+0000 7f3f38d8c700 1 --2- 172.21.15.39:0/3979944257 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f340fed10 0x7f3f340ff180 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:01.717 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.685+0000 7f3f38d8c700 1 -- 172.21.15.39:0/3979944257 >> 172.21.15.39:0/3979944257 conn(0x7f3f340f9fd0 msgr2=0x7f3f340fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:01.717 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.685+0000 7f3f38d8c700 1 -- 172.21.15.39:0/3979944257 shutdown_connections 2023-07-21T18:40:01.717 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.685+0000 7f3f38d8c700 1 -- 172.21.15.39:0/3979944257 wait complete. 2023-07-21T18:40:01.717 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.685+0000 7f3f38d8c700 1 Processor -- start 2023-07-21T18:40:01.718 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.686+0000 7f3f38d8c700 1 -- start start 2023-07-21T18:40:01.719 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.686+0000 7f3f38d8c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f340fe3c0 0x7f3f341979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:01.719 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.686+0000 7f3f38d8c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f340fed10 0x7f3f34197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:01.719 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.686+0000 7f3f38d8c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f340ff6c0 0x7f3f3419ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:01.720 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.686+0000 7f3f38d8c700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3419f540 con 0x7f3f340ff6c0 2023-07-21T18:40:01.720 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.686+0000 7f3f38d8c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3419f680 con 0x7f3f340fe3c0 2023-07-21T18:40:01.720 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.686+0000 7f3f38d8c700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3419f810 con 0x7f3f340fed10 2023-07-21T18:40:01.720 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.686+0000 7f3f32d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f340ff6c0 0x7f3f3419ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:01.720 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.686+0000 7f3f3259c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f340fe3c0 0x7f3f341979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:01.720 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.686+0000 7f3f32d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f340ff6c0 0x7f3f3419ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54960/0 (socket says 172.21.15.39:54960) 2023-07-21T18:40:01.720 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.686+0000 7f3f32d9d700 1 -- 172.21.15.39:0/1253946947 learned_addr learned my addr 172.21.15.39:0/1253946947 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:40:01.721 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.686+0000 7f3f31d9b700 1 --2- 172.21.15.39:0/1253946947 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f340fed10 0x7f3f34197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:01.721 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.686+0000 7f3f3259c700 1 -- 172.21.15.39:0/1253946947 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f340fed10 msgr2=0x7f3f34197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:01.721 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.686+0000 7f3f3259c700 1 --2- 172.21.15.39:0/1253946947 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f340fed10 0x7f3f34197f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:01.721 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.686+0000 7f3f3259c700 1 -- 172.21.15.39:0/1253946947 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f340ff6c0 msgr2=0x7f3f3419ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:01.721 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.686+0000 7f3f3259c700 1 --2- 172.21.15.39:0/1253946947 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f340ff6c0 0x7f3f3419ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:01.721 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.686+0000 7f3f3259c700 1 -- 172.21.15.39:0/1253946947 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f24009520 con 0x7f3f340fe3c0 2023-07-21T18:40:01.721 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.686+0000 7f3f3259c700 1 --2- 172.21.15.39:0/1253946947 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f340fe3c0 0x7f3f341979c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3f2400e9f0 tx=0x7f3f2400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:01.722 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.687+0000 7f3f237fe700 1 -- 172.21.15.39:0/1253946947 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f2400cb70 con 0x7f3f340fe3c0 2023-07-21T18:40:01.722 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.687+0000 7f3f237fe700 1 -- 172.21.15.39:0/1253946947 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f2400ee90 con 0x7f3f340fe3c0 2023-07-21T18:40:01.722 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.687+0000 7f3f38d8c700 1 -- 172.21.15.39:0/1253946947 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f280095f0 con 0x7f3f340fe3c0 2023-07-21T18:40:01.722 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.687+0000 7f3f237fe700 1 -- 172.21.15.39:0/1253946947 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f24003cf0 con 0x7f3f340fe3c0 2023-07-21T18:40:01.722 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.687+0000 7f3f38d8c700 1 -- 172.21.15.39:0/1253946947 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f3419fe70 con 0x7f3f340fe3c0 2023-07-21T18:40:01.722 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.687+0000 7f3f38d8c700 1 -- 172.21.15.39:0/1253946947 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f341091a0 con 0x7f3f340fe3c0 2023-07-21T18:40:01.723 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.689+0000 7f3f237fe700 1 -- 172.21.15.39:0/1253946947 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f3f2400ccd0 con 0x7f3f340fe3c0 2023-07-21T18:40:01.723 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.689+0000 7f3f237fe700 1 --2- 172.21.15.39:0/1253946947 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f3f1c0a11a0 0x7f3f1c0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:01.723 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.689+0000 7f3f237fe700 1 -- 172.21.15.39:0/1253946947 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f3f24014070 con 0x7f3f340fe3c0 2023-07-21T18:40:01.724 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.689+0000 7f3f31d9b700 1 --2- 172.21.15.39:0/1253946947 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f3f1c0a11a0 0x7f3f1c0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:01.724 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.690+0000 7f3f31d9b700 1 --2- 172.21.15.39:0/1253946947 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f3f1c0a11a0 0x7f3f1c0a3650 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3f2800b2e0 tx=0x7f3f28005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:01.724 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.693+0000 7f3f237fe700 1 -- 172.21.15.39:0/1253946947 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f3f24096920 con 0x7f3f340fe3c0 2023-07-21T18:40:01.944 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.918+0000 7f3f38d8c700 1 -- 172.21.15.39:0/1253946947 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f3f3402c940 con 0x7f3f340fe3c0 2023-07-21T18:40:01.945 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.918+0000 7f3f237fe700 1 -- 172.21.15.39:0/1253946947 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v75) v1 ==== 84+0+65 (secure 0 0 0) 0x7f3f24099f40 con 0x7f3f340fe3c0 2023-07-21T18:40:01.945 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:40:01.945 INFO:teuthology.orchestra.run.smithi039.stdout:[ 2023-07-21T18:40:01.945 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T18:40:01.946 INFO:teuthology.orchestra.run.smithi039.stdout: "poolnum": 1, 2023-07-21T18:40:01.946 INFO:teuthology.orchestra.run.smithi039.stdout: "poolname": ".mgr" 2023-07-21T18:40:01.946 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T18:40:01.946 INFO:teuthology.orchestra.run.smithi039.stdout:] 2023-07-21T18:40:01.947 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.920+0000 7f3f38d8c700 1 -- 172.21.15.39:0/1253946947 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f3f1c0a11a0 msgr2=0x7f3f1c0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:01.947 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.920+0000 7f3f38d8c700 1 --2- 172.21.15.39:0/1253946947 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f3f1c0a11a0 0x7f3f1c0a3650 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3f2800b2e0 tx=0x7f3f28005fb0 comp rx=0 tx=0).stop 2023-07-21T18:40:01.948 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.920+0000 7f3f38d8c700 1 -- 172.21.15.39:0/1253946947 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f340fe3c0 msgr2=0x7f3f341979c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:01.948 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.920+0000 7f3f38d8c700 1 --2- 172.21.15.39:0/1253946947 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f340fe3c0 0x7f3f341979c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3f2400e9f0 tx=0x7f3f2400ed00 comp rx=0 tx=0).stop 2023-07-21T18:40:01.948 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.920+0000 7f3f38d8c700 1 -- 172.21.15.39:0/1253946947 shutdown_connections 2023-07-21T18:40:01.948 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.920+0000 7f3f38d8c700 1 --2- 172.21.15.39:0/1253946947 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f3f1c0a11a0 0x7f3f1c0a3650 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:01.948 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.920+0000 7f3f38d8c700 1 --2- 172.21.15.39:0/1253946947 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f340fe3c0 0x7f3f341979c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:01.948 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.920+0000 7f3f38d8c700 1 --2- 172.21.15.39:0/1253946947 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f340ff6c0 0x7f3f3419ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:01.949 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.920+0000 7f3f38d8c700 1 --2- 172.21.15.39:0/1253946947 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f340fed10 0x7f3f34197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:01.949 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.921+0000 7f3f38d8c700 1 -- 172.21.15.39:0/1253946947 >> 172.21.15.39:0/1253946947 conn(0x7f3f340f9fd0 msgr2=0x7f3f340fbc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:01.949 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.921+0000 7f3f38d8c700 1 -- 172.21.15.39:0/1253946947 shutdown_connections 2023-07-21T18:40:01.949 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:01.921+0000 7f3f38d8c700 1 -- 172.21.15.39:0/1253946947 wait complete. 2023-07-21T18:40:01.960 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-07-21T18:40:01.960 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-07-21T18:40:02.096 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:02.067+0000 7f44d8bd3700 1 -- 172.21.15.39:0/1687024283 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44d4100560 msgr2=0x7f44d4106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:02.096 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:02.067+0000 7f44d8bd3700 1 --2- 172.21.15.39:0/1687024283 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44d4100560 0x7f44d4106e20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f44cc00b0a0 tx=0x7f44cc00b3b0 comp rx=0 tx=0).stop 2023-07-21T18:40:02.096 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:02.067+0000 7f44d8bd3700 1 -- 172.21.15.39:0/1687024283 shutdown_connections 2023-07-21T18:40:02.097 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:02.067+0000 7f44d8bd3700 1 --2- 172.21.15.39:0/1687024283 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44d4100560 0x7f44d4106e20 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:02.097 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:02.067+0000 7f44d8bd3700 1 --2- 172.21.15.39:0/1687024283 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44d40fe9e0 0x7f44d40fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:02.097 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:02.067+0000 7f44d8bd3700 1 --2- 172.21.15.39:0/1687024283 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44d40ffbd0 0x7f44d4100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:02.097 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:02.067+0000 7f44d8bd3700 1 -- 172.21.15.39:0/1687024283 >> 172.21.15.39:0/1687024283 conn(0x7f44d40fa000 msgr2=0x7f44d40fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:02.097 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:02.067+0000 7f44d8bd3700 1 -- 172.21.15.39:0/1687024283 shutdown_connections 2023-07-21T18:40:02.097 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:02.067+0000 7f44d8bd3700 1 -- 172.21.15.39:0/1687024283 wait complete. 2023-07-21T18:40:02.097 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:02.068+0000 7f44d8bd3700 1 Processor -- start 2023-07-21T18:40:02.097 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:02.068+0000 7f44d8bd3700 1 -- start start 2023-07-21T18:40:02.098 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:02.068+0000 7f44d8bd3700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44d40fe9e0 0x7f44d4068200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:02.098 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:02.068+0000 7f44d8bd3700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44d40ffbd0 0x7f44d419a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:02.098 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:02.068+0000 7f44d8bd3700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44d4100560 0x7f44d419cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:02.098 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:02.068+0000 7f44d8bd3700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d419d4e0 con 0x7f44d40fe9e0 2023-07-21T18:40:02.098 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:02.068+0000 7f44d8bd3700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d419d650 con 0x7f44d40ffbd0 2023-07-21T18:40:02.098 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:02.068+0000 7f44d8bd3700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d419d810 con 0x7f44d4100560 2023-07-21T18:40:02.098 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:02.068+0000 7f44d259c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44d40fe9e0 0x7f44d4068200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:02.099 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:02.068+0000 7f44d259c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44d40fe9e0 0x7f44d4068200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54970/0 (socket says 172.21.15.39:54970) 2023-07-21T18:40:02.099 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:02.068+0000 7f44d259c700 1 -- 172.21.15.39:0/2424580562 learned_addr learned my addr 172.21.15.39:0/2424580562 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:40:02.099 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:02.068+0000 7f44d1d9b700 1 --2- 172.21.15.39:0/2424580562 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44d40ffbd0 0x7f44d419a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:02.099 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:02.068+0000 7f44d2d9d700 1 --2- 172.21.15.39:0/2424580562 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44d4100560 0x7f44d419cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:02.099 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:02.068+0000 7f44d259c700 1 -- 172.21.15.39:0/2424580562 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44d4100560 msgr2=0x7f44d419cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:02.099 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:02.068+0000 7f44d259c700 1 --2- 172.21.15.39:0/2424580562 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44d4100560 0x7f44d419cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:02.099 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:02.068+0000 7f44d259c700 1 -- 172.21.15.39:0/2424580562 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44d40ffbd0 msgr2=0x7f44d419a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:02.099 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:02.068+0000 7f44d259c700 1 --2- 172.21.15.39:0/2424580562 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44d40ffbd0 0x7f44d419a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:02.100 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:02.068+0000 7f44d259c700 1 -- 172.21.15.39:0/2424580562 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44c4009520 con 0x7f44d40fe9e0 2023-07-21T18:40:02.100 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:02.068+0000 7f44d2d9d700 1 --2- 172.21.15.39:0/2424580562 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44d4100560 0x7f44d419cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:40:02.100 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:02.069+0000 7f44d259c700 1 --2- 172.21.15.39:0/2424580562 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44d40fe9e0 0x7f44d4068200 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f44bc00e970 tx=0x7f44bc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:02.100 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:02.069+0000 7f44cb7fe700 1 -- 172.21.15.39:0/2424580562 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44bc009790 con 0x7f44d40fe9e0 2023-07-21T18:40:02.100 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:02.069+0000 7f44cb7fe700 1 -- 172.21.15.39:0/2424580562 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f44bc00eec0 con 0x7f44d40fe9e0 2023-07-21T18:40:02.100 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:02.069+0000 7f44d8bd3700 1 -- 172.21.15.39:0/2424580562 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44cc009d00 con 0x7f44d40fe9e0 2023-07-21T18:40:02.100 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:02.069+0000 7f44cb7fe700 1 -- 172.21.15.39:0/2424580562 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44bc003cf0 con 0x7f44d40fe9e0 2023-07-21T18:40:02.101 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:02.069+0000 7f44d8bd3700 1 -- 172.21.15.39:0/2424580562 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44d419de70 con 0x7f44d40fe9e0 2023-07-21T18:40:02.101 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:02.070+0000 7f44d8bd3700 1 -- 172.21.15.39:0/2424580562 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44d410a210 con 0x7f44d40fe9e0 2023-07-21T18:40:02.102 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:02.072+0000 7f44cb7fe700 1 -- 172.21.15.39:0/2424580562 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f44bc00ccd0 con 0x7f44d40fe9e0 2023-07-21T18:40:02.102 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:02.072+0000 7f44cb7fe700 1 --2- 172.21.15.39:0/2424580562 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f44c00a1150 0x7f44c00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:02.102 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:02.073+0000 7f44cb7fe700 1 -- 172.21.15.39:0/2424580562 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f44bc014070 con 0x7f44d40fe9e0 2023-07-21T18:40:02.102 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:02.073+0000 7f44d1d9b700 1 --2- 172.21.15.39:0/2424580562 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f44c00a1150 0x7f44c00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:02.102 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:02.073+0000 7f44d1d9b700 1 --2- 172.21.15.39:0/2424580562 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f44c00a1150 0x7f44c00a3600 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f44d4067f70 tx=0x7f44c4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:02.103 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:02.075+0000 7f44cb7fe700 1 -- 172.21.15.39:0/2424580562 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f44bc0963f0 con 0x7f44d40fe9e0 2023-07-21T18:40:02.307 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:02.281+0000 7f44d8bd3700 1 -- 172.21.15.39:0/2424580562 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f44d410a400 con 0x7f44d40fe9e0 2023-07-21T18:40:02.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:02 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/1377300033' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T18:40:02.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:02 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:40:02.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:02 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:40:02.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:02 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:40:02.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:02 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:40:02.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:02 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:40:02.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:02 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:40:02.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:02 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/3493895650' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T18:40:02.443 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:02 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/1253946947' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-07-21T18:40:02.443 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:02 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:40:02.443 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:02 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:40:02.443 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:02 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:40:02.443 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:02 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:40:02.443 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:02 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:40:02.624 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:02 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/1377300033' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T18:40:02.624 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:02 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:40:02.624 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:02 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:40:02.624 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:02 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:40:02.624 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:02 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:40:02.624 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:02 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:40:02.625 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:02 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:40:02.625 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:02 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/3493895650' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T18:40:02.625 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:02 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/1253946947' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-07-21T18:40:02.625 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:02 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:40:02.625 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:02 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:40:02.625 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:02 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:40:02.625 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:02 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:40:02.626 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:02 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:40:02.642 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:02 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/1377300033' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T18:40:02.642 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:02 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:40:02.643 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:02 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:40:02.643 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:02 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:40:02.643 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:02 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:40:02.643 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:02 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:40:02.643 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:02 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:40:02.643 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:02 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/3493895650' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T18:40:02.643 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:02 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/1253946947' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-07-21T18:40:02.644 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:02 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:40:02.644 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:02 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:40:02.644 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:02 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:40:02.644 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:02 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:40:02.644 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:02 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:40:03.192 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:03.161+0000 7f44cb7fe700 1 -- 172.21.15.39:0/2424580562 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v76) v1 ==== 138+0+0 (secure 0 0 0) 0x7f44bc099a10 con 0x7f44d40fe9e0 2023-07-21T18:40:03.194 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:03.168+0000 7f44d8bd3700 1 -- 172.21.15.39:0/2424580562 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f44c00a1150 msgr2=0x7f44c00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:03.194 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:03.168+0000 7f44d8bd3700 1 --2- 172.21.15.39:0/2424580562 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f44c00a1150 0x7f44c00a3600 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f44d4067f70 tx=0x7f44c4009260 comp rx=0 tx=0).stop 2023-07-21T18:40:03.195 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:03.168+0000 7f44d8bd3700 1 -- 172.21.15.39:0/2424580562 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44d40fe9e0 msgr2=0x7f44d4068200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:03.195 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:03.168+0000 7f44d8bd3700 1 --2- 172.21.15.39:0/2424580562 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44d40fe9e0 0x7f44d4068200 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f44bc00e970 tx=0x7f44bc00ec80 comp rx=0 tx=0).stop 2023-07-21T18:40:03.195 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:03.168+0000 7f44d8bd3700 1 -- 172.21.15.39:0/2424580562 shutdown_connections 2023-07-21T18:40:03.195 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:03.168+0000 7f44d8bd3700 1 --2- 172.21.15.39:0/2424580562 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f44c00a1150 0x7f44c00a3600 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:03.195 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:03.168+0000 7f44d8bd3700 1 --2- 172.21.15.39:0/2424580562 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44d4100560 0x7f44d419cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:03.195 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:03.168+0000 7f44d8bd3700 1 --2- 172.21.15.39:0/2424580562 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44d40fe9e0 0x7f44d4068200 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:03.195 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:03.168+0000 7f44d8bd3700 1 --2- 172.21.15.39:0/2424580562 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44d40ffbd0 0x7f44d419a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:03.196 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:03.168+0000 7f44d8bd3700 1 -- 172.21.15.39:0/2424580562 >> 172.21.15.39:0/2424580562 conn(0x7f44d40fa000 msgr2=0x7f44d40fc1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:03.196 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:03.168+0000 7f44d8bd3700 1 -- 172.21.15.39:0/2424580562 shutdown_connections 2023-07-21T18:40:03.196 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:03.168+0000 7f44d8bd3700 1 -- 172.21.15.39:0/2424580562 wait complete. 2023-07-21T18:40:03.196 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_metadata' created 2023-07-21T18:40:03.207 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-07-21T18:40:03.355 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:03.328+0000 7fd8db834700 1 -- 172.21.15.39:0/893151220 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8d40fe9e0 msgr2=0x7fd8d40fedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:03.355 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:03.328+0000 7fd8db834700 1 --2- 172.21.15.39:0/893151220 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8d40fe9e0 0x7fd8d40fedf0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd8d0009990 tx=0x7fd8d0009ca0 comp rx=0 tx=0).stop 2023-07-21T18:40:03.355 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:03.328+0000 7fd8db834700 1 -- 172.21.15.39:0/893151220 shutdown_connections 2023-07-21T18:40:03.355 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:03.328+0000 7fd8db834700 1 --2- 172.21.15.39:0/893151220 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd8d4100560 0x7fd8d4106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:03.355 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:03.328+0000 7fd8db834700 1 --2- 172.21.15.39:0/893151220 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8d40fe9e0 0x7fd8d40fedf0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:03.355 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:03.328+0000 7fd8db834700 1 --2- 172.21.15.39:0/893151220 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8d40ffbd0 0x7fd8d4100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:03.356 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:03.328+0000 7fd8db834700 1 -- 172.21.15.39:0/893151220 >> 172.21.15.39:0/893151220 conn(0x7fd8d40fa000 msgr2=0x7fd8d40fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:03.356 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:03.328+0000 7fd8db834700 1 -- 172.21.15.39:0/893151220 shutdown_connections 2023-07-21T18:40:03.356 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:03.328+0000 7fd8db834700 1 -- 172.21.15.39:0/893151220 wait complete. 2023-07-21T18:40:03.358 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:03.329+0000 7fd8db834700 1 Processor -- start 2023-07-21T18:40:03.358 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:03.329+0000 7fd8db834700 1 -- start start 2023-07-21T18:40:03.358 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:03.329+0000 7fd8db834700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8d40fe9e0 0x7fd8d4194740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:03.358 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:03.329+0000 7fd8db834700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8d40ffbd0 0x7fd8d4194c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:03.358 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:03.329+0000 7fd8db834700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd8d4100560 0x7fd8d419bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:03.358 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:03.329+0000 7fd8db834700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d419c2c0 con 0x7fd8d40ffbd0 2023-07-21T18:40:03.359 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:03.329+0000 7fd8db834700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d419c400 con 0x7fd8d40fe9e0 2023-07-21T18:40:03.359 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:03.329+0000 7fd8db834700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d419c590 con 0x7fd8d4100560 2023-07-21T18:40:03.359 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:03.329+0000 7fd8d8dcf700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8d40ffbd0 0x7fd8d4194c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:03.359 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:03.329+0000 7fd8d8dcf700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8d40ffbd0 0x7fd8d4194c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54986/0 (socket says 172.21.15.39:54986) 2023-07-21T18:40:03.359 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:03.329+0000 7fd8d8dcf700 1 -- 172.21.15.39:0/3867475852 learned_addr learned my addr 172.21.15.39:0/3867475852 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:40:03.359 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:03.329+0000 7fd8d9dd1700 1 --2- 172.21.15.39:0/3867475852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd8d4100560 0x7fd8d419bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:03.359 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:03.329+0000 7fd8d95d0700 1 --2- 172.21.15.39:0/3867475852 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8d40fe9e0 0x7fd8d4194740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:03.359 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:03.329+0000 7fd8d9dd1700 1 -- 172.21.15.39:0/3867475852 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8d40fe9e0 msgr2=0x7fd8d4194740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:03.360 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:03.329+0000 7fd8d9dd1700 1 --2- 172.21.15.39:0/3867475852 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8d40fe9e0 0x7fd8d4194740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:03.360 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:03.329+0000 7fd8d9dd1700 1 -- 172.21.15.39:0/3867475852 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8d40ffbd0 msgr2=0x7fd8d4194c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:03.360 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:03.329+0000 7fd8d9dd1700 1 --2- 172.21.15.39:0/3867475852 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8d40ffbd0 0x7fd8d4194c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:03.360 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:03.329+0000 7fd8d9dd1700 1 -- 172.21.15.39:0/3867475852 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8d00095f0 con 0x7fd8d4100560 2023-07-21T18:40:03.360 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:03.330+0000 7fd8d9dd1700 1 --2- 172.21.15.39:0/3867475852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd8d4100560 0x7fd8d419bcf0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc00efc0 tx=0x7fd8cc00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:03.360 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:03.330+0000 7fd8ca7fc700 1 -- 172.21.15.39:0/3867475852 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8cc018070 con 0x7fd8d4100560 2023-07-21T18:40:03.360 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:03.330+0000 7fd8db834700 1 -- 172.21.15.39:0/3867475852 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8d419c810 con 0x7fd8d4100560 2023-07-21T18:40:03.361 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:03.330+0000 7fd8ca7fc700 1 -- 172.21.15.39:0/3867475852 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd8cc00f040 con 0x7fd8d4100560 2023-07-21T18:40:03.361 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:03.330+0000 7fd8ca7fc700 1 -- 172.21.15.39:0/3867475852 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8cc00cd40 con 0x7fd8d4100560 2023-07-21T18:40:03.361 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:03.330+0000 7fd8db834700 1 -- 172.21.15.39:0/3867475852 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8d419cd70 con 0x7fd8d4100560 2023-07-21T18:40:03.361 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:03.331+0000 7fd8db834700 1 -- 172.21.15.39:0/3867475852 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8d4105960 con 0x7fd8d4100560 2023-07-21T18:40:03.362 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:03.332+0000 7fd8ca7fc700 1 -- 172.21.15.39:0/3867475852 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd8cc003a00 con 0x7fd8d4100560 2023-07-21T18:40:03.362 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:03.333+0000 7fd8ca7fc700 1 --2- 172.21.15.39:0/3867475852 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fd8c00a1150 0x7fd8c00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:03.362 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:03.333+0000 7fd8d95d0700 1 --2- 172.21.15.39:0/3867475852 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fd8c00a1150 0x7fd8c00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:03.362 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:03.333+0000 7fd8ca7fc700 1 -- 172.21.15.39:0/3867475852 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7497+0+0 (secure 0 0 0) 0x7fd8cc0cc140 con 0x7fd8d4100560 2023-07-21T18:40:03.362 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:03.333+0000 7fd8d95d0700 1 --2- 172.21.15.39:0/3867475852 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fd8c00a1150 0x7fd8c00a3600 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd8d0006010 tx=0x7fd8d0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:03.364 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:03.337+0000 7fd8ca7fc700 1 -- 172.21.15.39:0/3867475852 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fd8cc095c50 con 0x7fd8d4100560 2023-07-21T18:40:03.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:03 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/2424580562' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-07-21T18:40:03.443 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:03 smithi039 ceph-mon[96489]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:40:03.571 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:03.544+0000 7fd8db834700 1 -- 172.21.15.39:0/3867475852 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fd8d4061120 con 0x7fd8d4100560 2023-07-21T18:40:03.624 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:03 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/2424580562' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-07-21T18:40:03.624 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:03 smithi193 ceph-mon[98204]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:40:03.642 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:03 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/2424580562' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-07-21T18:40:03.642 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:03 smithi155 ceph-mon[97934]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:40:04.195 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:04.168+0000 7fd8ca7fc700 1 -- 172.21.15.39:0/3867475852 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v77) v1 ==== 170+0+0 (secure 0 0 0) 0x7fd8cc099270 con 0x7fd8d4100560 2023-07-21T18:40:04.205 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:04.176+0000 7fd8db834700 1 -- 172.21.15.39:0/3867475852 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fd8c00a1150 msgr2=0x7fd8c00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:04.206 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:04.176+0000 7fd8db834700 1 --2- 172.21.15.39:0/3867475852 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fd8c00a1150 0x7fd8c00a3600 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd8d0006010 tx=0x7fd8d0005960 comp rx=0 tx=0).stop 2023-07-21T18:40:04.206 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:04.176+0000 7fd8db834700 1 -- 172.21.15.39:0/3867475852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd8d4100560 msgr2=0x7fd8d419bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:04.206 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:04.176+0000 7fd8db834700 1 --2- 172.21.15.39:0/3867475852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd8d4100560 0x7fd8d419bcf0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc00efc0 tx=0x7fd8cc00c3a0 comp rx=0 tx=0).stop 2023-07-21T18:40:04.208 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:04.182+0000 7fd8db834700 1 -- 172.21.15.39:0/3867475852 shutdown_connections 2023-07-21T18:40:04.209 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:04.182+0000 7fd8db834700 1 --2- 172.21.15.39:0/3867475852 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fd8c00a1150 0x7fd8c00a3600 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:04.209 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:04.182+0000 7fd8db834700 1 --2- 172.21.15.39:0/3867475852 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8d40fe9e0 0x7fd8d4194740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:04.209 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:04.182+0000 7fd8db834700 1 --2- 172.21.15.39:0/3867475852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd8d4100560 0x7fd8d419bcf0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:04.209 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:04.182+0000 7fd8db834700 1 --2- 172.21.15.39:0/3867475852 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8d40ffbd0 0x7fd8d4194c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:04.209 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:04.182+0000 7fd8db834700 1 -- 172.21.15.39:0/3867475852 >> 172.21.15.39:0/3867475852 conn(0x7fd8d40fa000 msgr2=0x7fd8d40fc2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:04.211 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:04.184+0000 7fd8db834700 1 -- 172.21.15.39:0/3867475852 shutdown_connections 2023-07-21T18:40:04.211 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:04.184+0000 7fd8db834700 1 -- 172.21.15.39:0/3867475852 wait complete. 2023-07-21T18:40:04.212 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_data' created 2023-07-21T18:40:04.224 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-07-21T18:40:04.360 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:04.331+0000 7f49e2aa7700 1 -- 172.21.15.39:0/1500255181 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49dc101660 msgr2=0x7f49dc107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:04.361 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:04.331+0000 7f49e2aa7700 1 --2- 172.21.15.39:0/1500255181 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49dc101660 0x7f49dc107f20 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f49cc009990 tx=0x7f49cc009ca0 comp rx=0 tx=0).stop 2023-07-21T18:40:04.361 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:04.331+0000 7f49e2aa7700 1 -- 172.21.15.39:0/1500255181 shutdown_connections 2023-07-21T18:40:04.361 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:04.331+0000 7f49e2aa7700 1 --2- 172.21.15.39:0/1500255181 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49dc101660 0x7f49dc107f20 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:04.362 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:04.331+0000 7f49e2aa7700 1 --2- 172.21.15.39:0/1500255181 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49dc0ffae0 0x7f49dc0ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:04.362 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:04.331+0000 7f49e2aa7700 1 --2- 172.21.15.39:0/1500255181 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49dc100cd0 0x7f49dc101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:04.362 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:04.331+0000 7f49e2aa7700 1 -- 172.21.15.39:0/1500255181 >> 172.21.15.39:0/1500255181 conn(0x7f49dc0fb0e0 msgr2=0x7f49dc0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:04.362 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:04.331+0000 7f49e2aa7700 1 -- 172.21.15.39:0/1500255181 shutdown_connections 2023-07-21T18:40:04.363 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:04.331+0000 7f49e2aa7700 1 -- 172.21.15.39:0/1500255181 wait complete. 2023-07-21T18:40:04.363 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:04.332+0000 7f49e2aa7700 1 Processor -- start 2023-07-21T18:40:04.363 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:04.332+0000 7f49e2aa7700 1 -- start start 2023-07-21T18:40:04.363 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:04.332+0000 7f49e2aa7700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49dc0ffae0 0x7f49dc1956e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:04.363 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:04.332+0000 7f49e2aa7700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49dc100cd0 0x7f49dc195c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:04.363 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:04.332+0000 7f49e2aa7700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49dc101660 0x7f49dc19cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:04.363 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:04.332+0000 7f49e2aa7700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49dc19d260 con 0x7f49dc100cd0 2023-07-21T18:40:04.364 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:04.332+0000 7f49e2aa7700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49dc19d3a0 con 0x7f49dc0ffae0 2023-07-21T18:40:04.364 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:04.332+0000 7f49e2aa7700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49dc19d530 con 0x7f49dc101660 2023-07-21T18:40:04.364 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:04.332+0000 7f49dbfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49dc100cd0 0x7f49dc195c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:04.364 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:04.332+0000 7f49dbfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49dc100cd0 0x7f49dc195c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55000/0 (socket says 172.21.15.39:55000) 2023-07-21T18:40:04.364 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:04.332+0000 7f49dbfff700 1 -- 172.21.15.39:0/1939311422 learned_addr learned my addr 172.21.15.39:0/1939311422 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:40:04.364 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:04.332+0000 7f49e0843700 1 --2- 172.21.15.39:0/1939311422 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49dc0ffae0 0x7f49dc1956e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:04.364 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:04.333+0000 7f49e1044700 1 --2- 172.21.15.39:0/1939311422 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49dc101660 0x7f49dc19cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:04.365 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:04.333+0000 7f49dbfff700 1 -- 172.21.15.39:0/1939311422 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49dc101660 msgr2=0x7f49dc19cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:04.365 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:04.333+0000 7f49dbfff700 1 --2- 172.21.15.39:0/1939311422 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49dc101660 0x7f49dc19cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:04.365 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:04.333+0000 7f49dbfff700 1 -- 172.21.15.39:0/1939311422 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49dc0ffae0 msgr2=0x7f49dc1956e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:04.365 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:04.333+0000 7f49dbfff700 1 --2- 172.21.15.39:0/1939311422 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49dc0ffae0 0x7f49dc1956e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:04.365 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:04.333+0000 7f49dbfff700 1 -- 172.21.15.39:0/1939311422 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49cc0095f0 con 0x7f49dc100cd0 2023-07-21T18:40:04.365 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:04.333+0000 7f49dbfff700 1 --2- 172.21.15.39:0/1939311422 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49dc100cd0 0x7f49dc195c20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f49c400e9a0 tx=0x7f49c400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:04.365 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:04.333+0000 7f49d9ffb700 1 -- 172.21.15.39:0/1939311422 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c4009790 con 0x7f49dc100cd0 2023-07-21T18:40:04.365 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:04.333+0000 7f49d9ffb700 1 -- 172.21.15.39:0/1939311422 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49c400cbd0 con 0x7f49dc100cd0 2023-07-21T18:40:04.366 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:04.333+0000 7f49e2aa7700 1 -- 172.21.15.39:0/1939311422 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49dc19d810 con 0x7f49dc100cd0 2023-07-21T18:40:04.366 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:04.333+0000 7f49d9ffb700 1 -- 172.21.15.39:0/1939311422 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c4018680 con 0x7f49dc100cd0 2023-07-21T18:40:04.366 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:04.333+0000 7f49e2aa7700 1 -- 172.21.15.39:0/1939311422 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49dc19dd70 con 0x7f49dc100cd0 2023-07-21T18:40:04.367 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:04.334+0000 7f49e2aa7700 1 -- 172.21.15.39:0/1939311422 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49dc10b2c0 con 0x7f49dc100cd0 2023-07-21T18:40:04.367 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:04.335+0000 7f49d9ffb700 1 -- 172.21.15.39:0/1939311422 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f49c400cd40 con 0x7f49dc100cd0 2023-07-21T18:40:04.367 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:04.336+0000 7f49d9ffb700 1 --2- 172.21.15.39:0/1939311422 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f49c80a11d0 0x7f49c80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:04.367 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:04.336+0000 7f49e0843700 1 --2- 172.21.15.39:0/1939311422 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f49c80a11d0 0x7f49c80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:04.367 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:04.336+0000 7f49d9ffb700 1 -- 172.21.15.39:0/1939311422 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f49c4014070 con 0x7f49dc100cd0 2023-07-21T18:40:04.368 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:04.336+0000 7f49e0843700 1 --2- 172.21.15.39:0/1939311422 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f49c80a11d0 0x7f49c80a3680 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f49dc100b30 tx=0x7f49d4007a30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:04.368 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:04.339+0000 7f49d9ffb700 1 -- 172.21.15.39:0/1939311422 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f49c40964f0 con 0x7f49dc100cd0 2023-07-21T18:40:04.618 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:04.592+0000 7f49e2aa7700 1 -- 172.21.15.39:0/1939311422 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f49dc19e500 con 0x7f49dc100cd0 2023-07-21T18:40:04.623 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:04 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/2424580562' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-07-21T18:40:04.624 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:04 smithi193 ceph-mon[98204]: osdmap e76: 12 total, 12 up, 12 in 2023-07-21T18:40:04.624 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:04 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/3867475852' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-21T18:40:04.624 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:04 smithi193 ceph-mon[98204]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-21T18:40:04.642 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:04 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/2424580562' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-07-21T18:40:04.642 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:04 smithi155 ceph-mon[97934]: osdmap e76: 12 total, 12 up, 12 in 2023-07-21T18:40:04.643 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:04 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/3867475852' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-21T18:40:04.643 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:04 smithi155 ceph-mon[97934]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-21T18:40:04.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:04 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/2424580562' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-07-21T18:40:04.696 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:04 smithi039 ceph-mon[96489]: osdmap e76: 12 total, 12 up, 12 in 2023-07-21T18:40:04.696 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:04 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/3867475852' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-21T18:40:04.696 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:04 smithi039 ceph-mon[96489]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-21T18:40:05.205 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.179+0000 7f49d9ffb700 1 -- 172.21.15.39:0/1939311422 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f49c40260c0 con 0x7f49dc100cd0 2023-07-21T18:40:05.208 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.181+0000 7f49e2aa7700 1 -- 172.21.15.39:0/1939311422 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f49c80a11d0 msgr2=0x7f49c80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:05.208 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.181+0000 7f49e2aa7700 1 --2- 172.21.15.39:0/1939311422 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f49c80a11d0 0x7f49c80a3680 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f49dc100b30 tx=0x7f49d4007a30 comp rx=0 tx=0).stop 2023-07-21T18:40:05.208 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.181+0000 7f49e2aa7700 1 -- 172.21.15.39:0/1939311422 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49dc100cd0 msgr2=0x7f49dc195c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:05.208 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.181+0000 7f49e2aa7700 1 --2- 172.21.15.39:0/1939311422 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49dc100cd0 0x7f49dc195c20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f49c400e9a0 tx=0x7f49c400ecb0 comp rx=0 tx=0).stop 2023-07-21T18:40:05.208 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.181+0000 7f49e2aa7700 1 -- 172.21.15.39:0/1939311422 shutdown_connections 2023-07-21T18:40:05.209 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.181+0000 7f49e2aa7700 1 --2- 172.21.15.39:0/1939311422 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f49c80a11d0 0x7f49c80a3680 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:05.209 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.181+0000 7f49e2aa7700 1 --2- 172.21.15.39:0/1939311422 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49dc0ffae0 0x7f49dc1956e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:05.209 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.181+0000 7f49e2aa7700 1 --2- 172.21.15.39:0/1939311422 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49dc101660 0x7f49dc19cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:05.209 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.181+0000 7f49e2aa7700 1 --2- 172.21.15.39:0/1939311422 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49dc100cd0 0x7f49dc195c20 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:05.209 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.181+0000 7f49e2aa7700 1 -- 172.21.15.39:0/1939311422 >> 172.21.15.39:0/1939311422 conn(0x7f49dc0fb0e0 msgr2=0x7f49dc0fd220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:05.209 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.181+0000 7f49e2aa7700 1 -- 172.21.15.39:0/1939311422 shutdown_connections 2023-07-21T18:40:05.209 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.181+0000 7f49e2aa7700 1 -- 172.21.15.39:0/1939311422 wait complete. 2023-07-21T18:40:05.210 INFO:teuthology.orchestra.run.smithi039.stderr:new fs with metadata pool 2 and data pool 3 2023-07-21T18:40:05.223 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T18:40:05.350 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.323+0000 7f9fd7c6d700 1 -- 172.21.15.39:0/369245650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9fd00ffbc0 msgr2=0x7f9fd0100010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:05.350 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.323+0000 7f9fd7c6d700 1 --2- 172.21.15.39:0/369245650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9fd00ffbc0 0x7f9fd0100010 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9fc0009990 tx=0x7f9fc0009ca0 comp rx=0 tx=0).stop 2023-07-21T18:40:05.351 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.323+0000 7f9fd7c6d700 1 -- 172.21.15.39:0/369245650 shutdown_connections 2023-07-21T18:40:05.351 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.323+0000 7f9fd7c6d700 1 --2- 172.21.15.39:0/369245650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9fd0100550 0x7f9fd0106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:05.351 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.323+0000 7f9fd7c6d700 1 --2- 172.21.15.39:0/369245650 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9fd00fe9d0 0x7f9fd00fede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:05.351 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.323+0000 7f9fd7c6d700 1 --2- 172.21.15.39:0/369245650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9fd00ffbc0 0x7f9fd0100010 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:05.351 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.323+0000 7f9fd7c6d700 1 -- 172.21.15.39:0/369245650 >> 172.21.15.39:0/369245650 conn(0x7f9fd00f9fd0 msgr2=0x7f9fd00fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:05.351 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.323+0000 7f9fd7c6d700 1 -- 172.21.15.39:0/369245650 shutdown_connections 2023-07-21T18:40:05.351 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.323+0000 7f9fd7c6d700 1 -- 172.21.15.39:0/369245650 wait complete. 2023-07-21T18:40:05.354 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.324+0000 7f9fd7c6d700 1 Processor -- start 2023-07-21T18:40:05.354 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.324+0000 7f9fd7c6d700 1 -- start start 2023-07-21T18:40:05.354 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.324+0000 7f9fd7c6d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9fd00fe9d0 0x7f9fd0068200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:05.354 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.324+0000 7f9fd7c6d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9fd00ffbc0 0x7f9fd019a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:05.355 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.324+0000 7f9fd7c6d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9fd0100550 0x7f9fd019cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:05.355 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.324+0000 7f9fd7c6d700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd019d320 con 0x7f9fd00fe9d0 2023-07-21T18:40:05.355 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.324+0000 7f9fd7c6d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd019d490 con 0x7f9fd00ffbc0 2023-07-21T18:40:05.356 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.324+0000 7f9fd7c6d700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd019d650 con 0x7f9fd0100550 2023-07-21T18:40:05.356 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.324+0000 7f9fd5a09700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9fd00fe9d0 0x7f9fd0068200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:05.356 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.324+0000 7f9fd5a09700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9fd00fe9d0 0x7f9fd0068200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55020/0 (socket says 172.21.15.39:55020) 2023-07-21T18:40:05.356 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.324+0000 7f9fd5a09700 1 -- 172.21.15.39:0/3825336784 learned_addr learned my addr 172.21.15.39:0/3825336784 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:40:05.356 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.324+0000 7f9fd5208700 1 --2- 172.21.15.39:0/3825336784 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9fd00ffbc0 0x7f9fd019a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:05.356 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.325+0000 7f9fd620a700 1 --2- 172.21.15.39:0/3825336784 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9fd0100550 0x7f9fd019cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:05.356 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.325+0000 7f9fd5a09700 1 -- 172.21.15.39:0/3825336784 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9fd0100550 msgr2=0x7f9fd019cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:05.357 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.325+0000 7f9fd5a09700 1 --2- 172.21.15.39:0/3825336784 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9fd0100550 0x7f9fd019cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:05.357 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.325+0000 7f9fd5a09700 1 -- 172.21.15.39:0/3825336784 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9fd00ffbc0 msgr2=0x7f9fd019a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:05.357 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.325+0000 7f9fd5a09700 1 --2- 172.21.15.39:0/3825336784 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9fd00ffbc0 0x7f9fd019a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:05.357 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.325+0000 7f9fd5a09700 1 -- 172.21.15.39:0/3825336784 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fc00095f0 con 0x7f9fd00fe9d0 2023-07-21T18:40:05.357 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.325+0000 7f9fd5a09700 1 --2- 172.21.15.39:0/3825336784 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9fd00fe9d0 0x7f9fd0068200 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc00b500 tx=0x7f9fcc00b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:05.357 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.325+0000 7f9fc6ffd700 1 -- 172.21.15.39:0/3825336784 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc011830 con 0x7f9fd00fe9d0 2023-07-21T18:40:05.357 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.325+0000 7f9fc6ffd700 1 -- 172.21.15.39:0/3825336784 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9fcc011e50 con 0x7f9fd00fe9d0 2023-07-21T18:40:05.357 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.325+0000 7f9fd7c6d700 1 -- 172.21.15.39:0/3825336784 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fd019d930 con 0x7f9fd00fe9d0 2023-07-21T18:40:05.358 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.325+0000 7f9fc6ffd700 1 -- 172.21.15.39:0/3825336784 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc00f2e0 con 0x7f9fd00fe9d0 2023-07-21T18:40:05.358 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.325+0000 7f9fd7c6d700 1 -- 172.21.15.39:0/3825336784 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fd019dec0 con 0x7f9fd00fe9d0 2023-07-21T18:40:05.358 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.326+0000 7f9fd7c6d700 1 -- 172.21.15.39:0/3825336784 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fd0105860 con 0x7f9fd00fe9d0 2023-07-21T18:40:05.358 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.327+0000 7f9fc6ffd700 1 -- 172.21.15.39:0/3825336784 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f9fcc010580 con 0x7f9fd00fe9d0 2023-07-21T18:40:05.359 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.328+0000 7f9fc6ffd700 1 --2- 172.21.15.39:0/3825336784 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f9fbc0a11a0 0x7f9fbc0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:05.359 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.328+0000 7f9fd5208700 1 --2- 172.21.15.39:0/3825336784 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f9fbc0a11a0 0x7f9fbc0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:05.359 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.328+0000 7f9fc6ffd700 1 -- 172.21.15.39:0/3825336784 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f9fcc0cc790 con 0x7f9fd00fe9d0 2023-07-21T18:40:05.359 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.329+0000 7f9fd5208700 1 --2- 172.21.15.39:0/3825336784 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f9fbc0a11a0 0x7f9fbc0a3650 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9fd0067f70 tx=0x7f9fc0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:05.359 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.332+0000 7f9fc6ffd700 1 -- 172.21.15.39:0/3825336784 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f9fcc096030 con 0x7f9fd00fe9d0 2023-07-21T18:40:05.558 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.532+0000 7f9fd7c6d700 1 -- 172.21.15.39:0/3825336784 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9fd002caa0 con 0x7f9fd00fe9d0 2023-07-21T18:40:05.560 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.533+0000 7f9fc6ffd700 1 -- 172.21.15.39:0/3825336784 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+22130 (secure 0 0 0) 0x7f9fcc015090 con 0x7f9fd00fe9d0 2023-07-21T18:40:05.560 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:40:05.561 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":78,"fsid":"eaf8e25a-27f4-11ee-9b34-001a4aab830c","created":"2023-07-21T18:33:06.504979+0000","modified":"2023-07-21T18:40:04.594151+0000","last_up_change":"2023-07-21T18:39:23.531009+0000","last_in_change":"2023-07-21T18:39:05.993024+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T18:36:13.572583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T18:40:02.726735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T18:40:03.990112+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03502c07-2d0f-4723-8d0b-d34c9f5b91bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6803","nonce":860193857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6805","nonce":860193857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6809","nonce":860193857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6807","nonce":860193857}]},"public_addr":"172.21.15.39:6803/860193857","cluster_addr":"172.21.15.39:6805/860193857","heartbeat_back_addr":"172.21.15.39:6809/860193857","heartbeat_front_addr":"172.21.15.39:6807/860193857","state":["exists","up"]},{"osd":1,"uuid":"0831a0c0-a1fa-4d30-9a28-513223aa69eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6801","nonce":3246721764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6803","nonce":3246721764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6807","nonce":3246721764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6805","nonce":3246721764}]},"public_addr":"172.21.15.155:6801/3246721764","cluster_addr":"172.21.15.155:6803/3246721764","heartbeat_back_addr":"172.21.15.155:6807/3246721764","heartbeat_front_addr":"172.21.15.155:6805/3246721764","state":["exists","up"]},{"osd":2,"uuid":"22df0003-bde9-48e4-b47a-cb8eafbab0db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6801","nonce":2563594449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6803","nonce":2563594449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6807","nonce":2563594449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6805","nonce":2563594449}]},"public_addr":"172.21.15.193:6801/2563594449","cluster_addr":"172.21.15.193:6803/2563594449","heartbeat_back_addr":"172.21.15.193:6807/2563594449","heartbeat_front_addr":"172.21.15.193:6805/2563594449","state":["exists","up"]},{"osd":3,"uuid":"54562558-a2e8-45dd-bc95-561569b9ae6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6811","nonce":863708386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6813","nonce":863708386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6817","nonce":863708386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6815","nonce":863708386}]},"public_addr":"172.21.15.39:6811/863708386","cluster_addr":"172.21.15.39:6813/863708386","heartbeat_back_addr":"172.21.15.39:6817/863708386","heartbeat_front_addr":"172.21.15.39:6815/863708386","state":["exists","up"]},{"osd":4,"uuid":"491118e7-ae1e-495c-9eac-ef1cdcae9600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6809","nonce":3361780673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6811","nonce":3361780673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6815","nonce":3361780673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6813","nonce":3361780673}]},"public_addr":"172.21.15.155:6809/3361780673","cluster_addr":"172.21.15.155:6811/3361780673","heartbeat_back_addr":"172.21.15.155:6815/3361780673","heartbeat_front_addr":"172.21.15.155:6813/3361780673","state":["exists","up"]},{"osd":5,"uuid":"d267bc2c-42b3-4d96-b2d1-b186dc7e6b18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6809","nonce":2376867721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6811","nonce":2376867721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6815","nonce":2376867721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6813","nonce":2376867721}]},"public_addr":"172.21.15.193:6809/2376867721","cluster_addr":"172.21.15.193:6811/2376867721","heartbeat_back_addr":"172.21.15.193:6815/2376867721","heartbeat_front_addr":"172.21.15.193:6813/2376867721","state":["exists","up"]},{"osd":6,"uuid":"beec8088-970a-473e-bd67-6eee7cbdef24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6819","nonce":2417770529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6821","nonce":2417770529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6825","nonce":2417770529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6823","nonce":2417770529}]},"public_addr":"172.21.15.39:6819/2417770529","cluster_addr":"172.21.15.39:6821/2417770529","heartbeat_back_addr":"172.21.15.39:6825/2417770529","heartbeat_front_addr":"172.21.15.39:6823/2417770529","state":["exists","up"]},{"osd":7,"uuid":"5a88ce30-1835-4254-90cc-804ed5cf1066","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6817","nonce":2542002111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6819","nonce":2542002111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6823","nonce":2542002111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6821","nonce":2542002111}]},"public_addr":"172.21.15.155:6817/2542002111","cluster_addr":"172.21.15.155:6819/2542002111","heartbeat_back_addr":"172.21.15.155:6823/2542002111","heartbeat_front_addr":"172.21.15.155:6821/2542002111","state":["exists","up"]},{"osd":8,"uuid":"d964a183-3021-4ace-9a0c-aad00f2570e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6817","nonce":2985835085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6819","nonce":2985835085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6823","nonce":2985835085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6821","nonce":2985835085}]},"public_addr":"172.21.15.193:6817/2985835085","cluster_addr":"172.21.15.193:6819/2985835085","heartbeat_back_addr":"172.21.15.193:6823/2985835085","heartbeat_front_addr":"172.21.15.193:6821/2985835085","state":["exists","up"]},{"osd":9,"uuid":"9725ea5c-49a3-4c99-bac4-e95069100b1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6827","nonce":1946493651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6829","nonce":1946493651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6833","nonce":1946493651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6831","nonce":1946493651}]},"public_addr":"172.21.15.39:6827/1946493651","cluster_addr":"172.21.15.39:6829/1946493651","heartbeat_back_addr":"172.21.15.39:6833/1946493651","heartbeat_front_addr":"172.21.15.39:6831/1946493651","state":["exists","up"]},{"osd":10,"uuid":"1fd60ccb-92db-45f0-a580-2003db0976e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6825","nonce":1234615022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6827","nonce":1234615022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6831","nonce":1234615022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6829","nonce":1234615022}]},"public_addr":"172.21.15.155:6825/1234615022","cluster_addr":"172.21.15.155:6827/1234615022","heartbeat_back_addr":"172.21.15.155:6831/1234615022","heartbeat_front_addr":"172.21.15.155:6829/1234615022","state":["exists","up"]},{"osd":11,"uuid":"efb31ea1-4775-4e5f-8740-728d6f4aeea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6825","nonce":215104663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6827","nonce":215104663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6831","nonce":215104663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6829","nonce":215104663}]},"public_addr":"172.21.15.193:6825/215104663","cluster_addr":"172.21.15.193:6827/215104663","heartbeat_back_addr":"172.21.15.193:6831/215104663","heartbeat_front_addr":"172.21.15.193:6829/215104663","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:35:30.211698+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:35:50.883846+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:36:11.124168+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:36:32.711905+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:36:54.442007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:37:14.843526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:37:36.587354+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:37:57.882374+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:38:18.729407+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:38:40.139231+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:39:01.290862+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:39:21.767537+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:0/2902600414":"2023-07-22T18:34:07.039823+0000","172.21.15.39:6801/2893866425":"2023-07-22T18:34:07.039823+0000","172.21.15.39:0/19717002":"2023-07-22T18:34:07.039823+0000","172.21.15.39:6800/2893866425":"2023-07-22T18:34:07.039823+0000","172.21.15.39:6800/889516631":"2023-07-22T18:33:40.852706+0000","172.21.15.39:6801/889516631":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/3636510379":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/1275748831":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/3819152602":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/1127767082":"2023-07-22T18:34:07.039823+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T18:40:05.562 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.535+0000 7f9fd7c6d700 1 -- 172.21.15.39:0/3825336784 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f9fbc0a11a0 msgr2=0x7f9fbc0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:05.562 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.535+0000 7f9fd7c6d700 1 --2- 172.21.15.39:0/3825336784 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f9fbc0a11a0 0x7f9fbc0a3650 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9fd0067f70 tx=0x7f9fc0005960 comp rx=0 tx=0).stop 2023-07-21T18:40:05.562 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.535+0000 7f9fd7c6d700 1 -- 172.21.15.39:0/3825336784 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9fd00fe9d0 msgr2=0x7f9fd0068200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:05.563 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.535+0000 7f9fd7c6d700 1 --2- 172.21.15.39:0/3825336784 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9fd00fe9d0 0x7f9fd0068200 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc00b500 tx=0x7f9fcc00b810 comp rx=0 tx=0).stop 2023-07-21T18:40:05.563 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.535+0000 7f9fd7c6d700 1 -- 172.21.15.39:0/3825336784 shutdown_connections 2023-07-21T18:40:05.563 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.535+0000 7f9fd7c6d700 1 --2- 172.21.15.39:0/3825336784 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f9fbc0a11a0 0x7f9fbc0a3650 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:05.563 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.535+0000 7f9fd7c6d700 1 --2- 172.21.15.39:0/3825336784 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9fd0100550 0x7f9fd019cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:05.563 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.535+0000 7f9fd7c6d700 1 --2- 172.21.15.39:0/3825336784 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9fd00fe9d0 0x7f9fd0068200 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:05.563 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.535+0000 7f9fd7c6d700 1 --2- 172.21.15.39:0/3825336784 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9fd00ffbc0 0x7f9fd019a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:05.564 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.535+0000 7f9fd7c6d700 1 -- 172.21.15.39:0/3825336784 >> 172.21.15.39:0/3825336784 conn(0x7f9fd00f9fd0 msgr2=0x7f9fd00fc190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:05.564 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.536+0000 7f9fd7c6d700 1 -- 172.21.15.39:0/3825336784 shutdown_connections 2023-07-21T18:40:05.564 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.536+0000 7f9fd7c6d700 1 -- 172.21.15.39:0/3825336784 wait complete. 2023-07-21T18:40:05.573 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:05 smithi039 ceph-mon[96489]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-07-21T18:40:05.574 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:05 smithi039 ceph-mon[96489]: osdmap e77: 12 total, 12 up, 12 in 2023-07-21T18:40:05.574 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:05 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/1939311422' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-07-21T18:40:05.575 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:05 smithi039 ceph-mon[96489]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-07-21T18:40:05.575 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:05 smithi039 ceph-mon[96489]: osdmap e78: 12 total, 12 up, 12 in 2023-07-21T18:40:05.575 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:05 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:40:05.575 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:05 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:40:05.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:05 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:40:05.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:05 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:40:05.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:05 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:40:05.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:05 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:40:05.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:05 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:40:05.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:05 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:40:05.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:05 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:40:05.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:05 smithi039 ceph-mon[96489]: pgmap v247: 97 pgs: 24 creating+peering, 3 active+clean, 70 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:40:05.580 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T18:40:05.623 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:05 smithi193 ceph-mon[98204]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-07-21T18:40:05.624 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:05 smithi193 ceph-mon[98204]: osdmap e77: 12 total, 12 up, 12 in 2023-07-21T18:40:05.624 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:05 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/1939311422' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-07-21T18:40:05.624 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:05 smithi193 ceph-mon[98204]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-07-21T18:40:05.625 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:05 smithi193 ceph-mon[98204]: osdmap e78: 12 total, 12 up, 12 in 2023-07-21T18:40:05.625 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:05 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:40:05.625 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:05 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:40:05.625 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:05 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:40:05.626 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:05 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:40:05.626 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:05 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:40:05.626 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:05 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:40:05.626 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:05 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:40:05.626 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:05 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:40:05.627 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:05 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:40:05.627 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:05 smithi193 ceph-mon[98204]: pgmap v247: 97 pgs: 24 creating+peering, 3 active+clean, 70 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:40:05.642 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:05 smithi155 ceph-mon[97934]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-07-21T18:40:05.642 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:05 smithi155 ceph-mon[97934]: osdmap e77: 12 total, 12 up, 12 in 2023-07-21T18:40:05.643 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:05 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/1939311422' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-07-21T18:40:05.643 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:05 smithi155 ceph-mon[97934]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-07-21T18:40:05.643 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:05 smithi155 ceph-mon[97934]: osdmap e78: 12 total, 12 up, 12 in 2023-07-21T18:40:05.643 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:05 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:40:05.643 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:05 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:40:05.643 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:05 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:40:05.643 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:05 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:40:05.644 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:05 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:40:05.644 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:05 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:40:05.644 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:05 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:40:05.644 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:05 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:40:05.644 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:05 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:40:05.644 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:05 smithi155 ceph-mon[97934]: pgmap v247: 97 pgs: 24 creating+peering, 3 active+clean, 70 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:40:05.708 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.679+0000 7fb1ec893700 1 -- 172.21.15.39:0/1383915751 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1e8100cf0 msgr2=0x7fb1e8101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:05.709 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.679+0000 7fb1ec893700 1 --2- 172.21.15.39:0/1383915751 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1e8100cf0 0x7fb1e8101140 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc009990 tx=0x7fb1dc009ca0 comp rx=0 tx=0).stop 2023-07-21T18:40:05.709 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.679+0000 7fb1ec893700 1 -- 172.21.15.39:0/1383915751 shutdown_connections 2023-07-21T18:40:05.709 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.679+0000 7fb1ec893700 1 --2- 172.21.15.39:0/1383915751 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1e80ffb00 0x7fb1e80fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:05.710 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.679+0000 7fb1ec893700 1 --2- 172.21.15.39:0/1383915751 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb1e8101680 0x7fb1e8107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:05.710 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.679+0000 7fb1ec893700 1 --2- 172.21.15.39:0/1383915751 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1e8100cf0 0x7fb1e8101140 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:05.710 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.679+0000 7fb1ec893700 1 -- 172.21.15.39:0/1383915751 >> 172.21.15.39:0/1383915751 conn(0x7fb1e80fb0e0 msgr2=0x7fb1e80fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:05.710 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.679+0000 7fb1ec893700 1 -- 172.21.15.39:0/1383915751 shutdown_connections 2023-07-21T18:40:05.710 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.679+0000 7fb1ec893700 1 -- 172.21.15.39:0/1383915751 wait complete. 2023-07-21T18:40:05.711 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.680+0000 7fb1ec893700 1 Processor -- start 2023-07-21T18:40:05.711 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.680+0000 7fb1ec893700 1 -- start start 2023-07-21T18:40:05.711 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.680+0000 7fb1ec893700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb1e80ffb00 0x7fb1e80742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:05.712 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.680+0000 7fb1ec893700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1e8100cf0 0x7fb1e8072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:05.712 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.680+0000 7fb1ec893700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1e8101680 0x7fb1e8072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:05.712 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.680+0000 7fb1ec893700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e80735e0 con 0x7fb1e8101680 2023-07-21T18:40:05.713 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.680+0000 7fb1ec893700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e8073750 con 0x7fb1e8100cf0 2023-07-21T18:40:05.713 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.680+0000 7fb1ec893700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e819f850 con 0x7fb1e80ffb00 2023-07-21T18:40:05.713 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.680+0000 7fb1e6d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1e8101680 0x7fb1e8072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:05.713 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.680+0000 7fb1e659c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb1e80ffb00 0x7fb1e80742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:05.714 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.680+0000 7fb1e6d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1e8101680 0x7fb1e8072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55038/0 (socket says 172.21.15.39:55038) 2023-07-21T18:40:05.714 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.680+0000 7fb1e6d9d700 1 -- 172.21.15.39:0/840749405 learned_addr learned my addr 172.21.15.39:0/840749405 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:40:05.714 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.680+0000 7fb1e5d9b700 1 --2- 172.21.15.39:0/840749405 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1e8100cf0 0x7fb1e8072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:05.714 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.680+0000 7fb1e6d9d700 1 -- 172.21.15.39:0/840749405 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb1e80ffb00 msgr2=0x7fb1e80742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:05.714 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.680+0000 7fb1e6d9d700 1 --2- 172.21.15.39:0/840749405 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb1e80ffb00 0x7fb1e80742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:05.715 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.680+0000 7fb1e6d9d700 1 -- 172.21.15.39:0/840749405 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1e8100cf0 msgr2=0x7fb1e8072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:05.715 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.680+0000 7fb1e6d9d700 1 --2- 172.21.15.39:0/840749405 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1e8100cf0 0x7fb1e8072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:05.715 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.680+0000 7fb1e6d9d700 1 -- 172.21.15.39:0/840749405 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1dc0095f0 con 0x7fb1e8101680 2023-07-21T18:40:05.715 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.681+0000 7fb1e6d9d700 1 --2- 172.21.15.39:0/840749405 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1e8101680 0x7fb1e8072e40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0007ee0 tx=0x7fb1e000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:05.716 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.681+0000 7fb1d77fe700 1 -- 172.21.15.39:0/840749405 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e0017070 con 0x7fb1e8101680 2023-07-21T18:40:05.716 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.681+0000 7fb1d77fe700 1 -- 172.21.15.39:0/840749405 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb1e000f040 con 0x7fb1e8101680 2023-07-21T18:40:05.716 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.681+0000 7fb1ec893700 1 -- 172.21.15.39:0/840749405 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1e819fb30 con 0x7fb1e8101680 2023-07-21T18:40:05.716 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.681+0000 7fb1d77fe700 1 -- 172.21.15.39:0/840749405 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e0005580 con 0x7fb1e8101680 2023-07-21T18:40:05.717 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.681+0000 7fb1ec893700 1 -- 172.21.15.39:0/840749405 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1e81a0040 con 0x7fb1e8101680 2023-07-21T18:40:05.717 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.682+0000 7fb1ec893700 1 -- 172.21.15.39:0/840749405 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1e810b2d0 con 0x7fb1e8101680 2023-07-21T18:40:05.718 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.683+0000 7fb1d77fe700 1 -- 172.21.15.39:0/840749405 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb1e000db20 con 0x7fb1e8101680 2023-07-21T18:40:05.718 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.684+0000 7fb1d77fe700 1 --2- 172.21.15.39:0/840749405 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fb1d00a1220 0x7fb1d00a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:05.719 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.684+0000 7fb1e659c700 1 --2- 172.21.15.39:0/840749405 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fb1d00a1220 0x7fb1d00a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:05.719 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.684+0000 7fb1d77fe700 1 -- 172.21.15.39:0/840749405 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb1e00cc4c0 con 0x7fb1e8101680 2023-07-21T18:40:05.719 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.684+0000 7fb1e659c700 1 --2- 172.21.15.39:0/840749405 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fb1d00a1220 0x7fb1d00a36d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb1e8100b50 tx=0x7fb1d8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:05.719 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.687+0000 7fb1d77fe700 1 -- 172.21.15.39:0/840749405 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fb1e0095d60 con 0x7fb1e8101680 2023-07-21T18:40:05.917 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.891+0000 7fb1ec893700 1 -- 172.21.15.39:0/840749405 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb1e810b4c0 con 0x7fb1e8101680 2023-07-21T18:40:05.919 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.892+0000 7fb1d77fe700 1 -- 172.21.15.39:0/840749405 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+22117 (secure 0 0 0) 0x7fb1e0099380 con 0x7fb1e8101680 2023-07-21T18:40:05.920 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:40:05.920 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":79,"fsid":"eaf8e25a-27f4-11ee-9b34-001a4aab830c","created":"2023-07-21T18:33:06.504979+0000","modified":"2023-07-21T18:40:05.769300+0000","last_up_change":"2023-07-21T18:39:23.531009+0000","last_in_change":"2023-07-21T18:39:05.993024+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T18:36:13.572583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T18:40:02.726735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T18:40:03.990112+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03502c07-2d0f-4723-8d0b-d34c9f5b91bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6803","nonce":860193857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6805","nonce":860193857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6809","nonce":860193857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6807","nonce":860193857}]},"public_addr":"172.21.15.39:6803/860193857","cluster_addr":"172.21.15.39:6805/860193857","heartbeat_back_addr":"172.21.15.39:6809/860193857","heartbeat_front_addr":"172.21.15.39:6807/860193857","state":["exists","up"]},{"osd":1,"uuid":"0831a0c0-a1fa-4d30-9a28-513223aa69eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6801","nonce":3246721764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6803","nonce":3246721764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6807","nonce":3246721764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6805","nonce":3246721764}]},"public_addr":"172.21.15.155:6801/3246721764","cluster_addr":"172.21.15.155:6803/3246721764","heartbeat_back_addr":"172.21.15.155:6807/3246721764","heartbeat_front_addr":"172.21.15.155:6805/3246721764","state":["exists","up"]},{"osd":2,"uuid":"22df0003-bde9-48e4-b47a-cb8eafbab0db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6801","nonce":2563594449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6803","nonce":2563594449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6807","nonce":2563594449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6805","nonce":2563594449}]},"public_addr":"172.21.15.193:6801/2563594449","cluster_addr":"172.21.15.193:6803/2563594449","heartbeat_back_addr":"172.21.15.193:6807/2563594449","heartbeat_front_addr":"172.21.15.193:6805/2563594449","state":["exists","up"]},{"osd":3,"uuid":"54562558-a2e8-45dd-bc95-561569b9ae6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6811","nonce":863708386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6813","nonce":863708386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6817","nonce":863708386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6815","nonce":863708386}]},"public_addr":"172.21.15.39:6811/863708386","cluster_addr":"172.21.15.39:6813/863708386","heartbeat_back_addr":"172.21.15.39:6817/863708386","heartbeat_front_addr":"172.21.15.39:6815/863708386","state":["exists","up"]},{"osd":4,"uuid":"491118e7-ae1e-495c-9eac-ef1cdcae9600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6809","nonce":3361780673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6811","nonce":3361780673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6815","nonce":3361780673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6813","nonce":3361780673}]},"public_addr":"172.21.15.155:6809/3361780673","cluster_addr":"172.21.15.155:6811/3361780673","heartbeat_back_addr":"172.21.15.155:6815/3361780673","heartbeat_front_addr":"172.21.15.155:6813/3361780673","state":["exists","up"]},{"osd":5,"uuid":"d267bc2c-42b3-4d96-b2d1-b186dc7e6b18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6809","nonce":2376867721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6811","nonce":2376867721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6815","nonce":2376867721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6813","nonce":2376867721}]},"public_addr":"172.21.15.193:6809/2376867721","cluster_addr":"172.21.15.193:6811/2376867721","heartbeat_back_addr":"172.21.15.193:6815/2376867721","heartbeat_front_addr":"172.21.15.193:6813/2376867721","state":["exists","up"]},{"osd":6,"uuid":"beec8088-970a-473e-bd67-6eee7cbdef24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6819","nonce":2417770529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6821","nonce":2417770529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6825","nonce":2417770529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6823","nonce":2417770529}]},"public_addr":"172.21.15.39:6819/2417770529","cluster_addr":"172.21.15.39:6821/2417770529","heartbeat_back_addr":"172.21.15.39:6825/2417770529","heartbeat_front_addr":"172.21.15.39:6823/2417770529","state":["exists","up"]},{"osd":7,"uuid":"5a88ce30-1835-4254-90cc-804ed5cf1066","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6817","nonce":2542002111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6819","nonce":2542002111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6823","nonce":2542002111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6821","nonce":2542002111}]},"public_addr":"172.21.15.155:6817/2542002111","cluster_addr":"172.21.15.155:6819/2542002111","heartbeat_back_addr":"172.21.15.155:6823/2542002111","heartbeat_front_addr":"172.21.15.155:6821/2542002111","state":["exists","up"]},{"osd":8,"uuid":"d964a183-3021-4ace-9a0c-aad00f2570e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6817","nonce":2985835085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6819","nonce":2985835085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6823","nonce":2985835085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6821","nonce":2985835085}]},"public_addr":"172.21.15.193:6817/2985835085","cluster_addr":"172.21.15.193:6819/2985835085","heartbeat_back_addr":"172.21.15.193:6823/2985835085","heartbeat_front_addr":"172.21.15.193:6821/2985835085","state":["exists","up"]},{"osd":9,"uuid":"9725ea5c-49a3-4c99-bac4-e95069100b1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6827","nonce":1946493651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6829","nonce":1946493651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6833","nonce":1946493651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6831","nonce":1946493651}]},"public_addr":"172.21.15.39:6827/1946493651","cluster_addr":"172.21.15.39:6829/1946493651","heartbeat_back_addr":"172.21.15.39:6833/1946493651","heartbeat_front_addr":"172.21.15.39:6831/1946493651","state":["exists","up"]},{"osd":10,"uuid":"1fd60ccb-92db-45f0-a580-2003db0976e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6825","nonce":1234615022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6827","nonce":1234615022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6831","nonce":1234615022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6829","nonce":1234615022}]},"public_addr":"172.21.15.155:6825/1234615022","cluster_addr":"172.21.15.155:6827/1234615022","heartbeat_back_addr":"172.21.15.155:6831/1234615022","heartbeat_front_addr":"172.21.15.155:6829/1234615022","state":["exists","up"]},{"osd":11,"uuid":"efb31ea1-4775-4e5f-8740-728d6f4aeea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6825","nonce":215104663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6827","nonce":215104663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6831","nonce":215104663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6829","nonce":215104663}]},"public_addr":"172.21.15.193:6825/215104663","cluster_addr":"172.21.15.193:6827/215104663","heartbeat_back_addr":"172.21.15.193:6831/215104663","heartbeat_front_addr":"172.21.15.193:6829/215104663","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:35:30.211698+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:35:50.883846+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:36:11.124168+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:36:32.711905+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:36:54.442007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:37:14.843526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:37:36.587354+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:37:57.882374+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:38:18.729407+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:38:40.139231+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:39:01.290862+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:39:21.767537+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:0/2902600414":"2023-07-22T18:34:07.039823+0000","172.21.15.39:6801/2893866425":"2023-07-22T18:34:07.039823+0000","172.21.15.39:0/19717002":"2023-07-22T18:34:07.039823+0000","172.21.15.39:6800/2893866425":"2023-07-22T18:34:07.039823+0000","172.21.15.39:6800/889516631":"2023-07-22T18:33:40.852706+0000","172.21.15.39:6801/889516631":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/3636510379":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/1275748831":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/3819152602":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/1127767082":"2023-07-22T18:34:07.039823+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T18:40:05.921 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.894+0000 7fb1ec893700 1 -- 172.21.15.39:0/840749405 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fb1d00a1220 msgr2=0x7fb1d00a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:05.921 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.894+0000 7fb1ec893700 1 --2- 172.21.15.39:0/840749405 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fb1d00a1220 0x7fb1d00a36d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb1e8100b50 tx=0x7fb1d8009260 comp rx=0 tx=0).stop 2023-07-21T18:40:05.922 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.894+0000 7fb1ec893700 1 -- 172.21.15.39:0/840749405 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1e8101680 msgr2=0x7fb1e8072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:05.922 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.894+0000 7fb1ec893700 1 --2- 172.21.15.39:0/840749405 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1e8101680 0x7fb1e8072e40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0007ee0 tx=0x7fb1e000d3b0 comp rx=0 tx=0).stop 2023-07-21T18:40:05.922 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.894+0000 7fb1ec893700 1 -- 172.21.15.39:0/840749405 shutdown_connections 2023-07-21T18:40:05.922 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.894+0000 7fb1ec893700 1 --2- 172.21.15.39:0/840749405 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fb1d00a1220 0x7fb1d00a36d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:05.922 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.894+0000 7fb1ec893700 1 --2- 172.21.15.39:0/840749405 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1e8101680 0x7fb1e8072e40 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:05.922 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.894+0000 7fb1ec893700 1 --2- 172.21.15.39:0/840749405 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb1e80ffb00 0x7fb1e80742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:05.923 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.894+0000 7fb1ec893700 1 --2- 172.21.15.39:0/840749405 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1e8100cf0 0x7fb1e8072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:05.923 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.895+0000 7fb1ec893700 1 -- 172.21.15.39:0/840749405 >> 172.21.15.39:0/840749405 conn(0x7fb1e80fb0e0 msgr2=0x7fb1e80fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:05.923 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.895+0000 7fb1ec893700 1 -- 172.21.15.39:0/840749405 shutdown_connections 2023-07-21T18:40:05.923 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:05.895+0000 7fb1ec893700 1 -- 172.21.15.39:0/840749405 wait complete. 2023-07-21T18:40:05.934 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-07-21T18:40:06.066 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:06.037+0000 7f219cbee700 1 -- 172.21.15.39:0/1589796 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21980ffbd0 msgr2=0x7f2198100020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:06.067 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:06.037+0000 7f219cbee700 1 --2- 172.21.15.39:0/1589796 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21980ffbd0 0x7f2198100020 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2188009990 tx=0x7f2188009ca0 comp rx=0 tx=0).stop 2023-07-21T18:40:06.067 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:06.037+0000 7f219cbee700 1 -- 172.21.15.39:0/1589796 shutdown_connections 2023-07-21T18:40:06.067 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:06.037+0000 7f219cbee700 1 --2- 172.21.15.39:0/1589796 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21980fe9e0 0x7f21980fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:06.067 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:06.037+0000 7f219cbee700 1 --2- 172.21.15.39:0/1589796 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2198100560 0x7f2198106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:06.068 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:06.037+0000 7f219cbee700 1 --2- 172.21.15.39:0/1589796 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21980ffbd0 0x7f2198100020 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:06.068 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:06.037+0000 7f219cbee700 1 -- 172.21.15.39:0/1589796 >> 172.21.15.39:0/1589796 conn(0x7f21980fa000 msgr2=0x7f21980fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:06.068 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:06.037+0000 7f219cbee700 1 -- 172.21.15.39:0/1589796 shutdown_connections 2023-07-21T18:40:06.069 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:06.037+0000 7f219cbee700 1 -- 172.21.15.39:0/1589796 wait complete. 2023-07-21T18:40:06.069 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:06.038+0000 7f219cbee700 1 Processor -- start 2023-07-21T18:40:06.069 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:06.038+0000 7f219cbee700 1 -- start start 2023-07-21T18:40:06.069 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:06.038+0000 7f219cbee700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21980fe9e0 0x7f21980681e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:06.070 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:06.038+0000 7f219cbee700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21980ffbd0 0x7f219819a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:06.070 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:06.038+0000 7f219cbee700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2198100560 0x7f219819cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:06.070 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:06.038+0000 7f219cbee700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f219819d3c0 con 0x7f21980fe9e0 2023-07-21T18:40:06.070 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:06.038+0000 7f219cbee700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f219819d530 con 0x7f21980ffbd0 2023-07-21T18:40:06.071 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:06.038+0000 7f219cbee700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f219819d6f0 con 0x7f2198100560 2023-07-21T18:40:06.071 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:06.038+0000 7f219659c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21980fe9e0 0x7f21980681e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:06.071 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:06.038+0000 7f219659c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21980fe9e0 0x7f21980681e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55048/0 (socket says 172.21.15.39:55048) 2023-07-21T18:40:06.072 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:06.038+0000 7f219659c700 1 -- 172.21.15.39:0/3785479282 learned_addr learned my addr 172.21.15.39:0/3785479282 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:40:06.072 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:06.038+0000 7f2196d9d700 1 --2- 172.21.15.39:0/3785479282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2198100560 0x7f219819cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:06.072 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:06.038+0000 7f2195d9b700 1 --2- 172.21.15.39:0/3785479282 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21980ffbd0 0x7f219819a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:06.072 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:06.038+0000 7f219659c700 1 -- 172.21.15.39:0/3785479282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2198100560 msgr2=0x7f219819cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:06.073 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:06.038+0000 7f219659c700 1 --2- 172.21.15.39:0/3785479282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2198100560 0x7f219819cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:06.073 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:06.038+0000 7f219659c700 1 -- 172.21.15.39:0/3785479282 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21980ffbd0 msgr2=0x7f219819a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:06.073 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:06.038+0000 7f219659c700 1 --2- 172.21.15.39:0/3785479282 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21980ffbd0 0x7f219819a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:06.073 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:06.038+0000 7f219659c700 1 -- 172.21.15.39:0/3785479282 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21880095f0 con 0x7f21980fe9e0 2023-07-21T18:40:06.074 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:06.039+0000 7f219659c700 1 --2- 172.21.15.39:0/3785479282 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21980fe9e0 0x7f21980681e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f218000d980 tx=0x7f218000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:06.074 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:06.039+0000 7f218f7fe700 1 -- 172.21.15.39:0/3785479282 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2180004020 con 0x7f21980fe9e0 2023-07-21T18:40:06.075 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:06.039+0000 7f219cbee700 1 -- 172.21.15.39:0/3785479282 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f219819d9d0 con 0x7f21980fe9e0 2023-07-21T18:40:06.075 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:06.039+0000 7f218f7fe700 1 -- 172.21.15.39:0/3785479282 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f218000f690 con 0x7f21980fe9e0 2023-07-21T18:40:06.075 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:06.039+0000 7f218f7fe700 1 -- 172.21.15.39:0/3785479282 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2180003a60 con 0x7f21980fe9e0 2023-07-21T18:40:06.075 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:06.039+0000 7f219cbee700 1 -- 172.21.15.39:0/3785479282 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f219819df30 con 0x7f21980fe9e0 2023-07-21T18:40:06.076 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:06.040+0000 7f219cbee700 1 -- 172.21.15.39:0/3785479282 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2198105960 con 0x7f21980fe9e0 2023-07-21T18:40:06.076 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:06.041+0000 7f218f7fe700 1 -- 172.21.15.39:0/3785479282 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f218000fb00 con 0x7f21980fe9e0 2023-07-21T18:40:06.077 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:06.041+0000 7f218f7fe700 1 --2- 172.21.15.39:0/3785479282 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f21840a1150 0x7f21840a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:06.077 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:06.042+0000 7f218f7fe700 1 -- 172.21.15.39:0/3785479282 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f21800cc6f0 con 0x7f21980fe9e0 2023-07-21T18:40:06.077 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:06.042+0000 7f2195d9b700 1 --2- 172.21.15.39:0/3785479282 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f21840a1150 0x7f21840a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:06.078 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:06.042+0000 7f2195d9b700 1 --2- 172.21.15.39:0/3785479282 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f21840a1150 0x7f21840a3600 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f218800b5c0 tx=0x7f2188005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:06.078 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:06.045+0000 7f218f7fe700 1 -- 172.21.15.39:0/3785479282 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f2180095f90 con 0x7f21980fe9e0 2023-07-21T18:40:06.318 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:06.291+0000 7f219cbee700 1 -- 172.21.15.39:0/3785479282 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f219802c850 con 0x7f21980fe9e0 2023-07-21T18:40:06.623 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:06 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/1939311422' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-07-21T18:40:06.624 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:06 smithi193 ceph-mon[98204]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-07-21T18:40:06.624 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:06 smithi193 ceph-mon[98204]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-07-21T18:40:06.624 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:06 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/3825336784' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T18:40:06.624 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:06 smithi193 ceph-mon[98204]: osdmap e79: 12 total, 12 up, 12 in 2023-07-21T18:40:06.624 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:06 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/840749405' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T18:40:06.625 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:06 smithi193 ceph-mon[98204]: mds.? [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] up:active 2023-07-21T18:40:06.625 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:06 smithi193 ceph-mon[98204]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-21T18:40:06.642 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:06 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/1939311422' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-07-21T18:40:06.642 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:06 smithi155 ceph-mon[97934]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-07-21T18:40:06.643 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:06 smithi155 ceph-mon[97934]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-07-21T18:40:06.643 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:06 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/3825336784' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T18:40:06.643 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:06 smithi155 ceph-mon[97934]: osdmap e79: 12 total, 12 up, 12 in 2023-07-21T18:40:06.643 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:06 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/840749405' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T18:40:06.643 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:06 smithi155 ceph-mon[97934]: mds.? [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] up:active 2023-07-21T18:40:06.643 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:06 smithi155 ceph-mon[97934]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-21T18:40:06.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:06 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/1939311422' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-07-21T18:40:06.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:06 smithi039 ceph-mon[96489]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-07-21T18:40:06.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:06 smithi039 ceph-mon[96489]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-07-21T18:40:06.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:06 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/3825336784' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T18:40:06.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:06 smithi039 ceph-mon[96489]: osdmap e79: 12 total, 12 up, 12 in 2023-07-21T18:40:06.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:06 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/840749405' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T18:40:06.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:06 smithi039 ceph-mon[96489]: mds.? [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] up:active 2023-07-21T18:40:06.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:06 smithi039 ceph-mon[96489]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-21T18:40:07.215 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:07.189+0000 7f218f7fe700 1 -- 172.21.15.39:0/3785479282 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) v1 ==== 118+0+0 (secure 0 0 0) 0x7f21800995b0 con 0x7f21980fe9e0 2023-07-21T18:40:07.218 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:07.191+0000 7f219cbee700 1 -- 172.21.15.39:0/3785479282 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f21840a1150 msgr2=0x7f21840a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:07.218 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:07.191+0000 7f219cbee700 1 --2- 172.21.15.39:0/3785479282 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f21840a1150 0x7f21840a3600 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f218800b5c0 tx=0x7f2188005fb0 comp rx=0 tx=0).stop 2023-07-21T18:40:07.218 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:07.191+0000 7f219cbee700 1 -- 172.21.15.39:0/3785479282 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21980fe9e0 msgr2=0x7f21980681e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:07.219 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:07.191+0000 7f219cbee700 1 --2- 172.21.15.39:0/3785479282 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21980fe9e0 0x7f21980681e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f218000d980 tx=0x7f218000dc90 comp rx=0 tx=0).stop 2023-07-21T18:40:07.219 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:07.191+0000 7f219cbee700 1 -- 172.21.15.39:0/3785479282 shutdown_connections 2023-07-21T18:40:07.219 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:07.191+0000 7f219cbee700 1 --2- 172.21.15.39:0/3785479282 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f21840a1150 0x7f21840a3600 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:07.219 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:07.191+0000 7f219cbee700 1 --2- 172.21.15.39:0/3785479282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2198100560 0x7f219819cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:07.220 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:07.191+0000 7f219cbee700 1 --2- 172.21.15.39:0/3785479282 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21980fe9e0 0x7f21980681e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:07.220 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:07.191+0000 7f219cbee700 1 --2- 172.21.15.39:0/3785479282 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21980ffbd0 0x7f219819a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:07.220 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:07.191+0000 7f219cbee700 1 -- 172.21.15.39:0/3785479282 >> 172.21.15.39:0/3785479282 conn(0x7f21980fa000 msgr2=0x7f21980fc2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:07.220 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:07.191+0000 7f219cbee700 1 -- 172.21.15.39:0/3785479282 shutdown_connections 2023-07-21T18:40:07.220 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:07.191+0000 7f219cbee700 1 -- 172.21.15.39:0/3785479282 wait complete. 2023-07-21T18:40:07.233 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2023-07-21T18:40:07.234 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2023-07-21T18:40:07.374 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:07.345+0000 7fa944481700 1 -- 172.21.15.39:0/3028555514 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa93c100ce0 msgr2=0x7fa93c101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:07.374 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:07.345+0000 7fa944481700 1 --2- 172.21.15.39:0/3028555514 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa93c100ce0 0x7fa93c101130 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa92c009990 tx=0x7fa92c009ca0 comp rx=0 tx=0).stop 2023-07-21T18:40:07.375 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:07.345+0000 7fa944481700 1 -- 172.21.15.39:0/3028555514 shutdown_connections 2023-07-21T18:40:07.375 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:07.345+0000 7fa944481700 1 --2- 172.21.15.39:0/3028555514 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa93c0ffaf0 0x7fa93c0fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:07.375 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:07.345+0000 7fa944481700 1 --2- 172.21.15.39:0/3028555514 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa93c101670 0x7fa93c107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:07.376 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:07.345+0000 7fa944481700 1 --2- 172.21.15.39:0/3028555514 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa93c100ce0 0x7fa93c101130 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:07.377 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:07.345+0000 7fa944481700 1 -- 172.21.15.39:0/3028555514 >> 172.21.15.39:0/3028555514 conn(0x7fa93c0fb130 msgr2=0x7fa93c0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:07.377 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:07.345+0000 7fa944481700 1 -- 172.21.15.39:0/3028555514 shutdown_connections 2023-07-21T18:40:07.377 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:07.345+0000 7fa944481700 1 -- 172.21.15.39:0/3028555514 wait complete. 2023-07-21T18:40:07.377 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:07.346+0000 7fa944481700 1 Processor -- start 2023-07-21T18:40:07.377 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:07.346+0000 7fa944481700 1 -- start start 2023-07-21T18:40:07.377 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:07.346+0000 7fa944481700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa93c0ffaf0 0x7fa93c195840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:07.378 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:07.346+0000 7fa944481700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa93c100ce0 0x7fa93c195d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:07.378 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:07.346+0000 7fa944481700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa93c101670 0x7fa93c19cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:07.378 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:07.346+0000 7fa944481700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa93c19d3c0 con 0x7fa93c101670 2023-07-21T18:40:07.378 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:07.346+0000 7fa944481700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa93c19d500 con 0x7fa93c100ce0 2023-07-21T18:40:07.378 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:07.346+0000 7fa944481700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa93c19d690 con 0x7fa93c0ffaf0 2023-07-21T18:40:07.378 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:07.346+0000 7fa942a1e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa93c101670 0x7fa93c19cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:07.378 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:07.346+0000 7fa942a1e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa93c101670 0x7fa93c19cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55066/0 (socket says 172.21.15.39:55066) 2023-07-21T18:40:07.379 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:07.346+0000 7fa942a1e700 1 -- 172.21.15.39:0/1352605301 learned_addr learned my addr 172.21.15.39:0/1352605301 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:40:07.379 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:07.346+0000 7fa941a1c700 1 --2- 172.21.15.39:0/1352605301 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa93c100ce0 0x7fa93c195d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:07.379 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:07.346+0000 7fa94221d700 1 --2- 172.21.15.39:0/1352605301 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa93c0ffaf0 0x7fa93c195840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:07.379 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:07.346+0000 7fa942a1e700 1 -- 172.21.15.39:0/1352605301 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa93c0ffaf0 msgr2=0x7fa93c195840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:07.379 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:07.346+0000 7fa942a1e700 1 --2- 172.21.15.39:0/1352605301 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa93c0ffaf0 0x7fa93c195840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:07.379 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:07.346+0000 7fa942a1e700 1 -- 172.21.15.39:0/1352605301 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa93c100ce0 msgr2=0x7fa93c195d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:07.379 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:07.346+0000 7fa942a1e700 1 --2- 172.21.15.39:0/1352605301 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa93c100ce0 0x7fa93c195d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:07.379 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:07.346+0000 7fa942a1e700 1 -- 172.21.15.39:0/1352605301 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa92c0095f0 con 0x7fa93c101670 2023-07-21T18:40:07.380 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:07.347+0000 7fa942a1e700 1 --2- 172.21.15.39:0/1352605301 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa93c101670 0x7fa93c19cdf0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa93400b970 tx=0x7fa93400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:07.380 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:07.347+0000 7fa9337fe700 1 -- 172.21.15.39:0/1352605301 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93400c930 con 0x7fa93c101670 2023-07-21T18:40:07.380 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:07.347+0000 7fa9337fe700 1 -- 172.21.15.39:0/1352605301 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa934014440 con 0x7fa93c101670 2023-07-21T18:40:07.380 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:07.347+0000 7fa944481700 1 -- 172.21.15.39:0/1352605301 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa93c19d970 con 0x7fa93c101670 2023-07-21T18:40:07.380 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:07.347+0000 7fa9337fe700 1 -- 172.21.15.39:0/1352605301 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9340075d0 con 0x7fa93c101670 2023-07-21T18:40:07.380 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:07.347+0000 7fa944481700 1 -- 172.21.15.39:0/1352605301 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa93c19deb0 con 0x7fa93c101670 2023-07-21T18:40:07.381 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:07.348+0000 7fa944481700 1 -- 172.21.15.39:0/1352605301 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa93c106fe0 con 0x7fa93c101670 2023-07-21T18:40:07.381 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:07.349+0000 7fa9337fe700 1 -- 172.21.15.39:0/1352605301 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fa9340145b0 con 0x7fa93c101670 2023-07-21T18:40:07.382 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:07.350+0000 7fa9337fe700 1 --2- 172.21.15.39:0/1352605301 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fa9280a1150 0x7fa9280a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:07.382 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:07.350+0000 7fa94221d700 1 --2- 172.21.15.39:0/1352605301 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fa9280a1150 0x7fa9280a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:07.382 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:07.350+0000 7fa9337fe700 1 -- 172.21.15.39:0/1352605301 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa934006e40 con 0x7fa93c101670 2023-07-21T18:40:07.382 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:07.350+0000 7fa94221d700 1 --2- 172.21.15.39:0/1352605301 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fa9280a1150 0x7fa9280a3600 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa938005950 tx=0x7fa9380058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:07.382 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:07.353+0000 7fa9337fe700 1 -- 172.21.15.39:0/1352605301 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fa934095b70 con 0x7fa93c101670 2023-07-21T18:40:07.623 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:07 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/3785479282' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-07-21T18:40:07.624 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:07 smithi193 ceph-mon[98204]: pgmap v249: 97 pgs: 24 creating+peering, 34 active+clean, 39 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 6 op/s 2023-07-21T18:40:07.625 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:07.599+0000 7fa944481700 1 -- 172.21.15.39:0/1352605301 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) v1 -- 0x7fa93c02cce0 con 0x7fa93c101670 2023-07-21T18:40:07.642 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:07 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/3785479282' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-07-21T18:40:07.642 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:07 smithi155 ceph-mon[97934]: pgmap v249: 97 pgs: 24 creating+peering, 34 active+clean, 39 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 6 op/s 2023-07-21T18:40:07.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:07 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/3785479282' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-07-21T18:40:07.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:07 smithi039 ceph-mon[96489]: pgmap v249: 97 pgs: 24 creating+peering, 34 active+clean, 39 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 6 op/s 2023-07-21T18:40:08.225 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:08.198+0000 7fa9337fe700 1 -- 172.21.15.39:0/1352605301 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v17) v1 ==== 105+0+0 (secure 0 0 0) 0x7fa9340b7e60 con 0x7fa93c101670 2023-07-21T18:40:08.228 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:08.202+0000 7fa944481700 1 -- 172.21.15.39:0/1352605301 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fa9280a1150 msgr2=0x7fa9280a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:08.229 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:08.202+0000 7fa944481700 1 --2- 172.21.15.39:0/1352605301 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fa9280a1150 0x7fa9280a3600 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa938005950 tx=0x7fa9380058e0 comp rx=0 tx=0).stop 2023-07-21T18:40:08.229 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:08.202+0000 7fa944481700 1 -- 172.21.15.39:0/1352605301 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa93c101670 msgr2=0x7fa93c19cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:08.229 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:08.202+0000 7fa944481700 1 --2- 172.21.15.39:0/1352605301 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa93c101670 0x7fa93c19cdf0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa93400b970 tx=0x7fa93400bc80 comp rx=0 tx=0).stop 2023-07-21T18:40:08.229 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:08.202+0000 7fa944481700 1 -- 172.21.15.39:0/1352605301 shutdown_connections 2023-07-21T18:40:08.230 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:08.202+0000 7fa944481700 1 --2- 172.21.15.39:0/1352605301 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fa9280a1150 0x7fa9280a3600 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:08.230 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:08.202+0000 7fa944481700 1 --2- 172.21.15.39:0/1352605301 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa93c101670 0x7fa93c19cdf0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:08.230 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:08.202+0000 7fa944481700 1 --2- 172.21.15.39:0/1352605301 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa93c0ffaf0 0x7fa93c195840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:08.230 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:08.202+0000 7fa944481700 1 --2- 172.21.15.39:0/1352605301 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa93c100ce0 0x7fa93c195d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:08.230 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:08.202+0000 7fa944481700 1 -- 172.21.15.39:0/1352605301 >> 172.21.15.39:0/1352605301 conn(0x7fa93c0fb130 msgr2=0x7fa93c104120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:08.230 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:08.202+0000 7fa944481700 1 -- 172.21.15.39:0/1352605301 shutdown_connections 2023-07-21T18:40:08.231 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:08.202+0000 7fa944481700 1 -- 172.21.15.39:0/1352605301 wait complete. 2023-07-21T18:40:08.247 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2023-07-21T18:40:08.383 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:08.357+0000 7f9e12649700 1 -- 172.21.15.39:0/3287841123 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e0c105800 msgr2=0x7f9e0c107c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:08.384 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:08.357+0000 7f9e12649700 1 --2- 172.21.15.39:0/3287841123 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e0c105800 0x7f9e0c107c00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9e0400b0a0 tx=0x7f9e0400b3b0 comp rx=0 tx=0).stop 2023-07-21T18:40:08.384 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:08.357+0000 7f9e12649700 1 -- 172.21.15.39:0/3287841123 shutdown_connections 2023-07-21T18:40:08.384 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:08.357+0000 7f9e12649700 1 --2- 172.21.15.39:0/3287841123 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e0c105800 0x7f9e0c107c00 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:08.384 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:08.357+0000 7f9e12649700 1 --2- 172.21.15.39:0/3287841123 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e0c1005c0 0x7f9e0c1029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:08.384 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:08.357+0000 7f9e12649700 1 --2- 172.21.15.39:0/3287841123 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9e0c102ee0 0x7f9e0c1052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:08.385 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:08.357+0000 7f9e12649700 1 -- 172.21.15.39:0/3287841123 >> 172.21.15.39:0/3287841123 conn(0x7f9e0c0fa000 msgr2=0x7f9e0c0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:08.385 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:08.357+0000 7f9e12649700 1 -- 172.21.15.39:0/3287841123 shutdown_connections 2023-07-21T18:40:08.385 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:08.357+0000 7f9e12649700 1 -- 172.21.15.39:0/3287841123 wait complete. 2023-07-21T18:40:08.385 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:08.358+0000 7f9e12649700 1 Processor -- start 2023-07-21T18:40:08.385 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:08.358+0000 7f9e12649700 1 -- start start 2023-07-21T18:40:08.387 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:08.358+0000 7f9e12649700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e0c1005c0 0x7f9e0c0fff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:08.387 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:08.358+0000 7f9e12649700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e0c102ee0 0x7f9e0c0fe570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:08.387 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:08.358+0000 7f9e12649700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9e0c105800 0x7f9e0c0feab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:08.387 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:08.358+0000 7f9e12649700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0c0ff250 con 0x7f9e0c102ee0 2023-07-21T18:40:08.387 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:08.358+0000 7f9e12649700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0c0ff3c0 con 0x7f9e0c105800 2023-07-21T18:40:08.388 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:08.358+0000 7f9e12649700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0c19f9a0 con 0x7f9e0c1005c0 2023-07-21T18:40:08.388 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:08.359+0000 7f9e0b7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e0c102ee0 0x7f9e0c0fe570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:08.388 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:08.359+0000 7f9e0b7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e0c102ee0 0x7f9e0c0fe570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55076/0 (socket says 172.21.15.39:55076) 2023-07-21T18:40:08.388 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:08.359+0000 7f9e0b7fe700 1 -- 172.21.15.39:0/3252238689 learned_addr learned my addr 172.21.15.39:0/3252238689 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:40:08.388 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:08.359+0000 7f9e10be6700 1 --2- 172.21.15.39:0/3252238689 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9e0c105800 0x7f9e0c0feab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:08.388 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:08.359+0000 7f9e0bfff700 1 --2- 172.21.15.39:0/3252238689 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e0c1005c0 0x7f9e0c0fff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:08.388 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:08.359+0000 7f9e0b7fe700 1 -- 172.21.15.39:0/3252238689 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e0c1005c0 msgr2=0x7f9e0c0fff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:08.389 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:08.359+0000 7f9e0b7fe700 1 --2- 172.21.15.39:0/3252238689 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e0c1005c0 0x7f9e0c0fff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:08.389 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:08.359+0000 7f9e0b7fe700 1 -- 172.21.15.39:0/3252238689 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9e0c105800 msgr2=0x7f9e0c0feab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:08.389 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:08.359+0000 7f9e0b7fe700 1 --2- 172.21.15.39:0/3252238689 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9e0c105800 0x7f9e0c0feab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:08.389 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:08.359+0000 7f9e0b7fe700 1 -- 172.21.15.39:0/3252238689 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e04009d00 con 0x7f9e0c102ee0 2023-07-21T18:40:08.389 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:08.359+0000 7f9e0b7fe700 1 --2- 172.21.15.39:0/3252238689 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e0c102ee0 0x7f9e0c0fe570 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9e0000d950 tx=0x7f9e0000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:08.389 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:08.359+0000 7f9e097fa700 1 -- 172.21.15.39:0/3252238689 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e00009940 con 0x7f9e0c102ee0 2023-07-21T18:40:08.389 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:08.359+0000 7f9e097fa700 1 -- 172.21.15.39:0/3252238689 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e00010460 con 0x7f9e0c102ee0 2023-07-21T18:40:08.389 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:08.359+0000 7f9e12649700 1 -- 172.21.15.39:0/3252238689 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e0c19fc80 con 0x7f9e0c102ee0 2023-07-21T18:40:08.390 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:08.359+0000 7f9e097fa700 1 -- 172.21.15.39:0/3252238689 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0000f620 con 0x7f9e0c102ee0 2023-07-21T18:40:08.390 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:08.359+0000 7f9e12649700 1 -- 172.21.15.39:0/3252238689 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e0c1a0190 con 0x7f9e0c102ee0 2023-07-21T18:40:08.390 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:08.360+0000 7f9e12649700 1 -- 172.21.15.39:0/3252238689 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e0c0fbb70 con 0x7f9e0c102ee0 2023-07-21T18:40:08.390 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:08.361+0000 7f9e097fa700 1 -- 172.21.15.39:0/3252238689 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f9e000108d0 con 0x7f9e0c102ee0 2023-07-21T18:40:08.391 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:08.362+0000 7f9e097fa700 1 --2- 172.21.15.39:0/3252238689 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f9df40a11d0 0x7f9df40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:08.391 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:08.362+0000 7f9e097fa700 1 -- 172.21.15.39:0/3252238689 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f9e000cc5c0 con 0x7f9e0c102ee0 2023-07-21T18:40:08.391 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:08.362+0000 7f9e0bfff700 1 --2- 172.21.15.39:0/3252238689 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f9df40a11d0 0x7f9df40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:08.391 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:08.363+0000 7f9e0bfff700 1 --2- 172.21.15.39:0/3252238689 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f9df40a11d0 0x7f9df40a3680 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc005950 tx=0x7f9dfc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:08.392 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:08.366+0000 7f9e097fa700 1 -- 172.21.15.39:0/3252238689 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f9e00095e60 con 0x7f9e0c102ee0 2023-07-21T18:40:08.623 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:08 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/3785479282' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-07-21T18:40:08.624 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:08 smithi193 ceph-mon[98204]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-21T18:40:08.624 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:08 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/1352605301' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-07-21T18:40:08.639 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:08.613+0000 7f9e12649700 1 -- 172.21.15.39:0/3252238689 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f9e0c02c850 con 0x7f9e0c102ee0 2023-07-21T18:40:08.642 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:08 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/3785479282' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-07-21T18:40:08.642 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:08 smithi155 ceph-mon[97934]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-21T18:40:08.643 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:08 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/1352605301' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-07-21T18:40:08.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:08 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/3785479282' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-07-21T18:40:08.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:08 smithi039 ceph-mon[96489]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-21T18:40:08.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:08 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/1352605301' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-07-21T18:40:09.239 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:09.214+0000 7f9e097fa700 1 -- 172.21.15.39:0/3252238689 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v19) v1 ==== 121+0+0 (secure 0 0 0) 0x7f9e00099480 con 0x7f9e0c102ee0 2023-07-21T18:40:09.242 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:09.216+0000 7f9e12649700 1 -- 172.21.15.39:0/3252238689 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f9df40a11d0 msgr2=0x7f9df40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:09.242 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:09.216+0000 7f9e12649700 1 --2- 172.21.15.39:0/3252238689 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f9df40a11d0 0x7f9df40a3680 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc005950 tx=0x7f9dfc0058e0 comp rx=0 tx=0).stop 2023-07-21T18:40:09.242 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:09.216+0000 7f9e12649700 1 -- 172.21.15.39:0/3252238689 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e0c102ee0 msgr2=0x7f9e0c0fe570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:09.242 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:09.216+0000 7f9e12649700 1 --2- 172.21.15.39:0/3252238689 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e0c102ee0 0x7f9e0c0fe570 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9e0000d950 tx=0x7f9e0000dc60 comp rx=0 tx=0).stop 2023-07-21T18:40:09.243 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:09.216+0000 7f9e12649700 1 -- 172.21.15.39:0/3252238689 shutdown_connections 2023-07-21T18:40:09.243 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:09.216+0000 7f9e12649700 1 --2- 172.21.15.39:0/3252238689 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f9df40a11d0 0x7f9df40a3680 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:09.243 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:09.216+0000 7f9e12649700 1 --2- 172.21.15.39:0/3252238689 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9e0c105800 0x7f9e0c0feab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:09.243 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:09.216+0000 7f9e12649700 1 --2- 172.21.15.39:0/3252238689 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e0c1005c0 0x7f9e0c0fff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:09.243 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:09.216+0000 7f9e12649700 1 --2- 172.21.15.39:0/3252238689 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e0c102ee0 0x7f9e0c0fe570 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:09.243 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:09.216+0000 7f9e12649700 1 -- 172.21.15.39:0/3252238689 >> 172.21.15.39:0/3252238689 conn(0x7f9e0c0fa000 msgr2=0x7f9e0c101220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:09.243 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:09.216+0000 7f9e12649700 1 -- 172.21.15.39:0/3252238689 shutdown_connections 2023-07-21T18:40:09.243 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:09.216+0000 7f9e12649700 1 -- 172.21.15.39:0/3252238689 wait complete. 2023-07-21T18:40:09.259 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2023-07-21T18:40:09.390 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:09.364+0000 7f265c14d700 1 -- 172.21.15.39:0/1900965970 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2654100560 msgr2=0x7f2654106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:09.390 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:09.364+0000 7f265c14d700 1 --2- 172.21.15.39:0/1900965970 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2654100560 0x7f2654106e20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f264c00b0a0 tx=0x7f264c00b3b0 comp rx=0 tx=0).stop 2023-07-21T18:40:09.391 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:09.364+0000 7f265c14d700 1 -- 172.21.15.39:0/1900965970 shutdown_connections 2023-07-21T18:40:09.391 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:09.364+0000 7f265c14d700 1 --2- 172.21.15.39:0/1900965970 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2654100560 0x7f2654106e20 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:09.391 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:09.364+0000 7f265c14d700 1 --2- 172.21.15.39:0/1900965970 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26540fe9e0 0x7f26540fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:09.391 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:09.364+0000 7f265c14d700 1 --2- 172.21.15.39:0/1900965970 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26540ffbd0 0x7f2654100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:09.391 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:09.364+0000 7f265c14d700 1 -- 172.21.15.39:0/1900965970 >> 172.21.15.39:0/1900965970 conn(0x7f26540fa000 msgr2=0x7f26540fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:09.391 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:09.364+0000 7f265c14d700 1 -- 172.21.15.39:0/1900965970 shutdown_connections 2023-07-21T18:40:09.391 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:09.364+0000 7f265c14d700 1 -- 172.21.15.39:0/1900965970 wait complete. 2023-07-21T18:40:09.392 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:09.365+0000 7f265c14d700 1 Processor -- start 2023-07-21T18:40:09.393 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:09.365+0000 7f265c14d700 1 -- start start 2023-07-21T18:40:09.394 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:09.365+0000 7f265c14d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26540fe9e0 0x7f2654194810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:09.394 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:09.365+0000 7f265c14d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26540ffbd0 0x7f2654194d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:09.394 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:09.365+0000 7f265c14d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2654100560 0x7f265419bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:09.394 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:09.365+0000 7f265c14d700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f265419c390 con 0x7f26540fe9e0 2023-07-21T18:40:09.394 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:09.365+0000 7f265c14d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f265419c4d0 con 0x7f26540ffbd0 2023-07-21T18:40:09.394 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:09.365+0000 7f265c14d700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f265419c660 con 0x7f2654100560 2023-07-21T18:40:09.394 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:09.365+0000 7f2659ee9700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26540fe9e0 0x7f2654194810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:09.395 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:09.365+0000 7f2659ee9700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26540fe9e0 0x7f2654194810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55088/0 (socket says 172.21.15.39:55088) 2023-07-21T18:40:09.395 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:09.365+0000 7f2659ee9700 1 -- 172.21.15.39:0/4148906896 learned_addr learned my addr 172.21.15.39:0/4148906896 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:40:09.395 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:09.365+0000 7f26596e8700 1 --2- 172.21.15.39:0/4148906896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26540ffbd0 0x7f2654194d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:09.395 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:09.365+0000 7f265a6ea700 1 --2- 172.21.15.39:0/4148906896 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2654100560 0x7f265419bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:09.395 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:09.366+0000 7f2659ee9700 1 -- 172.21.15.39:0/4148906896 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2654100560 msgr2=0x7f265419bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:09.395 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:09.366+0000 7f2659ee9700 1 --2- 172.21.15.39:0/4148906896 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2654100560 0x7f265419bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:09.395 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:09.366+0000 7f2659ee9700 1 -- 172.21.15.39:0/4148906896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26540ffbd0 msgr2=0x7f2654194d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:09.395 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:09.366+0000 7f2659ee9700 1 --2- 172.21.15.39:0/4148906896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26540ffbd0 0x7f2654194d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:09.396 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:09.366+0000 7f2659ee9700 1 -- 172.21.15.39:0/4148906896 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f264c009d00 con 0x7f26540fe9e0 2023-07-21T18:40:09.396 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:09.366+0000 7f265a6ea700 1 --2- 172.21.15.39:0/4148906896 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2654100560 0x7f265419bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:40:09.396 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:09.366+0000 7f2659ee9700 1 --2- 172.21.15.39:0/4148906896 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26540fe9e0 0x7f2654194810 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f265000c970 tx=0x7f265000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:09.396 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:09.366+0000 7f2646ffd700 1 -- 172.21.15.39:0/4148906896 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2650004220 con 0x7f26540fe9e0 2023-07-21T18:40:09.396 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:09.366+0000 7f2646ffd700 1 -- 172.21.15.39:0/4148906896 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2650007d60 con 0x7f26540fe9e0 2023-07-21T18:40:09.396 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:09.366+0000 7f265c14d700 1 -- 172.21.15.39:0/4148906896 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f265419c940 con 0x7f26540fe9e0 2023-07-21T18:40:09.396 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:09.366+0000 7f2646ffd700 1 -- 172.21.15.39:0/4148906896 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2650003c90 con 0x7f26540fe9e0 2023-07-21T18:40:09.396 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:09.366+0000 7f265c14d700 1 -- 172.21.15.39:0/4148906896 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f265419ce50 con 0x7f26540fe9e0 2023-07-21T18:40:09.397 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:09.367+0000 7f265c14d700 1 -- 172.21.15.39:0/4148906896 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2654105870 con 0x7f26540fe9e0 2023-07-21T18:40:09.397 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:09.368+0000 7f2646ffd700 1 -- 172.21.15.39:0/4148906896 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f2650004380 con 0x7f26540fe9e0 2023-07-21T18:40:09.397 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:09.369+0000 7f2646ffd700 1 --2- 172.21.15.39:0/4148906896 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f26400a1150 0x7f26400a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:09.398 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:09.369+0000 7f2646ffd700 1 -- 172.21.15.39:0/4148906896 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f26500cc7d0 con 0x7f26540fe9e0 2023-07-21T18:40:09.398 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:09.369+0000 7f26596e8700 1 --2- 172.21.15.39:0/4148906896 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f26400a1150 0x7f26400a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:09.398 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:09.369+0000 7f26596e8700 1 --2- 172.21.15.39:0/4148906896 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f26400a1150 0x7f26400a3600 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2648005950 tx=0x7f264800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:09.398 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:09.372+0000 7f2646ffd700 1 -- 172.21.15.39:0/4148906896 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f2650095fd0 con 0x7f26540fe9e0 2023-07-21T18:40:09.624 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:09 smithi193 ceph-mon[98204]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-07-21T18:40:09.624 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:09 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/1352605301' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-07-21T18:40:09.624 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:09 smithi193 ceph-mon[98204]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-07-21T18:40:09.624 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:09 smithi193 ceph-mon[98204]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-21T18:40:09.624 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:09 smithi193 ceph-mon[98204]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2023-07-21T18:40:09.625 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:09 smithi193 ceph-mon[98204]: daemon mds.h is now active in filesystem cephfs as rank 1 2023-07-21T18:40:09.625 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:09 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/3252238689' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-07-21T18:40:09.625 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:09 smithi193 ceph-mon[98204]: pgmap v250: 97 pgs: 15 creating+peering, 82 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 9 op/s 2023-07-21T18:40:09.642 INFO:journalctl@ceph.mds.b.smithi155.stdout:Jul 21 18:40:09 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-b[123270]: 2023-07-21T18:40:09.155+0000 7f8b91b33700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-07-21T18:40:09.643 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:09 smithi155 ceph-mon[97934]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-07-21T18:40:09.643 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:09 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/1352605301' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-07-21T18:40:09.643 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:09 smithi155 ceph-mon[97934]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-07-21T18:40:09.643 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:09 smithi155 ceph-mon[97934]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-21T18:40:09.643 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:09 smithi155 ceph-mon[97934]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2023-07-21T18:40:09.643 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:09 smithi155 ceph-mon[97934]: daemon mds.h is now active in filesystem cephfs as rank 1 2023-07-21T18:40:09.644 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:09 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/3252238689' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-07-21T18:40:09.644 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:09 smithi155 ceph-mon[97934]: pgmap v250: 97 pgs: 15 creating+peering, 82 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 9 op/s 2023-07-21T18:40:09.646 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:09.621+0000 7f265c14d700 1 -- 172.21.15.39:0/4148906896 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f265402c850 con 0x7f26540fe9e0 2023-07-21T18:40:09.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:09 smithi039 ceph-mon[96489]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-07-21T18:40:09.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:09 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/1352605301' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-07-21T18:40:09.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:09 smithi039 ceph-mon[96489]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-07-21T18:40:09.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:09 smithi039 ceph-mon[96489]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-21T18:40:09.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:09 smithi039 ceph-mon[96489]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2023-07-21T18:40:09.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:09 smithi039 ceph-mon[96489]: daemon mds.h is now active in filesystem cephfs as rank 1 2023-07-21T18:40:09.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:09 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/3252238689' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-07-21T18:40:09.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:09 smithi039 ceph-mon[96489]: pgmap v250: 97 pgs: 15 creating+peering, 82 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 9 op/s 2023-07-21T18:40:10.250 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.224+0000 7f2646ffd700 1 -- 172.21.15.39:0/4148906896 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v21) v1 ==== 115+0+0 (secure 0 0 0) 0x7f265001d0b0 con 0x7f26540fe9e0 2023-07-21T18:40:10.253 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.227+0000 7f265c14d700 1 -- 172.21.15.39:0/4148906896 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f26400a1150 msgr2=0x7f26400a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:10.253 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.227+0000 7f265c14d700 1 --2- 172.21.15.39:0/4148906896 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f26400a1150 0x7f26400a3600 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2648005950 tx=0x7f264800a300 comp rx=0 tx=0).stop 2023-07-21T18:40:10.253 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.227+0000 7f265c14d700 1 -- 172.21.15.39:0/4148906896 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26540fe9e0 msgr2=0x7f2654194810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:10.254 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.227+0000 7f265c14d700 1 --2- 172.21.15.39:0/4148906896 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26540fe9e0 0x7f2654194810 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f265000c970 tx=0x7f265000cc80 comp rx=0 tx=0).stop 2023-07-21T18:40:10.254 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.227+0000 7f265c14d700 1 -- 172.21.15.39:0/4148906896 shutdown_connections 2023-07-21T18:40:10.254 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.227+0000 7f265c14d700 1 --2- 172.21.15.39:0/4148906896 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f26400a1150 0x7f26400a3600 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:10.254 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.227+0000 7f265c14d700 1 --2- 172.21.15.39:0/4148906896 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2654100560 0x7f265419bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:10.254 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.227+0000 7f265c14d700 1 --2- 172.21.15.39:0/4148906896 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26540fe9e0 0x7f2654194810 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:10.254 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.227+0000 7f265c14d700 1 --2- 172.21.15.39:0/4148906896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26540ffbd0 0x7f2654194d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:10.254 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.227+0000 7f265c14d700 1 -- 172.21.15.39:0/4148906896 >> 172.21.15.39:0/4148906896 conn(0x7f26540fa000 msgr2=0x7f26540fc1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:10.255 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.227+0000 7f265c14d700 1 -- 172.21.15.39:0/4148906896 shutdown_connections 2023-07-21T18:40:10.255 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.227+0000 7f265c14d700 1 -- 172.21.15.39:0/4148906896 wait complete. 2023-07-21T18:40:10.271 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2023-07-21T18:40:10.271 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 '' 2023-07-21T18:40:10.406 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.380+0000 7f51b9253700 1 -- 172.21.15.39:0/3144788235 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51b4100560 msgr2=0x7f51b4106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:10.406 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.380+0000 7f51b9253700 1 --2- 172.21.15.39:0/3144788235 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51b4100560 0x7f51b4106e20 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f51ac00b0a0 tx=0x7f51ac00b3b0 comp rx=0 tx=0).stop 2023-07-21T18:40:10.406 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.380+0000 7f51b9253700 1 -- 172.21.15.39:0/3144788235 shutdown_connections 2023-07-21T18:40:10.406 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.380+0000 7f51b9253700 1 --2- 172.21.15.39:0/3144788235 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51b4100560 0x7f51b4106e20 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:10.407 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.380+0000 7f51b9253700 1 --2- 172.21.15.39:0/3144788235 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f51b40fe9e0 0x7f51b40fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:10.407 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.380+0000 7f51b9253700 1 --2- 172.21.15.39:0/3144788235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f51b40ffbd0 0x7f51b4100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:10.407 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.380+0000 7f51b9253700 1 -- 172.21.15.39:0/3144788235 >> 172.21.15.39:0/3144788235 conn(0x7f51b40fa000 msgr2=0x7f51b40fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:10.407 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.380+0000 7f51b9253700 1 -- 172.21.15.39:0/3144788235 shutdown_connections 2023-07-21T18:40:10.407 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.380+0000 7f51b9253700 1 -- 172.21.15.39:0/3144788235 wait complete. 2023-07-21T18:40:10.409 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.381+0000 7f51b9253700 1 Processor -- start 2023-07-21T18:40:10.409 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.381+0000 7f51b9253700 1 -- start start 2023-07-21T18:40:10.409 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.382+0000 7f51b9253700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f51b40fe9e0 0x7f51b40681e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:10.409 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.382+0000 7f51b9253700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f51b40ffbd0 0x7f51b419a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:10.409 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.382+0000 7f51b9253700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51b4100560 0x7f51b419cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:10.409 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.382+0000 7f51b9253700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b419d4c0 con 0x7f51b4100560 2023-07-21T18:40:10.410 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.382+0000 7f51b9253700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b419d630 con 0x7f51b40fe9e0 2023-07-21T18:40:10.410 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.382+0000 7f51b9253700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b419d7f0 con 0x7f51b40ffbd0 2023-07-21T18:40:10.410 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.382+0000 7f51b359e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51b4100560 0x7f51b419cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:10.410 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.382+0000 7f51b359e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51b4100560 0x7f51b419cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55106/0 (socket says 172.21.15.39:55106) 2023-07-21T18:40:10.410 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.382+0000 7f51b359e700 1 -- 172.21.15.39:0/2118002247 learned_addr learned my addr 172.21.15.39:0/2118002247 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:40:10.410 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.382+0000 7f51b259c700 1 --2- 172.21.15.39:0/2118002247 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f51b40ffbd0 0x7f51b419a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:10.410 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.382+0000 7f51b2d9d700 1 --2- 172.21.15.39:0/2118002247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f51b40fe9e0 0x7f51b40681e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:10.411 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.382+0000 7f51b359e700 1 -- 172.21.15.39:0/2118002247 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f51b40ffbd0 msgr2=0x7f51b419a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:10.411 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.382+0000 7f51b359e700 1 --2- 172.21.15.39:0/2118002247 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f51b40ffbd0 0x7f51b419a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:10.411 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.382+0000 7f51b359e700 1 -- 172.21.15.39:0/2118002247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f51b40fe9e0 msgr2=0x7f51b40681e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:10.411 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.382+0000 7f51b359e700 1 --2- 172.21.15.39:0/2118002247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f51b40fe9e0 0x7f51b40681e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:10.411 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.382+0000 7f51b359e700 1 -- 172.21.15.39:0/2118002247 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51ac009d00 con 0x7f51b4100560 2023-07-21T18:40:10.411 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.382+0000 7f51b359e700 1 --2- 172.21.15.39:0/2118002247 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51b4100560 0x7f51b419cd20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f51ac00d010 tx=0x7f51ac004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:10.411 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.382+0000 7f51abfff700 1 -- 172.21.15.39:0/2118002247 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ac00e040 con 0x7f51b4100560 2023-07-21T18:40:10.411 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.382+0000 7f51abfff700 1 -- 172.21.15.39:0/2118002247 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51ac004e30 con 0x7f51b4100560 2023-07-21T18:40:10.412 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.382+0000 7f51b9253700 1 -- 172.21.15.39:0/2118002247 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51b419da70 con 0x7f51b4100560 2023-07-21T18:40:10.412 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.382+0000 7f51abfff700 1 -- 172.21.15.39:0/2118002247 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ac004020 con 0x7f51b4100560 2023-07-21T18:40:10.412 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.383+0000 7f51b9253700 1 -- 172.21.15.39:0/2118002247 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51b419def0 con 0x7f51b4100560 2023-07-21T18:40:10.412 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.383+0000 7f51b9253700 1 -- 172.21.15.39:0/2118002247 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51b410a210 con 0x7f51b4100560 2023-07-21T18:40:10.413 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.385+0000 7f51abfff700 1 -- 172.21.15.39:0/2118002247 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f51ac01d690 con 0x7f51b4100560 2023-07-21T18:40:10.413 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.385+0000 7f51abfff700 1 --2- 172.21.15.39:0/2118002247 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f51a00a1150 0x7f51a00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:10.413 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.385+0000 7f51b2d9d700 1 --2- 172.21.15.39:0/2118002247 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f51a00a1150 0x7f51a00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:10.413 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.385+0000 7f51abfff700 1 -- 172.21.15.39:0/2118002247 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f51ac0ce5c0 con 0x7f51b4100560 2023-07-21T18:40:10.413 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.386+0000 7f51b2d9d700 1 --2- 172.21.15.39:0/2118002247 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f51a00a1150 0x7f51a00a3600 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f51b40ffa30 tx=0x7f51a400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:10.414 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.388+0000 7f51abfff700 1 -- 172.21.15.39:0/2118002247 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f51ac097ef0 con 0x7f51b4100560 2023-07-21T18:40:10.623 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:10 smithi193 ceph-mon[98204]: mds.? [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] up:active 2023-07-21T18:40:10.624 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:10 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/3252238689' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-07-21T18:40:10.624 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:10 smithi193 ceph-mon[98204]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-07-21T18:40:10.624 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:10 smithi193 ceph-mon[98204]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2023-07-21T18:40:10.624 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:10 smithi193 ceph-mon[98204]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2023-07-21T18:40:10.624 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:10 smithi193 ceph-mon[98204]: daemon mds.i is now active in filesystem cephfs as rank 2 2023-07-21T18:40:10.624 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:10 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/4148906896' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-07-21T18:40:10.642 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:10 smithi155 ceph-mon[97934]: mds.? [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] up:active 2023-07-21T18:40:10.642 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:10 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/3252238689' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-07-21T18:40:10.642 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:10 smithi155 ceph-mon[97934]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-07-21T18:40:10.643 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:10 smithi155 ceph-mon[97934]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2023-07-21T18:40:10.643 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:10 smithi155 ceph-mon[97934]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2023-07-21T18:40:10.643 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:10 smithi155 ceph-mon[97934]: daemon mds.i is now active in filesystem cephfs as rank 2 2023-07-21T18:40:10.643 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:10 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/4148906896' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-07-21T18:40:10.643 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.618+0000 7f51b9253700 1 -- 172.21.15.39:0/2118002247 --> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f51b4060950 con 0x7f51a00a1150 2023-07-21T18:40:10.650 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:10 smithi039 ceph-mon[96489]: mds.? [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] up:active 2023-07-21T18:40:10.651 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:10 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/3252238689' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-07-21T18:40:10.651 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:10 smithi039 ceph-mon[96489]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-07-21T18:40:10.651 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:10 smithi039 ceph-mon[96489]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2023-07-21T18:40:10.651 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:10 smithi039 ceph-mon[96489]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2023-07-21T18:40:10.651 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:10 smithi039 ceph-mon[96489]: daemon mds.i is now active in filesystem cephfs as rank 2 2023-07-21T18:40:10.651 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:10 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/4148906896' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-07-21T18:40:10.688 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.662+0000 7f51abfff700 1 -- 172.21.15.39:0/2118002247 <== mgr.14152 v2:172.21.15.39:6800/2116291712 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f51b4060950 con 0x7f51a00a1150 2023-07-21T18:40:10.690 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.664+0000 7f51b9253700 1 -- 172.21.15.39:0/2118002247 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f51a00a1150 msgr2=0x7f51a00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:10.690 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.664+0000 7f51b9253700 1 --2- 172.21.15.39:0/2118002247 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f51a00a1150 0x7f51a00a3600 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f51b40ffa30 tx=0x7f51a400b410 comp rx=0 tx=0).stop 2023-07-21T18:40:10.690 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.664+0000 7f51b9253700 1 -- 172.21.15.39:0/2118002247 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51b4100560 msgr2=0x7f51b419cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:10.691 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.664+0000 7f51b9253700 1 --2- 172.21.15.39:0/2118002247 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51b4100560 0x7f51b419cd20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f51ac00d010 tx=0x7f51ac004c80 comp rx=0 tx=0).stop 2023-07-21T18:40:10.691 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.664+0000 7f51b9253700 1 -- 172.21.15.39:0/2118002247 shutdown_connections 2023-07-21T18:40:10.691 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.664+0000 7f51b9253700 1 --2- 172.21.15.39:0/2118002247 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f51a00a1150 0x7f51a00a3600 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:10.692 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.664+0000 7f51b9253700 1 --2- 172.21.15.39:0/2118002247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f51b40fe9e0 0x7f51b40681e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:10.692 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.664+0000 7f51b9253700 1 --2- 172.21.15.39:0/2118002247 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51b4100560 0x7f51b419cd20 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:10.692 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.664+0000 7f51b9253700 1 --2- 172.21.15.39:0/2118002247 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f51b40ffbd0 0x7f51b419a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:10.692 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.664+0000 7f51b9253700 1 -- 172.21.15.39:0/2118002247 >> 172.21.15.39:0/2118002247 conn(0x7f51b40fa000 msgr2=0x7f51b40fc1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:10.692 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.664+0000 7f51b9253700 1 -- 172.21.15.39:0/2118002247 shutdown_connections 2023-07-21T18:40:10.693 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.664+0000 7f51b9253700 1 -- 172.21.15.39:0/2118002247 wait complete. 2023-07-21T18:40:10.703 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_0 2023-07-21T18:40:10.834 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.806+0000 7f07836d7700 1 -- 172.21.15.39:0/4076866377 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f077c0ffbc0 msgr2=0x7f077c100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:10.834 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.806+0000 7f07836d7700 1 --2- 172.21.15.39:0/4076866377 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f077c0ffbc0 0x7f077c100030 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0778009990 tx=0x7f0778009ca0 comp rx=0 tx=0).stop 2023-07-21T18:40:10.834 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.806+0000 7f07836d7700 1 -- 172.21.15.39:0/4076866377 shutdown_connections 2023-07-21T18:40:10.834 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.806+0000 7f07836d7700 1 --2- 172.21.15.39:0/4076866377 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f077c0feeb0 0x7f077c0ff2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:10.834 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.806+0000 7f07836d7700 1 --2- 172.21.15.39:0/4076866377 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f077c100570 0x7f077c106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:10.835 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.806+0000 7f07836d7700 1 --2- 172.21.15.39:0/4076866377 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f077c0ffbc0 0x7f077c100030 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:10.835 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.806+0000 7f07836d7700 1 -- 172.21.15.39:0/4076866377 >> 172.21.15.39:0/4076866377 conn(0x7f077c0f9730 msgr2=0x7f077c0fbb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:10.835 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.806+0000 7f07836d7700 1 -- 172.21.15.39:0/4076866377 shutdown_connections 2023-07-21T18:40:10.835 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.806+0000 7f07836d7700 1 -- 172.21.15.39:0/4076866377 wait complete. 2023-07-21T18:40:10.835 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.807+0000 7f07836d7700 1 Processor -- start 2023-07-21T18:40:10.836 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.807+0000 7f07836d7700 1 -- start start 2023-07-21T18:40:10.836 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.807+0000 7f07836d7700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f077c0feeb0 0x7f077c1957c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:10.836 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.807+0000 7f07836d7700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f077c0ffbc0 0x7f077c195d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:10.836 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.807+0000 7f07836d7700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f077c100570 0x7f077c19cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:10.836 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.807+0000 7f07836d7700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077c19d340 con 0x7f077c0ffbc0 2023-07-21T18:40:10.836 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.807+0000 7f07836d7700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077c19d480 con 0x7f077c0feeb0 2023-07-21T18:40:10.837 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.807+0000 7f07836d7700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077c19d610 con 0x7f077c100570 2023-07-21T18:40:10.837 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.807+0000 7f0780c72700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f077c0ffbc0 0x7f077c195d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:10.837 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.807+0000 7f0780c72700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f077c0ffbc0 0x7f077c195d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55134/0 (socket says 172.21.15.39:55134) 2023-07-21T18:40:10.837 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.807+0000 7f0780c72700 1 -- 172.21.15.39:0/2670349811 learned_addr learned my addr 172.21.15.39:0/2670349811 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:40:10.837 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.807+0000 7f0781473700 1 --2- 172.21.15.39:0/2670349811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f077c0feeb0 0x7f077c1957c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:10.837 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.807+0000 7f0781c74700 1 --2- 172.21.15.39:0/2670349811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f077c100570 0x7f077c19cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:10.837 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.808+0000 7f0780c72700 1 -- 172.21.15.39:0/2670349811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f077c100570 msgr2=0x7f077c19cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:10.837 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.808+0000 7f0780c72700 1 --2- 172.21.15.39:0/2670349811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f077c100570 0x7f077c19cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:10.838 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.808+0000 7f0780c72700 1 -- 172.21.15.39:0/2670349811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f077c0feeb0 msgr2=0x7f077c1957c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:10.838 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.808+0000 7f0780c72700 1 --2- 172.21.15.39:0/2670349811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f077c0feeb0 0x7f077c1957c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:10.838 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.808+0000 7f0780c72700 1 -- 172.21.15.39:0/2670349811 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07780095f0 con 0x7f077c0ffbc0 2023-07-21T18:40:10.838 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.808+0000 7f0780c72700 1 --2- 172.21.15.39:0/2670349811 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f077c0ffbc0 0x7f077c195d00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f07780095c0 tx=0x7f077800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:10.838 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.808+0000 7f076e7fc700 1 -- 172.21.15.39:0/2670349811 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077801d070 con 0x7f077c0ffbc0 2023-07-21T18:40:10.838 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.808+0000 7f076e7fc700 1 -- 172.21.15.39:0/2670349811 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0778005030 con 0x7f077c0ffbc0 2023-07-21T18:40:10.838 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.808+0000 7f07836d7700 1 -- 172.21.15.39:0/2670349811 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f077c19d890 con 0x7f077c0ffbc0 2023-07-21T18:40:10.839 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.808+0000 7f076e7fc700 1 -- 172.21.15.39:0/2670349811 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0778004ab0 con 0x7f077c0ffbc0 2023-07-21T18:40:10.839 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.808+0000 7f07836d7700 1 -- 172.21.15.39:0/2670349811 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f077c19dcb0 con 0x7f077c0ffbc0 2023-07-21T18:40:10.840 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.809+0000 7f07836d7700 1 -- 172.21.15.39:0/2670349811 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f077c10a170 con 0x7f077c0ffbc0 2023-07-21T18:40:10.840 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.810+0000 7f076e7fc700 1 -- 172.21.15.39:0/2670349811 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f0778004c50 con 0x7f077c0ffbc0 2023-07-21T18:40:10.840 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.810+0000 7f076e7fc700 1 --2- 172.21.15.39:0/2670349811 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f07680a1150 0x7f07680a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:10.840 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.811+0000 7f0781473700 1 --2- 172.21.15.39:0/2670349811 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f07680a1150 0x7f07680a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:10.840 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.811+0000 7f076e7fc700 1 -- 172.21.15.39:0/2670349811 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f07780cdb90 con 0x7f077c0ffbc0 2023-07-21T18:40:10.841 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.811+0000 7f0781473700 1 --2- 172.21.15.39:0/2670349811 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f07680a1150 0x7f07680a3600 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f077c0671e0 tx=0x7f077000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:10.841 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:10.814+0000 7f076e7fc700 1 -- 172.21.15.39:0/2670349811 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f0778097420 con 0x7f077c0ffbc0 2023-07-21T18:40:10.941 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:40:10 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:40:10.623+0000 7fe42d0b8700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T18:40:10.941 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:40:10 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:40:10.623+0000 7fe42d0b8700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T18:40:10.941 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:40:10 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:40:10.623+0000 7fe42d0b8700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T18:40:10.942 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:40:10 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:40:10.623+0000 7fe42d0b8700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T18:40:10.942 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:40:10 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:40:10.623+0000 7fe42d0b8700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T18:40:11.082 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.057+0000 7f07836d7700 1 -- 172.21.15.39:0/2670349811 --> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7f077c0fe130 con 0x7f07680a1150 2023-07-21T18:40:11.087 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.061+0000 7f076e7fc700 1 -- 172.21.15.39:0/2670349811 <== mgr.14152 v2:172.21.15.39:6800/2116291712 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7f077c0fe130 con 0x7f07680a1150 2023-07-21T18:40:11.087 INFO:teuthology.orchestra.run.smithi039.stdout:/volumes/_nogroup/sv_0/3bafb205-ece8-4d0d-bb97-282635031169 2023-07-21T18:40:11.089 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.063+0000 7f07836d7700 1 -- 172.21.15.39:0/2670349811 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f07680a1150 msgr2=0x7f07680a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:11.089 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.063+0000 7f07836d7700 1 --2- 172.21.15.39:0/2670349811 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f07680a1150 0x7f07680a3600 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f077c0671e0 tx=0x7f077000b410 comp rx=0 tx=0).stop 2023-07-21T18:40:11.089 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.063+0000 7f07836d7700 1 -- 172.21.15.39:0/2670349811 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f077c0ffbc0 msgr2=0x7f077c195d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:11.089 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.063+0000 7f07836d7700 1 --2- 172.21.15.39:0/2670349811 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f077c0ffbc0 0x7f077c195d00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f07780095c0 tx=0x7f077800f690 comp rx=0 tx=0).stop 2023-07-21T18:40:11.089 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.063+0000 7f07836d7700 1 -- 172.21.15.39:0/2670349811 shutdown_connections 2023-07-21T18:40:11.090 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.063+0000 7f07836d7700 1 --2- 172.21.15.39:0/2670349811 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f07680a1150 0x7f07680a3600 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:11.090 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.063+0000 7f07836d7700 1 --2- 172.21.15.39:0/2670349811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f077c0feeb0 0x7f077c1957c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:11.090 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.063+0000 7f07836d7700 1 --2- 172.21.15.39:0/2670349811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f077c100570 0x7f077c19cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:11.090 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.063+0000 7f07836d7700 1 --2- 172.21.15.39:0/2670349811 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f077c0ffbc0 0x7f077c195d00 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:11.090 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.063+0000 7f07836d7700 1 -- 172.21.15.39:0/2670349811 >> 172.21.15.39:0/2670349811 conn(0x7f077c0f9730 msgr2=0x7f077c0fda20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:11.090 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.063+0000 7f07836d7700 1 -- 172.21.15.39:0/2670349811 shutdown_connections 2023-07-21T18:40:11.090 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.064+0000 7f07836d7700 1 -- 172.21.15.39:0/2670349811 wait complete. 2023-07-21T18:40:11.101 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 '' 2023-07-21T18:40:11.232 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.207+0000 7fc5f88c1700 1 -- 172.21.15.39:0/1931923689 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc5f40fdb20 msgr2=0x7fc5f40fdf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:11.233 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.207+0000 7fc5f88c1700 1 --2- 172.21.15.39:0/1931923689 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc5f40fdb20 0x7fc5f40fdf70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc5e4009990 tx=0x7fc5e4009ca0 comp rx=0 tx=0).stop 2023-07-21T18:40:11.233 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.207+0000 7fc5f88c1700 1 -- 172.21.15.39:0/1931923689 shutdown_connections 2023-07-21T18:40:11.233 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.207+0000 7fc5f88c1700 1 --2- 172.21.15.39:0/1931923689 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5f40fe4b0 0x7fc5f4108090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:11.233 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.207+0000 7fc5f88c1700 1 --2- 172.21.15.39:0/1931923689 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5f41010b0 0x7fc5f41014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:11.233 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.207+0000 7fc5f88c1700 1 --2- 172.21.15.39:0/1931923689 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc5f40fdb20 0x7fc5f40fdf70 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:11.234 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.207+0000 7fc5f88c1700 1 -- 172.21.15.39:0/1931923689 >> 172.21.15.39:0/1931923689 conn(0x7fc5f40f9770 msgr2=0x7fc5f40fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:11.234 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.207+0000 7fc5f88c1700 1 -- 172.21.15.39:0/1931923689 shutdown_connections 2023-07-21T18:40:11.234 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.207+0000 7fc5f88c1700 1 -- 172.21.15.39:0/1931923689 wait complete. 2023-07-21T18:40:11.236 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.208+0000 7fc5f88c1700 1 Processor -- start 2023-07-21T18:40:11.236 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.208+0000 7fc5f88c1700 1 -- start start 2023-07-21T18:40:11.236 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.208+0000 7fc5f88c1700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5f40fdb20 0x7fc5f41007b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:11.236 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.208+0000 7fc5f88c1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc5f40fe4b0 0x7fc5f40fedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:11.237 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.208+0000 7fc5f88c1700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5f41010b0 0x7fc5f40ff330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:11.237 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.208+0000 7fc5f88c1700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f40ffad0 con 0x7fc5f40fe4b0 2023-07-21T18:40:11.237 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.208+0000 7fc5f88c1700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f40ffc40 con 0x7fc5f40fdb20 2023-07-21T18:40:11.237 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.208+0000 7fc5f88c1700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f419f8a0 con 0x7fc5f41010b0 2023-07-21T18:40:11.237 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.208+0000 7fc5f1d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc5f40fe4b0 0x7fc5f40fedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:11.237 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.208+0000 7fc5f1d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc5f40fe4b0 0x7fc5f40fedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55166/0 (socket says 172.21.15.39:55166) 2023-07-21T18:40:11.237 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.208+0000 7fc5f1d9b700 1 -- 172.21.15.39:0/1477787037 learned_addr learned my addr 172.21.15.39:0/1477787037 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:40:11.237 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.209+0000 7fc5f259c700 1 --2- 172.21.15.39:0/1477787037 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5f40fdb20 0x7fc5f41007b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:11.238 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.209+0000 7fc5f2d9d700 1 --2- 172.21.15.39:0/1477787037 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5f41010b0 0x7fc5f40ff330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:11.238 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.209+0000 7fc5f1d9b700 1 -- 172.21.15.39:0/1477787037 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5f41010b0 msgr2=0x7fc5f40ff330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:11.238 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.209+0000 7fc5f1d9b700 1 --2- 172.21.15.39:0/1477787037 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5f41010b0 0x7fc5f40ff330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:11.238 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.209+0000 7fc5f1d9b700 1 -- 172.21.15.39:0/1477787037 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5f40fdb20 msgr2=0x7fc5f41007b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:11.238 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.209+0000 7fc5f1d9b700 1 --2- 172.21.15.39:0/1477787037 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5f40fdb20 0x7fc5f41007b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:11.238 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.209+0000 7fc5f1d9b700 1 -- 172.21.15.39:0/1477787037 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5e40095f0 con 0x7fc5f40fe4b0 2023-07-21T18:40:11.239 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.209+0000 7fc5f1d9b700 1 --2- 172.21.15.39:0/1477787037 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc5f40fe4b0 0x7fc5f40fedf0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc5e4005910 tx=0x7fc5e400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:11.239 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.209+0000 7fc5e37fe700 1 -- 172.21.15.39:0/1477787037 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e401d070 con 0x7fc5f40fe4b0 2023-07-21T18:40:11.239 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.209+0000 7fc5e37fe700 1 -- 172.21.15.39:0/1477787037 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5e400fb80 con 0x7fc5f40fe4b0 2023-07-21T18:40:11.239 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.209+0000 7fc5e37fe700 1 -- 172.21.15.39:0/1477787037 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e4017650 con 0x7fc5f40fe4b0 2023-07-21T18:40:11.239 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.209+0000 7fc5f88c1700 1 -- 172.21.15.39:0/1477787037 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5f419fb20 con 0x7fc5f40fe4b0 2023-07-21T18:40:11.239 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.209+0000 7fc5f88c1700 1 -- 172.21.15.39:0/1477787037 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5f419ff40 con 0x7fc5f40fe4b0 2023-07-21T18:40:11.239 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.210+0000 7fc5f88c1700 1 -- 172.21.15.39:0/1477787037 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5f4106bd0 con 0x7fc5f40fe4b0 2023-07-21T18:40:11.240 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.211+0000 7fc5e37fe700 1 -- 172.21.15.39:0/1477787037 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fc5e4022470 con 0x7fc5f40fe4b0 2023-07-21T18:40:11.240 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.212+0000 7fc5e37fe700 1 --2- 172.21.15.39:0/1477787037 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fc5dc0a11d0 0x7fc5dc0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:11.241 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.212+0000 7fc5f259c700 1 --2- 172.21.15.39:0/1477787037 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fc5dc0a11d0 0x7fc5dc0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:11.241 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.212+0000 7fc5e37fe700 1 -- 172.21.15.39:0/1477787037 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc5e40ce100 con 0x7fc5f40fe4b0 2023-07-21T18:40:11.241 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.213+0000 7fc5f259c700 1 --2- 172.21.15.39:0/1477787037 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fc5dc0a11d0 0x7fc5dc0a3680 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc5f4067390 tx=0x7fc5e800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:11.242 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.216+0000 7fc5e37fe700 1 -- 172.21.15.39:0/1477787037 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fc5e4097a30 con 0x7fc5f40fe4b0 2023-07-21T18:40:11.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:11 smithi039 ceph-mon[96489]: mds.? [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] up:active 2023-07-21T18:40:11.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:11 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/4148906896' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-07-21T18:40:11.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:11 smithi039 ceph-mon[96489]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-07-21T18:40:11.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:11 smithi039 ceph-mon[96489]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2023-07-21T18:40:11.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:11 smithi039 ceph-mon[96489]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=e=up:creating} 8 up:standby 2023-07-21T18:40:11.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:11 smithi039 ceph-mon[96489]: daemon mds.e is now active in filesystem cephfs as rank 3 2023-07-21T18:40:11.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:11 smithi039 ceph-mon[96489]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:40:11.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:11 smithi039 ceph-mon[96489]: mds.? [v2:172.21.15.155:6834/3854856286,v1:172.21.15.155:6835/3854856286] up:active 2023-07-21T18:40:11.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:11 smithi039 ceph-mon[96489]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-07-21T18:40:11.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:11 smithi039 ceph-mon[96489]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-07-21T18:40:11.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:11 smithi039 ceph-mon[96489]: Cluster is now healthy 2023-07-21T18:40:11.443 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:11 smithi039 ceph-mon[96489]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=e=up:active} 4 up:standby-replay 4 up:standby 2023-07-21T18:40:11.443 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:11 smithi039 ceph-mon[96489]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2023-07-21T18:40:11.443 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:11 smithi039 ceph-mon[96489]: daemon mds.a is now active in filesystem cephfs as rank 4 2023-07-21T18:40:11.443 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:11 smithi039 ceph-mon[96489]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:40:11.443 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:11 smithi039 ceph-mon[96489]: pgmap v251: 97 pgs: 97 active+clean; 582 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 5.5 KiB/s wr, 17 op/s 2023-07-21T18:40:11.473 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.447+0000 7fc5f88c1700 1 -- 172.21.15.39:0/1477787037 --> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7fc5f4060950 con 0x7fc5dc0a11d0 2023-07-21T18:40:11.499 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.473+0000 7fc5e37fe700 1 -- 172.21.15.39:0/1477787037 <== mgr.14152 v2:172.21.15.39:6800/2116291712 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fc5f4060950 con 0x7fc5dc0a11d0 2023-07-21T18:40:11.501 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.475+0000 7fc5f88c1700 1 -- 172.21.15.39:0/1477787037 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fc5dc0a11d0 msgr2=0x7fc5dc0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:11.501 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.475+0000 7fc5f88c1700 1 --2- 172.21.15.39:0/1477787037 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fc5dc0a11d0 0x7fc5dc0a3680 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc5f4067390 tx=0x7fc5e800b410 comp rx=0 tx=0).stop 2023-07-21T18:40:11.501 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.475+0000 7fc5f88c1700 1 -- 172.21.15.39:0/1477787037 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc5f40fe4b0 msgr2=0x7fc5f40fedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:11.502 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.475+0000 7fc5f88c1700 1 --2- 172.21.15.39:0/1477787037 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc5f40fe4b0 0x7fc5f40fedf0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc5e4005910 tx=0x7fc5e400f690 comp rx=0 tx=0).stop 2023-07-21T18:40:11.502 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.476+0000 7fc5f88c1700 1 -- 172.21.15.39:0/1477787037 shutdown_connections 2023-07-21T18:40:11.502 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.476+0000 7fc5f88c1700 1 --2- 172.21.15.39:0/1477787037 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fc5dc0a11d0 0x7fc5dc0a3680 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:11.502 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.476+0000 7fc5f88c1700 1 --2- 172.21.15.39:0/1477787037 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5f40fdb20 0x7fc5f41007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:11.502 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.476+0000 7fc5f88c1700 1 --2- 172.21.15.39:0/1477787037 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5f41010b0 0x7fc5f40ff330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:11.502 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.476+0000 7fc5f88c1700 1 --2- 172.21.15.39:0/1477787037 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc5f40fe4b0 0x7fc5f40fedf0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:11.502 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.476+0000 7fc5f88c1700 1 -- 172.21.15.39:0/1477787037 >> 172.21.15.39:0/1477787037 conn(0x7fc5f40f9770 msgr2=0x7fc5f40fba00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:11.503 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.476+0000 7fc5f88c1700 1 -- 172.21.15.39:0/1477787037 shutdown_connections 2023-07-21T18:40:11.503 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.476+0000 7fc5f88c1700 1 -- 172.21.15.39:0/1477787037 wait complete. 2023-07-21T18:40:11.513 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_1 2023-07-21T18:40:11.623 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:11 smithi193 ceph-mon[98204]: mds.? [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] up:active 2023-07-21T18:40:11.624 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:11 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/4148906896' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-07-21T18:40:11.624 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:11 smithi193 ceph-mon[98204]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-07-21T18:40:11.624 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:11 smithi193 ceph-mon[98204]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2023-07-21T18:40:11.624 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:11 smithi193 ceph-mon[98204]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=e=up:creating} 8 up:standby 2023-07-21T18:40:11.624 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:11 smithi193 ceph-mon[98204]: daemon mds.e is now active in filesystem cephfs as rank 3 2023-07-21T18:40:11.624 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:11 smithi193 ceph-mon[98204]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:40:11.624 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:11 smithi193 ceph-mon[98204]: mds.? [v2:172.21.15.155:6834/3854856286,v1:172.21.15.155:6835/3854856286] up:active 2023-07-21T18:40:11.625 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:11 smithi193 ceph-mon[98204]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-07-21T18:40:11.625 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:11 smithi193 ceph-mon[98204]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-07-21T18:40:11.625 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:11 smithi193 ceph-mon[98204]: Cluster is now healthy 2023-07-21T18:40:11.625 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:11 smithi193 ceph-mon[98204]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=e=up:active} 4 up:standby-replay 4 up:standby 2023-07-21T18:40:11.625 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:11 smithi193 ceph-mon[98204]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2023-07-21T18:40:11.625 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:11 smithi193 ceph-mon[98204]: daemon mds.a is now active in filesystem cephfs as rank 4 2023-07-21T18:40:11.625 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:11 smithi193 ceph-mon[98204]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:40:11.626 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:11 smithi193 ceph-mon[98204]: pgmap v251: 97 pgs: 97 active+clean; 582 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 5.5 KiB/s wr, 17 op/s 2023-07-21T18:40:11.642 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.616+0000 7f83997e3700 1 -- 172.21.15.39:0/3726750928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83940ffbd0 msgr2=0x7f8394100020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:11.642 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.616+0000 7f83997e3700 1 --2- 172.21.15.39:0/3726750928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83940ffbd0 0x7f8394100020 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8384009990 tx=0x7f8384009ca0 comp rx=0 tx=0).stop 2023-07-21T18:40:11.642 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.616+0000 7f83997e3700 1 -- 172.21.15.39:0/3726750928 shutdown_connections 2023-07-21T18:40:11.642 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.616+0000 7f83997e3700 1 --2- 172.21.15.39:0/3726750928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8394100560 0x7f8394106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:11.642 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.616+0000 7f83997e3700 1 --2- 172.21.15.39:0/3726750928 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83940fe9e0 0x7f83940fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:11.643 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.616+0000 7f83997e3700 1 --2- 172.21.15.39:0/3726750928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83940ffbd0 0x7f8394100020 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:11.643 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.616+0000 7f83997e3700 1 -- 172.21.15.39:0/3726750928 >> 172.21.15.39:0/3726750928 conn(0x7f83940fa000 msgr2=0x7f83940fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:11.643 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.616+0000 7f83997e3700 1 -- 172.21.15.39:0/3726750928 shutdown_connections 2023-07-21T18:40:11.643 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.616+0000 7f83997e3700 1 -- 172.21.15.39:0/3726750928 wait complete. 2023-07-21T18:40:11.643 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.616+0000 7f83997e3700 1 Processor -- start 2023-07-21T18:40:11.644 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:11 smithi155 ceph-mon[97934]: mds.? [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] up:active 2023-07-21T18:40:11.644 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:11 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/4148906896' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-07-21T18:40:11.644 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:11 smithi155 ceph-mon[97934]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-07-21T18:40:11.644 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:11 smithi155 ceph-mon[97934]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2023-07-21T18:40:11.644 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:11 smithi155 ceph-mon[97934]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=e=up:creating} 8 up:standby 2023-07-21T18:40:11.644 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:11 smithi155 ceph-mon[97934]: daemon mds.e is now active in filesystem cephfs as rank 3 2023-07-21T18:40:11.645 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:11 smithi155 ceph-mon[97934]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:40:11.645 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:11 smithi155 ceph-mon[97934]: mds.? [v2:172.21.15.155:6834/3854856286,v1:172.21.15.155:6835/3854856286] up:active 2023-07-21T18:40:11.645 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:11 smithi155 ceph-mon[97934]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-07-21T18:40:11.645 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:11 smithi155 ceph-mon[97934]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-07-21T18:40:11.645 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:11 smithi155 ceph-mon[97934]: Cluster is now healthy 2023-07-21T18:40:11.645 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:11 smithi155 ceph-mon[97934]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=e=up:active} 4 up:standby-replay 4 up:standby 2023-07-21T18:40:11.645 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:11 smithi155 ceph-mon[97934]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2023-07-21T18:40:11.645 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:11 smithi155 ceph-mon[97934]: daemon mds.a is now active in filesystem cephfs as rank 4 2023-07-21T18:40:11.646 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:11 smithi155 ceph-mon[97934]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:40:11.646 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:11 smithi155 ceph-mon[97934]: pgmap v251: 97 pgs: 97 active+clean; 582 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 5.5 KiB/s wr, 17 op/s 2023-07-21T18:40:11.647 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.617+0000 7f83997e3700 1 -- start start 2023-07-21T18:40:11.648 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.617+0000 7f83997e3700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83940fe9e0 0x7f8394068200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:11.648 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.617+0000 7f83997e3700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83940ffbd0 0x7f839419a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:11.648 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.617+0000 7f83997e3700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8394100560 0x7f839419cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:11.648 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.617+0000 7f83997e3700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f839419d3a0 con 0x7f8394100560 2023-07-21T18:40:11.648 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.617+0000 7f83997e3700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f839419d510 con 0x7f83940ffbd0 2023-07-21T18:40:11.649 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.617+0000 7f83997e3700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f839419d6d0 con 0x7f83940fe9e0 2023-07-21T18:40:11.649 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.617+0000 7f83937fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8394100560 0x7f839419cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:11.649 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.617+0000 7f83937fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8394100560 0x7f839419cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55184/0 (socket says 172.21.15.39:55184) 2023-07-21T18:40:11.649 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.617+0000 7f83937fe700 1 -- 172.21.15.39:0/3537413448 learned_addr learned my addr 172.21.15.39:0/3537413448 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:40:11.649 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.617+0000 7f8392ffd700 1 --2- 172.21.15.39:0/3537413448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83940fe9e0 0x7f8394068200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:11.649 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.617+0000 7f83927fc700 1 --2- 172.21.15.39:0/3537413448 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83940ffbd0 0x7f839419a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:11.649 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.617+0000 7f83937fe700 1 -- 172.21.15.39:0/3537413448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83940fe9e0 msgr2=0x7f8394068200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:11.649 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.617+0000 7f83937fe700 1 --2- 172.21.15.39:0/3537413448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83940fe9e0 0x7f8394068200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:11.650 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.617+0000 7f83937fe700 1 -- 172.21.15.39:0/3537413448 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83940ffbd0 msgr2=0x7f839419a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:11.650 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.617+0000 7f83937fe700 1 --2- 172.21.15.39:0/3537413448 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83940ffbd0 0x7f839419a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:11.650 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.617+0000 7f83937fe700 1 -- 172.21.15.39:0/3537413448 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83840095f0 con 0x7f8394100560 2023-07-21T18:40:11.650 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.618+0000 7f83937fe700 1 --2- 172.21.15.39:0/3537413448 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8394100560 0x7f839419cbd0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f838c00ada0 tx=0x7f838c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:11.650 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.618+0000 7f837bfff700 1 -- 172.21.15.39:0/3537413448 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f838c00f070 con 0x7f8394100560 2023-07-21T18:40:11.650 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.618+0000 7f837bfff700 1 -- 172.21.15.39:0/3537413448 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f838c007b30 con 0x7f8394100560 2023-07-21T18:40:11.650 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.618+0000 7f83997e3700 1 -- 172.21.15.39:0/3537413448 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f839419d9b0 con 0x7f8394100560 2023-07-21T18:40:11.651 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.618+0000 7f837bfff700 1 -- 172.21.15.39:0/3537413448 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f838c005280 con 0x7f8394100560 2023-07-21T18:40:11.651 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.618+0000 7f83997e3700 1 -- 172.21.15.39:0/3537413448 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f839419de70 con 0x7f8394100560 2023-07-21T18:40:11.651 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.619+0000 7f83997e3700 1 -- 172.21.15.39:0/3537413448 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8394105fc0 con 0x7f8394100560 2023-07-21T18:40:11.651 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.620+0000 7f837bfff700 1 -- 172.21.15.39:0/3537413448 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f838c00c8d0 con 0x7f8394100560 2023-07-21T18:40:11.651 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.620+0000 7f837bfff700 1 --2- 172.21.15.39:0/3537413448 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f837c0a1150 0x7f837c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:11.651 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.621+0000 7f837bfff700 1 -- 172.21.15.39:0/3537413448 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f838c0cbdc0 con 0x7f8394100560 2023-07-21T18:40:11.651 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.621+0000 7f8392ffd700 1 --2- 172.21.15.39:0/3537413448 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f837c0a1150 0x7f837c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:11.651 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.621+0000 7f8392ffd700 1 --2- 172.21.15.39:0/3537413448 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f837c0a1150 0x7f837c0a3600 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f83940ffa30 tx=0x7f838800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:11.652 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.625+0000 7f837bfff700 1 -- 172.21.15.39:0/3537413448 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f838c0990e0 con 0x7f8394100560 2023-07-21T18:40:11.883 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.857+0000 7f83997e3700 1 -- 172.21.15.39:0/3537413448 --> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f8394060950 con 0x7f837c0a1150 2023-07-21T18:40:11.887 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.862+0000 7f837bfff700 1 -- 172.21.15.39:0/3537413448 <== mgr.14152 v2:172.21.15.39:6800/2116291712 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7f8394060950 con 0x7f837c0a1150 2023-07-21T18:40:11.888 INFO:teuthology.orchestra.run.smithi039.stdout:/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a 2023-07-21T18:40:11.890 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.864+0000 7f83997e3700 1 -- 172.21.15.39:0/3537413448 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f837c0a1150 msgr2=0x7f837c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:11.890 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.864+0000 7f83997e3700 1 --2- 172.21.15.39:0/3537413448 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f837c0a1150 0x7f837c0a3600 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f83940ffa30 tx=0x7f838800a380 comp rx=0 tx=0).stop 2023-07-21T18:40:11.890 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.864+0000 7f83997e3700 1 -- 172.21.15.39:0/3537413448 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8394100560 msgr2=0x7f839419cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:11.890 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.864+0000 7f83997e3700 1 --2- 172.21.15.39:0/3537413448 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8394100560 0x7f839419cbd0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f838c00ada0 tx=0x7f838c00c330 comp rx=0 tx=0).stop 2023-07-21T18:40:11.890 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.864+0000 7f83997e3700 1 -- 172.21.15.39:0/3537413448 shutdown_connections 2023-07-21T18:40:11.890 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.864+0000 7f83997e3700 1 --2- 172.21.15.39:0/3537413448 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f837c0a1150 0x7f837c0a3600 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:11.891 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.864+0000 7f83997e3700 1 --2- 172.21.15.39:0/3537413448 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8394100560 0x7f839419cbd0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:11.891 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.864+0000 7f83997e3700 1 --2- 172.21.15.39:0/3537413448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83940fe9e0 0x7f8394068200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:11.891 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.864+0000 7f83997e3700 1 --2- 172.21.15.39:0/3537413448 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83940ffbd0 0x7f839419a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:11.891 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.864+0000 7f83997e3700 1 -- 172.21.15.39:0/3537413448 >> 172.21.15.39:0/3537413448 conn(0x7f83940fa000 msgr2=0x7f8394103100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:11.891 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.865+0000 7f83997e3700 1 -- 172.21.15.39:0/3537413448 shutdown_connections 2023-07-21T18:40:11.892 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:11.865+0000 7f83997e3700 1 -- 172.21.15.39:0/3537413448 wait complete. 2023-07-21T18:40:11.902 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T18:40:12.038 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.013+0000 7f99805e4700 1 -- 172.21.15.39:0/681444540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99781005b0 msgr2=0x7f9978102990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:12.039 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.013+0000 7f99805e4700 1 --2- 172.21.15.39:0/681444540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99781005b0 0x7f9978102990 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f996800b0a0 tx=0x7f996800b3b0 comp rx=0 tx=0).stop 2023-07-21T18:40:12.039 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.013+0000 7f99805e4700 1 -- 172.21.15.39:0/681444540 shutdown_connections 2023-07-21T18:40:12.039 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.013+0000 7f99805e4700 1 --2- 172.21.15.39:0/681444540 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f99781057f0 0x7f9978107bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:12.039 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.013+0000 7f99805e4700 1 --2- 172.21.15.39:0/681444540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99781005b0 0x7f9978102990 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:12.039 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.013+0000 7f99805e4700 1 --2- 172.21.15.39:0/681444540 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9978102ed0 0x7f99781052b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:12.039 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.013+0000 7f99805e4700 1 -- 172.21.15.39:0/681444540 >> 172.21.15.39:0/681444540 conn(0x7f99780f9fd0 msgr2=0x7f99780fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:12.039 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.013+0000 7f99805e4700 1 -- 172.21.15.39:0/681444540 shutdown_connections 2023-07-21T18:40:12.040 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.013+0000 7f99805e4700 1 -- 172.21.15.39:0/681444540 wait complete. 2023-07-21T18:40:12.041 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.014+0000 7f99805e4700 1 Processor -- start 2023-07-21T18:40:12.041 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.014+0000 7f99805e4700 1 -- start start 2023-07-21T18:40:12.042 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.014+0000 7f99805e4700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f99781005b0 0x7f9978195920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:12.042 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.014+0000 7f99805e4700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9978102ed0 0x7f9978195e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:12.042 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.014+0000 7f99805e4700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f99781057f0 0x7f997819ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:12.042 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.014+0000 7f99805e4700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f997819d4a0 con 0x7f9978102ed0 2023-07-21T18:40:12.042 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.014+0000 7f99805e4700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f997819d5e0 con 0x7f99781005b0 2023-07-21T18:40:12.042 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.014+0000 7f99805e4700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f997819d770 con 0x7f99781057f0 2023-07-21T18:40:12.042 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.014+0000 7f997db7f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9978102ed0 0x7f9978195e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:12.042 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.014+0000 7f997db7f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9978102ed0 0x7f9978195e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55210/0 (socket says 172.21.15.39:55210) 2023-07-21T18:40:12.043 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.014+0000 7f997db7f700 1 -- 172.21.15.39:0/1917891167 learned_addr learned my addr 172.21.15.39:0/1917891167 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:40:12.043 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.014+0000 7f997eb81700 1 --2- 172.21.15.39:0/1917891167 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f99781057f0 0x7f997819ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:12.043 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.014+0000 7f997e380700 1 --2- 172.21.15.39:0/1917891167 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f99781005b0 0x7f9978195920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:12.043 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.014+0000 7f997db7f700 1 -- 172.21.15.39:0/1917891167 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f99781057f0 msgr2=0x7f997819ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:12.043 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.014+0000 7f997db7f700 1 --2- 172.21.15.39:0/1917891167 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f99781057f0 0x7f997819ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:12.043 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.014+0000 7f997db7f700 1 -- 172.21.15.39:0/1917891167 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f99781005b0 msgr2=0x7f9978195920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:12.043 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.014+0000 7f997db7f700 1 --2- 172.21.15.39:0/1917891167 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f99781005b0 0x7f9978195920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:12.044 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.014+0000 7f997db7f700 1 -- 172.21.15.39:0/1917891167 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9968009d00 con 0x7f9978102ed0 2023-07-21T18:40:12.044 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.014+0000 7f997db7f700 1 --2- 172.21.15.39:0/1917891167 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9978102ed0 0x7f9978195e60 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f996c00d950 tx=0x7f996c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:12.044 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.014+0000 7f99777fe700 1 -- 172.21.15.39:0/1917891167 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996c009930 con 0x7f9978102ed0 2023-07-21T18:40:12.044 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.015+0000 7f99805e4700 1 -- 172.21.15.39:0/1917891167 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f997819da50 con 0x7f9978102ed0 2023-07-21T18:40:12.044 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.015+0000 7f99777fe700 1 -- 172.21.15.39:0/1917891167 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f996c010460 con 0x7f9978102ed0 2023-07-21T18:40:12.044 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.015+0000 7f99777fe700 1 -- 172.21.15.39:0/1917891167 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996c00f620 con 0x7f9978102ed0 2023-07-21T18:40:12.044 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.015+0000 7f99805e4700 1 -- 172.21.15.39:0/1917891167 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f997819df60 con 0x7f9978102ed0 2023-07-21T18:40:12.044 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.016+0000 7f99805e4700 1 -- 172.21.15.39:0/1917891167 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99780fbb40 con 0x7f9978102ed0 2023-07-21T18:40:12.045 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.016+0000 7f99777fe700 1 -- 172.21.15.39:0/1917891167 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f996c0108d0 con 0x7f9978102ed0 2023-07-21T18:40:12.045 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.017+0000 7f99777fe700 1 --2- 172.21.15.39:0/1917891167 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f99600a1220 0x7f99600a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:12.046 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.017+0000 7f997e380700 1 --2- 172.21.15.39:0/1917891167 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f99600a1220 0x7f99600a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:12.046 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.017+0000 7f99777fe700 1 -- 172.21.15.39:0/1917891167 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f996c0cc6b0 con 0x7f9978102ed0 2023-07-21T18:40:12.046 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.018+0000 7f997e380700 1 --2- 172.21.15.39:0/1917891167 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f99600a1220 0x7f99600a36d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9968007ab0 tx=0x7f996800b7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:12.047 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.021+0000 7f99777fe700 1 -- 172.21.15.39:0/1917891167 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f996c095e50 con 0x7f9978102ed0 2023-07-21T18:40:12.295 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.270+0000 7f99805e4700 1 -- 172.21.15.39:0/1917891167 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f997802ca10 con 0x7f9978102ed0 2023-07-21T18:40:12.296 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.271+0000 7f99777fe700 1 -- 172.21.15.39:0/1917891167 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10887 (secure 0 0 0) 0x7f996c099470 con 0x7f9978102ed0 2023-07-21T18:40:12.296 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:40:12.297 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/147318681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":147318681},{"type":"v1","addr":"172.21.15.39:6835","nonce":147318681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/1062891760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1062891760},{"type":"v1","addr":"172.21.15.39:6837","nonce":1062891760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T18:40:04.594553+0000","modified":"2023-07-21T18:40:11.989039+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24536,"mds_1":24500,"mds_2":24487,"mds_3":24464,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/3184273349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":3184273349},{"type":"v1","addr":"172.21.15.39:6839","nonce":3184273349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6841/640981220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":640981220},{"type":"v1","addr":"172.21.15.39:6841","nonce":640981220}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3106166804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3106166804},{"type":"v1","addr":"172.21.15.193:6833","nonce":3106166804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24437":{"gid":24437,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/1034755277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1034755277},{"type":"v1","addr":"172.21.15.155:6833","nonce":1034755277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/381560657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":381560657},{"type":"v1","addr":"172.21.15.193:6835","nonce":381560657}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6837/2626761843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2626761843},{"type":"v1","addr":"172.21.15.193:6837","nonce":2626761843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24464":{"gid":24464,"name":"e","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.155:6835/3854856286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3854856286},{"type":"v1","addr":"172.21.15.155:6835","nonce":3854856286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.193:6839/3148488639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3148488639},{"type":"v1","addr":"172.21.15.193:6839","nonce":3148488639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.155:6837/947670056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":947670056},{"type":"v1","addr":"172.21.15.155:6837","nonce":947670056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6839/431690342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":431690342},{"type":"v1","addr":"172.21.15.155:6839","nonce":431690342}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T18:40:12.299 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.274+0000 7f99805e4700 1 -- 172.21.15.39:0/1917891167 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f99600a1220 msgr2=0x7f99600a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:12.300 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.274+0000 7f99805e4700 1 --2- 172.21.15.39:0/1917891167 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f99600a1220 0x7f99600a36d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9968007ab0 tx=0x7f996800b7b0 comp rx=0 tx=0).stop 2023-07-21T18:40:12.300 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.274+0000 7f99805e4700 1 -- 172.21.15.39:0/1917891167 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9978102ed0 msgr2=0x7f9978195e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:12.300 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.274+0000 7f99805e4700 1 --2- 172.21.15.39:0/1917891167 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9978102ed0 0x7f9978195e60 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f996c00d950 tx=0x7f996c00dc60 comp rx=0 tx=0).stop 2023-07-21T18:40:12.300 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.274+0000 7f99805e4700 1 -- 172.21.15.39:0/1917891167 shutdown_connections 2023-07-21T18:40:12.300 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.274+0000 7f99805e4700 1 --2- 172.21.15.39:0/1917891167 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f99600a1220 0x7f99600a36d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:12.300 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.274+0000 7f99805e4700 1 --2- 172.21.15.39:0/1917891167 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f99781005b0 0x7f9978195920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:12.301 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.274+0000 7f99805e4700 1 --2- 172.21.15.39:0/1917891167 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f99781057f0 0x7f997819ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:12.301 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.274+0000 7f99805e4700 1 --2- 172.21.15.39:0/1917891167 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9978102ed0 0x7f9978195e60 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:12.301 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.274+0000 7f99805e4700 1 -- 172.21.15.39:0/1917891167 >> 172.21.15.39:0/1917891167 conn(0x7f99780f9fd0 msgr2=0x7f9978101210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:12.301 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.274+0000 7f99805e4700 1 -- 172.21.15.39:0/1917891167 shutdown_connections 2023-07-21T18:40:12.301 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.274+0000 7f99805e4700 1 -- 172.21.15.39:0/1917891167 wait complete. 2023-07-21T18:40:12.301 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 26 2023-07-21T18:40:12.317 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T18:40:12.442 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.416+0000 7fc9e9df5700 1 -- 172.21.15.39:0/1526121305 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9e4100590 msgr2=0x7fc9e4106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:12.442 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.416+0000 7fc9e9df5700 1 --2- 172.21.15.39:0/1526121305 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9e4100590 0x7fc9e4106e50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc00b0a0 tx=0x7fc9dc00b3b0 comp rx=0 tx=0).stop 2023-07-21T18:40:12.442 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.416+0000 7fc9e9df5700 1 -- 172.21.15.39:0/1526121305 shutdown_connections 2023-07-21T18:40:12.442 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.416+0000 7fc9e9df5700 1 --2- 172.21.15.39:0/1526121305 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9e40fea10 0x7fc9e40fee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:12.442 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.416+0000 7fc9e9df5700 1 --2- 172.21.15.39:0/1526121305 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9e4100590 0x7fc9e4106e50 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:12.442 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.416+0000 7fc9e9df5700 1 --2- 172.21.15.39:0/1526121305 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc9e40ffc00 0x7fc9e4100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:12.443 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.416+0000 7fc9e9df5700 1 -- 172.21.15.39:0/1526121305 >> 172.21.15.39:0/1526121305 conn(0x7fc9e40fa030 msgr2=0x7fc9e40fc460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:12.443 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.416+0000 7fc9e9df5700 1 -- 172.21.15.39:0/1526121305 shutdown_connections 2023-07-21T18:40:12.443 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.416+0000 7fc9e9df5700 1 -- 172.21.15.39:0/1526121305 wait complete. 2023-07-21T18:40:12.443 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.416+0000 7fc9e9df5700 1 Processor -- start 2023-07-21T18:40:12.446 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.417+0000 7fc9e9df5700 1 -- start start 2023-07-21T18:40:12.447 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.417+0000 7fc9e9df5700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc9e40fea10 0x7fc9e4195860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:12.447 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.417+0000 7fc9e9df5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9e40ffc00 0x7fc9e4195da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:12.448 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.417+0000 7fc9e9df5700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9e4100590 0x7fc9e419ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:12.449 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.417+0000 7fc9e9df5700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e419d3e0 con 0x7fc9e40ffc00 2023-07-21T18:40:12.449 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.417+0000 7fc9e9df5700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e419d520 con 0x7fc9e4100590 2023-07-21T18:40:12.450 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.417+0000 7fc9e9df5700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e419d6b0 con 0x7fc9e40fea10 2023-07-21T18:40:12.450 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.417+0000 7fc9e2ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9e40ffc00 0x7fc9e4195da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:12.450 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.417+0000 7fc9e2ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9e40ffc00 0x7fc9e4195da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55224/0 (socket says 172.21.15.39:55224) 2023-07-21T18:40:12.451 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.417+0000 7fc9e2ffd700 1 -- 172.21.15.39:0/4202342743 learned_addr learned my addr 172.21.15.39:0/4202342743 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:40:12.451 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.417+0000 7fc9e37fe700 1 --2- 172.21.15.39:0/4202342743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc9e40fea10 0x7fc9e4195860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:12.451 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.417+0000 7fc9e3fff700 1 --2- 172.21.15.39:0/4202342743 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9e4100590 0x7fc9e419ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:12.451 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.417+0000 7fc9e2ffd700 1 -- 172.21.15.39:0/4202342743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc9e40fea10 msgr2=0x7fc9e4195860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:12.452 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.417+0000 7fc9e2ffd700 1 --2- 172.21.15.39:0/4202342743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc9e40fea10 0x7fc9e4195860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:12.452 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.417+0000 7fc9e2ffd700 1 -- 172.21.15.39:0/4202342743 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9e4100590 msgr2=0x7fc9e419ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:12.452 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.417+0000 7fc9e2ffd700 1 --2- 172.21.15.39:0/4202342743 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9e4100590 0x7fc9e419ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:12.453 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.417+0000 7fc9e2ffd700 1 -- 172.21.15.39:0/4202342743 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9dc009d00 con 0x7fc9e40ffc00 2023-07-21T18:40:12.453 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.417+0000 7fc9e3fff700 1 --2- 172.21.15.39:0/4202342743 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9e4100590 0x7fc9e419ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:40:12.453 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.417+0000 7fc9e2ffd700 1 --2- 172.21.15.39:0/4202342743 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9e40ffc00 0x7fc9e4195da0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc9d400c920 tx=0x7fc9d400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:12.453 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.418+0000 7fc9e0ff9700 1 -- 172.21.15.39:0/4202342743 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d4007b00 con 0x7fc9e40ffc00 2023-07-21T18:40:12.454 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.418+0000 7fc9e0ff9700 1 -- 172.21.15.39:0/4202342743 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc9d400ce70 con 0x7fc9e40ffc00 2023-07-21T18:40:12.454 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.418+0000 7fc9e9df5700 1 -- 172.21.15.39:0/4202342743 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9e419d990 con 0x7fc9e40ffc00 2023-07-21T18:40:12.454 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.418+0000 7fc9e0ff9700 1 -- 172.21.15.39:0/4202342743 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d40054e0 con 0x7fc9e40ffc00 2023-07-21T18:40:12.454 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.418+0000 7fc9e9df5700 1 -- 172.21.15.39:0/4202342743 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9e419dea0 con 0x7fc9e40ffc00 2023-07-21T18:40:12.455 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.419+0000 7fc9e9df5700 1 -- 172.21.15.39:0/4202342743 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9e410a1f0 con 0x7fc9e40ffc00 2023-07-21T18:40:12.455 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.420+0000 7fc9e0ff9700 1 -- 172.21.15.39:0/4202342743 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fc9d4004750 con 0x7fc9e40ffc00 2023-07-21T18:40:12.455 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.420+0000 7fc9e0ff9700 1 --2- 172.21.15.39:0/4202342743 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fc9d00a1220 0x7fc9d00a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:12.455 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.420+0000 7fc9e37fe700 1 --2- 172.21.15.39:0/4202342743 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fc9d00a1220 0x7fc9d00a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:12.456 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.420+0000 7fc9e0ff9700 1 -- 172.21.15.39:0/4202342743 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc9d40cc840 con 0x7fc9e40ffc00 2023-07-21T18:40:12.456 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.421+0000 7fc9e37fe700 1 --2- 172.21.15.39:0/4202342743 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fc9d00a1220 0x7fc9d00a36d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc9e40ffa60 tx=0x7fc9cc00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:12.457 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.424+0000 7fc9e0ff9700 1 -- 172.21.15.39:0/4202342743 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fc9d4095fe0 con 0x7fc9e40ffc00 2023-07-21T18:40:12.649 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.624+0000 7fc9e9df5700 1 -- 172.21.15.39:0/4202342743 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc9e410a410 con 0x7fc9e40ffc00 2023-07-21T18:40:12.651 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.626+0000 7fc9e0ff9700 1 -- 172.21.15.39:0/4202342743 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+22117 (secure 0 0 0) 0x7fc9d401d020 con 0x7fc9e40ffc00 2023-07-21T18:40:12.651 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:40:12.652 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":79,"fsid":"eaf8e25a-27f4-11ee-9b34-001a4aab830c","created":"2023-07-21T18:33:06.504979+0000","modified":"2023-07-21T18:40:05.769300+0000","last_up_change":"2023-07-21T18:39:23.531009+0000","last_in_change":"2023-07-21T18:39:05.993024+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T18:36:13.572583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T18:40:02.726735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T18:40:03.990112+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03502c07-2d0f-4723-8d0b-d34c9f5b91bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6803","nonce":860193857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6805","nonce":860193857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6809","nonce":860193857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6807","nonce":860193857}]},"public_addr":"172.21.15.39:6803/860193857","cluster_addr":"172.21.15.39:6805/860193857","heartbeat_back_addr":"172.21.15.39:6809/860193857","heartbeat_front_addr":"172.21.15.39:6807/860193857","state":["exists","up"]},{"osd":1,"uuid":"0831a0c0-a1fa-4d30-9a28-513223aa69eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6801","nonce":3246721764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6803","nonce":3246721764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6807","nonce":3246721764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6805","nonce":3246721764}]},"public_addr":"172.21.15.155:6801/3246721764","cluster_addr":"172.21.15.155:6803/3246721764","heartbeat_back_addr":"172.21.15.155:6807/3246721764","heartbeat_front_addr":"172.21.15.155:6805/3246721764","state":["exists","up"]},{"osd":2,"uuid":"22df0003-bde9-48e4-b47a-cb8eafbab0db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6801","nonce":2563594449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6803","nonce":2563594449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6807","nonce":2563594449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6805","nonce":2563594449}]},"public_addr":"172.21.15.193:6801/2563594449","cluster_addr":"172.21.15.193:6803/2563594449","heartbeat_back_addr":"172.21.15.193:6807/2563594449","heartbeat_front_addr":"172.21.15.193:6805/2563594449","state":["exists","up"]},{"osd":3,"uuid":"54562558-a2e8-45dd-bc95-561569b9ae6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6811","nonce":863708386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6813","nonce":863708386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6817","nonce":863708386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6815","nonce":863708386}]},"public_addr":"172.21.15.39:6811/863708386","cluster_addr":"172.21.15.39:6813/863708386","heartbeat_back_addr":"172.21.15.39:6817/863708386","heartbeat_front_addr":"172.21.15.39:6815/863708386","state":["exists","up"]},{"osd":4,"uuid":"491118e7-ae1e-495c-9eac-ef1cdcae9600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6809","nonce":3361780673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6811","nonce":3361780673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6815","nonce":3361780673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6813","nonce":3361780673}]},"public_addr":"172.21.15.155:6809/3361780673","cluster_addr":"172.21.15.155:6811/3361780673","heartbeat_back_addr":"172.21.15.155:6815/3361780673","heartbeat_front_addr":"172.21.15.155:6813/3361780673","state":["exists","up"]},{"osd":5,"uuid":"d267bc2c-42b3-4d96-b2d1-b186dc7e6b18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6809","nonce":2376867721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6811","nonce":2376867721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6815","nonce":2376867721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6813","nonce":2376867721}]},"public_addr":"172.21.15.193:6809/2376867721","cluster_addr":"172.21.15.193:6811/2376867721","heartbeat_back_addr":"172.21.15.193:6815/2376867721","heartbeat_front_addr":"172.21.15.193:6813/2376867721","state":["exists","up"]},{"osd":6,"uuid":"beec8088-970a-473e-bd67-6eee7cbdef24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6819","nonce":2417770529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6821","nonce":2417770529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6825","nonce":2417770529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6823","nonce":2417770529}]},"public_addr":"172.21.15.39:6819/2417770529","cluster_addr":"172.21.15.39:6821/2417770529","heartbeat_back_addr":"172.21.15.39:6825/2417770529","heartbeat_front_addr":"172.21.15.39:6823/2417770529","state":["exists","up"]},{"osd":7,"uuid":"5a88ce30-1835-4254-90cc-804ed5cf1066","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6817","nonce":2542002111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6819","nonce":2542002111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6823","nonce":2542002111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6821","nonce":2542002111}]},"public_addr":"172.21.15.155:6817/2542002111","cluster_addr":"172.21.15.155:6819/2542002111","heartbeat_back_addr":"172.21.15.155:6823/2542002111","heartbeat_front_addr":"172.21.15.155:6821/2542002111","state":["exists","up"]},{"osd":8,"uuid":"d964a183-3021-4ace-9a0c-aad00f2570e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6817","nonce":2985835085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6819","nonce":2985835085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6823","nonce":2985835085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6821","nonce":2985835085}]},"public_addr":"172.21.15.193:6817/2985835085","cluster_addr":"172.21.15.193:6819/2985835085","heartbeat_back_addr":"172.21.15.193:6823/2985835085","heartbeat_front_addr":"172.21.15.193:6821/2985835085","state":["exists","up"]},{"osd":9,"uuid":"9725ea5c-49a3-4c99-bac4-e95069100b1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6827","nonce":1946493651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6829","nonce":1946493651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6833","nonce":1946493651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6831","nonce":1946493651}]},"public_addr":"172.21.15.39:6827/1946493651","cluster_addr":"172.21.15.39:6829/1946493651","heartbeat_back_addr":"172.21.15.39:6833/1946493651","heartbeat_front_addr":"172.21.15.39:6831/1946493651","state":["exists","up"]},{"osd":10,"uuid":"1fd60ccb-92db-45f0-a580-2003db0976e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6825","nonce":1234615022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6827","nonce":1234615022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6831","nonce":1234615022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6829","nonce":1234615022}]},"public_addr":"172.21.15.155:6825/1234615022","cluster_addr":"172.21.15.155:6827/1234615022","heartbeat_back_addr":"172.21.15.155:6831/1234615022","heartbeat_front_addr":"172.21.15.155:6829/1234615022","state":["exists","up"]},{"osd":11,"uuid":"efb31ea1-4775-4e5f-8740-728d6f4aeea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6825","nonce":215104663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6827","nonce":215104663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6831","nonce":215104663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6829","nonce":215104663}]},"public_addr":"172.21.15.193:6825/215104663","cluster_addr":"172.21.15.193:6827/215104663","heartbeat_back_addr":"172.21.15.193:6831/215104663","heartbeat_front_addr":"172.21.15.193:6829/215104663","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:35:30.211698+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:35:50.883846+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:36:11.124168+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:36:32.711905+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:36:54.442007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:37:14.843526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:37:36.587354+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:37:57.882374+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:38:18.729407+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:38:40.139231+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:39:01.290862+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:39:21.767537+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:0/2902600414":"2023-07-22T18:34:07.039823+0000","172.21.15.39:6801/2893866425":"2023-07-22T18:34:07.039823+0000","172.21.15.39:0/19717002":"2023-07-22T18:34:07.039823+0000","172.21.15.39:6800/2893866425":"2023-07-22T18:34:07.039823+0000","172.21.15.39:6800/889516631":"2023-07-22T18:33:40.852706+0000","172.21.15.39:6801/889516631":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/3636510379":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/1275748831":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/3819152602":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/1127767082":"2023-07-22T18:34:07.039823+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T18:40:12.653 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.628+0000 7fc9e9df5700 1 -- 172.21.15.39:0/4202342743 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fc9d00a1220 msgr2=0x7fc9d00a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:12.654 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.628+0000 7fc9e9df5700 1 --2- 172.21.15.39:0/4202342743 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fc9d00a1220 0x7fc9d00a36d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc9e40ffa60 tx=0x7fc9cc00a380 comp rx=0 tx=0).stop 2023-07-21T18:40:12.654 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.628+0000 7fc9e9df5700 1 -- 172.21.15.39:0/4202342743 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9e40ffc00 msgr2=0x7fc9e4195da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:12.654 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.628+0000 7fc9e9df5700 1 --2- 172.21.15.39:0/4202342743 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9e40ffc00 0x7fc9e4195da0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc9d400c920 tx=0x7fc9d400cc30 comp rx=0 tx=0).stop 2023-07-21T18:40:12.654 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.628+0000 7fc9e9df5700 1 -- 172.21.15.39:0/4202342743 shutdown_connections 2023-07-21T18:40:12.654 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.628+0000 7fc9e9df5700 1 --2- 172.21.15.39:0/4202342743 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fc9d00a1220 0x7fc9d00a36d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:12.654 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.628+0000 7fc9e9df5700 1 --2- 172.21.15.39:0/4202342743 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9e4100590 0x7fc9e419ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:12.655 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.628+0000 7fc9e9df5700 1 --2- 172.21.15.39:0/4202342743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc9e40fea10 0x7fc9e4195860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:12.655 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.628+0000 7fc9e9df5700 1 --2- 172.21.15.39:0/4202342743 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9e40ffc00 0x7fc9e4195da0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:12.655 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.628+0000 7fc9e9df5700 1 -- 172.21.15.39:0/4202342743 >> 172.21.15.39:0/4202342743 conn(0x7fc9e40fa030 msgr2=0x7fc9e4068700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:12.655 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.628+0000 7fc9e9df5700 1 -- 172.21.15.39:0/4202342743 shutdown_connections 2023-07-21T18:40:12.655 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.628+0000 7fc9e9df5700 1 -- 172.21.15.39:0/4202342743 wait complete. 2023-07-21T18:40:12.666 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-07-21T18:40:12.667 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-21T18:40:12.794 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.767+0000 7fc3caf69700 1 -- 172.21.15.39:0/848274440 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3c4100550 msgr2=0x7fc3c4106e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:12.794 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.767+0000 7fc3caf69700 1 --2- 172.21.15.39:0/848274440 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3c4100550 0x7fc3c4106e10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc00b0a0 tx=0x7fc3bc00b3b0 comp rx=0 tx=0).stop 2023-07-21T18:40:12.794 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.767+0000 7fc3caf69700 1 -- 172.21.15.39:0/848274440 shutdown_connections 2023-07-21T18:40:12.795 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.767+0000 7fc3caf69700 1 --2- 172.21.15.39:0/848274440 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3c4100550 0x7fc3c4106e10 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:12.795 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.767+0000 7fc3caf69700 1 --2- 172.21.15.39:0/848274440 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3c40fe9d0 0x7fc3c40fede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:12.795 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.767+0000 7fc3caf69700 1 --2- 172.21.15.39:0/848274440 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3c40ffbc0 0x7fc3c4100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:12.795 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.767+0000 7fc3caf69700 1 -- 172.21.15.39:0/848274440 >> 172.21.15.39:0/848274440 conn(0x7fc3c40f9fd0 msgr2=0x7fc3c40fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:12.795 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.767+0000 7fc3caf69700 1 -- 172.21.15.39:0/848274440 shutdown_connections 2023-07-21T18:40:12.795 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.767+0000 7fc3caf69700 1 -- 172.21.15.39:0/848274440 wait complete. 2023-07-21T18:40:12.795 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.767+0000 7fc3caf69700 1 Processor -- start 2023-07-21T18:40:12.796 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.767+0000 7fc3caf69700 1 -- start start 2023-07-21T18:40:12.796 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.768+0000 7fc3caf69700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3c40fe9d0 0x7fc3c41958e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:12.796 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.768+0000 7fc3caf69700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3c40ffbc0 0x7fc3c4195e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:12.796 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.768+0000 7fc3caf69700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3c4100550 0x7fc3c419ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:12.796 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.768+0000 7fc3caf69700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c419d460 con 0x7fc3c4100550 2023-07-21T18:40:12.796 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.768+0000 7fc3caf69700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c419d5a0 con 0x7fc3c40fe9d0 2023-07-21T18:40:12.796 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.768+0000 7fc3caf69700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c419d730 con 0x7fc3c40ffbc0 2023-07-21T18:40:12.796 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.768+0000 7fc3c9506700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3c4100550 0x7fc3c419ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:12.797 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.768+0000 7fc3c9506700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3c4100550 0x7fc3c419ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55240/0 (socket says 172.21.15.39:55240) 2023-07-21T18:40:12.797 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.768+0000 7fc3c9506700 1 -- 172.21.15.39:0/2125918961 learned_addr learned my addr 172.21.15.39:0/2125918961 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:40:12.797 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.768+0000 7fc3c3fff700 1 --2- 172.21.15.39:0/2125918961 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3c40ffbc0 0x7fc3c4195e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:12.797 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.768+0000 7fc3c8d05700 1 --2- 172.21.15.39:0/2125918961 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3c40fe9d0 0x7fc3c41958e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:12.797 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.768+0000 7fc3c9506700 1 -- 172.21.15.39:0/2125918961 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3c40ffbc0 msgr2=0x7fc3c4195e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:12.797 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.768+0000 7fc3c9506700 1 --2- 172.21.15.39:0/2125918961 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3c40ffbc0 0x7fc3c4195e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:12.797 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.768+0000 7fc3c9506700 1 -- 172.21.15.39:0/2125918961 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3c40fe9d0 msgr2=0x7fc3c41958e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:12.798 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.768+0000 7fc3c9506700 1 --2- 172.21.15.39:0/2125918961 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3c40fe9d0 0x7fc3c41958e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:12.798 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.768+0000 7fc3c9506700 1 -- 172.21.15.39:0/2125918961 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3bc009d00 con 0x7fc3c4100550 2023-07-21T18:40:12.798 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.768+0000 7fc3c9506700 1 --2- 172.21.15.39:0/2125918961 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3c4100550 0x7fc3c419ce90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc00b830 tx=0x7fc3bc004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:12.798 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.768+0000 7fc3c1ffb700 1 -- 172.21.15.39:0/2125918961 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3bc00e040 con 0x7fc3c4100550 2023-07-21T18:40:12.798 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.768+0000 7fc3c1ffb700 1 -- 172.21.15.39:0/2125918961 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc3bc01db50 con 0x7fc3c4100550 2023-07-21T18:40:12.798 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.768+0000 7fc3caf69700 1 -- 172.21.15.39:0/2125918961 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3c419d9b0 con 0x7fc3c4100550 2023-07-21T18:40:12.798 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.768+0000 7fc3c1ffb700 1 -- 172.21.15.39:0/2125918961 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3bc01bb20 con 0x7fc3c4100550 2023-07-21T18:40:12.798 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.769+0000 7fc3caf69700 1 -- 172.21.15.39:0/2125918961 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3c419de30 con 0x7fc3c4100550 2023-07-21T18:40:12.799 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.769+0000 7fc3caf69700 1 -- 172.21.15.39:0/2125918961 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3c410a1d0 con 0x7fc3c4100550 2023-07-21T18:40:12.799 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.771+0000 7fc3c1ffb700 1 -- 172.21.15.39:0/2125918961 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fc3bc01dcc0 con 0x7fc3c4100550 2023-07-21T18:40:12.800 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.771+0000 7fc3c1ffb700 1 --2- 172.21.15.39:0/2125918961 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fc3b40a1220 0x7fc3b40a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:12.800 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.771+0000 7fc3c1ffb700 1 -- 172.21.15.39:0/2125918961 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc3bc0ce880 con 0x7fc3c4100550 2023-07-21T18:40:12.800 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.771+0000 7fc3c8d05700 1 --2- 172.21.15.39:0/2125918961 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fc3b40a1220 0x7fc3b40a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:12.800 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.772+0000 7fc3c8d05700 1 --2- 172.21.15.39:0/2125918961 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fc3b40a1220 0x7fc3b40a36d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc3c40ffa20 tx=0x7fc3b800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:12.800 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.774+0000 7fc3c1ffb700 1 -- 172.21.15.39:0/2125918961 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fc3bc098130 con 0x7fc3c4100550 2023-07-21T18:40:12.999 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.974+0000 7fc3caf69700 1 -- 172.21.15.39:0/2125918961 --> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc3c40fae60 con 0x7fc3b40a1220 2023-07-21T18:40:13.012 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.986+0000 7fc3c1ffb700 1 -- 172.21.15.39:0/2125918961 <== mgr.14152 v2:172.21.15.39:6800/2116291712 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+335622 (secure 0 0 0) 0x7fc3c40fae60 con 0x7fc3b40a1220 2023-07-21T18:40:13.017 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:40:13.021 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.990+0000 7fc3caf69700 1 -- 172.21.15.39:0/2125918961 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fc3b40a1220 msgr2=0x7fc3b40a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:13.022 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.990+0000 7fc3caf69700 1 --2- 172.21.15.39:0/2125918961 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fc3b40a1220 0x7fc3b40a36d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc3c40ffa20 tx=0x7fc3b800b410 comp rx=0 tx=0).stop 2023-07-21T18:40:13.022 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.990+0000 7fc3caf69700 1 -- 172.21.15.39:0/2125918961 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3c4100550 msgr2=0x7fc3c419ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:13.022 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.990+0000 7fc3caf69700 1 --2- 172.21.15.39:0/2125918961 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3c4100550 0x7fc3c419ce90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc00b830 tx=0x7fc3bc004c80 comp rx=0 tx=0).stop 2023-07-21T18:40:13.023 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.990+0000 7fc3caf69700 1 -- 172.21.15.39:0/2125918961 shutdown_connections 2023-07-21T18:40:13.023 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.990+0000 7fc3caf69700 1 --2- 172.21.15.39:0/2125918961 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fc3b40a1220 0x7fc3b40a36d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:13.023 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.990+0000 7fc3caf69700 1 --2- 172.21.15.39:0/2125918961 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3c40fe9d0 0x7fc3c41958e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:13.024 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.990+0000 7fc3caf69700 1 --2- 172.21.15.39:0/2125918961 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3c4100550 0x7fc3c419ce90 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:13.024 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.990+0000 7fc3caf69700 1 --2- 172.21.15.39:0/2125918961 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3c40ffbc0 0x7fc3c4195e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:13.024 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.990+0000 7fc3caf69700 1 -- 172.21.15.39:0/2125918961 >> 172.21.15.39:0/2125918961 conn(0x7fc3c40f9fd0 msgr2=0x7fc3c40fa750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:13.024 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.990+0000 7fc3caf69700 1 -- 172.21.15.39:0/2125918961 shutdown_connections 2023-07-21T18:40:13.025 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:12.990+0000 7fc3caf69700 1 -- 172.21.15.39:0/2125918961 wait complete. 2023-07-21T18:40:13.025 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2023-07-21T18:40:13.028 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":251,"stamp":"2023-07-21T18:40:11.120210+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595544,"num_objects":54,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":149,"num_read_kb":131,"num_write":288,"num_write_kb":2976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":235,"ondisk_log_size":235,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":265,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335216,"kb_used_data":11792,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124606608,"statfs":{"total":1151940427776,"available":1151597166592,"internally_reserved":0,"allocated":12075008,"data_stored":6656683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5176,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":13,"num_read_kb":12,"num_write":107,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.957070"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.773443+0000","last_change":"2023-07-21T18:40:04.792977+0000","last_active":"2023-07-21T18:40:05.773443+0000","last_peered":"2023-07-21T18:40:05.773443+0000","last_clean":"2023-07-21T18:40:05.773443+0000","last_became_active":"2023-07-21T18:40:04.792667+0000","last_became_peered":"2023-07-21T18:40:04.792667+0000","last_unstale":"2023-07-21T18:40:05.773443+0000","last_undegraded":"2023-07-21T18:40:05.773443+0000","last_fullsized":"2023-07-21T18:40:05.773443+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:37:51.249650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.035763+0000","last_change":"2023-07-21T18:40:04.792936+0000","last_active":"2023-07-21T18:40:07.035763+0000","last_peered":"2023-07-21T18:40:07.035763+0000","last_clean":"2023-07-21T18:40:07.035763+0000","last_became_active":"2023-07-21T18:40:04.792511+0000","last_became_peered":"2023-07-21T18:40:04.792511+0000","last_unstale":"2023-07-21T18:40:07.035763+0000","last_undegraded":"2023-07-21T18:40:07.035763+0000","last_fullsized":"2023-07-21T18:40:07.035763+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:05:00.595279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.035567+0000","last_change":"2023-07-21T18:40:04.796401+0000","last_active":"2023-07-21T18:40:07.035567+0000","last_peered":"2023-07-21T18:40:07.035567+0000","last_clean":"2023-07-21T18:40:07.035567+0000","last_became_active":"2023-07-21T18:40:04.796063+0000","last_became_peered":"2023-07-21T18:40:04.796063+0000","last_unstale":"2023-07-21T18:40:07.035567+0000","last_undegraded":"2023-07-21T18:40:07.035567+0000","last_fullsized":"2023-07-21T18:40:07.035567+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:11:42.127453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.035756+0000","last_change":"2023-07-21T18:40:04.792308+0000","last_active":"2023-07-21T18:40:07.035756+0000","last_peered":"2023-07-21T18:40:07.035756+0000","last_clean":"2023-07-21T18:40:07.035756+0000","last_became_active":"2023-07-21T18:40:04.791831+0000","last_became_peered":"2023-07-21T18:40:04.791831+0000","last_unstale":"2023-07-21T18:40:07.035756+0000","last_undegraded":"2023-07-21T18:40:07.035756+0000","last_fullsized":"2023-07-21T18:40:07.035756+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:31:30.077525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.006829+0000","last_change":"2023-07-21T18:40:04.794035+0000","last_active":"2023-07-21T18:40:06.006829+0000","last_peered":"2023-07-21T18:40:06.006829+0000","last_clean":"2023-07-21T18:40:06.006829+0000","last_became_active":"2023-07-21T18:40:04.793739+0000","last_became_peered":"2023-07-21T18:40:04.793739+0000","last_unstale":"2023-07-21T18:40:06.006829+0000","last_undegraded":"2023-07-21T18:40:06.006829+0000","last_fullsized":"2023-07-21T18:40:06.006829+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:40:42.480359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.035838+0000","last_change":"2023-07-21T18:40:04.792751+0000","last_active":"2023-07-21T18:40:07.035838+0000","last_peered":"2023-07-21T18:40:07.035838+0000","last_clean":"2023-07-21T18:40:07.035838+0000","last_became_active":"2023-07-21T18:40:04.792413+0000","last_became_peered":"2023-07-21T18:40:04.792413+0000","last_unstale":"2023-07-21T18:40:07.035838+0000","last_undegraded":"2023-07-21T18:40:07.035838+0000","last_fullsized":"2023-07-21T18:40:07.035838+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:14:40.371518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.089901+0000","last_change":"2023-07-21T18:40:04.787544+0000","last_active":"2023-07-21T18:40:06.089901+0000","last_peered":"2023-07-21T18:40:06.089901+0000","last_clean":"2023-07-21T18:40:06.089901+0000","last_became_active":"2023-07-21T18:40:04.787208+0000","last_became_peered":"2023-07-21T18:40:04.787208+0000","last_unstale":"2023-07-21T18:40:06.089901+0000","last_undegraded":"2023-07-21T18:40:06.089901+0000","last_fullsized":"2023-07-21T18:40:06.089901+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:40:52.809501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.035809+0000","last_change":"2023-07-21T18:40:04.787964+0000","last_active":"2023-07-21T18:40:07.035809+0000","last_peered":"2023-07-21T18:40:07.035809+0000","last_clean":"2023-07-21T18:40:07.035809+0000","last_became_active":"2023-07-21T18:40:04.787518+0000","last_became_peered":"2023-07-21T18:40:04.787518+0000","last_unstale":"2023-07-21T18:40:07.035809+0000","last_undegraded":"2023-07-21T18:40:07.035809+0000","last_fullsized":"2023-07-21T18:40:07.035809+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:40:44.640274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.777237+0000","last_change":"2023-07-21T18:40:04.782495+0000","last_active":"2023-07-21T18:40:05.777237+0000","last_peered":"2023-07-21T18:40:05.777237+0000","last_clean":"2023-07-21T18:40:05.777237+0000","last_became_active":"2023-07-21T18:40:04.782208+0000","last_became_peered":"2023-07-21T18:40:04.782208+0000","last_unstale":"2023-07-21T18:40:05.777237+0000","last_undegraded":"2023-07-21T18:40:05.777237+0000","last_fullsized":"2023-07-21T18:40:05.777237+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:42:34.649015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.036247+0000","last_change":"2023-07-21T18:40:04.790957+0000","last_active":"2023-07-21T18:40:07.036247+0000","last_peered":"2023-07-21T18:40:07.036247+0000","last_clean":"2023-07-21T18:40:07.036247+0000","last_became_active":"2023-07-21T18:40:04.790659+0000","last_became_peered":"2023-07-21T18:40:04.790659+0000","last_unstale":"2023-07-21T18:40:07.036247+0000","last_undegraded":"2023-07-21T18:40:07.036247+0000","last_fullsized":"2023-07-21T18:40:07.036247+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:06:45.518722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.090285+0000","last_change":"2023-07-21T18:40:04.789415+0000","last_active":"2023-07-21T18:40:06.090285+0000","last_peered":"2023-07-21T18:40:06.090285+0000","last_clean":"2023-07-21T18:40:06.090285+0000","last_became_active":"2023-07-21T18:40:04.789088+0000","last_became_peered":"2023-07-21T18:40:04.789088+0000","last_unstale":"2023-07-21T18:40:06.090285+0000","last_undegraded":"2023-07-21T18:40:06.090285+0000","last_fullsized":"2023-07-21T18:40:06.090285+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:10:43.584203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.035583+0000","last_change":"2023-07-21T18:40:04.789101+0000","last_active":"2023-07-21T18:40:07.035583+0000","last_peered":"2023-07-21T18:40:07.035583+0000","last_clean":"2023-07-21T18:40:07.035583+0000","last_became_active":"2023-07-21T18:40:04.788704+0000","last_became_peered":"2023-07-21T18:40:04.788704+0000","last_unstale":"2023-07-21T18:40:07.035583+0000","last_undegraded":"2023-07-21T18:40:07.035583+0000","last_fullsized":"2023-07-21T18:40:07.035583+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:10:15.256725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.772732+0000","last_change":"2023-07-21T18:40:04.784863+0000","last_active":"2023-07-21T18:40:05.772732+0000","last_peered":"2023-07-21T18:40:05.772732+0000","last_clean":"2023-07-21T18:40:05.772732+0000","last_became_active":"2023-07-21T18:40:04.784474+0000","last_became_peered":"2023-07-21T18:40:04.784474+0000","last_unstale":"2023-07-21T18:40:05.772732+0000","last_undegraded":"2023-07-21T18:40:05.772732+0000","last_fullsized":"2023-07-21T18:40:05.772732+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:53:41.983453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.036146+0000","last_change":"2023-07-21T18:40:04.797678+0000","last_active":"2023-07-21T18:40:07.036146+0000","last_peered":"2023-07-21T18:40:07.036146+0000","last_clean":"2023-07-21T18:40:07.036146+0000","last_became_active":"2023-07-21T18:40:04.797318+0000","last_became_peered":"2023-07-21T18:40:04.797318+0000","last_unstale":"2023-07-21T18:40:07.036146+0000","last_undegraded":"2023-07-21T18:40:07.036146+0000","last_fullsized":"2023-07-21T18:40:07.036146+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:44:17.374089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.035540+0000","last_change":"2023-07-21T18:40:04.793170+0000","last_active":"2023-07-21T18:40:07.035540+0000","last_peered":"2023-07-21T18:40:07.035540+0000","last_clean":"2023-07-21T18:40:07.035540+0000","last_became_active":"2023-07-21T18:40:04.792563+0000","last_became_peered":"2023-07-21T18:40:04.792563+0000","last_unstale":"2023-07-21T18:40:07.035540+0000","last_undegraded":"2023-07-21T18:40:07.035540+0000","last_fullsized":"2023-07-21T18:40:07.035540+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:36:53.352643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.776683+0000","last_change":"2023-07-21T18:40:04.791972+0000","last_active":"2023-07-21T18:40:05.776683+0000","last_peered":"2023-07-21T18:40:05.776683+0000","last_clean":"2023-07-21T18:40:05.776683+0000","last_became_active":"2023-07-21T18:40:04.791639+0000","last_became_peered":"2023-07-21T18:40:04.791639+0000","last_unstale":"2023-07-21T18:40:05.776683+0000","last_undegraded":"2023-07-21T18:40:05.776683+0000","last_fullsized":"2023-07-21T18:40:05.776683+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:35:06.204290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.773432+0000","last_change":"2023-07-21T18:40:04.789362+0000","last_active":"2023-07-21T18:40:05.773432+0000","last_peered":"2023-07-21T18:40:05.773432+0000","last_clean":"2023-07-21T18:40:05.773432+0000","last_became_active":"2023-07-21T18:40:04.788978+0000","last_became_peered":"2023-07-21T18:40:04.788978+0000","last_unstale":"2023-07-21T18:40:05.773432+0000","last_undegraded":"2023-07-21T18:40:05.773432+0000","last_fullsized":"2023-07-21T18:40:05.773432+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:55:04.759568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.036113+0000","last_change":"2023-07-21T18:40:04.788016+0000","last_active":"2023-07-21T18:40:07.036113+0000","last_peered":"2023-07-21T18:40:07.036113+0000","last_clean":"2023-07-21T18:40:07.036113+0000","last_became_active":"2023-07-21T18:40:04.787690+0000","last_became_peered":"2023-07-21T18:40:04.787690+0000","last_unstale":"2023-07-21T18:40:07.036113+0000","last_undegraded":"2023-07-21T18:40:07.036113+0000","last_fullsized":"2023-07-21T18:40:07.036113+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:55:35.585172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.592595+0000","last_change":"2023-07-21T18:40:04.793075+0000","last_active":"2023-07-21T18:40:06.592595+0000","last_peered":"2023-07-21T18:40:06.592595+0000","last_clean":"2023-07-21T18:40:06.592595+0000","last_became_active":"2023-07-21T18:40:04.792773+0000","last_became_peered":"2023-07-21T18:40:04.792773+0000","last_unstale":"2023-07-21T18:40:06.592595+0000","last_undegraded":"2023-07-21T18:40:06.592595+0000","last_fullsized":"2023-07-21T18:40:06.592595+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:48:23.084376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'4","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.036709+0000","last_change":"2023-07-21T18:40:04.178197+0000","last_active":"2023-07-21T18:40:07.036709+0000","last_peered":"2023-07-21T18:40:07.036709+0000","last_clean":"2023-07-21T18:40:07.036709+0000","last_became_active":"2023-07-21T18:40:04.177914+0000","last_became_peered":"2023-07-21T18:40:04.177914+0000","last_unstale":"2023-07-21T18:40:07.036709+0000","last_undegraded":"2023-07-21T18:40:07.036709+0000","last_fullsized":"2023-07-21T18:40:07.036709+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:33:10.544496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.037188+0000","last_change":"2023-07-21T18:40:04.791610+0000","last_active":"2023-07-21T18:40:07.037188+0000","last_peered":"2023-07-21T18:40:07.037188+0000","last_clean":"2023-07-21T18:40:07.037188+0000","last_became_active":"2023-07-21T18:40:04.791242+0000","last_became_peered":"2023-07-21T18:40:04.791242+0000","last_unstale":"2023-07-21T18:40:07.037188+0000","last_undegraded":"2023-07-21T18:40:07.037188+0000","last_fullsized":"2023-07-21T18:40:07.037188+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:02:43.400899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"78'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.037020+0000","last_change":"2023-07-21T18:40:04.182120+0000","last_active":"2023-07-21T18:40:07.037020+0000","last_peered":"2023-07-21T18:40:07.037020+0000","last_clean":"2023-07-21T18:40:07.037020+0000","last_became_active":"2023-07-21T18:40:04.181797+0000","last_became_peered":"2023-07-21T18:40:04.181797+0000","last_unstale":"2023-07-21T18:40:07.037020+0000","last_undegraded":"2023-07-21T18:40:07.037020+0000","last_fullsized":"2023-07-21T18:40:07.037020+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:02:06.127933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.036297+0000","last_change":"2023-07-21T18:40:04.796649+0000","last_active":"2023-07-21T18:40:07.036297+0000","last_peered":"2023-07-21T18:40:07.036297+0000","last_clean":"2023-07-21T18:40:07.036297+0000","last_became_active":"2023-07-21T18:40:04.795610+0000","last_became_peered":"2023-07-21T18:40:04.795610+0000","last_unstale":"2023-07-21T18:40:07.036297+0000","last_undegraded":"2023-07-21T18:40:07.036297+0000","last_fullsized":"2023-07-21T18:40:07.036297+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:11:11.969964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"79'11","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:10.975382+0000","last_change":"2023-07-21T18:40:04.178384+0000","last_active":"2023-07-21T18:40:10.975382+0000","last_peered":"2023-07-21T18:40:10.975382+0000","last_clean":"2023-07-21T18:40:10.975382+0000","last_became_active":"2023-07-21T18:40:04.178074+0000","last_became_peered":"2023-07-21T18:40:04.178074+0000","last_unstale":"2023-07-21T18:40:10.975382+0000","last_undegraded":"2023-07-21T18:40:10.975382+0000","last_fullsized":"2023-07-21T18:40:10.975382+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:21:02.336313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.776990+0000","last_change":"2023-07-21T18:40:04.785431+0000","last_active":"2023-07-21T18:40:05.776990+0000","last_peered":"2023-07-21T18:40:05.776990+0000","last_clean":"2023-07-21T18:40:05.776990+0000","last_became_active":"2023-07-21T18:40:04.785124+0000","last_became_peered":"2023-07-21T18:40:04.785124+0000","last_unstale":"2023-07-21T18:40:05.776990+0000","last_undegraded":"2023-07-21T18:40:05.776990+0000","last_fullsized":"2023-07-21T18:40:05.776990+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:19:44.604006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"79'5","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:10.975728+0000","last_change":"2023-07-21T18:40:04.176349+0000","last_active":"2023-07-21T18:40:10.975728+0000","last_peered":"2023-07-21T18:40:10.975728+0000","last_clean":"2023-07-21T18:40:10.975728+0000","last_became_active":"2023-07-21T18:40:04.175997+0000","last_became_peered":"2023-07-21T18:40:04.175997+0000","last_unstale":"2023-07-21T18:40:10.975728+0000","last_undegraded":"2023-07-21T18:40:10.975728+0000","last_fullsized":"2023-07-21T18:40:10.975728+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:13:57.068743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.995402+0000","last_change":"2023-07-21T18:40:04.784867+0000","last_active":"2023-07-21T18:40:06.995402+0000","last_peered":"2023-07-21T18:40:06.995402+0000","last_clean":"2023-07-21T18:40:06.995402+0000","last_became_active":"2023-07-21T18:40:04.784555+0000","last_became_peered":"2023-07-21T18:40:04.784555+0000","last_unstale":"2023-07-21T18:40:06.995402+0000","last_undegraded":"2023-07-21T18:40:06.995402+0000","last_fullsized":"2023-07-21T18:40:06.995402+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:57:13.354337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"79'5","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:10.964523+0000","last_change":"2023-07-21T18:40:04.177638+0000","last_active":"2023-07-21T18:40:10.964523+0000","last_peered":"2023-07-21T18:40:10.964523+0000","last_clean":"2023-07-21T18:40:10.964523+0000","last_became_active":"2023-07-21T18:40:04.177284+0000","last_became_peered":"2023-07-21T18:40:04.177284+0000","last_unstale":"2023-07-21T18:40:10.964523+0000","last_undegraded":"2023-07-21T18:40:10.964523+0000","last_fullsized":"2023-07-21T18:40:10.964523+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:21:19.903720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.006906+0000","last_change":"2023-07-21T18:40:04.791536+0000","last_active":"2023-07-21T18:40:06.006906+0000","last_peered":"2023-07-21T18:40:06.006906+0000","last_clean":"2023-07-21T18:40:06.006906+0000","last_became_active":"2023-07-21T18:40:04.791246+0000","last_became_peered":"2023-07-21T18:40:04.791246+0000","last_unstale":"2023-07-21T18:40:06.006906+0000","last_undegraded":"2023-07-21T18:40:06.006906+0000","last_fullsized":"2023-07-21T18:40:06.006906+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:44:08.906635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"78'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.035747+0000","last_change":"2023-07-21T18:40:04.179054+0000","last_active":"2023-07-21T18:40:07.035747+0000","last_peered":"2023-07-21T18:40:07.035747+0000","last_clean":"2023-07-21T18:40:07.035747+0000","last_became_active":"2023-07-21T18:40:04.178713+0000","last_became_peered":"2023-07-21T18:40:04.178713+0000","last_unstale":"2023-07-21T18:40:07.035747+0000","last_undegraded":"2023-07-21T18:40:07.035747+0000","last_fullsized":"2023-07-21T18:40:07.035747+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T18:45:12.898734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.995525+0000","last_change":"2023-07-21T18:40:04.793947+0000","last_active":"2023-07-21T18:40:06.995525+0000","last_peered":"2023-07-21T18:40:06.995525+0000","last_clean":"2023-07-21T18:40:06.995525+0000","last_became_active":"2023-07-21T18:40:04.793631+0000","last_became_peered":"2023-07-21T18:40:04.793631+0000","last_unstale":"2023-07-21T18:40:06.995525+0000","last_undegraded":"2023-07-21T18:40:06.995525+0000","last_fullsized":"2023-07-21T18:40:06.995525+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:38:22.934266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.772418+0000","last_change":"2023-07-21T18:40:04.182549+0000","last_active":"2023-07-21T18:40:05.772418+0000","last_peered":"2023-07-21T18:40:05.772418+0000","last_clean":"2023-07-21T18:40:05.772418+0000","last_became_active":"2023-07-21T18:40:04.182233+0000","last_became_peered":"2023-07-21T18:40:04.182233+0000","last_unstale":"2023-07-21T18:40:05.772418+0000","last_undegraded":"2023-07-21T18:40:05.772418+0000","last_fullsized":"2023-07-21T18:40:05.772418+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:47:27.131280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.036098+0000","last_change":"2023-07-21T18:40:04.789987+0000","last_active":"2023-07-21T18:40:07.036098+0000","last_peered":"2023-07-21T18:40:07.036098+0000","last_clean":"2023-07-21T18:40:07.036098+0000","last_became_active":"2023-07-21T18:40:04.789647+0000","last_became_peered":"2023-07-21T18:40:04.789647+0000","last_unstale":"2023-07-21T18:40:07.036098+0000","last_undegraded":"2023-07-21T18:40:07.036098+0000","last_fullsized":"2023-07-21T18:40:07.036098+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:38:39.403755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'3","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:10.965460+0000","last_change":"2023-07-21T18:40:04.177342+0000","last_active":"2023-07-21T18:40:10.965460+0000","last_peered":"2023-07-21T18:40:10.965460+0000","last_clean":"2023-07-21T18:40:10.965460+0000","last_became_active":"2023-07-21T18:40:04.176983+0000","last_became_peered":"2023-07-21T18:40:04.176983+0000","last_unstale":"2023-07-21T18:40:10.965460+0000","last_undegraded":"2023-07-21T18:40:10.965460+0000","last_fullsized":"2023-07-21T18:40:10.965460+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:51:36.023901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.035932+0000","last_change":"2023-07-21T18:40:04.794907+0000","last_active":"2023-07-21T18:40:07.035932+0000","last_peered":"2023-07-21T18:40:07.035932+0000","last_clean":"2023-07-21T18:40:07.035932+0000","last_became_active":"2023-07-21T18:40:04.794545+0000","last_became_peered":"2023-07-21T18:40:04.794545+0000","last_unstale":"2023-07-21T18:40:07.035932+0000","last_undegraded":"2023-07-21T18:40:07.035932+0000","last_fullsized":"2023-07-21T18:40:07.035932+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:47:34.101102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"79'3","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:08.216736+0000","last_change":"2023-07-21T18:40:04.179198+0000","last_active":"2023-07-21T18:40:08.216736+0000","last_peered":"2023-07-21T18:40:08.216736+0000","last_clean":"2023-07-21T18:40:08.216736+0000","last_became_active":"2023-07-21T18:40:04.178841+0000","last_became_peered":"2023-07-21T18:40:04.178841+0000","last_unstale":"2023-07-21T18:40:08.216736+0000","last_undegraded":"2023-07-21T18:40:08.216736+0000","last_fullsized":"2023-07-21T18:40:08.216736+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:12:24.650360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.777142+0000","last_change":"2023-07-21T18:40:04.786229+0000","last_active":"2023-07-21T18:40:05.777142+0000","last_peered":"2023-07-21T18:40:05.777142+0000","last_clean":"2023-07-21T18:40:05.777142+0000","last_became_active":"2023-07-21T18:40:04.785919+0000","last_became_peered":"2023-07-21T18:40:04.785919+0000","last_unstale":"2023-07-21T18:40:05.777142+0000","last_undegraded":"2023-07-21T18:40:05.777142+0000","last_fullsized":"2023-07-21T18:40:05.777142+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:19:04.949730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:08.211879+0000","last_change":"2023-07-21T18:40:04.177164+0000","last_active":"2023-07-21T18:40:08.211879+0000","last_peered":"2023-07-21T18:40:08.211879+0000","last_clean":"2023-07-21T18:40:08.211879+0000","last_became_active":"2023-07-21T18:40:04.176617+0000","last_became_peered":"2023-07-21T18:40:04.176617+0000","last_unstale":"2023-07-21T18:40:08.211879+0000","last_undegraded":"2023-07-21T18:40:08.211879+0000","last_fullsized":"2023-07-21T18:40:08.211879+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:38:06.655399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.036335+0000","last_change":"2023-07-21T18:40:04.787863+0000","last_active":"2023-07-21T18:40:07.036335+0000","last_peered":"2023-07-21T18:40:07.036335+0000","last_clean":"2023-07-21T18:40:07.036335+0000","last_became_active":"2023-07-21T18:40:04.787562+0000","last_became_peered":"2023-07-21T18:40:04.787562+0000","last_unstale":"2023-07-21T18:40:07.036335+0000","last_undegraded":"2023-07-21T18:40:07.036335+0000","last_fullsized":"2023-07-21T18:40:07.036335+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:30:30.392804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'5","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:10.244099+0000","last_change":"2023-07-21T18:40:04.179921+0000","last_active":"2023-07-21T18:40:10.244099+0000","last_peered":"2023-07-21T18:40:10.244099+0000","last_clean":"2023-07-21T18:40:10.244099+0000","last_became_active":"2023-07-21T18:40:04.179670+0000","last_became_peered":"2023-07-21T18:40:04.179670+0000","last_unstale":"2023-07-21T18:40:10.244099+0000","last_undegraded":"2023-07-21T18:40:10.244099+0000","last_fullsized":"2023-07-21T18:40:10.244099+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:11:15.619102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.777008+0000","last_change":"2023-07-21T18:40:04.792930+0000","last_active":"2023-07-21T18:40:05.777008+0000","last_peered":"2023-07-21T18:40:05.777008+0000","last_clean":"2023-07-21T18:40:05.777008+0000","last_became_active":"2023-07-21T18:40:04.792654+0000","last_became_peered":"2023-07-21T18:40:04.792654+0000","last_unstale":"2023-07-21T18:40:05.777008+0000","last_undegraded":"2023-07-21T18:40:05.777008+0000","last_fullsized":"2023-07-21T18:40:05.777008+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:50:21.387154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"79'8","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:08.211768+0000","last_change":"2023-07-21T18:40:04.178820+0000","last_active":"2023-07-21T18:40:08.211768+0000","last_peered":"2023-07-21T18:40:08.211768+0000","last_clean":"2023-07-21T18:40:08.211768+0000","last_became_active":"2023-07-21T18:40:04.178502+0000","last_became_peered":"2023-07-21T18:40:04.178502+0000","last_unstale":"2023-07-21T18:40:08.211768+0000","last_undegraded":"2023-07-21T18:40:08.211768+0000","last_fullsized":"2023-07-21T18:40:08.211768+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:48:50.193456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.006957+0000","last_change":"2023-07-21T18:40:04.792921+0000","last_active":"2023-07-21T18:40:06.006957+0000","last_peered":"2023-07-21T18:40:06.006957+0000","last_clean":"2023-07-21T18:40:06.006957+0000","last_became_active":"2023-07-21T18:40:04.792608+0000","last_became_peered":"2023-07-21T18:40:04.792608+0000","last_unstale":"2023-07-21T18:40:06.006957+0000","last_undegraded":"2023-07-21T18:40:06.006957+0000","last_fullsized":"2023-07-21T18:40:06.006957+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:04:23.180977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'4","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:08.211852+0000","last_change":"2023-07-21T18:40:04.184917+0000","last_active":"2023-07-21T18:40:08.211852+0000","last_peered":"2023-07-21T18:40:08.211852+0000","last_clean":"2023-07-21T18:40:08.211852+0000","last_became_active":"2023-07-21T18:40:04.184612+0000","last_became_peered":"2023-07-21T18:40:04.184612+0000","last_unstale":"2023-07-21T18:40:08.211852+0000","last_undegraded":"2023-07-21T18:40:08.211852+0000","last_fullsized":"2023-07-21T18:40:08.211852+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:54:45.871657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.772576+0000","last_change":"2023-07-21T18:40:04.794134+0000","last_active":"2023-07-21T18:40:05.772576+0000","last_peered":"2023-07-21T18:40:05.772576+0000","last_clean":"2023-07-21T18:40:05.772576+0000","last_became_active":"2023-07-21T18:40:04.793784+0000","last_became_peered":"2023-07-21T18:40:04.793784+0000","last_unstale":"2023-07-21T18:40:05.772576+0000","last_undegraded":"2023-07-21T18:40:05.772576+0000","last_fullsized":"2023-07-21T18:40:05.772576+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:24:12.380794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'5","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:10.241421+0000","last_change":"2023-07-21T18:40:04.184889+0000","last_active":"2023-07-21T18:40:10.241421+0000","last_peered":"2023-07-21T18:40:10.241421+0000","last_clean":"2023-07-21T18:40:10.241421+0000","last_became_active":"2023-07-21T18:40:04.184516+0000","last_became_peered":"2023-07-21T18:40:04.184516+0000","last_unstale":"2023-07-21T18:40:10.241421+0000","last_undegraded":"2023-07-21T18:40:10.241421+0000","last_fullsized":"2023-07-21T18:40:10.241421+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:56:48.211364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.090064+0000","last_change":"2023-07-21T18:40:04.790627+0000","last_active":"2023-07-21T18:40:06.090064+0000","last_peered":"2023-07-21T18:40:06.090064+0000","last_clean":"2023-07-21T18:40:06.090064+0000","last_became_active":"2023-07-21T18:40:04.790298+0000","last_became_peered":"2023-07-21T18:40:04.790298+0000","last_unstale":"2023-07-21T18:40:06.090064+0000","last_undegraded":"2023-07-21T18:40:06.090064+0000","last_fullsized":"2023-07-21T18:40:06.090064+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:23:15.541330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.035239+0000","last_change":"2023-07-21T18:40:04.176156+0000","last_active":"2023-07-21T18:40:07.035239+0000","last_peered":"2023-07-21T18:40:07.035239+0000","last_clean":"2023-07-21T18:40:07.035239+0000","last_became_active":"2023-07-21T18:40:04.175823+0000","last_became_peered":"2023-07-21T18:40:04.175823+0000","last_unstale":"2023-07-21T18:40:07.035239+0000","last_undegraded":"2023-07-21T18:40:07.035239+0000","last_fullsized":"2023-07-21T18:40:07.035239+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:43:06.713614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.089814+0000","last_change":"2023-07-21T18:40:04.790422+0000","last_active":"2023-07-21T18:40:06.089814+0000","last_peered":"2023-07-21T18:40:06.089814+0000","last_clean":"2023-07-21T18:40:06.089814+0000","last_became_active":"2023-07-21T18:40:04.790124+0000","last_became_peered":"2023-07-21T18:40:04.790124+0000","last_unstale":"2023-07-21T18:40:06.089814+0000","last_undegraded":"2023-07-21T18:40:06.089814+0000","last_fullsized":"2023-07-21T18:40:06.089814+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:50:38.399075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'17","reported_seq":32,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:10.962031+0000","last_change":"2023-07-21T18:40:04.177006+0000","last_active":"2023-07-21T18:40:10.962031+0000","last_peered":"2023-07-21T18:40:10.962031+0000","last_clean":"2023-07-21T18:40:10.962031+0000","last_became_active":"2023-07-21T18:40:04.176711+0000","last_became_peered":"2023-07-21T18:40:04.176711+0000","last_unstale":"2023-07-21T18:40:10.962031+0000","last_undegraded":"2023-07-21T18:40:10.962031+0000","last_fullsized":"2023-07-21T18:40:10.962031+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:51:47.735726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.995106+0000","last_change":"2023-07-21T18:40:04.785601+0000","last_active":"2023-07-21T18:40:06.995106+0000","last_peered":"2023-07-21T18:40:06.995106+0000","last_clean":"2023-07-21T18:40:06.995106+0000","last_became_active":"2023-07-21T18:40:04.785307+0000","last_became_peered":"2023-07-21T18:40:04.785307+0000","last_unstale":"2023-07-21T18:40:06.995106+0000","last_undegraded":"2023-07-21T18:40:06.995106+0000","last_fullsized":"2023-07-21T18:40:06.995106+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:17:07.221060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'1","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.053227+0000","last_change":"2023-07-21T18:40:04.171005+0000","last_active":"2023-07-21T18:40:06.053227+0000","last_peered":"2023-07-21T18:40:06.053227+0000","last_clean":"2023-07-21T18:40:06.053227+0000","last_became_active":"2023-07-21T18:40:04.170719+0000","last_became_peered":"2023-07-21T18:40:04.170719+0000","last_unstale":"2023-07-21T18:40:06.053227+0000","last_undegraded":"2023-07-21T18:40:06.053227+0000","last_fullsized":"2023-07-21T18:40:06.053227+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:06:46.353789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.773304+0000","last_change":"2023-07-21T18:40:04.783596+0000","last_active":"2023-07-21T18:40:05.773304+0000","last_peered":"2023-07-21T18:40:05.773304+0000","last_clean":"2023-07-21T18:40:05.773304+0000","last_became_active":"2023-07-21T18:40:04.783293+0000","last_became_peered":"2023-07-21T18:40:04.783293+0000","last_unstale":"2023-07-21T18:40:05.773304+0000","last_undegraded":"2023-07-21T18:40:05.773304+0000","last_fullsized":"2023-07-21T18:40:05.773304+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:17:45.360634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.036528+0000","last_change":"2023-07-21T18:40:04.181069+0000","last_active":"2023-07-21T18:40:07.036528+0000","last_peered":"2023-07-21T18:40:07.036528+0000","last_clean":"2023-07-21T18:40:07.036528+0000","last_became_active":"2023-07-21T18:40:04.180750+0000","last_became_peered":"2023-07-21T18:40:04.180750+0000","last_unstale":"2023-07-21T18:40:07.036528+0000","last_undegraded":"2023-07-21T18:40:07.036528+0000","last_fullsized":"2023-07-21T18:40:07.036528+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:44:48.158024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.036165+0000","last_change":"2023-07-21T18:40:04.789940+0000","last_active":"2023-07-21T18:40:07.036165+0000","last_peered":"2023-07-21T18:40:07.036165+0000","last_clean":"2023-07-21T18:40:07.036165+0000","last_became_active":"2023-07-21T18:40:04.789465+0000","last_became_peered":"2023-07-21T18:40:04.789465+0000","last_unstale":"2023-07-21T18:40:07.036165+0000","last_undegraded":"2023-07-21T18:40:07.036165+0000","last_fullsized":"2023-07-21T18:40:07.036165+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:05:44.817727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"79'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.090012+0000","last_change":"2023-07-21T18:40:04.180522+0000","last_active":"2023-07-21T18:40:06.090012+0000","last_peered":"2023-07-21T18:40:06.090012+0000","last_clean":"2023-07-21T18:40:06.090012+0000","last_became_active":"2023-07-21T18:40:04.180196+0000","last_became_peered":"2023-07-21T18:40:04.180196+0000","last_unstale":"2023-07-21T18:40:06.090012+0000","last_undegraded":"2023-07-21T18:40:06.090012+0000","last_fullsized":"2023-07-21T18:40:06.090012+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:10:53.779418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.036035+0000","last_change":"2023-07-21T18:40:04.794781+0000","last_active":"2023-07-21T18:40:07.036035+0000","last_peered":"2023-07-21T18:40:07.036035+0000","last_clean":"2023-07-21T18:40:07.036035+0000","last_became_active":"2023-07-21T18:40:04.794388+0000","last_became_peered":"2023-07-21T18:40:04.794388+0000","last_unstale":"2023-07-21T18:40:07.036035+0000","last_undegraded":"2023-07-21T18:40:07.036035+0000","last_fullsized":"2023-07-21T18:40:07.036035+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:26:47.679253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.995411+0000","last_change":"2023-07-21T18:40:04.786408+0000","last_active":"2023-07-21T18:40:06.995411+0000","last_peered":"2023-07-21T18:40:06.995411+0000","last_clean":"2023-07-21T18:40:06.995411+0000","last_became_active":"2023-07-21T18:40:04.785555+0000","last_became_peered":"2023-07-21T18:40:04.785555+0000","last_unstale":"2023-07-21T18:40:06.995411+0000","last_undegraded":"2023-07-21T18:40:06.995411+0000","last_fullsized":"2023-07-21T18:40:06.995411+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:32:35.090936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"79'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.772454+0000","last_change":"2023-07-21T18:40:04.185980+0000","last_active":"2023-07-21T18:40:05.772454+0000","last_peered":"2023-07-21T18:40:05.772454+0000","last_clean":"2023-07-21T18:40:05.772454+0000","last_became_active":"2023-07-21T18:40:04.185618+0000","last_became_peered":"2023-07-21T18:40:04.185618+0000","last_unstale":"2023-07-21T18:40:05.772454+0000","last_undegraded":"2023-07-21T18:40:05.772454+0000","last_fullsized":"2023-07-21T18:40:05.772454+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:44:22.202995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.036366+0000","last_change":"2023-07-21T18:40:04.792535+0000","last_active":"2023-07-21T18:40:07.036366+0000","last_peered":"2023-07-21T18:40:07.036366+0000","last_clean":"2023-07-21T18:40:07.036366+0000","last_became_active":"2023-07-21T18:40:04.792239+0000","last_became_peered":"2023-07-21T18:40:04.792239+0000","last_unstale":"2023-07-21T18:40:07.036366+0000","last_undegraded":"2023-07-21T18:40:07.036366+0000","last_fullsized":"2023-07-21T18:40:07.036366+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:51:14.670073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.035456+0000","last_change":"2023-07-21T18:40:04.173539+0000","last_active":"2023-07-21T18:40:07.035456+0000","last_peered":"2023-07-21T18:40:07.035456+0000","last_clean":"2023-07-21T18:40:07.035456+0000","last_became_active":"2023-07-21T18:40:04.173225+0000","last_became_peered":"2023-07-21T18:40:04.173225+0000","last_unstale":"2023-07-21T18:40:07.035456+0000","last_undegraded":"2023-07-21T18:40:07.035456+0000","last_fullsized":"2023-07-21T18:40:07.035456+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:38:21.873200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":337,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.089887+0000","last_change":"2023-07-21T18:39:05.999983+0000","last_active":"2023-07-21T18:40:06.089887+0000","last_peered":"2023-07-21T18:40:06.089887+0000","last_clean":"2023-07-21T18:40:06.089887+0000","last_became_active":"2023-07-21T18:39:05.999495+0000","last_became_peered":"2023-07-21T18:39:05.999495+0000","last_unstale":"2023-07-21T18:40:06.089887+0000","last_undegraded":"2023-07-21T18:40:06.089887+0000","last_fullsized":"2023-07-21T18:40:06.089887+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:36:14.004429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:36:14.004429+0000","last_clean_scrub_stamp":"2023-07-21T18:36:14.004429+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:34:11.106296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.772327+0000","last_change":"2023-07-21T18:40:04.785086+0000","last_active":"2023-07-21T18:40:05.772327+0000","last_peered":"2023-07-21T18:40:05.772327+0000","last_clean":"2023-07-21T18:40:05.772327+0000","last_became_active":"2023-07-21T18:40:04.784693+0000","last_became_peered":"2023-07-21T18:40:04.784693+0000","last_unstale":"2023-07-21T18:40:05.772327+0000","last_undegraded":"2023-07-21T18:40:05.772327+0000","last_fullsized":"2023-07-21T18:40:05.772327+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:53:45.159424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"79'3","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:09.230670+0000","last_change":"2023-07-21T18:40:04.177032+0000","last_active":"2023-07-21T18:40:09.230670+0000","last_peered":"2023-07-21T18:40:09.230670+0000","last_clean":"2023-07-21T18:40:09.230670+0000","last_became_active":"2023-07-21T18:40:04.176741+0000","last_became_peered":"2023-07-21T18:40:04.176741+0000","last_unstale":"2023-07-21T18:40:09.230670+0000","last_undegraded":"2023-07-21T18:40:09.230670+0000","last_fullsized":"2023-07-21T18:40:09.230670+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:17:23.675128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"79'4","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:10.962237+0000","last_change":"2023-07-21T18:40:04.180106+0000","last_active":"2023-07-21T18:40:10.962237+0000","last_peered":"2023-07-21T18:40:10.962237+0000","last_clean":"2023-07-21T18:40:10.962237+0000","last_became_active":"2023-07-21T18:40:04.179802+0000","last_became_peered":"2023-07-21T18:40:04.179802+0000","last_unstale":"2023-07-21T18:40:10.962237+0000","last_undegraded":"2023-07-21T18:40:10.962237+0000","last_fullsized":"2023-07-21T18:40:10.962237+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:38:16.465170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.772506+0000","last_change":"2023-07-21T18:40:04.789127+0000","last_active":"2023-07-21T18:40:05.772506+0000","last_peered":"2023-07-21T18:40:05.772506+0000","last_clean":"2023-07-21T18:40:05.772506+0000","last_became_active":"2023-07-21T18:40:04.788815+0000","last_became_peered":"2023-07-21T18:40:04.788815+0000","last_unstale":"2023-07-21T18:40:05.772506+0000","last_undegraded":"2023-07-21T18:40:05.772506+0000","last_fullsized":"2023-07-21T18:40:05.772506+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:14:00.509523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.036260+0000","last_change":"2023-07-21T18:40:04.184786+0000","last_active":"2023-07-21T18:40:07.036260+0000","last_peered":"2023-07-21T18:40:07.036260+0000","last_clean":"2023-07-21T18:40:07.036260+0000","last_became_active":"2023-07-21T18:40:04.184433+0000","last_became_peered":"2023-07-21T18:40:04.184433+0000","last_unstale":"2023-07-21T18:40:07.036260+0000","last_undegraded":"2023-07-21T18:40:07.036260+0000","last_fullsized":"2023-07-21T18:40:07.036260+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:39:47.800619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.037220+0000","last_change":"2023-07-21T18:40:04.791937+0000","last_active":"2023-07-21T18:40:07.037220+0000","last_peered":"2023-07-21T18:40:07.037220+0000","last_clean":"2023-07-21T18:40:07.037220+0000","last_became_active":"2023-07-21T18:40:04.791588+0000","last_became_peered":"2023-07-21T18:40:04.791588+0000","last_unstale":"2023-07-21T18:40:07.037220+0000","last_undegraded":"2023-07-21T18:40:07.037220+0000","last_fullsized":"2023-07-21T18:40:07.037220+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:33:01.443024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.035716+0000","last_change":"2023-07-21T18:40:04.177007+0000","last_active":"2023-07-21T18:40:07.035716+0000","last_peered":"2023-07-21T18:40:07.035716+0000","last_clean":"2023-07-21T18:40:07.035716+0000","last_became_active":"2023-07-21T18:40:04.176699+0000","last_became_peered":"2023-07-21T18:40:04.176699+0000","last_unstale":"2023-07-21T18:40:07.035716+0000","last_undegraded":"2023-07-21T18:40:07.035716+0000","last_fullsized":"2023-07-21T18:40:07.035716+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:21:08.441400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.035909+0000","last_change":"2023-07-21T18:40:04.795671+0000","last_active":"2023-07-21T18:40:07.035909+0000","last_peered":"2023-07-21T18:40:07.035909+0000","last_clean":"2023-07-21T18:40:07.035909+0000","last_became_active":"2023-07-21T18:40:04.795233+0000","last_became_peered":"2023-07-21T18:40:04.795233+0000","last_unstale":"2023-07-21T18:40:07.035909+0000","last_undegraded":"2023-07-21T18:40:07.035909+0000","last_fullsized":"2023-07-21T18:40:07.035909+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:26:32.284218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.776720+0000","last_change":"2023-07-21T18:40:04.176859+0000","last_active":"2023-07-21T18:40:05.776720+0000","last_peered":"2023-07-21T18:40:05.776720+0000","last_clean":"2023-07-21T18:40:05.776720+0000","last_became_active":"2023-07-21T18:40:04.176559+0000","last_became_peered":"2023-07-21T18:40:04.176559+0000","last_unstale":"2023-07-21T18:40:05.776720+0000","last_undegraded":"2023-07-21T18:40:05.776720+0000","last_fullsized":"2023-07-21T18:40:05.776720+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:34:23.667073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.006770+0000","last_change":"2023-07-21T18:40:04.793091+0000","last_active":"2023-07-21T18:40:06.006770+0000","last_peered":"2023-07-21T18:40:06.006770+0000","last_clean":"2023-07-21T18:40:06.006770+0000","last_became_active":"2023-07-21T18:40:04.792723+0000","last_became_peered":"2023-07-21T18:40:04.792723+0000","last_unstale":"2023-07-21T18:40:06.006770+0000","last_undegraded":"2023-07-21T18:40:06.006770+0000","last_fullsized":"2023-07-21T18:40:06.006770+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:27:04.440384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'3","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:08.214536+0000","last_change":"2023-07-21T18:40:04.182867+0000","last_active":"2023-07-21T18:40:08.214536+0000","last_peered":"2023-07-21T18:40:08.214536+0000","last_clean":"2023-07-21T18:40:08.214536+0000","last_became_active":"2023-07-21T18:40:04.181885+0000","last_became_peered":"2023-07-21T18:40:04.181885+0000","last_unstale":"2023-07-21T18:40:08.214536+0000","last_undegraded":"2023-07-21T18:40:08.214536+0000","last_fullsized":"2023-07-21T18:40:08.214536+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:03:28.800986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.773091+0000","last_change":"2023-07-21T18:40:04.790340+0000","last_active":"2023-07-21T18:40:05.773091+0000","last_peered":"2023-07-21T18:40:05.773091+0000","last_clean":"2023-07-21T18:40:05.773091+0000","last_became_active":"2023-07-21T18:40:04.790014+0000","last_became_peered":"2023-07-21T18:40:04.790014+0000","last_unstale":"2023-07-21T18:40:05.773091+0000","last_undegraded":"2023-07-21T18:40:05.773091+0000","last_fullsized":"2023-07-21T18:40:05.773091+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:40:26.891907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.592263+0000","last_change":"2023-07-21T18:40:04.184986+0000","last_active":"2023-07-21T18:40:06.592263+0000","last_peered":"2023-07-21T18:40:06.592263+0000","last_clean":"2023-07-21T18:40:06.592263+0000","last_became_active":"2023-07-21T18:40:04.184672+0000","last_became_peered":"2023-07-21T18:40:04.184672+0000","last_unstale":"2023-07-21T18:40:06.592263+0000","last_undegraded":"2023-07-21T18:40:06.592263+0000","last_fullsized":"2023-07-21T18:40:06.592263+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:40:47.955426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.036702+0000","last_change":"2023-07-21T18:40:04.793272+0000","last_active":"2023-07-21T18:40:07.036702+0000","last_peered":"2023-07-21T18:40:07.036702+0000","last_clean":"2023-07-21T18:40:07.036702+0000","last_became_active":"2023-07-21T18:40:04.792854+0000","last_became_peered":"2023-07-21T18:40:04.792854+0000","last_unstale":"2023-07-21T18:40:07.036702+0000","last_undegraded":"2023-07-21T18:40:07.036702+0000","last_fullsized":"2023-07-21T18:40:07.036702+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:22:05.555012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:08.212137+0000","last_change":"2023-07-21T18:40:04.176404+0000","last_active":"2023-07-21T18:40:08.212137+0000","last_peered":"2023-07-21T18:40:08.212137+0000","last_clean":"2023-07-21T18:40:08.212137+0000","last_became_active":"2023-07-21T18:40:04.176148+0000","last_became_peered":"2023-07-21T18:40:04.176148+0000","last_unstale":"2023-07-21T18:40:08.212137+0000","last_undegraded":"2023-07-21T18:40:08.212137+0000","last_fullsized":"2023-07-21T18:40:08.212137+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:29:41.873295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.090128+0000","last_change":"2023-07-21T18:40:04.795628+0000","last_active":"2023-07-21T18:40:06.090128+0000","last_peered":"2023-07-21T18:40:06.090128+0000","last_clean":"2023-07-21T18:40:06.090128+0000","last_became_active":"2023-07-21T18:40:04.795248+0000","last_became_peered":"2023-07-21T18:40:04.795248+0000","last_unstale":"2023-07-21T18:40:06.090128+0000","last_undegraded":"2023-07-21T18:40:06.090128+0000","last_fullsized":"2023-07-21T18:40:06.090128+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:05:26.124641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.592468+0000","last_change":"2023-07-21T18:40:04.184380+0000","last_active":"2023-07-21T18:40:06.592468+0000","last_peered":"2023-07-21T18:40:06.592468+0000","last_clean":"2023-07-21T18:40:06.592468+0000","last_became_active":"2023-07-21T18:40:04.184045+0000","last_became_peered":"2023-07-21T18:40:04.184045+0000","last_unstale":"2023-07-21T18:40:06.592468+0000","last_undegraded":"2023-07-21T18:40:06.592468+0000","last_fullsized":"2023-07-21T18:40:06.592468+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:58:31.782143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.090407+0000","last_change":"2023-07-21T18:40:04.787760+0000","last_active":"2023-07-21T18:40:06.090407+0000","last_peered":"2023-07-21T18:40:06.090407+0000","last_clean":"2023-07-21T18:40:06.090407+0000","last_became_active":"2023-07-21T18:40:04.787371+0000","last_became_peered":"2023-07-21T18:40:04.787371+0000","last_unstale":"2023-07-21T18:40:06.090407+0000","last_undegraded":"2023-07-21T18:40:06.090407+0000","last_fullsized":"2023-07-21T18:40:06.090407+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:10:11.576057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'3","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:10.962506+0000","last_change":"2023-07-21T18:40:04.184721+0000","last_active":"2023-07-21T18:40:10.962506+0000","last_peered":"2023-07-21T18:40:10.962506+0000","last_clean":"2023-07-21T18:40:10.962506+0000","last_became_active":"2023-07-21T18:40:04.184070+0000","last_became_peered":"2023-07-21T18:40:04.184070+0000","last_unstale":"2023-07-21T18:40:10.962506+0000","last_undegraded":"2023-07-21T18:40:10.962506+0000","last_fullsized":"2023-07-21T18:40:10.962506+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:57:46.774015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.090187+0000","last_change":"2023-07-21T18:40:04.786963+0000","last_active":"2023-07-21T18:40:06.090187+0000","last_peered":"2023-07-21T18:40:06.090187+0000","last_clean":"2023-07-21T18:40:06.090187+0000","last_became_active":"2023-07-21T18:40:04.786617+0000","last_became_peered":"2023-07-21T18:40:04.786617+0000","last_unstale":"2023-07-21T18:40:06.090187+0000","last_undegraded":"2023-07-21T18:40:06.090187+0000","last_fullsized":"2023-07-21T18:40:06.090187+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:22:45.940347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.772565+0000","last_change":"2023-07-21T18:40:04.184509+0000","last_active":"2023-07-21T18:40:05.772565+0000","last_peered":"2023-07-21T18:40:05.772565+0000","last_clean":"2023-07-21T18:40:05.772565+0000","last_became_active":"2023-07-21T18:40:04.184208+0000","last_became_peered":"2023-07-21T18:40:04.184208+0000","last_unstale":"2023-07-21T18:40:05.772565+0000","last_undegraded":"2023-07-21T18:40:05.772565+0000","last_fullsized":"2023-07-21T18:40:05.772565+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:18:05.113810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.090384+0000","last_change":"2023-07-21T18:40:04.788248+0000","last_active":"2023-07-21T18:40:06.090384+0000","last_peered":"2023-07-21T18:40:06.090384+0000","last_clean":"2023-07-21T18:40:06.090384+0000","last_became_active":"2023-07-21T18:40:04.787974+0000","last_became_peered":"2023-07-21T18:40:04.787974+0000","last_unstale":"2023-07-21T18:40:06.090384+0000","last_undegraded":"2023-07-21T18:40:06.090384+0000","last_fullsized":"2023-07-21T18:40:06.090384+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:58:18.957824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.773170+0000","last_change":"2023-07-21T18:40:04.790522+0000","last_active":"2023-07-21T18:40:05.773170+0000","last_peered":"2023-07-21T18:40:05.773170+0000","last_clean":"2023-07-21T18:40:05.773170+0000","last_became_active":"2023-07-21T18:40:04.790178+0000","last_became_peered":"2023-07-21T18:40:04.790178+0000","last_unstale":"2023-07-21T18:40:05.773170+0000","last_undegraded":"2023-07-21T18:40:05.773170+0000","last_fullsized":"2023-07-21T18:40:05.773170+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:39:58.451665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.592261+0000","last_change":"2023-07-21T18:40:04.790403+0000","last_active":"2023-07-21T18:40:06.592261+0000","last_peered":"2023-07-21T18:40:06.592261+0000","last_clean":"2023-07-21T18:40:06.592261+0000","last_became_active":"2023-07-21T18:40:04.790106+0000","last_became_peered":"2023-07-21T18:40:04.790106+0000","last_unstale":"2023-07-21T18:40:06.592261+0000","last_undegraded":"2023-07-21T18:40:06.592261+0000","last_fullsized":"2023-07-21T18:40:06.592261+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:22:57.358209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.592288+0000","last_change":"2023-07-21T18:40:04.795230+0000","last_active":"2023-07-21T18:40:06.592288+0000","last_peered":"2023-07-21T18:40:06.592288+0000","last_clean":"2023-07-21T18:40:06.592288+0000","last_became_active":"2023-07-21T18:40:04.794949+0000","last_became_peered":"2023-07-21T18:40:04.794949+0000","last_unstale":"2023-07-21T18:40:06.592288+0000","last_undegraded":"2023-07-21T18:40:06.592288+0000","last_fullsized":"2023-07-21T18:40:06.592288+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:25:52.872388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.037364+0000","last_change":"2023-07-21T18:40:04.792736+0000","last_active":"2023-07-21T18:40:07.037364+0000","last_peered":"2023-07-21T18:40:07.037364+0000","last_clean":"2023-07-21T18:40:07.037364+0000","last_became_active":"2023-07-21T18:40:04.792301+0000","last_became_peered":"2023-07-21T18:40:04.792301+0000","last_unstale":"2023-07-21T18:40:07.037364+0000","last_undegraded":"2023-07-21T18:40:07.037364+0000","last_fullsized":"2023-07-21T18:40:07.037364+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:40:20.170397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.772638+0000","last_change":"2023-07-21T18:40:04.790733+0000","last_active":"2023-07-21T18:40:05.772638+0000","last_peered":"2023-07-21T18:40:05.772638+0000","last_clean":"2023-07-21T18:40:05.772638+0000","last_became_active":"2023-07-21T18:40:04.790441+0000","last_became_peered":"2023-07-21T18:40:04.790441+0000","last_unstale":"2023-07-21T18:40:05.772638+0000","last_undegraded":"2023-07-21T18:40:05.772638+0000","last_fullsized":"2023-07-21T18:40:05.772638+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:29:55.466493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.773446+0000","last_change":"2023-07-21T18:40:04.790853+0000","last_active":"2023-07-21T18:40:05.773446+0000","last_peered":"2023-07-21T18:40:05.773446+0000","last_clean":"2023-07-21T18:40:05.773446+0000","last_became_active":"2023-07-21T18:40:04.790373+0000","last_became_peered":"2023-07-21T18:40:04.790373+0000","last_unstale":"2023-07-21T18:40:05.773446+0000","last_undegraded":"2023-07-21T18:40:05.773446+0000","last_fullsized":"2023-07-21T18:40:05.773446+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:59:25.687337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.592674+0000","last_change":"2023-07-21T18:40:04.794088+0000","last_active":"2023-07-21T18:40:06.592674+0000","last_peered":"2023-07-21T18:40:06.592674+0000","last_clean":"2023-07-21T18:40:06.592674+0000","last_became_active":"2023-07-21T18:40:04.793782+0000","last_became_peered":"2023-07-21T18:40:04.793782+0000","last_unstale":"2023-07-21T18:40:06.592674+0000","last_undegraded":"2023-07-21T18:40:06.592674+0000","last_fullsized":"2023-07-21T18:40:06.592674+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:14:52.604091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.037053+0000","last_change":"2023-07-21T18:40:04.791772+0000","last_active":"2023-07-21T18:40:07.037053+0000","last_peered":"2023-07-21T18:40:07.037053+0000","last_clean":"2023-07-21T18:40:07.037053+0000","last_became_active":"2023-07-21T18:40:04.791379+0000","last_became_peered":"2023-07-21T18:40:04.791379+0000","last_unstale":"2023-07-21T18:40:07.037053+0000","last_undegraded":"2023-07-21T18:40:07.037053+0000","last_fullsized":"2023-07-21T18:40:07.037053+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:37:36.712528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.994956+0000","last_change":"2023-07-21T18:40:04.786579+0000","last_active":"2023-07-21T18:40:06.994956+0000","last_peered":"2023-07-21T18:40:06.994956+0000","last_clean":"2023-07-21T18:40:06.994956+0000","last_became_active":"2023-07-21T18:40:04.786032+0000","last_became_peered":"2023-07-21T18:40:04.786032+0000","last_unstale":"2023-07-21T18:40:06.994956+0000","last_undegraded":"2023-07-21T18:40:06.994956+0000","last_fullsized":"2023-07-21T18:40:06.994956+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:34:12.940522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.036500+0000","last_change":"2023-07-21T18:40:04.793441+0000","last_active":"2023-07-21T18:40:07.036500+0000","last_peered":"2023-07-21T18:40:07.036500+0000","last_clean":"2023-07-21T18:40:07.036500+0000","last_became_active":"2023-07-21T18:40:04.791441+0000","last_became_peered":"2023-07-21T18:40:04.791441+0000","last_unstale":"2023-07-21T18:40:07.036500+0000","last_undegraded":"2023-07-21T18:40:07.036500+0000","last_fullsized":"2023-07-21T18:40:07.036500+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:01:49.043942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.777066+0000","last_change":"2023-07-21T18:40:04.794913+0000","last_active":"2023-07-21T18:40:05.777066+0000","last_peered":"2023-07-21T18:40:05.777066+0000","last_clean":"2023-07-21T18:40:05.777066+0000","last_became_active":"2023-07-21T18:40:04.794527+0000","last_became_peered":"2023-07-21T18:40:04.794527+0000","last_unstale":"2023-07-21T18:40:05.777066+0000","last_undegraded":"2023-07-21T18:40:05.777066+0000","last_fullsized":"2023-07-21T18:40:05.777066+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:13:56.682713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.772671+0000","last_change":"2023-07-21T18:40:04.789145+0000","last_active":"2023-07-21T18:40:05.772671+0000","last_peered":"2023-07-21T18:40:05.772671+0000","last_clean":"2023-07-21T18:40:05.772671+0000","last_became_active":"2023-07-21T18:40:04.788285+0000","last_became_peered":"2023-07-21T18:40:04.788285+0000","last_unstale":"2023-07-21T18:40:05.772671+0000","last_undegraded":"2023-07-21T18:40:05.772671+0000","last_fullsized":"2023-07-21T18:40:05.772671+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:42:10.814436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.773199+0000","last_change":"2023-07-21T18:40:04.788821+0000","last_active":"2023-07-21T18:40:05.773199+0000","last_peered":"2023-07-21T18:40:05.773199+0000","last_clean":"2023-07-21T18:40:05.773199+0000","last_became_active":"2023-07-21T18:40:04.788513+0000","last_became_peered":"2023-07-21T18:40:04.788513+0000","last_unstale":"2023-07-21T18:40:05.773199+0000","last_undegraded":"2023-07-21T18:40:05.773199+0000","last_fullsized":"2023-07-21T18:40:05.773199+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:35:28.640398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5176,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":13,"num_read_kb":12,"num_write":107,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":204800,"data_stored":14366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":98,"ondisk_log_size":98,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579916,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27780,"kb_used_data":828,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717372,"statfs":{"total":95995035648,"available":95966588928,"internally_reserved":0,"allocated":847872,"data_stored":407750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]}]},{"osd":10,"up_from":67,"seq":287762808847,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28392,"kb_used_data":1440,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716760,"statfs":{"total":95995035648,"available":95965962240,"internally_reserved":0,"allocated":1474560,"data_stored":1000307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":1,"last update":"Fri Jul 21 18:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.33200000000000002}]},{"osd":2,"last update":"Fri Jul 21 18:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.621},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.57199999999999995}]},{"osd":3,"last update":"Fri Jul 21 18:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.38600000000000001}]},{"osd":4,"last update":"Fri Jul 21 18:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.371}]},{"osd":5,"last update":"Fri Jul 21 18:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.73499999999999999}]},{"osd":6,"last update":"Fri Jul 21 18:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.505},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.46999999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":8,"last update":"Fri Jul 21 18:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.70999999999999996}]},{"osd":9,"last update":"Fri Jul 21 18:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.58599999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]}]},{"osd":9,"up_from":62,"seq":266287972372,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27772,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717380,"statfs":{"total":95995035648,"available":95966597120,"internally_reserved":0,"allocated":839680,"data_stored":399762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:39:45 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.6319999999999999,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.45400000000000001}]},{"osd":1,"last update":"Fri Jul 21 18:39:45 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":2.1000000000000001,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.681,"5min":1.681,"15min":1.681},"last":0.48899999999999999}]},{"osd":2,"last update":"Fri Jul 21 18:39:45 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.9530000000000001,"5min":1.9530000000000001,"15min":1.9530000000000001},"last":0.623},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.7170000000000001,"5min":1.7170000000000001,"15min":1.7170000000000001},"last":0.61099999999999999}]},{"osd":3,"last update":"Fri Jul 21 18:39:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":2.048,"5min":2.048,"15min":2.048},"last":0.54700000000000004}]},{"osd":4,"last update":"Fri Jul 21 18:39:45 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.8100000000000001,"5min":1.8100000000000001,"15min":1.8100000000000001},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.56000000000000005}]},{"osd":5,"last update":"Fri Jul 21 18:39:45 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.992,"5min":1.992,"15min":1.992},"last":0.72699999999999998}]},{"osd":6,"last update":"Fri Jul 21 18:39:45 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":2.0209999999999999,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.595,"5min":1.595,"15min":1.595},"last":0.57599999999999996}]},{"osd":7,"last update":"Fri Jul 21 18:39:45 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":2.0750000000000002,"5min":2.0750000000000002,"15min":2.0750000000000002},"last":0.66100000000000003}]},{"osd":8,"last update":"Fri Jul 21 18:39:45 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":2.2890000000000001,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.74199999999999999}]},{"osd":10,"last update":"Fri Jul 21 18:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":2.1240000000000001,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.67200000000000004}]}]},{"osd":8,"up_from":55,"seq":236223201305,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27796,"kb_used_data":844,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717356,"statfs":{"total":95995035648,"available":95966572544,"internally_reserved":0,"allocated":864256,"data_stored":408157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:39:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.38700000000000001}]},{"osd":1,"last update":"Fri Jul 21 18:39:23 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.79300000000000004}]},{"osd":2,"last update":"Fri Jul 21 18:39:23 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.434},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.33400000000000002}]},{"osd":3,"last update":"Fri Jul 21 18:39:23 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.41299999999999998}]},{"osd":4,"last update":"Fri Jul 21 18:39:23 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.72099999999999997}]},{"osd":5,"last update":"Fri Jul 21 18:39:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.46200000000000002}]},{"osd":6,"last update":"Fri Jul 21 18:39:23 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.65800000000000003}]},{"osd":7,"last update":"Fri Jul 21 18:39:23 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.64100000000000001}]},{"osd":9,"last update":"Fri Jul 21 18:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.621},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.73699999999999999}]},{"osd":10,"last update":"Fri Jul 21 18:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.86799999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]}]},{"osd":7,"up_from":48,"seq":206158430237,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28372,"kb_used_data":1420,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716780,"statfs":{"total":95995035648,"available":95965982720,"internally_reserved":0,"allocated":1454080,"data_stored":999541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.53400000000000003,"15min":0.52600000000000002},"min":{"1min":0.41599999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.53900000000000003,"15min":0.53600000000000003},"min":{"1min":0.34000000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.48899999999999999}]},{"osd":1,"last update":"Fri Jul 21 18:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.41099999999999998,"15min":0.40400000000000003},"min":{"1min":0.311,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.43099999999999999,"15min":0.42299999999999999},"min":{"1min":0.307,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.71099999999999997,"5min":0.75,"15min":0.75},"last":0.39100000000000001}]},{"osd":2,"last update":"Fri Jul 21 18:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.57599999999999996,"15min":0.57099999999999995},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58899999999999997,"15min":0.58999999999999997},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.72699999999999998}]},{"osd":3,"last update":"Fri Jul 21 18:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.54900000000000004,"15min":0.54000000000000004},"min":{"1min":0.39700000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.58299999999999996,"15min":0.57199999999999995},"min":{"1min":0.44500000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.86899999999999999,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.64600000000000002}]},{"osd":4,"last update":"Fri Jul 21 18:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.52700000000000002,"15min":0.52100000000000002},"min":{"1min":0.35399999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.52500000000000002,"15min":0.51700000000000002},"min":{"1min":0.39800000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.89500000000000002,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.33800000000000002}]},{"osd":5,"last update":"Fri Jul 21 18:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.64800000000000002,"15min":0.64100000000000001},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.64600000000000002,"15min":0.64200000000000002},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.59999999999999998}]},{"osd":6,"last update":"Fri Jul 21 18:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.66500000000000004,"15min":0.65500000000000003},"min":{"1min":0.51300000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.93500000000000005,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.67300000000000004,"15min":0.66100000000000003},"min":{"1min":0.46400000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0069999999999999,"5min":1.087,"15min":1.087},"last":0.58799999999999997}]},{"osd":8,"last update":"Fri Jul 21 18:39:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.71699999999999997}]},{"osd":9,"last update":"Fri Jul 21 18:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.51000000000000001}]},{"osd":10,"last update":"Fri Jul 21 18:40:06 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.63}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]}]},{"osd":6,"up_from":43,"seq":184683593761,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27776,"kb_used_data":824,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717376,"statfs":{"total":95995035648,"available":95966593024,"internally_reserved":0,"allocated":843776,"data_stored":407477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.41099999999999998,"15min":0.40799999999999997},"min":{"1min":0.30199999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.68200000000000005,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.43099999999999999,"15min":0.42799999999999999},"min":{"1min":0.28899999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.311}]},{"osd":1,"last update":"Fri Jul 21 18:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.61199999999999999,"15min":0.60799999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.872,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.621,"5min":0.58999999999999997,"15min":0.58499999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.629}]},{"osd":2,"last update":"Fri Jul 21 18:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.59799999999999998,"15min":0.58799999999999997},"min":{"1min":0.38500000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.441},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.59499999999999997,"15min":0.58099999999999996},"min":{"1min":0.47199999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.42099999999999999}]},{"osd":3,"last update":"Fri Jul 21 18:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.47599999999999998,"15min":0.47199999999999998},"min":{"1min":0.33600000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.81899999999999995,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.51600000000000001,"15min":0.51300000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.39900000000000002}]},{"osd":4,"last update":"Fri Jul 21 18:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.64800000000000002,"15min":0.64700000000000002},"min":{"1min":0.40899999999999997,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.65200000000000002,"15min":0.64800000000000002},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.57799999999999996}]},{"osd":5,"last update":"Fri Jul 21 18:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.66100000000000003,"15min":0.65100000000000002},"min":{"1min":0.438,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.70999999999999996,"15min":0.69799999999999995},"min":{"1min":0.53500000000000003,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.147,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.49299999999999999}]},{"osd":7,"last update":"Fri Jul 21 18:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.71299999999999997,"15min":0.70899999999999996},"min":{"1min":0.53500000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.70599999999999996,"15min":0.70499999999999996},"min":{"1min":0.54300000000000004,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.73199999999999998}]},{"osd":8,"last update":"Fri Jul 21 18:39:24 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.51000000000000001}]},{"osd":9,"last update":"Fri Jul 21 18:39:42 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.55300000000000005}]},{"osd":10,"last update":"Fri Jul 21 18:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.68999999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574902,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27788,"kb_used_data":836,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717364,"statfs":{"total":95995035648,"available":95966580736,"internally_reserved":0,"allocated":856064,"data_stored":409041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:39:59 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.50900000000000001,"15min":0.48499999999999999},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.63},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.52800000000000002,"15min":0.498},"min":{"1min":0.39200000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.51000000000000001}]},{"osd":2,"last update":"Fri Jul 21 18:39:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.53900000000000003,"15min":0.52100000000000002},"min":{"1min":0.38800000000000001,"5min":0.313,"15min":0.313},"max":{"1min":3.2959999999999998,"5min":3.2959999999999998,"15min":3.2959999999999998},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.625,"5min":0.53400000000000003,"15min":0.52000000000000002},"min":{"1min":0.34200000000000003,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":2.8650000000000002,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":0.75}]},{"osd":3,"last update":"Fri Jul 21 18:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.53600000000000003,"15min":0.52000000000000002},"min":{"1min":0.35299999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.88200000000000001,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.62,"5min":0.54500000000000004,"15min":0.52500000000000002},"min":{"1min":0.379,"5min":0.377,"15min":0.377},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.58899999999999997}]},{"osd":4,"last update":"Fri Jul 21 18:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.41799999999999998,"15min":0.40100000000000002},"min":{"1min":0.32600000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.497,"5min":0.41799999999999998,"15min":0.40500000000000003},"min":{"1min":0.31,"5min":0.254,"15min":0.254},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.46800000000000003}]},{"osd":5,"last update":"Fri Jul 21 18:39:19 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.58699999999999997,"15min":0.58099999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.653,"5min":1.653,"15min":1.653},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.59799999999999998,"15min":0.58999999999999997},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.641,"5min":1.641,"15min":1.641},"last":0.82099999999999995}]},{"osd":6,"last update":"Fri Jul 21 18:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.63600000000000001,"15min":0.63},"min":{"1min":0.45900000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.64100000000000001,"15min":0.63900000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.84299999999999997}]},{"osd":7,"last update":"Fri Jul 21 18:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.51900000000000002,"15min":0.502},"min":{"1min":0.41399999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.54800000000000004,"15min":0.53000000000000003},"min":{"1min":0.40699999999999997,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.93000000000000005}]},{"osd":8,"last update":"Fri Jul 21 18:39:22 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":2.7360000000000002,"5min":2.7360000000000002,"15min":2.7360000000000002},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":2.7210000000000001,"5min":2.7210000000000001,"15min":2.7210000000000001},"last":0.77400000000000002}]},{"osd":9,"last update":"Fri Jul 21 18:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":1.034}]},{"osd":10,"last update":"Fri Jul 21 18:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":1.159}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0800000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738426,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27784,"kb_used_data":832,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717368,"statfs":{"total":95995035648,"available":95966584832,"internally_reserved":0,"allocated":851968,"data_stored":407840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 21 18:39:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.59099999999999997,"15min":0.57399999999999995},"min":{"1min":0.36299999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.88200000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.57699999999999996,"15min":0.53700000000000003},"min":{"1min":0.42899999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.71499999999999997}]},{"osd":2,"last update":"Fri Jul 21 18:39:16 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.57399999999999995,"15min":0.56399999999999995},"min":{"1min":0.44800000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.81899999999999995,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.56999999999999995,"15min":0.57399999999999995},"min":{"1min":0.46000000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.83499999999999996,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":0.81499999999999995}]},{"osd":3,"last update":"Fri Jul 21 18:38:36 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.42799999999999999,"15min":0.42999999999999999},"min":{"1min":0.29999999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.621,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.48199999999999998,"15min":0.47699999999999998},"min":{"1min":0.33600000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.69199999999999995,"5min":0.879,"15min":0.879},"last":0.56999999999999995}]},{"osd":4,"last update":"Fri Jul 21 18:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.60599999999999998,"15min":0.57199999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":2.948,"5min":2.948,"15min":2.948},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.56200000000000006,"15min":0.54600000000000004},"min":{"1min":0.40600000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.80400000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.53800000000000003}]},{"osd":5,"last update":"Fri Jul 21 18:39:20 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.60099999999999998,"15min":0.58599999999999997},"min":{"1min":0.54200000000000004,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.61499999999999999,"15min":0.60199999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.93999999999999995}]},{"osd":6,"last update":"Fri Jul 21 18:39:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.52500000000000002,"15min":0.51200000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.58099999999999996,"15min":0.56699999999999995},"min":{"1min":0.36599999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.66500000000000004}]},{"osd":7,"last update":"Fri Jul 21 18:39:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.47699999999999998}]},{"osd":8,"last update":"Fri Jul 21 18:39:22 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.97099999999999997}]},{"osd":9,"last update":"Fri Jul 21 18:39:44 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.296,"5min":1.296,"15min":1.296},"last":0.73299999999999998}]},{"osd":10,"last update":"Fri Jul 21 18:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":2.9079999999999999,"5min":2.9079999999999999,"15min":2.9079999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.77800000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411378,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27804,"kb_used_data":852,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717348,"statfs":{"total":95995035648,"available":95966564352,"internally_reserved":0,"allocated":872448,"data_stored":409922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:39:17 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.50600000000000001,"15min":0.502},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.52000000000000002}]},{"osd":1,"last update":"Fri Jul 21 18:39:17 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.51600000000000001,"15min":0.51300000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.53800000000000003,"15min":0.53000000000000003},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.90800000000000003}]},{"osd":3,"last update":"Fri Jul 21 18:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.57299999999999995,"15min":0.56000000000000005},"min":{"1min":0.45000000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.56200000000000006,"15min":0.55800000000000005},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.63}]},{"osd":4,"last update":"Fri Jul 21 18:39:57 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.58999999999999997,"15min":0.56999999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.633,"5min":1.633,"15min":1.633},"last":0.87},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.626,"15min":0.60899999999999999},"min":{"1min":0.42999999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.79100000000000004}]},{"osd":5,"last update":"Fri Jul 21 18:39:17 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.44800000000000001,"15min":0.44900000000000001},"min":{"1min":0.32600000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.71799999999999997,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.45300000000000001,"15min":0.45100000000000001},"min":{"1min":0.32100000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.57299999999999995}]},{"osd":6,"last update":"Fri Jul 21 18:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.59999999999999998,"15min":0.58999999999999997},"min":{"1min":0.42999999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.61899999999999999,"15min":0.61699999999999999},"min":{"1min":0.39500000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.70599999999999996}]},{"osd":7,"last update":"Fri Jul 21 18:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.61699999999999999,"15min":0.60799999999999998},"min":{"1min":0.45100000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.577,"5min":1.577,"15min":1.577},"last":0.495},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.65500000000000003,"15min":0.65100000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.878,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.56499999999999995}]},{"osd":8,"last update":"Fri Jul 21 18:39:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.73299999999999998}]},{"osd":9,"last update":"Fri Jul 21 18:39:42 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.748}]},{"osd":10,"last update":"Fri Jul 21 18:40:06 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.95799999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93500000000000005}]}]},{"osd":3,"up_from":24,"seq":103079215149,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27784,"kb_used_data":832,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717368,"statfs":{"total":95995035648,"available":95966584832,"internally_reserved":0,"allocated":851968,"data_stored":407340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:39:36 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.34799999999999998,"15min":0.32800000000000001},"min":{"1min":0.28399999999999997,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.442,"5min":0.36899999999999999,"15min":0.34899999999999998},"min":{"1min":0.28399999999999997,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.38}]},{"osd":1,"last update":"Fri Jul 21 18:39:36 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.56999999999999995,"15min":0.55000000000000004},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.57899999999999996,"15min":0.54500000000000004},"min":{"1min":0.495,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.83899999999999997,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.57199999999999995}]},{"osd":2,"last update":"Fri Jul 21 18:39:36 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.57599999999999996,"15min":0.55300000000000005},"min":{"1min":0.48799999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.55300000000000005,"15min":0.53400000000000003},"min":{"1min":0.46400000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.67400000000000004}]},{"osd":4,"last update":"Fri Jul 21 18:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.61599999999999999,"15min":0.60499999999999998},"min":{"1min":0.42799999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.61699999999999999,"15min":0.59999999999999998},"min":{"1min":0.45300000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.69499999999999995}]},{"osd":5,"last update":"Fri Jul 21 18:39:19 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.60799999999999998,"15min":0.58599999999999997},"min":{"1min":0.49099999999999999,"5min":0.374,"15min":0.374},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.59099999999999997,"15min":0.57899999999999996},"min":{"1min":0.44,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.754}]},{"osd":6,"last update":"Fri Jul 21 18:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.5,"15min":0.48699999999999999},"min":{"1min":0.32300000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.57199999999999995,"15min":0.55900000000000005},"min":{"1min":0.40600000000000003,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.625}]},{"osd":7,"last update":"Fri Jul 21 18:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.66800000000000004,"15min":0.67000000000000004},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.97299999999999998,"5min":0.999,"15min":0.999},"last":0.5},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.73299999999999998,"15min":0.72499999999999998},"min":{"1min":0.55100000000000005,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.77600000000000002}]},{"osd":8,"last update":"Fri Jul 21 18:39:25 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.47299999999999998}]},{"osd":9,"last update":"Fri Jul 21 18:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.879}]},{"osd":10,"last update":"Fri Jul 21 18:40:07 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.81499999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]}]},{"osd":4,"up_from":31,"seq":133143986217,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27768,"kb_used_data":816,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717384,"statfs":{"total":95995035648,"available":95966601216,"internally_reserved":0,"allocated":835584,"data_stored":398929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:40:02 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.51800000000000002,"15min":0.48199999999999998},"min":{"1min":0.40500000000000003,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.94099999999999995,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.52500000000000002,"15min":0.50600000000000001},"min":{"1min":0.373,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.80600000000000005,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.58299999999999996}]},{"osd":1,"last update":"Fri Jul 21 18:40:02 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.373,"15min":0.34599999999999997},"min":{"1min":0.29799999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.40899999999999997,"15min":0.38600000000000001},"min":{"1min":0.30399999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.76200000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.43099999999999999}]},{"osd":2,"last update":"Fri Jul 21 18:40:02 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.54700000000000004,"15min":0.52900000000000003},"min":{"1min":0.44700000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.80700000000000005,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.56200000000000006,"15min":0.53200000000000003},"min":{"1min":0.41799999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.51500000000000001}]},{"osd":3,"last update":"Fri Jul 21 18:40:02 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.55800000000000005,"15min":0.53300000000000003},"min":{"1min":0.52300000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.85999999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56299999999999994,"15min":0.55700000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.97199999999999998,"5min":1.05,"15min":1.05},"last":0.57099999999999995}]},{"osd":5,"last update":"Fri Jul 21 18:39:18 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.61699999999999999,"15min":0.59799999999999998},"min":{"1min":0.42599999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.571,"5min":1.571,"15min":1.571},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.624,"15min":0.60899999999999999},"min":{"1min":0.50900000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":0.56000000000000005}]},{"osd":6,"last update":"Fri Jul 21 18:39:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.63100000000000001,"15min":0.61399999999999999},"min":{"1min":0.53000000000000003,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.63200000000000001,"15min":0.60799999999999998},"min":{"1min":0.54100000000000004,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.63}]},{"osd":7,"last update":"Fri Jul 21 18:40:02 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.57299999999999995,"15min":0.56100000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.54500000000000004,"15min":0.54000000000000004},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.873,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.41199999999999998}]},{"osd":8,"last update":"Fri Jul 21 18:39:21 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.505},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.59199999999999997}]},{"osd":9,"last update":"Fri Jul 21 18:39:44 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.621},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.64900000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]}]},{"osd":5,"up_from":36,"seq":154618822693,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28400,"kb_used_data":1448,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716752,"statfs":{"total":95995035648,"available":95965954048,"internally_reserved":0,"allocated":1482752,"data_stored":1000617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:39:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.53500000000000003,"15min":0.53300000000000003},"min":{"1min":0.29499999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.55600000000000005,"15min":0.56000000000000005},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.48499999999999999}]},{"osd":1,"last update":"Fri Jul 21 18:39:20 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.50700000000000001,"15min":0.499},"min":{"1min":0.35899999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.51800000000000002,"15min":0.50600000000000001},"min":{"1min":0.41099999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.75800000000000001}]},{"osd":2,"last update":"Fri Jul 21 18:39:20 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.38900000000000001,"15min":0.38400000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.40200000000000002,"15min":0.39300000000000002},"min":{"1min":0.29399999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.57199999999999995}]},{"osd":3,"last update":"Fri Jul 21 18:39:20 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.56899999999999995,"15min":0.57099999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58199999999999996,"15min":0.58099999999999996},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.69199999999999995}]},{"osd":4,"last update":"Fri Jul 21 18:39:20 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.55900000000000005,"15min":0.54900000000000004},"min":{"1min":0.433,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.54000000000000004,"15min":0.53200000000000003},"min":{"1min":0.41299999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.80000000000000004}]},{"osd":6,"last update":"Fri Jul 21 18:39:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.60099999999999998,"15min":0.60199999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.59799999999999998,"15min":0.58899999999999997},"min":{"1min":0.48299999999999998,"5min":0.434,"15min":0.434},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.78200000000000003}]},{"osd":7,"last update":"Fri Jul 21 18:40:02 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.66600000000000004,"15min":0.66800000000000004},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.64300000000000002,"15min":0.64000000000000001},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.85999999999999999}]},{"osd":8,"last update":"Fri Jul 21 18:39:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.71999999999999997}]},{"osd":9,"last update":"Fri Jul 21 18:39:45 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.877},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.88900000000000001}]},{"osd":10,"last update":"Fri Jul 21 18:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.94199999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-21T18:40:13.043 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-21T18:40:13.196 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:13.169+0000 7fdc3b324700 1 -- 172.21.15.39:0/1004012203 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc340ffbc0 msgr2=0x7fdc34100010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:13.196 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:13.169+0000 7fdc3b324700 1 --2- 172.21.15.39:0/1004012203 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc340ffbc0 0x7fdc34100010 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fdc24009990 tx=0x7fdc24009ca0 comp rx=0 tx=0).stop 2023-07-21T18:40:13.196 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:13.169+0000 7fdc3b324700 1 -- 172.21.15.39:0/1004012203 shutdown_connections 2023-07-21T18:40:13.197 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:13.169+0000 7fdc3b324700 1 --2- 172.21.15.39:0/1004012203 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc34100550 0x7fdc34106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:13.197 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:13.169+0000 7fdc3b324700 1 --2- 172.21.15.39:0/1004012203 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc340fe9d0 0x7fdc340fede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:13.197 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:13.169+0000 7fdc3b324700 1 --2- 172.21.15.39:0/1004012203 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc340ffbc0 0x7fdc34100010 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:13.197 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:13.169+0000 7fdc3b324700 1 -- 172.21.15.39:0/1004012203 >> 172.21.15.39:0/1004012203 conn(0x7fdc340f9fd0 msgr2=0x7fdc340fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:13.197 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:13.169+0000 7fdc3b324700 1 -- 172.21.15.39:0/1004012203 shutdown_connections 2023-07-21T18:40:13.197 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:13.169+0000 7fdc3b324700 1 -- 172.21.15.39:0/1004012203 wait complete. 2023-07-21T18:40:13.197 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:13.170+0000 7fdc3b324700 1 Processor -- start 2023-07-21T18:40:13.198 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:13.170+0000 7fdc3b324700 1 -- start start 2023-07-21T18:40:13.198 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:13.170+0000 7fdc3b324700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc340fe9d0 0x7fdc341092c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:13.198 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:13.170+0000 7fdc3b324700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc340ffbc0 0x7fdc34107900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:13.198 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:13.170+0000 7fdc3b324700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc34100550 0x7fdc34107e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:13.198 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:13.170+0000 7fdc3b324700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc341085e0 con 0x7fdc340fe9d0 2023-07-21T18:40:13.198 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:13.170+0000 7fdc3b324700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc34108750 con 0x7fdc34100550 2023-07-21T18:40:13.198 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:13.170+0000 7fdc3b324700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3419f840 con 0x7fdc340ffbc0 2023-07-21T18:40:13.198 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:13.170+0000 7fdc390c0700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc340fe9d0 0x7fdc341092c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:13.199 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:13.170+0000 7fdc390c0700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc340fe9d0 0x7fdc341092c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55250/0 (socket says 172.21.15.39:55250) 2023-07-21T18:40:13.199 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:13.170+0000 7fdc390c0700 1 -- 172.21.15.39:0/3343815859 learned_addr learned my addr 172.21.15.39:0/3343815859 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:40:13.199 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:13.170+0000 7fdc388bf700 1 --2- 172.21.15.39:0/3343815859 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc340ffbc0 0x7fdc34107900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:13.199 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:13.170+0000 7fdc398c1700 1 --2- 172.21.15.39:0/3343815859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc34100550 0x7fdc34107e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:13.199 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:13.170+0000 7fdc390c0700 1 -- 172.21.15.39:0/3343815859 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc340ffbc0 msgr2=0x7fdc34107900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:13.199 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:13.170+0000 7fdc390c0700 1 --2- 172.21.15.39:0/3343815859 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc340ffbc0 0x7fdc34107900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:13.199 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:13.170+0000 7fdc390c0700 1 -- 172.21.15.39:0/3343815859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc34100550 msgr2=0x7fdc34107e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:13.199 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:13.170+0000 7fdc390c0700 1 --2- 172.21.15.39:0/3343815859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc34100550 0x7fdc34107e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:13.200 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:13.170+0000 7fdc390c0700 1 -- 172.21.15.39:0/3343815859 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc240095f0 con 0x7fdc340fe9d0 2023-07-21T18:40:13.200 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:13.170+0000 7fdc398c1700 1 --2- 172.21.15.39:0/3343815859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc34100550 0x7fdc34107e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:40:13.200 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:13.170+0000 7fdc390c0700 1 --2- 172.21.15.39:0/3343815859 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc340fe9d0 0x7fdc341092c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fdc3000d980 tx=0x7fdc3000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:13.200 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:13.171+0000 7fdc2a7fc700 1 -- 172.21.15.39:0/3343815859 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc3000de60 con 0x7fdc340fe9d0 2023-07-21T18:40:13.200 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:13.171+0000 7fdc2a7fc700 1 -- 172.21.15.39:0/3343815859 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc300044e0 con 0x7fdc340fe9d0 2023-07-21T18:40:13.200 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:13.171+0000 7fdc3b324700 1 -- 172.21.15.39:0/3343815859 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc3419fb20 con 0x7fdc340fe9d0 2023-07-21T18:40:13.200 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:13.171+0000 7fdc2a7fc700 1 -- 172.21.15.39:0/3343815859 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc30003be0 con 0x7fdc340fe9d0 2023-07-21T18:40:13.201 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:13.171+0000 7fdc3b324700 1 -- 172.21.15.39:0/3343815859 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc341a0030 con 0x7fdc340fe9d0 2023-07-21T18:40:13.201 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:13.172+0000 7fdc3b324700 1 -- 172.21.15.39:0/3343815859 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc3410b3d0 con 0x7fdc340fe9d0 2023-07-21T18:40:13.202 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:13.173+0000 7fdc2a7fc700 1 -- 172.21.15.39:0/3343815859 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fdc30009bb0 con 0x7fdc340fe9d0 2023-07-21T18:40:13.202 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:13.173+0000 7fdc2a7fc700 1 --2- 172.21.15.39:0/3343815859 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fdc200a1220 0x7fdc200a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:13.202 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:13.173+0000 7fdc388bf700 1 --2- 172.21.15.39:0/3343815859 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fdc200a1220 0x7fdc200a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:13.202 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:13.173+0000 7fdc2a7fc700 1 -- 172.21.15.39:0/3343815859 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fdc300cc6a0 con 0x7fdc340fe9d0 2023-07-21T18:40:13.202 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:13.174+0000 7fdc388bf700 1 --2- 172.21.15.39:0/3343815859 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fdc200a1220 0x7fdc200a36d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fdc2400b5c0 tx=0x7fdc24005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:13.203 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:13.177+0000 7fdc2a7fc700 1 -- 172.21.15.39:0/3343815859 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fdc30095e40 con 0x7fdc340fe9d0 2023-07-21T18:40:13.373 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:12 smithi193 ceph-mon[98204]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:40:13.374 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:12 smithi193 ceph-mon[98204]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:40:13.374 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:12 smithi193 ceph-mon[98204]: mgrmap e17: x(active, since 6m), standbys: z, y 2023-07-21T18:40:13.374 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:12 smithi193 ceph-mon[98204]: mds.? [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] up:active 2023-07-21T18:40:13.374 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:12 smithi193 ceph-mon[98204]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2023-07-21T18:40:13.374 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:12 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:40:13.374 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:12 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/1917891167' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:40:13.374 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:12 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/4202342743' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T18:40:13.392 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:12 smithi155 ceph-mon[97934]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:40:13.392 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:12 smithi155 ceph-mon[97934]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:40:13.392 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:12 smithi155 ceph-mon[97934]: mgrmap e17: x(active, since 6m), standbys: z, y 2023-07-21T18:40:13.393 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:12 smithi155 ceph-mon[97934]: mds.? [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] up:active 2023-07-21T18:40:13.393 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:12 smithi155 ceph-mon[97934]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2023-07-21T18:40:13.393 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:12 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:40:13.393 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:12 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/1917891167' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:40:13.393 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:12 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/4202342743' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T18:40:13.399 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:13.373+0000 7fdc3b324700 1 -- 172.21.15.39:0/3343815859 --> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdc34060950 con 0x7fdc200a1220 2023-07-21T18:40:13.410 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:13.385+0000 7fdc2a7fc700 1 -- 172.21.15.39:0/3343815859 <== mgr.14152 v2:172.21.15.39:6800/2116291712 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+336369 (secure 0 0 0) 0x7fdc34060950 con 0x7fdc200a1220 2023-07-21T18:40:13.414 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:40:13.417 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:13.388+0000 7fdc3b324700 1 -- 172.21.15.39:0/3343815859 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fdc200a1220 msgr2=0x7fdc200a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:13.417 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:13.388+0000 7fdc3b324700 1 --2- 172.21.15.39:0/3343815859 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fdc200a1220 0x7fdc200a36d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fdc2400b5c0 tx=0x7fdc24005fb0 comp rx=0 tx=0).stop 2023-07-21T18:40:13.417 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:13.388+0000 7fdc3b324700 1 -- 172.21.15.39:0/3343815859 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc340fe9d0 msgr2=0x7fdc341092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:13.417 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:13.388+0000 7fdc3b324700 1 --2- 172.21.15.39:0/3343815859 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc340fe9d0 0x7fdc341092c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fdc3000d980 tx=0x7fdc3000dd40 comp rx=0 tx=0).stop 2023-07-21T18:40:13.417 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:13.388+0000 7fdc3b324700 1 -- 172.21.15.39:0/3343815859 shutdown_connections 2023-07-21T18:40:13.418 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:13.388+0000 7fdc3b324700 1 --2- 172.21.15.39:0/3343815859 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fdc200a1220 0x7fdc200a36d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:13.418 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:13.388+0000 7fdc3b324700 1 --2- 172.21.15.39:0/3343815859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc34100550 0x7fdc34107e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:13.418 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:13.388+0000 7fdc3b324700 1 --2- 172.21.15.39:0/3343815859 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc340fe9d0 0x7fdc341092c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:13.418 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:13.389+0000 7fdc3b324700 1 --2- 172.21.15.39:0/3343815859 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc340ffbc0 0x7fdc34107900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:13.419 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:13.389+0000 7fdc3b324700 1 -- 172.21.15.39:0/3343815859 >> 172.21.15.39:0/3343815859 conn(0x7fdc340f9fd0 msgr2=0x7fdc340fc180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:13.419 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:13.389+0000 7fdc3b324700 1 -- 172.21.15.39:0/3343815859 shutdown_connections 2023-07-21T18:40:13.419 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:13.389+0000 7fdc3b324700 1 -- 172.21.15.39:0/3343815859 wait complete. 2023-07-21T18:40:13.419 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2023-07-21T18:40:13.425 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:12 smithi039 ceph-mon[96489]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:40:13.425 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:12 smithi039 ceph-mon[96489]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:40:13.426 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:12 smithi039 ceph-mon[96489]: mgrmap e17: x(active, since 6m), standbys: z, y 2023-07-21T18:40:13.426 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:12 smithi039 ceph-mon[96489]: mds.? [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] up:active 2023-07-21T18:40:13.426 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:12 smithi039 ceph-mon[96489]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2023-07-21T18:40:13.426 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:12 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:40:13.426 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:12 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/1917891167' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:40:13.426 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:12 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/4202342743' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T18:40:13.427 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":252,"stamp":"2023-07-21T18:40:13.120869+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596654,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":0,"num_read":161,"num_read_kb":141,"num_write":332,"num_write_kb":2988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":47,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271,"ondisk_log_size":271,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335332,"kb_used_data":11908,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124606492,"statfs":{"total":1151940427776,"available":1151597047808,"internally_reserved":0,"allocated":12193792,"data_stored":6711760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6286,"num_objects":72,"num_object_clones":0,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":25,"num_read_kb":22,"num_write":151,"num_write_kb":50,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":47,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.346650"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.773443+0000","last_change":"2023-07-21T18:40:04.792977+0000","last_active":"2023-07-21T18:40:05.773443+0000","last_peered":"2023-07-21T18:40:05.773443+0000","last_clean":"2023-07-21T18:40:05.773443+0000","last_became_active":"2023-07-21T18:40:04.792667+0000","last_became_peered":"2023-07-21T18:40:04.792667+0000","last_unstale":"2023-07-21T18:40:05.773443+0000","last_undegraded":"2023-07-21T18:40:05.773443+0000","last_fullsized":"2023-07-21T18:40:05.773443+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:37:51.249650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.035763+0000","last_change":"2023-07-21T18:40:04.792936+0000","last_active":"2023-07-21T18:40:07.035763+0000","last_peered":"2023-07-21T18:40:07.035763+0000","last_clean":"2023-07-21T18:40:07.035763+0000","last_became_active":"2023-07-21T18:40:04.792511+0000","last_became_peered":"2023-07-21T18:40:04.792511+0000","last_unstale":"2023-07-21T18:40:07.035763+0000","last_undegraded":"2023-07-21T18:40:07.035763+0000","last_fullsized":"2023-07-21T18:40:07.035763+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:05:00.595279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.035567+0000","last_change":"2023-07-21T18:40:04.796401+0000","last_active":"2023-07-21T18:40:07.035567+0000","last_peered":"2023-07-21T18:40:07.035567+0000","last_clean":"2023-07-21T18:40:07.035567+0000","last_became_active":"2023-07-21T18:40:04.796063+0000","last_became_peered":"2023-07-21T18:40:04.796063+0000","last_unstale":"2023-07-21T18:40:07.035567+0000","last_undegraded":"2023-07-21T18:40:07.035567+0000","last_fullsized":"2023-07-21T18:40:07.035567+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:11:42.127453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.035756+0000","last_change":"2023-07-21T18:40:04.792308+0000","last_active":"2023-07-21T18:40:07.035756+0000","last_peered":"2023-07-21T18:40:07.035756+0000","last_clean":"2023-07-21T18:40:07.035756+0000","last_became_active":"2023-07-21T18:40:04.791831+0000","last_became_peered":"2023-07-21T18:40:04.791831+0000","last_unstale":"2023-07-21T18:40:07.035756+0000","last_undegraded":"2023-07-21T18:40:07.035756+0000","last_fullsized":"2023-07-21T18:40:07.035756+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:31:30.077525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.006829+0000","last_change":"2023-07-21T18:40:04.794035+0000","last_active":"2023-07-21T18:40:06.006829+0000","last_peered":"2023-07-21T18:40:06.006829+0000","last_clean":"2023-07-21T18:40:06.006829+0000","last_became_active":"2023-07-21T18:40:04.793739+0000","last_became_peered":"2023-07-21T18:40:04.793739+0000","last_unstale":"2023-07-21T18:40:06.006829+0000","last_undegraded":"2023-07-21T18:40:06.006829+0000","last_fullsized":"2023-07-21T18:40:06.006829+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:40:42.480359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.035838+0000","last_change":"2023-07-21T18:40:04.792751+0000","last_active":"2023-07-21T18:40:07.035838+0000","last_peered":"2023-07-21T18:40:07.035838+0000","last_clean":"2023-07-21T18:40:07.035838+0000","last_became_active":"2023-07-21T18:40:04.792413+0000","last_became_peered":"2023-07-21T18:40:04.792413+0000","last_unstale":"2023-07-21T18:40:07.035838+0000","last_undegraded":"2023-07-21T18:40:07.035838+0000","last_fullsized":"2023-07-21T18:40:07.035838+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:14:40.371518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.089901+0000","last_change":"2023-07-21T18:40:04.787544+0000","last_active":"2023-07-21T18:40:06.089901+0000","last_peered":"2023-07-21T18:40:06.089901+0000","last_clean":"2023-07-21T18:40:06.089901+0000","last_became_active":"2023-07-21T18:40:04.787208+0000","last_became_peered":"2023-07-21T18:40:04.787208+0000","last_unstale":"2023-07-21T18:40:06.089901+0000","last_undegraded":"2023-07-21T18:40:06.089901+0000","last_fullsized":"2023-07-21T18:40:06.089901+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:40:52.809501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.035809+0000","last_change":"2023-07-21T18:40:04.787964+0000","last_active":"2023-07-21T18:40:07.035809+0000","last_peered":"2023-07-21T18:40:07.035809+0000","last_clean":"2023-07-21T18:40:07.035809+0000","last_became_active":"2023-07-21T18:40:04.787518+0000","last_became_peered":"2023-07-21T18:40:04.787518+0000","last_unstale":"2023-07-21T18:40:07.035809+0000","last_undegraded":"2023-07-21T18:40:07.035809+0000","last_fullsized":"2023-07-21T18:40:07.035809+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:40:44.640274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.777237+0000","last_change":"2023-07-21T18:40:04.782495+0000","last_active":"2023-07-21T18:40:05.777237+0000","last_peered":"2023-07-21T18:40:05.777237+0000","last_clean":"2023-07-21T18:40:05.777237+0000","last_became_active":"2023-07-21T18:40:04.782208+0000","last_became_peered":"2023-07-21T18:40:04.782208+0000","last_unstale":"2023-07-21T18:40:05.777237+0000","last_undegraded":"2023-07-21T18:40:05.777237+0000","last_fullsized":"2023-07-21T18:40:05.777237+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:42:34.649015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.036247+0000","last_change":"2023-07-21T18:40:04.790957+0000","last_active":"2023-07-21T18:40:07.036247+0000","last_peered":"2023-07-21T18:40:07.036247+0000","last_clean":"2023-07-21T18:40:07.036247+0000","last_became_active":"2023-07-21T18:40:04.790659+0000","last_became_peered":"2023-07-21T18:40:04.790659+0000","last_unstale":"2023-07-21T18:40:07.036247+0000","last_undegraded":"2023-07-21T18:40:07.036247+0000","last_fullsized":"2023-07-21T18:40:07.036247+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:06:45.518722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.090285+0000","last_change":"2023-07-21T18:40:04.789415+0000","last_active":"2023-07-21T18:40:06.090285+0000","last_peered":"2023-07-21T18:40:06.090285+0000","last_clean":"2023-07-21T18:40:06.090285+0000","last_became_active":"2023-07-21T18:40:04.789088+0000","last_became_peered":"2023-07-21T18:40:04.789088+0000","last_unstale":"2023-07-21T18:40:06.090285+0000","last_undegraded":"2023-07-21T18:40:06.090285+0000","last_fullsized":"2023-07-21T18:40:06.090285+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:10:43.584203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.035583+0000","last_change":"2023-07-21T18:40:04.789101+0000","last_active":"2023-07-21T18:40:07.035583+0000","last_peered":"2023-07-21T18:40:07.035583+0000","last_clean":"2023-07-21T18:40:07.035583+0000","last_became_active":"2023-07-21T18:40:04.788704+0000","last_became_peered":"2023-07-21T18:40:04.788704+0000","last_unstale":"2023-07-21T18:40:07.035583+0000","last_undegraded":"2023-07-21T18:40:07.035583+0000","last_fullsized":"2023-07-21T18:40:07.035583+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:10:15.256725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.772732+0000","last_change":"2023-07-21T18:40:04.784863+0000","last_active":"2023-07-21T18:40:05.772732+0000","last_peered":"2023-07-21T18:40:05.772732+0000","last_clean":"2023-07-21T18:40:05.772732+0000","last_became_active":"2023-07-21T18:40:04.784474+0000","last_became_peered":"2023-07-21T18:40:04.784474+0000","last_unstale":"2023-07-21T18:40:05.772732+0000","last_undegraded":"2023-07-21T18:40:05.772732+0000","last_fullsized":"2023-07-21T18:40:05.772732+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:53:41.983453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.036146+0000","last_change":"2023-07-21T18:40:04.797678+0000","last_active":"2023-07-21T18:40:07.036146+0000","last_peered":"2023-07-21T18:40:07.036146+0000","last_clean":"2023-07-21T18:40:07.036146+0000","last_became_active":"2023-07-21T18:40:04.797318+0000","last_became_peered":"2023-07-21T18:40:04.797318+0000","last_unstale":"2023-07-21T18:40:07.036146+0000","last_undegraded":"2023-07-21T18:40:07.036146+0000","last_fullsized":"2023-07-21T18:40:07.036146+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:44:17.374089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.035540+0000","last_change":"2023-07-21T18:40:04.793170+0000","last_active":"2023-07-21T18:40:07.035540+0000","last_peered":"2023-07-21T18:40:07.035540+0000","last_clean":"2023-07-21T18:40:07.035540+0000","last_became_active":"2023-07-21T18:40:04.792563+0000","last_became_peered":"2023-07-21T18:40:04.792563+0000","last_unstale":"2023-07-21T18:40:07.035540+0000","last_undegraded":"2023-07-21T18:40:07.035540+0000","last_fullsized":"2023-07-21T18:40:07.035540+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:36:53.352643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.776683+0000","last_change":"2023-07-21T18:40:04.791972+0000","last_active":"2023-07-21T18:40:05.776683+0000","last_peered":"2023-07-21T18:40:05.776683+0000","last_clean":"2023-07-21T18:40:05.776683+0000","last_became_active":"2023-07-21T18:40:04.791639+0000","last_became_peered":"2023-07-21T18:40:04.791639+0000","last_unstale":"2023-07-21T18:40:05.776683+0000","last_undegraded":"2023-07-21T18:40:05.776683+0000","last_fullsized":"2023-07-21T18:40:05.776683+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:35:06.204290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.773432+0000","last_change":"2023-07-21T18:40:04.789362+0000","last_active":"2023-07-21T18:40:05.773432+0000","last_peered":"2023-07-21T18:40:05.773432+0000","last_clean":"2023-07-21T18:40:05.773432+0000","last_became_active":"2023-07-21T18:40:04.788978+0000","last_became_peered":"2023-07-21T18:40:04.788978+0000","last_unstale":"2023-07-21T18:40:05.773432+0000","last_undegraded":"2023-07-21T18:40:05.773432+0000","last_fullsized":"2023-07-21T18:40:05.773432+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:55:04.759568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.036113+0000","last_change":"2023-07-21T18:40:04.788016+0000","last_active":"2023-07-21T18:40:07.036113+0000","last_peered":"2023-07-21T18:40:07.036113+0000","last_clean":"2023-07-21T18:40:07.036113+0000","last_became_active":"2023-07-21T18:40:04.787690+0000","last_became_peered":"2023-07-21T18:40:04.787690+0000","last_unstale":"2023-07-21T18:40:07.036113+0000","last_undegraded":"2023-07-21T18:40:07.036113+0000","last_fullsized":"2023-07-21T18:40:07.036113+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:55:35.585172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.592595+0000","last_change":"2023-07-21T18:40:04.793075+0000","last_active":"2023-07-21T18:40:06.592595+0000","last_peered":"2023-07-21T18:40:06.592595+0000","last_clean":"2023-07-21T18:40:06.592595+0000","last_became_active":"2023-07-21T18:40:04.792773+0000","last_became_peered":"2023-07-21T18:40:04.792773+0000","last_unstale":"2023-07-21T18:40:06.592595+0000","last_undegraded":"2023-07-21T18:40:06.592595+0000","last_fullsized":"2023-07-21T18:40:06.592595+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:48:23.084376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"79'13","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:10.974115+0000","last_change":"2023-07-21T18:40:04.178197+0000","last_active":"2023-07-21T18:40:10.974115+0000","last_peered":"2023-07-21T18:40:10.974115+0000","last_clean":"2023-07-21T18:40:10.974115+0000","last_became_active":"2023-07-21T18:40:04.177914+0000","last_became_peered":"2023-07-21T18:40:04.177914+0000","last_unstale":"2023-07-21T18:40:10.974115+0000","last_undegraded":"2023-07-21T18:40:10.974115+0000","last_fullsized":"2023-07-21T18:40:10.974115+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:33:10.544496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.037188+0000","last_change":"2023-07-21T18:40:04.791610+0000","last_active":"2023-07-21T18:40:07.037188+0000","last_peered":"2023-07-21T18:40:07.037188+0000","last_clean":"2023-07-21T18:40:07.037188+0000","last_became_active":"2023-07-21T18:40:04.791242+0000","last_became_peered":"2023-07-21T18:40:04.791242+0000","last_unstale":"2023-07-21T18:40:07.037188+0000","last_undegraded":"2023-07-21T18:40:07.037188+0000","last_fullsized":"2023-07-21T18:40:07.037188+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:02:43.400899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"78'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.037020+0000","last_change":"2023-07-21T18:40:04.182120+0000","last_active":"2023-07-21T18:40:07.037020+0000","last_peered":"2023-07-21T18:40:07.037020+0000","last_clean":"2023-07-21T18:40:07.037020+0000","last_became_active":"2023-07-21T18:40:04.181797+0000","last_became_peered":"2023-07-21T18:40:04.181797+0000","last_unstale":"2023-07-21T18:40:07.037020+0000","last_undegraded":"2023-07-21T18:40:07.037020+0000","last_fullsized":"2023-07-21T18:40:07.037020+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:02:06.127933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.036297+0000","last_change":"2023-07-21T18:40:04.796649+0000","last_active":"2023-07-21T18:40:07.036297+0000","last_peered":"2023-07-21T18:40:07.036297+0000","last_clean":"2023-07-21T18:40:07.036297+0000","last_became_active":"2023-07-21T18:40:04.795610+0000","last_became_peered":"2023-07-21T18:40:04.795610+0000","last_unstale":"2023-07-21T18:40:07.036297+0000","last_undegraded":"2023-07-21T18:40:07.036297+0000","last_fullsized":"2023-07-21T18:40:07.036297+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:11:11.969964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"79'11","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:10.975382+0000","last_change":"2023-07-21T18:40:04.178384+0000","last_active":"2023-07-21T18:40:10.975382+0000","last_peered":"2023-07-21T18:40:10.975382+0000","last_clean":"2023-07-21T18:40:10.975382+0000","last_became_active":"2023-07-21T18:40:04.178074+0000","last_became_peered":"2023-07-21T18:40:04.178074+0000","last_unstale":"2023-07-21T18:40:10.975382+0000","last_undegraded":"2023-07-21T18:40:10.975382+0000","last_fullsized":"2023-07-21T18:40:10.975382+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:21:02.336313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.776990+0000","last_change":"2023-07-21T18:40:04.785431+0000","last_active":"2023-07-21T18:40:05.776990+0000","last_peered":"2023-07-21T18:40:05.776990+0000","last_clean":"2023-07-21T18:40:05.776990+0000","last_became_active":"2023-07-21T18:40:04.785124+0000","last_became_peered":"2023-07-21T18:40:04.785124+0000","last_unstale":"2023-07-21T18:40:05.776990+0000","last_undegraded":"2023-07-21T18:40:05.776990+0000","last_fullsized":"2023-07-21T18:40:05.776990+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:19:44.604006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"79'5","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:10.975728+0000","last_change":"2023-07-21T18:40:04.176349+0000","last_active":"2023-07-21T18:40:10.975728+0000","last_peered":"2023-07-21T18:40:10.975728+0000","last_clean":"2023-07-21T18:40:10.975728+0000","last_became_active":"2023-07-21T18:40:04.175997+0000","last_became_peered":"2023-07-21T18:40:04.175997+0000","last_unstale":"2023-07-21T18:40:10.975728+0000","last_undegraded":"2023-07-21T18:40:10.975728+0000","last_fullsized":"2023-07-21T18:40:10.975728+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:13:57.068743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.995402+0000","last_change":"2023-07-21T18:40:04.784867+0000","last_active":"2023-07-21T18:40:06.995402+0000","last_peered":"2023-07-21T18:40:06.995402+0000","last_clean":"2023-07-21T18:40:06.995402+0000","last_became_active":"2023-07-21T18:40:04.784555+0000","last_became_peered":"2023-07-21T18:40:04.784555+0000","last_unstale":"2023-07-21T18:40:06.995402+0000","last_undegraded":"2023-07-21T18:40:06.995402+0000","last_fullsized":"2023-07-21T18:40:06.995402+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:57:13.354337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"79'5","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:10.964523+0000","last_change":"2023-07-21T18:40:04.177638+0000","last_active":"2023-07-21T18:40:10.964523+0000","last_peered":"2023-07-21T18:40:10.964523+0000","last_clean":"2023-07-21T18:40:10.964523+0000","last_became_active":"2023-07-21T18:40:04.177284+0000","last_became_peered":"2023-07-21T18:40:04.177284+0000","last_unstale":"2023-07-21T18:40:10.964523+0000","last_undegraded":"2023-07-21T18:40:10.964523+0000","last_fullsized":"2023-07-21T18:40:10.964523+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:21:19.903720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.006906+0000","last_change":"2023-07-21T18:40:04.791536+0000","last_active":"2023-07-21T18:40:06.006906+0000","last_peered":"2023-07-21T18:40:06.006906+0000","last_clean":"2023-07-21T18:40:06.006906+0000","last_became_active":"2023-07-21T18:40:04.791246+0000","last_became_peered":"2023-07-21T18:40:04.791246+0000","last_unstale":"2023-07-21T18:40:06.006906+0000","last_undegraded":"2023-07-21T18:40:06.006906+0000","last_fullsized":"2023-07-21T18:40:06.006906+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:44:08.906635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"78'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.035747+0000","last_change":"2023-07-21T18:40:04.179054+0000","last_active":"2023-07-21T18:40:07.035747+0000","last_peered":"2023-07-21T18:40:07.035747+0000","last_clean":"2023-07-21T18:40:07.035747+0000","last_became_active":"2023-07-21T18:40:04.178713+0000","last_became_peered":"2023-07-21T18:40:04.178713+0000","last_unstale":"2023-07-21T18:40:07.035747+0000","last_undegraded":"2023-07-21T18:40:07.035747+0000","last_fullsized":"2023-07-21T18:40:07.035747+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T18:45:12.898734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.995525+0000","last_change":"2023-07-21T18:40:04.793947+0000","last_active":"2023-07-21T18:40:06.995525+0000","last_peered":"2023-07-21T18:40:06.995525+0000","last_clean":"2023-07-21T18:40:06.995525+0000","last_became_active":"2023-07-21T18:40:04.793631+0000","last_became_peered":"2023-07-21T18:40:04.793631+0000","last_unstale":"2023-07-21T18:40:06.995525+0000","last_undegraded":"2023-07-21T18:40:06.995525+0000","last_fullsized":"2023-07-21T18:40:06.995525+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:38:22.934266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.772418+0000","last_change":"2023-07-21T18:40:04.182549+0000","last_active":"2023-07-21T18:40:05.772418+0000","last_peered":"2023-07-21T18:40:05.772418+0000","last_clean":"2023-07-21T18:40:05.772418+0000","last_became_active":"2023-07-21T18:40:04.182233+0000","last_became_peered":"2023-07-21T18:40:04.182233+0000","last_unstale":"2023-07-21T18:40:05.772418+0000","last_undegraded":"2023-07-21T18:40:05.772418+0000","last_fullsized":"2023-07-21T18:40:05.772418+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:47:27.131280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.036098+0000","last_change":"2023-07-21T18:40:04.789987+0000","last_active":"2023-07-21T18:40:07.036098+0000","last_peered":"2023-07-21T18:40:07.036098+0000","last_clean":"2023-07-21T18:40:07.036098+0000","last_became_active":"2023-07-21T18:40:04.789647+0000","last_became_peered":"2023-07-21T18:40:04.789647+0000","last_unstale":"2023-07-21T18:40:07.036098+0000","last_undegraded":"2023-07-21T18:40:07.036098+0000","last_fullsized":"2023-07-21T18:40:07.036098+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:38:39.403755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'3","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:10.965460+0000","last_change":"2023-07-21T18:40:04.177342+0000","last_active":"2023-07-21T18:40:10.965460+0000","last_peered":"2023-07-21T18:40:10.965460+0000","last_clean":"2023-07-21T18:40:10.965460+0000","last_became_active":"2023-07-21T18:40:04.176983+0000","last_became_peered":"2023-07-21T18:40:04.176983+0000","last_unstale":"2023-07-21T18:40:10.965460+0000","last_undegraded":"2023-07-21T18:40:10.965460+0000","last_fullsized":"2023-07-21T18:40:10.965460+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:51:36.023901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.035932+0000","last_change":"2023-07-21T18:40:04.794907+0000","last_active":"2023-07-21T18:40:07.035932+0000","last_peered":"2023-07-21T18:40:07.035932+0000","last_clean":"2023-07-21T18:40:07.035932+0000","last_became_active":"2023-07-21T18:40:04.794545+0000","last_became_peered":"2023-07-21T18:40:04.794545+0000","last_unstale":"2023-07-21T18:40:07.035932+0000","last_undegraded":"2023-07-21T18:40:07.035932+0000","last_fullsized":"2023-07-21T18:40:07.035932+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:47:34.101102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"79'3","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:08.216736+0000","last_change":"2023-07-21T18:40:04.179198+0000","last_active":"2023-07-21T18:40:08.216736+0000","last_peered":"2023-07-21T18:40:08.216736+0000","last_clean":"2023-07-21T18:40:08.216736+0000","last_became_active":"2023-07-21T18:40:04.178841+0000","last_became_peered":"2023-07-21T18:40:04.178841+0000","last_unstale":"2023-07-21T18:40:08.216736+0000","last_undegraded":"2023-07-21T18:40:08.216736+0000","last_fullsized":"2023-07-21T18:40:08.216736+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:12:24.650360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.777142+0000","last_change":"2023-07-21T18:40:04.786229+0000","last_active":"2023-07-21T18:40:05.777142+0000","last_peered":"2023-07-21T18:40:05.777142+0000","last_clean":"2023-07-21T18:40:05.777142+0000","last_became_active":"2023-07-21T18:40:04.785919+0000","last_became_peered":"2023-07-21T18:40:04.785919+0000","last_unstale":"2023-07-21T18:40:05.777142+0000","last_undegraded":"2023-07-21T18:40:05.777142+0000","last_fullsized":"2023-07-21T18:40:05.777142+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:19:04.949730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:08.211879+0000","last_change":"2023-07-21T18:40:04.177164+0000","last_active":"2023-07-21T18:40:08.211879+0000","last_peered":"2023-07-21T18:40:08.211879+0000","last_clean":"2023-07-21T18:40:08.211879+0000","last_became_active":"2023-07-21T18:40:04.176617+0000","last_became_peered":"2023-07-21T18:40:04.176617+0000","last_unstale":"2023-07-21T18:40:08.211879+0000","last_undegraded":"2023-07-21T18:40:08.211879+0000","last_fullsized":"2023-07-21T18:40:08.211879+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:38:06.655399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.036335+0000","last_change":"2023-07-21T18:40:04.787863+0000","last_active":"2023-07-21T18:40:07.036335+0000","last_peered":"2023-07-21T18:40:07.036335+0000","last_clean":"2023-07-21T18:40:07.036335+0000","last_became_active":"2023-07-21T18:40:04.787562+0000","last_became_peered":"2023-07-21T18:40:04.787562+0000","last_unstale":"2023-07-21T18:40:07.036335+0000","last_undegraded":"2023-07-21T18:40:07.036335+0000","last_fullsized":"2023-07-21T18:40:07.036335+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:30:30.392804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'5","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:10.244099+0000","last_change":"2023-07-21T18:40:04.179921+0000","last_active":"2023-07-21T18:40:10.244099+0000","last_peered":"2023-07-21T18:40:10.244099+0000","last_clean":"2023-07-21T18:40:10.244099+0000","last_became_active":"2023-07-21T18:40:04.179670+0000","last_became_peered":"2023-07-21T18:40:04.179670+0000","last_unstale":"2023-07-21T18:40:10.244099+0000","last_undegraded":"2023-07-21T18:40:10.244099+0000","last_fullsized":"2023-07-21T18:40:10.244099+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:11:15.619102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.777008+0000","last_change":"2023-07-21T18:40:04.792930+0000","last_active":"2023-07-21T18:40:05.777008+0000","last_peered":"2023-07-21T18:40:05.777008+0000","last_clean":"2023-07-21T18:40:05.777008+0000","last_became_active":"2023-07-21T18:40:04.792654+0000","last_became_peered":"2023-07-21T18:40:04.792654+0000","last_unstale":"2023-07-21T18:40:05.777008+0000","last_undegraded":"2023-07-21T18:40:05.777008+0000","last_fullsized":"2023-07-21T18:40:05.777008+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:50:21.387154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"79'8","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:08.211768+0000","last_change":"2023-07-21T18:40:04.178820+0000","last_active":"2023-07-21T18:40:08.211768+0000","last_peered":"2023-07-21T18:40:08.211768+0000","last_clean":"2023-07-21T18:40:08.211768+0000","last_became_active":"2023-07-21T18:40:04.178502+0000","last_became_peered":"2023-07-21T18:40:04.178502+0000","last_unstale":"2023-07-21T18:40:08.211768+0000","last_undegraded":"2023-07-21T18:40:08.211768+0000","last_fullsized":"2023-07-21T18:40:08.211768+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:48:50.193456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.006957+0000","last_change":"2023-07-21T18:40:04.792921+0000","last_active":"2023-07-21T18:40:06.006957+0000","last_peered":"2023-07-21T18:40:06.006957+0000","last_clean":"2023-07-21T18:40:06.006957+0000","last_became_active":"2023-07-21T18:40:04.792608+0000","last_became_peered":"2023-07-21T18:40:04.792608+0000","last_unstale":"2023-07-21T18:40:06.006957+0000","last_undegraded":"2023-07-21T18:40:06.006957+0000","last_fullsized":"2023-07-21T18:40:06.006957+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:04:23.180977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'4","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:08.211852+0000","last_change":"2023-07-21T18:40:04.184917+0000","last_active":"2023-07-21T18:40:08.211852+0000","last_peered":"2023-07-21T18:40:08.211852+0000","last_clean":"2023-07-21T18:40:08.211852+0000","last_became_active":"2023-07-21T18:40:04.184612+0000","last_became_peered":"2023-07-21T18:40:04.184612+0000","last_unstale":"2023-07-21T18:40:08.211852+0000","last_undegraded":"2023-07-21T18:40:08.211852+0000","last_fullsized":"2023-07-21T18:40:08.211852+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:54:45.871657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.772576+0000","last_change":"2023-07-21T18:40:04.794134+0000","last_active":"2023-07-21T18:40:05.772576+0000","last_peered":"2023-07-21T18:40:05.772576+0000","last_clean":"2023-07-21T18:40:05.772576+0000","last_became_active":"2023-07-21T18:40:04.793784+0000","last_became_peered":"2023-07-21T18:40:04.793784+0000","last_unstale":"2023-07-21T18:40:05.772576+0000","last_undegraded":"2023-07-21T18:40:05.772576+0000","last_fullsized":"2023-07-21T18:40:05.772576+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:24:12.380794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'5","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:10.241421+0000","last_change":"2023-07-21T18:40:04.184889+0000","last_active":"2023-07-21T18:40:10.241421+0000","last_peered":"2023-07-21T18:40:10.241421+0000","last_clean":"2023-07-21T18:40:10.241421+0000","last_became_active":"2023-07-21T18:40:04.184516+0000","last_became_peered":"2023-07-21T18:40:04.184516+0000","last_unstale":"2023-07-21T18:40:10.241421+0000","last_undegraded":"2023-07-21T18:40:10.241421+0000","last_fullsized":"2023-07-21T18:40:10.241421+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:56:48.211364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.090064+0000","last_change":"2023-07-21T18:40:04.790627+0000","last_active":"2023-07-21T18:40:06.090064+0000","last_peered":"2023-07-21T18:40:06.090064+0000","last_clean":"2023-07-21T18:40:06.090064+0000","last_became_active":"2023-07-21T18:40:04.790298+0000","last_became_peered":"2023-07-21T18:40:04.790298+0000","last_unstale":"2023-07-21T18:40:06.090064+0000","last_undegraded":"2023-07-21T18:40:06.090064+0000","last_fullsized":"2023-07-21T18:40:06.090064+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:23:15.541330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"79'6","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:10.967434+0000","last_change":"2023-07-21T18:40:04.176156+0000","last_active":"2023-07-21T18:40:10.967434+0000","last_peered":"2023-07-21T18:40:10.967434+0000","last_clean":"2023-07-21T18:40:10.967434+0000","last_became_active":"2023-07-21T18:40:04.175823+0000","last_became_peered":"2023-07-21T18:40:04.175823+0000","last_unstale":"2023-07-21T18:40:10.967434+0000","last_undegraded":"2023-07-21T18:40:10.967434+0000","last_fullsized":"2023-07-21T18:40:10.967434+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:43:06.713614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":930,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.089814+0000","last_change":"2023-07-21T18:40:04.790422+0000","last_active":"2023-07-21T18:40:06.089814+0000","last_peered":"2023-07-21T18:40:06.089814+0000","last_clean":"2023-07-21T18:40:06.089814+0000","last_became_active":"2023-07-21T18:40:04.790124+0000","last_became_peered":"2023-07-21T18:40:04.790124+0000","last_unstale":"2023-07-21T18:40:06.089814+0000","last_undegraded":"2023-07-21T18:40:06.089814+0000","last_fullsized":"2023-07-21T18:40:06.089814+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:50:38.399075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'17","reported_seq":32,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:10.962031+0000","last_change":"2023-07-21T18:40:04.177006+0000","last_active":"2023-07-21T18:40:10.962031+0000","last_peered":"2023-07-21T18:40:10.962031+0000","last_clean":"2023-07-21T18:40:10.962031+0000","last_became_active":"2023-07-21T18:40:04.176711+0000","last_became_peered":"2023-07-21T18:40:04.176711+0000","last_unstale":"2023-07-21T18:40:10.962031+0000","last_undegraded":"2023-07-21T18:40:10.962031+0000","last_fullsized":"2023-07-21T18:40:10.962031+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:51:47.735726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.995106+0000","last_change":"2023-07-21T18:40:04.785601+0000","last_active":"2023-07-21T18:40:06.995106+0000","last_peered":"2023-07-21T18:40:06.995106+0000","last_clean":"2023-07-21T18:40:06.995106+0000","last_became_active":"2023-07-21T18:40:04.785307+0000","last_became_peered":"2023-07-21T18:40:04.785307+0000","last_unstale":"2023-07-21T18:40:06.995106+0000","last_undegraded":"2023-07-21T18:40:06.995106+0000","last_fullsized":"2023-07-21T18:40:06.995106+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:17:07.221060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'1","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.053227+0000","last_change":"2023-07-21T18:40:04.171005+0000","last_active":"2023-07-21T18:40:06.053227+0000","last_peered":"2023-07-21T18:40:06.053227+0000","last_clean":"2023-07-21T18:40:06.053227+0000","last_became_active":"2023-07-21T18:40:04.170719+0000","last_became_peered":"2023-07-21T18:40:04.170719+0000","last_unstale":"2023-07-21T18:40:06.053227+0000","last_undegraded":"2023-07-21T18:40:06.053227+0000","last_fullsized":"2023-07-21T18:40:06.053227+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:06:46.353789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.773304+0000","last_change":"2023-07-21T18:40:04.783596+0000","last_active":"2023-07-21T18:40:05.773304+0000","last_peered":"2023-07-21T18:40:05.773304+0000","last_clean":"2023-07-21T18:40:05.773304+0000","last_became_active":"2023-07-21T18:40:04.783293+0000","last_became_peered":"2023-07-21T18:40:04.783293+0000","last_unstale":"2023-07-21T18:40:05.773304+0000","last_undegraded":"2023-07-21T18:40:05.773304+0000","last_fullsized":"2023-07-21T18:40:05.773304+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:17:45.360634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"79'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.648456+0000","last_change":"2023-07-21T18:40:04.181069+0000","last_active":"2023-07-21T18:40:07.648456+0000","last_peered":"2023-07-21T18:40:07.648456+0000","last_clean":"2023-07-21T18:40:07.648456+0000","last_became_active":"2023-07-21T18:40:04.180750+0000","last_became_peered":"2023-07-21T18:40:04.180750+0000","last_unstale":"2023-07-21T18:40:07.648456+0000","last_undegraded":"2023-07-21T18:40:07.648456+0000","last_fullsized":"2023-07-21T18:40:07.648456+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:44:48.158024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.036165+0000","last_change":"2023-07-21T18:40:04.789940+0000","last_active":"2023-07-21T18:40:07.036165+0000","last_peered":"2023-07-21T18:40:07.036165+0000","last_clean":"2023-07-21T18:40:07.036165+0000","last_became_active":"2023-07-21T18:40:04.789465+0000","last_became_peered":"2023-07-21T18:40:04.789465+0000","last_unstale":"2023-07-21T18:40:07.036165+0000","last_undegraded":"2023-07-21T18:40:07.036165+0000","last_fullsized":"2023-07-21T18:40:07.036165+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:05:44.817727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"79'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.090012+0000","last_change":"2023-07-21T18:40:04.180522+0000","last_active":"2023-07-21T18:40:06.090012+0000","last_peered":"2023-07-21T18:40:06.090012+0000","last_clean":"2023-07-21T18:40:06.090012+0000","last_became_active":"2023-07-21T18:40:04.180196+0000","last_became_peered":"2023-07-21T18:40:04.180196+0000","last_unstale":"2023-07-21T18:40:06.090012+0000","last_undegraded":"2023-07-21T18:40:06.090012+0000","last_fullsized":"2023-07-21T18:40:06.090012+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:10:53.779418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.036035+0000","last_change":"2023-07-21T18:40:04.794781+0000","last_active":"2023-07-21T18:40:07.036035+0000","last_peered":"2023-07-21T18:40:07.036035+0000","last_clean":"2023-07-21T18:40:07.036035+0000","last_became_active":"2023-07-21T18:40:04.794388+0000","last_became_peered":"2023-07-21T18:40:04.794388+0000","last_unstale":"2023-07-21T18:40:07.036035+0000","last_undegraded":"2023-07-21T18:40:07.036035+0000","last_fullsized":"2023-07-21T18:40:07.036035+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:26:47.679253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.995411+0000","last_change":"2023-07-21T18:40:04.786408+0000","last_active":"2023-07-21T18:40:06.995411+0000","last_peered":"2023-07-21T18:40:06.995411+0000","last_clean":"2023-07-21T18:40:06.995411+0000","last_became_active":"2023-07-21T18:40:04.785555+0000","last_became_peered":"2023-07-21T18:40:04.785555+0000","last_unstale":"2023-07-21T18:40:06.995411+0000","last_undegraded":"2023-07-21T18:40:06.995411+0000","last_fullsized":"2023-07-21T18:40:06.995411+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:32:35.090936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"79'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.772454+0000","last_change":"2023-07-21T18:40:04.185980+0000","last_active":"2023-07-21T18:40:05.772454+0000","last_peered":"2023-07-21T18:40:05.772454+0000","last_clean":"2023-07-21T18:40:05.772454+0000","last_became_active":"2023-07-21T18:40:04.185618+0000","last_became_peered":"2023-07-21T18:40:04.185618+0000","last_unstale":"2023-07-21T18:40:05.772454+0000","last_undegraded":"2023-07-21T18:40:05.772454+0000","last_fullsized":"2023-07-21T18:40:05.772454+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:44:22.202995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.036366+0000","last_change":"2023-07-21T18:40:04.792535+0000","last_active":"2023-07-21T18:40:07.036366+0000","last_peered":"2023-07-21T18:40:07.036366+0000","last_clean":"2023-07-21T18:40:07.036366+0000","last_became_active":"2023-07-21T18:40:04.792239+0000","last_became_peered":"2023-07-21T18:40:04.792239+0000","last_unstale":"2023-07-21T18:40:07.036366+0000","last_undegraded":"2023-07-21T18:40:07.036366+0000","last_fullsized":"2023-07-21T18:40:07.036366+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:51:14.670073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"79'6","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:10.974758+0000","last_change":"2023-07-21T18:40:04.173539+0000","last_active":"2023-07-21T18:40:10.974758+0000","last_peered":"2023-07-21T18:40:10.974758+0000","last_clean":"2023-07-21T18:40:10.974758+0000","last_became_active":"2023-07-21T18:40:04.173225+0000","last_became_peered":"2023-07-21T18:40:04.173225+0000","last_unstale":"2023-07-21T18:40:10.974758+0000","last_undegraded":"2023-07-21T18:40:10.974758+0000","last_fullsized":"2023-07-21T18:40:10.974758+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:38:21.873200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":337,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.089887+0000","last_change":"2023-07-21T18:39:05.999983+0000","last_active":"2023-07-21T18:40:06.089887+0000","last_peered":"2023-07-21T18:40:06.089887+0000","last_clean":"2023-07-21T18:40:06.089887+0000","last_became_active":"2023-07-21T18:39:05.999495+0000","last_became_peered":"2023-07-21T18:39:05.999495+0000","last_unstale":"2023-07-21T18:40:06.089887+0000","last_undegraded":"2023-07-21T18:40:06.089887+0000","last_fullsized":"2023-07-21T18:40:06.089887+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:36:14.004429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:36:14.004429+0000","last_clean_scrub_stamp":"2023-07-21T18:36:14.004429+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:34:11.106296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.772327+0000","last_change":"2023-07-21T18:40:04.785086+0000","last_active":"2023-07-21T18:40:05.772327+0000","last_peered":"2023-07-21T18:40:05.772327+0000","last_clean":"2023-07-21T18:40:05.772327+0000","last_became_active":"2023-07-21T18:40:04.784693+0000","last_became_peered":"2023-07-21T18:40:04.784693+0000","last_unstale":"2023-07-21T18:40:05.772327+0000","last_undegraded":"2023-07-21T18:40:05.772327+0000","last_fullsized":"2023-07-21T18:40:05.772327+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:53:45.159424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"79'3","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:09.230670+0000","last_change":"2023-07-21T18:40:04.177032+0000","last_active":"2023-07-21T18:40:09.230670+0000","last_peered":"2023-07-21T18:40:09.230670+0000","last_clean":"2023-07-21T18:40:09.230670+0000","last_became_active":"2023-07-21T18:40:04.176741+0000","last_became_peered":"2023-07-21T18:40:04.176741+0000","last_unstale":"2023-07-21T18:40:09.230670+0000","last_undegraded":"2023-07-21T18:40:09.230670+0000","last_fullsized":"2023-07-21T18:40:09.230670+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:17:23.675128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"79'4","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:10.962237+0000","last_change":"2023-07-21T18:40:04.180106+0000","last_active":"2023-07-21T18:40:10.962237+0000","last_peered":"2023-07-21T18:40:10.962237+0000","last_clean":"2023-07-21T18:40:10.962237+0000","last_became_active":"2023-07-21T18:40:04.179802+0000","last_became_peered":"2023-07-21T18:40:04.179802+0000","last_unstale":"2023-07-21T18:40:10.962237+0000","last_undegraded":"2023-07-21T18:40:10.962237+0000","last_fullsized":"2023-07-21T18:40:10.962237+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:38:16.465170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.772506+0000","last_change":"2023-07-21T18:40:04.789127+0000","last_active":"2023-07-21T18:40:05.772506+0000","last_peered":"2023-07-21T18:40:05.772506+0000","last_clean":"2023-07-21T18:40:05.772506+0000","last_became_active":"2023-07-21T18:40:04.788815+0000","last_became_peered":"2023-07-21T18:40:04.788815+0000","last_unstale":"2023-07-21T18:40:05.772506+0000","last_undegraded":"2023-07-21T18:40:05.772506+0000","last_fullsized":"2023-07-21T18:40:05.772506+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:14:00.509523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"79'7","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:10.244485+0000","last_change":"2023-07-21T18:40:04.184786+0000","last_active":"2023-07-21T18:40:10.244485+0000","last_peered":"2023-07-21T18:40:10.244485+0000","last_clean":"2023-07-21T18:40:10.244485+0000","last_became_active":"2023-07-21T18:40:04.184433+0000","last_became_peered":"2023-07-21T18:40:04.184433+0000","last_unstale":"2023-07-21T18:40:10.244485+0000","last_undegraded":"2023-07-21T18:40:10.244485+0000","last_fullsized":"2023-07-21T18:40:10.244485+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:39:47.800619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.037220+0000","last_change":"2023-07-21T18:40:04.791937+0000","last_active":"2023-07-21T18:40:07.037220+0000","last_peered":"2023-07-21T18:40:07.037220+0000","last_clean":"2023-07-21T18:40:07.037220+0000","last_became_active":"2023-07-21T18:40:04.791588+0000","last_became_peered":"2023-07-21T18:40:04.791588+0000","last_unstale":"2023-07-21T18:40:07.037220+0000","last_undegraded":"2023-07-21T18:40:07.037220+0000","last_fullsized":"2023-07-21T18:40:07.037220+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:33:01.443024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'7","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:10.973433+0000","last_change":"2023-07-21T18:40:04.177007+0000","last_active":"2023-07-21T18:40:10.973433+0000","last_peered":"2023-07-21T18:40:10.973433+0000","last_clean":"2023-07-21T18:40:10.973433+0000","last_became_active":"2023-07-21T18:40:04.176699+0000","last_became_peered":"2023-07-21T18:40:04.176699+0000","last_unstale":"2023-07-21T18:40:10.973433+0000","last_undegraded":"2023-07-21T18:40:10.973433+0000","last_fullsized":"2023-07-21T18:40:10.973433+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:21:08.441400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.035909+0000","last_change":"2023-07-21T18:40:04.795671+0000","last_active":"2023-07-21T18:40:07.035909+0000","last_peered":"2023-07-21T18:40:07.035909+0000","last_clean":"2023-07-21T18:40:07.035909+0000","last_became_active":"2023-07-21T18:40:04.795233+0000","last_became_peered":"2023-07-21T18:40:04.795233+0000","last_unstale":"2023-07-21T18:40:07.035909+0000","last_undegraded":"2023-07-21T18:40:07.035909+0000","last_fullsized":"2023-07-21T18:40:07.035909+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:26:32.284218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:08.212881+0000","last_change":"2023-07-21T18:40:04.176859+0000","last_active":"2023-07-21T18:40:08.212881+0000","last_peered":"2023-07-21T18:40:08.212881+0000","last_clean":"2023-07-21T18:40:08.212881+0000","last_became_active":"2023-07-21T18:40:04.176559+0000","last_became_peered":"2023-07-21T18:40:04.176559+0000","last_unstale":"2023-07-21T18:40:08.212881+0000","last_undegraded":"2023-07-21T18:40:08.212881+0000","last_fullsized":"2023-07-21T18:40:08.212881+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:34:23.667073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.006770+0000","last_change":"2023-07-21T18:40:04.793091+0000","last_active":"2023-07-21T18:40:06.006770+0000","last_peered":"2023-07-21T18:40:06.006770+0000","last_clean":"2023-07-21T18:40:06.006770+0000","last_became_active":"2023-07-21T18:40:04.792723+0000","last_became_peered":"2023-07-21T18:40:04.792723+0000","last_unstale":"2023-07-21T18:40:06.006770+0000","last_undegraded":"2023-07-21T18:40:06.006770+0000","last_fullsized":"2023-07-21T18:40:06.006770+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:27:04.440384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'3","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:08.214536+0000","last_change":"2023-07-21T18:40:04.182867+0000","last_active":"2023-07-21T18:40:08.214536+0000","last_peered":"2023-07-21T18:40:08.214536+0000","last_clean":"2023-07-21T18:40:08.214536+0000","last_became_active":"2023-07-21T18:40:04.181885+0000","last_became_peered":"2023-07-21T18:40:04.181885+0000","last_unstale":"2023-07-21T18:40:08.214536+0000","last_undegraded":"2023-07-21T18:40:08.214536+0000","last_fullsized":"2023-07-21T18:40:08.214536+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:03:28.800986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.773091+0000","last_change":"2023-07-21T18:40:04.790340+0000","last_active":"2023-07-21T18:40:05.773091+0000","last_peered":"2023-07-21T18:40:05.773091+0000","last_clean":"2023-07-21T18:40:05.773091+0000","last_became_active":"2023-07-21T18:40:04.790014+0000","last_became_peered":"2023-07-21T18:40:04.790014+0000","last_unstale":"2023-07-21T18:40:05.773091+0000","last_undegraded":"2023-07-21T18:40:05.773091+0000","last_fullsized":"2023-07-21T18:40:05.773091+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:40:26.891907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.592263+0000","last_change":"2023-07-21T18:40:04.184986+0000","last_active":"2023-07-21T18:40:06.592263+0000","last_peered":"2023-07-21T18:40:06.592263+0000","last_clean":"2023-07-21T18:40:06.592263+0000","last_became_active":"2023-07-21T18:40:04.184672+0000","last_became_peered":"2023-07-21T18:40:04.184672+0000","last_unstale":"2023-07-21T18:40:06.592263+0000","last_undegraded":"2023-07-21T18:40:06.592263+0000","last_fullsized":"2023-07-21T18:40:06.592263+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:40:47.955426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.036702+0000","last_change":"2023-07-21T18:40:04.793272+0000","last_active":"2023-07-21T18:40:07.036702+0000","last_peered":"2023-07-21T18:40:07.036702+0000","last_clean":"2023-07-21T18:40:07.036702+0000","last_became_active":"2023-07-21T18:40:04.792854+0000","last_became_peered":"2023-07-21T18:40:04.792854+0000","last_unstale":"2023-07-21T18:40:07.036702+0000","last_undegraded":"2023-07-21T18:40:07.036702+0000","last_fullsized":"2023-07-21T18:40:07.036702+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:22:05.555012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:08.212137+0000","last_change":"2023-07-21T18:40:04.176404+0000","last_active":"2023-07-21T18:40:08.212137+0000","last_peered":"2023-07-21T18:40:08.212137+0000","last_clean":"2023-07-21T18:40:08.212137+0000","last_became_active":"2023-07-21T18:40:04.176148+0000","last_became_peered":"2023-07-21T18:40:04.176148+0000","last_unstale":"2023-07-21T18:40:08.212137+0000","last_undegraded":"2023-07-21T18:40:08.212137+0000","last_fullsized":"2023-07-21T18:40:08.212137+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:29:41.873295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.090128+0000","last_change":"2023-07-21T18:40:04.795628+0000","last_active":"2023-07-21T18:40:06.090128+0000","last_peered":"2023-07-21T18:40:06.090128+0000","last_clean":"2023-07-21T18:40:06.090128+0000","last_became_active":"2023-07-21T18:40:04.795248+0000","last_became_peered":"2023-07-21T18:40:04.795248+0000","last_unstale":"2023-07-21T18:40:06.090128+0000","last_undegraded":"2023-07-21T18:40:06.090128+0000","last_fullsized":"2023-07-21T18:40:06.090128+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:05:26.124641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.592468+0000","last_change":"2023-07-21T18:40:04.184380+0000","last_active":"2023-07-21T18:40:06.592468+0000","last_peered":"2023-07-21T18:40:06.592468+0000","last_clean":"2023-07-21T18:40:06.592468+0000","last_became_active":"2023-07-21T18:40:04.184045+0000","last_became_peered":"2023-07-21T18:40:04.184045+0000","last_unstale":"2023-07-21T18:40:06.592468+0000","last_undegraded":"2023-07-21T18:40:06.592468+0000","last_fullsized":"2023-07-21T18:40:06.592468+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:58:31.782143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.090407+0000","last_change":"2023-07-21T18:40:04.787760+0000","last_active":"2023-07-21T18:40:06.090407+0000","last_peered":"2023-07-21T18:40:06.090407+0000","last_clean":"2023-07-21T18:40:06.090407+0000","last_became_active":"2023-07-21T18:40:04.787371+0000","last_became_peered":"2023-07-21T18:40:04.787371+0000","last_unstale":"2023-07-21T18:40:06.090407+0000","last_undegraded":"2023-07-21T18:40:06.090407+0000","last_fullsized":"2023-07-21T18:40:06.090407+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:10:11.576057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'3","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:10.962506+0000","last_change":"2023-07-21T18:40:04.184721+0000","last_active":"2023-07-21T18:40:10.962506+0000","last_peered":"2023-07-21T18:40:10.962506+0000","last_clean":"2023-07-21T18:40:10.962506+0000","last_became_active":"2023-07-21T18:40:04.184070+0000","last_became_peered":"2023-07-21T18:40:04.184070+0000","last_unstale":"2023-07-21T18:40:10.962506+0000","last_undegraded":"2023-07-21T18:40:10.962506+0000","last_fullsized":"2023-07-21T18:40:10.962506+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:57:46.774015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.090187+0000","last_change":"2023-07-21T18:40:04.786963+0000","last_active":"2023-07-21T18:40:06.090187+0000","last_peered":"2023-07-21T18:40:06.090187+0000","last_clean":"2023-07-21T18:40:06.090187+0000","last_became_active":"2023-07-21T18:40:04.786617+0000","last_became_peered":"2023-07-21T18:40:04.786617+0000","last_unstale":"2023-07-21T18:40:06.090187+0000","last_undegraded":"2023-07-21T18:40:06.090187+0000","last_fullsized":"2023-07-21T18:40:06.090187+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:22:45.940347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.772565+0000","last_change":"2023-07-21T18:40:04.184509+0000","last_active":"2023-07-21T18:40:05.772565+0000","last_peered":"2023-07-21T18:40:05.772565+0000","last_clean":"2023-07-21T18:40:05.772565+0000","last_became_active":"2023-07-21T18:40:04.184208+0000","last_became_peered":"2023-07-21T18:40:04.184208+0000","last_unstale":"2023-07-21T18:40:05.772565+0000","last_undegraded":"2023-07-21T18:40:05.772565+0000","last_fullsized":"2023-07-21T18:40:05.772565+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:18:05.113810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.090384+0000","last_change":"2023-07-21T18:40:04.788248+0000","last_active":"2023-07-21T18:40:06.090384+0000","last_peered":"2023-07-21T18:40:06.090384+0000","last_clean":"2023-07-21T18:40:06.090384+0000","last_became_active":"2023-07-21T18:40:04.787974+0000","last_became_peered":"2023-07-21T18:40:04.787974+0000","last_unstale":"2023-07-21T18:40:06.090384+0000","last_undegraded":"2023-07-21T18:40:06.090384+0000","last_fullsized":"2023-07-21T18:40:06.090384+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:58:18.957824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.773170+0000","last_change":"2023-07-21T18:40:04.790522+0000","last_active":"2023-07-21T18:40:05.773170+0000","last_peered":"2023-07-21T18:40:05.773170+0000","last_clean":"2023-07-21T18:40:05.773170+0000","last_became_active":"2023-07-21T18:40:04.790178+0000","last_became_peered":"2023-07-21T18:40:04.790178+0000","last_unstale":"2023-07-21T18:40:05.773170+0000","last_undegraded":"2023-07-21T18:40:05.773170+0000","last_fullsized":"2023-07-21T18:40:05.773170+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:39:58.451665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.592261+0000","last_change":"2023-07-21T18:40:04.790403+0000","last_active":"2023-07-21T18:40:06.592261+0000","last_peered":"2023-07-21T18:40:06.592261+0000","last_clean":"2023-07-21T18:40:06.592261+0000","last_became_active":"2023-07-21T18:40:04.790106+0000","last_became_peered":"2023-07-21T18:40:04.790106+0000","last_unstale":"2023-07-21T18:40:06.592261+0000","last_undegraded":"2023-07-21T18:40:06.592261+0000","last_fullsized":"2023-07-21T18:40:06.592261+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:22:57.358209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.592288+0000","last_change":"2023-07-21T18:40:04.795230+0000","last_active":"2023-07-21T18:40:06.592288+0000","last_peered":"2023-07-21T18:40:06.592288+0000","last_clean":"2023-07-21T18:40:06.592288+0000","last_became_active":"2023-07-21T18:40:04.794949+0000","last_became_peered":"2023-07-21T18:40:04.794949+0000","last_unstale":"2023-07-21T18:40:06.592288+0000","last_undegraded":"2023-07-21T18:40:06.592288+0000","last_fullsized":"2023-07-21T18:40:06.592288+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:25:52.872388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.037364+0000","last_change":"2023-07-21T18:40:04.792736+0000","last_active":"2023-07-21T18:40:07.037364+0000","last_peered":"2023-07-21T18:40:07.037364+0000","last_clean":"2023-07-21T18:40:07.037364+0000","last_became_active":"2023-07-21T18:40:04.792301+0000","last_became_peered":"2023-07-21T18:40:04.792301+0000","last_unstale":"2023-07-21T18:40:07.037364+0000","last_undegraded":"2023-07-21T18:40:07.037364+0000","last_fullsized":"2023-07-21T18:40:07.037364+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:40:20.170397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.772638+0000","last_change":"2023-07-21T18:40:04.790733+0000","last_active":"2023-07-21T18:40:05.772638+0000","last_peered":"2023-07-21T18:40:05.772638+0000","last_clean":"2023-07-21T18:40:05.772638+0000","last_became_active":"2023-07-21T18:40:04.790441+0000","last_became_peered":"2023-07-21T18:40:04.790441+0000","last_unstale":"2023-07-21T18:40:05.772638+0000","last_undegraded":"2023-07-21T18:40:05.772638+0000","last_fullsized":"2023-07-21T18:40:05.772638+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:29:55.466493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.773446+0000","last_change":"2023-07-21T18:40:04.790853+0000","last_active":"2023-07-21T18:40:05.773446+0000","last_peered":"2023-07-21T18:40:05.773446+0000","last_clean":"2023-07-21T18:40:05.773446+0000","last_became_active":"2023-07-21T18:40:04.790373+0000","last_became_peered":"2023-07-21T18:40:04.790373+0000","last_unstale":"2023-07-21T18:40:05.773446+0000","last_undegraded":"2023-07-21T18:40:05.773446+0000","last_fullsized":"2023-07-21T18:40:05.773446+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:59:25.687337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.592674+0000","last_change":"2023-07-21T18:40:04.794088+0000","last_active":"2023-07-21T18:40:06.592674+0000","last_peered":"2023-07-21T18:40:06.592674+0000","last_clean":"2023-07-21T18:40:06.592674+0000","last_became_active":"2023-07-21T18:40:04.793782+0000","last_became_peered":"2023-07-21T18:40:04.793782+0000","last_unstale":"2023-07-21T18:40:06.592674+0000","last_undegraded":"2023-07-21T18:40:06.592674+0000","last_fullsized":"2023-07-21T18:40:06.592674+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:14:52.604091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.037053+0000","last_change":"2023-07-21T18:40:04.791772+0000","last_active":"2023-07-21T18:40:07.037053+0000","last_peered":"2023-07-21T18:40:07.037053+0000","last_clean":"2023-07-21T18:40:07.037053+0000","last_became_active":"2023-07-21T18:40:04.791379+0000","last_became_peered":"2023-07-21T18:40:04.791379+0000","last_unstale":"2023-07-21T18:40:07.037053+0000","last_undegraded":"2023-07-21T18:40:07.037053+0000","last_fullsized":"2023-07-21T18:40:07.037053+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:37:36.712528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.994956+0000","last_change":"2023-07-21T18:40:04.786579+0000","last_active":"2023-07-21T18:40:06.994956+0000","last_peered":"2023-07-21T18:40:06.994956+0000","last_clean":"2023-07-21T18:40:06.994956+0000","last_became_active":"2023-07-21T18:40:04.786032+0000","last_became_peered":"2023-07-21T18:40:04.786032+0000","last_unstale":"2023-07-21T18:40:06.994956+0000","last_undegraded":"2023-07-21T18:40:06.994956+0000","last_fullsized":"2023-07-21T18:40:06.994956+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:34:12.940522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.036500+0000","last_change":"2023-07-21T18:40:04.793441+0000","last_active":"2023-07-21T18:40:07.036500+0000","last_peered":"2023-07-21T18:40:07.036500+0000","last_clean":"2023-07-21T18:40:07.036500+0000","last_became_active":"2023-07-21T18:40:04.791441+0000","last_became_peered":"2023-07-21T18:40:04.791441+0000","last_unstale":"2023-07-21T18:40:07.036500+0000","last_undegraded":"2023-07-21T18:40:07.036500+0000","last_fullsized":"2023-07-21T18:40:07.036500+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:01:49.043942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.777066+0000","last_change":"2023-07-21T18:40:04.794913+0000","last_active":"2023-07-21T18:40:05.777066+0000","last_peered":"2023-07-21T18:40:05.777066+0000","last_clean":"2023-07-21T18:40:05.777066+0000","last_became_active":"2023-07-21T18:40:04.794527+0000","last_became_peered":"2023-07-21T18:40:04.794527+0000","last_unstale":"2023-07-21T18:40:05.777066+0000","last_undegraded":"2023-07-21T18:40:05.777066+0000","last_fullsized":"2023-07-21T18:40:05.777066+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:13:56.682713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.772671+0000","last_change":"2023-07-21T18:40:04.789145+0000","last_active":"2023-07-21T18:40:05.772671+0000","last_peered":"2023-07-21T18:40:05.772671+0000","last_clean":"2023-07-21T18:40:05.772671+0000","last_became_active":"2023-07-21T18:40:04.788285+0000","last_became_peered":"2023-07-21T18:40:04.788285+0000","last_unstale":"2023-07-21T18:40:05.772671+0000","last_undegraded":"2023-07-21T18:40:05.772671+0000","last_fullsized":"2023-07-21T18:40:05.772671+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:42:10.814436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.773199+0000","last_change":"2023-07-21T18:40:04.788821+0000","last_active":"2023-07-21T18:40:05.773199+0000","last_peered":"2023-07-21T18:40:05.773199+0000","last_clean":"2023-07-21T18:40:05.773199+0000","last_became_active":"2023-07-21T18:40:04.788513+0000","last_became_peered":"2023-07-21T18:40:04.788513+0000","last_unstale":"2023-07-21T18:40:05.773199+0000","last_undegraded":"2023-07-21T18:40:05.773199+0000","last_fullsized":"2023-07-21T18:40:05.773199+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:35:28.640398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6286,"num_objects":72,"num_object_clones":0,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":25,"num_read_kb":22,"num_write":151,"num_write_kb":50,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":47,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":299008,"data_stored":52303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":134,"ondisk_log_size":134,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579916,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27780,"kb_used_data":828,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717372,"statfs":{"total":95995035648,"available":95966588928,"internally_reserved":0,"allocated":847872,"data_stored":407750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]}]},{"osd":10,"up_from":67,"seq":287762808847,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28392,"kb_used_data":1440,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716760,"statfs":{"total":95995035648,"available":95965962240,"internally_reserved":0,"allocated":1474560,"data_stored":1000307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":1,"last update":"Fri Jul 21 18:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.33200000000000002}]},{"osd":2,"last update":"Fri Jul 21 18:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.621},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.57199999999999995}]},{"osd":3,"last update":"Fri Jul 21 18:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.38600000000000001}]},{"osd":4,"last update":"Fri Jul 21 18:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.371}]},{"osd":5,"last update":"Fri Jul 21 18:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.73499999999999999}]},{"osd":6,"last update":"Fri Jul 21 18:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.505},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.46999999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":8,"last update":"Fri Jul 21 18:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.70999999999999996}]},{"osd":9,"last update":"Fri Jul 21 18:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.58599999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]}]},{"osd":9,"up_from":62,"seq":266287972373,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27816,"kb_used_data":864,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717336,"statfs":{"total":95995035648,"available":95966552064,"internally_reserved":0,"allocated":884736,"data_stored":408872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:39:45 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.6319999999999999,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.40000000000000002}]},{"osd":1,"last update":"Fri Jul 21 18:39:45 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":2.1000000000000001,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.681,"5min":1.681,"15min":1.681},"last":0.53000000000000003}]},{"osd":2,"last update":"Fri Jul 21 18:39:45 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.9530000000000001,"5min":1.9530000000000001,"15min":1.9530000000000001},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.7170000000000001,"5min":1.7170000000000001,"15min":1.7170000000000001},"last":0.59699999999999998}]},{"osd":3,"last update":"Fri Jul 21 18:39:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":2.048,"5min":2.048,"15min":2.048},"last":0.51200000000000001}]},{"osd":4,"last update":"Fri Jul 21 18:39:45 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.8100000000000001,"5min":1.8100000000000001,"15min":1.8100000000000001},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.54900000000000004}]},{"osd":5,"last update":"Fri Jul 21 18:39:45 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.992,"5min":1.992,"15min":1.992},"last":0.68999999999999995}]},{"osd":6,"last update":"Fri Jul 21 18:39:45 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":2.0209999999999999,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.595,"5min":1.595,"15min":1.595},"last":0.56899999999999995}]},{"osd":7,"last update":"Fri Jul 21 18:39:45 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":2.0750000000000002,"5min":2.0750000000000002,"15min":2.0750000000000002},"last":0.63100000000000001}]},{"osd":8,"last update":"Fri Jul 21 18:39:45 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":2.2890000000000001,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.74399999999999999}]},{"osd":10,"last update":"Fri Jul 21 18:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":2.1240000000000001,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.70099999999999996}]}]},{"osd":8,"up_from":55,"seq":236223201305,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27796,"kb_used_data":844,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717356,"statfs":{"total":95995035648,"available":95966572544,"internally_reserved":0,"allocated":864256,"data_stored":408157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:39:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.38700000000000001}]},{"osd":1,"last update":"Fri Jul 21 18:39:23 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.79300000000000004}]},{"osd":2,"last update":"Fri Jul 21 18:39:23 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.434},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.33400000000000002}]},{"osd":3,"last update":"Fri Jul 21 18:39:23 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.41299999999999998}]},{"osd":4,"last update":"Fri Jul 21 18:39:23 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.72099999999999997}]},{"osd":5,"last update":"Fri Jul 21 18:39:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.46200000000000002}]},{"osd":6,"last update":"Fri Jul 21 18:39:23 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.65800000000000003}]},{"osd":7,"last update":"Fri Jul 21 18:39:23 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.64100000000000001}]},{"osd":9,"last update":"Fri Jul 21 18:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.621},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.73699999999999999}]},{"osd":10,"last update":"Fri Jul 21 18:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.86799999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]}]},{"osd":7,"up_from":48,"seq":206158430237,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28372,"kb_used_data":1420,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716780,"statfs":{"total":95995035648,"available":95965982720,"internally_reserved":0,"allocated":1454080,"data_stored":999541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.53400000000000003,"15min":0.52600000000000002},"min":{"1min":0.41599999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.53900000000000003,"15min":0.53600000000000003},"min":{"1min":0.34000000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.48899999999999999}]},{"osd":1,"last update":"Fri Jul 21 18:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.41099999999999998,"15min":0.40400000000000003},"min":{"1min":0.311,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.43099999999999999,"15min":0.42299999999999999},"min":{"1min":0.307,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.71099999999999997,"5min":0.75,"15min":0.75},"last":0.39100000000000001}]},{"osd":2,"last update":"Fri Jul 21 18:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.57599999999999996,"15min":0.57099999999999995},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58899999999999997,"15min":0.58999999999999997},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.72699999999999998}]},{"osd":3,"last update":"Fri Jul 21 18:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.54900000000000004,"15min":0.54000000000000004},"min":{"1min":0.39700000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.58299999999999996,"15min":0.57199999999999995},"min":{"1min":0.44500000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.86899999999999999,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.64600000000000002}]},{"osd":4,"last update":"Fri Jul 21 18:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.52700000000000002,"15min":0.52100000000000002},"min":{"1min":0.35399999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.52500000000000002,"15min":0.51700000000000002},"min":{"1min":0.39800000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.89500000000000002,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.33800000000000002}]},{"osd":5,"last update":"Fri Jul 21 18:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.64800000000000002,"15min":0.64100000000000001},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.64600000000000002,"15min":0.64200000000000002},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.59999999999999998}]},{"osd":6,"last update":"Fri Jul 21 18:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.66500000000000004,"15min":0.65500000000000003},"min":{"1min":0.51300000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.93500000000000005,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.67300000000000004,"15min":0.66100000000000003},"min":{"1min":0.46400000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0069999999999999,"5min":1.087,"15min":1.087},"last":0.58799999999999997}]},{"osd":8,"last update":"Fri Jul 21 18:39:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.71699999999999997}]},{"osd":9,"last update":"Fri Jul 21 18:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.51000000000000001}]},{"osd":10,"last update":"Fri Jul 21 18:40:06 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.63}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]}]},{"osd":6,"up_from":43,"seq":184683593761,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27776,"kb_used_data":824,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717376,"statfs":{"total":95995035648,"available":95966593024,"internally_reserved":0,"allocated":843776,"data_stored":407477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.41099999999999998,"15min":0.40799999999999997},"min":{"1min":0.30199999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.68200000000000005,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.43099999999999999,"15min":0.42799999999999999},"min":{"1min":0.28899999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.311}]},{"osd":1,"last update":"Fri Jul 21 18:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.61199999999999999,"15min":0.60799999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.872,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.621,"5min":0.58999999999999997,"15min":0.58499999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.629}]},{"osd":2,"last update":"Fri Jul 21 18:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.59799999999999998,"15min":0.58799999999999997},"min":{"1min":0.38500000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.441},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.59499999999999997,"15min":0.58099999999999996},"min":{"1min":0.47199999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.42099999999999999}]},{"osd":3,"last update":"Fri Jul 21 18:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.47599999999999998,"15min":0.47199999999999998},"min":{"1min":0.33600000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.81899999999999995,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.51600000000000001,"15min":0.51300000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.39900000000000002}]},{"osd":4,"last update":"Fri Jul 21 18:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.64800000000000002,"15min":0.64700000000000002},"min":{"1min":0.40899999999999997,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.65200000000000002,"15min":0.64800000000000002},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.57799999999999996}]},{"osd":5,"last update":"Fri Jul 21 18:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.66100000000000003,"15min":0.65100000000000002},"min":{"1min":0.438,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.70999999999999996,"15min":0.69799999999999995},"min":{"1min":0.53500000000000003,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.147,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.49299999999999999}]},{"osd":7,"last update":"Fri Jul 21 18:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.71299999999999997,"15min":0.70899999999999996},"min":{"1min":0.53500000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.70599999999999996,"15min":0.70499999999999996},"min":{"1min":0.54300000000000004,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.73199999999999998}]},{"osd":8,"last update":"Fri Jul 21 18:39:24 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.51000000000000001}]},{"osd":9,"last update":"Fri Jul 21 18:39:42 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.55300000000000005}]},{"osd":10,"last update":"Fri Jul 21 18:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.68999999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574903,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27832,"kb_used_data":880,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717320,"statfs":{"total":95995035648,"available":95966535680,"internally_reserved":0,"allocated":901120,"data_stored":445418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:39:59 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.50900000000000001,"15min":0.48499999999999999},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.52800000000000002,"15min":0.498},"min":{"1min":0.39200000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.51400000000000001}]},{"osd":2,"last update":"Fri Jul 21 18:39:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.53900000000000003,"15min":0.52100000000000002},"min":{"1min":0.38800000000000001,"5min":0.313,"15min":0.313},"max":{"1min":3.2959999999999998,"5min":3.2959999999999998,"15min":3.2959999999999998},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.625,"5min":0.53400000000000003,"15min":0.52000000000000002},"min":{"1min":0.34200000000000003,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":2.8650000000000002,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":0.73099999999999998}]},{"osd":3,"last update":"Fri Jul 21 18:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.53600000000000003,"15min":0.52000000000000002},"min":{"1min":0.35299999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.88200000000000001,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.62,"5min":0.54500000000000004,"15min":0.52500000000000002},"min":{"1min":0.379,"5min":0.377,"15min":0.377},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.60999999999999999}]},{"osd":4,"last update":"Fri Jul 21 18:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.41799999999999998,"15min":0.40100000000000002},"min":{"1min":0.32600000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.497,"5min":0.41799999999999998,"15min":0.40500000000000003},"min":{"1min":0.31,"5min":0.254,"15min":0.254},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.65600000000000003}]},{"osd":5,"last update":"Fri Jul 21 18:39:19 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.58699999999999997,"15min":0.58099999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.653,"5min":1.653,"15min":1.653},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.59799999999999998,"15min":0.58999999999999997},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.641,"5min":1.641,"15min":1.641},"last":0.94999999999999996}]},{"osd":6,"last update":"Fri Jul 21 18:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.63600000000000001,"15min":0.63},"min":{"1min":0.45900000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.64100000000000001,"15min":0.63900000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.68600000000000005}]},{"osd":7,"last update":"Fri Jul 21 18:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.51900000000000002,"15min":0.502},"min":{"1min":0.41399999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.54800000000000004,"15min":0.53000000000000003},"min":{"1min":0.40699999999999997,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.76300000000000001}]},{"osd":8,"last update":"Fri Jul 21 18:39:22 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":2.7360000000000002,"5min":2.7360000000000002,"15min":2.7360000000000002},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":2.7210000000000001,"5min":2.7210000000000001,"15min":2.7210000000000001},"last":0.96899999999999997}]},{"osd":9,"last update":"Fri Jul 21 18:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.99099999999999999},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.79500000000000004}]},{"osd":10,"last update":"Fri Jul 21 18:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":1.0149999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738426,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27784,"kb_used_data":832,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717368,"statfs":{"total":95995035648,"available":95966584832,"internally_reserved":0,"allocated":851968,"data_stored":407840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 21 18:39:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.59099999999999997,"15min":0.57399999999999995},"min":{"1min":0.36299999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.88200000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.57699999999999996,"15min":0.53700000000000003},"min":{"1min":0.42899999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.71499999999999997}]},{"osd":2,"last update":"Fri Jul 21 18:39:16 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.57399999999999995,"15min":0.56399999999999995},"min":{"1min":0.44800000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.81899999999999995,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.56999999999999995,"15min":0.57399999999999995},"min":{"1min":0.46000000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.83499999999999996,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":0.81499999999999995}]},{"osd":3,"last update":"Fri Jul 21 18:38:36 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.42799999999999999,"15min":0.42999999999999999},"min":{"1min":0.29999999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.621,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.48199999999999998,"15min":0.47699999999999998},"min":{"1min":0.33600000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.69199999999999995,"5min":0.879,"15min":0.879},"last":0.56999999999999995}]},{"osd":4,"last update":"Fri Jul 21 18:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.60599999999999998,"15min":0.57199999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":2.948,"5min":2.948,"15min":2.948},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.56200000000000006,"15min":0.54600000000000004},"min":{"1min":0.40600000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.80400000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.53800000000000003}]},{"osd":5,"last update":"Fri Jul 21 18:39:20 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.60099999999999998,"15min":0.58599999999999997},"min":{"1min":0.54200000000000004,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.61499999999999999,"15min":0.60199999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.93999999999999995}]},{"osd":6,"last update":"Fri Jul 21 18:39:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.52500000000000002,"15min":0.51200000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.58099999999999996,"15min":0.56699999999999995},"min":{"1min":0.36599999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.66500000000000004}]},{"osd":7,"last update":"Fri Jul 21 18:39:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.47699999999999998}]},{"osd":8,"last update":"Fri Jul 21 18:39:22 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.97099999999999997}]},{"osd":9,"last update":"Fri Jul 21 18:39:44 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.296,"5min":1.296,"15min":1.296},"last":0.73299999999999998}]},{"osd":10,"last update":"Fri Jul 21 18:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":2.9079999999999999,"5min":2.9079999999999999,"15min":2.9079999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.77800000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411378,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27804,"kb_used_data":852,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717348,"statfs":{"total":95995035648,"available":95966564352,"internally_reserved":0,"allocated":872448,"data_stored":409922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:39:17 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.50600000000000001,"15min":0.502},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.52000000000000002}]},{"osd":1,"last update":"Fri Jul 21 18:39:17 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.51600000000000001,"15min":0.51300000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.53800000000000003,"15min":0.53000000000000003},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.90800000000000003}]},{"osd":3,"last update":"Fri Jul 21 18:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.57299999999999995,"15min":0.56000000000000005},"min":{"1min":0.45000000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.56200000000000006,"15min":0.55800000000000005},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.63}]},{"osd":4,"last update":"Fri Jul 21 18:39:57 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.58999999999999997,"15min":0.56999999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.633,"5min":1.633,"15min":1.633},"last":0.87},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.626,"15min":0.60899999999999999},"min":{"1min":0.42999999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.79100000000000004}]},{"osd":5,"last update":"Fri Jul 21 18:39:17 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.44800000000000001,"15min":0.44900000000000001},"min":{"1min":0.32600000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.71799999999999997,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.45300000000000001,"15min":0.45100000000000001},"min":{"1min":0.32100000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.57299999999999995}]},{"osd":6,"last update":"Fri Jul 21 18:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.59999999999999998,"15min":0.58999999999999997},"min":{"1min":0.42999999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.61899999999999999,"15min":0.61699999999999999},"min":{"1min":0.39500000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.70599999999999996}]},{"osd":7,"last update":"Fri Jul 21 18:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.61699999999999999,"15min":0.60799999999999998},"min":{"1min":0.45100000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.577,"5min":1.577,"15min":1.577},"last":0.495},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.65500000000000003,"15min":0.65100000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.878,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.56499999999999995}]},{"osd":8,"last update":"Fri Jul 21 18:39:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.73299999999999998}]},{"osd":9,"last update":"Fri Jul 21 18:39:42 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.748}]},{"osd":10,"last update":"Fri Jul 21 18:40:06 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.95799999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93500000000000005}]}]},{"osd":3,"up_from":24,"seq":103079215149,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27784,"kb_used_data":832,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717368,"statfs":{"total":95995035648,"available":95966584832,"internally_reserved":0,"allocated":851968,"data_stored":407340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:39:36 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.34799999999999998,"15min":0.32800000000000001},"min":{"1min":0.28399999999999997,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.442,"5min":0.36899999999999999,"15min":0.34899999999999998},"min":{"1min":0.28399999999999997,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.38}]},{"osd":1,"last update":"Fri Jul 21 18:39:36 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.56999999999999995,"15min":0.55000000000000004},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.57899999999999996,"15min":0.54500000000000004},"min":{"1min":0.495,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.83899999999999997,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.57199999999999995}]},{"osd":2,"last update":"Fri Jul 21 18:39:36 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.57599999999999996,"15min":0.55300000000000005},"min":{"1min":0.48799999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.55300000000000005,"15min":0.53400000000000003},"min":{"1min":0.46400000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.67400000000000004}]},{"osd":4,"last update":"Fri Jul 21 18:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.61599999999999999,"15min":0.60499999999999998},"min":{"1min":0.42799999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.61699999999999999,"15min":0.59999999999999998},"min":{"1min":0.45300000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.69499999999999995}]},{"osd":5,"last update":"Fri Jul 21 18:39:19 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.60799999999999998,"15min":0.58599999999999997},"min":{"1min":0.49099999999999999,"5min":0.374,"15min":0.374},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.59099999999999997,"15min":0.57899999999999996},"min":{"1min":0.44,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.754}]},{"osd":6,"last update":"Fri Jul 21 18:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.5,"15min":0.48699999999999999},"min":{"1min":0.32300000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.57199999999999995,"15min":0.55900000000000005},"min":{"1min":0.40600000000000003,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.625}]},{"osd":7,"last update":"Fri Jul 21 18:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.66800000000000004,"15min":0.67000000000000004},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.97299999999999998,"5min":0.999,"15min":0.999},"last":0.5},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.73299999999999998,"15min":0.72499999999999998},"min":{"1min":0.55100000000000005,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.77600000000000002}]},{"osd":8,"last update":"Fri Jul 21 18:39:25 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.47299999999999998}]},{"osd":9,"last update":"Fri Jul 21 18:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.879}]},{"osd":10,"last update":"Fri Jul 21 18:40:07 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.81499999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]}]},{"osd":4,"up_from":31,"seq":133143986218,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27796,"kb_used_data":844,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717356,"statfs":{"total":95995035648,"available":95966572544,"internally_reserved":0,"allocated":864256,"data_stored":408519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:40:02 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.51800000000000002,"15min":0.48199999999999998},"min":{"1min":0.40500000000000003,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.94099999999999995,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.52500000000000002,"15min":0.50600000000000001},"min":{"1min":0.373,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.80600000000000005,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.58299999999999996}]},{"osd":1,"last update":"Fri Jul 21 18:40:02 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.373,"15min":0.34599999999999997},"min":{"1min":0.29799999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.40899999999999997,"15min":0.38600000000000001},"min":{"1min":0.30399999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.76200000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.36099999999999999}]},{"osd":2,"last update":"Fri Jul 21 18:40:02 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.54700000000000004,"15min":0.52900000000000003},"min":{"1min":0.44700000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.80700000000000005,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.56200000000000006,"15min":0.53200000000000003},"min":{"1min":0.41799999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.77200000000000002}]},{"osd":3,"last update":"Fri Jul 21 18:40:02 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.55800000000000005,"15min":0.53300000000000003},"min":{"1min":0.52300000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.85999999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56299999999999994,"15min":0.55700000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.97199999999999998,"5min":1.05,"15min":1.05},"last":0.39800000000000002}]},{"osd":5,"last update":"Fri Jul 21 18:39:18 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.61699999999999999,"15min":0.59799999999999998},"min":{"1min":0.42599999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.571,"5min":1.571,"15min":1.571},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.624,"15min":0.60899999999999999},"min":{"1min":0.50900000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":0.64400000000000002}]},{"osd":6,"last update":"Fri Jul 21 18:39:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.63100000000000001,"15min":0.61399999999999999},"min":{"1min":0.53000000000000003,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.63200000000000001,"15min":0.60799999999999998},"min":{"1min":0.54100000000000004,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.46100000000000002}]},{"osd":7,"last update":"Fri Jul 21 18:40:02 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.57299999999999995,"15min":0.56100000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.54500000000000004,"15min":0.54000000000000004},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.873,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.498}]},{"osd":8,"last update":"Fri Jul 21 18:39:21 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.745}]},{"osd":9,"last update":"Fri Jul 21 18:39:44 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.60499999999999998}]},{"osd":10,"last update":"Fri Jul 21 18:40:07 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.62}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]}]},{"osd":5,"up_from":36,"seq":154618822693,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28400,"kb_used_data":1448,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716752,"statfs":{"total":95995035648,"available":95965954048,"internally_reserved":0,"allocated":1482752,"data_stored":1000617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:39:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.53500000000000003,"15min":0.53300000000000003},"min":{"1min":0.29499999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.55600000000000005,"15min":0.56000000000000005},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.48499999999999999}]},{"osd":1,"last update":"Fri Jul 21 18:39:20 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.50700000000000001,"15min":0.499},"min":{"1min":0.35899999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.51800000000000002,"15min":0.50600000000000001},"min":{"1min":0.41099999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.75800000000000001}]},{"osd":2,"last update":"Fri Jul 21 18:39:20 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.38900000000000001,"15min":0.38400000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.40200000000000002,"15min":0.39300000000000002},"min":{"1min":0.29399999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.57199999999999995}]},{"osd":3,"last update":"Fri Jul 21 18:39:20 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.56899999999999995,"15min":0.57099999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58199999999999996,"15min":0.58099999999999996},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.69199999999999995}]},{"osd":4,"last update":"Fri Jul 21 18:39:20 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.55900000000000005,"15min":0.54900000000000004},"min":{"1min":0.433,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.54000000000000004,"15min":0.53200000000000003},"min":{"1min":0.41299999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.80000000000000004}]},{"osd":6,"last update":"Fri Jul 21 18:39:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.60099999999999998,"15min":0.60199999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.59799999999999998,"15min":0.58899999999999997},"min":{"1min":0.48299999999999998,"5min":0.434,"15min":0.434},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.78200000000000003}]},{"osd":7,"last update":"Fri Jul 21 18:40:02 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.66600000000000004,"15min":0.66800000000000004},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.64300000000000002,"15min":0.64000000000000001},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.85999999999999999}]},{"osd":8,"last update":"Fri Jul 21 18:39:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.71999999999999997}]},{"osd":9,"last update":"Fri Jul 21 18:39:45 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.877},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.88900000000000001}]},{"osd":10,"last update":"Fri Jul 21 18:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.94199999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":37941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-21T18:40:13.437 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-07-21T18:40:13.437 INFO:tasks.cephadm:Setting up client nodes... 2023-07-21T18:40:13.438 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2023-07-21T18:40:14.023 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:40:14.282 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:13 smithi039 ceph-mon[96489]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T18:40:14.282 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:13 smithi039 ceph-mon[96489]: pgmap v252: 97 pgs: 97 active+clean; 583 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 6.0 KiB/s wr, 21 op/s 2023-07-21T18:40:14.373 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:13 smithi193 ceph-mon[98204]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T18:40:14.373 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:13 smithi193 ceph-mon[98204]: pgmap v252: 97 pgs: 97 active+clean; 583 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 6.0 KiB/s wr, 21 op/s 2023-07-21T18:40:14.392 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:13 smithi155 ceph-mon[97934]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T18:40:14.392 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:13 smithi155 ceph-mon[97934]: pgmap v252: 97 pgs: 97 active+clean; 583 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 6.0 KiB/s wr, 21 op/s 2023-07-21T18:40:14.392 INFO:journalctl@ceph.mds.b.smithi155.stdout:Jul 21 18:40:14 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-b[123270]: 2023-07-21T18:40:14.154+0000 7f8b91b33700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2023-07-21T18:40:14.393 INFO:journalctl@ceph.mds.b.smithi155.stdout:Jul 21 18:40:14 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-b[123270]: 2023-07-21T18:40:14.154+0000 7f8b91b33700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2023-07-21T18:40:15.274 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:15.248+0000 7fbf814d8700 1 -- 172.21.15.39:0/3001668280 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf7c0ffcb0 msgr2=0x7fbf7c100100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:15.275 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:15.248+0000 7fbf814d8700 1 --2- 172.21.15.39:0/3001668280 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf7c0ffcb0 0x7fbf7c100100 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbf70009990 tx=0x7fbf70009ca0 comp rx=0 tx=0).stop 2023-07-21T18:40:15.275 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:15.248+0000 7fbf814d8700 1 -- 172.21.15.39:0/3001668280 shutdown_connections 2023-07-21T18:40:15.275 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:15.248+0000 7fbf814d8700 1 --2- 172.21.15.39:0/3001668280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf7c100640 0x7fbf7c106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:15.275 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:15.248+0000 7fbf814d8700 1 --2- 172.21.15.39:0/3001668280 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf7c0feab0 0x7fbf7c0feec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:15.275 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:15.248+0000 7fbf814d8700 1 --2- 172.21.15.39:0/3001668280 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf7c0ffcb0 0x7fbf7c100100 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:15.276 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:15.248+0000 7fbf814d8700 1 -- 172.21.15.39:0/3001668280 >> 172.21.15.39:0/3001668280 conn(0x7fbf7c0fa0e0 msgr2=0x7fbf7c0fc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:15.276 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:15.248+0000 7fbf814d8700 1 -- 172.21.15.39:0/3001668280 shutdown_connections 2023-07-21T18:40:15.276 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:15.248+0000 7fbf814d8700 1 -- 172.21.15.39:0/3001668280 wait complete. 2023-07-21T18:40:15.276 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:15.249+0000 7fbf814d8700 1 Processor -- start 2023-07-21T18:40:15.276 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:15.249+0000 7fbf814d8700 1 -- start start 2023-07-21T18:40:15.278 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:15.249+0000 7fbf814d8700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf7c0feab0 0x7fbf7c195960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:15.278 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:15.249+0000 7fbf814d8700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf7c0ffcb0 0x7fbf7c195ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:15.278 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:15.249+0000 7fbf814d8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf7c100640 0x7fbf7c19cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:15.278 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:15.249+0000 7fbf814d8700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7c19d4f0 con 0x7fbf7c100640 2023-07-21T18:40:15.279 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:15.249+0000 7fbf814d8700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7c19d630 con 0x7fbf7c0ffcb0 2023-07-21T18:40:15.279 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:15.249+0000 7fbf814d8700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7c19d7c0 con 0x7fbf7c0feab0 2023-07-21T18:40:15.279 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:15.250+0000 7fbf7b7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf7c100640 0x7fbf7c19cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:15.279 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:15.250+0000 7fbf7b7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf7c100640 0x7fbf7c19cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55266/0 (socket says 172.21.15.39:55266) 2023-07-21T18:40:15.279 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:15.250+0000 7fbf7b7fe700 1 -- 172.21.15.39:0/1165598197 learned_addr learned my addr 172.21.15.39:0/1165598197 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:40:15.279 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:15.250+0000 7fbf7affd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf7c0feab0 0x7fbf7c195960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:15.280 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:15.250+0000 7fbf7a7fc700 1 --2- 172.21.15.39:0/1165598197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf7c0ffcb0 0x7fbf7c195ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:15.280 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:15.250+0000 7fbf7b7fe700 1 -- 172.21.15.39:0/1165598197 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf7c0feab0 msgr2=0x7fbf7c195960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:15.280 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:15.250+0000 7fbf7b7fe700 1 --2- 172.21.15.39:0/1165598197 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf7c0feab0 0x7fbf7c195960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:15.280 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:15.250+0000 7fbf7b7fe700 1 -- 172.21.15.39:0/1165598197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf7c0ffcb0 msgr2=0x7fbf7c195ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:15.280 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:15.250+0000 7fbf7b7fe700 1 --2- 172.21.15.39:0/1165598197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf7c0ffcb0 0x7fbf7c195ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:15.280 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:15.250+0000 7fbf7b7fe700 1 -- 172.21.15.39:0/1165598197 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf700095f0 con 0x7fbf7c100640 2023-07-21T18:40:15.280 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:15.250+0000 7fbf7b7fe700 1 --2- 172.21.15.39:0/1165598197 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf7c100640 0x7fbf7c19cf20 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbf7400b9a0 tx=0x7fbf7400bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:15.280 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:15.250+0000 7fbf63fff700 1 -- 172.21.15.39:0/1165598197 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf7400d850 con 0x7fbf7c100640 2023-07-21T18:40:15.281 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:15.250+0000 7fbf63fff700 1 -- 172.21.15.39:0/1165598197 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf7400de70 con 0x7fbf7c100640 2023-07-21T18:40:15.281 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:15.250+0000 7fbf814d8700 1 -- 172.21.15.39:0/1165598197 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf7c19daa0 con 0x7fbf7c100640 2023-07-21T18:40:15.281 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:15.250+0000 7fbf63fff700 1 -- 172.21.15.39:0/1165598197 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf74017400 con 0x7fbf7c100640 2023-07-21T18:40:15.281 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:15.250+0000 7fbf814d8700 1 -- 172.21.15.39:0/1165598197 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf7c19dfd0 con 0x7fbf7c100640 2023-07-21T18:40:15.281 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:15.251+0000 7fbf814d8700 1 -- 172.21.15.39:0/1165598197 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf7c106b40 con 0x7fbf7c100640 2023-07-21T18:40:15.281 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:15.252+0000 7fbf63fff700 1 -- 172.21.15.39:0/1165598197 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fbf7400e600 con 0x7fbf7c100640 2023-07-21T18:40:15.282 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:15.253+0000 7fbf63fff700 1 --2- 172.21.15.39:0/1165598197 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fbf640a11e0 0x7fbf640a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:15.282 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:15.253+0000 7fbf7affd700 1 --2- 172.21.15.39:0/1165598197 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fbf640a11e0 0x7fbf640a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:15.282 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:15.253+0000 7fbf63fff700 1 -- 172.21.15.39:0/1165598197 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fbf74006e40 con 0x7fbf7c100640 2023-07-21T18:40:15.283 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:15.254+0000 7fbf7affd700 1 --2- 172.21.15.39:0/1165598197 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fbf640a11e0 0x7fbf640a3690 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c0ffb10 tx=0x7fbf6c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:15.284 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:15.257+0000 7fbf63fff700 1 -- 172.21.15.39:0/1165598197 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fbf74095c50 con 0x7fbf7c100640 2023-07-21T18:40:15.373 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:14 smithi193 ceph-mon[98204]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T18:40:15.392 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:14 smithi155 ceph-mon[97934]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T18:40:15.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:14 smithi039 ceph-mon[96489]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T18:40:15.561 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:15.536+0000 7fbf814d8700 1 -- 172.21.15.39:0/1165598197 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fbf7c02c850 con 0x7fbf7c100640 2023-07-21T18:40:15.567 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:15.542+0000 7fbf63fff700 1 -- 172.21.15.39:0/1165598197 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v31) v1 ==== 170+0+59 (secure 0 0 0) 0x7fbf74099270 con 0x7fbf7c100640 2023-07-21T18:40:15.567 INFO:teuthology.orchestra.run.smithi039.stdout:[client.0] 2023-07-21T18:40:15.567 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQAP0bpk+18GIBAAWKk3f3ppNPOvnFSFE4WFNw== 2023-07-21T18:40:15.569 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:15.544+0000 7fbf814d8700 1 -- 172.21.15.39:0/1165598197 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fbf640a11e0 msgr2=0x7fbf640a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:15.569 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:15.544+0000 7fbf814d8700 1 --2- 172.21.15.39:0/1165598197 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fbf640a11e0 0x7fbf640a3690 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c0ffb10 tx=0x7fbf6c009190 comp rx=0 tx=0).stop 2023-07-21T18:40:15.569 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:15.544+0000 7fbf814d8700 1 -- 172.21.15.39:0/1165598197 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf7c100640 msgr2=0x7fbf7c19cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:15.570 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:15.544+0000 7fbf814d8700 1 --2- 172.21.15.39:0/1165598197 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf7c100640 0x7fbf7c19cf20 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbf7400b9a0 tx=0x7fbf7400bcb0 comp rx=0 tx=0).stop 2023-07-21T18:40:15.570 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:15.544+0000 7fbf814d8700 1 -- 172.21.15.39:0/1165598197 shutdown_connections 2023-07-21T18:40:15.570 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:15.544+0000 7fbf814d8700 1 --2- 172.21.15.39:0/1165598197 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fbf640a11e0 0x7fbf640a3690 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:15.570 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:15.544+0000 7fbf814d8700 1 --2- 172.21.15.39:0/1165598197 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf7c100640 0x7fbf7c19cf20 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:15.570 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:15.544+0000 7fbf814d8700 1 --2- 172.21.15.39:0/1165598197 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf7c0feab0 0x7fbf7c195960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:15.570 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:15.544+0000 7fbf814d8700 1 --2- 172.21.15.39:0/1165598197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf7c0ffcb0 0x7fbf7c195ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:15.570 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:15.544+0000 7fbf814d8700 1 -- 172.21.15.39:0/1165598197 >> 172.21.15.39:0/1165598197 conn(0x7fbf7c0fa0e0 msgr2=0x7fbf7c0fc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:15.571 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:15.544+0000 7fbf814d8700 1 -- 172.21.15.39:0/1165598197 shutdown_connections 2023-07-21T18:40:15.571 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:15.544+0000 7fbf814d8700 1 -- 172.21.15.39:0/1165598197 wait complete. 2023-07-21T18:40:15.980 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-07-21T18:40:15.980 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2023-07-21T18:40:15.980 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-07-21T18:40:16.025 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-07-21T18:40:16.026 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2023-07-21T18:40:16.026 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph mgr dump --format=json 2023-07-21T18:40:16.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:15 smithi039 ceph-mon[96489]: pgmap v253: 97 pgs: 97 active+clean; 622 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 10 KiB/s wr, 26 op/s 2023-07-21T18:40:16.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:15 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/1165598197' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-07-21T18:40:16.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:15 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/1165598197' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-07-21T18:40:16.373 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:15 smithi193 ceph-mon[98204]: pgmap v253: 97 pgs: 97 active+clean; 622 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 10 KiB/s wr, 26 op/s 2023-07-21T18:40:16.373 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:15 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/1165598197' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-07-21T18:40:16.373 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:15 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/1165598197' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-07-21T18:40:16.392 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:16 smithi155 ceph-mon[97934]: pgmap v253: 97 pgs: 97 active+clean; 622 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 10 KiB/s wr, 26 op/s 2023-07-21T18:40:16.392 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:16 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/1165598197' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-07-21T18:40:16.392 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:16 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/1165598197' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-07-21T18:40:16.625 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:40:17.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:17 smithi039 ceph-mon[96489]: pgmap v254: 97 pgs: 97 active+clean; 623 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 9.2 KiB/s wr, 25 op/s 2023-07-21T18:40:17.623 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:17 smithi193 ceph-mon[98204]: pgmap v254: 97 pgs: 97 active+clean; 623 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 9.2 KiB/s wr, 25 op/s 2023-07-21T18:40:17.642 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:17 smithi155 ceph-mon[97934]: pgmap v254: 97 pgs: 97 active+clean; 623 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 9.2 KiB/s wr, 25 op/s 2023-07-21T18:40:17.922 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:17.896+0000 7fc4c98a7700 1 -- 172.21.15.39:0/2509858042 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4c410a100 msgr2=0x7fc4c410c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:17.922 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:17.896+0000 7fc4c98a7700 1 --2- 172.21.15.39:0/2509858042 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4c410a100 0x7fc4c410c560 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc00b0a0 tx=0x7fc4bc00b3b0 comp rx=0 tx=0).stop 2023-07-21T18:40:17.923 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:17.896+0000 7fc4c98a7700 1 -- 172.21.15.39:0/2509858042 shutdown_connections 2023-07-21T18:40:17.923 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:17.896+0000 7fc4c98a7700 1 --2- 172.21.15.39:0/2509858042 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc4c4100db0 0x7fc4c4101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:17.923 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:17.896+0000 7fc4c98a7700 1 --2- 172.21.15.39:0/2509858042 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4c410a100 0x7fc4c410c560 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:17.923 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:17.896+0000 7fc4c98a7700 1 --2- 172.21.15.39:0/2509858042 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4c41016c0 0x7fc4c4109bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:17.923 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:17.896+0000 7fc4c98a7700 1 -- 172.21.15.39:0/2509858042 >> 172.21.15.39:0/2509858042 conn(0x7fc4c40fa570 msgr2=0x7fc4c40fc980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:17.923 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:17.896+0000 7fc4c98a7700 1 -- 172.21.15.39:0/2509858042 shutdown_connections 2023-07-21T18:40:17.924 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:17.896+0000 7fc4c98a7700 1 -- 172.21.15.39:0/2509858042 wait complete. 2023-07-21T18:40:17.925 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:17.897+0000 7fc4c98a7700 1 Processor -- start 2023-07-21T18:40:17.926 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:17.897+0000 7fc4c98a7700 1 -- start start 2023-07-21T18:40:17.926 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:17.898+0000 7fc4c98a7700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4c4100db0 0x7fc4c419bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:17.926 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:17.898+0000 7fc4c98a7700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc4c41016c0 0x7fc4c419c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:17.926 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:17.898+0000 7fc4c98a7700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4c410a100 0x7fc4c4195b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:17.926 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:17.898+0000 7fc4c98a7700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c41961c0 con 0x7fc4c4100db0 2023-07-21T18:40:17.927 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:17.898+0000 7fc4c98a7700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c4196330 con 0x7fc4c41016c0 2023-07-21T18:40:17.927 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:17.898+0000 7fc4c98a7700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c41964f0 con 0x7fc4c410a100 2023-07-21T18:40:17.927 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:17.898+0000 7fc4c2ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4c4100db0 0x7fc4c419bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:17.927 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:17.898+0000 7fc4c2ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4c4100db0 0x7fc4c419bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55276/0 (socket says 172.21.15.39:55276) 2023-07-21T18:40:17.927 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:17.898+0000 7fc4c2ffd700 1 -- 172.21.15.39:0/2576911629 learned_addr learned my addr 172.21.15.39:0/2576911629 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:40:17.928 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:17.898+0000 7fc4c27fc700 1 --2- 172.21.15.39:0/2576911629 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc4c41016c0 0x7fc4c419c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:17.928 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:17.898+0000 7fc4c37fe700 1 --2- 172.21.15.39:0/2576911629 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4c410a100 0x7fc4c4195b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:17.928 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:17.898+0000 7fc4c2ffd700 1 -- 172.21.15.39:0/2576911629 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4c410a100 msgr2=0x7fc4c4195b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:17.928 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:17.898+0000 7fc4c2ffd700 1 --2- 172.21.15.39:0/2576911629 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4c410a100 0x7fc4c4195b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:17.928 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:17.898+0000 7fc4c2ffd700 1 -- 172.21.15.39:0/2576911629 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc4c41016c0 msgr2=0x7fc4c419c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:17.928 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:17.898+0000 7fc4c2ffd700 1 --2- 172.21.15.39:0/2576911629 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc4c41016c0 0x7fc4c419c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:17.928 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:17.898+0000 7fc4c2ffd700 1 -- 172.21.15.39:0/2576911629 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4bc009d00 con 0x7fc4c4100db0 2023-07-21T18:40:17.929 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:17.898+0000 7fc4c2ffd700 1 --2- 172.21.15.39:0/2576911629 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4c4100db0 0x7fc4c419bad0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc4b400c920 tx=0x7fc4b400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:17.929 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:17.898+0000 7fc4c88a5700 1 -- 172.21.15.39:0/2576911629 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b4007b00 con 0x7fc4c4100db0 2023-07-21T18:40:17.929 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:17.898+0000 7fc4c88a5700 1 -- 172.21.15.39:0/2576911629 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc4b4004cf0 con 0x7fc4c4100db0 2023-07-21T18:40:17.929 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:17.898+0000 7fc4c98a7700 1 -- 172.21.15.39:0/2576911629 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4c41967d0 con 0x7fc4c4100db0 2023-07-21T18:40:17.929 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:17.898+0000 7fc4c88a5700 1 -- 172.21.15.39:0/2576911629 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b4005610 con 0x7fc4c4100db0 2023-07-21T18:40:17.930 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:17.898+0000 7fc4c98a7700 1 -- 172.21.15.39:0/2576911629 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4c41a2ac0 con 0x7fc4c4100db0 2023-07-21T18:40:17.930 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:17.899+0000 7fc4c98a7700 1 -- 172.21.15.39:0/2576911629 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4c40681c0 con 0x7fc4c4100db0 2023-07-21T18:40:17.930 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:17.900+0000 7fc4c88a5700 1 -- 172.21.15.39:0/2576911629 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fc4b4004750 con 0x7fc4c4100db0 2023-07-21T18:40:17.931 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:17.901+0000 7fc4c88a5700 1 --2- 172.21.15.39:0/2576911629 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fc4ac0a1260 0x7fc4ac0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:17.931 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:17.901+0000 7fc4c27fc700 1 --2- 172.21.15.39:0/2576911629 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fc4ac0a1260 0x7fc4ac0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:17.931 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:17.901+0000 7fc4c88a5700 1 -- 172.21.15.39:0/2576911629 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc4b40cc920 con 0x7fc4c4100db0 2023-07-21T18:40:17.931 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:17.901+0000 7fc4c27fc700 1 --2- 172.21.15.39:0/2576911629 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fc4ac0a1260 0x7fc4ac0a3710 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8005fd0 tx=0x7fc4b8005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:17.931 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:17.905+0000 7fc4c88a5700 1 -- 172.21.15.39:0/2576911629 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fc4b4095fc0 con 0x7fc4c4100db0 2023-07-21T18:40:18.186 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:18.161+0000 7fc4c98a7700 1 -- 172.21.15.39:0/2576911629 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fc4c404e210 con 0x7fc4c4100db0 2023-07-21T18:40:18.191 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:18.166+0000 7fc4c88a5700 1 -- 172.21.15.39:0/2576911629 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+260125 (secure 0 0 0) 0x7fc4b401d020 con 0x7fc4c4100db0 2023-07-21T18:40:18.194 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:40:18.196 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:18.169+0000 7fc4c98a7700 1 -- 172.21.15.39:0/2576911629 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fc4ac0a1260 msgr2=0x7fc4ac0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:18.196 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:18.169+0000 7fc4c98a7700 1 --2- 172.21.15.39:0/2576911629 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fc4ac0a1260 0x7fc4ac0a3710 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8005fd0 tx=0x7fc4b8005db0 comp rx=0 tx=0).stop 2023-07-21T18:40:18.196 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:18.169+0000 7fc4c98a7700 1 -- 172.21.15.39:0/2576911629 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4c4100db0 msgr2=0x7fc4c419bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:18.196 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:18.169+0000 7fc4c98a7700 1 --2- 172.21.15.39:0/2576911629 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4c4100db0 0x7fc4c419bad0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc4b400c920 tx=0x7fc4b400cce0 comp rx=0 tx=0).stop 2023-07-21T18:40:18.196 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:18.169+0000 7fc4c98a7700 1 -- 172.21.15.39:0/2576911629 shutdown_connections 2023-07-21T18:40:18.196 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:18.169+0000 7fc4c98a7700 1 --2- 172.21.15.39:0/2576911629 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fc4ac0a1260 0x7fc4ac0a3710 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:18.197 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:18.169+0000 7fc4c98a7700 1 --2- 172.21.15.39:0/2576911629 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4c410a100 0x7fc4c4195b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:18.197 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:18.169+0000 7fc4c98a7700 1 --2- 172.21.15.39:0/2576911629 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4c4100db0 0x7fc4c419bad0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:18.197 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:18.169+0000 7fc4c98a7700 1 --2- 172.21.15.39:0/2576911629 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc4c41016c0 0x7fc4c419c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:18.197 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:18.169+0000 7fc4c98a7700 1 -- 172.21.15.39:0/2576911629 >> 172.21.15.39:0/2576911629 conn(0x7fc4c40fa570 msgr2=0x7fc4c410c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:18.197 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:18.169+0000 7fc4c98a7700 1 -- 172.21.15.39:0/2576911629 shutdown_connections 2023-07-21T18:40:18.197 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:18.169+0000 7fc4c98a7700 1 -- 172.21.15.39:0/2576911629 wait complete. 2023-07-21T18:40:18.568 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:18 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/2576911629' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-07-21T18:40:18.570 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":17,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":2116291712},{"type":"v1","addr":"172.21.15.39:6801","nonce":2116291712}]},"active_addr":"172.21.15.39:6801/2116291712","active_change":"2023-07-21T18:34:07.039986+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.39:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.39:0","nonce":545627125}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.39:0","nonce":4249110368}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.39:0","nonce":3797141824}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.39:0","nonce":1870486291}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.39:0","nonce":1762298871}]}]} 2023-07-21T18:40:18.580 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2023-07-21T18:40:18.580 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2023-07-21T18:40:18.580 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph osd dump --format=json 2023-07-21T18:40:18.623 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:18 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/2576911629' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-07-21T18:40:18.642 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:18 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/2576911629' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-07-21T18:40:19.142 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:40:19.537 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:19 smithi039 ceph-mon[96489]: pgmap v255: 97 pgs: 97 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.8 KiB/s wr, 25 op/s 2023-07-21T18:40:19.623 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:19 smithi193 ceph-mon[98204]: pgmap v255: 97 pgs: 97 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.8 KiB/s wr, 25 op/s 2023-07-21T18:40:19.642 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:19 smithi155 ceph-mon[97934]: pgmap v255: 97 pgs: 97 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.8 KiB/s wr, 25 op/s 2023-07-21T18:40:20.357 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:20.329+0000 7faf4448d700 1 -- 172.21.15.39:0/1172018434 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf3c104ab0 msgr2=0x7faf3c104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:20.357 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:20.329+0000 7faf4448d700 1 --2- 172.21.15.39:0/1172018434 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf3c104ab0 0x7faf3c104e80 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7faf38009990 tx=0x7faf38009ca0 comp rx=0 tx=0).stop 2023-07-21T18:40:20.358 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:20.329+0000 7faf4448d700 1 -- 172.21.15.39:0/1172018434 shutdown_connections 2023-07-21T18:40:20.358 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:20.329+0000 7faf4448d700 1 --2- 172.21.15.39:0/1172018434 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf3c0ff460 0x7faf3c10b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:20.358 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:20.329+0000 7faf4448d700 1 --2- 172.21.15.39:0/1172018434 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf3c104ab0 0x7faf3c104e80 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:20.358 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:20.329+0000 7faf4448d700 1 --2- 172.21.15.39:0/1172018434 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf3c0feab0 0x7faf3c0fef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:20.359 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:20.329+0000 7faf4448d700 1 -- 172.21.15.39:0/1172018434 >> 172.21.15.39:0/1172018434 conn(0x7faf3c075b70 msgr2=0x7faf3c075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:20.359 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:20.330+0000 7faf4448d700 1 -- 172.21.15.39:0/1172018434 shutdown_connections 2023-07-21T18:40:20.359 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:20.330+0000 7faf4448d700 1 -- 172.21.15.39:0/1172018434 wait complete. 2023-07-21T18:40:20.359 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:20.330+0000 7faf4448d700 1 Processor -- start 2023-07-21T18:40:20.359 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:20.330+0000 7faf4448d700 1 -- start start 2023-07-21T18:40:20.359 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:20.330+0000 7faf4448d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf3c0feab0 0x7faf3c196ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:20.360 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:20.330+0000 7faf4448d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf3c0ff460 0x7faf3c197000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:20.360 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:20.330+0000 7faf4448d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf3c104ab0 0x7faf3c1a1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:20.360 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:20.330+0000 7faf4448d700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3c197910 con 0x7faf3c0feab0 2023-07-21T18:40:20.360 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:20.330+0000 7faf4448d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3c1a2080 con 0x7faf3c104ab0 2023-07-21T18:40:20.360 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:20.330+0000 7faf4448d700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3c1a21f0 con 0x7faf3c0ff460 2023-07-21T18:40:20.360 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:20.331+0000 7faf42229700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf3c0feab0 0x7faf3c196ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:20.360 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:20.331+0000 7faf42229700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf3c0feab0 0x7faf3c196ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55294/0 (socket says 172.21.15.39:55294) 2023-07-21T18:40:20.361 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:20.331+0000 7faf42229700 1 -- 172.21.15.39:0/374246969 learned_addr learned my addr 172.21.15.39:0/374246969 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:40:20.361 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:20.331+0000 7faf41a28700 1 --2- 172.21.15.39:0/374246969 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf3c0ff460 0x7faf3c197000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:20.361 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:20.331+0000 7faf42229700 1 -- 172.21.15.39:0/374246969 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf3c0ff460 msgr2=0x7faf3c197000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:20.361 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:20.331+0000 7faf42229700 1 --2- 172.21.15.39:0/374246969 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf3c0ff460 0x7faf3c197000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:20.361 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:20.331+0000 7faf42a2a700 1 --2- 172.21.15.39:0/374246969 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf3c104ab0 0x7faf3c1a1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:20.361 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:20.331+0000 7faf42229700 1 -- 172.21.15.39:0/374246969 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf3c104ab0 msgr2=0x7faf3c1a1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:20.361 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:20.331+0000 7faf42229700 1 --2- 172.21.15.39:0/374246969 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf3c104ab0 0x7faf3c1a1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:20.361 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:20.331+0000 7faf42229700 1 -- 172.21.15.39:0/374246969 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf380095f0 con 0x7faf3c0feab0 2023-07-21T18:40:20.362 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:20.331+0000 7faf42a2a700 1 --2- 172.21.15.39:0/374246969 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf3c104ab0 0x7faf3c1a1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:40:20.362 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:20.331+0000 7faf42229700 1 --2- 172.21.15.39:0/374246969 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf3c0feab0 0x7faf3c196ac0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7faf38003730 tx=0x7faf3800f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:20.362 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:20.331+0000 7faf2f7fe700 1 -- 172.21.15.39:0/374246969 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf3801d070 con 0x7faf3c0feab0 2023-07-21T18:40:20.362 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:20.331+0000 7faf2f7fe700 1 -- 172.21.15.39:0/374246969 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf3800fc30 con 0x7faf3c0feab0 2023-07-21T18:40:20.362 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:20.331+0000 7faf4448d700 1 -- 172.21.15.39:0/374246969 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf3c1a2470 con 0x7faf3c0feab0 2023-07-21T18:40:20.362 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:20.331+0000 7faf2f7fe700 1 -- 172.21.15.39:0/374246969 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf380173f0 con 0x7faf3c0feab0 2023-07-21T18:40:20.362 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:20.331+0000 7faf4448d700 1 -- 172.21.15.39:0/374246969 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf3c1a2880 con 0x7faf3c0feab0 2023-07-21T18:40:20.363 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:20.332+0000 7faf4448d700 1 -- 172.21.15.39:0/374246969 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf3c04e210 con 0x7faf3c0feab0 2023-07-21T18:40:20.363 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:20.333+0000 7faf2f7fe700 1 -- 172.21.15.39:0/374246969 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7faf38022470 con 0x7faf3c0feab0 2023-07-21T18:40:20.364 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:20.334+0000 7faf2f7fe700 1 --2- 172.21.15.39:0/374246969 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7faf280a11e0 0x7faf280a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:20.364 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:20.334+0000 7faf2f7fe700 1 -- 172.21.15.39:0/374246969 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7faf380ce0a0 con 0x7faf3c0feab0 2023-07-21T18:40:20.364 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:20.334+0000 7faf41a28700 1 --2- 172.21.15.39:0/374246969 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7faf280a11e0 0x7faf280a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:20.364 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:20.335+0000 7faf41a28700 1 --2- 172.21.15.39:0/374246969 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7faf280a11e0 0x7faf280a3690 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7faf30005950 tx=0x7faf300058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:20.364 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:20.338+0000 7faf2f7fe700 1 -- 172.21.15.39:0/374246969 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7faf380978d0 con 0x7faf3c0feab0 2023-07-21T18:40:20.559 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:20.534+0000 7faf4448d700 1 -- 172.21.15.39:0/374246969 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faf3c1a2bb0 con 0x7faf3c0feab0 2023-07-21T18:40:20.561 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:20.536+0000 7faf2f7fe700 1 -- 172.21.15.39:0/374246969 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+22117 (secure 0 0 0) 0x7faf3809aef0 con 0x7faf3c0feab0 2023-07-21T18:40:20.561 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:40:20.562 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":79,"fsid":"eaf8e25a-27f4-11ee-9b34-001a4aab830c","created":"2023-07-21T18:33:06.504979+0000","modified":"2023-07-21T18:40:05.769300+0000","last_up_change":"2023-07-21T18:39:23.531009+0000","last_in_change":"2023-07-21T18:39:05.993024+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T18:36:13.572583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T18:40:02.726735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T18:40:03.990112+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03502c07-2d0f-4723-8d0b-d34c9f5b91bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6803","nonce":860193857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6805","nonce":860193857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6809","nonce":860193857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6807","nonce":860193857}]},"public_addr":"172.21.15.39:6803/860193857","cluster_addr":"172.21.15.39:6805/860193857","heartbeat_back_addr":"172.21.15.39:6809/860193857","heartbeat_front_addr":"172.21.15.39:6807/860193857","state":["exists","up"]},{"osd":1,"uuid":"0831a0c0-a1fa-4d30-9a28-513223aa69eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6801","nonce":3246721764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6803","nonce":3246721764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6807","nonce":3246721764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6805","nonce":3246721764}]},"public_addr":"172.21.15.155:6801/3246721764","cluster_addr":"172.21.15.155:6803/3246721764","heartbeat_back_addr":"172.21.15.155:6807/3246721764","heartbeat_front_addr":"172.21.15.155:6805/3246721764","state":["exists","up"]},{"osd":2,"uuid":"22df0003-bde9-48e4-b47a-cb8eafbab0db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6801","nonce":2563594449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6803","nonce":2563594449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6807","nonce":2563594449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6805","nonce":2563594449}]},"public_addr":"172.21.15.193:6801/2563594449","cluster_addr":"172.21.15.193:6803/2563594449","heartbeat_back_addr":"172.21.15.193:6807/2563594449","heartbeat_front_addr":"172.21.15.193:6805/2563594449","state":["exists","up"]},{"osd":3,"uuid":"54562558-a2e8-45dd-bc95-561569b9ae6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6811","nonce":863708386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6813","nonce":863708386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6817","nonce":863708386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6815","nonce":863708386}]},"public_addr":"172.21.15.39:6811/863708386","cluster_addr":"172.21.15.39:6813/863708386","heartbeat_back_addr":"172.21.15.39:6817/863708386","heartbeat_front_addr":"172.21.15.39:6815/863708386","state":["exists","up"]},{"osd":4,"uuid":"491118e7-ae1e-495c-9eac-ef1cdcae9600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6809","nonce":3361780673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6811","nonce":3361780673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6815","nonce":3361780673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6813","nonce":3361780673}]},"public_addr":"172.21.15.155:6809/3361780673","cluster_addr":"172.21.15.155:6811/3361780673","heartbeat_back_addr":"172.21.15.155:6815/3361780673","heartbeat_front_addr":"172.21.15.155:6813/3361780673","state":["exists","up"]},{"osd":5,"uuid":"d267bc2c-42b3-4d96-b2d1-b186dc7e6b18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6809","nonce":2376867721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6811","nonce":2376867721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6815","nonce":2376867721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6813","nonce":2376867721}]},"public_addr":"172.21.15.193:6809/2376867721","cluster_addr":"172.21.15.193:6811/2376867721","heartbeat_back_addr":"172.21.15.193:6815/2376867721","heartbeat_front_addr":"172.21.15.193:6813/2376867721","state":["exists","up"]},{"osd":6,"uuid":"beec8088-970a-473e-bd67-6eee7cbdef24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6819","nonce":2417770529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6821","nonce":2417770529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6825","nonce":2417770529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6823","nonce":2417770529}]},"public_addr":"172.21.15.39:6819/2417770529","cluster_addr":"172.21.15.39:6821/2417770529","heartbeat_back_addr":"172.21.15.39:6825/2417770529","heartbeat_front_addr":"172.21.15.39:6823/2417770529","state":["exists","up"]},{"osd":7,"uuid":"5a88ce30-1835-4254-90cc-804ed5cf1066","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6817","nonce":2542002111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6819","nonce":2542002111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6823","nonce":2542002111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6821","nonce":2542002111}]},"public_addr":"172.21.15.155:6817/2542002111","cluster_addr":"172.21.15.155:6819/2542002111","heartbeat_back_addr":"172.21.15.155:6823/2542002111","heartbeat_front_addr":"172.21.15.155:6821/2542002111","state":["exists","up"]},{"osd":8,"uuid":"d964a183-3021-4ace-9a0c-aad00f2570e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6817","nonce":2985835085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6819","nonce":2985835085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6823","nonce":2985835085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6821","nonce":2985835085}]},"public_addr":"172.21.15.193:6817/2985835085","cluster_addr":"172.21.15.193:6819/2985835085","heartbeat_back_addr":"172.21.15.193:6823/2985835085","heartbeat_front_addr":"172.21.15.193:6821/2985835085","state":["exists","up"]},{"osd":9,"uuid":"9725ea5c-49a3-4c99-bac4-e95069100b1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6827","nonce":1946493651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6829","nonce":1946493651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6833","nonce":1946493651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6831","nonce":1946493651}]},"public_addr":"172.21.15.39:6827/1946493651","cluster_addr":"172.21.15.39:6829/1946493651","heartbeat_back_addr":"172.21.15.39:6833/1946493651","heartbeat_front_addr":"172.21.15.39:6831/1946493651","state":["exists","up"]},{"osd":10,"uuid":"1fd60ccb-92db-45f0-a580-2003db0976e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6825","nonce":1234615022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6827","nonce":1234615022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6831","nonce":1234615022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6829","nonce":1234615022}]},"public_addr":"172.21.15.155:6825/1234615022","cluster_addr":"172.21.15.155:6827/1234615022","heartbeat_back_addr":"172.21.15.155:6831/1234615022","heartbeat_front_addr":"172.21.15.155:6829/1234615022","state":["exists","up"]},{"osd":11,"uuid":"efb31ea1-4775-4e5f-8740-728d6f4aeea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6825","nonce":215104663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6827","nonce":215104663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6831","nonce":215104663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6829","nonce":215104663}]},"public_addr":"172.21.15.193:6825/215104663","cluster_addr":"172.21.15.193:6827/215104663","heartbeat_back_addr":"172.21.15.193:6831/215104663","heartbeat_front_addr":"172.21.15.193:6829/215104663","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:35:30.211698+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:35:50.883846+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:36:11.124168+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:36:32.711905+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:36:54.442007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:37:14.843526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:37:36.587354+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:37:57.882374+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:38:18.729407+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:38:40.139231+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:39:01.290862+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:39:21.767537+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:0/2902600414":"2023-07-22T18:34:07.039823+0000","172.21.15.39:6801/2893866425":"2023-07-22T18:34:07.039823+0000","172.21.15.39:0/19717002":"2023-07-22T18:34:07.039823+0000","172.21.15.39:6800/2893866425":"2023-07-22T18:34:07.039823+0000","172.21.15.39:6800/889516631":"2023-07-22T18:33:40.852706+0000","172.21.15.39:6801/889516631":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/3636510379":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/1275748831":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/3819152602":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/1127767082":"2023-07-22T18:34:07.039823+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T18:40:20.563 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:20.538+0000 7faf4448d700 1 -- 172.21.15.39:0/374246969 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7faf280a11e0 msgr2=0x7faf280a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:20.563 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:20.538+0000 7faf4448d700 1 --2- 172.21.15.39:0/374246969 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7faf280a11e0 0x7faf280a3690 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7faf30005950 tx=0x7faf300058e0 comp rx=0 tx=0).stop 2023-07-21T18:40:20.564 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:20.538+0000 7faf4448d700 1 -- 172.21.15.39:0/374246969 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf3c0feab0 msgr2=0x7faf3c196ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:20.564 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:20.538+0000 7faf4448d700 1 --2- 172.21.15.39:0/374246969 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf3c0feab0 0x7faf3c196ac0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7faf38003730 tx=0x7faf3800f740 comp rx=0 tx=0).stop 2023-07-21T18:40:20.564 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:20.538+0000 7faf4448d700 1 -- 172.21.15.39:0/374246969 shutdown_connections 2023-07-21T18:40:20.564 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:20.538+0000 7faf4448d700 1 --2- 172.21.15.39:0/374246969 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7faf280a11e0 0x7faf280a3690 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:20.564 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:20.538+0000 7faf4448d700 1 --2- 172.21.15.39:0/374246969 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf3c104ab0 0x7faf3c1a1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:20.564 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:20.538+0000 7faf4448d700 1 --2- 172.21.15.39:0/374246969 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf3c0feab0 0x7faf3c196ac0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:20.565 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:20.538+0000 7faf4448d700 1 --2- 172.21.15.39:0/374246969 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf3c0ff460 0x7faf3c197000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:20.565 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:20.538+0000 7faf4448d700 1 -- 172.21.15.39:0/374246969 >> 172.21.15.39:0/374246969 conn(0x7faf3c075b70 msgr2=0x7faf3c102670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:20.565 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:20.538+0000 7faf4448d700 1 -- 172.21.15.39:0/374246969 shutdown_connections 2023-07-21T18:40:20.565 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:20.538+0000 7faf4448d700 1 -- 172.21.15.39:0/374246969 wait complete. 2023-07-21T18:40:20.873 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:20 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/374246969' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T18:40:20.891 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:20 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/374246969' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T18:40:20.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:20 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/374246969' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T18:40:20.988 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2023-07-21T18:40:20.988 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph osd dump --format=json 2023-07-21T18:40:21.552 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:40:22.062 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:21 smithi039 ceph-mon[96489]: pgmap v256: 97 pgs: 97 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.8 KiB/s wr, 23 op/s 2023-07-21T18:40:22.373 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:21 smithi193 ceph-mon[98204]: pgmap v256: 97 pgs: 97 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.8 KiB/s wr, 23 op/s 2023-07-21T18:40:22.391 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:21 smithi155 ceph-mon[97934]: pgmap v256: 97 pgs: 97 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.8 KiB/s wr, 23 op/s 2023-07-21T18:40:22.744 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:22.718+0000 7ff3cb27f700 1 -- 172.21.15.39:0/3599701752 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3c40fee80 msgr2=0x7ff3c40ff250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:22.745 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:22.718+0000 7ff3cb27f700 1 --2- 172.21.15.39:0/3599701752 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3c40fee80 0x7ff3c40ff250 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff3c00099c0 tx=0x7ff3c0009cd0 comp rx=0 tx=0).stop 2023-07-21T18:40:22.745 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:22.718+0000 7ff3cb27f700 1 -- 172.21.15.39:0/3599701752 shutdown_connections 2023-07-21T18:40:22.745 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:22.718+0000 7ff3cb27f700 1 --2- 172.21.15.39:0/3599701752 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3c4100f80 0x7ff3c410ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:22.745 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:22.718+0000 7ff3cb27f700 1 --2- 172.21.15.39:0/3599701752 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3c40fee80 0x7ff3c40ff250 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:22.745 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:22.718+0000 7ff3cb27f700 1 --2- 172.21.15.39:0/3599701752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff3c41005d0 0x7ff3c4100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:22.745 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:22.718+0000 7ff3cb27f700 1 -- 172.21.15.39:0/3599701752 >> 172.21.15.39:0/3599701752 conn(0x7ff3c4075b60 msgr2=0x7ff3c4075f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:22.746 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:22.718+0000 7ff3cb27f700 1 -- 172.21.15.39:0/3599701752 shutdown_connections 2023-07-21T18:40:22.746 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:22.718+0000 7ff3cb27f700 1 -- 172.21.15.39:0/3599701752 wait complete. 2023-07-21T18:40:22.746 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:22.718+0000 7ff3cb27f700 1 Processor -- start 2023-07-21T18:40:22.746 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:22.719+0000 7ff3cb27f700 1 -- start start 2023-07-21T18:40:22.746 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:22.719+0000 7ff3cb27f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3c40fee80 0x7ff3c4196ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:22.746 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:22.719+0000 7ff3cb27f700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff3c41005d0 0x7ff3c41970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:22.746 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:22.719+0000 7ff3cb27f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3c4100f80 0x7ff3c41a1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:22.747 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:22.719+0000 7ff3cb27f700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c41979f0 con 0x7ff3c4100f80 2023-07-21T18:40:22.747 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:22.719+0000 7ff3cb27f700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c41a2160 con 0x7ff3c40fee80 2023-07-21T18:40:22.747 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:22.719+0000 7ff3cb27f700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c41a22d0 con 0x7ff3c41005d0 2023-07-21T18:40:22.747 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:22.719+0000 7ff3c981c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3c4100f80 0x7ff3c41a1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:22.747 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:22.719+0000 7ff3c981c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3c4100f80 0x7ff3c41a1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55312/0 (socket says 172.21.15.39:55312) 2023-07-21T18:40:22.747 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:22.719+0000 7ff3c981c700 1 -- 172.21.15.39:0/1796903384 learned_addr learned my addr 172.21.15.39:0/1796903384 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:40:22.747 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:22.719+0000 7ff3c901b700 1 --2- 172.21.15.39:0/1796903384 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3c40fee80 0x7ff3c4196ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:22.748 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:22.719+0000 7ff3c881a700 1 --2- 172.21.15.39:0/1796903384 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff3c41005d0 0x7ff3c41970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:22.748 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:22.719+0000 7ff3c981c700 1 -- 172.21.15.39:0/1796903384 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff3c41005d0 msgr2=0x7ff3c41970e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:22.748 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:22.719+0000 7ff3c981c700 1 --2- 172.21.15.39:0/1796903384 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff3c41005d0 0x7ff3c41970e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:22.748 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:22.719+0000 7ff3c981c700 1 -- 172.21.15.39:0/1796903384 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3c40fee80 msgr2=0x7ff3c4196ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:22.748 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:22.719+0000 7ff3c981c700 1 --2- 172.21.15.39:0/1796903384 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3c40fee80 0x7ff3c4196ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:22.748 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:22.719+0000 7ff3c981c700 1 -- 172.21.15.39:0/1796903384 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3c00095f0 con 0x7ff3c4100f80 2023-07-21T18:40:22.748 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:22.719+0000 7ff3c881a700 1 --2- 172.21.15.39:0/1796903384 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff3c41005d0 0x7ff3c41970e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:40:22.749 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:22.719+0000 7ff3c981c700 1 --2- 172.21.15.39:0/1796903384 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3c4100f80 0x7ff3c41a1c20 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc007d70 tx=0x7ff3bc00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:22.750 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:22.720+0000 7ff3ba7fc700 1 -- 172.21.15.39:0/1796903384 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc00ad40 con 0x7ff3c4100f80 2023-07-21T18:40:22.750 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:22.720+0000 7ff3ba7fc700 1 -- 172.21.15.39:0/1796903384 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff3bc0128e0 con 0x7ff3c4100f80 2023-07-21T18:40:22.750 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:22.720+0000 7ff3cb27f700 1 -- 172.21.15.39:0/1796903384 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3c41a25b0 con 0x7ff3c4100f80 2023-07-21T18:40:22.750 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:22.720+0000 7ff3ba7fc700 1 -- 172.21.15.39:0/1796903384 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc0116b0 con 0x7ff3c4100f80 2023-07-21T18:40:22.750 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:22.720+0000 7ff3cb27f700 1 -- 172.21.15.39:0/1796903384 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3c41a2ab0 con 0x7ff3c4100f80 2023-07-21T18:40:22.750 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:22.721+0000 7ff3cb27f700 1 -- 172.21.15.39:0/1796903384 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3c404e210 con 0x7ff3c4100f80 2023-07-21T18:40:22.751 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:22.722+0000 7ff3ba7fc700 1 -- 172.21.15.39:0/1796903384 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7ff3bc012420 con 0x7ff3c4100f80 2023-07-21T18:40:22.751 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:22.722+0000 7ff3ba7fc700 1 --2- 172.21.15.39:0/1796903384 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7ff3b00a11e0 0x7ff3b00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:22.751 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:22.723+0000 7ff3c901b700 1 --2- 172.21.15.39:0/1796903384 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7ff3b00a11e0 0x7ff3b00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:22.751 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:22.723+0000 7ff3ba7fc700 1 -- 172.21.15.39:0/1796903384 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff3bc0cce20 con 0x7ff3c4100f80 2023-07-21T18:40:22.751 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:22.723+0000 7ff3c901b700 1 --2- 172.21.15.39:0/1796903384 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7ff3b00a11e0 0x7ff3b00a3690 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0000c00 tx=0x7ff3c0005c40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:22.752 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:22.726+0000 7ff3ba7fc700 1 -- 172.21.15.39:0/1796903384 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7ff3bc0966d0 con 0x7ff3c4100f80 2023-07-21T18:40:22.947 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:22.922+0000 7ff3cb27f700 1 -- 172.21.15.39:0/1796903384 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff3c41a2d30 con 0x7ff3c4100f80 2023-07-21T18:40:22.948 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:22.923+0000 7ff3ba7fc700 1 -- 172.21.15.39:0/1796903384 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+22117 (secure 0 0 0) 0x7ff3bc099cf0 con 0x7ff3c4100f80 2023-07-21T18:40:22.949 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:40:22.949 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":79,"fsid":"eaf8e25a-27f4-11ee-9b34-001a4aab830c","created":"2023-07-21T18:33:06.504979+0000","modified":"2023-07-21T18:40:05.769300+0000","last_up_change":"2023-07-21T18:39:23.531009+0000","last_in_change":"2023-07-21T18:39:05.993024+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T18:36:13.572583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T18:40:02.726735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T18:40:03.990112+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03502c07-2d0f-4723-8d0b-d34c9f5b91bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6803","nonce":860193857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6805","nonce":860193857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6809","nonce":860193857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6807","nonce":860193857}]},"public_addr":"172.21.15.39:6803/860193857","cluster_addr":"172.21.15.39:6805/860193857","heartbeat_back_addr":"172.21.15.39:6809/860193857","heartbeat_front_addr":"172.21.15.39:6807/860193857","state":["exists","up"]},{"osd":1,"uuid":"0831a0c0-a1fa-4d30-9a28-513223aa69eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6801","nonce":3246721764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6803","nonce":3246721764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6807","nonce":3246721764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6805","nonce":3246721764}]},"public_addr":"172.21.15.155:6801/3246721764","cluster_addr":"172.21.15.155:6803/3246721764","heartbeat_back_addr":"172.21.15.155:6807/3246721764","heartbeat_front_addr":"172.21.15.155:6805/3246721764","state":["exists","up"]},{"osd":2,"uuid":"22df0003-bde9-48e4-b47a-cb8eafbab0db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6801","nonce":2563594449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6803","nonce":2563594449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6807","nonce":2563594449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6805","nonce":2563594449}]},"public_addr":"172.21.15.193:6801/2563594449","cluster_addr":"172.21.15.193:6803/2563594449","heartbeat_back_addr":"172.21.15.193:6807/2563594449","heartbeat_front_addr":"172.21.15.193:6805/2563594449","state":["exists","up"]},{"osd":3,"uuid":"54562558-a2e8-45dd-bc95-561569b9ae6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6811","nonce":863708386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6813","nonce":863708386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6817","nonce":863708386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6815","nonce":863708386}]},"public_addr":"172.21.15.39:6811/863708386","cluster_addr":"172.21.15.39:6813/863708386","heartbeat_back_addr":"172.21.15.39:6817/863708386","heartbeat_front_addr":"172.21.15.39:6815/863708386","state":["exists","up"]},{"osd":4,"uuid":"491118e7-ae1e-495c-9eac-ef1cdcae9600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6809","nonce":3361780673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6811","nonce":3361780673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6815","nonce":3361780673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6813","nonce":3361780673}]},"public_addr":"172.21.15.155:6809/3361780673","cluster_addr":"172.21.15.155:6811/3361780673","heartbeat_back_addr":"172.21.15.155:6815/3361780673","heartbeat_front_addr":"172.21.15.155:6813/3361780673","state":["exists","up"]},{"osd":5,"uuid":"d267bc2c-42b3-4d96-b2d1-b186dc7e6b18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6809","nonce":2376867721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6811","nonce":2376867721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6815","nonce":2376867721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6813","nonce":2376867721}]},"public_addr":"172.21.15.193:6809/2376867721","cluster_addr":"172.21.15.193:6811/2376867721","heartbeat_back_addr":"172.21.15.193:6815/2376867721","heartbeat_front_addr":"172.21.15.193:6813/2376867721","state":["exists","up"]},{"osd":6,"uuid":"beec8088-970a-473e-bd67-6eee7cbdef24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6819","nonce":2417770529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6821","nonce":2417770529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6825","nonce":2417770529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6823","nonce":2417770529}]},"public_addr":"172.21.15.39:6819/2417770529","cluster_addr":"172.21.15.39:6821/2417770529","heartbeat_back_addr":"172.21.15.39:6825/2417770529","heartbeat_front_addr":"172.21.15.39:6823/2417770529","state":["exists","up"]},{"osd":7,"uuid":"5a88ce30-1835-4254-90cc-804ed5cf1066","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6817","nonce":2542002111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6819","nonce":2542002111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6823","nonce":2542002111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6821","nonce":2542002111}]},"public_addr":"172.21.15.155:6817/2542002111","cluster_addr":"172.21.15.155:6819/2542002111","heartbeat_back_addr":"172.21.15.155:6823/2542002111","heartbeat_front_addr":"172.21.15.155:6821/2542002111","state":["exists","up"]},{"osd":8,"uuid":"d964a183-3021-4ace-9a0c-aad00f2570e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6817","nonce":2985835085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6819","nonce":2985835085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6823","nonce":2985835085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6821","nonce":2985835085}]},"public_addr":"172.21.15.193:6817/2985835085","cluster_addr":"172.21.15.193:6819/2985835085","heartbeat_back_addr":"172.21.15.193:6823/2985835085","heartbeat_front_addr":"172.21.15.193:6821/2985835085","state":["exists","up"]},{"osd":9,"uuid":"9725ea5c-49a3-4c99-bac4-e95069100b1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6827","nonce":1946493651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6829","nonce":1946493651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6833","nonce":1946493651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6831","nonce":1946493651}]},"public_addr":"172.21.15.39:6827/1946493651","cluster_addr":"172.21.15.39:6829/1946493651","heartbeat_back_addr":"172.21.15.39:6833/1946493651","heartbeat_front_addr":"172.21.15.39:6831/1946493651","state":["exists","up"]},{"osd":10,"uuid":"1fd60ccb-92db-45f0-a580-2003db0976e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6825","nonce":1234615022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6827","nonce":1234615022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6831","nonce":1234615022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6829","nonce":1234615022}]},"public_addr":"172.21.15.155:6825/1234615022","cluster_addr":"172.21.15.155:6827/1234615022","heartbeat_back_addr":"172.21.15.155:6831/1234615022","heartbeat_front_addr":"172.21.15.155:6829/1234615022","state":["exists","up"]},{"osd":11,"uuid":"efb31ea1-4775-4e5f-8740-728d6f4aeea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6825","nonce":215104663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6827","nonce":215104663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6831","nonce":215104663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6829","nonce":215104663}]},"public_addr":"172.21.15.193:6825/215104663","cluster_addr":"172.21.15.193:6827/215104663","heartbeat_back_addr":"172.21.15.193:6831/215104663","heartbeat_front_addr":"172.21.15.193:6829/215104663","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:35:30.211698+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:35:50.883846+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:36:11.124168+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:36:32.711905+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:36:54.442007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:37:14.843526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:37:36.587354+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:37:57.882374+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:38:18.729407+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:38:40.139231+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:39:01.290862+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:39:21.767537+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:0/2902600414":"2023-07-22T18:34:07.039823+0000","172.21.15.39:6801/2893866425":"2023-07-22T18:34:07.039823+0000","172.21.15.39:0/19717002":"2023-07-22T18:34:07.039823+0000","172.21.15.39:6800/2893866425":"2023-07-22T18:34:07.039823+0000","172.21.15.39:6800/889516631":"2023-07-22T18:33:40.852706+0000","172.21.15.39:6801/889516631":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/3636510379":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/1275748831":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/3819152602":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/1127767082":"2023-07-22T18:34:07.039823+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T18:40:22.950 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:22.925+0000 7ff3cb27f700 1 -- 172.21.15.39:0/1796903384 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7ff3b00a11e0 msgr2=0x7ff3b00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:22.951 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:22.925+0000 7ff3cb27f700 1 --2- 172.21.15.39:0/1796903384 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7ff3b00a11e0 0x7ff3b00a3690 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0000c00 tx=0x7ff3c0005c40 comp rx=0 tx=0).stop 2023-07-21T18:40:22.951 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:22.925+0000 7ff3cb27f700 1 -- 172.21.15.39:0/1796903384 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3c4100f80 msgr2=0x7ff3c41a1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:22.951 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:22.925+0000 7ff3cb27f700 1 --2- 172.21.15.39:0/1796903384 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3c4100f80 0x7ff3c41a1c20 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc007d70 tx=0x7ff3bc00a340 comp rx=0 tx=0).stop 2023-07-21T18:40:22.951 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:22.925+0000 7ff3cb27f700 1 -- 172.21.15.39:0/1796903384 shutdown_connections 2023-07-21T18:40:22.951 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:22.925+0000 7ff3cb27f700 1 --2- 172.21.15.39:0/1796903384 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7ff3b00a11e0 0x7ff3b00a3690 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:22.952 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:22.925+0000 7ff3cb27f700 1 --2- 172.21.15.39:0/1796903384 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3c40fee80 0x7ff3c4196ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:22.952 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:22.925+0000 7ff3cb27f700 1 --2- 172.21.15.39:0/1796903384 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3c4100f80 0x7ff3c41a1c20 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:22.952 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:22.925+0000 7ff3cb27f700 1 --2- 172.21.15.39:0/1796903384 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff3c41005d0 0x7ff3c41970e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:22.952 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:22.925+0000 7ff3cb27f700 1 -- 172.21.15.39:0/1796903384 >> 172.21.15.39:0/1796903384 conn(0x7ff3c4075b60 msgr2=0x7ff3c40fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:22.952 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:22.925+0000 7ff3cb27f700 1 -- 172.21.15.39:0/1796903384 shutdown_connections 2023-07-21T18:40:22.952 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:22.925+0000 7ff3cb27f700 1 -- 172.21.15.39:0/1796903384 wait complete. 2023-07-21T18:40:23.331 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:22 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/1796903384' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T18:40:23.373 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:22 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/1796903384' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T18:40:23.391 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:22 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/1796903384' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T18:40:23.410 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph tell osd.0 flush_pg_stats 2023-07-21T18:40:23.411 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph tell osd.1 flush_pg_stats 2023-07-21T18:40:23.412 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph tell osd.2 flush_pg_stats 2023-07-21T18:40:23.412 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph tell osd.3 flush_pg_stats 2023-07-21T18:40:23.413 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph tell osd.4 flush_pg_stats 2023-07-21T18:40:23.413 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph tell osd.5 flush_pg_stats 2023-07-21T18:40:23.414 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph tell osd.6 flush_pg_stats 2023-07-21T18:40:23.414 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph tell osd.7 flush_pg_stats 2023-07-21T18:40:23.415 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph tell osd.8 flush_pg_stats 2023-07-21T18:40:23.416 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph tell osd.9 flush_pg_stats 2023-07-21T18:40:23.416 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph tell osd.10 flush_pg_stats 2023-07-21T18:40:23.417 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph tell osd.11 flush_pg_stats 2023-07-21T18:40:24.373 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:23 smithi193 ceph-mon[98204]: pgmap v257: 97 pgs: 97 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.9 KiB/s wr, 19 op/s 2023-07-21T18:40:24.391 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:23 smithi155 ceph-mon[97934]: pgmap v257: 97 pgs: 97 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.9 KiB/s wr, 19 op/s 2023-07-21T18:40:24.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:23 smithi039 ceph-mon[96489]: pgmap v257: 97 pgs: 97 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.9 KiB/s wr, 19 op/s 2023-07-21T18:40:24.524 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:40:24.833 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:40:24.939 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:40:24.940 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:40:25.402 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:40:25.405 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:40:25.409 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:25 smithi039 ceph-mon[96489]: pgmap v258: 97 pgs: 97 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.9 KiB/s wr, 17 op/s 2023-07-21T18:40:25.423 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:40:25.462 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:40:25.623 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:25 smithi193 ceph-mon[98204]: pgmap v258: 97 pgs: 97 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.9 KiB/s wr, 17 op/s 2023-07-21T18:40:25.641 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:40:25.642 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:25 smithi155 ceph-mon[97934]: pgmap v258: 97 pgs: 97 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.9 KiB/s wr, 17 op/s 2023-07-21T18:40:26.054 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:40:26.236 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:40:26.260 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:40:27.513 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:27 smithi039 ceph-mon[96489]: pgmap v259: 97 pgs: 97 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2023-07-21T18:40:27.623 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:27 smithi193 ceph-mon[98204]: pgmap v259: 97 pgs: 97 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2023-07-21T18:40:27.641 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:27 smithi155 ceph-mon[97934]: pgmap v259: 97 pgs: 97 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2023-07-21T18:40:28.951 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:28.880+0000 7fb7a8468700 1 -- 172.21.15.39:0/483706150 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb7a0104ab0 msgr2=0x7fb7a0104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:28.951 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:28.880+0000 7fb7a8468700 1 --2- 172.21.15.39:0/483706150 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb7a0104ab0 0x7fb7a0104e80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb79c009990 tx=0x7fb79c009ca0 comp rx=0 tx=0).stop 2023-07-21T18:40:28.952 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:28.880+0000 7fb7a8468700 1 -- 172.21.15.39:0/483706150 shutdown_connections 2023-07-21T18:40:28.952 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:28.880+0000 7fb7a8468700 1 --2- 172.21.15.39:0/483706150 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7a00ff460 0x7fb7a010b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:28.952 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:28.880+0000 7fb7a8468700 1 --2- 172.21.15.39:0/483706150 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb7a0104ab0 0x7fb7a0104e80 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:28.952 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:28.880+0000 7fb7a8468700 1 --2- 172.21.15.39:0/483706150 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7a00feab0 0x7fb7a00fef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:28.952 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:28.880+0000 7fb7a8468700 1 -- 172.21.15.39:0/483706150 >> 172.21.15.39:0/483706150 conn(0x7fb7a0075b70 msgr2=0x7fb7a0075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:28.952 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:28.880+0000 7fb7a8468700 1 -- 172.21.15.39:0/483706150 shutdown_connections 2023-07-21T18:40:28.953 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:28.880+0000 7fb7a8468700 1 -- 172.21.15.39:0/483706150 wait complete. 2023-07-21T18:40:28.953 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:28.881+0000 7fb7a8468700 1 Processor -- start 2023-07-21T18:40:28.953 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:28.881+0000 7fb7a8468700 1 -- start start 2023-07-21T18:40:28.953 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:28.881+0000 7fb7a8468700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb7a00feab0 0x7fb7a019a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:28.953 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:28.881+0000 7fb7a8468700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7a00ff460 0x7fb7a019ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:28.953 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:28.881+0000 7fb7a8468700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7a0104ab0 0x7fb7a0194940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:28.953 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:28.881+0000 7fb7a8468700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a0194fb0 con 0x7fb7a00feab0 2023-07-21T18:40:28.954 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:28.881+0000 7fb7a8468700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a0195120 con 0x7fb7a00ff460 2023-07-21T18:40:28.954 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:28.881+0000 7fb7a8468700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a01952e0 con 0x7fb7a0104ab0 2023-07-21T18:40:28.954 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:28.881+0000 7fb7a6204700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb7a00feab0 0x7fb7a019a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:28.954 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:28.881+0000 7fb7a6204700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb7a00feab0 0x7fb7a019a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55320/0 (socket says 172.21.15.39:55320) 2023-07-21T18:40:28.954 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:28.881+0000 7fb7a6204700 1 -- 172.21.15.39:0/14625498 learned_addr learned my addr 172.21.15.39:0/14625498 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:40:28.954 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:28.881+0000 7fb7a5a03700 1 --2- 172.21.15.39:0/14625498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7a00ff460 0x7fb7a019ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:28.954 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:28.881+0000 7fb7a6a05700 1 --2- 172.21.15.39:0/14625498 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7a0104ab0 0x7fb7a0194940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:28.955 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:28.882+0000 7fb7a6204700 1 -- 172.21.15.39:0/14625498 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7a0104ab0 msgr2=0x7fb7a0194940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:28.955 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:28.882+0000 7fb7a6204700 1 --2- 172.21.15.39:0/14625498 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7a0104ab0 0x7fb7a0194940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:28.955 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:28.882+0000 7fb7a6204700 1 -- 172.21.15.39:0/14625498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7a00ff460 msgr2=0x7fb7a019ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:28.955 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:28.882+0000 7fb7a6204700 1 --2- 172.21.15.39:0/14625498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7a00ff460 0x7fb7a019ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:28.955 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:28.882+0000 7fb7a6204700 1 -- 172.21.15.39:0/14625498 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb79c0095f0 con 0x7fb7a00feab0 2023-07-21T18:40:28.955 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:28.882+0000 7fb7a5a03700 1 --2- 172.21.15.39:0/14625498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7a00ff460 0x7fb7a019ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:40:28.955 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:28.882+0000 7fb7a6204700 1 --2- 172.21.15.39:0/14625498 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb7a00feab0 0x7fb7a019a8c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb79c00b2e0 tx=0x7fb79c005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:28.956 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:28.882+0000 7fb7977fe700 1 -- 172.21.15.39:0/14625498 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79c01d070 con 0x7fb7a00feab0 2023-07-21T18:40:28.956 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:28.882+0000 7fb7a8468700 1 -- 172.21.15.39:0/14625498 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7a0195560 con 0x7fb7a00feab0 2023-07-21T18:40:28.956 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:28.882+0000 7fb7977fe700 1 -- 172.21.15.39:0/14625498 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb79c0047e0 con 0x7fb7a00feab0 2023-07-21T18:40:28.956 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:28.882+0000 7fb7977fe700 1 -- 172.21.15.39:0/14625498 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79c005470 con 0x7fb7a00feab0 2023-07-21T18:40:28.956 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:28.882+0000 7fb7a8468700 1 -- 172.21.15.39:0/14625498 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7a01a18b0 con 0x7fb7a00feab0 2023-07-21T18:40:28.956 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:28.883+0000 7fb7a8468700 1 -- 172.21.15.39:0/14625498 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb7a004e210 con 0x7fb7a00feab0 2023-07-21T18:40:28.956 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:28.884+0000 7fb7977fe700 1 -- 172.21.15.39:0/14625498 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fb79c0055d0 con 0x7fb7a00feab0 2023-07-21T18:40:28.957 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:28.885+0000 7fb7977fe700 1 --2- 172.21.15.39:0/14625498 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fb78c0a11e0 0x7fb78c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:28.957 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:28.885+0000 7fb7a5a03700 1 --2- 172.21.15.39:0/14625498 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fb78c0a11e0 0x7fb78c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:28.957 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:28.885+0000 7fb7977fe700 1 -- 172.21.15.39:0/14625498 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb79c0ce120 con 0x7fb7a00feab0 2023-07-21T18:40:28.957 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:28.885+0000 7fb7977fe700 1 --2- 172.21.15.39:0/14625498 >> [v2:172.21.15.193:6800/2563594449,v1:172.21.15.193:6801/2563594449] conn(0x7fb78c0aa9b0 0x7fb78c0acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:28.958 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:28.885+0000 7fb7977fe700 1 -- 172.21.15.39:0/14625498 --> [v2:172.21.15.193:6800/2563594449,v1:172.21.15.193:6801/2563594449] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb78c0ad470 con 0x7fb78c0aa9b0 2023-07-21T18:40:28.958 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:28.885+0000 7fb7977fe700 1 -- 172.21.15.39:0/14625498 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb79c0ce5c0 con 0x7fb7a00feab0 2023-07-21T18:40:28.958 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:28.885+0000 7fb7a5a03700 1 --2- 172.21.15.39:0/14625498 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fb78c0a11e0 0x7fb78c0a3690 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb790005950 tx=0x7fb7900058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:28.958 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:28.885+0000 7fb7a6a05700 1 --2- 172.21.15.39:0/14625498 >> [v2:172.21.15.193:6800/2563594449,v1:172.21.15.193:6801/2563594449] conn(0x7fb78c0aa9b0 0x7fb78c0acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:28.958 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:28.886+0000 7fb7a6a05700 1 --2- 172.21.15.39:0/14625498 >> [v2:172.21.15.193:6800/2563594449,v1:172.21.15.193:6801/2563594449] conn(0x7fb78c0aa9b0 0x7fb78c0acdc0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:28.958 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:28.887+0000 7fb7977fe700 1 -- 172.21.15.39:0/14625498 <== osd.2 v2:172.21.15.193:6800/2563594449 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fb78c0ad470 con 0x7fb78c0aa9b0 2023-07-21T18:40:28.959 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:28.902+0000 7fb7a8468700 1 -- 172.21.15.39:0/14625498 --> [v2:172.21.15.193:6800/2563594449,v1:172.21.15.193:6801/2563594449] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb7a00fd7e0 con 0x7fb78c0aa9b0 2023-07-21T18:40:28.959 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:28.903+0000 7fb7977fe700 1 -- 172.21.15.39:0/14625498 <== osd.2 v2:172.21.15.193:6800/2563594449 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb7a00fd7e0 con 0x7fb78c0aa9b0 2023-07-21T18:40:28.959 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:28.903+0000 7fb7a8468700 1 -- 172.21.15.39:0/14625498 >> [v2:172.21.15.193:6800/2563594449,v1:172.21.15.193:6801/2563594449] conn(0x7fb78c0aa9b0 msgr2=0x7fb78c0acdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:28.959 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:28.903+0000 7fb7a8468700 1 --2- 172.21.15.39:0/14625498 >> [v2:172.21.15.193:6800/2563594449,v1:172.21.15.193:6801/2563594449] conn(0x7fb78c0aa9b0 0x7fb78c0acdc0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:28.959 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:28.903+0000 7fb7a8468700 1 -- 172.21.15.39:0/14625498 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fb78c0a11e0 msgr2=0x7fb78c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:28.959 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:28.903+0000 7fb7a8468700 1 --2- 172.21.15.39:0/14625498 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fb78c0a11e0 0x7fb78c0a3690 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb790005950 tx=0x7fb7900058e0 comp rx=0 tx=0).stop 2023-07-21T18:40:28.959 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:28.903+0000 7fb7a8468700 1 -- 172.21.15.39:0/14625498 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb7a00feab0 msgr2=0x7fb7a019a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:28.960 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:28.903+0000 7fb7a8468700 1 --2- 172.21.15.39:0/14625498 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb7a00feab0 0x7fb7a019a8c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb79c00b2e0 tx=0x7fb79c005ef0 comp rx=0 tx=0).stop 2023-07-21T18:40:28.960 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:28.903+0000 7fb7a6a05700 1 -- 172.21.15.39:0/14625498 reap_dead start 2023-07-21T18:40:28.960 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:28.903+0000 7fb7a8468700 1 -- 172.21.15.39:0/14625498 shutdown_connections 2023-07-21T18:40:28.960 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:28.903+0000 7fb7a8468700 1 -- 172.21.15.39:0/14625498 >> 172.21.15.39:0/14625498 conn(0x7fb7a0075b70 msgr2=0x7fb7a00fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:28.960 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:28.903+0000 7fb7a8468700 1 -- 172.21.15.39:0/14625498 shutdown_connections 2023-07-21T18:40:28.960 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:28.903+0000 7fb7a8468700 1 -- 172.21.15.39:0/14625498 wait complete. 2023-07-21T18:40:29.504 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:29 smithi039 ceph-mon[96489]: pgmap v260: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2023-07-21T18:40:29.584 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:29.338+0000 7f359e158700 1 -- 172.21.15.39:0/1558465555 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35981005d0 msgr2=0x7f3598100a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:29.585 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:29.338+0000 7f359e158700 1 --2- 172.21.15.39:0/1558465555 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35981005d0 0x7f3598100a40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f358c0099c0 tx=0x7f358c009cd0 comp rx=0 tx=0).stop 2023-07-21T18:40:29.585 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:29.338+0000 7f359e158700 1 -- 172.21.15.39:0/1558465555 shutdown_connections 2023-07-21T18:40:29.585 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:29.338+0000 7f359e158700 1 --2- 172.21.15.39:0/1558465555 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3598100f80 0x7f359810ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:29.585 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:29.338+0000 7f359e158700 1 --2- 172.21.15.39:0/1558465555 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35980fee80 0x7f35980ff250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:29.585 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:29.338+0000 7f359e158700 1 --2- 172.21.15.39:0/1558465555 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35981005d0 0x7f3598100a40 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:29.586 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:29.338+0000 7f359e158700 1 -- 172.21.15.39:0/1558465555 >> 172.21.15.39:0/1558465555 conn(0x7f3598075b60 msgr2=0x7f3598075f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:29.586 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:29.338+0000 7f359e158700 1 -- 172.21.15.39:0/1558465555 shutdown_connections 2023-07-21T18:40:29.586 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:29.338+0000 7f359e158700 1 -- 172.21.15.39:0/1558465555 wait complete. 2023-07-21T18:40:29.586 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:29.338+0000 7f359e158700 1 Processor -- start 2023-07-21T18:40:29.586 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:29.339+0000 7f359e158700 1 -- start start 2023-07-21T18:40:29.586 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:29.339+0000 7f359e158700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35980fee80 0x7f3598105a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:29.586 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:29.339+0000 7f359e158700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35981005d0 0x7f3598105f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:29.587 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:29.339+0000 7f359e158700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3598100f80 0x7f35981064e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:29.587 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:29.339+0000 7f359e158700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35981a41b0 con 0x7f35980fee80 2023-07-21T18:40:29.587 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:29.339+0000 7f359e158700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35981a4320 con 0x7f3598100f80 2023-07-21T18:40:29.587 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:29.339+0000 7f359e158700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35981a44e0 con 0x7f35981005d0 2023-07-21T18:40:29.587 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:29.339+0000 7f35977fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35980fee80 0x7f3598105a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:29.587 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:29.339+0000 7f35977fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35980fee80 0x7f3598105a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55336/0 (socket says 172.21.15.39:55336) 2023-07-21T18:40:29.587 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:29.339+0000 7f35977fe700 1 -- 172.21.15.39:0/653749069 learned_addr learned my addr 172.21.15.39:0/653749069 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:40:29.588 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:29.339+0000 7f3596ffd700 1 --2- 172.21.15.39:0/653749069 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35981005d0 0x7f3598105f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:29.588 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:29.339+0000 7f3597fff700 1 --2- 172.21.15.39:0/653749069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3598100f80 0x7f35981064e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:29.588 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:29.339+0000 7f35977fe700 1 -- 172.21.15.39:0/653749069 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35981005d0 msgr2=0x7f3598105f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:29.588 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:29.339+0000 7f35977fe700 1 --2- 172.21.15.39:0/653749069 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35981005d0 0x7f3598105f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:29.588 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:29.339+0000 7f35977fe700 1 -- 172.21.15.39:0/653749069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3598100f80 msgr2=0x7f35981064e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:29.589 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:29.339+0000 7f35977fe700 1 --2- 172.21.15.39:0/653749069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3598100f80 0x7f35981064e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:29.589 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:29.339+0000 7f35977fe700 1 -- 172.21.15.39:0/653749069 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f358c0095f0 con 0x7f35980fee80 2023-07-21T18:40:29.589 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:29.339+0000 7f3596ffd700 1 --2- 172.21.15.39:0/653749069 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35981005d0 0x7f3598105f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:40:29.589 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:29.339+0000 7f3597fff700 1 --2- 172.21.15.39:0/653749069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3598100f80 0x7f35981064e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:40:29.589 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:29.339+0000 7f35977fe700 1 --2- 172.21.15.39:0/653749069 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35980fee80 0x7f3598105a20 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f35880060b0 tx=0x7f3588007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:29.590 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:29.340+0000 7f3594ff9700 1 -- 172.21.15.39:0/653749069 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358800de20 con 0x7f35980fee80 2023-07-21T18:40:29.590 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:29.340+0000 7f359e158700 1 -- 172.21.15.39:0/653749069 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35981a47c0 con 0x7f35980fee80 2023-07-21T18:40:29.590 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:29.340+0000 7f3594ff9700 1 -- 172.21.15.39:0/653749069 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f358800f040 con 0x7f35980fee80 2023-07-21T18:40:29.590 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:29.340+0000 7f3594ff9700 1 -- 172.21.15.39:0/653749069 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35880146e0 con 0x7f35980fee80 2023-07-21T18:40:29.590 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:29.340+0000 7f359e158700 1 -- 172.21.15.39:0/653749069 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35981a4d10 con 0x7f35980fee80 2023-07-21T18:40:29.590 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:29.341+0000 7f359e158700 1 -- 172.21.15.39:0/653749069 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f359810edb0 con 0x7f35980fee80 2023-07-21T18:40:29.591 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:29.342+0000 7f3594ff9700 1 -- 172.21.15.39:0/653749069 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f3588015a30 con 0x7f35980fee80 2023-07-21T18:40:29.591 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:29.342+0000 7f3594ff9700 1 --2- 172.21.15.39:0/653749069 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f35800a11e0 0x7f35800a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:29.591 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:29.343+0000 7f3596ffd700 1 --2- 172.21.15.39:0/653749069 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f35800a11e0 0x7f35800a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:29.591 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:29.343+0000 7f3594ff9700 1 -- 172.21.15.39:0/653749069 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f35880ccc20 con 0x7f35980fee80 2023-07-21T18:40:29.592 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:29.343+0000 7f3594ff9700 1 --2- 172.21.15.39:0/653749069 >> [v2:172.21.15.39:6802/860193857,v1:172.21.15.39:6803/860193857] conn(0x7f35800aa9b0 0x7f35800acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:29.592 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:29.343+0000 7f3594ff9700 1 -- 172.21.15.39:0/653749069 --> [v2:172.21.15.39:6802/860193857,v1:172.21.15.39:6803/860193857] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f35800ad470 con 0x7f35800aa9b0 2023-07-21T18:40:29.592 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:29.343+0000 7f3594ff9700 1 -- 172.21.15.39:0/653749069 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35880ccf90 con 0x7f35980fee80 2023-07-21T18:40:29.592 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:29.343+0000 7f3597fff700 1 --2- 172.21.15.39:0/653749069 >> [v2:172.21.15.39:6802/860193857,v1:172.21.15.39:6803/860193857] conn(0x7f35800aa9b0 0x7f35800acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:29.592 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:29.343+0000 7f3596ffd700 1 --2- 172.21.15.39:0/653749069 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f35800a11e0 0x7f35800a3690 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f358c006010 tx=0x7f358c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:29.592 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:29.343+0000 7f3597fff700 1 --2- 172.21.15.39:0/653749069 >> [v2:172.21.15.39:6802/860193857,v1:172.21.15.39:6803/860193857] conn(0x7f35800aa9b0 0x7f35800acdc0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:29.593 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:29.344+0000 7f3594ff9700 1 -- 172.21.15.39:0/653749069 <== osd.0 v2:172.21.15.39:6802/860193857 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f35800ad470 con 0x7f35800aa9b0 2023-07-21T18:40:29.593 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:29.361+0000 7f359e158700 1 -- 172.21.15.39:0/653749069 --> [v2:172.21.15.39:6802/860193857,v1:172.21.15.39:6803/860193857] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f359804e210 con 0x7f35800aa9b0 2023-07-21T18:40:29.593 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:29.362+0000 7f3594ff9700 1 -- 172.21.15.39:0/653749069 <== osd.0 v2:172.21.15.39:6802/860193857 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f359804e210 con 0x7f35800aa9b0 2023-07-21T18:40:29.593 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:29.362+0000 7f359e158700 1 -- 172.21.15.39:0/653749069 >> [v2:172.21.15.39:6802/860193857,v1:172.21.15.39:6803/860193857] conn(0x7f35800aa9b0 msgr2=0x7f35800acdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:29.593 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:29.362+0000 7f359e158700 1 --2- 172.21.15.39:0/653749069 >> [v2:172.21.15.39:6802/860193857,v1:172.21.15.39:6803/860193857] conn(0x7f35800aa9b0 0x7f35800acdc0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:29.593 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:29.362+0000 7f359e158700 1 -- 172.21.15.39:0/653749069 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f35800a11e0 msgr2=0x7f35800a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:29.593 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:29.362+0000 7f359e158700 1 --2- 172.21.15.39:0/653749069 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f35800a11e0 0x7f35800a3690 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f358c006010 tx=0x7f358c000bc0 comp rx=0 tx=0).stop 2023-07-21T18:40:29.594 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:29.362+0000 7f359e158700 1 -- 172.21.15.39:0/653749069 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35980fee80 msgr2=0x7f3598105a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:29.594 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:29.362+0000 7f359e158700 1 --2- 172.21.15.39:0/653749069 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35980fee80 0x7f3598105a20 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f35880060b0 tx=0x7f3588007f90 comp rx=0 tx=0).stop 2023-07-21T18:40:29.594 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:29.362+0000 7f3597fff700 1 -- 172.21.15.39:0/653749069 reap_dead start 2023-07-21T18:40:29.594 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:29.362+0000 7f359e158700 1 -- 172.21.15.39:0/653749069 shutdown_connections 2023-07-21T18:40:29.594 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:29.362+0000 7f359e158700 1 -- 172.21.15.39:0/653749069 >> 172.21.15.39:0/653749069 conn(0x7f3598075b60 msgr2=0x7f35980688a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:29.594 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:29.362+0000 7f359e158700 1 -- 172.21.15.39:0/653749069 shutdown_connections 2023-07-21T18:40:29.594 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:29.362+0000 7f359e158700 1 -- 172.21.15.39:0/653749069 wait complete. 2023-07-21T18:40:29.622 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:29 smithi193 ceph-mon[98204]: pgmap v260: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2023-07-21T18:40:29.641 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:29 smithi155 ceph-mon[97934]: pgmap v260: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2023-07-21T18:40:31.212 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:31.187+0000 7f450e3e1700 1 -- 172.21.15.39:0/1495391669 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4508104ea0 msgr2=0x7f450810ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:31.212 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:31.187+0000 7f450e3e1700 1 --2- 172.21.15.39:0/1495391669 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4508104ea0 0x7f450810ca20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f450000b0d0 tx=0x7f450000b3e0 comp rx=0 tx=0).stop 2023-07-21T18:40:31.212 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:31.187+0000 7f450e3e1700 1 -- 172.21.15.39:0/1495391669 shutdown_connections 2023-07-21T18:40:31.213 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:31.187+0000 7f450e3e1700 1 --2- 172.21.15.39:0/1495391669 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4508104ea0 0x7f450810ca20 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:31.213 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:31.187+0000 7f450e3e1700 1 --2- 172.21.15.39:0/1495391669 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45080ffe00 0x7f45081001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:31.213 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:31.187+0000 7f450e3e1700 1 --2- 172.21.15.39:0/1495391669 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45081007a0 0x7f4508104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:31.213 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:31.187+0000 7f450e3e1700 1 -- 172.21.15.39:0/1495391669 >> 172.21.15.39:0/1495391669 conn(0x7f45080fb660 msgr2=0x7f45080fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:31.213 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:31.187+0000 7f450e3e1700 1 -- 172.21.15.39:0/1495391669 shutdown_connections 2023-07-21T18:40:31.213 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:31.187+0000 7f450e3e1700 1 -- 172.21.15.39:0/1495391669 wait complete. 2023-07-21T18:40:31.215 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:31.188+0000 7f450e3e1700 1 Processor -- start 2023-07-21T18:40:31.215 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:31.188+0000 7f450e3e1700 1 -- start start 2023-07-21T18:40:31.216 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:31.188+0000 7f450e3e1700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45080ffe00 0x7f4508195d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:31.216 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:31.188+0000 7f450e3e1700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45081007a0 0x7f4508196270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:31.216 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:31.188+0000 7f450e3e1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4508104ea0 0x7f450819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:31.216 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:31.188+0000 7f450e3e1700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f450819ac80 con 0x7f4508104ea0 2023-07-21T18:40:31.217 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:31.188+0000 7f450e3e1700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f450819adf0 con 0x7f45081007a0 2023-07-21T18:40:31.217 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:31.188+0000 7f450e3e1700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f450819afb0 con 0x7f45080ffe00 2023-07-21T18:40:31.217 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:31.188+0000 7f450c97e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4508104ea0 0x7f450819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:31.217 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:31.188+0000 7f450c97e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4508104ea0 0x7f450819a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55354/0 (socket says 172.21.15.39:55354) 2023-07-21T18:40:31.217 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:31.188+0000 7f450c97e700 1 -- 172.21.15.39:0/2757452693 learned_addr learned my addr 172.21.15.39:0/2757452693 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:40:31.218 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:31.188+0000 7f45077fe700 1 --2- 172.21.15.39:0/2757452693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45081007a0 0x7f4508196270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:31.218 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:31.188+0000 7f4507fff700 1 --2- 172.21.15.39:0/2757452693 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45080ffe00 0x7f4508195d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:31.218 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:31.189+0000 7f450c97e700 1 -- 172.21.15.39:0/2757452693 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45080ffe00 msgr2=0x7f4508195d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:31.218 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:31.189+0000 7f450c97e700 1 --2- 172.21.15.39:0/2757452693 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45080ffe00 0x7f4508195d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:31.218 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:31.189+0000 7f450c97e700 1 -- 172.21.15.39:0/2757452693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45081007a0 msgr2=0x7f4508196270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:31.218 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:31.189+0000 7f450c97e700 1 --2- 172.21.15.39:0/2757452693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45081007a0 0x7f4508196270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:31.218 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:31.189+0000 7f450c97e700 1 -- 172.21.15.39:0/2757452693 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4500009d00 con 0x7f4508104ea0 2023-07-21T18:40:31.219 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:31.189+0000 7f450c97e700 1 --2- 172.21.15.39:0/2757452693 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4508104ea0 0x7f450819a670 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4500007870 tx=0x7f4500007950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:31.219 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:31.189+0000 7f45057fa700 1 -- 172.21.15.39:0/2757452693 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f450000e040 con 0x7f4508104ea0 2023-07-21T18:40:31.219 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:31.189+0000 7f45057fa700 1 -- 172.21.15.39:0/2757452693 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4500003e10 con 0x7f4508104ea0 2023-07-21T18:40:31.219 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:31.189+0000 7f450e3e1700 1 -- 172.21.15.39:0/2757452693 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f450819b230 con 0x7f4508104ea0 2023-07-21T18:40:31.219 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:31.189+0000 7f45057fa700 1 -- 172.21.15.39:0/2757452693 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4500012af0 con 0x7f4508104ea0 2023-07-21T18:40:31.219 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:31.189+0000 7f450e3e1700 1 -- 172.21.15.39:0/2757452693 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45081a2920 con 0x7f4508104ea0 2023-07-21T18:40:31.219 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:31.190+0000 7f450e3e1700 1 -- 172.21.15.39:0/2757452693 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f450804e210 con 0x7f4508104ea0 2023-07-21T18:40:31.221 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:31.191+0000 7f45057fa700 1 -- 172.21.15.39:0/2757452693 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f45000056c0 con 0x7f4508104ea0 2023-07-21T18:40:31.221 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:31.191+0000 7f45057fa700 1 --2- 172.21.15.39:0/2757452693 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f44f00a11e0 0x7f44f00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:31.221 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:31.192+0000 7f4507fff700 1 --2- 172.21.15.39:0/2757452693 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f44f00a11e0 0x7f44f00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:31.221 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:31.192+0000 7f45057fa700 1 -- 172.21.15.39:0/2757452693 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f45000ce4e0 con 0x7f4508104ea0 2023-07-21T18:40:31.221 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:31.192+0000 7f45057fa700 1 --2- 172.21.15.39:0/2757452693 >> [v2:172.21.15.155:6824/1234615022,v1:172.21.15.155:6825/1234615022] conn(0x7f44f00aa880 0x7f44f00acc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:31.221 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:31.192+0000 7f45057fa700 1 -- 172.21.15.39:0/2757452693 --> [v2:172.21.15.155:6824/1234615022,v1:172.21.15.155:6825/1234615022] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f44f00ad340 con 0x7f44f00aa880 2023-07-21T18:40:31.222 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:31.192+0000 7f45057fa700 1 -- 172.21.15.39:0/2757452693 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45000ce980 con 0x7f4508104ea0 2023-07-21T18:40:31.222 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:31.192+0000 7f4507fff700 1 --2- 172.21.15.39:0/2757452693 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f44f00a11e0 0x7f44f00a3690 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f44f8005fd0 tx=0x7f44f800a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:31.222 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:31.192+0000 7f45077fe700 1 --2- 172.21.15.39:0/2757452693 >> [v2:172.21.15.155:6824/1234615022,v1:172.21.15.155:6825/1234615022] conn(0x7f44f00aa880 0x7f44f00acc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:31.222 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:31.193+0000 7f45077fe700 1 --2- 172.21.15.39:0/2757452693 >> [v2:172.21.15.155:6824/1234615022,v1:172.21.15.155:6825/1234615022] conn(0x7f44f00aa880 0x7f44f00acc90 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:31.222 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:31.194+0000 7f45057fa700 1 -- 172.21.15.39:0/2757452693 <== osd.10 v2:172.21.15.155:6824/1234615022 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f44f00ad340 con 0x7f44f00aa880 2023-07-21T18:40:31.234 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:31.209+0000 7f450e3e1700 1 -- 172.21.15.39:0/2757452693 --> [v2:172.21.15.155:6824/1234615022,v1:172.21.15.155:6825/1234615022] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f450810fd40 con 0x7f44f00aa880 2023-07-21T18:40:31.235 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:31.210+0000 7f45057fa700 1 -- 172.21.15.39:0/2757452693 <== osd.10 v2:172.21.15.155:6824/1234615022 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f44fc00d710 con 0x7f44f00aa880 2023-07-21T18:40:31.235 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:31.210+0000 7f450e3e1700 1 -- 172.21.15.39:0/2757452693 >> [v2:172.21.15.155:6824/1234615022,v1:172.21.15.155:6825/1234615022] conn(0x7f44f00aa880 msgr2=0x7f44f00acc90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:31.236 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:31.210+0000 7f450e3e1700 1 --2- 172.21.15.39:0/2757452693 >> [v2:172.21.15.155:6824/1234615022,v1:172.21.15.155:6825/1234615022] conn(0x7f44f00aa880 0x7f44f00acc90 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:31.236 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:31.210+0000 7f450e3e1700 1 -- 172.21.15.39:0/2757452693 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f44f00a11e0 msgr2=0x7f44f00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:31.236 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:31.210+0000 7f450e3e1700 1 --2- 172.21.15.39:0/2757452693 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f44f00a11e0 0x7f44f00a3690 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f44f8005fd0 tx=0x7f44f800a560 comp rx=0 tx=0).stop 2023-07-21T18:40:31.236 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:31.210+0000 7f450e3e1700 1 -- 172.21.15.39:0/2757452693 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4508104ea0 msgr2=0x7f450819a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:31.236 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:31.210+0000 7f450e3e1700 1 --2- 172.21.15.39:0/2757452693 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4508104ea0 0x7f450819a670 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4500007870 tx=0x7f4500007950 comp rx=0 tx=0).stop 2023-07-21T18:40:31.236 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:31.210+0000 7f450c97e700 1 -- 172.21.15.39:0/2757452693 reap_dead start 2023-07-21T18:40:31.237 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:31.210+0000 7f450e3e1700 1 -- 172.21.15.39:0/2757452693 shutdown_connections 2023-07-21T18:40:31.237 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:31.210+0000 7f450e3e1700 1 -- 172.21.15.39:0/2757452693 >> 172.21.15.39:0/2757452693 conn(0x7f45080fb660 msgr2=0x7f45080fc030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:31.237 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:31.211+0000 7f450e3e1700 1 -- 172.21.15.39:0/2757452693 shutdown_connections 2023-07-21T18:40:31.237 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:31.211+0000 7f450e3e1700 1 -- 172.21.15.39:0/2757452693 wait complete. 2023-07-21T18:40:31.471 INFO:teuthology.orchestra.run.smithi039.stdout:77309411382 2023-07-21T18:40:31.472 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.2 2023-07-21T18:40:31.855 INFO:teuthology.orchestra.run.smithi039.stdout:34359738431 2023-07-21T18:40:31.856 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.0 2023-07-21T18:40:32.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:31 smithi039 ceph-mon[96489]: pgmap v261: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 9 op/s 2023-07-21T18:40:32.335 INFO:teuthology.orchestra.run.smithi039.stdout:287762808852 2023-07-21T18:40:32.335 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.10 2023-07-21T18:40:32.372 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:31 smithi193 ceph-mon[98204]: pgmap v261: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 9 op/s 2023-07-21T18:40:32.391 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:31 smithi155 ceph-mon[97934]: pgmap v261: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 9 op/s 2023-07-21T18:40:32.495 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.469+0000 7f36f56cb700 1 -- 172.21.15.39:0/382452480 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36f010a1d0 msgr2=0x7f36f010c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:32.495 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.469+0000 7f36f56cb700 1 --2- 172.21.15.39:0/382452480 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36f010a1d0 0x7f36f010c630 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f36e800b0a0 tx=0x7f36e800b3b0 comp rx=0 tx=0).stop 2023-07-21T18:40:32.495 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.469+0000 7f36f56cb700 1 -- 172.21.15.39:0/382452480 shutdown_connections 2023-07-21T18:40:32.495 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.469+0000 7f36f56cb700 1 --2- 172.21.15.39:0/382452480 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36f010a1d0 0x7f36f010c630 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:32.495 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.469+0000 7f36f56cb700 1 --2- 172.21.15.39:0/382452480 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f36f0100df0 0x7f36f01011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:32.496 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.469+0000 7f36f56cb700 1 --2- 172.21.15.39:0/382452480 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36f0101700 0x7f36f0109c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:32.496 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.470+0000 7f36f56cb700 1 -- 172.21.15.39:0/382452480 >> 172.21.15.39:0/382452480 conn(0x7f36f00fa5f0 msgr2=0x7f36f00fca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:32.496 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.470+0000 7f36f56cb700 1 -- 172.21.15.39:0/382452480 shutdown_connections 2023-07-21T18:40:32.496 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.470+0000 7f36f56cb700 1 -- 172.21.15.39:0/382452480 wait complete. 2023-07-21T18:40:32.497 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.470+0000 7f36f56cb700 1 Processor -- start 2023-07-21T18:40:32.498 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.471+0000 7f36f56cb700 1 -- start start 2023-07-21T18:40:32.498 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.471+0000 7f36f56cb700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36f0100df0 0x7f36f0195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:32.498 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.471+0000 7f36f56cb700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36f0101700 0x7f36f01962a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:32.498 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.471+0000 7f36f56cb700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f36f010a1d0 0x7f36f019a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:32.498 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.471+0000 7f36f56cb700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36f019ad20 con 0x7f36f0101700 2023-07-21T18:40:32.498 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.471+0000 7f36f56cb700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36f019ae90 con 0x7f36f0100df0 2023-07-21T18:40:32.499 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.471+0000 7f36f56cb700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36f019b050 con 0x7f36f010a1d0 2023-07-21T18:40:32.499 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.471+0000 7f36ee7fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36f0101700 0x7f36f01962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:32.499 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.471+0000 7f36ee7fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36f0101700 0x7f36f01962a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55368/0 (socket says 172.21.15.39:55368) 2023-07-21T18:40:32.499 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.471+0000 7f36ee7fc700 1 -- 172.21.15.39:0/3857342325 learned_addr learned my addr 172.21.15.39:0/3857342325 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:40:32.499 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.471+0000 7f36eeffd700 1 --2- 172.21.15.39:0/3857342325 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36f0100df0 0x7f36f0195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:32.499 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.471+0000 7f36ef7fe700 1 --2- 172.21.15.39:0/3857342325 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f36f010a1d0 0x7f36f019a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:32.499 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.472+0000 7f36ee7fc700 1 -- 172.21.15.39:0/3857342325 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f36f010a1d0 msgr2=0x7f36f019a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:32.500 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.472+0000 7f36ee7fc700 1 --2- 172.21.15.39:0/3857342325 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f36f010a1d0 0x7f36f019a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:32.501 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.472+0000 7f36ee7fc700 1 -- 172.21.15.39:0/3857342325 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36f0100df0 msgr2=0x7f36f0195d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:32.501 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.472+0000 7f36ee7fc700 1 --2- 172.21.15.39:0/3857342325 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36f0100df0 0x7f36f0195d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:32.501 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.472+0000 7f36ee7fc700 1 -- 172.21.15.39:0/3857342325 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e8009d00 con 0x7f36f0101700 2023-07-21T18:40:32.501 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.472+0000 7f36ee7fc700 1 --2- 172.21.15.39:0/3857342325 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36f0101700 0x7f36f01962a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f36e400c920 tx=0x7f36e400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:32.501 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.472+0000 7f36d7fff700 1 -- 172.21.15.39:0/3857342325 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e4004a30 con 0x7f36f0101700 2023-07-21T18:40:32.501 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.472+0000 7f36d7fff700 1 -- 172.21.15.39:0/3857342325 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36e40054e0 con 0x7f36f0101700 2023-07-21T18:40:32.501 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.472+0000 7f36d7fff700 1 -- 172.21.15.39:0/3857342325 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e4007bd0 con 0x7f36f0101700 2023-07-21T18:40:32.502 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.472+0000 7f36f56cb700 1 -- 172.21.15.39:0/3857342325 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36f019b330 con 0x7f36f0101700 2023-07-21T18:40:32.502 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.472+0000 7f36f56cb700 1 -- 172.21.15.39:0/3857342325 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36f01a2b90 con 0x7f36f0101700 2023-07-21T18:40:32.502 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.473+0000 7f36f56cb700 1 -- 172.21.15.39:0/3857342325 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f36f00681a0 con 0x7f36f0101700 2023-07-21T18:40:32.503 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.475+0000 7f36d7fff700 1 -- 172.21.15.39:0/3857342325 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f36e4007d30 con 0x7f36f0101700 2023-07-21T18:40:32.503 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.476+0000 7f36d7fff700 1 --2- 172.21.15.39:0/3857342325 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f36d80a1260 0x7f36d80a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:32.503 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.476+0000 7f36eeffd700 1 --2- 172.21.15.39:0/3857342325 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f36d80a1260 0x7f36d80a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:32.503 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.476+0000 7f36d7fff700 1 -- 172.21.15.39:0/3857342325 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f36e40cc730 con 0x7f36f0101700 2023-07-21T18:40:32.503 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.476+0000 7f36d7fff700 1 --2- 172.21.15.39:0/3857342325 >> [v2:172.21.15.155:6800/3246721764,v1:172.21.15.155:6801/3246721764] conn(0x7f36d80aaa30 0x7f36d80ace40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:32.503 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.476+0000 7f36d7fff700 1 -- 172.21.15.39:0/3857342325 --> [v2:172.21.15.155:6800/3246721764,v1:172.21.15.155:6801/3246721764] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f36d80ad4f0 con 0x7f36d80aaa30 2023-07-21T18:40:32.504 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.476+0000 7f36d7fff700 1 -- 172.21.15.39:0/3857342325 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36e40ccbd0 con 0x7f36f0101700 2023-07-21T18:40:32.504 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.477+0000 7f36ef7fe700 1 --2- 172.21.15.39:0/3857342325 >> [v2:172.21.15.155:6800/3246721764,v1:172.21.15.155:6801/3246721764] conn(0x7f36d80aaa30 0x7f36d80ace40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:32.504 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.477+0000 7f36eeffd700 1 --2- 172.21.15.39:0/3857342325 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f36d80a1260 0x7f36d80a3710 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f36e0005950 tx=0x7f36e000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:32.504 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.477+0000 7f36ef7fe700 1 --2- 172.21.15.39:0/3857342325 >> [v2:172.21.15.155:6800/3246721764,v1:172.21.15.155:6801/3246721764] conn(0x7f36d80aaa30 0x7f36d80ace40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:32.504 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.478+0000 7f36d7fff700 1 -- 172.21.15.39:0/3857342325 <== osd.1 v2:172.21.15.155:6800/3246721764 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f36d80ad4f0 con 0x7f36d80aaa30 2023-07-21T18:40:32.526 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.501+0000 7f36f56cb700 1 -- 172.21.15.39:0/3857342325 --> [v2:172.21.15.155:6800/3246721764,v1:172.21.15.155:6801/3246721764] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f36f010fdb0 con 0x7f36d80aaa30 2023-07-21T18:40:32.527 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.502+0000 7f36d7fff700 1 -- 172.21.15.39:0/3857342325 <== osd.1 v2:172.21.15.155:6800/3246721764 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f36e8009b80 con 0x7f36d80aaa30 2023-07-21T18:40:32.528 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.503+0000 7f36f56cb700 1 -- 172.21.15.39:0/3857342325 >> [v2:172.21.15.155:6800/3246721764,v1:172.21.15.155:6801/3246721764] conn(0x7f36d80aaa30 msgr2=0x7f36d80ace40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:32.528 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.503+0000 7f36f56cb700 1 --2- 172.21.15.39:0/3857342325 >> [v2:172.21.15.155:6800/3246721764,v1:172.21.15.155:6801/3246721764] conn(0x7f36d80aaa30 0x7f36d80ace40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:32.529 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.503+0000 7f36f56cb700 1 -- 172.21.15.39:0/3857342325 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f36d80a1260 msgr2=0x7f36d80a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:32.529 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.503+0000 7f36f56cb700 1 --2- 172.21.15.39:0/3857342325 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f36d80a1260 0x7f36d80a3710 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f36e0005950 tx=0x7f36e000a300 comp rx=0 tx=0).stop 2023-07-21T18:40:32.529 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.503+0000 7f36f56cb700 1 -- 172.21.15.39:0/3857342325 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36f0101700 msgr2=0x7f36f01962a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:32.529 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.503+0000 7f36f56cb700 1 --2- 172.21.15.39:0/3857342325 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36f0101700 0x7f36f01962a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f36e400c920 tx=0x7f36e400cc30 comp rx=0 tx=0).stop 2023-07-21T18:40:32.529 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.503+0000 7f36ef7fe700 1 -- 172.21.15.39:0/3857342325 reap_dead start 2023-07-21T18:40:32.530 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.503+0000 7f36f56cb700 1 -- 172.21.15.39:0/3857342325 shutdown_connections 2023-07-21T18:40:32.530 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.503+0000 7f36f56cb700 1 -- 172.21.15.39:0/3857342325 >> 172.21.15.39:0/3857342325 conn(0x7f36f00fa5f0 msgr2=0x7f36f010c330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:32.530 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.503+0000 7f36f56cb700 1 -- 172.21.15.39:0/3857342325 shutdown_connections 2023-07-21T18:40:32.530 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.503+0000 7f36f56cb700 1 -- 172.21.15.39:0/3857342325 wait complete. 2023-07-21T18:40:32.601 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.576+0000 7f0ca67fc700 1 -- 172.21.15.39:0/2749923237 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c98003b50 con 0x7f0ca8104ac0 2023-07-21T18:40:32.601 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.576+0000 7f0cadd5d700 1 -- 172.21.15.39:0/2749923237 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ca8104ac0 msgr2=0x7f0ca8104e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:32.601 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.576+0000 7f0cadd5d700 1 --2- 172.21.15.39:0/2749923237 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ca8104ac0 0x7f0ca8104e90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0c98009990 tx=0x7f0c98009ca0 comp rx=0 tx=0).stop 2023-07-21T18:40:32.601 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.576+0000 7f0cadd5d700 1 -- 172.21.15.39:0/2749923237 shutdown_connections 2023-07-21T18:40:32.602 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.576+0000 7f0cadd5d700 1 --2- 172.21.15.39:0/2749923237 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0ca80ff470 0x7f0ca810b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:32.602 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.576+0000 7f0cadd5d700 1 --2- 172.21.15.39:0/2749923237 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ca8104ac0 0x7f0ca8104e90 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:32.602 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.576+0000 7f0cadd5d700 1 --2- 172.21.15.39:0/2749923237 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ca80feac0 0x7f0ca80fef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:32.602 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.576+0000 7f0cadd5d700 1 -- 172.21.15.39:0/2749923237 >> 172.21.15.39:0/2749923237 conn(0x7f0ca8075950 msgr2=0x7f0ca8075d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:32.602 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.576+0000 7f0cadd5d700 1 -- 172.21.15.39:0/2749923237 shutdown_connections 2023-07-21T18:40:32.602 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.576+0000 7f0cadd5d700 1 -- 172.21.15.39:0/2749923237 wait complete. 2023-07-21T18:40:32.603 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.577+0000 7f0cadd5d700 1 Processor -- start 2023-07-21T18:40:32.603 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.578+0000 7f0cadd5d700 1 -- start start 2023-07-21T18:40:32.604 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.578+0000 7f0cadd5d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0ca80feac0 0x7f0ca8193990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:32.604 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.578+0000 7f0cadd5d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ca80ff470 0x7f0ca8193ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:32.604 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.578+0000 7f0cadd5d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ca8104ac0 0x7f0ca8198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:32.604 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.578+0000 7f0cadd5d700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca8198940 con 0x7f0ca8104ac0 2023-07-21T18:40:32.604 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.578+0000 7f0cadd5d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca8198ab0 con 0x7f0ca80ff470 2023-07-21T18:40:32.604 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.578+0000 7f0cadd5d700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca8198c70 con 0x7f0ca80feac0 2023-07-21T18:40:32.604 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.578+0000 7f0ca7fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ca8104ac0 0x7f0ca8198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:32.605 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.578+0000 7f0ca77fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0ca80feac0 0x7f0ca8193990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:32.605 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.578+0000 7f0ca7fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ca8104ac0 0x7f0ca8198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55384/0 (socket says 172.21.15.39:55384) 2023-07-21T18:40:32.605 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.578+0000 7f0ca7fff700 1 -- 172.21.15.39:0/2134132912 learned_addr learned my addr 172.21.15.39:0/2134132912 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:40:32.605 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.578+0000 7f0ca6ffd700 1 --2- 172.21.15.39:0/2134132912 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ca80ff470 0x7f0ca8193ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:32.605 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.579+0000 7f0ca77fe700 1 -- 172.21.15.39:0/2134132912 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ca80ff470 msgr2=0x7f0ca8193ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:32.605 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.579+0000 7f0ca77fe700 1 --2- 172.21.15.39:0/2134132912 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ca80ff470 0x7f0ca8193ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:32.605 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.579+0000 7f0ca77fe700 1 -- 172.21.15.39:0/2134132912 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ca8104ac0 msgr2=0x7f0ca8198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:32.606 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.579+0000 7f0ca77fe700 1 --2- 172.21.15.39:0/2134132912 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ca8104ac0 0x7f0ca8198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:32.606 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.579+0000 7f0ca77fe700 1 -- 172.21.15.39:0/2134132912 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c980095f0 con 0x7f0ca80feac0 2023-07-21T18:40:32.606 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.579+0000 7f0ca7fff700 1 --2- 172.21.15.39:0/2134132912 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ca8104ac0 0x7f0ca8198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:40:32.606 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.579+0000 7f0ca77fe700 1 --2- 172.21.15.39:0/2134132912 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0ca80feac0 0x7f0ca8193990 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0c98000c00 tx=0x7f0c9800f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:32.607 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.579+0000 7f0ca4ff9700 1 -- 172.21.15.39:0/2134132912 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c9801d070 con 0x7f0ca80feac0 2023-07-21T18:40:32.607 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.579+0000 7f0ca4ff9700 1 -- 172.21.15.39:0/2134132912 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c98022470 con 0x7f0ca80feac0 2023-07-21T18:40:32.607 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.579+0000 7f0cadd5d700 1 -- 172.21.15.39:0/2134132912 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ca8198ef0 con 0x7f0ca80feac0 2023-07-21T18:40:32.608 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.579+0000 7f0ca4ff9700 1 -- 172.21.15.39:0/2134132912 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c98017650 con 0x7f0ca80feac0 2023-07-21T18:40:32.608 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.579+0000 7f0cadd5d700 1 -- 172.21.15.39:0/2134132912 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ca81991e0 con 0x7f0ca80feac0 2023-07-21T18:40:32.608 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.580+0000 7f0cadd5d700 1 -- 172.21.15.39:0/2134132912 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0ca804e210 con 0x7f0ca80feac0 2023-07-21T18:40:32.608 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.582+0000 7f0ca4ff9700 1 -- 172.21.15.39:0/2134132912 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f0c98022800 con 0x7f0ca80feac0 2023-07-21T18:40:32.608 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.582+0000 7f0ca4ff9700 1 --2- 172.21.15.39:0/2134132912 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f0c900a1240 0x7f0c900a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:32.608 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.582+0000 7f0ca4ff9700 1 -- 172.21.15.39:0/2134132912 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0c980cdb50 con 0x7f0ca80feac0 2023-07-21T18:40:32.608 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.582+0000 7f0ca6ffd700 1 --2- 172.21.15.39:0/2134132912 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f0c900a1240 0x7f0c900a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:32.609 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.583+0000 7f0ca4ff9700 1 --2- 172.21.15.39:0/2134132912 >> [v2:172.21.15.39:6810/863708386,v1:172.21.15.39:6811/863708386] conn(0x7f0c900aaa10 0x7f0c900ace20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:32.609 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.583+0000 7f0ca4ff9700 1 -- 172.21.15.39:0/2134132912 --> [v2:172.21.15.39:6810/863708386,v1:172.21.15.39:6811/863708386] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0c900ad4d0 con 0x7f0c900aaa10 2023-07-21T18:40:32.609 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.583+0000 7f0ca4ff9700 1 -- 172.21.15.39:0/2134132912 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c980cdff0 con 0x7f0ca80feac0 2023-07-21T18:40:32.609 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.583+0000 7f0ca7fff700 1 --2- 172.21.15.39:0/2134132912 >> [v2:172.21.15.39:6810/863708386,v1:172.21.15.39:6811/863708386] conn(0x7f0c900aaa10 0x7f0c900ace20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:32.610 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.583+0000 7f0ca7fff700 1 --2- 172.21.15.39:0/2134132912 >> [v2:172.21.15.39:6810/863708386,v1:172.21.15.39:6811/863708386] conn(0x7f0c900aaa10 0x7f0c900ace20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:32.610 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.583+0000 7f0ca6ffd700 1 --2- 172.21.15.39:0/2134132912 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f0c900a1240 0x7f0c900a36f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8194fb0 tx=0x7f0c9c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:32.610 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.584+0000 7f0ca4ff9700 1 -- 172.21.15.39:0/2134132912 <== osd.3 v2:172.21.15.39:6810/863708386 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f0c900ad4d0 con 0x7f0c900aaa10 2023-07-21T18:40:32.629 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.604+0000 7f0cadd5d700 1 -- 172.21.15.39:0/2134132912 --> [v2:172.21.15.39:6810/863708386,v1:172.21.15.39:6811/863708386] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0ca810dbf0 con 0x7f0c900aaa10 2023-07-21T18:40:32.629 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.604+0000 7f0ca4ff9700 1 -- 172.21.15.39:0/2134132912 <== osd.3 v2:172.21.15.39:6810/863708386 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f0ca000f040 con 0x7f0c900aaa10 2023-07-21T18:40:32.630 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.605+0000 7f0cadd5d700 1 -- 172.21.15.39:0/2134132912 >> [v2:172.21.15.39:6810/863708386,v1:172.21.15.39:6811/863708386] conn(0x7f0c900aaa10 msgr2=0x7f0c900ace20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:32.630 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.605+0000 7f0cadd5d700 1 --2- 172.21.15.39:0/2134132912 >> [v2:172.21.15.39:6810/863708386,v1:172.21.15.39:6811/863708386] conn(0x7f0c900aaa10 0x7f0c900ace20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:32.630 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.605+0000 7f0cadd5d700 1 -- 172.21.15.39:0/2134132912 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f0c900a1240 msgr2=0x7f0c900a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:32.630 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.605+0000 7f0cadd5d700 1 --2- 172.21.15.39:0/2134132912 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f0c900a1240 0x7f0c900a36f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8194fb0 tx=0x7f0c9c00b410 comp rx=0 tx=0).stop 2023-07-21T18:40:32.630 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.605+0000 7f0cadd5d700 1 -- 172.21.15.39:0/2134132912 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0ca80feac0 msgr2=0x7f0ca8193990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:32.631 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.605+0000 7f0cadd5d700 1 --2- 172.21.15.39:0/2134132912 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0ca80feac0 0x7f0ca8193990 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0c98000c00 tx=0x7f0c9800f690 comp rx=0 tx=0).stop 2023-07-21T18:40:32.631 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.605+0000 7f0ca7fff700 1 -- 172.21.15.39:0/2134132912 reap_dead start 2023-07-21T18:40:32.631 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.605+0000 7f0cadd5d700 1 -- 172.21.15.39:0/2134132912 shutdown_connections 2023-07-21T18:40:32.631 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.605+0000 7f0cadd5d700 1 -- 172.21.15.39:0/2134132912 >> 172.21.15.39:0/2134132912 conn(0x7f0ca8075950 msgr2=0x7f0ca8105920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:32.631 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.605+0000 7f0cadd5d700 1 -- 172.21.15.39:0/2134132912 shutdown_connections 2023-07-21T18:40:32.631 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:32.605+0000 7f0cadd5d700 1 -- 172.21.15.39:0/2134132912 wait complete. 2023-07-21T18:40:33.190 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.166+0000 7f9cce6a7700 1 -- 172.21.15.39:0/1334814833 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9cc80ff460 msgr2=0x7f9cc810b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:33.191 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.166+0000 7f9cce6a7700 1 --2- 172.21.15.39:0/1334814833 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9cc80ff460 0x7f9cc810b8f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9cc000b0a0 tx=0x7f9cc000b3b0 comp rx=0 tx=0).stop 2023-07-21T18:40:33.191 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.166+0000 7f9cce6a7700 1 -- 172.21.15.39:0/1334814833 shutdown_connections 2023-07-21T18:40:33.191 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.166+0000 7f9cce6a7700 1 --2- 172.21.15.39:0/1334814833 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9cc8104ab0 0x7f9cc8104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:33.191 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.166+0000 7f9cce6a7700 1 --2- 172.21.15.39:0/1334814833 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9cc80ff460 0x7f9cc810b8f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:33.191 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.166+0000 7f9cce6a7700 1 --2- 172.21.15.39:0/1334814833 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9cc80feab0 0x7f9cc80fef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:33.191 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.166+0000 7f9cce6a7700 1 -- 172.21.15.39:0/1334814833 >> 172.21.15.39:0/1334814833 conn(0x7f9cc8075b70 msgr2=0x7f9cc8075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:33.192 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.166+0000 7f9cce6a7700 1 -- 172.21.15.39:0/1334814833 shutdown_connections 2023-07-21T18:40:33.192 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.166+0000 7f9cce6a7700 1 -- 172.21.15.39:0/1334814833 wait complete. 2023-07-21T18:40:33.193 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.166+0000 7f9cce6a7700 1 Processor -- start 2023-07-21T18:40:33.193 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.166+0000 7f9cce6a7700 1 -- start start 2023-07-21T18:40:33.193 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.167+0000 7f9cce6a7700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9cc80feab0 0x7f9cc8196a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:33.194 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.167+0000 7f9cce6a7700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9cc80ff460 0x7f9cc8196fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:33.194 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.167+0000 7f9cce6a7700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9cc8104ab0 0x7f9cc81a1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:33.194 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.167+0000 7f9cce6a7700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc81978d0 con 0x7f9cc8104ab0 2023-07-21T18:40:33.194 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.167+0000 7f9cce6a7700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc81a2040 con 0x7f9cc80ff460 2023-07-21T18:40:33.194 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.167+0000 7f9cce6a7700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc81a21b0 con 0x7f9cc80feab0 2023-07-21T18:40:33.194 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.167+0000 7f9cccc44700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9cc8104ab0 0x7f9cc81a1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:33.194 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.167+0000 7f9cccc44700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9cc8104ab0 0x7f9cc81a1b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55402/0 (socket says 172.21.15.39:55402) 2023-07-21T18:40:33.195 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.167+0000 7f9cccc44700 1 -- 172.21.15.39:0/2239154261 learned_addr learned my addr 172.21.15.39:0/2239154261 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:40:33.195 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.167+0000 7f9cc77fe700 1 --2- 172.21.15.39:0/2239154261 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9cc80ff460 0x7f9cc8196fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:33.195 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.167+0000 7f9cc7fff700 1 --2- 172.21.15.39:0/2239154261 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9cc80feab0 0x7f9cc8196a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:33.195 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.167+0000 7f9cccc44700 1 -- 172.21.15.39:0/2239154261 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9cc80feab0 msgr2=0x7f9cc8196a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:33.195 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.167+0000 7f9cccc44700 1 --2- 172.21.15.39:0/2239154261 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9cc80feab0 0x7f9cc8196a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:33.195 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.167+0000 7f9cccc44700 1 -- 172.21.15.39:0/2239154261 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9cc80ff460 msgr2=0x7f9cc8196fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:33.195 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.167+0000 7f9cccc44700 1 --2- 172.21.15.39:0/2239154261 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9cc80ff460 0x7f9cc8196fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:33.196 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.167+0000 7f9cccc44700 1 -- 172.21.15.39:0/2239154261 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cc0009d00 con 0x7f9cc8104ab0 2023-07-21T18:40:33.196 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.167+0000 7f9cc7fff700 1 --2- 172.21.15.39:0/2239154261 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9cc80feab0 0x7f9cc8196a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:40:33.196 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.167+0000 7f9cc77fe700 1 --2- 172.21.15.39:0/2239154261 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9cc80ff460 0x7f9cc8196fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:40:33.196 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.167+0000 7f9cccc44700 1 --2- 172.21.15.39:0/2239154261 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9cc8104ab0 0x7f9cc81a1b00 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9cc00062a0 tx=0x7f9cc000cfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:33.197 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.167+0000 7f9cc57fa700 1 -- 172.21.15.39:0/2239154261 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc000e040 con 0x7f9cc8104ab0 2023-07-21T18:40:33.197 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.168+0000 7f9cce6a7700 1 -- 172.21.15.39:0/2239154261 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cc81a2430 con 0x7f9cc8104ab0 2023-07-21T18:40:33.197 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.168+0000 7f9cc57fa700 1 -- 172.21.15.39:0/2239154261 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9cc0004020 con 0x7f9cc8104ab0 2023-07-21T18:40:33.197 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.168+0000 7f9cc57fa700 1 -- 172.21.15.39:0/2239154261 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc001b690 con 0x7f9cc8104ab0 2023-07-21T18:40:33.197 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.168+0000 7f9cce6a7700 1 -- 172.21.15.39:0/2239154261 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cc81a28f0 con 0x7f9cc8104ab0 2023-07-21T18:40:33.197 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.168+0000 7f9cce6a7700 1 -- 172.21.15.39:0/2239154261 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9cc804e210 con 0x7f9cc8104ab0 2023-07-21T18:40:33.198 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.170+0000 7f9cc57fa700 1 -- 172.21.15.39:0/2239154261 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f9cc0004400 con 0x7f9cc8104ab0 2023-07-21T18:40:33.199 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.171+0000 7f9cc57fa700 1 --2- 172.21.15.39:0/2239154261 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f9cb00a11e0 0x7f9cb00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:33.199 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.171+0000 7f9cc7fff700 1 --2- 172.21.15.39:0/2239154261 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f9cb00a11e0 0x7f9cb00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:33.199 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.171+0000 7f9cc57fa700 1 -- 172.21.15.39:0/2239154261 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f9cc00ceb20 con 0x7f9cc8104ab0 2023-07-21T18:40:33.199 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.171+0000 7f9cc57fa700 1 --2- 172.21.15.39:0/2239154261 >> [v2:172.21.15.193:6808/2376867721,v1:172.21.15.193:6809/2376867721] conn(0x7f9cb00aa9b0 0x7f9cb00acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:33.199 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.171+0000 7f9cc57fa700 1 -- 172.21.15.39:0/2239154261 --> [v2:172.21.15.193:6808/2376867721,v1:172.21.15.193:6809/2376867721] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9cb00ad470 con 0x7f9cb00aa9b0 2023-07-21T18:40:33.199 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.171+0000 7f9cc57fa700 1 -- 172.21.15.39:0/2239154261 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9cc00cee90 con 0x7f9cc8104ab0 2023-07-21T18:40:33.199 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.172+0000 7f9cc7fff700 1 --2- 172.21.15.39:0/2239154261 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f9cb00a11e0 0x7f9cb00a3690 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8005fd0 tx=0x7f9cb8005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:33.199 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.172+0000 7f9cc77fe700 1 --2- 172.21.15.39:0/2239154261 >> [v2:172.21.15.193:6808/2376867721,v1:172.21.15.193:6809/2376867721] conn(0x7f9cb00aa9b0 0x7f9cb00acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:33.200 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.172+0000 7f9cc77fe700 1 --2- 172.21.15.39:0/2239154261 >> [v2:172.21.15.193:6808/2376867721,v1:172.21.15.193:6809/2376867721] conn(0x7f9cb00aa9b0 0x7f9cb00acdc0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:33.200 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.173+0000 7f9cc57fa700 1 -- 172.21.15.39:0/2239154261 <== osd.5 v2:172.21.15.193:6808/2376867721 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f9cb00ad470 con 0x7f9cb00aa9b0 2023-07-21T18:40:33.214 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.190+0000 7f9cce6a7700 1 -- 172.21.15.39:0/2239154261 --> [v2:172.21.15.193:6808/2376867721,v1:172.21.15.193:6809/2376867721] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9cc810ec50 con 0x7f9cb00aa9b0 2023-07-21T18:40:33.215 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.190+0000 7f9cc57fa700 1 -- 172.21.15.39:0/2239154261 <== osd.5 v2:172.21.15.193:6808/2376867721 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f9cbc008c00 con 0x7f9cb00aa9b0 2023-07-21T18:40:33.216 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.191+0000 7f9cce6a7700 1 -- 172.21.15.39:0/2239154261 >> [v2:172.21.15.193:6808/2376867721,v1:172.21.15.193:6809/2376867721] conn(0x7f9cb00aa9b0 msgr2=0x7f9cb00acdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:33.216 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.191+0000 7f9cce6a7700 1 --2- 172.21.15.39:0/2239154261 >> [v2:172.21.15.193:6808/2376867721,v1:172.21.15.193:6809/2376867721] conn(0x7f9cb00aa9b0 0x7f9cb00acdc0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:33.216 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.191+0000 7f9cce6a7700 1 -- 172.21.15.39:0/2239154261 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f9cb00a11e0 msgr2=0x7f9cb00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:33.216 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.191+0000 7f9cce6a7700 1 --2- 172.21.15.39:0/2239154261 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f9cb00a11e0 0x7f9cb00a3690 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8005fd0 tx=0x7f9cb8005dc0 comp rx=0 tx=0).stop 2023-07-21T18:40:33.216 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.191+0000 7f9cce6a7700 1 -- 172.21.15.39:0/2239154261 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9cc8104ab0 msgr2=0x7f9cc81a1b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:33.216 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.191+0000 7f9cce6a7700 1 --2- 172.21.15.39:0/2239154261 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9cc8104ab0 0x7f9cc81a1b00 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9cc00062a0 tx=0x7f9cc000cfb0 comp rx=0 tx=0).stop 2023-07-21T18:40:33.217 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.191+0000 7f9cccc44700 1 -- 172.21.15.39:0/2239154261 reap_dead start 2023-07-21T18:40:33.217 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.191+0000 7f9cce6a7700 1 -- 172.21.15.39:0/2239154261 shutdown_connections 2023-07-21T18:40:33.217 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.191+0000 7f9cce6a7700 1 -- 172.21.15.39:0/2239154261 >> 172.21.15.39:0/2239154261 conn(0x7f9cc8075b70 msgr2=0x7f9cc8102600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:33.217 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.191+0000 7f9cce6a7700 1 -- 172.21.15.39:0/2239154261 shutdown_connections 2023-07-21T18:40:33.217 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.191+0000 7f9cce6a7700 1 -- 172.21.15.39:0/2239154261 wait complete. 2023-07-21T18:40:33.514 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.490+0000 7f9dabec9700 1 -- 172.21.15.39:0/1934525852 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9da4104ab0 msgr2=0x7f9da4104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:33.515 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.490+0000 7f9dabec9700 1 --2- 172.21.15.39:0/1934525852 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9da4104ab0 0x7f9da4104e80 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9da0009990 tx=0x7f9da0009ca0 comp rx=0 tx=0).stop 2023-07-21T18:40:33.515 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.490+0000 7f9dabec9700 1 -- 172.21.15.39:0/1934525852 shutdown_connections 2023-07-21T18:40:33.515 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.490+0000 7f9dabec9700 1 --2- 172.21.15.39:0/1934525852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9da40ff460 0x7f9da410b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:33.515 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.490+0000 7f9dabec9700 1 --2- 172.21.15.39:0/1934525852 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9da4104ab0 0x7f9da4104e80 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:33.516 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.490+0000 7f9dabec9700 1 --2- 172.21.15.39:0/1934525852 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9da40feab0 0x7f9da40fef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:33.516 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.490+0000 7f9dabec9700 1 -- 172.21.15.39:0/1934525852 >> 172.21.15.39:0/1934525852 conn(0x7f9da4075b70 msgr2=0x7f9da4075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:33.516 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.490+0000 7f9dabec9700 1 -- 172.21.15.39:0/1934525852 shutdown_connections 2023-07-21T18:40:33.516 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.490+0000 7f9dabec9700 1 -- 172.21.15.39:0/1934525852 wait complete. 2023-07-21T18:40:33.518 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.491+0000 7f9dabec9700 1 Processor -- start 2023-07-21T18:40:33.518 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.491+0000 7f9dabec9700 1 -- start start 2023-07-21T18:40:33.518 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.491+0000 7f9dabec9700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9da40feab0 0x7f9da4196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:33.518 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.491+0000 7f9dabec9700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9da40ff460 0x7f9da4196f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:33.518 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.491+0000 7f9dabec9700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9da4104ab0 0x7f9da41a1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:33.519 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.491+0000 7f9dabec9700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da41a1fd0 con 0x7f9da4104ab0 2023-07-21T18:40:33.519 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.491+0000 7f9dabec9700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da41a2110 con 0x7f9da40feab0 2023-07-21T18:40:33.519 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.491+0000 7f9dabec9700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da41a2250 con 0x7f9da40ff460 2023-07-21T18:40:33.519 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.491+0000 7f9daa466700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9da4104ab0 0x7f9da41a1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:33.519 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.491+0000 7f9daa466700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9da4104ab0 0x7f9da41a1a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55418/0 (socket says 172.21.15.39:55418) 2023-07-21T18:40:33.519 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.491+0000 7f9daa466700 1 -- 172.21.15.39:0/2494193244 learned_addr learned my addr 172.21.15.39:0/2494193244 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:40:33.519 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.491+0000 7f9da9464700 1 --2- 172.21.15.39:0/2494193244 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9da40ff460 0x7f9da4196f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:33.520 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.491+0000 7f9da9c65700 1 --2- 172.21.15.39:0/2494193244 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9da40feab0 0x7f9da4196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:33.520 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.491+0000 7f9daa466700 1 -- 172.21.15.39:0/2494193244 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9da40ff460 msgr2=0x7f9da4196f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:33.520 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.491+0000 7f9daa466700 1 --2- 172.21.15.39:0/2494193244 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9da40ff460 0x7f9da4196f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:33.520 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.491+0000 7f9daa466700 1 -- 172.21.15.39:0/2494193244 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9da40feab0 msgr2=0x7f9da4196a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:33.520 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.491+0000 7f9daa466700 1 --2- 172.21.15.39:0/2494193244 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9da40feab0 0x7f9da4196a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:33.520 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.492+0000 7f9daa466700 1 -- 172.21.15.39:0/2494193244 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9da00095f0 con 0x7f9da4104ab0 2023-07-21T18:40:33.520 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.492+0000 7f9daa466700 1 --2- 172.21.15.39:0/2494193244 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9da4104ab0 0x7f9da41a1a90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c007f10 tx=0x7f9d9c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:33.521 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.492+0000 7f9d9affd700 1 -- 172.21.15.39:0/2494193244 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c017070 con 0x7f9da4104ab0 2023-07-21T18:40:33.521 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.492+0000 7f9d9affd700 1 -- 172.21.15.39:0/2494193244 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d9c00f040 con 0x7f9da4104ab0 2023-07-21T18:40:33.521 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.492+0000 7f9dabec9700 1 -- 172.21.15.39:0/2494193244 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9da41a24b0 con 0x7f9da4104ab0 2023-07-21T18:40:33.521 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.492+0000 7f9d9affd700 1 -- 172.21.15.39:0/2494193244 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c0054b0 con 0x7f9da4104ab0 2023-07-21T18:40:33.521 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.492+0000 7f9dabec9700 1 -- 172.21.15.39:0/2494193244 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9da41a2980 con 0x7f9da4104ab0 2023-07-21T18:40:33.522 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.493+0000 7f9dabec9700 1 -- 172.21.15.39:0/2494193244 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9da404e210 con 0x7f9da4104ab0 2023-07-21T18:40:33.522 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.494+0000 7f9d9affd700 1 -- 172.21.15.39:0/2494193244 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f9d9c004ad0 con 0x7f9da4104ab0 2023-07-21T18:40:33.523 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.494+0000 7f9d9affd700 1 --2- 172.21.15.39:0/2494193244 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f9d900a1260 0x7f9d900a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:33.523 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.495+0000 7f9da9c65700 1 --2- 172.21.15.39:0/2494193244 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f9d900a1260 0x7f9d900a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:33.523 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.495+0000 7f9d9affd700 1 -- 172.21.15.39:0/2494193244 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f9d9c0cc5e0 con 0x7f9da4104ab0 2023-07-21T18:40:33.523 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.495+0000 7f9d9affd700 1 --2- 172.21.15.39:0/2494193244 >> [v2:172.21.15.39:6818/2417770529,v1:172.21.15.39:6819/2417770529] conn(0x7f9d900aaa30 0x7f9d900ace40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:33.523 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.495+0000 7f9d9affd700 1 -- 172.21.15.39:0/2494193244 --> [v2:172.21.15.39:6818/2417770529,v1:172.21.15.39:6819/2417770529] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9d900ad4f0 con 0x7f9d900aaa30 2023-07-21T18:40:33.523 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.495+0000 7f9d9affd700 1 -- 172.21.15.39:0/2494193244 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d9c0cca80 con 0x7f9da4104ab0 2023-07-21T18:40:33.524 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.495+0000 7f9da9464700 1 --2- 172.21.15.39:0/2494193244 >> [v2:172.21.15.39:6818/2417770529,v1:172.21.15.39:6819/2417770529] conn(0x7f9d900aaa30 0x7f9d900ace40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:33.524 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.495+0000 7f9da9c65700 1 --2- 172.21.15.39:0/2494193244 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f9d900a1260 0x7f9d900a3710 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9da000b5c0 tx=0x7f9da0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:33.524 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.495+0000 7f9da9464700 1 --2- 172.21.15.39:0/2494193244 >> [v2:172.21.15.39:6818/2417770529,v1:172.21.15.39:6819/2417770529] conn(0x7f9d900aaa30 0x7f9d900ace40 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:33.524 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.496+0000 7f9d9affd700 1 -- 172.21.15.39:0/2494193244 <== osd.6 v2:172.21.15.39:6818/2417770529 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f9d900ad4f0 con 0x7f9d900aaa30 2023-07-21T18:40:33.537 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.512+0000 7f9dabec9700 1 -- 172.21.15.39:0/2494193244 --> [v2:172.21.15.39:6818/2417770529,v1:172.21.15.39:6819/2417770529] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9da410ec50 con 0x7f9d900aaa30 2023-07-21T18:40:33.537 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.512+0000 7f9d9affd700 1 -- 172.21.15.39:0/2494193244 <== osd.6 v2:172.21.15.39:6818/2417770529 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f9d9400a840 con 0x7f9d900aaa30 2023-07-21T18:40:33.538 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.512+0000 7f9dabec9700 1 -- 172.21.15.39:0/2494193244 >> [v2:172.21.15.39:6818/2417770529,v1:172.21.15.39:6819/2417770529] conn(0x7f9d900aaa30 msgr2=0x7f9d900ace40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:33.538 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.512+0000 7f9dabec9700 1 --2- 172.21.15.39:0/2494193244 >> [v2:172.21.15.39:6818/2417770529,v1:172.21.15.39:6819/2417770529] conn(0x7f9d900aaa30 0x7f9d900ace40 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:33.538 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.512+0000 7f9dabec9700 1 -- 172.21.15.39:0/2494193244 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f9d900a1260 msgr2=0x7f9d900a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:33.539 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.512+0000 7f9dabec9700 1 --2- 172.21.15.39:0/2494193244 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f9d900a1260 0x7f9d900a3710 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9da000b5c0 tx=0x7f9da0005fd0 comp rx=0 tx=0).stop 2023-07-21T18:40:33.539 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.512+0000 7f9dabec9700 1 -- 172.21.15.39:0/2494193244 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9da4104ab0 msgr2=0x7f9da41a1a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:33.539 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.512+0000 7f9dabec9700 1 --2- 172.21.15.39:0/2494193244 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9da4104ab0 0x7f9da41a1a90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c007f10 tx=0x7f9d9c00d3b0 comp rx=0 tx=0).stop 2023-07-21T18:40:33.539 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.513+0000 7f9daa466700 1 -- 172.21.15.39:0/2494193244 reap_dead start 2023-07-21T18:40:33.539 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.513+0000 7f9dabec9700 1 -- 172.21.15.39:0/2494193244 shutdown_connections 2023-07-21T18:40:33.539 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.513+0000 7f9dabec9700 1 -- 172.21.15.39:0/2494193244 >> 172.21.15.39:0/2494193244 conn(0x7f9da4075b70 msgr2=0x7f9da40fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:33.539 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.513+0000 7f9dabec9700 1 -- 172.21.15.39:0/2494193244 shutdown_connections 2023-07-21T18:40:33.540 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.513+0000 7f9dabec9700 1 -- 172.21.15.39:0/2494193244 wait complete. 2023-07-21T18:40:33.555 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:33 smithi039 ceph-mon[96489]: pgmap v262: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 426 B/s wr, 10 op/s 2023-07-21T18:40:33.622 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:33 smithi193 ceph-mon[98204]: pgmap v262: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 426 B/s wr, 10 op/s 2023-07-21T18:40:33.641 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:33 smithi155 ceph-mon[97934]: pgmap v262: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 426 B/s wr, 10 op/s 2023-07-21T18:40:33.816 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.792+0000 7fa55c05e700 1 -- 172.21.15.39:0/2153927728 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa554104ea0 msgr2=0x7fa55410ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:33.817 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.792+0000 7fa55c05e700 1 --2- 172.21.15.39:0/2153927728 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa554104ea0 0x7fa55410ca20 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa54c00b0d0 tx=0x7fa54c00b3e0 comp rx=0 tx=0).stop 2023-07-21T18:40:33.817 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.792+0000 7fa55c05e700 1 -- 172.21.15.39:0/2153927728 shutdown_connections 2023-07-21T18:40:33.817 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.792+0000 7fa55c05e700 1 --2- 172.21.15.39:0/2153927728 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa554104ea0 0x7fa55410ca20 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:33.817 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.792+0000 7fa55c05e700 1 --2- 172.21.15.39:0/2153927728 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5540ffe00 0x7fa5541001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:33.817 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.792+0000 7fa55c05e700 1 --2- 172.21.15.39:0/2153927728 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5541007a0 0x7fa554104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:33.817 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.792+0000 7fa55c05e700 1 -- 172.21.15.39:0/2153927728 >> 172.21.15.39:0/2153927728 conn(0x7fa5540fb660 msgr2=0x7fa5540fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:33.818 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.792+0000 7fa55c05e700 1 -- 172.21.15.39:0/2153927728 shutdown_connections 2023-07-21T18:40:33.818 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.792+0000 7fa55c05e700 1 -- 172.21.15.39:0/2153927728 wait complete. 2023-07-21T18:40:33.819 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.792+0000 7fa55c05e700 1 Processor -- start 2023-07-21T18:40:33.819 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.792+0000 7fa55c05e700 1 -- start start 2023-07-21T18:40:33.819 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.793+0000 7fa55c05e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5540ffe00 0x7fa554072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:33.819 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.793+0000 7fa55c05e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5541007a0 0x7fa554072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:33.819 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.793+0000 7fa55c05e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa554104ea0 0x7fa5541a3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:33.820 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.793+0000 7fa55c05e700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa554073640 con 0x7fa5541007a0 2023-07-21T18:40:33.820 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.793+0000 7fa55c05e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5541a4470 con 0x7fa554104ea0 2023-07-21T18:40:33.820 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.793+0000 7fa55c05e700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5541a45e0 con 0x7fa5540ffe00 2023-07-21T18:40:33.820 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.793+0000 7fa5595f9700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5541007a0 0x7fa554072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:33.821 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.793+0000 7fa5595f9700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5541007a0 0x7fa554072d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55432/0 (socket says 172.21.15.39:55432) 2023-07-21T18:40:33.821 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.793+0000 7fa5595f9700 1 -- 172.21.15.39:0/1312152540 learned_addr learned my addr 172.21.15.39:0/1312152540 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:40:33.821 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.793+0000 7fa559dfa700 1 --2- 172.21.15.39:0/1312152540 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5540ffe00 0x7fa554072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:33.821 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.793+0000 7fa55a5fb700 1 --2- 172.21.15.39:0/1312152540 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa554104ea0 0x7fa5541a3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:33.821 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.793+0000 7fa5595f9700 1 -- 172.21.15.39:0/1312152540 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5540ffe00 msgr2=0x7fa554072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:33.821 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.793+0000 7fa5595f9700 1 --2- 172.21.15.39:0/1312152540 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5540ffe00 0x7fa554072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:33.821 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.793+0000 7fa5595f9700 1 -- 172.21.15.39:0/1312152540 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa554104ea0 msgr2=0x7fa5541a3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:33.822 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.793+0000 7fa5595f9700 1 --2- 172.21.15.39:0/1312152540 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa554104ea0 0x7fa5541a3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:33.822 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.793+0000 7fa5595f9700 1 -- 172.21.15.39:0/1312152540 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa54c009d00 con 0x7fa5541007a0 2023-07-21T18:40:33.822 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.793+0000 7fa55a5fb700 1 --2- 172.21.15.39:0/1312152540 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa554104ea0 0x7fa5541a3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:40:33.822 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.794+0000 7fa5595f9700 1 --2- 172.21.15.39:0/1312152540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5541007a0 0x7fa554072d50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa54400e970 tx=0x7fa54400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:33.822 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.794+0000 7fa54affd700 1 -- 172.21.15.39:0/1312152540 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa544009790 con 0x7fa5541007a0 2023-07-21T18:40:33.822 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.794+0000 7fa55c05e700 1 -- 172.21.15.39:0/1312152540 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5541a4860 con 0x7fa5541007a0 2023-07-21T18:40:33.822 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.794+0000 7fa54affd700 1 -- 172.21.15.39:0/1312152540 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa544004cf0 con 0x7fa5541007a0 2023-07-21T18:40:33.822 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.794+0000 7fa54affd700 1 -- 172.21.15.39:0/1312152540 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa544005680 con 0x7fa5541007a0 2023-07-21T18:40:33.823 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.794+0000 7fa55c05e700 1 -- 172.21.15.39:0/1312152540 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5541a4db0 con 0x7fa5541007a0 2023-07-21T18:40:33.824 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.795+0000 7fa55c05e700 1 -- 172.21.15.39:0/1312152540 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa55410fca0 con 0x7fa5541007a0 2023-07-21T18:40:33.824 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.797+0000 7fa54affd700 1 -- 172.21.15.39:0/1312152540 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fa544004750 con 0x7fa5541007a0 2023-07-21T18:40:33.824 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.798+0000 7fa54affd700 1 --2- 172.21.15.39:0/1312152540 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fa5400a11e0 0x7fa5400a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:33.824 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.798+0000 7fa559dfa700 1 --2- 172.21.15.39:0/1312152540 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fa5400a11e0 0x7fa5400a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:33.824 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.798+0000 7fa54affd700 1 -- 172.21.15.39:0/1312152540 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa544014070 con 0x7fa5541007a0 2023-07-21T18:40:33.824 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.799+0000 7fa54affd700 1 --2- 172.21.15.39:0/1312152540 >> [v2:172.21.15.193:6824/215104663,v1:172.21.15.193:6825/215104663] conn(0x7fa5400aa9b0 0x7fa5400acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:33.824 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.799+0000 7fa54affd700 1 -- 172.21.15.39:0/1312152540 --> [v2:172.21.15.193:6824/215104663,v1:172.21.15.193:6825/215104663] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa5400ad470 con 0x7fa5400aa9b0 2023-07-21T18:40:33.825 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.799+0000 7fa54affd700 1 -- 172.21.15.39:0/1312152540 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5440cd270 con 0x7fa5541007a0 2023-07-21T18:40:33.825 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.799+0000 7fa559dfa700 1 --2- 172.21.15.39:0/1312152540 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fa5400a11e0 0x7fa5400a3690 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa550005fd0 tx=0x7fa550005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:33.825 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.799+0000 7fa55a5fb700 1 --2- 172.21.15.39:0/1312152540 >> [v2:172.21.15.193:6824/215104663,v1:172.21.15.193:6825/215104663] conn(0x7fa5400aa9b0 0x7fa5400acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:33.825 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.800+0000 7fa55a5fb700 1 --2- 172.21.15.39:0/1312152540 >> [v2:172.21.15.193:6824/215104663,v1:172.21.15.193:6825/215104663] conn(0x7fa5400aa9b0 0x7fa5400acdc0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:33.826 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.800+0000 7fa54affd700 1 -- 172.21.15.39:0/1312152540 <== osd.11 v2:172.21.15.193:6824/215104663 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fa5400ad470 con 0x7fa5400aa9b0 2023-07-21T18:40:33.842 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.817+0000 7fa55c05e700 1 -- 172.21.15.39:0/1312152540 --> [v2:172.21.15.193:6824/215104663,v1:172.21.15.193:6825/215104663] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa55404e210 con 0x7fa5400aa9b0 2023-07-21T18:40:33.842 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.817+0000 7fa54affd700 1 -- 172.21.15.39:0/1312152540 <== osd.11 v2:172.21.15.193:6824/215104663 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fa55404e210 con 0x7fa5400aa9b0 2023-07-21T18:40:33.843 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.818+0000 7fa55c05e700 1 -- 172.21.15.39:0/1312152540 >> [v2:172.21.15.193:6824/215104663,v1:172.21.15.193:6825/215104663] conn(0x7fa5400aa9b0 msgr2=0x7fa5400acdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:33.843 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.818+0000 7fa55c05e700 1 --2- 172.21.15.39:0/1312152540 >> [v2:172.21.15.193:6824/215104663,v1:172.21.15.193:6825/215104663] conn(0x7fa5400aa9b0 0x7fa5400acdc0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:33.843 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.818+0000 7fa55c05e700 1 -- 172.21.15.39:0/1312152540 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fa5400a11e0 msgr2=0x7fa5400a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:33.843 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.818+0000 7fa55c05e700 1 --2- 172.21.15.39:0/1312152540 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fa5400a11e0 0x7fa5400a3690 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa550005fd0 tx=0x7fa550005f60 comp rx=0 tx=0).stop 2023-07-21T18:40:33.843 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.818+0000 7fa55c05e700 1 -- 172.21.15.39:0/1312152540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5541007a0 msgr2=0x7fa554072d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:33.844 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.818+0000 7fa55c05e700 1 --2- 172.21.15.39:0/1312152540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5541007a0 0x7fa554072d50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa54400e970 tx=0x7fa54400ed30 comp rx=0 tx=0).stop 2023-07-21T18:40:33.844 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.818+0000 7fa55a5fb700 1 -- 172.21.15.39:0/1312152540 reap_dead start 2023-07-21T18:40:33.844 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.818+0000 7fa55c05e700 1 -- 172.21.15.39:0/1312152540 shutdown_connections 2023-07-21T18:40:33.844 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.818+0000 7fa55c05e700 1 -- 172.21.15.39:0/1312152540 >> 172.21.15.39:0/1312152540 conn(0x7fa5540fb660 msgr2=0x7fa5540fc070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:33.844 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.818+0000 7fa55c05e700 1 -- 172.21.15.39:0/1312152540 shutdown_connections 2023-07-21T18:40:33.844 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.818+0000 7fa55c05e700 1 -- 172.21.15.39:0/1312152540 wait complete. 2023-07-21T18:40:34.050 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.888+0000 7f2cb9482700 1 -- 172.21.15.39:0/987231122 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2cb40ffe80 msgr2=0x7f2cb4100250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:34.050 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.888+0000 7f2cb9482700 1 --2- 172.21.15.39:0/987231122 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2cb40ffe80 0x7f2cb4100250 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2ca40099c0 tx=0x7f2ca4009cd0 comp rx=0 tx=0).stop 2023-07-21T18:40:34.051 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.888+0000 7f2cb9482700 1 -- 172.21.15.39:0/987231122 shutdown_connections 2023-07-21T18:40:34.051 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.888+0000 7f2cb9482700 1 --2- 172.21.15.39:0/987231122 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2cb4104ed0 0x7f2cb410ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:34.051 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.888+0000 7f2cb9482700 1 --2- 172.21.15.39:0/987231122 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2cb40ffe80 0x7f2cb4100250 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:34.051 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.888+0000 7f2cb9482700 1 --2- 172.21.15.39:0/987231122 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cb4100820 0x7f2cb41047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:34.051 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.888+0000 7f2cb9482700 1 -- 172.21.15.39:0/987231122 >> 172.21.15.39:0/987231122 conn(0x7f2cb4077ba0 msgr2=0x7f2cb4077fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:34.051 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.889+0000 7f2cb9482700 1 -- 172.21.15.39:0/987231122 shutdown_connections 2023-07-21T18:40:34.051 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.889+0000 7f2cb9482700 1 -- 172.21.15.39:0/987231122 wait complete. 2023-07-21T18:40:34.052 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.889+0000 7f2cb9482700 1 Processor -- start 2023-07-21T18:40:34.052 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.890+0000 7f2cb9482700 1 -- start start 2023-07-21T18:40:34.052 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.890+0000 7f2cb9482700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cb40ffe80 0x7f2cb419ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:34.052 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.890+0000 7f2cb9482700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2cb4100820 0x7f2cb419bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:34.052 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.890+0000 7f2cb9482700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2cb4104ed0 0x7f2cb4195aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:34.053 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.890+0000 7f2cb9482700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb4196110 con 0x7f2cb4104ed0 2023-07-21T18:40:34.053 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.890+0000 7f2cb9482700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb4196280 con 0x7f2cb40ffe80 2023-07-21T18:40:34.053 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.890+0000 7f2cb9482700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb4196440 con 0x7f2cb4100820 2023-07-21T18:40:34.053 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.890+0000 7f2cb37fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2cb4104ed0 0x7f2cb4195aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:34.054 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.890+0000 7f2cb37fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2cb4104ed0 0x7f2cb4195aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55450/0 (socket says 172.21.15.39:55450) 2023-07-21T18:40:34.054 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.890+0000 7f2cb37fe700 1 -- 172.21.15.39:0/1520666706 learned_addr learned my addr 172.21.15.39:0/1520666706 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:40:34.055 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.890+0000 7f2cb27fc700 1 --2- 172.21.15.39:0/1520666706 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2cb4100820 0x7f2cb419bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:34.055 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.890+0000 7f2cb2ffd700 1 --2- 172.21.15.39:0/1520666706 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cb40ffe80 0x7f2cb419ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:34.055 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.890+0000 7f2cb37fe700 1 -- 172.21.15.39:0/1520666706 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2cb4100820 msgr2=0x7f2cb419bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:34.055 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.890+0000 7f2cb37fe700 1 --2- 172.21.15.39:0/1520666706 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2cb4100820 0x7f2cb419bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:34.056 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.890+0000 7f2cb37fe700 1 -- 172.21.15.39:0/1520666706 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cb40ffe80 msgr2=0x7f2cb419ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:34.056 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.890+0000 7f2cb37fe700 1 --2- 172.21.15.39:0/1520666706 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cb40ffe80 0x7f2cb419ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:34.056 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.890+0000 7f2cb37fe700 1 -- 172.21.15.39:0/1520666706 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ca40095f0 con 0x7f2cb4104ed0 2023-07-21T18:40:34.057 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.890+0000 7f2cb2ffd700 1 --2- 172.21.15.39:0/1520666706 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cb40ffe80 0x7f2cb419ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:40:34.057 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.891+0000 7f2cb37fe700 1 --2- 172.21.15.39:0/1520666706 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2cb4104ed0 0x7f2cb4195aa0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2cac00ef30 tx=0x7f2cac00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:34.057 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.891+0000 7f2c9bfff700 1 -- 172.21.15.39:0/1520666706 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cac018070 con 0x7f2cb4104ed0 2023-07-21T18:40:34.058 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.891+0000 7f2cb9482700 1 -- 172.21.15.39:0/1520666706 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cb41966c0 con 0x7f2cb4104ed0 2023-07-21T18:40:34.058 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.891+0000 7f2c9bfff700 1 -- 172.21.15.39:0/1520666706 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2cac00f040 con 0x7f2cb4104ed0 2023-07-21T18:40:34.058 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.891+0000 7f2c9bfff700 1 -- 172.21.15.39:0/1520666706 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cac00cdb0 con 0x7f2cb4104ed0 2023-07-21T18:40:34.058 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.891+0000 7f2cb9482700 1 -- 172.21.15.39:0/1520666706 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cb41a2a10 con 0x7f2cb4104ed0 2023-07-21T18:40:34.059 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.894+0000 7f2cb9482700 1 -- 172.21.15.39:0/1520666706 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2cb410fda0 con 0x7f2cb4104ed0 2023-07-21T18:40:34.059 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.894+0000 7f2c9bfff700 1 -- 172.21.15.39:0/1520666706 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f2cac003a00 con 0x7f2cb4104ed0 2023-07-21T18:40:34.059 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.894+0000 7f2c9bfff700 1 --2- 172.21.15.39:0/1520666706 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f2c9c0a11e0 0x7f2c9c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:34.059 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.894+0000 7f2cb2ffd700 1 --2- 172.21.15.39:0/1520666706 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f2c9c0a11e0 0x7f2c9c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:34.060 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.894+0000 7f2c9bfff700 1 -- 172.21.15.39:0/1520666706 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f2cac0cc430 con 0x7f2cb4104ed0 2023-07-21T18:40:34.060 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.895+0000 7f2c9bfff700 1 --2- 172.21.15.39:0/1520666706 >> [v2:172.21.15.155:6808/3361780673,v1:172.21.15.155:6809/3361780673] conn(0x7f2c9c0aa9b0 0x7f2c9c0acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:34.060 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.895+0000 7f2c9bfff700 1 -- 172.21.15.39:0/1520666706 --> [v2:172.21.15.155:6808/3361780673,v1:172.21.15.155:6809/3361780673] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2c9c0ad470 con 0x7f2c9c0aa9b0 2023-07-21T18:40:34.060 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.895+0000 7f2c9bfff700 1 -- 172.21.15.39:0/1520666706 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2cac0ceb40 con 0x7f2cb4104ed0 2023-07-21T18:40:34.061 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.895+0000 7f2cb2ffd700 1 --2- 172.21.15.39:0/1520666706 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f2c9c0a11e0 0x7f2c9c0a3690 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4009990 tx=0x7f2ca401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:34.061 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.895+0000 7f2cb27fc700 1 --2- 172.21.15.39:0/1520666706 >> [v2:172.21.15.155:6808/3361780673,v1:172.21.15.155:6809/3361780673] conn(0x7f2c9c0aa9b0 0x7f2c9c0acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:34.061 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.895+0000 7f2cb27fc700 1 --2- 172.21.15.39:0/1520666706 >> [v2:172.21.15.155:6808/3361780673,v1:172.21.15.155:6809/3361780673] conn(0x7f2c9c0aa9b0 0x7f2c9c0acdc0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:34.061 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.896+0000 7f2c9bfff700 1 -- 172.21.15.39:0/1520666706 <== osd.4 v2:172.21.15.155:6808/3361780673 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f2c9c0ad470 con 0x7f2c9c0aa9b0 2023-07-21T18:40:34.062 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.912+0000 7f2cb9482700 1 -- 172.21.15.39:0/1520666706 --> [v2:172.21.15.155:6808/3361780673,v1:172.21.15.155:6809/3361780673] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2cb40fe150 con 0x7f2c9c0aa9b0 2023-07-21T18:40:34.062 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.913+0000 7f2c9bfff700 1 -- 172.21.15.39:0/1520666706 <== osd.4 v2:172.21.15.155:6808/3361780673 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f2cb40fe150 con 0x7f2c9c0aa9b0 2023-07-21T18:40:34.063 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.913+0000 7f2cb9482700 1 -- 172.21.15.39:0/1520666706 >> [v2:172.21.15.155:6808/3361780673,v1:172.21.15.155:6809/3361780673] conn(0x7f2c9c0aa9b0 msgr2=0x7f2c9c0acdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:34.063 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.913+0000 7f2cb9482700 1 --2- 172.21.15.39:0/1520666706 >> [v2:172.21.15.155:6808/3361780673,v1:172.21.15.155:6809/3361780673] conn(0x7f2c9c0aa9b0 0x7f2c9c0acdc0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:34.063 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.913+0000 7f2cb9482700 1 -- 172.21.15.39:0/1520666706 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f2c9c0a11e0 msgr2=0x7f2c9c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:34.063 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.913+0000 7f2cb9482700 1 --2- 172.21.15.39:0/1520666706 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f2c9c0a11e0 0x7f2c9c0a3690 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4009990 tx=0x7f2ca401a040 comp rx=0 tx=0).stop 2023-07-21T18:40:34.064 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.913+0000 7f2cb9482700 1 -- 172.21.15.39:0/1520666706 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2cb4104ed0 msgr2=0x7f2cb4195aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:34.064 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.913+0000 7f2cb9482700 1 --2- 172.21.15.39:0/1520666706 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2cb4104ed0 0x7f2cb4195aa0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2cac00ef30 tx=0x7f2cac00c3a0 comp rx=0 tx=0).stop 2023-07-21T18:40:34.064 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.913+0000 7f2cb37fe700 1 -- 172.21.15.39:0/1520666706 reap_dead start 2023-07-21T18:40:34.064 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.913+0000 7f2cb9482700 1 -- 172.21.15.39:0/1520666706 shutdown_connections 2023-07-21T18:40:34.065 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.913+0000 7f2cb9482700 1 -- 172.21.15.39:0/1520666706 >> 172.21.15.39:0/1520666706 conn(0x7f2cb4077ba0 msgr2=0x7f2cb40fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:34.065 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.914+0000 7f2cb9482700 1 -- 172.21.15.39:0/1520666706 shutdown_connections 2023-07-21T18:40:34.065 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:33.914+0000 7f2cb9482700 1 -- 172.21.15.39:0/1520666706 wait complete. 2023-07-21T18:40:34.272 INFO:teuthology.orchestra.run.smithi039.stdout:103079215154 2023-07-21T18:40:34.272 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.3 2023-07-21T18:40:34.488 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:34.464+0000 7f1c1568d700 1 -- 172.21.15.39:0/3690312965 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c100ff460 msgr2=0x7f1c1010b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:34.489 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:34.464+0000 7f1c1568d700 1 --2- 172.21.15.39:0/3690312965 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c100ff460 0x7f1c1010b8f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1c00009990 tx=0x7f1c00009ca0 comp rx=0 tx=0).stop 2023-07-21T18:40:34.489 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:34.464+0000 7f1c1568d700 1 -- 172.21.15.39:0/3690312965 shutdown_connections 2023-07-21T18:40:34.489 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:34.464+0000 7f1c1568d700 1 --2- 172.21.15.39:0/3690312965 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c100ff460 0x7f1c1010b8f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:34.489 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:34.464+0000 7f1c1568d700 1 --2- 172.21.15.39:0/3690312965 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c10104ab0 0x7f1c10104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:34.489 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:34.464+0000 7f1c1568d700 1 --2- 172.21.15.39:0/3690312965 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c100feab0 0x7f1c100fef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:34.490 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:34.464+0000 7f1c1568d700 1 -- 172.21.15.39:0/3690312965 >> 172.21.15.39:0/3690312965 conn(0x7f1c10075b70 msgr2=0x7f1c10075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:34.490 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:34.464+0000 7f1c1568d700 1 -- 172.21.15.39:0/3690312965 shutdown_connections 2023-07-21T18:40:34.490 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:34.464+0000 7f1c1568d700 1 -- 172.21.15.39:0/3690312965 wait complete. 2023-07-21T18:40:34.491 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:34.464+0000 7f1c1568d700 1 Processor -- start 2023-07-21T18:40:34.492 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:34.465+0000 7f1c1568d700 1 -- start start 2023-07-21T18:40:34.492 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:34.465+0000 7f1c1568d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c100feab0 0x7f1c10196970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:34.492 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:34.465+0000 7f1c1568d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c100ff460 0x7f1c10196eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:34.492 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:34.465+0000 7f1c1568d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c10104ab0 0x7f1c101a19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:34.492 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:34.465+0000 7f1c1568d700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c101977f0 con 0x7f1c100ff460 2023-07-21T18:40:34.493 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:34.465+0000 7f1c1568d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c101a1f30 con 0x7f1c10104ab0 2023-07-21T18:40:34.493 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:34.465+0000 7f1c1568d700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c101a20f0 con 0x7f1c100feab0 2023-07-21T18:40:34.493 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:34.465+0000 7f1c0e7fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c100ff460 0x7f1c10196eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:34.493 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:34.465+0000 7f1c0e7fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c100ff460 0x7f1c10196eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55464/0 (socket says 172.21.15.39:55464) 2023-07-21T18:40:34.494 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:34.465+0000 7f1c0e7fc700 1 -- 172.21.15.39:0/2927621850 learned_addr learned my addr 172.21.15.39:0/2927621850 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:40:34.494 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:34.465+0000 7f1c0effd700 1 --2- 172.21.15.39:0/2927621850 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c100feab0 0x7f1c10196970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:34.494 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:34.465+0000 7f1c0f7fe700 1 --2- 172.21.15.39:0/2927621850 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c10104ab0 0x7f1c101a19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:34.494 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:34.465+0000 7f1c0e7fc700 1 -- 172.21.15.39:0/2927621850 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c100feab0 msgr2=0x7f1c10196970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:34.494 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:34.465+0000 7f1c0e7fc700 1 --2- 172.21.15.39:0/2927621850 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c100feab0 0x7f1c10196970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:34.494 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:34.465+0000 7f1c0e7fc700 1 -- 172.21.15.39:0/2927621850 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c10104ab0 msgr2=0x7f1c101a19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:34.495 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:34.465+0000 7f1c0e7fc700 1 --2- 172.21.15.39:0/2927621850 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c10104ab0 0x7f1c101a19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:34.495 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:34.465+0000 7f1c0e7fc700 1 -- 172.21.15.39:0/2927621850 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c000095f0 con 0x7f1c100ff460 2023-07-21T18:40:34.495 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:34.465+0000 7f1c0f7fe700 1 --2- 172.21.15.39:0/2927621850 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c10104ab0 0x7f1c101a19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:40:34.495 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:34.465+0000 7f1c0e7fc700 1 --2- 172.21.15.39:0/2927621850 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c100ff460 0x7f1c10196eb0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1bf800d980 tx=0x7f1bf800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:34.495 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:34.465+0000 7f1c07fff700 1 -- 172.21.15.39:0/2927621850 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf800bb60 con 0x7f1c100ff460 2023-07-21T18:40:34.495 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:34.466+0000 7f1c1568d700 1 -- 172.21.15.39:0/2927621850 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c101a23d0 con 0x7f1c100ff460 2023-07-21T18:40:34.495 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:34.466+0000 7f1c07fff700 1 -- 172.21.15.39:0/2927621850 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1bf8007640 con 0x7f1c100ff460 2023-07-21T18:40:34.495 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:34.466+0000 7f1c07fff700 1 -- 172.21.15.39:0/2927621850 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf8004e90 con 0x7f1c100ff460 2023-07-21T18:40:34.496 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:34.466+0000 7f1c1568d700 1 -- 172.21.15.39:0/2927621850 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c101a28d0 con 0x7f1c100ff460 2023-07-21T18:40:34.496 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:34.466+0000 7f1c1568d700 1 -- 172.21.15.39:0/2927621850 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1c1004e210 con 0x7f1c100ff460 2023-07-21T18:40:34.498 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:34.468+0000 7f1c07fff700 1 -- 172.21.15.39:0/2927621850 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f1bf800bcc0 con 0x7f1c100ff460 2023-07-21T18:40:34.498 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:34.469+0000 7f1c07fff700 1 --2- 172.21.15.39:0/2927621850 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f1bfc0a11e0 0x7f1bfc0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:34.499 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:34.469+0000 7f1c07fff700 1 -- 172.21.15.39:0/2927621850 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f1bf80cceb0 con 0x7f1c100ff460 2023-07-21T18:40:34.499 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:34.469+0000 7f1c0effd700 1 --2- 172.21.15.39:0/2927621850 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f1bfc0a11e0 0x7f1bfc0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:34.499 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:34.469+0000 7f1c07fff700 1 --2- 172.21.15.39:0/2927621850 >> [v2:172.21.15.193:6816/2985835085,v1:172.21.15.193:6817/2985835085] conn(0x7f1bfc0aa840 0x7f1bfc0acc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:34.499 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:34.469+0000 7f1c07fff700 1 -- 172.21.15.39:0/2927621850 --> [v2:172.21.15.193:6816/2985835085,v1:172.21.15.193:6817/2985835085] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1bfc0ad300 con 0x7f1bfc0aa840 2023-07-21T18:40:34.499 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:34.469+0000 7f1c07fff700 1 -- 172.21.15.39:0/2927621850 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1bf80cd350 con 0x7f1c100ff460 2023-07-21T18:40:34.499 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:34.469+0000 7f1c0effd700 1 --2- 172.21.15.39:0/2927621850 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f1bfc0a11e0 0x7f1bfc0a3690 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1c100ffbf0 tx=0x7f1c08007ae0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:34.499 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:34.469+0000 7f1c0f7fe700 1 --2- 172.21.15.39:0/2927621850 >> [v2:172.21.15.193:6816/2985835085,v1:172.21.15.193:6817/2985835085] conn(0x7f1bfc0aa840 0x7f1bfc0acc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:34.500 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:34.470+0000 7f1c0f7fe700 1 --2- 172.21.15.39:0/2927621850 >> [v2:172.21.15.193:6816/2985835085,v1:172.21.15.193:6817/2985835085] conn(0x7f1bfc0aa840 0x7f1bfc0acc50 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:34.500 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:34.471+0000 7f1c07fff700 1 -- 172.21.15.39:0/2927621850 <== osd.8 v2:172.21.15.193:6816/2985835085 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f1bfc0ad300 con 0x7f1bfc0aa840 2023-07-21T18:40:34.514 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:34.488+0000 7f1c1568d700 1 -- 172.21.15.39:0/2927621850 --> [v2:172.21.15.193:6816/2985835085,v1:172.21.15.193:6817/2985835085] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1c1010ecb0 con 0x7f1bfc0aa840 2023-07-21T18:40:34.514 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:34.489+0000 7f1c07fff700 1 -- 172.21.15.39:0/2927621850 <== osd.8 v2:172.21.15.193:6816/2985835085 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f1c0001d040 con 0x7f1bfc0aa840 2023-07-21T18:40:34.516 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:34.489+0000 7f1c1568d700 1 -- 172.21.15.39:0/2927621850 >> [v2:172.21.15.193:6816/2985835085,v1:172.21.15.193:6817/2985835085] conn(0x7f1bfc0aa840 msgr2=0x7f1bfc0acc50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:34.516 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:34.489+0000 7f1c1568d700 1 --2- 172.21.15.39:0/2927621850 >> [v2:172.21.15.193:6816/2985835085,v1:172.21.15.193:6817/2985835085] conn(0x7f1bfc0aa840 0x7f1bfc0acc50 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:34.516 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:34.489+0000 7f1c1568d700 1 -- 172.21.15.39:0/2927621850 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f1bfc0a11e0 msgr2=0x7f1bfc0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:34.516 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:34.489+0000 7f1c1568d700 1 --2- 172.21.15.39:0/2927621850 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f1bfc0a11e0 0x7f1bfc0a3690 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1c100ffbf0 tx=0x7f1c08007ae0 comp rx=0 tx=0).stop 2023-07-21T18:40:34.517 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:34.489+0000 7f1c1568d700 1 -- 172.21.15.39:0/2927621850 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c100ff460 msgr2=0x7f1c10196eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:34.517 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:34.489+0000 7f1c1568d700 1 --2- 172.21.15.39:0/2927621850 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c100ff460 0x7f1c10196eb0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1bf800d980 tx=0x7f1bf800dd40 comp rx=0 tx=0).stop 2023-07-21T18:40:34.517 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:34.489+0000 7f1c0f7fe700 1 -- 172.21.15.39:0/2927621850 reap_dead start 2023-07-21T18:40:34.517 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:34.490+0000 7f1c1568d700 1 -- 172.21.15.39:0/2927621850 shutdown_connections 2023-07-21T18:40:34.518 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:34.490+0000 7f1c1568d700 1 -- 172.21.15.39:0/2927621850 >> 172.21.15.39:0/2927621850 conn(0x7f1c10075b70 msgr2=0x7f1c10102600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:34.518 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:34.490+0000 7f1c1568d700 1 -- 172.21.15.39:0/2927621850 shutdown_connections 2023-07-21T18:40:34.518 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:34.490+0000 7f1c1568d700 1 -- 172.21.15.39:0/2927621850 wait complete. 2023-07-21T18:40:34.523 INFO:teuthology.orchestra.run.smithi039.stdout:317827579921 2023-07-21T18:40:34.523 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.11 2023-07-21T18:40:35.138 INFO:teuthology.orchestra.run.smithi039.stdout:154618822698 2023-07-21T18:40:35.139 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.5 2023-07-21T18:40:35.343 INFO:teuthology.orchestra.run.smithi039.stdout:55834574907 2023-07-21T18:40:35.344 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.1 2023-07-21T18:40:35.371 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:35 smithi039 ceph-mon[96489]: pgmap v263: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 426 B/s wr, 11 op/s 2023-07-21T18:40:35.508 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:35.337+0000 7fb87e979700 1 -- 172.21.15.39:0/3579789501 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8780fe0e0 msgr2=0x7fb8780fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:35.509 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:35.337+0000 7fb87e979700 1 --2- 172.21.15.39:0/3579789501 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8780fe0e0 0x7fb8780fe550 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb86c009990 tx=0x7fb86c009ca0 comp rx=0 tx=0).stop 2023-07-21T18:40:35.509 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:35.337+0000 7fb87e979700 1 -- 172.21.15.39:0/3579789501 shutdown_connections 2023-07-21T18:40:35.509 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:35.337+0000 7fb87e979700 1 --2- 172.21.15.39:0/3579789501 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb878104b40 0x7fb878104f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:35.509 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:35.337+0000 7fb87e979700 1 --2- 172.21.15.39:0/3579789501 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8780fea90 0x7fb87810ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:35.509 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:35.337+0000 7fb87e979700 1 --2- 172.21.15.39:0/3579789501 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8780fe0e0 0x7fb8780fe550 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:35.509 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:35.337+0000 7fb87e979700 1 -- 172.21.15.39:0/3579789501 >> 172.21.15.39:0/3579789501 conn(0x7fb878075150 msgr2=0x7fb878075550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:35.510 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:35.337+0000 7fb87e979700 1 -- 172.21.15.39:0/3579789501 shutdown_connections 2023-07-21T18:40:35.510 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:35.337+0000 7fb87e979700 1 -- 172.21.15.39:0/3579789501 wait complete. 2023-07-21T18:40:35.510 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:35.337+0000 7fb87e979700 1 Processor -- start 2023-07-21T18:40:35.510 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:35.337+0000 7fb87e979700 1 -- start start 2023-07-21T18:40:35.510 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:35.338+0000 7fb87e979700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8780fe0e0 0x7fb87819b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:35.510 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:35.338+0000 7fb87e979700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8780fea90 0x7fb87819bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:35.510 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:35.338+0000 7fb87e979700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb878104b40 0x7fb878195a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:35.510 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:35.338+0000 7fb87e979700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8781960a0 con 0x7fb8780fea90 2023-07-21T18:40:35.511 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:35.338+0000 7fb87e979700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb878196210 con 0x7fb878104b40 2023-07-21T18:40:35.511 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:35.338+0000 7fb87e979700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8781963d0 con 0x7fb8780fe0e0 2023-07-21T18:40:35.511 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:35.338+0000 7fb8777fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8780fea90 0x7fb87819bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:35.511 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:35.338+0000 7fb8777fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8780fea90 0x7fb87819bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55478/0 (socket says 172.21.15.39:55478) 2023-07-21T18:40:35.511 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:35.338+0000 7fb8777fe700 1 -- 172.21.15.39:0/224236334 learned_addr learned my addr 172.21.15.39:0/224236334 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:40:35.511 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:35.338+0000 7fb877fff700 1 --2- 172.21.15.39:0/224236334 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8780fe0e0 0x7fb87819b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:35.511 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:35.338+0000 7fb87cf16700 1 --2- 172.21.15.39:0/224236334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb878104b40 0x7fb878195a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:35.512 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:35.338+0000 7fb8777fe700 1 -- 172.21.15.39:0/224236334 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8780fe0e0 msgr2=0x7fb87819b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:35.512 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:35.338+0000 7fb8777fe700 1 --2- 172.21.15.39:0/224236334 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8780fe0e0 0x7fb87819b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:35.512 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:35.338+0000 7fb8777fe700 1 -- 172.21.15.39:0/224236334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb878104b40 msgr2=0x7fb878195a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:35.512 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:35.338+0000 7fb8777fe700 1 --2- 172.21.15.39:0/224236334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb878104b40 0x7fb878195a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:35.512 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:35.338+0000 7fb8777fe700 1 -- 172.21.15.39:0/224236334 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb86c0095f0 con 0x7fb8780fea90 2023-07-21T18:40:35.512 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:35.338+0000 7fb8777fe700 1 --2- 172.21.15.39:0/224236334 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8780fea90 0x7fb87819bef0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb86c0095c0 tx=0x7fb86c00f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:35.512 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:35.339+0000 7fb8757fa700 1 -- 172.21.15.39:0/224236334 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86c01d070 con 0x7fb8780fea90 2023-07-21T18:40:35.513 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:35.339+0000 7fb87e979700 1 -- 172.21.15.39:0/224236334 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb878196650 con 0x7fb8780fea90 2023-07-21T18:40:35.513 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:35.339+0000 7fb8757fa700 1 -- 172.21.15.39:0/224236334 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb86c0054f0 con 0x7fb8780fea90 2023-07-21T18:40:35.513 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:35.339+0000 7fb8757fa700 1 -- 172.21.15.39:0/224236334 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86c004bd0 con 0x7fb8780fea90 2023-07-21T18:40:35.513 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:35.339+0000 7fb87e979700 1 -- 172.21.15.39:0/224236334 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8781a29a0 con 0x7fb8780fea90 2023-07-21T18:40:35.513 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:35.340+0000 7fb87e979700 1 -- 172.21.15.39:0/224236334 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb87810fe80 con 0x7fb8780fea90 2023-07-21T18:40:35.513 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:35.341+0000 7fb8757fa700 1 -- 172.21.15.39:0/224236334 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fb86c00fc30 con 0x7fb8780fea90 2023-07-21T18:40:35.513 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:35.342+0000 7fb8757fa700 1 --2- 172.21.15.39:0/224236334 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fb8600a11e0 0x7fb8600a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:35.513 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:35.342+0000 7fb8757fa700 1 -- 172.21.15.39:0/224236334 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb86c0ce0a0 con 0x7fb8780fea90 2023-07-21T18:40:35.514 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:35.342+0000 7fb877fff700 1 --2- 172.21.15.39:0/224236334 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fb8600a11e0 0x7fb8600a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:35.514 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:35.342+0000 7fb8757fa700 1 --2- 172.21.15.39:0/224236334 >> [v2:172.21.15.155:6816/2542002111,v1:172.21.15.155:6817/2542002111] conn(0x7fb8600aa9b0 0x7fb8600acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:35.514 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:35.342+0000 7fb8757fa700 1 -- 172.21.15.39:0/224236334 --> [v2:172.21.15.155:6816/2542002111,v1:172.21.15.155:6817/2542002111] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb8600ad470 con 0x7fb8600aa9b0 2023-07-21T18:40:35.514 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:35.342+0000 7fb8757fa700 1 -- 172.21.15.39:0/224236334 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb86c0ce410 con 0x7fb8780fea90 2023-07-21T18:40:35.514 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:35.342+0000 7fb877fff700 1 --2- 172.21.15.39:0/224236334 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fb8600a11e0 0x7fb8600a3690 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb868005ea0 tx=0x7fb868005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:35.514 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:35.342+0000 7fb87cf16700 1 --2- 172.21.15.39:0/224236334 >> [v2:172.21.15.155:6816/2542002111,v1:172.21.15.155:6817/2542002111] conn(0x7fb8600aa9b0 0x7fb8600acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:35.514 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:35.343+0000 7fb87cf16700 1 --2- 172.21.15.39:0/224236334 >> [v2:172.21.15.155:6816/2542002111,v1:172.21.15.155:6817/2542002111] conn(0x7fb8600aa9b0 0x7fb8600acdc0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:35.515 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:35.344+0000 7fb8757fa700 1 -- 172.21.15.39:0/224236334 <== osd.7 v2:172.21.15.155:6816/2542002111 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fb8600ad470 con 0x7fb8600aa9b0 2023-07-21T18:40:35.515 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:35.360+0000 7fb87e979700 1 -- 172.21.15.39:0/224236334 --> [v2:172.21.15.155:6816/2542002111,v1:172.21.15.155:6817/2542002111] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb8780fd210 con 0x7fb8600aa9b0 2023-07-21T18:40:35.515 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:35.361+0000 7fb8757fa700 1 -- 172.21.15.39:0/224236334 <== osd.7 v2:172.21.15.155:6816/2542002111 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fb8780fd210 con 0x7fb8600aa9b0 2023-07-21T18:40:35.515 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:35.361+0000 7fb87e979700 1 -- 172.21.15.39:0/224236334 >> [v2:172.21.15.155:6816/2542002111,v1:172.21.15.155:6817/2542002111] conn(0x7fb8600aa9b0 msgr2=0x7fb8600acdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:35.515 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:35.361+0000 7fb87e979700 1 --2- 172.21.15.39:0/224236334 >> [v2:172.21.15.155:6816/2542002111,v1:172.21.15.155:6817/2542002111] conn(0x7fb8600aa9b0 0x7fb8600acdc0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:35.516 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:35.361+0000 7fb87e979700 1 -- 172.21.15.39:0/224236334 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fb8600a11e0 msgr2=0x7fb8600a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:35.516 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:35.361+0000 7fb87e979700 1 --2- 172.21.15.39:0/224236334 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fb8600a11e0 0x7fb8600a3690 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb868005ea0 tx=0x7fb868005e30 comp rx=0 tx=0).stop 2023-07-21T18:40:35.516 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:35.361+0000 7fb87e979700 1 -- 172.21.15.39:0/224236334 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8780fea90 msgr2=0x7fb87819bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:35.516 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:35.361+0000 7fb87e979700 1 --2- 172.21.15.39:0/224236334 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8780fea90 0x7fb87819bef0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb86c0095c0 tx=0x7fb86c00f740 comp rx=0 tx=0).stop 2023-07-21T18:40:35.516 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:35.361+0000 7fb87cf16700 1 -- 172.21.15.39:0/224236334 reap_dead start 2023-07-21T18:40:35.516 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:35.362+0000 7fb87e979700 1 -- 172.21.15.39:0/224236334 shutdown_connections 2023-07-21T18:40:35.516 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:35.362+0000 7fb87e979700 1 -- 172.21.15.39:0/224236334 >> 172.21.15.39:0/224236334 conn(0x7fb878075150 msgr2=0x7fb8780fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:35.517 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:35.362+0000 7fb87e979700 1 -- 172.21.15.39:0/224236334 shutdown_connections 2023-07-21T18:40:35.517 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:35.362+0000 7fb87e979700 1 -- 172.21.15.39:0/224236334 wait complete. 2023-07-21T18:40:35.622 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:35 smithi193 ceph-mon[98204]: pgmap v263: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 426 B/s wr, 11 op/s 2023-07-21T18:40:35.641 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:35 smithi155 ceph-mon[97934]: pgmap v263: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 426 B/s wr, 11 op/s 2023-07-21T18:40:35.674 INFO:teuthology.orchestra.run.smithi039.stdout:236223201310 2023-07-21T18:40:35.675 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.8 2023-07-21T18:40:35.675 INFO:teuthology.orchestra.run.smithi039.stdout:133143986223 2023-07-21T18:40:35.676 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.4 2023-07-21T18:40:36.140 INFO:teuthology.orchestra.run.smithi039.stdout:206158430242 2023-07-21T18:40:36.141 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.7 2023-07-21T18:40:36.475 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:36.419+0000 7f4c32ca0700 1 -- 172.21.15.39:0/976955535 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c2c104ab0 msgr2=0x7f4c2c104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:36.476 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:36.419+0000 7f4c32ca0700 1 --2- 172.21.15.39:0/976955535 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c2c104ab0 0x7f4c2c104e80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4c20009990 tx=0x7f4c20009ca0 comp rx=0 tx=0).stop 2023-07-21T18:40:36.476 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:36.419+0000 7f4c32ca0700 1 -- 172.21.15.39:0/976955535 shutdown_connections 2023-07-21T18:40:36.476 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:36.419+0000 7f4c32ca0700 1 --2- 172.21.15.39:0/976955535 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4c2c0ff460 0x7f4c2c10b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:36.476 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:36.419+0000 7f4c32ca0700 1 --2- 172.21.15.39:0/976955535 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c2c104ab0 0x7f4c2c104e80 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:36.476 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:36.419+0000 7f4c32ca0700 1 --2- 172.21.15.39:0/976955535 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c2c0feab0 0x7f4c2c0fef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:36.476 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:36.419+0000 7f4c32ca0700 1 -- 172.21.15.39:0/976955535 >> 172.21.15.39:0/976955535 conn(0x7f4c2c075b70 msgr2=0x7f4c2c075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:36.476 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:36.419+0000 7f4c32ca0700 1 -- 172.21.15.39:0/976955535 shutdown_connections 2023-07-21T18:40:36.477 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:36.419+0000 7f4c32ca0700 1 -- 172.21.15.39:0/976955535 wait complete. 2023-07-21T18:40:36.477 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:36.420+0000 7f4c32ca0700 1 Processor -- start 2023-07-21T18:40:36.477 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:36.420+0000 7f4c32ca0700 1 -- start start 2023-07-21T18:40:36.477 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:36.420+0000 7f4c32ca0700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c2c0feab0 0x7f4c2c196ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:36.477 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:36.420+0000 7f4c32ca0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c2c0ff460 0x7f4c2c197000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:36.477 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:36.420+0000 7f4c32ca0700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4c2c104ab0 0x7f4c2c1a1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:36.477 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:36.420+0000 7f4c32ca0700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2c197910 con 0x7f4c2c0feab0 2023-07-21T18:40:36.478 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:36.420+0000 7f4c32ca0700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2c1a2080 con 0x7f4c2c0ff460 2023-07-21T18:40:36.478 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:36.420+0000 7f4c32ca0700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2c1a21f0 con 0x7f4c2c104ab0 2023-07-21T18:40:36.478 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:36.420+0000 7f4c30a3c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c2c0feab0 0x7f4c2c196ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:36.478 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:36.420+0000 7f4c30a3c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c2c0feab0 0x7f4c2c196ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55492/0 (socket says 172.21.15.39:55492) 2023-07-21T18:40:36.478 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:36.420+0000 7f4c30a3c700 1 -- 172.21.15.39:0/607062323 learned_addr learned my addr 172.21.15.39:0/607062323 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:40:36.478 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:36.420+0000 7f4c3123d700 1 --2- 172.21.15.39:0/607062323 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4c2c104ab0 0x7f4c2c1a1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:36.478 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:36.420+0000 7f4c2bfff700 1 --2- 172.21.15.39:0/607062323 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c2c0ff460 0x7f4c2c197000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:36.478 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:36.420+0000 7f4c30a3c700 1 -- 172.21.15.39:0/607062323 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4c2c104ab0 msgr2=0x7f4c2c1a1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:36.479 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:36.420+0000 7f4c30a3c700 1 --2- 172.21.15.39:0/607062323 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4c2c104ab0 0x7f4c2c1a1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:36.479 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:36.420+0000 7f4c30a3c700 1 -- 172.21.15.39:0/607062323 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c2c0ff460 msgr2=0x7f4c2c197000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:36.479 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:36.420+0000 7f4c30a3c700 1 --2- 172.21.15.39:0/607062323 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c2c0ff460 0x7f4c2c197000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:36.479 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:36.421+0000 7f4c30a3c700 1 -- 172.21.15.39:0/607062323 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c200095f0 con 0x7f4c2c0feab0 2023-07-21T18:40:36.479 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:36.421+0000 7f4c30a3c700 1 --2- 172.21.15.39:0/607062323 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c2c0feab0 0x7f4c2c196ac0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4c20003730 tx=0x7f4c2000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:36.479 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:36.421+0000 7f4c29ffb700 1 -- 172.21.15.39:0/607062323 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2001d070 con 0x7f4c2c0feab0 2023-07-21T18:40:36.479 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:36.421+0000 7f4c32ca0700 1 -- 172.21.15.39:0/607062323 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c2c1a2470 con 0x7f4c2c0feab0 2023-07-21T18:40:36.480 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:36.421+0000 7f4c29ffb700 1 -- 172.21.15.39:0/607062323 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c20004c90 con 0x7f4c2c0feab0 2023-07-21T18:40:36.480 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:36.421+0000 7f4c29ffb700 1 -- 172.21.15.39:0/607062323 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c20005360 con 0x7f4c2c0feab0 2023-07-21T18:40:36.480 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:36.421+0000 7f4c32ca0700 1 -- 172.21.15.39:0/607062323 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c2c1a28e0 con 0x7f4c2c0feab0 2023-07-21T18:40:36.480 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:36.422+0000 7f4c32ca0700 1 -- 172.21.15.39:0/607062323 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4c2c04e210 con 0x7f4c2c0feab0 2023-07-21T18:40:36.480 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:36.423+0000 7f4c29ffb700 1 -- 172.21.15.39:0/607062323 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f4c200054e0 con 0x7f4c2c0feab0 2023-07-21T18:40:36.480 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:36.424+0000 7f4c29ffb700 1 --2- 172.21.15.39:0/607062323 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f4c1c0a11e0 0x7f4c1c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:36.481 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:36.424+0000 7f4c29ffb700 1 -- 172.21.15.39:0/607062323 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f4c200cdef0 con 0x7f4c2c0feab0 2023-07-21T18:40:36.481 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:36.424+0000 7f4c2bfff700 1 --2- 172.21.15.39:0/607062323 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f4c1c0a11e0 0x7f4c1c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:36.481 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:36.424+0000 7f4c29ffb700 1 --2- 172.21.15.39:0/607062323 >> [v2:172.21.15.39:6826/1946493651,v1:172.21.15.39:6827/1946493651] conn(0x7f4c1c0aa9b0 0x7f4c1c0acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:36.481 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:36.424+0000 7f4c29ffb700 1 -- 172.21.15.39:0/607062323 --> [v2:172.21.15.39:6826/1946493651,v1:172.21.15.39:6827/1946493651] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4c1c0ad470 con 0x7f4c1c0aa9b0 2023-07-21T18:40:36.481 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:36.424+0000 7f4c29ffb700 1 -- 172.21.15.39:0/607062323 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c200ce390 con 0x7f4c2c0feab0 2023-07-21T18:40:36.481 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:36.424+0000 7f4c3123d700 1 --2- 172.21.15.39:0/607062323 >> [v2:172.21.15.39:6826/1946493651,v1:172.21.15.39:6827/1946493651] conn(0x7f4c1c0aa9b0 0x7f4c1c0acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:36.482 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:36.424+0000 7f4c2bfff700 1 --2- 172.21.15.39:0/607062323 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f4c1c0a11e0 0x7f4c1c0a3690 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4c18005950 tx=0x7f4c180058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:36.482 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:36.425+0000 7f4c3123d700 1 --2- 172.21.15.39:0/607062323 >> [v2:172.21.15.39:6826/1946493651,v1:172.21.15.39:6827/1946493651] conn(0x7f4c1c0aa9b0 0x7f4c1c0acdc0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:36.482 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:36.425+0000 7f4c29ffb700 1 -- 172.21.15.39:0/607062323 <== osd.9 v2:172.21.15.39:6826/1946493651 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f4c1c0ad470 con 0x7f4c1c0aa9b0 2023-07-21T18:40:36.482 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:36.443+0000 7f4c32ca0700 1 -- 172.21.15.39:0/607062323 --> [v2:172.21.15.39:6826/1946493651,v1:172.21.15.39:6827/1946493651] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4c2c10eca0 con 0x7f4c1c0aa9b0 2023-07-21T18:40:36.482 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:36.443+0000 7f4c29ffb700 1 -- 172.21.15.39:0/607062323 <== osd.9 v2:172.21.15.39:6826/1946493651 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f4c2400f040 con 0x7f4c1c0aa9b0 2023-07-21T18:40:36.482 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:36.443+0000 7f4c32ca0700 1 -- 172.21.15.39:0/607062323 >> [v2:172.21.15.39:6826/1946493651,v1:172.21.15.39:6827/1946493651] conn(0x7f4c1c0aa9b0 msgr2=0x7f4c1c0acdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:36.482 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:36.443+0000 7f4c32ca0700 1 --2- 172.21.15.39:0/607062323 >> [v2:172.21.15.39:6826/1946493651,v1:172.21.15.39:6827/1946493651] conn(0x7f4c1c0aa9b0 0x7f4c1c0acdc0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:36.482 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:36.444+0000 7f4c32ca0700 1 -- 172.21.15.39:0/607062323 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f4c1c0a11e0 msgr2=0x7f4c1c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:36.483 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:36.444+0000 7f4c32ca0700 1 --2- 172.21.15.39:0/607062323 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f4c1c0a11e0 0x7f4c1c0a3690 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4c18005950 tx=0x7f4c180058e0 comp rx=0 tx=0).stop 2023-07-21T18:40:36.483 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:36.444+0000 7f4c32ca0700 1 -- 172.21.15.39:0/607062323 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c2c0feab0 msgr2=0x7f4c2c196ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:36.483 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:36.444+0000 7f4c32ca0700 1 --2- 172.21.15.39:0/607062323 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c2c0feab0 0x7f4c2c196ac0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4c20003730 tx=0x7f4c2000f690 comp rx=0 tx=0).stop 2023-07-21T18:40:36.483 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:36.444+0000 7f4c3123d700 1 -- 172.21.15.39:0/607062323 reap_dead start 2023-07-21T18:40:36.483 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:36.444+0000 7f4c32ca0700 1 -- 172.21.15.39:0/607062323 shutdown_connections 2023-07-21T18:40:36.483 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:36.444+0000 7f4c32ca0700 1 -- 172.21.15.39:0/607062323 >> 172.21.15.39:0/607062323 conn(0x7f4c2c075b70 msgr2=0x7f4c2c0fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:36.483 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:36.444+0000 7f4c32ca0700 1 -- 172.21.15.39:0/607062323 shutdown_connections 2023-07-21T18:40:36.483 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:36.444+0000 7f4c32ca0700 1 -- 172.21.15.39:0/607062323 wait complete. 2023-07-21T18:40:36.595 INFO:teuthology.orchestra.run.smithi039.stdout:184683593766 2023-07-21T18:40:36.595 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.6 2023-07-21T18:40:36.841 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:40:37.305 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:40:37.347 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:40:37.393 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:40:37.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:37 smithi039 ceph-mon[96489]: pgmap v264: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 426 B/s wr, 9 op/s 2023-07-21T18:40:37.571 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:40:37.622 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:37 smithi193 ceph-mon[98204]: pgmap v264: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 426 B/s wr, 9 op/s 2023-07-21T18:40:37.641 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:37 smithi155 ceph-mon[97934]: pgmap v264: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 426 B/s wr, 9 op/s 2023-07-21T18:40:37.827 INFO:teuthology.orchestra.run.smithi039.stdout:266287972378 2023-07-21T18:40:37.828 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.9 2023-07-21T18:40:37.928 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:40:38.233 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:40:38.345 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:40:39.264 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:39 smithi039 ceph-mon[96489]: pgmap v265: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 426 B/s wr, 11 op/s 2023-07-21T18:40:39.622 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:39 smithi193 ceph-mon[98204]: pgmap v265: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 426 B/s wr, 11 op/s 2023-07-21T18:40:39.641 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:39 smithi155 ceph-mon[97934]: pgmap v265: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 426 B/s wr, 11 op/s 2023-07-21T18:40:40.221 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:40:40.833 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:40 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/77778717' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-07-21T18:40:40.891 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:40 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/77778717' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-07-21T18:40:41.122 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:40 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/77778717' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-07-21T18:40:41.407 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:41.382+0000 7f0da355a700 1 -- 172.21.15.39:0/238197689 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d9c0fee80 msgr2=0x7f0d9c0ff250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:41.407 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:41.382+0000 7f0da355a700 1 --2- 172.21.15.39:0/238197689 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d9c0fee80 0x7f0d9c0ff250 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0d980099c0 tx=0x7f0d98009cd0 comp rx=0 tx=0).stop 2023-07-21T18:40:41.407 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:41.382+0000 7f0da355a700 1 -- 172.21.15.39:0/238197689 shutdown_connections 2023-07-21T18:40:41.407 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:41.382+0000 7f0da355a700 1 --2- 172.21.15.39:0/238197689 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d9c100f80 0x7f0d9c10ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:41.408 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:41.382+0000 7f0da355a700 1 --2- 172.21.15.39:0/238197689 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d9c0fee80 0x7f0d9c0ff250 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:41.408 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:41.382+0000 7f0da355a700 1 --2- 172.21.15.39:0/238197689 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d9c1005d0 0x7f0d9c100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:41.408 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:41.382+0000 7f0da355a700 1 -- 172.21.15.39:0/238197689 >> 172.21.15.39:0/238197689 conn(0x7f0d9c075b60 msgr2=0x7f0d9c075f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:41.408 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:41.382+0000 7f0da355a700 1 -- 172.21.15.39:0/238197689 shutdown_connections 2023-07-21T18:40:41.408 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:41.382+0000 7f0da355a700 1 -- 172.21.15.39:0/238197689 wait complete. 2023-07-21T18:40:41.409 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:41.383+0000 7f0da355a700 1 Processor -- start 2023-07-21T18:40:41.410 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:41.383+0000 7f0da355a700 1 -- start start 2023-07-21T18:40:41.410 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:41.383+0000 7f0da355a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d9c0fee80 0x7f0d9c196b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:41.410 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:41.383+0000 7f0da355a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d9c1005d0 0x7f0d9c197070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:41.410 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:41.383+0000 7f0da355a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d9c100f80 0x7f0d9c1a1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:41.410 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:41.383+0000 7f0da355a700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d9c1979b0 con 0x7f0d9c1005d0 2023-07-21T18:40:41.410 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:41.383+0000 7f0da355a700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d9c1a20f0 con 0x7f0d9c100f80 2023-07-21T18:40:41.411 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:41.383+0000 7f0da355a700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d9c1a22b0 con 0x7f0d9c0fee80 2023-07-21T18:40:41.411 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:41.383+0000 7f0da0af5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d9c1005d0 0x7f0d9c197070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:41.411 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:41.383+0000 7f0da0af5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d9c1005d0 0x7f0d9c197070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55524/0 (socket says 172.21.15.39:55524) 2023-07-21T18:40:41.411 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:41.383+0000 7f0da0af5700 1 -- 172.21.15.39:0/2167509777 learned_addr learned my addr 172.21.15.39:0/2167509777 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:40:41.411 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:41.383+0000 7f0da1af7700 1 --2- 172.21.15.39:0/2167509777 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d9c100f80 0x7f0d9c1a1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:41.411 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:41.383+0000 7f0da12f6700 1 --2- 172.21.15.39:0/2167509777 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d9c0fee80 0x7f0d9c196b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:41.411 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:41.384+0000 7f0da0af5700 1 -- 172.21.15.39:0/2167509777 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d9c0fee80 msgr2=0x7f0d9c196b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:41.411 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:41.384+0000 7f0da0af5700 1 --2- 172.21.15.39:0/2167509777 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d9c0fee80 0x7f0d9c196b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:41.412 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:41.384+0000 7f0da0af5700 1 -- 172.21.15.39:0/2167509777 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d9c100f80 msgr2=0x7f0d9c1a1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:41.412 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:41.384+0000 7f0da0af5700 1 --2- 172.21.15.39:0/2167509777 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d9c100f80 0x7f0d9c1a1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:41.412 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:41.384+0000 7f0da0af5700 1 -- 172.21.15.39:0/2167509777 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d980095f0 con 0x7f0d9c1005d0 2023-07-21T18:40:41.412 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:41.384+0000 7f0da0af5700 1 --2- 172.21.15.39:0/2167509777 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d9c1005d0 0x7f0d9c197070 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c00d950 tx=0x7f0d8c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:41.412 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:41.384+0000 7f0d927fc700 1 -- 172.21.15.39:0/2167509777 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c009940 con 0x7f0d9c1005d0 2023-07-21T18:40:41.412 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:41.384+0000 7f0d927fc700 1 -- 172.21.15.39:0/2167509777 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d8c010460 con 0x7f0d9c1005d0 2023-07-21T18:40:41.412 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:41.384+0000 7f0da355a700 1 -- 172.21.15.39:0/2167509777 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d9c1a2590 con 0x7f0d9c1005d0 2023-07-21T18:40:41.412 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:41.384+0000 7f0d927fc700 1 -- 172.21.15.39:0/2167509777 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c00f620 con 0x7f0d9c1005d0 2023-07-21T18:40:41.413 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:41.384+0000 7f0da355a700 1 -- 172.21.15.39:0/2167509777 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d9c1a2a90 con 0x7f0d9c1005d0 2023-07-21T18:40:41.413 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:41.386+0000 7f0da355a700 1 -- 172.21.15.39:0/2167509777 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d9c1070b0 con 0x7f0d9c1005d0 2023-07-21T18:40:41.414 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:41.386+0000 7f0d927fc700 1 -- 172.21.15.39:0/2167509777 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f0d8c0108d0 con 0x7f0d9c1005d0 2023-07-21T18:40:41.414 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:41.387+0000 7f0d927fc700 1 --2- 172.21.15.39:0/2167509777 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f0d880a11e0 0x7f0d880a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:41.414 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:41.387+0000 7f0da12f6700 1 --2- 172.21.15.39:0/2167509777 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f0d880a11e0 0x7f0d880a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:41.414 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:41.387+0000 7f0d927fc700 1 -- 172.21.15.39:0/2167509777 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0d8c0cc760 con 0x7f0d9c1005d0 2023-07-21T18:40:41.414 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:41.387+0000 7f0da12f6700 1 --2- 172.21.15.39:0/2167509777 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f0d880a11e0 0x7f0d880a3690 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0d98009fd0 tx=0x7f0d98005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:41.415 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:41.391+0000 7f0d927fc700 1 -- 172.21.15.39:0/2167509777 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f0d8c095f00 con 0x7f0d9c1005d0 2023-07-21T18:40:41.618 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:41.594+0000 7f0da355a700 1 -- 172.21.15.39:0/2167509777 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f0d9c068f10 con 0x7f0d9c1005d0 2023-07-21T18:40:41.619 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:41.594+0000 7f0d927fc700 1 -- 172.21.15.39:0/2167509777 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0d8c099520 con 0x7f0d9c1005d0 2023-07-21T18:40:41.619 INFO:teuthology.orchestra.run.smithi039.stdout:154618822699 2023-07-21T18:40:41.621 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:41.596+0000 7f0da355a700 1 -- 172.21.15.39:0/2167509777 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f0d880a11e0 msgr2=0x7f0d880a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:41.621 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:41.596+0000 7f0da355a700 1 --2- 172.21.15.39:0/2167509777 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f0d880a11e0 0x7f0d880a3690 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0d98009fd0 tx=0x7f0d98005fb0 comp rx=0 tx=0).stop 2023-07-21T18:40:41.621 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:41.596+0000 7f0da355a700 1 -- 172.21.15.39:0/2167509777 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d9c1005d0 msgr2=0x7f0d9c197070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:41.621 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:41.596+0000 7f0da355a700 1 --2- 172.21.15.39:0/2167509777 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d9c1005d0 0x7f0d9c197070 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c00d950 tx=0x7f0d8c00dc60 comp rx=0 tx=0).stop 2023-07-21T18:40:41.622 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:41.597+0000 7f0da355a700 1 -- 172.21.15.39:0/2167509777 shutdown_connections 2023-07-21T18:40:41.622 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:41.597+0000 7f0da355a700 1 --2- 172.21.15.39:0/2167509777 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f0d880a11e0 0x7f0d880a3690 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:41.622 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:41.597+0000 7f0da355a700 1 --2- 172.21.15.39:0/2167509777 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d9c100f80 0x7f0d9c1a1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:41.622 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:41.597+0000 7f0da355a700 1 --2- 172.21.15.39:0/2167509777 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d9c0fee80 0x7f0d9c196b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:41.622 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:41.597+0000 7f0da355a700 1 --2- 172.21.15.39:0/2167509777 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d9c1005d0 0x7f0d9c197070 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:41.622 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:41.597+0000 7f0da355a700 1 -- 172.21.15.39:0/2167509777 >> 172.21.15.39:0/2167509777 conn(0x7f0d9c075b60 msgr2=0x7f0d9c1018e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:41.622 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:41.597+0000 7f0da355a700 1 -- 172.21.15.39:0/2167509777 shutdown_connections 2023-07-21T18:40:41.623 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:41.597+0000 7f0da355a700 1 -- 172.21.15.39:0/2167509777 wait complete. 2023-07-21T18:40:41.640 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:40.334+0000 7f406beef700 1 -- 172.21.15.39:0/2661762161 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4064104ea0 msgr2=0x7f406410ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:41.641 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:40.334+0000 7f406beef700 1 --2- 172.21.15.39:0/2661762161 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4064104ea0 0x7f406410ca20 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f405c00b0d0 tx=0x7f405c00b3e0 comp rx=0 tx=0).stop 2023-07-21T18:40:41.641 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:40.334+0000 7f406beef700 1 -- 172.21.15.39:0/2661762161 shutdown_connections 2023-07-21T18:40:41.641 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:40.334+0000 7f406beef700 1 --2- 172.21.15.39:0/2661762161 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4064104ea0 0x7f406410ca20 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:41.641 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:40.334+0000 7f406beef700 1 --2- 172.21.15.39:0/2661762161 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40640ffe00 0x7f40641001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:41.641 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:40.334+0000 7f406beef700 1 --2- 172.21.15.39:0/2661762161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40641007a0 0x7f4064104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:41.641 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:40.334+0000 7f406beef700 1 -- 172.21.15.39:0/2661762161 >> 172.21.15.39:0/2661762161 conn(0x7f40640fb6a0 msgr2=0x7f40640fdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:41.641 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:40.334+0000 7f406beef700 1 -- 172.21.15.39:0/2661762161 shutdown_connections 2023-07-21T18:40:41.641 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:40.334+0000 7f406beef700 1 -- 172.21.15.39:0/2661762161 wait complete. 2023-07-21T18:40:41.642 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:40.335+0000 7f406beef700 1 Processor -- start 2023-07-21T18:40:41.642 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:40.335+0000 7f406beef700 1 -- start start 2023-07-21T18:40:41.642 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:40.335+0000 7f406beef700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40640ffe00 0x7f4064195e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:41.642 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:40.335+0000 7f406beef700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40641007a0 0x7f40641963d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:41.642 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:40.335+0000 7f406beef700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4064104ea0 0x7f406419a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:41.642 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:40.335+0000 7f406beef700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f406419ae50 con 0x7f4064104ea0 2023-07-21T18:40:41.642 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:40.335+0000 7f406beef700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f406419afc0 con 0x7f40641007a0 2023-07-21T18:40:41.643 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:40.335+0000 7f406beef700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f406419b180 con 0x7f40640ffe00 2023-07-21T18:40:41.643 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:40.335+0000 7f406a48c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4064104ea0 0x7f406419a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:41.643 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:40.335+0000 7f406a48c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4064104ea0 0x7f406419a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55512/0 (socket says 172.21.15.39:55512) 2023-07-21T18:40:41.643 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:40.335+0000 7f406a48c700 1 -- 172.21.15.39:0/77778717 learned_addr learned my addr 172.21.15.39:0/77778717 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:40:41.643 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:40.335+0000 7f4069c8b700 1 --2- 172.21.15.39:0/77778717 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40640ffe00 0x7f4064195e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:41.643 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:40.335+0000 7f406948a700 1 --2- 172.21.15.39:0/77778717 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40641007a0 0x7f40641963d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:41.644 INFO:teuthology.orchestra.run.smithi039.stdout:103079215155 2023-07-21T18:40:41.644 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:40.335+0000 7f406a48c700 1 -- 172.21.15.39:0/77778717 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40640ffe00 msgr2=0x7f4064195e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:41.644 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:40.335+0000 7f406a48c700 1 --2- 172.21.15.39:0/77778717 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40640ffe00 0x7f4064195e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:41.644 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:40.335+0000 7f406a48c700 1 -- 172.21.15.39:0/77778717 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40641007a0 msgr2=0x7f40641963d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:41.645 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:40.335+0000 7f406a48c700 1 --2- 172.21.15.39:0/77778717 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40641007a0 0x7f40641963d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:41.645 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:40.335+0000 7f406a48c700 1 -- 172.21.15.39:0/77778717 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f405c009d00 con 0x7f4064104ea0 2023-07-21T18:40:41.645 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:40.335+0000 7f406948a700 1 --2- 172.21.15.39:0/77778717 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40641007a0 0x7f40641963d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:40:41.645 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:40.336+0000 7f406a48c700 1 --2- 172.21.15.39:0/77778717 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4064104ea0 0x7f406419a840 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f405c007940 tx=0x7f405c007970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:41.645 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:40.336+0000 7f4056ffd700 1 -- 172.21.15.39:0/77778717 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f405c00e040 con 0x7f4064104ea0 2023-07-21T18:40:41.645 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:40.336+0000 7f4056ffd700 1 -- 172.21.15.39:0/77778717 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f405c003e10 con 0x7f4064104ea0 2023-07-21T18:40:41.645 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:40.336+0000 7f4056ffd700 1 -- 172.21.15.39:0/77778717 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f405c012a60 con 0x7f4064104ea0 2023-07-21T18:40:41.646 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:40.336+0000 7f406beef700 1 -- 172.21.15.39:0/77778717 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f406419b400 con 0x7f4064104ea0 2023-07-21T18:40:41.646 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:40.336+0000 7f406beef700 1 -- 172.21.15.39:0/77778717 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40641a2af0 con 0x7f4064104ea0 2023-07-21T18:40:41.646 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:40.337+0000 7f406beef700 1 -- 172.21.15.39:0/77778717 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f406404e210 con 0x7f4064104ea0 2023-07-21T18:40:41.646 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:40.338+0000 7f4056ffd700 1 -- 172.21.15.39:0/77778717 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f405c005730 con 0x7f4064104ea0 2023-07-21T18:40:41.646 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:40.338+0000 7f4056ffd700 1 --2- 172.21.15.39:0/77778717 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f40500a11e0 0x7f40500a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:41.646 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:40.339+0000 7f4069c8b700 1 --2- 172.21.15.39:0/77778717 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f40500a11e0 0x7f40500a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:41.646 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:40.339+0000 7f4056ffd700 1 -- 172.21.15.39:0/77778717 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f405c0ce190 con 0x7f4064104ea0 2023-07-21T18:40:41.646 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:40.339+0000 7f4069c8b700 1 --2- 172.21.15.39:0/77778717 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f40500a11e0 0x7f40500a3690 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4060009a10 tx=0x7f4060009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:41.647 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:40.343+0000 7f4056ffd700 1 -- 172.21.15.39:0/77778717 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f405c0978c0 con 0x7f4064104ea0 2023-07-21T18:40:41.647 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:40.549+0000 7f406beef700 1 -- 172.21.15.39:0/77778717 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f40641a2e10 con 0x7f4064104ea0 2023-07-21T18:40:41.647 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:40.550+0000 7f4056ffd700 1 -- 172.21.15.39:0/77778717 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f405c09aee0 con 0x7f4064104ea0 2023-07-21T18:40:41.647 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:40.552+0000 7f406beef700 1 -- 172.21.15.39:0/77778717 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f40500a11e0 msgr2=0x7f40500a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:41.647 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:40.552+0000 7f406beef700 1 --2- 172.21.15.39:0/77778717 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f40500a11e0 0x7f40500a3690 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4060009a10 tx=0x7f4060009190 comp rx=0 tx=0).stop 2023-07-21T18:40:41.647 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:40.552+0000 7f406beef700 1 -- 172.21.15.39:0/77778717 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4064104ea0 msgr2=0x7f406419a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:41.647 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:40.552+0000 7f406beef700 1 --2- 172.21.15.39:0/77778717 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4064104ea0 0x7f406419a840 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f405c007940 tx=0x7f405c007970 comp rx=0 tx=0).stop 2023-07-21T18:40:41.648 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:40.552+0000 7f406beef700 1 -- 172.21.15.39:0/77778717 shutdown_connections 2023-07-21T18:40:41.648 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:40.552+0000 7f406beef700 1 --2- 172.21.15.39:0/77778717 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f40500a11e0 0x7f40500a3690 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:41.648 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:40.552+0000 7f406beef700 1 --2- 172.21.15.39:0/77778717 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4064104ea0 0x7f406419a840 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:41.648 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:40.552+0000 7f406beef700 1 --2- 172.21.15.39:0/77778717 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40640ffe00 0x7f4064195e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:41.648 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:40.552+0000 7f406beef700 1 --2- 172.21.15.39:0/77778717 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40641007a0 0x7f40641963d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:41.648 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:40.552+0000 7f406beef700 1 -- 172.21.15.39:0/77778717 >> 172.21.15.39:0/77778717 conn(0x7f40640fb6a0 msgr2=0x7f40640fc110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:41.648 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:40.552+0000 7f406beef700 1 -- 172.21.15.39:0/77778717 shutdown_connections 2023-07-21T18:40:41.649 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:40.552+0000 7f406beef700 1 -- 172.21.15.39:0/77778717 wait complete. 2023-07-21T18:40:41.802 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:40:41.909 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:40:42.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:41 smithi039 ceph-mon[96489]: pgmap v266: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2023-07-21T18:40:42.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:41 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/2167509777' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-07-21T18:40:42.359 INFO:tasks.cephadm.ceph_manager.ceph:need seq 103079215154 got 103079215155 for osd.3 2023-07-21T18:40:42.360 DEBUG:teuthology.parallel:result is None 2023-07-21T18:40:42.372 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:41 smithi193 ceph-mon[98204]: pgmap v266: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2023-07-21T18:40:42.372 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:41 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/2167509777' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-07-21T18:40:42.391 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:41 smithi155 ceph-mon[97934]: pgmap v266: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2023-07-21T18:40:42.392 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:41 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/2167509777' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-07-21T18:40:42.878 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.852+0000 7febfdd31700 1 -- 172.21.15.39:0/24854479 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7febf8100df0 msgr2=0x7febf81011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:42.878 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.852+0000 7febfdd31700 1 --2- 172.21.15.39:0/24854479 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7febf8100df0 0x7febf81011c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7febe8009990 tx=0x7febe8009ca0 comp rx=0 tx=0).stop 2023-07-21T18:40:42.878 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.852+0000 7febfdd31700 1 -- 172.21.15.39:0/24854479 shutdown_connections 2023-07-21T18:40:42.879 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.852+0000 7febfdd31700 1 --2- 172.21.15.39:0/24854479 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7febf810a140 0x7febf810c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:42.879 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.852+0000 7febfdd31700 1 --2- 172.21.15.39:0/24854479 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7febf8100df0 0x7febf81011c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:42.879 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.852+0000 7febfdd31700 1 --2- 172.21.15.39:0/24854479 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7febf8101700 0x7febf8109c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:42.879 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.852+0000 7febfdd31700 1 -- 172.21.15.39:0/24854479 >> 172.21.15.39:0/24854479 conn(0x7febf8075b70 msgr2=0x7febf8075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:42.879 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.852+0000 7febfdd31700 1 -- 172.21.15.39:0/24854479 shutdown_connections 2023-07-21T18:40:42.879 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.852+0000 7febfdd31700 1 -- 172.21.15.39:0/24854479 wait complete. 2023-07-21T18:40:42.879 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.853+0000 7febfdd31700 1 Processor -- start 2023-07-21T18:40:42.881 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.853+0000 7febfdd31700 1 -- start start 2023-07-21T18:40:42.881 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.853+0000 7febfdd31700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7febf8100df0 0x7febf8197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:42.881 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.853+0000 7febfdd31700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7febf8101700 0x7febf8198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:42.881 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.853+0000 7febfdd31700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7febf810a140 0x7febf819c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:42.881 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.853+0000 7febfdd31700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf819cce0 con 0x7febf8101700 2023-07-21T18:40:42.882 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.853+0000 7febfdd31700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf819ce50 con 0x7febf8100df0 2023-07-21T18:40:42.882 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.853+0000 7febfdd31700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf819d010 con 0x7febf810a140 2023-07-21T18:40:42.882 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.853+0000 7febf6ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7febf8101700 0x7febf8198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:42.882 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.854+0000 7febf6ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7febf8101700 0x7febf8198260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55538/0 (socket says 172.21.15.39:55538) 2023-07-21T18:40:42.882 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.854+0000 7febf6ffd700 1 -- 172.21.15.39:0/3988262319 learned_addr learned my addr 172.21.15.39:0/3988262319 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:40:42.882 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.854+0000 7febf77fe700 1 --2- 172.21.15.39:0/3988262319 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7febf8100df0 0x7febf8197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:42.882 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.854+0000 7febf7fff700 1 --2- 172.21.15.39:0/3988262319 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7febf810a140 0x7febf819c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:42.883 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.854+0000 7febf6ffd700 1 -- 172.21.15.39:0/3988262319 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7febf810a140 msgr2=0x7febf819c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:42.883 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.854+0000 7febf6ffd700 1 --2- 172.21.15.39:0/3988262319 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7febf810a140 0x7febf819c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:42.883 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.854+0000 7febf6ffd700 1 -- 172.21.15.39:0/3988262319 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7febf8100df0 msgr2=0x7febf8197d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:42.883 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.854+0000 7febf6ffd700 1 --2- 172.21.15.39:0/3988262319 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7febf8100df0 0x7febf8197d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:42.883 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.854+0000 7febf6ffd700 1 -- 172.21.15.39:0/3988262319 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febe80095f0 con 0x7febf8101700 2023-07-21T18:40:42.883 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.854+0000 7febf7fff700 1 --2- 172.21.15.39:0/3988262319 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7febf810a140 0x7febf819c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:40:42.883 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.854+0000 7febf6ffd700 1 --2- 172.21.15.39:0/3988262319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7febf8101700 0x7febf8198260 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7febe000c9b0 tx=0x7febe000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:42.883 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.854+0000 7febf4ff9700 1 -- 172.21.15.39:0/3988262319 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe0007b00 con 0x7febf8101700 2023-07-21T18:40:42.884 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.854+0000 7febf4ff9700 1 -- 172.21.15.39:0/3988262319 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febe0007c60 con 0x7febf8101700 2023-07-21T18:40:42.884 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.854+0000 7febfdd31700 1 -- 172.21.15.39:0/3988262319 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febf819d2f0 con 0x7febf8101700 2023-07-21T18:40:42.884 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.854+0000 7febf4ff9700 1 -- 172.21.15.39:0/3988262319 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe0018660 con 0x7febf8101700 2023-07-21T18:40:42.884 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.854+0000 7febfdd31700 1 -- 172.21.15.39:0/3988262319 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febf81a4b50 con 0x7febf8101700 2023-07-21T18:40:42.884 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.855+0000 7febfdd31700 1 -- 172.21.15.39:0/3988262319 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febf8107680 con 0x7febf8101700 2023-07-21T18:40:42.885 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.856+0000 7febf4ff9700 1 -- 172.21.15.39:0/3988262319 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7febe0018880 con 0x7febf8101700 2023-07-21T18:40:42.885 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.857+0000 7febf4ff9700 1 --2- 172.21.15.39:0/3988262319 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7febe40a1260 0x7febe40a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:42.885 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.857+0000 7febf77fe700 1 --2- 172.21.15.39:0/3988262319 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7febe40a1260 0x7febe40a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:42.886 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.857+0000 7febf4ff9700 1 -- 172.21.15.39:0/3988262319 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7febe00ccb90 con 0x7febf8101700 2023-07-21T18:40:42.886 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.858+0000 7febf77fe700 1 --2- 172.21.15.39:0/3988262319 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7febe40a1260 0x7febe40a3710 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7febe8006010 tx=0x7febe800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:42.888 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.862+0000 7febf4ff9700 1 -- 172.21.15.39:0/3988262319 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7febe0096440 con 0x7febf8101700 2023-07-21T18:40:43.027 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:40:43.082 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.973+0000 7f981cd13700 1 -- 172.21.15.39:0/3398240232 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f98180ffe00 msgr2=0x7f98181001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:43.083 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.973+0000 7f981cd13700 1 --2- 172.21.15.39:0/3398240232 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f98180ffe00 0x7f98181001d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f98080099c0 tx=0x7f9808009cd0 comp rx=0 tx=0).stop 2023-07-21T18:40:43.083 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.973+0000 7f981cd13700 1 -- 172.21.15.39:0/3398240232 shutdown_connections 2023-07-21T18:40:43.083 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.973+0000 7f981cd13700 1 --2- 172.21.15.39:0/3398240232 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9818104ea0 0x7f981810ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:43.083 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.973+0000 7f981cd13700 1 --2- 172.21.15.39:0/3398240232 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f98180ffe00 0x7f98181001d0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:43.084 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.973+0000 7f981cd13700 1 --2- 172.21.15.39:0/3398240232 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98181007a0 0x7f9818104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:43.084 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.973+0000 7f981cd13700 1 -- 172.21.15.39:0/3398240232 >> 172.21.15.39:0/3398240232 conn(0x7f98180fb660 msgr2=0x7f98180fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:43.084 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.973+0000 7f981cd13700 1 -- 172.21.15.39:0/3398240232 shutdown_connections 2023-07-21T18:40:43.085 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.973+0000 7f981cd13700 1 -- 172.21.15.39:0/3398240232 wait complete. 2023-07-21T18:40:43.085 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.974+0000 7f981cd13700 1 Processor -- start 2023-07-21T18:40:43.085 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.974+0000 7f981cd13700 1 -- start start 2023-07-21T18:40:43.085 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.974+0000 7f981cd13700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f98180ffe00 0x7f9818195cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:43.086 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.974+0000 7f981cd13700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f98181007a0 0x7f9818196210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:43.086 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.974+0000 7f981cd13700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9818104ea0 0x7f98181995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:43.086 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.974+0000 7f981cd13700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9818199bd0 con 0x7f98180ffe00 2023-07-21T18:40:43.086 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.974+0000 7f981cd13700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9818199d40 con 0x7f9818104ea0 2023-07-21T18:40:43.087 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.974+0000 7f981cd13700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9818199f00 con 0x7f98181007a0 2023-07-21T18:40:43.087 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.974+0000 7f981659c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f98180ffe00 0x7f9818195cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:43.087 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.974+0000 7f981659c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f98180ffe00 0x7f9818195cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55550/0 (socket says 172.21.15.39:55550) 2023-07-21T18:40:43.087 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.974+0000 7f981659c700 1 -- 172.21.15.39:0/1852961709 learned_addr learned my addr 172.21.15.39:0/1852961709 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:40:43.088 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.974+0000 7f9815d9b700 1 --2- 172.21.15.39:0/1852961709 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f98181007a0 0x7f9818196210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:43.088 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.974+0000 7f9816d9d700 1 --2- 172.21.15.39:0/1852961709 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9818104ea0 0x7f98181995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:43.088 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.974+0000 7f981659c700 1 -- 172.21.15.39:0/1852961709 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f98181007a0 msgr2=0x7f9818196210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:43.089 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.975+0000 7f981659c700 1 --2- 172.21.15.39:0/1852961709 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f98181007a0 0x7f9818196210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:43.089 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.975+0000 7f981659c700 1 -- 172.21.15.39:0/1852961709 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9818104ea0 msgr2=0x7f98181995d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:43.089 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.975+0000 7f981659c700 1 --2- 172.21.15.39:0/1852961709 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9818104ea0 0x7f98181995d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:43.089 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.975+0000 7f981659c700 1 -- 172.21.15.39:0/1852961709 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98080095f0 con 0x7f98180ffe00 2023-07-21T18:40:43.090 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.975+0000 7f9816d9d700 1 --2- 172.21.15.39:0/1852961709 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9818104ea0 0x7f98181995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:40:43.090 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.975+0000 7f9815d9b700 1 --2- 172.21.15.39:0/1852961709 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f98181007a0 0x7f9818196210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:40:43.090 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.975+0000 7f981659c700 1 --2- 172.21.15.39:0/1852961709 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f98180ffe00 0x7f9818195cd0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f98080098e0 tx=0x7f980800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:43.090 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.975+0000 7f98077fe700 1 -- 172.21.15.39:0/1852961709 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980801d070 con 0x7f98180ffe00 2023-07-21T18:40:43.091 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.975+0000 7f981cd13700 1 -- 172.21.15.39:0/1852961709 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f981819a180 con 0x7f98180ffe00 2023-07-21T18:40:43.091 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.975+0000 7f98077fe700 1 -- 172.21.15.39:0/1852961709 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f980800fb80 con 0x7f98180ffe00 2023-07-21T18:40:43.091 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.975+0000 7f98077fe700 1 -- 172.21.15.39:0/1852961709 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98080173f0 con 0x7f98180ffe00 2023-07-21T18:40:43.091 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.975+0000 7f981cd13700 1 -- 172.21.15.39:0/1852961709 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98181a2b30 con 0x7f98180ffe00 2023-07-21T18:40:43.092 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.976+0000 7f981cd13700 1 -- 172.21.15.39:0/1852961709 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f981804e210 con 0x7f98180ffe00 2023-07-21T18:40:43.092 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.977+0000 7f98077fe700 1 -- 172.21.15.39:0/1852961709 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f980800fcf0 con 0x7f98180ffe00 2023-07-21T18:40:43.092 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.978+0000 7f98077fe700 1 --2- 172.21.15.39:0/1852961709 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f98000a11e0 0x7f98000a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:43.092 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.978+0000 7f98077fe700 1 -- 172.21.15.39:0/1852961709 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f98080cd5c0 con 0x7f98180ffe00 2023-07-21T18:40:43.093 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.978+0000 7f9815d9b700 1 --2- 172.21.15.39:0/1852961709 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f98000a11e0 0x7f98000a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:43.093 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.978+0000 7f9815d9b700 1 --2- 172.21.15.39:0/1852961709 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f98000a11e0 0x7f98000a3690 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f98181972b0 tx=0x7f980c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:43.093 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:42.982+0000 7f98077fe700 1 -- 172.21.15.39:0/1852961709 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f980805b790 con 0x7f98180ffe00 2023-07-21T18:40:43.140 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:43.114+0000 7febfdd31700 1 -- 172.21.15.39:0/3988262319 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7febf804e210 con 0x7febf8101700 2023-07-21T18:40:43.141 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:43.115+0000 7febf4ff9700 1 -- 172.21.15.39:0/3988262319 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7febe0099a60 con 0x7febf8101700 2023-07-21T18:40:43.141 INFO:teuthology.orchestra.run.smithi039.stdout:133143986224 2023-07-21T18:40:43.143 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:43.117+0000 7febfdd31700 1 -- 172.21.15.39:0/3988262319 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7febe40a1260 msgr2=0x7febe40a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:43.143 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:43.117+0000 7febfdd31700 1 --2- 172.21.15.39:0/3988262319 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7febe40a1260 0x7febe40a3710 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7febe8006010 tx=0x7febe800b540 comp rx=0 tx=0).stop 2023-07-21T18:40:43.143 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:43.117+0000 7febfdd31700 1 -- 172.21.15.39:0/3988262319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7febf8101700 msgr2=0x7febf8198260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:43.144 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:43.117+0000 7febfdd31700 1 --2- 172.21.15.39:0/3988262319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7febf8101700 0x7febf8198260 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7febe000c9b0 tx=0x7febe000cd70 comp rx=0 tx=0).stop 2023-07-21T18:40:43.144 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:43.117+0000 7febfdd31700 1 -- 172.21.15.39:0/3988262319 shutdown_connections 2023-07-21T18:40:43.144 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:43.117+0000 7febfdd31700 1 --2- 172.21.15.39:0/3988262319 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7febe40a1260 0x7febe40a3710 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:43.144 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:43.117+0000 7febfdd31700 1 --2- 172.21.15.39:0/3988262319 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7febf8100df0 0x7febf8197d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:43.145 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:43.117+0000 7febfdd31700 1 --2- 172.21.15.39:0/3988262319 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7febf810a140 0x7febf819c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:43.145 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:43.117+0000 7febfdd31700 1 --2- 172.21.15.39:0/3988262319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7febf8101700 0x7febf8198260 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:43.145 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:43.117+0000 7febfdd31700 1 -- 172.21.15.39:0/3988262319 >> 172.21.15.39:0/3988262319 conn(0x7febf8075b70 msgr2=0x7febf80feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:43.145 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:43.117+0000 7febfdd31700 1 -- 172.21.15.39:0/3988262319 shutdown_connections 2023-07-21T18:40:43.146 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:43.117+0000 7febfdd31700 1 -- 172.21.15.39:0/3988262319 wait complete. 2023-07-21T18:40:43.185 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822698 got 154618822699 for osd.5 2023-07-21T18:40:43.186 DEBUG:teuthology.parallel:result is None 2023-07-21T18:40:43.266 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:43.241+0000 7f981cd13700 1 -- 172.21.15.39:0/1852961709 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7f98181a2cc0 con 0x7f98180ffe00 2023-07-21T18:40:43.266 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:43.241+0000 7f98077fe700 1 -- 172.21.15.39:0/1852961709 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f9808096df0 con 0x7f98180ffe00 2023-07-21T18:40:43.267 INFO:teuthology.orchestra.run.smithi039.stdout:287762808854 2023-07-21T18:40:43.269 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:43.243+0000 7f981cd13700 1 -- 172.21.15.39:0/1852961709 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f98000a11e0 msgr2=0x7f98000a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:43.269 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:43.243+0000 7f981cd13700 1 --2- 172.21.15.39:0/1852961709 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f98000a11e0 0x7f98000a3690 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f98181972b0 tx=0x7f980c009260 comp rx=0 tx=0).stop 2023-07-21T18:40:43.269 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:43.243+0000 7f981cd13700 1 -- 172.21.15.39:0/1852961709 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f98180ffe00 msgr2=0x7f9818195cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:43.269 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:43.243+0000 7f981cd13700 1 --2- 172.21.15.39:0/1852961709 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f98180ffe00 0x7f9818195cd0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f98080098e0 tx=0x7f980800f690 comp rx=0 tx=0).stop 2023-07-21T18:40:43.269 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:43.243+0000 7f981cd13700 1 -- 172.21.15.39:0/1852961709 shutdown_connections 2023-07-21T18:40:43.270 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:43.243+0000 7f981cd13700 1 --2- 172.21.15.39:0/1852961709 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f98000a11e0 0x7f98000a3690 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:43.270 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:43.243+0000 7f981cd13700 1 --2- 172.21.15.39:0/1852961709 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9818104ea0 0x7f98181995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:43.270 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:43.243+0000 7f981cd13700 1 --2- 172.21.15.39:0/1852961709 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f98180ffe00 0x7f9818195cd0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:43.270 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:43.243+0000 7f981cd13700 1 --2- 172.21.15.39:0/1852961709 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f98181007a0 0x7f9818196210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:43.270 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:43.243+0000 7f981cd13700 1 -- 172.21.15.39:0/1852961709 >> 172.21.15.39:0/1852961709 conn(0x7f98180fb660 msgr2=0x7f98180fc170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:43.270 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:43.243+0000 7f981cd13700 1 -- 172.21.15.39:0/1852961709 shutdown_connections 2023-07-21T18:40:43.270 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:43.243+0000 7f981cd13700 1 -- 172.21.15.39:0/1852961709 wait complete. 2023-07-21T18:40:43.292 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:43 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/3988262319' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-07-21T18:40:43.292 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:43 smithi039 ceph-mon[96489]: pgmap v267: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2023-07-21T18:40:43.622 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:43 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/3988262319' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-07-21T18:40:43.623 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:43 smithi193 ceph-mon[98204]: pgmap v267: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2023-07-21T18:40:43.641 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:43 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/3988262319' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-07-21T18:40:43.641 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:43 smithi155 ceph-mon[97934]: pgmap v267: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2023-07-21T18:40:43.901 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:43.876+0000 7f9c757e8700 1 -- 172.21.15.39:0/1642837354 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c70104ab0 msgr2=0x7f9c70104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:43.902 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:43.876+0000 7f9c757e8700 1 --2- 172.21.15.39:0/1642837354 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c70104ab0 0x7f9c70104e80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9c60009990 tx=0x7f9c60009ca0 comp rx=0 tx=0).stop 2023-07-21T18:40:43.902 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:43.876+0000 7f9c757e8700 1 -- 172.21.15.39:0/1642837354 shutdown_connections 2023-07-21T18:40:43.902 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:43.876+0000 7f9c757e8700 1 --2- 172.21.15.39:0/1642837354 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c700ff460 0x7f9c7010b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:43.902 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:43.876+0000 7f9c757e8700 1 --2- 172.21.15.39:0/1642837354 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c70104ab0 0x7f9c70104e80 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:43.902 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:43.876+0000 7f9c757e8700 1 --2- 172.21.15.39:0/1642837354 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c700feab0 0x7f9c700fef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:43.902 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:43.876+0000 7f9c757e8700 1 -- 172.21.15.39:0/1642837354 >> 172.21.15.39:0/1642837354 conn(0x7f9c70075b70 msgr2=0x7f9c70075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:43.902 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:43.876+0000 7f9c757e8700 1 -- 172.21.15.39:0/1642837354 shutdown_connections 2023-07-21T18:40:43.903 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:43.876+0000 7f9c757e8700 1 -- 172.21.15.39:0/1642837354 wait complete. 2023-07-21T18:40:43.904 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:43.876+0000 7f9c757e8700 1 Processor -- start 2023-07-21T18:40:43.904 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:43.877+0000 7f9c757e8700 1 -- start start 2023-07-21T18:40:43.905 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:43.877+0000 7f9c757e8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c700feab0 0x7f9c70194920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:43.905 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:43.877+0000 7f9c757e8700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c700ff460 0x7f9c70194e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:43.905 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:43.877+0000 7f9c757e8700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c70104ab0 0x7f9c70199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:43.905 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:43.877+0000 7f9c757e8700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c70195770 con 0x7f9c700feab0 2023-07-21T18:40:43.905 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:43.877+0000 7f9c757e8700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c70199eb0 con 0x7f9c70104ab0 2023-07-21T18:40:43.905 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:43.877+0000 7f9c757e8700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c7019a020 con 0x7f9c700ff460 2023-07-21T18:40:43.905 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:43.877+0000 7f9c6effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c700feab0 0x7f9c70194920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:43.906 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:43.877+0000 7f9c6effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c700feab0 0x7f9c70194920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55564/0 (socket says 172.21.15.39:55564) 2023-07-21T18:40:43.906 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:43.877+0000 7f9c6effd700 1 -- 172.21.15.39:0/4115415963 learned_addr learned my addr 172.21.15.39:0/4115415963 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:40:43.906 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:43.877+0000 7f9c6f7fe700 1 --2- 172.21.15.39:0/4115415963 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c70104ab0 0x7f9c70199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:43.906 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:43.877+0000 7f9c6e7fc700 1 --2- 172.21.15.39:0/4115415963 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c700ff460 0x7f9c70194e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:43.906 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:43.877+0000 7f9c6effd700 1 -- 172.21.15.39:0/4115415963 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c700ff460 msgr2=0x7f9c70194e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:43.906 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:43.877+0000 7f9c6effd700 1 --2- 172.21.15.39:0/4115415963 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c700ff460 0x7f9c70194e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:43.906 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:43.877+0000 7f9c6effd700 1 -- 172.21.15.39:0/4115415963 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c70104ab0 msgr2=0x7f9c70199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:43.906 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:43.877+0000 7f9c6effd700 1 --2- 172.21.15.39:0/4115415963 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c70104ab0 0x7f9c70199970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:43.907 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:43.877+0000 7f9c6effd700 1 -- 172.21.15.39:0/4115415963 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c600095f0 con 0x7f9c700feab0 2023-07-21T18:40:43.907 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:43.878+0000 7f9c6effd700 1 --2- 172.21.15.39:0/4115415963 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c700feab0 0x7f9c70194920 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9c60005b40 tx=0x7f9c60009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:43.907 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:43.878+0000 7f9c67fff700 1 -- 172.21.15.39:0/4115415963 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c6001d070 con 0x7f9c700feab0 2023-07-21T18:40:43.907 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:43.878+0000 7f9c757e8700 1 -- 172.21.15.39:0/4115415963 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c7019a2a0 con 0x7f9c700feab0 2023-07-21T18:40:43.907 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:43.878+0000 7f9c67fff700 1 -- 172.21.15.39:0/4115415963 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c60005030 con 0x7f9c700feab0 2023-07-21T18:40:43.907 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:43.878+0000 7f9c67fff700 1 -- 172.21.15.39:0/4115415963 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c60004b50 con 0x7f9c700feab0 2023-07-21T18:40:43.907 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:43.878+0000 7f9c757e8700 1 -- 172.21.15.39:0/4115415963 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c7019a6e0 con 0x7f9c700feab0 2023-07-21T18:40:43.907 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:43.879+0000 7f9c757e8700 1 -- 172.21.15.39:0/4115415963 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c7004e210 con 0x7f9c700feab0 2023-07-21T18:40:43.908 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:43.880+0000 7f9c67fff700 1 -- 172.21.15.39:0/4115415963 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f9c600051a0 con 0x7f9c700feab0 2023-07-21T18:40:43.908 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:43.881+0000 7f9c67fff700 1 --2- 172.21.15.39:0/4115415963 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f9c5c0a11e0 0x7f9c5c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:43.909 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:43.881+0000 7f9c6e7fc700 1 --2- 172.21.15.39:0/4115415963 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f9c5c0a11e0 0x7f9c5c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:43.909 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:43.881+0000 7f9c67fff700 1 -- 172.21.15.39:0/4115415963 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f9c600cdf90 con 0x7f9c700feab0 2023-07-21T18:40:43.909 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:43.881+0000 7f9c6e7fc700 1 --2- 172.21.15.39:0/4115415963 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f9c5c0a11e0 0x7f9c5c0a3690 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9c70068890 tx=0x7f9c58009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:43.909 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:43.884+0000 7f9c67fff700 1 -- 172.21.15.39:0/4115415963 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f9c600977c0 con 0x7f9c700feab0 2023-07-21T18:40:44.117 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:44.092+0000 7f9c757e8700 1 -- 172.21.15.39:0/4115415963 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f9c7019a960 con 0x7f9c700feab0 2023-07-21T18:40:44.118 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:44.092+0000 7f9c67fff700 1 -- 172.21.15.39:0/4115415963 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f9c6009ade0 con 0x7f9c700feab0 2023-07-21T18:40:44.118 INFO:teuthology.orchestra.run.smithi039.stdout:317827579923 2023-07-21T18:40:44.120 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:44.095+0000 7f9c757e8700 1 -- 172.21.15.39:0/4115415963 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f9c5c0a11e0 msgr2=0x7f9c5c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:44.120 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:44.095+0000 7f9c757e8700 1 --2- 172.21.15.39:0/4115415963 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f9c5c0a11e0 0x7f9c5c0a3690 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9c70068890 tx=0x7f9c58009260 comp rx=0 tx=0).stop 2023-07-21T18:40:44.121 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:44.095+0000 7f9c757e8700 1 -- 172.21.15.39:0/4115415963 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c700feab0 msgr2=0x7f9c70194920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:44.121 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:44.095+0000 7f9c757e8700 1 --2- 172.21.15.39:0/4115415963 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c700feab0 0x7f9c70194920 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9c60005b40 tx=0x7f9c60009580 comp rx=0 tx=0).stop 2023-07-21T18:40:44.121 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:44.095+0000 7f9c757e8700 1 -- 172.21.15.39:0/4115415963 shutdown_connections 2023-07-21T18:40:44.121 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:44.095+0000 7f9c757e8700 1 --2- 172.21.15.39:0/4115415963 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f9c5c0a11e0 0x7f9c5c0a3690 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:44.121 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:44.095+0000 7f9c757e8700 1 --2- 172.21.15.39:0/4115415963 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c70104ab0 0x7f9c70199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:44.121 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:44.095+0000 7f9c757e8700 1 --2- 172.21.15.39:0/4115415963 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c700feab0 0x7f9c70194920 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:44.121 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:44.095+0000 7f9c757e8700 1 --2- 172.21.15.39:0/4115415963 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c700ff460 0x7f9c70194e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:44.122 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:44.095+0000 7f9c757e8700 1 -- 172.21.15.39:0/4115415963 >> 172.21.15.39:0/4115415963 conn(0x7f9c70075b70 msgr2=0x7f9c700fcec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:44.122 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:44.095+0000 7f9c757e8700 1 -- 172.21.15.39:0/4115415963 shutdown_connections 2023-07-21T18:40:44.122 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:44.095+0000 7f9c757e8700 1 -- 172.21.15.39:0/4115415963 wait complete. 2023-07-21T18:40:44.439 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:44 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/1852961709' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-07-21T18:40:44.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:44 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/4115415963' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-07-21T18:40:44.622 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:44 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/1852961709' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-07-21T18:40:44.622 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:44 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/4115415963' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-07-21T18:40:44.641 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:44 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/1852961709' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-07-21T18:40:44.641 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:44 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/4115415963' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-07-21T18:40:44.977 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808852 got 287762808854 for osd.10 2023-07-21T18:40:44.977 DEBUG:teuthology.parallel:result is None 2023-07-21T18:40:45.235 INFO:tasks.cephadm.ceph_manager.ceph:need seq 317827579921 got 317827579923 for osd.11 2023-07-21T18:40:45.235 DEBUG:teuthology.parallel:result is None 2023-07-21T18:40:45.557 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:45 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/4017030958' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-07-21T18:40:45.558 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:45 smithi039 ceph-mon[96489]: pgmap v268: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2023-07-21T18:40:45.622 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:45 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/4017030958' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-07-21T18:40:45.622 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:45 smithi193 ceph-mon[98204]: pgmap v268: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2023-07-21T18:40:45.627 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986223 got 133143986224 for osd.4 2023-07-21T18:40:45.627 DEBUG:teuthology.parallel:result is None 2023-07-21T18:40:45.641 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:45 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/4017030958' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-07-21T18:40:45.641 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:45 smithi155 ceph-mon[97934]: pgmap v268: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2023-07-21T18:40:45.815 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.715+0000 7f4904620700 1 -- 172.21.15.39:0/3485529588 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f48fc0ffe00 msgr2=0x7f48fc1001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:45.816 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.715+0000 7f4904620700 1 --2- 172.21.15.39:0/3485529588 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f48fc0ffe00 0x7f48fc1001d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f48ec00b0d0 tx=0x7f48ec00b3e0 comp rx=0 tx=0).stop 2023-07-21T18:40:45.816 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.715+0000 7f4904620700 1 -- 172.21.15.39:0/3485529588 shutdown_connections 2023-07-21T18:40:45.816 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.715+0000 7f4904620700 1 --2- 172.21.15.39:0/3485529588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48fc104ea0 0x7f48fc10ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:45.816 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.715+0000 7f4904620700 1 --2- 172.21.15.39:0/3485529588 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f48fc0ffe00 0x7f48fc1001d0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:45.816 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.715+0000 7f4904620700 1 --2- 172.21.15.39:0/3485529588 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f48fc1007a0 0x7f48fc104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:45.816 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.715+0000 7f4904620700 1 -- 172.21.15.39:0/3485529588 >> 172.21.15.39:0/3485529588 conn(0x7f48fc0fb660 msgr2=0x7f48fc0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:45.817 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.715+0000 7f4904620700 1 -- 172.21.15.39:0/3485529588 shutdown_connections 2023-07-21T18:40:45.817 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.715+0000 7f4904620700 1 -- 172.21.15.39:0/3485529588 wait complete. 2023-07-21T18:40:45.817 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.716+0000 7f4904620700 1 Processor -- start 2023-07-21T18:40:45.817 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.716+0000 7f4904620700 1 -- start start 2023-07-21T18:40:45.817 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.717+0000 7f4904620700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48fc0ffe00 0x7f48fc195e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:45.817 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.717+0000 7f4904620700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f48fc1007a0 0x7f48fc196380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:45.817 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.717+0000 7f4904620700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f48fc104ea0 0x7f48fc19a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:45.818 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.717+0000 7f4904620700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48fc19ae00 con 0x7f48fc104ea0 2023-07-21T18:40:45.818 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.717+0000 7f4904620700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48fc19af70 con 0x7f48fc0ffe00 2023-07-21T18:40:45.818 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.717+0000 7f4904620700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48fc19b130 con 0x7f48fc1007a0 2023-07-21T18:40:45.818 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.717+0000 7f4902bbd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f48fc104ea0 0x7f48fc19a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:45.818 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.717+0000 7f4902bbd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f48fc104ea0 0x7f48fc19a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55594/0 (socket says 172.21.15.39:55594) 2023-07-21T18:40:45.818 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.717+0000 7f4902bbd700 1 -- 172.21.15.39:0/106091874 learned_addr learned my addr 172.21.15.39:0/106091874 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:40:45.818 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.717+0000 7f49023bc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48fc0ffe00 0x7f48fc195e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:45.818 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.717+0000 7f4901bbb700 1 --2- 172.21.15.39:0/106091874 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f48fc1007a0 0x7f48fc196380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:45.819 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.717+0000 7f4902bbd700 1 -- 172.21.15.39:0/106091874 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f48fc1007a0 msgr2=0x7f48fc196380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:45.819 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.717+0000 7f4902bbd700 1 --2- 172.21.15.39:0/106091874 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f48fc1007a0 0x7f48fc196380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:45.819 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.717+0000 7f4902bbd700 1 -- 172.21.15.39:0/106091874 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48fc0ffe00 msgr2=0x7f48fc195e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:45.819 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.717+0000 7f4902bbd700 1 --2- 172.21.15.39:0/106091874 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48fc0ffe00 0x7f48fc195e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:45.819 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.717+0000 7f4902bbd700 1 -- 172.21.15.39:0/106091874 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48ec009d00 con 0x7f48fc104ea0 2023-07-21T18:40:45.819 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.717+0000 7f4902bbd700 1 --2- 172.21.15.39:0/106091874 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f48fc104ea0 0x7f48fc19a7f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f48f400d980 tx=0x7f48f400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:45.819 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.717+0000 7f48fb7fe700 1 -- 172.21.15.39:0/106091874 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f4004a30 con 0x7f48fc104ea0 2023-07-21T18:40:45.819 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.717+0000 7f4904620700 1 -- 172.21.15.39:0/106091874 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48fc19b410 con 0x7f48fc104ea0 2023-07-21T18:40:45.820 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.717+0000 7f48fb7fe700 1 -- 172.21.15.39:0/106091874 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48f40054e0 con 0x7f48fc104ea0 2023-07-21T18:40:45.820 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.717+0000 7f48fb7fe700 1 -- 172.21.15.39:0/106091874 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f400fbc0 con 0x7f48fc104ea0 2023-07-21T18:40:45.820 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.718+0000 7f4904620700 1 -- 172.21.15.39:0/106091874 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48fc1a2c70 con 0x7f48fc104ea0 2023-07-21T18:40:45.820 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.718+0000 7f4904620700 1 -- 172.21.15.39:0/106091874 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48fc04e210 con 0x7f48fc104ea0 2023-07-21T18:40:45.821 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.719+0000 7f48fb7fe700 1 -- 172.21.15.39:0/106091874 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f48f4004c20 con 0x7f48fc104ea0 2023-07-21T18:40:45.821 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.720+0000 7f48fb7fe700 1 --2- 172.21.15.39:0/106091874 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f48e40a11e0 0x7f48e40a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:45.821 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.720+0000 7f49023bc700 1 --2- 172.21.15.39:0/106091874 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f48e40a11e0 0x7f48e40a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:45.821 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.720+0000 7f48fb7fe700 1 -- 172.21.15.39:0/106091874 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f48f40cc5c0 con 0x7f48fc104ea0 2023-07-21T18:40:45.821 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.720+0000 7f49023bc700 1 --2- 172.21.15.39:0/106091874 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f48e40a11e0 0x7f48e40a3690 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f48ec009fd0 tx=0x7f48ec00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:45.821 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.724+0000 7f48fb7fe700 1 -- 172.21.15.39:0/106091874 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f48f4095d60 con 0x7f48fc104ea0 2023-07-21T18:40:45.865 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.839+0000 7efc09955700 1 -- 172.21.15.39:0/4012484556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc040ff290 msgr2=0x7efc0410bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:45.865 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.839+0000 7efc09955700 1 --2- 172.21.15.39:0/4012484556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc040ff290 0x7efc0410bff0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7efbf8009990 tx=0x7efbf8009ca0 comp rx=0 tx=0).stop 2023-07-21T18:40:45.865 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.839+0000 7efc09955700 1 -- 172.21.15.39:0/4012484556 shutdown_connections 2023-07-21T18:40:45.865 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.839+0000 7efc09955700 1 --2- 172.21.15.39:0/4012484556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc0410c530 0x7efc0410e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:45.865 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.839+0000 7efc09955700 1 --2- 172.21.15.39:0/4012484556 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc040fe980 0x7efc040fed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:45.866 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.839+0000 7efc09955700 1 --2- 172.21.15.39:0/4012484556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc040ff290 0x7efc0410bff0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:45.866 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.839+0000 7efc09955700 1 -- 172.21.15.39:0/4012484556 >> 172.21.15.39:0/4012484556 conn(0x7efc04075b70 msgr2=0x7efc04075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:45.866 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.839+0000 7efc09955700 1 -- 172.21.15.39:0/4012484556 shutdown_connections 2023-07-21T18:40:45.866 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.839+0000 7efc09955700 1 -- 172.21.15.39:0/4012484556 wait complete. 2023-07-21T18:40:45.867 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.840+0000 7efc09955700 1 Processor -- start 2023-07-21T18:40:45.868 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.840+0000 7efc09955700 1 -- start start 2023-07-21T18:40:45.868 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.840+0000 7efc09955700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc040fe980 0x7efc04197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:45.868 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.840+0000 7efc09955700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc040ff290 0x7efc04198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:45.868 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.840+0000 7efc09955700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc0410c530 0x7efc0419c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:45.868 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.840+0000 7efc09955700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc0419cf00 con 0x7efc040ff290 2023-07-21T18:40:45.868 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.840+0000 7efc09955700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc0419d070 con 0x7efc0410c530 2023-07-21T18:40:45.869 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.840+0000 7efc09955700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc0419d230 con 0x7efc040fe980 2023-07-21T18:40:45.869 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.841+0000 7efc027fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc040ff290 0x7efc04198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:45.869 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.841+0000 7efc027fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc040ff290 0x7efc04198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55606/0 (socket says 172.21.15.39:55606) 2023-07-21T18:40:45.869 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.841+0000 7efc027fc700 1 -- 172.21.15.39:0/3296040818 learned_addr learned my addr 172.21.15.39:0/3296040818 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:40:45.870 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.841+0000 7efc02ffd700 1 --2- 172.21.15.39:0/3296040818 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc040fe980 0x7efc04197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:45.870 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.841+0000 7efc037fe700 1 --2- 172.21.15.39:0/3296040818 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc0410c530 0x7efc0419c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:45.870 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.841+0000 7efc027fc700 1 -- 172.21.15.39:0/3296040818 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc040fe980 msgr2=0x7efc04197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:45.870 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.841+0000 7efc027fc700 1 --2- 172.21.15.39:0/3296040818 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc040fe980 0x7efc04197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:45.871 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.841+0000 7efc027fc700 1 -- 172.21.15.39:0/3296040818 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc0410c530 msgr2=0x7efc0419c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:45.871 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.841+0000 7efc027fc700 1 --2- 172.21.15.39:0/3296040818 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc0410c530 0x7efc0419c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:45.871 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.841+0000 7efc027fc700 1 -- 172.21.15.39:0/3296040818 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbf80095f0 con 0x7efc040ff290 2023-07-21T18:40:45.872 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.841+0000 7efc027fc700 1 --2- 172.21.15.39:0/3296040818 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc040ff290 0x7efc04198480 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7efbf80092e0 tx=0x7efbf800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:45.872 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.841+0000 7efc08953700 1 -- 172.21.15.39:0/3296040818 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf801d070 con 0x7efc040ff290 2023-07-21T18:40:45.872 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.841+0000 7efc09955700 1 -- 172.21.15.39:0/3296040818 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc0419d4b0 con 0x7efc040ff290 2023-07-21T18:40:45.873 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.841+0000 7efc08953700 1 -- 172.21.15.39:0/3296040818 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efbf8003d60 con 0x7efc040ff290 2023-07-21T18:40:45.873 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.841+0000 7efc08953700 1 -- 172.21.15.39:0/3296040818 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf80056c0 con 0x7efc040ff290 2023-07-21T18:40:45.873 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.841+0000 7efc09955700 1 -- 172.21.15.39:0/3296040818 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc041a4d70 con 0x7efc040ff290 2023-07-21T18:40:45.873 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.842+0000 7efc09955700 1 -- 172.21.15.39:0/3296040818 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc0404e210 con 0x7efc040ff290 2023-07-21T18:40:45.875 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.844+0000 7efc08953700 1 -- 172.21.15.39:0/3296040818 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7efbf800fcd0 con 0x7efc040ff290 2023-07-21T18:40:45.875 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.845+0000 7efc08953700 1 --2- 172.21.15.39:0/3296040818 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7efbec0a11e0 0x7efbec0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:45.876 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.845+0000 7efc02ffd700 1 --2- 172.21.15.39:0/3296040818 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7efbec0a11e0 0x7efbec0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:45.876 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.845+0000 7efc08953700 1 -- 172.21.15.39:0/3296040818 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7efbf80cdc90 con 0x7efc040ff290 2023-07-21T18:40:45.876 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.845+0000 7efc02ffd700 1 --2- 172.21.15.39:0/3296040818 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7efbec0a11e0 0x7efbec0a3690 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7efbf4005950 tx=0x7efbf400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:45.876 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.848+0000 7efc08953700 1 -- 172.21.15.39:0/3296040818 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7efbf80974c0 con 0x7efc040ff290 2023-07-21T18:40:45.955 INFO:teuthology.orchestra.run.smithi039.stdout:77309411386 2023-07-21T18:40:45.955 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.929+0000 7f4904620700 1 -- 172.21.15.39:0/106091874 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f48fc1971a0 con 0x7f48fc104ea0 2023-07-21T18:40:45.956 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.929+0000 7f48fb7fe700 1 -- 172.21.15.39:0/106091874 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f48f4099380 con 0x7f48fc104ea0 2023-07-21T18:40:45.957 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.931+0000 7f4904620700 1 -- 172.21.15.39:0/106091874 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f48e40a11e0 msgr2=0x7f48e40a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:45.957 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.931+0000 7f4904620700 1 --2- 172.21.15.39:0/106091874 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f48e40a11e0 0x7f48e40a3690 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f48ec009fd0 tx=0x7f48ec00b040 comp rx=0 tx=0).stop 2023-07-21T18:40:45.957 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.931+0000 7f4904620700 1 -- 172.21.15.39:0/106091874 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f48fc104ea0 msgr2=0x7f48fc19a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:45.957 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.931+0000 7f4904620700 1 --2- 172.21.15.39:0/106091874 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f48fc104ea0 0x7f48fc19a7f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f48f400d980 tx=0x7f48f400dc90 comp rx=0 tx=0).stop 2023-07-21T18:40:45.958 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.932+0000 7f4904620700 1 -- 172.21.15.39:0/106091874 shutdown_connections 2023-07-21T18:40:45.958 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.932+0000 7f4904620700 1 --2- 172.21.15.39:0/106091874 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f48e40a11e0 0x7f48e40a3690 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:45.958 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.932+0000 7f4904620700 1 --2- 172.21.15.39:0/106091874 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48fc0ffe00 0x7f48fc195e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:45.958 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.932+0000 7f4904620700 1 --2- 172.21.15.39:0/106091874 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f48fc104ea0 0x7f48fc19a7f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:45.958 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.932+0000 7f4904620700 1 --2- 172.21.15.39:0/106091874 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f48fc1007a0 0x7f48fc196380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:45.958 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.932+0000 7f4904620700 1 -- 172.21.15.39:0/106091874 >> 172.21.15.39:0/106091874 conn(0x7f48fc0fb660 msgr2=0x7f48fc0fc170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:45.958 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.932+0000 7f4904620700 1 -- 172.21.15.39:0/106091874 shutdown_connections 2023-07-21T18:40:45.959 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:45.932+0000 7f4904620700 1 -- 172.21.15.39:0/106091874 wait complete. 2023-07-21T18:40:46.040 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:44.458+0000 7f3f8e72e700 1 -- 172.21.15.39:0/2519550026 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f880ff460 msgr2=0x7f3f8810b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:46.041 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:44.458+0000 7f3f8e72e700 1 --2- 172.21.15.39:0/2519550026 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f880ff460 0x7f3f8810b8f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3f8000b0a0 tx=0x7f3f8000b3b0 comp rx=0 tx=0).stop 2023-07-21T18:40:46.041 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:44.458+0000 7f3f8e72e700 1 -- 172.21.15.39:0/2519550026 shutdown_connections 2023-07-21T18:40:46.041 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:44.458+0000 7f3f8e72e700 1 --2- 172.21.15.39:0/2519550026 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f880ff460 0x7f3f8810b8f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:46.041 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:44.458+0000 7f3f8e72e700 1 --2- 172.21.15.39:0/2519550026 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f88104ab0 0x7f3f88104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:46.041 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:44.458+0000 7f3f8e72e700 1 --2- 172.21.15.39:0/2519550026 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f880feab0 0x7f3f880fef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:46.041 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:44.458+0000 7f3f8e72e700 1 -- 172.21.15.39:0/2519550026 >> 172.21.15.39:0/2519550026 conn(0x7f3f88075b70 msgr2=0x7f3f88075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:46.042 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:44.458+0000 7f3f8e72e700 1 -- 172.21.15.39:0/2519550026 shutdown_connections 2023-07-21T18:40:46.042 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:44.459+0000 7f3f8e72e700 1 -- 172.21.15.39:0/2519550026 wait complete. 2023-07-21T18:40:46.042 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:44.459+0000 7f3f8e72e700 1 Processor -- start 2023-07-21T18:40:46.042 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:44.459+0000 7f3f8e72e700 1 -- start start 2023-07-21T18:40:46.042 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:44.459+0000 7f3f8e72e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f880feab0 0x7f3f88196b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:46.042 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:44.459+0000 7f3f8e72e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f880ff460 0x7f3f88197070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:46.042 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:44.459+0000 7f3f8e72e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f88104ab0 0x7f3f881a1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:46.042 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:44.459+0000 7f3f8e72e700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f88197980 con 0x7f3f88104ab0 2023-07-21T18:40:46.043 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:44.459+0000 7f3f8e72e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f881a20f0 con 0x7f3f880ff460 2023-07-21T18:40:46.043 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:44.459+0000 7f3f8e72e700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f881a2260 con 0x7f3f880feab0 2023-07-21T18:40:46.043 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:44.460+0000 7f3f8cccb700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f88104ab0 0x7f3f881a1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:46.043 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:44.460+0000 7f3f8cccb700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f88104ab0 0x7f3f881a1bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55582/0 (socket says 172.21.15.39:55582) 2023-07-21T18:40:46.043 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:44.460+0000 7f3f8cccb700 1 -- 172.21.15.39:0/4017030958 learned_addr learned my addr 172.21.15.39:0/4017030958 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:40:46.043 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:44.460+0000 7f3f877fe700 1 --2- 172.21.15.39:0/4017030958 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f880ff460 0x7f3f88197070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:46.043 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:44.460+0000 7f3f87fff700 1 --2- 172.21.15.39:0/4017030958 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f880feab0 0x7f3f88196b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:46.044 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:44.460+0000 7f3f8cccb700 1 -- 172.21.15.39:0/4017030958 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f880feab0 msgr2=0x7f3f88196b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:46.044 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:44.460+0000 7f3f8cccb700 1 --2- 172.21.15.39:0/4017030958 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f880feab0 0x7f3f88196b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:46.044 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:44.460+0000 7f3f8cccb700 1 -- 172.21.15.39:0/4017030958 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f880ff460 msgr2=0x7f3f88197070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:46.044 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:44.460+0000 7f3f8cccb700 1 --2- 172.21.15.39:0/4017030958 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f880ff460 0x7f3f88197070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:46.044 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:44.460+0000 7f3f8cccb700 1 -- 172.21.15.39:0/4017030958 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f80009d00 con 0x7f3f88104ab0 2023-07-21T18:40:46.044 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:44.460+0000 7f3f8cccb700 1 --2- 172.21.15.39:0/4017030958 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f88104ab0 0x7f3f881a1bb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3f8000b830 tx=0x7f3f800078c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:46.044 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:44.460+0000 7f3f857fa700 1 -- 172.21.15.39:0/4017030958 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f8000e040 con 0x7f3f88104ab0 2023-07-21T18:40:46.044 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:44.460+0000 7f3f8e72e700 1 -- 172.21.15.39:0/4017030958 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f881a24e0 con 0x7f3f88104ab0 2023-07-21T18:40:46.045 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:44.460+0000 7f3f857fa700 1 -- 172.21.15.39:0/4017030958 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f80004c60 con 0x7f3f88104ab0 2023-07-21T18:40:46.045 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:44.460+0000 7f3f857fa700 1 -- 172.21.15.39:0/4017030958 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f800048e0 con 0x7f3f88104ab0 2023-07-21T18:40:46.045 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:44.460+0000 7f3f8e72e700 1 -- 172.21.15.39:0/4017030958 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f881a2950 con 0x7f3f88104ab0 2023-07-21T18:40:46.045 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:44.462+0000 7f3f8e72e700 1 -- 172.21.15.39:0/4017030958 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f8804e210 con 0x7f3f88104ab0 2023-07-21T18:40:46.045 INFO:teuthology.orchestra.run.smithi039.stdout:55834574910 2023-07-21T18:40:46.046 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:44.462+0000 7f3f857fa700 1 -- 172.21.15.39:0/4017030958 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f3f80003a00 con 0x7f3f88104ab0 2023-07-21T18:40:46.046 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:44.463+0000 7f3f857fa700 1 --2- 172.21.15.39:0/4017030958 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f3f700a11e0 0x7f3f700a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:46.046 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:44.463+0000 7f3f87fff700 1 --2- 172.21.15.39:0/4017030958 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f3f700a11e0 0x7f3f700a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:46.046 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:44.463+0000 7f3f857fa700 1 -- 172.21.15.39:0/4017030958 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f3f800ce180 con 0x7f3f88104ab0 2023-07-21T18:40:46.046 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:44.463+0000 7f3f87fff700 1 --2- 172.21.15.39:0/4017030958 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f3f700a11e0 0x7f3f700a3690 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3f880ffbf0 tx=0x7f3f7800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:46.047 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:44.467+0000 7f3f857fa700 1 -- 172.21.15.39:0/4017030958 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f3f800979b0 con 0x7f3f88104ab0 2023-07-21T18:40:46.047 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:44.670+0000 7f3f8e72e700 1 -- 172.21.15.39:0/4017030958 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f3f881a2c30 con 0x7f3f88104ab0 2023-07-21T18:40:46.047 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:44.670+0000 7f3f857fa700 1 -- 172.21.15.39:0/4017030958 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3f8009afd0 con 0x7f3f88104ab0 2023-07-21T18:40:46.047 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:44.672+0000 7f3f8e72e700 1 -- 172.21.15.39:0/4017030958 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f3f700a11e0 msgr2=0x7f3f700a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:46.047 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:44.672+0000 7f3f8e72e700 1 --2- 172.21.15.39:0/4017030958 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f3f700a11e0 0x7f3f700a3690 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3f880ffbf0 tx=0x7f3f7800b410 comp rx=0 tx=0).stop 2023-07-21T18:40:46.047 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:44.672+0000 7f3f8e72e700 1 -- 172.21.15.39:0/4017030958 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f88104ab0 msgr2=0x7f3f881a1bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:46.047 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:44.672+0000 7f3f8e72e700 1 --2- 172.21.15.39:0/4017030958 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f88104ab0 0x7f3f881a1bb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3f8000b830 tx=0x7f3f800078c0 comp rx=0 tx=0).stop 2023-07-21T18:40:46.047 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:44.672+0000 7f3f8e72e700 1 -- 172.21.15.39:0/4017030958 shutdown_connections 2023-07-21T18:40:46.048 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:44.672+0000 7f3f8e72e700 1 --2- 172.21.15.39:0/4017030958 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f3f700a11e0 0x7f3f700a3690 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:46.048 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:44.672+0000 7f3f8e72e700 1 --2- 172.21.15.39:0/4017030958 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f88104ab0 0x7f3f881a1bb0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:46.048 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:44.672+0000 7f3f8e72e700 1 --2- 172.21.15.39:0/4017030958 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f880feab0 0x7f3f88196b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:46.048 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:44.672+0000 7f3f8e72e700 1 --2- 172.21.15.39:0/4017030958 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f880ff460 0x7f3f88197070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:46.048 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:44.672+0000 7f3f8e72e700 1 -- 172.21.15.39:0/4017030958 >> 172.21.15.39:0/4017030958 conn(0x7f3f88075b70 msgr2=0x7f3f88102600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:46.048 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:44.673+0000 7f3f8e72e700 1 -- 172.21.15.39:0/4017030958 shutdown_connections 2023-07-21T18:40:46.048 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:44.673+0000 7f3f8e72e700 1 -- 172.21.15.39:0/4017030958 wait complete. 2023-07-21T18:40:46.084 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:46.058+0000 7efc09955700 1 -- 172.21.15.39:0/3296040818 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7efc041a4f00 con 0x7efc040ff290 2023-07-21T18:40:46.084 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:46.058+0000 7efc08953700 1 -- 172.21.15.39:0/3296040818 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7efbf80163a0 con 0x7efc040ff290 2023-07-21T18:40:46.084 INFO:teuthology.orchestra.run.smithi039.stdout:236223201312 2023-07-21T18:40:46.086 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:46.061+0000 7efc09955700 1 -- 172.21.15.39:0/3296040818 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7efbec0a11e0 msgr2=0x7efbec0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:46.086 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:46.061+0000 7efc09955700 1 --2- 172.21.15.39:0/3296040818 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7efbec0a11e0 0x7efbec0a3690 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7efbf4005950 tx=0x7efbf400a300 comp rx=0 tx=0).stop 2023-07-21T18:40:46.086 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:46.061+0000 7efc09955700 1 -- 172.21.15.39:0/3296040818 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc040ff290 msgr2=0x7efc04198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:46.087 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:46.061+0000 7efc09955700 1 --2- 172.21.15.39:0/3296040818 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc040ff290 0x7efc04198480 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7efbf80092e0 tx=0x7efbf800f690 comp rx=0 tx=0).stop 2023-07-21T18:40:46.087 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:46.061+0000 7efc09955700 1 -- 172.21.15.39:0/3296040818 shutdown_connections 2023-07-21T18:40:46.087 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:46.061+0000 7efc09955700 1 --2- 172.21.15.39:0/3296040818 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7efbec0a11e0 0x7efbec0a3690 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:46.087 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:46.061+0000 7efc09955700 1 --2- 172.21.15.39:0/3296040818 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc0410c530 0x7efc0419c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:46.088 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:46.061+0000 7efc09955700 1 --2- 172.21.15.39:0/3296040818 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc040fe980 0x7efc04197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:46.088 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:46.061+0000 7efc09955700 1 --2- 172.21.15.39:0/3296040818 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc040ff290 0x7efc04198480 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:46.088 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:46.061+0000 7efc09955700 1 -- 172.21.15.39:0/3296040818 >> 172.21.15.39:0/3296040818 conn(0x7efc04075b70 msgr2=0x7efc040fc3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:46.088 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:46.061+0000 7efc09955700 1 -- 172.21.15.39:0/3296040818 shutdown_connections 2023-07-21T18:40:46.088 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:46.061+0000 7efc09955700 1 -- 172.21.15.39:0/3296040818 wait complete. 2023-07-21T18:40:46.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:46 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/106091874' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-07-21T18:40:46.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:46 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/3296040818' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-07-21T18:40:46.622 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:46 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/106091874' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-07-21T18:40:46.622 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:46 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/3296040818' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-07-21T18:40:46.641 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:46 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/106091874' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-07-21T18:40:46.641 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:46 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/3296040818' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-07-21T18:40:47.246 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411382 got 77309411386 for osd.2 2023-07-21T18:40:47.247 DEBUG:teuthology.parallel:result is None 2023-07-21T18:40:47.374 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:47 smithi039 ceph-mon[96489]: pgmap v269: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-07-21T18:40:47.391 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574907 got 55834574910 for osd.1 2023-07-21T18:40:47.391 DEBUG:teuthology.parallel:result is None 2023-07-21T18:40:47.622 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:47 smithi193 ceph-mon[98204]: pgmap v269: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-07-21T18:40:47.641 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:47 smithi155 ceph-mon[97934]: pgmap v269: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-07-21T18:40:47.696 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.670+0000 7f8df2011700 1 -- 172.21.15.39:0/4164660738 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8dec10a270 msgr2=0x7f8dec10c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:47.696 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.670+0000 7f8df2011700 1 --2- 172.21.15.39:0/4164660738 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8dec10a270 0x7f8dec10c6c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8de400b0a0 tx=0x7f8de400b3b0 comp rx=0 tx=0).stop 2023-07-21T18:40:47.696 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.671+0000 7f8df2011700 1 -- 172.21.15.39:0/4164660738 shutdown_connections 2023-07-21T18:40:47.696 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.671+0000 7f8df2011700 1 --2- 172.21.15.39:0/4164660738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8dec100df0 0x7f8dec1011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:47.697 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.671+0000 7f8df2011700 1 --2- 172.21.15.39:0/4164660738 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8dec10a270 0x7f8dec10c6c0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:47.697 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.671+0000 7f8df2011700 1 --2- 172.21.15.39:0/4164660738 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8dec101790 0x7f8dec109c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:47.697 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.671+0000 7f8df2011700 1 -- 172.21.15.39:0/4164660738 >> 172.21.15.39:0/4164660738 conn(0x7f8dec075b60 msgr2=0x7f8dec075f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:47.697 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.671+0000 7f8df2011700 1 -- 172.21.15.39:0/4164660738 shutdown_connections 2023-07-21T18:40:47.697 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.671+0000 7f8df2011700 1 -- 172.21.15.39:0/4164660738 wait complete. 2023-07-21T18:40:47.699 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.671+0000 7f8df2011700 1 Processor -- start 2023-07-21T18:40:47.699 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.672+0000 7f8df2011700 1 -- start start 2023-07-21T18:40:47.699 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.672+0000 7f8df2011700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8dec100df0 0x7f8dec196040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:47.699 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.672+0000 7f8df2011700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8dec101790 0x7f8dec196580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:47.699 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.672+0000 7f8df2011700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8dec10a270 0x7f8dec19a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:47.700 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.672+0000 7f8df2011700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dec19b000 con 0x7f8dec101790 2023-07-21T18:40:47.700 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.672+0000 7f8df2011700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dec19b170 con 0x7f8dec10a270 2023-07-21T18:40:47.700 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.672+0000 7f8df2011700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dec19b330 con 0x7f8dec100df0 2023-07-21T18:40:47.701 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.672+0000 7f8deaffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8dec101790 0x7f8dec196580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:47.701 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.672+0000 7f8deaffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8dec101790 0x7f8dec196580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55618/0 (socket says 172.21.15.39:55618) 2023-07-21T18:40:47.701 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.672+0000 7f8deaffd700 1 -- 172.21.15.39:0/3655513230 learned_addr learned my addr 172.21.15.39:0/3655513230 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:40:47.701 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.672+0000 7f8debfff700 1 --2- 172.21.15.39:0/3655513230 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8dec10a270 0x7f8dec19a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:47.701 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.672+0000 7f8deb7fe700 1 --2- 172.21.15.39:0/3655513230 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8dec100df0 0x7f8dec196040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:47.701 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.673+0000 7f8deaffd700 1 -- 172.21.15.39:0/3655513230 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8dec100df0 msgr2=0x7f8dec196040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:47.701 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.673+0000 7f8deaffd700 1 --2- 172.21.15.39:0/3655513230 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8dec100df0 0x7f8dec196040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:47.702 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.673+0000 7f8deaffd700 1 -- 172.21.15.39:0/3655513230 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8dec10a270 msgr2=0x7f8dec19a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:47.702 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.673+0000 7f8deaffd700 1 --2- 172.21.15.39:0/3655513230 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8dec10a270 0x7f8dec19a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:47.702 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.673+0000 7f8deaffd700 1 -- 172.21.15.39:0/3655513230 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8de4009d00 con 0x7f8dec101790 2023-07-21T18:40:47.702 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.673+0000 7f8deaffd700 1 --2- 172.21.15.39:0/3655513230 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8dec101790 0x7f8dec196580 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8de000b500 tx=0x7f8de000b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:47.702 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.673+0000 7f8de8ff9700 1 -- 172.21.15.39:0/3655513230 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de0011830 con 0x7f8dec101790 2023-07-21T18:40:47.702 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.673+0000 7f8de8ff9700 1 -- 172.21.15.39:0/3655513230 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8de0011e50 con 0x7f8dec101790 2023-07-21T18:40:47.702 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.673+0000 7f8df2011700 1 -- 172.21.15.39:0/3655513230 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dec19b610 con 0x7f8dec101790 2023-07-21T18:40:47.702 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.673+0000 7f8de8ff9700 1 -- 172.21.15.39:0/3655513230 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de0010de0 con 0x7f8dec101790 2023-07-21T18:40:47.703 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.673+0000 7f8df2011700 1 -- 172.21.15.39:0/3655513230 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dec1a2e70 con 0x7f8dec101790 2023-07-21T18:40:47.704 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.674+0000 7f8df2011700 1 -- 172.21.15.39:0/3655513230 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dec104a40 con 0x7f8dec101790 2023-07-21T18:40:47.705 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.676+0000 7f8de8ff9700 1 -- 172.21.15.39:0/3655513230 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f8de0010480 con 0x7f8dec101790 2023-07-21T18:40:47.705 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.677+0000 7f8de8ff9700 1 --2- 172.21.15.39:0/3655513230 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f8dd40a1260 0x7f8dd40a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:47.705 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.677+0000 7f8deb7fe700 1 --2- 172.21.15.39:0/3655513230 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f8dd40a1260 0x7f8dd40a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:47.705 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.677+0000 7f8de8ff9700 1 -- 172.21.15.39:0/3655513230 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f8de00cc710 con 0x7f8dec101790 2023-07-21T18:40:47.706 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.677+0000 7f8deb7fe700 1 --2- 172.21.15.39:0/3655513230 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f8dd40a1260 0x7f8dd40a3710 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc009a10 tx=0x7f8ddc009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:47.708 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.680+0000 7f8de8ff9700 1 -- 172.21.15.39:0/3655513230 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f8de0095fc0 con 0x7f8dec101790 2023-07-21T18:40:47.911 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.886+0000 7f8df2011700 1 -- 172.21.15.39:0/3655513230 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f8dec04e210 con 0x7f8dec101790 2023-07-21T18:40:47.912 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.887+0000 7f8de8ff9700 1 -- 172.21.15.39:0/3655513230 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8de0015020 con 0x7f8dec101790 2023-07-21T18:40:47.912 INFO:teuthology.orchestra.run.smithi039.stdout:184683593769 2023-07-21T18:40:47.914 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.889+0000 7f8df2011700 1 -- 172.21.15.39:0/3655513230 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f8dd40a1260 msgr2=0x7f8dd40a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:47.914 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.889+0000 7f8df2011700 1 --2- 172.21.15.39:0/3655513230 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f8dd40a1260 0x7f8dd40a3710 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc009a10 tx=0x7f8ddc009190 comp rx=0 tx=0).stop 2023-07-21T18:40:47.915 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.889+0000 7f8df2011700 1 -- 172.21.15.39:0/3655513230 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8dec101790 msgr2=0x7f8dec196580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:47.915 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.889+0000 7f8df2011700 1 --2- 172.21.15.39:0/3655513230 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8dec101790 0x7f8dec196580 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8de000b500 tx=0x7f8de000b810 comp rx=0 tx=0).stop 2023-07-21T18:40:47.915 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.889+0000 7f8df2011700 1 -- 172.21.15.39:0/3655513230 shutdown_connections 2023-07-21T18:40:47.915 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.889+0000 7f8df2011700 1 --2- 172.21.15.39:0/3655513230 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f8dd40a1260 0x7f8dd40a3710 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:47.915 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.889+0000 7f8df2011700 1 --2- 172.21.15.39:0/3655513230 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8dec10a270 0x7f8dec19a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:47.915 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.889+0000 7f8df2011700 1 --2- 172.21.15.39:0/3655513230 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8dec100df0 0x7f8dec196040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:47.915 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.889+0000 7f8df2011700 1 --2- 172.21.15.39:0/3655513230 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8dec101790 0x7f8dec196580 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:47.916 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.889+0000 7f8df2011700 1 -- 172.21.15.39:0/3655513230 >> 172.21.15.39:0/3655513230 conn(0x7f8dec075b60 msgr2=0x7f8dec101b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:47.916 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.889+0000 7f8df2011700 1 -- 172.21.15.39:0/3655513230 shutdown_connections 2023-07-21T18:40:47.916 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.889+0000 7f8df2011700 1 -- 172.21.15.39:0/3655513230 wait complete. 2023-07-21T18:40:47.945 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.920+0000 7fd797810700 1 -- 172.21.15.39:0/847423830 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd79010c450 msgr2=0x7fd79010e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:47.946 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.920+0000 7fd797810700 1 --2- 172.21.15.39:0/847423830 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd79010c450 0x7fd79010e830 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd78800b0a0 tx=0x7fd78800b3b0 comp rx=0 tx=0).stop 2023-07-21T18:40:47.946 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.920+0000 7fd797810700 1 -- 172.21.15.39:0/847423830 shutdown_connections 2023-07-21T18:40:47.946 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.920+0000 7fd797810700 1 --2- 172.21.15.39:0/847423830 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd79010c450 0x7fd79010e830 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:47.946 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.920+0000 7fd797810700 1 --2- 172.21.15.39:0/847423830 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd790072850 0x7fd790072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:47.947 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.920+0000 7fd797810700 1 --2- 172.21.15.39:0/847423830 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd790073160 0x7fd79010bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:47.947 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.920+0000 7fd797810700 1 -- 172.21.15.39:0/847423830 >> 172.21.15.39:0/847423830 conn(0x7fd7900fb690 msgr2=0x7fd7900fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:47.947 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.920+0000 7fd797810700 1 -- 172.21.15.39:0/847423830 shutdown_connections 2023-07-21T18:40:47.947 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.920+0000 7fd797810700 1 -- 172.21.15.39:0/847423830 wait complete. 2023-07-21T18:40:47.949 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.921+0000 7fd797810700 1 Processor -- start 2023-07-21T18:40:47.950 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.921+0000 7fd797810700 1 -- start start 2023-07-21T18:40:47.950 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.921+0000 7fd797810700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd790072850 0x7fd7900ffb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:47.950 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.921+0000 7fd797810700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd790073160 0x7fd7901000b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:47.951 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.921+0000 7fd797810700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd79010c450 0x7fd7901005f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:47.951 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.921+0000 7fd797810700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7901a8640 con 0x7fd790072850 2023-07-21T18:40:47.951 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.921+0000 7fd797810700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd790197d40 con 0x7fd790073160 2023-07-21T18:40:47.951 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.921+0000 7fd797810700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd790197f00 con 0x7fd79010c450 2023-07-21T18:40:47.952 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.921+0000 7fd7955ac700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd790072850 0x7fd7900ffb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:47.952 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.921+0000 7fd7955ac700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd790072850 0x7fd7900ffb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55630/0 (socket says 172.21.15.39:55630) 2023-07-21T18:40:47.952 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.921+0000 7fd7955ac700 1 -- 172.21.15.39:0/1792786297 learned_addr learned my addr 172.21.15.39:0/1792786297 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:40:47.952 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.921+0000 7fd795dad700 1 --2- 172.21.15.39:0/1792786297 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd79010c450 0x7fd7901005f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:47.952 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.921+0000 7fd794dab700 1 --2- 172.21.15.39:0/1792786297 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd790073160 0x7fd7901000b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:47.953 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.921+0000 7fd7955ac700 1 -- 172.21.15.39:0/1792786297 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd79010c450 msgr2=0x7fd7901005f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:47.953 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.921+0000 7fd7955ac700 1 --2- 172.21.15.39:0/1792786297 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd79010c450 0x7fd7901005f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:47.953 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.921+0000 7fd7955ac700 1 -- 172.21.15.39:0/1792786297 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd790073160 msgr2=0x7fd7901000b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:47.953 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.921+0000 7fd7955ac700 1 --2- 172.21.15.39:0/1792786297 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd790073160 0x7fd7901000b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:47.954 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.921+0000 7fd7955ac700 1 -- 172.21.15.39:0/1792786297 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd788009d00 con 0x7fd790072850 2023-07-21T18:40:47.954 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.922+0000 7fd7955ac700 1 --2- 172.21.15.39:0/1792786297 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd790072850 0x7fd7900ffb70 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd78c00e9a0 tx=0x7fd78c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:47.954 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.922+0000 7fd7867fc700 1 -- 172.21.15.39:0/1792786297 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78c009790 con 0x7fd790072850 2023-07-21T18:40:47.954 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.922+0000 7fd7867fc700 1 -- 172.21.15.39:0/1792786297 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd78c010430 con 0x7fd790072850 2023-07-21T18:40:47.955 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.922+0000 7fd797810700 1 -- 172.21.15.39:0/1792786297 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7901981e0 con 0x7fd790072850 2023-07-21T18:40:47.955 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.922+0000 7fd7867fc700 1 -- 172.21.15.39:0/1792786297 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78c018680 con 0x7fd790072850 2023-07-21T18:40:47.955 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.922+0000 7fd797810700 1 -- 172.21.15.39:0/1792786297 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7901986e0 con 0x7fd790072850 2023-07-21T18:40:47.955 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.923+0000 7fd797810700 1 -- 172.21.15.39:0/1792786297 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd790106d70 con 0x7fd790072850 2023-07-21T18:40:47.956 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.925+0000 7fd7867fc700 1 -- 172.21.15.39:0/1792786297 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fd78c0105a0 con 0x7fd790072850 2023-07-21T18:40:47.957 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.926+0000 7fd7867fc700 1 --2- 172.21.15.39:0/1792786297 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fd77c0a11e0 0x7fd77c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:47.957 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.926+0000 7fd794dab700 1 --2- 172.21.15.39:0/1792786297 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fd77c0a11e0 0x7fd77c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:47.957 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.926+0000 7fd7867fc700 1 -- 172.21.15.39:0/1792786297 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fd78c014070 con 0x7fd790072850 2023-07-21T18:40:47.957 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.926+0000 7fd794dab700 1 --2- 172.21.15.39:0/1792786297 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fd77c0a11e0 0x7fd77c0a3690 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd790101360 tx=0x7fd78000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:47.957 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.929+0000 7fd7867fc700 1 -- 172.21.15.39:0/1792786297 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fd78c096440 con 0x7fd790072850 2023-07-21T18:40:48.023 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.981+0000 7f3270ff3700 1 -- 172.21.15.39:0/2980141398 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f326c100e50 msgr2=0x7f326c101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:48.023 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.981+0000 7f3270ff3700 1 --2- 172.21.15.39:0/2980141398 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f326c100e50 0x7f326c101220 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f325c009990 tx=0x7f325c009ca0 comp rx=0 tx=0).stop 2023-07-21T18:40:48.023 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.981+0000 7f3270ff3700 1 -- 172.21.15.39:0/2980141398 shutdown_connections 2023-07-21T18:40:48.024 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.981+0000 7f3270ff3700 1 --2- 172.21.15.39:0/2980141398 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f326c10a1a0 0x7f326c10c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:48.024 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.981+0000 7f3270ff3700 1 --2- 172.21.15.39:0/2980141398 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f326c100e50 0x7f326c101220 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:48.024 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.981+0000 7f3270ff3700 1 --2- 172.21.15.39:0/2980141398 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f326c101760 0x7f326c109c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:48.024 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.981+0000 7f3270ff3700 1 -- 172.21.15.39:0/2980141398 >> 172.21.15.39:0/2980141398 conn(0x7f326c0fa5f0 msgr2=0x7f326c0fca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:48.024 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.981+0000 7f3270ff3700 1 -- 172.21.15.39:0/2980141398 shutdown_connections 2023-07-21T18:40:48.024 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.981+0000 7f3270ff3700 1 -- 172.21.15.39:0/2980141398 wait complete. 2023-07-21T18:40:48.024 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.982+0000 7f3270ff3700 1 Processor -- start 2023-07-21T18:40:48.025 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.982+0000 7f3270ff3700 1 -- start start 2023-07-21T18:40:48.025 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.982+0000 7f3270ff3700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f326c100e50 0x7f326c0feab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:48.025 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.982+0000 7f3270ff3700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f326c101760 0x7f326c0feff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:48.025 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.982+0000 7f3270ff3700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f326c10a1a0 0x7f326c0ff570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:48.025 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.982+0000 7f3270ff3700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326c1a40b0 con 0x7f326c100e50 2023-07-21T18:40:48.025 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.982+0000 7f3270ff3700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326c1a4220 con 0x7f326c10a1a0 2023-07-21T18:40:48.025 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.982+0000 7f3270ff3700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326c1a43e0 con 0x7f326c101760 2023-07-21T18:40:48.025 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.983+0000 7f326a59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f326c100e50 0x7f326c0feab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:48.026 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.983+0000 7f326a59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f326c100e50 0x7f326c0feab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55644/0 (socket says 172.21.15.39:55644) 2023-07-21T18:40:48.026 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.983+0000 7f326a59c700 1 -- 172.21.15.39:0/1956117533 learned_addr learned my addr 172.21.15.39:0/1956117533 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:40:48.027 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.983+0000 7f326ad9d700 1 --2- 172.21.15.39:0/1956117533 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f326c10a1a0 0x7f326c0ff570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:48.027 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.983+0000 7f3269d9b700 1 --2- 172.21.15.39:0/1956117533 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f326c101760 0x7f326c0feff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:48.027 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.983+0000 7f326a59c700 1 -- 172.21.15.39:0/1956117533 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f326c101760 msgr2=0x7f326c0feff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:48.027 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.983+0000 7f326a59c700 1 --2- 172.21.15.39:0/1956117533 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f326c101760 0x7f326c0feff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:48.027 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.983+0000 7f326a59c700 1 -- 172.21.15.39:0/1956117533 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f326c10a1a0 msgr2=0x7f326c0ff570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:48.028 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.983+0000 7f326a59c700 1 --2- 172.21.15.39:0/1956117533 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f326c10a1a0 0x7f326c0ff570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:48.028 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.983+0000 7f326a59c700 1 -- 172.21.15.39:0/1956117533 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f325c0095f0 con 0x7f326c100e50 2023-07-21T18:40:48.028 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.984+0000 7f3269d9b700 1 --2- 172.21.15.39:0/1956117533 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f326c101760 0x7f326c0feff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:40:48.028 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.984+0000 7f326a59c700 1 --2- 172.21.15.39:0/1956117533 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f326c100e50 0x7f326c0feab0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f325c0095c0 tx=0x7f325c00f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:48.028 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.984+0000 7f325b7fe700 1 -- 172.21.15.39:0/1956117533 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325c01d070 con 0x7f326c100e50 2023-07-21T18:40:48.028 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.984+0000 7f3270ff3700 1 -- 172.21.15.39:0/1956117533 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f326c1a4660 con 0x7f326c100e50 2023-07-21T18:40:48.028 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.984+0000 7f325b7fe700 1 -- 172.21.15.39:0/1956117533 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f325c00fc00 con 0x7f326c100e50 2023-07-21T18:40:48.029 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.984+0000 7f325b7fe700 1 -- 172.21.15.39:0/1956117533 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325c017650 con 0x7f326c100e50 2023-07-21T18:40:48.029 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.984+0000 7f3270ff3700 1 -- 172.21.15.39:0/1956117533 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f326c1a4b50 con 0x7f326c100e50 2023-07-21T18:40:48.029 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.985+0000 7f3270ff3700 1 -- 172.21.15.39:0/1956117533 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f326c1052e0 con 0x7f326c100e50 2023-07-21T18:40:48.029 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.987+0000 7f325b7fe700 1 -- 172.21.15.39:0/1956117533 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f325c022470 con 0x7f326c100e50 2023-07-21T18:40:48.029 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.988+0000 7f325b7fe700 1 --2- 172.21.15.39:0/1956117533 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f32540a11e0 0x7f32540a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:48.029 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.988+0000 7f325b7fe700 1 -- 172.21.15.39:0/1956117533 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f325c0cdd50 con 0x7f326c100e50 2023-07-21T18:40:48.029 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.988+0000 7f3269d9b700 1 --2- 172.21.15.39:0/1956117533 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f32540a11e0 0x7f32540a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:48.030 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.988+0000 7f3269d9b700 1 --2- 172.21.15.39:0/1956117533 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f32540a11e0 0x7f32540a3690 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3260005fd0 tx=0x7f3260005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:48.030 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:47.994+0000 7f325b7fe700 1 -- 172.21.15.39:0/1956117533 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f325c097580 con 0x7f326c100e50 2023-07-21T18:40:48.110 INFO:tasks.cephadm.ceph_manager.ceph:need seq 236223201310 got 236223201312 for osd.8 2023-07-21T18:40:48.111 DEBUG:teuthology.parallel:result is None 2023-07-21T18:40:48.171 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.145+0000 7fd797810700 1 -- 172.21.15.39:0/1792786297 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7fd79004e210 con 0x7fd790072850 2023-07-21T18:40:48.171 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.146+0000 7fd7867fc700 1 -- 172.21.15.39:0/1792786297 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd78c026090 con 0x7fd790072850 2023-07-21T18:40:48.171 INFO:teuthology.orchestra.run.smithi039.stdout:266287972380 2023-07-21T18:40:48.173 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.148+0000 7fd797810700 1 -- 172.21.15.39:0/1792786297 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fd77c0a11e0 msgr2=0x7fd77c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:48.173 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.148+0000 7fd797810700 1 --2- 172.21.15.39:0/1792786297 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fd77c0a11e0 0x7fd77c0a3690 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd790101360 tx=0x7fd78000b410 comp rx=0 tx=0).stop 2023-07-21T18:40:48.173 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.148+0000 7fd797810700 1 -- 172.21.15.39:0/1792786297 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd790072850 msgr2=0x7fd7900ffb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:48.173 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.148+0000 7fd797810700 1 --2- 172.21.15.39:0/1792786297 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd790072850 0x7fd7900ffb70 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd78c00e9a0 tx=0x7fd78c00ecb0 comp rx=0 tx=0).stop 2023-07-21T18:40:48.173 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.148+0000 7fd797810700 1 -- 172.21.15.39:0/1792786297 shutdown_connections 2023-07-21T18:40:48.174 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.148+0000 7fd797810700 1 --2- 172.21.15.39:0/1792786297 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fd77c0a11e0 0x7fd77c0a3690 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:48.174 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.148+0000 7fd797810700 1 --2- 172.21.15.39:0/1792786297 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd79010c450 0x7fd7901005f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:48.174 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.148+0000 7fd797810700 1 --2- 172.21.15.39:0/1792786297 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd790072850 0x7fd7900ffb70 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:48.174 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.148+0000 7fd797810700 1 --2- 172.21.15.39:0/1792786297 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd790073160 0x7fd7901000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:48.174 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.148+0000 7fd797810700 1 -- 172.21.15.39:0/1792786297 >> 172.21.15.39:0/1792786297 conn(0x7fd7900fb690 msgr2=0x7fd790103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:48.174 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.148+0000 7fd797810700 1 -- 172.21.15.39:0/1792786297 shutdown_connections 2023-07-21T18:40:48.174 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.148+0000 7fd797810700 1 -- 172.21.15.39:0/1792786297 wait complete. 2023-07-21T18:40:48.238 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.213+0000 7f3270ff3700 1 -- 172.21.15.39:0/1956117533 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f326c104cc0 con 0x7f326c100e50 2023-07-21T18:40:48.239 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.213+0000 7f325b7fe700 1 -- 172.21.15.39:0/1956117533 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f325c09aba0 con 0x7f326c100e50 2023-07-21T18:40:48.239 INFO:teuthology.orchestra.run.smithi039.stdout:206158430245 2023-07-21T18:40:48.241 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.216+0000 7f3270ff3700 1 -- 172.21.15.39:0/1956117533 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f32540a11e0 msgr2=0x7f32540a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:48.241 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.216+0000 7f3270ff3700 1 --2- 172.21.15.39:0/1956117533 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f32540a11e0 0x7f32540a3690 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3260005fd0 tx=0x7f3260005dc0 comp rx=0 tx=0).stop 2023-07-21T18:40:48.241 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.216+0000 7f3270ff3700 1 -- 172.21.15.39:0/1956117533 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f326c100e50 msgr2=0x7f326c0feab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:48.241 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.216+0000 7f3270ff3700 1 --2- 172.21.15.39:0/1956117533 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f326c100e50 0x7f326c0feab0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f325c0095c0 tx=0x7f325c00f710 comp rx=0 tx=0).stop 2023-07-21T18:40:48.242 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.216+0000 7f3270ff3700 1 -- 172.21.15.39:0/1956117533 shutdown_connections 2023-07-21T18:40:48.242 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.216+0000 7f3270ff3700 1 --2- 172.21.15.39:0/1956117533 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f32540a11e0 0x7f32540a3690 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:48.242 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.216+0000 7f3270ff3700 1 --2- 172.21.15.39:0/1956117533 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f326c10a1a0 0x7f326c0ff570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:48.242 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.216+0000 7f3270ff3700 1 --2- 172.21.15.39:0/1956117533 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f326c100e50 0x7f326c0feab0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:48.242 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.216+0000 7f3270ff3700 1 --2- 172.21.15.39:0/1956117533 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f326c101760 0x7f326c0feff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:48.242 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.216+0000 7f3270ff3700 1 -- 172.21.15.39:0/1956117533 >> 172.21.15.39:0/1956117533 conn(0x7f326c0fa5f0 msgr2=0x7f326c0fb250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:48.242 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.216+0000 7f3270ff3700 1 -- 172.21.15.39:0/1956117533 shutdown_connections 2023-07-21T18:40:48.243 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.216+0000 7f3270ff3700 1 -- 172.21.15.39:0/1956117533 wait complete. 2023-07-21T18:40:48.438 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:48 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/3655513230' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-07-21T18:40:48.438 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:48 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/1792786297' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-07-21T18:40:48.565 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.370+0000 7fd970aaf700 1 -- 172.21.15.39:0/1432381678 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd96c0fea90 msgr2=0x7fd96c0fef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:48.565 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.370+0000 7fd970aaf700 1 --2- 172.21.15.39:0/1432381678 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd96c0fea90 0x7fd96c0fef00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd960009990 tx=0x7fd960009ca0 comp rx=0 tx=0).stop 2023-07-21T18:40:48.565 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.370+0000 7fd970aaf700 1 -- 172.21.15.39:0/1432381678 shutdown_connections 2023-07-21T18:40:48.567 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.370+0000 7fd970aaf700 1 --2- 172.21.15.39:0/1432381678 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd96c104a90 0x7fd96c104e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:48.567 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.370+0000 7fd970aaf700 1 --2- 172.21.15.39:0/1432381678 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd96c0ff440 0x7fd96c10b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:48.567 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.370+0000 7fd970aaf700 1 --2- 172.21.15.39:0/1432381678 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd96c0fea90 0x7fd96c0fef00 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:48.567 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.370+0000 7fd970aaf700 1 -- 172.21.15.39:0/1432381678 >> 172.21.15.39:0/1432381678 conn(0x7fd96c0fa570 msgr2=0x7fd96c0fc980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:48.567 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.370+0000 7fd970aaf700 1 -- 172.21.15.39:0/1432381678 shutdown_connections 2023-07-21T18:40:48.568 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.370+0000 7fd970aaf700 1 -- 172.21.15.39:0/1432381678 wait complete. 2023-07-21T18:40:48.568 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.371+0000 7fd970aaf700 1 Processor -- start 2023-07-21T18:40:48.568 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.371+0000 7fd970aaf700 1 -- start start 2023-07-21T18:40:48.568 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.371+0000 7fd970aaf700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd96c0fea90 0x7fd96c19a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:48.568 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.371+0000 7fd970aaf700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd96c0ff440 0x7fd96c19ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:48.569 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.371+0000 7fd970aaf700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd96c104a90 0x7fd96c1948e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:48.569 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.371+0000 7fd970aaf700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd96c194f50 con 0x7fd96c0ff440 2023-07-21T18:40:48.569 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.371+0000 7fd970aaf700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd96c1950c0 con 0x7fd96c104a90 2023-07-21T18:40:48.569 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.371+0000 7fd970aaf700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd96c195280 con 0x7fd96c0fea90 2023-07-21T18:40:48.570 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.371+0000 7fd969d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd96c0ff440 0x7fd96c19ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:48.570 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.371+0000 7fd969d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd96c0ff440 0x7fd96c19ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55660/0 (socket says 172.21.15.39:55660) 2023-07-21T18:40:48.570 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.371+0000 7fd969d9b700 1 -- 172.21.15.39:0/4294524484 learned_addr learned my addr 172.21.15.39:0/4294524484 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:40:48.571 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.371+0000 7fd96ad9d700 1 --2- 172.21.15.39:0/4294524484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd96c104a90 0x7fd96c1948e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:48.571 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.371+0000 7fd96a59c700 1 --2- 172.21.15.39:0/4294524484 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd96c0fea90 0x7fd96c19a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:48.571 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.372+0000 7fd969d9b700 1 -- 172.21.15.39:0/4294524484 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd96c0fea90 msgr2=0x7fd96c19a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:48.572 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.372+0000 7fd969d9b700 1 --2- 172.21.15.39:0/4294524484 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd96c0fea90 0x7fd96c19a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:48.572 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.372+0000 7fd969d9b700 1 -- 172.21.15.39:0/4294524484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd96c104a90 msgr2=0x7fd96c1948e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:48.572 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.372+0000 7fd969d9b700 1 --2- 172.21.15.39:0/4294524484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd96c104a90 0x7fd96c1948e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:48.572 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.372+0000 7fd969d9b700 1 -- 172.21.15.39:0/4294524484 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9600095f0 con 0x7fd96c0ff440 2023-07-21T18:40:48.573 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.372+0000 7fd969d9b700 1 --2- 172.21.15.39:0/4294524484 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd96c0ff440 0x7fd96c19ada0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd9600095c0 tx=0x7fd96000f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:48.573 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.372+0000 7fd95b7fe700 1 -- 172.21.15.39:0/4294524484 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96001d070 con 0x7fd96c0ff440 2023-07-21T18:40:48.573 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.372+0000 7fd970aaf700 1 -- 172.21.15.39:0/4294524484 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd96c195500 con 0x7fd96c0ff440 2023-07-21T18:40:48.573 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.372+0000 7fd95b7fe700 1 -- 172.21.15.39:0/4294524484 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9600054f0 con 0x7fd96c0ff440 2023-07-21T18:40:48.573 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.372+0000 7fd95b7fe700 1 -- 172.21.15.39:0/4294524484 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd960004a70 con 0x7fd96c0ff440 2023-07-21T18:40:48.573 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.372+0000 7fd970aaf700 1 -- 172.21.15.39:0/4294524484 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd96c1a1850 con 0x7fd96c0ff440 2023-07-21T18:40:48.574 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.373+0000 7fd970aaf700 1 -- 172.21.15.39:0/4294524484 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd96c04e210 con 0x7fd96c0ff440 2023-07-21T18:40:48.574 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.374+0000 7fd95b7fe700 1 -- 172.21.15.39:0/4294524484 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fd960004bd0 con 0x7fd96c0ff440 2023-07-21T18:40:48.574 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.375+0000 7fd95b7fe700 1 --2- 172.21.15.39:0/4294524484 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fd9540a11e0 0x7fd9540a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:48.574 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.375+0000 7fd96a59c700 1 --2- 172.21.15.39:0/4294524484 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fd9540a11e0 0x7fd9540a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:48.574 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.375+0000 7fd95b7fe700 1 -- 172.21.15.39:0/4294524484 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fd9600cd660 con 0x7fd96c0ff440 2023-07-21T18:40:48.574 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.375+0000 7fd96a59c700 1 --2- 172.21.15.39:0/4294524484 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fd9540a11e0 0x7fd9540a3690 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd96c0ffbd0 tx=0x7fd95c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:48.574 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.379+0000 7fd95b7fe700 1 -- 172.21.15.39:0/4294524484 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fd960096e90 con 0x7fd96c0ff440 2023-07-21T18:40:48.612 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.586+0000 7fd970aaf700 1 -- 172.21.15.39:0/4294524484 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd96c1a19e0 con 0x7fd96c0ff440 2023-07-21T18:40:48.613 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.587+0000 7fd95b7fe700 1 -- 172.21.15.39:0/4294524484 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd96009a4b0 con 0x7fd96c0ff440 2023-07-21T18:40:48.613 INFO:teuthology.orchestra.run.smithi039.stdout:34359738434 2023-07-21T18:40:48.615 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.589+0000 7fd970aaf700 1 -- 172.21.15.39:0/4294524484 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fd9540a11e0 msgr2=0x7fd9540a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:48.615 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.589+0000 7fd970aaf700 1 --2- 172.21.15.39:0/4294524484 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fd9540a11e0 0x7fd9540a3690 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd96c0ffbd0 tx=0x7fd95c00a380 comp rx=0 tx=0).stop 2023-07-21T18:40:48.615 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.589+0000 7fd970aaf700 1 -- 172.21.15.39:0/4294524484 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd96c0ff440 msgr2=0x7fd96c19ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:48.615 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.589+0000 7fd970aaf700 1 --2- 172.21.15.39:0/4294524484 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd96c0ff440 0x7fd96c19ada0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd9600095c0 tx=0x7fd96000f710 comp rx=0 tx=0).stop 2023-07-21T18:40:48.615 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.589+0000 7fd970aaf700 1 -- 172.21.15.39:0/4294524484 shutdown_connections 2023-07-21T18:40:48.615 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.589+0000 7fd970aaf700 1 --2- 172.21.15.39:0/4294524484 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fd9540a11e0 0x7fd9540a3690 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:48.616 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.589+0000 7fd970aaf700 1 --2- 172.21.15.39:0/4294524484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd96c104a90 0x7fd96c1948e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:48.616 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.589+0000 7fd970aaf700 1 --2- 172.21.15.39:0/4294524484 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd96c0fea90 0x7fd96c19a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:48.616 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.589+0000 7fd970aaf700 1 --2- 172.21.15.39:0/4294524484 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd96c0ff440 0x7fd96c19ada0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:48.616 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.589+0000 7fd970aaf700 1 -- 172.21.15.39:0/4294524484 >> 172.21.15.39:0/4294524484 conn(0x7fd96c0fa570 msgr2=0x7fd96c0fbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:48.616 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.590+0000 7fd970aaf700 1 -- 172.21.15.39:0/4294524484 shutdown_connections 2023-07-21T18:40:48.616 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:48.590+0000 7fd970aaf700 1 -- 172.21.15.39:0/4294524484 wait complete. 2023-07-21T18:40:48.622 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:48 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/3655513230' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-07-21T18:40:48.622 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:48 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/1792786297' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-07-21T18:40:48.630 INFO:tasks.cephadm.ceph_manager.ceph:need seq 184683593766 got 184683593769 for osd.6 2023-07-21T18:40:48.630 DEBUG:teuthology.parallel:result is None 2023-07-21T18:40:48.641 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:48 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/3655513230' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-07-21T18:40:48.641 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:48 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/1792786297' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-07-21T18:40:49.156 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738431 got 34359738434 for osd.0 2023-07-21T18:40:49.156 DEBUG:teuthology.parallel:result is None 2023-07-21T18:40:49.200 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430242 got 206158430245 for osd.7 2023-07-21T18:40:49.201 DEBUG:teuthology.parallel:result is None 2023-07-21T18:40:49.497 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:49 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/1956117533' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-07-21T18:40:49.498 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:49 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/4294524484' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-07-21T18:40:49.498 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:49 smithi039 ceph-mon[96489]: pgmap v270: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s 2023-07-21T18:40:49.499 INFO:tasks.cephadm.ceph_manager.ceph:need seq 266287972378 got 266287972380 for osd.9 2023-07-21T18:40:49.499 DEBUG:teuthology.parallel:result is None 2023-07-21T18:40:49.500 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2023-07-21T18:40:49.500 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph pg dump --format=json 2023-07-21T18:40:49.622 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:49 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/1956117533' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-07-21T18:40:49.622 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:49 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/4294524484' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-07-21T18:40:49.622 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:49 smithi193 ceph-mon[98204]: pgmap v270: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s 2023-07-21T18:40:49.641 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:49 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/1956117533' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-07-21T18:40:49.641 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:49 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/4294524484' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-07-21T18:40:49.641 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:49 smithi155 ceph-mon[97934]: pgmap v270: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s 2023-07-21T18:40:50.063 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:40:51.262 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:51.236+0000 7fb49426e700 1 -- 172.21.15.39:0/2326673109 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb48c0ffe00 msgr2=0x7fb48c1001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:51.262 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:51.236+0000 7fb49426e700 1 --2- 172.21.15.39:0/2326673109 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb48c0ffe00 0x7fb48c1001d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb4880099c0 tx=0x7fb488009cd0 comp rx=0 tx=0).stop 2023-07-21T18:40:51.263 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:51.237+0000 7fb49426e700 1 -- 172.21.15.39:0/2326673109 shutdown_connections 2023-07-21T18:40:51.263 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:51.237+0000 7fb49426e700 1 --2- 172.21.15.39:0/2326673109 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb48c104ea0 0x7fb48c10ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:51.263 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:51.237+0000 7fb49426e700 1 --2- 172.21.15.39:0/2326673109 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb48c0ffe00 0x7fb48c1001d0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:51.263 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:51.237+0000 7fb49426e700 1 --2- 172.21.15.39:0/2326673109 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb48c1007a0 0x7fb48c104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:51.263 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:51.237+0000 7fb49426e700 1 -- 172.21.15.39:0/2326673109 >> 172.21.15.39:0/2326673109 conn(0x7fb48c0fb660 msgr2=0x7fb48c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:51.263 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:51.237+0000 7fb49426e700 1 -- 172.21.15.39:0/2326673109 shutdown_connections 2023-07-21T18:40:51.263 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:51.237+0000 7fb49426e700 1 -- 172.21.15.39:0/2326673109 wait complete. 2023-07-21T18:40:51.265 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:51.237+0000 7fb49426e700 1 Processor -- start 2023-07-21T18:40:51.265 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:51.237+0000 7fb49426e700 1 -- start start 2023-07-21T18:40:51.265 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:51.237+0000 7fb49426e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb48c0ffe00 0x7fb48c19ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:51.265 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:51.238+0000 7fb49426e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb48c1007a0 0x7fb48c19bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:51.265 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:51.238+0000 7fb49426e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb48c104ea0 0x7fb48c195a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:51.266 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:51.238+0000 7fb49426e700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb48c1960f0 con 0x7fb48c0ffe00 2023-07-21T18:40:51.266 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:51.238+0000 7fb49426e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb48c196260 con 0x7fb48c1007a0 2023-07-21T18:40:51.266 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:51.238+0000 7fb49426e700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb48c196420 con 0x7fb48c104ea0 2023-07-21T18:40:51.266 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:51.238+0000 7fb49200a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb48c0ffe00 0x7fb48c19ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:51.266 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:51.238+0000 7fb49200a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb48c0ffe00 0x7fb48c19ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55670/0 (socket says 172.21.15.39:55670) 2023-07-21T18:40:51.267 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:51.238+0000 7fb49200a700 1 -- 172.21.15.39:0/1411442271 learned_addr learned my addr 172.21.15.39:0/1411442271 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:40:51.267 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:51.238+0000 7fb491809700 1 --2- 172.21.15.39:0/1411442271 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb48c1007a0 0x7fb48c19bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:51.267 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:51.238+0000 7fb49280b700 1 --2- 172.21.15.39:0/1411442271 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb48c104ea0 0x7fb48c195a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:51.267 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:51.238+0000 7fb49200a700 1 -- 172.21.15.39:0/1411442271 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb48c104ea0 msgr2=0x7fb48c195a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:51.267 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:51.238+0000 7fb49200a700 1 --2- 172.21.15.39:0/1411442271 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb48c104ea0 0x7fb48c195a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:51.267 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:51.238+0000 7fb49200a700 1 -- 172.21.15.39:0/1411442271 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb48c1007a0 msgr2=0x7fb48c19bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:51.267 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:51.238+0000 7fb49200a700 1 --2- 172.21.15.39:0/1411442271 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb48c1007a0 0x7fb48c19bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:51.268 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:51.238+0000 7fb49200a700 1 -- 172.21.15.39:0/1411442271 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4880095f0 con 0x7fb48c0ffe00 2023-07-21T18:40:51.268 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:51.238+0000 7fb49200a700 1 --2- 172.21.15.39:0/1411442271 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb48c0ffe00 0x7fb48c19ba00 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb488009990 tx=0x7fb488009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:51.268 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:51.238+0000 7fb47f7fe700 1 -- 172.21.15.39:0/1411442271 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48801d070 con 0x7fb48c0ffe00 2023-07-21T18:40:51.268 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:51.238+0000 7fb49426e700 1 -- 172.21.15.39:0/1411442271 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb48c1966a0 con 0x7fb48c0ffe00 2023-07-21T18:40:51.268 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:51.238+0000 7fb47f7fe700 1 -- 172.21.15.39:0/1411442271 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb488004010 con 0x7fb48c0ffe00 2023-07-21T18:40:51.268 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:51.238+0000 7fb47f7fe700 1 -- 172.21.15.39:0/1411442271 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb488022d40 con 0x7fb48c0ffe00 2023-07-21T18:40:51.269 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:51.238+0000 7fb49426e700 1 -- 172.21.15.39:0/1411442271 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb48c1a29f0 con 0x7fb48c0ffe00 2023-07-21T18:40:51.269 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:51.240+0000 7fb49426e700 1 -- 172.21.15.39:0/1411442271 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb48c04e210 con 0x7fb48c0ffe00 2023-07-21T18:40:51.270 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:51.241+0000 7fb47f7fe700 1 -- 172.21.15.39:0/1411442271 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fb488022470 con 0x7fb48c0ffe00 2023-07-21T18:40:51.270 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:51.242+0000 7fb47f7fe700 1 --2- 172.21.15.39:0/1411442271 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fb4780a11e0 0x7fb4780a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:51.270 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:51.242+0000 7fb491809700 1 --2- 172.21.15.39:0/1411442271 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fb4780a11e0 0x7fb4780a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:51.270 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:51.242+0000 7fb47f7fe700 1 -- 172.21.15.39:0/1411442271 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb4880cdc70 con 0x7fb48c0ffe00 2023-07-21T18:40:51.270 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:51.242+0000 7fb491809700 1 --2- 172.21.15.39:0/1411442271 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fb4780a11e0 0x7fb4780a3690 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb480005950 tx=0x7fb4800058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:51.271 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:51.246+0000 7fb47f7fe700 1 -- 172.21.15.39:0/1411442271 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fb4880974a0 con 0x7fb48c0ffe00 2023-07-21T18:40:51.464 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:51.439+0000 7fb49426e700 1 -- 172.21.15.39:0/1411442271 --> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb48c103e60 con 0x7fb4780a11e0 2023-07-21T18:40:51.476 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:51.451+0000 7fb47f7fe700 1 -- 172.21.15.39:0/1411442271 <== mgr.14152 v2:172.21.15.39:6800/2116291712 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+341011 (secure 0 0 0) 0x7fb48c103e60 con 0x7fb4780a11e0 2023-07-21T18:40:51.480 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:40:51.482 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:51.455+0000 7fb49426e700 1 -- 172.21.15.39:0/1411442271 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fb4780a11e0 msgr2=0x7fb4780a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:51.482 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:51.455+0000 7fb49426e700 1 --2- 172.21.15.39:0/1411442271 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fb4780a11e0 0x7fb4780a3690 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb480005950 tx=0x7fb4800058e0 comp rx=0 tx=0).stop 2023-07-21T18:40:51.482 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:51.455+0000 7fb49426e700 1 -- 172.21.15.39:0/1411442271 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb48c0ffe00 msgr2=0x7fb48c19ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:51.482 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:51.455+0000 7fb49426e700 1 --2- 172.21.15.39:0/1411442271 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb48c0ffe00 0x7fb48c19ba00 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb488009990 tx=0x7fb488009580 comp rx=0 tx=0).stop 2023-07-21T18:40:51.482 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:51.455+0000 7fb49426e700 1 -- 172.21.15.39:0/1411442271 shutdown_connections 2023-07-21T18:40:51.482 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:51.455+0000 7fb49426e700 1 --2- 172.21.15.39:0/1411442271 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fb4780a11e0 0x7fb4780a3690 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:51.483 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:51.455+0000 7fb49426e700 1 --2- 172.21.15.39:0/1411442271 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb48c104ea0 0x7fb48c195a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:51.483 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:51.455+0000 7fb49426e700 1 --2- 172.21.15.39:0/1411442271 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb48c0ffe00 0x7fb48c19ba00 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:51.483 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:51.455+0000 7fb49426e700 1 --2- 172.21.15.39:0/1411442271 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb48c1007a0 0x7fb48c19bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:51.483 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:51.455+0000 7fb49426e700 1 -- 172.21.15.39:0/1411442271 >> 172.21.15.39:0/1411442271 conn(0x7fb48c0fb660 msgr2=0x7fb48c103750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:51.483 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:51.455+0000 7fb49426e700 1 -- 172.21.15.39:0/1411442271 shutdown_connections 2023-07-21T18:40:51.483 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:51.455+0000 7fb49426e700 1 -- 172.21.15.39:0/1411442271 wait complete. 2023-07-21T18:40:51.483 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2023-07-21T18:40:51.922 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":271,"stamp":"2023-07-21T18:40:51.130693+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654109,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":553,"num_read_kb":395,"num_write":411,"num_write_kb":3064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":339,"ondisk_log_size":339,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335628,"kb_used_data":12204,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124606196,"statfs":{"total":1151940427776,"available":1151596744704,"internally_reserved":0,"allocated":12496896,"data_stored":6852696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":109,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003206"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.773443+0000","last_change":"2023-07-21T18:40:04.792977+0000","last_active":"2023-07-21T18:40:05.773443+0000","last_peered":"2023-07-21T18:40:05.773443+0000","last_clean":"2023-07-21T18:40:05.773443+0000","last_became_active":"2023-07-21T18:40:04.792667+0000","last_became_peered":"2023-07-21T18:40:04.792667+0000","last_unstale":"2023-07-21T18:40:05.773443+0000","last_undegraded":"2023-07-21T18:40:05.773443+0000","last_fullsized":"2023-07-21T18:40:05.773443+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:37:51.249650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.035763+0000","last_change":"2023-07-21T18:40:04.792936+0000","last_active":"2023-07-21T18:40:07.035763+0000","last_peered":"2023-07-21T18:40:07.035763+0000","last_clean":"2023-07-21T18:40:07.035763+0000","last_became_active":"2023-07-21T18:40:04.792511+0000","last_became_peered":"2023-07-21T18:40:04.792511+0000","last_unstale":"2023-07-21T18:40:07.035763+0000","last_undegraded":"2023-07-21T18:40:07.035763+0000","last_fullsized":"2023-07-21T18:40:07.035763+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:05:00.595279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.035567+0000","last_change":"2023-07-21T18:40:04.796401+0000","last_active":"2023-07-21T18:40:07.035567+0000","last_peered":"2023-07-21T18:40:07.035567+0000","last_clean":"2023-07-21T18:40:07.035567+0000","last_became_active":"2023-07-21T18:40:04.796063+0000","last_became_peered":"2023-07-21T18:40:04.796063+0000","last_unstale":"2023-07-21T18:40:07.035567+0000","last_undegraded":"2023-07-21T18:40:07.035567+0000","last_fullsized":"2023-07-21T18:40:07.035567+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:11:42.127453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.035756+0000","last_change":"2023-07-21T18:40:04.792308+0000","last_active":"2023-07-21T18:40:07.035756+0000","last_peered":"2023-07-21T18:40:07.035756+0000","last_clean":"2023-07-21T18:40:07.035756+0000","last_became_active":"2023-07-21T18:40:04.791831+0000","last_became_peered":"2023-07-21T18:40:04.791831+0000","last_unstale":"2023-07-21T18:40:07.035756+0000","last_undegraded":"2023-07-21T18:40:07.035756+0000","last_fullsized":"2023-07-21T18:40:07.035756+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:31:30.077525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.006829+0000","last_change":"2023-07-21T18:40:04.794035+0000","last_active":"2023-07-21T18:40:06.006829+0000","last_peered":"2023-07-21T18:40:06.006829+0000","last_clean":"2023-07-21T18:40:06.006829+0000","last_became_active":"2023-07-21T18:40:04.793739+0000","last_became_peered":"2023-07-21T18:40:04.793739+0000","last_unstale":"2023-07-21T18:40:06.006829+0000","last_undegraded":"2023-07-21T18:40:06.006829+0000","last_fullsized":"2023-07-21T18:40:06.006829+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:40:42.480359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.035838+0000","last_change":"2023-07-21T18:40:04.792751+0000","last_active":"2023-07-21T18:40:07.035838+0000","last_peered":"2023-07-21T18:40:07.035838+0000","last_clean":"2023-07-21T18:40:07.035838+0000","last_became_active":"2023-07-21T18:40:04.792413+0000","last_became_peered":"2023-07-21T18:40:04.792413+0000","last_unstale":"2023-07-21T18:40:07.035838+0000","last_undegraded":"2023-07-21T18:40:07.035838+0000","last_fullsized":"2023-07-21T18:40:07.035838+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:14:40.371518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.089901+0000","last_change":"2023-07-21T18:40:04.787544+0000","last_active":"2023-07-21T18:40:06.089901+0000","last_peered":"2023-07-21T18:40:06.089901+0000","last_clean":"2023-07-21T18:40:06.089901+0000","last_became_active":"2023-07-21T18:40:04.787208+0000","last_became_peered":"2023-07-21T18:40:04.787208+0000","last_unstale":"2023-07-21T18:40:06.089901+0000","last_undegraded":"2023-07-21T18:40:06.089901+0000","last_fullsized":"2023-07-21T18:40:06.089901+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:40:52.809501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.035809+0000","last_change":"2023-07-21T18:40:04.787964+0000","last_active":"2023-07-21T18:40:07.035809+0000","last_peered":"2023-07-21T18:40:07.035809+0000","last_clean":"2023-07-21T18:40:07.035809+0000","last_became_active":"2023-07-21T18:40:04.787518+0000","last_became_peered":"2023-07-21T18:40:04.787518+0000","last_unstale":"2023-07-21T18:40:07.035809+0000","last_undegraded":"2023-07-21T18:40:07.035809+0000","last_fullsized":"2023-07-21T18:40:07.035809+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:40:44.640274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.777237+0000","last_change":"2023-07-21T18:40:04.782495+0000","last_active":"2023-07-21T18:40:05.777237+0000","last_peered":"2023-07-21T18:40:05.777237+0000","last_clean":"2023-07-21T18:40:05.777237+0000","last_became_active":"2023-07-21T18:40:04.782208+0000","last_became_peered":"2023-07-21T18:40:04.782208+0000","last_unstale":"2023-07-21T18:40:05.777237+0000","last_undegraded":"2023-07-21T18:40:05.777237+0000","last_fullsized":"2023-07-21T18:40:05.777237+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:42:34.649015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.036247+0000","last_change":"2023-07-21T18:40:04.790957+0000","last_active":"2023-07-21T18:40:07.036247+0000","last_peered":"2023-07-21T18:40:07.036247+0000","last_clean":"2023-07-21T18:40:07.036247+0000","last_became_active":"2023-07-21T18:40:04.790659+0000","last_became_peered":"2023-07-21T18:40:04.790659+0000","last_unstale":"2023-07-21T18:40:07.036247+0000","last_undegraded":"2023-07-21T18:40:07.036247+0000","last_fullsized":"2023-07-21T18:40:07.036247+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:06:45.518722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.090285+0000","last_change":"2023-07-21T18:40:04.789415+0000","last_active":"2023-07-21T18:40:06.090285+0000","last_peered":"2023-07-21T18:40:06.090285+0000","last_clean":"2023-07-21T18:40:06.090285+0000","last_became_active":"2023-07-21T18:40:04.789088+0000","last_became_peered":"2023-07-21T18:40:04.789088+0000","last_unstale":"2023-07-21T18:40:06.090285+0000","last_undegraded":"2023-07-21T18:40:06.090285+0000","last_fullsized":"2023-07-21T18:40:06.090285+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:10:43.584203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.035583+0000","last_change":"2023-07-21T18:40:04.789101+0000","last_active":"2023-07-21T18:40:07.035583+0000","last_peered":"2023-07-21T18:40:07.035583+0000","last_clean":"2023-07-21T18:40:07.035583+0000","last_became_active":"2023-07-21T18:40:04.788704+0000","last_became_peered":"2023-07-21T18:40:04.788704+0000","last_unstale":"2023-07-21T18:40:07.035583+0000","last_undegraded":"2023-07-21T18:40:07.035583+0000","last_fullsized":"2023-07-21T18:40:07.035583+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:10:15.256725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.772732+0000","last_change":"2023-07-21T18:40:04.784863+0000","last_active":"2023-07-21T18:40:05.772732+0000","last_peered":"2023-07-21T18:40:05.772732+0000","last_clean":"2023-07-21T18:40:05.772732+0000","last_became_active":"2023-07-21T18:40:04.784474+0000","last_became_peered":"2023-07-21T18:40:04.784474+0000","last_unstale":"2023-07-21T18:40:05.772732+0000","last_undegraded":"2023-07-21T18:40:05.772732+0000","last_fullsized":"2023-07-21T18:40:05.772732+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:53:41.983453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.036146+0000","last_change":"2023-07-21T18:40:04.797678+0000","last_active":"2023-07-21T18:40:07.036146+0000","last_peered":"2023-07-21T18:40:07.036146+0000","last_clean":"2023-07-21T18:40:07.036146+0000","last_became_active":"2023-07-21T18:40:04.797318+0000","last_became_peered":"2023-07-21T18:40:04.797318+0000","last_unstale":"2023-07-21T18:40:07.036146+0000","last_undegraded":"2023-07-21T18:40:07.036146+0000","last_fullsized":"2023-07-21T18:40:07.036146+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:44:17.374089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.035540+0000","last_change":"2023-07-21T18:40:04.793170+0000","last_active":"2023-07-21T18:40:07.035540+0000","last_peered":"2023-07-21T18:40:07.035540+0000","last_clean":"2023-07-21T18:40:07.035540+0000","last_became_active":"2023-07-21T18:40:04.792563+0000","last_became_peered":"2023-07-21T18:40:04.792563+0000","last_unstale":"2023-07-21T18:40:07.035540+0000","last_undegraded":"2023-07-21T18:40:07.035540+0000","last_fullsized":"2023-07-21T18:40:07.035540+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:36:53.352643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.776683+0000","last_change":"2023-07-21T18:40:04.791972+0000","last_active":"2023-07-21T18:40:05.776683+0000","last_peered":"2023-07-21T18:40:05.776683+0000","last_clean":"2023-07-21T18:40:05.776683+0000","last_became_active":"2023-07-21T18:40:04.791639+0000","last_became_peered":"2023-07-21T18:40:04.791639+0000","last_unstale":"2023-07-21T18:40:05.776683+0000","last_undegraded":"2023-07-21T18:40:05.776683+0000","last_fullsized":"2023-07-21T18:40:05.776683+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:35:06.204290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.773432+0000","last_change":"2023-07-21T18:40:04.789362+0000","last_active":"2023-07-21T18:40:05.773432+0000","last_peered":"2023-07-21T18:40:05.773432+0000","last_clean":"2023-07-21T18:40:05.773432+0000","last_became_active":"2023-07-21T18:40:04.788978+0000","last_became_peered":"2023-07-21T18:40:04.788978+0000","last_unstale":"2023-07-21T18:40:05.773432+0000","last_undegraded":"2023-07-21T18:40:05.773432+0000","last_fullsized":"2023-07-21T18:40:05.773432+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:55:04.759568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.036113+0000","last_change":"2023-07-21T18:40:04.788016+0000","last_active":"2023-07-21T18:40:07.036113+0000","last_peered":"2023-07-21T18:40:07.036113+0000","last_clean":"2023-07-21T18:40:07.036113+0000","last_became_active":"2023-07-21T18:40:04.787690+0000","last_became_peered":"2023-07-21T18:40:04.787690+0000","last_unstale":"2023-07-21T18:40:07.036113+0000","last_undegraded":"2023-07-21T18:40:07.036113+0000","last_fullsized":"2023-07-21T18:40:07.036113+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:55:35.585172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.592595+0000","last_change":"2023-07-21T18:40:04.793075+0000","last_active":"2023-07-21T18:40:06.592595+0000","last_peered":"2023-07-21T18:40:06.592595+0000","last_clean":"2023-07-21T18:40:06.592595+0000","last_became_active":"2023-07-21T18:40:04.792773+0000","last_became_peered":"2023-07-21T18:40:04.792773+0000","last_unstale":"2023-07-21T18:40:06.592595+0000","last_undegraded":"2023-07-21T18:40:06.592595+0000","last_fullsized":"2023-07-21T18:40:06.592595+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:48:23.084376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"79'13","reported_seq":64,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:46.048658+0000","last_change":"2023-07-21T18:40:04.178197+0000","last_active":"2023-07-21T18:40:46.048658+0000","last_peered":"2023-07-21T18:40:46.048658+0000","last_clean":"2023-07-21T18:40:46.048658+0000","last_became_active":"2023-07-21T18:40:04.177914+0000","last_became_peered":"2023-07-21T18:40:04.177914+0000","last_unstale":"2023-07-21T18:40:46.048658+0000","last_undegraded":"2023-07-21T18:40:46.048658+0000","last_fullsized":"2023-07-21T18:40:46.048658+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:33:10.544496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.037188+0000","last_change":"2023-07-21T18:40:04.791610+0000","last_active":"2023-07-21T18:40:07.037188+0000","last_peered":"2023-07-21T18:40:07.037188+0000","last_clean":"2023-07-21T18:40:07.037188+0000","last_became_active":"2023-07-21T18:40:04.791242+0000","last_became_peered":"2023-07-21T18:40:04.791242+0000","last_unstale":"2023-07-21T18:40:07.037188+0000","last_undegraded":"2023-07-21T18:40:07.037188+0000","last_fullsized":"2023-07-21T18:40:07.037188+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:02:43.400899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.037064+0000","last_change":"2023-07-21T18:40:04.182120+0000","last_active":"2023-07-21T18:40:07.037064+0000","last_peered":"2023-07-21T18:40:07.037064+0000","last_clean":"2023-07-21T18:40:07.037064+0000","last_became_active":"2023-07-21T18:40:04.181797+0000","last_became_peered":"2023-07-21T18:40:04.181797+0000","last_unstale":"2023-07-21T18:40:07.037064+0000","last_undegraded":"2023-07-21T18:40:07.037064+0000","last_fullsized":"2023-07-21T18:40:07.037064+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:02:06.127933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.036297+0000","last_change":"2023-07-21T18:40:04.796649+0000","last_active":"2023-07-21T18:40:07.036297+0000","last_peered":"2023-07-21T18:40:07.036297+0000","last_clean":"2023-07-21T18:40:07.036297+0000","last_became_active":"2023-07-21T18:40:04.795610+0000","last_became_peered":"2023-07-21T18:40:04.795610+0000","last_unstale":"2023-07-21T18:40:07.036297+0000","last_undegraded":"2023-07-21T18:40:07.036297+0000","last_fullsized":"2023-07-21T18:40:07.036297+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:11:11.969964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"79'12","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:10.975720+0000","last_change":"2023-07-21T18:40:04.178384+0000","last_active":"2023-07-21T18:40:10.975720+0000","last_peered":"2023-07-21T18:40:10.975720+0000","last_clean":"2023-07-21T18:40:10.975720+0000","last_became_active":"2023-07-21T18:40:04.178074+0000","last_became_peered":"2023-07-21T18:40:04.178074+0000","last_unstale":"2023-07-21T18:40:10.975720+0000","last_undegraded":"2023-07-21T18:40:10.975720+0000","last_fullsized":"2023-07-21T18:40:10.975720+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:21:02.336313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.776990+0000","last_change":"2023-07-21T18:40:04.785431+0000","last_active":"2023-07-21T18:40:05.776990+0000","last_peered":"2023-07-21T18:40:05.776990+0000","last_clean":"2023-07-21T18:40:05.776990+0000","last_became_active":"2023-07-21T18:40:04.785124+0000","last_became_peered":"2023-07-21T18:40:04.785124+0000","last_unstale":"2023-07-21T18:40:05.776990+0000","last_undegraded":"2023-07-21T18:40:05.776990+0000","last_fullsized":"2023-07-21T18:40:05.776990+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:19:44.604006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"79'5","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:10.975999+0000","last_change":"2023-07-21T18:40:04.176349+0000","last_active":"2023-07-21T18:40:10.975999+0000","last_peered":"2023-07-21T18:40:10.975999+0000","last_clean":"2023-07-21T18:40:10.975999+0000","last_became_active":"2023-07-21T18:40:04.175997+0000","last_became_peered":"2023-07-21T18:40:04.175997+0000","last_unstale":"2023-07-21T18:40:10.975999+0000","last_undegraded":"2023-07-21T18:40:10.975999+0000","last_fullsized":"2023-07-21T18:40:10.975999+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:13:57.068743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.995402+0000","last_change":"2023-07-21T18:40:04.784867+0000","last_active":"2023-07-21T18:40:06.995402+0000","last_peered":"2023-07-21T18:40:06.995402+0000","last_clean":"2023-07-21T18:40:06.995402+0000","last_became_active":"2023-07-21T18:40:04.784555+0000","last_became_peered":"2023-07-21T18:40:04.784555+0000","last_unstale":"2023-07-21T18:40:06.995402+0000","last_undegraded":"2023-07-21T18:40:06.995402+0000","last_fullsized":"2023-07-21T18:40:06.995402+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:57:13.354337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"79'5","reported_seq":62,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:50.026241+0000","last_change":"2023-07-21T18:40:04.177638+0000","last_active":"2023-07-21T18:40:50.026241+0000","last_peered":"2023-07-21T18:40:50.026241+0000","last_clean":"2023-07-21T18:40:50.026241+0000","last_became_active":"2023-07-21T18:40:04.177284+0000","last_became_peered":"2023-07-21T18:40:04.177284+0000","last_unstale":"2023-07-21T18:40:50.026241+0000","last_undegraded":"2023-07-21T18:40:50.026241+0000","last_fullsized":"2023-07-21T18:40:50.026241+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:21:19.903720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":43,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.006906+0000","last_change":"2023-07-21T18:40:04.791536+0000","last_active":"2023-07-21T18:40:06.006906+0000","last_peered":"2023-07-21T18:40:06.006906+0000","last_clean":"2023-07-21T18:40:06.006906+0000","last_became_active":"2023-07-21T18:40:04.791246+0000","last_became_peered":"2023-07-21T18:40:04.791246+0000","last_unstale":"2023-07-21T18:40:06.006906+0000","last_undegraded":"2023-07-21T18:40:06.006906+0000","last_fullsized":"2023-07-21T18:40:06.006906+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:44:08.906635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"79'7","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:10.271306+0000","last_change":"2023-07-21T18:40:04.179054+0000","last_active":"2023-07-21T18:40:10.271306+0000","last_peered":"2023-07-21T18:40:10.271306+0000","last_clean":"2023-07-21T18:40:10.271306+0000","last_became_active":"2023-07-21T18:40:04.178713+0000","last_became_peered":"2023-07-21T18:40:04.178713+0000","last_unstale":"2023-07-21T18:40:10.271306+0000","last_undegraded":"2023-07-21T18:40:10.271306+0000","last_fullsized":"2023-07-21T18:40:10.271306+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T18:45:12.898734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.995525+0000","last_change":"2023-07-21T18:40:04.793947+0000","last_active":"2023-07-21T18:40:06.995525+0000","last_peered":"2023-07-21T18:40:06.995525+0000","last_clean":"2023-07-21T18:40:06.995525+0000","last_became_active":"2023-07-21T18:40:04.793631+0000","last_became_peered":"2023-07-21T18:40:04.793631+0000","last_unstale":"2023-07-21T18:40:06.995525+0000","last_undegraded":"2023-07-21T18:40:06.995525+0000","last_fullsized":"2023-07-21T18:40:06.995525+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:38:22.934266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"79'3","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:09.232592+0000","last_change":"2023-07-21T18:40:04.182549+0000","last_active":"2023-07-21T18:40:09.232592+0000","last_peered":"2023-07-21T18:40:09.232592+0000","last_clean":"2023-07-21T18:40:09.232592+0000","last_became_active":"2023-07-21T18:40:04.182233+0000","last_became_peered":"2023-07-21T18:40:04.182233+0000","last_unstale":"2023-07-21T18:40:09.232592+0000","last_undegraded":"2023-07-21T18:40:09.232592+0000","last_fullsized":"2023-07-21T18:40:09.232592+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:47:27.131280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.036098+0000","last_change":"2023-07-21T18:40:04.789987+0000","last_active":"2023-07-21T18:40:07.036098+0000","last_peered":"2023-07-21T18:40:07.036098+0000","last_clean":"2023-07-21T18:40:07.036098+0000","last_became_active":"2023-07-21T18:40:04.789647+0000","last_became_peered":"2023-07-21T18:40:04.789647+0000","last_unstale":"2023-07-21T18:40:07.036098+0000","last_undegraded":"2023-07-21T18:40:07.036098+0000","last_fullsized":"2023-07-21T18:40:07.036098+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:38:39.403755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'3","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:10.965460+0000","last_change":"2023-07-21T18:40:04.177342+0000","last_active":"2023-07-21T18:40:10.965460+0000","last_peered":"2023-07-21T18:40:10.965460+0000","last_clean":"2023-07-21T18:40:10.965460+0000","last_became_active":"2023-07-21T18:40:04.176983+0000","last_became_peered":"2023-07-21T18:40:04.176983+0000","last_unstale":"2023-07-21T18:40:10.965460+0000","last_undegraded":"2023-07-21T18:40:10.965460+0000","last_fullsized":"2023-07-21T18:40:10.965460+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:51:36.023901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.035932+0000","last_change":"2023-07-21T18:40:04.794907+0000","last_active":"2023-07-21T18:40:07.035932+0000","last_peered":"2023-07-21T18:40:07.035932+0000","last_clean":"2023-07-21T18:40:07.035932+0000","last_became_active":"2023-07-21T18:40:04.794545+0000","last_became_peered":"2023-07-21T18:40:04.794545+0000","last_unstale":"2023-07-21T18:40:07.035932+0000","last_undegraded":"2023-07-21T18:40:07.035932+0000","last_fullsized":"2023-07-21T18:40:07.035932+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:47:34.101102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"79'4","reported_seq":94,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:48.022390+0000","last_change":"2023-07-21T18:40:04.179198+0000","last_active":"2023-07-21T18:40:48.022390+0000","last_peered":"2023-07-21T18:40:48.022390+0000","last_clean":"2023-07-21T18:40:48.022390+0000","last_became_active":"2023-07-21T18:40:04.178841+0000","last_became_peered":"2023-07-21T18:40:04.178841+0000","last_unstale":"2023-07-21T18:40:48.022390+0000","last_undegraded":"2023-07-21T18:40:48.022390+0000","last_fullsized":"2023-07-21T18:40:48.022390+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:12:24.650360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":76,"num_read_kb":76,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.777142+0000","last_change":"2023-07-21T18:40:04.786229+0000","last_active":"2023-07-21T18:40:05.777142+0000","last_peered":"2023-07-21T18:40:05.777142+0000","last_clean":"2023-07-21T18:40:05.777142+0000","last_became_active":"2023-07-21T18:40:04.785919+0000","last_became_peered":"2023-07-21T18:40:04.785919+0000","last_unstale":"2023-07-21T18:40:05.777142+0000","last_undegraded":"2023-07-21T18:40:05.777142+0000","last_fullsized":"2023-07-21T18:40:05.777142+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:19:04.949730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'4","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:10.974889+0000","last_change":"2023-07-21T18:40:04.177164+0000","last_active":"2023-07-21T18:40:10.974889+0000","last_peered":"2023-07-21T18:40:10.974889+0000","last_clean":"2023-07-21T18:40:10.974889+0000","last_became_active":"2023-07-21T18:40:04.176617+0000","last_became_peered":"2023-07-21T18:40:04.176617+0000","last_unstale":"2023-07-21T18:40:10.974889+0000","last_undegraded":"2023-07-21T18:40:10.974889+0000","last_fullsized":"2023-07-21T18:40:10.974889+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:38:06.655399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.036335+0000","last_change":"2023-07-21T18:40:04.787863+0000","last_active":"2023-07-21T18:40:07.036335+0000","last_peered":"2023-07-21T18:40:07.036335+0000","last_clean":"2023-07-21T18:40:07.036335+0000","last_became_active":"2023-07-21T18:40:04.787562+0000","last_became_peered":"2023-07-21T18:40:04.787562+0000","last_unstale":"2023-07-21T18:40:07.036335+0000","last_undegraded":"2023-07-21T18:40:07.036335+0000","last_fullsized":"2023-07-21T18:40:07.036335+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:30:30.392804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'5","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:10.244099+0000","last_change":"2023-07-21T18:40:04.179921+0000","last_active":"2023-07-21T18:40:10.244099+0000","last_peered":"2023-07-21T18:40:10.244099+0000","last_clean":"2023-07-21T18:40:10.244099+0000","last_became_active":"2023-07-21T18:40:04.179670+0000","last_became_peered":"2023-07-21T18:40:04.179670+0000","last_unstale":"2023-07-21T18:40:10.244099+0000","last_undegraded":"2023-07-21T18:40:10.244099+0000","last_fullsized":"2023-07-21T18:40:10.244099+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:11:15.619102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.777008+0000","last_change":"2023-07-21T18:40:04.792930+0000","last_active":"2023-07-21T18:40:05.777008+0000","last_peered":"2023-07-21T18:40:05.777008+0000","last_clean":"2023-07-21T18:40:05.777008+0000","last_became_active":"2023-07-21T18:40:04.792654+0000","last_became_peered":"2023-07-21T18:40:04.792654+0000","last_unstale":"2023-07-21T18:40:05.777008+0000","last_undegraded":"2023-07-21T18:40:05.777008+0000","last_fullsized":"2023-07-21T18:40:05.777008+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:50:21.387154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"79'28","reported_seq":124,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:48.026676+0000","last_change":"2023-07-21T18:40:04.178820+0000","last_active":"2023-07-21T18:40:48.026676+0000","last_peered":"2023-07-21T18:40:48.026676+0000","last_clean":"2023-07-21T18:40:48.026676+0000","last_became_active":"2023-07-21T18:40:04.178502+0000","last_became_peered":"2023-07-21T18:40:04.178502+0000","last_unstale":"2023-07-21T18:40:48.026676+0000","last_undegraded":"2023-07-21T18:40:48.026676+0000","last_fullsized":"2023-07-21T18:40:48.026676+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:48:50.193456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56875,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":82,"num_read_kb":96,"num_write":30,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.006957+0000","last_change":"2023-07-21T18:40:04.792921+0000","last_active":"2023-07-21T18:40:06.006957+0000","last_peered":"2023-07-21T18:40:06.006957+0000","last_clean":"2023-07-21T18:40:06.006957+0000","last_became_active":"2023-07-21T18:40:04.792608+0000","last_became_peered":"2023-07-21T18:40:04.792608+0000","last_unstale":"2023-07-21T18:40:06.006957+0000","last_undegraded":"2023-07-21T18:40:06.006957+0000","last_fullsized":"2023-07-21T18:40:06.006957+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:04:23.180977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'9","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:10.975237+0000","last_change":"2023-07-21T18:40:04.184917+0000","last_active":"2023-07-21T18:40:10.975237+0000","last_peered":"2023-07-21T18:40:10.975237+0000","last_clean":"2023-07-21T18:40:10.975237+0000","last_became_active":"2023-07-21T18:40:04.184612+0000","last_became_peered":"2023-07-21T18:40:04.184612+0000","last_unstale":"2023-07-21T18:40:10.975237+0000","last_undegraded":"2023-07-21T18:40:10.975237+0000","last_fullsized":"2023-07-21T18:40:10.975237+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:54:45.871657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.772576+0000","last_change":"2023-07-21T18:40:04.794134+0000","last_active":"2023-07-21T18:40:05.772576+0000","last_peered":"2023-07-21T18:40:05.772576+0000","last_clean":"2023-07-21T18:40:05.772576+0000","last_became_active":"2023-07-21T18:40:04.793784+0000","last_became_peered":"2023-07-21T18:40:04.793784+0000","last_unstale":"2023-07-21T18:40:05.772576+0000","last_undegraded":"2023-07-21T18:40:05.772576+0000","last_fullsized":"2023-07-21T18:40:05.772576+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:24:12.380794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'5","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:10.975199+0000","last_change":"2023-07-21T18:40:04.184889+0000","last_active":"2023-07-21T18:40:10.975199+0000","last_peered":"2023-07-21T18:40:10.975199+0000","last_clean":"2023-07-21T18:40:10.975199+0000","last_became_active":"2023-07-21T18:40:04.184516+0000","last_became_peered":"2023-07-21T18:40:04.184516+0000","last_unstale":"2023-07-21T18:40:10.975199+0000","last_undegraded":"2023-07-21T18:40:10.975199+0000","last_fullsized":"2023-07-21T18:40:10.975199+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:56:48.211364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.090064+0000","last_change":"2023-07-21T18:40:04.790627+0000","last_active":"2023-07-21T18:40:06.090064+0000","last_peered":"2023-07-21T18:40:06.090064+0000","last_clean":"2023-07-21T18:40:06.090064+0000","last_became_active":"2023-07-21T18:40:04.790298+0000","last_became_peered":"2023-07-21T18:40:04.790298+0000","last_unstale":"2023-07-21T18:40:06.090064+0000","last_undegraded":"2023-07-21T18:40:06.090064+0000","last_fullsized":"2023-07-21T18:40:06.090064+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:23:15.541330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"79'6","reported_seq":59,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:46.019054+0000","last_change":"2023-07-21T18:40:04.176156+0000","last_active":"2023-07-21T18:40:46.019054+0000","last_peered":"2023-07-21T18:40:46.019054+0000","last_clean":"2023-07-21T18:40:46.019054+0000","last_became_active":"2023-07-21T18:40:04.175823+0000","last_became_peered":"2023-07-21T18:40:04.175823+0000","last_unstale":"2023-07-21T18:40:46.019054+0000","last_undegraded":"2023-07-21T18:40:46.019054+0000","last_fullsized":"2023-07-21T18:40:46.019054+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:43:06.713614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":930,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":40,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.089814+0000","last_change":"2023-07-21T18:40:04.790422+0000","last_active":"2023-07-21T18:40:06.089814+0000","last_peered":"2023-07-21T18:40:06.089814+0000","last_clean":"2023-07-21T18:40:06.089814+0000","last_became_active":"2023-07-21T18:40:04.790124+0000","last_became_peered":"2023-07-21T18:40:04.790124+0000","last_unstale":"2023-07-21T18:40:06.089814+0000","last_undegraded":"2023-07-21T18:40:06.089814+0000","last_fullsized":"2023-07-21T18:40:06.089814+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:50:38.399075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'17","reported_seq":32,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:10.962031+0000","last_change":"2023-07-21T18:40:04.177006+0000","last_active":"2023-07-21T18:40:10.962031+0000","last_peered":"2023-07-21T18:40:10.962031+0000","last_clean":"2023-07-21T18:40:10.962031+0000","last_became_active":"2023-07-21T18:40:04.176711+0000","last_became_peered":"2023-07-21T18:40:04.176711+0000","last_unstale":"2023-07-21T18:40:10.962031+0000","last_undegraded":"2023-07-21T18:40:10.962031+0000","last_fullsized":"2023-07-21T18:40:10.962031+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:51:47.735726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.995106+0000","last_change":"2023-07-21T18:40:04.785601+0000","last_active":"2023-07-21T18:40:06.995106+0000","last_peered":"2023-07-21T18:40:06.995106+0000","last_clean":"2023-07-21T18:40:06.995106+0000","last_became_active":"2023-07-21T18:40:04.785307+0000","last_became_peered":"2023-07-21T18:40:04.785307+0000","last_unstale":"2023-07-21T18:40:06.995106+0000","last_undegraded":"2023-07-21T18:40:06.995106+0000","last_fullsized":"2023-07-21T18:40:06.995106+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:17:07.221060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:10.962183+0000","last_change":"2023-07-21T18:40:04.171005+0000","last_active":"2023-07-21T18:40:10.962183+0000","last_peered":"2023-07-21T18:40:10.962183+0000","last_clean":"2023-07-21T18:40:10.962183+0000","last_became_active":"2023-07-21T18:40:04.170719+0000","last_became_peered":"2023-07-21T18:40:04.170719+0000","last_unstale":"2023-07-21T18:40:10.962183+0000","last_undegraded":"2023-07-21T18:40:10.962183+0000","last_fullsized":"2023-07-21T18:40:10.962183+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:06:46.353789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.773304+0000","last_change":"2023-07-21T18:40:04.783596+0000","last_active":"2023-07-21T18:40:05.773304+0000","last_peered":"2023-07-21T18:40:05.773304+0000","last_clean":"2023-07-21T18:40:05.773304+0000","last_became_active":"2023-07-21T18:40:04.783293+0000","last_became_peered":"2023-07-21T18:40:04.783293+0000","last_unstale":"2023-07-21T18:40:05.773304+0000","last_undegraded":"2023-07-21T18:40:05.773304+0000","last_fullsized":"2023-07-21T18:40:05.773304+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:17:45.360634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"79'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.648456+0000","last_change":"2023-07-21T18:40:04.181069+0000","last_active":"2023-07-21T18:40:07.648456+0000","last_peered":"2023-07-21T18:40:07.648456+0000","last_clean":"2023-07-21T18:40:07.648456+0000","last_became_active":"2023-07-21T18:40:04.180750+0000","last_became_peered":"2023-07-21T18:40:04.180750+0000","last_unstale":"2023-07-21T18:40:07.648456+0000","last_undegraded":"2023-07-21T18:40:07.648456+0000","last_fullsized":"2023-07-21T18:40:07.648456+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:44:48.158024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"79'1","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:08.919470+0000","last_change":"2023-07-21T18:40:04.789940+0000","last_active":"2023-07-21T18:40:08.919470+0000","last_peered":"2023-07-21T18:40:08.919470+0000","last_clean":"2023-07-21T18:40:08.919470+0000","last_became_active":"2023-07-21T18:40:04.789465+0000","last_became_peered":"2023-07-21T18:40:04.789465+0000","last_unstale":"2023-07-21T18:40:08.919470+0000","last_undegraded":"2023-07-21T18:40:08.919470+0000","last_fullsized":"2023-07-21T18:40:08.919470+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:05:44.817727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"79'5","reported_seq":59,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:49.053040+0000","last_change":"2023-07-21T18:40:04.180522+0000","last_active":"2023-07-21T18:40:49.053040+0000","last_peered":"2023-07-21T18:40:49.053040+0000","last_clean":"2023-07-21T18:40:49.053040+0000","last_became_active":"2023-07-21T18:40:04.180196+0000","last_became_peered":"2023-07-21T18:40:04.180196+0000","last_unstale":"2023-07-21T18:40:49.053040+0000","last_undegraded":"2023-07-21T18:40:49.053040+0000","last_fullsized":"2023-07-21T18:40:49.053040+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:10:53.779418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":40,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.036035+0000","last_change":"2023-07-21T18:40:04.794781+0000","last_active":"2023-07-21T18:40:07.036035+0000","last_peered":"2023-07-21T18:40:07.036035+0000","last_clean":"2023-07-21T18:40:07.036035+0000","last_became_active":"2023-07-21T18:40:04.794388+0000","last_became_peered":"2023-07-21T18:40:04.794388+0000","last_unstale":"2023-07-21T18:40:07.036035+0000","last_undegraded":"2023-07-21T18:40:07.036035+0000","last_fullsized":"2023-07-21T18:40:07.036035+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:26:47.679253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.995411+0000","last_change":"2023-07-21T18:40:04.786408+0000","last_active":"2023-07-21T18:40:06.995411+0000","last_peered":"2023-07-21T18:40:06.995411+0000","last_clean":"2023-07-21T18:40:06.995411+0000","last_became_active":"2023-07-21T18:40:04.785555+0000","last_became_peered":"2023-07-21T18:40:04.785555+0000","last_unstale":"2023-07-21T18:40:06.995411+0000","last_undegraded":"2023-07-21T18:40:06.995411+0000","last_fullsized":"2023-07-21T18:40:06.995411+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:32:35.090936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"79'5","reported_seq":58,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:47.020157+0000","last_change":"2023-07-21T18:40:04.185980+0000","last_active":"2023-07-21T18:40:47.020157+0000","last_peered":"2023-07-21T18:40:47.020157+0000","last_clean":"2023-07-21T18:40:47.020157+0000","last_became_active":"2023-07-21T18:40:04.185618+0000","last_became_peered":"2023-07-21T18:40:04.185618+0000","last_unstale":"2023-07-21T18:40:47.020157+0000","last_undegraded":"2023-07-21T18:40:47.020157+0000","last_fullsized":"2023-07-21T18:40:47.020157+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:44:22.202995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":39,"num_read_kb":39,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.036366+0000","last_change":"2023-07-21T18:40:04.792535+0000","last_active":"2023-07-21T18:40:07.036366+0000","last_peered":"2023-07-21T18:40:07.036366+0000","last_clean":"2023-07-21T18:40:07.036366+0000","last_became_active":"2023-07-21T18:40:04.792239+0000","last_became_peered":"2023-07-21T18:40:04.792239+0000","last_unstale":"2023-07-21T18:40:07.036366+0000","last_undegraded":"2023-07-21T18:40:07.036366+0000","last_fullsized":"2023-07-21T18:40:07.036366+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:51:14.670073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"79'6","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:10.974758+0000","last_change":"2023-07-21T18:40:04.173539+0000","last_active":"2023-07-21T18:40:10.974758+0000","last_peered":"2023-07-21T18:40:10.974758+0000","last_clean":"2023-07-21T18:40:10.974758+0000","last_became_active":"2023-07-21T18:40:04.173225+0000","last_became_peered":"2023-07-21T18:40:04.173225+0000","last_unstale":"2023-07-21T18:40:10.974758+0000","last_undegraded":"2023-07-21T18:40:10.974758+0000","last_fullsized":"2023-07-21T18:40:10.974758+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:38:21.873200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":337,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.089887+0000","last_change":"2023-07-21T18:39:05.999983+0000","last_active":"2023-07-21T18:40:06.089887+0000","last_peered":"2023-07-21T18:40:06.089887+0000","last_clean":"2023-07-21T18:40:06.089887+0000","last_became_active":"2023-07-21T18:39:05.999495+0000","last_became_peered":"2023-07-21T18:39:05.999495+0000","last_unstale":"2023-07-21T18:40:06.089887+0000","last_undegraded":"2023-07-21T18:40:06.089887+0000","last_fullsized":"2023-07-21T18:40:06.089887+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:36:14.004429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:36:14.004429+0000","last_clean_scrub_stamp":"2023-07-21T18:36:14.004429+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:34:11.106296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.772327+0000","last_change":"2023-07-21T18:40:04.785086+0000","last_active":"2023-07-21T18:40:05.772327+0000","last_peered":"2023-07-21T18:40:05.772327+0000","last_clean":"2023-07-21T18:40:05.772327+0000","last_became_active":"2023-07-21T18:40:04.784693+0000","last_became_peered":"2023-07-21T18:40:04.784693+0000","last_unstale":"2023-07-21T18:40:05.772327+0000","last_undegraded":"2023-07-21T18:40:05.772327+0000","last_fullsized":"2023-07-21T18:40:05.772327+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:53:45.159424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"79'4","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:09.230935+0000","last_change":"2023-07-21T18:40:04.177032+0000","last_active":"2023-07-21T18:40:09.230935+0000","last_peered":"2023-07-21T18:40:09.230935+0000","last_clean":"2023-07-21T18:40:09.230935+0000","last_became_active":"2023-07-21T18:40:04.176741+0000","last_became_peered":"2023-07-21T18:40:04.176741+0000","last_unstale":"2023-07-21T18:40:09.230935+0000","last_undegraded":"2023-07-21T18:40:09.230935+0000","last_fullsized":"2023-07-21T18:40:09.230935+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:17:23.675128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"79'4","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:10.962237+0000","last_change":"2023-07-21T18:40:04.180106+0000","last_active":"2023-07-21T18:40:10.962237+0000","last_peered":"2023-07-21T18:40:10.962237+0000","last_clean":"2023-07-21T18:40:10.962237+0000","last_became_active":"2023-07-21T18:40:04.179802+0000","last_became_peered":"2023-07-21T18:40:04.179802+0000","last_unstale":"2023-07-21T18:40:10.962237+0000","last_undegraded":"2023-07-21T18:40:10.962237+0000","last_fullsized":"2023-07-21T18:40:10.962237+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:38:16.465170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.772506+0000","last_change":"2023-07-21T18:40:04.789127+0000","last_active":"2023-07-21T18:40:05.772506+0000","last_peered":"2023-07-21T18:40:05.772506+0000","last_clean":"2023-07-21T18:40:05.772506+0000","last_became_active":"2023-07-21T18:40:04.788815+0000","last_became_peered":"2023-07-21T18:40:04.788815+0000","last_unstale":"2023-07-21T18:40:05.772506+0000","last_undegraded":"2023-07-21T18:40:05.772506+0000","last_fullsized":"2023-07-21T18:40:05.772506+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:14:00.509523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"79'9","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:13.313053+0000","last_change":"2023-07-21T18:40:04.184786+0000","last_active":"2023-07-21T18:40:13.313053+0000","last_peered":"2023-07-21T18:40:13.313053+0000","last_clean":"2023-07-21T18:40:13.313053+0000","last_became_active":"2023-07-21T18:40:04.184433+0000","last_became_peered":"2023-07-21T18:40:04.184433+0000","last_unstale":"2023-07-21T18:40:13.313053+0000","last_undegraded":"2023-07-21T18:40:13.313053+0000","last_fullsized":"2023-07-21T18:40:13.313053+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:39:47.800619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.037220+0000","last_change":"2023-07-21T18:40:04.791937+0000","last_active":"2023-07-21T18:40:07.037220+0000","last_peered":"2023-07-21T18:40:07.037220+0000","last_clean":"2023-07-21T18:40:07.037220+0000","last_became_active":"2023-07-21T18:40:04.791588+0000","last_became_peered":"2023-07-21T18:40:04.791588+0000","last_unstale":"2023-07-21T18:40:07.037220+0000","last_undegraded":"2023-07-21T18:40:07.037220+0000","last_fullsized":"2023-07-21T18:40:07.037220+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:33:01.443024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'7","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:10.973433+0000","last_change":"2023-07-21T18:40:04.177007+0000","last_active":"2023-07-21T18:40:10.973433+0000","last_peered":"2023-07-21T18:40:10.973433+0000","last_clean":"2023-07-21T18:40:10.973433+0000","last_became_active":"2023-07-21T18:40:04.176699+0000","last_became_peered":"2023-07-21T18:40:04.176699+0000","last_unstale":"2023-07-21T18:40:10.973433+0000","last_undegraded":"2023-07-21T18:40:10.973433+0000","last_fullsized":"2023-07-21T18:40:10.973433+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:21:08.441400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.035909+0000","last_change":"2023-07-21T18:40:04.795671+0000","last_active":"2023-07-21T18:40:07.035909+0000","last_peered":"2023-07-21T18:40:07.035909+0000","last_clean":"2023-07-21T18:40:07.035909+0000","last_became_active":"2023-07-21T18:40:04.795233+0000","last_became_peered":"2023-07-21T18:40:04.795233+0000","last_unstale":"2023-07-21T18:40:07.035909+0000","last_undegraded":"2023-07-21T18:40:07.035909+0000","last_fullsized":"2023-07-21T18:40:07.035909+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:26:32.284218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:08.212881+0000","last_change":"2023-07-21T18:40:04.176859+0000","last_active":"2023-07-21T18:40:08.212881+0000","last_peered":"2023-07-21T18:40:08.212881+0000","last_clean":"2023-07-21T18:40:08.212881+0000","last_became_active":"2023-07-21T18:40:04.176559+0000","last_became_peered":"2023-07-21T18:40:04.176559+0000","last_unstale":"2023-07-21T18:40:08.212881+0000","last_undegraded":"2023-07-21T18:40:08.212881+0000","last_fullsized":"2023-07-21T18:40:08.212881+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:34:23.667073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.006770+0000","last_change":"2023-07-21T18:40:04.793091+0000","last_active":"2023-07-21T18:40:06.006770+0000","last_peered":"2023-07-21T18:40:06.006770+0000","last_clean":"2023-07-21T18:40:06.006770+0000","last_became_active":"2023-07-21T18:40:04.792723+0000","last_became_peered":"2023-07-21T18:40:04.792723+0000","last_unstale":"2023-07-21T18:40:06.006770+0000","last_undegraded":"2023-07-21T18:40:06.006770+0000","last_fullsized":"2023-07-21T18:40:06.006770+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:27:04.440384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'7","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:10.974001+0000","last_change":"2023-07-21T18:40:04.182867+0000","last_active":"2023-07-21T18:40:10.974001+0000","last_peered":"2023-07-21T18:40:10.974001+0000","last_clean":"2023-07-21T18:40:10.974001+0000","last_became_active":"2023-07-21T18:40:04.181885+0000","last_became_peered":"2023-07-21T18:40:04.181885+0000","last_unstale":"2023-07-21T18:40:10.974001+0000","last_undegraded":"2023-07-21T18:40:10.974001+0000","last_fullsized":"2023-07-21T18:40:10.974001+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:03:28.800986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.773091+0000","last_change":"2023-07-21T18:40:04.790340+0000","last_active":"2023-07-21T18:40:05.773091+0000","last_peered":"2023-07-21T18:40:05.773091+0000","last_clean":"2023-07-21T18:40:05.773091+0000","last_became_active":"2023-07-21T18:40:04.790014+0000","last_became_peered":"2023-07-21T18:40:04.790014+0000","last_unstale":"2023-07-21T18:40:05.773091+0000","last_undegraded":"2023-07-21T18:40:05.773091+0000","last_fullsized":"2023-07-21T18:40:05.773091+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:40:26.891907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'3","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:10.243533+0000","last_change":"2023-07-21T18:40:04.184986+0000","last_active":"2023-07-21T18:40:10.243533+0000","last_peered":"2023-07-21T18:40:10.243533+0000","last_clean":"2023-07-21T18:40:10.243533+0000","last_became_active":"2023-07-21T18:40:04.184672+0000","last_became_peered":"2023-07-21T18:40:04.184672+0000","last_unstale":"2023-07-21T18:40:10.243533+0000","last_undegraded":"2023-07-21T18:40:10.243533+0000","last_fullsized":"2023-07-21T18:40:10.243533+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:40:47.955426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.036702+0000","last_change":"2023-07-21T18:40:04.793272+0000","last_active":"2023-07-21T18:40:07.036702+0000","last_peered":"2023-07-21T18:40:07.036702+0000","last_clean":"2023-07-21T18:40:07.036702+0000","last_became_active":"2023-07-21T18:40:04.792854+0000","last_became_peered":"2023-07-21T18:40:04.792854+0000","last_unstale":"2023-07-21T18:40:07.036702+0000","last_undegraded":"2023-07-21T18:40:07.036702+0000","last_fullsized":"2023-07-21T18:40:07.036702+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:22:05.555012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:08.212137+0000","last_change":"2023-07-21T18:40:04.176404+0000","last_active":"2023-07-21T18:40:08.212137+0000","last_peered":"2023-07-21T18:40:08.212137+0000","last_clean":"2023-07-21T18:40:08.212137+0000","last_became_active":"2023-07-21T18:40:04.176148+0000","last_became_peered":"2023-07-21T18:40:04.176148+0000","last_unstale":"2023-07-21T18:40:08.212137+0000","last_undegraded":"2023-07-21T18:40:08.212137+0000","last_fullsized":"2023-07-21T18:40:08.212137+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:29:41.873295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.090128+0000","last_change":"2023-07-21T18:40:04.795628+0000","last_active":"2023-07-21T18:40:06.090128+0000","last_peered":"2023-07-21T18:40:06.090128+0000","last_clean":"2023-07-21T18:40:06.090128+0000","last_became_active":"2023-07-21T18:40:04.795248+0000","last_became_peered":"2023-07-21T18:40:04.795248+0000","last_unstale":"2023-07-21T18:40:06.090128+0000","last_undegraded":"2023-07-21T18:40:06.090128+0000","last_fullsized":"2023-07-21T18:40:06.090128+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:05:26.124641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'5","reported_seq":59,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:47.021669+0000","last_change":"2023-07-21T18:40:04.184380+0000","last_active":"2023-07-21T18:40:47.021669+0000","last_peered":"2023-07-21T18:40:47.021669+0000","last_clean":"2023-07-21T18:40:47.021669+0000","last_became_active":"2023-07-21T18:40:04.184045+0000","last_became_peered":"2023-07-21T18:40:04.184045+0000","last_unstale":"2023-07-21T18:40:47.021669+0000","last_undegraded":"2023-07-21T18:40:47.021669+0000","last_fullsized":"2023-07-21T18:40:47.021669+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:58:31.782143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":40,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.090407+0000","last_change":"2023-07-21T18:40:04.787760+0000","last_active":"2023-07-21T18:40:06.090407+0000","last_peered":"2023-07-21T18:40:06.090407+0000","last_clean":"2023-07-21T18:40:06.090407+0000","last_became_active":"2023-07-21T18:40:04.787371+0000","last_became_peered":"2023-07-21T18:40:04.787371+0000","last_unstale":"2023-07-21T18:40:06.090407+0000","last_undegraded":"2023-07-21T18:40:06.090407+0000","last_fullsized":"2023-07-21T18:40:06.090407+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:10:11.576057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'3","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:10.962506+0000","last_change":"2023-07-21T18:40:04.184721+0000","last_active":"2023-07-21T18:40:10.962506+0000","last_peered":"2023-07-21T18:40:10.962506+0000","last_clean":"2023-07-21T18:40:10.962506+0000","last_became_active":"2023-07-21T18:40:04.184070+0000","last_became_peered":"2023-07-21T18:40:04.184070+0000","last_unstale":"2023-07-21T18:40:10.962506+0000","last_undegraded":"2023-07-21T18:40:10.962506+0000","last_fullsized":"2023-07-21T18:40:10.962506+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:57:46.774015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"79'1","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.090270+0000","last_change":"2023-07-21T18:40:04.786963+0000","last_active":"2023-07-21T18:40:06.090270+0000","last_peered":"2023-07-21T18:40:06.090270+0000","last_clean":"2023-07-21T18:40:06.090270+0000","last_became_active":"2023-07-21T18:40:04.786617+0000","last_became_peered":"2023-07-21T18:40:04.786617+0000","last_unstale":"2023-07-21T18:40:06.090270+0000","last_undegraded":"2023-07-21T18:40:06.090270+0000","last_fullsized":"2023-07-21T18:40:06.090270+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:22:45.940347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'5","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:10.975017+0000","last_change":"2023-07-21T18:40:04.184509+0000","last_active":"2023-07-21T18:40:10.975017+0000","last_peered":"2023-07-21T18:40:10.975017+0000","last_clean":"2023-07-21T18:40:10.975017+0000","last_became_active":"2023-07-21T18:40:04.184208+0000","last_became_peered":"2023-07-21T18:40:04.184208+0000","last_unstale":"2023-07-21T18:40:10.975017+0000","last_undegraded":"2023-07-21T18:40:10.975017+0000","last_fullsized":"2023-07-21T18:40:10.975017+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:18:05.113810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.090384+0000","last_change":"2023-07-21T18:40:04.788248+0000","last_active":"2023-07-21T18:40:06.090384+0000","last_peered":"2023-07-21T18:40:06.090384+0000","last_clean":"2023-07-21T18:40:06.090384+0000","last_became_active":"2023-07-21T18:40:04.787974+0000","last_became_peered":"2023-07-21T18:40:04.787974+0000","last_unstale":"2023-07-21T18:40:06.090384+0000","last_undegraded":"2023-07-21T18:40:06.090384+0000","last_fullsized":"2023-07-21T18:40:06.090384+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:58:18.957824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.773170+0000","last_change":"2023-07-21T18:40:04.790522+0000","last_active":"2023-07-21T18:40:05.773170+0000","last_peered":"2023-07-21T18:40:05.773170+0000","last_clean":"2023-07-21T18:40:05.773170+0000","last_became_active":"2023-07-21T18:40:04.790178+0000","last_became_peered":"2023-07-21T18:40:04.790178+0000","last_unstale":"2023-07-21T18:40:05.773170+0000","last_undegraded":"2023-07-21T18:40:05.773170+0000","last_fullsized":"2023-07-21T18:40:05.773170+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:39:58.451665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.592261+0000","last_change":"2023-07-21T18:40:04.790403+0000","last_active":"2023-07-21T18:40:06.592261+0000","last_peered":"2023-07-21T18:40:06.592261+0000","last_clean":"2023-07-21T18:40:06.592261+0000","last_became_active":"2023-07-21T18:40:04.790106+0000","last_became_peered":"2023-07-21T18:40:04.790106+0000","last_unstale":"2023-07-21T18:40:06.592261+0000","last_undegraded":"2023-07-21T18:40:06.592261+0000","last_fullsized":"2023-07-21T18:40:06.592261+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:22:57.358209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.592288+0000","last_change":"2023-07-21T18:40:04.795230+0000","last_active":"2023-07-21T18:40:06.592288+0000","last_peered":"2023-07-21T18:40:06.592288+0000","last_clean":"2023-07-21T18:40:06.592288+0000","last_became_active":"2023-07-21T18:40:04.794949+0000","last_became_peered":"2023-07-21T18:40:04.794949+0000","last_unstale":"2023-07-21T18:40:06.592288+0000","last_undegraded":"2023-07-21T18:40:06.592288+0000","last_fullsized":"2023-07-21T18:40:06.592288+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:25:52.872388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.037364+0000","last_change":"2023-07-21T18:40:04.792736+0000","last_active":"2023-07-21T18:40:07.037364+0000","last_peered":"2023-07-21T18:40:07.037364+0000","last_clean":"2023-07-21T18:40:07.037364+0000","last_became_active":"2023-07-21T18:40:04.792301+0000","last_became_peered":"2023-07-21T18:40:04.792301+0000","last_unstale":"2023-07-21T18:40:07.037364+0000","last_undegraded":"2023-07-21T18:40:07.037364+0000","last_fullsized":"2023-07-21T18:40:07.037364+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:40:20.170397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.772638+0000","last_change":"2023-07-21T18:40:04.790733+0000","last_active":"2023-07-21T18:40:05.772638+0000","last_peered":"2023-07-21T18:40:05.772638+0000","last_clean":"2023-07-21T18:40:05.772638+0000","last_became_active":"2023-07-21T18:40:04.790441+0000","last_became_peered":"2023-07-21T18:40:04.790441+0000","last_unstale":"2023-07-21T18:40:05.772638+0000","last_undegraded":"2023-07-21T18:40:05.772638+0000","last_fullsized":"2023-07-21T18:40:05.772638+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:29:55.466493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.773446+0000","last_change":"2023-07-21T18:40:04.790853+0000","last_active":"2023-07-21T18:40:05.773446+0000","last_peered":"2023-07-21T18:40:05.773446+0000","last_clean":"2023-07-21T18:40:05.773446+0000","last_became_active":"2023-07-21T18:40:04.790373+0000","last_became_peered":"2023-07-21T18:40:04.790373+0000","last_unstale":"2023-07-21T18:40:05.773446+0000","last_undegraded":"2023-07-21T18:40:05.773446+0000","last_fullsized":"2023-07-21T18:40:05.773446+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:59:25.687337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.592674+0000","last_change":"2023-07-21T18:40:04.794088+0000","last_active":"2023-07-21T18:40:06.592674+0000","last_peered":"2023-07-21T18:40:06.592674+0000","last_clean":"2023-07-21T18:40:06.592674+0000","last_became_active":"2023-07-21T18:40:04.793782+0000","last_became_peered":"2023-07-21T18:40:04.793782+0000","last_unstale":"2023-07-21T18:40:06.592674+0000","last_undegraded":"2023-07-21T18:40:06.592674+0000","last_fullsized":"2023-07-21T18:40:06.592674+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:14:52.604091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.037053+0000","last_change":"2023-07-21T18:40:04.791772+0000","last_active":"2023-07-21T18:40:07.037053+0000","last_peered":"2023-07-21T18:40:07.037053+0000","last_clean":"2023-07-21T18:40:07.037053+0000","last_became_active":"2023-07-21T18:40:04.791379+0000","last_became_peered":"2023-07-21T18:40:04.791379+0000","last_unstale":"2023-07-21T18:40:07.037053+0000","last_undegraded":"2023-07-21T18:40:07.037053+0000","last_fullsized":"2023-07-21T18:40:07.037053+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:37:36.712528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.994956+0000","last_change":"2023-07-21T18:40:04.786579+0000","last_active":"2023-07-21T18:40:06.994956+0000","last_peered":"2023-07-21T18:40:06.994956+0000","last_clean":"2023-07-21T18:40:06.994956+0000","last_became_active":"2023-07-21T18:40:04.786032+0000","last_became_peered":"2023-07-21T18:40:04.786032+0000","last_unstale":"2023-07-21T18:40:06.994956+0000","last_undegraded":"2023-07-21T18:40:06.994956+0000","last_fullsized":"2023-07-21T18:40:06.994956+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:34:12.940522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.036500+0000","last_change":"2023-07-21T18:40:04.793441+0000","last_active":"2023-07-21T18:40:07.036500+0000","last_peered":"2023-07-21T18:40:07.036500+0000","last_clean":"2023-07-21T18:40:07.036500+0000","last_became_active":"2023-07-21T18:40:04.791441+0000","last_became_peered":"2023-07-21T18:40:04.791441+0000","last_unstale":"2023-07-21T18:40:07.036500+0000","last_undegraded":"2023-07-21T18:40:07.036500+0000","last_fullsized":"2023-07-21T18:40:07.036500+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:01:49.043942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.777066+0000","last_change":"2023-07-21T18:40:04.794913+0000","last_active":"2023-07-21T18:40:05.777066+0000","last_peered":"2023-07-21T18:40:05.777066+0000","last_clean":"2023-07-21T18:40:05.777066+0000","last_became_active":"2023-07-21T18:40:04.794527+0000","last_became_peered":"2023-07-21T18:40:04.794527+0000","last_unstale":"2023-07-21T18:40:05.777066+0000","last_undegraded":"2023-07-21T18:40:05.777066+0000","last_fullsized":"2023-07-21T18:40:05.777066+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:13:56.682713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.772671+0000","last_change":"2023-07-21T18:40:04.789145+0000","last_active":"2023-07-21T18:40:05.772671+0000","last_peered":"2023-07-21T18:40:05.772671+0000","last_clean":"2023-07-21T18:40:05.772671+0000","last_became_active":"2023-07-21T18:40:04.788285+0000","last_became_peered":"2023-07-21T18:40:04.788285+0000","last_unstale":"2023-07-21T18:40:05.772671+0000","last_undegraded":"2023-07-21T18:40:05.772671+0000","last_fullsized":"2023-07-21T18:40:05.772671+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:42:10.814436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.773199+0000","last_change":"2023-07-21T18:40:04.788821+0000","last_active":"2023-07-21T18:40:05.773199+0000","last_peered":"2023-07-21T18:40:05.773199+0000","last_clean":"2023-07-21T18:40:05.773199+0000","last_became_active":"2023-07-21T18:40:04.788513+0000","last_became_peered":"2023-07-21T18:40:04.788513+0000","last_unstale":"2023-07-21T18:40:05.773199+0000","last_undegraded":"2023-07-21T18:40:05.773199+0000","last_fullsized":"2023-07-21T18:40:05.773199+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:35:28.640398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63495,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":417,"num_read_kb":276,"num_write":228,"num_write_kb":124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":190485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":200,"ondisk_log_size":200,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579925,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27792,"kb_used_data":840,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717360,"statfs":{"total":95995035648,"available":95966576640,"internally_reserved":0,"allocated":860160,"data_stored":408894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.4990000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.59599999999999997}]},{"osd":1,"last update":"Fri Jul 21 18:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.77500000000000002}]},{"osd":2,"last update":"Fri Jul 21 18:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.64400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":4,"last update":"Fri Jul 21 18:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.68700000000000006}]},{"osd":5,"last update":"Fri Jul 21 18:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.69799999999999995}]},{"osd":6,"last update":"Fri Jul 21 18:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.72499999999999998}]},{"osd":7,"last update":"Fri Jul 21 18:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.87},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.85699999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78900000000000003}]},{"osd":9,"last update":"Fri Jul 21 18:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.643,"5min":1.643,"15min":1.643},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.95899999999999996}]},{"osd":10,"last update":"Fri Jul 21 18:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.89000000000000001}]}]},{"osd":10,"up_from":67,"seq":287762808856,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28468,"kb_used_data":1516,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716684,"statfs":{"total":95995035648,"available":95965884416,"internally_reserved":0,"allocated":1552384,"data_stored":1056863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":1,"last update":"Fri Jul 21 18:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.624}]},{"osd":2,"last update":"Fri Jul 21 18:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.79800000000000004}]},{"osd":3,"last update":"Fri Jul 21 18:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.755}]},{"osd":4,"last update":"Fri Jul 21 18:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.81799999999999995}]},{"osd":5,"last update":"Fri Jul 21 18:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.96899999999999997}]},{"osd":6,"last update":"Fri Jul 21 18:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.874},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":1.0069999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":8,"last update":"Fri Jul 21 18:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.98399999999999999}]},{"osd":9,"last update":"Fri Jul 21 18:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.89000000000000001}]},{"osd":11,"last update":"Fri Jul 21 18:40:28 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":1.032}]}]},{"osd":9,"up_from":62,"seq":266287972381,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27816,"kb_used_data":864,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717336,"statfs":{"total":95995035648,"available":95966552064,"internally_reserved":0,"allocated":884736,"data_stored":408872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.52500000000000002,"15min":0.53900000000000003},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.996,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.438,"5min":0.505,"15min":0.51600000000000001},"min":{"1min":0.316,"5min":0.252,"15min":0.252},"max":{"1min":0.621,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.621}]},{"osd":1,"last update":"Fri Jul 21 18:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.73199999999999998,"15min":0.749},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.099,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.71299999999999997,"15min":0.72099999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.169,"5min":1.681,"15min":1.681},"last":0.59599999999999997}]},{"osd":2,"last update":"Fri Jul 21 18:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.70799999999999996,"15min":0.71399999999999997},"min":{"1min":0.47299999999999998,"5min":0.38,"15min":0.38},"max":{"1min":1.222,"5min":1.9530000000000001,"15min":1.9530000000000001},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.78300000000000003,"15min":0.753},"min":{"1min":0.46100000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":6.4020000000000001,"5min":6.4020000000000001,"15min":6.4020000000000001},"last":0.46100000000000002}]},{"osd":3,"last update":"Fri Jul 21 18:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.627,"15min":0.64900000000000002},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.89000000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.627,"15min":0.64500000000000002},"min":{"1min":0.35199999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.1910000000000001,"5min":2.048,"15min":2.048},"last":0.65000000000000002}]},{"osd":4,"last update":"Fri Jul 21 18:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.80700000000000005,"15min":0.83699999999999997},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.127,"5min":1.8100000000000001,"15min":1.8100000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.79800000000000004,"15min":0.82299999999999995},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.94799999999999995,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.69899999999999995}]},{"osd":5,"last update":"Fri Jul 21 18:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.80500000000000005,"15min":0.77700000000000002},"min":{"1min":0.49299999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":6.5069999999999997,"5min":6.5069999999999997,"15min":6.5069999999999997},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.78600000000000003,"15min":0.80400000000000005},"min":{"1min":0.503,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.1399999999999999,"5min":1.992,"15min":1.992},"last":0.76400000000000001}]},{"osd":6,"last update":"Fri Jul 21 18:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.74299999999999999,"15min":0.77300000000000002},"min":{"1min":0.375,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.1240000000000001,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.78600000000000003,"15min":0.81399999999999995},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.2410000000000001,"5min":1.595,"15min":1.595},"last":0.73199999999999998}]},{"osd":7,"last update":"Fri Jul 21 18:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.85799999999999998,"15min":0.88200000000000001},"min":{"1min":0.54700000000000004,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.026,"5min":1.284,"15min":1.284},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.89300000000000002,"15min":0.92000000000000004},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.175,"5min":2.0750000000000002,"15min":2.0750000000000002},"last":0.80900000000000005}]},{"osd":8,"last update":"Fri Jul 21 18:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.90700000000000003,"15min":0.92900000000000005},"min":{"1min":0.61499999999999999,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.2749999999999999,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":0.79400000000000004},{"interface":"front","average":{"1min":1.0669999999999999,"5min":0.90600000000000003,"15min":0.879},"min":{"1min":0.60899999999999999,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":6.5679999999999996,"5min":6.5679999999999996,"15min":6.5679999999999996},"last":0.82599999999999996}]},{"osd":10,"last update":"Fri Jul 21 18:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":2.1240000000000001,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.85699999999999998}]}]},{"osd":8,"up_from":55,"seq":236223201314,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27808,"kb_used_data":856,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717344,"statfs":{"total":95995035648,"available":95966560256,"internally_reserved":0,"allocated":876544,"data_stored":408919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:40:23 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.61699999999999999,"15min":0.60899999999999999},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":4.2000000000000002,"5min":4.2000000000000002,"15min":4.2000000000000002},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.59399999999999997,"15min":0.58599999999999997},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":3.145,"5min":3.145,"15min":3.145},"last":0.54200000000000004}]},{"osd":1,"last update":"Fri Jul 21 18:40:23 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.59099999999999997,"15min":0.58199999999999996},"min":{"1min":0.38800000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.90400000000000003,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.624,"5min":0.61399999999999999,"15min":0.61199999999999999},"min":{"1min":0.41999999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.878,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.71699999999999997}]},{"osd":2,"last update":"Fri Jul 21 18:40:23 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.46100000000000002,"15min":0.45900000000000002},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.72499999999999998,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.45600000000000002}]},{"osd":3,"last update":"Fri Jul 21 18:40:23 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":0.73799999999999999,"15min":0.68600000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":11.260999999999999,"5min":11.260999999999999,"15min":11.260999999999999},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.627,"15min":0.623},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":3.2719999999999998,"5min":3.2719999999999998,"15min":3.2719999999999998},"last":0.61899999999999999}]},{"osd":4,"last update":"Fri Jul 21 18:40:23 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.56999999999999995,"15min":0.56499999999999995},"min":{"1min":0.39500000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.629,"15min":0.628},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.78000000000000003,"5min":0.875,"15min":0.875},"last":0.66200000000000003}]},{"osd":5,"last update":"Fri Jul 21 18:40:23 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.58099999999999996,"15min":0.58299999999999996},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.83499999999999996,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.748},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52900000000000003,"15min":0.53100000000000003},"min":{"1min":0.35599999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.72299999999999998,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.49399999999999999}]},{"osd":6,"last update":"Fri Jul 21 18:40:23 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":0.78500000000000003,"15min":0.73699999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":10.292999999999999,"5min":10.292999999999999,"15min":10.292999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.77700000000000002,"15min":0.76500000000000001},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":4.1029999999999998,"5min":4.1029999999999998,"15min":4.1029999999999998},"last":0.69799999999999995}]},{"osd":7,"last update":"Fri Jul 21 18:40:23 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.71099999999999997,"15min":0.71499999999999997},"min":{"1min":0.44500000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.97599999999999998,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.66900000000000004,"15min":0.67100000000000004},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.80600000000000005,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.72999999999999998}]},{"osd":9,"last update":"Fri Jul 21 18:40:46 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.54100000000000004,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.95199999999999996,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":1.2509999999999999,"5min":0.86699999999999999,"15min":0.80300000000000005},"min":{"1min":0.54900000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":11.151,"5min":11.151,"15min":11.151},"last":0.84199999999999997}]},{"osd":10,"last update":"Fri Jul 21 18:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.89800000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]}]},{"osd":7,"up_from":48,"seq":206158430246,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28392,"kb_used_data":1440,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716760,"statfs":{"total":95995035648,"available":95965962240,"internally_reserved":0,"allocated":1474560,"data_stored":1001356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.53400000000000003,"15min":0.52600000000000002},"min":{"1min":0.41599999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.53900000000000003,"15min":0.53600000000000003},"min":{"1min":0.34000000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.48899999999999999}]},{"osd":1,"last update":"Fri Jul 21 18:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.41099999999999998,"15min":0.40400000000000003},"min":{"1min":0.311,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.43099999999999999,"15min":0.42299999999999999},"min":{"1min":0.307,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.71099999999999997,"5min":0.75,"15min":0.75},"last":0.41499999999999998}]},{"osd":2,"last update":"Fri Jul 21 18:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.57599999999999996,"15min":0.57099999999999995},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58899999999999997,"15min":0.58999999999999997},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.83299999999999996}]},{"osd":3,"last update":"Fri Jul 21 18:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.54900000000000004,"15min":0.54000000000000004},"min":{"1min":0.39700000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.58299999999999996,"15min":0.57199999999999995},"min":{"1min":0.44500000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.86899999999999999,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.628}]},{"osd":4,"last update":"Fri Jul 21 18:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.52700000000000002,"15min":0.52100000000000002},"min":{"1min":0.35399999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.52500000000000002,"15min":0.51700000000000002},"min":{"1min":0.39800000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.89500000000000002,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.44500000000000001}]},{"osd":5,"last update":"Fri Jul 21 18:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.64800000000000002,"15min":0.64100000000000001},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.64600000000000002,"15min":0.64200000000000002},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.79800000000000004}]},{"osd":6,"last update":"Fri Jul 21 18:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.66500000000000004,"15min":0.65500000000000003},"min":{"1min":0.51300000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.93500000000000005,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.67300000000000004,"15min":0.66100000000000003},"min":{"1min":0.46400000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0069999999999999,"5min":1.087,"15min":1.087},"last":0.56299999999999994}]},{"osd":8,"last update":"Fri Jul 21 18:40:22 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.69199999999999995,"15min":0.68999999999999995},"min":{"1min":0.54200000000000004,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.746,"5min":0.71999999999999997,"15min":0.71599999999999997},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.85399999999999998}]},{"osd":9,"last update":"Fri Jul 21 18:40:46 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.74199999999999999,"15min":0.73499999999999999},"min":{"1min":0.46500000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.98099999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.74199999999999999,"15min":0.73599999999999999},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.73299999999999998}]},{"osd":10,"last update":"Fri Jul 21 18:40:06 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.76900000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91200000000000003}]}]},{"osd":6,"up_from":43,"seq":184683593770,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27796,"kb_used_data":844,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717356,"statfs":{"total":95995035648,"available":95966572544,"internally_reserved":0,"allocated":864256,"data_stored":408274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:40:40 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.41299999999999998,"15min":0.40899999999999997},"min":{"1min":0.29599999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.504},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.48099999999999998,"15min":0.44500000000000001},"min":{"1min":0.311,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":4.859,"5min":4.859,"15min":4.859},"last":0.51800000000000002}]},{"osd":1,"last update":"Fri Jul 21 18:40:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.44500000000000001,"5min":0.37,"15min":0.37},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.39000000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.84399999999999997,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.65300000000000002}]},{"osd":2,"last update":"Fri Jul 21 18:40:40 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.64800000000000002,"15min":0.60399999999999998},"min":{"1min":0.39900000000000002,"5min":0.376,"15min":0.376},"max":{"1min":4.9180000000000001,"5min":4.9180000000000001,"15min":4.9180000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.59599999999999997,"15min":0.58199999999999996},"min":{"1min":0.42099999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.83399999999999996}]},{"osd":3,"last update":"Fri Jul 21 18:40:40 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.52300000000000002,"15min":0.48799999999999999},"min":{"1min":0.33800000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":4.4720000000000004,"5min":4.4720000000000004,"15min":4.4720000000000004},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.56999999999999995,"15min":0.53100000000000003},"min":{"1min":0.32200000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":5.2910000000000004,"5min":5.2910000000000004,"15min":5.2910000000000004},"last":0.441}]},{"osd":4,"last update":"Fri Jul 21 18:40:40 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.63900000000000001,"15min":0.64400000000000002},"min":{"1min":0.40300000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.022,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.64100000000000001,"15min":0.64400000000000002},"min":{"1min":0.46500000000000002,"5min":0.442,"15min":0.442},"max":{"1min":0.89600000000000002,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.84899999999999998}]},{"osd":5,"last update":"Fri Jul 21 18:40:40 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.70699999999999996,"15min":0.66600000000000004},"min":{"1min":0.47499999999999998,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":4.9960000000000004,"5min":4.9960000000000004,"15min":4.9960000000000004},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.70299999999999996,"15min":0.69499999999999995},"min":{"1min":0.49299999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.145,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.63700000000000001}]},{"osd":7,"last update":"Fri Jul 21 18:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.71299999999999997,"15min":0.70899999999999996},"min":{"1min":0.53500000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.874},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.70599999999999996,"15min":0.70499999999999996},"min":{"1min":0.54300000000000004,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.68600000000000005}]},{"osd":8,"last update":"Fri Jul 21 18:40:26 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.79000000000000004,"15min":0.80600000000000005},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.96099999999999997,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.77500000000000002,"15min":0.78700000000000003},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.91800000000000004,"5min":1.171,"15min":1.171},"last":0.81299999999999994}]},{"osd":9,"last update":"Fri Jul 21 18:40:44 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.75,"15min":0.75600000000000001},"min":{"1min":0.56799999999999995,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.194,"5min":1.196,"15min":1.196},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.72599999999999998,"15min":0.73399999999999999},"min":{"1min":0.52600000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.872,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.54300000000000004}]},{"osd":10,"last update":"Fri Jul 21 18:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.82299999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":900,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":921600,"data_stored":466056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:39:59 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.50900000000000001,"15min":0.48499999999999999},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.52800000000000002,"15min":0.498},"min":{"1min":0.39200000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.65100000000000002}]},{"osd":2,"last update":"Fri Jul 21 18:40:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.55900000000000005,"15min":0.52800000000000002},"min":{"1min":0.38600000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.84099999999999997,"5min":3.2959999999999998,"15min":3.2959999999999998},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.55400000000000005,"15min":0.52600000000000002},"min":{"1min":0.34999999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.93300000000000005,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":0.85099999999999998}]},{"osd":3,"last update":"Fri Jul 21 18:40:44 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.55300000000000005,"15min":0.52500000000000002},"min":{"1min":0.38400000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.499},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.58099999999999996,"15min":0.53700000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":3.7789999999999999,"5min":3.7789999999999999,"15min":3.7789999999999999},"last":0.53400000000000003}]},{"osd":4,"last update":"Fri Jul 21 18:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.41799999999999998,"15min":0.40100000000000002},"min":{"1min":0.32600000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.497,"5min":0.41799999999999998,"15min":0.40500000000000003},"min":{"1min":0.31,"5min":0.254,"15min":0.254},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.77900000000000003}]},{"osd":5,"last update":"Fri Jul 21 18:40:19 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.60699999999999998,"15min":0.58799999999999997},"min":{"1min":0.45100000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.089,"5min":1.653,"15min":1.653},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.623,"15min":0.59799999999999998},"min":{"1min":0.50800000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.94999999999999996,"5min":1.641,"15min":1.641},"last":0.88600000000000001}]},{"osd":6,"last update":"Fri Jul 21 18:40:44 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.64400000000000002,"15min":0.63200000000000001},"min":{"1min":0.46200000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.92900000000000005,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.64500000000000002,"15min":0.64000000000000001},"min":{"1min":0.41599999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.96099999999999997,"5min":1.022,"15min":1.022},"last":0.71399999999999997}]},{"osd":7,"last update":"Fri Jul 21 18:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.51900000000000002,"15min":0.502},"min":{"1min":0.41399999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.54800000000000004,"15min":0.53000000000000003},"min":{"1min":0.40699999999999997,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.871}]},{"osd":8,"last update":"Fri Jul 21 18:40:26 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.73599999999999999,"15min":0.72799999999999998},"min":{"1min":0.51500000000000001,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.1910000000000001,"5min":2.7360000000000002,"15min":2.7360000000000002},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.72699999999999998,"15min":0.72499999999999998},"min":{"1min":0.52400000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.96899999999999997,"5min":2.7210000000000001,"15min":2.7210000000000001},"last":0.94399999999999995}]},{"osd":9,"last update":"Fri Jul 21 18:40:44 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.745,"15min":0.73899999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.753,"15min":0.746},"min":{"1min":0.55100000000000005,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.91600000000000004}]},{"osd":10,"last update":"Fri Jul 21 18:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":1.0049999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99299999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738435,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27820,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717332,"statfs":{"total":95995035648,"available":95966547968,"internally_reserved":0,"allocated":888832,"data_stored":409880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 21 18:39:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.59099999999999997,"15min":0.57399999999999995},"min":{"1min":0.36299999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.88200000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.57699999999999996,"15min":0.53700000000000003},"min":{"1min":0.42899999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.82699999999999996}]},{"osd":2,"last update":"Fri Jul 21 18:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.58199999999999996,"15min":0.56699999999999995},"min":{"1min":0.39400000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.92800000000000005,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.60599999999999998,"15min":0.58599999999999997},"min":{"1min":0.44900000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":3.0470000000000002,"5min":3.0470000000000002,"15min":3.0470000000000002},"last":0.73799999999999999}]},{"osd":3,"last update":"Fri Jul 21 18:38:36 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.42799999999999999,"15min":0.42999999999999999},"min":{"1min":0.29999999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.621,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.48199999999999998,"15min":0.47699999999999998},"min":{"1min":0.33600000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.69199999999999995,"5min":0.879,"15min":0.879},"last":0.44500000000000001}]},{"osd":4,"last update":"Fri Jul 21 18:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.60599999999999998,"15min":0.57199999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":2.948,"5min":2.948,"15min":2.948},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.56200000000000006,"15min":0.54600000000000004},"min":{"1min":0.40600000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.80400000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.49099999999999999}]},{"osd":5,"last update":"Fri Jul 21 18:40:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.58799999999999997},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.748,"5min":0.64600000000000002,"15min":0.61199999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":3.0150000000000001,"5min":3.0150000000000001,"15min":3.0150000000000001},"last":0.50700000000000001}]},{"osd":6,"last update":"Fri Jul 21 18:40:46 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.52200000000000002,"15min":0.51100000000000001},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":2.077,"5min":2.077,"15min":2.077},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.58799999999999997,"15min":0.56899999999999995},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":2.4500000000000002,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":0.53800000000000003}]},{"osd":7,"last update":"Fri Jul 21 18:39:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.47699999999999998}]},{"osd":8,"last update":"Fri Jul 21 18:40:23 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.79700000000000004,"15min":0.81499999999999995},"min":{"1min":0.54600000000000004,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.90900000000000003,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81599999999999995,"15min":0.81699999999999995},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":2.9830000000000001,"5min":2.9830000000000001,"15min":2.9830000000000001},"last":0.58799999999999997}]},{"osd":9,"last update":"Fri Jul 21 18:40:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.71999999999999997,"15min":0.74199999999999999},"min":{"1min":0.37,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.78800000000000003,"15min":0.81299999999999994},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.421,"5min":1.421,"15min":1.421},"last":0.66800000000000004}]},{"osd":10,"last update":"Fri Jul 21 18:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":2.9079999999999999,"5min":2.9079999999999999,"15min":2.9079999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.84999999999999998}]},{"osd":11,"last update":"Fri Jul 21 18:40:26 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":3.0840000000000001,"5min":3.0840000000000001,"15min":3.0840000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.60599999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411387,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":924,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":946176,"data_stored":466501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:40:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.54000000000000004,"15min":0.53200000000000003},"min":{"1min":0.40000000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.92100000000000004,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.50800000000000001,"15min":0.503},"min":{"1min":0.31,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.80500000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.57499999999999996}]},{"osd":1,"last update":"Fri Jul 21 18:40:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.53500000000000003,"15min":0.51900000000000002},"min":{"1min":0.45200000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.76200000000000001,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.56000000000000005,"15min":0.53800000000000003},"min":{"1min":0.39300000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.79400000000000004}]},{"osd":3,"last update":"Fri Jul 21 18:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.58899999999999997,"15min":0.56499999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.92400000000000004,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.56799999999999995,"15min":0.56000000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.79600000000000004,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.70199999999999996}]},{"osd":4,"last update":"Fri Jul 21 18:39:57 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.58999999999999997,"15min":0.56999999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.633,"5min":1.633,"15min":1.633},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.626,"15min":0.60899999999999999},"min":{"1min":0.42999999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.77400000000000002}]},{"osd":5,"last update":"Fri Jul 21 18:40:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.47099999999999997,"15min":0.45700000000000002},"min":{"1min":0.29699999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.46700000000000003,"15min":0.45600000000000002},"min":{"1min":0.35799999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.60899999999999999}]},{"osd":6,"last update":"Fri Jul 21 18:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.61899999999999999,"15min":0.59599999999999997},"min":{"1min":0.53000000000000003,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.85799999999999998,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.628,"15min":0.62},"min":{"1min":0.436,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.84399999999999997,"5min":0.873,"15min":0.873},"last":0.81599999999999995}]},{"osd":7,"last update":"Fri Jul 21 18:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.61699999999999999,"15min":0.60799999999999998},"min":{"1min":0.45100000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.577,"5min":1.577,"15min":1.577},"last":0.495},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.65500000000000003,"15min":0.65100000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.878,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.56499999999999995}]},{"osd":8,"last update":"Fri Jul 21 18:40:26 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.58899999999999997,"15min":0.57099999999999995},"min":{"1min":0.40799999999999997,"5min":0.376,"15min":0.376},"max":{"1min":0.91300000000000003,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.60799999999999998,"15min":0.59799999999999998},"min":{"1min":0.40799999999999997,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.88600000000000001}]},{"osd":9,"last update":"Fri Jul 21 18:40:46 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.76000000000000001,"15min":0.75600000000000001},"min":{"1min":0.59499999999999997,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.95999999999999996,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.69199999999999995,"15min":0.68999999999999995},"min":{"1min":0.47499999999999998,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.89600000000000002,"5min":1.01,"15min":1.01},"last":0.87}]},{"osd":10,"last update":"Fri Jul 21 18:40:06 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.94699999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92800000000000005}]}]},{"osd":3,"up_from":24,"seq":103079215158,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27808,"kb_used_data":856,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717344,"statfs":{"total":95995035648,"available":95966560256,"internally_reserved":0,"allocated":876544,"data_stored":407822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.35699999999999998,"15min":0.33100000000000002},"min":{"1min":0.26800000000000002,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.54000000000000004,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.373,"15min":0.35099999999999998},"min":{"1min":0.28100000000000003,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.51600000000000001,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.32300000000000001}]},{"osd":1,"last update":"Fri Jul 21 18:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.57199999999999995,"15min":0.55100000000000005},"min":{"1min":0.39900000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.75800000000000001,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59199999999999997,"15min":0.55000000000000004},"min":{"1min":0.39400000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.83099999999999996,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.55100000000000005}]},{"osd":2,"last update":"Fri Jul 21 18:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.57699999999999996,"15min":0.55300000000000005},"min":{"1min":0.36799999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.81599999999999995,"5min":0.876,"15min":0.876},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.56799999999999995,"15min":0.53900000000000003},"min":{"1min":0.373,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.52900000000000003}]},{"osd":4,"last update":"Fri Jul 21 18:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.61599999999999999,"15min":0.60499999999999998},"min":{"1min":0.42799999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.61699999999999999,"15min":0.59999999999999998},"min":{"1min":0.45300000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.50800000000000001}]},{"osd":5,"last update":"Fri Jul 21 18:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.622,"15min":0.58999999999999997},"min":{"1min":0.45400000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.89000000000000001,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.442},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.59799999999999998,"15min":0.58199999999999996},"min":{"1min":0.39500000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.90500000000000003,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.46300000000000002}]},{"osd":6,"last update":"Fri Jul 21 18:40:41 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.498,"15min":0.48599999999999999},"min":{"1min":0.33600000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.80900000000000005,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.55800000000000005,"15min":0.55400000000000005},"min":{"1min":0.36599999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.63400000000000001,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.42399999999999999}]},{"osd":7,"last update":"Fri Jul 21 18:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.66800000000000004,"15min":0.67000000000000004},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.97299999999999998,"5min":0.999,"15min":0.999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.73299999999999998,"15min":0.72499999999999998},"min":{"1min":0.55100000000000005,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.65000000000000002}]},{"osd":8,"last update":"Fri Jul 21 18:39:25 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.47299999999999998}]},{"osd":9,"last update":"Fri Jul 21 18:40:45 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.68600000000000005,"15min":0.71499999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.77900000000000003,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.624,"5min":0.81999999999999995,"15min":0.85299999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.88100000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.57199999999999995}]},{"osd":10,"last update":"Fri Jul 21 18:40:07 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.69599999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]}]},{"osd":4,"up_from":31,"seq":133143986226,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27796,"kb_used_data":844,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717356,"statfs":{"total":95995035648,"available":95966572544,"internally_reserved":0,"allocated":864256,"data_stored":408519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:40:02 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.51800000000000002,"15min":0.48199999999999998},"min":{"1min":0.40500000000000003,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.94099999999999995,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.52500000000000002,"15min":0.50600000000000001},"min":{"1min":0.373,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.80600000000000005,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.58299999999999996}]},{"osd":1,"last update":"Fri Jul 21 18:40:02 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.373,"15min":0.34599999999999997},"min":{"1min":0.29799999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.40899999999999997,"15min":0.38600000000000001},"min":{"1min":0.30399999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.76200000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.51600000000000001}]},{"osd":2,"last update":"Fri Jul 21 18:40:02 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.54700000000000004,"15min":0.52900000000000003},"min":{"1min":0.44700000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.80700000000000005,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.56200000000000006,"15min":0.53200000000000003},"min":{"1min":0.41799999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.65400000000000003}]},{"osd":3,"last update":"Fri Jul 21 18:40:02 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.55800000000000005,"15min":0.53300000000000003},"min":{"1min":0.52300000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.85999999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56299999999999994,"15min":0.55700000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.97199999999999998,"5min":1.05,"15min":1.05},"last":0.73299999999999998}]},{"osd":5,"last update":"Fri Jul 21 18:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.64000000000000001,"15min":0.60599999999999998},"min":{"1min":0.48599999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.90800000000000003,"5min":1.571,"15min":1.571},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.64100000000000001,"15min":0.61499999999999999},"min":{"1min":0.434,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.90800000000000003,"5min":1.377,"15min":1.377},"last":0.68100000000000005}]},{"osd":6,"last update":"Fri Jul 21 18:40:44 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.63900000000000001,"15min":0.61699999999999999},"min":{"1min":0.498,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.83699999999999997,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.65000000000000002,"15min":0.61399999999999999},"min":{"1min":0.46100000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.81299999999999994,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.76200000000000001}]},{"osd":7,"last update":"Fri Jul 21 18:40:02 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.57299999999999995,"15min":0.56100000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.54500000000000004,"15min":0.54000000000000004},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.873,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.81499999999999995}]},{"osd":8,"last update":"Fri Jul 21 18:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.65900000000000003,"15min":0.66100000000000003},"min":{"1min":0.46200000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.92400000000000004,"5min":1.026,"15min":1.026},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.73099999999999998,"15min":0.72499999999999998},"min":{"1min":0.56899999999999995,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.91200000000000003}]},{"osd":9,"last update":"Fri Jul 21 18:40:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.79000000000000004,"15min":0.79900000000000004},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.93999999999999995,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.83599999999999997,"15min":0.84599999999999997},"min":{"1min":0.60499999999999998,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.95499999999999996,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.95399999999999996}]},{"osd":10,"last update":"Fri Jul 21 18:40:07 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.97499999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.879}]}]},{"osd":5,"up_from":36,"seq":154618822702,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28404,"kb_used_data":1452,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716748,"statfs":{"total":95995035648,"available":95965949952,"internally_reserved":0,"allocated":1486848,"data_stored":1000740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.52600000000000002,"15min":0.53000000000000003},"min":{"1min":0.313,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.54800000000000004,"15min":0.55700000000000005},"min":{"1min":0.316,"5min":0.314,"15min":0.314},"max":{"1min":0.78700000000000003,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.40799999999999997}]},{"osd":1,"last update":"Fri Jul 21 18:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.53400000000000003,"15min":0.50800000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.54000000000000004,"15min":0.51300000000000001},"min":{"1min":0.43099999999999999,"5min":0.375,"15min":0.375},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.505}]},{"osd":2,"last update":"Fri Jul 21 18:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.40000000000000002,"15min":0.38800000000000001},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.59299999999999997,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.41699999999999998,"15min":0.39800000000000002},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.623,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.47599999999999998}]},{"osd":3,"last update":"Fri Jul 21 18:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.56000000000000005,"15min":0.56799999999999995},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58199999999999996,"15min":0.58099999999999996},"min":{"1min":0.434,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.79800000000000004,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.442}]},{"osd":4,"last update":"Fri Jul 21 18:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.57199999999999995,"15min":0.55400000000000005},"min":{"1min":0.36899999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.83899999999999997,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.56200000000000006,"15min":0.54000000000000004},"min":{"1min":0.45700000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.63100000000000001}]},{"osd":6,"last update":"Fri Jul 21 18:40:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.59599999999999997,"15min":0.59999999999999998},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.60399999999999998,"15min":0.59099999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.66200000000000003}]},{"osd":7,"last update":"Fri Jul 21 18:40:02 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.66600000000000004,"15min":0.66800000000000004},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.64300000000000002,"15min":0.64000000000000001},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.748}]},{"osd":8,"last update":"Fri Jul 21 18:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.92200000000000004,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.626,"5min":0.63200000000000001,"15min":0.63300000000000001},"min":{"1min":0.40999999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.81399999999999995,"5min":0.874,"15min":0.874},"last":0.72999999999999998}]},{"osd":9,"last update":"Fri Jul 21 18:40:45 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.72499999999999998,"15min":0.72899999999999998},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.877,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.73199999999999998,"15min":0.73399999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.71399999999999997}]},{"osd":10,"last update":"Fri Jul 21 18:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.77900000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":58579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":58751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":58953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-21T18:40:51.930 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph pg dump --format=json 2023-07-21T18:40:52.189 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:51 smithi039 ceph-mon[96489]: pgmap v271: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-07-21T18:40:52.372 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:51 smithi193 ceph-mon[98204]: pgmap v271: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-07-21T18:40:52.390 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:51 smithi155 ceph-mon[97934]: pgmap v271: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-07-21T18:40:52.491 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:40:53.372 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:52 smithi193 ceph-mon[98204]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T18:40:53.391 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:52 smithi155 ceph-mon[97934]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T18:40:53.439 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:52 smithi039 ceph-mon[96489]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T18:40:53.751 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:53.725+0000 7f9a699a2700 1 -- 172.21.15.39:0/546164419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a64103550 msgr2=0x7f9a6410b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:53.751 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:53.725+0000 7f9a699a2700 1 --2- 172.21.15.39:0/546164419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a64103550 0x7f9a6410b0d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c00b0d0 tx=0x7f9a5c00b3e0 comp rx=0 tx=0).stop 2023-07-21T18:40:53.751 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:53.725+0000 7f9a699a2700 1 -- 172.21.15.39:0/546164419 shutdown_connections 2023-07-21T18:40:53.751 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:53.725+0000 7f9a699a2700 1 --2- 172.21.15.39:0/546164419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a64103550 0x7f9a6410b0d0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:53.751 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:53.725+0000 7f9a699a2700 1 --2- 172.21.15.39:0/546164419 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a640fe4b0 0x7f9a640fe880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:53.752 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:53.725+0000 7f9a699a2700 1 --2- 172.21.15.39:0/546164419 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a640fee50 0x7f9a64102e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:53.752 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:53.725+0000 7f9a699a2700 1 -- 172.21.15.39:0/546164419 >> 172.21.15.39:0/546164419 conn(0x7f9a64075150 msgr2=0x7f9a64075550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:53.752 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:53.725+0000 7f9a699a2700 1 -- 172.21.15.39:0/546164419 shutdown_connections 2023-07-21T18:40:53.752 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:53.725+0000 7f9a699a2700 1 -- 172.21.15.39:0/546164419 wait complete. 2023-07-21T18:40:53.752 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:53.726+0000 7f9a699a2700 1 Processor -- start 2023-07-21T18:40:53.752 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:53.726+0000 7f9a699a2700 1 -- start start 2023-07-21T18:40:53.754 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:53.726+0000 7f9a699a2700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a640fe4b0 0x7f9a64195e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:53.754 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:53.726+0000 7f9a699a2700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a640fee50 0x7f9a641963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:53.754 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:53.726+0000 7f9a699a2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a64103550 0x7f9a6419a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:53.754 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:53.727+0000 7f9a699a2700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6419ae30 con 0x7f9a64103550 2023-07-21T18:40:53.754 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:53.727+0000 7f9a699a2700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6419afa0 con 0x7f9a640fee50 2023-07-21T18:40:53.755 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:53.727+0000 7f9a699a2700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6419b160 con 0x7f9a640fe4b0 2023-07-21T18:40:53.755 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:53.727+0000 7f9a637fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a64103550 0x7f9a6419a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:53.755 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:53.727+0000 7f9a637fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a64103550 0x7f9a6419a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55688/0 (socket says 172.21.15.39:55688) 2023-07-21T18:40:53.755 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:53.727+0000 7f9a637fe700 1 -- 172.21.15.39:0/2601641266 learned_addr learned my addr 172.21.15.39:0/2601641266 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:40:53.755 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:53.727+0000 7f9a5a7fc700 1 --2- 172.21.15.39:0/2601641266 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a640fee50 0x7f9a641963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:53.755 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:53.727+0000 7f9a62ffd700 1 --2- 172.21.15.39:0/2601641266 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a640fe4b0 0x7f9a64195e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:53.755 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:53.727+0000 7f9a637fe700 1 -- 172.21.15.39:0/2601641266 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a640fe4b0 msgr2=0x7f9a64195e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:53.756 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:53.727+0000 7f9a637fe700 1 --2- 172.21.15.39:0/2601641266 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a640fe4b0 0x7f9a64195e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:53.756 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:53.727+0000 7f9a637fe700 1 -- 172.21.15.39:0/2601641266 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a640fee50 msgr2=0x7f9a641963b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:53.756 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:53.727+0000 7f9a637fe700 1 --2- 172.21.15.39:0/2601641266 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a640fee50 0x7f9a641963b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:53.756 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:53.727+0000 7f9a637fe700 1 -- 172.21.15.39:0/2601641266 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a5c009d00 con 0x7f9a64103550 2023-07-21T18:40:53.756 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:53.727+0000 7f9a5a7fc700 1 --2- 172.21.15.39:0/2601641266 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a640fee50 0x7f9a641963b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:40:53.756 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:53.727+0000 7f9a637fe700 1 --2- 172.21.15.39:0/2601641266 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a64103550 0x7f9a6419a820 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c007940 tx=0x7f9a5c007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:53.756 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:53.727+0000 7f9a689a0700 1 -- 172.21.15.39:0/2601641266 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a5c00e040 con 0x7f9a64103550 2023-07-21T18:40:53.756 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:53.727+0000 7f9a699a2700 1 -- 172.21.15.39:0/2601641266 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a6419b3e0 con 0x7f9a64103550 2023-07-21T18:40:53.757 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:53.727+0000 7f9a689a0700 1 -- 172.21.15.39:0/2601641266 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a5c003e10 con 0x7f9a64103550 2023-07-21T18:40:53.757 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:53.727+0000 7f9a689a0700 1 -- 172.21.15.39:0/2601641266 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a5c005620 con 0x7f9a64103550 2023-07-21T18:40:53.757 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:53.727+0000 7f9a699a2700 1 -- 172.21.15.39:0/2601641266 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a641a2ca0 con 0x7f9a64103550 2023-07-21T18:40:53.757 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:53.728+0000 7f9a699a2700 1 -- 172.21.15.39:0/2601641266 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a64107830 con 0x7f9a64103550 2023-07-21T18:40:53.758 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:53.729+0000 7f9a689a0700 1 -- 172.21.15.39:0/2601641266 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f9a5c005780 con 0x7f9a64103550 2023-07-21T18:40:53.758 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:53.730+0000 7f9a689a0700 1 --2- 172.21.15.39:0/2601641266 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f9a500a11e0 0x7f9a500a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:53.758 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:53.730+0000 7f9a62ffd700 1 --2- 172.21.15.39:0/2601641266 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f9a500a11e0 0x7f9a500a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:53.758 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:53.730+0000 7f9a689a0700 1 -- 172.21.15.39:0/2601641266 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f9a5c0ce4f0 con 0x7f9a64103550 2023-07-21T18:40:53.758 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:53.731+0000 7f9a62ffd700 1 --2- 172.21.15.39:0/2601641266 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f9a500a11e0 0x7f9a500a3690 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9a54005fd0 tx=0x7f9a5400a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:53.759 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:53.734+0000 7f9a689a0700 1 -- 172.21.15.39:0/2601641266 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f9a5c097d20 con 0x7f9a64103550 2023-07-21T18:40:53.952 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:53.927+0000 7f9a699a2700 1 -- 172.21.15.39:0/2601641266 --> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9a64197240 con 0x7f9a500a11e0 2023-07-21T18:40:53.964 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:53.939+0000 7f9a689a0700 1 -- 172.21.15.39:0/2601641266 <== mgr.14152 v2:172.21.15.39:6800/2116291712 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340951 (secure 0 0 0) 0x7f9a64197240 con 0x7f9a500a11e0 2023-07-21T18:40:53.969 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:40:53.970 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:53.942+0000 7f9a699a2700 1 -- 172.21.15.39:0/2601641266 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f9a500a11e0 msgr2=0x7f9a500a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:53.971 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:53.942+0000 7f9a699a2700 1 --2- 172.21.15.39:0/2601641266 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f9a500a11e0 0x7f9a500a3690 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9a54005fd0 tx=0x7f9a5400a560 comp rx=0 tx=0).stop 2023-07-21T18:40:53.971 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:53.942+0000 7f9a699a2700 1 -- 172.21.15.39:0/2601641266 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a64103550 msgr2=0x7f9a6419a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:53.971 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:53.942+0000 7f9a699a2700 1 --2- 172.21.15.39:0/2601641266 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a64103550 0x7f9a6419a820 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c007940 tx=0x7f9a5c007a20 comp rx=0 tx=0).stop 2023-07-21T18:40:53.971 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:53.943+0000 7f9a699a2700 1 -- 172.21.15.39:0/2601641266 shutdown_connections 2023-07-21T18:40:53.971 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:53.943+0000 7f9a699a2700 1 --2- 172.21.15.39:0/2601641266 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f9a500a11e0 0x7f9a500a3690 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:53.971 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:53.943+0000 7f9a699a2700 1 --2- 172.21.15.39:0/2601641266 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a64103550 0x7f9a6419a820 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:53.971 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:53.943+0000 7f9a699a2700 1 --2- 172.21.15.39:0/2601641266 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a640fe4b0 0x7f9a64195e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:53.972 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:53.943+0000 7f9a699a2700 1 --2- 172.21.15.39:0/2601641266 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a640fee50 0x7f9a641963b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:53.972 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:53.943+0000 7f9a699a2700 1 -- 172.21.15.39:0/2601641266 >> 172.21.15.39:0/2601641266 conn(0x7f9a64075150 msgr2=0x7f9a64102890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:53.972 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:53.943+0000 7f9a699a2700 1 -- 172.21.15.39:0/2601641266 shutdown_connections 2023-07-21T18:40:53.972 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:53.943+0000 7f9a699a2700 1 -- 172.21.15.39:0/2601641266 wait complete. 2023-07-21T18:40:53.972 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2023-07-21T18:40:54.365 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:53 smithi039 ceph-mon[96489]: pgmap v272: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-07-21T18:40:54.371 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:53 smithi193 ceph-mon[98204]: pgmap v272: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-07-21T18:40:54.390 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:53 smithi155 ceph-mon[97934]: pgmap v272: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-07-21T18:40:54.438 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":272,"stamp":"2023-07-21T18:40:53.131183+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654109,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":563,"num_read_kb":400,"num_write":411,"num_write_kb":3064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":339,"ondisk_log_size":339,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335628,"kb_used_data":12204,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124606196,"statfs":{"total":1151940427776,"available":1151596744704,"internally_reserved":0,"allocated":12496896,"data_stored":6852696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":109,"num_read_kb":55,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003146"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.773443+0000","last_change":"2023-07-21T18:40:04.792977+0000","last_active":"2023-07-21T18:40:05.773443+0000","last_peered":"2023-07-21T18:40:05.773443+0000","last_clean":"2023-07-21T18:40:05.773443+0000","last_became_active":"2023-07-21T18:40:04.792667+0000","last_became_peered":"2023-07-21T18:40:04.792667+0000","last_unstale":"2023-07-21T18:40:05.773443+0000","last_undegraded":"2023-07-21T18:40:05.773443+0000","last_fullsized":"2023-07-21T18:40:05.773443+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:37:51.249650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.035763+0000","last_change":"2023-07-21T18:40:04.792936+0000","last_active":"2023-07-21T18:40:07.035763+0000","last_peered":"2023-07-21T18:40:07.035763+0000","last_clean":"2023-07-21T18:40:07.035763+0000","last_became_active":"2023-07-21T18:40:04.792511+0000","last_became_peered":"2023-07-21T18:40:04.792511+0000","last_unstale":"2023-07-21T18:40:07.035763+0000","last_undegraded":"2023-07-21T18:40:07.035763+0000","last_fullsized":"2023-07-21T18:40:07.035763+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:05:00.595279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.035567+0000","last_change":"2023-07-21T18:40:04.796401+0000","last_active":"2023-07-21T18:40:07.035567+0000","last_peered":"2023-07-21T18:40:07.035567+0000","last_clean":"2023-07-21T18:40:07.035567+0000","last_became_active":"2023-07-21T18:40:04.796063+0000","last_became_peered":"2023-07-21T18:40:04.796063+0000","last_unstale":"2023-07-21T18:40:07.035567+0000","last_undegraded":"2023-07-21T18:40:07.035567+0000","last_fullsized":"2023-07-21T18:40:07.035567+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:11:42.127453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.035756+0000","last_change":"2023-07-21T18:40:04.792308+0000","last_active":"2023-07-21T18:40:07.035756+0000","last_peered":"2023-07-21T18:40:07.035756+0000","last_clean":"2023-07-21T18:40:07.035756+0000","last_became_active":"2023-07-21T18:40:04.791831+0000","last_became_peered":"2023-07-21T18:40:04.791831+0000","last_unstale":"2023-07-21T18:40:07.035756+0000","last_undegraded":"2023-07-21T18:40:07.035756+0000","last_fullsized":"2023-07-21T18:40:07.035756+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:31:30.077525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.006829+0000","last_change":"2023-07-21T18:40:04.794035+0000","last_active":"2023-07-21T18:40:06.006829+0000","last_peered":"2023-07-21T18:40:06.006829+0000","last_clean":"2023-07-21T18:40:06.006829+0000","last_became_active":"2023-07-21T18:40:04.793739+0000","last_became_peered":"2023-07-21T18:40:04.793739+0000","last_unstale":"2023-07-21T18:40:06.006829+0000","last_undegraded":"2023-07-21T18:40:06.006829+0000","last_fullsized":"2023-07-21T18:40:06.006829+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:40:42.480359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.035838+0000","last_change":"2023-07-21T18:40:04.792751+0000","last_active":"2023-07-21T18:40:07.035838+0000","last_peered":"2023-07-21T18:40:07.035838+0000","last_clean":"2023-07-21T18:40:07.035838+0000","last_became_active":"2023-07-21T18:40:04.792413+0000","last_became_peered":"2023-07-21T18:40:04.792413+0000","last_unstale":"2023-07-21T18:40:07.035838+0000","last_undegraded":"2023-07-21T18:40:07.035838+0000","last_fullsized":"2023-07-21T18:40:07.035838+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:14:40.371518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.089901+0000","last_change":"2023-07-21T18:40:04.787544+0000","last_active":"2023-07-21T18:40:06.089901+0000","last_peered":"2023-07-21T18:40:06.089901+0000","last_clean":"2023-07-21T18:40:06.089901+0000","last_became_active":"2023-07-21T18:40:04.787208+0000","last_became_peered":"2023-07-21T18:40:04.787208+0000","last_unstale":"2023-07-21T18:40:06.089901+0000","last_undegraded":"2023-07-21T18:40:06.089901+0000","last_fullsized":"2023-07-21T18:40:06.089901+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:40:52.809501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.035809+0000","last_change":"2023-07-21T18:40:04.787964+0000","last_active":"2023-07-21T18:40:07.035809+0000","last_peered":"2023-07-21T18:40:07.035809+0000","last_clean":"2023-07-21T18:40:07.035809+0000","last_became_active":"2023-07-21T18:40:04.787518+0000","last_became_peered":"2023-07-21T18:40:04.787518+0000","last_unstale":"2023-07-21T18:40:07.035809+0000","last_undegraded":"2023-07-21T18:40:07.035809+0000","last_fullsized":"2023-07-21T18:40:07.035809+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:40:44.640274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.777237+0000","last_change":"2023-07-21T18:40:04.782495+0000","last_active":"2023-07-21T18:40:05.777237+0000","last_peered":"2023-07-21T18:40:05.777237+0000","last_clean":"2023-07-21T18:40:05.777237+0000","last_became_active":"2023-07-21T18:40:04.782208+0000","last_became_peered":"2023-07-21T18:40:04.782208+0000","last_unstale":"2023-07-21T18:40:05.777237+0000","last_undegraded":"2023-07-21T18:40:05.777237+0000","last_fullsized":"2023-07-21T18:40:05.777237+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:42:34.649015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.036247+0000","last_change":"2023-07-21T18:40:04.790957+0000","last_active":"2023-07-21T18:40:07.036247+0000","last_peered":"2023-07-21T18:40:07.036247+0000","last_clean":"2023-07-21T18:40:07.036247+0000","last_became_active":"2023-07-21T18:40:04.790659+0000","last_became_peered":"2023-07-21T18:40:04.790659+0000","last_unstale":"2023-07-21T18:40:07.036247+0000","last_undegraded":"2023-07-21T18:40:07.036247+0000","last_fullsized":"2023-07-21T18:40:07.036247+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:06:45.518722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.090285+0000","last_change":"2023-07-21T18:40:04.789415+0000","last_active":"2023-07-21T18:40:06.090285+0000","last_peered":"2023-07-21T18:40:06.090285+0000","last_clean":"2023-07-21T18:40:06.090285+0000","last_became_active":"2023-07-21T18:40:04.789088+0000","last_became_peered":"2023-07-21T18:40:04.789088+0000","last_unstale":"2023-07-21T18:40:06.090285+0000","last_undegraded":"2023-07-21T18:40:06.090285+0000","last_fullsized":"2023-07-21T18:40:06.090285+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:10:43.584203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.035583+0000","last_change":"2023-07-21T18:40:04.789101+0000","last_active":"2023-07-21T18:40:07.035583+0000","last_peered":"2023-07-21T18:40:07.035583+0000","last_clean":"2023-07-21T18:40:07.035583+0000","last_became_active":"2023-07-21T18:40:04.788704+0000","last_became_peered":"2023-07-21T18:40:04.788704+0000","last_unstale":"2023-07-21T18:40:07.035583+0000","last_undegraded":"2023-07-21T18:40:07.035583+0000","last_fullsized":"2023-07-21T18:40:07.035583+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:10:15.256725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.772732+0000","last_change":"2023-07-21T18:40:04.784863+0000","last_active":"2023-07-21T18:40:05.772732+0000","last_peered":"2023-07-21T18:40:05.772732+0000","last_clean":"2023-07-21T18:40:05.772732+0000","last_became_active":"2023-07-21T18:40:04.784474+0000","last_became_peered":"2023-07-21T18:40:04.784474+0000","last_unstale":"2023-07-21T18:40:05.772732+0000","last_undegraded":"2023-07-21T18:40:05.772732+0000","last_fullsized":"2023-07-21T18:40:05.772732+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:53:41.983453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.036146+0000","last_change":"2023-07-21T18:40:04.797678+0000","last_active":"2023-07-21T18:40:07.036146+0000","last_peered":"2023-07-21T18:40:07.036146+0000","last_clean":"2023-07-21T18:40:07.036146+0000","last_became_active":"2023-07-21T18:40:04.797318+0000","last_became_peered":"2023-07-21T18:40:04.797318+0000","last_unstale":"2023-07-21T18:40:07.036146+0000","last_undegraded":"2023-07-21T18:40:07.036146+0000","last_fullsized":"2023-07-21T18:40:07.036146+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:44:17.374089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.035540+0000","last_change":"2023-07-21T18:40:04.793170+0000","last_active":"2023-07-21T18:40:07.035540+0000","last_peered":"2023-07-21T18:40:07.035540+0000","last_clean":"2023-07-21T18:40:07.035540+0000","last_became_active":"2023-07-21T18:40:04.792563+0000","last_became_peered":"2023-07-21T18:40:04.792563+0000","last_unstale":"2023-07-21T18:40:07.035540+0000","last_undegraded":"2023-07-21T18:40:07.035540+0000","last_fullsized":"2023-07-21T18:40:07.035540+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:36:53.352643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.776683+0000","last_change":"2023-07-21T18:40:04.791972+0000","last_active":"2023-07-21T18:40:05.776683+0000","last_peered":"2023-07-21T18:40:05.776683+0000","last_clean":"2023-07-21T18:40:05.776683+0000","last_became_active":"2023-07-21T18:40:04.791639+0000","last_became_peered":"2023-07-21T18:40:04.791639+0000","last_unstale":"2023-07-21T18:40:05.776683+0000","last_undegraded":"2023-07-21T18:40:05.776683+0000","last_fullsized":"2023-07-21T18:40:05.776683+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:35:06.204290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.773432+0000","last_change":"2023-07-21T18:40:04.789362+0000","last_active":"2023-07-21T18:40:05.773432+0000","last_peered":"2023-07-21T18:40:05.773432+0000","last_clean":"2023-07-21T18:40:05.773432+0000","last_became_active":"2023-07-21T18:40:04.788978+0000","last_became_peered":"2023-07-21T18:40:04.788978+0000","last_unstale":"2023-07-21T18:40:05.773432+0000","last_undegraded":"2023-07-21T18:40:05.773432+0000","last_fullsized":"2023-07-21T18:40:05.773432+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:55:04.759568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.036113+0000","last_change":"2023-07-21T18:40:04.788016+0000","last_active":"2023-07-21T18:40:07.036113+0000","last_peered":"2023-07-21T18:40:07.036113+0000","last_clean":"2023-07-21T18:40:07.036113+0000","last_became_active":"2023-07-21T18:40:04.787690+0000","last_became_peered":"2023-07-21T18:40:04.787690+0000","last_unstale":"2023-07-21T18:40:07.036113+0000","last_undegraded":"2023-07-21T18:40:07.036113+0000","last_fullsized":"2023-07-21T18:40:07.036113+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:55:35.585172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.592595+0000","last_change":"2023-07-21T18:40:04.793075+0000","last_active":"2023-07-21T18:40:06.592595+0000","last_peered":"2023-07-21T18:40:06.592595+0000","last_clean":"2023-07-21T18:40:06.592595+0000","last_became_active":"2023-07-21T18:40:04.792773+0000","last_became_peered":"2023-07-21T18:40:04.792773+0000","last_unstale":"2023-07-21T18:40:06.592595+0000","last_undegraded":"2023-07-21T18:40:06.592595+0000","last_fullsized":"2023-07-21T18:40:06.592595+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:48:23.084376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"79'13","reported_seq":69,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:51.055126+0000","last_change":"2023-07-21T18:40:04.178197+0000","last_active":"2023-07-21T18:40:51.055126+0000","last_peered":"2023-07-21T18:40:51.055126+0000","last_clean":"2023-07-21T18:40:51.055126+0000","last_became_active":"2023-07-21T18:40:04.177914+0000","last_became_peered":"2023-07-21T18:40:04.177914+0000","last_unstale":"2023-07-21T18:40:51.055126+0000","last_undegraded":"2023-07-21T18:40:51.055126+0000","last_fullsized":"2023-07-21T18:40:51.055126+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:33:10.544496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":42,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.037188+0000","last_change":"2023-07-21T18:40:04.791610+0000","last_active":"2023-07-21T18:40:07.037188+0000","last_peered":"2023-07-21T18:40:07.037188+0000","last_clean":"2023-07-21T18:40:07.037188+0000","last_became_active":"2023-07-21T18:40:04.791242+0000","last_became_peered":"2023-07-21T18:40:04.791242+0000","last_unstale":"2023-07-21T18:40:07.037188+0000","last_undegraded":"2023-07-21T18:40:07.037188+0000","last_fullsized":"2023-07-21T18:40:07.037188+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:02:43.400899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.037064+0000","last_change":"2023-07-21T18:40:04.182120+0000","last_active":"2023-07-21T18:40:07.037064+0000","last_peered":"2023-07-21T18:40:07.037064+0000","last_clean":"2023-07-21T18:40:07.037064+0000","last_became_active":"2023-07-21T18:40:04.181797+0000","last_became_peered":"2023-07-21T18:40:04.181797+0000","last_unstale":"2023-07-21T18:40:07.037064+0000","last_undegraded":"2023-07-21T18:40:07.037064+0000","last_fullsized":"2023-07-21T18:40:07.037064+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:02:06.127933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.036297+0000","last_change":"2023-07-21T18:40:04.796649+0000","last_active":"2023-07-21T18:40:07.036297+0000","last_peered":"2023-07-21T18:40:07.036297+0000","last_clean":"2023-07-21T18:40:07.036297+0000","last_became_active":"2023-07-21T18:40:04.795610+0000","last_became_peered":"2023-07-21T18:40:04.795610+0000","last_unstale":"2023-07-21T18:40:07.036297+0000","last_undegraded":"2023-07-21T18:40:07.036297+0000","last_fullsized":"2023-07-21T18:40:07.036297+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:11:11.969964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"79'12","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:10.975720+0000","last_change":"2023-07-21T18:40:04.178384+0000","last_active":"2023-07-21T18:40:10.975720+0000","last_peered":"2023-07-21T18:40:10.975720+0000","last_clean":"2023-07-21T18:40:10.975720+0000","last_became_active":"2023-07-21T18:40:04.178074+0000","last_became_peered":"2023-07-21T18:40:04.178074+0000","last_unstale":"2023-07-21T18:40:10.975720+0000","last_undegraded":"2023-07-21T18:40:10.975720+0000","last_fullsized":"2023-07-21T18:40:10.975720+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:21:02.336313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.776990+0000","last_change":"2023-07-21T18:40:04.785431+0000","last_active":"2023-07-21T18:40:05.776990+0000","last_peered":"2023-07-21T18:40:05.776990+0000","last_clean":"2023-07-21T18:40:05.776990+0000","last_became_active":"2023-07-21T18:40:04.785124+0000","last_became_peered":"2023-07-21T18:40:04.785124+0000","last_unstale":"2023-07-21T18:40:05.776990+0000","last_undegraded":"2023-07-21T18:40:05.776990+0000","last_fullsized":"2023-07-21T18:40:05.776990+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:19:44.604006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"79'5","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:10.975999+0000","last_change":"2023-07-21T18:40:04.176349+0000","last_active":"2023-07-21T18:40:10.975999+0000","last_peered":"2023-07-21T18:40:10.975999+0000","last_clean":"2023-07-21T18:40:10.975999+0000","last_became_active":"2023-07-21T18:40:04.175997+0000","last_became_peered":"2023-07-21T18:40:04.175997+0000","last_unstale":"2023-07-21T18:40:10.975999+0000","last_undegraded":"2023-07-21T18:40:10.975999+0000","last_fullsized":"2023-07-21T18:40:10.975999+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:13:57.068743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.995402+0000","last_change":"2023-07-21T18:40:04.784867+0000","last_active":"2023-07-21T18:40:06.995402+0000","last_peered":"2023-07-21T18:40:06.995402+0000","last_clean":"2023-07-21T18:40:06.995402+0000","last_became_active":"2023-07-21T18:40:04.784555+0000","last_became_peered":"2023-07-21T18:40:04.784555+0000","last_unstale":"2023-07-21T18:40:06.995402+0000","last_undegraded":"2023-07-21T18:40:06.995402+0000","last_fullsized":"2023-07-21T18:40:06.995402+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:57:13.354337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"79'5","reported_seq":62,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:50.026241+0000","last_change":"2023-07-21T18:40:04.177638+0000","last_active":"2023-07-21T18:40:50.026241+0000","last_peered":"2023-07-21T18:40:50.026241+0000","last_clean":"2023-07-21T18:40:50.026241+0000","last_became_active":"2023-07-21T18:40:04.177284+0000","last_became_peered":"2023-07-21T18:40:04.177284+0000","last_unstale":"2023-07-21T18:40:50.026241+0000","last_undegraded":"2023-07-21T18:40:50.026241+0000","last_fullsized":"2023-07-21T18:40:50.026241+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:21:19.903720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":43,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.006906+0000","last_change":"2023-07-21T18:40:04.791536+0000","last_active":"2023-07-21T18:40:06.006906+0000","last_peered":"2023-07-21T18:40:06.006906+0000","last_clean":"2023-07-21T18:40:06.006906+0000","last_became_active":"2023-07-21T18:40:04.791246+0000","last_became_peered":"2023-07-21T18:40:04.791246+0000","last_unstale":"2023-07-21T18:40:06.006906+0000","last_undegraded":"2023-07-21T18:40:06.006906+0000","last_fullsized":"2023-07-21T18:40:06.006906+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:44:08.906635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"79'7","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:10.271306+0000","last_change":"2023-07-21T18:40:04.179054+0000","last_active":"2023-07-21T18:40:10.271306+0000","last_peered":"2023-07-21T18:40:10.271306+0000","last_clean":"2023-07-21T18:40:10.271306+0000","last_became_active":"2023-07-21T18:40:04.178713+0000","last_became_peered":"2023-07-21T18:40:04.178713+0000","last_unstale":"2023-07-21T18:40:10.271306+0000","last_undegraded":"2023-07-21T18:40:10.271306+0000","last_fullsized":"2023-07-21T18:40:10.271306+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T18:45:12.898734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.995525+0000","last_change":"2023-07-21T18:40:04.793947+0000","last_active":"2023-07-21T18:40:06.995525+0000","last_peered":"2023-07-21T18:40:06.995525+0000","last_clean":"2023-07-21T18:40:06.995525+0000","last_became_active":"2023-07-21T18:40:04.793631+0000","last_became_peered":"2023-07-21T18:40:04.793631+0000","last_unstale":"2023-07-21T18:40:06.995525+0000","last_undegraded":"2023-07-21T18:40:06.995525+0000","last_fullsized":"2023-07-21T18:40:06.995525+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:38:22.934266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"79'3","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:09.232592+0000","last_change":"2023-07-21T18:40:04.182549+0000","last_active":"2023-07-21T18:40:09.232592+0000","last_peered":"2023-07-21T18:40:09.232592+0000","last_clean":"2023-07-21T18:40:09.232592+0000","last_became_active":"2023-07-21T18:40:04.182233+0000","last_became_peered":"2023-07-21T18:40:04.182233+0000","last_unstale":"2023-07-21T18:40:09.232592+0000","last_undegraded":"2023-07-21T18:40:09.232592+0000","last_fullsized":"2023-07-21T18:40:09.232592+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:47:27.131280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.036098+0000","last_change":"2023-07-21T18:40:04.789987+0000","last_active":"2023-07-21T18:40:07.036098+0000","last_peered":"2023-07-21T18:40:07.036098+0000","last_clean":"2023-07-21T18:40:07.036098+0000","last_became_active":"2023-07-21T18:40:04.789647+0000","last_became_peered":"2023-07-21T18:40:04.789647+0000","last_unstale":"2023-07-21T18:40:07.036098+0000","last_undegraded":"2023-07-21T18:40:07.036098+0000","last_fullsized":"2023-07-21T18:40:07.036098+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:38:39.403755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'3","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:10.965460+0000","last_change":"2023-07-21T18:40:04.177342+0000","last_active":"2023-07-21T18:40:10.965460+0000","last_peered":"2023-07-21T18:40:10.965460+0000","last_clean":"2023-07-21T18:40:10.965460+0000","last_became_active":"2023-07-21T18:40:04.176983+0000","last_became_peered":"2023-07-21T18:40:04.176983+0000","last_unstale":"2023-07-21T18:40:10.965460+0000","last_undegraded":"2023-07-21T18:40:10.965460+0000","last_fullsized":"2023-07-21T18:40:10.965460+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:51:36.023901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.035932+0000","last_change":"2023-07-21T18:40:04.794907+0000","last_active":"2023-07-21T18:40:07.035932+0000","last_peered":"2023-07-21T18:40:07.035932+0000","last_clean":"2023-07-21T18:40:07.035932+0000","last_became_active":"2023-07-21T18:40:04.794545+0000","last_became_peered":"2023-07-21T18:40:04.794545+0000","last_unstale":"2023-07-21T18:40:07.035932+0000","last_undegraded":"2023-07-21T18:40:07.035932+0000","last_fullsized":"2023-07-21T18:40:07.035932+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:47:34.101102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"79'4","reported_seq":94,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:48.022390+0000","last_change":"2023-07-21T18:40:04.179198+0000","last_active":"2023-07-21T18:40:48.022390+0000","last_peered":"2023-07-21T18:40:48.022390+0000","last_clean":"2023-07-21T18:40:48.022390+0000","last_became_active":"2023-07-21T18:40:04.178841+0000","last_became_peered":"2023-07-21T18:40:04.178841+0000","last_unstale":"2023-07-21T18:40:48.022390+0000","last_undegraded":"2023-07-21T18:40:48.022390+0000","last_fullsized":"2023-07-21T18:40:48.022390+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:12:24.650360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":76,"num_read_kb":76,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.777142+0000","last_change":"2023-07-21T18:40:04.786229+0000","last_active":"2023-07-21T18:40:05.777142+0000","last_peered":"2023-07-21T18:40:05.777142+0000","last_clean":"2023-07-21T18:40:05.777142+0000","last_became_active":"2023-07-21T18:40:04.785919+0000","last_became_peered":"2023-07-21T18:40:04.785919+0000","last_unstale":"2023-07-21T18:40:05.777142+0000","last_undegraded":"2023-07-21T18:40:05.777142+0000","last_fullsized":"2023-07-21T18:40:05.777142+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:19:04.949730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'4","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:10.974889+0000","last_change":"2023-07-21T18:40:04.177164+0000","last_active":"2023-07-21T18:40:10.974889+0000","last_peered":"2023-07-21T18:40:10.974889+0000","last_clean":"2023-07-21T18:40:10.974889+0000","last_became_active":"2023-07-21T18:40:04.176617+0000","last_became_peered":"2023-07-21T18:40:04.176617+0000","last_unstale":"2023-07-21T18:40:10.974889+0000","last_undegraded":"2023-07-21T18:40:10.974889+0000","last_fullsized":"2023-07-21T18:40:10.974889+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:38:06.655399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.036335+0000","last_change":"2023-07-21T18:40:04.787863+0000","last_active":"2023-07-21T18:40:07.036335+0000","last_peered":"2023-07-21T18:40:07.036335+0000","last_clean":"2023-07-21T18:40:07.036335+0000","last_became_active":"2023-07-21T18:40:04.787562+0000","last_became_peered":"2023-07-21T18:40:04.787562+0000","last_unstale":"2023-07-21T18:40:07.036335+0000","last_undegraded":"2023-07-21T18:40:07.036335+0000","last_fullsized":"2023-07-21T18:40:07.036335+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:30:30.392804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'5","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:10.244099+0000","last_change":"2023-07-21T18:40:04.179921+0000","last_active":"2023-07-21T18:40:10.244099+0000","last_peered":"2023-07-21T18:40:10.244099+0000","last_clean":"2023-07-21T18:40:10.244099+0000","last_became_active":"2023-07-21T18:40:04.179670+0000","last_became_peered":"2023-07-21T18:40:04.179670+0000","last_unstale":"2023-07-21T18:40:10.244099+0000","last_undegraded":"2023-07-21T18:40:10.244099+0000","last_fullsized":"2023-07-21T18:40:10.244099+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:11:15.619102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.777008+0000","last_change":"2023-07-21T18:40:04.792930+0000","last_active":"2023-07-21T18:40:05.777008+0000","last_peered":"2023-07-21T18:40:05.777008+0000","last_clean":"2023-07-21T18:40:05.777008+0000","last_became_active":"2023-07-21T18:40:04.792654+0000","last_became_peered":"2023-07-21T18:40:04.792654+0000","last_unstale":"2023-07-21T18:40:05.777008+0000","last_undegraded":"2023-07-21T18:40:05.777008+0000","last_fullsized":"2023-07-21T18:40:05.777008+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:50:21.387154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"79'28","reported_seq":124,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:48.026676+0000","last_change":"2023-07-21T18:40:04.178820+0000","last_active":"2023-07-21T18:40:48.026676+0000","last_peered":"2023-07-21T18:40:48.026676+0000","last_clean":"2023-07-21T18:40:48.026676+0000","last_became_active":"2023-07-21T18:40:04.178502+0000","last_became_peered":"2023-07-21T18:40:04.178502+0000","last_unstale":"2023-07-21T18:40:48.026676+0000","last_undegraded":"2023-07-21T18:40:48.026676+0000","last_fullsized":"2023-07-21T18:40:48.026676+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:48:50.193456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56875,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":82,"num_read_kb":96,"num_write":30,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.006957+0000","last_change":"2023-07-21T18:40:04.792921+0000","last_active":"2023-07-21T18:40:06.006957+0000","last_peered":"2023-07-21T18:40:06.006957+0000","last_clean":"2023-07-21T18:40:06.006957+0000","last_became_active":"2023-07-21T18:40:04.792608+0000","last_became_peered":"2023-07-21T18:40:04.792608+0000","last_unstale":"2023-07-21T18:40:06.006957+0000","last_undegraded":"2023-07-21T18:40:06.006957+0000","last_fullsized":"2023-07-21T18:40:06.006957+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:04:23.180977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'9","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:10.975237+0000","last_change":"2023-07-21T18:40:04.184917+0000","last_active":"2023-07-21T18:40:10.975237+0000","last_peered":"2023-07-21T18:40:10.975237+0000","last_clean":"2023-07-21T18:40:10.975237+0000","last_became_active":"2023-07-21T18:40:04.184612+0000","last_became_peered":"2023-07-21T18:40:04.184612+0000","last_unstale":"2023-07-21T18:40:10.975237+0000","last_undegraded":"2023-07-21T18:40:10.975237+0000","last_fullsized":"2023-07-21T18:40:10.975237+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:54:45.871657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.772576+0000","last_change":"2023-07-21T18:40:04.794134+0000","last_active":"2023-07-21T18:40:05.772576+0000","last_peered":"2023-07-21T18:40:05.772576+0000","last_clean":"2023-07-21T18:40:05.772576+0000","last_became_active":"2023-07-21T18:40:04.793784+0000","last_became_peered":"2023-07-21T18:40:04.793784+0000","last_unstale":"2023-07-21T18:40:05.772576+0000","last_undegraded":"2023-07-21T18:40:05.772576+0000","last_fullsized":"2023-07-21T18:40:05.772576+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:24:12.380794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'5","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:10.975199+0000","last_change":"2023-07-21T18:40:04.184889+0000","last_active":"2023-07-21T18:40:10.975199+0000","last_peered":"2023-07-21T18:40:10.975199+0000","last_clean":"2023-07-21T18:40:10.975199+0000","last_became_active":"2023-07-21T18:40:04.184516+0000","last_became_peered":"2023-07-21T18:40:04.184516+0000","last_unstale":"2023-07-21T18:40:10.975199+0000","last_undegraded":"2023-07-21T18:40:10.975199+0000","last_fullsized":"2023-07-21T18:40:10.975199+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:56:48.211364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.090064+0000","last_change":"2023-07-21T18:40:04.790627+0000","last_active":"2023-07-21T18:40:06.090064+0000","last_peered":"2023-07-21T18:40:06.090064+0000","last_clean":"2023-07-21T18:40:06.090064+0000","last_became_active":"2023-07-21T18:40:04.790298+0000","last_became_peered":"2023-07-21T18:40:04.790298+0000","last_unstale":"2023-07-21T18:40:06.090064+0000","last_undegraded":"2023-07-21T18:40:06.090064+0000","last_fullsized":"2023-07-21T18:40:06.090064+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:23:15.541330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"79'6","reported_seq":64,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:51.026943+0000","last_change":"2023-07-21T18:40:04.176156+0000","last_active":"2023-07-21T18:40:51.026943+0000","last_peered":"2023-07-21T18:40:51.026943+0000","last_clean":"2023-07-21T18:40:51.026943+0000","last_became_active":"2023-07-21T18:40:04.175823+0000","last_became_peered":"2023-07-21T18:40:04.175823+0000","last_unstale":"2023-07-21T18:40:51.026943+0000","last_undegraded":"2023-07-21T18:40:51.026943+0000","last_fullsized":"2023-07-21T18:40:51.026943+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:43:06.713614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":930,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":45,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.089814+0000","last_change":"2023-07-21T18:40:04.790422+0000","last_active":"2023-07-21T18:40:06.089814+0000","last_peered":"2023-07-21T18:40:06.089814+0000","last_clean":"2023-07-21T18:40:06.089814+0000","last_became_active":"2023-07-21T18:40:04.790124+0000","last_became_peered":"2023-07-21T18:40:04.790124+0000","last_unstale":"2023-07-21T18:40:06.089814+0000","last_undegraded":"2023-07-21T18:40:06.089814+0000","last_fullsized":"2023-07-21T18:40:06.089814+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:50:38.399075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'17","reported_seq":32,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:10.962031+0000","last_change":"2023-07-21T18:40:04.177006+0000","last_active":"2023-07-21T18:40:10.962031+0000","last_peered":"2023-07-21T18:40:10.962031+0000","last_clean":"2023-07-21T18:40:10.962031+0000","last_became_active":"2023-07-21T18:40:04.176711+0000","last_became_peered":"2023-07-21T18:40:04.176711+0000","last_unstale":"2023-07-21T18:40:10.962031+0000","last_undegraded":"2023-07-21T18:40:10.962031+0000","last_fullsized":"2023-07-21T18:40:10.962031+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:51:47.735726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.995106+0000","last_change":"2023-07-21T18:40:04.785601+0000","last_active":"2023-07-21T18:40:06.995106+0000","last_peered":"2023-07-21T18:40:06.995106+0000","last_clean":"2023-07-21T18:40:06.995106+0000","last_became_active":"2023-07-21T18:40:04.785307+0000","last_became_peered":"2023-07-21T18:40:04.785307+0000","last_unstale":"2023-07-21T18:40:06.995106+0000","last_undegraded":"2023-07-21T18:40:06.995106+0000","last_fullsized":"2023-07-21T18:40:06.995106+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:17:07.221060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:10.962183+0000","last_change":"2023-07-21T18:40:04.171005+0000","last_active":"2023-07-21T18:40:10.962183+0000","last_peered":"2023-07-21T18:40:10.962183+0000","last_clean":"2023-07-21T18:40:10.962183+0000","last_became_active":"2023-07-21T18:40:04.170719+0000","last_became_peered":"2023-07-21T18:40:04.170719+0000","last_unstale":"2023-07-21T18:40:10.962183+0000","last_undegraded":"2023-07-21T18:40:10.962183+0000","last_fullsized":"2023-07-21T18:40:10.962183+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:06:46.353789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.773304+0000","last_change":"2023-07-21T18:40:04.783596+0000","last_active":"2023-07-21T18:40:05.773304+0000","last_peered":"2023-07-21T18:40:05.773304+0000","last_clean":"2023-07-21T18:40:05.773304+0000","last_became_active":"2023-07-21T18:40:04.783293+0000","last_became_peered":"2023-07-21T18:40:04.783293+0000","last_unstale":"2023-07-21T18:40:05.773304+0000","last_undegraded":"2023-07-21T18:40:05.773304+0000","last_fullsized":"2023-07-21T18:40:05.773304+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:17:45.360634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"79'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.648456+0000","last_change":"2023-07-21T18:40:04.181069+0000","last_active":"2023-07-21T18:40:07.648456+0000","last_peered":"2023-07-21T18:40:07.648456+0000","last_clean":"2023-07-21T18:40:07.648456+0000","last_became_active":"2023-07-21T18:40:04.180750+0000","last_became_peered":"2023-07-21T18:40:04.180750+0000","last_unstale":"2023-07-21T18:40:07.648456+0000","last_undegraded":"2023-07-21T18:40:07.648456+0000","last_fullsized":"2023-07-21T18:40:07.648456+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:44:48.158024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"79'1","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:08.919470+0000","last_change":"2023-07-21T18:40:04.789940+0000","last_active":"2023-07-21T18:40:08.919470+0000","last_peered":"2023-07-21T18:40:08.919470+0000","last_clean":"2023-07-21T18:40:08.919470+0000","last_became_active":"2023-07-21T18:40:04.789465+0000","last_became_peered":"2023-07-21T18:40:04.789465+0000","last_unstale":"2023-07-21T18:40:08.919470+0000","last_undegraded":"2023-07-21T18:40:08.919470+0000","last_fullsized":"2023-07-21T18:40:08.919470+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:05:44.817727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"79'5","reported_seq":59,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:49.053040+0000","last_change":"2023-07-21T18:40:04.180522+0000","last_active":"2023-07-21T18:40:49.053040+0000","last_peered":"2023-07-21T18:40:49.053040+0000","last_clean":"2023-07-21T18:40:49.053040+0000","last_became_active":"2023-07-21T18:40:04.180196+0000","last_became_peered":"2023-07-21T18:40:04.180196+0000","last_unstale":"2023-07-21T18:40:49.053040+0000","last_undegraded":"2023-07-21T18:40:49.053040+0000","last_fullsized":"2023-07-21T18:40:49.053040+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:10:53.779418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":40,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.036035+0000","last_change":"2023-07-21T18:40:04.794781+0000","last_active":"2023-07-21T18:40:07.036035+0000","last_peered":"2023-07-21T18:40:07.036035+0000","last_clean":"2023-07-21T18:40:07.036035+0000","last_became_active":"2023-07-21T18:40:04.794388+0000","last_became_peered":"2023-07-21T18:40:04.794388+0000","last_unstale":"2023-07-21T18:40:07.036035+0000","last_undegraded":"2023-07-21T18:40:07.036035+0000","last_fullsized":"2023-07-21T18:40:07.036035+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:26:47.679253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.995411+0000","last_change":"2023-07-21T18:40:04.786408+0000","last_active":"2023-07-21T18:40:06.995411+0000","last_peered":"2023-07-21T18:40:06.995411+0000","last_clean":"2023-07-21T18:40:06.995411+0000","last_became_active":"2023-07-21T18:40:04.785555+0000","last_became_peered":"2023-07-21T18:40:04.785555+0000","last_unstale":"2023-07-21T18:40:06.995411+0000","last_undegraded":"2023-07-21T18:40:06.995411+0000","last_fullsized":"2023-07-21T18:40:06.995411+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:32:35.090936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"79'5","reported_seq":58,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:47.020157+0000","last_change":"2023-07-21T18:40:04.185980+0000","last_active":"2023-07-21T18:40:47.020157+0000","last_peered":"2023-07-21T18:40:47.020157+0000","last_clean":"2023-07-21T18:40:47.020157+0000","last_became_active":"2023-07-21T18:40:04.185618+0000","last_became_peered":"2023-07-21T18:40:04.185618+0000","last_unstale":"2023-07-21T18:40:47.020157+0000","last_undegraded":"2023-07-21T18:40:47.020157+0000","last_fullsized":"2023-07-21T18:40:47.020157+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:44:22.202995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":39,"num_read_kb":39,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.036366+0000","last_change":"2023-07-21T18:40:04.792535+0000","last_active":"2023-07-21T18:40:07.036366+0000","last_peered":"2023-07-21T18:40:07.036366+0000","last_clean":"2023-07-21T18:40:07.036366+0000","last_became_active":"2023-07-21T18:40:04.792239+0000","last_became_peered":"2023-07-21T18:40:04.792239+0000","last_unstale":"2023-07-21T18:40:07.036366+0000","last_undegraded":"2023-07-21T18:40:07.036366+0000","last_fullsized":"2023-07-21T18:40:07.036366+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:51:14.670073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"79'6","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:10.974758+0000","last_change":"2023-07-21T18:40:04.173539+0000","last_active":"2023-07-21T18:40:10.974758+0000","last_peered":"2023-07-21T18:40:10.974758+0000","last_clean":"2023-07-21T18:40:10.974758+0000","last_became_active":"2023-07-21T18:40:04.173225+0000","last_became_peered":"2023-07-21T18:40:04.173225+0000","last_unstale":"2023-07-21T18:40:10.974758+0000","last_undegraded":"2023-07-21T18:40:10.974758+0000","last_fullsized":"2023-07-21T18:40:10.974758+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:38:21.873200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":337,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.089887+0000","last_change":"2023-07-21T18:39:05.999983+0000","last_active":"2023-07-21T18:40:06.089887+0000","last_peered":"2023-07-21T18:40:06.089887+0000","last_clean":"2023-07-21T18:40:06.089887+0000","last_became_active":"2023-07-21T18:39:05.999495+0000","last_became_peered":"2023-07-21T18:39:05.999495+0000","last_unstale":"2023-07-21T18:40:06.089887+0000","last_undegraded":"2023-07-21T18:40:06.089887+0000","last_fullsized":"2023-07-21T18:40:06.089887+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:36:14.004429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:36:14.004429+0000","last_clean_scrub_stamp":"2023-07-21T18:36:14.004429+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:34:11.106296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.772327+0000","last_change":"2023-07-21T18:40:04.785086+0000","last_active":"2023-07-21T18:40:05.772327+0000","last_peered":"2023-07-21T18:40:05.772327+0000","last_clean":"2023-07-21T18:40:05.772327+0000","last_became_active":"2023-07-21T18:40:04.784693+0000","last_became_peered":"2023-07-21T18:40:04.784693+0000","last_unstale":"2023-07-21T18:40:05.772327+0000","last_undegraded":"2023-07-21T18:40:05.772327+0000","last_fullsized":"2023-07-21T18:40:05.772327+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:53:45.159424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"79'4","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:09.230935+0000","last_change":"2023-07-21T18:40:04.177032+0000","last_active":"2023-07-21T18:40:09.230935+0000","last_peered":"2023-07-21T18:40:09.230935+0000","last_clean":"2023-07-21T18:40:09.230935+0000","last_became_active":"2023-07-21T18:40:04.176741+0000","last_became_peered":"2023-07-21T18:40:04.176741+0000","last_unstale":"2023-07-21T18:40:09.230935+0000","last_undegraded":"2023-07-21T18:40:09.230935+0000","last_fullsized":"2023-07-21T18:40:09.230935+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:17:23.675128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"79'4","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:10.962237+0000","last_change":"2023-07-21T18:40:04.180106+0000","last_active":"2023-07-21T18:40:10.962237+0000","last_peered":"2023-07-21T18:40:10.962237+0000","last_clean":"2023-07-21T18:40:10.962237+0000","last_became_active":"2023-07-21T18:40:04.179802+0000","last_became_peered":"2023-07-21T18:40:04.179802+0000","last_unstale":"2023-07-21T18:40:10.962237+0000","last_undegraded":"2023-07-21T18:40:10.962237+0000","last_fullsized":"2023-07-21T18:40:10.962237+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:38:16.465170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.772506+0000","last_change":"2023-07-21T18:40:04.789127+0000","last_active":"2023-07-21T18:40:05.772506+0000","last_peered":"2023-07-21T18:40:05.772506+0000","last_clean":"2023-07-21T18:40:05.772506+0000","last_became_active":"2023-07-21T18:40:04.788815+0000","last_became_peered":"2023-07-21T18:40:04.788815+0000","last_unstale":"2023-07-21T18:40:05.772506+0000","last_undegraded":"2023-07-21T18:40:05.772506+0000","last_fullsized":"2023-07-21T18:40:05.772506+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:14:00.509523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"79'9","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:13.313053+0000","last_change":"2023-07-21T18:40:04.184786+0000","last_active":"2023-07-21T18:40:13.313053+0000","last_peered":"2023-07-21T18:40:13.313053+0000","last_clean":"2023-07-21T18:40:13.313053+0000","last_became_active":"2023-07-21T18:40:04.184433+0000","last_became_peered":"2023-07-21T18:40:04.184433+0000","last_unstale":"2023-07-21T18:40:13.313053+0000","last_undegraded":"2023-07-21T18:40:13.313053+0000","last_fullsized":"2023-07-21T18:40:13.313053+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:39:47.800619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.037220+0000","last_change":"2023-07-21T18:40:04.791937+0000","last_active":"2023-07-21T18:40:07.037220+0000","last_peered":"2023-07-21T18:40:07.037220+0000","last_clean":"2023-07-21T18:40:07.037220+0000","last_became_active":"2023-07-21T18:40:04.791588+0000","last_became_peered":"2023-07-21T18:40:04.791588+0000","last_unstale":"2023-07-21T18:40:07.037220+0000","last_undegraded":"2023-07-21T18:40:07.037220+0000","last_fullsized":"2023-07-21T18:40:07.037220+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:33:01.443024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'7","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:10.973433+0000","last_change":"2023-07-21T18:40:04.177007+0000","last_active":"2023-07-21T18:40:10.973433+0000","last_peered":"2023-07-21T18:40:10.973433+0000","last_clean":"2023-07-21T18:40:10.973433+0000","last_became_active":"2023-07-21T18:40:04.176699+0000","last_became_peered":"2023-07-21T18:40:04.176699+0000","last_unstale":"2023-07-21T18:40:10.973433+0000","last_undegraded":"2023-07-21T18:40:10.973433+0000","last_fullsized":"2023-07-21T18:40:10.973433+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:21:08.441400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.035909+0000","last_change":"2023-07-21T18:40:04.795671+0000","last_active":"2023-07-21T18:40:07.035909+0000","last_peered":"2023-07-21T18:40:07.035909+0000","last_clean":"2023-07-21T18:40:07.035909+0000","last_became_active":"2023-07-21T18:40:04.795233+0000","last_became_peered":"2023-07-21T18:40:04.795233+0000","last_unstale":"2023-07-21T18:40:07.035909+0000","last_undegraded":"2023-07-21T18:40:07.035909+0000","last_fullsized":"2023-07-21T18:40:07.035909+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:26:32.284218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:08.212881+0000","last_change":"2023-07-21T18:40:04.176859+0000","last_active":"2023-07-21T18:40:08.212881+0000","last_peered":"2023-07-21T18:40:08.212881+0000","last_clean":"2023-07-21T18:40:08.212881+0000","last_became_active":"2023-07-21T18:40:04.176559+0000","last_became_peered":"2023-07-21T18:40:04.176559+0000","last_unstale":"2023-07-21T18:40:08.212881+0000","last_undegraded":"2023-07-21T18:40:08.212881+0000","last_fullsized":"2023-07-21T18:40:08.212881+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:34:23.667073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.006770+0000","last_change":"2023-07-21T18:40:04.793091+0000","last_active":"2023-07-21T18:40:06.006770+0000","last_peered":"2023-07-21T18:40:06.006770+0000","last_clean":"2023-07-21T18:40:06.006770+0000","last_became_active":"2023-07-21T18:40:04.792723+0000","last_became_peered":"2023-07-21T18:40:04.792723+0000","last_unstale":"2023-07-21T18:40:06.006770+0000","last_undegraded":"2023-07-21T18:40:06.006770+0000","last_fullsized":"2023-07-21T18:40:06.006770+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:27:04.440384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'7","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:10.974001+0000","last_change":"2023-07-21T18:40:04.182867+0000","last_active":"2023-07-21T18:40:10.974001+0000","last_peered":"2023-07-21T18:40:10.974001+0000","last_clean":"2023-07-21T18:40:10.974001+0000","last_became_active":"2023-07-21T18:40:04.181885+0000","last_became_peered":"2023-07-21T18:40:04.181885+0000","last_unstale":"2023-07-21T18:40:10.974001+0000","last_undegraded":"2023-07-21T18:40:10.974001+0000","last_fullsized":"2023-07-21T18:40:10.974001+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:03:28.800986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.773091+0000","last_change":"2023-07-21T18:40:04.790340+0000","last_active":"2023-07-21T18:40:05.773091+0000","last_peered":"2023-07-21T18:40:05.773091+0000","last_clean":"2023-07-21T18:40:05.773091+0000","last_became_active":"2023-07-21T18:40:04.790014+0000","last_became_peered":"2023-07-21T18:40:04.790014+0000","last_unstale":"2023-07-21T18:40:05.773091+0000","last_undegraded":"2023-07-21T18:40:05.773091+0000","last_fullsized":"2023-07-21T18:40:05.773091+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:40:26.891907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'3","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:10.243533+0000","last_change":"2023-07-21T18:40:04.184986+0000","last_active":"2023-07-21T18:40:10.243533+0000","last_peered":"2023-07-21T18:40:10.243533+0000","last_clean":"2023-07-21T18:40:10.243533+0000","last_became_active":"2023-07-21T18:40:04.184672+0000","last_became_peered":"2023-07-21T18:40:04.184672+0000","last_unstale":"2023-07-21T18:40:10.243533+0000","last_undegraded":"2023-07-21T18:40:10.243533+0000","last_fullsized":"2023-07-21T18:40:10.243533+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:40:47.955426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.036702+0000","last_change":"2023-07-21T18:40:04.793272+0000","last_active":"2023-07-21T18:40:07.036702+0000","last_peered":"2023-07-21T18:40:07.036702+0000","last_clean":"2023-07-21T18:40:07.036702+0000","last_became_active":"2023-07-21T18:40:04.792854+0000","last_became_peered":"2023-07-21T18:40:04.792854+0000","last_unstale":"2023-07-21T18:40:07.036702+0000","last_undegraded":"2023-07-21T18:40:07.036702+0000","last_fullsized":"2023-07-21T18:40:07.036702+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:22:05.555012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:08.212137+0000","last_change":"2023-07-21T18:40:04.176404+0000","last_active":"2023-07-21T18:40:08.212137+0000","last_peered":"2023-07-21T18:40:08.212137+0000","last_clean":"2023-07-21T18:40:08.212137+0000","last_became_active":"2023-07-21T18:40:04.176148+0000","last_became_peered":"2023-07-21T18:40:04.176148+0000","last_unstale":"2023-07-21T18:40:08.212137+0000","last_undegraded":"2023-07-21T18:40:08.212137+0000","last_fullsized":"2023-07-21T18:40:08.212137+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:29:41.873295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.090128+0000","last_change":"2023-07-21T18:40:04.795628+0000","last_active":"2023-07-21T18:40:06.090128+0000","last_peered":"2023-07-21T18:40:06.090128+0000","last_clean":"2023-07-21T18:40:06.090128+0000","last_became_active":"2023-07-21T18:40:04.795248+0000","last_became_peered":"2023-07-21T18:40:04.795248+0000","last_unstale":"2023-07-21T18:40:06.090128+0000","last_undegraded":"2023-07-21T18:40:06.090128+0000","last_fullsized":"2023-07-21T18:40:06.090128+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:05:26.124641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'5","reported_seq":59,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:47.021669+0000","last_change":"2023-07-21T18:40:04.184380+0000","last_active":"2023-07-21T18:40:47.021669+0000","last_peered":"2023-07-21T18:40:47.021669+0000","last_clean":"2023-07-21T18:40:47.021669+0000","last_became_active":"2023-07-21T18:40:04.184045+0000","last_became_peered":"2023-07-21T18:40:04.184045+0000","last_unstale":"2023-07-21T18:40:47.021669+0000","last_undegraded":"2023-07-21T18:40:47.021669+0000","last_fullsized":"2023-07-21T18:40:47.021669+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:58:31.782143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":40,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.090407+0000","last_change":"2023-07-21T18:40:04.787760+0000","last_active":"2023-07-21T18:40:06.090407+0000","last_peered":"2023-07-21T18:40:06.090407+0000","last_clean":"2023-07-21T18:40:06.090407+0000","last_became_active":"2023-07-21T18:40:04.787371+0000","last_became_peered":"2023-07-21T18:40:04.787371+0000","last_unstale":"2023-07-21T18:40:06.090407+0000","last_undegraded":"2023-07-21T18:40:06.090407+0000","last_fullsized":"2023-07-21T18:40:06.090407+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:10:11.576057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'3","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:10.962506+0000","last_change":"2023-07-21T18:40:04.184721+0000","last_active":"2023-07-21T18:40:10.962506+0000","last_peered":"2023-07-21T18:40:10.962506+0000","last_clean":"2023-07-21T18:40:10.962506+0000","last_became_active":"2023-07-21T18:40:04.184070+0000","last_became_peered":"2023-07-21T18:40:04.184070+0000","last_unstale":"2023-07-21T18:40:10.962506+0000","last_undegraded":"2023-07-21T18:40:10.962506+0000","last_fullsized":"2023-07-21T18:40:10.962506+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:57:46.774015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"79'1","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.090270+0000","last_change":"2023-07-21T18:40:04.786963+0000","last_active":"2023-07-21T18:40:06.090270+0000","last_peered":"2023-07-21T18:40:06.090270+0000","last_clean":"2023-07-21T18:40:06.090270+0000","last_became_active":"2023-07-21T18:40:04.786617+0000","last_became_peered":"2023-07-21T18:40:04.786617+0000","last_unstale":"2023-07-21T18:40:06.090270+0000","last_undegraded":"2023-07-21T18:40:06.090270+0000","last_fullsized":"2023-07-21T18:40:06.090270+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:22:45.940347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'5","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:10.975017+0000","last_change":"2023-07-21T18:40:04.184509+0000","last_active":"2023-07-21T18:40:10.975017+0000","last_peered":"2023-07-21T18:40:10.975017+0000","last_clean":"2023-07-21T18:40:10.975017+0000","last_became_active":"2023-07-21T18:40:04.184208+0000","last_became_peered":"2023-07-21T18:40:04.184208+0000","last_unstale":"2023-07-21T18:40:10.975017+0000","last_undegraded":"2023-07-21T18:40:10.975017+0000","last_fullsized":"2023-07-21T18:40:10.975017+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:18:05.113810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.090384+0000","last_change":"2023-07-21T18:40:04.788248+0000","last_active":"2023-07-21T18:40:06.090384+0000","last_peered":"2023-07-21T18:40:06.090384+0000","last_clean":"2023-07-21T18:40:06.090384+0000","last_became_active":"2023-07-21T18:40:04.787974+0000","last_became_peered":"2023-07-21T18:40:04.787974+0000","last_unstale":"2023-07-21T18:40:06.090384+0000","last_undegraded":"2023-07-21T18:40:06.090384+0000","last_fullsized":"2023-07-21T18:40:06.090384+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:58:18.957824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.773170+0000","last_change":"2023-07-21T18:40:04.790522+0000","last_active":"2023-07-21T18:40:05.773170+0000","last_peered":"2023-07-21T18:40:05.773170+0000","last_clean":"2023-07-21T18:40:05.773170+0000","last_became_active":"2023-07-21T18:40:04.790178+0000","last_became_peered":"2023-07-21T18:40:04.790178+0000","last_unstale":"2023-07-21T18:40:05.773170+0000","last_undegraded":"2023-07-21T18:40:05.773170+0000","last_fullsized":"2023-07-21T18:40:05.773170+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:39:58.451665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.592261+0000","last_change":"2023-07-21T18:40:04.790403+0000","last_active":"2023-07-21T18:40:06.592261+0000","last_peered":"2023-07-21T18:40:06.592261+0000","last_clean":"2023-07-21T18:40:06.592261+0000","last_became_active":"2023-07-21T18:40:04.790106+0000","last_became_peered":"2023-07-21T18:40:04.790106+0000","last_unstale":"2023-07-21T18:40:06.592261+0000","last_undegraded":"2023-07-21T18:40:06.592261+0000","last_fullsized":"2023-07-21T18:40:06.592261+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:22:57.358209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.592288+0000","last_change":"2023-07-21T18:40:04.795230+0000","last_active":"2023-07-21T18:40:06.592288+0000","last_peered":"2023-07-21T18:40:06.592288+0000","last_clean":"2023-07-21T18:40:06.592288+0000","last_became_active":"2023-07-21T18:40:04.794949+0000","last_became_peered":"2023-07-21T18:40:04.794949+0000","last_unstale":"2023-07-21T18:40:06.592288+0000","last_undegraded":"2023-07-21T18:40:06.592288+0000","last_fullsized":"2023-07-21T18:40:06.592288+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:25:52.872388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.037364+0000","last_change":"2023-07-21T18:40:04.792736+0000","last_active":"2023-07-21T18:40:07.037364+0000","last_peered":"2023-07-21T18:40:07.037364+0000","last_clean":"2023-07-21T18:40:07.037364+0000","last_became_active":"2023-07-21T18:40:04.792301+0000","last_became_peered":"2023-07-21T18:40:04.792301+0000","last_unstale":"2023-07-21T18:40:07.037364+0000","last_undegraded":"2023-07-21T18:40:07.037364+0000","last_fullsized":"2023-07-21T18:40:07.037364+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:40:20.170397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.772638+0000","last_change":"2023-07-21T18:40:04.790733+0000","last_active":"2023-07-21T18:40:05.772638+0000","last_peered":"2023-07-21T18:40:05.772638+0000","last_clean":"2023-07-21T18:40:05.772638+0000","last_became_active":"2023-07-21T18:40:04.790441+0000","last_became_peered":"2023-07-21T18:40:04.790441+0000","last_unstale":"2023-07-21T18:40:05.772638+0000","last_undegraded":"2023-07-21T18:40:05.772638+0000","last_fullsized":"2023-07-21T18:40:05.772638+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:29:55.466493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.773446+0000","last_change":"2023-07-21T18:40:04.790853+0000","last_active":"2023-07-21T18:40:05.773446+0000","last_peered":"2023-07-21T18:40:05.773446+0000","last_clean":"2023-07-21T18:40:05.773446+0000","last_became_active":"2023-07-21T18:40:04.790373+0000","last_became_peered":"2023-07-21T18:40:04.790373+0000","last_unstale":"2023-07-21T18:40:05.773446+0000","last_undegraded":"2023-07-21T18:40:05.773446+0000","last_fullsized":"2023-07-21T18:40:05.773446+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:59:25.687337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.592674+0000","last_change":"2023-07-21T18:40:04.794088+0000","last_active":"2023-07-21T18:40:06.592674+0000","last_peered":"2023-07-21T18:40:06.592674+0000","last_clean":"2023-07-21T18:40:06.592674+0000","last_became_active":"2023-07-21T18:40:04.793782+0000","last_became_peered":"2023-07-21T18:40:04.793782+0000","last_unstale":"2023-07-21T18:40:06.592674+0000","last_undegraded":"2023-07-21T18:40:06.592674+0000","last_fullsized":"2023-07-21T18:40:06.592674+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:14:52.604091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.037053+0000","last_change":"2023-07-21T18:40:04.791772+0000","last_active":"2023-07-21T18:40:07.037053+0000","last_peered":"2023-07-21T18:40:07.037053+0000","last_clean":"2023-07-21T18:40:07.037053+0000","last_became_active":"2023-07-21T18:40:04.791379+0000","last_became_peered":"2023-07-21T18:40:04.791379+0000","last_unstale":"2023-07-21T18:40:07.037053+0000","last_undegraded":"2023-07-21T18:40:07.037053+0000","last_fullsized":"2023-07-21T18:40:07.037053+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:37:36.712528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:06.994956+0000","last_change":"2023-07-21T18:40:04.786579+0000","last_active":"2023-07-21T18:40:06.994956+0000","last_peered":"2023-07-21T18:40:06.994956+0000","last_clean":"2023-07-21T18:40:06.994956+0000","last_became_active":"2023-07-21T18:40:04.786032+0000","last_became_peered":"2023-07-21T18:40:04.786032+0000","last_unstale":"2023-07-21T18:40:06.994956+0000","last_undegraded":"2023-07-21T18:40:06.994956+0000","last_fullsized":"2023-07-21T18:40:06.994956+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:34:12.940522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:07.036500+0000","last_change":"2023-07-21T18:40:04.793441+0000","last_active":"2023-07-21T18:40:07.036500+0000","last_peered":"2023-07-21T18:40:07.036500+0000","last_clean":"2023-07-21T18:40:07.036500+0000","last_became_active":"2023-07-21T18:40:04.791441+0000","last_became_peered":"2023-07-21T18:40:04.791441+0000","last_unstale":"2023-07-21T18:40:07.036500+0000","last_undegraded":"2023-07-21T18:40:07.036500+0000","last_fullsized":"2023-07-21T18:40:07.036500+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:01:49.043942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.777066+0000","last_change":"2023-07-21T18:40:04.794913+0000","last_active":"2023-07-21T18:40:05.777066+0000","last_peered":"2023-07-21T18:40:05.777066+0000","last_clean":"2023-07-21T18:40:05.777066+0000","last_became_active":"2023-07-21T18:40:04.794527+0000","last_became_peered":"2023-07-21T18:40:04.794527+0000","last_unstale":"2023-07-21T18:40:05.777066+0000","last_undegraded":"2023-07-21T18:40:05.777066+0000","last_fullsized":"2023-07-21T18:40:05.777066+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:13:56.682713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.772671+0000","last_change":"2023-07-21T18:40:04.789145+0000","last_active":"2023-07-21T18:40:05.772671+0000","last_peered":"2023-07-21T18:40:05.772671+0000","last_clean":"2023-07-21T18:40:05.772671+0000","last_became_active":"2023-07-21T18:40:04.788285+0000","last_became_peered":"2023-07-21T18:40:04.788285+0000","last_unstale":"2023-07-21T18:40:05.772671+0000","last_undegraded":"2023-07-21T18:40:05.772671+0000","last_fullsized":"2023-07-21T18:40:05.772671+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:42:10.814436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-07-21T18:40:05.773199+0000","last_change":"2023-07-21T18:40:04.788821+0000","last_active":"2023-07-21T18:40:05.773199+0000","last_peered":"2023-07-21T18:40:05.773199+0000","last_clean":"2023-07-21T18:40:05.773199+0000","last_became_active":"2023-07-21T18:40:04.788513+0000","last_became_peered":"2023-07-21T18:40:04.788513+0000","last_unstale":"2023-07-21T18:40:05.773199+0000","last_undegraded":"2023-07-21T18:40:05.773199+0000","last_fullsized":"2023-07-21T18:40:05.773199+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:35:28.640398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63495,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":427,"num_read_kb":281,"num_write":228,"num_write_kb":124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":190485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":200,"ondisk_log_size":200,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579925,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27792,"kb_used_data":840,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717360,"statfs":{"total":95995035648,"available":95966576640,"internally_reserved":0,"allocated":860160,"data_stored":408894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.4990000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.59599999999999997}]},{"osd":1,"last update":"Fri Jul 21 18:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.77500000000000002}]},{"osd":2,"last update":"Fri Jul 21 18:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.64400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":4,"last update":"Fri Jul 21 18:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.68700000000000006}]},{"osd":5,"last update":"Fri Jul 21 18:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.69799999999999995}]},{"osd":6,"last update":"Fri Jul 21 18:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.72499999999999998}]},{"osd":7,"last update":"Fri Jul 21 18:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.87},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.85699999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78900000000000003}]},{"osd":9,"last update":"Fri Jul 21 18:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.643,"5min":1.643,"15min":1.643},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.95899999999999996}]},{"osd":10,"last update":"Fri Jul 21 18:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.89000000000000001}]}]},{"osd":10,"up_from":67,"seq":287762808856,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28468,"kb_used_data":1516,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716684,"statfs":{"total":95995035648,"available":95965884416,"internally_reserved":0,"allocated":1552384,"data_stored":1056863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":1,"last update":"Fri Jul 21 18:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.624}]},{"osd":2,"last update":"Fri Jul 21 18:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.79800000000000004}]},{"osd":3,"last update":"Fri Jul 21 18:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.755}]},{"osd":4,"last update":"Fri Jul 21 18:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.81799999999999995}]},{"osd":5,"last update":"Fri Jul 21 18:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.96899999999999997}]},{"osd":6,"last update":"Fri Jul 21 18:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.874},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":1.0069999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":8,"last update":"Fri Jul 21 18:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.98399999999999999}]},{"osd":9,"last update":"Fri Jul 21 18:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.89000000000000001}]},{"osd":11,"last update":"Fri Jul 21 18:40:28 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":1.032}]}]},{"osd":9,"up_from":62,"seq":266287972382,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27816,"kb_used_data":864,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717336,"statfs":{"total":95995035648,"available":95966552064,"internally_reserved":0,"allocated":884736,"data_stored":408872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.52500000000000002,"15min":0.53900000000000003},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.996,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.438,"5min":0.505,"15min":0.51600000000000001},"min":{"1min":0.316,"5min":0.252,"15min":0.252},"max":{"1min":0.621,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.42099999999999999}]},{"osd":1,"last update":"Fri Jul 21 18:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.73199999999999998,"15min":0.749},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.099,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":1.1830000000000001},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.71299999999999997,"15min":0.72099999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.169,"5min":1.681,"15min":1.681},"last":0.57599999999999996}]},{"osd":2,"last update":"Fri Jul 21 18:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.70799999999999996,"15min":0.71399999999999997},"min":{"1min":0.47299999999999998,"5min":0.38,"15min":0.38},"max":{"1min":1.222,"5min":1.9530000000000001,"15min":1.9530000000000001},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.78300000000000003,"15min":0.753},"min":{"1min":0.46100000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":6.4020000000000001,"5min":6.4020000000000001,"15min":6.4020000000000001},"last":0.60999999999999999}]},{"osd":3,"last update":"Fri Jul 21 18:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.627,"15min":0.64900000000000002},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.89000000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.627,"15min":0.64500000000000002},"min":{"1min":0.35199999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.1910000000000001,"5min":2.048,"15min":2.048},"last":0.39500000000000002}]},{"osd":4,"last update":"Fri Jul 21 18:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.80700000000000005,"15min":0.83699999999999997},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.127,"5min":1.8100000000000001,"15min":1.8100000000000001},"last":0.875},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.79800000000000004,"15min":0.82299999999999995},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.94799999999999995,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.67300000000000004}]},{"osd":5,"last update":"Fri Jul 21 18:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.80500000000000005,"15min":0.77700000000000002},"min":{"1min":0.49299999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":6.5069999999999997,"5min":6.5069999999999997,"15min":6.5069999999999997},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.78600000000000003,"15min":0.80400000000000005},"min":{"1min":0.503,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.1399999999999999,"5min":1.992,"15min":1.992},"last":0.51300000000000001}]},{"osd":6,"last update":"Fri Jul 21 18:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.74299999999999999,"15min":0.77300000000000002},"min":{"1min":0.375,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.1240000000000001,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.78600000000000003,"15min":0.81399999999999995},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.2410000000000001,"5min":1.595,"15min":1.595},"last":0.48799999999999999}]},{"osd":7,"last update":"Fri Jul 21 18:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.85799999999999998,"15min":0.88200000000000001},"min":{"1min":0.54700000000000004,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.026,"5min":1.284,"15min":1.284},"last":1.3819999999999999},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.89300000000000002,"15min":0.92000000000000004},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.175,"5min":2.0750000000000002,"15min":2.0750000000000002},"last":0.999}]},{"osd":8,"last update":"Fri Jul 21 18:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.90700000000000003,"15min":0.92900000000000005},"min":{"1min":0.61499999999999999,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.2749999999999999,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":1.2390000000000001},{"interface":"front","average":{"1min":1.0669999999999999,"5min":0.90600000000000003,"15min":0.879},"min":{"1min":0.60899999999999999,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":6.5679999999999996,"5min":6.5679999999999996,"15min":6.5679999999999996},"last":0.71499999999999997}]},{"osd":10,"last update":"Fri Jul 21 18:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":2.1240000000000001,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":1.071},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":1.405}]}]},{"osd":8,"up_from":55,"seq":236223201314,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27808,"kb_used_data":856,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717344,"statfs":{"total":95995035648,"available":95966560256,"internally_reserved":0,"allocated":876544,"data_stored":408919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:40:23 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.61699999999999999,"15min":0.60899999999999999},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":4.2000000000000002,"5min":4.2000000000000002,"15min":4.2000000000000002},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.59399999999999997,"15min":0.58599999999999997},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":3.145,"5min":3.145,"15min":3.145},"last":0.54200000000000004}]},{"osd":1,"last update":"Fri Jul 21 18:40:23 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.59099999999999997,"15min":0.58199999999999996},"min":{"1min":0.38800000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.90400000000000003,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.624,"5min":0.61399999999999999,"15min":0.61199999999999999},"min":{"1min":0.41999999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.878,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.71699999999999997}]},{"osd":2,"last update":"Fri Jul 21 18:40:23 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.46100000000000002,"15min":0.45900000000000002},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.72499999999999998,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.45600000000000002}]},{"osd":3,"last update":"Fri Jul 21 18:40:23 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":0.73799999999999999,"15min":0.68600000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":11.260999999999999,"5min":11.260999999999999,"15min":11.260999999999999},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.627,"15min":0.623},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":3.2719999999999998,"5min":3.2719999999999998,"15min":3.2719999999999998},"last":0.61899999999999999}]},{"osd":4,"last update":"Fri Jul 21 18:40:23 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.56999999999999995,"15min":0.56499999999999995},"min":{"1min":0.39500000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.629,"15min":0.628},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.78000000000000003,"5min":0.875,"15min":0.875},"last":0.66200000000000003}]},{"osd":5,"last update":"Fri Jul 21 18:40:23 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.58099999999999996,"15min":0.58299999999999996},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.83499999999999996,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.748},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52900000000000003,"15min":0.53100000000000003},"min":{"1min":0.35599999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.72299999999999998,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.49399999999999999}]},{"osd":6,"last update":"Fri Jul 21 18:40:23 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":0.78500000000000003,"15min":0.73699999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":10.292999999999999,"5min":10.292999999999999,"15min":10.292999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.77700000000000002,"15min":0.76500000000000001},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":4.1029999999999998,"5min":4.1029999999999998,"15min":4.1029999999999998},"last":0.69799999999999995}]},{"osd":7,"last update":"Fri Jul 21 18:40:23 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.71099999999999997,"15min":0.71499999999999997},"min":{"1min":0.44500000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.97599999999999998,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.66900000000000004,"15min":0.67100000000000004},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.80600000000000005,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.72999999999999998}]},{"osd":9,"last update":"Fri Jul 21 18:40:46 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.54100000000000004,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.95199999999999996,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":1.2509999999999999,"5min":0.86699999999999999,"15min":0.80300000000000005},"min":{"1min":0.54900000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":11.151,"5min":11.151,"15min":11.151},"last":0.84199999999999997}]},{"osd":10,"last update":"Fri Jul 21 18:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.89800000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]}]},{"osd":7,"up_from":48,"seq":206158430246,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28392,"kb_used_data":1440,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716760,"statfs":{"total":95995035648,"available":95965962240,"internally_reserved":0,"allocated":1474560,"data_stored":1001356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.53400000000000003,"15min":0.52600000000000002},"min":{"1min":0.41599999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.53900000000000003,"15min":0.53600000000000003},"min":{"1min":0.34000000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.48899999999999999}]},{"osd":1,"last update":"Fri Jul 21 18:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.41099999999999998,"15min":0.40400000000000003},"min":{"1min":0.311,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.43099999999999999,"15min":0.42299999999999999},"min":{"1min":0.307,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.71099999999999997,"5min":0.75,"15min":0.75},"last":0.41499999999999998}]},{"osd":2,"last update":"Fri Jul 21 18:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.57599999999999996,"15min":0.57099999999999995},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58899999999999997,"15min":0.58999999999999997},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.83299999999999996}]},{"osd":3,"last update":"Fri Jul 21 18:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.54900000000000004,"15min":0.54000000000000004},"min":{"1min":0.39700000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.58299999999999996,"15min":0.57199999999999995},"min":{"1min":0.44500000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.86899999999999999,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.628}]},{"osd":4,"last update":"Fri Jul 21 18:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.52700000000000002,"15min":0.52100000000000002},"min":{"1min":0.35399999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.52500000000000002,"15min":0.51700000000000002},"min":{"1min":0.39800000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.89500000000000002,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.44500000000000001}]},{"osd":5,"last update":"Fri Jul 21 18:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.64800000000000002,"15min":0.64100000000000001},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.64600000000000002,"15min":0.64200000000000002},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.79800000000000004}]},{"osd":6,"last update":"Fri Jul 21 18:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.66500000000000004,"15min":0.65500000000000003},"min":{"1min":0.51300000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.93500000000000005,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.67300000000000004,"15min":0.66100000000000003},"min":{"1min":0.46400000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0069999999999999,"5min":1.087,"15min":1.087},"last":0.56299999999999994}]},{"osd":8,"last update":"Fri Jul 21 18:40:22 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.69199999999999995,"15min":0.68999999999999995},"min":{"1min":0.54200000000000004,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.746,"5min":0.71999999999999997,"15min":0.71599999999999997},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.85399999999999998}]},{"osd":9,"last update":"Fri Jul 21 18:40:46 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.74199999999999999,"15min":0.73499999999999999},"min":{"1min":0.46500000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.98099999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.74199999999999999,"15min":0.73599999999999999},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.73299999999999998}]},{"osd":10,"last update":"Fri Jul 21 18:40:06 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.76900000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91200000000000003}]}]},{"osd":6,"up_from":43,"seq":184683593770,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27796,"kb_used_data":844,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717356,"statfs":{"total":95995035648,"available":95966572544,"internally_reserved":0,"allocated":864256,"data_stored":408274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:40:40 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.41299999999999998,"15min":0.40899999999999997},"min":{"1min":0.29599999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.504},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.48099999999999998,"15min":0.44500000000000001},"min":{"1min":0.311,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":4.859,"5min":4.859,"15min":4.859},"last":0.51800000000000002}]},{"osd":1,"last update":"Fri Jul 21 18:40:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.44500000000000001,"5min":0.37,"15min":0.37},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.39000000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.84399999999999997,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.65300000000000002}]},{"osd":2,"last update":"Fri Jul 21 18:40:40 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.64800000000000002,"15min":0.60399999999999998},"min":{"1min":0.39900000000000002,"5min":0.376,"15min":0.376},"max":{"1min":4.9180000000000001,"5min":4.9180000000000001,"15min":4.9180000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.59599999999999997,"15min":0.58199999999999996},"min":{"1min":0.42099999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.83399999999999996}]},{"osd":3,"last update":"Fri Jul 21 18:40:40 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.52300000000000002,"15min":0.48799999999999999},"min":{"1min":0.33800000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":4.4720000000000004,"5min":4.4720000000000004,"15min":4.4720000000000004},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.56999999999999995,"15min":0.53100000000000003},"min":{"1min":0.32200000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":5.2910000000000004,"5min":5.2910000000000004,"15min":5.2910000000000004},"last":0.441}]},{"osd":4,"last update":"Fri Jul 21 18:40:40 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.63900000000000001,"15min":0.64400000000000002},"min":{"1min":0.40300000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.022,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.64100000000000001,"15min":0.64400000000000002},"min":{"1min":0.46500000000000002,"5min":0.442,"15min":0.442},"max":{"1min":0.89600000000000002,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.84899999999999998}]},{"osd":5,"last update":"Fri Jul 21 18:40:40 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.70699999999999996,"15min":0.66600000000000004},"min":{"1min":0.47499999999999998,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":4.9960000000000004,"5min":4.9960000000000004,"15min":4.9960000000000004},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.70299999999999996,"15min":0.69499999999999995},"min":{"1min":0.49299999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.145,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.63700000000000001}]},{"osd":7,"last update":"Fri Jul 21 18:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.71299999999999997,"15min":0.70899999999999996},"min":{"1min":0.53500000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.874},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.70599999999999996,"15min":0.70499999999999996},"min":{"1min":0.54300000000000004,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.68600000000000005}]},{"osd":8,"last update":"Fri Jul 21 18:40:26 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.79000000000000004,"15min":0.80600000000000005},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.96099999999999997,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.77500000000000002,"15min":0.78700000000000003},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.91800000000000004,"5min":1.171,"15min":1.171},"last":0.81299999999999994}]},{"osd":9,"last update":"Fri Jul 21 18:40:44 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.75,"15min":0.75600000000000001},"min":{"1min":0.56799999999999995,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.194,"5min":1.196,"15min":1.196},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.72599999999999998,"15min":0.73399999999999999},"min":{"1min":0.52600000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.872,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.54300000000000004}]},{"osd":10,"last update":"Fri Jul 21 18:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.82299999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":900,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":921600,"data_stored":466056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:39:59 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.50900000000000001,"15min":0.48499999999999999},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.52800000000000002,"15min":0.498},"min":{"1min":0.39200000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.69499999999999995}]},{"osd":2,"last update":"Fri Jul 21 18:40:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.55900000000000005,"15min":0.52800000000000002},"min":{"1min":0.38600000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.84099999999999997,"5min":3.2959999999999998,"15min":3.2959999999999998},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.55400000000000005,"15min":0.52600000000000002},"min":{"1min":0.34999999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.93300000000000005,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":0.80700000000000005}]},{"osd":3,"last update":"Fri Jul 21 18:40:44 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.55300000000000005,"15min":0.52500000000000002},"min":{"1min":0.38400000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.58099999999999996,"15min":0.53700000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":3.7789999999999999,"5min":3.7789999999999999,"15min":3.7789999999999999},"last":0.57599999999999996}]},{"osd":4,"last update":"Fri Jul 21 18:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.41799999999999998,"15min":0.40100000000000002},"min":{"1min":0.32600000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.501},{"interface":"front","average":{"1min":0.497,"5min":0.41799999999999998,"15min":0.40500000000000003},"min":{"1min":0.31,"5min":0.254,"15min":0.254},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.72499999999999998}]},{"osd":5,"last update":"Fri Jul 21 18:40:19 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.60699999999999998,"15min":0.58799999999999997},"min":{"1min":0.45100000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.089,"5min":1.653,"15min":1.653},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.623,"15min":0.59799999999999998},"min":{"1min":0.50800000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.94999999999999996,"5min":1.641,"15min":1.641},"last":0.95899999999999996}]},{"osd":6,"last update":"Fri Jul 21 18:40:44 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.64400000000000002,"15min":0.63200000000000001},"min":{"1min":0.46200000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.92900000000000005,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.64500000000000002,"15min":0.64000000000000001},"min":{"1min":0.41599999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.96099999999999997,"5min":1.022,"15min":1.022},"last":0.84899999999999998}]},{"osd":7,"last update":"Fri Jul 21 18:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.51900000000000002,"15min":0.502},"min":{"1min":0.41399999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.54800000000000004,"15min":0.53000000000000003},"min":{"1min":0.40699999999999997,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.78400000000000003}]},{"osd":8,"last update":"Fri Jul 21 18:40:26 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.73599999999999999,"15min":0.72799999999999998},"min":{"1min":0.51500000000000001,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.1910000000000001,"5min":2.7360000000000002,"15min":2.7360000000000002},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.72699999999999998,"15min":0.72499999999999998},"min":{"1min":0.52400000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.96899999999999997,"5min":2.7210000000000001,"15min":2.7210000000000001},"last":0.82199999999999995}]},{"osd":9,"last update":"Fri Jul 21 18:40:44 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.745,"15min":0.73899999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.753,"15min":0.746},"min":{"1min":0.55100000000000005,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":1.0189999999999999}]},{"osd":10,"last update":"Fri Jul 21 18:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.748},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.92500000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97899999999999998}]}]},{"osd":0,"up_from":8,"seq":34359738435,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27820,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717332,"statfs":{"total":95995035648,"available":95966547968,"internally_reserved":0,"allocated":888832,"data_stored":409880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 21 18:39:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.59099999999999997,"15min":0.57399999999999995},"min":{"1min":0.36299999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.88200000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.57699999999999996,"15min":0.53700000000000003},"min":{"1min":0.42899999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.82699999999999996}]},{"osd":2,"last update":"Fri Jul 21 18:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.58199999999999996,"15min":0.56699999999999995},"min":{"1min":0.39400000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.92800000000000005,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.60599999999999998,"15min":0.58599999999999997},"min":{"1min":0.44900000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":3.0470000000000002,"5min":3.0470000000000002,"15min":3.0470000000000002},"last":0.73799999999999999}]},{"osd":3,"last update":"Fri Jul 21 18:38:36 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.42799999999999999,"15min":0.42999999999999999},"min":{"1min":0.29999999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.621,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.48199999999999998,"15min":0.47699999999999998},"min":{"1min":0.33600000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.69199999999999995,"5min":0.879,"15min":0.879},"last":0.44500000000000001}]},{"osd":4,"last update":"Fri Jul 21 18:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.60599999999999998,"15min":0.57199999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":2.948,"5min":2.948,"15min":2.948},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.56200000000000006,"15min":0.54600000000000004},"min":{"1min":0.40600000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.80400000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.49099999999999999}]},{"osd":5,"last update":"Fri Jul 21 18:40:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.58799999999999997},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.748,"5min":0.64600000000000002,"15min":0.61199999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":3.0150000000000001,"5min":3.0150000000000001,"15min":3.0150000000000001},"last":0.50700000000000001}]},{"osd":6,"last update":"Fri Jul 21 18:40:46 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.52200000000000002,"15min":0.51100000000000001},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":2.077,"5min":2.077,"15min":2.077},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.58799999999999997,"15min":0.56899999999999995},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":2.4500000000000002,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":0.53800000000000003}]},{"osd":7,"last update":"Fri Jul 21 18:39:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.47699999999999998}]},{"osd":8,"last update":"Fri Jul 21 18:40:23 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.79700000000000004,"15min":0.81499999999999995},"min":{"1min":0.54600000000000004,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.90900000000000003,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81599999999999995,"15min":0.81699999999999995},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":2.9830000000000001,"5min":2.9830000000000001,"15min":2.9830000000000001},"last":0.58799999999999997}]},{"osd":9,"last update":"Fri Jul 21 18:40:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.71999999999999997,"15min":0.74199999999999999},"min":{"1min":0.37,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.78800000000000003,"15min":0.81299999999999994},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.421,"5min":1.421,"15min":1.421},"last":0.66800000000000004}]},{"osd":10,"last update":"Fri Jul 21 18:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":2.9079999999999999,"5min":2.9079999999999999,"15min":2.9079999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.84999999999999998}]},{"osd":11,"last update":"Fri Jul 21 18:40:26 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":3.0840000000000001,"5min":3.0840000000000001,"15min":3.0840000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.60599999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411387,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":924,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":946176,"data_stored":466501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:40:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.54000000000000004,"15min":0.53200000000000003},"min":{"1min":0.40000000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.92100000000000004,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.50800000000000001,"15min":0.503},"min":{"1min":0.31,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.80500000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.57499999999999996}]},{"osd":1,"last update":"Fri Jul 21 18:40:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.53500000000000003,"15min":0.51900000000000002},"min":{"1min":0.45200000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.76200000000000001,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.56000000000000005,"15min":0.53800000000000003},"min":{"1min":0.39300000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.79400000000000004}]},{"osd":3,"last update":"Fri Jul 21 18:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.58899999999999997,"15min":0.56499999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.92400000000000004,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.56799999999999995,"15min":0.56000000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.79600000000000004,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.70199999999999996}]},{"osd":4,"last update":"Fri Jul 21 18:39:57 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.58999999999999997,"15min":0.56999999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.633,"5min":1.633,"15min":1.633},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.626,"15min":0.60899999999999999},"min":{"1min":0.42999999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.77400000000000002}]},{"osd":5,"last update":"Fri Jul 21 18:40:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.47099999999999997,"15min":0.45700000000000002},"min":{"1min":0.29699999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.46700000000000003,"15min":0.45600000000000002},"min":{"1min":0.35799999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.60899999999999999}]},{"osd":6,"last update":"Fri Jul 21 18:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.61899999999999999,"15min":0.59599999999999997},"min":{"1min":0.53000000000000003,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.85799999999999998,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.628,"15min":0.62},"min":{"1min":0.436,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.84399999999999997,"5min":0.873,"15min":0.873},"last":0.81599999999999995}]},{"osd":7,"last update":"Fri Jul 21 18:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.61699999999999999,"15min":0.60799999999999998},"min":{"1min":0.45100000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.577,"5min":1.577,"15min":1.577},"last":0.495},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.65500000000000003,"15min":0.65100000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.878,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.56499999999999995}]},{"osd":8,"last update":"Fri Jul 21 18:40:26 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.58899999999999997,"15min":0.57099999999999995},"min":{"1min":0.40799999999999997,"5min":0.376,"15min":0.376},"max":{"1min":0.91300000000000003,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.60799999999999998,"15min":0.59799999999999998},"min":{"1min":0.40799999999999997,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.88600000000000001}]},{"osd":9,"last update":"Fri Jul 21 18:40:46 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.76000000000000001,"15min":0.75600000000000001},"min":{"1min":0.59499999999999997,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.95999999999999996,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.69199999999999995,"15min":0.68999999999999995},"min":{"1min":0.47499999999999998,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.89600000000000002,"5min":1.01,"15min":1.01},"last":0.87}]},{"osd":10,"last update":"Fri Jul 21 18:40:06 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.94699999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92800000000000005}]}]},{"osd":3,"up_from":24,"seq":103079215158,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27808,"kb_used_data":856,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717344,"statfs":{"total":95995035648,"available":95966560256,"internally_reserved":0,"allocated":876544,"data_stored":407822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.35699999999999998,"15min":0.33100000000000002},"min":{"1min":0.26800000000000002,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.54000000000000004,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.373,"15min":0.35099999999999998},"min":{"1min":0.28100000000000003,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.51600000000000001,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.32300000000000001}]},{"osd":1,"last update":"Fri Jul 21 18:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.57199999999999995,"15min":0.55100000000000005},"min":{"1min":0.39900000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.75800000000000001,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59199999999999997,"15min":0.55000000000000004},"min":{"1min":0.39400000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.83099999999999996,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.55100000000000005}]},{"osd":2,"last update":"Fri Jul 21 18:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.57699999999999996,"15min":0.55300000000000005},"min":{"1min":0.36799999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.81599999999999995,"5min":0.876,"15min":0.876},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.56799999999999995,"15min":0.53900000000000003},"min":{"1min":0.373,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.52900000000000003}]},{"osd":4,"last update":"Fri Jul 21 18:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.61599999999999999,"15min":0.60499999999999998},"min":{"1min":0.42799999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.61699999999999999,"15min":0.59999999999999998},"min":{"1min":0.45300000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.50800000000000001}]},{"osd":5,"last update":"Fri Jul 21 18:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.622,"15min":0.58999999999999997},"min":{"1min":0.45400000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.89000000000000001,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.442},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.59799999999999998,"15min":0.58199999999999996},"min":{"1min":0.39500000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.90500000000000003,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.46300000000000002}]},{"osd":6,"last update":"Fri Jul 21 18:40:41 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.498,"15min":0.48599999999999999},"min":{"1min":0.33600000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.80900000000000005,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.55800000000000005,"15min":0.55400000000000005},"min":{"1min":0.36599999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.63400000000000001,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.42399999999999999}]},{"osd":7,"last update":"Fri Jul 21 18:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.66800000000000004,"15min":0.67000000000000004},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.97299999999999998,"5min":0.999,"15min":0.999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.73299999999999998,"15min":0.72499999999999998},"min":{"1min":0.55100000000000005,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.65000000000000002}]},{"osd":8,"last update":"Fri Jul 21 18:39:25 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.47299999999999998}]},{"osd":9,"last update":"Fri Jul 21 18:40:45 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.68600000000000005,"15min":0.71499999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.77900000000000003,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.624,"5min":0.81999999999999995,"15min":0.85299999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.88100000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.57199999999999995}]},{"osd":10,"last update":"Fri Jul 21 18:40:07 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.69599999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]}]},{"osd":4,"up_from":31,"seq":133143986227,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27796,"kb_used_data":844,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717356,"statfs":{"total":95995035648,"available":95966572544,"internally_reserved":0,"allocated":864256,"data_stored":408519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:40:02 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.51800000000000002,"15min":0.48199999999999998},"min":{"1min":0.40500000000000003,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.94099999999999995,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.52500000000000002,"15min":0.50600000000000001},"min":{"1min":0.373,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.80600000000000005,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.58299999999999996}]},{"osd":1,"last update":"Fri Jul 21 18:40:02 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.373,"15min":0.34599999999999997},"min":{"1min":0.29799999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.376},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.40899999999999997,"15min":0.38600000000000001},"min":{"1min":0.30399999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.76200000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.52100000000000002}]},{"osd":2,"last update":"Fri Jul 21 18:40:02 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.54700000000000004,"15min":0.52900000000000003},"min":{"1min":0.44700000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.80700000000000005,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.56200000000000006,"15min":0.53200000000000003},"min":{"1min":0.41799999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.55000000000000004}]},{"osd":3,"last update":"Fri Jul 21 18:40:02 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.55800000000000005,"15min":0.53300000000000003},"min":{"1min":0.52300000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.85999999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56299999999999994,"15min":0.55700000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.97199999999999998,"5min":1.05,"15min":1.05},"last":0.71999999999999997}]},{"osd":5,"last update":"Fri Jul 21 18:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.64000000000000001,"15min":0.60599999999999998},"min":{"1min":0.48599999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.90800000000000003,"5min":1.571,"15min":1.571},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.64100000000000001,"15min":0.61499999999999999},"min":{"1min":0.434,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.90800000000000003,"5min":1.377,"15min":1.377},"last":0.62}]},{"osd":6,"last update":"Fri Jul 21 18:40:44 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.63900000000000001,"15min":0.61699999999999999},"min":{"1min":0.498,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.83699999999999997,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.65000000000000002,"15min":0.61399999999999999},"min":{"1min":0.46100000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.81299999999999994,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.76400000000000001}]},{"osd":7,"last update":"Fri Jul 21 18:40:02 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.57299999999999995,"15min":0.56100000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.54500000000000004,"15min":0.54000000000000004},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.873,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.63100000000000001}]},{"osd":8,"last update":"Fri Jul 21 18:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.65900000000000003,"15min":0.66100000000000003},"min":{"1min":0.46200000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.92400000000000004,"5min":1.026,"15min":1.026},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.73099999999999998,"15min":0.72499999999999998},"min":{"1min":0.56899999999999995,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.86399999999999999}]},{"osd":9,"last update":"Fri Jul 21 18:40:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.79000000000000004,"15min":0.79900000000000004},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.93999999999999995,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.83599999999999997,"15min":0.84599999999999997},"min":{"1min":0.60499999999999998,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.95499999999999996,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.94499999999999995}]},{"osd":10,"last update":"Fri Jul 21 18:40:07 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.79800000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82599999999999996}]}]},{"osd":5,"up_from":36,"seq":154618822702,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28404,"kb_used_data":1452,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716748,"statfs":{"total":95995035648,"available":95965949952,"internally_reserved":0,"allocated":1486848,"data_stored":1000740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.52600000000000002,"15min":0.53000000000000003},"min":{"1min":0.313,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.54800000000000004,"15min":0.55700000000000005},"min":{"1min":0.316,"5min":0.314,"15min":0.314},"max":{"1min":0.78700000000000003,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.40799999999999997}]},{"osd":1,"last update":"Fri Jul 21 18:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.53400000000000003,"15min":0.50800000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.54000000000000004,"15min":0.51300000000000001},"min":{"1min":0.43099999999999999,"5min":0.375,"15min":0.375},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.505}]},{"osd":2,"last update":"Fri Jul 21 18:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.40000000000000002,"15min":0.38800000000000001},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.59299999999999997,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.41699999999999998,"15min":0.39800000000000002},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.623,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.47599999999999998}]},{"osd":3,"last update":"Fri Jul 21 18:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.56000000000000005,"15min":0.56799999999999995},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58199999999999996,"15min":0.58099999999999996},"min":{"1min":0.434,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.79800000000000004,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.442}]},{"osd":4,"last update":"Fri Jul 21 18:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.57199999999999995,"15min":0.55400000000000005},"min":{"1min":0.36899999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.83899999999999997,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.56200000000000006,"15min":0.54000000000000004},"min":{"1min":0.45700000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.63100000000000001}]},{"osd":6,"last update":"Fri Jul 21 18:40:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.59599999999999997,"15min":0.59999999999999998},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.60399999999999998,"15min":0.59099999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.66200000000000003}]},{"osd":7,"last update":"Fri Jul 21 18:40:02 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.66600000000000004,"15min":0.66800000000000004},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.64300000000000002,"15min":0.64000000000000001},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.748}]},{"osd":8,"last update":"Fri Jul 21 18:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.92200000000000004,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.626,"5min":0.63200000000000001,"15min":0.63300000000000001},"min":{"1min":0.40999999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.81399999999999995,"5min":0.874,"15min":0.874},"last":0.72999999999999998}]},{"osd":9,"last update":"Fri Jul 21 18:40:45 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.72499999999999998,"15min":0.72899999999999998},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.877,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.73199999999999998,"15min":0.73399999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.71399999999999997}]},{"osd":10,"last update":"Fri Jul 21 18:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.77900000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":58579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":58751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":58953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-21T18:40:54.447 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2023-07-21T18:40:54.447 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-07-21T18:40:54.447 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2023-07-21T18:40:54.448 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph health --format=json 2023-07-21T18:40:55.008 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:40:55.282 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:54 smithi039 ceph-mon[96489]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T18:40:55.371 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:54 smithi193 ceph-mon[98204]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T18:40:55.390 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:54 smithi155 ceph-mon[97934]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T18:40:56.123 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.097+0000 7f1b1061d700 1 -- 172.21.15.39:0/1688343665 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b08072850 msgr2=0x7f1b08072c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:56.123 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.097+0000 7f1b1061d700 1 --2- 172.21.15.39:0/1688343665 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b08072850 0x7f1b08072c20 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1af8009990 tx=0x7f1af8009ca0 comp rx=0 tx=0).stop 2023-07-21T18:40:56.123 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.097+0000 7f1b1061d700 1 -- 172.21.15.39:0/1688343665 shutdown_connections 2023-07-21T18:40:56.123 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.097+0000 7f1b1061d700 1 --2- 172.21.15.39:0/1688343665 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1b0810c440 0x7f1b0810e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:56.123 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.097+0000 7f1b1061d700 1 --2- 172.21.15.39:0/1688343665 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b08072850 0x7f1b08072c20 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:56.123 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.097+0000 7f1b1061d700 1 --2- 172.21.15.39:0/1688343665 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b08073160 0x7f1b0810bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:56.124 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.097+0000 7f1b1061d700 1 -- 172.21.15.39:0/1688343665 >> 172.21.15.39:0/1688343665 conn(0x7f1b080fb660 msgr2=0x7f1b080fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:56.124 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.098+0000 7f1b1061d700 1 -- 172.21.15.39:0/1688343665 shutdown_connections 2023-07-21T18:40:56.124 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.098+0000 7f1b1061d700 1 -- 172.21.15.39:0/1688343665 wait complete. 2023-07-21T18:40:56.125 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.098+0000 7f1b1061d700 1 Processor -- start 2023-07-21T18:40:56.126 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.098+0000 7f1b1061d700 1 -- start start 2023-07-21T18:40:56.126 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.099+0000 7f1b1061d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1b08072850 0x7f1b08197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:56.126 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.099+0000 7f1b1061d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b08073160 0x7f1b081983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:56.126 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.099+0000 7f1b1061d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b0810c440 0x7f1b0819c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:56.126 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.099+0000 7f1b1061d700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0819ce30 con 0x7f1b08073160 2023-07-21T18:40:56.126 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.099+0000 7f1b1061d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0819cfa0 con 0x7f1b0810c440 2023-07-21T18:40:56.126 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.099+0000 7f1b1061d700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0819d160 con 0x7f1b08072850 2023-07-21T18:40:56.127 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.099+0000 7f1b0dbb8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b08073160 0x7f1b081983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:56.127 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.099+0000 7f1b0dbb8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b08073160 0x7f1b081983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55700/0 (socket says 172.21.15.39:55700) 2023-07-21T18:40:56.127 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.099+0000 7f1b0dbb8700 1 -- 172.21.15.39:0/3605691598 learned_addr learned my addr 172.21.15.39:0/3605691598 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:40:56.127 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.099+0000 7f1b0e3b9700 1 --2- 172.21.15.39:0/3605691598 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1b08072850 0x7f1b08197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:56.127 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.099+0000 7f1b0ebba700 1 --2- 172.21.15.39:0/3605691598 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b0810c440 0x7f1b0819c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:56.127 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.099+0000 7f1b0dbb8700 1 -- 172.21.15.39:0/3605691598 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1b08072850 msgr2=0x7f1b08197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:56.127 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.099+0000 7f1b0dbb8700 1 --2- 172.21.15.39:0/3605691598 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1b08072850 0x7f1b08197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:56.128 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.099+0000 7f1b0dbb8700 1 -- 172.21.15.39:0/3605691598 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b0810c440 msgr2=0x7f1b0819c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:56.128 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.099+0000 7f1b0dbb8700 1 --2- 172.21.15.39:0/3605691598 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b0810c440 0x7f1b0819c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:56.128 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.099+0000 7f1b0dbb8700 1 -- 172.21.15.39:0/3605691598 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1af80095f0 con 0x7f1b08073160 2023-07-21T18:40:56.128 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.099+0000 7f1b0dbb8700 1 --2- 172.21.15.39:0/3605691598 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b08073160 0x7f1b081983b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1b0400d980 tx=0x7f1b0400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:56.128 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.100+0000 7f1aff7fe700 1 -- 172.21.15.39:0/3605691598 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b04004220 con 0x7f1b08073160 2023-07-21T18:40:56.128 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.100+0000 7f1aff7fe700 1 -- 172.21.15.39:0/3605691598 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b04007640 con 0x7f1b08073160 2023-07-21T18:40:56.128 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.100+0000 7f1b1061d700 1 -- 172.21.15.39:0/3605691598 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b0819d440 con 0x7f1b08073160 2023-07-21T18:40:56.128 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.100+0000 7f1aff7fe700 1 -- 172.21.15.39:0/3605691598 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b04005610 con 0x7f1b08073160 2023-07-21T18:40:56.129 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.100+0000 7f1b1061d700 1 -- 172.21.15.39:0/3605691598 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b081a4ca0 con 0x7f1b08073160 2023-07-21T18:40:56.129 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.101+0000 7f1b1061d700 1 -- 172.21.15.39:0/3605691598 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b0804e210 con 0x7f1b08073160 2023-07-21T18:40:56.129 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.102+0000 7f1aff7fe700 1 -- 172.21.15.39:0/3605691598 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f1b04004750 con 0x7f1b08073160 2023-07-21T18:40:56.130 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.102+0000 7f1aff7fe700 1 --2- 172.21.15.39:0/3605691598 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f1af40a11e0 0x7f1af40a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:56.130 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.102+0000 7f1b0e3b9700 1 --2- 172.21.15.39:0/3605691598 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f1af40a11e0 0x7f1af40a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:56.130 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.102+0000 7f1aff7fe700 1 -- 172.21.15.39:0/3605691598 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f1b040ccc50 con 0x7f1b08073160 2023-07-21T18:40:56.130 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.103+0000 7f1b0e3b9700 1 --2- 172.21.15.39:0/3605691598 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f1af40a11e0 0x7f1af40a3690 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1af800b5c0 tx=0x7f1af8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:56.130 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.106+0000 7f1aff7fe700 1 -- 172.21.15.39:0/3605691598 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f1b040963f0 con 0x7f1b08073160 2023-07-21T18:40:56.189 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:56 smithi039 ceph-mon[96489]: pgmap v273: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-07-21T18:40:56.371 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:56 smithi193 ceph-mon[98204]: pgmap v273: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-07-21T18:40:56.373 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.348+0000 7f1b1061d700 1 -- 172.21.15.39:0/3605691598 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f1b081991d0 con 0x7f1b08073160 2023-07-21T18:40:56.374 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.349+0000 7f1aff7fe700 1 -- 172.21.15.39:0/3605691598 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f1b04016040 con 0x7f1b08073160 2023-07-21T18:40:56.374 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:40:56.374 INFO:teuthology.orchestra.run.smithi039.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-07-21T18:40:56.375 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.351+0000 7f1b1061d700 1 -- 172.21.15.39:0/3605691598 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f1af40a11e0 msgr2=0x7f1af40a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:56.376 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.351+0000 7f1b1061d700 1 --2- 172.21.15.39:0/3605691598 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f1af40a11e0 0x7f1af40a3690 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1af800b5c0 tx=0x7f1af8005fd0 comp rx=0 tx=0).stop 2023-07-21T18:40:56.376 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.351+0000 7f1b1061d700 1 -- 172.21.15.39:0/3605691598 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b08073160 msgr2=0x7f1b081983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:56.376 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.351+0000 7f1b1061d700 1 --2- 172.21.15.39:0/3605691598 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b08073160 0x7f1b081983b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1b0400d980 tx=0x7f1b0400dc90 comp rx=0 tx=0).stop 2023-07-21T18:40:56.376 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.351+0000 7f1b1061d700 1 -- 172.21.15.39:0/3605691598 shutdown_connections 2023-07-21T18:40:56.376 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.351+0000 7f1b1061d700 1 --2- 172.21.15.39:0/3605691598 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f1af40a11e0 0x7f1af40a3690 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:56.376 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.351+0000 7f1b1061d700 1 --2- 172.21.15.39:0/3605691598 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b0810c440 0x7f1b0819c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:56.377 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.351+0000 7f1b1061d700 1 --2- 172.21.15.39:0/3605691598 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1b08072850 0x7f1b08197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:56.377 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.351+0000 7f1b1061d700 1 --2- 172.21.15.39:0/3605691598 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b08073160 0x7f1b081983b0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:56.377 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.351+0000 7f1b1061d700 1 -- 172.21.15.39:0/3605691598 >> 172.21.15.39:0/3605691598 conn(0x7f1b080fb660 msgr2=0x7f1b0810e640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:56.377 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.351+0000 7f1b1061d700 1 -- 172.21.15.39:0/3605691598 shutdown_connections 2023-07-21T18:40:56.377 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.351+0000 7f1b1061d700 1 -- 172.21.15.39:0/3605691598 wait complete. 2023-07-21T18:40:56.390 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:56 smithi155 ceph-mon[97934]: pgmap v273: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-07-21T18:40:56.869 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2023-07-21T18:40:56.870 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T18:40:57.000 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.975+0000 7efe10a29700 1 -- 172.21.15.39:0/1259013444 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe0c0fe9d0 msgr2=0x7efe0c0fede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:57.000 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.975+0000 7efe10a29700 1 --2- 172.21.15.39:0/1259013444 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe0c0fe9d0 0x7efe0c0fede0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7efdfc009990 tx=0x7efdfc009ca0 comp rx=0 tx=0).stop 2023-07-21T18:40:57.000 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.975+0000 7efe10a29700 1 -- 172.21.15.39:0/1259013444 shutdown_connections 2023-07-21T18:40:57.000 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.975+0000 7efe10a29700 1 --2- 172.21.15.39:0/1259013444 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe0c100550 0x7efe0c106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:57.000 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.975+0000 7efe10a29700 1 --2- 172.21.15.39:0/1259013444 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe0c0fe9d0 0x7efe0c0fede0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:57.001 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.975+0000 7efe10a29700 1 --2- 172.21.15.39:0/1259013444 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe0c0ffbc0 0x7efe0c100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:57.001 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.975+0000 7efe10a29700 1 -- 172.21.15.39:0/1259013444 >> 172.21.15.39:0/1259013444 conn(0x7efe0c0f9fd0 msgr2=0x7efe0c0fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:57.001 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.975+0000 7efe10a29700 1 -- 172.21.15.39:0/1259013444 shutdown_connections 2023-07-21T18:40:57.001 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.975+0000 7efe10a29700 1 -- 172.21.15.39:0/1259013444 wait complete. 2023-07-21T18:40:57.003 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.975+0000 7efe10a29700 1 Processor -- start 2023-07-21T18:40:57.003 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.976+0000 7efe10a29700 1 -- start start 2023-07-21T18:40:57.003 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.976+0000 7efe10a29700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe0c0fe9d0 0x7efe0c102c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:57.003 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.976+0000 7efe10a29700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe0c0ffbc0 0x7efe0c101260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:57.003 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.976+0000 7efe10a29700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe0c100550 0x7efe0c1017a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:57.003 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.976+0000 7efe10a29700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe0c101f40 con 0x7efe0c0fe9d0 2023-07-21T18:40:57.003 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.976+0000 7efe10a29700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe0c1020b0 con 0x7efe0c0ffbc0 2023-07-21T18:40:57.004 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.976+0000 7efe10a29700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe0c19f910 con 0x7efe0c100550 2023-07-21T18:40:57.004 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.976+0000 7efe0a59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe0c0fe9d0 0x7efe0c102c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:57.004 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.976+0000 7efe0a59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe0c0fe9d0 0x7efe0c102c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55710/0 (socket says 172.21.15.39:55710) 2023-07-21T18:40:57.004 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.976+0000 7efe0a59c700 1 -- 172.21.15.39:0/3104069947 learned_addr learned my addr 172.21.15.39:0/3104069947 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:40:57.004 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.976+0000 7efe09d9b700 1 --2- 172.21.15.39:0/3104069947 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe0c0ffbc0 0x7efe0c101260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:57.004 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.976+0000 7efe0ad9d700 1 --2- 172.21.15.39:0/3104069947 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe0c100550 0x7efe0c1017a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:57.004 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.976+0000 7efe0a59c700 1 -- 172.21.15.39:0/3104069947 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe0c100550 msgr2=0x7efe0c1017a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:57.005 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.976+0000 7efe0a59c700 1 --2- 172.21.15.39:0/3104069947 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe0c100550 0x7efe0c1017a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:57.005 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.976+0000 7efe0a59c700 1 -- 172.21.15.39:0/3104069947 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe0c0ffbc0 msgr2=0x7efe0c101260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:57.005 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.976+0000 7efe0a59c700 1 --2- 172.21.15.39:0/3104069947 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe0c0ffbc0 0x7efe0c101260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:57.005 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.976+0000 7efe0a59c700 1 -- 172.21.15.39:0/3104069947 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdfc0095f0 con 0x7efe0c0fe9d0 2023-07-21T18:40:57.005 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.976+0000 7efe0ad9d700 1 --2- 172.21.15.39:0/3104069947 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe0c100550 0x7efe0c1017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:40:57.005 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.976+0000 7efe0a59c700 1 --2- 172.21.15.39:0/3104069947 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe0c0fe9d0 0x7efe0c102c20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7efdfc004760 tx=0x7efdfc009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:57.005 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.976+0000 7efdfb7fe700 1 -- 172.21.15.39:0/3104069947 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdfc01d070 con 0x7efe0c0fe9d0 2023-07-21T18:40:57.006 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.976+0000 7efe10a29700 1 -- 172.21.15.39:0/3104069947 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe0c19fb90 con 0x7efe0c0fe9d0 2023-07-21T18:40:57.006 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.976+0000 7efdfb7fe700 1 -- 172.21.15.39:0/3104069947 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efdfc004120 con 0x7efe0c0fe9d0 2023-07-21T18:40:57.006 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.976+0000 7efdfb7fe700 1 -- 172.21.15.39:0/3104069947 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdfc005290 con 0x7efe0c0fe9d0 2023-07-21T18:40:57.006 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.977+0000 7efe10a29700 1 -- 172.21.15.39:0/3104069947 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe0c19ffb0 con 0x7efe0c0fe9d0 2023-07-21T18:40:57.006 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.977+0000 7efe10a29700 1 -- 172.21.15.39:0/3104069947 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe0c106a60 con 0x7efe0c0fe9d0 2023-07-21T18:40:57.007 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.978+0000 7efdfb7fe700 1 -- 172.21.15.39:0/3104069947 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7efdfc004810 con 0x7efe0c0fe9d0 2023-07-21T18:40:57.007 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.979+0000 7efdfb7fe700 1 --2- 172.21.15.39:0/3104069947 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7efdf40a1220 0x7efdf40a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:57.007 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.979+0000 7efe09d9b700 1 --2- 172.21.15.39:0/3104069947 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7efdf40a1220 0x7efdf40a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:57.007 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.979+0000 7efdfb7fe700 1 -- 172.21.15.39:0/3104069947 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7efdfc0cd930 con 0x7efe0c0fe9d0 2023-07-21T18:40:57.007 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.980+0000 7efe09d9b700 1 --2- 172.21.15.39:0/3104069947 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7efdf40a1220 0x7efdf40a36d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7efe00006fd0 tx=0x7efe00008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:57.008 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:56.983+0000 7efdfb7fe700 1 -- 172.21.15.39:0/3104069947 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7efdfc097160 con 0x7efe0c0fe9d0 2023-07-21T18:40:57.189 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:57 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/3605691598' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-07-21T18:40:57.215 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.190+0000 7efe10a29700 1 -- 172.21.15.39:0/3104069947 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efe0c02ca70 con 0x7efe0c0fe9d0 2023-07-21T18:40:57.216 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.192+0000 7efdfb7fe700 1 -- 172.21.15.39:0/3104069947 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+22117 (secure 0 0 0) 0x7efdfc09a780 con 0x7efe0c0fe9d0 2023-07-21T18:40:57.217 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:40:57.217 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":79,"fsid":"eaf8e25a-27f4-11ee-9b34-001a4aab830c","created":"2023-07-21T18:33:06.504979+0000","modified":"2023-07-21T18:40:05.769300+0000","last_up_change":"2023-07-21T18:39:23.531009+0000","last_in_change":"2023-07-21T18:39:05.993024+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T18:36:13.572583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T18:40:02.726735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T18:40:03.990112+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03502c07-2d0f-4723-8d0b-d34c9f5b91bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6803","nonce":860193857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6805","nonce":860193857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6809","nonce":860193857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6807","nonce":860193857}]},"public_addr":"172.21.15.39:6803/860193857","cluster_addr":"172.21.15.39:6805/860193857","heartbeat_back_addr":"172.21.15.39:6809/860193857","heartbeat_front_addr":"172.21.15.39:6807/860193857","state":["exists","up"]},{"osd":1,"uuid":"0831a0c0-a1fa-4d30-9a28-513223aa69eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6801","nonce":3246721764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6803","nonce":3246721764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6807","nonce":3246721764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6805","nonce":3246721764}]},"public_addr":"172.21.15.155:6801/3246721764","cluster_addr":"172.21.15.155:6803/3246721764","heartbeat_back_addr":"172.21.15.155:6807/3246721764","heartbeat_front_addr":"172.21.15.155:6805/3246721764","state":["exists","up"]},{"osd":2,"uuid":"22df0003-bde9-48e4-b47a-cb8eafbab0db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6801","nonce":2563594449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6803","nonce":2563594449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6807","nonce":2563594449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6805","nonce":2563594449}]},"public_addr":"172.21.15.193:6801/2563594449","cluster_addr":"172.21.15.193:6803/2563594449","heartbeat_back_addr":"172.21.15.193:6807/2563594449","heartbeat_front_addr":"172.21.15.193:6805/2563594449","state":["exists","up"]},{"osd":3,"uuid":"54562558-a2e8-45dd-bc95-561569b9ae6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6811","nonce":863708386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6813","nonce":863708386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6817","nonce":863708386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6815","nonce":863708386}]},"public_addr":"172.21.15.39:6811/863708386","cluster_addr":"172.21.15.39:6813/863708386","heartbeat_back_addr":"172.21.15.39:6817/863708386","heartbeat_front_addr":"172.21.15.39:6815/863708386","state":["exists","up"]},{"osd":4,"uuid":"491118e7-ae1e-495c-9eac-ef1cdcae9600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6809","nonce":3361780673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6811","nonce":3361780673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6815","nonce":3361780673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6813","nonce":3361780673}]},"public_addr":"172.21.15.155:6809/3361780673","cluster_addr":"172.21.15.155:6811/3361780673","heartbeat_back_addr":"172.21.15.155:6815/3361780673","heartbeat_front_addr":"172.21.15.155:6813/3361780673","state":["exists","up"]},{"osd":5,"uuid":"d267bc2c-42b3-4d96-b2d1-b186dc7e6b18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6809","nonce":2376867721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6811","nonce":2376867721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6815","nonce":2376867721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6813","nonce":2376867721}]},"public_addr":"172.21.15.193:6809/2376867721","cluster_addr":"172.21.15.193:6811/2376867721","heartbeat_back_addr":"172.21.15.193:6815/2376867721","heartbeat_front_addr":"172.21.15.193:6813/2376867721","state":["exists","up"]},{"osd":6,"uuid":"beec8088-970a-473e-bd67-6eee7cbdef24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6819","nonce":2417770529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6821","nonce":2417770529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6825","nonce":2417770529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6823","nonce":2417770529}]},"public_addr":"172.21.15.39:6819/2417770529","cluster_addr":"172.21.15.39:6821/2417770529","heartbeat_back_addr":"172.21.15.39:6825/2417770529","heartbeat_front_addr":"172.21.15.39:6823/2417770529","state":["exists","up"]},{"osd":7,"uuid":"5a88ce30-1835-4254-90cc-804ed5cf1066","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6817","nonce":2542002111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6819","nonce":2542002111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6823","nonce":2542002111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6821","nonce":2542002111}]},"public_addr":"172.21.15.155:6817/2542002111","cluster_addr":"172.21.15.155:6819/2542002111","heartbeat_back_addr":"172.21.15.155:6823/2542002111","heartbeat_front_addr":"172.21.15.155:6821/2542002111","state":["exists","up"]},{"osd":8,"uuid":"d964a183-3021-4ace-9a0c-aad00f2570e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6817","nonce":2985835085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6819","nonce":2985835085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6823","nonce":2985835085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6821","nonce":2985835085}]},"public_addr":"172.21.15.193:6817/2985835085","cluster_addr":"172.21.15.193:6819/2985835085","heartbeat_back_addr":"172.21.15.193:6823/2985835085","heartbeat_front_addr":"172.21.15.193:6821/2985835085","state":["exists","up"]},{"osd":9,"uuid":"9725ea5c-49a3-4c99-bac4-e95069100b1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6827","nonce":1946493651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6829","nonce":1946493651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6833","nonce":1946493651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6831","nonce":1946493651}]},"public_addr":"172.21.15.39:6827/1946493651","cluster_addr":"172.21.15.39:6829/1946493651","heartbeat_back_addr":"172.21.15.39:6833/1946493651","heartbeat_front_addr":"172.21.15.39:6831/1946493651","state":["exists","up"]},{"osd":10,"uuid":"1fd60ccb-92db-45f0-a580-2003db0976e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6825","nonce":1234615022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6827","nonce":1234615022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6831","nonce":1234615022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6829","nonce":1234615022}]},"public_addr":"172.21.15.155:6825/1234615022","cluster_addr":"172.21.15.155:6827/1234615022","heartbeat_back_addr":"172.21.15.155:6831/1234615022","heartbeat_front_addr":"172.21.15.155:6829/1234615022","state":["exists","up"]},{"osd":11,"uuid":"efb31ea1-4775-4e5f-8740-728d6f4aeea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6825","nonce":215104663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6827","nonce":215104663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6831","nonce":215104663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6829","nonce":215104663}]},"public_addr":"172.21.15.193:6825/215104663","cluster_addr":"172.21.15.193:6827/215104663","heartbeat_back_addr":"172.21.15.193:6831/215104663","heartbeat_front_addr":"172.21.15.193:6829/215104663","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:35:30.211698+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:35:50.883846+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:36:11.124168+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:36:32.711905+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:36:54.442007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:37:14.843526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:37:36.587354+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:37:57.882374+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:38:18.729407+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:38:40.139231+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:39:01.290862+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:39:21.767537+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:0/2902600414":"2023-07-22T18:34:07.039823+0000","172.21.15.39:6801/2893866425":"2023-07-22T18:34:07.039823+0000","172.21.15.39:0/19717002":"2023-07-22T18:34:07.039823+0000","172.21.15.39:6800/2893866425":"2023-07-22T18:34:07.039823+0000","172.21.15.39:6800/889516631":"2023-07-22T18:33:40.852706+0000","172.21.15.39:6801/889516631":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/3636510379":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/1275748831":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/3819152602":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/1127767082":"2023-07-22T18:34:07.039823+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T18:40:57.219 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.194+0000 7efe10a29700 1 -- 172.21.15.39:0/3104069947 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7efdf40a1220 msgr2=0x7efdf40a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:57.219 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.194+0000 7efe10a29700 1 --2- 172.21.15.39:0/3104069947 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7efdf40a1220 0x7efdf40a36d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7efe00006fd0 tx=0x7efe00008040 comp rx=0 tx=0).stop 2023-07-21T18:40:57.219 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.194+0000 7efe10a29700 1 -- 172.21.15.39:0/3104069947 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe0c0fe9d0 msgr2=0x7efe0c102c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:57.219 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.194+0000 7efe10a29700 1 --2- 172.21.15.39:0/3104069947 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe0c0fe9d0 0x7efe0c102c20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7efdfc004760 tx=0x7efdfc009580 comp rx=0 tx=0).stop 2023-07-21T18:40:57.220 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.194+0000 7efe10a29700 1 -- 172.21.15.39:0/3104069947 shutdown_connections 2023-07-21T18:40:57.220 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.194+0000 7efe10a29700 1 --2- 172.21.15.39:0/3104069947 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7efdf40a1220 0x7efdf40a36d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:57.220 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.194+0000 7efe10a29700 1 --2- 172.21.15.39:0/3104069947 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe0c100550 0x7efe0c1017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:57.220 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.194+0000 7efe10a29700 1 --2- 172.21.15.39:0/3104069947 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe0c0fe9d0 0x7efe0c102c20 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:57.220 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.194+0000 7efe10a29700 1 --2- 172.21.15.39:0/3104069947 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe0c0ffbc0 0x7efe0c101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:57.220 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.194+0000 7efe10a29700 1 -- 172.21.15.39:0/3104069947 >> 172.21.15.39:0/3104069947 conn(0x7efe0c0f9fd0 msgr2=0x7efe0c0fc1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:57.221 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.194+0000 7efe10a29700 1 -- 172.21.15.39:0/3104069947 shutdown_connections 2023-07-21T18:40:57.221 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.194+0000 7efe10a29700 1 -- 172.21.15.39:0/3104069947 wait complete. 2023-07-21T18:40:57.232 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-21T18:36:13.572583+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-21T18:40:02.726735+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-21T18:40:03.990112+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-21T18:40:57.233 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-21T18:40:57.363 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.337+0000 7f3702b03700 1 -- 172.21.15.39:0/1780347651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36fc102e80 msgr2=0x7f36fc105260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:57.363 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.337+0000 7f3702b03700 1 --2- 172.21.15.39:0/1780347651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36fc102e80 0x7f36fc105260 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f36e8009990 tx=0x7f36e8009ca0 comp rx=0 tx=0).stop 2023-07-21T18:40:57.364 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.337+0000 7f3702b03700 1 -- 172.21.15.39:0/1780347651 shutdown_connections 2023-07-21T18:40:57.364 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.337+0000 7f3702b03700 1 --2- 172.21.15.39:0/1780347651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36fc100560 0x7f36fc102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:57.364 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.337+0000 7f3702b03700 1 --2- 172.21.15.39:0/1780347651 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f36fc1057a0 0x7f36fc107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:57.364 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.337+0000 7f3702b03700 1 --2- 172.21.15.39:0/1780347651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36fc102e80 0x7f36fc105260 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:57.364 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.337+0000 7f3702b03700 1 -- 172.21.15.39:0/1780347651 >> 172.21.15.39:0/1780347651 conn(0x7f36fc0fa000 msgr2=0x7f36fc0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:57.365 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.337+0000 7f3702b03700 1 -- 172.21.15.39:0/1780347651 shutdown_connections 2023-07-21T18:40:57.365 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.337+0000 7f3702b03700 1 -- 172.21.15.39:0/1780347651 wait complete. 2023-07-21T18:40:57.365 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.337+0000 7f3702b03700 1 Processor -- start 2023-07-21T18:40:57.365 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.337+0000 7f3702b03700 1 -- start start 2023-07-21T18:40:57.365 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.338+0000 7f3702b03700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36fc100560 0x7f36fc195810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:57.365 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.338+0000 7f3702b03700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36fc102e80 0x7f36fc195d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:57.366 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.338+0000 7f3702b03700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f36fc1057a0 0x7f36fc19cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:57.366 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.338+0000 7f3702b03700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36fc19d390 con 0x7f36fc100560 2023-07-21T18:40:57.366 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.338+0000 7f3702b03700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36fc19d4d0 con 0x7f36fc102e80 2023-07-21T18:40:57.366 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.338+0000 7f3702b03700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36fc19d660 con 0x7f36fc1057a0 2023-07-21T18:40:57.366 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.338+0000 7f370089f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36fc100560 0x7f36fc195810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:57.366 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.338+0000 7f370089f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36fc100560 0x7f36fc195810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55724/0 (socket says 172.21.15.39:55724) 2023-07-21T18:40:57.366 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.338+0000 7f370089f700 1 -- 172.21.15.39:0/4121866451 learned_addr learned my addr 172.21.15.39:0/4121866451 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:40:57.367 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.338+0000 7f36fbfff700 1 --2- 172.21.15.39:0/4121866451 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36fc102e80 0x7f36fc195d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:57.368 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.338+0000 7f370089f700 1 -- 172.21.15.39:0/4121866451 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f36fc1057a0 msgr2=0x7f36fc19cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:57.368 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.338+0000 7f370089f700 1 --2- 172.21.15.39:0/4121866451 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f36fc1057a0 0x7f36fc19cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:57.368 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.338+0000 7f370089f700 1 -- 172.21.15.39:0/4121866451 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36fc102e80 msgr2=0x7f36fc195d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:57.368 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.338+0000 7f370089f700 1 --2- 172.21.15.39:0/4121866451 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36fc102e80 0x7f36fc195d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:57.368 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.338+0000 7f370089f700 1 -- 172.21.15.39:0/4121866451 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e80095f0 con 0x7f36fc100560 2023-07-21T18:40:57.369 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.338+0000 7f370089f700 1 --2- 172.21.15.39:0/4121866451 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36fc100560 0x7f36fc195810 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f36f000d950 tx=0x7f36f000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:57.369 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.338+0000 7f36f9ffb700 1 -- 172.21.15.39:0/4121866451 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f0009940 con 0x7f36fc100560 2023-07-21T18:40:57.369 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.338+0000 7f36f9ffb700 1 -- 172.21.15.39:0/4121866451 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36f0010460 con 0x7f36fc100560 2023-07-21T18:40:57.369 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.338+0000 7f36f9ffb700 1 -- 172.21.15.39:0/4121866451 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f000f620 con 0x7f36fc100560 2023-07-21T18:40:57.369 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.338+0000 7f3702b03700 1 -- 172.21.15.39:0/4121866451 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36fc19d940 con 0x7f36fc100560 2023-07-21T18:40:57.369 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.339+0000 7f3702b03700 1 -- 172.21.15.39:0/4121866451 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36fc19de00 con 0x7f36fc100560 2023-07-21T18:40:57.370 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.340+0000 7f3702b03700 1 -- 172.21.15.39:0/4121866451 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36fc10b2a0 con 0x7f36fc100560 2023-07-21T18:40:57.370 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.341+0000 7f36f9ffb700 1 -- 172.21.15.39:0/4121866451 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f36f00108d0 con 0x7f36fc100560 2023-07-21T18:40:57.370 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.341+0000 7f36f9ffb700 1 --2- 172.21.15.39:0/4121866451 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f36ec0a11e0 0x7f36ec0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:57.371 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.341+0000 7f36fbfff700 1 --2- 172.21.15.39:0/4121866451 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f36ec0a11e0 0x7f36ec0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:57.371 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.341+0000 7f36f9ffb700 1 -- 172.21.15.39:0/4121866451 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f36f00cc760 con 0x7f36fc100560 2023-07-21T18:40:57.371 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.342+0000 7f36fbfff700 1 --2- 172.21.15.39:0/4121866451 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f36ec0a11e0 0x7f36ec0a3690 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f36e800b5c0 tx=0x7f36e8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:57.371 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.345+0000 7f36f9ffb700 1 -- 172.21.15.39:0/4121866451 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f36f0095f00 con 0x7f36fc100560 2023-07-21T18:40:57.371 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:57 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/3605691598' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-07-21T18:40:57.390 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:57 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/3605691598' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-07-21T18:40:57.568 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.543+0000 7f3702b03700 1 -- 172.21.15.39:0/4121866451 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f36fc19e3e0 con 0x7f36fc100560 2023-07-21T18:40:57.568 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.543+0000 7f36f9ffb700 1 -- 172.21.15.39:0/4121866451 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7f36f0099520 con 0x7f36fc100560 2023-07-21T18:40:57.569 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2023-07-21T18:40:57.570 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.545+0000 7f3702b03700 1 -- 172.21.15.39:0/4121866451 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f36ec0a11e0 msgr2=0x7f36ec0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:57.571 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.545+0000 7f3702b03700 1 --2- 172.21.15.39:0/4121866451 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f36ec0a11e0 0x7f36ec0a3690 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f36e800b5c0 tx=0x7f36e8005fd0 comp rx=0 tx=0).stop 2023-07-21T18:40:57.571 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.545+0000 7f3702b03700 1 -- 172.21.15.39:0/4121866451 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36fc100560 msgr2=0x7f36fc195810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:57.571 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.545+0000 7f3702b03700 1 --2- 172.21.15.39:0/4121866451 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36fc100560 0x7f36fc195810 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f36f000d950 tx=0x7f36f000dc60 comp rx=0 tx=0).stop 2023-07-21T18:40:57.572 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.545+0000 7f3702b03700 1 -- 172.21.15.39:0/4121866451 shutdown_connections 2023-07-21T18:40:57.572 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.545+0000 7f3702b03700 1 --2- 172.21.15.39:0/4121866451 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f36ec0a11e0 0x7f36ec0a3690 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:57.572 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.545+0000 7f3702b03700 1 --2- 172.21.15.39:0/4121866451 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f36fc1057a0 0x7f36fc19cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:57.572 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.545+0000 7f3702b03700 1 --2- 172.21.15.39:0/4121866451 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36fc100560 0x7f36fc195810 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:57.573 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.545+0000 7f3702b03700 1 --2- 172.21.15.39:0/4121866451 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36fc102e80 0x7f36fc195d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:57.573 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.545+0000 7f3702b03700 1 -- 172.21.15.39:0/4121866451 >> 172.21.15.39:0/4121866451 conn(0x7f36fc0fa000 msgr2=0x7f36fc1019c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:57.573 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.545+0000 7f3702b03700 1 -- 172.21.15.39:0/4121866451 shutdown_connections 2023-07-21T18:40:57.574 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.545+0000 7f3702b03700 1 -- 172.21.15.39:0/4121866451 wait complete. 2023-07-21T18:40:57.582 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-21T18:40:57.712 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.687+0000 7fe2aa12d700 1 -- 172.21.15.39:0/2971204804 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2a4100560 msgr2=0x7fe2a4106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:57.712 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.687+0000 7fe2aa12d700 1 --2- 172.21.15.39:0/2971204804 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2a4100560 0x7fe2a4106e20 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe29c00b0a0 tx=0x7fe29c00b3b0 comp rx=0 tx=0).stop 2023-07-21T18:40:57.712 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.687+0000 7fe2aa12d700 1 -- 172.21.15.39:0/2971204804 shutdown_connections 2023-07-21T18:40:57.713 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.687+0000 7fe2aa12d700 1 --2- 172.21.15.39:0/2971204804 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2a40fe9e0 0x7fe2a40fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:57.713 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.687+0000 7fe2aa12d700 1 --2- 172.21.15.39:0/2971204804 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2a4100560 0x7fe2a4106e20 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:57.713 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.687+0000 7fe2aa12d700 1 --2- 172.21.15.39:0/2971204804 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2a40ffbd0 0x7fe2a4100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:57.713 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.687+0000 7fe2aa12d700 1 -- 172.21.15.39:0/2971204804 >> 172.21.15.39:0/2971204804 conn(0x7fe2a40fa000 msgr2=0x7fe2a40fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:57.713 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.687+0000 7fe2aa12d700 1 -- 172.21.15.39:0/2971204804 shutdown_connections 2023-07-21T18:40:57.713 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.687+0000 7fe2aa12d700 1 -- 172.21.15.39:0/2971204804 wait complete. 2023-07-21T18:40:57.715 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.688+0000 7fe2aa12d700 1 Processor -- start 2023-07-21T18:40:57.715 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.688+0000 7fe2aa12d700 1 -- start start 2023-07-21T18:40:57.715 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.688+0000 7fe2aa12d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2a40fe9e0 0x7fe2a41958f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:57.715 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.688+0000 7fe2aa12d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2a40ffbd0 0x7fe2a4195e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:57.716 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.688+0000 7fe2aa12d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2a4100560 0x7fe2a419cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:57.716 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.688+0000 7fe2aa12d700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a419d470 con 0x7fe2a4100560 2023-07-21T18:40:57.716 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.688+0000 7fe2aa12d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a419d5b0 con 0x7fe2a40fe9e0 2023-07-21T18:40:57.716 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.688+0000 7fe2aa12d700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a419d740 con 0x7fe2a40ffbd0 2023-07-21T18:40:57.717 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.688+0000 7fe2a3fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2a4100560 0x7fe2a419cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:57.717 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.688+0000 7fe2a3fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2a4100560 0x7fe2a419cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55742/0 (socket says 172.21.15.39:55742) 2023-07-21T18:40:57.717 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.688+0000 7fe2a3fff700 1 -- 172.21.15.39:0/1932242907 learned_addr learned my addr 172.21.15.39:0/1932242907 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:40:57.717 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.688+0000 7fe2a37fe700 1 --2- 172.21.15.39:0/1932242907 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2a40fe9e0 0x7fe2a41958f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:57.718 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.688+0000 7fe2a2ffd700 1 --2- 172.21.15.39:0/1932242907 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2a40ffbd0 0x7fe2a4195e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:57.718 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.688+0000 7fe2a3fff700 1 -- 172.21.15.39:0/1932242907 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2a40ffbd0 msgr2=0x7fe2a4195e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:57.718 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.688+0000 7fe2a3fff700 1 --2- 172.21.15.39:0/1932242907 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2a40ffbd0 0x7fe2a4195e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:57.718 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.688+0000 7fe2a3fff700 1 -- 172.21.15.39:0/1932242907 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2a40fe9e0 msgr2=0x7fe2a41958f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:57.719 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.688+0000 7fe2a3fff700 1 --2- 172.21.15.39:0/1932242907 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2a40fe9e0 0x7fe2a41958f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:57.719 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.688+0000 7fe2a3fff700 1 -- 172.21.15.39:0/1932242907 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe29c009d00 con 0x7fe2a4100560 2023-07-21T18:40:57.719 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.689+0000 7fe2a3fff700 1 --2- 172.21.15.39:0/1932242907 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2a4100560 0x7fe2a419cea0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe29c00d010 tx=0x7fe29c004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:57.719 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.689+0000 7fe2a0ff9700 1 -- 172.21.15.39:0/1932242907 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe29c00e040 con 0x7fe2a4100560 2023-07-21T18:40:57.719 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.689+0000 7fe2a0ff9700 1 -- 172.21.15.39:0/1932242907 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe29c009440 con 0x7fe2a4100560 2023-07-21T18:40:57.719 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.689+0000 7fe2aa12d700 1 -- 172.21.15.39:0/1932242907 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2a419d9c0 con 0x7fe2a4100560 2023-07-21T18:40:57.719 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.689+0000 7fe2a0ff9700 1 -- 172.21.15.39:0/1932242907 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe29c004020 con 0x7fe2a4100560 2023-07-21T18:40:57.720 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.689+0000 7fe2aa12d700 1 -- 172.21.15.39:0/1932242907 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2a419de40 con 0x7fe2a4100560 2023-07-21T18:40:57.720 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.690+0000 7fe2aa12d700 1 -- 172.21.15.39:0/1932242907 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2a4105810 con 0x7fe2a4100560 2023-07-21T18:40:57.720 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.691+0000 7fe2a0ff9700 1 -- 172.21.15.39:0/1932242907 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fe29c01d690 con 0x7fe2a4100560 2023-07-21T18:40:57.721 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.691+0000 7fe2a0ff9700 1 --2- 172.21.15.39:0/1932242907 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fe28c0a11e0 0x7fe28c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:57.721 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.692+0000 7fe2a37fe700 1 --2- 172.21.15.39:0/1932242907 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fe28c0a11e0 0x7fe28c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:57.721 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.692+0000 7fe2a0ff9700 1 -- 172.21.15.39:0/1932242907 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe29c0ce200 con 0x7fe2a4100560 2023-07-21T18:40:57.721 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.692+0000 7fe2a37fe700 1 --2- 172.21.15.39:0/1932242907 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fe28c0a11e0 0x7fe28c0a3690 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe2a40ffa30 tx=0x7fe29400a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:57.721 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.696+0000 7fe2a0ff9700 1 -- 172.21.15.39:0/1932242907 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fe29c097930 con 0x7fe2a4100560 2023-07-21T18:40:57.919 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.894+0000 7fe2aa12d700 1 -- 172.21.15.39:0/1932242907 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe2a402ca60 con 0x7fe2a4100560 2023-07-21T18:40:57.919 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.894+0000 7fe2a0ff9700 1 -- 172.21.15.39:0/1932242907 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v79) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe29c09af50 con 0x7fe2a4100560 2023-07-21T18:40:57.919 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2023-07-21T18:40:57.921 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.896+0000 7fe2aa12d700 1 -- 172.21.15.39:0/1932242907 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fe28c0a11e0 msgr2=0x7fe28c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:57.921 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.896+0000 7fe2aa12d700 1 --2- 172.21.15.39:0/1932242907 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fe28c0a11e0 0x7fe28c0a3690 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe2a40ffa30 tx=0x7fe29400a3b0 comp rx=0 tx=0).stop 2023-07-21T18:40:57.922 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.896+0000 7fe2aa12d700 1 -- 172.21.15.39:0/1932242907 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2a4100560 msgr2=0x7fe2a419cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:57.922 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.896+0000 7fe2aa12d700 1 --2- 172.21.15.39:0/1932242907 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2a4100560 0x7fe2a419cea0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe29c00d010 tx=0x7fe29c004c80 comp rx=0 tx=0).stop 2023-07-21T18:40:57.922 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.896+0000 7fe2aa12d700 1 -- 172.21.15.39:0/1932242907 shutdown_connections 2023-07-21T18:40:57.922 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.896+0000 7fe2aa12d700 1 --2- 172.21.15.39:0/1932242907 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fe28c0a11e0 0x7fe28c0a3690 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:57.922 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.896+0000 7fe2aa12d700 1 --2- 172.21.15.39:0/1932242907 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2a40fe9e0 0x7fe2a41958f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:57.922 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.896+0000 7fe2aa12d700 1 --2- 172.21.15.39:0/1932242907 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2a4100560 0x7fe2a419cea0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:57.922 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.896+0000 7fe2aa12d700 1 --2- 172.21.15.39:0/1932242907 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2a40ffbd0 0x7fe2a4195e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:57.922 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.896+0000 7fe2aa12d700 1 -- 172.21.15.39:0/1932242907 >> 172.21.15.39:0/1932242907 conn(0x7fe2a40fa000 msgr2=0x7fe2a40fc1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:57.923 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.897+0000 7fe2aa12d700 1 -- 172.21.15.39:0/1932242907 shutdown_connections 2023-07-21T18:40:57.923 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:57.897+0000 7fe2aa12d700 1 -- 172.21.15.39:0/1932242907 wait complete. 2023-07-21T18:40:57.934 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-21T18:40:58.063 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.038+0000 7f8e35136700 1 -- 172.21.15.39:0/942252053 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e30102e80 msgr2=0x7f8e30105260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:58.064 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.038+0000 7f8e35136700 1 --2- 172.21.15.39:0/942252053 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e30102e80 0x7f8e30105260 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8e24009990 tx=0x7f8e24009ca0 comp rx=0 tx=0).stop 2023-07-21T18:40:58.064 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.038+0000 7f8e35136700 1 -- 172.21.15.39:0/942252053 shutdown_connections 2023-07-21T18:40:58.064 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.038+0000 7f8e35136700 1 --2- 172.21.15.39:0/942252053 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e30100560 0x7f8e30102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:58.064 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.038+0000 7f8e35136700 1 --2- 172.21.15.39:0/942252053 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e301057a0 0x7f8e30107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:58.064 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.038+0000 7f8e35136700 1 --2- 172.21.15.39:0/942252053 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e30102e80 0x7f8e30105260 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:58.065 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.038+0000 7f8e35136700 1 -- 172.21.15.39:0/942252053 >> 172.21.15.39:0/942252053 conn(0x7f8e300fa000 msgr2=0x7f8e300fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:58.065 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.038+0000 7f8e35136700 1 -- 172.21.15.39:0/942252053 shutdown_connections 2023-07-21T18:40:58.065 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.038+0000 7f8e35136700 1 -- 172.21.15.39:0/942252053 wait complete. 2023-07-21T18:40:58.065 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.039+0000 7f8e35136700 1 Processor -- start 2023-07-21T18:40:58.067 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.039+0000 7f8e35136700 1 -- start start 2023-07-21T18:40:58.067 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.039+0000 7f8e35136700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e30100560 0x7f8e30195860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:58.067 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.039+0000 7f8e35136700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e30102e80 0x7f8e30195da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:58.067 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.039+0000 7f8e35136700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e301057a0 0x7f8e3019ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:58.067 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.039+0000 7f8e35136700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3019d3e0 con 0x7f8e301057a0 2023-07-21T18:40:58.067 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.039+0000 7f8e35136700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3019d520 con 0x7f8e30100560 2023-07-21T18:40:58.067 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.039+0000 7f8e35136700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3019d6b0 con 0x7f8e30102e80 2023-07-21T18:40:58.068 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.040+0000 7f8e2f59e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e301057a0 0x7f8e3019ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:58.068 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.040+0000 7f8e2f59e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e301057a0 0x7f8e3019ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55756/0 (socket says 172.21.15.39:55756) 2023-07-21T18:40:58.068 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.040+0000 7f8e2f59e700 1 -- 172.21.15.39:0/2485111859 learned_addr learned my addr 172.21.15.39:0/2485111859 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:40:58.068 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.040+0000 7f8e2e59c700 1 --2- 172.21.15.39:0/2485111859 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e30102e80 0x7f8e30195da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:58.068 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.040+0000 7f8e2f59e700 1 -- 172.21.15.39:0/2485111859 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e30102e80 msgr2=0x7f8e30195da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:58.068 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.040+0000 7f8e2f59e700 1 --2- 172.21.15.39:0/2485111859 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e30102e80 0x7f8e30195da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:58.069 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.040+0000 7f8e2f59e700 1 -- 172.21.15.39:0/2485111859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e30100560 msgr2=0x7f8e30195860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:58.069 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.040+0000 7f8e2f59e700 1 --2- 172.21.15.39:0/2485111859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e30100560 0x7f8e30195860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:58.069 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.040+0000 7f8e2f59e700 1 -- 172.21.15.39:0/2485111859 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e240095f0 con 0x7f8e301057a0 2023-07-21T18:40:58.069 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.040+0000 7f8e2e59c700 1 --2- 172.21.15.39:0/2485111859 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e30102e80 0x7f8e30195da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:40:58.069 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.040+0000 7f8e2f59e700 1 --2- 172.21.15.39:0/2485111859 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e301057a0 0x7f8e3019ce10 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8e2800ef30 tx=0x7f8e2800c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:58.069 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.040+0000 7f8e17fff700 1 -- 172.21.15.39:0/2485111859 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e28018070 con 0x7f8e301057a0 2023-07-21T18:40:58.069 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.040+0000 7f8e17fff700 1 -- 172.21.15.39:0/2485111859 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e2800f040 con 0x7f8e301057a0 2023-07-21T18:40:58.069 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.040+0000 7f8e35136700 1 -- 172.21.15.39:0/2485111859 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e3019d930 con 0x7f8e301057a0 2023-07-21T18:40:58.070 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.040+0000 7f8e17fff700 1 -- 172.21.15.39:0/2485111859 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e28003dc0 con 0x7f8e301057a0 2023-07-21T18:40:58.070 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.040+0000 7f8e35136700 1 -- 172.21.15.39:0/2485111859 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e3019ddf0 con 0x7f8e301057a0 2023-07-21T18:40:58.070 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.041+0000 7f8e35136700 1 -- 172.21.15.39:0/2485111859 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e3010b2f0 con 0x7f8e301057a0 2023-07-21T18:40:58.070 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.042+0000 7f8e17fff700 1 -- 172.21.15.39:0/2485111859 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f8e2800caa0 con 0x7f8e301057a0 2023-07-21T18:40:58.071 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.043+0000 7f8e17fff700 1 --2- 172.21.15.39:0/2485111859 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f8e180a11e0 0x7f8e180a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:58.071 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.043+0000 7f8e2ed9d700 1 --2- 172.21.15.39:0/2485111859 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f8e180a11e0 0x7f8e180a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:58.071 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.043+0000 7f8e17fff700 1 -- 172.21.15.39:0/2485111859 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f8e280cc5c0 con 0x7f8e301057a0 2023-07-21T18:40:58.071 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.044+0000 7f8e2ed9d700 1 --2- 172.21.15.39:0/2485111859 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f8e180a11e0 0x7f8e180a3690 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8e20005950 tx=0x7f8e200058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:58.072 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.048+0000 7f8e17fff700 1 -- 172.21.15.39:0/2485111859 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f8e28095df0 con 0x7f8e301057a0 2023-07-21T18:40:58.189 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:58 smithi039 ceph-mon[96489]: pgmap v274: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-07-21T18:40:58.189 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:58 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/3104069947' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T18:40:58.189 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:58 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/4121866451' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T18:40:58.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:58 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/1932242907' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T18:40:58.273 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.248+0000 7f8e35136700 1 -- 172.21.15.39:0/2485111859 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8e3019e430 con 0x7f8e301057a0 2023-07-21T18:40:58.273 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.248+0000 7f8e17fff700 1 -- 172.21.15.39:0/2485111859 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v79) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8e28099410 con 0x7f8e301057a0 2023-07-21T18:40:58.274 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 64 2023-07-21T18:40:58.275 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.250+0000 7f8e35136700 1 -- 172.21.15.39:0/2485111859 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f8e180a11e0 msgr2=0x7f8e180a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:58.275 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.250+0000 7f8e35136700 1 --2- 172.21.15.39:0/2485111859 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f8e180a11e0 0x7f8e180a3690 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8e20005950 tx=0x7f8e200058e0 comp rx=0 tx=0).stop 2023-07-21T18:40:58.276 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.250+0000 7f8e35136700 1 -- 172.21.15.39:0/2485111859 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e301057a0 msgr2=0x7f8e3019ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:58.276 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.250+0000 7f8e35136700 1 --2- 172.21.15.39:0/2485111859 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e301057a0 0x7f8e3019ce10 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8e2800ef30 tx=0x7f8e2800c450 comp rx=0 tx=0).stop 2023-07-21T18:40:58.276 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.250+0000 7f8e35136700 1 -- 172.21.15.39:0/2485111859 shutdown_connections 2023-07-21T18:40:58.276 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.250+0000 7f8e35136700 1 --2- 172.21.15.39:0/2485111859 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f8e180a11e0 0x7f8e180a3690 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:58.276 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.250+0000 7f8e35136700 1 --2- 172.21.15.39:0/2485111859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e30100560 0x7f8e30195860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:58.276 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.250+0000 7f8e35136700 1 --2- 172.21.15.39:0/2485111859 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e301057a0 0x7f8e3019ce10 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:58.276 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.251+0000 7f8e35136700 1 --2- 172.21.15.39:0/2485111859 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e30102e80 0x7f8e30195da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:58.277 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.251+0000 7f8e35136700 1 -- 172.21.15.39:0/2485111859 >> 172.21.15.39:0/2485111859 conn(0x7f8e300fa000 msgr2=0x7f8e301019c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:58.277 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.251+0000 7f8e35136700 1 -- 172.21.15.39:0/2485111859 shutdown_connections 2023-07-21T18:40:58.277 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.251+0000 7f8e35136700 1 -- 172.21.15.39:0/2485111859 wait complete. 2023-07-21T18:40:58.288 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T18:40:58.371 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:58 smithi193 ceph-mon[98204]: pgmap v274: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-07-21T18:40:58.372 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:58 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/3104069947' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T18:40:58.372 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:58 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/4121866451' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T18:40:58.372 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:58 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/1932242907' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T18:40:58.390 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:58 smithi155 ceph-mon[97934]: pgmap v274: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-07-21T18:40:58.391 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:58 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/3104069947' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T18:40:58.391 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:58 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/4121866451' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T18:40:58.391 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:58 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/1932242907' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T18:40:58.413 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.388+0000 7f9e745fb700 1 -- 172.21.15.39:0/975974728 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e6c100550 msgr2=0x7f9e6c106e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:58.413 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.388+0000 7f9e745fb700 1 --2- 172.21.15.39:0/975974728 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e6c100550 0x7f9e6c106e10 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9e6400b0a0 tx=0x7f9e6400b3b0 comp rx=0 tx=0).stop 2023-07-21T18:40:58.414 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.388+0000 7f9e745fb700 1 -- 172.21.15.39:0/975974728 shutdown_connections 2023-07-21T18:40:58.414 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.388+0000 7f9e745fb700 1 --2- 172.21.15.39:0/975974728 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9e6c0fe9d0 0x7f9e6c0fede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:58.414 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.388+0000 7f9e745fb700 1 --2- 172.21.15.39:0/975974728 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e6c100550 0x7f9e6c106e10 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:58.414 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.388+0000 7f9e745fb700 1 --2- 172.21.15.39:0/975974728 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e6c0ffbc0 0x7f9e6c100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:58.414 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.388+0000 7f9e745fb700 1 -- 172.21.15.39:0/975974728 >> 172.21.15.39:0/975974728 conn(0x7f9e6c0f9fd0 msgr2=0x7f9e6c0fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:58.414 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.388+0000 7f9e745fb700 1 -- 172.21.15.39:0/975974728 shutdown_connections 2023-07-21T18:40:58.414 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.388+0000 7f9e745fb700 1 -- 172.21.15.39:0/975974728 wait complete. 2023-07-21T18:40:58.416 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.389+0000 7f9e745fb700 1 Processor -- start 2023-07-21T18:40:58.416 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.389+0000 7f9e745fb700 1 -- start start 2023-07-21T18:40:58.416 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.390+0000 7f9e745fb700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e6c0fe9d0 0x7f9e6c194820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:58.416 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.390+0000 7f9e745fb700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9e6c0ffbc0 0x7f9e6c194d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:58.417 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.390+0000 7f9e745fb700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e6c100550 0x7f9e6c19bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:58.417 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.390+0000 7f9e745fb700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6c19c3a0 con 0x7f9e6c0fe9d0 2023-07-21T18:40:58.417 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.390+0000 7f9e745fb700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6c19c4e0 con 0x7f9e6c0ffbc0 2023-07-21T18:40:58.417 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.390+0000 7f9e745fb700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6c19c670 con 0x7f9e6c100550 2023-07-21T18:40:58.417 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.390+0000 7f9e72397700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e6c0fe9d0 0x7f9e6c194820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:58.417 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.390+0000 7f9e72397700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e6c0fe9d0 0x7f9e6c194820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55766/0 (socket says 172.21.15.39:55766) 2023-07-21T18:40:58.417 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.390+0000 7f9e72397700 1 -- 172.21.15.39:0/533857347 learned_addr learned my addr 172.21.15.39:0/533857347 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:40:58.418 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.390+0000 7f9e71b96700 1 --2- 172.21.15.39:0/533857347 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9e6c0ffbc0 0x7f9e6c194d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:58.418 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.390+0000 7f9e72397700 1 -- 172.21.15.39:0/533857347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e6c100550 msgr2=0x7f9e6c19bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:58.418 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.390+0000 7f9e72397700 1 --2- 172.21.15.39:0/533857347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e6c100550 0x7f9e6c19bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:58.418 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.390+0000 7f9e72397700 1 -- 172.21.15.39:0/533857347 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9e6c0ffbc0 msgr2=0x7f9e6c194d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:58.418 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.390+0000 7f9e72397700 1 --2- 172.21.15.39:0/533857347 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9e6c0ffbc0 0x7f9e6c194d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:58.418 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.390+0000 7f9e72397700 1 -- 172.21.15.39:0/533857347 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e64009d00 con 0x7f9e6c0fe9d0 2023-07-21T18:40:58.418 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.390+0000 7f9e72397700 1 --2- 172.21.15.39:0/533857347 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e6c0fe9d0 0x7f9e6c194820 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f9e6800d980 tx=0x7f9e6800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:58.418 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.390+0000 7f9e5f7fe700 1 -- 172.21.15.39:0/533857347 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e6800de60 con 0x7f9e6c0fe9d0 2023-07-21T18:40:58.419 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.390+0000 7f9e5f7fe700 1 -- 172.21.15.39:0/533857347 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e680044e0 con 0x7f9e6c0fe9d0 2023-07-21T18:40:58.419 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.390+0000 7f9e745fb700 1 -- 172.21.15.39:0/533857347 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e6c19c950 con 0x7f9e6c0fe9d0 2023-07-21T18:40:58.419 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.390+0000 7f9e5f7fe700 1 -- 172.21.15.39:0/533857347 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e68003be0 con 0x7f9e6c0fe9d0 2023-07-21T18:40:58.419 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.390+0000 7f9e745fb700 1 -- 172.21.15.39:0/533857347 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e6c19ceb0 con 0x7f9e6c0fe9d0 2023-07-21T18:40:58.419 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.391+0000 7f9e745fb700 1 -- 172.21.15.39:0/533857347 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e6c105ec0 con 0x7f9e6c0fe9d0 2023-07-21T18:40:58.420 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.392+0000 7f9e5f7fe700 1 -- 172.21.15.39:0/533857347 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f9e68009bb0 con 0x7f9e6c0fe9d0 2023-07-21T18:40:58.420 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.393+0000 7f9e5f7fe700 1 --2- 172.21.15.39:0/533857347 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f9e580a12a0 0x7f9e580a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:58.420 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.393+0000 7f9e71b96700 1 --2- 172.21.15.39:0/533857347 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f9e580a12a0 0x7f9e580a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:58.420 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.393+0000 7f9e5f7fe700 1 -- 172.21.15.39:0/533857347 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f9e680cc620 con 0x7f9e6c0fe9d0 2023-07-21T18:40:58.420 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.393+0000 7f9e71b96700 1 --2- 172.21.15.39:0/533857347 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f9e580a12a0 0x7f9e580a3750 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9e60005950 tx=0x7f9e600058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:58.422 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.397+0000 7f9e5f7fe700 1 -- 172.21.15.39:0/533857347 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f9e68095dc0 con 0x7f9e6c0fe9d0 2023-07-21T18:40:58.666 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.642+0000 7f9e745fb700 1 -- 172.21.15.39:0/533857347 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9e6c02c850 con 0x7f9e6c0fe9d0 2023-07-21T18:40:58.667 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.642+0000 7f9e5f7fe700 1 -- 172.21.15.39:0/533857347 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10887 (secure 0 0 0) 0x7f9e68009940 con 0x7f9e6c0fe9d0 2023-07-21T18:40:58.667 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:40:58.667 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/147318681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":147318681},{"type":"v1","addr":"172.21.15.39:6835","nonce":147318681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/1062891760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1062891760},{"type":"v1","addr":"172.21.15.39:6837","nonce":1062891760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T18:40:04.594553+0000","modified":"2023-07-21T18:40:11.989039+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24536,"mds_1":24500,"mds_2":24487,"mds_3":24464,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/3184273349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":3184273349},{"type":"v1","addr":"172.21.15.39:6839","nonce":3184273349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6841/640981220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":640981220},{"type":"v1","addr":"172.21.15.39:6841","nonce":640981220}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3106166804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3106166804},{"type":"v1","addr":"172.21.15.193:6833","nonce":3106166804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24437":{"gid":24437,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/1034755277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1034755277},{"type":"v1","addr":"172.21.15.155:6833","nonce":1034755277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/381560657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":381560657},{"type":"v1","addr":"172.21.15.193:6835","nonce":381560657}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6837/2626761843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2626761843},{"type":"v1","addr":"172.21.15.193:6837","nonce":2626761843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24464":{"gid":24464,"name":"e","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.155:6835/3854856286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3854856286},{"type":"v1","addr":"172.21.15.155:6835","nonce":3854856286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.193:6839/3148488639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3148488639},{"type":"v1","addr":"172.21.15.193:6839","nonce":3148488639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.155:6837/947670056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":947670056},{"type":"v1","addr":"172.21.15.155:6837","nonce":947670056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6839/431690342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":431690342},{"type":"v1","addr":"172.21.15.155:6839","nonce":431690342}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T18:40:58.669 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.644+0000 7f9e745fb700 1 -- 172.21.15.39:0/533857347 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f9e580a12a0 msgr2=0x7f9e580a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:58.669 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.644+0000 7f9e745fb700 1 --2- 172.21.15.39:0/533857347 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f9e580a12a0 0x7f9e580a3750 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9e60005950 tx=0x7f9e600058e0 comp rx=0 tx=0).stop 2023-07-21T18:40:58.669 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.644+0000 7f9e745fb700 1 -- 172.21.15.39:0/533857347 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e6c0fe9d0 msgr2=0x7f9e6c194820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:58.669 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.644+0000 7f9e745fb700 1 --2- 172.21.15.39:0/533857347 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e6c0fe9d0 0x7f9e6c194820 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f9e6800d980 tx=0x7f9e6800dd40 comp rx=0 tx=0).stop 2023-07-21T18:40:58.670 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.644+0000 7f9e745fb700 1 -- 172.21.15.39:0/533857347 shutdown_connections 2023-07-21T18:40:58.670 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.644+0000 7f9e745fb700 1 --2- 172.21.15.39:0/533857347 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f9e580a12a0 0x7f9e580a3750 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:58.670 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.644+0000 7f9e745fb700 1 --2- 172.21.15.39:0/533857347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e6c100550 0x7f9e6c19bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:58.670 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.644+0000 7f9e745fb700 1 --2- 172.21.15.39:0/533857347 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e6c0fe9d0 0x7f9e6c194820 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:58.670 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.644+0000 7f9e745fb700 1 --2- 172.21.15.39:0/533857347 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9e6c0ffbc0 0x7f9e6c194d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:58.670 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.644+0000 7f9e745fb700 1 -- 172.21.15.39:0/533857347 >> 172.21.15.39:0/533857347 conn(0x7f9e6c0f9fd0 msgr2=0x7f9e6c103000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:58.670 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.644+0000 7f9e745fb700 1 -- 172.21.15.39:0/533857347 shutdown_connections 2023-07-21T18:40:58.670 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.644+0000 7f9e745fb700 1 -- 172.21.15.39:0/533857347 wait complete. 2023-07-21T18:40:58.671 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 26 2023-07-21T18:40:58.682 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T18:40:58.808 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.782+0000 7f888bde7700 1 -- 172.21.15.39:0/2798740202 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88840fea00 msgr2=0x7f88840fee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:58.808 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.782+0000 7f888bde7700 1 --2- 172.21.15.39:0/2798740202 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88840fea00 0x7f88840fee10 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8880009990 tx=0x7f8880009ca0 comp rx=0 tx=0).stop 2023-07-21T18:40:58.808 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.783+0000 7f888bde7700 1 -- 172.21.15.39:0/2798740202 shutdown_connections 2023-07-21T18:40:58.808 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.783+0000 7f888bde7700 1 --2- 172.21.15.39:0/2798740202 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8884100580 0x7f8884106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:58.809 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.783+0000 7f888bde7700 1 --2- 172.21.15.39:0/2798740202 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88840fea00 0x7f88840fee10 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:58.809 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.783+0000 7f888bde7700 1 --2- 172.21.15.39:0/2798740202 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f88840ffbf0 0x7f8884100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:58.809 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.783+0000 7f888bde7700 1 -- 172.21.15.39:0/2798740202 >> 172.21.15.39:0/2798740202 conn(0x7f88840f9fe0 msgr2=0x7f88840fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:58.809 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.783+0000 7f888bde7700 1 -- 172.21.15.39:0/2798740202 shutdown_connections 2023-07-21T18:40:58.809 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.783+0000 7f888bde7700 1 -- 172.21.15.39:0/2798740202 wait complete. 2023-07-21T18:40:58.811 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.783+0000 7f888bde7700 1 Processor -- start 2023-07-21T18:40:58.811 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.783+0000 7f888bde7700 1 -- start start 2023-07-21T18:40:58.811 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.784+0000 7f888bde7700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f88840fea00 0x7f88841946e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:58.811 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.784+0000 7f888bde7700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88840ffbf0 0x7f8884194c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:58.811 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.784+0000 7f888bde7700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8884100580 0x7f888419bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:58.811 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.784+0000 7f888bde7700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f888419c260 con 0x7f8884100580 2023-07-21T18:40:58.812 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.784+0000 7f888bde7700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f888419c3a0 con 0x7f88840ffbf0 2023-07-21T18:40:58.812 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.784+0000 7f888bde7700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f888419c530 con 0x7f88840fea00 2023-07-21T18:40:58.812 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.784+0000 7f888a384700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8884100580 0x7f888419bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:58.812 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.784+0000 7f888a384700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8884100580 0x7f888419bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55784/0 (socket says 172.21.15.39:55784) 2023-07-21T18:40:58.812 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.784+0000 7f888a384700 1 -- 172.21.15.39:0/59184222 learned_addr learned my addr 172.21.15.39:0/59184222 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:40:58.812 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.784+0000 7f8889b83700 1 --2- 172.21.15.39:0/59184222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f88840fea00 0x7f88841946e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:58.812 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.784+0000 7f8889382700 1 --2- 172.21.15.39:0/59184222 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88840ffbf0 0x7f8884194c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:58.812 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.784+0000 7f888a384700 1 -- 172.21.15.39:0/59184222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f88840fea00 msgr2=0x7f88841946e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:58.813 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.784+0000 7f888a384700 1 --2- 172.21.15.39:0/59184222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f88840fea00 0x7f88841946e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:58.813 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.784+0000 7f888a384700 1 -- 172.21.15.39:0/59184222 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88840ffbf0 msgr2=0x7f8884194c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:58.813 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.784+0000 7f888a384700 1 --2- 172.21.15.39:0/59184222 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88840ffbf0 0x7f8884194c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:58.813 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.784+0000 7f888a384700 1 -- 172.21.15.39:0/59184222 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88800095f0 con 0x7f8884100580 2023-07-21T18:40:58.813 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.784+0000 7f888a384700 1 --2- 172.21.15.39:0/59184222 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8884100580 0x7f888419bc90 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f887c007ee0 tx=0x7f887c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:58.813 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.784+0000 7f887affd700 1 -- 172.21.15.39:0/59184222 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f887c017070 con 0x7f8884100580 2023-07-21T18:40:58.813 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.784+0000 7f887affd700 1 -- 172.21.15.39:0/59184222 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f887c00f040 con 0x7f8884100580 2023-07-21T18:40:58.814 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.784+0000 7f888bde7700 1 -- 172.21.15.39:0/59184222 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f888419c810 con 0x7f8884100580 2023-07-21T18:40:58.814 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.784+0000 7f887affd700 1 -- 172.21.15.39:0/59184222 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f887c005740 con 0x7f8884100580 2023-07-21T18:40:58.814 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.785+0000 7f888bde7700 1 -- 172.21.15.39:0/59184222 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f888419ccd0 con 0x7f8884100580 2023-07-21T18:40:58.814 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.786+0000 7f888bde7700 1 -- 172.21.15.39:0/59184222 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8884105f70 con 0x7f8884100580 2023-07-21T18:40:58.815 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.787+0000 7f887affd700 1 -- 172.21.15.39:0/59184222 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f887c00dc60 con 0x7f8884100580 2023-07-21T18:40:58.815 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.787+0000 7f887affd700 1 --2- 172.21.15.39:0/59184222 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f88700a1220 0x7f88700a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:58.815 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.787+0000 7f8889b83700 1 --2- 172.21.15.39:0/59184222 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f88700a1220 0x7f88700a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:58.815 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.787+0000 7f887affd700 1 -- 172.21.15.39:0/59184222 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f887c0cc740 con 0x7f8884100580 2023-07-21T18:40:58.815 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.788+0000 7f8889b83700 1 --2- 172.21.15.39:0/59184222 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f88700a1220 0x7f88700a36d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8880006010 tx=0x7f8880005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:58.815 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.790+0000 7f887affd700 1 -- 172.21.15.39:0/59184222 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f887c095ee0 con 0x7f8884100580 2023-07-21T18:40:59.018 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.994+0000 7f888bde7700 1 -- 172.21.15.39:0/59184222 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f888402caa0 con 0x7f8884100580 2023-07-21T18:40:59.020 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.995+0000 7f887affd700 1 -- 172.21.15.39:0/59184222 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+22117 (secure 0 0 0) 0x7f887c099500 con 0x7f8884100580 2023-07-21T18:40:59.020 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:40:59.021 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":79,"fsid":"eaf8e25a-27f4-11ee-9b34-001a4aab830c","created":"2023-07-21T18:33:06.504979+0000","modified":"2023-07-21T18:40:05.769300+0000","last_up_change":"2023-07-21T18:39:23.531009+0000","last_in_change":"2023-07-21T18:39:05.993024+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T18:36:13.572583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T18:40:02.726735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T18:40:03.990112+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03502c07-2d0f-4723-8d0b-d34c9f5b91bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6803","nonce":860193857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6805","nonce":860193857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6809","nonce":860193857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6807","nonce":860193857}]},"public_addr":"172.21.15.39:6803/860193857","cluster_addr":"172.21.15.39:6805/860193857","heartbeat_back_addr":"172.21.15.39:6809/860193857","heartbeat_front_addr":"172.21.15.39:6807/860193857","state":["exists","up"]},{"osd":1,"uuid":"0831a0c0-a1fa-4d30-9a28-513223aa69eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6801","nonce":3246721764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6803","nonce":3246721764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6807","nonce":3246721764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6805","nonce":3246721764}]},"public_addr":"172.21.15.155:6801/3246721764","cluster_addr":"172.21.15.155:6803/3246721764","heartbeat_back_addr":"172.21.15.155:6807/3246721764","heartbeat_front_addr":"172.21.15.155:6805/3246721764","state":["exists","up"]},{"osd":2,"uuid":"22df0003-bde9-48e4-b47a-cb8eafbab0db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6801","nonce":2563594449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6803","nonce":2563594449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6807","nonce":2563594449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6805","nonce":2563594449}]},"public_addr":"172.21.15.193:6801/2563594449","cluster_addr":"172.21.15.193:6803/2563594449","heartbeat_back_addr":"172.21.15.193:6807/2563594449","heartbeat_front_addr":"172.21.15.193:6805/2563594449","state":["exists","up"]},{"osd":3,"uuid":"54562558-a2e8-45dd-bc95-561569b9ae6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6811","nonce":863708386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6813","nonce":863708386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6817","nonce":863708386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6815","nonce":863708386}]},"public_addr":"172.21.15.39:6811/863708386","cluster_addr":"172.21.15.39:6813/863708386","heartbeat_back_addr":"172.21.15.39:6817/863708386","heartbeat_front_addr":"172.21.15.39:6815/863708386","state":["exists","up"]},{"osd":4,"uuid":"491118e7-ae1e-495c-9eac-ef1cdcae9600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6809","nonce":3361780673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6811","nonce":3361780673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6815","nonce":3361780673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6813","nonce":3361780673}]},"public_addr":"172.21.15.155:6809/3361780673","cluster_addr":"172.21.15.155:6811/3361780673","heartbeat_back_addr":"172.21.15.155:6815/3361780673","heartbeat_front_addr":"172.21.15.155:6813/3361780673","state":["exists","up"]},{"osd":5,"uuid":"d267bc2c-42b3-4d96-b2d1-b186dc7e6b18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6809","nonce":2376867721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6811","nonce":2376867721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6815","nonce":2376867721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6813","nonce":2376867721}]},"public_addr":"172.21.15.193:6809/2376867721","cluster_addr":"172.21.15.193:6811/2376867721","heartbeat_back_addr":"172.21.15.193:6815/2376867721","heartbeat_front_addr":"172.21.15.193:6813/2376867721","state":["exists","up"]},{"osd":6,"uuid":"beec8088-970a-473e-bd67-6eee7cbdef24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6819","nonce":2417770529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6821","nonce":2417770529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6825","nonce":2417770529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6823","nonce":2417770529}]},"public_addr":"172.21.15.39:6819/2417770529","cluster_addr":"172.21.15.39:6821/2417770529","heartbeat_back_addr":"172.21.15.39:6825/2417770529","heartbeat_front_addr":"172.21.15.39:6823/2417770529","state":["exists","up"]},{"osd":7,"uuid":"5a88ce30-1835-4254-90cc-804ed5cf1066","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6817","nonce":2542002111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6819","nonce":2542002111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6823","nonce":2542002111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6821","nonce":2542002111}]},"public_addr":"172.21.15.155:6817/2542002111","cluster_addr":"172.21.15.155:6819/2542002111","heartbeat_back_addr":"172.21.15.155:6823/2542002111","heartbeat_front_addr":"172.21.15.155:6821/2542002111","state":["exists","up"]},{"osd":8,"uuid":"d964a183-3021-4ace-9a0c-aad00f2570e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6817","nonce":2985835085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6819","nonce":2985835085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6823","nonce":2985835085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6821","nonce":2985835085}]},"public_addr":"172.21.15.193:6817/2985835085","cluster_addr":"172.21.15.193:6819/2985835085","heartbeat_back_addr":"172.21.15.193:6823/2985835085","heartbeat_front_addr":"172.21.15.193:6821/2985835085","state":["exists","up"]},{"osd":9,"uuid":"9725ea5c-49a3-4c99-bac4-e95069100b1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6827","nonce":1946493651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6829","nonce":1946493651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6833","nonce":1946493651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6831","nonce":1946493651}]},"public_addr":"172.21.15.39:6827/1946493651","cluster_addr":"172.21.15.39:6829/1946493651","heartbeat_back_addr":"172.21.15.39:6833/1946493651","heartbeat_front_addr":"172.21.15.39:6831/1946493651","state":["exists","up"]},{"osd":10,"uuid":"1fd60ccb-92db-45f0-a580-2003db0976e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6825","nonce":1234615022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6827","nonce":1234615022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6831","nonce":1234615022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6829","nonce":1234615022}]},"public_addr":"172.21.15.155:6825/1234615022","cluster_addr":"172.21.15.155:6827/1234615022","heartbeat_back_addr":"172.21.15.155:6831/1234615022","heartbeat_front_addr":"172.21.15.155:6829/1234615022","state":["exists","up"]},{"osd":11,"uuid":"efb31ea1-4775-4e5f-8740-728d6f4aeea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6825","nonce":215104663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6827","nonce":215104663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6831","nonce":215104663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6829","nonce":215104663}]},"public_addr":"172.21.15.193:6825/215104663","cluster_addr":"172.21.15.193:6827/215104663","heartbeat_back_addr":"172.21.15.193:6831/215104663","heartbeat_front_addr":"172.21.15.193:6829/215104663","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:35:30.211698+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:35:50.883846+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:36:11.124168+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:36:32.711905+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:36:54.442007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:37:14.843526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:37:36.587354+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:37:57.882374+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:38:18.729407+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:38:40.139231+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:39:01.290862+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:39:21.767537+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:0/2902600414":"2023-07-22T18:34:07.039823+0000","172.21.15.39:6801/2893866425":"2023-07-22T18:34:07.039823+0000","172.21.15.39:0/19717002":"2023-07-22T18:34:07.039823+0000","172.21.15.39:6800/2893866425":"2023-07-22T18:34:07.039823+0000","172.21.15.39:6800/889516631":"2023-07-22T18:33:40.852706+0000","172.21.15.39:6801/889516631":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/3636510379":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/1275748831":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/3819152602":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/1127767082":"2023-07-22T18:34:07.039823+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T18:40:59.022 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.997+0000 7f888bde7700 1 -- 172.21.15.39:0/59184222 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f88700a1220 msgr2=0x7f88700a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:59.023 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.997+0000 7f888bde7700 1 --2- 172.21.15.39:0/59184222 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f88700a1220 0x7f88700a36d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8880006010 tx=0x7f8880005960 comp rx=0 tx=0).stop 2023-07-21T18:40:59.023 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.997+0000 7f888bde7700 1 -- 172.21.15.39:0/59184222 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8884100580 msgr2=0x7f888419bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:59.023 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.997+0000 7f888bde7700 1 --2- 172.21.15.39:0/59184222 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8884100580 0x7f888419bc90 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f887c007ee0 tx=0x7f887c00d3b0 comp rx=0 tx=0).stop 2023-07-21T18:40:59.023 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.998+0000 7f888bde7700 1 -- 172.21.15.39:0/59184222 shutdown_connections 2023-07-21T18:40:59.023 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.998+0000 7f888bde7700 1 --2- 172.21.15.39:0/59184222 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f88700a1220 0x7f88700a36d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:59.023 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.998+0000 7f888bde7700 1 --2- 172.21.15.39:0/59184222 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8884100580 0x7f888419bc90 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:59.023 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.998+0000 7f888bde7700 1 --2- 172.21.15.39:0/59184222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f88840fea00 0x7f88841946e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:59.024 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.998+0000 7f888bde7700 1 --2- 172.21.15.39:0/59184222 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88840ffbf0 0x7f8884194c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:59.024 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.998+0000 7f888bde7700 1 -- 172.21.15.39:0/59184222 >> 172.21.15.39:0/59184222 conn(0x7f88840f9fe0 msgr2=0x7f88841030b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:59.024 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.998+0000 7f888bde7700 1 -- 172.21.15.39:0/59184222 shutdown_connections 2023-07-21T18:40:59.024 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:58.998+0000 7f888bde7700 1 -- 172.21.15.39:0/59184222 wait complete. 2023-07-21T18:40:59.035 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-21T18:36:13.572583+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-21T18:40:02.726735+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-21T18:40:03.990112+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-21T18:40:59.036 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-21T18:40:59.165 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.140+0000 7f339447e700 1 -- 172.21.15.39:0/641130580 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f338c1057a0 msgr2=0x7f338c107c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:59.166 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.140+0000 7f339447e700 1 --2- 172.21.15.39:0/641130580 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f338c1057a0 0x7f338c107c00 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f338400b0a0 tx=0x7f338400b3b0 comp rx=0 tx=0).stop 2023-07-21T18:40:59.166 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.140+0000 7f339447e700 1 -- 172.21.15.39:0/641130580 shutdown_connections 2023-07-21T18:40:59.166 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.140+0000 7f339447e700 1 --2- 172.21.15.39:0/641130580 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f338c100560 0x7f338c102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:59.166 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.140+0000 7f339447e700 1 --2- 172.21.15.39:0/641130580 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f338c1057a0 0x7f338c107c00 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:59.166 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.140+0000 7f339447e700 1 --2- 172.21.15.39:0/641130580 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f338c102e80 0x7f338c105260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:59.167 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.140+0000 7f339447e700 1 -- 172.21.15.39:0/641130580 >> 172.21.15.39:0/641130580 conn(0x7f338c0fa000 msgr2=0x7f338c0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:59.167 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.140+0000 7f339447e700 1 -- 172.21.15.39:0/641130580 shutdown_connections 2023-07-21T18:40:59.167 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.140+0000 7f339447e700 1 -- 172.21.15.39:0/641130580 wait complete. 2023-07-21T18:40:59.167 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.141+0000 7f339447e700 1 Processor -- start 2023-07-21T18:40:59.168 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.141+0000 7f339447e700 1 -- start start 2023-07-21T18:40:59.169 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.141+0000 7f339447e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f338c100560 0x7f338c195920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:59.169 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.141+0000 7f339447e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f338c102e80 0x7f338c195e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:59.169 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.141+0000 7f339447e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f338c1057a0 0x7f338c19ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:59.169 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.141+0000 7f339447e700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f338c19d4a0 con 0x7f338c1057a0 2023-07-21T18:40:59.169 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.141+0000 7f339447e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f338c19d5e0 con 0x7f338c100560 2023-07-21T18:40:59.169 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.141+0000 7f339447e700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f338c19d770 con 0x7f338c102e80 2023-07-21T18:40:59.170 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.141+0000 7f3392a1b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f338c1057a0 0x7f338c19ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:59.170 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.141+0000 7f3392a1b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f338c1057a0 0x7f338c19ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55798/0 (socket says 172.21.15.39:55798) 2023-07-21T18:40:59.170 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.141+0000 7f3392a1b700 1 -- 172.21.15.39:0/4206443480 learned_addr learned my addr 172.21.15.39:0/4206443480 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:40:59.170 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.141+0000 7f3392a1b700 1 -- 172.21.15.39:0/4206443480 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f338c102e80 msgr2=0x7f338c195e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:59.170 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.141+0000 7f3392a1b700 1 --2- 172.21.15.39:0/4206443480 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f338c102e80 0x7f338c195e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:59.170 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.141+0000 7f3392a1b700 1 -- 172.21.15.39:0/4206443480 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f338c100560 msgr2=0x7f338c195920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:59.171 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.141+0000 7f3392a1b700 1 --2- 172.21.15.39:0/4206443480 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f338c100560 0x7f338c195920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:59.171 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.141+0000 7f3392a1b700 1 -- 172.21.15.39:0/4206443480 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3384009d00 con 0x7f338c1057a0 2023-07-21T18:40:59.171 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.142+0000 7f3392a1b700 1 --2- 172.21.15.39:0/4206443480 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f338c1057a0 0x7f338c19ced0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f33840062a0 tx=0x7f3384004db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:59.171 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.142+0000 7f33837fe700 1 -- 172.21.15.39:0/4206443480 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f338400e040 con 0x7f338c1057a0 2023-07-21T18:40:59.171 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.142+0000 7f33837fe700 1 -- 172.21.15.39:0/4206443480 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f338401db50 con 0x7f338c1057a0 2023-07-21T18:40:59.171 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.142+0000 7f339447e700 1 -- 172.21.15.39:0/4206443480 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f338c19d9f0 con 0x7f338c1057a0 2023-07-21T18:40:59.172 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.142+0000 7f33837fe700 1 -- 172.21.15.39:0/4206443480 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f338401bc40 con 0x7f338c1057a0 2023-07-21T18:40:59.172 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.142+0000 7f339447e700 1 -- 172.21.15.39:0/4206443480 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f338c19de70 con 0x7f338c1057a0 2023-07-21T18:40:59.172 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.143+0000 7f339447e700 1 -- 172.21.15.39:0/4206443480 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f338c0fbb70 con 0x7f338c1057a0 2023-07-21T18:40:59.172 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.144+0000 7f33837fe700 1 -- 172.21.15.39:0/4206443480 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f338401d690 con 0x7f338c1057a0 2023-07-21T18:40:59.172 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.144+0000 7f33837fe700 1 --2- 172.21.15.39:0/4206443480 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f33780a11e0 0x7f33780a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:59.173 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.144+0000 7f33837fe700 1 -- 172.21.15.39:0/4206443480 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f33840ce4f0 con 0x7f338c1057a0 2023-07-21T18:40:59.173 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.144+0000 7f339221a700 1 --2- 172.21.15.39:0/4206443480 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f33780a11e0 0x7f33780a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:59.173 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.145+0000 7f339221a700 1 --2- 172.21.15.39:0/4206443480 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f33780a11e0 0x7f33780a3690 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3388005d90 tx=0x7f3388005d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:59.173 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.149+0000 7f33837fe700 1 -- 172.21.15.39:0/4206443480 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f3384097d20 con 0x7f338c1057a0 2023-07-21T18:40:59.371 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:59 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/2485111859' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T18:40:59.372 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:59 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/533857347' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:40:59.372 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:40:59 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/59184222' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T18:40:59.372 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.348+0000 7f339447e700 1 -- 172.21.15.39:0/4206443480 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f338c02cab0 con 0x7f338c1057a0 2023-07-21T18:40:59.373 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.348+0000 7f33837fe700 1 -- 172.21.15.39:0/4206443480 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7f338409b340 con 0x7f338c1057a0 2023-07-21T18:40:59.373 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2023-07-21T18:40:59.375 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.350+0000 7f339447e700 1 -- 172.21.15.39:0/4206443480 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f33780a11e0 msgr2=0x7f33780a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:59.375 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.350+0000 7f339447e700 1 --2- 172.21.15.39:0/4206443480 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f33780a11e0 0x7f33780a3690 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3388005d90 tx=0x7f3388005d00 comp rx=0 tx=0).stop 2023-07-21T18:40:59.375 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.350+0000 7f339447e700 1 -- 172.21.15.39:0/4206443480 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f338c1057a0 msgr2=0x7f338c19ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:59.376 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.350+0000 7f339447e700 1 --2- 172.21.15.39:0/4206443480 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f338c1057a0 0x7f338c19ced0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f33840062a0 tx=0x7f3384004db0 comp rx=0 tx=0).stop 2023-07-21T18:40:59.376 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.350+0000 7f339447e700 1 -- 172.21.15.39:0/4206443480 shutdown_connections 2023-07-21T18:40:59.376 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.350+0000 7f339447e700 1 --2- 172.21.15.39:0/4206443480 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f33780a11e0 0x7f33780a3690 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:59.376 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.350+0000 7f339447e700 1 --2- 172.21.15.39:0/4206443480 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f338c100560 0x7f338c195920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:59.376 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.350+0000 7f339447e700 1 --2- 172.21.15.39:0/4206443480 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f338c1057a0 0x7f338c19ced0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:59.376 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.350+0000 7f339447e700 1 --2- 172.21.15.39:0/4206443480 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f338c102e80 0x7f338c195e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:59.376 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.350+0000 7f339447e700 1 -- 172.21.15.39:0/4206443480 >> 172.21.15.39:0/4206443480 conn(0x7f338c0fa000 msgr2=0x7f338c101260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:59.377 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.350+0000 7f339447e700 1 -- 172.21.15.39:0/4206443480 shutdown_connections 2023-07-21T18:40:59.377 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.350+0000 7f339447e700 1 -- 172.21.15.39:0/4206443480 wait complete. 2023-07-21T18:40:59.387 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:59 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/2485111859' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T18:40:59.387 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:59 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/533857347' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:40:59.387 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:40:59 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/59184222' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T18:40:59.389 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-21T18:40:59.391 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:59 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/2485111859' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T18:40:59.391 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:59 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/533857347' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:40:59.392 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:40:59 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/59184222' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T18:40:59.520 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.494+0000 7f9a94709700 1 -- 172.21.15.39:0/2604345699 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a8c105830 msgr2=0x7f9a8c107c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:59.520 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.494+0000 7f9a94709700 1 --2- 172.21.15.39:0/2604345699 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a8c105830 0x7f9a8c107c30 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9a8400b0a0 tx=0x7f9a8400b3b0 comp rx=0 tx=0).stop 2023-07-21T18:40:59.520 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.495+0000 7f9a94709700 1 -- 172.21.15.39:0/2604345699 shutdown_connections 2023-07-21T18:40:59.520 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.495+0000 7f9a94709700 1 --2- 172.21.15.39:0/2604345699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a8c1005f0 0x7f9a8c1029d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:59.521 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.495+0000 7f9a94709700 1 --2- 172.21.15.39:0/2604345699 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a8c105830 0x7f9a8c107c30 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:59.521 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.495+0000 7f9a94709700 1 --2- 172.21.15.39:0/2604345699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a8c102f10 0x7f9a8c1052f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:59.521 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.495+0000 7f9a94709700 1 -- 172.21.15.39:0/2604345699 >> 172.21.15.39:0/2604345699 conn(0x7f9a8c0fa030 msgr2=0x7f9a8c0fc460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:59.521 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.495+0000 7f9a94709700 1 -- 172.21.15.39:0/2604345699 shutdown_connections 2023-07-21T18:40:59.521 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.495+0000 7f9a94709700 1 -- 172.21.15.39:0/2604345699 wait complete. 2023-07-21T18:40:59.521 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.495+0000 7f9a94709700 1 Processor -- start 2023-07-21T18:40:59.521 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.495+0000 7f9a94709700 1 -- start start 2023-07-21T18:40:59.523 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.495+0000 7f9a94709700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a8c1005f0 0x7f9a8c0fff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:59.523 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.495+0000 7f9a94709700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a8c102f10 0x7f9a8c0fe5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:59.524 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.495+0000 7f9a94709700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a8c105830 0x7f9a8c0feae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:59.524 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.495+0000 7f9a94709700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8c0ff280 con 0x7f9a8c1005f0 2023-07-21T18:40:59.524 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.496+0000 7f9a94709700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8c0ff3f0 con 0x7f9a8c102f10 2023-07-21T18:40:59.524 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.496+0000 7f9a94709700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8c19f9c0 con 0x7f9a8c105830 2023-07-21T18:40:59.524 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.496+0000 7f9a924a5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a8c1005f0 0x7f9a8c0fff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:59.524 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.496+0000 7f9a924a5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a8c1005f0 0x7f9a8c0fff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55808/0 (socket says 172.21.15.39:55808) 2023-07-21T18:40:59.524 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.496+0000 7f9a924a5700 1 -- 172.21.15.39:0/3625994973 learned_addr learned my addr 172.21.15.39:0/3625994973 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:40:59.525 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.496+0000 7f9a91ca4700 1 --2- 172.21.15.39:0/3625994973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a8c102f10 0x7f9a8c0fe5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:59.525 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.496+0000 7f9a92ca6700 1 --2- 172.21.15.39:0/3625994973 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a8c105830 0x7f9a8c0feae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:59.525 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.496+0000 7f9a924a5700 1 -- 172.21.15.39:0/3625994973 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a8c105830 msgr2=0x7f9a8c0feae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:59.525 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.496+0000 7f9a924a5700 1 --2- 172.21.15.39:0/3625994973 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a8c105830 0x7f9a8c0feae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:59.526 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.496+0000 7f9a924a5700 1 -- 172.21.15.39:0/3625994973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a8c102f10 msgr2=0x7f9a8c0fe5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:59.526 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.496+0000 7f9a924a5700 1 --2- 172.21.15.39:0/3625994973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a8c102f10 0x7f9a8c0fe5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:59.526 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.496+0000 7f9a924a5700 1 -- 172.21.15.39:0/3625994973 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a84009d00 con 0x7f9a8c1005f0 2023-07-21T18:40:59.526 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.496+0000 7f9a924a5700 1 --2- 172.21.15.39:0/3625994973 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a8c1005f0 0x7f9a8c0fff60 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9a8800e9a0 tx=0x7f9a8800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:59.526 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.496+0000 7f9a7f7fe700 1 -- 172.21.15.39:0/3625994973 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a88009790 con 0x7f9a8c1005f0 2023-07-21T18:40:59.526 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.496+0000 7f9a7f7fe700 1 -- 172.21.15.39:0/3625994973 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a88010430 con 0x7f9a8c1005f0 2023-07-21T18:40:59.526 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.496+0000 7f9a94709700 1 -- 172.21.15.39:0/3625994973 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a8c19fca0 con 0x7f9a8c1005f0 2023-07-21T18:40:59.527 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.496+0000 7f9a7f7fe700 1 -- 172.21.15.39:0/3625994973 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a88018680 con 0x7f9a8c1005f0 2023-07-21T18:40:59.527 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.496+0000 7f9a94709700 1 -- 172.21.15.39:0/3625994973 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a8c1a0180 con 0x7f9a8c1005f0 2023-07-21T18:40:59.527 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.497+0000 7f9a94709700 1 -- 172.21.15.39:0/3625994973 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a8c0fbba0 con 0x7f9a8c1005f0 2023-07-21T18:40:59.527 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.498+0000 7f9a7f7fe700 1 -- 172.21.15.39:0/3625994973 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f9a880105a0 con 0x7f9a8c1005f0 2023-07-21T18:40:59.527 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.499+0000 7f9a7f7fe700 1 --2- 172.21.15.39:0/3625994973 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f9a780a11e0 0x7f9a780a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:59.528 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.499+0000 7f9a91ca4700 1 --2- 172.21.15.39:0/3625994973 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f9a780a11e0 0x7f9a780a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:59.528 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.499+0000 7f9a7f7fe700 1 -- 172.21.15.39:0/3625994973 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f9a88014070 con 0x7f9a8c1005f0 2023-07-21T18:40:59.529 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.499+0000 7f9a91ca4700 1 --2- 172.21.15.39:0/3625994973 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f9a780a11e0 0x7f9a780a3690 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9a80009c60 tx=0x7f9a80009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:59.529 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.503+0000 7f9a7f7fe700 1 -- 172.21.15.39:0/3625994973 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f9a88051790 con 0x7f9a8c1005f0 2023-07-21T18:40:59.727 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.702+0000 7f9a94709700 1 -- 172.21.15.39:0/3625994973 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9a8c02c850 con 0x7f9a8c1005f0 2023-07-21T18:40:59.728 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.703+0000 7f9a7f7fe700 1 -- 172.21.15.39:0/3625994973 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v79) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9a8c02c850 con 0x7f9a8c1005f0 2023-07-21T18:40:59.728 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2023-07-21T18:40:59.729 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.705+0000 7f9a94709700 1 -- 172.21.15.39:0/3625994973 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f9a780a11e0 msgr2=0x7f9a780a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:59.730 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.705+0000 7f9a94709700 1 --2- 172.21.15.39:0/3625994973 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f9a780a11e0 0x7f9a780a3690 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9a80009c60 tx=0x7f9a80009260 comp rx=0 tx=0).stop 2023-07-21T18:40:59.730 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.705+0000 7f9a94709700 1 -- 172.21.15.39:0/3625994973 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a8c1005f0 msgr2=0x7f9a8c0fff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:59.730 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.705+0000 7f9a94709700 1 --2- 172.21.15.39:0/3625994973 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a8c1005f0 0x7f9a8c0fff60 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9a8800e9a0 tx=0x7f9a8800ecb0 comp rx=0 tx=0).stop 2023-07-21T18:40:59.730 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.705+0000 7f9a94709700 1 -- 172.21.15.39:0/3625994973 shutdown_connections 2023-07-21T18:40:59.730 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.705+0000 7f9a94709700 1 --2- 172.21.15.39:0/3625994973 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f9a780a11e0 0x7f9a780a3690 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:59.730 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.705+0000 7f9a94709700 1 --2- 172.21.15.39:0/3625994973 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a8c105830 0x7f9a8c0feae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:59.731 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.705+0000 7f9a94709700 1 --2- 172.21.15.39:0/3625994973 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a8c1005f0 0x7f9a8c0fff60 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:59.731 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.705+0000 7f9a94709700 1 --2- 172.21.15.39:0/3625994973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a8c102f10 0x7f9a8c0fe5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:59.731 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.705+0000 7f9a94709700 1 -- 172.21.15.39:0/3625994973 >> 172.21.15.39:0/3625994973 conn(0x7f9a8c0fa030 msgr2=0x7f9a8c101250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:59.731 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.705+0000 7f9a94709700 1 -- 172.21.15.39:0/3625994973 shutdown_connections 2023-07-21T18:40:59.731 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.705+0000 7f9a94709700 1 -- 172.21.15.39:0/3625994973 wait complete. 2023-07-21T18:40:59.742 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-21T18:40:59.874 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.846+0000 7f6a7f5a1700 1 -- 172.21.15.39:0/1550981541 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a78100560 msgr2=0x7f6a78106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:59.875 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.846+0000 7f6a7f5a1700 1 --2- 172.21.15.39:0/1550981541 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a78100560 0x7f6a78106e20 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f6a7000b0a0 tx=0x7f6a7000b3b0 comp rx=0 tx=0).stop 2023-07-21T18:40:59.875 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.846+0000 7f6a7f5a1700 1 -- 172.21.15.39:0/1550981541 shutdown_connections 2023-07-21T18:40:59.875 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.846+0000 7f6a7f5a1700 1 --2- 172.21.15.39:0/1550981541 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a78100560 0x7f6a78106e20 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:59.875 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.846+0000 7f6a7f5a1700 1 --2- 172.21.15.39:0/1550981541 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a780fe9e0 0x7f6a780fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:59.876 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.846+0000 7f6a7f5a1700 1 --2- 172.21.15.39:0/1550981541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a780ffbd0 0x7f6a78100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:59.876 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.846+0000 7f6a7f5a1700 1 -- 172.21.15.39:0/1550981541 >> 172.21.15.39:0/1550981541 conn(0x7f6a780fa000 msgr2=0x7f6a780fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:40:59.876 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.846+0000 7f6a7f5a1700 1 -- 172.21.15.39:0/1550981541 shutdown_connections 2023-07-21T18:40:59.876 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.846+0000 7f6a7f5a1700 1 -- 172.21.15.39:0/1550981541 wait complete. 2023-07-21T18:40:59.877 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.847+0000 7f6a7f5a1700 1 Processor -- start 2023-07-21T18:40:59.877 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.847+0000 7f6a7f5a1700 1 -- start start 2023-07-21T18:40:59.877 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.847+0000 7f6a7f5a1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a780fe9e0 0x7f6a78068200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:59.877 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.847+0000 7f6a7f5a1700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a780ffbd0 0x7f6a7819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:59.878 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.847+0000 7f6a7f5a1700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a78100560 0x7f6a7819cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:59.878 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.847+0000 7f6a7f5a1700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7819d450 con 0x7f6a780fe9e0 2023-07-21T18:40:59.879 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.847+0000 7f6a7f5a1700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7819d5c0 con 0x7f6a780ffbd0 2023-07-21T18:40:59.879 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.847+0000 7f6a7f5a1700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7819d780 con 0x7f6a78100560 2023-07-21T18:40:59.879 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.847+0000 7f6a7d33d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a780fe9e0 0x7f6a78068200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:59.880 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.847+0000 7f6a7d33d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a780fe9e0 0x7f6a78068200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55822/0 (socket says 172.21.15.39:55822) 2023-07-21T18:40:59.880 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.847+0000 7f6a7d33d700 1 -- 172.21.15.39:0/3209722474 learned_addr learned my addr 172.21.15.39:0/3209722474 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:40:59.880 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.847+0000 7f6a7cb3c700 1 --2- 172.21.15.39:0/3209722474 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a780ffbd0 0x7f6a7819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:59.881 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.847+0000 7f6a7db3e700 1 --2- 172.21.15.39:0/3209722474 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a78100560 0x7f6a7819cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:59.881 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.847+0000 7f6a7d33d700 1 -- 172.21.15.39:0/3209722474 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a78100560 msgr2=0x7f6a7819cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:59.881 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.847+0000 7f6a7d33d700 1 --2- 172.21.15.39:0/3209722474 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a78100560 0x7f6a7819cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:59.881 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.848+0000 7f6a7d33d700 1 -- 172.21.15.39:0/3209722474 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a780ffbd0 msgr2=0x7f6a7819a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:40:59.882 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.848+0000 7f6a7d33d700 1 --2- 172.21.15.39:0/3209722474 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a780ffbd0 0x7f6a7819a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:40:59.882 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.848+0000 7f6a7d33d700 1 -- 172.21.15.39:0/3209722474 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a70009d00 con 0x7f6a780fe9e0 2023-07-21T18:40:59.882 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.848+0000 7f6a7d33d700 1 --2- 172.21.15.39:0/3209722474 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a780fe9e0 0x7f6a78068200 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f6a7400d980 tx=0x7f6a7400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:59.882 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.848+0000 7f6a6e7fc700 1 -- 172.21.15.39:0/3209722474 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a74004020 con 0x7f6a780fe9e0 2023-07-21T18:40:59.883 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.848+0000 7f6a7f5a1700 1 -- 172.21.15.39:0/3209722474 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a7819da60 con 0x7f6a780fe9e0 2023-07-21T18:40:59.883 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.848+0000 7f6a6e7fc700 1 -- 172.21.15.39:0/3209722474 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a7400f690 con 0x7f6a780fe9e0 2023-07-21T18:40:59.883 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.848+0000 7f6a6e7fc700 1 -- 172.21.15.39:0/3209722474 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a74003a60 con 0x7f6a780fe9e0 2023-07-21T18:40:59.883 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.848+0000 7f6a7f5a1700 1 -- 172.21.15.39:0/3209722474 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a7819df70 con 0x7f6a780fe9e0 2023-07-21T18:40:59.884 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.849+0000 7f6a7f5a1700 1 -- 172.21.15.39:0/3209722474 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a78105870 con 0x7f6a780fe9e0 2023-07-21T18:40:59.885 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.850+0000 7f6a6e7fc700 1 -- 172.21.15.39:0/3209722474 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f6a7400fb00 con 0x7f6a780fe9e0 2023-07-21T18:40:59.885 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.850+0000 7f6a6e7fc700 1 --2- 172.21.15.39:0/3209722474 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f6a640a11e0 0x7f6a640a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:40:59.886 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.851+0000 7f6a6e7fc700 1 -- 172.21.15.39:0/3209722474 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f6a740cc4e0 con 0x7f6a780fe9e0 2023-07-21T18:40:59.886 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.851+0000 7f6a7cb3c700 1 --2- 172.21.15.39:0/3209722474 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f6a640a11e0 0x7f6a640a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:40:59.886 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.851+0000 7f6a7cb3c700 1 --2- 172.21.15.39:0/3209722474 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f6a640a11e0 0x7f6a640a3690 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6a78067f70 tx=0x7f6a6800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:40:59.886 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:40:59.855+0000 7f6a6e7fc700 1 -- 172.21.15.39:0/3209722474 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6a74095c80 con 0x7f6a780fe9e0 2023-07-21T18:41:00.080 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.055+0000 7f6a7f5a1700 1 -- 172.21.15.39:0/3209722474 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6a7802ca60 con 0x7f6a780fe9e0 2023-07-21T18:41:00.080 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.056+0000 7f6a6e7fc700 1 -- 172.21.15.39:0/3209722474 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v79) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6a740992a0 con 0x7f6a780fe9e0 2023-07-21T18:41:00.081 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 64 2023-07-21T18:41:00.083 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.058+0000 7f6a7f5a1700 1 -- 172.21.15.39:0/3209722474 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f6a640a11e0 msgr2=0x7f6a640a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:00.083 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.058+0000 7f6a7f5a1700 1 --2- 172.21.15.39:0/3209722474 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f6a640a11e0 0x7f6a640a3690 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6a78067f70 tx=0x7f6a6800b410 comp rx=0 tx=0).stop 2023-07-21T18:41:00.083 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.058+0000 7f6a7f5a1700 1 -- 172.21.15.39:0/3209722474 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a780fe9e0 msgr2=0x7f6a78068200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:00.083 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.058+0000 7f6a7f5a1700 1 --2- 172.21.15.39:0/3209722474 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a780fe9e0 0x7f6a78068200 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f6a7400d980 tx=0x7f6a7400dc90 comp rx=0 tx=0).stop 2023-07-21T18:41:00.084 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.058+0000 7f6a7f5a1700 1 -- 172.21.15.39:0/3209722474 shutdown_connections 2023-07-21T18:41:00.084 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.058+0000 7f6a7f5a1700 1 --2- 172.21.15.39:0/3209722474 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f6a640a11e0 0x7f6a640a3690 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:00.084 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.058+0000 7f6a7f5a1700 1 --2- 172.21.15.39:0/3209722474 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a78100560 0x7f6a7819cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:00.084 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.058+0000 7f6a7f5a1700 1 --2- 172.21.15.39:0/3209722474 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a780fe9e0 0x7f6a78068200 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:00.084 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.058+0000 7f6a7f5a1700 1 --2- 172.21.15.39:0/3209722474 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a780ffbd0 0x7f6a7819a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:00.084 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.058+0000 7f6a7f5a1700 1 -- 172.21.15.39:0/3209722474 >> 172.21.15.39:0/3209722474 conn(0x7f6a780fa000 msgr2=0x7f6a780fc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:00.085 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.058+0000 7f6a7f5a1700 1 -- 172.21.15.39:0/3209722474 shutdown_connections 2023-07-21T18:41:00.085 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.058+0000 7f6a7f5a1700 1 -- 172.21.15.39:0/3209722474 wait complete. 2023-07-21T18:41:00.095 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T18:41:00.222 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.197+0000 7f84d5de3700 1 -- 172.21.15.39:0/1293884584 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84d00fe9d0 msgr2=0x7f84d00fede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:00.222 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.197+0000 7f84d5de3700 1 --2- 172.21.15.39:0/1293884584 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84d00fe9d0 0x7f84d00fede0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f84c0009990 tx=0x7f84c0009ca0 comp rx=0 tx=0).stop 2023-07-21T18:41:00.222 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.197+0000 7f84d5de3700 1 -- 172.21.15.39:0/1293884584 shutdown_connections 2023-07-21T18:41:00.223 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.197+0000 7f84d5de3700 1 --2- 172.21.15.39:0/1293884584 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f84d0100550 0x7f84d0106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:00.223 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.197+0000 7f84d5de3700 1 --2- 172.21.15.39:0/1293884584 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84d00fe9d0 0x7f84d00fede0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:00.223 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.197+0000 7f84d5de3700 1 --2- 172.21.15.39:0/1293884584 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84d00ffbc0 0x7f84d0100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:00.223 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.197+0000 7f84d5de3700 1 -- 172.21.15.39:0/1293884584 >> 172.21.15.39:0/1293884584 conn(0x7f84d00f9fd0 msgr2=0x7f84d00fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:00.223 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.197+0000 7f84d5de3700 1 -- 172.21.15.39:0/1293884584 shutdown_connections 2023-07-21T18:41:00.223 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.197+0000 7f84d5de3700 1 -- 172.21.15.39:0/1293884584 wait complete. 2023-07-21T18:41:00.223 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.198+0000 7f84d5de3700 1 Processor -- start 2023-07-21T18:41:00.225 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.198+0000 7f84d5de3700 1 -- start start 2023-07-21T18:41:00.225 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.198+0000 7f84d5de3700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84d00fe9d0 0x7f84d01957c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:00.225 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.198+0000 7f84d5de3700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f84d00ffbc0 0x7f84d0195d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:00.226 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.198+0000 7f84d5de3700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84d0100550 0x7f84d019cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:00.226 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.198+0000 7f84d5de3700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d019d340 con 0x7f84d0100550 2023-07-21T18:41:00.226 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.198+0000 7f84d5de3700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d019d480 con 0x7f84d00fe9d0 2023-07-21T18:41:00.226 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.198+0000 7f84d5de3700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d019d610 con 0x7f84d00ffbc0 2023-07-21T18:41:00.226 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.198+0000 7f84cffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84d0100550 0x7f84d019cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:00.226 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.198+0000 7f84cffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84d0100550 0x7f84d019cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55840/0 (socket says 172.21.15.39:55840) 2023-07-21T18:41:00.226 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.198+0000 7f84cffff700 1 -- 172.21.15.39:0/378083044 learned_addr learned my addr 172.21.15.39:0/378083044 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:00.226 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.198+0000 7f84ceffd700 1 --2- 172.21.15.39:0/378083044 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f84d00ffbc0 0x7f84d0195d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:00.227 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.198+0000 7f84cf7fe700 1 --2- 172.21.15.39:0/378083044 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84d00fe9d0 0x7f84d01957c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:00.227 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.198+0000 7f84cffff700 1 -- 172.21.15.39:0/378083044 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f84d00ffbc0 msgr2=0x7f84d0195d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:00.227 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.198+0000 7f84cffff700 1 --2- 172.21.15.39:0/378083044 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f84d00ffbc0 0x7f84d0195d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:00.227 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.198+0000 7f84cffff700 1 -- 172.21.15.39:0/378083044 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84d00fe9d0 msgr2=0x7f84d01957c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:00.227 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.198+0000 7f84cffff700 1 --2- 172.21.15.39:0/378083044 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84d00fe9d0 0x7f84d01957c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:00.227 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.198+0000 7f84cffff700 1 -- 172.21.15.39:0/378083044 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84c00095f0 con 0x7f84d0100550 2023-07-21T18:41:00.227 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.199+0000 7f84cffff700 1 --2- 172.21.15.39:0/378083044 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84d0100550 0x7f84d019cd70 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f84c800b970 tx=0x7f84c800bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:00.228 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.199+0000 7f84ccff9700 1 -- 172.21.15.39:0/378083044 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c800c9b0 con 0x7f84d0100550 2023-07-21T18:41:00.228 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.199+0000 7f84d5de3700 1 -- 172.21.15.39:0/378083044 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84d019d8f0 con 0x7f84d0100550 2023-07-21T18:41:00.228 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.199+0000 7f84ccff9700 1 -- 172.21.15.39:0/378083044 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84c8014440 con 0x7f84d0100550 2023-07-21T18:41:00.228 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.199+0000 7f84ccff9700 1 -- 172.21.15.39:0/378083044 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c80075d0 con 0x7f84d0100550 2023-07-21T18:41:00.228 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.199+0000 7f84d5de3700 1 -- 172.21.15.39:0/378083044 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84d019de80 con 0x7f84d0100550 2023-07-21T18:41:00.228 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.200+0000 7f84d5de3700 1 -- 172.21.15.39:0/378083044 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84d010a220 con 0x7f84d0100550 2023-07-21T18:41:00.229 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.201+0000 7f84ccff9700 1 -- 172.21.15.39:0/378083044 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f84c80145b0 con 0x7f84d0100550 2023-07-21T18:41:00.229 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.202+0000 7f84ccff9700 1 --2- 172.21.15.39:0/378083044 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f84b80a1220 0x7f84b80a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:00.229 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.202+0000 7f84cf7fe700 1 --2- 172.21.15.39:0/378083044 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f84b80a1220 0x7f84b80a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:00.229 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.202+0000 7f84ccff9700 1 -- 172.21.15.39:0/378083044 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f84c8006e40 con 0x7f84d0100550 2023-07-21T18:41:00.229 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.202+0000 7f84cf7fe700 1 --2- 172.21.15.39:0/378083044 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f84b80a1220 0x7f84b80a36d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f84c0006010 tx=0x7f84c000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:00.230 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.205+0000 7f84ccff9700 1 -- 172.21.15.39:0/378083044 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f84c8095ca0 con 0x7f84d0100550 2023-07-21T18:41:00.371 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:00 smithi193 ceph-mon[98204]: pgmap v275: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-07-21T18:41:00.372 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:00 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/4206443480' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T18:41:00.372 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:00 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/3625994973' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T18:41:00.390 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:00 smithi155 ceph-mon[97934]: pgmap v275: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-07-21T18:41:00.390 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:00 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/4206443480' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T18:41:00.391 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:00 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/3625994973' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T18:41:00.439 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:00 smithi039 ceph-mon[96489]: pgmap v275: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-07-21T18:41:00.439 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:00 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/4206443480' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T18:41:00.439 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:00 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/3625994973' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T18:41:00.473 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.449+0000 7f84d5de3700 1 -- 172.21.15.39:0/378083044 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f84d010a410 con 0x7f84d0100550 2023-07-21T18:41:00.474 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.449+0000 7f84ccff9700 1 -- 172.21.15.39:0/378083044 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10887 (secure 0 0 0) 0x7f84c80992c0 con 0x7f84d0100550 2023-07-21T18:41:00.474 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:41:00.475 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/147318681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":147318681},{"type":"v1","addr":"172.21.15.39:6835","nonce":147318681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/1062891760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1062891760},{"type":"v1","addr":"172.21.15.39:6837","nonce":1062891760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T18:40:04.594553+0000","modified":"2023-07-21T18:40:11.989039+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24536,"mds_1":24500,"mds_2":24487,"mds_3":24464,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/3184273349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":3184273349},{"type":"v1","addr":"172.21.15.39:6839","nonce":3184273349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6841/640981220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":640981220},{"type":"v1","addr":"172.21.15.39:6841","nonce":640981220}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3106166804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3106166804},{"type":"v1","addr":"172.21.15.193:6833","nonce":3106166804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24437":{"gid":24437,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/1034755277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1034755277},{"type":"v1","addr":"172.21.15.155:6833","nonce":1034755277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/381560657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":381560657},{"type":"v1","addr":"172.21.15.193:6835","nonce":381560657}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6837/2626761843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2626761843},{"type":"v1","addr":"172.21.15.193:6837","nonce":2626761843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24464":{"gid":24464,"name":"e","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.155:6835/3854856286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3854856286},{"type":"v1","addr":"172.21.15.155:6835","nonce":3854856286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.193:6839/3148488639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3148488639},{"type":"v1","addr":"172.21.15.193:6839","nonce":3148488639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.155:6837/947670056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":947670056},{"type":"v1","addr":"172.21.15.155:6837","nonce":947670056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6839/431690342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":431690342},{"type":"v1","addr":"172.21.15.155:6839","nonce":431690342}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T18:41:00.476 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.451+0000 7f84d5de3700 1 -- 172.21.15.39:0/378083044 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f84b80a1220 msgr2=0x7f84b80a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:00.476 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.451+0000 7f84d5de3700 1 --2- 172.21.15.39:0/378083044 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f84b80a1220 0x7f84b80a36d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f84c0006010 tx=0x7f84c000b540 comp rx=0 tx=0).stop 2023-07-21T18:41:00.477 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.452+0000 7f84d5de3700 1 -- 172.21.15.39:0/378083044 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84d0100550 msgr2=0x7f84d019cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:00.477 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.452+0000 7f84d5de3700 1 --2- 172.21.15.39:0/378083044 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84d0100550 0x7f84d019cd70 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f84c800b970 tx=0x7f84c800bd30 comp rx=0 tx=0).stop 2023-07-21T18:41:00.477 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.452+0000 7f84d5de3700 1 -- 172.21.15.39:0/378083044 shutdown_connections 2023-07-21T18:41:00.477 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.452+0000 7f84d5de3700 1 --2- 172.21.15.39:0/378083044 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f84b80a1220 0x7f84b80a36d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:00.477 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.452+0000 7f84d5de3700 1 --2- 172.21.15.39:0/378083044 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84d00fe9d0 0x7f84d01957c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:00.477 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.452+0000 7f84d5de3700 1 --2- 172.21.15.39:0/378083044 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84d0100550 0x7f84d019cd70 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:00.477 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.452+0000 7f84d5de3700 1 --2- 172.21.15.39:0/378083044 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f84d00ffbc0 0x7f84d0195d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:00.478 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.452+0000 7f84d5de3700 1 -- 172.21.15.39:0/378083044 >> 172.21.15.39:0/378083044 conn(0x7f84d00f9fd0 msgr2=0x7f84d0067730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:00.478 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.452+0000 7f84d5de3700 1 -- 172.21.15.39:0/378083044 shutdown_connections 2023-07-21T18:41:00.478 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.452+0000 7f84d5de3700 1 -- 172.21.15.39:0/378083044 wait complete. 2023-07-21T18:41:00.478 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 26 2023-07-21T18:41:00.489 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T18:41:00.614 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.589+0000 7f12357b2700 1 -- 172.21.15.39:0/3984005113 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1230100550 msgr2=0x7f1230102930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:00.615 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.589+0000 7f12357b2700 1 --2- 172.21.15.39:0/3984005113 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1230100550 0x7f1230102930 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1220009990 tx=0x7f1220009ca0 comp rx=0 tx=0).stop 2023-07-21T18:41:00.615 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.590+0000 7f12357b2700 1 -- 172.21.15.39:0/3984005113 shutdown_connections 2023-07-21T18:41:00.615 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.590+0000 7f12357b2700 1 --2- 172.21.15.39:0/3984005113 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1230105790 0x7f1230107bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:00.615 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.590+0000 7f12357b2700 1 --2- 172.21.15.39:0/3984005113 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1230100550 0x7f1230102930 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:00.615 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.590+0000 7f12357b2700 1 --2- 172.21.15.39:0/3984005113 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1230102e70 0x7f1230105250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:00.615 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.590+0000 7f12357b2700 1 -- 172.21.15.39:0/3984005113 >> 172.21.15.39:0/3984005113 conn(0x7f12300f9fd0 msgr2=0x7f12300fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:00.616 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.590+0000 7f12357b2700 1 -- 172.21.15.39:0/3984005113 shutdown_connections 2023-07-21T18:41:00.616 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.590+0000 7f12357b2700 1 -- 172.21.15.39:0/3984005113 wait complete. 2023-07-21T18:41:00.617 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.590+0000 7f12357b2700 1 Processor -- start 2023-07-21T18:41:00.618 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.591+0000 7f12357b2700 1 -- start start 2023-07-21T18:41:00.618 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.591+0000 7f12357b2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1230100550 0x7f12300ffec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:00.618 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.591+0000 7f12357b2700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1230102e70 0x7f12300fe500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:00.618 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.591+0000 7f12357b2700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1230105790 0x7f12300fea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:00.618 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.591+0000 7f12357b2700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12300ff1e0 con 0x7f1230100550 2023-07-21T18:41:00.618 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.591+0000 7f12357b2700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12300ff350 con 0x7f1230105790 2023-07-21T18:41:00.619 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.591+0000 7f12357b2700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f123019f8b0 con 0x7f1230102e70 2023-07-21T18:41:00.619 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.591+0000 7f122effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1230100550 0x7f12300ffec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:00.619 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.591+0000 7f122effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1230100550 0x7f12300ffec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55850/0 (socket says 172.21.15.39:55850) 2023-07-21T18:41:00.619 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.591+0000 7f122effd700 1 -- 172.21.15.39:0/3700900770 learned_addr learned my addr 172.21.15.39:0/3700900770 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:00.619 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.591+0000 7f122f7fe700 1 --2- 172.21.15.39:0/3700900770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1230105790 0x7f12300fea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:00.619 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.591+0000 7f122effd700 1 -- 172.21.15.39:0/3700900770 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1230102e70 msgr2=0x7f12300fe500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:00.619 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.591+0000 7f122e7fc700 1 --2- 172.21.15.39:0/3700900770 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1230102e70 0x7f12300fe500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:00.619 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.591+0000 7f122effd700 1 --2- 172.21.15.39:0/3700900770 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1230102e70 0x7f12300fe500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:00.620 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.591+0000 7f122effd700 1 -- 172.21.15.39:0/3700900770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1230105790 msgr2=0x7f12300fea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:00.620 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.591+0000 7f122effd700 1 --2- 172.21.15.39:0/3700900770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1230105790 0x7f12300fea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:00.620 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.591+0000 7f122effd700 1 -- 172.21.15.39:0/3700900770 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12200095f0 con 0x7f1230100550 2023-07-21T18:41:00.620 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.591+0000 7f122e7fc700 1 --2- 172.21.15.39:0/3700900770 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1230102e70 0x7f12300fe500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:41:00.620 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.591+0000 7f122effd700 1 --2- 172.21.15.39:0/3700900770 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1230100550 0x7f12300ffec0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1220000c00 tx=0x7f122000f830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:00.620 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.592+0000 7f1217fff700 1 -- 172.21.15.39:0/3700900770 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f122001d070 con 0x7f1230100550 2023-07-21T18:41:00.620 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.592+0000 7f12357b2700 1 -- 172.21.15.39:0/3700900770 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f123019fb30 con 0x7f1230100550 2023-07-21T18:41:00.621 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.592+0000 7f1217fff700 1 -- 172.21.15.39:0/3700900770 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f122000fce0 con 0x7f1230100550 2023-07-21T18:41:00.621 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.592+0000 7f1217fff700 1 -- 172.21.15.39:0/3700900770 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12200176e0 con 0x7f1230100550 2023-07-21T18:41:00.621 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.592+0000 7f12357b2700 1 -- 172.21.15.39:0/3700900770 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f123019ffe0 con 0x7f1230100550 2023-07-21T18:41:00.621 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.593+0000 7f12357b2700 1 -- 172.21.15.39:0/3700900770 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12300fbb40 con 0x7f1230100550 2023-07-21T18:41:00.622 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.594+0000 7f1217fff700 1 -- 172.21.15.39:0/3700900770 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f1220022470 con 0x7f1230100550 2023-07-21T18:41:00.622 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.594+0000 7f1217fff700 1 --2- 172.21.15.39:0/3700900770 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f12180a1220 0x7f12180a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:00.622 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.595+0000 7f1217fff700 1 -- 172.21.15.39:0/3700900770 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f12200ce1f0 con 0x7f1230100550 2023-07-21T18:41:00.622 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.595+0000 7f122e7fc700 1 --2- 172.21.15.39:0/3700900770 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f12180a1220 0x7f12180a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:00.622 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.595+0000 7f122e7fc700 1 --2- 172.21.15.39:0/3700900770 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f12180a1220 0x7f12180a36d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1224005fd0 tx=0x7f1224005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:00.623 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.598+0000 7f1217fff700 1 -- 172.21.15.39:0/3700900770 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f1220097a20 con 0x7f1230100550 2023-07-21T18:41:00.825 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.801+0000 7f12357b2700 1 -- 172.21.15.39:0/3700900770 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f123002caa0 con 0x7f1230100550 2023-07-21T18:41:00.827 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.802+0000 7f1217fff700 1 -- 172.21.15.39:0/3700900770 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+22117 (secure 0 0 0) 0x7f122009b040 con 0x7f1230100550 2023-07-21T18:41:00.827 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:41:00.827 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":79,"fsid":"eaf8e25a-27f4-11ee-9b34-001a4aab830c","created":"2023-07-21T18:33:06.504979+0000","modified":"2023-07-21T18:40:05.769300+0000","last_up_change":"2023-07-21T18:39:23.531009+0000","last_in_change":"2023-07-21T18:39:05.993024+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T18:36:13.572583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T18:40:02.726735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T18:40:03.990112+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03502c07-2d0f-4723-8d0b-d34c9f5b91bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6803","nonce":860193857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6805","nonce":860193857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6809","nonce":860193857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6807","nonce":860193857}]},"public_addr":"172.21.15.39:6803/860193857","cluster_addr":"172.21.15.39:6805/860193857","heartbeat_back_addr":"172.21.15.39:6809/860193857","heartbeat_front_addr":"172.21.15.39:6807/860193857","state":["exists","up"]},{"osd":1,"uuid":"0831a0c0-a1fa-4d30-9a28-513223aa69eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6801","nonce":3246721764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6803","nonce":3246721764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6807","nonce":3246721764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6805","nonce":3246721764}]},"public_addr":"172.21.15.155:6801/3246721764","cluster_addr":"172.21.15.155:6803/3246721764","heartbeat_back_addr":"172.21.15.155:6807/3246721764","heartbeat_front_addr":"172.21.15.155:6805/3246721764","state":["exists","up"]},{"osd":2,"uuid":"22df0003-bde9-48e4-b47a-cb8eafbab0db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6801","nonce":2563594449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6803","nonce":2563594449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6807","nonce":2563594449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6805","nonce":2563594449}]},"public_addr":"172.21.15.193:6801/2563594449","cluster_addr":"172.21.15.193:6803/2563594449","heartbeat_back_addr":"172.21.15.193:6807/2563594449","heartbeat_front_addr":"172.21.15.193:6805/2563594449","state":["exists","up"]},{"osd":3,"uuid":"54562558-a2e8-45dd-bc95-561569b9ae6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6811","nonce":863708386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6813","nonce":863708386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6817","nonce":863708386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6815","nonce":863708386}]},"public_addr":"172.21.15.39:6811/863708386","cluster_addr":"172.21.15.39:6813/863708386","heartbeat_back_addr":"172.21.15.39:6817/863708386","heartbeat_front_addr":"172.21.15.39:6815/863708386","state":["exists","up"]},{"osd":4,"uuid":"491118e7-ae1e-495c-9eac-ef1cdcae9600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6809","nonce":3361780673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6811","nonce":3361780673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6815","nonce":3361780673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6813","nonce":3361780673}]},"public_addr":"172.21.15.155:6809/3361780673","cluster_addr":"172.21.15.155:6811/3361780673","heartbeat_back_addr":"172.21.15.155:6815/3361780673","heartbeat_front_addr":"172.21.15.155:6813/3361780673","state":["exists","up"]},{"osd":5,"uuid":"d267bc2c-42b3-4d96-b2d1-b186dc7e6b18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6809","nonce":2376867721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6811","nonce":2376867721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6815","nonce":2376867721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6813","nonce":2376867721}]},"public_addr":"172.21.15.193:6809/2376867721","cluster_addr":"172.21.15.193:6811/2376867721","heartbeat_back_addr":"172.21.15.193:6815/2376867721","heartbeat_front_addr":"172.21.15.193:6813/2376867721","state":["exists","up"]},{"osd":6,"uuid":"beec8088-970a-473e-bd67-6eee7cbdef24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6819","nonce":2417770529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6821","nonce":2417770529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6825","nonce":2417770529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6823","nonce":2417770529}]},"public_addr":"172.21.15.39:6819/2417770529","cluster_addr":"172.21.15.39:6821/2417770529","heartbeat_back_addr":"172.21.15.39:6825/2417770529","heartbeat_front_addr":"172.21.15.39:6823/2417770529","state":["exists","up"]},{"osd":7,"uuid":"5a88ce30-1835-4254-90cc-804ed5cf1066","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6817","nonce":2542002111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6819","nonce":2542002111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6823","nonce":2542002111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6821","nonce":2542002111}]},"public_addr":"172.21.15.155:6817/2542002111","cluster_addr":"172.21.15.155:6819/2542002111","heartbeat_back_addr":"172.21.15.155:6823/2542002111","heartbeat_front_addr":"172.21.15.155:6821/2542002111","state":["exists","up"]},{"osd":8,"uuid":"d964a183-3021-4ace-9a0c-aad00f2570e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6817","nonce":2985835085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6819","nonce":2985835085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6823","nonce":2985835085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6821","nonce":2985835085}]},"public_addr":"172.21.15.193:6817/2985835085","cluster_addr":"172.21.15.193:6819/2985835085","heartbeat_back_addr":"172.21.15.193:6823/2985835085","heartbeat_front_addr":"172.21.15.193:6821/2985835085","state":["exists","up"]},{"osd":9,"uuid":"9725ea5c-49a3-4c99-bac4-e95069100b1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6827","nonce":1946493651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6829","nonce":1946493651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6833","nonce":1946493651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6831","nonce":1946493651}]},"public_addr":"172.21.15.39:6827/1946493651","cluster_addr":"172.21.15.39:6829/1946493651","heartbeat_back_addr":"172.21.15.39:6833/1946493651","heartbeat_front_addr":"172.21.15.39:6831/1946493651","state":["exists","up"]},{"osd":10,"uuid":"1fd60ccb-92db-45f0-a580-2003db0976e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6825","nonce":1234615022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6827","nonce":1234615022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6831","nonce":1234615022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6829","nonce":1234615022}]},"public_addr":"172.21.15.155:6825/1234615022","cluster_addr":"172.21.15.155:6827/1234615022","heartbeat_back_addr":"172.21.15.155:6831/1234615022","heartbeat_front_addr":"172.21.15.155:6829/1234615022","state":["exists","up"]},{"osd":11,"uuid":"efb31ea1-4775-4e5f-8740-728d6f4aeea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6825","nonce":215104663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6827","nonce":215104663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6831","nonce":215104663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6829","nonce":215104663}]},"public_addr":"172.21.15.193:6825/215104663","cluster_addr":"172.21.15.193:6827/215104663","heartbeat_back_addr":"172.21.15.193:6831/215104663","heartbeat_front_addr":"172.21.15.193:6829/215104663","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:35:30.211698+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:35:50.883846+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:36:11.124168+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:36:32.711905+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:36:54.442007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:37:14.843526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:37:36.587354+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:37:57.882374+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:38:18.729407+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:38:40.139231+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:39:01.290862+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:39:21.767537+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:0/2902600414":"2023-07-22T18:34:07.039823+0000","172.21.15.39:6801/2893866425":"2023-07-22T18:34:07.039823+0000","172.21.15.39:0/19717002":"2023-07-22T18:34:07.039823+0000","172.21.15.39:6800/2893866425":"2023-07-22T18:34:07.039823+0000","172.21.15.39:6800/889516631":"2023-07-22T18:33:40.852706+0000","172.21.15.39:6801/889516631":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/3636510379":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/1275748831":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/3819152602":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/1127767082":"2023-07-22T18:34:07.039823+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T18:41:00.829 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.804+0000 7f12357b2700 1 -- 172.21.15.39:0/3700900770 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f12180a1220 msgr2=0x7f12180a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:00.829 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.804+0000 7f12357b2700 1 --2- 172.21.15.39:0/3700900770 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f12180a1220 0x7f12180a36d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1224005fd0 tx=0x7f1224005dc0 comp rx=0 tx=0).stop 2023-07-21T18:41:00.829 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.804+0000 7f12357b2700 1 -- 172.21.15.39:0/3700900770 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1230100550 msgr2=0x7f12300ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:00.829 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.804+0000 7f12357b2700 1 --2- 172.21.15.39:0/3700900770 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1230100550 0x7f12300ffec0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1220000c00 tx=0x7f122000f830 comp rx=0 tx=0).stop 2023-07-21T18:41:00.830 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.804+0000 7f12357b2700 1 -- 172.21.15.39:0/3700900770 shutdown_connections 2023-07-21T18:41:00.830 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.804+0000 7f12357b2700 1 --2- 172.21.15.39:0/3700900770 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f12180a1220 0x7f12180a36d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:00.830 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.804+0000 7f12357b2700 1 --2- 172.21.15.39:0/3700900770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1230105790 0x7f12300fea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:00.830 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.804+0000 7f12357b2700 1 --2- 172.21.15.39:0/3700900770 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1230100550 0x7f12300ffec0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:00.830 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.804+0000 7f12357b2700 1 --2- 172.21.15.39:0/3700900770 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1230102e70 0x7f12300fe500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:00.830 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.804+0000 7f12357b2700 1 -- 172.21.15.39:0/3700900770 >> 172.21.15.39:0/3700900770 conn(0x7f12300f9fd0 msgr2=0x7f12301011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:00.830 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.804+0000 7f12357b2700 1 -- 172.21.15.39:0/3700900770 shutdown_connections 2023-07-21T18:41:00.830 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.804+0000 7f12357b2700 1 -- 172.21.15.39:0/3700900770 wait complete. 2023-07-21T18:41:00.842 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T18:41:00.968 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.943+0000 7f1147841700 1 -- 172.21.15.39:0/938131598 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11400fe9f0 msgr2=0x7f11400fee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:00.968 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.943+0000 7f1147841700 1 --2- 172.21.15.39:0/938131598 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11400fe9f0 0x7f11400fee00 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1130009990 tx=0x7f1130009ca0 comp rx=0 tx=0).stop 2023-07-21T18:41:00.968 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.943+0000 7f1147841700 1 -- 172.21.15.39:0/938131598 shutdown_connections 2023-07-21T18:41:00.968 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.943+0000 7f1147841700 1 --2- 172.21.15.39:0/938131598 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1140100570 0x7f1140106e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:00.969 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.943+0000 7f1147841700 1 --2- 172.21.15.39:0/938131598 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11400fe9f0 0x7f11400fee00 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:00.969 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.943+0000 7f1147841700 1 --2- 172.21.15.39:0/938131598 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11400ffbe0 0x7f1140100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:00.969 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.943+0000 7f1147841700 1 -- 172.21.15.39:0/938131598 >> 172.21.15.39:0/938131598 conn(0x7f11400f9fd0 msgr2=0x7f11400fc440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:00.969 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.943+0000 7f1147841700 1 -- 172.21.15.39:0/938131598 shutdown_connections 2023-07-21T18:41:00.969 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.943+0000 7f1147841700 1 -- 172.21.15.39:0/938131598 wait complete. 2023-07-21T18:41:00.971 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.944+0000 7f1147841700 1 Processor -- start 2023-07-21T18:41:00.971 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.944+0000 7f1147841700 1 -- start start 2023-07-21T18:41:00.971 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.944+0000 7f1147841700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11400fe9f0 0x7f1140069230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:00.971 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.944+0000 7f1147841700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11400ffbe0 0x7f114019a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:00.971 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.944+0000 7f1147841700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1140100570 0x7f114019cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:00.971 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.944+0000 7f1147841700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f114019d350 con 0x7f11400ffbe0 2023-07-21T18:41:00.972 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.944+0000 7f1147841700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f114019d4c0 con 0x7f1140100570 2023-07-21T18:41:00.972 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.944+0000 7f1147841700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f114019d680 con 0x7f11400fe9f0 2023-07-21T18:41:00.972 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.944+0000 7f1144ddc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11400ffbe0 0x7f114019a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:00.972 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.944+0000 7f1144ddc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11400ffbe0 0x7f114019a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55866/0 (socket says 172.21.15.39:55866) 2023-07-21T18:41:00.972 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.944+0000 7f1144ddc700 1 -- 172.21.15.39:0/1695775236 learned_addr learned my addr 172.21.15.39:0/1695775236 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:00.972 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.944+0000 7f11455dd700 1 --2- 172.21.15.39:0/1695775236 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11400fe9f0 0x7f1140069230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:00.972 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.944+0000 7f1144ddc700 1 -- 172.21.15.39:0/1695775236 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11400fe9f0 msgr2=0x7f1140069230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:00.973 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.944+0000 7f1144ddc700 1 --2- 172.21.15.39:0/1695775236 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11400fe9f0 0x7f1140069230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:00.973 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.944+0000 7f1144ddc700 1 -- 172.21.15.39:0/1695775236 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1140100570 msgr2=0x7f114019cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:00.973 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.944+0000 7f1144ddc700 1 --2- 172.21.15.39:0/1695775236 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1140100570 0x7f114019cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:00.973 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.944+0000 7f1144ddc700 1 -- 172.21.15.39:0/1695775236 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11300095f0 con 0x7f11400ffbe0 2023-07-21T18:41:00.973 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.944+0000 7f1144ddc700 1 --2- 172.21.15.39:0/1695775236 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11400ffbe0 0x7f114019a530 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f113c00e9a0 tx=0x7f113c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:00.973 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.945+0000 7f11367fc700 1 -- 172.21.15.39:0/1695775236 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113c009790 con 0x7f11400ffbe0 2023-07-21T18:41:00.973 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.945+0000 7f11367fc700 1 -- 172.21.15.39:0/1695775236 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f113c00cbd0 con 0x7f11400ffbe0 2023-07-21T18:41:00.973 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.945+0000 7f11367fc700 1 -- 172.21.15.39:0/1695775236 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113c018680 con 0x7f11400ffbe0 2023-07-21T18:41:00.974 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.945+0000 7f1147841700 1 -- 172.21.15.39:0/1695775236 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f114019d960 con 0x7f11400ffbe0 2023-07-21T18:41:00.974 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.945+0000 7f1147841700 1 -- 172.21.15.39:0/1695775236 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f114019de70 con 0x7f11400ffbe0 2023-07-21T18:41:00.974 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.946+0000 7f1147841700 1 -- 172.21.15.39:0/1695775236 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1140105f20 con 0x7f11400ffbe0 2023-07-21T18:41:00.974 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.947+0000 7f11367fc700 1 -- 172.21.15.39:0/1695775236 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f113c00cd40 con 0x7f11400ffbe0 2023-07-21T18:41:00.975 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.947+0000 7f11367fc700 1 --2- 172.21.15.39:0/1695775236 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f112c0a12a0 0x7f112c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:00.975 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.947+0000 7f11367fc700 1 -- 172.21.15.39:0/1695775236 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f113c014070 con 0x7f11400ffbe0 2023-07-21T18:41:00.975 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.947+0000 7f11455dd700 1 --2- 172.21.15.39:0/1695775236 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f112c0a12a0 0x7f112c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:00.975 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.948+0000 7f11455dd700 1 --2- 172.21.15.39:0/1695775236 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f112c0a12a0 0x7f112c0a3750 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1130006010 tx=0x7f113000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:00.976 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:00.951+0000 7f11367fc700 1 -- 172.21.15.39:0/1695775236 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f113c051790 con 0x7f11400ffbe0 2023-07-21T18:41:01.189 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:01 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/3209722474' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T18:41:01.189 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:01 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/378083044' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:41:01.189 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:01 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/3700900770' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T18:41:01.223 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.199+0000 7f1147841700 1 -- 172.21.15.39:0/1695775236 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f114002c850 con 0x7f11400ffbe0 2023-07-21T18:41:01.224 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.199+0000 7f11367fc700 1 -- 172.21.15.39:0/1695775236 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10887 (secure 0 0 0) 0x7f113c099e30 con 0x7f11400ffbe0 2023-07-21T18:41:01.224 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:41:01.225 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/147318681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":147318681},{"type":"v1","addr":"172.21.15.39:6835","nonce":147318681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/1062891760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1062891760},{"type":"v1","addr":"172.21.15.39:6837","nonce":1062891760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T18:40:04.594553+0000","modified":"2023-07-21T18:40:11.989039+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24536,"mds_1":24500,"mds_2":24487,"mds_3":24464,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/3184273349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":3184273349},{"type":"v1","addr":"172.21.15.39:6839","nonce":3184273349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6841/640981220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":640981220},{"type":"v1","addr":"172.21.15.39:6841","nonce":640981220}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3106166804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3106166804},{"type":"v1","addr":"172.21.15.193:6833","nonce":3106166804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24437":{"gid":24437,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/1034755277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1034755277},{"type":"v1","addr":"172.21.15.155:6833","nonce":1034755277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/381560657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":381560657},{"type":"v1","addr":"172.21.15.193:6835","nonce":381560657}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6837/2626761843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2626761843},{"type":"v1","addr":"172.21.15.193:6837","nonce":2626761843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24464":{"gid":24464,"name":"e","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.155:6835/3854856286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3854856286},{"type":"v1","addr":"172.21.15.155:6835","nonce":3854856286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.193:6839/3148488639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3148488639},{"type":"v1","addr":"172.21.15.193:6839","nonce":3148488639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.155:6837/947670056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":947670056},{"type":"v1","addr":"172.21.15.155:6837","nonce":947670056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6839/431690342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":431690342},{"type":"v1","addr":"172.21.15.155:6839","nonce":431690342}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T18:41:01.226 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.202+0000 7f1147841700 1 -- 172.21.15.39:0/1695775236 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f112c0a12a0 msgr2=0x7f112c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:01.227 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.202+0000 7f1147841700 1 --2- 172.21.15.39:0/1695775236 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f112c0a12a0 0x7f112c0a3750 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1130006010 tx=0x7f113000b260 comp rx=0 tx=0).stop 2023-07-21T18:41:01.227 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.202+0000 7f1147841700 1 -- 172.21.15.39:0/1695775236 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11400ffbe0 msgr2=0x7f114019a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:01.227 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.202+0000 7f1147841700 1 --2- 172.21.15.39:0/1695775236 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11400ffbe0 0x7f114019a530 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f113c00e9a0 tx=0x7f113c00ed60 comp rx=0 tx=0).stop 2023-07-21T18:41:01.227 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.202+0000 7f1147841700 1 -- 172.21.15.39:0/1695775236 shutdown_connections 2023-07-21T18:41:01.227 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.202+0000 7f1147841700 1 --2- 172.21.15.39:0/1695775236 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f112c0a12a0 0x7f112c0a3750 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.227 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.202+0000 7f1147841700 1 --2- 172.21.15.39:0/1695775236 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1140100570 0x7f114019cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.227 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.202+0000 7f1147841700 1 --2- 172.21.15.39:0/1695775236 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11400fe9f0 0x7f1140069230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.228 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.202+0000 7f1147841700 1 --2- 172.21.15.39:0/1695775236 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11400ffbe0 0x7f114019a530 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.228 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.202+0000 7f1147841700 1 -- 172.21.15.39:0/1695775236 >> 172.21.15.39:0/1695775236 conn(0x7f11400f9fd0 msgr2=0x7f1140103060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:01.228 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.202+0000 7f1147841700 1 -- 172.21.15.39:0/1695775236 shutdown_connections 2023-07-21T18:41:01.228 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.202+0000 7f1147841700 1 -- 172.21.15.39:0/1695775236 wait complete. 2023-07-21T18:41:01.228 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 26 2023-07-21T18:41:01.239 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-07-21T18:40:04.594553+0000', 'modified': '2023-07-21T18:40:11.989039+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24536, 'mds_1': 24500, 'mds_2': 24487, 'mds_3': 24464, 'mds_4': 14604}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14592': {'gid': 14592, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.39:6839/3184273349', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6838', 'nonce': 3184273349}, {'type': 'v1', 'addr': '172.21.15.39:6839', 'nonce': 3184273349}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_14604': {'gid': 14604, 'name': 'a', 'rank': 4, 'incarnation': 24, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.39:6841/640981220', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6840', 'nonce': 640981220}, {'type': 'v1', 'addr': '172.21.15.39:6841', 'nonce': 640981220}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24415': {'gid': 24415, 'name': 'l', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.193:6833/3106166804', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6832', 'nonce': 3106166804}, {'type': 'v1', 'addr': '172.21.15.193:6833', 'nonce': 3106166804}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24437': {'gid': 24437, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.155:6833/1034755277', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6832', 'nonce': 1034755277}, {'type': 'v1', 'addr': '172.21.15.155:6833', 'nonce': 1034755277}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24451': {'gid': 24451, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.193:6835/381560657', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6834', 'nonce': 381560657}, {'type': 'v1', 'addr': '172.21.15.193:6835', 'nonce': 381560657}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24463': {'gid': 24463, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.193:6837/2626761843', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6836', 'nonce': 2626761843}, {'type': 'v1', 'addr': '172.21.15.193:6837', 'nonce': 2626761843}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24464': {'gid': 24464, 'name': 'e', 'rank': 3, 'incarnation': 22, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.155:6835/3854856286', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 3854856286}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 3854856286}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24487': {'gid': 24487, 'name': 'i', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.193:6839/3148488639', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6838', 'nonce': 3148488639}, {'type': 'v1', 'addr': '172.21.15.193:6839', 'nonce': 3148488639}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24500': {'gid': 24500, 'name': 'h', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.155:6837/947670056', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6836', 'nonce': 947670056}, {'type': 'v1', 'addr': '172.21.15.155:6837', 'nonce': 947670056}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24536': {'gid': 24536, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.155:6839/431690342', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6838', 'nonce': 431690342}, {'type': 'v1', 'addr': '172.21.15.155:6839', 'nonce': 431690342}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-07-21T18:41:01.239 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2023-07-21T18:41:01.240 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-07-21T18:41:01.371 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:01 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/3209722474' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T18:41:01.372 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:01 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/378083044' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:41:01.372 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:01 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/3700900770' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T18:41:01.377 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.352+0000 7ff91e82b700 1 -- 172.21.15.39:0/1732251869 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9181057a0 msgr2=0x7ff918107c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:01.377 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.352+0000 7ff91e82b700 1 --2- 172.21.15.39:0/1732251869 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9181057a0 0x7ff918107c00 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff91000b0a0 tx=0x7ff91000b3b0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.378 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.352+0000 7ff91e82b700 1 -- 172.21.15.39:0/1732251869 shutdown_connections 2023-07-21T18:41:01.378 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.352+0000 7ff91e82b700 1 --2- 172.21.15.39:0/1732251869 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9181057a0 0x7ff918107c00 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.378 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.352+0000 7ff91e82b700 1 --2- 172.21.15.39:0/1732251869 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff918100560 0x7ff918102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.378 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.352+0000 7ff91e82b700 1 --2- 172.21.15.39:0/1732251869 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff918102e80 0x7ff918105260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.378 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.352+0000 7ff91e82b700 1 -- 172.21.15.39:0/1732251869 >> 172.21.15.39:0/1732251869 conn(0x7ff9180fa000 msgr2=0x7ff9180fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:01.378 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.353+0000 7ff91e82b700 1 -- 172.21.15.39:0/1732251869 shutdown_connections 2023-07-21T18:41:01.378 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.353+0000 7ff91e82b700 1 -- 172.21.15.39:0/1732251869 wait complete. 2023-07-21T18:41:01.380 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.353+0000 7ff91e82b700 1 Processor -- start 2023-07-21T18:41:01.380 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.353+0000 7ff91e82b700 1 -- start start 2023-07-21T18:41:01.380 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.353+0000 7ff91e82b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff918100560 0x7ff918195910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.381 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.353+0000 7ff91e82b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff918102e80 0x7ff918195e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.381 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.353+0000 7ff91e82b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9181057a0 0x7ff91819cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.381 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.354+0000 7ff91e82b700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff91819d490 con 0x7ff918100560 2023-07-21T18:41:01.382 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.354+0000 7ff91e82b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff91819d5d0 con 0x7ff9181057a0 2023-07-21T18:41:01.382 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.354+0000 7ff91e82b700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff91819d760 con 0x7ff918102e80 2023-07-21T18:41:01.382 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.354+0000 7ff917fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff918100560 0x7ff918195910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:01.382 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.354+0000 7ff917fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff918100560 0x7ff918195910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55878/0 (socket says 172.21.15.39:55878) 2023-07-21T18:41:01.382 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.354+0000 7ff917fff700 1 -- 172.21.15.39:0/579481434 learned_addr learned my addr 172.21.15.39:0/579481434 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:01.383 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.354+0000 7ff917fff700 1 -- 172.21.15.39:0/579481434 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff918102e80 msgr2=0x7ff918195e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:01.383 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.354+0000 7ff917fff700 1 --2- 172.21.15.39:0/579481434 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff918102e80 0x7ff918195e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.383 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.354+0000 7ff917fff700 1 -- 172.21.15.39:0/579481434 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9181057a0 msgr2=0x7ff91819cec0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-21T18:41:01.383 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.354+0000 7ff917fff700 1 --2- 172.21.15.39:0/579481434 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9181057a0 0x7ff91819cec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.384 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.354+0000 7ff917fff700 1 -- 172.21.15.39:0/579481434 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff910009d00 con 0x7ff918100560 2023-07-21T18:41:01.384 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.354+0000 7ff917fff700 1 --2- 172.21.15.39:0/579481434 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff918100560 0x7ff918195910 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff90800e9a0 tx=0x7ff90800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:01.384 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.354+0000 7ff9157fa700 1 -- 172.21.15.39:0/579481434 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff908009790 con 0x7ff918100560 2023-07-21T18:41:01.384 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.354+0000 7ff9157fa700 1 -- 172.21.15.39:0/579481434 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff90800cbd0 con 0x7ff918100560 2023-07-21T18:41:01.385 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.354+0000 7ff91e82b700 1 -- 172.21.15.39:0/579481434 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff91819da40 con 0x7ff918100560 2023-07-21T18:41:01.385 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.354+0000 7ff9157fa700 1 -- 172.21.15.39:0/579481434 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff908018680 con 0x7ff918100560 2023-07-21T18:41:01.385 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.354+0000 7ff91e82b700 1 -- 172.21.15.39:0/579481434 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff91819dfa0 con 0x7ff918100560 2023-07-21T18:41:01.390 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.357+0000 7ff9157fa700 1 -- 172.21.15.39:0/579481434 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7ff90800cd40 con 0x7ff918100560 2023-07-21T18:41:01.390 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.357+0000 7ff9157fa700 1 --2- 172.21.15.39:0/579481434 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7ff9000a1260 0x7ff9000a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.390 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.357+0000 7ff9157fa700 1 -- 172.21.15.39:0/579481434 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff908014070 con 0x7ff918100560 2023-07-21T18:41:01.391 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.357+0000 7ff9177fe700 1 --2- 172.21.15.39:0/579481434 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7ff9000a1260 0x7ff9000a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:01.391 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.358+0000 7ff9177fe700 1 --2- 172.21.15.39:0/579481434 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7ff9000a1260 0x7ff9000a3710 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff90c005950 tx=0x7ff90c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:01.391 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.358+0000 7ff92a9b1b80 1 Processor -- start 2023-07-21T18:41:01.391 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.358+0000 7ff92a9b1b80 1 -- start start 2023-07-21T18:41:01.392 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.358+0000 7ff92a9b1b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559108ebf890 0x559108ebfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.392 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.358+0000 7ff92a9b1b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559108ec01e0 0x559108ec0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.392 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.358+0000 7ff92a9b1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559108ec1030 0x559108ec14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.393 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.358+0000 7ff92a9b1b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x559108dc0c60 con 0x559108ebf890 2023-07-21T18:41:01.393 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.358+0000 7ff92a9b1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559108e139f0 con 0x559108ec1030 2023-07-21T18:41:01.393 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.358+0000 7ff92a9b1b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x559108e0ce80 con 0x559108ec01e0 2023-07-21T18:41:01.393 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.358+0000 7ff917fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559108ebf890 0x559108ebfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:01.394 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.358+0000 7ff917fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559108ebf890 0x559108ebfca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55886/0 (socket says 172.21.15.39:55886) 2023-07-21T18:41:01.394 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.358+0000 7ff917fff700 1 -- 172.21.15.39:0/2251468062 learned_addr learned my addr 172.21.15.39:0/2251468062 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:01.394 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.358+0000 7ff91cdc8700 1 --2- 172.21.15.39:0/2251468062 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559108ec1030 0x559108ec14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:01.395 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.358+0000 7ff917fff700 1 -- 172.21.15.39:0/2251468062 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559108ec01e0 msgr2=0x559108ec0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:01.395 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.358+0000 7ff917fff700 1 --2- 172.21.15.39:0/2251468062 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559108ec01e0 0x559108ec0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.395 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.358+0000 7ff917fff700 1 -- 172.21.15.39:0/2251468062 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559108ec1030 msgr2=0x559108ec14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:01.395 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.358+0000 7ff917fff700 1 --2- 172.21.15.39:0/2251468062 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559108ec1030 0x559108ec14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.396 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.359+0000 7ff917fff700 1 -- 172.21.15.39:0/2251468062 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559108c3ad90 con 0x559108ebf890 2023-07-21T18:41:01.396 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.359+0000 7ff91cdc8700 1 --2- 172.21.15.39:0/2251468062 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559108ec1030 0x559108ec14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:41:01.396 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.359+0000 7ff917fff700 1 --2- 172.21.15.39:0/2251468062 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559108ebf890 0x559108ebfca0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff9080a1f70 tx=0x7ff908099cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56963777e5cfd01a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:01.396 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.359+0000 7ff8fe7fc700 1 -- 172.21.15.39:0/2251468062 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff908099540 con 0x559108ebf890 2023-07-21T18:41:01.397 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.359+0000 7ff8fe7fc700 1 -- 172.21.15.39:0/2251468062 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9080996a0 con 0x559108ebf890 2023-07-21T18:41:01.397 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.359+0000 7ff8fe7fc700 1 -- 172.21.15.39:0/2251468062 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff908099310 con 0x559108ebf890 2023-07-21T18:41:01.397 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.359+0000 7ff92a9b1b80 1 -- 172.21.15.39:0/2251468062 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559108ebf890 msgr2=0x559108ebfca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:01.397 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.359+0000 7ff92a9b1b80 1 --2- 172.21.15.39:0/2251468062 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559108ebf890 0x559108ebfca0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff9080a1f70 tx=0x7ff908099cc0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.398 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.360+0000 7ff92a9b1b80 1 -- 172.21.15.39:0/2251468062 shutdown_connections 2023-07-21T18:41:01.398 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.360+0000 7ff92a9b1b80 1 --2- 172.21.15.39:0/2251468062 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559108ec1030 0x559108ec14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.398 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.360+0000 7ff92a9b1b80 1 --2- 172.21.15.39:0/2251468062 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559108ebf890 0x559108ebfca0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.398 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.360+0000 7ff92a9b1b80 1 --2- 172.21.15.39:0/2251468062 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559108ec01e0 0x559108ec0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.398 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.360+0000 7ff92a9b1b80 1 -- 172.21.15.39:0/2251468062 >> 172.21.15.39:0/2251468062 conn(0x559108eb0d50 msgr2=0x559108eb6180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:01.399 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.360+0000 7ff92a9b1b80 1 -- 172.21.15.39:0/2251468062 shutdown_connections 2023-07-21T18:41:01.399 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.360+0000 7ff92a9b1b80 1 -- 172.21.15.39:0/2251468062 wait complete. 2023-07-21T18:41:01.399 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.360+0000 7ff92a9b1b80 1 -- start start 2023-07-21T18:41:01.399 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.360+0000 7ff92a9b1b80 1 Processor -- start 2023-07-21T18:41:01.399 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.361+0000 7ff92a9b1b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559108ebf890 0x559108ec94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.399 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.361+0000 7ff92a9b1b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559108ec01e0 0x559108ec9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.399 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.361+0000 7ff92a9b1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559108ec1030 0x559108ed0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.399 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.361+0000 7ff92a9b1b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x559108daca70 con 0x559108ec01e0 2023-07-21T18:41:01.400 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.361+0000 7ff92a9b1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559108dac770 con 0x559108ec1030 2023-07-21T18:41:01.400 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.361+0000 7ff92a9b1b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x559108c908d0 con 0x559108ebf890 2023-07-21T18:41:01.400 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.361+0000 7ff92a9b1b80 10 client.0 _ll_register_callbacks cb 0x559108daa710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:41:01.400 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.361+0000 7ff9177fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559108ec01e0 0x559108ec9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:01.400 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.361+0000 7ff9177fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559108ec01e0 0x559108ec9a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55894/0 (socket says 172.21.15.39:55894) 2023-07-21T18:41:01.400 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.361+0000 7ff9177fe700 1 -- 172.21.15.39:0/1907236935 learned_addr learned my addr 172.21.15.39:0/1907236935 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:01.400 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.361+0000 7ff91cdc8700 1 --2- 172.21.15.39:0/1907236935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559108ec1030 0x559108ed0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:01.401 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.361+0000 7ff917fff700 1 --2- 172.21.15.39:0/1907236935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559108ebf890 0x559108ec94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:01.401 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.361+0000 7ff9177fe700 1 -- 172.21.15.39:0/1907236935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559108ebf890 msgr2=0x559108ec94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:01.401 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.361+0000 7ff9177fe700 1 --2- 172.21.15.39:0/1907236935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559108ebf890 0x559108ec94f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.401 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.361+0000 7ff9177fe700 1 -- 172.21.15.39:0/1907236935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559108ec1030 msgr2=0x559108ed0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:01.401 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.361+0000 7ff9177fe700 1 --2- 172.21.15.39:0/1907236935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559108ec1030 0x559108ed0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.401 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.361+0000 7ff9177fe700 1 -- 172.21.15.39:0/1907236935 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff91819dfa0 con 0x559108ec01e0 2023-07-21T18:41:01.402 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.361+0000 7ff9177fe700 1 --2- 172.21.15.39:0/1907236935 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559108ec01e0 0x559108ec9a30 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff90c00ddd0 tx=0x7ff90c010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3266d3632fe57a9d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:01.402 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.361+0000 7ff8f77fe700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:41:01.402 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.361+0000 7ff8f77fe700 1 -- 172.21.15.39:0/1907236935 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90c010e70 con 0x559108ec01e0 2023-07-21T18:41:01.402 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.361+0000 7ff8f77fe700 1 -- 172.21.15.39:0/1907236935 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff90c01d440 con 0x559108ec01e0 2023-07-21T18:41:01.402 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.361+0000 7ff92a9b1b80 1 -- 172.21.15.39:0/1907236935 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559108d664a0 con 0x559108ec01e0 2023-07-21T18:41:01.402 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.361+0000 7ff8f77fe700 1 -- 172.21.15.39:0/1907236935 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff90c00deb0 con 0x559108ec01e0 2023-07-21T18:41:01.402 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.362+0000 7ff92a9b1b80 10 client.15018 fetch_fsmap learned FSMap version 26 2023-07-21T18:41:01.403 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.362+0000 7ff92a9b1b80 1 -- 172.21.15.39:0/1907236935 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x559108c23c10 con 0x559108ec01e0 2023-07-21T18:41:01.403 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.362+0000 7ff8f77fe700 1 -- 172.21.15.39:0/1907236935 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559108d664a0 con 0x559108ec01e0 2023-07-21T18:41:01.403 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.362+0000 7ff8f77fe700 1 -- 172.21.15.39:0/1907236935 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff90c02a820 con 0x559108ec01e0 2023-07-21T18:41:01.403 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.362+0000 7ff92a9b1b80 10 client.15018 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T18:41:01.403 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.362+0000 7ff92a9b1b80 10 client.15018 resolve_mds: resolved name 'a' to daemon mds.a 2023-07-21T18:41:01.403 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.362+0000 7ff92a9b1b80 20 client.15018 populate_metadata read hostname 'smithi039' 2023-07-21T18:41:01.403 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.362+0000 7ff92a9b1b80 1 --2- 172.21.15.39:0/1907236935 >> [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] conn(0x559108ed4d40 0x559108ed3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.403 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.362+0000 7ff92a9b1b80 4 client.15018 mds_command: new command op to 14604 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:41:01.404 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.362+0000 7ff92a9b1b80 1 -- 172.21.15.39:0/1907236935 --> [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559108e05e60 con 0x559108ed4d40 2023-07-21T18:41:01.404 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.362+0000 7ff917fff700 1 --2- 172.21.15.39:0/1907236935 >> [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] conn(0x559108ed4d40 0x559108ed3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:01.404 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.362+0000 7ff8f77fe700 10 client.15018 ms_handle_connect on v2:172.21.15.39:6840/640981220 2023-07-21T18:41:01.404 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.362+0000 7ff917fff700 1 --2- 172.21.15.39:0/1907236935 >> [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] conn(0x559108ed4d40 0x559108ed3b40 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7c3de7b2177718c9 server_cookie=ab8dadfe9700443a in_seq=0 out_seq=0 2023-07-21T18:41:01.404 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.363+0000 7ff8f77fe700 1 -- 172.21.15.39:0/1907236935 <== mds.4 v2:172.21.15.39:6840/640981220 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559108e05e60 con 0x559108ed4d40 2023-07-21T18:41:01.404 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.363+0000 7ff8f77fe700 10 client.15018 handle_command_reply: tid=0 2023-07-21T18:41:01.405 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.363+0000 7ff92a9b1b80 1 client.15018 shutdown 2023-07-21T18:41:01.405 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.363+0000 7ff92a9b1b80 1 -- 172.21.15.39:0/1907236935 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559108ec01e0 msgr2=0x559108ec9a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:01.405 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.363+0000 7ff92a9b1b80 1 --2- 172.21.15.39:0/1907236935 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559108ec01e0 0x559108ec9a30 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff90c00ddd0 tx=0x7ff90c010740 comp rx=0 tx=0).stop 2023-07-21T18:41:01.405 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.363+0000 7ff92a9b1b80 1 -- 172.21.15.39:0/1907236935 shutdown_connections 2023-07-21T18:41:01.405 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.363+0000 7ff92a9b1b80 1 --2- 172.21.15.39:0/1907236935 >> [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] conn(0x559108ed4d40 0x559108ed3b40 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.405 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.363+0000 7ff92a9b1b80 1 --2- 172.21.15.39:0/1907236935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559108ec1030 0x559108ed0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.405 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.363+0000 7ff92a9b1b80 1 --2- 172.21.15.39:0/1907236935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559108ebf890 0x559108ec94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.406 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.363+0000 7ff8f77fe700 0 client.15018 ms_handle_reset on v2:172.21.15.39:6840/640981220 2023-07-21T18:41:01.406 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.363+0000 7ff92a9b1b80 1 --2- 172.21.15.39:0/1907236935 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559108ec01e0 0x559108ec9a30 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.406 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.363+0000 7ff92a9b1b80 1 -- 172.21.15.39:0/1907236935 >> 172.21.15.39:0/1907236935 conn(0x559108eb0d50 msgr2=0x559108eb5860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:01.406 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.364+0000 7ff92a9b1b80 1 -- 172.21.15.39:0/1907236935 shutdown_connections 2023-07-21T18:41:01.406 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.364+0000 7ff92a9b1b80 1 -- 172.21.15.39:0/1907236935 wait complete. 2023-07-21T18:41:01.406 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.364+0000 7ff92a9b1b80 20 client.15018 trim_cache size 0 max 16384 2023-07-21T18:41:01.407 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:01 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/3209722474' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T18:41:01.407 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:01 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/378083044' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:41:01.407 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:01 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/3700900770' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T18:41:01.411 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.381+0000 7ff92a9b1b80 1 Processor -- start 2023-07-21T18:41:01.411 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.381+0000 7ff92a9b1b80 1 -- start start 2023-07-21T18:41:01.412 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.381+0000 7ff92a9b1b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559108ebf890 0x559108ec81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.412 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.381+0000 7ff92a9b1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559108ec01e0 0x559108ed0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.412 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.381+0000 7ff92a9b1b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559108ec1030 0x559108ed1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.412 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.381+0000 7ff92a9b1b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x559108eb7cf0 con 0x559108ebf890 2023-07-21T18:41:01.412 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.381+0000 7ff92a9b1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559108ddb790 con 0x559108ec01e0 2023-07-21T18:41:01.412 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.381+0000 7ff92a9b1b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x559108c9cde0 con 0x559108ec1030 2023-07-21T18:41:01.413 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.382+0000 7ff917fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559108ebf890 0x559108ec81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:01.413 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.382+0000 7ff917fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559108ebf890 0x559108ec81b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55900/0 (socket says 172.21.15.39:55900) 2023-07-21T18:41:01.413 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.382+0000 7ff917fff700 1 -- 172.21.15.39:0/3138083723 learned_addr learned my addr 172.21.15.39:0/3138083723 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:01.413 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.382+0000 7ff9177fe700 1 --2- 172.21.15.39:0/3138083723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559108ec01e0 0x559108ed0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:01.413 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.382+0000 7ff917fff700 1 -- 172.21.15.39:0/3138083723 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559108ec1030 msgr2=0x559108ed1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:01.413 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.382+0000 7ff91cdc8700 1 --2- 172.21.15.39:0/3138083723 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559108ec1030 0x559108ed1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:01.413 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.382+0000 7ff917fff700 1 --2- 172.21.15.39:0/3138083723 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559108ec1030 0x559108ed1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.414 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.382+0000 7ff917fff700 1 -- 172.21.15.39:0/3138083723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559108ec01e0 msgr2=0x559108ed0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:01.414 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.382+0000 7ff917fff700 1 --2- 172.21.15.39:0/3138083723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559108ec01e0 0x559108ed0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.414 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.382+0000 7ff917fff700 1 -- 172.21.15.39:0/3138083723 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559108c61200 con 0x559108ebf890 2023-07-21T18:41:01.414 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.382+0000 7ff91cdc8700 1 --2- 172.21.15.39:0/3138083723 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559108ec1030 0x559108ed1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:41:01.414 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.382+0000 7ff917fff700 1 --2- 172.21.15.39:0/3138083723 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559108ebf890 0x559108ec81b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff908099c90 tx=0x7ff908099c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d1f595975c8ffe5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:01.414 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.382+0000 7ff8feffd700 1 -- 172.21.15.39:0/3138083723 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff908099540 con 0x559108ebf890 2023-07-21T18:41:01.414 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.382+0000 7ff8feffd700 1 -- 172.21.15.39:0/3138083723 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9080983f0 con 0x559108ebf890 2023-07-21T18:41:01.415 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.382+0000 7ff8feffd700 1 -- 172.21.15.39:0/3138083723 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff908099810 con 0x559108ebf890 2023-07-21T18:41:01.415 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.383+0000 7ff92a9b1b80 1 -- 172.21.15.39:0/3138083723 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559108ebf890 msgr2=0x559108ec81b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:01.415 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.383+0000 7ff92a9b1b80 1 --2- 172.21.15.39:0/3138083723 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559108ebf890 0x559108ec81b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff908099c90 tx=0x7ff908099c20 comp rx=0 tx=0).stop 2023-07-21T18:41:01.415 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.383+0000 7ff92a9b1b80 1 -- 172.21.15.39:0/3138083723 shutdown_connections 2023-07-21T18:41:01.415 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.383+0000 7ff92a9b1b80 1 --2- 172.21.15.39:0/3138083723 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559108ec1030 0x559108ed1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.415 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.383+0000 7ff92a9b1b80 1 --2- 172.21.15.39:0/3138083723 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559108ebf890 0x559108ec81b0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.415 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.383+0000 7ff92a9b1b80 1 --2- 172.21.15.39:0/3138083723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559108ec01e0 0x559108ed0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.416 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.383+0000 7ff92a9b1b80 1 -- 172.21.15.39:0/3138083723 >> 172.21.15.39:0/3138083723 conn(0x559108eb0d50 msgr2=0x559108ec74d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:01.416 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.383+0000 7ff92a9b1b80 1 -- 172.21.15.39:0/3138083723 shutdown_connections 2023-07-21T18:41:01.416 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.383+0000 7ff92a9b1b80 1 -- 172.21.15.39:0/3138083723 wait complete. 2023-07-21T18:41:01.416 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.383+0000 7ff92a9b1b80 1 -- start start 2023-07-21T18:41:01.416 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.383+0000 7ff92a9b1b80 1 Processor -- start 2023-07-21T18:41:01.416 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.384+0000 7ff92a9b1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559108ebf890 0x559108ecdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.416 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.384+0000 7ff92a9b1b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559108ec01e0 0x559108ece070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.417 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.384+0000 7ff92a9b1b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559108ec1030 0x559108ecf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.417 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.384+0000 7ff92a9b1b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x559108ecf5c0 con 0x559108ec01e0 2023-07-21T18:41:01.417 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.384+0000 7ff92a9b1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559108ecf730 con 0x559108ebf890 2023-07-21T18:41:01.417 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.384+0000 7ff92a9b1b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x559108ecf8f0 con 0x559108ec1030 2023-07-21T18:41:01.417 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.384+0000 7ff92a9b1b80 10 client.0 _ll_register_callbacks cb 0x7ff90c00d980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:41:01.417 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.384+0000 7ff9177fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559108ec01e0 0x559108ece070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:01.417 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.384+0000 7ff9177fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559108ec01e0 0x559108ece070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55908/0 (socket says 172.21.15.39:55908) 2023-07-21T18:41:01.418 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.384+0000 7ff9177fe700 1 -- 172.21.15.39:0/29914227 learned_addr learned my addr 172.21.15.39:0/29914227 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:01.418 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.384+0000 7ff917fff700 1 --2- 172.21.15.39:0/29914227 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559108ebf890 0x559108ecdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:01.418 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.384+0000 7ff9177fe700 1 -- 172.21.15.39:0/29914227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559108ec1030 msgr2=0x559108ecf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:01.418 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.384+0000 7ff9177fe700 1 --2- 172.21.15.39:0/29914227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559108ec1030 0x559108ecf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.418 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.384+0000 7ff9177fe700 1 -- 172.21.15.39:0/29914227 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559108ebf890 msgr2=0x559108ecdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:01.418 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.384+0000 7ff9177fe700 1 --2- 172.21.15.39:0/29914227 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559108ebf890 0x559108ecdb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.418 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.384+0000 7ff9177fe700 1 -- 172.21.15.39:0/29914227 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559108c3ad90 con 0x559108ec01e0 2023-07-21T18:41:01.418 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.384+0000 7ff917fff700 1 --2- 172.21.15.39:0/29914227 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559108ebf890 0x559108ecdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:41:01.419 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.384+0000 7ff9177fe700 1 --2- 172.21.15.39:0/29914227 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559108ec01e0 0x559108ece070 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff90c005a80 tx=0x7ff90c02a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c7d2609c6a688f4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:01.419 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.384+0000 7ff8f77fe700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:41:01.419 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.384+0000 7ff8f77fe700 1 -- 172.21.15.39:0/29914227 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90c01d640 con 0x559108ec01e0 2023-07-21T18:41:01.419 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.384+0000 7ff8f77fe700 1 -- 172.21.15.39:0/29914227 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff90c01d7a0 con 0x559108ec01e0 2023-07-21T18:41:01.419 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.384+0000 7ff92a9b1b80 1 -- 172.21.15.39:0/29914227 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff9080996a0 con 0x559108ec01e0 2023-07-21T18:41:01.419 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.384+0000 7ff8f77fe700 1 -- 172.21.15.39:0/29914227 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff90c01d960 con 0x559108ec01e0 2023-07-21T18:41:01.419 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.384+0000 7ff92a9b1b80 10 client.15024 fetch_fsmap learned FSMap version 26 2023-07-21T18:41:01.420 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.385+0000 7ff92a9b1b80 1 -- 172.21.15.39:0/29914227 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x559108c39260 con 0x559108ec01e0 2023-07-21T18:41:01.420 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.385+0000 7ff8f77fe700 1 -- 172.21.15.39:0/29914227 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff90c045020 con 0x559108ec01e0 2023-07-21T18:41:01.420 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.385+0000 7ff8f77fe700 1 -- 172.21.15.39:0/29914227 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff90c041bc0 con 0x559108ec01e0 2023-07-21T18:41:01.420 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.385+0000 7ff92a9b1b80 10 client.15024 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T18:41:01.420 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.385+0000 7ff92a9b1b80 10 client.15024 resolve_mds: resolved name 'a' to daemon mds.a 2023-07-21T18:41:01.420 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.385+0000 7ff92a9b1b80 20 client.15024 populate_metadata read hostname 'smithi039' 2023-07-21T18:41:01.421 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.385+0000 7ff92a9b1b80 1 --2- 172.21.15.39:0/29914227 >> [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] conn(0x559108f2dcd0 0x559108f2a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.421 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.385+0000 7ff92a9b1b80 4 client.15024 mds_command: new command op to 14604 tid=0[{"prefix": "status"}] 2023-07-21T18:41:01.421 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.385+0000 7ff92a9b1b80 1 -- 172.21.15.39:0/29914227 --> [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559108dc8000 con 0x559108f2dcd0 2023-07-21T18:41:01.421 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.385+0000 7ff917fff700 1 --2- 172.21.15.39:0/29914227 >> [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] conn(0x559108f2dcd0 0x559108f2a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:01.421 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.385+0000 7ff917fff700 1 --2- 172.21.15.39:0/29914227 >> [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] conn(0x559108f2dcd0 0x559108f2a5e0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b4f2826d66da82f3 server_cookie=716c1cd61c039472 in_seq=0 out_seq=0 2023-07-21T18:41:01.421 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.385+0000 7ff8f77fe700 10 client.15024 ms_handle_connect on v2:172.21.15.39:6840/640981220 2023-07-21T18:41:01.421 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.385+0000 7ff8f77fe700 1 -- 172.21.15.39:0/29914227 <== mds.4 v2:172.21.15.39:6840/640981220 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x559108dc8000 con 0x559108f2dcd0 2023-07-21T18:41:01.422 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.385+0000 7ff8f77fe700 10 client.15024 handle_command_reply: tid=0 2023-07-21T18:41:01.422 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.386+0000 7ff92a9b1b80 1 client.15024 shutdown 2023-07-21T18:41:01.422 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.386+0000 7ff92a9b1b80 1 -- 172.21.15.39:0/29914227 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559108ec01e0 msgr2=0x559108ece070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:01.422 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.386+0000 7ff92a9b1b80 1 --2- 172.21.15.39:0/29914227 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559108ec01e0 0x559108ece070 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff90c005a80 tx=0x7ff90c02a760 comp rx=0 tx=0).stop 2023-07-21T18:41:01.422 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.386+0000 7ff92a9b1b80 1 -- 172.21.15.39:0/29914227 shutdown_connections 2023-07-21T18:41:01.422 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.386+0000 7ff92a9b1b80 1 --2- 172.21.15.39:0/29914227 >> [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] conn(0x559108f2dcd0 0x559108f2a5e0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.422 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.386+0000 7ff92a9b1b80 1 --2- 172.21.15.39:0/29914227 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559108ebf890 0x559108ecdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.423 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.386+0000 7ff92a9b1b80 1 --2- 172.21.15.39:0/29914227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559108ec1030 0x559108ecf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.423 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.386+0000 7ff92a9b1b80 1 --2- 172.21.15.39:0/29914227 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559108ec01e0 0x559108ece070 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.423 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.386+0000 7ff8f77fe700 0 client.15024 ms_handle_reset on v2:172.21.15.39:6840/640981220 2023-07-21T18:41:01.423 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.386+0000 7ff92a9b1b80 1 -- 172.21.15.39:0/29914227 >> 172.21.15.39:0/29914227 conn(0x559108eb0d50 msgr2=0x559108ec81b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:01.423 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.386+0000 7ff92a9b1b80 1 -- 172.21.15.39:0/29914227 shutdown_connections 2023-07-21T18:41:01.423 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.386+0000 7ff92a9b1b80 1 -- 172.21.15.39:0/29914227 wait complete. 2023-07-21T18:41:01.423 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.386+0000 7ff92a9b1b80 20 client.15024 trim_cache size 0 max 16384 2023-07-21T18:41:01.424 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2023-07-21T18:41:01.424 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "eaf8e25a-27f4-11ee-9b34-001a4aab830c", 2023-07-21T18:41:01.424 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 4, 2023-07-21T18:41:01.424 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 14604, 2023-07-21T18:41:01.424 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2023-07-21T18:41:01.425 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2023-07-21T18:41:01.425 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2023-07-21T18:41:01.425 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 50.421967866000003, 2023-07-21T18:41:01.425 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 26, 2023-07-21T18:41:01.425 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 79, 2023-07-21T18:41:01.425 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 79, 2023-07-21T18:41:01.425 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 65.350959666999998 2023-07-21T18:41:01.425 INFO:teuthology.orchestra.run.smithi039.stdout:} 2023-07-21T18:41:01.427 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.386+0000 7ff91e82b700 1 -- 172.21.15.39:0/579481434 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7ff9000a1260 msgr2=0x7ff9000a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:01.427 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.386+0000 7ff91e82b700 1 --2- 172.21.15.39:0/579481434 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7ff9000a1260 0x7ff9000a3710 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff90c005950 tx=0x7ff90c0058e0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.427 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.386+0000 7ff91e82b700 1 -- 172.21.15.39:0/579481434 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff918100560 msgr2=0x7ff918195910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:01.427 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.386+0000 7ff91e82b700 1 --2- 172.21.15.39:0/579481434 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff918100560 0x7ff918195910 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff90800e9a0 tx=0x7ff90800ed60 comp rx=0 tx=0).stop 2023-07-21T18:41:01.427 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.386+0000 7ff91e82b700 1 -- 172.21.15.39:0/579481434 shutdown_connections 2023-07-21T18:41:01.428 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.386+0000 7ff91e82b700 1 --2- 172.21.15.39:0/579481434 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7ff9000a1260 0x7ff9000a3710 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.428 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.386+0000 7ff91e82b700 1 --2- 172.21.15.39:0/579481434 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9181057a0 0x7ff91819cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.428 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.386+0000 7ff91e82b700 1 --2- 172.21.15.39:0/579481434 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff918100560 0x7ff918195910 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.428 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.386+0000 7ff91e82b700 1 --2- 172.21.15.39:0/579481434 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff918102e80 0x7ff918195e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.428 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.387+0000 7ff91e82b700 1 -- 172.21.15.39:0/579481434 >> 172.21.15.39:0/579481434 conn(0x7ff9180fa000 msgr2=0x7ff9181011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:01.428 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.387+0000 7ff91e82b700 1 -- 172.21.15.39:0/579481434 shutdown_connections 2023-07-21T18:41:01.428 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.387+0000 7ff91e82b700 1 -- 172.21.15.39:0/579481434 wait complete. 2023-07-21T18:41:01.429 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2023-07-21T18:41:01.555 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.530+0000 7fed15ab7700 1 -- 172.21.15.39:0/2670204386 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed100ffae0 msgr2=0x7fed100ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:01.556 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.530+0000 7fed15ab7700 1 --2- 172.21.15.39:0/2670204386 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed100ffae0 0x7fed100ffef0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fed00009990 tx=0x7fed00009ca0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.556 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.531+0000 7fed15ab7700 1 -- 172.21.15.39:0/2670204386 shutdown_connections 2023-07-21T18:41:01.556 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.531+0000 7fed15ab7700 1 --2- 172.21.15.39:0/2670204386 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed10101660 0x7fed10107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.556 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.531+0000 7fed15ab7700 1 --2- 172.21.15.39:0/2670204386 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed100ffae0 0x7fed100ffef0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.557 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.531+0000 7fed15ab7700 1 --2- 172.21.15.39:0/2670204386 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fed10100cd0 0x7fed10101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.557 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.531+0000 7fed15ab7700 1 -- 172.21.15.39:0/2670204386 >> 172.21.15.39:0/2670204386 conn(0x7fed100fb120 msgr2=0x7fed100fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:01.557 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.531+0000 7fed15ab7700 1 -- 172.21.15.39:0/2670204386 shutdown_connections 2023-07-21T18:41:01.557 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.531+0000 7fed15ab7700 1 -- 172.21.15.39:0/2670204386 wait complete. 2023-07-21T18:41:01.557 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.531+0000 7fed15ab7700 1 Processor -- start 2023-07-21T18:41:01.559 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.531+0000 7fed15ab7700 1 -- start start 2023-07-21T18:41:01.559 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.531+0000 7fed15ab7700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed100ffae0 0x7fed100742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.559 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.532+0000 7fed15ab7700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed10100cd0 0x7fed10072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.559 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.532+0000 7fed15ab7700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fed10101660 0x7fed10072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.559 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.532+0000 7fed15ab7700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed100735e0 con 0x7fed100ffae0 2023-07-21T18:41:01.559 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.532+0000 7fed15ab7700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed10073750 con 0x7fed10100cd0 2023-07-21T18:41:01.560 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.532+0000 7fed15ab7700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed1019f850 con 0x7fed10101660 2023-07-21T18:41:01.560 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.532+0000 7fed0f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed100ffae0 0x7fed100742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:01.560 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.532+0000 7fed0f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed100ffae0 0x7fed100742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55920/0 (socket says 172.21.15.39:55920) 2023-07-21T18:41:01.560 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.532+0000 7fed0f7fe700 1 -- 172.21.15.39:0/417981779 learned_addr learned my addr 172.21.15.39:0/417981779 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:01.560 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.532+0000 7fed0effd700 1 --2- 172.21.15.39:0/417981779 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed10100cd0 0x7fed10072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:01.560 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.532+0000 7fed0ffff700 1 --2- 172.21.15.39:0/417981779 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fed10101660 0x7fed10072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:01.560 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.532+0000 7fed0f7fe700 1 -- 172.21.15.39:0/417981779 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fed10101660 msgr2=0x7fed10072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:01.561 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.532+0000 7fed0f7fe700 1 --2- 172.21.15.39:0/417981779 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fed10101660 0x7fed10072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.561 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.532+0000 7fed0f7fe700 1 -- 172.21.15.39:0/417981779 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed10100cd0 msgr2=0x7fed10072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:01.561 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.532+0000 7fed0f7fe700 1 --2- 172.21.15.39:0/417981779 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed10100cd0 0x7fed10072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.561 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.532+0000 7fed0f7fe700 1 -- 172.21.15.39:0/417981779 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed000095f0 con 0x7fed100ffae0 2023-07-21T18:41:01.561 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.532+0000 7fed0effd700 1 --2- 172.21.15.39:0/417981779 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed10100cd0 0x7fed10072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:41:01.561 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.532+0000 7fed0f7fe700 1 --2- 172.21.15.39:0/417981779 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed100ffae0 0x7fed100742c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fed00000c00 tx=0x7fed000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:01.561 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.532+0000 7fed0cff9700 1 -- 172.21.15.39:0/417981779 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed0001d070 con 0x7fed100ffae0 2023-07-21T18:41:01.561 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.532+0000 7fed15ab7700 1 -- 172.21.15.39:0/417981779 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed1019fad0 con 0x7fed100ffae0 2023-07-21T18:41:01.562 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.532+0000 7fed0cff9700 1 -- 172.21.15.39:0/417981779 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed00003fe0 con 0x7fed100ffae0 2023-07-21T18:41:01.562 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.532+0000 7fed0cff9700 1 -- 172.21.15.39:0/417981779 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed00022df0 con 0x7fed100ffae0 2023-07-21T18:41:01.562 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.532+0000 7fed15ab7700 1 -- 172.21.15.39:0/417981779 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed1019fef0 con 0x7fed100ffae0 2023-07-21T18:41:01.562 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.534+0000 7fed0cff9700 1 -- 172.21.15.39:0/417981779 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fed00022470 con 0x7fed100ffae0 2023-07-21T18:41:01.566 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.535+0000 7fed0cff9700 1 --2- 172.21.15.39:0/417981779 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fecf80a11e0 0x7fecf80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.566 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.535+0000 7fed0cff9700 1 -- 172.21.15.39:0/417981779 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fed000cdd90 con 0x7fed100ffae0 2023-07-21T18:41:01.566 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.535+0000 7fed0effd700 1 --2- 172.21.15.39:0/417981779 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fecf80a11e0 0x7fecf80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:01.566 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.535+0000 7fed0effd700 1 --2- 172.21.15.39:0/417981779 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fecf80a11e0 0x7fecf80a3690 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fed04005fd0 tx=0x7fed04005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:01.567 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.536+0000 7fed21c3db80 1 Processor -- start 2023-07-21T18:41:01.567 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.536+0000 7fed21c3db80 1 -- start start 2023-07-21T18:41:01.567 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.536+0000 7fed21c3db80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590d62b1480 0x5590d62b1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.567 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.536+0000 7fed21c3db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5590d62b1dd0 0x5590d62b2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.567 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.536+0000 7fed21c3db80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5590d62b2c20 0x5590d62b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.567 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.536+0000 7fed21c3db80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590d61b6c60 con 0x5590d62b2c20 2023-07-21T18:41:01.567 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.536+0000 7fed21c3db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590d62099f0 con 0x5590d62b1dd0 2023-07-21T18:41:01.568 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.536+0000 7fed21c3db80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590d6202e80 con 0x5590d62b1480 2023-07-21T18:41:01.568 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.536+0000 7fed0ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5590d62b2c20 0x5590d62b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:01.568 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.536+0000 7fed0ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5590d62b2c20 0x5590d62b3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55932/0 (socket says 172.21.15.39:55932) 2023-07-21T18:41:01.568 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.536+0000 7fed0ffff700 1 -- 172.21.15.39:0/3136881842 learned_addr learned my addr 172.21.15.39:0/3136881842 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:01.568 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.536+0000 7fed0f7fe700 1 --2- 172.21.15.39:0/3136881842 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590d62b1480 0x5590d62b1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:01.568 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.536+0000 7fed0effd700 1 --2- 172.21.15.39:0/3136881842 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5590d62b1dd0 0x5590d62b2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:01.569 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.537+0000 7fed0f7fe700 1 -- 172.21.15.39:0/3136881842 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5590d62b1dd0 msgr2=0x5590d62b2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:01.569 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.537+0000 7fed0f7fe700 1 --2- 172.21.15.39:0/3136881842 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5590d62b1dd0 0x5590d62b2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.569 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.537+0000 7fed0f7fe700 1 -- 172.21.15.39:0/3136881842 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5590d62b2c20 msgr2=0x5590d62b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:01.569 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.537+0000 7fed0f7fe700 1 --2- 172.21.15.39:0/3136881842 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5590d62b2c20 0x5590d62b3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.569 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.537+0000 7fed0f7fe700 1 -- 172.21.15.39:0/3136881842 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590d6030d90 con 0x5590d62b1480 2023-07-21T18:41:01.569 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.537+0000 7fed0ffff700 1 --2- 172.21.15.39:0/3136881842 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5590d62b2c20 0x5590d62b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:41:01.569 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.537+0000 7fed0f7fe700 1 --2- 172.21.15.39:0/3136881842 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590d62b1480 0x5590d62b1890 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fed000cb8f0 tx=0x7fed0009ac60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=347d62bc55c5b921 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:01.569 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.537+0000 7fecf5ffb700 1 -- 172.21.15.39:0/3136881842 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed0009b280 con 0x5590d62b1480 2023-07-21T18:41:01.570 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.537+0000 7fecf5ffb700 1 -- 172.21.15.39:0/3136881842 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed00095e90 con 0x5590d62b1480 2023-07-21T18:41:01.570 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.537+0000 7fecf5ffb700 1 -- 172.21.15.39:0/3136881842 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed0009a9d0 con 0x5590d62b1480 2023-07-21T18:41:01.570 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.538+0000 7fed21c3db80 1 -- 172.21.15.39:0/3136881842 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590d62b1480 msgr2=0x5590d62b1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:01.570 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.538+0000 7fed21c3db80 1 --2- 172.21.15.39:0/3136881842 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590d62b1480 0x5590d62b1890 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fed000cb8f0 tx=0x7fed0009ac60 comp rx=0 tx=0).stop 2023-07-21T18:41:01.570 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.538+0000 7fed21c3db80 1 -- 172.21.15.39:0/3136881842 shutdown_connections 2023-07-21T18:41:01.570 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.538+0000 7fed21c3db80 1 --2- 172.21.15.39:0/3136881842 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5590d62b2c20 0x5590d62b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.570 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.538+0000 7fed21c3db80 1 --2- 172.21.15.39:0/3136881842 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590d62b1480 0x5590d62b1890 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.570 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.538+0000 7fed21c3db80 1 --2- 172.21.15.39:0/3136881842 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5590d62b1dd0 0x5590d62b2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.571 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.538+0000 7fed21c3db80 1 -- 172.21.15.39:0/3136881842 >> 172.21.15.39:0/3136881842 conn(0x5590d62a6d50 msgr2=0x5590d62ac180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:01.571 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.538+0000 7fed21c3db80 1 -- 172.21.15.39:0/3136881842 shutdown_connections 2023-07-21T18:41:01.571 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.538+0000 7fed21c3db80 1 -- 172.21.15.39:0/3136881842 wait complete. 2023-07-21T18:41:01.571 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.538+0000 7fed21c3db80 1 -- start start 2023-07-21T18:41:01.571 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.538+0000 7fed21c3db80 1 Processor -- start 2023-07-21T18:41:01.571 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.539+0000 7fed21c3db80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5590d62b1480 0x5590d62bad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.571 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.539+0000 7fed21c3db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5590d62b1dd0 0x5590d62bb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.572 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.539+0000 7fed21c3db80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590d62b2c20 0x5590d62c2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.572 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.539+0000 7fed21c3db80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590d61a2a70 con 0x5590d62b1480 2023-07-21T18:41:01.572 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.539+0000 7fed21c3db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590d61a2770 con 0x5590d62b1dd0 2023-07-21T18:41:01.572 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.539+0000 7fed21c3db80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590d60868d0 con 0x5590d62b2c20 2023-07-21T18:41:01.572 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.539+0000 7fed21c3db80 10 client.0 _ll_register_callbacks cb 0x5590d61a0710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:41:01.572 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.539+0000 7fed0f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5590d62b1480 0x5590d62bad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:01.572 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.539+0000 7fed0f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5590d62b1480 0x5590d62bad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55934/0 (socket says 172.21.15.39:55934) 2023-07-21T18:41:01.573 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.539+0000 7fed0f7fe700 1 -- 172.21.15.39:0/2864066313 learned_addr learned my addr 172.21.15.39:0/2864066313 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:01.573 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.539+0000 7fed0effd700 1 --2- 172.21.15.39:0/2864066313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5590d62b1dd0 0x5590d62bb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:01.573 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.539+0000 7fed0ffff700 1 --2- 172.21.15.39:0/2864066313 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590d62b2c20 0x5590d62c2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:01.573 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.539+0000 7fed0f7fe700 1 -- 172.21.15.39:0/2864066313 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590d62b2c20 msgr2=0x5590d62c2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:01.573 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.539+0000 7fed0f7fe700 1 --2- 172.21.15.39:0/2864066313 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590d62b2c20 0x5590d62c2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.573 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.539+0000 7fed0f7fe700 1 -- 172.21.15.39:0/2864066313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5590d62b1dd0 msgr2=0x5590d62bb240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:01.573 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.539+0000 7fed0f7fe700 1 --2- 172.21.15.39:0/2864066313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5590d62b1dd0 0x5590d62bb240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.573 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.539+0000 7fed0f7fe700 1 -- 172.21.15.39:0/2864066313 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed080098a0 con 0x5590d62b1480 2023-07-21T18:41:01.574 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.539+0000 7fed0ffff700 1 --2- 172.21.15.39:0/2864066313 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590d62b2c20 0x5590d62c2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:41:01.574 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.539+0000 7fed0f7fe700 1 --2- 172.21.15.39:0/2864066313 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5590d62b1480 0x5590d62bad00 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fed0000b5c0 tx=0x7fed000a33e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8377d35c51cdc0b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:01.574 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.539+0000 7feceeffd700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:41:01.574 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.539+0000 7feceeffd700 1 -- 172.21.15.39:0/2864066313 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed000b3dd0 con 0x5590d62b1480 2023-07-21T18:41:01.574 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.540+0000 7feceeffd700 1 -- 172.21.15.39:0/2864066313 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fed0009f2b0 con 0x5590d62b1480 2023-07-21T18:41:01.574 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.540+0000 7fed21c3db80 1 -- 172.21.15.39:0/2864066313 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5590d615c4a0 con 0x5590d62b1480 2023-07-21T18:41:01.574 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.540+0000 7feceeffd700 1 -- 172.21.15.39:0/2864066313 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed0009b280 con 0x5590d62b1480 2023-07-21T18:41:01.575 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.540+0000 7fed21c3db80 10 client.15036 fetch_fsmap learned FSMap version 26 2023-07-21T18:41:01.575 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.540+0000 7fed21c3db80 1 -- 172.21.15.39:0/2864066313 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7fed1019fef0 con 0x5590d62b1480 2023-07-21T18:41:01.575 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.540+0000 7feceeffd700 1 -- 172.21.15.39:0/2864066313 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5590d615c4a0 con 0x5590d62b1480 2023-07-21T18:41:01.575 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.540+0000 7feceeffd700 1 -- 172.21.15.39:0/2864066313 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fed00097100 con 0x5590d62b1480 2023-07-21T18:41:01.575 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.540+0000 7fed21c3db80 10 client.15036 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T18:41:01.575 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.540+0000 7fed21c3db80 10 client.15036 resolve_mds: resolved name 'e' to daemon mds.e 2023-07-21T18:41:01.576 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.540+0000 7fed21c3db80 20 client.15036 populate_metadata read hostname 'smithi039' 2023-07-21T18:41:01.576 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.540+0000 7fed21c3db80 1 --2- 172.21.15.39:0/2864066313 >> [v2:172.21.15.155:6834/3854856286,v1:172.21.15.155:6835/3854856286] conn(0x5590d62c5b80 0x5590d62c51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.576 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.540+0000 7fed21c3db80 4 client.15036 mds_command: new command op to 24464 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:41:01.576 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.540+0000 7fed21c3db80 1 -- 172.21.15.39:0/2864066313 --> [v2:172.21.15.155:6834/3854856286,v1:172.21.15.155:6835/3854856286] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5590d61fbe60 con 0x5590d62c5b80 2023-07-21T18:41:01.576 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.541+0000 7fed0effd700 1 --2- 172.21.15.39:0/2864066313 >> [v2:172.21.15.155:6834/3854856286,v1:172.21.15.155:6835/3854856286] conn(0x5590d62c5b80 0x5590d62c51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:01.576 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.542+0000 7feceeffd700 10 client.15036 ms_handle_connect on v2:172.21.15.155:6834/3854856286 2023-07-21T18:41:01.576 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.542+0000 7fed0effd700 1 --2- 172.21.15.39:0/2864066313 >> [v2:172.21.15.155:6834/3854856286,v1:172.21.15.155:6835/3854856286] conn(0x5590d62c5b80 0x5590d62c51b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=73f31c3debbba31a server_cookie=7c153e9a3d2eefaf in_seq=0 out_seq=0 2023-07-21T18:41:01.577 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.542+0000 7feceeffd700 1 -- 172.21.15.39:0/2864066313 <== mds.3 v2:172.21.15.155:6834/3854856286 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5590d61fbe60 con 0x5590d62c5b80 2023-07-21T18:41:01.578 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.542+0000 7feceeffd700 10 client.15036 handle_command_reply: tid=0 2023-07-21T18:41:01.578 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.542+0000 7fed21c3db80 1 client.15036 shutdown 2023-07-21T18:41:01.578 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.543+0000 7fed21c3db80 1 -- 172.21.15.39:0/2864066313 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5590d62b1480 msgr2=0x5590d62bad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:01.578 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.543+0000 7fed21c3db80 1 --2- 172.21.15.39:0/2864066313 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5590d62b1480 0x5590d62bad00 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fed0000b5c0 tx=0x7fed000a33e0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.578 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.543+0000 7fed21c3db80 1 -- 172.21.15.39:0/2864066313 shutdown_connections 2023-07-21T18:41:01.578 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.543+0000 7fed21c3db80 1 --2- 172.21.15.39:0/2864066313 >> [v2:172.21.15.155:6834/3854856286,v1:172.21.15.155:6835/3854856286] conn(0x5590d62c5b80 0x5590d62c51b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.578 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.543+0000 7fed21c3db80 1 --2- 172.21.15.39:0/2864066313 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590d62b2c20 0x5590d62c2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.579 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.543+0000 7feceeffd700 0 client.15036 ms_handle_reset on v2:172.21.15.155:6834/3854856286 2023-07-21T18:41:01.579 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.543+0000 7fed21c3db80 1 --2- 172.21.15.39:0/2864066313 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5590d62b1480 0x5590d62bad00 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.579 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.543+0000 7fed21c3db80 1 --2- 172.21.15.39:0/2864066313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5590d62b1dd0 0x5590d62bb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.579 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.543+0000 7fed21c3db80 1 -- 172.21.15.39:0/2864066313 >> 172.21.15.39:0/2864066313 conn(0x5590d62a6d50 msgr2=0x5590d62ab860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:01.579 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.543+0000 7fed21c3db80 1 -- 172.21.15.39:0/2864066313 shutdown_connections 2023-07-21T18:41:01.579 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.543+0000 7fed21c3db80 1 -- 172.21.15.39:0/2864066313 wait complete. 2023-07-21T18:41:01.579 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.543+0000 7fed21c3db80 20 client.15036 trim_cache size 0 max 16384 2023-07-21T18:41:01.586 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.560+0000 7fed21c3db80 1 Processor -- start 2023-07-21T18:41:01.586 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.560+0000 7fed21c3db80 1 -- start start 2023-07-21T18:41:01.586 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.561+0000 7fed21c3db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5590d62b1480 0x5590d62c2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.586 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.561+0000 7fed21c3db80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5590d62b1dd0 0x5590d62c2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.587 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.561+0000 7fed21c3db80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590d62b2c20 0x5590d62b5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.587 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.561+0000 7fed21c3db80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590d62c58c0 con 0x5590d62b1dd0 2023-07-21T18:41:01.587 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.561+0000 7fed21c3db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590d62ad0a0 con 0x5590d62b1480 2023-07-21T18:41:01.587 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.561+0000 7fed21c3db80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590d62c6e40 con 0x5590d62b2c20 2023-07-21T18:41:01.587 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.561+0000 7fed0effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5590d62b1dd0 0x5590d62c2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:01.587 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.561+0000 7fed0effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5590d62b1dd0 0x5590d62c2240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55944/0 (socket says 172.21.15.39:55944) 2023-07-21T18:41:01.588 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.561+0000 7fed0effd700 1 -- 172.21.15.39:0/3578250355 learned_addr learned my addr 172.21.15.39:0/3578250355 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:01.588 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.561+0000 7fed0f7fe700 1 --2- 172.21.15.39:0/3578250355 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5590d62b1480 0x5590d62c2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:01.588 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.561+0000 7fed0ffff700 1 --2- 172.21.15.39:0/3578250355 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590d62b2c20 0x5590d62b5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:01.589 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.561+0000 7fed0effd700 1 -- 172.21.15.39:0/3578250355 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590d62b2c20 msgr2=0x5590d62b5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:01.589 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.561+0000 7fed0effd700 1 --2- 172.21.15.39:0/3578250355 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590d62b2c20 0x5590d62b5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.589 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.561+0000 7fed0effd700 1 -- 172.21.15.39:0/3578250355 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5590d62b1480 msgr2=0x5590d62c2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:01.589 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.561+0000 7fed0effd700 1 --2- 172.21.15.39:0/3578250355 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5590d62b1480 0x5590d62c2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.589 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.561+0000 7fed0effd700 1 -- 172.21.15.39:0/3578250355 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590d6019c10 con 0x5590d62b1dd0 2023-07-21T18:41:01.590 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.561+0000 7fed0effd700 1 --2- 172.21.15.39:0/3578250355 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5590d62b1dd0 0x5590d62c2240 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x5590d62bb890 tx=0x7fed04014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ece3631f55778abb server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:01.590 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.561+0000 7fecf67fc700 1 -- 172.21.15.39:0/3578250355 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed04024040 con 0x5590d62b1dd0 2023-07-21T18:41:01.590 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.561+0000 7fecf67fc700 1 -- 172.21.15.39:0/3578250355 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed040264c0 con 0x5590d62b1dd0 2023-07-21T18:41:01.590 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.561+0000 7fecf67fc700 1 -- 172.21.15.39:0/3578250355 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed04010690 con 0x5590d62b1dd0 2023-07-21T18:41:01.593 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.562+0000 7fed21c3db80 1 -- 172.21.15.39:0/3578250355 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5590d62b1dd0 msgr2=0x5590d62c2240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:01.593 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.562+0000 7fed21c3db80 1 --2- 172.21.15.39:0/3578250355 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5590d62b1dd0 0x5590d62c2240 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x5590d62bb890 tx=0x7fed04014000 comp rx=0 tx=0).stop 2023-07-21T18:41:01.594 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.562+0000 7fed21c3db80 1 -- 172.21.15.39:0/3578250355 shutdown_connections 2023-07-21T18:41:01.594 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.562+0000 7fed21c3db80 1 --2- 172.21.15.39:0/3578250355 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5590d62b1480 0x5590d62c2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.594 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.562+0000 7fed21c3db80 1 --2- 172.21.15.39:0/3578250355 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590d62b2c20 0x5590d62b5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.594 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.562+0000 7fed21c3db80 1 --2- 172.21.15.39:0/3578250355 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5590d62b1dd0 0x5590d62c2240 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.595 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.562+0000 7fed21c3db80 1 -- 172.21.15.39:0/3578250355 >> 172.21.15.39:0/3578250355 conn(0x5590d62a6d50 msgr2=0x5590d62c51b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:01.595 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.562+0000 7fed21c3db80 1 -- 172.21.15.39:0/3578250355 shutdown_connections 2023-07-21T18:41:01.595 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.562+0000 7fed21c3db80 1 -- 172.21.15.39:0/3578250355 wait complete. 2023-07-21T18:41:01.595 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.562+0000 7fed21c3db80 1 -- start start 2023-07-21T18:41:01.596 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.562+0000 7fed21c3db80 1 Processor -- start 2023-07-21T18:41:01.596 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.563+0000 7fed21c3db80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5590d62b1480 0x5590d62be440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.596 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.563+0000 7fed21c3db80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590d62b1dd0 0x5590d62be980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.596 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.563+0000 7fed21c3db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5590d62b2c20 0x5590d62c1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.597 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.563+0000 7fed21c3db80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590d62c1790 con 0x5590d62b1480 2023-07-21T18:41:01.597 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.563+0000 7fed21c3db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590d62c1900 con 0x5590d62b2c20 2023-07-21T18:41:01.597 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.563+0000 7fed21c3db80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590d62c1ac0 con 0x5590d62b1dd0 2023-07-21T18:41:01.597 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.563+0000 7fed21c3db80 10 client.0 _ll_register_callbacks cb 0x7fed08009230 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:41:01.597 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.563+0000 7fed0f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5590d62b1480 0x5590d62be440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:01.598 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.563+0000 7fed0f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5590d62b1480 0x5590d62be440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55948/0 (socket says 172.21.15.39:55948) 2023-07-21T18:41:01.598 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.563+0000 7fed0f7fe700 1 -- 172.21.15.39:0/2780093409 learned_addr learned my addr 172.21.15.39:0/2780093409 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:01.598 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.563+0000 7fed0effd700 1 --2- 172.21.15.39:0/2780093409 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590d62b1dd0 0x5590d62be980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:01.598 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.563+0000 7fed0f7fe700 1 -- 172.21.15.39:0/2780093409 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590d62b1dd0 msgr2=0x5590d62be980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:01.598 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.563+0000 7fed0f7fe700 1 --2- 172.21.15.39:0/2780093409 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590d62b1dd0 0x5590d62be980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.598 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.563+0000 7fed0f7fe700 1 -- 172.21.15.39:0/2780093409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5590d62b2c20 msgr2=0x5590d62c1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:01.599 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.563+0000 7fed0f7fe700 1 --2- 172.21.15.39:0/2780093409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5590d62b2c20 0x5590d62c1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.599 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.563+0000 7fed0f7fe700 1 -- 172.21.15.39:0/2780093409 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed0400ed40 con 0x5590d62b1480 2023-07-21T18:41:01.599 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.563+0000 7fed0effd700 1 --2- 172.21.15.39:0/2780093409 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590d62b1dd0 0x5590d62be980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:41:01.599 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.563+0000 7fed0f7fe700 1 --2- 172.21.15.39:0/2780093409 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5590d62b1480 0x5590d62be440 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fed00097cf0 tx=0x7fed00005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c50a6232c54394b3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:01.599 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.563+0000 7feceeffd700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:41:01.599 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.563+0000 7feceeffd700 1 -- 172.21.15.39:0/2780093409 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed00097280 con 0x5590d62b1480 2023-07-21T18:41:01.600 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.563+0000 7feceeffd700 1 -- 172.21.15.39:0/2780093409 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fed000a3640 con 0x5590d62b1480 2023-07-21T18:41:01.600 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.563+0000 7fed21c3db80 1 -- 172.21.15.39:0/2780093409 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fed00095e90 con 0x5590d62b1480 2023-07-21T18:41:01.600 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.563+0000 7feceeffd700 1 -- 172.21.15.39:0/2780093409 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed000cb020 con 0x5590d62b1480 2023-07-21T18:41:01.600 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.563+0000 7fed21c3db80 10 client.15042 fetch_fsmap learned FSMap version 26 2023-07-21T18:41:01.601 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.563+0000 7fed21c3db80 1 -- 172.21.15.39:0/2780093409 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5590d6057200 con 0x5590d62b1480 2023-07-21T18:41:01.601 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.564+0000 7feceeffd700 1 -- 172.21.15.39:0/2780093409 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed00095e90 con 0x5590d62b1480 2023-07-21T18:41:01.601 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.564+0000 7feceeffd700 1 -- 172.21.15.39:0/2780093409 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fed0009a1e0 con 0x5590d62b1480 2023-07-21T18:41:01.601 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.564+0000 7fed21c3db80 10 client.15042 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T18:41:01.601 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.564+0000 7fed21c3db80 10 client.15042 resolve_mds: resolved name 'e' to daemon mds.e 2023-07-21T18:41:01.602 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.564+0000 7fed21c3db80 20 client.15042 populate_metadata read hostname 'smithi039' 2023-07-21T18:41:01.602 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.564+0000 7fed21c3db80 1 --2- 172.21.15.39:0/2780093409 >> [v2:172.21.15.155:6834/3854856286,v1:172.21.15.155:6835/3854856286] conn(0x5590d631f4b0 0x5590d631bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.602 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.564+0000 7fed21c3db80 4 client.15042 mds_command: new command op to 24464 tid=0[{"prefix": "status"}] 2023-07-21T18:41:01.602 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.564+0000 7fed21c3db80 1 -- 172.21.15.39:0/2780093409 --> [v2:172.21.15.155:6834/3854856286,v1:172.21.15.155:6835/3854856286] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5590d61be000 con 0x5590d631f4b0 2023-07-21T18:41:01.602 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.564+0000 7fed0effd700 1 --2- 172.21.15.39:0/2780093409 >> [v2:172.21.15.155:6834/3854856286,v1:172.21.15.155:6835/3854856286] conn(0x5590d631f4b0 0x5590d631bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:01.602 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.565+0000 7feceeffd700 10 client.15042 ms_handle_connect on v2:172.21.15.155:6834/3854856286 2023-07-21T18:41:01.603 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.565+0000 7fed0effd700 1 --2- 172.21.15.39:0/2780093409 >> [v2:172.21.15.155:6834/3854856286,v1:172.21.15.155:6835/3854856286] conn(0x5590d631f4b0 0x5590d631bf20 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=45cd5db2c450bb4b server_cookie=74108bf68469580d in_seq=0 out_seq=0 2023-07-21T18:41:01.603 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.565+0000 7feceeffd700 1 -- 172.21.15.39:0/2780093409 <== mds.3 v2:172.21.15.155:6834/3854856286 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5590d61be000 con 0x5590d631f4b0 2023-07-21T18:41:01.603 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.565+0000 7feceeffd700 10 client.15042 handle_command_reply: tid=0 2023-07-21T18:41:01.603 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.565+0000 7fed21c3db80 1 client.15042 shutdown 2023-07-21T18:41:01.603 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.565+0000 7fed21c3db80 1 -- 172.21.15.39:0/2780093409 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5590d62b1480 msgr2=0x5590d62be440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:01.603 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.565+0000 7fed21c3db80 1 --2- 172.21.15.39:0/2780093409 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5590d62b1480 0x5590d62be440 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fed00097cf0 tx=0x7fed00005c90 comp rx=0 tx=0).stop 2023-07-21T18:41:01.603 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.565+0000 7fed21c3db80 1 -- 172.21.15.39:0/2780093409 shutdown_connections 2023-07-21T18:41:01.604 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.565+0000 7fed21c3db80 1 --2- 172.21.15.39:0/2780093409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5590d62b2c20 0x5590d62c1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.604 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.565+0000 7fed21c3db80 1 --2- 172.21.15.39:0/2780093409 >> [v2:172.21.15.155:6834/3854856286,v1:172.21.15.155:6835/3854856286] conn(0x5590d631f4b0 0x5590d631bf20 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.604 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.565+0000 7fed21c3db80 1 --2- 172.21.15.39:0/2780093409 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5590d62b1480 0x5590d62be440 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.604 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.565+0000 7fed21c3db80 1 --2- 172.21.15.39:0/2780093409 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590d62b1dd0 0x5590d62be980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.604 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.565+0000 7feceeffd700 0 client.15042 ms_handle_reset on v2:172.21.15.155:6834/3854856286 2023-07-21T18:41:01.604 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.565+0000 7fed21c3db80 1 -- 172.21.15.39:0/2780093409 >> 172.21.15.39:0/2780093409 conn(0x5590d62a6d50 msgr2=0x5590d62c2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:01.604 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.565+0000 7fed21c3db80 1 -- 172.21.15.39:0/2780093409 shutdown_connections 2023-07-21T18:41:01.605 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.565+0000 7fed21c3db80 1 -- 172.21.15.39:0/2780093409 wait complete. 2023-07-21T18:41:01.605 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.565+0000 7fed21c3db80 20 client.15042 trim_cache size 0 max 16384 2023-07-21T18:41:01.605 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2023-07-21T18:41:01.605 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "eaf8e25a-27f4-11ee-9b34-001a4aab830c", 2023-07-21T18:41:01.605 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 3, 2023-07-21T18:41:01.606 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 24464, 2023-07-21T18:41:01.606 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2023-07-21T18:41:01.606 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2023-07-21T18:41:01.606 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2023-07-21T18:41:01.606 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 51.332637583999997, 2023-07-21T18:41:01.606 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 26, 2023-07-21T18:41:01.606 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 79, 2023-07-21T18:41:01.607 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 79, 2023-07-21T18:41:01.607 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 78.712947975999995 2023-07-21T18:41:01.607 INFO:teuthology.orchestra.run.smithi039.stdout:} 2023-07-21T18:41:01.608 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.566+0000 7fed15ab7700 1 -- 172.21.15.39:0/417981779 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fecf80a11e0 msgr2=0x7fecf80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:01.608 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.566+0000 7fed15ab7700 1 --2- 172.21.15.39:0/417981779 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fecf80a11e0 0x7fecf80a3690 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fed04005fd0 tx=0x7fed04005e30 comp rx=0 tx=0).stop 2023-07-21T18:41:01.609 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.566+0000 7fed15ab7700 1 -- 172.21.15.39:0/417981779 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed100ffae0 msgr2=0x7fed100742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:01.609 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.566+0000 7fed15ab7700 1 --2- 172.21.15.39:0/417981779 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed100ffae0 0x7fed100742c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fed00000c00 tx=0x7fed000057f0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.609 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.566+0000 7fed15ab7700 1 -- 172.21.15.39:0/417981779 shutdown_connections 2023-07-21T18:41:01.609 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.566+0000 7fed15ab7700 1 --2- 172.21.15.39:0/417981779 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fecf80a11e0 0x7fecf80a3690 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.609 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.566+0000 7fed15ab7700 1 --2- 172.21.15.39:0/417981779 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fed10101660 0x7fed10072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.609 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.566+0000 7fed15ab7700 1 --2- 172.21.15.39:0/417981779 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed100ffae0 0x7fed100742c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.609 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.566+0000 7fed15ab7700 1 --2- 172.21.15.39:0/417981779 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed10100cd0 0x7fed10072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.610 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.566+0000 7fed15ab7700 1 -- 172.21.15.39:0/417981779 >> 172.21.15.39:0/417981779 conn(0x7fed100fb120 msgr2=0x7fed100fd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:01.610 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.566+0000 7fed15ab7700 1 -- 172.21.15.39:0/417981779 shutdown_connections 2023-07-21T18:41:01.610 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.566+0000 7fed15ab7700 1 -- 172.21.15.39:0/417981779 wait complete. 2023-07-21T18:41:01.610 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-07-21T18:41:01.737 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.712+0000 7f51d3347700 1 -- 172.21.15.39:0/596368296 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51cc0fea10 msgr2=0x7f51cc0fee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:01.738 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.712+0000 7f51d3347700 1 --2- 172.21.15.39:0/596368296 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51cc0fea10 0x7f51cc0fee20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f51c8009990 tx=0x7f51c8009ca0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.738 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.712+0000 7f51d3347700 1 -- 172.21.15.39:0/596368296 shutdown_connections 2023-07-21T18:41:01.738 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.712+0000 7f51d3347700 1 --2- 172.21.15.39:0/596368296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f51cc100590 0x7f51cc106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.738 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.712+0000 7f51d3347700 1 --2- 172.21.15.39:0/596368296 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51cc0fea10 0x7f51cc0fee20 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.739 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.712+0000 7f51d3347700 1 --2- 172.21.15.39:0/596368296 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f51cc0ffc00 0x7f51cc100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.739 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.712+0000 7f51d3347700 1 -- 172.21.15.39:0/596368296 >> 172.21.15.39:0/596368296 conn(0x7f51cc0fa030 msgr2=0x7f51cc0fc460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:01.739 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.712+0000 7f51d3347700 1 -- 172.21.15.39:0/596368296 shutdown_connections 2023-07-21T18:41:01.739 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.712+0000 7f51d3347700 1 -- 172.21.15.39:0/596368296 wait complete. 2023-07-21T18:41:01.739 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.713+0000 7f51d3347700 1 Processor -- start 2023-07-21T18:41:01.739 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.713+0000 7f51d3347700 1 -- start start 2023-07-21T18:41:01.741 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.713+0000 7f51d3347700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51cc0fea10 0x7f51cc197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.741 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.713+0000 7f51d3347700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f51cc0ffc00 0x7f51cc197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.741 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.713+0000 7f51d3347700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f51cc100590 0x7f51cc19f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.741 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.713+0000 7f51d3347700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51cc19f620 con 0x7f51cc0fea10 2023-07-21T18:41:01.741 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.713+0000 7f51d3347700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51cc19f760 con 0x7f51cc0ffc00 2023-07-21T18:41:01.741 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.713+0000 7f51d3347700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51cc19f8f0 con 0x7f51cc100590 2023-07-21T18:41:01.742 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.713+0000 7f51d10e3700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51cc0fea10 0x7f51cc197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:01.742 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.713+0000 7f51d10e3700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51cc0fea10 0x7f51cc197aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55962/0 (socket says 172.21.15.39:55962) 2023-07-21T18:41:01.742 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.713+0000 7f51d10e3700 1 -- 172.21.15.39:0/1386275956 learned_addr learned my addr 172.21.15.39:0/1386275956 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:01.742 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.713+0000 7f51d08e2700 1 --2- 172.21.15.39:0/1386275956 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f51cc0ffc00 0x7f51cc197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:01.742 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.713+0000 7f51d18e4700 1 --2- 172.21.15.39:0/1386275956 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f51cc100590 0x7f51cc19f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:01.742 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.714+0000 7f51d10e3700 1 -- 172.21.15.39:0/1386275956 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f51cc100590 msgr2=0x7f51cc19f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:01.743 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.714+0000 7f51d10e3700 1 --2- 172.21.15.39:0/1386275956 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f51cc100590 0x7f51cc19f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.743 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.714+0000 7f51d10e3700 1 -- 172.21.15.39:0/1386275956 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f51cc0ffc00 msgr2=0x7f51cc197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:01.743 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.714+0000 7f51d10e3700 1 --2- 172.21.15.39:0/1386275956 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f51cc0ffc00 0x7f51cc197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.743 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.714+0000 7f51d10e3700 1 -- 172.21.15.39:0/1386275956 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c80095f0 con 0x7f51cc0fea10 2023-07-21T18:41:01.743 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.714+0000 7f51d10e3700 1 --2- 172.21.15.39:0/1386275956 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51cc0fea10 0x7f51cc197aa0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f51c8000c00 tx=0x7f51c8005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:01.743 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.714+0000 7f51c27fc700 1 -- 172.21.15.39:0/1386275956 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c801d070 con 0x7f51cc0fea10 2023-07-21T18:41:01.744 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.714+0000 7f51c27fc700 1 -- 172.21.15.39:0/1386275956 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51c8003f60 con 0x7f51cc0fea10 2023-07-21T18:41:01.744 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.714+0000 7f51c27fc700 1 -- 172.21.15.39:0/1386275956 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c8022b50 con 0x7f51cc0fea10 2023-07-21T18:41:01.744 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.714+0000 7f51d3347700 1 -- 172.21.15.39:0/1386275956 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51cc19fb70 con 0x7f51cc0fea10 2023-07-21T18:41:01.744 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.714+0000 7f51d3347700 1 -- 172.21.15.39:0/1386275956 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51cc19ff40 con 0x7f51cc0fea10 2023-07-21T18:41:01.744 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.716+0000 7f51c27fc700 1 -- 172.21.15.39:0/1386275956 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f51c80040d0 con 0x7f51cc0fea10 2023-07-21T18:41:01.749 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.717+0000 7f51c27fc700 1 --2- 172.21.15.39:0/1386275956 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f51b80a11e0 0x7f51b80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.750 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.717+0000 7f51c27fc700 1 -- 172.21.15.39:0/1386275956 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f51c80ce090 con 0x7f51cc0fea10 2023-07-21T18:41:01.750 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.717+0000 7f51d08e2700 1 --2- 172.21.15.39:0/1386275956 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f51b80a11e0 0x7f51b80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:01.750 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.717+0000 7f51d08e2700 1 --2- 172.21.15.39:0/1386275956 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f51b80a11e0 0x7f51b80a3690 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f51bc005950 tx=0x7f51bc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:01.750 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.718+0000 7f51df4cdb80 1 Processor -- start 2023-07-21T18:41:01.751 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.718+0000 7f51df4cdb80 1 -- start start 2023-07-21T18:41:01.751 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.718+0000 7f51df4cdb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e3f0262480 0x55e3f0262890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.751 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.718+0000 7f51df4cdb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3f0262dd0 0x55e3f0263690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.751 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.718+0000 7f51df4cdb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3f0263c20 0x55e3f0264090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.752 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.718+0000 7f51df4cdb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3f0167c60 con 0x55e3f0262dd0 2023-07-21T18:41:01.752 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.718+0000 7f51df4cdb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3f01ba9f0 con 0x55e3f0262480 2023-07-21T18:41:01.752 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.718+0000 7f51df4cdb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3f01b3e80 con 0x55e3f0263c20 2023-07-21T18:41:01.753 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.718+0000 7f51d08e2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3f0262dd0 0x55e3f0263690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:01.753 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.718+0000 7f51d08e2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3f0262dd0 0x55e3f0263690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55972/0 (socket says 172.21.15.39:55972) 2023-07-21T18:41:01.753 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.718+0000 7f51d08e2700 1 -- 172.21.15.39:0/1722331062 learned_addr learned my addr 172.21.15.39:0/1722331062 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:01.753 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.718+0000 7f51d18e4700 1 --2- 172.21.15.39:0/1722331062 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3f0263c20 0x55e3f0264090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:01.753 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.719+0000 7f51d10e3700 1 --2- 172.21.15.39:0/1722331062 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e3f0262480 0x55e3f0262890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:01.753 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.719+0000 7f51d08e2700 1 -- 172.21.15.39:0/1722331062 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3f0263c20 msgr2=0x55e3f0264090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:01.753 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.719+0000 7f51d08e2700 1 --2- 172.21.15.39:0/1722331062 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3f0263c20 0x55e3f0264090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.753 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.719+0000 7f51d08e2700 1 -- 172.21.15.39:0/1722331062 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e3f0262480 msgr2=0x55e3f0262890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:01.754 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.719+0000 7f51d08e2700 1 --2- 172.21.15.39:0/1722331062 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e3f0262480 0x55e3f0262890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.754 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.719+0000 7f51d08e2700 1 -- 172.21.15.39:0/1722331062 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3effe1d90 con 0x55e3f0262dd0 2023-07-21T18:41:01.754 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.719+0000 7f51d18e4700 1 --2- 172.21.15.39:0/1722331062 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3f0263c20 0x55e3f0264090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:41:01.754 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.719+0000 7f51d10e3700 1 --2- 172.21.15.39:0/1722331062 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e3f0262480 0x55e3f0262890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:41:01.754 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.719+0000 7f51d08e2700 1 --2- 172.21.15.39:0/1722331062 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3f0262dd0 0x55e3f0263690 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f51bc010fa0 tx=0x7f51bc013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=910d8681d9bfc7e9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:01.754 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.719+0000 7f51b77fe700 1 -- 172.21.15.39:0/1722331062 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51bc01e620 con 0x55e3f0262dd0 2023-07-21T18:41:01.754 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.719+0000 7f51b77fe700 1 -- 172.21.15.39:0/1722331062 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51bc015070 con 0x55e3f0262dd0 2023-07-21T18:41:01.754 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.719+0000 7f51b77fe700 1 -- 172.21.15.39:0/1722331062 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51bc01ca20 con 0x55e3f0262dd0 2023-07-21T18:41:01.755 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.720+0000 7f51df4cdb80 1 -- 172.21.15.39:0/1722331062 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3f0262dd0 msgr2=0x55e3f0263690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:01.755 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.720+0000 7f51df4cdb80 1 --2- 172.21.15.39:0/1722331062 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3f0262dd0 0x55e3f0263690 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f51bc010fa0 tx=0x7f51bc013000 comp rx=0 tx=0).stop 2023-07-21T18:41:01.755 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.720+0000 7f51df4cdb80 1 -- 172.21.15.39:0/1722331062 shutdown_connections 2023-07-21T18:41:01.755 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.720+0000 7f51df4cdb80 1 --2- 172.21.15.39:0/1722331062 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e3f0262480 0x55e3f0262890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.755 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.720+0000 7f51df4cdb80 1 --2- 172.21.15.39:0/1722331062 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3f0263c20 0x55e3f0264090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.755 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.720+0000 7f51df4cdb80 1 --2- 172.21.15.39:0/1722331062 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3f0262dd0 0x55e3f0263690 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.755 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.720+0000 7f51df4cdb80 1 -- 172.21.15.39:0/1722331062 >> 172.21.15.39:0/1722331062 conn(0x55e3f0257d50 msgr2=0x55e3f025d180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:01.756 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.720+0000 7f51df4cdb80 1 -- 172.21.15.39:0/1722331062 shutdown_connections 2023-07-21T18:41:01.756 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.720+0000 7f51df4cdb80 1 -- 172.21.15.39:0/1722331062 wait complete. 2023-07-21T18:41:01.756 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.720+0000 7f51df4cdb80 1 -- start start 2023-07-21T18:41:01.756 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.720+0000 7f51df4cdb80 1 Processor -- start 2023-07-21T18:41:01.756 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.721+0000 7f51df4cdb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e3f0262480 0x55e3f026c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.756 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.721+0000 7f51df4cdb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3f0262dd0 0x55e3f026c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.756 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.721+0000 7f51df4cdb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3f0263c20 0x55e3f0273650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.757 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.721+0000 7f51df4cdb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3f0153a70 con 0x55e3f0262dd0 2023-07-21T18:41:01.757 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.721+0000 7f51df4cdb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3f0153770 con 0x55e3f0262480 2023-07-21T18:41:01.757 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.721+0000 7f51df4cdb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3f00378d0 con 0x55e3f0263c20 2023-07-21T18:41:01.757 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.721+0000 7f51df4cdb80 10 client.0 _ll_register_callbacks cb 0x55e3f0151710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:41:01.757 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.721+0000 7f51d08e2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3f0262dd0 0x55e3f026c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:01.757 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.721+0000 7f51d08e2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3f0262dd0 0x55e3f026c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55978/0 (socket says 172.21.15.39:55978) 2023-07-21T18:41:01.757 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.721+0000 7f51d08e2700 1 -- 172.21.15.39:0/3466691712 learned_addr learned my addr 172.21.15.39:0/3466691712 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:01.757 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.721+0000 7f51d10e3700 1 --2- 172.21.15.39:0/3466691712 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e3f0262480 0x55e3f026c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:01.758 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.721+0000 7f51d18e4700 1 --2- 172.21.15.39:0/3466691712 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3f0263c20 0x55e3f0273650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:01.758 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.721+0000 7f51d08e2700 1 -- 172.21.15.39:0/3466691712 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3f0263c20 msgr2=0x55e3f0273650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:01.758 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.721+0000 7f51d08e2700 1 --2- 172.21.15.39:0/3466691712 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3f0263c20 0x55e3f0273650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.758 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.721+0000 7f51d08e2700 1 -- 172.21.15.39:0/3466691712 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e3f0262480 msgr2=0x55e3f026c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:01.758 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.721+0000 7f51d08e2700 1 --2- 172.21.15.39:0/3466691712 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e3f0262480 0x55e3f026c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.758 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.721+0000 7f51d08e2700 1 -- 172.21.15.39:0/3466691712 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51bc010c50 con 0x55e3f0262dd0 2023-07-21T18:41:01.758 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.721+0000 7f51d08e2700 1 --2- 172.21.15.39:0/3466691712 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3f0262dd0 0x55e3f026c610 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f51bc010f70 tx=0x7f51bc005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f60a4dd564d28eba server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:01.758 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.721+0000 7f51b4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:41:01.759 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.721+0000 7f51b4ff9700 1 -- 172.21.15.39:0/3466691712 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51bc02d070 con 0x55e3f0262dd0 2023-07-21T18:41:01.759 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.722+0000 7f51b4ff9700 1 -- 172.21.15.39:0/3466691712 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f51bc017070 con 0x55e3f0262dd0 2023-07-21T18:41:01.759 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.722+0000 7f51df4cdb80 1 -- 172.21.15.39:0/3466691712 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e3f010d4a0 con 0x55e3f0262dd0 2023-07-21T18:41:01.759 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.722+0000 7f51b4ff9700 1 -- 172.21.15.39:0/3466691712 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51bc005710 con 0x55e3f0262dd0 2023-07-21T18:41:01.760 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.722+0000 7f51df4cdb80 10 client.15054 fetch_fsmap learned FSMap version 26 2023-07-21T18:41:01.760 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.722+0000 7f51df4cdb80 1 -- 172.21.15.39:0/3466691712 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55e3effcac10 con 0x55e3f0262dd0 2023-07-21T18:41:01.760 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.722+0000 7f51b4ff9700 1 -- 172.21.15.39:0/3466691712 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e3f010d4a0 con 0x55e3f0262dd0 2023-07-21T18:41:01.760 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.722+0000 7f51b4ff9700 1 -- 172.21.15.39:0/3466691712 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f51bc01cbf0 con 0x55e3f0262dd0 2023-07-21T18:41:01.761 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.722+0000 7f51df4cdb80 10 client.15054 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T18:41:01.761 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.722+0000 7f51df4cdb80 10 client.15054 resolve_mds: resolved name 'i' to daemon mds.i 2023-07-21T18:41:01.761 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.722+0000 7f51df4cdb80 20 client.15054 populate_metadata read hostname 'smithi039' 2023-07-21T18:41:01.761 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.722+0000 7f51df4cdb80 1 --2- 172.21.15.39:0/3466691712 >> [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] conn(0x55e3f0277750 0x55e3f02765b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.761 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.722+0000 7f51df4cdb80 4 client.15054 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:41:01.761 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.722+0000 7f51df4cdb80 1 -- 172.21.15.39:0/3466691712 --> [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e3f01ace60 con 0x55e3f0277750 2023-07-21T18:41:01.761 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.722+0000 7f51d10e3700 1 --2- 172.21.15.39:0/3466691712 >> [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] conn(0x55e3f0277750 0x55e3f02765b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:01.762 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.723+0000 7f51d10e3700 1 --2- 172.21.15.39:0/3466691712 >> [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] conn(0x55e3f0277750 0x55e3f02765b0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f0db294d45806219 server_cookie=6080d05b4fdc7689 in_seq=0 out_seq=0 2023-07-21T18:41:01.762 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.723+0000 7f51b4ff9700 10 client.15054 ms_handle_connect on v2:172.21.15.193:6838/3148488639 2023-07-21T18:41:01.762 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.724+0000 7f51b4ff9700 1 -- 172.21.15.39:0/3466691712 <== mds.2 v2:172.21.15.193:6838/3148488639 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e3f01ace60 con 0x55e3f0277750 2023-07-21T18:41:01.762 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.724+0000 7f51b4ff9700 10 client.15054 handle_command_reply: tid=0 2023-07-21T18:41:01.762 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.724+0000 7f51df4cdb80 1 client.15054 shutdown 2023-07-21T18:41:01.762 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.724+0000 7f51df4cdb80 1 -- 172.21.15.39:0/3466691712 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3f0262dd0 msgr2=0x55e3f026c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:01.762 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.724+0000 7f51df4cdb80 1 --2- 172.21.15.39:0/3466691712 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3f0262dd0 0x55e3f026c610 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f51bc010f70 tx=0x7f51bc005270 comp rx=0 tx=0).stop 2023-07-21T18:41:01.763 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.724+0000 7f51df4cdb80 1 -- 172.21.15.39:0/3466691712 shutdown_connections 2023-07-21T18:41:01.763 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.724+0000 7f51df4cdb80 1 --2- 172.21.15.39:0/3466691712 >> [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] conn(0x55e3f0277750 0x55e3f02765b0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.763 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.724+0000 7f51df4cdb80 1 --2- 172.21.15.39:0/3466691712 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e3f0262480 0x55e3f026c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.763 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.724+0000 7f51df4cdb80 1 --2- 172.21.15.39:0/3466691712 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3f0263c20 0x55e3f0273650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.763 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.724+0000 7f51b4ff9700 0 client.15054 ms_handle_reset on v2:172.21.15.193:6838/3148488639 2023-07-21T18:41:01.763 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.724+0000 7f51df4cdb80 1 --2- 172.21.15.39:0/3466691712 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3f0262dd0 0x55e3f026c610 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.763 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.724+0000 7f51df4cdb80 1 -- 172.21.15.39:0/3466691712 >> 172.21.15.39:0/3466691712 conn(0x55e3f0257d50 msgr2=0x55e3f025c860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:01.763 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.724+0000 7f51df4cdb80 1 -- 172.21.15.39:0/3466691712 shutdown_connections 2023-07-21T18:41:01.764 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.724+0000 7f51df4cdb80 1 -- 172.21.15.39:0/3466691712 wait complete. 2023-07-21T18:41:01.764 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.724+0000 7f51df4cdb80 20 client.15054 trim_cache size 0 max 16384 2023-07-21T18:41:01.768 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.742+0000 7f51df4cdb80 1 Processor -- start 2023-07-21T18:41:01.768 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.742+0000 7f51df4cdb80 1 -- start start 2023-07-21T18:41:01.768 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.742+0000 7f51df4cdb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3f0262480 0x55e3f02765b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.769 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.742+0000 7f51df4cdb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3f0262dd0 0x55e3f0273570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.769 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.742+0000 7f51df4cdb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e3f0263c20 0x55e3f0269f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.769 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.742+0000 7f51df4cdb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3f0182790 con 0x55e3f0262480 2023-07-21T18:41:01.769 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.742+0000 7f51df4cdb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3f0043de0 con 0x55e3f0263c20 2023-07-21T18:41:01.769 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.742+0000 7f51df4cdb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3f0172480 con 0x55e3f0262dd0 2023-07-21T18:41:01.770 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.742+0000 7f51d10e3700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3f0262480 0x55e3f02765b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:01.770 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.742+0000 7f51d10e3700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3f0262480 0x55e3f02765b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55984/0 (socket says 172.21.15.39:55984) 2023-07-21T18:41:01.770 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.742+0000 7f51d10e3700 1 -- 172.21.15.39:0/1470924933 learned_addr learned my addr 172.21.15.39:0/1470924933 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:01.770 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.742+0000 7f51d10e3700 1 -- 172.21.15.39:0/1470924933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3f0262dd0 msgr2=0x55e3f0273570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:01.770 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.742+0000 7f51d10e3700 1 --2- 172.21.15.39:0/1470924933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3f0262dd0 0x55e3f0273570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.770 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.742+0000 7f51d10e3700 1 -- 172.21.15.39:0/1470924933 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e3f0263c20 msgr2=0x55e3f0269f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:01.771 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.742+0000 7f51d10e3700 1 --2- 172.21.15.39:0/1470924933 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e3f0263c20 0x55e3f0269f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.771 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.742+0000 7f51d10e3700 1 -- 172.21.15.39:0/1470924933 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3f0008200 con 0x55e3f0262480 2023-07-21T18:41:01.771 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.742+0000 7f51d10e3700 1 --2- 172.21.15.39:0/1470924933 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3f0262480 0x55e3f02765b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f51c8097980 tx=0x7f51c8022470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd17cdcf77360f58 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:01.771 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.742+0000 7f51b7fff700 1 -- 172.21.15.39:0/1470924933 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c80a3840 con 0x55e3f0262480 2023-07-21T18:41:01.771 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.743+0000 7f51b7fff700 1 -- 172.21.15.39:0/1470924933 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51c8099820 con 0x55e3f0262480 2023-07-21T18:41:01.772 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.743+0000 7f51b7fff700 1 -- 172.21.15.39:0/1470924933 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c809abd0 con 0x55e3f0262480 2023-07-21T18:41:01.772 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.743+0000 7f51df4cdb80 1 -- 172.21.15.39:0/1470924933 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3f0262480 msgr2=0x55e3f02765b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:01.772 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.743+0000 7f51df4cdb80 1 --2- 172.21.15.39:0/1470924933 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3f0262480 0x55e3f02765b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f51c8097980 tx=0x7f51c8022470 comp rx=0 tx=0).stop 2023-07-21T18:41:01.772 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.743+0000 7f51df4cdb80 1 -- 172.21.15.39:0/1470924933 shutdown_connections 2023-07-21T18:41:01.772 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.743+0000 7f51df4cdb80 1 --2- 172.21.15.39:0/1470924933 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e3f0263c20 0x55e3f0269f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.772 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.743+0000 7f51df4cdb80 1 --2- 172.21.15.39:0/1470924933 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3f0262480 0x55e3f02765b0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.772 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.743+0000 7f51df4cdb80 1 --2- 172.21.15.39:0/1470924933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3f0262dd0 0x55e3f0273570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.773 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.743+0000 7f51df4cdb80 1 -- 172.21.15.39:0/1470924933 >> 172.21.15.39:0/1470924933 conn(0x55e3f0257d50 msgr2=0x55e3f0273e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:01.773 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.743+0000 7f51df4cdb80 1 -- 172.21.15.39:0/1470924933 shutdown_connections 2023-07-21T18:41:01.773 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.743+0000 7f51df4cdb80 1 -- 172.21.15.39:0/1470924933 wait complete. 2023-07-21T18:41:01.773 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.743+0000 7f51df4cdb80 1 -- start start 2023-07-21T18:41:01.773 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.744+0000 7f51df4cdb80 1 Processor -- start 2023-07-21T18:41:01.776 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.744+0000 7f51df4cdb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e3f0262480 0x55e3f026ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.776 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.744+0000 7f51df4cdb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3f0262dd0 0x55e3f026f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.776 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.744+0000 7f51df4cdb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3f0263c20 0x55e3f02722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.776 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.744+0000 7f51df4cdb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3f0272800 con 0x55e3f0262dd0 2023-07-21T18:41:01.776 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.744+0000 7f51df4cdb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3f0272970 con 0x55e3f0262480 2023-07-21T18:41:01.777 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.744+0000 7f51df4cdb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3f0272b30 con 0x55e3f0263c20 2023-07-21T18:41:01.777 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.744+0000 7f51df4cdb80 10 client.0 _ll_register_callbacks cb 0x7f51bc010e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:41:01.777 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.744+0000 7f51d08e2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3f0262dd0 0x55e3f026f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:01.777 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.744+0000 7f51d08e2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3f0262dd0 0x55e3f026f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55992/0 (socket says 172.21.15.39:55992) 2023-07-21T18:41:01.777 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.744+0000 7f51d08e2700 1 -- 172.21.15.39:0/2365031216 learned_addr learned my addr 172.21.15.39:0/2365031216 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:01.777 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.744+0000 7f51d18e4700 1 --2- 172.21.15.39:0/2365031216 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3f0263c20 0x55e3f02722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:01.777 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.744+0000 7f51d08e2700 1 -- 172.21.15.39:0/2365031216 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3f0263c20 msgr2=0x55e3f02722c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:01.777 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.744+0000 7f51d08e2700 1 --2- 172.21.15.39:0/2365031216 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3f0263c20 0x55e3f02722c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.778 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.744+0000 7f51d08e2700 1 -- 172.21.15.39:0/2365031216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e3f0262480 msgr2=0x55e3f026ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:01.778 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.744+0000 7f51d08e2700 1 --2- 172.21.15.39:0/2365031216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e3f0262480 0x55e3f026ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.778 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.744+0000 7f51d08e2700 1 -- 172.21.15.39:0/2365031216 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51cc19ff40 con 0x55e3f0262dd0 2023-07-21T18:41:01.778 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.745+0000 7f51d08e2700 1 --2- 172.21.15.39:0/2365031216 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3f0262dd0 0x55e3f026f380 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f51bc010f40 tx=0x7f51bc005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb5abf20180bae44 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:01.778 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.745+0000 7f51b4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:41:01.779 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.745+0000 7f51b4ff9700 1 -- 172.21.15.39:0/2365031216 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51bc042070 con 0x55e3f0262dd0 2023-07-21T18:41:01.779 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.745+0000 7f51b4ff9700 1 -- 172.21.15.39:0/2365031216 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f51bc017070 con 0x55e3f0262dd0 2023-07-21T18:41:01.779 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.745+0000 7f51df4cdb80 1 -- 172.21.15.39:0/2365031216 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f51bc015070 con 0x55e3f0262dd0 2023-07-21T18:41:01.779 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.745+0000 7f51b4ff9700 1 -- 172.21.15.39:0/2365031216 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51bc02d070 con 0x55e3f0262dd0 2023-07-21T18:41:01.779 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.745+0000 7f51df4cdb80 10 client.15060 fetch_fsmap learned FSMap version 26 2023-07-21T18:41:01.779 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.745+0000 7f51df4cdb80 1 -- 172.21.15.39:0/2365031216 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55e3effe0260 con 0x55e3f0262dd0 2023-07-21T18:41:01.779 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.745+0000 7f51b4ff9700 1 -- 172.21.15.39:0/2365031216 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51bc015070 con 0x55e3f0262dd0 2023-07-21T18:41:01.780 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.745+0000 7f51b4ff9700 1 -- 172.21.15.39:0/2365031216 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f51bc045020 con 0x55e3f0262dd0 2023-07-21T18:41:01.780 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.745+0000 7f51df4cdb80 10 client.15060 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T18:41:01.780 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.745+0000 7f51df4cdb80 10 client.15060 resolve_mds: resolved name 'i' to daemon mds.i 2023-07-21T18:41:01.780 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.745+0000 7f51df4cdb80 20 client.15060 populate_metadata read hostname 'smithi039' 2023-07-21T18:41:01.780 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.745+0000 7f51df4cdb80 1 --2- 172.21.15.39:0/2365031216 >> [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] conn(0x55e3f02d0890 0x55e3f02cd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.780 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.745+0000 7f51df4cdb80 4 client.15060 mds_command: new command op to 24487 tid=0[{"prefix": "status"}] 2023-07-21T18:41:01.780 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.745+0000 7f51df4cdb80 1 -- 172.21.15.39:0/2365031216 --> [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e3f016f000 con 0x55e3f02d0890 2023-07-21T18:41:01.780 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.746+0000 7f51d10e3700 1 --2- 172.21.15.39:0/2365031216 >> [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] conn(0x55e3f02d0890 0x55e3f02cd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:01.781 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.746+0000 7f51d10e3700 1 --2- 172.21.15.39:0/2365031216 >> [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] conn(0x55e3f02d0890 0x55e3f02cd220 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c557e676865a9cab server_cookie=a4f107fd84c5be28 in_seq=0 out_seq=0 2023-07-21T18:41:01.781 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.746+0000 7f51b4ff9700 10 client.15060 ms_handle_connect on v2:172.21.15.193:6838/3148488639 2023-07-21T18:41:01.781 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.746+0000 7f51b4ff9700 1 -- 172.21.15.39:0/2365031216 <== mds.2 v2:172.21.15.193:6838/3148488639 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55e3f016f000 con 0x55e3f02d0890 2023-07-21T18:41:01.781 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.746+0000 7f51b4ff9700 10 client.15060 handle_command_reply: tid=0 2023-07-21T18:41:01.781 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.746+0000 7f51df4cdb80 1 client.15060 shutdown 2023-07-21T18:41:01.781 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.746+0000 7f51df4cdb80 1 -- 172.21.15.39:0/2365031216 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3f0262dd0 msgr2=0x55e3f026f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:01.781 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.746+0000 7f51df4cdb80 1 --2- 172.21.15.39:0/2365031216 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3f0262dd0 0x55e3f026f380 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f51bc010f40 tx=0x7f51bc005d30 comp rx=0 tx=0).stop 2023-07-21T18:41:01.782 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.746+0000 7f51df4cdb80 1 -- 172.21.15.39:0/2365031216 shutdown_connections 2023-07-21T18:41:01.782 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.747+0000 7f51df4cdb80 1 --2- 172.21.15.39:0/2365031216 >> [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] conn(0x55e3f02d0890 0x55e3f02cd220 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.782 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.747+0000 7f51df4cdb80 1 --2- 172.21.15.39:0/2365031216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e3f0262480 0x55e3f026ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.782 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.747+0000 7f51df4cdb80 1 --2- 172.21.15.39:0/2365031216 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3f0263c20 0x55e3f02722c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.782 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.747+0000 7f51b4ff9700 0 client.15060 ms_handle_reset on v2:172.21.15.193:6838/3148488639 2023-07-21T18:41:01.782 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.747+0000 7f51df4cdb80 1 --2- 172.21.15.39:0/2365031216 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3f0262dd0 0x55e3f026f380 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.782 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.747+0000 7f51df4cdb80 1 -- 172.21.15.39:0/2365031216 >> 172.21.15.39:0/2365031216 conn(0x55e3f0257d50 msgr2=0x55e3f02765b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:01.782 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.747+0000 7f51df4cdb80 1 -- 172.21.15.39:0/2365031216 shutdown_connections 2023-07-21T18:41:01.783 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.747+0000 7f51df4cdb80 1 -- 172.21.15.39:0/2365031216 wait complete. 2023-07-21T18:41:01.783 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.747+0000 7f51df4cdb80 20 client.15060 trim_cache size 0 max 16384 2023-07-21T18:41:01.783 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2023-07-21T18:41:01.783 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "eaf8e25a-27f4-11ee-9b34-001a4aab830c", 2023-07-21T18:41:01.783 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 2, 2023-07-21T18:41:01.784 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 24487, 2023-07-21T18:41:01.784 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2023-07-21T18:41:01.784 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2023-07-21T18:41:01.784 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2023-07-21T18:41:01.784 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 52.525999327000001, 2023-07-21T18:41:01.784 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 26, 2023-07-21T18:41:01.784 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 79, 2023-07-21T18:41:01.785 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 79, 2023-07-21T18:41:01.785 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 70.074646354999999 2023-07-21T18:41:01.785 INFO:teuthology.orchestra.run.smithi039.stdout:} 2023-07-21T18:41:01.786 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.747+0000 7f51d3347700 1 -- 172.21.15.39:0/1386275956 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f51b80a11e0 msgr2=0x7f51b80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:01.786 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.747+0000 7f51d3347700 1 --2- 172.21.15.39:0/1386275956 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f51b80a11e0 0x7f51b80a3690 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f51bc005950 tx=0x7f51bc00b410 comp rx=0 tx=0).stop 2023-07-21T18:41:01.786 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.747+0000 7f51d3347700 1 -- 172.21.15.39:0/1386275956 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51cc0fea10 msgr2=0x7f51cc197aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:01.786 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.747+0000 7f51d3347700 1 --2- 172.21.15.39:0/1386275956 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51cc0fea10 0x7f51cc197aa0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f51c8000c00 tx=0x7f51c8005740 comp rx=0 tx=0).stop 2023-07-21T18:41:01.786 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.747+0000 7f51d3347700 1 -- 172.21.15.39:0/1386275956 shutdown_connections 2023-07-21T18:41:01.787 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.747+0000 7f51d3347700 1 --2- 172.21.15.39:0/1386275956 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f51b80a11e0 0x7f51b80a3690 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.787 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.747+0000 7f51d3347700 1 --2- 172.21.15.39:0/1386275956 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f51cc100590 0x7f51cc19f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.787 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.747+0000 7f51d3347700 1 --2- 172.21.15.39:0/1386275956 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51cc0fea10 0x7f51cc197aa0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.787 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.747+0000 7f51d3347700 1 --2- 172.21.15.39:0/1386275956 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f51cc0ffc00 0x7f51cc197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.787 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.747+0000 7f51d3347700 1 -- 172.21.15.39:0/1386275956 >> 172.21.15.39:0/1386275956 conn(0x7f51cc0fa030 msgr2=0x7f51cc068740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:01.787 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.747+0000 7f51d3347700 1 -- 172.21.15.39:0/1386275956 shutdown_connections 2023-07-21T18:41:01.787 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.747+0000 7f51d3347700 1 -- 172.21.15.39:0/1386275956 wait complete. 2023-07-21T18:41:01.788 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2023-07-21T18:41:01.917 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.890+0000 7f0e560c6700 1 -- 172.21.15.39:0/2953295147 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e50100e00 msgr2=0x7f0e50101250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:01.917 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.890+0000 7f0e560c6700 1 --2- 172.21.15.39:0/2953295147 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e50100e00 0x7f0e50101250 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0e38009990 tx=0x7f0e38009ca0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.918 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.890+0000 7f0e560c6700 1 -- 172.21.15.39:0/2953295147 shutdown_connections 2023-07-21T18:41:01.918 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.890+0000 7f0e560c6700 1 --2- 172.21.15.39:0/2953295147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e500fdd90 0x7f0e500fe1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.918 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.890+0000 7f0e560c6700 1 --2- 172.21.15.39:0/2953295147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e50101790 0x7f0e50108050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.918 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.890+0000 7f0e560c6700 1 --2- 172.21.15.39:0/2953295147 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e50100e00 0x7f0e50101250 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.919 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.890+0000 7f0e560c6700 1 -- 172.21.15.39:0/2953295147 >> 172.21.15.39:0/2953295147 conn(0x7f0e500f9770 msgr2=0x7f0e500fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:01.919 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.890+0000 7f0e560c6700 1 -- 172.21.15.39:0/2953295147 shutdown_connections 2023-07-21T18:41:01.919 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.890+0000 7f0e560c6700 1 -- 172.21.15.39:0/2953295147 wait complete. 2023-07-21T18:41:01.920 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.890+0000 7f0e560c6700 1 Processor -- start 2023-07-21T18:41:01.920 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.891+0000 7f0e560c6700 1 -- start start 2023-07-21T18:41:01.920 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.891+0000 7f0e560c6700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e500fdd90 0x7f0e50195830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.920 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.891+0000 7f0e560c6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e50100e00 0x7f0e50195d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.921 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.891+0000 7f0e560c6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e50101790 0x7f0e5019cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.921 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.891+0000 7f0e560c6700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5019d3b0 con 0x7f0e50100e00 2023-07-21T18:41:01.921 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.891+0000 7f0e560c6700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5019d4f0 con 0x7f0e50101790 2023-07-21T18:41:01.921 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.891+0000 7f0e560c6700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5019d680 con 0x7f0e500fdd90 2023-07-21T18:41:01.922 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.891+0000 7f0e4effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e50100e00 0x7f0e50195d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:01.922 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.891+0000 7f0e4effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e50100e00 0x7f0e50195d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56006/0 (socket says 172.21.15.39:56006) 2023-07-21T18:41:01.922 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.891+0000 7f0e4effd700 1 -- 172.21.15.39:0/88305486 learned_addr learned my addr 172.21.15.39:0/88305486 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:01.923 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.891+0000 7f0e4effd700 1 -- 172.21.15.39:0/88305486 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e500fdd90 msgr2=0x7f0e50195830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:01.923 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.891+0000 7f0e4effd700 1 --2- 172.21.15.39:0/88305486 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e500fdd90 0x7f0e50195830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.923 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.891+0000 7f0e4effd700 1 -- 172.21.15.39:0/88305486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e50101790 msgr2=0x7f0e5019cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:01.923 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.891+0000 7f0e4effd700 1 --2- 172.21.15.39:0/88305486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e50101790 0x7f0e5019cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.924 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.891+0000 7f0e4effd700 1 -- 172.21.15.39:0/88305486 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e380095f0 con 0x7f0e50100e00 2023-07-21T18:41:01.924 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.891+0000 7f0e4effd700 1 --2- 172.21.15.39:0/88305486 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e50100e00 0x7f0e50195d70 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0e38004760 tx=0x7f0e38009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:01.924 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.891+0000 7f0e4cff9700 1 -- 172.21.15.39:0/88305486 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e3801d070 con 0x7f0e50100e00 2023-07-21T18:41:01.924 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.891+0000 7f0e4cff9700 1 -- 172.21.15.39:0/88305486 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e38004200 con 0x7f0e50100e00 2023-07-21T18:41:01.925 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.891+0000 7f0e560c6700 1 -- 172.21.15.39:0/88305486 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e5019d900 con 0x7f0e50100e00 2023-07-21T18:41:01.925 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.891+0000 7f0e4cff9700 1 -- 172.21.15.39:0/88305486 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e380049f0 con 0x7f0e50100e00 2023-07-21T18:41:01.925 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.892+0000 7f0e560c6700 1 -- 172.21.15.39:0/88305486 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e5019ddb0 con 0x7f0e50100e00 2023-07-21T18:41:01.934 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.893+0000 7f0e4cff9700 1 -- 172.21.15.39:0/88305486 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f0e38003d40 con 0x7f0e50100e00 2023-07-21T18:41:01.934 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.894+0000 7f0e4cff9700 1 --2- 172.21.15.39:0/88305486 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f0e3c0a11e0 0x7f0e3c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.934 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.895+0000 7f0e4f7fe700 1 --2- 172.21.15.39:0/88305486 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f0e3c0a11e0 0x7f0e3c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:01.934 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.895+0000 7f0e4cff9700 1 -- 172.21.15.39:0/88305486 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0e380cd940 con 0x7f0e50100e00 2023-07-21T18:41:01.935 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.895+0000 7f0e4f7fe700 1 --2- 172.21.15.39:0/88305486 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f0e3c0a11e0 0x7f0e3c0a3690 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0e500673d0 tx=0x7f0e40009470 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:01.935 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.895+0000 7f0e6224cb80 1 Processor -- start 2023-07-21T18:41:01.935 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.895+0000 7f0e6224cb80 1 -- start start 2023-07-21T18:41:01.936 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.896+0000 7f0e6224cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a892201890 0x55a892201ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.936 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.896+0000 7f0e6224cb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a8922021e0 0x55a892202aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.936 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.896+0000 7f0e6224cb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a892203030 0x55a8922034a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.936 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.896+0000 7f0e6224cb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a892102c60 con 0x55a8922021e0 2023-07-21T18:41:01.937 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.896+0000 7f0e6224cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8921559f0 con 0x55a892201890 2023-07-21T18:41:01.937 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.896+0000 7f0e6224cb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a89214ee80 con 0x55a892203030 2023-07-21T18:41:01.937 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.896+0000 7f0e4effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a8922021e0 0x55a892202aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:01.937 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.896+0000 7f0e4effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a8922021e0 0x55a892202aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56014/0 (socket says 172.21.15.39:56014) 2023-07-21T18:41:01.938 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.896+0000 7f0e4effd700 1 -- 172.21.15.39:0/2605662797 learned_addr learned my addr 172.21.15.39:0/2605662797 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:01.938 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.896+0000 7f0e4effd700 1 -- 172.21.15.39:0/2605662797 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a892203030 msgr2=0x55a8922034a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:01.938 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.896+0000 7f0e4effd700 1 --2- 172.21.15.39:0/2605662797 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a892203030 0x55a8922034a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.938 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.896+0000 7f0e4effd700 1 -- 172.21.15.39:0/2605662797 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a892201890 msgr2=0x55a892201ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:01.939 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.896+0000 7f0e4effd700 1 --2- 172.21.15.39:0/2605662797 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a892201890 0x55a892201ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.939 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.896+0000 7f0e4effd700 1 -- 172.21.15.39:0/2605662797 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a891f7cd90 con 0x55a8922021e0 2023-07-21T18:41:01.939 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.896+0000 7f0e4effd700 1 --2- 172.21.15.39:0/2605662797 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a8922021e0 0x55a892202aa0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f0e3809acc0 tx=0x7f0e38005550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44aaec62505fc75c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:01.939 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.896+0000 7f0e45ffb700 1 -- 172.21.15.39:0/2605662797 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e380a30f0 con 0x55a8922021e0 2023-07-21T18:41:01.940 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.896+0000 7f0e45ffb700 1 -- 172.21.15.39:0/2605662797 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e380990d0 con 0x55a8922021e0 2023-07-21T18:41:01.940 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.896+0000 7f0e45ffb700 1 -- 172.21.15.39:0/2605662797 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e380a2c60 con 0x55a8922021e0 2023-07-21T18:41:01.940 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.897+0000 7f0e6224cb80 1 -- 172.21.15.39:0/2605662797 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a8922021e0 msgr2=0x55a892202aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:01.940 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.897+0000 7f0e6224cb80 1 --2- 172.21.15.39:0/2605662797 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a8922021e0 0x55a892202aa0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f0e3809acc0 tx=0x7f0e38005550 comp rx=0 tx=0).stop 2023-07-21T18:41:01.941 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.897+0000 7f0e6224cb80 1 -- 172.21.15.39:0/2605662797 shutdown_connections 2023-07-21T18:41:01.941 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.897+0000 7f0e6224cb80 1 --2- 172.21.15.39:0/2605662797 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a892201890 0x55a892201ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.941 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.897+0000 7f0e6224cb80 1 --2- 172.21.15.39:0/2605662797 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a892203030 0x55a8922034a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.942 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.897+0000 7f0e6224cb80 1 --2- 172.21.15.39:0/2605662797 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a8922021e0 0x55a892202aa0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.942 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.897+0000 7f0e6224cb80 1 -- 172.21.15.39:0/2605662797 >> 172.21.15.39:0/2605662797 conn(0x55a8921f2d50 msgr2=0x55a8921f8180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:01.942 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.897+0000 7f0e6224cb80 1 -- 172.21.15.39:0/2605662797 shutdown_connections 2023-07-21T18:41:01.942 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.897+0000 7f0e6224cb80 1 -- 172.21.15.39:0/2605662797 wait complete. 2023-07-21T18:41:01.943 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.897+0000 7f0e6224cb80 1 -- start start 2023-07-21T18:41:01.943 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.898+0000 7f0e6224cb80 1 Processor -- start 2023-07-21T18:41:01.943 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.898+0000 7f0e6224cb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a892201890 0x55a89220b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.943 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.898+0000 7f0e6224cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a8922021e0 0x55a89220b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.944 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.898+0000 7f0e6224cb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a892203030 0x55a8922127c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.944 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.898+0000 7f0e6224cb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8920eea70 con 0x55a892201890 2023-07-21T18:41:01.944 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.898+0000 7f0e6224cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8920ee770 con 0x55a8922021e0 2023-07-21T18:41:01.944 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.898+0000 7f0e6224cb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a891fd28d0 con 0x55a892203030 2023-07-21T18:41:01.945 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.898+0000 7f0e6224cb80 10 client.0 _ll_register_callbacks cb 0x55a8920ec710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:41:01.945 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.898+0000 7f0e4f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a892201890 0x55a89220b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:01.945 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.898+0000 7f0e4f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a892201890 0x55a89220b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56018/0 (socket says 172.21.15.39:56018) 2023-07-21T18:41:01.946 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.898+0000 7f0e4f7fe700 1 -- 172.21.15.39:0/4133297485 learned_addr learned my addr 172.21.15.39:0/4133297485 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:01.946 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.898+0000 7f0e4ffff700 1 --2- 172.21.15.39:0/4133297485 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a892203030 0x55a8922127c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:01.946 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.898+0000 7f0e4effd700 1 --2- 172.21.15.39:0/4133297485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a8922021e0 0x55a89220b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:01.946 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.898+0000 7f0e4f7fe700 1 -- 172.21.15.39:0/4133297485 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a892203030 msgr2=0x55a8922127c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:01.947 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.898+0000 7f0e4f7fe700 1 --2- 172.21.15.39:0/4133297485 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a892203030 0x55a8922127c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.947 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.898+0000 7f0e4f7fe700 1 -- 172.21.15.39:0/4133297485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a8922021e0 msgr2=0x55a89220b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:01.947 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.898+0000 7f0e4f7fe700 1 --2- 172.21.15.39:0/4133297485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a8922021e0 0x55a89220b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.947 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.898+0000 7f0e4f7fe700 1 -- 172.21.15.39:0/4133297485 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e5019ddb0 con 0x55a892201890 2023-07-21T18:41:01.948 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.898+0000 7f0e4effd700 1 --2- 172.21.15.39:0/4133297485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a8922021e0 0x55a89220b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:41:01.948 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.898+0000 7f0e4ffff700 1 --2- 172.21.15.39:0/4133297485 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a892203030 0x55a8922127c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:41:01.948 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.898+0000 7f0e4f7fe700 1 --2- 172.21.15.39:0/4133297485 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a892201890 0x55a89220b240 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f0e4000dd20 tx=0x7f0e40010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b242a9b5a444e21 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:01.948 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.898+0000 7f0e2effd700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:41:01.949 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.899+0000 7f0e2effd700 1 -- 172.21.15.39:0/4133297485 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e40010e70 con 0x55a892201890 2023-07-21T18:41:01.949 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.899+0000 7f0e2effd700 1 -- 172.21.15.39:0/4133297485 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0e4001d440 con 0x55a892201890 2023-07-21T18:41:01.949 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.899+0000 7f0e6224cb80 1 -- 172.21.15.39:0/4133297485 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a8920a84a0 con 0x55a892201890 2023-07-21T18:41:01.950 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.899+0000 7f0e2effd700 1 -- 172.21.15.39:0/4133297485 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e4000dec0 con 0x55a892201890 2023-07-21T18:41:01.950 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.899+0000 7f0e6224cb80 10 client.15072 fetch_fsmap learned FSMap version 26 2023-07-21T18:41:01.950 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.899+0000 7f0e6224cb80 1 -- 172.21.15.39:0/4133297485 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55a891f65c10 con 0x55a892201890 2023-07-21T18:41:01.950 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.899+0000 7f0e2effd700 1 -- 172.21.15.39:0/4133297485 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a8920a84a0 con 0x55a892201890 2023-07-21T18:41:01.951 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.899+0000 7f0e2effd700 1 -- 172.21.15.39:0/4133297485 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0e4000f790 con 0x55a892201890 2023-07-21T18:41:01.951 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.899+0000 7f0e6224cb80 10 client.15072 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T18:41:01.951 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.899+0000 7f0e6224cb80 10 client.15072 resolve_mds: resolved name 'h' to daemon mds.h 2023-07-21T18:41:01.951 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.899+0000 7f0e6224cb80 20 client.15072 populate_metadata read hostname 'smithi039' 2023-07-21T18:41:01.952 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.899+0000 7f0e6224cb80 1 --2- 172.21.15.39:0/4133297485 >> [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] conn(0x55a8922165c0 0x55a892215970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.952 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.899+0000 7f0e6224cb80 4 client.15072 mds_command: new command op to 24500 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:41:01.952 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.899+0000 7f0e6224cb80 1 -- 172.21.15.39:0/4133297485 --> [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a892147e60 con 0x55a8922165c0 2023-07-21T18:41:01.952 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.900+0000 7f0e4effd700 1 --2- 172.21.15.39:0/4133297485 >> [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] conn(0x55a8922165c0 0x55a892215970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:01.953 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.900+0000 7f0e2effd700 10 client.15072 ms_handle_connect on v2:172.21.15.155:6836/947670056 2023-07-21T18:41:01.953 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.900+0000 7f0e4effd700 1 --2- 172.21.15.39:0/4133297485 >> [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] conn(0x55a8922165c0 0x55a892215970 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3b144b25f652eae2 server_cookie=40135adeed196823 in_seq=0 out_seq=0 2023-07-21T18:41:01.953 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.901+0000 7f0e2effd700 1 -- 172.21.15.39:0/4133297485 <== mds.1 v2:172.21.15.155:6836/947670056 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a892147e60 con 0x55a8922165c0 2023-07-21T18:41:01.954 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.901+0000 7f0e2effd700 10 client.15072 handle_command_reply: tid=0 2023-07-21T18:41:01.954 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.901+0000 7f0e6224cb80 1 client.15072 shutdown 2023-07-21T18:41:01.954 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.901+0000 7f0e6224cb80 1 -- 172.21.15.39:0/4133297485 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a892201890 msgr2=0x55a89220b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:01.955 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.901+0000 7f0e6224cb80 1 --2- 172.21.15.39:0/4133297485 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a892201890 0x55a89220b240 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f0e4000dd20 tx=0x7f0e40010740 comp rx=0 tx=0).stop 2023-07-21T18:41:01.955 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.901+0000 7f0e6224cb80 1 -- 172.21.15.39:0/4133297485 shutdown_connections 2023-07-21T18:41:01.955 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.901+0000 7f0e6224cb80 1 --2- 172.21.15.39:0/4133297485 >> [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] conn(0x55a8922165c0 0x55a892215970 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.955 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.901+0000 7f0e6224cb80 1 --2- 172.21.15.39:0/4133297485 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a892203030 0x55a8922127c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.956 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.901+0000 7f0e2effd700 0 client.15072 ms_handle_reset on v2:172.21.15.155:6836/947670056 2023-07-21T18:41:01.956 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.901+0000 7f0e6224cb80 1 --2- 172.21.15.39:0/4133297485 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a892201890 0x55a89220b240 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.956 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.901+0000 7f0e6224cb80 1 --2- 172.21.15.39:0/4133297485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a8922021e0 0x55a89220b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.957 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.901+0000 7f0e6224cb80 1 -- 172.21.15.39:0/4133297485 >> 172.21.15.39:0/4133297485 conn(0x55a8921f2d50 msgr2=0x55a8921f7860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:01.957 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.901+0000 7f0e6224cb80 1 -- 172.21.15.39:0/4133297485 shutdown_connections 2023-07-21T18:41:01.957 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.901+0000 7f0e6224cb80 1 -- 172.21.15.39:0/4133297485 wait complete. 2023-07-21T18:41:01.957 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.901+0000 7f0e6224cb80 20 client.15072 trim_cache size 0 max 16384 2023-07-21T18:41:01.965 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.919+0000 7f0e6224cb80 1 Processor -- start 2023-07-21T18:41:01.966 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.919+0000 7f0e6224cb80 1 -- start start 2023-07-21T18:41:01.966 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.919+0000 7f0e6224cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a892201890 0x55a89221f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.966 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.919+0000 7f0e6224cb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a8922021e0 0x55a892212780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.967 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.919+0000 7f0e6224cb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a892203030 0x55a8922094c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.967 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.919+0000 7f0e6224cb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8920a84a0 con 0x55a892203030 2023-07-21T18:41:01.967 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.919+0000 7f0e6224cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a89211d790 con 0x55a892201890 2023-07-21T18:41:01.967 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.919+0000 7f0e6224cb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a891fdede0 con 0x55a8922021e0 2023-07-21T18:41:01.968 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.919+0000 7f0e4ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a892203030 0x55a8922094c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:01.968 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.920+0000 7f0e4ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a892203030 0x55a8922094c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56030/0 (socket says 172.21.15.39:56030) 2023-07-21T18:41:01.968 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.920+0000 7f0e4ffff700 1 -- 172.21.15.39:0/3419026472 learned_addr learned my addr 172.21.15.39:0/3419026472 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:01.968 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.920+0000 7f0e4effd700 1 --2- 172.21.15.39:0/3419026472 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a8922021e0 0x55a892212780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:01.969 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.920+0000 7f0e4f7fe700 1 --2- 172.21.15.39:0/3419026472 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a892201890 0x55a89221f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:01.969 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.920+0000 7f0e4ffff700 1 -- 172.21.15.39:0/3419026472 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a8922021e0 msgr2=0x55a892212780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:01.969 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.920+0000 7f0e4ffff700 1 --2- 172.21.15.39:0/3419026472 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a8922021e0 0x55a892212780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.969 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.920+0000 7f0e4ffff700 1 -- 172.21.15.39:0/3419026472 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a892201890 msgr2=0x55a89221f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:01.970 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.920+0000 7f0e4ffff700 1 --2- 172.21.15.39:0/3419026472 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a892201890 0x55a89221f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.970 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.920+0000 7f0e4ffff700 1 -- 172.21.15.39:0/3419026472 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a891fa3200 con 0x55a892203030 2023-07-21T18:41:01.970 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.920+0000 7f0e4ffff700 1 --2- 172.21.15.39:0/3419026472 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a892203030 0x55a8922094c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0e48009c90 tx=0x7f0e48008450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b09c2f9579820410 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:01.971 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.920+0000 7f0e467fc700 1 -- 172.21.15.39:0/3419026472 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e48008e00 con 0x55a892203030 2023-07-21T18:41:01.971 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.920+0000 7f0e467fc700 1 -- 172.21.15.39:0/3419026472 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e48004bb0 con 0x55a892203030 2023-07-21T18:41:01.971 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.920+0000 7f0e467fc700 1 -- 172.21.15.39:0/3419026472 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e48014660 con 0x55a892203030 2023-07-21T18:41:01.971 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.921+0000 7f0e6224cb80 1 -- 172.21.15.39:0/3419026472 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a892203030 msgr2=0x55a8922094c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:01.972 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.921+0000 7f0e6224cb80 1 --2- 172.21.15.39:0/3419026472 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a892203030 0x55a8922094c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0e48009c90 tx=0x7f0e48008450 comp rx=0 tx=0).stop 2023-07-21T18:41:01.972 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.921+0000 7f0e6224cb80 1 -- 172.21.15.39:0/3419026472 shutdown_connections 2023-07-21T18:41:01.972 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.921+0000 7f0e6224cb80 1 --2- 172.21.15.39:0/3419026472 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a892201890 0x55a89221f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.972 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.921+0000 7f0e6224cb80 1 --2- 172.21.15.39:0/3419026472 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a892203030 0x55a8922094c0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.973 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.921+0000 7f0e6224cb80 1 --2- 172.21.15.39:0/3419026472 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a8922021e0 0x55a892212780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.973 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.921+0000 7f0e6224cb80 1 -- 172.21.15.39:0/3419026472 >> 172.21.15.39:0/3419026472 conn(0x55a8921f2d50 msgr2=0x55a8921f42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:01.973 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.921+0000 7f0e6224cb80 1 -- 172.21.15.39:0/3419026472 shutdown_connections 2023-07-21T18:41:01.974 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.921+0000 7f0e6224cb80 1 -- 172.21.15.39:0/3419026472 wait complete. 2023-07-21T18:41:01.974 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.921+0000 7f0e6224cb80 1 -- start start 2023-07-21T18:41:01.974 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.921+0000 7f0e6224cb80 1 Processor -- start 2023-07-21T18:41:01.974 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.922+0000 7f0e6224cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a892201890 0x55a89220f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.975 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.922+0000 7f0e6224cb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a8922021e0 0x55a89220ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.975 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.922+0000 7f0e6224cb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a892203030 0x55a89226a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.975 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.922+0000 7f0e6224cb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8922128d0 con 0x55a892203030 2023-07-21T18:41:01.975 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.922+0000 7f0e6224cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a892212a40 con 0x55a892201890 2023-07-21T18:41:01.976 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.922+0000 7f0e6224cb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a89226a980 con 0x55a8922021e0 2023-07-21T18:41:01.976 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.922+0000 7f0e6224cb80 10 client.0 _ll_register_callbacks cb 0x7f0e4000d900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:41:01.976 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.922+0000 7f0e4ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a892203030 0x55a89226a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:01.976 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.922+0000 7f0e4ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a892203030 0x55a89226a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56036/0 (socket says 172.21.15.39:56036) 2023-07-21T18:41:01.977 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.922+0000 7f0e4ffff700 1 -- 172.21.15.39:0/1363282573 learned_addr learned my addr 172.21.15.39:0/1363282573 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:01.977 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.922+0000 7f0e4f7fe700 1 --2- 172.21.15.39:0/1363282573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a892201890 0x55a89220f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:01.977 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.922+0000 7f0e4effd700 1 --2- 172.21.15.39:0/1363282573 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a8922021e0 0x55a89220ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:01.977 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.922+0000 7f0e4ffff700 1 -- 172.21.15.39:0/1363282573 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a8922021e0 msgr2=0x55a89220ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:01.978 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.922+0000 7f0e4ffff700 1 --2- 172.21.15.39:0/1363282573 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a8922021e0 0x55a89220ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.978 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.922+0000 7f0e4ffff700 1 -- 172.21.15.39:0/1363282573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a892201890 msgr2=0x55a89220f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:01.978 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.922+0000 7f0e4ffff700 1 --2- 172.21.15.39:0/1363282573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a892201890 0x55a89220f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.979 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.922+0000 7f0e4ffff700 1 -- 172.21.15.39:0/1363282573 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e48009940 con 0x55a892203030 2023-07-21T18:41:01.979 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.922+0000 7f0e4f7fe700 1 --2- 172.21.15.39:0/1363282573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a892201890 0x55a89220f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:41:01.979 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.922+0000 7f0e4ffff700 1 --2- 172.21.15.39:0/1363282573 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a892203030 0x55a89226a440 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f0e48009300 tx=0x7f0e48004d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48d28683c7b17dce server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:01.979 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.922+0000 7f0e2effd700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:41:01.980 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.922+0000 7f0e2effd700 1 -- 172.21.15.39:0/1363282573 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e480054b0 con 0x55a892203030 2023-07-21T18:41:01.980 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.922+0000 7f0e2effd700 1 -- 172.21.15.39:0/1363282573 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0e4801e460 con 0x55a892203030 2023-07-21T18:41:01.980 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.922+0000 7f0e6224cb80 1 -- 172.21.15.39:0/1363282573 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0e380990d0 con 0x55a892203030 2023-07-21T18:41:01.980 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.922+0000 7f0e2effd700 1 -- 172.21.15.39:0/1363282573 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e48005610 con 0x55a892203030 2023-07-21T18:41:01.981 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.922+0000 7f0e6224cb80 10 client.15078 fetch_fsmap learned FSMap version 26 2023-07-21T18:41:01.981 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.922+0000 7f0e6224cb80 1 -- 172.21.15.39:0/1363282573 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55a891f7b260 con 0x55a892203030 2023-07-21T18:41:01.981 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.923+0000 7f0e2effd700 1 -- 172.21.15.39:0/1363282573 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e4802e020 con 0x55a892203030 2023-07-21T18:41:01.981 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.923+0000 7f0e2effd700 1 -- 172.21.15.39:0/1363282573 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0e4801c040 con 0x55a892203030 2023-07-21T18:41:01.982 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.923+0000 7f0e6224cb80 10 client.15078 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T18:41:01.982 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.923+0000 7f0e6224cb80 10 client.15078 resolve_mds: resolved name 'h' to daemon mds.h 2023-07-21T18:41:01.982 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.923+0000 7f0e6224cb80 20 client.15078 populate_metadata read hostname 'smithi039' 2023-07-21T18:41:01.982 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.923+0000 7f0e6224cb80 1 --2- 172.21.15.39:0/1363282573 >> [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] conn(0x55a89226f5f0 0x55a89226bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:01.983 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.923+0000 7f0e6224cb80 4 client.15078 mds_command: new command op to 24500 tid=0[{"prefix": "status"}] 2023-07-21T18:41:01.983 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.923+0000 7f0e6224cb80 1 -- 172.21.15.39:0/1363282573 --> [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a89210a000 con 0x55a89226f5f0 2023-07-21T18:41:01.983 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.923+0000 7f0e4f7fe700 1 --2- 172.21.15.39:0/1363282573 >> [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] conn(0x55a89226f5f0 0x55a89226bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:01.984 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.923+0000 7f0e2effd700 10 client.15078 ms_handle_connect on v2:172.21.15.155:6836/947670056 2023-07-21T18:41:01.984 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.923+0000 7f0e4f7fe700 1 --2- 172.21.15.39:0/1363282573 >> [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] conn(0x55a89226f5f0 0x55a89226bf40 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9b2db3eebe4bbace server_cookie=c8e64d076c00793b in_seq=0 out_seq=0 2023-07-21T18:41:01.984 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.924+0000 7f0e2effd700 1 -- 172.21.15.39:0/1363282573 <== mds.1 v2:172.21.15.155:6836/947670056 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55a89210a000 con 0x55a89226f5f0 2023-07-21T18:41:01.984 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.924+0000 7f0e2effd700 10 client.15078 handle_command_reply: tid=0 2023-07-21T18:41:01.984 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.924+0000 7f0e6224cb80 1 client.15078 shutdown 2023-07-21T18:41:01.985 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.924+0000 7f0e6224cb80 1 -- 172.21.15.39:0/1363282573 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a892203030 msgr2=0x55a89226a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:01.985 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.924+0000 7f0e6224cb80 1 --2- 172.21.15.39:0/1363282573 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a892203030 0x55a89226a440 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f0e48009300 tx=0x7f0e48004d20 comp rx=0 tx=0).stop 2023-07-21T18:41:01.985 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.924+0000 7f0e6224cb80 1 -- 172.21.15.39:0/1363282573 shutdown_connections 2023-07-21T18:41:01.985 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.924+0000 7f0e6224cb80 1 --2- 172.21.15.39:0/1363282573 >> [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] conn(0x55a89226f5f0 0x55a89226bf40 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.985 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.924+0000 7f0e6224cb80 1 --2- 172.21.15.39:0/1363282573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a892201890 0x55a89220f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.985 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.924+0000 7f0e2effd700 0 client.15078 ms_handle_reset on v2:172.21.15.155:6836/947670056 2023-07-21T18:41:01.985 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.924+0000 7f0e6224cb80 1 --2- 172.21.15.39:0/1363282573 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a892203030 0x55a89226a440 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.986 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.924+0000 7f0e6224cb80 1 --2- 172.21.15.39:0/1363282573 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a8922021e0 0x55a89220ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.986 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.924+0000 7f0e6224cb80 1 -- 172.21.15.39:0/1363282573 >> 172.21.15.39:0/1363282573 conn(0x55a8921f2d50 msgr2=0x55a89221f310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:01.986 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.924+0000 7f0e6224cb80 1 -- 172.21.15.39:0/1363282573 shutdown_connections 2023-07-21T18:41:01.986 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.924+0000 7f0e6224cb80 1 -- 172.21.15.39:0/1363282573 wait complete. 2023-07-21T18:41:01.986 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.924+0000 7f0e6224cb80 20 client.15078 trim_cache size 0 max 16384 2023-07-21T18:41:01.986 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2023-07-21T18:41:01.987 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "eaf8e25a-27f4-11ee-9b34-001a4aab830c", 2023-07-21T18:41:01.987 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 1, 2023-07-21T18:41:01.987 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 24500, 2023-07-21T18:41:01.987 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2023-07-21T18:41:01.987 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2023-07-21T18:41:01.987 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2023-07-21T18:41:01.987 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 53.722465945000003, 2023-07-21T18:41:01.988 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 26, 2023-07-21T18:41:01.988 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 79, 2023-07-21T18:41:01.988 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 79, 2023-07-21T18:41:01.988 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 72.454977033000006 2023-07-21T18:41:01.988 INFO:teuthology.orchestra.run.smithi039.stdout:} 2023-07-21T18:41:01.989 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.925+0000 7f0e560c6700 1 -- 172.21.15.39:0/88305486 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f0e3c0a11e0 msgr2=0x7f0e3c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:01.989 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.925+0000 7f0e560c6700 1 --2- 172.21.15.39:0/88305486 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f0e3c0a11e0 0x7f0e3c0a3690 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0e500673d0 tx=0x7f0e40009470 comp rx=0 tx=0).stop 2023-07-21T18:41:01.990 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.925+0000 7f0e560c6700 1 -- 172.21.15.39:0/88305486 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e50100e00 msgr2=0x7f0e50195d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:01.990 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.925+0000 7f0e560c6700 1 --2- 172.21.15.39:0/88305486 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e50100e00 0x7f0e50195d70 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0e38004760 tx=0x7f0e38009f90 comp rx=0 tx=0).stop 2023-07-21T18:41:01.990 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.925+0000 7f0e560c6700 1 -- 172.21.15.39:0/88305486 shutdown_connections 2023-07-21T18:41:01.990 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.925+0000 7f0e560c6700 1 --2- 172.21.15.39:0/88305486 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f0e3c0a11e0 0x7f0e3c0a3690 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.990 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.925+0000 7f0e560c6700 1 --2- 172.21.15.39:0/88305486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e50101790 0x7f0e5019cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.990 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.925+0000 7f0e560c6700 1 --2- 172.21.15.39:0/88305486 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e500fdd90 0x7f0e50195830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.990 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.925+0000 7f0e560c6700 1 --2- 172.21.15.39:0/88305486 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e50100e00 0x7f0e50195d70 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:01.991 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.925+0000 7f0e560c6700 1 -- 172.21.15.39:0/88305486 >> 172.21.15.39:0/88305486 conn(0x7f0e500f9770 msgr2=0x7f0e500fba60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:01.991 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.925+0000 7f0e560c6700 1 -- 172.21.15.39:0/88305486 shutdown_connections 2023-07-21T18:41:01.991 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:01.925+0000 7f0e560c6700 1 -- 172.21.15.39:0/88305486 wait complete. 2023-07-21T18:41:01.991 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-07-21T18:41:02.123 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.099+0000 7ff3cc372700 1 -- 172.21.15.39:0/1511378437 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3c40fe9e0 msgr2=0x7ff3c40fedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:02.124 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.099+0000 7ff3cc372700 1 --2- 172.21.15.39:0/1511378437 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3c40fe9e0 0x7ff3c40fedf0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff3b4009990 tx=0x7ff3b4009ca0 comp rx=0 tx=0).stop 2023-07-21T18:41:02.124 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.099+0000 7ff3cc372700 1 -- 172.21.15.39:0/1511378437 shutdown_connections 2023-07-21T18:41:02.124 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.099+0000 7ff3cc372700 1 --2- 172.21.15.39:0/1511378437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3c4100560 0x7ff3c4106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:02.124 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.099+0000 7ff3cc372700 1 --2- 172.21.15.39:0/1511378437 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3c40fe9e0 0x7ff3c40fedf0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:02.124 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.099+0000 7ff3cc372700 1 --2- 172.21.15.39:0/1511378437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff3c40ffbd0 0x7ff3c4100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:02.124 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.099+0000 7ff3cc372700 1 -- 172.21.15.39:0/1511378437 >> 172.21.15.39:0/1511378437 conn(0x7ff3c40fa000 msgr2=0x7ff3c40fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:02.124 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.099+0000 7ff3cc372700 1 -- 172.21.15.39:0/1511378437 shutdown_connections 2023-07-21T18:41:02.125 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.099+0000 7ff3cc372700 1 -- 172.21.15.39:0/1511378437 wait complete. 2023-07-21T18:41:02.126 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.099+0000 7ff3cc372700 1 Processor -- start 2023-07-21T18:41:02.126 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.099+0000 7ff3cc372700 1 -- start start 2023-07-21T18:41:02.127 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.099+0000 7ff3cc372700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3c40fe9e0 0x7ff3c41958e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:02.127 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.099+0000 7ff3cc372700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3c40ffbd0 0x7ff3c4195e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:02.127 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.099+0000 7ff3cc372700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff3c4100560 0x7ff3c419ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:02.127 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.099+0000 7ff3cc372700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c419d460 con 0x7ff3c40fe9e0 2023-07-21T18:41:02.127 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.099+0000 7ff3cc372700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c419d5a0 con 0x7ff3c40ffbd0 2023-07-21T18:41:02.127 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.099+0000 7ff3cc372700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c419d730 con 0x7ff3c4100560 2023-07-21T18:41:02.127 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.100+0000 7ff3ca10e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3c40fe9e0 0x7ff3c41958e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:02.128 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.100+0000 7ff3ca10e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3c40fe9e0 0x7ff3c41958e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56046/0 (socket says 172.21.15.39:56046) 2023-07-21T18:41:02.128 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.100+0000 7ff3ca10e700 1 -- 172.21.15.39:0/3347440548 learned_addr learned my addr 172.21.15.39:0/3347440548 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:02.128 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.100+0000 7ff3c990d700 1 --2- 172.21.15.39:0/3347440548 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3c40ffbd0 0x7ff3c4195e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:02.128 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.100+0000 7ff3ca10e700 1 -- 172.21.15.39:0/3347440548 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff3c4100560 msgr2=0x7ff3c419ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:02.128 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.100+0000 7ff3ca90f700 1 --2- 172.21.15.39:0/3347440548 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff3c4100560 0x7ff3c419ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:02.128 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.100+0000 7ff3ca10e700 1 --2- 172.21.15.39:0/3347440548 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff3c4100560 0x7ff3c419ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:02.128 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.100+0000 7ff3ca10e700 1 -- 172.21.15.39:0/3347440548 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3c40ffbd0 msgr2=0x7ff3c4195e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:02.128 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.100+0000 7ff3ca10e700 1 --2- 172.21.15.39:0/3347440548 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3c40ffbd0 0x7ff3c4195e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:02.129 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.100+0000 7ff3ca10e700 1 -- 172.21.15.39:0/3347440548 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3b40095f0 con 0x7ff3c40fe9e0 2023-07-21T18:41:02.129 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.100+0000 7ff3ca10e700 1 --2- 172.21.15.39:0/3347440548 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3c40fe9e0 0x7ff3c41958e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff3b40059d0 tx=0x7ff3b4005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:02.129 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.100+0000 7ff3bb7fe700 1 -- 172.21.15.39:0/3347440548 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b401d070 con 0x7ff3c40fe9e0 2023-07-21T18:41:02.129 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.100+0000 7ff3bb7fe700 1 -- 172.21.15.39:0/3347440548 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff3b40040e0 con 0x7ff3c40fe9e0 2023-07-21T18:41:02.129 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.100+0000 7ff3bb7fe700 1 -- 172.21.15.39:0/3347440548 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b400f460 con 0x7ff3c40fe9e0 2023-07-21T18:41:02.129 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.100+0000 7ff3cc372700 1 -- 172.21.15.39:0/3347440548 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3c419d9b0 con 0x7ff3c40fe9e0 2023-07-21T18:41:02.129 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.100+0000 7ff3cc372700 1 -- 172.21.15.39:0/3347440548 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3c419de00 con 0x7ff3c40fe9e0 2023-07-21T18:41:02.134 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.102+0000 7ff3bb7fe700 1 -- 172.21.15.39:0/3347440548 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7ff3b4003c20 con 0x7ff3c40fe9e0 2023-07-21T18:41:02.134 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.103+0000 7ff3bb7fe700 1 --2- 172.21.15.39:0/3347440548 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7ff3b00a11e0 0x7ff3b00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:02.134 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.103+0000 7ff3bb7fe700 1 -- 172.21.15.39:0/3347440548 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff3b40cde40 con 0x7ff3c40fe9e0 2023-07-21T18:41:02.134 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.103+0000 7ff3c990d700 1 --2- 172.21.15.39:0/3347440548 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7ff3b00a11e0 0x7ff3b00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:02.134 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.103+0000 7ff3c990d700 1 --2- 172.21.15.39:0/3347440548 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7ff3b00a11e0 0x7ff3b00a3690 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0009a10 tx=0x7ff3c0009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:02.134 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.104+0000 7ff3d84f8b80 1 Processor -- start 2023-07-21T18:41:02.134 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.104+0000 7ff3d84f8b80 1 -- start start 2023-07-21T18:41:02.135 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.104+0000 7ff3d84f8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561156ce8480 0x561156ce8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:02.135 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.104+0000 7ff3d84f8b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561156ce8dd0 0x561156ce9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:02.135 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.104+0000 7ff3d84f8b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561156ce9c20 0x561156cea090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:02.135 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.104+0000 7ff3d84f8b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x561156bedc60 con 0x561156ce8dd0 2023-07-21T18:41:02.135 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.104+0000 7ff3d84f8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561156c409f0 con 0x561156ce8480 2023-07-21T18:41:02.135 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.104+0000 7ff3d84f8b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x561156c39e80 con 0x561156ce9c20 2023-07-21T18:41:02.135 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.104+0000 7ff3c990d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561156ce8dd0 0x561156ce9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:02.136 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.104+0000 7ff3c990d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561156ce8dd0 0x561156ce9690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56056/0 (socket says 172.21.15.39:56056) 2023-07-21T18:41:02.136 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.104+0000 7ff3c990d700 1 -- 172.21.15.39:0/3995444082 learned_addr learned my addr 172.21.15.39:0/3995444082 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:02.136 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.105+0000 7ff3ca10e700 1 --2- 172.21.15.39:0/3995444082 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561156ce8480 0x561156ce8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:02.136 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.105+0000 7ff3ca90f700 1 --2- 172.21.15.39:0/3995444082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561156ce9c20 0x561156cea090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:02.136 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.105+0000 7ff3c990d700 1 -- 172.21.15.39:0/3995444082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561156ce9c20 msgr2=0x561156cea090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:02.136 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.105+0000 7ff3c990d700 1 --2- 172.21.15.39:0/3995444082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561156ce9c20 0x561156cea090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:02.136 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.105+0000 7ff3c990d700 1 -- 172.21.15.39:0/3995444082 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561156ce8480 msgr2=0x561156ce8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:02.136 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.105+0000 7ff3c990d700 1 --2- 172.21.15.39:0/3995444082 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561156ce8480 0x561156ce8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:02.137 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.105+0000 7ff3c990d700 1 -- 172.21.15.39:0/3995444082 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561156a67d90 con 0x561156ce8dd0 2023-07-21T18:41:02.137 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.105+0000 7ff3c990d700 1 --2- 172.21.15.39:0/3995444082 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561156ce8dd0 0x561156ce9690 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff3c000ea00 tx=0x7ff3c0011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45e0b81eb12e2297 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:02.137 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.105+0000 7ff3b8dac700 1 -- 172.21.15.39:0/3995444082 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c000eec0 con 0x561156ce8dd0 2023-07-21T18:41:02.137 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.105+0000 7ff3b8dac700 1 -- 172.21.15.39:0/3995444082 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff3c0015940 con 0x561156ce8dd0 2023-07-21T18:41:02.137 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.105+0000 7ff3b8dac700 1 -- 172.21.15.39:0/3995444082 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c0013970 con 0x561156ce8dd0 2023-07-21T18:41:02.137 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.106+0000 7ff3d84f8b80 1 -- 172.21.15.39:0/3995444082 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561156ce8dd0 msgr2=0x561156ce9690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:02.137 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.106+0000 7ff3d84f8b80 1 --2- 172.21.15.39:0/3995444082 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561156ce8dd0 0x561156ce9690 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff3c000ea00 tx=0x7ff3c0011000 comp rx=0 tx=0).stop 2023-07-21T18:41:02.138 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.106+0000 7ff3d84f8b80 1 -- 172.21.15.39:0/3995444082 shutdown_connections 2023-07-21T18:41:02.138 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.106+0000 7ff3d84f8b80 1 --2- 172.21.15.39:0/3995444082 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561156ce8480 0x561156ce8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:02.138 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.106+0000 7ff3d84f8b80 1 --2- 172.21.15.39:0/3995444082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561156ce9c20 0x561156cea090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:02.138 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.106+0000 7ff3d84f8b80 1 --2- 172.21.15.39:0/3995444082 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561156ce8dd0 0x561156ce9690 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:02.138 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.106+0000 7ff3d84f8b80 1 -- 172.21.15.39:0/3995444082 >> 172.21.15.39:0/3995444082 conn(0x561156cddd50 msgr2=0x561156ce3180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:02.138 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.106+0000 7ff3d84f8b80 1 -- 172.21.15.39:0/3995444082 shutdown_connections 2023-07-21T18:41:02.138 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.106+0000 7ff3d84f8b80 1 -- 172.21.15.39:0/3995444082 wait complete. 2023-07-21T18:41:02.139 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.106+0000 7ff3d84f8b80 1 -- start start 2023-07-21T18:41:02.139 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.106+0000 7ff3d84f8b80 1 Processor -- start 2023-07-21T18:41:02.139 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.107+0000 7ff3d84f8b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561156ce8480 0x561156cf2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:02.139 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.107+0000 7ff3d84f8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561156ce8dd0 0x561156cf2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:02.139 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.107+0000 7ff3d84f8b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561156ce9c20 0x561156cf9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:02.139 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.107+0000 7ff3d84f8b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x561156bd9a70 con 0x561156ce9c20 2023-07-21T18:41:02.139 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.107+0000 7ff3d84f8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561156bd9770 con 0x561156ce8dd0 2023-07-21T18:41:02.140 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.107+0000 7ff3d84f8b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x561156abd8d0 con 0x561156ce8480 2023-07-21T18:41:02.140 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.107+0000 7ff3d84f8b80 10 client.0 _ll_register_callbacks cb 0x561156bd7710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:41:02.140 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.107+0000 7ff3ca90f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561156ce9c20 0x561156cf9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:02.140 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.107+0000 7ff3ca90f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561156ce9c20 0x561156cf9680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56064/0 (socket says 172.21.15.39:56064) 2023-07-21T18:41:02.140 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.107+0000 7ff3ca90f700 1 -- 172.21.15.39:0/3966415582 learned_addr learned my addr 172.21.15.39:0/3966415582 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:02.140 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.107+0000 7ff3ca10e700 1 --2- 172.21.15.39:0/3966415582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561156ce8480 0x561156cf2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:02.140 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.107+0000 7ff3c990d700 1 --2- 172.21.15.39:0/3966415582 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561156ce8dd0 0x561156cf2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:02.141 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.107+0000 7ff3ca90f700 1 -- 172.21.15.39:0/3966415582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561156ce8480 msgr2=0x561156cf2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:02.141 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.107+0000 7ff3ca90f700 1 --2- 172.21.15.39:0/3966415582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561156ce8480 0x561156cf2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:02.141 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.107+0000 7ff3ca90f700 1 -- 172.21.15.39:0/3966415582 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561156ce8dd0 msgr2=0x561156cf2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:02.141 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.107+0000 7ff3ca90f700 1 --2- 172.21.15.39:0/3966415582 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561156ce8dd0 0x561156cf2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:02.141 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.107+0000 7ff3ca90f700 1 -- 172.21.15.39:0/3966415582 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3c000e6e0 con 0x561156ce9c20 2023-07-21T18:41:02.141 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.107+0000 7ff3c990d700 1 --2- 172.21.15.39:0/3966415582 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561156ce8dd0 0x561156cf2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:41:02.141 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.107+0000 7ff3ca10e700 1 --2- 172.21.15.39:0/3966415582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561156ce8480 0x561156cf2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:41:02.142 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.107+0000 7ff3ca90f700 1 --2- 172.21.15.39:0/3966415582 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561156ce9c20 0x561156cf9680 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc007bd0 tx=0x7ff3bc007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=555c9f7e92653439 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:02.142 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.107+0000 7ff3adffb700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:41:02.142 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.107+0000 7ff3adffb700 1 -- 172.21.15.39:0/3966415582 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc00fa30 con 0x561156ce9c20 2023-07-21T18:41:02.142 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.107+0000 7ff3d84f8b80 1 -- 172.21.15.39:0/3966415582 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561156b934a0 con 0x561156ce9c20 2023-07-21T18:41:02.142 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.107+0000 7ff3adffb700 1 -- 172.21.15.39:0/3966415582 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff3bc00fb90 con 0x561156ce9c20 2023-07-21T18:41:02.142 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.107+0000 7ff3adffb700 1 -- 172.21.15.39:0/3966415582 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff3bc00fd50 con 0x561156ce9c20 2023-07-21T18:41:02.142 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.107+0000 7ff3d84f8b80 10 client.15090 fetch_fsmap learned FSMap version 26 2023-07-21T18:41:02.142 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.108+0000 7ff3d84f8b80 1 -- 172.21.15.39:0/3966415582 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x561156a50c10 con 0x561156ce9c20 2023-07-21T18:41:02.143 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.108+0000 7ff3adffb700 1 -- 172.21.15.39:0/3966415582 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff3bc026080 con 0x561156ce9c20 2023-07-21T18:41:02.143 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.108+0000 7ff3adffb700 1 -- 172.21.15.39:0/3966415582 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff3bc022450 con 0x561156ce9c20 2023-07-21T18:41:02.143 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.108+0000 7ff3d84f8b80 10 client.15090 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T18:41:02.143 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.108+0000 7ff3d84f8b80 10 client.15090 resolve_mds: resolved name 'b' to daemon mds.b 2023-07-21T18:41:02.143 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.108+0000 7ff3d84f8b80 20 client.15090 populate_metadata read hostname 'smithi039' 2023-07-21T18:41:02.143 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.108+0000 7ff3d84f8b80 1 --2- 172.21.15.39:0/3966415582 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x561156cfd480 0x561156cfc830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:02.143 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.108+0000 7ff3d84f8b80 4 client.15090 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:41:02.143 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.108+0000 7ff3d84f8b80 1 -- 172.21.15.39:0/3966415582 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561156c32e60 con 0x561156cfd480 2023-07-21T18:41:02.144 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.108+0000 7ff3ca10e700 1 --2- 172.21.15.39:0/3966415582 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x561156cfd480 0x561156cfc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:02.144 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.109+0000 7ff3ca10e700 1 --2- 172.21.15.39:0/3966415582 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x561156cfd480 0x561156cfc830 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8046082f6a70b9b4 server_cookie=be0bedcc8a257464 in_seq=0 out_seq=0 2023-07-21T18:41:02.144 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.109+0000 7ff3adffb700 10 client.15090 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:41:02.145 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.110+0000 7ff3adffb700 1 -- 172.21.15.39:0/3966415582 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561156c32e60 con 0x561156cfd480 2023-07-21T18:41:02.145 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.110+0000 7ff3adffb700 10 client.15090 handle_command_reply: tid=0 2023-07-21T18:41:02.145 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.110+0000 7ff3d84f8b80 1 client.15090 shutdown 2023-07-21T18:41:02.145 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.110+0000 7ff3d84f8b80 1 -- 172.21.15.39:0/3966415582 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561156ce9c20 msgr2=0x561156cf9680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:02.145 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.110+0000 7ff3d84f8b80 1 --2- 172.21.15.39:0/3966415582 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561156ce9c20 0x561156cf9680 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc007bd0 tx=0x7ff3bc007f90 comp rx=0 tx=0).stop 2023-07-21T18:41:02.146 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.110+0000 7ff3d84f8b80 1 -- 172.21.15.39:0/3966415582 shutdown_connections 2023-07-21T18:41:02.146 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.110+0000 7ff3d84f8b80 1 --2- 172.21.15.39:0/3966415582 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x561156cfd480 0x561156cfc830 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:02.146 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.110+0000 7ff3d84f8b80 1 --2- 172.21.15.39:0/3966415582 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561156ce9c20 0x561156cf9680 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:02.146 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.110+0000 7ff3d84f8b80 1 --2- 172.21.15.39:0/3966415582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561156ce8480 0x561156cf2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:02.146 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.110+0000 7ff3adffb700 0 client.15090 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:41:02.146 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.110+0000 7ff3d84f8b80 1 --2- 172.21.15.39:0/3966415582 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561156ce8dd0 0x561156cf2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:02.146 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.110+0000 7ff3d84f8b80 1 -- 172.21.15.39:0/3966415582 >> 172.21.15.39:0/3966415582 conn(0x561156cddd50 msgr2=0x561156ce2860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:02.146 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.110+0000 7ff3d84f8b80 1 -- 172.21.15.39:0/3966415582 shutdown_connections 2023-07-21T18:41:02.147 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.110+0000 7ff3d84f8b80 1 -- 172.21.15.39:0/3966415582 wait complete. 2023-07-21T18:41:02.147 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.110+0000 7ff3d84f8b80 20 client.15090 trim_cache size 0 max 16384 2023-07-21T18:41:02.153 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.128+0000 7ff3d84f8b80 1 Processor -- start 2023-07-21T18:41:02.153 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.128+0000 7ff3d84f8b80 1 -- start start 2023-07-21T18:41:02.153 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.128+0000 7ff3d84f8b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561156ce8480 0x561156cf9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:02.154 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.128+0000 7ff3d84f8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561156ce8dd0 0x561156cf95a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:02.154 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.128+0000 7ff3d84f8b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561156ce9c20 0x561156ceffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:02.154 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.128+0000 7ff3d84f8b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x561156cf1cd0 con 0x561156ce8480 2023-07-21T18:41:02.154 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.128+0000 7ff3d84f8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561156c08790 con 0x561156ce8dd0 2023-07-21T18:41:02.154 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.128+0000 7ff3d84f8b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x561156bf8480 con 0x561156ce9c20 2023-07-21T18:41:02.154 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.128+0000 7ff3ca10e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561156ce8480 0x561156cf9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:02.154 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.128+0000 7ff3ca10e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561156ce8480 0x561156cf9e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56068/0 (socket says 172.21.15.39:56068) 2023-07-21T18:41:02.155 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.128+0000 7ff3ca10e700 1 -- 172.21.15.39:0/2274642156 learned_addr learned my addr 172.21.15.39:0/2274642156 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:02.155 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.128+0000 7ff3ca90f700 1 --2- 172.21.15.39:0/2274642156 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561156ce9c20 0x561156ceffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:02.155 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.128+0000 7ff3c990d700 1 --2- 172.21.15.39:0/2274642156 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561156ce8dd0 0x561156cf95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:02.155 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.128+0000 7ff3ca10e700 1 -- 172.21.15.39:0/2274642156 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561156ce9c20 msgr2=0x561156ceffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:02.155 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.128+0000 7ff3ca10e700 1 --2- 172.21.15.39:0/2274642156 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561156ce9c20 0x561156ceffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:02.155 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.128+0000 7ff3ca10e700 1 -- 172.21.15.39:0/2274642156 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561156ce8dd0 msgr2=0x561156cf95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:02.155 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.128+0000 7ff3ca10e700 1 --2- 172.21.15.39:0/2274642156 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561156ce8dd0 0x561156cf95a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:02.155 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.128+0000 7ff3ca10e700 1 -- 172.21.15.39:0/2274642156 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561156cea2c0 con 0x561156ce8480 2023-07-21T18:41:02.156 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.129+0000 7ff3ca10e700 1 --2- 172.21.15.39:0/2274642156 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561156ce8480 0x561156cf9e50 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff3b40861b0 tx=0x7ff3b40a1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df06fd648bb36375 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:02.156 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.129+0000 7ff3b95ad700 1 -- 172.21.15.39:0/2274642156 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b409f170 con 0x561156ce8480 2023-07-21T18:41:02.156 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.129+0000 7ff3b95ad700 1 -- 172.21.15.39:0/2274642156 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff3b40a36f0 con 0x561156ce8480 2023-07-21T18:41:02.156 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.129+0000 7ff3b95ad700 1 -- 172.21.15.39:0/2274642156 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b409a820 con 0x561156ce8480 2023-07-21T18:41:02.159 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.129+0000 7ff3d84f8b80 1 -- 172.21.15.39:0/2274642156 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561156ce8480 msgr2=0x561156cf9e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:02.159 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.129+0000 7ff3d84f8b80 1 --2- 172.21.15.39:0/2274642156 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561156ce8480 0x561156cf9e50 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff3b40861b0 tx=0x7ff3b40a1040 comp rx=0 tx=0).stop 2023-07-21T18:41:02.159 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.129+0000 7ff3d84f8b80 1 -- 172.21.15.39:0/2274642156 shutdown_connections 2023-07-21T18:41:02.160 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.129+0000 7ff3d84f8b80 1 --2- 172.21.15.39:0/2274642156 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561156ce9c20 0x561156ceffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:02.160 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.129+0000 7ff3d84f8b80 1 --2- 172.21.15.39:0/2274642156 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561156ce8480 0x561156cf9e50 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:02.160 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.129+0000 7ff3d84f8b80 1 --2- 172.21.15.39:0/2274642156 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561156ce8dd0 0x561156cf95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:02.160 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.129+0000 7ff3d84f8b80 1 -- 172.21.15.39:0/2274642156 >> 172.21.15.39:0/2274642156 conn(0x561156cddd50 msgr2=0x561156cf0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:02.160 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.129+0000 7ff3d84f8b80 1 -- 172.21.15.39:0/2274642156 shutdown_connections 2023-07-21T18:41:02.160 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.129+0000 7ff3d84f8b80 1 -- 172.21.15.39:0/2274642156 wait complete. 2023-07-21T18:41:02.160 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.130+0000 7ff3d84f8b80 1 -- start start 2023-07-21T18:41:02.161 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.130+0000 7ff3d84f8b80 1 Processor -- start 2023-07-21T18:41:02.161 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.130+0000 7ff3d84f8b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561156ce8480 0x561156ceee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:02.161 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.130+0000 7ff3d84f8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561156ce8dd0 0x561156cf95a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:02.161 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.130+0000 7ff3d84f8b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561156ce9c20 0x561156cef4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:02.161 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.130+0000 7ff3d84f8b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x561156d514d0 con 0x561156ce9c20 2023-07-21T18:41:02.161 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.130+0000 7ff3d84f8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561156d51640 con 0x561156ce8dd0 2023-07-21T18:41:02.161 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.130+0000 7ff3d84f8b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x561156d51800 con 0x561156ce8480 2023-07-21T18:41:02.162 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.130+0000 7ff3d84f8b80 10 client.0 _ll_register_callbacks cb 0x7ff3bc007410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:41:02.162 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.130+0000 7ff3ca90f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561156ce9c20 0x561156cef4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:02.162 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.130+0000 7ff3ca90f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561156ce9c20 0x561156cef4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56078/0 (socket says 172.21.15.39:56078) 2023-07-21T18:41:02.162 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.130+0000 7ff3ca90f700 1 -- 172.21.15.39:0/4023413221 learned_addr learned my addr 172.21.15.39:0/4023413221 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:02.162 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.130+0000 7ff3ca10e700 1 --2- 172.21.15.39:0/4023413221 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561156ce8480 0x561156ceee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:02.162 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.130+0000 7ff3c990d700 1 --2- 172.21.15.39:0/4023413221 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561156ce8dd0 0x561156cf95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:02.162 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.130+0000 7ff3ca90f700 1 -- 172.21.15.39:0/4023413221 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561156ce8480 msgr2=0x561156ceee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:02.162 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.130+0000 7ff3ca90f700 1 --2- 172.21.15.39:0/4023413221 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561156ce8480 0x561156ceee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:02.163 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.130+0000 7ff3ca90f700 1 -- 172.21.15.39:0/4023413221 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561156ce8dd0 msgr2=0x561156cf95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:02.163 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.130+0000 7ff3ca90f700 1 --2- 172.21.15.39:0/4023413221 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561156ce8dd0 0x561156cf95a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:02.163 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.130+0000 7ff3ca90f700 1 -- 172.21.15.39:0/4023413221 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3c419de00 con 0x561156ce9c20 2023-07-21T18:41:02.163 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.130+0000 7ff3c990d700 1 --2- 172.21.15.39:0/4023413221 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561156ce8dd0 0x561156cf95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:41:02.163 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.131+0000 7ff3ca90f700 1 --2- 172.21.15.39:0/4023413221 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561156ce9c20 0x561156cef4f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc022800 tx=0x7ff3bc0224b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ade3de815ebd557e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:02.163 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.131+0000 7ff3adffb700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:41:02.163 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.131+0000 7ff3adffb700 1 -- 172.21.15.39:0/4023413221 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc00c040 con 0x561156ce9c20 2023-07-21T18:41:02.164 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.131+0000 7ff3adffb700 1 -- 172.21.15.39:0/4023413221 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff3bc004ca0 con 0x561156ce9c20 2023-07-21T18:41:02.164 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.131+0000 7ff3d84f8b80 1 -- 172.21.15.39:0/4023413221 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff3c0015940 con 0x561156ce9c20 2023-07-21T18:41:02.164 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.131+0000 7ff3adffb700 1 -- 172.21.15.39:0/4023413221 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff3bc004e60 con 0x561156ce9c20 2023-07-21T18:41:02.164 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.131+0000 7ff3d84f8b80 10 client.15096 fetch_fsmap learned FSMap version 26 2023-07-21T18:41:02.164 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.131+0000 7ff3d84f8b80 1 -- 172.21.15.39:0/4023413221 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x561156a8e200 con 0x561156ce9c20 2023-07-21T18:41:02.165 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.131+0000 7ff3adffb700 1 -- 172.21.15.39:0/4023413221 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff3c0015940 con 0x561156ce9c20 2023-07-21T18:41:02.165 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.131+0000 7ff3adffb700 1 -- 172.21.15.39:0/4023413221 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff3bc0156e0 con 0x561156ce9c20 2023-07-21T18:41:02.165 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.131+0000 7ff3d84f8b80 10 client.15096 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T18:41:02.165 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.131+0000 7ff3d84f8b80 10 client.15096 resolve_mds: resolved name 'b' to daemon mds.b 2023-07-21T18:41:02.165 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.131+0000 7ff3d84f8b80 20 client.15096 populate_metadata read hostname 'smithi039' 2023-07-21T18:41:02.165 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.131+0000 7ff3d84f8b80 1 --2- 172.21.15.39:0/4023413221 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x561156cfd480 0x561156d52d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:02.166 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.131+0000 7ff3d84f8b80 4 client.15096 mds_command: new command op to 24536 tid=0[{"prefix": "status"}] 2023-07-21T18:41:02.166 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.131+0000 7ff3d84f8b80 1 -- 172.21.15.39:0/4023413221 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "status"}) v1 -- 0x561156bf5000 con 0x561156cfd480 2023-07-21T18:41:02.166 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.132+0000 7ff3ca10e700 1 --2- 172.21.15.39:0/4023413221 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x561156cfd480 0x561156d52d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:02.166 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.132+0000 7ff3adffb700 10 client.15096 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:41:02.166 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.132+0000 7ff3ca10e700 1 --2- 172.21.15.39:0/4023413221 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x561156cfd480 0x561156d52d80 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa93b7c265be61e2 server_cookie=4be5aa34767d6520 in_seq=0 out_seq=0 2023-07-21T18:41:02.166 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.132+0000 7ff3adffb700 1 -- 172.21.15.39:0/4023413221 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x561156bf5000 con 0x561156cfd480 2023-07-21T18:41:02.166 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.132+0000 7ff3adffb700 10 client.15096 handle_command_reply: tid=0 2023-07-21T18:41:02.166 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.132+0000 7ff3d84f8b80 1 client.15096 shutdown 2023-07-21T18:41:02.167 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.132+0000 7ff3d84f8b80 1 -- 172.21.15.39:0/4023413221 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561156ce9c20 msgr2=0x561156cef4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:02.167 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.132+0000 7ff3d84f8b80 1 --2- 172.21.15.39:0/4023413221 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561156ce9c20 0x561156cef4f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc022800 tx=0x7ff3bc0224b0 comp rx=0 tx=0).stop 2023-07-21T18:41:02.167 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.132+0000 7ff3d84f8b80 1 -- 172.21.15.39:0/4023413221 shutdown_connections 2023-07-21T18:41:02.167 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.132+0000 7ff3d84f8b80 1 --2- 172.21.15.39:0/4023413221 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x561156cfd480 0x561156d52d80 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:02.167 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.132+0000 7ff3d84f8b80 1 --2- 172.21.15.39:0/4023413221 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561156ce9c20 0x561156cef4f0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:02.167 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.132+0000 7ff3d84f8b80 1 --2- 172.21.15.39:0/4023413221 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561156ce8480 0x561156ceee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:02.167 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.132+0000 7ff3d84f8b80 1 --2- 172.21.15.39:0/4023413221 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561156ce8dd0 0x561156cf95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:02.167 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.132+0000 7ff3adffb700 0 client.15096 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:41:02.168 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.132+0000 7ff3d84f8b80 1 -- 172.21.15.39:0/4023413221 >> 172.21.15.39:0/4023413221 conn(0x561156cddd50 msgr2=0x561156ceffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:02.168 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.132+0000 7ff3d84f8b80 1 -- 172.21.15.39:0/4023413221 shutdown_connections 2023-07-21T18:41:02.168 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.132+0000 7ff3d84f8b80 1 -- 172.21.15.39:0/4023413221 wait complete. 2023-07-21T18:41:02.168 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.133+0000 7ff3d84f8b80 20 client.15096 trim_cache size 0 max 16384 2023-07-21T18:41:02.169 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2023-07-21T18:41:02.169 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "eaf8e25a-27f4-11ee-9b34-001a4aab830c", 2023-07-21T18:41:02.169 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 0, 2023-07-21T18:41:02.169 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 24536, 2023-07-21T18:41:02.169 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2023-07-21T18:41:02.169 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2023-07-21T18:41:02.169 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2023-07-21T18:41:02.170 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 56.953694767999998, 2023-07-21T18:41:02.170 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 26, 2023-07-21T18:41:02.170 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 79, 2023-07-21T18:41:02.170 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 79, 2023-07-21T18:41:02.170 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 63.825077137999997 2023-07-21T18:41:02.170 INFO:teuthology.orchestra.run.smithi039.stdout:} 2023-07-21T18:41:02.171 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.133+0000 7ff3cc372700 1 -- 172.21.15.39:0/3347440548 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7ff3b00a11e0 msgr2=0x7ff3b00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:02.171 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.133+0000 7ff3cc372700 1 --2- 172.21.15.39:0/3347440548 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7ff3b00a11e0 0x7ff3b00a3690 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0009a10 tx=0x7ff3c0009190 comp rx=0 tx=0).stop 2023-07-21T18:41:02.172 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.133+0000 7ff3cc372700 1 -- 172.21.15.39:0/3347440548 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3c40fe9e0 msgr2=0x7ff3c41958e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:02.172 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.133+0000 7ff3cc372700 1 --2- 172.21.15.39:0/3347440548 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3c40fe9e0 0x7ff3c41958e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff3b40059d0 tx=0x7ff3b4005740 comp rx=0 tx=0).stop 2023-07-21T18:41:02.172 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.133+0000 7ff3cc372700 1 -- 172.21.15.39:0/3347440548 shutdown_connections 2023-07-21T18:41:02.172 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.133+0000 7ff3cc372700 1 --2- 172.21.15.39:0/3347440548 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7ff3b00a11e0 0x7ff3b00a3690 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:02.172 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.133+0000 7ff3cc372700 1 --2- 172.21.15.39:0/3347440548 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff3c4100560 0x7ff3c419ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:02.172 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.133+0000 7ff3cc372700 1 --2- 172.21.15.39:0/3347440548 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3c40fe9e0 0x7ff3c41958e0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:02.172 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.133+0000 7ff3cc372700 1 --2- 172.21.15.39:0/3347440548 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3c40ffbd0 0x7ff3c4195e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:02.173 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.133+0000 7ff3cc372700 1 -- 172.21.15.39:0/3347440548 >> 172.21.15.39:0/3347440548 conn(0x7ff3c40fa000 msgr2=0x7ff3c40fc310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:02.173 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.133+0000 7ff3cc372700 1 -- 172.21.15.39:0/3347440548 shutdown_connections 2023-07-21T18:41:02.173 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:02.133+0000 7ff3cc372700 1 -- 172.21.15.39:0/3347440548 wait complete. 2023-07-21T18:41:02.173 INFO:tasks.cephadm:Setup complete, yielding 2023-07-21T18:41:02.174 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-07-21T18:41:02.184 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi039.front.sepia.ceph.com 2023-07-21T18:41:02.185 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- bash -c 'ceph orch status' 2023-07-21T18:41:02.371 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:02 smithi193 ceph-mon[98204]: pgmap v276: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-07-21T18:41:02.372 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:02 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/1695775236' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:41:02.390 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:02 smithi155 ceph-mon[97934]: pgmap v276: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-07-21T18:41:02.390 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:02 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/1695775236' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:41:02.439 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:02 smithi039 ceph-mon[96489]: pgmap v276: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-07-21T18:41:02.439 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:02 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/1695775236' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:41:02.745 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:41:03.621 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:03 smithi193 ceph-mon[98204]: pgmap v277: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-07-21T18:41:03.640 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:03 smithi155 ceph-mon[97934]: pgmap v277: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-07-21T18:41:03.689 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:03 smithi039 ceph-mon[96489]: pgmap v277: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-07-21T18:41:04.029 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:03.941+0000 7f792d556700 1 -- 172.21.15.39:0/358014433 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f79280ffc80 msgr2=0x7f79281000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:04.029 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:03.941+0000 7f792d556700 1 --2- 172.21.15.39:0/358014433 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f79280ffc80 0x7f79281000d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f791c009990 tx=0x7f791c009ca0 comp rx=0 tx=0).stop 2023-07-21T18:41:04.029 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:03.941+0000 7f792d556700 1 -- 172.21.15.39:0/358014433 shutdown_connections 2023-07-21T18:41:04.029 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:03.941+0000 7f792d556700 1 --2- 172.21.15.39:0/358014433 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7928100610 0x7f7928106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:04.030 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:03.941+0000 7f792d556700 1 --2- 172.21.15.39:0/358014433 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79280fea80 0x7f79280fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:04.030 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:03.941+0000 7f792d556700 1 --2- 172.21.15.39:0/358014433 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f79280ffc80 0x7f79281000d0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:04.030 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:03.941+0000 7f792d556700 1 -- 172.21.15.39:0/358014433 >> 172.21.15.39:0/358014433 conn(0x7f79280fa0b0 msgr2=0x7f79280fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:04.030 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:03.941+0000 7f792d556700 1 -- 172.21.15.39:0/358014433 shutdown_connections 2023-07-21T18:41:04.030 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:03.941+0000 7f792d556700 1 -- 172.21.15.39:0/358014433 wait complete. 2023-07-21T18:41:04.030 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:03.942+0000 7f792d556700 1 Processor -- start 2023-07-21T18:41:04.030 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:03.942+0000 7f792d556700 1 -- start start 2023-07-21T18:41:04.030 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:03.942+0000 7f792d556700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79280fea80 0x7f79281947c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:04.031 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:03.942+0000 7f792d556700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f79280ffc80 0x7f7928194d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:04.031 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:03.942+0000 7f792d556700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7928100610 0x7f792819bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:04.031 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:03.942+0000 7f792d556700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f792819c350 con 0x7f79280ffc80 2023-07-21T18:41:04.031 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:03.942+0000 7f792d556700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f792819c490 con 0x7f79280fea80 2023-07-21T18:41:04.031 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:03.942+0000 7f792d556700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f792819c620 con 0x7f7928100610 2023-07-21T18:41:04.031 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:03.943+0000 7f79267fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f79280ffc80 0x7f7928194d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:04.031 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:03.943+0000 7f79267fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f79280ffc80 0x7f7928194d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56090/0 (socket says 172.21.15.39:56090) 2023-07-21T18:41:04.032 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:03.943+0000 7f79267fc700 1 -- 172.21.15.39:0/3143497750 learned_addr learned my addr 172.21.15.39:0/3143497750 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:04.032 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:03.943+0000 7f7926ffd700 1 --2- 172.21.15.39:0/3143497750 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79280fea80 0x7f79281947c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:04.032 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:03.943+0000 7f79267fc700 1 -- 172.21.15.39:0/3143497750 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7928100610 msgr2=0x7f792819bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:04.032 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:03.943+0000 7f79267fc700 1 --2- 172.21.15.39:0/3143497750 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7928100610 0x7f792819bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:04.032 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:03.943+0000 7f79267fc700 1 -- 172.21.15.39:0/3143497750 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79280fea80 msgr2=0x7f79281947c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:04.032 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:03.943+0000 7f79267fc700 1 --2- 172.21.15.39:0/3143497750 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79280fea80 0x7f79281947c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:04.032 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:03.943+0000 7f79267fc700 1 -- 172.21.15.39:0/3143497750 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f791c0095f0 con 0x7f79280ffc80 2023-07-21T18:41:04.032 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:03.943+0000 7f79267fc700 1 --2- 172.21.15.39:0/3143497750 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f79280ffc80 0x7f7928194d00 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f791c00c010 tx=0x7f791c01a8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:04.033 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:03.943+0000 7f790ffff700 1 -- 172.21.15.39:0/3143497750 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f791c01f070 con 0x7f79280ffc80 2023-07-21T18:41:04.033 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:03.943+0000 7f790ffff700 1 -- 172.21.15.39:0/3143497750 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f791c01abc0 con 0x7f79280ffc80 2023-07-21T18:41:04.033 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:03.943+0000 7f792d556700 1 -- 172.21.15.39:0/3143497750 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7928067fd0 con 0x7f79280ffc80 2023-07-21T18:41:04.033 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:03.943+0000 7f790ffff700 1 -- 172.21.15.39:0/3143497750 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f791c025680 con 0x7f79280ffc80 2023-07-21T18:41:04.033 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:03.943+0000 7f792d556700 1 -- 172.21.15.39:0/3143497750 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7928068470 con 0x7f79280ffc80 2023-07-21T18:41:04.033 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:03.944+0000 7f792d556700 1 -- 172.21.15.39:0/3143497750 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7928068e60 con 0x7f79280ffc80 2023-07-21T18:41:04.033 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:03.945+0000 7f790ffff700 1 -- 172.21.15.39:0/3143497750 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f791c023070 con 0x7f79280ffc80 2023-07-21T18:41:04.033 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:03.946+0000 7f790ffff700 1 --2- 172.21.15.39:0/3143497750 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f79100a11e0 0x7f79100a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:04.034 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:03.946+0000 7f7926ffd700 1 --2- 172.21.15.39:0/3143497750 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f79100a11e0 0x7f79100a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:04.034 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:03.946+0000 7f790ffff700 1 -- 172.21.15.39:0/3143497750 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f791c0cffa0 con 0x7f79280ffc80 2023-07-21T18:41:04.034 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:03.946+0000 7f7926ffd700 1 --2- 172.21.15.39:0/3143497750 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f79100a11e0 0x7f79100a3690 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f79280ffae0 tx=0x7f791800b5b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:04.034 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:03.949+0000 7f790ffff700 1 -- 172.21.15.39:0/3143497750 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f791c0996d0 con 0x7f79280ffc80 2023-07-21T18:41:04.188 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:04.165+0000 7f792d556700 1 -- 172.21.15.39:0/3143497750 --> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7928068690 con 0x7f79100a11e0 2023-07-21T18:41:04.189 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:04.166+0000 7f790ffff700 1 -- 172.21.15.39:0/3143497750 <== mgr.14152 v2:172.21.15.39:6800/2116291712 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f7928068690 con 0x7f79100a11e0 2023-07-21T18:41:04.189 INFO:teuthology.orchestra.run.smithi039.stdout:Backend: cephadm 2023-07-21T18:41:04.190 INFO:teuthology.orchestra.run.smithi039.stdout:Available: Yes 2023-07-21T18:41:04.190 INFO:teuthology.orchestra.run.smithi039.stdout:Paused: No 2023-07-21T18:41:04.192 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:04.168+0000 7f792d556700 1 -- 172.21.15.39:0/3143497750 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f79100a11e0 msgr2=0x7f79100a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:04.192 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:04.168+0000 7f792d556700 1 --2- 172.21.15.39:0/3143497750 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f79100a11e0 0x7f79100a3690 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f79280ffae0 tx=0x7f791800b5b0 comp rx=0 tx=0).stop 2023-07-21T18:41:04.192 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:04.168+0000 7f792d556700 1 -- 172.21.15.39:0/3143497750 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f79280ffc80 msgr2=0x7f7928194d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:04.192 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:04.168+0000 7f792d556700 1 --2- 172.21.15.39:0/3143497750 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f79280ffc80 0x7f7928194d00 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f791c00c010 tx=0x7f791c01a8f0 comp rx=0 tx=0).stop 2023-07-21T18:41:04.192 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:04.168+0000 7f792d556700 1 -- 172.21.15.39:0/3143497750 shutdown_connections 2023-07-21T18:41:04.192 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:04.168+0000 7f792d556700 1 --2- 172.21.15.39:0/3143497750 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f79100a11e0 0x7f79100a3690 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:04.193 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:04.168+0000 7f792d556700 1 --2- 172.21.15.39:0/3143497750 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79280fea80 0x7f79281947c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:04.193 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:04.168+0000 7f792d556700 1 --2- 172.21.15.39:0/3143497750 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7928100610 0x7f792819bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:04.193 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:04.168+0000 7f792d556700 1 --2- 172.21.15.39:0/3143497750 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f79280ffc80 0x7f7928194d00 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:04.193 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:04.168+0000 7f792d556700 1 -- 172.21.15.39:0/3143497750 >> 172.21.15.39:0/3143497750 conn(0x7f79280fa0b0 msgr2=0x7f79281031c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:04.193 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:04.168+0000 7f792d556700 1 -- 172.21.15.39:0/3143497750 shutdown_connections 2023-07-21T18:41:04.193 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:04.168+0000 7f792d556700 1 -- 172.21.15.39:0/3143497750 wait complete. 2023-07-21T18:41:04.594 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- bash -c 'ceph orch ps' 2023-07-21T18:41:05.156 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:41:05.579 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:05 smithi039 ceph-mon[96489]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:41:05.579 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:05 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:41:05.579 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:05 smithi039 ceph-mon[96489]: pgmap v278: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-07-21T18:41:05.621 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:05 smithi193 ceph-mon[98204]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:41:05.622 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:05 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:41:05.622 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:05 smithi193 ceph-mon[98204]: pgmap v278: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-07-21T18:41:05.640 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:05 smithi155 ceph-mon[97934]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:41:05.640 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:05 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:41:05.640 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:05 smithi155 ceph-mon[97934]: pgmap v278: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-07-21T18:41:06.550 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:06.525+0000 7f1baafd6700 1 -- 172.21.15.39:0/3140390872 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ba40ffc20 msgr2=0x7f1ba4100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:06.551 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:06.525+0000 7f1baafd6700 1 --2- 172.21.15.39:0/3140390872 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ba40ffc20 0x7f1ba4100030 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1ba000b0a0 tx=0x7f1ba000b3b0 comp rx=0 tx=0).stop 2023-07-21T18:41:06.551 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:06.525+0000 7f1baafd6700 1 -- 172.21.15.39:0/3140390872 shutdown_connections 2023-07-21T18:41:06.551 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:06.525+0000 7f1baafd6700 1 --2- 172.21.15.39:0/3140390872 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ba41017b0 0x7f1ba4108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:06.551 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:06.525+0000 7f1baafd6700 1 --2- 172.21.15.39:0/3140390872 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ba40ffc20 0x7f1ba4100030 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:06.551 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:06.525+0000 7f1baafd6700 1 --2- 172.21.15.39:0/3140390872 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ba4100e20 0x7f1ba4101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:06.551 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:06.525+0000 7f1baafd6700 1 -- 172.21.15.39:0/3140390872 >> 172.21.15.39:0/3140390872 conn(0x7f1ba40fb250 msgr2=0x7f1ba40fd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:06.551 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:06.525+0000 7f1baafd6700 1 -- 172.21.15.39:0/3140390872 shutdown_connections 2023-07-21T18:41:06.552 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:06.525+0000 7f1baafd6700 1 -- 172.21.15.39:0/3140390872 wait complete. 2023-07-21T18:41:06.552 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:06.526+0000 7f1baafd6700 1 Processor -- start 2023-07-21T18:41:06.552 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:06.526+0000 7f1baafd6700 1 -- start start 2023-07-21T18:41:06.552 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:06.526+0000 7f1baafd6700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ba40ffc20 0x7f1ba4195a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:06.552 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:06.526+0000 7f1baafd6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ba4100e20 0x7f1ba4195f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:06.552 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:06.526+0000 7f1baafd6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ba41017b0 0x7f1ba419cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:06.553 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:06.526+0000 7f1baafd6700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba419d5a0 con 0x7f1ba4100e20 2023-07-21T18:41:06.553 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:06.526+0000 7f1baafd6700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba419d6e0 con 0x7f1ba41017b0 2023-07-21T18:41:06.553 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:06.526+0000 7f1baafd6700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba419d870 con 0x7f1ba40ffc20 2023-07-21T18:41:06.553 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:06.526+0000 7f1b9bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ba4100e20 0x7f1ba4195f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:06.553 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:06.526+0000 7f1b9bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ba4100e20 0x7f1ba4195f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56104/0 (socket says 172.21.15.39:56104) 2023-07-21T18:41:06.554 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:06.526+0000 7f1b9bfff700 1 -- 172.21.15.39:0/3459722377 learned_addr learned my addr 172.21.15.39:0/3459722377 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:06.554 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:06.526+0000 7f1ba8d72700 1 --2- 172.21.15.39:0/3459722377 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ba40ffc20 0x7f1ba4195a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:06.554 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:06.526+0000 7f1ba9573700 1 --2- 172.21.15.39:0/3459722377 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ba41017b0 0x7f1ba419cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:06.554 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:06.527+0000 7f1b9bfff700 1 -- 172.21.15.39:0/3459722377 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ba40ffc20 msgr2=0x7f1ba4195a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:06.554 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:06.527+0000 7f1b9bfff700 1 --2- 172.21.15.39:0/3459722377 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ba40ffc20 0x7f1ba4195a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:06.554 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:06.527+0000 7f1b9bfff700 1 -- 172.21.15.39:0/3459722377 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ba41017b0 msgr2=0x7f1ba419cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:06.554 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:06.527+0000 7f1b9bfff700 1 --2- 172.21.15.39:0/3459722377 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ba41017b0 0x7f1ba419cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:06.555 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:06.527+0000 7f1b9bfff700 1 -- 172.21.15.39:0/3459722377 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ba0009d00 con 0x7f1ba4100e20 2023-07-21T18:41:06.555 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:06.527+0000 7f1ba8d72700 1 --2- 172.21.15.39:0/3459722377 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ba40ffc20 0x7f1ba4195a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:41:06.555 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:06.527+0000 7f1ba9573700 1 --2- 172.21.15.39:0/3459722377 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ba41017b0 0x7f1ba419cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:41:06.555 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:06.527+0000 7f1b9bfff700 1 --2- 172.21.15.39:0/3459722377 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ba4100e20 0x7f1ba4195f50 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1b9400d980 tx=0x7f1b9400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:06.555 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:06.527+0000 7f1b99ffb700 1 -- 172.21.15.39:0/3459722377 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b940098d0 con 0x7f1ba4100e20 2023-07-21T18:41:06.555 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:06.527+0000 7f1baafd6700 1 -- 172.21.15.39:0/3459722377 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ba419db50 con 0x7f1ba4100e20 2023-07-21T18:41:06.555 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:06.527+0000 7f1b99ffb700 1 -- 172.21.15.39:0/3459722377 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b94010460 con 0x7f1ba4100e20 2023-07-21T18:41:06.556 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:06.527+0000 7f1b99ffb700 1 -- 172.21.15.39:0/3459722377 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b9400f620 con 0x7f1ba4100e20 2023-07-21T18:41:06.556 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:06.527+0000 7f1baafd6700 1 -- 172.21.15.39:0/3459722377 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ba419e0a0 con 0x7f1ba4100e20 2023-07-21T18:41:06.557 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:06.528+0000 7f1baafd6700 1 -- 172.21.15.39:0/3459722377 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ba410b3e0 con 0x7f1ba4100e20 2023-07-21T18:41:06.557 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:06.529+0000 7f1b99ffb700 1 -- 172.21.15.39:0/3459722377 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f1b94010860 con 0x7f1ba4100e20 2023-07-21T18:41:06.557 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:06.530+0000 7f1b99ffb700 1 --2- 172.21.15.39:0/3459722377 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f1b8c0a11e0 0x7f1b8c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:06.557 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:06.530+0000 7f1ba8d72700 1 --2- 172.21.15.39:0/3459722377 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f1b8c0a11e0 0x7f1b8c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:06.557 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:06.530+0000 7f1b99ffb700 1 -- 172.21.15.39:0/3459722377 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f1b940cc760 con 0x7f1ba4100e20 2023-07-21T18:41:06.557 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:06.530+0000 7f1ba8d72700 1 --2- 172.21.15.39:0/3459722377 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f1b8c0a11e0 0x7f1b8c0a3690 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1ba0007a60 tx=0x7f1ba000b700 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:06.558 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:06.534+0000 7f1b99ffb700 1 -- 172.21.15.39:0/3459722377 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f1b94095e00 con 0x7f1ba4100e20 2023-07-21T18:41:06.772 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:06.749+0000 7f1baafd6700 1 -- 172.21.15.39:0/3459722377 --> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1ba4060950 con 0x7f1b8c0a11e0 2023-07-21T18:41:06.793 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:06.770+0000 7f1b99ffb700 1 -- 172.21.15.39:0/3459722377 <== mgr.14152 v2:172.21.15.39:6800/2116291712 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7f1ba4060950 con 0x7f1b8c0a11e0 2023-07-21T18:41:06.794 INFO:teuthology.orchestra.run.smithi039.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-07-21T18:41:06.794 INFO:teuthology.orchestra.run.smithi039.stdout:mds.a smithi039 running (71s) 61s ago 70s 14.3M - 18.0.0-4746-g81cf785a 66a4f7b015b5 a5de16bb3f5d 2023-07-21T18:41:06.794 INFO:teuthology.orchestra.run.smithi039.stdout:mds.b smithi155 running (68s) 61s ago 68s 13.2M - 18.0.0-4746-g81cf785a 66a4f7b015b5 009e64f168b0 2023-07-21T18:41:06.794 INFO:teuthology.orchestra.run.smithi039.stdout:mds.c smithi193 running (80s) 61s ago 79s 15.7M - 18.0.0-4746-g81cf785a 66a4f7b015b5 24117b151724 2023-07-21T18:41:06.794 INFO:teuthology.orchestra.run.smithi039.stdout:mds.d smithi039 running (86s) 61s ago 86s 14.4M - 18.0.0-4746-g81cf785a 66a4f7b015b5 6a19ff935597 2023-07-21T18:41:06.795 INFO:teuthology.orchestra.run.smithi039.stdout:mds.e smithi155 running (84s) 61s ago 83s 15.6M - 18.0.0-4746-g81cf785a 66a4f7b015b5 665fc908fce1 2023-07-21T18:41:06.795 INFO:teuthology.orchestra.run.smithi039.stdout:mds.f smithi193 running (82s) 61s ago 81s 15.7M - 18.0.0-4746-g81cf785a 66a4f7b015b5 d7af23d66669 2023-07-21T18:41:06.795 INFO:teuthology.orchestra.run.smithi039.stdout:mds.g smithi039 running (73s) 61s ago 72s 14.0M - 18.0.0-4746-g81cf785a 66a4f7b015b5 032128f34a00 2023-07-21T18:41:06.795 INFO:teuthology.orchestra.run.smithi039.stdout:mds.h smithi155 running (77s) 61s ago 77s 16.8M - 18.0.0-4746-g81cf785a 66a4f7b015b5 a4a7b8902523 2023-07-21T18:41:06.795 INFO:teuthology.orchestra.run.smithi039.stdout:mds.i smithi193 running (75s) 61s ago 75s 15.6M - 18.0.0-4746-g81cf785a 66a4f7b015b5 226e6661067a 2023-07-21T18:41:06.795 INFO:teuthology.orchestra.run.smithi039.stdout:mds.j smithi039 running (88s) 61s ago 88s 14.2M - 18.0.0-4746-g81cf785a 66a4f7b015b5 6c7a67820dde 2023-07-21T18:41:06.795 INFO:teuthology.orchestra.run.smithi039.stdout:mds.k smithi155 running (90s) 61s ago 90s 14.0M - 18.0.0-4746-g81cf785a 66a4f7b015b5 5d4b3ec0067a 2023-07-21T18:41:06.796 INFO:teuthology.orchestra.run.smithi039.stdout:mds.l smithi193 running (92s) 61s ago 92s 14.2M - 18.0.0-4746-g81cf785a 66a4f7b015b5 7c8836bfc045 2023-07-21T18:41:06.796 INFO:teuthology.orchestra.run.smithi039.stdout:mgr.x smithi039 *:9283,8765 running (7m) 61s ago 7m 510M - 18.0.0-4746-g81cf785a 66a4f7b015b5 9506db3d7e28 2023-07-21T18:41:06.796 INFO:teuthology.orchestra.run.smithi039.stdout:mgr.y smithi155 *:8443,8765 running (5m) 61s ago 5m 445M - 18.0.0-4746-g81cf785a 66a4f7b015b5 50bbf712a793 2023-07-21T18:41:06.796 INFO:teuthology.orchestra.run.smithi039.stdout:mgr.z smithi193 *:8443,8765 running (6m) 61s ago 6m 446M - 18.0.0-4746-g81cf785a 66a4f7b015b5 1d812b344ad6 2023-07-21T18:41:06.796 INFO:teuthology.orchestra.run.smithi039.stdout:mon.a smithi039 running (7m) 61s ago 8m 83.5M 2048M 18.0.0-4746-g81cf785a 66a4f7b015b5 a136b64d7de7 2023-07-21T18:41:06.796 INFO:teuthology.orchestra.run.smithi039.stdout:mon.b smithi155 running (6m) 61s ago 6m 59.7M 2048M 18.0.0-4746-g81cf785a 66a4f7b015b5 b4501ab044c1 2023-07-21T18:41:06.796 INFO:teuthology.orchestra.run.smithi039.stdout:mon.c smithi193 running (6m) 61s ago 6m 58.5M 2048M 18.0.0-4746-g81cf785a 66a4f7b015b5 5a7b2b6ffbc0 2023-07-21T18:41:06.797 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0 smithi039 running (5m) 61s ago 5m 55.3M 4292M 18.0.0-4746-g81cf785a 66a4f7b015b5 72da1d988aca 2023-07-21T18:41:06.797 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1 smithi155 running (5m) 61s ago 5m 58.5M 4292M 18.0.0-4746-g81cf785a 66a4f7b015b5 179c0201da43 2023-07-21T18:41:06.797 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2 smithi193 running (5m) 61s ago 5m 53.5M 4292M 18.0.0-4746-g81cf785a 66a4f7b015b5 edbb63e29628 2023-07-21T18:41:06.797 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3 smithi039 running (4m) 61s ago 4m 51.7M 4292M 18.0.0-4746-g81cf785a 66a4f7b015b5 f5c39898f963 2023-07-21T18:41:06.797 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4 smithi155 running (4m) 61s ago 4m 51.1M 4292M 18.0.0-4746-g81cf785a 66a4f7b015b5 d7cf29f19023 2023-07-21T18:41:06.797 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5 smithi193 running (3m) 61s ago 3m 52.5M 4292M 18.0.0-4746-g81cf785a 66a4f7b015b5 805cdcaea7d5 2023-07-21T18:41:06.797 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6 smithi039 running (3m) 61s ago 3m 47.4M 4292M 18.0.0-4746-g81cf785a 66a4f7b015b5 b072ea5e0409 2023-07-21T18:41:06.798 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7 smithi155 running (3m) 61s ago 3m 54.7M 4292M 18.0.0-4746-g81cf785a 66a4f7b015b5 eddd3cc20586 2023-07-21T18:41:06.798 INFO:teuthology.orchestra.run.smithi039.stdout:osd.8 smithi193 running (2m) 61s ago 2m 50.8M 4292M 18.0.0-4746-g81cf785a 66a4f7b015b5 e3a12aa3460a 2023-07-21T18:41:06.798 INFO:teuthology.orchestra.run.smithi039.stdout:osd.9 smithi039 running (2m) 61s ago 2m 46.5M 4292M 18.0.0-4746-g81cf785a 66a4f7b015b5 e32421ee31e3 2023-07-21T18:41:06.798 INFO:teuthology.orchestra.run.smithi039.stdout:osd.10 smithi155 running (2m) 61s ago 2m 49.2M 4292M 18.0.0-4746-g81cf785a 66a4f7b015b5 3175499b85cb 2023-07-21T18:41:06.798 INFO:teuthology.orchestra.run.smithi039.stdout:osd.11 smithi193 running (110s) 61s ago 109s 46.5M 4292M 18.0.0-4746-g81cf785a 66a4f7b015b5 e9efdc0ffa02 2023-07-21T18:41:06.799 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:06.772+0000 7f1baafd6700 1 -- 172.21.15.39:0/3459722377 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f1b8c0a11e0 msgr2=0x7f1b8c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:06.799 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:06.772+0000 7f1baafd6700 1 --2- 172.21.15.39:0/3459722377 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f1b8c0a11e0 0x7f1b8c0a3690 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1ba0007a60 tx=0x7f1ba000b700 comp rx=0 tx=0).stop 2023-07-21T18:41:06.799 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:06.772+0000 7f1baafd6700 1 -- 172.21.15.39:0/3459722377 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ba4100e20 msgr2=0x7f1ba4195f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:06.799 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:06.772+0000 7f1baafd6700 1 --2- 172.21.15.39:0/3459722377 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ba4100e20 0x7f1ba4195f50 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1b9400d980 tx=0x7f1b9400dc90 comp rx=0 tx=0).stop 2023-07-21T18:41:06.800 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:06.772+0000 7f1baafd6700 1 -- 172.21.15.39:0/3459722377 shutdown_connections 2023-07-21T18:41:06.800 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:06.772+0000 7f1baafd6700 1 --2- 172.21.15.39:0/3459722377 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f1b8c0a11e0 0x7f1b8c0a3690 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:06.800 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:06.772+0000 7f1baafd6700 1 --2- 172.21.15.39:0/3459722377 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ba41017b0 0x7f1ba419cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:06.800 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:06.772+0000 7f1baafd6700 1 --2- 172.21.15.39:0/3459722377 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ba40ffc20 0x7f1ba4195a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:06.800 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:06.772+0000 7f1baafd6700 1 --2- 172.21.15.39:0/3459722377 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ba4100e20 0x7f1ba4195f50 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:06.800 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:06.772+0000 7f1baafd6700 1 -- 172.21.15.39:0/3459722377 >> 172.21.15.39:0/3459722377 conn(0x7f1ba40fb250 msgr2=0x7f1ba40fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:06.800 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:06.773+0000 7f1baafd6700 1 -- 172.21.15.39:0/3459722377 shutdown_connections 2023-07-21T18:41:06.801 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:06.773+0000 7f1baafd6700 1 -- 172.21.15.39:0/3459722377 wait complete. 2023-07-21T18:41:07.172 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:07 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2023-07-21T18:41:07.172 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:07 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2023-07-21T18:41:07.172 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:07 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]: dispatch 2023-07-21T18:41:07.172 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:07 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]: dispatch 2023-07-21T18:41:07.173 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:07 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2023-07-21T18:41:07.173 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:07 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2023-07-21T18:41:07.173 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:07 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2023-07-21T18:41:07.274 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- bash -c 'ceph orch ls' 2023-07-21T18:41:07.621 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:07 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2023-07-21T18:41:07.622 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:07 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2023-07-21T18:41:07.622 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:07 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]: dispatch 2023-07-21T18:41:07.622 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:07 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]: dispatch 2023-07-21T18:41:07.622 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:07 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2023-07-21T18:41:07.622 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:07 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2023-07-21T18:41:07.622 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:07 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2023-07-21T18:41:07.640 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:07 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2023-07-21T18:41:07.640 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:07 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2023-07-21T18:41:07.641 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:07 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]: dispatch 2023-07-21T18:41:07.641 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:07 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]: dispatch 2023-07-21T18:41:07.641 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:07 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2023-07-21T18:41:07.641 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:07 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2023-07-21T18:41:07.641 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:07 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2023-07-21T18:41:07.845 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:41:08.346 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:08 smithi039 ceph-mon[96489]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:41:08.346 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:08 smithi039 ceph-mon[96489]: pgmap v279: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-07-21T18:41:08.346 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:08 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2023-07-21T18:41:08.346 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:08 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2023-07-21T18:41:08.347 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:08 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]': finished 2023-07-21T18:41:08.347 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:08 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]': finished 2023-07-21T18:41:08.347 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:08 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2023-07-21T18:41:08.347 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:08 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2023-07-21T18:41:08.347 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:08 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2023-07-21T18:41:08.347 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:08 smithi039 ceph-mon[96489]: osdmap e80: 12 total, 12 up, 12 in 2023-07-21T18:41:08.348 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:08 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:41:08.348 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:08 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:41:08.348 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:08 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:41:08.348 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:08 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:41:08.348 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:08 smithi039 ceph-mon[96489]: osdmap e81: 12 total, 12 up, 12 in 2023-07-21T18:41:08.621 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:08 smithi193 ceph-mon[98204]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:41:08.621 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:08 smithi193 ceph-mon[98204]: pgmap v279: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-07-21T18:41:08.622 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:08 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2023-07-21T18:41:08.622 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:08 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2023-07-21T18:41:08.622 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:08 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]': finished 2023-07-21T18:41:08.622 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:08 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]': finished 2023-07-21T18:41:08.622 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:08 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2023-07-21T18:41:08.622 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:08 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2023-07-21T18:41:08.622 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:08 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2023-07-21T18:41:08.623 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:08 smithi193 ceph-mon[98204]: osdmap e80: 12 total, 12 up, 12 in 2023-07-21T18:41:08.623 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:08 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:41:08.623 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:08 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:41:08.623 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:08 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:41:08.623 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:08 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:41:08.623 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:08 smithi193 ceph-mon[98204]: osdmap e81: 12 total, 12 up, 12 in 2023-07-21T18:41:08.640 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:08 smithi155 ceph-mon[97934]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:41:08.640 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:08 smithi155 ceph-mon[97934]: pgmap v279: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-07-21T18:41:08.641 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:08 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2023-07-21T18:41:08.641 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:08 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2023-07-21T18:41:08.641 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:08 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]': finished 2023-07-21T18:41:08.641 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:08 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]': finished 2023-07-21T18:41:08.641 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:08 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2023-07-21T18:41:08.641 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:08 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2023-07-21T18:41:08.641 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:08 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2023-07-21T18:41:08.642 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:08 smithi155 ceph-mon[97934]: osdmap e80: 12 total, 12 up, 12 in 2023-07-21T18:41:08.642 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:08 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:41:08.642 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:08 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:41:08.642 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:08 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:41:08.642 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:08 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:41:08.642 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:08 smithi155 ceph-mon[97934]: osdmap e81: 12 total, 12 up, 12 in 2023-07-21T18:41:09.139 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:09.116+0000 7f1de86bd700 1 -- 172.21.15.39:0/1095382365 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1de0101720 msgr2=0x7f1de0107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:09.140 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:09.116+0000 7f1de86bd700 1 --2- 172.21.15.39:0/1095382365 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1de0101720 0x7f1de0107fe0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1dd800b0a0 tx=0x7f1dd800b3b0 comp rx=0 tx=0).stop 2023-07-21T18:41:09.140 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:09.116+0000 7f1de86bd700 1 -- 172.21.15.39:0/1095382365 shutdown_connections 2023-07-21T18:41:09.140 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:09.116+0000 7f1de86bd700 1 --2- 172.21.15.39:0/1095382365 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1de0101720 0x7f1de0107fe0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:09.140 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:09.116+0000 7f1de86bd700 1 --2- 172.21.15.39:0/1095382365 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1de00ffb90 0x7f1de00fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:09.140 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:09.116+0000 7f1de86bd700 1 --2- 172.21.15.39:0/1095382365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1de0100d90 0x7f1de01011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:09.140 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:09.116+0000 7f1de86bd700 1 -- 172.21.15.39:0/1095382365 >> 172.21.15.39:0/1095382365 conn(0x7f1de00fb1a0 msgr2=0x7f1de00fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:09.140 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:09.116+0000 7f1de86bd700 1 -- 172.21.15.39:0/1095382365 shutdown_connections 2023-07-21T18:41:09.141 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:09.116+0000 7f1de86bd700 1 -- 172.21.15.39:0/1095382365 wait complete. 2023-07-21T18:41:09.142 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:09.116+0000 7f1de86bd700 1 Processor -- start 2023-07-21T18:41:09.142 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:09.116+0000 7f1de86bd700 1 -- start start 2023-07-21T18:41:09.142 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:09.117+0000 7f1de86bd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1de00ffb90 0x7f1de00742b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:09.143 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:09.117+0000 7f1de86bd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1de0100d90 0x7f1de0072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:09.143 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:09.117+0000 7f1de86bd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1de0101720 0x7f1de0072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:09.143 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:09.117+0000 7f1de86bd700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de00735c0 con 0x7f1de0100d90 2023-07-21T18:41:09.143 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:09.117+0000 7f1de86bd700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de0073730 con 0x7f1de0101720 2023-07-21T18:41:09.143 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:09.117+0000 7f1de86bd700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de019f9e0 con 0x7f1de00ffb90 2023-07-21T18:41:09.143 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:09.117+0000 7f1de5c58700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1de0100d90 0x7f1de0072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:09.143 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:09.117+0000 7f1de5c58700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1de0100d90 0x7f1de0072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56120/0 (socket says 172.21.15.39:56120) 2023-07-21T18:41:09.143 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:09.117+0000 7f1de5c58700 1 -- 172.21.15.39:0/2789475080 learned_addr learned my addr 172.21.15.39:0/2789475080 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:09.144 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:09.117+0000 7f1de6459700 1 --2- 172.21.15.39:0/2789475080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1de00ffb90 0x7f1de00742b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:09.144 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:09.117+0000 7f1de6c5a700 1 --2- 172.21.15.39:0/2789475080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1de0101720 0x7f1de0072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:09.144 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:09.117+0000 7f1de5c58700 1 -- 172.21.15.39:0/2789475080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1de00ffb90 msgr2=0x7f1de00742b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:09.144 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:09.117+0000 7f1de5c58700 1 --2- 172.21.15.39:0/2789475080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1de00ffb90 0x7f1de00742b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:09.144 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:09.117+0000 7f1de5c58700 1 -- 172.21.15.39:0/2789475080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1de0101720 msgr2=0x7f1de0072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:09.144 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:09.117+0000 7f1de5c58700 1 --2- 172.21.15.39:0/2789475080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1de0101720 0x7f1de0072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:09.144 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:09.117+0000 7f1de5c58700 1 -- 172.21.15.39:0/2789475080 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dd8009d00 con 0x7f1de0100d90 2023-07-21T18:41:09.145 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:09.117+0000 7f1de5c58700 1 --2- 172.21.15.39:0/2789475080 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1de0100d90 0x7f1de0072900 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1dd000e9a0 tx=0x7f1dd000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:09.145 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:09.117+0000 7f1dd77fe700 1 -- 172.21.15.39:0/2789475080 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd0009790 con 0x7f1de0100d90 2023-07-21T18:41:09.145 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:09.117+0000 7f1dd77fe700 1 -- 172.21.15.39:0/2789475080 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1dd0010430 con 0x7f1de0100d90 2023-07-21T18:41:09.145 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:09.117+0000 7f1de86bd700 1 -- 172.21.15.39:0/2789475080 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1de019fcc0 con 0x7f1de0100d90 2023-07-21T18:41:09.145 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:09.118+0000 7f1dd77fe700 1 -- 172.21.15.39:0/2789475080 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd0018680 con 0x7f1de0100d90 2023-07-21T18:41:09.145 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:09.118+0000 7f1de86bd700 1 -- 172.21.15.39:0/2789475080 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1de01a01c0 con 0x7f1de0100d90 2023-07-21T18:41:09.145 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:09.119+0000 7f1de86bd700 1 -- 172.21.15.39:0/2789475080 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1de010b360 con 0x7f1de0100d90 2023-07-21T18:41:09.146 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:09.120+0000 7f1dd77fe700 1 -- 172.21.15.39:0/2789475080 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f1dd00105a0 con 0x7f1de0100d90 2023-07-21T18:41:09.147 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:09.121+0000 7f1dd77fe700 1 --2- 172.21.15.39:0/2789475080 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f1dcc0a1260 0x7f1dcc0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:09.147 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:09.121+0000 7f1dd77fe700 1 -- 172.21.15.39:0/2789475080 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f1dd0014070 con 0x7f1de0100d90 2023-07-21T18:41:09.147 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:09.121+0000 7f1de6459700 1 --2- 172.21.15.39:0/2789475080 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f1dcc0a1260 0x7f1dcc0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:09.147 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:09.122+0000 7f1de6459700 1 --2- 172.21.15.39:0/2789475080 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f1dcc0a1260 0x7f1dcc0a3710 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1de0100bf0 tx=0x7f1ddc007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:09.147 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:09.124+0000 7f1dd77fe700 1 -- 172.21.15.39:0/2789475080 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f1dd00963a0 con 0x7f1de0100d90 2023-07-21T18:41:09.377 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:09.354+0000 7f1de86bd700 1 -- 172.21.15.39:0/2789475080 --> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f1de0060950 con 0x7f1dcc0a1260 2023-07-21T18:41:09.384 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:09.361+0000 7f1dd77fe700 1 -- 172.21.15.39:0/2789475080 <== mgr.14152 v2:172.21.15.39:6800/2116291712 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7f1de0060950 con 0x7f1dcc0a1260 2023-07-21T18:41:09.385 INFO:teuthology.orchestra.run.smithi039.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2023-07-21T18:41:09.385 INFO:teuthology.orchestra.run.smithi039.stdout:mds.all 12/12 1s ago 96s smithi039=a;smithi039=d;smithi039=g;smithi039=j;smithi155=b;smithi155=e;smithi155=h;smithi155=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l;count:12 2023-07-21T18:41:09.385 INFO:teuthology.orchestra.run.smithi039.stdout:mgr 3/3 1s ago 6m smithi039=x;smithi155=y;smithi193=z;count:3 2023-07-21T18:41:09.385 INFO:teuthology.orchestra.run.smithi039.stdout:mon 3/3 1s ago 6m smithi039:172.21.15.39=a;smithi155:172.21.15.155=b;smithi193:172.21.15.193=c;count:3 2023-07-21T18:41:09.385 INFO:teuthology.orchestra.run.smithi039.stdout:osd 12 1s ago - 2023-07-21T18:41:09.388 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:09.365+0000 7f1de86bd700 1 -- 172.21.15.39:0/2789475080 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f1dcc0a1260 msgr2=0x7f1dcc0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:09.389 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:09.365+0000 7f1de86bd700 1 --2- 172.21.15.39:0/2789475080 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f1dcc0a1260 0x7f1dcc0a3710 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1de0100bf0 tx=0x7f1ddc007400 comp rx=0 tx=0).stop 2023-07-21T18:41:09.389 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:09.365+0000 7f1de86bd700 1 -- 172.21.15.39:0/2789475080 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1de0100d90 msgr2=0x7f1de0072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:09.389 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:09.365+0000 7f1de86bd700 1 --2- 172.21.15.39:0/2789475080 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1de0100d90 0x7f1de0072900 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1dd000e9a0 tx=0x7f1dd000ecb0 comp rx=0 tx=0).stop 2023-07-21T18:41:09.389 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:09.365+0000 7f1de86bd700 1 -- 172.21.15.39:0/2789475080 shutdown_connections 2023-07-21T18:41:09.389 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:09.365+0000 7f1de86bd700 1 --2- 172.21.15.39:0/2789475080 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f1dcc0a1260 0x7f1dcc0a3710 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:09.389 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:09.365+0000 7f1de86bd700 1 --2- 172.21.15.39:0/2789475080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1de0101720 0x7f1de0072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:09.389 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:09.365+0000 7f1de86bd700 1 --2- 172.21.15.39:0/2789475080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1de00ffb90 0x7f1de00742b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:09.390 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:09.365+0000 7f1de86bd700 1 --2- 172.21.15.39:0/2789475080 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1de0100d90 0x7f1de0072900 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:09.390 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:09.365+0000 7f1de86bd700 1 -- 172.21.15.39:0/2789475080 >> 172.21.15.39:0/2789475080 conn(0x7f1de00fb1a0 msgr2=0x7f1de00fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:09.390 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:09.365+0000 7f1de86bd700 1 -- 172.21.15.39:0/2789475080 shutdown_connections 2023-07-21T18:41:09.390 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:09.365+0000 7f1de86bd700 1 -- 172.21.15.39:0/2789475080 wait complete. 2023-07-21T18:41:09.806 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- bash -c 'ceph orch host ls' 2023-07-21T18:41:10.121 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:09 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:41:10.121 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:09 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:41:10.121 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:09 smithi193 ceph-mon[98204]: pgmap v282: 97 pgs: 5 peering, 92 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-07-21T18:41:10.122 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:09 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:41:10.122 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:09 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:41:10.122 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:09 smithi193 ceph-mon[98204]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:41:10.140 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:09 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:41:10.140 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:09 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:41:10.140 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:09 smithi155 ceph-mon[97934]: pgmap v282: 97 pgs: 5 peering, 92 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-07-21T18:41:10.140 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:09 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:41:10.140 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:09 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:41:10.141 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:09 smithi155 ceph-mon[97934]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:41:10.189 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:09 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:41:10.189 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:09 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:41:10.189 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:09 smithi039 ceph-mon[96489]: pgmap v282: 97 pgs: 5 peering, 92 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-07-21T18:41:10.189 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:09 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:41:10.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:09 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:41:10.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:09 smithi039 ceph-mon[96489]: from='mgr.14152 172.21.15.39:0/4015368534' entity='mgr.x' 2023-07-21T18:41:10.367 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:41:10.879 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:10 smithi039 ceph-mon[96489]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:41:10.879 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:10 smithi039 ceph-mon[96489]: Health check failed: Reduced data availability: 2 pgs inactive, 4 pgs peering (PG_AVAILABILITY) 2023-07-21T18:41:11.121 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:10 smithi193 ceph-mon[98204]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:41:11.122 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:10 smithi193 ceph-mon[98204]: Health check failed: Reduced data availability: 2 pgs inactive, 4 pgs peering (PG_AVAILABILITY) 2023-07-21T18:41:11.140 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:10 smithi155 ceph-mon[97934]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:41:11.140 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:10 smithi155 ceph-mon[97934]: Health check failed: Reduced data availability: 2 pgs inactive, 4 pgs peering (PG_AVAILABILITY) 2023-07-21T18:41:11.603 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:11.579+0000 7fa19d6df700 1 -- 172.21.15.39:0/3486095460 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1980fea80 msgr2=0x7fa1980fee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:11.603 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:11.579+0000 7fa19d6df700 1 --2- 172.21.15.39:0/3486095460 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1980fea80 0x7fa1980fee90 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa188009990 tx=0x7fa188009ca0 comp rx=0 tx=0).stop 2023-07-21T18:41:11.604 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:11.579+0000 7fa19d6df700 1 -- 172.21.15.39:0/3486095460 shutdown_connections 2023-07-21T18:41:11.604 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:11.579+0000 7fa19d6df700 1 --2- 172.21.15.39:0/3486095460 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa198100610 0x7fa198106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:11.604 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:11.579+0000 7fa19d6df700 1 --2- 172.21.15.39:0/3486095460 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1980fea80 0x7fa1980fee90 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:11.604 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:11.579+0000 7fa19d6df700 1 --2- 172.21.15.39:0/3486095460 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1980ffc80 0x7fa1981000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:11.604 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:11.579+0000 7fa19d6df700 1 -- 172.21.15.39:0/3486095460 >> 172.21.15.39:0/3486095460 conn(0x7fa1980fa0b0 msgr2=0x7fa1980fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:11.604 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:11.579+0000 7fa19d6df700 1 -- 172.21.15.39:0/3486095460 shutdown_connections 2023-07-21T18:41:11.604 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:11.579+0000 7fa19d6df700 1 -- 172.21.15.39:0/3486095460 wait complete. 2023-07-21T18:41:11.605 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:11.580+0000 7fa19d6df700 1 Processor -- start 2023-07-21T18:41:11.605 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:11.580+0000 7fa19d6df700 1 -- start start 2023-07-21T18:41:11.606 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:11.580+0000 7fa19d6df700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1980fea80 0x7fa198102a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:11.606 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:11.580+0000 7fa19d6df700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1980ffc80 0x7fa198101060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:11.606 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:11.580+0000 7fa19d6df700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa198100610 0x7fa1981015a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:11.607 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:11.580+0000 7fa19d6df700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa198101d20 con 0x7fa1980ffc80 2023-07-21T18:41:11.607 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:11.580+0000 7fa19d6df700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa198101e90 con 0x7fa1980fea80 2023-07-21T18:41:11.607 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:11.580+0000 7fa19d6df700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa19819f860 con 0x7fa198100610 2023-07-21T18:41:11.607 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:11.580+0000 7fa1967fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1980ffc80 0x7fa198101060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:11.607 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:11.580+0000 7fa1967fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1980ffc80 0x7fa198101060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56132/0 (socket says 172.21.15.39:56132) 2023-07-21T18:41:11.607 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:11.580+0000 7fa196ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1980fea80 0x7fa198102a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:11.607 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:11.580+0000 7fa1967fc700 1 -- 172.21.15.39:0/3189596679 learned_addr learned my addr 172.21.15.39:0/3189596679 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:11.608 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:11.580+0000 7fa1977fe700 1 --2- 172.21.15.39:0/3189596679 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa198100610 0x7fa1981015a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:11.608 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:11.581+0000 7fa1967fc700 1 -- 172.21.15.39:0/3189596679 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa198100610 msgr2=0x7fa1981015a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:11.608 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:11.581+0000 7fa1967fc700 1 --2- 172.21.15.39:0/3189596679 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa198100610 0x7fa1981015a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:11.608 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:11.581+0000 7fa1967fc700 1 -- 172.21.15.39:0/3189596679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1980fea80 msgr2=0x7fa198102a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:11.608 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:11.581+0000 7fa1967fc700 1 --2- 172.21.15.39:0/3189596679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1980fea80 0x7fa198102a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:11.608 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:11.581+0000 7fa1967fc700 1 -- 172.21.15.39:0/3189596679 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1880095f0 con 0x7fa1980ffc80 2023-07-21T18:41:11.608 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:11.581+0000 7fa1967fc700 1 --2- 172.21.15.39:0/3189596679 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1980ffc80 0x7fa198101060 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa18000c9b0 tx=0x7fa18000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:11.608 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:11.581+0000 7fa18ffff700 1 -- 172.21.15.39:0/3189596679 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa180007b00 con 0x7fa1980ffc80 2023-07-21T18:41:11.609 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:11.581+0000 7fa19d6df700 1 -- 172.21.15.39:0/3189596679 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa19819fb40 con 0x7fa1980ffc80 2023-07-21T18:41:11.609 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:11.581+0000 7fa18ffff700 1 -- 172.21.15.39:0/3189596679 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa18000ce50 con 0x7fa1980ffc80 2023-07-21T18:41:11.609 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:11.581+0000 7fa18ffff700 1 -- 172.21.15.39:0/3189596679 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa180018660 con 0x7fa1980ffc80 2023-07-21T18:41:11.609 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:11.581+0000 7fa19d6df700 1 -- 172.21.15.39:0/3189596679 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1981a0040 con 0x7fa1980ffc80 2023-07-21T18:41:11.609 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:11.582+0000 7fa19d6df700 1 -- 172.21.15.39:0/3189596679 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa19810a240 con 0x7fa1980ffc80 2023-07-21T18:41:11.609 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:11.583+0000 7fa18ffff700 1 -- 172.21.15.39:0/3189596679 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fa18000f450 con 0x7fa1980ffc80 2023-07-21T18:41:11.610 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:11.584+0000 7fa18ffff700 1 --2- 172.21.15.39:0/3189596679 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fa1840a11e0 0x7fa1840a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:11.610 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:11.584+0000 7fa196ffd700 1 --2- 172.21.15.39:0/3189596679 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fa1840a11e0 0x7fa1840a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:11.610 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:11.584+0000 7fa18ffff700 1 -- 172.21.15.39:0/3189596679 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fa1800cc9b0 con 0x7fa1980ffc80 2023-07-21T18:41:11.610 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:11.584+0000 7fa196ffd700 1 --2- 172.21.15.39:0/3189596679 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fa1840a11e0 0x7fa1840a3690 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa188006010 tx=0x7fa188005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:11.612 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:11.588+0000 7fa18ffff700 1 -- 172.21.15.39:0/3189596679 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fa180095fd0 con 0x7fa1980ffc80 2023-07-21T18:41:11.820 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:11.796+0000 7fa19d6df700 1 -- 172.21.15.39:0/3189596679 --> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fa198068780 con 0x7fa1840a11e0 2023-07-21T18:41:11.821 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:11.798+0000 7fa18ffff700 1 -- 172.21.15.39:0/3189596679 <== mgr.14152 v2:172.21.15.39:6800/2116291712 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7fa198068780 con 0x7fa1840a11e0 2023-07-21T18:41:11.821 INFO:teuthology.orchestra.run.smithi039.stdout:HOST ADDR LABELS STATUS 2023-07-21T18:41:11.821 INFO:teuthology.orchestra.run.smithi039.stdout:smithi039 172.21.15.39 2023-07-21T18:41:11.822 INFO:teuthology.orchestra.run.smithi039.stdout:smithi155 172.21.15.155 2023-07-21T18:41:11.822 INFO:teuthology.orchestra.run.smithi039.stdout:smithi193 172.21.15.193 2023-07-21T18:41:11.822 INFO:teuthology.orchestra.run.smithi039.stdout:3 hosts in cluster 2023-07-21T18:41:11.823 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:11.800+0000 7fa19d6df700 1 -- 172.21.15.39:0/3189596679 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fa1840a11e0 msgr2=0x7fa1840a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:11.824 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:11.800+0000 7fa19d6df700 1 --2- 172.21.15.39:0/3189596679 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fa1840a11e0 0x7fa1840a3690 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa188006010 tx=0x7fa188005960 comp rx=0 tx=0).stop 2023-07-21T18:41:11.824 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:11.800+0000 7fa19d6df700 1 -- 172.21.15.39:0/3189596679 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1980ffc80 msgr2=0x7fa198101060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:11.824 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:11.800+0000 7fa19d6df700 1 --2- 172.21.15.39:0/3189596679 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1980ffc80 0x7fa198101060 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa18000c9b0 tx=0x7fa18000ccc0 comp rx=0 tx=0).stop 2023-07-21T18:41:11.824 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:11.800+0000 7fa19d6df700 1 -- 172.21.15.39:0/3189596679 shutdown_connections 2023-07-21T18:41:11.824 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:11.800+0000 7fa19d6df700 1 --2- 172.21.15.39:0/3189596679 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fa1840a11e0 0x7fa1840a3690 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:11.824 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:11.800+0000 7fa19d6df700 1 --2- 172.21.15.39:0/3189596679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1980fea80 0x7fa198102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:11.824 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:11.800+0000 7fa19d6df700 1 --2- 172.21.15.39:0/3189596679 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa198100610 0x7fa1981015a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:11.825 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:11.800+0000 7fa19d6df700 1 --2- 172.21.15.39:0/3189596679 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1980ffc80 0x7fa198101060 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:11.825 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:11.800+0000 7fa19d6df700 1 -- 172.21.15.39:0/3189596679 >> 172.21.15.39:0/3189596679 conn(0x7fa1980fa0b0 msgr2=0x7fa198068070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:11.825 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:11.800+0000 7fa19d6df700 1 -- 172.21.15.39:0/3189596679 shutdown_connections 2023-07-21T18:41:11.825 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:11.800+0000 7fa19d6df700 1 -- 172.21.15.39:0/3189596679 wait complete. 2023-07-21T18:41:12.188 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:11 smithi039 ceph-mon[96489]: pgmap v283: 97 pgs: 5 peering, 92 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2023-07-21T18:41:12.226 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- bash -c 'ceph orch device ls' 2023-07-21T18:41:12.371 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:11 smithi193 ceph-mon[98204]: pgmap v283: 97 pgs: 5 peering, 92 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2023-07-21T18:41:12.390 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:11 smithi155 ceph-mon[97934]: pgmap v283: 97 pgs: 5 peering, 92 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2023-07-21T18:41:12.788 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:41:13.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:12 smithi039 ceph-mon[96489]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:41:13.371 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:12 smithi193 ceph-mon[98204]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:41:13.390 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:12 smithi155 ceph-mon[97934]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:41:14.003 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:13.979+0000 7f2225b30700 1 -- 172.21.15.39:0/1551814231 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22200744c0 msgr2=0x7f2220072920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:14.003 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:13.979+0000 7f2225b30700 1 --2- 172.21.15.39:0/1551814231 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22200744c0 0x7f2220072920 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f2210009990 tx=0x7f2210009ca0 comp rx=0 tx=0).stop 2023-07-21T18:41:14.003 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:13.979+0000 7f2225b30700 1 -- 172.21.15.39:0/1551814231 shutdown_connections 2023-07-21T18:41:14.003 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:13.979+0000 7f2225b30700 1 --2- 172.21.15.39:0/1551814231 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2220107a80 0x7f2220109f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:14.003 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:13.979+0000 7f2225b30700 1 --2- 172.21.15.39:0/1551814231 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22200744c0 0x7f2220072920 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:14.004 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:13.979+0000 7f2225b30700 1 --2- 172.21.15.39:0/1551814231 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2220072ef0 0x7f2220073360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:14.004 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:13.979+0000 7f2225b30700 1 -- 172.21.15.39:0/1551814231 >> 172.21.15.39:0/1551814231 conn(0x7f22200fb1d0 msgr2=0x7f22200fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:14.004 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:13.979+0000 7f2225b30700 1 -- 172.21.15.39:0/1551814231 shutdown_connections 2023-07-21T18:41:14.004 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:13.979+0000 7f2225b30700 1 -- 172.21.15.39:0/1551814231 wait complete. 2023-07-21T18:41:14.004 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:13.980+0000 7f2225b30700 1 Processor -- start 2023-07-21T18:41:14.005 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:13.980+0000 7f2225b30700 1 -- start start 2023-07-21T18:41:14.006 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:13.980+0000 7f2225b30700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2220072ef0 0x7f2220197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:14.006 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:13.980+0000 7f2225b30700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22200744c0 0x7f2220197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:14.006 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:13.980+0000 7f2225b30700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2220107a80 0x7f222019f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:14.006 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:13.980+0000 7f2225b30700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f222019f640 con 0x7f2220072ef0 2023-07-21T18:41:14.006 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:13.980+0000 7f2225b30700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f222019f780 con 0x7f2220107a80 2023-07-21T18:41:14.006 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:13.980+0000 7f2225b30700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f222019f910 con 0x7f22200744c0 2023-07-21T18:41:14.006 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:13.980+0000 7f221f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2220072ef0 0x7f2220197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:14.007 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:13.980+0000 7f221f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2220072ef0 0x7f2220197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56142/0 (socket says 172.21.15.39:56142) 2023-07-21T18:41:14.007 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:13.980+0000 7f221f7fe700 1 -- 172.21.15.39:0/2937797329 learned_addr learned my addr 172.21.15.39:0/2937797329 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:14.007 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:13.980+0000 7f221effd700 1 --2- 172.21.15.39:0/2937797329 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22200744c0 0x7f2220197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:14.007 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:13.980+0000 7f221ffff700 1 --2- 172.21.15.39:0/2937797329 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2220107a80 0x7f222019f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:14.007 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:13.980+0000 7f221f7fe700 1 -- 172.21.15.39:0/2937797329 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22200744c0 msgr2=0x7f2220197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:14.007 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:13.980+0000 7f221f7fe700 1 --2- 172.21.15.39:0/2937797329 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22200744c0 0x7f2220197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:14.007 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:13.980+0000 7f221f7fe700 1 -- 172.21.15.39:0/2937797329 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2220107a80 msgr2=0x7f222019f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:14.007 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:13.980+0000 7f221f7fe700 1 --2- 172.21.15.39:0/2937797329 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2220107a80 0x7f222019f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:14.008 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:13.980+0000 7f221f7fe700 1 -- 172.21.15.39:0/2937797329 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22100095f0 con 0x7f2220072ef0 2023-07-21T18:41:14.008 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:13.981+0000 7f221f7fe700 1 --2- 172.21.15.39:0/2937797329 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2220072ef0 0x7f2220197ab0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2210006010 tx=0x7f22100047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:14.008 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:13.981+0000 7f221cff9700 1 -- 172.21.15.39:0/2937797329 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221001d070 con 0x7f2220072ef0 2023-07-21T18:41:14.008 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:13.981+0000 7f221cff9700 1 -- 172.21.15.39:0/2937797329 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f221000fb90 con 0x7f2220072ef0 2023-07-21T18:41:14.008 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:13.981+0000 7f2225b30700 1 -- 172.21.15.39:0/2937797329 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f222019fb90 con 0x7f2220072ef0 2023-07-21T18:41:14.008 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:13.981+0000 7f221cff9700 1 -- 172.21.15.39:0/2937797329 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2210017650 con 0x7f2220072ef0 2023-07-21T18:41:14.008 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:13.981+0000 7f2225b30700 1 -- 172.21.15.39:0/2937797329 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f222019ffa0 con 0x7f2220072ef0 2023-07-21T18:41:14.009 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:13.982+0000 7f2225b30700 1 -- 172.21.15.39:0/2937797329 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22201061c0 con 0x7f2220072ef0 2023-07-21T18:41:14.009 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:13.983+0000 7f221cff9700 1 -- 172.21.15.39:0/2937797329 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f221000fd00 con 0x7f2220072ef0 2023-07-21T18:41:14.009 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:13.983+0000 7f221cff9700 1 --2- 172.21.15.39:0/2937797329 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f22080a11e0 0x7f22080a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:14.010 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:13.984+0000 7f221cff9700 1 -- 172.21.15.39:0/2937797329 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f22100ce110 con 0x7f2220072ef0 2023-07-21T18:41:14.010 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:13.984+0000 7f221effd700 1 --2- 172.21.15.39:0/2937797329 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f22080a11e0 0x7f22080a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:14.010 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:13.984+0000 7f221effd700 1 --2- 172.21.15.39:0/2937797329 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f22080a11e0 0x7f22080a3690 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2214009c60 tx=0x7f2214009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:14.011 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:13.988+0000 7f221cff9700 1 -- 172.21.15.39:0/2937797329 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f22100977c0 con 0x7f2220072ef0 2023-07-21T18:41:14.189 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:13 smithi039 ceph-mon[96489]: pgmap v284: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-07-21T18:41:14.217 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:14.193+0000 7f2225b30700 1 -- 172.21.15.39:0/2937797329 --> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f2220060950 con 0x7f22080a11e0 2023-07-21T18:41:14.218 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:14.195+0000 7f221cff9700 1 -- 172.21.15.39:0/2937797329 <== mgr.14152 v2:172.21.15.39:6800/2116291712 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+492 (secure 0 0 0) 0x7f2220060950 con 0x7f22080a11e0 2023-07-21T18:41:14.219 INFO:teuthology.orchestra.run.smithi039.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2023-07-21T18:41:14.219 INFO:teuthology.orchestra.run.smithi039.stdout:smithi039 /dev/nvme0n1 ssd INTEL_SSDPEDKE040T7_BTLE736102QX4P0KGN 3726G No 2m ago LVM detected, locked 2023-07-21T18:41:14.219 INFO:teuthology.orchestra.run.smithi039.stdout:smithi155 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GW400BGN 372G No 2m ago LVM detected, locked 2023-07-21T18:41:14.219 INFO:teuthology.orchestra.run.smithi039.stdout:smithi193 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500HB400BGN 372G No 112s ago LVM detected, locked 2023-07-21T18:41:14.221 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:14.197+0000 7f2225b30700 1 -- 172.21.15.39:0/2937797329 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f22080a11e0 msgr2=0x7f22080a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:14.221 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:14.197+0000 7f2225b30700 1 --2- 172.21.15.39:0/2937797329 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f22080a11e0 0x7f22080a3690 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2214009c60 tx=0x7f2214009260 comp rx=0 tx=0).stop 2023-07-21T18:41:14.221 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:14.197+0000 7f2225b30700 1 -- 172.21.15.39:0/2937797329 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2220072ef0 msgr2=0x7f2220197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:14.221 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:14.197+0000 7f2225b30700 1 --2- 172.21.15.39:0/2937797329 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2220072ef0 0x7f2220197ab0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2210006010 tx=0x7f22100047a0 comp rx=0 tx=0).stop 2023-07-21T18:41:14.221 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:14.198+0000 7f2225b30700 1 -- 172.21.15.39:0/2937797329 shutdown_connections 2023-07-21T18:41:14.222 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:14.198+0000 7f2225b30700 1 --2- 172.21.15.39:0/2937797329 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f22080a11e0 0x7f22080a3690 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:14.222 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:14.198+0000 7f2225b30700 1 --2- 172.21.15.39:0/2937797329 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2220107a80 0x7f222019f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:14.222 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:14.198+0000 7f2225b30700 1 --2- 172.21.15.39:0/2937797329 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2220072ef0 0x7f2220197ab0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:14.222 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:14.198+0000 7f2225b30700 1 --2- 172.21.15.39:0/2937797329 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22200744c0 0x7f2220197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:14.222 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:14.198+0000 7f2225b30700 1 -- 172.21.15.39:0/2937797329 >> 172.21.15.39:0/2937797329 conn(0x7f22200fb1d0 msgr2=0x7f22200fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:14.222 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:14.198+0000 7f2225b30700 1 -- 172.21.15.39:0/2937797329 shutdown_connections 2023-07-21T18:41:14.222 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:14.198+0000 7f2225b30700 1 -- 172.21.15.39:0/2937797329 wait complete. 2023-07-21T18:41:14.371 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:13 smithi193 ceph-mon[98204]: pgmap v284: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-07-21T18:41:14.390 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:13 smithi155 ceph-mon[97934]: pgmap v284: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-07-21T18:41:14.652 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-07-21T18:41:14.661 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi039.front.sepia.ceph.com 2023-07-21T18:41:14.662 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- bash -c 'ceph fs dump' 2023-07-21T18:41:15.225 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:41:15.371 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:14 smithi193 ceph-mon[98204]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs inactive, 4 pgs peering) 2023-07-21T18:41:15.371 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:14 smithi193 ceph-mon[98204]: Cluster is now healthy 2023-07-21T18:41:15.390 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:14 smithi155 ceph-mon[97934]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs inactive, 4 pgs peering) 2023-07-21T18:41:15.432 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:14 smithi155 ceph-mon[97934]: Cluster is now healthy 2023-07-21T18:41:15.432 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:14 smithi039 ceph-mon[96489]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs inactive, 4 pgs peering) 2023-07-21T18:41:15.433 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:14 smithi039 ceph-mon[96489]: Cluster is now healthy 2023-07-21T18:41:16.259 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:16 smithi039 ceph-mon[96489]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:41:16.260 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:16 smithi039 ceph-mon[96489]: pgmap v285: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-07-21T18:41:16.371 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:16 smithi193 ceph-mon[98204]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:41:16.371 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:16 smithi193 ceph-mon[98204]: pgmap v285: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-07-21T18:41:16.390 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:16 smithi155 ceph-mon[97934]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:41:16.390 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:16 smithi155 ceph-mon[97934]: pgmap v285: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-07-21T18:41:16.442 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:16.418+0000 7f8d11fe9700 1 -- 172.21.15.39:0/3838712459 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d0c100690 msgr2=0x7f8d0c102a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:16.443 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:16.418+0000 7f8d11fe9700 1 --2- 172.21.15.39:0/3838712459 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d0c100690 0x7f8d0c102a70 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc00b0a0 tx=0x7f8cfc00b3b0 comp rx=0 tx=0).stop 2023-07-21T18:41:16.443 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:16.418+0000 7f8d11fe9700 1 -- 172.21.15.39:0/3838712459 shutdown_connections 2023-07-21T18:41:16.443 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:16.418+0000 7f8d11fe9700 1 --2- 172.21.15.39:0/3838712459 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d0c1058d0 0x7f8d0c107cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:16.443 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:16.418+0000 7f8d11fe9700 1 --2- 172.21.15.39:0/3838712459 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d0c100690 0x7f8d0c102a70 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:16.443 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:16.418+0000 7f8d11fe9700 1 --2- 172.21.15.39:0/3838712459 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d0c102fb0 0x7f8d0c105390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:16.443 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:16.418+0000 7f8d11fe9700 1 -- 172.21.15.39:0/3838712459 >> 172.21.15.39:0/3838712459 conn(0x7f8d0c0fa0b0 msgr2=0x7f8d0c0fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:16.444 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:16.419+0000 7f8d11fe9700 1 -- 172.21.15.39:0/3838712459 shutdown_connections 2023-07-21T18:41:16.444 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:16.419+0000 7f8d11fe9700 1 -- 172.21.15.39:0/3838712459 wait complete. 2023-07-21T18:41:16.444 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:16.419+0000 7f8d11fe9700 1 Processor -- start 2023-07-21T18:41:16.444 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:16.419+0000 7f8d11fe9700 1 -- start start 2023-07-21T18:41:16.445 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:16.419+0000 7f8d11fe9700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d0c100690 0x7f8d0c0ffff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:16.445 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:16.420+0000 7f8d11fe9700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d0c102fb0 0x7f8d0c0fe640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:16.446 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:16.420+0000 7f8d11fe9700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d0c1058d0 0x7f8d0c0feb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:16.446 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:16.420+0000 7f8d11fe9700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d0c0ff300 con 0x7f8d0c1058d0 2023-07-21T18:41:16.446 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:16.420+0000 7f8d11fe9700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d0c0ff470 con 0x7f8d0c100690 2023-07-21T18:41:16.446 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:16.420+0000 7f8d11fe9700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d0c19f9e0 con 0x7f8d0c102fb0 2023-07-21T18:41:16.446 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:16.420+0000 7f8d0bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d0c1058d0 0x7f8d0c0feb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:16.446 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:16.420+0000 7f8d0bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d0c1058d0 0x7f8d0c0feb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56160/0 (socket says 172.21.15.39:56160) 2023-07-21T18:41:16.447 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:16.420+0000 7f8d0bfff700 1 -- 172.21.15.39:0/2378436973 learned_addr learned my addr 172.21.15.39:0/2378436973 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:16.447 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:16.420+0000 7f8d0affd700 1 --2- 172.21.15.39:0/2378436973 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d0c102fb0 0x7f8d0c0fe640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:16.447 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:16.420+0000 7f8d0b7fe700 1 --2- 172.21.15.39:0/2378436973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d0c100690 0x7f8d0c0ffff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:16.447 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:16.420+0000 7f8d0bfff700 1 -- 172.21.15.39:0/2378436973 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d0c102fb0 msgr2=0x7f8d0c0fe640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:16.447 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:16.420+0000 7f8d0bfff700 1 --2- 172.21.15.39:0/2378436973 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d0c102fb0 0x7f8d0c0fe640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:16.447 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:16.420+0000 7f8d0bfff700 1 -- 172.21.15.39:0/2378436973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d0c100690 msgr2=0x7f8d0c0ffff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:16.447 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:16.420+0000 7f8d0bfff700 1 --2- 172.21.15.39:0/2378436973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d0c100690 0x7f8d0c0ffff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:16.447 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:16.420+0000 7f8d0bfff700 1 -- 172.21.15.39:0/2378436973 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cfc009d00 con 0x7f8d0c1058d0 2023-07-21T18:41:16.448 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:16.420+0000 7f8d0bfff700 1 --2- 172.21.15.39:0/2378436973 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d0c1058d0 0x7f8d0c0feb80 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8d0400c970 tx=0x7f8d0400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:16.448 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:16.420+0000 7f8d08ff9700 1 -- 172.21.15.39:0/2378436973 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d04004a30 con 0x7f8d0c1058d0 2023-07-21T18:41:16.448 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:16.420+0000 7f8d08ff9700 1 -- 172.21.15.39:0/2378436973 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d040054e0 con 0x7f8d0c1058d0 2023-07-21T18:41:16.448 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:16.420+0000 7f8d11fe9700 1 -- 172.21.15.39:0/2378436973 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d0c19fcc0 con 0x7f8d0c1058d0 2023-07-21T18:41:16.448 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:16.420+0000 7f8d08ff9700 1 -- 172.21.15.39:0/2378436973 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d0400eb70 con 0x7f8d0c1058d0 2023-07-21T18:41:16.448 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:16.421+0000 7f8d11fe9700 1 -- 172.21.15.39:0/2378436973 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d0c1a0170 con 0x7f8d0c1058d0 2023-07-21T18:41:16.448 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:16.422+0000 7f8d11fe9700 1 -- 172.21.15.39:0/2378436973 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d0c0fbc20 con 0x7f8d0c1058d0 2023-07-21T18:41:16.449 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:16.423+0000 7f8d08ff9700 1 -- 172.21.15.39:0/2378436973 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f8d0400edb0 con 0x7f8d0c1058d0 2023-07-21T18:41:16.449 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:16.423+0000 7f8d08ff9700 1 --2- 172.21.15.39:0/2378436973 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f8cf40a11e0 0x7f8cf40a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:16.449 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:16.424+0000 7f8d0b7fe700 1 --2- 172.21.15.39:0/2378436973 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f8cf40a11e0 0x7f8cf40a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:16.450 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:16.424+0000 7f8d08ff9700 1 -- 172.21.15.39:0/2378436973 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f8d040cc8b0 con 0x7f8d0c1058d0 2023-07-21T18:41:16.450 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:16.424+0000 7f8d0b7fe700 1 --2- 172.21.15.39:0/2378436973 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f8cf40a11e0 0x7f8cf40a3690 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc007ab0 tx=0x7f8cfc00b7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:16.450 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:16.427+0000 7f8d08ff9700 1 -- 172.21.15.39:0/2378436973 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f8d04095ed0 con 0x7f8d0c1058d0 2023-07-21T18:41:16.696 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:16.673+0000 7f8d11fe9700 1 -- 172.21.15.39:0/2378436973 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f8d0c02c850 con 0x7f8d0c1058d0 2023-07-21T18:41:16.697 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:16.673+0000 7f8d08ff9700 1 -- 172.21.15.39:0/2378436973 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+2923 (secure 0 0 0) 0x7f8d0401d020 con 0x7f8d0c1058d0 2023-07-21T18:41:16.697 INFO:teuthology.orchestra.run.smithi039.stdout:e26 2023-07-21T18:41:16.697 INFO:teuthology.orchestra.run.smithi039.stdout:enable_multiple, ever_enabled_multiple: 1,1 2023-07-21T18:41:16.697 INFO:teuthology.orchestra.run.smithi039.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-21T18:41:16.698 INFO:teuthology.orchestra.run.smithi039.stdout:legacy client fscid: 1 2023-07-21T18:41:16.698 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:41:16.698 INFO:teuthology.orchestra.run.smithi039.stdout:Filesystem 'cephfs' (1) 2023-07-21T18:41:16.698 INFO:teuthology.orchestra.run.smithi039.stdout:fs_name cephfs 2023-07-21T18:41:16.698 INFO:teuthology.orchestra.run.smithi039.stdout:epoch 26 2023-07-21T18:41:16.698 INFO:teuthology.orchestra.run.smithi039.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2023-07-21T18:41:16.698 INFO:teuthology.orchestra.run.smithi039.stdout:created 2023-07-21T18:40:04.594553+0000 2023-07-21T18:41:16.698 INFO:teuthology.orchestra.run.smithi039.stdout:modified 2023-07-21T18:40:11.989039+0000 2023-07-21T18:41:16.699 INFO:teuthology.orchestra.run.smithi039.stdout:tableserver 0 2023-07-21T18:41:16.699 INFO:teuthology.orchestra.run.smithi039.stdout:root 0 2023-07-21T18:41:16.699 INFO:teuthology.orchestra.run.smithi039.stdout:session_timeout 300 2023-07-21T18:41:16.699 INFO:teuthology.orchestra.run.smithi039.stdout:session_autoclose 300 2023-07-21T18:41:16.699 INFO:teuthology.orchestra.run.smithi039.stdout:max_file_size 1099511627776 2023-07-21T18:41:16.699 INFO:teuthology.orchestra.run.smithi039.stdout:max_xattr_size 65536 2023-07-21T18:41:16.699 INFO:teuthology.orchestra.run.smithi039.stdout:required_client_features {} 2023-07-21T18:41:16.699 INFO:teuthology.orchestra.run.smithi039.stdout:last_failure 0 2023-07-21T18:41:16.700 INFO:teuthology.orchestra.run.smithi039.stdout:last_failure_osd_epoch 0 2023-07-21T18:41:16.700 INFO:teuthology.orchestra.run.smithi039.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-21T18:41:16.700 INFO:teuthology.orchestra.run.smithi039.stdout:max_mds 5 2023-07-21T18:41:16.700 INFO:teuthology.orchestra.run.smithi039.stdout:in 0,1,2,3,4 2023-07-21T18:41:16.700 INFO:teuthology.orchestra.run.smithi039.stdout:up {0=24536,1=24500,2=24487,3=24464,4=14604} 2023-07-21T18:41:16.700 INFO:teuthology.orchestra.run.smithi039.stdout:failed 2023-07-21T18:41:16.700 INFO:teuthology.orchestra.run.smithi039.stdout:damaged 2023-07-21T18:41:16.701 INFO:teuthology.orchestra.run.smithi039.stdout:stopped 2023-07-21T18:41:16.701 INFO:teuthology.orchestra.run.smithi039.stdout:data_pools [3] 2023-07-21T18:41:16.701 INFO:teuthology.orchestra.run.smithi039.stdout:metadata_pool 2 2023-07-21T18:41:16.701 INFO:teuthology.orchestra.run.smithi039.stdout:inline_data disabled 2023-07-21T18:41:16.701 INFO:teuthology.orchestra.run.smithi039.stdout:balancer 2023-07-21T18:41:16.701 INFO:teuthology.orchestra.run.smithi039.stdout:bal_rank_mask -1 2023-07-21T18:41:16.701 INFO:teuthology.orchestra.run.smithi039.stdout:standby_count_wanted 0 2023-07-21T18:41:16.701 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.b{0:24536} state up:active seq 3 addr [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] compat {c=[1],r=[1],i=[7ff]}] 2023-07-21T18:41:16.702 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.c{0:24463} state up:standby-replay seq 1 addr [v2:172.21.15.193:6836/2626761843,v1:172.21.15.193:6837/2626761843] compat {c=[1],r=[1],i=[7ff]}] 2023-07-21T18:41:16.702 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.h{1:24500} state up:active seq 6 addr [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] compat {c=[1],r=[1],i=[7ff]}] 2023-07-21T18:41:16.702 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.f{1:24451} state up:standby-replay seq 1 addr [v2:172.21.15.193:6834/381560657,v1:172.21.15.193:6835/381560657] compat {c=[1],r=[1],i=[7ff]}] 2023-07-21T18:41:16.702 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.i{2:24487} state up:active seq 6 addr [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] compat {c=[1],r=[1],i=[7ff]}] 2023-07-21T18:41:16.702 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.k{2:24437} state up:standby-replay seq 1 addr [v2:172.21.15.155:6832/1034755277,v1:172.21.15.155:6833/1034755277] compat {c=[1],r=[1],i=[7ff]}] 2023-07-21T18:41:16.702 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.e{3:24464} state up:active seq 8 addr [v2:172.21.15.155:6834/3854856286,v1:172.21.15.155:6835/3854856286] compat {c=[1],r=[1],i=[7ff]}] 2023-07-21T18:41:16.702 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.l{3:24415} state up:standby-replay seq 1 addr [v2:172.21.15.193:6832/3106166804,v1:172.21.15.193:6833/3106166804] compat {c=[1],r=[1],i=[7ff]}] 2023-07-21T18:41:16.702 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.a{4:14604} state up:active seq 5 addr [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] compat {c=[1],r=[1],i=[7ff]}] 2023-07-21T18:41:16.703 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.g{4:14592} state up:standby-replay seq 1 addr [v2:172.21.15.39:6838/3184273349,v1:172.21.15.39:6839/3184273349] compat {c=[1],r=[1],i=[7ff]}] 2023-07-21T18:41:16.703 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:41:16.703 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:41:16.703 INFO:teuthology.orchestra.run.smithi039.stdout:Standby daemons: 2023-07-21T18:41:16.703 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:41:16.703 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.j{-1:14526} state up:standby seq 1 addr [v2:172.21.15.39:6834/147318681,v1:172.21.15.39:6835/147318681] compat {c=[1],r=[1],i=[7ff]}] 2023-07-21T18:41:16.703 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.d{-1:14538} state up:standby seq 1 addr [v2:172.21.15.39:6836/1062891760,v1:172.21.15.39:6837/1062891760] compat {c=[1],r=[1],i=[7ff]}] 2023-07-21T18:41:16.704 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:16.676+0000 7f8d11fe9700 1 -- 172.21.15.39:0/2378436973 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f8cf40a11e0 msgr2=0x7f8cf40a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:16.705 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:16.676+0000 7f8d11fe9700 1 --2- 172.21.15.39:0/2378436973 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f8cf40a11e0 0x7f8cf40a3690 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc007ab0 tx=0x7f8cfc00b7b0 comp rx=0 tx=0).stop 2023-07-21T18:41:16.705 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:16.676+0000 7f8d11fe9700 1 -- 172.21.15.39:0/2378436973 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d0c1058d0 msgr2=0x7f8d0c0feb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:16.705 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:16.676+0000 7f8d11fe9700 1 --2- 172.21.15.39:0/2378436973 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d0c1058d0 0x7f8d0c0feb80 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8d0400c970 tx=0x7f8d0400cc80 comp rx=0 tx=0).stop 2023-07-21T18:41:16.705 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:16.676+0000 7f8d11fe9700 1 -- 172.21.15.39:0/2378436973 shutdown_connections 2023-07-21T18:41:16.705 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:16.676+0000 7f8d11fe9700 1 --2- 172.21.15.39:0/2378436973 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f8cf40a11e0 0x7f8cf40a3690 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:16.706 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:16.676+0000 7f8d11fe9700 1 --2- 172.21.15.39:0/2378436973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d0c100690 0x7f8d0c0ffff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:16.706 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:16.676+0000 7f8d11fe9700 1 --2- 172.21.15.39:0/2378436973 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d0c1058d0 0x7f8d0c0feb80 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:16.706 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:16.676+0000 7f8d11fe9700 1 --2- 172.21.15.39:0/2378436973 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d0c102fb0 0x7f8d0c0fe640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:16.706 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:16.676+0000 7f8d11fe9700 1 -- 172.21.15.39:0/2378436973 >> 172.21.15.39:0/2378436973 conn(0x7f8d0c0fa0b0 msgr2=0x7f8d0c1012f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:16.706 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:16.676+0000 7f8d11fe9700 1 -- 172.21.15.39:0/2378436973 shutdown_connections 2023-07-21T18:41:16.706 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:16.676+0000 7f8d11fe9700 1 -- 172.21.15.39:0/2378436973 wait complete. 2023-07-21T18:41:16.706 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 26 2023-07-21T18:41:17.086 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- bash -c 'ceph osd dump' 2023-07-21T18:41:17.371 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:17 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/2378436973' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-07-21T18:41:17.389 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:17 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/2378436973' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-07-21T18:41:17.439 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:17 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/2378436973' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-07-21T18:41:17.646 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:41:18.127 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:18 smithi039 ceph-mon[96489]: pgmap v286: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-07-21T18:41:18.371 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:18 smithi193 ceph-mon[98204]: pgmap v286: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-07-21T18:41:18.389 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:18 smithi155 ceph-mon[97934]: pgmap v286: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-07-21T18:41:18.833 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:18.809+0000 7f837d703700 1 -- 172.21.15.39:0/4163030173 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83780ffd10 msgr2=0x7f8378100160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:18.833 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:18.809+0000 7f837d703700 1 --2- 172.21.15.39:0/4163030173 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83780ffd10 0x7f8378100160 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f836c009990 tx=0x7f836c009ca0 comp rx=0 tx=0).stop 2023-07-21T18:41:18.834 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:18.809+0000 7f837d703700 1 -- 172.21.15.39:0/4163030173 shutdown_connections 2023-07-21T18:41:18.834 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:18.809+0000 7f837d703700 1 --2- 172.21.15.39:0/4163030173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83780feb10 0x7f83780fef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:18.834 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:18.809+0000 7f837d703700 1 --2- 172.21.15.39:0/4163030173 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83781006a0 0x7f8378106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:18.834 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:18.809+0000 7f837d703700 1 --2- 172.21.15.39:0/4163030173 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83780ffd10 0x7f8378100160 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:18.834 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:18.809+0000 7f837d703700 1 -- 172.21.15.39:0/4163030173 >> 172.21.15.39:0/4163030173 conn(0x7f83780fa160 msgr2=0x7f83780fc590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:18.834 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:18.809+0000 7f837d703700 1 -- 172.21.15.39:0/4163030173 shutdown_connections 2023-07-21T18:41:18.834 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:18.809+0000 7f837d703700 1 -- 172.21.15.39:0/4163030173 wait complete. 2023-07-21T18:41:18.835 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:18.810+0000 7f837d703700 1 Processor -- start 2023-07-21T18:41:18.835 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:18.810+0000 7f837d703700 1 -- start start 2023-07-21T18:41:18.835 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:18.810+0000 7f837d703700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83780feb10 0x7f83781947b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:18.835 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:18.810+0000 7f837d703700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83780ffd10 0x7f8378194cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:18.835 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:18.810+0000 7f837d703700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83781006a0 0x7f837819bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:18.835 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:18.810+0000 7f837d703700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f837819c340 con 0x7f83781006a0 2023-07-21T18:41:18.835 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:18.810+0000 7f837d703700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f837819c480 con 0x7f83780ffd10 2023-07-21T18:41:18.835 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:18.810+0000 7f837d703700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f837819c610 con 0x7f83780feb10 2023-07-21T18:41:18.837 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:18.810+0000 7f83777fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83781006a0 0x7f837819bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:18.837 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:18.810+0000 7f83767fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83780ffd10 0x7f8378194cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:18.837 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:18.810+0000 7f83767fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83780ffd10 0x7f8378194cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:40534/0 (socket says 172.21.15.39:40534) 2023-07-21T18:41:18.837 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:18.811+0000 7f83767fc700 1 -- 172.21.15.39:0/4054605177 learned_addr learned my addr 172.21.15.39:0/4054605177 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:18.837 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:18.810+0000 7f83777fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83781006a0 0x7f837819bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56172/0 (socket says 172.21.15.39:56172) 2023-07-21T18:41:18.838 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:18.811+0000 7f8376ffd700 1 --2- 172.21.15.39:0/4054605177 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83780feb10 0x7f83781947b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:18.838 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:18.811+0000 7f83777fe700 1 -- 172.21.15.39:0/4054605177 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83780feb10 msgr2=0x7f83781947b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:18.838 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:18.811+0000 7f83777fe700 1 --2- 172.21.15.39:0/4054605177 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83780feb10 0x7f83781947b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:18.838 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:18.811+0000 7f83777fe700 1 -- 172.21.15.39:0/4054605177 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83780ffd10 msgr2=0x7f8378194cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:18.838 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:18.811+0000 7f83777fe700 1 --2- 172.21.15.39:0/4054605177 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83780ffd10 0x7f8378194cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:18.838 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:18.811+0000 7f83777fe700 1 -- 172.21.15.39:0/4054605177 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f836c0095f0 con 0x7f83781006a0 2023-07-21T18:41:18.838 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:18.811+0000 7f83777fe700 1 --2- 172.21.15.39:0/4054605177 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83781006a0 0x7f837819bd70 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f837000ab70 tx=0x7f837000ae80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:18.839 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:18.811+0000 7f835ffff700 1 -- 172.21.15.39:0/4054605177 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f837000f070 con 0x7f83781006a0 2023-07-21T18:41:18.839 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:18.811+0000 7f835ffff700 1 -- 172.21.15.39:0/4054605177 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8370007a70 con 0x7f83781006a0 2023-07-21T18:41:18.839 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:18.811+0000 7f837d703700 1 -- 172.21.15.39:0/4054605177 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f837819c8f0 con 0x7f83781006a0 2023-07-21T18:41:18.839 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:18.811+0000 7f835ffff700 1 -- 172.21.15.39:0/4054605177 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8370005280 con 0x7f83781006a0 2023-07-21T18:41:18.839 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:18.811+0000 7f837d703700 1 -- 172.21.15.39:0/4054605177 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f837819cda0 con 0x7f83781006a0 2023-07-21T18:41:18.839 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:18.812+0000 7f837d703700 1 -- 172.21.15.39:0/4054605177 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f837810a2d0 con 0x7f83781006a0 2023-07-21T18:41:18.839 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:18.813+0000 7f835ffff700 1 -- 172.21.15.39:0/4054605177 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f8370004ad0 con 0x7f83781006a0 2023-07-21T18:41:18.839 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:18.814+0000 7f835ffff700 1 --2- 172.21.15.39:0/4054605177 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f83600a11e0 0x7f83600a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:18.840 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:18.814+0000 7f8376ffd700 1 --2- 172.21.15.39:0/4054605177 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f83600a11e0 0x7f83600a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:18.840 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:18.814+0000 7f835ffff700 1 -- 172.21.15.39:0/4054605177 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f83700cbf30 con 0x7f83781006a0 2023-07-21T18:41:18.840 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:18.814+0000 7f8376ffd700 1 --2- 172.21.15.39:0/4054605177 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f83600a11e0 0x7f83600a3690 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8368005950 tx=0x7f83680058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:18.841 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:18.818+0000 7f835ffff700 1 -- 172.21.15.39:0/4054605177 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f83700955e0 con 0x7f83781006a0 2023-07-21T18:41:19.041 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.018+0000 7f837d703700 1 -- 172.21.15.39:0/4054605177 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f8378105ab0 con 0x7f83781006a0 2023-07-21T18:41:19.042 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.019+0000 7f835ffff700 1 -- 172.21.15.39:0/4054605177 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v81) v1 ==== 56+0+5254 (secure 0 0 0) 0x7f8370098c00 con 0x7f83781006a0 2023-07-21T18:41:19.042 INFO:teuthology.orchestra.run.smithi039.stdout:epoch 81 2023-07-21T18:41:19.042 INFO:teuthology.orchestra.run.smithi039.stdout:fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c 2023-07-21T18:41:19.042 INFO:teuthology.orchestra.run.smithi039.stdout:created 2023-07-21T18:33:06.504979+0000 2023-07-21T18:41:19.043 INFO:teuthology.orchestra.run.smithi039.stdout:modified 2023-07-21T18:41:08.140284+0000 2023-07-21T18:41:19.043 INFO:teuthology.orchestra.run.smithi039.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-07-21T18:41:19.043 INFO:teuthology.orchestra.run.smithi039.stdout:crush_version 26 2023-07-21T18:41:19.043 INFO:teuthology.orchestra.run.smithi039.stdout:full_ratio 0.95 2023-07-21T18:41:19.043 INFO:teuthology.orchestra.run.smithi039.stdout:backfillfull_ratio 0.9 2023-07-21T18:41:19.043 INFO:teuthology.orchestra.run.smithi039.stdout:nearfull_ratio 0.85 2023-07-21T18:41:19.043 INFO:teuthology.orchestra.run.smithi039.stdout:require_min_compat_client luminous 2023-07-21T18:41:19.044 INFO:teuthology.orchestra.run.smithi039.stdout:min_compat_client luminous 2023-07-21T18:41:19.044 INFO:teuthology.orchestra.run.smithi039.stdout:require_osd_release reef 2023-07-21T18:41:19.044 INFO:teuthology.orchestra.run.smithi039.stdout:stretch_mode_enabled false 2023-07-21T18:41:19.044 INFO:teuthology.orchestra.run.smithi039.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2023-07-21T18:41:19.044 INFO:teuthology.orchestra.run.smithi039.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 78 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2023-07-21T18:41:19.044 INFO:teuthology.orchestra.run.smithi039.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 79 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2023-07-21T18:41:19.044 INFO:teuthology.orchestra.run.smithi039.stdout:max_osd 12 2023-07-21T18:41:19.045 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0 up in weight 1 up_from 8 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.39:6802/860193857,v1:172.21.15.39:6803/860193857] [v2:172.21.15.39:6804/860193857,v1:172.21.15.39:6805/860193857] exists,up 03502c07-2d0f-4723-8d0b-d34c9f5b91bf 2023-07-21T18:41:19.045 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1 up in weight 1 up_from 13 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.155:6800/3246721764,v1:172.21.15.155:6801/3246721764] [v2:172.21.15.155:6802/3246721764,v1:172.21.15.155:6803/3246721764] exists,up 0831a0c0-a1fa-4d30-9a28-513223aa69eb 2023-07-21T18:41:19.045 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2 up in weight 1 up_from 18 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.193:6800/2563594449,v1:172.21.15.193:6801/2563594449] [v2:172.21.15.193:6802/2563594449,v1:172.21.15.193:6803/2563594449] exists,up 22df0003-bde9-48e4-b47a-cb8eafbab0db 2023-07-21T18:41:19.045 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3 up in weight 1 up_from 24 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.39:6810/863708386,v1:172.21.15.39:6811/863708386] [v2:172.21.15.39:6812/863708386,v1:172.21.15.39:6813/863708386] exists,up 54562558-a2e8-45dd-bc95-561569b9ae6b 2023-07-21T18:41:19.045 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4 up in weight 1 up_from 31 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.155:6808/3361780673,v1:172.21.15.155:6809/3361780673] [v2:172.21.15.155:6810/3361780673,v1:172.21.15.155:6811/3361780673] exists,up 491118e7-ae1e-495c-9eac-ef1cdcae9600 2023-07-21T18:41:19.045 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5 up in weight 1 up_from 36 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.193:6808/2376867721,v1:172.21.15.193:6809/2376867721] [v2:172.21.15.193:6810/2376867721,v1:172.21.15.193:6811/2376867721] exists,up d267bc2c-42b3-4d96-b2d1-b186dc7e6b18 2023-07-21T18:41:19.045 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6 up in weight 1 up_from 43 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.39:6818/2417770529,v1:172.21.15.39:6819/2417770529] [v2:172.21.15.39:6820/2417770529,v1:172.21.15.39:6821/2417770529] exists,up beec8088-970a-473e-bd67-6eee7cbdef24 2023-07-21T18:41:19.045 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7 up in weight 1 up_from 48 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.155:6816/2542002111,v1:172.21.15.155:6817/2542002111] [v2:172.21.15.155:6818/2542002111,v1:172.21.15.155:6819/2542002111] exists,up 5a88ce30-1835-4254-90cc-804ed5cf1066 2023-07-21T18:41:19.046 INFO:teuthology.orchestra.run.smithi039.stdout:osd.8 up in weight 1 up_from 55 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.193:6816/2985835085,v1:172.21.15.193:6817/2985835085] [v2:172.21.15.193:6818/2985835085,v1:172.21.15.193:6819/2985835085] exists,up d964a183-3021-4ace-9a0c-aad00f2570e4 2023-07-21T18:41:19.046 INFO:teuthology.orchestra.run.smithi039.stdout:osd.9 up in weight 1 up_from 62 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.39:6826/1946493651,v1:172.21.15.39:6827/1946493651] [v2:172.21.15.39:6828/1946493651,v1:172.21.15.39:6829/1946493651] exists,up 9725ea5c-49a3-4c99-bac4-e95069100b1e 2023-07-21T18:41:19.046 INFO:teuthology.orchestra.run.smithi039.stdout:osd.10 up in weight 1 up_from 67 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.155:6824/1234615022,v1:172.21.15.155:6825/1234615022] [v2:172.21.15.155:6826/1234615022,v1:172.21.15.155:6827/1234615022] exists,up 1fd60ccb-92db-45f0-a580-2003db0976e4 2023-07-21T18:41:19.046 INFO:teuthology.orchestra.run.smithi039.stdout:osd.11 up in weight 1 up_from 74 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.193:6824/215104663,v1:172.21.15.193:6825/215104663] [v2:172.21.15.193:6826/215104663,v1:172.21.15.193:6827/215104663] exists,up efb31ea1-4775-4e5f-8740-728d6f4aeea6 2023-07-21T18:41:19.046 INFO:teuthology.orchestra.run.smithi039.stdout:pg_upmap_items 2.6 [8,11] 2023-07-21T18:41:19.046 INFO:teuthology.orchestra.run.smithi039.stdout:pg_upmap_items 2.8 [2,11] 2023-07-21T18:41:19.046 INFO:teuthology.orchestra.run.smithi039.stdout:pg_upmap_items 3.e [4,6] 2023-07-21T18:41:19.047 INFO:teuthology.orchestra.run.smithi039.stdout:pg_upmap_items 3.1d [5,9] 2023-07-21T18:41:19.047 INFO:teuthology.orchestra.run.smithi039.stdout:pg_upmap_items 3.38 [3,9] 2023-07-21T18:41:19.047 INFO:teuthology.orchestra.run.smithi039.stdout:pg_upmap_items 3.3a [4,8] 2023-07-21T18:41:19.047 INFO:teuthology.orchestra.run.smithi039.stdout:pg_upmap_items 3.3c [4,9] 2023-07-21T18:41:19.047 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:0/2902600414 expires 2023-07-22T18:34:07.039823+0000 2023-07-21T18:41:19.047 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:6801/2893866425 expires 2023-07-22T18:34:07.039823+0000 2023-07-21T18:41:19.047 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:0/19717002 expires 2023-07-22T18:34:07.039823+0000 2023-07-21T18:41:19.047 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:6800/2893866425 expires 2023-07-22T18:34:07.039823+0000 2023-07-21T18:41:19.048 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:6800/889516631 expires 2023-07-22T18:33:40.852706+0000 2023-07-21T18:41:19.048 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:6801/889516631 expires 2023-07-22T18:33:40.852706+0000 2023-07-21T18:41:19.048 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:0/3636510379 expires 2023-07-22T18:33:40.852706+0000 2023-07-21T18:41:19.048 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:0/1275748831 expires 2023-07-22T18:33:40.852706+0000 2023-07-21T18:41:19.048 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:0/3819152602 expires 2023-07-22T18:33:40.852706+0000 2023-07-21T18:41:19.048 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:0/1127767082 expires 2023-07-22T18:34:07.039823+0000 2023-07-21T18:41:19.049 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.021+0000 7f837d703700 1 -- 172.21.15.39:0/4054605177 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f83600a11e0 msgr2=0x7f83600a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:19.049 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.021+0000 7f837d703700 1 --2- 172.21.15.39:0/4054605177 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f83600a11e0 0x7f83600a3690 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8368005950 tx=0x7f83680058e0 comp rx=0 tx=0).stop 2023-07-21T18:41:19.049 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.021+0000 7f837d703700 1 -- 172.21.15.39:0/4054605177 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83781006a0 msgr2=0x7f837819bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:19.049 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.021+0000 7f837d703700 1 --2- 172.21.15.39:0/4054605177 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83781006a0 0x7f837819bd70 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f837000ab70 tx=0x7f837000ae80 comp rx=0 tx=0).stop 2023-07-21T18:41:19.049 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.021+0000 7f837d703700 1 -- 172.21.15.39:0/4054605177 shutdown_connections 2023-07-21T18:41:19.050 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.021+0000 7f837d703700 1 --2- 172.21.15.39:0/4054605177 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f83600a11e0 0x7f83600a3690 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:19.050 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.021+0000 7f837d703700 1 --2- 172.21.15.39:0/4054605177 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83781006a0 0x7f837819bd70 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:19.050 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.021+0000 7f837d703700 1 --2- 172.21.15.39:0/4054605177 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83780feb10 0x7f83781947b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:19.050 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.021+0000 7f837d703700 1 --2- 172.21.15.39:0/4054605177 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83780ffd10 0x7f8378194cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:19.050 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.021+0000 7f837d703700 1 -- 172.21.15.39:0/4054605177 >> 172.21.15.39:0/4054605177 conn(0x7f83780fa160 msgr2=0x7f83780fc460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:19.050 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.021+0000 7f837d703700 1 -- 172.21.15.39:0/4054605177 shutdown_connections 2023-07-21T18:41:19.050 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.021+0000 7f837d703700 1 -- 172.21.15.39:0/4054605177 wait complete. 2023-07-21T18:41:19.371 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:19 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/4054605177' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-07-21T18:41:19.389 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:19 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/4054605177' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-07-21T18:41:19.421 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:19 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/4054605177' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-07-21T18:41:19.506 INFO:teuthology.run_tasks:Running task fs.ready... 2023-07-21T18:41:19.523 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T18:41:19.653 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.630+0000 7f99778ff700 1 -- 172.21.15.39:0/1626150477 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99700fdb20 msgr2=0x7f99700fdf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:19.654 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.630+0000 7f99778ff700 1 --2- 172.21.15.39:0/1626150477 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99700fdb20 0x7f99700fdf70 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f9960009990 tx=0x7f9960009ca0 comp rx=0 tx=0).stop 2023-07-21T18:41:19.654 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.630+0000 7f99778ff700 1 -- 172.21.15.39:0/1626150477 shutdown_connections 2023-07-21T18:41:19.654 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.630+0000 7f99778ff700 1 --2- 172.21.15.39:0/1626150477 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f99701010b0 0x7f99701014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:19.654 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.630+0000 7f99778ff700 1 --2- 172.21.15.39:0/1626150477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f99700fe4b0 0x7f9970108090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:19.655 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.630+0000 7f99778ff700 1 --2- 172.21.15.39:0/1626150477 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99700fdb20 0x7f99700fdf70 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:19.655 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.630+0000 7f99778ff700 1 -- 172.21.15.39:0/1626150477 >> 172.21.15.39:0/1626150477 conn(0x7f99700f9770 msgr2=0x7f99700fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:19.655 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.630+0000 7f99778ff700 1 -- 172.21.15.39:0/1626150477 shutdown_connections 2023-07-21T18:41:19.656 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.630+0000 7f99778ff700 1 -- 172.21.15.39:0/1626150477 wait complete. 2023-07-21T18:41:19.659 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.631+0000 7f99778ff700 1 Processor -- start 2023-07-21T18:41:19.660 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.631+0000 7f99778ff700 1 -- start start 2023-07-21T18:41:19.660 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.631+0000 7f99778ff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f99700fdb20 0x7f99701007b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:19.660 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.631+0000 7f99778ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f99700fe4b0 0x7f99700fedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:19.661 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.631+0000 7f99778ff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99701010b0 0x7f99700ff330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:19.661 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.631+0000 7f99778ff700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99700ffad0 con 0x7f99701010b0 2023-07-21T18:41:19.661 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.631+0000 7f99778ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99700ffc40 con 0x7f99700fe4b0 2023-07-21T18:41:19.661 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.631+0000 7f99778ff700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f997019f830 con 0x7f99700fdb20 2023-07-21T18:41:19.662 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.631+0000 7f9975e9c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99701010b0 0x7f99700ff330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:19.662 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.631+0000 7f9975e9c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99701010b0 0x7f99700ff330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56184/0 (socket says 172.21.15.39:56184) 2023-07-21T18:41:19.663 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.631+0000 7f9975e9c700 1 -- 172.21.15.39:0/1668835619 learned_addr learned my addr 172.21.15.39:0/1668835619 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:19.663 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.631+0000 7f997569b700 1 --2- 172.21.15.39:0/1668835619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f99700fdb20 0x7f99701007b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:19.663 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.631+0000 7f9974e9a700 1 --2- 172.21.15.39:0/1668835619 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f99700fe4b0 0x7f99700fedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:19.663 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.631+0000 7f9975e9c700 1 -- 172.21.15.39:0/1668835619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f99700fdb20 msgr2=0x7f99701007b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:19.664 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.631+0000 7f9975e9c700 1 --2- 172.21.15.39:0/1668835619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f99700fdb20 0x7f99701007b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:19.664 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.631+0000 7f9975e9c700 1 -- 172.21.15.39:0/1668835619 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f99700fe4b0 msgr2=0x7f99700fedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:19.664 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.631+0000 7f9975e9c700 1 --2- 172.21.15.39:0/1668835619 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f99700fe4b0 0x7f99700fedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:19.664 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.631+0000 7f9975e9c700 1 -- 172.21.15.39:0/1668835619 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99600095f0 con 0x7f99701010b0 2023-07-21T18:41:19.665 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.631+0000 7f9974e9a700 1 --2- 172.21.15.39:0/1668835619 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f99700fe4b0 0x7f99700fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:41:19.665 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.632+0000 7f9975e9c700 1 --2- 172.21.15.39:0/1668835619 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99701010b0 0x7f99700ff330 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9968007ee0 tx=0x7f996800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:19.665 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.632+0000 7f99667fc700 1 -- 172.21.15.39:0/1668835619 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9968017070 con 0x7f99701010b0 2023-07-21T18:41:19.665 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.632+0000 7f99667fc700 1 -- 172.21.15.39:0/1668835619 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f996800f040 con 0x7f99701010b0 2023-07-21T18:41:19.665 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.632+0000 7f99778ff700 1 -- 172.21.15.39:0/1668835619 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f997019fb10 con 0x7f99701010b0 2023-07-21T18:41:19.665 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.632+0000 7f99667fc700 1 -- 172.21.15.39:0/1668835619 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9968005740 con 0x7f99701010b0 2023-07-21T18:41:19.665 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.632+0000 7f99778ff700 1 -- 172.21.15.39:0/1668835619 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f997019ffd0 con 0x7f99701010b0 2023-07-21T18:41:19.665 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.633+0000 7f99778ff700 1 -- 172.21.15.39:0/1668835619 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99701071a0 con 0x7f99701010b0 2023-07-21T18:41:19.666 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.634+0000 7f99667fc700 1 -- 172.21.15.39:0/1668835619 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f996800dc60 con 0x7f99701010b0 2023-07-21T18:41:19.666 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.634+0000 7f99667fc700 1 --2- 172.21.15.39:0/1668835619 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f995c0a1220 0x7f995c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:19.666 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.634+0000 7f997569b700 1 --2- 172.21.15.39:0/1668835619 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f995c0a1220 0x7f995c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:19.666 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.634+0000 7f99667fc700 1 -- 172.21.15.39:0/1668835619 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f99680cc090 con 0x7f99701010b0 2023-07-21T18:41:19.666 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.635+0000 7f997569b700 1 --2- 172.21.15.39:0/1668835619 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f995c0a1220 0x7f995c0a36d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9970067390 tx=0x7f996c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:19.667 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.638+0000 7f99667fc700 1 -- 172.21.15.39:0/1668835619 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f99680956b0 con 0x7f99701010b0 2023-07-21T18:41:19.862 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.839+0000 7f99778ff700 1 -- 172.21.15.39:0/1668835619 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f997002caa0 con 0x7f99701010b0 2023-07-21T18:41:19.863 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.840+0000 7f99667fc700 1 -- 172.21.15.39:0/1668835619 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+22447 (secure 0 0 0) 0x7f9968098cd0 con 0x7f99701010b0 2023-07-21T18:41:19.864 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:41:19.864 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":81,"fsid":"eaf8e25a-27f4-11ee-9b34-001a4aab830c","created":"2023-07-21T18:33:06.504979+0000","modified":"2023-07-21T18:41:08.140284+0000","last_up_change":"2023-07-21T18:39:23.531009+0000","last_in_change":"2023-07-21T18:39:05.993024+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T18:36:13.572583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T18:40:02.726735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T18:40:03.990112+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03502c07-2d0f-4723-8d0b-d34c9f5b91bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6803","nonce":860193857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6805","nonce":860193857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6809","nonce":860193857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6807","nonce":860193857}]},"public_addr":"172.21.15.39:6803/860193857","cluster_addr":"172.21.15.39:6805/860193857","heartbeat_back_addr":"172.21.15.39:6809/860193857","heartbeat_front_addr":"172.21.15.39:6807/860193857","state":["exists","up"]},{"osd":1,"uuid":"0831a0c0-a1fa-4d30-9a28-513223aa69eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6801","nonce":3246721764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6803","nonce":3246721764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6807","nonce":3246721764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6805","nonce":3246721764}]},"public_addr":"172.21.15.155:6801/3246721764","cluster_addr":"172.21.15.155:6803/3246721764","heartbeat_back_addr":"172.21.15.155:6807/3246721764","heartbeat_front_addr":"172.21.15.155:6805/3246721764","state":["exists","up"]},{"osd":2,"uuid":"22df0003-bde9-48e4-b47a-cb8eafbab0db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6801","nonce":2563594449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6803","nonce":2563594449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6807","nonce":2563594449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6805","nonce":2563594449}]},"public_addr":"172.21.15.193:6801/2563594449","cluster_addr":"172.21.15.193:6803/2563594449","heartbeat_back_addr":"172.21.15.193:6807/2563594449","heartbeat_front_addr":"172.21.15.193:6805/2563594449","state":["exists","up"]},{"osd":3,"uuid":"54562558-a2e8-45dd-bc95-561569b9ae6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6811","nonce":863708386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6813","nonce":863708386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6817","nonce":863708386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6815","nonce":863708386}]},"public_addr":"172.21.15.39:6811/863708386","cluster_addr":"172.21.15.39:6813/863708386","heartbeat_back_addr":"172.21.15.39:6817/863708386","heartbeat_front_addr":"172.21.15.39:6815/863708386","state":["exists","up"]},{"osd":4,"uuid":"491118e7-ae1e-495c-9eac-ef1cdcae9600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6809","nonce":3361780673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6811","nonce":3361780673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6815","nonce":3361780673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6813","nonce":3361780673}]},"public_addr":"172.21.15.155:6809/3361780673","cluster_addr":"172.21.15.155:6811/3361780673","heartbeat_back_addr":"172.21.15.155:6815/3361780673","heartbeat_front_addr":"172.21.15.155:6813/3361780673","state":["exists","up"]},{"osd":5,"uuid":"d267bc2c-42b3-4d96-b2d1-b186dc7e6b18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6809","nonce":2376867721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6811","nonce":2376867721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6815","nonce":2376867721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6813","nonce":2376867721}]},"public_addr":"172.21.15.193:6809/2376867721","cluster_addr":"172.21.15.193:6811/2376867721","heartbeat_back_addr":"172.21.15.193:6815/2376867721","heartbeat_front_addr":"172.21.15.193:6813/2376867721","state":["exists","up"]},{"osd":6,"uuid":"beec8088-970a-473e-bd67-6eee7cbdef24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6819","nonce":2417770529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6821","nonce":2417770529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6825","nonce":2417770529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6823","nonce":2417770529}]},"public_addr":"172.21.15.39:6819/2417770529","cluster_addr":"172.21.15.39:6821/2417770529","heartbeat_back_addr":"172.21.15.39:6825/2417770529","heartbeat_front_addr":"172.21.15.39:6823/2417770529","state":["exists","up"]},{"osd":7,"uuid":"5a88ce30-1835-4254-90cc-804ed5cf1066","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6817","nonce":2542002111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6819","nonce":2542002111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6823","nonce":2542002111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6821","nonce":2542002111}]},"public_addr":"172.21.15.155:6817/2542002111","cluster_addr":"172.21.15.155:6819/2542002111","heartbeat_back_addr":"172.21.15.155:6823/2542002111","heartbeat_front_addr":"172.21.15.155:6821/2542002111","state":["exists","up"]},{"osd":8,"uuid":"d964a183-3021-4ace-9a0c-aad00f2570e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6817","nonce":2985835085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6819","nonce":2985835085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6823","nonce":2985835085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6821","nonce":2985835085}]},"public_addr":"172.21.15.193:6817/2985835085","cluster_addr":"172.21.15.193:6819/2985835085","heartbeat_back_addr":"172.21.15.193:6823/2985835085","heartbeat_front_addr":"172.21.15.193:6821/2985835085","state":["exists","up"]},{"osd":9,"uuid":"9725ea5c-49a3-4c99-bac4-e95069100b1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6827","nonce":1946493651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6829","nonce":1946493651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6833","nonce":1946493651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6831","nonce":1946493651}]},"public_addr":"172.21.15.39:6827/1946493651","cluster_addr":"172.21.15.39:6829/1946493651","heartbeat_back_addr":"172.21.15.39:6833/1946493651","heartbeat_front_addr":"172.21.15.39:6831/1946493651","state":["exists","up"]},{"osd":10,"uuid":"1fd60ccb-92db-45f0-a580-2003db0976e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6825","nonce":1234615022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6827","nonce":1234615022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6831","nonce":1234615022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6829","nonce":1234615022}]},"public_addr":"172.21.15.155:6825/1234615022","cluster_addr":"172.21.15.155:6827/1234615022","heartbeat_back_addr":"172.21.15.155:6831/1234615022","heartbeat_front_addr":"172.21.15.155:6829/1234615022","state":["exists","up"]},{"osd":11,"uuid":"efb31ea1-4775-4e5f-8740-728d6f4aeea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6825","nonce":215104663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6827","nonce":215104663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6831","nonce":215104663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6829","nonce":215104663}]},"public_addr":"172.21.15.193:6825/215104663","cluster_addr":"172.21.15.193:6827/215104663","heartbeat_back_addr":"172.21.15.193:6831/215104663","heartbeat_front_addr":"172.21.15.193:6829/215104663","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:35:30.211698+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:35:50.883846+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:36:11.124168+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:36:32.711905+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:36:54.442007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:37:14.843526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:37:36.587354+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:37:57.882374+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:38:18.729407+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:38:40.139231+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:39:01.290862+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:39:21.767537+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:0/2902600414":"2023-07-22T18:34:07.039823+0000","172.21.15.39:6801/2893866425":"2023-07-22T18:34:07.039823+0000","172.21.15.39:0/19717002":"2023-07-22T18:34:07.039823+0000","172.21.15.39:6800/2893866425":"2023-07-22T18:34:07.039823+0000","172.21.15.39:6800/889516631":"2023-07-22T18:33:40.852706+0000","172.21.15.39:6801/889516631":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/3636510379":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/1275748831":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/3819152602":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/1127767082":"2023-07-22T18:34:07.039823+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T18:41:19.866 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.843+0000 7f99778ff700 1 -- 172.21.15.39:0/1668835619 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f995c0a1220 msgr2=0x7f995c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:19.866 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.843+0000 7f99778ff700 1 --2- 172.21.15.39:0/1668835619 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f995c0a1220 0x7f995c0a36d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9970067390 tx=0x7f996c00b410 comp rx=0 tx=0).stop 2023-07-21T18:41:19.866 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.843+0000 7f99778ff700 1 -- 172.21.15.39:0/1668835619 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99701010b0 msgr2=0x7f99700ff330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:19.866 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.843+0000 7f99778ff700 1 --2- 172.21.15.39:0/1668835619 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99701010b0 0x7f99700ff330 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9968007ee0 tx=0x7f996800d3b0 comp rx=0 tx=0).stop 2023-07-21T18:41:19.867 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.843+0000 7f99778ff700 1 -- 172.21.15.39:0/1668835619 shutdown_connections 2023-07-21T18:41:19.867 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.843+0000 7f99778ff700 1 --2- 172.21.15.39:0/1668835619 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f995c0a1220 0x7f995c0a36d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:19.867 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.843+0000 7f99778ff700 1 --2- 172.21.15.39:0/1668835619 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99701010b0 0x7f99700ff330 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:19.867 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.843+0000 7f99778ff700 1 --2- 172.21.15.39:0/1668835619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f99700fdb20 0x7f99701007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:19.867 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.843+0000 7f99778ff700 1 --2- 172.21.15.39:0/1668835619 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f99700fe4b0 0x7f99700fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:19.867 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.843+0000 7f99778ff700 1 -- 172.21.15.39:0/1668835619 >> 172.21.15.39:0/1668835619 conn(0x7f99700f9770 msgr2=0x7f99701042e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:19.867 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.843+0000 7f99778ff700 1 -- 172.21.15.39:0/1668835619 shutdown_connections 2023-07-21T18:41:19.867 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.843+0000 7f99778ff700 1 -- 172.21.15.39:0/1668835619 wait complete. 2023-07-21T18:41:19.878 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-21T18:36:13.572583+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-21T18:40:02.726735+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-21T18:40:03.990112+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-21T18:41:19.879 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-21T18:41:20.010 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.984+0000 7f373048e700 1 -- 172.21.15.39:0/3712706623 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f37280ffbd0 msgr2=0x7f3728100020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:20.010 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.984+0000 7f373048e700 1 --2- 172.21.15.39:0/3712706623 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f37280ffbd0 0x7f3728100020 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3718009990 tx=0x7f3718009ca0 comp rx=0 tx=0).stop 2023-07-21T18:41:20.010 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.985+0000 7f373048e700 1 -- 172.21.15.39:0/3712706623 shutdown_connections 2023-07-21T18:41:20.010 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.985+0000 7f373048e700 1 --2- 172.21.15.39:0/3712706623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37280fe9e0 0x7f37280fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:20.010 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.985+0000 7f373048e700 1 --2- 172.21.15.39:0/3712706623 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3728100560 0x7f3728106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:20.011 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.985+0000 7f373048e700 1 --2- 172.21.15.39:0/3712706623 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f37280ffbd0 0x7f3728100020 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:20.011 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.985+0000 7f373048e700 1 -- 172.21.15.39:0/3712706623 >> 172.21.15.39:0/3712706623 conn(0x7f37280fa000 msgr2=0x7f37280fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:20.011 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.985+0000 7f373048e700 1 -- 172.21.15.39:0/3712706623 shutdown_connections 2023-07-21T18:41:20.011 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.985+0000 7f373048e700 1 -- 172.21.15.39:0/3712706623 wait complete. 2023-07-21T18:41:20.011 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.985+0000 7f373048e700 1 Processor -- start 2023-07-21T18:41:20.011 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.985+0000 7f373048e700 1 -- start start 2023-07-21T18:41:20.011 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.985+0000 7f373048e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37280fe9e0 0x7f3728194750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:20.011 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.985+0000 7f373048e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37280ffbd0 0x7f3728194c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:20.012 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.985+0000 7f373048e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3728100560 0x7f372819bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:20.012 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.985+0000 7f373048e700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f372819c2d0 con 0x7f3728100560 2023-07-21T18:41:20.012 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.985+0000 7f373048e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f372819c410 con 0x7f37280fe9e0 2023-07-21T18:41:20.012 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.985+0000 7f373048e700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f372819c5a0 con 0x7f37280ffbd0 2023-07-21T18:41:20.012 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.986+0000 7f372ea2b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3728100560 0x7f372819bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:20.012 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.986+0000 7f372ea2b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3728100560 0x7f372819bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56196/0 (socket says 172.21.15.39:56196) 2023-07-21T18:41:20.012 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.986+0000 7f372ea2b700 1 -- 172.21.15.39:0/2142895776 learned_addr learned my addr 172.21.15.39:0/2142895776 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:20.013 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.986+0000 7f372e22a700 1 --2- 172.21.15.39:0/2142895776 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37280fe9e0 0x7f3728194750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:20.013 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.986+0000 7f372da29700 1 --2- 172.21.15.39:0/2142895776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37280ffbd0 0x7f3728194c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:20.013 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.986+0000 7f372ea2b700 1 -- 172.21.15.39:0/2142895776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37280ffbd0 msgr2=0x7f3728194c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:20.013 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.986+0000 7f372ea2b700 1 --2- 172.21.15.39:0/2142895776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37280ffbd0 0x7f3728194c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:20.013 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.986+0000 7f372ea2b700 1 -- 172.21.15.39:0/2142895776 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37280fe9e0 msgr2=0x7f3728194750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:20.013 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.986+0000 7f372ea2b700 1 --2- 172.21.15.39:0/2142895776 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37280fe9e0 0x7f3728194750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:20.013 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.986+0000 7f372ea2b700 1 -- 172.21.15.39:0/2142895776 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37180095f0 con 0x7f3728100560 2023-07-21T18:41:20.013 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.986+0000 7f372da29700 1 --2- 172.21.15.39:0/2142895776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37280ffbd0 0x7f3728194c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:41:20.014 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.986+0000 7f372ea2b700 1 --2- 172.21.15.39:0/2142895776 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3728100560 0x7f372819bd00 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3720007ee0 tx=0x7f372000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:20.014 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.986+0000 7f371f7fe700 1 -- 172.21.15.39:0/2142895776 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3720017070 con 0x7f3728100560 2023-07-21T18:41:20.014 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.986+0000 7f371f7fe700 1 -- 172.21.15.39:0/2142895776 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f372000f040 con 0x7f3728100560 2023-07-21T18:41:20.014 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.986+0000 7f373048e700 1 -- 172.21.15.39:0/2142895776 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f372819c880 con 0x7f3728100560 2023-07-21T18:41:20.014 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.986+0000 7f371f7fe700 1 -- 172.21.15.39:0/2142895776 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3720005740 con 0x7f3728100560 2023-07-21T18:41:20.015 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.986+0000 7f373048e700 1 -- 172.21.15.39:0/2142895776 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f372819cd40 con 0x7f3728100560 2023-07-21T18:41:20.015 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.987+0000 7f373048e700 1 -- 172.21.15.39:0/2142895776 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3728105fc0 con 0x7f3728100560 2023-07-21T18:41:20.015 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.988+0000 7f371f7fe700 1 -- 172.21.15.39:0/2142895776 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f372000dc60 con 0x7f3728100560 2023-07-21T18:41:20.015 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.989+0000 7f371f7fe700 1 --2- 172.21.15.39:0/2142895776 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f37140a11e0 0x7f37140a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:20.015 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.989+0000 7f371f7fe700 1 -- 172.21.15.39:0/2142895776 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f37200cc8c0 con 0x7f3728100560 2023-07-21T18:41:20.016 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.989+0000 7f372e22a700 1 --2- 172.21.15.39:0/2142895776 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f37140a11e0 0x7f37140a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:20.016 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.990+0000 7f372e22a700 1 --2- 172.21.15.39:0/2142895776 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f37140a11e0 0x7f37140a3690 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f37280ffa30 tx=0x7f372400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:20.016 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:19.992+0000 7f371f7fe700 1 -- 172.21.15.39:0/2142895776 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f3720095ee0 con 0x7f3728100560 2023-07-21T18:41:20.188 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:20 smithi039 ceph-mon[96489]: pgmap v287: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-07-21T18:41:20.189 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:20 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/1668835619' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T18:41:20.216 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.193+0000 7f373048e700 1 -- 172.21.15.39:0/2142895776 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f372802ccd0 con 0x7f3728100560 2023-07-21T18:41:20.216 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.193+0000 7f371f7fe700 1 -- 172.21.15.39:0/2142895776 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f372802ccd0 con 0x7f3728100560 2023-07-21T18:41:20.217 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2023-07-21T18:41:20.219 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.195+0000 7f373048e700 1 -- 172.21.15.39:0/2142895776 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f37140a11e0 msgr2=0x7f37140a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:20.219 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.195+0000 7f373048e700 1 --2- 172.21.15.39:0/2142895776 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f37140a11e0 0x7f37140a3690 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f37280ffa30 tx=0x7f372400b410 comp rx=0 tx=0).stop 2023-07-21T18:41:20.219 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.195+0000 7f373048e700 1 -- 172.21.15.39:0/2142895776 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3728100560 msgr2=0x7f372819bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:20.219 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.195+0000 7f373048e700 1 --2- 172.21.15.39:0/2142895776 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3728100560 0x7f372819bd00 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3720007ee0 tx=0x7f372000d3b0 comp rx=0 tx=0).stop 2023-07-21T18:41:20.219 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.195+0000 7f373048e700 1 -- 172.21.15.39:0/2142895776 shutdown_connections 2023-07-21T18:41:20.219 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.195+0000 7f373048e700 1 --2- 172.21.15.39:0/2142895776 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f37140a11e0 0x7f37140a3690 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:20.220 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.195+0000 7f373048e700 1 --2- 172.21.15.39:0/2142895776 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37280fe9e0 0x7f3728194750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:20.220 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.195+0000 7f373048e700 1 --2- 172.21.15.39:0/2142895776 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3728100560 0x7f372819bd00 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:20.220 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.195+0000 7f373048e700 1 --2- 172.21.15.39:0/2142895776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37280ffbd0 0x7f3728194c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:20.220 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.195+0000 7f373048e700 1 -- 172.21.15.39:0/2142895776 >> 172.21.15.39:0/2142895776 conn(0x7f37280fa000 msgr2=0x7f3728103100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:20.220 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.196+0000 7f373048e700 1 -- 172.21.15.39:0/2142895776 shutdown_connections 2023-07-21T18:41:20.220 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.196+0000 7f373048e700 1 -- 172.21.15.39:0/2142895776 wait complete. 2023-07-21T18:41:20.231 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-21T18:41:20.361 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.338+0000 7ff172129700 1 -- 172.21.15.39:0/3494380287 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff16c0ffbd0 msgr2=0x7ff16c100020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:20.361 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.338+0000 7ff172129700 1 --2- 172.21.15.39:0/3494380287 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff16c0ffbd0 0x7ff16c100020 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff15c009990 tx=0x7ff15c009ca0 comp rx=0 tx=0).stop 2023-07-21T18:41:20.361 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.338+0000 7ff172129700 1 -- 172.21.15.39:0/3494380287 shutdown_connections 2023-07-21T18:41:20.362 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.338+0000 7ff172129700 1 --2- 172.21.15.39:0/3494380287 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff16c0fe9e0 0x7ff16c0fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:20.362 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.338+0000 7ff172129700 1 --2- 172.21.15.39:0/3494380287 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff16c100560 0x7ff16c106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:20.362 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.338+0000 7ff172129700 1 --2- 172.21.15.39:0/3494380287 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff16c0ffbd0 0x7ff16c100020 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:20.362 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.338+0000 7ff172129700 1 -- 172.21.15.39:0/3494380287 >> 172.21.15.39:0/3494380287 conn(0x7ff16c0fa000 msgr2=0x7ff16c0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:20.362 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.338+0000 7ff172129700 1 -- 172.21.15.39:0/3494380287 shutdown_connections 2023-07-21T18:41:20.362 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.338+0000 7ff172129700 1 -- 172.21.15.39:0/3494380287 wait complete. 2023-07-21T18:41:20.362 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.338+0000 7ff172129700 1 Processor -- start 2023-07-21T18:41:20.365 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.338+0000 7ff172129700 1 -- start start 2023-07-21T18:41:20.365 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.339+0000 7ff172129700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff16c0fe9e0 0x7ff16c194740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:20.365 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.339+0000 7ff172129700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff16c0ffbd0 0x7ff16c194c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:20.366 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.339+0000 7ff172129700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff16c100560 0x7ff16c19bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:20.366 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.339+0000 7ff172129700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16c19c2c0 con 0x7ff16c0ffbd0 2023-07-21T18:41:20.366 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.339+0000 7ff172129700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16c19c400 con 0x7ff16c100560 2023-07-21T18:41:20.366 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.339+0000 7ff172129700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16c19c590 con 0x7ff16c0fe9e0 2023-07-21T18:41:20.366 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.339+0000 7ff16affd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff16c0ffbd0 0x7ff16c194c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:20.366 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.339+0000 7ff16affd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff16c0ffbd0 0x7ff16c194c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56208/0 (socket says 172.21.15.39:56208) 2023-07-21T18:41:20.366 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.339+0000 7ff16affd700 1 -- 172.21.15.39:0/733087412 learned_addr learned my addr 172.21.15.39:0/733087412 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:20.366 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.339+0000 7ff16b7fe700 1 --2- 172.21.15.39:0/733087412 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff16c0fe9e0 0x7ff16c194740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:20.367 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.339+0000 7ff16affd700 1 -- 172.21.15.39:0/733087412 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff16c0fe9e0 msgr2=0x7ff16c194740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:20.367 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.339+0000 7ff16affd700 1 --2- 172.21.15.39:0/733087412 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff16c0fe9e0 0x7ff16c194740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:20.367 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.339+0000 7ff16affd700 1 -- 172.21.15.39:0/733087412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff16c100560 msgr2=0x7ff16c19bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:20.367 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.339+0000 7ff16affd700 1 --2- 172.21.15.39:0/733087412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff16c100560 0x7ff16c19bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:20.367 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.339+0000 7ff16affd700 1 -- 172.21.15.39:0/733087412 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff15c0095f0 con 0x7ff16c0ffbd0 2023-07-21T18:41:20.367 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.339+0000 7ff16affd700 1 --2- 172.21.15.39:0/733087412 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff16c0ffbd0 0x7ff16c194c80 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff15c004760 tx=0x7ff15c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:20.367 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.339+0000 7ff168ff9700 1 -- 172.21.15.39:0/733087412 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15c01d070 con 0x7ff16c0ffbd0 2023-07-21T18:41:20.367 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.339+0000 7ff168ff9700 1 -- 172.21.15.39:0/733087412 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff15c004200 con 0x7ff16c0ffbd0 2023-07-21T18:41:20.368 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.339+0000 7ff172129700 1 -- 172.21.15.39:0/733087412 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff16c19c810 con 0x7ff16c0ffbd0 2023-07-21T18:41:20.368 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.339+0000 7ff168ff9700 1 -- 172.21.15.39:0/733087412 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15c0049f0 con 0x7ff16c0ffbd0 2023-07-21T18:41:20.368 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.339+0000 7ff172129700 1 -- 172.21.15.39:0/733087412 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff16c19cd10 con 0x7ff16c0ffbd0 2023-07-21T18:41:20.368 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.340+0000 7ff172129700 1 -- 172.21.15.39:0/733087412 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff16c10a1e0 con 0x7ff16c0ffbd0 2023-07-21T18:41:20.368 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.341+0000 7ff168ff9700 1 -- 172.21.15.39:0/733087412 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7ff15c003d40 con 0x7ff16c0ffbd0 2023-07-21T18:41:20.368 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.342+0000 7ff168ff9700 1 --2- 172.21.15.39:0/733087412 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7ff1540a1260 0x7ff1540a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:20.368 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.342+0000 7ff168ff9700 1 -- 172.21.15.39:0/733087412 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8182+0+0 (secure 0 0 0) 0x7ff15c0cdd60 con 0x7ff16c0ffbd0 2023-07-21T18:41:20.369 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.342+0000 7ff16b7fe700 1 --2- 172.21.15.39:0/733087412 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7ff1540a1260 0x7ff1540a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:20.369 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.342+0000 7ff16b7fe700 1 --2- 172.21.15.39:0/733087412 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7ff1540a1260 0x7ff1540a3710 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff160005950 tx=0x7ff1600058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:20.369 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.345+0000 7ff168ff9700 1 -- 172.21.15.39:0/733087412 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7ff15c097410 con 0x7ff16c0ffbd0 2023-07-21T18:41:20.371 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:20 smithi193 ceph-mon[98204]: pgmap v287: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-07-21T18:41:20.371 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:20 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/1668835619' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T18:41:20.389 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:20 smithi155 ceph-mon[97934]: pgmap v287: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-07-21T18:41:20.390 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:20 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/1668835619' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T18:41:20.571 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.548+0000 7ff172129700 1 -- 172.21.15.39:0/733087412 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff16c105910 con 0x7ff16c0ffbd0 2023-07-21T18:41:20.572 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.548+0000 7ff168ff9700 1 -- 172.21.15.39:0/733087412 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v81) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff15c09aa30 con 0x7ff16c0ffbd0 2023-07-21T18:41:20.572 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2023-07-21T18:41:20.574 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.550+0000 7ff172129700 1 -- 172.21.15.39:0/733087412 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7ff1540a1260 msgr2=0x7ff1540a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:20.574 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.550+0000 7ff172129700 1 --2- 172.21.15.39:0/733087412 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7ff1540a1260 0x7ff1540a3710 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff160005950 tx=0x7ff1600058e0 comp rx=0 tx=0).stop 2023-07-21T18:41:20.574 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.550+0000 7ff172129700 1 -- 172.21.15.39:0/733087412 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff16c0ffbd0 msgr2=0x7ff16c194c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:20.574 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.550+0000 7ff172129700 1 --2- 172.21.15.39:0/733087412 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff16c0ffbd0 0x7ff16c194c80 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff15c004760 tx=0x7ff15c009f90 comp rx=0 tx=0).stop 2023-07-21T18:41:20.575 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.550+0000 7ff172129700 1 -- 172.21.15.39:0/733087412 shutdown_connections 2023-07-21T18:41:20.575 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.550+0000 7ff172129700 1 --2- 172.21.15.39:0/733087412 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7ff1540a1260 0x7ff1540a3710 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:20.575 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.550+0000 7ff172129700 1 --2- 172.21.15.39:0/733087412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff16c100560 0x7ff16c19bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:20.575 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.550+0000 7ff172129700 1 --2- 172.21.15.39:0/733087412 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff16c0fe9e0 0x7ff16c194740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:20.575 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.550+0000 7ff172129700 1 --2- 172.21.15.39:0/733087412 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff16c0ffbd0 0x7ff16c194c80 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:20.576 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.550+0000 7ff172129700 1 -- 172.21.15.39:0/733087412 >> 172.21.15.39:0/733087412 conn(0x7ff16c0fa000 msgr2=0x7ff16c0fc280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:20.576 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.550+0000 7ff172129700 1 -- 172.21.15.39:0/733087412 shutdown_connections 2023-07-21T18:41:20.576 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.550+0000 7ff172129700 1 -- 172.21.15.39:0/733087412 wait complete. 2023-07-21T18:41:20.586 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-21T18:41:20.716 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.692+0000 7f78073ef700 1 -- 172.21.15.39:0/2203013136 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78000ffc00 msgr2=0x7f7800100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:20.716 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.692+0000 7f78073ef700 1 --2- 172.21.15.39:0/2203013136 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78000ffc00 0x7f7800100050 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f77fc009990 tx=0x7f77fc009ca0 comp rx=0 tx=0).stop 2023-07-21T18:41:20.716 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.692+0000 7f78073ef700 1 -- 172.21.15.39:0/2203013136 shutdown_connections 2023-07-21T18:41:20.716 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.692+0000 7f78073ef700 1 --2- 172.21.15.39:0/2203013136 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78000fea10 0x7f78000fee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:20.717 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.692+0000 7f78073ef700 1 --2- 172.21.15.39:0/2203013136 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7800100590 0x7f7800106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:20.717 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.692+0000 7f78073ef700 1 --2- 172.21.15.39:0/2203013136 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78000ffc00 0x7f7800100050 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:20.717 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.692+0000 7f78073ef700 1 -- 172.21.15.39:0/2203013136 >> 172.21.15.39:0/2203013136 conn(0x7f78000fa030 msgr2=0x7f78000fc460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:20.717 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.692+0000 7f78073ef700 1 -- 172.21.15.39:0/2203013136 shutdown_connections 2023-07-21T18:41:20.717 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.692+0000 7f78073ef700 1 -- 172.21.15.39:0/2203013136 wait complete. 2023-07-21T18:41:20.717 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.693+0000 7f78073ef700 1 Processor -- start 2023-07-21T18:41:20.717 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.693+0000 7f78073ef700 1 -- start start 2023-07-21T18:41:20.719 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.693+0000 7f78073ef700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78000fea10 0x7f7800193680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:20.719 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.693+0000 7f78073ef700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78000ffc00 0x7f7800193bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:20.719 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.693+0000 7f78073ef700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7800100590 0x7f780019ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:20.720 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.693+0000 7f78073ef700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780019b200 con 0x7f78000ffc00 2023-07-21T18:41:20.720 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.693+0000 7f78073ef700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780019b340 con 0x7f7800100590 2023-07-21T18:41:20.720 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.693+0000 7f78073ef700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780019b4d0 con 0x7f78000fea10 2023-07-21T18:41:20.720 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.693+0000 7f780498a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78000ffc00 0x7f7800193bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:20.720 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.693+0000 7f780498a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78000ffc00 0x7f7800193bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56222/0 (socket says 172.21.15.39:56222) 2023-07-21T18:41:20.720 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.693+0000 7f780498a700 1 -- 172.21.15.39:0/3970044523 learned_addr learned my addr 172.21.15.39:0/3970044523 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:20.721 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.693+0000 7f780518b700 1 --2- 172.21.15.39:0/3970044523 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78000fea10 0x7f7800193680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:20.721 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.694+0000 7f780598c700 1 --2- 172.21.15.39:0/3970044523 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7800100590 0x7f780019ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:20.721 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.694+0000 7f780498a700 1 -- 172.21.15.39:0/3970044523 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78000fea10 msgr2=0x7f7800193680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:20.721 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.694+0000 7f780498a700 1 --2- 172.21.15.39:0/3970044523 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78000fea10 0x7f7800193680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:20.721 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.694+0000 7f780498a700 1 -- 172.21.15.39:0/3970044523 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7800100590 msgr2=0x7f780019ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:20.721 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.694+0000 7f780498a700 1 --2- 172.21.15.39:0/3970044523 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7800100590 0x7f780019ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:20.721 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.694+0000 7f780498a700 1 -- 172.21.15.39:0/3970044523 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77fc0095f0 con 0x7f78000ffc00 2023-07-21T18:41:20.722 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.694+0000 7f780518b700 1 --2- 172.21.15.39:0/3970044523 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78000fea10 0x7f7800193680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:41:20.722 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.694+0000 7f780598c700 1 --2- 172.21.15.39:0/3970044523 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7800100590 0x7f780019ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:41:20.722 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.694+0000 7f780498a700 1 --2- 172.21.15.39:0/3970044523 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78000ffc00 0x7f7800193bc0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f77fc00c010 tx=0x7f77fc005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:20.722 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.694+0000 7f77f67fc700 1 -- 172.21.15.39:0/3970044523 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77fc01d070 con 0x7f78000ffc00 2023-07-21T18:41:20.722 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.694+0000 7f77f67fc700 1 -- 172.21.15.39:0/3970044523 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77fc004ca0 con 0x7f78000ffc00 2023-07-21T18:41:20.722 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.694+0000 7f77f67fc700 1 -- 172.21.15.39:0/3970044523 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77fc0055e0 con 0x7f78000ffc00 2023-07-21T18:41:20.722 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.694+0000 7f78073ef700 1 -- 172.21.15.39:0/3970044523 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f780019b750 con 0x7f78000ffc00 2023-07-21T18:41:20.722 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.694+0000 7f78073ef700 1 -- 172.21.15.39:0/3970044523 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f780019bb10 con 0x7f78000ffc00 2023-07-21T18:41:20.723 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.695+0000 7f78073ef700 1 -- 172.21.15.39:0/3970044523 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7800109200 con 0x7f78000ffc00 2023-07-21T18:41:20.723 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.696+0000 7f77f67fc700 1 -- 172.21.15.39:0/3970044523 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f77fc00fc70 con 0x7f78000ffc00 2023-07-21T18:41:20.723 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.697+0000 7f77f67fc700 1 --2- 172.21.15.39:0/3970044523 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f77ec0a1260 0x7f77ec0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:20.723 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.697+0000 7f77f67fc700 1 -- 172.21.15.39:0/3970044523 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f77fc0ce270 con 0x7f78000ffc00 2023-07-21T18:41:20.724 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.697+0000 7f780518b700 1 --2- 172.21.15.39:0/3970044523 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f77ec0a1260 0x7f77ec0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:20.724 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.697+0000 7f780518b700 1 --2- 172.21.15.39:0/3970044523 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f77ec0a1260 0x7f77ec0a3710 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f78000ffa60 tx=0x7f77f0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:20.724 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.700+0000 7f77f67fc700 1 -- 172.21.15.39:0/3970044523 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f77fc0978c0 con 0x7f78000ffc00 2023-07-21T18:41:20.922 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.899+0000 7f78073ef700 1 -- 172.21.15.39:0/3970044523 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f78001058c0 con 0x7f78000ffc00 2023-07-21T18:41:20.923 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.900+0000 7f77f67fc700 1 -- 172.21.15.39:0/3970044523 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v81) v1 ==== 100+0+11 (secure 0 0 0) 0x7f77fc09aee0 con 0x7f78000ffc00 2023-07-21T18:41:20.923 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 64 2023-07-21T18:41:20.925 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.902+0000 7f78073ef700 1 -- 172.21.15.39:0/3970044523 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f77ec0a1260 msgr2=0x7f77ec0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:20.925 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.902+0000 7f78073ef700 1 --2- 172.21.15.39:0/3970044523 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f77ec0a1260 0x7f77ec0a3710 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f78000ffa60 tx=0x7f77f0009260 comp rx=0 tx=0).stop 2023-07-21T18:41:20.926 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.902+0000 7f78073ef700 1 -- 172.21.15.39:0/3970044523 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78000ffc00 msgr2=0x7f7800193bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:20.926 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.902+0000 7f78073ef700 1 --2- 172.21.15.39:0/3970044523 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78000ffc00 0x7f7800193bc0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f77fc00c010 tx=0x7f77fc005e80 comp rx=0 tx=0).stop 2023-07-21T18:41:20.926 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.902+0000 7f78073ef700 1 -- 172.21.15.39:0/3970044523 shutdown_connections 2023-07-21T18:41:20.926 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.902+0000 7f78073ef700 1 --2- 172.21.15.39:0/3970044523 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f77ec0a1260 0x7f77ec0a3710 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:20.926 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.902+0000 7f78073ef700 1 --2- 172.21.15.39:0/3970044523 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7800100590 0x7f780019ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:20.926 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.902+0000 7f78073ef700 1 --2- 172.21.15.39:0/3970044523 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78000fea10 0x7f7800193680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:20.926 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.902+0000 7f78073ef700 1 --2- 172.21.15.39:0/3970044523 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78000ffc00 0x7f7800193bc0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:20.927 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.902+0000 7f78073ef700 1 -- 172.21.15.39:0/3970044523 >> 172.21.15.39:0/3970044523 conn(0x7f78000fa030 msgr2=0x7f78000fc240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:20.927 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.902+0000 7f78073ef700 1 -- 172.21.15.39:0/3970044523 shutdown_connections 2023-07-21T18:41:20.927 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:20.902+0000 7f78073ef700 1 -- 172.21.15.39:0/3970044523 wait complete. 2023-07-21T18:41:20.937 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T18:41:21.064 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.041+0000 7f72d11a6700 1 -- 172.21.15.39:0/2921750838 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72cc100550 msgr2=0x7f72cc106e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:21.065 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.041+0000 7f72d11a6700 1 --2- 172.21.15.39:0/2921750838 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72cc100550 0x7f72cc106e10 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f72c400b0a0 tx=0x7f72c400b3b0 comp rx=0 tx=0).stop 2023-07-21T18:41:21.065 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.041+0000 7f72d11a6700 1 -- 172.21.15.39:0/2921750838 shutdown_connections 2023-07-21T18:41:21.065 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.041+0000 7f72d11a6700 1 --2- 172.21.15.39:0/2921750838 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72cc0fe9d0 0x7f72cc0fede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:21.065 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.041+0000 7f72d11a6700 1 --2- 172.21.15.39:0/2921750838 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72cc100550 0x7f72cc106e10 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:21.065 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.041+0000 7f72d11a6700 1 --2- 172.21.15.39:0/2921750838 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72cc0ffbc0 0x7f72cc100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:21.066 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.041+0000 7f72d11a6700 1 -- 172.21.15.39:0/2921750838 >> 172.21.15.39:0/2921750838 conn(0x7f72cc0f9fd0 msgr2=0x7f72cc0fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:21.066 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.041+0000 7f72d11a6700 1 -- 172.21.15.39:0/2921750838 shutdown_connections 2023-07-21T18:41:21.066 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.041+0000 7f72d11a6700 1 -- 172.21.15.39:0/2921750838 wait complete. 2023-07-21T18:41:21.068 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.042+0000 7f72d11a6700 1 Processor -- start 2023-07-21T18:41:21.068 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.042+0000 7f72d11a6700 1 -- start start 2023-07-21T18:41:21.069 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.042+0000 7f72d11a6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72cc0fe9d0 0x7f72cc068200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:21.069 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.042+0000 7f72d11a6700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72cc0ffbc0 0x7f72cc19a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:21.069 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.042+0000 7f72d11a6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72cc100550 0x7f72cc19cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:21.069 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.042+0000 7f72d11a6700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72cc19d430 con 0x7f72cc100550 2023-07-21T18:41:21.070 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.042+0000 7f72d11a6700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72cc19d5a0 con 0x7f72cc0fe9d0 2023-07-21T18:41:21.070 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.042+0000 7f72d11a6700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72cc19d760 con 0x7f72cc0ffbc0 2023-07-21T18:41:21.070 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.042+0000 7f72cb59e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72cc100550 0x7f72cc19cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:21.070 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.042+0000 7f72cb59e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72cc100550 0x7f72cc19cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56238/0 (socket says 172.21.15.39:56238) 2023-07-21T18:41:21.070 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.042+0000 7f72cb59e700 1 -- 172.21.15.39:0/3837481129 learned_addr learned my addr 172.21.15.39:0/3837481129 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:21.070 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.042+0000 7f72ca59c700 1 --2- 172.21.15.39:0/3837481129 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72cc0ffbc0 0x7f72cc19a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:21.070 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.042+0000 7f72cad9d700 1 --2- 172.21.15.39:0/3837481129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72cc0fe9d0 0x7f72cc068200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:21.070 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.043+0000 7f72cb59e700 1 -- 172.21.15.39:0/3837481129 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72cc0ffbc0 msgr2=0x7f72cc19a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:21.071 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.043+0000 7f72cb59e700 1 --2- 172.21.15.39:0/3837481129 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72cc0ffbc0 0x7f72cc19a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:21.071 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.043+0000 7f72cb59e700 1 -- 172.21.15.39:0/3837481129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72cc0fe9d0 msgr2=0x7f72cc068200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:21.071 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.043+0000 7f72cb59e700 1 --2- 172.21.15.39:0/3837481129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72cc0fe9d0 0x7f72cc068200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:21.071 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.043+0000 7f72cb59e700 1 -- 172.21.15.39:0/3837481129 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72c4009d00 con 0x7f72cc100550 2023-07-21T18:41:21.071 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.043+0000 7f72ca59c700 1 --2- 172.21.15.39:0/3837481129 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72cc0ffbc0 0x7f72cc19a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:41:21.071 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.043+0000 7f72cad9d700 1 --2- 172.21.15.39:0/3837481129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72cc0fe9d0 0x7f72cc068200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:41:21.071 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.043+0000 7f72cb59e700 1 --2- 172.21.15.39:0/3837481129 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72cc100550 0x7f72cc19cc60 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f72c400d010 tx=0x7f72c4004d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:21.072 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.043+0000 7f72b3fff700 1 -- 172.21.15.39:0/3837481129 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c400e040 con 0x7f72cc100550 2023-07-21T18:41:21.072 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.043+0000 7f72b3fff700 1 -- 172.21.15.39:0/3837481129 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f72c4009440 con 0x7f72cc100550 2023-07-21T18:41:21.072 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.043+0000 7f72d11a6700 1 -- 172.21.15.39:0/3837481129 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72cc19d9e0 con 0x7f72cc100550 2023-07-21T18:41:21.072 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.043+0000 7f72b3fff700 1 -- 172.21.15.39:0/3837481129 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c4004280 con 0x7f72cc100550 2023-07-21T18:41:21.072 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.043+0000 7f72d11a6700 1 -- 172.21.15.39:0/3837481129 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72cc19de60 con 0x7f72cc100550 2023-07-21T18:41:21.072 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.044+0000 7f72d11a6700 1 -- 172.21.15.39:0/3837481129 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72cc105ea0 con 0x7f72cc100550 2023-07-21T18:41:21.073 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.045+0000 7f72b3fff700 1 -- 172.21.15.39:0/3837481129 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f72c401d720 con 0x7f72cc100550 2023-07-21T18:41:21.073 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.046+0000 7f72b3fff700 1 --2- 172.21.15.39:0/3837481129 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f72b40a1220 0x7f72b40a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:21.074 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.046+0000 7f72cad9d700 1 --2- 172.21.15.39:0/3837481129 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f72b40a1220 0x7f72b40a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:21.074 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.046+0000 7f72b3fff700 1 -- 172.21.15.39:0/3837481129 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f72c401de40 con 0x7f72cc100550 2023-07-21T18:41:21.074 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.046+0000 7f72cad9d700 1 --2- 172.21.15.39:0/3837481129 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f72b40a1220 0x7f72b40a36d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f72bc005fd0 tx=0x7f72bc005f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:21.074 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.050+0000 7f72b3fff700 1 -- 172.21.15.39:0/3837481129 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f72c4098170 con 0x7f72cc100550 2023-07-21T18:41:21.188 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:21 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/2142895776' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T18:41:21.189 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:21 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/733087412' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T18:41:21.189 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:21 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/3970044523' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T18:41:21.320 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.297+0000 7f72d11a6700 1 -- 172.21.15.39:0/3837481129 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f72cc02c850 con 0x7f72cc100550 2023-07-21T18:41:21.321 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.298+0000 7f72b3fff700 1 -- 172.21.15.39:0/3837481129 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10887 (secure 0 0 0) 0x7f72c4021020 con 0x7f72cc100550 2023-07-21T18:41:21.321 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:41:21.321 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/147318681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":147318681},{"type":"v1","addr":"172.21.15.39:6835","nonce":147318681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/1062891760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1062891760},{"type":"v1","addr":"172.21.15.39:6837","nonce":1062891760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T18:40:04.594553+0000","modified":"2023-07-21T18:40:11.989039+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24536,"mds_1":24500,"mds_2":24487,"mds_3":24464,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/3184273349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":3184273349},{"type":"v1","addr":"172.21.15.39:6839","nonce":3184273349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6841/640981220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":640981220},{"type":"v1","addr":"172.21.15.39:6841","nonce":640981220}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3106166804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3106166804},{"type":"v1","addr":"172.21.15.193:6833","nonce":3106166804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24437":{"gid":24437,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/1034755277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1034755277},{"type":"v1","addr":"172.21.15.155:6833","nonce":1034755277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/381560657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":381560657},{"type":"v1","addr":"172.21.15.193:6835","nonce":381560657}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6837/2626761843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2626761843},{"type":"v1","addr":"172.21.15.193:6837","nonce":2626761843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24464":{"gid":24464,"name":"e","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.155:6835/3854856286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3854856286},{"type":"v1","addr":"172.21.15.155:6835","nonce":3854856286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.193:6839/3148488639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3148488639},{"type":"v1","addr":"172.21.15.193:6839","nonce":3148488639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.155:6837/947670056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":947670056},{"type":"v1","addr":"172.21.15.155:6837","nonce":947670056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6839/431690342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":431690342},{"type":"v1","addr":"172.21.15.155:6839","nonce":431690342}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T18:41:21.323 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.300+0000 7f72d11a6700 1 -- 172.21.15.39:0/3837481129 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f72b40a1220 msgr2=0x7f72b40a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:21.323 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.300+0000 7f72d11a6700 1 --2- 172.21.15.39:0/3837481129 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f72b40a1220 0x7f72b40a36d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f72bc005fd0 tx=0x7f72bc005f00 comp rx=0 tx=0).stop 2023-07-21T18:41:21.324 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.300+0000 7f72d11a6700 1 -- 172.21.15.39:0/3837481129 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72cc100550 msgr2=0x7f72cc19cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:21.324 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.300+0000 7f72d11a6700 1 --2- 172.21.15.39:0/3837481129 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72cc100550 0x7f72cc19cc60 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f72c400d010 tx=0x7f72c4004d30 comp rx=0 tx=0).stop 2023-07-21T18:41:21.324 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.300+0000 7f72d11a6700 1 -- 172.21.15.39:0/3837481129 shutdown_connections 2023-07-21T18:41:21.324 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.300+0000 7f72d11a6700 1 --2- 172.21.15.39:0/3837481129 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f72b40a1220 0x7f72b40a36d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:21.324 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.300+0000 7f72d11a6700 1 --2- 172.21.15.39:0/3837481129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72cc0fe9d0 0x7f72cc068200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:21.325 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.300+0000 7f72d11a6700 1 --2- 172.21.15.39:0/3837481129 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72cc100550 0x7f72cc19cc60 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:21.325 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.300+0000 7f72d11a6700 1 --2- 172.21.15.39:0/3837481129 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72cc0ffbc0 0x7f72cc19a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:21.325 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.300+0000 7f72d11a6700 1 -- 172.21.15.39:0/3837481129 >> 172.21.15.39:0/3837481129 conn(0x7f72cc0f9fd0 msgr2=0x7f72cc102fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:21.325 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.300+0000 7f72d11a6700 1 -- 172.21.15.39:0/3837481129 shutdown_connections 2023-07-21T18:41:21.325 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.300+0000 7f72d11a6700 1 -- 172.21.15.39:0/3837481129 wait complete. 2023-07-21T18:41:21.325 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 26 2023-07-21T18:41:21.336 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T18:41:21.371 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:21 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/2142895776' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T18:41:21.371 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:21 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/733087412' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T18:41:21.371 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:21 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/3970044523' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T18:41:21.389 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:21 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/2142895776' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T18:41:21.390 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:21 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/733087412' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T18:41:21.390 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:21 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/3970044523' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T18:41:21.461 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.438+0000 7f9e527cb700 1 -- 172.21.15.39:0/415114661 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e4c100550 msgr2=0x7f9e4c106e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:21.461 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.438+0000 7f9e527cb700 1 --2- 172.21.15.39:0/415114661 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e4c100550 0x7f9e4c106e10 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9e4400b0a0 tx=0x7f9e4400b3b0 comp rx=0 tx=0).stop 2023-07-21T18:41:21.462 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.438+0000 7f9e527cb700 1 -- 172.21.15.39:0/415114661 shutdown_connections 2023-07-21T18:41:21.462 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.438+0000 7f9e527cb700 1 --2- 172.21.15.39:0/415114661 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9e4c0fe9d0 0x7f9e4c0fede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:21.462 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.438+0000 7f9e527cb700 1 --2- 172.21.15.39:0/415114661 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e4c100550 0x7f9e4c106e10 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:21.462 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.438+0000 7f9e527cb700 1 --2- 172.21.15.39:0/415114661 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e4c0ffbc0 0x7f9e4c100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:21.462 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.438+0000 7f9e527cb700 1 -- 172.21.15.39:0/415114661 >> 172.21.15.39:0/415114661 conn(0x7f9e4c0f9fd0 msgr2=0x7f9e4c0fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:21.463 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.438+0000 7f9e527cb700 1 -- 172.21.15.39:0/415114661 shutdown_connections 2023-07-21T18:41:21.463 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.438+0000 7f9e527cb700 1 -- 172.21.15.39:0/415114661 wait complete. 2023-07-21T18:41:21.464 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.439+0000 7f9e527cb700 1 Processor -- start 2023-07-21T18:41:21.464 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.439+0000 7f9e527cb700 1 -- start start 2023-07-21T18:41:21.464 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.439+0000 7f9e527cb700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e4c0fe9d0 0x7f9e4c1979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:21.465 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.439+0000 7f9e527cb700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e4c0ffbc0 0x7f9e4c197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:21.465 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.439+0000 7f9e527cb700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9e4c100550 0x7f9e4c19efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:21.465 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.439+0000 7f9e527cb700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4c19f570 con 0x7f9e4c0fe9d0 2023-07-21T18:41:21.465 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.439+0000 7f9e527cb700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4c19f6b0 con 0x7f9e4c100550 2023-07-21T18:41:21.465 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.439+0000 7f9e527cb700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4c19f840 con 0x7f9e4c0ffbc0 2023-07-21T18:41:21.465 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.439+0000 7f9e4bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e4c0fe9d0 0x7f9e4c1979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:21.465 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.439+0000 7f9e4bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e4c0fe9d0 0x7f9e4c1979f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56248/0 (socket says 172.21.15.39:56248) 2023-07-21T18:41:21.466 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.439+0000 7f9e4bfff700 1 -- 172.21.15.39:0/1324425699 learned_addr learned my addr 172.21.15.39:0/1324425699 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:21.466 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.439+0000 7f9e4bfff700 1 -- 172.21.15.39:0/1324425699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e4c0ffbc0 msgr2=0x7f9e4c197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:21.466 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.439+0000 7f9e4bfff700 1 --2- 172.21.15.39:0/1324425699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e4c0ffbc0 0x7f9e4c197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:21.466 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.439+0000 7f9e4bfff700 1 -- 172.21.15.39:0/1324425699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9e4c100550 msgr2=0x7f9e4c19efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:21.466 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.439+0000 7f9e4bfff700 1 --2- 172.21.15.39:0/1324425699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9e4c100550 0x7f9e4c19efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:21.466 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.439+0000 7f9e4bfff700 1 -- 172.21.15.39:0/1324425699 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e44009d00 con 0x7f9e4c0fe9d0 2023-07-21T18:41:21.466 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.440+0000 7f9e4bfff700 1 --2- 172.21.15.39:0/1324425699 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e4c0fe9d0 0x7f9e4c1979f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c00c990 tx=0x7f9e3c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:21.466 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.440+0000 7f9e497fa700 1 -- 172.21.15.39:0/1324425699 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c007b00 con 0x7f9e4c0fe9d0 2023-07-21T18:41:21.467 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.440+0000 7f9e497fa700 1 -- 172.21.15.39:0/1324425699 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e3c007c60 con 0x7f9e4c0fe9d0 2023-07-21T18:41:21.467 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.440+0000 7f9e527cb700 1 -- 172.21.15.39:0/1324425699 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e4c19fb20 con 0x7f9e4c0fe9d0 2023-07-21T18:41:21.467 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.440+0000 7f9e497fa700 1 -- 172.21.15.39:0/1324425699 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c018660 con 0x7f9e4c0fe9d0 2023-07-21T18:41:21.467 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.440+0000 7f9e527cb700 1 -- 172.21.15.39:0/1324425699 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e4c1a0030 con 0x7f9e4c0fe9d0 2023-07-21T18:41:21.467 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.441+0000 7f9e527cb700 1 -- 172.21.15.39:0/1324425699 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e4c105720 con 0x7f9e4c0fe9d0 2023-07-21T18:41:21.468 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.442+0000 7f9e497fa700 1 -- 172.21.15.39:0/1324425699 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f9e3c004020 con 0x7f9e4c0fe9d0 2023-07-21T18:41:21.468 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.442+0000 7f9e497fa700 1 --2- 172.21.15.39:0/1324425699 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f9e340a1210 0x7f9e340a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:21.468 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.443+0000 7f9e4b7fe700 1 --2- 172.21.15.39:0/1324425699 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f9e340a1210 0x7f9e340a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:21.468 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.443+0000 7f9e497fa700 1 -- 172.21.15.39:0/1324425699 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f9e3c0ccb50 con 0x7f9e4c0fe9d0 2023-07-21T18:41:21.468 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.443+0000 7f9e4b7fe700 1 --2- 172.21.15.39:0/1324425699 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f9e340a1210 0x7f9e340a36c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9e40005950 tx=0x7f9e400058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:21.469 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.446+0000 7f9e497fa700 1 -- 172.21.15.39:0/1324425699 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f9e3c096200 con 0x7f9e4c0fe9d0 2023-07-21T18:41:21.670 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.647+0000 7f9e527cb700 1 -- 172.21.15.39:0/1324425699 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9e4c02cb00 con 0x7f9e4c0fe9d0 2023-07-21T18:41:21.671 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.648+0000 7f9e497fa700 1 -- 172.21.15.39:0/1324425699 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+22447 (secure 0 0 0) 0x7f9e3c099820 con 0x7f9e4c0fe9d0 2023-07-21T18:41:21.672 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:41:21.672 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":81,"fsid":"eaf8e25a-27f4-11ee-9b34-001a4aab830c","created":"2023-07-21T18:33:06.504979+0000","modified":"2023-07-21T18:41:08.140284+0000","last_up_change":"2023-07-21T18:39:23.531009+0000","last_in_change":"2023-07-21T18:39:05.993024+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T18:36:13.572583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T18:40:02.726735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T18:40:03.990112+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03502c07-2d0f-4723-8d0b-d34c9f5b91bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6803","nonce":860193857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6805","nonce":860193857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6809","nonce":860193857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6807","nonce":860193857}]},"public_addr":"172.21.15.39:6803/860193857","cluster_addr":"172.21.15.39:6805/860193857","heartbeat_back_addr":"172.21.15.39:6809/860193857","heartbeat_front_addr":"172.21.15.39:6807/860193857","state":["exists","up"]},{"osd":1,"uuid":"0831a0c0-a1fa-4d30-9a28-513223aa69eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6801","nonce":3246721764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6803","nonce":3246721764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6807","nonce":3246721764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6805","nonce":3246721764}]},"public_addr":"172.21.15.155:6801/3246721764","cluster_addr":"172.21.15.155:6803/3246721764","heartbeat_back_addr":"172.21.15.155:6807/3246721764","heartbeat_front_addr":"172.21.15.155:6805/3246721764","state":["exists","up"]},{"osd":2,"uuid":"22df0003-bde9-48e4-b47a-cb8eafbab0db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6801","nonce":2563594449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6803","nonce":2563594449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6807","nonce":2563594449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6805","nonce":2563594449}]},"public_addr":"172.21.15.193:6801/2563594449","cluster_addr":"172.21.15.193:6803/2563594449","heartbeat_back_addr":"172.21.15.193:6807/2563594449","heartbeat_front_addr":"172.21.15.193:6805/2563594449","state":["exists","up"]},{"osd":3,"uuid":"54562558-a2e8-45dd-bc95-561569b9ae6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6811","nonce":863708386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6813","nonce":863708386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6817","nonce":863708386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6815","nonce":863708386}]},"public_addr":"172.21.15.39:6811/863708386","cluster_addr":"172.21.15.39:6813/863708386","heartbeat_back_addr":"172.21.15.39:6817/863708386","heartbeat_front_addr":"172.21.15.39:6815/863708386","state":["exists","up"]},{"osd":4,"uuid":"491118e7-ae1e-495c-9eac-ef1cdcae9600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6809","nonce":3361780673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6811","nonce":3361780673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6815","nonce":3361780673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6813","nonce":3361780673}]},"public_addr":"172.21.15.155:6809/3361780673","cluster_addr":"172.21.15.155:6811/3361780673","heartbeat_back_addr":"172.21.15.155:6815/3361780673","heartbeat_front_addr":"172.21.15.155:6813/3361780673","state":["exists","up"]},{"osd":5,"uuid":"d267bc2c-42b3-4d96-b2d1-b186dc7e6b18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6809","nonce":2376867721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6811","nonce":2376867721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6815","nonce":2376867721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6813","nonce":2376867721}]},"public_addr":"172.21.15.193:6809/2376867721","cluster_addr":"172.21.15.193:6811/2376867721","heartbeat_back_addr":"172.21.15.193:6815/2376867721","heartbeat_front_addr":"172.21.15.193:6813/2376867721","state":["exists","up"]},{"osd":6,"uuid":"beec8088-970a-473e-bd67-6eee7cbdef24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6819","nonce":2417770529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6821","nonce":2417770529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6825","nonce":2417770529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6823","nonce":2417770529}]},"public_addr":"172.21.15.39:6819/2417770529","cluster_addr":"172.21.15.39:6821/2417770529","heartbeat_back_addr":"172.21.15.39:6825/2417770529","heartbeat_front_addr":"172.21.15.39:6823/2417770529","state":["exists","up"]},{"osd":7,"uuid":"5a88ce30-1835-4254-90cc-804ed5cf1066","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6817","nonce":2542002111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6819","nonce":2542002111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6823","nonce":2542002111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6821","nonce":2542002111}]},"public_addr":"172.21.15.155:6817/2542002111","cluster_addr":"172.21.15.155:6819/2542002111","heartbeat_back_addr":"172.21.15.155:6823/2542002111","heartbeat_front_addr":"172.21.15.155:6821/2542002111","state":["exists","up"]},{"osd":8,"uuid":"d964a183-3021-4ace-9a0c-aad00f2570e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6817","nonce":2985835085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6819","nonce":2985835085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6823","nonce":2985835085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6821","nonce":2985835085}]},"public_addr":"172.21.15.193:6817/2985835085","cluster_addr":"172.21.15.193:6819/2985835085","heartbeat_back_addr":"172.21.15.193:6823/2985835085","heartbeat_front_addr":"172.21.15.193:6821/2985835085","state":["exists","up"]},{"osd":9,"uuid":"9725ea5c-49a3-4c99-bac4-e95069100b1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6827","nonce":1946493651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6829","nonce":1946493651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6833","nonce":1946493651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6831","nonce":1946493651}]},"public_addr":"172.21.15.39:6827/1946493651","cluster_addr":"172.21.15.39:6829/1946493651","heartbeat_back_addr":"172.21.15.39:6833/1946493651","heartbeat_front_addr":"172.21.15.39:6831/1946493651","state":["exists","up"]},{"osd":10,"uuid":"1fd60ccb-92db-45f0-a580-2003db0976e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6825","nonce":1234615022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6827","nonce":1234615022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6831","nonce":1234615022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6829","nonce":1234615022}]},"public_addr":"172.21.15.155:6825/1234615022","cluster_addr":"172.21.15.155:6827/1234615022","heartbeat_back_addr":"172.21.15.155:6831/1234615022","heartbeat_front_addr":"172.21.15.155:6829/1234615022","state":["exists","up"]},{"osd":11,"uuid":"efb31ea1-4775-4e5f-8740-728d6f4aeea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6825","nonce":215104663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6827","nonce":215104663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6831","nonce":215104663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6829","nonce":215104663}]},"public_addr":"172.21.15.193:6825/215104663","cluster_addr":"172.21.15.193:6827/215104663","heartbeat_back_addr":"172.21.15.193:6831/215104663","heartbeat_front_addr":"172.21.15.193:6829/215104663","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:35:30.211698+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:35:50.883846+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:36:11.124168+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:36:32.711905+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:36:54.442007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:37:14.843526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:37:36.587354+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:37:57.882374+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:38:18.729407+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:38:40.139231+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:39:01.290862+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:39:21.767537+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:0/2902600414":"2023-07-22T18:34:07.039823+0000","172.21.15.39:6801/2893866425":"2023-07-22T18:34:07.039823+0000","172.21.15.39:0/19717002":"2023-07-22T18:34:07.039823+0000","172.21.15.39:6800/2893866425":"2023-07-22T18:34:07.039823+0000","172.21.15.39:6800/889516631":"2023-07-22T18:33:40.852706+0000","172.21.15.39:6801/889516631":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/3636510379":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/1275748831":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/3819152602":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/1127767082":"2023-07-22T18:34:07.039823+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T18:41:21.674 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.651+0000 7f9e527cb700 1 -- 172.21.15.39:0/1324425699 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f9e340a1210 msgr2=0x7f9e340a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:21.674 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.651+0000 7f9e527cb700 1 --2- 172.21.15.39:0/1324425699 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f9e340a1210 0x7f9e340a36c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9e40005950 tx=0x7f9e400058e0 comp rx=0 tx=0).stop 2023-07-21T18:41:21.674 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.651+0000 7f9e527cb700 1 -- 172.21.15.39:0/1324425699 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e4c0fe9d0 msgr2=0x7f9e4c1979f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:21.674 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.651+0000 7f9e527cb700 1 --2- 172.21.15.39:0/1324425699 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e4c0fe9d0 0x7f9e4c1979f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c00c990 tx=0x7f9e3c00cd50 comp rx=0 tx=0).stop 2023-07-21T18:41:21.674 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.651+0000 7f9e527cb700 1 -- 172.21.15.39:0/1324425699 shutdown_connections 2023-07-21T18:41:21.674 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.651+0000 7f9e527cb700 1 --2- 172.21.15.39:0/1324425699 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f9e340a1210 0x7f9e340a36c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:21.675 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.651+0000 7f9e527cb700 1 --2- 172.21.15.39:0/1324425699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9e4c100550 0x7f9e4c19efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:21.675 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.651+0000 7f9e527cb700 1 --2- 172.21.15.39:0/1324425699 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e4c0fe9d0 0x7f9e4c1979f0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:21.675 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.651+0000 7f9e527cb700 1 --2- 172.21.15.39:0/1324425699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e4c0ffbc0 0x7f9e4c197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:21.675 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.651+0000 7f9e527cb700 1 -- 172.21.15.39:0/1324425699 >> 172.21.15.39:0/1324425699 conn(0x7f9e4c0f9fd0 msgr2=0x7f9e4c0fc070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:21.675 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.651+0000 7f9e527cb700 1 -- 172.21.15.39:0/1324425699 shutdown_connections 2023-07-21T18:41:21.675 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.651+0000 7f9e527cb700 1 -- 172.21.15.39:0/1324425699 wait complete. 2023-07-21T18:41:21.686 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-21T18:36:13.572583+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-21T18:40:02.726735+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-21T18:40:03.990112+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-21T18:41:21.687 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-21T18:41:21.816 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.793+0000 7f60415e8700 1 -- 172.21.15.39:0/983205319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f603c0fea00 msgr2=0x7f603c0fee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:21.817 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.793+0000 7f60415e8700 1 --2- 172.21.15.39:0/983205319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f603c0fea00 0x7f603c0fee10 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f602c009990 tx=0x7f602c009ca0 comp rx=0 tx=0).stop 2023-07-21T18:41:21.817 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.793+0000 7f60415e8700 1 -- 172.21.15.39:0/983205319 shutdown_connections 2023-07-21T18:41:21.817 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.793+0000 7f60415e8700 1 --2- 172.21.15.39:0/983205319 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f603c100580 0x7f603c106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:21.817 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.793+0000 7f60415e8700 1 --2- 172.21.15.39:0/983205319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f603c0fea00 0x7f603c0fee10 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:21.817 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.793+0000 7f60415e8700 1 --2- 172.21.15.39:0/983205319 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f603c0ffbf0 0x7f603c100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:21.817 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.793+0000 7f60415e8700 1 -- 172.21.15.39:0/983205319 >> 172.21.15.39:0/983205319 conn(0x7f603c0fa060 msgr2=0x7f603c0fc470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:21.817 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.793+0000 7f60415e8700 1 -- 172.21.15.39:0/983205319 shutdown_connections 2023-07-21T18:41:21.818 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.793+0000 7f60415e8700 1 -- 172.21.15.39:0/983205319 wait complete. 2023-07-21T18:41:21.819 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.794+0000 7f60415e8700 1 Processor -- start 2023-07-21T18:41:21.820 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.794+0000 7f60415e8700 1 -- start start 2023-07-21T18:41:21.820 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.794+0000 7f60415e8700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f603c0fea00 0x7f603c0693f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:21.820 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.794+0000 7f60415e8700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f603c0ffbf0 0x7f603c19a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:21.820 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.794+0000 7f60415e8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f603c100580 0x7f603c19cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:21.820 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.794+0000 7f60415e8700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f603c19d320 con 0x7f603c100580 2023-07-21T18:41:21.820 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.794+0000 7f60415e8700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f603c19d490 con 0x7f603c0ffbf0 2023-07-21T18:41:21.820 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.794+0000 7f60415e8700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f603c19d650 con 0x7f603c0fea00 2023-07-21T18:41:21.821 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.794+0000 7f603b7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f603c100580 0x7f603c19cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:21.821 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.794+0000 7f603b7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f603c100580 0x7f603c19cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56264/0 (socket says 172.21.15.39:56264) 2023-07-21T18:41:21.821 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.794+0000 7f603b7fe700 1 -- 172.21.15.39:0/3322256208 learned_addr learned my addr 172.21.15.39:0/3322256208 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:21.821 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.795+0000 7f603a7fc700 1 --2- 172.21.15.39:0/3322256208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f603c0ffbf0 0x7f603c19a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:21.821 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.795+0000 7f603affd700 1 --2- 172.21.15.39:0/3322256208 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f603c0fea00 0x7f603c0693f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:21.821 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.795+0000 7f603b7fe700 1 -- 172.21.15.39:0/3322256208 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f603c0fea00 msgr2=0x7f603c0693f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:21.821 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.795+0000 7f603b7fe700 1 --2- 172.21.15.39:0/3322256208 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f603c0fea00 0x7f603c0693f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:21.821 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.795+0000 7f603b7fe700 1 -- 172.21.15.39:0/3322256208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f603c0ffbf0 msgr2=0x7f603c19a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:21.822 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.795+0000 7f603b7fe700 1 --2- 172.21.15.39:0/3322256208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f603c0ffbf0 0x7f603c19a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:21.822 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.795+0000 7f603b7fe700 1 -- 172.21.15.39:0/3322256208 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f602c0095f0 con 0x7f603c100580 2023-07-21T18:41:21.822 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.795+0000 7f603affd700 1 --2- 172.21.15.39:0/3322256208 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f603c0fea00 0x7f603c0693f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:41:21.822 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.795+0000 7f603b7fe700 1 --2- 172.21.15.39:0/3322256208 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f603c100580 0x7f603c19cb50 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6034007d70 tx=0x7f603400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:21.822 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.795+0000 7f6023fff700 1 -- 172.21.15.39:0/3322256208 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603400ad40 con 0x7f603c100580 2023-07-21T18:41:21.822 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.795+0000 7f6023fff700 1 -- 172.21.15.39:0/3322256208 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60340128e0 con 0x7f603c100580 2023-07-21T18:41:21.822 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.795+0000 7f6023fff700 1 -- 172.21.15.39:0/3322256208 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6034011780 con 0x7f603c100580 2023-07-21T18:41:21.822 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.795+0000 7f60415e8700 1 -- 172.21.15.39:0/3322256208 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f603c19d930 con 0x7f603c100580 2023-07-21T18:41:21.823 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.795+0000 7f60415e8700 1 -- 172.21.15.39:0/3322256208 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f603c19ddf0 con 0x7f603c100580 2023-07-21T18:41:21.823 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.796+0000 7f60415e8700 1 -- 172.21.15.39:0/3322256208 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f603c1058e0 con 0x7f603c100580 2023-07-21T18:41:21.823 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.797+0000 7f6023fff700 1 -- 172.21.15.39:0/3322256208 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f6034012a50 con 0x7f603c100580 2023-07-21T18:41:21.824 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.798+0000 7f6023fff700 1 --2- 172.21.15.39:0/3322256208 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f60240a1260 0x7f60240a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:21.824 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.798+0000 7f603affd700 1 --2- 172.21.15.39:0/3322256208 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f60240a1260 0x7f60240a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:21.824 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.798+0000 7f6023fff700 1 -- 172.21.15.39:0/3322256208 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f60340ccfa0 con 0x7f603c100580 2023-07-21T18:41:21.824 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.798+0000 7f603affd700 1 --2- 172.21.15.39:0/3322256208 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f60240a1260 0x7f60240a3710 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f602c000c00 tx=0x7f602c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:21.824 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:21.802+0000 7f6023fff700 1 -- 172.21.15.39:0/3322256208 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6034096770 con 0x7f603c100580 2023-07-21T18:41:22.023 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.000+0000 7f60415e8700 1 -- 172.21.15.39:0/3322256208 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f603c02c9e0 con 0x7f603c100580 2023-07-21T18:41:22.024 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.000+0000 7f6023fff700 1 -- 172.21.15.39:0/3322256208 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f60340160a0 con 0x7f603c100580 2023-07-21T18:41:22.024 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2023-07-21T18:41:22.025 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.002+0000 7f60415e8700 1 -- 172.21.15.39:0/3322256208 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f60240a1260 msgr2=0x7f60240a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:22.026 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.002+0000 7f60415e8700 1 --2- 172.21.15.39:0/3322256208 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f60240a1260 0x7f60240a3710 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f602c000c00 tx=0x7f602c005fb0 comp rx=0 tx=0).stop 2023-07-21T18:41:22.026 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.002+0000 7f60415e8700 1 -- 172.21.15.39:0/3322256208 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f603c100580 msgr2=0x7f603c19cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:22.026 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.002+0000 7f60415e8700 1 --2- 172.21.15.39:0/3322256208 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f603c100580 0x7f603c19cb50 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6034007d70 tx=0x7f603400a340 comp rx=0 tx=0).stop 2023-07-21T18:41:22.026 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.003+0000 7f60415e8700 1 -- 172.21.15.39:0/3322256208 shutdown_connections 2023-07-21T18:41:22.026 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.003+0000 7f60415e8700 1 --2- 172.21.15.39:0/3322256208 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f60240a1260 0x7f60240a3710 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:22.026 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.003+0000 7f60415e8700 1 --2- 172.21.15.39:0/3322256208 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f603c100580 0x7f603c19cb50 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:22.026 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.003+0000 7f60415e8700 1 --2- 172.21.15.39:0/3322256208 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f603c0fea00 0x7f603c0693f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:22.027 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.003+0000 7f60415e8700 1 --2- 172.21.15.39:0/3322256208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f603c0ffbf0 0x7f603c19a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:22.027 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.003+0000 7f60415e8700 1 -- 172.21.15.39:0/3322256208 >> 172.21.15.39:0/3322256208 conn(0x7f603c0fa060 msgr2=0x7f603c0fd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:22.027 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.003+0000 7f60415e8700 1 -- 172.21.15.39:0/3322256208 shutdown_connections 2023-07-21T18:41:22.027 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.003+0000 7f60415e8700 1 -- 172.21.15.39:0/3322256208 wait complete. 2023-07-21T18:41:22.038 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-21T18:41:22.168 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.145+0000 7f427e2e9700 1 -- 172.21.15.39:0/2466433184 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4278100560 msgr2=0x7f4278106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:22.169 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.145+0000 7f427e2e9700 1 --2- 172.21.15.39:0/2466433184 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4278100560 0x7f4278106e20 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f427000b0a0 tx=0x7f427000b3b0 comp rx=0 tx=0).stop 2023-07-21T18:41:22.169 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.145+0000 7f427e2e9700 1 -- 172.21.15.39:0/2466433184 shutdown_connections 2023-07-21T18:41:22.169 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.145+0000 7f427e2e9700 1 --2- 172.21.15.39:0/2466433184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f42780fe9e0 0x7f42780fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:22.169 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.145+0000 7f427e2e9700 1 --2- 172.21.15.39:0/2466433184 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4278100560 0x7f4278106e20 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:22.169 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.145+0000 7f427e2e9700 1 --2- 172.21.15.39:0/2466433184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f42780ffbd0 0x7f4278100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:22.170 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.145+0000 7f427e2e9700 1 -- 172.21.15.39:0/2466433184 >> 172.21.15.39:0/2466433184 conn(0x7f42780fa000 msgr2=0x7f42780fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:22.170 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.145+0000 7f427e2e9700 1 -- 172.21.15.39:0/2466433184 shutdown_connections 2023-07-21T18:41:22.170 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.145+0000 7f427e2e9700 1 -- 172.21.15.39:0/2466433184 wait complete. 2023-07-21T18:41:22.170 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.146+0000 7f427e2e9700 1 Processor -- start 2023-07-21T18:41:22.170 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.146+0000 7f427e2e9700 1 -- start start 2023-07-21T18:41:22.172 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.146+0000 7f427e2e9700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f42780fe9e0 0x7f4278195980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:22.172 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.146+0000 7f427e2e9700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f42780ffbd0 0x7f4278195ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:22.172 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.146+0000 7f427e2e9700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4278100560 0x7f427819cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:22.172 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.146+0000 7f427e2e9700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f427819d500 con 0x7f4278100560 2023-07-21T18:41:22.173 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.146+0000 7f427e2e9700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f427819d640 con 0x7f42780ffbd0 2023-07-21T18:41:22.173 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.146+0000 7f427e2e9700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f427819d7d0 con 0x7f42780fe9e0 2023-07-21T18:41:22.173 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.146+0000 7f427c886700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4278100560 0x7f427819cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:22.173 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.146+0000 7f427c886700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4278100560 0x7f427819cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56278/0 (socket says 172.21.15.39:56278) 2023-07-21T18:41:22.173 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.146+0000 7f427c886700 1 -- 172.21.15.39:0/3439362154 learned_addr learned my addr 172.21.15.39:0/3439362154 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:22.173 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.146+0000 7f42777fe700 1 --2- 172.21.15.39:0/3439362154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f42780ffbd0 0x7f4278195ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:22.173 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.146+0000 7f4277fff700 1 --2- 172.21.15.39:0/3439362154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f42780fe9e0 0x7f4278195980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:22.174 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.147+0000 7f427c886700 1 -- 172.21.15.39:0/3439362154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f42780fe9e0 msgr2=0x7f4278195980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:22.174 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.147+0000 7f427c886700 1 --2- 172.21.15.39:0/3439362154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f42780fe9e0 0x7f4278195980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:22.174 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.147+0000 7f427c886700 1 -- 172.21.15.39:0/3439362154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f42780ffbd0 msgr2=0x7f4278195ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:22.174 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.147+0000 7f427c886700 1 --2- 172.21.15.39:0/3439362154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f42780ffbd0 0x7f4278195ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:22.174 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.147+0000 7f427c886700 1 -- 172.21.15.39:0/3439362154 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4270009d00 con 0x7f4278100560 2023-07-21T18:41:22.174 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.147+0000 7f427c886700 1 --2- 172.21.15.39:0/3439362154 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4278100560 0x7f427819cf30 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f427000d010 tx=0x7f427000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:22.174 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.147+0000 7f42757fa700 1 -- 172.21.15.39:0/3439362154 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f427000e040 con 0x7f4278100560 2023-07-21T18:41:22.175 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.147+0000 7f42757fa700 1 -- 172.21.15.39:0/3439362154 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4270003e10 con 0x7f4278100560 2023-07-21T18:41:22.175 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.147+0000 7f42757fa700 1 -- 172.21.15.39:0/3439362154 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4270005260 con 0x7f4278100560 2023-07-21T18:41:22.175 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.147+0000 7f427e2e9700 1 -- 172.21.15.39:0/3439362154 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f427819da50 con 0x7f4278100560 2023-07-21T18:41:22.175 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.147+0000 7f427e2e9700 1 -- 172.21.15.39:0/3439362154 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f427819de70 con 0x7f4278100560 2023-07-21T18:41:22.175 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.148+0000 7f427e2e9700 1 -- 172.21.15.39:0/3439362154 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4278106a70 con 0x7f4278100560 2023-07-21T18:41:22.175 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.149+0000 7f42757fa700 1 -- 172.21.15.39:0/3439362154 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f4270005400 con 0x7f4278100560 2023-07-21T18:41:22.176 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.150+0000 7f42757fa700 1 --2- 172.21.15.39:0/3439362154 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f42600a1260 0x7f42600a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:22.176 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.150+0000 7f4277fff700 1 --2- 172.21.15.39:0/3439362154 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f42600a1260 0x7f42600a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:22.176 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.150+0000 7f42757fa700 1 -- 172.21.15.39:0/3439362154 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f42700cdc10 con 0x7f4278100560 2023-07-21T18:41:22.176 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.150+0000 7f4277fff700 1 --2- 172.21.15.39:0/3439362154 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f42600a1260 0x7f42600a3710 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f42780ffa30 tx=0x7f4268009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:22.177 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.154+0000 7f42757fa700 1 -- 172.21.15.39:0/3439362154 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f42700972c0 con 0x7f4278100560 2023-07-21T18:41:22.371 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:22 smithi193 ceph-mon[98204]: pgmap v288: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-07-21T18:41:22.371 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:22 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/3837481129' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:41:22.371 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:22 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/1324425699' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T18:41:22.372 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:22 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/3322256208' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T18:41:22.377 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.354+0000 7f427e2e9700 1 -- 172.21.15.39:0/3439362154 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f427802caa0 con 0x7f4278100560 2023-07-21T18:41:22.378 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.355+0000 7f42757fa700 1 -- 172.21.15.39:0/3439362154 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v81) v1 ==== 104+0+11 (secure 0 0 0) 0x7f427009a8e0 con 0x7f4278100560 2023-07-21T18:41:22.378 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2023-07-21T18:41:22.380 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.357+0000 7f427e2e9700 1 -- 172.21.15.39:0/3439362154 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f42600a1260 msgr2=0x7f42600a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:22.380 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.357+0000 7f427e2e9700 1 --2- 172.21.15.39:0/3439362154 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f42600a1260 0x7f42600a3710 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f42780ffa30 tx=0x7f4268009260 comp rx=0 tx=0).stop 2023-07-21T18:41:22.380 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.357+0000 7f427e2e9700 1 -- 172.21.15.39:0/3439362154 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4278100560 msgr2=0x7f427819cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:22.380 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.357+0000 7f427e2e9700 1 --2- 172.21.15.39:0/3439362154 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4278100560 0x7f427819cf30 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f427000d010 tx=0x7f427000bc80 comp rx=0 tx=0).stop 2023-07-21T18:41:22.380 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.357+0000 7f427e2e9700 1 -- 172.21.15.39:0/3439362154 shutdown_connections 2023-07-21T18:41:22.380 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.357+0000 7f427e2e9700 1 --2- 172.21.15.39:0/3439362154 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f42600a1260 0x7f42600a3710 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:22.381 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.357+0000 7f427e2e9700 1 --2- 172.21.15.39:0/3439362154 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4278100560 0x7f427819cf30 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:22.381 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.357+0000 7f427e2e9700 1 --2- 172.21.15.39:0/3439362154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f42780fe9e0 0x7f4278195980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:22.381 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.357+0000 7f427e2e9700 1 --2- 172.21.15.39:0/3439362154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f42780ffbd0 0x7f4278195ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:22.381 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.357+0000 7f427e2e9700 1 -- 172.21.15.39:0/3439362154 >> 172.21.15.39:0/3439362154 conn(0x7f42780fa000 msgr2=0x7f42780fc1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:22.381 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.357+0000 7f427e2e9700 1 -- 172.21.15.39:0/3439362154 shutdown_connections 2023-07-21T18:41:22.381 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.357+0000 7f427e2e9700 1 -- 172.21.15.39:0/3439362154 wait complete. 2023-07-21T18:41:22.389 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:22 smithi155 ceph-mon[97934]: pgmap v288: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-07-21T18:41:22.389 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:22 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/3837481129' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:41:22.390 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:22 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/1324425699' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T18:41:22.390 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:22 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/3322256208' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T18:41:22.391 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:22 smithi039 ceph-mon[96489]: pgmap v288: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-07-21T18:41:22.391 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:22 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/3837481129' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:41:22.391 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:22 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/1324425699' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T18:41:22.392 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:22 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/3322256208' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T18:41:22.392 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-21T18:41:22.525 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.500+0000 7fe1d3092700 1 -- 172.21.15.39:0/1079694974 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1cc0ffbd0 msgr2=0x7fe1cc100020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:22.525 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.500+0000 7fe1d3092700 1 --2- 172.21.15.39:0/1079694974 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1cc0ffbd0 0x7fe1cc100020 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8009990 tx=0x7fe1b8009ca0 comp rx=0 tx=0).stop 2023-07-21T18:41:22.525 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.500+0000 7fe1d3092700 1 -- 172.21.15.39:0/1079694974 shutdown_connections 2023-07-21T18:41:22.526 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.500+0000 7fe1d3092700 1 --2- 172.21.15.39:0/1079694974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1cc100560 0x7fe1cc106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:22.526 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.500+0000 7fe1d3092700 1 --2- 172.21.15.39:0/1079694974 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe1cc0fe9e0 0x7fe1cc0fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:22.526 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.500+0000 7fe1d3092700 1 --2- 172.21.15.39:0/1079694974 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1cc0ffbd0 0x7fe1cc100020 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:22.526 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.500+0000 7fe1d3092700 1 -- 172.21.15.39:0/1079694974 >> 172.21.15.39:0/1079694974 conn(0x7fe1cc0fa000 msgr2=0x7fe1cc0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:22.526 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.500+0000 7fe1d3092700 1 -- 172.21.15.39:0/1079694974 shutdown_connections 2023-07-21T18:41:22.526 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.500+0000 7fe1d3092700 1 -- 172.21.15.39:0/1079694974 wait complete. 2023-07-21T18:41:22.526 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.501+0000 7fe1d3092700 1 Processor -- start 2023-07-21T18:41:22.527 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.501+0000 7fe1d3092700 1 -- start start 2023-07-21T18:41:22.527 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.501+0000 7fe1d3092700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1cc0fe9e0 0x7fe1cc1947a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:22.527 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.501+0000 7fe1d3092700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe1cc0ffbd0 0x7fe1cc194ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:22.527 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.501+0000 7fe1d3092700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1cc100560 0x7fe1cc19bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:22.527 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.501+0000 7fe1d3092700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1cc19c320 con 0x7fe1cc0fe9e0 2023-07-21T18:41:22.527 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.501+0000 7fe1d3092700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1cc19c460 con 0x7fe1cc100560 2023-07-21T18:41:22.527 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.501+0000 7fe1d3092700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1cc19c5f0 con 0x7fe1cc0ffbd0 2023-07-21T18:41:22.527 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.501+0000 7fe1d0e2e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1cc0fe9e0 0x7fe1cc1947a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:22.528 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.501+0000 7fe1d0e2e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1cc0fe9e0 0x7fe1cc1947a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56288/0 (socket says 172.21.15.39:56288) 2023-07-21T18:41:22.528 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.501+0000 7fe1d0e2e700 1 -- 172.21.15.39:0/264470155 learned_addr learned my addr 172.21.15.39:0/264470155 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:22.528 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.501+0000 7fe1cbfff700 1 --2- 172.21.15.39:0/264470155 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe1cc0ffbd0 0x7fe1cc194ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:22.528 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.501+0000 7fe1d162f700 1 --2- 172.21.15.39:0/264470155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1cc100560 0x7fe1cc19bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:22.529 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.501+0000 7fe1d0e2e700 1 -- 172.21.15.39:0/264470155 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe1cc0ffbd0 msgr2=0x7fe1cc194ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:22.529 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.501+0000 7fe1d0e2e700 1 --2- 172.21.15.39:0/264470155 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe1cc0ffbd0 0x7fe1cc194ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:22.529 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.501+0000 7fe1d0e2e700 1 -- 172.21.15.39:0/264470155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1cc100560 msgr2=0x7fe1cc19bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:22.529 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.501+0000 7fe1d0e2e700 1 --2- 172.21.15.39:0/264470155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1cc100560 0x7fe1cc19bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:22.529 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.502+0000 7fe1d0e2e700 1 -- 172.21.15.39:0/264470155 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1b80095f0 con 0x7fe1cc0fe9e0 2023-07-21T18:41:22.530 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.502+0000 7fe1d0e2e700 1 --2- 172.21.15.39:0/264470155 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1cc0fe9e0 0x7fe1cc1947a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe1c000d980 tx=0x7fe1c000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:22.530 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.502+0000 7fe1c9ffb700 1 -- 172.21.15.39:0/264470155 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c0004020 con 0x7fe1cc0fe9e0 2023-07-21T18:41:22.530 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.502+0000 7fe1c9ffb700 1 -- 172.21.15.39:0/264470155 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe1c0009730 con 0x7fe1cc0fe9e0 2023-07-21T18:41:22.530 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.502+0000 7fe1d3092700 1 -- 172.21.15.39:0/264470155 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1cc19c8d0 con 0x7fe1cc0fe9e0 2023-07-21T18:41:22.530 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.502+0000 7fe1c9ffb700 1 -- 172.21.15.39:0/264470155 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c0003a30 con 0x7fe1cc0fe9e0 2023-07-21T18:41:22.531 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.502+0000 7fe1d3092700 1 -- 172.21.15.39:0/264470155 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1cc19ce60 con 0x7fe1cc0fe9e0 2023-07-21T18:41:22.531 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.503+0000 7fe1d3092700 1 -- 172.21.15.39:0/264470155 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1cc105960 con 0x7fe1cc0fe9e0 2023-07-21T18:41:22.532 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.504+0000 7fe1c9ffb700 1 -- 172.21.15.39:0/264470155 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fe1c00098a0 con 0x7fe1cc0fe9e0 2023-07-21T18:41:22.532 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.505+0000 7fe1c9ffb700 1 --2- 172.21.15.39:0/264470155 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fe1bc0a11e0 0x7fe1bc0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:22.532 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.505+0000 7fe1cbfff700 1 --2- 172.21.15.39:0/264470155 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fe1bc0a11e0 0x7fe1bc0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:22.532 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.505+0000 7fe1c9ffb700 1 -- 172.21.15.39:0/264470155 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fe1c00cc770 con 0x7fe1cc0fe9e0 2023-07-21T18:41:22.532 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.505+0000 7fe1cbfff700 1 --2- 172.21.15.39:0/264470155 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fe1bc0a11e0 0x7fe1bc0a3690 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe1b800b5c0 tx=0x7fe1b8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:22.533 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.509+0000 7fe1c9ffb700 1 -- 172.21.15.39:0/264470155 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fe1c0095d90 con 0x7fe1cc0fe9e0 2023-07-21T18:41:22.731 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.708+0000 7fe1d3092700 1 -- 172.21.15.39:0/264470155 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe1cc02cce0 con 0x7fe1cc0fe9e0 2023-07-21T18:41:22.731 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.708+0000 7fe1c9ffb700 1 -- 172.21.15.39:0/264470155 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v81) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe1c00993b0 con 0x7fe1cc0fe9e0 2023-07-21T18:41:22.732 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 64 2023-07-21T18:41:22.733 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.710+0000 7fe1d3092700 1 -- 172.21.15.39:0/264470155 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fe1bc0a11e0 msgr2=0x7fe1bc0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:22.734 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.710+0000 7fe1d3092700 1 --2- 172.21.15.39:0/264470155 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fe1bc0a11e0 0x7fe1bc0a3690 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe1b800b5c0 tx=0x7fe1b8005960 comp rx=0 tx=0).stop 2023-07-21T18:41:22.734 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.710+0000 7fe1d3092700 1 -- 172.21.15.39:0/264470155 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1cc0fe9e0 msgr2=0x7fe1cc1947a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:22.734 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.710+0000 7fe1d3092700 1 --2- 172.21.15.39:0/264470155 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1cc0fe9e0 0x7fe1cc1947a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe1c000d980 tx=0x7fe1c000dc90 comp rx=0 tx=0).stop 2023-07-21T18:41:22.734 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.711+0000 7fe1d3092700 1 -- 172.21.15.39:0/264470155 shutdown_connections 2023-07-21T18:41:22.734 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.711+0000 7fe1d3092700 1 --2- 172.21.15.39:0/264470155 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fe1bc0a11e0 0x7fe1bc0a3690 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:22.734 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.711+0000 7fe1d3092700 1 --2- 172.21.15.39:0/264470155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1cc100560 0x7fe1cc19bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:22.735 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.711+0000 7fe1d3092700 1 --2- 172.21.15.39:0/264470155 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1cc0fe9e0 0x7fe1cc1947a0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:22.735 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.711+0000 7fe1d3092700 1 --2- 172.21.15.39:0/264470155 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe1cc0ffbd0 0x7fe1cc194ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:22.735 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.711+0000 7fe1d3092700 1 -- 172.21.15.39:0/264470155 >> 172.21.15.39:0/264470155 conn(0x7fe1cc0fa000 msgr2=0x7fe1cc0fc290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:22.735 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.711+0000 7fe1d3092700 1 -- 172.21.15.39:0/264470155 shutdown_connections 2023-07-21T18:41:22.735 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.711+0000 7fe1d3092700 1 -- 172.21.15.39:0/264470155 wait complete. 2023-07-21T18:41:22.746 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T18:41:22.874 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.849+0000 7f2a767aa700 1 -- 172.21.15.39:0/1121764102 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a700fea00 msgr2=0x7f2a700fee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:22.874 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.849+0000 7f2a767aa700 1 --2- 172.21.15.39:0/1121764102 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a700fea00 0x7f2a700fee10 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f2a60009990 tx=0x7f2a60009ca0 comp rx=0 tx=0).stop 2023-07-21T18:41:22.874 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.849+0000 7f2a767aa700 1 -- 172.21.15.39:0/1121764102 shutdown_connections 2023-07-21T18:41:22.874 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.849+0000 7f2a767aa700 1 --2- 172.21.15.39:0/1121764102 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a70100580 0x7f2a70106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:22.874 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.849+0000 7f2a767aa700 1 --2- 172.21.15.39:0/1121764102 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a700fea00 0x7f2a700fee10 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:22.875 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.849+0000 7f2a767aa700 1 --2- 172.21.15.39:0/1121764102 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a700ffbf0 0x7f2a70100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:22.875 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.849+0000 7f2a767aa700 1 -- 172.21.15.39:0/1121764102 >> 172.21.15.39:0/1121764102 conn(0x7f2a700f9fe0 msgr2=0x7f2a700fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:22.875 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.849+0000 7f2a767aa700 1 -- 172.21.15.39:0/1121764102 shutdown_connections 2023-07-21T18:41:22.875 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.849+0000 7f2a767aa700 1 -- 172.21.15.39:0/1121764102 wait complete. 2023-07-21T18:41:22.875 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.849+0000 7f2a767aa700 1 Processor -- start 2023-07-21T18:41:22.876 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.849+0000 7f2a767aa700 1 -- start start 2023-07-21T18:41:22.876 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.850+0000 7f2a767aa700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a700fea00 0x7f2a70102990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:22.876 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.850+0000 7f2a767aa700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a700ffbf0 0x7f2a70100fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:22.876 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.850+0000 7f2a767aa700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a70100580 0x7f2a70101510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:22.876 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.850+0000 7f2a767aa700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a70101cb0 con 0x7f2a70100580 2023-07-21T18:41:22.876 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.850+0000 7f2a767aa700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a70101e20 con 0x7f2a700ffbf0 2023-07-21T18:41:22.877 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.850+0000 7f2a767aa700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7019f870 con 0x7f2a700fea00 2023-07-21T18:41:22.877 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.850+0000 7f2a74d47700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a70100580 0x7f2a70101510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:22.877 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.850+0000 7f2a74d47700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a70100580 0x7f2a70101510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56306/0 (socket says 172.21.15.39:56306) 2023-07-21T18:41:22.877 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.850+0000 7f2a74d47700 1 -- 172.21.15.39:0/2448255077 learned_addr learned my addr 172.21.15.39:0/2448255077 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:22.877 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.850+0000 7f2a6ffff700 1 --2- 172.21.15.39:0/2448255077 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a700fea00 0x7f2a70102990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:22.877 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.850+0000 7f2a6f7fe700 1 --2- 172.21.15.39:0/2448255077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a700ffbf0 0x7f2a70100fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:22.877 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.850+0000 7f2a74d47700 1 -- 172.21.15.39:0/2448255077 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a700fea00 msgr2=0x7f2a70102990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:22.878 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.850+0000 7f2a74d47700 1 --2- 172.21.15.39:0/2448255077 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a700fea00 0x7f2a70102990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:22.878 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.850+0000 7f2a74d47700 1 -- 172.21.15.39:0/2448255077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a700ffbf0 msgr2=0x7f2a70100fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:22.878 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.850+0000 7f2a74d47700 1 --2- 172.21.15.39:0/2448255077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a700ffbf0 0x7f2a70100fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:22.878 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.850+0000 7f2a74d47700 1 -- 172.21.15.39:0/2448255077 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a600095f0 con 0x7f2a70100580 2023-07-21T18:41:22.878 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.850+0000 7f2a74d47700 1 --2- 172.21.15.39:0/2448255077 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a70100580 0x7f2a70101510 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2a6800ada0 tx=0x7f2a6800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:22.878 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.850+0000 7f2a6d7fa700 1 -- 172.21.15.39:0/2448255077 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a6800f070 con 0x7f2a70100580 2023-07-21T18:41:22.878 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.851+0000 7f2a6d7fa700 1 -- 172.21.15.39:0/2448255077 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a68007b30 con 0x7f2a70100580 2023-07-21T18:41:22.878 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.851+0000 7f2a767aa700 1 -- 172.21.15.39:0/2448255077 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a7019fb50 con 0x7f2a70100580 2023-07-21T18:41:22.879 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.851+0000 7f2a6d7fa700 1 -- 172.21.15.39:0/2448255077 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a68005440 con 0x7f2a70100580 2023-07-21T18:41:22.879 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.851+0000 7f2a767aa700 1 -- 172.21.15.39:0/2448255077 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a701a0060 con 0x7f2a70100580 2023-07-21T18:41:22.880 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.851+0000 7f2a767aa700 1 -- 172.21.15.39:0/2448255077 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a7010a220 con 0x7f2a70100580 2023-07-21T18:41:22.880 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.853+0000 7f2a6d7fa700 1 -- 172.21.15.39:0/2448255077 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f2a68004ad0 con 0x7f2a70100580 2023-07-21T18:41:22.880 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.853+0000 7f2a6d7fa700 1 --2- 172.21.15.39:0/2448255077 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f2a5c0a12a0 0x7f2a5c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:22.880 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.853+0000 7f2a6ffff700 1 --2- 172.21.15.39:0/2448255077 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f2a5c0a12a0 0x7f2a5c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:22.880 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.853+0000 7f2a6d7fa700 1 -- 172.21.15.39:0/2448255077 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f2a680cc6a0 con 0x7f2a70100580 2023-07-21T18:41:22.880 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.854+0000 7f2a6ffff700 1 --2- 172.21.15.39:0/2448255077 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f2a5c0a12a0 0x7f2a5c0a3750 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2a6000b5c0 tx=0x7f2a60005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:22.881 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:22.857+0000 7f2a6d7fa700 1 -- 172.21.15.39:0/2448255077 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f2a68095d50 con 0x7f2a70100580 2023-07-21T18:41:23.125 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.102+0000 7f2a767aa700 1 -- 172.21.15.39:0/2448255077 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2a701a0610 con 0x7f2a70100580 2023-07-21T18:41:23.126 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.103+0000 7f2a6d7fa700 1 -- 172.21.15.39:0/2448255077 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10887 (secure 0 0 0) 0x7f2a68099370 con 0x7f2a70100580 2023-07-21T18:41:23.126 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:41:23.127 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/147318681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":147318681},{"type":"v1","addr":"172.21.15.39:6835","nonce":147318681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/1062891760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1062891760},{"type":"v1","addr":"172.21.15.39:6837","nonce":1062891760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T18:40:04.594553+0000","modified":"2023-07-21T18:40:11.989039+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24536,"mds_1":24500,"mds_2":24487,"mds_3":24464,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/3184273349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":3184273349},{"type":"v1","addr":"172.21.15.39:6839","nonce":3184273349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6841/640981220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":640981220},{"type":"v1","addr":"172.21.15.39:6841","nonce":640981220}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3106166804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3106166804},{"type":"v1","addr":"172.21.15.193:6833","nonce":3106166804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24437":{"gid":24437,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/1034755277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1034755277},{"type":"v1","addr":"172.21.15.155:6833","nonce":1034755277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/381560657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":381560657},{"type":"v1","addr":"172.21.15.193:6835","nonce":381560657}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6837/2626761843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2626761843},{"type":"v1","addr":"172.21.15.193:6837","nonce":2626761843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24464":{"gid":24464,"name":"e","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.155:6835/3854856286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3854856286},{"type":"v1","addr":"172.21.15.155:6835","nonce":3854856286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.193:6839/3148488639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3148488639},{"type":"v1","addr":"172.21.15.193:6839","nonce":3148488639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.155:6837/947670056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":947670056},{"type":"v1","addr":"172.21.15.155:6837","nonce":947670056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6839/431690342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":431690342},{"type":"v1","addr":"172.21.15.155:6839","nonce":431690342}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T18:41:23.128 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.105+0000 7f2a767aa700 1 -- 172.21.15.39:0/2448255077 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f2a5c0a12a0 msgr2=0x7f2a5c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:23.128 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.105+0000 7f2a767aa700 1 --2- 172.21.15.39:0/2448255077 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f2a5c0a12a0 0x7f2a5c0a3750 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2a6000b5c0 tx=0x7f2a60005fb0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.129 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.105+0000 7f2a767aa700 1 -- 172.21.15.39:0/2448255077 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a70100580 msgr2=0x7f2a70101510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:23.129 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.105+0000 7f2a767aa700 1 --2- 172.21.15.39:0/2448255077 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a70100580 0x7f2a70101510 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2a6800ada0 tx=0x7f2a6800c330 comp rx=0 tx=0).stop 2023-07-21T18:41:23.129 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.105+0000 7f2a767aa700 1 -- 172.21.15.39:0/2448255077 shutdown_connections 2023-07-21T18:41:23.130 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.105+0000 7f2a767aa700 1 --2- 172.21.15.39:0/2448255077 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f2a5c0a12a0 0x7f2a5c0a3750 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.130 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.105+0000 7f2a767aa700 1 --2- 172.21.15.39:0/2448255077 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a70100580 0x7f2a70101510 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.131 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.105+0000 7f2a767aa700 1 --2- 172.21.15.39:0/2448255077 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a700fea00 0x7f2a70102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.131 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.105+0000 7f2a767aa700 1 --2- 172.21.15.39:0/2448255077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a700ffbf0 0x7f2a70100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.131 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.105+0000 7f2a767aa700 1 -- 172.21.15.39:0/2448255077 >> 172.21.15.39:0/2448255077 conn(0x7f2a700f9fe0 msgr2=0x7f2a700686e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:23.131 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.105+0000 7f2a767aa700 1 -- 172.21.15.39:0/2448255077 shutdown_connections 2023-07-21T18:41:23.131 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.105+0000 7f2a767aa700 1 -- 172.21.15.39:0/2448255077 wait complete. 2023-07-21T18:41:23.132 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 26 2023-07-21T18:41:23.140 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T18:41:23.267 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.243+0000 7f1f5770d700 1 -- 172.21.15.39:0/309530039 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f50100550 msgr2=0x7f1f50106e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:23.267 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.243+0000 7f1f5770d700 1 --2- 172.21.15.39:0/309530039 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f50100550 0x7f1f50106e10 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f1f4800b0a0 tx=0x7f1f4800b3b0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.267 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.243+0000 7f1f5770d700 1 -- 172.21.15.39:0/309530039 shutdown_connections 2023-07-21T18:41:23.267 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.243+0000 7f1f5770d700 1 --2- 172.21.15.39:0/309530039 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f50100550 0x7f1f50106e10 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.267 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.243+0000 7f1f5770d700 1 --2- 172.21.15.39:0/309530039 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1f500fe9d0 0x7f1f500fede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.268 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.243+0000 7f1f5770d700 1 --2- 172.21.15.39:0/309530039 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f500ffbc0 0x7f1f50100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.268 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.243+0000 7f1f5770d700 1 -- 172.21.15.39:0/309530039 >> 172.21.15.39:0/309530039 conn(0x7f1f500f9fd0 msgr2=0x7f1f500fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:23.268 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.243+0000 7f1f5770d700 1 -- 172.21.15.39:0/309530039 shutdown_connections 2023-07-21T18:41:23.268 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.243+0000 7f1f5770d700 1 -- 172.21.15.39:0/309530039 wait complete. 2023-07-21T18:41:23.268 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.244+0000 7f1f5770d700 1 Processor -- start 2023-07-21T18:41:23.268 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.244+0000 7f1f5770d700 1 -- start start 2023-07-21T18:41:23.270 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.244+0000 7f1f5770d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1f500fe9d0 0x7f1f50194780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:23.270 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.244+0000 7f1f5770d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f500ffbc0 0x7f1f50194cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:23.270 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.244+0000 7f1f5770d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f50100550 0x7f1f5019bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:23.270 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.244+0000 7f1f5770d700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5019c300 con 0x7f1f500ffbc0 2023-07-21T18:41:23.270 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.244+0000 7f1f5770d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5019c440 con 0x7f1f50100550 2023-07-21T18:41:23.271 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.244+0000 7f1f5770d700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5019c5d0 con 0x7f1f500fe9d0 2023-07-21T18:41:23.271 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.244+0000 7f1f54ca8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f500ffbc0 0x7f1f50194cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:23.271 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.244+0000 7f1f54ca8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f500ffbc0 0x7f1f50194cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56318/0 (socket says 172.21.15.39:56318) 2023-07-21T18:41:23.271 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.244+0000 7f1f54ca8700 1 -- 172.21.15.39:0/2248819757 learned_addr learned my addr 172.21.15.39:0/2248819757 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:23.271 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.245+0000 7f1f55caa700 1 --2- 172.21.15.39:0/2248819757 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f50100550 0x7f1f5019bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:23.271 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.245+0000 7f1f554a9700 1 --2- 172.21.15.39:0/2248819757 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1f500fe9d0 0x7f1f50194780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:23.271 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.245+0000 7f1f54ca8700 1 -- 172.21.15.39:0/2248819757 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1f500fe9d0 msgr2=0x7f1f50194780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:23.271 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.245+0000 7f1f54ca8700 1 --2- 172.21.15.39:0/2248819757 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1f500fe9d0 0x7f1f50194780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.272 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.245+0000 7f1f54ca8700 1 -- 172.21.15.39:0/2248819757 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f50100550 msgr2=0x7f1f5019bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:23.272 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.245+0000 7f1f54ca8700 1 --2- 172.21.15.39:0/2248819757 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f50100550 0x7f1f5019bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.272 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.245+0000 7f1f54ca8700 1 -- 172.21.15.39:0/2248819757 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f48009d00 con 0x7f1f500ffbc0 2023-07-21T18:41:23.272 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.245+0000 7f1f54ca8700 1 --2- 172.21.15.39:0/2248819757 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f500ffbc0 0x7f1f50194cc0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1f4400e9a0 tx=0x7f1f4400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:23.272 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.245+0000 7f1f427fc700 1 -- 172.21.15.39:0/2248819757 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f44009790 con 0x7f1f500ffbc0 2023-07-21T18:41:23.272 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.245+0000 7f1f427fc700 1 -- 172.21.15.39:0/2248819757 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f44010430 con 0x7f1f500ffbc0 2023-07-21T18:41:23.272 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.245+0000 7f1f5770d700 1 -- 172.21.15.39:0/2248819757 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f5019c8b0 con 0x7f1f500ffbc0 2023-07-21T18:41:23.273 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.245+0000 7f1f427fc700 1 -- 172.21.15.39:0/2248819757 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f44018680 con 0x7f1f500ffbc0 2023-07-21T18:41:23.273 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.245+0000 7f1f5770d700 1 -- 172.21.15.39:0/2248819757 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f5019ce10 con 0x7f1f500ffbc0 2023-07-21T18:41:23.273 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.246+0000 7f1f5770d700 1 -- 172.21.15.39:0/2248819757 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f5010a1b0 con 0x7f1f500ffbc0 2023-07-21T18:41:23.273 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.247+0000 7f1f427fc700 1 -- 172.21.15.39:0/2248819757 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f1f440105a0 con 0x7f1f500ffbc0 2023-07-21T18:41:23.274 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.248+0000 7f1f427fc700 1 --2- 172.21.15.39:0/2248819757 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f1f3c0a12a0 0x7f1f3c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:23.274 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.248+0000 7f1f554a9700 1 --2- 172.21.15.39:0/2248819757 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f1f3c0a12a0 0x7f1f3c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:23.274 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.248+0000 7f1f427fc700 1 -- 172.21.15.39:0/2248819757 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f1f44014070 con 0x7f1f500ffbc0 2023-07-21T18:41:23.274 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.249+0000 7f1f554a9700 1 --2- 172.21.15.39:0/2248819757 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f1f3c0a12a0 0x7f1f3c0a3750 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1f500ffa20 tx=0x7f1f4c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:23.274 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.251+0000 7f1f427fc700 1 -- 172.21.15.39:0/2248819757 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f1f44096410 con 0x7f1f500ffbc0 2023-07-21T18:41:23.371 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:23 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/3439362154' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T18:41:23.371 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:23 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/264470155' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T18:41:23.389 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:23 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/3439362154' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T18:41:23.390 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:23 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/264470155' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T18:41:23.438 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:23 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/3439362154' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T18:41:23.438 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:23 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/264470155' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T18:41:23.474 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.451+0000 7f1f5770d700 1 -- 172.21.15.39:0/2248819757 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1f5019d510 con 0x7f1f500ffbc0 2023-07-21T18:41:23.476 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.453+0000 7f1f427fc700 1 -- 172.21.15.39:0/2248819757 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+22447 (secure 0 0 0) 0x7f1f44026070 con 0x7f1f500ffbc0 2023-07-21T18:41:23.476 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:41:23.477 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":81,"fsid":"eaf8e25a-27f4-11ee-9b34-001a4aab830c","created":"2023-07-21T18:33:06.504979+0000","modified":"2023-07-21T18:41:08.140284+0000","last_up_change":"2023-07-21T18:39:23.531009+0000","last_in_change":"2023-07-21T18:39:05.993024+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T18:36:13.572583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T18:40:02.726735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T18:40:03.990112+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03502c07-2d0f-4723-8d0b-d34c9f5b91bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6803","nonce":860193857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6805","nonce":860193857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6809","nonce":860193857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6807","nonce":860193857}]},"public_addr":"172.21.15.39:6803/860193857","cluster_addr":"172.21.15.39:6805/860193857","heartbeat_back_addr":"172.21.15.39:6809/860193857","heartbeat_front_addr":"172.21.15.39:6807/860193857","state":["exists","up"]},{"osd":1,"uuid":"0831a0c0-a1fa-4d30-9a28-513223aa69eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6801","nonce":3246721764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6803","nonce":3246721764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6807","nonce":3246721764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6805","nonce":3246721764}]},"public_addr":"172.21.15.155:6801/3246721764","cluster_addr":"172.21.15.155:6803/3246721764","heartbeat_back_addr":"172.21.15.155:6807/3246721764","heartbeat_front_addr":"172.21.15.155:6805/3246721764","state":["exists","up"]},{"osd":2,"uuid":"22df0003-bde9-48e4-b47a-cb8eafbab0db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6801","nonce":2563594449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6803","nonce":2563594449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6807","nonce":2563594449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6805","nonce":2563594449}]},"public_addr":"172.21.15.193:6801/2563594449","cluster_addr":"172.21.15.193:6803/2563594449","heartbeat_back_addr":"172.21.15.193:6807/2563594449","heartbeat_front_addr":"172.21.15.193:6805/2563594449","state":["exists","up"]},{"osd":3,"uuid":"54562558-a2e8-45dd-bc95-561569b9ae6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6811","nonce":863708386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6813","nonce":863708386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6817","nonce":863708386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6815","nonce":863708386}]},"public_addr":"172.21.15.39:6811/863708386","cluster_addr":"172.21.15.39:6813/863708386","heartbeat_back_addr":"172.21.15.39:6817/863708386","heartbeat_front_addr":"172.21.15.39:6815/863708386","state":["exists","up"]},{"osd":4,"uuid":"491118e7-ae1e-495c-9eac-ef1cdcae9600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6809","nonce":3361780673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6811","nonce":3361780673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6815","nonce":3361780673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6813","nonce":3361780673}]},"public_addr":"172.21.15.155:6809/3361780673","cluster_addr":"172.21.15.155:6811/3361780673","heartbeat_back_addr":"172.21.15.155:6815/3361780673","heartbeat_front_addr":"172.21.15.155:6813/3361780673","state":["exists","up"]},{"osd":5,"uuid":"d267bc2c-42b3-4d96-b2d1-b186dc7e6b18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6809","nonce":2376867721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6811","nonce":2376867721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6815","nonce":2376867721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6813","nonce":2376867721}]},"public_addr":"172.21.15.193:6809/2376867721","cluster_addr":"172.21.15.193:6811/2376867721","heartbeat_back_addr":"172.21.15.193:6815/2376867721","heartbeat_front_addr":"172.21.15.193:6813/2376867721","state":["exists","up"]},{"osd":6,"uuid":"beec8088-970a-473e-bd67-6eee7cbdef24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6819","nonce":2417770529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6821","nonce":2417770529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6825","nonce":2417770529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6823","nonce":2417770529}]},"public_addr":"172.21.15.39:6819/2417770529","cluster_addr":"172.21.15.39:6821/2417770529","heartbeat_back_addr":"172.21.15.39:6825/2417770529","heartbeat_front_addr":"172.21.15.39:6823/2417770529","state":["exists","up"]},{"osd":7,"uuid":"5a88ce30-1835-4254-90cc-804ed5cf1066","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6817","nonce":2542002111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6819","nonce":2542002111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6823","nonce":2542002111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6821","nonce":2542002111}]},"public_addr":"172.21.15.155:6817/2542002111","cluster_addr":"172.21.15.155:6819/2542002111","heartbeat_back_addr":"172.21.15.155:6823/2542002111","heartbeat_front_addr":"172.21.15.155:6821/2542002111","state":["exists","up"]},{"osd":8,"uuid":"d964a183-3021-4ace-9a0c-aad00f2570e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6817","nonce":2985835085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6819","nonce":2985835085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6823","nonce":2985835085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6821","nonce":2985835085}]},"public_addr":"172.21.15.193:6817/2985835085","cluster_addr":"172.21.15.193:6819/2985835085","heartbeat_back_addr":"172.21.15.193:6823/2985835085","heartbeat_front_addr":"172.21.15.193:6821/2985835085","state":["exists","up"]},{"osd":9,"uuid":"9725ea5c-49a3-4c99-bac4-e95069100b1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6827","nonce":1946493651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6829","nonce":1946493651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6833","nonce":1946493651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6831","nonce":1946493651}]},"public_addr":"172.21.15.39:6827/1946493651","cluster_addr":"172.21.15.39:6829/1946493651","heartbeat_back_addr":"172.21.15.39:6833/1946493651","heartbeat_front_addr":"172.21.15.39:6831/1946493651","state":["exists","up"]},{"osd":10,"uuid":"1fd60ccb-92db-45f0-a580-2003db0976e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6825","nonce":1234615022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6827","nonce":1234615022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6831","nonce":1234615022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6829","nonce":1234615022}]},"public_addr":"172.21.15.155:6825/1234615022","cluster_addr":"172.21.15.155:6827/1234615022","heartbeat_back_addr":"172.21.15.155:6831/1234615022","heartbeat_front_addr":"172.21.15.155:6829/1234615022","state":["exists","up"]},{"osd":11,"uuid":"efb31ea1-4775-4e5f-8740-728d6f4aeea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6825","nonce":215104663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6827","nonce":215104663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6831","nonce":215104663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6829","nonce":215104663}]},"public_addr":"172.21.15.193:6825/215104663","cluster_addr":"172.21.15.193:6827/215104663","heartbeat_back_addr":"172.21.15.193:6831/215104663","heartbeat_front_addr":"172.21.15.193:6829/215104663","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:35:30.211698+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:35:50.883846+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:36:11.124168+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:36:32.711905+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:36:54.442007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:37:14.843526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:37:36.587354+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:37:57.882374+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:38:18.729407+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:38:40.139231+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:39:01.290862+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:39:21.767537+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:0/2902600414":"2023-07-22T18:34:07.039823+0000","172.21.15.39:6801/2893866425":"2023-07-22T18:34:07.039823+0000","172.21.15.39:0/19717002":"2023-07-22T18:34:07.039823+0000","172.21.15.39:6800/2893866425":"2023-07-22T18:34:07.039823+0000","172.21.15.39:6800/889516631":"2023-07-22T18:33:40.852706+0000","172.21.15.39:6801/889516631":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/3636510379":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/1275748831":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/3819152602":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/1127767082":"2023-07-22T18:34:07.039823+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T18:41:23.478 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.455+0000 7f1f5770d700 1 -- 172.21.15.39:0/2248819757 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f1f3c0a12a0 msgr2=0x7f1f3c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:23.478 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.455+0000 7f1f5770d700 1 --2- 172.21.15.39:0/2248819757 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f1f3c0a12a0 0x7f1f3c0a3750 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1f500ffa20 tx=0x7f1f4c00b410 comp rx=0 tx=0).stop 2023-07-21T18:41:23.479 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.455+0000 7f1f5770d700 1 -- 172.21.15.39:0/2248819757 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f500ffbc0 msgr2=0x7f1f50194cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:23.479 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.455+0000 7f1f5770d700 1 --2- 172.21.15.39:0/2248819757 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f500ffbc0 0x7f1f50194cc0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1f4400e9a0 tx=0x7f1f4400ecb0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.479 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.455+0000 7f1f5770d700 1 -- 172.21.15.39:0/2248819757 shutdown_connections 2023-07-21T18:41:23.479 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.455+0000 7f1f5770d700 1 --2- 172.21.15.39:0/2248819757 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f1f3c0a12a0 0x7f1f3c0a3750 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.479 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.455+0000 7f1f5770d700 1 --2- 172.21.15.39:0/2248819757 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f50100550 0x7f1f5019bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.479 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.455+0000 7f1f5770d700 1 --2- 172.21.15.39:0/2248819757 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1f500fe9d0 0x7f1f50194780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.479 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.455+0000 7f1f5770d700 1 --2- 172.21.15.39:0/2248819757 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f500ffbc0 0x7f1f50194cc0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.480 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.455+0000 7f1f5770d700 1 -- 172.21.15.39:0/2248819757 >> 172.21.15.39:0/2248819757 conn(0x7f1f500f9fd0 msgr2=0x7f1f500fc180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:23.480 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.455+0000 7f1f5770d700 1 -- 172.21.15.39:0/2248819757 shutdown_connections 2023-07-21T18:41:23.480 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.455+0000 7f1f5770d700 1 -- 172.21.15.39:0/2248819757 wait complete. 2023-07-21T18:41:23.491 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-07-21T18:40:04.594553+0000', 'modified': '2023-07-21T18:40:11.989039+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24536, 'mds_1': 24500, 'mds_2': 24487, 'mds_3': 24464, 'mds_4': 14604}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14592': {'gid': 14592, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.39:6839/3184273349', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6838', 'nonce': 3184273349}, {'type': 'v1', 'addr': '172.21.15.39:6839', 'nonce': 3184273349}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_14604': {'gid': 14604, 'name': 'a', 'rank': 4, 'incarnation': 24, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.39:6841/640981220', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6840', 'nonce': 640981220}, {'type': 'v1', 'addr': '172.21.15.39:6841', 'nonce': 640981220}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24415': {'gid': 24415, 'name': 'l', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.193:6833/3106166804', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6832', 'nonce': 3106166804}, {'type': 'v1', 'addr': '172.21.15.193:6833', 'nonce': 3106166804}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24437': {'gid': 24437, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.155:6833/1034755277', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6832', 'nonce': 1034755277}, {'type': 'v1', 'addr': '172.21.15.155:6833', 'nonce': 1034755277}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24451': {'gid': 24451, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.193:6835/381560657', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6834', 'nonce': 381560657}, {'type': 'v1', 'addr': '172.21.15.193:6835', 'nonce': 381560657}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24463': {'gid': 24463, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.193:6837/2626761843', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6836', 'nonce': 2626761843}, {'type': 'v1', 'addr': '172.21.15.193:6837', 'nonce': 2626761843}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24464': {'gid': 24464, 'name': 'e', 'rank': 3, 'incarnation': 22, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.155:6835/3854856286', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 3854856286}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 3854856286}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24487': {'gid': 24487, 'name': 'i', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.193:6839/3148488639', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6838', 'nonce': 3148488639}, {'type': 'v1', 'addr': '172.21.15.193:6839', 'nonce': 3148488639}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24500': {'gid': 24500, 'name': 'h', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.155:6837/947670056', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6836', 'nonce': 947670056}, {'type': 'v1', 'addr': '172.21.15.155:6837', 'nonce': 947670056}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24536': {'gid': 24536, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.155:6839/431690342', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6838', 'nonce': 431690342}, {'type': 'v1', 'addr': '172.21.15.155:6839', 'nonce': 431690342}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-07-21T18:41:23.492 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2023-07-21T18:41:23.492 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-07-21T18:41:23.619 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.596+0000 7fd71422c700 1 -- 172.21.15.39:0/3659414824 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd70c0fea10 msgr2=0x7fd70c0fee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:23.619 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.596+0000 7fd71422c700 1 --2- 172.21.15.39:0/3659414824 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd70c0fea10 0x7fd70c0fee20 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc009990 tx=0x7fd6fc009ca0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.620 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.596+0000 7fd71422c700 1 -- 172.21.15.39:0/3659414824 shutdown_connections 2023-07-21T18:41:23.620 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.596+0000 7fd71422c700 1 --2- 172.21.15.39:0/3659414824 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd70c100590 0x7fd70c106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.620 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.596+0000 7fd71422c700 1 --2- 172.21.15.39:0/3659414824 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd70c0fea10 0x7fd70c0fee20 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.620 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.596+0000 7fd71422c700 1 --2- 172.21.15.39:0/3659414824 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd70c0ffc00 0x7fd70c100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.620 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.596+0000 7fd71422c700 1 -- 172.21.15.39:0/3659414824 >> 172.21.15.39:0/3659414824 conn(0x7fd70c0fa030 msgr2=0x7fd70c0fc460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:23.621 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.596+0000 7fd71422c700 1 -- 172.21.15.39:0/3659414824 shutdown_connections 2023-07-21T18:41:23.621 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.596+0000 7fd71422c700 1 -- 172.21.15.39:0/3659414824 wait complete. 2023-07-21T18:41:23.623 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.597+0000 7fd71422c700 1 Processor -- start 2023-07-21T18:41:23.623 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.597+0000 7fd71422c700 1 -- start start 2023-07-21T18:41:23.623 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.597+0000 7fd71422c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd70c0fea10 0x7fd70c1029a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:23.623 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.597+0000 7fd71422c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd70c0ffc00 0x7fd70c100fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:23.623 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.597+0000 7fd71422c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd70c100590 0x7fd70c101520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:23.623 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.597+0000 7fd71422c700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd70c101cc0 con 0x7fd70c100590 2023-07-21T18:41:23.624 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.597+0000 7fd71422c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd70c101e30 con 0x7fd70c0fea10 2023-07-21T18:41:23.624 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.597+0000 7fd71422c700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd70c19f8c0 con 0x7fd70c0ffc00 2023-07-21T18:41:23.625 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.597+0000 7fd7127c9700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd70c100590 0x7fd70c101520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:23.625 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.597+0000 7fd7127c9700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd70c100590 0x7fd70c101520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56334/0 (socket says 172.21.15.39:56334) 2023-07-21T18:41:23.625 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.597+0000 7fd7127c9700 1 -- 172.21.15.39:0/2386669356 learned_addr learned my addr 172.21.15.39:0/2386669356 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:23.625 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.597+0000 7fd7117c7700 1 --2- 172.21.15.39:0/2386669356 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd70c0ffc00 0x7fd70c100fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:23.625 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.597+0000 7fd711fc8700 1 --2- 172.21.15.39:0/2386669356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd70c0fea10 0x7fd70c1029a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:23.625 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.597+0000 7fd7127c9700 1 -- 172.21.15.39:0/2386669356 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd70c0ffc00 msgr2=0x7fd70c100fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:23.625 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.597+0000 7fd7127c9700 1 --2- 172.21.15.39:0/2386669356 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd70c0ffc00 0x7fd70c100fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.626 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.597+0000 7fd7127c9700 1 -- 172.21.15.39:0/2386669356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd70c0fea10 msgr2=0x7fd70c1029a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:23.626 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.597+0000 7fd7127c9700 1 --2- 172.21.15.39:0/2386669356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd70c0fea10 0x7fd70c1029a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.626 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.597+0000 7fd7127c9700 1 -- 172.21.15.39:0/2386669356 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6fc0095f0 con 0x7fd70c100590 2023-07-21T18:41:23.626 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.597+0000 7fd7117c7700 1 --2- 172.21.15.39:0/2386669356 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd70c0ffc00 0x7fd70c100fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:41:23.626 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.597+0000 7fd7127c9700 1 --2- 172.21.15.39:0/2386669356 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd70c100590 0x7fd70c101520 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd704007ee0 tx=0x7fd70400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:23.626 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.597+0000 7fd702ffd700 1 -- 172.21.15.39:0/2386669356 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd704017070 con 0x7fd70c100590 2023-07-21T18:41:23.626 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.598+0000 7fd702ffd700 1 -- 172.21.15.39:0/2386669356 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd70400f040 con 0x7fd70c100590 2023-07-21T18:41:23.626 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.598+0000 7fd71422c700 1 -- 172.21.15.39:0/2386669356 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd70c19fba0 con 0x7fd70c100590 2023-07-21T18:41:23.627 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.598+0000 7fd702ffd700 1 -- 172.21.15.39:0/2386669356 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd704005740 con 0x7fd70c100590 2023-07-21T18:41:23.627 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.598+0000 7fd71422c700 1 -- 172.21.15.39:0/2386669356 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd70c1a00b0 con 0x7fd70c100590 2023-07-21T18:41:23.627 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.600+0000 7fd702ffd700 1 -- 172.21.15.39:0/2386669356 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fd7040058a0 con 0x7fd70c100590 2023-07-21T18:41:23.631 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.600+0000 7fd702ffd700 1 --2- 172.21.15.39:0/2386669356 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fd6f80a1260 0x7fd6f80a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:23.632 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.601+0000 7fd702ffd700 1 -- 172.21.15.39:0/2386669356 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fd7040cc280 con 0x7fd70c100590 2023-07-21T18:41:23.632 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.601+0000 7fd711fc8700 1 --2- 172.21.15.39:0/2386669356 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fd6f80a1260 0x7fd6f80a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:23.632 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.601+0000 7fd711fc8700 1 --2- 172.21.15.39:0/2386669356 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fd6f80a1260 0x7fd6f80a3710 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc006010 tx=0x7fd6fc00b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:23.632 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.601+0000 7fd7203b2b80 1 Processor -- start 2023-07-21T18:41:23.632 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.601+0000 7fd7203b2b80 1 -- start start 2023-07-21T18:41:23.632 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.602+0000 7fd7203b2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be53ed1480 0x55be53ed1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:23.633 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.602+0000 7fd7203b2b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be53ed1dd0 0x55be53ed2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:23.633 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.602+0000 7fd7203b2b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be53ed2c20 0x55be53ed3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:23.633 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.602+0000 7fd7203b2b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be53dd6c60 con 0x55be53ed1dd0 2023-07-21T18:41:23.633 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.602+0000 7fd7203b2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be53e299f0 con 0x55be53ed1480 2023-07-21T18:41:23.633 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.602+0000 7fd7203b2b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be53e22e80 con 0x55be53ed2c20 2023-07-21T18:41:23.633 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.602+0000 7fd7117c7700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be53ed1dd0 0x55be53ed2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:23.633 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.602+0000 7fd7117c7700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be53ed1dd0 0x55be53ed2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56340/0 (socket says 172.21.15.39:56340) 2023-07-21T18:41:23.633 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.602+0000 7fd7117c7700 1 -- 172.21.15.39:0/3001089605 learned_addr learned my addr 172.21.15.39:0/3001089605 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:23.634 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.602+0000 7fd711fc8700 1 --2- 172.21.15.39:0/3001089605 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be53ed1480 0x55be53ed1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:23.634 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.602+0000 7fd7127c9700 1 --2- 172.21.15.39:0/3001089605 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be53ed2c20 0x55be53ed3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:23.634 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.602+0000 7fd7117c7700 1 -- 172.21.15.39:0/3001089605 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be53ed2c20 msgr2=0x55be53ed3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:23.634 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.602+0000 7fd7117c7700 1 --2- 172.21.15.39:0/3001089605 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be53ed2c20 0x55be53ed3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.635 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.602+0000 7fd7117c7700 1 -- 172.21.15.39:0/3001089605 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be53ed1480 msgr2=0x55be53ed1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:23.635 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.602+0000 7fd7117c7700 1 --2- 172.21.15.39:0/3001089605 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be53ed1480 0x55be53ed1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.635 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.602+0000 7fd7117c7700 1 -- 172.21.15.39:0/3001089605 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be53c50d90 con 0x55be53ed1dd0 2023-07-21T18:41:23.635 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.602+0000 7fd7127c9700 1 --2- 172.21.15.39:0/3001089605 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be53ed2c20 0x55be53ed3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:41:23.635 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.602+0000 7fd711fc8700 1 --2- 172.21.15.39:0/3001089605 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be53ed1480 0x55be53ed1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:41:23.635 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.602+0000 7fd7117c7700 1 --2- 172.21.15.39:0/3001089605 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be53ed1dd0 0x55be53ed2690 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd708009de0 tx=0x7fd7080116c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7d63df045a3c960 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:23.635 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.602+0000 7fd6f7fff700 1 -- 172.21.15.39:0/3001089605 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd708011ed0 con 0x55be53ed1dd0 2023-07-21T18:41:23.635 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.602+0000 7fd6f7fff700 1 -- 172.21.15.39:0/3001089605 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd708012910 con 0x55be53ed1dd0 2023-07-21T18:41:23.636 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.602+0000 7fd6f7fff700 1 -- 172.21.15.39:0/3001089605 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70800f870 con 0x55be53ed1dd0 2023-07-21T18:41:23.636 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.603+0000 7fd7203b2b80 1 -- 172.21.15.39:0/3001089605 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be53ed1dd0 msgr2=0x55be53ed2690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:23.636 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.603+0000 7fd7203b2b80 1 --2- 172.21.15.39:0/3001089605 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be53ed1dd0 0x55be53ed2690 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd708009de0 tx=0x7fd7080116c0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.636 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.603+0000 7fd7203b2b80 1 -- 172.21.15.39:0/3001089605 shutdown_connections 2023-07-21T18:41:23.636 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.603+0000 7fd7203b2b80 1 --2- 172.21.15.39:0/3001089605 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be53ed1480 0x55be53ed1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.636 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.603+0000 7fd7203b2b80 1 --2- 172.21.15.39:0/3001089605 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be53ed2c20 0x55be53ed3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.636 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.603+0000 7fd7203b2b80 1 --2- 172.21.15.39:0/3001089605 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be53ed1dd0 0x55be53ed2690 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.636 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.603+0000 7fd7203b2b80 1 -- 172.21.15.39:0/3001089605 >> 172.21.15.39:0/3001089605 conn(0x55be53ec6d50 msgr2=0x55be53ecc180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:23.637 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.603+0000 7fd7203b2b80 1 -- 172.21.15.39:0/3001089605 shutdown_connections 2023-07-21T18:41:23.637 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.603+0000 7fd7203b2b80 1 -- 172.21.15.39:0/3001089605 wait complete. 2023-07-21T18:41:23.637 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.604+0000 7fd7203b2b80 1 -- start start 2023-07-21T18:41:23.637 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.604+0000 7fd7203b2b80 1 Processor -- start 2023-07-21T18:41:23.637 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.604+0000 7fd7203b2b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be53ed1480 0x55be53edb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:23.637 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.604+0000 7fd7203b2b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be53ed1dd0 0x55be53edb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:23.637 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.604+0000 7fd7203b2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be53ed2c20 0x55be53ee2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:23.638 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.604+0000 7fd7203b2b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be53dc2a70 con 0x55be53ed1480 2023-07-21T18:41:23.638 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.604+0000 7fd7203b2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be53dc2770 con 0x55be53ed2c20 2023-07-21T18:41:23.638 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.604+0000 7fd7203b2b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be53ca68d0 con 0x55be53ed1dd0 2023-07-21T18:41:23.638 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.604+0000 7fd711fc8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be53ed1480 0x55be53edb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:23.638 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.604+0000 7fd711fc8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be53ed1480 0x55be53edb0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56344/0 (socket says 172.21.15.39:56344) 2023-07-21T18:41:23.638 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.604+0000 7fd711fc8700 1 -- 172.21.15.39:0/3506739319 learned_addr learned my addr 172.21.15.39:0/3506739319 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:23.638 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.604+0000 7fd7203b2b80 10 client.0 _ll_register_callbacks cb 0x55be53dc0710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:41:23.638 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.604+0000 7fd7117c7700 1 --2- 172.21.15.39:0/3506739319 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be53ed1dd0 0x55be53edb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:23.639 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.604+0000 7fd7127c9700 1 --2- 172.21.15.39:0/3506739319 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be53ed2c20 0x55be53ee2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:23.639 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.605+0000 7fd711fc8700 1 -- 172.21.15.39:0/3506739319 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be53ed1dd0 msgr2=0x55be53edb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:23.639 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.605+0000 7fd711fc8700 1 --2- 172.21.15.39:0/3506739319 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be53ed1dd0 0x55be53edb5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.639 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.605+0000 7fd711fc8700 1 -- 172.21.15.39:0/3506739319 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be53ed2c20 msgr2=0x55be53ee2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:23.639 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.605+0000 7fd711fc8700 1 --2- 172.21.15.39:0/3506739319 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be53ed2c20 0x55be53ee2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.639 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.605+0000 7fd711fc8700 1 -- 172.21.15.39:0/3506739319 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd708009a10 con 0x55be53ed1480 2023-07-21T18:41:23.639 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.605+0000 7fd7127c9700 1 --2- 172.21.15.39:0/3506739319 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be53ed2c20 0x55be53ee2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:41:23.640 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.605+0000 7fd711fc8700 1 --2- 172.21.15.39:0/3506739319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be53ed1480 0x55be53edb0b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc015000 tx=0x7fd6fc009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=912d8e4346fa0a79 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:23.640 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.605+0000 7fd6f57fa700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:41:23.640 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.605+0000 7fd6f57fa700 1 -- 172.21.15.39:0/3506739319 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc004da0 con 0x55be53ed1480 2023-07-21T18:41:23.640 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.605+0000 7fd6f57fa700 1 -- 172.21.15.39:0/3506739319 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fd6fc0056a0 con 0x55be53ed1480 2023-07-21T18:41:23.640 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.605+0000 7fd7203b2b80 1 -- 172.21.15.39:0/3506739319 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55be53d7c4a0 con 0x55be53ed1480 2023-07-21T18:41:23.640 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.605+0000 7fd6f57fa700 1 -- 172.21.15.39:0/3506739319 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd6fc00f460 con 0x55be53ed1480 2023-07-21T18:41:23.640 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.605+0000 7fd7203b2b80 10 client.15216 fetch_fsmap learned FSMap version 26 2023-07-21T18:41:23.640 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.605+0000 7fd7203b2b80 1 -- 172.21.15.39:0/3506739319 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55be53c39c10 con 0x55be53ed1480 2023-07-21T18:41:23.641 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.605+0000 7fd6f57fa700 1 -- 172.21.15.39:0/3506739319 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55be53d7c4a0 con 0x55be53ed1480 2023-07-21T18:41:23.641 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.605+0000 7fd6f57fa700 1 -- 172.21.15.39:0/3506739319 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd6fc030690 con 0x55be53ed1480 2023-07-21T18:41:23.641 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.606+0000 7fd7203b2b80 10 client.15216 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T18:41:23.641 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.606+0000 7fd7203b2b80 10 client.15216 resolve_mds: resolved name 'a' to daemon mds.a 2023-07-21T18:41:23.641 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.606+0000 7fd7203b2b80 20 client.15216 populate_metadata read hostname 'smithi039' 2023-07-21T18:41:23.641 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.606+0000 7fd7203b2b80 1 --2- 172.21.15.39:0/3506739319 >> [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] conn(0x55be53ee62f0 0x55be53ee56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:23.641 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.606+0000 7fd7203b2b80 4 client.15216 mds_command: new command op to 14604 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:41:23.641 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.606+0000 7fd7203b2b80 1 -- 172.21.15.39:0/3506739319 --> [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55be53e1be60 con 0x55be53ee62f0 2023-07-21T18:41:23.642 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.606+0000 7fd7117c7700 1 --2- 172.21.15.39:0/3506739319 >> [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] conn(0x55be53ee62f0 0x55be53ee56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:23.642 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.606+0000 7fd6f57fa700 10 client.15216 ms_handle_connect on v2:172.21.15.39:6840/640981220 2023-07-21T18:41:23.642 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.606+0000 7fd7117c7700 1 --2- 172.21.15.39:0/3506739319 >> [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] conn(0x55be53ee62f0 0x55be53ee56f0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=57a73b6ae9c04e70 server_cookie=656366c02c51a3be in_seq=0 out_seq=0 2023-07-21T18:41:23.642 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.607+0000 7fd6f57fa700 1 -- 172.21.15.39:0/3506739319 <== mds.4 v2:172.21.15.39:6840/640981220 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55be53e1be60 con 0x55be53ee62f0 2023-07-21T18:41:23.642 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.607+0000 7fd6f57fa700 10 client.15216 handle_command_reply: tid=0 2023-07-21T18:41:23.642 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.607+0000 7fd7203b2b80 1 client.15216 shutdown 2023-07-21T18:41:23.642 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.607+0000 7fd7203b2b80 1 -- 172.21.15.39:0/3506739319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be53ed1480 msgr2=0x55be53edb0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:23.643 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.607+0000 7fd7203b2b80 1 --2- 172.21.15.39:0/3506739319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be53ed1480 0x55be53edb0b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc015000 tx=0x7fd6fc009ca0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.643 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.607+0000 7fd7203b2b80 1 -- 172.21.15.39:0/3506739319 shutdown_connections 2023-07-21T18:41:23.643 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.607+0000 7fd7203b2b80 1 --2- 172.21.15.39:0/3506739319 >> [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] conn(0x55be53ee62f0 0x55be53ee56f0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.643 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.607+0000 7fd7203b2b80 1 --2- 172.21.15.39:0/3506739319 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be53ed2c20 0x55be53ee2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.643 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.607+0000 7fd7203b2b80 1 --2- 172.21.15.39:0/3506739319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be53ed1480 0x55be53edb0b0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.643 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.607+0000 7fd6f57fa700 0 client.15216 ms_handle_reset on v2:172.21.15.39:6840/640981220 2023-07-21T18:41:23.643 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.607+0000 7fd7203b2b80 1 --2- 172.21.15.39:0/3506739319 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be53ed1dd0 0x55be53edb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.643 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.607+0000 7fd7203b2b80 1 -- 172.21.15.39:0/3506739319 >> 172.21.15.39:0/3506739319 conn(0x55be53ec6d50 msgr2=0x55be53ecb860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:23.644 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.607+0000 7fd7203b2b80 1 -- 172.21.15.39:0/3506739319 shutdown_connections 2023-07-21T18:41:23.644 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.607+0000 7fd7203b2b80 1 -- 172.21.15.39:0/3506739319 wait complete. 2023-07-21T18:41:23.644 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.607+0000 7fd7203b2b80 20 client.15216 trim_cache size 0 max 16384 2023-07-21T18:41:23.649 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.625+0000 7fd7203b2b80 1 Processor -- start 2023-07-21T18:41:23.649 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.625+0000 7fd7203b2b80 1 -- start start 2023-07-21T18:41:23.650 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.625+0000 7fd7203b2b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be53ed1480 0x55be53ed9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:23.650 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.625+0000 7fd7203b2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be53ed1dd0 0x55be53ee25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:23.650 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.625+0000 7fd7203b2b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be53ed2c20 0x55be53ed9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:23.650 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.625+0000 7fd7203b2b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be53df1790 con 0x55be53ed2c20 2023-07-21T18:41:23.650 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.625+0000 7fd7203b2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be53cb2de0 con 0x55be53ed1dd0 2023-07-21T18:41:23.650 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.625+0000 7fd7203b2b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be53de1480 con 0x55be53ed1480 2023-07-21T18:41:23.651 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.625+0000 7fd7127c9700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be53ed2c20 0x55be53ed9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:23.651 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.625+0000 7fd7127c9700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be53ed2c20 0x55be53ed9030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56356/0 (socket says 172.21.15.39:56356) 2023-07-21T18:41:23.651 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.625+0000 7fd7127c9700 1 -- 172.21.15.39:0/3169204322 learned_addr learned my addr 172.21.15.39:0/3169204322 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:23.651 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.625+0000 7fd711fc8700 1 --2- 172.21.15.39:0/3169204322 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be53ed1480 0x55be53ed9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:23.651 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.625+0000 7fd7127c9700 1 -- 172.21.15.39:0/3169204322 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be53ed1480 msgr2=0x55be53ed9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:23.651 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.625+0000 7fd7127c9700 1 --2- 172.21.15.39:0/3169204322 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be53ed1480 0x55be53ed9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.651 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.625+0000 7fd7117c7700 1 --2- 172.21.15.39:0/3169204322 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be53ed1dd0 0x55be53ee25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:23.652 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.625+0000 7fd7127c9700 1 -- 172.21.15.39:0/3169204322 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be53ed1dd0 msgr2=0x55be53ee25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:23.652 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.625+0000 7fd7127c9700 1 --2- 172.21.15.39:0/3169204322 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be53ed1dd0 0x55be53ee25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.652 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.625+0000 7fd7127c9700 1 -- 172.21.15.39:0/3169204322 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be53c77200 con 0x55be53ed2c20 2023-07-21T18:41:23.652 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.625+0000 7fd711fc8700 1 --2- 172.21.15.39:0/3169204322 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be53ed1480 0x55be53ed9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:41:23.652 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.625+0000 7fd7127c9700 1 --2- 172.21.15.39:0/3169204322 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be53ed2c20 0x55be53ed9030 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd704099260 tx=0x7fd7040b9040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afe096b9440f3fb8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:23.652 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.626+0000 7fd700ff9700 1 -- 172.21.15.39:0/3169204322 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70409d590 con 0x55be53ed2c20 2023-07-21T18:41:23.653 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.626+0000 7fd700ff9700 1 -- 172.21.15.39:0/3169204322 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7040a19b0 con 0x55be53ed2c20 2023-07-21T18:41:23.653 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.626+0000 7fd700ff9700 1 -- 172.21.15.39:0/3169204322 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70409d150 con 0x55be53ed2c20 2023-07-21T18:41:23.653 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.626+0000 7fd7203b2b80 1 -- 172.21.15.39:0/3169204322 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be53ed2c20 msgr2=0x55be53ed9030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:23.653 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.626+0000 7fd7203b2b80 1 --2- 172.21.15.39:0/3169204322 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be53ed2c20 0x55be53ed9030 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd704099260 tx=0x7fd7040b9040 comp rx=0 tx=0).stop 2023-07-21T18:41:23.653 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.626+0000 7fd7203b2b80 1 -- 172.21.15.39:0/3169204322 shutdown_connections 2023-07-21T18:41:23.653 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.626+0000 7fd7203b2b80 1 --2- 172.21.15.39:0/3169204322 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be53ed2c20 0x55be53ed9030 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.653 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.626+0000 7fd7203b2b80 1 --2- 172.21.15.39:0/3169204322 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be53ed1480 0x55be53ed9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.654 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.626+0000 7fd7203b2b80 1 --2- 172.21.15.39:0/3169204322 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be53ed1dd0 0x55be53ee25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.654 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.626+0000 7fd7203b2b80 1 -- 172.21.15.39:0/3169204322 >> 172.21.15.39:0/3169204322 conn(0x55be53ec6d50 msgr2=0x55be53eef080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:23.654 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.626+0000 7fd7203b2b80 1 -- 172.21.15.39:0/3169204322 shutdown_connections 2023-07-21T18:41:23.654 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.626+0000 7fd7203b2b80 1 -- 172.21.15.39:0/3169204322 wait complete. 2023-07-21T18:41:23.657 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.627+0000 7fd7203b2b80 1 -- start start 2023-07-21T18:41:23.657 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.627+0000 7fd7203b2b80 1 Processor -- start 2023-07-21T18:41:23.658 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.627+0000 7fd7203b2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be53ed1480 0x55be53edd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:23.658 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.627+0000 7fd7203b2b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be53ed1dd0 0x55be53edd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:23.658 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.627+0000 7fd7203b2b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be53ed2c20 0x55be53f3a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:23.658 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.627+0000 7fd7203b2b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be53eddd60 con 0x55be53ed1dd0 2023-07-21T18:41:23.658 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.627+0000 7fd7203b2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be53edded0 con 0x55be53ed1480 2023-07-21T18:41:23.658 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.627+0000 7fd7203b2b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be53f3a6f0 con 0x55be53ed2c20 2023-07-21T18:41:23.658 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.627+0000 7fd7203b2b80 10 client.0 _ll_register_callbacks cb 0x7fd6fc00bb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:41:23.658 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.627+0000 7fd7117c7700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be53ed1dd0 0x55be53edd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:23.659 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.627+0000 7fd7117c7700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be53ed1dd0 0x55be53edd820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56360/0 (socket says 172.21.15.39:56360) 2023-07-21T18:41:23.659 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.627+0000 7fd7117c7700 1 -- 172.21.15.39:0/3253530697 learned_addr learned my addr 172.21.15.39:0/3253530697 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:23.659 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.627+0000 7fd711fc8700 1 --2- 172.21.15.39:0/3253530697 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be53ed1480 0x55be53edd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:23.659 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.628+0000 7fd7127c9700 1 --2- 172.21.15.39:0/3253530697 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be53ed2c20 0x55be53f3a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:23.659 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.628+0000 7fd7117c7700 1 -- 172.21.15.39:0/3253530697 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be53ed2c20 msgr2=0x55be53f3a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:23.659 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.628+0000 7fd7117c7700 1 --2- 172.21.15.39:0/3253530697 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be53ed2c20 0x55be53f3a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.659 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.628+0000 7fd7117c7700 1 -- 172.21.15.39:0/3253530697 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be53ed1480 msgr2=0x55be53edd2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:23.660 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.628+0000 7fd7117c7700 1 --2- 172.21.15.39:0/3253530697 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be53ed1480 0x55be53edd2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.660 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.628+0000 7fd7117c7700 1 -- 172.21.15.39:0/3253530697 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd70c1a00b0 con 0x55be53ed1dd0 2023-07-21T18:41:23.660 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.628+0000 7fd711fc8700 1 --2- 172.21.15.39:0/3253530697 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be53ed1480 0x55be53edd2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:41:23.660 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.628+0000 7fd7117c7700 1 --2- 172.21.15.39:0/3253530697 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be53ed1dd0 0x55be53edd820 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd708011ac0 tx=0x7fd708011ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=470085ce4aa066f9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:23.660 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.628+0000 7fd6f57fa700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:41:23.660 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.628+0000 7fd6f57fa700 1 -- 172.21.15.39:0/3253530697 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70802a070 con 0x55be53ed1dd0 2023-07-21T18:41:23.660 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.628+0000 7fd7203b2b80 1 -- 172.21.15.39:0/3253530697 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd708012910 con 0x55be53ed1dd0 2023-07-21T18:41:23.661 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.628+0000 7fd6f57fa700 1 -- 172.21.15.39:0/3253530697 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fd708009830 con 0x55be53ed1dd0 2023-07-21T18:41:23.661 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.628+0000 7fd6f57fa700 1 -- 172.21.15.39:0/3253530697 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd708012450 con 0x55be53ed1dd0 2023-07-21T18:41:23.661 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.628+0000 7fd7203b2b80 10 client.15222 fetch_fsmap learned FSMap version 26 2023-07-21T18:41:23.661 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.628+0000 7fd7203b2b80 1 -- 172.21.15.39:0/3253530697 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55be53c4f260 con 0x55be53ed1dd0 2023-07-21T18:41:23.661 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.628+0000 7fd6f57fa700 1 -- 172.21.15.39:0/3253530697 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd708012910 con 0x55be53ed1dd0 2023-07-21T18:41:23.661 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.628+0000 7fd6f57fa700 1 -- 172.21.15.39:0/3253530697 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd7080113e0 con 0x55be53ed1dd0 2023-07-21T18:41:23.661 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.628+0000 7fd7203b2b80 10 client.15222 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T18:41:23.662 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.628+0000 7fd7203b2b80 10 client.15222 resolve_mds: resolved name 'a' to daemon mds.a 2023-07-21T18:41:23.662 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.628+0000 7fd7203b2b80 20 client.15222 populate_metadata read hostname 'smithi039' 2023-07-21T18:41:23.662 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.629+0000 7fd7203b2b80 1 --2- 172.21.15.39:0/3253530697 >> [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] conn(0x55be53f3f490 0x55be53f3be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:23.662 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.629+0000 7fd7203b2b80 4 client.15222 mds_command: new command op to 14604 tid=0[{"prefix": "status"}] 2023-07-21T18:41:23.662 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.629+0000 7fd7203b2b80 1 -- 172.21.15.39:0/3253530697 --> [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55be53dde000 con 0x55be53f3f490 2023-07-21T18:41:23.662 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.629+0000 7fd711fc8700 1 --2- 172.21.15.39:0/3253530697 >> [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] conn(0x55be53f3f490 0x55be53f3be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:23.662 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.629+0000 7fd6f57fa700 10 client.15222 ms_handle_connect on v2:172.21.15.39:6840/640981220 2023-07-21T18:41:23.662 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.629+0000 7fd711fc8700 1 --2- 172.21.15.39:0/3253530697 >> [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] conn(0x55be53f3f490 0x55be53f3be00 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=eda8cc0a93382073 server_cookie=6bf252f434808d80 in_seq=0 out_seq=0 2023-07-21T18:41:23.663 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.629+0000 7fd6f57fa700 1 -- 172.21.15.39:0/3253530697 <== mds.4 v2:172.21.15.39:6840/640981220 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55be53dde000 con 0x55be53f3f490 2023-07-21T18:41:23.663 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.629+0000 7fd6f57fa700 10 client.15222 handle_command_reply: tid=0 2023-07-21T18:41:23.663 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.629+0000 7fd7203b2b80 1 client.15222 shutdown 2023-07-21T18:41:23.663 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.629+0000 7fd7203b2b80 1 -- 172.21.15.39:0/3253530697 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be53ed1dd0 msgr2=0x55be53edd820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:23.663 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.629+0000 7fd7203b2b80 1 --2- 172.21.15.39:0/3253530697 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be53ed1dd0 0x55be53edd820 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd708011ac0 tx=0x7fd708011ea0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.663 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.629+0000 7fd7203b2b80 1 -- 172.21.15.39:0/3253530697 shutdown_connections 2023-07-21T18:41:23.663 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.629+0000 7fd7203b2b80 1 --2- 172.21.15.39:0/3253530697 >> [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] conn(0x55be53f3f490 0x55be53f3be00 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.663 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.630+0000 7fd7203b2b80 1 --2- 172.21.15.39:0/3253530697 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be53ed1480 0x55be53edd2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.664 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.630+0000 7fd6f57fa700 0 client.15222 ms_handle_reset on v2:172.21.15.39:6840/640981220 2023-07-21T18:41:23.664 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.630+0000 7fd7203b2b80 1 --2- 172.21.15.39:0/3253530697 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be53ed2c20 0x55be53f3a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.664 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.630+0000 7fd7203b2b80 1 --2- 172.21.15.39:0/3253530697 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be53ed1dd0 0x55be53edd820 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.664 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.630+0000 7fd7203b2b80 1 -- 172.21.15.39:0/3253530697 >> 172.21.15.39:0/3253530697 conn(0x55be53ec6d50 msgr2=0x55be53ed9e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:23.664 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.630+0000 7fd7203b2b80 1 -- 172.21.15.39:0/3253530697 shutdown_connections 2023-07-21T18:41:23.664 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.630+0000 7fd7203b2b80 1 -- 172.21.15.39:0/3253530697 wait complete. 2023-07-21T18:41:23.665 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.630+0000 7fd7203b2b80 20 client.15222 trim_cache size 0 max 16384 2023-07-21T18:41:23.665 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2023-07-21T18:41:23.665 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "eaf8e25a-27f4-11ee-9b34-001a4aab830c", 2023-07-21T18:41:23.665 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 4, 2023-07-21T18:41:23.666 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 14604, 2023-07-21T18:41:23.666 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2023-07-21T18:41:23.666 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2023-07-21T18:41:23.666 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2023-07-21T18:41:23.666 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 72.664703635999999, 2023-07-21T18:41:23.666 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 26, 2023-07-21T18:41:23.666 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 81, 2023-07-21T18:41:23.667 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 79, 2023-07-21T18:41:23.667 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 87.593698239999995 2023-07-21T18:41:23.667 INFO:teuthology.orchestra.run.smithi039.stdout:} 2023-07-21T18:41:23.668 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.630+0000 7fd71422c700 1 -- 172.21.15.39:0/2386669356 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fd6f80a1260 msgr2=0x7fd6f80a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:23.668 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.630+0000 7fd71422c700 1 --2- 172.21.15.39:0/2386669356 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fd6f80a1260 0x7fd6f80a3710 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc006010 tx=0x7fd6fc00b260 comp rx=0 tx=0).stop 2023-07-21T18:41:23.668 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.630+0000 7fd71422c700 1 -- 172.21.15.39:0/2386669356 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd70c100590 msgr2=0x7fd70c101520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:23.668 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.630+0000 7fd71422c700 1 --2- 172.21.15.39:0/2386669356 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd70c100590 0x7fd70c101520 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd704007ee0 tx=0x7fd70400d3b0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.669 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.630+0000 7fd71422c700 1 -- 172.21.15.39:0/2386669356 shutdown_connections 2023-07-21T18:41:23.669 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.630+0000 7fd71422c700 1 --2- 172.21.15.39:0/2386669356 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fd6f80a1260 0x7fd6f80a3710 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.669 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.630+0000 7fd71422c700 1 --2- 172.21.15.39:0/2386669356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd70c0fea10 0x7fd70c1029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.669 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.630+0000 7fd71422c700 1 --2- 172.21.15.39:0/2386669356 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd70c100590 0x7fd70c101520 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.669 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.630+0000 7fd71422c700 1 --2- 172.21.15.39:0/2386669356 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd70c0ffc00 0x7fd70c100fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.669 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.630+0000 7fd71422c700 1 -- 172.21.15.39:0/2386669356 >> 172.21.15.39:0/2386669356 conn(0x7fd70c0fa030 msgr2=0x7fd70c068680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:23.669 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.630+0000 7fd71422c700 1 -- 172.21.15.39:0/2386669356 shutdown_connections 2023-07-21T18:41:23.670 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.630+0000 7fd71422c700 1 -- 172.21.15.39:0/2386669356 wait complete. 2023-07-21T18:41:23.670 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2023-07-21T18:41:23.797 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.774+0000 7fdf37199700 1 -- 172.21.15.39:0/3687430423 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf300ffbd0 msgr2=0x7fdf30100020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:23.798 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.774+0000 7fdf37199700 1 --2- 172.21.15.39:0/3687430423 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf300ffbd0 0x7fdf30100020 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c009990 tx=0x7fdf1c009ca0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.798 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.774+0000 7fdf37199700 1 -- 172.21.15.39:0/3687430423 shutdown_connections 2023-07-21T18:41:23.798 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.774+0000 7fdf37199700 1 --2- 172.21.15.39:0/3687430423 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf30100560 0x7fdf30106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.798 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.774+0000 7fdf37199700 1 --2- 172.21.15.39:0/3687430423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf300fe9e0 0x7fdf300fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.798 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.774+0000 7fdf37199700 1 --2- 172.21.15.39:0/3687430423 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf300ffbd0 0x7fdf30100020 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.798 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.774+0000 7fdf37199700 1 -- 172.21.15.39:0/3687430423 >> 172.21.15.39:0/3687430423 conn(0x7fdf300fa000 msgr2=0x7fdf300fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:23.798 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.774+0000 7fdf37199700 1 -- 172.21.15.39:0/3687430423 shutdown_connections 2023-07-21T18:41:23.799 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.774+0000 7fdf37199700 1 -- 172.21.15.39:0/3687430423 wait complete. 2023-07-21T18:41:23.799 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.775+0000 7fdf37199700 1 Processor -- start 2023-07-21T18:41:23.799 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.775+0000 7fdf37199700 1 -- start start 2023-07-21T18:41:23.800 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.775+0000 7fdf37199700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf300fe9e0 0x7fdf30068200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:23.801 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.775+0000 7fdf37199700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf300ffbd0 0x7fdf3019a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:23.801 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.775+0000 7fdf37199700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf30100560 0x7fdf3019cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:23.801 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.775+0000 7fdf37199700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3019d330 con 0x7fdf300fe9e0 2023-07-21T18:41:23.801 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.775+0000 7fdf37199700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3019d4a0 con 0x7fdf300ffbd0 2023-07-21T18:41:23.801 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.775+0000 7fdf37199700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3019d660 con 0x7fdf30100560 2023-07-21T18:41:23.801 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.775+0000 7fdf34f35700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf300fe9e0 0x7fdf30068200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:23.802 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.775+0000 7fdf34f35700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf300fe9e0 0x7fdf30068200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56372/0 (socket says 172.21.15.39:56372) 2023-07-21T18:41:23.802 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.775+0000 7fdf34f35700 1 -- 172.21.15.39:0/3287849270 learned_addr learned my addr 172.21.15.39:0/3287849270 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:23.802 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.775+0000 7fdf35736700 1 --2- 172.21.15.39:0/3287849270 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf30100560 0x7fdf3019cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:23.802 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.775+0000 7fdf34f35700 1 -- 172.21.15.39:0/3287849270 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf30100560 msgr2=0x7fdf3019cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:23.803 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.775+0000 7fdf34f35700 1 --2- 172.21.15.39:0/3287849270 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf30100560 0x7fdf3019cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.803 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.775+0000 7fdf34f35700 1 -- 172.21.15.39:0/3287849270 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf300ffbd0 msgr2=0x7fdf3019a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:23.803 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.775+0000 7fdf34f35700 1 --2- 172.21.15.39:0/3287849270 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf300ffbd0 0x7fdf3019a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.803 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.775+0000 7fdf34f35700 1 -- 172.21.15.39:0/3287849270 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf1c0095f0 con 0x7fdf300fe9e0 2023-07-21T18:41:23.803 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.775+0000 7fdf34f35700 1 --2- 172.21.15.39:0/3287849270 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf300fe9e0 0x7fdf30068200 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fdf2400d980 tx=0x7fdf2400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:23.803 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.775+0000 7fdf2dffb700 1 -- 172.21.15.39:0/3287849270 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2400de60 con 0x7fdf300fe9e0 2023-07-21T18:41:23.804 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.776+0000 7fdf2dffb700 1 -- 172.21.15.39:0/3287849270 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf240044e0 con 0x7fdf300fe9e0 2023-07-21T18:41:23.804 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.776+0000 7fdf37199700 1 -- 172.21.15.39:0/3287849270 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf3019d940 con 0x7fdf300fe9e0 2023-07-21T18:41:23.804 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.776+0000 7fdf2dffb700 1 -- 172.21.15.39:0/3287849270 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf24003be0 con 0x7fdf300fe9e0 2023-07-21T18:41:23.804 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.776+0000 7fdf37199700 1 -- 172.21.15.39:0/3287849270 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf3019de50 con 0x7fdf300fe9e0 2023-07-21T18:41:23.804 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.778+0000 7fdf2dffb700 1 -- 172.21.15.39:0/3287849270 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fdf24009bb0 con 0x7fdf300fe9e0 2023-07-21T18:41:23.808 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.778+0000 7fdf2dffb700 1 --2- 172.21.15.39:0/3287849270 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fdf200a11e0 0x7fdf200a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:23.809 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.778+0000 7fdf2dffb700 1 -- 172.21.15.39:0/3287849270 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fdf240cc810 con 0x7fdf300fe9e0 2023-07-21T18:41:23.809 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.778+0000 7fdf2ffff700 1 --2- 172.21.15.39:0/3287849270 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fdf200a11e0 0x7fdf200a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:23.809 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.779+0000 7fdf2ffff700 1 --2- 172.21.15.39:0/3287849270 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fdf200a11e0 0x7fdf200a3690 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c00b5c0 tx=0x7fdf1c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:23.809 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.779+0000 7fdf4331fb80 1 Processor -- start 2023-07-21T18:41:23.809 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.779+0000 7fdf4331fb80 1 -- start start 2023-07-21T18:41:23.809 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.779+0000 7fdf4331fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f45d64890 0x559f45d64ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:23.810 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.779+0000 7fdf4331fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f45d651e0 0x559f45d65aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:23.810 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.779+0000 7fdf4331fb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f45d66030 0x559f45d664a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:23.810 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.779+0000 7fdf4331fb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f45c65c60 con 0x559f45d66030 2023-07-21T18:41:23.810 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.779+0000 7fdf4331fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f45cb89f0 con 0x559f45d651e0 2023-07-21T18:41:23.810 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.779+0000 7fdf4331fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f45cb1e80 con 0x559f45d64890 2023-07-21T18:41:23.810 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.780+0000 7fdf35736700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f45d66030 0x559f45d664a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:23.810 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.780+0000 7fdf35736700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f45d66030 0x559f45d664a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56384/0 (socket says 172.21.15.39:56384) 2023-07-21T18:41:23.810 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.780+0000 7fdf35736700 1 -- 172.21.15.39:0/2535989313 learned_addr learned my addr 172.21.15.39:0/2535989313 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:23.811 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.780+0000 7fdf2ffff700 1 --2- 172.21.15.39:0/2535989313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f45d651e0 0x559f45d65aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:23.811 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.780+0000 7fdf34f35700 1 --2- 172.21.15.39:0/2535989313 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f45d64890 0x559f45d64ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:23.811 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.780+0000 7fdf35736700 1 -- 172.21.15.39:0/2535989313 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f45d64890 msgr2=0x559f45d64ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:23.811 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.780+0000 7fdf35736700 1 --2- 172.21.15.39:0/2535989313 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f45d64890 0x559f45d64ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.811 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.780+0000 7fdf35736700 1 -- 172.21.15.39:0/2535989313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f45d651e0 msgr2=0x559f45d65aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:23.811 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.780+0000 7fdf35736700 1 --2- 172.21.15.39:0/2535989313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f45d651e0 0x559f45d65aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.811 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.780+0000 7fdf35736700 1 -- 172.21.15.39:0/2535989313 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f45adfd90 con 0x559f45d66030 2023-07-21T18:41:23.811 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.780+0000 7fdf35736700 1 --2- 172.21.15.39:0/2535989313 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f45d66030 0x559f45d664a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fdf2800e190 tx=0x7fdf2800e4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fadca543ec5d651d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:23.812 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.780+0000 7fdf1affd700 1 -- 172.21.15.39:0/2535989313 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf28018070 con 0x559f45d66030 2023-07-21T18:41:23.812 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.780+0000 7fdf1affd700 1 -- 172.21.15.39:0/2535989313 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf28007c30 con 0x559f45d66030 2023-07-21T18:41:23.812 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.780+0000 7fdf1affd700 1 -- 172.21.15.39:0/2535989313 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf28014d80 con 0x559f45d66030 2023-07-21T18:41:23.812 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.781+0000 7fdf4331fb80 1 -- 172.21.15.39:0/2535989313 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f45d66030 msgr2=0x559f45d664a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:23.812 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.781+0000 7fdf4331fb80 1 --2- 172.21.15.39:0/2535989313 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f45d66030 0x559f45d664a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fdf2800e190 tx=0x7fdf2800e4a0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.813 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.781+0000 7fdf4331fb80 1 -- 172.21.15.39:0/2535989313 shutdown_connections 2023-07-21T18:41:23.813 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.781+0000 7fdf4331fb80 1 --2- 172.21.15.39:0/2535989313 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f45d66030 0x559f45d664a0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.813 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.781+0000 7fdf4331fb80 1 --2- 172.21.15.39:0/2535989313 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f45d64890 0x559f45d64ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.813 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.781+0000 7fdf4331fb80 1 --2- 172.21.15.39:0/2535989313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f45d651e0 0x559f45d65aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.813 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.781+0000 7fdf4331fb80 1 -- 172.21.15.39:0/2535989313 >> 172.21.15.39:0/2535989313 conn(0x559f45d55d50 msgr2=0x559f45d5b180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:23.813 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.781+0000 7fdf4331fb80 1 -- 172.21.15.39:0/2535989313 shutdown_connections 2023-07-21T18:41:23.813 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.781+0000 7fdf4331fb80 1 -- 172.21.15.39:0/2535989313 wait complete. 2023-07-21T18:41:23.813 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.781+0000 7fdf4331fb80 1 -- start start 2023-07-21T18:41:23.814 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.781+0000 7fdf4331fb80 1 Processor -- start 2023-07-21T18:41:23.814 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.782+0000 7fdf4331fb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f45d64890 0x559f45d6e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:23.814 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.782+0000 7fdf4331fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f45d651e0 0x559f45d6e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:23.814 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.782+0000 7fdf4331fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f45d66030 0x559f45d75a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:23.814 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.782+0000 7fdf4331fb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f45c51a70 con 0x559f45d64890 2023-07-21T18:41:23.814 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.782+0000 7fdf4331fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f45c51770 con 0x559f45d66030 2023-07-21T18:41:23.814 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.782+0000 7fdf4331fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f45b358d0 con 0x559f45d651e0 2023-07-21T18:41:23.815 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.782+0000 7fdf4331fb80 10 client.0 _ll_register_callbacks cb 0x559f45c4f710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:41:23.815 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.782+0000 7fdf34f35700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f45d64890 0x559f45d6e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:23.815 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.782+0000 7fdf34f35700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f45d64890 0x559f45d6e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56386/0 (socket says 172.21.15.39:56386) 2023-07-21T18:41:23.815 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.782+0000 7fdf34f35700 1 -- 172.21.15.39:0/2907803365 learned_addr learned my addr 172.21.15.39:0/2907803365 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:23.815 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.782+0000 7fdf2ffff700 1 --2- 172.21.15.39:0/2907803365 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f45d651e0 0x559f45d6e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:23.815 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.782+0000 7fdf35736700 1 --2- 172.21.15.39:0/2907803365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f45d66030 0x559f45d75a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:23.815 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.782+0000 7fdf34f35700 1 -- 172.21.15.39:0/2907803365 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f45d651e0 msgr2=0x559f45d6e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:23.815 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.782+0000 7fdf34f35700 1 --2- 172.21.15.39:0/2907803365 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f45d651e0 0x559f45d6e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.816 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.782+0000 7fdf34f35700 1 -- 172.21.15.39:0/2907803365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f45d66030 msgr2=0x559f45d75a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:23.816 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.782+0000 7fdf34f35700 1 --2- 172.21.15.39:0/2907803365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f45d66030 0x559f45d75a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.816 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.782+0000 7fdf34f35700 1 -- 172.21.15.39:0/2907803365 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf2800f040 con 0x559f45d64890 2023-07-21T18:41:23.816 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.782+0000 7fdf35736700 1 --2- 172.21.15.39:0/2907803365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f45d66030 0x559f45d75a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:41:23.816 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.782+0000 7fdf34f35700 1 --2- 172.21.15.39:0/2907803365 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f45d64890 0x559f45d6e4a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fdf24096560 tx=0x7fdf2409db20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9ec2dec99c96001 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:23.816 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.782+0000 7fdf0ffff700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:41:23.816 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.782+0000 7fdf0ffff700 1 -- 172.21.15.39:0/2907803365 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf24098fe0 con 0x559f45d64890 2023-07-21T18:41:23.817 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.782+0000 7fdf0ffff700 1 -- 172.21.15.39:0/2907803365 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fdf24099140 con 0x559f45d64890 2023-07-21T18:41:23.817 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.783+0000 7fdf4331fb80 1 -- 172.21.15.39:0/2907803365 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559f45caae60 con 0x559f45d64890 2023-07-21T18:41:23.817 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.783+0000 7fdf0ffff700 1 -- 172.21.15.39:0/2907803365 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf240a1ab0 con 0x559f45d64890 2023-07-21T18:41:23.817 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.783+0000 7fdf4331fb80 10 client.15234 fetch_fsmap learned FSMap version 26 2023-07-21T18:41:23.817 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.783+0000 7fdf4331fb80 1 -- 172.21.15.39:0/2907803365 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x559f45ac8c10 con 0x559f45d64890 2023-07-21T18:41:23.817 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.783+0000 7fdf0ffff700 1 -- 172.21.15.39:0/2907803365 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559f45caae60 con 0x559f45d64890 2023-07-21T18:41:23.817 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.783+0000 7fdf0ffff700 1 -- 172.21.15.39:0/2907803365 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fdf24098c50 con 0x559f45d64890 2023-07-21T18:41:23.817 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.783+0000 7fdf4331fb80 10 client.15234 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T18:41:23.818 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.783+0000 7fdf4331fb80 10 client.15234 resolve_mds: resolved name 'e' to daemon mds.e 2023-07-21T18:41:23.818 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.783+0000 7fdf4331fb80 20 client.15234 populate_metadata read hostname 'smithi039' 2023-07-21T18:41:23.818 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.783+0000 7fdf4331fb80 1 --2- 172.21.15.39:0/2907803365 >> [v2:172.21.15.155:6834/3854856286,v1:172.21.15.155:6835/3854856286] conn(0x559f45d796f0 0x559f45d78b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:23.818 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.783+0000 7fdf4331fb80 4 client.15234 mds_command: new command op to 24464 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:41:23.819 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.783+0000 7fdf4331fb80 1 -- 172.21.15.39:0/2907803365 --> [v2:172.21.15.155:6834/3854856286,v1:172.21.15.155:6835/3854856286] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559f45c6d000 con 0x559f45d796f0 2023-07-21T18:41:23.819 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.783+0000 7fdf2ffff700 1 --2- 172.21.15.39:0/2907803365 >> [v2:172.21.15.155:6834/3854856286,v1:172.21.15.155:6835/3854856286] conn(0x559f45d796f0 0x559f45d78b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:23.819 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.784+0000 7fdf0ffff700 10 client.15234 ms_handle_connect on v2:172.21.15.155:6834/3854856286 2023-07-21T18:41:23.819 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.784+0000 7fdf2ffff700 1 --2- 172.21.15.39:0/2907803365 >> [v2:172.21.15.155:6834/3854856286,v1:172.21.15.155:6835/3854856286] conn(0x559f45d796f0 0x559f45d78b80 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d6efe64ae4345fb server_cookie=86c3e76b340efbb5 in_seq=0 out_seq=0 2023-07-21T18:41:23.820 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.785+0000 7fdf0ffff700 1 -- 172.21.15.39:0/2907803365 <== mds.3 v2:172.21.15.155:6834/3854856286 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559f45c6d000 con 0x559f45d796f0 2023-07-21T18:41:23.820 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.785+0000 7fdf0ffff700 10 client.15234 handle_command_reply: tid=0 2023-07-21T18:41:23.820 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.785+0000 7fdf4331fb80 1 client.15234 shutdown 2023-07-21T18:41:23.820 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.785+0000 7fdf4331fb80 1 -- 172.21.15.39:0/2907803365 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f45d64890 msgr2=0x559f45d6e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:23.821 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.785+0000 7fdf4331fb80 1 --2- 172.21.15.39:0/2907803365 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f45d64890 0x559f45d6e4a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fdf24096560 tx=0x7fdf2409db20 comp rx=0 tx=0).stop 2023-07-21T18:41:23.821 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.785+0000 7fdf4331fb80 1 -- 172.21.15.39:0/2907803365 shutdown_connections 2023-07-21T18:41:23.821 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.785+0000 7fdf4331fb80 1 --2- 172.21.15.39:0/2907803365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f45d66030 0x559f45d75a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.822 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.785+0000 7fdf4331fb80 1 --2- 172.21.15.39:0/2907803365 >> [v2:172.21.15.155:6834/3854856286,v1:172.21.15.155:6835/3854856286] conn(0x559f45d796f0 0x559f45d78b80 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.822 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.785+0000 7fdf4331fb80 1 --2- 172.21.15.39:0/2907803365 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f45d64890 0x559f45d6e4a0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.822 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.785+0000 7fdf0ffff700 0 client.15234 ms_handle_reset on v2:172.21.15.155:6834/3854856286 2023-07-21T18:41:23.822 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.785+0000 7fdf4331fb80 1 --2- 172.21.15.39:0/2907803365 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f45d651e0 0x559f45d6e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.823 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.785+0000 7fdf4331fb80 1 -- 172.21.15.39:0/2907803365 >> 172.21.15.39:0/2907803365 conn(0x559f45d55d50 msgr2=0x559f45d5a860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:23.823 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.785+0000 7fdf4331fb80 1 -- 172.21.15.39:0/2907803365 shutdown_connections 2023-07-21T18:41:23.823 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.785+0000 7fdf4331fb80 1 -- 172.21.15.39:0/2907803365 wait complete. 2023-07-21T18:41:23.823 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.785+0000 7fdf4331fb80 20 client.15234 trim_cache size 0 max 16384 2023-07-21T18:41:23.826 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.803+0000 7fdf4331fb80 1 Processor -- start 2023-07-21T18:41:23.827 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.803+0000 7fdf4331fb80 1 -- start start 2023-07-21T18:41:23.827 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.803+0000 7fdf4331fb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f45d64890 0x559f45d78b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:23.827 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.803+0000 7fdf4331fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f45d651e0 0x559f45d759b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:23.827 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.803+0000 7fdf4331fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f45d66030 0x559f45d68f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:23.827 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.803+0000 7fdf4331fb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f45d52fc0 con 0x559f45d64890 2023-07-21T18:41:23.827 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.803+0000 7fdf4331fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f45c6d000 con 0x559f45d651e0 2023-07-21T18:41:23.827 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.803+0000 7fdf4331fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f45c5b470 con 0x559f45d66030 2023-07-21T18:41:23.828 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.803+0000 7fdf34f35700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f45d64890 0x559f45d78b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:23.828 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.803+0000 7fdf34f35700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f45d64890 0x559f45d78b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56394/0 (socket says 172.21.15.39:56394) 2023-07-21T18:41:23.828 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.803+0000 7fdf34f35700 1 -- 172.21.15.39:0/959327457 learned_addr learned my addr 172.21.15.39:0/959327457 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:23.828 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.803+0000 7fdf35736700 1 --2- 172.21.15.39:0/959327457 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f45d66030 0x559f45d68f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:23.828 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.803+0000 7fdf34f35700 1 -- 172.21.15.39:0/959327457 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f45d66030 msgr2=0x559f45d68f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:23.828 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.803+0000 7fdf34f35700 1 --2- 172.21.15.39:0/959327457 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f45d66030 0x559f45d68f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.828 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.803+0000 7fdf2ffff700 1 --2- 172.21.15.39:0/959327457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f45d651e0 0x559f45d759b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:23.829 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.803+0000 7fdf34f35700 1 -- 172.21.15.39:0/959327457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f45d651e0 msgr2=0x559f45d759b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:23.829 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.803+0000 7fdf34f35700 1 --2- 172.21.15.39:0/959327457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f45d651e0 0x559f45d759b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.829 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.803+0000 7fdf34f35700 1 -- 172.21.15.39:0/959327457 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f45b06200 con 0x559f45d64890 2023-07-21T18:41:23.829 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.803+0000 7fdf2ffff700 1 --2- 172.21.15.39:0/959327457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f45d651e0 0x559f45d759b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:41:23.829 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.803+0000 7fdf34f35700 1 --2- 172.21.15.39:0/959327457 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f45d64890 0x559f45d78b80 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fdf24009520 tx=0x7fdf24003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89800784431adf server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:23.829 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.804+0000 7fdf1b7fe700 1 -- 172.21.15.39:0/959327457 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf240998c0 con 0x559f45d64890 2023-07-21T18:41:23.829 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.804+0000 7fdf1b7fe700 1 -- 172.21.15.39:0/959327457 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf24098fe0 con 0x559f45d64890 2023-07-21T18:41:23.829 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.804+0000 7fdf1b7fe700 1 -- 172.21.15.39:0/959327457 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf24095b40 con 0x559f45d64890 2023-07-21T18:41:23.833 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.804+0000 7fdf4331fb80 1 -- 172.21.15.39:0/959327457 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f45d64890 msgr2=0x559f45d78b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:23.833 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.804+0000 7fdf4331fb80 1 --2- 172.21.15.39:0/959327457 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f45d64890 0x559f45d78b80 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fdf24009520 tx=0x7fdf24003680 comp rx=0 tx=0).stop 2023-07-21T18:41:23.833 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.804+0000 7fdf4331fb80 1 -- 172.21.15.39:0/959327457 shutdown_connections 2023-07-21T18:41:23.833 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.804+0000 7fdf4331fb80 1 --2- 172.21.15.39:0/959327457 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f45d66030 0x559f45d68f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.834 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.804+0000 7fdf4331fb80 1 --2- 172.21.15.39:0/959327457 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f45d64890 0x559f45d78b80 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.834 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.804+0000 7fdf4331fb80 1 --2- 172.21.15.39:0/959327457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f45d651e0 0x559f45d759b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.834 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.804+0000 7fdf4331fb80 1 -- 172.21.15.39:0/959327457 >> 172.21.15.39:0/959327457 conn(0x559f45d55d50 msgr2=0x559f45d76250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:23.834 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.804+0000 7fdf4331fb80 1 -- 172.21.15.39:0/959327457 shutdown_connections 2023-07-21T18:41:23.834 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.804+0000 7fdf4331fb80 1 -- 172.21.15.39:0/959327457 wait complete. 2023-07-21T18:41:23.834 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.805+0000 7fdf4331fb80 1 -- start start 2023-07-21T18:41:23.834 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.805+0000 7fdf4331fb80 1 Processor -- start 2023-07-21T18:41:23.834 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.805+0000 7fdf4331fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f45d64890 0x559f45d71240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:23.835 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.805+0000 7fdf4331fb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f45d651e0 0x559f45d71780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:23.835 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.805+0000 7fdf4331fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f45d66030 0x559f45d745c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:23.835 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.805+0000 7fdf4331fb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f45d74b00 con 0x559f45d651e0 2023-07-21T18:41:23.835 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.805+0000 7fdf4331fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f45d74c70 con 0x559f45d64890 2023-07-21T18:41:23.835 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.805+0000 7fdf4331fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f45d74e30 con 0x559f45d66030 2023-07-21T18:41:23.835 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.805+0000 7fdf4331fb80 10 client.0 _ll_register_callbacks cb 0x7fdf24099490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:41:23.835 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.805+0000 7fdf2ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f45d651e0 0x559f45d71780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:23.835 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.805+0000 7fdf2ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f45d651e0 0x559f45d71780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56402/0 (socket says 172.21.15.39:56402) 2023-07-21T18:41:23.836 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.805+0000 7fdf2ffff700 1 -- 172.21.15.39:0/3561714106 learned_addr learned my addr 172.21.15.39:0/3561714106 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:23.836 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.805+0000 7fdf35736700 1 --2- 172.21.15.39:0/3561714106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f45d66030 0x559f45d745c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:23.836 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.805+0000 7fdf2ffff700 1 -- 172.21.15.39:0/3561714106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f45d66030 msgr2=0x559f45d745c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:23.836 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.805+0000 7fdf2ffff700 1 --2- 172.21.15.39:0/3561714106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f45d66030 0x559f45d745c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.836 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.805+0000 7fdf2ffff700 1 -- 172.21.15.39:0/3561714106 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f45d64890 msgr2=0x559f45d71240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:23.836 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.805+0000 7fdf34f35700 1 --2- 172.21.15.39:0/3561714106 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f45d64890 0x559f45d71240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:23.836 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.805+0000 7fdf2ffff700 1 --2- 172.21.15.39:0/3561714106 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f45d64890 0x559f45d71240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.837 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.806+0000 7fdf2ffff700 1 -- 172.21.15.39:0/3561714106 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559f45ac8c10 con 0x559f45d651e0 2023-07-21T18:41:23.837 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.806+0000 7fdf34f35700 1 --2- 172.21.15.39:0/3561714106 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f45d64890 0x559f45d71240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:41:23.837 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.806+0000 7fdf2ffff700 1 --2- 172.21.15.39:0/3561714106 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f45d651e0 0x559f45d71780 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fdf30067f70 tx=0x7fdf1c01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e9e7f88b515c2ba server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:23.837 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.806+0000 7fdf0ffff700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:41:23.837 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.806+0000 7fdf0ffff700 1 -- 172.21.15.39:0/3561714106 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c01d070 con 0x559f45d651e0 2023-07-21T18:41:23.837 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.806+0000 7fdf4331fb80 1 -- 172.21.15.39:0/3561714106 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559f45c0b4a0 con 0x559f45d651e0 2023-07-21T18:41:23.837 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.806+0000 7fdf0ffff700 1 -- 172.21.15.39:0/3561714106 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fdf1c032e30 con 0x559f45d651e0 2023-07-21T18:41:23.837 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.806+0000 7fdf0ffff700 1 -- 172.21.15.39:0/3561714106 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf1c00b770 con 0x559f45d651e0 2023-07-21T18:41:23.838 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.806+0000 7fdf4331fb80 10 client.15240 fetch_fsmap learned FSMap version 26 2023-07-21T18:41:23.838 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.806+0000 7fdf4331fb80 1 -- 172.21.15.39:0/3561714106 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x559f45ade260 con 0x559f45d651e0 2023-07-21T18:41:23.838 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.806+0000 7fdf0ffff700 1 -- 172.21.15.39:0/3561714106 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559f45c0b4a0 con 0x559f45d651e0 2023-07-21T18:41:23.838 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.806+0000 7fdf0ffff700 1 -- 172.21.15.39:0/3561714106 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fdf1c004bc0 con 0x559f45d651e0 2023-07-21T18:41:23.838 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.806+0000 7fdf4331fb80 10 client.15240 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T18:41:23.838 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.806+0000 7fdf4331fb80 10 client.15240 resolve_mds: resolved name 'e' to daemon mds.e 2023-07-21T18:41:23.838 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.806+0000 7fdf4331fb80 20 client.15240 populate_metadata read hostname 'smithi039' 2023-07-21T18:41:23.839 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.806+0000 7fdf4331fb80 1 --2- 172.21.15.39:0/3561714106 >> [v2:172.21.15.155:6834/3854856286,v1:172.21.15.155:6835/3854856286] conn(0x559f45dd30e0 0x559f45dcfa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:23.839 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.806+0000 7fdf4331fb80 4 client.15240 mds_command: new command op to 24464 tid=0[{"prefix": "status"}] 2023-07-21T18:41:23.839 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.806+0000 7fdf4331fb80 1 -- 172.21.15.39:0/3561714106 --> [v2:172.21.15.155:6834/3854856286,v1:172.21.15.155:6835/3854856286] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559f45b00f20 con 0x559f45dd30e0 2023-07-21T18:41:23.839 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.807+0000 7fdf34f35700 1 --2- 172.21.15.39:0/3561714106 >> [v2:172.21.15.155:6834/3854856286,v1:172.21.15.155:6835/3854856286] conn(0x559f45dd30e0 0x559f45dcfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:23.839 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.807+0000 7fdf0ffff700 10 client.15240 ms_handle_connect on v2:172.21.15.155:6834/3854856286 2023-07-21T18:41:23.839 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.807+0000 7fdf34f35700 1 --2- 172.21.15.39:0/3561714106 >> [v2:172.21.15.155:6834/3854856286,v1:172.21.15.155:6835/3854856286] conn(0x559f45dd30e0 0x559f45dcfa30 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dcb2cc15ca21b33b server_cookie=e15f3a47e05d287a in_seq=0 out_seq=0 2023-07-21T18:41:23.839 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.807+0000 7fdf0ffff700 1 -- 172.21.15.39:0/3561714106 <== mds.3 v2:172.21.15.155:6834/3854856286 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x559f45b00f20 con 0x559f45dd30e0 2023-07-21T18:41:23.839 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.807+0000 7fdf0ffff700 10 client.15240 handle_command_reply: tid=0 2023-07-21T18:41:23.840 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.807+0000 7fdf4331fb80 1 client.15240 shutdown 2023-07-21T18:41:23.840 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.807+0000 7fdf4331fb80 1 -- 172.21.15.39:0/3561714106 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f45d651e0 msgr2=0x559f45d71780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:23.840 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.807+0000 7fdf4331fb80 1 --2- 172.21.15.39:0/3561714106 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f45d651e0 0x559f45d71780 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fdf30067f70 tx=0x7fdf1c01d000 comp rx=0 tx=0).stop 2023-07-21T18:41:23.840 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.807+0000 7fdf4331fb80 1 -- 172.21.15.39:0/3561714106 shutdown_connections 2023-07-21T18:41:23.840 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.807+0000 7fdf4331fb80 1 --2- 172.21.15.39:0/3561714106 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f45d64890 0x559f45d71240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.840 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.807+0000 7fdf4331fb80 1 --2- 172.21.15.39:0/3561714106 >> [v2:172.21.15.155:6834/3854856286,v1:172.21.15.155:6835/3854856286] conn(0x559f45dd30e0 0x559f45dcfa30 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.840 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.807+0000 7fdf4331fb80 1 --2- 172.21.15.39:0/3561714106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f45d66030 0x559f45d745c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.841 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.807+0000 7fdf0ffff700 0 client.15240 ms_handle_reset on v2:172.21.15.155:6834/3854856286 2023-07-21T18:41:23.841 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.807+0000 7fdf4331fb80 1 --2- 172.21.15.39:0/3561714106 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f45d651e0 0x559f45d71780 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.841 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.807+0000 7fdf4331fb80 1 -- 172.21.15.39:0/3561714106 >> 172.21.15.39:0/3561714106 conn(0x559f45d55d50 msgr2=0x559f45d76250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:23.841 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.808+0000 7fdf4331fb80 1 -- 172.21.15.39:0/3561714106 shutdown_connections 2023-07-21T18:41:23.841 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.808+0000 7fdf4331fb80 1 -- 172.21.15.39:0/3561714106 wait complete. 2023-07-21T18:41:23.841 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.808+0000 7fdf4331fb80 20 client.15240 trim_cache size 0 max 16384 2023-07-21T18:41:23.841 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2023-07-21T18:41:23.842 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "eaf8e25a-27f4-11ee-9b34-001a4aab830c", 2023-07-21T18:41:23.842 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 3, 2023-07-21T18:41:23.842 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 24464, 2023-07-21T18:41:23.842 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2023-07-21T18:41:23.842 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2023-07-21T18:41:23.842 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2023-07-21T18:41:23.842 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 73.574014874, 2023-07-21T18:41:23.843 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 26, 2023-07-21T18:41:23.843 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 81, 2023-07-21T18:41:23.843 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 79, 2023-07-21T18:41:23.843 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 100.95432544499999 2023-07-21T18:41:23.843 INFO:teuthology.orchestra.run.smithi039.stdout:} 2023-07-21T18:41:23.844 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.808+0000 7fdf37199700 1 -- 172.21.15.39:0/3287849270 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fdf200a11e0 msgr2=0x7fdf200a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:23.844 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.808+0000 7fdf37199700 1 --2- 172.21.15.39:0/3287849270 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fdf200a11e0 0x7fdf200a3690 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c00b5c0 tx=0x7fdf1c005fb0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.845 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.808+0000 7fdf37199700 1 -- 172.21.15.39:0/3287849270 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf300fe9e0 msgr2=0x7fdf30068200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:23.845 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.808+0000 7fdf37199700 1 --2- 172.21.15.39:0/3287849270 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf300fe9e0 0x7fdf30068200 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fdf2400d980 tx=0x7fdf2400dd40 comp rx=0 tx=0).stop 2023-07-21T18:41:23.845 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.808+0000 7fdf37199700 1 -- 172.21.15.39:0/3287849270 shutdown_connections 2023-07-21T18:41:23.845 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.808+0000 7fdf37199700 1 --2- 172.21.15.39:0/3287849270 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fdf200a11e0 0x7fdf200a3690 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.845 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.808+0000 7fdf37199700 1 --2- 172.21.15.39:0/3287849270 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf30100560 0x7fdf3019cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.846 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.808+0000 7fdf37199700 1 --2- 172.21.15.39:0/3287849270 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf300fe9e0 0x7fdf30068200 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.846 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.808+0000 7fdf37199700 1 --2- 172.21.15.39:0/3287849270 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf300ffbd0 0x7fdf3019a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.846 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.808+0000 7fdf37199700 1 -- 172.21.15.39:0/3287849270 >> 172.21.15.39:0/3287849270 conn(0x7fdf300fa000 msgr2=0x7fdf300fc200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:23.846 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.808+0000 7fdf37199700 1 -- 172.21.15.39:0/3287849270 shutdown_connections 2023-07-21T18:41:23.846 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.808+0000 7fdf37199700 1 -- 172.21.15.39:0/3287849270 wait complete. 2023-07-21T18:41:23.846 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-07-21T18:41:23.973 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.950+0000 7f8209b18700 1 -- 172.21.15.39:0/1832761107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8204100560 msgr2=0x7f8204106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:23.974 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.950+0000 7f8209b18700 1 --2- 172.21.15.39:0/1832761107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8204100560 0x7f8204106e20 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f81fc00b0a0 tx=0x7f81fc00b3b0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.974 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.950+0000 7f8209b18700 1 -- 172.21.15.39:0/1832761107 shutdown_connections 2023-07-21T18:41:23.974 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.950+0000 7f8209b18700 1 --2- 172.21.15.39:0/1832761107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8204100560 0x7f8204106e20 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.974 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.950+0000 7f8209b18700 1 --2- 172.21.15.39:0/1832761107 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82040fe9e0 0x7f82040fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.974 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.950+0000 7f8209b18700 1 --2- 172.21.15.39:0/1832761107 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82040ffbd0 0x7f8204100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.974 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.950+0000 7f8209b18700 1 -- 172.21.15.39:0/1832761107 >> 172.21.15.39:0/1832761107 conn(0x7f82040fa000 msgr2=0x7f82040fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:23.975 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.950+0000 7f8209b18700 1 -- 172.21.15.39:0/1832761107 shutdown_connections 2023-07-21T18:41:23.975 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.950+0000 7f8209b18700 1 -- 172.21.15.39:0/1832761107 wait complete. 2023-07-21T18:41:23.975 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.951+0000 7f8209b18700 1 Processor -- start 2023-07-21T18:41:23.976 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.951+0000 7f8209b18700 1 -- start start 2023-07-21T18:41:23.977 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.951+0000 7f8209b18700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82040fe9e0 0x7f8204195970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:23.977 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.951+0000 7f8209b18700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82040ffbd0 0x7f8204195eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:23.977 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.951+0000 7f8209b18700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8204100560 0x7f820419cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:23.977 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.951+0000 7f8209b18700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f820419d4f0 con 0x7f8204100560 2023-07-21T18:41:23.977 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.951+0000 7f8209b18700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f820419d630 con 0x7f82040fe9e0 2023-07-21T18:41:23.977 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.951+0000 7f8209b18700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f820419d7c0 con 0x7f82040ffbd0 2023-07-21T18:41:23.977 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.951+0000 7f8203fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8204100560 0x7f820419cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:23.978 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.951+0000 7f8203fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8204100560 0x7f820419cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56418/0 (socket says 172.21.15.39:56418) 2023-07-21T18:41:23.978 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.951+0000 7f8203fff700 1 -- 172.21.15.39:0/1061812137 learned_addr learned my addr 172.21.15.39:0/1061812137 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:23.978 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.951+0000 7f82037fe700 1 --2- 172.21.15.39:0/1061812137 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82040fe9e0 0x7f8204195970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:23.978 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.951+0000 7f8202ffd700 1 --2- 172.21.15.39:0/1061812137 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82040ffbd0 0x7f8204195eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:23.978 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.951+0000 7f8203fff700 1 -- 172.21.15.39:0/1061812137 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82040ffbd0 msgr2=0x7f8204195eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:23.978 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.951+0000 7f8203fff700 1 --2- 172.21.15.39:0/1061812137 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82040ffbd0 0x7f8204195eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.978 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.951+0000 7f8203fff700 1 -- 172.21.15.39:0/1061812137 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82040fe9e0 msgr2=0x7f8204195970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:23.979 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.951+0000 7f8203fff700 1 --2- 172.21.15.39:0/1061812137 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82040fe9e0 0x7f8204195970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.979 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.951+0000 7f8203fff700 1 -- 172.21.15.39:0/1061812137 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81fc009d00 con 0x7f8204100560 2023-07-21T18:41:23.979 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.952+0000 7f8203fff700 1 --2- 172.21.15.39:0/1061812137 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8204100560 0x7f820419cf20 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f81fc00d010 tx=0x7f81fc004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:23.979 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.952+0000 7f8200ff9700 1 -- 172.21.15.39:0/1061812137 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81fc00e040 con 0x7f8204100560 2023-07-21T18:41:23.979 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.952+0000 7f8200ff9700 1 -- 172.21.15.39:0/1061812137 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f81fc004e30 con 0x7f8204100560 2023-07-21T18:41:23.979 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.952+0000 7f8209b18700 1 -- 172.21.15.39:0/1061812137 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f820419da40 con 0x7f8204100560 2023-07-21T18:41:23.979 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.952+0000 7f8200ff9700 1 -- 172.21.15.39:0/1061812137 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81fc004020 con 0x7f8204100560 2023-07-21T18:41:23.980 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.952+0000 7f8209b18700 1 -- 172.21.15.39:0/1061812137 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f820419dec0 con 0x7f8204100560 2023-07-21T18:41:23.984 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.954+0000 7f8200ff9700 1 -- 172.21.15.39:0/1061812137 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f81fc01d690 con 0x7f8204100560 2023-07-21T18:41:23.984 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.955+0000 7f8200ff9700 1 --2- 172.21.15.39:0/1061812137 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f81ec0a1260 0x7f81ec0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:23.984 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.955+0000 7f8200ff9700 1 -- 172.21.15.39:0/1061812137 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f81fc0ce4b0 con 0x7f8204100560 2023-07-21T18:41:23.985 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.955+0000 7f82037fe700 1 --2- 172.21.15.39:0/1061812137 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f81ec0a1260 0x7f81ec0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:23.985 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.955+0000 7f82037fe700 1 --2- 172.21.15.39:0/1061812137 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f81ec0a1260 0x7f81ec0a3710 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f82040ffa30 tx=0x7f81f400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:23.985 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.955+0000 7f8215c9eb80 1 Processor -- start 2023-07-21T18:41:23.985 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.955+0000 7f8215c9eb80 1 -- start start 2023-07-21T18:41:23.985 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.956+0000 7f8215c9eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a4ff350480 0x55a4ff350890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:23.985 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.956+0000 7f8215c9eb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4ff350dd0 0x55a4ff351690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:23.985 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.956+0000 7f8215c9eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4ff351c20 0x55a4ff352090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:23.985 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.956+0000 7f8215c9eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4ff255c60 con 0x55a4ff351c20 2023-07-21T18:41:23.986 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.956+0000 7f8215c9eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4ff2a89f0 con 0x55a4ff350480 2023-07-21T18:41:23.986 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.956+0000 7f8215c9eb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4ff2a1e80 con 0x55a4ff350dd0 2023-07-21T18:41:23.986 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.956+0000 7f8203fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4ff351c20 0x55a4ff352090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:23.986 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.956+0000 7f8203fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4ff351c20 0x55a4ff352090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56424/0 (socket says 172.21.15.39:56424) 2023-07-21T18:41:23.986 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.956+0000 7f8203fff700 1 -- 172.21.15.39:0/172974986 learned_addr learned my addr 172.21.15.39:0/172974986 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:23.986 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.956+0000 7f82037fe700 1 --2- 172.21.15.39:0/172974986 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a4ff350480 0x55a4ff350890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:23.986 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.956+0000 7f8202ffd700 1 --2- 172.21.15.39:0/172974986 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4ff350dd0 0x55a4ff351690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:23.986 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.956+0000 7f8203fff700 1 -- 172.21.15.39:0/172974986 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4ff350dd0 msgr2=0x55a4ff351690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:23.987 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.956+0000 7f8203fff700 1 --2- 172.21.15.39:0/172974986 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4ff350dd0 0x55a4ff351690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.987 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.956+0000 7f8203fff700 1 -- 172.21.15.39:0/172974986 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a4ff350480 msgr2=0x55a4ff350890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:23.987 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.956+0000 7f8203fff700 1 --2- 172.21.15.39:0/172974986 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a4ff350480 0x55a4ff350890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.987 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.956+0000 7f8203fff700 1 -- 172.21.15.39:0/172974986 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a4ff0cfd90 con 0x55a4ff351c20 2023-07-21T18:41:23.987 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.956+0000 7f8202ffd700 1 --2- 172.21.15.39:0/172974986 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4ff350dd0 0x55a4ff351690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:41:23.987 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.956+0000 7f82037fe700 1 --2- 172.21.15.39:0/172974986 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a4ff350480 0x55a4ff350890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:41:23.987 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.956+0000 7f8203fff700 1 --2- 172.21.15.39:0/172974986 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4ff351c20 0x55a4ff352090 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f81fc098230 tx=0x7f81fc09b590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4999d334fd8c7e6a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:23.988 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.956+0000 7f81e9ffb700 1 -- 172.21.15.39:0/172974986 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81fc09b7c0 con 0x55a4ff351c20 2023-07-21T18:41:23.988 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.956+0000 7f81e9ffb700 1 -- 172.21.15.39:0/172974986 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f81fc09f7f0 con 0x55a4ff351c20 2023-07-21T18:41:23.988 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.956+0000 7f81e9ffb700 1 -- 172.21.15.39:0/172974986 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81fc0bb200 con 0x55a4ff351c20 2023-07-21T18:41:23.988 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.957+0000 7f8215c9eb80 1 -- 172.21.15.39:0/172974986 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4ff351c20 msgr2=0x55a4ff352090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:23.988 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.957+0000 7f8215c9eb80 1 --2- 172.21.15.39:0/172974986 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4ff351c20 0x55a4ff352090 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f81fc098230 tx=0x7f81fc09b590 comp rx=0 tx=0).stop 2023-07-21T18:41:23.988 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.957+0000 7f8215c9eb80 1 -- 172.21.15.39:0/172974986 shutdown_connections 2023-07-21T18:41:23.988 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.957+0000 7f8215c9eb80 1 --2- 172.21.15.39:0/172974986 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a4ff350480 0x55a4ff350890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.988 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.957+0000 7f8215c9eb80 1 --2- 172.21.15.39:0/172974986 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4ff351c20 0x55a4ff352090 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.989 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.957+0000 7f8215c9eb80 1 --2- 172.21.15.39:0/172974986 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4ff350dd0 0x55a4ff351690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.989 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.957+0000 7f8215c9eb80 1 -- 172.21.15.39:0/172974986 >> 172.21.15.39:0/172974986 conn(0x55a4ff345d50 msgr2=0x55a4ff34b180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:23.989 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.957+0000 7f8215c9eb80 1 -- 172.21.15.39:0/172974986 shutdown_connections 2023-07-21T18:41:23.989 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.957+0000 7f8215c9eb80 1 -- 172.21.15.39:0/172974986 wait complete. 2023-07-21T18:41:23.989 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.957+0000 7f8215c9eb80 1 -- start start 2023-07-21T18:41:23.989 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.958+0000 7f8215c9eb80 1 Processor -- start 2023-07-21T18:41:23.989 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.958+0000 7f8215c9eb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4ff350480 0x55a4ff359ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:23.989 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.958+0000 7f8215c9eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a4ff350dd0 0x55a4ff35a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:23.990 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.958+0000 7f8215c9eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4ff351c20 0x55a4ff361470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:23.990 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.958+0000 7f8215c9eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4ff241a70 con 0x55a4ff351c20 2023-07-21T18:41:23.990 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.958+0000 7f8215c9eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4ff241770 con 0x55a4ff350dd0 2023-07-21T18:41:23.990 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.958+0000 7f8215c9eb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4ff1258d0 con 0x55a4ff350480 2023-07-21T18:41:23.990 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.958+0000 7f8215c9eb80 10 client.0 _ll_register_callbacks cb 0x55a4ff23f710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:41:23.990 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.958+0000 7f8203fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4ff351c20 0x55a4ff361470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:23.990 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.958+0000 7f8203fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4ff351c20 0x55a4ff361470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56432/0 (socket says 172.21.15.39:56432) 2023-07-21T18:41:23.991 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.958+0000 7f8203fff700 1 -- 172.21.15.39:0/2654343959 learned_addr learned my addr 172.21.15.39:0/2654343959 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:23.991 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.959+0000 7f82037fe700 1 --2- 172.21.15.39:0/2654343959 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4ff350480 0x55a4ff359ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:23.991 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.959+0000 7f8202ffd700 1 --2- 172.21.15.39:0/2654343959 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a4ff350dd0 0x55a4ff35a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:23.991 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.959+0000 7f8203fff700 1 -- 172.21.15.39:0/2654343959 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4ff350480 msgr2=0x55a4ff359ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:23.991 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.959+0000 7f8203fff700 1 --2- 172.21.15.39:0/2654343959 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4ff350480 0x55a4ff359ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.991 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.959+0000 7f8203fff700 1 -- 172.21.15.39:0/2654343959 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a4ff350dd0 msgr2=0x55a4ff35a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:23.991 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.959+0000 7f8203fff700 1 --2- 172.21.15.39:0/2654343959 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a4ff350dd0 0x55a4ff35a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.992 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.959+0000 7f8203fff700 1 -- 172.21.15.39:0/2654343959 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f820419dec0 con 0x55a4ff351c20 2023-07-21T18:41:23.992 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.959+0000 7f82037fe700 1 --2- 172.21.15.39:0/2654343959 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4ff350480 0x55a4ff359ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:41:23.992 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.959+0000 7f8202ffd700 1 --2- 172.21.15.39:0/2654343959 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a4ff350dd0 0x55a4ff35a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:41:23.992 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.959+0000 7f8203fff700 1 --2- 172.21.15.39:0/2654343959 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4ff351c20 0x55a4ff361470 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f81fc0a3650 tx=0x7f81fc0bb2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=571702e9f32802d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:23.992 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.959+0000 7f81e2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:41:23.992 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.959+0000 7f81e2ffd700 1 -- 172.21.15.39:0/2654343959 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81fc09bc30 con 0x55a4ff351c20 2023-07-21T18:41:23.992 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.959+0000 7f81e2ffd700 1 -- 172.21.15.39:0/2654343959 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f81fc09acb0 con 0x55a4ff351c20 2023-07-21T18:41:23.992 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.959+0000 7f8215c9eb80 1 -- 172.21.15.39:0/2654343959 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a4ff1fb4a0 con 0x55a4ff351c20 2023-07-21T18:41:23.993 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.959+0000 7f81e2ffd700 1 -- 172.21.15.39:0/2654343959 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f81fc09ae70 con 0x55a4ff351c20 2023-07-21T18:41:23.993 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.959+0000 7f8215c9eb80 10 client.15252 fetch_fsmap learned FSMap version 26 2023-07-21T18:41:23.993 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.959+0000 7f8215c9eb80 1 -- 172.21.15.39:0/2654343959 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55a4ff0b8c10 con 0x55a4ff351c20 2023-07-21T18:41:23.993 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.959+0000 7f81e2ffd700 1 -- 172.21.15.39:0/2654343959 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f81fc07e080 con 0x55a4ff351c20 2023-07-21T18:41:23.993 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.959+0000 7f81e2ffd700 1 -- 172.21.15.39:0/2654343959 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f81fc09a980 con 0x55a4ff351c20 2023-07-21T18:41:23.993 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.959+0000 7f8215c9eb80 10 client.15252 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T18:41:23.993 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.959+0000 7f8215c9eb80 10 client.15252 resolve_mds: resolved name 'i' to daemon mds.i 2023-07-21T18:41:23.993 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.960+0000 7f8215c9eb80 20 client.15252 populate_metadata read hostname 'smithi039' 2023-07-21T18:41:23.994 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.960+0000 7f8215c9eb80 1 --2- 172.21.15.39:0/2654343959 >> [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] conn(0x55a4ff364c30 0x55a4ff364310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:23.994 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.960+0000 7f8215c9eb80 4 client.15252 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:41:23.994 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.960+0000 7f8215c9eb80 1 -- 172.21.15.39:0/2654343959 --> [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a4ff29ae60 con 0x55a4ff364c30 2023-07-21T18:41:23.994 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.960+0000 7f82037fe700 1 --2- 172.21.15.39:0/2654343959 >> [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] conn(0x55a4ff364c30 0x55a4ff364310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:23.994 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.960+0000 7f81e2ffd700 10 client.15252 ms_handle_connect on v2:172.21.15.193:6838/3148488639 2023-07-21T18:41:23.994 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.960+0000 7f82037fe700 1 --2- 172.21.15.39:0/2654343959 >> [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] conn(0x55a4ff364c30 0x55a4ff364310 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=803768a26a9173e3 server_cookie=764c09c31afca54 in_seq=0 out_seq=0 2023-07-21T18:41:23.995 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.961+0000 7f81e2ffd700 1 -- 172.21.15.39:0/2654343959 <== mds.2 v2:172.21.15.193:6838/3148488639 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a4ff29ae60 con 0x55a4ff364c30 2023-07-21T18:41:23.995 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.961+0000 7f81e2ffd700 10 client.15252 handle_command_reply: tid=0 2023-07-21T18:41:23.995 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.961+0000 7f8215c9eb80 1 client.15252 shutdown 2023-07-21T18:41:23.995 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.961+0000 7f8215c9eb80 1 -- 172.21.15.39:0/2654343959 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4ff351c20 msgr2=0x55a4ff361470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:23.995 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.961+0000 7f8215c9eb80 1 --2- 172.21.15.39:0/2654343959 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4ff351c20 0x55a4ff361470 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f81fc0a3650 tx=0x7f81fc0bb2b0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.996 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.961+0000 7f8215c9eb80 1 -- 172.21.15.39:0/2654343959 shutdown_connections 2023-07-21T18:41:23.996 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.961+0000 7f8215c9eb80 1 --2- 172.21.15.39:0/2654343959 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4ff351c20 0x55a4ff361470 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.996 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.961+0000 7f8215c9eb80 1 --2- 172.21.15.39:0/2654343959 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4ff350480 0x55a4ff359ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.996 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.961+0000 7f8215c9eb80 1 --2- 172.21.15.39:0/2654343959 >> [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] conn(0x55a4ff364c30 0x55a4ff364310 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.996 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.961+0000 7f8215c9eb80 1 --2- 172.21.15.39:0/2654343959 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a4ff350dd0 0x55a4ff35a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:23.996 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.961+0000 7f81e2ffd700 0 client.15252 ms_handle_reset on v2:172.21.15.193:6838/3148488639 2023-07-21T18:41:23.996 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.961+0000 7f8215c9eb80 1 -- 172.21.15.39:0/2654343959 >> 172.21.15.39:0/2654343959 conn(0x55a4ff345d50 msgr2=0x55a4ff34a860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:23.997 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.961+0000 7f8215c9eb80 1 -- 172.21.15.39:0/2654343959 shutdown_connections 2023-07-21T18:41:23.997 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.961+0000 7f8215c9eb80 1 -- 172.21.15.39:0/2654343959 wait complete. 2023-07-21T18:41:23.997 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.961+0000 7f8215c9eb80 20 client.15252 trim_cache size 0 max 16384 2023-07-21T18:41:24.003 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.979+0000 7f8215c9eb80 1 Processor -- start 2023-07-21T18:41:24.003 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.979+0000 7f8215c9eb80 1 -- start start 2023-07-21T18:41:24.003 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.979+0000 7f8215c9eb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4ff350480 0x55a4ff364310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:24.003 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.979+0000 7f8215c9eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a4ff350dd0 0x55a4ff361400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:24.003 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.979+0000 7f8215c9eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4ff351c20 0x55a4ff34a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:24.003 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.979+0000 7f8215c9eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4ff35adf0 con 0x55a4ff351c20 2023-07-21T18:41:24.004 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.979+0000 7f8215c9eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4ff29ae60 con 0x55a4ff350dd0 2023-07-21T18:41:24.004 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.979+0000 7f8215c9eb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4ff270790 con 0x55a4ff350480 2023-07-21T18:41:24.004 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.979+0000 7f8203fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4ff351c20 0x55a4ff34a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:24.004 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.979+0000 7f8203fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4ff351c20 0x55a4ff34a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56440/0 (socket says 172.21.15.39:56440) 2023-07-21T18:41:24.004 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.979+0000 7f8203fff700 1 -- 172.21.15.39:0/3914869433 learned_addr learned my addr 172.21.15.39:0/3914869433 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:24.004 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.979+0000 7f8202ffd700 1 --2- 172.21.15.39:0/3914869433 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a4ff350dd0 0x55a4ff361400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:24.004 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.980+0000 7f82037fe700 1 --2- 172.21.15.39:0/3914869433 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4ff350480 0x55a4ff364310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:24.005 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.980+0000 7f8203fff700 1 -- 172.21.15.39:0/3914869433 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4ff350480 msgr2=0x55a4ff364310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:24.005 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.980+0000 7f8203fff700 1 --2- 172.21.15.39:0/3914869433 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4ff350480 0x55a4ff364310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.005 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.980+0000 7f8203fff700 1 -- 172.21.15.39:0/3914869433 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a4ff350dd0 msgr2=0x55a4ff361400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:24.005 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.980+0000 7f8203fff700 1 --2- 172.21.15.39:0/3914869433 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a4ff350dd0 0x55a4ff361400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.005 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.980+0000 7f8203fff700 1 -- 172.21.15.39:0/3914869433 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a4ff0f6200 con 0x55a4ff351c20 2023-07-21T18:41:24.005 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.980+0000 7f8202ffd700 1 --2- 172.21.15.39:0/3914869433 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a4ff350dd0 0x55a4ff361400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:41:24.005 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.980+0000 7f8203fff700 1 --2- 172.21.15.39:0/3914869433 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4ff351c20 0x55a4ff34a860 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f81fc098230 tx=0x7f81fc09b590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6072c5de1c6f7759 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:24.005 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.980+0000 7f81ea7fc700 1 -- 172.21.15.39:0/3914869433 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81fc09bc30 con 0x55a4ff351c20 2023-07-21T18:41:24.006 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.980+0000 7f81ea7fc700 1 -- 172.21.15.39:0/3914869433 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a4ff1fb4a0 con 0x55a4ff351c20 2023-07-21T18:41:24.006 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.980+0000 7f81ea7fc700 1 -- 172.21.15.39:0/3914869433 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81fc09acb0 con 0x55a4ff351c20 2023-07-21T18:41:24.009 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.980+0000 7f8215c9eb80 1 -- 172.21.15.39:0/3914869433 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4ff351c20 msgr2=0x55a4ff34a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:24.009 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.980+0000 7f8215c9eb80 1 --2- 172.21.15.39:0/3914869433 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4ff351c20 0x55a4ff34a860 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f81fc098230 tx=0x7f81fc09b590 comp rx=0 tx=0).stop 2023-07-21T18:41:24.009 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.980+0000 7f8215c9eb80 1 -- 172.21.15.39:0/3914869433 shutdown_connections 2023-07-21T18:41:24.009 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.980+0000 7f8215c9eb80 1 --2- 172.21.15.39:0/3914869433 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4ff351c20 0x55a4ff34a860 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.009 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.980+0000 7f8215c9eb80 1 --2- 172.21.15.39:0/3914869433 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4ff350480 0x55a4ff364310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.009 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.980+0000 7f8215c9eb80 1 --2- 172.21.15.39:0/3914869433 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a4ff350dd0 0x55a4ff361400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.010 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.980+0000 7f8215c9eb80 1 -- 172.21.15.39:0/3914869433 >> 172.21.15.39:0/3914869433 conn(0x55a4ff345d50 msgr2=0x55a4ff36d980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:24.010 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.981+0000 7f8215c9eb80 1 -- 172.21.15.39:0/3914869433 shutdown_connections 2023-07-21T18:41:24.010 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.981+0000 7f8215c9eb80 1 -- 172.21.15.39:0/3914869433 wait complete. 2023-07-21T18:41:24.010 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.981+0000 7f8215c9eb80 1 -- start start 2023-07-21T18:41:24.010 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.981+0000 7f8215c9eb80 1 Processor -- start 2023-07-21T18:41:24.010 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.981+0000 7f8215c9eb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4ff350480 0x55a4ff35c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:24.010 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.981+0000 7f8215c9eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a4ff350dd0 0x55a4ff35cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:24.011 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.981+0000 7f8215c9eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4ff351c20 0x55a4ff3b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:24.011 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.981+0000 7f8215c9eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4ff35d080 con 0x55a4ff351c20 2023-07-21T18:41:24.011 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.981+0000 7f8215c9eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4ff35d1f0 con 0x55a4ff350dd0 2023-07-21T18:41:24.011 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.981+0000 7f8215c9eb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4ff3b8ff0 con 0x55a4ff350480 2023-07-21T18:41:24.011 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.981+0000 7f8215c9eb80 10 client.0 _ll_register_callbacks cb 0x7f81fc097bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:41:24.011 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.981+0000 7f8203fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4ff351c20 0x55a4ff3b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:24.011 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.981+0000 7f8203fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4ff351c20 0x55a4ff3b8ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56446/0 (socket says 172.21.15.39:56446) 2023-07-21T18:41:24.011 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.981+0000 7f8203fff700 1 -- 172.21.15.39:0/394636406 learned_addr learned my addr 172.21.15.39:0/394636406 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:24.012 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.981+0000 7f82037fe700 1 --2- 172.21.15.39:0/394636406 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4ff350480 0x55a4ff35c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:24.012 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.981+0000 7f8202ffd700 1 --2- 172.21.15.39:0/394636406 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a4ff350dd0 0x55a4ff35cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:24.012 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.982+0000 7f8203fff700 1 -- 172.21.15.39:0/394636406 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4ff350480 msgr2=0x55a4ff35c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:24.012 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.982+0000 7f8203fff700 1 --2- 172.21.15.39:0/394636406 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4ff350480 0x55a4ff35c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.012 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.982+0000 7f8203fff700 1 -- 172.21.15.39:0/394636406 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a4ff350dd0 msgr2=0x55a4ff35cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:24.012 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.982+0000 7f8203fff700 1 --2- 172.21.15.39:0/394636406 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a4ff350dd0 0x55a4ff35cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.012 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.982+0000 7f8203fff700 1 -- 172.21.15.39:0/394636406 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a4ff0b8c10 con 0x55a4ff351c20 2023-07-21T18:41:24.012 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.982+0000 7f82037fe700 1 --2- 172.21.15.39:0/394636406 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4ff350480 0x55a4ff35c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:41:24.013 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.982+0000 7f8202ffd700 1 --2- 172.21.15.39:0/394636406 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a4ff350dd0 0x55a4ff35cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:41:24.013 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.982+0000 7f8203fff700 1 --2- 172.21.15.39:0/394636406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4ff351c20 0x55a4ff3b8ab0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f81fc0ce420 tx=0x7f81fc00b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9216b087d720b3a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:24.013 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.982+0000 7f81e2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:41:24.013 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.982+0000 7f81e2ffd700 1 -- 172.21.15.39:0/394636406 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81fc0975b0 con 0x55a4ff351c20 2023-07-21T18:41:24.013 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.982+0000 7f81e2ffd700 1 -- 172.21.15.39:0/394636406 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f81fc097710 con 0x55a4ff351c20 2023-07-21T18:41:24.013 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.982+0000 7f8215c9eb80 1 -- 172.21.15.39:0/394636406 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f81fc09f7f0 con 0x55a4ff351c20 2023-07-21T18:41:24.013 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.982+0000 7f81e2ffd700 1 -- 172.21.15.39:0/394636406 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f81fc0978d0 con 0x55a4ff351c20 2023-07-21T18:41:24.013 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.982+0000 7f8215c9eb80 10 client.15258 fetch_fsmap learned FSMap version 26 2023-07-21T18:41:24.014 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.982+0000 7f8215c9eb80 1 -- 172.21.15.39:0/394636406 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55a4ff0ce260 con 0x55a4ff351c20 2023-07-21T18:41:24.014 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.982+0000 7f81e2ffd700 1 -- 172.21.15.39:0/394636406 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f81fc09f7f0 con 0x55a4ff351c20 2023-07-21T18:41:24.014 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.983+0000 7f81e2ffd700 1 -- 172.21.15.39:0/394636406 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f81fc081080 con 0x55a4ff351c20 2023-07-21T18:41:24.014 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.983+0000 7f8215c9eb80 10 client.15258 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T18:41:24.014 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.983+0000 7f8215c9eb80 10 client.15258 resolve_mds: resolved name 'i' to daemon mds.i 2023-07-21T18:41:24.014 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.983+0000 7f8215c9eb80 20 client.15258 populate_metadata read hostname 'smithi039' 2023-07-21T18:41:24.014 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.983+0000 7f8215c9eb80 1 --2- 172.21.15.39:0/394636406 >> [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] conn(0x55a4ff3bd660 0x55a4ff3ba1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:24.015 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.983+0000 7f8215c9eb80 4 client.15258 mds_command: new command op to 24487 tid=0[{"prefix": "status"}] 2023-07-21T18:41:24.015 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.983+0000 7f8215c9eb80 1 -- 172.21.15.39:0/394636406 --> [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a4ff25d000 con 0x55a4ff3bd660 2023-07-21T18:41:24.015 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.983+0000 7f82037fe700 1 --2- 172.21.15.39:0/394636406 >> [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] conn(0x55a4ff3bd660 0x55a4ff3ba1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:24.015 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.983+0000 7f81e2ffd700 10 client.15258 ms_handle_connect on v2:172.21.15.193:6838/3148488639 2023-07-21T18:41:24.015 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.983+0000 7f82037fe700 1 --2- 172.21.15.39:0/394636406 >> [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] conn(0x55a4ff3bd660 0x55a4ff3ba1b0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a876f11c8fc601bb server_cookie=5b461722401abbb2 in_seq=0 out_seq=0 2023-07-21T18:41:24.015 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.984+0000 7f81e2ffd700 1 -- 172.21.15.39:0/394636406 <== mds.2 v2:172.21.15.193:6838/3148488639 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55a4ff25d000 con 0x55a4ff3bd660 2023-07-21T18:41:24.015 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.984+0000 7f81e2ffd700 10 client.15258 handle_command_reply: tid=0 2023-07-21T18:41:24.016 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.984+0000 7f8215c9eb80 1 client.15258 shutdown 2023-07-21T18:41:24.016 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.984+0000 7f8215c9eb80 1 -- 172.21.15.39:0/394636406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4ff351c20 msgr2=0x55a4ff3b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:24.016 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.984+0000 7f8215c9eb80 1 --2- 172.21.15.39:0/394636406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4ff351c20 0x55a4ff3b8ab0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f81fc0ce420 tx=0x7f81fc00b3b0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.016 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.984+0000 7f8215c9eb80 1 -- 172.21.15.39:0/394636406 shutdown_connections 2023-07-21T18:41:24.017 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.984+0000 7f8215c9eb80 1 --2- 172.21.15.39:0/394636406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4ff351c20 0x55a4ff3b8ab0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.017 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.984+0000 7f8215c9eb80 1 --2- 172.21.15.39:0/394636406 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4ff350480 0x55a4ff35c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.017 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.984+0000 7f8215c9eb80 1 --2- 172.21.15.39:0/394636406 >> [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] conn(0x55a4ff3bd660 0x55a4ff3ba1b0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.017 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.984+0000 7f8215c9eb80 1 --2- 172.21.15.39:0/394636406 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a4ff350dd0 0x55a4ff35cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.017 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.984+0000 7f81e2ffd700 0 client.15258 ms_handle_reset on v2:172.21.15.193:6838/3148488639 2023-07-21T18:41:24.017 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.984+0000 7f8215c9eb80 1 -- 172.21.15.39:0/394636406 >> 172.21.15.39:0/394636406 conn(0x55a4ff345d50 msgr2=0x55a4ff364310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:24.017 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.984+0000 7f8215c9eb80 1 -- 172.21.15.39:0/394636406 shutdown_connections 2023-07-21T18:41:24.017 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.984+0000 7f8215c9eb80 1 -- 172.21.15.39:0/394636406 wait complete. 2023-07-21T18:41:24.018 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.984+0000 7f8215c9eb80 20 client.15258 trim_cache size 0 max 16384 2023-07-21T18:41:24.018 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2023-07-21T18:41:24.018 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "eaf8e25a-27f4-11ee-9b34-001a4aab830c", 2023-07-21T18:41:24.018 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 2, 2023-07-21T18:41:24.018 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 24487, 2023-07-21T18:41:24.019 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2023-07-21T18:41:24.019 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2023-07-21T18:41:24.019 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2023-07-21T18:41:24.019 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 74.76231421, 2023-07-21T18:41:24.019 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 26, 2023-07-21T18:41:24.019 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 81, 2023-07-21T18:41:24.019 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 79, 2023-07-21T18:41:24.019 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 92.310960786999999 2023-07-21T18:41:24.020 INFO:teuthology.orchestra.run.smithi039.stdout:} 2023-07-21T18:41:24.021 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.984+0000 7f8209b18700 1 -- 172.21.15.39:0/1061812137 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f81ec0a1260 msgr2=0x7f81ec0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:24.021 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.984+0000 7f8209b18700 1 --2- 172.21.15.39:0/1061812137 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f81ec0a1260 0x7f81ec0a3710 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f82040ffa30 tx=0x7f81f400a380 comp rx=0 tx=0).stop 2023-07-21T18:41:24.021 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.984+0000 7f8209b18700 1 -- 172.21.15.39:0/1061812137 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8204100560 msgr2=0x7f820419cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:24.021 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.985+0000 7f8209b18700 1 --2- 172.21.15.39:0/1061812137 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8204100560 0x7f820419cf20 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f81fc00d010 tx=0x7f81fc004c80 comp rx=0 tx=0).stop 2023-07-21T18:41:24.021 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.985+0000 7f8209b18700 1 -- 172.21.15.39:0/1061812137 shutdown_connections 2023-07-21T18:41:24.021 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.985+0000 7f8209b18700 1 --2- 172.21.15.39:0/1061812137 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f81ec0a1260 0x7f81ec0a3710 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.022 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.985+0000 7f8209b18700 1 --2- 172.21.15.39:0/1061812137 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82040fe9e0 0x7f8204195970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.022 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.985+0000 7f8209b18700 1 --2- 172.21.15.39:0/1061812137 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8204100560 0x7f820419cf20 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.022 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.985+0000 7f8209b18700 1 --2- 172.21.15.39:0/1061812137 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82040ffbd0 0x7f8204195eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.022 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.985+0000 7f8209b18700 1 -- 172.21.15.39:0/1061812137 >> 172.21.15.39:0/1061812137 conn(0x7f82040fa000 msgr2=0x7f8204067690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:24.022 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.985+0000 7f8209b18700 1 -- 172.21.15.39:0/1061812137 shutdown_connections 2023-07-21T18:41:24.022 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:23.985+0000 7f8209b18700 1 -- 172.21.15.39:0/1061812137 wait complete. 2023-07-21T18:41:24.023 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2023-07-21T18:41:24.150 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.127+0000 7f405d896700 1 -- 172.21.15.39:0/996216162 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40580fea50 msgr2=0x7f40580fee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:24.151 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.127+0000 7f405d896700 1 --2- 172.21.15.39:0/996216162 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40580fea50 0x7f40580fee60 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f4048009990 tx=0x7f4048009ca0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.151 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.127+0000 7f405d896700 1 -- 172.21.15.39:0/996216162 shutdown_connections 2023-07-21T18:41:24.151 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.127+0000 7f405d896700 1 --2- 172.21.15.39:0/996216162 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40581005d0 0x7f4058106e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.151 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.127+0000 7f405d896700 1 --2- 172.21.15.39:0/996216162 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40580fea50 0x7f40580fee60 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.151 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.127+0000 7f405d896700 1 --2- 172.21.15.39:0/996216162 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40580ffc40 0x7f4058100090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.152 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.127+0000 7f405d896700 1 -- 172.21.15.39:0/996216162 >> 172.21.15.39:0/996216162 conn(0x7f40580fa070 msgr2=0x7f40580fc4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:24.152 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.127+0000 7f405d896700 1 -- 172.21.15.39:0/996216162 shutdown_connections 2023-07-21T18:41:24.152 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.128+0000 7f405d896700 1 -- 172.21.15.39:0/996216162 wait complete. 2023-07-21T18:41:24.153 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.128+0000 7f405d896700 1 Processor -- start 2023-07-21T18:41:24.154 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.128+0000 7f405d896700 1 -- start start 2023-07-21T18:41:24.154 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.128+0000 7f405d896700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40580fea50 0x7f40580693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:24.154 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.128+0000 7f405d896700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40580ffc40 0x7f405819a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:24.154 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.128+0000 7f405d896700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40581005d0 0x7f405819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:24.154 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.128+0000 7f405d896700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405819d410 con 0x7f40580ffc40 2023-07-21T18:41:24.154 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.128+0000 7f405d896700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405819d580 con 0x7f40580fea50 2023-07-21T18:41:24.154 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.128+0000 7f405d896700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405819d740 con 0x7f40581005d0 2023-07-21T18:41:24.155 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.129+0000 7f40567fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40580ffc40 0x7f405819a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:24.155 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.129+0000 7f40567fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40580ffc40 0x7f405819a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56458/0 (socket says 172.21.15.39:56458) 2023-07-21T18:41:24.155 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.129+0000 7f40567fc700 1 -- 172.21.15.39:0/2504871448 learned_addr learned my addr 172.21.15.39:0/2504871448 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:24.155 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.129+0000 7f4056ffd700 1 --2- 172.21.15.39:0/2504871448 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40580fea50 0x7f40580693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:24.155 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.129+0000 7f40577fe700 1 --2- 172.21.15.39:0/2504871448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40581005d0 0x7f405819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:24.155 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.129+0000 7f40567fc700 1 -- 172.21.15.39:0/2504871448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40581005d0 msgr2=0x7f405819cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:24.155 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.129+0000 7f40567fc700 1 --2- 172.21.15.39:0/2504871448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40581005d0 0x7f405819cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.156 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.129+0000 7f40567fc700 1 -- 172.21.15.39:0/2504871448 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40580fea50 msgr2=0x7f40580693b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:24.156 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.129+0000 7f40567fc700 1 --2- 172.21.15.39:0/2504871448 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40580fea50 0x7f40580693b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.156 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.129+0000 7f40567fc700 1 -- 172.21.15.39:0/2504871448 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40480095f0 con 0x7f40580ffc40 2023-07-21T18:41:24.156 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.129+0000 7f40577fe700 1 --2- 172.21.15.39:0/2504871448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40581005d0 0x7f405819cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:41:24.156 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.129+0000 7f40567fc700 1 --2- 172.21.15.39:0/2504871448 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40580ffc40 0x7f405819a5f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f404c00d950 tx=0x7f404c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:24.156 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.129+0000 7f405c894700 1 -- 172.21.15.39:0/2504871448 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404c009940 con 0x7f40580ffc40 2023-07-21T18:41:24.156 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.129+0000 7f405c894700 1 -- 172.21.15.39:0/2504871448 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f404c010460 con 0x7f40580ffc40 2023-07-21T18:41:24.157 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.129+0000 7f405c894700 1 -- 172.21.15.39:0/2504871448 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404c00f620 con 0x7f40580ffc40 2023-07-21T18:41:24.157 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.129+0000 7f405d896700 1 -- 172.21.15.39:0/2504871448 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f405819da20 con 0x7f40580ffc40 2023-07-21T18:41:24.157 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.129+0000 7f405d896700 1 -- 172.21.15.39:0/2504871448 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f405819df30 con 0x7f40580ffc40 2023-07-21T18:41:24.162 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.131+0000 7f405c894700 1 -- 172.21.15.39:0/2504871448 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f404c0108d0 con 0x7f40580ffc40 2023-07-21T18:41:24.162 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.132+0000 7f405c894700 1 --2- 172.21.15.39:0/2504871448 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f40400a1260 0x7f40400a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:24.162 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.132+0000 7f405c894700 1 -- 172.21.15.39:0/2504871448 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f404c0cc840 con 0x7f40580ffc40 2023-07-21T18:41:24.162 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.132+0000 7f4056ffd700 1 --2- 172.21.15.39:0/2504871448 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f40400a1260 0x7f40400a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:24.163 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.132+0000 7f4056ffd700 1 --2- 172.21.15.39:0/2504871448 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f40400a1260 0x7f40400a3710 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f404800b5c0 tx=0x7f4048005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:24.163 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.133+0000 7f4069a1cb80 1 Processor -- start 2023-07-21T18:41:24.163 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.133+0000 7f4069a1cb80 1 -- start start 2023-07-21T18:41:24.163 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.133+0000 7f4069a1cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dfe1d6890 0x564dfe1d6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:24.163 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.133+0000 7f4069a1cb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dfe1d71e0 0x564dfe1d7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:24.163 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.133+0000 7f4069a1cb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564dfe1d8030 0x564dfe1d84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:24.163 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.133+0000 7f4069a1cb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dfe0d7c60 con 0x564dfe1d71e0 2023-07-21T18:41:24.164 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.133+0000 7f4069a1cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dfe12a9f0 con 0x564dfe1d6890 2023-07-21T18:41:24.164 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.133+0000 7f4069a1cb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dfe123e80 con 0x564dfe1d8030 2023-07-21T18:41:24.164 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.133+0000 7f40567fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dfe1d71e0 0x564dfe1d7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:24.164 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.133+0000 7f40567fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dfe1d71e0 0x564dfe1d7aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56466/0 (socket says 172.21.15.39:56466) 2023-07-21T18:41:24.164 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.133+0000 7f40567fc700 1 -- 172.21.15.39:0/697492151 learned_addr learned my addr 172.21.15.39:0/697492151 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:24.164 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.133+0000 7f40577fe700 1 --2- 172.21.15.39:0/697492151 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564dfe1d8030 0x564dfe1d84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:24.164 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.133+0000 7f4056ffd700 1 --2- 172.21.15.39:0/697492151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dfe1d6890 0x564dfe1d6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:24.165 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.133+0000 7f40567fc700 1 -- 172.21.15.39:0/697492151 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564dfe1d8030 msgr2=0x564dfe1d84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:24.165 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.133+0000 7f40567fc700 1 --2- 172.21.15.39:0/697492151 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564dfe1d8030 0x564dfe1d84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.165 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.133+0000 7f40567fc700 1 -- 172.21.15.39:0/697492151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dfe1d6890 msgr2=0x564dfe1d6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:24.165 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.133+0000 7f40567fc700 1 --2- 172.21.15.39:0/697492151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dfe1d6890 0x564dfe1d6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.165 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.133+0000 7f40567fc700 1 -- 172.21.15.39:0/697492151 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564dfdf51d90 con 0x564dfe1d71e0 2023-07-21T18:41:24.165 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.133+0000 7f4056ffd700 1 --2- 172.21.15.39:0/697492151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dfe1d6890 0x564dfe1d6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:41:24.165 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.134+0000 7f40567fc700 1 --2- 172.21.15.39:0/697492151 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dfe1d71e0 0x564dfe1d7aa0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f404c096590 tx=0x7f404c099580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec21a61c79be5e16 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:24.165 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.134+0000 7f403dffb700 1 -- 172.21.15.39:0/697492151 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404c0ca470 con 0x564dfe1d71e0 2023-07-21T18:41:24.166 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.134+0000 7f403dffb700 1 -- 172.21.15.39:0/697492151 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f404c099d80 con 0x564dfe1d71e0 2023-07-21T18:41:24.166 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.134+0000 7f403dffb700 1 -- 172.21.15.39:0/697492151 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404c0a1ae0 con 0x564dfe1d71e0 2023-07-21T18:41:24.166 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.134+0000 7f4069a1cb80 1 -- 172.21.15.39:0/697492151 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dfe1d71e0 msgr2=0x564dfe1d7aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:24.166 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.134+0000 7f4069a1cb80 1 --2- 172.21.15.39:0/697492151 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dfe1d71e0 0x564dfe1d7aa0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f404c096590 tx=0x7f404c099580 comp rx=0 tx=0).stop 2023-07-21T18:41:24.166 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.134+0000 7f4069a1cb80 1 -- 172.21.15.39:0/697492151 shutdown_connections 2023-07-21T18:41:24.166 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.134+0000 7f4069a1cb80 1 --2- 172.21.15.39:0/697492151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dfe1d6890 0x564dfe1d6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.166 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.134+0000 7f4069a1cb80 1 --2- 172.21.15.39:0/697492151 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564dfe1d8030 0x564dfe1d84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.167 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.134+0000 7f4069a1cb80 1 --2- 172.21.15.39:0/697492151 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dfe1d71e0 0x564dfe1d7aa0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.167 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.134+0000 7f4069a1cb80 1 -- 172.21.15.39:0/697492151 >> 172.21.15.39:0/697492151 conn(0x564dfe1c7d50 msgr2=0x564dfe1cd180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:24.167 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.134+0000 7f4069a1cb80 1 -- 172.21.15.39:0/697492151 shutdown_connections 2023-07-21T18:41:24.167 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.135+0000 7f4069a1cb80 1 -- 172.21.15.39:0/697492151 wait complete. 2023-07-21T18:41:24.167 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.135+0000 7f4069a1cb80 1 -- start start 2023-07-21T18:41:24.168 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.135+0000 7f4069a1cb80 1 Processor -- start 2023-07-21T18:41:24.168 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.135+0000 7f4069a1cb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564dfe1d6890 0x564dfe1e0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:24.168 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.135+0000 7f4069a1cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dfe1d71e0 0x564dfe1e0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:24.168 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.135+0000 7f4069a1cb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dfe1d8030 0x564dfe1e78d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:24.168 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.135+0000 7f4069a1cb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dfe0c3a70 con 0x564dfe1d8030 2023-07-21T18:41:24.169 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.135+0000 7f4069a1cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dfe0c3770 con 0x564dfe1d71e0 2023-07-21T18:41:24.169 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.135+0000 7f4069a1cb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dfdfa78d0 con 0x564dfe1d6890 2023-07-21T18:41:24.169 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.135+0000 7f4069a1cb80 10 client.0 _ll_register_callbacks cb 0x564dfe0c1710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:41:24.169 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.135+0000 7f40577fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dfe1d8030 0x564dfe1e78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:24.169 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.135+0000 7f40577fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dfe1d8030 0x564dfe1e78d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56474/0 (socket says 172.21.15.39:56474) 2023-07-21T18:41:24.169 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.135+0000 7f40577fe700 1 -- 172.21.15.39:0/2042750527 learned_addr learned my addr 172.21.15.39:0/2042750527 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:24.169 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.136+0000 7f4056ffd700 1 --2- 172.21.15.39:0/2042750527 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564dfe1d6890 0x564dfe1e0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:24.170 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.136+0000 7f40567fc700 1 --2- 172.21.15.39:0/2042750527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dfe1d71e0 0x564dfe1e0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:24.170 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.136+0000 7f40577fe700 1 -- 172.21.15.39:0/2042750527 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564dfe1d6890 msgr2=0x564dfe1e0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:24.170 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.136+0000 7f40577fe700 1 --2- 172.21.15.39:0/2042750527 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564dfe1d6890 0x564dfe1e0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.170 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.136+0000 7f40577fe700 1 -- 172.21.15.39:0/2042750527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dfe1d71e0 msgr2=0x564dfe1e0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:24.170 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.136+0000 7f40577fe700 1 --2- 172.21.15.39:0/2042750527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dfe1d71e0 0x564dfe1e0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.170 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.136+0000 7f40577fe700 1 -- 172.21.15.39:0/2042750527 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f405819df30 con 0x564dfe1d8030 2023-07-21T18:41:24.170 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.136+0000 7f40577fe700 1 --2- 172.21.15.39:0/2042750527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dfe1d8030 0x564dfe1e78d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f4050008c10 tx=0x7f4050008f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4714da10379e8822 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:24.170 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.136+0000 7f4036ffd700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:41:24.171 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.136+0000 7f4036ffd700 1 -- 172.21.15.39:0/2042750527 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f405000f920 con 0x564dfe1d8030 2023-07-21T18:41:24.171 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.136+0000 7f4036ffd700 1 -- 172.21.15.39:0/2042750527 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f405000fa80 con 0x564dfe1d8030 2023-07-21T18:41:24.171 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.136+0000 7f4069a1cb80 1 -- 172.21.15.39:0/2042750527 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564dfe07d4a0 con 0x564dfe1d8030 2023-07-21T18:41:24.171 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.136+0000 7f4036ffd700 1 -- 172.21.15.39:0/2042750527 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f405000fc40 con 0x564dfe1d8030 2023-07-21T18:41:24.171 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.136+0000 7f4069a1cb80 10 client.15270 fetch_fsmap learned FSMap version 26 2023-07-21T18:41:24.171 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.136+0000 7f4069a1cb80 1 -- 172.21.15.39:0/2042750527 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x564dfdf3ac10 con 0x564dfe1d8030 2023-07-21T18:41:24.171 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.136+0000 7f4036ffd700 1 -- 172.21.15.39:0/2042750527 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564dfe07d4a0 con 0x564dfe1d8030 2023-07-21T18:41:24.171 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.137+0000 7f4036ffd700 1 -- 172.21.15.39:0/2042750527 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f405001e420 con 0x564dfe1d8030 2023-07-21T18:41:24.172 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.137+0000 7f4069a1cb80 10 client.15270 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T18:41:24.172 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.137+0000 7f4069a1cb80 10 client.15270 resolve_mds: resolved name 'h' to daemon mds.h 2023-07-21T18:41:24.172 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.137+0000 7f4069a1cb80 20 client.15270 populate_metadata read hostname 'smithi039' 2023-07-21T18:41:24.172 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.137+0000 7f4069a1cb80 1 --2- 172.21.15.39:0/2042750527 >> [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] conn(0x564dfe1eb640 0x564dfe1ea9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:24.172 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.137+0000 7f4069a1cb80 4 client.15270 mds_command: new command op to 24500 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:41:24.172 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.137+0000 7f4069a1cb80 1 -- 172.21.15.39:0/2042750527 --> [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564dfe11ce60 con 0x564dfe1eb640 2023-07-21T18:41:24.172 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.137+0000 7f4056ffd700 1 --2- 172.21.15.39:0/2042750527 >> [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] conn(0x564dfe1eb640 0x564dfe1ea9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:24.173 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.137+0000 7f4036ffd700 10 client.15270 ms_handle_connect on v2:172.21.15.155:6836/947670056 2023-07-21T18:41:24.173 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.137+0000 7f4056ffd700 1 --2- 172.21.15.39:0/2042750527 >> [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] conn(0x564dfe1eb640 0x564dfe1ea9f0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c49d5c1ddba31331 server_cookie=62b837ec4ddf9bb9 in_seq=0 out_seq=0 2023-07-21T18:41:24.173 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.138+0000 7f4036ffd700 1 -- 172.21.15.39:0/2042750527 <== mds.1 v2:172.21.15.155:6836/947670056 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564dfe11ce60 con 0x564dfe1eb640 2023-07-21T18:41:24.173 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.138+0000 7f4036ffd700 10 client.15270 handle_command_reply: tid=0 2023-07-21T18:41:24.173 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.138+0000 7f4069a1cb80 1 client.15270 shutdown 2023-07-21T18:41:24.173 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.138+0000 7f4069a1cb80 1 -- 172.21.15.39:0/2042750527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dfe1d8030 msgr2=0x564dfe1e78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:24.174 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.138+0000 7f4069a1cb80 1 --2- 172.21.15.39:0/2042750527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dfe1d8030 0x564dfe1e78d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f4050008c10 tx=0x7f4050008f20 comp rx=0 tx=0).stop 2023-07-21T18:41:24.174 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.138+0000 7f4069a1cb80 1 -- 172.21.15.39:0/2042750527 shutdown_connections 2023-07-21T18:41:24.174 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.138+0000 7f4069a1cb80 1 --2- 172.21.15.39:0/2042750527 >> [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] conn(0x564dfe1eb640 0x564dfe1ea9f0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.174 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.138+0000 7f4069a1cb80 1 --2- 172.21.15.39:0/2042750527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dfe1d8030 0x564dfe1e78d0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.174 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.138+0000 7f4069a1cb80 1 --2- 172.21.15.39:0/2042750527 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564dfe1d6890 0x564dfe1e0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.175 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.138+0000 7f4036ffd700 0 client.15270 ms_handle_reset on v2:172.21.15.155:6836/947670056 2023-07-21T18:41:24.175 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.138+0000 7f4069a1cb80 1 --2- 172.21.15.39:0/2042750527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dfe1d71e0 0x564dfe1e0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.175 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.138+0000 7f4069a1cb80 1 -- 172.21.15.39:0/2042750527 >> 172.21.15.39:0/2042750527 conn(0x564dfe1c7d50 msgr2=0x564dfe1cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:24.175 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.138+0000 7f4069a1cb80 1 -- 172.21.15.39:0/2042750527 shutdown_connections 2023-07-21T18:41:24.175 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.138+0000 7f4069a1cb80 1 -- 172.21.15.39:0/2042750527 wait complete. 2023-07-21T18:41:24.175 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.138+0000 7f4069a1cb80 20 client.15270 trim_cache size 0 max 16384 2023-07-21T18:41:24.180 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.156+0000 7f4069a1cb80 1 Processor -- start 2023-07-21T18:41:24.180 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.156+0000 7f4069a1cb80 1 -- start start 2023-07-21T18:41:24.180 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.157+0000 7f4069a1cb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564dfe1d6890 0x564dfe1e8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:24.181 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.157+0000 7f4069a1cb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dfe1d71e0 0x564dfe1e77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:24.181 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.157+0000 7f4069a1cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dfe1d8030 0x564dfe1de420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:24.181 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.157+0000 7f4069a1cb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dfe1df850 con 0x564dfe1d71e0 2023-07-21T18:41:24.181 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.157+0000 7f4069a1cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dfe0d7c60 con 0x564dfe1d8030 2023-07-21T18:41:24.181 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.157+0000 7f4069a1cb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dfe1dee50 con 0x564dfe1d6890 2023-07-21T18:41:24.181 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.157+0000 7f40567fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dfe1d71e0 0x564dfe1e77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:24.181 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.157+0000 7f40567fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dfe1d71e0 0x564dfe1e77f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56480/0 (socket says 172.21.15.39:56480) 2023-07-21T18:41:24.182 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.157+0000 7f40567fc700 1 -- 172.21.15.39:0/1158426586 learned_addr learned my addr 172.21.15.39:0/1158426586 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:24.182 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.157+0000 7f40577fe700 1 --2- 172.21.15.39:0/1158426586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dfe1d8030 0x564dfe1de420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:24.182 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.157+0000 7f4056ffd700 1 --2- 172.21.15.39:0/1158426586 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564dfe1d6890 0x564dfe1e8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:24.182 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.157+0000 7f40567fc700 1 -- 172.21.15.39:0/1158426586 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564dfe1d6890 msgr2=0x564dfe1e8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:24.182 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.157+0000 7f40567fc700 1 --2- 172.21.15.39:0/1158426586 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564dfe1d6890 0x564dfe1e8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.182 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.157+0000 7f40567fc700 1 -- 172.21.15.39:0/1158426586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dfe1d8030 msgr2=0x564dfe1de420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:24.182 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.157+0000 7f40567fc700 1 --2- 172.21.15.39:0/1158426586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dfe1d8030 0x564dfe1de420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.182 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.157+0000 7f40567fc700 1 -- 172.21.15.39:0/1158426586 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564dfe1c95d0 con 0x564dfe1d71e0 2023-07-21T18:41:24.183 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.157+0000 7f40567fc700 1 --2- 172.21.15.39:0/1158426586 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dfe1d71e0 0x564dfe1e77f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f404c097e70 tx=0x7f404c005910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8fbf391e67ae924 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:24.183 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.157+0000 7f403e7fc700 1 -- 172.21.15.39:0/1158426586 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404c0b2250 con 0x564dfe1d71e0 2023-07-21T18:41:24.183 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.157+0000 7f403e7fc700 1 -- 172.21.15.39:0/1158426586 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f404c0ca470 con 0x564dfe1d71e0 2023-07-21T18:41:24.183 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.157+0000 7f403e7fc700 1 -- 172.21.15.39:0/1158426586 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404c0a1d20 con 0x564dfe1d71e0 2023-07-21T18:41:24.186 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.158+0000 7f4069a1cb80 1 -- 172.21.15.39:0/1158426586 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dfe1d71e0 msgr2=0x564dfe1e77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:24.186 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.158+0000 7f4069a1cb80 1 --2- 172.21.15.39:0/1158426586 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dfe1d71e0 0x564dfe1e77f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f404c097e70 tx=0x7f404c005910 comp rx=0 tx=0).stop 2023-07-21T18:41:24.187 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.158+0000 7f4069a1cb80 1 -- 172.21.15.39:0/1158426586 shutdown_connections 2023-07-21T18:41:24.187 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.158+0000 7f4069a1cb80 1 --2- 172.21.15.39:0/1158426586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dfe1d8030 0x564dfe1de420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.187 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.158+0000 7f4069a1cb80 1 --2- 172.21.15.39:0/1158426586 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564dfe1d6890 0x564dfe1e8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.187 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.158+0000 7f4069a1cb80 1 --2- 172.21.15.39:0/1158426586 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dfe1d71e0 0x564dfe1e77f0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.187 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.158+0000 7f4069a1cb80 1 -- 172.21.15.39:0/1158426586 >> 172.21.15.39:0/1158426586 conn(0x564dfe1c7d50 msgr2=0x564dfe1df040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:24.187 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.158+0000 7f4069a1cb80 1 -- 172.21.15.39:0/1158426586 shutdown_connections 2023-07-21T18:41:24.187 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.158+0000 7f4069a1cb80 1 -- 172.21.15.39:0/1158426586 wait complete. 2023-07-21T18:41:24.187 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.158+0000 7f4069a1cb80 1 -- start start 2023-07-21T18:41:24.188 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.159+0000 7f4069a1cb80 1 Processor -- start 2023-07-21T18:41:24.188 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.159+0000 7f4069a1cb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dfe1d6890 0x564dfe1e2c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:24.188 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.159+0000 7f4069a1cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dfe1d71e0 0x564dfe1e31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:24.188 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.159+0000 7f4069a1cb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564dfe1d8030 0x564dfe1e6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:24.188 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.159+0000 7f4069a1cb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dfe1e6660 con 0x564dfe1d6890 2023-07-21T18:41:24.189 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.159+0000 7f4069a1cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dfe1e67d0 con 0x564dfe1d71e0 2023-07-21T18:41:24.189 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.159+0000 7f4069a1cb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dfe1e6990 con 0x564dfe1d8030 2023-07-21T18:41:24.189 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.159+0000 7f4069a1cb80 10 client.0 _ll_register_callbacks cb 0x7f4050008480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:41:24.189 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.159+0000 7f4056ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dfe1d6890 0x564dfe1e2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:24.189 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.159+0000 7f4056ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dfe1d6890 0x564dfe1e2c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56484/0 (socket says 172.21.15.39:56484) 2023-07-21T18:41:24.189 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.159+0000 7f4056ffd700 1 -- 172.21.15.39:0/3522642607 learned_addr learned my addr 172.21.15.39:0/3522642607 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:24.189 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.159+0000 7f40577fe700 1 --2- 172.21.15.39:0/3522642607 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564dfe1d8030 0x564dfe1e6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:24.189 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.159+0000 7f40567fc700 1 --2- 172.21.15.39:0/3522642607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dfe1d71e0 0x564dfe1e31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:24.190 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.159+0000 7f4056ffd700 1 -- 172.21.15.39:0/3522642607 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564dfe1d8030 msgr2=0x564dfe1e6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:24.190 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.159+0000 7f4056ffd700 1 --2- 172.21.15.39:0/3522642607 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564dfe1d8030 0x564dfe1e6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.190 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.159+0000 7f4056ffd700 1 -- 172.21.15.39:0/3522642607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dfe1d71e0 msgr2=0x564dfe1e31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:24.190 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.159+0000 7f4056ffd700 1 --2- 172.21.15.39:0/3522642607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dfe1d71e0 0x564dfe1e31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.190 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.159+0000 7f4056ffd700 1 -- 172.21.15.39:0/3522642607 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564dfdf51d90 con 0x564dfe1d6890 2023-07-21T18:41:24.190 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.159+0000 7f40567fc700 1 --2- 172.21.15.39:0/3522642607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dfe1d71e0 0x564dfe1e31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:41:24.190 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.159+0000 7f4056ffd700 1 --2- 172.21.15.39:0/3522642607 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dfe1d6890 0x564dfe1e2c60 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f404800a010 tx=0x7f4048009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35414f89d09555d1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:24.191 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.159+0000 7f4036ffd700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:41:24.191 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.159+0000 7f4036ffd700 1 -- 172.21.15.39:0/3522642607 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4048034cd0 con 0x564dfe1d6890 2023-07-21T18:41:24.191 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.160+0000 7f4069a1cb80 1 -- 172.21.15.39:0/3522642607 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f404c099d80 con 0x564dfe1d6890 2023-07-21T18:41:24.191 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.160+0000 7f4036ffd700 1 -- 172.21.15.39:0/3522642607 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f4048034e30 con 0x564dfe1d6890 2023-07-21T18:41:24.191 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.160+0000 7f4036ffd700 1 -- 172.21.15.39:0/3522642607 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4048034590 con 0x564dfe1d6890 2023-07-21T18:41:24.191 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.160+0000 7f4069a1cb80 10 client.15276 fetch_fsmap learned FSMap version 26 2023-07-21T18:41:24.191 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.160+0000 7f4069a1cb80 1 -- 172.21.15.39:0/3522642607 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x564dfdf78200 con 0x564dfe1d6890 2023-07-21T18:41:24.191 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.160+0000 7f4036ffd700 1 -- 172.21.15.39:0/3522642607 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f404c099d80 con 0x564dfe1d6890 2023-07-21T18:41:24.192 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.160+0000 7f4036ffd700 1 -- 172.21.15.39:0/3522642607 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4048046050 con 0x564dfe1d6890 2023-07-21T18:41:24.192 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.160+0000 7f4069a1cb80 10 client.15276 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T18:41:24.192 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.160+0000 7f4069a1cb80 10 client.15276 resolve_mds: resolved name 'h' to daemon mds.h 2023-07-21T18:41:24.192 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.160+0000 7f4069a1cb80 20 client.15276 populate_metadata read hostname 'smithi039' 2023-07-21T18:41:24.192 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.160+0000 7f4069a1cb80 1 --2- 172.21.15.39:0/3522642607 >> [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] conn(0x564dfe244db0 0x564dfe241700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:24.192 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.160+0000 7f4069a1cb80 4 client.15276 mds_command: new command op to 24500 tid=0[{"prefix": "status"}] 2023-07-21T18:41:24.192 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.160+0000 7f4069a1cb80 1 -- 172.21.15.39:0/3522642607 --> [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564dfe0df000 con 0x564dfe244db0 2023-07-21T18:41:24.193 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.161+0000 7f40567fc700 1 --2- 172.21.15.39:0/3522642607 >> [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] conn(0x564dfe244db0 0x564dfe241700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:24.193 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.161+0000 7f40567fc700 1 --2- 172.21.15.39:0/3522642607 >> [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] conn(0x564dfe244db0 0x564dfe241700 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=413aaad7e062f24c server_cookie=a9d0f9c7863c058c in_seq=0 out_seq=0 2023-07-21T18:41:24.193 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.161+0000 7f4036ffd700 10 client.15276 ms_handle_connect on v2:172.21.15.155:6836/947670056 2023-07-21T18:41:24.193 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.161+0000 7f4036ffd700 1 -- 172.21.15.39:0/3522642607 <== mds.1 v2:172.21.15.155:6836/947670056 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x564dfe0df000 con 0x564dfe244db0 2023-07-21T18:41:24.193 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.161+0000 7f4036ffd700 10 client.15276 handle_command_reply: tid=0 2023-07-21T18:41:24.193 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.161+0000 7f4069a1cb80 1 client.15276 shutdown 2023-07-21T18:41:24.193 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.161+0000 7f4069a1cb80 1 -- 172.21.15.39:0/3522642607 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dfe1d6890 msgr2=0x564dfe1e2c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:24.193 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.161+0000 7f4069a1cb80 1 --2- 172.21.15.39:0/3522642607 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dfe1d6890 0x564dfe1e2c60 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f404800a010 tx=0x7f4048009ca0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.194 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.161+0000 7f4069a1cb80 1 -- 172.21.15.39:0/3522642607 shutdown_connections 2023-07-21T18:41:24.194 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.161+0000 7f4069a1cb80 1 --2- 172.21.15.39:0/3522642607 >> [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] conn(0x564dfe244db0 0x564dfe241700 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.194 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.161+0000 7f4069a1cb80 1 --2- 172.21.15.39:0/3522642607 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564dfe1d8030 0x564dfe1e6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.194 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.161+0000 7f4069a1cb80 1 --2- 172.21.15.39:0/3522642607 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dfe1d6890 0x564dfe1e2c60 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.194 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.161+0000 7f4036ffd700 0 client.15276 ms_handle_reset on v2:172.21.15.155:6836/947670056 2023-07-21T18:41:24.194 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.161+0000 7f4069a1cb80 1 --2- 172.21.15.39:0/3522642607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dfe1d71e0 0x564dfe1e31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.194 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.161+0000 7f4069a1cb80 1 -- 172.21.15.39:0/3522642607 >> 172.21.15.39:0/3522642607 conn(0x564dfe1c7d50 msgr2=0x564dfe1de420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:24.194 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.161+0000 7f4069a1cb80 1 -- 172.21.15.39:0/3522642607 shutdown_connections 2023-07-21T18:41:24.195 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.162+0000 7f4069a1cb80 1 -- 172.21.15.39:0/3522642607 wait complete. 2023-07-21T18:41:24.195 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.162+0000 7f4069a1cb80 20 client.15276 trim_cache size 0 max 16384 2023-07-21T18:41:24.195 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2023-07-21T18:41:24.195 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "eaf8e25a-27f4-11ee-9b34-001a4aab830c", 2023-07-21T18:41:24.195 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 1, 2023-07-21T18:41:24.195 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 24500, 2023-07-21T18:41:24.196 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2023-07-21T18:41:24.196 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2023-07-21T18:41:24.196 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2023-07-21T18:41:24.196 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 75.958908989999998, 2023-07-21T18:41:24.196 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 26, 2023-07-21T18:41:24.196 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 81, 2023-07-21T18:41:24.196 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 79, 2023-07-21T18:41:24.197 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 94.691418526000007 2023-07-21T18:41:24.197 INFO:teuthology.orchestra.run.smithi039.stdout:} 2023-07-21T18:41:24.198 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.162+0000 7f405d896700 1 -- 172.21.15.39:0/2504871448 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f40400a1260 msgr2=0x7f40400a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:24.198 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.162+0000 7f405d896700 1 --2- 172.21.15.39:0/2504871448 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f40400a1260 0x7f40400a3710 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f404800b5c0 tx=0x7f4048005960 comp rx=0 tx=0).stop 2023-07-21T18:41:24.198 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.162+0000 7f405d896700 1 -- 172.21.15.39:0/2504871448 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40580ffc40 msgr2=0x7f405819a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:24.198 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.162+0000 7f405d896700 1 --2- 172.21.15.39:0/2504871448 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40580ffc40 0x7f405819a5f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f404c00d950 tx=0x7f404c00dc60 comp rx=0 tx=0).stop 2023-07-21T18:41:24.198 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.162+0000 7f405d896700 1 -- 172.21.15.39:0/2504871448 shutdown_connections 2023-07-21T18:41:24.199 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.162+0000 7f405d896700 1 --2- 172.21.15.39:0/2504871448 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f40400a1260 0x7f40400a3710 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.199 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.162+0000 7f405d896700 1 --2- 172.21.15.39:0/2504871448 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40580fea50 0x7f40580693b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.199 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.162+0000 7f405d896700 1 --2- 172.21.15.39:0/2504871448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40581005d0 0x7f405819cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.199 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.162+0000 7f405d896700 1 --2- 172.21.15.39:0/2504871448 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40580ffc40 0x7f405819a5f0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.199 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.162+0000 7f405d896700 1 -- 172.21.15.39:0/2504871448 >> 172.21.15.39:0/2504871448 conn(0x7f40580fa070 msgr2=0x7f4058103180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:24.199 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.162+0000 7f405d896700 1 -- 172.21.15.39:0/2504871448 shutdown_connections 2023-07-21T18:41:24.200 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.162+0000 7f405d896700 1 -- 172.21.15.39:0/2504871448 wait complete. 2023-07-21T18:41:24.200 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-07-21T18:41:24.328 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.304+0000 7fef9002b700 1 -- 172.21.15.39:0/1599624368 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef881010b0 msgr2=0x7fef881014c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:24.328 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.304+0000 7fef9002b700 1 --2- 172.21.15.39:0/1599624368 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef881010b0 0x7fef881014c0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fef7c009990 tx=0x7fef7c009ca0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.328 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.304+0000 7fef9002b700 1 -- 172.21.15.39:0/1599624368 shutdown_connections 2023-07-21T18:41:24.329 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.304+0000 7fef9002b700 1 --2- 172.21.15.39:0/1599624368 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef880fe4b0 0x7fef88108090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.329 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.304+0000 7fef9002b700 1 --2- 172.21.15.39:0/1599624368 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef881010b0 0x7fef881014c0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.329 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.304+0000 7fef9002b700 1 --2- 172.21.15.39:0/1599624368 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef880fdb20 0x7fef880fdf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.329 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.304+0000 7fef9002b700 1 -- 172.21.15.39:0/1599624368 >> 172.21.15.39:0/1599624368 conn(0x7fef880f9770 msgr2=0x7fef880fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:24.329 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.304+0000 7fef9002b700 1 -- 172.21.15.39:0/1599624368 shutdown_connections 2023-07-21T18:41:24.329 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.305+0000 7fef9002b700 1 -- 172.21.15.39:0/1599624368 wait complete. 2023-07-21T18:41:24.329 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.305+0000 7fef9002b700 1 Processor -- start 2023-07-21T18:41:24.330 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.305+0000 7fef9002b700 1 -- start start 2023-07-21T18:41:24.331 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.305+0000 7fef9002b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef880fdb20 0x7fef88195820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:24.331 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.305+0000 7fef9002b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef880fe4b0 0x7fef88195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:24.332 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.305+0000 7fef9002b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef881010b0 0x7fef8819cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:24.332 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.305+0000 7fef9002b700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef8819d3a0 con 0x7fef881010b0 2023-07-21T18:41:24.332 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.305+0000 7fef9002b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef8819d4e0 con 0x7fef880fe4b0 2023-07-21T18:41:24.332 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.305+0000 7fef9002b700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef8819d670 con 0x7fef880fdb20 2023-07-21T18:41:24.332 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.305+0000 7fef8e5c8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef881010b0 0x7fef8819cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:24.332 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.305+0000 7fef8e5c8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef881010b0 0x7fef8819cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56502/0 (socket says 172.21.15.39:56502) 2023-07-21T18:41:24.333 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.305+0000 7fef8e5c8700 1 -- 172.21.15.39:0/1377915730 learned_addr learned my addr 172.21.15.39:0/1377915730 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:24.333 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.306+0000 7fef8ddc7700 1 --2- 172.21.15.39:0/1377915730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef880fdb20 0x7fef88195820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:24.333 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.306+0000 7fef8d5c6700 1 --2- 172.21.15.39:0/1377915730 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef880fe4b0 0x7fef88195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:24.333 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.306+0000 7fef8e5c8700 1 -- 172.21.15.39:0/1377915730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef880fdb20 msgr2=0x7fef88195820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:24.333 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.306+0000 7fef8e5c8700 1 --2- 172.21.15.39:0/1377915730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef880fdb20 0x7fef88195820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.333 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.306+0000 7fef8e5c8700 1 -- 172.21.15.39:0/1377915730 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef880fe4b0 msgr2=0x7fef88195d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:24.333 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.306+0000 7fef8e5c8700 1 --2- 172.21.15.39:0/1377915730 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef880fe4b0 0x7fef88195d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.333 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.306+0000 7fef8e5c8700 1 -- 172.21.15.39:0/1377915730 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef7c0095f0 con 0x7fef881010b0 2023-07-21T18:41:24.334 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.306+0000 7fef8ddc7700 1 --2- 172.21.15.39:0/1377915730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef880fdb20 0x7fef88195820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:41:24.334 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.306+0000 7fef8e5c8700 1 --2- 172.21.15.39:0/1377915730 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef881010b0 0x7fef8819cdd0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fef80007ee0 tx=0x7fef8000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:24.334 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.306+0000 7fef7affd700 1 -- 172.21.15.39:0/1377915730 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef80017070 con 0x7fef881010b0 2023-07-21T18:41:24.334 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.306+0000 7fef7affd700 1 -- 172.21.15.39:0/1377915730 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef8000f040 con 0x7fef881010b0 2023-07-21T18:41:24.334 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.306+0000 7fef9002b700 1 -- 172.21.15.39:0/1377915730 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef8819d950 con 0x7fef881010b0 2023-07-21T18:41:24.334 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.306+0000 7fef7affd700 1 -- 172.21.15.39:0/1377915730 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef80005740 con 0x7fef881010b0 2023-07-21T18:41:24.334 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.306+0000 7fef9002b700 1 -- 172.21.15.39:0/1377915730 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef8819de60 con 0x7fef881010b0 2023-07-21T18:41:24.335 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.308+0000 7fef7affd700 1 -- 172.21.15.39:0/1377915730 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fef800058a0 con 0x7fef881010b0 2023-07-21T18:41:24.339 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.309+0000 7fef7affd700 1 --2- 172.21.15.39:0/1377915730 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fef740a1260 0x7fef740a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:24.340 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.309+0000 7fef7affd700 1 -- 172.21.15.39:0/1377915730 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fef800cc280 con 0x7fef881010b0 2023-07-21T18:41:24.340 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.309+0000 7fef8ddc7700 1 --2- 172.21.15.39:0/1377915730 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fef740a1260 0x7fef740a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:24.340 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.309+0000 7fef8ddc7700 1 --2- 172.21.15.39:0/1377915730 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fef740a1260 0x7fef740a3710 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fef7c005f50 tx=0x7fef7c005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:24.340 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.310+0000 7fef9c1b1b80 1 Processor -- start 2023-07-21T18:41:24.340 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.310+0000 7fef9c1b1b80 1 -- start start 2023-07-21T18:41:24.340 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.310+0000 7fef9c1b1b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ea31be7480 0x55ea31be7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:24.341 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.310+0000 7fef9c1b1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea31be7dd0 0x55ea31be8690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:24.341 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.310+0000 7fef9c1b1b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ea31be8c20 0x55ea31be9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:24.341 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.310+0000 7fef9c1b1b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea31aecc60 con 0x55ea31be7480 2023-07-21T18:41:24.341 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.310+0000 7fef9c1b1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea31b3f9f0 con 0x55ea31be7dd0 2023-07-21T18:41:24.341 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.310+0000 7fef9c1b1b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea31b38e80 con 0x55ea31be8c20 2023-07-21T18:41:24.341 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.310+0000 7fef8ddc7700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ea31be7480 0x55ea31be7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:24.341 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.310+0000 7fef8ddc7700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ea31be7480 0x55ea31be7890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56506/0 (socket says 172.21.15.39:56506) 2023-07-21T18:41:24.341 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.310+0000 7fef8ddc7700 1 -- 172.21.15.39:0/3768368591 learned_addr learned my addr 172.21.15.39:0/3768368591 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:24.342 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.310+0000 7fef8e5c8700 1 --2- 172.21.15.39:0/3768368591 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ea31be8c20 0x55ea31be9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:24.342 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.310+0000 7fef8d5c6700 1 --2- 172.21.15.39:0/3768368591 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea31be7dd0 0x55ea31be8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:24.342 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.310+0000 7fef8ddc7700 1 -- 172.21.15.39:0/3768368591 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ea31be8c20 msgr2=0x55ea31be9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:24.342 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.310+0000 7fef8ddc7700 1 --2- 172.21.15.39:0/3768368591 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ea31be8c20 0x55ea31be9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.342 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.310+0000 7fef8ddc7700 1 -- 172.21.15.39:0/3768368591 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea31be7dd0 msgr2=0x55ea31be8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:24.342 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.310+0000 7fef8ddc7700 1 --2- 172.21.15.39:0/3768368591 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea31be7dd0 0x55ea31be8690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.343 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.310+0000 7fef8ddc7700 1 -- 172.21.15.39:0/3768368591 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea31966d90 con 0x55ea31be7480 2023-07-21T18:41:24.343 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.311+0000 7fef8ddc7700 1 --2- 172.21.15.39:0/3768368591 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ea31be7480 0x55ea31be7890 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fef88067390 tx=0x7fef7c01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d83a16363a4fe60 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:24.343 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.311+0000 7fef73fff700 1 -- 172.21.15.39:0/3768368591 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef7c02a070 con 0x55ea31be7480 2023-07-21T18:41:24.343 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.311+0000 7fef73fff700 1 -- 172.21.15.39:0/3768368591 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef7c01d040 con 0x55ea31be7480 2023-07-21T18:41:24.343 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.311+0000 7fef73fff700 1 -- 172.21.15.39:0/3768368591 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef7c00f4e0 con 0x55ea31be7480 2023-07-21T18:41:24.343 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.311+0000 7fef9c1b1b80 1 -- 172.21.15.39:0/3768368591 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ea31be7480 msgr2=0x55ea31be7890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:24.344 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.311+0000 7fef9c1b1b80 1 --2- 172.21.15.39:0/3768368591 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ea31be7480 0x55ea31be7890 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fef88067390 tx=0x7fef7c01e000 comp rx=0 tx=0).stop 2023-07-21T18:41:24.344 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.311+0000 7fef9c1b1b80 1 -- 172.21.15.39:0/3768368591 shutdown_connections 2023-07-21T18:41:24.344 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.311+0000 7fef9c1b1b80 1 --2- 172.21.15.39:0/3768368591 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ea31be8c20 0x55ea31be9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.344 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.311+0000 7fef9c1b1b80 1 --2- 172.21.15.39:0/3768368591 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ea31be7480 0x55ea31be7890 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.344 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.311+0000 7fef9c1b1b80 1 --2- 172.21.15.39:0/3768368591 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea31be7dd0 0x55ea31be8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.344 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.311+0000 7fef9c1b1b80 1 -- 172.21.15.39:0/3768368591 >> 172.21.15.39:0/3768368591 conn(0x55ea31bdcd50 msgr2=0x55ea31be2180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:24.344 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.312+0000 7fef9c1b1b80 1 -- 172.21.15.39:0/3768368591 shutdown_connections 2023-07-21T18:41:24.345 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.312+0000 7fef9c1b1b80 1 -- 172.21.15.39:0/3768368591 wait complete. 2023-07-21T18:41:24.345 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.312+0000 7fef9c1b1b80 1 -- start start 2023-07-21T18:41:24.345 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.312+0000 7fef9c1b1b80 1 Processor -- start 2023-07-21T18:41:24.345 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.312+0000 7fef9c1b1b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ea31be7480 0x55ea31bf1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:24.345 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.312+0000 7fef9c1b1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea31be7dd0 0x55ea31bf1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:24.345 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.312+0000 7fef9c1b1b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ea31be8c20 0x55ea31bf8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:24.345 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.312+0000 7fef9c1b1b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea31ad8a70 con 0x55ea31be8c20 2023-07-21T18:41:24.346 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.312+0000 7fef9c1b1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea31ad8770 con 0x55ea31be7dd0 2023-07-21T18:41:24.346 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.312+0000 7fef9c1b1b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea319bc8d0 con 0x55ea31be7480 2023-07-21T18:41:24.346 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.312+0000 7fef9c1b1b80 10 client.0 _ll_register_callbacks cb 0x55ea31ad6710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:41:24.346 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.312+0000 7fef8e5c8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ea31be8c20 0x55ea31bf8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:24.346 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.313+0000 7fef8e5c8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ea31be8c20 0x55ea31bf8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56516/0 (socket says 172.21.15.39:56516) 2023-07-21T18:41:24.346 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.313+0000 7fef8e5c8700 1 -- 172.21.15.39:0/563742887 learned_addr learned my addr 172.21.15.39:0/563742887 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:24.346 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.313+0000 7fef8d5c6700 1 --2- 172.21.15.39:0/563742887 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea31be7dd0 0x55ea31bf1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:24.346 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.313+0000 7fef8ddc7700 1 --2- 172.21.15.39:0/563742887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ea31be7480 0x55ea31bf1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:24.347 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.313+0000 7fef8e5c8700 1 -- 172.21.15.39:0/563742887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ea31be7480 msgr2=0x55ea31bf1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:24.347 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.313+0000 7fef8e5c8700 1 --2- 172.21.15.39:0/563742887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ea31be7480 0x55ea31bf1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.347 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.313+0000 7fef8e5c8700 1 -- 172.21.15.39:0/563742887 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea31be7dd0 msgr2=0x55ea31bf1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:24.347 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.313+0000 7fef8e5c8700 1 --2- 172.21.15.39:0/563742887 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea31be7dd0 0x55ea31bf1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.347 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.313+0000 7fef8e5c8700 1 -- 172.21.15.39:0/563742887 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef88108850 con 0x55ea31be8c20 2023-07-21T18:41:24.347 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.313+0000 7fef8e5c8700 1 --2- 172.21.15.39:0/563742887 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ea31be8c20 0x55ea31bf8680 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fef800994f0 tx=0x7fef80004c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb792d6cd34949b6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:24.347 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.313+0000 7fef717fa700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:41:24.348 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.313+0000 7fef717fa700 1 -- 172.21.15.39:0/563742887 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8000ddd0 con 0x55ea31be8c20 2023-07-21T18:41:24.348 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.313+0000 7fef9c1b1b80 1 -- 172.21.15.39:0/563742887 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ea31a924a0 con 0x55ea31be8c20 2023-07-21T18:41:24.348 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.313+0000 7fef717fa700 1 -- 172.21.15.39:0/563742887 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fef800a19b0 con 0x55ea31be8c20 2023-07-21T18:41:24.348 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.313+0000 7fef717fa700 1 -- 172.21.15.39:0/563742887 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef800b9070 con 0x55ea31be8c20 2023-07-21T18:41:24.348 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.313+0000 7fef9c1b1b80 10 client.15288 fetch_fsmap learned FSMap version 26 2023-07-21T18:41:24.348 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.313+0000 7fef9c1b1b80 1 -- 172.21.15.39:0/563742887 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55ea3194fc10 con 0x55ea31be8c20 2023-07-21T18:41:24.348 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.313+0000 7fef717fa700 1 -- 172.21.15.39:0/563742887 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ea31a924a0 con 0x55ea31be8c20 2023-07-21T18:41:24.349 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.314+0000 7fef717fa700 1 -- 172.21.15.39:0/563742887 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fef800a1520 con 0x55ea31be8c20 2023-07-21T18:41:24.349 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.314+0000 7fef9c1b1b80 10 client.15288 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T18:41:24.349 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.314+0000 7fef9c1b1b80 10 client.15288 resolve_mds: resolved name 'b' to daemon mds.b 2023-07-21T18:41:24.349 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.314+0000 7fef9c1b1b80 20 client.15288 populate_metadata read hostname 'smithi039' 2023-07-21T18:41:24.349 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.314+0000 7fef9c1b1b80 1 --2- 172.21.15.39:0/563742887 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55ea31bfc350 0x55ea31bfb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:24.349 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.314+0000 7fef9c1b1b80 4 client.15288 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:41:24.349 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.314+0000 7fef9c1b1b80 1 -- 172.21.15.39:0/563742887 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ea31b31e60 con 0x55ea31bfc350 2023-07-21T18:41:24.349 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.314+0000 7fef8ddc7700 1 --2- 172.21.15.39:0/563742887 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55ea31bfc350 0x55ea31bfb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:24.350 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.314+0000 7fef717fa700 10 client.15288 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:41:24.350 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.314+0000 7fef8ddc7700 1 --2- 172.21.15.39:0/563742887 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55ea31bfc350 0x55ea31bfb7e0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2caac440cbca6d42 server_cookie=2e2b62538bdbd94b in_seq=0 out_seq=0 2023-07-21T18:41:24.350 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.315+0000 7fef717fa700 1 -- 172.21.15.39:0/563742887 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ea31b31e60 con 0x55ea31bfc350 2023-07-21T18:41:24.350 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.315+0000 7fef717fa700 10 client.15288 handle_command_reply: tid=0 2023-07-21T18:41:24.350 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.315+0000 7fef9c1b1b80 1 client.15288 shutdown 2023-07-21T18:41:24.350 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.315+0000 7fef9c1b1b80 1 -- 172.21.15.39:0/563742887 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ea31be8c20 msgr2=0x55ea31bf8680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:24.350 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.315+0000 7fef9c1b1b80 1 --2- 172.21.15.39:0/563742887 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ea31be8c20 0x55ea31bf8680 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fef800994f0 tx=0x7fef80004c20 comp rx=0 tx=0).stop 2023-07-21T18:41:24.351 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.315+0000 7fef9c1b1b80 1 -- 172.21.15.39:0/563742887 shutdown_connections 2023-07-21T18:41:24.351 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.315+0000 7fef9c1b1b80 1 --2- 172.21.15.39:0/563742887 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55ea31bfc350 0x55ea31bfb7e0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.351 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.315+0000 7fef9c1b1b80 1 --2- 172.21.15.39:0/563742887 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ea31be8c20 0x55ea31bf8680 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.351 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.315+0000 7fef9c1b1b80 1 --2- 172.21.15.39:0/563742887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ea31be7480 0x55ea31bf1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.351 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.315+0000 7fef9c1b1b80 1 --2- 172.21.15.39:0/563742887 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea31be7dd0 0x55ea31bf1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.351 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.315+0000 7fef9c1b1b80 1 -- 172.21.15.39:0/563742887 >> 172.21.15.39:0/563742887 conn(0x55ea31bdcd50 msgr2=0x55ea31be1860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:24.351 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.315+0000 7fef717fa700 0 client.15288 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:41:24.351 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.315+0000 7fef9c1b1b80 1 -- 172.21.15.39:0/563742887 shutdown_connections 2023-07-21T18:41:24.352 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.315+0000 7fef9c1b1b80 1 -- 172.21.15.39:0/563742887 wait complete. 2023-07-21T18:41:24.352 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.315+0000 7fef9c1b1b80 20 client.15288 trim_cache size 0 max 16384 2023-07-21T18:41:24.357 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.333+0000 7fef9c1b1b80 1 Processor -- start 2023-07-21T18:41:24.357 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.333+0000 7fef9c1b1b80 1 -- start start 2023-07-21T18:41:24.357 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.333+0000 7fef9c1b1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea31be7480 0x55ea31befdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:24.357 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.333+0000 7fef9c1b1b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ea31be7dd0 0x55ea31bf8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:24.357 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.333+0000 7fef9c1b1b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ea31be8c20 0x55ea31bf8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:24.358 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.333+0000 7fef9c1b1b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea31b31e60 con 0x55ea31be8c20 2023-07-21T18:41:24.358 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.333+0000 7fef9c1b1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea31a924a0 con 0x55ea31be7480 2023-07-21T18:41:24.358 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.333+0000 7fef9c1b1b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea31b07790 con 0x55ea31be7dd0 2023-07-21T18:41:24.358 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.333+0000 7fef8e5c8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ea31be8c20 0x55ea31bf8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:24.358 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.333+0000 7fef8e5c8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ea31be8c20 0x55ea31bf8e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56524/0 (socket says 172.21.15.39:56524) 2023-07-21T18:41:24.358 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.333+0000 7fef8e5c8700 1 -- 172.21.15.39:0/2075210751 learned_addr learned my addr 172.21.15.39:0/2075210751 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:24.358 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.333+0000 7fef8ddc7700 1 --2- 172.21.15.39:0/2075210751 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea31be7480 0x55ea31befdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:24.359 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.333+0000 7fef8e5c8700 1 -- 172.21.15.39:0/2075210751 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ea31be7dd0 msgr2=0x55ea31bf8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:24.359 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.333+0000 7fef8e5c8700 1 --2- 172.21.15.39:0/2075210751 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ea31be7dd0 0x55ea31bf8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.359 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.333+0000 7fef8e5c8700 1 -- 172.21.15.39:0/2075210751 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea31be7480 msgr2=0x55ea31befdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:24.359 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.333+0000 7fef8e5c8700 1 --2- 172.21.15.39:0/2075210751 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea31be7480 0x55ea31befdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.359 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.333+0000 7fef8e5c8700 1 -- 172.21.15.39:0/2075210751 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea3198d200 con 0x55ea31be8c20 2023-07-21T18:41:24.359 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.334+0000 7fef8e5c8700 1 --2- 172.21.15.39:0/2075210751 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ea31be8c20 0x55ea31bf8e50 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fef80099260 tx=0x7fef8009d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cac7953abf38b0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:24.359 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.334+0000 7fef78ff9700 1 -- 172.21.15.39:0/2075210751 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef800b7c30 con 0x55ea31be8c20 2023-07-21T18:41:24.360 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.334+0000 7fef78ff9700 1 -- 172.21.15.39:0/2075210751 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef800a19b0 con 0x55ea31be8c20 2023-07-21T18:41:24.360 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.334+0000 7fef78ff9700 1 -- 172.21.15.39:0/2075210751 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8009d180 con 0x55ea31be8c20 2023-07-21T18:41:24.363 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.334+0000 7fef9c1b1b80 1 -- 172.21.15.39:0/2075210751 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ea31be8c20 msgr2=0x55ea31bf8e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:24.364 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.334+0000 7fef9c1b1b80 1 --2- 172.21.15.39:0/2075210751 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ea31be8c20 0x55ea31bf8e50 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fef80099260 tx=0x7fef8009d710 comp rx=0 tx=0).stop 2023-07-21T18:41:24.364 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.334+0000 7fef9c1b1b80 1 -- 172.21.15.39:0/2075210751 shutdown_connections 2023-07-21T18:41:24.364 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.334+0000 7fef9c1b1b80 1 --2- 172.21.15.39:0/2075210751 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea31be7480 0x55ea31befdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.364 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.334+0000 7fef9c1b1b80 1 --2- 172.21.15.39:0/2075210751 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ea31be8c20 0x55ea31bf8e50 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.364 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.334+0000 7fef9c1b1b80 1 --2- 172.21.15.39:0/2075210751 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ea31be7dd0 0x55ea31bf8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.364 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.334+0000 7fef9c1b1b80 1 -- 172.21.15.39:0/2075210751 >> 172.21.15.39:0/2075210751 conn(0x55ea31bdcd50 msgr2=0x55ea31c050a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:24.364 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.334+0000 7fef9c1b1b80 1 -- 172.21.15.39:0/2075210751 shutdown_connections 2023-07-21T18:41:24.365 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.334+0000 7fef9c1b1b80 1 -- 172.21.15.39:0/2075210751 wait complete. 2023-07-21T18:41:24.365 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.335+0000 7fef9c1b1b80 1 -- start start 2023-07-21T18:41:24.365 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.335+0000 7fef9c1b1b80 1 Processor -- start 2023-07-21T18:41:24.365 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.335+0000 7fef9c1b1b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ea31be7480 0x55ea31bf58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:24.365 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.335+0000 7fef9c1b1b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ea31be7dd0 0x55ea31bf5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:24.365 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.335+0000 7fef9c1b1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea31be8c20 0x55ea31c501d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:24.365 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.335+0000 7fef9c1b1b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea31bf8700 con 0x55ea31be7dd0 2023-07-21T18:41:24.366 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.335+0000 7fef9c1b1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea31bf8870 con 0x55ea31be8c20 2023-07-21T18:41:24.366 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.335+0000 7fef9c1b1b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea31bf8a30 con 0x55ea31be7480 2023-07-21T18:41:24.366 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.335+0000 7fef8d5c6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ea31be7dd0 0x55ea31bf5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:24.366 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.335+0000 7fef9c1b1b80 10 client.0 _ll_register_callbacks cb 0x55ea31ad6710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:41:24.366 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.335+0000 7fef8d5c6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ea31be7dd0 0x55ea31bf5e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56528/0 (socket says 172.21.15.39:56528) 2023-07-21T18:41:24.366 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.335+0000 7fef8d5c6700 1 -- 172.21.15.39:0/4005348868 learned_addr learned my addr 172.21.15.39:0/4005348868 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:24.366 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.335+0000 7fef8ddc7700 1 --2- 172.21.15.39:0/4005348868 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ea31be7480 0x55ea31bf58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:24.366 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.335+0000 7fef8d5c6700 1 -- 172.21.15.39:0/4005348868 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ea31be7480 msgr2=0x55ea31bf58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:24.367 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.335+0000 7fef8e5c8700 1 --2- 172.21.15.39:0/4005348868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea31be8c20 0x55ea31c501d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:24.367 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.335+0000 7fef8d5c6700 1 --2- 172.21.15.39:0/4005348868 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ea31be7480 0x55ea31bf58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.367 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.335+0000 7fef8d5c6700 1 -- 172.21.15.39:0/4005348868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea31be8c20 msgr2=0x55ea31c501d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:24.367 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.336+0000 7fef8d5c6700 1 --2- 172.21.15.39:0/4005348868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea31be8c20 0x55ea31c501d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.367 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.336+0000 7fef8d5c6700 1 -- 172.21.15.39:0/4005348868 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ea3194fc10 con 0x55ea31be7dd0 2023-07-21T18:41:24.367 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.336+0000 7fef8d5c6700 1 --2- 172.21.15.39:0/4005348868 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ea31be7dd0 0x55ea31bf5e00 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fef8400cfd0 tx=0x7fef8400f980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36803a9adf0efd02 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:24.367 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.336+0000 7fef717fa700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:41:24.367 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.336+0000 7fef717fa700 1 -- 172.21.15.39:0/4005348868 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef84014070 con 0x55ea31be7dd0 2023-07-21T18:41:24.368 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.336+0000 7fef717fa700 1 -- 172.21.15.39:0/4005348868 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fef8400e620 con 0x55ea31be7dd0 2023-07-21T18:41:24.368 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.336+0000 7fef9c1b1b80 1 -- 172.21.15.39:0/4005348868 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fef7c01d040 con 0x55ea31be7dd0 2023-07-21T18:41:24.368 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.336+0000 7fef717fa700 1 -- 172.21.15.39:0/4005348868 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef8400e7e0 con 0x55ea31be7dd0 2023-07-21T18:41:24.368 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.336+0000 7fef9c1b1b80 10 client.15294 fetch_fsmap learned FSMap version 26 2023-07-21T18:41:24.368 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.336+0000 7fef9c1b1b80 1 -- 172.21.15.39:0/4005348868 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55ea31965260 con 0x55ea31be7dd0 2023-07-21T18:41:24.368 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.336+0000 7fef717fa700 1 -- 172.21.15.39:0/4005348868 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef7c01d040 con 0x55ea31be7dd0 2023-07-21T18:41:24.368 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.336+0000 7fef717fa700 1 -- 172.21.15.39:0/4005348868 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fef84023b70 con 0x55ea31be7dd0 2023-07-21T18:41:24.369 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.336+0000 7fef9c1b1b80 10 client.15294 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T18:41:24.369 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.336+0000 7fef9c1b1b80 10 client.15294 resolve_mds: resolved name 'b' to daemon mds.b 2023-07-21T18:41:24.369 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.336+0000 7fef9c1b1b80 20 client.15294 populate_metadata read hostname 'smithi039' 2023-07-21T18:41:24.369 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.336+0000 7fef9c1b1b80 1 --2- 172.21.15.39:0/4005348868 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55ea31c55350 0x55ea31c51cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:24.369 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.336+0000 7fef9c1b1b80 4 client.15294 mds_command: new command op to 24536 tid=0[{"prefix": "status"}] 2023-07-21T18:41:24.369 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.336+0000 7fef9c1b1b80 1 -- 172.21.15.39:0/4005348868 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ea31af4000 con 0x55ea31c55350 2023-07-21T18:41:24.369 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.337+0000 7fef8ddc7700 1 --2- 172.21.15.39:0/4005348868 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55ea31c55350 0x55ea31c51cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:24.369 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.337+0000 7fef8ddc7700 1 --2- 172.21.15.39:0/4005348868 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55ea31c55350 0x55ea31c51cc0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5dd9d9711b3f562d server_cookie=262169be1ec57555 in_seq=0 out_seq=0 2023-07-21T18:41:24.370 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.337+0000 7fef717fa700 10 client.15294 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:41:24.370 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.337+0000 7fef717fa700 1 -- 172.21.15.39:0/4005348868 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55ea31af4000 con 0x55ea31c55350 2023-07-21T18:41:24.370 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.337+0000 7fef717fa700 10 client.15294 handle_command_reply: tid=0 2023-07-21T18:41:24.370 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.337+0000 7fef9c1b1b80 1 client.15294 shutdown 2023-07-21T18:41:24.370 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.337+0000 7fef9c1b1b80 1 -- 172.21.15.39:0/4005348868 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ea31be7dd0 msgr2=0x55ea31bf5e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:24.370 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.337+0000 7fef9c1b1b80 1 --2- 172.21.15.39:0/4005348868 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ea31be7dd0 0x55ea31bf5e00 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fef8400cfd0 tx=0x7fef8400f980 comp rx=0 tx=0).stop 2023-07-21T18:41:24.370 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.337+0000 7fef9c1b1b80 1 -- 172.21.15.39:0/4005348868 shutdown_connections 2023-07-21T18:41:24.371 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.337+0000 7fef9c1b1b80 1 --2- 172.21.15.39:0/4005348868 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55ea31c55350 0x55ea31c51cc0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.371 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.337+0000 7fef9c1b1b80 1 --2- 172.21.15.39:0/4005348868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea31be8c20 0x55ea31c501d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.371 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.337+0000 7fef9c1b1b80 1 --2- 172.21.15.39:0/4005348868 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ea31be7480 0x55ea31bf58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.371 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.337+0000 7fef717fa700 0 client.15294 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:41:24.371 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.337+0000 7fef9c1b1b80 1 --2- 172.21.15.39:0/4005348868 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ea31be7dd0 0x55ea31bf5e00 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.371 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.337+0000 7fef9c1b1b80 1 -- 172.21.15.39:0/4005348868 >> 172.21.15.39:0/4005348868 conn(0x55ea31bdcd50 msgr2=0x55ea31befdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:24.371 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.337+0000 7fef9c1b1b80 1 -- 172.21.15.39:0/4005348868 shutdown_connections 2023-07-21T18:41:24.371 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.337+0000 7fef9c1b1b80 1 -- 172.21.15.39:0/4005348868 wait complete. 2023-07-21T18:41:24.372 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.338+0000 7fef9c1b1b80 20 client.15294 trim_cache size 0 max 16384 2023-07-21T18:41:24.372 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2023-07-21T18:41:24.372 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "eaf8e25a-27f4-11ee-9b34-001a4aab830c", 2023-07-21T18:41:24.373 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 0, 2023-07-21T18:41:24.373 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 24536, 2023-07-21T18:41:24.373 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2023-07-21T18:41:24.373 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2023-07-21T18:41:24.373 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2023-07-21T18:41:24.373 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 79.157696467999997, 2023-07-21T18:41:24.374 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 26, 2023-07-21T18:41:24.374 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 81, 2023-07-21T18:41:24.374 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 79, 2023-07-21T18:41:24.374 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 86.029080938000007 2023-07-21T18:41:24.374 INFO:teuthology.orchestra.run.smithi039.stdout:} 2023-07-21T18:41:24.375 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.338+0000 7fef9002b700 1 -- 172.21.15.39:0/1377915730 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fef740a1260 msgr2=0x7fef740a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:24.375 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.338+0000 7fef9002b700 1 --2- 172.21.15.39:0/1377915730 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fef740a1260 0x7fef740a3710 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fef7c005f50 tx=0x7fef7c005e80 comp rx=0 tx=0).stop 2023-07-21T18:41:24.375 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.338+0000 7fef9002b700 1 -- 172.21.15.39:0/1377915730 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef881010b0 msgr2=0x7fef8819cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:24.375 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.338+0000 7fef9002b700 1 --2- 172.21.15.39:0/1377915730 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef881010b0 0x7fef8819cdd0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fef80007ee0 tx=0x7fef8000d3b0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.375 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.338+0000 7fef9002b700 1 -- 172.21.15.39:0/1377915730 shutdown_connections 2023-07-21T18:41:24.376 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.338+0000 7fef9002b700 1 --2- 172.21.15.39:0/1377915730 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fef740a1260 0x7fef740a3710 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.376 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.338+0000 7fef9002b700 1 --2- 172.21.15.39:0/1377915730 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef881010b0 0x7fef8819cdd0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.376 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.338+0000 7fef9002b700 1 --2- 172.21.15.39:0/1377915730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef880fdb20 0x7fef88195820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.376 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.338+0000 7fef9002b700 1 --2- 172.21.15.39:0/1377915730 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef880fe4b0 0x7fef88195d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:24.376 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.338+0000 7fef9002b700 1 -- 172.21.15.39:0/1377915730 >> 172.21.15.39:0/1377915730 conn(0x7fef880f9770 msgr2=0x7fef880fba60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:24.376 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.338+0000 7fef9002b700 1 -- 172.21.15.39:0/1377915730 shutdown_connections 2023-07-21T18:41:24.376 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:24.338+0000 7fef9002b700 1 -- 172.21.15.39:0/1377915730 wait complete. 2023-07-21T18:41:24.377 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:24 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/2448255077' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:41:24.377 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:24 smithi039 ceph-mon[96489]: pgmap v289: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-07-21T18:41:24.377 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:24 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/2248819757' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T18:41:24.378 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:24 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/2448255077' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:41:24.378 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:24 smithi193 ceph-mon[98204]: pgmap v289: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-07-21T18:41:24.378 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:24 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/2248819757' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T18:41:24.379 INFO:teuthology.run_tasks:Running task kclient... 2023-07-21T18:41:24.407 INFO:tasks.kclient:Mounting kernel clients... 2023-07-21T18:41:24.408 INFO:tasks.kclient:config is {'client.0': {}} 2023-07-21T18:41:24.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T18:41:24.409 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list 2023-07-21T18:41:24.409 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:24 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/2448255077' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:41:24.410 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:24 smithi155 ceph-mon[97934]: pgmap v289: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-07-21T18:41:24.410 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:24 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/2248819757' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T18:41:24.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T18:41:24.429 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link delete ceph-brx 2023-07-21T18:41:24.497 INFO:teuthology.orchestra.run.smithi039.stderr:Cannot find device "ceph-brx" 2023-07-21T18:41:24.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T18:41:24.499 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'nowsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2023-07-21T18:41:24.500 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-07-21T18:41:24.500 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-21T18:41:24.500 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph fs ls 2023-07-21T18:41:25.099 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:41:25.495 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:25 smithi039 ceph-mon[96489]: pgmap v290: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-07-21T18:41:25.621 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:25 smithi193 ceph-mon[98204]: pgmap v290: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-07-21T18:41:25.639 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:25 smithi155 ceph-mon[97934]: pgmap v290: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-07-21T18:41:26.324 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:26.300+0000 7facddf31700 1 -- 172.21.15.39:0/380640424 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7facd80feab0 msgr2=0x7facd80fef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:26.324 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:26.300+0000 7facddf31700 1 --2- 172.21.15.39:0/380640424 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7facd80feab0 0x7facd80fef20 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7faccc009990 tx=0x7faccc009ca0 comp rx=0 tx=0).stop 2023-07-21T18:41:26.324 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:26.300+0000 7facddf31700 1 -- 172.21.15.39:0/380640424 shutdown_connections 2023-07-21T18:41:26.325 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:26.300+0000 7facddf31700 1 --2- 172.21.15.39:0/380640424 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7facd80ff460 0x7facd810b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:26.325 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:26.300+0000 7facddf31700 1 --2- 172.21.15.39:0/380640424 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7facd8104ab0 0x7facd8104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:26.325 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:26.300+0000 7facddf31700 1 --2- 172.21.15.39:0/380640424 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7facd80feab0 0x7facd80fef20 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:26.325 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:26.300+0000 7facddf31700 1 -- 172.21.15.39:0/380640424 >> 172.21.15.39:0/380640424 conn(0x7facd8075b70 msgr2=0x7facd8075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:26.325 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:26.300+0000 7facddf31700 1 -- 172.21.15.39:0/380640424 shutdown_connections 2023-07-21T18:41:26.325 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:26.300+0000 7facddf31700 1 -- 172.21.15.39:0/380640424 wait complete. 2023-07-21T18:41:26.327 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:26.301+0000 7facddf31700 1 Processor -- start 2023-07-21T18:41:26.327 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:26.301+0000 7facddf31700 1 -- start start 2023-07-21T18:41:26.327 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:26.301+0000 7facddf31700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7facd80feab0 0x7facd819a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:26.327 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:26.301+0000 7facddf31700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7facd80ff460 0x7facd819ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:26.327 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:26.301+0000 7facddf31700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7facd8104ab0 0x7facd81949d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:26.327 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:26.301+0000 7facddf31700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd8195040 con 0x7facd80ff460 2023-07-21T18:41:26.328 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:26.301+0000 7facddf31700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd81951b0 con 0x7facd80feab0 2023-07-21T18:41:26.328 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:26.301+0000 7facddf31700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd8195370 con 0x7facd8104ab0 2023-07-21T18:41:26.328 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:26.302+0000 7facd6ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7facd80ff460 0x7facd819ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:26.328 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:26.302+0000 7facd6ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7facd80ff460 0x7facd819ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56540/0 (socket says 172.21.15.39:56540) 2023-07-21T18:41:26.328 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:26.302+0000 7facd6ffd700 1 -- 172.21.15.39:0/2993817424 learned_addr learned my addr 172.21.15.39:0/2993817424 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:26.328 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:26.302+0000 7facd77fe700 1 --2- 172.21.15.39:0/2993817424 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7facd80feab0 0x7facd819a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:26.329 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:26.302+0000 7facd7fff700 1 --2- 172.21.15.39:0/2993817424 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7facd8104ab0 0x7facd81949d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:26.329 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:26.302+0000 7facd6ffd700 1 -- 172.21.15.39:0/2993817424 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7facd8104ab0 msgr2=0x7facd81949d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:26.329 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:26.302+0000 7facd6ffd700 1 --2- 172.21.15.39:0/2993817424 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7facd8104ab0 0x7facd81949d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:26.329 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:26.302+0000 7facd6ffd700 1 -- 172.21.15.39:0/2993817424 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7facd80feab0 msgr2=0x7facd819a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:26.329 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:26.302+0000 7facd6ffd700 1 --2- 172.21.15.39:0/2993817424 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7facd80feab0 0x7facd819a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:26.329 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:26.302+0000 7facd6ffd700 1 -- 172.21.15.39:0/2993817424 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faccc0095f0 con 0x7facd80ff460 2023-07-21T18:41:26.329 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:26.302+0000 7facd6ffd700 1 --2- 172.21.15.39:0/2993817424 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7facd80ff460 0x7facd819ae90 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7faccc0095c0 tx=0x7faccc00f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:26.329 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:26.302+0000 7facd4ff9700 1 -- 172.21.15.39:0/2993817424 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faccc01d070 con 0x7facd80ff460 2023-07-21T18:41:26.330 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:26.302+0000 7facddf31700 1 -- 172.21.15.39:0/2993817424 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facd81955f0 con 0x7facd80ff460 2023-07-21T18:41:26.330 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:26.302+0000 7facd4ff9700 1 -- 172.21.15.39:0/2993817424 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faccc00fb50 con 0x7facd80ff460 2023-07-21T18:41:26.330 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:26.302+0000 7facd4ff9700 1 -- 172.21.15.39:0/2993817424 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faccc0173f0 con 0x7facd80ff460 2023-07-21T18:41:26.330 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:26.302+0000 7facddf31700 1 -- 172.21.15.39:0/2993817424 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facd81a1940 con 0x7facd80ff460 2023-07-21T18:41:26.330 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:26.303+0000 7facddf31700 1 -- 172.21.15.39:0/2993817424 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facd804e210 con 0x7facd80ff460 2023-07-21T18:41:26.331 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:26.305+0000 7facd4ff9700 1 -- 172.21.15.39:0/2993817424 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7faccc00fcc0 con 0x7facd80ff460 2023-07-21T18:41:26.331 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:26.305+0000 7facd4ff9700 1 --2- 172.21.15.39:0/2993817424 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7facc00a1260 0x7facc00a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:26.331 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:26.306+0000 7facd77fe700 1 --2- 172.21.15.39:0/2993817424 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7facc00a1260 0x7facc00a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:26.331 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:26.306+0000 7facd4ff9700 1 -- 172.21.15.39:0/2993817424 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8182+0+0 (secure 0 0 0) 0x7faccc0cdd80 con 0x7facd80ff460 2023-07-21T18:41:26.331 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:26.306+0000 7facd77fe700 1 --2- 172.21.15.39:0/2993817424 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7facc00a1260 0x7facc00a3710 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7facd80ffbf0 tx=0x7facc8009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:26.332 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:26.309+0000 7facd4ff9700 1 -- 172.21.15.39:0/2993817424 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7faccc0973d0 con 0x7facd80ff460 2023-07-21T18:41:26.574 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:26.551+0000 7facddf31700 1 -- 172.21.15.39:0/2993817424 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7facd8195ee0 con 0x7facd80ff460 2023-07-21T18:41:26.574 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:26.551+0000 7facd4ff9700 1 -- 172.21.15.39:0/2993817424 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) v1 ==== 53+0+73 (secure 0 0 0) 0x7faccc09a9f0 con 0x7facd80ff460 2023-07-21T18:41:26.575 INFO:teuthology.orchestra.run.smithi039.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-21T18:41:26.576 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:26.553+0000 7facddf31700 1 -- 172.21.15.39:0/2993817424 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7facc00a1260 msgr2=0x7facc00a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:26.576 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:26.553+0000 7facddf31700 1 --2- 172.21.15.39:0/2993817424 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7facc00a1260 0x7facc00a3710 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7facd80ffbf0 tx=0x7facc8009190 comp rx=0 tx=0).stop 2023-07-21T18:41:26.577 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:26.553+0000 7facddf31700 1 -- 172.21.15.39:0/2993817424 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7facd80ff460 msgr2=0x7facd819ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:26.577 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:26.553+0000 7facddf31700 1 --2- 172.21.15.39:0/2993817424 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7facd80ff460 0x7facd819ae90 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7faccc0095c0 tx=0x7faccc00f710 comp rx=0 tx=0).stop 2023-07-21T18:41:26.577 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:26.553+0000 7facddf31700 1 -- 172.21.15.39:0/2993817424 shutdown_connections 2023-07-21T18:41:26.577 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:26.553+0000 7facddf31700 1 --2- 172.21.15.39:0/2993817424 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7facc00a1260 0x7facc00a3710 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:26.577 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:26.553+0000 7facddf31700 1 --2- 172.21.15.39:0/2993817424 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7facd80feab0 0x7facd819a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:26.577 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:26.553+0000 7facddf31700 1 --2- 172.21.15.39:0/2993817424 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7facd8104ab0 0x7facd81949d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:26.577 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:26.553+0000 7facddf31700 1 --2- 172.21.15.39:0/2993817424 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7facd80ff460 0x7facd819ae90 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:26.578 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:26.553+0000 7facddf31700 1 -- 172.21.15.39:0/2993817424 >> 172.21.15.39:0/2993817424 conn(0x7facd8075b70 msgr2=0x7facd81026e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:26.578 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:26.553+0000 7facddf31700 1 -- 172.21.15.39:0/2993817424 shutdown_connections 2023-07-21T18:41:26.578 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:26.554+0000 7facddf31700 1 -- 172.21.15.39:0/2993817424 wait complete. 2023-07-21T18:41:26.958 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-21T18:41:26.958 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-21T18:41:26.958 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi039.front.sepia.ceph.com 2023-07-21T18:41:26.959 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-07-21T18:41:26.959 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-21T18:41:26.959 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-21T18:41:26.960 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-21T18:41:26.960 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-21T18:41:26.960 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-21T18:41:26.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T18:41:26.961 DEBUG:teuthology.orchestra.run.smithi039:> ip addr 2023-07-21T18:41:26.979 INFO:teuthology.orchestra.run.smithi039.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-21T18:41:26.979 INFO:teuthology.orchestra.run.smithi039.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-21T18:41:26.979 INFO:teuthology.orchestra.run.smithi039.stdout: inet 127.0.0.1/8 scope host lo 2023-07-21T18:41:26.980 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2023-07-21T18:41:26.980 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 ::1/128 scope host 2023-07-21T18:41:26.980 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2023-07-21T18:41:26.980 INFO:teuthology.orchestra.run.smithi039.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-21T18:41:26.980 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c6 brd ff:ff:ff:ff:ff:ff 2023-07-21T18:41:26.980 INFO:teuthology.orchestra.run.smithi039.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-21T18:41:26.980 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c7 brd ff:ff:ff:ff:ff:ff 2023-07-21T18:41:26.981 INFO:teuthology.orchestra.run.smithi039.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-21T18:41:26.981 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:70 brd ff:ff:ff:ff:ff:ff 2023-07-21T18:41:26.981 INFO:teuthology.orchestra.run.smithi039.stdout: inet 172.21.15.39/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-07-21T18:41:26.981 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft 41808sec preferred_lft 41808sec 2023-07-21T18:41:26.981 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::ec4:7aff:febd:1370/64 scope link 2023-07-21T18:41:26.981 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2023-07-21T18:41:26.981 INFO:teuthology.orchestra.run.smithi039.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-21T18:41:26.981 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:71 brd ff:ff:ff:ff:ff:ff 2023-07-21T18:41:26.982 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-07-21T18:41:26.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T18:41:26.982 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec bash -c ' 2023-07-21T18:41:26.983 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2023-07-21T18:41:26.983 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link add name ceph-brx type bridge 2023-07-21T18:41:26.983 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip addr flush dev ceph-brx 2023-07-21T18:41:26.983 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set ceph-brx up 2023-07-21T18:41:26.984 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-07-21T18:41:26.984 DEBUG:teuthology.orchestra.run.smithi039:> ') 2023-07-21T18:41:27.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T18:41:27.154 DEBUG:teuthology.orchestra.run.smithi039:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-21T18:41:27.183 INFO:teuthology.orchestra.run.smithi039.stdout:1 2023-07-21T18:41:27.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T18:41:27.185 DEBUG:teuthology.orchestra.run.smithi039:> route 2023-07-21T18:41:27.255 INFO:teuthology.orchestra.run.smithi039.stdout:Kernel IP routing table 2023-07-21T18:41:27.256 INFO:teuthology.orchestra.run.smithi039.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-21T18:41:27.256 INFO:teuthology.orchestra.run.smithi039.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-07-21T18:41:27.256 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-07-21T18:41:27.256 INFO:teuthology.orchestra.run.smithi039.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-21T18:41:27.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T18:41:27.257 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec bash -c ' 2023-07-21T18:41:27.257 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2023-07-21T18:41:27.258 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-07-21T18:41:27.258 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-07-21T18:41:27.258 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-07-21T18:41:27.258 DEBUG:teuthology.orchestra.run.smithi039:> ') 2023-07-21T18:41:27.336 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:26 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/2993817424' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-07-21T18:41:27.371 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:26 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/2993817424' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-07-21T18:41:27.389 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:26 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/2993817424' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-07-21T18:41:27.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T18:41:27.532 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list 2023-07-21T18:41:27.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T18:41:27.590 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list-id 2023-07-21T18:41:27.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T18:41:27.646 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec bash -c ' 2023-07-21T18:41:27.646 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2023-07-21T18:41:27.647 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-21T18:41:27.647 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-07-21T18:41:27.647 DEBUG:teuthology.orchestra.run.smithi039:> ') 2023-07-21T18:41:27.742 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-07-21T18:41:27.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T18:41:27.743 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec bash -c ' 2023-07-21T18:41:27.743 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2023-07-21T18:41:27.744 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-07-21T18:41:27.744 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-07-21T18:41:27.744 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-07-21T18:41:27.744 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-07-21T18:41:27.744 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-07-21T18:41:27.745 DEBUG:teuthology.orchestra.run.smithi039:> ') 2023-07-21T18:41:27.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T18:41:27.951 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec bash -c ' 2023-07-21T18:41:27.951 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2023-07-21T18:41:27.951 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set brx.0 up 2023-07-21T18:41:27.951 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set dev brx.0 master ceph-brx 2023-07-21T18:41:27.952 DEBUG:teuthology.orchestra.run.smithi039:> ') 2023-07-21T18:41:28.051 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-21T18:41:28.052 DEBUG:teuthology.orchestra.run.smithi039:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-07-21T18:41:28.069 INFO:teuthology.orchestra.run.smithi039.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-07-21T18:41:28.069 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-21T18:41:28.069 DEBUG:teuthology.orchestra.run.smithi039:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-07-21T18:41:28.125 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2023-07-21T18:41:28.126 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2023-07-21T18:41:28.126 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a 2023-07-21T18:41:28.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T18:41:28.126 DEBUG:teuthology.orchestra.run.smithi039:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2023-07-21T18:41:28.224 INFO:teuthology.orchestra.run.smithi039.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2023-07-21T18:41:28.225 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync". 2023-07-21T18:41:28.244 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:28 smithi039 ceph-mon[96489]: pgmap v291: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-07-21T18:41:28.359 INFO:teuthology.orchestra.run.smithi039.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2023-07-21T18:41:28.359 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync". 2023-07-21T18:41:28.360 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: resolved to: "172.21.15.39:6789,172.21.15.155:6789,172.21.15.193:6789" 2023-07-21T18:41:28.360 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: trying mount with new device syntax: 0@eaf8e25a-27f4-11ee-9b34-001a4aab830c.cephfs=/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a 2023-07-21T18:41:28.360 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync,name=0,key=0,mon_addr=172.21.15.39:6789/172.21.15.155:6789/172.21.15.193:6789" will pass to kernel 2023-07-21T18:41:28.360 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: v2 mount support check returned 2 2023-07-21T18:41:28.360 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: kernel does not support v2 syntax 2023-07-21T18:41:28.360 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.39:6789,172.21.15.155:6789,172.21.15.193:6789:/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a 2023-07-21T18:41:28.360 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync,name=0,key=0,mds_namespace=cephfs,fsid=eaf8e25a-27f4-11ee-9b34-001a4aab830c" will pass to kernel 2023-07-21T18:41:28.364 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-07-21T18:41:28.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T18:41:28.364 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-07-21T18:41:28.370 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:28 smithi193 ceph-mon[98204]: pgmap v291: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-07-21T18:41:28.389 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:28 smithi155 ceph-mon[97934]: pgmap v291: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-07-21T18:41:28.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T18:41:28.423 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-21T18:41:28.477 INFO:teuthology.orchestra.run.smithi039.stdout:client15303 2023-07-21T18:41:28.478 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-07-21T18:41:28.478 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-21T18:41:28.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T18:41:28.498 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-21T18:41:28.591 INFO:teuthology.orchestra.run.smithi039.stdout:client15303 2023-07-21T18:41:28.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T18:41:28.592 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-21T18:41:28.646 INFO:teuthology.orchestra.run.smithi039.stdout:client15303 2023-07-21T18:41:28.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T18:41:28.646 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c.client15303/status') 2023-07-21T18:41:28.705 INFO:teuthology.orchestra.run.smithi039.stdout:instance: client.15303 (3)192.168.0.1:0/4213250051 2023-07-21T18:41:28.705 INFO:teuthology.orchestra.run.smithi039.stdout:blocklisted: false 2023-07-21T18:41:28.706 INFO:teuthology.orchestra.run.smithi039.stderr:0+1 records in 2023-07-21T18:41:28.706 INFO:teuthology.orchestra.run.smithi039.stderr:0+1 records out 2023-07-21T18:41:28.706 INFO:teuthology.orchestra.run.smithi039.stderr:70 bytes copied, 2.3478e-05 s, 3.0 MB/s 2023-07-21T18:41:28.707 INFO:teuthology.run_tasks:Running task check-counter... 2023-07-21T18:41:28.718 INFO:teuthology.run_tasks:Running task fwd_scrub... 2023-07-21T18:41:28.756 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T18:41:28.883 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:28.860+0000 7fe26d967700 1 -- 172.21.15.39:0/2081811831 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2680ff670 msgr2=0x7fe2680ffae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:28.883 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:28.860+0000 7fe26d967700 1 --2- 172.21.15.39:0/2081811831 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2680ff670 0x7fe2680ffae0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe26000b0a0 tx=0x7fe26000b3b0 comp rx=0 tx=0).stop 2023-07-21T18:41:28.883 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:28.860+0000 7fe26d967700 1 -- 172.21.15.39:0/2081811831 shutdown_connections 2023-07-21T18:41:28.883 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:28.860+0000 7fe26d967700 1 --2- 172.21.15.39:0/2081811831 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2680fe370 0x7fe2680fe780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:28.884 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:28.860+0000 7fe26d967700 1 --2- 172.21.15.39:0/2081811831 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2680ff670 0x7fe2680ffae0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:28.884 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:28.860+0000 7fe26d967700 1 --2- 172.21.15.39:0/2081811831 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2680fecc0 0x7fe2680ff130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:28.884 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:28.860+0000 7fe26d967700 1 -- 172.21.15.39:0/2081811831 >> 172.21.15.39:0/2081811831 conn(0x7fe2680fa020 msgr2=0x7fe2680fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:28.884 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:28.860+0000 7fe26d967700 1 -- 172.21.15.39:0/2081811831 shutdown_connections 2023-07-21T18:41:28.884 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:28.860+0000 7fe26d967700 1 -- 172.21.15.39:0/2081811831 wait complete. 2023-07-21T18:41:28.886 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:28.861+0000 7fe26d967700 1 Processor -- start 2023-07-21T18:41:28.886 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:28.861+0000 7fe26d967700 1 -- start start 2023-07-21T18:41:28.886 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:28.861+0000 7fe26d967700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2680fe370 0x7fe268197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:28.886 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:28.861+0000 7fe26d967700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2680fecc0 0x7fe268197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:28.886 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:28.861+0000 7fe26d967700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2680ff670 0x7fe26819f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:28.886 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:28.861+0000 7fe26d967700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe26819f600 con 0x7fe2680ff670 2023-07-21T18:41:28.887 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:28.861+0000 7fe26d967700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe26819f740 con 0x7fe2680fecc0 2023-07-21T18:41:28.887 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:28.861+0000 7fe26d967700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe26819f8d0 con 0x7fe2680fe370 2023-07-21T18:41:28.887 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:28.861+0000 7fe2677fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2680ff670 0x7fe26819f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:28.887 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:28.861+0000 7fe2677fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2680ff670 0x7fe26819f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56564/0 (socket says 172.21.15.39:56564) 2023-07-21T18:41:28.887 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:28.861+0000 7fe2677fe700 1 -- 172.21.15.39:0/49284339 learned_addr learned my addr 172.21.15.39:0/49284339 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:28.887 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:28.861+0000 7fe2667fc700 1 --2- 172.21.15.39:0/49284339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2680fecc0 0x7fe268197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:28.887 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:28.861+0000 7fe266ffd700 1 --2- 172.21.15.39:0/49284339 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2680fe370 0x7fe268197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:28.888 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:28.861+0000 7fe2677fe700 1 -- 172.21.15.39:0/49284339 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2680fe370 msgr2=0x7fe268197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:28.888 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:28.861+0000 7fe2677fe700 1 --2- 172.21.15.39:0/49284339 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2680fe370 0x7fe268197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:28.888 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:28.861+0000 7fe2677fe700 1 -- 172.21.15.39:0/49284339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2680fecc0 msgr2=0x7fe268197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:28.888 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:28.861+0000 7fe2677fe700 1 --2- 172.21.15.39:0/49284339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2680fecc0 0x7fe268197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:28.889 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:28.861+0000 7fe2677fe700 1 -- 172.21.15.39:0/49284339 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe260009d00 con 0x7fe2680ff670 2023-07-21T18:41:28.889 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:28.861+0000 7fe266ffd700 1 --2- 172.21.15.39:0/49284339 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2680fe370 0x7fe268197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:41:28.889 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:28.861+0000 7fe2667fc700 1 --2- 172.21.15.39:0/49284339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2680fecc0 0x7fe268197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:41:28.889 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:28.862+0000 7fe2677fe700 1 --2- 172.21.15.39:0/49284339 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2680ff670 0x7fe26819f030 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fe26000b780 tx=0x7fe26000be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:28.890 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:28.862+0000 7fe26c965700 1 -- 172.21.15.39:0/49284339 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe26000e040 con 0x7fe2680ff670 2023-07-21T18:41:28.890 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:28.862+0000 7fe26c965700 1 -- 172.21.15.39:0/49284339 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe260004e60 con 0x7fe2680ff670 2023-07-21T18:41:28.890 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:28.862+0000 7fe26d967700 1 -- 172.21.15.39:0/49284339 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe26819fb50 con 0x7fe2680ff670 2023-07-21T18:41:28.890 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:28.862+0000 7fe26c965700 1 -- 172.21.15.39:0/49284339 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2600053b0 con 0x7fe2680ff670 2023-07-21T18:41:28.891 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:28.862+0000 7fe26d967700 1 -- 172.21.15.39:0/49284339 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe26819ff80 con 0x7fe2680ff670 2023-07-21T18:41:28.891 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:28.863+0000 7fe26d967700 1 -- 172.21.15.39:0/49284339 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe268109150 con 0x7fe2680ff670 2023-07-21T18:41:28.891 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:28.864+0000 7fe26c965700 1 -- 172.21.15.39:0/49284339 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fe260005510 con 0x7fe2680ff670 2023-07-21T18:41:28.892 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:28.864+0000 7fe26c965700 1 --2- 172.21.15.39:0/49284339 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fe2540a1220 0x7fe2540a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:28.892 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:28.865+0000 7fe26c965700 1 -- 172.21.15.39:0/49284339 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fe2600ce2f0 con 0x7fe2680ff670 2023-07-21T18:41:28.892 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:28.865+0000 7fe266ffd700 1 --2- 172.21.15.39:0/49284339 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fe2540a1220 0x7fe2540a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:28.892 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:28.865+0000 7fe266ffd700 1 --2- 172.21.15.39:0/49284339 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fe2540a1220 0x7fe2540a36d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe258009be0 tx=0x7fe258009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:28.892 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:28.869+0000 7fe26c965700 1 -- 172.21.15.39:0/49284339 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fe2600979a0 con 0x7fe2680ff670 2023-07-21T18:41:29.093 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.070+0000 7fe26d967700 1 -- 172.21.15.39:0/49284339 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe26802caa0 con 0x7fe2680ff670 2023-07-21T18:41:29.094 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.071+0000 7fe26c965700 1 -- 172.21.15.39:0/49284339 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+22447 (secure 0 0 0) 0x7fe26009afc0 con 0x7fe2680ff670 2023-07-21T18:41:29.094 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:41:29.095 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":81,"fsid":"eaf8e25a-27f4-11ee-9b34-001a4aab830c","created":"2023-07-21T18:33:06.504979+0000","modified":"2023-07-21T18:41:08.140284+0000","last_up_change":"2023-07-21T18:39:23.531009+0000","last_in_change":"2023-07-21T18:39:05.993024+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T18:36:13.572583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T18:40:02.726735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T18:40:03.990112+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03502c07-2d0f-4723-8d0b-d34c9f5b91bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6803","nonce":860193857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6805","nonce":860193857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6809","nonce":860193857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6807","nonce":860193857}]},"public_addr":"172.21.15.39:6803/860193857","cluster_addr":"172.21.15.39:6805/860193857","heartbeat_back_addr":"172.21.15.39:6809/860193857","heartbeat_front_addr":"172.21.15.39:6807/860193857","state":["exists","up"]},{"osd":1,"uuid":"0831a0c0-a1fa-4d30-9a28-513223aa69eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6801","nonce":3246721764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6803","nonce":3246721764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6807","nonce":3246721764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6805","nonce":3246721764}]},"public_addr":"172.21.15.155:6801/3246721764","cluster_addr":"172.21.15.155:6803/3246721764","heartbeat_back_addr":"172.21.15.155:6807/3246721764","heartbeat_front_addr":"172.21.15.155:6805/3246721764","state":["exists","up"]},{"osd":2,"uuid":"22df0003-bde9-48e4-b47a-cb8eafbab0db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6801","nonce":2563594449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6803","nonce":2563594449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6807","nonce":2563594449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6805","nonce":2563594449}]},"public_addr":"172.21.15.193:6801/2563594449","cluster_addr":"172.21.15.193:6803/2563594449","heartbeat_back_addr":"172.21.15.193:6807/2563594449","heartbeat_front_addr":"172.21.15.193:6805/2563594449","state":["exists","up"]},{"osd":3,"uuid":"54562558-a2e8-45dd-bc95-561569b9ae6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6811","nonce":863708386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6813","nonce":863708386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6817","nonce":863708386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6815","nonce":863708386}]},"public_addr":"172.21.15.39:6811/863708386","cluster_addr":"172.21.15.39:6813/863708386","heartbeat_back_addr":"172.21.15.39:6817/863708386","heartbeat_front_addr":"172.21.15.39:6815/863708386","state":["exists","up"]},{"osd":4,"uuid":"491118e7-ae1e-495c-9eac-ef1cdcae9600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6809","nonce":3361780673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6811","nonce":3361780673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6815","nonce":3361780673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6813","nonce":3361780673}]},"public_addr":"172.21.15.155:6809/3361780673","cluster_addr":"172.21.15.155:6811/3361780673","heartbeat_back_addr":"172.21.15.155:6815/3361780673","heartbeat_front_addr":"172.21.15.155:6813/3361780673","state":["exists","up"]},{"osd":5,"uuid":"d267bc2c-42b3-4d96-b2d1-b186dc7e6b18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6809","nonce":2376867721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6811","nonce":2376867721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6815","nonce":2376867721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6813","nonce":2376867721}]},"public_addr":"172.21.15.193:6809/2376867721","cluster_addr":"172.21.15.193:6811/2376867721","heartbeat_back_addr":"172.21.15.193:6815/2376867721","heartbeat_front_addr":"172.21.15.193:6813/2376867721","state":["exists","up"]},{"osd":6,"uuid":"beec8088-970a-473e-bd67-6eee7cbdef24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6819","nonce":2417770529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6821","nonce":2417770529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6825","nonce":2417770529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6823","nonce":2417770529}]},"public_addr":"172.21.15.39:6819/2417770529","cluster_addr":"172.21.15.39:6821/2417770529","heartbeat_back_addr":"172.21.15.39:6825/2417770529","heartbeat_front_addr":"172.21.15.39:6823/2417770529","state":["exists","up"]},{"osd":7,"uuid":"5a88ce30-1835-4254-90cc-804ed5cf1066","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6817","nonce":2542002111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6819","nonce":2542002111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6823","nonce":2542002111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6821","nonce":2542002111}]},"public_addr":"172.21.15.155:6817/2542002111","cluster_addr":"172.21.15.155:6819/2542002111","heartbeat_back_addr":"172.21.15.155:6823/2542002111","heartbeat_front_addr":"172.21.15.155:6821/2542002111","state":["exists","up"]},{"osd":8,"uuid":"d964a183-3021-4ace-9a0c-aad00f2570e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6817","nonce":2985835085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6819","nonce":2985835085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6823","nonce":2985835085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6821","nonce":2985835085}]},"public_addr":"172.21.15.193:6817/2985835085","cluster_addr":"172.21.15.193:6819/2985835085","heartbeat_back_addr":"172.21.15.193:6823/2985835085","heartbeat_front_addr":"172.21.15.193:6821/2985835085","state":["exists","up"]},{"osd":9,"uuid":"9725ea5c-49a3-4c99-bac4-e95069100b1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6827","nonce":1946493651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6829","nonce":1946493651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6833","nonce":1946493651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6831","nonce":1946493651}]},"public_addr":"172.21.15.39:6827/1946493651","cluster_addr":"172.21.15.39:6829/1946493651","heartbeat_back_addr":"172.21.15.39:6833/1946493651","heartbeat_front_addr":"172.21.15.39:6831/1946493651","state":["exists","up"]},{"osd":10,"uuid":"1fd60ccb-92db-45f0-a580-2003db0976e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6825","nonce":1234615022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6827","nonce":1234615022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6831","nonce":1234615022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6829","nonce":1234615022}]},"public_addr":"172.21.15.155:6825/1234615022","cluster_addr":"172.21.15.155:6827/1234615022","heartbeat_back_addr":"172.21.15.155:6831/1234615022","heartbeat_front_addr":"172.21.15.155:6829/1234615022","state":["exists","up"]},{"osd":11,"uuid":"efb31ea1-4775-4e5f-8740-728d6f4aeea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6825","nonce":215104663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6827","nonce":215104663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6831","nonce":215104663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6829","nonce":215104663}]},"public_addr":"172.21.15.193:6825/215104663","cluster_addr":"172.21.15.193:6827/215104663","heartbeat_back_addr":"172.21.15.193:6831/215104663","heartbeat_front_addr":"172.21.15.193:6829/215104663","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:35:30.211698+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:35:50.883846+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:36:11.124168+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:36:32.711905+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:36:54.442007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:37:14.843526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:37:36.587354+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:37:57.882374+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:38:18.729407+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:38:40.139231+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:39:01.290862+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:39:21.767537+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:0/2902600414":"2023-07-22T18:34:07.039823+0000","172.21.15.39:6801/2893866425":"2023-07-22T18:34:07.039823+0000","172.21.15.39:0/19717002":"2023-07-22T18:34:07.039823+0000","172.21.15.39:6800/2893866425":"2023-07-22T18:34:07.039823+0000","172.21.15.39:6800/889516631":"2023-07-22T18:33:40.852706+0000","172.21.15.39:6801/889516631":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/3636510379":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/1275748831":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/3819152602":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/1127767082":"2023-07-22T18:34:07.039823+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T18:41:29.096 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.073+0000 7fe26d967700 1 -- 172.21.15.39:0/49284339 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fe2540a1220 msgr2=0x7fe2540a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:29.097 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.073+0000 7fe26d967700 1 --2- 172.21.15.39:0/49284339 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fe2540a1220 0x7fe2540a36d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe258009be0 tx=0x7fe258009260 comp rx=0 tx=0).stop 2023-07-21T18:41:29.097 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.073+0000 7fe26d967700 1 -- 172.21.15.39:0/49284339 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2680ff670 msgr2=0x7fe26819f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:29.097 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.073+0000 7fe26d967700 1 --2- 172.21.15.39:0/49284339 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2680ff670 0x7fe26819f030 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fe26000b780 tx=0x7fe26000be10 comp rx=0 tx=0).stop 2023-07-21T18:41:29.097 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.073+0000 7fe26d967700 1 -- 172.21.15.39:0/49284339 shutdown_connections 2023-07-21T18:41:29.097 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.073+0000 7fe26d967700 1 --2- 172.21.15.39:0/49284339 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fe2540a1220 0x7fe2540a36d0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:29.097 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.073+0000 7fe26d967700 1 --2- 172.21.15.39:0/49284339 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2680ff670 0x7fe26819f030 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:29.097 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.073+0000 7fe26d967700 1 --2- 172.21.15.39:0/49284339 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2680fe370 0x7fe268197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:29.098 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.073+0000 7fe26d967700 1 --2- 172.21.15.39:0/49284339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2680fecc0 0x7fe268197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:29.098 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.073+0000 7fe26d967700 1 -- 172.21.15.39:0/49284339 >> 172.21.15.39:0/49284339 conn(0x7fe2680fa020 msgr2=0x7fe2680fbb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:29.098 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.073+0000 7fe26d967700 1 -- 172.21.15.39:0/49284339 shutdown_connections 2023-07-21T18:41:29.098 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.073+0000 7fe26d967700 1 -- 172.21.15.39:0/49284339 wait complete. 2023-07-21T18:41:29.109 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-21T18:36:13.572583+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-21T18:40:02.726735+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-21T18:40:03.990112+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-21T18:41:29.110 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-21T18:41:29.240 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.216+0000 7f49871fe700 1 -- 172.21.15.39:0/987627008 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4980100560 msgr2=0x7f4980106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:29.240 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.216+0000 7f49871fe700 1 --2- 172.21.15.39:0/987627008 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4980100560 0x7f4980106e20 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f497800b0a0 tx=0x7f497800b3b0 comp rx=0 tx=0).stop 2023-07-21T18:41:29.240 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.217+0000 7f49871fe700 1 -- 172.21.15.39:0/987627008 shutdown_connections 2023-07-21T18:41:29.240 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.217+0000 7f49871fe700 1 --2- 172.21.15.39:0/987627008 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4980100560 0x7f4980106e20 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:29.241 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.217+0000 7f49871fe700 1 --2- 172.21.15.39:0/987627008 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49800fe9e0 0x7f49800fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:29.241 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.217+0000 7f49871fe700 1 --2- 172.21.15.39:0/987627008 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49800ffbd0 0x7f4980100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:29.241 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.217+0000 7f49871fe700 1 -- 172.21.15.39:0/987627008 >> 172.21.15.39:0/987627008 conn(0x7f49800fa000 msgr2=0x7f49800fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:29.241 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.217+0000 7f49871fe700 1 -- 172.21.15.39:0/987627008 shutdown_connections 2023-07-21T18:41:29.241 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.217+0000 7f49871fe700 1 -- 172.21.15.39:0/987627008 wait complete. 2023-07-21T18:41:29.241 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.217+0000 7f49871fe700 1 Processor -- start 2023-07-21T18:41:29.241 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.217+0000 7f49871fe700 1 -- start start 2023-07-21T18:41:29.243 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.217+0000 7f49871fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49800fe9e0 0x7f4980102c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:29.243 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.217+0000 7f49871fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49800ffbd0 0x7f4980101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:29.243 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.217+0000 7f49871fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4980100560 0x7f49801017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:29.243 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.217+0000 7f49871fe700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4980101f50 con 0x7f4980100560 2023-07-21T18:41:29.243 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.217+0000 7f49871fe700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49801020c0 con 0x7f49800fe9e0 2023-07-21T18:41:29.243 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.217+0000 7f49871fe700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f498019f970 con 0x7f49800ffbd0 2023-07-21T18:41:29.244 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.218+0000 7f498579b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4980100560 0x7f49801017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:29.244 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.218+0000 7f498579b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4980100560 0x7f49801017b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56578/0 (socket says 172.21.15.39:56578) 2023-07-21T18:41:29.244 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.218+0000 7f498579b700 1 -- 172.21.15.39:0/3935803527 learned_addr learned my addr 172.21.15.39:0/3935803527 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:29.244 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.218+0000 7f4984f9a700 1 --2- 172.21.15.39:0/3935803527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49800fe9e0 0x7f4980102c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:29.244 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.218+0000 7f497ffff700 1 --2- 172.21.15.39:0/3935803527 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49800ffbd0 0x7f4980101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:29.244 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.218+0000 7f498579b700 1 -- 172.21.15.39:0/3935803527 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49800ffbd0 msgr2=0x7f4980101270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:29.244 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.218+0000 7f498579b700 1 --2- 172.21.15.39:0/3935803527 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49800ffbd0 0x7f4980101270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:29.245 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.218+0000 7f498579b700 1 -- 172.21.15.39:0/3935803527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49800fe9e0 msgr2=0x7f4980102c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:29.245 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.218+0000 7f498579b700 1 --2- 172.21.15.39:0/3935803527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49800fe9e0 0x7f4980102c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:29.245 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.218+0000 7f498579b700 1 -- 172.21.15.39:0/3935803527 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4978009d00 con 0x7f4980100560 2023-07-21T18:41:29.245 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.218+0000 7f498579b700 1 --2- 172.21.15.39:0/3935803527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4980100560 0x7f49801017b0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f4978009fd0 tx=0x7f4978004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:29.245 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.218+0000 7f497dffb700 1 -- 172.21.15.39:0/3935803527 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497800e040 con 0x7f4980100560 2023-07-21T18:41:29.245 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.218+0000 7f497dffb700 1 -- 172.21.15.39:0/3935803527 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f497801d690 con 0x7f4980100560 2023-07-21T18:41:29.245 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.218+0000 7f49871fe700 1 -- 172.21.15.39:0/3935803527 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f498019fbf0 con 0x7f4980100560 2023-07-21T18:41:29.245 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.218+0000 7f497dffb700 1 -- 172.21.15.39:0/3935803527 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4978004260 con 0x7f4980100560 2023-07-21T18:41:29.246 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.219+0000 7f49871fe700 1 -- 172.21.15.39:0/3935803527 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49801a0070 con 0x7f4980100560 2023-07-21T18:41:29.246 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.220+0000 7f49871fe700 1 -- 172.21.15.39:0/3935803527 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4980106a70 con 0x7f4980100560 2023-07-21T18:41:29.246 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.221+0000 7f497dffb700 1 -- 172.21.15.39:0/3935803527 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f497801d800 con 0x7f4980100560 2023-07-21T18:41:29.247 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.221+0000 7f497dffb700 1 --2- 172.21.15.39:0/3935803527 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f49700a11e0 0x7f49700a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:29.247 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.221+0000 7f497dffb700 1 -- 172.21.15.39:0/3935803527 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f49780ce300 con 0x7f4980100560 2023-07-21T18:41:29.247 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.221+0000 7f4984f9a700 1 --2- 172.21.15.39:0/3935803527 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f49700a11e0 0x7f49700a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:29.247 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.222+0000 7f4984f9a700 1 --2- 172.21.15.39:0/3935803527 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f49700a11e0 0x7f49700a3690 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f49800ffa30 tx=0x7f497400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:29.248 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.225+0000 7f497dffb700 1 -- 172.21.15.39:0/3935803527 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f49780979b0 con 0x7f4980100560 2023-07-21T18:41:29.438 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:29 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/49284339' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T18:41:29.448 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.425+0000 7f49871fe700 1 -- 172.21.15.39:0/3935803527 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f498002ca70 con 0x7f4980100560 2023-07-21T18:41:29.449 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.426+0000 7f497dffb700 1 -- 172.21.15.39:0/3935803527 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f497809afd0 con 0x7f4980100560 2023-07-21T18:41:29.449 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2023-07-21T18:41:29.451 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.428+0000 7f49871fe700 1 -- 172.21.15.39:0/3935803527 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f49700a11e0 msgr2=0x7f49700a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:29.451 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.428+0000 7f49871fe700 1 --2- 172.21.15.39:0/3935803527 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f49700a11e0 0x7f49700a3690 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f49800ffa30 tx=0x7f497400a380 comp rx=0 tx=0).stop 2023-07-21T18:41:29.451 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.428+0000 7f49871fe700 1 -- 172.21.15.39:0/3935803527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4980100560 msgr2=0x7f49801017b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:29.451 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.428+0000 7f49871fe700 1 --2- 172.21.15.39:0/3935803527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4980100560 0x7f49801017b0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f4978009fd0 tx=0x7f4978004c80 comp rx=0 tx=0).stop 2023-07-21T18:41:29.451 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.428+0000 7f49871fe700 1 -- 172.21.15.39:0/3935803527 shutdown_connections 2023-07-21T18:41:29.451 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.428+0000 7f49871fe700 1 --2- 172.21.15.39:0/3935803527 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f49700a11e0 0x7f49700a3690 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:29.452 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.428+0000 7f49871fe700 1 --2- 172.21.15.39:0/3935803527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49800fe9e0 0x7f4980102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:29.452 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.428+0000 7f49871fe700 1 --2- 172.21.15.39:0/3935803527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4980100560 0x7f49801017b0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:29.452 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.428+0000 7f49871fe700 1 --2- 172.21.15.39:0/3935803527 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49800ffbd0 0x7f4980101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:29.452 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.428+0000 7f49871fe700 1 -- 172.21.15.39:0/3935803527 >> 172.21.15.39:0/3935803527 conn(0x7f49800fa000 msgr2=0x7f49800fc1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:29.452 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.428+0000 7f49871fe700 1 -- 172.21.15.39:0/3935803527 shutdown_connections 2023-07-21T18:41:29.452 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.428+0000 7f49871fe700 1 -- 172.21.15.39:0/3935803527 wait complete. 2023-07-21T18:41:29.463 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-21T18:41:29.592 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.569+0000 7f84adc6b700 1 -- 172.21.15.39:0/778374000 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84a80ffbd0 msgr2=0x7f84a8100020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:29.593 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.569+0000 7f84adc6b700 1 --2- 172.21.15.39:0/778374000 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84a80ffbd0 0x7f84a8100020 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f849c009990 tx=0x7f849c009ca0 comp rx=0 tx=0).stop 2023-07-21T18:41:29.593 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.569+0000 7f84adc6b700 1 -- 172.21.15.39:0/778374000 shutdown_connections 2023-07-21T18:41:29.593 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.569+0000 7f84adc6b700 1 --2- 172.21.15.39:0/778374000 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84a80fe9e0 0x7f84a80fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:29.593 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.569+0000 7f84adc6b700 1 --2- 172.21.15.39:0/778374000 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f84a8100560 0x7f84a8106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:29.593 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.569+0000 7f84adc6b700 1 --2- 172.21.15.39:0/778374000 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84a80ffbd0 0x7f84a8100020 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:29.593 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.569+0000 7f84adc6b700 1 -- 172.21.15.39:0/778374000 >> 172.21.15.39:0/778374000 conn(0x7f84a80fa000 msgr2=0x7f84a80fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:29.594 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.569+0000 7f84adc6b700 1 -- 172.21.15.39:0/778374000 shutdown_connections 2023-07-21T18:41:29.594 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.569+0000 7f84adc6b700 1 -- 172.21.15.39:0/778374000 wait complete. 2023-07-21T18:41:29.595 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.570+0000 7f84adc6b700 1 Processor -- start 2023-07-21T18:41:29.596 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.570+0000 7f84adc6b700 1 -- start start 2023-07-21T18:41:29.596 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.571+0000 7f84adc6b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84a80fe9e0 0x7f84a81947b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:29.596 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.571+0000 7f84adc6b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84a80ffbd0 0x7f84a8194cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:29.596 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.571+0000 7f84adc6b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f84a8100560 0x7f84a819bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:29.596 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.571+0000 7f84adc6b700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a819c330 con 0x7f84a80fe9e0 2023-07-21T18:41:29.597 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.571+0000 7f84adc6b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a819c470 con 0x7f84a80ffbd0 2023-07-21T18:41:29.597 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.571+0000 7f84adc6b700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a819c600 con 0x7f84a8100560 2023-07-21T18:41:29.597 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.571+0000 7f84a77fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84a80fe9e0 0x7f84a81947b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:29.597 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.571+0000 7f84a77fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84a80fe9e0 0x7f84a81947b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56588/0 (socket says 172.21.15.39:56588) 2023-07-21T18:41:29.597 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.571+0000 7f84a77fe700 1 -- 172.21.15.39:0/649896149 learned_addr learned my addr 172.21.15.39:0/649896149 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:29.597 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.571+0000 7f84a7fff700 1 --2- 172.21.15.39:0/649896149 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f84a8100560 0x7f84a819bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:29.597 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.571+0000 7f84a6ffd700 1 --2- 172.21.15.39:0/649896149 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84a80ffbd0 0x7f84a8194cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:29.598 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.571+0000 7f84a77fe700 1 -- 172.21.15.39:0/649896149 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f84a8100560 msgr2=0x7f84a819bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:29.598 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.571+0000 7f84a77fe700 1 --2- 172.21.15.39:0/649896149 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f84a8100560 0x7f84a819bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:29.598 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.571+0000 7f84a77fe700 1 -- 172.21.15.39:0/649896149 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84a80ffbd0 msgr2=0x7f84a8194cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:29.598 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.571+0000 7f84a77fe700 1 --2- 172.21.15.39:0/649896149 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84a80ffbd0 0x7f84a8194cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:29.598 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.571+0000 7f84a77fe700 1 -- 172.21.15.39:0/649896149 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f849c0095f0 con 0x7f84a80fe9e0 2023-07-21T18:41:29.598 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.571+0000 7f84a7fff700 1 --2- 172.21.15.39:0/649896149 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f84a8100560 0x7f84a819bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:41:29.598 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.571+0000 7f84a6ffd700 1 --2- 172.21.15.39:0/649896149 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84a80ffbd0 0x7f84a8194cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:41:29.598 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.571+0000 7f84a77fe700 1 --2- 172.21.15.39:0/649896149 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84a80fe9e0 0x7f84a81947b0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f849800d980 tx=0x7f849800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:29.599 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.571+0000 7f84a4ff9700 1 -- 172.21.15.39:0/649896149 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f849800de60 con 0x7f84a80fe9e0 2023-07-21T18:41:29.599 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.571+0000 7f84adc6b700 1 -- 172.21.15.39:0/649896149 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84a819c8e0 con 0x7f84a80fe9e0 2023-07-21T18:41:29.599 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.571+0000 7f84a4ff9700 1 -- 172.21.15.39:0/649896149 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84980044e0 con 0x7f84a80fe9e0 2023-07-21T18:41:29.599 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.571+0000 7f84a4ff9700 1 -- 172.21.15.39:0/649896149 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8498003be0 con 0x7f84a80fe9e0 2023-07-21T18:41:29.599 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.571+0000 7f84adc6b700 1 -- 172.21.15.39:0/649896149 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84a819cdf0 con 0x7f84a80fe9e0 2023-07-21T18:41:29.599 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.572+0000 7f84adc6b700 1 -- 172.21.15.39:0/649896149 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84a8106020 con 0x7f84a80fe9e0 2023-07-21T18:41:29.600 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.574+0000 7f84a4ff9700 1 -- 172.21.15.39:0/649896149 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f8498009bb0 con 0x7f84a80fe9e0 2023-07-21T18:41:29.600 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.574+0000 7f84a4ff9700 1 --2- 172.21.15.39:0/649896149 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f84900a11e0 0x7f84900a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:29.601 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.574+0000 7f84a4ff9700 1 -- 172.21.15.39:0/649896149 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f84980ccbb0 con 0x7f84a80fe9e0 2023-07-21T18:41:29.601 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.574+0000 7f84a6ffd700 1 --2- 172.21.15.39:0/649896149 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f84900a11e0 0x7f84900a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:29.601 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.575+0000 7f84a6ffd700 1 --2- 172.21.15.39:0/649896149 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f84900a11e0 0x7f84900a3690 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f849c00b2e0 tx=0x7f849c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:29.601 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.578+0000 7f84a4ff9700 1 -- 172.21.15.39:0/649896149 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f84980961d0 con 0x7f84a80fe9e0 2023-07-21T18:41:29.620 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:29 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/49284339' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T18:41:29.639 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:29 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/49284339' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T18:41:29.800 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.777+0000 7f84adc6b700 1 -- 172.21.15.39:0/649896149 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f84a802ca00 con 0x7f84a80fe9e0 2023-07-21T18:41:29.800 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.777+0000 7f84a4ff9700 1 -- 172.21.15.39:0/649896149 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v81) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8498009940 con 0x7f84a80fe9e0 2023-07-21T18:41:29.801 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2023-07-21T18:41:29.803 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.780+0000 7f84adc6b700 1 -- 172.21.15.39:0/649896149 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f84900a11e0 msgr2=0x7f84900a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:29.803 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.780+0000 7f84adc6b700 1 --2- 172.21.15.39:0/649896149 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f84900a11e0 0x7f84900a3690 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f849c00b2e0 tx=0x7f849c005fd0 comp rx=0 tx=0).stop 2023-07-21T18:41:29.803 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.780+0000 7f84adc6b700 1 -- 172.21.15.39:0/649896149 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84a80fe9e0 msgr2=0x7f84a81947b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:29.803 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.780+0000 7f84adc6b700 1 --2- 172.21.15.39:0/649896149 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84a80fe9e0 0x7f84a81947b0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f849800d980 tx=0x7f849800dd40 comp rx=0 tx=0).stop 2023-07-21T18:41:29.803 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.780+0000 7f84adc6b700 1 -- 172.21.15.39:0/649896149 shutdown_connections 2023-07-21T18:41:29.803 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.780+0000 7f84adc6b700 1 --2- 172.21.15.39:0/649896149 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f84900a11e0 0x7f84900a3690 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:29.804 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.780+0000 7f84adc6b700 1 --2- 172.21.15.39:0/649896149 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f84a8100560 0x7f84a819bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:29.804 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.780+0000 7f84adc6b700 1 --2- 172.21.15.39:0/649896149 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84a80fe9e0 0x7f84a81947b0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:29.804 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.780+0000 7f84adc6b700 1 --2- 172.21.15.39:0/649896149 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84a80ffbd0 0x7f84a8194cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:29.804 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.780+0000 7f84adc6b700 1 -- 172.21.15.39:0/649896149 >> 172.21.15.39:0/649896149 conn(0x7f84a80fa000 msgr2=0x7f84a8103160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:29.804 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.780+0000 7f84adc6b700 1 -- 172.21.15.39:0/649896149 shutdown_connections 2023-07-21T18:41:29.804 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.780+0000 7f84adc6b700 1 -- 172.21.15.39:0/649896149 wait complete. 2023-07-21T18:41:29.815 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-21T18:41:29.945 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.922+0000 7f9273100700 1 -- 172.21.15.39:0/2672916054 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f926c100d00 msgr2=0x7f926c101150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:29.945 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.922+0000 7f9273100700 1 --2- 172.21.15.39:0/2672916054 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f926c100d00 0x7f926c101150 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9258009990 tx=0x7f9258009ca0 comp rx=0 tx=0).stop 2023-07-21T18:41:29.945 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.922+0000 7f9273100700 1 -- 172.21.15.39:0/2672916054 shutdown_connections 2023-07-21T18:41:29.945 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.922+0000 7f9273100700 1 --2- 172.21.15.39:0/2672916054 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f926c0ffb10 0x7f926c0fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:29.945 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.922+0000 7f9273100700 1 --2- 172.21.15.39:0/2672916054 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f926c101690 0x7f926c107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:29.946 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.922+0000 7f9273100700 1 --2- 172.21.15.39:0/2672916054 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f926c100d00 0x7f926c101150 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:29.946 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.922+0000 7f9273100700 1 -- 172.21.15.39:0/2672916054 >> 172.21.15.39:0/2672916054 conn(0x7f926c0fb130 msgr2=0x7f926c0fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:29.946 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.922+0000 7f9273100700 1 -- 172.21.15.39:0/2672916054 shutdown_connections 2023-07-21T18:41:29.946 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.922+0000 7f9273100700 1 -- 172.21.15.39:0/2672916054 wait complete. 2023-07-21T18:41:29.948 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.922+0000 7f9273100700 1 Processor -- start 2023-07-21T18:41:29.948 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.922+0000 7f9273100700 1 -- start start 2023-07-21T18:41:29.948 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.923+0000 7f9273100700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f926c0ffb10 0x7f926c0742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:29.948 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.923+0000 7f9273100700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f926c100d00 0x7f926c072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:29.948 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.923+0000 7f9273100700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f926c101690 0x7f926c072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:29.949 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.923+0000 7f9273100700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f926c0735e0 con 0x7f926c101690 2023-07-21T18:41:29.949 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.923+0000 7f9273100700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f926c073750 con 0x7f926c0ffb10 2023-07-21T18:41:29.949 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.923+0000 7f9273100700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f926c19f900 con 0x7f926c100d00 2023-07-21T18:41:29.949 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.923+0000 7f927169d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f926c101690 0x7f926c072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:29.949 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.923+0000 7f927169d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f926c101690 0x7f926c072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56604/0 (socket says 172.21.15.39:56604) 2023-07-21T18:41:29.949 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.923+0000 7f927169d700 1 -- 172.21.15.39:0/1822928010 learned_addr learned my addr 172.21.15.39:0/1822928010 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:29.949 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.923+0000 7f926bfff700 1 --2- 172.21.15.39:0/1822928010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f926c100d00 0x7f926c072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:29.950 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.923+0000 7f9270e9c700 1 --2- 172.21.15.39:0/1822928010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f926c0ffb10 0x7f926c0742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:29.950 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.923+0000 7f927169d700 1 -- 172.21.15.39:0/1822928010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f926c100d00 msgr2=0x7f926c072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:29.950 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.923+0000 7f927169d700 1 --2- 172.21.15.39:0/1822928010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f926c100d00 0x7f926c072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:29.950 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.923+0000 7f927169d700 1 -- 172.21.15.39:0/1822928010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f926c0ffb10 msgr2=0x7f926c0742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:29.950 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.923+0000 7f927169d700 1 --2- 172.21.15.39:0/1822928010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f926c0ffb10 0x7f926c0742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:29.950 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.923+0000 7f927169d700 1 -- 172.21.15.39:0/1822928010 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92580095f0 con 0x7f926c101690 2023-07-21T18:41:29.950 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.923+0000 7f927169d700 1 --2- 172.21.15.39:0/1822928010 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f926c101690 0x7f926c072e40 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f926400ef30 tx=0x7f926400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:29.951 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.923+0000 7f9269ffb700 1 -- 172.21.15.39:0/1822928010 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9264018070 con 0x7f926c101690 2023-07-21T18:41:29.951 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.923+0000 7f9269ffb700 1 -- 172.21.15.39:0/1822928010 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f926400f040 con 0x7f926c101690 2023-07-21T18:41:29.951 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.923+0000 7f9273100700 1 -- 172.21.15.39:0/1822928010 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f926c19fb80 con 0x7f926c101690 2023-07-21T18:41:29.951 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.923+0000 7f9269ffb700 1 -- 172.21.15.39:0/1822928010 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f926400cdb0 con 0x7f926c101690 2023-07-21T18:41:29.951 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.924+0000 7f9273100700 1 -- 172.21.15.39:0/1822928010 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f926c1a0090 con 0x7f926c101690 2023-07-21T18:41:29.951 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.925+0000 7f9273100700 1 -- 172.21.15.39:0/1822928010 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f926c10b330 con 0x7f926c101690 2023-07-21T18:41:29.952 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.925+0000 7f9269ffb700 1 -- 172.21.15.39:0/1822928010 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f9264003a00 con 0x7f926c101690 2023-07-21T18:41:29.952 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.926+0000 7f9269ffb700 1 --2- 172.21.15.39:0/1822928010 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f925c0a1260 0x7f925c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:29.952 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.926+0000 7f9270e9c700 1 --2- 172.21.15.39:0/1822928010 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f925c0a1260 0x7f925c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:29.953 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.927+0000 7f9269ffb700 1 -- 172.21.15.39:0/1822928010 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f92640cc610 con 0x7f926c101690 2023-07-21T18:41:29.953 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.927+0000 7f9270e9c700 1 --2- 172.21.15.39:0/1822928010 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f925c0a1260 0x7f925c0a3710 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f926c100b60 tx=0x7f9260009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:29.953 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:29.931+0000 7f9269ffb700 1 -- 172.21.15.39:0/1822928010 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f9264095cc0 con 0x7f926c101690 2023-07-21T18:41:30.153 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.130+0000 7f9273100700 1 -- 172.21.15.39:0/1822928010 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f926c106a00 con 0x7f926c101690 2023-07-21T18:41:30.154 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.131+0000 7f9269ffb700 1 -- 172.21.15.39:0/1822928010 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v81) v1 ==== 100+0+11 (secure 0 0 0) 0x7f92640992e0 con 0x7f926c101690 2023-07-21T18:41:30.154 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 64 2023-07-21T18:41:30.156 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.133+0000 7f9273100700 1 -- 172.21.15.39:0/1822928010 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f925c0a1260 msgr2=0x7f925c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:30.157 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.133+0000 7f9273100700 1 --2- 172.21.15.39:0/1822928010 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f925c0a1260 0x7f925c0a3710 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f926c100b60 tx=0x7f9260009260 comp rx=0 tx=0).stop 2023-07-21T18:41:30.157 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.133+0000 7f9273100700 1 -- 172.21.15.39:0/1822928010 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f926c101690 msgr2=0x7f926c072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:30.157 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.133+0000 7f9273100700 1 --2- 172.21.15.39:0/1822928010 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f926c101690 0x7f926c072e40 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f926400ef30 tx=0x7f926400c3a0 comp rx=0 tx=0).stop 2023-07-21T18:41:30.157 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.133+0000 7f9273100700 1 -- 172.21.15.39:0/1822928010 shutdown_connections 2023-07-21T18:41:30.158 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.133+0000 7f9273100700 1 --2- 172.21.15.39:0/1822928010 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f925c0a1260 0x7f925c0a3710 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:30.158 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.133+0000 7f9273100700 1 --2- 172.21.15.39:0/1822928010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f926c0ffb10 0x7f926c0742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:30.158 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.133+0000 7f9273100700 1 --2- 172.21.15.39:0/1822928010 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f926c101690 0x7f926c072e40 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:30.159 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.133+0000 7f9273100700 1 --2- 172.21.15.39:0/1822928010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f926c100d00 0x7f926c072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:30.159 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.133+0000 7f9273100700 1 -- 172.21.15.39:0/1822928010 >> 172.21.15.39:0/1822928010 conn(0x7f926c0fb130 msgr2=0x7f926c0fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:30.159 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.133+0000 7f9273100700 1 -- 172.21.15.39:0/1822928010 shutdown_connections 2023-07-21T18:41:30.159 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.133+0000 7f9273100700 1 -- 172.21.15.39:0/1822928010 wait complete. 2023-07-21T18:41:30.170 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T18:41:30.300 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.274+0000 7f84da21d700 1 -- 172.21.15.39:0/1355520550 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84d4100550 msgr2=0x7f84d4102930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:30.300 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.274+0000 7f84da21d700 1 --2- 172.21.15.39:0/1355520550 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84d4100550 0x7f84d4102930 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f84bc009990 tx=0x7f84bc009ca0 comp rx=0 tx=0).stop 2023-07-21T18:41:30.301 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.274+0000 7f84da21d700 1 -- 172.21.15.39:0/1355520550 shutdown_connections 2023-07-21T18:41:30.301 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.274+0000 7f84da21d700 1 --2- 172.21.15.39:0/1355520550 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f84d4105790 0x7f84d4107bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:30.301 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.274+0000 7f84da21d700 1 --2- 172.21.15.39:0/1355520550 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84d4100550 0x7f84d4102930 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:30.301 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.274+0000 7f84da21d700 1 --2- 172.21.15.39:0/1355520550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84d4102e70 0x7f84d4105250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:30.302 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.274+0000 7f84da21d700 1 -- 172.21.15.39:0/1355520550 >> 172.21.15.39:0/1355520550 conn(0x7f84d40f9fd0 msgr2=0x7f84d40fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:30.302 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.274+0000 7f84da21d700 1 -- 172.21.15.39:0/1355520550 shutdown_connections 2023-07-21T18:41:30.302 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.274+0000 7f84da21d700 1 -- 172.21.15.39:0/1355520550 wait complete. 2023-07-21T18:41:30.302 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.274+0000 7f84da21d700 1 Processor -- start 2023-07-21T18:41:30.303 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.274+0000 7f84da21d700 1 -- start start 2023-07-21T18:41:30.303 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.275+0000 7f84da21d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f84d4100550 0x7f84d4195730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:30.303 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.275+0000 7f84da21d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84d4102e70 0x7f84d4195c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:30.304 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.275+0000 7f84da21d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84d4105790 0x7f84d419cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:30.304 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.275+0000 7f84da21d700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d419d2b0 con 0x7f84d4102e70 2023-07-21T18:41:30.304 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.275+0000 7f84da21d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d419d3f0 con 0x7f84d4105790 2023-07-21T18:41:30.304 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.275+0000 7f84da21d700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d419d580 con 0x7f84d4100550 2023-07-21T18:41:30.305 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.275+0000 7f84d2ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84d4102e70 0x7f84d4195c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:30.305 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.275+0000 7f84d2ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84d4102e70 0x7f84d4195c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56616/0 (socket says 172.21.15.39:56616) 2023-07-21T18:41:30.305 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.275+0000 7f84d2ffd700 1 -- 172.21.15.39:0/2414396009 learned_addr learned my addr 172.21.15.39:0/2414396009 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:30.305 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.275+0000 7f84d37fe700 1 --2- 172.21.15.39:0/2414396009 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f84d4100550 0x7f84d4195730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:30.306 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.275+0000 7f84d3fff700 1 --2- 172.21.15.39:0/2414396009 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84d4105790 0x7f84d419cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:30.306 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.275+0000 7f84d2ffd700 1 -- 172.21.15.39:0/2414396009 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f84d4100550 msgr2=0x7f84d4195730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:30.306 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.275+0000 7f84d2ffd700 1 --2- 172.21.15.39:0/2414396009 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f84d4100550 0x7f84d4195730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:30.306 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.275+0000 7f84d2ffd700 1 -- 172.21.15.39:0/2414396009 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84d4105790 msgr2=0x7f84d419cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:30.307 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.275+0000 7f84d2ffd700 1 --2- 172.21.15.39:0/2414396009 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84d4105790 0x7f84d419cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:30.307 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.275+0000 7f84d2ffd700 1 -- 172.21.15.39:0/2414396009 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84bc0095f0 con 0x7f84d4102e70 2023-07-21T18:41:30.307 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.275+0000 7f84d3fff700 1 --2- 172.21.15.39:0/2414396009 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84d4105790 0x7f84d419cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:41:30.307 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.275+0000 7f84d2ffd700 1 --2- 172.21.15.39:0/2414396009 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84d4102e70 0x7f84d4195c70 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f84c400c9b0 tx=0x7f84c400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:30.308 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.275+0000 7f84d0ff9700 1 -- 172.21.15.39:0/2414396009 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c4007b00 con 0x7f84d4102e70 2023-07-21T18:41:30.308 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.275+0000 7f84d0ff9700 1 -- 172.21.15.39:0/2414396009 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84c4007c60 con 0x7f84d4102e70 2023-07-21T18:41:30.308 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.275+0000 7f84da21d700 1 -- 172.21.15.39:0/2414396009 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84d419d860 con 0x7f84d4102e70 2023-07-21T18:41:30.308 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.276+0000 7f84d0ff9700 1 -- 172.21.15.39:0/2414396009 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c4018660 con 0x7f84d4102e70 2023-07-21T18:41:30.309 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.276+0000 7f84da21d700 1 -- 172.21.15.39:0/2414396009 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84d419dd70 con 0x7f84d4102e70 2023-07-21T18:41:30.309 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.277+0000 7f84da21d700 1 -- 172.21.15.39:0/2414396009 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84d40fbb40 con 0x7f84d4102e70 2023-07-21T18:41:30.310 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.278+0000 7f84d0ff9700 1 -- 172.21.15.39:0/2414396009 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f84c400f450 con 0x7f84d4102e70 2023-07-21T18:41:30.311 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.278+0000 7f84d0ff9700 1 --2- 172.21.15.39:0/2414396009 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f84c00a1220 0x7f84c00a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:30.311 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.279+0000 7f84d37fe700 1 --2- 172.21.15.39:0/2414396009 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f84c00a1220 0x7f84c00a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:30.311 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.279+0000 7f84d0ff9700 1 -- 172.21.15.39:0/2414396009 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f84c40ccd00 con 0x7f84d4102e70 2023-07-21T18:41:30.311 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.279+0000 7f84d37fe700 1 --2- 172.21.15.39:0/2414396009 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f84c00a1220 0x7f84c00a36d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f84bc00b5c0 tx=0x7f84bc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:30.312 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.282+0000 7f84d0ff9700 1 -- 172.21.15.39:0/2414396009 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f84c40963b0 con 0x7f84d4102e70 2023-07-21T18:41:30.438 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:30 smithi039 ceph-mon[96489]: pgmap v292: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 170 B/s wr, 11 op/s 2023-07-21T18:41:30.438 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:30 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/3935803527' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T18:41:30.439 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:30 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/649896149' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T18:41:30.506 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.483+0000 7f84da21d700 1 -- 172.21.15.39:0/2414396009 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f84d402caa0 con 0x7f84d4102e70 2023-07-21T18:41:30.507 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.484+0000 7f84d0ff9700 1 -- 172.21.15.39:0/2414396009 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+22447 (secure 0 0 0) 0x7f84c40999d0 con 0x7f84d4102e70 2023-07-21T18:41:30.507 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:41:30.508 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":81,"fsid":"eaf8e25a-27f4-11ee-9b34-001a4aab830c","created":"2023-07-21T18:33:06.504979+0000","modified":"2023-07-21T18:41:08.140284+0000","last_up_change":"2023-07-21T18:39:23.531009+0000","last_in_change":"2023-07-21T18:39:05.993024+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T18:36:13.572583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T18:40:02.726735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T18:40:03.990112+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03502c07-2d0f-4723-8d0b-d34c9f5b91bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6803","nonce":860193857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6805","nonce":860193857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6809","nonce":860193857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6807","nonce":860193857}]},"public_addr":"172.21.15.39:6803/860193857","cluster_addr":"172.21.15.39:6805/860193857","heartbeat_back_addr":"172.21.15.39:6809/860193857","heartbeat_front_addr":"172.21.15.39:6807/860193857","state":["exists","up"]},{"osd":1,"uuid":"0831a0c0-a1fa-4d30-9a28-513223aa69eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6801","nonce":3246721764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6803","nonce":3246721764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6807","nonce":3246721764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6805","nonce":3246721764}]},"public_addr":"172.21.15.155:6801/3246721764","cluster_addr":"172.21.15.155:6803/3246721764","heartbeat_back_addr":"172.21.15.155:6807/3246721764","heartbeat_front_addr":"172.21.15.155:6805/3246721764","state":["exists","up"]},{"osd":2,"uuid":"22df0003-bde9-48e4-b47a-cb8eafbab0db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6801","nonce":2563594449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6803","nonce":2563594449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6807","nonce":2563594449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6805","nonce":2563594449}]},"public_addr":"172.21.15.193:6801/2563594449","cluster_addr":"172.21.15.193:6803/2563594449","heartbeat_back_addr":"172.21.15.193:6807/2563594449","heartbeat_front_addr":"172.21.15.193:6805/2563594449","state":["exists","up"]},{"osd":3,"uuid":"54562558-a2e8-45dd-bc95-561569b9ae6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6811","nonce":863708386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6813","nonce":863708386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6817","nonce":863708386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6815","nonce":863708386}]},"public_addr":"172.21.15.39:6811/863708386","cluster_addr":"172.21.15.39:6813/863708386","heartbeat_back_addr":"172.21.15.39:6817/863708386","heartbeat_front_addr":"172.21.15.39:6815/863708386","state":["exists","up"]},{"osd":4,"uuid":"491118e7-ae1e-495c-9eac-ef1cdcae9600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6809","nonce":3361780673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6811","nonce":3361780673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6815","nonce":3361780673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6813","nonce":3361780673}]},"public_addr":"172.21.15.155:6809/3361780673","cluster_addr":"172.21.15.155:6811/3361780673","heartbeat_back_addr":"172.21.15.155:6815/3361780673","heartbeat_front_addr":"172.21.15.155:6813/3361780673","state":["exists","up"]},{"osd":5,"uuid":"d267bc2c-42b3-4d96-b2d1-b186dc7e6b18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6809","nonce":2376867721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6811","nonce":2376867721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6815","nonce":2376867721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6813","nonce":2376867721}]},"public_addr":"172.21.15.193:6809/2376867721","cluster_addr":"172.21.15.193:6811/2376867721","heartbeat_back_addr":"172.21.15.193:6815/2376867721","heartbeat_front_addr":"172.21.15.193:6813/2376867721","state":["exists","up"]},{"osd":6,"uuid":"beec8088-970a-473e-bd67-6eee7cbdef24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6819","nonce":2417770529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6821","nonce":2417770529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6825","nonce":2417770529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6823","nonce":2417770529}]},"public_addr":"172.21.15.39:6819/2417770529","cluster_addr":"172.21.15.39:6821/2417770529","heartbeat_back_addr":"172.21.15.39:6825/2417770529","heartbeat_front_addr":"172.21.15.39:6823/2417770529","state":["exists","up"]},{"osd":7,"uuid":"5a88ce30-1835-4254-90cc-804ed5cf1066","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6817","nonce":2542002111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6819","nonce":2542002111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6823","nonce":2542002111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6821","nonce":2542002111}]},"public_addr":"172.21.15.155:6817/2542002111","cluster_addr":"172.21.15.155:6819/2542002111","heartbeat_back_addr":"172.21.15.155:6823/2542002111","heartbeat_front_addr":"172.21.15.155:6821/2542002111","state":["exists","up"]},{"osd":8,"uuid":"d964a183-3021-4ace-9a0c-aad00f2570e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6817","nonce":2985835085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6819","nonce":2985835085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6823","nonce":2985835085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6821","nonce":2985835085}]},"public_addr":"172.21.15.193:6817/2985835085","cluster_addr":"172.21.15.193:6819/2985835085","heartbeat_back_addr":"172.21.15.193:6823/2985835085","heartbeat_front_addr":"172.21.15.193:6821/2985835085","state":["exists","up"]},{"osd":9,"uuid":"9725ea5c-49a3-4c99-bac4-e95069100b1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6827","nonce":1946493651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6829","nonce":1946493651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6833","nonce":1946493651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6831","nonce":1946493651}]},"public_addr":"172.21.15.39:6827/1946493651","cluster_addr":"172.21.15.39:6829/1946493651","heartbeat_back_addr":"172.21.15.39:6833/1946493651","heartbeat_front_addr":"172.21.15.39:6831/1946493651","state":["exists","up"]},{"osd":10,"uuid":"1fd60ccb-92db-45f0-a580-2003db0976e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6825","nonce":1234615022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6827","nonce":1234615022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6831","nonce":1234615022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6829","nonce":1234615022}]},"public_addr":"172.21.15.155:6825/1234615022","cluster_addr":"172.21.15.155:6827/1234615022","heartbeat_back_addr":"172.21.15.155:6831/1234615022","heartbeat_front_addr":"172.21.15.155:6829/1234615022","state":["exists","up"]},{"osd":11,"uuid":"efb31ea1-4775-4e5f-8740-728d6f4aeea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6825","nonce":215104663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6827","nonce":215104663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6831","nonce":215104663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6829","nonce":215104663}]},"public_addr":"172.21.15.193:6825/215104663","cluster_addr":"172.21.15.193:6827/215104663","heartbeat_back_addr":"172.21.15.193:6831/215104663","heartbeat_front_addr":"172.21.15.193:6829/215104663","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:35:30.211698+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:35:50.883846+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:36:11.124168+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:36:32.711905+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:36:54.442007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:37:14.843526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:37:36.587354+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:37:57.882374+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:38:18.729407+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:38:40.139231+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:39:01.290862+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:39:21.767537+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:0/2902600414":"2023-07-22T18:34:07.039823+0000","172.21.15.39:6801/2893866425":"2023-07-22T18:34:07.039823+0000","172.21.15.39:0/19717002":"2023-07-22T18:34:07.039823+0000","172.21.15.39:6800/2893866425":"2023-07-22T18:34:07.039823+0000","172.21.15.39:6800/889516631":"2023-07-22T18:33:40.852706+0000","172.21.15.39:6801/889516631":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/3636510379":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/1275748831":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/3819152602":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/1127767082":"2023-07-22T18:34:07.039823+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T18:41:30.509 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.487+0000 7f84da21d700 1 -- 172.21.15.39:0/2414396009 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f84c00a1220 msgr2=0x7f84c00a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:30.510 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.487+0000 7f84da21d700 1 --2- 172.21.15.39:0/2414396009 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f84c00a1220 0x7f84c00a36d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f84bc00b5c0 tx=0x7f84bc005fd0 comp rx=0 tx=0).stop 2023-07-21T18:41:30.510 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.487+0000 7f84da21d700 1 -- 172.21.15.39:0/2414396009 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84d4102e70 msgr2=0x7f84d4195c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:30.510 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.487+0000 7f84da21d700 1 --2- 172.21.15.39:0/2414396009 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84d4102e70 0x7f84d4195c70 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f84c400c9b0 tx=0x7f84c400ccc0 comp rx=0 tx=0).stop 2023-07-21T18:41:30.510 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.487+0000 7f84da21d700 1 -- 172.21.15.39:0/2414396009 shutdown_connections 2023-07-21T18:41:30.510 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.487+0000 7f84da21d700 1 --2- 172.21.15.39:0/2414396009 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f84c00a1220 0x7f84c00a36d0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:30.510 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.487+0000 7f84da21d700 1 --2- 172.21.15.39:0/2414396009 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84d4105790 0x7f84d419cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:30.510 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.487+0000 7f84da21d700 1 --2- 172.21.15.39:0/2414396009 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f84d4100550 0x7f84d4195730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:30.511 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.487+0000 7f84da21d700 1 --2- 172.21.15.39:0/2414396009 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84d4102e70 0x7f84d4195c70 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:30.511 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.487+0000 7f84da21d700 1 -- 172.21.15.39:0/2414396009 >> 172.21.15.39:0/2414396009 conn(0x7f84d40f9fd0 msgr2=0x7f84d41011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:30.511 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.487+0000 7f84da21d700 1 -- 172.21.15.39:0/2414396009 shutdown_connections 2023-07-21T18:41:30.511 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.487+0000 7f84da21d700 1 -- 172.21.15.39:0/2414396009 wait complete. 2023-07-21T18:41:30.522 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-21T18:36:13.572583+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-21T18:40:02.726735+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-21T18:40:03.990112+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-21T18:41:30.523 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-21T18:41:30.620 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:30 smithi193 ceph-mon[98204]: pgmap v292: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 170 B/s wr, 11 op/s 2023-07-21T18:41:30.621 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:30 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/3935803527' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T18:41:30.621 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:30 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/649896149' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T18:41:30.639 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:30 smithi155 ceph-mon[97934]: pgmap v292: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 170 B/s wr, 11 op/s 2023-07-21T18:41:30.640 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:30 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/3935803527' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T18:41:30.640 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:30 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/649896149' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T18:41:30.653 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.630+0000 7f3275f64700 1 -- 172.21.15.39:0/2395136545 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f32700ffbd0 msgr2=0x7f3270100020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:30.653 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.630+0000 7f3275f64700 1 --2- 172.21.15.39:0/2395136545 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f32700ffbd0 0x7f3270100020 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f3258009990 tx=0x7f3258009ca0 comp rx=0 tx=0).stop 2023-07-21T18:41:30.653 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.630+0000 7f3275f64700 1 -- 172.21.15.39:0/2395136545 shutdown_connections 2023-07-21T18:41:30.653 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.630+0000 7f3275f64700 1 --2- 172.21.15.39:0/2395136545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3270100560 0x7f3270106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:30.653 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.630+0000 7f3275f64700 1 --2- 172.21.15.39:0/2395136545 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32700fe9e0 0x7f32700fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:30.654 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.630+0000 7f3275f64700 1 --2- 172.21.15.39:0/2395136545 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f32700ffbd0 0x7f3270100020 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:30.654 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.630+0000 7f3275f64700 1 -- 172.21.15.39:0/2395136545 >> 172.21.15.39:0/2395136545 conn(0x7f32700fa000 msgr2=0x7f32700fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:30.654 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.630+0000 7f3275f64700 1 -- 172.21.15.39:0/2395136545 shutdown_connections 2023-07-21T18:41:30.654 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.630+0000 7f3275f64700 1 -- 172.21.15.39:0/2395136545 wait complete. 2023-07-21T18:41:30.654 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.630+0000 7f3275f64700 1 Processor -- start 2023-07-21T18:41:30.655 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.631+0000 7f3275f64700 1 -- start start 2023-07-21T18:41:30.656 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.631+0000 7f3275f64700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f32700fe9e0 0x7f3270195990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:30.656 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.631+0000 7f3275f64700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32700ffbd0 0x7f3270195ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:30.656 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.631+0000 7f3275f64700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3270100560 0x7f327019cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:30.656 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.631+0000 7f3275f64700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f327019d510 con 0x7f3270100560 2023-07-21T18:41:30.656 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.631+0000 7f3275f64700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f327019d650 con 0x7f32700fe9e0 2023-07-21T18:41:30.656 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.631+0000 7f3275f64700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f327019d7e0 con 0x7f32700ffbd0 2023-07-21T18:41:30.657 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.631+0000 7f326ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3270100560 0x7f327019cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:30.657 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.631+0000 7f326ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3270100560 0x7f327019cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56632/0 (socket says 172.21.15.39:56632) 2023-07-21T18:41:30.657 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.631+0000 7f326ffff700 1 -- 172.21.15.39:0/1816419852 learned_addr learned my addr 172.21.15.39:0/1816419852 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:30.657 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.631+0000 7f326f7fe700 1 --2- 172.21.15.39:0/1816419852 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f32700fe9e0 0x7f3270195990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:30.658 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.631+0000 7f326effd700 1 --2- 172.21.15.39:0/1816419852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32700ffbd0 0x7f3270195ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:30.658 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.631+0000 7f326ffff700 1 -- 172.21.15.39:0/1816419852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32700ffbd0 msgr2=0x7f3270195ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:30.658 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.631+0000 7f326ffff700 1 --2- 172.21.15.39:0/1816419852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32700ffbd0 0x7f3270195ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:30.658 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.631+0000 7f326ffff700 1 -- 172.21.15.39:0/1816419852 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f32700fe9e0 msgr2=0x7f3270195990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:30.658 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.631+0000 7f326ffff700 1 --2- 172.21.15.39:0/1816419852 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f32700fe9e0 0x7f3270195990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:30.658 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.631+0000 7f326ffff700 1 -- 172.21.15.39:0/1816419852 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32580095f0 con 0x7f3270100560 2023-07-21T18:41:30.658 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.631+0000 7f326ffff700 1 --2- 172.21.15.39:0/1816419852 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3270100560 0x7f327019cf40 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f3268007ee0 tx=0x7f326800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:30.659 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.631+0000 7f326cff9700 1 -- 172.21.15.39:0/1816419852 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3268017070 con 0x7f3270100560 2023-07-21T18:41:30.659 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.632+0000 7f326cff9700 1 -- 172.21.15.39:0/1816419852 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f326800f040 con 0x7f3270100560 2023-07-21T18:41:30.659 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.632+0000 7f3275f64700 1 -- 172.21.15.39:0/1816419852 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f327019dac0 con 0x7f3270100560 2023-07-21T18:41:30.659 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.632+0000 7f326cff9700 1 -- 172.21.15.39:0/1816419852 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3268005740 con 0x7f3270100560 2023-07-21T18:41:30.659 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.632+0000 7f3275f64700 1 -- 172.21.15.39:0/1816419852 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f327019df80 con 0x7f3270100560 2023-07-21T18:41:30.659 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.632+0000 7f3275f64700 1 -- 172.21.15.39:0/1816419852 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f327010a2a0 con 0x7f3270100560 2023-07-21T18:41:30.660 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.634+0000 7f326cff9700 1 -- 172.21.15.39:0/1816419852 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f326800dc60 con 0x7f3270100560 2023-07-21T18:41:30.660 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.634+0000 7f326cff9700 1 --2- 172.21.15.39:0/1816419852 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f325c0a11e0 0x7f325c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:30.660 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.634+0000 7f326f7fe700 1 --2- 172.21.15.39:0/1816419852 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f325c0a11e0 0x7f325c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:30.660 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.634+0000 7f326cff9700 1 -- 172.21.15.39:0/1816419852 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f32680cc2b0 con 0x7f3270100560 2023-07-21T18:41:30.660 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.635+0000 7f326f7fe700 1 --2- 172.21.15.39:0/1816419852 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f325c0a11e0 0x7f325c0a3690 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f32700ffa30 tx=0x7f3260009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:30.661 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.639+0000 7f326cff9700 1 -- 172.21.15.39:0/1816419852 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f32680958d0 con 0x7f3270100560 2023-07-21T18:41:30.856 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.833+0000 7f3275f64700 1 -- 172.21.15.39:0/1816419852 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f327019e530 con 0x7f3270100560 2023-07-21T18:41:30.857 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.834+0000 7f326cff9700 1 -- 172.21.15.39:0/1816419852 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3268098ef0 con 0x7f3270100560 2023-07-21T18:41:30.857 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2023-07-21T18:41:30.859 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.836+0000 7f3275f64700 1 -- 172.21.15.39:0/1816419852 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f325c0a11e0 msgr2=0x7f325c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:30.859 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.836+0000 7f3275f64700 1 --2- 172.21.15.39:0/1816419852 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f325c0a11e0 0x7f325c0a3690 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f32700ffa30 tx=0x7f3260009190 comp rx=0 tx=0).stop 2023-07-21T18:41:30.859 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.836+0000 7f3275f64700 1 -- 172.21.15.39:0/1816419852 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3270100560 msgr2=0x7f327019cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:30.859 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.836+0000 7f3275f64700 1 --2- 172.21.15.39:0/1816419852 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3270100560 0x7f327019cf40 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f3268007ee0 tx=0x7f326800d3b0 comp rx=0 tx=0).stop 2023-07-21T18:41:30.859 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.836+0000 7f3275f64700 1 -- 172.21.15.39:0/1816419852 shutdown_connections 2023-07-21T18:41:30.859 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.836+0000 7f3275f64700 1 --2- 172.21.15.39:0/1816419852 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f325c0a11e0 0x7f325c0a3690 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:30.860 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.836+0000 7f3275f64700 1 --2- 172.21.15.39:0/1816419852 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f32700fe9e0 0x7f3270195990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:30.860 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.836+0000 7f3275f64700 1 --2- 172.21.15.39:0/1816419852 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3270100560 0x7f327019cf40 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:30.860 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.836+0000 7f3275f64700 1 --2- 172.21.15.39:0/1816419852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32700ffbd0 0x7f3270195ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:30.860 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.836+0000 7f3275f64700 1 -- 172.21.15.39:0/1816419852 >> 172.21.15.39:0/1816419852 conn(0x7f32700fa000 msgr2=0x7f3270067670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:30.860 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.836+0000 7f3275f64700 1 -- 172.21.15.39:0/1816419852 shutdown_connections 2023-07-21T18:41:30.860 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.836+0000 7f3275f64700 1 -- 172.21.15.39:0/1816419852 wait complete. 2023-07-21T18:41:30.871 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-21T18:41:31.001 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.978+0000 7fa39896d700 1 -- 172.21.15.39:0/1054512093 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa394100560 msgr2=0x7fa394102940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:31.001 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.978+0000 7fa39896d700 1 --2- 172.21.15.39:0/1054512093 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa394100560 0x7fa394102940 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fa384009990 tx=0x7fa384009ca0 comp rx=0 tx=0).stop 2023-07-21T18:41:31.001 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.978+0000 7fa39896d700 1 -- 172.21.15.39:0/1054512093 shutdown_connections 2023-07-21T18:41:31.001 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.978+0000 7fa39896d700 1 --2- 172.21.15.39:0/1054512093 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3941057a0 0x7fa394107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:31.001 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.978+0000 7fa39896d700 1 --2- 172.21.15.39:0/1054512093 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa394100560 0x7fa394102940 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:31.001 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.978+0000 7fa39896d700 1 --2- 172.21.15.39:0/1054512093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa394102e80 0x7fa394105260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:31.002 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.978+0000 7fa39896d700 1 -- 172.21.15.39:0/1054512093 >> 172.21.15.39:0/1054512093 conn(0x7fa3940fa000 msgr2=0x7fa3940fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:31.002 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.978+0000 7fa39896d700 1 -- 172.21.15.39:0/1054512093 shutdown_connections 2023-07-21T18:41:31.002 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.978+0000 7fa39896d700 1 -- 172.21.15.39:0/1054512093 wait complete. 2023-07-21T18:41:31.003 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.978+0000 7fa39896d700 1 Processor -- start 2023-07-21T18:41:31.004 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.979+0000 7fa39896d700 1 -- start start 2023-07-21T18:41:31.004 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.979+0000 7fa39896d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa394100560 0x7fa3941957c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:31.004 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.979+0000 7fa39896d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa394102e80 0x7fa394195d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:31.004 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.979+0000 7fa39896d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3941057a0 0x7fa39419cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:31.004 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.979+0000 7fa39896d700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39419d340 con 0x7fa3941057a0 2023-07-21T18:41:31.005 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.979+0000 7fa39896d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39419d480 con 0x7fa394100560 2023-07-21T18:41:31.005 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.979+0000 7fa39896d700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39419d610 con 0x7fa394102e80 2023-07-21T18:41:31.005 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.979+0000 7fa392d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3941057a0 0x7fa39419cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:31.005 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.979+0000 7fa392d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3941057a0 0x7fa39419cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56646/0 (socket says 172.21.15.39:56646) 2023-07-21T18:41:31.005 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.979+0000 7fa392d9d700 1 -- 172.21.15.39:0/453869244 learned_addr learned my addr 172.21.15.39:0/453869244 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:31.006 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.979+0000 7fa39259c700 1 --2- 172.21.15.39:0/453869244 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa394100560 0x7fa3941957c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:31.006 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.979+0000 7fa38bfff700 1 --2- 172.21.15.39:0/453869244 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa394102e80 0x7fa394195d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:31.006 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.979+0000 7fa392d9d700 1 -- 172.21.15.39:0/453869244 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa394102e80 msgr2=0x7fa394195d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:31.006 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.979+0000 7fa392d9d700 1 --2- 172.21.15.39:0/453869244 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa394102e80 0x7fa394195d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:31.006 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.979+0000 7fa392d9d700 1 -- 172.21.15.39:0/453869244 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa394100560 msgr2=0x7fa3941957c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:31.006 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.979+0000 7fa392d9d700 1 --2- 172.21.15.39:0/453869244 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa394100560 0x7fa3941957c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:31.006 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.979+0000 7fa392d9d700 1 -- 172.21.15.39:0/453869244 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3840095f0 con 0x7fa3941057a0 2023-07-21T18:41:31.007 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.979+0000 7fa392d9d700 1 --2- 172.21.15.39:0/453869244 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3941057a0 0x7fa39419cd70 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa38c007ee0 tx=0x7fa38c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:31.007 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.980+0000 7fa38b7fe700 1 -- 172.21.15.39:0/453869244 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38c017070 con 0x7fa3941057a0 2023-07-21T18:41:31.007 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.980+0000 7fa38b7fe700 1 -- 172.21.15.39:0/453869244 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa38c00f040 con 0x7fa3941057a0 2023-07-21T18:41:31.007 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.980+0000 7fa39896d700 1 -- 172.21.15.39:0/453869244 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa39419d8f0 con 0x7fa3941057a0 2023-07-21T18:41:31.007 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.980+0000 7fa38b7fe700 1 -- 172.21.15.39:0/453869244 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38c005740 con 0x7fa3941057a0 2023-07-21T18:41:31.007 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.980+0000 7fa39896d700 1 -- 172.21.15.39:0/453869244 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa39419ddb0 con 0x7fa3941057a0 2023-07-21T18:41:31.007 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.981+0000 7fa39896d700 1 -- 172.21.15.39:0/453869244 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3940fbb70 con 0x7fa3941057a0 2023-07-21T18:41:31.008 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.982+0000 7fa38b7fe700 1 -- 172.21.15.39:0/453869244 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fa38c00dc60 con 0x7fa3941057a0 2023-07-21T18:41:31.008 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.983+0000 7fa38b7fe700 1 --2- 172.21.15.39:0/453869244 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fa3800a11e0 0x7fa3800a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:31.009 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.983+0000 7fa38b7fe700 1 -- 172.21.15.39:0/453869244 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fa38c0cc8c0 con 0x7fa3941057a0 2023-07-21T18:41:31.009 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.983+0000 7fa39259c700 1 --2- 172.21.15.39:0/453869244 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fa3800a11e0 0x7fa3800a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:31.009 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.983+0000 7fa39259c700 1 --2- 172.21.15.39:0/453869244 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fa3800a11e0 0x7fa3800a3690 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa38400b5c0 tx=0x7fa384005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:31.009 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:30.986+0000 7fa38b7fe700 1 -- 172.21.15.39:0/453869244 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fa38c095ee0 con 0x7fa3941057a0 2023-07-21T18:41:31.208 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.185+0000 7fa39896d700 1 -- 172.21.15.39:0/453869244 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa39402caa0 con 0x7fa3941057a0 2023-07-21T18:41:31.209 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.186+0000 7fa38b7fe700 1 -- 172.21.15.39:0/453869244 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v81) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa38c099500 con 0x7fa3941057a0 2023-07-21T18:41:31.209 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2023-07-21T18:41:31.210 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.188+0000 7fa39896d700 1 -- 172.21.15.39:0/453869244 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fa3800a11e0 msgr2=0x7fa3800a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:31.211 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.188+0000 7fa39896d700 1 --2- 172.21.15.39:0/453869244 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fa3800a11e0 0x7fa3800a3690 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa38400b5c0 tx=0x7fa384005fd0 comp rx=0 tx=0).stop 2023-07-21T18:41:31.211 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.188+0000 7fa39896d700 1 -- 172.21.15.39:0/453869244 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3941057a0 msgr2=0x7fa39419cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:31.211 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.188+0000 7fa39896d700 1 --2- 172.21.15.39:0/453869244 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3941057a0 0x7fa39419cd70 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa38c007ee0 tx=0x7fa38c00d3b0 comp rx=0 tx=0).stop 2023-07-21T18:41:31.211 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.188+0000 7fa39896d700 1 -- 172.21.15.39:0/453869244 shutdown_connections 2023-07-21T18:41:31.211 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.188+0000 7fa39896d700 1 --2- 172.21.15.39:0/453869244 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fa3800a11e0 0x7fa3800a3690 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:31.211 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.188+0000 7fa39896d700 1 --2- 172.21.15.39:0/453869244 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa394100560 0x7fa3941957c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:31.211 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.188+0000 7fa39896d700 1 --2- 172.21.15.39:0/453869244 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3941057a0 0x7fa39419cd70 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:31.212 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.188+0000 7fa39896d700 1 --2- 172.21.15.39:0/453869244 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa394102e80 0x7fa394195d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:31.212 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.188+0000 7fa39896d700 1 -- 172.21.15.39:0/453869244 >> 172.21.15.39:0/453869244 conn(0x7fa3940fa000 msgr2=0x7fa3941011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:31.212 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.188+0000 7fa39896d700 1 -- 172.21.15.39:0/453869244 shutdown_connections 2023-07-21T18:41:31.212 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.188+0000 7fa39896d700 1 -- 172.21.15.39:0/453869244 wait complete. 2023-07-21T18:41:31.223 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-21T18:41:31.352 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.329+0000 7f21c4058700 1 -- 172.21.15.39:0/3539505098 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21bc0ffc10 msgr2=0x7f21bc100060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:31.353 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.329+0000 7f21c4058700 1 --2- 172.21.15.39:0/3539505098 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21bc0ffc10 0x7f21bc100060 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f21ac009990 tx=0x7f21ac009ca0 comp rx=0 tx=0).stop 2023-07-21T18:41:31.353 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.329+0000 7f21c4058700 1 -- 172.21.15.39:0/3539505098 shutdown_connections 2023-07-21T18:41:31.353 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.329+0000 7f21c4058700 1 --2- 172.21.15.39:0/3539505098 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21bc0fea20 0x7f21bc0fee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:31.353 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.329+0000 7f21c4058700 1 --2- 172.21.15.39:0/3539505098 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21bc1005a0 0x7f21bc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:31.353 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.329+0000 7f21c4058700 1 --2- 172.21.15.39:0/3539505098 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21bc0ffc10 0x7f21bc100060 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:31.353 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.329+0000 7f21c4058700 1 -- 172.21.15.39:0/3539505098 >> 172.21.15.39:0/3539505098 conn(0x7f21bc0fa020 msgr2=0x7f21bc0fc470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:31.353 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.329+0000 7f21c4058700 1 -- 172.21.15.39:0/3539505098 shutdown_connections 2023-07-21T18:41:31.354 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.329+0000 7f21c4058700 1 -- 172.21.15.39:0/3539505098 wait complete. 2023-07-21T18:41:31.355 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.330+0000 7f21c4058700 1 Processor -- start 2023-07-21T18:41:31.355 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.330+0000 7f21c4058700 1 -- start start 2023-07-21T18:41:31.356 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.330+0000 7f21c4058700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21bc0fea20 0x7f21bc195800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:31.356 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.330+0000 7f21c4058700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21bc0ffc10 0x7f21bc195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:31.356 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.330+0000 7f21c4058700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21bc1005a0 0x7f21bc19cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:31.356 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.330+0000 7f21c4058700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21bc19d380 con 0x7f21bc0fea20 2023-07-21T18:41:31.356 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.330+0000 7f21c4058700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21bc19d4c0 con 0x7f21bc0ffc10 2023-07-21T18:41:31.356 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.330+0000 7f21c4058700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21bc19d650 con 0x7f21bc1005a0 2023-07-21T18:41:31.356 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.330+0000 7f21c1df4700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21bc0fea20 0x7f21bc195800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:31.357 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.330+0000 7f21c1df4700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21bc0fea20 0x7f21bc195800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56656/0 (socket says 172.21.15.39:56656) 2023-07-21T18:41:31.357 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.330+0000 7f21c1df4700 1 -- 172.21.15.39:0/1494681701 learned_addr learned my addr 172.21.15.39:0/1494681701 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:31.357 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.331+0000 7f21c15f3700 1 --2- 172.21.15.39:0/1494681701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21bc0ffc10 0x7f21bc195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:31.357 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.331+0000 7f21c1df4700 1 -- 172.21.15.39:0/1494681701 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21bc1005a0 msgr2=0x7f21bc19cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:31.357 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.331+0000 7f21c1df4700 1 --2- 172.21.15.39:0/1494681701 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21bc1005a0 0x7f21bc19cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:31.357 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.331+0000 7f21c1df4700 1 -- 172.21.15.39:0/1494681701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21bc0ffc10 msgr2=0x7f21bc195d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:31.357 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.331+0000 7f21c1df4700 1 --2- 172.21.15.39:0/1494681701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21bc0ffc10 0x7f21bc195d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:31.357 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.331+0000 7f21c1df4700 1 -- 172.21.15.39:0/1494681701 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21ac0095f0 con 0x7f21bc0fea20 2023-07-21T18:41:31.358 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.331+0000 7f21c15f3700 1 --2- 172.21.15.39:0/1494681701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21bc0ffc10 0x7f21bc195d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:41:31.358 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.331+0000 7f21c1df4700 1 --2- 172.21.15.39:0/1494681701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21bc0fea20 0x7f21bc195800 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f21b800c990 tx=0x7f21b800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:31.358 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.331+0000 7f21b2ffd700 1 -- 172.21.15.39:0/1494681701 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b8007b00 con 0x7f21bc0fea20 2023-07-21T18:41:31.358 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.331+0000 7f21b2ffd700 1 -- 172.21.15.39:0/1494681701 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f21b8007c60 con 0x7f21bc0fea20 2023-07-21T18:41:31.358 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.331+0000 7f21c4058700 1 -- 172.21.15.39:0/1494681701 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21bc19d930 con 0x7f21bc0fea20 2023-07-21T18:41:31.358 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.331+0000 7f21b2ffd700 1 -- 172.21.15.39:0/1494681701 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b8018660 con 0x7f21bc0fea20 2023-07-21T18:41:31.358 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.331+0000 7f21c4058700 1 -- 172.21.15.39:0/1494681701 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21bc19de90 con 0x7f21bc0fea20 2023-07-21T18:41:31.358 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.332+0000 7f21c4058700 1 -- 172.21.15.39:0/1494681701 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21bc10a1f0 con 0x7f21bc0fea20 2023-07-21T18:41:31.359 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.333+0000 7f21b2ffd700 1 -- 172.21.15.39:0/1494681701 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f21b8004020 con 0x7f21bc0fea20 2023-07-21T18:41:31.359 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.334+0000 7f21b2ffd700 1 --2- 172.21.15.39:0/1494681701 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f21a80a1260 0x7f21a80a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:31.359 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.334+0000 7f21b2ffd700 1 -- 172.21.15.39:0/1494681701 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f21b80ccb60 con 0x7f21bc0fea20 2023-07-21T18:41:31.360 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.334+0000 7f21c15f3700 1 --2- 172.21.15.39:0/1494681701 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f21a80a1260 0x7f21a80a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:31.360 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.334+0000 7f21c15f3700 1 --2- 172.21.15.39:0/1494681701 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f21a80a1260 0x7f21a80a3710 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f21ac006010 tx=0x7f21ac00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:31.360 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.337+0000 7f21b2ffd700 1 -- 172.21.15.39:0/1494681701 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f21b8096290 con 0x7f21bc0fea20 2023-07-21T18:41:31.438 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:31 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/1822928010' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T18:41:31.438 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:31 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/2414396009' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T18:41:31.438 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:31 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/1816419852' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T18:41:31.561 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.539+0000 7f21c4058700 1 -- 172.21.15.39:0/1494681701 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f21bc19e500 con 0x7f21bc0fea20 2023-07-21T18:41:31.562 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.539+0000 7f21b2ffd700 1 -- 172.21.15.39:0/1494681701 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v81) v1 ==== 100+0+11 (secure 0 0 0) 0x7f21b80998b0 con 0x7f21bc0fea20 2023-07-21T18:41:31.562 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 64 2023-07-21T18:41:31.564 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.541+0000 7f21c4058700 1 -- 172.21.15.39:0/1494681701 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f21a80a1260 msgr2=0x7f21a80a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:31.564 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.541+0000 7f21c4058700 1 --2- 172.21.15.39:0/1494681701 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f21a80a1260 0x7f21a80a3710 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f21ac006010 tx=0x7f21ac00b540 comp rx=0 tx=0).stop 2023-07-21T18:41:31.565 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.541+0000 7f21c4058700 1 -- 172.21.15.39:0/1494681701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21bc0fea20 msgr2=0x7f21bc195800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:31.565 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.541+0000 7f21c4058700 1 --2- 172.21.15.39:0/1494681701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21bc0fea20 0x7f21bc195800 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f21b800c990 tx=0x7f21b800cd50 comp rx=0 tx=0).stop 2023-07-21T18:41:31.565 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.541+0000 7f21c4058700 1 -- 172.21.15.39:0/1494681701 shutdown_connections 2023-07-21T18:41:31.565 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.541+0000 7f21c4058700 1 --2- 172.21.15.39:0/1494681701 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f21a80a1260 0x7f21a80a3710 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:31.565 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.541+0000 7f21c4058700 1 --2- 172.21.15.39:0/1494681701 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21bc1005a0 0x7f21bc19cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:31.565 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.541+0000 7f21c4058700 1 --2- 172.21.15.39:0/1494681701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21bc0fea20 0x7f21bc195800 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:31.565 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.541+0000 7f21c4058700 1 --2- 172.21.15.39:0/1494681701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21bc0ffc10 0x7f21bc195d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:31.565 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.541+0000 7f21c4058700 1 -- 172.21.15.39:0/1494681701 >> 172.21.15.39:0/1494681701 conn(0x7f21bc0fa020 msgr2=0x7f21bc0685c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:31.566 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.542+0000 7f21c4058700 1 -- 172.21.15.39:0/1494681701 shutdown_connections 2023-07-21T18:41:31.566 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.542+0000 7f21c4058700 1 -- 172.21.15.39:0/1494681701 wait complete. 2023-07-21T18:41:31.577 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2023-07-21T18:41:31.577 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T18:41:31.621 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:31 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/1822928010' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T18:41:31.621 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:31 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/2414396009' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T18:41:31.621 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:31 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/1816419852' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T18:41:31.639 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:31 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/1822928010' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T18:41:31.640 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:31 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/2414396009' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T18:41:31.640 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:31 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/1816419852' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T18:41:31.705 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.681+0000 7fb107242700 1 -- 172.21.15.39:0/278064433 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1000fe150 msgr2=0x7fb1000fe560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:31.706 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.681+0000 7fb107242700 1 --2- 172.21.15.39:0/278064433 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1000fe150 0x7fb1000fe560 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4009990 tx=0x7fb0f4009ca0 comp rx=0 tx=0).stop 2023-07-21T18:41:31.706 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.681+0000 7fb107242700 1 -- 172.21.15.39:0/278064433 shutdown_connections 2023-07-21T18:41:31.706 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.681+0000 7fb107242700 1 --2- 172.21.15.39:0/278064433 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb1000ffcd0 0x7fb100106590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:31.706 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.681+0000 7fb107242700 1 --2- 172.21.15.39:0/278064433 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1000fe150 0x7fb1000fe560 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:31.706 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.681+0000 7fb107242700 1 --2- 172.21.15.39:0/278064433 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1000ff340 0x7fb1000ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:31.706 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.681+0000 7fb107242700 1 -- 172.21.15.39:0/278064433 >> 172.21.15.39:0/278064433 conn(0x7fb1000f9770 msgr2=0x7fb1000fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:31.707 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.681+0000 7fb107242700 1 -- 172.21.15.39:0/278064433 shutdown_connections 2023-07-21T18:41:31.707 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.681+0000 7fb107242700 1 -- 172.21.15.39:0/278064433 wait complete. 2023-07-21T18:41:31.707 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.682+0000 7fb107242700 1 Processor -- start 2023-07-21T18:41:31.707 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.682+0000 7fb107242700 1 -- start start 2023-07-21T18:41:31.707 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.682+0000 7fb107242700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1000fe150 0x7fb1001957a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:31.707 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.682+0000 7fb107242700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb1000ff340 0x7fb100195ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:31.708 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.682+0000 7fb107242700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1000ffcd0 0x7fb10019cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:31.708 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.682+0000 7fb107242700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10019d320 con 0x7fb1000fe150 2023-07-21T18:41:31.708 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.682+0000 7fb107242700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10019d460 con 0x7fb1000ffcd0 2023-07-21T18:41:31.708 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.682+0000 7fb107242700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10019d5f0 con 0x7fb1000ff340 2023-07-21T18:41:31.708 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.682+0000 7fb104fde700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1000fe150 0x7fb1001957a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:31.708 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.682+0000 7fb104fde700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1000fe150 0x7fb1001957a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56670/0 (socket says 172.21.15.39:56670) 2023-07-21T18:41:31.708 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.682+0000 7fb104fde700 1 -- 172.21.15.39:0/2001220702 learned_addr learned my addr 172.21.15.39:0/2001220702 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:31.709 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.682+0000 7fb0fffff700 1 --2- 172.21.15.39:0/2001220702 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb1000ff340 0x7fb100195ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:31.709 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.682+0000 7fb1057df700 1 --2- 172.21.15.39:0/2001220702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1000ffcd0 0x7fb10019cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:31.709 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.682+0000 7fb104fde700 1 -- 172.21.15.39:0/2001220702 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb1000ff340 msgr2=0x7fb100195ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:31.709 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.682+0000 7fb104fde700 1 --2- 172.21.15.39:0/2001220702 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb1000ff340 0x7fb100195ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:31.709 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.682+0000 7fb104fde700 1 -- 172.21.15.39:0/2001220702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1000ffcd0 msgr2=0x7fb10019cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:31.709 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.682+0000 7fb104fde700 1 --2- 172.21.15.39:0/2001220702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1000ffcd0 0x7fb10019cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:31.709 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.682+0000 7fb104fde700 1 -- 172.21.15.39:0/2001220702 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f40095f0 con 0x7fb1000fe150 2023-07-21T18:41:31.709 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.682+0000 7fb104fde700 1 --2- 172.21.15.39:0/2001220702 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1000fe150 0x7fb1001957a0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4000c00 tx=0x7fb0f400f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:31.710 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.683+0000 7fb0fdffb700 1 -- 172.21.15.39:0/2001220702 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f401d070 con 0x7fb1000fe150 2023-07-21T18:41:31.710 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.683+0000 7fb107242700 1 -- 172.21.15.39:0/2001220702 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb10019d870 con 0x7fb1000fe150 2023-07-21T18:41:31.710 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.683+0000 7fb0fdffb700 1 -- 172.21.15.39:0/2001220702 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0f400fc60 con 0x7fb1000fe150 2023-07-21T18:41:31.710 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.683+0000 7fb0fdffb700 1 -- 172.21.15.39:0/2001220702 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f4017650 con 0x7fb1000fe150 2023-07-21T18:41:31.710 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.683+0000 7fb107242700 1 -- 172.21.15.39:0/2001220702 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb10019dcf0 con 0x7fb1000fe150 2023-07-21T18:41:31.711 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.684+0000 7fb107242700 1 -- 172.21.15.39:0/2001220702 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb10010a1d0 con 0x7fb1000fe150 2023-07-21T18:41:31.711 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.685+0000 7fb0fdffb700 1 -- 172.21.15.39:0/2001220702 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fb0f40177b0 con 0x7fb1000fe150 2023-07-21T18:41:31.711 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.685+0000 7fb0fdffb700 1 --2- 172.21.15.39:0/2001220702 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fb0f00a1220 0x7fb0f00a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:31.711 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.686+0000 7fb0fdffb700 1 -- 172.21.15.39:0/2001220702 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fb0f40ce3b0 con 0x7fb1000fe150 2023-07-21T18:41:31.712 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.686+0000 7fb0fffff700 1 --2- 172.21.15.39:0/2001220702 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fb0f00a1220 0x7fb0f00a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:31.712 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.686+0000 7fb0fffff700 1 --2- 172.21.15.39:0/2001220702 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fb0f00a1220 0x7fb0f00a36d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec009b30 tx=0x7fb0ec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:31.712 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.689+0000 7fb0fdffb700 1 -- 172.21.15.39:0/2001220702 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fb0f4097b20 con 0x7fb1000fe150 2023-07-21T18:41:31.959 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.937+0000 7fb107242700 1 -- 172.21.15.39:0/2001220702 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb10010a3c0 con 0x7fb1000fe150 2023-07-21T18:41:31.960 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.937+0000 7fb0fdffb700 1 -- 172.21.15.39:0/2001220702 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10887 (secure 0 0 0) 0x7fb0f409b140 con 0x7fb1000fe150 2023-07-21T18:41:31.961 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:41:31.961 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/147318681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":147318681},{"type":"v1","addr":"172.21.15.39:6835","nonce":147318681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/1062891760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1062891760},{"type":"v1","addr":"172.21.15.39:6837","nonce":1062891760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T18:40:04.594553+0000","modified":"2023-07-21T18:40:11.989039+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24536,"mds_1":24500,"mds_2":24487,"mds_3":24464,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/3184273349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":3184273349},{"type":"v1","addr":"172.21.15.39:6839","nonce":3184273349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6841/640981220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":640981220},{"type":"v1","addr":"172.21.15.39:6841","nonce":640981220}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3106166804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3106166804},{"type":"v1","addr":"172.21.15.193:6833","nonce":3106166804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24437":{"gid":24437,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/1034755277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1034755277},{"type":"v1","addr":"172.21.15.155:6833","nonce":1034755277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/381560657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":381560657},{"type":"v1","addr":"172.21.15.193:6835","nonce":381560657}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6837/2626761843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2626761843},{"type":"v1","addr":"172.21.15.193:6837","nonce":2626761843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24464":{"gid":24464,"name":"e","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.155:6835/3854856286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3854856286},{"type":"v1","addr":"172.21.15.155:6835","nonce":3854856286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.193:6839/3148488639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3148488639},{"type":"v1","addr":"172.21.15.193:6839","nonce":3148488639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.155:6837/947670056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":947670056},{"type":"v1","addr":"172.21.15.155:6837","nonce":947670056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6839/431690342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":431690342},{"type":"v1","addr":"172.21.15.155:6839","nonce":431690342}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T18:41:31.962 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.940+0000 7fb107242700 1 -- 172.21.15.39:0/2001220702 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fb0f00a1220 msgr2=0x7fb0f00a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:31.963 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.940+0000 7fb107242700 1 --2- 172.21.15.39:0/2001220702 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fb0f00a1220 0x7fb0f00a36d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec009b30 tx=0x7fb0ec009260 comp rx=0 tx=0).stop 2023-07-21T18:41:31.963 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.940+0000 7fb107242700 1 -- 172.21.15.39:0/2001220702 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1000fe150 msgr2=0x7fb1001957a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:31.963 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.940+0000 7fb107242700 1 --2- 172.21.15.39:0/2001220702 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1000fe150 0x7fb1001957a0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4000c00 tx=0x7fb0f400f510 comp rx=0 tx=0).stop 2023-07-21T18:41:31.963 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.940+0000 7fb107242700 1 -- 172.21.15.39:0/2001220702 shutdown_connections 2023-07-21T18:41:31.963 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.940+0000 7fb107242700 1 --2- 172.21.15.39:0/2001220702 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fb0f00a1220 0x7fb0f00a36d0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:31.963 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.940+0000 7fb107242700 1 --2- 172.21.15.39:0/2001220702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1000ffcd0 0x7fb10019cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:31.964 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.940+0000 7fb107242700 1 --2- 172.21.15.39:0/2001220702 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1000fe150 0x7fb1001957a0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:31.964 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.940+0000 7fb107242700 1 --2- 172.21.15.39:0/2001220702 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb1000ff340 0x7fb100195ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:31.964 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.940+0000 7fb107242700 1 -- 172.21.15.39:0/2001220702 >> 172.21.15.39:0/2001220702 conn(0x7fb1000f9770 msgr2=0x7fb100108540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:31.964 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.940+0000 7fb107242700 1 -- 172.21.15.39:0/2001220702 shutdown_connections 2023-07-21T18:41:31.964 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:31.940+0000 7fb107242700 1 -- 172.21.15.39:0/2001220702 wait complete. 2023-07-21T18:41:31.964 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 26 2023-07-21T18:41:31.975 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2023-07-21T18:41:31.975 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2023-07-21T18:41:31.976 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-21T18:41:32.102 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.079+0000 7fa9b65eb700 1 -- 172.21.15.39:0/3009967472 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa9b0100550 msgr2=0x7fa9b0106e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:32.103 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.079+0000 7fa9b65eb700 1 --2- 172.21.15.39:0/3009967472 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa9b0100550 0x7fa9b0106e10 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fa9a800b0a0 tx=0x7fa9a800b3b0 comp rx=0 tx=0).stop 2023-07-21T18:41:32.103 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.079+0000 7fa9b65eb700 1 -- 172.21.15.39:0/3009967472 shutdown_connections 2023-07-21T18:41:32.103 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.079+0000 7fa9b65eb700 1 --2- 172.21.15.39:0/3009967472 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9b00fe9d0 0x7fa9b00fede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:32.103 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.079+0000 7fa9b65eb700 1 --2- 172.21.15.39:0/3009967472 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa9b0100550 0x7fa9b0106e10 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:32.103 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.079+0000 7fa9b65eb700 1 --2- 172.21.15.39:0/3009967472 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9b00ffbc0 0x7fa9b0100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:32.103 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.079+0000 7fa9b65eb700 1 -- 172.21.15.39:0/3009967472 >> 172.21.15.39:0/3009967472 conn(0x7fa9b00f9fd0 msgr2=0x7fa9b00fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:32.103 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.079+0000 7fa9b65eb700 1 -- 172.21.15.39:0/3009967472 shutdown_connections 2023-07-21T18:41:32.104 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.079+0000 7fa9b65eb700 1 -- 172.21.15.39:0/3009967472 wait complete. 2023-07-21T18:41:32.105 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.080+0000 7fa9b65eb700 1 Processor -- start 2023-07-21T18:41:32.105 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.080+0000 7fa9b65eb700 1 -- start start 2023-07-21T18:41:32.106 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.080+0000 7fa9b65eb700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9b00fe9d0 0x7fa9b0195890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:32.106 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.080+0000 7fa9b65eb700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9b00ffbc0 0x7fa9b0195dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:32.106 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.080+0000 7fa9b65eb700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa9b0100550 0x7fa9b019ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:32.106 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.080+0000 7fa9b65eb700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b019d410 con 0x7fa9b0100550 2023-07-21T18:41:32.106 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.080+0000 7fa9b65eb700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b019d550 con 0x7fa9b00fe9d0 2023-07-21T18:41:32.106 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.080+0000 7fa9b65eb700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b019d6e0 con 0x7fa9b00ffbc0 2023-07-21T18:41:32.106 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.081+0000 7fa9b4b88700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa9b0100550 0x7fa9b019ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:32.107 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.081+0000 7fa9b4b88700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa9b0100550 0x7fa9b019ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56688/0 (socket says 172.21.15.39:56688) 2023-07-21T18:41:32.107 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.081+0000 7fa9b4b88700 1 -- 172.21.15.39:0/1486909819 learned_addr learned my addr 172.21.15.39:0/1486909819 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:32.107 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.081+0000 7fa9affff700 1 --2- 172.21.15.39:0/1486909819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9b00fe9d0 0x7fa9b0195890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:32.107 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.081+0000 7fa9af7fe700 1 --2- 172.21.15.39:0/1486909819 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9b00ffbc0 0x7fa9b0195dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:32.107 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.081+0000 7fa9b4b88700 1 -- 172.21.15.39:0/1486909819 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9b00ffbc0 msgr2=0x7fa9b0195dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:32.107 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.081+0000 7fa9b4b88700 1 --2- 172.21.15.39:0/1486909819 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9b00ffbc0 0x7fa9b0195dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:32.107 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.081+0000 7fa9b4b88700 1 -- 172.21.15.39:0/1486909819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9b00fe9d0 msgr2=0x7fa9b0195890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:32.107 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.081+0000 7fa9b4b88700 1 --2- 172.21.15.39:0/1486909819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9b00fe9d0 0x7fa9b0195890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:32.108 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.081+0000 7fa9b4b88700 1 -- 172.21.15.39:0/1486909819 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9a8009d00 con 0x7fa9b0100550 2023-07-21T18:41:32.108 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.081+0000 7fa9b4b88700 1 --2- 172.21.15.39:0/1486909819 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa9b0100550 0x7fa9b019ce40 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fa9a800b780 tx=0x7fa9a800bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:32.108 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.081+0000 7fa9ad7fa700 1 -- 172.21.15.39:0/1486909819 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a800e040 con 0x7fa9b0100550 2023-07-21T18:41:32.108 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.081+0000 7fa9ad7fa700 1 -- 172.21.15.39:0/1486909819 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9a8004e60 con 0x7fa9b0100550 2023-07-21T18:41:32.108 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.081+0000 7fa9b65eb700 1 -- 172.21.15.39:0/1486909819 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9b019d960 con 0x7fa9b0100550 2023-07-21T18:41:32.108 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.081+0000 7fa9ad7fa700 1 -- 172.21.15.39:0/1486909819 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a801b870 con 0x7fa9b0100550 2023-07-21T18:41:32.108 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.081+0000 7fa9b65eb700 1 -- 172.21.15.39:0/1486909819 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9b019de30 con 0x7fa9b0100550 2023-07-21T18:41:32.109 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.082+0000 7fa9b65eb700 1 -- 172.21.15.39:0/1486909819 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9b010a1b0 con 0x7fa9b0100550 2023-07-21T18:41:32.109 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.083+0000 7fa9ad7fa700 1 -- 172.21.15.39:0/1486909819 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fa9a801b9d0 con 0x7fa9b0100550 2023-07-21T18:41:32.109 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.084+0000 7fa9ad7fa700 1 --2- 172.21.15.39:0/1486909819 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fa9980a12a0 0x7fa9980a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:32.110 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.084+0000 7fa9ad7fa700 1 -- 172.21.15.39:0/1486909819 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fa9a80ce920 con 0x7fa9b0100550 2023-07-21T18:41:32.110 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.084+0000 7fa9affff700 1 --2- 172.21.15.39:0/1486909819 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fa9980a12a0 0x7fa9980a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:32.110 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.084+0000 7fa9affff700 1 --2- 172.21.15.39:0/1486909819 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fa9980a12a0 0x7fa9980a3750 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa9b00ffa20 tx=0x7fa9a0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:32.110 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.087+0000 7fa9ad7fa700 1 -- 172.21.15.39:0/1486909819 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fa9a8097fd0 con 0x7fa9b0100550 2023-07-21T18:41:32.309 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.287+0000 7fa9b65eb700 1 -- 172.21.15.39:0/1486909819 --> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa9b0067e80 con 0x7fa9980a12a0 2023-07-21T18:41:32.322 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.299+0000 7fa9ad7fa700 1 -- 172.21.15.39:0/1486909819 <== mgr.14152 v2:172.21.15.39:6800/2116291712 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+344482 (secure 0 0 0) 0x7fa9b0067e80 con 0x7fa9980a12a0 2023-07-21T18:41:32.325 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:41:32.328 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.303+0000 7fa9b65eb700 1 -- 172.21.15.39:0/1486909819 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fa9980a12a0 msgr2=0x7fa9980a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:32.329 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.303+0000 7fa9b65eb700 1 --2- 172.21.15.39:0/1486909819 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fa9980a12a0 0x7fa9980a3750 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa9b00ffa20 tx=0x7fa9a0009260 comp rx=0 tx=0).stop 2023-07-21T18:41:32.329 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.303+0000 7fa9b65eb700 1 -- 172.21.15.39:0/1486909819 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa9b0100550 msgr2=0x7fa9b019ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:32.329 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.303+0000 7fa9b65eb700 1 --2- 172.21.15.39:0/1486909819 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa9b0100550 0x7fa9b019ce40 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fa9a800b780 tx=0x7fa9a800bdb0 comp rx=0 tx=0).stop 2023-07-21T18:41:32.329 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.303+0000 7fa9b65eb700 1 -- 172.21.15.39:0/1486909819 shutdown_connections 2023-07-21T18:41:32.329 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.303+0000 7fa9b65eb700 1 --2- 172.21.15.39:0/1486909819 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fa9980a12a0 0x7fa9980a3750 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:32.329 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.303+0000 7fa9b65eb700 1 --2- 172.21.15.39:0/1486909819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9b00fe9d0 0x7fa9b0195890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:32.330 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.303+0000 7fa9b65eb700 1 --2- 172.21.15.39:0/1486909819 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa9b0100550 0x7fa9b019ce40 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:32.330 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.303+0000 7fa9b65eb700 1 --2- 172.21.15.39:0/1486909819 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9b00ffbc0 0x7fa9b0195dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:32.330 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.303+0000 7fa9b65eb700 1 -- 172.21.15.39:0/1486909819 >> 172.21.15.39:0/1486909819 conn(0x7fa9b00f9fd0 msgr2=0x7fa9b0067770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:32.330 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.303+0000 7fa9b65eb700 1 -- 172.21.15.39:0/1486909819 shutdown_connections 2023-07-21T18:41:32.330 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.303+0000 7fa9b65eb700 1 -- 172.21.15.39:0/1486909819 wait complete. 2023-07-21T18:41:32.330 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2023-07-21T18:41:32.337 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:32 smithi039 ceph-mon[96489]: pgmap v293: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 9 op/s 2023-07-21T18:41:32.337 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:32 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/453869244' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T18:41:32.337 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:32 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/1494681701' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T18:41:32.338 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:32 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/2001220702' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:41:32.338 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2023-07-21T18:41:31.141306+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654745,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":953,"num_read_kb":596,"num_write":416,"num_write_kb":3068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180736,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":343,"ondisk_log_size":343,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335916,"kb_used_data":12492,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124605908,"statfs":{"total":1151940427776,"available":1151596449792,"internally_reserved":0,"allocated":12791808,"data_stored":7060074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":636,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":111,"num_read_kb":51,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003031"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.144002+0000","last_change":"2023-07-21T18:40:04.792977+0000","last_active":"2023-07-21T18:41:08.144002+0000","last_peered":"2023-07-21T18:41:08.144002+0000","last_clean":"2023-07-21T18:41:08.144002+0000","last_became_active":"2023-07-21T18:40:04.792667+0000","last_became_peered":"2023-07-21T18:40:04.792667+0000","last_unstale":"2023-07-21T18:41:08.144002+0000","last_undegraded":"2023-07-21T18:41:08.144002+0000","last_fullsized":"2023-07-21T18:41:08.144002+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:45:33.822350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.154989+0000","last_change":"2023-07-21T18:40:04.792936+0000","last_active":"2023-07-21T18:41:08.154989+0000","last_peered":"2023-07-21T18:41:08.154989+0000","last_clean":"2023-07-21T18:41:08.154989+0000","last_became_active":"2023-07-21T18:40:04.792511+0000","last_became_peered":"2023-07-21T18:40:04.792511+0000","last_unstale":"2023-07-21T18:41:08.154989+0000","last_undegraded":"2023-07-21T18:41:08.154989+0000","last_fullsized":"2023-07-21T18:41:08.154989+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:18:24.383764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.154716+0000","last_change":"2023-07-21T18:40:04.796401+0000","last_active":"2023-07-21T18:41:08.154716+0000","last_peered":"2023-07-21T18:41:08.154716+0000","last_clean":"2023-07-21T18:41:08.154716+0000","last_became_active":"2023-07-21T18:40:04.796063+0000","last_became_peered":"2023-07-21T18:40:04.796063+0000","last_unstale":"2023-07-21T18:41:08.154716+0000","last_undegraded":"2023-07-21T18:41:08.154716+0000","last_fullsized":"2023-07-21T18:41:08.154716+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:53:37.099167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.154119+0000","last_change":"2023-07-21T18:41:08.154119+0000","last_active":"2023-07-21T18:41:08.154119+0000","last_peered":"2023-07-21T18:41:08.154119+0000","last_clean":"2023-07-21T18:41:08.154119+0000","last_became_active":"2023-07-21T18:41:08.153698+0000","last_became_peered":"2023-07-21T18:41:08.153698+0000","last_unstale":"2023-07-21T18:41:08.154119+0000","last_undegraded":"2023-07-21T18:41:08.154119+0000","last_fullsized":"2023-07-21T18:41:08.154119+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T18:41:49.057189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.148468+0000","last_change":"2023-07-21T18:40:04.794035+0000","last_active":"2023-07-21T18:41:08.148468+0000","last_peered":"2023-07-21T18:41:08.148468+0000","last_clean":"2023-07-21T18:41:08.148468+0000","last_became_active":"2023-07-21T18:40:04.793739+0000","last_became_peered":"2023-07-21T18:40:04.793739+0000","last_unstale":"2023-07-21T18:41:08.148468+0000","last_undegraded":"2023-07-21T18:41:08.148468+0000","last_fullsized":"2023-07-21T18:41:08.148468+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:25:53.830527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.150742+0000","last_change":"2023-07-21T18:41:08.150742+0000","last_active":"2023-07-21T18:41:08.150742+0000","last_peered":"2023-07-21T18:41:08.150742+0000","last_clean":"2023-07-21T18:41:08.150742+0000","last_became_active":"2023-07-21T18:41:08.150346+0000","last_became_peered":"2023-07-21T18:41:08.150346+0000","last_unstale":"2023-07-21T18:41:08.150742+0000","last_undegraded":"2023-07-21T18:41:08.150742+0000","last_fullsized":"2023-07-21T18:41:08.150742+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:06:31.088681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.595238+0000","last_change":"2023-07-21T18:40:04.787544+0000","last_active":"2023-07-21T18:41:08.595238+0000","last_peered":"2023-07-21T18:41:08.595238+0000","last_clean":"2023-07-21T18:41:08.595238+0000","last_became_active":"2023-07-21T18:40:04.787208+0000","last_became_peered":"2023-07-21T18:40:04.787208+0000","last_unstale":"2023-07-21T18:41:08.595238+0000","last_undegraded":"2023-07-21T18:41:08.595238+0000","last_fullsized":"2023-07-21T18:41:08.595238+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:57:13.333034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.249534+0000","last_change":"2023-07-21T18:41:08.249534+0000","last_active":"2023-07-21T18:41:08.249534+0000","last_peered":"2023-07-21T18:41:08.249534+0000","last_clean":"2023-07-21T18:41:08.249534+0000","last_became_active":"2023-07-21T18:41:08.249228+0000","last_became_peered":"2023-07-21T18:41:08.249228+0000","last_unstale":"2023-07-21T18:41:08.249534+0000","last_undegraded":"2023-07-21T18:41:08.249534+0000","last_fullsized":"2023-07-21T18:41:08.249534+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:19:58.325032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.151247+0000","last_change":"2023-07-21T18:40:04.782495+0000","last_active":"2023-07-21T18:41:08.151247+0000","last_peered":"2023-07-21T18:41:08.151247+0000","last_clean":"2023-07-21T18:41:08.151247+0000","last_became_active":"2023-07-21T18:40:04.782208+0000","last_became_peered":"2023-07-21T18:40:04.782208+0000","last_unstale":"2023-07-21T18:41:08.151247+0000","last_undegraded":"2023-07-21T18:41:08.151247+0000","last_fullsized":"2023-07-21T18:41:08.151247+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:22:50.428618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.148205+0000","last_change":"2023-07-21T18:40:04.790957+0000","last_active":"2023-07-21T18:41:08.148205+0000","last_peered":"2023-07-21T18:41:08.148205+0000","last_clean":"2023-07-21T18:41:08.148205+0000","last_became_active":"2023-07-21T18:40:04.790659+0000","last_became_peered":"2023-07-21T18:40:04.790659+0000","last_unstale":"2023-07-21T18:41:08.148205+0000","last_undegraded":"2023-07-21T18:41:08.148205+0000","last_fullsized":"2023-07-21T18:41:08.148205+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:29:03.090784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.595484+0000","last_change":"2023-07-21T18:40:04.789415+0000","last_active":"2023-07-21T18:41:08.595484+0000","last_peered":"2023-07-21T18:41:08.595484+0000","last_clean":"2023-07-21T18:41:08.595484+0000","last_became_active":"2023-07-21T18:40:04.789088+0000","last_became_peered":"2023-07-21T18:40:04.789088+0000","last_unstale":"2023-07-21T18:41:08.595484+0000","last_undegraded":"2023-07-21T18:41:08.595484+0000","last_fullsized":"2023-07-21T18:41:08.595484+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:40:03.977217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.245377+0000","last_change":"2023-07-21T18:40:04.789101+0000","last_active":"2023-07-21T18:41:08.245377+0000","last_peered":"2023-07-21T18:41:08.245377+0000","last_clean":"2023-07-21T18:41:08.245377+0000","last_became_active":"2023-07-21T18:40:04.788704+0000","last_became_peered":"2023-07-21T18:40:04.788704+0000","last_unstale":"2023-07-21T18:41:08.245377+0000","last_undegraded":"2023-07-21T18:41:08.245377+0000","last_fullsized":"2023-07-21T18:41:08.245377+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:33:01.699609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.144358+0000","last_change":"2023-07-21T18:40:04.784863+0000","last_active":"2023-07-21T18:41:08.144358+0000","last_peered":"2023-07-21T18:41:08.144358+0000","last_clean":"2023-07-21T18:41:08.144358+0000","last_became_active":"2023-07-21T18:40:04.784474+0000","last_became_peered":"2023-07-21T18:40:04.784474+0000","last_unstale":"2023-07-21T18:41:08.144358+0000","last_undegraded":"2023-07-21T18:41:08.144358+0000","last_fullsized":"2023-07-21T18:41:08.144358+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:20:04.409410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.154148+0000","last_change":"2023-07-21T18:40:04.797678+0000","last_active":"2023-07-21T18:41:08.154148+0000","last_peered":"2023-07-21T18:41:08.154148+0000","last_clean":"2023-07-21T18:41:08.154148+0000","last_became_active":"2023-07-21T18:40:04.797318+0000","last_became_peered":"2023-07-21T18:40:04.797318+0000","last_unstale":"2023-07-21T18:41:08.154148+0000","last_undegraded":"2023-07-21T18:41:08.154148+0000","last_fullsized":"2023-07-21T18:41:08.154148+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:28:01.333981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.153924+0000","last_change":"2023-07-21T18:40:04.793170+0000","last_active":"2023-07-21T18:41:08.153924+0000","last_peered":"2023-07-21T18:41:08.153924+0000","last_clean":"2023-07-21T18:41:08.153924+0000","last_became_active":"2023-07-21T18:40:04.792563+0000","last_became_peered":"2023-07-21T18:40:04.792563+0000","last_unstale":"2023-07-21T18:41:08.153924+0000","last_undegraded":"2023-07-21T18:41:08.153924+0000","last_fullsized":"2023-07-21T18:41:08.153924+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:07:51.115805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.150518+0000","last_change":"2023-07-21T18:40:04.791972+0000","last_active":"2023-07-21T18:41:08.150518+0000","last_peered":"2023-07-21T18:41:08.150518+0000","last_clean":"2023-07-21T18:41:08.150518+0000","last_became_active":"2023-07-21T18:40:04.791639+0000","last_became_peered":"2023-07-21T18:40:04.791639+0000","last_unstale":"2023-07-21T18:41:08.150518+0000","last_undegraded":"2023-07-21T18:41:08.150518+0000","last_fullsized":"2023-07-21T18:41:08.150518+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:31:09.972680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.144015+0000","last_change":"2023-07-21T18:40:04.789362+0000","last_active":"2023-07-21T18:41:08.144015+0000","last_peered":"2023-07-21T18:41:08.144015+0000","last_clean":"2023-07-21T18:41:08.144015+0000","last_became_active":"2023-07-21T18:40:04.788978+0000","last_became_peered":"2023-07-21T18:40:04.788978+0000","last_unstale":"2023-07-21T18:41:08.144015+0000","last_undegraded":"2023-07-21T18:41:08.144015+0000","last_fullsized":"2023-07-21T18:41:08.144015+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:06:43.785294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.146563+0000","last_change":"2023-07-21T18:40:04.788016+0000","last_active":"2023-07-21T18:41:08.146563+0000","last_peered":"2023-07-21T18:41:08.146563+0000","last_clean":"2023-07-21T18:41:08.146563+0000","last_became_active":"2023-07-21T18:40:04.787690+0000","last_became_peered":"2023-07-21T18:40:04.787690+0000","last_unstale":"2023-07-21T18:41:08.146563+0000","last_undegraded":"2023-07-21T18:41:08.146563+0000","last_fullsized":"2023-07-21T18:41:08.146563+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:40:42.379190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.152768+0000","last_change":"2023-07-21T18:40:04.793075+0000","last_active":"2023-07-21T18:41:08.152768+0000","last_peered":"2023-07-21T18:41:08.152768+0000","last_clean":"2023-07-21T18:41:08.152768+0000","last_became_active":"2023-07-21T18:40:04.792773+0000","last_became_peered":"2023-07-21T18:40:04.792773+0000","last_unstale":"2023-07-21T18:41:08.152768+0000","last_undegraded":"2023-07-21T18:41:08.152768+0000","last_fullsized":"2023-07-21T18:41:08.152768+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:46:27.429800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"79'13","reported_seq":108,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:26.100464+0000","last_change":"2023-07-21T18:40:04.178197+0000","last_active":"2023-07-21T18:41:26.100464+0000","last_peered":"2023-07-21T18:41:26.100464+0000","last_clean":"2023-07-21T18:41:26.100464+0000","last_became_active":"2023-07-21T18:40:04.177914+0000","last_became_peered":"2023-07-21T18:40:04.177914+0000","last_unstale":"2023-07-21T18:41:26.100464+0000","last_undegraded":"2023-07-21T18:41:26.100464+0000","last_fullsized":"2023-07-21T18:41:26.100464+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:55:39.788580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":77,"num_read_kb":77,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.149760+0000","last_change":"2023-07-21T18:40:04.791610+0000","last_active":"2023-07-21T18:41:08.149760+0000","last_peered":"2023-07-21T18:41:08.149760+0000","last_clean":"2023-07-21T18:41:08.149760+0000","last_became_active":"2023-07-21T18:40:04.791242+0000","last_became_peered":"2023-07-21T18:40:04.791242+0000","last_unstale":"2023-07-21T18:41:08.149760+0000","last_undegraded":"2023-07-21T18:41:08.149760+0000","last_fullsized":"2023-07-21T18:41:08.149760+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:27:52.116304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.149806+0000","last_change":"2023-07-21T18:40:04.182120+0000","last_active":"2023-07-21T18:41:08.149806+0000","last_peered":"2023-07-21T18:41:08.149806+0000","last_clean":"2023-07-21T18:41:08.149806+0000","last_became_active":"2023-07-21T18:40:04.181797+0000","last_became_peered":"2023-07-21T18:40:04.181797+0000","last_unstale":"2023-07-21T18:41:08.149806+0000","last_undegraded":"2023-07-21T18:41:08.149806+0000","last_fullsized":"2023-07-21T18:41:08.149806+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:47:45.817595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.155369+0000","last_change":"2023-07-21T18:40:04.796649+0000","last_active":"2023-07-21T18:41:08.155369+0000","last_peered":"2023-07-21T18:41:08.155369+0000","last_clean":"2023-07-21T18:41:08.155369+0000","last_became_active":"2023-07-21T18:40:04.795610+0000","last_became_peered":"2023-07-21T18:40:04.795610+0000","last_unstale":"2023-07-21T18:41:08.155369+0000","last_undegraded":"2023-07-21T18:41:08.155369+0000","last_fullsized":"2023-07-21T18:41:08.155369+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:22:37.727432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"81'13","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.149058+0000","last_change":"2023-07-21T18:40:04.178384+0000","last_active":"2023-07-21T18:41:08.149058+0000","last_peered":"2023-07-21T18:41:08.149058+0000","last_clean":"2023-07-21T18:41:08.149058+0000","last_became_active":"2023-07-21T18:40:04.178074+0000","last_became_peered":"2023-07-21T18:40:04.178074+0000","last_unstale":"2023-07-21T18:41:08.149058+0000","last_undegraded":"2023-07-21T18:41:08.149058+0000","last_fullsized":"2023-07-21T18:41:08.149058+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:20:16.081243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.150712+0000","last_change":"2023-07-21T18:40:04.785431+0000","last_active":"2023-07-21T18:41:08.150712+0000","last_peered":"2023-07-21T18:41:08.150712+0000","last_clean":"2023-07-21T18:41:08.150712+0000","last_became_active":"2023-07-21T18:40:04.785124+0000","last_became_peered":"2023-07-21T18:40:04.785124+0000","last_unstale":"2023-07-21T18:41:08.150712+0000","last_undegraded":"2023-07-21T18:41:08.150712+0000","last_fullsized":"2023-07-21T18:41:08.150712+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:07:13.594091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"79'5","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.148841+0000","last_change":"2023-07-21T18:40:04.176349+0000","last_active":"2023-07-21T18:41:08.148841+0000","last_peered":"2023-07-21T18:41:08.148841+0000","last_clean":"2023-07-21T18:41:08.148841+0000","last_became_active":"2023-07-21T18:40:04.175997+0000","last_became_peered":"2023-07-21T18:40:04.175997+0000","last_unstale":"2023-07-21T18:41:08.148841+0000","last_undegraded":"2023-07-21T18:41:08.148841+0000","last_fullsized":"2023-07-21T18:41:08.148841+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:24:53.545517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.246653+0000","last_change":"2023-07-21T18:41:08.246653+0000","last_active":"2023-07-21T18:41:08.246653+0000","last_peered":"2023-07-21T18:41:08.246653+0000","last_clean":"2023-07-21T18:41:08.246653+0000","last_became_active":"2023-07-21T18:41:08.246035+0000","last_became_peered":"2023-07-21T18:41:08.246035+0000","last_unstale":"2023-07-21T18:41:08.246653+0000","last_undegraded":"2023-07-21T18:41:08.246653+0000","last_fullsized":"2023-07-21T18:41:08.246653+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:30:36.054664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1b","version":"79'5","reported_seq":105,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:29.084542+0000","last_change":"2023-07-21T18:40:04.177638+0000","last_active":"2023-07-21T18:41:29.084542+0000","last_peered":"2023-07-21T18:41:29.084542+0000","last_clean":"2023-07-21T18:41:29.084542+0000","last_became_active":"2023-07-21T18:40:04.177284+0000","last_became_peered":"2023-07-21T18:40:04.177284+0000","last_unstale":"2023-07-21T18:41:29.084542+0000","last_undegraded":"2023-07-21T18:41:29.084542+0000","last_fullsized":"2023-07-21T18:41:29.084542+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:34:36.750937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.148514+0000","last_change":"2023-07-21T18:40:04.791536+0000","last_active":"2023-07-21T18:41:08.148514+0000","last_peered":"2023-07-21T18:41:08.148514+0000","last_clean":"2023-07-21T18:41:08.148514+0000","last_became_active":"2023-07-21T18:40:04.791246+0000","last_became_peered":"2023-07-21T18:40:04.791246+0000","last_unstale":"2023-07-21T18:41:08.148514+0000","last_undegraded":"2023-07-21T18:41:08.148514+0000","last_fullsized":"2023-07-21T18:41:08.148514+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:30:46.562521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"79'7","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.147995+0000","last_change":"2023-07-21T18:40:04.179054+0000","last_active":"2023-07-21T18:41:08.147995+0000","last_peered":"2023-07-21T18:41:08.147995+0000","last_clean":"2023-07-21T18:41:08.147995+0000","last_became_active":"2023-07-21T18:40:04.178713+0000","last_became_peered":"2023-07-21T18:40:04.178713+0000","last_unstale":"2023-07-21T18:41:08.147995+0000","last_undegraded":"2023-07-21T18:41:08.147995+0000","last_fullsized":"2023-07-21T18:41:08.147995+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:54:00.028673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.249387+0000","last_change":"2023-07-21T18:40:04.793947+0000","last_active":"2023-07-21T18:41:08.249387+0000","last_peered":"2023-07-21T18:41:08.249387+0000","last_clean":"2023-07-21T18:41:08.249387+0000","last_became_active":"2023-07-21T18:40:04.793631+0000","last_became_peered":"2023-07-21T18:40:04.793631+0000","last_unstale":"2023-07-21T18:41:08.249387+0000","last_undegraded":"2023-07-21T18:41:08.249387+0000","last_fullsized":"2023-07-21T18:41:08.249387+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T18:56:26.416040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"79'3","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.143902+0000","last_change":"2023-07-21T18:40:04.182549+0000","last_active":"2023-07-21T18:41:08.143902+0000","last_peered":"2023-07-21T18:41:08.143902+0000","last_clean":"2023-07-21T18:41:08.143902+0000","last_became_active":"2023-07-21T18:40:04.182233+0000","last_became_peered":"2023-07-21T18:40:04.182233+0000","last_unstale":"2023-07-21T18:41:08.143902+0000","last_undegraded":"2023-07-21T18:41:08.143902+0000","last_fullsized":"2023-07-21T18:41:08.143902+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:38:34.232736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.149083+0000","last_change":"2023-07-21T18:40:04.789987+0000","last_active":"2023-07-21T18:41:08.149083+0000","last_peered":"2023-07-21T18:41:08.149083+0000","last_clean":"2023-07-21T18:41:08.149083+0000","last_became_active":"2023-07-21T18:40:04.789647+0000","last_became_peered":"2023-07-21T18:40:04.789647+0000","last_unstale":"2023-07-21T18:41:08.149083+0000","last_undegraded":"2023-07-21T18:41:08.149083+0000","last_fullsized":"2023-07-21T18:41:08.149083+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:33:16.733179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'3","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.248336+0000","last_change":"2023-07-21T18:40:04.177342+0000","last_active":"2023-07-21T18:41:08.248336+0000","last_peered":"2023-07-21T18:41:08.248336+0000","last_clean":"2023-07-21T18:41:08.248336+0000","last_became_active":"2023-07-21T18:40:04.176983+0000","last_became_peered":"2023-07-21T18:40:04.176983+0000","last_unstale":"2023-07-21T18:41:08.248336+0000","last_undegraded":"2023-07-21T18:41:08.248336+0000","last_fullsized":"2023-07-21T18:41:08.248336+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:25:28.759619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.146313+0000","last_change":"2023-07-21T18:40:04.794907+0000","last_active":"2023-07-21T18:41:08.146313+0000","last_peered":"2023-07-21T18:41:08.146313+0000","last_clean":"2023-07-21T18:41:08.146313+0000","last_became_active":"2023-07-21T18:40:04.794545+0000","last_became_peered":"2023-07-21T18:40:04.794545+0000","last_unstale":"2023-07-21T18:41:08.146313+0000","last_undegraded":"2023-07-21T18:41:08.146313+0000","last_fullsized":"2023-07-21T18:41:08.146313+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:42:33.019351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"79'4","reported_seq":178,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:28.082056+0000","last_change":"2023-07-21T18:40:04.179198+0000","last_active":"2023-07-21T18:41:28.082056+0000","last_peered":"2023-07-21T18:41:28.082056+0000","last_clean":"2023-07-21T18:41:28.082056+0000","last_became_active":"2023-07-21T18:40:04.178841+0000","last_became_peered":"2023-07-21T18:40:04.178841+0000","last_unstale":"2023-07-21T18:41:28.082056+0000","last_undegraded":"2023-07-21T18:41:28.082056+0000","last_fullsized":"2023-07-21T18:41:28.082056+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:12:15.233249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":156,"num_read_kb":156,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.151068+0000","last_change":"2023-07-21T18:40:04.786229+0000","last_active":"2023-07-21T18:41:08.151068+0000","last_peered":"2023-07-21T18:41:08.151068+0000","last_clean":"2023-07-21T18:41:08.151068+0000","last_became_active":"2023-07-21T18:40:04.785919+0000","last_became_peered":"2023-07-21T18:40:04.785919+0000","last_unstale":"2023-07-21T18:41:08.151068+0000","last_undegraded":"2023-07-21T18:41:08.151068+0000","last_fullsized":"2023-07-21T18:41:08.151068+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:37:35.592913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'4","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.595131+0000","last_change":"2023-07-21T18:40:04.177164+0000","last_active":"2023-07-21T18:41:08.595131+0000","last_peered":"2023-07-21T18:41:08.595131+0000","last_clean":"2023-07-21T18:41:08.595131+0000","last_became_active":"2023-07-21T18:40:04.176617+0000","last_became_peered":"2023-07-21T18:40:04.176617+0000","last_unstale":"2023-07-21T18:41:08.595131+0000","last_undegraded":"2023-07-21T18:41:08.595131+0000","last_fullsized":"2023-07-21T18:41:08.595131+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:43:44.771285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.155486+0000","last_change":"2023-07-21T18:40:04.787863+0000","last_active":"2023-07-21T18:41:08.155486+0000","last_peered":"2023-07-21T18:41:08.155486+0000","last_clean":"2023-07-21T18:41:08.155486+0000","last_became_active":"2023-07-21T18:40:04.787562+0000","last_became_peered":"2023-07-21T18:40:04.787562+0000","last_unstale":"2023-07-21T18:41:08.155486+0000","last_undegraded":"2023-07-21T18:41:08.155486+0000","last_fullsized":"2023-07-21T18:41:08.155486+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:15:49.834031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'5","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.248499+0000","last_change":"2023-07-21T18:40:04.179921+0000","last_active":"2023-07-21T18:41:08.248499+0000","last_peered":"2023-07-21T18:41:08.248499+0000","last_clean":"2023-07-21T18:41:08.248499+0000","last_became_active":"2023-07-21T18:40:04.179670+0000","last_became_peered":"2023-07-21T18:40:04.179670+0000","last_unstale":"2023-07-21T18:41:08.248499+0000","last_undegraded":"2023-07-21T18:41:08.248499+0000","last_fullsized":"2023-07-21T18:41:08.248499+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:15:29.991358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.150860+0000","last_change":"2023-07-21T18:40:04.792930+0000","last_active":"2023-07-21T18:41:08.150860+0000","last_peered":"2023-07-21T18:41:08.150860+0000","last_clean":"2023-07-21T18:41:08.150860+0000","last_became_active":"2023-07-21T18:40:04.792654+0000","last_became_peered":"2023-07-21T18:40:04.792654+0000","last_unstale":"2023-07-21T18:41:08.150860+0000","last_undegraded":"2023-07-21T18:41:08.150860+0000","last_fullsized":"2023-07-21T18:41:08.150860+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:03:06.046049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'30","reported_seq":210,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:28.377347+0000","last_change":"2023-07-21T18:40:04.178820+0000","last_active":"2023-07-21T18:41:28.377347+0000","last_peered":"2023-07-21T18:41:28.377347+0000","last_clean":"2023-07-21T18:41:28.377347+0000","last_became_active":"2023-07-21T18:40:04.178502+0000","last_became_peered":"2023-07-21T18:40:04.178502+0000","last_unstale":"2023-07-21T18:41:28.377347+0000","last_undegraded":"2023-07-21T18:41:28.377347+0000","last_fullsized":"2023-07-21T18:41:28.377347+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:19:51.865558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57193,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":162,"num_read_kb":136,"num_write":32,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.148850+0000","last_change":"2023-07-21T18:40:04.792921+0000","last_active":"2023-07-21T18:41:08.148850+0000","last_peered":"2023-07-21T18:41:08.148850+0000","last_clean":"2023-07-21T18:41:08.148850+0000","last_became_active":"2023-07-21T18:40:04.792608+0000","last_became_peered":"2023-07-21T18:40:04.792608+0000","last_unstale":"2023-07-21T18:41:08.148850+0000","last_undegraded":"2023-07-21T18:41:08.148850+0000","last_fullsized":"2023-07-21T18:41:08.148850+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:58:38.532900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'9","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.153057+0000","last_change":"2023-07-21T18:40:04.184917+0000","last_active":"2023-07-21T18:41:08.153057+0000","last_peered":"2023-07-21T18:41:08.153057+0000","last_clean":"2023-07-21T18:41:08.153057+0000","last_became_active":"2023-07-21T18:40:04.184612+0000","last_became_peered":"2023-07-21T18:40:04.184612+0000","last_unstale":"2023-07-21T18:41:08.153057+0000","last_undegraded":"2023-07-21T18:41:08.153057+0000","last_fullsized":"2023-07-21T18:41:08.153057+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:58:56.789436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.143966+0000","last_change":"2023-07-21T18:40:04.794134+0000","last_active":"2023-07-21T18:41:08.143966+0000","last_peered":"2023-07-21T18:41:08.143966+0000","last_clean":"2023-07-21T18:41:08.143966+0000","last_became_active":"2023-07-21T18:40:04.793784+0000","last_became_peered":"2023-07-21T18:40:04.793784+0000","last_unstale":"2023-07-21T18:41:08.143966+0000","last_undegraded":"2023-07-21T18:41:08.143966+0000","last_fullsized":"2023-07-21T18:41:08.143966+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:30:56.528513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'5","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.248332+0000","last_change":"2023-07-21T18:40:04.184889+0000","last_active":"2023-07-21T18:41:08.248332+0000","last_peered":"2023-07-21T18:41:08.248332+0000","last_clean":"2023-07-21T18:41:08.248332+0000","last_became_active":"2023-07-21T18:40:04.184516+0000","last_became_peered":"2023-07-21T18:40:04.184516+0000","last_unstale":"2023-07-21T18:41:08.248332+0000","last_undegraded":"2023-07-21T18:41:08.248332+0000","last_fullsized":"2023-07-21T18:41:08.248332+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:28:26.823194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.595621+0000","last_change":"2023-07-21T18:40:04.790627+0000","last_active":"2023-07-21T18:41:08.595621+0000","last_peered":"2023-07-21T18:41:08.595621+0000","last_clean":"2023-07-21T18:41:08.595621+0000","last_became_active":"2023-07-21T18:40:04.790298+0000","last_became_peered":"2023-07-21T18:40:04.790298+0000","last_unstale":"2023-07-21T18:41:08.595621+0000","last_undegraded":"2023-07-21T18:41:08.595621+0000","last_fullsized":"2023-07-21T18:41:08.595621+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:07:48.083182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"79'6","reported_seq":103,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:26.079011+0000","last_change":"2023-07-21T18:40:04.176156+0000","last_active":"2023-07-21T18:41:26.079011+0000","last_peered":"2023-07-21T18:41:26.079011+0000","last_clean":"2023-07-21T18:41:26.079011+0000","last_became_active":"2023-07-21T18:40:04.175823+0000","last_became_peered":"2023-07-21T18:40:04.175823+0000","last_unstale":"2023-07-21T18:41:26.079011+0000","last_undegraded":"2023-07-21T18:41:26.079011+0000","last_fullsized":"2023-07-21T18:41:26.079011+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:40:30.078845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":930,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":80,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.595323+0000","last_change":"2023-07-21T18:40:04.790422+0000","last_active":"2023-07-21T18:41:08.595323+0000","last_peered":"2023-07-21T18:41:08.595323+0000","last_clean":"2023-07-21T18:41:08.595323+0000","last_became_active":"2023-07-21T18:40:04.790124+0000","last_became_peered":"2023-07-21T18:40:04.790124+0000","last_unstale":"2023-07-21T18:41:08.595323+0000","last_undegraded":"2023-07-21T18:41:08.595323+0000","last_fullsized":"2023-07-21T18:41:08.595323+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:11:35.114265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'17","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.148291+0000","last_change":"2023-07-21T18:40:04.177006+0000","last_active":"2023-07-21T18:41:08.148291+0000","last_peered":"2023-07-21T18:41:08.148291+0000","last_clean":"2023-07-21T18:41:08.148291+0000","last_became_active":"2023-07-21T18:40:04.176711+0000","last_became_peered":"2023-07-21T18:40:04.176711+0000","last_unstale":"2023-07-21T18:41:08.148291+0000","last_undegraded":"2023-07-21T18:41:08.148291+0000","last_fullsized":"2023-07-21T18:41:08.148291+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:05:40.549509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.249206+0000","last_change":"2023-07-21T18:40:04.785601+0000","last_active":"2023-07-21T18:41:08.249206+0000","last_peered":"2023-07-21T18:41:08.249206+0000","last_clean":"2023-07-21T18:41:08.249206+0000","last_became_active":"2023-07-21T18:40:04.785307+0000","last_became_peered":"2023-07-21T18:40:04.785307+0000","last_unstale":"2023-07-21T18:41:08.249206+0000","last_undegraded":"2023-07-21T18:41:08.249206+0000","last_fullsized":"2023-07-21T18:41:08.249206+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:04:30.580644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.149118+0000","last_change":"2023-07-21T18:40:04.171005+0000","last_active":"2023-07-21T18:41:08.149118+0000","last_peered":"2023-07-21T18:41:08.149118+0000","last_clean":"2023-07-21T18:41:08.149118+0000","last_became_active":"2023-07-21T18:40:04.170719+0000","last_became_peered":"2023-07-21T18:40:04.170719+0000","last_unstale":"2023-07-21T18:41:08.149118+0000","last_undegraded":"2023-07-21T18:41:08.149118+0000","last_fullsized":"2023-07-21T18:41:08.149118+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:06:18.122712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.249987+0000","last_change":"2023-07-21T18:41:08.249987+0000","last_active":"2023-07-21T18:41:08.249987+0000","last_peered":"2023-07-21T18:41:08.249987+0000","last_clean":"2023-07-21T18:41:08.249987+0000","last_became_active":"2023-07-21T18:41:08.249610+0000","last_became_peered":"2023-07-21T18:41:08.249610+0000","last_unstale":"2023-07-21T18:41:08.249987+0000","last_undegraded":"2023-07-21T18:41:08.249987+0000","last_fullsized":"2023-07-21T18:41:08.249987+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:40:12.815463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"79'1","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.146358+0000","last_change":"2023-07-21T18:40:04.181069+0000","last_active":"2023-07-21T18:41:08.146358+0000","last_peered":"2023-07-21T18:41:08.146358+0000","last_clean":"2023-07-21T18:41:08.146358+0000","last_became_active":"2023-07-21T18:40:04.180750+0000","last_became_peered":"2023-07-21T18:40:04.180750+0000","last_unstale":"2023-07-21T18:41:08.146358+0000","last_undegraded":"2023-07-21T18:41:08.146358+0000","last_fullsized":"2023-07-21T18:41:08.146358+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:33:23.897099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"79'1","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.147933+0000","last_change":"2023-07-21T18:40:04.789940+0000","last_active":"2023-07-21T18:41:08.147933+0000","last_peered":"2023-07-21T18:41:08.147933+0000","last_clean":"2023-07-21T18:41:08.147933+0000","last_became_active":"2023-07-21T18:40:04.789465+0000","last_became_peered":"2023-07-21T18:40:04.789465+0000","last_unstale":"2023-07-21T18:41:08.147933+0000","last_undegraded":"2023-07-21T18:41:08.147933+0000","last_fullsized":"2023-07-21T18:41:08.147933+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:31:28.465807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"81'6","reported_seq":105,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:29.105557+0000","last_change":"2023-07-21T18:40:04.180522+0000","last_active":"2023-07-21T18:41:29.105557+0000","last_peered":"2023-07-21T18:41:29.105557+0000","last_clean":"2023-07-21T18:41:29.105557+0000","last_became_active":"2023-07-21T18:40:04.180196+0000","last_became_peered":"2023-07-21T18:40:04.180196+0000","last_unstale":"2023-07-21T18:41:29.105557+0000","last_undegraded":"2023-07-21T18:41:29.105557+0000","last_fullsized":"2023-07-21T18:41:29.105557+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:24:49.698830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":858,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.148148+0000","last_change":"2023-07-21T18:40:04.794781+0000","last_active":"2023-07-21T18:41:08.148148+0000","last_peered":"2023-07-21T18:41:08.148148+0000","last_clean":"2023-07-21T18:41:08.148148+0000","last_became_active":"2023-07-21T18:40:04.794388+0000","last_became_peered":"2023-07-21T18:40:04.794388+0000","last_unstale":"2023-07-21T18:41:08.148148+0000","last_undegraded":"2023-07-21T18:41:08.148148+0000","last_fullsized":"2023-07-21T18:41:08.148148+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:01:02.955504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.248717+0000","last_change":"2023-07-21T18:40:04.786408+0000","last_active":"2023-07-21T18:41:08.248717+0000","last_peered":"2023-07-21T18:41:08.248717+0000","last_clean":"2023-07-21T18:41:08.248717+0000","last_became_active":"2023-07-21T18:40:04.785555+0000","last_became_peered":"2023-07-21T18:40:04.785555+0000","last_unstale":"2023-07-21T18:41:08.248717+0000","last_undegraded":"2023-07-21T18:41:08.248717+0000","last_fullsized":"2023-07-21T18:41:08.248717+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:45:51.047733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"79'5","reported_seq":102,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:27.080056+0000","last_change":"2023-07-21T18:40:04.185980+0000","last_active":"2023-07-21T18:41:27.080056+0000","last_peered":"2023-07-21T18:41:27.080056+0000","last_clean":"2023-07-21T18:41:27.080056+0000","last_became_active":"2023-07-21T18:40:04.185618+0000","last_became_peered":"2023-07-21T18:40:04.185618+0000","last_unstale":"2023-07-21T18:41:27.080056+0000","last_undegraded":"2023-07-21T18:41:27.080056+0000","last_fullsized":"2023-07-21T18:41:27.080056+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:37:27.318460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":79,"num_read_kb":79,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.149558+0000","last_change":"2023-07-21T18:40:04.792535+0000","last_active":"2023-07-21T18:41:08.149558+0000","last_peered":"2023-07-21T18:41:08.149558+0000","last_clean":"2023-07-21T18:41:08.149558+0000","last_became_active":"2023-07-21T18:40:04.792239+0000","last_became_peered":"2023-07-21T18:40:04.792239+0000","last_unstale":"2023-07-21T18:41:08.149558+0000","last_undegraded":"2023-07-21T18:41:08.149558+0000","last_fullsized":"2023-07-21T18:41:08.149558+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:34:19.596250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"79'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.244642+0000","last_change":"2023-07-21T18:40:04.173539+0000","last_active":"2023-07-21T18:41:08.244642+0000","last_peered":"2023-07-21T18:41:08.244642+0000","last_clean":"2023-07-21T18:41:08.244642+0000","last_became_active":"2023-07-21T18:40:04.173225+0000","last_became_peered":"2023-07-21T18:40:04.173225+0000","last_unstale":"2023-07-21T18:41:08.244642+0000","last_undegraded":"2023-07-21T18:41:08.244642+0000","last_fullsized":"2023-07-21T18:41:08.244642+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:14:31.670842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":341,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.595315+0000","last_change":"2023-07-21T18:39:05.999983+0000","last_active":"2023-07-21T18:41:08.595315+0000","last_peered":"2023-07-21T18:41:08.595315+0000","last_clean":"2023-07-21T18:41:08.595315+0000","last_became_active":"2023-07-21T18:39:05.999495+0000","last_became_peered":"2023-07-21T18:39:05.999495+0000","last_unstale":"2023-07-21T18:41:08.595315+0000","last_undegraded":"2023-07-21T18:41:08.595315+0000","last_fullsized":"2023-07-21T18:41:08.595315+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:36:14.004429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:36:14.004429+0000","last_clean_scrub_stamp":"2023-07-21T18:36:14.004429+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:13:55.308992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.143732+0000","last_change":"2023-07-21T18:40:04.785086+0000","last_active":"2023-07-21T18:41:08.143732+0000","last_peered":"2023-07-21T18:41:08.143732+0000","last_clean":"2023-07-21T18:41:08.143732+0000","last_became_active":"2023-07-21T18:40:04.784693+0000","last_became_peered":"2023-07-21T18:40:04.784693+0000","last_unstale":"2023-07-21T18:41:08.143732+0000","last_undegraded":"2023-07-21T18:41:08.143732+0000","last_fullsized":"2023-07-21T18:41:08.143732+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:20:44.967039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"79'4","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.595119+0000","last_change":"2023-07-21T18:40:04.177032+0000","last_active":"2023-07-21T18:41:08.595119+0000","last_peered":"2023-07-21T18:41:08.595119+0000","last_clean":"2023-07-21T18:41:08.595119+0000","last_became_active":"2023-07-21T18:40:04.176741+0000","last_became_peered":"2023-07-21T18:40:04.176741+0000","last_unstale":"2023-07-21T18:41:08.595119+0000","last_undegraded":"2023-07-21T18:41:08.595119+0000","last_fullsized":"2023-07-21T18:41:08.595119+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:30:00.615431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"79'4","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.148885+0000","last_change":"2023-07-21T18:40:04.180106+0000","last_active":"2023-07-21T18:41:08.148885+0000","last_peered":"2023-07-21T18:41:08.148885+0000","last_clean":"2023-07-21T18:41:08.148885+0000","last_became_active":"2023-07-21T18:40:04.179802+0000","last_became_peered":"2023-07-21T18:40:04.179802+0000","last_unstale":"2023-07-21T18:41:08.148885+0000","last_undegraded":"2023-07-21T18:41:08.148885+0000","last_fullsized":"2023-07-21T18:41:08.148885+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:58:17.009228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.144906+0000","last_change":"2023-07-21T18:40:04.789127+0000","last_active":"2023-07-21T18:41:08.144906+0000","last_peered":"2023-07-21T18:41:08.144906+0000","last_clean":"2023-07-21T18:41:08.144906+0000","last_became_active":"2023-07-21T18:40:04.788815+0000","last_became_peered":"2023-07-21T18:40:04.788815+0000","last_unstale":"2023-07-21T18:41:08.144906+0000","last_undegraded":"2023-07-21T18:41:08.144906+0000","last_fullsized":"2023-07-21T18:41:08.144906+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:31:07.061759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"79'9","reported_seq":43,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.156947+0000","last_change":"2023-07-21T18:41:08.156947+0000","last_active":"2023-07-21T18:41:08.156947+0000","last_peered":"2023-07-21T18:41:08.156947+0000","last_clean":"2023-07-21T18:41:08.156947+0000","last_became_active":"2023-07-21T18:41:08.149966+0000","last_became_peered":"2023-07-21T18:41:08.149966+0000","last_unstale":"2023-07-21T18:41:08.156947+0000","last_undegraded":"2023-07-21T18:41:08.156947+0000","last_fullsized":"2023-07-21T18:41:08.156947+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:17:05.137573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.149731+0000","last_change":"2023-07-21T18:40:04.791937+0000","last_active":"2023-07-21T18:41:08.149731+0000","last_peered":"2023-07-21T18:41:08.149731+0000","last_clean":"2023-07-21T18:41:08.149731+0000","last_became_active":"2023-07-21T18:40:04.791588+0000","last_became_peered":"2023-07-21T18:40:04.791588+0000","last_unstale":"2023-07-21T18:41:08.149731+0000","last_undegraded":"2023-07-21T18:41:08.149731+0000","last_fullsized":"2023-07-21T18:41:08.149731+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:17:39.327055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'7","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.244822+0000","last_change":"2023-07-21T18:40:04.177007+0000","last_active":"2023-07-21T18:41:08.244822+0000","last_peered":"2023-07-21T18:41:08.244822+0000","last_clean":"2023-07-21T18:41:08.244822+0000","last_became_active":"2023-07-21T18:40:04.176699+0000","last_became_peered":"2023-07-21T18:40:04.176699+0000","last_unstale":"2023-07-21T18:41:08.244822+0000","last_undegraded":"2023-07-21T18:41:08.244822+0000","last_fullsized":"2023-07-21T18:41:08.244822+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:20:14.547927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.155110+0000","last_change":"2023-07-21T18:40:04.795671+0000","last_active":"2023-07-21T18:41:08.155110+0000","last_peered":"2023-07-21T18:41:08.155110+0000","last_clean":"2023-07-21T18:41:08.155110+0000","last_became_active":"2023-07-21T18:40:04.795233+0000","last_became_peered":"2023-07-21T18:40:04.795233+0000","last_unstale":"2023-07-21T18:41:08.155110+0000","last_undegraded":"2023-07-21T18:41:08.155110+0000","last_fullsized":"2023-07-21T18:41:08.155110+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:57:42.725756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.159479+0000","last_change":"2023-07-21T18:41:08.159479+0000","last_active":"2023-07-21T18:41:08.159479+0000","last_peered":"2023-07-21T18:41:08.159479+0000","last_clean":"2023-07-21T18:41:08.159479+0000","last_became_active":"2023-07-21T18:41:08.156027+0000","last_became_peered":"2023-07-21T18:41:08.156027+0000","last_unstale":"2023-07-21T18:41:08.159479+0000","last_undegraded":"2023-07-21T18:41:08.159479+0000","last_fullsized":"2023-07-21T18:41:08.159479+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:36:55.454682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.148310+0000","last_change":"2023-07-21T18:40:04.793091+0000","last_active":"2023-07-21T18:41:08.148310+0000","last_peered":"2023-07-21T18:41:08.148310+0000","last_clean":"2023-07-21T18:41:08.148310+0000","last_became_active":"2023-07-21T18:40:04.792723+0000","last_became_peered":"2023-07-21T18:40:04.792723+0000","last_unstale":"2023-07-21T18:41:08.148310+0000","last_undegraded":"2023-07-21T18:41:08.148310+0000","last_fullsized":"2023-07-21T18:41:08.148310+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:25:52.341093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'7","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.149262+0000","last_change":"2023-07-21T18:40:04.182867+0000","last_active":"2023-07-21T18:41:08.149262+0000","last_peered":"2023-07-21T18:41:08.149262+0000","last_clean":"2023-07-21T18:41:08.149262+0000","last_became_active":"2023-07-21T18:40:04.181885+0000","last_became_peered":"2023-07-21T18:40:04.181885+0000","last_unstale":"2023-07-21T18:41:08.149262+0000","last_undegraded":"2023-07-21T18:41:08.149262+0000","last_fullsized":"2023-07-21T18:41:08.149262+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:44:58.697667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.143574+0000","last_change":"2023-07-21T18:40:04.790340+0000","last_active":"2023-07-21T18:41:08.143574+0000","last_peered":"2023-07-21T18:41:08.143574+0000","last_clean":"2023-07-21T18:41:08.143574+0000","last_became_active":"2023-07-21T18:40:04.790014+0000","last_became_peered":"2023-07-21T18:40:04.790014+0000","last_unstale":"2023-07-21T18:41:08.143574+0000","last_undegraded":"2023-07-21T18:41:08.143574+0000","last_fullsized":"2023-07-21T18:41:08.143574+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:01:01.514954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'3","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.152586+0000","last_change":"2023-07-21T18:40:04.184986+0000","last_active":"2023-07-21T18:41:08.152586+0000","last_peered":"2023-07-21T18:41:08.152586+0000","last_clean":"2023-07-21T18:41:08.152586+0000","last_became_active":"2023-07-21T18:40:04.184672+0000","last_became_peered":"2023-07-21T18:40:04.184672+0000","last_unstale":"2023-07-21T18:41:08.152586+0000","last_undegraded":"2023-07-21T18:41:08.152586+0000","last_fullsized":"2023-07-21T18:41:08.152586+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:56:51.290795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.149573+0000","last_change":"2023-07-21T18:40:04.793272+0000","last_active":"2023-07-21T18:41:08.149573+0000","last_peered":"2023-07-21T18:41:08.149573+0000","last_clean":"2023-07-21T18:41:08.149573+0000","last_became_active":"2023-07-21T18:40:04.792854+0000","last_became_peered":"2023-07-21T18:40:04.792854+0000","last_unstale":"2023-07-21T18:41:08.149573+0000","last_undegraded":"2023-07-21T18:41:08.149573+0000","last_fullsized":"2023-07-21T18:41:08.149573+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:13:19.927149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.148619+0000","last_change":"2023-07-21T18:40:04.176404+0000","last_active":"2023-07-21T18:41:08.148619+0000","last_peered":"2023-07-21T18:41:08.148619+0000","last_clean":"2023-07-21T18:41:08.148619+0000","last_became_active":"2023-07-21T18:40:04.176148+0000","last_became_peered":"2023-07-21T18:40:04.176148+0000","last_unstale":"2023-07-21T18:41:08.148619+0000","last_undegraded":"2023-07-21T18:41:08.148619+0000","last_fullsized":"2023-07-21T18:41:08.148619+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:06:45.638197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.595565+0000","last_change":"2023-07-21T18:40:04.795628+0000","last_active":"2023-07-21T18:41:08.595565+0000","last_peered":"2023-07-21T18:41:08.595565+0000","last_clean":"2023-07-21T18:41:08.595565+0000","last_became_active":"2023-07-21T18:40:04.795248+0000","last_became_peered":"2023-07-21T18:40:04.795248+0000","last_unstale":"2023-07-21T18:41:08.595565+0000","last_undegraded":"2023-07-21T18:41:08.595565+0000","last_fullsized":"2023-07-21T18:41:08.595565+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:54:09.726387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'5","reported_seq":103,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:27.081526+0000","last_change":"2023-07-21T18:40:04.184380+0000","last_active":"2023-07-21T18:41:27.081526+0000","last_peered":"2023-07-21T18:41:27.081526+0000","last_clean":"2023-07-21T18:41:27.081526+0000","last_became_active":"2023-07-21T18:40:04.184045+0000","last_became_peered":"2023-07-21T18:40:04.184045+0000","last_unstale":"2023-07-21T18:41:27.081526+0000","last_undegraded":"2023-07-21T18:41:27.081526+0000","last_fullsized":"2023-07-21T18:41:27.081526+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:33:19.018900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":80,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.595539+0000","last_change":"2023-07-21T18:40:04.787760+0000","last_active":"2023-07-21T18:41:08.595539+0000","last_peered":"2023-07-21T18:41:08.595539+0000","last_clean":"2023-07-21T18:41:08.595539+0000","last_became_active":"2023-07-21T18:40:04.787371+0000","last_became_peered":"2023-07-21T18:40:04.787371+0000","last_unstale":"2023-07-21T18:41:08.595539+0000","last_undegraded":"2023-07-21T18:41:08.595539+0000","last_fullsized":"2023-07-21T18:41:08.595539+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:24:52.752764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'3","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.248814+0000","last_change":"2023-07-21T18:40:04.184721+0000","last_active":"2023-07-21T18:41:08.248814+0000","last_peered":"2023-07-21T18:41:08.248814+0000","last_clean":"2023-07-21T18:41:08.248814+0000","last_became_active":"2023-07-21T18:40:04.184070+0000","last_became_peered":"2023-07-21T18:40:04.184070+0000","last_unstale":"2023-07-21T18:41:08.248814+0000","last_undegraded":"2023-07-21T18:41:08.248814+0000","last_fullsized":"2023-07-21T18:41:08.248814+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:57:29.552146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"79'1","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.595437+0000","last_change":"2023-07-21T18:40:04.786963+0000","last_active":"2023-07-21T18:41:08.595437+0000","last_peered":"2023-07-21T18:41:08.595437+0000","last_clean":"2023-07-21T18:41:08.595437+0000","last_became_active":"2023-07-21T18:40:04.786617+0000","last_became_peered":"2023-07-21T18:40:04.786617+0000","last_unstale":"2023-07-21T18:41:08.595437+0000","last_undegraded":"2023-07-21T18:41:08.595437+0000","last_fullsized":"2023-07-21T18:41:08.595437+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:28:47.943499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'5","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.144548+0000","last_change":"2023-07-21T18:40:04.184509+0000","last_active":"2023-07-21T18:41:08.144548+0000","last_peered":"2023-07-21T18:41:08.144548+0000","last_clean":"2023-07-21T18:41:08.144548+0000","last_became_active":"2023-07-21T18:40:04.184208+0000","last_became_peered":"2023-07-21T18:40:04.184208+0000","last_unstale":"2023-07-21T18:41:08.144548+0000","last_undegraded":"2023-07-21T18:41:08.144548+0000","last_fullsized":"2023-07-21T18:41:08.144548+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:21:46.199399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.595579+0000","last_change":"2023-07-21T18:40:04.788248+0000","last_active":"2023-07-21T18:41:08.595579+0000","last_peered":"2023-07-21T18:41:08.595579+0000","last_clean":"2023-07-21T18:41:08.595579+0000","last_became_active":"2023-07-21T18:40:04.787974+0000","last_became_peered":"2023-07-21T18:40:04.787974+0000","last_unstale":"2023-07-21T18:41:08.595579+0000","last_undegraded":"2023-07-21T18:41:08.595579+0000","last_fullsized":"2023-07-21T18:41:08.595579+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:28:27.418555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.143620+0000","last_change":"2023-07-21T18:40:04.790522+0000","last_active":"2023-07-21T18:41:08.143620+0000","last_peered":"2023-07-21T18:41:08.143620+0000","last_clean":"2023-07-21T18:41:08.143620+0000","last_became_active":"2023-07-21T18:40:04.790178+0000","last_became_peered":"2023-07-21T18:40:04.790178+0000","last_unstale":"2023-07-21T18:41:08.143620+0000","last_undegraded":"2023-07-21T18:41:08.143620+0000","last_fullsized":"2023-07-21T18:41:08.143620+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:06:13.808345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.152539+0000","last_change":"2023-07-21T18:40:04.790403+0000","last_active":"2023-07-21T18:41:08.152539+0000","last_peered":"2023-07-21T18:41:08.152539+0000","last_clean":"2023-07-21T18:41:08.152539+0000","last_became_active":"2023-07-21T18:40:04.790106+0000","last_became_peered":"2023-07-21T18:40:04.790106+0000","last_unstale":"2023-07-21T18:41:08.152539+0000","last_undegraded":"2023-07-21T18:41:08.152539+0000","last_fullsized":"2023-07-21T18:41:08.152539+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:04:45.154675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.152548+0000","last_change":"2023-07-21T18:40:04.795230+0000","last_active":"2023-07-21T18:41:08.152548+0000","last_peered":"2023-07-21T18:41:08.152548+0000","last_clean":"2023-07-21T18:41:08.152548+0000","last_became_active":"2023-07-21T18:40:04.794949+0000","last_became_peered":"2023-07-21T18:40:04.794949+0000","last_unstale":"2023-07-21T18:41:08.152548+0000","last_undegraded":"2023-07-21T18:41:08.152548+0000","last_fullsized":"2023-07-21T18:41:08.152548+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T18:42:31.029744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.154417+0000","last_change":"2023-07-21T18:40:04.792736+0000","last_active":"2023-07-21T18:41:08.154417+0000","last_peered":"2023-07-21T18:41:08.154417+0000","last_clean":"2023-07-21T18:41:08.154417+0000","last_became_active":"2023-07-21T18:40:04.792301+0000","last_became_peered":"2023-07-21T18:40:04.792301+0000","last_unstale":"2023-07-21T18:41:08.154417+0000","last_undegraded":"2023-07-21T18:41:08.154417+0000","last_fullsized":"2023-07-21T18:41:08.154417+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:45:51.320918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.145009+0000","last_change":"2023-07-21T18:40:04.790733+0000","last_active":"2023-07-21T18:41:08.145009+0000","last_peered":"2023-07-21T18:41:08.145009+0000","last_clean":"2023-07-21T18:41:08.145009+0000","last_became_active":"2023-07-21T18:40:04.790441+0000","last_became_peered":"2023-07-21T18:40:04.790441+0000","last_unstale":"2023-07-21T18:41:08.145009+0000","last_undegraded":"2023-07-21T18:41:08.145009+0000","last_fullsized":"2023-07-21T18:41:08.145009+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:40:00.802774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.143768+0000","last_change":"2023-07-21T18:40:04.790853+0000","last_active":"2023-07-21T18:41:08.143768+0000","last_peered":"2023-07-21T18:41:08.143768+0000","last_clean":"2023-07-21T18:41:08.143768+0000","last_became_active":"2023-07-21T18:40:04.790373+0000","last_became_peered":"2023-07-21T18:40:04.790373+0000","last_unstale":"2023-07-21T18:41:08.143768+0000","last_undegraded":"2023-07-21T18:41:08.143768+0000","last_fullsized":"2023-07-21T18:41:08.143768+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:29:27.642249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.153055+0000","last_change":"2023-07-21T18:40:04.794088+0000","last_active":"2023-07-21T18:41:08.153055+0000","last_peered":"2023-07-21T18:41:08.153055+0000","last_clean":"2023-07-21T18:41:08.153055+0000","last_became_active":"2023-07-21T18:40:04.793782+0000","last_became_peered":"2023-07-21T18:40:04.793782+0000","last_unstale":"2023-07-21T18:41:08.153055+0000","last_undegraded":"2023-07-21T18:41:08.153055+0000","last_fullsized":"2023-07-21T18:41:08.153055+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:10:46.116604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.149451+0000","last_change":"2023-07-21T18:40:04.791772+0000","last_active":"2023-07-21T18:41:08.149451+0000","last_peered":"2023-07-21T18:41:08.149451+0000","last_clean":"2023-07-21T18:41:08.149451+0000","last_became_active":"2023-07-21T18:40:04.791379+0000","last_became_peered":"2023-07-21T18:40:04.791379+0000","last_unstale":"2023-07-21T18:41:08.149451+0000","last_undegraded":"2023-07-21T18:41:08.149451+0000","last_fullsized":"2023-07-21T18:41:08.149451+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:01:22.491116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.248658+0000","last_change":"2023-07-21T18:40:04.786579+0000","last_active":"2023-07-21T18:41:08.248658+0000","last_peered":"2023-07-21T18:41:08.248658+0000","last_clean":"2023-07-21T18:41:08.248658+0000","last_became_active":"2023-07-21T18:40:04.786032+0000","last_became_peered":"2023-07-21T18:40:04.786032+0000","last_unstale":"2023-07-21T18:41:08.248658+0000","last_undegraded":"2023-07-21T18:41:08.248658+0000","last_fullsized":"2023-07-21T18:41:08.248658+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:26:58.238429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.149279+0000","last_change":"2023-07-21T18:40:04.793441+0000","last_active":"2023-07-21T18:41:08.149279+0000","last_peered":"2023-07-21T18:41:08.149279+0000","last_clean":"2023-07-21T18:41:08.149279+0000","last_became_active":"2023-07-21T18:40:04.791441+0000","last_became_peered":"2023-07-21T18:40:04.791441+0000","last_unstale":"2023-07-21T18:41:08.149279+0000","last_undegraded":"2023-07-21T18:41:08.149279+0000","last_fullsized":"2023-07-21T18:41:08.149279+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T18:47:16.762634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.151030+0000","last_change":"2023-07-21T18:40:04.794913+0000","last_active":"2023-07-21T18:41:08.151030+0000","last_peered":"2023-07-21T18:41:08.151030+0000","last_clean":"2023-07-21T18:41:08.151030+0000","last_became_active":"2023-07-21T18:40:04.794527+0000","last_became_peered":"2023-07-21T18:40:04.794527+0000","last_unstale":"2023-07-21T18:41:08.151030+0000","last_undegraded":"2023-07-21T18:41:08.151030+0000","last_fullsized":"2023-07-21T18:41:08.151030+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:14:55.276602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.144282+0000","last_change":"2023-07-21T18:40:04.789145+0000","last_active":"2023-07-21T18:41:08.144282+0000","last_peered":"2023-07-21T18:41:08.144282+0000","last_clean":"2023-07-21T18:41:08.144282+0000","last_became_active":"2023-07-21T18:40:04.788285+0000","last_became_peered":"2023-07-21T18:40:04.788285+0000","last_unstale":"2023-07-21T18:41:08.144282+0000","last_undegraded":"2023-07-21T18:41:08.144282+0000","last_fullsized":"2023-07-21T18:41:08.144282+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:50:38.851180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.143670+0000","last_change":"2023-07-21T18:40:04.788821+0000","last_active":"2023-07-21T18:41:08.143670+0000","last_peered":"2023-07-21T18:41:08.143670+0000","last_clean":"2023-07-21T18:41:08.143670+0000","last_became_active":"2023-07-21T18:40:04.788513+0000","last_became_peered":"2023-07-21T18:40:04.788513+0000","last_unstale":"2023-07-21T18:41:08.143670+0000","last_undegraded":"2023-07-21T18:41:08.143670+0000","last_fullsized":"2023-07-21T18:41:08.143670+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:37:17.944763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64131,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":817,"num_read_kb":477,"num_write":233,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":191757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":204,"ondisk_log_size":204,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579933,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27816,"kb_used_data":864,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717336,"statfs":{"total":95995035648,"available":95966552064,"internally_reserved":0,"allocated":884736,"data_stored":426096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:41:24 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.54300000000000004,"15min":0.54700000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.78600000000000003,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.629,"15min":0.63900000000000001},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.748,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.54700000000000004}]},{"osd":1,"last update":"Fri Jul 21 18:41:24 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.63900000000000001,"15min":0.63300000000000001},"min":{"1min":0.50800000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.747},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.61699999999999999,"15min":0.61099999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.85899999999999999,"5min":1.137,"15min":1.137},"last":0.64000000000000001}]},{"osd":2,"last update":"Fri Jul 21 18:41:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.502,"15min":0.495},"min":{"1min":0.378,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.502,"5min":0.47299999999999998,"15min":0.46800000000000003},"min":{"1min":0.32600000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.64400000000000002,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.60299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":4,"last update":"Fri Jul 21 18:41:24 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.64300000000000002,"15min":0.63300000000000001},"min":{"1min":0.45700000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.60199999999999998,"15min":0.59099999999999997},"min":{"1min":0.46700000000000003,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.79300000000000004}]},{"osd":5,"last update":"Fri Jul 21 18:41:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.56799999999999995,"15min":0.56000000000000005},"min":{"1min":0.50600000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.57299999999999995,"15min":0.56599999999999995},"min":{"1min":0.39300000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.75700000000000001,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.81200000000000006}]},{"osd":6,"last update":"Fri Jul 21 18:41:24 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.64800000000000002,"15min":0.65100000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.88700000000000001,"5min":1.181,"15min":1.181},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.755,"15min":0.75900000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.92500000000000004,"5min":1.425,"15min":1.425},"last":0.88400000000000001}]},{"osd":7,"last update":"Fri Jul 21 18:41:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.72799999999999998,"15min":0.71999999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.91200000000000003,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.752,"5min":0.70999999999999996,"15min":0.70299999999999996},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.91500000000000004,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.98099999999999998}]},{"osd":8,"last update":"Fri Jul 21 18:41:06 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.999}]},{"osd":9,"last update":"Fri Jul 21 18:41:24 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.80500000000000005,"15min":0.80200000000000005},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.98799999999999999,"5min":1.643,"15min":1.643},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.78800000000000003,"15min":0.78400000000000003},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.95899999999999996,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":1.0269999999999999}]},{"osd":10,"last update":"Fri Jul 21 18:41:24 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.76900000000000002,"15min":0.76200000000000001},"min":{"1min":0.56000000000000005,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.95899999999999996,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.56999999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.98099999999999998,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.96499999999999997}]}]},{"osd":10,"up_from":67,"seq":287762808864,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28492,"kb_used_data":1540,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716660,"statfs":{"total":95995035648,"available":95965859840,"internally_reserved":0,"allocated":1576960,"data_stored":1074383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:41:08 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.40999999999999998}]},{"osd":1,"last update":"Fri Jul 21 18:41:08 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.46800000000000003,"15min":0.46600000000000003},"min":{"1min":0.29399999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.68600000000000005,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.46500000000000002,"15min":0.46100000000000002},"min":{"1min":0.33200000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.624,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.49099999999999999}]},{"osd":2,"last update":"Fri Jul 21 18:41:08 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.60699999999999998,"15min":0.56200000000000006},"min":{"1min":0.54100000000000004,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":4.0129999999999999,"5min":4.0129999999999999,"15min":4.0129999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.622,"15min":0.60999999999999999},"min":{"1min":0.45600000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.82799999999999996,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.59199999999999997}]},{"osd":3,"last update":"Fri Jul 21 18:41:08 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.60499999999999998,"15min":0.60099999999999998},"min":{"1min":0.434,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.746,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.621,"5min":0.60499999999999998,"15min":0.60199999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.439}]},{"osd":4,"last update":"Fri Jul 21 18:41:08 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.57699999999999996,"15min":0.57199999999999995},"min":{"1min":0.42599999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.73099999999999998,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.59099999999999997,"15min":0.59299999999999997},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.84699999999999998,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.52600000000000002}]},{"osd":5,"last update":"Fri Jul 21 18:41:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.69299999999999995,"15min":0.68700000000000006},"min":{"1min":0.49399999999999999,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.69699999999999995,"15min":0.68300000000000005},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.621}]},{"osd":6,"last update":"Fri Jul 21 18:41:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.69299999999999995,"15min":0.68899999999999995},"min":{"1min":0.505,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.874,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.46999999999999997,"5min":0.442,"15min":0.442},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.60499999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":8,"last update":"Fri Jul 21 18:41:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.73699999999999999,"15min":0.72899999999999998},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.65700000000000003},{"interface":"front","average":{"1min":1.01,"5min":0.81100000000000005,"15min":0.77800000000000002},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":4.0419999999999998,"5min":4.0419999999999998,"15min":4.0419999999999998},"last":0.72299999999999998}]},{"osd":9,"last update":"Fri Jul 21 18:41:08 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.75900000000000001,"15min":0.75700000000000001},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.75,"5min":0.77800000000000002,"15min":0.78200000000000003},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.92100000000000004,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.68899999999999995}]},{"osd":11,"last update":"Fri Jul 21 18:40:28 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.75}]}]},{"osd":9,"up_from":62,"seq":266287972389,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27840,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717312,"statfs":{"total":95995035648,"available":95966527488,"internally_reserved":0,"allocated":909312,"data_stored":426074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.52500000000000002,"15min":0.53900000000000003},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.996,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.438,"5min":0.505,"15min":0.51600000000000001},"min":{"1min":0.316,"5min":0.252,"15min":0.252},"max":{"1min":0.621,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.33100000000000002}]},{"osd":1,"last update":"Fri Jul 21 18:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.73199999999999998,"15min":0.749},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.099,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":0.627},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.71299999999999997,"15min":0.72099999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.169,"5min":1.681,"15min":1.681},"last":0.59899999999999998}]},{"osd":2,"last update":"Fri Jul 21 18:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.70799999999999996,"15min":0.71399999999999997},"min":{"1min":0.47299999999999998,"5min":0.38,"15min":0.38},"max":{"1min":1.222,"5min":1.9530000000000001,"15min":1.9530000000000001},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.78300000000000003,"15min":0.753},"min":{"1min":0.46100000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":6.4020000000000001,"5min":6.4020000000000001,"15min":6.4020000000000001},"last":0.63900000000000001}]},{"osd":3,"last update":"Fri Jul 21 18:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.627,"15min":0.64900000000000002},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.89000000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.627,"15min":0.64500000000000002},"min":{"1min":0.35199999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.1910000000000001,"5min":2.048,"15min":2.048},"last":0.38}]},{"osd":4,"last update":"Fri Jul 21 18:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.80700000000000005,"15min":0.83699999999999997},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.127,"5min":1.8100000000000001,"15min":1.8100000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.79800000000000004,"15min":0.82299999999999995},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.94799999999999995,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.748}]},{"osd":5,"last update":"Fri Jul 21 18:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.80500000000000005,"15min":0.77700000000000002},"min":{"1min":0.49299999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":6.5069999999999997,"5min":6.5069999999999997,"15min":6.5069999999999997},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.78600000000000003,"15min":0.80400000000000005},"min":{"1min":0.503,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.1399999999999999,"5min":1.992,"15min":1.992},"last":0.56299999999999994}]},{"osd":6,"last update":"Fri Jul 21 18:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.74299999999999999,"15min":0.77300000000000002},"min":{"1min":0.375,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.1240000000000001,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.78600000000000003,"15min":0.81399999999999995},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.2410000000000001,"5min":1.595,"15min":1.595},"last":0.77900000000000003}]},{"osd":7,"last update":"Fri Jul 21 18:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.85799999999999998,"15min":0.88200000000000001},"min":{"1min":0.54700000000000004,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.026,"5min":1.284,"15min":1.284},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.89300000000000002,"15min":0.92000000000000004},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.175,"5min":2.0750000000000002,"15min":2.0750000000000002},"last":0.60999999999999999}]},{"osd":8,"last update":"Fri Jul 21 18:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.90700000000000003,"15min":0.92900000000000005},"min":{"1min":0.61499999999999999,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.2749999999999999,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":1.0669999999999999,"5min":0.90600000000000003,"15min":0.879},"min":{"1min":0.60899999999999999,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":6.5679999999999996,"5min":6.5679999999999996,"15min":6.5679999999999996},"last":0.79900000000000004}]},{"osd":10,"last update":"Fri Jul 21 18:41:08 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.86699999999999999,"15min":0.88700000000000001},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.155,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.86399999999999999,"15min":0.88200000000000001},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":0.76500000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78900000000000003}]}]},{"osd":8,"up_from":55,"seq":236223201322,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27832,"kb_used_data":880,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717320,"statfs":{"total":95995035648,"available":95966535680,"internally_reserved":0,"allocated":901120,"data_stored":426121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:41:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.36299999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.81899999999999995,"5min":4.2000000000000002,"15min":4.2000000000000002},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.58699999999999997,"15min":0.58399999999999996},"min":{"1min":0.33200000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.747,"5min":3.145,"15min":3.145},"last":0.50800000000000001}]},{"osd":1,"last update":"Fri Jul 21 18:41:27 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.622,"15min":0.59199999999999997},"min":{"1min":0.45500000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.91300000000000003,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.627,"15min":0.61699999999999999},"min":{"1min":0.49399999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.92600000000000005,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.87}]},{"osd":2,"last update":"Fri Jul 21 18:41:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.47999999999999998,"15min":0.46500000000000002},"min":{"1min":0.39400000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.48699999999999999,"15min":0.47099999999999997},"min":{"1min":0.34399999999999997,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.77700000000000002,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.85199999999999998}]},{"osd":3,"last update":"Fri Jul 21 18:41:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.72799999999999998,"15min":0.68300000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.85099999999999998,"5min":11.260999999999999,"15min":11.260999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.61799999999999999,"15min":0.62},"min":{"1min":0.441,"5min":0.371,"15min":0.371},"max":{"1min":0.72499999999999998,"5min":3.2719999999999998,"15min":3.2719999999999998},"last":0.56200000000000006}]},{"osd":4,"last update":"Fri Jul 21 18:41:27 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.58399999999999996,"15min":0.56999999999999995},"min":{"1min":0.46899999999999997,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.80000000000000004,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.64600000000000002,"15min":0.63400000000000001},"min":{"1min":0.54600000000000004,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.86299999999999999,"5min":0.875,"15min":0.875},"last":0.82699999999999996}]},{"osd":5,"last update":"Fri Jul 21 18:41:27 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.59399999999999997,"15min":0.58699999999999997},"min":{"1min":0.308,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.53900000000000003,"15min":0.53500000000000003},"min":{"1min":0.36599999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.70299999999999996}]},{"osd":6,"last update":"Fri Jul 21 18:41:27 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.78400000000000003,"15min":0.73699999999999999},"min":{"1min":0.47299999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.98099999999999998,"5min":10.292999999999999,"15min":10.292999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.76600000000000001},"min":{"1min":0.53600000000000003,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.018,"5min":4.1029999999999998,"15min":4.1029999999999998},"last":0.90300000000000002}]},{"osd":7,"last update":"Fri Jul 21 18:41:27 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.73099999999999998,"15min":0.72099999999999997},"min":{"1min":0.56699999999999995,"5min":0.432,"15min":0.432},"max":{"1min":1.0029999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.67900000000000005,"15min":0.67500000000000004},"min":{"1min":0.497,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.88700000000000001}]},{"osd":9,"last update":"Fri Jul 21 18:40:46 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.54100000000000004,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.95199999999999996,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.92000000000000004},{"interface":"front","average":{"1min":1.2509999999999999,"5min":0.86699999999999999,"15min":0.80300000000000005},"min":{"1min":0.54900000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":11.151,"5min":11.151,"15min":11.151},"last":0.98299999999999998}]},{"osd":10,"last update":"Fri Jul 21 18:41:05 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.71599999999999997,"15min":0.69799999999999995},"min":{"1min":0.59899999999999998,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.996},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.73499999999999999,"15min":0.71399999999999997},"min":{"1min":0.73599999999999999,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":1.0089999999999999}]},{"osd":11,"last update":"Fri Jul 21 18:41:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.96199999999999997}]}]},{"osd":7,"up_from":48,"seq":206158430254,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28416,"kb_used_data":1464,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716736,"statfs":{"total":95995035648,"available":95965937664,"internally_reserved":0,"allocated":1499136,"data_stored":1018876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.53400000000000003,"15min":0.52600000000000002},"min":{"1min":0.41599999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.53900000000000003,"15min":0.53600000000000003},"min":{"1min":0.34000000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.48899999999999999}]},{"osd":1,"last update":"Fri Jul 21 18:41:03 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.42099999999999999,"15min":0.40699999999999997},"min":{"1min":0.29199999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.70799999999999996,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.442,"15min":0.42599999999999999},"min":{"1min":0.32400000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.66700000000000004,"5min":0.75,"15min":0.75},"last":0.379}]},{"osd":2,"last update":"Fri Jul 21 18:41:03 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.59299999999999997,"15min":0.57699999999999996},"min":{"1min":0.41899999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.89700000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.60799999999999998,"15min":0.59599999999999997},"min":{"1min":0.47999999999999998,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.83299999999999996,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.73699999999999999}]},{"osd":3,"last update":"Fri Jul 21 18:41:03 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.55900000000000005,"15min":0.54300000000000004},"min":{"1min":0.42599999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.70499999999999996,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.59099999999999997,"15min":0.57499999999999996},"min":{"1min":0.433,"5min":0.372,"15min":0.372},"max":{"1min":0.81200000000000006,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.63100000000000001}]},{"osd":4,"last update":"Fri Jul 21 18:41:03 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53000000000000003,"15min":0.52200000000000002},"min":{"1min":0.39300000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.73499999999999999,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.51900000000000002,"15min":0.51500000000000001},"min":{"1min":0.33800000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.60999999999999999,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.40200000000000002}]},{"osd":5,"last update":"Fri Jul 21 18:41:03 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.66400000000000003,"15min":0.64600000000000002},"min":{"1min":0.51700000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.65700000000000003,"15min":0.64600000000000002},"min":{"1min":0.505,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.56200000000000006}]},{"osd":6,"last update":"Fri Jul 21 18:41:03 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66800000000000004,"15min":0.65600000000000003},"min":{"1min":0.51600000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.86899999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.67700000000000005,"15min":0.66300000000000003},"min":{"1min":0.56299999999999994,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.97399999999999998,"5min":1.087,"15min":1.087},"last":0.59999999999999998}]},{"osd":8,"last update":"Fri Jul 21 18:41:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.70799999999999996,"15min":0.69499999999999995},"min":{"1min":0.64500000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.91400000000000003,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.72899999999999998,"15min":0.71899999999999997},"min":{"1min":0.57399999999999995,"5min":0.495,"15min":0.495},"max":{"1min":0.93799999999999994,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.78900000000000003}]},{"osd":9,"last update":"Fri Jul 21 18:40:46 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.74199999999999999,"15min":0.73499999999999999},"min":{"1min":0.46500000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.98099999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.753},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.74199999999999999,"15min":0.73599999999999999},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.83499999999999996}]},{"osd":10,"last update":"Fri Jul 21 18:41:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.75800000000000001,"15min":0.76200000000000001},"min":{"1min":0.50900000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.94899999999999995,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.746,"15min":0.74399999999999999},"min":{"1min":0.63,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":0.95399999999999996,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.68999999999999995}]},{"osd":11,"last update":"Fri Jul 21 18:41:04 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.77700000000000002}]}]},{"osd":6,"up_from":43,"seq":184683593778,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27820,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717332,"statfs":{"total":95995035648,"available":95966547968,"internally_reserved":0,"allocated":888832,"data_stored":425794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:40:40 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.41299999999999998,"15min":0.40899999999999997},"min":{"1min":0.29599999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.313},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.48099999999999998,"15min":0.44500000000000001},"min":{"1min":0.311,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":4.859,"5min":4.859,"15min":4.859},"last":0.37}]},{"osd":1,"last update":"Fri Jul 21 18:40:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.44500000000000001,"5min":0.37,"15min":0.37},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.625},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.39000000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.84399999999999997,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.56599999999999995}]},{"osd":2,"last update":"Fri Jul 21 18:40:40 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.64800000000000002,"15min":0.60399999999999998},"min":{"1min":0.39900000000000002,"5min":0.376,"15min":0.376},"max":{"1min":4.9180000000000001,"5min":4.9180000000000001,"15min":4.9180000000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.59599999999999997,"15min":0.58199999999999996},"min":{"1min":0.42099999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.502}]},{"osd":3,"last update":"Fri Jul 21 18:40:40 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.52300000000000002,"15min":0.48799999999999999},"min":{"1min":0.33800000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":4.4720000000000004,"5min":4.4720000000000004,"15min":4.4720000000000004},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.56999999999999995,"15min":0.53100000000000003},"min":{"1min":0.32200000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":5.2910000000000004,"5min":5.2910000000000004,"15min":5.2910000000000004},"last":0.40799999999999997}]},{"osd":4,"last update":"Fri Jul 21 18:40:40 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.63900000000000001,"15min":0.64400000000000002},"min":{"1min":0.40300000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.022,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.64100000000000001,"15min":0.64400000000000002},"min":{"1min":0.46500000000000002,"5min":0.442,"15min":0.442},"max":{"1min":0.89600000000000002,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.61199999999999999}]},{"osd":5,"last update":"Fri Jul 21 18:40:40 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.70699999999999996,"15min":0.66600000000000004},"min":{"1min":0.47499999999999998,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":4.9960000000000004,"5min":4.9960000000000004,"15min":4.9960000000000004},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.70299999999999996,"15min":0.69499999999999995},"min":{"1min":0.49299999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.145,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.53900000000000003}]},{"osd":7,"last update":"Fri Jul 21 18:41:03 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.70199999999999996,"15min":0.70499999999999996},"min":{"1min":0.495,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.874,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.75,"15min":0.71999999999999997},"min":{"1min":0.55200000000000005,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":5.0330000000000004,"5min":5.0330000000000004,"15min":5.0330000000000004},"last":0.69699999999999995}]},{"osd":8,"last update":"Fri Jul 21 18:40:26 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.79000000000000004,"15min":0.80600000000000005},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.96099999999999997,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.77500000000000002,"15min":0.78700000000000003},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.91800000000000004,"5min":1.171,"15min":1.171},"last":0.68000000000000005}]},{"osd":9,"last update":"Fri Jul 21 18:40:44 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.75,"15min":0.75600000000000001},"min":{"1min":0.56799999999999995,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.194,"5min":1.196,"15min":1.196},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.72599999999999998,"15min":0.73399999999999999},"min":{"1min":0.52600000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.872,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.71199999999999997}]},{"osd":10,"last update":"Fri Jul 21 18:41:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.78300000000000003,"15min":0.79100000000000004},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.86499999999999999,"5min":1.216,"15min":1.216},"last":0.749},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.80500000000000005,"15min":0.77900000000000003},"min":{"1min":0.60699999999999998,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":4.9580000000000002,"5min":4.9580000000000002,"15min":4.9580000000000002},"last":0.72199999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574919,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":924,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":946176,"data_stored":483258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:41:04 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.52300000000000002,"15min":0.48899999999999999},"min":{"1min":0.377,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.77100000000000002,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.56899999999999995,"15min":0.51100000000000001},"min":{"1min":0.38500000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":3.7690000000000001,"5min":3.7690000000000001,"15min":3.7690000000000001},"last":0.58799999999999997}]},{"osd":2,"last update":"Fri Jul 21 18:41:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.59299999999999997,"15min":0.53900000000000003},"min":{"1min":0.49399999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.85099999999999998,"5min":3.2959999999999998,"15min":3.2959999999999998},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.59399999999999997,"15min":0.54000000000000004},"min":{"1min":0.38200000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.85099999999999998,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":0.79800000000000004}]},{"osd":3,"last update":"Fri Jul 21 18:40:44 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.55300000000000005,"15min":0.52500000000000002},"min":{"1min":0.38400000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.58099999999999996,"15min":0.53700000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":3.7789999999999999,"5min":3.7789999999999999,"15min":3.7789999999999999},"last":0.622}]},{"osd":4,"last update":"Fri Jul 21 18:41:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.45100000000000001,"15min":0.41199999999999998},"min":{"1min":0.32400000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.44800000000000001,"15min":0.41499999999999998},"min":{"1min":0.28000000000000003,"5min":0.254,"15min":0.254},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.55900000000000005}]},{"osd":5,"last update":"Fri Jul 21 18:41:20 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.64800000000000002,"15min":0.60199999999999998},"min":{"1min":0.56399999999999995,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.97999999999999998,"5min":1.653,"15min":1.653},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.67100000000000004,"15min":0.61399999999999999},"min":{"1min":0.65700000000000003,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.95899999999999996,"5min":1.641,"15min":1.641},"last":0.85099999999999998}]},{"osd":6,"last update":"Fri Jul 21 18:40:44 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.64400000000000002,"15min":0.63200000000000001},"min":{"1min":0.46200000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.92900000000000005,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.64500000000000002,"15min":0.64000000000000001},"min":{"1min":0.41599999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.96099999999999997,"5min":1.022,"15min":1.022},"last":0.81699999999999995}]},{"osd":7,"last update":"Fri Jul 21 18:41:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.55600000000000005,"15min":0.51400000000000001},"min":{"1min":0.33200000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.749,"5min":0.59399999999999997,"15min":0.54500000000000004},"min":{"1min":0.35999999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.94699999999999995,"5min":1.014,"15min":1.014},"last":0.71999999999999997}]},{"osd":8,"last update":"Fri Jul 21 18:40:26 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.73599999999999999,"15min":0.72799999999999998},"min":{"1min":0.51500000000000001,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.1910000000000001,"5min":2.7360000000000002,"15min":2.7360000000000002},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.72699999999999998,"15min":0.72499999999999998},"min":{"1min":0.52400000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.96899999999999997,"5min":2.7210000000000001,"15min":2.7210000000000001},"last":0.77200000000000002}]},{"osd":9,"last update":"Fri Jul 21 18:40:44 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.745,"15min":0.73899999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.753,"15min":0.746},"min":{"1min":0.55100000000000005,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.93300000000000005}]},{"osd":10,"last update":"Fri Jul 21 18:41:04 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.75700000000000001,"15min":0.749},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.875,"5min":0.78500000000000003,"15min":0.77000000000000002},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.94999999999999996}]},{"osd":11,"last update":"Fri Jul 21 18:41:05 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.91300000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738443,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27844,"kb_used_data":892,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717308,"statfs":{"total":95995035648,"available":95966523392,"internally_reserved":0,"allocated":913408,"data_stored":427082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 21 18:41:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.59499999999999997,"15min":0.57499999999999996},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.875,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.504},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58999999999999997,"15min":0.54100000000000004},"min":{"1min":0.38100000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.82699999999999996,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.54900000000000004}]},{"osd":2,"last update":"Fri Jul 21 18:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.58999999999999997,"15min":0.56999999999999995},"min":{"1min":0.439,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.90300000000000002,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.61099999999999999,"15min":0.58699999999999997},"min":{"1min":0.45400000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.79300000000000004,"5min":3.0470000000000002,"15min":3.0470000000000002},"last":0.53400000000000003}]},{"osd":3,"last update":"Fri Jul 21 18:41:07 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.32100000000000001}]},{"osd":4,"last update":"Fri Jul 21 18:41:03 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.60199999999999998,"15min":0.56999999999999995},"min":{"1min":0.35799999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.77800000000000002,"5min":2.948,"15min":2.948},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.56299999999999994,"15min":0.54600000000000004},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.51400000000000001}]},{"osd":5,"last update":"Fri Jul 21 18:41:24 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.61499999999999999,"15min":0.59099999999999997},"min":{"1min":0.41599999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.94099999999999995,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.64700000000000002,"15min":0.61299999999999999},"min":{"1min":0.434,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.1419999999999999,"5min":3.0150000000000001,"15min":3.0150000000000001},"last":0.434}]},{"osd":6,"last update":"Fri Jul 21 18:40:46 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.52200000000000002,"15min":0.51100000000000001},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":2.077,"5min":2.077,"15min":2.077},"last":0.377},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.58799999999999997,"15min":0.56899999999999995},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":2.4500000000000002,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":0.56100000000000005}]},{"osd":7,"last update":"Fri Jul 21 18:39:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.47699999999999998}]},{"osd":8,"last update":"Fri Jul 21 18:41:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.76900000000000002,"15min":0.80600000000000005},"min":{"1min":0.54600000000000004,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.96299999999999997,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.78800000000000003,"15min":0.80800000000000005},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.1679999999999999,"5min":2.9830000000000001,"15min":2.9830000000000001},"last":0.61799999999999999}]},{"osd":9,"last update":"Fri Jul 21 18:40:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.71999999999999997,"15min":0.74199999999999999},"min":{"1min":0.37,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.78800000000000003,"15min":0.81299999999999994},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.421,"5min":1.421,"15min":1.421},"last":0.63800000000000001}]},{"osd":10,"last update":"Fri Jul 21 18:41:07 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.84899999999999998,"15min":0.88100000000000001},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.95599999999999996,"5min":2.9079999999999999,"15min":2.9079999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.78400000000000003,"15min":0.80400000000000005},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.96899999999999997,"5min":1.111,"15min":1.111},"last":0.59199999999999997}]},{"osd":11,"last update":"Fri Jul 21 18:40:26 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":3.0840000000000001,"5min":3.0840000000000001,"15min":3.0840000000000001},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.63}]}]},{"osd":2,"up_from":18,"seq":77309411395,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":970752,"data_stored":483703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:41:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.54700000000000004,"15min":0.53500000000000003},"min":{"1min":0.38100000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.92200000000000004,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.51600000000000001,"15min":0.50600000000000001},"min":{"1min":0.38,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.68600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.54600000000000004}]},{"osd":1,"last update":"Fri Jul 21 18:41:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.56200000000000006,"15min":0.52800000000000002},"min":{"1min":0.42999999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.59699999999999998,"15min":0.55000000000000004},"min":{"1min":0.435,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.91800000000000004,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.83299999999999996}]},{"osd":3,"last update":"Fri Jul 21 18:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.58899999999999997,"15min":0.56499999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.92400000000000004,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.56799999999999995,"15min":0.56000000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.79600000000000004,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.63900000000000001}]},{"osd":4,"last update":"Fri Jul 21 18:41:04 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.73899999999999999}]},{"osd":5,"last update":"Fri Jul 21 18:41:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.51300000000000001,"15min":0.47099999999999997},"min":{"1min":0.36899999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.89100000000000001,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.501,"15min":0.46700000000000003},"min":{"1min":0.39800000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.85999999999999999}]},{"osd":6,"last update":"Fri Jul 21 18:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.61899999999999999,"15min":0.59599999999999997},"min":{"1min":0.53000000000000003,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.85799999999999998,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.628,"15min":0.62},"min":{"1min":0.436,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.84399999999999997,"5min":0.873,"15min":0.873},"last":0.79700000000000004}]},{"osd":7,"last update":"Fri Jul 21 18:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.61699999999999999,"15min":0.60799999999999998},"min":{"1min":0.45100000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.577,"5min":1.577,"15min":1.577},"last":0.495},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.65500000000000003,"15min":0.65100000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.878,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.56499999999999995}]},{"osd":8,"last update":"Fri Jul 21 18:40:26 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.58899999999999997,"15min":0.57099999999999995},"min":{"1min":0.40799999999999997,"5min":0.376,"15min":0.376},"max":{"1min":0.91300000000000003,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.60799999999999998,"15min":0.59799999999999998},"min":{"1min":0.40799999999999997,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.95299999999999996}]},{"osd":9,"last update":"Fri Jul 21 18:40:46 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.76000000000000001,"15min":0.75600000000000001},"min":{"1min":0.59499999999999997,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.95999999999999996,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.998},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.69199999999999995,"15min":0.68999999999999995},"min":{"1min":0.47499999999999998,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.89600000000000002,"5min":1.01,"15min":1.01},"last":0.89900000000000002}]},{"osd":10,"last update":"Fri Jul 21 18:41:10 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.78200000000000003,"15min":0.77600000000000002},"min":{"1min":0.67500000000000004,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":0.91600000000000004,"5min":1.669,"15min":1.669},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.874,"5min":0.80300000000000005,"15min":0.79100000000000004},"min":{"1min":0.75600000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.97299999999999998,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":1.056}]},{"osd":11,"last update":"Fri Jul 21 18:41:04 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":1.034},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":1.0209999999999999}]}]},{"osd":3,"up_from":24,"seq":103079215166,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27832,"kb_used_data":880,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717320,"statfs":{"total":95995035648,"available":95966535680,"internally_reserved":0,"allocated":901120,"data_stored":425024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.35699999999999998,"15min":0.33100000000000002},"min":{"1min":0.26800000000000002,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.54000000000000004,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.373,"15min":0.35099999999999998},"min":{"1min":0.28100000000000003,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.51600000000000001,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.32000000000000001}]},{"osd":1,"last update":"Fri Jul 21 18:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.57199999999999995,"15min":0.55100000000000005},"min":{"1min":0.39900000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.75800000000000001,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59199999999999997,"15min":0.55000000000000004},"min":{"1min":0.39400000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.83099999999999996,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.70499999999999996}]},{"osd":2,"last update":"Fri Jul 21 18:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.57699999999999996,"15min":0.55300000000000005},"min":{"1min":0.36799999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.81599999999999995,"5min":0.876,"15min":0.876},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.56799999999999995,"15min":0.53900000000000003},"min":{"1min":0.373,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.57399999999999995}]},{"osd":4,"last update":"Fri Jul 21 18:41:05 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.61299999999999999,"15min":0.60399999999999998},"min":{"1min":0.443,"5min":0.374,"15min":0.374},"max":{"1min":0.76900000000000002,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.61799999999999999,"15min":0.59999999999999998},"min":{"1min":0.42599999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.86699999999999999,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.61299999999999999}]},{"osd":5,"last update":"Fri Jul 21 18:41:20 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.621,"15min":0.58999999999999997},"min":{"1min":0.42099999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.71299999999999997,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.59799999999999998,"15min":0.58199999999999996},"min":{"1min":0.434,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.70099999999999996,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.68799999999999994}]},{"osd":6,"last update":"Fri Jul 21 18:40:41 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.498,"15min":0.48599999999999999},"min":{"1min":0.33600000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.80900000000000005,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.55800000000000005,"15min":0.55400000000000005},"min":{"1min":0.36599999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.63400000000000001,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.51800000000000002}]},{"osd":7,"last update":"Fri Jul 21 18:41:05 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.65500000000000003,"15min":0.66600000000000004},"min":{"1min":0.5,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.84299999999999997,"5min":0.999,"15min":0.999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.72499999999999998,"15min":0.72199999999999998},"min":{"1min":0.53600000000000003,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.93600000000000005,"5min":1.181,"15min":1.181},"last":0.67600000000000005}]},{"osd":8,"last update":"Fri Jul 21 18:39:25 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.47299999999999998}]},{"osd":9,"last update":"Fri Jul 21 18:40:45 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.68600000000000005,"15min":0.71499999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.77900000000000003,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.629},{"interface":"front","average":{"1min":0.624,"5min":0.81999999999999995,"15min":0.85299999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.88100000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.748}]},{"osd":10,"last update":"Fri Jul 21 18:41:08 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.71799999999999997,"15min":0.72499999999999998},"min":{"1min":0.48999999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.80500000000000005,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.84199999999999997,"15min":0.85999999999999999},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":0.96699999999999997,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.76900000000000002}]},{"osd":11,"last update":"Fri Jul 21 18:41:05 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.79600000000000004}]}]},{"osd":4,"up_from":31,"seq":133143986234,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27820,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717332,"statfs":{"total":95995035648,"available":95966547968,"internally_reserved":0,"allocated":888832,"data_stored":425721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:40:02 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.51800000000000002,"15min":0.48199999999999998},"min":{"1min":0.40500000000000003,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.94099999999999995,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.52500000000000002,"15min":0.50600000000000001},"min":{"1min":0.373,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.80600000000000005,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.58299999999999996}]},{"osd":1,"last update":"Fri Jul 21 18:41:02 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.39100000000000001,"15min":0.35199999999999998},"min":{"1min":0.28299999999999997,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.51900000000000002,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.42499999999999999,"15min":0.39100000000000001},"min":{"1min":0.312,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.57599999999999996,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.46500000000000002}]},{"osd":2,"last update":"Fri Jul 21 18:41:02 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.56299999999999994,"15min":0.53400000000000003},"min":{"1min":0.45900000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.76600000000000001,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.59899999999999998,"15min":0.54400000000000004},"min":{"1min":0.45700000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.96499999999999997}]},{"osd":3,"last update":"Fri Jul 21 18:41:02 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.57899999999999996,"15min":0.54000000000000004},"min":{"1min":0.47499999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.78200000000000003,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56399999999999995,"15min":0.55700000000000005},"min":{"1min":0.39200000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.73299999999999998,"5min":1.05,"15min":1.05},"last":0.501}]},{"osd":5,"last update":"Fri Jul 21 18:41:23 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.66100000000000003,"15min":0.61299999999999999},"min":{"1min":0.56399999999999995,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.87,"5min":1.571,"15min":1.571},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.66400000000000003,"15min":0.622},"min":{"1min":0.46200000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.89600000000000002,"5min":1.377,"15min":1.377},"last":0.88200000000000001}]},{"osd":6,"last update":"Fri Jul 21 18:40:44 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.63900000000000001,"15min":0.61699999999999999},"min":{"1min":0.498,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.83699999999999997,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.627},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.65000000000000002,"15min":0.61399999999999999},"min":{"1min":0.46100000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.81299999999999994,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.65900000000000003}]},{"osd":7,"last update":"Fri Jul 21 18:41:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.59999999999999998,"15min":0.56999999999999995},"min":{"1min":0.39600000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.93700000000000006,"5min":1.073,"15min":1.073},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.55000000000000004,"15min":0.54200000000000004},"min":{"1min":0.41499999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.81499999999999995,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.55900000000000005}]},{"osd":8,"last update":"Fri Jul 21 18:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.65900000000000003,"15min":0.66100000000000003},"min":{"1min":0.46200000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.92400000000000004,"5min":1.026,"15min":1.026},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.73099999999999998,"15min":0.72499999999999998},"min":{"1min":0.56899999999999995,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.92500000000000004}]},{"osd":9,"last update":"Fri Jul 21 18:40:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.79000000000000004,"15min":0.79900000000000004},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.93999999999999995,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.83599999999999997,"15min":0.84599999999999997},"min":{"1min":0.60499999999999998,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.95499999999999996,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.78000000000000003}]},{"osd":10,"last update":"Fri Jul 21 18:41:08 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.78500000000000003,"15min":0.78300000000000003},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.98699999999999999,"5min":1.153,"15min":1.153},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.745,"15min":0.73899999999999999},"min":{"1min":0.55900000000000005,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.97499999999999998,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.79700000000000004}]},{"osd":11,"last update":"Fri Jul 21 18:41:05 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.94099999999999995}]}]},{"osd":5,"up_from":36,"seq":154618822710,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28428,"kb_used_data":1476,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716724,"statfs":{"total":95995035648,"available":95965925376,"internally_reserved":0,"allocated":1511424,"data_stored":1017942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:41:24 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.52200000000000002,"15min":0.52900000000000003},"min":{"1min":0.35799999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.73099999999999998,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.503,"5min":0.53600000000000003,"15min":0.55300000000000005},"min":{"1min":0.34799999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.42799999999999999}]},{"osd":1,"last update":"Fri Jul 21 18:41:24 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.55400000000000005,"15min":0.51400000000000001},"min":{"1min":0.39200000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.76900000000000002,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.622,"5min":0.56399999999999995,"15min":0.52100000000000002},"min":{"1min":0.48599999999999999,"5min":0.375,"15min":0.375},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.60099999999999998}]},{"osd":2,"last update":"Fri Jul 21 18:41:24 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.41299999999999998,"15min":0.39200000000000002},"min":{"1min":0.27600000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.439,"5min":0.42699999999999999,"15min":0.40100000000000002},"min":{"1min":0.32000000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.69799999999999995,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.501}]},{"osd":3,"last update":"Fri Jul 21 18:41:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.56100000000000005,"15min":0.56899999999999995},"min":{"1min":0.437,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.80100000000000005,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.436,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.622}]},{"osd":4,"last update":"Fri Jul 21 18:41:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.58999999999999997,"15min":0.56000000000000005},"min":{"1min":0.46600000000000003,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.86799999999999999,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.58299999999999996,"15min":0.54700000000000004},"min":{"1min":0.49399999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81999999999999995,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.73799999999999999}]},{"osd":6,"last update":"Fri Jul 21 18:40:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.59599999999999997,"15min":0.59999999999999998},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.60399999999999998,"15min":0.59099999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.71599999999999997}]},{"osd":7,"last update":"Fri Jul 21 18:41:05 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.65900000000000003,"15min":0.66600000000000004},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.65100000000000002,"15min":0.64300000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.85999999999999999,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.77000000000000002}]},{"osd":8,"last update":"Fri Jul 21 18:41:24 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.63100000000000001,"15min":0.63300000000000001},"min":{"1min":0.35099999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.95599999999999996,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.63700000000000001,"15min":0.63500000000000001},"min":{"1min":0.41999999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.77700000000000002}]},{"osd":9,"last update":"Fri Jul 21 18:40:45 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.72499999999999998,"15min":0.72899999999999998},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.877,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.73199999999999998,"15min":0.73399999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.81499999999999995}]},{"osd":10,"last update":"Fri Jul 21 18:41:05 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.71899999999999997,"15min":0.71199999999999997},"min":{"1min":0.54600000000000004,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.73599999999999999,"15min":0.73099999999999998},"min":{"1min":0.55700000000000005,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.83799999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":58579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":59069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":59271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-21T18:41:32.349 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-21T18:41:32.505 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.482+0000 7f9731a8d700 1 -- 172.21.15.39:0/1987698290 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f972c0ffbc0 msgr2=0x7f972c100010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:32.506 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.482+0000 7f9731a8d700 1 --2- 172.21.15.39:0/1987698290 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f972c0ffbc0 0x7f972c100010 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f9720009990 tx=0x7f9720009ca0 comp rx=0 tx=0).stop 2023-07-21T18:41:32.506 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.482+0000 7f9731a8d700 1 -- 172.21.15.39:0/1987698290 shutdown_connections 2023-07-21T18:41:32.506 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.482+0000 7f9731a8d700 1 --2- 172.21.15.39:0/1987698290 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f972c0fe9d0 0x7f972c0fede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:32.506 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.482+0000 7f9731a8d700 1 --2- 172.21.15.39:0/1987698290 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f972c100550 0x7f972c106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:32.506 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.482+0000 7f9731a8d700 1 --2- 172.21.15.39:0/1987698290 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f972c0ffbc0 0x7f972c100010 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:32.506 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.482+0000 7f9731a8d700 1 -- 172.21.15.39:0/1987698290 >> 172.21.15.39:0/1987698290 conn(0x7f972c0f9fd0 msgr2=0x7f972c0fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:32.506 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.482+0000 7f9731a8d700 1 -- 172.21.15.39:0/1987698290 shutdown_connections 2023-07-21T18:41:32.507 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.482+0000 7f9731a8d700 1 -- 172.21.15.39:0/1987698290 wait complete. 2023-07-21T18:41:32.507 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.482+0000 7f9731a8d700 1 Processor -- start 2023-07-21T18:41:32.507 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.483+0000 7f9731a8d700 1 -- start start 2023-07-21T18:41:32.509 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.483+0000 7f9731a8d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f972c0fe9d0 0x7f972c194750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:32.509 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.483+0000 7f9731a8d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f972c0ffbc0 0x7f972c194c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:32.509 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.483+0000 7f9731a8d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f972c100550 0x7f972c19bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:32.509 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.483+0000 7f9731a8d700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f972c19c2d0 con 0x7f972c100550 2023-07-21T18:41:32.509 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.483+0000 7f9731a8d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f972c19c410 con 0x7f972c0fe9d0 2023-07-21T18:41:32.509 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.483+0000 7f9731a8d700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f972c19c5a0 con 0x7f972c0ffbc0 2023-07-21T18:41:32.509 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.483+0000 7f972bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f972c100550 0x7f972c19bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:32.510 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.483+0000 7f972bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f972c100550 0x7f972c19bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56718/0 (socket says 172.21.15.39:56718) 2023-07-21T18:41:32.510 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.483+0000 7f972bfff700 1 -- 172.21.15.39:0/2857776647 learned_addr learned my addr 172.21.15.39:0/2857776647 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:32.510 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.483+0000 7f972affd700 1 --2- 172.21.15.39:0/2857776647 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f972c0ffbc0 0x7f972c194c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:32.510 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.483+0000 7f972b7fe700 1 --2- 172.21.15.39:0/2857776647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f972c0fe9d0 0x7f972c194750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:32.510 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.483+0000 7f972bfff700 1 -- 172.21.15.39:0/2857776647 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f972c0ffbc0 msgr2=0x7f972c194c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:32.510 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.484+0000 7f972bfff700 1 --2- 172.21.15.39:0/2857776647 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f972c0ffbc0 0x7f972c194c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:32.510 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.484+0000 7f972bfff700 1 -- 172.21.15.39:0/2857776647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f972c0fe9d0 msgr2=0x7f972c194750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:32.511 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.484+0000 7f972bfff700 1 --2- 172.21.15.39:0/2857776647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f972c0fe9d0 0x7f972c194750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:32.511 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.484+0000 7f972bfff700 1 -- 172.21.15.39:0/2857776647 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97200095f0 con 0x7f972c100550 2023-07-21T18:41:32.511 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.484+0000 7f972bfff700 1 --2- 172.21.15.39:0/2857776647 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f972c100550 0x7f972c19bd00 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f972400cfb0 tx=0x7f972400e330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:32.511 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.484+0000 7f9728ff9700 1 -- 172.21.15.39:0/2857776647 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9724011070 con 0x7f972c100550 2023-07-21T18:41:32.511 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.484+0000 7f9728ff9700 1 -- 172.21.15.39:0/2857776647 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9724007b30 con 0x7f972c100550 2023-07-21T18:41:32.511 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.484+0000 7f9731a8d700 1 -- 172.21.15.39:0/2857776647 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f972c0676d0 con 0x7f972c100550 2023-07-21T18:41:32.511 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.484+0000 7f9728ff9700 1 -- 172.21.15.39:0/2857776647 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f972400adf0 con 0x7f972c100550 2023-07-21T18:41:32.512 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.484+0000 7f9731a8d700 1 -- 172.21.15.39:0/2857776647 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f972c067b90 con 0x7f972c100550 2023-07-21T18:41:32.512 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.485+0000 7f9731a8d700 1 -- 172.21.15.39:0/2857776647 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f972c067300 con 0x7f972c100550 2023-07-21T18:41:32.512 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.486+0000 7f9728ff9700 1 -- 172.21.15.39:0/2857776647 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f972400e980 con 0x7f972c100550 2023-07-21T18:41:32.513 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.486+0000 7f9728ff9700 1 --2- 172.21.15.39:0/2857776647 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f97140a1220 0x7f97140a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:32.513 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.487+0000 7f972b7fe700 1 --2- 172.21.15.39:0/2857776647 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f97140a1220 0x7f97140a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:32.513 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.487+0000 7f9728ff9700 1 -- 172.21.15.39:0/2857776647 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f97240ce580 con 0x7f972c100550 2023-07-21T18:41:32.513 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.487+0000 7f972b7fe700 1 --2- 172.21.15.39:0/2857776647 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f97140a1220 0x7f97140a36d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f972c0ffa20 tx=0x7f971c00bef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:32.513 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.491+0000 7f9728ff9700 1 -- 172.21.15.39:0/2857776647 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f9724097b40 con 0x7f972c100550 2023-07-21T18:41:32.620 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:32 smithi193 ceph-mon[98204]: pgmap v293: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 9 op/s 2023-07-21T18:41:32.621 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:32 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/453869244' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T18:41:32.621 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:32 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/1494681701' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T18:41:32.621 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:32 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/2001220702' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:41:32.639 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:32 smithi155 ceph-mon[97934]: pgmap v293: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 9 op/s 2023-07-21T18:41:32.640 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:32 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/453869244' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T18:41:32.641 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:32 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/1494681701' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T18:41:32.641 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:32 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/2001220702' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:41:32.709 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.686+0000 7f9731a8d700 1 -- 172.21.15.39:0/2857776647 --> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f972c067db0 con 0x7f97140a1220 2023-07-21T18:41:32.721 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.698+0000 7f9728ff9700 1 -- 172.21.15.39:0/2857776647 <== mgr.14152 v2:172.21.15.39:6800/2116291712 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+344482 (secure 0 0 0) 0x7f972c067db0 con 0x7f97140a1220 2023-07-21T18:41:32.725 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:41:32.727 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.702+0000 7f9731a8d700 1 -- 172.21.15.39:0/2857776647 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f97140a1220 msgr2=0x7f97140a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:32.727 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.702+0000 7f9731a8d700 1 --2- 172.21.15.39:0/2857776647 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f97140a1220 0x7f97140a36d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f972c0ffa20 tx=0x7f971c00bef0 comp rx=0 tx=0).stop 2023-07-21T18:41:32.727 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.702+0000 7f9731a8d700 1 -- 172.21.15.39:0/2857776647 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f972c100550 msgr2=0x7f972c19bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:32.727 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.702+0000 7f9731a8d700 1 --2- 172.21.15.39:0/2857776647 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f972c100550 0x7f972c19bd00 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f972400cfb0 tx=0x7f972400e330 comp rx=0 tx=0).stop 2023-07-21T18:41:32.727 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.702+0000 7f9731a8d700 1 -- 172.21.15.39:0/2857776647 shutdown_connections 2023-07-21T18:41:32.728 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.702+0000 7f9731a8d700 1 --2- 172.21.15.39:0/2857776647 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f97140a1220 0x7f97140a36d0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:32.728 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.702+0000 7f9731a8d700 1 --2- 172.21.15.39:0/2857776647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f972c0fe9d0 0x7f972c194750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:32.728 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.702+0000 7f9731a8d700 1 --2- 172.21.15.39:0/2857776647 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f972c100550 0x7f972c19bd00 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:32.728 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.702+0000 7f9731a8d700 1 --2- 172.21.15.39:0/2857776647 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f972c0ffbc0 0x7f972c194c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:32.728 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.702+0000 7f9731a8d700 1 -- 172.21.15.39:0/2857776647 >> 172.21.15.39:0/2857776647 conn(0x7f972c0f9fd0 msgr2=0x7f972c103050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:32.728 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.702+0000 7f9731a8d700 1 -- 172.21.15.39:0/2857776647 shutdown_connections 2023-07-21T18:41:32.728 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.702+0000 7f9731a8d700 1 -- 172.21.15.39:0/2857776647 wait complete. 2023-07-21T18:41:32.728 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2023-07-21T18:41:32.737 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2023-07-21T18:41:31.141306+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654745,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":953,"num_read_kb":596,"num_write":416,"num_write_kb":3068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180736,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":343,"ondisk_log_size":343,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335916,"kb_used_data":12492,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124605908,"statfs":{"total":1151940427776,"available":1151596449792,"internally_reserved":0,"allocated":12791808,"data_stored":7060074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":636,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":111,"num_read_kb":51,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003031"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.144002+0000","last_change":"2023-07-21T18:40:04.792977+0000","last_active":"2023-07-21T18:41:08.144002+0000","last_peered":"2023-07-21T18:41:08.144002+0000","last_clean":"2023-07-21T18:41:08.144002+0000","last_became_active":"2023-07-21T18:40:04.792667+0000","last_became_peered":"2023-07-21T18:40:04.792667+0000","last_unstale":"2023-07-21T18:41:08.144002+0000","last_undegraded":"2023-07-21T18:41:08.144002+0000","last_fullsized":"2023-07-21T18:41:08.144002+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:45:33.822350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.154989+0000","last_change":"2023-07-21T18:40:04.792936+0000","last_active":"2023-07-21T18:41:08.154989+0000","last_peered":"2023-07-21T18:41:08.154989+0000","last_clean":"2023-07-21T18:41:08.154989+0000","last_became_active":"2023-07-21T18:40:04.792511+0000","last_became_peered":"2023-07-21T18:40:04.792511+0000","last_unstale":"2023-07-21T18:41:08.154989+0000","last_undegraded":"2023-07-21T18:41:08.154989+0000","last_fullsized":"2023-07-21T18:41:08.154989+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:18:24.383764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.154716+0000","last_change":"2023-07-21T18:40:04.796401+0000","last_active":"2023-07-21T18:41:08.154716+0000","last_peered":"2023-07-21T18:41:08.154716+0000","last_clean":"2023-07-21T18:41:08.154716+0000","last_became_active":"2023-07-21T18:40:04.796063+0000","last_became_peered":"2023-07-21T18:40:04.796063+0000","last_unstale":"2023-07-21T18:41:08.154716+0000","last_undegraded":"2023-07-21T18:41:08.154716+0000","last_fullsized":"2023-07-21T18:41:08.154716+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:53:37.099167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.154119+0000","last_change":"2023-07-21T18:41:08.154119+0000","last_active":"2023-07-21T18:41:08.154119+0000","last_peered":"2023-07-21T18:41:08.154119+0000","last_clean":"2023-07-21T18:41:08.154119+0000","last_became_active":"2023-07-21T18:41:08.153698+0000","last_became_peered":"2023-07-21T18:41:08.153698+0000","last_unstale":"2023-07-21T18:41:08.154119+0000","last_undegraded":"2023-07-21T18:41:08.154119+0000","last_fullsized":"2023-07-21T18:41:08.154119+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T18:41:49.057189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.148468+0000","last_change":"2023-07-21T18:40:04.794035+0000","last_active":"2023-07-21T18:41:08.148468+0000","last_peered":"2023-07-21T18:41:08.148468+0000","last_clean":"2023-07-21T18:41:08.148468+0000","last_became_active":"2023-07-21T18:40:04.793739+0000","last_became_peered":"2023-07-21T18:40:04.793739+0000","last_unstale":"2023-07-21T18:41:08.148468+0000","last_undegraded":"2023-07-21T18:41:08.148468+0000","last_fullsized":"2023-07-21T18:41:08.148468+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:25:53.830527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.150742+0000","last_change":"2023-07-21T18:41:08.150742+0000","last_active":"2023-07-21T18:41:08.150742+0000","last_peered":"2023-07-21T18:41:08.150742+0000","last_clean":"2023-07-21T18:41:08.150742+0000","last_became_active":"2023-07-21T18:41:08.150346+0000","last_became_peered":"2023-07-21T18:41:08.150346+0000","last_unstale":"2023-07-21T18:41:08.150742+0000","last_undegraded":"2023-07-21T18:41:08.150742+0000","last_fullsized":"2023-07-21T18:41:08.150742+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:06:31.088681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.595238+0000","last_change":"2023-07-21T18:40:04.787544+0000","last_active":"2023-07-21T18:41:08.595238+0000","last_peered":"2023-07-21T18:41:08.595238+0000","last_clean":"2023-07-21T18:41:08.595238+0000","last_became_active":"2023-07-21T18:40:04.787208+0000","last_became_peered":"2023-07-21T18:40:04.787208+0000","last_unstale":"2023-07-21T18:41:08.595238+0000","last_undegraded":"2023-07-21T18:41:08.595238+0000","last_fullsized":"2023-07-21T18:41:08.595238+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:57:13.333034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.249534+0000","last_change":"2023-07-21T18:41:08.249534+0000","last_active":"2023-07-21T18:41:08.249534+0000","last_peered":"2023-07-21T18:41:08.249534+0000","last_clean":"2023-07-21T18:41:08.249534+0000","last_became_active":"2023-07-21T18:41:08.249228+0000","last_became_peered":"2023-07-21T18:41:08.249228+0000","last_unstale":"2023-07-21T18:41:08.249534+0000","last_undegraded":"2023-07-21T18:41:08.249534+0000","last_fullsized":"2023-07-21T18:41:08.249534+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:19:58.325032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.151247+0000","last_change":"2023-07-21T18:40:04.782495+0000","last_active":"2023-07-21T18:41:08.151247+0000","last_peered":"2023-07-21T18:41:08.151247+0000","last_clean":"2023-07-21T18:41:08.151247+0000","last_became_active":"2023-07-21T18:40:04.782208+0000","last_became_peered":"2023-07-21T18:40:04.782208+0000","last_unstale":"2023-07-21T18:41:08.151247+0000","last_undegraded":"2023-07-21T18:41:08.151247+0000","last_fullsized":"2023-07-21T18:41:08.151247+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:22:50.428618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.148205+0000","last_change":"2023-07-21T18:40:04.790957+0000","last_active":"2023-07-21T18:41:08.148205+0000","last_peered":"2023-07-21T18:41:08.148205+0000","last_clean":"2023-07-21T18:41:08.148205+0000","last_became_active":"2023-07-21T18:40:04.790659+0000","last_became_peered":"2023-07-21T18:40:04.790659+0000","last_unstale":"2023-07-21T18:41:08.148205+0000","last_undegraded":"2023-07-21T18:41:08.148205+0000","last_fullsized":"2023-07-21T18:41:08.148205+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:29:03.090784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.595484+0000","last_change":"2023-07-21T18:40:04.789415+0000","last_active":"2023-07-21T18:41:08.595484+0000","last_peered":"2023-07-21T18:41:08.595484+0000","last_clean":"2023-07-21T18:41:08.595484+0000","last_became_active":"2023-07-21T18:40:04.789088+0000","last_became_peered":"2023-07-21T18:40:04.789088+0000","last_unstale":"2023-07-21T18:41:08.595484+0000","last_undegraded":"2023-07-21T18:41:08.595484+0000","last_fullsized":"2023-07-21T18:41:08.595484+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:40:03.977217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.245377+0000","last_change":"2023-07-21T18:40:04.789101+0000","last_active":"2023-07-21T18:41:08.245377+0000","last_peered":"2023-07-21T18:41:08.245377+0000","last_clean":"2023-07-21T18:41:08.245377+0000","last_became_active":"2023-07-21T18:40:04.788704+0000","last_became_peered":"2023-07-21T18:40:04.788704+0000","last_unstale":"2023-07-21T18:41:08.245377+0000","last_undegraded":"2023-07-21T18:41:08.245377+0000","last_fullsized":"2023-07-21T18:41:08.245377+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:33:01.699609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.144358+0000","last_change":"2023-07-21T18:40:04.784863+0000","last_active":"2023-07-21T18:41:08.144358+0000","last_peered":"2023-07-21T18:41:08.144358+0000","last_clean":"2023-07-21T18:41:08.144358+0000","last_became_active":"2023-07-21T18:40:04.784474+0000","last_became_peered":"2023-07-21T18:40:04.784474+0000","last_unstale":"2023-07-21T18:41:08.144358+0000","last_undegraded":"2023-07-21T18:41:08.144358+0000","last_fullsized":"2023-07-21T18:41:08.144358+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:20:04.409410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.154148+0000","last_change":"2023-07-21T18:40:04.797678+0000","last_active":"2023-07-21T18:41:08.154148+0000","last_peered":"2023-07-21T18:41:08.154148+0000","last_clean":"2023-07-21T18:41:08.154148+0000","last_became_active":"2023-07-21T18:40:04.797318+0000","last_became_peered":"2023-07-21T18:40:04.797318+0000","last_unstale":"2023-07-21T18:41:08.154148+0000","last_undegraded":"2023-07-21T18:41:08.154148+0000","last_fullsized":"2023-07-21T18:41:08.154148+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:28:01.333981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.153924+0000","last_change":"2023-07-21T18:40:04.793170+0000","last_active":"2023-07-21T18:41:08.153924+0000","last_peered":"2023-07-21T18:41:08.153924+0000","last_clean":"2023-07-21T18:41:08.153924+0000","last_became_active":"2023-07-21T18:40:04.792563+0000","last_became_peered":"2023-07-21T18:40:04.792563+0000","last_unstale":"2023-07-21T18:41:08.153924+0000","last_undegraded":"2023-07-21T18:41:08.153924+0000","last_fullsized":"2023-07-21T18:41:08.153924+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:07:51.115805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.150518+0000","last_change":"2023-07-21T18:40:04.791972+0000","last_active":"2023-07-21T18:41:08.150518+0000","last_peered":"2023-07-21T18:41:08.150518+0000","last_clean":"2023-07-21T18:41:08.150518+0000","last_became_active":"2023-07-21T18:40:04.791639+0000","last_became_peered":"2023-07-21T18:40:04.791639+0000","last_unstale":"2023-07-21T18:41:08.150518+0000","last_undegraded":"2023-07-21T18:41:08.150518+0000","last_fullsized":"2023-07-21T18:41:08.150518+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:31:09.972680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.144015+0000","last_change":"2023-07-21T18:40:04.789362+0000","last_active":"2023-07-21T18:41:08.144015+0000","last_peered":"2023-07-21T18:41:08.144015+0000","last_clean":"2023-07-21T18:41:08.144015+0000","last_became_active":"2023-07-21T18:40:04.788978+0000","last_became_peered":"2023-07-21T18:40:04.788978+0000","last_unstale":"2023-07-21T18:41:08.144015+0000","last_undegraded":"2023-07-21T18:41:08.144015+0000","last_fullsized":"2023-07-21T18:41:08.144015+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:06:43.785294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.146563+0000","last_change":"2023-07-21T18:40:04.788016+0000","last_active":"2023-07-21T18:41:08.146563+0000","last_peered":"2023-07-21T18:41:08.146563+0000","last_clean":"2023-07-21T18:41:08.146563+0000","last_became_active":"2023-07-21T18:40:04.787690+0000","last_became_peered":"2023-07-21T18:40:04.787690+0000","last_unstale":"2023-07-21T18:41:08.146563+0000","last_undegraded":"2023-07-21T18:41:08.146563+0000","last_fullsized":"2023-07-21T18:41:08.146563+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:40:42.379190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.152768+0000","last_change":"2023-07-21T18:40:04.793075+0000","last_active":"2023-07-21T18:41:08.152768+0000","last_peered":"2023-07-21T18:41:08.152768+0000","last_clean":"2023-07-21T18:41:08.152768+0000","last_became_active":"2023-07-21T18:40:04.792773+0000","last_became_peered":"2023-07-21T18:40:04.792773+0000","last_unstale":"2023-07-21T18:41:08.152768+0000","last_undegraded":"2023-07-21T18:41:08.152768+0000","last_fullsized":"2023-07-21T18:41:08.152768+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:46:27.429800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"79'13","reported_seq":108,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:26.100464+0000","last_change":"2023-07-21T18:40:04.178197+0000","last_active":"2023-07-21T18:41:26.100464+0000","last_peered":"2023-07-21T18:41:26.100464+0000","last_clean":"2023-07-21T18:41:26.100464+0000","last_became_active":"2023-07-21T18:40:04.177914+0000","last_became_peered":"2023-07-21T18:40:04.177914+0000","last_unstale":"2023-07-21T18:41:26.100464+0000","last_undegraded":"2023-07-21T18:41:26.100464+0000","last_fullsized":"2023-07-21T18:41:26.100464+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:55:39.788580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":77,"num_read_kb":77,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.149760+0000","last_change":"2023-07-21T18:40:04.791610+0000","last_active":"2023-07-21T18:41:08.149760+0000","last_peered":"2023-07-21T18:41:08.149760+0000","last_clean":"2023-07-21T18:41:08.149760+0000","last_became_active":"2023-07-21T18:40:04.791242+0000","last_became_peered":"2023-07-21T18:40:04.791242+0000","last_unstale":"2023-07-21T18:41:08.149760+0000","last_undegraded":"2023-07-21T18:41:08.149760+0000","last_fullsized":"2023-07-21T18:41:08.149760+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:27:52.116304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.149806+0000","last_change":"2023-07-21T18:40:04.182120+0000","last_active":"2023-07-21T18:41:08.149806+0000","last_peered":"2023-07-21T18:41:08.149806+0000","last_clean":"2023-07-21T18:41:08.149806+0000","last_became_active":"2023-07-21T18:40:04.181797+0000","last_became_peered":"2023-07-21T18:40:04.181797+0000","last_unstale":"2023-07-21T18:41:08.149806+0000","last_undegraded":"2023-07-21T18:41:08.149806+0000","last_fullsized":"2023-07-21T18:41:08.149806+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:47:45.817595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.155369+0000","last_change":"2023-07-21T18:40:04.796649+0000","last_active":"2023-07-21T18:41:08.155369+0000","last_peered":"2023-07-21T18:41:08.155369+0000","last_clean":"2023-07-21T18:41:08.155369+0000","last_became_active":"2023-07-21T18:40:04.795610+0000","last_became_peered":"2023-07-21T18:40:04.795610+0000","last_unstale":"2023-07-21T18:41:08.155369+0000","last_undegraded":"2023-07-21T18:41:08.155369+0000","last_fullsized":"2023-07-21T18:41:08.155369+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:22:37.727432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"81'13","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.149058+0000","last_change":"2023-07-21T18:40:04.178384+0000","last_active":"2023-07-21T18:41:08.149058+0000","last_peered":"2023-07-21T18:41:08.149058+0000","last_clean":"2023-07-21T18:41:08.149058+0000","last_became_active":"2023-07-21T18:40:04.178074+0000","last_became_peered":"2023-07-21T18:40:04.178074+0000","last_unstale":"2023-07-21T18:41:08.149058+0000","last_undegraded":"2023-07-21T18:41:08.149058+0000","last_fullsized":"2023-07-21T18:41:08.149058+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:20:16.081243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.150712+0000","last_change":"2023-07-21T18:40:04.785431+0000","last_active":"2023-07-21T18:41:08.150712+0000","last_peered":"2023-07-21T18:41:08.150712+0000","last_clean":"2023-07-21T18:41:08.150712+0000","last_became_active":"2023-07-21T18:40:04.785124+0000","last_became_peered":"2023-07-21T18:40:04.785124+0000","last_unstale":"2023-07-21T18:41:08.150712+0000","last_undegraded":"2023-07-21T18:41:08.150712+0000","last_fullsized":"2023-07-21T18:41:08.150712+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:07:13.594091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"79'5","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.148841+0000","last_change":"2023-07-21T18:40:04.176349+0000","last_active":"2023-07-21T18:41:08.148841+0000","last_peered":"2023-07-21T18:41:08.148841+0000","last_clean":"2023-07-21T18:41:08.148841+0000","last_became_active":"2023-07-21T18:40:04.175997+0000","last_became_peered":"2023-07-21T18:40:04.175997+0000","last_unstale":"2023-07-21T18:41:08.148841+0000","last_undegraded":"2023-07-21T18:41:08.148841+0000","last_fullsized":"2023-07-21T18:41:08.148841+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:24:53.545517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.246653+0000","last_change":"2023-07-21T18:41:08.246653+0000","last_active":"2023-07-21T18:41:08.246653+0000","last_peered":"2023-07-21T18:41:08.246653+0000","last_clean":"2023-07-21T18:41:08.246653+0000","last_became_active":"2023-07-21T18:41:08.246035+0000","last_became_peered":"2023-07-21T18:41:08.246035+0000","last_unstale":"2023-07-21T18:41:08.246653+0000","last_undegraded":"2023-07-21T18:41:08.246653+0000","last_fullsized":"2023-07-21T18:41:08.246653+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:30:36.054664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1b","version":"79'5","reported_seq":105,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:29.084542+0000","last_change":"2023-07-21T18:40:04.177638+0000","last_active":"2023-07-21T18:41:29.084542+0000","last_peered":"2023-07-21T18:41:29.084542+0000","last_clean":"2023-07-21T18:41:29.084542+0000","last_became_active":"2023-07-21T18:40:04.177284+0000","last_became_peered":"2023-07-21T18:40:04.177284+0000","last_unstale":"2023-07-21T18:41:29.084542+0000","last_undegraded":"2023-07-21T18:41:29.084542+0000","last_fullsized":"2023-07-21T18:41:29.084542+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:34:36.750937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.148514+0000","last_change":"2023-07-21T18:40:04.791536+0000","last_active":"2023-07-21T18:41:08.148514+0000","last_peered":"2023-07-21T18:41:08.148514+0000","last_clean":"2023-07-21T18:41:08.148514+0000","last_became_active":"2023-07-21T18:40:04.791246+0000","last_became_peered":"2023-07-21T18:40:04.791246+0000","last_unstale":"2023-07-21T18:41:08.148514+0000","last_undegraded":"2023-07-21T18:41:08.148514+0000","last_fullsized":"2023-07-21T18:41:08.148514+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:30:46.562521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"79'7","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.147995+0000","last_change":"2023-07-21T18:40:04.179054+0000","last_active":"2023-07-21T18:41:08.147995+0000","last_peered":"2023-07-21T18:41:08.147995+0000","last_clean":"2023-07-21T18:41:08.147995+0000","last_became_active":"2023-07-21T18:40:04.178713+0000","last_became_peered":"2023-07-21T18:40:04.178713+0000","last_unstale":"2023-07-21T18:41:08.147995+0000","last_undegraded":"2023-07-21T18:41:08.147995+0000","last_fullsized":"2023-07-21T18:41:08.147995+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:54:00.028673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.249387+0000","last_change":"2023-07-21T18:40:04.793947+0000","last_active":"2023-07-21T18:41:08.249387+0000","last_peered":"2023-07-21T18:41:08.249387+0000","last_clean":"2023-07-21T18:41:08.249387+0000","last_became_active":"2023-07-21T18:40:04.793631+0000","last_became_peered":"2023-07-21T18:40:04.793631+0000","last_unstale":"2023-07-21T18:41:08.249387+0000","last_undegraded":"2023-07-21T18:41:08.249387+0000","last_fullsized":"2023-07-21T18:41:08.249387+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T18:56:26.416040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"79'3","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.143902+0000","last_change":"2023-07-21T18:40:04.182549+0000","last_active":"2023-07-21T18:41:08.143902+0000","last_peered":"2023-07-21T18:41:08.143902+0000","last_clean":"2023-07-21T18:41:08.143902+0000","last_became_active":"2023-07-21T18:40:04.182233+0000","last_became_peered":"2023-07-21T18:40:04.182233+0000","last_unstale":"2023-07-21T18:41:08.143902+0000","last_undegraded":"2023-07-21T18:41:08.143902+0000","last_fullsized":"2023-07-21T18:41:08.143902+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:38:34.232736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.149083+0000","last_change":"2023-07-21T18:40:04.789987+0000","last_active":"2023-07-21T18:41:08.149083+0000","last_peered":"2023-07-21T18:41:08.149083+0000","last_clean":"2023-07-21T18:41:08.149083+0000","last_became_active":"2023-07-21T18:40:04.789647+0000","last_became_peered":"2023-07-21T18:40:04.789647+0000","last_unstale":"2023-07-21T18:41:08.149083+0000","last_undegraded":"2023-07-21T18:41:08.149083+0000","last_fullsized":"2023-07-21T18:41:08.149083+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:33:16.733179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'3","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.248336+0000","last_change":"2023-07-21T18:40:04.177342+0000","last_active":"2023-07-21T18:41:08.248336+0000","last_peered":"2023-07-21T18:41:08.248336+0000","last_clean":"2023-07-21T18:41:08.248336+0000","last_became_active":"2023-07-21T18:40:04.176983+0000","last_became_peered":"2023-07-21T18:40:04.176983+0000","last_unstale":"2023-07-21T18:41:08.248336+0000","last_undegraded":"2023-07-21T18:41:08.248336+0000","last_fullsized":"2023-07-21T18:41:08.248336+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:25:28.759619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.146313+0000","last_change":"2023-07-21T18:40:04.794907+0000","last_active":"2023-07-21T18:41:08.146313+0000","last_peered":"2023-07-21T18:41:08.146313+0000","last_clean":"2023-07-21T18:41:08.146313+0000","last_became_active":"2023-07-21T18:40:04.794545+0000","last_became_peered":"2023-07-21T18:40:04.794545+0000","last_unstale":"2023-07-21T18:41:08.146313+0000","last_undegraded":"2023-07-21T18:41:08.146313+0000","last_fullsized":"2023-07-21T18:41:08.146313+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:42:33.019351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"79'4","reported_seq":178,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:28.082056+0000","last_change":"2023-07-21T18:40:04.179198+0000","last_active":"2023-07-21T18:41:28.082056+0000","last_peered":"2023-07-21T18:41:28.082056+0000","last_clean":"2023-07-21T18:41:28.082056+0000","last_became_active":"2023-07-21T18:40:04.178841+0000","last_became_peered":"2023-07-21T18:40:04.178841+0000","last_unstale":"2023-07-21T18:41:28.082056+0000","last_undegraded":"2023-07-21T18:41:28.082056+0000","last_fullsized":"2023-07-21T18:41:28.082056+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:12:15.233249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":156,"num_read_kb":156,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.151068+0000","last_change":"2023-07-21T18:40:04.786229+0000","last_active":"2023-07-21T18:41:08.151068+0000","last_peered":"2023-07-21T18:41:08.151068+0000","last_clean":"2023-07-21T18:41:08.151068+0000","last_became_active":"2023-07-21T18:40:04.785919+0000","last_became_peered":"2023-07-21T18:40:04.785919+0000","last_unstale":"2023-07-21T18:41:08.151068+0000","last_undegraded":"2023-07-21T18:41:08.151068+0000","last_fullsized":"2023-07-21T18:41:08.151068+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:37:35.592913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'4","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.595131+0000","last_change":"2023-07-21T18:40:04.177164+0000","last_active":"2023-07-21T18:41:08.595131+0000","last_peered":"2023-07-21T18:41:08.595131+0000","last_clean":"2023-07-21T18:41:08.595131+0000","last_became_active":"2023-07-21T18:40:04.176617+0000","last_became_peered":"2023-07-21T18:40:04.176617+0000","last_unstale":"2023-07-21T18:41:08.595131+0000","last_undegraded":"2023-07-21T18:41:08.595131+0000","last_fullsized":"2023-07-21T18:41:08.595131+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:43:44.771285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.155486+0000","last_change":"2023-07-21T18:40:04.787863+0000","last_active":"2023-07-21T18:41:08.155486+0000","last_peered":"2023-07-21T18:41:08.155486+0000","last_clean":"2023-07-21T18:41:08.155486+0000","last_became_active":"2023-07-21T18:40:04.787562+0000","last_became_peered":"2023-07-21T18:40:04.787562+0000","last_unstale":"2023-07-21T18:41:08.155486+0000","last_undegraded":"2023-07-21T18:41:08.155486+0000","last_fullsized":"2023-07-21T18:41:08.155486+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:15:49.834031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'5","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.248499+0000","last_change":"2023-07-21T18:40:04.179921+0000","last_active":"2023-07-21T18:41:08.248499+0000","last_peered":"2023-07-21T18:41:08.248499+0000","last_clean":"2023-07-21T18:41:08.248499+0000","last_became_active":"2023-07-21T18:40:04.179670+0000","last_became_peered":"2023-07-21T18:40:04.179670+0000","last_unstale":"2023-07-21T18:41:08.248499+0000","last_undegraded":"2023-07-21T18:41:08.248499+0000","last_fullsized":"2023-07-21T18:41:08.248499+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:15:29.991358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.150860+0000","last_change":"2023-07-21T18:40:04.792930+0000","last_active":"2023-07-21T18:41:08.150860+0000","last_peered":"2023-07-21T18:41:08.150860+0000","last_clean":"2023-07-21T18:41:08.150860+0000","last_became_active":"2023-07-21T18:40:04.792654+0000","last_became_peered":"2023-07-21T18:40:04.792654+0000","last_unstale":"2023-07-21T18:41:08.150860+0000","last_undegraded":"2023-07-21T18:41:08.150860+0000","last_fullsized":"2023-07-21T18:41:08.150860+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:03:06.046049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'30","reported_seq":210,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:28.377347+0000","last_change":"2023-07-21T18:40:04.178820+0000","last_active":"2023-07-21T18:41:28.377347+0000","last_peered":"2023-07-21T18:41:28.377347+0000","last_clean":"2023-07-21T18:41:28.377347+0000","last_became_active":"2023-07-21T18:40:04.178502+0000","last_became_peered":"2023-07-21T18:40:04.178502+0000","last_unstale":"2023-07-21T18:41:28.377347+0000","last_undegraded":"2023-07-21T18:41:28.377347+0000","last_fullsized":"2023-07-21T18:41:28.377347+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:19:51.865558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57193,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":162,"num_read_kb":136,"num_write":32,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.148850+0000","last_change":"2023-07-21T18:40:04.792921+0000","last_active":"2023-07-21T18:41:08.148850+0000","last_peered":"2023-07-21T18:41:08.148850+0000","last_clean":"2023-07-21T18:41:08.148850+0000","last_became_active":"2023-07-21T18:40:04.792608+0000","last_became_peered":"2023-07-21T18:40:04.792608+0000","last_unstale":"2023-07-21T18:41:08.148850+0000","last_undegraded":"2023-07-21T18:41:08.148850+0000","last_fullsized":"2023-07-21T18:41:08.148850+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:58:38.532900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'9","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.153057+0000","last_change":"2023-07-21T18:40:04.184917+0000","last_active":"2023-07-21T18:41:08.153057+0000","last_peered":"2023-07-21T18:41:08.153057+0000","last_clean":"2023-07-21T18:41:08.153057+0000","last_became_active":"2023-07-21T18:40:04.184612+0000","last_became_peered":"2023-07-21T18:40:04.184612+0000","last_unstale":"2023-07-21T18:41:08.153057+0000","last_undegraded":"2023-07-21T18:41:08.153057+0000","last_fullsized":"2023-07-21T18:41:08.153057+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:58:56.789436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.143966+0000","last_change":"2023-07-21T18:40:04.794134+0000","last_active":"2023-07-21T18:41:08.143966+0000","last_peered":"2023-07-21T18:41:08.143966+0000","last_clean":"2023-07-21T18:41:08.143966+0000","last_became_active":"2023-07-21T18:40:04.793784+0000","last_became_peered":"2023-07-21T18:40:04.793784+0000","last_unstale":"2023-07-21T18:41:08.143966+0000","last_undegraded":"2023-07-21T18:41:08.143966+0000","last_fullsized":"2023-07-21T18:41:08.143966+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:30:56.528513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'5","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.248332+0000","last_change":"2023-07-21T18:40:04.184889+0000","last_active":"2023-07-21T18:41:08.248332+0000","last_peered":"2023-07-21T18:41:08.248332+0000","last_clean":"2023-07-21T18:41:08.248332+0000","last_became_active":"2023-07-21T18:40:04.184516+0000","last_became_peered":"2023-07-21T18:40:04.184516+0000","last_unstale":"2023-07-21T18:41:08.248332+0000","last_undegraded":"2023-07-21T18:41:08.248332+0000","last_fullsized":"2023-07-21T18:41:08.248332+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:28:26.823194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.595621+0000","last_change":"2023-07-21T18:40:04.790627+0000","last_active":"2023-07-21T18:41:08.595621+0000","last_peered":"2023-07-21T18:41:08.595621+0000","last_clean":"2023-07-21T18:41:08.595621+0000","last_became_active":"2023-07-21T18:40:04.790298+0000","last_became_peered":"2023-07-21T18:40:04.790298+0000","last_unstale":"2023-07-21T18:41:08.595621+0000","last_undegraded":"2023-07-21T18:41:08.595621+0000","last_fullsized":"2023-07-21T18:41:08.595621+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:07:48.083182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"79'6","reported_seq":103,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:26.079011+0000","last_change":"2023-07-21T18:40:04.176156+0000","last_active":"2023-07-21T18:41:26.079011+0000","last_peered":"2023-07-21T18:41:26.079011+0000","last_clean":"2023-07-21T18:41:26.079011+0000","last_became_active":"2023-07-21T18:40:04.175823+0000","last_became_peered":"2023-07-21T18:40:04.175823+0000","last_unstale":"2023-07-21T18:41:26.079011+0000","last_undegraded":"2023-07-21T18:41:26.079011+0000","last_fullsized":"2023-07-21T18:41:26.079011+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:40:30.078845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":930,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":80,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.595323+0000","last_change":"2023-07-21T18:40:04.790422+0000","last_active":"2023-07-21T18:41:08.595323+0000","last_peered":"2023-07-21T18:41:08.595323+0000","last_clean":"2023-07-21T18:41:08.595323+0000","last_became_active":"2023-07-21T18:40:04.790124+0000","last_became_peered":"2023-07-21T18:40:04.790124+0000","last_unstale":"2023-07-21T18:41:08.595323+0000","last_undegraded":"2023-07-21T18:41:08.595323+0000","last_fullsized":"2023-07-21T18:41:08.595323+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:11:35.114265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'17","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.148291+0000","last_change":"2023-07-21T18:40:04.177006+0000","last_active":"2023-07-21T18:41:08.148291+0000","last_peered":"2023-07-21T18:41:08.148291+0000","last_clean":"2023-07-21T18:41:08.148291+0000","last_became_active":"2023-07-21T18:40:04.176711+0000","last_became_peered":"2023-07-21T18:40:04.176711+0000","last_unstale":"2023-07-21T18:41:08.148291+0000","last_undegraded":"2023-07-21T18:41:08.148291+0000","last_fullsized":"2023-07-21T18:41:08.148291+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:05:40.549509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.249206+0000","last_change":"2023-07-21T18:40:04.785601+0000","last_active":"2023-07-21T18:41:08.249206+0000","last_peered":"2023-07-21T18:41:08.249206+0000","last_clean":"2023-07-21T18:41:08.249206+0000","last_became_active":"2023-07-21T18:40:04.785307+0000","last_became_peered":"2023-07-21T18:40:04.785307+0000","last_unstale":"2023-07-21T18:41:08.249206+0000","last_undegraded":"2023-07-21T18:41:08.249206+0000","last_fullsized":"2023-07-21T18:41:08.249206+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:04:30.580644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.149118+0000","last_change":"2023-07-21T18:40:04.171005+0000","last_active":"2023-07-21T18:41:08.149118+0000","last_peered":"2023-07-21T18:41:08.149118+0000","last_clean":"2023-07-21T18:41:08.149118+0000","last_became_active":"2023-07-21T18:40:04.170719+0000","last_became_peered":"2023-07-21T18:40:04.170719+0000","last_unstale":"2023-07-21T18:41:08.149118+0000","last_undegraded":"2023-07-21T18:41:08.149118+0000","last_fullsized":"2023-07-21T18:41:08.149118+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:06:18.122712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.249987+0000","last_change":"2023-07-21T18:41:08.249987+0000","last_active":"2023-07-21T18:41:08.249987+0000","last_peered":"2023-07-21T18:41:08.249987+0000","last_clean":"2023-07-21T18:41:08.249987+0000","last_became_active":"2023-07-21T18:41:08.249610+0000","last_became_peered":"2023-07-21T18:41:08.249610+0000","last_unstale":"2023-07-21T18:41:08.249987+0000","last_undegraded":"2023-07-21T18:41:08.249987+0000","last_fullsized":"2023-07-21T18:41:08.249987+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:40:12.815463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"79'1","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.146358+0000","last_change":"2023-07-21T18:40:04.181069+0000","last_active":"2023-07-21T18:41:08.146358+0000","last_peered":"2023-07-21T18:41:08.146358+0000","last_clean":"2023-07-21T18:41:08.146358+0000","last_became_active":"2023-07-21T18:40:04.180750+0000","last_became_peered":"2023-07-21T18:40:04.180750+0000","last_unstale":"2023-07-21T18:41:08.146358+0000","last_undegraded":"2023-07-21T18:41:08.146358+0000","last_fullsized":"2023-07-21T18:41:08.146358+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:33:23.897099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"79'1","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.147933+0000","last_change":"2023-07-21T18:40:04.789940+0000","last_active":"2023-07-21T18:41:08.147933+0000","last_peered":"2023-07-21T18:41:08.147933+0000","last_clean":"2023-07-21T18:41:08.147933+0000","last_became_active":"2023-07-21T18:40:04.789465+0000","last_became_peered":"2023-07-21T18:40:04.789465+0000","last_unstale":"2023-07-21T18:41:08.147933+0000","last_undegraded":"2023-07-21T18:41:08.147933+0000","last_fullsized":"2023-07-21T18:41:08.147933+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:31:28.465807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"81'6","reported_seq":105,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:29.105557+0000","last_change":"2023-07-21T18:40:04.180522+0000","last_active":"2023-07-21T18:41:29.105557+0000","last_peered":"2023-07-21T18:41:29.105557+0000","last_clean":"2023-07-21T18:41:29.105557+0000","last_became_active":"2023-07-21T18:40:04.180196+0000","last_became_peered":"2023-07-21T18:40:04.180196+0000","last_unstale":"2023-07-21T18:41:29.105557+0000","last_undegraded":"2023-07-21T18:41:29.105557+0000","last_fullsized":"2023-07-21T18:41:29.105557+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:24:49.698830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":858,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.148148+0000","last_change":"2023-07-21T18:40:04.794781+0000","last_active":"2023-07-21T18:41:08.148148+0000","last_peered":"2023-07-21T18:41:08.148148+0000","last_clean":"2023-07-21T18:41:08.148148+0000","last_became_active":"2023-07-21T18:40:04.794388+0000","last_became_peered":"2023-07-21T18:40:04.794388+0000","last_unstale":"2023-07-21T18:41:08.148148+0000","last_undegraded":"2023-07-21T18:41:08.148148+0000","last_fullsized":"2023-07-21T18:41:08.148148+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:01:02.955504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.248717+0000","last_change":"2023-07-21T18:40:04.786408+0000","last_active":"2023-07-21T18:41:08.248717+0000","last_peered":"2023-07-21T18:41:08.248717+0000","last_clean":"2023-07-21T18:41:08.248717+0000","last_became_active":"2023-07-21T18:40:04.785555+0000","last_became_peered":"2023-07-21T18:40:04.785555+0000","last_unstale":"2023-07-21T18:41:08.248717+0000","last_undegraded":"2023-07-21T18:41:08.248717+0000","last_fullsized":"2023-07-21T18:41:08.248717+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:45:51.047733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"79'5","reported_seq":102,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:27.080056+0000","last_change":"2023-07-21T18:40:04.185980+0000","last_active":"2023-07-21T18:41:27.080056+0000","last_peered":"2023-07-21T18:41:27.080056+0000","last_clean":"2023-07-21T18:41:27.080056+0000","last_became_active":"2023-07-21T18:40:04.185618+0000","last_became_peered":"2023-07-21T18:40:04.185618+0000","last_unstale":"2023-07-21T18:41:27.080056+0000","last_undegraded":"2023-07-21T18:41:27.080056+0000","last_fullsized":"2023-07-21T18:41:27.080056+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:37:27.318460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":79,"num_read_kb":79,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.149558+0000","last_change":"2023-07-21T18:40:04.792535+0000","last_active":"2023-07-21T18:41:08.149558+0000","last_peered":"2023-07-21T18:41:08.149558+0000","last_clean":"2023-07-21T18:41:08.149558+0000","last_became_active":"2023-07-21T18:40:04.792239+0000","last_became_peered":"2023-07-21T18:40:04.792239+0000","last_unstale":"2023-07-21T18:41:08.149558+0000","last_undegraded":"2023-07-21T18:41:08.149558+0000","last_fullsized":"2023-07-21T18:41:08.149558+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:34:19.596250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"79'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.244642+0000","last_change":"2023-07-21T18:40:04.173539+0000","last_active":"2023-07-21T18:41:08.244642+0000","last_peered":"2023-07-21T18:41:08.244642+0000","last_clean":"2023-07-21T18:41:08.244642+0000","last_became_active":"2023-07-21T18:40:04.173225+0000","last_became_peered":"2023-07-21T18:40:04.173225+0000","last_unstale":"2023-07-21T18:41:08.244642+0000","last_undegraded":"2023-07-21T18:41:08.244642+0000","last_fullsized":"2023-07-21T18:41:08.244642+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:14:31.670842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":341,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.595315+0000","last_change":"2023-07-21T18:39:05.999983+0000","last_active":"2023-07-21T18:41:08.595315+0000","last_peered":"2023-07-21T18:41:08.595315+0000","last_clean":"2023-07-21T18:41:08.595315+0000","last_became_active":"2023-07-21T18:39:05.999495+0000","last_became_peered":"2023-07-21T18:39:05.999495+0000","last_unstale":"2023-07-21T18:41:08.595315+0000","last_undegraded":"2023-07-21T18:41:08.595315+0000","last_fullsized":"2023-07-21T18:41:08.595315+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:36:14.004429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:36:14.004429+0000","last_clean_scrub_stamp":"2023-07-21T18:36:14.004429+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:13:55.308992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.143732+0000","last_change":"2023-07-21T18:40:04.785086+0000","last_active":"2023-07-21T18:41:08.143732+0000","last_peered":"2023-07-21T18:41:08.143732+0000","last_clean":"2023-07-21T18:41:08.143732+0000","last_became_active":"2023-07-21T18:40:04.784693+0000","last_became_peered":"2023-07-21T18:40:04.784693+0000","last_unstale":"2023-07-21T18:41:08.143732+0000","last_undegraded":"2023-07-21T18:41:08.143732+0000","last_fullsized":"2023-07-21T18:41:08.143732+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:20:44.967039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"79'4","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.595119+0000","last_change":"2023-07-21T18:40:04.177032+0000","last_active":"2023-07-21T18:41:08.595119+0000","last_peered":"2023-07-21T18:41:08.595119+0000","last_clean":"2023-07-21T18:41:08.595119+0000","last_became_active":"2023-07-21T18:40:04.176741+0000","last_became_peered":"2023-07-21T18:40:04.176741+0000","last_unstale":"2023-07-21T18:41:08.595119+0000","last_undegraded":"2023-07-21T18:41:08.595119+0000","last_fullsized":"2023-07-21T18:41:08.595119+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:30:00.615431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"79'4","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.148885+0000","last_change":"2023-07-21T18:40:04.180106+0000","last_active":"2023-07-21T18:41:08.148885+0000","last_peered":"2023-07-21T18:41:08.148885+0000","last_clean":"2023-07-21T18:41:08.148885+0000","last_became_active":"2023-07-21T18:40:04.179802+0000","last_became_peered":"2023-07-21T18:40:04.179802+0000","last_unstale":"2023-07-21T18:41:08.148885+0000","last_undegraded":"2023-07-21T18:41:08.148885+0000","last_fullsized":"2023-07-21T18:41:08.148885+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:58:17.009228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.144906+0000","last_change":"2023-07-21T18:40:04.789127+0000","last_active":"2023-07-21T18:41:08.144906+0000","last_peered":"2023-07-21T18:41:08.144906+0000","last_clean":"2023-07-21T18:41:08.144906+0000","last_became_active":"2023-07-21T18:40:04.788815+0000","last_became_peered":"2023-07-21T18:40:04.788815+0000","last_unstale":"2023-07-21T18:41:08.144906+0000","last_undegraded":"2023-07-21T18:41:08.144906+0000","last_fullsized":"2023-07-21T18:41:08.144906+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:31:07.061759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"79'9","reported_seq":43,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.156947+0000","last_change":"2023-07-21T18:41:08.156947+0000","last_active":"2023-07-21T18:41:08.156947+0000","last_peered":"2023-07-21T18:41:08.156947+0000","last_clean":"2023-07-21T18:41:08.156947+0000","last_became_active":"2023-07-21T18:41:08.149966+0000","last_became_peered":"2023-07-21T18:41:08.149966+0000","last_unstale":"2023-07-21T18:41:08.156947+0000","last_undegraded":"2023-07-21T18:41:08.156947+0000","last_fullsized":"2023-07-21T18:41:08.156947+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:17:05.137573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.149731+0000","last_change":"2023-07-21T18:40:04.791937+0000","last_active":"2023-07-21T18:41:08.149731+0000","last_peered":"2023-07-21T18:41:08.149731+0000","last_clean":"2023-07-21T18:41:08.149731+0000","last_became_active":"2023-07-21T18:40:04.791588+0000","last_became_peered":"2023-07-21T18:40:04.791588+0000","last_unstale":"2023-07-21T18:41:08.149731+0000","last_undegraded":"2023-07-21T18:41:08.149731+0000","last_fullsized":"2023-07-21T18:41:08.149731+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:17:39.327055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'7","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.244822+0000","last_change":"2023-07-21T18:40:04.177007+0000","last_active":"2023-07-21T18:41:08.244822+0000","last_peered":"2023-07-21T18:41:08.244822+0000","last_clean":"2023-07-21T18:41:08.244822+0000","last_became_active":"2023-07-21T18:40:04.176699+0000","last_became_peered":"2023-07-21T18:40:04.176699+0000","last_unstale":"2023-07-21T18:41:08.244822+0000","last_undegraded":"2023-07-21T18:41:08.244822+0000","last_fullsized":"2023-07-21T18:41:08.244822+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:20:14.547927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.155110+0000","last_change":"2023-07-21T18:40:04.795671+0000","last_active":"2023-07-21T18:41:08.155110+0000","last_peered":"2023-07-21T18:41:08.155110+0000","last_clean":"2023-07-21T18:41:08.155110+0000","last_became_active":"2023-07-21T18:40:04.795233+0000","last_became_peered":"2023-07-21T18:40:04.795233+0000","last_unstale":"2023-07-21T18:41:08.155110+0000","last_undegraded":"2023-07-21T18:41:08.155110+0000","last_fullsized":"2023-07-21T18:41:08.155110+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:57:42.725756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.159479+0000","last_change":"2023-07-21T18:41:08.159479+0000","last_active":"2023-07-21T18:41:08.159479+0000","last_peered":"2023-07-21T18:41:08.159479+0000","last_clean":"2023-07-21T18:41:08.159479+0000","last_became_active":"2023-07-21T18:41:08.156027+0000","last_became_peered":"2023-07-21T18:41:08.156027+0000","last_unstale":"2023-07-21T18:41:08.159479+0000","last_undegraded":"2023-07-21T18:41:08.159479+0000","last_fullsized":"2023-07-21T18:41:08.159479+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:36:55.454682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.148310+0000","last_change":"2023-07-21T18:40:04.793091+0000","last_active":"2023-07-21T18:41:08.148310+0000","last_peered":"2023-07-21T18:41:08.148310+0000","last_clean":"2023-07-21T18:41:08.148310+0000","last_became_active":"2023-07-21T18:40:04.792723+0000","last_became_peered":"2023-07-21T18:40:04.792723+0000","last_unstale":"2023-07-21T18:41:08.148310+0000","last_undegraded":"2023-07-21T18:41:08.148310+0000","last_fullsized":"2023-07-21T18:41:08.148310+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:25:52.341093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'7","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.149262+0000","last_change":"2023-07-21T18:40:04.182867+0000","last_active":"2023-07-21T18:41:08.149262+0000","last_peered":"2023-07-21T18:41:08.149262+0000","last_clean":"2023-07-21T18:41:08.149262+0000","last_became_active":"2023-07-21T18:40:04.181885+0000","last_became_peered":"2023-07-21T18:40:04.181885+0000","last_unstale":"2023-07-21T18:41:08.149262+0000","last_undegraded":"2023-07-21T18:41:08.149262+0000","last_fullsized":"2023-07-21T18:41:08.149262+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:44:58.697667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.143574+0000","last_change":"2023-07-21T18:40:04.790340+0000","last_active":"2023-07-21T18:41:08.143574+0000","last_peered":"2023-07-21T18:41:08.143574+0000","last_clean":"2023-07-21T18:41:08.143574+0000","last_became_active":"2023-07-21T18:40:04.790014+0000","last_became_peered":"2023-07-21T18:40:04.790014+0000","last_unstale":"2023-07-21T18:41:08.143574+0000","last_undegraded":"2023-07-21T18:41:08.143574+0000","last_fullsized":"2023-07-21T18:41:08.143574+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:01:01.514954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'3","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.152586+0000","last_change":"2023-07-21T18:40:04.184986+0000","last_active":"2023-07-21T18:41:08.152586+0000","last_peered":"2023-07-21T18:41:08.152586+0000","last_clean":"2023-07-21T18:41:08.152586+0000","last_became_active":"2023-07-21T18:40:04.184672+0000","last_became_peered":"2023-07-21T18:40:04.184672+0000","last_unstale":"2023-07-21T18:41:08.152586+0000","last_undegraded":"2023-07-21T18:41:08.152586+0000","last_fullsized":"2023-07-21T18:41:08.152586+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:56:51.290795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.149573+0000","last_change":"2023-07-21T18:40:04.793272+0000","last_active":"2023-07-21T18:41:08.149573+0000","last_peered":"2023-07-21T18:41:08.149573+0000","last_clean":"2023-07-21T18:41:08.149573+0000","last_became_active":"2023-07-21T18:40:04.792854+0000","last_became_peered":"2023-07-21T18:40:04.792854+0000","last_unstale":"2023-07-21T18:41:08.149573+0000","last_undegraded":"2023-07-21T18:41:08.149573+0000","last_fullsized":"2023-07-21T18:41:08.149573+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:13:19.927149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.148619+0000","last_change":"2023-07-21T18:40:04.176404+0000","last_active":"2023-07-21T18:41:08.148619+0000","last_peered":"2023-07-21T18:41:08.148619+0000","last_clean":"2023-07-21T18:41:08.148619+0000","last_became_active":"2023-07-21T18:40:04.176148+0000","last_became_peered":"2023-07-21T18:40:04.176148+0000","last_unstale":"2023-07-21T18:41:08.148619+0000","last_undegraded":"2023-07-21T18:41:08.148619+0000","last_fullsized":"2023-07-21T18:41:08.148619+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:06:45.638197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.595565+0000","last_change":"2023-07-21T18:40:04.795628+0000","last_active":"2023-07-21T18:41:08.595565+0000","last_peered":"2023-07-21T18:41:08.595565+0000","last_clean":"2023-07-21T18:41:08.595565+0000","last_became_active":"2023-07-21T18:40:04.795248+0000","last_became_peered":"2023-07-21T18:40:04.795248+0000","last_unstale":"2023-07-21T18:41:08.595565+0000","last_undegraded":"2023-07-21T18:41:08.595565+0000","last_fullsized":"2023-07-21T18:41:08.595565+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:54:09.726387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'5","reported_seq":103,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:27.081526+0000","last_change":"2023-07-21T18:40:04.184380+0000","last_active":"2023-07-21T18:41:27.081526+0000","last_peered":"2023-07-21T18:41:27.081526+0000","last_clean":"2023-07-21T18:41:27.081526+0000","last_became_active":"2023-07-21T18:40:04.184045+0000","last_became_peered":"2023-07-21T18:40:04.184045+0000","last_unstale":"2023-07-21T18:41:27.081526+0000","last_undegraded":"2023-07-21T18:41:27.081526+0000","last_fullsized":"2023-07-21T18:41:27.081526+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:33:19.018900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":80,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.595539+0000","last_change":"2023-07-21T18:40:04.787760+0000","last_active":"2023-07-21T18:41:08.595539+0000","last_peered":"2023-07-21T18:41:08.595539+0000","last_clean":"2023-07-21T18:41:08.595539+0000","last_became_active":"2023-07-21T18:40:04.787371+0000","last_became_peered":"2023-07-21T18:40:04.787371+0000","last_unstale":"2023-07-21T18:41:08.595539+0000","last_undegraded":"2023-07-21T18:41:08.595539+0000","last_fullsized":"2023-07-21T18:41:08.595539+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:24:52.752764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'3","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.248814+0000","last_change":"2023-07-21T18:40:04.184721+0000","last_active":"2023-07-21T18:41:08.248814+0000","last_peered":"2023-07-21T18:41:08.248814+0000","last_clean":"2023-07-21T18:41:08.248814+0000","last_became_active":"2023-07-21T18:40:04.184070+0000","last_became_peered":"2023-07-21T18:40:04.184070+0000","last_unstale":"2023-07-21T18:41:08.248814+0000","last_undegraded":"2023-07-21T18:41:08.248814+0000","last_fullsized":"2023-07-21T18:41:08.248814+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:57:29.552146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"79'1","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.595437+0000","last_change":"2023-07-21T18:40:04.786963+0000","last_active":"2023-07-21T18:41:08.595437+0000","last_peered":"2023-07-21T18:41:08.595437+0000","last_clean":"2023-07-21T18:41:08.595437+0000","last_became_active":"2023-07-21T18:40:04.786617+0000","last_became_peered":"2023-07-21T18:40:04.786617+0000","last_unstale":"2023-07-21T18:41:08.595437+0000","last_undegraded":"2023-07-21T18:41:08.595437+0000","last_fullsized":"2023-07-21T18:41:08.595437+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:28:47.943499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'5","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.144548+0000","last_change":"2023-07-21T18:40:04.184509+0000","last_active":"2023-07-21T18:41:08.144548+0000","last_peered":"2023-07-21T18:41:08.144548+0000","last_clean":"2023-07-21T18:41:08.144548+0000","last_became_active":"2023-07-21T18:40:04.184208+0000","last_became_peered":"2023-07-21T18:40:04.184208+0000","last_unstale":"2023-07-21T18:41:08.144548+0000","last_undegraded":"2023-07-21T18:41:08.144548+0000","last_fullsized":"2023-07-21T18:41:08.144548+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:21:46.199399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.595579+0000","last_change":"2023-07-21T18:40:04.788248+0000","last_active":"2023-07-21T18:41:08.595579+0000","last_peered":"2023-07-21T18:41:08.595579+0000","last_clean":"2023-07-21T18:41:08.595579+0000","last_became_active":"2023-07-21T18:40:04.787974+0000","last_became_peered":"2023-07-21T18:40:04.787974+0000","last_unstale":"2023-07-21T18:41:08.595579+0000","last_undegraded":"2023-07-21T18:41:08.595579+0000","last_fullsized":"2023-07-21T18:41:08.595579+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:28:27.418555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.143620+0000","last_change":"2023-07-21T18:40:04.790522+0000","last_active":"2023-07-21T18:41:08.143620+0000","last_peered":"2023-07-21T18:41:08.143620+0000","last_clean":"2023-07-21T18:41:08.143620+0000","last_became_active":"2023-07-21T18:40:04.790178+0000","last_became_peered":"2023-07-21T18:40:04.790178+0000","last_unstale":"2023-07-21T18:41:08.143620+0000","last_undegraded":"2023-07-21T18:41:08.143620+0000","last_fullsized":"2023-07-21T18:41:08.143620+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:06:13.808345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.152539+0000","last_change":"2023-07-21T18:40:04.790403+0000","last_active":"2023-07-21T18:41:08.152539+0000","last_peered":"2023-07-21T18:41:08.152539+0000","last_clean":"2023-07-21T18:41:08.152539+0000","last_became_active":"2023-07-21T18:40:04.790106+0000","last_became_peered":"2023-07-21T18:40:04.790106+0000","last_unstale":"2023-07-21T18:41:08.152539+0000","last_undegraded":"2023-07-21T18:41:08.152539+0000","last_fullsized":"2023-07-21T18:41:08.152539+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:04:45.154675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.152548+0000","last_change":"2023-07-21T18:40:04.795230+0000","last_active":"2023-07-21T18:41:08.152548+0000","last_peered":"2023-07-21T18:41:08.152548+0000","last_clean":"2023-07-21T18:41:08.152548+0000","last_became_active":"2023-07-21T18:40:04.794949+0000","last_became_peered":"2023-07-21T18:40:04.794949+0000","last_unstale":"2023-07-21T18:41:08.152548+0000","last_undegraded":"2023-07-21T18:41:08.152548+0000","last_fullsized":"2023-07-21T18:41:08.152548+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T18:42:31.029744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.154417+0000","last_change":"2023-07-21T18:40:04.792736+0000","last_active":"2023-07-21T18:41:08.154417+0000","last_peered":"2023-07-21T18:41:08.154417+0000","last_clean":"2023-07-21T18:41:08.154417+0000","last_became_active":"2023-07-21T18:40:04.792301+0000","last_became_peered":"2023-07-21T18:40:04.792301+0000","last_unstale":"2023-07-21T18:41:08.154417+0000","last_undegraded":"2023-07-21T18:41:08.154417+0000","last_fullsized":"2023-07-21T18:41:08.154417+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:45:51.320918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.145009+0000","last_change":"2023-07-21T18:40:04.790733+0000","last_active":"2023-07-21T18:41:08.145009+0000","last_peered":"2023-07-21T18:41:08.145009+0000","last_clean":"2023-07-21T18:41:08.145009+0000","last_became_active":"2023-07-21T18:40:04.790441+0000","last_became_peered":"2023-07-21T18:40:04.790441+0000","last_unstale":"2023-07-21T18:41:08.145009+0000","last_undegraded":"2023-07-21T18:41:08.145009+0000","last_fullsized":"2023-07-21T18:41:08.145009+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:40:00.802774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.143768+0000","last_change":"2023-07-21T18:40:04.790853+0000","last_active":"2023-07-21T18:41:08.143768+0000","last_peered":"2023-07-21T18:41:08.143768+0000","last_clean":"2023-07-21T18:41:08.143768+0000","last_became_active":"2023-07-21T18:40:04.790373+0000","last_became_peered":"2023-07-21T18:40:04.790373+0000","last_unstale":"2023-07-21T18:41:08.143768+0000","last_undegraded":"2023-07-21T18:41:08.143768+0000","last_fullsized":"2023-07-21T18:41:08.143768+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:29:27.642249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.153055+0000","last_change":"2023-07-21T18:40:04.794088+0000","last_active":"2023-07-21T18:41:08.153055+0000","last_peered":"2023-07-21T18:41:08.153055+0000","last_clean":"2023-07-21T18:41:08.153055+0000","last_became_active":"2023-07-21T18:40:04.793782+0000","last_became_peered":"2023-07-21T18:40:04.793782+0000","last_unstale":"2023-07-21T18:41:08.153055+0000","last_undegraded":"2023-07-21T18:41:08.153055+0000","last_fullsized":"2023-07-21T18:41:08.153055+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:10:46.116604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.149451+0000","last_change":"2023-07-21T18:40:04.791772+0000","last_active":"2023-07-21T18:41:08.149451+0000","last_peered":"2023-07-21T18:41:08.149451+0000","last_clean":"2023-07-21T18:41:08.149451+0000","last_became_active":"2023-07-21T18:40:04.791379+0000","last_became_peered":"2023-07-21T18:40:04.791379+0000","last_unstale":"2023-07-21T18:41:08.149451+0000","last_undegraded":"2023-07-21T18:41:08.149451+0000","last_fullsized":"2023-07-21T18:41:08.149451+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:01:22.491116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.248658+0000","last_change":"2023-07-21T18:40:04.786579+0000","last_active":"2023-07-21T18:41:08.248658+0000","last_peered":"2023-07-21T18:41:08.248658+0000","last_clean":"2023-07-21T18:41:08.248658+0000","last_became_active":"2023-07-21T18:40:04.786032+0000","last_became_peered":"2023-07-21T18:40:04.786032+0000","last_unstale":"2023-07-21T18:41:08.248658+0000","last_undegraded":"2023-07-21T18:41:08.248658+0000","last_fullsized":"2023-07-21T18:41:08.248658+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:26:58.238429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.149279+0000","last_change":"2023-07-21T18:40:04.793441+0000","last_active":"2023-07-21T18:41:08.149279+0000","last_peered":"2023-07-21T18:41:08.149279+0000","last_clean":"2023-07-21T18:41:08.149279+0000","last_became_active":"2023-07-21T18:40:04.791441+0000","last_became_peered":"2023-07-21T18:40:04.791441+0000","last_unstale":"2023-07-21T18:41:08.149279+0000","last_undegraded":"2023-07-21T18:41:08.149279+0000","last_fullsized":"2023-07-21T18:41:08.149279+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T18:47:16.762634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.151030+0000","last_change":"2023-07-21T18:40:04.794913+0000","last_active":"2023-07-21T18:41:08.151030+0000","last_peered":"2023-07-21T18:41:08.151030+0000","last_clean":"2023-07-21T18:41:08.151030+0000","last_became_active":"2023-07-21T18:40:04.794527+0000","last_became_peered":"2023-07-21T18:40:04.794527+0000","last_unstale":"2023-07-21T18:41:08.151030+0000","last_undegraded":"2023-07-21T18:41:08.151030+0000","last_fullsized":"2023-07-21T18:41:08.151030+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:14:55.276602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.144282+0000","last_change":"2023-07-21T18:40:04.789145+0000","last_active":"2023-07-21T18:41:08.144282+0000","last_peered":"2023-07-21T18:41:08.144282+0000","last_clean":"2023-07-21T18:41:08.144282+0000","last_became_active":"2023-07-21T18:40:04.788285+0000","last_became_peered":"2023-07-21T18:40:04.788285+0000","last_unstale":"2023-07-21T18:41:08.144282+0000","last_undegraded":"2023-07-21T18:41:08.144282+0000","last_fullsized":"2023-07-21T18:41:08.144282+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:50:38.851180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.143670+0000","last_change":"2023-07-21T18:40:04.788821+0000","last_active":"2023-07-21T18:41:08.143670+0000","last_peered":"2023-07-21T18:41:08.143670+0000","last_clean":"2023-07-21T18:41:08.143670+0000","last_became_active":"2023-07-21T18:40:04.788513+0000","last_became_peered":"2023-07-21T18:40:04.788513+0000","last_unstale":"2023-07-21T18:41:08.143670+0000","last_undegraded":"2023-07-21T18:41:08.143670+0000","last_fullsized":"2023-07-21T18:41:08.143670+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:37:17.944763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64131,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":817,"num_read_kb":477,"num_write":233,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":191757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":204,"ondisk_log_size":204,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579933,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27816,"kb_used_data":864,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717336,"statfs":{"total":95995035648,"available":95966552064,"internally_reserved":0,"allocated":884736,"data_stored":426096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:41:24 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.54300000000000004,"15min":0.54700000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.78600000000000003,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.629,"15min":0.63900000000000001},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.748,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.54700000000000004}]},{"osd":1,"last update":"Fri Jul 21 18:41:24 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.63900000000000001,"15min":0.63300000000000001},"min":{"1min":0.50800000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.747},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.61699999999999999,"15min":0.61099999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.85899999999999999,"5min":1.137,"15min":1.137},"last":0.64000000000000001}]},{"osd":2,"last update":"Fri Jul 21 18:41:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.502,"15min":0.495},"min":{"1min":0.378,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.502,"5min":0.47299999999999998,"15min":0.46800000000000003},"min":{"1min":0.32600000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.64400000000000002,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.60299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":4,"last update":"Fri Jul 21 18:41:24 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.64300000000000002,"15min":0.63300000000000001},"min":{"1min":0.45700000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.60199999999999998,"15min":0.59099999999999997},"min":{"1min":0.46700000000000003,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.79300000000000004}]},{"osd":5,"last update":"Fri Jul 21 18:41:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.56799999999999995,"15min":0.56000000000000005},"min":{"1min":0.50600000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.57299999999999995,"15min":0.56599999999999995},"min":{"1min":0.39300000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.75700000000000001,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.81200000000000006}]},{"osd":6,"last update":"Fri Jul 21 18:41:24 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.64800000000000002,"15min":0.65100000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.88700000000000001,"5min":1.181,"15min":1.181},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.755,"15min":0.75900000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.92500000000000004,"5min":1.425,"15min":1.425},"last":0.88400000000000001}]},{"osd":7,"last update":"Fri Jul 21 18:41:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.72799999999999998,"15min":0.71999999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.91200000000000003,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.752,"5min":0.70999999999999996,"15min":0.70299999999999996},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.91500000000000004,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.98099999999999998}]},{"osd":8,"last update":"Fri Jul 21 18:41:06 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.999}]},{"osd":9,"last update":"Fri Jul 21 18:41:24 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.80500000000000005,"15min":0.80200000000000005},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.98799999999999999,"5min":1.643,"15min":1.643},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.78800000000000003,"15min":0.78400000000000003},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.95899999999999996,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":1.0269999999999999}]},{"osd":10,"last update":"Fri Jul 21 18:41:24 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.76900000000000002,"15min":0.76200000000000001},"min":{"1min":0.56000000000000005,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.95899999999999996,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.56999999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.98099999999999998,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.96499999999999997}]}]},{"osd":10,"up_from":67,"seq":287762808864,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28492,"kb_used_data":1540,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716660,"statfs":{"total":95995035648,"available":95965859840,"internally_reserved":0,"allocated":1576960,"data_stored":1074383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:41:08 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.40999999999999998}]},{"osd":1,"last update":"Fri Jul 21 18:41:08 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.46800000000000003,"15min":0.46600000000000003},"min":{"1min":0.29399999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.68600000000000005,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.46500000000000002,"15min":0.46100000000000002},"min":{"1min":0.33200000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.624,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.49099999999999999}]},{"osd":2,"last update":"Fri Jul 21 18:41:08 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.60699999999999998,"15min":0.56200000000000006},"min":{"1min":0.54100000000000004,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":4.0129999999999999,"5min":4.0129999999999999,"15min":4.0129999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.622,"15min":0.60999999999999999},"min":{"1min":0.45600000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.82799999999999996,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.59199999999999997}]},{"osd":3,"last update":"Fri Jul 21 18:41:08 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.60499999999999998,"15min":0.60099999999999998},"min":{"1min":0.434,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.746,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.621,"5min":0.60499999999999998,"15min":0.60199999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.439}]},{"osd":4,"last update":"Fri Jul 21 18:41:08 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.57699999999999996,"15min":0.57199999999999995},"min":{"1min":0.42599999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.73099999999999998,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.59099999999999997,"15min":0.59299999999999997},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.84699999999999998,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.52600000000000002}]},{"osd":5,"last update":"Fri Jul 21 18:41:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.69299999999999995,"15min":0.68700000000000006},"min":{"1min":0.49399999999999999,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.69699999999999995,"15min":0.68300000000000005},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.621}]},{"osd":6,"last update":"Fri Jul 21 18:41:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.69299999999999995,"15min":0.68899999999999995},"min":{"1min":0.505,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.874,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.46999999999999997,"5min":0.442,"15min":0.442},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.60499999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":8,"last update":"Fri Jul 21 18:41:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.73699999999999999,"15min":0.72899999999999998},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.65700000000000003},{"interface":"front","average":{"1min":1.01,"5min":0.81100000000000005,"15min":0.77800000000000002},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":4.0419999999999998,"5min":4.0419999999999998,"15min":4.0419999999999998},"last":0.72299999999999998}]},{"osd":9,"last update":"Fri Jul 21 18:41:08 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.75900000000000001,"15min":0.75700000000000001},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.75,"5min":0.77800000000000002,"15min":0.78200000000000003},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.92100000000000004,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.68899999999999995}]},{"osd":11,"last update":"Fri Jul 21 18:40:28 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.75}]}]},{"osd":9,"up_from":62,"seq":266287972389,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27840,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717312,"statfs":{"total":95995035648,"available":95966527488,"internally_reserved":0,"allocated":909312,"data_stored":426074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.52500000000000002,"15min":0.53900000000000003},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.996,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.438,"5min":0.505,"15min":0.51600000000000001},"min":{"1min":0.316,"5min":0.252,"15min":0.252},"max":{"1min":0.621,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.33100000000000002}]},{"osd":1,"last update":"Fri Jul 21 18:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.73199999999999998,"15min":0.749},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.099,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":0.627},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.71299999999999997,"15min":0.72099999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.169,"5min":1.681,"15min":1.681},"last":0.59899999999999998}]},{"osd":2,"last update":"Fri Jul 21 18:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.70799999999999996,"15min":0.71399999999999997},"min":{"1min":0.47299999999999998,"5min":0.38,"15min":0.38},"max":{"1min":1.222,"5min":1.9530000000000001,"15min":1.9530000000000001},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.78300000000000003,"15min":0.753},"min":{"1min":0.46100000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":6.4020000000000001,"5min":6.4020000000000001,"15min":6.4020000000000001},"last":0.63900000000000001}]},{"osd":3,"last update":"Fri Jul 21 18:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.627,"15min":0.64900000000000002},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.89000000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.627,"15min":0.64500000000000002},"min":{"1min":0.35199999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.1910000000000001,"5min":2.048,"15min":2.048},"last":0.38}]},{"osd":4,"last update":"Fri Jul 21 18:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.80700000000000005,"15min":0.83699999999999997},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.127,"5min":1.8100000000000001,"15min":1.8100000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.79800000000000004,"15min":0.82299999999999995},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.94799999999999995,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.748}]},{"osd":5,"last update":"Fri Jul 21 18:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.80500000000000005,"15min":0.77700000000000002},"min":{"1min":0.49299999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":6.5069999999999997,"5min":6.5069999999999997,"15min":6.5069999999999997},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.78600000000000003,"15min":0.80400000000000005},"min":{"1min":0.503,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.1399999999999999,"5min":1.992,"15min":1.992},"last":0.56299999999999994}]},{"osd":6,"last update":"Fri Jul 21 18:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.74299999999999999,"15min":0.77300000000000002},"min":{"1min":0.375,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.1240000000000001,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.78600000000000003,"15min":0.81399999999999995},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.2410000000000001,"5min":1.595,"15min":1.595},"last":0.77900000000000003}]},{"osd":7,"last update":"Fri Jul 21 18:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.85799999999999998,"15min":0.88200000000000001},"min":{"1min":0.54700000000000004,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.026,"5min":1.284,"15min":1.284},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.89300000000000002,"15min":0.92000000000000004},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.175,"5min":2.0750000000000002,"15min":2.0750000000000002},"last":0.60999999999999999}]},{"osd":8,"last update":"Fri Jul 21 18:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.90700000000000003,"15min":0.92900000000000005},"min":{"1min":0.61499999999999999,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.2749999999999999,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":1.0669999999999999,"5min":0.90600000000000003,"15min":0.879},"min":{"1min":0.60899999999999999,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":6.5679999999999996,"5min":6.5679999999999996,"15min":6.5679999999999996},"last":0.79900000000000004}]},{"osd":10,"last update":"Fri Jul 21 18:41:08 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.86699999999999999,"15min":0.88700000000000001},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.155,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.86399999999999999,"15min":0.88200000000000001},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":0.76500000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78900000000000003}]}]},{"osd":8,"up_from":55,"seq":236223201322,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27832,"kb_used_data":880,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717320,"statfs":{"total":95995035648,"available":95966535680,"internally_reserved":0,"allocated":901120,"data_stored":426121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:41:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.36299999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.81899999999999995,"5min":4.2000000000000002,"15min":4.2000000000000002},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.58699999999999997,"15min":0.58399999999999996},"min":{"1min":0.33200000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.747,"5min":3.145,"15min":3.145},"last":0.50800000000000001}]},{"osd":1,"last update":"Fri Jul 21 18:41:27 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.622,"15min":0.59199999999999997},"min":{"1min":0.45500000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.91300000000000003,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.627,"15min":0.61699999999999999},"min":{"1min":0.49399999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.92600000000000005,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.87}]},{"osd":2,"last update":"Fri Jul 21 18:41:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.47999999999999998,"15min":0.46500000000000002},"min":{"1min":0.39400000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.48699999999999999,"15min":0.47099999999999997},"min":{"1min":0.34399999999999997,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.77700000000000002,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.85199999999999998}]},{"osd":3,"last update":"Fri Jul 21 18:41:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.72799999999999998,"15min":0.68300000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.85099999999999998,"5min":11.260999999999999,"15min":11.260999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.61799999999999999,"15min":0.62},"min":{"1min":0.441,"5min":0.371,"15min":0.371},"max":{"1min":0.72499999999999998,"5min":3.2719999999999998,"15min":3.2719999999999998},"last":0.56200000000000006}]},{"osd":4,"last update":"Fri Jul 21 18:41:27 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.58399999999999996,"15min":0.56999999999999995},"min":{"1min":0.46899999999999997,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.80000000000000004,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.64600000000000002,"15min":0.63400000000000001},"min":{"1min":0.54600000000000004,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.86299999999999999,"5min":0.875,"15min":0.875},"last":0.82699999999999996}]},{"osd":5,"last update":"Fri Jul 21 18:41:27 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.59399999999999997,"15min":0.58699999999999997},"min":{"1min":0.308,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.53900000000000003,"15min":0.53500000000000003},"min":{"1min":0.36599999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.70299999999999996}]},{"osd":6,"last update":"Fri Jul 21 18:41:27 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.78400000000000003,"15min":0.73699999999999999},"min":{"1min":0.47299999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.98099999999999998,"5min":10.292999999999999,"15min":10.292999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.76600000000000001},"min":{"1min":0.53600000000000003,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.018,"5min":4.1029999999999998,"15min":4.1029999999999998},"last":0.90300000000000002}]},{"osd":7,"last update":"Fri Jul 21 18:41:27 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.73099999999999998,"15min":0.72099999999999997},"min":{"1min":0.56699999999999995,"5min":0.432,"15min":0.432},"max":{"1min":1.0029999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.67900000000000005,"15min":0.67500000000000004},"min":{"1min":0.497,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.88700000000000001}]},{"osd":9,"last update":"Fri Jul 21 18:40:46 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.54100000000000004,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.95199999999999996,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.92000000000000004},{"interface":"front","average":{"1min":1.2509999999999999,"5min":0.86699999999999999,"15min":0.80300000000000005},"min":{"1min":0.54900000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":11.151,"5min":11.151,"15min":11.151},"last":0.98299999999999998}]},{"osd":10,"last update":"Fri Jul 21 18:41:05 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.71599999999999997,"15min":0.69799999999999995},"min":{"1min":0.59899999999999998,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.996},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.73499999999999999,"15min":0.71399999999999997},"min":{"1min":0.73599999999999999,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":1.0089999999999999}]},{"osd":11,"last update":"Fri Jul 21 18:41:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.96199999999999997}]}]},{"osd":7,"up_from":48,"seq":206158430254,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28416,"kb_used_data":1464,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716736,"statfs":{"total":95995035648,"available":95965937664,"internally_reserved":0,"allocated":1499136,"data_stored":1018876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.53400000000000003,"15min":0.52600000000000002},"min":{"1min":0.41599999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.53900000000000003,"15min":0.53600000000000003},"min":{"1min":0.34000000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.48899999999999999}]},{"osd":1,"last update":"Fri Jul 21 18:41:03 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.42099999999999999,"15min":0.40699999999999997},"min":{"1min":0.29199999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.70799999999999996,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.442,"15min":0.42599999999999999},"min":{"1min":0.32400000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.66700000000000004,"5min":0.75,"15min":0.75},"last":0.379}]},{"osd":2,"last update":"Fri Jul 21 18:41:03 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.59299999999999997,"15min":0.57699999999999996},"min":{"1min":0.41899999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.89700000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.60799999999999998,"15min":0.59599999999999997},"min":{"1min":0.47999999999999998,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.83299999999999996,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.73699999999999999}]},{"osd":3,"last update":"Fri Jul 21 18:41:03 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.55900000000000005,"15min":0.54300000000000004},"min":{"1min":0.42599999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.70499999999999996,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.59099999999999997,"15min":0.57499999999999996},"min":{"1min":0.433,"5min":0.372,"15min":0.372},"max":{"1min":0.81200000000000006,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.63100000000000001}]},{"osd":4,"last update":"Fri Jul 21 18:41:03 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53000000000000003,"15min":0.52200000000000002},"min":{"1min":0.39300000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.73499999999999999,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.51900000000000002,"15min":0.51500000000000001},"min":{"1min":0.33800000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.60999999999999999,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.40200000000000002}]},{"osd":5,"last update":"Fri Jul 21 18:41:03 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.66400000000000003,"15min":0.64600000000000002},"min":{"1min":0.51700000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.65700000000000003,"15min":0.64600000000000002},"min":{"1min":0.505,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.56200000000000006}]},{"osd":6,"last update":"Fri Jul 21 18:41:03 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66800000000000004,"15min":0.65600000000000003},"min":{"1min":0.51600000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.86899999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.67700000000000005,"15min":0.66300000000000003},"min":{"1min":0.56299999999999994,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.97399999999999998,"5min":1.087,"15min":1.087},"last":0.59999999999999998}]},{"osd":8,"last update":"Fri Jul 21 18:41:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.70799999999999996,"15min":0.69499999999999995},"min":{"1min":0.64500000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.91400000000000003,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.72899999999999998,"15min":0.71899999999999997},"min":{"1min":0.57399999999999995,"5min":0.495,"15min":0.495},"max":{"1min":0.93799999999999994,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.78900000000000003}]},{"osd":9,"last update":"Fri Jul 21 18:40:46 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.74199999999999999,"15min":0.73499999999999999},"min":{"1min":0.46500000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.98099999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.753},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.74199999999999999,"15min":0.73599999999999999},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.83499999999999996}]},{"osd":10,"last update":"Fri Jul 21 18:41:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.75800000000000001,"15min":0.76200000000000001},"min":{"1min":0.50900000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.94899999999999995,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.746,"15min":0.74399999999999999},"min":{"1min":0.63,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":0.95399999999999996,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.68999999999999995}]},{"osd":11,"last update":"Fri Jul 21 18:41:04 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.77700000000000002}]}]},{"osd":6,"up_from":43,"seq":184683593778,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27820,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717332,"statfs":{"total":95995035648,"available":95966547968,"internally_reserved":0,"allocated":888832,"data_stored":425794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:40:40 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.41299999999999998,"15min":0.40899999999999997},"min":{"1min":0.29599999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.313},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.48099999999999998,"15min":0.44500000000000001},"min":{"1min":0.311,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":4.859,"5min":4.859,"15min":4.859},"last":0.37}]},{"osd":1,"last update":"Fri Jul 21 18:40:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.44500000000000001,"5min":0.37,"15min":0.37},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.625},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.39000000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.84399999999999997,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.56599999999999995}]},{"osd":2,"last update":"Fri Jul 21 18:40:40 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.64800000000000002,"15min":0.60399999999999998},"min":{"1min":0.39900000000000002,"5min":0.376,"15min":0.376},"max":{"1min":4.9180000000000001,"5min":4.9180000000000001,"15min":4.9180000000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.59599999999999997,"15min":0.58199999999999996},"min":{"1min":0.42099999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.502}]},{"osd":3,"last update":"Fri Jul 21 18:40:40 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.52300000000000002,"15min":0.48799999999999999},"min":{"1min":0.33800000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":4.4720000000000004,"5min":4.4720000000000004,"15min":4.4720000000000004},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.56999999999999995,"15min":0.53100000000000003},"min":{"1min":0.32200000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":5.2910000000000004,"5min":5.2910000000000004,"15min":5.2910000000000004},"last":0.40799999999999997}]},{"osd":4,"last update":"Fri Jul 21 18:40:40 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.63900000000000001,"15min":0.64400000000000002},"min":{"1min":0.40300000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.022,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.64100000000000001,"15min":0.64400000000000002},"min":{"1min":0.46500000000000002,"5min":0.442,"15min":0.442},"max":{"1min":0.89600000000000002,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.61199999999999999}]},{"osd":5,"last update":"Fri Jul 21 18:40:40 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.70699999999999996,"15min":0.66600000000000004},"min":{"1min":0.47499999999999998,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":4.9960000000000004,"5min":4.9960000000000004,"15min":4.9960000000000004},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.70299999999999996,"15min":0.69499999999999995},"min":{"1min":0.49299999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.145,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.53900000000000003}]},{"osd":7,"last update":"Fri Jul 21 18:41:03 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.70199999999999996,"15min":0.70499999999999996},"min":{"1min":0.495,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.874,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.75,"15min":0.71999999999999997},"min":{"1min":0.55200000000000005,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":5.0330000000000004,"5min":5.0330000000000004,"15min":5.0330000000000004},"last":0.69699999999999995}]},{"osd":8,"last update":"Fri Jul 21 18:40:26 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.79000000000000004,"15min":0.80600000000000005},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.96099999999999997,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.77500000000000002,"15min":0.78700000000000003},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.91800000000000004,"5min":1.171,"15min":1.171},"last":0.68000000000000005}]},{"osd":9,"last update":"Fri Jul 21 18:40:44 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.75,"15min":0.75600000000000001},"min":{"1min":0.56799999999999995,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.194,"5min":1.196,"15min":1.196},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.72599999999999998,"15min":0.73399999999999999},"min":{"1min":0.52600000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.872,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.71199999999999997}]},{"osd":10,"last update":"Fri Jul 21 18:41:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.78300000000000003,"15min":0.79100000000000004},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.86499999999999999,"5min":1.216,"15min":1.216},"last":0.749},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.80500000000000005,"15min":0.77900000000000003},"min":{"1min":0.60699999999999998,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":4.9580000000000002,"5min":4.9580000000000002,"15min":4.9580000000000002},"last":0.72199999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574919,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":924,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":946176,"data_stored":483258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:41:04 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.52300000000000002,"15min":0.48899999999999999},"min":{"1min":0.377,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.77100000000000002,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.56899999999999995,"15min":0.51100000000000001},"min":{"1min":0.38500000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":3.7690000000000001,"5min":3.7690000000000001,"15min":3.7690000000000001},"last":0.58799999999999997}]},{"osd":2,"last update":"Fri Jul 21 18:41:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.59299999999999997,"15min":0.53900000000000003},"min":{"1min":0.49399999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.85099999999999998,"5min":3.2959999999999998,"15min":3.2959999999999998},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.59399999999999997,"15min":0.54000000000000004},"min":{"1min":0.38200000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.85099999999999998,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":0.79800000000000004}]},{"osd":3,"last update":"Fri Jul 21 18:40:44 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.55300000000000005,"15min":0.52500000000000002},"min":{"1min":0.38400000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.58099999999999996,"15min":0.53700000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":3.7789999999999999,"5min":3.7789999999999999,"15min":3.7789999999999999},"last":0.622}]},{"osd":4,"last update":"Fri Jul 21 18:41:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.45100000000000001,"15min":0.41199999999999998},"min":{"1min":0.32400000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.44800000000000001,"15min":0.41499999999999998},"min":{"1min":0.28000000000000003,"5min":0.254,"15min":0.254},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.55900000000000005}]},{"osd":5,"last update":"Fri Jul 21 18:41:20 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.64800000000000002,"15min":0.60199999999999998},"min":{"1min":0.56399999999999995,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.97999999999999998,"5min":1.653,"15min":1.653},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.67100000000000004,"15min":0.61399999999999999},"min":{"1min":0.65700000000000003,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.95899999999999996,"5min":1.641,"15min":1.641},"last":0.85099999999999998}]},{"osd":6,"last update":"Fri Jul 21 18:40:44 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.64400000000000002,"15min":0.63200000000000001},"min":{"1min":0.46200000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.92900000000000005,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.64500000000000002,"15min":0.64000000000000001},"min":{"1min":0.41599999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.96099999999999997,"5min":1.022,"15min":1.022},"last":0.81699999999999995}]},{"osd":7,"last update":"Fri Jul 21 18:41:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.55600000000000005,"15min":0.51400000000000001},"min":{"1min":0.33200000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.749,"5min":0.59399999999999997,"15min":0.54500000000000004},"min":{"1min":0.35999999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.94699999999999995,"5min":1.014,"15min":1.014},"last":0.71999999999999997}]},{"osd":8,"last update":"Fri Jul 21 18:40:26 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.73599999999999999,"15min":0.72799999999999998},"min":{"1min":0.51500000000000001,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.1910000000000001,"5min":2.7360000000000002,"15min":2.7360000000000002},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.72699999999999998,"15min":0.72499999999999998},"min":{"1min":0.52400000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.96899999999999997,"5min":2.7210000000000001,"15min":2.7210000000000001},"last":0.77200000000000002}]},{"osd":9,"last update":"Fri Jul 21 18:40:44 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.745,"15min":0.73899999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.753,"15min":0.746},"min":{"1min":0.55100000000000005,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.93300000000000005}]},{"osd":10,"last update":"Fri Jul 21 18:41:04 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.75700000000000001,"15min":0.749},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.875,"5min":0.78500000000000003,"15min":0.77000000000000002},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.94999999999999996}]},{"osd":11,"last update":"Fri Jul 21 18:41:05 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.91300000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738443,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27844,"kb_used_data":892,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717308,"statfs":{"total":95995035648,"available":95966523392,"internally_reserved":0,"allocated":913408,"data_stored":427082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 21 18:41:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.59499999999999997,"15min":0.57499999999999996},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.875,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.504},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58999999999999997,"15min":0.54100000000000004},"min":{"1min":0.38100000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.82699999999999996,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.54900000000000004}]},{"osd":2,"last update":"Fri Jul 21 18:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.58999999999999997,"15min":0.56999999999999995},"min":{"1min":0.439,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.90300000000000002,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.61099999999999999,"15min":0.58699999999999997},"min":{"1min":0.45400000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.79300000000000004,"5min":3.0470000000000002,"15min":3.0470000000000002},"last":0.53400000000000003}]},{"osd":3,"last update":"Fri Jul 21 18:41:07 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.32100000000000001}]},{"osd":4,"last update":"Fri Jul 21 18:41:03 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.60199999999999998,"15min":0.56999999999999995},"min":{"1min":0.35799999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.77800000000000002,"5min":2.948,"15min":2.948},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.56299999999999994,"15min":0.54600000000000004},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.51400000000000001}]},{"osd":5,"last update":"Fri Jul 21 18:41:24 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.61499999999999999,"15min":0.59099999999999997},"min":{"1min":0.41599999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.94099999999999995,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.64700000000000002,"15min":0.61299999999999999},"min":{"1min":0.434,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.1419999999999999,"5min":3.0150000000000001,"15min":3.0150000000000001},"last":0.434}]},{"osd":6,"last update":"Fri Jul 21 18:40:46 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.52200000000000002,"15min":0.51100000000000001},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":2.077,"5min":2.077,"15min":2.077},"last":0.377},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.58799999999999997,"15min":0.56899999999999995},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":2.4500000000000002,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":0.56100000000000005}]},{"osd":7,"last update":"Fri Jul 21 18:39:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.47699999999999998}]},{"osd":8,"last update":"Fri Jul 21 18:41:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.76900000000000002,"15min":0.80600000000000005},"min":{"1min":0.54600000000000004,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.96299999999999997,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.78800000000000003,"15min":0.80800000000000005},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.1679999999999999,"5min":2.9830000000000001,"15min":2.9830000000000001},"last":0.61799999999999999}]},{"osd":9,"last update":"Fri Jul 21 18:40:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.71999999999999997,"15min":0.74199999999999999},"min":{"1min":0.37,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.78800000000000003,"15min":0.81299999999999994},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.421,"5min":1.421,"15min":1.421},"last":0.63800000000000001}]},{"osd":10,"last update":"Fri Jul 21 18:41:07 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.84899999999999998,"15min":0.88100000000000001},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.95599999999999996,"5min":2.9079999999999999,"15min":2.9079999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.78400000000000003,"15min":0.80400000000000005},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.96899999999999997,"5min":1.111,"15min":1.111},"last":0.59199999999999997}]},{"osd":11,"last update":"Fri Jul 21 18:40:26 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":3.0840000000000001,"5min":3.0840000000000001,"15min":3.0840000000000001},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.63}]}]},{"osd":2,"up_from":18,"seq":77309411395,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":970752,"data_stored":483703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:41:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.54700000000000004,"15min":0.53500000000000003},"min":{"1min":0.38100000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.92200000000000004,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.51600000000000001,"15min":0.50600000000000001},"min":{"1min":0.38,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.68600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.54600000000000004}]},{"osd":1,"last update":"Fri Jul 21 18:41:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.56200000000000006,"15min":0.52800000000000002},"min":{"1min":0.42999999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.59699999999999998,"15min":0.55000000000000004},"min":{"1min":0.435,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.91800000000000004,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.83299999999999996}]},{"osd":3,"last update":"Fri Jul 21 18:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.58899999999999997,"15min":0.56499999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.92400000000000004,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.56799999999999995,"15min":0.56000000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.79600000000000004,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.63900000000000001}]},{"osd":4,"last update":"Fri Jul 21 18:41:04 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.73899999999999999}]},{"osd":5,"last update":"Fri Jul 21 18:41:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.51300000000000001,"15min":0.47099999999999997},"min":{"1min":0.36899999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.89100000000000001,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.501,"15min":0.46700000000000003},"min":{"1min":0.39800000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.85999999999999999}]},{"osd":6,"last update":"Fri Jul 21 18:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.61899999999999999,"15min":0.59599999999999997},"min":{"1min":0.53000000000000003,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.85799999999999998,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.628,"15min":0.62},"min":{"1min":0.436,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.84399999999999997,"5min":0.873,"15min":0.873},"last":0.79700000000000004}]},{"osd":7,"last update":"Fri Jul 21 18:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.61699999999999999,"15min":0.60799999999999998},"min":{"1min":0.45100000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.577,"5min":1.577,"15min":1.577},"last":0.495},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.65500000000000003,"15min":0.65100000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.878,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.56499999999999995}]},{"osd":8,"last update":"Fri Jul 21 18:40:26 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.58899999999999997,"15min":0.57099999999999995},"min":{"1min":0.40799999999999997,"5min":0.376,"15min":0.376},"max":{"1min":0.91300000000000003,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.60799999999999998,"15min":0.59799999999999998},"min":{"1min":0.40799999999999997,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.95299999999999996}]},{"osd":9,"last update":"Fri Jul 21 18:40:46 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.76000000000000001,"15min":0.75600000000000001},"min":{"1min":0.59499999999999997,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.95999999999999996,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.998},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.69199999999999995,"15min":0.68999999999999995},"min":{"1min":0.47499999999999998,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.89600000000000002,"5min":1.01,"15min":1.01},"last":0.89900000000000002}]},{"osd":10,"last update":"Fri Jul 21 18:41:10 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.78200000000000003,"15min":0.77600000000000002},"min":{"1min":0.67500000000000004,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":0.91600000000000004,"5min":1.669,"15min":1.669},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.874,"5min":0.80300000000000005,"15min":0.79100000000000004},"min":{"1min":0.75600000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.97299999999999998,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":1.056}]},{"osd":11,"last update":"Fri Jul 21 18:41:04 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":1.034},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":1.0209999999999999}]}]},{"osd":3,"up_from":24,"seq":103079215166,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27832,"kb_used_data":880,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717320,"statfs":{"total":95995035648,"available":95966535680,"internally_reserved":0,"allocated":901120,"data_stored":425024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.35699999999999998,"15min":0.33100000000000002},"min":{"1min":0.26800000000000002,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.54000000000000004,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.373,"15min":0.35099999999999998},"min":{"1min":0.28100000000000003,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.51600000000000001,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.32000000000000001}]},{"osd":1,"last update":"Fri Jul 21 18:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.57199999999999995,"15min":0.55100000000000005},"min":{"1min":0.39900000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.75800000000000001,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59199999999999997,"15min":0.55000000000000004},"min":{"1min":0.39400000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.83099999999999996,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.70499999999999996}]},{"osd":2,"last update":"Fri Jul 21 18:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.57699999999999996,"15min":0.55300000000000005},"min":{"1min":0.36799999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.81599999999999995,"5min":0.876,"15min":0.876},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.56799999999999995,"15min":0.53900000000000003},"min":{"1min":0.373,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.57399999999999995}]},{"osd":4,"last update":"Fri Jul 21 18:41:05 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.61299999999999999,"15min":0.60399999999999998},"min":{"1min":0.443,"5min":0.374,"15min":0.374},"max":{"1min":0.76900000000000002,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.61799999999999999,"15min":0.59999999999999998},"min":{"1min":0.42599999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.86699999999999999,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.61299999999999999}]},{"osd":5,"last update":"Fri Jul 21 18:41:20 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.621,"15min":0.58999999999999997},"min":{"1min":0.42099999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.71299999999999997,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.59799999999999998,"15min":0.58199999999999996},"min":{"1min":0.434,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.70099999999999996,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.68799999999999994}]},{"osd":6,"last update":"Fri Jul 21 18:40:41 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.498,"15min":0.48599999999999999},"min":{"1min":0.33600000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.80900000000000005,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.55800000000000005,"15min":0.55400000000000005},"min":{"1min":0.36599999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.63400000000000001,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.51800000000000002}]},{"osd":7,"last update":"Fri Jul 21 18:41:05 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.65500000000000003,"15min":0.66600000000000004},"min":{"1min":0.5,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.84299999999999997,"5min":0.999,"15min":0.999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.72499999999999998,"15min":0.72199999999999998},"min":{"1min":0.53600000000000003,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.93600000000000005,"5min":1.181,"15min":1.181},"last":0.67600000000000005}]},{"osd":8,"last update":"Fri Jul 21 18:39:25 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.47299999999999998}]},{"osd":9,"last update":"Fri Jul 21 18:40:45 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.68600000000000005,"15min":0.71499999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.77900000000000003,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.629},{"interface":"front","average":{"1min":0.624,"5min":0.81999999999999995,"15min":0.85299999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.88100000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.748}]},{"osd":10,"last update":"Fri Jul 21 18:41:08 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.71799999999999997,"15min":0.72499999999999998},"min":{"1min":0.48999999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.80500000000000005,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.84199999999999997,"15min":0.85999999999999999},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":0.96699999999999997,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.76900000000000002}]},{"osd":11,"last update":"Fri Jul 21 18:41:05 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.79600000000000004}]}]},{"osd":4,"up_from":31,"seq":133143986234,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27820,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717332,"statfs":{"total":95995035648,"available":95966547968,"internally_reserved":0,"allocated":888832,"data_stored":425721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:40:02 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.51800000000000002,"15min":0.48199999999999998},"min":{"1min":0.40500000000000003,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.94099999999999995,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.52500000000000002,"15min":0.50600000000000001},"min":{"1min":0.373,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.80600000000000005,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.58299999999999996}]},{"osd":1,"last update":"Fri Jul 21 18:41:02 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.39100000000000001,"15min":0.35199999999999998},"min":{"1min":0.28299999999999997,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.51900000000000002,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.42499999999999999,"15min":0.39100000000000001},"min":{"1min":0.312,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.57599999999999996,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.46500000000000002}]},{"osd":2,"last update":"Fri Jul 21 18:41:02 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.56299999999999994,"15min":0.53400000000000003},"min":{"1min":0.45900000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.76600000000000001,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.59899999999999998,"15min":0.54400000000000004},"min":{"1min":0.45700000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.96499999999999997}]},{"osd":3,"last update":"Fri Jul 21 18:41:02 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.57899999999999996,"15min":0.54000000000000004},"min":{"1min":0.47499999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.78200000000000003,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56399999999999995,"15min":0.55700000000000005},"min":{"1min":0.39200000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.73299999999999998,"5min":1.05,"15min":1.05},"last":0.501}]},{"osd":5,"last update":"Fri Jul 21 18:41:23 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.66100000000000003,"15min":0.61299999999999999},"min":{"1min":0.56399999999999995,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.87,"5min":1.571,"15min":1.571},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.66400000000000003,"15min":0.622},"min":{"1min":0.46200000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.89600000000000002,"5min":1.377,"15min":1.377},"last":0.88200000000000001}]},{"osd":6,"last update":"Fri Jul 21 18:40:44 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.63900000000000001,"15min":0.61699999999999999},"min":{"1min":0.498,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.83699999999999997,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.627},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.65000000000000002,"15min":0.61399999999999999},"min":{"1min":0.46100000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.81299999999999994,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.65900000000000003}]},{"osd":7,"last update":"Fri Jul 21 18:41:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.59999999999999998,"15min":0.56999999999999995},"min":{"1min":0.39600000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.93700000000000006,"5min":1.073,"15min":1.073},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.55000000000000004,"15min":0.54200000000000004},"min":{"1min":0.41499999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.81499999999999995,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.55900000000000005}]},{"osd":8,"last update":"Fri Jul 21 18:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.65900000000000003,"15min":0.66100000000000003},"min":{"1min":0.46200000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.92400000000000004,"5min":1.026,"15min":1.026},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.73099999999999998,"15min":0.72499999999999998},"min":{"1min":0.56899999999999995,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.92500000000000004}]},{"osd":9,"last update":"Fri Jul 21 18:40:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.79000000000000004,"15min":0.79900000000000004},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.93999999999999995,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.83599999999999997,"15min":0.84599999999999997},"min":{"1min":0.60499999999999998,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.95499999999999996,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.78000000000000003}]},{"osd":10,"last update":"Fri Jul 21 18:41:08 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.78500000000000003,"15min":0.78300000000000003},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.98699999999999999,"5min":1.153,"15min":1.153},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.745,"15min":0.73899999999999999},"min":{"1min":0.55900000000000005,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.97499999999999998,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.79700000000000004}]},{"osd":11,"last update":"Fri Jul 21 18:41:05 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.94099999999999995}]}]},{"osd":5,"up_from":36,"seq":154618822710,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28428,"kb_used_data":1476,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716724,"statfs":{"total":95995035648,"available":95965925376,"internally_reserved":0,"allocated":1511424,"data_stored":1017942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:41:24 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.52200000000000002,"15min":0.52900000000000003},"min":{"1min":0.35799999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.73099999999999998,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.503,"5min":0.53600000000000003,"15min":0.55300000000000005},"min":{"1min":0.34799999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.42799999999999999}]},{"osd":1,"last update":"Fri Jul 21 18:41:24 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.55400000000000005,"15min":0.51400000000000001},"min":{"1min":0.39200000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.76900000000000002,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.622,"5min":0.56399999999999995,"15min":0.52100000000000002},"min":{"1min":0.48599999999999999,"5min":0.375,"15min":0.375},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.60099999999999998}]},{"osd":2,"last update":"Fri Jul 21 18:41:24 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.41299999999999998,"15min":0.39200000000000002},"min":{"1min":0.27600000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.439,"5min":0.42699999999999999,"15min":0.40100000000000002},"min":{"1min":0.32000000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.69799999999999995,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.501}]},{"osd":3,"last update":"Fri Jul 21 18:41:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.56100000000000005,"15min":0.56899999999999995},"min":{"1min":0.437,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.80100000000000005,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.436,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.622}]},{"osd":4,"last update":"Fri Jul 21 18:41:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.58999999999999997,"15min":0.56000000000000005},"min":{"1min":0.46600000000000003,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.86799999999999999,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.58299999999999996,"15min":0.54700000000000004},"min":{"1min":0.49399999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81999999999999995,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.73799999999999999}]},{"osd":6,"last update":"Fri Jul 21 18:40:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.59599999999999997,"15min":0.59999999999999998},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.60399999999999998,"15min":0.59099999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.71599999999999997}]},{"osd":7,"last update":"Fri Jul 21 18:41:05 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.65900000000000003,"15min":0.66600000000000004},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.65100000000000002,"15min":0.64300000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.85999999999999999,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.77000000000000002}]},{"osd":8,"last update":"Fri Jul 21 18:41:24 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.63100000000000001,"15min":0.63300000000000001},"min":{"1min":0.35099999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.95599999999999996,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.63700000000000001,"15min":0.63500000000000001},"min":{"1min":0.41999999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.77700000000000002}]},{"osd":9,"last update":"Fri Jul 21 18:40:45 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.72499999999999998,"15min":0.72899999999999998},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.877,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.73199999999999998,"15min":0.73399999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.81499999999999995}]},{"osd":10,"last update":"Fri Jul 21 18:41:05 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.71899999999999997,"15min":0.71199999999999997},"min":{"1min":0.54600000000000004,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.73599999999999999,"15min":0.73099999999999998},"min":{"1min":0.55700000000000005,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.83799999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":58579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":59069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":59271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-21T18:41:32.746 INFO:tasks.fwd_scrub.ceph_manager:clean! 2023-07-21T18:41:32.746 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-21T18:41:32.907 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.884+0000 7fdd2f8bb700 1 -- 172.21.15.39:0/3215227914 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd280ffc00 msgr2=0x7fdd28100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:32.907 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.884+0000 7fdd2f8bb700 1 --2- 172.21.15.39:0/3215227914 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd280ffc00 0x7fdd28100050 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fdd24009990 tx=0x7fdd24009ca0 comp rx=0 tx=0).stop 2023-07-21T18:41:32.907 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.884+0000 7fdd2f8bb700 1 -- 172.21.15.39:0/3215227914 shutdown_connections 2023-07-21T18:41:32.907 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.884+0000 7fdd2f8bb700 1 --2- 172.21.15.39:0/3215227914 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd280fea10 0x7fdd280fee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:32.908 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.884+0000 7fdd2f8bb700 1 --2- 172.21.15.39:0/3215227914 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd28100590 0x7fdd28106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:32.908 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.884+0000 7fdd2f8bb700 1 --2- 172.21.15.39:0/3215227914 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd280ffc00 0x7fdd28100050 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:32.908 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.884+0000 7fdd2f8bb700 1 -- 172.21.15.39:0/3215227914 >> 172.21.15.39:0/3215227914 conn(0x7fdd280fa030 msgr2=0x7fdd280fc460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:32.908 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.884+0000 7fdd2f8bb700 1 -- 172.21.15.39:0/3215227914 shutdown_connections 2023-07-21T18:41:32.908 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.884+0000 7fdd2f8bb700 1 -- 172.21.15.39:0/3215227914 wait complete. 2023-07-21T18:41:32.910 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.885+0000 7fdd2f8bb700 1 Processor -- start 2023-07-21T18:41:32.910 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.885+0000 7fdd2f8bb700 1 -- start start 2023-07-21T18:41:32.910 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.885+0000 7fdd2f8bb700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd280fea10 0x7fdd28194780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:32.910 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.885+0000 7fdd2f8bb700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd280ffc00 0x7fdd28194cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:32.911 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.885+0000 7fdd2f8bb700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd28100590 0x7fdd2819bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:32.911 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.885+0000 7fdd2f8bb700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2819c300 con 0x7fdd280ffc00 2023-07-21T18:41:32.911 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.885+0000 7fdd2f8bb700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2819c440 con 0x7fdd28100590 2023-07-21T18:41:32.911 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.885+0000 7fdd2f8bb700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2819c5d0 con 0x7fdd280fea10 2023-07-21T18:41:32.911 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.885+0000 7fdd2ce56700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd280ffc00 0x7fdd28194cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:32.911 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.885+0000 7fdd2ce56700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd280ffc00 0x7fdd28194cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56736/0 (socket says 172.21.15.39:56736) 2023-07-21T18:41:32.911 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.885+0000 7fdd2ce56700 1 -- 172.21.15.39:0/3386100776 learned_addr learned my addr 172.21.15.39:0/3386100776 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:32.911 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.885+0000 7fdd2d657700 1 --2- 172.21.15.39:0/3386100776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd280fea10 0x7fdd28194780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:32.912 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.885+0000 7fdd2ce56700 1 -- 172.21.15.39:0/3386100776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd280fea10 msgr2=0x7fdd28194780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:32.912 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.885+0000 7fdd2ce56700 1 --2- 172.21.15.39:0/3386100776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd280fea10 0x7fdd28194780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:32.912 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.885+0000 7fdd2de58700 1 --2- 172.21.15.39:0/3386100776 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd28100590 0x7fdd2819bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:32.912 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.885+0000 7fdd2ce56700 1 -- 172.21.15.39:0/3386100776 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd28100590 msgr2=0x7fdd2819bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:32.912 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.885+0000 7fdd2ce56700 1 --2- 172.21.15.39:0/3386100776 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd28100590 0x7fdd2819bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:32.912 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.885+0000 7fdd2ce56700 1 -- 172.21.15.39:0/3386100776 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd240095f0 con 0x7fdd280ffc00 2023-07-21T18:41:32.912 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.885+0000 7fdd2ce56700 1 --2- 172.21.15.39:0/3386100776 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd280ffc00 0x7fdd28194cc0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fdd24004760 tx=0x7fdd24009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:32.913 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.886+0000 7fdd1a7fc700 1 -- 172.21.15.39:0/3386100776 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd2401d070 con 0x7fdd280ffc00 2023-07-21T18:41:32.913 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.886+0000 7fdd1a7fc700 1 -- 172.21.15.39:0/3386100776 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd24004200 con 0x7fdd280ffc00 2023-07-21T18:41:32.913 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.886+0000 7fdd2f8bb700 1 -- 172.21.15.39:0/3386100776 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd2819c850 con 0x7fdd280ffc00 2023-07-21T18:41:32.913 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.886+0000 7fdd1a7fc700 1 -- 172.21.15.39:0/3386100776 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd240049f0 con 0x7fdd280ffc00 2023-07-21T18:41:32.913 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.886+0000 7fdd2f8bb700 1 -- 172.21.15.39:0/3386100776 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd2819cd50 con 0x7fdd280ffc00 2023-07-21T18:41:32.913 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.887+0000 7fdd2f8bb700 1 -- 172.21.15.39:0/3386100776 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd28105f60 con 0x7fdd280ffc00 2023-07-21T18:41:32.914 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.888+0000 7fdd1a7fc700 1 -- 172.21.15.39:0/3386100776 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fdd24003d40 con 0x7fdd280ffc00 2023-07-21T18:41:32.914 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.888+0000 7fdd1a7fc700 1 --2- 172.21.15.39:0/3386100776 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fdd140a12a0 0x7fdd140a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:32.914 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.889+0000 7fdd2d657700 1 --2- 172.21.15.39:0/3386100776 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fdd140a12a0 0x7fdd140a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:32.914 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.889+0000 7fdd1a7fc700 1 -- 172.21.15.39:0/3386100776 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fdd240cd920 con 0x7fdd280ffc00 2023-07-21T18:41:32.915 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.889+0000 7fdd2d657700 1 --2- 172.21.15.39:0/3386100776 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fdd140a12a0 0x7fdd140a3750 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c005950 tx=0x7fdd1c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:32.915 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:32.892+0000 7fdd1a7fc700 1 -- 172.21.15.39:0/3386100776 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fdd24096fd0 con 0x7fdd280ffc00 2023-07-21T18:41:33.112 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.089+0000 7fdd2f8bb700 1 -- 172.21.15.39:0/3386100776 --> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdd28060950 con 0x7fdd140a12a0 2023-07-21T18:41:33.124 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.101+0000 7fdd1a7fc700 1 -- 172.21.15.39:0/3386100776 <== mgr.14152 v2:172.21.15.39:6800/2116291712 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+344482 (secure 0 0 0) 0x7fdd28060950 con 0x7fdd140a12a0 2023-07-21T18:41:33.128 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:41:33.130 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.105+0000 7fdd2f8bb700 1 -- 172.21.15.39:0/3386100776 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fdd140a12a0 msgr2=0x7fdd140a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:33.130 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.105+0000 7fdd2f8bb700 1 --2- 172.21.15.39:0/3386100776 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fdd140a12a0 0x7fdd140a3750 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c005950 tx=0x7fdd1c0058e0 comp rx=0 tx=0).stop 2023-07-21T18:41:33.130 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.105+0000 7fdd2f8bb700 1 -- 172.21.15.39:0/3386100776 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd280ffc00 msgr2=0x7fdd28194cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:33.130 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.105+0000 7fdd2f8bb700 1 --2- 172.21.15.39:0/3386100776 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd280ffc00 0x7fdd28194cc0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fdd24004760 tx=0x7fdd24009f90 comp rx=0 tx=0).stop 2023-07-21T18:41:33.130 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.105+0000 7fdd2f8bb700 1 -- 172.21.15.39:0/3386100776 shutdown_connections 2023-07-21T18:41:33.130 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.105+0000 7fdd2f8bb700 1 --2- 172.21.15.39:0/3386100776 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fdd140a12a0 0x7fdd140a3750 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:33.130 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.105+0000 7fdd2f8bb700 1 --2- 172.21.15.39:0/3386100776 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd28100590 0x7fdd2819bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:33.131 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.105+0000 7fdd2f8bb700 1 --2- 172.21.15.39:0/3386100776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd280fea10 0x7fdd28194780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:33.131 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.105+0000 7fdd2f8bb700 1 --2- 172.21.15.39:0/3386100776 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd280ffc00 0x7fdd28194cc0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:33.131 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.105+0000 7fdd2f8bb700 1 -- 172.21.15.39:0/3386100776 >> 172.21.15.39:0/3386100776 conn(0x7fdd280fa030 msgr2=0x7fdd281030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:33.131 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.105+0000 7fdd2f8bb700 1 -- 172.21.15.39:0/3386100776 shutdown_connections 2023-07-21T18:41:33.131 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.105+0000 7fdd2f8bb700 1 -- 172.21.15.39:0/3386100776 wait complete. 2023-07-21T18:41:33.131 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2023-07-21T18:41:33.140 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2023-07-21T18:41:31.141306+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654745,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":953,"num_read_kb":596,"num_write":416,"num_write_kb":3068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180736,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":343,"ondisk_log_size":343,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335916,"kb_used_data":12492,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124605908,"statfs":{"total":1151940427776,"available":1151596449792,"internally_reserved":0,"allocated":12791808,"data_stored":7060074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":636,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":111,"num_read_kb":51,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003031"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.144002+0000","last_change":"2023-07-21T18:40:04.792977+0000","last_active":"2023-07-21T18:41:08.144002+0000","last_peered":"2023-07-21T18:41:08.144002+0000","last_clean":"2023-07-21T18:41:08.144002+0000","last_became_active":"2023-07-21T18:40:04.792667+0000","last_became_peered":"2023-07-21T18:40:04.792667+0000","last_unstale":"2023-07-21T18:41:08.144002+0000","last_undegraded":"2023-07-21T18:41:08.144002+0000","last_fullsized":"2023-07-21T18:41:08.144002+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:45:33.822350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.154989+0000","last_change":"2023-07-21T18:40:04.792936+0000","last_active":"2023-07-21T18:41:08.154989+0000","last_peered":"2023-07-21T18:41:08.154989+0000","last_clean":"2023-07-21T18:41:08.154989+0000","last_became_active":"2023-07-21T18:40:04.792511+0000","last_became_peered":"2023-07-21T18:40:04.792511+0000","last_unstale":"2023-07-21T18:41:08.154989+0000","last_undegraded":"2023-07-21T18:41:08.154989+0000","last_fullsized":"2023-07-21T18:41:08.154989+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:18:24.383764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.154716+0000","last_change":"2023-07-21T18:40:04.796401+0000","last_active":"2023-07-21T18:41:08.154716+0000","last_peered":"2023-07-21T18:41:08.154716+0000","last_clean":"2023-07-21T18:41:08.154716+0000","last_became_active":"2023-07-21T18:40:04.796063+0000","last_became_peered":"2023-07-21T18:40:04.796063+0000","last_unstale":"2023-07-21T18:41:08.154716+0000","last_undegraded":"2023-07-21T18:41:08.154716+0000","last_fullsized":"2023-07-21T18:41:08.154716+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:53:37.099167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.154119+0000","last_change":"2023-07-21T18:41:08.154119+0000","last_active":"2023-07-21T18:41:08.154119+0000","last_peered":"2023-07-21T18:41:08.154119+0000","last_clean":"2023-07-21T18:41:08.154119+0000","last_became_active":"2023-07-21T18:41:08.153698+0000","last_became_peered":"2023-07-21T18:41:08.153698+0000","last_unstale":"2023-07-21T18:41:08.154119+0000","last_undegraded":"2023-07-21T18:41:08.154119+0000","last_fullsized":"2023-07-21T18:41:08.154119+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T18:41:49.057189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.148468+0000","last_change":"2023-07-21T18:40:04.794035+0000","last_active":"2023-07-21T18:41:08.148468+0000","last_peered":"2023-07-21T18:41:08.148468+0000","last_clean":"2023-07-21T18:41:08.148468+0000","last_became_active":"2023-07-21T18:40:04.793739+0000","last_became_peered":"2023-07-21T18:40:04.793739+0000","last_unstale":"2023-07-21T18:41:08.148468+0000","last_undegraded":"2023-07-21T18:41:08.148468+0000","last_fullsized":"2023-07-21T18:41:08.148468+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:25:53.830527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.150742+0000","last_change":"2023-07-21T18:41:08.150742+0000","last_active":"2023-07-21T18:41:08.150742+0000","last_peered":"2023-07-21T18:41:08.150742+0000","last_clean":"2023-07-21T18:41:08.150742+0000","last_became_active":"2023-07-21T18:41:08.150346+0000","last_became_peered":"2023-07-21T18:41:08.150346+0000","last_unstale":"2023-07-21T18:41:08.150742+0000","last_undegraded":"2023-07-21T18:41:08.150742+0000","last_fullsized":"2023-07-21T18:41:08.150742+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:06:31.088681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.595238+0000","last_change":"2023-07-21T18:40:04.787544+0000","last_active":"2023-07-21T18:41:08.595238+0000","last_peered":"2023-07-21T18:41:08.595238+0000","last_clean":"2023-07-21T18:41:08.595238+0000","last_became_active":"2023-07-21T18:40:04.787208+0000","last_became_peered":"2023-07-21T18:40:04.787208+0000","last_unstale":"2023-07-21T18:41:08.595238+0000","last_undegraded":"2023-07-21T18:41:08.595238+0000","last_fullsized":"2023-07-21T18:41:08.595238+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:57:13.333034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.249534+0000","last_change":"2023-07-21T18:41:08.249534+0000","last_active":"2023-07-21T18:41:08.249534+0000","last_peered":"2023-07-21T18:41:08.249534+0000","last_clean":"2023-07-21T18:41:08.249534+0000","last_became_active":"2023-07-21T18:41:08.249228+0000","last_became_peered":"2023-07-21T18:41:08.249228+0000","last_unstale":"2023-07-21T18:41:08.249534+0000","last_undegraded":"2023-07-21T18:41:08.249534+0000","last_fullsized":"2023-07-21T18:41:08.249534+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:19:58.325032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.151247+0000","last_change":"2023-07-21T18:40:04.782495+0000","last_active":"2023-07-21T18:41:08.151247+0000","last_peered":"2023-07-21T18:41:08.151247+0000","last_clean":"2023-07-21T18:41:08.151247+0000","last_became_active":"2023-07-21T18:40:04.782208+0000","last_became_peered":"2023-07-21T18:40:04.782208+0000","last_unstale":"2023-07-21T18:41:08.151247+0000","last_undegraded":"2023-07-21T18:41:08.151247+0000","last_fullsized":"2023-07-21T18:41:08.151247+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:22:50.428618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.148205+0000","last_change":"2023-07-21T18:40:04.790957+0000","last_active":"2023-07-21T18:41:08.148205+0000","last_peered":"2023-07-21T18:41:08.148205+0000","last_clean":"2023-07-21T18:41:08.148205+0000","last_became_active":"2023-07-21T18:40:04.790659+0000","last_became_peered":"2023-07-21T18:40:04.790659+0000","last_unstale":"2023-07-21T18:41:08.148205+0000","last_undegraded":"2023-07-21T18:41:08.148205+0000","last_fullsized":"2023-07-21T18:41:08.148205+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:29:03.090784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.595484+0000","last_change":"2023-07-21T18:40:04.789415+0000","last_active":"2023-07-21T18:41:08.595484+0000","last_peered":"2023-07-21T18:41:08.595484+0000","last_clean":"2023-07-21T18:41:08.595484+0000","last_became_active":"2023-07-21T18:40:04.789088+0000","last_became_peered":"2023-07-21T18:40:04.789088+0000","last_unstale":"2023-07-21T18:41:08.595484+0000","last_undegraded":"2023-07-21T18:41:08.595484+0000","last_fullsized":"2023-07-21T18:41:08.595484+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:40:03.977217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.245377+0000","last_change":"2023-07-21T18:40:04.789101+0000","last_active":"2023-07-21T18:41:08.245377+0000","last_peered":"2023-07-21T18:41:08.245377+0000","last_clean":"2023-07-21T18:41:08.245377+0000","last_became_active":"2023-07-21T18:40:04.788704+0000","last_became_peered":"2023-07-21T18:40:04.788704+0000","last_unstale":"2023-07-21T18:41:08.245377+0000","last_undegraded":"2023-07-21T18:41:08.245377+0000","last_fullsized":"2023-07-21T18:41:08.245377+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:33:01.699609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.144358+0000","last_change":"2023-07-21T18:40:04.784863+0000","last_active":"2023-07-21T18:41:08.144358+0000","last_peered":"2023-07-21T18:41:08.144358+0000","last_clean":"2023-07-21T18:41:08.144358+0000","last_became_active":"2023-07-21T18:40:04.784474+0000","last_became_peered":"2023-07-21T18:40:04.784474+0000","last_unstale":"2023-07-21T18:41:08.144358+0000","last_undegraded":"2023-07-21T18:41:08.144358+0000","last_fullsized":"2023-07-21T18:41:08.144358+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:20:04.409410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.154148+0000","last_change":"2023-07-21T18:40:04.797678+0000","last_active":"2023-07-21T18:41:08.154148+0000","last_peered":"2023-07-21T18:41:08.154148+0000","last_clean":"2023-07-21T18:41:08.154148+0000","last_became_active":"2023-07-21T18:40:04.797318+0000","last_became_peered":"2023-07-21T18:40:04.797318+0000","last_unstale":"2023-07-21T18:41:08.154148+0000","last_undegraded":"2023-07-21T18:41:08.154148+0000","last_fullsized":"2023-07-21T18:41:08.154148+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:28:01.333981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.153924+0000","last_change":"2023-07-21T18:40:04.793170+0000","last_active":"2023-07-21T18:41:08.153924+0000","last_peered":"2023-07-21T18:41:08.153924+0000","last_clean":"2023-07-21T18:41:08.153924+0000","last_became_active":"2023-07-21T18:40:04.792563+0000","last_became_peered":"2023-07-21T18:40:04.792563+0000","last_unstale":"2023-07-21T18:41:08.153924+0000","last_undegraded":"2023-07-21T18:41:08.153924+0000","last_fullsized":"2023-07-21T18:41:08.153924+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:07:51.115805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.150518+0000","last_change":"2023-07-21T18:40:04.791972+0000","last_active":"2023-07-21T18:41:08.150518+0000","last_peered":"2023-07-21T18:41:08.150518+0000","last_clean":"2023-07-21T18:41:08.150518+0000","last_became_active":"2023-07-21T18:40:04.791639+0000","last_became_peered":"2023-07-21T18:40:04.791639+0000","last_unstale":"2023-07-21T18:41:08.150518+0000","last_undegraded":"2023-07-21T18:41:08.150518+0000","last_fullsized":"2023-07-21T18:41:08.150518+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:31:09.972680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.144015+0000","last_change":"2023-07-21T18:40:04.789362+0000","last_active":"2023-07-21T18:41:08.144015+0000","last_peered":"2023-07-21T18:41:08.144015+0000","last_clean":"2023-07-21T18:41:08.144015+0000","last_became_active":"2023-07-21T18:40:04.788978+0000","last_became_peered":"2023-07-21T18:40:04.788978+0000","last_unstale":"2023-07-21T18:41:08.144015+0000","last_undegraded":"2023-07-21T18:41:08.144015+0000","last_fullsized":"2023-07-21T18:41:08.144015+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:06:43.785294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.146563+0000","last_change":"2023-07-21T18:40:04.788016+0000","last_active":"2023-07-21T18:41:08.146563+0000","last_peered":"2023-07-21T18:41:08.146563+0000","last_clean":"2023-07-21T18:41:08.146563+0000","last_became_active":"2023-07-21T18:40:04.787690+0000","last_became_peered":"2023-07-21T18:40:04.787690+0000","last_unstale":"2023-07-21T18:41:08.146563+0000","last_undegraded":"2023-07-21T18:41:08.146563+0000","last_fullsized":"2023-07-21T18:41:08.146563+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:40:42.379190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.152768+0000","last_change":"2023-07-21T18:40:04.793075+0000","last_active":"2023-07-21T18:41:08.152768+0000","last_peered":"2023-07-21T18:41:08.152768+0000","last_clean":"2023-07-21T18:41:08.152768+0000","last_became_active":"2023-07-21T18:40:04.792773+0000","last_became_peered":"2023-07-21T18:40:04.792773+0000","last_unstale":"2023-07-21T18:41:08.152768+0000","last_undegraded":"2023-07-21T18:41:08.152768+0000","last_fullsized":"2023-07-21T18:41:08.152768+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:46:27.429800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"79'13","reported_seq":108,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:26.100464+0000","last_change":"2023-07-21T18:40:04.178197+0000","last_active":"2023-07-21T18:41:26.100464+0000","last_peered":"2023-07-21T18:41:26.100464+0000","last_clean":"2023-07-21T18:41:26.100464+0000","last_became_active":"2023-07-21T18:40:04.177914+0000","last_became_peered":"2023-07-21T18:40:04.177914+0000","last_unstale":"2023-07-21T18:41:26.100464+0000","last_undegraded":"2023-07-21T18:41:26.100464+0000","last_fullsized":"2023-07-21T18:41:26.100464+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:55:39.788580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":77,"num_read_kb":77,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.149760+0000","last_change":"2023-07-21T18:40:04.791610+0000","last_active":"2023-07-21T18:41:08.149760+0000","last_peered":"2023-07-21T18:41:08.149760+0000","last_clean":"2023-07-21T18:41:08.149760+0000","last_became_active":"2023-07-21T18:40:04.791242+0000","last_became_peered":"2023-07-21T18:40:04.791242+0000","last_unstale":"2023-07-21T18:41:08.149760+0000","last_undegraded":"2023-07-21T18:41:08.149760+0000","last_fullsized":"2023-07-21T18:41:08.149760+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:27:52.116304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.149806+0000","last_change":"2023-07-21T18:40:04.182120+0000","last_active":"2023-07-21T18:41:08.149806+0000","last_peered":"2023-07-21T18:41:08.149806+0000","last_clean":"2023-07-21T18:41:08.149806+0000","last_became_active":"2023-07-21T18:40:04.181797+0000","last_became_peered":"2023-07-21T18:40:04.181797+0000","last_unstale":"2023-07-21T18:41:08.149806+0000","last_undegraded":"2023-07-21T18:41:08.149806+0000","last_fullsized":"2023-07-21T18:41:08.149806+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:47:45.817595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.155369+0000","last_change":"2023-07-21T18:40:04.796649+0000","last_active":"2023-07-21T18:41:08.155369+0000","last_peered":"2023-07-21T18:41:08.155369+0000","last_clean":"2023-07-21T18:41:08.155369+0000","last_became_active":"2023-07-21T18:40:04.795610+0000","last_became_peered":"2023-07-21T18:40:04.795610+0000","last_unstale":"2023-07-21T18:41:08.155369+0000","last_undegraded":"2023-07-21T18:41:08.155369+0000","last_fullsized":"2023-07-21T18:41:08.155369+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:22:37.727432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"81'13","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.149058+0000","last_change":"2023-07-21T18:40:04.178384+0000","last_active":"2023-07-21T18:41:08.149058+0000","last_peered":"2023-07-21T18:41:08.149058+0000","last_clean":"2023-07-21T18:41:08.149058+0000","last_became_active":"2023-07-21T18:40:04.178074+0000","last_became_peered":"2023-07-21T18:40:04.178074+0000","last_unstale":"2023-07-21T18:41:08.149058+0000","last_undegraded":"2023-07-21T18:41:08.149058+0000","last_fullsized":"2023-07-21T18:41:08.149058+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:20:16.081243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.150712+0000","last_change":"2023-07-21T18:40:04.785431+0000","last_active":"2023-07-21T18:41:08.150712+0000","last_peered":"2023-07-21T18:41:08.150712+0000","last_clean":"2023-07-21T18:41:08.150712+0000","last_became_active":"2023-07-21T18:40:04.785124+0000","last_became_peered":"2023-07-21T18:40:04.785124+0000","last_unstale":"2023-07-21T18:41:08.150712+0000","last_undegraded":"2023-07-21T18:41:08.150712+0000","last_fullsized":"2023-07-21T18:41:08.150712+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:07:13.594091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"79'5","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.148841+0000","last_change":"2023-07-21T18:40:04.176349+0000","last_active":"2023-07-21T18:41:08.148841+0000","last_peered":"2023-07-21T18:41:08.148841+0000","last_clean":"2023-07-21T18:41:08.148841+0000","last_became_active":"2023-07-21T18:40:04.175997+0000","last_became_peered":"2023-07-21T18:40:04.175997+0000","last_unstale":"2023-07-21T18:41:08.148841+0000","last_undegraded":"2023-07-21T18:41:08.148841+0000","last_fullsized":"2023-07-21T18:41:08.148841+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:24:53.545517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.246653+0000","last_change":"2023-07-21T18:41:08.246653+0000","last_active":"2023-07-21T18:41:08.246653+0000","last_peered":"2023-07-21T18:41:08.246653+0000","last_clean":"2023-07-21T18:41:08.246653+0000","last_became_active":"2023-07-21T18:41:08.246035+0000","last_became_peered":"2023-07-21T18:41:08.246035+0000","last_unstale":"2023-07-21T18:41:08.246653+0000","last_undegraded":"2023-07-21T18:41:08.246653+0000","last_fullsized":"2023-07-21T18:41:08.246653+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:30:36.054664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1b","version":"79'5","reported_seq":105,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:29.084542+0000","last_change":"2023-07-21T18:40:04.177638+0000","last_active":"2023-07-21T18:41:29.084542+0000","last_peered":"2023-07-21T18:41:29.084542+0000","last_clean":"2023-07-21T18:41:29.084542+0000","last_became_active":"2023-07-21T18:40:04.177284+0000","last_became_peered":"2023-07-21T18:40:04.177284+0000","last_unstale":"2023-07-21T18:41:29.084542+0000","last_undegraded":"2023-07-21T18:41:29.084542+0000","last_fullsized":"2023-07-21T18:41:29.084542+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:34:36.750937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.148514+0000","last_change":"2023-07-21T18:40:04.791536+0000","last_active":"2023-07-21T18:41:08.148514+0000","last_peered":"2023-07-21T18:41:08.148514+0000","last_clean":"2023-07-21T18:41:08.148514+0000","last_became_active":"2023-07-21T18:40:04.791246+0000","last_became_peered":"2023-07-21T18:40:04.791246+0000","last_unstale":"2023-07-21T18:41:08.148514+0000","last_undegraded":"2023-07-21T18:41:08.148514+0000","last_fullsized":"2023-07-21T18:41:08.148514+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:30:46.562521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"79'7","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.147995+0000","last_change":"2023-07-21T18:40:04.179054+0000","last_active":"2023-07-21T18:41:08.147995+0000","last_peered":"2023-07-21T18:41:08.147995+0000","last_clean":"2023-07-21T18:41:08.147995+0000","last_became_active":"2023-07-21T18:40:04.178713+0000","last_became_peered":"2023-07-21T18:40:04.178713+0000","last_unstale":"2023-07-21T18:41:08.147995+0000","last_undegraded":"2023-07-21T18:41:08.147995+0000","last_fullsized":"2023-07-21T18:41:08.147995+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:54:00.028673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.249387+0000","last_change":"2023-07-21T18:40:04.793947+0000","last_active":"2023-07-21T18:41:08.249387+0000","last_peered":"2023-07-21T18:41:08.249387+0000","last_clean":"2023-07-21T18:41:08.249387+0000","last_became_active":"2023-07-21T18:40:04.793631+0000","last_became_peered":"2023-07-21T18:40:04.793631+0000","last_unstale":"2023-07-21T18:41:08.249387+0000","last_undegraded":"2023-07-21T18:41:08.249387+0000","last_fullsized":"2023-07-21T18:41:08.249387+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T18:56:26.416040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"79'3","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.143902+0000","last_change":"2023-07-21T18:40:04.182549+0000","last_active":"2023-07-21T18:41:08.143902+0000","last_peered":"2023-07-21T18:41:08.143902+0000","last_clean":"2023-07-21T18:41:08.143902+0000","last_became_active":"2023-07-21T18:40:04.182233+0000","last_became_peered":"2023-07-21T18:40:04.182233+0000","last_unstale":"2023-07-21T18:41:08.143902+0000","last_undegraded":"2023-07-21T18:41:08.143902+0000","last_fullsized":"2023-07-21T18:41:08.143902+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:38:34.232736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.149083+0000","last_change":"2023-07-21T18:40:04.789987+0000","last_active":"2023-07-21T18:41:08.149083+0000","last_peered":"2023-07-21T18:41:08.149083+0000","last_clean":"2023-07-21T18:41:08.149083+0000","last_became_active":"2023-07-21T18:40:04.789647+0000","last_became_peered":"2023-07-21T18:40:04.789647+0000","last_unstale":"2023-07-21T18:41:08.149083+0000","last_undegraded":"2023-07-21T18:41:08.149083+0000","last_fullsized":"2023-07-21T18:41:08.149083+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:33:16.733179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'3","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.248336+0000","last_change":"2023-07-21T18:40:04.177342+0000","last_active":"2023-07-21T18:41:08.248336+0000","last_peered":"2023-07-21T18:41:08.248336+0000","last_clean":"2023-07-21T18:41:08.248336+0000","last_became_active":"2023-07-21T18:40:04.176983+0000","last_became_peered":"2023-07-21T18:40:04.176983+0000","last_unstale":"2023-07-21T18:41:08.248336+0000","last_undegraded":"2023-07-21T18:41:08.248336+0000","last_fullsized":"2023-07-21T18:41:08.248336+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:25:28.759619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.146313+0000","last_change":"2023-07-21T18:40:04.794907+0000","last_active":"2023-07-21T18:41:08.146313+0000","last_peered":"2023-07-21T18:41:08.146313+0000","last_clean":"2023-07-21T18:41:08.146313+0000","last_became_active":"2023-07-21T18:40:04.794545+0000","last_became_peered":"2023-07-21T18:40:04.794545+0000","last_unstale":"2023-07-21T18:41:08.146313+0000","last_undegraded":"2023-07-21T18:41:08.146313+0000","last_fullsized":"2023-07-21T18:41:08.146313+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:42:33.019351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"79'4","reported_seq":178,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:28.082056+0000","last_change":"2023-07-21T18:40:04.179198+0000","last_active":"2023-07-21T18:41:28.082056+0000","last_peered":"2023-07-21T18:41:28.082056+0000","last_clean":"2023-07-21T18:41:28.082056+0000","last_became_active":"2023-07-21T18:40:04.178841+0000","last_became_peered":"2023-07-21T18:40:04.178841+0000","last_unstale":"2023-07-21T18:41:28.082056+0000","last_undegraded":"2023-07-21T18:41:28.082056+0000","last_fullsized":"2023-07-21T18:41:28.082056+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:12:15.233249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":156,"num_read_kb":156,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.151068+0000","last_change":"2023-07-21T18:40:04.786229+0000","last_active":"2023-07-21T18:41:08.151068+0000","last_peered":"2023-07-21T18:41:08.151068+0000","last_clean":"2023-07-21T18:41:08.151068+0000","last_became_active":"2023-07-21T18:40:04.785919+0000","last_became_peered":"2023-07-21T18:40:04.785919+0000","last_unstale":"2023-07-21T18:41:08.151068+0000","last_undegraded":"2023-07-21T18:41:08.151068+0000","last_fullsized":"2023-07-21T18:41:08.151068+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:37:35.592913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'4","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.595131+0000","last_change":"2023-07-21T18:40:04.177164+0000","last_active":"2023-07-21T18:41:08.595131+0000","last_peered":"2023-07-21T18:41:08.595131+0000","last_clean":"2023-07-21T18:41:08.595131+0000","last_became_active":"2023-07-21T18:40:04.176617+0000","last_became_peered":"2023-07-21T18:40:04.176617+0000","last_unstale":"2023-07-21T18:41:08.595131+0000","last_undegraded":"2023-07-21T18:41:08.595131+0000","last_fullsized":"2023-07-21T18:41:08.595131+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:43:44.771285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.155486+0000","last_change":"2023-07-21T18:40:04.787863+0000","last_active":"2023-07-21T18:41:08.155486+0000","last_peered":"2023-07-21T18:41:08.155486+0000","last_clean":"2023-07-21T18:41:08.155486+0000","last_became_active":"2023-07-21T18:40:04.787562+0000","last_became_peered":"2023-07-21T18:40:04.787562+0000","last_unstale":"2023-07-21T18:41:08.155486+0000","last_undegraded":"2023-07-21T18:41:08.155486+0000","last_fullsized":"2023-07-21T18:41:08.155486+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:15:49.834031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'5","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.248499+0000","last_change":"2023-07-21T18:40:04.179921+0000","last_active":"2023-07-21T18:41:08.248499+0000","last_peered":"2023-07-21T18:41:08.248499+0000","last_clean":"2023-07-21T18:41:08.248499+0000","last_became_active":"2023-07-21T18:40:04.179670+0000","last_became_peered":"2023-07-21T18:40:04.179670+0000","last_unstale":"2023-07-21T18:41:08.248499+0000","last_undegraded":"2023-07-21T18:41:08.248499+0000","last_fullsized":"2023-07-21T18:41:08.248499+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:15:29.991358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.150860+0000","last_change":"2023-07-21T18:40:04.792930+0000","last_active":"2023-07-21T18:41:08.150860+0000","last_peered":"2023-07-21T18:41:08.150860+0000","last_clean":"2023-07-21T18:41:08.150860+0000","last_became_active":"2023-07-21T18:40:04.792654+0000","last_became_peered":"2023-07-21T18:40:04.792654+0000","last_unstale":"2023-07-21T18:41:08.150860+0000","last_undegraded":"2023-07-21T18:41:08.150860+0000","last_fullsized":"2023-07-21T18:41:08.150860+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:03:06.046049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'30","reported_seq":210,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:28.377347+0000","last_change":"2023-07-21T18:40:04.178820+0000","last_active":"2023-07-21T18:41:28.377347+0000","last_peered":"2023-07-21T18:41:28.377347+0000","last_clean":"2023-07-21T18:41:28.377347+0000","last_became_active":"2023-07-21T18:40:04.178502+0000","last_became_peered":"2023-07-21T18:40:04.178502+0000","last_unstale":"2023-07-21T18:41:28.377347+0000","last_undegraded":"2023-07-21T18:41:28.377347+0000","last_fullsized":"2023-07-21T18:41:28.377347+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:19:51.865558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57193,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":162,"num_read_kb":136,"num_write":32,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.148850+0000","last_change":"2023-07-21T18:40:04.792921+0000","last_active":"2023-07-21T18:41:08.148850+0000","last_peered":"2023-07-21T18:41:08.148850+0000","last_clean":"2023-07-21T18:41:08.148850+0000","last_became_active":"2023-07-21T18:40:04.792608+0000","last_became_peered":"2023-07-21T18:40:04.792608+0000","last_unstale":"2023-07-21T18:41:08.148850+0000","last_undegraded":"2023-07-21T18:41:08.148850+0000","last_fullsized":"2023-07-21T18:41:08.148850+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:58:38.532900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'9","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.153057+0000","last_change":"2023-07-21T18:40:04.184917+0000","last_active":"2023-07-21T18:41:08.153057+0000","last_peered":"2023-07-21T18:41:08.153057+0000","last_clean":"2023-07-21T18:41:08.153057+0000","last_became_active":"2023-07-21T18:40:04.184612+0000","last_became_peered":"2023-07-21T18:40:04.184612+0000","last_unstale":"2023-07-21T18:41:08.153057+0000","last_undegraded":"2023-07-21T18:41:08.153057+0000","last_fullsized":"2023-07-21T18:41:08.153057+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:58:56.789436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.143966+0000","last_change":"2023-07-21T18:40:04.794134+0000","last_active":"2023-07-21T18:41:08.143966+0000","last_peered":"2023-07-21T18:41:08.143966+0000","last_clean":"2023-07-21T18:41:08.143966+0000","last_became_active":"2023-07-21T18:40:04.793784+0000","last_became_peered":"2023-07-21T18:40:04.793784+0000","last_unstale":"2023-07-21T18:41:08.143966+0000","last_undegraded":"2023-07-21T18:41:08.143966+0000","last_fullsized":"2023-07-21T18:41:08.143966+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:30:56.528513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'5","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.248332+0000","last_change":"2023-07-21T18:40:04.184889+0000","last_active":"2023-07-21T18:41:08.248332+0000","last_peered":"2023-07-21T18:41:08.248332+0000","last_clean":"2023-07-21T18:41:08.248332+0000","last_became_active":"2023-07-21T18:40:04.184516+0000","last_became_peered":"2023-07-21T18:40:04.184516+0000","last_unstale":"2023-07-21T18:41:08.248332+0000","last_undegraded":"2023-07-21T18:41:08.248332+0000","last_fullsized":"2023-07-21T18:41:08.248332+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:28:26.823194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.595621+0000","last_change":"2023-07-21T18:40:04.790627+0000","last_active":"2023-07-21T18:41:08.595621+0000","last_peered":"2023-07-21T18:41:08.595621+0000","last_clean":"2023-07-21T18:41:08.595621+0000","last_became_active":"2023-07-21T18:40:04.790298+0000","last_became_peered":"2023-07-21T18:40:04.790298+0000","last_unstale":"2023-07-21T18:41:08.595621+0000","last_undegraded":"2023-07-21T18:41:08.595621+0000","last_fullsized":"2023-07-21T18:41:08.595621+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:07:48.083182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"79'6","reported_seq":103,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:26.079011+0000","last_change":"2023-07-21T18:40:04.176156+0000","last_active":"2023-07-21T18:41:26.079011+0000","last_peered":"2023-07-21T18:41:26.079011+0000","last_clean":"2023-07-21T18:41:26.079011+0000","last_became_active":"2023-07-21T18:40:04.175823+0000","last_became_peered":"2023-07-21T18:40:04.175823+0000","last_unstale":"2023-07-21T18:41:26.079011+0000","last_undegraded":"2023-07-21T18:41:26.079011+0000","last_fullsized":"2023-07-21T18:41:26.079011+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:40:30.078845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":930,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":80,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.595323+0000","last_change":"2023-07-21T18:40:04.790422+0000","last_active":"2023-07-21T18:41:08.595323+0000","last_peered":"2023-07-21T18:41:08.595323+0000","last_clean":"2023-07-21T18:41:08.595323+0000","last_became_active":"2023-07-21T18:40:04.790124+0000","last_became_peered":"2023-07-21T18:40:04.790124+0000","last_unstale":"2023-07-21T18:41:08.595323+0000","last_undegraded":"2023-07-21T18:41:08.595323+0000","last_fullsized":"2023-07-21T18:41:08.595323+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:11:35.114265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'17","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.148291+0000","last_change":"2023-07-21T18:40:04.177006+0000","last_active":"2023-07-21T18:41:08.148291+0000","last_peered":"2023-07-21T18:41:08.148291+0000","last_clean":"2023-07-21T18:41:08.148291+0000","last_became_active":"2023-07-21T18:40:04.176711+0000","last_became_peered":"2023-07-21T18:40:04.176711+0000","last_unstale":"2023-07-21T18:41:08.148291+0000","last_undegraded":"2023-07-21T18:41:08.148291+0000","last_fullsized":"2023-07-21T18:41:08.148291+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:05:40.549509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.249206+0000","last_change":"2023-07-21T18:40:04.785601+0000","last_active":"2023-07-21T18:41:08.249206+0000","last_peered":"2023-07-21T18:41:08.249206+0000","last_clean":"2023-07-21T18:41:08.249206+0000","last_became_active":"2023-07-21T18:40:04.785307+0000","last_became_peered":"2023-07-21T18:40:04.785307+0000","last_unstale":"2023-07-21T18:41:08.249206+0000","last_undegraded":"2023-07-21T18:41:08.249206+0000","last_fullsized":"2023-07-21T18:41:08.249206+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:04:30.580644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.149118+0000","last_change":"2023-07-21T18:40:04.171005+0000","last_active":"2023-07-21T18:41:08.149118+0000","last_peered":"2023-07-21T18:41:08.149118+0000","last_clean":"2023-07-21T18:41:08.149118+0000","last_became_active":"2023-07-21T18:40:04.170719+0000","last_became_peered":"2023-07-21T18:40:04.170719+0000","last_unstale":"2023-07-21T18:41:08.149118+0000","last_undegraded":"2023-07-21T18:41:08.149118+0000","last_fullsized":"2023-07-21T18:41:08.149118+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:06:18.122712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.249987+0000","last_change":"2023-07-21T18:41:08.249987+0000","last_active":"2023-07-21T18:41:08.249987+0000","last_peered":"2023-07-21T18:41:08.249987+0000","last_clean":"2023-07-21T18:41:08.249987+0000","last_became_active":"2023-07-21T18:41:08.249610+0000","last_became_peered":"2023-07-21T18:41:08.249610+0000","last_unstale":"2023-07-21T18:41:08.249987+0000","last_undegraded":"2023-07-21T18:41:08.249987+0000","last_fullsized":"2023-07-21T18:41:08.249987+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:40:12.815463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"79'1","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.146358+0000","last_change":"2023-07-21T18:40:04.181069+0000","last_active":"2023-07-21T18:41:08.146358+0000","last_peered":"2023-07-21T18:41:08.146358+0000","last_clean":"2023-07-21T18:41:08.146358+0000","last_became_active":"2023-07-21T18:40:04.180750+0000","last_became_peered":"2023-07-21T18:40:04.180750+0000","last_unstale":"2023-07-21T18:41:08.146358+0000","last_undegraded":"2023-07-21T18:41:08.146358+0000","last_fullsized":"2023-07-21T18:41:08.146358+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:33:23.897099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"79'1","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.147933+0000","last_change":"2023-07-21T18:40:04.789940+0000","last_active":"2023-07-21T18:41:08.147933+0000","last_peered":"2023-07-21T18:41:08.147933+0000","last_clean":"2023-07-21T18:41:08.147933+0000","last_became_active":"2023-07-21T18:40:04.789465+0000","last_became_peered":"2023-07-21T18:40:04.789465+0000","last_unstale":"2023-07-21T18:41:08.147933+0000","last_undegraded":"2023-07-21T18:41:08.147933+0000","last_fullsized":"2023-07-21T18:41:08.147933+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:31:28.465807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"81'6","reported_seq":105,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:29.105557+0000","last_change":"2023-07-21T18:40:04.180522+0000","last_active":"2023-07-21T18:41:29.105557+0000","last_peered":"2023-07-21T18:41:29.105557+0000","last_clean":"2023-07-21T18:41:29.105557+0000","last_became_active":"2023-07-21T18:40:04.180196+0000","last_became_peered":"2023-07-21T18:40:04.180196+0000","last_unstale":"2023-07-21T18:41:29.105557+0000","last_undegraded":"2023-07-21T18:41:29.105557+0000","last_fullsized":"2023-07-21T18:41:29.105557+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:24:49.698830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":858,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.148148+0000","last_change":"2023-07-21T18:40:04.794781+0000","last_active":"2023-07-21T18:41:08.148148+0000","last_peered":"2023-07-21T18:41:08.148148+0000","last_clean":"2023-07-21T18:41:08.148148+0000","last_became_active":"2023-07-21T18:40:04.794388+0000","last_became_peered":"2023-07-21T18:40:04.794388+0000","last_unstale":"2023-07-21T18:41:08.148148+0000","last_undegraded":"2023-07-21T18:41:08.148148+0000","last_fullsized":"2023-07-21T18:41:08.148148+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:01:02.955504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.248717+0000","last_change":"2023-07-21T18:40:04.786408+0000","last_active":"2023-07-21T18:41:08.248717+0000","last_peered":"2023-07-21T18:41:08.248717+0000","last_clean":"2023-07-21T18:41:08.248717+0000","last_became_active":"2023-07-21T18:40:04.785555+0000","last_became_peered":"2023-07-21T18:40:04.785555+0000","last_unstale":"2023-07-21T18:41:08.248717+0000","last_undegraded":"2023-07-21T18:41:08.248717+0000","last_fullsized":"2023-07-21T18:41:08.248717+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:45:51.047733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"79'5","reported_seq":102,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:27.080056+0000","last_change":"2023-07-21T18:40:04.185980+0000","last_active":"2023-07-21T18:41:27.080056+0000","last_peered":"2023-07-21T18:41:27.080056+0000","last_clean":"2023-07-21T18:41:27.080056+0000","last_became_active":"2023-07-21T18:40:04.185618+0000","last_became_peered":"2023-07-21T18:40:04.185618+0000","last_unstale":"2023-07-21T18:41:27.080056+0000","last_undegraded":"2023-07-21T18:41:27.080056+0000","last_fullsized":"2023-07-21T18:41:27.080056+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:37:27.318460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":79,"num_read_kb":79,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.149558+0000","last_change":"2023-07-21T18:40:04.792535+0000","last_active":"2023-07-21T18:41:08.149558+0000","last_peered":"2023-07-21T18:41:08.149558+0000","last_clean":"2023-07-21T18:41:08.149558+0000","last_became_active":"2023-07-21T18:40:04.792239+0000","last_became_peered":"2023-07-21T18:40:04.792239+0000","last_unstale":"2023-07-21T18:41:08.149558+0000","last_undegraded":"2023-07-21T18:41:08.149558+0000","last_fullsized":"2023-07-21T18:41:08.149558+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:34:19.596250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"79'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.244642+0000","last_change":"2023-07-21T18:40:04.173539+0000","last_active":"2023-07-21T18:41:08.244642+0000","last_peered":"2023-07-21T18:41:08.244642+0000","last_clean":"2023-07-21T18:41:08.244642+0000","last_became_active":"2023-07-21T18:40:04.173225+0000","last_became_peered":"2023-07-21T18:40:04.173225+0000","last_unstale":"2023-07-21T18:41:08.244642+0000","last_undegraded":"2023-07-21T18:41:08.244642+0000","last_fullsized":"2023-07-21T18:41:08.244642+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:14:31.670842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":341,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.595315+0000","last_change":"2023-07-21T18:39:05.999983+0000","last_active":"2023-07-21T18:41:08.595315+0000","last_peered":"2023-07-21T18:41:08.595315+0000","last_clean":"2023-07-21T18:41:08.595315+0000","last_became_active":"2023-07-21T18:39:05.999495+0000","last_became_peered":"2023-07-21T18:39:05.999495+0000","last_unstale":"2023-07-21T18:41:08.595315+0000","last_undegraded":"2023-07-21T18:41:08.595315+0000","last_fullsized":"2023-07-21T18:41:08.595315+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:36:14.004429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:36:14.004429+0000","last_clean_scrub_stamp":"2023-07-21T18:36:14.004429+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:13:55.308992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.143732+0000","last_change":"2023-07-21T18:40:04.785086+0000","last_active":"2023-07-21T18:41:08.143732+0000","last_peered":"2023-07-21T18:41:08.143732+0000","last_clean":"2023-07-21T18:41:08.143732+0000","last_became_active":"2023-07-21T18:40:04.784693+0000","last_became_peered":"2023-07-21T18:40:04.784693+0000","last_unstale":"2023-07-21T18:41:08.143732+0000","last_undegraded":"2023-07-21T18:41:08.143732+0000","last_fullsized":"2023-07-21T18:41:08.143732+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:20:44.967039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"79'4","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.595119+0000","last_change":"2023-07-21T18:40:04.177032+0000","last_active":"2023-07-21T18:41:08.595119+0000","last_peered":"2023-07-21T18:41:08.595119+0000","last_clean":"2023-07-21T18:41:08.595119+0000","last_became_active":"2023-07-21T18:40:04.176741+0000","last_became_peered":"2023-07-21T18:40:04.176741+0000","last_unstale":"2023-07-21T18:41:08.595119+0000","last_undegraded":"2023-07-21T18:41:08.595119+0000","last_fullsized":"2023-07-21T18:41:08.595119+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:30:00.615431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"79'4","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.148885+0000","last_change":"2023-07-21T18:40:04.180106+0000","last_active":"2023-07-21T18:41:08.148885+0000","last_peered":"2023-07-21T18:41:08.148885+0000","last_clean":"2023-07-21T18:41:08.148885+0000","last_became_active":"2023-07-21T18:40:04.179802+0000","last_became_peered":"2023-07-21T18:40:04.179802+0000","last_unstale":"2023-07-21T18:41:08.148885+0000","last_undegraded":"2023-07-21T18:41:08.148885+0000","last_fullsized":"2023-07-21T18:41:08.148885+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:58:17.009228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.144906+0000","last_change":"2023-07-21T18:40:04.789127+0000","last_active":"2023-07-21T18:41:08.144906+0000","last_peered":"2023-07-21T18:41:08.144906+0000","last_clean":"2023-07-21T18:41:08.144906+0000","last_became_active":"2023-07-21T18:40:04.788815+0000","last_became_peered":"2023-07-21T18:40:04.788815+0000","last_unstale":"2023-07-21T18:41:08.144906+0000","last_undegraded":"2023-07-21T18:41:08.144906+0000","last_fullsized":"2023-07-21T18:41:08.144906+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:31:07.061759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"79'9","reported_seq":43,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.156947+0000","last_change":"2023-07-21T18:41:08.156947+0000","last_active":"2023-07-21T18:41:08.156947+0000","last_peered":"2023-07-21T18:41:08.156947+0000","last_clean":"2023-07-21T18:41:08.156947+0000","last_became_active":"2023-07-21T18:41:08.149966+0000","last_became_peered":"2023-07-21T18:41:08.149966+0000","last_unstale":"2023-07-21T18:41:08.156947+0000","last_undegraded":"2023-07-21T18:41:08.156947+0000","last_fullsized":"2023-07-21T18:41:08.156947+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:17:05.137573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.149731+0000","last_change":"2023-07-21T18:40:04.791937+0000","last_active":"2023-07-21T18:41:08.149731+0000","last_peered":"2023-07-21T18:41:08.149731+0000","last_clean":"2023-07-21T18:41:08.149731+0000","last_became_active":"2023-07-21T18:40:04.791588+0000","last_became_peered":"2023-07-21T18:40:04.791588+0000","last_unstale":"2023-07-21T18:41:08.149731+0000","last_undegraded":"2023-07-21T18:41:08.149731+0000","last_fullsized":"2023-07-21T18:41:08.149731+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:17:39.327055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'7","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.244822+0000","last_change":"2023-07-21T18:40:04.177007+0000","last_active":"2023-07-21T18:41:08.244822+0000","last_peered":"2023-07-21T18:41:08.244822+0000","last_clean":"2023-07-21T18:41:08.244822+0000","last_became_active":"2023-07-21T18:40:04.176699+0000","last_became_peered":"2023-07-21T18:40:04.176699+0000","last_unstale":"2023-07-21T18:41:08.244822+0000","last_undegraded":"2023-07-21T18:41:08.244822+0000","last_fullsized":"2023-07-21T18:41:08.244822+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:20:14.547927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.155110+0000","last_change":"2023-07-21T18:40:04.795671+0000","last_active":"2023-07-21T18:41:08.155110+0000","last_peered":"2023-07-21T18:41:08.155110+0000","last_clean":"2023-07-21T18:41:08.155110+0000","last_became_active":"2023-07-21T18:40:04.795233+0000","last_became_peered":"2023-07-21T18:40:04.795233+0000","last_unstale":"2023-07-21T18:41:08.155110+0000","last_undegraded":"2023-07-21T18:41:08.155110+0000","last_fullsized":"2023-07-21T18:41:08.155110+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:57:42.725756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.159479+0000","last_change":"2023-07-21T18:41:08.159479+0000","last_active":"2023-07-21T18:41:08.159479+0000","last_peered":"2023-07-21T18:41:08.159479+0000","last_clean":"2023-07-21T18:41:08.159479+0000","last_became_active":"2023-07-21T18:41:08.156027+0000","last_became_peered":"2023-07-21T18:41:08.156027+0000","last_unstale":"2023-07-21T18:41:08.159479+0000","last_undegraded":"2023-07-21T18:41:08.159479+0000","last_fullsized":"2023-07-21T18:41:08.159479+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:36:55.454682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.148310+0000","last_change":"2023-07-21T18:40:04.793091+0000","last_active":"2023-07-21T18:41:08.148310+0000","last_peered":"2023-07-21T18:41:08.148310+0000","last_clean":"2023-07-21T18:41:08.148310+0000","last_became_active":"2023-07-21T18:40:04.792723+0000","last_became_peered":"2023-07-21T18:40:04.792723+0000","last_unstale":"2023-07-21T18:41:08.148310+0000","last_undegraded":"2023-07-21T18:41:08.148310+0000","last_fullsized":"2023-07-21T18:41:08.148310+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:25:52.341093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'7","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.149262+0000","last_change":"2023-07-21T18:40:04.182867+0000","last_active":"2023-07-21T18:41:08.149262+0000","last_peered":"2023-07-21T18:41:08.149262+0000","last_clean":"2023-07-21T18:41:08.149262+0000","last_became_active":"2023-07-21T18:40:04.181885+0000","last_became_peered":"2023-07-21T18:40:04.181885+0000","last_unstale":"2023-07-21T18:41:08.149262+0000","last_undegraded":"2023-07-21T18:41:08.149262+0000","last_fullsized":"2023-07-21T18:41:08.149262+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:44:58.697667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.143574+0000","last_change":"2023-07-21T18:40:04.790340+0000","last_active":"2023-07-21T18:41:08.143574+0000","last_peered":"2023-07-21T18:41:08.143574+0000","last_clean":"2023-07-21T18:41:08.143574+0000","last_became_active":"2023-07-21T18:40:04.790014+0000","last_became_peered":"2023-07-21T18:40:04.790014+0000","last_unstale":"2023-07-21T18:41:08.143574+0000","last_undegraded":"2023-07-21T18:41:08.143574+0000","last_fullsized":"2023-07-21T18:41:08.143574+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:01:01.514954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'3","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.152586+0000","last_change":"2023-07-21T18:40:04.184986+0000","last_active":"2023-07-21T18:41:08.152586+0000","last_peered":"2023-07-21T18:41:08.152586+0000","last_clean":"2023-07-21T18:41:08.152586+0000","last_became_active":"2023-07-21T18:40:04.184672+0000","last_became_peered":"2023-07-21T18:40:04.184672+0000","last_unstale":"2023-07-21T18:41:08.152586+0000","last_undegraded":"2023-07-21T18:41:08.152586+0000","last_fullsized":"2023-07-21T18:41:08.152586+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:56:51.290795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.149573+0000","last_change":"2023-07-21T18:40:04.793272+0000","last_active":"2023-07-21T18:41:08.149573+0000","last_peered":"2023-07-21T18:41:08.149573+0000","last_clean":"2023-07-21T18:41:08.149573+0000","last_became_active":"2023-07-21T18:40:04.792854+0000","last_became_peered":"2023-07-21T18:40:04.792854+0000","last_unstale":"2023-07-21T18:41:08.149573+0000","last_undegraded":"2023-07-21T18:41:08.149573+0000","last_fullsized":"2023-07-21T18:41:08.149573+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:13:19.927149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.148619+0000","last_change":"2023-07-21T18:40:04.176404+0000","last_active":"2023-07-21T18:41:08.148619+0000","last_peered":"2023-07-21T18:41:08.148619+0000","last_clean":"2023-07-21T18:41:08.148619+0000","last_became_active":"2023-07-21T18:40:04.176148+0000","last_became_peered":"2023-07-21T18:40:04.176148+0000","last_unstale":"2023-07-21T18:41:08.148619+0000","last_undegraded":"2023-07-21T18:41:08.148619+0000","last_fullsized":"2023-07-21T18:41:08.148619+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:06:45.638197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.595565+0000","last_change":"2023-07-21T18:40:04.795628+0000","last_active":"2023-07-21T18:41:08.595565+0000","last_peered":"2023-07-21T18:41:08.595565+0000","last_clean":"2023-07-21T18:41:08.595565+0000","last_became_active":"2023-07-21T18:40:04.795248+0000","last_became_peered":"2023-07-21T18:40:04.795248+0000","last_unstale":"2023-07-21T18:41:08.595565+0000","last_undegraded":"2023-07-21T18:41:08.595565+0000","last_fullsized":"2023-07-21T18:41:08.595565+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:54:09.726387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'5","reported_seq":103,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:27.081526+0000","last_change":"2023-07-21T18:40:04.184380+0000","last_active":"2023-07-21T18:41:27.081526+0000","last_peered":"2023-07-21T18:41:27.081526+0000","last_clean":"2023-07-21T18:41:27.081526+0000","last_became_active":"2023-07-21T18:40:04.184045+0000","last_became_peered":"2023-07-21T18:40:04.184045+0000","last_unstale":"2023-07-21T18:41:27.081526+0000","last_undegraded":"2023-07-21T18:41:27.081526+0000","last_fullsized":"2023-07-21T18:41:27.081526+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:33:19.018900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":80,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.595539+0000","last_change":"2023-07-21T18:40:04.787760+0000","last_active":"2023-07-21T18:41:08.595539+0000","last_peered":"2023-07-21T18:41:08.595539+0000","last_clean":"2023-07-21T18:41:08.595539+0000","last_became_active":"2023-07-21T18:40:04.787371+0000","last_became_peered":"2023-07-21T18:40:04.787371+0000","last_unstale":"2023-07-21T18:41:08.595539+0000","last_undegraded":"2023-07-21T18:41:08.595539+0000","last_fullsized":"2023-07-21T18:41:08.595539+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:24:52.752764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'3","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.248814+0000","last_change":"2023-07-21T18:40:04.184721+0000","last_active":"2023-07-21T18:41:08.248814+0000","last_peered":"2023-07-21T18:41:08.248814+0000","last_clean":"2023-07-21T18:41:08.248814+0000","last_became_active":"2023-07-21T18:40:04.184070+0000","last_became_peered":"2023-07-21T18:40:04.184070+0000","last_unstale":"2023-07-21T18:41:08.248814+0000","last_undegraded":"2023-07-21T18:41:08.248814+0000","last_fullsized":"2023-07-21T18:41:08.248814+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:57:29.552146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"79'1","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.595437+0000","last_change":"2023-07-21T18:40:04.786963+0000","last_active":"2023-07-21T18:41:08.595437+0000","last_peered":"2023-07-21T18:41:08.595437+0000","last_clean":"2023-07-21T18:41:08.595437+0000","last_became_active":"2023-07-21T18:40:04.786617+0000","last_became_peered":"2023-07-21T18:40:04.786617+0000","last_unstale":"2023-07-21T18:41:08.595437+0000","last_undegraded":"2023-07-21T18:41:08.595437+0000","last_fullsized":"2023-07-21T18:41:08.595437+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:28:47.943499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'5","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.144548+0000","last_change":"2023-07-21T18:40:04.184509+0000","last_active":"2023-07-21T18:41:08.144548+0000","last_peered":"2023-07-21T18:41:08.144548+0000","last_clean":"2023-07-21T18:41:08.144548+0000","last_became_active":"2023-07-21T18:40:04.184208+0000","last_became_peered":"2023-07-21T18:40:04.184208+0000","last_unstale":"2023-07-21T18:41:08.144548+0000","last_undegraded":"2023-07-21T18:41:08.144548+0000","last_fullsized":"2023-07-21T18:41:08.144548+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:03.153723+0000","last_clean_scrub_stamp":"2023-07-21T18:40:03.153723+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:21:46.199399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.595579+0000","last_change":"2023-07-21T18:40:04.788248+0000","last_active":"2023-07-21T18:41:08.595579+0000","last_peered":"2023-07-21T18:41:08.595579+0000","last_clean":"2023-07-21T18:41:08.595579+0000","last_became_active":"2023-07-21T18:40:04.787974+0000","last_became_peered":"2023-07-21T18:40:04.787974+0000","last_unstale":"2023-07-21T18:41:08.595579+0000","last_undegraded":"2023-07-21T18:41:08.595579+0000","last_fullsized":"2023-07-21T18:41:08.595579+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:28:27.418555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.143620+0000","last_change":"2023-07-21T18:40:04.790522+0000","last_active":"2023-07-21T18:41:08.143620+0000","last_peered":"2023-07-21T18:41:08.143620+0000","last_clean":"2023-07-21T18:41:08.143620+0000","last_became_active":"2023-07-21T18:40:04.790178+0000","last_became_peered":"2023-07-21T18:40:04.790178+0000","last_unstale":"2023-07-21T18:41:08.143620+0000","last_undegraded":"2023-07-21T18:41:08.143620+0000","last_fullsized":"2023-07-21T18:41:08.143620+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:06:13.808345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.152539+0000","last_change":"2023-07-21T18:40:04.790403+0000","last_active":"2023-07-21T18:41:08.152539+0000","last_peered":"2023-07-21T18:41:08.152539+0000","last_clean":"2023-07-21T18:41:08.152539+0000","last_became_active":"2023-07-21T18:40:04.790106+0000","last_became_peered":"2023-07-21T18:40:04.790106+0000","last_unstale":"2023-07-21T18:41:08.152539+0000","last_undegraded":"2023-07-21T18:41:08.152539+0000","last_fullsized":"2023-07-21T18:41:08.152539+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:04:45.154675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.152548+0000","last_change":"2023-07-21T18:40:04.795230+0000","last_active":"2023-07-21T18:41:08.152548+0000","last_peered":"2023-07-21T18:41:08.152548+0000","last_clean":"2023-07-21T18:41:08.152548+0000","last_became_active":"2023-07-21T18:40:04.794949+0000","last_became_peered":"2023-07-21T18:40:04.794949+0000","last_unstale":"2023-07-21T18:41:08.152548+0000","last_undegraded":"2023-07-21T18:41:08.152548+0000","last_fullsized":"2023-07-21T18:41:08.152548+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T18:42:31.029744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.154417+0000","last_change":"2023-07-21T18:40:04.792736+0000","last_active":"2023-07-21T18:41:08.154417+0000","last_peered":"2023-07-21T18:41:08.154417+0000","last_clean":"2023-07-21T18:41:08.154417+0000","last_became_active":"2023-07-21T18:40:04.792301+0000","last_became_peered":"2023-07-21T18:40:04.792301+0000","last_unstale":"2023-07-21T18:41:08.154417+0000","last_undegraded":"2023-07-21T18:41:08.154417+0000","last_fullsized":"2023-07-21T18:41:08.154417+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:45:51.320918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.145009+0000","last_change":"2023-07-21T18:40:04.790733+0000","last_active":"2023-07-21T18:41:08.145009+0000","last_peered":"2023-07-21T18:41:08.145009+0000","last_clean":"2023-07-21T18:41:08.145009+0000","last_became_active":"2023-07-21T18:40:04.790441+0000","last_became_peered":"2023-07-21T18:40:04.790441+0000","last_unstale":"2023-07-21T18:41:08.145009+0000","last_undegraded":"2023-07-21T18:41:08.145009+0000","last_fullsized":"2023-07-21T18:41:08.145009+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:40:00.802774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.143768+0000","last_change":"2023-07-21T18:40:04.790853+0000","last_active":"2023-07-21T18:41:08.143768+0000","last_peered":"2023-07-21T18:41:08.143768+0000","last_clean":"2023-07-21T18:41:08.143768+0000","last_became_active":"2023-07-21T18:40:04.790373+0000","last_became_peered":"2023-07-21T18:40:04.790373+0000","last_unstale":"2023-07-21T18:41:08.143768+0000","last_undegraded":"2023-07-21T18:41:08.143768+0000","last_fullsized":"2023-07-21T18:41:08.143768+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:29:27.642249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.153055+0000","last_change":"2023-07-21T18:40:04.794088+0000","last_active":"2023-07-21T18:41:08.153055+0000","last_peered":"2023-07-21T18:41:08.153055+0000","last_clean":"2023-07-21T18:41:08.153055+0000","last_became_active":"2023-07-21T18:40:04.793782+0000","last_became_peered":"2023-07-21T18:40:04.793782+0000","last_unstale":"2023-07-21T18:41:08.153055+0000","last_undegraded":"2023-07-21T18:41:08.153055+0000","last_fullsized":"2023-07-21T18:41:08.153055+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:10:46.116604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.149451+0000","last_change":"2023-07-21T18:40:04.791772+0000","last_active":"2023-07-21T18:41:08.149451+0000","last_peered":"2023-07-21T18:41:08.149451+0000","last_clean":"2023-07-21T18:41:08.149451+0000","last_became_active":"2023-07-21T18:40:04.791379+0000","last_became_peered":"2023-07-21T18:40:04.791379+0000","last_unstale":"2023-07-21T18:41:08.149451+0000","last_undegraded":"2023-07-21T18:41:08.149451+0000","last_fullsized":"2023-07-21T18:41:08.149451+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:01:22.491116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.248658+0000","last_change":"2023-07-21T18:40:04.786579+0000","last_active":"2023-07-21T18:41:08.248658+0000","last_peered":"2023-07-21T18:41:08.248658+0000","last_clean":"2023-07-21T18:41:08.248658+0000","last_became_active":"2023-07-21T18:40:04.786032+0000","last_became_peered":"2023-07-21T18:40:04.786032+0000","last_unstale":"2023-07-21T18:41:08.248658+0000","last_undegraded":"2023-07-21T18:41:08.248658+0000","last_fullsized":"2023-07-21T18:41:08.248658+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:26:58.238429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.149279+0000","last_change":"2023-07-21T18:40:04.793441+0000","last_active":"2023-07-21T18:41:08.149279+0000","last_peered":"2023-07-21T18:41:08.149279+0000","last_clean":"2023-07-21T18:41:08.149279+0000","last_became_active":"2023-07-21T18:40:04.791441+0000","last_became_peered":"2023-07-21T18:40:04.791441+0000","last_unstale":"2023-07-21T18:41:08.149279+0000","last_undegraded":"2023-07-21T18:41:08.149279+0000","last_fullsized":"2023-07-21T18:41:08.149279+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T18:47:16.762634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.151030+0000","last_change":"2023-07-21T18:40:04.794913+0000","last_active":"2023-07-21T18:41:08.151030+0000","last_peered":"2023-07-21T18:41:08.151030+0000","last_clean":"2023-07-21T18:41:08.151030+0000","last_became_active":"2023-07-21T18:40:04.794527+0000","last_became_peered":"2023-07-21T18:40:04.794527+0000","last_unstale":"2023-07-21T18:41:08.151030+0000","last_undegraded":"2023-07-21T18:41:08.151030+0000","last_fullsized":"2023-07-21T18:41:08.151030+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:14:55.276602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.144282+0000","last_change":"2023-07-21T18:40:04.789145+0000","last_active":"2023-07-21T18:41:08.144282+0000","last_peered":"2023-07-21T18:41:08.144282+0000","last_clean":"2023-07-21T18:41:08.144282+0000","last_became_active":"2023-07-21T18:40:04.788285+0000","last_became_peered":"2023-07-21T18:40:04.788285+0000","last_unstale":"2023-07-21T18:41:08.144282+0000","last_undegraded":"2023-07-21T18:41:08.144282+0000","last_fullsized":"2023-07-21T18:41:08.144282+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:50:38.851180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-21T18:41:08.143670+0000","last_change":"2023-07-21T18:40:04.788821+0000","last_active":"2023-07-21T18:41:08.143670+0000","last_peered":"2023-07-21T18:41:08.143670+0000","last_clean":"2023-07-21T18:41:08.143670+0000","last_became_active":"2023-07-21T18:40:04.788513+0000","last_became_peered":"2023-07-21T18:40:04.788513+0000","last_unstale":"2023-07-21T18:41:08.143670+0000","last_undegraded":"2023-07-21T18:41:08.143670+0000","last_fullsized":"2023-07-21T18:41:08.143670+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T18:40:04.159126+0000","last_clean_scrub_stamp":"2023-07-21T18:40:04.159126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:37:17.944763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64131,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":817,"num_read_kb":477,"num_write":233,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":191757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":204,"ondisk_log_size":204,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579933,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27816,"kb_used_data":864,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717336,"statfs":{"total":95995035648,"available":95966552064,"internally_reserved":0,"allocated":884736,"data_stored":426096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:41:24 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.54300000000000004,"15min":0.54700000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.78600000000000003,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.629,"15min":0.63900000000000001},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.748,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.54700000000000004}]},{"osd":1,"last update":"Fri Jul 21 18:41:24 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.63900000000000001,"15min":0.63300000000000001},"min":{"1min":0.50800000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.747},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.61699999999999999,"15min":0.61099999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.85899999999999999,"5min":1.137,"15min":1.137},"last":0.64000000000000001}]},{"osd":2,"last update":"Fri Jul 21 18:41:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.502,"15min":0.495},"min":{"1min":0.378,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.502,"5min":0.47299999999999998,"15min":0.46800000000000003},"min":{"1min":0.32600000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.64400000000000002,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.60299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":4,"last update":"Fri Jul 21 18:41:24 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.64300000000000002,"15min":0.63300000000000001},"min":{"1min":0.45700000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.60199999999999998,"15min":0.59099999999999997},"min":{"1min":0.46700000000000003,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.79300000000000004}]},{"osd":5,"last update":"Fri Jul 21 18:41:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.56799999999999995,"15min":0.56000000000000005},"min":{"1min":0.50600000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.57299999999999995,"15min":0.56599999999999995},"min":{"1min":0.39300000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.75700000000000001,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.81200000000000006}]},{"osd":6,"last update":"Fri Jul 21 18:41:24 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.64800000000000002,"15min":0.65100000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.88700000000000001,"5min":1.181,"15min":1.181},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.755,"15min":0.75900000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.92500000000000004,"5min":1.425,"15min":1.425},"last":0.88400000000000001}]},{"osd":7,"last update":"Fri Jul 21 18:41:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.72799999999999998,"15min":0.71999999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.91200000000000003,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.752,"5min":0.70999999999999996,"15min":0.70299999999999996},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.91500000000000004,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.98099999999999998}]},{"osd":8,"last update":"Fri Jul 21 18:41:06 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.999}]},{"osd":9,"last update":"Fri Jul 21 18:41:24 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.80500000000000005,"15min":0.80200000000000005},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.98799999999999999,"5min":1.643,"15min":1.643},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.78800000000000003,"15min":0.78400000000000003},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.95899999999999996,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":1.0269999999999999}]},{"osd":10,"last update":"Fri Jul 21 18:41:24 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.76900000000000002,"15min":0.76200000000000001},"min":{"1min":0.56000000000000005,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.95899999999999996,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.56999999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.98099999999999998,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.96499999999999997}]}]},{"osd":10,"up_from":67,"seq":287762808864,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28492,"kb_used_data":1540,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716660,"statfs":{"total":95995035648,"available":95965859840,"internally_reserved":0,"allocated":1576960,"data_stored":1074383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:41:08 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.40999999999999998}]},{"osd":1,"last update":"Fri Jul 21 18:41:08 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.46800000000000003,"15min":0.46600000000000003},"min":{"1min":0.29399999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.68600000000000005,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.46500000000000002,"15min":0.46100000000000002},"min":{"1min":0.33200000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.624,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.49099999999999999}]},{"osd":2,"last update":"Fri Jul 21 18:41:08 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.60699999999999998,"15min":0.56200000000000006},"min":{"1min":0.54100000000000004,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":4.0129999999999999,"5min":4.0129999999999999,"15min":4.0129999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.622,"15min":0.60999999999999999},"min":{"1min":0.45600000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.82799999999999996,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.59199999999999997}]},{"osd":3,"last update":"Fri Jul 21 18:41:08 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.60499999999999998,"15min":0.60099999999999998},"min":{"1min":0.434,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.746,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.621,"5min":0.60499999999999998,"15min":0.60199999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.439}]},{"osd":4,"last update":"Fri Jul 21 18:41:08 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.57699999999999996,"15min":0.57199999999999995},"min":{"1min":0.42599999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.73099999999999998,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.59099999999999997,"15min":0.59299999999999997},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.84699999999999998,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.52600000000000002}]},{"osd":5,"last update":"Fri Jul 21 18:41:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.69299999999999995,"15min":0.68700000000000006},"min":{"1min":0.49399999999999999,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.69699999999999995,"15min":0.68300000000000005},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.621}]},{"osd":6,"last update":"Fri Jul 21 18:41:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.69299999999999995,"15min":0.68899999999999995},"min":{"1min":0.505,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.874,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.46999999999999997,"5min":0.442,"15min":0.442},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.60499999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":8,"last update":"Fri Jul 21 18:41:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.73699999999999999,"15min":0.72899999999999998},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.65700000000000003},{"interface":"front","average":{"1min":1.01,"5min":0.81100000000000005,"15min":0.77800000000000002},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":4.0419999999999998,"5min":4.0419999999999998,"15min":4.0419999999999998},"last":0.72299999999999998}]},{"osd":9,"last update":"Fri Jul 21 18:41:08 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.75900000000000001,"15min":0.75700000000000001},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.75,"5min":0.77800000000000002,"15min":0.78200000000000003},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.92100000000000004,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.68899999999999995}]},{"osd":11,"last update":"Fri Jul 21 18:40:28 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.75}]}]},{"osd":9,"up_from":62,"seq":266287972389,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27840,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717312,"statfs":{"total":95995035648,"available":95966527488,"internally_reserved":0,"allocated":909312,"data_stored":426074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.52500000000000002,"15min":0.53900000000000003},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.996,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.438,"5min":0.505,"15min":0.51600000000000001},"min":{"1min":0.316,"5min":0.252,"15min":0.252},"max":{"1min":0.621,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.33100000000000002}]},{"osd":1,"last update":"Fri Jul 21 18:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.73199999999999998,"15min":0.749},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.099,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":0.627},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.71299999999999997,"15min":0.72099999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.169,"5min":1.681,"15min":1.681},"last":0.59899999999999998}]},{"osd":2,"last update":"Fri Jul 21 18:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.70799999999999996,"15min":0.71399999999999997},"min":{"1min":0.47299999999999998,"5min":0.38,"15min":0.38},"max":{"1min":1.222,"5min":1.9530000000000001,"15min":1.9530000000000001},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.78300000000000003,"15min":0.753},"min":{"1min":0.46100000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":6.4020000000000001,"5min":6.4020000000000001,"15min":6.4020000000000001},"last":0.63900000000000001}]},{"osd":3,"last update":"Fri Jul 21 18:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.627,"15min":0.64900000000000002},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.89000000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.627,"15min":0.64500000000000002},"min":{"1min":0.35199999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.1910000000000001,"5min":2.048,"15min":2.048},"last":0.38}]},{"osd":4,"last update":"Fri Jul 21 18:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.80700000000000005,"15min":0.83699999999999997},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.127,"5min":1.8100000000000001,"15min":1.8100000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.79800000000000004,"15min":0.82299999999999995},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.94799999999999995,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.748}]},{"osd":5,"last update":"Fri Jul 21 18:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.80500000000000005,"15min":0.77700000000000002},"min":{"1min":0.49299999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":6.5069999999999997,"5min":6.5069999999999997,"15min":6.5069999999999997},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.78600000000000003,"15min":0.80400000000000005},"min":{"1min":0.503,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.1399999999999999,"5min":1.992,"15min":1.992},"last":0.56299999999999994}]},{"osd":6,"last update":"Fri Jul 21 18:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.74299999999999999,"15min":0.77300000000000002},"min":{"1min":0.375,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.1240000000000001,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.78600000000000003,"15min":0.81399999999999995},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.2410000000000001,"5min":1.595,"15min":1.595},"last":0.77900000000000003}]},{"osd":7,"last update":"Fri Jul 21 18:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.85799999999999998,"15min":0.88200000000000001},"min":{"1min":0.54700000000000004,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.026,"5min":1.284,"15min":1.284},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.89300000000000002,"15min":0.92000000000000004},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.175,"5min":2.0750000000000002,"15min":2.0750000000000002},"last":0.60999999999999999}]},{"osd":8,"last update":"Fri Jul 21 18:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.90700000000000003,"15min":0.92900000000000005},"min":{"1min":0.61499999999999999,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.2749999999999999,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":1.0669999999999999,"5min":0.90600000000000003,"15min":0.879},"min":{"1min":0.60899999999999999,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":6.5679999999999996,"5min":6.5679999999999996,"15min":6.5679999999999996},"last":0.79900000000000004}]},{"osd":10,"last update":"Fri Jul 21 18:41:08 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.86699999999999999,"15min":0.88700000000000001},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.155,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.86399999999999999,"15min":0.88200000000000001},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":0.76500000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78900000000000003}]}]},{"osd":8,"up_from":55,"seq":236223201322,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27832,"kb_used_data":880,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717320,"statfs":{"total":95995035648,"available":95966535680,"internally_reserved":0,"allocated":901120,"data_stored":426121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:41:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.36299999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.81899999999999995,"5min":4.2000000000000002,"15min":4.2000000000000002},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.58699999999999997,"15min":0.58399999999999996},"min":{"1min":0.33200000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.747,"5min":3.145,"15min":3.145},"last":0.50800000000000001}]},{"osd":1,"last update":"Fri Jul 21 18:41:27 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.622,"15min":0.59199999999999997},"min":{"1min":0.45500000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.91300000000000003,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.627,"15min":0.61699999999999999},"min":{"1min":0.49399999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.92600000000000005,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.87}]},{"osd":2,"last update":"Fri Jul 21 18:41:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.47999999999999998,"15min":0.46500000000000002},"min":{"1min":0.39400000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.48699999999999999,"15min":0.47099999999999997},"min":{"1min":0.34399999999999997,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.77700000000000002,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.85199999999999998}]},{"osd":3,"last update":"Fri Jul 21 18:41:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.72799999999999998,"15min":0.68300000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.85099999999999998,"5min":11.260999999999999,"15min":11.260999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.61799999999999999,"15min":0.62},"min":{"1min":0.441,"5min":0.371,"15min":0.371},"max":{"1min":0.72499999999999998,"5min":3.2719999999999998,"15min":3.2719999999999998},"last":0.56200000000000006}]},{"osd":4,"last update":"Fri Jul 21 18:41:27 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.58399999999999996,"15min":0.56999999999999995},"min":{"1min":0.46899999999999997,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.80000000000000004,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.64600000000000002,"15min":0.63400000000000001},"min":{"1min":0.54600000000000004,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.86299999999999999,"5min":0.875,"15min":0.875},"last":0.82699999999999996}]},{"osd":5,"last update":"Fri Jul 21 18:41:27 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.59399999999999997,"15min":0.58699999999999997},"min":{"1min":0.308,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.53900000000000003,"15min":0.53500000000000003},"min":{"1min":0.36599999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.70299999999999996}]},{"osd":6,"last update":"Fri Jul 21 18:41:27 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.78400000000000003,"15min":0.73699999999999999},"min":{"1min":0.47299999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.98099999999999998,"5min":10.292999999999999,"15min":10.292999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.76600000000000001},"min":{"1min":0.53600000000000003,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.018,"5min":4.1029999999999998,"15min":4.1029999999999998},"last":0.90300000000000002}]},{"osd":7,"last update":"Fri Jul 21 18:41:27 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.73099999999999998,"15min":0.72099999999999997},"min":{"1min":0.56699999999999995,"5min":0.432,"15min":0.432},"max":{"1min":1.0029999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.67900000000000005,"15min":0.67500000000000004},"min":{"1min":0.497,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.88700000000000001}]},{"osd":9,"last update":"Fri Jul 21 18:40:46 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.54100000000000004,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.95199999999999996,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.92000000000000004},{"interface":"front","average":{"1min":1.2509999999999999,"5min":0.86699999999999999,"15min":0.80300000000000005},"min":{"1min":0.54900000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":11.151,"5min":11.151,"15min":11.151},"last":0.98299999999999998}]},{"osd":10,"last update":"Fri Jul 21 18:41:05 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.71599999999999997,"15min":0.69799999999999995},"min":{"1min":0.59899999999999998,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.996},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.73499999999999999,"15min":0.71399999999999997},"min":{"1min":0.73599999999999999,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":1.0089999999999999}]},{"osd":11,"last update":"Fri Jul 21 18:41:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.96199999999999997}]}]},{"osd":7,"up_from":48,"seq":206158430254,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28416,"kb_used_data":1464,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716736,"statfs":{"total":95995035648,"available":95965937664,"internally_reserved":0,"allocated":1499136,"data_stored":1018876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.53400000000000003,"15min":0.52600000000000002},"min":{"1min":0.41599999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.53900000000000003,"15min":0.53600000000000003},"min":{"1min":0.34000000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.48899999999999999}]},{"osd":1,"last update":"Fri Jul 21 18:41:03 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.42099999999999999,"15min":0.40699999999999997},"min":{"1min":0.29199999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.70799999999999996,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.442,"15min":0.42599999999999999},"min":{"1min":0.32400000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.66700000000000004,"5min":0.75,"15min":0.75},"last":0.379}]},{"osd":2,"last update":"Fri Jul 21 18:41:03 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.59299999999999997,"15min":0.57699999999999996},"min":{"1min":0.41899999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.89700000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.60799999999999998,"15min":0.59599999999999997},"min":{"1min":0.47999999999999998,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.83299999999999996,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.73699999999999999}]},{"osd":3,"last update":"Fri Jul 21 18:41:03 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.55900000000000005,"15min":0.54300000000000004},"min":{"1min":0.42599999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.70499999999999996,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.59099999999999997,"15min":0.57499999999999996},"min":{"1min":0.433,"5min":0.372,"15min":0.372},"max":{"1min":0.81200000000000006,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.63100000000000001}]},{"osd":4,"last update":"Fri Jul 21 18:41:03 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53000000000000003,"15min":0.52200000000000002},"min":{"1min":0.39300000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.73499999999999999,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.51900000000000002,"15min":0.51500000000000001},"min":{"1min":0.33800000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.60999999999999999,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.40200000000000002}]},{"osd":5,"last update":"Fri Jul 21 18:41:03 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.66400000000000003,"15min":0.64600000000000002},"min":{"1min":0.51700000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.65700000000000003,"15min":0.64600000000000002},"min":{"1min":0.505,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.56200000000000006}]},{"osd":6,"last update":"Fri Jul 21 18:41:03 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66800000000000004,"15min":0.65600000000000003},"min":{"1min":0.51600000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.86899999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.67700000000000005,"15min":0.66300000000000003},"min":{"1min":0.56299999999999994,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.97399999999999998,"5min":1.087,"15min":1.087},"last":0.59999999999999998}]},{"osd":8,"last update":"Fri Jul 21 18:41:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.70799999999999996,"15min":0.69499999999999995},"min":{"1min":0.64500000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.91400000000000003,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.72899999999999998,"15min":0.71899999999999997},"min":{"1min":0.57399999999999995,"5min":0.495,"15min":0.495},"max":{"1min":0.93799999999999994,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.78900000000000003}]},{"osd":9,"last update":"Fri Jul 21 18:40:46 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.74199999999999999,"15min":0.73499999999999999},"min":{"1min":0.46500000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.98099999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.753},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.74199999999999999,"15min":0.73599999999999999},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.83499999999999996}]},{"osd":10,"last update":"Fri Jul 21 18:41:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.75800000000000001,"15min":0.76200000000000001},"min":{"1min":0.50900000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.94899999999999995,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.746,"15min":0.74399999999999999},"min":{"1min":0.63,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":0.95399999999999996,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.68999999999999995}]},{"osd":11,"last update":"Fri Jul 21 18:41:04 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.77700000000000002}]}]},{"osd":6,"up_from":43,"seq":184683593778,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27820,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717332,"statfs":{"total":95995035648,"available":95966547968,"internally_reserved":0,"allocated":888832,"data_stored":425794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:40:40 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.41299999999999998,"15min":0.40899999999999997},"min":{"1min":0.29599999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.313},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.48099999999999998,"15min":0.44500000000000001},"min":{"1min":0.311,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":4.859,"5min":4.859,"15min":4.859},"last":0.37}]},{"osd":1,"last update":"Fri Jul 21 18:40:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.44500000000000001,"5min":0.37,"15min":0.37},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.625},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.39000000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.84399999999999997,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.56599999999999995}]},{"osd":2,"last update":"Fri Jul 21 18:40:40 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.64800000000000002,"15min":0.60399999999999998},"min":{"1min":0.39900000000000002,"5min":0.376,"15min":0.376},"max":{"1min":4.9180000000000001,"5min":4.9180000000000001,"15min":4.9180000000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.59599999999999997,"15min":0.58199999999999996},"min":{"1min":0.42099999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.502}]},{"osd":3,"last update":"Fri Jul 21 18:40:40 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.52300000000000002,"15min":0.48799999999999999},"min":{"1min":0.33800000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":4.4720000000000004,"5min":4.4720000000000004,"15min":4.4720000000000004},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.56999999999999995,"15min":0.53100000000000003},"min":{"1min":0.32200000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":5.2910000000000004,"5min":5.2910000000000004,"15min":5.2910000000000004},"last":0.40799999999999997}]},{"osd":4,"last update":"Fri Jul 21 18:40:40 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.63900000000000001,"15min":0.64400000000000002},"min":{"1min":0.40300000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.022,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.64100000000000001,"15min":0.64400000000000002},"min":{"1min":0.46500000000000002,"5min":0.442,"15min":0.442},"max":{"1min":0.89600000000000002,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.61199999999999999}]},{"osd":5,"last update":"Fri Jul 21 18:40:40 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.70699999999999996,"15min":0.66600000000000004},"min":{"1min":0.47499999999999998,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":4.9960000000000004,"5min":4.9960000000000004,"15min":4.9960000000000004},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.70299999999999996,"15min":0.69499999999999995},"min":{"1min":0.49299999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.145,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.53900000000000003}]},{"osd":7,"last update":"Fri Jul 21 18:41:03 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.70199999999999996,"15min":0.70499999999999996},"min":{"1min":0.495,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.874,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.75,"15min":0.71999999999999997},"min":{"1min":0.55200000000000005,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":5.0330000000000004,"5min":5.0330000000000004,"15min":5.0330000000000004},"last":0.69699999999999995}]},{"osd":8,"last update":"Fri Jul 21 18:40:26 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.79000000000000004,"15min":0.80600000000000005},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.96099999999999997,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.77500000000000002,"15min":0.78700000000000003},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.91800000000000004,"5min":1.171,"15min":1.171},"last":0.68000000000000005}]},{"osd":9,"last update":"Fri Jul 21 18:40:44 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.75,"15min":0.75600000000000001},"min":{"1min":0.56799999999999995,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.194,"5min":1.196,"15min":1.196},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.72599999999999998,"15min":0.73399999999999999},"min":{"1min":0.52600000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.872,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.71199999999999997}]},{"osd":10,"last update":"Fri Jul 21 18:41:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.78300000000000003,"15min":0.79100000000000004},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.86499999999999999,"5min":1.216,"15min":1.216},"last":0.749},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.80500000000000005,"15min":0.77900000000000003},"min":{"1min":0.60699999999999998,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":4.9580000000000002,"5min":4.9580000000000002,"15min":4.9580000000000002},"last":0.72199999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574919,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":924,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":946176,"data_stored":483258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:41:04 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.52300000000000002,"15min":0.48899999999999999},"min":{"1min":0.377,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.77100000000000002,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.56899999999999995,"15min":0.51100000000000001},"min":{"1min":0.38500000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":3.7690000000000001,"5min":3.7690000000000001,"15min":3.7690000000000001},"last":0.58799999999999997}]},{"osd":2,"last update":"Fri Jul 21 18:41:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.59299999999999997,"15min":0.53900000000000003},"min":{"1min":0.49399999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.85099999999999998,"5min":3.2959999999999998,"15min":3.2959999999999998},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.59399999999999997,"15min":0.54000000000000004},"min":{"1min":0.38200000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.85099999999999998,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":0.79800000000000004}]},{"osd":3,"last update":"Fri Jul 21 18:40:44 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.55300000000000005,"15min":0.52500000000000002},"min":{"1min":0.38400000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.58099999999999996,"15min":0.53700000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":3.7789999999999999,"5min":3.7789999999999999,"15min":3.7789999999999999},"last":0.622}]},{"osd":4,"last update":"Fri Jul 21 18:41:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.45100000000000001,"15min":0.41199999999999998},"min":{"1min":0.32400000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.44800000000000001,"15min":0.41499999999999998},"min":{"1min":0.28000000000000003,"5min":0.254,"15min":0.254},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.55900000000000005}]},{"osd":5,"last update":"Fri Jul 21 18:41:20 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.64800000000000002,"15min":0.60199999999999998},"min":{"1min":0.56399999999999995,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.97999999999999998,"5min":1.653,"15min":1.653},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.67100000000000004,"15min":0.61399999999999999},"min":{"1min":0.65700000000000003,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.95899999999999996,"5min":1.641,"15min":1.641},"last":0.85099999999999998}]},{"osd":6,"last update":"Fri Jul 21 18:40:44 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.64400000000000002,"15min":0.63200000000000001},"min":{"1min":0.46200000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.92900000000000005,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.64500000000000002,"15min":0.64000000000000001},"min":{"1min":0.41599999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.96099999999999997,"5min":1.022,"15min":1.022},"last":0.81699999999999995}]},{"osd":7,"last update":"Fri Jul 21 18:41:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.55600000000000005,"15min":0.51400000000000001},"min":{"1min":0.33200000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.749,"5min":0.59399999999999997,"15min":0.54500000000000004},"min":{"1min":0.35999999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.94699999999999995,"5min":1.014,"15min":1.014},"last":0.71999999999999997}]},{"osd":8,"last update":"Fri Jul 21 18:40:26 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.73599999999999999,"15min":0.72799999999999998},"min":{"1min":0.51500000000000001,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.1910000000000001,"5min":2.7360000000000002,"15min":2.7360000000000002},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.72699999999999998,"15min":0.72499999999999998},"min":{"1min":0.52400000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.96899999999999997,"5min":2.7210000000000001,"15min":2.7210000000000001},"last":0.77200000000000002}]},{"osd":9,"last update":"Fri Jul 21 18:40:44 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.745,"15min":0.73899999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.753,"15min":0.746},"min":{"1min":0.55100000000000005,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.93300000000000005}]},{"osd":10,"last update":"Fri Jul 21 18:41:04 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.75700000000000001,"15min":0.749},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.875,"5min":0.78500000000000003,"15min":0.77000000000000002},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.94999999999999996}]},{"osd":11,"last update":"Fri Jul 21 18:41:05 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.91300000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738443,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27844,"kb_used_data":892,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717308,"statfs":{"total":95995035648,"available":95966523392,"internally_reserved":0,"allocated":913408,"data_stored":427082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 21 18:41:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.59499999999999997,"15min":0.57499999999999996},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.875,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.504},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58999999999999997,"15min":0.54100000000000004},"min":{"1min":0.38100000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.82699999999999996,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.54900000000000004}]},{"osd":2,"last update":"Fri Jul 21 18:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.58999999999999997,"15min":0.56999999999999995},"min":{"1min":0.439,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.90300000000000002,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.61099999999999999,"15min":0.58699999999999997},"min":{"1min":0.45400000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.79300000000000004,"5min":3.0470000000000002,"15min":3.0470000000000002},"last":0.53400000000000003}]},{"osd":3,"last update":"Fri Jul 21 18:41:07 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.32100000000000001}]},{"osd":4,"last update":"Fri Jul 21 18:41:03 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.60199999999999998,"15min":0.56999999999999995},"min":{"1min":0.35799999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.77800000000000002,"5min":2.948,"15min":2.948},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.56299999999999994,"15min":0.54600000000000004},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.51400000000000001}]},{"osd":5,"last update":"Fri Jul 21 18:41:24 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.61499999999999999,"15min":0.59099999999999997},"min":{"1min":0.41599999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.94099999999999995,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.64700000000000002,"15min":0.61299999999999999},"min":{"1min":0.434,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.1419999999999999,"5min":3.0150000000000001,"15min":3.0150000000000001},"last":0.434}]},{"osd":6,"last update":"Fri Jul 21 18:40:46 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.52200000000000002,"15min":0.51100000000000001},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":2.077,"5min":2.077,"15min":2.077},"last":0.377},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.58799999999999997,"15min":0.56899999999999995},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":2.4500000000000002,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":0.56100000000000005}]},{"osd":7,"last update":"Fri Jul 21 18:39:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.47699999999999998}]},{"osd":8,"last update":"Fri Jul 21 18:41:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.76900000000000002,"15min":0.80600000000000005},"min":{"1min":0.54600000000000004,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.96299999999999997,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.78800000000000003,"15min":0.80800000000000005},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.1679999999999999,"5min":2.9830000000000001,"15min":2.9830000000000001},"last":0.61799999999999999}]},{"osd":9,"last update":"Fri Jul 21 18:40:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.71999999999999997,"15min":0.74199999999999999},"min":{"1min":0.37,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.78800000000000003,"15min":0.81299999999999994},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.421,"5min":1.421,"15min":1.421},"last":0.63800000000000001}]},{"osd":10,"last update":"Fri Jul 21 18:41:07 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.84899999999999998,"15min":0.88100000000000001},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.95599999999999996,"5min":2.9079999999999999,"15min":2.9079999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.78400000000000003,"15min":0.80400000000000005},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.96899999999999997,"5min":1.111,"15min":1.111},"last":0.59199999999999997}]},{"osd":11,"last update":"Fri Jul 21 18:40:26 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":3.0840000000000001,"5min":3.0840000000000001,"15min":3.0840000000000001},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.63}]}]},{"osd":2,"up_from":18,"seq":77309411395,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":970752,"data_stored":483703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:41:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.54700000000000004,"15min":0.53500000000000003},"min":{"1min":0.38100000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.92200000000000004,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.51600000000000001,"15min":0.50600000000000001},"min":{"1min":0.38,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.68600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.54600000000000004}]},{"osd":1,"last update":"Fri Jul 21 18:41:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.56200000000000006,"15min":0.52800000000000002},"min":{"1min":0.42999999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.59699999999999998,"15min":0.55000000000000004},"min":{"1min":0.435,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.91800000000000004,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.83299999999999996}]},{"osd":3,"last update":"Fri Jul 21 18:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.58899999999999997,"15min":0.56499999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.92400000000000004,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.56799999999999995,"15min":0.56000000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.79600000000000004,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.63900000000000001}]},{"osd":4,"last update":"Fri Jul 21 18:41:04 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.73899999999999999}]},{"osd":5,"last update":"Fri Jul 21 18:41:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.51300000000000001,"15min":0.47099999999999997},"min":{"1min":0.36899999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.89100000000000001,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.501,"15min":0.46700000000000003},"min":{"1min":0.39800000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.85999999999999999}]},{"osd":6,"last update":"Fri Jul 21 18:40:42 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.61899999999999999,"15min":0.59599999999999997},"min":{"1min":0.53000000000000003,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.85799999999999998,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.628,"15min":0.62},"min":{"1min":0.436,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.84399999999999997,"5min":0.873,"15min":0.873},"last":0.79700000000000004}]},{"osd":7,"last update":"Fri Jul 21 18:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.61699999999999999,"15min":0.60799999999999998},"min":{"1min":0.45100000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.577,"5min":1.577,"15min":1.577},"last":0.495},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.65500000000000003,"15min":0.65100000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.878,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.56499999999999995}]},{"osd":8,"last update":"Fri Jul 21 18:40:26 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.58899999999999997,"15min":0.57099999999999995},"min":{"1min":0.40799999999999997,"5min":0.376,"15min":0.376},"max":{"1min":0.91300000000000003,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.60799999999999998,"15min":0.59799999999999998},"min":{"1min":0.40799999999999997,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.95299999999999996}]},{"osd":9,"last update":"Fri Jul 21 18:40:46 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.76000000000000001,"15min":0.75600000000000001},"min":{"1min":0.59499999999999997,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.95999999999999996,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.998},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.69199999999999995,"15min":0.68999999999999995},"min":{"1min":0.47499999999999998,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.89600000000000002,"5min":1.01,"15min":1.01},"last":0.89900000000000002}]},{"osd":10,"last update":"Fri Jul 21 18:41:10 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.78200000000000003,"15min":0.77600000000000002},"min":{"1min":0.67500000000000004,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":0.91600000000000004,"5min":1.669,"15min":1.669},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.874,"5min":0.80300000000000005,"15min":0.79100000000000004},"min":{"1min":0.75600000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.97299999999999998,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":1.056}]},{"osd":11,"last update":"Fri Jul 21 18:41:04 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":1.034},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":1.0209999999999999}]}]},{"osd":3,"up_from":24,"seq":103079215166,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27832,"kb_used_data":880,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717320,"statfs":{"total":95995035648,"available":95966535680,"internally_reserved":0,"allocated":901120,"data_stored":425024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.35699999999999998,"15min":0.33100000000000002},"min":{"1min":0.26800000000000002,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.54000000000000004,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.373,"15min":0.35099999999999998},"min":{"1min":0.28100000000000003,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.51600000000000001,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.32000000000000001}]},{"osd":1,"last update":"Fri Jul 21 18:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.57199999999999995,"15min":0.55100000000000005},"min":{"1min":0.39900000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.75800000000000001,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59199999999999997,"15min":0.55000000000000004},"min":{"1min":0.39400000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.83099999999999996,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.70499999999999996}]},{"osd":2,"last update":"Fri Jul 21 18:40:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.57699999999999996,"15min":0.55300000000000005},"min":{"1min":0.36799999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.81599999999999995,"5min":0.876,"15min":0.876},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.56799999999999995,"15min":0.53900000000000003},"min":{"1min":0.373,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.57399999999999995}]},{"osd":4,"last update":"Fri Jul 21 18:41:05 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.61299999999999999,"15min":0.60399999999999998},"min":{"1min":0.443,"5min":0.374,"15min":0.374},"max":{"1min":0.76900000000000002,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.61799999999999999,"15min":0.59999999999999998},"min":{"1min":0.42599999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.86699999999999999,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.61299999999999999}]},{"osd":5,"last update":"Fri Jul 21 18:41:20 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.621,"15min":0.58999999999999997},"min":{"1min":0.42099999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.71299999999999997,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.59799999999999998,"15min":0.58199999999999996},"min":{"1min":0.434,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.70099999999999996,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.68799999999999994}]},{"osd":6,"last update":"Fri Jul 21 18:40:41 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.498,"15min":0.48599999999999999},"min":{"1min":0.33600000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.80900000000000005,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.55800000000000005,"15min":0.55400000000000005},"min":{"1min":0.36599999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.63400000000000001,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.51800000000000002}]},{"osd":7,"last update":"Fri Jul 21 18:41:05 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.65500000000000003,"15min":0.66600000000000004},"min":{"1min":0.5,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.84299999999999997,"5min":0.999,"15min":0.999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.72499999999999998,"15min":0.72199999999999998},"min":{"1min":0.53600000000000003,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.93600000000000005,"5min":1.181,"15min":1.181},"last":0.67600000000000005}]},{"osd":8,"last update":"Fri Jul 21 18:39:25 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.47299999999999998}]},{"osd":9,"last update":"Fri Jul 21 18:40:45 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.68600000000000005,"15min":0.71499999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.77900000000000003,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.629},{"interface":"front","average":{"1min":0.624,"5min":0.81999999999999995,"15min":0.85299999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.88100000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.748}]},{"osd":10,"last update":"Fri Jul 21 18:41:08 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.71799999999999997,"15min":0.72499999999999998},"min":{"1min":0.48999999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.80500000000000005,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.84199999999999997,"15min":0.85999999999999999},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":0.96699999999999997,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.76900000000000002}]},{"osd":11,"last update":"Fri Jul 21 18:41:05 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.79600000000000004}]}]},{"osd":4,"up_from":31,"seq":133143986234,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27820,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717332,"statfs":{"total":95995035648,"available":95966547968,"internally_reserved":0,"allocated":888832,"data_stored":425721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:40:02 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.51800000000000002,"15min":0.48199999999999998},"min":{"1min":0.40500000000000003,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.94099999999999995,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.52500000000000002,"15min":0.50600000000000001},"min":{"1min":0.373,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.80600000000000005,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.58299999999999996}]},{"osd":1,"last update":"Fri Jul 21 18:41:02 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.39100000000000001,"15min":0.35199999999999998},"min":{"1min":0.28299999999999997,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.51900000000000002,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.42499999999999999,"15min":0.39100000000000001},"min":{"1min":0.312,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.57599999999999996,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.46500000000000002}]},{"osd":2,"last update":"Fri Jul 21 18:41:02 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.56299999999999994,"15min":0.53400000000000003},"min":{"1min":0.45900000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.76600000000000001,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.59899999999999998,"15min":0.54400000000000004},"min":{"1min":0.45700000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.96499999999999997}]},{"osd":3,"last update":"Fri Jul 21 18:41:02 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.57899999999999996,"15min":0.54000000000000004},"min":{"1min":0.47499999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.78200000000000003,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56399999999999995,"15min":0.55700000000000005},"min":{"1min":0.39200000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.73299999999999998,"5min":1.05,"15min":1.05},"last":0.501}]},{"osd":5,"last update":"Fri Jul 21 18:41:23 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.66100000000000003,"15min":0.61299999999999999},"min":{"1min":0.56399999999999995,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.87,"5min":1.571,"15min":1.571},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.66400000000000003,"15min":0.622},"min":{"1min":0.46200000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.89600000000000002,"5min":1.377,"15min":1.377},"last":0.88200000000000001}]},{"osd":6,"last update":"Fri Jul 21 18:40:44 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.63900000000000001,"15min":0.61699999999999999},"min":{"1min":0.498,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.83699999999999997,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.627},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.65000000000000002,"15min":0.61399999999999999},"min":{"1min":0.46100000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.81299999999999994,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.65900000000000003}]},{"osd":7,"last update":"Fri Jul 21 18:41:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.59999999999999998,"15min":0.56999999999999995},"min":{"1min":0.39600000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.93700000000000006,"5min":1.073,"15min":1.073},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.55000000000000004,"15min":0.54200000000000004},"min":{"1min":0.41499999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.81499999999999995,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.55900000000000005}]},{"osd":8,"last update":"Fri Jul 21 18:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.65900000000000003,"15min":0.66100000000000003},"min":{"1min":0.46200000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.92400000000000004,"5min":1.026,"15min":1.026},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.73099999999999998,"15min":0.72499999999999998},"min":{"1min":0.56899999999999995,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.92500000000000004}]},{"osd":9,"last update":"Fri Jul 21 18:40:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.79000000000000004,"15min":0.79900000000000004},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.93999999999999995,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.83599999999999997,"15min":0.84599999999999997},"min":{"1min":0.60499999999999998,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.95499999999999996,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.78000000000000003}]},{"osd":10,"last update":"Fri Jul 21 18:41:08 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.78500000000000003,"15min":0.78300000000000003},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.98699999999999999,"5min":1.153,"15min":1.153},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.745,"15min":0.73899999999999999},"min":{"1min":0.55900000000000005,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.97499999999999998,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.79700000000000004}]},{"osd":11,"last update":"Fri Jul 21 18:41:05 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.94099999999999995}]}]},{"osd":5,"up_from":36,"seq":154618822710,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28428,"kb_used_data":1476,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716724,"statfs":{"total":95995035648,"available":95965925376,"internally_reserved":0,"allocated":1511424,"data_stored":1017942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 18:41:24 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.52200000000000002,"15min":0.52900000000000003},"min":{"1min":0.35799999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.73099999999999998,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.503,"5min":0.53600000000000003,"15min":0.55300000000000005},"min":{"1min":0.34799999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.42799999999999999}]},{"osd":1,"last update":"Fri Jul 21 18:41:24 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.55400000000000005,"15min":0.51400000000000001},"min":{"1min":0.39200000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.76900000000000002,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.622,"5min":0.56399999999999995,"15min":0.52100000000000002},"min":{"1min":0.48599999999999999,"5min":0.375,"15min":0.375},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.60099999999999998}]},{"osd":2,"last update":"Fri Jul 21 18:41:24 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.41299999999999998,"15min":0.39200000000000002},"min":{"1min":0.27600000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.439,"5min":0.42699999999999999,"15min":0.40100000000000002},"min":{"1min":0.32000000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.69799999999999995,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.501}]},{"osd":3,"last update":"Fri Jul 21 18:41:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.56100000000000005,"15min":0.56899999999999995},"min":{"1min":0.437,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.80100000000000005,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.436,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.622}]},{"osd":4,"last update":"Fri Jul 21 18:41:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.58999999999999997,"15min":0.56000000000000005},"min":{"1min":0.46600000000000003,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.86799999999999999,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.58299999999999996,"15min":0.54700000000000004},"min":{"1min":0.49399999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81999999999999995,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.73799999999999999}]},{"osd":6,"last update":"Fri Jul 21 18:40:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.59599999999999997,"15min":0.59999999999999998},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.60399999999999998,"15min":0.59099999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.71599999999999997}]},{"osd":7,"last update":"Fri Jul 21 18:41:05 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.65900000000000003,"15min":0.66600000000000004},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.65100000000000002,"15min":0.64300000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.85999999999999999,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.77000000000000002}]},{"osd":8,"last update":"Fri Jul 21 18:41:24 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.63100000000000001,"15min":0.63300000000000001},"min":{"1min":0.35099999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.95599999999999996,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.63700000000000001,"15min":0.63500000000000001},"min":{"1min":0.41999999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.77700000000000002}]},{"osd":9,"last update":"Fri Jul 21 18:40:45 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.72499999999999998,"15min":0.72899999999999998},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.877,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.73199999999999998,"15min":0.73399999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.81499999999999995}]},{"osd":10,"last update":"Fri Jul 21 18:41:05 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.71899999999999997,"15min":0.71199999999999997},"min":{"1min":0.54600000000000004,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.73599999999999999,"15min":0.73099999999999998},"min":{"1min":0.55700000000000005,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.83799999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":58579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":59069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":59271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-21T18:41:33.149 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T18:41:33.306 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.283+0000 7fcc0cbe0700 1 -- 172.21.15.39:0/3175661521 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc080fec10 msgr2=0x7fcc080ff080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:33.307 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.283+0000 7fcc0cbe0700 1 --2- 172.21.15.39:0/3175661521 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc080fec10 0x7fcc080ff080 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0009990 tx=0x7fcbf0009ca0 comp rx=0 tx=0).stop 2023-07-21T18:41:33.307 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.283+0000 7fcc0cbe0700 1 -- 172.21.15.39:0/3175661521 shutdown_connections 2023-07-21T18:41:33.307 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.283+0000 7fcc0cbe0700 1 --2- 172.21.15.39:0/3175661521 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc08100050 0x7fcc08100460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:33.307 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.284+0000 7fcc0cbe0700 1 --2- 172.21.15.39:0/3175661521 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc080ff5c0 0x7fcc08106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:33.307 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.284+0000 7fcc0cbe0700 1 --2- 172.21.15.39:0/3175661521 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc080fec10 0x7fcc080ff080 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:33.308 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.284+0000 7fcc0cbe0700 1 -- 172.21.15.39:0/3175661521 >> 172.21.15.39:0/3175661521 conn(0x7fcc080f9770 msgr2=0x7fcc080fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:33.308 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.284+0000 7fcc0cbe0700 1 -- 172.21.15.39:0/3175661521 shutdown_connections 2023-07-21T18:41:33.308 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.284+0000 7fcc0cbe0700 1 -- 172.21.15.39:0/3175661521 wait complete. 2023-07-21T18:41:33.310 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.284+0000 7fcc0cbe0700 1 Processor -- start 2023-07-21T18:41:33.310 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.284+0000 7fcc0cbe0700 1 -- start start 2023-07-21T18:41:33.310 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.285+0000 7fcc0cbe0700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc080fec10 0x7fcc080fe670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:33.310 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.285+0000 7fcc0cbe0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc080ff5c0 0x7fcc0819a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:33.310 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.285+0000 7fcc0cbe0700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc08100050 0x7fcc0819cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:33.310 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.285+0000 7fcc0cbe0700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0819d260 con 0x7fcc080fec10 2023-07-21T18:41:33.310 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.285+0000 7fcc0cbe0700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0819d3d0 con 0x7fcc080ff5c0 2023-07-21T18:41:33.311 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.285+0000 7fcc0cbe0700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0819d590 con 0x7fcc08100050 2023-07-21T18:41:33.311 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.285+0000 7fcc0659c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc080fec10 0x7fcc080fe670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:33.311 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.285+0000 7fcc0659c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc080fec10 0x7fcc080fe670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56748/0 (socket says 172.21.15.39:56748) 2023-07-21T18:41:33.311 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.285+0000 7fcc0659c700 1 -- 172.21.15.39:0/1482430331 learned_addr learned my addr 172.21.15.39:0/1482430331 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:33.311 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.285+0000 7fcc05d9b700 1 --2- 172.21.15.39:0/1482430331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc080ff5c0 0x7fcc0819a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:33.311 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.285+0000 7fcc06d9d700 1 --2- 172.21.15.39:0/1482430331 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc08100050 0x7fcc0819cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:33.311 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.285+0000 7fcc0659c700 1 -- 172.21.15.39:0/1482430331 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc08100050 msgr2=0x7fcc0819cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:33.312 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.285+0000 7fcc0659c700 1 --2- 172.21.15.39:0/1482430331 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc08100050 0x7fcc0819cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:33.312 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.285+0000 7fcc0659c700 1 -- 172.21.15.39:0/1482430331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc080ff5c0 msgr2=0x7fcc0819a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:33.312 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.285+0000 7fcc0659c700 1 --2- 172.21.15.39:0/1482430331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc080ff5c0 0x7fcc0819a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:33.312 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.285+0000 7fcc0659c700 1 -- 172.21.15.39:0/1482430331 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbf00095f0 con 0x7fcc080fec10 2023-07-21T18:41:33.312 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.285+0000 7fcc0659c700 1 --2- 172.21.15.39:0/1482430331 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc080fec10 0x7fcc080fe670 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fcbf800d950 tx=0x7fcbf800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:33.312 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.285+0000 7fcbff7fe700 1 -- 172.21.15.39:0/1482430331 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf8009b40 con 0x7fcc080fec10 2023-07-21T18:41:33.312 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.285+0000 7fcbff7fe700 1 -- 172.21.15.39:0/1482430331 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcbf8009ca0 con 0x7fcc080fec10 2023-07-21T18:41:33.313 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.285+0000 7fcbff7fe700 1 -- 172.21.15.39:0/1482430331 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf800f620 con 0x7fcc080fec10 2023-07-21T18:41:33.313 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.285+0000 7fcc0cbe0700 1 -- 172.21.15.39:0/1482430331 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc0819d870 con 0x7fcc080fec10 2023-07-21T18:41:33.313 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.286+0000 7fcc0cbe0700 1 -- 172.21.15.39:0/1482430331 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc0819dd80 con 0x7fcc080fec10 2023-07-21T18:41:33.313 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.287+0000 7fcc0cbe0700 1 -- 172.21.15.39:0/1482430331 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc08105f20 con 0x7fcc080fec10 2023-07-21T18:41:33.313 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.288+0000 7fcbff7fe700 1 -- 172.21.15.39:0/1482430331 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fcbf8010460 con 0x7fcc080fec10 2023-07-21T18:41:33.314 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.288+0000 7fcbff7fe700 1 --2- 172.21.15.39:0/1482430331 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fcbf40a12a0 0x7fcbf40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:33.314 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.288+0000 7fcc05d9b700 1 --2- 172.21.15.39:0/1482430331 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fcbf40a12a0 0x7fcbf40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:33.314 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.288+0000 7fcbff7fe700 1 -- 172.21.15.39:0/1482430331 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fcbf80ccd70 con 0x7fcc080fec10 2023-07-21T18:41:33.314 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.289+0000 7fcc05d9b700 1 --2- 172.21.15.39:0/1482430331 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fcbf40a12a0 0x7fcbf40a3750 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0006010 tx=0x7fcbf000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:33.315 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.292+0000 7fcbff7fe700 1 -- 172.21.15.39:0/1482430331 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fcbf8096390 con 0x7fcc080fec10 2023-07-21T18:41:33.516 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.494+0000 7fcc0cbe0700 1 -- 172.21.15.39:0/1482430331 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcc0802cac0 con 0x7fcc080fec10 2023-07-21T18:41:33.518 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.495+0000 7fcbff7fe700 1 -- 172.21.15.39:0/1482430331 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+22447 (secure 0 0 0) 0x7fcbf8020090 con 0x7fcc080fec10 2023-07-21T18:41:33.519 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:41:33.519 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":81,"fsid":"eaf8e25a-27f4-11ee-9b34-001a4aab830c","created":"2023-07-21T18:33:06.504979+0000","modified":"2023-07-21T18:41:08.140284+0000","last_up_change":"2023-07-21T18:39:23.531009+0000","last_in_change":"2023-07-21T18:39:05.993024+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T18:36:13.572583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T18:40:02.726735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T18:40:03.990112+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03502c07-2d0f-4723-8d0b-d34c9f5b91bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6803","nonce":860193857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6805","nonce":860193857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6809","nonce":860193857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6807","nonce":860193857}]},"public_addr":"172.21.15.39:6803/860193857","cluster_addr":"172.21.15.39:6805/860193857","heartbeat_back_addr":"172.21.15.39:6809/860193857","heartbeat_front_addr":"172.21.15.39:6807/860193857","state":["exists","up"]},{"osd":1,"uuid":"0831a0c0-a1fa-4d30-9a28-513223aa69eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6801","nonce":3246721764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6803","nonce":3246721764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6807","nonce":3246721764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6805","nonce":3246721764}]},"public_addr":"172.21.15.155:6801/3246721764","cluster_addr":"172.21.15.155:6803/3246721764","heartbeat_back_addr":"172.21.15.155:6807/3246721764","heartbeat_front_addr":"172.21.15.155:6805/3246721764","state":["exists","up"]},{"osd":2,"uuid":"22df0003-bde9-48e4-b47a-cb8eafbab0db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6801","nonce":2563594449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6803","nonce":2563594449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6807","nonce":2563594449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6805","nonce":2563594449}]},"public_addr":"172.21.15.193:6801/2563594449","cluster_addr":"172.21.15.193:6803/2563594449","heartbeat_back_addr":"172.21.15.193:6807/2563594449","heartbeat_front_addr":"172.21.15.193:6805/2563594449","state":["exists","up"]},{"osd":3,"uuid":"54562558-a2e8-45dd-bc95-561569b9ae6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6811","nonce":863708386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6813","nonce":863708386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6817","nonce":863708386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6815","nonce":863708386}]},"public_addr":"172.21.15.39:6811/863708386","cluster_addr":"172.21.15.39:6813/863708386","heartbeat_back_addr":"172.21.15.39:6817/863708386","heartbeat_front_addr":"172.21.15.39:6815/863708386","state":["exists","up"]},{"osd":4,"uuid":"491118e7-ae1e-495c-9eac-ef1cdcae9600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6809","nonce":3361780673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6811","nonce":3361780673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6815","nonce":3361780673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6813","nonce":3361780673}]},"public_addr":"172.21.15.155:6809/3361780673","cluster_addr":"172.21.15.155:6811/3361780673","heartbeat_back_addr":"172.21.15.155:6815/3361780673","heartbeat_front_addr":"172.21.15.155:6813/3361780673","state":["exists","up"]},{"osd":5,"uuid":"d267bc2c-42b3-4d96-b2d1-b186dc7e6b18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6809","nonce":2376867721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6811","nonce":2376867721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6815","nonce":2376867721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6813","nonce":2376867721}]},"public_addr":"172.21.15.193:6809/2376867721","cluster_addr":"172.21.15.193:6811/2376867721","heartbeat_back_addr":"172.21.15.193:6815/2376867721","heartbeat_front_addr":"172.21.15.193:6813/2376867721","state":["exists","up"]},{"osd":6,"uuid":"beec8088-970a-473e-bd67-6eee7cbdef24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6819","nonce":2417770529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6821","nonce":2417770529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6825","nonce":2417770529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6823","nonce":2417770529}]},"public_addr":"172.21.15.39:6819/2417770529","cluster_addr":"172.21.15.39:6821/2417770529","heartbeat_back_addr":"172.21.15.39:6825/2417770529","heartbeat_front_addr":"172.21.15.39:6823/2417770529","state":["exists","up"]},{"osd":7,"uuid":"5a88ce30-1835-4254-90cc-804ed5cf1066","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6817","nonce":2542002111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6819","nonce":2542002111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6823","nonce":2542002111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6821","nonce":2542002111}]},"public_addr":"172.21.15.155:6817/2542002111","cluster_addr":"172.21.15.155:6819/2542002111","heartbeat_back_addr":"172.21.15.155:6823/2542002111","heartbeat_front_addr":"172.21.15.155:6821/2542002111","state":["exists","up"]},{"osd":8,"uuid":"d964a183-3021-4ace-9a0c-aad00f2570e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6817","nonce":2985835085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6819","nonce":2985835085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6823","nonce":2985835085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6821","nonce":2985835085}]},"public_addr":"172.21.15.193:6817/2985835085","cluster_addr":"172.21.15.193:6819/2985835085","heartbeat_back_addr":"172.21.15.193:6823/2985835085","heartbeat_front_addr":"172.21.15.193:6821/2985835085","state":["exists","up"]},{"osd":9,"uuid":"9725ea5c-49a3-4c99-bac4-e95069100b1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6827","nonce":1946493651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6829","nonce":1946493651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6833","nonce":1946493651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6831","nonce":1946493651}]},"public_addr":"172.21.15.39:6827/1946493651","cluster_addr":"172.21.15.39:6829/1946493651","heartbeat_back_addr":"172.21.15.39:6833/1946493651","heartbeat_front_addr":"172.21.15.39:6831/1946493651","state":["exists","up"]},{"osd":10,"uuid":"1fd60ccb-92db-45f0-a580-2003db0976e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6825","nonce":1234615022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6827","nonce":1234615022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6831","nonce":1234615022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6829","nonce":1234615022}]},"public_addr":"172.21.15.155:6825/1234615022","cluster_addr":"172.21.15.155:6827/1234615022","heartbeat_back_addr":"172.21.15.155:6831/1234615022","heartbeat_front_addr":"172.21.15.155:6829/1234615022","state":["exists","up"]},{"osd":11,"uuid":"efb31ea1-4775-4e5f-8740-728d6f4aeea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6825","nonce":215104663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6827","nonce":215104663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6831","nonce":215104663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6829","nonce":215104663}]},"public_addr":"172.21.15.193:6825/215104663","cluster_addr":"172.21.15.193:6827/215104663","heartbeat_back_addr":"172.21.15.193:6831/215104663","heartbeat_front_addr":"172.21.15.193:6829/215104663","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:35:30.211698+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:35:50.883846+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:36:11.124168+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:36:32.711905+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:36:54.442007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:37:14.843526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:37:36.587354+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:37:57.882374+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:38:18.729407+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:38:40.139231+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:39:01.290862+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:39:21.767537+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:0/2902600414":"2023-07-22T18:34:07.039823+0000","172.21.15.39:6801/2893866425":"2023-07-22T18:34:07.039823+0000","172.21.15.39:0/19717002":"2023-07-22T18:34:07.039823+0000","172.21.15.39:6800/2893866425":"2023-07-22T18:34:07.039823+0000","172.21.15.39:6800/889516631":"2023-07-22T18:33:40.852706+0000","172.21.15.39:6801/889516631":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/3636510379":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/1275748831":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/3819152602":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/1127767082":"2023-07-22T18:34:07.039823+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T18:41:33.520 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.498+0000 7fcc0cbe0700 1 -- 172.21.15.39:0/1482430331 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fcbf40a12a0 msgr2=0x7fcbf40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:33.521 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.498+0000 7fcc0cbe0700 1 --2- 172.21.15.39:0/1482430331 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fcbf40a12a0 0x7fcbf40a3750 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0006010 tx=0x7fcbf000b540 comp rx=0 tx=0).stop 2023-07-21T18:41:33.521 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.498+0000 7fcc0cbe0700 1 -- 172.21.15.39:0/1482430331 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc080fec10 msgr2=0x7fcc080fe670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:33.521 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.498+0000 7fcc0cbe0700 1 --2- 172.21.15.39:0/1482430331 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc080fec10 0x7fcc080fe670 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fcbf800d950 tx=0x7fcbf800dd10 comp rx=0 tx=0).stop 2023-07-21T18:41:33.521 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.498+0000 7fcc0cbe0700 1 -- 172.21.15.39:0/1482430331 shutdown_connections 2023-07-21T18:41:33.521 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.498+0000 7fcc0cbe0700 1 --2- 172.21.15.39:0/1482430331 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fcbf40a12a0 0x7fcbf40a3750 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:33.521 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.498+0000 7fcc0cbe0700 1 --2- 172.21.15.39:0/1482430331 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc08100050 0x7fcc0819cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:33.521 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.498+0000 7fcc0cbe0700 1 --2- 172.21.15.39:0/1482430331 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc080fec10 0x7fcc080fe670 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:33.522 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.498+0000 7fcc0cbe0700 1 --2- 172.21.15.39:0/1482430331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc080ff5c0 0x7fcc0819a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:33.522 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.498+0000 7fcc0cbe0700 1 -- 172.21.15.39:0/1482430331 >> 172.21.15.39:0/1482430331 conn(0x7fcc080f9770 msgr2=0x7fcc08103060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:33.522 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.498+0000 7fcc0cbe0700 1 -- 172.21.15.39:0/1482430331 shutdown_connections 2023-07-21T18:41:33.522 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.498+0000 7fcc0cbe0700 1 -- 172.21.15.39:0/1482430331 wait complete. 2023-07-21T18:41:33.533 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-21T18:36:13.572583+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-21T18:40:02.726735+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-21T18:40:03.990112+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-21T18:41:33.534 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-21T18:41:33.663 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.640+0000 7f7bff4ff700 1 -- 172.21.15.39:0/2534063058 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7bf81005d0 msgr2=0x7f7bf8106e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:33.663 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.640+0000 7f7bff4ff700 1 --2- 172.21.15.39:0/2534063058 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7bf81005d0 0x7f7bf8106e90 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f7bf000b0a0 tx=0x7f7bf000b3b0 comp rx=0 tx=0).stop 2023-07-21T18:41:33.663 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.640+0000 7f7bff4ff700 1 -- 172.21.15.39:0/2534063058 shutdown_connections 2023-07-21T18:41:33.663 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.640+0000 7f7bff4ff700 1 --2- 172.21.15.39:0/2534063058 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7bf81005d0 0x7f7bf8106e90 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:33.663 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.640+0000 7f7bff4ff700 1 --2- 172.21.15.39:0/2534063058 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7bf80fea50 0x7f7bf80fee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:33.664 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.640+0000 7f7bff4ff700 1 --2- 172.21.15.39:0/2534063058 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7bf80ffc40 0x7f7bf8100090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:33.664 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.640+0000 7f7bff4ff700 1 -- 172.21.15.39:0/2534063058 >> 172.21.15.39:0/2534063058 conn(0x7f7bf80fa070 msgr2=0x7f7bf80fc4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:33.664 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.640+0000 7f7bff4ff700 1 -- 172.21.15.39:0/2534063058 shutdown_connections 2023-07-21T18:41:33.664 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.640+0000 7f7bff4ff700 1 -- 172.21.15.39:0/2534063058 wait complete. 2023-07-21T18:41:33.666 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.641+0000 7f7bff4ff700 1 Processor -- start 2023-07-21T18:41:33.666 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.641+0000 7f7bff4ff700 1 -- start start 2023-07-21T18:41:33.666 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.641+0000 7f7bff4ff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7bf80fea50 0x7f7bf81958c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:33.666 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.641+0000 7f7bff4ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7bf80ffc40 0x7f7bf8195e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:33.666 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.641+0000 7f7bff4ff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7bf81005d0 0x7f7bf819ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:33.667 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.641+0000 7f7bff4ff700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf819d440 con 0x7f7bf80fea50 2023-07-21T18:41:33.667 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.641+0000 7f7bff4ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf819d580 con 0x7f7bf80ffc40 2023-07-21T18:41:33.667 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.641+0000 7f7bff4ff700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf819d710 con 0x7f7bf81005d0 2023-07-21T18:41:33.667 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.641+0000 7f7bfd29b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7bf80fea50 0x7f7bf81958c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:33.667 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.641+0000 7f7bfd29b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7bf80fea50 0x7f7bf81958c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56762/0 (socket says 172.21.15.39:56762) 2023-07-21T18:41:33.667 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.641+0000 7f7bfd29b700 1 -- 172.21.15.39:0/147191410 learned_addr learned my addr 172.21.15.39:0/147191410 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:33.667 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.641+0000 7f7bfda9c700 1 --2- 172.21.15.39:0/147191410 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7bf81005d0 0x7f7bf819ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:33.668 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.641+0000 7f7bfca9a700 1 --2- 172.21.15.39:0/147191410 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7bf80ffc40 0x7f7bf8195e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:33.668 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.641+0000 7f7bfd29b700 1 -- 172.21.15.39:0/147191410 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7bf81005d0 msgr2=0x7f7bf819ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:33.668 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.641+0000 7f7bfd29b700 1 --2- 172.21.15.39:0/147191410 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7bf81005d0 0x7f7bf819ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:33.668 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.641+0000 7f7bfd29b700 1 -- 172.21.15.39:0/147191410 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7bf80ffc40 msgr2=0x7f7bf8195e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:33.668 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.641+0000 7f7bfd29b700 1 --2- 172.21.15.39:0/147191410 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7bf80ffc40 0x7f7bf8195e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:33.668 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.641+0000 7f7bfd29b700 1 -- 172.21.15.39:0/147191410 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bf0009d00 con 0x7f7bf80fea50 2023-07-21T18:41:33.668 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.641+0000 7f7bfca9a700 1 --2- 172.21.15.39:0/147191410 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7bf80ffc40 0x7f7bf8195e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:41:33.669 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.641+0000 7f7bfd29b700 1 --2- 172.21.15.39:0/147191410 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7bf80fea50 0x7f7bf81958c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f7bf400c970 tx=0x7f7bf400cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:33.669 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.642+0000 7f7bea7fc700 1 -- 172.21.15.39:0/147191410 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf4004220 con 0x7f7bf80fea50 2023-07-21T18:41:33.669 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.642+0000 7f7bea7fc700 1 -- 172.21.15.39:0/147191410 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bf4007d60 con 0x7f7bf80fea50 2023-07-21T18:41:33.669 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.642+0000 7f7bff4ff700 1 -- 172.21.15.39:0/147191410 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bf819d9f0 con 0x7f7bf80fea50 2023-07-21T18:41:33.669 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.642+0000 7f7bea7fc700 1 -- 172.21.15.39:0/147191410 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf4003d00 con 0x7f7bf80fea50 2023-07-21T18:41:33.669 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.642+0000 7f7bff4ff700 1 -- 172.21.15.39:0/147191410 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bf819df00 con 0x7f7bf80fea50 2023-07-21T18:41:33.669 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.643+0000 7f7bff4ff700 1 -- 172.21.15.39:0/147191410 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bf8106a30 con 0x7f7bf80fea50 2023-07-21T18:41:33.670 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.644+0000 7f7bea7fc700 1 -- 172.21.15.39:0/147191410 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f7bf4004380 con 0x7f7bf80fea50 2023-07-21T18:41:33.670 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.645+0000 7f7bea7fc700 1 --2- 172.21.15.39:0/147191410 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f7be40a11e0 0x7f7be40a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:33.670 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.645+0000 7f7bfca9a700 1 --2- 172.21.15.39:0/147191410 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f7be40a11e0 0x7f7be40a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:33.670 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.645+0000 7f7bea7fc700 1 -- 172.21.15.39:0/147191410 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f7bf40ccac0 con 0x7f7bf80fea50 2023-07-21T18:41:33.671 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.646+0000 7f7bfca9a700 1 --2- 172.21.15.39:0/147191410 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f7be40a11e0 0x7f7be40a3690 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7bec005fd0 tx=0x7f7bec005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:33.671 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.648+0000 7f7bea7fc700 1 -- 172.21.15.39:0/147191410 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f7bf4096080 con 0x7f7bf80fea50 2023-07-21T18:41:33.870 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.847+0000 7f7bff4ff700 1 -- 172.21.15.39:0/147191410 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7bf802cac0 con 0x7f7bf80fea50 2023-07-21T18:41:33.871 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.848+0000 7f7bea7fc700 1 -- 172.21.15.39:0/147191410 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7bf40996a0 con 0x7f7bf80fea50 2023-07-21T18:41:33.871 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2023-07-21T18:41:33.872 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.850+0000 7f7bff4ff700 1 -- 172.21.15.39:0/147191410 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f7be40a11e0 msgr2=0x7f7be40a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:33.873 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.850+0000 7f7bff4ff700 1 --2- 172.21.15.39:0/147191410 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f7be40a11e0 0x7f7be40a3690 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7bec005fd0 tx=0x7f7bec005ee0 comp rx=0 tx=0).stop 2023-07-21T18:41:33.873 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.850+0000 7f7bff4ff700 1 -- 172.21.15.39:0/147191410 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7bf80fea50 msgr2=0x7f7bf81958c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:33.873 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.850+0000 7f7bff4ff700 1 --2- 172.21.15.39:0/147191410 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7bf80fea50 0x7f7bf81958c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f7bf400c970 tx=0x7f7bf400cd30 comp rx=0 tx=0).stop 2023-07-21T18:41:33.873 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.850+0000 7f7bff4ff700 1 -- 172.21.15.39:0/147191410 shutdown_connections 2023-07-21T18:41:33.873 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.850+0000 7f7bff4ff700 1 --2- 172.21.15.39:0/147191410 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f7be40a11e0 0x7f7be40a3690 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:33.873 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.850+0000 7f7bff4ff700 1 --2- 172.21.15.39:0/147191410 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7bf81005d0 0x7f7bf819ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:33.874 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.850+0000 7f7bff4ff700 1 --2- 172.21.15.39:0/147191410 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7bf80fea50 0x7f7bf81958c0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:33.874 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.850+0000 7f7bff4ff700 1 --2- 172.21.15.39:0/147191410 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7bf80ffc40 0x7f7bf8195e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:33.874 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.850+0000 7f7bff4ff700 1 -- 172.21.15.39:0/147191410 >> 172.21.15.39:0/147191410 conn(0x7f7bf80fa070 msgr2=0x7f7bf80fc190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:33.874 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.850+0000 7f7bff4ff700 1 -- 172.21.15.39:0/147191410 shutdown_connections 2023-07-21T18:41:33.874 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.850+0000 7f7bff4ff700 1 -- 172.21.15.39:0/147191410 wait complete. 2023-07-21T18:41:33.885 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-21T18:41:34.014 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.991+0000 7fefe9c16700 1 -- 172.21.15.39:0/3828098353 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefe4101680 msgr2=0x7fefe4107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:34.015 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.991+0000 7fefe9c16700 1 --2- 172.21.15.39:0/3828098353 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefe4101680 0x7fefe4107f40 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fefdc00b0a0 tx=0x7fefdc00b3b0 comp rx=0 tx=0).stop 2023-07-21T18:41:34.015 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.991+0000 7fefe9c16700 1 -- 172.21.15.39:0/3828098353 shutdown_connections 2023-07-21T18:41:34.015 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.991+0000 7fefe9c16700 1 --2- 172.21.15.39:0/3828098353 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fefe40ffb00 0x7fefe40fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:34.015 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.991+0000 7fefe9c16700 1 --2- 172.21.15.39:0/3828098353 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefe4101680 0x7fefe4107f40 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:34.015 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.991+0000 7fefe9c16700 1 --2- 172.21.15.39:0/3828098353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefe4100cf0 0x7fefe4101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:34.015 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.991+0000 7fefe9c16700 1 -- 172.21.15.39:0/3828098353 >> 172.21.15.39:0/3828098353 conn(0x7fefe40fb160 msgr2=0x7fefe40fd570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:34.016 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.991+0000 7fefe9c16700 1 -- 172.21.15.39:0/3828098353 shutdown_connections 2023-07-21T18:41:34.016 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.991+0000 7fefe9c16700 1 -- 172.21.15.39:0/3828098353 wait complete. 2023-07-21T18:41:34.016 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.992+0000 7fefe9c16700 1 Processor -- start 2023-07-21T18:41:34.018 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.992+0000 7fefe9c16700 1 -- start start 2023-07-21T18:41:34.018 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.992+0000 7fefe9c16700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fefe40ffb00 0x7fefe40742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:34.018 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.992+0000 7fefe9c16700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefe4100cf0 0x7fefe4072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:34.018 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.992+0000 7fefe9c16700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefe4101680 0x7fefe4072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:34.018 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.992+0000 7fefe9c16700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe40735e0 con 0x7fefe4100cf0 2023-07-21T18:41:34.018 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.992+0000 7fefe9c16700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe4073750 con 0x7fefe40ffb00 2023-07-21T18:41:34.019 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.992+0000 7fefe9c16700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe419f960 con 0x7fefe4101680 2023-07-21T18:41:34.019 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.993+0000 7fefe2ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefe4100cf0 0x7fefe4072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:34.019 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.993+0000 7fefe2ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefe4100cf0 0x7fefe4072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56778/0 (socket says 172.21.15.39:56778) 2023-07-21T18:41:34.019 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.993+0000 7fefe2ffd700 1 -- 172.21.15.39:0/3939272131 learned_addr learned my addr 172.21.15.39:0/3939272131 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:34.019 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.993+0000 7fefe37fe700 1 --2- 172.21.15.39:0/3939272131 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fefe40ffb00 0x7fefe40742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:34.019 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.993+0000 7fefe3fff700 1 --2- 172.21.15.39:0/3939272131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefe4101680 0x7fefe4072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:34.019 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.993+0000 7fefe2ffd700 1 -- 172.21.15.39:0/3939272131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefe4101680 msgr2=0x7fefe4072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:34.020 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.993+0000 7fefe2ffd700 1 --2- 172.21.15.39:0/3939272131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefe4101680 0x7fefe4072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:34.020 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.993+0000 7fefe2ffd700 1 -- 172.21.15.39:0/3939272131 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fefe40ffb00 msgr2=0x7fefe40742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:34.020 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.993+0000 7fefe2ffd700 1 --2- 172.21.15.39:0/3939272131 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fefe40ffb00 0x7fefe40742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:34.020 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.993+0000 7fefe2ffd700 1 -- 172.21.15.39:0/3939272131 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefdc009d00 con 0x7fefe4100cf0 2023-07-21T18:41:34.020 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.993+0000 7fefe2ffd700 1 --2- 172.21.15.39:0/3939272131 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefe4100cf0 0x7fefe4072900 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fefd800d950 tx=0x7fefd800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:34.020 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.993+0000 7fefe0ff9700 1 -- 172.21.15.39:0/3939272131 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd8009b40 con 0x7fefe4100cf0 2023-07-21T18:41:34.020 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.993+0000 7fefe0ff9700 1 -- 172.21.15.39:0/3939272131 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fefd8009ca0 con 0x7fefe4100cf0 2023-07-21T18:41:34.021 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.993+0000 7fefe9c16700 1 -- 172.21.15.39:0/3939272131 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefe419fc40 con 0x7fefe4100cf0 2023-07-21T18:41:34.021 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.993+0000 7fefe0ff9700 1 -- 172.21.15.39:0/3939272131 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd800f620 con 0x7fefe4100cf0 2023-07-21T18:41:34.021 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.993+0000 7fefe9c16700 1 -- 172.21.15.39:0/3939272131 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefe41a01a0 con 0x7fefe4100cf0 2023-07-21T18:41:34.021 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.994+0000 7fefe9c16700 1 -- 172.21.15.39:0/3939272131 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefe4107000 con 0x7fefe4100cf0 2023-07-21T18:41:34.022 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.996+0000 7fefe0ff9700 1 -- 172.21.15.39:0/3939272131 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fefd8010460 con 0x7fefe4100cf0 2023-07-21T18:41:34.022 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.996+0000 7fefe0ff9700 1 --2- 172.21.15.39:0/3939272131 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fefcc0a1260 0x7fefcc0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:34.022 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.996+0000 7fefe37fe700 1 --2- 172.21.15.39:0/3939272131 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fefcc0a1260 0x7fefcc0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:34.022 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.996+0000 7fefe0ff9700 1 -- 172.21.15.39:0/3939272131 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fefd80ccd50 con 0x7fefe4100cf0 2023-07-21T18:41:34.022 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.997+0000 7fefe37fe700 1 --2- 172.21.15.39:0/3939272131 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fefcc0a1260 0x7fefcc0a3710 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fefd4005950 tx=0x7fefd40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:34.023 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:33.999+0000 7fefe0ff9700 1 -- 172.21.15.39:0/3939272131 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fefd8096370 con 0x7fefe4100cf0 2023-07-21T18:41:34.222 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.199+0000 7fefe9c16700 1 -- 172.21.15.39:0/3939272131 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fefe402caa0 con 0x7fefe4100cf0 2023-07-21T18:41:34.223 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.200+0000 7fefe0ff9700 1 -- 172.21.15.39:0/3939272131 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v81) v1 ==== 104+0+11 (secure 0 0 0) 0x7fefd80200a0 con 0x7fefe4100cf0 2023-07-21T18:41:34.224 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2023-07-21T18:41:34.225 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.202+0000 7fefe9c16700 1 -- 172.21.15.39:0/3939272131 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fefcc0a1260 msgr2=0x7fefcc0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:34.225 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.202+0000 7fefe9c16700 1 --2- 172.21.15.39:0/3939272131 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fefcc0a1260 0x7fefcc0a3710 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fefd4005950 tx=0x7fefd40058e0 comp rx=0 tx=0).stop 2023-07-21T18:41:34.225 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.202+0000 7fefe9c16700 1 -- 172.21.15.39:0/3939272131 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefe4100cf0 msgr2=0x7fefe4072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:34.225 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.202+0000 7fefe9c16700 1 --2- 172.21.15.39:0/3939272131 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefe4100cf0 0x7fefe4072900 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fefd800d950 tx=0x7fefd800dd10 comp rx=0 tx=0).stop 2023-07-21T18:41:34.225 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.202+0000 7fefe9c16700 1 -- 172.21.15.39:0/3939272131 shutdown_connections 2023-07-21T18:41:34.225 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.202+0000 7fefe9c16700 1 --2- 172.21.15.39:0/3939272131 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fefcc0a1260 0x7fefcc0a3710 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:34.226 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.202+0000 7fefe9c16700 1 --2- 172.21.15.39:0/3939272131 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fefe40ffb00 0x7fefe40742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:34.226 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.202+0000 7fefe9c16700 1 --2- 172.21.15.39:0/3939272131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefe4101680 0x7fefe4072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:34.226 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.202+0000 7fefe9c16700 1 --2- 172.21.15.39:0/3939272131 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefe4100cf0 0x7fefe4072900 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:34.226 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.202+0000 7fefe9c16700 1 -- 172.21.15.39:0/3939272131 >> 172.21.15.39:0/3939272131 conn(0x7fefe40fb160 msgr2=0x7fefe4104140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:34.226 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.202+0000 7fefe9c16700 1 -- 172.21.15.39:0/3939272131 shutdown_connections 2023-07-21T18:41:34.226 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.202+0000 7fefe9c16700 1 -- 172.21.15.39:0/3939272131 wait complete. 2023-07-21T18:41:34.237 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-21T18:41:34.368 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.344+0000 7ff7e2fae700 1 -- 172.21.15.39:0/399504718 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7dc100560 msgr2=0x7ff7dc106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:34.368 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.344+0000 7ff7e2fae700 1 --2- 172.21.15.39:0/399504718 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7dc100560 0x7ff7dc106e20 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ff7d400b0a0 tx=0x7ff7d400b3b0 comp rx=0 tx=0).stop 2023-07-21T18:41:34.368 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.344+0000 7ff7e2fae700 1 -- 172.21.15.39:0/399504718 shutdown_connections 2023-07-21T18:41:34.368 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.344+0000 7ff7e2fae700 1 --2- 172.21.15.39:0/399504718 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7dc100560 0x7ff7dc106e20 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:34.368 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.344+0000 7ff7e2fae700 1 --2- 172.21.15.39:0/399504718 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7dc0fe9e0 0x7ff7dc0fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:34.369 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.344+0000 7ff7e2fae700 1 --2- 172.21.15.39:0/399504718 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7dc0ffbd0 0x7ff7dc100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:34.369 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.344+0000 7ff7e2fae700 1 -- 172.21.15.39:0/399504718 >> 172.21.15.39:0/399504718 conn(0x7ff7dc0fa000 msgr2=0x7ff7dc0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:34.369 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.344+0000 7ff7e2fae700 1 -- 172.21.15.39:0/399504718 shutdown_connections 2023-07-21T18:41:34.369 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.344+0000 7ff7e2fae700 1 -- 172.21.15.39:0/399504718 wait complete. 2023-07-21T18:41:34.369 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.344+0000 7ff7e2fae700 1 Processor -- start 2023-07-21T18:41:34.369 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.344+0000 7ff7e2fae700 1 -- start start 2023-07-21T18:41:34.369 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.344+0000 7ff7e2fae700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7dc0fe9e0 0x7ff7dc102970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:34.370 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.344+0000 7ff7e2fae700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7dc0ffbd0 0x7ff7dc100fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:34.370 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.344+0000 7ff7e2fae700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7dc100560 0x7ff7dc1014f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:34.370 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.344+0000 7ff7e2fae700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7dc101c90 con 0x7ff7dc0fe9e0 2023-07-21T18:41:34.370 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.344+0000 7ff7e2fae700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7dc101e00 con 0x7ff7dc100560 2023-07-21T18:41:34.370 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.344+0000 7ff7e2fae700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7dc19f8d0 con 0x7ff7dc0ffbd0 2023-07-21T18:41:34.370 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.345+0000 7ff7e0d4a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7dc0fe9e0 0x7ff7dc102970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:34.370 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.345+0000 7ff7e0d4a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7dc0fe9e0 0x7ff7dc102970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56790/0 (socket says 172.21.15.39:56790) 2023-07-21T18:41:34.370 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.345+0000 7ff7e0d4a700 1 -- 172.21.15.39:0/2322606878 learned_addr learned my addr 172.21.15.39:0/2322606878 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:34.371 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.345+0000 7ff7e154b700 1 --2- 172.21.15.39:0/2322606878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7dc100560 0x7ff7dc1014f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:34.371 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.345+0000 7ff7e0d4a700 1 -- 172.21.15.39:0/2322606878 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7dc0ffbd0 msgr2=0x7ff7dc100fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:34.371 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.345+0000 7ff7e0d4a700 1 --2- 172.21.15.39:0/2322606878 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7dc0ffbd0 0x7ff7dc100fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:34.371 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.345+0000 7ff7e0d4a700 1 -- 172.21.15.39:0/2322606878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7dc100560 msgr2=0x7ff7dc1014f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:34.371 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.345+0000 7ff7e0d4a700 1 --2- 172.21.15.39:0/2322606878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7dc100560 0x7ff7dc1014f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:34.371 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.345+0000 7ff7e0d4a700 1 -- 172.21.15.39:0/2322606878 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7d4009d00 con 0x7ff7dc0fe9e0 2023-07-21T18:41:34.371 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.345+0000 7ff7e154b700 1 --2- 172.21.15.39:0/2322606878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7dc100560 0x7ff7dc1014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:41:34.371 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.345+0000 7ff7e0d4a700 1 --2- 172.21.15.39:0/2322606878 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7dc0fe9e0 0x7ff7dc102970 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff7d000d980 tx=0x7ff7d000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:34.372 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.345+0000 7ff7d9ffb700 1 -- 172.21.15.39:0/2322606878 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d000de60 con 0x7ff7dc0fe9e0 2023-07-21T18:41:34.372 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.345+0000 7ff7e2fae700 1 -- 172.21.15.39:0/2322606878 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7dc19fbb0 con 0x7ff7dc0fe9e0 2023-07-21T18:41:34.372 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.345+0000 7ff7d9ffb700 1 -- 172.21.15.39:0/2322606878 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7d00044e0 con 0x7ff7dc0fe9e0 2023-07-21T18:41:34.372 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.345+0000 7ff7d9ffb700 1 -- 172.21.15.39:0/2322606878 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d0003be0 con 0x7ff7dc0fe9e0 2023-07-21T18:41:34.373 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.345+0000 7ff7e2fae700 1 -- 172.21.15.39:0/2322606878 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7dc1a0110 con 0x7ff7dc0fe9e0 2023-07-21T18:41:34.373 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.346+0000 7ff7e2fae700 1 -- 172.21.15.39:0/2322606878 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7dc10a1c0 con 0x7ff7dc0fe9e0 2023-07-21T18:41:34.373 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.347+0000 7ff7d9ffb700 1 -- 172.21.15.39:0/2322606878 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7ff7d0009bb0 con 0x7ff7dc0fe9e0 2023-07-21T18:41:34.373 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.348+0000 7ff7d9ffb700 1 --2- 172.21.15.39:0/2322606878 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7ff7cc0a11e0 0x7ff7cc0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:34.373 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.348+0000 7ff7dbfff700 1 --2- 172.21.15.39:0/2322606878 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7ff7cc0a11e0 0x7ff7cc0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:34.374 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.348+0000 7ff7d9ffb700 1 -- 172.21.15.39:0/2322606878 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8182+0+0 (secure 0 0 0) 0x7ff7d00ccbc0 con 0x7ff7dc0fe9e0 2023-07-21T18:41:34.374 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.349+0000 7ff7dbfff700 1 --2- 172.21.15.39:0/2322606878 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7ff7cc0a11e0 0x7ff7cc0a3690 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff7c8005fd0 tx=0x7ff7c8005d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:34.374 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.352+0000 7ff7d9ffb700 1 -- 172.21.15.39:0/2322606878 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7ff7d00961e0 con 0x7ff7dc0fe9e0 2023-07-21T18:41:34.438 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:34 smithi039 ceph-mon[96489]: from='client.15363 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T18:41:34.438 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:34 smithi039 ceph-mon[96489]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T18:41:34.438 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:34 smithi039 ceph-mon[96489]: from='client.15375 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T18:41:34.438 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:34 smithi039 ceph-mon[96489]: pgmap v294: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2023-07-21T18:41:34.439 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:34 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/1482430331' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T18:41:34.439 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:34 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/147191410' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T18:41:34.573 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.550+0000 7ff7e2fae700 1 -- 172.21.15.39:0/2322606878 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff7dc10a3b0 con 0x7ff7dc0fe9e0 2023-07-21T18:41:34.573 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.550+0000 7ff7d9ffb700 1 -- 172.21.15.39:0/2322606878 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v81) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff7d0009940 con 0x7ff7dc0fe9e0 2023-07-21T18:41:34.573 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 64 2023-07-21T18:41:34.575 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.552+0000 7ff7e2fae700 1 -- 172.21.15.39:0/2322606878 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7ff7cc0a11e0 msgr2=0x7ff7cc0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:34.575 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.552+0000 7ff7e2fae700 1 --2- 172.21.15.39:0/2322606878 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7ff7cc0a11e0 0x7ff7cc0a3690 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff7c8005fd0 tx=0x7ff7c8005d00 comp rx=0 tx=0).stop 2023-07-21T18:41:34.575 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.552+0000 7ff7e2fae700 1 -- 172.21.15.39:0/2322606878 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7dc0fe9e0 msgr2=0x7ff7dc102970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:34.576 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.552+0000 7ff7e2fae700 1 --2- 172.21.15.39:0/2322606878 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7dc0fe9e0 0x7ff7dc102970 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff7d000d980 tx=0x7ff7d000dd40 comp rx=0 tx=0).stop 2023-07-21T18:41:34.576 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.552+0000 7ff7e2fae700 1 -- 172.21.15.39:0/2322606878 shutdown_connections 2023-07-21T18:41:34.576 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.552+0000 7ff7e2fae700 1 --2- 172.21.15.39:0/2322606878 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7ff7cc0a11e0 0x7ff7cc0a3690 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:34.576 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.552+0000 7ff7e2fae700 1 --2- 172.21.15.39:0/2322606878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7dc100560 0x7ff7dc1014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:34.576 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.552+0000 7ff7e2fae700 1 --2- 172.21.15.39:0/2322606878 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7dc0fe9e0 0x7ff7dc102970 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:34.576 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.552+0000 7ff7e2fae700 1 --2- 172.21.15.39:0/2322606878 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7dc0ffbd0 0x7ff7dc100fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:34.576 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.552+0000 7ff7e2fae700 1 -- 172.21.15.39:0/2322606878 >> 172.21.15.39:0/2322606878 conn(0x7ff7dc0fa000 msgr2=0x7ff7dc067670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:34.577 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.553+0000 7ff7e2fae700 1 -- 172.21.15.39:0/2322606878 shutdown_connections 2023-07-21T18:41:34.577 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.553+0000 7ff7e2fae700 1 -- 172.21.15.39:0/2322606878 wait complete. 2023-07-21T18:41:34.588 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T18:41:34.620 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:34 smithi193 ceph-mon[98204]: from='client.15363 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T18:41:34.621 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:34 smithi193 ceph-mon[98204]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T18:41:34.621 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:34 smithi193 ceph-mon[98204]: from='client.15375 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T18:41:34.621 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:34 smithi193 ceph-mon[98204]: pgmap v294: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2023-07-21T18:41:34.621 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:34 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/1482430331' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T18:41:34.621 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:34 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/147191410' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T18:41:34.639 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:34 smithi155 ceph-mon[97934]: from='client.15363 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T18:41:34.639 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:34 smithi155 ceph-mon[97934]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T18:41:34.639 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:34 smithi155 ceph-mon[97934]: from='client.15375 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T18:41:34.640 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:34 smithi155 ceph-mon[97934]: pgmap v294: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2023-07-21T18:41:34.640 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:34 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/1482430331' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T18:41:34.640 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:34 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/147191410' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T18:41:34.714 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.691+0000 7f58cd4ee700 1 -- 172.21.15.39:0/447563966 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58c8105800 msgr2=0x7f58c8107c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:34.714 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.691+0000 7f58cd4ee700 1 --2- 172.21.15.39:0/447563966 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58c8105800 0x7f58c8107c00 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f58c000b0a0 tx=0x7f58c000b3b0 comp rx=0 tx=0).stop 2023-07-21T18:41:34.714 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.691+0000 7f58cd4ee700 1 -- 172.21.15.39:0/447563966 shutdown_connections 2023-07-21T18:41:34.714 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.691+0000 7f58cd4ee700 1 --2- 172.21.15.39:0/447563966 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58c8105800 0x7f58c8107c00 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:34.715 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.691+0000 7f58cd4ee700 1 --2- 172.21.15.39:0/447563966 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58c81005c0 0x7f58c81029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:34.715 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.691+0000 7f58cd4ee700 1 --2- 172.21.15.39:0/447563966 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58c8102ee0 0x7f58c81052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:34.715 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.691+0000 7f58cd4ee700 1 -- 172.21.15.39:0/447563966 >> 172.21.15.39:0/447563966 conn(0x7f58c80fa000 msgr2=0x7f58c80fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:34.715 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.691+0000 7f58cd4ee700 1 -- 172.21.15.39:0/447563966 shutdown_connections 2023-07-21T18:41:34.715 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.691+0000 7f58cd4ee700 1 -- 172.21.15.39:0/447563966 wait complete. 2023-07-21T18:41:34.717 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.692+0000 7f58cd4ee700 1 Processor -- start 2023-07-21T18:41:34.717 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.692+0000 7f58cd4ee700 1 -- start start 2023-07-21T18:41:34.717 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.692+0000 7f58cd4ee700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58c81005c0 0x7f58c80fff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:34.717 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.692+0000 7f58cd4ee700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58c8102ee0 0x7f58c80fe570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:34.717 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.692+0000 7f58cd4ee700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58c8105800 0x7f58c80feab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:34.717 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.692+0000 7f58cd4ee700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c80ff250 con 0x7f58c8102ee0 2023-07-21T18:41:34.717 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.692+0000 7f58cd4ee700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c80ff3c0 con 0x7f58c8105800 2023-07-21T18:41:34.718 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.692+0000 7f58cd4ee700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c819f910 con 0x7f58c81005c0 2023-07-21T18:41:34.718 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.692+0000 7f58c67fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58c8102ee0 0x7f58c80fe570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:34.718 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.692+0000 7f58c67fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58c8102ee0 0x7f58c80fe570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56806/0 (socket says 172.21.15.39:56806) 2023-07-21T18:41:34.718 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.692+0000 7f58c67fc700 1 -- 172.21.15.39:0/378523360 learned_addr learned my addr 172.21.15.39:0/378523360 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:34.718 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.692+0000 7f58c77fe700 1 --2- 172.21.15.39:0/378523360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58c8105800 0x7f58c80feab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:34.718 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.692+0000 7f58c6ffd700 1 --2- 172.21.15.39:0/378523360 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58c81005c0 0x7f58c80fff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:34.718 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.693+0000 7f58c67fc700 1 -- 172.21.15.39:0/378523360 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58c81005c0 msgr2=0x7f58c80fff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:34.719 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.693+0000 7f58c67fc700 1 --2- 172.21.15.39:0/378523360 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58c81005c0 0x7f58c80fff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:34.719 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.693+0000 7f58c67fc700 1 -- 172.21.15.39:0/378523360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58c8105800 msgr2=0x7f58c80feab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:34.719 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.693+0000 7f58c67fc700 1 --2- 172.21.15.39:0/378523360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58c8105800 0x7f58c80feab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:34.719 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.693+0000 7f58c67fc700 1 -- 172.21.15.39:0/378523360 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c0009d00 con 0x7f58c8102ee0 2023-07-21T18:41:34.719 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.693+0000 7f58c67fc700 1 --2- 172.21.15.39:0/378523360 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58c8102ee0 0x7f58c80fe570 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f58bc00d980 tx=0x7f58bc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:34.719 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.693+0000 7f58affff700 1 -- 172.21.15.39:0/378523360 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58bc004220 con 0x7f58c8102ee0 2023-07-21T18:41:34.719 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.693+0000 7f58cd4ee700 1 -- 172.21.15.39:0/378523360 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58c819fbf0 con 0x7f58c8102ee0 2023-07-21T18:41:34.719 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.693+0000 7f58affff700 1 -- 172.21.15.39:0/378523360 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58bc007640 con 0x7f58c8102ee0 2023-07-21T18:41:34.720 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.693+0000 7f58affff700 1 -- 172.21.15.39:0/378523360 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58bc005690 con 0x7f58c8102ee0 2023-07-21T18:41:34.720 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.693+0000 7f58cd4ee700 1 -- 172.21.15.39:0/378523360 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58c81a0100 con 0x7f58c8102ee0 2023-07-21T18:41:34.720 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.694+0000 7f58cd4ee700 1 -- 172.21.15.39:0/378523360 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58c80fbb70 con 0x7f58c8102ee0 2023-07-21T18:41:34.721 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.695+0000 7f58affff700 1 -- 172.21.15.39:0/378523360 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f58bc004750 con 0x7f58c8102ee0 2023-07-21T18:41:34.721 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.696+0000 7f58affff700 1 --2- 172.21.15.39:0/378523360 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f58b00a1220 0x7f58b00a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:34.721 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.696+0000 7f58c6ffd700 1 --2- 172.21.15.39:0/378523360 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f58b00a1220 0x7f58b00a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:34.721 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.696+0000 7f58affff700 1 -- 172.21.15.39:0/378523360 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f58bc0cc8a0 con 0x7f58c8102ee0 2023-07-21T18:41:34.721 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.696+0000 7f58c6ffd700 1 --2- 172.21.15.39:0/378523360 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f58b00a1220 0x7f58b00a36d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f58b8005950 tx=0x7f58b800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:34.723 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.700+0000 7f58affff700 1 -- 172.21.15.39:0/378523360 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f58bc095e60 con 0x7f58c8102ee0 2023-07-21T18:41:34.966 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.943+0000 7f58cd4ee700 1 -- 172.21.15.39:0/378523360 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f58c802c850 con 0x7f58c8102ee0 2023-07-21T18:41:34.966 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.944+0000 7f58affff700 1 -- 172.21.15.39:0/378523360 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10887 (secure 0 0 0) 0x7f58bc016020 con 0x7f58c8102ee0 2023-07-21T18:41:34.967 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:41:34.967 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/147318681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":147318681},{"type":"v1","addr":"172.21.15.39:6835","nonce":147318681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/1062891760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1062891760},{"type":"v1","addr":"172.21.15.39:6837","nonce":1062891760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T18:40:04.594553+0000","modified":"2023-07-21T18:40:11.989039+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24536,"mds_1":24500,"mds_2":24487,"mds_3":24464,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/3184273349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":3184273349},{"type":"v1","addr":"172.21.15.39:6839","nonce":3184273349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6841/640981220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":640981220},{"type":"v1","addr":"172.21.15.39:6841","nonce":640981220}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3106166804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3106166804},{"type":"v1","addr":"172.21.15.193:6833","nonce":3106166804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24437":{"gid":24437,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/1034755277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1034755277},{"type":"v1","addr":"172.21.15.155:6833","nonce":1034755277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/381560657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":381560657},{"type":"v1","addr":"172.21.15.193:6835","nonce":381560657}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6837/2626761843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2626761843},{"type":"v1","addr":"172.21.15.193:6837","nonce":2626761843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24464":{"gid":24464,"name":"e","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.155:6835/3854856286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3854856286},{"type":"v1","addr":"172.21.15.155:6835","nonce":3854856286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.193:6839/3148488639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3148488639},{"type":"v1","addr":"172.21.15.193:6839","nonce":3148488639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.155:6837/947670056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":947670056},{"type":"v1","addr":"172.21.15.155:6837","nonce":947670056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6839/431690342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":431690342},{"type":"v1","addr":"172.21.15.155:6839","nonce":431690342}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T18:41:34.969 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.946+0000 7f58cd4ee700 1 -- 172.21.15.39:0/378523360 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f58b00a1220 msgr2=0x7f58b00a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:34.969 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.946+0000 7f58cd4ee700 1 --2- 172.21.15.39:0/378523360 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f58b00a1220 0x7f58b00a36d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f58b8005950 tx=0x7f58b800b410 comp rx=0 tx=0).stop 2023-07-21T18:41:34.970 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.946+0000 7f58cd4ee700 1 -- 172.21.15.39:0/378523360 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58c8102ee0 msgr2=0x7f58c80fe570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:34.970 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.946+0000 7f58cd4ee700 1 --2- 172.21.15.39:0/378523360 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58c8102ee0 0x7f58c80fe570 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f58bc00d980 tx=0x7f58bc00dc90 comp rx=0 tx=0).stop 2023-07-21T18:41:34.970 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.946+0000 7f58cd4ee700 1 -- 172.21.15.39:0/378523360 shutdown_connections 2023-07-21T18:41:34.970 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.946+0000 7f58cd4ee700 1 --2- 172.21.15.39:0/378523360 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f58b00a1220 0x7f58b00a36d0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:34.970 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.946+0000 7f58cd4ee700 1 --2- 172.21.15.39:0/378523360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58c8105800 0x7f58c80feab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:34.970 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.946+0000 7f58cd4ee700 1 --2- 172.21.15.39:0/378523360 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58c81005c0 0x7f58c80fff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:34.970 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.946+0000 7f58cd4ee700 1 --2- 172.21.15.39:0/378523360 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58c8102ee0 0x7f58c80fe570 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:34.970 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.946+0000 7f58cd4ee700 1 -- 172.21.15.39:0/378523360 >> 172.21.15.39:0/378523360 conn(0x7f58c80fa000 msgr2=0x7f58c8101220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:34.971 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.946+0000 7f58cd4ee700 1 -- 172.21.15.39:0/378523360 shutdown_connections 2023-07-21T18:41:34.971 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:34.946+0000 7f58cd4ee700 1 -- 172.21.15.39:0/378523360 wait complete. 2023-07-21T18:41:34.971 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 26 2023-07-21T18:41:34.982 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T18:41:35.109 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.086+0000 7fae35657700 1 -- 172.21.15.39:0/3174455978 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae300fe9d0 msgr2=0x7fae300fede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:35.109 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.086+0000 7fae35657700 1 --2- 172.21.15.39:0/3174455978 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae300fe9d0 0x7fae300fede0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fae20009990 tx=0x7fae20009ca0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.109 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.086+0000 7fae35657700 1 -- 172.21.15.39:0/3174455978 shutdown_connections 2023-07-21T18:41:35.109 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.086+0000 7fae35657700 1 --2- 172.21.15.39:0/3174455978 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fae30100550 0x7fae30106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.109 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.086+0000 7fae35657700 1 --2- 172.21.15.39:0/3174455978 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae300fe9d0 0x7fae300fede0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.110 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.086+0000 7fae35657700 1 --2- 172.21.15.39:0/3174455978 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae300ffbc0 0x7fae30100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.110 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.086+0000 7fae35657700 1 -- 172.21.15.39:0/3174455978 >> 172.21.15.39:0/3174455978 conn(0x7fae300f9fd0 msgr2=0x7fae300fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:35.110 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.086+0000 7fae35657700 1 -- 172.21.15.39:0/3174455978 shutdown_connections 2023-07-21T18:41:35.110 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.086+0000 7fae35657700 1 -- 172.21.15.39:0/3174455978 wait complete. 2023-07-21T18:41:35.111 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.086+0000 7fae35657700 1 Processor -- start 2023-07-21T18:41:35.112 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.087+0000 7fae35657700 1 -- start start 2023-07-21T18:41:35.112 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.087+0000 7fae35657700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae300fe9d0 0x7fae30102960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:35.112 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.087+0000 7fae35657700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae300ffbc0 0x7fae30100fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:35.112 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.087+0000 7fae35657700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fae30100550 0x7fae301014e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:35.112 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.087+0000 7fae35657700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae30101c80 con 0x7fae300ffbc0 2023-07-21T18:41:35.112 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.087+0000 7fae35657700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae30101df0 con 0x7fae30100550 2023-07-21T18:41:35.113 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.087+0000 7fae35657700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3019f830 con 0x7fae300fe9d0 2023-07-21T18:41:35.113 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.087+0000 7fae2e7fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae300ffbc0 0x7fae30100fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:35.113 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.087+0000 7fae2e7fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae300ffbc0 0x7fae30100fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56818/0 (socket says 172.21.15.39:56818) 2023-07-21T18:41:35.113 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.087+0000 7fae2e7fc700 1 -- 172.21.15.39:0/3782582119 learned_addr learned my addr 172.21.15.39:0/3782582119 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:35.113 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.087+0000 7fae2f7fe700 1 --2- 172.21.15.39:0/3782582119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fae30100550 0x7fae301014e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:35.113 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.087+0000 7fae2effd700 1 --2- 172.21.15.39:0/3782582119 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae300fe9d0 0x7fae30102960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:35.113 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.087+0000 7fae2e7fc700 1 -- 172.21.15.39:0/3782582119 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae300fe9d0 msgr2=0x7fae30102960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:35.113 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.087+0000 7fae2e7fc700 1 --2- 172.21.15.39:0/3782582119 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae300fe9d0 0x7fae30102960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.114 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.087+0000 7fae2e7fc700 1 -- 172.21.15.39:0/3782582119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fae30100550 msgr2=0x7fae301014e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:35.114 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.087+0000 7fae2e7fc700 1 --2- 172.21.15.39:0/3782582119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fae30100550 0x7fae301014e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.114 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.087+0000 7fae2e7fc700 1 -- 172.21.15.39:0/3782582119 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae200095f0 con 0x7fae300ffbc0 2023-07-21T18:41:35.114 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.087+0000 7fae2f7fe700 1 --2- 172.21.15.39:0/3782582119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fae30100550 0x7fae301014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:41:35.114 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.087+0000 7fae2e7fc700 1 --2- 172.21.15.39:0/3782582119 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae300ffbc0 0x7fae30100fa0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fae2400d950 tx=0x7fae2400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:35.114 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.088+0000 7fae17fff700 1 -- 172.21.15.39:0/3782582119 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae24009940 con 0x7fae300ffbc0 2023-07-21T18:41:35.114 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.088+0000 7fae17fff700 1 -- 172.21.15.39:0/3782582119 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae24010460 con 0x7fae300ffbc0 2023-07-21T18:41:35.115 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.088+0000 7fae35657700 1 -- 172.21.15.39:0/3782582119 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae3019fb10 con 0x7fae300ffbc0 2023-07-21T18:41:35.115 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.088+0000 7fae17fff700 1 -- 172.21.15.39:0/3782582119 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2400f620 con 0x7fae300ffbc0 2023-07-21T18:41:35.115 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.088+0000 7fae35657700 1 -- 172.21.15.39:0/3782582119 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae301a0020 con 0x7fae300ffbc0 2023-07-21T18:41:35.115 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.089+0000 7fae35657700 1 -- 172.21.15.39:0/3782582119 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae3010a1a0 con 0x7fae300ffbc0 2023-07-21T18:41:35.116 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.090+0000 7fae17fff700 1 -- 172.21.15.39:0/3782582119 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fae240108d0 con 0x7fae300ffbc0 2023-07-21T18:41:35.116 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.091+0000 7fae17fff700 1 --2- 172.21.15.39:0/3782582119 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fae180a1220 0x7fae180a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:35.116 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.091+0000 7fae2effd700 1 --2- 172.21.15.39:0/3782582119 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fae180a1220 0x7fae180a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:35.116 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.091+0000 7fae17fff700 1 -- 172.21.15.39:0/3782582119 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fae240cc8f0 con 0x7fae300ffbc0 2023-07-21T18:41:35.116 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.091+0000 7fae2effd700 1 --2- 172.21.15.39:0/3782582119 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fae180a1220 0x7fae180a36d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fae2000b5c0 tx=0x7fae20005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:35.117 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.094+0000 7fae17fff700 1 -- 172.21.15.39:0/3782582119 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fae24095eb0 con 0x7fae300ffbc0 2023-07-21T18:41:35.320 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.297+0000 7fae35657700 1 -- 172.21.15.39:0/3782582119 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fae301a05d0 con 0x7fae300ffbc0 2023-07-21T18:41:35.321 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.299+0000 7fae17fff700 1 -- 172.21.15.39:0/3782582119 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+22447 (secure 0 0 0) 0x7fae24016090 con 0x7fae300ffbc0 2023-07-21T18:41:35.322 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:41:35.323 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":81,"fsid":"eaf8e25a-27f4-11ee-9b34-001a4aab830c","created":"2023-07-21T18:33:06.504979+0000","modified":"2023-07-21T18:41:08.140284+0000","last_up_change":"2023-07-21T18:39:23.531009+0000","last_in_change":"2023-07-21T18:39:05.993024+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T18:36:13.572583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T18:40:02.726735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T18:40:03.990112+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03502c07-2d0f-4723-8d0b-d34c9f5b91bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6803","nonce":860193857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6805","nonce":860193857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6809","nonce":860193857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6807","nonce":860193857}]},"public_addr":"172.21.15.39:6803/860193857","cluster_addr":"172.21.15.39:6805/860193857","heartbeat_back_addr":"172.21.15.39:6809/860193857","heartbeat_front_addr":"172.21.15.39:6807/860193857","state":["exists","up"]},{"osd":1,"uuid":"0831a0c0-a1fa-4d30-9a28-513223aa69eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6801","nonce":3246721764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6803","nonce":3246721764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6807","nonce":3246721764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6805","nonce":3246721764}]},"public_addr":"172.21.15.155:6801/3246721764","cluster_addr":"172.21.15.155:6803/3246721764","heartbeat_back_addr":"172.21.15.155:6807/3246721764","heartbeat_front_addr":"172.21.15.155:6805/3246721764","state":["exists","up"]},{"osd":2,"uuid":"22df0003-bde9-48e4-b47a-cb8eafbab0db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6801","nonce":2563594449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6803","nonce":2563594449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6807","nonce":2563594449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6805","nonce":2563594449}]},"public_addr":"172.21.15.193:6801/2563594449","cluster_addr":"172.21.15.193:6803/2563594449","heartbeat_back_addr":"172.21.15.193:6807/2563594449","heartbeat_front_addr":"172.21.15.193:6805/2563594449","state":["exists","up"]},{"osd":3,"uuid":"54562558-a2e8-45dd-bc95-561569b9ae6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6811","nonce":863708386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6813","nonce":863708386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6817","nonce":863708386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6815","nonce":863708386}]},"public_addr":"172.21.15.39:6811/863708386","cluster_addr":"172.21.15.39:6813/863708386","heartbeat_back_addr":"172.21.15.39:6817/863708386","heartbeat_front_addr":"172.21.15.39:6815/863708386","state":["exists","up"]},{"osd":4,"uuid":"491118e7-ae1e-495c-9eac-ef1cdcae9600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6809","nonce":3361780673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6811","nonce":3361780673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6815","nonce":3361780673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6813","nonce":3361780673}]},"public_addr":"172.21.15.155:6809/3361780673","cluster_addr":"172.21.15.155:6811/3361780673","heartbeat_back_addr":"172.21.15.155:6815/3361780673","heartbeat_front_addr":"172.21.15.155:6813/3361780673","state":["exists","up"]},{"osd":5,"uuid":"d267bc2c-42b3-4d96-b2d1-b186dc7e6b18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6809","nonce":2376867721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6811","nonce":2376867721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6815","nonce":2376867721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6813","nonce":2376867721}]},"public_addr":"172.21.15.193:6809/2376867721","cluster_addr":"172.21.15.193:6811/2376867721","heartbeat_back_addr":"172.21.15.193:6815/2376867721","heartbeat_front_addr":"172.21.15.193:6813/2376867721","state":["exists","up"]},{"osd":6,"uuid":"beec8088-970a-473e-bd67-6eee7cbdef24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6819","nonce":2417770529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6821","nonce":2417770529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6825","nonce":2417770529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6823","nonce":2417770529}]},"public_addr":"172.21.15.39:6819/2417770529","cluster_addr":"172.21.15.39:6821/2417770529","heartbeat_back_addr":"172.21.15.39:6825/2417770529","heartbeat_front_addr":"172.21.15.39:6823/2417770529","state":["exists","up"]},{"osd":7,"uuid":"5a88ce30-1835-4254-90cc-804ed5cf1066","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6817","nonce":2542002111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6819","nonce":2542002111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6823","nonce":2542002111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6821","nonce":2542002111}]},"public_addr":"172.21.15.155:6817/2542002111","cluster_addr":"172.21.15.155:6819/2542002111","heartbeat_back_addr":"172.21.15.155:6823/2542002111","heartbeat_front_addr":"172.21.15.155:6821/2542002111","state":["exists","up"]},{"osd":8,"uuid":"d964a183-3021-4ace-9a0c-aad00f2570e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6817","nonce":2985835085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6819","nonce":2985835085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6823","nonce":2985835085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6821","nonce":2985835085}]},"public_addr":"172.21.15.193:6817/2985835085","cluster_addr":"172.21.15.193:6819/2985835085","heartbeat_back_addr":"172.21.15.193:6823/2985835085","heartbeat_front_addr":"172.21.15.193:6821/2985835085","state":["exists","up"]},{"osd":9,"uuid":"9725ea5c-49a3-4c99-bac4-e95069100b1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6827","nonce":1946493651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6829","nonce":1946493651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6833","nonce":1946493651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6831","nonce":1946493651}]},"public_addr":"172.21.15.39:6827/1946493651","cluster_addr":"172.21.15.39:6829/1946493651","heartbeat_back_addr":"172.21.15.39:6833/1946493651","heartbeat_front_addr":"172.21.15.39:6831/1946493651","state":["exists","up"]},{"osd":10,"uuid":"1fd60ccb-92db-45f0-a580-2003db0976e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6825","nonce":1234615022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6827","nonce":1234615022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6831","nonce":1234615022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6829","nonce":1234615022}]},"public_addr":"172.21.15.155:6825/1234615022","cluster_addr":"172.21.15.155:6827/1234615022","heartbeat_back_addr":"172.21.15.155:6831/1234615022","heartbeat_front_addr":"172.21.15.155:6829/1234615022","state":["exists","up"]},{"osd":11,"uuid":"efb31ea1-4775-4e5f-8740-728d6f4aeea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6825","nonce":215104663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6827","nonce":215104663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6831","nonce":215104663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6829","nonce":215104663}]},"public_addr":"172.21.15.193:6825/215104663","cluster_addr":"172.21.15.193:6827/215104663","heartbeat_back_addr":"172.21.15.193:6831/215104663","heartbeat_front_addr":"172.21.15.193:6829/215104663","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:35:30.211698+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:35:50.883846+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:36:11.124168+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:36:32.711905+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:36:54.442007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:37:14.843526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:37:36.587354+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:37:57.882374+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:38:18.729407+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:38:40.139231+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:39:01.290862+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:39:21.767537+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:0/2902600414":"2023-07-22T18:34:07.039823+0000","172.21.15.39:6801/2893866425":"2023-07-22T18:34:07.039823+0000","172.21.15.39:0/19717002":"2023-07-22T18:34:07.039823+0000","172.21.15.39:6800/2893866425":"2023-07-22T18:34:07.039823+0000","172.21.15.39:6800/889516631":"2023-07-22T18:33:40.852706+0000","172.21.15.39:6801/889516631":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/3636510379":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/1275748831":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/3819152602":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/1127767082":"2023-07-22T18:34:07.039823+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T18:41:35.324 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.301+0000 7fae35657700 1 -- 172.21.15.39:0/3782582119 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fae180a1220 msgr2=0x7fae180a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:35.324 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.301+0000 7fae35657700 1 --2- 172.21.15.39:0/3782582119 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fae180a1220 0x7fae180a36d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fae2000b5c0 tx=0x7fae20005960 comp rx=0 tx=0).stop 2023-07-21T18:41:35.325 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.301+0000 7fae35657700 1 -- 172.21.15.39:0/3782582119 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae300ffbc0 msgr2=0x7fae30100fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:35.325 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.301+0000 7fae35657700 1 --2- 172.21.15.39:0/3782582119 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae300ffbc0 0x7fae30100fa0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fae2400d950 tx=0x7fae2400dc60 comp rx=0 tx=0).stop 2023-07-21T18:41:35.325 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.301+0000 7fae35657700 1 -- 172.21.15.39:0/3782582119 shutdown_connections 2023-07-21T18:41:35.325 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.301+0000 7fae35657700 1 --2- 172.21.15.39:0/3782582119 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fae180a1220 0x7fae180a36d0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.325 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.301+0000 7fae35657700 1 --2- 172.21.15.39:0/3782582119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fae30100550 0x7fae301014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.325 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.301+0000 7fae35657700 1 --2- 172.21.15.39:0/3782582119 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae300fe9d0 0x7fae30102960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.325 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.301+0000 7fae35657700 1 --2- 172.21.15.39:0/3782582119 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae300ffbc0 0x7fae30100fa0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.326 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.301+0000 7fae35657700 1 -- 172.21.15.39:0/3782582119 >> 172.21.15.39:0/3782582119 conn(0x7fae300f9fd0 msgr2=0x7fae300fa7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:35.326 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.301+0000 7fae35657700 1 -- 172.21.15.39:0/3782582119 shutdown_connections 2023-07-21T18:41:35.326 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.301+0000 7fae35657700 1 -- 172.21.15.39:0/3782582119 wait complete. 2023-07-21T18:41:35.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:35 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/3939272131' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T18:41:35.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:35 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/2322606878' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T18:41:35.336 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:35 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/378523360' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:41:35.338 DEBUG:tasks.fwd_scrub:Yielding 2023-07-21T18:41:35.339 INFO:teuthology.run_tasks:Running task exec... 2023-07-21T18:41:35.354 INFO:teuthology.task.exec:Executing custom commands... 2023-07-21T18:41:35.355 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi039.front.sepia.ceph.com 2023-07-21T18:41:35.355 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2023-07-21T18:41:35.357 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2023-07-21T18:41:35.357 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-21T18:41:35.357 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-21T18:41:35.486 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.463+0000 7fbafce9d700 1 -- 172.21.15.39:0/129495209 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbaf80fea00 msgr2=0x7fbaf80fee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:35.486 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.463+0000 7fbafce9d700 1 --2- 172.21.15.39:0/129495209 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbaf80fea00 0x7fbaf80fee10 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fbae8009990 tx=0x7fbae8009ca0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.486 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.463+0000 7fbafce9d700 1 -- 172.21.15.39:0/129495209 shutdown_connections 2023-07-21T18:41:35.486 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.463+0000 7fbafce9d700 1 --2- 172.21.15.39:0/129495209 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbaf8100640 0x7fbaf8106ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.487 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.463+0000 7fbafce9d700 1 --2- 172.21.15.39:0/129495209 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbaf80fea00 0x7fbaf80fee10 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.487 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.463+0000 7fbafce9d700 1 --2- 172.21.15.39:0/129495209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbaf80ffc90 0x7fbaf8100100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.487 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.463+0000 7fbafce9d700 1 -- 172.21.15.39:0/129495209 >> 172.21.15.39:0/129495209 conn(0x7fbaf80fa000 msgr2=0x7fbaf80fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:35.487 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.463+0000 7fbafce9d700 1 -- 172.21.15.39:0/129495209 shutdown_connections 2023-07-21T18:41:35.487 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.463+0000 7fbafce9d700 1 -- 172.21.15.39:0/129495209 wait complete. 2023-07-21T18:41:35.489 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.464+0000 7fbafce9d700 1 Processor -- start 2023-07-21T18:41:35.489 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.464+0000 7fbafce9d700 1 -- start start 2023-07-21T18:41:35.489 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.464+0000 7fbafce9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbaf80fea00 0x7fbaf8198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:35.489 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.464+0000 7fbafce9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbaf80ffc90 0x7fbaf819aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:35.489 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.464+0000 7fbafce9d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbaf8100640 0x7fbaf819cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:35.490 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.464+0000 7fbafce9d700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf819d5b0 con 0x7fbaf80ffc90 2023-07-21T18:41:35.490 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.464+0000 7fbafce9d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf819d6f0 con 0x7fbaf80fea00 2023-07-21T18:41:35.490 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.464+0000 7fbafce9d700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf819d8b0 con 0x7fbaf8100640 2023-07-21T18:41:35.490 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.465+0000 7fbaf5d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbaf80ffc90 0x7fbaf819aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:35.490 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.465+0000 7fbaf5d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbaf80ffc90 0x7fbaf819aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56832/0 (socket says 172.21.15.39:56832) 2023-07-21T18:41:35.490 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.465+0000 7fbaf5d9b700 1 -- 172.21.15.39:0/2880058095 learned_addr learned my addr 172.21.15.39:0/2880058095 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:35.490 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.465+0000 7fbaf659c700 1 --2- 172.21.15.39:0/2880058095 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbaf80fea00 0x7fbaf8198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:35.491 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.465+0000 7fbaf6d9d700 1 --2- 172.21.15.39:0/2880058095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbaf8100640 0x7fbaf819cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:35.491 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.465+0000 7fbaf5d9b700 1 -- 172.21.15.39:0/2880058095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbaf8100640 msgr2=0x7fbaf819cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:35.491 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.465+0000 7fbaf5d9b700 1 --2- 172.21.15.39:0/2880058095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbaf8100640 0x7fbaf819cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.491 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.465+0000 7fbaf5d9b700 1 -- 172.21.15.39:0/2880058095 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbaf80fea00 msgr2=0x7fbaf8198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:35.491 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.465+0000 7fbaf5d9b700 1 --2- 172.21.15.39:0/2880058095 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbaf80fea00 0x7fbaf8198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.491 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.465+0000 7fbaf5d9b700 1 -- 172.21.15.39:0/2880058095 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbae80095f0 con 0x7fbaf80ffc90 2023-07-21T18:41:35.491 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.465+0000 7fbaf6d9d700 1 --2- 172.21.15.39:0/2880058095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbaf8100640 0x7fbaf819cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:41:35.491 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.465+0000 7fbaf659c700 1 --2- 172.21.15.39:0/2880058095 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbaf80fea00 0x7fbaf8198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:41:35.492 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.465+0000 7fbaf5d9b700 1 --2- 172.21.15.39:0/2880058095 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbaf80ffc90 0x7fbaf819aa90 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fbaec00e9a0 tx=0x7fbaec00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:35.492 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.465+0000 7fbae77fe700 1 -- 172.21.15.39:0/2880058095 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaec009790 con 0x7fbaf80ffc90 2023-07-21T18:41:35.492 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.465+0000 7fbae77fe700 1 -- 172.21.15.39:0/2880058095 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbaec00cbd0 con 0x7fbaf80ffc90 2023-07-21T18:41:35.492 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.465+0000 7fbae77fe700 1 -- 172.21.15.39:0/2880058095 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaec018680 con 0x7fbaf80ffc90 2023-07-21T18:41:35.492 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.466+0000 7fbafce9d700 1 -- 172.21.15.39:0/2880058095 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbaf819db90 con 0x7fbaf80ffc90 2023-07-21T18:41:35.492 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.466+0000 7fbafce9d700 1 -- 172.21.15.39:0/2880058095 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaf819e020 con 0x7fbaf80ffc90 2023-07-21T18:41:35.493 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.467+0000 7f6eb9e78700 1 -- 172.21.15.39:0/2765904080 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6eb4101a80 msgr2=0x7f6eb40fdba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:35.493 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.467+0000 7f6eb9e78700 1 --2- 172.21.15.39:0/2765904080 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6eb4101a80 0x7f6eb40fdba0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f6ea8009990 tx=0x7f6ea8009ca0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.493 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.467+0000 7f6eb9e78700 1 -- 172.21.15.39:0/2765904080 shutdown_connections 2023-07-21T18:41:35.494 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.467+0000 7f6eb9e78700 1 --2- 172.21.15.39:0/2765904080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6eb40fe1c0 0x7f6eb40fe640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.494 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.467+0000 7f6eb9e78700 1 --2- 172.21.15.39:0/2765904080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6eb4101130 0x7f6eb4101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.494 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.467+0000 7f6eb9e78700 1 --2- 172.21.15.39:0/2765904080 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6eb4101a80 0x7f6eb40fdba0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.495 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.467+0000 7f6eb9e78700 1 -- 172.21.15.39:0/2765904080 >> 172.21.15.39:0/2765904080 conn(0x7f6eb40f9770 msgr2=0x7f6eb40fbbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:35.495 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.467+0000 7f6eb9e78700 1 -- 172.21.15.39:0/2765904080 shutdown_connections 2023-07-21T18:41:35.495 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.467+0000 7fbafce9d700 1 -- 172.21.15.39:0/2880058095 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbaf8105980 con 0x7fbaf80ffc90 2023-07-21T18:41:35.496 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.467+0000 7f6eb9e78700 1 -- 172.21.15.39:0/2765904080 wait complete. 2023-07-21T18:41:35.497 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.468+0000 7f6eb9e78700 1 Processor -- start 2023-07-21T18:41:35.497 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.468+0000 7fbae77fe700 1 -- 172.21.15.39:0/2880058095 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fbaec00cd40 con 0x7fbaf80ffc90 2023-07-21T18:41:35.499 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.468+0000 7f6eb9e78700 1 -- start start 2023-07-21T18:41:35.499 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.469+0000 7f6eb9e78700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6eb40fe1c0 0x7f6eb4195880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:35.499 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.469+0000 7f6eb9e78700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6eb4101130 0x7f6eb4195dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:35.499 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.469+0000 7f6eb9e78700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6eb4101a80 0x7f6eb419ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:35.500 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.469+0000 7f6eb9e78700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb419d400 con 0x7f6eb4101a80 2023-07-21T18:41:35.500 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.469+0000 7f6eb9e78700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb419d540 con 0x7f6eb40fe1c0 2023-07-21T18:41:35.500 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.469+0000 7f6eb9e78700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb419d6d0 con 0x7f6eb4101130 2023-07-21T18:41:35.501 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.469+0000 7fbae77fe700 1 --2- 172.21.15.39:0/2880058095 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fbae00a11e0 0x7fbae00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:35.502 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.469+0000 7f6eb3fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6eb4101a80 0x7f6eb419ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:35.502 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.469+0000 7f6eb3fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6eb4101a80 0x7f6eb419ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56846/0 (socket says 172.21.15.39:56846) 2023-07-21T18:41:35.502 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.469+0000 7f6eb3fff700 1 -- 172.21.15.39:0/2255600385 learned_addr learned my addr 172.21.15.39:0/2255600385 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:35.503 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.469+0000 7f6eb37fe700 1 --2- 172.21.15.39:0/2255600385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6eb40fe1c0 0x7f6eb4195880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:35.504 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.469+0000 7fbaf659c700 1 --2- 172.21.15.39:0/2880058095 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fbae00a11e0 0x7fbae00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:35.504 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.469+0000 7fbae77fe700 1 -- 172.21.15.39:0/2880058095 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fbaec014070 con 0x7fbaf80ffc90 2023-07-21T18:41:35.505 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.469+0000 7f6eb2ffd700 1 --2- 172.21.15.39:0/2255600385 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6eb4101130 0x7f6eb4195dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:35.505 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.469+0000 7f6eb3fff700 1 -- 172.21.15.39:0/2255600385 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6eb4101130 msgr2=0x7f6eb4195dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:35.506 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.469+0000 7f6eb3fff700 1 --2- 172.21.15.39:0/2255600385 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6eb4101130 0x7f6eb4195dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.506 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.469+0000 7f6eb3fff700 1 -- 172.21.15.39:0/2255600385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6eb40fe1c0 msgr2=0x7f6eb4195880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:35.506 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.469+0000 7f6eb3fff700 1 --2- 172.21.15.39:0/2255600385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6eb40fe1c0 0x7f6eb4195880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.506 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.469+0000 7f6eb3fff700 1 -- 172.21.15.39:0/2255600385 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ea80095f0 con 0x7f6eb4101a80 2023-07-21T18:41:35.507 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.469+0000 7f6eb37fe700 1 --2- 172.21.15.39:0/2255600385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6eb40fe1c0 0x7f6eb4195880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:41:35.507 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.470+0000 7fbaf659c700 1 --2- 172.21.15.39:0/2880058095 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fbae00a11e0 0x7fbae00a3690 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbae800b2e0 tx=0x7fbae8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:35.509 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.470+0000 7f6eb3fff700 1 --2- 172.21.15.39:0/2255600385 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6eb4101a80 0x7f6eb419ce30 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f6eac007ec0 tx=0x7f6eac00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:35.509 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.473+0000 7f6eb0ff9700 1 -- 172.21.15.39:0/2255600385 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eac00dd20 con 0x7f6eb4101a80 2023-07-21T18:41:35.509 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.473+0000 7f6eb0ff9700 1 -- 172.21.15.39:0/2255600385 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6eac00f040 con 0x7f6eb4101a80 2023-07-21T18:41:35.509 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.473+0000 7f6eb9e78700 1 -- 172.21.15.39:0/2255600385 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6eb419d9b0 con 0x7f6eb4101a80 2023-07-21T18:41:35.510 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.473+0000 7f6eb0ff9700 1 -- 172.21.15.39:0/2255600385 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eac012590 con 0x7f6eb4101a80 2023-07-21T18:41:35.510 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.473+0000 7fbae77fe700 1 -- 172.21.15.39:0/2880058095 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fbaec096560 con 0x7fbaf80ffc90 2023-07-21T18:41:35.519 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.473+0000 7f6eb9e78700 1 -- 172.21.15.39:0/2255600385 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6eb419df10 con 0x7f6eb4101a80 2023-07-21T18:41:35.520 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.476+0000 7f6eb0ff9700 1 -- 172.21.15.39:0/2255600385 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f6eac0133f0 con 0x7f6eb4101a80 2023-07-21T18:41:35.520 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.476+0000 7f6eb0ff9700 1 --2- 172.21.15.39:0/2255600385 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f6e9c0a1260 0x7f6e9c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:35.520 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.476+0000 7f6eb0ff9700 1 -- 172.21.15.39:0/2255600385 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f6eac0cc860 con 0x7f6eb4101a80 2023-07-21T18:41:35.521 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.476+0000 7f6eb37fe700 1 --2- 172.21.15.39:0/2255600385 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f6e9c0a1260 0x7f6e9c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:35.521 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.477+0000 7f6eb37fe700 1 --2- 172.21.15.39:0/2255600385 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f6e9c0a1260 0x7f6e9c0a3710 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4005950 tx=0x7f6ea400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:35.521 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.478+0000 7f6ec5ffeb80 1 Processor -- start 2023-07-21T18:41:35.521 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.478+0000 7f6ec5ffeb80 1 -- start start 2023-07-21T18:41:35.522 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.479+0000 7f6ec5ffeb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562401ed3960 0x562401ed3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:35.522 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.479+0000 7f6ec5ffeb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562401ed42b0 0x562401ed4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:35.522 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.479+0000 7f6ec5ffeb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562401ed5100 0x562401ed5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:35.523 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.479+0000 7f6ec5ffeb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x562401dd4d30 con 0x562401ed5100 2023-07-21T18:41:35.523 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.479+0000 7f6ec5ffeb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562401e27ac0 con 0x562401ed42b0 2023-07-21T18:41:35.523 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.479+0000 7f6ec5ffeb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x562401e20f50 con 0x562401ed3960 2023-07-21T18:41:35.523 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.479+0000 7f6eb3fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562401ed5100 0x562401ed5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:35.524 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.479+0000 7f6eb3fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562401ed5100 0x562401ed5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56856/0 (socket says 172.21.15.39:56856) 2023-07-21T18:41:35.524 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.479+0000 7f6eb3fff700 1 -- 172.21.15.39:0/1750331136 learned_addr learned my addr 172.21.15.39:0/1750331136 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:35.524 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.479+0000 7f6eb37fe700 1 --2- 172.21.15.39:0/1750331136 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562401ed3960 0x562401ed3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:35.525 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.479+0000 7f6eb2ffd700 1 --2- 172.21.15.39:0/1750331136 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562401ed42b0 0x562401ed4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:35.525 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.479+0000 7f6eb3fff700 1 -- 172.21.15.39:0/1750331136 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562401ed3960 msgr2=0x562401ed3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:35.525 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.479+0000 7f6eb3fff700 1 --2- 172.21.15.39:0/1750331136 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562401ed3960 0x562401ed3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.525 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.479+0000 7f6eb3fff700 1 -- 172.21.15.39:0/1750331136 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562401ed42b0 msgr2=0x562401ed4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:35.526 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.479+0000 7f6eb3fff700 1 --2- 172.21.15.39:0/1750331136 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562401ed42b0 0x562401ed4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.526 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.479+0000 7f6eb3fff700 1 -- 172.21.15.39:0/1750331136 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562401c4ee60 con 0x562401ed5100 2023-07-21T18:41:35.526 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.479+0000 7f6eb3fff700 1 --2- 172.21.15.39:0/1750331136 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562401ed5100 0x562401ed5570 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f6eac0a1a60 tx=0x7f6eac0957e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c330321015a922e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:35.527 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.480+0000 7f6e99ffb700 1 -- 172.21.15.39:0/1750331136 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eac09db70 con 0x562401ed5100 2023-07-21T18:41:35.527 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.480+0000 7f6e99ffb700 1 -- 172.21.15.39:0/1750331136 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6eac0a1f90 con 0x562401ed5100 2023-07-21T18:41:35.527 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.480+0000 7f6e99ffb700 1 -- 172.21.15.39:0/1750331136 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eac09a000 con 0x562401ed5100 2023-07-21T18:41:35.528 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.480+0000 7f6ec5ffeb80 1 -- 172.21.15.39:0/1750331136 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562401ed5100 msgr2=0x562401ed5570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:35.528 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.480+0000 7f6ec5ffeb80 1 --2- 172.21.15.39:0/1750331136 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562401ed5100 0x562401ed5570 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f6eac0a1a60 tx=0x7f6eac0957e0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.528 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.480+0000 7f6ec5ffeb80 1 -- 172.21.15.39:0/1750331136 shutdown_connections 2023-07-21T18:41:35.528 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.480+0000 7f6ec5ffeb80 1 --2- 172.21.15.39:0/1750331136 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562401ed5100 0x562401ed5570 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.529 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.480+0000 7f6ec5ffeb80 1 --2- 172.21.15.39:0/1750331136 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562401ed3960 0x562401ed3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.529 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.480+0000 7f6ec5ffeb80 1 --2- 172.21.15.39:0/1750331136 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562401ed42b0 0x562401ed4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.529 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.480+0000 7f6ec5ffeb80 1 -- 172.21.15.39:0/1750331136 >> 172.21.15.39:0/1750331136 conn(0x562401ec4e20 msgr2=0x562401eca250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:35.529 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.480+0000 7f6ec5ffeb80 1 -- 172.21.15.39:0/1750331136 shutdown_connections 2023-07-21T18:41:35.530 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.480+0000 7f6ec5ffeb80 1 -- 172.21.15.39:0/1750331136 wait complete. 2023-07-21T18:41:35.530 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.481+0000 7f6ec5ffeb80 1 -- start start 2023-07-21T18:41:35.530 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.481+0000 7f6ec5ffeb80 1 Processor -- start 2023-07-21T18:41:35.531 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.481+0000 7f6ec5ffeb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562401ed3960 0x562401edd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:35.531 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.481+0000 7f6ec5ffeb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562401ed42b0 0x562401eddad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:35.531 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.481+0000 7f6ec5ffeb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562401ed5100 0x562401ee4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:35.531 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.481+0000 7f6ec5ffeb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x562401dc0b40 con 0x562401ed3960 2023-07-21T18:41:35.531 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.481+0000 7f6ec5ffeb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562401dc0840 con 0x562401ed5100 2023-07-21T18:41:35.531 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.481+0000 7f6ec5ffeb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x562401ca49a0 con 0x562401ed42b0 2023-07-21T18:41:35.531 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.481+0000 7f6ec5ffeb80 10 client.0 _ll_register_callbacks cb 0x562401dbe7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:41:35.532 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.481+0000 7f6eb37fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562401ed3960 0x562401edd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:35.532 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.481+0000 7f6eb37fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562401ed3960 0x562401edd590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56858/0 (socket says 172.21.15.39:56858) 2023-07-21T18:41:35.532 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.481+0000 7f6eb37fe700 1 -- 172.21.15.39:0/611865647 learned_addr learned my addr 172.21.15.39:0/611865647 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:35.532 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.482+0000 7f6eb3fff700 1 --2- 172.21.15.39:0/611865647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562401ed5100 0x562401ee4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:35.532 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.482+0000 7f6eb37fe700 1 -- 172.21.15.39:0/611865647 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562401ed42b0 msgr2=0x562401eddad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:35.532 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.482+0000 7f6eb2ffd700 1 --2- 172.21.15.39:0/611865647 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562401ed42b0 0x562401eddad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:35.532 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.482+0000 7f6eb37fe700 1 --2- 172.21.15.39:0/611865647 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562401ed42b0 0x562401eddad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.533 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.482+0000 7f6eb37fe700 1 -- 172.21.15.39:0/611865647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562401ed5100 msgr2=0x562401ee4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:35.533 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.482+0000 7f6eb37fe700 1 --2- 172.21.15.39:0/611865647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562401ed5100 0x562401ee4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.533 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.482+0000 7f6eb37fe700 1 -- 172.21.15.39:0/611865647 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6eb419df10 con 0x562401ed3960 2023-07-21T18:41:35.533 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.482+0000 7f6eb2ffd700 1 --2- 172.21.15.39:0/611865647 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562401ed42b0 0x562401eddad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:41:35.533 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.482+0000 7f6eb3fff700 1 --2- 172.21.15.39:0/611865647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562401ed5100 0x562401ee4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:41:35.533 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.482+0000 7f6eb37fe700 1 --2- 172.21.15.39:0/611865647 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562401ed3960 0x562401edd590 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f6ea401de30 tx=0x7f6ea40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbdddfcd17ac4bec server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:35.533 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.482+0000 7f6e92ffd700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:41:35.533 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.482+0000 7f6e92ffd700 1 -- 172.21.15.39:0/611865647 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea4010cd0 con 0x562401ed3960 2023-07-21T18:41:35.534 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.482+0000 7f6e92ffd700 1 -- 172.21.15.39:0/611865647 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f6ea4010e30 con 0x562401ed3960 2023-07-21T18:41:35.534 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.482+0000 7f6ec5ffeb80 1 -- 172.21.15.39:0/611865647 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562401d7a570 con 0x562401ed3960 2023-07-21T18:41:35.534 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.482+0000 7f6e92ffd700 1 -- 172.21.15.39:0/611865647 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ea40273f0 con 0x562401ed3960 2023-07-21T18:41:35.534 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.482+0000 7f6ec5ffeb80 10 client.15429 fetch_fsmap learned FSMap version 26 2023-07-21T18:41:35.534 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.482+0000 7f6ec5ffeb80 1 -- 172.21.15.39:0/611865647 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x562401c37ce0 con 0x562401ed3960 2023-07-21T18:41:35.534 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.482+0000 7f6e92ffd700 1 -- 172.21.15.39:0/611865647 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562401d7a570 con 0x562401ed3960 2023-07-21T18:41:35.534 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.483+0000 7f6e92ffd700 1 -- 172.21.15.39:0/611865647 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6ea40337e0 con 0x562401ed3960 2023-07-21T18:41:35.534 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.483+0000 7f6ec5ffeb80 10 client.15429 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T18:41:35.535 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.483+0000 7f6ec5ffeb80 10 client.15429 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T18:41:35.535 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.483+0000 7f6ec5ffeb80 20 client.15429 populate_metadata read hostname 'smithi039' 2023-07-21T18:41:35.535 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.483+0000 7f6ec5ffeb80 1 --2- 172.21.15.39:0/611865647 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x562401ee93c0 0x562401eeb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:35.535 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.483+0000 7f6ec5ffeb80 4 client.15429 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:41:35.535 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.483+0000 7f6ec5ffeb80 1 -- 172.21.15.39:0/611865647 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562401e19f30 con 0x562401ee93c0 2023-07-21T18:41:35.535 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.483+0000 7f6eb2ffd700 1 --2- 172.21.15.39:0/611865647 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x562401ee93c0 0x562401eeb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:35.535 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.484+0000 7f6eb2ffd700 1 --2- 172.21.15.39:0/611865647 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x562401ee93c0 0x562401eeb7d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f2f9936df29d08a server_cookie=7579fd3f61dc739a in_seq=0 out_seq=0 2023-07-21T18:41:35.535 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.484+0000 7f6e92ffd700 10 client.15429 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:41:35.536 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.484+0000 7f6e92ffd700 1 -- 172.21.15.39:0/611865647 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562401e19f30 con 0x562401ee93c0 2023-07-21T18:41:35.536 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.484+0000 7f6e92ffd700 10 client.15429 handle_command_reply: tid=0 2023-07-21T18:41:35.536 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.484+0000 7f6ec5ffeb80 1 client.15429 shutdown 2023-07-21T18:41:35.536 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.484+0000 7f6ec5ffeb80 1 -- 172.21.15.39:0/611865647 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562401ed3960 msgr2=0x562401edd590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:35.536 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.484+0000 7f6ec5ffeb80 1 --2- 172.21.15.39:0/611865647 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562401ed3960 0x562401edd590 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f6ea401de30 tx=0x7f6ea40047c0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.536 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.485+0000 7f6ec5ffeb80 1 -- 172.21.15.39:0/611865647 shutdown_connections 2023-07-21T18:41:35.537 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.485+0000 7f6ec5ffeb80 1 --2- 172.21.15.39:0/611865647 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x562401ee93c0 0x562401eeb7d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.537 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.485+0000 7f6ec5ffeb80 1 --2- 172.21.15.39:0/611865647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562401ed5100 0x562401ee4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.537 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.485+0000 7f6e92ffd700 0 client.15429 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:41:35.537 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.485+0000 7f6ec5ffeb80 1 --2- 172.21.15.39:0/611865647 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562401ed3960 0x562401edd590 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.537 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.485+0000 7f6ec5ffeb80 1 --2- 172.21.15.39:0/611865647 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562401ed42b0 0x562401eddad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.537 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.485+0000 7f6ec5ffeb80 1 -- 172.21.15.39:0/611865647 >> 172.21.15.39:0/611865647 conn(0x562401ec4e20 msgr2=0x562401ec9930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:35.538 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.485+0000 7f6ec5ffeb80 1 -- 172.21.15.39:0/611865647 shutdown_connections 2023-07-21T18:41:35.538 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.485+0000 7f6ec5ffeb80 1 -- 172.21.15.39:0/611865647 wait complete. 2023-07-21T18:41:35.538 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.485+0000 7f6ec5ffeb80 20 client.15429 trim_cache size 0 max 16384 2023-07-21T18:41:35.542 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.514+0000 7f6ec5ffeb80 1 Processor -- start 2023-07-21T18:41:35.542 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.514+0000 7f6ec5ffeb80 1 -- start start 2023-07-21T18:41:35.542 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.514+0000 7f6ec5ffeb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562401ed3960 0x562401edc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:35.542 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.514+0000 7f6ec5ffeb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562401ed42b0 0x562401ee4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:35.542 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.514+0000 7f6ec5ffeb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562401ed5100 0x562401edb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:35.542 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.514+0000 7f6ec5ffeb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x562401ed4a00 con 0x562401ed42b0 2023-07-21T18:41:35.543 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.514+0000 7f6ec5ffeb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562401edd100 con 0x562401ed3960 2023-07-21T18:41:35.543 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.514+0000 7f6ec5ffeb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x562401ecbd80 con 0x562401ed5100 2023-07-21T18:41:35.543 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.514+0000 7f6eb2ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562401ed42b0 0x562401ee4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:35.543 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.514+0000 7f6eb2ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562401ed42b0 0x562401ee4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56866/0 (socket says 172.21.15.39:56866) 2023-07-21T18:41:35.543 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.514+0000 7f6eb2ffd700 1 -- 172.21.15.39:0/3348270426 learned_addr learned my addr 172.21.15.39:0/3348270426 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:35.543 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.514+0000 7f6eb3fff700 1 --2- 172.21.15.39:0/3348270426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562401ed5100 0x562401edb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:35.544 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.514+0000 7f6eb37fe700 1 --2- 172.21.15.39:0/3348270426 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562401ed3960 0x562401edc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:35.544 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.514+0000 7f6eb2ffd700 1 -- 172.21.15.39:0/3348270426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562401ed5100 msgr2=0x562401edb4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:35.544 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.514+0000 7f6eb2ffd700 1 --2- 172.21.15.39:0/3348270426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562401ed5100 0x562401edb4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.544 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.514+0000 7f6eb2ffd700 1 -- 172.21.15.39:0/3348270426 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562401ed3960 msgr2=0x562401edc2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:35.544 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.514+0000 7f6eb2ffd700 1 --2- 172.21.15.39:0/3348270426 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562401ed3960 0x562401edc2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.544 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.514+0000 7f6eb2ffd700 1 -- 172.21.15.39:0/3348270426 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562401c752d0 con 0x562401ed42b0 2023-07-21T18:41:35.544 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.514+0000 7f6eb2ffd700 1 --2- 172.21.15.39:0/3348270426 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562401ed42b0 0x562401ee4ad0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f6ea800b5c0 tx=0x7f6ea8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=266e4118872f054d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:35.545 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.515+0000 7f6e9a7fc700 1 -- 172.21.15.39:0/3348270426 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea801d040 con 0x562401ed42b0 2023-07-21T18:41:35.545 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.515+0000 7f6e9a7fc700 1 -- 172.21.15.39:0/3348270426 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ea801e040 con 0x562401ed42b0 2023-07-21T18:41:35.545 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.515+0000 7f6e9a7fc700 1 -- 172.21.15.39:0/3348270426 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea8003cb0 con 0x562401ed42b0 2023-07-21T18:41:35.545 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.515+0000 7f6ec5ffeb80 1 -- 172.21.15.39:0/3348270426 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562401ed42b0 msgr2=0x562401ee4ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:35.545 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.515+0000 7f6ec5ffeb80 1 --2- 172.21.15.39:0/3348270426 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562401ed42b0 0x562401ee4ad0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f6ea800b5c0 tx=0x7f6ea8005fd0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.545 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.515+0000 7f6ec5ffeb80 1 -- 172.21.15.39:0/3348270426 shutdown_connections 2023-07-21T18:41:35.545 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.515+0000 7f6ec5ffeb80 1 --2- 172.21.15.39:0/3348270426 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562401ed3960 0x562401edc2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.545 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.515+0000 7f6ec5ffeb80 1 --2- 172.21.15.39:0/3348270426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562401ed5100 0x562401edb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.546 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.515+0000 7f6ec5ffeb80 1 --2- 172.21.15.39:0/3348270426 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562401ed42b0 0x562401ee4ad0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.546 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.515+0000 7f6ec5ffeb80 1 -- 172.21.15.39:0/3348270426 >> 172.21.15.39:0/3348270426 conn(0x562401ec4e20 msgr2=0x562401ee5400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:35.546 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.515+0000 7f6ec5ffeb80 1 -- 172.21.15.39:0/3348270426 shutdown_connections 2023-07-21T18:41:35.546 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.516+0000 7f6ec5ffeb80 1 -- 172.21.15.39:0/3348270426 wait complete. 2023-07-21T18:41:35.546 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.516+0000 7f6ec5ffeb80 1 -- start start 2023-07-21T18:41:35.546 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.516+0000 7f6ec5ffeb80 1 Processor -- start 2023-07-21T18:41:35.546 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.516+0000 7f6ec5ffeb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562401ed3960 0x562401ee1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:35.547 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.516+0000 7f6ec5ffeb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562401ed42b0 0x562401ee1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:35.547 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.516+0000 7f6ec5ffeb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562401ed5100 0x562401f3d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:35.547 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.516+0000 7f6ec5ffeb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x562401f3d7b0 con 0x562401ed5100 2023-07-21T18:41:35.547 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.516+0000 7f6ec5ffeb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562401f3d8f0 con 0x562401ed3960 2023-07-21T18:41:35.547 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.516+0000 7f6ec5ffeb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x562401f3da30 con 0x562401ed42b0 2023-07-21T18:41:35.547 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.516+0000 7f6ec5ffeb80 10 client.0 _ll_register_callbacks cb 0x7f6ea401d7d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:41:35.547 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.516+0000 7f6eb3fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562401ed5100 0x562401f3d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:35.548 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.516+0000 7f6eb3fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562401ed5100 0x562401f3d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56876/0 (socket says 172.21.15.39:56876) 2023-07-21T18:41:35.548 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.517+0000 7f6eb3fff700 1 -- 172.21.15.39:0/1599003237 learned_addr learned my addr 172.21.15.39:0/1599003237 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:35.548 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.517+0000 7f6eb37fe700 1 --2- 172.21.15.39:0/1599003237 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562401ed3960 0x562401ee1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:35.548 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.517+0000 7f6eb2ffd700 1 --2- 172.21.15.39:0/1599003237 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562401ed42b0 0x562401ee1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:35.548 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.517+0000 7f6eb3fff700 1 -- 172.21.15.39:0/1599003237 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562401ed42b0 msgr2=0x562401ee1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:35.548 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.517+0000 7f6eb3fff700 1 --2- 172.21.15.39:0/1599003237 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562401ed42b0 0x562401ee1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.548 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.517+0000 7f6eb3fff700 1 -- 172.21.15.39:0/1599003237 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562401ed3960 msgr2=0x562401ee1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:35.549 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.517+0000 7f6eb3fff700 1 --2- 172.21.15.39:0/1599003237 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562401ed3960 0x562401ee1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.549 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.517+0000 7f6eb3fff700 1 -- 172.21.15.39:0/1599003237 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6eb4108840 con 0x562401ed5100 2023-07-21T18:41:35.549 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.517+0000 7f6eb2ffd700 1 --2- 172.21.15.39:0/1599003237 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562401ed42b0 0x562401ee1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:41:35.549 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.517+0000 7f6eb3fff700 1 --2- 172.21.15.39:0/1599003237 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562401ed5100 0x562401f3d270 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f6eac096040 tx=0x7f6eac009000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=488d7e16606d4706 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:35.549 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.517+0000 7f6e92ffd700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:41:35.549 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.517+0000 7f6e92ffd700 1 -- 172.21.15.39:0/1599003237 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eac09db70 con 0x562401ed5100 2023-07-21T18:41:35.549 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.517+0000 7f6e92ffd700 1 -- 172.21.15.39:0/1599003237 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f6eac0b2240 con 0x562401ed5100 2023-07-21T18:41:35.549 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.517+0000 7f6ec5ffeb80 1 -- 172.21.15.39:0/1599003237 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6eac0a1f90 con 0x562401ed5100 2023-07-21T18:41:35.550 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.517+0000 7f6e92ffd700 1 -- 172.21.15.39:0/1599003237 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6eac0ca490 con 0x562401ed5100 2023-07-21T18:41:35.550 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.517+0000 7f6ec5ffeb80 10 client.15435 fetch_fsmap learned FSMap version 26 2023-07-21T18:41:35.550 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.517+0000 7f6ec5ffeb80 1 -- 172.21.15.39:0/1599003237 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x562401c4d330 con 0x562401ed5100 2023-07-21T18:41:35.550 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.518+0000 7f6e92ffd700 1 -- 172.21.15.39:0/1599003237 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6eac0a1f90 con 0x562401ed5100 2023-07-21T18:41:35.550 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.518+0000 7f6e92ffd700 1 -- 172.21.15.39:0/1599003237 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6eac004bd0 con 0x562401ed5100 2023-07-21T18:41:35.550 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.518+0000 7f6ec5ffeb80 10 client.15435 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T18:41:35.550 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.518+0000 7f6ec5ffeb80 10 client.15435 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T18:41:35.551 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.518+0000 7f6ec5ffeb80 20 client.15435 populate_metadata read hostname 'smithi039' 2023-07-21T18:41:35.551 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.518+0000 7f6ec5ffeb80 1 --2- 172.21.15.39:0/1599003237 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x562401f3f190 0x562401f415a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:35.551 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.518+0000 7f6ec5ffeb80 4 client.15435 mds_command: new command op to 24536 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-21T18:41:35.551 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.518+0000 7f6ec5ffeb80 1 -- 172.21.15.39:0/1599003237 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x562401ddc0d0 con 0x562401f3f190 2023-07-21T18:41:35.551 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.518+0000 7f6eb37fe700 1 --2- 172.21.15.39:0/1599003237 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x562401f3f190 0x562401f415a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:35.551 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.518+0000 7f6e92ffd700 10 client.15435 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:41:35.551 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.518+0000 7f6eb37fe700 1 --2- 172.21.15.39:0/1599003237 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x562401f3f190 0x562401f415a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f69177d236612967 server_cookie=ad8215ea38d3d2ce in_seq=0 out_seq=0 2023-07-21T18:41:35.552 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.520+0000 7f6e92ffd700 1 -- 172.21.15.39:0/1599003237 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x562401ddc0d0 con 0x562401f3f190 2023-07-21T18:41:35.553 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.520+0000 7f6e92ffd700 10 client.15435 handle_command_reply: tid=0 2023-07-21T18:41:35.553 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.520+0000 7f6ec5ffeb80 1 client.15435 shutdown 2023-07-21T18:41:35.553 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.520+0000 7f6ec5ffeb80 1 -- 172.21.15.39:0/1599003237 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562401ed5100 msgr2=0x562401f3d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:35.553 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.520+0000 7f6ec5ffeb80 1 --2- 172.21.15.39:0/1599003237 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562401ed5100 0x562401f3d270 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f6eac096040 tx=0x7f6eac009000 comp rx=0 tx=0).stop 2023-07-21T18:41:35.553 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.520+0000 7f6ec5ffeb80 1 -- 172.21.15.39:0/1599003237 shutdown_connections 2023-07-21T18:41:35.553 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.520+0000 7f6ec5ffeb80 1 --2- 172.21.15.39:0/1599003237 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x562401f3f190 0x562401f415a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.553 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.520+0000 7f6ec5ffeb80 1 --2- 172.21.15.39:0/1599003237 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562401ed3960 0x562401ee1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.553 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.520+0000 7f6ec5ffeb80 1 --2- 172.21.15.39:0/1599003237 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562401ed5100 0x562401f3d270 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.554 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.520+0000 7f6ec5ffeb80 1 --2- 172.21.15.39:0/1599003237 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562401ed42b0 0x562401ee1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.554 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.520+0000 7f6ec5ffeb80 1 -- 172.21.15.39:0/1599003237 >> 172.21.15.39:0/1599003237 conn(0x562401ec4e20 msgr2=0x562401edc2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:35.554 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.520+0000 7f6e92ffd700 0 client.15435 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:41:35.554 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.520+0000 7f6ec5ffeb80 1 -- 172.21.15.39:0/1599003237 shutdown_connections 2023-07-21T18:41:35.554 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.520+0000 7f6ec5ffeb80 1 -- 172.21.15.39:0/1599003237 wait complete. 2023-07-21T18:41:35.554 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.520+0000 7f6ec5ffeb80 20 client.15435 trim_cache size 0 max 16384 2023-07-21T18:41:35.555 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2023-07-21T18:41:35.555 INFO:teuthology.orchestra.run.smithi039.stdout: "return_code": 0, 2023-07-21T18:41:35.555 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_tag": "ddc2b241-385e-45b5-b13a-84178d7135cb", 2023-07-21T18:41:35.555 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": "asynchronous" 2023-07-21T18:41:35.555 INFO:teuthology.orchestra.run.smithi039.stdout:} 2023-07-21T18:41:35.557 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.521+0000 7f6eb9e78700 1 -- 172.21.15.39:0/2255600385 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f6e9c0a1260 msgr2=0x7f6e9c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:35.558 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.521+0000 7f6eb9e78700 1 --2- 172.21.15.39:0/2255600385 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f6e9c0a1260 0x7f6e9c0a3710 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4005950 tx=0x7f6ea400a300 comp rx=0 tx=0).stop 2023-07-21T18:41:35.558 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.521+0000 7f6eb9e78700 1 -- 172.21.15.39:0/2255600385 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6eb4101a80 msgr2=0x7f6eb419ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:35.558 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.521+0000 7f6eb9e78700 1 --2- 172.21.15.39:0/2255600385 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6eb4101a80 0x7f6eb419ce30 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f6eac007ec0 tx=0x7f6eac00d3b0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.558 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.521+0000 7f6eb9e78700 1 -- 172.21.15.39:0/2255600385 shutdown_connections 2023-07-21T18:41:35.559 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.521+0000 7f6eb9e78700 1 --2- 172.21.15.39:0/2255600385 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f6e9c0a1260 0x7f6e9c0a3710 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.559 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.521+0000 7f6eb9e78700 1 --2- 172.21.15.39:0/2255600385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6eb40fe1c0 0x7f6eb4195880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.559 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.521+0000 7f6eb9e78700 1 --2- 172.21.15.39:0/2255600385 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6eb4101a80 0x7f6eb419ce30 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.560 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.521+0000 7f6eb9e78700 1 --2- 172.21.15.39:0/2255600385 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6eb4101130 0x7f6eb4195dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.560 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.521+0000 7f6eb9e78700 1 -- 172.21.15.39:0/2255600385 >> 172.21.15.39:0/2255600385 conn(0x7f6eb40f9770 msgr2=0x7f6eb40fb3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:35.560 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.521+0000 7f6eb9e78700 1 -- 172.21.15.39:0/2255600385 shutdown_connections 2023-07-21T18:41:35.560 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.521+0000 7f6eb9e78700 1 -- 172.21.15.39:0/2255600385 wait complete. 2023-07-21T18:41:35.561 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T18:41:35.620 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:35 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/3939272131' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T18:41:35.621 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:35 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/2322606878' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T18:41:35.621 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:35 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/378523360' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:41:35.639 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:35 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/3939272131' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T18:41:35.639 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:35 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/2322606878' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T18:41:35.640 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:35 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/378523360' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:41:35.691 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.668+0000 7f313aac4700 1 -- 172.21.15.39:0/4020742047 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3134105800 msgr2=0x7f3134107c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:35.691 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.668+0000 7f313aac4700 1 --2- 172.21.15.39:0/4020742047 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3134105800 0x7f3134107c00 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f312c00b0a0 tx=0x7f312c00b3b0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.691 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.668+0000 7f313aac4700 1 -- 172.21.15.39:0/4020742047 shutdown_connections 2023-07-21T18:41:35.692 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.668+0000 7f313aac4700 1 --2- 172.21.15.39:0/4020742047 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3134105800 0x7f3134107c00 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.692 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.668+0000 7f313aac4700 1 --2- 172.21.15.39:0/4020742047 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31341005c0 0x7f31341029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.692 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.668+0000 7f313aac4700 1 --2- 172.21.15.39:0/4020742047 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3134102ee0 0x7f31341052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.692 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.668+0000 7f313aac4700 1 -- 172.21.15.39:0/4020742047 >> 172.21.15.39:0/4020742047 conn(0x7f31340fa000 msgr2=0x7f31340fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:35.692 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.668+0000 7f313aac4700 1 -- 172.21.15.39:0/4020742047 shutdown_connections 2023-07-21T18:41:35.692 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.668+0000 7f313aac4700 1 -- 172.21.15.39:0/4020742047 wait complete. 2023-07-21T18:41:35.692 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.669+0000 7f313aac4700 1 Processor -- start 2023-07-21T18:41:35.693 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.669+0000 7f313aac4700 1 -- start start 2023-07-21T18:41:35.694 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.669+0000 7f313aac4700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31341005c0 0x7f3134195930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:35.694 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.669+0000 7f313aac4700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3134102ee0 0x7f3134195e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:35.695 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.669+0000 7f313aac4700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3134105800 0x7f313419cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:35.695 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.669+0000 7f313aac4700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f313419d4b0 con 0x7f3134102ee0 2023-07-21T18:41:35.695 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.669+0000 7f313aac4700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f313419d5f0 con 0x7f31341005c0 2023-07-21T18:41:35.695 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.669+0000 7f313aac4700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f313419d780 con 0x7f3134105800 2023-07-21T18:41:35.695 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.669+0000 7f3133fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3134102ee0 0x7f3134195e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:35.695 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.669+0000 7f3133fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3134102ee0 0x7f3134195e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56888/0 (socket says 172.21.15.39:56888) 2023-07-21T18:41:35.695 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.669+0000 7f3133fff700 1 -- 172.21.15.39:0/1330873930 learned_addr learned my addr 172.21.15.39:0/1330873930 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:35.696 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.669+0000 7f3139061700 1 --2- 172.21.15.39:0/1330873930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3134105800 0x7f313419cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:35.696 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.669+0000 7f3138860700 1 --2- 172.21.15.39:0/1330873930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31341005c0 0x7f3134195930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:35.696 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.669+0000 7f3133fff700 1 -- 172.21.15.39:0/1330873930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3134105800 msgr2=0x7f313419cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:35.696 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.669+0000 7f3133fff700 1 --2- 172.21.15.39:0/1330873930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3134105800 0x7f313419cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.696 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.669+0000 7f3133fff700 1 -- 172.21.15.39:0/1330873930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31341005c0 msgr2=0x7f3134195930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:35.696 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.669+0000 7f3133fff700 1 --2- 172.21.15.39:0/1330873930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31341005c0 0x7f3134195930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.696 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.669+0000 7f3133fff700 1 -- 172.21.15.39:0/1330873930 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f312c009d00 con 0x7f3134102ee0 2023-07-21T18:41:35.697 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.670+0000 7f3133fff700 1 --2- 172.21.15.39:0/1330873930 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3134102ee0 0x7f3134195e70 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f312000d950 tx=0x7f312000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:35.697 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.670+0000 7f3131ffb700 1 -- 172.21.15.39:0/1330873930 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3120009940 con 0x7f3134102ee0 2023-07-21T18:41:35.697 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.670+0000 7f313aac4700 1 -- 172.21.15.39:0/1330873930 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f313419da60 con 0x7f3134102ee0 2023-07-21T18:41:35.697 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.670+0000 7f3131ffb700 1 -- 172.21.15.39:0/1330873930 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3120010460 con 0x7f3134102ee0 2023-07-21T18:41:35.697 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.670+0000 7f3131ffb700 1 -- 172.21.15.39:0/1330873930 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f312000f620 con 0x7f3134102ee0 2023-07-21T18:41:35.697 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.670+0000 7f313aac4700 1 -- 172.21.15.39:0/1330873930 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f313419df70 con 0x7f3134102ee0 2023-07-21T18:41:35.697 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.672+0000 7f3131ffb700 1 -- 172.21.15.39:0/1330873930 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f31200108d0 con 0x7f3134102ee0 2023-07-21T18:41:35.700 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.673+0000 7f3131ffb700 1 --2- 172.21.15.39:0/1330873930 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f31240a11e0 0x7f31240a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:35.700 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.673+0000 7f3131ffb700 1 -- 172.21.15.39:0/1330873930 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f31200cc8e0 con 0x7f3134102ee0 2023-07-21T18:41:35.700 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.673+0000 7f3138860700 1 --2- 172.21.15.39:0/1330873930 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f31240a11e0 0x7f31240a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:35.700 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.673+0000 7f3138860700 1 --2- 172.21.15.39:0/1330873930 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f31240a11e0 0x7f31240a3690 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3128005fd0 tx=0x7f312800a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:35.700 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.675+0000 7f3146c4ab80 1 Processor -- start 2023-07-21T18:41:35.700 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.675+0000 7f3146c4ab80 1 -- start start 2023-07-21T18:41:35.701 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.675+0000 7f3146c4ab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b97ec274d0 0x55b97ec278e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:35.701 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.675+0000 7f3146c4ab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b97ec27e20 0x55b97ec286e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:35.701 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.675+0000 7f3146c4ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b97ec28c70 0x55b97ec290e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:35.701 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.675+0000 7f3146c4ab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b97eb2ccb0 con 0x55b97ec27e20 2023-07-21T18:41:35.701 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.675+0000 7f3146c4ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b97eb7fa40 con 0x55b97ec28c70 2023-07-21T18:41:35.701 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.675+0000 7f3146c4ab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b97eb78ed0 con 0x55b97ec274d0 2023-07-21T18:41:35.701 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.675+0000 7f3133fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b97ec27e20 0x55b97ec286e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:35.701 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.675+0000 7f3133fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b97ec27e20 0x55b97ec286e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56896/0 (socket says 172.21.15.39:56896) 2023-07-21T18:41:35.702 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.675+0000 7f3133fff700 1 -- 172.21.15.39:0/3671477167 learned_addr learned my addr 172.21.15.39:0/3671477167 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:35.702 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.676+0000 7f3138860700 1 --2- 172.21.15.39:0/3671477167 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b97ec274d0 0x55b97ec278e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:35.702 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.676+0000 7f3139061700 1 --2- 172.21.15.39:0/3671477167 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b97ec28c70 0x55b97ec290e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:35.702 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.676+0000 7f3133fff700 1 -- 172.21.15.39:0/3671477167 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b97ec274d0 msgr2=0x55b97ec278e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:35.702 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.676+0000 7f3133fff700 1 --2- 172.21.15.39:0/3671477167 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b97ec274d0 0x55b97ec278e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.703 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.676+0000 7f3133fff700 1 -- 172.21.15.39:0/3671477167 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b97ec28c70 msgr2=0x55b97ec290e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:35.703 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.676+0000 7f3133fff700 1 --2- 172.21.15.39:0/3671477167 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b97ec28c70 0x55b97ec290e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.703 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.676+0000 7f3133fff700 1 -- 172.21.15.39:0/3671477167 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b97e9a6de0 con 0x55b97ec27e20 2023-07-21T18:41:35.703 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.676+0000 7f3139061700 1 --2- 172.21.15.39:0/3671477167 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b97ec28c70 0x55b97ec290e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:41:35.703 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.676+0000 7f3133fff700 1 --2- 172.21.15.39:0/3671477167 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b97ec27e20 0x55b97ec286e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f3120096630 tx=0x7f3120099620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3f83faf0e5bbb61 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:35.703 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.676+0000 7f311effd700 1 -- 172.21.15.39:0/3671477167 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31200ca510 con 0x55b97ec27e20 2023-07-21T18:41:35.704 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.676+0000 7f311effd700 1 -- 172.21.15.39:0/3671477167 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3120099e20 con 0x55b97ec27e20 2023-07-21T18:41:35.704 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.676+0000 7f311effd700 1 -- 172.21.15.39:0/3671477167 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31200a1de0 con 0x55b97ec27e20 2023-07-21T18:41:35.704 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.677+0000 7f3146c4ab80 1 -- 172.21.15.39:0/3671477167 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b97ec27e20 msgr2=0x55b97ec286e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:35.704 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.677+0000 7f3146c4ab80 1 --2- 172.21.15.39:0/3671477167 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b97ec27e20 0x55b97ec286e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f3120096630 tx=0x7f3120099620 comp rx=0 tx=0).stop 2023-07-21T18:41:35.704 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.677+0000 7f3146c4ab80 1 -- 172.21.15.39:0/3671477167 shutdown_connections 2023-07-21T18:41:35.704 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.677+0000 7f3146c4ab80 1 --2- 172.21.15.39:0/3671477167 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b97ec28c70 0x55b97ec290e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.704 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.677+0000 7f3146c4ab80 1 --2- 172.21.15.39:0/3671477167 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b97ec274d0 0x55b97ec278e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.705 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.677+0000 7f3146c4ab80 1 --2- 172.21.15.39:0/3671477167 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b97ec27e20 0x55b97ec286e0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.705 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.677+0000 7f3146c4ab80 1 -- 172.21.15.39:0/3671477167 >> 172.21.15.39:0/3671477167 conn(0x55b97ec1cda0 msgr2=0x55b97ec221d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:35.705 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.677+0000 7f3146c4ab80 1 -- 172.21.15.39:0/3671477167 shutdown_connections 2023-07-21T18:41:35.705 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.677+0000 7f3146c4ab80 1 -- 172.21.15.39:0/3671477167 wait complete. 2023-07-21T18:41:35.705 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.677+0000 7f3146c4ab80 1 -- start start 2023-07-21T18:41:35.705 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.677+0000 7f3146c4ab80 1 Processor -- start 2023-07-21T18:41:35.707 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.678+0000 7f3146c4ab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b97ec274d0 0x55b97ec311c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:35.708 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.678+0000 7f3146c4ab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b97ec27e20 0x55b97ec31700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:35.708 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.678+0000 7f3146c4ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b97ec28c70 0x55b97ec38740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:35.708 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.678+0000 7f3146c4ab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b97eb18ac0 con 0x55b97ec27e20 2023-07-21T18:41:35.708 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.678+0000 7f3146c4ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b97eb187c0 con 0x55b97ec28c70 2023-07-21T18:41:35.708 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.678+0000 7f3146c4ab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b97e9fc920 con 0x55b97ec274d0 2023-07-21T18:41:35.708 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.678+0000 7f3146c4ab80 10 client.0 _ll_register_callbacks cb 0x55b97eb16760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:41:35.708 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.678+0000 7f3133fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b97ec27e20 0x55b97ec31700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:35.709 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.678+0000 7f3133fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b97ec27e20 0x55b97ec31700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56902/0 (socket says 172.21.15.39:56902) 2023-07-21T18:41:35.709 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.678+0000 7f3133fff700 1 -- 172.21.15.39:0/899608797 learned_addr learned my addr 172.21.15.39:0/899608797 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:35.709 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.678+0000 7f3139061700 1 --2- 172.21.15.39:0/899608797 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b97ec28c70 0x55b97ec38740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:35.709 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.678+0000 7f3133fff700 1 -- 172.21.15.39:0/899608797 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b97ec274d0 msgr2=0x55b97ec311c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:35.709 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.678+0000 7f3133fff700 1 --2- 172.21.15.39:0/899608797 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b97ec274d0 0x55b97ec311c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.709 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.678+0000 7f3133fff700 1 -- 172.21.15.39:0/899608797 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b97ec28c70 msgr2=0x55b97ec38740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:35.709 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.678+0000 7f3133fff700 1 --2- 172.21.15.39:0/899608797 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b97ec28c70 0x55b97ec38740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.710 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.678+0000 7f3133fff700 1 -- 172.21.15.39:0/899608797 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f313419df70 con 0x55b97ec27e20 2023-07-21T18:41:35.710 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.678+0000 7f3133fff700 1 --2- 172.21.15.39:0/899608797 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b97ec27e20 0x55b97ec31700 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f31200960c0 tx=0x7f3120010cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21d851947a473871 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:35.710 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.678+0000 7f3113fff700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:41:35.710 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.678+0000 7f3113fff700 1 -- 172.21.15.39:0/899608797 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31200b2150 con 0x55b97ec27e20 2023-07-21T18:41:35.710 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.678+0000 7f3113fff700 1 -- 172.21.15.39:0/899608797 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f31200b22b0 con 0x55b97ec27e20 2023-07-21T18:41:35.710 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.678+0000 7f3146c4ab80 1 -- 172.21.15.39:0/899608797 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b97ead24f0 con 0x55b97ec27e20 2023-07-21T18:41:35.710 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.679+0000 7f3113fff700 1 -- 172.21.15.39:0/899608797 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f31200a1b80 con 0x55b97ec27e20 2023-07-21T18:41:35.710 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.679+0000 7f3146c4ab80 10 client.15447 fetch_fsmap learned FSMap version 26 2023-07-21T18:41:35.711 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.679+0000 7f3146c4ab80 1 -- 172.21.15.39:0/899608797 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55b97e98fc60 con 0x55b97ec27e20 2023-07-21T18:41:35.711 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.679+0000 7f3113fff700 1 -- 172.21.15.39:0/899608797 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3120063020 con 0x55b97ec27e20 2023-07-21T18:41:35.711 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.679+0000 7f3113fff700 1 -- 172.21.15.39:0/899608797 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3120009aa0 con 0x55b97ec27e20 2023-07-21T18:41:35.711 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.679+0000 7f3146c4ab80 10 client.15447 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T18:41:35.711 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.679+0000 7f3146c4ab80 10 client.15447 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T18:41:35.711 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.679+0000 7f3146c4ab80 20 client.15447 populate_metadata read hostname 'smithi039' 2023-07-21T18:41:35.711 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.679+0000 7f3146c4ab80 1 --2- 172.21.15.39:0/899608797 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55b97ec3ca10 0x55b97ec3ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:35.711 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.679+0000 7f3146c4ab80 4 client.15447 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:41:35.712 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.679+0000 7f3146c4ab80 1 -- 172.21.15.39:0/899608797 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b97eb71eb0 con 0x55b97ec3ca10 2023-07-21T18:41:35.712 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.679+0000 7f3138860700 1 --2- 172.21.15.39:0/899608797 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55b97ec3ca10 0x55b97ec3ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:35.712 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.680+0000 7f3113fff700 10 client.15447 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:41:35.712 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.680+0000 7f3138860700 1 --2- 172.21.15.39:0/899608797 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55b97ec3ca10 0x55b97ec3ee20 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fba4d5dc173426f server_cookie=aa3d0974717e14fd in_seq=0 out_seq=0 2023-07-21T18:41:35.712 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.680+0000 7f3113fff700 1 -- 172.21.15.39:0/899608797 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b97eb71eb0 con 0x55b97ec3ca10 2023-07-21T18:41:35.712 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.680+0000 7f3113fff700 10 client.15447 handle_command_reply: tid=0 2023-07-21T18:41:35.712 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.680+0000 7f3146c4ab80 1 client.15447 shutdown 2023-07-21T18:41:35.713 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.680+0000 7f3146c4ab80 1 -- 172.21.15.39:0/899608797 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b97ec27e20 msgr2=0x55b97ec31700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:35.713 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.680+0000 7f3146c4ab80 1 --2- 172.21.15.39:0/899608797 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b97ec27e20 0x55b97ec31700 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f31200960c0 tx=0x7f3120010cd0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.713 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.680+0000 7f3146c4ab80 1 -- 172.21.15.39:0/899608797 shutdown_connections 2023-07-21T18:41:35.713 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.680+0000 7f3146c4ab80 1 --2- 172.21.15.39:0/899608797 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55b97ec3ca10 0x55b97ec3ee20 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.713 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.680+0000 7f3146c4ab80 1 --2- 172.21.15.39:0/899608797 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b97ec28c70 0x55b97ec38740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.713 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.680+0000 7f3113fff700 0 client.15447 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:41:35.713 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.680+0000 7f3146c4ab80 1 --2- 172.21.15.39:0/899608797 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b97ec274d0 0x55b97ec311c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.714 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.680+0000 7f3146c4ab80 1 --2- 172.21.15.39:0/899608797 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b97ec27e20 0x55b97ec31700 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.714 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.680+0000 7f3146c4ab80 1 -- 172.21.15.39:0/899608797 >> 172.21.15.39:0/899608797 conn(0x55b97ec1cda0 msgr2=0x55b97ec218b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:35.714 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.681+0000 7f3146c4ab80 1 -- 172.21.15.39:0/899608797 shutdown_connections 2023-07-21T18:41:35.714 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.681+0000 7f3146c4ab80 1 -- 172.21.15.39:0/899608797 wait complete. 2023-07-21T18:41:35.714 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.681+0000 7f3146c4ab80 20 client.15447 trim_cache size 0 max 16384 2023-07-21T18:41:35.727 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.698+0000 7f3146c4ab80 1 Processor -- start 2023-07-21T18:41:35.727 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.698+0000 7f3146c4ab80 1 -- start start 2023-07-21T18:41:35.727 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.698+0000 7f3146c4ab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b97ec274d0 0x55b97ec2f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:35.727 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.698+0000 7f3146c4ab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b97ec27e20 0x55b97ec38700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:35.727 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.698+0000 7f3146c4ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b97ec28c70 0x55b97ec2bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:35.728 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.698+0000 7f3146c4ab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b97ec30d90 con 0x55b97ec274d0 2023-07-21T18:41:35.728 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.698+0000 7f3146c4ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b97eb71eb0 con 0x55b97ec28c70 2023-07-21T18:41:35.728 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.698+0000 7f3146c4ab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b97ec234c0 con 0x55b97ec27e20 2023-07-21T18:41:35.728 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.698+0000 7f3138860700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b97ec274d0 0x55b97ec2f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:35.728 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.698+0000 7f3138860700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b97ec274d0 0x55b97ec2f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56908/0 (socket says 172.21.15.39:56908) 2023-07-21T18:41:35.728 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.698+0000 7f3138860700 1 -- 172.21.15.39:0/3724745567 learned_addr learned my addr 172.21.15.39:0/3724745567 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:35.728 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.698+0000 7f3133fff700 1 --2- 172.21.15.39:0/3724745567 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b97ec27e20 0x55b97ec38700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:35.729 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.698+0000 7f3139061700 1 --2- 172.21.15.39:0/3724745567 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b97ec28c70 0x55b97ec2bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:35.729 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.698+0000 7f3138860700 1 -- 172.21.15.39:0/3724745567 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b97ec27e20 msgr2=0x55b97ec38700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:35.729 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.698+0000 7f3138860700 1 --2- 172.21.15.39:0/3724745567 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b97ec27e20 0x55b97ec38700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.729 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.698+0000 7f3138860700 1 -- 172.21.15.39:0/3724745567 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b97ec28c70 msgr2=0x55b97ec2bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:35.729 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.698+0000 7f3138860700 1 --2- 172.21.15.39:0/3724745567 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b97ec28c70 0x55b97ec2bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.729 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.698+0000 7f3138860700 1 -- 172.21.15.39:0/3724745567 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b97e9cd250 con 0x55b97ec274d0 2023-07-21T18:41:35.729 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.699+0000 7f3138860700 1 --2- 172.21.15.39:0/3724745567 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b97ec274d0 0x55b97ec2f110 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f312800aa60 tx=0x7f3128011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78f3a6f124dd9a9f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:35.730 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.699+0000 7f311f7fe700 1 -- 172.21.15.39:0/3724745567 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3128023040 con 0x55b97ec274d0 2023-07-21T18:41:35.730 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.699+0000 7f311f7fe700 1 -- 172.21.15.39:0/3724745567 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f312802a070 con 0x55b97ec274d0 2023-07-21T18:41:35.730 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.699+0000 7f311f7fe700 1 -- 172.21.15.39:0/3724745567 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3128005420 con 0x55b97ec274d0 2023-07-21T18:41:35.730 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.699+0000 7f3146c4ab80 1 -- 172.21.15.39:0/3724745567 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b97ec274d0 msgr2=0x55b97ec2f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:35.730 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.699+0000 7f3146c4ab80 1 --2- 172.21.15.39:0/3724745567 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b97ec274d0 0x55b97ec2f110 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f312800aa60 tx=0x7f3128011000 comp rx=0 tx=0).stop 2023-07-21T18:41:35.730 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.699+0000 7f3146c4ab80 1 -- 172.21.15.39:0/3724745567 shutdown_connections 2023-07-21T18:41:35.730 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.699+0000 7f3146c4ab80 1 --2- 172.21.15.39:0/3724745567 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b97ec28c70 0x55b97ec2bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.730 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.699+0000 7f3146c4ab80 1 --2- 172.21.15.39:0/3724745567 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b97ec274d0 0x55b97ec2f110 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.731 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.699+0000 7f3146c4ab80 1 --2- 172.21.15.39:0/3724745567 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b97ec27e20 0x55b97ec38700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.731 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.699+0000 7f3146c4ab80 1 -- 172.21.15.39:0/3724745567 >> 172.21.15.39:0/3724745567 conn(0x55b97ec1cda0 msgr2=0x55b97ec2fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:35.731 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.700+0000 7f3146c4ab80 1 -- 172.21.15.39:0/3724745567 shutdown_connections 2023-07-21T18:41:35.731 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.700+0000 7f3146c4ab80 1 -- 172.21.15.39:0/3724745567 wait complete. 2023-07-21T18:41:35.731 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.700+0000 7f3146c4ab80 1 -- start start 2023-07-21T18:41:35.731 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.700+0000 7f3146c4ab80 1 Processor -- start 2023-07-21T18:41:35.731 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.700+0000 7f3146c4ab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b97ec274d0 0x55b97ec38220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:35.731 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.700+0000 7f3146c4ab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b97ec27e20 0x55b97ec36830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:35.732 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.700+0000 7f3146c4ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b97ec28c70 0x55b97ec36d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:35.732 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.700+0000 7f3146c4ab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b97ec38a40 con 0x55b97ec27e20 2023-07-21T18:41:35.732 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.700+0000 7f3146c4ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b97ec372b0 con 0x55b97ec28c70 2023-07-21T18:41:35.732 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.700+0000 7f3146c4ab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b97ec37470 con 0x55b97ec274d0 2023-07-21T18:41:35.732 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.700+0000 7f3146c4ab80 10 client.0 _ll_register_callbacks cb 0x7f3128014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:41:35.732 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.700+0000 7f3133fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b97ec27e20 0x55b97ec36830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:35.732 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.700+0000 7f3133fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b97ec27e20 0x55b97ec36830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56916/0 (socket says 172.21.15.39:56916) 2023-07-21T18:41:35.733 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.700+0000 7f3133fff700 1 -- 172.21.15.39:0/2486510258 learned_addr learned my addr 172.21.15.39:0/2486510258 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:35.733 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.701+0000 7f3139061700 1 --2- 172.21.15.39:0/2486510258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b97ec28c70 0x55b97ec36d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:35.733 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.701+0000 7f3138860700 1 --2- 172.21.15.39:0/2486510258 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b97ec274d0 0x55b97ec38220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:35.733 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.701+0000 7f3133fff700 1 -- 172.21.15.39:0/2486510258 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b97ec274d0 msgr2=0x55b97ec38220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:35.733 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.701+0000 7f3133fff700 1 --2- 172.21.15.39:0/2486510258 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b97ec274d0 0x55b97ec38220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.733 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.701+0000 7f3133fff700 1 -- 172.21.15.39:0/2486510258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b97ec28c70 msgr2=0x55b97ec36d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:35.734 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.701+0000 7f3133fff700 1 --2- 172.21.15.39:0/2486510258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b97ec28c70 0x55b97ec36d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.734 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.701+0000 7f3133fff700 1 -- 172.21.15.39:0/2486510258 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f312800eb30 con 0x55b97ec27e20 2023-07-21T18:41:35.734 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.701+0000 7f3133fff700 1 --2- 172.21.15.39:0/2486510258 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b97ec27e20 0x55b97ec36830 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f3120099b50 tx=0x7f31200995d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f634fd7ec90d1265 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:35.734 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.701+0000 7f3113fff700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:41:35.734 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.701+0000 7f3113fff700 1 -- 172.21.15.39:0/2486510258 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31200a1d80 con 0x55b97ec27e20 2023-07-21T18:41:35.734 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.701+0000 7f3146c4ab80 1 -- 172.21.15.39:0/2486510258 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3120099e20 con 0x55b97ec27e20 2023-07-21T18:41:35.734 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.701+0000 7f3113fff700 1 -- 172.21.15.39:0/2486510258 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f3120064080 con 0x55b97ec27e20 2023-07-21T18:41:35.734 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.701+0000 7f3113fff700 1 -- 172.21.15.39:0/2486510258 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3120098010 con 0x55b97ec27e20 2023-07-21T18:41:35.735 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.701+0000 7f3146c4ab80 10 client.15453 fetch_fsmap learned FSMap version 26 2023-07-21T18:41:35.735 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.701+0000 7f3146c4ab80 1 -- 172.21.15.39:0/2486510258 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55b97e9a52b0 con 0x55b97ec27e20 2023-07-21T18:41:35.735 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.701+0000 7f3113fff700 1 -- 172.21.15.39:0/2486510258 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3120099e20 con 0x55b97ec27e20 2023-07-21T18:41:35.735 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.702+0000 7f3113fff700 1 -- 172.21.15.39:0/2486510258 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3120098b20 con 0x55b97ec27e20 2023-07-21T18:41:35.735 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.702+0000 7f3146c4ab80 10 client.15453 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T18:41:35.735 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.702+0000 7f3146c4ab80 10 client.15453 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T18:41:35.736 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.702+0000 7f3146c4ab80 20 client.15453 populate_metadata read hostname 'smithi039' 2023-07-21T18:41:35.736 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.702+0000 7f3146c4ab80 1 --2- 172.21.15.39:0/2486510258 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55b97ec919b0 0x55b97ec93dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:35.736 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.702+0000 7f3146c4ab80 4 client.15453 mds_command: new command op to 24536 tid=0[{"prefix": "scrub status"}] 2023-07-21T18:41:35.736 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.702+0000 7f3146c4ab80 1 -- 172.21.15.39:0/2486510258 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b97eb34050 con 0x55b97ec919b0 2023-07-21T18:41:35.736 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.702+0000 7f3138860700 1 --2- 172.21.15.39:0/2486510258 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55b97ec919b0 0x55b97ec93dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:35.736 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.702+0000 7f3113fff700 10 client.15453 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:41:35.736 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.702+0000 7f3138860700 1 --2- 172.21.15.39:0/2486510258 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55b97ec919b0 0x55b97ec93dc0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee84c1b16181989b server_cookie=85df8d6197fa178c in_seq=0 out_seq=0 2023-07-21T18:41:35.737 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.703+0000 7f3113fff700 1 -- 172.21.15.39:0/2486510258 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55b97eb34050 con 0x55b97ec919b0 2023-07-21T18:41:35.737 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.703+0000 7f3113fff700 10 client.15453 handle_command_reply: tid=0 2023-07-21T18:41:35.737 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.703+0000 7f3146c4ab80 1 client.15453 shutdown 2023-07-21T18:41:35.737 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.703+0000 7f3146c4ab80 1 -- 172.21.15.39:0/2486510258 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b97ec27e20 msgr2=0x55b97ec36830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:35.738 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.703+0000 7f3146c4ab80 1 --2- 172.21.15.39:0/2486510258 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b97ec27e20 0x55b97ec36830 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f3120099b50 tx=0x7f31200995d0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.738 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.703+0000 7f3146c4ab80 1 -- 172.21.15.39:0/2486510258 shutdown_connections 2023-07-21T18:41:35.738 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.703+0000 7f3146c4ab80 1 --2- 172.21.15.39:0/2486510258 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55b97ec919b0 0x55b97ec93dc0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.739 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.703+0000 7f3146c4ab80 1 --2- 172.21.15.39:0/2486510258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b97ec28c70 0x55b97ec36d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.739 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.703+0000 7f3113fff700 0 client.15453 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:41:35.739 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.703+0000 7f3146c4ab80 1 --2- 172.21.15.39:0/2486510258 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b97ec274d0 0x55b97ec38220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.739 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.703+0000 7f3146c4ab80 1 --2- 172.21.15.39:0/2486510258 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b97ec27e20 0x55b97ec36830 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.740 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.703+0000 7f3146c4ab80 1 -- 172.21.15.39:0/2486510258 >> 172.21.15.39:0/2486510258 conn(0x55b97ec1cda0 msgr2=0x55b97ec2f110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:35.740 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.703+0000 7f3146c4ab80 1 -- 172.21.15.39:0/2486510258 shutdown_connections 2023-07-21T18:41:35.740 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.703+0000 7f3146c4ab80 1 -- 172.21.15.39:0/2486510258 wait complete. 2023-07-21T18:41:35.740 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.703+0000 7f3146c4ab80 20 client.15453 trim_cache size 0 max 16384 2023-07-21T18:41:35.740 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2023-07-21T18:41:35.741 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T18:41:35.741 INFO:teuthology.orchestra.run.smithi039.stdout: "scrubs": { 2023-07-21T18:41:35.741 INFO:teuthology.orchestra.run.smithi039.stdout: "ddc2b241-385e-45b5-b13a-84178d7135cb": { 2023-07-21T18:41:35.741 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2023-07-21T18:41:35.741 INFO:teuthology.orchestra.run.smithi039.stdout: "tag": "ddc2b241-385e-45b5-b13a-84178d7135cb", 2023-07-21T18:41:35.741 INFO:teuthology.orchestra.run.smithi039.stdout: "options": "recursive,force" 2023-07-21T18:41:35.741 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T18:41:35.742 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T18:41:35.742 INFO:teuthology.orchestra.run.smithi039.stdout:} 2023-07-21T18:41:35.743 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.703+0000 7f313aac4700 1 -- 172.21.15.39:0/1330873930 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f31240a11e0 msgr2=0x7f31240a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:35.744 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.703+0000 7f313aac4700 1 --2- 172.21.15.39:0/1330873930 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f31240a11e0 0x7f31240a3690 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3128005fd0 tx=0x7f312800a560 comp rx=0 tx=0).stop 2023-07-21T18:41:35.744 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.703+0000 7f313aac4700 1 -- 172.21.15.39:0/1330873930 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3134102ee0 msgr2=0x7f3134195e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:35.744 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.703+0000 7f313aac4700 1 --2- 172.21.15.39:0/1330873930 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3134102ee0 0x7f3134195e70 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f312000d950 tx=0x7f312000dc60 comp rx=0 tx=0).stop 2023-07-21T18:41:35.744 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.703+0000 7f313aac4700 1 -- 172.21.15.39:0/1330873930 shutdown_connections 2023-07-21T18:41:35.744 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.703+0000 7f313aac4700 1 --2- 172.21.15.39:0/1330873930 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7f31240a11e0 0x7f31240a3690 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.744 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.703+0000 7f313aac4700 1 --2- 172.21.15.39:0/1330873930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31341005c0 0x7f3134195930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.745 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.703+0000 7f313aac4700 1 --2- 172.21.15.39:0/1330873930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3134105800 0x7f313419cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.745 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.703+0000 7f313aac4700 1 --2- 172.21.15.39:0/1330873930 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3134102ee0 0x7f3134195e70 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:35.745 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.703+0000 7f313aac4700 1 -- 172.21.15.39:0/1330873930 >> 172.21.15.39:0/1330873930 conn(0x7f31340fa000 msgr2=0x7f3134101220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:35.745 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.704+0000 7f313aac4700 1 -- 172.21.15.39:0/1330873930 shutdown_connections 2023-07-21T18:41:35.745 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.704+0000 7f313aac4700 1 -- 172.21.15.39:0/1330873930 wait complete. 2023-07-21T18:41:35.745 INFO:tasks.cephfs.filesystem:scrub status for tag:ddc2b241-385e-45b5-b13a-84178d7135cb - {'path': '/', 'tag': 'ddc2b241-385e-45b5-b13a-84178d7135cb', 'options': 'recursive,force'} 2023-07-21T18:41:35.781 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:35.758+0000 7fbafce9d700 1 -- 172.21.15.39:0/2880058095 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fbaf802c850 con 0x7fbaf80ffc90 2023-07-21T18:41:36.207 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.183+0000 7fbae77fe700 1 -- 172.21.15.39:0/2880058095 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mgrmap(e 18) v1 ==== 135783+0+0 (secure 0 0 0) 0x7fbaec096930 con 0x7fbaf80ffc90 2023-07-21T18:41:36.208 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.184+0000 7fbae77fe700 1 -- 172.21.15.39:0/2880058095 <== mon.0 v2:172.21.15.39:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v18) v1 ==== 92+0+0 (secure 0 0 0) 0x7fbaec0ab680 con 0x7fbaf80ffc90 2023-07-21T18:41:36.211 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.188+0000 7fbafce9d700 1 -- 172.21.15.39:0/2880058095 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fbae00a11e0 msgr2=0x7fbae00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:36.211 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.188+0000 7fbafce9d700 1 --2- 172.21.15.39:0/2880058095 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fbae00a11e0 0x7fbae00a3690 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbae800b2e0 tx=0x7fbae8005fb0 comp rx=0 tx=0).stop 2023-07-21T18:41:36.211 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.188+0000 7fbafce9d700 1 -- 172.21.15.39:0/2880058095 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbaf80ffc90 msgr2=0x7fbaf819aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:36.211 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.188+0000 7fbafce9d700 1 --2- 172.21.15.39:0/2880058095 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbaf80ffc90 0x7fbaf819aa90 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fbaec00e9a0 tx=0x7fbaec00ecb0 comp rx=0 tx=0).stop 2023-07-21T18:41:36.212 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.188+0000 7fbafce9d700 1 -- 172.21.15.39:0/2880058095 shutdown_connections 2023-07-21T18:41:36.212 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.188+0000 7fbafce9d700 1 --2- 172.21.15.39:0/2880058095 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fbae00a11e0 0x7fbae00a3690 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:36.212 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.188+0000 7fbafce9d700 1 --2- 172.21.15.39:0/2880058095 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbaf80fea00 0x7fbaf8198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:36.212 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.188+0000 7fbafce9d700 1 --2- 172.21.15.39:0/2880058095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbaf8100640 0x7fbaf819cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:36.213 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.188+0000 7fbafce9d700 1 --2- 172.21.15.39:0/2880058095 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbaf80ffc90 0x7fbaf819aa90 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:36.213 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.188+0000 7fbafce9d700 1 -- 172.21.15.39:0/2880058095 >> 172.21.15.39:0/2880058095 conn(0x7fbaf80fa000 msgr2=0x7fbaf80fc250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:36.213 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.188+0000 7fbafce9d700 1 -- 172.21.15.39:0/2880058095 shutdown_connections 2023-07-21T18:41:36.213 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.188+0000 7fbafce9d700 1 -- 172.21.15.39:0/2880058095 wait complete. 2023-07-21T18:41:36.229 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2023-07-21T18:41:36.399 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.375+0000 7fb82f1c0700 1 -- 172.21.15.39:0/1613643438 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb828101690 msgr2=0x7fb828107f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:36.399 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.375+0000 7fb82f1c0700 1 --2- 172.21.15.39:0/1613643438 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb828101690 0x7fb828107f50 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fb82000b0a0 tx=0x7fb82000b3b0 comp rx=0 tx=0).stop 2023-07-21T18:41:36.400 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.375+0000 7fb82f1c0700 1 -- 172.21.15.39:0/1613643438 shutdown_connections 2023-07-21T18:41:36.400 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.375+0000 7fb82f1c0700 1 --2- 172.21.15.39:0/1613643438 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8280ffb10 0x7fb8280fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:36.400 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.375+0000 7fb82f1c0700 1 --2- 172.21.15.39:0/1613643438 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb828101690 0x7fb828107f50 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:36.400 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.375+0000 7fb82f1c0700 1 --2- 172.21.15.39:0/1613643438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb828100d00 0x7fb828101150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:36.400 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.375+0000 7fb82f1c0700 1 -- 172.21.15.39:0/1613643438 >> 172.21.15.39:0/1613643438 conn(0x7fb8280fb130 msgr2=0x7fb8280fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:36.400 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.375+0000 7fb82f1c0700 1 -- 172.21.15.39:0/1613643438 shutdown_connections 2023-07-21T18:41:36.400 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.375+0000 7fb82f1c0700 1 -- 172.21.15.39:0/1613643438 wait complete. 2023-07-21T18:41:36.402 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.376+0000 7fb82f1c0700 1 Processor -- start 2023-07-21T18:41:36.402 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.376+0000 7fb82f1c0700 1 -- start start 2023-07-21T18:41:36.402 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.377+0000 7fb82f1c0700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8280ffb10 0x7fb828195a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:36.403 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.377+0000 7fb82f1c0700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb828100d00 0x7fb828195fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:36.403 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.377+0000 7fb82f1c0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb828101690 0x7fb828196e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:36.403 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.377+0000 7fb82f1c0700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb828197490 con 0x7fb828100d00 2023-07-21T18:41:36.403 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.377+0000 7fb82f1c0700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb828197600 con 0x7fb828101690 2023-07-21T18:41:36.403 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.377+0000 7fb82f1c0700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8281977c0 con 0x7fb8280ffb10 2023-07-21T18:41:36.403 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.377+0000 7fb827fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb828100d00 0x7fb828195fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:36.403 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.377+0000 7fb82cf5c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8280ffb10 0x7fb828195a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:36.404 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.377+0000 7fb827fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb828100d00 0x7fb828195fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56956/0 (socket says 172.21.15.39:56956) 2023-07-21T18:41:36.404 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.377+0000 7fb827fff700 1 -- 172.21.15.39:0/262606771 learned_addr learned my addr 172.21.15.39:0/262606771 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:36.404 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.377+0000 7fb82d75d700 1 --2- 172.21.15.39:0/262606771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb828101690 0x7fb828196e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:36.404 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.377+0000 7fb827fff700 1 -- 172.21.15.39:0/262606771 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8280ffb10 msgr2=0x7fb828195a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:36.404 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.377+0000 7fb827fff700 1 --2- 172.21.15.39:0/262606771 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8280ffb10 0x7fb828195a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:36.404 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.377+0000 7fb827fff700 1 -- 172.21.15.39:0/262606771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb828101690 msgr2=0x7fb828196e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:36.404 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.377+0000 7fb827fff700 1 --2- 172.21.15.39:0/262606771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb828101690 0x7fb828196e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:36.404 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.377+0000 7fb827fff700 1 -- 172.21.15.39:0/262606771 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb81c009520 con 0x7fb828100d00 2023-07-21T18:41:36.405 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.377+0000 7fb82cf5c700 1 --2- 172.21.15.39:0/262606771 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8280ffb10 0x7fb828195a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:41:36.405 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.377+0000 7fb827fff700 1 --2- 172.21.15.39:0/262606771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb828100d00 0x7fb828195fb0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fb81400e9a0 tx=0x7fb81400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:36.405 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.377+0000 7fb825ffb700 1 -- 172.21.15.39:0/262606771 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb814009790 con 0x7fb828100d00 2023-07-21T18:41:36.405 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.377+0000 7fb825ffb700 1 -- 172.21.15.39:0/262606771 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb814010430 con 0x7fb828100d00 2023-07-21T18:41:36.405 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.377+0000 7fb82f1c0700 1 -- 172.21.15.39:0/262606771 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb820009d00 con 0x7fb828100d00 2023-07-21T18:41:36.405 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.377+0000 7fb825ffb700 1 -- 172.21.15.39:0/262606771 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb814018680 con 0x7fb828100d00 2023-07-21T18:41:36.405 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.378+0000 7fb82f1c0700 1 -- 172.21.15.39:0/262606771 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb82819e080 con 0x7fb828100d00 2023-07-21T18:41:36.406 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.379+0000 7fb82f1c0700 1 -- 172.21.15.39:0/262606771 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb828107030 con 0x7fb828100d00 2023-07-21T18:41:36.406 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.380+0000 7fb825ffb700 1 -- 172.21.15.39:0/262606771 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 18) v1 ==== 135783+0+0 (secure 0 0 0) 0x7fb8140105a0 con 0x7fb828100d00 2023-07-21T18:41:36.406 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.381+0000 7fb825ffb700 1 --2- 172.21.15.39:0/262606771 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fb8180a1230 0x7fb8180a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:36.407 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.381+0000 7fb82cf5c700 1 -- 172.21.15.39:0/262606771 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fb8180a1230 msgr2=0x7fb8180a36e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/2116291712 2023-07-21T18:41:36.407 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.381+0000 7fb825ffb700 1 -- 172.21.15.39:0/262606771 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fb814014070 con 0x7fb828100d00 2023-07-21T18:41:36.407 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.381+0000 7fb82cf5c700 1 --2- 172.21.15.39:0/262606771 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fb8180a1230 0x7fb8180a36e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-21T18:41:36.408 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.384+0000 7fb825ffb700 1 -- 172.21.15.39:0/262606771 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fb8140964a0 con 0x7fb828100d00 2023-07-21T18:41:36.438 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:36 smithi039 ceph-mon[96489]: pgmap v295: 97 pgs: 97 active+clean; 644 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 853 B/s wr, 11 op/s 2023-07-21T18:41:36.439 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:36 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/3782582119' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T18:41:36.439 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:36 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/2880058095' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-07-21T18:41:36.439 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:41:36 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: ignoring --setuser ceph since I am not root 2023-07-21T18:41:36.439 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:41:36 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: ignoring --setgroup ceph since I am not root 2023-07-21T18:41:36.439 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:41:36 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:41:36.381+0000 7f93cfe50200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-21T18:41:36.453 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:36 smithi155 ceph-mon[97934]: pgmap v295: 97 pgs: 97 active+clean; 644 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 853 B/s wr, 11 op/s 2023-07-21T18:41:36.453 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:36 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/3782582119' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T18:41:36.453 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:36 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/2880058095' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-07-21T18:41:36.454 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 21 18:41:36 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-y[99389]: ignoring --setuser ceph since I am not root 2023-07-21T18:41:36.454 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 21 18:41:36 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-y[99389]: ignoring --setgroup ceph since I am not root 2023-07-21T18:41:36.454 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 21 18:41:36 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-y[99389]: 2023-07-21T18:41:36.350+0000 7fa3efc74200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-21T18:41:36.454 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 21 18:41:36 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-y[99389]: 2023-07-21T18:41:36.430+0000 7fa3efc74200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-21T18:41:36.456 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:36 smithi193 ceph-mon[98204]: pgmap v295: 97 pgs: 97 active+clean; 644 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 853 B/s wr, 11 op/s 2023-07-21T18:41:36.456 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:36 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/3782582119' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T18:41:36.456 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:36 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/2880058095' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-07-21T18:41:36.457 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Jul 21 18:41:36 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-z[100072]: ignoring --setuser ceph since I am not root 2023-07-21T18:41:36.457 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Jul 21 18:41:36 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-z[100072]: ignoring --setgroup ceph since I am not root 2023-07-21T18:41:36.457 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Jul 21 18:41:36 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-z[100072]: 2023-07-21T18:41:36.352+0000 7ff0a29cc200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-21T18:41:36.457 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Jul 21 18:41:36 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-z[100072]: 2023-07-21T18:41:36.432+0000 7ff0a29cc200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-21T18:41:36.605 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.582+0000 7fb82cf5c700 1 -- 172.21.15.39:0/262606771 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fb8180a1230 msgr2=0x7fb8180a36e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/2116291712 2023-07-21T18:41:36.605 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.582+0000 7fb82cf5c700 1 --2- 172.21.15.39:0/262606771 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fb8180a1230 0x7fb8180a36e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-21T18:41:36.608 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.585+0000 7fb82f1c0700 1 -- 172.21.15.39:0/262606771 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7fb82804e2f0 con 0x7fb828100d00 2023-07-21T18:41:36.620 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.596+0000 7fb825ffb700 1 -- 172.21.15.39:0/262606771 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v23)=0 v23) v1 ==== 138+0+0 (secure 0 0 0) 0x7fb8140260d0 con 0x7fb828100d00 2023-07-21T18:41:36.622 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.599+0000 7fb82f1c0700 1 -- 172.21.15.39:0/262606771 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fb8180a1230 msgr2=0x7fb8180a36e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-21T18:41:36.623 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.599+0000 7fb82f1c0700 1 --2- 172.21.15.39:0/262606771 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fb8180a1230 0x7fb8180a36e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:36.623 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.599+0000 7fb82f1c0700 1 -- 172.21.15.39:0/262606771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb828100d00 msgr2=0x7fb828195fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:36.623 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.599+0000 7fb82f1c0700 1 --2- 172.21.15.39:0/262606771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb828100d00 0x7fb828195fb0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fb81400e9a0 tx=0x7fb81400ecb0 comp rx=0 tx=0).stop 2023-07-21T18:41:36.623 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.599+0000 7fb82f1c0700 1 -- 172.21.15.39:0/262606771 shutdown_connections 2023-07-21T18:41:36.623 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.599+0000 7fb82f1c0700 1 --2- 172.21.15.39:0/262606771 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fb8180a1230 0x7fb8180a36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:36.623 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.599+0000 7fb82f1c0700 1 --2- 172.21.15.39:0/262606771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb828101690 0x7fb828196e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:36.624 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.599+0000 7fb82f1c0700 1 --2- 172.21.15.39:0/262606771 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8280ffb10 0x7fb828195a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:36.624 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.599+0000 7fb82f1c0700 1 --2- 172.21.15.39:0/262606771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb828100d00 0x7fb828195fb0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:36.624 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.599+0000 7fb82f1c0700 1 -- 172.21.15.39:0/262606771 >> 172.21.15.39:0/262606771 conn(0x7fb8280fb130 msgr2=0x7fb828104170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:36.624 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.599+0000 7fb82f1c0700 1 -- 172.21.15.39:0/262606771 shutdown_connections 2023-07-21T18:41:36.624 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.599+0000 7fb82f1c0700 1 -- 172.21.15.39:0/262606771 wait complete. 2023-07-21T18:41:36.636 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2023-07-21T18:41:36.764 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.740+0000 7fd8d3e43700 1 -- 172.21.15.39:0/2968690521 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8cc100640 msgr2=0x7fd8cc106ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:36.764 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.740+0000 7fd8d3e43700 1 --2- 172.21.15.39:0/2968690521 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8cc100640 0x7fd8cc106ee0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fd8c4009ec0 tx=0x7fd8c400b330 comp rx=0 tx=0).stop 2023-07-21T18:41:36.765 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.740+0000 7fd8d3e43700 1 -- 172.21.15.39:0/2968690521 shutdown_connections 2023-07-21T18:41:36.765 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.740+0000 7fd8d3e43700 1 --2- 172.21.15.39:0/2968690521 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8cc100640 0x7fd8cc106ee0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:36.765 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.740+0000 7fd8d3e43700 1 --2- 172.21.15.39:0/2968690521 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd8cc0fea00 0x7fd8cc0fee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:36.765 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.740+0000 7fd8d3e43700 1 --2- 172.21.15.39:0/2968690521 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8cc0ffc90 0x7fd8cc100100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:36.765 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.740+0000 7fd8d3e43700 1 -- 172.21.15.39:0/2968690521 >> 172.21.15.39:0/2968690521 conn(0x7fd8cc0fa000 msgr2=0x7fd8cc0fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:36.765 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.741+0000 7fd8d3e43700 1 -- 172.21.15.39:0/2968690521 shutdown_connections 2023-07-21T18:41:36.765 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.741+0000 7fd8d3e43700 1 -- 172.21.15.39:0/2968690521 wait complete. 2023-07-21T18:41:36.767 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.741+0000 7fd8d3e43700 1 Processor -- start 2023-07-21T18:41:36.767 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.741+0000 7fd8d3e43700 1 -- start start 2023-07-21T18:41:36.767 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.741+0000 7fd8d3e43700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8cc0fea00 0x7fd8cc195880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:36.767 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.741+0000 7fd8d3e43700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd8cc0ffc90 0x7fd8cc195dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:36.767 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.741+0000 7fd8d3e43700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8cc100640 0x7fd8cc196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:36.767 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.741+0000 7fd8d3e43700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8cc1972a0 con 0x7fd8cc100640 2023-07-21T18:41:36.768 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.741+0000 7fd8d3e43700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8cc197410 con 0x7fd8cc0fea00 2023-07-21T18:41:36.768 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.741+0000 7fd8d3e43700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8cc1975d0 con 0x7fd8cc0ffc90 2023-07-21T18:41:36.768 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.741+0000 7fd8d23e0700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8cc100640 0x7fd8cc196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:36.768 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.741+0000 7fd8d23e0700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8cc100640 0x7fd8cc196c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57022/0 (socket says 172.21.15.39:57022) 2023-07-21T18:41:36.768 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.741+0000 7fd8d23e0700 1 -- 172.21.15.39:0/2756222737 learned_addr learned my addr 172.21.15.39:0/2756222737 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:36.768 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.741+0000 7fd8d1bdf700 1 --2- 172.21.15.39:0/2756222737 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8cc0fea00 0x7fd8cc195880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:36.768 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.741+0000 7fd8d13de700 1 --2- 172.21.15.39:0/2756222737 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd8cc0ffc90 0x7fd8cc195dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:36.769 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.742+0000 7fd8d23e0700 1 -- 172.21.15.39:0/2756222737 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd8cc0ffc90 msgr2=0x7fd8cc195dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:36.769 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.742+0000 7fd8d23e0700 1 --2- 172.21.15.39:0/2756222737 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd8cc0ffc90 0x7fd8cc195dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:36.769 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.742+0000 7fd8d23e0700 1 -- 172.21.15.39:0/2756222737 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8cc0fea00 msgr2=0x7fd8cc195880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:36.769 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.742+0000 7fd8d23e0700 1 --2- 172.21.15.39:0/2756222737 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8cc0fea00 0x7fd8cc195880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:36.769 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.742+0000 7fd8d23e0700 1 -- 172.21.15.39:0/2756222737 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8c4005e70 con 0x7fd8cc100640 2023-07-21T18:41:36.769 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.742+0000 7fd8d1bdf700 1 --2- 172.21.15.39:0/2756222737 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8cc0fea00 0x7fd8cc195880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:41:36.769 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.742+0000 7fd8d23e0700 1 --2- 172.21.15.39:0/2756222737 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8cc100640 0x7fd8cc196c40 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd8c4009fd0 tx=0x7fd8c4007cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:36.770 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.742+0000 7fd8c2ffd700 1 -- 172.21.15.39:0/2756222737 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c400e040 con 0x7fd8cc100640 2023-07-21T18:41:36.770 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.742+0000 7fd8c2ffd700 1 -- 172.21.15.39:0/2756222737 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd8c401db50 con 0x7fd8cc100640 2023-07-21T18:41:36.770 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.742+0000 7fd8d3e43700 1 -- 172.21.15.39:0/2756222737 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8cc197850 con 0x7fd8cc100640 2023-07-21T18:41:36.770 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.742+0000 7fd8c2ffd700 1 -- 172.21.15.39:0/2756222737 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c401bb00 con 0x7fd8cc100640 2023-07-21T18:41:36.770 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.742+0000 7fd8d3e43700 1 -- 172.21.15.39:0/2756222737 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8cc19e070 con 0x7fd8cc100640 2023-07-21T18:41:36.770 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.743+0000 7fd8d3e43700 1 -- 172.21.15.39:0/2756222737 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8cc04e2f0 con 0x7fd8cc100640 2023-07-21T18:41:36.771 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.744+0000 7fd8c2ffd700 1 -- 172.21.15.39:0/2756222737 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 18) v1 ==== 135783+0+0 (secure 0 0 0) 0x7fd8c401dcc0 con 0x7fd8cc100640 2023-07-21T18:41:36.771 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.745+0000 7fd8c2ffd700 1 --2- 172.21.15.39:0/2756222737 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fd8b80a1230 0x7fd8b80a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:36.771 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.745+0000 7fd8d1bdf700 1 -- 172.21.15.39:0/2756222737 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fd8b80a1230 msgr2=0x7fd8b80a36e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/2116291712 2023-07-21T18:41:36.771 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.745+0000 7fd8c2ffd700 1 -- 172.21.15.39:0/2756222737 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fd8c40ce700 con 0x7fd8cc100640 2023-07-21T18:41:36.771 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.745+0000 7fd8d1bdf700 1 --2- 172.21.15.39:0/2756222737 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fd8b80a1230 0x7fd8b80a36e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-21T18:41:36.772 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.749+0000 7fd8c2ffd700 1 -- 172.21.15.39:0/2756222737 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fd8c4097db0 con 0x7fd8cc100640 2023-07-21T18:41:36.804 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 21 18:41:36 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-y[99389]: 2023-07-21T18:41:36.519+0000 7fa3efc74200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-21T18:41:36.805 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 21 18:41:36 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-y[99389]: 2023-07-21T18:41:36.613+0000 7fa3efc74200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-21T18:41:36.808 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Jul 21 18:41:36 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-z[100072]: 2023-07-21T18:41:36.521+0000 7ff0a29cc200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-21T18:41:36.808 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Jul 21 18:41:36 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-z[100072]: 2023-07-21T18:41:36.615+0000 7ff0a29cc200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-21T18:41:36.938 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:41:36 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:41:36.480+0000 7f93cfe50200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-21T18:41:36.938 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:41:36 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:41:36.589+0000 7f93cfe50200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-21T18:41:36.938 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:41:36 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:41:36.708+0000 7f93cfe50200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-21T18:41:36.969 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.946+0000 7fd8d1bdf700 1 -- 172.21.15.39:0/2756222737 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fd8b80a1230 msgr2=0x7fd8b80a36e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/2116291712 2023-07-21T18:41:36.969 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.946+0000 7fd8d1bdf700 1 --2- 172.21.15.39:0/2756222737 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fd8b80a1230 0x7fd8b80a36e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-21T18:41:36.972 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.949+0000 7fd8d3e43700 1 -- 172.21.15.39:0/2756222737 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fd8cc19e200 con 0x7fd8cc100640 2023-07-21T18:41:36.984 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.960+0000 7fd8c2ffd700 1 -- 172.21.15.39:0/2756222737 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v24)=0 v24) v1 ==== 133+0+0 (secure 0 0 0) 0x7fd8c409b3d0 con 0x7fd8cc100640 2023-07-21T18:41:36.986 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.963+0000 7fd8d3e43700 1 -- 172.21.15.39:0/2756222737 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fd8b80a1230 msgr2=0x7fd8b80a36e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-21T18:41:36.986 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.963+0000 7fd8d3e43700 1 --2- 172.21.15.39:0/2756222737 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fd8b80a1230 0x7fd8b80a36e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:36.987 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.963+0000 7fd8d3e43700 1 -- 172.21.15.39:0/2756222737 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8cc100640 msgr2=0x7fd8cc196c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:36.987 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.963+0000 7fd8d3e43700 1 --2- 172.21.15.39:0/2756222737 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8cc100640 0x7fd8cc196c40 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd8c4009fd0 tx=0x7fd8c4007cf0 comp rx=0 tx=0).stop 2023-07-21T18:41:36.987 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.963+0000 7fd8d3e43700 1 -- 172.21.15.39:0/2756222737 shutdown_connections 2023-07-21T18:41:36.987 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.963+0000 7fd8d3e43700 1 --2- 172.21.15.39:0/2756222737 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7fd8b80a1230 0x7fd8b80a36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:36.987 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.963+0000 7fd8d3e43700 1 --2- 172.21.15.39:0/2756222737 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8cc0fea00 0x7fd8cc195880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:36.987 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.963+0000 7fd8d3e43700 1 --2- 172.21.15.39:0/2756222737 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8cc100640 0x7fd8cc196c40 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:36.987 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.963+0000 7fd8d3e43700 1 --2- 172.21.15.39:0/2756222737 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd8cc0ffc90 0x7fd8cc195dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:36.987 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.963+0000 7fd8d3e43700 1 -- 172.21.15.39:0/2756222737 >> 172.21.15.39:0/2756222737 conn(0x7fd8cc0fa000 msgr2=0x7fd8cc067920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:36.988 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.963+0000 7fd8d3e43700 1 -- 172.21.15.39:0/2756222737 shutdown_connections 2023-07-21T18:41:36.988 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:36.963+0000 7fd8d3e43700 1 -- 172.21.15.39:0/2756222737 wait complete. 2023-07-21T18:41:36.998 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1M' 2023-07-21T18:41:37.116 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Jul 21 18:41:36 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-z[100072]: 2023-07-21T18:41:36.785+0000 7ff0a29cc200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-21T18:41:37.117 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Jul 21 18:41:37 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-z[100072]: 2023-07-21T18:41:37.093+0000 7ff0a29cc200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-21T18:41:37.128 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:37.104+0000 7feb46553700 1 -- 172.21.15.39:0/1770444196 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb40102c10 msgr2=0x7feb40105050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:37.128 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:37.104+0000 7feb46553700 1 --2- 172.21.15.39:0/1770444196 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb40102c10 0x7feb40105050 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7feb340098f0 tx=0x7feb34009c00 comp rx=0 tx=0).stop 2023-07-21T18:41:37.128 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:37.104+0000 7feb46553700 1 -- 172.21.15.39:0/1770444196 shutdown_connections 2023-07-21T18:41:37.128 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:37.104+0000 7feb46553700 1 --2- 172.21.15.39:0/1770444196 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb401056e0 0x7feb40107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:37.128 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:37.104+0000 7feb46553700 1 --2- 172.21.15.39:0/1770444196 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb40068600 0x7feb401026d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:37.129 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:37.104+0000 7feb46553700 1 --2- 172.21.15.39:0/1770444196 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb40102c10 0x7feb40105050 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:37.129 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:37.104+0000 7feb46553700 1 -- 172.21.15.39:0/1770444196 >> 172.21.15.39:0/1770444196 conn(0x7feb400fa000 msgr2=0x7feb400fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:37.129 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:37.104+0000 7feb46553700 1 -- 172.21.15.39:0/1770444196 shutdown_connections 2023-07-21T18:41:37.129 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:37.104+0000 7feb46553700 1 -- 172.21.15.39:0/1770444196 wait complete. 2023-07-21T18:41:37.129 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:37.104+0000 7feb46553700 1 Processor -- start 2023-07-21T18:41:37.129 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:37.104+0000 7feb46553700 1 -- start start 2023-07-21T18:41:37.131 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:37.105+0000 7feb46553700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb40068600 0x7feb40197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:37.131 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:37.105+0000 7feb46553700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb40102c10 0x7feb40198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:37.131 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:37.105+0000 7feb46553700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb401056e0 0x7feb40198f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:37.131 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:37.105+0000 7feb46553700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb401995e0 con 0x7feb401056e0 2023-07-21T18:41:37.131 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:37.105+0000 7feb46553700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb40199750 con 0x7feb40068600 2023-07-21T18:41:37.132 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:37.105+0000 7feb46553700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb40199910 con 0x7feb40102c10 2023-07-21T18:41:37.132 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:37.105+0000 7feb44af0700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb401056e0 0x7feb40198f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:37.132 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:37.105+0000 7feb44af0700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb401056e0 0x7feb40198f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57088/0 (socket says 172.21.15.39:57088) 2023-07-21T18:41:37.132 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:37.105+0000 7feb44af0700 1 -- 172.21.15.39:0/792163655 learned_addr learned my addr 172.21.15.39:0/792163655 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:37.132 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:37.105+0000 7feb3f7fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb40102c10 0x7feb40198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:37.132 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:37.105+0000 7feb3ffff700 1 --2- 172.21.15.39:0/792163655 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb40068600 0x7feb40197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:37.132 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:37.105+0000 7feb3f7fe700 1 -- 172.21.15.39:0/792163655 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb40068600 msgr2=0x7feb40197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:37.133 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:37.105+0000 7feb3f7fe700 1 --2- 172.21.15.39:0/792163655 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb40068600 0x7feb40197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:37.133 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:37.105+0000 7feb3f7fe700 1 -- 172.21.15.39:0/792163655 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb401056e0 msgr2=0x7feb40198f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:37.133 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:37.105+0000 7feb3f7fe700 1 --2- 172.21.15.39:0/792163655 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb401056e0 0x7feb40198f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:37.133 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:37.105+0000 7feb3f7fe700 1 -- 172.21.15.39:0/792163655 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb300095f0 con 0x7feb40102c10 2023-07-21T18:41:37.133 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:37.105+0000 7feb44af0700 1 --2- 172.21.15.39:0/792163655 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb401056e0 0x7feb40198f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:41:37.133 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:37.105+0000 7feb3ffff700 1 --2- 172.21.15.39:0/792163655 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb40068600 0x7feb40197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:41:37.133 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:37.106+0000 7feb3f7fe700 1 --2- 172.21.15.39:0/792163655 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb40102c10 0x7feb40198100 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7feb34000c00 tx=0x7feb3400b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:37.133 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:37.106+0000 7feb3d7fa700 1 -- 172.21.15.39:0/792163655 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3401c070 con 0x7feb40102c10 2023-07-21T18:41:37.134 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:37.106+0000 7feb46553700 1 -- 172.21.15.39:0/792163655 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb34009520 con 0x7feb40102c10 2023-07-21T18:41:37.134 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:37.106+0000 7feb3d7fa700 1 -- 172.21.15.39:0/792163655 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb3400f7d0 con 0x7feb40102c10 2023-07-21T18:41:37.134 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:37.106+0000 7feb3d7fa700 1 -- 172.21.15.39:0/792163655 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb34003990 con 0x7feb40102c10 2023-07-21T18:41:37.134 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:37.106+0000 7feb46553700 1 -- 172.21.15.39:0/792163655 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb401a01d0 con 0x7feb40102c10 2023-07-21T18:41:37.134 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:37.107+0000 7feb46553700 1 -- 172.21.15.39:0/792163655 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb400ffc30 con 0x7feb40102c10 2023-07-21T18:41:37.135 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:37.108+0000 7feb3d7fa700 1 -- 172.21.15.39:0/792163655 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 18) v1 ==== 135783+0+0 (secure 0 0 0) 0x7feb34020030 con 0x7feb40102c10 2023-07-21T18:41:37.135 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:37.109+0000 7feb3d7fa700 1 --2- 172.21.15.39:0/792163655 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7feb280a1230 0x7feb280a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:37.135 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:37.109+0000 7feb3ffff700 1 -- 172.21.15.39:0/792163655 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7feb280a1230 msgr2=0x7feb280a36e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/2116291712 2023-07-21T18:41:37.135 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:37.109+0000 7feb3d7fa700 1 -- 172.21.15.39:0/792163655 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8182+0+0 (secure 0 0 0) 0x7feb340cde20 con 0x7feb40102c10 2023-07-21T18:41:37.136 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:37.109+0000 7feb3ffff700 1 --2- 172.21.15.39:0/792163655 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7feb280a1230 0x7feb280a36e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-21T18:41:37.139 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:37.116+0000 7feb3d7fa700 1 -- 172.21.15.39:0/792163655 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7feb340974d0 con 0x7feb40102c10 2023-07-21T18:41:37.202 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:41:36 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:41:36.920+0000 7f93cfe50200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-21T18:41:37.333 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:37.310+0000 7feb3ffff700 1 -- 172.21.15.39:0/792163655 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7feb280a1230 msgr2=0x7feb280a36e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/2116291712 2023-07-21T18:41:37.333 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:37.310+0000 7feb3ffff700 1 --2- 172.21.15.39:0/792163655 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7feb280a1230 0x7feb280a36e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-21T18:41:37.366 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:37.343+0000 7feb46553700 1 -- 172.21.15.39:0/792163655 --> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7feb40060950 con 0x7feb280a1230 2023-07-21T18:41:37.370 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Jul 21 18:41:37 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-z[100072]: 2023-07-21T18:41:37.189+0000 7ff0a29cc200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-21T18:41:37.370 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Jul 21 18:41:37 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-z[100072]: 2023-07-21T18:41:37.319+0000 7ff0a29cc200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-21T18:41:37.370 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:37 smithi193 ceph-mon[98204]: scrub queued for path: / 2023-07-21T18:41:37.371 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:37 smithi193 ceph-mon[98204]: scrub summary: idle+waiting paths [/] 2023-07-21T18:41:37.371 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:37 smithi193 ceph-mon[98204]: scrub summary: active paths [/] 2023-07-21T18:41:37.371 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:37 smithi193 ceph-mon[98204]: scrub summary: idle+waiting paths [/] 2023-07-21T18:41:37.371 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:37 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/2880058095' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-07-21T18:41:37.371 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:37 smithi193 ceph-mon[98204]: mgrmap e18: x(active, since 7m), standbys: z, y 2023-07-21T18:41:37.371 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:37 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/262606771' entity='client.admin' 2023-07-21T18:41:37.371 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:37 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/2756222737' entity='client.admin' 2023-07-21T18:41:37.389 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 21 18:41:36 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-y[99389]: 2023-07-21T18:41:36.782+0000 7fa3efc74200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-21T18:41:37.389 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 21 18:41:37 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-y[99389]: 2023-07-21T18:41:37.091+0000 7fa3efc74200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-21T18:41:37.389 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 21 18:41:37 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-y[99389]: 2023-07-21T18:41:37.182+0000 7fa3efc74200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-21T18:41:37.390 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 21 18:41:37 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-y[99389]: 2023-07-21T18:41:37.312+0000 7fa3efc74200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-21T18:41:37.390 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:37 smithi155 ceph-mon[97934]: scrub queued for path: / 2023-07-21T18:41:37.390 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:37 smithi155 ceph-mon[97934]: scrub summary: idle+waiting paths [/] 2023-07-21T18:41:37.390 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:37 smithi155 ceph-mon[97934]: scrub summary: active paths [/] 2023-07-21T18:41:37.390 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:37 smithi155 ceph-mon[97934]: scrub summary: idle+waiting paths [/] 2023-07-21T18:41:37.390 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:37 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/2880058095' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-07-21T18:41:37.391 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:37 smithi155 ceph-mon[97934]: mgrmap e18: x(active, since 7m), standbys: z, y 2023-07-21T18:41:37.391 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:37 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/262606771' entity='client.admin' 2023-07-21T18:41:37.391 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:37 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/2756222737' entity='client.admin' 2023-07-21T18:41:37.460 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:41:37 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:41:37.327+0000 7f93cfe50200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-21T18:41:37.460 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:37 smithi039 ceph-mon[96489]: scrub queued for path: / 2023-07-21T18:41:37.461 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:37 smithi039 ceph-mon[96489]: scrub summary: idle+waiting paths [/] 2023-07-21T18:41:37.461 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:37 smithi039 ceph-mon[96489]: scrub summary: active paths [/] 2023-07-21T18:41:37.461 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:37 smithi039 ceph-mon[96489]: scrub summary: idle+waiting paths [/] 2023-07-21T18:41:37.461 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:37 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/2880058095' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-07-21T18:41:37.461 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:37 smithi039 ceph-mon[96489]: mgrmap e18: x(active, since 7m), standbys: z, y 2023-07-21T18:41:37.461 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:37 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/262606771' entity='client.admin' 2023-07-21T18:41:37.461 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:37 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/2756222737' entity='client.admin' 2023-07-21T18:41:37.728 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:41:37 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:41:37.436+0000 7f93cfe50200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-21T18:41:37.728 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:41:37 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:41:37.588+0000 7f93cfe50200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-21T18:41:37.735 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:37.711+0000 7feb3ffff700 1 -- 172.21.15.39:0/792163655 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7feb280a1230 msgr2=0x7feb280a36e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/2116291712 2023-07-21T18:41:37.735 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:37.711+0000 7feb3ffff700 1 --2- 172.21.15.39:0/792163655 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7feb280a1230 0x7feb280a36e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-07-21T18:41:37.849 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 21 18:41:37 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-y[99389]: 2023-07-21T18:41:37.407+0000 7fa3efc74200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-21T18:41:37.855 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Jul 21 18:41:37 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-z[100072]: 2023-07-21T18:41:37.414+0000 7ff0a29cc200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-21T18:41:38.120 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Jul 21 18:41:37 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-z[100072]: 2023-07-21T18:41:37.832+0000 7ff0a29cc200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-21T18:41:38.139 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 21 18:41:37 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-y[99389]: 2023-07-21T18:41:37.826+0000 7fa3efc74200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-21T18:41:38.188 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:41:37 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:41:37.704+0000 7f93cfe50200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-21T18:41:38.535 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:38.512+0000 7feb3ffff700 1 -- 172.21.15.39:0/792163655 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7feb280a1230 msgr2=0x7feb280a36e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/2116291712 2023-07-21T18:41:38.535 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:38.512+0000 7feb3ffff700 1 --2- 172.21.15.39:0/792163655 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7feb280a1230 0x7feb280a36e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-07-21T18:41:38.688 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:38 smithi039 ceph-mon[96489]: scrub summary: idle 2023-07-21T18:41:38.688 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:41:38 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:41:38.214+0000 7f93cfe50200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-21T18:41:38.870 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:38 smithi193 ceph-mon[98204]: scrub summary: idle 2023-07-21T18:41:38.870 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Jul 21 18:41:38 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-z[100072]: 2023-07-21T18:41:38.396+0000 7ff0a29cc200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-21T18:41:38.871 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Jul 21 18:41:38 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-z[100072]: 2023-07-21T18:41:38.567+0000 7ff0a29cc200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-21T18:41:38.889 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:38 smithi155 ceph-mon[97934]: scrub summary: idle 2023-07-21T18:41:38.889 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 21 18:41:38 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-y[99389]: 2023-07-21T18:41:38.392+0000 7fa3efc74200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-21T18:41:38.890 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 21 18:41:38 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-y[99389]: 2023-07-21T18:41:38.562+0000 7fa3efc74200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-21T18:41:39.188 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:41:38 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:41:38.896+0000 7f93cfe50200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-21T18:41:39.188 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:41:39 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:41:39.089+0000 7f93cfe50200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-21T18:41:39.607 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 21 18:41:39 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-y[99389]: 2023-07-21T18:41:39.324+0000 7fa3efc74200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-21T18:41:39.607 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 21 18:41:39 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-y[99389]: 2023-07-21T18:41:39.415+0000 7fa3efc74200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-21T18:41:39.609 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Jul 21 18:41:39 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-z[100072]: 2023-07-21T18:41:39.327+0000 7ff0a29cc200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-21T18:41:39.610 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Jul 21 18:41:39 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-z[100072]: 2023-07-21T18:41:39.418+0000 7ff0a29cc200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-21T18:41:39.883 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 21 18:41:39 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-y[99389]: 2023-07-21T18:41:39.584+0000 7fa3efc74200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-21T18:41:40.137 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:40.113+0000 7feb3ffff700 1 -- 172.21.15.39:0/792163655 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7feb280a1230 msgr2=0x7feb280a36e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/2116291712 2023-07-21T18:41:40.137 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:40.113+0000 7feb3ffff700 1 --2- 172.21.15.39:0/792163655 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7feb280a1230 0x7feb280a36e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-07-21T18:41:40.139 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 21 18:41:39 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-y[99389]: 2023-07-21T18:41:39.860+0000 7fa3efc74200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-21T18:41:40.139 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 21 18:41:40 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-y[99389]: 2023-07-21T18:41:40.087+0000 7fa3efc74200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-21T18:41:40.262 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Jul 21 18:41:39 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-z[100072]: 2023-07-21T18:41:39.586+0000 7ff0a29cc200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-21T18:41:40.262 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Jul 21 18:41:39 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-z[100072]: 2023-07-21T18:41:39.862+0000 7ff0a29cc200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-21T18:41:40.262 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Jul 21 18:41:40 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-z[100072]: 2023-07-21T18:41:40.089+0000 7ff0a29cc200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-21T18:41:40.414 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:41:40 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:41:40.057+0000 7f93cfe50200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-21T18:41:40.414 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:41:40 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:41:40.169+0000 7f93cfe50200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-21T18:41:40.519 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 21 18:41:40 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-y[99389]: 2023-07-21T18:41:40.237+0000 7fa3efc74200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-21T18:41:40.688 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:41:40 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:41:40.390+0000 7f93cfe50200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-21T18:41:40.870 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Jul 21 18:41:40 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-z[100072]: 2023-07-21T18:41:40.239+0000 7ff0a29cc200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-21T18:41:40.870 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Jul 21 18:41:40 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-z[100072]: 2023-07-21T18:41:40.498+0000 7ff0a29cc200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-21T18:41:40.889 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 21 18:41:40 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-y[99389]: 2023-07-21T18:41:40.497+0000 7fa3efc74200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-21T18:41:41.021 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:41:40 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:41:40.742+0000 7f93cfe50200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-21T18:41:41.438 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:41:40 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:41:40.998+0000 7f93cfe50200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-21T18:41:41.438 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:41:41 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:41:41.181+0000 7f93cfe50200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-21T18:41:41.540 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 21 18:41:41 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-y[99389]: 2023-07-21T18:41:41.199+0000 7fa3efc74200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-21T18:41:41.540 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 21 18:41:41 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-y[99389]: 2023-07-21T18:41:41.330+0000 7fa3efc74200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-21T18:41:41.545 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Jul 21 18:41:41 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-z[100072]: 2023-07-21T18:41:41.202+0000 7ff0a29cc200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-21T18:41:41.545 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Jul 21 18:41:41 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-z[100072]: 2023-07-21T18:41:41.334+0000 7ff0a29cc200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-21T18:41:41.865 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 21 18:41:41 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-y[99389]: 2023-07-21T18:41:41.517+0000 7fa3efc74200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-21T18:41:41.865 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 21 18:41:41 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-y[99389]: 2023-07-21T18:41:41.630+0000 7fa3efc74200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-21T18:41:41.870 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Jul 21 18:41:41 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-z[100072]: 2023-07-21T18:41:41.522+0000 7ff0a29cc200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-21T18:41:41.870 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Jul 21 18:41:41 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-z[100072]: 2023-07-21T18:41:41.636+0000 7ff0a29cc200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-21T18:41:41.938 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:41:41 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:41:41.514+0000 7f93cfe50200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-21T18:41:42.370 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Jul 21 18:41:41 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-z[100072]: 2023-07-21T18:41:41.851+0000 7ff0a29cc200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-21T18:41:42.688 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:41:42 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:41:42.339+0000 7f93cfe50200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-21T18:41:42.688 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:41:42 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:41:42.499+0000 7f93cfe50200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-21T18:41:42.796 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:42.773+0000 7feb3d7fa700 1 -- 172.21.15.39:0/792163655 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mgrmap(e 19) v1 ==== 135783+0+0 (secure 0 0 0) 0x7feb340978a0 con 0x7feb40102c10 2023-07-21T18:41:43.120 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Jul 21 18:41:42 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-z[100072]: 2023-07-21T18:41:42.703+0000 7ff0a29cc200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-21T18:41:43.121 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:42 smithi193 ceph-mon[98204]: Standby manager daemon y restarted 2023-07-21T18:41:43.121 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:42 smithi193 ceph-mon[98204]: Standby manager daemon y started 2023-07-21T18:41:43.121 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:42 smithi193 ceph-mon[98204]: from='mgr.? 172.21.15.155:0/576128027' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-07-21T18:41:43.121 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:42 smithi193 ceph-mon[98204]: from='mgr.? 172.21.15.155:0/576128027' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-21T18:41:43.121 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:42 smithi193 ceph-mon[98204]: Standby manager daemon z restarted 2023-07-21T18:41:43.121 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:42 smithi193 ceph-mon[98204]: Standby manager daemon z started 2023-07-21T18:41:43.121 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:42 smithi193 ceph-mon[98204]: from='mgr.? 172.21.15.155:0/576128027' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-07-21T18:41:43.122 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:42 smithi193 ceph-mon[98204]: from='mgr.? 172.21.15.155:0/576128027' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-21T18:41:43.122 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:42 smithi193 ceph-mon[98204]: from='mgr.? 172.21.15.193:0/2951439463' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-07-21T18:41:43.122 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:42 smithi193 ceph-mon[98204]: from='mgr.? 172.21.15.193:0/2951439463' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-21T18:41:43.122 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:42 smithi193 ceph-mon[98204]: from='mgr.? 172.21.15.193:0/2951439463' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-07-21T18:41:43.122 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:42 smithi193 ceph-mon[98204]: from='mgr.? 172.21.15.193:0/2951439463' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-21T18:41:43.139 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 21 18:41:41 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-y[99389]: 2023-07-21T18:41:41.843+0000 7fa3efc74200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-21T18:41:43.139 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 21 18:41:42 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-y[99389]: 2023-07-21T18:41:42.700+0000 7fa3efc74200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-21T18:41:43.139 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:42 smithi155 ceph-mon[97934]: Standby manager daemon y restarted 2023-07-21T18:41:43.139 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:42 smithi155 ceph-mon[97934]: Standby manager daemon y started 2023-07-21T18:41:43.140 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:42 smithi155 ceph-mon[97934]: from='mgr.? 172.21.15.155:0/576128027' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-07-21T18:41:43.140 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:42 smithi155 ceph-mon[97934]: from='mgr.? 172.21.15.155:0/576128027' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-21T18:41:43.140 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:42 smithi155 ceph-mon[97934]: Standby manager daemon z restarted 2023-07-21T18:41:43.140 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:42 smithi155 ceph-mon[97934]: Standby manager daemon z started 2023-07-21T18:41:43.140 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:42 smithi155 ceph-mon[97934]: from='mgr.? 172.21.15.155:0/576128027' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-07-21T18:41:43.140 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:42 smithi155 ceph-mon[97934]: from='mgr.? 172.21.15.155:0/576128027' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-21T18:41:43.140 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:42 smithi155 ceph-mon[97934]: from='mgr.? 172.21.15.193:0/2951439463' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-07-21T18:41:43.141 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:42 smithi155 ceph-mon[97934]: from='mgr.? 172.21.15.193:0/2951439463' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-21T18:41:43.141 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:42 smithi155 ceph-mon[97934]: from='mgr.? 172.21.15.193:0/2951439463' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-07-21T18:41:43.141 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:42 smithi155 ceph-mon[97934]: from='mgr.? 172.21.15.193:0/2951439463' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-21T18:41:43.188 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:41:42 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:41:42.748+0000 7f93cfe50200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-21T18:41:43.188 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:41:42 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:41:42.891+0000 7f93cfe50200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-21T18:41:43.188 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:42 smithi039 ceph-mon[96489]: Standby manager daemon y restarted 2023-07-21T18:41:43.188 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:42 smithi039 ceph-mon[96489]: Standby manager daemon y started 2023-07-21T18:41:43.188 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:42 smithi039 ceph-mon[96489]: from='mgr.? 172.21.15.155:0/576128027' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-07-21T18:41:43.189 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:42 smithi039 ceph-mon[96489]: from='mgr.? 172.21.15.155:0/576128027' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-21T18:41:43.189 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:42 smithi039 ceph-mon[96489]: Standby manager daemon z restarted 2023-07-21T18:41:43.189 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:42 smithi039 ceph-mon[96489]: Standby manager daemon z started 2023-07-21T18:41:43.189 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:42 smithi039 ceph-mon[96489]: from='mgr.? 172.21.15.155:0/576128027' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-07-21T18:41:43.189 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:42 smithi039 ceph-mon[96489]: from='mgr.? 172.21.15.155:0/576128027' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-21T18:41:43.189 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:42 smithi039 ceph-mon[96489]: from='mgr.? 172.21.15.193:0/2951439463' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-07-21T18:41:43.189 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:42 smithi039 ceph-mon[96489]: from='mgr.? 172.21.15.193:0/2951439463' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-21T18:41:43.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:42 smithi039 ceph-mon[96489]: from='mgr.? 172.21.15.193:0/2951439463' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-07-21T18:41:43.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:42 smithi039 ceph-mon[96489]: from='mgr.? 172.21.15.193:0/2951439463' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-21T18:41:43.339 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:43.316+0000 7feb3ffff700 1 -- 172.21.15.39:0/792163655 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7feb280a1230 msgr2=0x7feb280a36e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/2116291712 2023-07-21T18:41:43.339 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:43.316+0000 7feb3ffff700 1 --2- 172.21.15.39:0/792163655 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7feb280a1230 0x7feb280a36e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-07-21T18:41:43.688 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:41:43 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:41:43.169+0000 7f93cfe50200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-21T18:41:44.120 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:43 smithi193 ceph-mon[98204]: mgrmap e19: x(active, since 7m), standbys: y, z 2023-07-21T18:41:44.139 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:43 smithi155 ceph-mon[97934]: mgrmap e19: x(active, since 7m), standbys: y, z 2023-07-21T18:41:44.187 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:43 smithi039 ceph-mon[96489]: mgrmap e19: x(active, since 7m), standbys: y, z 2023-07-21T18:41:44.271 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:44.247+0000 7feb3d7fa700 1 -- 172.21.15.39:0/792163655 <== mon.1 v2:172.21.15.193:3300/0 8 ==== mgrmap(e 20) v1 ==== 135400+0+0 (secure 0 0 0) 0x7feb340a39e0 con 0x7feb40102c10 2023-07-21T18:41:44.271 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:44.248+0000 7feb3d7fa700 1 -- 172.21.15.39:0/792163655 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7feb280a1230 msgr2=0x7feb280a36e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-21T18:41:44.272 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:44.248+0000 7feb3d7fa700 1 --2- 172.21.15.39:0/792163655 >> [v2:172.21.15.39:6800/2116291712,v1:172.21.15.39:6801/2116291712] conn(0x7feb280a1230 0x7feb280a36e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:44.497 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:41:44 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:41:44.216+0000 7f93cfe50200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-21T18:41:44.497 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:41:44 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:41:44.473+0000 7f932f3af700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T18:41:44.498 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:41:44 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:41:44.473+0000 7f932f3af700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T18:41:44.802 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:41:44 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:41:44.473+0000 7f932f3af700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T18:41:44.802 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:41:44 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:41:44.473+0000 7f932f3af700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T18:41:44.802 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:41:44 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:41:44.473+0000 7f932f3af700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T18:41:44.802 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:41:44 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:41:44.475+0000 7f932b3a7700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T18:41:44.802 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:41:44 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:41:44.475+0000 7f932b3a7700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T18:41:44.803 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:41:44 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:41:44.475+0000 7f932b3a7700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T18:41:44.803 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:41:44 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:41:44.475+0000 7f932b3a7700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T18:41:44.803 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:41:44 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:41:44.475+0000 7f932b3a7700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T18:41:45.121 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:44 smithi193 ceph-mon[98204]: Active manager daemon x restarted 2023-07-21T18:41:45.121 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:44 smithi193 ceph-mon[98204]: Activating manager daemon x 2023-07-21T18:41:45.121 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:44 smithi193 ceph-mon[98204]: osdmap e82: 12 total, 12 up, 12 in 2023-07-21T18:41:45.122 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:44 smithi193 ceph-mon[98204]: mgrmap e20: x(active, starting, since 0.0185895s), standbys: y, z 2023-07-21T18:41:45.122 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:44 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T18:41:45.122 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:44 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T18:41:45.122 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:44 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T18:41:45.123 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:44 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-07-21T18:41:45.123 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:44 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-07-21T18:41:45.123 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:44 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-21T18:41:45.123 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:44 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-21T18:41:45.124 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:44 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-21T18:41:45.124 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:44 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-21T18:41:45.124 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:44 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-21T18:41:45.124 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:44 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-21T18:41:45.125 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:44 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-21T18:41:45.125 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:44 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-21T18:41:45.125 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:44 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-21T18:41:45.125 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:44 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-21T18:41:45.126 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:44 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-07-21T18:41:45.126 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:44 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-07-21T18:41:45.126 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:44 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-07-21T18:41:45.126 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:44 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T18:41:45.127 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:44 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T18:41:45.127 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:44 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T18:41:45.127 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:44 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T18:41:45.127 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:44 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T18:41:45.128 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:44 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T18:41:45.128 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:44 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T18:41:45.128 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:44 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T18:41:45.129 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:44 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T18:41:45.129 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:44 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T18:41:45.129 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:44 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T18:41:45.129 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:44 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T18:41:45.130 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:44 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-07-21T18:41:45.130 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:44 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-07-21T18:41:45.130 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:44 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-07-21T18:41:45.130 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:44 smithi193 ceph-mon[98204]: Manager daemon x is now available 2023-07-21T18:41:45.131 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:44 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:41:45.131 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:44 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:41:45.131 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:44 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-21T18:41:45.131 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:44 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-21T18:41:45.132 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:44 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-21T18:41:45.132 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:44 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-21T18:41:45.139 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:44 smithi155 ceph-mon[97934]: Active manager daemon x restarted 2023-07-21T18:41:45.139 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:44 smithi155 ceph-mon[97934]: Activating manager daemon x 2023-07-21T18:41:45.140 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:44 smithi155 ceph-mon[97934]: osdmap e82: 12 total, 12 up, 12 in 2023-07-21T18:41:45.140 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:44 smithi155 ceph-mon[97934]: mgrmap e20: x(active, starting, since 0.0185895s), standbys: y, z 2023-07-21T18:41:45.140 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:44 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T18:41:45.141 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:44 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T18:41:45.141 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:44 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T18:41:45.141 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:44 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-07-21T18:41:45.141 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:44 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-07-21T18:41:45.142 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:44 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-21T18:41:45.142 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:44 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-21T18:41:45.142 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:44 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-21T18:41:45.142 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:44 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-21T18:41:45.143 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:44 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-21T18:41:45.143 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:44 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-21T18:41:45.143 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:44 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-21T18:41:45.143 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:44 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-21T18:41:45.144 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:44 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-21T18:41:45.144 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:44 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-21T18:41:45.144 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:44 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-07-21T18:41:45.144 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:44 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-07-21T18:41:45.145 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:44 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-07-21T18:41:45.145 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:44 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T18:41:45.145 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:44 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T18:41:45.145 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:44 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T18:41:45.146 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:44 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T18:41:45.146 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:44 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T18:41:45.146 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:44 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T18:41:45.146 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:44 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T18:41:45.147 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:44 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T18:41:45.147 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:44 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T18:41:45.147 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:44 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T18:41:45.147 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:44 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T18:41:45.148 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:44 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T18:41:45.148 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:44 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-07-21T18:41:45.148 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:44 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-07-21T18:41:45.148 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:44 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-07-21T18:41:45.149 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:44 smithi155 ceph-mon[97934]: Manager daemon x is now available 2023-07-21T18:41:45.149 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:44 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:41:45.149 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:44 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:41:45.149 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:44 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-21T18:41:45.150 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:44 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-21T18:41:45.150 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:44 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-21T18:41:45.150 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:44 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-21T18:41:45.188 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:44 smithi039 ceph-mon[96489]: Active manager daemon x restarted 2023-07-21T18:41:45.189 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:44 smithi039 ceph-mon[96489]: Activating manager daemon x 2023-07-21T18:41:45.189 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:44 smithi039 ceph-mon[96489]: osdmap e82: 12 total, 12 up, 12 in 2023-07-21T18:41:45.189 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:44 smithi039 ceph-mon[96489]: mgrmap e20: x(active, starting, since 0.0185895s), standbys: y, z 2023-07-21T18:41:45.189 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:44 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T18:41:45.189 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:44 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T18:41:45.189 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:44 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T18:41:45.189 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:44 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-07-21T18:41:45.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:44 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-07-21T18:41:45.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:44 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-21T18:41:45.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:44 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-21T18:41:45.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:44 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-21T18:41:45.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:44 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-21T18:41:45.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:44 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-21T18:41:45.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:44 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-21T18:41:45.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:44 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-21T18:41:45.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:44 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-21T18:41:45.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:44 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-21T18:41:45.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:44 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-21T18:41:45.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:44 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-07-21T18:41:45.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:44 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-07-21T18:41:45.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:44 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-07-21T18:41:45.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:44 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T18:41:45.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:44 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T18:41:45.192 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:44 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T18:41:45.192 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:44 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T18:41:45.192 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:44 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T18:41:45.192 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:44 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T18:41:45.192 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:44 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T18:41:45.192 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:44 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T18:41:45.192 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:44 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T18:41:45.192 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:44 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T18:41:45.193 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:44 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T18:41:45.193 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:44 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T18:41:45.193 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:44 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-07-21T18:41:45.193 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:44 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-07-21T18:41:45.193 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:44 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-07-21T18:41:45.193 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:44 smithi039 ceph-mon[96489]: Manager daemon x is now available 2023-07-21T18:41:45.193 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:44 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:41:45.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:44 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:41:45.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:44 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-21T18:41:45.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:44 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-21T18:41:45.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:44 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-21T18:41:45.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:44 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-21T18:41:45.274 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.250+0000 7feb3d7fa700 1 -- 172.21.15.39:0/792163655 <== mon.1 v2:172.21.15.193:3300/0 9 ==== mgrmap(e 21) v1 ==== 135527+0+0 (secure 0 0 0) 0x7feb34066690 con 0x7feb40102c10 2023-07-21T18:41:45.274 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.251+0000 7feb3d7fa700 1 --2- 172.21.15.39:0/792163655 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7feb2801b8f0 0x7feb2801bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:45.275 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.251+0000 7feb3d7fa700 1 -- 172.21.15.39:0/792163655 --> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7feb40060950 con 0x7feb2801b8f0 2023-07-21T18:41:45.275 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.252+0000 7feb3ffff700 1 --2- 172.21.15.39:0/792163655 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7feb2801b8f0 0x7feb2801bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:45.275 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.252+0000 7feb3ffff700 1 --2- 172.21.15.39:0/792163655 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7feb2801b8f0 0x7feb2801bcc0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7feb40068320 tx=0x7feb30009310 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:45.339 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.315+0000 7feb3d7fa700 1 -- 172.21.15.39:0/792163655 <== mgr.15471 v2:172.21.15.39:6800/13616514 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7feb40060950 con 0x7feb2801b8f0 2023-07-21T18:41:45.339 INFO:teuthology.orchestra.run.smithi039.stdout:Schedule set for path / 2023-07-21T18:41:45.341 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.317+0000 7feb46553700 1 -- 172.21.15.39:0/792163655 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7feb2801b8f0 msgr2=0x7feb2801bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:45.341 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.317+0000 7feb46553700 1 --2- 172.21.15.39:0/792163655 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7feb2801b8f0 0x7feb2801bcc0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7feb40068320 tx=0x7feb30009310 comp rx=0 tx=0).stop 2023-07-21T18:41:45.341 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.317+0000 7feb46553700 1 -- 172.21.15.39:0/792163655 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb40102c10 msgr2=0x7feb40198100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:45.341 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.317+0000 7feb46553700 1 --2- 172.21.15.39:0/792163655 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb40102c10 0x7feb40198100 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7feb34000c00 tx=0x7feb3400b790 comp rx=0 tx=0).stop 2023-07-21T18:41:45.341 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.317+0000 7feb44af0700 1 -- 172.21.15.39:0/792163655 reap_dead start 2023-07-21T18:41:45.341 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.317+0000 7feb46553700 1 -- 172.21.15.39:0/792163655 shutdown_connections 2023-07-21T18:41:45.342 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.317+0000 7feb46553700 1 -- 172.21.15.39:0/792163655 >> 172.21.15.39:0/792163655 conn(0x7feb400fa000 msgr2=0x7feb400fc3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:45.342 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.318+0000 7feb46553700 1 -- 172.21.15.39:0/792163655 shutdown_connections 2023-07-21T18:41:45.342 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.318+0000 7feb46553700 1 -- 172.21.15.39:0/792163655 wait complete. 2023-07-21T18:41:45.353 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6M3h' 2023-07-21T18:41:45.494 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.466+0000 7fd284805700 1 -- 172.21.15.39:0/2499263764 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd2800fe180 msgr2=0x7fd2800fe590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:45.495 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.466+0000 7fd284805700 1 --2- 172.21.15.39:0/2499263764 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd2800fe180 0x7fd2800fe590 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fd270009990 tx=0x7fd270009ca0 comp rx=0 tx=0).stop 2023-07-21T18:41:45.495 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.466+0000 7fd284805700 1 -- 172.21.15.39:0/2499263764 shutdown_connections 2023-07-21T18:41:45.495 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.466+0000 7fd284805700 1 --2- 172.21.15.39:0/2499263764 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2800ffd00 0x7fd2801065c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:45.495 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.466+0000 7fd284805700 1 --2- 172.21.15.39:0/2499263764 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd2800fe180 0x7fd2800fe590 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:45.495 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.466+0000 7fd284805700 1 --2- 172.21.15.39:0/2499263764 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2800ff370 0x7fd2800ff7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:45.495 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.466+0000 7fd284805700 1 -- 172.21.15.39:0/2499263764 >> 172.21.15.39:0/2499263764 conn(0x7fd2800f9780 msgr2=0x7fd2800fbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:45.496 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.466+0000 7fd284805700 1 -- 172.21.15.39:0/2499263764 shutdown_connections 2023-07-21T18:41:45.496 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.466+0000 7fd284805700 1 -- 172.21.15.39:0/2499263764 wait complete. 2023-07-21T18:41:45.496 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.467+0000 7fd284805700 1 Processor -- start 2023-07-21T18:41:45.496 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.467+0000 7fd284805700 1 -- start start 2023-07-21T18:41:45.496 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.467+0000 7fd284805700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd2800fe180 0x7fd280197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:45.496 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.467+0000 7fd284805700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2800ff370 0x7fd280198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:45.496 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.467+0000 7fd284805700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2800ffd00 0x7fd280198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:45.496 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.467+0000 7fd284805700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd280199550 con 0x7fd2800fe180 2023-07-21T18:41:45.497 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.467+0000 7fd284805700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2801996c0 con 0x7fd2800ff370 2023-07-21T18:41:45.497 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.467+0000 7fd284805700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd280199880 con 0x7fd2800ffd00 2023-07-21T18:41:45.497 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.467+0000 7fd27dd9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2800ff370 0x7fd280198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:45.497 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.467+0000 7fd27dd9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2800ff370 0x7fd280198070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:41782/0 (socket says 172.21.15.39:41782) 2023-07-21T18:41:45.497 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.467+0000 7fd27dd9b700 1 -- 172.21.15.39:0/817709206 learned_addr learned my addr 172.21.15.39:0/817709206 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:45.497 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.467+0000 7fd27ed9d700 1 --2- 172.21.15.39:0/817709206 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2800ffd00 0x7fd280198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:45.497 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.468+0000 7fd27e59c700 1 --2- 172.21.15.39:0/817709206 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd2800fe180 0x7fd280197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:45.498 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.468+0000 7fd27dd9b700 1 -- 172.21.15.39:0/817709206 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2800ffd00 msgr2=0x7fd280198ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:45.498 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.468+0000 7fd27dd9b700 1 --2- 172.21.15.39:0/817709206 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2800ffd00 0x7fd280198ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:45.498 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.468+0000 7fd27dd9b700 1 -- 172.21.15.39:0/817709206 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd2800fe180 msgr2=0x7fd280197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:45.498 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.468+0000 7fd27dd9b700 1 --2- 172.21.15.39:0/817709206 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd2800fe180 0x7fd280197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:45.498 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.468+0000 7fd27dd9b700 1 -- 172.21.15.39:0/817709206 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2700095f0 con 0x7fd2800ff370 2023-07-21T18:41:45.498 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.468+0000 7fd27e59c700 1 --2- 172.21.15.39:0/817709206 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd2800fe180 0x7fd280197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:41:45.498 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.468+0000 7fd27dd9b700 1 --2- 172.21.15.39:0/817709206 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2800ff370 0x7fd280198070 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd27400d980 tx=0x7fd27400dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:45.498 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.468+0000 7fd26f7fe700 1 -- 172.21.15.39:0/817709206 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd274004220 con 0x7fd2800ff370 2023-07-21T18:41:45.499 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.468+0000 7fd284805700 1 -- 172.21.15.39:0/817709206 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd280199b60 con 0x7fd2800ff370 2023-07-21T18:41:45.499 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.468+0000 7fd26f7fe700 1 -- 172.21.15.39:0/817709206 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd274007640 con 0x7fd2800ff370 2023-07-21T18:41:45.499 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.468+0000 7fd26f7fe700 1 -- 172.21.15.39:0/817709206 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd274005610 con 0x7fd2800ff370 2023-07-21T18:41:45.499 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.468+0000 7fd284805700 1 -- 172.21.15.39:0/817709206 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2801a0320 con 0x7fd2800ff370 2023-07-21T18:41:45.499 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.469+0000 7fd284805700 1 -- 172.21.15.39:0/817709206 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd28004e2f0 con 0x7fd2800ff370 2023-07-21T18:41:45.499 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.470+0000 7fd26f7fe700 1 -- 172.21.15.39:0/817709206 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 21) v1 ==== 135527+0+0 (secure 0 0 0) 0x7fd274004750 con 0x7fd2800ff370 2023-07-21T18:41:45.499 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.471+0000 7fd26f7fe700 1 --2- 172.21.15.39:0/817709206 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fd2680a0f50 0x7fd2680a3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:45.499 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.471+0000 7fd27e59c700 1 --2- 172.21.15.39:0/817709206 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fd2680a0f50 0x7fd2680a3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:45.500 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.471+0000 7fd26f7fe700 1 -- 172.21.15.39:0/817709206 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fd2740cc7d0 con 0x7fd2800ff370 2023-07-21T18:41:45.500 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.471+0000 7fd27e59c700 1 --2- 172.21.15.39:0/817709206 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fd2680a0f50 0x7fd2680a3400 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd27000b2e0 tx=0x7fd270005fd0 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:45.500 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.475+0000 7fd26f7fe700 1 -- 172.21.15.39:0/817709206 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fd2740960d0 con 0x7fd2800ff370 2023-07-21T18:41:45.728 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.705+0000 7fd284805700 1 -- 172.21.15.39:0/817709206 --> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}) v1 -- 0x7fd2801076a0 con 0x7fd2680a0f50 2023-07-21T18:41:45.787 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.763+0000 7fd26f7fe700 1 -- 172.21.15.39:0/817709206 <== mgr.15471 v2:172.21.15.39:6800/13616514 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7fd2801076a0 con 0x7fd2680a0f50 2023-07-21T18:41:45.787 INFO:teuthology.orchestra.run.smithi039.stdout:Retention added to path / 2023-07-21T18:41:45.789 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.766+0000 7fd284805700 1 -- 172.21.15.39:0/817709206 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fd2680a0f50 msgr2=0x7fd2680a3400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:45.789 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.766+0000 7fd284805700 1 --2- 172.21.15.39:0/817709206 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fd2680a0f50 0x7fd2680a3400 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd27000b2e0 tx=0x7fd270005fd0 comp rx=0 tx=0).stop 2023-07-21T18:41:45.789 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.766+0000 7fd284805700 1 -- 172.21.15.39:0/817709206 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2800ff370 msgr2=0x7fd280198070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:45.790 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.766+0000 7fd284805700 1 --2- 172.21.15.39:0/817709206 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2800ff370 0x7fd280198070 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd27400d980 tx=0x7fd27400dc90 comp rx=0 tx=0).stop 2023-07-21T18:41:45.790 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.766+0000 7fd284805700 1 -- 172.21.15.39:0/817709206 shutdown_connections 2023-07-21T18:41:45.790 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.766+0000 7fd284805700 1 --2- 172.21.15.39:0/817709206 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2800ffd00 0x7fd280198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:45.790 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.766+0000 7fd284805700 1 --2- 172.21.15.39:0/817709206 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd2800fe180 0x7fd280197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:45.790 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.766+0000 7fd284805700 1 --2- 172.21.15.39:0/817709206 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fd2680a0f50 0x7fd2680a3400 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:45.790 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.766+0000 7fd284805700 1 --2- 172.21.15.39:0/817709206 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2800ff370 0x7fd280198070 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:45.790 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.766+0000 7fd284805700 1 -- 172.21.15.39:0/817709206 >> 172.21.15.39:0/817709206 conn(0x7fd2800f9780 msgr2=0x7fd280106f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:45.791 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.766+0000 7fd284805700 1 -- 172.21.15.39:0/817709206 shutdown_connections 2023-07-21T18:41:45.791 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.766+0000 7fd284805700 1 -- 172.21.15.39:0/817709206 wait complete. 2023-07-21T18:41:45.814 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2023-07-21T18:41:45.949 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.925+0000 7f2a67291700 1 -- 172.21.15.39:0/4208562743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a601058a0 msgr2=0x7f2a60107c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:45.949 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.925+0000 7f2a67291700 1 --2- 172.21.15.39:0/4208562743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a601058a0 0x7f2a60107c80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2a58009ec0 tx=0x7f2a5800b330 comp rx=0 tx=0).stop 2023-07-21T18:41:45.949 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.926+0000 7f2a67291700 1 -- 172.21.15.39:0/4208562743 shutdown_connections 2023-07-21T18:41:45.950 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.926+0000 7f2a67291700 1 --2- 172.21.15.39:0/4208562743 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a60100660 0x7f2a60102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:45.950 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.926+0000 7f2a67291700 1 --2- 172.21.15.39:0/4208562743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a601058a0 0x7f2a60107c80 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:45.950 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.926+0000 7f2a67291700 1 --2- 172.21.15.39:0/4208562743 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a60102f80 0x7f2a60105360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:45.950 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.926+0000 7f2a67291700 1 -- 172.21.15.39:0/4208562743 >> 172.21.15.39:0/4208562743 conn(0x7f2a600fa000 msgr2=0x7f2a600fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:45.950 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.926+0000 7f2a67291700 1 -- 172.21.15.39:0/4208562743 shutdown_connections 2023-07-21T18:41:45.950 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.926+0000 7f2a67291700 1 -- 172.21.15.39:0/4208562743 wait complete. 2023-07-21T18:41:45.952 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.926+0000 7f2a67291700 1 Processor -- start 2023-07-21T18:41:45.952 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.927+0000 7f2a67291700 1 -- start start 2023-07-21T18:41:45.952 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.927+0000 7f2a67291700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a60100660 0x7f2a600fe610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:45.952 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.927+0000 7f2a67291700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a60102f80 0x7f2a600feb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:45.952 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.927+0000 7f2a67291700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a601058a0 0x7f2a600ff090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:45.952 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.927+0000 7f2a67291700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a60070bf0 con 0x7f2a601058a0 2023-07-21T18:41:45.953 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.927+0000 7f2a67291700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a60070d60 con 0x7f2a60100660 2023-07-21T18:41:45.953 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.927+0000 7f2a67291700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a60070f20 con 0x7f2a60102f80 2023-07-21T18:41:45.953 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.927+0000 7f2a6482c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a60102f80 0x7f2a600feb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:45.953 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.927+0000 7f2a6582e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a601058a0 0x7f2a600ff090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:45.953 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.927+0000 7f2a6502d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a60100660 0x7f2a600fe610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:45.953 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.927+0000 7f2a6582e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a601058a0 0x7f2a600ff090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57434/0 (socket says 172.21.15.39:57434) 2023-07-21T18:41:45.953 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.927+0000 7f2a6482c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a60102f80 0x7f2a600feb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:47752/0 (socket says 172.21.15.39:47752) 2023-07-21T18:41:45.953 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.927+0000 7f2a6582e700 1 -- 172.21.15.39:0/1440283500 learned_addr learned my addr 172.21.15.39:0/1440283500 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:45.954 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.927+0000 7f2a6502d700 1 -- 172.21.15.39:0/1440283500 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a60102f80 msgr2=0x7f2a600feb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:45.954 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.927+0000 7f2a6502d700 1 --2- 172.21.15.39:0/1440283500 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a60102f80 0x7f2a600feb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:45.954 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.927+0000 7f2a6502d700 1 -- 172.21.15.39:0/1440283500 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a601058a0 msgr2=0x7f2a600ff090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:45.954 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.927+0000 7f2a6502d700 1 --2- 172.21.15.39:0/1440283500 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a601058a0 0x7f2a600ff090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:45.954 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.927+0000 7f2a6502d700 1 -- 172.21.15.39:0/1440283500 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a58005e70 con 0x7f2a60100660 2023-07-21T18:41:45.954 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.927+0000 7f2a6482c700 1 --2- 172.21.15.39:0/1440283500 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a60102f80 0x7f2a600feb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:41:45.954 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.927+0000 7f2a6582e700 1 --2- 172.21.15.39:0/1440283500 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a601058a0 0x7f2a600ff090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:41:45.955 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.928+0000 7f2a6502d700 1 --2- 172.21.15.39:0/1440283500 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a60100660 0x7f2a600fe610 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c00e9c0 tx=0x7f2a5c00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:45.955 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.928+0000 7f2a567fc700 1 -- 172.21.15.39:0/1440283500 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c00c9c0 con 0x7f2a60100660 2023-07-21T18:41:45.955 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.928+0000 7f2a567fc700 1 -- 172.21.15.39:0/1440283500 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a5c010430 con 0x7f2a60100660 2023-07-21T18:41:45.955 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.928+0000 7f2a67291700 1 -- 172.21.15.39:0/1440283500 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a60071200 con 0x7f2a60100660 2023-07-21T18:41:45.955 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.928+0000 7f2a567fc700 1 -- 172.21.15.39:0/1440283500 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c018680 con 0x7f2a60100660 2023-07-21T18:41:45.955 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.928+0000 7f2a67291700 1 -- 172.21.15.39:0/1440283500 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a60071710 con 0x7f2a60100660 2023-07-21T18:41:45.956 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.929+0000 7f2a67291700 1 -- 172.21.15.39:0/1440283500 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a6004e2f0 con 0x7f2a60100660 2023-07-21T18:41:45.956 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.930+0000 7f2a567fc700 1 -- 172.21.15.39:0/1440283500 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 21) v1 ==== 135527+0+0 (secure 0 0 0) 0x7f2a5c00cb20 con 0x7f2a60100660 2023-07-21T18:41:45.956 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.930+0000 7f2a567fc700 1 --2- 172.21.15.39:0/1440283500 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f2a4c0a0f50 0x7f2a4c0a3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:45.956 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.931+0000 7f2a6482c700 1 --2- 172.21.15.39:0/1440283500 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f2a4c0a0f50 0x7f2a4c0a3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:45.957 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.931+0000 7f2a567fc700 1 -- 172.21.15.39:0/1440283500 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f2a5c014070 con 0x7f2a60100660 2023-07-21T18:41:45.957 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.931+0000 7f2a6482c700 1 --2- 172.21.15.39:0/1440283500 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f2a4c0a0f50 0x7f2a4c0a3400 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2a600ff910 tx=0x7f2a5000b410 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:45.958 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:45.935+0000 7f2a567fc700 1 -- 172.21.15.39:0/1440283500 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f2a5c051790 con 0x7f2a60100660 2023-07-21T18:41:46.201 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.178+0000 7f2a67291700 1 -- 172.21.15.39:0/1440283500 --> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f2a60071930 con 0x7f2a4c0a0f50 2023-07-21T18:41:46.209 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.186+0000 7f2a567fc700 1 -- 172.21.15.39:0/1440283500 <== mgr.15471 v2:172.21.15.39:6800/13616514 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+283 (secure 0 0 0) 0x7f2a60071930 con 0x7f2a4c0a0f50 2023-07-21T18:41:46.210 INFO:teuthology.orchestra.run.smithi039.stdout:{"fs": "cephfs", "subvol": null, "path": "/", "rel_path": "/", "schedule": "1M", "retention": {"h": 3, "M": 6}, "start": "2023-07-21T00:00:00", "created": "2023-07-21T18:41:45", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2023-07-21T18:41:46.212 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.188+0000 7f2a67291700 1 -- 172.21.15.39:0/1440283500 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f2a4c0a0f50 msgr2=0x7f2a4c0a3400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:46.212 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.188+0000 7f2a67291700 1 --2- 172.21.15.39:0/1440283500 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f2a4c0a0f50 0x7f2a4c0a3400 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2a600ff910 tx=0x7f2a5000b410 comp rx=0 tx=0).stop 2023-07-21T18:41:46.212 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.188+0000 7f2a67291700 1 -- 172.21.15.39:0/1440283500 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a60100660 msgr2=0x7f2a600fe610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:46.212 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.188+0000 7f2a67291700 1 --2- 172.21.15.39:0/1440283500 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a60100660 0x7f2a600fe610 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c00e9c0 tx=0x7f2a5c00ecd0 comp rx=0 tx=0).stop 2023-07-21T18:41:46.212 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.188+0000 7f2a67291700 1 -- 172.21.15.39:0/1440283500 shutdown_connections 2023-07-21T18:41:46.212 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.188+0000 7f2a67291700 1 --2- 172.21.15.39:0/1440283500 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f2a4c0a0f50 0x7f2a4c0a3400 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:46.213 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.188+0000 7f2a67291700 1 --2- 172.21.15.39:0/1440283500 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a60100660 0x7f2a600fe610 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:46.213 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.188+0000 7f2a67291700 1 --2- 172.21.15.39:0/1440283500 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a601058a0 0x7f2a600ff090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:46.213 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.188+0000 7f2a67291700 1 --2- 172.21.15.39:0/1440283500 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a60102f80 0x7f2a600feb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:46.213 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.188+0000 7f2a67291700 1 -- 172.21.15.39:0/1440283500 >> 172.21.15.39:0/1440283500 conn(0x7f2a600fa000 msgr2=0x7f2a601012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:46.213 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.188+0000 7f2a67291700 1 -- 172.21.15.39:0/1440283500 shutdown_connections 2023-07-21T18:41:46.213 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.188+0000 7f2a67291700 1 -- 172.21.15.39:0/1440283500 wait complete. 2023-07-21T18:41:46.224 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2023-07-21T18:41:46.361 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.338+0000 7fcb6a47a700 1 -- 172.21.15.39:0/292027350 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb640744d0 msgr2=0x7fcb64072920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:46.361 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.338+0000 7fcb6a47a700 1 --2- 172.21.15.39:0/292027350 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb640744d0 0x7fcb64072920 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fcb54009990 tx=0x7fcb54009ca0 comp rx=0 tx=0).stop 2023-07-21T18:41:46.362 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.338+0000 7fcb6a47a700 1 -- 172.21.15.39:0/292027350 shutdown_connections 2023-07-21T18:41:46.362 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.338+0000 7fcb6a47a700 1 --2- 172.21.15.39:0/292027350 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb641079c0 0x7fcb64109e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:46.362 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.338+0000 7fcb6a47a700 1 --2- 172.21.15.39:0/292027350 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb640744d0 0x7fcb64072920 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:46.362 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.338+0000 7fcb6a47a700 1 --2- 172.21.15.39:0/292027350 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb64072ef0 0x7fcb64073360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:46.362 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.338+0000 7fcb6a47a700 1 -- 172.21.15.39:0/292027350 >> 172.21.15.39:0/292027350 conn(0x7fcb640fb130 msgr2=0x7fcb640fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:46.362 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.338+0000 7fcb6a47a700 1 -- 172.21.15.39:0/292027350 shutdown_connections 2023-07-21T18:41:46.362 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.338+0000 7fcb6a47a700 1 -- 172.21.15.39:0/292027350 wait complete. 2023-07-21T18:41:46.364 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.338+0000 7fcb6a47a700 1 Processor -- start 2023-07-21T18:41:46.364 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.338+0000 7fcb6a47a700 1 -- start start 2023-07-21T18:41:46.365 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.339+0000 7fcb6a47a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb64072ef0 0x7fcb64197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:46.365 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.339+0000 7fcb6a47a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb640744d0 0x7fcb64198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:46.365 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.339+0000 7fcb6a47a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb641079c0 0x7fcb64198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:46.365 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.339+0000 7fcb6a47a700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb64199530 con 0x7fcb640744d0 2023-07-21T18:41:46.365 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.339+0000 7fcb6a47a700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb641996a0 con 0x7fcb64072ef0 2023-07-21T18:41:46.365 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.339+0000 7fcb6a47a700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb64199860 con 0x7fcb641079c0 2023-07-21T18:41:46.365 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.339+0000 7fcb637fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb640744d0 0x7fcb64198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:46.366 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.339+0000 7fcb637fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb640744d0 0x7fcb64198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57446/0 (socket says 172.21.15.39:57446) 2023-07-21T18:41:46.366 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.339+0000 7fcb637fe700 1 -- 172.21.15.39:0/232970046 learned_addr learned my addr 172.21.15.39:0/232970046 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:46.366 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.339+0000 7fcb63fff700 1 --2- 172.21.15.39:0/232970046 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb64072ef0 0x7fcb64197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:46.366 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.339+0000 7fcb68a17700 1 --2- 172.21.15.39:0/232970046 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb641079c0 0x7fcb64198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:46.366 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.339+0000 7fcb637fe700 1 -- 172.21.15.39:0/232970046 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb641079c0 msgr2=0x7fcb64198ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:46.366 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.339+0000 7fcb637fe700 1 --2- 172.21.15.39:0/232970046 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb641079c0 0x7fcb64198ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:46.366 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.339+0000 7fcb637fe700 1 -- 172.21.15.39:0/232970046 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb64072ef0 msgr2=0x7fcb64197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:46.367 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.339+0000 7fcb637fe700 1 --2- 172.21.15.39:0/232970046 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb64072ef0 0x7fcb64197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:46.367 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.339+0000 7fcb637fe700 1 -- 172.21.15.39:0/232970046 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb540095f0 con 0x7fcb640744d0 2023-07-21T18:41:46.367 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.339+0000 7fcb68a17700 1 --2- 172.21.15.39:0/232970046 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb641079c0 0x7fcb64198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:41:46.367 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.339+0000 7fcb637fe700 1 --2- 172.21.15.39:0/232970046 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb640744d0 0x7fcb64198050 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fcb5800d950 tx=0x7fcb5800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:46.367 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.340+0000 7fcb617fa700 1 -- 172.21.15.39:0/232970046 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb58009940 con 0x7fcb640744d0 2023-07-21T18:41:46.367 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.340+0000 7fcb6a47a700 1 -- 172.21.15.39:0/232970046 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb64199b40 con 0x7fcb640744d0 2023-07-21T18:41:46.367 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.340+0000 7fcb617fa700 1 -- 172.21.15.39:0/232970046 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb58010460 con 0x7fcb640744d0 2023-07-21T18:41:46.367 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.340+0000 7fcb617fa700 1 -- 172.21.15.39:0/232970046 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5800f620 con 0x7fcb640744d0 2023-07-21T18:41:46.368 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.340+0000 7fcb6a47a700 1 -- 172.21.15.39:0/232970046 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb641a0300 con 0x7fcb640744d0 2023-07-21T18:41:46.368 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.341+0000 7fcb6a47a700 1 -- 172.21.15.39:0/232970046 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb64106100 con 0x7fcb640744d0 2023-07-21T18:41:46.368 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.343+0000 7fcb617fa700 1 -- 172.21.15.39:0/232970046 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 21) v1 ==== 135527+0+0 (secure 0 0 0) 0x7fcb580108d0 con 0x7fcb640744d0 2023-07-21T18:41:46.369 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.343+0000 7fcb617fa700 1 --2- 172.21.15.39:0/232970046 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fcb4c0a0f50 0x7fcb4c0a3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:46.369 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.343+0000 7fcb63fff700 1 --2- 172.21.15.39:0/232970046 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fcb4c0a0f50 0x7fcb4c0a3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:46.369 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.344+0000 7fcb617fa700 1 -- 172.21.15.39:0/232970046 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fcb580cc4e0 con 0x7fcb640744d0 2023-07-21T18:41:46.369 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.344+0000 7fcb63fff700 1 --2- 172.21.15.39:0/232970046 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fcb4c0a0f50 0x7fcb4c0a3400 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fcb5400b5c0 tx=0x7fcb54005fd0 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:46.370 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.347+0000 7fcb617fa700 1 -- 172.21.15.39:0/232970046 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fcb58095de0 con 0x7fcb640744d0 2023-07-21T18:41:46.620 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:46 smithi193 ceph-mon[98204]: mgrmap e21: x(active, since 1.02076s), standbys: y, z 2023-07-21T18:41:46.622 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.599+0000 7fcb6a47a700 1 -- 172.21.15.39:0/232970046 --> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fcb64060950 con 0x7fcb4c0a0f50 2023-07-21T18:41:46.629 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.606+0000 7fcb617fa700 1 -- 172.21.15.39:0/232970046 <== mgr.15471 v2:172.21.15.39:6800/13616514 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7fcb64060950 con 0x7fcb4c0a0f50 2023-07-21T18:41:46.629 INFO:teuthology.orchestra.run.smithi039.stdout:/ 1M 6M3h 2023-07-21T18:41:46.632 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.608+0000 7fcb6a47a700 1 -- 172.21.15.39:0/232970046 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fcb4c0a0f50 msgr2=0x7fcb4c0a3400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:46.633 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.608+0000 7fcb6a47a700 1 --2- 172.21.15.39:0/232970046 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fcb4c0a0f50 0x7fcb4c0a3400 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fcb5400b5c0 tx=0x7fcb54005fd0 comp rx=0 tx=0).stop 2023-07-21T18:41:46.633 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.608+0000 7fcb6a47a700 1 -- 172.21.15.39:0/232970046 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb640744d0 msgr2=0x7fcb64198050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:46.633 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.608+0000 7fcb6a47a700 1 --2- 172.21.15.39:0/232970046 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb640744d0 0x7fcb64198050 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fcb5800d950 tx=0x7fcb5800dc60 comp rx=0 tx=0).stop 2023-07-21T18:41:46.634 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.609+0000 7fcb6a47a700 1 -- 172.21.15.39:0/232970046 shutdown_connections 2023-07-21T18:41:46.634 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.609+0000 7fcb6a47a700 1 --2- 172.21.15.39:0/232970046 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fcb4c0a0f50 0x7fcb4c0a3400 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:46.634 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.609+0000 7fcb6a47a700 1 --2- 172.21.15.39:0/232970046 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb64072ef0 0x7fcb64197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:46.634 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.609+0000 7fcb6a47a700 1 --2- 172.21.15.39:0/232970046 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb641079c0 0x7fcb64198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:46.635 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.609+0000 7fcb6a47a700 1 --2- 172.21.15.39:0/232970046 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb640744d0 0x7fcb64198050 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:46.635 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.609+0000 7fcb6a47a700 1 -- 172.21.15.39:0/232970046 >> 172.21.15.39:0/232970046 conn(0x7fcb640fb130 msgr2=0x7fcb640fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:46.635 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.609+0000 7fcb6a47a700 1 -- 172.21.15.39:0/232970046 shutdown_connections 2023-07-21T18:41:46.636 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:41:46.609+0000 7fcb6a47a700 1 -- 172.21.15.39:0/232970046 wait complete. 2023-07-21T18:41:46.639 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:46 smithi155 ceph-mon[97934]: mgrmap e21: x(active, since 1.02076s), standbys: y, z 2023-07-21T18:41:46.643 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:46 smithi039 ceph-mon[96489]: mgrmap e21: x(active, since 1.02076s), standbys: y, z 2023-07-21T18:41:46.644 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2023-07-21T18:41:46.673 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2023-07-21T18:41:46.687 INFO:teuthology.run_tasks:Running task background_exec... 2023-07-21T18:41:46.727 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi039.front.sepia.ceph.com 2023-07-21T18:41:46.728 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2023-07-21T18:41:46.730 INFO:teuthology.run_tasks:Running task workunit... 2023-07-21T18:41:46.783 INFO:tasks.workunit:Pulling workunits from ref 81cf785aa7781c4e08381c540f890cb9bc987ef9 2023-07-21T18:41:46.784 INFO:tasks.workunit:Making a separate scratch dir for every client... 2023-07-21T18:41:46.784 INFO:tasks.workunit:timeout=3h 2023-07-21T18:41:46.785 INFO:tasks.workunit:cleanup=True 2023-07-21T18:41:46.785 DEBUG:teuthology.orchestra.run.smithi039:> stat -- /home/ubuntu/cephtest/mnt.0 2023-07-21T18:41:46.804 INFO:teuthology.orchestra.run.smithi039.stdout: File: /home/ubuntu/cephtest/mnt.0 2023-07-21T18:41:46.804 INFO:teuthology.orchestra.run.smithi039.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2023-07-21T18:41:46.804 INFO:teuthology.orchestra.run.smithi039.stdout:Device: 7ch/124d Inode: 1099511628283 Links: 2 2023-07-21T18:41:46.804 INFO:teuthology.orchestra.run.smithi039.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-21T18:41:46.804 INFO:teuthology.orchestra.run.smithi039.stdout:Context: system_u:object_r:cephfs_t:s0 2023-07-21T18:41:46.805 INFO:teuthology.orchestra.run.smithi039.stdout:Access: 2023-07-21 18:40:11.452445726 +0000 2023-07-21T18:41:46.805 INFO:teuthology.orchestra.run.smithi039.stdout:Modify: 2023-07-21 18:40:11.452445726 +0000 2023-07-21T18:41:46.805 INFO:teuthology.orchestra.run.smithi039.stdout:Change: 2023-07-21 18:41:28.373145450 +0000 2023-07-21T18:41:46.805 INFO:teuthology.orchestra.run.smithi039.stdout: Birth: 2023-07-21 18:40:11.452445726 +0000 2023-07-21T18:41:46.805 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2023-07-21T18:41:46.805 DEBUG:teuthology.orchestra.run.smithi039:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2023-07-21T18:41:46.906 DEBUG:teuthology.orchestra.run.smithi039:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 81cf785aa7781c4e08381c540f890cb9bc987ef9 2023-07-21T18:41:46.925 INFO:tasks.workunit.client.0.smithi039.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2023-07-21T18:41:47.620 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:47 smithi193 ceph-mon[98204]: from='client.25111 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:41:47.620 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:47 smithi193 ceph-mon[98204]: pgmap v3: 97 pgs: 97 active+clean; 1.0 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:41:47.621 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:47 smithi193 ceph-mon[98204]: from='client.25178 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:41:47.621 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:47 smithi193 ceph-mon[98204]: from='client.25184 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:41:47.621 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:47 smithi193 ceph-mon[98204]: mgrmap e22: x(active, since 2s), standbys: y, z 2023-07-21T18:41:47.638 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:47 smithi155 ceph-mon[97934]: from='client.25111 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:41:47.639 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:47 smithi155 ceph-mon[97934]: pgmap v3: 97 pgs: 97 active+clean; 1.0 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:41:47.639 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:47 smithi155 ceph-mon[97934]: from='client.25178 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:41:47.639 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:47 smithi155 ceph-mon[97934]: from='client.25184 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:41:47.639 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:47 smithi155 ceph-mon[97934]: mgrmap e22: x(active, since 2s), standbys: y, z 2023-07-21T18:41:47.687 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:47 smithi039 ceph-mon[96489]: from='client.25111 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:41:47.688 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:47 smithi039 ceph-mon[96489]: pgmap v3: 97 pgs: 97 active+clean; 1.0 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:41:47.688 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:47 smithi039 ceph-mon[96489]: from='client.25178 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:41:47.688 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:47 smithi039 ceph-mon[96489]: from='client.25184 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:41:47.688 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:47 smithi039 ceph-mon[96489]: mgrmap e22: x(active, since 2s), standbys: y, z 2023-07-21T18:41:48.620 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:48 smithi193 ceph-mon[98204]: [21/Jul/2023:18:41:46] ENGINE Bus STARTING 2023-07-21T18:41:48.620 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:48 smithi193 ceph-mon[98204]: from='client.15546 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:41:48.621 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:48 smithi193 ceph-mon[98204]: [21/Jul/2023:18:41:46] ENGINE Serving on http://172.21.15.39:8765 2023-07-21T18:41:48.621 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:48 smithi193 ceph-mon[98204]: [21/Jul/2023:18:41:46] ENGINE Serving on https://172.21.15.39:7150 2023-07-21T18:41:48.621 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:48 smithi193 ceph-mon[98204]: [21/Jul/2023:18:41:46] ENGINE Bus STARTED 2023-07-21T18:41:48.621 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:48 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:41:48.621 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:48 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:41:48.621 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:48 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:41:48.622 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:48 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:41:48.622 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:48 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:41:48.622 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:48 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:41:48.639 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:48 smithi155 ceph-mon[97934]: [21/Jul/2023:18:41:46] ENGINE Bus STARTING 2023-07-21T18:41:48.639 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:48 smithi155 ceph-mon[97934]: from='client.15546 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:41:48.639 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:48 smithi155 ceph-mon[97934]: [21/Jul/2023:18:41:46] ENGINE Serving on http://172.21.15.39:8765 2023-07-21T18:41:48.639 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:48 smithi155 ceph-mon[97934]: [21/Jul/2023:18:41:46] ENGINE Serving on https://172.21.15.39:7150 2023-07-21T18:41:48.639 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:48 smithi155 ceph-mon[97934]: [21/Jul/2023:18:41:46] ENGINE Bus STARTED 2023-07-21T18:41:48.640 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:48 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:41:48.640 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:48 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:41:48.640 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:48 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:41:48.640 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:48 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:41:48.640 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:48 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:41:48.640 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:48 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:41:48.687 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:48 smithi039 ceph-mon[96489]: [21/Jul/2023:18:41:46] ENGINE Bus STARTING 2023-07-21T18:41:48.688 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:48 smithi039 ceph-mon[96489]: from='client.15546 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:41:48.688 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:48 smithi039 ceph-mon[96489]: [21/Jul/2023:18:41:46] ENGINE Serving on http://172.21.15.39:8765 2023-07-21T18:41:48.688 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:48 smithi039 ceph-mon[96489]: [21/Jul/2023:18:41:46] ENGINE Serving on https://172.21.15.39:7150 2023-07-21T18:41:48.688 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:48 smithi039 ceph-mon[96489]: [21/Jul/2023:18:41:46] ENGINE Bus STARTED 2023-07-21T18:41:48.688 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:48 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:41:48.689 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:48 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:41:48.689 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:48 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:41:48.689 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:48 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:41:48.689 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:48 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:41:48.689 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:48 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:41:49.620 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:49 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:41:49.621 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:49 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:41:49.621 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:49 smithi193 ceph-mon[98204]: pgmap v5: 97 pgs: 97 active+clean; 1.0 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:41:49.621 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:49 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.622 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:49 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.622 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:49 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.622 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:49 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.622 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:49 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.623 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:49 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.623 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:49 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.623 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:49 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.624 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:49 smithi193 ceph-mon[98204]: Adjusting osd_memory_target on smithi193 to 196.2M 2023-07-21T18:41:49.624 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:49 smithi193 ceph-mon[98204]: Unable to set osd_memory_target on smithi193 to 205773926: error parsing value: Value '205773926' is below minimum 939524096 2023-07-21T18:41:49.624 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:49 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:41:49.624 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:49 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:41:49.625 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:49 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.625 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:49 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:41:49.625 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:49 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.626 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:49 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.626 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:49 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.626 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:49 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:41:49.626 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:49 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.627 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:49 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.627 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:49 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.627 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:49 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.627 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:49 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.628 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:49 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.628 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:49 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.628 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:49 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.629 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:49 smithi193 ceph-mon[98204]: Adjusting osd_memory_target on smithi039 to 196.2M 2023-07-21T18:41:49.629 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:49 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.629 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:49 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.629 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:49 smithi193 ceph-mon[98204]: Unable to set osd_memory_target on smithi039 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-07-21T18:41:49.629 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:49 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.629 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:49 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.629 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:49 smithi193 ceph-mon[98204]: Adjusting osd_memory_target on smithi155 to 196.2M 2023-07-21T18:41:49.630 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:49 smithi193 ceph-mon[98204]: Unable to set osd_memory_target on smithi155 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-07-21T18:41:49.630 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:49 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:41:49.630 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:49 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:41:49.630 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:49 smithi193 ceph-mon[98204]: Updating smithi039:/etc/ceph/ceph.conf 2023-07-21T18:41:49.630 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:49 smithi193 ceph-mon[98204]: Updating smithi155:/etc/ceph/ceph.conf 2023-07-21T18:41:49.630 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:49 smithi193 ceph-mon[98204]: Updating smithi193:/etc/ceph/ceph.conf 2023-07-21T18:41:49.630 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:49 smithi193 ceph-mon[98204]: Updating smithi155:/var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T18:41:49.630 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:49 smithi193 ceph-mon[98204]: Updating smithi039:/var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T18:41:49.631 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:49 smithi193 ceph-mon[98204]: Updating smithi193:/var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T18:41:49.631 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:49 smithi193 ceph-mon[98204]: Updating smithi155:/etc/ceph/ceph.client.admin.keyring 2023-07-21T18:41:49.631 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:49 smithi193 ceph-mon[98204]: Updating smithi039:/etc/ceph/ceph.client.admin.keyring 2023-07-21T18:41:49.631 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:49 smithi193 ceph-mon[98204]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2023-07-21T18:41:49.639 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:49 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:41:49.639 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:49 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:41:49.639 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:49 smithi155 ceph-mon[97934]: pgmap v5: 97 pgs: 97 active+clean; 1.0 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:41:49.639 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:49 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.639 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:49 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.640 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:49 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.640 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:49 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.640 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:49 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.640 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:49 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.640 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:49 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.640 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:49 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.640 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:49 smithi155 ceph-mon[97934]: Adjusting osd_memory_target on smithi193 to 196.2M 2023-07-21T18:41:49.641 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:49 smithi155 ceph-mon[97934]: Unable to set osd_memory_target on smithi193 to 205773926: error parsing value: Value '205773926' is below minimum 939524096 2023-07-21T18:41:49.641 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:49 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:41:49.641 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:49 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:41:49.641 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:49 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.641 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:49 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:41:49.641 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:49 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.641 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:49 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.642 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:49 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.642 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:49 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:41:49.642 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:49 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.642 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:49 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.642 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:49 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.642 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:49 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.642 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:49 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.643 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:49 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.643 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:49 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.643 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:49 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.643 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:49 smithi155 ceph-mon[97934]: Adjusting osd_memory_target on smithi039 to 196.2M 2023-07-21T18:41:49.643 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:49 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.643 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:49 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.643 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:49 smithi155 ceph-mon[97934]: Unable to set osd_memory_target on smithi039 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-07-21T18:41:49.644 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:49 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.644 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:49 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.644 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:49 smithi155 ceph-mon[97934]: Adjusting osd_memory_target on smithi155 to 196.2M 2023-07-21T18:41:49.644 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:49 smithi155 ceph-mon[97934]: Unable to set osd_memory_target on smithi155 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-07-21T18:41:49.644 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:49 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:41:49.644 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:49 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:41:49.644 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:49 smithi155 ceph-mon[97934]: Updating smithi039:/etc/ceph/ceph.conf 2023-07-21T18:41:49.645 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:49 smithi155 ceph-mon[97934]: Updating smithi155:/etc/ceph/ceph.conf 2023-07-21T18:41:49.645 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:49 smithi155 ceph-mon[97934]: Updating smithi193:/etc/ceph/ceph.conf 2023-07-21T18:41:49.645 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:49 smithi155 ceph-mon[97934]: Updating smithi155:/var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T18:41:49.645 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:49 smithi155 ceph-mon[97934]: Updating smithi039:/var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T18:41:49.645 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:49 smithi155 ceph-mon[97934]: Updating smithi193:/var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T18:41:49.645 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:49 smithi155 ceph-mon[97934]: Updating smithi155:/etc/ceph/ceph.client.admin.keyring 2023-07-21T18:41:49.645 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:49 smithi155 ceph-mon[97934]: Updating smithi039:/etc/ceph/ceph.client.admin.keyring 2023-07-21T18:41:49.645 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:49 smithi155 ceph-mon[97934]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2023-07-21T18:41:49.688 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:49 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:41:49.689 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:49 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:41:49.689 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:49 smithi039 ceph-mon[96489]: pgmap v5: 97 pgs: 97 active+clean; 1.0 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T18:41:49.689 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:49 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:49 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:49 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:49 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:49 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:49 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:49 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:49 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:49 smithi039 ceph-mon[96489]: Adjusting osd_memory_target on smithi193 to 196.2M 2023-07-21T18:41:49.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:49 smithi039 ceph-mon[96489]: Unable to set osd_memory_target on smithi193 to 205773926: error parsing value: Value '205773926' is below minimum 939524096 2023-07-21T18:41:49.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:49 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:41:49.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:49 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:41:49.693 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:49 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.693 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:49 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:41:49.693 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:49 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:49 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:49 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:49 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:41:49.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:49 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:49 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:49 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:49 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.696 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:49 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.696 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:49 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.696 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:49 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.696 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:49 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.696 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:49 smithi039 ceph-mon[96489]: Adjusting osd_memory_target on smithi039 to 196.2M 2023-07-21T18:41:49.696 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:49 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.696 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:49 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.697 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:49 smithi039 ceph-mon[96489]: Unable to set osd_memory_target on smithi039 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-07-21T18:41:49.697 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:49 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.697 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:49 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:41:49.697 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:49 smithi039 ceph-mon[96489]: Adjusting osd_memory_target on smithi155 to 196.2M 2023-07-21T18:41:49.697 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:49 smithi039 ceph-mon[96489]: Unable to set osd_memory_target on smithi155 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-07-21T18:41:49.697 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:49 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:41:49.697 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:49 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:41:49.697 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:49 smithi039 ceph-mon[96489]: Updating smithi039:/etc/ceph/ceph.conf 2023-07-21T18:41:49.698 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:49 smithi039 ceph-mon[96489]: Updating smithi155:/etc/ceph/ceph.conf 2023-07-21T18:41:49.698 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:49 smithi039 ceph-mon[96489]: Updating smithi193:/etc/ceph/ceph.conf 2023-07-21T18:41:49.698 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:49 smithi039 ceph-mon[96489]: Updating smithi155:/var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T18:41:49.698 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:49 smithi039 ceph-mon[96489]: Updating smithi039:/var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T18:41:49.698 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:49 smithi039 ceph-mon[96489]: Updating smithi193:/var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T18:41:49.698 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:49 smithi039 ceph-mon[96489]: Updating smithi155:/etc/ceph/ceph.client.admin.keyring 2023-07-21T18:41:49.698 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:49 smithi039 ceph-mon[96489]: Updating smithi039:/etc/ceph/ceph.client.admin.keyring 2023-07-21T18:41:49.699 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:49 smithi039 ceph-mon[96489]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2023-07-21T18:41:50.870 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:50 smithi193 ceph-mon[98204]: Updating smithi155:/var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-21T18:41:50.870 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:50 smithi193 ceph-mon[98204]: Updating smithi039:/var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-21T18:41:50.871 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:50 smithi193 ceph-mon[98204]: Updating smithi193:/var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-21T18:41:50.871 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:50 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:41:50.871 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:50 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:41:50.871 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:50 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:41:50.871 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:50 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:41:50.871 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:50 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:41:50.871 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:50 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:41:50.871 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:50 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:41:50.872 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:50 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:41:50.872 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:50 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:41:50.872 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:50 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:41:50.872 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:50 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:41:50.889 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:50 smithi155 ceph-mon[97934]: Updating smithi155:/var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-21T18:41:50.889 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:50 smithi155 ceph-mon[97934]: Updating smithi039:/var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-21T18:41:50.889 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:50 smithi155 ceph-mon[97934]: Updating smithi193:/var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-21T18:41:50.889 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:50 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:41:50.890 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:50 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:41:50.890 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:50 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:41:50.890 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:50 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:41:50.890 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:50 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:41:50.890 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:50 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:41:50.890 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:50 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:41:50.890 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:50 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:41:50.890 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:50 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:41:50.891 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:50 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:41:50.891 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:50 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:41:50.938 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:50 smithi039 ceph-mon[96489]: Updating smithi155:/var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-21T18:41:50.938 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:50 smithi039 ceph-mon[96489]: Updating smithi039:/var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-21T18:41:50.938 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:50 smithi039 ceph-mon[96489]: Updating smithi193:/var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-21T18:41:50.938 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:50 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:41:50.938 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:50 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:41:50.938 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:50 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:41:50.939 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:50 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:41:50.939 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:50 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:41:50.939 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:50 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:41:50.939 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:50 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:41:50.939 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:50 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:41:50.939 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:50 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:41:50.939 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:50 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:41:50.939 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:50 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:41:52.370 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:51 smithi193 ceph-mon[98204]: pgmap v6: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 185 KiB/s wr, 75 op/s 2023-07-21T18:41:52.389 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:51 smithi155 ceph-mon[97934]: pgmap v6: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 185 KiB/s wr, 75 op/s 2023-07-21T18:41:52.437 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:51 smithi039 ceph-mon[96489]: pgmap v6: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 185 KiB/s wr, 75 op/s 2023-07-21T18:41:54.370 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:53 smithi193 ceph-mon[98204]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 132 KiB/s wr, 54 op/s 2023-07-21T18:41:54.388 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:53 smithi155 ceph-mon[97934]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 132 KiB/s wr, 54 op/s 2023-07-21T18:41:54.437 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:53 smithi039 ceph-mon[96489]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 132 KiB/s wr, 54 op/s 2023-07-21T18:41:56.370 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:55 smithi193 ceph-mon[98204]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 103 KiB/s wr, 42 op/s 2023-07-21T18:41:56.388 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:55 smithi155 ceph-mon[97934]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 103 KiB/s wr, 42 op/s 2023-07-21T18:41:56.437 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:55 smithi039 ceph-mon[96489]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 103 KiB/s wr, 42 op/s 2023-07-21T18:41:58.370 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:41:57 smithi193 ceph-mon[98204]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 84 KiB/s wr, 39 op/s 2023-07-21T18:41:58.388 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:41:57 smithi155 ceph-mon[97934]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 84 KiB/s wr, 39 op/s 2023-07-21T18:41:58.437 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:41:57 smithi039 ceph-mon[96489]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 84 KiB/s wr, 39 op/s 2023-07-21T18:41:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.880+0000 7f3565da1700 1 -- 172.21.15.39:0/2028229940 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35600fea30 msgr2=0x7f35600feea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.880+0000 7f3565da1700 1 --2- 172.21.15.39:0/2028229940 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35600fea30 0x7f35600feea0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f3554009a10 tx=0x7f3554009d20 comp rx=0 tx=0).stop 2023-07-21T18:41:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.880+0000 7f3565da1700 1 -- 172.21.15.39:0/2028229940 shutdown_connections 2023-07-21T18:41:59.903 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.880+0000 7f3565da1700 1 --2- 172.21.15.39:0/2028229940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35600ff3e0 0x7f356010b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:59.903 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.880+0000 7f3565da1700 1 --2- 172.21.15.39:0/2028229940 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3560104a30 0x7f3560104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:59.903 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.880+0000 7f3565da1700 1 --2- 172.21.15.39:0/2028229940 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35600fea30 0x7f35600feea0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:59.903 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.880+0000 7f3565da1700 1 -- 172.21.15.39:0/2028229940 >> 172.21.15.39:0/2028229940 conn(0x7f3560074760 msgr2=0x7f3560074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:59.903 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.880+0000 7f3565da1700 1 -- 172.21.15.39:0/2028229940 shutdown_connections 2023-07-21T18:41:59.903 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.880+0000 7f3565da1700 1 -- 172.21.15.39:0/2028229940 wait complete. 2023-07-21T18:41:59.904 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.880+0000 7f3565da1700 1 Processor -- start 2023-07-21T18:41:59.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.881+0000 7f3565da1700 1 -- start start 2023-07-21T18:41:59.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.881+0000 7f3565da1700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35600fea30 0x7f35601998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:59.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.881+0000 7f3565da1700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35600ff3e0 0x7f3560199de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:59.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.881+0000 7f3565da1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3560104a30 0x7f3560193700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:59.907 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.881+0000 7f3565da1700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3560193d70 con 0x7f3560104a30 2023-07-21T18:41:59.907 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.881+0000 7f3565da1700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3560193ee0 con 0x7f35600ff3e0 2023-07-21T18:41:59.907 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.881+0000 7f3565da1700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35601940a0 con 0x7f35600fea30 2023-07-21T18:41:59.907 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.881+0000 7f355f7fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35600fea30 0x7f35601998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:59.907 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.881+0000 7f355ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3560104a30 0x7f3560193700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:59.907 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.881+0000 7f355f7fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35600fea30 0x7f35601998a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:47790/0 (socket says 172.21.15.39:47790) 2023-07-21T18:41:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.881+0000 7f355f7fe700 1 -- 172.21.15.39:0/3791771041 learned_addr learned my addr 172.21.15.39:0/3791771041 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.881+0000 7f355ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3560104a30 0x7f3560193700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57474/0 (socket says 172.21.15.39:57474) 2023-07-21T18:41:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.881+0000 7f355effd700 1 --2- 172.21.15.39:0/3791771041 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35600ff3e0 0x7f3560199de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.881+0000 7f355ffff700 1 -- 172.21.15.39:0/3791771041 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35600fea30 msgr2=0x7f35601998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.881+0000 7f355ffff700 1 --2- 172.21.15.39:0/3791771041 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35600fea30 0x7f35601998a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.881+0000 7f355ffff700 1 -- 172.21.15.39:0/3791771041 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35600ff3e0 msgr2=0x7f3560199de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:59.909 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.881+0000 7f355ffff700 1 --2- 172.21.15.39:0/3791771041 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35600ff3e0 0x7f3560199de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:59.909 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.881+0000 7f355ffff700 1 -- 172.21.15.39:0/3791771041 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35540095f0 con 0x7f3560104a30 2023-07-21T18:41:59.909 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.881+0000 7f355f7fe700 1 --2- 172.21.15.39:0/3791771041 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35600fea30 0x7f35601998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:41:59.909 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.882+0000 7f355ffff700 1 --2- 172.21.15.39:0/3791771041 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3560104a30 0x7f3560193700 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f355800cc50 tx=0x7f355800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:59.909 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.882+0000 7f355cff9700 1 -- 172.21.15.39:0/3791771041 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355800aa90 con 0x7f3560104a30 2023-07-21T18:41:59.909 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.882+0000 7f355cff9700 1 -- 172.21.15.39:0/3791771041 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f355800abf0 con 0x7f3560104a30 2023-07-21T18:41:59.909 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.882+0000 7f3565da1700 1 -- 172.21.15.39:0/3791771041 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3560194380 con 0x7f3560104a30 2023-07-21T18:41:59.910 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.882+0000 7f355cff9700 1 -- 172.21.15.39:0/3791771041 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35580115e0 con 0x7f3560104a30 2023-07-21T18:41:59.910 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.882+0000 7f3565da1700 1 -- 172.21.15.39:0/3791771041 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35601a0890 con 0x7f3560104a30 2023-07-21T18:41:59.910 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.884+0000 7f355cff9700 1 -- 172.21.15.39:0/3791771041 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) v1 ==== 135891+0+0 (secure 0 0 0) 0x7f3558012420 con 0x7f3560104a30 2023-07-21T18:41:59.915 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.885+0000 7f355cff9700 1 --2- 172.21.15.39:0/3791771041 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f35480a1350 0x7f35480a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:59.915 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.885+0000 7f355cff9700 1 -- 172.21.15.39:0/3791771041 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f35580cced0 con 0x7f3560104a30 2023-07-21T18:41:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.885+0000 7f355f7fe700 1 --2- 172.21.15.39:0/3791771041 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f35480a1350 0x7f35480a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.885+0000 7f355f7fe700 1 --2- 172.21.15.39:0/3791771041 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f35480a1350 0x7f35480a3800 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f35500105c0 tx=0x7f355000b4a0 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.886+0000 7f3571f27b80 1 Processor -- start 2023-07-21T18:41:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.886+0000 7f3571f27b80 1 -- start start 2023-07-21T18:41:59.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.886+0000 7f3571f27b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632b2b556b0 0x5632b2d03160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:59.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.886+0000 7f3571f27b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5632b2d036a0 0x5632b2cfd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:59.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.886+0000 7f3571f27b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632b2d03b10 0x5632b2cfd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:59.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.886+0000 7f3571f27b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632b2bfec50 con 0x5632b2d036a0 2023-07-21T18:41:59.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.886+0000 7f3571f27b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632b2c519e0 con 0x5632b2d03b10 2023-07-21T18:41:59.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.886+0000 7f3571f27b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632b2c4ae70 con 0x5632b2b556b0 2023-07-21T18:41:59.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.886+0000 7f355effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5632b2d036a0 0x5632b2cfd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:59.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.886+0000 7f355effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5632b2d036a0 0x5632b2cfd450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57480/0 (socket says 172.21.15.39:57480) 2023-07-21T18:41:59.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.886+0000 7f355effd700 1 -- 172.21.15.39:0/3660217158 learned_addr learned my addr 172.21.15.39:0/3660217158 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:59.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.886+0000 7f355f7fe700 1 --2- 172.21.15.39:0/3660217158 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632b2b556b0 0x5632b2d03160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:59.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.886+0000 7f355ffff700 1 --2- 172.21.15.39:0/3660217158 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632b2d03b10 0x5632b2cfd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:59.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.886+0000 7f355effd700 1 -- 172.21.15.39:0/3660217158 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632b2b556b0 msgr2=0x5632b2d03160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:59.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.886+0000 7f355effd700 1 --2- 172.21.15.39:0/3660217158 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632b2b556b0 0x5632b2d03160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:59.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.886+0000 7f355effd700 1 -- 172.21.15.39:0/3660217158 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632b2d03b10 msgr2=0x5632b2cfd9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.886+0000 7f355effd700 1 --2- 172.21.15.39:0/3660217158 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632b2d03b10 0x5632b2cfd9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.887+0000 7f355effd700 1 -- 172.21.15.39:0/3660217158 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632b2a78d80 con 0x5632b2d036a0 2023-07-21T18:41:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.887+0000 7f355effd700 1 --2- 172.21.15.39:0/3660217158 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5632b2d036a0 0x5632b2cfd450 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f3554009a10 tx=0x7f35540037b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf2ee8a1c49cf620 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.887+0000 7f3545ffb700 1 -- 172.21.15.39:0/3660217158 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3554003be0 con 0x5632b2d036a0 2023-07-21T18:41:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.887+0000 7f3545ffb700 1 -- 172.21.15.39:0/3660217158 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3554005040 con 0x5632b2d036a0 2023-07-21T18:41:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.887+0000 7f3545ffb700 1 -- 172.21.15.39:0/3660217158 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355400b990 con 0x5632b2d036a0 2023-07-21T18:41:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.888+0000 7f3571f27b80 1 -- 172.21.15.39:0/3660217158 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5632b2d036a0 msgr2=0x5632b2cfd450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:59.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.888+0000 7f3571f27b80 1 --2- 172.21.15.39:0/3660217158 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5632b2d036a0 0x5632b2cfd450 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f3554009a10 tx=0x7f35540037b0 comp rx=0 tx=0).stop 2023-07-21T18:41:59.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.888+0000 7f3571f27b80 1 -- 172.21.15.39:0/3660217158 shutdown_connections 2023-07-21T18:41:59.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.888+0000 7f3571f27b80 1 --2- 172.21.15.39:0/3660217158 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632b2d03b10 0x5632b2cfd9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:59.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.888+0000 7f3571f27b80 1 --2- 172.21.15.39:0/3660217158 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632b2b556b0 0x5632b2d03160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:59.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.888+0000 7f3571f27b80 1 --2- 172.21.15.39:0/3660217158 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5632b2d036a0 0x5632b2cfd450 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:59.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.888+0000 7f3571f27b80 1 -- 172.21.15.39:0/3660217158 >> 172.21.15.39:0/3660217158 conn(0x5632b2ceebe0 msgr2=0x5632b2cf3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:59.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.888+0000 7f3571f27b80 1 -- 172.21.15.39:0/3660217158 shutdown_connections 2023-07-21T18:41:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.888+0000 7f3571f27b80 1 -- 172.21.15.39:0/3660217158 wait complete. 2023-07-21T18:41:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.888+0000 7f3571f27b80 1 -- start start 2023-07-21T18:41:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.888+0000 7f3571f27b80 1 Processor -- start 2023-07-21T18:41:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.889+0000 7f3571f27b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5632b2b556b0 0x5632b2d09930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.889+0000 7f3571f27b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632b2d036a0 0x5632b2d06bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.889+0000 7f3571f27b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632b2d03b10 0x5632b2d07150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.889+0000 7f3571f27b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632b2beaa60 con 0x5632b2b556b0 2023-07-21T18:41:59.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.889+0000 7f3571f27b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632b2bea760 con 0x5632b2d036a0 2023-07-21T18:41:59.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.889+0000 7f3571f27b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632b2ace8c0 con 0x5632b2d03b10 2023-07-21T18:41:59.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.889+0000 7f3571f27b80 10 client.0 _ll_register_callbacks cb 0x5632b2be8700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:41:59.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.889+0000 7f355f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5632b2b556b0 0x5632b2d09930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:59.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.889+0000 7f355f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5632b2b556b0 0x5632b2d09930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57484/0 (socket says 172.21.15.39:57484) 2023-07-21T18:41:59.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.889+0000 7f355f7fe700 1 -- 172.21.15.39:0/946496778 learned_addr learned my addr 172.21.15.39:0/946496778 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:59.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.889+0000 7f355effd700 1 --2- 172.21.15.39:0/946496778 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632b2d036a0 0x5632b2d06bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:59.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.889+0000 7f355ffff700 1 --2- 172.21.15.39:0/946496778 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632b2d03b10 0x5632b2d07150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:59.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.889+0000 7f355f7fe700 1 -- 172.21.15.39:0/946496778 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632b2d03b10 msgr2=0x5632b2d07150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:59.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.889+0000 7f355f7fe700 1 --2- 172.21.15.39:0/946496778 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632b2d03b10 0x5632b2d07150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:59.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.889+0000 7f355f7fe700 1 -- 172.21.15.39:0/946496778 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632b2d036a0 msgr2=0x5632b2d06bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:59.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.889+0000 7f355f7fe700 1 --2- 172.21.15.39:0/946496778 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632b2d036a0 0x5632b2d06bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:59.924 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.889+0000 7f355f7fe700 1 -- 172.21.15.39:0/946496778 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f356010c160 con 0x5632b2b556b0 2023-07-21T18:41:59.924 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.889+0000 7f355effd700 1 --2- 172.21.15.39:0/946496778 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632b2d036a0 0x5632b2d06bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:41:59.924 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.889+0000 7f355f7fe700 1 --2- 172.21.15.39:0/946496778 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5632b2b556b0 0x5632b2d09930 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f3550011b30 tx=0x7f35500047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38f86e42183c62d0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:59.924 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.889+0000 7f353effd700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:41:59.924 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.889+0000 7f353effd700 1 -- 172.21.15.39:0/946496778 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355001ea50 con 0x5632b2b556b0 2023-07-21T18:41:59.924 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.889+0000 7f353effd700 1 -- 172.21.15.39:0/946496778 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f3550017070 con 0x5632b2b556b0 2023-07-21T18:41:59.924 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.889+0000 7f3571f27b80 1 -- 172.21.15.39:0/946496778 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5632b2ba4490 con 0x5632b2b556b0 2023-07-21T18:41:59.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.890+0000 7f353effd700 1 -- 172.21.15.39:0/946496778 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f355001ebb0 con 0x5632b2b556b0 2023-07-21T18:41:59.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.890+0000 7f3571f27b80 10 client.15558 fetch_fsmap learned FSMap version 26 2023-07-21T18:41:59.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.890+0000 7f3571f27b80 1 -- 172.21.15.39:0/946496778 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5632b2a61c00 con 0x5632b2b556b0 2023-07-21T18:41:59.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.890+0000 7f353effd700 1 -- 172.21.15.39:0/946496778 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5632b2ba4490 con 0x5632b2b556b0 2023-07-21T18:41:59.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.890+0000 7f353effd700 1 -- 172.21.15.39:0/946496778 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f355001f430 con 0x5632b2b556b0 2023-07-21T18:41:59.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.890+0000 7f3571f27b80 10 client.15558 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T18:41:59.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.890+0000 7f3571f27b80 10 client.15558 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:41:59.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.890+0000 7f3571f27b80 20 client.15558 populate_metadata read hostname 'smithi039' 2023-07-21T18:41:59.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.890+0000 7f3571f27b80 1 --2- 172.21.15.39:0/946496778 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5632b2d16f30 0x5632b2d19340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:59.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.890+0000 7f3571f27b80 4 client.15558 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:41:59.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.890+0000 7f3571f27b80 1 -- 172.21.15.39:0/946496778 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5632b2c43e50 con 0x5632b2d16f30 2023-07-21T18:41:59.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.890+0000 7f355effd700 1 --2- 172.21.15.39:0/946496778 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5632b2d16f30 0x5632b2d19340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:59.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.891+0000 7f355effd700 1 --2- 172.21.15.39:0/946496778 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5632b2d16f30 0x5632b2d19340 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7232426bba86349a server_cookie=64fa9d13eed4539b in_seq=0 out_seq=0 2023-07-21T18:41:59.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.891+0000 7f353effd700 10 client.15558 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:41:59.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.891+0000 7f353effd700 1 -- 172.21.15.39:0/946496778 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5632b2c43e50 con 0x5632b2d16f30 2023-07-21T18:41:59.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.891+0000 7f353effd700 10 client.15558 handle_command_reply: tid=0 2023-07-21T18:41:59.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.892+0000 7f3571f27b80 1 client.15558 shutdown 2023-07-21T18:41:59.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.892+0000 7f3571f27b80 1 -- 172.21.15.39:0/946496778 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5632b2b556b0 msgr2=0x5632b2d09930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:59.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.892+0000 7f3571f27b80 1 --2- 172.21.15.39:0/946496778 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5632b2b556b0 0x5632b2d09930 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f3550011b30 tx=0x7f35500047c0 comp rx=0 tx=0).stop 2023-07-21T18:41:59.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.892+0000 7f3571f27b80 1 -- 172.21.15.39:0/946496778 shutdown_connections 2023-07-21T18:41:59.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.892+0000 7f3571f27b80 1 --2- 172.21.15.39:0/946496778 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5632b2d16f30 0x5632b2d19340 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:59.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.892+0000 7f3571f27b80 1 --2- 172.21.15.39:0/946496778 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632b2d03b10 0x5632b2d07150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:59.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.892+0000 7f3571f27b80 1 --2- 172.21.15.39:0/946496778 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5632b2b556b0 0x5632b2d09930 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:59.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.892+0000 7f353effd700 0 client.15558 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:41:59.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.892+0000 7f3571f27b80 1 --2- 172.21.15.39:0/946496778 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632b2d036a0 0x5632b2d06bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:59.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.892+0000 7f3571f27b80 1 -- 172.21.15.39:0/946496778 >> 172.21.15.39:0/946496778 conn(0x5632b2ceebe0 msgr2=0x5632b2cf34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:59.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.892+0000 7f3571f27b80 1 -- 172.21.15.39:0/946496778 shutdown_connections 2023-07-21T18:41:59.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.892+0000 7f3571f27b80 1 -- 172.21.15.39:0/946496778 wait complete. 2023-07-21T18:41:59.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.892+0000 7f3571f27b80 20 client.15558 trim_cache size 0 max 16384 2023-07-21T18:41:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.907+0000 7f3571f27b80 1 Processor -- start 2023-07-21T18:41:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.907+0000 7f3571f27b80 1 -- start start 2023-07-21T18:41:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.907+0000 7f3571f27b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632b2ceebe0 0x5632b2d0df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.907+0000 7f3571f27b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632b2b556b0 0x5632b2d045f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.907+0000 7f3571f27b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5632b2d036a0 0x5632b2d094d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.907+0000 7f3571f27b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632b2cbf060 con 0x5632b2d036a0 2023-07-21T18:41:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.907+0000 7f3571f27b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632b2cfe200 con 0x5632b2ceebe0 2023-07-21T18:41:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.907+0000 7f3571f27b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632b2d07cf0 con 0x5632b2b556b0 2023-07-21T18:41:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.907+0000 7f355ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5632b2d036a0 0x5632b2d094d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.907+0000 7f355ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5632b2d036a0 0x5632b2d094d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57496/0 (socket says 172.21.15.39:57496) 2023-07-21T18:41:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.908+0000 7f355ffff700 1 -- 172.21.15.39:0/3369497953 learned_addr learned my addr 172.21.15.39:0/3369497953 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.908+0000 7f355f7fe700 1 --2- 172.21.15.39:0/3369497953 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632b2ceebe0 0x5632b2d0df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.908+0000 7f355effd700 1 --2- 172.21.15.39:0/3369497953 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632b2b556b0 0x5632b2d045f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.908+0000 7f355ffff700 1 -- 172.21.15.39:0/3369497953 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632b2b556b0 msgr2=0x5632b2d045f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.908+0000 7f355ffff700 1 --2- 172.21.15.39:0/3369497953 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632b2b556b0 0x5632b2d045f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.908+0000 7f355ffff700 1 -- 172.21.15.39:0/3369497953 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632b2ceebe0 msgr2=0x5632b2d0df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.908+0000 7f355ffff700 1 --2- 172.21.15.39:0/3369497953 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632b2ceebe0 0x5632b2d0df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.908+0000 7f355ffff700 1 -- 172.21.15.39:0/3369497953 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632b2a9f1f0 con 0x5632b2d036a0 2023-07-21T18:41:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.908+0000 7f355ffff700 1 --2- 172.21.15.39:0/3369497953 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5632b2d036a0 0x5632b2d094d0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f35580969d0 tx=0x7f355800ae30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bead2179362faa66 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.908+0000 7f35467fc700 1 -- 172.21.15.39:0/3369497953 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3558099f60 con 0x5632b2d036a0 2023-07-21T18:41:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.908+0000 7f35467fc700 1 -- 172.21.15.39:0/3369497953 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f355809df90 con 0x5632b2d036a0 2023-07-21T18:41:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.908+0000 7f35467fc700 1 -- 172.21.15.39:0/3369497953 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355809a420 con 0x5632b2d036a0 2023-07-21T18:41:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.909+0000 7f3571f27b80 1 -- 172.21.15.39:0/3369497953 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5632b2d036a0 msgr2=0x5632b2d094d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:59.938 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.909+0000 7f3571f27b80 1 --2- 172.21.15.39:0/3369497953 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5632b2d036a0 0x5632b2d094d0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f35580969d0 tx=0x7f355800ae30 comp rx=0 tx=0).stop 2023-07-21T18:41:59.938 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.909+0000 7f3571f27b80 1 -- 172.21.15.39:0/3369497953 shutdown_connections 2023-07-21T18:41:59.938 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.909+0000 7f3571f27b80 1 --2- 172.21.15.39:0/3369497953 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632b2ceebe0 0x5632b2d0df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:59.938 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.909+0000 7f3571f27b80 1 --2- 172.21.15.39:0/3369497953 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5632b2d036a0 0x5632b2d094d0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:59.938 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.909+0000 7f3571f27b80 1 --2- 172.21.15.39:0/3369497953 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632b2b556b0 0x5632b2d045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:59.938 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.909+0000 7f3571f27b80 1 -- 172.21.15.39:0/3369497953 >> 172.21.15.39:0/3369497953 conn(0x5632b2d12a40 msgr2=0x5632b2d03ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:59.938 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.909+0000 7f3571f27b80 1 -- 172.21.15.39:0/3369497953 shutdown_connections 2023-07-21T18:41:59.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.909+0000 7f3571f27b80 1 -- 172.21.15.39:0/3369497953 wait complete. 2023-07-21T18:41:59.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.909+0000 7f3571f27b80 1 -- start start 2023-07-21T18:41:59.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.909+0000 7f3571f27b80 1 Processor -- start 2023-07-21T18:41:59.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.910+0000 7f3571f27b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632b2b556b0 0x5632b2d0e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:59.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.910+0000 7f3571f27b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632b2ceebe0 0x5632b2cfefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:59.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.910+0000 7f3571f27b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5632b2d036a0 0x5632b2cff600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:59.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.910+0000 7f3571f27b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632b2cffd10 con 0x5632b2d036a0 2023-07-21T18:41:59.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.910+0000 7f3571f27b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632b2cffe80 con 0x5632b2b556b0 2023-07-21T18:41:59.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.910+0000 7f3571f27b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632b2d6ae30 con 0x5632b2ceebe0 2023-07-21T18:41:59.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.910+0000 7f3571f27b80 10 client.0 _ll_register_callbacks cb 0x7f35500116e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:41:59.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.910+0000 7f355ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5632b2d036a0 0x5632b2cff600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:59.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.910+0000 7f355ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5632b2d036a0 0x5632b2cff600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57502/0 (socket says 172.21.15.39:57502) 2023-07-21T18:41:59.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.910+0000 7f355ffff700 1 -- 172.21.15.39:0/1838281883 learned_addr learned my addr 172.21.15.39:0/1838281883 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:41:59.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.910+0000 7f355effd700 1 --2- 172.21.15.39:0/1838281883 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632b2ceebe0 0x5632b2cfefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:59.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.910+0000 7f355f7fe700 1 --2- 172.21.15.39:0/1838281883 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632b2b556b0 0x5632b2d0e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:59.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.910+0000 7f355ffff700 1 -- 172.21.15.39:0/1838281883 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632b2ceebe0 msgr2=0x5632b2cfefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:59.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.910+0000 7f355ffff700 1 --2- 172.21.15.39:0/1838281883 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632b2ceebe0 0x5632b2cfefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:59.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.910+0000 7f355ffff700 1 -- 172.21.15.39:0/1838281883 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632b2b556b0 msgr2=0x5632b2d0e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:41:59.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.910+0000 7f355ffff700 1 --2- 172.21.15.39:0/1838281883 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632b2b556b0 0x5632b2d0e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:59.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.910+0000 7f355ffff700 1 -- 172.21.15.39:0/1838281883 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35601a0890 con 0x5632b2d036a0 2023-07-21T18:41:59.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.910+0000 7f355ffff700 1 --2- 172.21.15.39:0/1838281883 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5632b2d036a0 0x5632b2cff600 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f3558099c10 tx=0x7f3558007b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25a9a725b3932b45 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:41:59.942 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.910+0000 7f353effd700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:41:59.942 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.910+0000 7f353effd700 1 -- 172.21.15.39:0/1838281883 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3558099f60 con 0x5632b2d036a0 2023-07-21T18:41:59.942 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.910+0000 7f353effd700 1 -- 172.21.15.39:0/1838281883 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f35580128e0 con 0x5632b2d036a0 2023-07-21T18:41:59.942 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.911+0000 7f3571f27b80 1 -- 172.21.15.39:0/1838281883 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3554005040 con 0x5632b2d036a0 2023-07-21T18:41:59.942 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.911+0000 7f353effd700 1 -- 172.21.15.39:0/1838281883 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35580b82c0 con 0x5632b2d036a0 2023-07-21T18:41:59.942 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.911+0000 7f3571f27b80 10 client.15564 fetch_fsmap learned FSMap version 26 2023-07-21T18:41:59.942 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.911+0000 7f3571f27b80 1 -- 172.21.15.39:0/1838281883 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5632b2a77250 con 0x5632b2d036a0 2023-07-21T18:41:59.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.911+0000 7f353effd700 1 -- 172.21.15.39:0/1838281883 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3558063080 con 0x5632b2d036a0 2023-07-21T18:41:59.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.911+0000 7f353effd700 1 -- 172.21.15.39:0/1838281883 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f355809a1c0 con 0x5632b2d036a0 2023-07-21T18:41:59.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.911+0000 7f3571f27b80 10 client.15564 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T18:41:59.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.911+0000 7f3571f27b80 10 client.15564 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:41:59.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.911+0000 7f3571f27b80 20 client.15564 populate_metadata read hostname 'smithi039' 2023-07-21T18:41:59.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.911+0000 7f3571f27b80 1 --2- 172.21.15.39:0/1838281883 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5632b2d03b10 0x5632b2d6e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:41:59.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.911+0000 7f3571f27b80 4 client.15564 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:41:59.944 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.911+0000 7f3571f27b80 1 -- 172.21.15.39:0/1838281883 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5632b2c05ff0 con 0x5632b2d03b10 2023-07-21T18:41:59.944 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.911+0000 7f355f7fe700 1 --2- 172.21.15.39:0/1838281883 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5632b2d03b10 0x5632b2d6e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:41:59.944 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.912+0000 7f355f7fe700 1 --2- 172.21.15.39:0/1838281883 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5632b2d03b10 0x5632b2d6e2c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=578f4f45a4ff1ca6 server_cookie=815ca2884bf4e38c in_seq=0 out_seq=0 2023-07-21T18:41:59.944 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.912+0000 7f353effd700 10 client.15564 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:41:59.946 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.923+0000 7f353effd700 1 -- 172.21.15.39:0/1838281883 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5632b2c05ff0 con 0x5632b2d03b10 2023-07-21T18:41:59.946 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.923+0000 7f353effd700 10 client.15564 handle_command_reply: tid=0 2023-07-21T18:41:59.946 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.923+0000 7f3571f27b80 1 client.15564 shutdown 2023-07-21T18:41:59.947 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.924+0000 7f3571f27b80 1 -- 172.21.15.39:0/1838281883 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5632b2d036a0 msgr2=0x5632b2cff600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:59.947 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.924+0000 7f3571f27b80 1 --2- 172.21.15.39:0/1838281883 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5632b2d036a0 0x5632b2cff600 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f3558099c10 tx=0x7f3558007b20 comp rx=0 tx=0).stop 2023-07-21T18:41:59.947 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.924+0000 7f3571f27b80 1 -- 172.21.15.39:0/1838281883 shutdown_connections 2023-07-21T18:41:59.947 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.924+0000 7f3571f27b80 1 --2- 172.21.15.39:0/1838281883 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5632b2d03b10 0x5632b2d6e2c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:59.947 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.924+0000 7f3571f27b80 1 --2- 172.21.15.39:0/1838281883 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632b2b556b0 0x5632b2d0e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:59.948 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.924+0000 7f353effd700 0 client.15564 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:41:59.948 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.924+0000 7f3571f27b80 1 --2- 172.21.15.39:0/1838281883 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5632b2d036a0 0x5632b2cff600 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:59.948 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.924+0000 7f3571f27b80 1 --2- 172.21.15.39:0/1838281883 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632b2ceebe0 0x5632b2cfefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:59.949 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.924+0000 7f3571f27b80 1 -- 172.21.15.39:0/1838281883 >> 172.21.15.39:0/1838281883 conn(0x5632b2d12a40 msgr2=0x5632b2d04770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:59.949 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.924+0000 7f3571f27b80 1 -- 172.21.15.39:0/1838281883 shutdown_connections 2023-07-21T18:41:59.949 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.924+0000 7f3571f27b80 1 -- 172.21.15.39:0/1838281883 wait complete. 2023-07-21T18:41:59.949 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.924+0000 7f3571f27b80 20 client.15564 trim_cache size 0 max 16384 2023-07-21T18:41:59.949 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:41:59.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:41:59.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:41:59.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:41:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.924+0000 7f3565da1700 1 -- 172.21.15.39:0/3791771041 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f35480a1350 msgr2=0x7f35480a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.924+0000 7f3565da1700 1 --2- 172.21.15.39:0/3791771041 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f35480a1350 0x7f35480a3800 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f35500105c0 tx=0x7f355000b4a0 comp rx=0 tx=0).stop 2023-07-21T18:41:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.924+0000 7f3565da1700 1 -- 172.21.15.39:0/3791771041 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3560104a30 msgr2=0x7f3560193700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:41:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.924+0000 7f3565da1700 1 --2- 172.21.15.39:0/3791771041 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3560104a30 0x7f3560193700 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f355800cc50 tx=0x7f355800cf60 comp rx=0 tx=0).stop 2023-07-21T18:41:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.925+0000 7f3565da1700 1 -- 172.21.15.39:0/3791771041 shutdown_connections 2023-07-21T18:41:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.925+0000 7f3565da1700 1 --2- 172.21.15.39:0/3791771041 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3560104a30 0x7f3560193700 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:59.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.925+0000 7f3565da1700 1 --2- 172.21.15.39:0/3791771041 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35600fea30 0x7f35601998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:59.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.925+0000 7f3565da1700 1 --2- 172.21.15.39:0/3791771041 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f35480a1350 0x7f35480a3800 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:59.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.925+0000 7f3565da1700 1 --2- 172.21.15.39:0/3791771041 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35600ff3e0 0x7f3560199de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:41:59.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.925+0000 7f3565da1700 1 -- 172.21.15.39:0/3791771041 >> 172.21.15.39:0/3791771041 conn(0x7f3560074760 msgr2=0x7f3560105830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:41:59.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.925+0000 7f3565da1700 1 -- 172.21.15.39:0/3791771041 shutdown_connections 2023-07-21T18:41:59.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:41:59.925+0000 7f3565da1700 1 -- 172.21.15.39:0/3791771041 wait complete. 2023-07-21T18:42:00.369 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:42:00 smithi193 ceph-mon[98204]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 77 KiB/s wr, 35 op/s 2023-07-21T18:42:00.388 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:42:00 smithi155 ceph-mon[97934]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 77 KiB/s wr, 35 op/s 2023-07-21T18:42:00.437 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:42:00 smithi039 ceph-mon[96489]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 77 KiB/s wr, 35 op/s 2023-07-21T18:42:01.188 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:42:00 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:42:00.777+0000 7f9317fc1700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T18:42:01.188 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:42:00 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:42:00.777+0000 7f9317fc1700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T18:42:01.188 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:42:00 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:42:00.777+0000 7f9317fc1700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T18:42:01.189 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:42:00 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:42:00.777+0000 7f9317fc1700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T18:42:01.189 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 18:42:00 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T18:42:00.777+0000 7f9317fc1700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T18:42:02.369 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:42:02 smithi193 ceph-mon[98204]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 78 KiB/s wr, 44 op/s 2023-07-21T18:42:02.388 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:42:02 smithi155 ceph-mon[97934]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 78 KiB/s wr, 44 op/s 2023-07-21T18:42:02.437 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:42:02 smithi039 ceph-mon[96489]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 78 KiB/s wr, 44 op/s 2023-07-21T18:42:04.370 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:42:04 smithi193 ceph-mon[98204]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2023-07-21T18:42:04.370 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:42:04 smithi193 ceph-mon[98204]: mgrmap e23: x(active, since 18s), standbys: y, z 2023-07-21T18:42:04.388 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:42:04 smithi155 ceph-mon[97934]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2023-07-21T18:42:04.388 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:42:04 smithi155 ceph-mon[97934]: mgrmap e23: x(active, since 18s), standbys: y, z 2023-07-21T18:42:04.437 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:42:04 smithi039 ceph-mon[96489]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2023-07-21T18:42:04.437 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:42:04 smithi039 ceph-mon[96489]: mgrmap e23: x(active, since 18s), standbys: y, z 2023-07-21T18:42:05.746 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T18:42:05.889 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.867+0000 7f89409db700 1 -- 172.21.15.39:0/2872561364 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f893c0fe9e0 msgr2=0x7f893c0fee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:05.890 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.867+0000 7f89409db700 1 --2- 172.21.15.39:0/2872561364 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f893c0fe9e0 0x7f893c0fee50 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f8924009990 tx=0x7f8924009ca0 comp rx=0 tx=0).stop 2023-07-21T18:42:05.890 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.868+0000 7f89409db700 1 -- 172.21.15.39:0/2872561364 shutdown_connections 2023-07-21T18:42:05.890 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.868+0000 7f89409db700 1 --2- 172.21.15.39:0/2872561364 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f893c1049f0 0x7f893c104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:05.890 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.868+0000 7f89409db700 1 --2- 172.21.15.39:0/2872561364 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f893c0ff390 0x7f893c10b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:05.890 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.868+0000 7f89409db700 1 --2- 172.21.15.39:0/2872561364 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f893c0fe9e0 0x7f893c0fee50 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:05.890 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.868+0000 7f89409db700 1 -- 172.21.15.39:0/2872561364 >> 172.21.15.39:0/2872561364 conn(0x7f893c0fa4e0 msgr2=0x7f893c0fc8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:05.890 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.868+0000 7f89409db700 1 -- 172.21.15.39:0/2872561364 shutdown_connections 2023-07-21T18:42:05.891 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.868+0000 7f89409db700 1 -- 172.21.15.39:0/2872561364 wait complete. 2023-07-21T18:42:05.892 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.868+0000 7f89409db700 1 Processor -- start 2023-07-21T18:42:05.893 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.868+0000 7f89409db700 1 -- start start 2023-07-21T18:42:05.893 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.869+0000 7f89409db700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f893c0fe9e0 0x7f893c19b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:05.893 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.869+0000 7f89409db700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f893c0ff390 0x7f893c19bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:05.893 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.869+0000 7f89409db700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f893c1049f0 0x7f893c1959f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:05.893 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.869+0000 7f89409db700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f893c196060 con 0x7f893c0fe9e0 2023-07-21T18:42:05.893 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.869+0000 7f89409db700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f893c1961d0 con 0x7f893c1049f0 2023-07-21T18:42:05.893 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.869+0000 7f89409db700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f893c196390 con 0x7f893c0ff390 2023-07-21T18:42:05.894 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.869+0000 7f893a59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f893c0fe9e0 0x7f893c19b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:05.894 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.869+0000 7f893a59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f893c0fe9e0 0x7f893c19b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57528/0 (socket says 172.21.15.39:57528) 2023-07-21T18:42:05.894 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.869+0000 7f893a59c700 1 -- 172.21.15.39:0/1414258810 learned_addr learned my addr 172.21.15.39:0/1414258810 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:05.894 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.869+0000 7f893ad9d700 1 --2- 172.21.15.39:0/1414258810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f893c1049f0 0x7f893c1959f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:05.894 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.869+0000 7f8939d9b700 1 --2- 172.21.15.39:0/1414258810 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f893c0ff390 0x7f893c19bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:05.894 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.869+0000 7f893a59c700 1 -- 172.21.15.39:0/1414258810 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f893c0ff390 msgr2=0x7f893c19bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:05.894 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.869+0000 7f893a59c700 1 --2- 172.21.15.39:0/1414258810 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f893c0ff390 0x7f893c19bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:05.894 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.869+0000 7f893a59c700 1 -- 172.21.15.39:0/1414258810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f893c1049f0 msgr2=0x7f893c1959f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:05.895 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.869+0000 7f893a59c700 1 --2- 172.21.15.39:0/1414258810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f893c1049f0 0x7f893c1959f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:05.895 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.869+0000 7f893a59c700 1 -- 172.21.15.39:0/1414258810 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89240095f0 con 0x7f893c0fe9e0 2023-07-21T18:42:05.895 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.869+0000 7f893ad9d700 1 --2- 172.21.15.39:0/1414258810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f893c1049f0 0x7f893c1959f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:42:05.895 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.869+0000 7f893a59c700 1 --2- 172.21.15.39:0/1414258810 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f893c0fe9e0 0x7f893c19b980 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f892c00c970 tx=0x7f892c00cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:05.895 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.869+0000 7f89337fe700 1 -- 172.21.15.39:0/1414258810 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892c004220 con 0x7f893c0fe9e0 2023-07-21T18:42:05.895 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.870+0000 7f89337fe700 1 -- 172.21.15.39:0/1414258810 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f892c007d60 con 0x7f893c0fe9e0 2023-07-21T18:42:05.895 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.870+0000 7f89409db700 1 -- 172.21.15.39:0/1414258810 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f893c196670 con 0x7f893c0fe9e0 2023-07-21T18:42:05.896 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.870+0000 7f89337fe700 1 -- 172.21.15.39:0/1414258810 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892c003d00 con 0x7f893c0fe9e0 2023-07-21T18:42:05.896 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.870+0000 7f89409db700 1 -- 172.21.15.39:0/1414258810 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f893c1a2970 con 0x7f893c0fe9e0 2023-07-21T18:42:05.900 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.872+0000 7f89337fe700 1 -- 172.21.15.39:0/1414258810 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f892c004380 con 0x7f893c0fe9e0 2023-07-21T18:42:05.900 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.872+0000 7f89337fe700 1 --2- 172.21.15.39:0/1414258810 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f89280a13e0 0x7f89280a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:05.900 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.873+0000 7f89337fe700 1 -- 172.21.15.39:0/1414258810 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f892c0ccf00 con 0x7f893c0fe9e0 2023-07-21T18:42:05.900 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.873+0000 7f8939d9b700 1 --2- 172.21.15.39:0/1414258810 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f89280a13e0 0x7f89280a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:05.900 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.873+0000 7f8939d9b700 1 --2- 172.21.15.39:0/1414258810 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f89280a13e0 0x7f89280a3890 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8924006010 tx=0x7f892400b540 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:05.901 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.873+0000 7f894cb61b80 1 Processor -- start 2023-07-21T18:42:05.901 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.873+0000 7f894cb61b80 1 -- start start 2023-07-21T18:42:05.901 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.874+0000 7f894cb61b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55905a7c62d0 0x55905a7c66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:05.901 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.874+0000 7f894cb61b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55905a7c6be0 0x55905a7d1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:05.901 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.874+0000 7f894cb61b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55905a7d1ad0 0x55905a7d3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:05.901 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.874+0000 7f894cb61b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55905a6cccb0 con 0x55905a7c62d0 2023-07-21T18:42:05.901 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.874+0000 7f894cb61b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55905a71fa40 con 0x55905a7d1ad0 2023-07-21T18:42:05.901 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.874+0000 7f894cb61b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55905a718ed0 con 0x55905a7c6be0 2023-07-21T18:42:05.902 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.874+0000 7f893a59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55905a7c62d0 0x55905a7c66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:05.902 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.874+0000 7f893a59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55905a7c62d0 0x55905a7c66a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57536/0 (socket says 172.21.15.39:57536) 2023-07-21T18:42:05.902 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.874+0000 7f893a59c700 1 -- 172.21.15.39:0/3255672131 learned_addr learned my addr 172.21.15.39:0/3255672131 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:05.902 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.874+0000 7f8939d9b700 1 --2- 172.21.15.39:0/3255672131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55905a7c6be0 0x55905a7d1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:05.902 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.874+0000 7f893ad9d700 1 --2- 172.21.15.39:0/3255672131 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55905a7d1ad0 0x55905a7d3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:05.902 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.874+0000 7f893a59c700 1 -- 172.21.15.39:0/3255672131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55905a7c6be0 msgr2=0x55905a7d1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:05.902 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.874+0000 7f893a59c700 1 --2- 172.21.15.39:0/3255672131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55905a7c6be0 0x55905a7d1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:05.903 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.874+0000 7f893a59c700 1 -- 172.21.15.39:0/3255672131 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55905a7d1ad0 msgr2=0x55905a7d3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:05.903 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.874+0000 7f893a59c700 1 --2- 172.21.15.39:0/3255672131 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55905a7d1ad0 0x55905a7d3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:05.903 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.874+0000 7f893a59c700 1 -- 172.21.15.39:0/3255672131 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55905a546de0 con 0x55905a7c62d0 2023-07-21T18:42:05.903 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.874+0000 7f8939d9b700 1 --2- 172.21.15.39:0/3255672131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55905a7c6be0 0x55905a7d1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:42:05.903 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.874+0000 7f893ad9d700 1 --2- 172.21.15.39:0/3255672131 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55905a7d1ad0 0x55905a7d3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:42:05.903 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.874+0000 7f893a59c700 1 --2- 172.21.15.39:0/3255672131 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55905a7c62d0 0x55905a7c66a0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f892c0a1c60 tx=0x7f892c099880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2c64c788676787d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:05.903 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.874+0000 7f8930ff9700 1 -- 172.21.15.39:0/3255672131 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892c099d40 con 0x55905a7c62d0 2023-07-21T18:42:05.903 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.875+0000 7f8930ff9700 1 -- 172.21.15.39:0/3255672131 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f892c09dd70 con 0x55905a7c62d0 2023-07-21T18:42:05.904 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.875+0000 7f8930ff9700 1 -- 172.21.15.39:0/3255672131 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892c099230 con 0x55905a7c62d0 2023-07-21T18:42:05.904 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.875+0000 7f894cb61b80 1 -- 172.21.15.39:0/3255672131 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55905a7c62d0 msgr2=0x55905a7c66a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:05.904 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.875+0000 7f894cb61b80 1 --2- 172.21.15.39:0/3255672131 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55905a7c62d0 0x55905a7c66a0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f892c0a1c60 tx=0x7f892c099880 comp rx=0 tx=0).stop 2023-07-21T18:42:05.904 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.875+0000 7f894cb61b80 1 -- 172.21.15.39:0/3255672131 shutdown_connections 2023-07-21T18:42:05.904 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.875+0000 7f894cb61b80 1 --2- 172.21.15.39:0/3255672131 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55905a7d1ad0 0x55905a7d3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:05.904 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.875+0000 7f894cb61b80 1 --2- 172.21.15.39:0/3255672131 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55905a7c62d0 0x55905a7c66a0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:05.904 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.875+0000 7f894cb61b80 1 --2- 172.21.15.39:0/3255672131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55905a7c6be0 0x55905a7d1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:05.904 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.875+0000 7f894cb61b80 1 -- 172.21.15.39:0/3255672131 >> 172.21.15.39:0/3255672131 conn(0x55905a7bcda0 msgr2=0x55905a7c26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:05.905 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.875+0000 7f894cb61b80 1 -- 172.21.15.39:0/3255672131 shutdown_connections 2023-07-21T18:42:05.905 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.875+0000 7f894cb61b80 1 -- 172.21.15.39:0/3255672131 wait complete. 2023-07-21T18:42:05.905 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.876+0000 7f894cb61b80 1 -- start start 2023-07-21T18:42:05.905 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.876+0000 7f894cb61b80 1 Processor -- start 2023-07-21T18:42:05.905 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.876+0000 7f894cb61b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55905a7c62d0 0x55905a7d1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:05.905 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.876+0000 7f894cb61b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55905a7c6be0 0x55905a7ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:05.905 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.876+0000 7f894cb61b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55905a7d1ad0 0x55905a7ceec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:05.905 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.876+0000 7f894cb61b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55905a6b8ac0 con 0x55905a7c62d0 2023-07-21T18:42:05.906 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.876+0000 7f894cb61b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55905a6b87c0 con 0x55905a7d1ad0 2023-07-21T18:42:05.906 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.876+0000 7f894cb61b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55905a59c920 con 0x55905a7c6be0 2023-07-21T18:42:05.906 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.876+0000 7f894cb61b80 10 client.0 _ll_register_callbacks cb 0x55905a6b6760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:42:05.906 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.876+0000 7f893a59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55905a7c62d0 0x55905a7d1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:05.906 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.876+0000 7f893a59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55905a7c62d0 0x55905a7d1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57542/0 (socket says 172.21.15.39:57542) 2023-07-21T18:42:05.906 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.876+0000 7f893a59c700 1 -- 172.21.15.39:0/2564067303 learned_addr learned my addr 172.21.15.39:0/2564067303 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:05.906 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.876+0000 7f8939d9b700 1 --2- 172.21.15.39:0/2564067303 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55905a7c6be0 0x55905a7ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:05.907 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.877+0000 7f893ad9d700 1 --2- 172.21.15.39:0/2564067303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55905a7d1ad0 0x55905a7ceec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:05.907 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.877+0000 7f893a59c700 1 -- 172.21.15.39:0/2564067303 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55905a7c6be0 msgr2=0x55905a7ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:05.907 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.877+0000 7f893a59c700 1 --2- 172.21.15.39:0/2564067303 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55905a7c6be0 0x55905a7ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:05.907 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.877+0000 7f893a59c700 1 -- 172.21.15.39:0/2564067303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55905a7d1ad0 msgr2=0x55905a7ceec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:05.907 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.877+0000 7f893a59c700 1 --2- 172.21.15.39:0/2564067303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55905a7d1ad0 0x55905a7ceec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:05.907 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.877+0000 7f893a59c700 1 -- 172.21.15.39:0/2564067303 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f893c1a2970 con 0x55905a7c62d0 2023-07-21T18:42:05.907 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.877+0000 7f893ad9d700 1 --2- 172.21.15.39:0/2564067303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55905a7d1ad0 0x55905a7ceec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:42:05.907 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.877+0000 7f893a59c700 1 --2- 172.21.15.39:0/2564067303 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55905a7c62d0 0x55905a7d1500 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f892c099cd0 tx=0x7f892c099510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=264f42e42224ac80 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:05.908 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.877+0000 7f8921ffb700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:42:05.908 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.877+0000 7f8921ffb700 1 -- 172.21.15.39:0/2564067303 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892c09a1b0 con 0x55905a7c62d0 2023-07-21T18:42:05.908 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.877+0000 7f8921ffb700 1 -- 172.21.15.39:0/2564067303 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f892c0a1d00 con 0x55905a7c62d0 2023-07-21T18:42:05.908 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.877+0000 7f894cb61b80 1 -- 172.21.15.39:0/2564067303 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55905a6724f0 con 0x55905a7c62d0 2023-07-21T18:42:05.908 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.877+0000 7f8921ffb700 1 -- 172.21.15.39:0/2564067303 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f892c0a1ec0 con 0x55905a7c62d0 2023-07-21T18:42:05.908 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.877+0000 7f894cb61b80 10 client.15582 fetch_fsmap learned FSMap version 26 2023-07-21T18:42:05.909 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.877+0000 7f894cb61b80 1 -- 172.21.15.39:0/2564067303 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55905a52fc60 con 0x55905a7c62d0 2023-07-21T18:42:05.909 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.877+0000 7f8921ffb700 1 -- 172.21.15.39:0/2564067303 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f892c063080 con 0x55905a7c62d0 2023-07-21T18:42:05.909 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.877+0000 7f8921ffb700 1 -- 172.21.15.39:0/2564067303 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f892c0ca020 con 0x55905a7c62d0 2023-07-21T18:42:05.909 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.878+0000 7f894cb61b80 10 client.15582 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T18:42:05.909 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.878+0000 7f894cb61b80 10 client.15582 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T18:42:05.909 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.878+0000 7f894cb61b80 20 client.15582 populate_metadata read hostname 'smithi039' 2023-07-21T18:42:05.909 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.878+0000 7f894cb61b80 1 --2- 172.21.15.39:0/2564067303 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55905a7e0f80 0x55905a7e3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:05.909 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.878+0000 7f894cb61b80 4 client.15582 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:42:05.910 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.878+0000 7f894cb61b80 1 -- 172.21.15.39:0/2564067303 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55905a711eb0 con 0x55905a7e0f80 2023-07-21T18:42:05.910 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.878+0000 7f8939d9b700 1 --2- 172.21.15.39:0/2564067303 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55905a7e0f80 0x55905a7e3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:05.910 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.878+0000 7f8939d9b700 1 --2- 172.21.15.39:0/2564067303 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55905a7e0f80 0x55905a7e3360 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58b780bb0bec48a7 server_cookie=b40196975887d420 in_seq=0 out_seq=0 2023-07-21T18:42:05.910 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.878+0000 7f8921ffb700 10 client.15582 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:42:05.911 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.879+0000 7f8921ffb700 1 -- 172.21.15.39:0/2564067303 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55905a711eb0 con 0x55905a7e0f80 2023-07-21T18:42:05.911 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.879+0000 7f8921ffb700 10 client.15582 handle_command_reply: tid=0 2023-07-21T18:42:05.911 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.879+0000 7f894cb61b80 1 client.15582 shutdown 2023-07-21T18:42:05.911 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.879+0000 7f894cb61b80 1 -- 172.21.15.39:0/2564067303 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55905a7c62d0 msgr2=0x55905a7d1500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:05.911 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.879+0000 7f894cb61b80 1 --2- 172.21.15.39:0/2564067303 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55905a7c62d0 0x55905a7d1500 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f892c099cd0 tx=0x7f892c099510 comp rx=0 tx=0).stop 2023-07-21T18:42:05.912 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.879+0000 7f894cb61b80 1 -- 172.21.15.39:0/2564067303 shutdown_connections 2023-07-21T18:42:05.912 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.879+0000 7f894cb61b80 1 --2- 172.21.15.39:0/2564067303 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55905a7e0f80 0x55905a7e3360 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:05.912 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.879+0000 7f894cb61b80 1 --2- 172.21.15.39:0/2564067303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55905a7d1ad0 0x55905a7ceec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:05.912 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.879+0000 7f894cb61b80 1 --2- 172.21.15.39:0/2564067303 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55905a7c62d0 0x55905a7d1500 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:05.912 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.879+0000 7f8921ffb700 0 client.15582 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:42:05.912 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.879+0000 7f894cb61b80 1 --2- 172.21.15.39:0/2564067303 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55905a7c6be0 0x55905a7ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:05.912 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.879+0000 7f894cb61b80 1 -- 172.21.15.39:0/2564067303 >> 172.21.15.39:0/2564067303 conn(0x55905a7bcda0 msgr2=0x55905a7c2290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:05.913 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.879+0000 7f894cb61b80 1 -- 172.21.15.39:0/2564067303 shutdown_connections 2023-07-21T18:42:05.913 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.879+0000 7f894cb61b80 1 -- 172.21.15.39:0/2564067303 wait complete. 2023-07-21T18:42:05.913 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.879+0000 7f894cb61b80 20 client.15582 trim_cache size 0 max 16384 2023-07-21T18:42:05.919 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.896+0000 7f894cb61b80 1 Processor -- start 2023-07-21T18:42:05.919 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.896+0000 7f894cb61b80 1 -- start start 2023-07-21T18:42:05.919 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.896+0000 7f894cb61b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55905a7c62d0 0x55905a7d10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:05.919 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.897+0000 7f894cb61b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55905a7c6be0 0x55905a7cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:05.919 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.897+0000 7f894cb61b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55905a7d1ad0 0x55905a7ce480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:05.919 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.897+0000 7f894cb61b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55905a7c6fb0 con 0x55905a7c6be0 2023-07-21T18:42:05.920 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.897+0000 7f894cb61b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55905a7c6a20 con 0x55905a7d1ad0 2023-07-21T18:42:05.920 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.897+0000 7f894cb61b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55905a7c3ad0 con 0x55905a7c62d0 2023-07-21T18:42:05.920 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.897+0000 7f8939d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55905a7c6be0 0x55905a7cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:05.920 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.897+0000 7f8939d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55905a7c6be0 0x55905a7cdf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57552/0 (socket says 172.21.15.39:57552) 2023-07-21T18:42:05.920 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.897+0000 7f8939d9b700 1 -- 172.21.15.39:0/2891047544 learned_addr learned my addr 172.21.15.39:0/2891047544 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:05.920 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.897+0000 7f893a59c700 1 --2- 172.21.15.39:0/2891047544 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55905a7c62d0 0x55905a7d10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:05.920 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.897+0000 7f893ad9d700 1 --2- 172.21.15.39:0/2891047544 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55905a7d1ad0 0x55905a7ce480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:05.920 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.897+0000 7f8939d9b700 1 -- 172.21.15.39:0/2891047544 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55905a7c62d0 msgr2=0x55905a7d10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:05.921 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.897+0000 7f8939d9b700 1 --2- 172.21.15.39:0/2891047544 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55905a7c62d0 0x55905a7d10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:05.921 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.897+0000 7f8939d9b700 1 -- 172.21.15.39:0/2891047544 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55905a7d1ad0 msgr2=0x55905a7ce480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:05.921 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.897+0000 7f8939d9b700 1 --2- 172.21.15.39:0/2891047544 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55905a7d1ad0 0x55905a7ce480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:05.921 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.897+0000 7f8939d9b700 1 -- 172.21.15.39:0/2891047544 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55905a56d250 con 0x55905a7c6be0 2023-07-21T18:42:05.921 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.897+0000 7f8939d9b700 1 --2- 172.21.15.39:0/2891047544 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55905a7c6be0 0x55905a7cdf40 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f8924015000 tx=0x7f892401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16d86a0dd8d42b22 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:05.921 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.897+0000 7f89317fa700 1 -- 172.21.15.39:0/2891047544 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8924032450 con 0x55905a7c6be0 2023-07-21T18:42:05.921 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.897+0000 7f89317fa700 1 -- 172.21.15.39:0/2891047544 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f892401e040 con 0x55905a7c6be0 2023-07-21T18:42:05.922 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.897+0000 7f89317fa700 1 -- 172.21.15.39:0/2891047544 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8924024410 con 0x55905a7c6be0 2023-07-21T18:42:05.925 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.898+0000 7f894cb61b80 1 -- 172.21.15.39:0/2891047544 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55905a7c6be0 msgr2=0x55905a7cdf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:05.925 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.898+0000 7f894cb61b80 1 --2- 172.21.15.39:0/2891047544 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55905a7c6be0 0x55905a7cdf40 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f8924015000 tx=0x7f892401f000 comp rx=0 tx=0).stop 2023-07-21T18:42:05.925 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.898+0000 7f894cb61b80 1 -- 172.21.15.39:0/2891047544 shutdown_connections 2023-07-21T18:42:05.925 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.898+0000 7f894cb61b80 1 --2- 172.21.15.39:0/2891047544 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55905a7d1ad0 0x55905a7ce480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:05.925 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.898+0000 7f894cb61b80 1 --2- 172.21.15.39:0/2891047544 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55905a7c62d0 0x55905a7d10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:05.925 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.898+0000 7f894cb61b80 1 --2- 172.21.15.39:0/2891047544 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55905a7c6be0 0x55905a7cdf40 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:05.926 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.898+0000 7f894cb61b80 1 -- 172.21.15.39:0/2891047544 >> 172.21.15.39:0/2891047544 conn(0x55905a7bcda0 msgr2=0x55905a7dd760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:05.926 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.898+0000 7f894cb61b80 1 -- 172.21.15.39:0/2891047544 shutdown_connections 2023-07-21T18:42:05.926 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.898+0000 7f894cb61b80 1 -- 172.21.15.39:0/2891047544 wait complete. 2023-07-21T18:42:05.926 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.898+0000 7f894cb61b80 1 -- start start 2023-07-21T18:42:05.926 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.898+0000 7f894cb61b80 1 Processor -- start 2023-07-21T18:42:05.926 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.899+0000 7f894cb61b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55905a7c62d0 0x55905a7d85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:05.926 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.899+0000 7f894cb61b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55905a7c6be0 0x55905a7d68c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:05.927 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.899+0000 7f894cb61b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55905a7d1ad0 0x55905a7d6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:05.927 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.899+0000 7f894cb61b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55905a7d7570 con 0x55905a7d1ad0 2023-07-21T18:42:05.927 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.899+0000 7f894cb61b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55905a7d76e0 con 0x55905a7c62d0 2023-07-21T18:42:05.927 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.899+0000 7f894cb61b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55905a834e20 con 0x55905a7c6be0 2023-07-21T18:42:05.927 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.899+0000 7f894cb61b80 10 client.0 _ll_register_callbacks cb 0x7f892c0a2190 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:42:05.927 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.899+0000 7f893ad9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55905a7d1ad0 0x55905a7d6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:05.927 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.899+0000 7f893ad9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55905a7d1ad0 0x55905a7d6ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57560/0 (socket says 172.21.15.39:57560) 2023-07-21T18:42:05.927 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.899+0000 7f893ad9d700 1 -- 172.21.15.39:0/3647422676 learned_addr learned my addr 172.21.15.39:0/3647422676 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:05.928 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.899+0000 7f8939d9b700 1 --2- 172.21.15.39:0/3647422676 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55905a7c6be0 0x55905a7d68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:05.928 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.899+0000 7f893a59c700 1 --2- 172.21.15.39:0/3647422676 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55905a7c62d0 0x55905a7d85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:05.928 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.899+0000 7f893ad9d700 1 -- 172.21.15.39:0/3647422676 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55905a7c6be0 msgr2=0x55905a7d68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:05.928 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.899+0000 7f893ad9d700 1 --2- 172.21.15.39:0/3647422676 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55905a7c6be0 0x55905a7d68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:05.928 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.899+0000 7f893ad9d700 1 -- 172.21.15.39:0/3647422676 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55905a7c62d0 msgr2=0x55905a7d85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:05.928 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.899+0000 7f893ad9d700 1 --2- 172.21.15.39:0/3647422676 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55905a7c62d0 0x55905a7d85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:05.928 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.899+0000 7f893ad9d700 1 -- 172.21.15.39:0/3647422676 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f892402c040 con 0x55905a7d1ad0 2023-07-21T18:42:05.928 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.899+0000 7f893a59c700 1 --2- 172.21.15.39:0/3647422676 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55905a7c62d0 0x55905a7d85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:42:05.929 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.899+0000 7f893ad9d700 1 --2- 172.21.15.39:0/3647422676 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55905a7d1ad0 0x55905a7d6ea0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f8934007720 tx=0x7f8934007a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f63010dfc2a42f5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:05.929 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.899+0000 7f8921ffb700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:42:05.929 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.899+0000 7f8921ffb700 1 -- 172.21.15.39:0/3647422676 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8934016830 con 0x55905a7d1ad0 2023-07-21T18:42:05.929 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.899+0000 7f8921ffb700 1 -- 172.21.15.39:0/3647422676 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f893400e050 con 0x55905a7d1ad0 2023-07-21T18:42:05.929 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.900+0000 7f894cb61b80 1 -- 172.21.15.39:0/3647422676 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f892c09dd70 con 0x55905a7d1ad0 2023-07-21T18:42:05.929 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.900+0000 7f8921ffb700 1 -- 172.21.15.39:0/3647422676 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8934016990 con 0x55905a7d1ad0 2023-07-21T18:42:05.929 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.900+0000 7f894cb61b80 10 client.15588 fetch_fsmap learned FSMap version 26 2023-07-21T18:42:05.930 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.900+0000 7f894cb61b80 1 -- 172.21.15.39:0/3647422676 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55905a5452b0 con 0x55905a7d1ad0 2023-07-21T18:42:05.930 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.900+0000 7f8921ffb700 1 -- 172.21.15.39:0/3647422676 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f892c09dd70 con 0x55905a7d1ad0 2023-07-21T18:42:05.930 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.900+0000 7f8921ffb700 1 -- 172.21.15.39:0/3647422676 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8934014040 con 0x55905a7d1ad0 2023-07-21T18:42:05.930 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.900+0000 7f894cb61b80 10 client.15588 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T18:42:05.930 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.900+0000 7f894cb61b80 10 client.15588 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T18:42:05.930 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.900+0000 7f894cb61b80 20 client.15588 populate_metadata read hostname 'smithi039' 2023-07-21T18:42:05.930 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.900+0000 7f894cb61b80 1 --2- 172.21.15.39:0/3647422676 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55905a8365e0 0x55905a8389f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:05.930 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.900+0000 7f894cb61b80 4 client.15588 mds_command: new command op to 24536 tid=0[{"prefix": "scrub status"}] 2023-07-21T18:42:05.931 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.900+0000 7f894cb61b80 1 -- 172.21.15.39:0/3647422676 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55905a6d4050 con 0x55905a8365e0 2023-07-21T18:42:05.931 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.900+0000 7f893a59c700 1 --2- 172.21.15.39:0/3647422676 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55905a8365e0 0x55905a8389f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:05.931 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.901+0000 7f8921ffb700 10 client.15588 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:42:05.931 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.901+0000 7f893a59c700 1 --2- 172.21.15.39:0/3647422676 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55905a8365e0 0x55905a8389f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52a4a8b9a5f10d7d server_cookie=27b5221caad11dc4 in_seq=0 out_seq=0 2023-07-21T18:42:05.931 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.901+0000 7f8921ffb700 1 -- 172.21.15.39:0/3647422676 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55905a6d4050 con 0x55905a8365e0 2023-07-21T18:42:05.931 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.901+0000 7f8921ffb700 10 client.15588 handle_command_reply: tid=0 2023-07-21T18:42:05.931 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.901+0000 7f894cb61b80 1 client.15588 shutdown 2023-07-21T18:42:05.931 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.901+0000 7f894cb61b80 1 -- 172.21.15.39:0/3647422676 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55905a7d1ad0 msgr2=0x55905a7d6ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:05.932 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.901+0000 7f894cb61b80 1 --2- 172.21.15.39:0/3647422676 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55905a7d1ad0 0x55905a7d6ea0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f8934007720 tx=0x7f8934007a30 comp rx=0 tx=0).stop 2023-07-21T18:42:05.932 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.901+0000 7f894cb61b80 1 -- 172.21.15.39:0/3647422676 shutdown_connections 2023-07-21T18:42:05.932 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.901+0000 7f894cb61b80 1 --2- 172.21.15.39:0/3647422676 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55905a8365e0 0x55905a8389f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:05.932 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.901+0000 7f894cb61b80 1 --2- 172.21.15.39:0/3647422676 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55905a7c62d0 0x55905a7d85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:05.932 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.901+0000 7f8921ffb700 0 client.15588 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:42:05.932 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.901+0000 7f894cb61b80 1 --2- 172.21.15.39:0/3647422676 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55905a7d1ad0 0x55905a7d6ea0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:05.932 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.901+0000 7f894cb61b80 1 --2- 172.21.15.39:0/3647422676 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55905a7c6be0 0x55905a7d68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:05.933 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.901+0000 7f894cb61b80 1 -- 172.21.15.39:0/3647422676 >> 172.21.15.39:0/3647422676 conn(0x55905a7bcda0 msgr2=0x55905a7c18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:05.933 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.901+0000 7f894cb61b80 1 -- 172.21.15.39:0/3647422676 shutdown_connections 2023-07-21T18:42:05.933 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.901+0000 7f894cb61b80 1 -- 172.21.15.39:0/3647422676 wait complete. 2023-07-21T18:42:05.933 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.901+0000 7f894cb61b80 20 client.15588 trim_cache size 0 max 16384 2023-07-21T18:42:05.933 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2023-07-21T18:42:05.933 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "no active scrubs running", 2023-07-21T18:42:05.934 INFO:teuthology.orchestra.run.smithi039.stdout: "scrubs": {} 2023-07-21T18:42:05.934 INFO:teuthology.orchestra.run.smithi039.stdout:} 2023-07-21T18:42:05.935 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.902+0000 7f89409db700 1 -- 172.21.15.39:0/1414258810 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f89280a13e0 msgr2=0x7f89280a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:05.935 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.902+0000 7f89409db700 1 --2- 172.21.15.39:0/1414258810 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f89280a13e0 0x7f89280a3890 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8924006010 tx=0x7f892400b540 comp rx=0 tx=0).stop 2023-07-21T18:42:05.935 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.902+0000 7f89409db700 1 -- 172.21.15.39:0/1414258810 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f893c0fe9e0 msgr2=0x7f893c19b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:05.935 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.902+0000 7f89409db700 1 --2- 172.21.15.39:0/1414258810 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f893c0fe9e0 0x7f893c19b980 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f892c00c970 tx=0x7f892c00cd30 comp rx=0 tx=0).stop 2023-07-21T18:42:05.935 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.902+0000 7f89409db700 1 -- 172.21.15.39:0/1414258810 shutdown_connections 2023-07-21T18:42:05.935 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.902+0000 7f89409db700 1 --2- 172.21.15.39:0/1414258810 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f89280a13e0 0x7f89280a3890 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:05.936 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.902+0000 7f89409db700 1 --2- 172.21.15.39:0/1414258810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f893c1049f0 0x7f893c1959f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:05.936 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.902+0000 7f89409db700 1 --2- 172.21.15.39:0/1414258810 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f893c0fe9e0 0x7f893c19b980 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:05.936 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.902+0000 7f89409db700 1 --2- 172.21.15.39:0/1414258810 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f893c0ff390 0x7f893c19bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:05.936 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.902+0000 7f89409db700 1 -- 172.21.15.39:0/1414258810 >> 172.21.15.39:0/1414258810 conn(0x7f893c0fa4e0 msgr2=0x7f893c0fbf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:05.936 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.902+0000 7f89409db700 1 -- 172.21.15.39:0/1414258810 shutdown_connections 2023-07-21T18:42:05.936 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:05.902+0000 7f89409db700 1 -- 172.21.15.39:0/1414258810 wait complete. 2023-07-21T18:42:05.937 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-21T18:42:05.937 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T18:42:06.064 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.042+0000 7f4cf69aa700 1 -- 172.21.15.39:0/4230963760 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4cf01049e0 msgr2=0x7f4cf0104db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:06.064 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.042+0000 7f4cf69aa700 1 --2- 172.21.15.39:0/4230963760 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4cf01049e0 0x7f4cf0104db0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0009990 tx=0x7f4ce0009ca0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.065 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.042+0000 7f4cf69aa700 1 -- 172.21.15.39:0/4230963760 shutdown_connections 2023-07-21T18:42:06.065 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.042+0000 7f4cf69aa700 1 --2- 172.21.15.39:0/4230963760 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4cf00ff380 0x7f4cf010b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.065 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.042+0000 7f4cf69aa700 1 --2- 172.21.15.39:0/4230963760 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4cf01049e0 0x7f4cf0104db0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.065 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.042+0000 7f4cf69aa700 1 --2- 172.21.15.39:0/4230963760 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4cf00fe9d0 0x7f4cf00fee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.065 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.042+0000 7f4cf69aa700 1 -- 172.21.15.39:0/4230963760 >> 172.21.15.39:0/4230963760 conn(0x7f4cf00fa490 msgr2=0x7f4cf00fc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:06.066 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.042+0000 7f4cf69aa700 1 -- 172.21.15.39:0/4230963760 shutdown_connections 2023-07-21T18:42:06.066 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.042+0000 7f4cf69aa700 1 -- 172.21.15.39:0/4230963760 wait complete. 2023-07-21T18:42:06.067 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.043+0000 7f4cf69aa700 1 Processor -- start 2023-07-21T18:42:06.068 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.043+0000 7f4cf69aa700 1 -- start start 2023-07-21T18:42:06.068 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.043+0000 7f4cf69aa700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4cf00fe9d0 0x7f4cf0196a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.068 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.043+0000 7f4cf69aa700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4cf00ff380 0x7f4cf0196f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.068 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.043+0000 7f4cf69aa700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4cf01049e0 0x7f4cf01a1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.069 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.043+0000 7f4cf69aa700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf0197880 con 0x7f4cf00fe9d0 2023-07-21T18:42:06.069 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.043+0000 7f4cf69aa700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf01a1fc0 con 0x7f4cf01049e0 2023-07-21T18:42:06.069 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.043+0000 7f4cf69aa700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf01a2180 con 0x7f4cf00ff380 2023-07-21T18:42:06.069 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.044+0000 7f4ceffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4cf00fe9d0 0x7f4cf0196a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.069 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.044+0000 7f4ceffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4cf00fe9d0 0x7f4cf0196a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57570/0 (socket says 172.21.15.39:57570) 2023-07-21T18:42:06.069 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.044+0000 7f4ceffff700 1 -- 172.21.15.39:0/313026987 learned_addr learned my addr 172.21.15.39:0/313026987 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:06.069 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.044+0000 7f4cf4f47700 1 --2- 172.21.15.39:0/313026987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4cf01049e0 0x7f4cf01a1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.070 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.044+0000 7f4cef7fe700 1 --2- 172.21.15.39:0/313026987 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4cf00ff380 0x7f4cf0196f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.070 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.044+0000 7f4ceffff700 1 -- 172.21.15.39:0/313026987 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4cf00ff380 msgr2=0x7f4cf0196f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:06.070 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.044+0000 7f4ceffff700 1 --2- 172.21.15.39:0/313026987 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4cf00ff380 0x7f4cf0196f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.070 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.044+0000 7f4ceffff700 1 -- 172.21.15.39:0/313026987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4cf01049e0 msgr2=0x7f4cf01a1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:06.070 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.044+0000 7f4ceffff700 1 --2- 172.21.15.39:0/313026987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4cf01049e0 0x7f4cf01a1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.071 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.044+0000 7f4ceffff700 1 -- 172.21.15.39:0/313026987 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ce00095f0 con 0x7f4cf00fe9d0 2023-07-21T18:42:06.071 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.044+0000 7f4cef7fe700 1 --2- 172.21.15.39:0/313026987 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4cf00ff380 0x7f4cf0196f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:42:06.071 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.044+0000 7f4ceffff700 1 --2- 172.21.15.39:0/313026987 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4cf00fe9d0 0x7f4cf0196a00 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f4ce000b5c0 tx=0x7f4ce000f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:06.071 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.044+0000 7f4ced7fa700 1 -- 172.21.15.39:0/313026987 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce001d070 con 0x7f4cf00fe9d0 2023-07-21T18:42:06.071 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.044+0000 7f4cf69aa700 1 -- 172.21.15.39:0/313026987 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cf01a2400 con 0x7f4cf00fe9d0 2023-07-21T18:42:06.071 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.044+0000 7f4ced7fa700 1 -- 172.21.15.39:0/313026987 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ce000fb60 con 0x7f4cf00fe9d0 2023-07-21T18:42:06.071 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.044+0000 7f4ced7fa700 1 -- 172.21.15.39:0/313026987 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce0017650 con 0x7f4cf00fe9d0 2023-07-21T18:42:06.071 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.044+0000 7f4cf69aa700 1 -- 172.21.15.39:0/313026987 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cf01a2880 con 0x7f4cf00fe9d0 2023-07-21T18:42:06.072 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.045+0000 7f4cf69aa700 1 -- 172.21.15.39:0/313026987 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cf0067300 con 0x7f4cf00fe9d0 2023-07-21T18:42:06.073 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.047+0000 7f4ced7fa700 1 -- 172.21.15.39:0/313026987 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f4ce000fcd0 con 0x7f4cf00fe9d0 2023-07-21T18:42:06.073 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.047+0000 7f4ced7fa700 1 --2- 172.21.15.39:0/313026987 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f4cd80a1430 0x7f4cd80a38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.073 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.047+0000 7f4cef7fe700 1 --2- 172.21.15.39:0/313026987 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f4cd80a1430 0x7f4cd80a38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.073 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.047+0000 7f4ced7fa700 1 -- 172.21.15.39:0/313026987 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f4ce00ce570 con 0x7f4cf00fe9d0 2023-07-21T18:42:06.074 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.048+0000 7f4cef7fe700 1 --2- 172.21.15.39:0/313026987 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f4cd80a1430 0x7f4cd80a38e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0067f40 tx=0x7f4ce400a680 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:06.074 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.051+0000 7f4ced7fa700 1 -- 172.21.15.39:0/313026987 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f4ce00977e0 con 0x7f4cf00fe9d0 2023-07-21T18:42:06.316 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.294+0000 7f4cf69aa700 1 -- 172.21.15.39:0/313026987 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4cf01a2aa0 con 0x7f4cf00fe9d0 2023-07-21T18:42:06.317 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.295+0000 7f4ced7fa700 1 -- 172.21.15.39:0/313026987 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10887 (secure 0 0 0) 0x7f4ce009ae00 con 0x7f4cf00fe9d0 2023-07-21T18:42:06.317 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:42:06.317 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/147318681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":147318681},{"type":"v1","addr":"172.21.15.39:6835","nonce":147318681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/1062891760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1062891760},{"type":"v1","addr":"172.21.15.39:6837","nonce":1062891760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T18:40:04.594553+0000","modified":"2023-07-21T18:40:11.989039+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24536,"mds_1":24500,"mds_2":24487,"mds_3":24464,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/3184273349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":3184273349},{"type":"v1","addr":"172.21.15.39:6839","nonce":3184273349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6841/640981220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":640981220},{"type":"v1","addr":"172.21.15.39:6841","nonce":640981220}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3106166804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3106166804},{"type":"v1","addr":"172.21.15.193:6833","nonce":3106166804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24437":{"gid":24437,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/1034755277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1034755277},{"type":"v1","addr":"172.21.15.155:6833","nonce":1034755277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/381560657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":381560657},{"type":"v1","addr":"172.21.15.193:6835","nonce":381560657}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6837/2626761843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2626761843},{"type":"v1","addr":"172.21.15.193:6837","nonce":2626761843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24464":{"gid":24464,"name":"e","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.155:6835/3854856286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3854856286},{"type":"v1","addr":"172.21.15.155:6835","nonce":3854856286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.193:6839/3148488639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3148488639},{"type":"v1","addr":"172.21.15.193:6839","nonce":3148488639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.155:6837/947670056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":947670056},{"type":"v1","addr":"172.21.15.155:6837","nonce":947670056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6839/431690342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":431690342},{"type":"v1","addr":"172.21.15.155:6839","nonce":431690342}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T18:42:06.319 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.297+0000 7f4cf69aa700 1 -- 172.21.15.39:0/313026987 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f4cd80a1430 msgr2=0x7f4cd80a38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:06.319 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.297+0000 7f4cf69aa700 1 --2- 172.21.15.39:0/313026987 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f4cd80a1430 0x7f4cd80a38e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0067f40 tx=0x7f4ce400a680 comp rx=0 tx=0).stop 2023-07-21T18:42:06.319 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.297+0000 7f4cf69aa700 1 -- 172.21.15.39:0/313026987 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4cf00fe9d0 msgr2=0x7f4cf0196a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:06.319 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.297+0000 7f4cf69aa700 1 --2- 172.21.15.39:0/313026987 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4cf00fe9d0 0x7f4cf0196a00 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f4ce000b5c0 tx=0x7f4ce000f670 comp rx=0 tx=0).stop 2023-07-21T18:42:06.320 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.297+0000 7f4cf69aa700 1 -- 172.21.15.39:0/313026987 shutdown_connections 2023-07-21T18:42:06.320 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.297+0000 7f4cf69aa700 1 --2- 172.21.15.39:0/313026987 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f4cd80a1430 0x7f4cd80a38e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.320 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.297+0000 7f4cf69aa700 1 --2- 172.21.15.39:0/313026987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4cf01049e0 0x7f4cf01a1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.320 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.297+0000 7f4cf69aa700 1 --2- 172.21.15.39:0/313026987 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4cf00fe9d0 0x7f4cf0196a00 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.320 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.297+0000 7f4cf69aa700 1 --2- 172.21.15.39:0/313026987 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4cf00ff380 0x7f4cf0196f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.320 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.297+0000 7f4cf69aa700 1 -- 172.21.15.39:0/313026987 >> 172.21.15.39:0/313026987 conn(0x7f4cf00fa490 msgr2=0x7f4cf0105830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:06.320 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.298+0000 7f4cf69aa700 1 -- 172.21.15.39:0/313026987 shutdown_connections 2023-07-21T18:42:06.321 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.298+0000 7f4cf69aa700 1 -- 172.21.15.39:0/313026987 wait complete. 2023-07-21T18:42:06.321 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 26 2023-07-21T18:42:06.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:42:06 smithi039 ceph-mon[96489]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2023-07-21T18:42:06.331 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-07-21T18:42:06.369 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:42:06 smithi193 ceph-mon[98204]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2023-07-21T18:42:06.388 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:42:06 smithi155 ceph-mon[97934]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2023-07-21T18:42:06.459 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.437+0000 7ff44b196700 1 -- 172.21.15.39:0/1030637551 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4440fe9c0 msgr2=0x7ff4440fee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:06.459 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.437+0000 7ff44b196700 1 --2- 172.21.15.39:0/1030637551 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4440fe9c0 0x7ff4440fee30 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ff430009990 tx=0x7ff430009ca0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.459 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.437+0000 7ff44b196700 1 -- 172.21.15.39:0/1030637551 shutdown_connections 2023-07-21T18:42:06.459 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.437+0000 7ff44b196700 1 --2- 172.21.15.39:0/1030637551 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4441049d0 0x7ff444104da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.460 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.437+0000 7ff44b196700 1 --2- 172.21.15.39:0/1030637551 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4440ff370 0x7ff44410b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.460 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.437+0000 7ff44b196700 1 --2- 172.21.15.39:0/1030637551 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4440fe9c0 0x7ff4440fee30 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.460 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.437+0000 7ff44b196700 1 -- 172.21.15.39:0/1030637551 >> 172.21.15.39:0/1030637551 conn(0x7ff4440fa4a0 msgr2=0x7ff4440fc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:06.460 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.437+0000 7ff44b196700 1 -- 172.21.15.39:0/1030637551 shutdown_connections 2023-07-21T18:42:06.460 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.437+0000 7ff44b196700 1 -- 172.21.15.39:0/1030637551 wait complete. 2023-07-21T18:42:06.460 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.437+0000 7ff44b196700 1 Processor -- start 2023-07-21T18:42:06.460 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.437+0000 7ff44b196700 1 -- start start 2023-07-21T18:42:06.462 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.438+0000 7ff44b196700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4440fe9c0 0x7ff44419a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.462 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.438+0000 7ff44b196700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4440ff370 0x7ff44419ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.462 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.438+0000 7ff44b196700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4441049d0 0x7ff4441948c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.462 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.438+0000 7ff44b196700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff444194f30 con 0x7ff4440fe9c0 2023-07-21T18:42:06.463 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.438+0000 7ff44b196700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4441950a0 con 0x7ff4441049d0 2023-07-21T18:42:06.463 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.438+0000 7ff44b196700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff444195260 con 0x7ff4440ff370 2023-07-21T18:42:06.463 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.438+0000 7ff448f32700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4440fe9c0 0x7ff44419a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.463 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.438+0000 7ff448f32700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4440fe9c0 0x7ff44419a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57582/0 (socket says 172.21.15.39:57582) 2023-07-21T18:42:06.463 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.438+0000 7ff448f32700 1 -- 172.21.15.39:0/549805333 learned_addr learned my addr 172.21.15.39:0/549805333 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:06.463 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.438+0000 7ff443fff700 1 --2- 172.21.15.39:0/549805333 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4440ff370 0x7ff44419ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.463 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.438+0000 7ff448f32700 1 -- 172.21.15.39:0/549805333 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4440ff370 msgr2=0x7ff44419ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:06.463 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.438+0000 7ff448f32700 1 --2- 172.21.15.39:0/549805333 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4440ff370 0x7ff44419ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.464 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.438+0000 7ff448f32700 1 -- 172.21.15.39:0/549805333 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4441049d0 msgr2=0x7ff4441948c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:06.464 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.438+0000 7ff449733700 1 --2- 172.21.15.39:0/549805333 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4441049d0 0x7ff4441948c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.464 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.438+0000 7ff448f32700 1 --2- 172.21.15.39:0/549805333 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4441049d0 0x7ff4441948c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.464 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.438+0000 7ff448f32700 1 -- 172.21.15.39:0/549805333 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4300095f0 con 0x7ff4440fe9c0 2023-07-21T18:42:06.464 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.438+0000 7ff443fff700 1 --2- 172.21.15.39:0/549805333 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4440ff370 0x7ff44419ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:42:06.464 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.438+0000 7ff448f32700 1 --2- 172.21.15.39:0/549805333 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4440fe9c0 0x7ff44419a850 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7ff43800d980 tx=0x7ff43800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:06.464 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.438+0000 7ff441ffb700 1 -- 172.21.15.39:0/549805333 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43800bb60 con 0x7ff4440fe9c0 2023-07-21T18:42:06.464 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.439+0000 7ff441ffb700 1 -- 172.21.15.39:0/549805333 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff438007620 con 0x7ff4440fe9c0 2023-07-21T18:42:06.465 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.439+0000 7ff44b196700 1 -- 172.21.15.39:0/549805333 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff444195540 con 0x7ff4440fe9c0 2023-07-21T18:42:06.465 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.439+0000 7ff441ffb700 1 -- 172.21.15.39:0/549805333 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43800f620 con 0x7ff4440fe9c0 2023-07-21T18:42:06.465 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.439+0000 7ff44b196700 1 -- 172.21.15.39:0/549805333 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4441a1840 con 0x7ff4440fe9c0 2023-07-21T18:42:06.465 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.441+0000 7ff441ffb700 1 -- 172.21.15.39:0/549805333 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7ff438010460 con 0x7ff4440fe9c0 2023-07-21T18:42:06.470 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.441+0000 7ff441ffb700 1 --2- 172.21.15.39:0/549805333 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7ff4340a1460 0x7ff4340a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.470 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.441+0000 7ff441ffb700 1 -- 172.21.15.39:0/549805333 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7ff4380cce90 con 0x7ff4440fe9c0 2023-07-21T18:42:06.470 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.441+0000 7ff443fff700 1 --2- 172.21.15.39:0/549805333 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7ff4340a1460 0x7ff4340a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.470 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.442+0000 7ff443fff700 1 --2- 172.21.15.39:0/549805333 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7ff4340a1460 0x7ff4340a3910 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff43000b5c0 tx=0x7ff430005fb0 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:06.470 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.442+0000 7ff45731cb80 1 Processor -- start 2023-07-21T18:42:06.470 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.442+0000 7ff45731cb80 1 -- start start 2023-07-21T18:42:06.471 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.442+0000 7ff45731cb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562eebda72d0 0x562eebda76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.471 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.442+0000 7ff45731cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eebda7be0 0x562eebdb47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.471 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.442+0000 7ff45731cb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562eebdb4ce0 0x562eebdb70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.471 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.442+0000 7ff45731cb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eebcadcb0 con 0x562eebda72d0 2023-07-21T18:42:06.471 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.442+0000 7ff45731cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eebd00a40 con 0x562eebda7be0 2023-07-21T18:42:06.471 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.442+0000 7ff45731cb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eebcf9ed0 con 0x562eebdb4ce0 2023-07-21T18:42:06.471 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.443+0000 7ff448f32700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562eebda72d0 0x562eebda76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.472 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.443+0000 7ff448f32700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562eebda72d0 0x562eebda76a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57590/0 (socket says 172.21.15.39:57590) 2023-07-21T18:42:06.472 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.443+0000 7ff448f32700 1 -- 172.21.15.39:0/4288010907 learned_addr learned my addr 172.21.15.39:0/4288010907 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:06.472 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.443+0000 7ff443fff700 1 --2- 172.21.15.39:0/4288010907 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eebda7be0 0x562eebdb47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.472 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.443+0000 7ff449733700 1 --2- 172.21.15.39:0/4288010907 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562eebdb4ce0 0x562eebdb70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.472 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.443+0000 7ff448f32700 1 -- 172.21.15.39:0/4288010907 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562eebdb4ce0 msgr2=0x562eebdb70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:06.472 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.443+0000 7ff448f32700 1 --2- 172.21.15.39:0/4288010907 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562eebdb4ce0 0x562eebdb70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.472 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.443+0000 7ff448f32700 1 -- 172.21.15.39:0/4288010907 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eebda7be0 msgr2=0x562eebdb47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:06.472 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.443+0000 7ff448f32700 1 --2- 172.21.15.39:0/4288010907 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eebda7be0 0x562eebdb47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.473 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.443+0000 7ff448f32700 1 -- 172.21.15.39:0/4288010907 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562eebb27de0 con 0x562eebda72d0 2023-07-21T18:42:06.473 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.443+0000 7ff449733700 1 --2- 172.21.15.39:0/4288010907 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562eebdb4ce0 0x562eebdb70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:42:06.473 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.443+0000 7ff448f32700 1 --2- 172.21.15.39:0/4288010907 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562eebda72d0 0x562eebda76a0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7ff438099a30 tx=0x7ff43800dee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b81a7d7845dfb85e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:06.473 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.443+0000 7ff42effd700 1 -- 172.21.15.39:0/4288010907 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43800bdc0 con 0x562eebda72d0 2023-07-21T18:42:06.473 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.443+0000 7ff42effd700 1 -- 172.21.15.39:0/4288010907 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff438099d30 con 0x562eebda72d0 2023-07-21T18:42:06.473 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.443+0000 7ff42effd700 1 -- 172.21.15.39:0/4288010907 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff438099220 con 0x562eebda72d0 2023-07-21T18:42:06.473 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.444+0000 7ff45731cb80 1 -- 172.21.15.39:0/4288010907 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562eebda72d0 msgr2=0x562eebda76a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:06.473 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.444+0000 7ff45731cb80 1 --2- 172.21.15.39:0/4288010907 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562eebda72d0 0x562eebda76a0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7ff438099a30 tx=0x7ff43800dee0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.474 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.444+0000 7ff45731cb80 1 -- 172.21.15.39:0/4288010907 shutdown_connections 2023-07-21T18:42:06.474 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.444+0000 7ff45731cb80 1 --2- 172.21.15.39:0/4288010907 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562eebdb4ce0 0x562eebdb70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.474 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.444+0000 7ff45731cb80 1 --2- 172.21.15.39:0/4288010907 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562eebda72d0 0x562eebda76a0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.474 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.444+0000 7ff45731cb80 1 --2- 172.21.15.39:0/4288010907 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eebda7be0 0x562eebdb47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.474 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.444+0000 7ff45731cb80 1 -- 172.21.15.39:0/4288010907 >> 172.21.15.39:0/4288010907 conn(0x562eebd9dda0 msgr2=0x562eebda36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:06.474 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.444+0000 7ff45731cb80 1 -- 172.21.15.39:0/4288010907 shutdown_connections 2023-07-21T18:42:06.474 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.444+0000 7ff45731cb80 1 -- 172.21.15.39:0/4288010907 wait complete. 2023-07-21T18:42:06.475 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.444+0000 7ff45731cb80 1 -- start start 2023-07-21T18:42:06.475 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.444+0000 7ff45731cb80 1 Processor -- start 2023-07-21T18:42:06.475 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.445+0000 7ff45731cb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562eebda72d0 0x562eebdb1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.475 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.445+0000 7ff45731cb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562eebda7be0 0x562eebdb1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.475 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.445+0000 7ff45731cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eebdb4ce0 0x562eebdb24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.475 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.445+0000 7ff45731cb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eebc99ac0 con 0x562eebda72d0 2023-07-21T18:42:06.475 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.445+0000 7ff45731cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eebc997c0 con 0x562eebdb4ce0 2023-07-21T18:42:06.475 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.445+0000 7ff45731cb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eebb7d920 con 0x562eebda7be0 2023-07-21T18:42:06.476 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.445+0000 7ff45731cb80 10 client.0 _ll_register_callbacks cb 0x562eebc97760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:42:06.476 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.445+0000 7ff448f32700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562eebda72d0 0x562eebdb1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.476 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.445+0000 7ff448f32700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562eebda72d0 0x562eebdb1a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57596/0 (socket says 172.21.15.39:57596) 2023-07-21T18:42:06.476 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.445+0000 7ff448f32700 1 -- 172.21.15.39:0/3538823520 learned_addr learned my addr 172.21.15.39:0/3538823520 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:06.476 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.445+0000 7ff449733700 1 --2- 172.21.15.39:0/3538823520 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eebdb4ce0 0x562eebdb24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.476 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.445+0000 7ff443fff700 1 --2- 172.21.15.39:0/3538823520 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562eebda7be0 0x562eebdb1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.476 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.445+0000 7ff448f32700 1 -- 172.21.15.39:0/3538823520 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562eebda7be0 msgr2=0x562eebdb1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:06.476 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.445+0000 7ff448f32700 1 --2- 172.21.15.39:0/3538823520 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562eebda7be0 0x562eebdb1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.477 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.445+0000 7ff448f32700 1 -- 172.21.15.39:0/3538823520 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eebdb4ce0 msgr2=0x562eebdb24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:06.477 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.445+0000 7ff448f32700 1 --2- 172.21.15.39:0/3538823520 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eebdb4ce0 0x562eebdb24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.477 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.445+0000 7ff448f32700 1 -- 172.21.15.39:0/3538823520 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4441a1840 con 0x562eebda72d0 2023-07-21T18:42:06.477 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.445+0000 7ff443fff700 1 --2- 172.21.15.39:0/3538823520 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562eebda7be0 0x562eebdb1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:42:06.477 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.445+0000 7ff449733700 1 --2- 172.21.15.39:0/3538823520 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eebdb4ce0 0x562eebdb24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:42:06.477 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.445+0000 7ff448f32700 1 --2- 172.21.15.39:0/3538823520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562eebda72d0 0x562eebdb1a30 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7ff4380a1c50 tx=0x7ff43809dd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ac8b59d72f68257 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:06.477 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.445+0000 7ff423fff700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:42:06.478 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.446+0000 7ff423fff700 1 -- 172.21.15.39:0/3538823520 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4380b8170 con 0x562eebda72d0 2023-07-21T18:42:06.478 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.446+0000 7ff423fff700 1 -- 172.21.15.39:0/3538823520 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7ff43809c070 con 0x562eebda72d0 2023-07-21T18:42:06.478 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.446+0000 7ff45731cb80 1 -- 172.21.15.39:0/3538823520 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562eebc534f0 con 0x562eebda72d0 2023-07-21T18:42:06.478 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.446+0000 7ff423fff700 1 -- 172.21.15.39:0/3538823520 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4380a2180 con 0x562eebda72d0 2023-07-21T18:42:06.478 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.446+0000 7ff45731cb80 10 client.15606 fetch_fsmap learned FSMap version 26 2023-07-21T18:42:06.478 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.446+0000 7ff45731cb80 1 -- 172.21.15.39:0/3538823520 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x562eebb10c60 con 0x562eebda72d0 2023-07-21T18:42:06.478 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.446+0000 7ff423fff700 1 -- 172.21.15.39:0/3538823520 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562eebc534f0 con 0x562eebda72d0 2023-07-21T18:42:06.478 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.446+0000 7ff423fff700 1 -- 172.21.15.39:0/3538823520 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff4380a1e10 con 0x562eebda72d0 2023-07-21T18:42:06.479 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.446+0000 7ff45731cb80 10 client.15606 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T18:42:06.479 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.446+0000 7ff45731cb80 10 client.15606 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2023-07-21T18:42:06.479 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.446+0000 7ff45731cb80 20 client.15606 populate_metadata read hostname 'smithi039' 2023-07-21T18:42:06.479 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.446+0000 7ff45731cb80 1 --2- 172.21.15.39:0/3538823520 >> [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] conn(0x562eebdc40a0 0x562eebdc6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.479 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.446+0000 7ff45731cb80 4 client.15606 mds_command: new command op to 14604 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:42:06.479 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.446+0000 7ff45731cb80 1 -- 172.21.15.39:0/3538823520 --> [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562eebcf2eb0 con 0x562eebdc40a0 2023-07-21T18:42:06.479 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.446+0000 7ff443fff700 1 --2- 172.21.15.39:0/3538823520 >> [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] conn(0x562eebdc40a0 0x562eebdc6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.479 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.447+0000 7ff423fff700 10 client.15606 ms_handle_connect on v2:172.21.15.39:6840/640981220 2023-07-21T18:42:06.480 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.447+0000 7ff443fff700 1 --2- 172.21.15.39:0/3538823520 >> [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] conn(0x562eebdc40a0 0x562eebdc6480 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c6494e3d543d3083 server_cookie=c9ac01a85094d1ea in_seq=0 out_seq=0 2023-07-21T18:42:06.480 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.447+0000 7ff423fff700 1 -- 172.21.15.39:0/3538823520 <== mds.4 v2:172.21.15.39:6840/640981220 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562eebcf2eb0 con 0x562eebdc40a0 2023-07-21T18:42:06.480 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.447+0000 7ff423fff700 10 client.15606 handle_command_reply: tid=0 2023-07-21T18:42:06.480 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.447+0000 7ff45731cb80 1 client.15606 shutdown 2023-07-21T18:42:06.480 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.447+0000 7ff45731cb80 1 -- 172.21.15.39:0/3538823520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562eebda72d0 msgr2=0x562eebdb1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:06.480 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.447+0000 7ff45731cb80 1 --2- 172.21.15.39:0/3538823520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562eebda72d0 0x562eebdb1a30 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7ff4380a1c50 tx=0x7ff43809dd60 comp rx=0 tx=0).stop 2023-07-21T18:42:06.480 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.448+0000 7ff45731cb80 1 -- 172.21.15.39:0/3538823520 shutdown_connections 2023-07-21T18:42:06.480 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.448+0000 7ff45731cb80 1 --2- 172.21.15.39:0/3538823520 >> [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] conn(0x562eebdc40a0 0x562eebdc6480 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.481 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.448+0000 7ff45731cb80 1 --2- 172.21.15.39:0/3538823520 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eebdb4ce0 0x562eebdb24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.481 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.448+0000 7ff423fff700 0 client.15606 ms_handle_reset on v2:172.21.15.39:6840/640981220 2023-07-21T18:42:06.481 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.448+0000 7ff45731cb80 1 --2- 172.21.15.39:0/3538823520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562eebda72d0 0x562eebdb1a30 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.481 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.448+0000 7ff45731cb80 1 --2- 172.21.15.39:0/3538823520 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562eebda7be0 0x562eebdb1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.481 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.448+0000 7ff45731cb80 1 -- 172.21.15.39:0/3538823520 >> 172.21.15.39:0/3538823520 conn(0x562eebd9dda0 msgr2=0x562eebda3290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:06.481 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.448+0000 7ff45731cb80 1 -- 172.21.15.39:0/3538823520 shutdown_connections 2023-07-21T18:42:06.481 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.448+0000 7ff45731cb80 1 -- 172.21.15.39:0/3538823520 wait complete. 2023-07-21T18:42:06.481 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.448+0000 7ff45731cb80 20 client.15606 trim_cache size 0 max 16384 2023-07-21T18:42:06.483 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.461+0000 7ff45731cb80 1 Processor -- start 2023-07-21T18:42:06.483 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.461+0000 7ff45731cb80 1 -- start start 2023-07-21T18:42:06.483 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.461+0000 7ff45731cb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562eebda72d0 0x562eebdbb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.483 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.461+0000 7ff45731cb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562eebda7be0 0x562eebdb1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.483 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.461+0000 7ff45731cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eebdb4ce0 0x562eebdb2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.484 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.461+0000 7ff45731cb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eebd9fec0 con 0x562eebda72d0 2023-07-21T18:42:06.484 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.461+0000 7ff45731cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eebda4d20 con 0x562eebdb4ce0 2023-07-21T18:42:06.484 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.461+0000 7ff45731cb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eebdb2fa0 con 0x562eebda7be0 2023-07-21T18:42:06.484 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.461+0000 7ff448f32700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562eebda72d0 0x562eebdbb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.484 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.461+0000 7ff448f32700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562eebda72d0 0x562eebdbb6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57604/0 (socket says 172.21.15.39:57604) 2023-07-21T18:42:06.484 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.461+0000 7ff448f32700 1 -- 172.21.15.39:0/3766999090 learned_addr learned my addr 172.21.15.39:0/3766999090 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:06.484 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.461+0000 7ff443fff700 1 --2- 172.21.15.39:0/3766999090 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562eebda7be0 0x562eebdb1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.484 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.461+0000 7ff448f32700 1 -- 172.21.15.39:0/3766999090 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562eebda7be0 msgr2=0x562eebdb1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:06.485 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.461+0000 7ff448f32700 1 --2- 172.21.15.39:0/3766999090 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562eebda7be0 0x562eebdb1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.485 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.461+0000 7ff448f32700 1 -- 172.21.15.39:0/3766999090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eebdb4ce0 msgr2=0x562eebdb2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:06.485 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.461+0000 7ff448f32700 1 --2- 172.21.15.39:0/3766999090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eebdb4ce0 0x562eebdb2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.485 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.461+0000 7ff448f32700 1 -- 172.21.15.39:0/3766999090 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562eebd9f5e0 con 0x562eebda72d0 2023-07-21T18:42:06.485 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.461+0000 7ff448f32700 1 --2- 172.21.15.39:0/3766999090 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562eebda72d0 0x562eebdbb6c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ff438096230 tx=0x7ff4380961a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7233174b27e97fd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:06.488 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.462+0000 7ff42f7fe700 1 -- 172.21.15.39:0/3766999090 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4380b8170 con 0x562eebda72d0 2023-07-21T18:42:06.489 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.462+0000 7ff42f7fe700 1 -- 172.21.15.39:0/3766999090 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4380ca950 con 0x562eebda72d0 2023-07-21T18:42:06.489 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.462+0000 7ff42f7fe700 1 -- 172.21.15.39:0/3766999090 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43809a1f0 con 0x562eebda72d0 2023-07-21T18:42:06.489 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.462+0000 7ff45731cb80 1 -- 172.21.15.39:0/3766999090 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562eebda72d0 msgr2=0x562eebdbb6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:06.489 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.462+0000 7ff45731cb80 1 --2- 172.21.15.39:0/3766999090 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562eebda72d0 0x562eebdbb6c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ff438096230 tx=0x7ff4380961a0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.489 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.462+0000 7ff45731cb80 1 -- 172.21.15.39:0/3766999090 shutdown_connections 2023-07-21T18:42:06.489 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.462+0000 7ff45731cb80 1 --2- 172.21.15.39:0/3766999090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eebdb4ce0 0x562eebdb2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.490 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.462+0000 7ff45731cb80 1 --2- 172.21.15.39:0/3766999090 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562eebda72d0 0x562eebdbb6c0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.490 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.462+0000 7ff45731cb80 1 --2- 172.21.15.39:0/3766999090 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562eebda7be0 0x562eebdb1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.490 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.462+0000 7ff45731cb80 1 -- 172.21.15.39:0/3766999090 >> 172.21.15.39:0/3766999090 conn(0x562eebd9dda0 msgr2=0x562eebdc0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:06.490 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.462+0000 7ff45731cb80 1 -- 172.21.15.39:0/3766999090 shutdown_connections 2023-07-21T18:42:06.490 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.462+0000 7ff45731cb80 1 -- 172.21.15.39:0/3766999090 wait complete. 2023-07-21T18:42:06.490 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.463+0000 7ff45731cb80 1 -- start start 2023-07-21T18:42:06.490 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.463+0000 7ff45731cb80 1 Processor -- start 2023-07-21T18:42:06.490 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.463+0000 7ff45731cb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562eebda72d0 0x562eebdb8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.491 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.463+0000 7ff45731cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eebda7be0 0x562eebdb9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.491 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.463+0000 7ff45731cb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562eebdb4ce0 0x562eebdb99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.491 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.463+0000 7ff45731cb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eebe16ff0 con 0x562eebda72d0 2023-07-21T18:42:06.491 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.463+0000 7ff45731cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eebe17160 con 0x562eebda7be0 2023-07-21T18:42:06.491 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.463+0000 7ff45731cb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eebe17320 con 0x562eebdb4ce0 2023-07-21T18:42:06.491 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.463+0000 7ff45731cb80 10 client.0 _ll_register_callbacks cb 0x7ff430005e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:42:06.491 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.463+0000 7ff448f32700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562eebda72d0 0x562eebdb8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.491 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.463+0000 7ff448f32700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562eebda72d0 0x562eebdb8f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57610/0 (socket says 172.21.15.39:57610) 2023-07-21T18:42:06.492 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.463+0000 7ff448f32700 1 -- 172.21.15.39:0/2852362160 learned_addr learned my addr 172.21.15.39:0/2852362160 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:06.492 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.463+0000 7ff449733700 1 --2- 172.21.15.39:0/2852362160 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562eebdb4ce0 0x562eebdb99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.492 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.463+0000 7ff443fff700 1 --2- 172.21.15.39:0/2852362160 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eebda7be0 0x562eebdb9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.492 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.464+0000 7ff448f32700 1 -- 172.21.15.39:0/2852362160 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562eebdb4ce0 msgr2=0x562eebdb99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:06.492 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.464+0000 7ff448f32700 1 --2- 172.21.15.39:0/2852362160 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562eebdb4ce0 0x562eebdb99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.492 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.464+0000 7ff448f32700 1 -- 172.21.15.39:0/2852362160 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eebda7be0 msgr2=0x562eebdb9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:06.492 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.464+0000 7ff448f32700 1 --2- 172.21.15.39:0/2852362160 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eebda7be0 0x562eebdb9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.493 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.464+0000 7ff448f32700 1 -- 172.21.15.39:0/2852362160 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562eebb10c60 con 0x562eebda72d0 2023-07-21T18:42:06.493 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.464+0000 7ff449733700 1 --2- 172.21.15.39:0/2852362160 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562eebdb4ce0 0x562eebdb99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:42:06.493 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.464+0000 7ff448f32700 1 --2- 172.21.15.39:0/2852362160 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562eebda72d0 0x562eebdb8f30 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ff4380967a0 tx=0x7ff43809a320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1d6fc068b1f84b3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:06.493 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.464+0000 7ff423fff700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:42:06.493 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.464+0000 7ff423fff700 1 -- 172.21.15.39:0/2852362160 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4380a1ef0 con 0x562eebda72d0 2023-07-21T18:42:06.493 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.464+0000 7ff423fff700 1 -- 172.21.15.39:0/2852362160 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7ff43809c050 con 0x562eebda72d0 2023-07-21T18:42:06.493 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.464+0000 7ff45731cb80 1 -- 172.21.15.39:0/2852362160 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff438099d30 con 0x562eebda72d0 2023-07-21T18:42:06.493 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.464+0000 7ff423fff700 1 -- 172.21.15.39:0/2852362160 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4380b8170 con 0x562eebda72d0 2023-07-21T18:42:06.494 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.464+0000 7ff45731cb80 10 client.15612 fetch_fsmap learned FSMap version 26 2023-07-21T18:42:06.494 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.464+0000 7ff45731cb80 1 -- 172.21.15.39:0/2852362160 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x562eebb4e250 con 0x562eebda72d0 2023-07-21T18:42:06.494 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.464+0000 7ff423fff700 1 -- 172.21.15.39:0/2852362160 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff438099d30 con 0x562eebda72d0 2023-07-21T18:42:06.494 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.464+0000 7ff423fff700 1 -- 172.21.15.39:0/2852362160 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff438083030 con 0x562eebda72d0 2023-07-21T18:42:06.494 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.464+0000 7ff45731cb80 10 client.15612 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T18:42:06.494 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.464+0000 7ff45731cb80 10 client.15612 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2023-07-21T18:42:06.494 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.464+0000 7ff45731cb80 20 client.15612 populate_metadata read hostname 'smithi039' 2023-07-21T18:42:06.494 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.464+0000 7ff45731cb80 1 --2- 172.21.15.39:0/2852362160 >> [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] conn(0x562eebe18530 0x562eebe1a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.495 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.464+0000 7ff45731cb80 4 client.15612 mds_command: new command op to 14604 tid=0[{"prefix": "damage ls"}] 2023-07-21T18:42:06.495 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.464+0000 7ff45731cb80 1 -- 172.21.15.39:0/2852362160 --> [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562eebcb5050 con 0x562eebe18530 2023-07-21T18:42:06.495 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.465+0000 7ff443fff700 1 --2- 172.21.15.39:0/2852362160 >> [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] conn(0x562eebe18530 0x562eebe1a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.495 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.465+0000 7ff423fff700 10 client.15612 ms_handle_connect on v2:172.21.15.39:6840/640981220 2023-07-21T18:42:06.495 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.465+0000 7ff443fff700 1 --2- 172.21.15.39:0/2852362160 >> [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] conn(0x562eebe18530 0x562eebe1a910 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=dbb9312e47b674d0 server_cookie=aefb0d220c4cf281 in_seq=0 out_seq=0 2023-07-21T18:42:06.495 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.465+0000 7ff423fff700 1 -- 172.21.15.39:0/2852362160 <== mds.4 v2:172.21.15.39:6840/640981220 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562eebcb5050 con 0x562eebe18530 2023-07-21T18:42:06.495 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.465+0000 7ff423fff700 10 client.15612 handle_command_reply: tid=0 2023-07-21T18:42:06.495 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.465+0000 7ff45731cb80 1 client.15612 shutdown 2023-07-21T18:42:06.496 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.465+0000 7ff45731cb80 1 -- 172.21.15.39:0/2852362160 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562eebda72d0 msgr2=0x562eebdb8f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:06.496 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.465+0000 7ff45731cb80 1 --2- 172.21.15.39:0/2852362160 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562eebda72d0 0x562eebdb8f30 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ff4380967a0 tx=0x7ff43809a320 comp rx=0 tx=0).stop 2023-07-21T18:42:06.496 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.465+0000 7ff45731cb80 1 -- 172.21.15.39:0/2852362160 shutdown_connections 2023-07-21T18:42:06.496 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.465+0000 7ff45731cb80 1 --2- 172.21.15.39:0/2852362160 >> [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] conn(0x562eebe18530 0x562eebe1a910 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.496 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.465+0000 7ff45731cb80 1 --2- 172.21.15.39:0/2852362160 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562eebdb4ce0 0x562eebdb99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.496 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.465+0000 7ff423fff700 0 client.15612 ms_handle_reset on v2:172.21.15.39:6840/640981220 2023-07-21T18:42:06.496 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.465+0000 7ff45731cb80 1 --2- 172.21.15.39:0/2852362160 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562eebda72d0 0x562eebdb8f30 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.497 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.465+0000 7ff45731cb80 1 --2- 172.21.15.39:0/2852362160 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eebda7be0 0x562eebdb9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.497 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.465+0000 7ff45731cb80 1 -- 172.21.15.39:0/2852362160 >> 172.21.15.39:0/2852362160 conn(0x562eebd9dda0 msgr2=0x562eebdb1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:06.497 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.465+0000 7ff45731cb80 1 -- 172.21.15.39:0/2852362160 shutdown_connections 2023-07-21T18:42:06.497 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.465+0000 7ff45731cb80 1 -- 172.21.15.39:0/2852362160 wait complete. 2023-07-21T18:42:06.497 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.465+0000 7ff45731cb80 20 client.15612 trim_cache size 0 max 16384 2023-07-21T18:42:06.497 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2023-07-21T18:42:06.498 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.466+0000 7ff44b196700 1 -- 172.21.15.39:0/549805333 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7ff4340a1460 msgr2=0x7ff4340a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:06.498 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.466+0000 7ff44b196700 1 --2- 172.21.15.39:0/549805333 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7ff4340a1460 0x7ff4340a3910 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff43000b5c0 tx=0x7ff430005fb0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.498 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.466+0000 7ff44b196700 1 -- 172.21.15.39:0/549805333 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4440fe9c0 msgr2=0x7ff44419a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:06.499 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.466+0000 7ff44b196700 1 --2- 172.21.15.39:0/549805333 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4440fe9c0 0x7ff44419a850 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7ff43800d980 tx=0x7ff43800dd40 comp rx=0 tx=0).stop 2023-07-21T18:42:06.499 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.466+0000 7ff44b196700 1 -- 172.21.15.39:0/549805333 shutdown_connections 2023-07-21T18:42:06.499 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.466+0000 7ff44b196700 1 --2- 172.21.15.39:0/549805333 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7ff4340a1460 0x7ff4340a3910 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.499 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.466+0000 7ff44b196700 1 --2- 172.21.15.39:0/549805333 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4441049d0 0x7ff4441948c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.499 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.466+0000 7ff44b196700 1 --2- 172.21.15.39:0/549805333 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4440fe9c0 0x7ff44419a850 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.499 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.466+0000 7ff44b196700 1 --2- 172.21.15.39:0/549805333 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4440ff370 0x7ff44419ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.499 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.466+0000 7ff44b196700 1 -- 172.21.15.39:0/549805333 >> 172.21.15.39:0/549805333 conn(0x7ff4440fa4a0 msgr2=0x7ff4440fba30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:06.500 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.466+0000 7ff44b196700 1 -- 172.21.15.39:0/549805333 shutdown_connections 2023-07-21T18:42:06.500 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.466+0000 7ff44b196700 1 -- 172.21.15.39:0/549805333 wait complete. 2023-07-21T18:42:06.500 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-07-21T18:42:06.628 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.606+0000 7f24a71b8700 1 -- 172.21.15.39:0/3534933816 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24a01015b0 msgr2=0x7f24a0109aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:06.629 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.606+0000 7f24a71b8700 1 --2- 172.21.15.39:0/3534933816 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24a01015b0 0x7f24a0109aa0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f248c009990 tx=0x7f248c009ca0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.629 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.606+0000 7f24a71b8700 1 -- 172.21.15.39:0/3534933816 shutdown_connections 2023-07-21T18:42:06.629 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.606+0000 7f24a71b8700 1 --2- 172.21.15.39:0/3534933816 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24a0100ca0 0x7f24a0101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.629 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.606+0000 7f24a71b8700 1 --2- 172.21.15.39:0/3534933816 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24a010a070 0x7f24a010c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.629 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.606+0000 7f24a71b8700 1 --2- 172.21.15.39:0/3534933816 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24a01015b0 0x7f24a0109aa0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.629 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.606+0000 7f24a71b8700 1 -- 172.21.15.39:0/3534933816 >> 172.21.15.39:0/3534933816 conn(0x7f24a00fa4b0 msgr2=0x7f24a00fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:06.629 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.607+0000 7f24a71b8700 1 -- 172.21.15.39:0/3534933816 shutdown_connections 2023-07-21T18:42:06.629 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.607+0000 7f24a71b8700 1 -- 172.21.15.39:0/3534933816 wait complete. 2023-07-21T18:42:06.631 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.607+0000 7f24a71b8700 1 Processor -- start 2023-07-21T18:42:06.631 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.607+0000 7f24a71b8700 1 -- start start 2023-07-21T18:42:06.631 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.608+0000 7f24a71b8700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24a0100ca0 0x7f24a019b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.632 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.608+0000 7f24a71b8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24a01015b0 0x7f24a019bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.632 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.608+0000 7f24a71b8700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24a010a070 0x7f24a0195a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.632 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.608+0000 7f24a71b8700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a0196080 con 0x7f24a01015b0 2023-07-21T18:42:06.632 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.608+0000 7f24a71b8700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a01961f0 con 0x7f24a010a070 2023-07-21T18:42:06.632 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.608+0000 7f24a71b8700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a01963b0 con 0x7f24a0100ca0 2023-07-21T18:42:06.632 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.608+0000 7f249ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24a01015b0 0x7f24a019bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.632 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.608+0000 7f249ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24a01015b0 0x7f24a019bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57626/0 (socket says 172.21.15.39:57626) 2023-07-21T18:42:06.632 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.608+0000 7f249ffff700 1 -- 172.21.15.39:0/3672159104 learned_addr learned my addr 172.21.15.39:0/3672159104 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:06.633 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.608+0000 7f24a4f54700 1 --2- 172.21.15.39:0/3672159104 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24a0100ca0 0x7f24a019b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.633 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.608+0000 7f24a5755700 1 --2- 172.21.15.39:0/3672159104 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24a010a070 0x7f24a0195a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.633 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.608+0000 7f249ffff700 1 -- 172.21.15.39:0/3672159104 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24a0100ca0 msgr2=0x7f24a019b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:06.633 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.608+0000 7f249ffff700 1 --2- 172.21.15.39:0/3672159104 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24a0100ca0 0x7f24a019b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.633 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.608+0000 7f249ffff700 1 -- 172.21.15.39:0/3672159104 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24a010a070 msgr2=0x7f24a0195a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:06.633 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.608+0000 7f249ffff700 1 --2- 172.21.15.39:0/3672159104 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24a010a070 0x7f24a0195a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.633 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.608+0000 7f249ffff700 1 -- 172.21.15.39:0/3672159104 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f248c0095f0 con 0x7f24a01015b0 2023-07-21T18:42:06.633 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.608+0000 7f24a5755700 1 --2- 172.21.15.39:0/3672159104 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24a010a070 0x7f24a0195a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:42:06.634 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.608+0000 7f24a4f54700 1 --2- 172.21.15.39:0/3672159104 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24a0100ca0 0x7f24a019b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:42:06.634 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.608+0000 7f249ffff700 1 --2- 172.21.15.39:0/3672159104 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24a01015b0 0x7f24a019bee0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f248c000c00 tx=0x7f248c005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:06.634 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.608+0000 7f249dffb700 1 -- 172.21.15.39:0/3672159104 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248c01d070 con 0x7f24a01015b0 2023-07-21T18:42:06.634 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.609+0000 7f249dffb700 1 -- 172.21.15.39:0/3672159104 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f248c00fbc0 con 0x7f24a01015b0 2023-07-21T18:42:06.634 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.609+0000 7f24a71b8700 1 -- 172.21.15.39:0/3672159104 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24a0196630 con 0x7f24a01015b0 2023-07-21T18:42:06.634 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.609+0000 7f249dffb700 1 -- 172.21.15.39:0/3672159104 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248c017650 con 0x7f24a01015b0 2023-07-21T18:42:06.634 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.609+0000 7f24a71b8700 1 -- 172.21.15.39:0/3672159104 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24a01a2990 con 0x7f24a01015b0 2023-07-21T18:42:06.636 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.611+0000 7f249dffb700 1 -- 172.21.15.39:0/3672159104 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f248c021410 con 0x7f24a01015b0 2023-07-21T18:42:06.637 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.612+0000 7f249dffb700 1 --2- 172.21.15.39:0/3672159104 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f24900a13e0 0x7f24900a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.637 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.612+0000 7f24a4f54700 1 --2- 172.21.15.39:0/3672159104 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f24900a13e0 0x7f24900a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.637 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.612+0000 7f249dffb700 1 -- 172.21.15.39:0/3672159104 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f248c0ce480 con 0x7f24a01015b0 2023-07-21T18:42:06.637 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.612+0000 7f24a4f54700 1 --2- 172.21.15.39:0/3672159104 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f24900a13e0 0x7f24900a3890 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2494009cb0 tx=0x7f2494009260 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:06.637 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.613+0000 7f24b333eb80 1 Processor -- start 2023-07-21T18:42:06.637 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.613+0000 7f24b333eb80 1 -- start start 2023-07-21T18:42:06.638 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.613+0000 7f24b333eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56132fe4a2d0 0x56132fe4a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.638 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.613+0000 7f24b333eb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56132fe4abe0 0x56132fe55590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.638 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.613+0000 7f24b333eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56132fe55ad0 0x56132fe57eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.638 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.613+0000 7f24b333eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x56132fd50cb0 con 0x56132fe4a2d0 2023-07-21T18:42:06.638 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.613+0000 7f24b333eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56132fda3a40 con 0x56132fe55ad0 2023-07-21T18:42:06.638 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.613+0000 7f24b333eb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x56132fd9ced0 con 0x56132fe4abe0 2023-07-21T18:42:06.638 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.613+0000 7f24a4f54700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56132fe4a2d0 0x56132fe4a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.638 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.613+0000 7f24a4f54700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56132fe4a2d0 0x56132fe4a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57632/0 (socket says 172.21.15.39:57632) 2023-07-21T18:42:06.639 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.613+0000 7f24a4f54700 1 -- 172.21.15.39:0/3879607483 learned_addr learned my addr 172.21.15.39:0/3879607483 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:06.639 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.613+0000 7f24a5755700 1 --2- 172.21.15.39:0/3879607483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56132fe55ad0 0x56132fe57eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.639 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.613+0000 7f249ffff700 1 --2- 172.21.15.39:0/3879607483 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56132fe4abe0 0x56132fe55590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.639 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.613+0000 7f24a4f54700 1 -- 172.21.15.39:0/3879607483 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56132fe4abe0 msgr2=0x56132fe55590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:06.639 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.613+0000 7f24a4f54700 1 --2- 172.21.15.39:0/3879607483 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56132fe4abe0 0x56132fe55590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.639 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.613+0000 7f24a4f54700 1 -- 172.21.15.39:0/3879607483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56132fe55ad0 msgr2=0x56132fe57eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:06.639 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.613+0000 7f24a4f54700 1 --2- 172.21.15.39:0/3879607483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56132fe55ad0 0x56132fe57eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.639 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.614+0000 7f24a4f54700 1 -- 172.21.15.39:0/3879607483 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56132fbcade0 con 0x56132fe4a2d0 2023-07-21T18:42:06.640 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.614+0000 7f24a4f54700 1 --2- 172.21.15.39:0/3879607483 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56132fe4a2d0 0x56132fe4a6a0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f2494011fb0 tx=0x7f2494014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=592b67eefcbcba26 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:06.640 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.614+0000 7f248affd700 1 -- 172.21.15.39:0/3879607483 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2494003680 con 0x56132fe4a2d0 2023-07-21T18:42:06.640 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.614+0000 7f248affd700 1 -- 172.21.15.39:0/3879607483 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2494003ca0 con 0x56132fe4a2d0 2023-07-21T18:42:06.640 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.614+0000 7f248affd700 1 -- 172.21.15.39:0/3879607483 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2494012c60 con 0x56132fe4a2d0 2023-07-21T18:42:06.640 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.614+0000 7f24b333eb80 1 -- 172.21.15.39:0/3879607483 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56132fe4a2d0 msgr2=0x56132fe4a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:06.640 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.614+0000 7f24b333eb80 1 --2- 172.21.15.39:0/3879607483 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56132fe4a2d0 0x56132fe4a6a0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f2494011fb0 tx=0x7f2494014000 comp rx=0 tx=0).stop 2023-07-21T18:42:06.640 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.615+0000 7f24b333eb80 1 -- 172.21.15.39:0/3879607483 shutdown_connections 2023-07-21T18:42:06.641 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.615+0000 7f24b333eb80 1 --2- 172.21.15.39:0/3879607483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56132fe55ad0 0x56132fe57eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.641 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.615+0000 7f24b333eb80 1 --2- 172.21.15.39:0/3879607483 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56132fe4a2d0 0x56132fe4a6a0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.641 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.615+0000 7f24b333eb80 1 --2- 172.21.15.39:0/3879607483 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56132fe4abe0 0x56132fe55590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.641 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.615+0000 7f24b333eb80 1 -- 172.21.15.39:0/3879607483 >> 172.21.15.39:0/3879607483 conn(0x56132fe40da0 msgr2=0x56132fe466a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:06.641 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.615+0000 7f24b333eb80 1 -- 172.21.15.39:0/3879607483 shutdown_connections 2023-07-21T18:42:06.641 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.615+0000 7f24b333eb80 1 -- 172.21.15.39:0/3879607483 wait complete. 2023-07-21T18:42:06.641 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.615+0000 7f24b333eb80 1 -- start start 2023-07-21T18:42:06.641 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.615+0000 7f24b333eb80 1 Processor -- start 2023-07-21T18:42:06.644 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.615+0000 7f24b333eb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56132fe4a2d0 0x56132fe52850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.644 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.616+0000 7f24b333eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56132fe4abe0 0x56132fe52d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.644 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.616+0000 7f24b333eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56132fe55ad0 0x56132fe532d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.644 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.616+0000 7f24b333eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x56132fd3cac0 con 0x56132fe4abe0 2023-07-21T18:42:06.644 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.616+0000 7f24b333eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56132fd3c7c0 con 0x56132fe55ad0 2023-07-21T18:42:06.644 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.616+0000 7f24b333eb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x56132fc20920 con 0x56132fe4a2d0 2023-07-21T18:42:06.645 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.616+0000 7f24b333eb80 10 client.0 _ll_register_callbacks cb 0x56132fd3a760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:42:06.645 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.616+0000 7f249ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56132fe4abe0 0x56132fe52d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.645 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.616+0000 7f249ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56132fe4abe0 0x56132fe52d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57640/0 (socket says 172.21.15.39:57640) 2023-07-21T18:42:06.645 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.616+0000 7f249ffff700 1 -- 172.21.15.39:0/1870068168 learned_addr learned my addr 172.21.15.39:0/1870068168 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:06.645 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.616+0000 7f24a4f54700 1 --2- 172.21.15.39:0/1870068168 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56132fe4a2d0 0x56132fe52850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.645 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.616+0000 7f24a5755700 1 --2- 172.21.15.39:0/1870068168 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56132fe55ad0 0x56132fe532d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.645 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.616+0000 7f249ffff700 1 -- 172.21.15.39:0/1870068168 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56132fe4a2d0 msgr2=0x56132fe52850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:06.646 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.616+0000 7f249ffff700 1 --2- 172.21.15.39:0/1870068168 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56132fe4a2d0 0x56132fe52850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.646 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.616+0000 7f249ffff700 1 -- 172.21.15.39:0/1870068168 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56132fe55ad0 msgr2=0x56132fe532d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:06.646 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.616+0000 7f249ffff700 1 --2- 172.21.15.39:0/1870068168 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56132fe55ad0 0x56132fe532d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.646 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.616+0000 7f249ffff700 1 -- 172.21.15.39:0/1870068168 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2494011c60 con 0x56132fe4abe0 2023-07-21T18:42:06.646 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.616+0000 7f24a5755700 1 --2- 172.21.15.39:0/1870068168 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56132fe55ad0 0x56132fe532d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:42:06.646 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.616+0000 7f249ffff700 1 --2- 172.21.15.39:0/1870068168 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56132fe4abe0 0x56132fe52d90 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f248c097e20 tx=0x7f248c09b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=836756ebdd5c784 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:06.646 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.616+0000 7f247ffff700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:42:06.646 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.616+0000 7f247ffff700 1 -- 172.21.15.39:0/1870068168 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248c09b3b0 con 0x56132fe4abe0 2023-07-21T18:42:06.647 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.616+0000 7f24b333eb80 1 -- 172.21.15.39:0/1870068168 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56132fcf64f0 con 0x56132fe4abe0 2023-07-21T18:42:06.647 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.616+0000 7f247ffff700 1 -- 172.21.15.39:0/1870068168 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f248c09f3e0 con 0x56132fe4abe0 2023-07-21T18:42:06.647 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.616+0000 7f247ffff700 1 -- 172.21.15.39:0/1870068168 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f248c0a3770 con 0x56132fe4abe0 2023-07-21T18:42:06.647 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.616+0000 7f24b333eb80 10 client.15624 fetch_fsmap learned FSMap version 26 2023-07-21T18:42:06.647 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.617+0000 7f24b333eb80 1 -- 172.21.15.39:0/1870068168 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x56132fbb3c60 con 0x56132fe4abe0 2023-07-21T18:42:06.647 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.617+0000 7f247ffff700 1 -- 172.21.15.39:0/1870068168 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56132fcf64f0 con 0x56132fe4abe0 2023-07-21T18:42:06.647 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.617+0000 7f247ffff700 1 -- 172.21.15.39:0/1870068168 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f248c084bb0 con 0x56132fe4abe0 2023-07-21T18:42:06.647 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.617+0000 7f24b333eb80 10 client.15624 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T18:42:06.648 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.617+0000 7f24b333eb80 10 client.15624 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2023-07-21T18:42:06.648 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.617+0000 7f24b333eb80 20 client.15624 populate_metadata read hostname 'smithi039' 2023-07-21T18:42:06.648 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.617+0000 7f24b333eb80 1 --2- 172.21.15.39:0/1870068168 >> [v2:172.21.15.155:6834/3854856286,v1:172.21.15.155:6835/3854856286] conn(0x56132fe650c0 0x56132fe674d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.648 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.617+0000 7f24b333eb80 4 client.15624 mds_command: new command op to 24464 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:42:06.648 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.617+0000 7f24b333eb80 1 -- 172.21.15.39:0/1870068168 --> [v2:172.21.15.155:6834/3854856286,v1:172.21.15.155:6835/3854856286] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56132fd95eb0 con 0x56132fe650c0 2023-07-21T18:42:06.648 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.617+0000 7f24a4f54700 1 --2- 172.21.15.39:0/1870068168 >> [v2:172.21.15.155:6834/3854856286,v1:172.21.15.155:6835/3854856286] conn(0x56132fe650c0 0x56132fe674d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.648 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.618+0000 7f247ffff700 10 client.15624 ms_handle_connect on v2:172.21.15.155:6834/3854856286 2023-07-21T18:42:06.648 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.618+0000 7f24a4f54700 1 --2- 172.21.15.39:0/1870068168 >> [v2:172.21.15.155:6834/3854856286,v1:172.21.15.155:6835/3854856286] conn(0x56132fe650c0 0x56132fe674d0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ff98b36f05f9c50c server_cookie=a532049158ce3b8d in_seq=0 out_seq=0 2023-07-21T18:42:06.649 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.618+0000 7f247ffff700 1 -- 172.21.15.39:0/1870068168 <== mds.3 v2:172.21.15.155:6834/3854856286 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56132fd95eb0 con 0x56132fe650c0 2023-07-21T18:42:06.649 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.618+0000 7f247ffff700 10 client.15624 handle_command_reply: tid=0 2023-07-21T18:42:06.649 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.618+0000 7f24b333eb80 1 client.15624 shutdown 2023-07-21T18:42:06.649 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.618+0000 7f24b333eb80 1 -- 172.21.15.39:0/1870068168 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56132fe4abe0 msgr2=0x56132fe52d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:06.649 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.618+0000 7f24b333eb80 1 --2- 172.21.15.39:0/1870068168 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56132fe4abe0 0x56132fe52d90 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f248c097e20 tx=0x7f248c09b8f0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.649 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.619+0000 7f24b333eb80 1 -- 172.21.15.39:0/1870068168 shutdown_connections 2023-07-21T18:42:06.649 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.619+0000 7f24b333eb80 1 --2- 172.21.15.39:0/1870068168 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56132fe55ad0 0x56132fe532d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.650 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.619+0000 7f24b333eb80 1 --2- 172.21.15.39:0/1870068168 >> [v2:172.21.15.155:6834/3854856286,v1:172.21.15.155:6835/3854856286] conn(0x56132fe650c0 0x56132fe674d0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.650 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.619+0000 7f24b333eb80 1 --2- 172.21.15.39:0/1870068168 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56132fe4a2d0 0x56132fe52850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.650 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.619+0000 7f24b333eb80 1 --2- 172.21.15.39:0/1870068168 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56132fe4abe0 0x56132fe52d90 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.650 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.619+0000 7f247ffff700 0 client.15624 ms_handle_reset on v2:172.21.15.155:6834/3854856286 2023-07-21T18:42:06.650 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.619+0000 7f24b333eb80 1 -- 172.21.15.39:0/1870068168 >> 172.21.15.39:0/1870068168 conn(0x56132fe40da0 msgr2=0x56132fe46230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:06.650 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.619+0000 7f24b333eb80 1 -- 172.21.15.39:0/1870068168 shutdown_connections 2023-07-21T18:42:06.650 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.619+0000 7f24b333eb80 1 -- 172.21.15.39:0/1870068168 wait complete. 2023-07-21T18:42:06.650 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.619+0000 7f24b333eb80 20 client.15624 trim_cache size 0 max 16384 2023-07-21T18:42:06.654 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.632+0000 7f24b333eb80 1 Processor -- start 2023-07-21T18:42:06.654 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.632+0000 7f24b333eb80 1 -- start start 2023-07-21T18:42:06.654 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.632+0000 7f24b333eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56132fe4a2d0 0x56132fe61760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.654 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.632+0000 7f24b333eb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56132fe4abe0 0x56132fe4db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.654 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.632+0000 7f24b333eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56132fe55ad0 0x56132fe4e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.655 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.632+0000 7f24b333eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x56132fe40970 con 0x56132fe55ad0 2023-07-21T18:42:06.655 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.632+0000 7f24b333eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56132fe50db0 con 0x56132fe4a2d0 2023-07-21T18:42:06.655 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.632+0000 7f24b333eb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x56132fe4a8f0 con 0x56132fe4abe0 2023-07-21T18:42:06.655 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.632+0000 7f24a5755700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56132fe55ad0 0x56132fe4e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.655 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.632+0000 7f24a5755700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56132fe55ad0 0x56132fe4e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57650/0 (socket says 172.21.15.39:57650) 2023-07-21T18:42:06.655 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.632+0000 7f24a5755700 1 -- 172.21.15.39:0/2880848369 learned_addr learned my addr 172.21.15.39:0/2880848369 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:06.655 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.632+0000 7f249ffff700 1 --2- 172.21.15.39:0/2880848369 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56132fe4abe0 0x56132fe4db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.655 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.632+0000 7f24a4f54700 1 --2- 172.21.15.39:0/2880848369 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56132fe4a2d0 0x56132fe61760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.656 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.632+0000 7f24a5755700 1 -- 172.21.15.39:0/2880848369 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56132fe4abe0 msgr2=0x56132fe4db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:06.656 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.632+0000 7f24a5755700 1 --2- 172.21.15.39:0/2880848369 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56132fe4abe0 0x56132fe4db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.656 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.632+0000 7f24a5755700 1 -- 172.21.15.39:0/2880848369 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56132fe4a2d0 msgr2=0x56132fe61760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:06.656 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.632+0000 7f24a5755700 1 --2- 172.21.15.39:0/2880848369 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56132fe4a2d0 0x56132fe61760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.656 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.632+0000 7f24a5755700 1 -- 172.21.15.39:0/2880848369 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56132fbf1250 con 0x56132fe55ad0 2023-07-21T18:42:06.656 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.632+0000 7f24a5755700 1 --2- 172.21.15.39:0/2880848369 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56132fe55ad0 0x56132fe4e070 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f2498008580 tx=0x7f249800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e49f50779c50192 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:06.659 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.633+0000 7f248b7fe700 1 -- 172.21.15.39:0/2880848369 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2498017860 con 0x56132fe55ad0 2023-07-21T18:42:06.659 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.633+0000 7f248b7fe700 1 -- 172.21.15.39:0/2880848369 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2498011040 con 0x56132fe55ad0 2023-07-21T18:42:06.659 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.633+0000 7f248b7fe700 1 -- 172.21.15.39:0/2880848369 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2498007b50 con 0x56132fe55ad0 2023-07-21T18:42:06.660 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.633+0000 7f24b333eb80 1 -- 172.21.15.39:0/2880848369 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56132fe55ad0 msgr2=0x56132fe4e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:06.660 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.633+0000 7f24b333eb80 1 --2- 172.21.15.39:0/2880848369 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56132fe55ad0 0x56132fe4e070 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f2498008580 tx=0x7f249800b040 comp rx=0 tx=0).stop 2023-07-21T18:42:06.660 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.633+0000 7f24b333eb80 1 -- 172.21.15.39:0/2880848369 shutdown_connections 2023-07-21T18:42:06.660 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.633+0000 7f24b333eb80 1 --2- 172.21.15.39:0/2880848369 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56132fe4a2d0 0x56132fe61760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.660 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.633+0000 7f24b333eb80 1 --2- 172.21.15.39:0/2880848369 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56132fe55ad0 0x56132fe4e070 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.660 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.633+0000 7f24b333eb80 1 --2- 172.21.15.39:0/2880848369 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56132fe4abe0 0x56132fe4db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.660 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.633+0000 7f24b333eb80 1 -- 172.21.15.39:0/2880848369 >> 172.21.15.39:0/2880848369 conn(0x56132fe40da0 msgr2=0x56132fe6be30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:06.661 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.633+0000 7f24b333eb80 1 -- 172.21.15.39:0/2880848369 shutdown_connections 2023-07-21T18:42:06.661 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.633+0000 7f24b333eb80 1 -- 172.21.15.39:0/2880848369 wait complete. 2023-07-21T18:42:06.661 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.634+0000 7f24b333eb80 1 -- start start 2023-07-21T18:42:06.661 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.634+0000 7f24b333eb80 1 Processor -- start 2023-07-21T18:42:06.661 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.634+0000 7f24b333eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56132fe4a2d0 0x56132fe5c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.661 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.634+0000 7f24b333eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56132fe4abe0 0x56132fe5aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.661 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.634+0000 7f24b333eb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56132fe55ad0 0x56132fe5b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.661 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.634+0000 7f24b333eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x56132fe5b8b0 con 0x56132fe4a2d0 2023-07-21T18:42:06.662 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.634+0000 7f24b333eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56132febd3a0 con 0x56132fe4abe0 2023-07-21T18:42:06.662 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.634+0000 7f24b333eb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x56132febd4e0 con 0x56132fe55ad0 2023-07-21T18:42:06.662 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.634+0000 7f24b333eb80 10 client.0 _ll_register_callbacks cb 0x56132fd3a760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:42:06.662 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.634+0000 7f24a4f54700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56132fe4a2d0 0x56132fe5c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.662 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.634+0000 7f24a4f54700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56132fe4a2d0 0x56132fe5c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57652/0 (socket says 172.21.15.39:57652) 2023-07-21T18:42:06.662 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.634+0000 7f24a4f54700 1 -- 172.21.15.39:0/3836404401 learned_addr learned my addr 172.21.15.39:0/3836404401 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:06.662 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.634+0000 7f249ffff700 1 --2- 172.21.15.39:0/3836404401 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56132fe4abe0 0x56132fe5aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.662 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.634+0000 7f24a5755700 1 --2- 172.21.15.39:0/3836404401 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56132fe55ad0 0x56132fe5b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.663 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.635+0000 7f24a4f54700 1 -- 172.21.15.39:0/3836404401 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56132fe55ad0 msgr2=0x56132fe5b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:06.663 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.635+0000 7f24a4f54700 1 --2- 172.21.15.39:0/3836404401 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56132fe55ad0 0x56132fe5b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.663 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.635+0000 7f24a4f54700 1 -- 172.21.15.39:0/3836404401 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56132fe4abe0 msgr2=0x56132fe5aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:06.663 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.635+0000 7f24a4f54700 1 --2- 172.21.15.39:0/3836404401 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56132fe4abe0 0x56132fe5aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.663 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.635+0000 7f24a4f54700 1 -- 172.21.15.39:0/3836404401 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24980092c0 con 0x56132fe4a2d0 2023-07-21T18:42:06.663 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.635+0000 7f24a4f54700 1 --2- 172.21.15.39:0/3836404401 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56132fe4a2d0 0x56132fe5c730 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f2494023040 tx=0x7f2494004310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0d2ecaa8c1771f2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:06.663 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.635+0000 7f247ffff700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:42:06.663 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.635+0000 7f247ffff700 1 -- 172.21.15.39:0/3836404401 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2494032070 con 0x56132fe4a2d0 2023-07-21T18:42:06.664 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.635+0000 7f247ffff700 1 -- 172.21.15.39:0/3836404401 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f2494018070 con 0x56132fe4a2d0 2023-07-21T18:42:06.664 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.635+0000 7f24b333eb80 1 -- 172.21.15.39:0/3836404401 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2494003ca0 con 0x56132fe4a2d0 2023-07-21T18:42:06.664 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.635+0000 7f247ffff700 1 -- 172.21.15.39:0/3836404401 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2494003e10 con 0x56132fe4a2d0 2023-07-21T18:42:06.664 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.635+0000 7f24b333eb80 10 client.15630 fetch_fsmap learned FSMap version 26 2023-07-21T18:42:06.664 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.635+0000 7f24b333eb80 1 -- 172.21.15.39:0/3836404401 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x56132fbc92b0 con 0x56132fe4a2d0 2023-07-21T18:42:06.664 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.635+0000 7f247ffff700 1 -- 172.21.15.39:0/3836404401 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2494003ca0 con 0x56132fe4a2d0 2023-07-21T18:42:06.664 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.635+0000 7f247ffff700 1 -- 172.21.15.39:0/3836404401 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f24940327f0 con 0x56132fe4a2d0 2023-07-21T18:42:06.665 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.635+0000 7f24b333eb80 10 client.15630 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T18:42:06.665 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.635+0000 7f24b333eb80 10 client.15630 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2023-07-21T18:42:06.665 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.635+0000 7f24b333eb80 20 client.15630 populate_metadata read hostname 'smithi039' 2023-07-21T18:42:06.665 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.635+0000 7f24b333eb80 1 --2- 172.21.15.39:0/3836404401 >> [v2:172.21.15.155:6834/3854856286,v1:172.21.15.155:6835/3854856286] conn(0x56132febebe0 0x56132fec0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.665 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.636+0000 7f24b333eb80 4 client.15630 mds_command: new command op to 24464 tid=0[{"prefix": "damage ls"}] 2023-07-21T18:42:06.665 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.636+0000 7f24b333eb80 1 -- 172.21.15.39:0/3836404401 --> [v2:172.21.15.155:6834/3854856286,v1:172.21.15.155:6835/3854856286] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56132fd58050 con 0x56132febebe0 2023-07-21T18:42:06.665 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.636+0000 7f249ffff700 1 --2- 172.21.15.39:0/3836404401 >> [v2:172.21.15.155:6834/3854856286,v1:172.21.15.155:6835/3854856286] conn(0x56132febebe0 0x56132fec0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.665 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.636+0000 7f249ffff700 1 --2- 172.21.15.39:0/3836404401 >> [v2:172.21.15.155:6834/3854856286,v1:172.21.15.155:6835/3854856286] conn(0x56132febebe0 0x56132fec0ff0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=70f29efbcd7ea4f2 server_cookie=45cf8e5163d3e49e in_seq=0 out_seq=0 2023-07-21T18:42:06.666 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.636+0000 7f247ffff700 10 client.15630 ms_handle_connect on v2:172.21.15.155:6834/3854856286 2023-07-21T18:42:06.666 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.636+0000 7f247ffff700 1 -- 172.21.15.39:0/3836404401 <== mds.3 v2:172.21.15.155:6834/3854856286 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56132fd58050 con 0x56132febebe0 2023-07-21T18:42:06.666 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.636+0000 7f247ffff700 10 client.15630 handle_command_reply: tid=0 2023-07-21T18:42:06.666 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.636+0000 7f24b333eb80 1 client.15630 shutdown 2023-07-21T18:42:06.666 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.636+0000 7f24b333eb80 1 -- 172.21.15.39:0/3836404401 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56132fe4a2d0 msgr2=0x56132fe5c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:06.666 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.636+0000 7f24b333eb80 1 --2- 172.21.15.39:0/3836404401 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56132fe4a2d0 0x56132fe5c730 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f2494023040 tx=0x7f2494004310 comp rx=0 tx=0).stop 2023-07-21T18:42:06.666 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.637+0000 7f24b333eb80 1 -- 172.21.15.39:0/3836404401 shutdown_connections 2023-07-21T18:42:06.666 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.637+0000 7f24b333eb80 1 --2- 172.21.15.39:0/3836404401 >> [v2:172.21.15.155:6834/3854856286,v1:172.21.15.155:6835/3854856286] conn(0x56132febebe0 0x56132fec0ff0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.667 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.637+0000 7f24b333eb80 1 --2- 172.21.15.39:0/3836404401 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56132fe55ad0 0x56132fe5b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.667 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.637+0000 7f24b333eb80 1 --2- 172.21.15.39:0/3836404401 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56132fe4a2d0 0x56132fe5c730 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.667 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.637+0000 7f24b333eb80 1 --2- 172.21.15.39:0/3836404401 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56132fe4abe0 0x56132fe5aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.667 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.637+0000 7f247ffff700 0 client.15630 ms_handle_reset on v2:172.21.15.155:6834/3854856286 2023-07-21T18:42:06.667 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.637+0000 7f24b333eb80 1 -- 172.21.15.39:0/3836404401 >> 172.21.15.39:0/3836404401 conn(0x56132fe40da0 msgr2=0x56132fe61760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:06.667 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.637+0000 7f24b333eb80 1 -- 172.21.15.39:0/3836404401 shutdown_connections 2023-07-21T18:42:06.667 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.637+0000 7f24b333eb80 1 -- 172.21.15.39:0/3836404401 wait complete. 2023-07-21T18:42:06.667 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.637+0000 7f24b333eb80 20 client.15630 trim_cache size 0 max 16384 2023-07-21T18:42:06.668 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2023-07-21T18:42:06.668 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.637+0000 7f24a71b8700 1 -- 172.21.15.39:0/3672159104 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f24900a13e0 msgr2=0x7f24900a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:06.669 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.637+0000 7f24a71b8700 1 --2- 172.21.15.39:0/3672159104 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f24900a13e0 0x7f24900a3890 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2494009cb0 tx=0x7f2494009260 comp rx=0 tx=0).stop 2023-07-21T18:42:06.669 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.637+0000 7f24a71b8700 1 -- 172.21.15.39:0/3672159104 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24a01015b0 msgr2=0x7f24a019bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:06.669 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.637+0000 7f24a71b8700 1 --2- 172.21.15.39:0/3672159104 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24a01015b0 0x7f24a019bee0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f248c000c00 tx=0x7f248c005ef0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.669 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.637+0000 7f24a71b8700 1 -- 172.21.15.39:0/3672159104 shutdown_connections 2023-07-21T18:42:06.669 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.637+0000 7f24a71b8700 1 --2- 172.21.15.39:0/3672159104 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f24900a13e0 0x7f24900a3890 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.669 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.637+0000 7f24a71b8700 1 --2- 172.21.15.39:0/3672159104 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24a010a070 0x7f24a0195a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.669 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.637+0000 7f24a71b8700 1 --2- 172.21.15.39:0/3672159104 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24a0100ca0 0x7f24a019b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.670 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.637+0000 7f24a71b8700 1 --2- 172.21.15.39:0/3672159104 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24a01015b0 0x7f24a019bee0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.670 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.637+0000 7f24a71b8700 1 -- 172.21.15.39:0/3672159104 >> 172.21.15.39:0/3672159104 conn(0x7f24a00fa4b0 msgr2=0x7f24a010c220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:06.670 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.637+0000 7f24a71b8700 1 -- 172.21.15.39:0/3672159104 shutdown_connections 2023-07-21T18:42:06.670 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.637+0000 7f24a71b8700 1 -- 172.21.15.39:0/3672159104 wait complete. 2023-07-21T18:42:06.671 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-21T18:42:06.798 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.776+0000 7f16adaad700 1 -- 172.21.15.39:0/1072752738 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16a80ff380 msgr2=0x7f16a810b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:06.798 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.776+0000 7f16adaad700 1 --2- 172.21.15.39:0/1072752738 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16a80ff380 0x7f16a810b810 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f16a000b0a0 tx=0x7f16a000b3b0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.798 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.776+0000 7f16adaad700 1 -- 172.21.15.39:0/1072752738 shutdown_connections 2023-07-21T18:42:06.798 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.776+0000 7f16adaad700 1 --2- 172.21.15.39:0/1072752738 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16a80ff380 0x7f16a810b810 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.799 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.776+0000 7f16adaad700 1 --2- 172.21.15.39:0/1072752738 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16a81049e0 0x7f16a8104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.799 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.776+0000 7f16adaad700 1 --2- 172.21.15.39:0/1072752738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16a80fe9d0 0x7f16a80fee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.799 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.776+0000 7f16adaad700 1 -- 172.21.15.39:0/1072752738 >> 172.21.15.39:0/1072752738 conn(0x7f16a80fa4b0 msgr2=0x7f16a80fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:06.799 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.776+0000 7f16adaad700 1 -- 172.21.15.39:0/1072752738 shutdown_connections 2023-07-21T18:42:06.799 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.776+0000 7f16adaad700 1 -- 172.21.15.39:0/1072752738 wait complete. 2023-07-21T18:42:06.801 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.777+0000 7f16adaad700 1 Processor -- start 2023-07-21T18:42:06.801 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.777+0000 7f16adaad700 1 -- start start 2023-07-21T18:42:06.801 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.777+0000 7f16adaad700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16a80fe9d0 0x7f16a819ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.801 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.777+0000 7f16adaad700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16a80ff380 0x7f16a819bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.801 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.777+0000 7f16adaad700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16a81049e0 0x7f16a8195ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.801 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.777+0000 7f16adaad700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a8196120 con 0x7f16a81049e0 2023-07-21T18:42:06.801 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.777+0000 7f16adaad700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a8196290 con 0x7f16a80fe9d0 2023-07-21T18:42:06.802 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.777+0000 7f16adaad700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a8196450 con 0x7f16a80ff380 2023-07-21T18:42:06.802 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.777+0000 7f16a7fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16a81049e0 0x7f16a8195ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.802 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.777+0000 7f16a7fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16a81049e0 0x7f16a8195ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57670/0 (socket says 172.21.15.39:57670) 2023-07-21T18:42:06.802 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.777+0000 7f16a7fff700 1 -- 172.21.15.39:0/974951443 learned_addr learned my addr 172.21.15.39:0/974951443 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:06.802 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.777+0000 7f16a6ffd700 1 --2- 172.21.15.39:0/974951443 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16a80ff380 0x7f16a819bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.802 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.777+0000 7f16a77fe700 1 --2- 172.21.15.39:0/974951443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16a80fe9d0 0x7f16a819ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.802 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.778+0000 7f16a7fff700 1 -- 172.21.15.39:0/974951443 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16a80ff380 msgr2=0x7f16a819bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:06.803 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.778+0000 7f16a7fff700 1 --2- 172.21.15.39:0/974951443 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16a80ff380 0x7f16a819bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.803 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.778+0000 7f16a7fff700 1 -- 172.21.15.39:0/974951443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16a80fe9d0 msgr2=0x7f16a819ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:06.803 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.778+0000 7f16a7fff700 1 --2- 172.21.15.39:0/974951443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16a80fe9d0 0x7f16a819ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.803 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.778+0000 7f16a7fff700 1 -- 172.21.15.39:0/974951443 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a0009d00 con 0x7f16a81049e0 2023-07-21T18:42:06.803 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.778+0000 7f16a77fe700 1 --2- 172.21.15.39:0/974951443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16a80fe9d0 0x7f16a819ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:42:06.803 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.778+0000 7f16a7fff700 1 --2- 172.21.15.39:0/974951443 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16a81049e0 0x7f16a8195ab0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f16a000b780 tx=0x7f16a000be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:06.803 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.778+0000 7f16a4ff9700 1 -- 172.21.15.39:0/974951443 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a000e040 con 0x7f16a81049e0 2023-07-21T18:42:06.803 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.778+0000 7f16a4ff9700 1 -- 172.21.15.39:0/974951443 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16a0004e60 con 0x7f16a81049e0 2023-07-21T18:42:06.804 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.778+0000 7f16adaad700 1 -- 172.21.15.39:0/974951443 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16a81966d0 con 0x7f16a81049e0 2023-07-21T18:42:06.804 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.778+0000 7f16a4ff9700 1 -- 172.21.15.39:0/974951443 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a0005280 con 0x7f16a81049e0 2023-07-21T18:42:06.804 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.778+0000 7f16adaad700 1 -- 172.21.15.39:0/974951443 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16a81a2a30 con 0x7f16a81049e0 2023-07-21T18:42:06.807 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.780+0000 7f16a4ff9700 1 -- 172.21.15.39:0/974951443 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f16a001d470 con 0x7f16a81049e0 2023-07-21T18:42:06.807 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.781+0000 7f16a4ff9700 1 --2- 172.21.15.39:0/974951443 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f16900a13e0 0x7f16900a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.808 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.781+0000 7f16a77fe700 1 --2- 172.21.15.39:0/974951443 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f16900a13e0 0x7f16900a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.808 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.781+0000 7f16a4ff9700 1 -- 172.21.15.39:0/974951443 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f16a00ce630 con 0x7f16a81049e0 2023-07-21T18:42:06.808 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.781+0000 7f16b9c33b80 1 Processor -- start 2023-07-21T18:42:06.808 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.782+0000 7f16a77fe700 1 --2- 172.21.15.39:0/974951443 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f16900a13e0 0x7f16900a3890 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1698005fd0 tx=0x7f1698005f00 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:06.808 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.782+0000 7f16b9c33b80 1 -- start start 2023-07-21T18:42:06.808 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.782+0000 7f16b9c33b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d6b32c32d0 0x55d6b32c36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.808 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.782+0000 7f16b9c33b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d6b32c3be0 0x55d6b32ce590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.809 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.782+0000 7f16b9c33b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6b32cead0 0x55d6b32d0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.809 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.782+0000 7f16b9c33b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6b31c9cb0 con 0x55d6b32c32d0 2023-07-21T18:42:06.809 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.782+0000 7f16b9c33b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6b321ca40 con 0x55d6b32c3be0 2023-07-21T18:42:06.809 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.782+0000 7f16b9c33b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6b3215ed0 con 0x55d6b32cead0 2023-07-21T18:42:06.809 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.782+0000 7f16a77fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d6b32c32d0 0x55d6b32c36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.809 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.782+0000 7f16a77fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d6b32c32d0 0x55d6b32c36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57674/0 (socket says 172.21.15.39:57674) 2023-07-21T18:42:06.809 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.782+0000 7f16a77fe700 1 -- 172.21.15.39:0/253781670 learned_addr learned my addr 172.21.15.39:0/253781670 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:06.809 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.782+0000 7f16a6ffd700 1 --2- 172.21.15.39:0/253781670 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d6b32c3be0 0x55d6b32ce590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.810 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.782+0000 7f16a7fff700 1 --2- 172.21.15.39:0/253781670 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6b32cead0 0x55d6b32d0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.810 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.782+0000 7f16a77fe700 1 -- 172.21.15.39:0/253781670 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6b32cead0 msgr2=0x55d6b32d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:06.810 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.782+0000 7f16a77fe700 1 --2- 172.21.15.39:0/253781670 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6b32cead0 0x55d6b32d0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.810 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.782+0000 7f16a77fe700 1 -- 172.21.15.39:0/253781670 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d6b32c3be0 msgr2=0x55d6b32ce590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:06.810 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.782+0000 7f16a77fe700 1 --2- 172.21.15.39:0/253781670 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d6b32c3be0 0x55d6b32ce590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.810 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.782+0000 7f16a77fe700 1 -- 172.21.15.39:0/253781670 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d6b3043de0 con 0x55d6b32c32d0 2023-07-21T18:42:06.810 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.782+0000 7f16a7fff700 1 --2- 172.21.15.39:0/253781670 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6b32cead0 0x55d6b32d0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:42:06.810 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.783+0000 7f16a77fe700 1 --2- 172.21.15.39:0/253781670 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d6b32c32d0 0x55d6b32c36a0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f169800edb0 tx=0x7f1698011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e1d0479edaf0ffd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:06.811 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.783+0000 7f168dffb700 1 -- 172.21.15.39:0/253781670 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1698011be0 con 0x55d6b32c32d0 2023-07-21T18:42:06.811 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.783+0000 7f168dffb700 1 -- 172.21.15.39:0/253781670 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1698011d40 con 0x55d6b32c32d0 2023-07-21T18:42:06.811 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.783+0000 7f168dffb700 1 -- 172.21.15.39:0/253781670 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1698010630 con 0x55d6b32c32d0 2023-07-21T18:42:06.811 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.783+0000 7f16b9c33b80 1 -- 172.21.15.39:0/253781670 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d6b32c32d0 msgr2=0x55d6b32c36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:06.811 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.783+0000 7f16b9c33b80 1 --2- 172.21.15.39:0/253781670 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d6b32c32d0 0x55d6b32c36a0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f169800edb0 tx=0x7f1698011000 comp rx=0 tx=0).stop 2023-07-21T18:42:06.811 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.783+0000 7f16b9c33b80 1 -- 172.21.15.39:0/253781670 shutdown_connections 2023-07-21T18:42:06.811 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.783+0000 7f16b9c33b80 1 --2- 172.21.15.39:0/253781670 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6b32cead0 0x55d6b32d0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.811 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.783+0000 7f16b9c33b80 1 --2- 172.21.15.39:0/253781670 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d6b32c32d0 0x55d6b32c36a0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.812 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.783+0000 7f16b9c33b80 1 --2- 172.21.15.39:0/253781670 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d6b32c3be0 0x55d6b32ce590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.812 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.783+0000 7f16b9c33b80 1 -- 172.21.15.39:0/253781670 >> 172.21.15.39:0/253781670 conn(0x55d6b32b9da0 msgr2=0x55d6b32bf6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:06.812 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.784+0000 7f16b9c33b80 1 -- 172.21.15.39:0/253781670 shutdown_connections 2023-07-21T18:42:06.812 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.784+0000 7f16b9c33b80 1 -- 172.21.15.39:0/253781670 wait complete. 2023-07-21T18:42:06.812 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.784+0000 7f16b9c33b80 1 -- start start 2023-07-21T18:42:06.812 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.784+0000 7f16b9c33b80 1 Processor -- start 2023-07-21T18:42:06.812 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.784+0000 7f16b9c33b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d6b32c32d0 0x55d6b32ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.813 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.784+0000 7f16b9c33b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6b32c3be0 0x55d6b32cb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.813 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.784+0000 7f16b9c33b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d6b32cead0 0x55d6b32cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.813 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.784+0000 7f16b9c33b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6b31b5ac0 con 0x55d6b32cead0 2023-07-21T18:42:06.813 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.784+0000 7f16b9c33b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6b31b57c0 con 0x55d6b32c32d0 2023-07-21T18:42:06.813 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.784+0000 7f16b9c33b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6b3099920 con 0x55d6b32c3be0 2023-07-21T18:42:06.813 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.784+0000 7f16b9c33b80 10 client.0 _ll_register_callbacks cb 0x55d6b31b3760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:42:06.813 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.784+0000 7f16a7fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d6b32cead0 0x55d6b32cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.813 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.785+0000 7f16a7fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d6b32cead0 0x55d6b32cbe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57684/0 (socket says 172.21.15.39:57684) 2023-07-21T18:42:06.814 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.785+0000 7f16a7fff700 1 -- 172.21.15.39:0/4185594007 learned_addr learned my addr 172.21.15.39:0/4185594007 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:06.814 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.785+0000 7f16a6ffd700 1 --2- 172.21.15.39:0/4185594007 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6b32c3be0 0x55d6b32cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.814 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.785+0000 7f16a77fe700 1 --2- 172.21.15.39:0/4185594007 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d6b32c32d0 0x55d6b32ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.814 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.785+0000 7f16a7fff700 1 -- 172.21.15.39:0/4185594007 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6b32c3be0 msgr2=0x55d6b32cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:06.814 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.785+0000 7f16a7fff700 1 --2- 172.21.15.39:0/4185594007 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6b32c3be0 0x55d6b32cb790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.814 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.785+0000 7f16a7fff700 1 -- 172.21.15.39:0/4185594007 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d6b32c32d0 msgr2=0x55d6b32ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:06.814 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.785+0000 7f16a7fff700 1 --2- 172.21.15.39:0/4185594007 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d6b32c32d0 0x55d6b32ce4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.814 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.785+0000 7f16a7fff700 1 -- 172.21.15.39:0/4185594007 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f169800ea90 con 0x55d6b32cead0 2023-07-21T18:42:06.815 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.785+0000 7f16a7fff700 1 --2- 172.21.15.39:0/4185594007 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d6b32cead0 0x55d6b32cbe10 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f16a0004fd0 tx=0x7f16a000b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77016f7622755c6c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:06.815 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.785+0000 7f1686ffd700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:42:06.815 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.785+0000 7f1686ffd700 1 -- 172.21.15.39:0/4185594007 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a009f360 con 0x55d6b32cead0 2023-07-21T18:42:06.815 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.785+0000 7f1686ffd700 1 -- 172.21.15.39:0/4185594007 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f16a002d2f0 con 0x55d6b32cead0 2023-07-21T18:42:06.815 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.785+0000 7f16b9c33b80 1 -- 172.21.15.39:0/4185594007 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d6b316f4f0 con 0x55d6b32cead0 2023-07-21T18:42:06.815 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.785+0000 7f1686ffd700 1 -- 172.21.15.39:0/4185594007 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16a009b560 con 0x55d6b32cead0 2023-07-21T18:42:06.815 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.785+0000 7f16b9c33b80 10 client.15642 fetch_fsmap learned FSMap version 26 2023-07-21T18:42:06.815 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.785+0000 7f16b9c33b80 1 -- 172.21.15.39:0/4185594007 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55d6b302cc60 con 0x55d6b32cead0 2023-07-21T18:42:06.816 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.785+0000 7f1686ffd700 1 -- 172.21.15.39:0/4185594007 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16a00b7080 con 0x55d6b32cead0 2023-07-21T18:42:06.816 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.786+0000 7f1686ffd700 1 -- 172.21.15.39:0/4185594007 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f16a00a3490 con 0x55d6b32cead0 2023-07-21T18:42:06.816 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.786+0000 7f16b9c33b80 10 client.15642 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T18:42:06.816 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.786+0000 7f16b9c33b80 10 client.15642 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T18:42:06.816 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.786+0000 7f16b9c33b80 20 client.15642 populate_metadata read hostname 'smithi039' 2023-07-21T18:42:06.816 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.786+0000 7f16b9c33b80 1 --2- 172.21.15.39:0/4185594007 >> [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] conn(0x55d6b32de1a0 0x55d6b32e05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.816 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.786+0000 7f16b9c33b80 4 client.15642 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:42:06.816 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.786+0000 7f16b9c33b80 1 -- 172.21.15.39:0/4185594007 --> [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d6b320eeb0 con 0x55d6b32de1a0 2023-07-21T18:42:06.818 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.786+0000 7f16a77fe700 1 --2- 172.21.15.39:0/4185594007 >> [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] conn(0x55d6b32de1a0 0x55d6b32e05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.818 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.787+0000 7f1686ffd700 10 client.15642 ms_handle_connect on v2:172.21.15.193:6838/3148488639 2023-07-21T18:42:06.818 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.787+0000 7f16a77fe700 1 --2- 172.21.15.39:0/4185594007 >> [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] conn(0x55d6b32de1a0 0x55d6b32e05b0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a545f76c13d87ff0 server_cookie=44c00802239ca3b2 in_seq=0 out_seq=0 2023-07-21T18:42:06.818 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.787+0000 7f1686ffd700 1 -- 172.21.15.39:0/4185594007 <== mds.2 v2:172.21.15.193:6838/3148488639 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d6b320eeb0 con 0x55d6b32de1a0 2023-07-21T18:42:06.818 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.787+0000 7f1686ffd700 10 client.15642 handle_command_reply: tid=0 2023-07-21T18:42:06.818 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.787+0000 7f16b9c33b80 1 client.15642 shutdown 2023-07-21T18:42:06.818 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.787+0000 7f16b9c33b80 1 -- 172.21.15.39:0/4185594007 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d6b32cead0 msgr2=0x55d6b32cbe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:06.819 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.787+0000 7f16b9c33b80 1 --2- 172.21.15.39:0/4185594007 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d6b32cead0 0x55d6b32cbe10 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f16a0004fd0 tx=0x7f16a000b3b0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.819 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.787+0000 7f16b9c33b80 1 -- 172.21.15.39:0/4185594007 shutdown_connections 2023-07-21T18:42:06.819 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.787+0000 7f16b9c33b80 1 --2- 172.21.15.39:0/4185594007 >> [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] conn(0x55d6b32de1a0 0x55d6b32e05b0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.819 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.787+0000 7f16b9c33b80 1 --2- 172.21.15.39:0/4185594007 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d6b32c32d0 0x55d6b32ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.819 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.787+0000 7f1686ffd700 0 client.15642 ms_handle_reset on v2:172.21.15.193:6838/3148488639 2023-07-21T18:42:06.819 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.787+0000 7f16b9c33b80 1 --2- 172.21.15.39:0/4185594007 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d6b32cead0 0x55d6b32cbe10 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.819 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.787+0000 7f16b9c33b80 1 --2- 172.21.15.39:0/4185594007 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6b32c3be0 0x55d6b32cb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.819 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.787+0000 7f16b9c33b80 1 -- 172.21.15.39:0/4185594007 >> 172.21.15.39:0/4185594007 conn(0x55d6b32b9da0 msgr2=0x55d6b32bf230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:06.820 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.788+0000 7f16b9c33b80 1 -- 172.21.15.39:0/4185594007 shutdown_connections 2023-07-21T18:42:06.820 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.788+0000 7f16b9c33b80 1 -- 172.21.15.39:0/4185594007 wait complete. 2023-07-21T18:42:06.820 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.788+0000 7f16b9c33b80 20 client.15642 trim_cache size 0 max 16384 2023-07-21T18:42:06.823 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.801+0000 7f16b9c33b80 1 Processor -- start 2023-07-21T18:42:06.823 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.801+0000 7f16b9c33b80 1 -- start start 2023-07-21T18:42:06.823 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.801+0000 7f16b9c33b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d6b32c32d0 0x55d6b32c6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.823 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.801+0000 7f16b9c33b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d6b32c3be0 0x55d6b32c6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.823 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.801+0000 7f16b9c33b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6b32cead0 0x55d6b32caf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.823 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.801+0000 7f16b9c33b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6b32b6fa0 con 0x55d6b32c3be0 2023-07-21T18:42:06.824 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.801+0000 7f16b9c33b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6b32ddf80 con 0x55d6b32c32d0 2023-07-21T18:42:06.824 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.801+0000 7f16b9c33b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6b32b9970 con 0x55d6b32cead0 2023-07-21T18:42:06.824 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.801+0000 7f16a6ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d6b32c3be0 0x55d6b32c6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.824 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.801+0000 7f16a6ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d6b32c3be0 0x55d6b32c6fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57690/0 (socket says 172.21.15.39:57690) 2023-07-21T18:42:06.824 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.801+0000 7f16a6ffd700 1 -- 172.21.15.39:0/4086135963 learned_addr learned my addr 172.21.15.39:0/4086135963 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:06.824 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.801+0000 7f16a77fe700 1 --2- 172.21.15.39:0/4086135963 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d6b32c32d0 0x55d6b32c6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.824 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.801+0000 7f16a7fff700 1 --2- 172.21.15.39:0/4086135963 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6b32cead0 0x55d6b32caf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.825 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.801+0000 7f16a6ffd700 1 -- 172.21.15.39:0/4086135963 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6b32cead0 msgr2=0x55d6b32caf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:06.825 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.801+0000 7f16a6ffd700 1 --2- 172.21.15.39:0/4086135963 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6b32cead0 0x55d6b32caf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.825 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.801+0000 7f16a6ffd700 1 -- 172.21.15.39:0/4086135963 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d6b32c32d0 msgr2=0x55d6b32c6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:06.825 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.801+0000 7f16a6ffd700 1 --2- 172.21.15.39:0/4086135963 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d6b32c32d0 0x55d6b32c6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.825 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.801+0000 7f16a6ffd700 1 -- 172.21.15.39:0/4086135963 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d6b306a250 con 0x55d6b32c3be0 2023-07-21T18:42:06.825 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.801+0000 7f16a6ffd700 1 --2- 172.21.15.39:0/4086135963 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d6b32c3be0 0x55d6b32c6fd0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f169c009e00 tx=0x7f169c012730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b86b5a543e442449 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:06.828 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.802+0000 7f168e7fc700 1 -- 172.21.15.39:0/4086135963 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169c012d90 con 0x55d6b32c3be0 2023-07-21T18:42:06.829 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.802+0000 7f168e7fc700 1 -- 172.21.15.39:0/4086135963 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f169c018040 con 0x55d6b32c3be0 2023-07-21T18:42:06.829 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.802+0000 7f168e7fc700 1 -- 172.21.15.39:0/4086135963 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169c00f450 con 0x55d6b32c3be0 2023-07-21T18:42:06.829 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.802+0000 7f16b9c33b80 1 -- 172.21.15.39:0/4086135963 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d6b32c3be0 msgr2=0x55d6b32c6fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:06.829 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.802+0000 7f16b9c33b80 1 --2- 172.21.15.39:0/4086135963 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d6b32c3be0 0x55d6b32c6fd0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f169c009e00 tx=0x7f169c012730 comp rx=0 tx=0).stop 2023-07-21T18:42:06.829 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.802+0000 7f16b9c33b80 1 -- 172.21.15.39:0/4086135963 shutdown_connections 2023-07-21T18:42:06.829 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.802+0000 7f16b9c33b80 1 --2- 172.21.15.39:0/4086135963 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d6b32c32d0 0x55d6b32c6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.829 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.802+0000 7f16b9c33b80 1 --2- 172.21.15.39:0/4086135963 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6b32cead0 0x55d6b32caf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.830 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.802+0000 7f16b9c33b80 1 --2- 172.21.15.39:0/4086135963 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d6b32c3be0 0x55d6b32c6fd0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.830 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.802+0000 7f16b9c33b80 1 -- 172.21.15.39:0/4086135963 >> 172.21.15.39:0/4086135963 conn(0x55d6b32b9da0 msgr2=0x55d6b32d9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:06.830 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.802+0000 7f16b9c33b80 1 -- 172.21.15.39:0/4086135963 shutdown_connections 2023-07-21T18:42:06.830 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.802+0000 7f16b9c33b80 1 -- 172.21.15.39:0/4086135963 wait complete. 2023-07-21T18:42:06.830 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.803+0000 7f16b9c33b80 1 -- start start 2023-07-21T18:42:06.830 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.803+0000 7f16b9c33b80 1 Processor -- start 2023-07-21T18:42:06.830 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.803+0000 7f16b9c33b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d6b32c32d0 0x55d6b32d3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.830 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.803+0000 7f16b9c33b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6b32c3be0 0x55d6b32d4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.831 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.803+0000 7f16b9c33b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d6b32cead0 0x55d6b32d45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.831 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.803+0000 7f16b9c33b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6b3336480 con 0x55d6b32c32d0 2023-07-21T18:42:06.831 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.803+0000 7f16b9c33b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6b33365c0 con 0x55d6b32cead0 2023-07-21T18:42:06.831 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.803+0000 7f16b9c33b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6b3336700 con 0x55d6b32c3be0 2023-07-21T18:42:06.831 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.803+0000 7f16b9c33b80 10 client.0 _ll_register_callbacks cb 0x7f16a0009c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:42:06.831 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.803+0000 7f16a77fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d6b32c32d0 0x55d6b32d3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.831 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.803+0000 7f16a77fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d6b32c32d0 0x55d6b32d3b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57694/0 (socket says 172.21.15.39:57694) 2023-07-21T18:42:06.831 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.803+0000 7f16a77fe700 1 -- 172.21.15.39:0/3839837145 learned_addr learned my addr 172.21.15.39:0/3839837145 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:06.832 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.803+0000 7f16a6ffd700 1 --2- 172.21.15.39:0/3839837145 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6b32c3be0 0x55d6b32d4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.832 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.803+0000 7f16a7fff700 1 --2- 172.21.15.39:0/3839837145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d6b32cead0 0x55d6b32d45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.832 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.803+0000 7f16a77fe700 1 -- 172.21.15.39:0/3839837145 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6b32c3be0 msgr2=0x55d6b32d4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:06.832 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.803+0000 7f16a77fe700 1 --2- 172.21.15.39:0/3839837145 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6b32c3be0 0x55d6b32d4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.832 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.803+0000 7f16a77fe700 1 -- 172.21.15.39:0/3839837145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d6b32cead0 msgr2=0x55d6b32d45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:06.832 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.803+0000 7f16a77fe700 1 --2- 172.21.15.39:0/3839837145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d6b32cead0 0x55d6b32d45a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.832 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.804+0000 7f16a77fe700 1 -- 172.21.15.39:0/3839837145 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f169c009a80 con 0x55d6b32c32d0 2023-07-21T18:42:06.832 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.804+0000 7f16a77fe700 1 --2- 172.21.15.39:0/3839837145 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d6b32c32d0 0x55d6b32d3b20 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f169801ee90 tx=0x7f169800e700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49525467d2feae39 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:06.833 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.804+0000 7f1686ffd700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:42:06.833 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.804+0000 7f1686ffd700 1 -- 172.21.15.39:0/3839837145 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169802e030 con 0x55d6b32c32d0 2023-07-21T18:42:06.833 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.804+0000 7f1686ffd700 1 -- 172.21.15.39:0/3839837145 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f1698030220 con 0x55d6b32c32d0 2023-07-21T18:42:06.833 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.804+0000 7f16b9c33b80 1 -- 172.21.15.39:0/3839837145 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1698011d40 con 0x55d6b32c32d0 2023-07-21T18:42:06.833 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.804+0000 7f1686ffd700 1 -- 172.21.15.39:0/3839837145 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16980303e0 con 0x55d6b32c32d0 2023-07-21T18:42:06.833 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.804+0000 7f16b9c33b80 10 client.15648 fetch_fsmap learned FSMap version 26 2023-07-21T18:42:06.833 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.804+0000 7f16b9c33b80 1 -- 172.21.15.39:0/3839837145 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55d6b30422b0 con 0x55d6b32c32d0 2023-07-21T18:42:06.833 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.804+0000 7f1686ffd700 1 -- 172.21.15.39:0/3839837145 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1698011d40 con 0x55d6b32c32d0 2023-07-21T18:42:06.834 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.804+0000 7f1686ffd700 1 -- 172.21.15.39:0/3839837145 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f169801e6a0 con 0x55d6b32c32d0 2023-07-21T18:42:06.834 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.804+0000 7f16b9c33b80 10 client.15648 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T18:42:06.834 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.804+0000 7f16b9c33b80 10 client.15648 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T18:42:06.834 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.804+0000 7f16b9c33b80 20 client.15648 populate_metadata read hostname 'smithi039' 2023-07-21T18:42:06.834 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.804+0000 7f16b9c33b80 1 --2- 172.21.15.39:0/3839837145 >> [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] conn(0x55d6b3337bd0 0x55d6b3339fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.834 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.804+0000 7f16b9c33b80 4 client.15648 mds_command: new command op to 24487 tid=0[{"prefix": "damage ls"}] 2023-07-21T18:42:06.834 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.804+0000 7f16b9c33b80 1 -- 172.21.15.39:0/3839837145 --> [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d6b31d1050 con 0x55d6b3337bd0 2023-07-21T18:42:06.835 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.805+0000 7f16a6ffd700 1 --2- 172.21.15.39:0/3839837145 >> [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] conn(0x55d6b3337bd0 0x55d6b3339fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.835 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.805+0000 7f16a6ffd700 1 --2- 172.21.15.39:0/3839837145 >> [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] conn(0x55d6b3337bd0 0x55d6b3339fe0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=73424e233f287f3c server_cookie=c9c8f7f32afc64b6 in_seq=0 out_seq=0 2023-07-21T18:42:06.835 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.805+0000 7f1686ffd700 10 client.15648 ms_handle_connect on v2:172.21.15.193:6838/3148488639 2023-07-21T18:42:06.835 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.805+0000 7f1686ffd700 1 -- 172.21.15.39:0/3839837145 <== mds.2 v2:172.21.15.193:6838/3148488639 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d6b31d1050 con 0x55d6b3337bd0 2023-07-21T18:42:06.835 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.805+0000 7f1686ffd700 10 client.15648 handle_command_reply: tid=0 2023-07-21T18:42:06.835 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.805+0000 7f16b9c33b80 1 client.15648 shutdown 2023-07-21T18:42:06.835 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.805+0000 7f16b9c33b80 1 -- 172.21.15.39:0/3839837145 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d6b32c32d0 msgr2=0x55d6b32d3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:06.835 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.805+0000 7f16b9c33b80 1 --2- 172.21.15.39:0/3839837145 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d6b32c32d0 0x55d6b32d3b20 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f169801ee90 tx=0x7f169800e700 comp rx=0 tx=0).stop 2023-07-21T18:42:06.836 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.805+0000 7f16b9c33b80 1 -- 172.21.15.39:0/3839837145 shutdown_connections 2023-07-21T18:42:06.836 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.805+0000 7f16b9c33b80 1 --2- 172.21.15.39:0/3839837145 >> [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] conn(0x55d6b3337bd0 0x55d6b3339fe0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.836 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.805+0000 7f16b9c33b80 1 --2- 172.21.15.39:0/3839837145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d6b32cead0 0x55d6b32d45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.836 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.806+0000 7f16b9c33b80 1 --2- 172.21.15.39:0/3839837145 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d6b32c32d0 0x55d6b32d3b20 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.836 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.806+0000 7f1686ffd700 0 client.15648 ms_handle_reset on v2:172.21.15.193:6838/3148488639 2023-07-21T18:42:06.836 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.806+0000 7f16b9c33b80 1 --2- 172.21.15.39:0/3839837145 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6b32c3be0 0x55d6b32d4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.836 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.806+0000 7f16b9c33b80 1 -- 172.21.15.39:0/3839837145 >> 172.21.15.39:0/3839837145 conn(0x55d6b32b9da0 msgr2=0x55d6b32c6a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:06.836 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.806+0000 7f16b9c33b80 1 -- 172.21.15.39:0/3839837145 shutdown_connections 2023-07-21T18:42:06.837 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.806+0000 7f16b9c33b80 1 -- 172.21.15.39:0/3839837145 wait complete. 2023-07-21T18:42:06.837 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.806+0000 7f16b9c33b80 20 client.15648 trim_cache size 0 max 16384 2023-07-21T18:42:06.837 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2023-07-21T18:42:06.838 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.806+0000 7f16adaad700 1 -- 172.21.15.39:0/974951443 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f16900a13e0 msgr2=0x7f16900a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:06.838 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.806+0000 7f16adaad700 1 --2- 172.21.15.39:0/974951443 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f16900a13e0 0x7f16900a3890 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1698005fd0 tx=0x7f1698005f00 comp rx=0 tx=0).stop 2023-07-21T18:42:06.838 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.806+0000 7f16adaad700 1 -- 172.21.15.39:0/974951443 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16a81049e0 msgr2=0x7f16a8195ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:06.838 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.806+0000 7f16adaad700 1 --2- 172.21.15.39:0/974951443 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16a81049e0 0x7f16a8195ab0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f16a000b780 tx=0x7f16a000be10 comp rx=0 tx=0).stop 2023-07-21T18:42:06.838 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.806+0000 7f16adaad700 1 -- 172.21.15.39:0/974951443 shutdown_connections 2023-07-21T18:42:06.838 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.806+0000 7f16adaad700 1 --2- 172.21.15.39:0/974951443 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f16900a13e0 0x7f16900a3890 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.839 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.806+0000 7f16adaad700 1 --2- 172.21.15.39:0/974951443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16a80fe9d0 0x7f16a819ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.839 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.806+0000 7f16adaad700 1 --2- 172.21.15.39:0/974951443 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16a81049e0 0x7f16a8195ab0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.839 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.806+0000 7f16adaad700 1 --2- 172.21.15.39:0/974951443 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16a80ff380 0x7f16a819bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.839 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.806+0000 7f16adaad700 1 -- 172.21.15.39:0/974951443 >> 172.21.15.39:0/974951443 conn(0x7f16a80fa4b0 msgr2=0x7f16a80fbec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:06.839 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.806+0000 7f16adaad700 1 -- 172.21.15.39:0/974951443 shutdown_connections 2023-07-21T18:42:06.839 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.806+0000 7f16adaad700 1 -- 172.21.15.39:0/974951443 wait complete. 2023-07-21T18:42:06.840 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-21T18:42:06.968 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.946+0000 7f19105f6700 1 -- 172.21.15.39:0/1676740467 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1908105b00 msgr2=0x7f1908105ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:06.969 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.946+0000 7f19105f6700 1 --2- 172.21.15.39:0/1676740467 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1908105b00 0x7f1908105ed0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f1904009990 tx=0x7f1904009ca0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.969 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.947+0000 7f19105f6700 1 -- 172.21.15.39:0/1676740467 shutdown_connections 2023-07-21T18:42:06.969 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.947+0000 7f19105f6700 1 --2- 172.21.15.39:0/1676740467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19081004a0 0x7f190810c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.969 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.947+0000 7f19105f6700 1 --2- 172.21.15.39:0/1676740467 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1908105b00 0x7f1908105ed0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.969 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.947+0000 7f19105f6700 1 --2- 172.21.15.39:0/1676740467 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f19080ffaf0 0x7f19080fff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.969 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.947+0000 7f19105f6700 1 -- 172.21.15.39:0/1676740467 >> 172.21.15.39:0/1676740467 conn(0x7f19080fb650 msgr2=0x7f19080fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:06.969 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.947+0000 7f19105f6700 1 -- 172.21.15.39:0/1676740467 shutdown_connections 2023-07-21T18:42:06.969 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.947+0000 7f19105f6700 1 -- 172.21.15.39:0/1676740467 wait complete. 2023-07-21T18:42:06.971 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.947+0000 7f19105f6700 1 Processor -- start 2023-07-21T18:42:06.971 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.947+0000 7f19105f6700 1 -- start start 2023-07-21T18:42:06.971 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.947+0000 7f19105f6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19080ffaf0 0x7f190819b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.971 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.947+0000 7f19105f6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19081004a0 0x7f190819bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.972 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.947+0000 7f19105f6700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1908105b00 0x7f1908195a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.972 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.947+0000 7f19105f6700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1908196080 con 0x7f19081004a0 2023-07-21T18:42:06.972 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.947+0000 7f19105f6700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19081961f0 con 0x7f19080ffaf0 2023-07-21T18:42:06.972 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.947+0000 7f19105f6700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19081963b0 con 0x7f1908105b00 2023-07-21T18:42:06.972 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.948+0000 7f190db91700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19081004a0 0x7f190819bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.972 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.948+0000 7f190db91700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19081004a0 0x7f190819bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57710/0 (socket says 172.21.15.39:57710) 2023-07-21T18:42:06.972 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.948+0000 7f190db91700 1 -- 172.21.15.39:0/2778360306 learned_addr learned my addr 172.21.15.39:0/2778360306 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:06.973 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.948+0000 7f190e392700 1 --2- 172.21.15.39:0/2778360306 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19080ffaf0 0x7f190819b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.973 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.948+0000 7f190eb93700 1 --2- 172.21.15.39:0/2778360306 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1908105b00 0x7f1908195a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.973 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.948+0000 7f190db91700 1 -- 172.21.15.39:0/2778360306 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1908105b00 msgr2=0x7f1908195a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:06.973 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.948+0000 7f190db91700 1 --2- 172.21.15.39:0/2778360306 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1908105b00 0x7f1908195a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.973 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.948+0000 7f190db91700 1 -- 172.21.15.39:0/2778360306 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19080ffaf0 msgr2=0x7f190819b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:06.973 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.948+0000 7f190db91700 1 --2- 172.21.15.39:0/2778360306 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19080ffaf0 0x7f190819b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.973 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.948+0000 7f190db91700 1 -- 172.21.15.39:0/2778360306 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19040095f0 con 0x7f19081004a0 2023-07-21T18:42:06.973 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.948+0000 7f190db91700 1 --2- 172.21.15.39:0/2778360306 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19081004a0 0x7f190819bee0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f18f800d950 tx=0x7f18f800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:06.974 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.948+0000 7f18ff7fe700 1 -- 172.21.15.39:0/2778360306 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f8009940 con 0x7f19081004a0 2023-07-21T18:42:06.974 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.948+0000 7f19105f6700 1 -- 172.21.15.39:0/2778360306 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1908196690 con 0x7f19081004a0 2023-07-21T18:42:06.974 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.948+0000 7f18ff7fe700 1 -- 172.21.15.39:0/2778360306 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f18f8010460 con 0x7f19081004a0 2023-07-21T18:42:06.974 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.948+0000 7f18ff7fe700 1 -- 172.21.15.39:0/2778360306 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f800f620 con 0x7f19081004a0 2023-07-21T18:42:06.974 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.948+0000 7f19105f6700 1 -- 172.21.15.39:0/2778360306 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19081a2990 con 0x7f19081004a0 2023-07-21T18:42:06.978 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.950+0000 7f18ff7fe700 1 -- 172.21.15.39:0/2778360306 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f18f80108d0 con 0x7f19081004a0 2023-07-21T18:42:06.978 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.951+0000 7f18ff7fe700 1 --2- 172.21.15.39:0/2778360306 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f18f40a13e0 0x7f18f40a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.978 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.951+0000 7f190e392700 1 --2- 172.21.15.39:0/2778360306 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f18f40a13e0 0x7f18f40a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.978 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.951+0000 7f18ff7fe700 1 -- 172.21.15.39:0/2778360306 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f18f80cce20 con 0x7f19081004a0 2023-07-21T18:42:06.978 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.952+0000 7f190e392700 1 --2- 172.21.15.39:0/2778360306 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f18f40a13e0 0x7f18f40a3890 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f190400b5c0 tx=0x7f1904005fd0 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:06.978 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.952+0000 7f191c77cb80 1 Processor -- start 2023-07-21T18:42:06.979 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.952+0000 7f191c77cb80 1 -- start start 2023-07-21T18:42:06.979 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.953+0000 7f191c77cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b869082d0 0x560b869086a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.979 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.953+0000 7f191c77cb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560b86908be0 0x560b86913590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.979 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.953+0000 7f191c77cb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560b86913ad0 0x560b86915eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.979 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.953+0000 7f191c77cb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b8680ecb0 con 0x560b86908be0 2023-07-21T18:42:06.979 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.953+0000 7f191c77cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b86861a40 con 0x560b869082d0 2023-07-21T18:42:06.979 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.953+0000 7f191c77cb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b8685aed0 con 0x560b86913ad0 2023-07-21T18:42:06.980 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.953+0000 7f190db91700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560b86908be0 0x560b86913590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.980 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.953+0000 7f190db91700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560b86908be0 0x560b86913590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57718/0 (socket says 172.21.15.39:57718) 2023-07-21T18:42:06.980 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.953+0000 7f190db91700 1 -- 172.21.15.39:0/1161810143 learned_addr learned my addr 172.21.15.39:0/1161810143 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:06.980 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.953+0000 7f190e392700 1 --2- 172.21.15.39:0/1161810143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b869082d0 0x560b869086a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.980 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.953+0000 7f190eb93700 1 --2- 172.21.15.39:0/1161810143 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560b86913ad0 0x560b86915eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.980 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.953+0000 7f190db91700 1 -- 172.21.15.39:0/1161810143 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560b86913ad0 msgr2=0x560b86915eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:06.980 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.953+0000 7f190db91700 1 --2- 172.21.15.39:0/1161810143 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560b86913ad0 0x560b86915eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.980 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.953+0000 7f190db91700 1 -- 172.21.15.39:0/1161810143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b869082d0 msgr2=0x560b869086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:06.981 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.953+0000 7f190db91700 1 --2- 172.21.15.39:0/1161810143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b869082d0 0x560b869086a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.981 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.953+0000 7f190db91700 1 -- 172.21.15.39:0/1161810143 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b86688de0 con 0x560b86908be0 2023-07-21T18:42:06.981 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.953+0000 7f190eb93700 1 --2- 172.21.15.39:0/1161810143 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560b86913ad0 0x560b86915eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:42:06.981 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.953+0000 7f190e392700 1 --2- 172.21.15.39:0/1161810143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b869082d0 0x560b869086a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:42:06.981 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.953+0000 7f190db91700 1 --2- 172.21.15.39:0/1161810143 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560b86908be0 0x560b86913590 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f18f80961c0 tx=0x7f18f80996f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bff2d4fb3f311548 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:06.981 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.953+0000 7f18fcff9700 1 -- 172.21.15.39:0/1161810143 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f80a2110 con 0x560b86908be0 2023-07-21T18:42:06.981 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.953+0000 7f18fcff9700 1 -- 172.21.15.39:0/1161810143 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f18f8099f20 con 0x560b86908be0 2023-07-21T18:42:06.981 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.953+0000 7f18fcff9700 1 -- 172.21.15.39:0/1161810143 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f80a1f30 con 0x560b86908be0 2023-07-21T18:42:06.982 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.954+0000 7f191c77cb80 1 -- 172.21.15.39:0/1161810143 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560b86908be0 msgr2=0x560b86913590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:06.982 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.954+0000 7f191c77cb80 1 --2- 172.21.15.39:0/1161810143 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560b86908be0 0x560b86913590 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f18f80961c0 tx=0x7f18f80996f0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.982 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.954+0000 7f191c77cb80 1 -- 172.21.15.39:0/1161810143 shutdown_connections 2023-07-21T18:42:06.982 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.954+0000 7f191c77cb80 1 --2- 172.21.15.39:0/1161810143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b869082d0 0x560b869086a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.982 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.954+0000 7f191c77cb80 1 --2- 172.21.15.39:0/1161810143 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560b86913ad0 0x560b86915eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.982 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.954+0000 7f191c77cb80 1 --2- 172.21.15.39:0/1161810143 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560b86908be0 0x560b86913590 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.982 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.954+0000 7f191c77cb80 1 -- 172.21.15.39:0/1161810143 >> 172.21.15.39:0/1161810143 conn(0x560b868feda0 msgr2=0x560b869046a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:06.983 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.954+0000 7f191c77cb80 1 -- 172.21.15.39:0/1161810143 shutdown_connections 2023-07-21T18:42:06.983 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.954+0000 7f191c77cb80 1 -- 172.21.15.39:0/1161810143 wait complete. 2023-07-21T18:42:06.983 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.955+0000 7f191c77cb80 1 -- start start 2023-07-21T18:42:06.983 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.955+0000 7f191c77cb80 1 Processor -- start 2023-07-21T18:42:06.983 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.955+0000 7f191c77cb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560b869082d0 0x560b86913530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.983 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.955+0000 7f191c77cb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560b86908be0 0x560b869107b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.983 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.955+0000 7f191c77cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b86913ad0 0x560b86910ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.983 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.955+0000 7f191c77cb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b867faac0 con 0x560b86908be0 2023-07-21T18:42:06.984 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.955+0000 7f191c77cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b867fa7c0 con 0x560b86913ad0 2023-07-21T18:42:06.984 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.955+0000 7f191c77cb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b866de920 con 0x560b869082d0 2023-07-21T18:42:06.984 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.955+0000 7f191c77cb80 10 client.0 _ll_register_callbacks cb 0x560b867f8760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:42:06.984 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.955+0000 7f190db91700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560b86908be0 0x560b869107b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.984 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.955+0000 7f190db91700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560b86908be0 0x560b869107b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57724/0 (socket says 172.21.15.39:57724) 2023-07-21T18:42:06.984 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.955+0000 7f190db91700 1 -- 172.21.15.39:0/3597823899 learned_addr learned my addr 172.21.15.39:0/3597823899 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:06.984 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.955+0000 7f190e392700 1 --2- 172.21.15.39:0/3597823899 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560b869082d0 0x560b86913530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.984 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.955+0000 7f190db91700 1 -- 172.21.15.39:0/3597823899 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560b869082d0 msgr2=0x560b86913530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:06.985 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.955+0000 7f190db91700 1 --2- 172.21.15.39:0/3597823899 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560b869082d0 0x560b86913530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.985 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.955+0000 7f190eb93700 1 --2- 172.21.15.39:0/3597823899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b86913ad0 0x560b86910ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.985 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.955+0000 7f190db91700 1 -- 172.21.15.39:0/3597823899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b86913ad0 msgr2=0x560b86910ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:06.985 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.955+0000 7f190db91700 1 --2- 172.21.15.39:0/3597823899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b86913ad0 0x560b86910ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.985 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.955+0000 7f190db91700 1 -- 172.21.15.39:0/3597823899 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19081a2990 con 0x560b86908be0 2023-07-21T18:42:06.985 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.956+0000 7f190db91700 1 --2- 172.21.15.39:0/3597823899 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560b86908be0 0x560b869107b0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f18f800b2f0 tx=0x7f18f80ca410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=591c60a8c7abcddb server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:06.985 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.956+0000 7f18f1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:42:06.985 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.956+0000 7f18f1ffb700 1 -- 172.21.15.39:0/3597823899 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f80a1e90 con 0x560b86908be0 2023-07-21T18:42:06.986 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.956+0000 7f18f1ffb700 1 -- 172.21.15.39:0/3597823899 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f18f80991b0 con 0x560b86908be0 2023-07-21T18:42:06.986 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.956+0000 7f191c77cb80 1 -- 172.21.15.39:0/3597823899 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560b867b44f0 con 0x560b86908be0 2023-07-21T18:42:06.986 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.956+0000 7f18f1ffb700 1 -- 172.21.15.39:0/3597823899 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f18f8099370 con 0x560b86908be0 2023-07-21T18:42:06.986 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.956+0000 7f191c77cb80 10 client.15660 fetch_fsmap learned FSMap version 26 2023-07-21T18:42:06.986 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.956+0000 7f191c77cb80 1 -- 172.21.15.39:0/3597823899 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x560b86671c60 con 0x560b86908be0 2023-07-21T18:42:06.986 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.956+0000 7f18f1ffb700 1 -- 172.21.15.39:0/3597823899 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f18f8063020 con 0x560b86908be0 2023-07-21T18:42:06.986 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.956+0000 7f18f1ffb700 1 -- 172.21.15.39:0/3597823899 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f18f8095ab0 con 0x560b86908be0 2023-07-21T18:42:06.986 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.956+0000 7f191c77cb80 10 client.15660 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T18:42:06.987 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.956+0000 7f191c77cb80 10 client.15660 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T18:42:06.987 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.956+0000 7f191c77cb80 20 client.15660 populate_metadata read hostname 'smithi039' 2023-07-21T18:42:06.987 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.956+0000 7f191c77cb80 1 --2- 172.21.15.39:0/3597823899 >> [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] conn(0x560b86923020 0x560b86925430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.987 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.956+0000 7f191c77cb80 4 client.15660 mds_command: new command op to 24500 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:42:06.987 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.956+0000 7f191c77cb80 1 -- 172.21.15.39:0/3597823899 --> [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560b86853eb0 con 0x560b86923020 2023-07-21T18:42:06.988 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.957+0000 7f190e392700 1 --2- 172.21.15.39:0/3597823899 >> [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] conn(0x560b86923020 0x560b86925430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.988 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.957+0000 7f190e392700 1 --2- 172.21.15.39:0/3597823899 >> [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] conn(0x560b86923020 0x560b86925430 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d7ebcdbe147520e4 server_cookie=5e883c708a50b33b in_seq=0 out_seq=0 2023-07-21T18:42:06.989 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.957+0000 7f18f1ffb700 10 client.15660 ms_handle_connect on v2:172.21.15.155:6836/947670056 2023-07-21T18:42:06.989 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.958+0000 7f18f1ffb700 1 -- 172.21.15.39:0/3597823899 <== mds.1 v2:172.21.15.155:6836/947670056 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560b86853eb0 con 0x560b86923020 2023-07-21T18:42:06.989 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.958+0000 7f18f1ffb700 10 client.15660 handle_command_reply: tid=0 2023-07-21T18:42:06.989 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.958+0000 7f191c77cb80 1 client.15660 shutdown 2023-07-21T18:42:06.989 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.958+0000 7f191c77cb80 1 -- 172.21.15.39:0/3597823899 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560b86908be0 msgr2=0x560b869107b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:06.989 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.958+0000 7f191c77cb80 1 --2- 172.21.15.39:0/3597823899 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560b86908be0 0x560b869107b0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f18f800b2f0 tx=0x7f18f80ca410 comp rx=0 tx=0).stop 2023-07-21T18:42:06.989 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.958+0000 7f191c77cb80 1 -- 172.21.15.39:0/3597823899 shutdown_connections 2023-07-21T18:42:06.989 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.958+0000 7f191c77cb80 1 --2- 172.21.15.39:0/3597823899 >> [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] conn(0x560b86923020 0x560b86925430 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.990 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.958+0000 7f191c77cb80 1 --2- 172.21.15.39:0/3597823899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b86913ad0 0x560b86910ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.990 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.958+0000 7f191c77cb80 1 --2- 172.21.15.39:0/3597823899 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560b869082d0 0x560b86913530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.990 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.958+0000 7f191c77cb80 1 --2- 172.21.15.39:0/3597823899 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560b86908be0 0x560b869107b0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.990 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.958+0000 7f18f1ffb700 0 client.15660 ms_handle_reset on v2:172.21.15.155:6836/947670056 2023-07-21T18:42:06.990 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.958+0000 7f191c77cb80 1 -- 172.21.15.39:0/3597823899 >> 172.21.15.39:0/3597823899 conn(0x560b868feda0 msgr2=0x560b86904230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:06.990 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.958+0000 7f191c77cb80 1 -- 172.21.15.39:0/3597823899 shutdown_connections 2023-07-21T18:42:06.990 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.958+0000 7f191c77cb80 1 -- 172.21.15.39:0/3597823899 wait complete. 2023-07-21T18:42:06.990 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.958+0000 7f191c77cb80 20 client.15660 trim_cache size 0 max 16384 2023-07-21T18:42:06.993 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.971+0000 7f191c77cb80 1 Processor -- start 2023-07-21T18:42:06.993 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.971+0000 7f191c77cb80 1 -- start start 2023-07-21T18:42:06.993 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.971+0000 7f191c77cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b869082d0 0x560b86910ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.993 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.971+0000 7f191c77cb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560b86908be0 0x560b8691a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.994 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.971+0000 7f191c77cb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560b86913ad0 0x560b8690bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:06.994 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.971+0000 7f191c77cb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b86900ec0 con 0x560b86908be0 2023-07-21T18:42:06.994 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.971+0000 7f191c77cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b86909540 con 0x560b869082d0 2023-07-21T18:42:06.994 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.971+0000 7f191c77cb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b8691f720 con 0x560b86913ad0 2023-07-21T18:42:06.994 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.971+0000 7f190db91700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560b86908be0 0x560b8691a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.994 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.971+0000 7f190db91700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560b86908be0 0x560b8691a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57730/0 (socket says 172.21.15.39:57730) 2023-07-21T18:42:06.994 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.971+0000 7f190db91700 1 -- 172.21.15.39:0/222417974 learned_addr learned my addr 172.21.15.39:0/222417974 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:06.995 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.971+0000 7f190eb93700 1 --2- 172.21.15.39:0/222417974 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560b86913ad0 0x560b8690bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.995 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.971+0000 7f190e392700 1 --2- 172.21.15.39:0/222417974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b869082d0 0x560b86910ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:06.995 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.971+0000 7f190db91700 1 -- 172.21.15.39:0/222417974 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560b86913ad0 msgr2=0x560b8690bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:06.995 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.971+0000 7f190db91700 1 --2- 172.21.15.39:0/222417974 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560b86913ad0 0x560b8690bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.995 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.971+0000 7f190db91700 1 -- 172.21.15.39:0/222417974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b869082d0 msgr2=0x560b86910ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:06.995 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.971+0000 7f190db91700 1 --2- 172.21.15.39:0/222417974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b869082d0 0x560b86910ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.995 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.971+0000 7f190db91700 1 -- 172.21.15.39:0/222417974 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b866af250 con 0x560b86908be0 2023-07-21T18:42:06.995 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.971+0000 7f190e392700 1 --2- 172.21.15.39:0/222417974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b869082d0 0x560b86910ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:42:06.996 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.971+0000 7f190db91700 1 --2- 172.21.15.39:0/222417974 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560b86908be0 0x560b8691a4b0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f18f80961c0 tx=0x7f18f8099a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94021e5cde28c13f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:06.996 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.972+0000 7f18fd7fa700 1 -- 172.21.15.39:0/222417974 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f80a1e90 con 0x560b86908be0 2023-07-21T18:42:06.996 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.972+0000 7f18fd7fa700 1 -- 172.21.15.39:0/222417974 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f18f8063070 con 0x560b86908be0 2023-07-21T18:42:06.996 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.972+0000 7f18fd7fa700 1 -- 172.21.15.39:0/222417974 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f80b21a0 con 0x560b86908be0 2023-07-21T18:42:06.999 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.972+0000 7f191c77cb80 1 -- 172.21.15.39:0/222417974 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560b86908be0 msgr2=0x560b8691a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:06.999 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.972+0000 7f191c77cb80 1 --2- 172.21.15.39:0/222417974 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560b86908be0 0x560b8691a4b0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f18f80961c0 tx=0x7f18f8099a90 comp rx=0 tx=0).stop 2023-07-21T18:42:06.999 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.972+0000 7f191c77cb80 1 -- 172.21.15.39:0/222417974 shutdown_connections 2023-07-21T18:42:06.999 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.972+0000 7f191c77cb80 1 --2- 172.21.15.39:0/222417974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b869082d0 0x560b86910ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:06.999 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.972+0000 7f191c77cb80 1 --2- 172.21.15.39:0/222417974 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560b86913ad0 0x560b8690bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:07.000 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.972+0000 7f191c77cb80 1 --2- 172.21.15.39:0/222417974 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560b86908be0 0x560b8691a4b0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:07.000 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.972+0000 7f191c77cb80 1 -- 172.21.15.39:0/222417974 >> 172.21.15.39:0/222417974 conn(0x560b868feda0 msgr2=0x560b86902e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:07.000 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.972+0000 7f191c77cb80 1 -- 172.21.15.39:0/222417974 shutdown_connections 2023-07-21T18:42:07.000 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.972+0000 7f191c77cb80 1 -- 172.21.15.39:0/222417974 wait complete. 2023-07-21T18:42:07.000 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.972+0000 7f191c77cb80 1 -- start start 2023-07-21T18:42:07.000 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.973+0000 7f191c77cb80 1 Processor -- start 2023-07-21T18:42:07.000 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.973+0000 7f191c77cb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560b869082d0 0x560b8696b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:07.000 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.973+0000 7f191c77cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b86908be0 0x560b8696b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:07.001 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.973+0000 7f191c77cb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560b86913ad0 0x560b86975eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:07.001 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.973+0000 7f191c77cb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b869763f0 con 0x560b869082d0 2023-07-21T18:42:07.001 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.973+0000 7f191c77cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b86976530 con 0x560b86908be0 2023-07-21T18:42:07.001 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.973+0000 7f191c77cb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b86976670 con 0x560b86913ad0 2023-07-21T18:42:07.001 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.973+0000 7f191c77cb80 10 client.0 _ll_register_callbacks cb 0x560b867f8760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:42:07.001 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.973+0000 7f190e392700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560b869082d0 0x560b8696b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:07.001 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.973+0000 7f190e392700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560b869082d0 0x560b8696b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57734/0 (socket says 172.21.15.39:57734) 2023-07-21T18:42:07.001 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.973+0000 7f190e392700 1 -- 172.21.15.39:0/227179341 learned_addr learned my addr 172.21.15.39:0/227179341 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:07.002 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.973+0000 7f190eb93700 1 --2- 172.21.15.39:0/227179341 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560b86913ad0 0x560b86975eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:07.002 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.973+0000 7f190db91700 1 --2- 172.21.15.39:0/227179341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b86908be0 0x560b8696b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:07.002 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.973+0000 7f190e392700 1 -- 172.21.15.39:0/227179341 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560b86913ad0 msgr2=0x560b86975eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:07.002 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.973+0000 7f190e392700 1 --2- 172.21.15.39:0/227179341 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560b86913ad0 0x560b86975eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:07.002 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.973+0000 7f190e392700 1 -- 172.21.15.39:0/227179341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b86908be0 msgr2=0x560b8696b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:07.002 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.973+0000 7f190e392700 1 --2- 172.21.15.39:0/227179341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b86908be0 0x560b8696b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:07.002 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.973+0000 7f190e392700 1 -- 172.21.15.39:0/227179341 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b86671c60 con 0x560b869082d0 2023-07-21T18:42:07.003 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.973+0000 7f190eb93700 1 --2- 172.21.15.39:0/227179341 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560b86913ad0 0x560b86975eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:42:07.003 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.973+0000 7f190e392700 1 --2- 172.21.15.39:0/227179341 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560b869082d0 0x560b8696b020 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f19040098e0 tx=0x7f1904020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa35cd6e3baf6794 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:07.003 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.973+0000 7f18f1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:42:07.003 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.974+0000 7f18f1ffb700 1 -- 172.21.15.39:0/227179341 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1904033a50 con 0x560b869082d0 2023-07-21T18:42:07.003 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.974+0000 7f18f1ffb700 1 -- 172.21.15.39:0/227179341 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f1904033bb0 con 0x560b869082d0 2023-07-21T18:42:07.003 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.974+0000 7f191c77cb80 1 -- 172.21.15.39:0/227179341 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f18f8099f20 con 0x560b869082d0 2023-07-21T18:42:07.003 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.974+0000 7f18f1ffb700 1 -- 172.21.15.39:0/227179341 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1904033d70 con 0x560b869082d0 2023-07-21T18:42:07.003 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.974+0000 7f191c77cb80 10 client.15666 fetch_fsmap learned FSMap version 26 2023-07-21T18:42:07.004 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.974+0000 7f191c77cb80 1 -- 172.21.15.39:0/227179341 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x560b866872b0 con 0x560b869082d0 2023-07-21T18:42:07.004 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.974+0000 7f18f1ffb700 1 -- 172.21.15.39:0/227179341 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1904040020 con 0x560b869082d0 2023-07-21T18:42:07.004 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.974+0000 7f18f1ffb700 1 -- 172.21.15.39:0/227179341 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1904023770 con 0x560b869082d0 2023-07-21T18:42:07.004 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.974+0000 7f191c77cb80 10 client.15666 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T18:42:07.004 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.974+0000 7f191c77cb80 10 client.15666 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T18:42:07.004 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.974+0000 7f191c77cb80 20 client.15666 populate_metadata read hostname 'smithi039' 2023-07-21T18:42:07.004 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.974+0000 7f191c77cb80 1 --2- 172.21.15.39:0/227179341 >> [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] conn(0x560b86977c20 0x560b8697a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:07.004 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.974+0000 7f191c77cb80 4 client.15666 mds_command: new command op to 24500 tid=0[{"prefix": "damage ls"}] 2023-07-21T18:42:07.005 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.974+0000 7f191c77cb80 1 -- 172.21.15.39:0/227179341 --> [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560b86816050 con 0x560b86977c20 2023-07-21T18:42:07.005 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.974+0000 7f190db91700 1 --2- 172.21.15.39:0/227179341 >> [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] conn(0x560b86977c20 0x560b8697a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:07.005 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.975+0000 7f18f1ffb700 10 client.15666 ms_handle_connect on v2:172.21.15.155:6836/947670056 2023-07-21T18:42:07.005 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.975+0000 7f190db91700 1 --2- 172.21.15.39:0/227179341 >> [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] conn(0x560b86977c20 0x560b8697a030 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=79060f7795ac636d server_cookie=87bb553b18c62ad3 in_seq=0 out_seq=0 2023-07-21T18:42:07.005 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.975+0000 7f18f1ffb700 1 -- 172.21.15.39:0/227179341 <== mds.1 v2:172.21.15.155:6836/947670056 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560b86816050 con 0x560b86977c20 2023-07-21T18:42:07.005 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.975+0000 7f18f1ffb700 10 client.15666 handle_command_reply: tid=0 2023-07-21T18:42:07.005 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.975+0000 7f191c77cb80 1 client.15666 shutdown 2023-07-21T18:42:07.005 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.975+0000 7f191c77cb80 1 -- 172.21.15.39:0/227179341 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560b869082d0 msgr2=0x560b8696b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:07.006 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.975+0000 7f191c77cb80 1 --2- 172.21.15.39:0/227179341 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560b869082d0 0x560b8696b020 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f19040098e0 tx=0x7f1904020000 comp rx=0 tx=0).stop 2023-07-21T18:42:07.006 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.975+0000 7f191c77cb80 1 -- 172.21.15.39:0/227179341 shutdown_connections 2023-07-21T18:42:07.006 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.975+0000 7f191c77cb80 1 --2- 172.21.15.39:0/227179341 >> [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] conn(0x560b86977c20 0x560b8697a030 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:07.006 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.975+0000 7f191c77cb80 1 --2- 172.21.15.39:0/227179341 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560b86913ad0 0x560b86975eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:07.006 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.975+0000 7f191c77cb80 1 --2- 172.21.15.39:0/227179341 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560b869082d0 0x560b8696b020 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:07.006 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.975+0000 7f18f1ffb700 0 client.15666 ms_handle_reset on v2:172.21.15.155:6836/947670056 2023-07-21T18:42:07.006 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.975+0000 7f191c77cb80 1 --2- 172.21.15.39:0/227179341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b86908be0 0x560b8696b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:07.006 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.975+0000 7f191c77cb80 1 -- 172.21.15.39:0/227179341 >> 172.21.15.39:0/227179341 conn(0x560b868feda0 msgr2=0x560b8690bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:07.007 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.975+0000 7f191c77cb80 1 -- 172.21.15.39:0/227179341 shutdown_connections 2023-07-21T18:42:07.007 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.975+0000 7f191c77cb80 1 -- 172.21.15.39:0/227179341 wait complete. 2023-07-21T18:42:07.007 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.975+0000 7f191c77cb80 20 client.15666 trim_cache size 0 max 16384 2023-07-21T18:42:07.007 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2023-07-21T18:42:07.008 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.976+0000 7f19105f6700 1 -- 172.21.15.39:0/2778360306 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f18f40a13e0 msgr2=0x7f18f40a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:07.008 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.976+0000 7f19105f6700 1 --2- 172.21.15.39:0/2778360306 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f18f40a13e0 0x7f18f40a3890 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f190400b5c0 tx=0x7f1904005fd0 comp rx=0 tx=0).stop 2023-07-21T18:42:07.008 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.976+0000 7f19105f6700 1 -- 172.21.15.39:0/2778360306 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19081004a0 msgr2=0x7f190819bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:07.008 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.976+0000 7f19105f6700 1 --2- 172.21.15.39:0/2778360306 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19081004a0 0x7f190819bee0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f18f800d950 tx=0x7f18f800dc60 comp rx=0 tx=0).stop 2023-07-21T18:42:07.008 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.976+0000 7f19105f6700 1 -- 172.21.15.39:0/2778360306 shutdown_connections 2023-07-21T18:42:07.009 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.976+0000 7f19105f6700 1 --2- 172.21.15.39:0/2778360306 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f18f40a13e0 0x7f18f40a3890 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:07.009 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.976+0000 7f19105f6700 1 --2- 172.21.15.39:0/2778360306 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19080ffaf0 0x7f190819b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:07.009 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.976+0000 7f19105f6700 1 --2- 172.21.15.39:0/2778360306 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1908105b00 0x7f1908195a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:07.009 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.976+0000 7f19105f6700 1 --2- 172.21.15.39:0/2778360306 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19081004a0 0x7f190819bee0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:07.009 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.976+0000 7f19105f6700 1 -- 172.21.15.39:0/2778360306 >> 172.21.15.39:0/2778360306 conn(0x7f19080fb650 msgr2=0x7f1908103600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:07.009 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.976+0000 7f19105f6700 1 -- 172.21.15.39:0/2778360306 shutdown_connections 2023-07-21T18:42:07.009 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:06.976+0000 7f19105f6700 1 -- 172.21.15.39:0/2778360306 wait complete. 2023-07-21T18:42:07.010 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-21T18:42:07.140 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.119+0000 7fdc57fc7700 1 -- 172.21.15.39:0/1407756846 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc500fee10 msgr2=0x7fdc50102d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:07.141 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.119+0000 7fdc57fc7700 1 --2- 172.21.15.39:0/1407756846 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc500fee10 0x7fdc50102d50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fdc400098f0 tx=0x7fdc40009c00 comp rx=0 tx=0).stop 2023-07-21T18:42:07.141 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.119+0000 7fdc57fc7700 1 -- 172.21.15.39:0/1407756846 shutdown_connections 2023-07-21T18:42:07.141 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.119+0000 7fdc57fc7700 1 --2- 172.21.15.39:0/1407756846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc50103430 0x7fdc5010b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:07.141 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.119+0000 7fdc57fc7700 1 --2- 172.21.15.39:0/1407756846 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc500fe470 0x7fdc500fe840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:07.141 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.119+0000 7fdc57fc7700 1 --2- 172.21.15.39:0/1407756846 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc500fee10 0x7fdc50102d50 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:07.141 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.119+0000 7fdc57fc7700 1 -- 172.21.15.39:0/1407756846 >> 172.21.15.39:0/1407756846 conn(0x7fdc50076200 msgr2=0x7fdc50076600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:07.141 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.119+0000 7fdc57fc7700 1 -- 172.21.15.39:0/1407756846 shutdown_connections 2023-07-21T18:42:07.142 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.119+0000 7fdc57fc7700 1 -- 172.21.15.39:0/1407756846 wait complete. 2023-07-21T18:42:07.143 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.119+0000 7fdc57fc7700 1 Processor -- start 2023-07-21T18:42:07.143 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.119+0000 7fdc57fc7700 1 -- start start 2023-07-21T18:42:07.143 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.120+0000 7fdc57fc7700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc500fe470 0x7fdc50195ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:07.144 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.120+0000 7fdc57fc7700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc500fee10 0x7fdc50196220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:07.144 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.120+0000 7fdc57fc7700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc50103430 0x7fdc501995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:07.144 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.120+0000 7fdc57fc7700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc50199bd0 con 0x7fdc50103430 2023-07-21T18:42:07.144 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.120+0000 7fdc57fc7700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc50199d40 con 0x7fdc500fee10 2023-07-21T18:42:07.144 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.120+0000 7fdc57fc7700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc50199f00 con 0x7fdc500fe470 2023-07-21T18:42:07.144 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.120+0000 7fdc56564700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc50103430 0x7fdc501995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:07.144 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.120+0000 7fdc56564700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc50103430 0x7fdc501995d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57752/0 (socket says 172.21.15.39:57752) 2023-07-21T18:42:07.144 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.120+0000 7fdc56564700 1 -- 172.21.15.39:0/3565814654 learned_addr learned my addr 172.21.15.39:0/3565814654 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:07.145 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.120+0000 7fdc55d63700 1 --2- 172.21.15.39:0/3565814654 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc500fe470 0x7fdc50195ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:07.145 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.120+0000 7fdc55562700 1 --2- 172.21.15.39:0/3565814654 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc500fee10 0x7fdc50196220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:07.145 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.120+0000 7fdc56564700 1 -- 172.21.15.39:0/3565814654 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc500fe470 msgr2=0x7fdc50195ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:07.145 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.120+0000 7fdc56564700 1 --2- 172.21.15.39:0/3565814654 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc500fe470 0x7fdc50195ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:07.145 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.120+0000 7fdc56564700 1 -- 172.21.15.39:0/3565814654 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc500fee10 msgr2=0x7fdc50196220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:07.145 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.120+0000 7fdc56564700 1 --2- 172.21.15.39:0/3565814654 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc500fee10 0x7fdc50196220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:07.145 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.120+0000 7fdc56564700 1 -- 172.21.15.39:0/3565814654 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc4c00a700 con 0x7fdc50103430 2023-07-21T18:42:07.146 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.120+0000 7fdc55562700 1 --2- 172.21.15.39:0/3565814654 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc500fee10 0x7fdc50196220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:42:07.146 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.120+0000 7fdc56564700 1 --2- 172.21.15.39:0/3565814654 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc50103430 0x7fdc501995d0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fdc4800ef30 tx=0x7fdc4800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:07.146 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.121+0000 7fdc46ffd700 1 -- 172.21.15.39:0/3565814654 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc48018070 con 0x7fdc50103430 2023-07-21T18:42:07.146 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.121+0000 7fdc46ffd700 1 -- 172.21.15.39:0/3565814654 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc4800f040 con 0x7fdc50103430 2023-07-21T18:42:07.146 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.121+0000 7fdc46ffd700 1 -- 172.21.15.39:0/3565814654 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc4800cdb0 con 0x7fdc50103430 2023-07-21T18:42:07.146 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.121+0000 7fdc57fc7700 1 -- 172.21.15.39:0/3565814654 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc40009520 con 0x7fdc50103430 2023-07-21T18:42:07.146 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.121+0000 7fdc57fc7700 1 -- 172.21.15.39:0/3565814654 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc501a2960 con 0x7fdc50103430 2023-07-21T18:42:07.148 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.123+0000 7fdc46ffd700 1 -- 172.21.15.39:0/3565814654 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7fdc48003a00 con 0x7fdc50103430 2023-07-21T18:42:07.149 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.124+0000 7fdc46ffd700 1 --2- 172.21.15.39:0/3565814654 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fdc3c0a13e0 0x7fdc3c0a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:07.149 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.124+0000 7fdc55d63700 1 --2- 172.21.15.39:0/3565814654 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fdc3c0a13e0 0x7fdc3c0a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:07.149 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.124+0000 7fdc46ffd700 1 -- 172.21.15.39:0/3565814654 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fdc480ccaf0 con 0x7fdc50103430 2023-07-21T18:42:07.149 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.124+0000 7fdc6414db80 1 Processor -- start 2023-07-21T18:42:07.149 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.124+0000 7fdc55d63700 1 --2- 172.21.15.39:0/3565814654 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fdc3c0a13e0 0x7fdc3c0a3890 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c005fd0 tx=0x7fdc4c010040 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:07.149 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.125+0000 7fdc6414db80 1 -- start start 2023-07-21T18:42:07.149 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.125+0000 7fdc6414db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b71289b2d0 0x55b71289b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:07.150 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.125+0000 7fdc6414db80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b71289bbe0 0x55b7128a6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:07.150 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.125+0000 7fdc6414db80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b7128a6ad0 0x55b7128a8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:07.150 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.125+0000 7fdc6414db80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7127a1cb0 con 0x55b7128a6ad0 2023-07-21T18:42:07.150 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.125+0000 7fdc6414db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7127f4a40 con 0x55b71289b2d0 2023-07-21T18:42:07.150 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.125+0000 7fdc6414db80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7127eded0 con 0x55b71289bbe0 2023-07-21T18:42:07.150 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.125+0000 7fdc56564700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b7128a6ad0 0x55b7128a8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:07.150 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.125+0000 7fdc56564700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b7128a6ad0 0x55b7128a8eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57760/0 (socket says 172.21.15.39:57760) 2023-07-21T18:42:07.150 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.125+0000 7fdc56564700 1 -- 172.21.15.39:0/518484582 learned_addr learned my addr 172.21.15.39:0/518484582 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:07.151 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.125+0000 7fdc55d63700 1 --2- 172.21.15.39:0/518484582 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b71289b2d0 0x55b71289b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:07.151 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.125+0000 7fdc55562700 1 --2- 172.21.15.39:0/518484582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b71289bbe0 0x55b7128a6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:07.151 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.125+0000 7fdc56564700 1 -- 172.21.15.39:0/518484582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b71289bbe0 msgr2=0x55b7128a6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:07.151 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.125+0000 7fdc56564700 1 --2- 172.21.15.39:0/518484582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b71289bbe0 0x55b7128a6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:07.151 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.125+0000 7fdc56564700 1 -- 172.21.15.39:0/518484582 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b71289b2d0 msgr2=0x55b71289b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:07.151 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.125+0000 7fdc56564700 1 --2- 172.21.15.39:0/518484582 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b71289b2d0 0x55b71289b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:07.151 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.125+0000 7fdc56564700 1 -- 172.21.15.39:0/518484582 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b71261bde0 con 0x55b7128a6ad0 2023-07-21T18:42:07.151 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.125+0000 7fdc55d63700 1 --2- 172.21.15.39:0/518484582 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b71289b2d0 0x55b71289b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:42:07.152 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.125+0000 7fdc56564700 1 --2- 172.21.15.39:0/518484582 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b7128a6ad0 0x55b7128a8eb0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fdc48096490 tx=0x7fdc48099870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f767d491a5cdf81e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:07.152 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.126+0000 7fdc3bfff700 1 -- 172.21.15.39:0/518484582 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc48004ad0 con 0x55b7128a6ad0 2023-07-21T18:42:07.152 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.126+0000 7fdc3bfff700 1 -- 172.21.15.39:0/518484582 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc48099a20 con 0x55b7128a6ad0 2023-07-21T18:42:07.152 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.126+0000 7fdc3bfff700 1 -- 172.21.15.39:0/518484582 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc48099170 con 0x55b7128a6ad0 2023-07-21T18:42:07.152 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.126+0000 7fdc6414db80 1 -- 172.21.15.39:0/518484582 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b7128a6ad0 msgr2=0x55b7128a8eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:07.152 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.126+0000 7fdc6414db80 1 --2- 172.21.15.39:0/518484582 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b7128a6ad0 0x55b7128a8eb0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fdc48096490 tx=0x7fdc48099870 comp rx=0 tx=0).stop 2023-07-21T18:42:07.152 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.126+0000 7fdc6414db80 1 -- 172.21.15.39:0/518484582 shutdown_connections 2023-07-21T18:42:07.153 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.126+0000 7fdc6414db80 1 --2- 172.21.15.39:0/518484582 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b71289b2d0 0x55b71289b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:07.153 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.126+0000 7fdc6414db80 1 --2- 172.21.15.39:0/518484582 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b7128a6ad0 0x55b7128a8eb0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:07.153 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.126+0000 7fdc6414db80 1 --2- 172.21.15.39:0/518484582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b71289bbe0 0x55b7128a6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:07.153 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.126+0000 7fdc6414db80 1 -- 172.21.15.39:0/518484582 >> 172.21.15.39:0/518484582 conn(0x55b712891da0 msgr2=0x55b7128976a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:07.153 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.126+0000 7fdc6414db80 1 -- 172.21.15.39:0/518484582 shutdown_connections 2023-07-21T18:42:07.153 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.126+0000 7fdc6414db80 1 -- 172.21.15.39:0/518484582 wait complete. 2023-07-21T18:42:07.153 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.127+0000 7fdc6414db80 1 -- start start 2023-07-21T18:42:07.153 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.127+0000 7fdc6414db80 1 Processor -- start 2023-07-21T18:42:07.156 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.127+0000 7fdc6414db80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b71289b2d0 0x55b7128a6550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:07.156 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.127+0000 7fdc6414db80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b71289bbe0 0x55b7128a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:07.156 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.127+0000 7fdc6414db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b7128a6ad0 0x55b7128a3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:07.156 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.127+0000 7fdc6414db80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b71278dac0 con 0x55b71289b2d0 2023-07-21T18:42:07.156 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.127+0000 7fdc6414db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b71278d7c0 con 0x55b7128a6ad0 2023-07-21T18:42:07.157 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.127+0000 7fdc6414db80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b712671920 con 0x55b71289bbe0 2023-07-21T18:42:07.157 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.127+0000 7fdc6414db80 10 client.0 _ll_register_callbacks cb 0x55b71278b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:42:07.157 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.128+0000 7fdc55d63700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b71289b2d0 0x55b7128a6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:07.157 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.128+0000 7fdc55d63700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b71289b2d0 0x55b7128a6550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57762/0 (socket says 172.21.15.39:57762) 2023-07-21T18:42:07.157 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.128+0000 7fdc55d63700 1 -- 172.21.15.39:0/170110575 learned_addr learned my addr 172.21.15.39:0/170110575 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:07.157 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.128+0000 7fdc55562700 1 --2- 172.21.15.39:0/170110575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b71289bbe0 0x55b7128a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:07.157 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.128+0000 7fdc56564700 1 --2- 172.21.15.39:0/170110575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b7128a6ad0 0x55b7128a3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:07.157 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.128+0000 7fdc55d63700 1 -- 172.21.15.39:0/170110575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b71289bbe0 msgr2=0x55b7128a37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:07.158 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.128+0000 7fdc55d63700 1 --2- 172.21.15.39:0/170110575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b71289bbe0 0x55b7128a37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:07.158 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.128+0000 7fdc55d63700 1 -- 172.21.15.39:0/170110575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b7128a6ad0 msgr2=0x55b7128a3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:07.158 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.128+0000 7fdc55d63700 1 --2- 172.21.15.39:0/170110575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b7128a6ad0 0x55b7128a3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:07.158 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.128+0000 7fdc55d63700 1 -- 172.21.15.39:0/170110575 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc501a2960 con 0x55b71289b2d0 2023-07-21T18:42:07.158 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.128+0000 7fdc56564700 1 --2- 172.21.15.39:0/170110575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b7128a6ad0 0x55b7128a3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:42:07.158 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.128+0000 7fdc55d63700 1 --2- 172.21.15.39:0/170110575 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b71289b2d0 0x55b7128a6550 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c00ae30 tx=0x7fdc4c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=838895e4ea319403 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:07.158 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.128+0000 7fdc397fa700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:42:07.158 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.128+0000 7fdc397fa700 1 -- 172.21.15.39:0/170110575 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc4c013040 con 0x55b71289b2d0 2023-07-21T18:42:07.159 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.128+0000 7fdc397fa700 1 -- 172.21.15.39:0/170110575 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fdc4c00ee40 con 0x55b71289b2d0 2023-07-21T18:42:07.159 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.128+0000 7fdc6414db80 1 -- 172.21.15.39:0/170110575 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b7127474f0 con 0x55b71289b2d0 2023-07-21T18:42:07.159 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.129+0000 7fdc397fa700 1 -- 172.21.15.39:0/170110575 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc4c00ca70 con 0x55b71289b2d0 2023-07-21T18:42:07.159 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.129+0000 7fdc6414db80 10 client.15678 fetch_fsmap learned FSMap version 26 2023-07-21T18:42:07.159 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.129+0000 7fdc6414db80 1 -- 172.21.15.39:0/170110575 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55b712604c60 con 0x55b71289b2d0 2023-07-21T18:42:07.159 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.129+0000 7fdc397fa700 1 -- 172.21.15.39:0/170110575 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc4c032020 con 0x55b71289b2d0 2023-07-21T18:42:07.159 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.129+0000 7fdc397fa700 1 -- 172.21.15.39:0/170110575 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fdc4c025cf0 con 0x55b71289b2d0 2023-07-21T18:42:07.159 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.129+0000 7fdc6414db80 10 client.15678 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T18:42:07.160 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.129+0000 7fdc6414db80 10 client.15678 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T18:42:07.160 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.129+0000 7fdc6414db80 20 client.15678 populate_metadata read hostname 'smithi039' 2023-07-21T18:42:07.160 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.129+0000 7fdc6414db80 1 --2- 172.21.15.39:0/170110575 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55b7128b6480 0x55b7128b8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:07.160 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.129+0000 7fdc6414db80 4 client.15678 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:42:07.160 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.129+0000 7fdc6414db80 1 -- 172.21.15.39:0/170110575 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b7127e6eb0 con 0x55b7128b6480 2023-07-21T18:42:07.160 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.129+0000 7fdc55562700 1 --2- 172.21.15.39:0/170110575 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55b7128b6480 0x55b7128b8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:07.160 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.130+0000 7fdc397fa700 10 client.15678 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:42:07.161 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.130+0000 7fdc55562700 1 --2- 172.21.15.39:0/170110575 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55b7128b6480 0x55b7128b8890 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9aac2830ce91519 server_cookie=6c8d112a10912bbb in_seq=0 out_seq=0 2023-07-21T18:42:07.161 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.130+0000 7fdc397fa700 1 -- 172.21.15.39:0/170110575 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b7127e6eb0 con 0x55b7128b6480 2023-07-21T18:42:07.161 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.130+0000 7fdc397fa700 10 client.15678 handle_command_reply: tid=0 2023-07-21T18:42:07.161 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.130+0000 7fdc6414db80 1 client.15678 shutdown 2023-07-21T18:42:07.161 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.130+0000 7fdc6414db80 1 -- 172.21.15.39:0/170110575 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b71289b2d0 msgr2=0x55b7128a6550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:07.161 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.130+0000 7fdc6414db80 1 --2- 172.21.15.39:0/170110575 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b71289b2d0 0x55b7128a6550 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c00ae30 tx=0x7fdc4c004750 comp rx=0 tx=0).stop 2023-07-21T18:42:07.161 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.131+0000 7fdc6414db80 1 -- 172.21.15.39:0/170110575 shutdown_connections 2023-07-21T18:42:07.161 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.131+0000 7fdc6414db80 1 --2- 172.21.15.39:0/170110575 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55b7128b6480 0x55b7128b8890 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:07.162 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.131+0000 7fdc6414db80 1 --2- 172.21.15.39:0/170110575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b7128a6ad0 0x55b7128a3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:07.162 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.131+0000 7fdc6414db80 1 --2- 172.21.15.39:0/170110575 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b71289b2d0 0x55b7128a6550 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:07.162 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.131+0000 7fdc397fa700 0 client.15678 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:42:07.162 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.131+0000 7fdc6414db80 1 --2- 172.21.15.39:0/170110575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b71289bbe0 0x55b7128a37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:07.162 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.131+0000 7fdc6414db80 1 -- 172.21.15.39:0/170110575 >> 172.21.15.39:0/170110575 conn(0x55b712891da0 msgr2=0x55b712897290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:07.162 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.131+0000 7fdc6414db80 1 -- 172.21.15.39:0/170110575 shutdown_connections 2023-07-21T18:42:07.162 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.131+0000 7fdc6414db80 1 -- 172.21.15.39:0/170110575 wait complete. 2023-07-21T18:42:07.162 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.131+0000 7fdc6414db80 20 client.15678 trim_cache size 0 max 16384 2023-07-21T18:42:07.166 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.144+0000 7fdc6414db80 1 Processor -- start 2023-07-21T18:42:07.166 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.144+0000 7fdc6414db80 1 -- start start 2023-07-21T18:42:07.167 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.145+0000 7fdc6414db80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b71289b2d0 0x55b7128a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:07.167 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.145+0000 7fdc6414db80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b71289bbe0 0x55b71289eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:07.167 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.145+0000 7fdc6414db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b7128a6ad0 0x55b71289f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:07.167 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.145+0000 7fdc6414db80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7127e6eb0 con 0x55b71289bbe0 2023-07-21T18:42:07.167 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.145+0000 7fdc6414db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b71289e680 con 0x55b7128a6ad0 2023-07-21T18:42:07.167 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.145+0000 7fdc6414db80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b71289c540 con 0x55b71289b2d0 2023-07-21T18:42:07.167 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.145+0000 7fdc55562700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b71289bbe0 0x55b71289eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:07.167 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.145+0000 7fdc55562700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b71289bbe0 0x55b71289eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57772/0 (socket says 172.21.15.39:57772) 2023-07-21T18:42:07.168 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.145+0000 7fdc55562700 1 -- 172.21.15.39:0/368017575 learned_addr learned my addr 172.21.15.39:0/368017575 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:07.168 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.145+0000 7fdc55d63700 1 --2- 172.21.15.39:0/368017575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b71289b2d0 0x55b7128a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:07.171 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.145+0000 7fdc56564700 1 --2- 172.21.15.39:0/368017575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b7128a6ad0 0x55b71289f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:07.171 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.145+0000 7fdc55562700 1 -- 172.21.15.39:0/368017575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b71289b2d0 msgr2=0x55b7128a60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:07.171 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.145+0000 7fdc55562700 1 --2- 172.21.15.39:0/368017575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b71289b2d0 0x55b7128a60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:07.171 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.145+0000 7fdc55562700 1 -- 172.21.15.39:0/368017575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b7128a6ad0 msgr2=0x55b71289f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:07.171 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.145+0000 7fdc55562700 1 --2- 172.21.15.39:0/368017575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b7128a6ad0 0x55b71289f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:07.171 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.145+0000 7fdc55562700 1 -- 172.21.15.39:0/368017575 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b712642250 con 0x55b71289bbe0 2023-07-21T18:42:07.172 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.145+0000 7fdc56564700 1 --2- 172.21.15.39:0/368017575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b7128a6ad0 0x55b71289f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:42:07.172 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.145+0000 7fdc55562700 1 --2- 172.21.15.39:0/368017575 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b71289bbe0 0x55b71289eaf0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fdc500ff1e0 tx=0x7fdc40009230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaedf0eb2b12a173 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:07.172 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.146+0000 7fdc44ff9700 1 -- 172.21.15.39:0/368017575 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc4001c040 con 0x55b71289bbe0 2023-07-21T18:42:07.172 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.146+0000 7fdc44ff9700 1 -- 172.21.15.39:0/368017575 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc4000f8a0 con 0x55b71289bbe0 2023-07-21T18:42:07.172 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.146+0000 7fdc44ff9700 1 -- 172.21.15.39:0/368017575 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc4000b560 con 0x55b71289bbe0 2023-07-21T18:42:07.172 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.146+0000 7fdc6414db80 1 -- 172.21.15.39:0/368017575 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b71289bbe0 msgr2=0x55b71289eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:07.172 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.146+0000 7fdc6414db80 1 --2- 172.21.15.39:0/368017575 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b71289bbe0 0x55b71289eaf0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fdc500ff1e0 tx=0x7fdc40009230 comp rx=0 tx=0).stop 2023-07-21T18:42:07.172 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.146+0000 7fdc6414db80 1 -- 172.21.15.39:0/368017575 shutdown_connections 2023-07-21T18:42:07.173 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.146+0000 7fdc6414db80 1 --2- 172.21.15.39:0/368017575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b7128a6ad0 0x55b71289f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:07.173 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.146+0000 7fdc6414db80 1 --2- 172.21.15.39:0/368017575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b71289b2d0 0x55b7128a60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:07.173 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.146+0000 7fdc6414db80 1 --2- 172.21.15.39:0/368017575 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b71289bbe0 0x55b71289eaf0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:07.173 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.146+0000 7fdc6414db80 1 -- 172.21.15.39:0/368017575 >> 172.21.15.39:0/368017575 conn(0x55b712891da0 msgr2=0x55b712895e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:07.173 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.146+0000 7fdc6414db80 1 -- 172.21.15.39:0/368017575 shutdown_connections 2023-07-21T18:42:07.173 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.146+0000 7fdc6414db80 1 -- 172.21.15.39:0/368017575 wait complete. 2023-07-21T18:42:07.173 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.147+0000 7fdc6414db80 1 -- start start 2023-07-21T18:42:07.173 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.147+0000 7fdc6414db80 1 Processor -- start 2023-07-21T18:42:07.174 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.147+0000 7fdc6414db80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b71289b2d0 0x55b7128ad170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:07.174 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.147+0000 7fdc6414db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b71289bbe0 0x55b7128ad6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:07.174 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.147+0000 7fdc6414db80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b7128a6ad0 0x55b7128ab640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:07.174 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.147+0000 7fdc6414db80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7128abd40 con 0x55b71289b2d0 2023-07-21T18:42:07.174 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.147+0000 7fdc6414db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7128abeb0 con 0x55b71289bbe0 2023-07-21T18:42:07.174 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.147+0000 7fdc6414db80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7128ac070 con 0x55b7128a6ad0 2023-07-21T18:42:07.174 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.148+0000 7fdc6414db80 10 client.0 _ll_register_callbacks cb 0x7fdc4c019890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:42:07.175 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.148+0000 7fdc55d63700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b71289b2d0 0x55b7128ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:07.175 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.148+0000 7fdc55d63700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b71289b2d0 0x55b7128ad170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57776/0 (socket says 172.21.15.39:57776) 2023-07-21T18:42:07.175 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.148+0000 7fdc55d63700 1 -- 172.21.15.39:0/3239351569 learned_addr learned my addr 172.21.15.39:0/3239351569 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:07.175 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.148+0000 7fdc55562700 1 --2- 172.21.15.39:0/3239351569 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b71289bbe0 0x55b7128ad6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:07.175 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.148+0000 7fdc56564700 1 --2- 172.21.15.39:0/3239351569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b7128a6ad0 0x55b7128ab640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:07.175 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.148+0000 7fdc55d63700 1 -- 172.21.15.39:0/3239351569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b7128a6ad0 msgr2=0x55b7128ab640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:07.175 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.148+0000 7fdc55d63700 1 --2- 172.21.15.39:0/3239351569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b7128a6ad0 0x55b7128ab640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:07.175 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.148+0000 7fdc55d63700 1 -- 172.21.15.39:0/3239351569 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b71289bbe0 msgr2=0x55b7128ad6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:07.176 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.148+0000 7fdc55d63700 1 --2- 172.21.15.39:0/3239351569 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b71289bbe0 0x55b7128ad6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:07.176 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.148+0000 7fdc55d63700 1 -- 172.21.15.39:0/3239351569 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc5010b810 con 0x55b71289b2d0 2023-07-21T18:42:07.176 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.148+0000 7fdc55562700 1 --2- 172.21.15.39:0/3239351569 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b71289bbe0 0x55b7128ad6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:42:07.176 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.148+0000 7fdc55d63700 1 --2- 172.21.15.39:0/3239351569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b71289b2d0 0x55b7128ad170 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c00eb70 tx=0x7fdc4c00cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4d67e4748a281fa server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:07.176 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.148+0000 7fdc397fa700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:42:07.176 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.148+0000 7fdc397fa700 1 -- 172.21.15.39:0/3239351569 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc4c013070 con 0x55b71289b2d0 2023-07-21T18:42:07.176 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.148+0000 7fdc397fa700 1 -- 172.21.15.39:0/3239351569 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fdc4c033080 con 0x55b71289b2d0 2023-07-21T18:42:07.176 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.148+0000 7fdc6414db80 1 -- 172.21.15.39:0/3239351569 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdc48099a20 con 0x55b71289b2d0 2023-07-21T18:42:07.177 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.149+0000 7fdc397fa700 1 -- 172.21.15.39:0/3239351569 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc4c005ce0 con 0x55b71289b2d0 2023-07-21T18:42:07.177 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.149+0000 7fdc6414db80 10 client.15684 fetch_fsmap learned FSMap version 26 2023-07-21T18:42:07.177 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.149+0000 7fdc6414db80 1 -- 172.21.15.39:0/3239351569 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55b71261a2b0 con 0x55b71289b2d0 2023-07-21T18:42:07.177 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.149+0000 7fdc397fa700 1 -- 172.21.15.39:0/3239351569 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc48099a20 con 0x55b71289b2d0 2023-07-21T18:42:07.177 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.149+0000 7fdc397fa700 1 -- 172.21.15.39:0/3239351569 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fdc4c00dd10 con 0x55b71289b2d0 2023-07-21T18:42:07.177 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.149+0000 7fdc6414db80 10 client.15684 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T18:42:07.177 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.149+0000 7fdc6414db80 10 client.15684 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T18:42:07.177 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.149+0000 7fdc6414db80 20 client.15684 populate_metadata read hostname 'smithi039' 2023-07-21T18:42:07.178 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.149+0000 7fdc6414db80 1 --2- 172.21.15.39:0/3239351569 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55b71290b560 0x55b71290d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:07.178 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.149+0000 7fdc6414db80 4 client.15684 mds_command: new command op to 24536 tid=0[{"prefix": "damage ls"}] 2023-07-21T18:42:07.178 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.149+0000 7fdc6414db80 1 -- 172.21.15.39:0/3239351569 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b7127a9050 con 0x55b71290b560 2023-07-21T18:42:07.178 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.149+0000 7fdc55562700 1 --2- 172.21.15.39:0/3239351569 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55b71290b560 0x55b71290d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:07.179 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.150+0000 7fdc397fa700 10 client.15684 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:42:07.179 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.150+0000 7fdc55562700 1 --2- 172.21.15.39:0/3239351569 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55b71290b560 0x55b71290d970 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5816f4b935df0b14 server_cookie=53f285dbde9ba24d in_seq=0 out_seq=0 2023-07-21T18:42:07.179 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.150+0000 7fdc397fa700 1 -- 172.21.15.39:0/3239351569 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b7127a9050 con 0x55b71290b560 2023-07-21T18:42:07.179 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.150+0000 7fdc397fa700 10 client.15684 handle_command_reply: tid=0 2023-07-21T18:42:07.180 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.150+0000 7fdc6414db80 1 client.15684 shutdown 2023-07-21T18:42:07.180 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.150+0000 7fdc6414db80 1 -- 172.21.15.39:0/3239351569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b71289b2d0 msgr2=0x55b7128ad170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:07.180 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.150+0000 7fdc6414db80 1 --2- 172.21.15.39:0/3239351569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b71289b2d0 0x55b7128ad170 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c00eb70 tx=0x7fdc4c00cd20 comp rx=0 tx=0).stop 2023-07-21T18:42:07.180 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.150+0000 7fdc6414db80 1 -- 172.21.15.39:0/3239351569 shutdown_connections 2023-07-21T18:42:07.180 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.150+0000 7fdc6414db80 1 --2- 172.21.15.39:0/3239351569 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55b71290b560 0x55b71290d970 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:07.180 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.150+0000 7fdc6414db80 1 --2- 172.21.15.39:0/3239351569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b7128a6ad0 0x55b7128ab640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:07.180 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.150+0000 7fdc397fa700 0 client.15684 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:42:07.181 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.150+0000 7fdc6414db80 1 --2- 172.21.15.39:0/3239351569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b71289b2d0 0x55b7128ad170 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:07.181 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.150+0000 7fdc6414db80 1 --2- 172.21.15.39:0/3239351569 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b71289bbe0 0x55b7128ad6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:07.181 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.150+0000 7fdc6414db80 1 -- 172.21.15.39:0/3239351569 >> 172.21.15.39:0/3239351569 conn(0x55b712891da0 msgr2=0x55b71289eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:07.181 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.150+0000 7fdc6414db80 1 -- 172.21.15.39:0/3239351569 shutdown_connections 2023-07-21T18:42:07.181 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.150+0000 7fdc6414db80 1 -- 172.21.15.39:0/3239351569 wait complete. 2023-07-21T18:42:07.181 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.151+0000 7fdc6414db80 20 client.15684 trim_cache size 0 max 16384 2023-07-21T18:42:07.181 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2023-07-21T18:42:07.182 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.151+0000 7fdc57fc7700 1 -- 172.21.15.39:0/3565814654 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fdc3c0a13e0 msgr2=0x7fdc3c0a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:07.182 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.151+0000 7fdc57fc7700 1 --2- 172.21.15.39:0/3565814654 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fdc3c0a13e0 0x7fdc3c0a3890 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c005fd0 tx=0x7fdc4c010040 comp rx=0 tx=0).stop 2023-07-21T18:42:07.182 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.151+0000 7fdc57fc7700 1 -- 172.21.15.39:0/3565814654 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc50103430 msgr2=0x7fdc501995d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:07.183 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.151+0000 7fdc57fc7700 1 --2- 172.21.15.39:0/3565814654 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc50103430 0x7fdc501995d0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fdc4800ef30 tx=0x7fdc4800c3a0 comp rx=0 tx=0).stop 2023-07-21T18:42:07.183 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.151+0000 7fdc57fc7700 1 -- 172.21.15.39:0/3565814654 shutdown_connections 2023-07-21T18:42:07.183 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.151+0000 7fdc57fc7700 1 --2- 172.21.15.39:0/3565814654 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc50103430 0x7fdc501995d0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:07.183 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.151+0000 7fdc57fc7700 1 --2- 172.21.15.39:0/3565814654 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc500fe470 0x7fdc50195ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:07.183 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.151+0000 7fdc57fc7700 1 --2- 172.21.15.39:0/3565814654 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fdc3c0a13e0 0x7fdc3c0a3890 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:07.183 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.151+0000 7fdc57fc7700 1 --2- 172.21.15.39:0/3565814654 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc500fee10 0x7fdc50196220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:07.183 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.151+0000 7fdc57fc7700 1 -- 172.21.15.39:0/3565814654 >> 172.21.15.39:0/3565814654 conn(0x7fdc50076200 msgr2=0x7fdc500fc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:07.183 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.151+0000 7fdc57fc7700 1 -- 172.21.15.39:0/3565814654 shutdown_connections 2023-07-21T18:42:07.184 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:07.151+0000 7fdc57fc7700 1 -- 172.21.15.39:0/3565814654 wait complete. 2023-07-21T18:42:07.369 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:42:07 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/313026987' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:42:07.388 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:42:07 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/313026987' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:42:07.437 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:42:07 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/313026987' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:42:08.185 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-21T18:42:08.186 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-21T18:42:08.319 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.297+0000 7f1ed8b8e700 1 -- 172.21.15.39:0/2823336961 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ed40ff1c0 msgr2=0x7f1ed410be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:08.320 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.297+0000 7f1ed8b8e700 1 --2- 172.21.15.39:0/2823336961 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ed40ff1c0 0x7f1ed410be90 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc009990 tx=0x7f1ebc009ca0 comp rx=0 tx=0).stop 2023-07-21T18:42:08.320 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.297+0000 7f1ed8b8e700 1 -- 172.21.15.39:0/2823336961 shutdown_connections 2023-07-21T18:42:08.320 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.297+0000 7f1ed8b8e700 1 --2- 172.21.15.39:0/2823336961 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ed40fe820 0x7f1ed40febf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:08.320 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.297+0000 7f1ed8b8e700 1 --2- 172.21.15.39:0/2823336961 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ed410c3d0 0x7f1ed410e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:08.320 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.297+0000 7f1ed8b8e700 1 --2- 172.21.15.39:0/2823336961 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ed40ff1c0 0x7f1ed410be90 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:08.321 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.297+0000 7f1ed8b8e700 1 -- 172.21.15.39:0/2823336961 >> 172.21.15.39:0/2823336961 conn(0x7f1ed40fa4b0 msgr2=0x7f1ed40fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:08.321 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.298+0000 7f1ed8b8e700 1 -- 172.21.15.39:0/2823336961 shutdown_connections 2023-07-21T18:42:08.321 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.298+0000 7f1ed8b8e700 1 -- 172.21.15.39:0/2823336961 wait complete. 2023-07-21T18:42:08.323 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.298+0000 7f1ed8b8e700 1 Processor -- start 2023-07-21T18:42:08.323 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.299+0000 7f1ed8b8e700 1 -- start start 2023-07-21T18:42:08.323 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.299+0000 7f1ed8b8e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ed40fe820 0x7f1ed4197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:08.323 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.299+0000 7f1ed8b8e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ed40ff1c0 0x7f1ed4198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:08.323 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.299+0000 7f1ed8b8e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ed410c3d0 0x7f1ed419c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:08.323 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.299+0000 7f1ed8b8e700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed419ce20 con 0x7f1ed40fe820 2023-07-21T18:42:08.324 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.299+0000 7f1ed8b8e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed419cf90 con 0x7f1ed410c3d0 2023-07-21T18:42:08.324 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.299+0000 7f1ed8b8e700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed419d150 con 0x7f1ed40ff1c0 2023-07-21T18:42:08.324 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.299+0000 7f1ed259c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ed40fe820 0x7f1ed4197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:08.324 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.299+0000 7f1ed259c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ed40fe820 0x7f1ed4197e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57790/0 (socket says 172.21.15.39:57790) 2023-07-21T18:42:08.324 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.299+0000 7f1ed259c700 1 -- 172.21.15.39:0/1128497802 learned_addr learned my addr 172.21.15.39:0/1128497802 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:08.324 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.299+0000 7f1ec9d9b700 1 --2- 172.21.15.39:0/1128497802 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ed40ff1c0 0x7f1ed4198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:08.325 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.299+0000 7f1ed2d9d700 1 --2- 172.21.15.39:0/1128497802 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ed410c3d0 0x7f1ed419c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:08.325 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.299+0000 7f1ed259c700 1 -- 172.21.15.39:0/1128497802 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ed40ff1c0 msgr2=0x7f1ed4198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:08.325 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.299+0000 7f1ed259c700 1 --2- 172.21.15.39:0/1128497802 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ed40ff1c0 0x7f1ed4198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:08.325 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.299+0000 7f1ed259c700 1 -- 172.21.15.39:0/1128497802 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ed410c3d0 msgr2=0x7f1ed419c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:08.325 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.299+0000 7f1ed259c700 1 --2- 172.21.15.39:0/1128497802 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ed410c3d0 0x7f1ed419c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:08.325 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.299+0000 7f1ed259c700 1 -- 172.21.15.39:0/1128497802 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ebc0095f0 con 0x7f1ed40fe820 2023-07-21T18:42:08.325 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.299+0000 7f1ec9d9b700 1 --2- 172.21.15.39:0/1128497802 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ed40ff1c0 0x7f1ed4198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:42:08.326 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.299+0000 7f1ed259c700 1 --2- 172.21.15.39:0/1128497802 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ed40fe820 0x7f1ed4197e40 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f1ec400d950 tx=0x7f1ec400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:08.326 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.300+0000 7f1ecbfff700 1 -- 172.21.15.39:0/1128497802 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec4009b40 con 0x7f1ed40fe820 2023-07-21T18:42:08.326 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.300+0000 7f1ecbfff700 1 -- 172.21.15.39:0/1128497802 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ec4009ca0 con 0x7f1ed40fe820 2023-07-21T18:42:08.326 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.300+0000 7f1ed8b8e700 1 -- 172.21.15.39:0/1128497802 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ed419d430 con 0x7f1ed40fe820 2023-07-21T18:42:08.326 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.300+0000 7f1ecbfff700 1 -- 172.21.15.39:0/1128497802 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec400f620 con 0x7f1ed40fe820 2023-07-21T18:42:08.326 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.300+0000 7f1ed8b8e700 1 -- 172.21.15.39:0/1128497802 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ed41a4c70 con 0x7f1ed40fe820 2023-07-21T18:42:08.328 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.302+0000 7f1ecbfff700 1 -- 172.21.15.39:0/1128497802 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f1ec4010460 con 0x7f1ed40fe820 2023-07-21T18:42:08.328 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.302+0000 7f1ecbfff700 1 --2- 172.21.15.39:0/1128497802 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f1ec00a13e0 0x7f1ec00a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:08.328 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.303+0000 7f1ec9d9b700 1 --2- 172.21.15.39:0/1128497802 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f1ec00a13e0 0x7f1ec00a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:08.328 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.303+0000 7f1ecbfff700 1 -- 172.21.15.39:0/1128497802 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f1ec40cd330 con 0x7f1ed40fe820 2023-07-21T18:42:08.329 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.303+0000 7f1ec9d9b700 1 --2- 172.21.15.39:0/1128497802 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f1ec00a13e0 0x7f1ec00a3890 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1ed4199450 tx=0x7f1ebc005fb0 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:08.329 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.305+0000 7f1ee4d14b80 1 Processor -- start 2023-07-21T18:42:08.329 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.305+0000 7f1ee4d14b80 1 -- start start 2023-07-21T18:42:08.329 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.305+0000 7f1ee4d14b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a685c60350 0x55a685c60720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:08.329 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.305+0000 7f1ee4d14b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a685c60c60 0x55a685c6b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:08.329 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.305+0000 7f1ee4d14b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a685c6bb50 0x55a685c6df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:08.329 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.305+0000 7f1ee4d14b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a685b66d30 con 0x55a685c6bb50 2023-07-21T18:42:08.330 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.305+0000 7f1ee4d14b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a685bb9ac0 con 0x55a685c60350 2023-07-21T18:42:08.330 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.305+0000 7f1ee4d14b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a685bb2f50 con 0x55a685c60c60 2023-07-21T18:42:08.330 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.305+0000 7f1ed2d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a685c6bb50 0x55a685c6df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:08.330 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.305+0000 7f1ed2d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a685c6bb50 0x55a685c6df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57802/0 (socket says 172.21.15.39:57802) 2023-07-21T18:42:08.330 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.305+0000 7f1ed2d9d700 1 -- 172.21.15.39:0/2426175478 learned_addr learned my addr 172.21.15.39:0/2426175478 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:08.330 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.305+0000 7f1ec9d9b700 1 --2- 172.21.15.39:0/2426175478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a685c60c60 0x55a685c6b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:08.330 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.305+0000 7f1ed259c700 1 --2- 172.21.15.39:0/2426175478 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a685c60350 0x55a685c60720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:08.331 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.306+0000 7f1ed2d9d700 1 -- 172.21.15.39:0/2426175478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a685c60c60 msgr2=0x55a685c6b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:08.331 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.306+0000 7f1ed2d9d700 1 --2- 172.21.15.39:0/2426175478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a685c60c60 0x55a685c6b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:08.331 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.306+0000 7f1ed2d9d700 1 -- 172.21.15.39:0/2426175478 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a685c60350 msgr2=0x55a685c60720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:08.331 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.306+0000 7f1ed2d9d700 1 --2- 172.21.15.39:0/2426175478 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a685c60350 0x55a685c60720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:08.331 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.306+0000 7f1ed2d9d700 1 -- 172.21.15.39:0/2426175478 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6859e0e60 con 0x55a685c6bb50 2023-07-21T18:42:08.331 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.306+0000 7f1ec9d9b700 1 --2- 172.21.15.39:0/2426175478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a685c60c60 0x55a685c6b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:42:08.331 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.306+0000 7f1ed2d9d700 1 --2- 172.21.15.39:0/2426175478 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a685c6bb50 0x55a685c6df30 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc009370 tx=0x7f1ecc009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e4f70f4a9c5cfb server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:08.332 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.306+0000 7f1ec8d99700 1 -- 172.21.15.39:0/2426175478 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ecc00e070 con 0x55a685c6bb50 2023-07-21T18:42:08.332 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.306+0000 7f1ec8d99700 1 -- 172.21.15.39:0/2426175478 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ecc00f040 con 0x55a685c6bb50 2023-07-21T18:42:08.332 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.306+0000 7f1ec8d99700 1 -- 172.21.15.39:0/2426175478 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ecc0147a0 con 0x55a685c6bb50 2023-07-21T18:42:08.335 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.307+0000 7f1ee4d14b80 1 -- 172.21.15.39:0/2426175478 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a685c6bb50 msgr2=0x55a685c6df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:08.335 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.307+0000 7f1ee4d14b80 1 --2- 172.21.15.39:0/2426175478 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a685c6bb50 0x55a685c6df30 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc009370 tx=0x7f1ecc009920 comp rx=0 tx=0).stop 2023-07-21T18:42:08.335 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.307+0000 7f1ee4d14b80 1 -- 172.21.15.39:0/2426175478 shutdown_connections 2023-07-21T18:42:08.335 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.307+0000 7f1ee4d14b80 1 --2- 172.21.15.39:0/2426175478 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a685c60350 0x55a685c60720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:08.336 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.307+0000 7f1ee4d14b80 1 --2- 172.21.15.39:0/2426175478 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a685c6bb50 0x55a685c6df30 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:08.336 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.307+0000 7f1ee4d14b80 1 --2- 172.21.15.39:0/2426175478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a685c60c60 0x55a685c6b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:08.336 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.307+0000 7f1ee4d14b80 1 -- 172.21.15.39:0/2426175478 >> 172.21.15.39:0/2426175478 conn(0x55a685c56e20 msgr2=0x55a685c5c720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:08.336 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.307+0000 7f1ee4d14b80 1 -- 172.21.15.39:0/2426175478 shutdown_connections 2023-07-21T18:42:08.336 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.307+0000 7f1ee4d14b80 1 -- 172.21.15.39:0/2426175478 wait complete. 2023-07-21T18:42:08.336 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.308+0000 7f1ee4d14b80 1 -- start start 2023-07-21T18:42:08.337 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.308+0000 7f1ee4d14b80 1 Processor -- start 2023-07-21T18:42:08.337 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.308+0000 7f1ee4d14b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a685c60350 0x55a685c6b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:08.337 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.308+0000 7f1ee4d14b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a685c60c60 0x55a685c687e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:08.337 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.308+0000 7f1ee4d14b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a685c6bb50 0x55a685c68ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:08.337 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.308+0000 7f1ee4d14b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a685b52b40 con 0x55a685c6bb50 2023-07-21T18:42:08.337 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.308+0000 7f1ee4d14b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a685b52840 con 0x55a685c60350 2023-07-21T18:42:08.337 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.308+0000 7f1ee4d14b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a685a369a0 con 0x55a685c60c60 2023-07-21T18:42:08.338 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.308+0000 7f1ee4d14b80 10 client.0 _ll_register_callbacks cb 0x55a685b507e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:42:08.338 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.308+0000 7f1ed2d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a685c6bb50 0x55a685c68ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:08.338 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.308+0000 7f1ed2d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a685c6bb50 0x55a685c68ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57808/0 (socket says 172.21.15.39:57808) 2023-07-21T18:42:08.338 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.308+0000 7f1ed2d9d700 1 -- 172.21.15.39:0/2576605713 learned_addr learned my addr 172.21.15.39:0/2576605713 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:08.338 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.308+0000 7f1ec9d9b700 1 --2- 172.21.15.39:0/2576605713 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a685c60c60 0x55a685c687e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:08.338 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.309+0000 7f1ed259c700 1 --2- 172.21.15.39:0/2576605713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a685c60350 0x55a685c6b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:08.338 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.309+0000 7f1ed2d9d700 1 -- 172.21.15.39:0/2576605713 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a685c60c60 msgr2=0x55a685c687e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:08.339 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.309+0000 7f1ed2d9d700 1 --2- 172.21.15.39:0/2576605713 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a685c60c60 0x55a685c687e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:08.339 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.309+0000 7f1ed2d9d700 1 -- 172.21.15.39:0/2576605713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a685c60350 msgr2=0x55a685c6b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:08.339 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.309+0000 7f1ed2d9d700 1 --2- 172.21.15.39:0/2576605713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a685c60350 0x55a685c6b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:08.339 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.309+0000 7f1ed2d9d700 1 -- 172.21.15.39:0/2576605713 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ed410f3f0 con 0x55a685c6bb50 2023-07-21T18:42:08.339 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.309+0000 7f1ec9d9b700 1 --2- 172.21.15.39:0/2576605713 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a685c60c60 0x55a685c687e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:42:08.339 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.309+0000 7f1ed259c700 1 --2- 172.21.15.39:0/2576605713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a685c60350 0x55a685c6b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:42:08.339 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.309+0000 7f1ed2d9d700 1 --2- 172.21.15.39:0/2576605713 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a685c6bb50 0x55a685c68ed0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc007ac0 tx=0x7f1ecc015dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=427b8288eb80dfbf server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:08.339 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.309+0000 7f1eb9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:42:08.340 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.309+0000 7f1eb9ffb700 1 -- 172.21.15.39:0/2576605713 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ecc00e040 con 0x55a685c6bb50 2023-07-21T18:42:08.340 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.309+0000 7f1eb9ffb700 1 -- 172.21.15.39:0/2576605713 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f1ecc004cd0 con 0x55a685c6bb50 2023-07-21T18:42:08.340 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.309+0000 7f1ee4d14b80 1 -- 172.21.15.39:0/2576605713 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a685b0c570 con 0x55a685c6bb50 2023-07-21T18:42:08.340 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.309+0000 7f1eb9ffb700 1 -- 172.21.15.39:0/2576605713 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ecc004e90 con 0x55a685c6bb50 2023-07-21T18:42:08.340 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.309+0000 7f1ee4d14b80 10 client.15696 fetch_fsmap learned FSMap version 26 2023-07-21T18:42:08.340 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.309+0000 7f1ee4d14b80 1 -- 172.21.15.39:0/2576605713 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f1ecc009050 con 0x55a685c6bb50 2023-07-21T18:42:08.341 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.309+0000 7f1eb9ffb700 1 -- 172.21.15.39:0/2576605713 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a685b0c570 con 0x55a685c6bb50 2023-07-21T18:42:08.341 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.309+0000 7f1eb9ffb700 1 -- 172.21.15.39:0/2576605713 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1ecc01e4e0 con 0x55a685c6bb50 2023-07-21T18:42:08.341 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.310+0000 7f1ee4d14b80 10 client.15696 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T18:42:08.341 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.310+0000 7f1ee4d14b80 10 client.15696 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T18:42:08.341 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.310+0000 7f1ee4d14b80 20 client.15696 populate_metadata read hostname 'smithi039' 2023-07-21T18:42:08.341 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.310+0000 7f1ee4d14b80 1 --2- 172.21.15.39:0/2576605713 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55a685c7b410 0x55a685c7d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:08.341 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.310+0000 7f1ee4d14b80 4 client.15696 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:42:08.342 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.310+0000 7f1ee4d14b80 1 -- 172.21.15.39:0/2576605713 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a685babf30 con 0x55a685c7b410 2023-07-21T18:42:08.342 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.310+0000 7f1ed259c700 1 --2- 172.21.15.39:0/2576605713 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55a685c7b410 0x55a685c7d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:08.342 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.310+0000 7f1eb9ffb700 10 client.15696 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:42:08.342 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.310+0000 7f1ed259c700 1 --2- 172.21.15.39:0/2576605713 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55a685c7b410 0x55a685c7d820 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7d56a4f391ae7c6 server_cookie=ba4fbac4a7a1150 in_seq=0 out_seq=0 2023-07-21T18:42:08.342 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.311+0000 7f1eb9ffb700 1 -- 172.21.15.39:0/2576605713 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a685babf30 con 0x55a685c7b410 2023-07-21T18:42:08.342 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.311+0000 7f1eb9ffb700 10 client.15696 handle_command_reply: tid=0 2023-07-21T18:42:08.342 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.311+0000 7f1ee4d14b80 1 client.15696 shutdown 2023-07-21T18:42:08.343 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.311+0000 7f1ee4d14b80 1 -- 172.21.15.39:0/2576605713 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a685c6bb50 msgr2=0x55a685c68ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:08.343 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.311+0000 7f1ee4d14b80 1 --2- 172.21.15.39:0/2576605713 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a685c6bb50 0x55a685c68ed0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc007ac0 tx=0x7f1ecc015dd0 comp rx=0 tx=0).stop 2023-07-21T18:42:08.343 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.311+0000 7f1ee4d14b80 1 -- 172.21.15.39:0/2576605713 shutdown_connections 2023-07-21T18:42:08.343 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.311+0000 7f1ee4d14b80 1 --2- 172.21.15.39:0/2576605713 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55a685c7b410 0x55a685c7d820 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:08.343 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.311+0000 7f1ee4d14b80 1 --2- 172.21.15.39:0/2576605713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a685c60350 0x55a685c6b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:08.343 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.311+0000 7f1ee4d14b80 1 --2- 172.21.15.39:0/2576605713 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a685c6bb50 0x55a685c68ed0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:08.343 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.311+0000 7f1eb9ffb700 0 client.15696 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:42:08.344 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.311+0000 7f1ee4d14b80 1 --2- 172.21.15.39:0/2576605713 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a685c60c60 0x55a685c687e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:08.344 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.311+0000 7f1ee4d14b80 1 -- 172.21.15.39:0/2576605713 >> 172.21.15.39:0/2576605713 conn(0x55a685c56e20 msgr2=0x55a685c5c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:08.344 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.311+0000 7f1ee4d14b80 1 -- 172.21.15.39:0/2576605713 shutdown_connections 2023-07-21T18:42:08.344 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.311+0000 7f1ee4d14b80 1 -- 172.21.15.39:0/2576605713 wait complete. 2023-07-21T18:42:08.344 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.311+0000 7f1ee4d14b80 20 client.15696 trim_cache size 0 max 16384 2023-07-21T18:42:08.351 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.329+0000 7f1ee4d14b80 1 Processor -- start 2023-07-21T18:42:08.351 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.329+0000 7f1ee4d14b80 1 -- start start 2023-07-21T18:42:08.351 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.329+0000 7f1ee4d14b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a685c60350 0x55a685c6b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:08.351 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.329+0000 7f1ee4d14b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a685c60c60 0x55a685c63b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:08.351 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.329+0000 7f1ee4d14b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a685c6bb50 0x55a685c64050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:08.352 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.329+0000 7f1ee4d14b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a685c5dda0 con 0x55a685c60350 2023-07-21T18:42:08.352 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.329+0000 7f1ee4d14b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a685c69280 con 0x55a685c60c60 2023-07-21T18:42:08.352 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.329+0000 7f1ee4d14b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a685c615c0 con 0x55a685c6bb50 2023-07-21T18:42:08.352 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.329+0000 7f1ed259c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a685c60350 0x55a685c6b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:08.352 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.329+0000 7f1ed259c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a685c60350 0x55a685c6b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57812/0 (socket says 172.21.15.39:57812) 2023-07-21T18:42:08.352 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.329+0000 7f1ed259c700 1 -- 172.21.15.39:0/2483954023 learned_addr learned my addr 172.21.15.39:0/2483954023 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:08.353 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.329+0000 7f1ed2d9d700 1 --2- 172.21.15.39:0/2483954023 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a685c6bb50 0x55a685c64050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:08.353 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.329+0000 7f1ec9d9b700 1 --2- 172.21.15.39:0/2483954023 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a685c60c60 0x55a685c63b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:08.353 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.329+0000 7f1ed259c700 1 -- 172.21.15.39:0/2483954023 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a685c6bb50 msgr2=0x55a685c64050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:08.353 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.329+0000 7f1ed259c700 1 --2- 172.21.15.39:0/2483954023 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a685c6bb50 0x55a685c64050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:08.353 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.329+0000 7f1ed259c700 1 -- 172.21.15.39:0/2483954023 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a685c60c60 msgr2=0x55a685c63b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:08.353 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.329+0000 7f1ed259c700 1 --2- 172.21.15.39:0/2483954023 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a685c60c60 0x55a685c63b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:08.353 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.329+0000 7f1ed259c700 1 -- 172.21.15.39:0/2483954023 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6859c9ce0 con 0x55a685c60350 2023-07-21T18:42:08.354 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.330+0000 7f1ed259c700 1 --2- 172.21.15.39:0/2483954023 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a685c60350 0x55a685c6b100 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f1ec40caa30 tx=0x7f1ec4005910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a0520870521ce8c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:08.357 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.330+0000 7f1ec959a700 1 -- 172.21.15.39:0/2483954023 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec409e200 con 0x55a685c60350 2023-07-21T18:42:08.357 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.330+0000 7f1ec959a700 1 -- 172.21.15.39:0/2483954023 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ec40a2620 con 0x55a685c60350 2023-07-21T18:42:08.357 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.330+0000 7f1ec959a700 1 -- 172.21.15.39:0/2483954023 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec40996c0 con 0x55a685c60350 2023-07-21T18:42:08.357 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.330+0000 7f1ee4d14b80 1 -- 172.21.15.39:0/2483954023 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a685c60350 msgr2=0x55a685c6b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:08.357 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.330+0000 7f1ee4d14b80 1 --2- 172.21.15.39:0/2483954023 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a685c60350 0x55a685c6b100 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f1ec40caa30 tx=0x7f1ec4005910 comp rx=0 tx=0).stop 2023-07-21T18:42:08.358 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.331+0000 7f1ee4d14b80 1 -- 172.21.15.39:0/2483954023 shutdown_connections 2023-07-21T18:42:08.358 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.331+0000 7f1ee4d14b80 1 --2- 172.21.15.39:0/2483954023 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a685c6bb50 0x55a685c64050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:08.358 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.331+0000 7f1ee4d14b80 1 --2- 172.21.15.39:0/2483954023 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a685c60350 0x55a685c6b100 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:08.358 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.331+0000 7f1ee4d14b80 1 --2- 172.21.15.39:0/2483954023 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a685c60c60 0x55a685c63b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:08.358 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.331+0000 7f1ee4d14b80 1 -- 172.21.15.39:0/2483954023 >> 172.21.15.39:0/2483954023 conn(0x55a685c56e20 msgr2=0x55a685c5ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:08.358 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.331+0000 7f1ee4d14b80 1 -- 172.21.15.39:0/2483954023 shutdown_connections 2023-07-21T18:42:08.359 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.331+0000 7f1ee4d14b80 1 -- 172.21.15.39:0/2483954023 wait complete. 2023-07-21T18:42:08.359 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.331+0000 7f1ee4d14b80 1 -- start start 2023-07-21T18:42:08.359 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.331+0000 7f1ee4d14b80 1 Processor -- start 2023-07-21T18:42:08.359 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.331+0000 7f1ee4d14b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a685c60350 0x55a685c70d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:08.359 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.331+0000 7f1ee4d14b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a685c60c60 0x55a685c71240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:08.359 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.331+0000 7f1ee4d14b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a685c6bb50 0x55a685c71780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:08.359 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.331+0000 7f1ee4d14b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a685ccf380 con 0x55a685c6bb50 2023-07-21T18:42:08.360 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.331+0000 7f1ee4d14b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a685ccf4f0 con 0x55a685c60350 2023-07-21T18:42:08.360 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.331+0000 7f1ee4d14b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a685ccf6b0 con 0x55a685c60c60 2023-07-21T18:42:08.360 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.331+0000 7f1ee4d14b80 10 client.0 _ll_register_callbacks cb 0x7f1ec400df90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:42:08.360 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.331+0000 7f1ed2d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a685c6bb50 0x55a685c71780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:08.360 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.331+0000 7f1ed2d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a685c6bb50 0x55a685c71780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57822/0 (socket says 172.21.15.39:57822) 2023-07-21T18:42:08.360 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.331+0000 7f1ed2d9d700 1 -- 172.21.15.39:0/2862449774 learned_addr learned my addr 172.21.15.39:0/2862449774 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:08.360 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.331+0000 7f1ed259c700 1 --2- 172.21.15.39:0/2862449774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a685c60350 0x55a685c70d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:08.361 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.332+0000 7f1ec9d9b700 1 --2- 172.21.15.39:0/2862449774 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a685c60c60 0x55a685c71240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:08.361 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.332+0000 7f1ed2d9d700 1 -- 172.21.15.39:0/2862449774 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a685c60c60 msgr2=0x55a685c71240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:08.361 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.332+0000 7f1ed2d9d700 1 --2- 172.21.15.39:0/2862449774 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a685c60c60 0x55a685c71240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:08.361 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.332+0000 7f1ed2d9d700 1 -- 172.21.15.39:0/2862449774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a685c60350 msgr2=0x55a685c70d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:08.361 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.332+0000 7f1ed2d9d700 1 --2- 172.21.15.39:0/2862449774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a685c60350 0x55a685c70d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:08.361 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.332+0000 7f1ed2d9d700 1 -- 172.21.15.39:0/2862449774 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ed41a4c70 con 0x55a685c6bb50 2023-07-21T18:42:08.361 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.332+0000 7f1ec9d9b700 1 --2- 172.21.15.39:0/2862449774 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a685c60c60 0x55a685c71240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:42:08.362 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.332+0000 7f1ed2d9d700 1 --2- 172.21.15.39:0/2862449774 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a685c6bb50 0x55a685c71780 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc018040 tx=0x7f1ecc015ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c828d5c241336de1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:08.362 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.332+0000 7f1eb9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:42:08.362 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.332+0000 7f1eb9ffb700 1 -- 172.21.15.39:0/2862449774 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ecc01e650 con 0x55a685c6bb50 2023-07-21T18:42:08.362 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.332+0000 7f1eb9ffb700 1 -- 172.21.15.39:0/2862449774 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f1ecc004a90 con 0x55a685c6bb50 2023-07-21T18:42:08.362 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.332+0000 7f1ee4d14b80 1 -- 172.21.15.39:0/2862449774 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1ecc00f040 con 0x55a685c6bb50 2023-07-21T18:42:08.362 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.332+0000 7f1eb9ffb700 1 -- 172.21.15.39:0/2862449774 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ecc01ecc0 con 0x55a685c6bb50 2023-07-21T18:42:08.362 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.332+0000 7f1ee4d14b80 10 client.15702 fetch_fsmap learned FSMap version 26 2023-07-21T18:42:08.363 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.332+0000 7f1ee4d14b80 1 -- 172.21.15.39:0/2862449774 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55a685a072d0 con 0x55a685c6bb50 2023-07-21T18:42:08.363 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.333+0000 7f1eb9ffb700 1 -- 172.21.15.39:0/2862449774 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ecc00f040 con 0x55a685c6bb50 2023-07-21T18:42:08.363 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.333+0000 7f1eb9ffb700 1 -- 172.21.15.39:0/2862449774 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1ecc031cc0 con 0x55a685c6bb50 2023-07-21T18:42:08.363 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.333+0000 7f1ee4d14b80 10 client.15702 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T18:42:08.363 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.333+0000 7f1ee4d14b80 10 client.15702 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T18:42:08.363 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.333+0000 7f1ee4d14b80 20 client.15702 populate_metadata read hostname 'smithi039' 2023-07-21T18:42:08.363 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.333+0000 7f1ee4d14b80 1 --2- 172.21.15.39:0/2862449774 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55a685c7b410 0x55a685cd2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:08.364 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.333+0000 7f1ee4d14b80 4 client.15702 mds_command: new command op to 24536 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-21T18:42:08.364 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.333+0000 7f1ee4d14b80 1 -- 172.21.15.39:0/2862449774 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55a685b6e0d0 con 0x55a685c7b410 2023-07-21T18:42:08.364 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.333+0000 7f1ed259c700 1 --2- 172.21.15.39:0/2862449774 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55a685c7b410 0x55a685cd2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:08.364 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.333+0000 7f1ed259c700 1 --2- 172.21.15.39:0/2862449774 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55a685c7b410 0x55a685cd2b20 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1212ae7cc8b203a server_cookie=190d2d927c217f32 in_seq=0 out_seq=0 2023-07-21T18:42:08.364 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.333+0000 7f1eb9ffb700 10 client.15702 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:42:08.364 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.335+0000 7f1eb9ffb700 1 -- 172.21.15.39:0/2862449774 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55a685b6e0d0 con 0x55a685c7b410 2023-07-21T18:42:08.364 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.335+0000 7f1eb9ffb700 10 client.15702 handle_command_reply: tid=0 2023-07-21T18:42:08.365 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.335+0000 7f1ee4d14b80 1 client.15702 shutdown 2023-07-21T18:42:08.365 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.335+0000 7f1ee4d14b80 1 -- 172.21.15.39:0/2862449774 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a685c6bb50 msgr2=0x55a685c71780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:08.365 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.335+0000 7f1ee4d14b80 1 --2- 172.21.15.39:0/2862449774 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a685c6bb50 0x55a685c71780 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc018040 tx=0x7f1ecc015ec0 comp rx=0 tx=0).stop 2023-07-21T18:42:08.365 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.335+0000 7f1ee4d14b80 1 -- 172.21.15.39:0/2862449774 shutdown_connections 2023-07-21T18:42:08.365 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.335+0000 7f1ee4d14b80 1 --2- 172.21.15.39:0/2862449774 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55a685c7b410 0x55a685cd2b20 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:08.365 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.335+0000 7f1ee4d14b80 1 --2- 172.21.15.39:0/2862449774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a685c60350 0x55a685c70d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:08.365 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.335+0000 7f1ee4d14b80 1 --2- 172.21.15.39:0/2862449774 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a685c6bb50 0x55a685c71780 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:08.365 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.335+0000 7f1ee4d14b80 1 --2- 172.21.15.39:0/2862449774 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a685c60c60 0x55a685c71240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:08.366 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.335+0000 7f1ee4d14b80 1 -- 172.21.15.39:0/2862449774 >> 172.21.15.39:0/2862449774 conn(0x55a685c56e20 msgr2=0x55a685c63b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:08.366 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.335+0000 7f1eb9ffb700 0 client.15702 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:42:08.366 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.335+0000 7f1ee4d14b80 1 -- 172.21.15.39:0/2862449774 shutdown_connections 2023-07-21T18:42:08.366 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.335+0000 7f1ee4d14b80 1 -- 172.21.15.39:0/2862449774 wait complete. 2023-07-21T18:42:08.366 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.335+0000 7f1ee4d14b80 20 client.15702 trim_cache size 0 max 16384 2023-07-21T18:42:08.367 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2023-07-21T18:42:08.367 INFO:teuthology.orchestra.run.smithi039.stdout: "return_code": 0, 2023-07-21T18:42:08.367 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_tag": "fda23896-ce3d-43c0-b8e4-d5ef6a9da871", 2023-07-21T18:42:08.367 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": "asynchronous" 2023-07-21T18:42:08.367 INFO:teuthology.orchestra.run.smithi039.stdout:} 2023-07-21T18:42:08.368 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.336+0000 7f1ed8b8e700 1 -- 172.21.15.39:0/1128497802 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f1ec00a13e0 msgr2=0x7f1ec00a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:08.368 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.336+0000 7f1ed8b8e700 1 --2- 172.21.15.39:0/1128497802 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f1ec00a13e0 0x7f1ec00a3890 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1ed4199450 tx=0x7f1ebc005fb0 comp rx=0 tx=0).stop 2023-07-21T18:42:08.368 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.336+0000 7f1ed8b8e700 1 -- 172.21.15.39:0/1128497802 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ed40fe820 msgr2=0x7f1ed4197e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:08.368 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.336+0000 7f1ed8b8e700 1 --2- 172.21.15.39:0/1128497802 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ed40fe820 0x7f1ed4197e40 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f1ec400d950 tx=0x7f1ec400dd10 comp rx=0 tx=0).stop 2023-07-21T18:42:08.369 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.336+0000 7f1ed8b8e700 1 -- 172.21.15.39:0/1128497802 shutdown_connections 2023-07-21T18:42:08.369 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.336+0000 7f1ed8b8e700 1 --2- 172.21.15.39:0/1128497802 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f1ec00a13e0 0x7f1ec00a3890 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:08.369 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.336+0000 7f1ed8b8e700 1 --2- 172.21.15.39:0/1128497802 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ed410c3d0 0x7f1ed419c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:08.369 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.336+0000 7f1ed8b8e700 1 --2- 172.21.15.39:0/1128497802 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ed40fe820 0x7f1ed4197e40 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:08.369 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.336+0000 7f1ed8b8e700 1 --2- 172.21.15.39:0/1128497802 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ed40ff1c0 0x7f1ed4198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:08.369 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.336+0000 7f1ed8b8e700 1 -- 172.21.15.39:0/1128497802 >> 172.21.15.39:0/1128497802 conn(0x7f1ed40fa4b0 msgr2=0x7f1ed410ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:08.369 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.336+0000 7f1ed8b8e700 1 -- 172.21.15.39:0/1128497802 shutdown_connections 2023-07-21T18:42:08.370 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.336+0000 7f1ed8b8e700 1 -- 172.21.15.39:0/1128497802 wait complete. 2023-07-21T18:42:08.370 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:42:08 smithi193 ceph-mon[98204]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 29 op/s 2023-07-21T18:42:08.370 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:42:08 smithi039 ceph-mon[96489]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 29 op/s 2023-07-21T18:42:08.371 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T18:42:08.388 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:42:08 smithi155 ceph-mon[97934]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 29 op/s 2023-07-21T18:42:08.502 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.481+0000 7f64a624c700 1 -- 172.21.15.39:0/1399684168 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64a0100d00 msgr2=0x7f64a01010d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:08.503 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.481+0000 7f64a624c700 1 --2- 172.21.15.39:0/1399684168 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64a0100d00 0x7f64a01010d0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f6490009990 tx=0x7f6490009ca0 comp rx=0 tx=0).stop 2023-07-21T18:42:08.503 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.481+0000 7f64a624c700 1 -- 172.21.15.39:0/1399684168 shutdown_connections 2023-07-21T18:42:08.503 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.481+0000 7f64a624c700 1 --2- 172.21.15.39:0/1399684168 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64a010a040 0x7f64a010c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:08.503 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.481+0000 7f64a624c700 1 --2- 172.21.15.39:0/1399684168 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64a0100d00 0x7f64a01010d0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:08.503 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.481+0000 7f64a624c700 1 --2- 172.21.15.39:0/1399684168 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f64a0101610 0x7f64a0109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:08.503 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.481+0000 7f64a624c700 1 -- 172.21.15.39:0/1399684168 >> 172.21.15.39:0/1399684168 conn(0x7f64a00fa4b0 msgr2=0x7f64a00fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:08.504 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.481+0000 7f64a624c700 1 -- 172.21.15.39:0/1399684168 shutdown_connections 2023-07-21T18:42:08.504 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.481+0000 7f64a624c700 1 -- 172.21.15.39:0/1399684168 wait complete. 2023-07-21T18:42:08.505 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.481+0000 7f64a624c700 1 Processor -- start 2023-07-21T18:42:08.506 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.481+0000 7f64a624c700 1 -- start start 2023-07-21T18:42:08.506 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.482+0000 7f64a624c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f64a0100d00 0x7f64a019b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:08.506 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.482+0000 7f64a624c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64a0101610 0x7f64a019bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:08.506 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.482+0000 7f64a624c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64a010a040 0x7f64a0195a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:08.506 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.482+0000 7f64a624c700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a0196090 con 0x7f64a0101610 2023-07-21T18:42:08.506 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.482+0000 7f64a624c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a0196200 con 0x7f64a010a040 2023-07-21T18:42:08.507 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.482+0000 7f64a624c700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a01963c0 con 0x7f64a0100d00 2023-07-21T18:42:08.507 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.482+0000 7f649effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64a0101610 0x7f64a019bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:08.507 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.482+0000 7f649effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64a0101610 0x7f64a019bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57834/0 (socket says 172.21.15.39:57834) 2023-07-21T18:42:08.507 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.482+0000 7f649effd700 1 -- 172.21.15.39:0/379833583 learned_addr learned my addr 172.21.15.39:0/379833583 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:08.507 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.482+0000 7f649ffff700 1 --2- 172.21.15.39:0/379833583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64a010a040 0x7f64a0195a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:08.507 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.482+0000 7f649f7fe700 1 --2- 172.21.15.39:0/379833583 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f64a0100d00 0x7f64a019b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:08.507 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.482+0000 7f649effd700 1 -- 172.21.15.39:0/379833583 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f64a0100d00 msgr2=0x7f64a019b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:08.508 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.482+0000 7f649effd700 1 --2- 172.21.15.39:0/379833583 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f64a0100d00 0x7f64a019b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:08.508 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.482+0000 7f649effd700 1 -- 172.21.15.39:0/379833583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64a010a040 msgr2=0x7f64a0195a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:08.508 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.482+0000 7f649effd700 1 --2- 172.21.15.39:0/379833583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64a010a040 0x7f64a0195a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:08.508 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.482+0000 7f649effd700 1 -- 172.21.15.39:0/379833583 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64900095f0 con 0x7f64a0101610 2023-07-21T18:42:08.508 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.482+0000 7f649ffff700 1 --2- 172.21.15.39:0/379833583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64a010a040 0x7f64a0195a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:42:08.508 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.482+0000 7f649f7fe700 1 --2- 172.21.15.39:0/379833583 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f64a0100d00 0x7f64a019b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:42:08.509 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.482+0000 7f649effd700 1 --2- 172.21.15.39:0/379833583 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64a0101610 0x7f64a019bef0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f649400d950 tx=0x7f649400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:08.509 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.482+0000 7f649cff9700 1 -- 172.21.15.39:0/379833583 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6494009b40 con 0x7f64a0101610 2023-07-21T18:42:08.509 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.482+0000 7f64a624c700 1 -- 172.21.15.39:0/379833583 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64a01966a0 con 0x7f64a0101610 2023-07-21T18:42:08.509 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.482+0000 7f649cff9700 1 -- 172.21.15.39:0/379833583 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6494009ca0 con 0x7f64a0101610 2023-07-21T18:42:08.509 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.482+0000 7f649cff9700 1 -- 172.21.15.39:0/379833583 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f649400f620 con 0x7f64a0101610 2023-07-21T18:42:08.509 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.483+0000 7f64a624c700 1 -- 172.21.15.39:0/379833583 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64a01a29a0 con 0x7f64a0101610 2023-07-21T18:42:08.514 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.485+0000 7f649cff9700 1 -- 172.21.15.39:0/379833583 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f6494010460 con 0x7f64a0101610 2023-07-21T18:42:08.514 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.485+0000 7f649cff9700 1 --2- 172.21.15.39:0/379833583 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f64880a13e0 0x7f64880a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:08.514 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.486+0000 7f649cff9700 1 -- 172.21.15.39:0/379833583 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f64940cd330 con 0x7f64a0101610 2023-07-21T18:42:08.514 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.486+0000 7f649f7fe700 1 --2- 172.21.15.39:0/379833583 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f64880a13e0 0x7f64880a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:08.514 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.486+0000 7f649f7fe700 1 --2- 172.21.15.39:0/379833583 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f64880a13e0 0x7f64880a3890 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f649000b5c0 tx=0x7f6490005fb0 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:08.515 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.486+0000 7f64b23d2b80 1 Processor -- start 2023-07-21T18:42:08.515 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.486+0000 7f64b23d2b80 1 -- start start 2023-07-21T18:42:08.515 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.487+0000 7f64b23d2b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571a58352d0 0x5571a58356a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:08.515 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.487+0000 7f64b23d2b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5571a5835be0 0x5571a5840590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:08.515 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.487+0000 7f64b23d2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571a5840ad0 0x5571a5842eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:08.515 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.487+0000 7f64b23d2b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a573bcb0 con 0x5571a5835be0 2023-07-21T18:42:08.515 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.487+0000 7f64b23d2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a578ea40 con 0x5571a5840ad0 2023-07-21T18:42:08.516 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.487+0000 7f64b23d2b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a5787ed0 con 0x5571a58352d0 2023-07-21T18:42:08.516 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.487+0000 7f649effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5571a5835be0 0x5571a5840590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:08.516 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.487+0000 7f649effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5571a5835be0 0x5571a5840590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57842/0 (socket says 172.21.15.39:57842) 2023-07-21T18:42:08.516 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.487+0000 7f649effd700 1 -- 172.21.15.39:0/2842183293 learned_addr learned my addr 172.21.15.39:0/2842183293 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:08.516 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.487+0000 7f649f7fe700 1 --2- 172.21.15.39:0/2842183293 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571a58352d0 0x5571a58356a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:08.516 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.487+0000 7f649ffff700 1 --2- 172.21.15.39:0/2842183293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571a5840ad0 0x5571a5842eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:08.516 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.487+0000 7f649effd700 1 -- 172.21.15.39:0/2842183293 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571a58352d0 msgr2=0x5571a58356a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:08.517 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.487+0000 7f649effd700 1 --2- 172.21.15.39:0/2842183293 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571a58352d0 0x5571a58356a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:08.517 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.487+0000 7f649effd700 1 -- 172.21.15.39:0/2842183293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571a5840ad0 msgr2=0x5571a5842eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:08.517 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.487+0000 7f649effd700 1 --2- 172.21.15.39:0/2842183293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571a5840ad0 0x5571a5842eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:08.517 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.487+0000 7f649effd700 1 -- 172.21.15.39:0/2842183293 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5571a55b5de0 con 0x5571a5835be0 2023-07-21T18:42:08.517 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.487+0000 7f649f7fe700 1 --2- 172.21.15.39:0/2842183293 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571a58352d0 0x5571a58356a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:42:08.517 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.487+0000 7f649effd700 1 --2- 172.21.15.39:0/2842183293 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5571a5835be0 0x5571a5840590 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f649409a160 tx=0x7f6494099e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc5c026ac7d8e68 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:08.517 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.487+0000 7f6485ffb700 1 -- 172.21.15.39:0/2842183293 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64940a2620 con 0x5571a5835be0 2023-07-21T18:42:08.518 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.487+0000 7f6485ffb700 1 -- 172.21.15.39:0/2842183293 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6494010710 con 0x5571a5835be0 2023-07-21T18:42:08.518 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.487+0000 7f6485ffb700 1 -- 172.21.15.39:0/2842183293 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6494099920 con 0x5571a5835be0 2023-07-21T18:42:08.518 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.488+0000 7f64b23d2b80 1 -- 172.21.15.39:0/2842183293 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5571a5835be0 msgr2=0x5571a5840590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:08.518 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.488+0000 7f64b23d2b80 1 --2- 172.21.15.39:0/2842183293 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5571a5835be0 0x5571a5840590 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f649409a160 tx=0x7f6494099e40 comp rx=0 tx=0).stop 2023-07-21T18:42:08.518 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.488+0000 7f64b23d2b80 1 -- 172.21.15.39:0/2842183293 shutdown_connections 2023-07-21T18:42:08.518 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.488+0000 7f64b23d2b80 1 --2- 172.21.15.39:0/2842183293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571a5840ad0 0x5571a5842eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:08.518 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.488+0000 7f64b23d2b80 1 --2- 172.21.15.39:0/2842183293 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571a58352d0 0x5571a58356a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:08.519 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.488+0000 7f64b23d2b80 1 --2- 172.21.15.39:0/2842183293 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5571a5835be0 0x5571a5840590 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:08.519 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.488+0000 7f64b23d2b80 1 -- 172.21.15.39:0/2842183293 >> 172.21.15.39:0/2842183293 conn(0x5571a582bda0 msgr2=0x5571a58316a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:08.519 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.488+0000 7f64b23d2b80 1 -- 172.21.15.39:0/2842183293 shutdown_connections 2023-07-21T18:42:08.519 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.488+0000 7f64b23d2b80 1 -- 172.21.15.39:0/2842183293 wait complete. 2023-07-21T18:42:08.519 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.489+0000 7f64b23d2b80 1 -- start start 2023-07-21T18:42:08.519 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.489+0000 7f64b23d2b80 1 Processor -- start 2023-07-21T18:42:08.519 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.489+0000 7f64b23d2b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5571a58352d0 0x5571a583d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:08.520 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.489+0000 7f64b23d2b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571a5835be0 0x5571a583dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:08.520 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.489+0000 7f64b23d2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571a5840ad0 0x5571a583e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:08.520 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.489+0000 7f64b23d2b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a5727ac0 con 0x5571a58352d0 2023-07-21T18:42:08.520 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.489+0000 7f64b23d2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a57277c0 con 0x5571a5840ad0 2023-07-21T18:42:08.520 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.489+0000 7f64b23d2b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a560b920 con 0x5571a5835be0 2023-07-21T18:42:08.520 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.489+0000 7f64b23d2b80 10 client.0 _ll_register_callbacks cb 0x5571a5725760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:42:08.520 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.489+0000 7f649f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5571a58352d0 0x5571a583d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:08.521 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.489+0000 7f649f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5571a58352d0 0x5571a583d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57846/0 (socket says 172.21.15.39:57846) 2023-07-21T18:42:08.521 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.489+0000 7f649effd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571a5835be0 0x5571a583dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:08.521 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.489+0000 7f649f7fe700 1 -- 172.21.15.39:0/1240793640 learned_addr learned my addr 172.21.15.39:0/1240793640 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:08.521 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.489+0000 7f649ffff700 1 --2- 172.21.15.39:0/1240793640 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571a5840ad0 0x5571a583e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:08.521 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.490+0000 7f649effd700 1 -- 172.21.15.39:0/1240793640 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571a5840ad0 msgr2=0x5571a583e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:08.521 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.490+0000 7f649effd700 1 --2- 172.21.15.39:0/1240793640 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571a5840ad0 0x5571a583e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:08.522 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.490+0000 7f649effd700 1 -- 172.21.15.39:0/1240793640 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5571a58352d0 msgr2=0x5571a583d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:08.522 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.490+0000 7f649effd700 1 --2- 172.21.15.39:0/1240793640 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5571a58352d0 0x5571a583d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:08.522 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.490+0000 7f649effd700 1 -- 172.21.15.39:0/1240793640 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64a01a29a0 con 0x5571a5835be0 2023-07-21T18:42:08.522 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.490+0000 7f649f7fe700 1 --2- 172.21.15.39:0/1240793640 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5571a58352d0 0x5571a583d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:42:08.522 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.490+0000 7f649effd700 1 --2- 172.21.15.39:0/1240793640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571a5835be0 0x5571a583dd80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x5571a5837350 tx=0x7f6494010a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=300d534eb57bf4c7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:08.522 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.490+0000 7f647effd700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2023-07-21T18:42:08.522 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.490+0000 7f647effd700 1 -- 172.21.15.39:0/1240793640 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64940b6070 con 0x5571a5835be0 2023-07-21T18:42:08.523 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.490+0000 7f64b23d2b80 1 -- 172.21.15.39:0/1240793640 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5571a56e14f0 con 0x5571a5835be0 2023-07-21T18:42:08.523 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.490+0000 7f647effd700 1 -- 172.21.15.39:0/1240793640 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f6494098570 con 0x5571a5835be0 2023-07-21T18:42:08.523 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.490+0000 7f647effd700 1 -- 172.21.15.39:0/1240793640 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64940998d0 con 0x5571a5835be0 2023-07-21T18:42:08.523 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.490+0000 7f64b23d2b80 10 client.25300 fetch_fsmap learned FSMap version 26 2023-07-21T18:42:08.523 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.490+0000 7f64b23d2b80 1 -- 172.21.15.39:0/1240793640 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5571a559ec60 con 0x5571a5835be0 2023-07-21T18:42:08.523 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.491+0000 7f647effd700 1 -- 172.21.15.39:0/1240793640 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f64940ca020 con 0x5571a5835be0 2023-07-21T18:42:08.523 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.491+0000 7f647effd700 1 -- 172.21.15.39:0/1240793640 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f64940b2bb0 con 0x5571a5835be0 2023-07-21T18:42:08.524 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.491+0000 7f64b23d2b80 10 client.25300 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T18:42:08.524 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.491+0000 7f64b23d2b80 10 client.25300 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T18:42:08.524 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.491+0000 7f64b23d2b80 20 client.25300 populate_metadata read hostname 'smithi039' 2023-07-21T18:42:08.524 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.491+0000 7f64b23d2b80 1 --2- 172.21.15.39:0/1240793640 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5571a584fea0 0x5571a5852280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:08.524 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.491+0000 7f64b23d2b80 4 client.25300 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:42:08.524 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.491+0000 7f64b23d2b80 1 -- 172.21.15.39:0/1240793640 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5571a5780eb0 con 0x5571a584fea0 2023-07-21T18:42:08.524 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.491+0000 7f649f7fe700 1 --2- 172.21.15.39:0/1240793640 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5571a584fea0 0x5571a5852280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:08.525 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.492+0000 7f647effd700 10 client.25300 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:42:08.525 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.492+0000 7f649f7fe700 1 --2- 172.21.15.39:0/1240793640 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5571a584fea0 0x5571a5852280 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3ebed4a5ca2ced8 server_cookie=9dcd3a6fa23ca1c9 in_seq=0 out_seq=0 2023-07-21T18:42:08.525 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.492+0000 7f647effd700 1 -- 172.21.15.39:0/1240793640 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5571a5780eb0 con 0x5571a584fea0 2023-07-21T18:42:08.525 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.492+0000 7f647effd700 10 client.25300 handle_command_reply: tid=0 2023-07-21T18:42:08.525 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.492+0000 7f64b23d2b80 1 client.25300 shutdown 2023-07-21T18:42:08.526 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.493+0000 7f64b23d2b80 1 -- 172.21.15.39:0/1240793640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571a5835be0 msgr2=0x5571a583dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:08.526 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.493+0000 7f64b23d2b80 1 --2- 172.21.15.39:0/1240793640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571a5835be0 0x5571a583dd80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x5571a5837350 tx=0x7f6494010a90 comp rx=0 tx=0).stop 2023-07-21T18:42:08.526 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.493+0000 7f64b23d2b80 1 -- 172.21.15.39:0/1240793640 shutdown_connections 2023-07-21T18:42:08.526 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.493+0000 7f64b23d2b80 1 --2- 172.21.15.39:0/1240793640 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5571a584fea0 0x5571a5852280 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:08.527 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.493+0000 7f64b23d2b80 1 --2- 172.21.15.39:0/1240793640 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571a5840ad0 0x5571a583e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:08.527 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.493+0000 7f64b23d2b80 1 --2- 172.21.15.39:0/1240793640 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5571a58352d0 0x5571a583d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:08.527 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.493+0000 7f64b23d2b80 1 --2- 172.21.15.39:0/1240793640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571a5835be0 0x5571a583dd80 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:08.527 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.493+0000 7f647effd700 0 client.25300 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:42:08.527 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.493+0000 7f64b23d2b80 1 -- 172.21.15.39:0/1240793640 >> 172.21.15.39:0/1240793640 conn(0x5571a582bda0 msgr2=0x5571a5831230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:08.527 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.493+0000 7f64b23d2b80 1 -- 172.21.15.39:0/1240793640 shutdown_connections 2023-07-21T18:42:08.527 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.493+0000 7f64b23d2b80 1 -- 172.21.15.39:0/1240793640 wait complete. 2023-07-21T18:42:08.528 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.493+0000 7f64b23d2b80 20 client.25300 trim_cache size 0 max 16384 2023-07-21T18:42:08.532 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.510+0000 7f64b23d2b80 1 Processor -- start 2023-07-21T18:42:08.532 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.510+0000 7f64b23d2b80 1 -- start start 2023-07-21T18:42:08.533 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.510+0000 7f64b23d2b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571a58352d0 0x5571a58474b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:08.533 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.510+0000 7f64b23d2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571a5835be0 0x5571a583ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:08.533 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.510+0000 7f64b23d2b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5571a5840ad0 0x5571a583e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:08.533 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.510+0000 7f64b23d2b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a5832ad0 con 0x5571a5840ad0 2023-07-21T18:42:08.533 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.510+0000 7f64b23d2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a5836200 con 0x5571a5835be0 2023-07-21T18:42:08.533 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.510+0000 7f64b23d2b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a5828ef0 con 0x5571a58352d0 2023-07-21T18:42:08.533 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.510+0000 7f649ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5571a5840ad0 0x5571a583e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:08.534 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.510+0000 7f649ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5571a5840ad0 0x5571a583e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57858/0 (socket says 172.21.15.39:57858) 2023-07-21T18:42:08.534 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.510+0000 7f649ffff700 1 -- 172.21.15.39:0/3379483959 learned_addr learned my addr 172.21.15.39:0/3379483959 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:08.534 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.510+0000 7f649f7fe700 1 --2- 172.21.15.39:0/3379483959 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571a58352d0 0x5571a58474b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:08.534 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.510+0000 7f649effd700 1 --2- 172.21.15.39:0/3379483959 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571a5835be0 0x5571a583ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:08.534 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.510+0000 7f649ffff700 1 -- 172.21.15.39:0/3379483959 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571a58352d0 msgr2=0x5571a58474b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:08.534 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.510+0000 7f649ffff700 1 --2- 172.21.15.39:0/3379483959 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571a58352d0 0x5571a58474b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:08.534 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.510+0000 7f649ffff700 1 -- 172.21.15.39:0/3379483959 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571a5835be0 msgr2=0x5571a583ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:08.535 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.510+0000 7f649ffff700 1 --2- 172.21.15.39:0/3379483959 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571a5835be0 0x5571a583ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:08.535 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.510+0000 7f649ffff700 1 -- 172.21.15.39:0/3379483959 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5571a55dc250 con 0x5571a5840ad0 2023-07-21T18:42:08.535 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.510+0000 7f649f7fe700 1 --2- 172.21.15.39:0/3379483959 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571a58352d0 0x5571a58474b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:42:08.535 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.511+0000 7f649ffff700 1 --2- 172.21.15.39:0/3379483959 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5571a5840ad0 0x5571a583e2f0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f6498009b50 tx=0x7f6498009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2089654db68e9593 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:08.535 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.511+0000 7f64867fc700 1 -- 172.21.15.39:0/3379483959 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6498007cc0 con 0x5571a5840ad0 2023-07-21T18:42:08.535 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.511+0000 7f64867fc700 1 -- 172.21.15.39:0/3379483959 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6498007e20 con 0x5571a5840ad0 2023-07-21T18:42:08.535 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.511+0000 7f64867fc700 1 -- 172.21.15.39:0/3379483959 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6498004820 con 0x5571a5840ad0 2023-07-21T18:42:08.538 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.511+0000 7f64b23d2b80 1 -- 172.21.15.39:0/3379483959 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5571a5840ad0 msgr2=0x5571a583e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:08.539 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.511+0000 7f64b23d2b80 1 --2- 172.21.15.39:0/3379483959 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5571a5840ad0 0x5571a583e2f0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f6498009b50 tx=0x7f6498009040 comp rx=0 tx=0).stop 2023-07-21T18:42:08.539 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.512+0000 7f64b23d2b80 1 -- 172.21.15.39:0/3379483959 shutdown_connections 2023-07-21T18:42:08.539 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.512+0000 7f64b23d2b80 1 --2- 172.21.15.39:0/3379483959 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5571a5840ad0 0x5571a583e2f0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:08.539 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.512+0000 7f64b23d2b80 1 --2- 172.21.15.39:0/3379483959 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571a58352d0 0x5571a58474b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:08.539 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.512+0000 7f64b23d2b80 1 --2- 172.21.15.39:0/3379483959 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571a5835be0 0x5571a583ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:08.540 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.512+0000 7f64b23d2b80 1 -- 172.21.15.39:0/3379483959 >> 172.21.15.39:0/3379483959 conn(0x5571a582bda0 msgr2=0x5571a584c730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:08.540 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.512+0000 7f64b23d2b80 1 -- 172.21.15.39:0/3379483959 shutdown_connections 2023-07-21T18:42:08.540 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.512+0000 7f64b23d2b80 1 -- 172.21.15.39:0/3379483959 wait complete. 2023-07-21T18:42:08.540 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.512+0000 7f64b23d2b80 1 -- start start 2023-07-21T18:42:08.540 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.512+0000 7f64b23d2b80 1 Processor -- start 2023-07-21T18:42:08.540 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.512+0000 7f64b23d2b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571a58352d0 0x5571a5897e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:08.540 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.512+0000 7f64b23d2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571a5835be0 0x5571a5898410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:08.541 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.512+0000 7f64b23d2b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5571a5840ad0 0x5571a589bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:08.541 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.512+0000 7f64b23d2b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a589c570 con 0x5571a5840ad0 2023-07-21T18:42:08.541 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.512+0000 7f64b23d2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a589c6e0 con 0x5571a5835be0 2023-07-21T18:42:08.541 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.512+0000 7f64b23d2b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a589c8a0 con 0x5571a58352d0 2023-07-21T18:42:08.541 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.512+0000 7f64b23d2b80 10 client.0 _ll_register_callbacks cb 0x7f6494099d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:42:08.541 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.513+0000 7f649ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5571a5840ad0 0x5571a589bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:08.541 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.513+0000 7f649ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5571a5840ad0 0x5571a589bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57864/0 (socket says 172.21.15.39:57864) 2023-07-21T18:42:08.542 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.513+0000 7f649ffff700 1 -- 172.21.15.39:0/3491545557 learned_addr learned my addr 172.21.15.39:0/3491545557 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:08.542 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.513+0000 7f649effd700 1 --2- 172.21.15.39:0/3491545557 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571a5835be0 0x5571a5898410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:08.542 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.513+0000 7f649f7fe700 1 --2- 172.21.15.39:0/3491545557 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571a58352d0 0x5571a5897e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:08.542 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.513+0000 7f649ffff700 1 -- 172.21.15.39:0/3491545557 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571a58352d0 msgr2=0x5571a5897e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:08.542 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.513+0000 7f649ffff700 1 --2- 172.21.15.39:0/3491545557 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571a58352d0 0x5571a5897e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:08.542 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.513+0000 7f649ffff700 1 -- 172.21.15.39:0/3491545557 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571a5835be0 msgr2=0x5571a5898410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:08.542 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.513+0000 7f649ffff700 1 --2- 172.21.15.39:0/3491545557 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571a5835be0 0x5571a5898410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:08.543 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.513+0000 7f649ffff700 1 -- 172.21.15.39:0/3491545557 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6498011040 con 0x5571a5840ad0 2023-07-21T18:42:08.543 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.513+0000 7f649effd700 1 --2- 172.21.15.39:0/3491545557 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571a5835be0 0x5571a5898410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:42:08.543 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.513+0000 7f649ffff700 1 --2- 172.21.15.39:0/3491545557 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5571a5840ad0 0x5571a589bfc0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f64980077e0 tx=0x7f6498004920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2892ca89d8139529 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:08.543 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.513+0000 7f647effd700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:42:08.543 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.513+0000 7f647effd700 1 -- 172.21.15.39:0/3491545557 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6498003c10 con 0x5571a5840ad0 2023-07-21T18:42:08.543 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.513+0000 7f647effd700 1 -- 172.21.15.39:0/3491545557 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f6498003d70 con 0x5571a5840ad0 2023-07-21T18:42:08.543 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.513+0000 7f64b23d2b80 1 -- 172.21.15.39:0/3491545557 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6494010710 con 0x5571a5840ad0 2023-07-21T18:42:08.544 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.513+0000 7f647effd700 1 -- 172.21.15.39:0/3491545557 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6498020470 con 0x5571a5840ad0 2023-07-21T18:42:08.544 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.513+0000 7f64b23d2b80 10 client.15720 fetch_fsmap learned FSMap version 26 2023-07-21T18:42:08.544 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.513+0000 7f64b23d2b80 1 -- 172.21.15.39:0/3491545557 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5571a55b42b0 con 0x5571a5840ad0 2023-07-21T18:42:08.544 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.514+0000 7f647effd700 1 -- 172.21.15.39:0/3491545557 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6494010710 con 0x5571a5840ad0 2023-07-21T18:42:08.544 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.514+0000 7f647effd700 1 -- 172.21.15.39:0/3491545557 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6498017780 con 0x5571a5840ad0 2023-07-21T18:42:08.544 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.514+0000 7f64b23d2b80 10 client.15720 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T18:42:08.545 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.514+0000 7f64b23d2b80 10 client.15720 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T18:42:08.545 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.514+0000 7f64b23d2b80 20 client.15720 populate_metadata read hostname 'smithi039' 2023-07-21T18:42:08.545 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.514+0000 7f64b23d2b80 1 --2- 172.21.15.39:0/3491545557 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5571a58a5120 0x5571a58a7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:08.545 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.514+0000 7f64b23d2b80 4 client.15720 mds_command: new command op to 24536 tid=0[{"prefix": "scrub status"}] 2023-07-21T18:42:08.545 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.514+0000 7f64b23d2b80 1 -- 172.21.15.39:0/3491545557 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5571a5743050 con 0x5571a58a5120 2023-07-21T18:42:08.545 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.514+0000 7f649f7fe700 1 --2- 172.21.15.39:0/3491545557 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5571a58a5120 0x5571a58a7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:08.545 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.514+0000 7f647effd700 10 client.15720 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:42:08.546 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.514+0000 7f649f7fe700 1 --2- 172.21.15.39:0/3491545557 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5571a58a5120 0x5571a58a7530 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6df051cec1560a34 server_cookie=92e1100ba0476b6 in_seq=0 out_seq=0 2023-07-21T18:42:08.546 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.515+0000 7f647effd700 1 -- 172.21.15.39:0/3491545557 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5571a5743050 con 0x5571a58a5120 2023-07-21T18:42:08.546 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.515+0000 7f647effd700 10 client.15720 handle_command_reply: tid=0 2023-07-21T18:42:08.546 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.515+0000 7f64b23d2b80 1 client.15720 shutdown 2023-07-21T18:42:08.546 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.515+0000 7f64b23d2b80 1 -- 172.21.15.39:0/3491545557 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5571a5840ad0 msgr2=0x5571a589bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:08.546 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.515+0000 7f64b23d2b80 1 --2- 172.21.15.39:0/3491545557 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5571a5840ad0 0x5571a589bfc0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f64980077e0 tx=0x7f6498004920 comp rx=0 tx=0).stop 2023-07-21T18:42:08.546 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.515+0000 7f64b23d2b80 1 -- 172.21.15.39:0/3491545557 shutdown_connections 2023-07-21T18:42:08.547 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.515+0000 7f64b23d2b80 1 --2- 172.21.15.39:0/3491545557 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5571a58a5120 0x5571a58a7530 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:08.547 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.515+0000 7f64b23d2b80 1 --2- 172.21.15.39:0/3491545557 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5571a5840ad0 0x5571a589bfc0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:08.547 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.515+0000 7f64b23d2b80 1 --2- 172.21.15.39:0/3491545557 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571a58352d0 0x5571a5897e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:08.547 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.515+0000 7f647effd700 0 client.15720 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:42:08.547 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.515+0000 7f64b23d2b80 1 --2- 172.21.15.39:0/3491545557 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571a5835be0 0x5571a5898410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:08.547 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.515+0000 7f64b23d2b80 1 -- 172.21.15.39:0/3491545557 >> 172.21.15.39:0/3491545557 conn(0x5571a582bda0 msgr2=0x5571a583ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:08.547 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.515+0000 7f64b23d2b80 1 -- 172.21.15.39:0/3491545557 shutdown_connections 2023-07-21T18:42:08.548 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.515+0000 7f64b23d2b80 1 -- 172.21.15.39:0/3491545557 wait complete. 2023-07-21T18:42:08.548 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.515+0000 7f64b23d2b80 20 client.15720 trim_cache size 0 max 16384 2023-07-21T18:42:08.548 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2023-07-21T18:42:08.548 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T18:42:08.548 INFO:teuthology.orchestra.run.smithi039.stdout: "scrubs": { 2023-07-21T18:42:08.548 INFO:teuthology.orchestra.run.smithi039.stdout: "fda23896-ce3d-43c0-b8e4-d5ef6a9da871": { 2023-07-21T18:42:08.549 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2023-07-21T18:42:08.549 INFO:teuthology.orchestra.run.smithi039.stdout: "tag": "fda23896-ce3d-43c0-b8e4-d5ef6a9da871", 2023-07-21T18:42:08.549 INFO:teuthology.orchestra.run.smithi039.stdout: "options": "recursive,force" 2023-07-21T18:42:08.549 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T18:42:08.549 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T18:42:08.549 INFO:teuthology.orchestra.run.smithi039.stdout:} 2023-07-21T18:42:08.550 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.516+0000 7f64a624c700 1 -- 172.21.15.39:0/379833583 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f64880a13e0 msgr2=0x7f64880a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:08.551 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.516+0000 7f64a624c700 1 --2- 172.21.15.39:0/379833583 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f64880a13e0 0x7f64880a3890 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f649000b5c0 tx=0x7f6490005fb0 comp rx=0 tx=0).stop 2023-07-21T18:42:08.551 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.516+0000 7f64a624c700 1 -- 172.21.15.39:0/379833583 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64a0101610 msgr2=0x7f64a019bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:08.551 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.516+0000 7f64a624c700 1 --2- 172.21.15.39:0/379833583 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64a0101610 0x7f64a019bef0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f649400d950 tx=0x7f649400dd10 comp rx=0 tx=0).stop 2023-07-21T18:42:08.551 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.516+0000 7f64a624c700 1 -- 172.21.15.39:0/379833583 shutdown_connections 2023-07-21T18:42:08.551 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.516+0000 7f64a624c700 1 --2- 172.21.15.39:0/379833583 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f64880a13e0 0x7f64880a3890 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:08.551 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.516+0000 7f64a624c700 1 --2- 172.21.15.39:0/379833583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64a010a040 0x7f64a0195a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:08.551 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.516+0000 7f64a624c700 1 --2- 172.21.15.39:0/379833583 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f64a0100d00 0x7f64a019b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:08.552 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.516+0000 7f64a624c700 1 --2- 172.21.15.39:0/379833583 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64a0101610 0x7f64a019bef0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:08.552 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.516+0000 7f64a624c700 1 -- 172.21.15.39:0/379833583 >> 172.21.15.39:0/379833583 conn(0x7f64a00fa4b0 msgr2=0x7f64a010c220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:08.552 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.516+0000 7f64a624c700 1 -- 172.21.15.39:0/379833583 shutdown_connections 2023-07-21T18:42:08.552 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:08.516+0000 7f64a624c700 1 -- 172.21.15.39:0/379833583 wait complete. 2023-07-21T18:42:08.552 INFO:tasks.cephfs.filesystem:scrub status for tag:fda23896-ce3d-43c0-b8e4-d5ef6a9da871 - {'path': '/', 'tag': 'fda23896-ce3d-43c0-b8e4-d5ef6a9da871', 'options': 'recursive,force'} 2023-07-21T18:42:10.369 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:42:10 smithi193 ceph-mon[98204]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 KiB/s wr, 25 op/s 2023-07-21T18:42:10.370 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:42:10 smithi193 ceph-mon[98204]: scrub queued for path: / 2023-07-21T18:42:10.370 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:42:10 smithi193 ceph-mon[98204]: scrub summary: idle+waiting paths [/] 2023-07-21T18:42:10.370 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:42:10 smithi193 ceph-mon[98204]: scrub summary: active paths [/] 2023-07-21T18:42:10.370 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:42:10 smithi193 ceph-mon[98204]: scrub summary: idle+waiting paths [/] 2023-07-21T18:42:10.388 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:42:10 smithi155 ceph-mon[97934]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 KiB/s wr, 25 op/s 2023-07-21T18:42:10.388 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:42:10 smithi155 ceph-mon[97934]: scrub queued for path: / 2023-07-21T18:42:10.388 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:42:10 smithi155 ceph-mon[97934]: scrub summary: idle+waiting paths [/] 2023-07-21T18:42:10.389 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:42:10 smithi155 ceph-mon[97934]: scrub summary: active paths [/] 2023-07-21T18:42:10.389 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:42:10 smithi155 ceph-mon[97934]: scrub summary: idle+waiting paths [/] 2023-07-21T18:42:10.437 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:42:10 smithi039 ceph-mon[96489]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 KiB/s wr, 25 op/s 2023-07-21T18:42:10.437 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:42:10 smithi039 ceph-mon[96489]: scrub queued for path: / 2023-07-21T18:42:10.437 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:42:10 smithi039 ceph-mon[96489]: scrub summary: idle+waiting paths [/] 2023-07-21T18:42:10.437 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:42:10 smithi039 ceph-mon[96489]: scrub summary: active paths [/] 2023-07-21T18:42:10.437 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:42:10 smithi039 ceph-mon[96489]: scrub summary: idle+waiting paths [/] 2023-07-21T18:42:12.369 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:42:12 smithi193 ceph-mon[98204]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 KiB/s wr, 31 op/s 2023-07-21T18:42:12.388 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:42:12 smithi155 ceph-mon[97934]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 KiB/s wr, 31 op/s 2023-07-21T18:42:12.437 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:42:12 smithi039 ceph-mon[96489]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 KiB/s wr, 31 op/s 2023-07-21T18:42:13.062 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.040+0000 7f84bb614700 1 -- 172.21.15.39:0/306275199 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84b40ff3e0 msgr2=0x7f84b410b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:13.062 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.040+0000 7f84bb614700 1 --2- 172.21.15.39:0/306275199 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84b40ff3e0 0x7f84b410b860 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f84ac00b0a0 tx=0x7f84ac00b3b0 comp rx=0 tx=0).stop 2023-07-21T18:42:13.062 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.040+0000 7f84bb614700 1 -- 172.21.15.39:0/306275199 shutdown_connections 2023-07-21T18:42:13.062 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.040+0000 7f84bb614700 1 --2- 172.21.15.39:0/306275199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84b4104a30 0x7f84b4104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:13.063 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.040+0000 7f84bb614700 1 --2- 172.21.15.39:0/306275199 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84b40ff3e0 0x7f84b410b860 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:13.063 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.040+0000 7f84bb614700 1 --2- 172.21.15.39:0/306275199 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f84b40fea30 0x7f84b40feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:13.063 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.040+0000 7f84bb614700 1 -- 172.21.15.39:0/306275199 >> 172.21.15.39:0/306275199 conn(0x7f84b4074760 msgr2=0x7f84b4074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:13.063 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.040+0000 7f84bb614700 1 -- 172.21.15.39:0/306275199 shutdown_connections 2023-07-21T18:42:13.063 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.040+0000 7f84bb614700 1 -- 172.21.15.39:0/306275199 wait complete. 2023-07-21T18:42:13.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.041+0000 7f84bb614700 1 Processor -- start 2023-07-21T18:42:13.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.041+0000 7f84bb614700 1 -- start start 2023-07-21T18:42:13.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.041+0000 7f84bb614700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84b40fea30 0x7f84b419aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:13.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.041+0000 7f84bb614700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84b40ff3e0 0x7f84b419afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:13.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.041+0000 7f84bb614700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f84b4104a30 0x7f84b41948d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:13.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.041+0000 7f84bb614700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b4194f40 con 0x7f84b40ff3e0 2023-07-21T18:42:13.066 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.041+0000 7f84bb614700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b41950b0 con 0x7f84b40fea30 2023-07-21T18:42:13.066 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.041+0000 7f84bb614700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b4195270 con 0x7f84b4104a30 2023-07-21T18:42:13.066 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.041+0000 7f84b8baf700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84b40ff3e0 0x7f84b419afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:13.066 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.041+0000 7f84b8baf700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84b40ff3e0 0x7f84b419afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57876/0 (socket says 172.21.15.39:57876) 2023-07-21T18:42:13.066 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.041+0000 7f84b8baf700 1 -- 172.21.15.39:0/186107561 learned_addr learned my addr 172.21.15.39:0/186107561 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:13.066 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.042+0000 7f84b93b0700 1 --2- 172.21.15.39:0/186107561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84b40fea30 0x7f84b419aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:13.066 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.042+0000 7f84b9bb1700 1 --2- 172.21.15.39:0/186107561 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f84b4104a30 0x7f84b41948d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:13.066 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.042+0000 7f84b8baf700 1 -- 172.21.15.39:0/186107561 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f84b4104a30 msgr2=0x7f84b41948d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:13.067 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.042+0000 7f84b8baf700 1 --2- 172.21.15.39:0/186107561 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f84b4104a30 0x7f84b41948d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:13.067 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.042+0000 7f84b8baf700 1 -- 172.21.15.39:0/186107561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84b40fea30 msgr2=0x7f84b419aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:13.067 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.042+0000 7f84b8baf700 1 --2- 172.21.15.39:0/186107561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84b40fea30 0x7f84b419aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:13.067 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.042+0000 7f84b8baf700 1 -- 172.21.15.39:0/186107561 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84ac009d00 con 0x7f84b40ff3e0 2023-07-21T18:42:13.067 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.042+0000 7f84b93b0700 1 --2- 172.21.15.39:0/186107561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84b40fea30 0x7f84b419aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:42:13.067 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.042+0000 7f84b8baf700 1 --2- 172.21.15.39:0/186107561 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84b40ff3e0 0x7f84b419afb0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f84a400c9b0 tx=0x7f84a400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:13.067 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.042+0000 7f84aa7fc700 1 -- 172.21.15.39:0/186107561 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a4007b00 con 0x7f84b40ff3e0 2023-07-21T18:42:13.068 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.042+0000 7f84bb614700 1 -- 172.21.15.39:0/186107561 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84b4195550 con 0x7f84b40ff3e0 2023-07-21T18:42:13.068 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.042+0000 7f84aa7fc700 1 -- 172.21.15.39:0/186107561 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84a4007c60 con 0x7f84b40ff3e0 2023-07-21T18:42:13.068 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.042+0000 7f84aa7fc700 1 -- 172.21.15.39:0/186107561 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a4018660 con 0x7f84b40ff3e0 2023-07-21T18:42:13.068 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.042+0000 7f84bb614700 1 -- 172.21.15.39:0/186107561 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84b41a1a60 con 0x7f84b40ff3e0 2023-07-21T18:42:13.070 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.045+0000 7f84aa7fc700 1 -- 172.21.15.39:0/186107561 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f84a4018880 con 0x7f84b40ff3e0 2023-07-21T18:42:13.070 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.045+0000 7f84aa7fc700 1 --2- 172.21.15.39:0/186107561 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f84a00a13e0 0x7f84a00a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:13.070 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.045+0000 7f84aa7fc700 1 -- 172.21.15.39:0/186107561 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f84a40cd020 con 0x7f84b40ff3e0 2023-07-21T18:42:13.071 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.045+0000 7f84b93b0700 1 --2- 172.21.15.39:0/186107561 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f84a00a13e0 0x7f84a00a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:13.071 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.046+0000 7f84b93b0700 1 --2- 172.21.15.39:0/186107561 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f84a00a13e0 0x7f84a00a3890 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f84b00059d0 tx=0x7f84b0005900 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:13.071 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.046+0000 7f84c779ab80 1 Processor -- start 2023-07-21T18:42:13.071 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.046+0000 7f84c779ab80 1 -- start start 2023-07-21T18:42:13.071 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.047+0000 7f84c779ab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5649c41886b0 0x5649c4333fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:13.071 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.047+0000 7f84c779ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649c43344e0 0x5649c432e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:13.071 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.047+0000 7f84c779ab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649c4334950 0x5649c432e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:13.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.047+0000 7f84c779ab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649c4231c50 con 0x5649c41886b0 2023-07-21T18:42:13.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.047+0000 7f84c779ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649c42849e0 con 0x5649c43344e0 2023-07-21T18:42:13.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.047+0000 7f84c779ab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649c427de70 con 0x5649c4334950 2023-07-21T18:42:13.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.047+0000 7f84b93b0700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5649c41886b0 0x5649c4333fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:13.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.047+0000 7f84b93b0700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5649c41886b0 0x5649c4333fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57882/0 (socket says 172.21.15.39:57882) 2023-07-21T18:42:13.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.047+0000 7f84b93b0700 1 -- 172.21.15.39:0/3971869549 learned_addr learned my addr 172.21.15.39:0/3971869549 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:13.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.047+0000 7f84b8baf700 1 --2- 172.21.15.39:0/3971869549 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649c43344e0 0x5649c432e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:13.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.047+0000 7f84b9bb1700 1 --2- 172.21.15.39:0/3971869549 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649c4334950 0x5649c432e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:13.073 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.047+0000 7f84b93b0700 1 -- 172.21.15.39:0/3971869549 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649c4334950 msgr2=0x5649c432e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:13.073 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.047+0000 7f84b93b0700 1 --2- 172.21.15.39:0/3971869549 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649c4334950 0x5649c432e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:13.073 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.047+0000 7f84b93b0700 1 -- 172.21.15.39:0/3971869549 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649c43344e0 msgr2=0x5649c432e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:13.073 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.047+0000 7f84b93b0700 1 --2- 172.21.15.39:0/3971869549 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649c43344e0 0x5649c432e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:13.073 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.047+0000 7f84b93b0700 1 -- 172.21.15.39:0/3971869549 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5649c40abd80 con 0x5649c41886b0 2023-07-21T18:42:13.073 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.047+0000 7f84b9bb1700 1 --2- 172.21.15.39:0/3971869549 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649c4334950 0x5649c432e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:42:13.073 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.047+0000 7f84b8baf700 1 --2- 172.21.15.39:0/3971869549 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649c43344e0 0x5649c432e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:42:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.047+0000 7f84b93b0700 1 --2- 172.21.15.39:0/3971869549 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5649c41886b0 0x5649c4333fa0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f84b000ecc0 tx=0x7f84b0011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29e6f09649defba3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.047+0000 7f849f7fe700 1 -- 172.21.15.39:0/3971869549 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b0010d80 con 0x5649c41886b0 2023-07-21T18:42:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.047+0000 7f849f7fe700 1 -- 172.21.15.39:0/3971869549 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84b001d910 con 0x5649c41886b0 2023-07-21T18:42:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.047+0000 7f849f7fe700 1 -- 172.21.15.39:0/3971869549 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b000f7e0 con 0x5649c41886b0 2023-07-21T18:42:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.048+0000 7f84c779ab80 1 -- 172.21.15.39:0/3971869549 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5649c41886b0 msgr2=0x5649c4333fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.048+0000 7f84c779ab80 1 --2- 172.21.15.39:0/3971869549 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5649c41886b0 0x5649c4333fa0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f84b000ecc0 tx=0x7f84b0011000 comp rx=0 tx=0).stop 2023-07-21T18:42:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.048+0000 7f84c779ab80 1 -- 172.21.15.39:0/3971869549 shutdown_connections 2023-07-21T18:42:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.048+0000 7f84c779ab80 1 --2- 172.21.15.39:0/3971869549 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649c4334950 0x5649c432e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:13.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.048+0000 7f84c779ab80 1 --2- 172.21.15.39:0/3971869549 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5649c41886b0 0x5649c4333fa0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:13.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.048+0000 7f84c779ab80 1 --2- 172.21.15.39:0/3971869549 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649c43344e0 0x5649c432e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:13.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.048+0000 7f84c779ab80 1 -- 172.21.15.39:0/3971869549 >> 172.21.15.39:0/3971869549 conn(0x5649c4321be0 msgr2=0x5649c4326f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:13.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.048+0000 7f84c779ab80 1 -- 172.21.15.39:0/3971869549 shutdown_connections 2023-07-21T18:42:13.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.048+0000 7f84c779ab80 1 -- 172.21.15.39:0/3971869549 wait complete. 2023-07-21T18:42:13.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.049+0000 7f84c779ab80 1 -- start start 2023-07-21T18:42:13.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.049+0000 7f84c779ab80 1 Processor -- start 2023-07-21T18:42:13.078 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.049+0000 7f84c779ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649c41886b0 0x5649c4337b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:13.078 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.049+0000 7f84c779ab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649c43344e0 0x5649c4338040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:13.078 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.049+0000 7f84c779ab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5649c4338580 0x5649c4343eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:13.078 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.049+0000 7f84c779ab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649c421da60 con 0x5649c4338580 2023-07-21T18:42:13.079 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.049+0000 7f84c779ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649c421d760 con 0x5649c41886b0 2023-07-21T18:42:13.079 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.049+0000 7f84c779ab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649c41018c0 con 0x5649c43344e0 2023-07-21T18:42:13.079 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.049+0000 7f84c779ab80 10 client.0 _ll_register_callbacks cb 0x5649c421b700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:42:13.079 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.049+0000 7f84b9bb1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5649c4338580 0x5649c4343eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:13.079 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.049+0000 7f84b9bb1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5649c4338580 0x5649c4343eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57892/0 (socket says 172.21.15.39:57892) 2023-07-21T18:42:13.079 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.049+0000 7f84b9bb1700 1 -- 172.21.15.39:0/47180963 learned_addr learned my addr 172.21.15.39:0/47180963 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:13.079 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.050+0000 7f84b8baf700 1 --2- 172.21.15.39:0/47180963 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649c43344e0 0x5649c4338040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:13.079 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.050+0000 7f84b93b0700 1 --2- 172.21.15.39:0/47180963 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649c41886b0 0x5649c4337b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:13.080 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.050+0000 7f84b9bb1700 1 -- 172.21.15.39:0/47180963 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649c43344e0 msgr2=0x5649c4338040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:13.080 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.050+0000 7f84b9bb1700 1 --2- 172.21.15.39:0/47180963 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649c43344e0 0x5649c4338040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:13.080 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.050+0000 7f84b9bb1700 1 -- 172.21.15.39:0/47180963 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649c41886b0 msgr2=0x5649c4337b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:13.080 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.050+0000 7f84b9bb1700 1 --2- 172.21.15.39:0/47180963 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649c41886b0 0x5649c4337b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:13.080 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.050+0000 7f84b9bb1700 1 -- 172.21.15.39:0/47180963 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84b000e9a0 con 0x5649c4338580 2023-07-21T18:42:13.080 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.050+0000 7f84b9bb1700 1 --2- 172.21.15.39:0/47180963 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5649c4338580 0x5649c4343eb0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f84ac015000 tx=0x7f84ac004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2b3e2ce1512b439 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:13.080 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.050+0000 7f849cff9700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:42:13.081 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.050+0000 7f849cff9700 1 -- 172.21.15.39:0/47180963 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ac00e040 con 0x5649c4338580 2023-07-21T18:42:13.081 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.050+0000 7f849cff9700 1 -- 172.21.15.39:0/47180963 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f84ac0076e0 con 0x5649c4338580 2023-07-21T18:42:13.081 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.050+0000 7f84c779ab80 1 -- 172.21.15.39:0/47180963 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5649c41d7490 con 0x5649c4338580 2023-07-21T18:42:13.081 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.050+0000 7f849cff9700 1 -- 172.21.15.39:0/47180963 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84ac0078a0 con 0x5649c4338580 2023-07-21T18:42:13.081 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.050+0000 7f84c779ab80 10 client.15732 fetch_fsmap learned FSMap version 26 2023-07-21T18:42:13.081 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.050+0000 7f84c779ab80 1 -- 172.21.15.39:0/47180963 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5649c4094c00 con 0x5649c4338580 2023-07-21T18:42:13.081 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.051+0000 7f849cff9700 1 -- 172.21.15.39:0/47180963 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5649c41d7490 con 0x5649c4338580 2023-07-21T18:42:13.081 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.051+0000 7f849cff9700 1 -- 172.21.15.39:0/47180963 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f84ac019050 con 0x5649c4338580 2023-07-21T18:42:13.082 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.051+0000 7f84c779ab80 10 client.15732 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T18:42:13.082 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.051+0000 7f84c779ab80 10 client.15732 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:42:13.082 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.051+0000 7f84c779ab80 20 client.15732 populate_metadata read hostname 'smithi039' 2023-07-21T18:42:13.082 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.051+0000 7f84c779ab80 1 --2- 172.21.15.39:0/47180963 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5649c4348070 0x5649c434a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:13.082 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.051+0000 7f84c779ab80 4 client.15732 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:42:13.082 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.051+0000 7f84c779ab80 1 -- 172.21.15.39:0/47180963 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5649c4276e50 con 0x5649c4348070 2023-07-21T18:42:13.082 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.051+0000 7f84b93b0700 1 --2- 172.21.15.39:0/47180963 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5649c4348070 0x5649c434a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:13.082 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.052+0000 7f84b93b0700 1 --2- 172.21.15.39:0/47180963 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5649c4348070 0x5649c434a480 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96034f068e6790cc server_cookie=8a1fced7340c285f in_seq=0 out_seq=0 2023-07-21T18:42:13.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.052+0000 7f849cff9700 10 client.15732 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:42:13.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.052+0000 7f849cff9700 1 -- 172.21.15.39:0/47180963 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5649c4276e50 con 0x5649c4348070 2023-07-21T18:42:13.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.052+0000 7f849cff9700 10 client.15732 handle_command_reply: tid=0 2023-07-21T18:42:13.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.052+0000 7f84c779ab80 1 client.15732 shutdown 2023-07-21T18:42:13.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.052+0000 7f84c779ab80 1 -- 172.21.15.39:0/47180963 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5649c4338580 msgr2=0x5649c4343eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:13.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.052+0000 7f84c779ab80 1 --2- 172.21.15.39:0/47180963 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5649c4338580 0x5649c4343eb0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f84ac015000 tx=0x7f84ac004c80 comp rx=0 tx=0).stop 2023-07-21T18:42:13.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.052+0000 7f84c779ab80 1 -- 172.21.15.39:0/47180963 shutdown_connections 2023-07-21T18:42:13.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.053+0000 7f84c779ab80 1 --2- 172.21.15.39:0/47180963 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5649c4348070 0x5649c434a480 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:13.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.053+0000 7f84c779ab80 1 --2- 172.21.15.39:0/47180963 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649c41886b0 0x5649c4337b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:13.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.053+0000 7f84c779ab80 1 --2- 172.21.15.39:0/47180963 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5649c4338580 0x5649c4343eb0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:13.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.053+0000 7f849cff9700 0 client.15732 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:42:13.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.053+0000 7f84c779ab80 1 --2- 172.21.15.39:0/47180963 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649c43344e0 0x5649c4338040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:13.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.053+0000 7f84c779ab80 1 -- 172.21.15.39:0/47180963 >> 172.21.15.39:0/47180963 conn(0x5649c4321be0 msgr2=0x5649c43264e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:13.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.053+0000 7f84c779ab80 1 -- 172.21.15.39:0/47180963 shutdown_connections 2023-07-21T18:42:13.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.053+0000 7f84c779ab80 1 -- 172.21.15.39:0/47180963 wait complete. 2023-07-21T18:42:13.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.053+0000 7f84c779ab80 20 client.15732 trim_cache size 0 max 16384 2023-07-21T18:42:13.090 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.068+0000 7f84c779ab80 1 Processor -- start 2023-07-21T18:42:13.090 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.068+0000 7f84c779ab80 1 -- start start 2023-07-21T18:42:13.090 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.068+0000 7f84c779ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649c41886b0 0x5649c433ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:13.090 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.069+0000 7f84c779ab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649c43344e0 0x5649c4326c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:13.090 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.069+0000 7f84c779ab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5649c4338580 0x5649c43435e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:13.091 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.069+0000 7f84c779ab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649c41d7490 con 0x5649c4338580 2023-07-21T18:42:13.091 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.069+0000 7f84c779ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649c42f2060 con 0x5649c41886b0 2023-07-21T18:42:13.091 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.069+0000 7f84c779ab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649c4327b70 con 0x5649c43344e0 2023-07-21T18:42:13.094 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.069+0000 7f84b9bb1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5649c4338580 0x5649c43435e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:13.094 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.069+0000 7f84b9bb1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5649c4338580 0x5649c43435e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57900/0 (socket says 172.21.15.39:57900) 2023-07-21T18:42:13.094 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.069+0000 7f84b9bb1700 1 -- 172.21.15.39:0/4280969194 learned_addr learned my addr 172.21.15.39:0/4280969194 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:13.094 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.069+0000 7f84b93b0700 1 --2- 172.21.15.39:0/4280969194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649c41886b0 0x5649c433ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:13.094 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.069+0000 7f84b8baf700 1 --2- 172.21.15.39:0/4280969194 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649c43344e0 0x5649c4326c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:13.095 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.069+0000 7f84b9bb1700 1 -- 172.21.15.39:0/4280969194 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649c43344e0 msgr2=0x5649c4326c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:13.095 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.069+0000 7f84b9bb1700 1 --2- 172.21.15.39:0/4280969194 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649c43344e0 0x5649c4326c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:13.095 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.069+0000 7f84b9bb1700 1 -- 172.21.15.39:0/4280969194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649c41886b0 msgr2=0x5649c433ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:13.095 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.069+0000 7f84b9bb1700 1 --2- 172.21.15.39:0/4280969194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649c41886b0 0x5649c433ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:13.095 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.069+0000 7f84b9bb1700 1 -- 172.21.15.39:0/4280969194 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5649c40d21f0 con 0x5649c4338580 2023-07-21T18:42:13.095 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.069+0000 7f84b9bb1700 1 --2- 172.21.15.39:0/4280969194 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5649c4338580 0x5649c43435e0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f84ac007ab0 tx=0x7f84ac004d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3049cdc4b9a1bd21 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:13.095 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.069+0000 7f849ffff700 1 -- 172.21.15.39:0/4280969194 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ac00e070 con 0x5649c4338580 2023-07-21T18:42:13.096 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.069+0000 7f849ffff700 1 -- 172.21.15.39:0/4280969194 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84ac00f040 con 0x5649c4338580 2023-07-21T18:42:13.096 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.069+0000 7f849ffff700 1 -- 172.21.15.39:0/4280969194 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ac01c6a0 con 0x5649c4338580 2023-07-21T18:42:13.096 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.070+0000 7f84c779ab80 1 -- 172.21.15.39:0/4280969194 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5649c4338580 msgr2=0x5649c43435e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:13.096 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.070+0000 7f84c779ab80 1 --2- 172.21.15.39:0/4280969194 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5649c4338580 0x5649c43435e0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f84ac007ab0 tx=0x7f84ac004d70 comp rx=0 tx=0).stop 2023-07-21T18:42:13.096 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.070+0000 7f84c779ab80 1 -- 172.21.15.39:0/4280969194 shutdown_connections 2023-07-21T18:42:13.096 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.070+0000 7f84c779ab80 1 --2- 172.21.15.39:0/4280969194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649c41886b0 0x5649c433ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:13.096 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.070+0000 7f84c779ab80 1 --2- 172.21.15.39:0/4280969194 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5649c4338580 0x5649c43435e0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:13.096 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.070+0000 7f84c779ab80 1 --2- 172.21.15.39:0/4280969194 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649c43344e0 0x5649c4326c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.070+0000 7f84c779ab80 1 -- 172.21.15.39:0/4280969194 >> 172.21.15.39:0/4280969194 conn(0x5649c4321be0 msgr2=0x5649c43444b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.070+0000 7f84c779ab80 1 -- 172.21.15.39:0/4280969194 shutdown_connections 2023-07-21T18:42:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.070+0000 7f84c779ab80 1 -- 172.21.15.39:0/4280969194 wait complete. 2023-07-21T18:42:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.071+0000 7f84c779ab80 1 -- start start 2023-07-21T18:42:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.071+0000 7f84c779ab80 1 Processor -- start 2023-07-21T18:42:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.071+0000 7f84c779ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649c41886b0 0x5649c4330d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.071+0000 7f84c779ab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5649c43344e0 0x5649c4331280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.071+0000 7f84c779ab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649c4338580 0x5649c432f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.071+0000 7f84c779ab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649c433f430 con 0x5649c43344e0 2023-07-21T18:42:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.071+0000 7f84c779ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649c432ff00 con 0x5649c41886b0 2023-07-21T18:42:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.071+0000 7f84c779ab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649c43300c0 con 0x5649c4338580 2023-07-21T18:42:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.071+0000 7f84c779ab80 10 client.0 _ll_register_callbacks cb 0x7f84ac009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:42:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.071+0000 7f84b8baf700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5649c43344e0 0x5649c4331280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.071+0000 7f84b8baf700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5649c43344e0 0x5649c4331280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57904/0 (socket says 172.21.15.39:57904) 2023-07-21T18:42:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.071+0000 7f84b8baf700 1 -- 172.21.15.39:0/3027318261 learned_addr learned my addr 172.21.15.39:0/3027318261 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.071+0000 7f84b93b0700 1 --2- 172.21.15.39:0/3027318261 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649c41886b0 0x5649c4330d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.071+0000 7f84b9bb1700 1 --2- 172.21.15.39:0/3027318261 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649c4338580 0x5649c432f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.072+0000 7f84b8baf700 1 -- 172.21.15.39:0/3027318261 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649c4338580 msgr2=0x5649c432f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.072+0000 7f84b8baf700 1 --2- 172.21.15.39:0/3027318261 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649c4338580 0x5649c432f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.072+0000 7f84b8baf700 1 -- 172.21.15.39:0/3027318261 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649c41886b0 msgr2=0x5649c4330d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.072+0000 7f84b8baf700 1 --2- 172.21.15.39:0/3027318261 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649c41886b0 0x5649c4330d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.072+0000 7f84b8baf700 1 -- 172.21.15.39:0/3027318261 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5649c4094c00 con 0x5649c43344e0 2023-07-21T18:42:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.072+0000 7f84b93b0700 1 --2- 172.21.15.39:0/3027318261 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649c41886b0 0x5649c4330d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:42:13.100 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.072+0000 7f84b8baf700 1 --2- 172.21.15.39:0/3027318261 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5649c43344e0 0x5649c4331280 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f84a4099e50 tx=0x7f84a409def0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c697d08117b97e2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:13.100 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.072+0000 7f849cff9700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:42:13.100 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.072+0000 7f849cff9700 1 -- 172.21.15.39:0/3027318261 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a40b82c0 con 0x5649c43344e0 2023-07-21T18:42:13.100 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.072+0000 7f849cff9700 1 -- 172.21.15.39:0/3027318261 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f84a40a2310 con 0x5649c43344e0 2023-07-21T18:42:13.100 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.072+0000 7f84c779ab80 1 -- 172.21.15.39:0/3027318261 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f84b001d910 con 0x5649c43344e0 2023-07-21T18:42:13.100 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.072+0000 7f849cff9700 1 -- 172.21.15.39:0/3027318261 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84a40ca2f0 con 0x5649c43344e0 2023-07-21T18:42:13.100 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.072+0000 7f84c779ab80 10 client.15738 fetch_fsmap learned FSMap version 26 2023-07-21T18:42:13.100 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.072+0000 7f84c779ab80 1 -- 172.21.15.39:0/3027318261 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5649c40aa250 con 0x5649c43344e0 2023-07-21T18:42:13.101 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.072+0000 7f849cff9700 1 -- 172.21.15.39:0/3027318261 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84a4063020 con 0x5649c43344e0 2023-07-21T18:42:13.101 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.072+0000 7f849cff9700 1 -- 172.21.15.39:0/3027318261 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f84a40ca8f0 con 0x5649c43344e0 2023-07-21T18:42:13.101 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.073+0000 7f84c779ab80 10 client.15738 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T18:42:13.101 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.073+0000 7f84c779ab80 10 client.15738 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:42:13.101 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.073+0000 7f84c779ab80 20 client.15738 populate_metadata read hostname 'smithi039' 2023-07-21T18:42:13.101 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.073+0000 7f84c779ab80 1 --2- 172.21.15.39:0/3027318261 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5649c439cd50 0x5649c439f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:13.101 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.073+0000 7f84c779ab80 4 client.15738 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:42:13.102 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.073+0000 7f84c779ab80 1 -- 172.21.15.39:0/3027318261 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5649c4238ff0 con 0x5649c439cd50 2023-07-21T18:42:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.073+0000 7f84b93b0700 1 --2- 172.21.15.39:0/3027318261 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5649c439cd50 0x5649c439f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.073+0000 7f849cff9700 10 client.15738 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:42:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.073+0000 7f84b93b0700 1 --2- 172.21.15.39:0/3027318261 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5649c439cd50 0x5649c439f160 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0da3abfeb285029 server_cookie=aa80b454a57e1a02 in_seq=0 out_seq=0 2023-07-21T18:42:13.106 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.081+0000 7f849cff9700 1 -- 172.21.15.39:0/3027318261 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5649c4238ff0 con 0x5649c439cd50 2023-07-21T18:42:13.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.081+0000 7f849cff9700 10 client.15738 handle_command_reply: tid=0 2023-07-21T18:42:13.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.081+0000 7f84c779ab80 1 client.15738 shutdown 2023-07-21T18:42:13.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.081+0000 7f84c779ab80 1 -- 172.21.15.39:0/3027318261 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5649c43344e0 msgr2=0x5649c4331280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:13.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.081+0000 7f84c779ab80 1 --2- 172.21.15.39:0/3027318261 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5649c43344e0 0x5649c4331280 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f84a4099e50 tx=0x7f84a409def0 comp rx=0 tx=0).stop 2023-07-21T18:42:13.109 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.081+0000 7f84c779ab80 1 -- 172.21.15.39:0/3027318261 shutdown_connections 2023-07-21T18:42:13.109 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.081+0000 7f84c779ab80 1 --2- 172.21.15.39:0/3027318261 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5649c439cd50 0x5649c439f160 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:13.109 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.081+0000 7f84c779ab80 1 --2- 172.21.15.39:0/3027318261 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649c41886b0 0x5649c4330d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.081+0000 7f84c779ab80 1 --2- 172.21.15.39:0/3027318261 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649c4338580 0x5649c432f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.081+0000 7f849cff9700 0 client.15738 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:42:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.081+0000 7f84c779ab80 1 --2- 172.21.15.39:0/3027318261 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5649c43344e0 0x5649c4331280 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.081+0000 7f84c779ab80 1 -- 172.21.15.39:0/3027318261 >> 172.21.15.39:0/3027318261 conn(0x5649c4321be0 msgr2=0x5649c43435e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.082+0000 7f84c779ab80 1 -- 172.21.15.39:0/3027318261 shutdown_connections 2023-07-21T18:42:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.082+0000 7f84c779ab80 1 -- 172.21.15.39:0/3027318261 wait complete. 2023-07-21T18:42:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.082+0000 7f84c779ab80 20 client.15738 trim_cache size 0 max 16384 2023-07-21T18:42:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:42:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:42:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:42:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:42:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.082+0000 7f84bb614700 1 -- 172.21.15.39:0/186107561 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f84a00a13e0 msgr2=0x7f84a00a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.082+0000 7f84bb614700 1 --2- 172.21.15.39:0/186107561 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f84a00a13e0 0x7f84a00a3890 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f84b00059d0 tx=0x7f84b0005900 comp rx=0 tx=0).stop 2023-07-21T18:42:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.082+0000 7f84bb614700 1 -- 172.21.15.39:0/186107561 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84b40ff3e0 msgr2=0x7f84b419afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.082+0000 7f84bb614700 1 --2- 172.21.15.39:0/186107561 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84b40ff3e0 0x7f84b419afb0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f84a400c9b0 tx=0x7f84a400cd70 comp rx=0 tx=0).stop 2023-07-21T18:42:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.082+0000 7f84bb614700 1 -- 172.21.15.39:0/186107561 shutdown_connections 2023-07-21T18:42:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.082+0000 7f84bb614700 1 --2- 172.21.15.39:0/186107561 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f84a00a13e0 0x7f84a00a3890 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.082+0000 7f84bb614700 1 --2- 172.21.15.39:0/186107561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84b40fea30 0x7f84b419aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.082+0000 7f84bb614700 1 --2- 172.21.15.39:0/186107561 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f84b4104a30 0x7f84b41948d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.082+0000 7f84bb614700 1 --2- 172.21.15.39:0/186107561 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84b40ff3e0 0x7f84b419afb0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.082+0000 7f84bb614700 1 -- 172.21.15.39:0/186107561 >> 172.21.15.39:0/186107561 conn(0x7f84b4074760 msgr2=0x7f84b4105810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.083+0000 7f84bb614700 1 -- 172.21.15.39:0/186107561 shutdown_connections 2023-07-21T18:42:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:13.083+0000 7f84bb614700 1 -- 172.21.15.39:0/186107561 wait complete. 2023-07-21T18:42:14.369 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:42:14 smithi193 ceph-mon[98204]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 22 op/s 2023-07-21T18:42:14.369 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:42:14 smithi193 ceph-mon[98204]: scrub summary: idle 2023-07-21T18:42:14.388 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:42:14 smithi155 ceph-mon[97934]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 22 op/s 2023-07-21T18:42:14.388 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:42:14 smithi155 ceph-mon[97934]: scrub summary: idle 2023-07-21T18:42:14.437 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:42:14 smithi039 ceph-mon[96489]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 22 op/s 2023-07-21T18:42:14.437 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:42:14 smithi039 ceph-mon[96489]: scrub summary: idle 2023-07-21T18:42:16.370 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:42:16 smithi193 ceph-mon[98204]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 22 op/s 2023-07-21T18:42:16.388 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:42:16 smithi155 ceph-mon[97934]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 22 op/s 2023-07-21T18:42:16.437 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:42:16 smithi039 ceph-mon[96489]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 22 op/s 2023-07-21T18:42:18.369 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:42:18 smithi193 ceph-mon[98204]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 28 op/s 2023-07-21T18:42:18.388 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:42:18 smithi155 ceph-mon[97934]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 28 op/s 2023-07-21T18:42:18.437 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:42:18 smithi039 ceph-mon[96489]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 28 op/s 2023-07-21T18:42:20.369 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:42:20 smithi193 ceph-mon[98204]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2023-07-21T18:42:20.388 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:42:20 smithi155 ceph-mon[97934]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2023-07-21T18:42:20.436 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:42:20 smithi039 ceph-mon[96489]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2023-07-21T18:42:22.369 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:42:22 smithi193 ceph-mon[98204]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 2.6 KiB/s wr, 16 op/s 2023-07-21T18:42:22.388 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:42:22 smithi155 ceph-mon[97934]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 2.6 KiB/s wr, 16 op/s 2023-07-21T18:42:22.437 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:42:22 smithi039 ceph-mon[96489]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 2.6 KiB/s wr, 16 op/s 2023-07-21T18:42:24.369 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:42:24 smithi193 ceph-mon[98204]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2023-07-21T18:42:24.388 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:42:24 smithi155 ceph-mon[97934]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2023-07-21T18:42:24.437 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:42:24 smithi039 ceph-mon[96489]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2023-07-21T18:42:26.273 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.252+0000 7f8b5c1db700 1 -- 172.21.15.39:0/941261642 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b540fe260 msgr2=0x7f8b540fe630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:26.273 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.252+0000 7f8b5c1db700 1 --2- 172.21.15.39:0/941261642 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b540fe260 0x7f8b540fe630 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8b50009990 tx=0x7f8b50009ca0 comp rx=0 tx=0).stop 2023-07-21T18:42:26.274 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.252+0000 7f8b5c1db700 1 -- 172.21.15.39:0/941261642 shutdown_connections 2023-07-21T18:42:26.274 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.252+0000 7f8b5c1db700 1 --2- 172.21.15.39:0/941261642 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b540fe260 0x7f8b540fe630 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:26.274 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.252+0000 7f8b5c1db700 1 --2- 172.21.15.39:0/941261642 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b54104db0 0x7f8b5410c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:26.274 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.252+0000 7f8b5c1db700 1 --2- 172.21.15.39:0/941261642 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b540fec00 0x7f8b541046d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:26.274 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.252+0000 7f8b5c1db700 1 -- 172.21.15.39:0/941261642 >> 172.21.15.39:0/941261642 conn(0x7f8b54075f90 msgr2=0x7f8b54076390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:26.274 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.252+0000 7f8b5c1db700 1 -- 172.21.15.39:0/941261642 shutdown_connections 2023-07-21T18:42:26.275 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.252+0000 7f8b5c1db700 1 -- 172.21.15.39:0/941261642 wait complete. 2023-07-21T18:42:26.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.253+0000 7f8b5c1db700 1 Processor -- start 2023-07-21T18:42:26.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.253+0000 7f8b5c1db700 1 -- start start 2023-07-21T18:42:26.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.253+0000 7f8b5c1db700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b540fe260 0x7f8b54195ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:26.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.253+0000 7f8b5c1db700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b540fec00 0x7f8b54196220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.253+0000 7f8b5c1db700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b54104db0 0x7f8b541997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.253+0000 7f8b5c1db700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b54199de0 con 0x7f8b54104db0 2023-07-21T18:42:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.253+0000 7f8b5c1db700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b54199f50 con 0x7f8b540fe260 2023-07-21T18:42:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.253+0000 7f8b5c1db700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5419a110 con 0x7f8b540fec00 2023-07-21T18:42:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.254+0000 7f8b5a778700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b54104db0 0x7f8b541997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.254+0000 7f8b5a778700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b54104db0 0x7f8b541997e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57920/0 (socket says 172.21.15.39:57920) 2023-07-21T18:42:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.254+0000 7f8b5a778700 1 -- 172.21.15.39:0/1491988994 learned_addr learned my addr 172.21.15.39:0/1491988994 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.254+0000 7f8b59f77700 1 --2- 172.21.15.39:0/1491988994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b540fe260 0x7f8b54195ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.254+0000 7f8b59776700 1 --2- 172.21.15.39:0/1491988994 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b540fec00 0x7f8b54196220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.254+0000 7f8b5a778700 1 -- 172.21.15.39:0/1491988994 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b540fec00 msgr2=0x7f8b54196220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.254+0000 7f8b5a778700 1 --2- 172.21.15.39:0/1491988994 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b540fec00 0x7f8b54196220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.254+0000 7f8b5a778700 1 -- 172.21.15.39:0/1491988994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b540fe260 msgr2=0x7f8b54195ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.254+0000 7f8b5a778700 1 --2- 172.21.15.39:0/1491988994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b540fe260 0x7f8b54195ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.254+0000 7f8b5a778700 1 -- 172.21.15.39:0/1491988994 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b500095f0 con 0x7f8b54104db0 2023-07-21T18:42:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.254+0000 7f8b5a778700 1 --2- 172.21.15.39:0/1491988994 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b54104db0 0x7f8b541997e0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c00ef30 tx=0x7f8b4c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.254+0000 7f8b4affd700 1 -- 172.21.15.39:0/1491988994 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c018070 con 0x7f8b54104db0 2023-07-21T18:42:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.254+0000 7f8b5c1db700 1 -- 172.21.15.39:0/1491988994 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b5419a390 con 0x7f8b54104db0 2023-07-21T18:42:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.254+0000 7f8b5c1db700 1 -- 172.21.15.39:0/1491988994 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b541a2d50 con 0x7f8b54104db0 2023-07-21T18:42:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.254+0000 7f8b4affd700 1 -- 172.21.15.39:0/1491988994 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b4c00f040 con 0x7f8b54104db0 2023-07-21T18:42:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.255+0000 7f8b4affd700 1 -- 172.21.15.39:0/1491988994 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c00cd00 con 0x7f8b54104db0 2023-07-21T18:42:26.280 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.259+0000 7f8b68361b80 1 Processor -- start 2023-07-21T18:42:26.281 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.259+0000 7f8b68361b80 1 -- start start 2023-07-21T18:42:26.281 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.260+0000 7f8b68361b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556175ae36b0 0x556175c8ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:26.281 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.260+0000 7f8b68361b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556175c87040 0x556175c874b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:26.281 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.260+0000 7f8b68361b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556175c879f0 0x556175c9bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:26.281 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.260+0000 7f8b68361b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x556175b8cc50 con 0x556175ae36b0 2023-07-21T18:42:26.281 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.260+0000 7f8b68361b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556175bdf9e0 con 0x556175c879f0 2023-07-21T18:42:26.282 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.260+0000 7f8b68361b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x556175bd8e70 con 0x556175c87040 2023-07-21T18:42:26.282 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.260+0000 7f8b59776700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556175ae36b0 0x556175c8ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:26.283 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.260+0000 7f8b59776700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556175ae36b0 0x556175c8ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57922/0 (socket says 172.21.15.39:57922) 2023-07-21T18:42:26.283 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.260+0000 7f8b59776700 1 -- 172.21.15.39:0/4160472205 learned_addr learned my addr 172.21.15.39:0/4160472205 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:26.283 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.260+0000 7f8b59776700 1 --2- 172.21.15.39:0/4160472205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556175c879f0 0x556175c9bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:26.283 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.260+0000 7f8b59f77700 1 --2- 172.21.15.39:0/4160472205 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556175c87040 0x556175c874b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:26.283 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.260+0000 7f8b59776700 1 -- 172.21.15.39:0/4160472205 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556175c87040 msgr2=0x556175c874b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:26.283 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.260+0000 7f8b59776700 1 --2- 172.21.15.39:0/4160472205 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556175c87040 0x556175c874b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:26.283 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.260+0000 7f8b59776700 1 -- 172.21.15.39:0/4160472205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556175c879f0 msgr2=0x556175c9bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:26.284 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.260+0000 7f8b59776700 1 --2- 172.21.15.39:0/4160472205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556175c879f0 0x556175c9bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:26.284 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.260+0000 7f8b59776700 1 -- 172.21.15.39:0/4160472205 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556175a06d80 con 0x556175ae36b0 2023-07-21T18:42:26.284 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.261+0000 7f8b59f77700 1 --2- 172.21.15.39:0/4160472205 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556175c87040 0x556175c874b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:42:26.284 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.261+0000 7f8b59776700 1 --2- 172.21.15.39:0/4160472205 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556175ae36b0 0x556175c8ade0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f8b44005950 tx=0x7f8b440058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca0a9c55d517232 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:26.284 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.261+0000 7f8b3ffff700 1 -- 172.21.15.39:0/4160472205 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b44004050 con 0x556175ae36b0 2023-07-21T18:42:26.284 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.261+0000 7f8b3ffff700 1 -- 172.21.15.39:0/4160472205 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b44003680 con 0x556175ae36b0 2023-07-21T18:42:26.284 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.261+0000 7f8b3ffff700 1 -- 172.21.15.39:0/4160472205 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4400e610 con 0x556175ae36b0 2023-07-21T18:42:26.286 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.262+0000 7f8b68361b80 1 -- 172.21.15.39:0/4160472205 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556175ae36b0 msgr2=0x556175c8ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:26.286 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.262+0000 7f8b68361b80 1 --2- 172.21.15.39:0/4160472205 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556175ae36b0 0x556175c8ade0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f8b44005950 tx=0x7f8b440058e0 comp rx=0 tx=0).stop 2023-07-21T18:42:26.286 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.263+0000 7f8b4affd700 1 -- 172.21.15.39:0/1491988994 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f8b4c003a00 con 0x7f8b54104db0 2023-07-21T18:42:26.286 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.263+0000 7f8b68361b80 1 -- 172.21.15.39:0/4160472205 shutdown_connections 2023-07-21T18:42:26.286 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.263+0000 7f8b68361b80 1 --2- 172.21.15.39:0/4160472205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556175c879f0 0x556175c9bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:26.286 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.263+0000 7f8b68361b80 1 --2- 172.21.15.39:0/4160472205 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556175ae36b0 0x556175c8ade0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:26.286 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.263+0000 7f8b68361b80 1 --2- 172.21.15.39:0/4160472205 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556175c87040 0x556175c874b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.263+0000 7f8b68361b80 1 -- 172.21.15.39:0/4160472205 >> 172.21.15.39:0/4160472205 conn(0x556175c7cbe0 msgr2=0x556175c81f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.263+0000 7f8b4affd700 1 --2- 172.21.15.39:0/1491988994 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f8b400a1460 0x7f8b400a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.264+0000 7f8b59f77700 1 --2- 172.21.15.39:0/1491988994 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f8b400a1460 0x7f8b400a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.264+0000 7f8b4affd700 1 -- 172.21.15.39:0/1491988994 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f8b4c0cc9b0 con 0x7f8b54104db0 2023-07-21T18:42:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.264+0000 7f8b68361b80 1 -- 172.21.15.39:0/4160472205 shutdown_connections 2023-07-21T18:42:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.264+0000 7f8b68361b80 1 -- 172.21.15.39:0/4160472205 wait complete. 2023-07-21T18:42:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.264+0000 7f8b59f77700 1 --2- 172.21.15.39:0/1491988994 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f8b400a1460 0x7f8b400a3910 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8b5000b2e0 tx=0x7f8b50005fd0 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.264+0000 7f8b68361b80 1 -- start start 2023-07-21T18:42:26.288 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.264+0000 7f8b68361b80 1 Processor -- start 2023-07-21T18:42:26.290 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.265+0000 7f8b68361b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556175ae36b0 0x556175c95580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:26.290 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.265+0000 7f8b68361b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556175c87040 0x556175c95ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:26.290 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.265+0000 7f8b68361b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556175c879f0 0x556175c9b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:26.290 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.265+0000 7f8b68361b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x556175b78a60 con 0x556175ae36b0 2023-07-21T18:42:26.290 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.265+0000 7f8b68361b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556175b78760 con 0x556175c87040 2023-07-21T18:42:26.290 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.265+0000 7f8b68361b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x556175a5c8c0 con 0x556175c879f0 2023-07-21T18:42:26.290 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.265+0000 7f8b68361b80 10 client.0 _ll_register_callbacks cb 0x556175b76700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:42:26.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.265+0000 7f8b59776700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556175c87040 0x556175c95ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:26.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.265+0000 7f8b59776700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556175c87040 0x556175c95ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:42296/0 (socket says 172.21.15.39:42296) 2023-07-21T18:42:26.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.265+0000 7f8b59776700 1 -- 172.21.15.39:0/1451031874 learned_addr learned my addr 172.21.15.39:0/1451031874 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:26.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.265+0000 7f8b5a778700 1 --2- 172.21.15.39:0/1451031874 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556175c879f0 0x556175c9b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:26.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.266+0000 7f8b59776700 1 -- 172.21.15.39:0/1451031874 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556175c879f0 msgr2=0x556175c9b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:26.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.266+0000 7f8b59776700 1 --2- 172.21.15.39:0/1451031874 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556175c879f0 0x556175c9b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:26.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.266+0000 7f8b59776700 1 -- 172.21.15.39:0/1451031874 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556175ae36b0 msgr2=0x556175c95580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.266+0000 7f8b59776700 1 --2- 172.21.15.39:0/1451031874 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556175ae36b0 0x556175c95580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.266+0000 7f8b59776700 1 -- 172.21.15.39:0/1451031874 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b44019040 con 0x556175c87040 2023-07-21T18:42:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.266+0000 7f8b59776700 1 --2- 172.21.15.39:0/1451031874 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556175c87040 0x556175c95ac0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8b44003c40 tx=0x7f8b44009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=937f10ec28660b0e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.266+0000 7f8b3d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-21T18:42:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.266+0000 7f8b3d7fa700 1 -- 172.21.15.39:0/1451031874 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b44025070 con 0x556175c87040 2023-07-21T18:42:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.266+0000 7f8b3d7fa700 1 -- 172.21.15.39:0/1451031874 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f8b44004180 con 0x556175c87040 2023-07-21T18:42:26.293 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.266+0000 7f8b68361b80 1 -- 172.21.15.39:0/1451031874 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556175b32490 con 0x556175c87040 2023-07-21T18:42:26.293 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.267+0000 7f8b3d7fa700 1 -- 172.21.15.39:0/1451031874 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b44004340 con 0x556175c87040 2023-07-21T18:42:26.293 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.267+0000 7f8b68361b80 10 client.25337 fetch_fsmap learned FSMap version 26 2023-07-21T18:42:26.293 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.267+0000 7f8b68361b80 1 -- 172.21.15.39:0/1451031874 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5561759efc00 con 0x556175c87040 2023-07-21T18:42:26.293 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.267+0000 7f8b3d7fa700 1 -- 172.21.15.39:0/1451031874 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556175b32490 con 0x556175c87040 2023-07-21T18:42:26.293 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.267+0000 7f8b3d7fa700 1 -- 172.21.15.39:0/1451031874 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8b44004870 con 0x556175c87040 2023-07-21T18:42:26.293 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.267+0000 7f8b68361b80 10 client.25337 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T18:42:26.294 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.267+0000 7f8b68361b80 10 client.25337 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:42:26.294 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.267+0000 7f8b68361b80 20 client.25337 populate_metadata read hostname 'smithi039' 2023-07-21T18:42:26.294 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.267+0000 7f8b68361b80 1 --2- 172.21.15.39:0/1451031874 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x556175cb19f0 0x556175cb3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:26.294 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.267+0000 7f8b68361b80 4 client.25337 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:42:26.294 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.267+0000 7f8b68361b80 1 -- 172.21.15.39:0/1451031874 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556175bd1e50 con 0x556175cb19f0 2023-07-21T18:42:26.294 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.267+0000 7f8b59f77700 1 --2- 172.21.15.39:0/1451031874 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x556175cb19f0 0x556175cb3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:26.294 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.268+0000 7f8b3d7fa700 10 client.25337 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:42:26.294 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.268+0000 7f8b59f77700 1 --2- 172.21.15.39:0/1451031874 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x556175cb19f0 0x556175cb3e00 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c79df05d05aa6f21 server_cookie=1c5d0686cd3b56c5 in_seq=0 out_seq=0 2023-07-21T18:42:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.269+0000 7f8b3d7fa700 1 -- 172.21.15.39:0/1451031874 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556175bd1e50 con 0x556175cb19f0 2023-07-21T18:42:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.269+0000 7f8b3d7fa700 10 client.25337 handle_command_reply: tid=0 2023-07-21T18:42:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.269+0000 7f8b68361b80 1 client.25337 shutdown 2023-07-21T18:42:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.269+0000 7f8b68361b80 1 -- 172.21.15.39:0/1451031874 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556175c87040 msgr2=0x556175c95ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.269+0000 7f8b68361b80 1 --2- 172.21.15.39:0/1451031874 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556175c87040 0x556175c95ac0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8b44003c40 tx=0x7f8b44009780 comp rx=0 tx=0).stop 2023-07-21T18:42:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.269+0000 7f8b68361b80 1 -- 172.21.15.39:0/1451031874 shutdown_connections 2023-07-21T18:42:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.269+0000 7f8b68361b80 1 --2- 172.21.15.39:0/1451031874 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x556175cb19f0 0x556175cb3e00 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.269+0000 7f8b68361b80 1 --2- 172.21.15.39:0/1451031874 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556175c879f0 0x556175c9b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.269+0000 7f8b68361b80 1 --2- 172.21.15.39:0/1451031874 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556175ae36b0 0x556175c95580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.269+0000 7f8b68361b80 1 --2- 172.21.15.39:0/1451031874 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556175c87040 0x556175c95ac0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.269+0000 7f8b68361b80 1 -- 172.21.15.39:0/1451031874 >> 172.21.15.39:0/1451031874 conn(0x556175c7cbe0 msgr2=0x556175c814e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.269+0000 7f8b3d7fa700 0 client.25337 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:42:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.269+0000 7f8b68361b80 1 -- 172.21.15.39:0/1451031874 shutdown_connections 2023-07-21T18:42:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.269+0000 7f8b68361b80 1 -- 172.21.15.39:0/1451031874 wait complete. 2023-07-21T18:42:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.269+0000 7f8b68361b80 20 client.25337 trim_cache size 0 max 16384 2023-07-21T18:42:26.319 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.298+0000 7f8b68361b80 1 Processor -- start 2023-07-21T18:42:26.319 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.298+0000 7f8b68361b80 1 -- start start 2023-07-21T18:42:26.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.298+0000 7f8b68361b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556175ae36b0 0x556175c866b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:26.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.298+0000 7f8b68361b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556175c87040 0x556175c81c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.298+0000 7f8b68361b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556175c879f0 0x556175c95180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.298+0000 7f8b68361b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x556175c7ed30 con 0x556175c87040 2023-07-21T18:42:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.298+0000 7f8b68361b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556175c88210 con 0x556175c879f0 2023-07-21T18:42:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.298+0000 7f8b68361b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x556175c91e30 con 0x556175ae36b0 2023-07-21T18:42:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.298+0000 7f8b5a778700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556175c879f0 0x556175c95180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.298+0000 7f8b5a778700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556175c879f0 0x556175c95180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:42304/0 (socket says 172.21.15.39:42304) 2023-07-21T18:42:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.298+0000 7f8b59f77700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556175ae36b0 0x556175c866b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:26.322 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.298+0000 7f8b5a778700 1 -- 172.21.15.39:0/194472569 learned_addr learned my addr 172.21.15.39:0/194472569 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:26.322 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.299+0000 7f8b59776700 1 --2- 172.21.15.39:0/194472569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556175c87040 0x556175c81c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:26.322 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.299+0000 7f8b5a778700 1 -- 172.21.15.39:0/194472569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556175ae36b0 msgr2=0x556175c866b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:26.322 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.299+0000 7f8b5a778700 1 --2- 172.21.15.39:0/194472569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556175ae36b0 0x556175c866b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:26.322 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.299+0000 7f8b5a778700 1 -- 172.21.15.39:0/194472569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556175c87040 msgr2=0x556175c81c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:26.322 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.299+0000 7f8b5a778700 1 --2- 172.21.15.39:0/194472569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556175c87040 0x556175c81c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:26.323 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.299+0000 7f8b5a778700 1 -- 172.21.15.39:0/194472569 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556175cb8a10 con 0x556175c879f0 2023-07-21T18:42:26.323 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.299+0000 7f8b59f77700 1 --2- 172.21.15.39:0/194472569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556175ae36b0 0x556175c866b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:42:26.323 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.299+0000 7f8b59776700 1 --2- 172.21.15.39:0/194472569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556175c87040 0x556175c81c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:42:26.323 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.299+0000 7f8b5a778700 1 --2- 172.21.15.39:0/194472569 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556175c879f0 0x556175c95180 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c095de0 tx=0x7f8b4c0997b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4813b9b6edf2781 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:26.323 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.299+0000 7f8b48ff9700 1 -- 172.21.15.39:0/194472569 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c004ad0 con 0x556175c879f0 2023-07-21T18:42:26.324 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.299+0000 7f8b48ff9700 1 -- 172.21.15.39:0/194472569 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b4c0092e0 con 0x556175c879f0 2023-07-21T18:42:26.324 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.299+0000 7f8b48ff9700 1 -- 172.21.15.39:0/194472569 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c099b40 con 0x556175c879f0 2023-07-21T18:42:26.326 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.300+0000 7f8b68361b80 1 -- 172.21.15.39:0/194472569 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556175c879f0 msgr2=0x556175c95180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:26.326 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.300+0000 7f8b68361b80 1 --2- 172.21.15.39:0/194472569 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556175c879f0 0x556175c95180 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c095de0 tx=0x7f8b4c0997b0 comp rx=0 tx=0).stop 2023-07-21T18:42:26.326 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.300+0000 7f8b68361b80 1 -- 172.21.15.39:0/194472569 shutdown_connections 2023-07-21T18:42:26.326 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.300+0000 7f8b68361b80 1 --2- 172.21.15.39:0/194472569 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556175c879f0 0x556175c95180 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:26.327 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.300+0000 7f8b68361b80 1 --2- 172.21.15.39:0/194472569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556175ae36b0 0x556175c866b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:26.327 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.300+0000 7f8b68361b80 1 --2- 172.21.15.39:0/194472569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556175c87040 0x556175c81c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:26.327 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.300+0000 7f8b68361b80 1 -- 172.21.15.39:0/194472569 >> 172.21.15.39:0/194472569 conn(0x556175c7cbe0 msgr2=0x556175c8ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:26.327 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.300+0000 7f8b68361b80 1 -- 172.21.15.39:0/194472569 shutdown_connections 2023-07-21T18:42:26.327 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.300+0000 7f8b68361b80 1 -- 172.21.15.39:0/194472569 wait complete. 2023-07-21T18:42:26.327 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.301+0000 7f8b68361b80 1 -- start start 2023-07-21T18:42:26.327 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.301+0000 7f8b68361b80 1 Processor -- start 2023-07-21T18:42:26.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.301+0000 7f8b68361b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556175ae36b0 0x556175c98d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:26.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.301+0000 7f8b68361b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556175c87040 0x556175c99280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:26.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.301+0000 7f8b68361b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556175c879f0 0x556175d04890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:26.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.301+0000 7f8b68361b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x556175c999b0 con 0x556175c87040 2023-07-21T18:42:26.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.301+0000 7f8b68361b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556175d04dd0 con 0x556175ae36b0 2023-07-21T18:42:26.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.301+0000 7f8b68361b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x556175d04f60 con 0x556175c879f0 2023-07-21T18:42:26.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.301+0000 7f8b68361b80 10 client.0 _ll_register_callbacks cb 0x7f8b44009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:42:26.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.302+0000 7f8b59f77700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556175ae36b0 0x556175c98d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:26.329 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.302+0000 7f8b59f77700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556175ae36b0 0x556175c98d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:42308/0 (socket says 172.21.15.39:42308) 2023-07-21T18:42:26.329 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.302+0000 7f8b59f77700 1 -- 172.21.15.39:0/3877947933 learned_addr learned my addr 172.21.15.39:0/3877947933 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:26.329 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.302+0000 7f8b5a778700 1 --2- 172.21.15.39:0/3877947933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556175c879f0 0x556175d04890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:26.329 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.302+0000 7f8b59f77700 1 -- 172.21.15.39:0/3877947933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556175c879f0 msgr2=0x556175d04890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:26.329 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.302+0000 7f8b59776700 1 --2- 172.21.15.39:0/3877947933 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556175c87040 0x556175c99280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:26.329 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.302+0000 7f8b59f77700 1 --2- 172.21.15.39:0/3877947933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556175c879f0 0x556175d04890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:26.329 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.302+0000 7f8b59f77700 1 -- 172.21.15.39:0/3877947933 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556175c87040 msgr2=0x556175c99280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:26.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.302+0000 7f8b59f77700 1 --2- 172.21.15.39:0/3877947933 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556175c87040 0x556175c99280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:26.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.302+0000 7f8b59f77700 1 -- 172.21.15.39:0/3877947933 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b5410d1c0 con 0x556175ae36b0 2023-07-21T18:42:26.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.302+0000 7f8b59776700 1 --2- 172.21.15.39:0/3877947933 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556175c87040 0x556175c99280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:42:26.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.302+0000 7f8b59f77700 1 --2- 172.21.15.39:0/3877947933 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556175ae36b0 0x556175c98d40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8b50009990 tx=0x7f8b500050e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37a9d53bff379db server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:26.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.302+0000 7f8b3d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-21T18:42:26.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.303+0000 7f8b3d7fa700 1 -- 172.21.15.39:0/3877947933 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b5001d050 con 0x556175ae36b0 2023-07-21T18:42:26.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.303+0000 7f8b3d7fa700 1 -- 172.21.15.39:0/3877947933 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f8b50031a00 con 0x556175ae36b0 2023-07-21T18:42:26.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.303+0000 7f8b68361b80 1 -- 172.21.15.39:0/3877947933 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8b44003680 con 0x556175ae36b0 2023-07-21T18:42:26.331 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.303+0000 7f8b3d7fa700 1 -- 172.21.15.39:0/3877947933 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b50031bc0 con 0x556175ae36b0 2023-07-21T18:42:26.331 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.303+0000 7f8b68361b80 10 client.25343 fetch_fsmap learned FSMap version 26 2023-07-21T18:42:26.331 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.303+0000 7f8b68361b80 1 -- 172.21.15.39:0/3877947933 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f8b541a2d50 con 0x556175ae36b0 2023-07-21T18:42:26.331 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.303+0000 7f8b3d7fa700 1 -- 172.21.15.39:0/3877947933 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b44003680 con 0x556175ae36b0 2023-07-21T18:42:26.331 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.303+0000 7f8b3d7fa700 1 -- 172.21.15.39:0/3877947933 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8b50043460 con 0x556175ae36b0 2023-07-21T18:42:26.331 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.303+0000 7f8b68361b80 10 client.25343 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T18:42:26.331 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.303+0000 7f8b68361b80 10 client.25343 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:42:26.331 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.303+0000 7f8b68361b80 20 client.25343 populate_metadata read hostname 'smithi039' 2023-07-21T18:42:26.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.303+0000 7f8b68361b80 1 --2- 172.21.15.39:0/3877947933 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x556175d06670 0x556175d08a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:26.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.303+0000 7f8b68361b80 4 client.25343 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:42:26.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.303+0000 7f8b68361b80 1 -- 172.21.15.39:0/3877947933 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556175b93ff0 con 0x556175d06670 2023-07-21T18:42:26.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.304+0000 7f8b59776700 1 --2- 172.21.15.39:0/3877947933 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x556175d06670 0x556175d08a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:26.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.304+0000 7f8b3d7fa700 10 client.25343 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:42:26.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.304+0000 7f8b59776700 1 --2- 172.21.15.39:0/3877947933 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x556175d06670 0x556175d08a80 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40bdb88ccc48871e server_cookie=b3f41d07ef6e4217 in_seq=0 out_seq=0 2023-07-21T18:42:26.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.317+0000 7f8b3d7fa700 1 -- 172.21.15.39:0/3877947933 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556175b93ff0 con 0x556175d06670 2023-07-21T18:42:26.338 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.317+0000 7f8b3d7fa700 10 client.25343 handle_command_reply: tid=0 2023-07-21T18:42:26.338 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.317+0000 7f8b68361b80 1 client.25343 shutdown 2023-07-21T18:42:26.338 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.317+0000 7f8b68361b80 1 -- 172.21.15.39:0/3877947933 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556175ae36b0 msgr2=0x556175c98d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:26.338 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.317+0000 7f8b68361b80 1 --2- 172.21.15.39:0/3877947933 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556175ae36b0 0x556175c98d40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8b50009990 tx=0x7f8b500050e0 comp rx=0 tx=0).stop 2023-07-21T18:42:26.339 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.317+0000 7f8b68361b80 1 -- 172.21.15.39:0/3877947933 shutdown_connections 2023-07-21T18:42:26.339 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.317+0000 7f8b68361b80 1 --2- 172.21.15.39:0/3877947933 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x556175d06670 0x556175d08a80 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:26.339 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.317+0000 7f8b68361b80 1 --2- 172.21.15.39:0/3877947933 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556175ae36b0 0x556175c98d40 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:26.339 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.317+0000 7f8b68361b80 1 --2- 172.21.15.39:0/3877947933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556175c879f0 0x556175d04890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:26.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.317+0000 7f8b68361b80 1 --2- 172.21.15.39:0/3877947933 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556175c87040 0x556175c99280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:26.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.317+0000 7f8b68361b80 1 -- 172.21.15.39:0/3877947933 >> 172.21.15.39:0/3877947933 conn(0x556175c7cbe0 msgr2=0x556175c80c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:26.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.317+0000 7f8b3d7fa700 0 client.25343 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:42:26.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.317+0000 7f8b68361b80 1 -- 172.21.15.39:0/3877947933 shutdown_connections 2023-07-21T18:42:26.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.317+0000 7f8b68361b80 1 -- 172.21.15.39:0/3877947933 wait complete. 2023-07-21T18:42:26.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.317+0000 7f8b68361b80 20 client.25343 trim_cache size 0 max 16384 2023-07-21T18:42:26.341 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:42:26.341 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:42:26.341 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:42:26.341 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:42:26.342 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.318+0000 7f8b5c1db700 1 -- 172.21.15.39:0/1491988994 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f8b400a1460 msgr2=0x7f8b400a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:26.342 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.318+0000 7f8b5c1db700 1 --2- 172.21.15.39:0/1491988994 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f8b400a1460 0x7f8b400a3910 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8b5000b2e0 tx=0x7f8b50005fd0 comp rx=0 tx=0).stop 2023-07-21T18:42:26.342 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.318+0000 7f8b5c1db700 1 -- 172.21.15.39:0/1491988994 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b54104db0 msgr2=0x7f8b541997e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:26.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.318+0000 7f8b5c1db700 1 --2- 172.21.15.39:0/1491988994 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b54104db0 0x7f8b541997e0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c00ef30 tx=0x7f8b4c00c3a0 comp rx=0 tx=0).stop 2023-07-21T18:42:26.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.318+0000 7f8b5c1db700 1 -- 172.21.15.39:0/1491988994 shutdown_connections 2023-07-21T18:42:26.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.318+0000 7f8b5c1db700 1 --2- 172.21.15.39:0/1491988994 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f8b400a1460 0x7f8b400a3910 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:26.344 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.318+0000 7f8b5c1db700 1 --2- 172.21.15.39:0/1491988994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b540fe260 0x7f8b54195ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:26.344 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.318+0000 7f8b5c1db700 1 --2- 172.21.15.39:0/1491988994 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b54104db0 0x7f8b541997e0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:26.344 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.318+0000 7f8b5c1db700 1 --2- 172.21.15.39:0/1491988994 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b540fec00 0x7f8b54196220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:26.344 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.318+0000 7f8b5c1db700 1 -- 172.21.15.39:0/1491988994 >> 172.21.15.39:0/1491988994 conn(0x7f8b54075f90 msgr2=0x7f8b54103710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:26.345 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.318+0000 7f8b5c1db700 1 -- 172.21.15.39:0/1491988994 shutdown_connections 2023-07-21T18:42:26.345 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:26.318+0000 7f8b5c1db700 1 -- 172.21.15.39:0/1491988994 wait complete. 2023-07-21T18:42:26.369 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:42:26 smithi193 ceph-mon[98204]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2023-07-21T18:42:26.387 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:42:26 smithi155 ceph-mon[97934]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2023-07-21T18:42:26.436 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:42:26 smithi039 ceph-mon[96489]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2023-07-21T18:42:28.369 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:42:28 smithi193 ceph-mon[98204]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.5 KiB/s wr, 14 op/s 2023-07-21T18:42:28.387 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:42:28 smithi155 ceph-mon[97934]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.5 KiB/s wr, 14 op/s 2023-07-21T18:42:28.436 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:42:28 smithi039 ceph-mon[96489]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.5 KiB/s wr, 14 op/s 2023-07-21T18:42:30.369 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:42:30 smithi193 ceph-mon[98204]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2023-07-21T18:42:30.387 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:42:30 smithi155 ceph-mon[97934]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2023-07-21T18:42:30.436 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:42:30 smithi039 ceph-mon[96489]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2023-07-21T18:42:32.369 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:42:32 smithi193 ceph-mon[98204]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.5 KiB/s wr, 14 op/s 2023-07-21T18:42:32.387 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:42:32 smithi155 ceph-mon[97934]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.5 KiB/s wr, 14 op/s 2023-07-21T18:42:32.436 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:42:32 smithi039 ceph-mon[96489]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.5 KiB/s wr, 14 op/s 2023-07-21T18:42:34.368 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:42:34 smithi193 ceph-mon[98204]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-07-21T18:42:34.387 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:42:34 smithi155 ceph-mon[97934]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-07-21T18:42:34.436 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:42:34 smithi039 ceph-mon[96489]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-07-21T18:42:36.369 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:42:36 smithi193 ceph-mon[98204]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-07-21T18:42:36.387 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:42:36 smithi155 ceph-mon[97934]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-07-21T18:42:36.436 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:42:36 smithi039 ceph-mon[96489]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-07-21T18:42:38.387 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:42:38 smithi155 ceph-mon[97934]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2023-07-21T18:42:38.436 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:42:38 smithi039 ceph-mon[96489]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2023-07-21T18:42:38.553 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T18:42:38.618 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:42:38 smithi193 ceph-mon[98204]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2023-07-21T18:42:38.771 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.749+0000 7fd7bc5ed700 1 -- 172.21.15.39:0/354876139 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7b41049e0 msgr2=0x7fd7b4104db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:38.771 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.749+0000 7fd7bc5ed700 1 --2- 172.21.15.39:0/354876139 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7b41049e0 0x7fd7b4104db0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0009990 tx=0x7fd7b0009ca0 comp rx=0 tx=0).stop 2023-07-21T18:42:38.771 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.749+0000 7fd7bc5ed700 1 -- 172.21.15.39:0/354876139 shutdown_connections 2023-07-21T18:42:38.771 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.749+0000 7fd7bc5ed700 1 --2- 172.21.15.39:0/354876139 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd7b40ff380 0x7fd7b410b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:38.771 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.749+0000 7fd7bc5ed700 1 --2- 172.21.15.39:0/354876139 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7b41049e0 0x7fd7b4104db0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:38.772 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.749+0000 7fd7bc5ed700 1 --2- 172.21.15.39:0/354876139 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7b40fe9d0 0x7fd7b40fee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:38.772 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.749+0000 7fd7bc5ed700 1 -- 172.21.15.39:0/354876139 >> 172.21.15.39:0/354876139 conn(0x7fd7b40fa4b0 msgr2=0x7fd7b40fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:38.772 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.749+0000 7fd7bc5ed700 1 -- 172.21.15.39:0/354876139 shutdown_connections 2023-07-21T18:42:38.772 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.749+0000 7fd7bc5ed700 1 -- 172.21.15.39:0/354876139 wait complete. 2023-07-21T18:42:38.773 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.750+0000 7fd7bc5ed700 1 Processor -- start 2023-07-21T18:42:38.774 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.750+0000 7fd7bc5ed700 1 -- start start 2023-07-21T18:42:38.774 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.750+0000 7fd7bc5ed700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd7b40fe9d0 0x7fd7b4197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:38.774 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.750+0000 7fd7bc5ed700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7b40ff380 0x7fd7b4198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:38.774 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.750+0000 7fd7bc5ed700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7b41049e0 0x7fd7b419c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:38.774 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.750+0000 7fd7bc5ed700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b419ceb0 con 0x7fd7b41049e0 2023-07-21T18:42:38.774 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.750+0000 7fd7bc5ed700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b419d020 con 0x7fd7b40ff380 2023-07-21T18:42:38.774 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.750+0000 7fd7bc5ed700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b419d1e0 con 0x7fd7b40fe9d0 2023-07-21T18:42:38.775 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.750+0000 7fd7bab8a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7b41049e0 0x7fd7b419c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:38.775 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.750+0000 7fd7bab8a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7b41049e0 0x7fd7b419c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57962/0 (socket says 172.21.15.39:57962) 2023-07-21T18:42:38.775 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.750+0000 7fd7bab8a700 1 -- 172.21.15.39:0/1797902849 learned_addr learned my addr 172.21.15.39:0/1797902849 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:38.775 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.750+0000 7fd7b9b88700 1 --2- 172.21.15.39:0/1797902849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7b40ff380 0x7fd7b4198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:38.775 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.751+0000 7fd7ba389700 1 --2- 172.21.15.39:0/1797902849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd7b40fe9d0 0x7fd7b4197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:38.775 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.751+0000 7fd7bab8a700 1 -- 172.21.15.39:0/1797902849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd7b40fe9d0 msgr2=0x7fd7b4197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:38.776 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.751+0000 7fd7bab8a700 1 --2- 172.21.15.39:0/1797902849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd7b40fe9d0 0x7fd7b4197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:38.776 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.751+0000 7fd7bab8a700 1 -- 172.21.15.39:0/1797902849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7b40ff380 msgr2=0x7fd7b4198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:38.776 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.751+0000 7fd7bab8a700 1 --2- 172.21.15.39:0/1797902849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7b40ff380 0x7fd7b4198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:38.776 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.751+0000 7fd7bab8a700 1 -- 172.21.15.39:0/1797902849 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b00095f0 con 0x7fd7b41049e0 2023-07-21T18:42:38.776 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.751+0000 7fd7bab8a700 1 --2- 172.21.15.39:0/1797902849 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7b41049e0 0x7fd7b419c870 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac007ee0 tx=0x7fd7ac00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:38.776 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.751+0000 7fd7ab7fe700 1 -- 172.21.15.39:0/1797902849 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ac00ddd0 con 0x7fd7b41049e0 2023-07-21T18:42:38.776 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.751+0000 7fd7bc5ed700 1 -- 172.21.15.39:0/1797902849 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7b419d4c0 con 0x7fd7b41049e0 2023-07-21T18:42:38.777 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.751+0000 7fd7ab7fe700 1 -- 172.21.15.39:0/1797902849 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7ac00f040 con 0x7fd7b41049e0 2023-07-21T18:42:38.777 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.751+0000 7fd7ab7fe700 1 -- 172.21.15.39:0/1797902849 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ac015400 con 0x7fd7b41049e0 2023-07-21T18:42:38.777 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.751+0000 7fd7bc5ed700 1 -- 172.21.15.39:0/1797902849 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7b41a4d00 con 0x7fd7b41049e0 2023-07-21T18:42:38.777 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.755+0000 7fd7ab7fe700 1 -- 172.21.15.39:0/1797902849 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7fd7ac004020 con 0x7fd7b41049e0 2023-07-21T18:42:38.778 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.756+0000 7fd7ab7fe700 1 --2- 172.21.15.39:0/1797902849 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fd7a00a1460 0x7fd7a00a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:38.778 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.756+0000 7fd7ab7fe700 1 -- 172.21.15.39:0/1797902849 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fd7ac0cc7d0 con 0x7fd7b41049e0 2023-07-21T18:42:38.778 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.756+0000 7fd7c8773b80 1 Processor -- start 2023-07-21T18:42:38.779 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.756+0000 7fd7ba389700 1 --2- 172.21.15.39:0/1797902849 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fd7a00a1460 0x7fd7a00a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:38.779 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.757+0000 7fd7c8773b80 1 -- start start 2023-07-21T18:42:38.779 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.757+0000 7fd7c8773b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a5578aae0 0x560a5578aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:38.779 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.757+0000 7fd7c8773b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560a5578b3f0 0x560a55788240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:38.780 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.757+0000 7fd7c8773b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a5579ed50 0x560a557a1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:38.780 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.757+0000 7fd7c8773b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a55688cb0 con 0x560a5578b3f0 2023-07-21T18:42:38.780 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.757+0000 7fd7c8773b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a556dba40 con 0x560a5578aae0 2023-07-21T18:42:38.780 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.757+0000 7fd7c8773b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a556d4ed0 con 0x560a5579ed50 2023-07-21T18:42:38.782 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.757+0000 7fd7ba389700 1 --2- 172.21.15.39:0/1797902849 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fd7a00a1460 0x7fd7a00a3910 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd7b00058e0 tx=0x7fd7b0005fd0 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:38.782 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.757+0000 7fd7b9b88700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560a5578b3f0 0x560a55788240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:38.782 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.757+0000 7fd7b9b88700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560a5578b3f0 0x560a55788240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57970/0 (socket says 172.21.15.39:57970) 2023-07-21T18:42:38.782 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.757+0000 7fd7b9b88700 1 -- 172.21.15.39:0/2198395862 learned_addr learned my addr 172.21.15.39:0/2198395862 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:38.782 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.757+0000 7fd7ba389700 1 --2- 172.21.15.39:0/2198395862 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a5578aae0 0x560a5578aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:38.783 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.757+0000 7fd7b9b88700 1 -- 172.21.15.39:0/2198395862 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a5579ed50 msgr2=0x560a557a1130 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T18:42:38.783 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.757+0000 7fd7b9b88700 1 --2- 172.21.15.39:0/2198395862 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a5579ed50 0x560a557a1130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:38.783 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.757+0000 7fd7b9b88700 1 -- 172.21.15.39:0/2198395862 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a5578aae0 msgr2=0x560a5578aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:38.783 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.757+0000 7fd7b9b88700 1 --2- 172.21.15.39:0/2198395862 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a5578aae0 0x560a5578aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:38.783 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.758+0000 7fd7b9b88700 1 -- 172.21.15.39:0/2198395862 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a55502de0 con 0x560a5578b3f0 2023-07-21T18:42:38.783 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.758+0000 7fd7ba389700 1 --2- 172.21.15.39:0/2198395862 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a5578aae0 0x560a5578aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:42:38.783 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.758+0000 7fd7b9b88700 1 --2- 172.21.15.39:0/2198395862 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560a5578b3f0 0x560a55788240 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fd7a4009d20 tx=0x7fd7a4011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95d2f6f7c35eb0ea server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:38.784 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.758+0000 7fd7a8ff9700 1 -- 172.21.15.39:0/2198395862 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a4019630 con 0x560a5578b3f0 2023-07-21T18:42:38.784 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.758+0000 7fd7a8ff9700 1 -- 172.21.15.39:0/2198395862 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7a4019c50 con 0x560a5578b3f0 2023-07-21T18:42:38.784 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.758+0000 7fd7a8ff9700 1 -- 172.21.15.39:0/2198395862 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a400ebe0 con 0x560a5578b3f0 2023-07-21T18:42:38.784 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.759+0000 7fd7c8773b80 1 -- 172.21.15.39:0/2198395862 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560a5578b3f0 msgr2=0x560a55788240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:38.784 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.759+0000 7fd7c8773b80 1 --2- 172.21.15.39:0/2198395862 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560a5578b3f0 0x560a55788240 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fd7a4009d20 tx=0x7fd7a4011720 comp rx=0 tx=0).stop 2023-07-21T18:42:38.784 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.759+0000 7fd7c8773b80 1 -- 172.21.15.39:0/2198395862 shutdown_connections 2023-07-21T18:42:38.784 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.759+0000 7fd7c8773b80 1 --2- 172.21.15.39:0/2198395862 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a5578aae0 0x560a5578aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:38.784 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.759+0000 7fd7c8773b80 1 --2- 172.21.15.39:0/2198395862 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a5579ed50 0x560a557a1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:38.785 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.759+0000 7fd7c8773b80 1 --2- 172.21.15.39:0/2198395862 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560a5578b3f0 0x560a55788240 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:38.785 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.759+0000 7fd7c8773b80 1 -- 172.21.15.39:0/2198395862 >> 172.21.15.39:0/2198395862 conn(0x560a55778da0 msgr2=0x560a5577e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:38.785 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.760+0000 7fd7c8773b80 1 -- 172.21.15.39:0/2198395862 shutdown_connections 2023-07-21T18:42:38.785 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.760+0000 7fd7c8773b80 1 -- 172.21.15.39:0/2198395862 wait complete. 2023-07-21T18:42:38.785 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.760+0000 7fd7c8773b80 1 -- start start 2023-07-21T18:42:38.785 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.760+0000 7fd7c8773b80 1 Processor -- start 2023-07-21T18:42:38.790 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.760+0000 7fd7c8773b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560a5578aae0 0x560a5579f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:38.790 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.761+0000 7fd7c8773b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a5578b3f0 0x560a5579f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:38.790 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.761+0000 7fd7c8773b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a5579ed50 0x560a557aa1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:38.791 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.761+0000 7fd7c8773b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a55674ac0 con 0x560a5578aae0 2023-07-21T18:42:38.791 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.761+0000 7fd7c8773b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a556747c0 con 0x560a5578b3f0 2023-07-21T18:42:38.791 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.761+0000 7fd7c8773b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a55558920 con 0x560a5579ed50 2023-07-21T18:42:38.792 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.761+0000 7fd7c8773b80 10 client.0 _ll_register_callbacks cb 0x560a55672760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:42:38.792 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.761+0000 7fd7ba389700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560a5578aae0 0x560a5579f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:38.792 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.761+0000 7fd7ba389700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560a5578aae0 0x560a5579f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57972/0 (socket says 172.21.15.39:57972) 2023-07-21T18:42:38.792 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.761+0000 7fd7ba389700 1 -- 172.21.15.39:0/1736051622 learned_addr learned my addr 172.21.15.39:0/1736051622 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:38.792 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.761+0000 7fd7ba389700 1 -- 172.21.15.39:0/1736051622 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a5579ed50 msgr2=0x560a557aa1a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T18:42:38.792 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.761+0000 7fd7ba389700 1 --2- 172.21.15.39:0/1736051622 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a5579ed50 0x560a557aa1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:38.792 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.761+0000 7fd7ba389700 1 -- 172.21.15.39:0/1736051622 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a5578b3f0 msgr2=0x560a5579f660 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T18:42:38.793 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.761+0000 7fd7ba389700 1 --2- 172.21.15.39:0/1736051622 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a5578b3f0 0x560a5579f660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:38.793 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.761+0000 7fd7ba389700 1 -- 172.21.15.39:0/1736051622 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7b410c110 con 0x560a5578aae0 2023-07-21T18:42:38.793 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.761+0000 7fd7ba389700 1 --2- 172.21.15.39:0/1736051622 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560a5578aae0 0x560a5579f120 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0009990 tx=0x7fd7b0009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8372a6e7aaca02f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:38.793 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.761+0000 7fd79dffb700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:42:38.793 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.761+0000 7fd79dffb700 1 -- 172.21.15.39:0/1736051622 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b0020680 con 0x560a5578aae0 2023-07-21T18:42:38.793 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.762+0000 7fd79dffb700 1 -- 172.21.15.39:0/1736051622 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fd7b00207e0 con 0x560a5578aae0 2023-07-21T18:42:38.793 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.762+0000 7fd7c8773b80 1 -- 172.21.15.39:0/1736051622 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560a5562e4f0 con 0x560a5578aae0 2023-07-21T18:42:38.793 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.762+0000 7fd79dffb700 1 -- 172.21.15.39:0/1736051622 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7b00209a0 con 0x560a5578aae0 2023-07-21T18:42:38.794 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.762+0000 7fd7c8773b80 10 client.15762 fetch_fsmap learned FSMap version 26 2023-07-21T18:42:38.794 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.762+0000 7fd7c8773b80 1 -- 172.21.15.39:0/1736051622 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7fd7a4009980 con 0x560a5578aae0 2023-07-21T18:42:38.794 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.762+0000 7fd79dffb700 1 -- 172.21.15.39:0/1736051622 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560a5562e4f0 con 0x560a5578aae0 2023-07-21T18:42:38.794 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.762+0000 7fd79dffb700 1 -- 172.21.15.39:0/1736051622 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd7b0015c50 con 0x560a5578aae0 2023-07-21T18:42:38.794 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.762+0000 7fd7c8773b80 10 client.15762 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T18:42:38.794 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.762+0000 7fd7c8773b80 10 client.15762 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T18:42:38.794 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.762+0000 7fd7c8773b80 20 client.15762 populate_metadata read hostname 'smithi039' 2023-07-21T18:42:38.795 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.762+0000 7fd7c8773b80 1 --2- 172.21.15.39:0/1736051622 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x560a557ae5c0 0x560a557b09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:38.795 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.762+0000 7fd7c8773b80 4 client.15762 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:42:38.795 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.762+0000 7fd7c8773b80 1 -- 172.21.15.39:0/1736051622 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560a556cdeb0 con 0x560a557ae5c0 2023-07-21T18:42:38.795 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.763+0000 7fd7b9b88700 1 --2- 172.21.15.39:0/1736051622 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x560a557ae5c0 0x560a557b09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:38.795 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.763+0000 7fd7b9b88700 1 --2- 172.21.15.39:0/1736051622 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x560a557ae5c0 0x560a557b09d0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cc20c58fc6a720f server_cookie=58fdfef5b2e4f6fe in_seq=0 out_seq=0 2023-07-21T18:42:38.795 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.763+0000 7fd79dffb700 10 client.15762 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:42:38.795 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.764+0000 7fd79dffb700 1 -- 172.21.15.39:0/1736051622 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560a556cdeb0 con 0x560a557ae5c0 2023-07-21T18:42:38.795 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.764+0000 7fd79dffb700 10 client.15762 handle_command_reply: tid=0 2023-07-21T18:42:38.796 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.764+0000 7fd7c8773b80 1 client.15762 shutdown 2023-07-21T18:42:38.796 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.764+0000 7fd7c8773b80 1 -- 172.21.15.39:0/1736051622 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560a5578aae0 msgr2=0x560a5579f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:38.796 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.764+0000 7fd7c8773b80 1 --2- 172.21.15.39:0/1736051622 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560a5578aae0 0x560a5579f120 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0009990 tx=0x7fd7b0009310 comp rx=0 tx=0).stop 2023-07-21T18:42:38.796 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.764+0000 7fd7c8773b80 1 -- 172.21.15.39:0/1736051622 shutdown_connections 2023-07-21T18:42:38.796 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.764+0000 7fd7c8773b80 1 --2- 172.21.15.39:0/1736051622 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x560a557ae5c0 0x560a557b09d0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:38.796 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.764+0000 7fd7c8773b80 1 --2- 172.21.15.39:0/1736051622 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a5579ed50 0x560a557aa1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:38.796 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.764+0000 7fd7c8773b80 1 --2- 172.21.15.39:0/1736051622 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560a5578aae0 0x560a5579f120 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:38.797 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.764+0000 7fd7c8773b80 1 --2- 172.21.15.39:0/1736051622 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a5578b3f0 0x560a5579f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:38.797 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.764+0000 7fd7c8773b80 1 -- 172.21.15.39:0/1736051622 >> 172.21.15.39:0/1736051622 conn(0x560a55778da0 msgr2=0x560a5577e290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:38.797 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.764+0000 7fd79dffb700 0 client.15762 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:42:38.797 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.764+0000 7fd7c8773b80 1 -- 172.21.15.39:0/1736051622 shutdown_connections 2023-07-21T18:42:38.797 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.764+0000 7fd7c8773b80 1 -- 172.21.15.39:0/1736051622 wait complete. 2023-07-21T18:42:38.797 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.764+0000 7fd7c8773b80 20 client.15762 trim_cache size 0 max 16384 2023-07-21T18:42:38.819 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.797+0000 7fd7c8773b80 1 Processor -- start 2023-07-21T18:42:38.819 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.797+0000 7fd7c8773b80 1 -- start start 2023-07-21T18:42:38.819 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.797+0000 7fd7c8773b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560a5578aae0 0x560a557a4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:38.819 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.797+0000 7fd7c8773b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a5578b3f0 0x560a55782bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:38.820 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.797+0000 7fd7c8773b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a5579ed50 0x560a55783130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:38.820 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.797+0000 7fd7c8773b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a5578c570 con 0x560a5578aae0 2023-07-21T18:42:38.820 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.797+0000 7fd7c8773b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a5578b7c0 con 0x560a5579ed50 2023-07-21T18:42:38.820 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.797+0000 7fd7c8773b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a55787f30 con 0x560a5578b3f0 2023-07-21T18:42:38.822 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.798+0000 7fd7ba389700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560a5578aae0 0x560a557a4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:38.822 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.798+0000 7fd7ba389700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560a5578aae0 0x560a557a4120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57980/0 (socket says 172.21.15.39:57980) 2023-07-21T18:42:38.822 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.798+0000 7fd7ba389700 1 -- 172.21.15.39:0/3505695791 learned_addr learned my addr 172.21.15.39:0/3505695791 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:38.822 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.798+0000 7fd7bab8a700 1 --2- 172.21.15.39:0/3505695791 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a5579ed50 0x560a55783130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:38.822 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.798+0000 7fd7ba389700 1 -- 172.21.15.39:0/3505695791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a5578b3f0 msgr2=0x560a55782bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:38.822 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.798+0000 7fd7ba389700 1 --2- 172.21.15.39:0/3505695791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a5578b3f0 0x560a55782bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:38.822 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.798+0000 7fd7ba389700 1 -- 172.21.15.39:0/3505695791 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a5579ed50 msgr2=0x560a55783130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:38.823 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.798+0000 7fd7ba389700 1 --2- 172.21.15.39:0/3505695791 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a5579ed50 0x560a55783130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:38.823 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.798+0000 7fd7ba389700 1 -- 172.21.15.39:0/3505695791 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a5577a5e0 con 0x560a5578aae0 2023-07-21T18:42:38.823 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.798+0000 7fd7bab8a700 1 --2- 172.21.15.39:0/3505695791 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a5579ed50 0x560a55783130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:42:38.823 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.798+0000 7fd7ba389700 1 --2- 172.21.15.39:0/3505695791 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560a5578aae0 0x560a557a4120 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0000c00 tx=0x7fd7b0005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca514ebb16cdd312 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:38.823 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.798+0000 7fd7a97fa700 1 -- 172.21.15.39:0/3505695791 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b002dac0 con 0x560a5578aae0 2023-07-21T18:42:38.823 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.799+0000 7fd7a97fa700 1 -- 172.21.15.39:0/3505695791 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7b0025030 con 0x560a5578aae0 2023-07-21T18:42:38.823 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.799+0000 7fd7a97fa700 1 -- 172.21.15.39:0/3505695791 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b00156b0 con 0x560a5578aae0 2023-07-21T18:42:38.824 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.799+0000 7fd7c8773b80 1 -- 172.21.15.39:0/3505695791 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560a5578aae0 msgr2=0x560a557a4120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:38.824 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.799+0000 7fd7c8773b80 1 --2- 172.21.15.39:0/3505695791 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560a5578aae0 0x560a557a4120 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0000c00 tx=0x7fd7b0005740 comp rx=0 tx=0).stop 2023-07-21T18:42:38.824 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.800+0000 7fd7c8773b80 1 -- 172.21.15.39:0/3505695791 shutdown_connections 2023-07-21T18:42:38.824 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.800+0000 7fd7c8773b80 1 --2- 172.21.15.39:0/3505695791 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a5579ed50 0x560a55783130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:38.824 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.800+0000 7fd7c8773b80 1 --2- 172.21.15.39:0/3505695791 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560a5578aae0 0x560a557a4120 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:38.824 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.800+0000 7fd7c8773b80 1 --2- 172.21.15.39:0/3505695791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a5578b3f0 0x560a55782bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:38.824 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.800+0000 7fd7c8773b80 1 -- 172.21.15.39:0/3505695791 >> 172.21.15.39:0/3505695791 conn(0x560a55778da0 msgr2=0x560a55787150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:38.824 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.800+0000 7fd7c8773b80 1 -- 172.21.15.39:0/3505695791 shutdown_connections 2023-07-21T18:42:38.825 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.800+0000 7fd7c8773b80 1 -- 172.21.15.39:0/3505695791 wait complete. 2023-07-21T18:42:38.825 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.800+0000 7fd7c8773b80 1 -- start start 2023-07-21T18:42:38.825 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.800+0000 7fd7c8773b80 1 Processor -- start 2023-07-21T18:42:38.827 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.800+0000 7fd7c8773b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a5578aae0 0x560a557a0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:38.827 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.800+0000 7fd7c8773b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560a5578b3f0 0x560a557a0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:38.827 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.801+0000 7fd7c8773b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a5579ed50 0x560a55801470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:38.827 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.801+0000 7fd7c8773b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a558019b0 con 0x560a5578b3f0 2023-07-21T18:42:38.828 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.801+0000 7fd7c8773b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a55801af0 con 0x560a5578aae0 2023-07-21T18:42:38.828 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.801+0000 7fd7c8773b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a55801c30 con 0x560a5579ed50 2023-07-21T18:42:38.828 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.801+0000 7fd7c8773b80 10 client.0 _ll_register_callbacks cb 0x7fd7b000dce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:42:38.828 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.801+0000 7fd7ba389700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a5578aae0 0x560a557a0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:38.828 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.801+0000 7fd7b9b88700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560a5578b3f0 0x560a557a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:38.828 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.801+0000 7fd7ba389700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a5578aae0 0x560a557a0230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:42350/0 (socket says 172.21.15.39:42350) 2023-07-21T18:42:38.828 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.801+0000 7fd7ba389700 1 -- 172.21.15.39:0/831268618 learned_addr learned my addr 172.21.15.39:0/831268618 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:38.829 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.801+0000 7fd7bab8a700 1 --2- 172.21.15.39:0/831268618 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a5579ed50 0x560a55801470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:38.829 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.801+0000 7fd7ba389700 1 -- 172.21.15.39:0/831268618 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a5579ed50 msgr2=0x560a55801470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:38.829 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.801+0000 7fd7ba389700 1 --2- 172.21.15.39:0/831268618 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a5579ed50 0x560a55801470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:38.829 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.801+0000 7fd7ba389700 1 -- 172.21.15.39:0/831268618 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560a5578b3f0 msgr2=0x560a557a0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:38.829 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.801+0000 7fd7ba389700 1 --2- 172.21.15.39:0/831268618 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560a5578b3f0 0x560a557a0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:38.829 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.801+0000 7fd7ba389700 1 -- 172.21.15.39:0/831268618 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7ac082e70 con 0x560a5578aae0 2023-07-21T18:42:38.829 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.802+0000 7fd7ba389700 1 --2- 172.21.15.39:0/831268618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a5578aae0 0x560a557a0230 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0009550 tx=0x7fd7b0005c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9161a4d3fd9481f3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:38.829 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.802+0000 7fd79dffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-21T18:42:38.830 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.802+0000 7fd79dffb700 1 -- 172.21.15.39:0/831268618 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b002dc80 con 0x560a5578aae0 2023-07-21T18:42:38.830 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.802+0000 7fd79dffb700 1 -- 172.21.15.39:0/831268618 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fd7b002dde0 con 0x560a5578aae0 2023-07-21T18:42:38.830 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.802+0000 7fd7c8773b80 1 -- 172.21.15.39:0/831268618 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd7a4019c50 con 0x560a5578aae0 2023-07-21T18:42:38.830 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.802+0000 7fd79dffb700 1 -- 172.21.15.39:0/831268618 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7b0015450 con 0x560a5578aae0 2023-07-21T18:42:38.830 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.802+0000 7fd7c8773b80 10 client.25358 fetch_fsmap learned FSMap version 26 2023-07-21T18:42:38.830 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.802+0000 7fd7c8773b80 1 -- 172.21.15.39:0/831268618 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7fd7a4009980 con 0x560a5578aae0 2023-07-21T18:42:38.830 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.802+0000 7fd79dffb700 1 -- 172.21.15.39:0/831268618 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7a4019c50 con 0x560a5578aae0 2023-07-21T18:42:38.831 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.802+0000 7fd79dffb700 1 -- 172.21.15.39:0/831268618 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd7b003f8f0 con 0x560a5578aae0 2023-07-21T18:42:38.831 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.803+0000 7fd7c8773b80 10 client.25358 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T18:42:38.831 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.803+0000 7fd7c8773b80 10 client.25358 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T18:42:38.831 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.803+0000 7fd7c8773b80 20 client.25358 populate_metadata read hostname 'smithi039' 2023-07-21T18:42:38.831 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.803+0000 7fd7c8773b80 1 --2- 172.21.15.39:0/831268618 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x560a55802ec0 0x560a558052d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:38.831 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.803+0000 7fd7c8773b80 4 client.25358 mds_command: new command op to 24536 tid=0[{"prefix": "scrub status"}] 2023-07-21T18:42:38.831 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.803+0000 7fd7c8773b80 1 -- 172.21.15.39:0/831268618 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560a55690050 con 0x560a55802ec0 2023-07-21T18:42:38.831 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.803+0000 7fd7b9b88700 1 --2- 172.21.15.39:0/831268618 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x560a55802ec0 0x560a558052d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:38.832 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.803+0000 7fd79dffb700 10 client.25358 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:42:38.832 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.803+0000 7fd7b9b88700 1 --2- 172.21.15.39:0/831268618 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x560a55802ec0 0x560a558052d0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=850fb4127fb1d8e1 server_cookie=1d21f2c04b090cae in_seq=0 out_seq=0 2023-07-21T18:42:38.832 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.804+0000 7fd79dffb700 1 -- 172.21.15.39:0/831268618 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x560a55690050 con 0x560a55802ec0 2023-07-21T18:42:38.832 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.804+0000 7fd79dffb700 10 client.25358 handle_command_reply: tid=0 2023-07-21T18:42:38.832 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.804+0000 7fd7c8773b80 1 client.25358 shutdown 2023-07-21T18:42:38.832 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.804+0000 7fd7c8773b80 1 -- 172.21.15.39:0/831268618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a5578aae0 msgr2=0x560a557a0230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:38.832 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.804+0000 7fd7c8773b80 1 --2- 172.21.15.39:0/831268618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a5578aae0 0x560a557a0230 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0009550 tx=0x7fd7b0005c80 comp rx=0 tx=0).stop 2023-07-21T18:42:38.833 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.804+0000 7fd7c8773b80 1 -- 172.21.15.39:0/831268618 shutdown_connections 2023-07-21T18:42:38.833 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.804+0000 7fd7c8773b80 1 --2- 172.21.15.39:0/831268618 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x560a55802ec0 0x560a558052d0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:38.833 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.804+0000 7fd7c8773b80 1 --2- 172.21.15.39:0/831268618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a5578aae0 0x560a557a0230 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:38.833 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.804+0000 7fd7c8773b80 1 --2- 172.21.15.39:0/831268618 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a5579ed50 0x560a55801470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:38.833 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.804+0000 7fd7c8773b80 1 --2- 172.21.15.39:0/831268618 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560a5578b3f0 0x560a557a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:38.833 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.804+0000 7fd7c8773b80 1 -- 172.21.15.39:0/831268618 >> 172.21.15.39:0/831268618 conn(0x560a55778da0 msgr2=0x560a55782bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:38.833 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.804+0000 7fd79dffb700 0 client.25358 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:42:38.833 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.804+0000 7fd7c8773b80 1 -- 172.21.15.39:0/831268618 shutdown_connections 2023-07-21T18:42:38.834 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.804+0000 7fd7c8773b80 1 -- 172.21.15.39:0/831268618 wait complete. 2023-07-21T18:42:38.834 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.804+0000 7fd7c8773b80 20 client.25358 trim_cache size 0 max 16384 2023-07-21T18:42:38.834 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2023-07-21T18:42:38.834 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "no active scrubs running", 2023-07-21T18:42:38.834 INFO:teuthology.orchestra.run.smithi039.stdout: "scrubs": {} 2023-07-21T18:42:38.834 INFO:teuthology.orchestra.run.smithi039.stdout:} 2023-07-21T18:42:38.835 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.805+0000 7fd7bc5ed700 1 -- 172.21.15.39:0/1797902849 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fd7a00a1460 msgr2=0x7fd7a00a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:38.835 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.805+0000 7fd7bc5ed700 1 --2- 172.21.15.39:0/1797902849 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fd7a00a1460 0x7fd7a00a3910 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd7b00058e0 tx=0x7fd7b0005fd0 comp rx=0 tx=0).stop 2023-07-21T18:42:38.835 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.805+0000 7fd7bc5ed700 1 -- 172.21.15.39:0/1797902849 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7b41049e0 msgr2=0x7fd7b419c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:38.836 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.805+0000 7fd7bc5ed700 1 --2- 172.21.15.39:0/1797902849 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7b41049e0 0x7fd7b419c870 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac007ee0 tx=0x7fd7ac00d3b0 comp rx=0 tx=0).stop 2023-07-21T18:42:38.836 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.805+0000 7fd7bc5ed700 1 -- 172.21.15.39:0/1797902849 shutdown_connections 2023-07-21T18:42:38.836 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.805+0000 7fd7bc5ed700 1 --2- 172.21.15.39:0/1797902849 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7b41049e0 0x7fd7b419c870 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:38.836 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.805+0000 7fd7bc5ed700 1 --2- 172.21.15.39:0/1797902849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd7b40fe9d0 0x7fd7b4197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:38.836 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.805+0000 7fd7bc5ed700 1 --2- 172.21.15.39:0/1797902849 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fd7a00a1460 0x7fd7a00a3910 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:38.836 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.805+0000 7fd7bc5ed700 1 --2- 172.21.15.39:0/1797902849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7b40ff380 0x7fd7b4198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:38.836 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.805+0000 7fd7bc5ed700 1 -- 172.21.15.39:0/1797902849 >> 172.21.15.39:0/1797902849 conn(0x7fd7b40fa4b0 msgr2=0x7fd7b40fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:38.836 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.805+0000 7fd7bc5ed700 1 -- 172.21.15.39:0/1797902849 shutdown_connections 2023-07-21T18:42:38.837 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:38.805+0000 7fd7bc5ed700 1 -- 172.21.15.39:0/1797902849 wait complete. 2023-07-21T18:42:38.843 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-21T18:42:38.843 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T18:42:39.036 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.014+0000 7f33e130e700 1 -- 172.21.15.39:0/1940307927 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33dc1093b0 msgr2=0x7f33dc10b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:39.037 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.014+0000 7f33e130e700 1 --2- 172.21.15.39:0/1940307927 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33dc1093b0 0x7f33dc10b790 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f33d400b0a0 tx=0x7f33d400b3b0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.037 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.014+0000 7f33e130e700 1 -- 172.21.15.39:0/1940307927 shutdown_connections 2023-07-21T18:42:39.037 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.014+0000 7f33e130e700 1 --2- 172.21.15.39:0/1940307927 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33dc1093b0 0x7f33dc10b790 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.037 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.014+0000 7f33e130e700 1 --2- 172.21.15.39:0/1940307927 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33dc0ffe00 0x7f33dc1001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.037 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.014+0000 7f33e130e700 1 --2- 172.21.15.39:0/1940307927 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33dc100710 0x7f33dc108e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.037 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.014+0000 7f33e130e700 1 -- 172.21.15.39:0/1940307927 >> 172.21.15.39:0/1940307927 conn(0x7f33dc075240 msgr2=0x7f33dc075640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:39.037 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.015+0000 7f33e130e700 1 -- 172.21.15.39:0/1940307927 shutdown_connections 2023-07-21T18:42:39.038 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.015+0000 7f33e130e700 1 -- 172.21.15.39:0/1940307927 wait complete. 2023-07-21T18:42:39.039 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.015+0000 7f33e130e700 1 Processor -- start 2023-07-21T18:42:39.040 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.016+0000 7f33e130e700 1 -- start start 2023-07-21T18:42:39.040 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.016+0000 7f33e130e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33dc0ffe00 0x7f33dc193980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:39.040 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.016+0000 7f33e130e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33dc100710 0x7f33dc193ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:39.041 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.016+0000 7f33e130e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33dc1093b0 0x7f33dc198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:39.041 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.016+0000 7f33e130e700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33dc198920 con 0x7f33dc0ffe00 2023-07-21T18:42:39.041 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.016+0000 7f33e130e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33dc198a90 con 0x7f33dc100710 2023-07-21T18:42:39.041 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.016+0000 7f33e130e700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33dc198c50 con 0x7f33dc1093b0 2023-07-21T18:42:39.042 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.016+0000 7f33daffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33dc0ffe00 0x7f33dc193980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:39.042 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.016+0000 7f33daffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33dc0ffe00 0x7f33dc193980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58000/0 (socket says 172.21.15.39:58000) 2023-07-21T18:42:39.042 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.016+0000 7f33daffd700 1 -- 172.21.15.39:0/3923885066 learned_addr learned my addr 172.21.15.39:0/3923885066 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:39.042 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.016+0000 7f33db7fe700 1 --2- 172.21.15.39:0/3923885066 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33dc1093b0 0x7f33dc198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:39.043 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.017+0000 7f33daffd700 1 -- 172.21.15.39:0/3923885066 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33dc1093b0 msgr2=0x7f33dc198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:39.043 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.017+0000 7f33daffd700 1 --2- 172.21.15.39:0/3923885066 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33dc1093b0 0x7f33dc198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.043 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.017+0000 7f33daffd700 1 -- 172.21.15.39:0/3923885066 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33dc100710 msgr2=0x7f33dc193ec0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-21T18:42:39.044 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.017+0000 7f33daffd700 1 --2- 172.21.15.39:0/3923885066 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33dc100710 0x7f33dc193ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.044 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.017+0000 7f33daffd700 1 -- 172.21.15.39:0/3923885066 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33d4009d00 con 0x7f33dc0ffe00 2023-07-21T18:42:39.044 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.017+0000 7f33daffd700 1 --2- 172.21.15.39:0/3923885066 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33dc0ffe00 0x7f33dc193980 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f33cc00c920 tx=0x7f33cc00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:39.044 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.017+0000 7f33c3fff700 1 -- 172.21.15.39:0/3923885066 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33cc007b00 con 0x7f33dc0ffe00 2023-07-21T18:42:39.044 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.017+0000 7f33e130e700 1 -- 172.21.15.39:0/3923885066 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33dc198f30 con 0x7f33dc0ffe00 2023-07-21T18:42:39.044 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.017+0000 7f33c3fff700 1 -- 172.21.15.39:0/3923885066 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f33cc00ce70 con 0x7f33dc0ffe00 2023-07-21T18:42:39.045 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.017+0000 7f33c3fff700 1 -- 172.21.15.39:0/3923885066 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33cc005460 con 0x7f33dc0ffe00 2023-07-21T18:42:39.045 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.017+0000 7f33e130e700 1 -- 172.21.15.39:0/3923885066 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33dc1a07e0 con 0x7f33dc0ffe00 2023-07-21T18:42:39.045 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.018+0000 7f33e130e700 1 -- 172.21.15.39:0/3923885066 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33dc103b90 con 0x7f33dc0ffe00 2023-07-21T18:42:39.046 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.020+0000 7f33c3fff700 1 -- 172.21.15.39:0/3923885066 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f33cc0055c0 con 0x7f33dc0ffe00 2023-07-21T18:42:39.046 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.021+0000 7f33c3fff700 1 --2- 172.21.15.39:0/3923885066 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f33c40a1430 0x7f33c40a38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:39.046 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.021+0000 7f33c3fff700 1 -- 172.21.15.39:0/3923885066 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f33cc0ccbe0 con 0x7f33dc0ffe00 2023-07-21T18:42:39.046 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.022+0000 7f33da7fc700 1 --2- 172.21.15.39:0/3923885066 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f33c40a1430 0x7f33c40a38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:39.046 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.022+0000 7f33da7fc700 1 --2- 172.21.15.39:0/3923885066 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f33c40a1430 0x7f33c40a38e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f33dc194f90 tx=0x7f33d0009260 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:39.049 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.028+0000 7f33c3fff700 1 -- 172.21.15.39:0/3923885066 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f33cc095dc0 con 0x7f33dc0ffe00 2023-07-21T18:42:39.511 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.489+0000 7f51f1c3f700 1 -- 172.21.15.39:0/2761972274 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51ec0ff3e0 msgr2=0x7f51ec10b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:39.511 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.489+0000 7f51f1c3f700 1 --2- 172.21.15.39:0/2761972274 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51ec0ff3e0 0x7f51ec10b860 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f51e400b0a0 tx=0x7f51e400b3b0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.511 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.489+0000 7f51f1c3f700 1 -- 172.21.15.39:0/2761972274 shutdown_connections 2023-07-21T18:42:39.512 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.489+0000 7f51f1c3f700 1 --2- 172.21.15.39:0/2761972274 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51ec0ff3e0 0x7f51ec10b860 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.512 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.489+0000 7f51f1c3f700 1 --2- 172.21.15.39:0/2761972274 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f51ec104a30 0x7f51ec104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.512 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.489+0000 7f51f1c3f700 1 --2- 172.21.15.39:0/2761972274 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f51ec0fea30 0x7f51ec0feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.512 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.489+0000 7f51f1c3f700 1 -- 172.21.15.39:0/2761972274 >> 172.21.15.39:0/2761972274 conn(0x7f51ec074760 msgr2=0x7f51ec074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:39.512 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.489+0000 7f51f1c3f700 1 -- 172.21.15.39:0/2761972274 shutdown_connections 2023-07-21T18:42:39.512 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.489+0000 7f51f1c3f700 1 -- 172.21.15.39:0/2761972274 wait complete. 2023-07-21T18:42:39.513 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.490+0000 7f51f1c3f700 1 Processor -- start 2023-07-21T18:42:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.490+0000 7f51f1c3f700 1 -- start start 2023-07-21T18:42:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.491+0000 7f51f1c3f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51ec0fea30 0x7f51ec19bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.491+0000 7f51f1c3f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f51ec0ff3e0 0x7f51ec19c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.491+0000 7f51f1c3f700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f51ec104a30 0x7f51ec195a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.491+0000 7f51f1c3f700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ec196070 con 0x7f51ec0fea30 2023-07-21T18:42:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.491+0000 7f51f1c3f700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ec1961e0 con 0x7f51ec0ff3e0 2023-07-21T18:42:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.491+0000 7f51f1c3f700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ec1963a0 con 0x7f51ec104a30 2023-07-21T18:42:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.491+0000 7f51eb7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51ec0fea30 0x7f51ec19bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.491+0000 7f51eb7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51ec0fea30 0x7f51ec19bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58014/0 (socket says 172.21.15.39:58014) 2023-07-21T18:42:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.491+0000 7f51eb7fe700 1 -- 172.21.15.39:0/2496347582 learned_addr learned my addr 172.21.15.39:0/2496347582 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.491+0000 7f51eb7fe700 1 -- 172.21.15.39:0/2496347582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f51ec104a30 msgr2=0x7f51ec195a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.491+0000 7f51eb7fe700 1 --2- 172.21.15.39:0/2496347582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f51ec104a30 0x7f51ec195a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.491+0000 7f51eb7fe700 1 -- 172.21.15.39:0/2496347582 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f51ec0ff3e0 msgr2=0x7f51ec19c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.491+0000 7f51eb7fe700 1 --2- 172.21.15.39:0/2496347582 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f51ec0ff3e0 0x7f51ec19c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.491+0000 7f51eb7fe700 1 -- 172.21.15.39:0/2496347582 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51e4009d00 con 0x7f51ec0fea30 2023-07-21T18:42:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.492+0000 7f51eb7fe700 1 --2- 172.21.15.39:0/2496347582 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51ec0fea30 0x7f51ec19bba0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f51d400b550 tx=0x7f51d400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.492+0000 7f51e8ff9700 1 -- 172.21.15.39:0/2496347582 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d4011830 con 0x7f51ec0fea30 2023-07-21T18:42:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.492+0000 7f51e8ff9700 1 -- 172.21.15.39:0/2496347582 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51d4011e50 con 0x7f51ec0fea30 2023-07-21T18:42:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.492+0000 7f51e8ff9700 1 -- 172.21.15.39:0/2496347582 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d400f2e0 con 0x7f51ec0fea30 2023-07-21T18:42:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.492+0000 7f51f1c3f700 1 -- 172.21.15.39:0/2496347582 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51ec196680 con 0x7f51ec0fea30 2023-07-21T18:42:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.492+0000 7f51f1c3f700 1 -- 172.21.15.39:0/2496347582 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51ec06a190 con 0x7f51ec0fea30 2023-07-21T18:42:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.496+0000 7f51e8ff9700 1 -- 172.21.15.39:0/2496347582 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f51d4010580 con 0x7f51ec0fea30 2023-07-21T18:42:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.497+0000 7f51fddc5b80 1 Processor -- start 2023-07-21T18:42:39.519 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.497+0000 7f51fddc5b80 1 -- start start 2023-07-21T18:42:39.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.497+0000 7f51e8ff9700 1 --2- 172.21.15.39:0/2496347582 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f51d80a13e0 0x7f51d80a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:39.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.497+0000 7f51fddc5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa6ff256b0 0x55aa700ced90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:39.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.497+0000 7f51fddc5b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aa700cf2d0 0x55aa700c9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:39.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.497+0000 7f51fddc5b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa700c9580 0x55aa700c99d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:39.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.497+0000 7f51fddc5b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa6ffcec50 con 0x55aa700c9580 2023-07-21T18:42:39.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.497+0000 7f51fddc5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa700219e0 con 0x55aa6ff256b0 2023-07-21T18:42:39.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.497+0000 7f51fddc5b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa7001ae70 con 0x55aa700cf2d0 2023-07-21T18:42:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.497+0000 7f51e8ff9700 1 -- 172.21.15.39:0/2496347582 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f51d40cc950 con 0x7f51ec0fea30 2023-07-21T18:42:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.498+0000 7f51ebfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa700c9580 0x55aa700c99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.498+0000 7f51ebfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa700c9580 0x55aa700c99d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58024/0 (socket says 172.21.15.39:58024) 2023-07-21T18:42:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.498+0000 7f51ebfff700 1 -- 172.21.15.39:0/1269783950 learned_addr learned my addr 172.21.15.39:0/1269783950 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.498+0000 7f51eaffd700 1 --2- 172.21.15.39:0/1269783950 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aa700cf2d0 0x55aa700c9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.498+0000 7f51ebfff700 1 --2- 172.21.15.39:0/2496347582 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f51d80a13e0 0x7f51d80a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.498+0000 7f51eaffd700 1 -- 172.21.15.39:0/1269783950 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa6ff256b0 msgr2=0x55aa700ced90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T18:42:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.498+0000 7f51eaffd700 1 --2- 172.21.15.39:0/1269783950 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa6ff256b0 0x55aa700ced90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.498+0000 7f51eaffd700 1 -- 172.21.15.39:0/1269783950 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa700c9580 msgr2=0x55aa700c99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.498+0000 7f51eaffd700 1 --2- 172.21.15.39:0/1269783950 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa700c9580 0x55aa700c99d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.523 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.498+0000 7f51eaffd700 1 -- 172.21.15.39:0/1269783950 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa6fe48d80 con 0x55aa700cf2d0 2023-07-21T18:42:39.523 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.498+0000 7f51ebfff700 1 --2- 172.21.15.39:0/2496347582 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f51d80a13e0 0x7f51d80a3890 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f51e4009cd0 tx=0x7f51e4009570 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:39.523 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.498+0000 7f51eaffd700 1 --2- 172.21.15.39:0/1269783950 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aa700cf2d0 0x55aa700c9040 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f51dc009ba0 tx=0x7f51dc009eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb953fd6f8d33cef server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:39.523 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.500+0000 7f51e1ffb700 1 -- 172.21.15.39:0/1269783950 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51dc004230 con 0x55aa700cf2d0 2023-07-21T18:42:39.523 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.500+0000 7f51e1ffb700 1 -- 172.21.15.39:0/1269783950 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51dc00ed70 con 0x55aa700cf2d0 2023-07-21T18:42:39.523 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.500+0000 7f51e1ffb700 1 -- 172.21.15.39:0/1269783950 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51dc005020 con 0x55aa700cf2d0 2023-07-21T18:42:39.524 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.501+0000 7f51fddc5b80 1 -- 172.21.15.39:0/1269783950 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aa700cf2d0 msgr2=0x55aa700c9040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.501+0000 7f51fddc5b80 1 --2- 172.21.15.39:0/1269783950 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aa700cf2d0 0x55aa700c9040 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f51dc009ba0 tx=0x7f51dc009eb0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.501+0000 7f51fddc5b80 1 -- 172.21.15.39:0/1269783950 shutdown_connections 2023-07-21T18:42:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.501+0000 7f51fddc5b80 1 --2- 172.21.15.39:0/1269783950 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa6ff256b0 0x55aa700ced90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.501+0000 7f51fddc5b80 1 --2- 172.21.15.39:0/1269783950 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa700c9580 0x55aa700c99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.501+0000 7f51fddc5b80 1 --2- 172.21.15.39:0/1269783950 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aa700cf2d0 0x55aa700c9040 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.501+0000 7f51fddc5b80 1 -- 172.21.15.39:0/1269783950 >> 172.21.15.39:0/1269783950 conn(0x55aa700bebe0 msgr2=0x55aa700c3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.501+0000 7f51fddc5b80 1 -- 172.21.15.39:0/1269783950 shutdown_connections 2023-07-21T18:42:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.501+0000 7f51fddc5b80 1 -- 172.21.15.39:0/1269783950 wait complete. 2023-07-21T18:42:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.501+0000 7f51fddc5b80 1 -- start start 2023-07-21T18:42:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.502+0000 7f51fddc5b80 1 Processor -- start 2023-07-21T18:42:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.502+0000 7f51fddc5b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa6ff256b0 0x55aa700d29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.502+0000 7f51fddc5b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aa700c9580 0x55aa700d2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.502+0000 7f51fddc5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa700cf2d0 0x55aa700d3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.502+0000 7f51fddc5b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa6ffbaa60 con 0x55aa6ff256b0 2023-07-21T18:42:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.502+0000 7f51fddc5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa6ffba760 con 0x55aa700cf2d0 2023-07-21T18:42:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.502+0000 7f51fddc5b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa6fe9e8c0 con 0x55aa700c9580 2023-07-21T18:42:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.502+0000 7f51fddc5b80 10 client.0 _ll_register_callbacks cb 0x55aa6ffb8700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:42:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.503+0000 7f51ebfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa700cf2d0 0x55aa700d3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.503+0000 7f51ebfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa700cf2d0 0x55aa700d3440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:42394/0 (socket says 172.21.15.39:42394) 2023-07-21T18:42:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.503+0000 7f51ebfff700 1 -- 172.21.15.39:0/2366051837 learned_addr learned my addr 172.21.15.39:0/2366051837 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.503+0000 7f51eb7fe700 1 --2- 172.21.15.39:0/2366051837 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa6ff256b0 0x55aa700d29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:39.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.503+0000 7f51ebfff700 1 -- 172.21.15.39:0/2366051837 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aa700c9580 msgr2=0x55aa700d2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:39.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.503+0000 7f51ebfff700 1 --2- 172.21.15.39:0/2366051837 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aa700c9580 0x55aa700d2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.503+0000 7f51ebfff700 1 -- 172.21.15.39:0/2366051837 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa6ff256b0 msgr2=0x55aa700d29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:39.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.503+0000 7f51ebfff700 1 --2- 172.21.15.39:0/2366051837 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa6ff256b0 0x55aa700d29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.503+0000 7f51ebfff700 1 -- 172.21.15.39:0/2366051837 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51dc009880 con 0x55aa700cf2d0 2023-07-21T18:42:39.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.503+0000 7f51ebfff700 1 --2- 172.21.15.39:0/2366051837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa700cf2d0 0x55aa700d3440 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f51e401e6e0 tx=0x7f51e4009c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5dbb3895e614841a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:39.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.503+0000 7f51caffd700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-21T18:42:39.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.503+0000 7f51caffd700 1 -- 172.21.15.39:0/2366051837 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e400e070 con 0x55aa700cf2d0 2023-07-21T18:42:39.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.504+0000 7f51caffd700 1 -- 172.21.15.39:0/2366051837 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f51e4028430 con 0x55aa700cf2d0 2023-07-21T18:42:39.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.504+0000 7f51fddc5b80 1 -- 172.21.15.39:0/2366051837 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aa6ff74490 con 0x55aa700cf2d0 2023-07-21T18:42:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.504+0000 7f51caffd700 1 -- 172.21.15.39:0/2366051837 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51e4003a00 con 0x55aa700cf2d0 2023-07-21T18:42:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.504+0000 7f51fddc5b80 10 client.25367 fetch_fsmap learned FSMap version 26 2023-07-21T18:42:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.504+0000 7f51fddc5b80 1 -- 172.21.15.39:0/2366051837 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55aa6fe31c00 con 0x55aa700cf2d0 2023-07-21T18:42:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.504+0000 7f51caffd700 1 -- 172.21.15.39:0/2366051837 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55aa6ff74490 con 0x55aa700cf2d0 2023-07-21T18:42:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.504+0000 7f51caffd700 1 -- 172.21.15.39:0/2366051837 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f51e40285f0 con 0x55aa700cf2d0 2023-07-21T18:42:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.505+0000 7f51fddc5b80 10 client.25367 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T18:42:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.505+0000 7f51fddc5b80 10 client.25367 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:42:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.505+0000 7f51fddc5b80 20 client.25367 populate_metadata read hostname 'smithi039' 2023-07-21T18:42:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.505+0000 7f51fddc5b80 1 --2- 172.21.15.39:0/2366051837 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55aa700eb4c0 0x55aa700ed8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.505+0000 7f51fddc5b80 4 client.25367 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:42:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.505+0000 7f51fddc5b80 1 -- 172.21.15.39:0/2366051837 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aa70013e50 con 0x55aa700eb4c0 2023-07-21T18:42:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.506+0000 7f51eb7fe700 1 --2- 172.21.15.39:0/2366051837 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55aa700eb4c0 0x55aa700ed8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.506+0000 7f51caffd700 10 client.25367 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:42:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.506+0000 7f51eb7fe700 1 --2- 172.21.15.39:0/2366051837 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55aa700eb4c0 0x55aa700ed8d0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e27c0a2b7112abe1 server_cookie=bedab584161e16c3 in_seq=0 out_seq=0 2023-07-21T18:42:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.507+0000 7f51caffd700 1 -- 172.21.15.39:0/2366051837 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55aa70013e50 con 0x55aa700eb4c0 2023-07-21T18:42:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.507+0000 7f51caffd700 10 client.25367 handle_command_reply: tid=0 2023-07-21T18:42:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.507+0000 7f51fddc5b80 1 client.25367 shutdown 2023-07-21T18:42:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.507+0000 7f51fddc5b80 1 -- 172.21.15.39:0/2366051837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa700cf2d0 msgr2=0x55aa700d3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.507+0000 7f51fddc5b80 1 --2- 172.21.15.39:0/2366051837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa700cf2d0 0x55aa700d3440 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f51e401e6e0 tx=0x7f51e4009c40 comp rx=0 tx=0).stop 2023-07-21T18:42:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.507+0000 7f51fddc5b80 1 -- 172.21.15.39:0/2366051837 shutdown_connections 2023-07-21T18:42:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.507+0000 7f51fddc5b80 1 --2- 172.21.15.39:0/2366051837 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55aa700eb4c0 0x55aa700ed8d0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.507+0000 7f51fddc5b80 1 --2- 172.21.15.39:0/2366051837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa700cf2d0 0x55aa700d3440 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.507+0000 7f51fddc5b80 1 --2- 172.21.15.39:0/2366051837 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa6ff256b0 0x55aa700d29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.507+0000 7f51fddc5b80 1 --2- 172.21.15.39:0/2366051837 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aa700c9580 0x55aa700d2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.507+0000 7f51fddc5b80 1 -- 172.21.15.39:0/2366051837 >> 172.21.15.39:0/2366051837 conn(0x55aa700bebe0 msgr2=0x55aa700c34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.507+0000 7f51caffd700 0 client.25367 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:42:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.507+0000 7f51fddc5b80 1 -- 172.21.15.39:0/2366051837 shutdown_connections 2023-07-21T18:42:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.507+0000 7f51fddc5b80 1 -- 172.21.15.39:0/2366051837 wait complete. 2023-07-21T18:42:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.508+0000 7f51fddc5b80 20 client.25367 trim_cache size 0 max 16384 2023-07-21T18:42:39.538 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.515+0000 7f33e130e700 1 -- 172.21.15.39:0/3923885066 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f33dc067750 con 0x7f33dc0ffe00 2023-07-21T18:42:39.538 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.516+0000 7f33c3fff700 1 -- 172.21.15.39:0/3923885066 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10887 (secure 0 0 0) 0x7f33cc01d020 con 0x7f33dc0ffe00 2023-07-21T18:42:39.539 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:42:39.539 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/147318681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":147318681},{"type":"v1","addr":"172.21.15.39:6835","nonce":147318681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/1062891760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1062891760},{"type":"v1","addr":"172.21.15.39:6837","nonce":1062891760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T18:40:04.594553+0000","modified":"2023-07-21T18:40:11.989039+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24536,"mds_1":24500,"mds_2":24487,"mds_3":24464,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/3184273349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":3184273349},{"type":"v1","addr":"172.21.15.39:6839","nonce":3184273349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6841/640981220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":640981220},{"type":"v1","addr":"172.21.15.39:6841","nonce":640981220}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3106166804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3106166804},{"type":"v1","addr":"172.21.15.193:6833","nonce":3106166804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24437":{"gid":24437,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/1034755277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1034755277},{"type":"v1","addr":"172.21.15.155:6833","nonce":1034755277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/381560657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":381560657},{"type":"v1","addr":"172.21.15.193:6835","nonce":381560657}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6837/2626761843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2626761843},{"type":"v1","addr":"172.21.15.193:6837","nonce":2626761843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24464":{"gid":24464,"name":"e","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.155:6835/3854856286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3854856286},{"type":"v1","addr":"172.21.15.155:6835","nonce":3854856286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.193:6839/3148488639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3148488639},{"type":"v1","addr":"172.21.15.193:6839","nonce":3148488639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.155:6837/947670056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":947670056},{"type":"v1","addr":"172.21.15.155:6837","nonce":947670056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6839/431690342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":431690342},{"type":"v1","addr":"172.21.15.155:6839","nonce":431690342}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T18:42:39.541 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.519+0000 7f33e130e700 1 -- 172.21.15.39:0/3923885066 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f33c40a1430 msgr2=0x7f33c40a38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:39.541 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.519+0000 7f33e130e700 1 --2- 172.21.15.39:0/3923885066 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f33c40a1430 0x7f33c40a38e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f33dc194f90 tx=0x7f33d0009260 comp rx=0 tx=0).stop 2023-07-21T18:42:39.541 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.519+0000 7f33e130e700 1 -- 172.21.15.39:0/3923885066 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33dc0ffe00 msgr2=0x7f33dc193980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:39.541 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.519+0000 7f33e130e700 1 --2- 172.21.15.39:0/3923885066 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33dc0ffe00 0x7f33dc193980 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f33cc00c920 tx=0x7f33cc00cc30 comp rx=0 tx=0).stop 2023-07-21T18:42:39.542 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.519+0000 7f33e130e700 1 -- 172.21.15.39:0/3923885066 shutdown_connections 2023-07-21T18:42:39.542 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.519+0000 7f33e130e700 1 --2- 172.21.15.39:0/3923885066 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33dc1093b0 0x7f33dc198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.542 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.519+0000 7f33e130e700 1 --2- 172.21.15.39:0/3923885066 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33dc0ffe00 0x7f33dc193980 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.542 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.519+0000 7f33e130e700 1 --2- 172.21.15.39:0/3923885066 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f33c40a1430 0x7f33c40a38e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.542 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.519+0000 7f33e130e700 1 --2- 172.21.15.39:0/3923885066 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33dc100710 0x7f33dc193ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.542 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.519+0000 7f33e130e700 1 -- 172.21.15.39:0/3923885066 >> 172.21.15.39:0/3923885066 conn(0x7f33dc075240 msgr2=0x7f33dc100cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:39.543 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.520+0000 7f33e130e700 1 -- 172.21.15.39:0/3923885066 shutdown_connections 2023-07-21T18:42:39.543 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.520+0000 7f33e130e700 1 -- 172.21.15.39:0/3923885066 wait complete. 2023-07-21T18:42:39.544 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 26 2023-07-21T18:42:39.558 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-07-21T18:42:39.559 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.537+0000 7f51fddc5b80 1 Processor -- start 2023-07-21T18:42:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.537+0000 7f51fddc5b80 1 -- start start 2023-07-21T18:42:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.537+0000 7f51fddc5b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa6ff256b0 0x55aa700c9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.537+0000 7f51fddc5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa700c9580 0x55aa700ce3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.537+0000 7f51fddc5b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aa700cf2d0 0x55aa700d3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.537+0000 7f51fddc5b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa700e7ed0 con 0x55aa6ff256b0 2023-07-21T18:42:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.537+0000 7f51fddc5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa700c4b50 con 0x55aa700c9580 2023-07-21T18:42:39.561 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.537+0000 7f51fddc5b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa700cc7a0 con 0x55aa700cf2d0 2023-07-21T18:42:39.561 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.537+0000 7f51eb7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa6ff256b0 0x55aa700c9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:39.561 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.537+0000 7f51eb7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa6ff256b0 0x55aa700c9040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58036/0 (socket says 172.21.15.39:58036) 2023-07-21T18:42:39.561 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.537+0000 7f51eb7fe700 1 -- 172.21.15.39:0/676192567 learned_addr learned my addr 172.21.15.39:0/676192567 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:39.561 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.537+0000 7f51eaffd700 1 --2- 172.21.15.39:0/676192567 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa700c9580 0x55aa700ce3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:39.561 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.537+0000 7f51ebfff700 1 --2- 172.21.15.39:0/676192567 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aa700cf2d0 0x55aa700d3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:39.561 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.538+0000 7f51eb7fe700 1 -- 172.21.15.39:0/676192567 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aa700cf2d0 msgr2=0x55aa700d3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:39.561 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.538+0000 7f51eb7fe700 1 --2- 172.21.15.39:0/676192567 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aa700cf2d0 0x55aa700d3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.562 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.538+0000 7f51eb7fe700 1 -- 172.21.15.39:0/676192567 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa700c9580 msgr2=0x55aa700ce3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:39.562 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.538+0000 7f51eb7fe700 1 --2- 172.21.15.39:0/676192567 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa700c9580 0x55aa700ce3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.562 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.538+0000 7f51eb7fe700 1 -- 172.21.15.39:0/676192567 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa6fe6f1f0 con 0x55aa6ff256b0 2023-07-21T18:42:39.562 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.538+0000 7f51eb7fe700 1 --2- 172.21.15.39:0/676192567 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa6ff256b0 0x55aa700c9040 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f51d4095e00 tx=0x7f51d4010980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1590eb10642dbc0b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:39.563 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.538+0000 7f51e27fc700 1 -- 172.21.15.39:0/676192567 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d40b1070 con 0x55aa6ff256b0 2023-07-21T18:42:39.564 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.538+0000 7f51e27fc700 1 -- 172.21.15.39:0/676192567 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51d4099900 con 0x55aa6ff256b0 2023-07-21T18:42:39.564 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.538+0000 7f51e27fc700 1 -- 172.21.15.39:0/676192567 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d4098df0 con 0x55aa6ff256b0 2023-07-21T18:42:39.564 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.539+0000 7f51fddc5b80 1 -- 172.21.15.39:0/676192567 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa6ff256b0 msgr2=0x55aa700c9040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:39.564 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.539+0000 7f51fddc5b80 1 --2- 172.21.15.39:0/676192567 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa6ff256b0 0x55aa700c9040 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f51d4095e00 tx=0x7f51d4010980 comp rx=0 tx=0).stop 2023-07-21T18:42:39.564 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.539+0000 7f51fddc5b80 1 -- 172.21.15.39:0/676192567 shutdown_connections 2023-07-21T18:42:39.564 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.539+0000 7f51fddc5b80 1 --2- 172.21.15.39:0/676192567 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aa700cf2d0 0x55aa700d3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.564 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.539+0000 7f51fddc5b80 1 --2- 172.21.15.39:0/676192567 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa6ff256b0 0x55aa700c9040 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.565 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.539+0000 7f51fddc5b80 1 --2- 172.21.15.39:0/676192567 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa700c9580 0x55aa700ce3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.565 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.539+0000 7f51fddc5b80 1 -- 172.21.15.39:0/676192567 >> 172.21.15.39:0/676192567 conn(0x55aa700bebe0 msgr2=0x55aa700d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:39.565 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.541+0000 7f51fddc5b80 1 -- 172.21.15.39:0/676192567 shutdown_connections 2023-07-21T18:42:39.565 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.541+0000 7f51fddc5b80 1 -- 172.21.15.39:0/676192567 wait complete. 2023-07-21T18:42:39.565 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.541+0000 7f51fddc5b80 1 -- start start 2023-07-21T18:42:39.565 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.541+0000 7f51fddc5b80 1 Processor -- start 2023-07-21T18:42:39.565 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.542+0000 7f51fddc5b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa6ff256b0 0x55aa700e0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.542+0000 7f51fddc5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa700c9580 0x55aa700e0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.542+0000 7f51fddc5b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aa700cf2d0 0x55aa701383c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.542+0000 7f51fddc5b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa70138900 con 0x55aa6ff256b0 2023-07-21T18:42:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.542+0000 7f51fddc5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa70138a40 con 0x55aa700c9580 2023-07-21T18:42:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.542+0000 7f51fddc5b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa70138c00 con 0x55aa700cf2d0 2023-07-21T18:42:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.542+0000 7f51fddc5b80 10 client.0 _ll_register_callbacks cb 0x7f51e40077b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:42:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.542+0000 7f51eb7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa6ff256b0 0x55aa700e0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.542+0000 7f51eb7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa6ff256b0 0x55aa700e0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58042/0 (socket says 172.21.15.39:58042) 2023-07-21T18:42:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.542+0000 7f51eb7fe700 1 -- 172.21.15.39:0/4194591893 learned_addr learned my addr 172.21.15.39:0/4194591893 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.543+0000 7f51eb7fe700 1 -- 172.21.15.39:0/4194591893 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aa700cf2d0 msgr2=0x55aa701383c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.543+0000 7f51eb7fe700 1 --2- 172.21.15.39:0/4194591893 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aa700cf2d0 0x55aa701383c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.543+0000 7f51eb7fe700 1 -- 172.21.15.39:0/4194591893 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa700c9580 msgr2=0x55aa700e0bd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T18:42:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.543+0000 7f51eb7fe700 1 --2- 172.21.15.39:0/4194591893 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa700c9580 0x55aa700e0bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.543+0000 7f51eb7fe700 1 -- 172.21.15.39:0/4194591893 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51ec06a190 con 0x55aa6ff256b0 2023-07-21T18:42:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.543+0000 7f51eb7fe700 1 --2- 172.21.15.39:0/4194591893 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa6ff256b0 0x55aa700e0690 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f51d4000c00 tx=0x7f51d40a1e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48e5c8bea826d3f0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.543+0000 7f51caffd700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:42:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.543+0000 7f51caffd700 1 -- 172.21.15.39:0/4194591893 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d40c5070 con 0x55aa6ff256b0 2023-07-21T18:42:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.543+0000 7f51fddc5b80 1 -- 172.21.15.39:0/4194591893 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f51dc00ed70 con 0x55aa6ff256b0 2023-07-21T18:42:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.543+0000 7f51caffd700 1 -- 172.21.15.39:0/4194591893 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f51d4097ca0 con 0x55aa6ff256b0 2023-07-21T18:42:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.544+0000 7f51caffd700 1 -- 172.21.15.39:0/4194591893 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51d40b1070 con 0x55aa6ff256b0 2023-07-21T18:42:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.544+0000 7f51fddc5b80 10 client.15792 fetch_fsmap learned FSMap version 26 2023-07-21T18:42:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.544+0000 7f51fddc5b80 1 -- 172.21.15.39:0/4194591893 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55aa6fe47250 con 0x55aa6ff256b0 2023-07-21T18:42:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.544+0000 7f51caffd700 1 -- 172.21.15.39:0/4194591893 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51dc00ed70 con 0x55aa6ff256b0 2023-07-21T18:42:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.544+0000 7f51caffd700 1 -- 172.21.15.39:0/4194591893 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f51d40c8030 con 0x55aa6ff256b0 2023-07-21T18:42:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.544+0000 7f51fddc5b80 10 client.15792 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T18:42:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.544+0000 7f51fddc5b80 10 client.15792 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:42:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.544+0000 7f51fddc5b80 20 client.15792 populate_metadata read hostname 'smithi039' 2023-07-21T18:42:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.544+0000 7f51fddc5b80 1 --2- 172.21.15.39:0/4194591893 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55aa7013ffc0 0x55aa701423d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.544+0000 7f51fddc5b80 4 client.15792 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:42:39.571 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.544+0000 7f51fddc5b80 1 -- 172.21.15.39:0/4194591893 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55aa6ffd5ff0 con 0x55aa7013ffc0 2023-07-21T18:42:39.571 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.545+0000 7f51eaffd700 1 --2- 172.21.15.39:0/4194591893 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55aa7013ffc0 0x55aa701423d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:39.571 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.545+0000 7f51caffd700 10 client.15792 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:42:39.571 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.545+0000 7f51eaffd700 1 --2- 172.21.15.39:0/4194591893 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55aa7013ffc0 0x55aa701423d0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5bbece0328c2200 server_cookie=be24fd3f644b8316 in_seq=0 out_seq=0 2023-07-21T18:42:39.572 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.549+0000 7f51caffd700 1 -- 172.21.15.39:0/4194591893 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55aa6ffd5ff0 con 0x55aa7013ffc0 2023-07-21T18:42:39.572 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.549+0000 7f51caffd700 10 client.15792 handle_command_reply: tid=0 2023-07-21T18:42:39.572 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.549+0000 7f51fddc5b80 1 client.15792 shutdown 2023-07-21T18:42:39.572 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.550+0000 7f51fddc5b80 1 -- 172.21.15.39:0/4194591893 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa6ff256b0 msgr2=0x55aa700e0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:39.572 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.550+0000 7f51fddc5b80 1 --2- 172.21.15.39:0/4194591893 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa6ff256b0 0x55aa700e0690 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f51d4000c00 tx=0x7f51d40a1e90 comp rx=0 tx=0).stop 2023-07-21T18:42:39.572 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.550+0000 7f51fddc5b80 1 -- 172.21.15.39:0/4194591893 shutdown_connections 2023-07-21T18:42:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.550+0000 7f51fddc5b80 1 --2- 172.21.15.39:0/4194591893 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55aa7013ffc0 0x55aa701423d0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.550+0000 7f51fddc5b80 1 --2- 172.21.15.39:0/4194591893 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aa700cf2d0 0x55aa701383c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.550+0000 7f51fddc5b80 1 --2- 172.21.15.39:0/4194591893 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aa6ff256b0 0x55aa700e0690 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.550+0000 7f51fddc5b80 1 --2- 172.21.15.39:0/4194591893 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa700c9580 0x55aa700e0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.550+0000 7f51fddc5b80 1 -- 172.21.15.39:0/4194591893 >> 172.21.15.39:0/4194591893 conn(0x55aa700bebe0 msgr2=0x55aa700ce240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.550+0000 7f51caffd700 0 client.15792 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:42:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.550+0000 7f51fddc5b80 1 -- 172.21.15.39:0/4194591893 shutdown_connections 2023-07-21T18:42:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.550+0000 7f51fddc5b80 1 -- 172.21.15.39:0/4194591893 wait complete. 2023-07-21T18:42:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.550+0000 7f51fddc5b80 20 client.15792 trim_cache size 0 max 16384 2023-07-21T18:42:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:42:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:42:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:42:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:42:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.551+0000 7f51f1c3f700 1 -- 172.21.15.39:0/2496347582 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f51d80a13e0 msgr2=0x7f51d80a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.551+0000 7f51f1c3f700 1 --2- 172.21.15.39:0/2496347582 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f51d80a13e0 0x7f51d80a3890 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f51e4009cd0 tx=0x7f51e4009570 comp rx=0 tx=0).stop 2023-07-21T18:42:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.551+0000 7f51f1c3f700 1 -- 172.21.15.39:0/2496347582 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51ec0fea30 msgr2=0x7f51ec19bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:39.576 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.551+0000 7f51f1c3f700 1 --2- 172.21.15.39:0/2496347582 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51ec0fea30 0x7f51ec19bba0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f51d400b550 tx=0x7f51d400b910 comp rx=0 tx=0).stop 2023-07-21T18:42:39.576 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.551+0000 7f51f1c3f700 1 -- 172.21.15.39:0/2496347582 shutdown_connections 2023-07-21T18:42:39.576 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.551+0000 7f51f1c3f700 1 --2- 172.21.15.39:0/2496347582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f51ec104a30 0x7f51ec195a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.576 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.551+0000 7f51f1c3f700 1 --2- 172.21.15.39:0/2496347582 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51ec0fea30 0x7f51ec19bba0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.576 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.551+0000 7f51f1c3f700 1 --2- 172.21.15.39:0/2496347582 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f51d80a13e0 0x7f51d80a3890 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.576 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.551+0000 7f51f1c3f700 1 --2- 172.21.15.39:0/2496347582 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f51ec0ff3e0 0x7f51ec19c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.576 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.551+0000 7f51f1c3f700 1 -- 172.21.15.39:0/2496347582 >> 172.21.15.39:0/2496347582 conn(0x7f51ec074760 msgr2=0x7f51ec1058a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:39.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.552+0000 7f51f1c3f700 1 -- 172.21.15.39:0/2496347582 shutdown_connections 2023-07-21T18:42:39.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:39.552+0000 7f51f1c3f700 1 -- 172.21.15.39:0/2496347582 wait complete. 2023-07-21T18:42:39.756 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.734+0000 7f8afac45700 1 -- 172.21.15.39:0/677838106 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8af41049e0 msgr2=0x7f8af4104db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:39.757 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.734+0000 7f8afac45700 1 --2- 172.21.15.39:0/677838106 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8af41049e0 0x7f8af4104db0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8009990 tx=0x7f8ae8009ca0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.757 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.735+0000 7f8afac45700 1 -- 172.21.15.39:0/677838106 shutdown_connections 2023-07-21T18:42:39.757 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.735+0000 7f8afac45700 1 --2- 172.21.15.39:0/677838106 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8af40ff380 0x7f8af410b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.757 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.735+0000 7f8afac45700 1 --2- 172.21.15.39:0/677838106 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8af41049e0 0x7f8af4104db0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.757 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.735+0000 7f8afac45700 1 --2- 172.21.15.39:0/677838106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8af40fe9d0 0x7f8af40fee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.757 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.735+0000 7f8afac45700 1 -- 172.21.15.39:0/677838106 >> 172.21.15.39:0/677838106 conn(0x7f8af40fa4b0 msgr2=0x7f8af40fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:39.757 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.735+0000 7f8afac45700 1 -- 172.21.15.39:0/677838106 shutdown_connections 2023-07-21T18:42:39.758 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.735+0000 7f8afac45700 1 -- 172.21.15.39:0/677838106 wait complete. 2023-07-21T18:42:39.759 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.735+0000 7f8afac45700 1 Processor -- start 2023-07-21T18:42:39.759 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.735+0000 7f8afac45700 1 -- start start 2023-07-21T18:42:39.759 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.736+0000 7f8afac45700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8af40fe9d0 0x7f8af4197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:39.759 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.736+0000 7f8afac45700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8af40ff380 0x7f8af4198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:39.759 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.736+0000 7f8afac45700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8af41049e0 0x7f8af419c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:39.760 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.736+0000 7f8afac45700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af419cdf0 con 0x7f8af41049e0 2023-07-21T18:42:39.760 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.736+0000 7f8afac45700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af419cf60 con 0x7f8af40ff380 2023-07-21T18:42:39.760 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.736+0000 7f8afac45700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af419d120 con 0x7f8af40fe9d0 2023-07-21T18:42:39.760 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.736+0000 7f8af91e2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8af41049e0 0x7f8af419c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:39.760 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.736+0000 7f8af91e2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8af41049e0 0x7f8af419c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58058/0 (socket says 172.21.15.39:58058) 2023-07-21T18:42:39.760 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.736+0000 7f8af91e2700 1 -- 172.21.15.39:0/1936443076 learned_addr learned my addr 172.21.15.39:0/1936443076 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:39.761 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.736+0000 7f8af91e2700 1 -- 172.21.15.39:0/1936443076 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8af40fe9d0 msgr2=0x7f8af4197e10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-21T18:42:39.761 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.736+0000 7f8af91e2700 1 --2- 172.21.15.39:0/1936443076 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8af40fe9d0 0x7f8af4197e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.761 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.736+0000 7f8af91e2700 1 -- 172.21.15.39:0/1936443076 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8af40ff380 msgr2=0x7f8af4198350 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-21T18:42:39.761 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.736+0000 7f8af91e2700 1 --2- 172.21.15.39:0/1936443076 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8af40ff380 0x7f8af4198350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.761 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.736+0000 7f8af91e2700 1 -- 172.21.15.39:0/1936443076 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae80095f0 con 0x7f8af41049e0 2023-07-21T18:42:39.761 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.737+0000 7f8af91e2700 1 --2- 172.21.15.39:0/1936443076 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8af41049e0 0x7f8af419c7b0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f8aec00ef30 tx=0x7f8aec00c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:39.761 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.737+0000 7f8af1ffb700 1 -- 172.21.15.39:0/1936443076 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aec018070 con 0x7f8af41049e0 2023-07-21T18:42:39.762 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.737+0000 7f8afac45700 1 -- 172.21.15.39:0/1936443076 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8af419d3a0 con 0x7f8af41049e0 2023-07-21T18:42:39.762 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.737+0000 7f8af1ffb700 1 -- 172.21.15.39:0/1936443076 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8aec00f040 con 0x7f8af41049e0 2023-07-21T18:42:39.762 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.737+0000 7f8af1ffb700 1 -- 172.21.15.39:0/1936443076 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aec004020 con 0x7f8af41049e0 2023-07-21T18:42:39.762 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.737+0000 7f8afac45700 1 -- 172.21.15.39:0/1936443076 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8af41a4c40 con 0x7f8af41049e0 2023-07-21T18:42:39.762 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.741+0000 7f8af1ffb700 1 -- 172.21.15.39:0/1936443076 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f8aec004ad0 con 0x7f8af41049e0 2023-07-21T18:42:39.763 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.742+0000 7f8af1ffb700 1 --2- 172.21.15.39:0/1936443076 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f8ae40a13e0 0x7f8ae40a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:39.763 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.742+0000 7f8b06dcbb80 1 Processor -- start 2023-07-21T18:42:39.764 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.742+0000 7f8af1ffb700 1 -- 172.21.15.39:0/1936443076 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f8aec0ccb40 con 0x7f8af41049e0 2023-07-21T18:42:39.764 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.742+0000 7f8b06dcbb80 1 -- start start 2023-07-21T18:42:39.765 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.742+0000 7f8b06dcbb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d73a0a62d0 0x55d73a0a66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:39.765 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.742+0000 7f8b06dcbb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d73a0a6be0 0x55d73a0b9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:39.765 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.742+0000 7f8b06dcbb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d73a0ba2e0 0x55d73a0bc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:39.765 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.742+0000 7f8b06dcbb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d739faccb0 con 0x55d73a0a62d0 2023-07-21T18:42:39.765 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.742+0000 7f8b06dcbb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d739fffa40 con 0x55d73a0a6be0 2023-07-21T18:42:39.765 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.742+0000 7f8b06dcbb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d739ff8ed0 con 0x55d73a0ba2e0 2023-07-21T18:42:39.765 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.742+0000 7f8af89e1700 1 --2- 172.21.15.39:0/1936443076 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f8ae40a13e0 0x7f8ae40a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:39.766 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.743+0000 7f8af3fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d73a0a6be0 0x55d73a0b9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:39.766 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.743+0000 7f8af3fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d73a0a6be0 0x55d73a0b9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:42426/0 (socket says 172.21.15.39:42426) 2023-07-21T18:42:39.766 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.743+0000 7f8af3fff700 1 -- 172.21.15.39:0/1206900723 learned_addr learned my addr 172.21.15.39:0/1206900723 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:39.766 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.743+0000 7f8af89e1700 1 --2- 172.21.15.39:0/1206900723 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d73a0a62d0 0x55d73a0a66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:39.766 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.743+0000 7f8af91e2700 1 --2- 172.21.15.39:0/1206900723 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d73a0ba2e0 0x55d73a0bc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:39.768 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.743+0000 7f8af3fff700 1 -- 172.21.15.39:0/1206900723 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d73a0ba2e0 msgr2=0x55d73a0bc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:39.769 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.743+0000 7f8af3fff700 1 --2- 172.21.15.39:0/1206900723 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d73a0ba2e0 0x55d73a0bc6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.769 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.743+0000 7f8af3fff700 1 -- 172.21.15.39:0/1206900723 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d73a0a62d0 msgr2=0x55d73a0a66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:39.769 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.743+0000 7f8af3fff700 1 --2- 172.21.15.39:0/1206900723 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d73a0a62d0 0x55d73a0a66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.769 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.743+0000 7f8af3fff700 1 -- 172.21.15.39:0/1206900723 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d739e26de0 con 0x55d73a0a6be0 2023-07-21T18:42:39.769 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.743+0000 7f8af89e1700 1 --2- 172.21.15.39:0/1936443076 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f8ae40a13e0 0x7f8ae40a3890 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8005cb0 tx=0x7f8ae8005770 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:39.769 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.743+0000 7f8af3fff700 1 --2- 172.21.15.39:0/1206900723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d73a0a6be0 0x55d73a0b9da0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8ae0009c00 tx=0x7f8ae0009f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4eb7c880ccbb4fa2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:39.770 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.744+0000 7f8adeffd700 1 -- 172.21.15.39:0/1206900723 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae0009750 con 0x55d73a0a6be0 2023-07-21T18:42:39.770 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.744+0000 7f8adeffd700 1 -- 172.21.15.39:0/1206900723 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ae000e860 con 0x55d73a0a6be0 2023-07-21T18:42:39.770 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.744+0000 7f8adeffd700 1 -- 172.21.15.39:0/1206900723 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae0005550 con 0x55d73a0a6be0 2023-07-21T18:42:39.770 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.744+0000 7f8b06dcbb80 1 -- 172.21.15.39:0/1206900723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d73a0a6be0 msgr2=0x55d73a0b9da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:39.770 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.744+0000 7f8b06dcbb80 1 --2- 172.21.15.39:0/1206900723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d73a0a6be0 0x55d73a0b9da0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8ae0009c00 tx=0x7f8ae0009f10 comp rx=0 tx=0).stop 2023-07-21T18:42:39.770 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.745+0000 7f8b06dcbb80 1 -- 172.21.15.39:0/1206900723 shutdown_connections 2023-07-21T18:42:39.770 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.745+0000 7f8b06dcbb80 1 --2- 172.21.15.39:0/1206900723 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d73a0ba2e0 0x55d73a0bc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.770 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.745+0000 7f8b06dcbb80 1 --2- 172.21.15.39:0/1206900723 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d73a0a62d0 0x55d73a0a66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.771 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.745+0000 7f8b06dcbb80 1 --2- 172.21.15.39:0/1206900723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d73a0a6be0 0x55d73a0b9da0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.771 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.745+0000 7f8b06dcbb80 1 -- 172.21.15.39:0/1206900723 >> 172.21.15.39:0/1206900723 conn(0x55d73a09cda0 msgr2=0x55d73a0a26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:39.771 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.745+0000 7f8b06dcbb80 1 -- 172.21.15.39:0/1206900723 shutdown_connections 2023-07-21T18:42:39.771 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.745+0000 7f8b06dcbb80 1 -- 172.21.15.39:0/1206900723 wait complete. 2023-07-21T18:42:39.771 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.745+0000 7f8b06dcbb80 1 -- start start 2023-07-21T18:42:39.771 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.745+0000 7f8b06dcbb80 1 Processor -- start 2023-07-21T18:42:39.771 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.746+0000 7f8b06dcbb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d73a0a62d0 0x55d73a0b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:39.771 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.746+0000 7f8b06dcbb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d73a0a6be0 0x55d73a0b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:39.772 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.746+0000 7f8b06dcbb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d73a0ba2e0 0x55d73a0b9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:39.772 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.746+0000 7f8b06dcbb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d739f98ac0 con 0x55d73a0ba2e0 2023-07-21T18:42:39.772 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.746+0000 7f8b06dcbb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d739f987c0 con 0x55d73a0a6be0 2023-07-21T18:42:39.772 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.746+0000 7f8b06dcbb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d739e7c920 con 0x55d73a0a62d0 2023-07-21T18:42:39.772 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.746+0000 7f8b06dcbb80 10 client.0 _ll_register_callbacks cb 0x55d739f96760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:42:39.772 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.746+0000 7f8af91e2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d73a0ba2e0 0x55d73a0b9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:39.772 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.746+0000 7f8af91e2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d73a0ba2e0 0x55d73a0b9350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58072/0 (socket says 172.21.15.39:58072) 2023-07-21T18:42:39.773 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.746+0000 7f8af91e2700 1 -- 172.21.15.39:0/1941815340 learned_addr learned my addr 172.21.15.39:0/1941815340 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:39.773 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.746+0000 7f8af89e1700 1 --2- 172.21.15.39:0/1941815340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d73a0a62d0 0x55d73a0b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:39.773 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.746+0000 7f8af91e2700 1 -- 172.21.15.39:0/1941815340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d73a0a62d0 msgr2=0x55d73a0b3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:39.773 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.746+0000 7f8af91e2700 1 --2- 172.21.15.39:0/1941815340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d73a0a62d0 0x55d73a0b3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.773 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.746+0000 7f8af91e2700 1 -- 172.21.15.39:0/1941815340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d73a0a6be0 msgr2=0x55d73a0b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:39.773 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.746+0000 7f8af91e2700 1 --2- 172.21.15.39:0/1941815340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d73a0a6be0 0x55d73a0b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.774 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.746+0000 7f8af91e2700 1 -- 172.21.15.39:0/1941815340 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ae00098b0 con 0x55d73a0ba2e0 2023-07-21T18:42:39.774 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.747+0000 7f8af91e2700 1 --2- 172.21.15.39:0/1941815340 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d73a0ba2e0 0x55d73a0b9350 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f8aec0ca8b0 tx=0x7f8aec099fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b86fccf6864968d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:39.774 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.747+0000 7f8ad3fff700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:42:39.774 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.747+0000 7f8ad3fff700 1 -- 172.21.15.39:0/1941815340 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aec09daa0 con 0x55d73a0ba2e0 2023-07-21T18:42:39.774 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.747+0000 7f8ad3fff700 1 -- 172.21.15.39:0/1941815340 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f8aec0a1e30 con 0x55d73a0ba2e0 2023-07-21T18:42:39.774 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.747+0000 7f8b06dcbb80 1 -- 172.21.15.39:0/1941815340 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d739f524f0 con 0x55d73a0ba2e0 2023-07-21T18:42:39.774 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.747+0000 7f8ad3fff700 1 -- 172.21.15.39:0/1941815340 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8aec0ca410 con 0x55d73a0ba2e0 2023-07-21T18:42:39.775 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.747+0000 7f8b06dcbb80 10 client.15804 fetch_fsmap learned FSMap version 26 2023-07-21T18:42:39.775 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.747+0000 7f8b06dcbb80 1 -- 172.21.15.39:0/1941815340 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55d739e0fc60 con 0x55d73a0ba2e0 2023-07-21T18:42:39.776 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.747+0000 7f8ad3fff700 1 -- 172.21.15.39:0/1941815340 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d739f524f0 con 0x55d73a0ba2e0 2023-07-21T18:42:39.776 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.747+0000 7f8ad3fff700 1 -- 172.21.15.39:0/1941815340 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8aec095a60 con 0x55d73a0ba2e0 2023-07-21T18:42:39.776 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.747+0000 7f8b06dcbb80 10 client.15804 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T18:42:39.777 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.747+0000 7f8b06dcbb80 10 client.15804 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2023-07-21T18:42:39.777 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.748+0000 7f8b06dcbb80 20 client.15804 populate_metadata read hostname 'smithi039' 2023-07-21T18:42:39.777 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.748+0000 7f8b06dcbb80 1 --2- 172.21.15.39:0/1941815340 >> [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] conn(0x55d73a0d2450 0x55d73a0d4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:39.777 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.748+0000 7f8b06dcbb80 4 client.15804 mds_command: new command op to 14604 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:42:39.777 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.748+0000 7f8b06dcbb80 1 -- 172.21.15.39:0/1941815340 --> [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d739ff1eb0 con 0x55d73a0d2450 2023-07-21T18:42:39.777 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.748+0000 7f8af89e1700 1 --2- 172.21.15.39:0/1941815340 >> [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] conn(0x55d73a0d2450 0x55d73a0d4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:39.777 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.749+0000 7f8af89e1700 1 --2- 172.21.15.39:0/1941815340 >> [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] conn(0x55d73a0d2450 0x55d73a0d4860 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d134a78f6afa4510 server_cookie=6a050e473bc62a84 in_seq=0 out_seq=0 2023-07-21T18:42:39.778 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.749+0000 7f8ad3fff700 10 client.15804 ms_handle_connect on v2:172.21.15.39:6840/640981220 2023-07-21T18:42:39.778 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.749+0000 7f8ad3fff700 1 -- 172.21.15.39:0/1941815340 <== mds.4 v2:172.21.15.39:6840/640981220 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d739ff1eb0 con 0x55d73a0d2450 2023-07-21T18:42:39.778 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.749+0000 7f8ad3fff700 10 client.15804 handle_command_reply: tid=0 2023-07-21T18:42:39.778 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.750+0000 7f8b06dcbb80 1 client.15804 shutdown 2023-07-21T18:42:39.778 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.750+0000 7f8b06dcbb80 1 -- 172.21.15.39:0/1941815340 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d73a0ba2e0 msgr2=0x55d73a0b9350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:39.778 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.750+0000 7f8b06dcbb80 1 --2- 172.21.15.39:0/1941815340 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d73a0ba2e0 0x55d73a0b9350 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f8aec0ca8b0 tx=0x7f8aec099fb0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.778 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.750+0000 7f8b06dcbb80 1 -- 172.21.15.39:0/1941815340 shutdown_connections 2023-07-21T18:42:39.778 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.750+0000 7f8b06dcbb80 1 --2- 172.21.15.39:0/1941815340 >> [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] conn(0x55d73a0d2450 0x55d73a0d4860 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.779 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.750+0000 7f8b06dcbb80 1 --2- 172.21.15.39:0/1941815340 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d73a0ba2e0 0x55d73a0b9350 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.779 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.750+0000 7f8b06dcbb80 1 --2- 172.21.15.39:0/1941815340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d73a0a62d0 0x55d73a0b3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.779 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.750+0000 7f8b06dcbb80 1 --2- 172.21.15.39:0/1941815340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d73a0a6be0 0x55d73a0b3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.779 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.750+0000 7f8b06dcbb80 1 -- 172.21.15.39:0/1941815340 >> 172.21.15.39:0/1941815340 conn(0x55d73a09cda0 msgr2=0x55d73a0a2290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:39.779 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.750+0000 7f8ad3fff700 0 client.15804 ms_handle_reset on v2:172.21.15.39:6840/640981220 2023-07-21T18:42:39.779 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.750+0000 7f8b06dcbb80 1 -- 172.21.15.39:0/1941815340 shutdown_connections 2023-07-21T18:42:39.779 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.750+0000 7f8b06dcbb80 1 -- 172.21.15.39:0/1941815340 wait complete. 2023-07-21T18:42:39.779 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.750+0000 7f8b06dcbb80 20 client.15804 trim_cache size 0 max 16384 2023-07-21T18:42:39.797 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.776+0000 7f8b06dcbb80 1 Processor -- start 2023-07-21T18:42:39.798 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.776+0000 7f8b06dcbb80 1 -- start start 2023-07-21T18:42:39.798 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.776+0000 7f8b06dcbb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d73a0a62d0 0x55d73a0a9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:39.798 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.776+0000 7f8b06dcbb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d73a0a6be0 0x55d73a0aa180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:39.798 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.776+0000 7f8b06dcbb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d73a0ba2e0 0x55d73a0b31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:39.798 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.776+0000 7f8b06dcbb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d73a0a3330 con 0x55d73a0a62d0 2023-07-21T18:42:39.798 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.776+0000 7f8b06dcbb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d73a0a7540 con 0x55d73a0a6be0 2023-07-21T18:42:39.799 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.776+0000 7f8b06dcbb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d73a0d2230 con 0x55d73a0ba2e0 2023-07-21T18:42:39.800 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.776+0000 7f8af89e1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d73a0a62d0 0x55d73a0a9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:39.800 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.776+0000 7f8af89e1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d73a0a62d0 0x55d73a0a9c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58076/0 (socket says 172.21.15.39:58076) 2023-07-21T18:42:39.800 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.776+0000 7f8af89e1700 1 -- 172.21.15.39:0/2293308034 learned_addr learned my addr 172.21.15.39:0/2293308034 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:39.800 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.776+0000 7f8af91e2700 1 --2- 172.21.15.39:0/2293308034 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d73a0ba2e0 0x55d73a0b31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:39.800 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.777+0000 7f8af3fff700 1 --2- 172.21.15.39:0/2293308034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d73a0a6be0 0x55d73a0aa180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:39.801 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.777+0000 7f8af89e1700 1 -- 172.21.15.39:0/2293308034 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d73a0ba2e0 msgr2=0x55d73a0b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:39.801 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.777+0000 7f8af89e1700 1 --2- 172.21.15.39:0/2293308034 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d73a0ba2e0 0x55d73a0b31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.801 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.777+0000 7f8af89e1700 1 -- 172.21.15.39:0/2293308034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d73a0a6be0 msgr2=0x55d73a0aa180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:39.801 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.777+0000 7f8af89e1700 1 --2- 172.21.15.39:0/2293308034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d73a0a6be0 0x55d73a0aa180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.801 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.777+0000 7f8af89e1700 1 -- 172.21.15.39:0/2293308034 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d739e4d250 con 0x55d73a0a62d0 2023-07-21T18:42:39.801 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.777+0000 7f8af89e1700 1 --2- 172.21.15.39:0/2293308034 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d73a0a62d0 0x55d73a0a9c40 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f8af40ffb10 tx=0x7f8ae8027000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=612119b603bb3c7c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:39.801 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.777+0000 7f8adf7fe700 1 -- 172.21.15.39:0/2293308034 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae8036b30 con 0x55d73a0a62d0 2023-07-21T18:42:39.801 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.777+0000 7f8adf7fe700 1 -- 172.21.15.39:0/2293308034 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ae80365b0 con 0x55d73a0a62d0 2023-07-21T18:42:39.802 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.777+0000 7f8adf7fe700 1 -- 172.21.15.39:0/2293308034 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae800fb80 con 0x55d73a0a62d0 2023-07-21T18:42:39.802 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.778+0000 7f8b06dcbb80 1 -- 172.21.15.39:0/2293308034 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d73a0a62d0 msgr2=0x55d73a0a9c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:39.802 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.778+0000 7f8b06dcbb80 1 --2- 172.21.15.39:0/2293308034 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d73a0a62d0 0x55d73a0a9c40 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f8af40ffb10 tx=0x7f8ae8027000 comp rx=0 tx=0).stop 2023-07-21T18:42:39.802 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.778+0000 7f8b06dcbb80 1 -- 172.21.15.39:0/2293308034 shutdown_connections 2023-07-21T18:42:39.802 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.778+0000 7f8b06dcbb80 1 --2- 172.21.15.39:0/2293308034 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d73a0ba2e0 0x55d73a0b31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.802 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.778+0000 7f8b06dcbb80 1 --2- 172.21.15.39:0/2293308034 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d73a0a62d0 0x55d73a0a9c40 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.802 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.778+0000 7f8b06dcbb80 1 --2- 172.21.15.39:0/2293308034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d73a0a6be0 0x55d73a0aa180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.802 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.778+0000 7f8b06dcbb80 1 -- 172.21.15.39:0/2293308034 >> 172.21.15.39:0/2293308034 conn(0x55d73a09cda0 msgr2=0x55d73a0d91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:39.803 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.778+0000 7f8b06dcbb80 1 -- 172.21.15.39:0/2293308034 shutdown_connections 2023-07-21T18:42:39.803 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.778+0000 7f8b06dcbb80 1 -- 172.21.15.39:0/2293308034 wait complete. 2023-07-21T18:42:39.805 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.779+0000 7f8b06dcbb80 1 -- start start 2023-07-21T18:42:39.805 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.779+0000 7f8b06dcbb80 1 Processor -- start 2023-07-21T18:42:39.805 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.779+0000 7f8b06dcbb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d73a0a6be0 0x55d73a0b4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:39.806 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.779+0000 7f8b06dcbb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d73a0ba2e0 0x55d73a0b52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:39.806 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.779+0000 7f8b06dcbb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d73a1232f0 0x55d73a1256d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:39.806 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.779+0000 7f8b06dcbb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d73a0b5a50 con 0x55d73a0ba2e0 2023-07-21T18:42:39.806 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.779+0000 7f8b06dcbb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d73a125c10 con 0x55d73a0a6be0 2023-07-21T18:42:39.806 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.779+0000 7f8b06dcbb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d73a125dd0 con 0x55d73a1232f0 2023-07-21T18:42:39.806 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.779+0000 7f8b06dcbb80 10 client.0 _ll_register_callbacks cb 0x7f8aec099930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:42:39.806 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.780+0000 7f8af91e2700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d73a1232f0 0x55d73a1256d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:39.806 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.780+0000 7f8af91e2700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d73a1232f0 0x55d73a1256d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:48404/0 (socket says 172.21.15.39:48404) 2023-07-21T18:42:39.807 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.780+0000 7f8af89e1700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d73a0a6be0 0x55d73a0b4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:39.807 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.780+0000 7f8af91e2700 1 -- 172.21.15.39:0/1409205086 learned_addr learned my addr 172.21.15.39:0/1409205086 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:39.807 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.780+0000 7f8af3fff700 1 --2- 172.21.15.39:0/1409205086 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d73a0ba2e0 0x55d73a0b52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:39.807 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.780+0000 7f8af91e2700 1 -- 172.21.15.39:0/1409205086 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d73a0a6be0 msgr2=0x55d73a0b4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:39.807 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.780+0000 7f8af91e2700 1 --2- 172.21.15.39:0/1409205086 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d73a0a6be0 0x55d73a0b4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.807 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.780+0000 7f8af91e2700 1 -- 172.21.15.39:0/1409205086 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d73a0ba2e0 msgr2=0x55d73a0b52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:39.807 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.780+0000 7f8af91e2700 1 --2- 172.21.15.39:0/1409205086 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d73a0ba2e0 0x55d73a0b52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.808 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.780+0000 7f8af91e2700 1 -- 172.21.15.39:0/1409205086 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ae802d040 con 0x55d73a1232f0 2023-07-21T18:42:39.808 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.780+0000 7f8af3fff700 1 --2- 172.21.15.39:0/1409205086 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d73a0ba2e0 0x55d73a0b52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:42:39.808 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.780+0000 7f8af91e2700 1 --2- 172.21.15.39:0/1409205086 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d73a1232f0 0x55d73a1256d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8aec0090a0 tx=0x7f8aec095ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbe95151f611e9c4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:39.808 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.780+0000 7f8ad3fff700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2023-07-21T18:42:39.808 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.780+0000 7f8ad3fff700 1 -- 172.21.15.39:0/1409205086 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aec01bdb0 con 0x55d73a1232f0 2023-07-21T18:42:39.808 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.780+0000 7f8ad3fff700 1 -- 172.21.15.39:0/1409205086 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f8aec0a1e30 con 0x55d73a1232f0 2023-07-21T18:42:39.808 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.780+0000 7f8b06dcbb80 1 -- 172.21.15.39:0/1409205086 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8ae000e860 con 0x55d73a1232f0 2023-07-21T18:42:39.808 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.781+0000 7f8ad3fff700 1 -- 172.21.15.39:0/1409205086 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8aec09daa0 con 0x55d73a1232f0 2023-07-21T18:42:39.809 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.781+0000 7f8b06dcbb80 10 client.25369 fetch_fsmap learned FSMap version 26 2023-07-21T18:42:39.809 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.781+0000 7f8b06dcbb80 1 -- 172.21.15.39:0/1409205086 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55d739e252b0 con 0x55d73a1232f0 2023-07-21T18:42:39.809 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.781+0000 7f8ad3fff700 1 -- 172.21.15.39:0/1409205086 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ae000e860 con 0x55d73a1232f0 2023-07-21T18:42:39.809 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.781+0000 7f8ad3fff700 1 -- 172.21.15.39:0/1409205086 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8aec00cc30 con 0x55d73a1232f0 2023-07-21T18:42:39.809 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.781+0000 7f8b06dcbb80 10 client.25369 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T18:42:39.809 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.781+0000 7f8b06dcbb80 10 client.25369 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2023-07-21T18:42:39.809 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.781+0000 7f8b06dcbb80 20 client.25369 populate_metadata read hostname 'smithi039' 2023-07-21T18:42:39.809 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.781+0000 7f8b06dcbb80 1 --2- 172.21.15.39:0/1409205086 >> [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] conn(0x55d73a1271b0 0x55d73a129590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:39.810 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.781+0000 7f8b06dcbb80 4 client.25369 mds_command: new command op to 14604 tid=0[{"prefix": "damage ls"}] 2023-07-21T18:42:39.810 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.781+0000 7f8b06dcbb80 1 -- 172.21.15.39:0/1409205086 --> [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d739fb4050 con 0x55d73a1271b0 2023-07-21T18:42:39.810 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.781+0000 7f8af89e1700 1 --2- 172.21.15.39:0/1409205086 >> [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] conn(0x55d73a1271b0 0x55d73a129590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:39.810 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.782+0000 7f8ad3fff700 10 client.25369 ms_handle_connect on v2:172.21.15.39:6840/640981220 2023-07-21T18:42:39.810 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.782+0000 7f8af89e1700 1 --2- 172.21.15.39:0/1409205086 >> [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] conn(0x55d73a1271b0 0x55d73a129590 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=af5aa650935205d4 server_cookie=1a88eb693c0b65de in_seq=0 out_seq=0 2023-07-21T18:42:39.810 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.782+0000 7f8ad3fff700 1 -- 172.21.15.39:0/1409205086 <== mds.4 v2:172.21.15.39:6840/640981220 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d739fb4050 con 0x55d73a1271b0 2023-07-21T18:42:39.810 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.782+0000 7f8ad3fff700 10 client.25369 handle_command_reply: tid=0 2023-07-21T18:42:39.811 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.782+0000 7f8b06dcbb80 1 client.25369 shutdown 2023-07-21T18:42:39.811 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.782+0000 7f8b06dcbb80 1 -- 172.21.15.39:0/1409205086 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d73a1232f0 msgr2=0x55d73a1256d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:39.811 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.782+0000 7f8b06dcbb80 1 --2- 172.21.15.39:0/1409205086 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d73a1232f0 0x55d73a1256d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8aec0090a0 tx=0x7f8aec095ee0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.811 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.782+0000 7f8b06dcbb80 1 -- 172.21.15.39:0/1409205086 shutdown_connections 2023-07-21T18:42:39.811 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.782+0000 7f8b06dcbb80 1 --2- 172.21.15.39:0/1409205086 >> [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] conn(0x55d73a1271b0 0x55d73a129590 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.811 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.782+0000 7f8b06dcbb80 1 --2- 172.21.15.39:0/1409205086 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d73a0a6be0 0x55d73a0b4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.811 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.782+0000 7f8b06dcbb80 1 --2- 172.21.15.39:0/1409205086 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d73a1232f0 0x55d73a1256d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.812 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.782+0000 7f8b06dcbb80 1 --2- 172.21.15.39:0/1409205086 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d73a0ba2e0 0x55d73a0b52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.812 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.782+0000 7f8b06dcbb80 1 -- 172.21.15.39:0/1409205086 >> 172.21.15.39:0/1409205086 conn(0x55d73a09cda0 msgr2=0x55d73a0a9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:39.812 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.782+0000 7f8ad3fff700 0 client.25369 ms_handle_reset on v2:172.21.15.39:6840/640981220 2023-07-21T18:42:39.812 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.783+0000 7f8b06dcbb80 1 -- 172.21.15.39:0/1409205086 shutdown_connections 2023-07-21T18:42:39.812 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.783+0000 7f8b06dcbb80 1 -- 172.21.15.39:0/1409205086 wait complete. 2023-07-21T18:42:39.812 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.783+0000 7f8b06dcbb80 20 client.25369 trim_cache size 0 max 16384 2023-07-21T18:42:39.813 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2023-07-21T18:42:39.813 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.783+0000 7f8afac45700 1 -- 172.21.15.39:0/1936443076 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f8ae40a13e0 msgr2=0x7f8ae40a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:39.814 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.783+0000 7f8afac45700 1 --2- 172.21.15.39:0/1936443076 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f8ae40a13e0 0x7f8ae40a3890 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8005cb0 tx=0x7f8ae8005770 comp rx=0 tx=0).stop 2023-07-21T18:42:39.814 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.783+0000 7f8afac45700 1 -- 172.21.15.39:0/1936443076 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8af41049e0 msgr2=0x7f8af419c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:39.814 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.783+0000 7f8afac45700 1 --2- 172.21.15.39:0/1936443076 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8af41049e0 0x7f8af419c7b0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f8aec00ef30 tx=0x7f8aec00c450 comp rx=0 tx=0).stop 2023-07-21T18:42:39.814 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.784+0000 7f8afac45700 1 -- 172.21.15.39:0/1936443076 shutdown_connections 2023-07-21T18:42:39.814 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.784+0000 7f8afac45700 1 --2- 172.21.15.39:0/1936443076 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8af41049e0 0x7f8af419c7b0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.814 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.784+0000 7f8afac45700 1 --2- 172.21.15.39:0/1936443076 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8af40fe9d0 0x7f8af4197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.814 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.784+0000 7f8afac45700 1 --2- 172.21.15.39:0/1936443076 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f8ae40a13e0 0x7f8ae40a3890 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.814 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.784+0000 7f8afac45700 1 --2- 172.21.15.39:0/1936443076 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8af40ff380 0x7f8af4198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:39.815 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.784+0000 7f8afac45700 1 -- 172.21.15.39:0/1936443076 >> 172.21.15.39:0/1936443076 conn(0x7f8af40fa4b0 msgr2=0x7f8af40fba80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:39.815 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.784+0000 7f8afac45700 1 -- 172.21.15.39:0/1936443076 shutdown_connections 2023-07-21T18:42:39.815 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:39.784+0000 7f8afac45700 1 -- 172.21.15.39:0/1936443076 wait complete. 2023-07-21T18:42:39.821 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-07-21T18:42:40.057 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.035+0000 7f4ba1300700 1 -- 172.21.15.39:0/127278250 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b9c0ff580 msgr2=0x7f4b9c0ff950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:40.058 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.035+0000 7f4ba1300700 1 --2- 172.21.15.39:0/127278250 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b9c0ff580 0x7f4b9c0ff950 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c009920 tx=0x7f4b8c009c30 comp rx=0 tx=0).stop 2023-07-21T18:42:40.058 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.035+0000 7f4ba1300700 1 -- 172.21.15.39:0/127278250 shutdown_connections 2023-07-21T18:42:40.058 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.035+0000 7f4ba1300700 1 --2- 172.21.15.39:0/127278250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b9c103ce0 0x7f4b9c10b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.058 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.035+0000 7f4ba1300700 1 --2- 172.21.15.39:0/127278250 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b9c0ff580 0x7f4b9c0ff950 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.058 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.035+0000 7f4ba1300700 1 --2- 172.21.15.39:0/127278250 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b9c0fff20 0x7f4b9c103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.058 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.035+0000 7f4ba1300700 1 -- 172.21.15.39:0/127278250 >> 172.21.15.39:0/127278250 conn(0x7f4b9c076200 msgr2=0x7f4b9c076600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:40.058 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.036+0000 7f4ba1300700 1 -- 172.21.15.39:0/127278250 shutdown_connections 2023-07-21T18:42:40.059 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.036+0000 7f4ba1300700 1 -- 172.21.15.39:0/127278250 wait complete. 2023-07-21T18:42:40.060 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.036+0000 7f4ba1300700 1 Processor -- start 2023-07-21T18:42:40.060 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.037+0000 7f4ba1300700 1 -- start start 2023-07-21T18:42:40.060 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.037+0000 7f4ba1300700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b9c0ff580 0x7f4b9c196af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.060 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.037+0000 7f4ba1300700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b9c0fff20 0x7f4b9c197030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.061 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.037+0000 7f4ba1300700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b9c103ce0 0x7f4b9c1a1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.061 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.037+0000 7f4ba1300700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b9c197970 con 0x7f4b9c0fff20 2023-07-21T18:42:40.061 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.037+0000 7f4ba1300700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b9c1a20b0 con 0x7f4b9c0ff580 2023-07-21T18:42:40.061 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.037+0000 7f4ba1300700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b9c1a2270 con 0x7f4b9c103ce0 2023-07-21T18:42:40.061 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.037+0000 7f4b9a7fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b9c0fff20 0x7f4b9c197030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:40.061 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.037+0000 7f4b9affd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b9c0ff580 0x7f4b9c196af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:40.061 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.037+0000 7f4b9a7fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b9c0fff20 0x7f4b9c197030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58098/0 (socket says 172.21.15.39:58098) 2023-07-21T18:42:40.062 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.037+0000 7f4b9a7fc700 1 -- 172.21.15.39:0/3363041117 learned_addr learned my addr 172.21.15.39:0/3363041117 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:40.062 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.037+0000 7f4b9affd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b9c0ff580 0x7f4b9c196af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:42460/0 (socket says 172.21.15.39:42460) 2023-07-21T18:42:40.062 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.037+0000 7f4b9b7fe700 1 --2- 172.21.15.39:0/3363041117 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b9c103ce0 0x7f4b9c1a1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:40.062 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.038+0000 7f4b9affd700 1 -- 172.21.15.39:0/3363041117 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b9c103ce0 msgr2=0x7f4b9c1a1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:40.062 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.038+0000 7f4b9affd700 1 --2- 172.21.15.39:0/3363041117 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b9c103ce0 0x7f4b9c1a1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.062 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.038+0000 7f4b9affd700 1 -- 172.21.15.39:0/3363041117 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b9c0fff20 msgr2=0x7f4b9c197030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:40.062 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.038+0000 7f4b9affd700 1 --2- 172.21.15.39:0/3363041117 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b9c0fff20 0x7f4b9c197030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.062 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.038+0000 7f4b9affd700 1 -- 172.21.15.39:0/3363041117 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b8c009520 con 0x7f4b9c0ff580 2023-07-21T18:42:40.063 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.038+0000 7f4b9a7fc700 1 --2- 172.21.15.39:0/3363041117 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b9c0fff20 0x7f4b9c197030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:42:40.063 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.038+0000 7f4b9affd700 1 --2- 172.21.15.39:0/3363041117 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b9c0ff580 0x7f4b9c196af0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c009230 tx=0x7f4b8c00fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:40.063 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.038+0000 7f4b83fff700 1 -- 172.21.15.39:0/3363041117 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b8c01c070 con 0x7f4b9c0ff580 2023-07-21T18:42:40.063 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.038+0000 7f4ba1300700 1 -- 172.21.15.39:0/3363041117 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b9c1a24f0 con 0x7f4b9c0ff580 2023-07-21T18:42:40.063 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.038+0000 7f4b83fff700 1 -- 172.21.15.39:0/3363041117 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b8c00bbd0 con 0x7f4b9c0ff580 2023-07-21T18:42:40.063 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.038+0000 7f4b83fff700 1 -- 172.21.15.39:0/3363041117 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b8c0038e0 con 0x7f4b9c0ff580 2023-07-21T18:42:40.063 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.038+0000 7f4ba1300700 1 -- 172.21.15.39:0/3363041117 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b9c1a2910 con 0x7f4b9c0ff580 2023-07-21T18:42:40.064 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.041+0000 7f4b83fff700 1 -- 172.21.15.39:0/3363041117 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f4b8c020030 con 0x7f4b9c0ff580 2023-07-21T18:42:40.064 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.042+0000 7f4b83fff700 1 --2- 172.21.15.39:0/3363041117 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f4b840a13e0 0x7f4b840a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.065 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.043+0000 7f4b83fff700 1 -- 172.21.15.39:0/3363041117 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f4b8c0cde50 con 0x7f4b9c0ff580 2023-07-21T18:42:40.065 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.042+0000 7f4b9a7fc700 1 --2- 172.21.15.39:0/3363041117 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f4b840a13e0 0x7f4b840a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:40.065 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.043+0000 7f4bad486b80 1 Processor -- start 2023-07-21T18:42:40.065 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.043+0000 7f4bad486b80 1 -- start start 2023-07-21T18:42:40.065 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.043+0000 7f4b9a7fc700 1 --2- 172.21.15.39:0/3363041117 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f4b840a13e0 0x7f4b840a3890 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c0fead0 tx=0x7f4b90009310 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:40.066 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.043+0000 7f4bad486b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbfdca32d0 0x55bbfdca36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.066 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.043+0000 7f4bad486b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbfdca3be0 0x55bbfdca8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.067 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.043+0000 7f4bad486b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bbfdca8a80 0x55bbfdca8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.067 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.043+0000 7f4bad486b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbfdba9cb0 con 0x55bbfdca32d0 2023-07-21T18:42:40.067 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.043+0000 7f4bad486b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbfdbfca40 con 0x55bbfdca8a80 2023-07-21T18:42:40.067 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.043+0000 7f4bad486b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbfdbf5ed0 con 0x55bbfdca3be0 2023-07-21T18:42:40.067 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.044+0000 7f4b9affd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbfdca32d0 0x55bbfdca36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:40.067 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.044+0000 7f4b9affd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbfdca32d0 0x55bbfdca36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58104/0 (socket says 172.21.15.39:58104) 2023-07-21T18:42:40.067 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.044+0000 7f4b9affd700 1 -- 172.21.15.39:0/2907797568 learned_addr learned my addr 172.21.15.39:0/2907797568 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:40.067 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.044+0000 7f4b9a7fc700 1 --2- 172.21.15.39:0/2907797568 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbfdca3be0 0x55bbfdca8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:40.068 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.044+0000 7f4b9b7fe700 1 --2- 172.21.15.39:0/2907797568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bbfdca8a80 0x55bbfdca8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:40.068 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.044+0000 7f4b9affd700 1 -- 172.21.15.39:0/2907797568 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbfdca3be0 msgr2=0x55bbfdca8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:40.068 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.044+0000 7f4b9affd700 1 --2- 172.21.15.39:0/2907797568 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbfdca3be0 0x55bbfdca8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.068 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.044+0000 7f4b9affd700 1 -- 172.21.15.39:0/2907797568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bbfdca8a80 msgr2=0x55bbfdca8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:40.068 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.044+0000 7f4b9affd700 1 --2- 172.21.15.39:0/2907797568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bbfdca8a80 0x55bbfdca8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.068 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.044+0000 7f4b9affd700 1 -- 172.21.15.39:0/2907797568 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bbfda23de0 con 0x55bbfdca32d0 2023-07-21T18:42:40.068 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.044+0000 7f4b9affd700 1 --2- 172.21.15.39:0/2907797568 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbfdca32d0 0x55bbfdca36a0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c009810 tx=0x7f4b8c009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e87121851836d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:40.069 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.044+0000 7f4b817fa700 1 -- 172.21.15.39:0/2907797568 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b8c0b2070 con 0x55bbfdca32d0 2023-07-21T18:42:40.069 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.045+0000 7f4b817fa700 1 -- 172.21.15.39:0/2907797568 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b8c0970c0 con 0x55bbfdca32d0 2023-07-21T18:42:40.069 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.045+0000 7f4b817fa700 1 -- 172.21.15.39:0/2907797568 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b8c005140 con 0x55bbfdca32d0 2023-07-21T18:42:40.071 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.045+0000 7f4bad486b80 1 -- 172.21.15.39:0/2907797568 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbfdca32d0 msgr2=0x55bbfdca36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:40.071 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.045+0000 7f4bad486b80 1 --2- 172.21.15.39:0/2907797568 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbfdca32d0 0x55bbfdca36a0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c009810 tx=0x7f4b8c009c30 comp rx=0 tx=0).stop 2023-07-21T18:42:40.071 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.045+0000 7f4bad486b80 1 -- 172.21.15.39:0/2907797568 shutdown_connections 2023-07-21T18:42:40.071 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.045+0000 7f4bad486b80 1 --2- 172.21.15.39:0/2907797568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bbfdca8a80 0x55bbfdca8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.071 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.045+0000 7f4bad486b80 1 --2- 172.21.15.39:0/2907797568 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbfdca32d0 0x55bbfdca36a0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.071 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.045+0000 7f4bad486b80 1 --2- 172.21.15.39:0/2907797568 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbfdca3be0 0x55bbfdca8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.071 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.046+0000 7f4bad486b80 1 -- 172.21.15.39:0/2907797568 >> 172.21.15.39:0/2907797568 conn(0x55bbfdc99da0 msgr2=0x55bbfdc9f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:40.072 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.046+0000 7f4bad486b80 1 -- 172.21.15.39:0/2907797568 shutdown_connections 2023-07-21T18:42:40.072 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.046+0000 7f4bad486b80 1 -- 172.21.15.39:0/2907797568 wait complete. 2023-07-21T18:42:40.072 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.046+0000 7f4bad486b80 1 -- start start 2023-07-21T18:42:40.072 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.046+0000 7f4bad486b80 1 Processor -- start 2023-07-21T18:42:40.072 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.046+0000 7f4bad486b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bbfdca32d0 0x55bbfdcae060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.072 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.046+0000 7f4bad486b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbfdca3be0 0x55bbfdcae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.072 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.047+0000 7f4bad486b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbfdca8a80 0x55bbfdcbc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.073 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.047+0000 7f4bad486b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbfdb95ac0 con 0x55bbfdca3be0 2023-07-21T18:42:40.073 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.047+0000 7f4bad486b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbfdb957c0 con 0x55bbfdca32d0 2023-07-21T18:42:40.073 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.047+0000 7f4bad486b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbfda79920 con 0x55bbfdca8a80 2023-07-21T18:42:40.073 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.047+0000 7f4bad486b80 10 client.0 _ll_register_callbacks cb 0x55bbfdb93760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:42:40.073 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.047+0000 7f4b9a7fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbfdca3be0 0x55bbfdcae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:40.073 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.047+0000 7f4b9a7fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbfdca3be0 0x55bbfdcae5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58114/0 (socket says 172.21.15.39:58114) 2023-07-21T18:42:40.073 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.047+0000 7f4b9a7fc700 1 -- 172.21.15.39:0/4203348743 learned_addr learned my addr 172.21.15.39:0/4203348743 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:40.073 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.047+0000 7f4b9b7fe700 1 --2- 172.21.15.39:0/4203348743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbfdca8a80 0x55bbfdcbc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:40.074 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.047+0000 7f4b9a7fc700 1 -- 172.21.15.39:0/4203348743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbfdca8a80 msgr2=0x55bbfdcbc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:40.074 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.047+0000 7f4b9a7fc700 1 --2- 172.21.15.39:0/4203348743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbfdca8a80 0x55bbfdcbc1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.074 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.047+0000 7f4b9a7fc700 1 -- 172.21.15.39:0/4203348743 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bbfdca32d0 msgr2=0x55bbfdcae060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:40.074 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.047+0000 7f4b9a7fc700 1 --2- 172.21.15.39:0/4203348743 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bbfdca32d0 0x55bbfdcae060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.074 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.047+0000 7f4b9a7fc700 1 -- 172.21.15.39:0/4203348743 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b9c1a2910 con 0x55bbfdca3be0 2023-07-21T18:42:40.074 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.047+0000 7f4b9a7fc700 1 --2- 172.21.15.39:0/4203348743 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbfdca3be0 0x55bbfdcae5a0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f4b900039d0 tx=0x7f4b900047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66b596791c7f1b93 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:40.074 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.047+0000 7f4b7eff5700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:42:40.074 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.048+0000 7f4b7eff5700 1 -- 172.21.15.39:0/4203348743 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b9001fcd0 con 0x55bbfdca3be0 2023-07-21T18:42:40.075 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.048+0000 7f4b7eff5700 1 -- 172.21.15.39:0/4203348743 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f4b90003e40 con 0x55bbfdca3be0 2023-07-21T18:42:40.075 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.048+0000 7f4bad486b80 1 -- 172.21.15.39:0/4203348743 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bbfdb4f4f0 con 0x55bbfdca3be0 2023-07-21T18:42:40.075 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.048+0000 7f4b7eff5700 1 -- 172.21.15.39:0/4203348743 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b9001fe30 con 0x55bbfdca3be0 2023-07-21T18:42:40.075 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.049+0000 7f4bad486b80 10 client.15822 fetch_fsmap learned FSMap version 26 2023-07-21T18:42:40.075 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.049+0000 7f4bad486b80 1 -- 172.21.15.39:0/4203348743 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55bbfda0cc60 con 0x55bbfdca3be0 2023-07-21T18:42:40.075 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.049+0000 7f4b7eff5700 1 -- 172.21.15.39:0/4203348743 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bbfdb4f4f0 con 0x55bbfdca3be0 2023-07-21T18:42:40.075 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.049+0000 7f4b7eff5700 1 -- 172.21.15.39:0/4203348743 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4b900305a0 con 0x55bbfdca3be0 2023-07-21T18:42:40.075 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.049+0000 7f4bad486b80 10 client.15822 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T18:42:40.076 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.049+0000 7f4bad486b80 10 client.15822 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2023-07-21T18:42:40.076 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.049+0000 7f4bad486b80 20 client.15822 populate_metadata read hostname 'smithi039' 2023-07-21T18:42:40.076 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.049+0000 7f4bad486b80 1 --2- 172.21.15.39:0/4203348743 >> [v2:172.21.15.155:6834/3854856286,v1:172.21.15.155:6835/3854856286] conn(0x55bbfdcc6a30 0x55bbfdcc8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.076 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.049+0000 7f4bad486b80 4 client.15822 mds_command: new command op to 24464 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:42:40.076 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.049+0000 7f4bad486b80 1 -- 172.21.15.39:0/4203348743 --> [v2:172.21.15.155:6834/3854856286,v1:172.21.15.155:6835/3854856286] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bbfdbeeeb0 con 0x55bbfdcc6a30 2023-07-21T18:42:40.077 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.049+0000 7f4b9affd700 1 --2- 172.21.15.39:0/4203348743 >> [v2:172.21.15.155:6834/3854856286,v1:172.21.15.155:6835/3854856286] conn(0x55bbfdcc6a30 0x55bbfdcc8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:40.077 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.050+0000 7f4b7eff5700 10 client.15822 ms_handle_connect on v2:172.21.15.155:6834/3854856286 2023-07-21T18:42:40.077 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.050+0000 7f4b9affd700 1 --2- 172.21.15.39:0/4203348743 >> [v2:172.21.15.155:6834/3854856286,v1:172.21.15.155:6835/3854856286] conn(0x55bbfdcc6a30 0x55bbfdcc8e40 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ba361e7c854151af server_cookie=f2a4401530491018 in_seq=0 out_seq=0 2023-07-21T18:42:40.078 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.050+0000 7f4b7eff5700 1 -- 172.21.15.39:0/4203348743 <== mds.3 v2:172.21.15.155:6834/3854856286 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55bbfdbeeeb0 con 0x55bbfdcc6a30 2023-07-21T18:42:40.078 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.050+0000 7f4b7eff5700 10 client.15822 handle_command_reply: tid=0 2023-07-21T18:42:40.078 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.050+0000 7f4bad486b80 1 client.15822 shutdown 2023-07-21T18:42:40.078 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.051+0000 7f4bad486b80 1 -- 172.21.15.39:0/4203348743 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbfdca3be0 msgr2=0x55bbfdcae5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:40.078 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.051+0000 7f4bad486b80 1 --2- 172.21.15.39:0/4203348743 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbfdca3be0 0x55bbfdcae5a0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f4b900039d0 tx=0x7f4b900047c0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.078 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.051+0000 7f4bad486b80 1 -- 172.21.15.39:0/4203348743 shutdown_connections 2023-07-21T18:42:40.078 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.051+0000 7f4bad486b80 1 --2- 172.21.15.39:0/4203348743 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bbfdca32d0 0x55bbfdcae060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.078 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.051+0000 7f4bad486b80 1 --2- 172.21.15.39:0/4203348743 >> [v2:172.21.15.155:6834/3854856286,v1:172.21.15.155:6835/3854856286] conn(0x55bbfdcc6a30 0x55bbfdcc8e40 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.079 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.051+0000 7f4bad486b80 1 --2- 172.21.15.39:0/4203348743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbfdca8a80 0x55bbfdcbc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.079 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.051+0000 7f4bad486b80 1 --2- 172.21.15.39:0/4203348743 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbfdca3be0 0x55bbfdcae5a0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.079 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.051+0000 7f4bad486b80 1 -- 172.21.15.39:0/4203348743 >> 172.21.15.39:0/4203348743 conn(0x55bbfdc99da0 msgr2=0x55bbfdc9f230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:40.079 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.051+0000 7f4b7eff5700 0 client.15822 ms_handle_reset on v2:172.21.15.155:6834/3854856286 2023-07-21T18:42:40.079 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.051+0000 7f4bad486b80 1 -- 172.21.15.39:0/4203348743 shutdown_connections 2023-07-21T18:42:40.079 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.051+0000 7f4bad486b80 1 -- 172.21.15.39:0/4203348743 wait complete. 2023-07-21T18:42:40.079 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.051+0000 7f4bad486b80 20 client.15822 trim_cache size 0 max 16384 2023-07-21T18:42:40.099 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.076+0000 7f4bad486b80 1 Processor -- start 2023-07-21T18:42:40.100 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.076+0000 7f4bad486b80 1 -- start start 2023-07-21T18:42:40.100 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.077+0000 7f4bad486b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbfdca32d0 0x55bbfdcbc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.100 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.077+0000 7f4bad486b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbfdca3be0 0x55bbfdcab100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.100 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.077+0000 7f4bad486b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bbfdca8a80 0x55bbfdcab640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.100 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.077+0000 7f4bad486b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbfdca0d20 con 0x55bbfdca3be0 2023-07-21T18:42:40.100 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.077+0000 7f4bad486b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbfdca4540 con 0x55bbfdca8a80 2023-07-21T18:42:40.101 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.077+0000 7f4bad486b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbfdca9a90 con 0x55bbfdca32d0 2023-07-21T18:42:40.101 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.077+0000 7f4b9a7fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbfdca3be0 0x55bbfdcab100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:40.101 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.077+0000 7f4b9a7fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbfdca3be0 0x55bbfdcab100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58120/0 (socket says 172.21.15.39:58120) 2023-07-21T18:42:40.101 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.077+0000 7f4b9a7fc700 1 -- 172.21.15.39:0/2068812010 learned_addr learned my addr 172.21.15.39:0/2068812010 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:40.101 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.077+0000 7f4b9affd700 1 --2- 172.21.15.39:0/2068812010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbfdca32d0 0x55bbfdcbc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:40.101 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.077+0000 7f4b9b7fe700 1 --2- 172.21.15.39:0/2068812010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bbfdca8a80 0x55bbfdcab640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:40.101 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.077+0000 7f4b9a7fc700 1 -- 172.21.15.39:0/2068812010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbfdca32d0 msgr2=0x55bbfdcbc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:40.101 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.077+0000 7f4b9a7fc700 1 --2- 172.21.15.39:0/2068812010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbfdca32d0 0x55bbfdcbc1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.102 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.077+0000 7f4b9a7fc700 1 -- 172.21.15.39:0/2068812010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bbfdca8a80 msgr2=0x55bbfdcab640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:40.102 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.077+0000 7f4b9a7fc700 1 --2- 172.21.15.39:0/2068812010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bbfdca8a80 0x55bbfdcab640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.102 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.077+0000 7f4b9a7fc700 1 -- 172.21.15.39:0/2068812010 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bbfda4a250 con 0x55bbfdca3be0 2023-07-21T18:42:40.102 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.077+0000 7f4b9b7fe700 1 --2- 172.21.15.39:0/2068812010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bbfdca8a80 0x55bbfdcab640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:42:40.102 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.078+0000 7f4b9a7fc700 1 --2- 172.21.15.39:0/2068812010 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbfdca3be0 0x55bbfdcab100 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f4b900309a0 tx=0x7f4b90004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f64f4767c481214 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:40.102 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.078+0000 7f4b81ffb700 1 -- 172.21.15.39:0/2068812010 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b900305e0 con 0x55bbfdca3be0 2023-07-21T18:42:40.102 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.078+0000 7f4b81ffb700 1 -- 172.21.15.39:0/2068812010 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b9001fad0 con 0x55bbfdca3be0 2023-07-21T18:42:40.102 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.078+0000 7f4b81ffb700 1 -- 172.21.15.39:0/2068812010 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b90028900 con 0x55bbfdca3be0 2023-07-21T18:42:40.105 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.079+0000 7f4bad486b80 1 -- 172.21.15.39:0/2068812010 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbfdca3be0 msgr2=0x55bbfdcab100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:40.105 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.079+0000 7f4bad486b80 1 --2- 172.21.15.39:0/2068812010 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbfdca3be0 0x55bbfdcab100 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f4b900309a0 tx=0x7f4b90004750 comp rx=0 tx=0).stop 2023-07-21T18:42:40.105 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.079+0000 7f4bad486b80 1 -- 172.21.15.39:0/2068812010 shutdown_connections 2023-07-21T18:42:40.106 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.079+0000 7f4bad486b80 1 --2- 172.21.15.39:0/2068812010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bbfdca8a80 0x55bbfdcab640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.106 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.079+0000 7f4bad486b80 1 --2- 172.21.15.39:0/2068812010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbfdca32d0 0x55bbfdcbc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.106 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.079+0000 7f4bad486b80 1 --2- 172.21.15.39:0/2068812010 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbfdca3be0 0x55bbfdcab100 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.106 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.079+0000 7f4bad486b80 1 -- 172.21.15.39:0/2068812010 >> 172.21.15.39:0/2068812010 conn(0x55bbfdc99da0 msgr2=0x55bbfdc9de10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:40.106 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.079+0000 7f4bad486b80 1 -- 172.21.15.39:0/2068812010 shutdown_connections 2023-07-21T18:42:40.106 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.079+0000 7f4bad486b80 1 -- 172.21.15.39:0/2068812010 wait complete. 2023-07-21T18:42:40.106 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.079+0000 7f4bad486b80 1 -- start start 2023-07-21T18:42:40.106 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.080+0000 7f4bad486b80 1 Processor -- start 2023-07-21T18:42:40.107 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.080+0000 7f4bad486b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bbfdca32d0 0x55bbfdd0ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.107 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.080+0000 7f4bad486b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbfdca3be0 0x55bbfdd0efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.107 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.080+0000 7f4bad486b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbfdca8a80 0x55bbfdd13940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.107 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.080+0000 7f4bad486b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbfdd13eb0 con 0x55bbfdca3be0 2023-07-21T18:42:40.107 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.080+0000 7f4bad486b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbfdd14020 con 0x55bbfdca32d0 2023-07-21T18:42:40.107 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.080+0000 7f4bad486b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbfdd141e0 con 0x55bbfdca8a80 2023-07-21T18:42:40.107 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.080+0000 7f4bad486b80 10 client.0 _ll_register_callbacks cb 0x55bbfdb93760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:42:40.107 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.080+0000 7f4b9a7fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbfdca3be0 0x55bbfdd0efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:40.108 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.080+0000 7f4b9a7fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbfdca3be0 0x55bbfdd0efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58126/0 (socket says 172.21.15.39:58126) 2023-07-21T18:42:40.108 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.080+0000 7f4b9a7fc700 1 -- 172.21.15.39:0/2335961948 learned_addr learned my addr 172.21.15.39:0/2335961948 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:40.108 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.080+0000 7f4b9b7fe700 1 --2- 172.21.15.39:0/2335961948 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbfdca8a80 0x55bbfdd13940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:40.108 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.080+0000 7f4b9affd700 1 --2- 172.21.15.39:0/2335961948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bbfdca32d0 0x55bbfdd0ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:40.108 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.081+0000 7f4b9a7fc700 1 -- 172.21.15.39:0/2335961948 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbfdca8a80 msgr2=0x55bbfdd13940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:40.108 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.081+0000 7f4b9a7fc700 1 --2- 172.21.15.39:0/2335961948 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbfdca8a80 0x55bbfdd13940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.108 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.081+0000 7f4b9a7fc700 1 -- 172.21.15.39:0/2335961948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bbfdca32d0 msgr2=0x55bbfdd0ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:40.109 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.081+0000 7f4b9a7fc700 1 --2- 172.21.15.39:0/2335961948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bbfdca32d0 0x55bbfdd0ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.109 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.081+0000 7f4b9a7fc700 1 -- 172.21.15.39:0/2335961948 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bbfda0cc60 con 0x55bbfdca3be0 2023-07-21T18:42:40.109 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.081+0000 7f4b9a7fc700 1 --2- 172.21.15.39:0/2335961948 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbfdca3be0 0x55bbfdd0efb0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f4b90009ed0 tx=0x7f4b900094c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19bb3aebbf462844 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:40.109 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.081+0000 7f4b7eff5700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:42:40.109 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.081+0000 7f4b7eff5700 1 -- 172.21.15.39:0/2335961948 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b90005040 con 0x55bbfdca3be0 2023-07-21T18:42:40.109 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.081+0000 7f4b7eff5700 1 -- 172.21.15.39:0/2335961948 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f4b90059050 con 0x55bbfdca3be0 2023-07-21T18:42:40.109 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.081+0000 7f4bad486b80 1 -- 172.21.15.39:0/2335961948 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4b8c0970c0 con 0x55bbfdca3be0 2023-07-21T18:42:40.109 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.081+0000 7f4b7eff5700 1 -- 172.21.15.39:0/2335961948 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b900051a0 con 0x55bbfdca3be0 2023-07-21T18:42:40.110 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.081+0000 7f4bad486b80 10 client.15828 fetch_fsmap learned FSMap version 26 2023-07-21T18:42:40.110 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.081+0000 7f4bad486b80 1 -- 172.21.15.39:0/2335961948 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55bbfda222b0 con 0x55bbfdca3be0 2023-07-21T18:42:40.110 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.082+0000 7f4b7eff5700 1 -- 172.21.15.39:0/2335961948 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b8c0970c0 con 0x55bbfdca3be0 2023-07-21T18:42:40.110 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.082+0000 7f4b7eff5700 1 -- 172.21.15.39:0/2335961948 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4b90064030 con 0x55bbfdca3be0 2023-07-21T18:42:40.110 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.082+0000 7f4bad486b80 10 client.15828 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T18:42:40.110 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.082+0000 7f4bad486b80 10 client.15828 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2023-07-21T18:42:40.110 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.082+0000 7f4bad486b80 20 client.15828 populate_metadata read hostname 'smithi039' 2023-07-21T18:42:40.110 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.082+0000 7f4bad486b80 1 --2- 172.21.15.39:0/2335961948 >> [v2:172.21.15.155:6834/3854856286,v1:172.21.15.155:6835/3854856286] conn(0x55bbfdd1b5d0 0x55bbfdd1d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.111 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.082+0000 7f4bad486b80 4 client.15828 mds_command: new command op to 24464 tid=0[{"prefix": "damage ls"}] 2023-07-21T18:42:40.111 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.082+0000 7f4bad486b80 1 -- 172.21.15.39:0/2335961948 --> [v2:172.21.15.155:6834/3854856286,v1:172.21.15.155:6835/3854856286] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bbfdbb1050 con 0x55bbfdd1b5d0 2023-07-21T18:42:40.111 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.082+0000 7f4b9affd700 1 --2- 172.21.15.39:0/2335961948 >> [v2:172.21.15.155:6834/3854856286,v1:172.21.15.155:6835/3854856286] conn(0x55bbfdd1b5d0 0x55bbfdd1d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:40.111 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.083+0000 7f4b9affd700 1 --2- 172.21.15.39:0/2335961948 >> [v2:172.21.15.155:6834/3854856286,v1:172.21.15.155:6835/3854856286] conn(0x55bbfdd1b5d0 0x55bbfdd1d9b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=893b371b785358c2 server_cookie=2950306ff9fb953 in_seq=0 out_seq=0 2023-07-21T18:42:40.111 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.083+0000 7f4b7eff5700 10 client.15828 ms_handle_connect on v2:172.21.15.155:6834/3854856286 2023-07-21T18:42:40.112 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.083+0000 7f4b7eff5700 1 -- 172.21.15.39:0/2335961948 <== mds.3 v2:172.21.15.155:6834/3854856286 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bbfdbb1050 con 0x55bbfdd1b5d0 2023-07-21T18:42:40.112 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.083+0000 7f4b7eff5700 10 client.15828 handle_command_reply: tid=0 2023-07-21T18:42:40.112 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.083+0000 7f4bad486b80 1 client.15828 shutdown 2023-07-21T18:42:40.112 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.083+0000 7f4bad486b80 1 -- 172.21.15.39:0/2335961948 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbfdca3be0 msgr2=0x55bbfdd0efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:40.112 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.083+0000 7f4bad486b80 1 --2- 172.21.15.39:0/2335961948 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbfdca3be0 0x55bbfdd0efb0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f4b90009ed0 tx=0x7f4b900094c0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.112 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.083+0000 7f4bad486b80 1 -- 172.21.15.39:0/2335961948 shutdown_connections 2023-07-21T18:42:40.112 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.083+0000 7f4bad486b80 1 --2- 172.21.15.39:0/2335961948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bbfdca32d0 0x55bbfdd0ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.113 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.083+0000 7f4bad486b80 1 --2- 172.21.15.39:0/2335961948 >> [v2:172.21.15.155:6834/3854856286,v1:172.21.15.155:6835/3854856286] conn(0x55bbfdd1b5d0 0x55bbfdd1d9b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.113 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.083+0000 7f4bad486b80 1 --2- 172.21.15.39:0/2335961948 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbfdca8a80 0x55bbfdd13940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.113 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.083+0000 7f4bad486b80 1 --2- 172.21.15.39:0/2335961948 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbfdca3be0 0x55bbfdd0efb0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.113 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.083+0000 7f4bad486b80 1 -- 172.21.15.39:0/2335961948 >> 172.21.15.39:0/2335961948 conn(0x55bbfdc99da0 msgr2=0x55bbfdca53e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:40.113 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.083+0000 7f4b7eff5700 0 client.15828 ms_handle_reset on v2:172.21.15.155:6834/3854856286 2023-07-21T18:42:40.113 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.083+0000 7f4bad486b80 1 -- 172.21.15.39:0/2335961948 shutdown_connections 2023-07-21T18:42:40.113 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.083+0000 7f4bad486b80 1 -- 172.21.15.39:0/2335961948 wait complete. 2023-07-21T18:42:40.113 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.083+0000 7f4bad486b80 20 client.15828 trim_cache size 0 max 16384 2023-07-21T18:42:40.114 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2023-07-21T18:42:40.114 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.084+0000 7f4ba1300700 1 -- 172.21.15.39:0/3363041117 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f4b840a13e0 msgr2=0x7f4b840a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:40.115 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.084+0000 7f4ba1300700 1 --2- 172.21.15.39:0/3363041117 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f4b840a13e0 0x7f4b840a3890 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c0fead0 tx=0x7f4b90009310 comp rx=0 tx=0).stop 2023-07-21T18:42:40.115 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.084+0000 7f4ba1300700 1 -- 172.21.15.39:0/3363041117 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b9c0ff580 msgr2=0x7f4b9c196af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:40.115 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.084+0000 7f4ba1300700 1 --2- 172.21.15.39:0/3363041117 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b9c0ff580 0x7f4b9c196af0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c009230 tx=0x7f4b8c00fa00 comp rx=0 tx=0).stop 2023-07-21T18:42:40.115 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.084+0000 7f4ba1300700 1 -- 172.21.15.39:0/3363041117 shutdown_connections 2023-07-21T18:42:40.115 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.084+0000 7f4ba1300700 1 --2- 172.21.15.39:0/3363041117 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f4b840a13e0 0x7f4b840a3890 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.115 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.084+0000 7f4ba1300700 1 --2- 172.21.15.39:0/3363041117 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b9c0ff580 0x7f4b9c196af0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.116 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.084+0000 7f4ba1300700 1 --2- 172.21.15.39:0/3363041117 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b9c103ce0 0x7f4b9c1a1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.116 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.084+0000 7f4ba1300700 1 --2- 172.21.15.39:0/3363041117 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b9c0fff20 0x7f4b9c197030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.116 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.084+0000 7f4ba1300700 1 -- 172.21.15.39:0/3363041117 >> 172.21.15.39:0/3363041117 conn(0x7f4b9c076200 msgr2=0x7f4b9c0fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:40.116 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.084+0000 7f4ba1300700 1 -- 172.21.15.39:0/3363041117 shutdown_connections 2023-07-21T18:42:40.116 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.084+0000 7f4ba1300700 1 -- 172.21.15.39:0/3363041117 wait complete. 2023-07-21T18:42:40.122 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-21T18:42:40.319 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.297+0000 7f4486a45700 1 -- 172.21.15.39:0/2513636173 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44800ff380 msgr2=0x7f448010b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:40.319 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.297+0000 7f4486a45700 1 --2- 172.21.15.39:0/2513636173 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44800ff380 0x7f448010b810 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f447800b0a0 tx=0x7f447800b3b0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.319 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.297+0000 7f4486a45700 1 -- 172.21.15.39:0/2513636173 shutdown_connections 2023-07-21T18:42:40.319 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.297+0000 7f4486a45700 1 --2- 172.21.15.39:0/2513636173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44801049e0 0x7f4480104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.320 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.297+0000 7f4486a45700 1 --2- 172.21.15.39:0/2513636173 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44800ff380 0x7f448010b810 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.320 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.297+0000 7f4486a45700 1 --2- 172.21.15.39:0/2513636173 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44800fe9d0 0x7f44800fee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.320 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.297+0000 7f4486a45700 1 -- 172.21.15.39:0/2513636173 >> 172.21.15.39:0/2513636173 conn(0x7f44800fa4b0 msgr2=0x7f44800fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:40.320 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.297+0000 7f4486a45700 1 -- 172.21.15.39:0/2513636173 shutdown_connections 2023-07-21T18:42:40.320 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.297+0000 7f4486a45700 1 -- 172.21.15.39:0/2513636173 wait complete. 2023-07-21T18:42:40.322 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.298+0000 7f4486a45700 1 Processor -- start 2023-07-21T18:42:40.322 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.298+0000 7f4486a45700 1 -- start start 2023-07-21T18:42:40.322 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.299+0000 7f4486a45700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44800fe9d0 0x7f448019dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.322 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.299+0000 7f4486a45700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44800ff380 0x7f448019e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.322 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.299+0000 7f4486a45700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44801049e0 0x7f4480197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.322 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.299+0000 7f4486a45700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4480198210 con 0x7f44800ff380 2023-07-21T18:42:40.323 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.299+0000 7f4486a45700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4480198380 con 0x7f44801049e0 2023-07-21T18:42:40.323 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.299+0000 7f4486a45700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4480198540 con 0x7f44800fe9d0 2023-07-21T18:42:40.323 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.299+0000 7f447f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44800ff380 0x7f448019e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:40.323 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.299+0000 7f447f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44800ff380 0x7f448019e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58142/0 (socket says 172.21.15.39:58142) 2023-07-21T18:42:40.323 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.299+0000 7f447f7fe700 1 -- 172.21.15.39:0/4014816537 learned_addr learned my addr 172.21.15.39:0/4014816537 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:40.323 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.299+0000 7f447ffff700 1 --2- 172.21.15.39:0/4014816537 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44800fe9d0 0x7f448019dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:40.323 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.299+0000 7f447f7fe700 1 -- 172.21.15.39:0/4014816537 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44800fe9d0 msgr2=0x7f448019dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:40.323 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.299+0000 7f447f7fe700 1 --2- 172.21.15.39:0/4014816537 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44800fe9d0 0x7f448019dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.324 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.299+0000 7f447f7fe700 1 -- 172.21.15.39:0/4014816537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44801049e0 msgr2=0x7f4480197ba0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-21T18:42:40.324 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.299+0000 7f447f7fe700 1 --2- 172.21.15.39:0/4014816537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44801049e0 0x7f4480197ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.324 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.299+0000 7f447f7fe700 1 -- 172.21.15.39:0/4014816537 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4478009d00 con 0x7f44800ff380 2023-07-21T18:42:40.324 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.299+0000 7f447ffff700 1 --2- 172.21.15.39:0/4014816537 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44800fe9d0 0x7f448019dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:42:40.324 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.299+0000 7f447f7fe700 1 --2- 172.21.15.39:0/4014816537 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44800ff380 0x7f448019e020 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f447400e9a0 tx=0x7f447400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:40.324 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.300+0000 7f447d7fa700 1 -- 172.21.15.39:0/4014816537 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447400cbd0 con 0x7f44800ff380 2023-07-21T18:42:40.324 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.300+0000 7f4486a45700 1 -- 172.21.15.39:0/4014816537 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4480198820 con 0x7f44800ff380 2023-07-21T18:42:40.325 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.300+0000 7f447d7fa700 1 -- 172.21.15.39:0/4014816537 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f447400cd30 con 0x7f44800ff380 2023-07-21T18:42:40.325 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.300+0000 7f447d7fa700 1 -- 172.21.15.39:0/4014816537 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4474018680 con 0x7f44800ff380 2023-07-21T18:42:40.325 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.300+0000 7f4486a45700 1 -- 172.21.15.39:0/4014816537 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44801a4ad0 con 0x7f44800ff380 2023-07-21T18:42:40.326 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.305+0000 7f4492bcbb80 1 Processor -- start 2023-07-21T18:42:40.327 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.306+0000 7f447d7fa700 1 -- 172.21.15.39:0/4014816537 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f44740187e0 con 0x7f44800ff380 2023-07-21T18:42:40.328 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.307+0000 7f447d7fa700 1 --2- 172.21.15.39:0/4014816537 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f44680a13c0 0x7f44680a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.329 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.307+0000 7f447d7fa700 1 -- 172.21.15.39:0/4014816537 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f4474014070 con 0x7f44800ff380 2023-07-21T18:42:40.330 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.309+0000 7f4492bcbb80 1 -- start start 2023-07-21T18:42:40.331 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.309+0000 7f4484fe2700 1 --2- 172.21.15.39:0/4014816537 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f44680a13c0 0x7f44680a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:40.331 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.309+0000 7f4492bcbb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5588fa6b12d0 0x5588fa6b16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.331 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.309+0000 7f4492bcbb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588fa6b1be0 0x5588fa6c4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.331 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.309+0000 7f4492bcbb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588fa6c52e0 0x5588fa6c76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.331 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.309+0000 7f4492bcbb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588fa5b7cb0 con 0x5588fa6b1be0 2023-07-21T18:42:40.331 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.309+0000 7f4492bcbb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588fa60aa40 con 0x5588fa6b12d0 2023-07-21T18:42:40.331 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.309+0000 7f4492bcbb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588fa603ed0 con 0x5588fa6c52e0 2023-07-21T18:42:40.332 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.309+0000 7f447f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588fa6b1be0 0x5588fa6c4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:40.332 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.309+0000 7f447f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588fa6b1be0 0x5588fa6c4da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58146/0 (socket says 172.21.15.39:58146) 2023-07-21T18:42:40.333 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.309+0000 7f447f7fe700 1 -- 172.21.15.39:0/3065465209 learned_addr learned my addr 172.21.15.39:0/3065465209 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:40.333 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.310+0000 7f447f7fe700 1 -- 172.21.15.39:0/3065465209 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588fa6c52e0 msgr2=0x5588fa6c76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:40.333 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.310+0000 7f4484fe2700 1 --2- 172.21.15.39:0/3065465209 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588fa6c52e0 0x5588fa6c76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:40.333 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.310+0000 7f447f7fe700 1 --2- 172.21.15.39:0/3065465209 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588fa6c52e0 0x5588fa6c76c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.333 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.310+0000 7f447ffff700 1 --2- 172.21.15.39:0/3065465209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5588fa6b12d0 0x5588fa6b16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:40.333 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.310+0000 7f447f7fe700 1 -- 172.21.15.39:0/3065465209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5588fa6b12d0 msgr2=0x5588fa6b16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:40.333 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.310+0000 7f447f7fe700 1 --2- 172.21.15.39:0/3065465209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5588fa6b12d0 0x5588fa6b16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.334 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.310+0000 7f447f7fe700 1 -- 172.21.15.39:0/3065465209 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588fa431de0 con 0x5588fa6b1be0 2023-07-21T18:42:40.334 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.310+0000 7f447ffff700 1 --2- 172.21.15.39:0/3065465209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5588fa6b12d0 0x5588fa6b16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:42:40.334 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.310+0000 7f4484fe2700 1 --2- 172.21.15.39:0/4014816537 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f44680a13c0 0x7f44680a3870 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4478009610 tx=0x7f4478008e60 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:40.334 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.310+0000 7f447f7fe700 1 --2- 172.21.15.39:0/3065465209 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588fa6b1be0 0x5588fa6c4da0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f4474099dd0 tx=0x7f447401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adaf7f7f82fbd8b4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:40.334 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.310+0000 7f44667fc700 1 -- 172.21.15.39:0/3065465209 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447409a0d0 con 0x5588fa6b1be0 2023-07-21T18:42:40.334 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.310+0000 7f44667fc700 1 -- 172.21.15.39:0/3065465209 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f447409e100 con 0x5588fa6b1be0 2023-07-21T18:42:40.334 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.310+0000 7f44667fc700 1 -- 172.21.15.39:0/3065465209 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44740a22d0 con 0x5588fa6b1be0 2023-07-21T18:42:40.337 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.311+0000 7f4492bcbb80 1 -- 172.21.15.39:0/3065465209 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588fa6b1be0 msgr2=0x5588fa6c4da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:40.337 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.311+0000 7f4492bcbb80 1 --2- 172.21.15.39:0/3065465209 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588fa6b1be0 0x5588fa6c4da0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f4474099dd0 tx=0x7f447401f000 comp rx=0 tx=0).stop 2023-07-21T18:42:40.337 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.311+0000 7f4492bcbb80 1 -- 172.21.15.39:0/3065465209 shutdown_connections 2023-07-21T18:42:40.337 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.311+0000 7f4492bcbb80 1 --2- 172.21.15.39:0/3065465209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5588fa6b12d0 0x5588fa6b16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.337 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.311+0000 7f4492bcbb80 1 --2- 172.21.15.39:0/3065465209 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588fa6c52e0 0x5588fa6c76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.337 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.311+0000 7f4492bcbb80 1 --2- 172.21.15.39:0/3065465209 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588fa6b1be0 0x5588fa6c4da0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.338 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.311+0000 7f4492bcbb80 1 -- 172.21.15.39:0/3065465209 >> 172.21.15.39:0/3065465209 conn(0x5588fa6a7da0 msgr2=0x5588fa6ad6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:40.338 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.311+0000 7f4492bcbb80 1 -- 172.21.15.39:0/3065465209 shutdown_connections 2023-07-21T18:42:40.338 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.311+0000 7f4492bcbb80 1 -- 172.21.15.39:0/3065465209 wait complete. 2023-07-21T18:42:40.338 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.312+0000 7f4492bcbb80 1 -- start start 2023-07-21T18:42:40.338 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.312+0000 7f4492bcbb80 1 Processor -- start 2023-07-21T18:42:40.338 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.312+0000 7f4492bcbb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588fa6b12d0 0x5588fa6be5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.338 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.312+0000 7f4492bcbb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588fa6b1be0 0x5588fa6beae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.338 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.312+0000 7f4492bcbb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5588fa6c52e0 0x5588fa6c4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.339 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.312+0000 7f4492bcbb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588fa5a3ac0 con 0x5588fa6b1be0 2023-07-21T18:42:40.339 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.312+0000 7f4492bcbb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588fa5a37c0 con 0x5588fa6c52e0 2023-07-21T18:42:40.339 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.312+0000 7f4492bcbb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588fa487920 con 0x5588fa6b12d0 2023-07-21T18:42:40.339 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.312+0000 7f4492bcbb80 10 client.0 _ll_register_callbacks cb 0x5588fa5a1760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:42:40.339 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.313+0000 7f4484fe2700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5588fa6c52e0 0x5588fa6c4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:40.339 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.313+0000 7f447ffff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588fa6b12d0 0x5588fa6be5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:40.339 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.313+0000 7f4484fe2700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5588fa6c52e0 0x5588fa6c4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:42518/0 (socket says 172.21.15.39:42518) 2023-07-21T18:42:40.339 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.313+0000 7f4484fe2700 1 -- 172.21.15.39:0/1315229211 learned_addr learned my addr 172.21.15.39:0/1315229211 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:40.340 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.313+0000 7f447f7fe700 1 --2- 172.21.15.39:0/1315229211 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588fa6b1be0 0x5588fa6beae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:40.340 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.313+0000 7f447ffff700 1 -- 172.21.15.39:0/1315229211 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5588fa6c52e0 msgr2=0x5588fa6c4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:40.340 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.313+0000 7f447ffff700 1 --2- 172.21.15.39:0/1315229211 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5588fa6c52e0 0x5588fa6c4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.340 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.313+0000 7f447ffff700 1 -- 172.21.15.39:0/1315229211 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588fa6b1be0 msgr2=0x5588fa6beae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:40.340 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.313+0000 7f447ffff700 1 --2- 172.21.15.39:0/1315229211 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588fa6b1be0 0x5588fa6beae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.340 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.313+0000 7f447ffff700 1 -- 172.21.15.39:0/1315229211 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44801a4ad0 con 0x5588fa6b12d0 2023-07-21T18:42:40.340 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.313+0000 7f447ffff700 1 --2- 172.21.15.39:0/1315229211 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588fa6b12d0 0x5588fa6be5a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f44700106d0 tx=0x7f44700109e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3a89b25d6246286 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:40.341 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.313+0000 7f445f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2023-07-21T18:42:40.341 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.314+0000 7f445f7fe700 1 -- 172.21.15.39:0/1315229211 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4470004a30 con 0x5588fa6b12d0 2023-07-21T18:42:40.341 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.314+0000 7f445f7fe700 1 -- 172.21.15.39:0/1315229211 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f4470004b90 con 0x5588fa6b12d0 2023-07-21T18:42:40.341 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.314+0000 7f4492bcbb80 1 -- 172.21.15.39:0/1315229211 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5588fa55d4f0 con 0x5588fa6b12d0 2023-07-21T18:42:40.341 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.314+0000 7f445f7fe700 1 -- 172.21.15.39:0/1315229211 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4470004d50 con 0x5588fa6b12d0 2023-07-21T18:42:40.341 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.314+0000 7f4492bcbb80 10 client.25396 fetch_fsmap learned FSMap version 26 2023-07-21T18:42:40.341 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.314+0000 7f4492bcbb80 1 -- 172.21.15.39:0/1315229211 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5588fa41ac60 con 0x5588fa6b12d0 2023-07-21T18:42:40.341 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.314+0000 7f445f7fe700 1 -- 172.21.15.39:0/1315229211 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5588fa55d4f0 con 0x5588fa6b12d0 2023-07-21T18:42:40.342 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.314+0000 7f445f7fe700 1 -- 172.21.15.39:0/1315229211 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4470019030 con 0x5588fa6b12d0 2023-07-21T18:42:40.342 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.314+0000 7f4492bcbb80 10 client.25396 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T18:42:40.342 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.314+0000 7f4492bcbb80 10 client.25396 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T18:42:40.342 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.314+0000 7f4492bcbb80 20 client.25396 populate_metadata read hostname 'smithi039' 2023-07-21T18:42:40.342 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.314+0000 7f4492bcbb80 1 --2- 172.21.15.39:0/1315229211 >> [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] conn(0x5588fa6dd340 0x5588fa6df750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.342 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.314+0000 7f4492bcbb80 4 client.25396 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:42:40.342 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.314+0000 7f4492bcbb80 1 -- 172.21.15.39:0/1315229211 --> [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5588fa5fceb0 con 0x5588fa6dd340 2023-07-21T18:42:40.342 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.315+0000 7f447f7fe700 1 --2- 172.21.15.39:0/1315229211 >> [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] conn(0x5588fa6dd340 0x5588fa6df750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:40.343 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.315+0000 7f445f7fe700 10 client.25396 ms_handle_connect on v2:172.21.15.193:6838/3148488639 2023-07-21T18:42:40.343 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.315+0000 7f447f7fe700 1 --2- 172.21.15.39:0/1315229211 >> [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] conn(0x5588fa6dd340 0x5588fa6df750 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a8ce4ab45bde6e5f server_cookie=acb518fc98252cd5 in_seq=0 out_seq=0 2023-07-21T18:42:40.344 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.316+0000 7f445f7fe700 1 -- 172.21.15.39:0/1315229211 <== mds.2 v2:172.21.15.193:6838/3148488639 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5588fa5fceb0 con 0x5588fa6dd340 2023-07-21T18:42:40.344 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.316+0000 7f445f7fe700 10 client.25396 handle_command_reply: tid=0 2023-07-21T18:42:40.344 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.316+0000 7f4492bcbb80 1 client.25396 shutdown 2023-07-21T18:42:40.344 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.316+0000 7f4492bcbb80 1 -- 172.21.15.39:0/1315229211 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588fa6b12d0 msgr2=0x5588fa6be5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:40.344 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.316+0000 7f4492bcbb80 1 --2- 172.21.15.39:0/1315229211 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588fa6b12d0 0x5588fa6be5a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f44700106d0 tx=0x7f44700109e0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.344 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.316+0000 7f4492bcbb80 1 -- 172.21.15.39:0/1315229211 shutdown_connections 2023-07-21T18:42:40.345 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.316+0000 7f4492bcbb80 1 --2- 172.21.15.39:0/1315229211 >> [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] conn(0x5588fa6dd340 0x5588fa6df750 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.345 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.316+0000 7f4492bcbb80 1 --2- 172.21.15.39:0/1315229211 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5588fa6c52e0 0x5588fa6c4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.345 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.316+0000 7f4492bcbb80 1 --2- 172.21.15.39:0/1315229211 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588fa6b12d0 0x5588fa6be5a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.345 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.316+0000 7f4492bcbb80 1 --2- 172.21.15.39:0/1315229211 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588fa6b1be0 0x5588fa6beae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.345 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.316+0000 7f4492bcbb80 1 -- 172.21.15.39:0/1315229211 >> 172.21.15.39:0/1315229211 conn(0x5588fa6a7da0 msgr2=0x5588fa6ad290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:40.345 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.316+0000 7f445f7fe700 0 client.25396 ms_handle_reset on v2:172.21.15.193:6838/3148488639 2023-07-21T18:42:40.345 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.316+0000 7f4492bcbb80 1 -- 172.21.15.39:0/1315229211 shutdown_connections 2023-07-21T18:42:40.345 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.316+0000 7f4492bcbb80 1 -- 172.21.15.39:0/1315229211 wait complete. 2023-07-21T18:42:40.346 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.316+0000 7f4492bcbb80 20 client.25396 trim_cache size 0 max 16384 2023-07-21T18:42:40.364 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.342+0000 7f4492bcbb80 1 Processor -- start 2023-07-21T18:42:40.364 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.342+0000 7f4492bcbb80 1 -- start start 2023-07-21T18:42:40.364 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.342+0000 7f4492bcbb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5588fa6b12d0 0x5588fa6be1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.364 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.342+0000 7f4492bcbb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588fa6b1be0 0x5588fa6be6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.364 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.342+0000 7f4492bcbb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588fa6c52e0 0x5588fa6c66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.365 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.342+0000 7f4492bcbb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588fa6c4910 con 0x5588fa6c52e0 2023-07-21T18:42:40.365 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.342+0000 7f4492bcbb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588fa6ae4c0 con 0x5588fa6b12d0 2023-07-21T18:42:40.365 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.342+0000 7f4492bcbb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588fa6ac490 con 0x5588fa6b1be0 2023-07-21T18:42:40.366 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.343+0000 7f447ffff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5588fa6b12d0 0x5588fa6be1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:40.366 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.343+0000 7f447ffff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5588fa6b12d0 0x5588fa6be1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:42524/0 (socket says 172.21.15.39:42524) 2023-07-21T18:42:40.366 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.343+0000 7f447ffff700 1 -- 172.21.15.39:0/3159288551 learned_addr learned my addr 172.21.15.39:0/3159288551 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:40.366 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.343+0000 7f4484fe2700 1 --2- 172.21.15.39:0/3159288551 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588fa6c52e0 0x5588fa6c66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:40.366 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.343+0000 7f447ffff700 1 -- 172.21.15.39:0/3159288551 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588fa6b1be0 msgr2=0x5588fa6be6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:40.366 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.343+0000 7f447ffff700 1 --2- 172.21.15.39:0/3159288551 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588fa6b1be0 0x5588fa6be6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.367 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.343+0000 7f447ffff700 1 -- 172.21.15.39:0/3159288551 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588fa6c52e0 msgr2=0x5588fa6c66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:40.367 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.343+0000 7f447ffff700 1 --2- 172.21.15.39:0/3159288551 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588fa6c52e0 0x5588fa6c66c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.367 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.343+0000 7f447ffff700 1 -- 172.21.15.39:0/3159288551 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588fa458250 con 0x5588fa6b12d0 2023-07-21T18:42:40.367 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.343+0000 7f447ffff700 1 --2- 172.21.15.39:0/3159288551 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5588fa6b12d0 0x5588fa6be1a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4470005260 tx=0x7f4470010d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23b1c678c9f3b03e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:40.367 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.344+0000 7f4466ffd700 1 -- 172.21.15.39:0/3159288551 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4470005680 con 0x5588fa6b12d0 2023-07-21T18:42:40.367 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.344+0000 7f4466ffd700 1 -- 172.21.15.39:0/3159288551 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4470015070 con 0x5588fa6b12d0 2023-07-21T18:42:40.367 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.344+0000 7f4466ffd700 1 -- 172.21.15.39:0/3159288551 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4470004b90 con 0x5588fa6b12d0 2023-07-21T18:42:40.369 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.345+0000 7f4492bcbb80 1 -- 172.21.15.39:0/3159288551 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5588fa6b12d0 msgr2=0x5588fa6be1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:40.370 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.345+0000 7f4492bcbb80 1 --2- 172.21.15.39:0/3159288551 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5588fa6b12d0 0x5588fa6be1a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4470005260 tx=0x7f4470010d30 comp rx=0 tx=0).stop 2023-07-21T18:42:40.370 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.345+0000 7f4492bcbb80 1 -- 172.21.15.39:0/3159288551 shutdown_connections 2023-07-21T18:42:40.370 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.345+0000 7f4492bcbb80 1 --2- 172.21.15.39:0/3159288551 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5588fa6b12d0 0x5588fa6be1a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.370 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.345+0000 7f4492bcbb80 1 --2- 172.21.15.39:0/3159288551 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588fa6c52e0 0x5588fa6c66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.370 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.345+0000 7f4492bcbb80 1 --2- 172.21.15.39:0/3159288551 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588fa6b1be0 0x5588fa6be6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.370 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.345+0000 7f4492bcbb80 1 -- 172.21.15.39:0/3159288551 >> 172.21.15.39:0/3159288551 conn(0x5588fa6a7da0 msgr2=0x5588fa6b4ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:40.370 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.345+0000 7f4492bcbb80 1 -- 172.21.15.39:0/3159288551 shutdown_connections 2023-07-21T18:42:40.371 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.345+0000 7f4492bcbb80 1 -- 172.21.15.39:0/3159288551 wait complete. 2023-07-21T18:42:40.371 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.345+0000 7f4492bcbb80 1 -- start start 2023-07-21T18:42:40.371 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.345+0000 7f4492bcbb80 1 Processor -- start 2023-07-21T18:42:40.371 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.346+0000 7f4492bcbb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5588fa6b12d0 0x5588fa6c4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.371 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.346+0000 7f4492bcbb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588fa6b1be0 0x5588fa6c1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.371 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.346+0000 7f4492bcbb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588fa6c52e0 0x5588fa6c1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.371 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.346+0000 7f4492bcbb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588fa6c1b90 con 0x5588fa6b1be0 2023-07-21T18:42:40.371 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.346+0000 7f4492bcbb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588fa6c1d00 con 0x5588fa6b12d0 2023-07-21T18:42:40.372 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.346+0000 7f4492bcbb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588fa6c1e70 con 0x5588fa6c52e0 2023-07-21T18:42:40.372 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.346+0000 7f4492bcbb80 10 client.0 _ll_register_callbacks cb 0x7f447000ebd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:42:40.372 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.346+0000 7f447f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588fa6b1be0 0x5588fa6c1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:40.372 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.346+0000 7f447f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588fa6b1be0 0x5588fa6c1110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58168/0 (socket says 172.21.15.39:58168) 2023-07-21T18:42:40.372 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.346+0000 7f447f7fe700 1 -- 172.21.15.39:0/540051619 learned_addr learned my addr 172.21.15.39:0/540051619 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:40.373 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.346+0000 7f4484fe2700 1 --2- 172.21.15.39:0/540051619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588fa6c52e0 0x5588fa6c1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:40.373 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.346+0000 7f447ffff700 1 --2- 172.21.15.39:0/540051619 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5588fa6b12d0 0x5588fa6c4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:40.373 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.346+0000 7f447f7fe700 1 -- 172.21.15.39:0/540051619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588fa6c52e0 msgr2=0x5588fa6c1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:40.373 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.346+0000 7f447f7fe700 1 --2- 172.21.15.39:0/540051619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588fa6c52e0 0x5588fa6c1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.373 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.346+0000 7f447f7fe700 1 -- 172.21.15.39:0/540051619 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5588fa6b12d0 msgr2=0x5588fa6c4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:40.373 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.346+0000 7f447f7fe700 1 --2- 172.21.15.39:0/540051619 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5588fa6b12d0 0x5588fa6c4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.373 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.346+0000 7f447f7fe700 1 -- 172.21.15.39:0/540051619 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5588fa41ac60 con 0x5588fa6b1be0 2023-07-21T18:42:40.373 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.346+0000 7f447ffff700 1 --2- 172.21.15.39:0/540051619 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5588fa6b12d0 0x5588fa6c4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:42:40.374 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.347+0000 7f447f7fe700 1 --2- 172.21.15.39:0/540051619 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588fa6b1be0 0x5588fa6c1110 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f447400e970 tx=0x7f447409a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d67f42fd201440a4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:40.374 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.347+0000 7f445f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:42:40.374 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.347+0000 7f445f7fe700 1 -- 172.21.15.39:0/540051619 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44740b6070 con 0x5588fa6b1be0 2023-07-21T18:42:40.374 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.347+0000 7f445f7fe700 1 -- 172.21.15.39:0/540051619 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f44740b82b0 con 0x5588fa6b1be0 2023-07-21T18:42:40.374 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.347+0000 7f4492bcbb80 1 -- 172.21.15.39:0/540051619 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f447409e100 con 0x5588fa6b1be0 2023-07-21T18:42:40.374 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.347+0000 7f445f7fe700 1 -- 172.21.15.39:0/540051619 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44740b8470 con 0x5588fa6b1be0 2023-07-21T18:42:40.374 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.347+0000 7f4492bcbb80 10 client.15846 fetch_fsmap learned FSMap version 26 2023-07-21T18:42:40.374 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.347+0000 7f4492bcbb80 1 -- 172.21.15.39:0/540051619 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5588fa4302b0 con 0x5588fa6b1be0 2023-07-21T18:42:40.375 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.347+0000 7f445f7fe700 1 -- 172.21.15.39:0/540051619 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4474063080 con 0x5588fa6b1be0 2023-07-21T18:42:40.375 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.348+0000 7f445f7fe700 1 -- 172.21.15.39:0/540051619 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f44740cab00 con 0x5588fa6b1be0 2023-07-21T18:42:40.375 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.348+0000 7f4492bcbb80 10 client.15846 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T18:42:40.375 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.348+0000 7f4492bcbb80 10 client.15846 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T18:42:40.375 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.348+0000 7f4492bcbb80 20 client.15846 populate_metadata read hostname 'smithi039' 2023-07-21T18:42:40.375 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.348+0000 7f4492bcbb80 1 --2- 172.21.15.39:0/540051619 >> [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] conn(0x5588fa732480 0x5588fa734890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.375 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.348+0000 7f4492bcbb80 4 client.15846 mds_command: new command op to 24487 tid=0[{"prefix": "damage ls"}] 2023-07-21T18:42:40.375 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.348+0000 7f4492bcbb80 1 -- 172.21.15.39:0/540051619 --> [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5588fa5bf050 con 0x5588fa732480 2023-07-21T18:42:40.376 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.348+0000 7f447ffff700 1 --2- 172.21.15.39:0/540051619 >> [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] conn(0x5588fa732480 0x5588fa734890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:40.377 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.349+0000 7f445f7fe700 10 client.15846 ms_handle_connect on v2:172.21.15.193:6838/3148488639 2023-07-21T18:42:40.377 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.349+0000 7f447ffff700 1 --2- 172.21.15.39:0/540051619 >> [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] conn(0x5588fa732480 0x5588fa734890 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=feddec2e3c05e882 server_cookie=a429ae4ac83f369d in_seq=0 out_seq=0 2023-07-21T18:42:40.377 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.349+0000 7f445f7fe700 1 -- 172.21.15.39:0/540051619 <== mds.2 v2:172.21.15.193:6838/3148488639 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5588fa5bf050 con 0x5588fa732480 2023-07-21T18:42:40.377 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.349+0000 7f445f7fe700 10 client.15846 handle_command_reply: tid=0 2023-07-21T18:42:40.377 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.349+0000 7f4492bcbb80 1 client.15846 shutdown 2023-07-21T18:42:40.377 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.349+0000 7f4492bcbb80 1 -- 172.21.15.39:0/540051619 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588fa6b1be0 msgr2=0x5588fa6c1110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:40.377 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.349+0000 7f4492bcbb80 1 --2- 172.21.15.39:0/540051619 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588fa6b1be0 0x5588fa6c1110 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f447400e970 tx=0x7f447409a650 comp rx=0 tx=0).stop 2023-07-21T18:42:40.378 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.349+0000 7f4492bcbb80 1 -- 172.21.15.39:0/540051619 shutdown_connections 2023-07-21T18:42:40.378 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.349+0000 7f4492bcbb80 1 --2- 172.21.15.39:0/540051619 >> [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] conn(0x5588fa732480 0x5588fa734890 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.378 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.349+0000 7f4492bcbb80 1 --2- 172.21.15.39:0/540051619 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5588fa6b12d0 0x5588fa6c4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.378 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.349+0000 7f4492bcbb80 1 --2- 172.21.15.39:0/540051619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588fa6c52e0 0x5588fa6c1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.378 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.349+0000 7f4492bcbb80 1 --2- 172.21.15.39:0/540051619 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588fa6b1be0 0x5588fa6c1110 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.378 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.349+0000 7f4492bcbb80 1 -- 172.21.15.39:0/540051619 >> 172.21.15.39:0/540051619 conn(0x5588fa6a7da0 msgr2=0x5588fa6b3c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:40.378 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.349+0000 7f445f7fe700 0 client.15846 ms_handle_reset on v2:172.21.15.193:6838/3148488639 2023-07-21T18:42:40.378 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.349+0000 7f4492bcbb80 1 -- 172.21.15.39:0/540051619 shutdown_connections 2023-07-21T18:42:40.379 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.349+0000 7f4492bcbb80 1 -- 172.21.15.39:0/540051619 wait complete. 2023-07-21T18:42:40.379 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.349+0000 7f4492bcbb80 20 client.15846 trim_cache size 0 max 16384 2023-07-21T18:42:40.379 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2023-07-21T18:42:40.380 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.350+0000 7f4486a45700 1 -- 172.21.15.39:0/4014816537 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f44680a13c0 msgr2=0x7f44680a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:40.380 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.350+0000 7f4486a45700 1 --2- 172.21.15.39:0/4014816537 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f44680a13c0 0x7f44680a3870 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4478009610 tx=0x7f4478008e60 comp rx=0 tx=0).stop 2023-07-21T18:42:40.380 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.350+0000 7f4486a45700 1 -- 172.21.15.39:0/4014816537 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44800ff380 msgr2=0x7f448019e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:40.380 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.350+0000 7f4486a45700 1 --2- 172.21.15.39:0/4014816537 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44800ff380 0x7f448019e020 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f447400e9a0 tx=0x7f447400ed60 comp rx=0 tx=0).stop 2023-07-21T18:42:40.380 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.350+0000 7f4486a45700 1 -- 172.21.15.39:0/4014816537 shutdown_connections 2023-07-21T18:42:40.380 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.350+0000 7f4486a45700 1 --2- 172.21.15.39:0/4014816537 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f44680a13c0 0x7f44680a3870 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.381 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.350+0000 7f4486a45700 1 --2- 172.21.15.39:0/4014816537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44801049e0 0x7f4480197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.381 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.350+0000 7f4486a45700 1 --2- 172.21.15.39:0/4014816537 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44800fe9d0 0x7f448019dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.381 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.350+0000 7f4486a45700 1 --2- 172.21.15.39:0/4014816537 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44800ff380 0x7f448019e020 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.381 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.350+0000 7f4486a45700 1 -- 172.21.15.39:0/4014816537 >> 172.21.15.39:0/4014816537 conn(0x7f44800fa4b0 msgr2=0x7f44800fba60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:40.381 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.350+0000 7f4486a45700 1 -- 172.21.15.39:0/4014816537 shutdown_connections 2023-07-21T18:42:40.381 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.350+0000 7f4486a45700 1 -- 172.21.15.39:0/4014816537 wait complete. 2023-07-21T18:42:40.387 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:42:40 smithi039 ceph-mon[96489]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-07-21T18:42:40.387 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:42:40 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/3923885066' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:42:40.387 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:42:40 smithi155 ceph-mon[97934]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-07-21T18:42:40.387 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:42:40 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/3923885066' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:42:40.388 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-21T18:42:40.585 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.563+0000 7f78a8c89700 1 -- 172.21.15.39:0/2229169331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78a4103ce0 msgr2=0x7f78a410b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:40.585 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.563+0000 7f78a8c89700 1 --2- 172.21.15.39:0/2229169331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78a4103ce0 0x7f78a410b8f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f789c00b0a0 tx=0x7f789c00b3b0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.586 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.563+0000 7f78a8c89700 1 -- 172.21.15.39:0/2229169331 shutdown_connections 2023-07-21T18:42:40.586 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.563+0000 7f78a8c89700 1 --2- 172.21.15.39:0/2229169331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78a4103ce0 0x7f78a410b8f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.586 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.563+0000 7f78a8c89700 1 --2- 172.21.15.39:0/2229169331 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78a40ff580 0x7f78a40ff950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.586 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.563+0000 7f78a8c89700 1 --2- 172.21.15.39:0/2229169331 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78a40fff20 0x7f78a4103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.586 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.563+0000 7f78a8c89700 1 -- 172.21.15.39:0/2229169331 >> 172.21.15.39:0/2229169331 conn(0x7f78a4076200 msgr2=0x7f78a4076600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:40.587 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.563+0000 7f78a8c89700 1 -- 172.21.15.39:0/2229169331 shutdown_connections 2023-07-21T18:42:40.587 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.563+0000 7f78a8c89700 1 -- 172.21.15.39:0/2229169331 wait complete. 2023-07-21T18:42:40.589 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.564+0000 7f78a8c89700 1 Processor -- start 2023-07-21T18:42:40.589 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.564+0000 7f78a8c89700 1 -- start start 2023-07-21T18:42:40.589 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.564+0000 7f78a8c89700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78a40ff580 0x7f78a4196cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.589 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.564+0000 7f78a8c89700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78a40fff20 0x7f78a4197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.589 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.564+0000 7f78a8c89700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78a4103ce0 0x7f78a41a1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.589 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.564+0000 7f78a8c89700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a4197b40 con 0x7f78a40fff20 2023-07-21T18:42:40.590 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.564+0000 7f78a8c89700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a41a2280 con 0x7f78a40ff580 2023-07-21T18:42:40.590 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.564+0000 7f78a8c89700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a4194b30 con 0x7f78a4103ce0 2023-07-21T18:42:40.590 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.565+0000 7f78a1d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78a40fff20 0x7f78a4197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:40.590 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.565+0000 7f78a1d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78a40fff20 0x7f78a4197200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58182/0 (socket says 172.21.15.39:58182) 2023-07-21T18:42:40.590 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.565+0000 7f78a1d9b700 1 -- 172.21.15.39:0/2193629753 learned_addr learned my addr 172.21.15.39:0/2193629753 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:40.590 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.565+0000 7f78a259c700 1 --2- 172.21.15.39:0/2193629753 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78a40ff580 0x7f78a4196cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:40.590 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.565+0000 7f78a1d9b700 1 -- 172.21.15.39:0/2193629753 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78a4103ce0 msgr2=0x7f78a41a1d40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-21T18:42:40.591 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.565+0000 7f78a1d9b700 1 --2- 172.21.15.39:0/2193629753 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78a4103ce0 0x7f78a41a1d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.591 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.565+0000 7f78a1d9b700 1 -- 172.21.15.39:0/2193629753 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78a40ff580 msgr2=0x7f78a4196cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:40.591 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.565+0000 7f78a1d9b700 1 --2- 172.21.15.39:0/2193629753 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78a40ff580 0x7f78a4196cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.591 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.565+0000 7f78a1d9b700 1 -- 172.21.15.39:0/2193629753 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7898009520 con 0x7f78a40fff20 2023-07-21T18:42:40.591 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.565+0000 7f78a1d9b700 1 --2- 172.21.15.39:0/2193629753 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78a40fff20 0x7f78a4197200 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f7898009fd0 tx=0x7f789800ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:40.591 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.565+0000 7f78937fe700 1 -- 172.21.15.39:0/2193629753 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789800cbd0 con 0x7f78a40fff20 2023-07-21T18:42:40.592 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.566+0000 7f78a8c89700 1 -- 172.21.15.39:0/2193629753 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f789c009d00 con 0x7f78a40fff20 2023-07-21T18:42:40.592 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.566+0000 7f78937fe700 1 -- 172.21.15.39:0/2193629753 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f78980044e0 con 0x7f78a40fff20 2023-07-21T18:42:40.592 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.566+0000 7f78937fe700 1 -- 172.21.15.39:0/2193629753 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7898003dc0 con 0x7f78a40fff20 2023-07-21T18:42:40.592 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.566+0000 7f78a8c89700 1 -- 172.21.15.39:0/2193629753 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78a41951e0 con 0x7f78a40fff20 2023-07-21T18:42:40.593 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.570+0000 7f78b4e0fb80 1 Processor -- start 2023-07-21T18:42:40.593 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.571+0000 7f78b4e0fb80 1 -- start start 2023-07-21T18:42:40.593 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.571+0000 7f78b4e0fb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556f071362d0 0x556f071366a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.593 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.571+0000 7f78b4e0fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f07136be0 0x556f07141590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.594 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.571+0000 7f78b4e0fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556f07141ad0 0x556f07143eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.594 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.571+0000 7f78b4e0fb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f0703ccb0 con 0x556f071362d0 2023-07-21T18:42:40.594 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.571+0000 7f78b4e0fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f0708fa40 con 0x556f07136be0 2023-07-21T18:42:40.594 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.571+0000 7f78b4e0fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f07088ed0 con 0x556f07141ad0 2023-07-21T18:42:40.594 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.571+0000 7f78a2d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556f071362d0 0x556f071366a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:40.595 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.571+0000 7f78a2d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556f071362d0 0x556f071366a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58186/0 (socket says 172.21.15.39:58186) 2023-07-21T18:42:40.595 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.571+0000 7f78a2d9d700 1 -- 172.21.15.39:0/1689637111 learned_addr learned my addr 172.21.15.39:0/1689637111 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:40.596 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.571+0000 7f78937fe700 1 -- 172.21.15.39:0/2193629753 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f7898004000 con 0x7f78a40fff20 2023-07-21T18:42:40.597 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.571+0000 7f78a2d9d700 1 -- 172.21.15.39:0/1689637111 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556f07141ad0 msgr2=0x556f07143eb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T18:42:40.597 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.571+0000 7f78a2d9d700 1 --2- 172.21.15.39:0/1689637111 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556f07141ad0 0x556f07143eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.597 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.571+0000 7f78a2d9d700 1 -- 172.21.15.39:0/1689637111 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f07136be0 msgr2=0x556f07141590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T18:42:40.597 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.572+0000 7f78a2d9d700 1 --2- 172.21.15.39:0/1689637111 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f07136be0 0x556f07141590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.597 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.572+0000 7f78a2d9d700 1 -- 172.21.15.39:0/1689637111 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f06eb6de0 con 0x556f071362d0 2023-07-21T18:42:40.597 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.572+0000 7f78a2d9d700 1 --2- 172.21.15.39:0/1689637111 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556f071362d0 0x556f071366a0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f789c009fd0 tx=0x7f789c009c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d3ce854de7f69af server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:40.597 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.572+0000 7f7890ff9700 1 -- 172.21.15.39:0/1689637111 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789c00e040 con 0x556f071362d0 2023-07-21T18:42:40.598 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.572+0000 7f78937fe700 1 --2- 172.21.15.39:0/2193629753 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f788c0a13e0 0x7f788c0a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.598 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.572+0000 7f78a259c700 1 --2- 172.21.15.39:0/2193629753 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f788c0a13e0 0x7f788c0a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:40.598 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.573+0000 7f78937fe700 1 -- 172.21.15.39:0/2193629753 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f7898014070 con 0x7f78a40fff20 2023-07-21T18:42:40.598 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.573+0000 7f78a259c700 1 --2- 172.21.15.39:0/2193629753 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f788c0a13e0 0x7f788c0a3890 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7894009a10 tx=0x7f7894009190 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:40.598 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.572+0000 7f7890ff9700 1 -- 172.21.15.39:0/1689637111 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f789c00bcb0 con 0x556f071362d0 2023-07-21T18:42:40.598 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.573+0000 7f7890ff9700 1 -- 172.21.15.39:0/1689637111 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789c005280 con 0x556f071362d0 2023-07-21T18:42:40.598 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.574+0000 7f78b4e0fb80 1 -- 172.21.15.39:0/1689637111 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556f071362d0 msgr2=0x556f071366a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:40.598 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.574+0000 7f78b4e0fb80 1 --2- 172.21.15.39:0/1689637111 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556f071362d0 0x556f071366a0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f789c009fd0 tx=0x7f789c009c40 comp rx=0 tx=0).stop 2023-07-21T18:42:40.599 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.574+0000 7f78b4e0fb80 1 -- 172.21.15.39:0/1689637111 shutdown_connections 2023-07-21T18:42:40.599 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.574+0000 7f78b4e0fb80 1 --2- 172.21.15.39:0/1689637111 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556f07141ad0 0x556f07143eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.599 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.574+0000 7f78b4e0fb80 1 --2- 172.21.15.39:0/1689637111 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556f071362d0 0x556f071366a0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.599 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.574+0000 7f78b4e0fb80 1 --2- 172.21.15.39:0/1689637111 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f07136be0 0x556f07141590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.599 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.574+0000 7f78b4e0fb80 1 -- 172.21.15.39:0/1689637111 >> 172.21.15.39:0/1689637111 conn(0x556f0712cda0 msgr2=0x556f071326a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:40.599 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.575+0000 7f78b4e0fb80 1 -- 172.21.15.39:0/1689637111 shutdown_connections 2023-07-21T18:42:40.599 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.575+0000 7f78b4e0fb80 1 -- 172.21.15.39:0/1689637111 wait complete. 2023-07-21T18:42:40.602 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.575+0000 7f78b4e0fb80 1 -- start start 2023-07-21T18:42:40.602 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.575+0000 7f78b4e0fb80 1 Processor -- start 2023-07-21T18:42:40.602 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.576+0000 7f78b4e0fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f071362d0 0x556f0713e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.602 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.576+0000 7f78b4e0fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556f07136be0 0x556f0713ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.602 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.576+0000 7f78b4e0fb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556f07141ad0 0x556f0713f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.603 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.576+0000 7f78b4e0fb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f07028ac0 con 0x556f07141ad0 2023-07-21T18:42:40.603 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.576+0000 7f78b4e0fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f070287c0 con 0x556f071362d0 2023-07-21T18:42:40.603 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.576+0000 7f78b4e0fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f06f0c920 con 0x556f07136be0 2023-07-21T18:42:40.603 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.576+0000 7f78b4e0fb80 10 client.0 _ll_register_callbacks cb 0x556f07026760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:42:40.603 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.576+0000 7f78a2d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556f07141ad0 0x556f0713f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:40.603 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.576+0000 7f78a2d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556f07141ad0 0x556f0713f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58196/0 (socket says 172.21.15.39:58196) 2023-07-21T18:42:40.604 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.576+0000 7f78a2d9d700 1 -- 172.21.15.39:0/2300740130 learned_addr learned my addr 172.21.15.39:0/2300740130 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:40.604 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.576+0000 7f78a259c700 1 --2- 172.21.15.39:0/2300740130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f071362d0 0x556f0713e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:40.604 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.576+0000 7f78a2d9d700 1 -- 172.21.15.39:0/2300740130 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556f07136be0 msgr2=0x556f0713ed70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T18:42:40.604 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.576+0000 7f78a2d9d700 1 --2- 172.21.15.39:0/2300740130 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556f07136be0 0x556f0713ed70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.604 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.576+0000 7f78a2d9d700 1 -- 172.21.15.39:0/2300740130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f071362d0 msgr2=0x556f0713e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:40.604 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.576+0000 7f78a2d9d700 1 --2- 172.21.15.39:0/2300740130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f071362d0 0x556f0713e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.605 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.576+0000 7f78a2d9d700 1 -- 172.21.15.39:0/2300740130 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78a410c130 con 0x556f07141ad0 2023-07-21T18:42:40.605 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.576+0000 7f78a259c700 1 --2- 172.21.15.39:0/2300740130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f071362d0 0x556f0713e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:42:40.605 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.576+0000 7f78a2d9d700 1 --2- 172.21.15.39:0/2300740130 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556f07141ad0 0x556f0713f2b0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f789c007670 tx=0x7f789c003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cdb598616391a3a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:40.605 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.577+0000 7f7889ffb700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:42:40.605 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.577+0000 7f7889ffb700 1 -- 172.21.15.39:0/2300740130 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789c00e040 con 0x556f07141ad0 2023-07-21T18:42:40.605 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.577+0000 7f7889ffb700 1 -- 172.21.15.39:0/2300740130 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f789c029030 con 0x556f07141ad0 2023-07-21T18:42:40.605 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.577+0000 7f78b4e0fb80 1 -- 172.21.15.39:0/2300740130 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556f06fe24f0 con 0x556f07141ad0 2023-07-21T18:42:40.606 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.577+0000 7f7889ffb700 1 -- 172.21.15.39:0/2300740130 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f789c0076d0 con 0x556f07141ad0 2023-07-21T18:42:40.606 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.577+0000 7f78b4e0fb80 10 client.15855 fetch_fsmap learned FSMap version 26 2023-07-21T18:42:40.606 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.577+0000 7f78b4e0fb80 1 -- 172.21.15.39:0/2300740130 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x556f06e9fc60 con 0x556f07141ad0 2023-07-21T18:42:40.606 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.577+0000 7f7889ffb700 1 -- 172.21.15.39:0/2300740130 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556f06fe24f0 con 0x556f07141ad0 2023-07-21T18:42:40.606 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.578+0000 7f7889ffb700 1 -- 172.21.15.39:0/2300740130 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f789c019040 con 0x556f07141ad0 2023-07-21T18:42:40.606 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.578+0000 7f78b4e0fb80 10 client.15855 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T18:42:40.606 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.578+0000 7f78b4e0fb80 10 client.15855 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T18:42:40.607 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.578+0000 7f78b4e0fb80 20 client.15855 populate_metadata read hostname 'smithi039' 2023-07-21T18:42:40.607 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.578+0000 7f78b4e0fb80 1 --2- 172.21.15.39:0/2300740130 >> [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] conn(0x556f07150fb0 0x556f071533c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.607 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.578+0000 7f78b4e0fb80 4 client.15855 mds_command: new command op to 24500 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:42:40.607 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.578+0000 7f78b4e0fb80 1 -- 172.21.15.39:0/2300740130 --> [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556f07081eb0 con 0x556f07150fb0 2023-07-21T18:42:40.607 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.579+0000 7f78a259c700 1 --2- 172.21.15.39:0/2300740130 >> [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] conn(0x556f07150fb0 0x556f071533c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:40.607 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.579+0000 7f7889ffb700 10 client.15855 ms_handle_connect on v2:172.21.15.155:6836/947670056 2023-07-21T18:42:40.607 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.579+0000 7f78a259c700 1 --2- 172.21.15.39:0/2300740130 >> [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] conn(0x556f07150fb0 0x556f071533c0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5917ae74df1faece server_cookie=16f3c4846006f4ff in_seq=0 out_seq=0 2023-07-21T18:42:40.607 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.580+0000 7f7889ffb700 1 -- 172.21.15.39:0/2300740130 <== mds.1 v2:172.21.15.155:6836/947670056 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556f07081eb0 con 0x556f07150fb0 2023-07-21T18:42:40.608 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.580+0000 7f7889ffb700 10 client.15855 handle_command_reply: tid=0 2023-07-21T18:42:40.608 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.580+0000 7f78b4e0fb80 1 client.15855 shutdown 2023-07-21T18:42:40.608 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.580+0000 7f78b4e0fb80 1 -- 172.21.15.39:0/2300740130 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556f07141ad0 msgr2=0x556f0713f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:40.608 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.580+0000 7f78b4e0fb80 1 --2- 172.21.15.39:0/2300740130 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556f07141ad0 0x556f0713f2b0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f789c007670 tx=0x7f789c003f90 comp rx=0 tx=0).stop 2023-07-21T18:42:40.608 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.580+0000 7f78b4e0fb80 1 -- 172.21.15.39:0/2300740130 shutdown_connections 2023-07-21T18:42:40.608 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.580+0000 7f78b4e0fb80 1 --2- 172.21.15.39:0/2300740130 >> [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] conn(0x556f07150fb0 0x556f071533c0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.608 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.580+0000 7f78b4e0fb80 1 --2- 172.21.15.39:0/2300740130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f071362d0 0x556f0713e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.609 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.580+0000 7f78b4e0fb80 1 --2- 172.21.15.39:0/2300740130 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556f07141ad0 0x556f0713f2b0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.609 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.580+0000 7f78b4e0fb80 1 --2- 172.21.15.39:0/2300740130 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556f07136be0 0x556f0713ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.609 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.580+0000 7f78b4e0fb80 1 -- 172.21.15.39:0/2300740130 >> 172.21.15.39:0/2300740130 conn(0x556f0712cda0 msgr2=0x556f07132230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:40.609 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.580+0000 7f7889ffb700 0 client.15855 ms_handle_reset on v2:172.21.15.155:6836/947670056 2023-07-21T18:42:40.609 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.582+0000 7f78b4e0fb80 1 -- 172.21.15.39:0/2300740130 shutdown_connections 2023-07-21T18:42:40.610 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.582+0000 7f78b4e0fb80 1 -- 172.21.15.39:0/2300740130 wait complete. 2023-07-21T18:42:40.610 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.582+0000 7f78b4e0fb80 20 client.15855 trim_cache size 0 max 16384 2023-07-21T18:42:40.618 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:42:40 smithi193 ceph-mon[98204]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-07-21T18:42:40.619 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:42:40 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/3923885066' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:42:40.629 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.607+0000 7f78b4e0fb80 1 Processor -- start 2023-07-21T18:42:40.629 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.607+0000 7f78b4e0fb80 1 -- start start 2023-07-21T18:42:40.630 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.607+0000 7f78b4e0fb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556f071362d0 0x556f07148510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.631 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.608+0000 7f78b4e0fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f07136be0 0x556f0713baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.631 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.608+0000 7f78b4e0fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556f07141ad0 0x556f0713bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.631 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.608+0000 7f78b4e0fb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f07081eb0 con 0x556f071362d0 2023-07-21T18:42:40.631 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.608+0000 7f78b4e0fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f06fe24f0 con 0x556f07136be0 2023-07-21T18:42:40.632 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.608+0000 7f78b4e0fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f07137350 con 0x556f07141ad0 2023-07-21T18:42:40.632 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.608+0000 7f78a259c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556f071362d0 0x556f07148510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:40.632 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.608+0000 7f78a259c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556f071362d0 0x556f07148510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58202/0 (socket says 172.21.15.39:58202) 2023-07-21T18:42:40.633 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.608+0000 7f78a259c700 1 -- 172.21.15.39:0/2037697125 learned_addr learned my addr 172.21.15.39:0/2037697125 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:40.633 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.608+0000 7f78a1d9b700 1 --2- 172.21.15.39:0/2037697125 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f07136be0 0x556f0713baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:40.633 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.608+0000 7f78a259c700 1 -- 172.21.15.39:0/2037697125 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556f07141ad0 msgr2=0x556f0713bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:40.633 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.608+0000 7f78a2d9d700 1 --2- 172.21.15.39:0/2037697125 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556f07141ad0 0x556f0713bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:40.633 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.608+0000 7f78a259c700 1 --2- 172.21.15.39:0/2037697125 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556f07141ad0 0x556f0713bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.633 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.608+0000 7f78a259c700 1 -- 172.21.15.39:0/2037697125 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f07136be0 msgr2=0x556f0713baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:40.634 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.608+0000 7f78a259c700 1 --2- 172.21.15.39:0/2037697125 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f07136be0 0x556f0713baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.634 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.608+0000 7f78a259c700 1 -- 172.21.15.39:0/2037697125 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f06edd250 con 0x556f071362d0 2023-07-21T18:42:40.634 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.608+0000 7f78a259c700 1 --2- 172.21.15.39:0/2037697125 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556f071362d0 0x556f07148510 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x556f07153c90 tx=0x7f7894011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b35d96803d1407d2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:40.634 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.609+0000 7f78917fa700 1 -- 172.21.15.39:0/2037697125 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789400ee90 con 0x556f071362d0 2023-07-21T18:42:40.634 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.609+0000 7f78917fa700 1 -- 172.21.15.39:0/2037697125 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f789402a070 con 0x556f071362d0 2023-07-21T18:42:40.634 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.609+0000 7f78917fa700 1 -- 172.21.15.39:0/2037697125 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7894021be0 con 0x556f071362d0 2023-07-21T18:42:40.638 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.610+0000 7f78b4e0fb80 1 -- 172.21.15.39:0/2037697125 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556f071362d0 msgr2=0x556f07148510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:40.638 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.610+0000 7f78b4e0fb80 1 --2- 172.21.15.39:0/2037697125 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556f071362d0 0x556f07148510 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x556f07153c90 tx=0x7f7894011000 comp rx=0 tx=0).stop 2023-07-21T18:42:40.638 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.610+0000 7f78b4e0fb80 1 -- 172.21.15.39:0/2037697125 shutdown_connections 2023-07-21T18:42:40.638 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.610+0000 7f78b4e0fb80 1 --2- 172.21.15.39:0/2037697125 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556f07141ad0 0x556f0713bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.638 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.610+0000 7f78b4e0fb80 1 --2- 172.21.15.39:0/2037697125 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556f071362d0 0x556f07148510 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.638 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.610+0000 7f78b4e0fb80 1 --2- 172.21.15.39:0/2037697125 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f07136be0 0x556f0713baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.638 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.610+0000 7f78b4e0fb80 1 -- 172.21.15.39:0/2037697125 >> 172.21.15.39:0/2037697125 conn(0x556f0712cda0 msgr2=0x556f0714d850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:40.639 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.610+0000 7f78b4e0fb80 1 -- 172.21.15.39:0/2037697125 shutdown_connections 2023-07-21T18:42:40.639 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.610+0000 7f78b4e0fb80 1 -- 172.21.15.39:0/2037697125 wait complete. 2023-07-21T18:42:40.639 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.610+0000 7f78b4e0fb80 1 -- start start 2023-07-21T18:42:40.639 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.610+0000 7f78b4e0fb80 1 Processor -- start 2023-07-21T18:42:40.639 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.611+0000 7f78b4e0fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f07136be0 0x556f071462d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.639 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.611+0000 7f78b4e0fb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556f07141ad0 0x556f07146810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.639 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.611+0000 7f78b4e0fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556f071a1e20 0x556f071a4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.640 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.611+0000 7f78b4e0fb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f071a4740 con 0x556f07141ad0 2023-07-21T18:42:40.640 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.611+0000 7f78b4e0fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f071a4880 con 0x556f07136be0 2023-07-21T18:42:40.640 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.611+0000 7f78b4e0fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f071a49c0 con 0x556f071a1e20 2023-07-21T18:42:40.640 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.611+0000 7f78b4e0fb80 10 client.0 _ll_register_callbacks cb 0x7f7898005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:42:40.640 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.611+0000 7f78a1d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556f07141ad0 0x556f07146810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:40.641 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.611+0000 7f78a1d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556f07141ad0 0x556f07146810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58210/0 (socket says 172.21.15.39:58210) 2023-07-21T18:42:40.641 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.611+0000 7f78a1d9b700 1 -- 172.21.15.39:0/344257212 learned_addr learned my addr 172.21.15.39:0/344257212 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:40.641 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.611+0000 7f78a259c700 1 --2- 172.21.15.39:0/344257212 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f07136be0 0x556f071462d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:40.641 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.611+0000 7f78a1d9b700 1 -- 172.21.15.39:0/344257212 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556f071a1e20 msgr2=0x556f071a4200 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T18:42:40.641 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.611+0000 7f78a1d9b700 1 --2- 172.21.15.39:0/344257212 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556f071a1e20 0x556f071a4200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.641 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.611+0000 7f78a1d9b700 1 -- 172.21.15.39:0/344257212 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f07136be0 msgr2=0x556f071462d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:40.641 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.611+0000 7f78a1d9b700 1 --2- 172.21.15.39:0/344257212 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f07136be0 0x556f071462d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.642 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.611+0000 7f78a1d9b700 1 -- 172.21.15.39:0/344257212 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f789401a040 con 0x556f07141ad0 2023-07-21T18:42:40.642 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.611+0000 7f78a259c700 1 --2- 172.21.15.39:0/344257212 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f07136be0 0x556f071462d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:42:40.642 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.612+0000 7f78a1d9b700 1 --2- 172.21.15.39:0/344257212 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556f07141ad0 0x556f07146810 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f789809a110 tx=0x7f789800cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16e60133d11a4e2c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:40.642 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.612+0000 7f7889ffb700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:42:40.642 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.612+0000 7f7889ffb700 1 -- 172.21.15.39:0/344257212 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78980a25d0 con 0x556f07141ad0 2023-07-21T18:42:40.642 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.612+0000 7f7889ffb700 1 -- 172.21.15.39:0/344257212 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f7898099670 con 0x556f07141ad0 2023-07-21T18:42:40.643 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.612+0000 7f78b4e0fb80 1 -- 172.21.15.39:0/344257212 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f789c00bcb0 con 0x556f07141ad0 2023-07-21T18:42:40.643 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.612+0000 7f7889ffb700 1 -- 172.21.15.39:0/344257212 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7898099830 con 0x556f07141ad0 2023-07-21T18:42:40.643 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.612+0000 7f78b4e0fb80 10 client.15861 fetch_fsmap learned FSMap version 26 2023-07-21T18:42:40.643 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.612+0000 7f78b4e0fb80 1 -- 172.21.15.39:0/344257212 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x556f06eb52b0 con 0x556f07141ad0 2023-07-21T18:42:40.643 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.612+0000 7f7889ffb700 1 -- 172.21.15.39:0/344257212 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f789c00bcb0 con 0x556f07141ad0 2023-07-21T18:42:40.643 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.612+0000 7f7889ffb700 1 -- 172.21.15.39:0/344257212 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f78980b82f0 con 0x556f07141ad0 2023-07-21T18:42:40.644 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.612+0000 7f78b4e0fb80 10 client.15861 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T18:42:40.644 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.612+0000 7f78b4e0fb80 10 client.15861 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T18:42:40.644 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.612+0000 7f78b4e0fb80 20 client.15861 populate_metadata read hostname 'smithi039' 2023-07-21T18:42:40.644 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.612+0000 7f78b4e0fb80 1 --2- 172.21.15.39:0/344257212 >> [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] conn(0x556f071a60d0 0x556f071a84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.644 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.612+0000 7f78b4e0fb80 4 client.15861 mds_command: new command op to 24500 tid=0[{"prefix": "damage ls"}] 2023-07-21T18:42:40.644 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.612+0000 7f78b4e0fb80 1 -- 172.21.15.39:0/344257212 --> [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556f07044050 con 0x556f071a60d0 2023-07-21T18:42:40.644 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.613+0000 7f78a259c700 1 --2- 172.21.15.39:0/344257212 >> [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] conn(0x556f071a60d0 0x556f071a84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:40.645 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.613+0000 7f7889ffb700 10 client.15861 ms_handle_connect on v2:172.21.15.155:6836/947670056 2023-07-21T18:42:40.645 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.613+0000 7f78a259c700 1 --2- 172.21.15.39:0/344257212 >> [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] conn(0x556f071a60d0 0x556f071a84e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bfab80622a4855c5 server_cookie=e915b39afb31fd07 in_seq=0 out_seq=0 2023-07-21T18:42:40.645 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.613+0000 7f7889ffb700 1 -- 172.21.15.39:0/344257212 <== mds.1 v2:172.21.15.155:6836/947670056 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556f07044050 con 0x556f071a60d0 2023-07-21T18:42:40.645 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.613+0000 7f7889ffb700 10 client.15861 handle_command_reply: tid=0 2023-07-21T18:42:40.645 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.613+0000 7f78b4e0fb80 1 client.15861 shutdown 2023-07-21T18:42:40.645 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.614+0000 7f78b4e0fb80 1 -- 172.21.15.39:0/344257212 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556f07141ad0 msgr2=0x556f07146810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:40.646 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.614+0000 7f78b4e0fb80 1 --2- 172.21.15.39:0/344257212 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556f07141ad0 0x556f07146810 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f789809a110 tx=0x7f789800cdb0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.646 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.615+0000 7f78b4e0fb80 1 -- 172.21.15.39:0/344257212 shutdown_connections 2023-07-21T18:42:40.646 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.615+0000 7f78b4e0fb80 1 --2- 172.21.15.39:0/344257212 >> [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] conn(0x556f071a60d0 0x556f071a84e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.646 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.615+0000 7f78b4e0fb80 1 --2- 172.21.15.39:0/344257212 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f07136be0 0x556f071462d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.646 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.615+0000 7f78b4e0fb80 1 --2- 172.21.15.39:0/344257212 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556f071a1e20 0x556f071a4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.646 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.615+0000 7f7889ffb700 0 client.15861 ms_handle_reset on v2:172.21.15.155:6836/947670056 2023-07-21T18:42:40.647 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.615+0000 7f78b4e0fb80 1 --2- 172.21.15.39:0/344257212 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556f07141ad0 0x556f07146810 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.647 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.615+0000 7f78b4e0fb80 1 -- 172.21.15.39:0/344257212 >> 172.21.15.39:0/344257212 conn(0x556f0712cda0 msgr2=0x556f071383e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:40.647 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.615+0000 7f78b4e0fb80 1 -- 172.21.15.39:0/344257212 shutdown_connections 2023-07-21T18:42:40.647 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.615+0000 7f78b4e0fb80 1 -- 172.21.15.39:0/344257212 wait complete. 2023-07-21T18:42:40.647 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.615+0000 7f78b4e0fb80 20 client.15861 trim_cache size 0 max 16384 2023-07-21T18:42:40.648 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2023-07-21T18:42:40.649 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.615+0000 7f78a8c89700 1 -- 172.21.15.39:0/2193629753 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f788c0a13e0 msgr2=0x7f788c0a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:40.649 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.615+0000 7f78a8c89700 1 --2- 172.21.15.39:0/2193629753 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f788c0a13e0 0x7f788c0a3890 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7894009a10 tx=0x7f7894009190 comp rx=0 tx=0).stop 2023-07-21T18:42:40.649 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.615+0000 7f78a8c89700 1 -- 172.21.15.39:0/2193629753 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78a40fff20 msgr2=0x7f78a4197200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:40.650 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.615+0000 7f78a8c89700 1 --2- 172.21.15.39:0/2193629753 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78a40fff20 0x7f78a4197200 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f7898009fd0 tx=0x7f789800ed10 comp rx=0 tx=0).stop 2023-07-21T18:42:40.650 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.615+0000 7f78a8c89700 1 -- 172.21.15.39:0/2193629753 shutdown_connections 2023-07-21T18:42:40.650 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.615+0000 7f78a8c89700 1 --2- 172.21.15.39:0/2193629753 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f788c0a13e0 0x7f788c0a3890 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.650 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.615+0000 7f78a8c89700 1 --2- 172.21.15.39:0/2193629753 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78a40ff580 0x7f78a4196cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.650 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.615+0000 7f78a8c89700 1 --2- 172.21.15.39:0/2193629753 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78a4103ce0 0x7f78a41a1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.651 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.615+0000 7f78a8c89700 1 --2- 172.21.15.39:0/2193629753 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78a40fff20 0x7f78a4197200 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.651 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.615+0000 7f78a8c89700 1 -- 172.21.15.39:0/2193629753 >> 172.21.15.39:0/2193629753 conn(0x7f78a4076200 msgr2=0x7f78a40fbe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:40.651 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.616+0000 7f78a8c89700 1 -- 172.21.15.39:0/2193629753 shutdown_connections 2023-07-21T18:42:40.651 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.616+0000 7f78a8c89700 1 -- 172.21.15.39:0/2193629753 wait complete. 2023-07-21T18:42:40.654 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-21T18:42:40.853 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.830+0000 7f62ee37e700 1 -- 172.21.15.39:0/4026298027 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62e81049e0 msgr2=0x7f62e8104db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:40.854 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.830+0000 7f62ee37e700 1 --2- 172.21.15.39:0/4026298027 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62e81049e0 0x7f62e8104db0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f62d8009990 tx=0x7f62d8009ca0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.854 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.831+0000 7f62ee37e700 1 -- 172.21.15.39:0/4026298027 shutdown_connections 2023-07-21T18:42:40.854 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.831+0000 7f62ee37e700 1 --2- 172.21.15.39:0/4026298027 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62e80ff380 0x7f62e810b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.854 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.831+0000 7f62ee37e700 1 --2- 172.21.15.39:0/4026298027 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62e81049e0 0x7f62e8104db0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.854 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.831+0000 7f62ee37e700 1 --2- 172.21.15.39:0/4026298027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62e80fe9d0 0x7f62e80fee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.855 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.831+0000 7f62ee37e700 1 -- 172.21.15.39:0/4026298027 >> 172.21.15.39:0/4026298027 conn(0x7f62e80fa4d0 msgr2=0x7f62e80fc8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:40.855 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.831+0000 7f62ee37e700 1 -- 172.21.15.39:0/4026298027 shutdown_connections 2023-07-21T18:42:40.855 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.831+0000 7f62ee37e700 1 -- 172.21.15.39:0/4026298027 wait complete. 2023-07-21T18:42:40.855 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.832+0000 7f62ee37e700 1 Processor -- start 2023-07-21T18:42:40.855 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.832+0000 7f62ee37e700 1 -- start start 2023-07-21T18:42:40.856 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.832+0000 7f62ee37e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62e80fe9d0 0x7f62e8197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.857 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.832+0000 7f62ee37e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62e80ff380 0x7f62e8198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.857 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.832+0000 7f62ee37e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62e81049e0 0x7f62e819c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.857 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.832+0000 7f62ee37e700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62e819cd30 con 0x7f62e80ff380 2023-07-21T18:42:40.858 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.832+0000 7f62ee37e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62e819cea0 con 0x7f62e80fe9d0 2023-07-21T18:42:40.858 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.832+0000 7f62ee37e700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62e819d060 con 0x7f62e81049e0 2023-07-21T18:42:40.858 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.832+0000 7f62e77fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62e80ff380 0x7f62e8198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:40.858 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.832+0000 7f62e77fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62e80ff380 0x7f62e8198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58226/0 (socket says 172.21.15.39:58226) 2023-07-21T18:42:40.858 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.832+0000 7f62e77fe700 1 -- 172.21.15.39:0/1992579043 learned_addr learned my addr 172.21.15.39:0/1992579043 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:40.858 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.832+0000 7f62ec91b700 1 --2- 172.21.15.39:0/1992579043 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62e81049e0 0x7f62e819c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:40.858 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.832+0000 7f62e7fff700 1 --2- 172.21.15.39:0/1992579043 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62e80fe9d0 0x7f62e8197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:40.858 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.833+0000 7f62e77fe700 1 -- 172.21.15.39:0/1992579043 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62e81049e0 msgr2=0x7f62e819c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:40.859 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.833+0000 7f62e77fe700 1 --2- 172.21.15.39:0/1992579043 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62e81049e0 0x7f62e819c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.859 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.833+0000 7f62e77fe700 1 -- 172.21.15.39:0/1992579043 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62e80fe9d0 msgr2=0x7f62e8197d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:40.859 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.833+0000 7f62e77fe700 1 --2- 172.21.15.39:0/1992579043 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62e80fe9d0 0x7f62e8197d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.859 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.833+0000 7f62e77fe700 1 -- 172.21.15.39:0/1992579043 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d80095f0 con 0x7f62e80ff380 2023-07-21T18:42:40.859 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.833+0000 7f62e77fe700 1 --2- 172.21.15.39:0/1992579043 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62e80ff380 0x7f62e8198290 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f62dc0097d0 tx=0x7f62dc009ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:40.859 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.833+0000 7f62e57fa700 1 -- 172.21.15.39:0/1992579043 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62dc010040 con 0x7f62e80ff380 2023-07-21T18:42:40.859 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.833+0000 7f62ee37e700 1 -- 172.21.15.39:0/1992579043 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62e819d340 con 0x7f62e80ff380 2023-07-21T18:42:40.859 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.833+0000 7f62e57fa700 1 -- 172.21.15.39:0/1992579043 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f62dc014cd0 con 0x7f62e80ff380 2023-07-21T18:42:40.860 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.833+0000 7f62e57fa700 1 -- 172.21.15.39:0/1992579043 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62dc013da0 con 0x7f62e80ff380 2023-07-21T18:42:40.860 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.833+0000 7f62ee37e700 1 -- 172.21.15.39:0/1992579043 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62e81a4b80 con 0x7f62e80ff380 2023-07-21T18:42:40.861 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.837+0000 7f62e57fa700 1 -- 172.21.15.39:0/1992579043 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f62dc014810 con 0x7f62e80ff380 2023-07-21T18:42:40.861 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.838+0000 7f62e57fa700 1 --2- 172.21.15.39:0/1992579043 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f62d00a13e0 0x7f62d00a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.861 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.838+0000 7f62e57fa700 1 -- 172.21.15.39:0/1992579043 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f62dc0cce30 con 0x7f62e80ff380 2023-07-21T18:42:40.861 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.838+0000 7f62e7fff700 1 --2- 172.21.15.39:0/1992579043 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f62d00a13e0 0x7f62d00a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:40.862 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.838+0000 7f62fa504b80 1 Processor -- start 2023-07-21T18:42:40.862 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.838+0000 7f62fa504b80 1 -- start start 2023-07-21T18:42:40.862 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.839+0000 7f62fa504b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9888ad2d0 0x55d9888ad6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.862 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.839+0000 7f62fa504b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9888adbe0 0x55d9888b8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.862 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.839+0000 7f62fa504b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9888b8ad0 0x55d9888baeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.862 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.839+0000 7f62fa504b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9887b3cb0 con 0x55d9888ad2d0 2023-07-21T18:42:40.862 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.839+0000 7f62fa504b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d988806a40 con 0x55d9888b8ad0 2023-07-21T18:42:40.863 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.839+0000 7f62fa504b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9887ffed0 con 0x55d9888adbe0 2023-07-21T18:42:40.863 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.839+0000 7f62ec91b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9888b8ad0 0x55d9888baeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:40.863 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.839+0000 7f62ec91b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9888b8ad0 0x55d9888baeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:42594/0 (socket says 172.21.15.39:42594) 2023-07-21T18:42:40.863 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.839+0000 7f62e77fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9888adbe0 0x55d9888b8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:40.863 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.839+0000 7f62ec91b700 1 -- 172.21.15.39:0/4203241506 learned_addr learned my addr 172.21.15.39:0/4203241506 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:40.863 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.839+0000 7f62ec91b700 1 -- 172.21.15.39:0/4203241506 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9888adbe0 msgr2=0x55d9888b8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:40.863 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.839+0000 7f62ec91b700 1 --2- 172.21.15.39:0/4203241506 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9888adbe0 0x55d9888b8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.863 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.839+0000 7f62ec91b700 1 -- 172.21.15.39:0/4203241506 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9888ad2d0 msgr2=0x55d9888ad6a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T18:42:40.864 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.839+0000 7f62ec91b700 1 --2- 172.21.15.39:0/4203241506 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9888ad2d0 0x55d9888ad6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.864 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.839+0000 7f62ec91b700 1 -- 172.21.15.39:0/4203241506 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d98862dde0 con 0x55d9888b8ad0 2023-07-21T18:42:40.864 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.839+0000 7f62ec91b700 1 --2- 172.21.15.39:0/4203241506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9888b8ad0 0x55d9888baeb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f62e0009dc0 tx=0x7f62e00077e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=121414cd807486c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:40.864 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.840+0000 7f62e7fff700 1 --2- 172.21.15.39:0/1992579043 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f62d00a13e0 0x7f62d00a3890 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f62d80098e0 tx=0x7f62d8005960 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:40.866 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.841+0000 7f62ce7fc700 1 -- 172.21.15.39:0/4203241506 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e0014610 con 0x55d9888b8ad0 2023-07-21T18:42:40.866 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.841+0000 7f62ce7fc700 1 -- 172.21.15.39:0/4203241506 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f62e000f050 con 0x55d9888b8ad0 2023-07-21T18:42:40.866 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.841+0000 7f62ce7fc700 1 -- 172.21.15.39:0/4203241506 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e0013960 con 0x55d9888b8ad0 2023-07-21T18:42:40.866 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.842+0000 7f62fa504b80 1 -- 172.21.15.39:0/4203241506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9888b8ad0 msgr2=0x55d9888baeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:40.867 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.842+0000 7f62fa504b80 1 --2- 172.21.15.39:0/4203241506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9888b8ad0 0x55d9888baeb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f62e0009dc0 tx=0x7f62e00077e0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.867 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.842+0000 7f62fa504b80 1 -- 172.21.15.39:0/4203241506 shutdown_connections 2023-07-21T18:42:40.867 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.842+0000 7f62fa504b80 1 --2- 172.21.15.39:0/4203241506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9888b8ad0 0x55d9888baeb0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.867 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.842+0000 7f62fa504b80 1 --2- 172.21.15.39:0/4203241506 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9888ad2d0 0x55d9888ad6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.867 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.842+0000 7f62fa504b80 1 --2- 172.21.15.39:0/4203241506 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9888adbe0 0x55d9888b8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.867 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.842+0000 7f62fa504b80 1 -- 172.21.15.39:0/4203241506 >> 172.21.15.39:0/4203241506 conn(0x55d9888a3da0 msgr2=0x55d9888a96a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:40.867 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.842+0000 7f62fa504b80 1 -- 172.21.15.39:0/4203241506 shutdown_connections 2023-07-21T18:42:40.868 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.842+0000 7f62fa504b80 1 -- 172.21.15.39:0/4203241506 wait complete. 2023-07-21T18:42:40.868 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.842+0000 7f62fa504b80 1 -- start start 2023-07-21T18:42:40.868 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.842+0000 7f62fa504b80 1 Processor -- start 2023-07-21T18:42:40.868 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.843+0000 7f62fa504b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9888ad2d0 0x55d9888b84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.868 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.843+0000 7f62fa504b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9888adbe0 0x55d9888b5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.868 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.843+0000 7f62fa504b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9888b8ad0 0x55d9888b5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.868 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.843+0000 7f62fa504b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d98879fac0 con 0x55d9888b8ad0 2023-07-21T18:42:40.868 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.843+0000 7f62fa504b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d98879f7c0 con 0x55d9888ad2d0 2023-07-21T18:42:40.869 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.843+0000 7f62fa504b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d988683920 con 0x55d9888adbe0 2023-07-21T18:42:40.869 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.843+0000 7f62fa504b80 10 client.0 _ll_register_callbacks cb 0x55d98879d760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:42:40.869 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.843+0000 7f62ec91b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9888b8ad0 0x55d9888b5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:40.869 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.843+0000 7f62ec91b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9888b8ad0 0x55d9888b5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58236/0 (socket says 172.21.15.39:58236) 2023-07-21T18:42:40.869 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.843+0000 7f62ec91b700 1 -- 172.21.15.39:0/2968349405 learned_addr learned my addr 172.21.15.39:0/2968349405 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:40.869 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.843+0000 7f62e77fe700 1 --2- 172.21.15.39:0/2968349405 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9888adbe0 0x55d9888b5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:40.869 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.843+0000 7f62e7fff700 1 --2- 172.21.15.39:0/2968349405 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9888ad2d0 0x55d9888b84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:40.870 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.844+0000 7f62ec91b700 1 -- 172.21.15.39:0/2968349405 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9888adbe0 msgr2=0x55d9888b5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:40.870 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.844+0000 7f62ec91b700 1 --2- 172.21.15.39:0/2968349405 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9888adbe0 0x55d9888b5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.870 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.844+0000 7f62ec91b700 1 -- 172.21.15.39:0/2968349405 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9888ad2d0 msgr2=0x55d9888b84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:40.870 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.844+0000 7f62ec91b700 1 --2- 172.21.15.39:0/2968349405 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9888ad2d0 0x55d9888b84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.870 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.844+0000 7f62ec91b700 1 -- 172.21.15.39:0/2968349405 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62e0009a20 con 0x55d9888b8ad0 2023-07-21T18:42:40.870 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.844+0000 7f62e7fff700 1 --2- 172.21.15.39:0/2968349405 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9888ad2d0 0x55d9888b84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:42:40.870 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.844+0000 7f62ec91b700 1 --2- 172.21.15.39:0/2968349405 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9888b8ad0 0x55d9888b5d20 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f62e0009f50 tx=0x7f62e0014b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d68358213820183 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:40.870 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.844+0000 7f62c77fe700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:42:40.871 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.844+0000 7f62c77fe700 1 -- 172.21.15.39:0/2968349405 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e0006000 con 0x55d9888b8ad0 2023-07-21T18:42:40.871 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.844+0000 7f62c77fe700 1 -- 172.21.15.39:0/2968349405 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f62e00135e0 con 0x55d9888b8ad0 2023-07-21T18:42:40.871 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.844+0000 7f62fa504b80 1 -- 172.21.15.39:0/2968349405 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d9887594f0 con 0x55d9888b8ad0 2023-07-21T18:42:40.871 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.844+0000 7f62c77fe700 1 -- 172.21.15.39:0/2968349405 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f62e00137a0 con 0x55d9888b8ad0 2023-07-21T18:42:40.871 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.844+0000 7f62fa504b80 10 client.15870 fetch_fsmap learned FSMap version 26 2023-07-21T18:42:40.871 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.844+0000 7f62fa504b80 1 -- 172.21.15.39:0/2968349405 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55d988616c60 con 0x55d9888b8ad0 2023-07-21T18:42:40.871 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.845+0000 7f62c77fe700 1 -- 172.21.15.39:0/2968349405 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d9887594f0 con 0x55d9888b8ad0 2023-07-21T18:42:40.873 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.845+0000 7f62c77fe700 1 -- 172.21.15.39:0/2968349405 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f62e0026770 con 0x55d9888b8ad0 2023-07-21T18:42:40.873 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.845+0000 7f62fa504b80 10 client.15870 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T18:42:40.873 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.845+0000 7f62fa504b80 10 client.15870 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T18:42:40.873 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.845+0000 7f62fa504b80 20 client.15870 populate_metadata read hostname 'smithi039' 2023-07-21T18:42:40.873 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.845+0000 7f62fa504b80 1 --2- 172.21.15.39:0/2968349405 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d9888c8330 0x55d9888ca740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.873 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.845+0000 7f62fa504b80 4 client.15870 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:42:40.873 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.845+0000 7f62fa504b80 1 -- 172.21.15.39:0/2968349405 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d9887f8eb0 con 0x55d9888c8330 2023-07-21T18:42:40.874 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.845+0000 7f62e7fff700 1 --2- 172.21.15.39:0/2968349405 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d9888c8330 0x55d9888ca740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:40.874 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.846+0000 7f62c77fe700 10 client.15870 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:42:40.874 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.846+0000 7f62e7fff700 1 --2- 172.21.15.39:0/2968349405 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d9888c8330 0x55d9888ca740 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2085a9464c4264e4 server_cookie=f2711fc9f7fb5936 in_seq=0 out_seq=0 2023-07-21T18:42:40.874 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.846+0000 7f62c77fe700 1 -- 172.21.15.39:0/2968349405 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d9887f8eb0 con 0x55d9888c8330 2023-07-21T18:42:40.874 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.846+0000 7f62c77fe700 10 client.15870 handle_command_reply: tid=0 2023-07-21T18:42:40.874 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.846+0000 7f62fa504b80 1 client.15870 shutdown 2023-07-21T18:42:40.874 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.847+0000 7f62fa504b80 1 -- 172.21.15.39:0/2968349405 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9888b8ad0 msgr2=0x55d9888b5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:40.875 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.847+0000 7f62fa504b80 1 --2- 172.21.15.39:0/2968349405 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9888b8ad0 0x55d9888b5d20 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f62e0009f50 tx=0x7f62e0014b20 comp rx=0 tx=0).stop 2023-07-21T18:42:40.875 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.847+0000 7f62fa504b80 1 -- 172.21.15.39:0/2968349405 shutdown_connections 2023-07-21T18:42:40.875 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.847+0000 7f62fa504b80 1 --2- 172.21.15.39:0/2968349405 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d9888c8330 0x55d9888ca740 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.875 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.847+0000 7f62fa504b80 1 --2- 172.21.15.39:0/2968349405 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9888ad2d0 0x55d9888b84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.875 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.847+0000 7f62fa504b80 1 --2- 172.21.15.39:0/2968349405 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9888b8ad0 0x55d9888b5d20 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.875 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.847+0000 7f62fa504b80 1 --2- 172.21.15.39:0/2968349405 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9888adbe0 0x55d9888b5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.875 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.847+0000 7f62fa504b80 1 -- 172.21.15.39:0/2968349405 >> 172.21.15.39:0/2968349405 conn(0x55d9888a3da0 msgr2=0x55d9888a9230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:40.875 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.847+0000 7f62c77fe700 0 client.15870 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:42:40.876 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.847+0000 7f62fa504b80 1 -- 172.21.15.39:0/2968349405 shutdown_connections 2023-07-21T18:42:40.876 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.847+0000 7f62fa504b80 1 -- 172.21.15.39:0/2968349405 wait complete. 2023-07-21T18:42:40.876 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.847+0000 7f62fa504b80 20 client.15870 trim_cache size 0 max 16384 2023-07-21T18:42:40.895 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.873+0000 7f62fa504b80 1 Processor -- start 2023-07-21T18:42:40.895 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.873+0000 7f62fa504b80 1 -- start start 2023-07-21T18:42:40.895 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.873+0000 7f62fa504b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9888ad2d0 0x55d9888bf4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.895 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.873+0000 7f62fa504b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9888adbe0 0x55d9888ca340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.896 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.873+0000 7f62fa504b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9888b8ad0 0x55d9888b7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.896 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.873+0000 7f62fa504b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9888b6300 con 0x55d9888b8ad0 2023-07-21T18:42:40.896 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.873+0000 7f62fa504b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9888b3c90 con 0x55d9888ad2d0 2023-07-21T18:42:40.896 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.873+0000 7f62fa504b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9888bfa20 con 0x55d9888adbe0 2023-07-21T18:42:40.897 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.874+0000 7f62e77fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9888adbe0 0x55d9888ca340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:40.898 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.874+0000 7f62e7fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9888ad2d0 0x55d9888bf4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:40.898 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.874+0000 7f62e77fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9888adbe0 0x55d9888ca340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:48564/0 (socket says 172.21.15.39:48564) 2023-07-21T18:42:40.898 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.874+0000 7f62e7fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9888ad2d0 0x55d9888bf4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:42606/0 (socket says 172.21.15.39:42606) 2023-07-21T18:42:40.898 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.874+0000 7f62e77fe700 1 -- 172.21.15.39:0/2543474662 learned_addr learned my addr 172.21.15.39:0/2543474662 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:40.898 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.874+0000 7f62ec91b700 1 --2- 172.21.15.39:0/2543474662 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9888b8ad0 0x55d9888b7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:40.898 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.874+0000 7f62e77fe700 1 -- 172.21.15.39:0/2543474662 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9888ad2d0 msgr2=0x55d9888bf4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:40.898 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.874+0000 7f62e77fe700 1 --2- 172.21.15.39:0/2543474662 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9888ad2d0 0x55d9888bf4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.899 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.874+0000 7f62e77fe700 1 -- 172.21.15.39:0/2543474662 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9888b8ad0 msgr2=0x55d9888b7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:40.899 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.874+0000 7f62e77fe700 1 --2- 172.21.15.39:0/2543474662 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9888b8ad0 0x55d9888b7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.899 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.874+0000 7f62e77fe700 1 -- 172.21.15.39:0/2543474662 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d988654250 con 0x55d9888adbe0 2023-07-21T18:42:40.899 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.874+0000 7f62e77fe700 1 --2- 172.21.15.39:0/2543474662 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9888adbe0 0x55d9888ca340 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f62dc096740 tx=0x7f62dc0cc040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3832c23db8129cde server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:40.899 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.874+0000 7f62ceffd700 1 -- 172.21.15.39:0/2543474662 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62dc0b6070 con 0x55d9888adbe0 2023-07-21T18:42:40.899 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.874+0000 7f62ceffd700 1 -- 172.21.15.39:0/2543474662 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f62dc0b1070 con 0x55d9888adbe0 2023-07-21T18:42:40.899 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.874+0000 7f62ceffd700 1 -- 172.21.15.39:0/2543474662 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62dc09c070 con 0x55d9888adbe0 2023-07-21T18:42:40.900 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.875+0000 7f62fa504b80 1 -- 172.21.15.39:0/2543474662 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9888adbe0 msgr2=0x55d9888ca340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:40.900 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.875+0000 7f62fa504b80 1 --2- 172.21.15.39:0/2543474662 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9888adbe0 0x55d9888ca340 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f62dc096740 tx=0x7f62dc0cc040 comp rx=0 tx=0).stop 2023-07-21T18:42:40.900 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.875+0000 7f62fa504b80 1 -- 172.21.15.39:0/2543474662 shutdown_connections 2023-07-21T18:42:40.900 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.875+0000 7f62fa504b80 1 --2- 172.21.15.39:0/2543474662 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9888ad2d0 0x55d9888bf4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.901 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.875+0000 7f62fa504b80 1 --2- 172.21.15.39:0/2543474662 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9888b8ad0 0x55d9888b7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.901 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.875+0000 7f62fa504b80 1 --2- 172.21.15.39:0/2543474662 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9888adbe0 0x55d9888ca340 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.901 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.875+0000 7f62fa504b80 1 -- 172.21.15.39:0/2543474662 >> 172.21.15.39:0/2543474662 conn(0x55d9888a3da0 msgr2=0x55d9888c4660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:40.901 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.875+0000 7f62fa504b80 1 -- 172.21.15.39:0/2543474662 shutdown_connections 2023-07-21T18:42:40.901 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.876+0000 7f62fa504b80 1 -- 172.21.15.39:0/2543474662 wait complete. 2023-07-21T18:42:40.901 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.876+0000 7f62fa504b80 1 -- start start 2023-07-21T18:42:40.904 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.876+0000 7f62fa504b80 1 Processor -- start 2023-07-21T18:42:40.904 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.876+0000 7f62fa504b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9888ad2d0 0x55d9888b2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.904 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.877+0000 7f62fa504b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9888adbe0 0x55d9888b3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.904 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.877+0000 7f62fa504b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9888b8ad0 0x55d9888b1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.905 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.877+0000 7f62fa504b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9888b21b0 con 0x55d9888b8ad0 2023-07-21T18:42:40.905 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.877+0000 7f62fa504b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9888b2320 con 0x55d9888ad2d0 2023-07-21T18:42:40.905 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.877+0000 7f62fa504b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9888b24e0 con 0x55d9888adbe0 2023-07-21T18:42:40.905 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.877+0000 7f62fa504b80 10 client.0 _ll_register_callbacks cb 0x7f62dc0958f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:42:40.905 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.877+0000 7f62e7fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9888ad2d0 0x55d9888b2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:40.905 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.877+0000 7f62e7fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9888ad2d0 0x55d9888b2ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:42612/0 (socket says 172.21.15.39:42612) 2023-07-21T18:42:40.905 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.877+0000 7f62e7fff700 1 -- 172.21.15.39:0/1080805231 learned_addr learned my addr 172.21.15.39:0/1080805231 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:40.905 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.877+0000 7f62ec91b700 1 --2- 172.21.15.39:0/1080805231 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9888b8ad0 0x55d9888b1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:40.906 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.877+0000 7f62e77fe700 1 --2- 172.21.15.39:0/1080805231 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9888adbe0 0x55d9888b3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:40.906 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.877+0000 7f62e7fff700 1 -- 172.21.15.39:0/1080805231 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9888adbe0 msgr2=0x55d9888b3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:40.906 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.877+0000 7f62e7fff700 1 --2- 172.21.15.39:0/1080805231 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9888adbe0 0x55d9888b3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.906 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.877+0000 7f62e7fff700 1 -- 172.21.15.39:0/1080805231 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9888b8ad0 msgr2=0x55d9888b1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:40.906 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.877+0000 7f62e7fff700 1 --2- 172.21.15.39:0/1080805231 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9888b8ad0 0x55d9888b1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.906 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.877+0000 7f62e7fff700 1 -- 172.21.15.39:0/1080805231 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62e81a4b80 con 0x55d9888ad2d0 2023-07-21T18:42:40.906 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.878+0000 7f62e7fff700 1 --2- 172.21.15.39:0/1080805231 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9888ad2d0 0x55d9888b2ee0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f62d8009990 tx=0x7f62d8009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b8bee0f314633e5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:40.907 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.878+0000 7f62c77fe700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-21T18:42:40.907 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.878+0000 7f62c77fe700 1 -- 172.21.15.39:0/1080805231 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d8033e50 con 0x55d9888ad2d0 2023-07-21T18:42:40.907 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.878+0000 7f62c77fe700 1 -- 172.21.15.39:0/1080805231 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f62d800f460 con 0x55d9888ad2d0 2023-07-21T18:42:40.907 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.878+0000 7f62fa504b80 1 -- 172.21.15.39:0/1080805231 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f62e000f050 con 0x55d9888ad2d0 2023-07-21T18:42:40.907 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.878+0000 7f62c77fe700 1 -- 172.21.15.39:0/1080805231 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f62d800f620 con 0x55d9888ad2d0 2023-07-21T18:42:40.907 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.878+0000 7f62fa504b80 10 client.25442 fetch_fsmap learned FSMap version 26 2023-07-21T18:42:40.907 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.878+0000 7f62fa504b80 1 -- 172.21.15.39:0/1080805231 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55d98862c2b0 con 0x55d9888ad2d0 2023-07-21T18:42:40.908 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.878+0000 7f62c77fe700 1 -- 172.21.15.39:0/1080805231 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f62e000f050 con 0x55d9888ad2d0 2023-07-21T18:42:40.908 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.878+0000 7f62c77fe700 1 -- 172.21.15.39:0/1080805231 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f62d8004880 con 0x55d9888ad2d0 2023-07-21T18:42:40.908 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.878+0000 7f62fa504b80 10 client.25442 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T18:42:40.908 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.878+0000 7f62fa504b80 10 client.25442 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T18:42:40.908 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.878+0000 7f62fa504b80 20 client.25442 populate_metadata read hostname 'smithi039' 2023-07-21T18:42:40.908 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.879+0000 7f62fa504b80 1 --2- 172.21.15.39:0/1080805231 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d988925ce0 0x55d9889280f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:40.908 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.879+0000 7f62fa504b80 4 client.25442 mds_command: new command op to 24536 tid=0[{"prefix": "damage ls"}] 2023-07-21T18:42:40.909 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.879+0000 7f62fa504b80 1 -- 172.21.15.39:0/1080805231 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d9887bb050 con 0x55d988925ce0 2023-07-21T18:42:40.909 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.879+0000 7f62e77fe700 1 --2- 172.21.15.39:0/1080805231 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d988925ce0 0x55d9889280f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:40.909 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.879+0000 7f62c77fe700 10 client.25442 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:42:40.909 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.879+0000 7f62e77fe700 1 --2- 172.21.15.39:0/1080805231 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d988925ce0 0x55d9889280f0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9db29b327b0e2166 server_cookie=47c89ddc68c6c778 in_seq=0 out_seq=0 2023-07-21T18:42:40.909 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.879+0000 7f62c77fe700 1 -- 172.21.15.39:0/1080805231 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d9887bb050 con 0x55d988925ce0 2023-07-21T18:42:40.909 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.879+0000 7f62c77fe700 10 client.25442 handle_command_reply: tid=0 2023-07-21T18:42:40.910 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.879+0000 7f62fa504b80 1 client.25442 shutdown 2023-07-21T18:42:40.910 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.880+0000 7f62fa504b80 1 -- 172.21.15.39:0/1080805231 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9888ad2d0 msgr2=0x55d9888b2ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:40.910 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.880+0000 7f62fa504b80 1 --2- 172.21.15.39:0/1080805231 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9888ad2d0 0x55d9888b2ee0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f62d8009990 tx=0x7f62d8009ca0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.910 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.880+0000 7f62fa504b80 1 -- 172.21.15.39:0/1080805231 shutdown_connections 2023-07-21T18:42:40.910 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.880+0000 7f62fa504b80 1 --2- 172.21.15.39:0/1080805231 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d988925ce0 0x55d9889280f0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.910 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.880+0000 7f62fa504b80 1 --2- 172.21.15.39:0/1080805231 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9888ad2d0 0x55d9888b2ee0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.911 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.880+0000 7f62fa504b80 1 --2- 172.21.15.39:0/1080805231 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9888b8ad0 0x55d9888b1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.911 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.880+0000 7f62fa504b80 1 --2- 172.21.15.39:0/1080805231 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9888adbe0 0x55d9888b3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.911 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.880+0000 7f62fa504b80 1 -- 172.21.15.39:0/1080805231 >> 172.21.15.39:0/1080805231 conn(0x55d9888a3da0 msgr2=0x55d9888ca340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:40.911 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.880+0000 7f62c77fe700 0 client.25442 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:42:40.911 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.880+0000 7f62fa504b80 1 -- 172.21.15.39:0/1080805231 shutdown_connections 2023-07-21T18:42:40.911 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.880+0000 7f62fa504b80 1 -- 172.21.15.39:0/1080805231 wait complete. 2023-07-21T18:42:40.911 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.880+0000 7f62fa504b80 20 client.25442 trim_cache size 0 max 16384 2023-07-21T18:42:40.912 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2023-07-21T18:42:40.913 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.881+0000 7f62ee37e700 1 -- 172.21.15.39:0/1992579043 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f62d00a13e0 msgr2=0x7f62d00a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:40.913 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.881+0000 7f62ee37e700 1 --2- 172.21.15.39:0/1992579043 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f62d00a13e0 0x7f62d00a3890 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f62d80098e0 tx=0x7f62d8005960 comp rx=0 tx=0).stop 2023-07-21T18:42:40.913 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.881+0000 7f62ee37e700 1 -- 172.21.15.39:0/1992579043 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62e80ff380 msgr2=0x7f62e8198290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:40.913 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.881+0000 7f62ee37e700 1 --2- 172.21.15.39:0/1992579043 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62e80ff380 0x7f62e8198290 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f62dc0097d0 tx=0x7f62dc009ae0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.913 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.881+0000 7f62ee37e700 1 -- 172.21.15.39:0/1992579043 shutdown_connections 2023-07-21T18:42:40.913 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.881+0000 7f62ee37e700 1 --2- 172.21.15.39:0/1992579043 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f62d00a13e0 0x7f62d00a3890 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.914 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.881+0000 7f62ee37e700 1 --2- 172.21.15.39:0/1992579043 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62e80fe9d0 0x7f62e8197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.914 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.881+0000 7f62ee37e700 1 --2- 172.21.15.39:0/1992579043 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62e81049e0 0x7f62e819c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.914 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.881+0000 7f62ee37e700 1 --2- 172.21.15.39:0/1992579043 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62e80ff380 0x7f62e8198290 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:40.914 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.881+0000 7f62ee37e700 1 -- 172.21.15.39:0/1992579043 >> 172.21.15.39:0/1992579043 conn(0x7f62e80fa4d0 msgr2=0x7f62e80fbab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:40.914 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.881+0000 7f62ee37e700 1 -- 172.21.15.39:0/1992579043 shutdown_connections 2023-07-21T18:42:40.915 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:40.881+0000 7f62ee37e700 1 -- 172.21.15.39:0/1992579043 wait complete. 2023-07-21T18:42:41.919 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-21T18:42:41.919 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-21T18:42:42.055 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.030+0000 7f20438b8700 1 -- 172.21.15.39:0/218974017 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f203c103ce0 msgr2=0x7f203c10b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:42.056 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.030+0000 7f20438b8700 1 --2- 172.21.15.39:0/218974017 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f203c103ce0 0x7f203c10b8f0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f203400b0d0 tx=0x7f203400b3e0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.056 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.030+0000 7f20438b8700 1 -- 172.21.15.39:0/218974017 shutdown_connections 2023-07-21T18:42:42.056 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.030+0000 7f20438b8700 1 --2- 172.21.15.39:0/218974017 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f203c0ff580 0x7f203c0ff950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.057 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.030+0000 7f20438b8700 1 --2- 172.21.15.39:0/218974017 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f203c103ce0 0x7f203c10b8f0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.057 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.030+0000 7f20438b8700 1 --2- 172.21.15.39:0/218974017 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f203c0fff20 0x7f203c103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.057 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.030+0000 7f20438b8700 1 -- 172.21.15.39:0/218974017 >> 172.21.15.39:0/218974017 conn(0x7f203c076200 msgr2=0x7f203c076600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:42.058 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.030+0000 7f20438b8700 1 -- 172.21.15.39:0/218974017 shutdown_connections 2023-07-21T18:42:42.058 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.030+0000 7f20438b8700 1 -- 172.21.15.39:0/218974017 wait complete. 2023-07-21T18:42:42.058 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.031+0000 7f20438b8700 1 Processor -- start 2023-07-21T18:42:42.058 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.031+0000 7f20438b8700 1 -- start start 2023-07-21T18:42:42.059 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.031+0000 7f20438b8700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f203c0ff580 0x7f203c196b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:42.059 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.031+0000 7f20438b8700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f203c0fff20 0x7f203c197040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:42.059 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.031+0000 7f20438b8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f203c103ce0 0x7f203c1a1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:42.059 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.031+0000 7f20438b8700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f203c197980 con 0x7f203c103ce0 2023-07-21T18:42:42.060 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.031+0000 7f20438b8700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f203c1a20c0 con 0x7f203c0fff20 2023-07-21T18:42:42.060 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.031+0000 7f20438b8700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f203c1a2280 con 0x7f203c0ff580 2023-07-21T18:42:42.060 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.032+0000 7f2041e55700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f203c103ce0 0x7f203c1a1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:42.060 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.032+0000 7f2041e55700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f203c103ce0 0x7f203c1a1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58264/0 (socket says 172.21.15.39:58264) 2023-07-21T18:42:42.061 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.032+0000 7f2041e55700 1 -- 172.21.15.39:0/2387083689 learned_addr learned my addr 172.21.15.39:0/2387083689 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:42.061 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.032+0000 7f2041654700 1 --2- 172.21.15.39:0/2387083689 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f203c0ff580 0x7f203c196b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:42.061 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.032+0000 7f2040e53700 1 --2- 172.21.15.39:0/2387083689 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f203c0fff20 0x7f203c197040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:42.062 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.032+0000 7f2041e55700 1 -- 172.21.15.39:0/2387083689 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f203c0ff580 msgr2=0x7f203c196b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:42.062 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.032+0000 7f2041e55700 1 --2- 172.21.15.39:0/2387083689 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f203c0ff580 0x7f203c196b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.062 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.032+0000 7f2041e55700 1 -- 172.21.15.39:0/2387083689 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f203c0fff20 msgr2=0x7f203c197040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:42.062 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.032+0000 7f2041e55700 1 --2- 172.21.15.39:0/2387083689 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f203c0fff20 0x7f203c197040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.063 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.032+0000 7f2041e55700 1 -- 172.21.15.39:0/2387083689 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2034009d00 con 0x7f203c103ce0 2023-07-21T18:42:42.063 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.032+0000 7f2040e53700 1 --2- 172.21.15.39:0/2387083689 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f203c0fff20 0x7f203c197040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:42:42.063 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.032+0000 7f2041e55700 1 --2- 172.21.15.39:0/2387083689 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f203c103ce0 0x7f203c1a1b80 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f2034007870 tx=0x7f20340078a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:42.063 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.032+0000 7f20327fc700 1 -- 172.21.15.39:0/2387083689 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203400e040 con 0x7f203c103ce0 2023-07-21T18:42:42.064 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.032+0000 7f20327fc700 1 -- 172.21.15.39:0/2387083689 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2034003e10 con 0x7f203c103ce0 2023-07-21T18:42:42.064 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.032+0000 7f20327fc700 1 -- 172.21.15.39:0/2387083689 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203401b980 con 0x7f203c103ce0 2023-07-21T18:42:42.064 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.032+0000 7f20438b8700 1 -- 172.21.15.39:0/2387083689 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f203c1a2500 con 0x7f203c103ce0 2023-07-21T18:42:42.064 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.032+0000 7f20438b8700 1 -- 172.21.15.39:0/2387083689 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f203c1a28d0 con 0x7f203c103ce0 2023-07-21T18:42:42.074 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.035+0000 7f20327fc700 1 -- 172.21.15.39:0/2387083689 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f2034024410 con 0x7f203c103ce0 2023-07-21T18:42:42.074 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.036+0000 7f20327fc700 1 --2- 172.21.15.39:0/2387083689 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f20280a13e0 0x7f20280a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:42.075 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.036+0000 7f2041654700 1 --2- 172.21.15.39:0/2387083689 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f20280a13e0 0x7f20280a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:42.075 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.036+0000 7f20327fc700 1 -- 172.21.15.39:0/2387083689 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f20340ce650 con 0x7f203c103ce0 2023-07-21T18:42:42.075 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.036+0000 7f2041654700 1 --2- 172.21.15.39:0/2387083689 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f20280a13e0 0x7f20280a3890 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f202c00ac50 tx=0x7f202c00a380 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:42.075 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.037+0000 7f204fa3eb80 1 Processor -- start 2023-07-21T18:42:42.076 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.037+0000 7f204fa3eb80 1 -- start start 2023-07-21T18:42:42.076 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.037+0000 7f204fa3eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5573d7089350 0x5573d7089720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:42.076 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.037+0000 7f204fa3eb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5573d7089c60 0x5573d7098a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:42.077 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.037+0000 7f204fa3eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5573d7098f60 0x5573d709b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:42.077 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.037+0000 7f204fa3eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573d6f8fd30 con 0x5573d7098f60 2023-07-21T18:42:42.077 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.037+0000 7f204fa3eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573d6fe2ac0 con 0x5573d7089350 2023-07-21T18:42:42.077 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.037+0000 7f204fa3eb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573d6fdbf50 con 0x5573d7089c60 2023-07-21T18:42:42.078 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.037+0000 7f2041e55700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5573d7098f60 0x5573d709b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:42.078 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.037+0000 7f2040e53700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5573d7089c60 0x5573d7098a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:42.078 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.037+0000 7f2041e55700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5573d7098f60 0x5573d709b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58272/0 (socket says 172.21.15.39:58272) 2023-07-21T18:42:42.078 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.037+0000 7f2041e55700 1 -- 172.21.15.39:0/2907863765 learned_addr learned my addr 172.21.15.39:0/2907863765 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:42.079 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.037+0000 7f2041654700 1 --2- 172.21.15.39:0/2907863765 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5573d7089350 0x5573d7089720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:42.079 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.037+0000 7f2040e53700 1 -- 172.21.15.39:0/2907863765 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5573d7089350 msgr2=0x5573d7089720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:42.079 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.037+0000 7f2040e53700 1 --2- 172.21.15.39:0/2907863765 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5573d7089350 0x5573d7089720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.079 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.037+0000 7f2040e53700 1 -- 172.21.15.39:0/2907863765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5573d7098f60 msgr2=0x5573d709b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:42.080 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.037+0000 7f2040e53700 1 --2- 172.21.15.39:0/2907863765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5573d7098f60 0x5573d709b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.080 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.037+0000 7f2040e53700 1 -- 172.21.15.39:0/2907863765 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573d6e09e60 con 0x5573d7089c60 2023-07-21T18:42:42.080 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.037+0000 7f2041e55700 1 --2- 172.21.15.39:0/2907863765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5573d7098f60 0x5573d709b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:42:42.080 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.038+0000 7f2040e53700 1 --2- 172.21.15.39:0/2907863765 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5573d7089c60 0x5573d7098a20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f203c105290 tx=0x7f203800df00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee924580cfdce124 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:42.081 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.038+0000 7f20277fe700 1 -- 172.21.15.39:0/2907863765 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2038004230 con 0x5573d7089c60 2023-07-21T18:42:42.081 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.038+0000 7f20277fe700 1 -- 172.21.15.39:0/2907863765 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f203800fd40 con 0x5573d7089c60 2023-07-21T18:42:42.081 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.038+0000 7f20277fe700 1 -- 172.21.15.39:0/2907863765 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2038003d60 con 0x5573d7089c60 2023-07-21T18:42:42.081 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.039+0000 7f204fa3eb80 1 -- 172.21.15.39:0/2907863765 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5573d7089c60 msgr2=0x5573d7098a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:42.082 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.039+0000 7f204fa3eb80 1 --2- 172.21.15.39:0/2907863765 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5573d7089c60 0x5573d7098a20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f203c105290 tx=0x7f203800df00 comp rx=0 tx=0).stop 2023-07-21T18:42:42.082 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.039+0000 7f204fa3eb80 1 -- 172.21.15.39:0/2907863765 shutdown_connections 2023-07-21T18:42:42.082 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.039+0000 7f204fa3eb80 1 --2- 172.21.15.39:0/2907863765 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5573d7089350 0x5573d7089720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.082 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.039+0000 7f204fa3eb80 1 --2- 172.21.15.39:0/2907863765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5573d7098f60 0x5573d709b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.083 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.039+0000 7f204fa3eb80 1 --2- 172.21.15.39:0/2907863765 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5573d7089c60 0x5573d7098a20 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.083 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.039+0000 7f204fa3eb80 1 -- 172.21.15.39:0/2907863765 >> 172.21.15.39:0/2907863765 conn(0x5573d707fe20 msgr2=0x5573d7085720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:42.083 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.039+0000 7f204fa3eb80 1 -- 172.21.15.39:0/2907863765 shutdown_connections 2023-07-21T18:42:42.083 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.039+0000 7f204fa3eb80 1 -- 172.21.15.39:0/2907863765 wait complete. 2023-07-21T18:42:42.083 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.039+0000 7f204fa3eb80 1 -- start start 2023-07-21T18:42:42.084 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.039+0000 7f204fa3eb80 1 Processor -- start 2023-07-21T18:42:42.084 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.040+0000 7f204fa3eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5573d7089350 0x5573d7098950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:42.084 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.040+0000 7f204fa3eb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5573d7089c60 0x5573d7095c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:42.085 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.040+0000 7f204fa3eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5573d7098f60 0x5573d70962c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:42.085 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.040+0000 7f204fa3eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573d6f7bb40 con 0x5573d7098f60 2023-07-21T18:42:42.085 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.040+0000 7f204fa3eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573d6f7b840 con 0x5573d7089350 2023-07-21T18:42:42.085 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.040+0000 7f204fa3eb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573d6e5f9a0 con 0x5573d7089c60 2023-07-21T18:42:42.086 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.040+0000 7f204fa3eb80 10 client.0 _ll_register_callbacks cb 0x5573d6f797e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:42:42.086 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.040+0000 7f2041e55700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5573d7098f60 0x5573d70962c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:42.086 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.040+0000 7f2041e55700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5573d7098f60 0x5573d70962c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58278/0 (socket says 172.21.15.39:58278) 2023-07-21T18:42:42.087 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.040+0000 7f2041e55700 1 -- 172.21.15.39:0/3737542484 learned_addr learned my addr 172.21.15.39:0/3737542484 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:42.087 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.040+0000 7f2040e53700 1 --2- 172.21.15.39:0/3737542484 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5573d7089c60 0x5573d7095c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:42.087 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.040+0000 7f2041654700 1 --2- 172.21.15.39:0/3737542484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5573d7089350 0x5573d7098950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:42.088 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.040+0000 7f2041e55700 1 -- 172.21.15.39:0/3737542484 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5573d7089c60 msgr2=0x5573d7095c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:42.088 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.040+0000 7f2041e55700 1 --2- 172.21.15.39:0/3737542484 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5573d7089c60 0x5573d7095c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.088 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.040+0000 7f2041e55700 1 -- 172.21.15.39:0/3737542484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5573d7089350 msgr2=0x5573d7098950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:42.088 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.040+0000 7f2041e55700 1 --2- 172.21.15.39:0/3737542484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5573d7089350 0x5573d7098950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.089 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.040+0000 7f2041e55700 1 -- 172.21.15.39:0/3737542484 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f203c1a28d0 con 0x5573d7098f60 2023-07-21T18:42:42.089 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.040+0000 7f2041654700 1 --2- 172.21.15.39:0/3737542484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5573d7089350 0x5573d7098950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:42:42.089 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.040+0000 7f2041e55700 1 --2- 172.21.15.39:0/3737542484 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5573d7098f60 0x5573d70962c0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f203409b220 tx=0x7f203409b370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdf68d0fb37718bc server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:42.089 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.040+0000 7f2024ff9700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:42:42.090 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.040+0000 7f2024ff9700 1 -- 172.21.15.39:0/3737542484 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2034097040 con 0x5573d7098f60 2023-07-21T18:42:42.090 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.040+0000 7f2024ff9700 1 -- 172.21.15.39:0/3737542484 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f20340b0020 con 0x5573d7098f60 2023-07-21T18:42:42.090 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.040+0000 7f204fa3eb80 1 -- 172.21.15.39:0/3737542484 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5573d6f35570 con 0x5573d7098f60 2023-07-21T18:42:42.090 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.041+0000 7f2024ff9700 1 -- 172.21.15.39:0/3737542484 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f203409b520 con 0x5573d7098f60 2023-07-21T18:42:42.091 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.041+0000 7f204fa3eb80 10 client.15888 fetch_fsmap learned FSMap version 26 2023-07-21T18:42:42.091 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.041+0000 7f204fa3eb80 1 -- 172.21.15.39:0/3737542484 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f203800d880 con 0x5573d7098f60 2023-07-21T18:42:42.091 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.041+0000 7f2024ff9700 1 -- 172.21.15.39:0/3737542484 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5573d6f35570 con 0x5573d7098f60 2023-07-21T18:42:42.092 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.041+0000 7f2024ff9700 1 -- 172.21.15.39:0/3737542484 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2034084d20 con 0x5573d7098f60 2023-07-21T18:42:42.092 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.041+0000 7f204fa3eb80 10 client.15888 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T18:42:42.092 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.041+0000 7f204fa3eb80 10 client.15888 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T18:42:42.092 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.041+0000 7f204fa3eb80 20 client.15888 populate_metadata read hostname 'smithi039' 2023-07-21T18:42:42.093 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.041+0000 7f204fa3eb80 1 --2- 172.21.15.39:0/3737542484 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5573d70a8670 0x5573d70aaa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:42.093 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.041+0000 7f204fa3eb80 4 client.15888 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:42:42.093 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.041+0000 7f204fa3eb80 1 -- 172.21.15.39:0/3737542484 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5573d6fd4f30 con 0x5573d70a8670 2023-07-21T18:42:42.093 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.041+0000 7f2041654700 1 --2- 172.21.15.39:0/3737542484 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5573d70a8670 0x5573d70aaa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:42.093 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.042+0000 7f2041654700 1 --2- 172.21.15.39:0/3737542484 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5573d70a8670 0x5573d70aaa50 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55ff22d86a963ae4 server_cookie=e36a47bb9f716a1b in_seq=0 out_seq=0 2023-07-21T18:42:42.093 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.042+0000 7f2024ff9700 10 client.15888 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:42:42.093 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.043+0000 7f2024ff9700 1 -- 172.21.15.39:0/3737542484 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5573d6fd4f30 con 0x5573d70a8670 2023-07-21T18:42:42.094 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.043+0000 7f2024ff9700 10 client.15888 handle_command_reply: tid=0 2023-07-21T18:42:42.094 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.043+0000 7f204fa3eb80 1 client.15888 shutdown 2023-07-21T18:42:42.094 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.043+0000 7f204fa3eb80 1 -- 172.21.15.39:0/3737542484 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5573d7098f60 msgr2=0x5573d70962c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:42.094 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.043+0000 7f204fa3eb80 1 --2- 172.21.15.39:0/3737542484 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5573d7098f60 0x5573d70962c0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f203409b220 tx=0x7f203409b370 comp rx=0 tx=0).stop 2023-07-21T18:42:42.094 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.043+0000 7f204fa3eb80 1 -- 172.21.15.39:0/3737542484 shutdown_connections 2023-07-21T18:42:42.094 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.043+0000 7f204fa3eb80 1 --2- 172.21.15.39:0/3737542484 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5573d70a8670 0x5573d70aaa50 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.094 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.043+0000 7f204fa3eb80 1 --2- 172.21.15.39:0/3737542484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5573d7089350 0x5573d7098950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.095 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.043+0000 7f204fa3eb80 1 --2- 172.21.15.39:0/3737542484 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5573d7098f60 0x5573d70962c0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.095 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.043+0000 7f204fa3eb80 1 --2- 172.21.15.39:0/3737542484 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5573d7089c60 0x5573d7095c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.095 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.043+0000 7f2024ff9700 0 client.15888 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:42:42.095 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.043+0000 7f204fa3eb80 1 -- 172.21.15.39:0/3737542484 >> 172.21.15.39:0/3737542484 conn(0x5573d707fe20 msgr2=0x5573d7085310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:42.095 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.043+0000 7f204fa3eb80 1 -- 172.21.15.39:0/3737542484 shutdown_connections 2023-07-21T18:42:42.095 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.043+0000 7f204fa3eb80 1 -- 172.21.15.39:0/3737542484 wait complete. 2023-07-21T18:42:42.095 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.043+0000 7f204fa3eb80 20 client.15888 trim_cache size 0 max 16384 2023-07-21T18:42:42.099 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.060+0000 7f204fa3eb80 1 Processor -- start 2023-07-21T18:42:42.100 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.060+0000 7f204fa3eb80 1 -- start start 2023-07-21T18:42:42.100 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.060+0000 7f204fa3eb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5573d7089350 0x5573d709f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:42.100 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.060+0000 7f204fa3eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5573d7089c60 0x5573d70984f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:42.100 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.060+0000 7f204fa3eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5573d7098f60 0x5573d708bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:42.100 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.060+0000 7f204fa3eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573d70971b0 con 0x5573d7098f60 2023-07-21T18:42:42.100 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.060+0000 7f204fa3eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573d708b220 con 0x5573d7089c60 2023-07-21T18:42:42.101 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.060+0000 7f204fa3eb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573d7086cd0 con 0x5573d7089350 2023-07-21T18:42:42.101 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.060+0000 7f2041e55700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5573d7098f60 0x5573d708bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:42.101 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.060+0000 7f2041e55700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5573d7098f60 0x5573d708bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58286/0 (socket says 172.21.15.39:58286) 2023-07-21T18:42:42.101 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.060+0000 7f2041e55700 1 -- 172.21.15.39:0/3283043535 learned_addr learned my addr 172.21.15.39:0/3283043535 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:42.101 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.061+0000 7f2041654700 1 --2- 172.21.15.39:0/3283043535 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5573d7089350 0x5573d709f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:42.101 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.061+0000 7f2040e53700 1 --2- 172.21.15.39:0/3283043535 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5573d7089c60 0x5573d70984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:42.101 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.061+0000 7f2041e55700 1 -- 172.21.15.39:0/3283043535 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5573d7089350 msgr2=0x5573d709f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:42.102 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.061+0000 7f2041e55700 1 --2- 172.21.15.39:0/3283043535 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5573d7089350 0x5573d709f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.102 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.061+0000 7f2041e55700 1 -- 172.21.15.39:0/3283043535 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5573d7089c60 msgr2=0x5573d70984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:42.102 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.061+0000 7f2041e55700 1 --2- 172.21.15.39:0/3283043535 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5573d7089c60 0x5573d70984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.102 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.061+0000 7f2041e55700 1 -- 172.21.15.39:0/3283043535 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573d6df2ce0 con 0x5573d7098f60 2023-07-21T18:42:42.102 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.061+0000 7f2040e53700 1 --2- 172.21.15.39:0/3283043535 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5573d7089c60 0x5573d70984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:42:42.102 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.061+0000 7f2041e55700 1 --2- 172.21.15.39:0/3283043535 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5573d7098f60 0x5573d708bb70 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f2034009830 tx=0x7f2034005990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67be64caea9eb657 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:42.102 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.061+0000 7f2027fff700 1 -- 172.21.15.39:0/3283043535 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2034097040 con 0x5573d7098f60 2023-07-21T18:42:42.102 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.061+0000 7f2027fff700 1 -- 172.21.15.39:0/3283043535 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f203409c0d0 con 0x5573d7098f60 2023-07-21T18:42:42.103 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.061+0000 7f2027fff700 1 -- 172.21.15.39:0/3283043535 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203409f550 con 0x5573d7098f60 2023-07-21T18:42:42.103 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.062+0000 7f204fa3eb80 1 -- 172.21.15.39:0/3283043535 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5573d7098f60 msgr2=0x5573d708bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:42.103 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.062+0000 7f204fa3eb80 1 --2- 172.21.15.39:0/3283043535 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5573d7098f60 0x5573d708bb70 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f2034009830 tx=0x7f2034005990 comp rx=0 tx=0).stop 2023-07-21T18:42:42.103 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.062+0000 7f204fa3eb80 1 -- 172.21.15.39:0/3283043535 shutdown_connections 2023-07-21T18:42:42.103 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.062+0000 7f204fa3eb80 1 --2- 172.21.15.39:0/3283043535 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5573d7098f60 0x5573d708bb70 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.103 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.062+0000 7f204fa3eb80 1 --2- 172.21.15.39:0/3283043535 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5573d7089350 0x5573d709f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.103 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.062+0000 7f204fa3eb80 1 --2- 172.21.15.39:0/3283043535 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5573d7089c60 0x5573d70984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.103 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.062+0000 7f204fa3eb80 1 -- 172.21.15.39:0/3283043535 >> 172.21.15.39:0/3283043535 conn(0x5573d707fe20 msgr2=0x5573d708b460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:42.104 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.062+0000 7f204fa3eb80 1 -- 172.21.15.39:0/3283043535 shutdown_connections 2023-07-21T18:42:42.104 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.062+0000 7f204fa3eb80 1 -- 172.21.15.39:0/3283043535 wait complete. 2023-07-21T18:42:42.104 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.062+0000 7f204fa3eb80 1 -- start start 2023-07-21T18:42:42.104 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.062+0000 7f204fa3eb80 1 Processor -- start 2023-07-21T18:42:42.104 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.063+0000 7f204fa3eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5573d7089350 0x5573d709e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:42.104 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.063+0000 7f204fa3eb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5573d7089c60 0x5573d709e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:42.104 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.063+0000 7f204fa3eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5573d7098f60 0x5573d709ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:42.105 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.063+0000 7f204fa3eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573d70fc550 con 0x5573d7098f60 2023-07-21T18:42:42.105 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.063+0000 7f204fa3eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573d70fc690 con 0x5573d7089350 2023-07-21T18:42:42.105 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.063+0000 7f204fa3eb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573d70fc850 con 0x5573d7089c60 2023-07-21T18:42:42.105 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.063+0000 7f204fa3eb80 10 client.0 _ll_register_callbacks cb 0x5573d6f797e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:42:42.105 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.063+0000 7f2041e55700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5573d7098f60 0x5573d709ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:42.105 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.063+0000 7f2041e55700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5573d7098f60 0x5573d709ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58292/0 (socket says 172.21.15.39:58292) 2023-07-21T18:42:42.106 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.063+0000 7f2041e55700 1 -- 172.21.15.39:0/3768315385 learned_addr learned my addr 172.21.15.39:0/3768315385 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:42.106 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.063+0000 7f2041654700 1 --2- 172.21.15.39:0/3768315385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5573d7089350 0x5573d709e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:42.106 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.063+0000 7f2040e53700 1 --2- 172.21.15.39:0/3768315385 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5573d7089c60 0x5573d709e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:42.106 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.063+0000 7f2041e55700 1 -- 172.21.15.39:0/3768315385 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5573d7089c60 msgr2=0x5573d709e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:42.106 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.063+0000 7f2041e55700 1 --2- 172.21.15.39:0/3768315385 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5573d7089c60 0x5573d709e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.106 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.063+0000 7f2041e55700 1 -- 172.21.15.39:0/3768315385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5573d7089350 msgr2=0x5573d709e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:42.106 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.063+0000 7f2041e55700 1 --2- 172.21.15.39:0/3768315385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5573d7089350 0x5573d709e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.106 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.063+0000 7f2041e55700 1 -- 172.21.15.39:0/3768315385 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f203800d880 con 0x5573d7098f60 2023-07-21T18:42:42.107 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.063+0000 7f2041e55700 1 --2- 172.21.15.39:0/3768315385 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5573d7098f60 0x5573d709ec10 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f20340a33b0 tx=0x7f20340099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8f578ef73da3e3b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:42.107 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.063+0000 7f2024ff9700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:42:42.107 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.064+0000 7f2024ff9700 1 -- 172.21.15.39:0/3768315385 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2034097040 con 0x5573d7098f60 2023-07-21T18:42:42.107 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.064+0000 7f204fa3eb80 1 -- 172.21.15.39:0/3768315385 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5573d6f970d0 con 0x5573d7098f60 2023-07-21T18:42:42.107 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.064+0000 7f2024ff9700 1 -- 172.21.15.39:0/3768315385 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f20340b0070 con 0x5573d7098f60 2023-07-21T18:42:42.107 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.064+0000 7f2024ff9700 1 -- 172.21.15.39:0/3768315385 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f203409f370 con 0x5573d7098f60 2023-07-21T18:42:42.107 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.064+0000 7f204fa3eb80 10 client.15894 fetch_fsmap learned FSMap version 26 2023-07-21T18:42:42.108 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.064+0000 7f204fa3eb80 1 -- 172.21.15.39:0/3768315385 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5573d6e302d0 con 0x5573d7098f60 2023-07-21T18:42:42.108 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.064+0000 7f2024ff9700 1 -- 172.21.15.39:0/3768315385 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5573d6f970d0 con 0x5573d7098f60 2023-07-21T18:42:42.108 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.064+0000 7f2024ff9700 1 -- 172.21.15.39:0/3768315385 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2034051040 con 0x5573d7098f60 2023-07-21T18:42:42.108 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.064+0000 7f204fa3eb80 10 client.15894 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T18:42:42.108 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.064+0000 7f204fa3eb80 10 client.15894 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T18:42:42.108 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.064+0000 7f204fa3eb80 20 client.15894 populate_metadata read hostname 'smithi039' 2023-07-21T18:42:42.108 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.064+0000 7f204fa3eb80 1 --2- 172.21.15.39:0/3768315385 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5573d70fda50 0x5573d70ffe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:42.109 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.064+0000 7f204fa3eb80 4 client.15894 mds_command: new command op to 24536 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-21T18:42:42.109 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.064+0000 7f204fa3eb80 1 -- 172.21.15.39:0/3768315385 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5573d6e2aff0 con 0x5573d70fda50 2023-07-21T18:42:42.109 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.064+0000 7f2041654700 1 --2- 172.21.15.39:0/3768315385 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5573d70fda50 0x5573d70ffe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:42.109 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.065+0000 7f2024ff9700 10 client.15894 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:42:42.109 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.065+0000 7f2041654700 1 --2- 172.21.15.39:0/3768315385 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5573d70fda50 0x5573d70ffe30 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcf4b63ed802c0d1 server_cookie=44198973c0b92256 in_seq=0 out_seq=0 2023-07-21T18:42:42.109 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.066+0000 7f2024ff9700 1 -- 172.21.15.39:0/3768315385 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5573d6e2aff0 con 0x5573d70fda50 2023-07-21T18:42:42.109 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.066+0000 7f2024ff9700 10 client.15894 handle_command_reply: tid=0 2023-07-21T18:42:42.109 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.066+0000 7f204fa3eb80 1 client.15894 shutdown 2023-07-21T18:42:42.110 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.066+0000 7f204fa3eb80 1 -- 172.21.15.39:0/3768315385 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5573d7098f60 msgr2=0x5573d709ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:42.110 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.066+0000 7f204fa3eb80 1 --2- 172.21.15.39:0/3768315385 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5573d7098f60 0x5573d709ec10 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f20340a33b0 tx=0x7f20340099e0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.110 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.066+0000 7f204fa3eb80 1 -- 172.21.15.39:0/3768315385 shutdown_connections 2023-07-21T18:42:42.110 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.066+0000 7f204fa3eb80 1 --2- 172.21.15.39:0/3768315385 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5573d70fda50 0x5573d70ffe30 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.110 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.066+0000 7f204fa3eb80 1 --2- 172.21.15.39:0/3768315385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5573d7089350 0x5573d709e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.110 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.066+0000 7f204fa3eb80 1 --2- 172.21.15.39:0/3768315385 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5573d7098f60 0x5573d709ec10 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.110 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.066+0000 7f2024ff9700 0 client.15894 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:42:42.110 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.066+0000 7f204fa3eb80 1 --2- 172.21.15.39:0/3768315385 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5573d7089c60 0x5573d709e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.111 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.066+0000 7f204fa3eb80 1 -- 172.21.15.39:0/3768315385 >> 172.21.15.39:0/3768315385 conn(0x5573d707fe20 msgr2=0x5573d7084930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:42.111 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.067+0000 7f204fa3eb80 1 -- 172.21.15.39:0/3768315385 shutdown_connections 2023-07-21T18:42:42.111 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.067+0000 7f204fa3eb80 1 -- 172.21.15.39:0/3768315385 wait complete. 2023-07-21T18:42:42.111 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.067+0000 7f204fa3eb80 20 client.15894 trim_cache size 0 max 16384 2023-07-21T18:42:42.111 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2023-07-21T18:42:42.112 INFO:teuthology.orchestra.run.smithi039.stdout: "return_code": 0, 2023-07-21T18:42:42.112 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_tag": "e64b5b31-b4f8-42de-af6b-50e6288c6e95", 2023-07-21T18:42:42.112 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": "asynchronous" 2023-07-21T18:42:42.112 INFO:teuthology.orchestra.run.smithi039.stdout:} 2023-07-21T18:42:42.113 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.067+0000 7f20438b8700 1 -- 172.21.15.39:0/2387083689 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f20280a13e0 msgr2=0x7f20280a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:42.113 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.067+0000 7f20438b8700 1 --2- 172.21.15.39:0/2387083689 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f20280a13e0 0x7f20280a3890 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f202c00ac50 tx=0x7f202c00a380 comp rx=0 tx=0).stop 2023-07-21T18:42:42.113 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.067+0000 7f20438b8700 1 -- 172.21.15.39:0/2387083689 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f203c103ce0 msgr2=0x7f203c1a1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:42.114 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.067+0000 7f20438b8700 1 --2- 172.21.15.39:0/2387083689 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f203c103ce0 0x7f203c1a1b80 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f2034007870 tx=0x7f20340078a0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.114 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.067+0000 7f20438b8700 1 -- 172.21.15.39:0/2387083689 shutdown_connections 2023-07-21T18:42:42.114 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.067+0000 7f20438b8700 1 --2- 172.21.15.39:0/2387083689 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f203c103ce0 0x7f203c1a1b80 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.114 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.067+0000 7f20438b8700 1 --2- 172.21.15.39:0/2387083689 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f203c0ff580 0x7f203c196b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.114 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.067+0000 7f20438b8700 1 --2- 172.21.15.39:0/2387083689 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f20280a13e0 0x7f20280a3890 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.114 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.067+0000 7f20438b8700 1 --2- 172.21.15.39:0/2387083689 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f203c0fff20 0x7f203c197040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.114 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.067+0000 7f20438b8700 1 -- 172.21.15.39:0/2387083689 >> 172.21.15.39:0/2387083689 conn(0x7f203c076200 msgr2=0x7f203c0fbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:42.114 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.067+0000 7f20438b8700 1 -- 172.21.15.39:0/2387083689 shutdown_connections 2023-07-21T18:42:42.115 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.067+0000 7f20438b8700 1 -- 172.21.15.39:0/2387083689 wait complete. 2023-07-21T18:42:42.115 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T18:42:42.244 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.222+0000 7ff7fc26b700 1 -- 172.21.15.39:0/2376802199 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7f41015b0 msgr2=0x7ff7f4109aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:42.244 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.222+0000 7ff7fc26b700 1 --2- 172.21.15.39:0/2376802199 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7f41015b0 0x7ff7f4109aa0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8009990 tx=0x7ff7e8009ca0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.244 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.222+0000 7ff7fc26b700 1 -- 172.21.15.39:0/2376802199 shutdown_connections 2023-07-21T18:42:42.245 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.222+0000 7ff7fc26b700 1 --2- 172.21.15.39:0/2376802199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7f410a070 0x7ff7f410c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.245 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.222+0000 7ff7fc26b700 1 --2- 172.21.15.39:0/2376802199 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7f4100ca0 0x7ff7f4101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.245 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.222+0000 7ff7fc26b700 1 --2- 172.21.15.39:0/2376802199 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7f41015b0 0x7ff7f4109aa0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.245 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.222+0000 7ff7fc26b700 1 -- 172.21.15.39:0/2376802199 >> 172.21.15.39:0/2376802199 conn(0x7ff7f40fa4b0 msgr2=0x7ff7f40fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:42.245 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.222+0000 7ff7fc26b700 1 -- 172.21.15.39:0/2376802199 shutdown_connections 2023-07-21T18:42:42.245 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.222+0000 7ff7fc26b700 1 -- 172.21.15.39:0/2376802199 wait complete. 2023-07-21T18:42:42.247 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.223+0000 7ff7fc26b700 1 Processor -- start 2023-07-21T18:42:42.247 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.223+0000 7ff7fc26b700 1 -- start start 2023-07-21T18:42:42.247 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.223+0000 7ff7fc26b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7f4100ca0 0x7ff7f41981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:42.247 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.223+0000 7ff7fc26b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7f41015b0 0x7ff7f41986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:42.247 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.223+0000 7ff7fc26b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7f410a070 0x7ff7f419c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:42.248 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.223+0000 7ff7fc26b700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f419ce10 con 0x7ff7f4100ca0 2023-07-21T18:42:42.248 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.223+0000 7ff7fc26b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f419cf80 con 0x7ff7f41015b0 2023-07-21T18:42:42.248 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.223+0000 7ff7fc26b700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f419d140 con 0x7ff7f410a070 2023-07-21T18:42:42.248 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.224+0000 7ff7fa007700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7f4100ca0 0x7ff7f41981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:42.248 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.224+0000 7ff7fa007700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7f4100ca0 0x7ff7f41981a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58300/0 (socket says 172.21.15.39:58300) 2023-07-21T18:42:42.248 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.224+0000 7ff7fa007700 1 -- 172.21.15.39:0/1746345002 learned_addr learned my addr 172.21.15.39:0/1746345002 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:42.248 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.224+0000 7ff7fa808700 1 --2- 172.21.15.39:0/1746345002 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7f410a070 0x7ff7f419c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:42.249 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.224+0000 7ff7f9806700 1 --2- 172.21.15.39:0/1746345002 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7f41015b0 0x7ff7f41986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:42.249 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.224+0000 7ff7fa007700 1 -- 172.21.15.39:0/1746345002 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7f410a070 msgr2=0x7ff7f419c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:42.249 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.224+0000 7ff7fa007700 1 --2- 172.21.15.39:0/1746345002 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7f410a070 0x7ff7f419c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.249 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.224+0000 7ff7fa007700 1 -- 172.21.15.39:0/1746345002 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7f41015b0 msgr2=0x7ff7f41986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:42.249 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.224+0000 7ff7fa007700 1 --2- 172.21.15.39:0/1746345002 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7f41015b0 0x7ff7f41986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.249 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.224+0000 7ff7fa007700 1 -- 172.21.15.39:0/1746345002 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7e80095f0 con 0x7ff7f4100ca0 2023-07-21T18:42:42.249 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.224+0000 7ff7fa007700 1 --2- 172.21.15.39:0/1746345002 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7f4100ca0 0x7ff7f41981a0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7ff7f000e9a0 tx=0x7ff7f000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:42.249 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.224+0000 7ff7e77fe700 1 -- 172.21.15.39:0/1746345002 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f0009790 con 0x7ff7f4100ca0 2023-07-21T18:42:42.250 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.224+0000 7ff7e77fe700 1 -- 172.21.15.39:0/1746345002 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7f0010430 con 0x7ff7f4100ca0 2023-07-21T18:42:42.250 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.224+0000 7ff7fc26b700 1 -- 172.21.15.39:0/1746345002 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7f419d420 con 0x7ff7f4100ca0 2023-07-21T18:42:42.250 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.224+0000 7ff7e77fe700 1 -- 172.21.15.39:0/1746345002 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f0018680 con 0x7ff7f4100ca0 2023-07-21T18:42:42.250 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.224+0000 7ff7fc26b700 1 -- 172.21.15.39:0/1746345002 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7f41a4c60 con 0x7ff7f4100ca0 2023-07-21T18:42:42.255 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.226+0000 7ff7e77fe700 1 -- 172.21.15.39:0/1746345002 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7ff7f00105a0 con 0x7ff7f4100ca0 2023-07-21T18:42:42.255 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.227+0000 7ff7e77fe700 1 --2- 172.21.15.39:0/1746345002 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7ff7e00a1460 0x7ff7e00a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:42.255 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.227+0000 7ff7e77fe700 1 -- 172.21.15.39:0/1746345002 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7ff7f0014070 con 0x7ff7f4100ca0 2023-07-21T18:42:42.255 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.227+0000 7ff7f9806700 1 --2- 172.21.15.39:0/1746345002 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7ff7e00a1460 0x7ff7e00a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:42.255 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.227+0000 7ff7f9806700 1 --2- 172.21.15.39:0/1746345002 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7ff7e00a1460 0x7ff7e00a3910 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff7f4199440 tx=0x7ff7e8005fb0 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:42.255 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.228+0000 7ff8083f1b80 1 Processor -- start 2023-07-21T18:42:42.255 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.228+0000 7ff8083f1b80 1 -- start start 2023-07-21T18:42:42.256 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.228+0000 7ff8083f1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55891ecf22d0 0x55891ecf26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:42.257 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.228+0000 7ff8083f1b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55891ecf2be0 0x55891ecfd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:42.257 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.228+0000 7ff8083f1b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55891ecfdad0 0x55891ecffeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:42.257 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.228+0000 7ff8083f1b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55891ebf8cb0 con 0x55891ecfdad0 2023-07-21T18:42:42.257 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.228+0000 7ff8083f1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55891ec4ba40 con 0x55891ecf22d0 2023-07-21T18:42:42.257 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.228+0000 7ff8083f1b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55891ec44ed0 con 0x55891ecf2be0 2023-07-21T18:42:42.257 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.228+0000 7ff7fa808700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55891ecfdad0 0x55891ecffeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:42.258 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.228+0000 7ff7fa808700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55891ecfdad0 0x55891ecffeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58312/0 (socket says 172.21.15.39:58312) 2023-07-21T18:42:42.258 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.228+0000 7ff7fa808700 1 -- 172.21.15.39:0/2419808765 learned_addr learned my addr 172.21.15.39:0/2419808765 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:42.258 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.229+0000 7ff7fa007700 1 --2- 172.21.15.39:0/2419808765 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55891ecf22d0 0x55891ecf26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:42.258 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.229+0000 7ff7f9806700 1 --2- 172.21.15.39:0/2419808765 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55891ecf2be0 0x55891ecfd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:42.258 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.229+0000 7ff7fa808700 1 -- 172.21.15.39:0/2419808765 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55891ecf2be0 msgr2=0x55891ecfd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:42.258 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.229+0000 7ff7fa808700 1 --2- 172.21.15.39:0/2419808765 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55891ecf2be0 0x55891ecfd590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.258 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.229+0000 7ff7fa808700 1 -- 172.21.15.39:0/2419808765 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55891ecf22d0 msgr2=0x55891ecf26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:42.258 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.229+0000 7ff7fa808700 1 --2- 172.21.15.39:0/2419808765 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55891ecf22d0 0x55891ecf26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.259 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.229+0000 7ff7fa808700 1 -- 172.21.15.39:0/2419808765 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55891ea72de0 con 0x55891ecfdad0 2023-07-21T18:42:42.259 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.229+0000 7ff7f9806700 1 --2- 172.21.15.39:0/2419808765 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55891ecf2be0 0x55891ecfd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:42:42.259 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.229+0000 7ff7fa808700 1 --2- 172.21.15.39:0/2419808765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55891ecfdad0 0x55891ecffeb0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec003f30 tx=0x7ff7ec009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f81467ee2718e70c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:42.259 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.229+0000 7ff7e4dac700 1 -- 172.21.15.39:0/2419808765 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec003a00 con 0x55891ecfdad0 2023-07-21T18:42:42.259 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.229+0000 7ff7e4dac700 1 -- 172.21.15.39:0/2419808765 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7ec00f040 con 0x55891ecfdad0 2023-07-21T18:42:42.259 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.229+0000 7ff7e4dac700 1 -- 172.21.15.39:0/2419808765 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec0156e0 con 0x55891ecfdad0 2023-07-21T18:42:42.260 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.230+0000 7ff8083f1b80 1 -- 172.21.15.39:0/2419808765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55891ecfdad0 msgr2=0x55891ecffeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:42.260 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.230+0000 7ff8083f1b80 1 --2- 172.21.15.39:0/2419808765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55891ecfdad0 0x55891ecffeb0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec003f30 tx=0x7ff7ec009c20 comp rx=0 tx=0).stop 2023-07-21T18:42:42.260 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.230+0000 7ff8083f1b80 1 -- 172.21.15.39:0/2419808765 shutdown_connections 2023-07-21T18:42:42.260 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.230+0000 7ff8083f1b80 1 --2- 172.21.15.39:0/2419808765 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55891ecf22d0 0x55891ecf26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.260 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.230+0000 7ff8083f1b80 1 --2- 172.21.15.39:0/2419808765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55891ecfdad0 0x55891ecffeb0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.261 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.230+0000 7ff8083f1b80 1 --2- 172.21.15.39:0/2419808765 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55891ecf2be0 0x55891ecfd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.261 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.230+0000 7ff8083f1b80 1 -- 172.21.15.39:0/2419808765 >> 172.21.15.39:0/2419808765 conn(0x55891ece8da0 msgr2=0x55891ecee6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:42.261 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.230+0000 7ff8083f1b80 1 -- 172.21.15.39:0/2419808765 shutdown_connections 2023-07-21T18:42:42.261 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.230+0000 7ff8083f1b80 1 -- 172.21.15.39:0/2419808765 wait complete. 2023-07-21T18:42:42.261 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.230+0000 7ff8083f1b80 1 -- start start 2023-07-21T18:42:42.261 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.230+0000 7ff8083f1b80 1 Processor -- start 2023-07-21T18:42:42.261 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.231+0000 7ff8083f1b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55891ecf22d0 0x55891ecfd520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:42.262 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.231+0000 7ff8083f1b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55891ecf2be0 0x55891ecfa730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:42.262 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.231+0000 7ff8083f1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55891ecfdad0 0x55891ecfae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:42.262 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.231+0000 7ff8083f1b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55891ebe4ac0 con 0x55891ecf2be0 2023-07-21T18:42:42.262 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.231+0000 7ff8083f1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55891ebe47c0 con 0x55891ecfdad0 2023-07-21T18:42:42.262 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.231+0000 7ff8083f1b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55891eac8920 con 0x55891ecf22d0 2023-07-21T18:42:42.262 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.231+0000 7ff8083f1b80 10 client.0 _ll_register_callbacks cb 0x55891ebe2760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:42:42.262 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.231+0000 7ff7f9806700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55891ecf2be0 0x55891ecfa730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:42.262 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.231+0000 7ff7f9806700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55891ecf2be0 0x55891ecfa730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58316/0 (socket says 172.21.15.39:58316) 2023-07-21T18:42:42.263 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.231+0000 7ff7f9806700 1 -- 172.21.15.39:0/561297422 learned_addr learned my addr 172.21.15.39:0/561297422 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:42.263 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.231+0000 7ff7fa808700 1 --2- 172.21.15.39:0/561297422 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55891ecfdad0 0x55891ecfae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:42.263 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.231+0000 7ff7fa007700 1 --2- 172.21.15.39:0/561297422 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55891ecf22d0 0x55891ecfd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:42.263 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.231+0000 7ff7f9806700 1 -- 172.21.15.39:0/561297422 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55891ecf22d0 msgr2=0x55891ecfd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:42.263 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.231+0000 7ff7f9806700 1 --2- 172.21.15.39:0/561297422 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55891ecf22d0 0x55891ecfd520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.263 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.231+0000 7ff7f9806700 1 -- 172.21.15.39:0/561297422 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55891ecfdad0 msgr2=0x55891ecfae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:42.263 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.231+0000 7ff7f9806700 1 --2- 172.21.15.39:0/561297422 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55891ecfdad0 0x55891ecfae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.263 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.231+0000 7ff7f9806700 1 -- 172.21.15.39:0/561297422 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7ec003b60 con 0x55891ecf2be0 2023-07-21T18:42:42.264 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.231+0000 7ff7f9806700 1 --2- 172.21.15.39:0/561297422 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55891ecf2be0 0x55891ecfa730 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8009990 tx=0x7ff7e8009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17bc6bccc7a16a42 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:42.264 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.231+0000 7ff7ddffb700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:42:42.264 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.232+0000 7ff7ddffb700 1 -- 172.21.15.39:0/561297422 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e8004d70 con 0x55891ecf2be0 2023-07-21T18:42:42.264 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.232+0000 7ff7ddffb700 1 -- 172.21.15.39:0/561297422 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7ff7e800f460 con 0x55891ecf2be0 2023-07-21T18:42:42.264 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.232+0000 7ff8083f1b80 1 -- 172.21.15.39:0/561297422 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55891eb9e4f0 con 0x55891ecf2be0 2023-07-21T18:42:42.264 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.232+0000 7ff7ddffb700 1 -- 172.21.15.39:0/561297422 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7e800f620 con 0x55891ecf2be0 2023-07-21T18:42:42.264 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.232+0000 7ff8083f1b80 10 client.15906 fetch_fsmap learned FSMap version 26 2023-07-21T18:42:42.265 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.232+0000 7ff8083f1b80 1 -- 172.21.15.39:0/561297422 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55891ea5bc60 con 0x55891ecf2be0 2023-07-21T18:42:42.265 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.232+0000 7ff7ddffb700 1 -- 172.21.15.39:0/561297422 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7e803c020 con 0x55891ecf2be0 2023-07-21T18:42:42.265 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.232+0000 7ff7ddffb700 1 -- 172.21.15.39:0/561297422 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff7e80388b0 con 0x55891ecf2be0 2023-07-21T18:42:42.265 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.232+0000 7ff8083f1b80 10 client.15906 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T18:42:42.265 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.232+0000 7ff8083f1b80 10 client.15906 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T18:42:42.265 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.232+0000 7ff8083f1b80 20 client.15906 populate_metadata read hostname 'smithi039' 2023-07-21T18:42:42.265 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.232+0000 7ff8083f1b80 1 --2- 172.21.15.39:0/561297422 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55891ed0d440 0x55891ed0f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:42.266 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.232+0000 7ff8083f1b80 4 client.15906 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:42:42.266 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.232+0000 7ff8083f1b80 1 -- 172.21.15.39:0/561297422 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55891ec3deb0 con 0x55891ed0d440 2023-07-21T18:42:42.266 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.232+0000 7ff7fa007700 1 --2- 172.21.15.39:0/561297422 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55891ed0d440 0x55891ed0f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:42.266 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.233+0000 7ff7ddffb700 10 client.15906 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:42:42.266 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.233+0000 7ff7fa007700 1 --2- 172.21.15.39:0/561297422 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55891ed0d440 0x55891ed0f850 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd15769f63eb5bdf server_cookie=e873d622f8138fb1 in_seq=0 out_seq=0 2023-07-21T18:42:42.267 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.233+0000 7ff7ddffb700 1 -- 172.21.15.39:0/561297422 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55891ec3deb0 con 0x55891ed0d440 2023-07-21T18:42:42.267 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.233+0000 7ff7ddffb700 10 client.15906 handle_command_reply: tid=0 2023-07-21T18:42:42.268 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.233+0000 7ff8083f1b80 1 client.15906 shutdown 2023-07-21T18:42:42.268 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.234+0000 7ff8083f1b80 1 -- 172.21.15.39:0/561297422 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55891ecf2be0 msgr2=0x55891ecfa730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:42.268 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.234+0000 7ff8083f1b80 1 --2- 172.21.15.39:0/561297422 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55891ecf2be0 0x55891ecfa730 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8009990 tx=0x7ff7e8009ca0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.268 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.234+0000 7ff8083f1b80 1 -- 172.21.15.39:0/561297422 shutdown_connections 2023-07-21T18:42:42.268 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.234+0000 7ff8083f1b80 1 --2- 172.21.15.39:0/561297422 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55891ed0d440 0x55891ed0f850 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.268 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.234+0000 7ff8083f1b80 1 --2- 172.21.15.39:0/561297422 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55891ecfdad0 0x55891ecfae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.268 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.234+0000 7ff7ddffb700 0 client.15906 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:42:42.268 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.234+0000 7ff8083f1b80 1 --2- 172.21.15.39:0/561297422 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55891ecf22d0 0x55891ecfd520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.269 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.234+0000 7ff8083f1b80 1 --2- 172.21.15.39:0/561297422 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55891ecf2be0 0x55891ecfa730 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.269 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.234+0000 7ff8083f1b80 1 -- 172.21.15.39:0/561297422 >> 172.21.15.39:0/561297422 conn(0x55891ece8da0 msgr2=0x55891ecee230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:42.269 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.234+0000 7ff8083f1b80 1 -- 172.21.15.39:0/561297422 shutdown_connections 2023-07-21T18:42:42.269 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.234+0000 7ff8083f1b80 1 -- 172.21.15.39:0/561297422 wait complete. 2023-07-21T18:42:42.269 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.234+0000 7ff8083f1b80 20 client.15906 trim_cache size 0 max 16384 2023-07-21T18:42:42.273 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.251+0000 7ff8083f1b80 1 Processor -- start 2023-07-21T18:42:42.274 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.251+0000 7ff8083f1b80 1 -- start start 2023-07-21T18:42:42.274 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.251+0000 7ff8083f1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55891ecf22d0 0x55891ed09790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:42.274 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.251+0000 7ff8083f1b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55891ecf2be0 0x55891ecfd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:42.274 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.251+0000 7ff8083f1b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55891ecfdad0 0x55891ecf79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:42.274 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.251+0000 7ff8083f1b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55891ecf6280 con 0x55891ecfdad0 2023-07-21T18:42:42.274 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.251+0000 7ff8083f1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55891ecf63f0 con 0x55891ecf22d0 2023-07-21T18:42:42.274 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.251+0000 7ff8083f1b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55891ecfae30 con 0x55891ecf2be0 2023-07-21T18:42:42.275 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.251+0000 7ff7fa808700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55891ecfdad0 0x55891ecf79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:42.275 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.251+0000 7ff7fa808700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55891ecfdad0 0x55891ecf79a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58326/0 (socket says 172.21.15.39:58326) 2023-07-21T18:42:42.275 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.251+0000 7ff7fa808700 1 -- 172.21.15.39:0/1282270299 learned_addr learned my addr 172.21.15.39:0/1282270299 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:42.275 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.251+0000 7ff7f9806700 1 --2- 172.21.15.39:0/1282270299 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55891ecf2be0 0x55891ecfd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:42.275 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.251+0000 7ff7fa808700 1 -- 172.21.15.39:0/1282270299 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55891ecf2be0 msgr2=0x55891ecfd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:42.275 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.251+0000 7ff7fa808700 1 --2- 172.21.15.39:0/1282270299 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55891ecf2be0 0x55891ecfd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.275 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.251+0000 7ff7fa808700 1 -- 172.21.15.39:0/1282270299 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55891ecf22d0 msgr2=0x55891ed09790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:42.276 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.251+0000 7ff7fa007700 1 --2- 172.21.15.39:0/1282270299 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55891ecf22d0 0x55891ed09790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:42.276 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.251+0000 7ff7fa808700 1 --2- 172.21.15.39:0/1282270299 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55891ecf22d0 0x55891ed09790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.276 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.251+0000 7ff7fa808700 1 -- 172.21.15.39:0/1282270299 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55891ea99250 con 0x55891ecfdad0 2023-07-21T18:42:42.276 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.252+0000 7ff7fa808700 1 --2- 172.21.15.39:0/1282270299 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55891ecfdad0 0x55891ecf79a0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec007ce0 tx=0x7ff7ec007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abf233e7a49534fb server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:42.276 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.252+0000 7ff7e55ad700 1 -- 172.21.15.39:0/1282270299 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec004a90 con 0x55891ecfdad0 2023-07-21T18:42:42.276 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.252+0000 7ff7e55ad700 1 -- 172.21.15.39:0/1282270299 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7ec0079c0 con 0x55891ecfdad0 2023-07-21T18:42:42.276 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.252+0000 7ff7e55ad700 1 -- 172.21.15.39:0/1282270299 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec015af0 con 0x55891ecfdad0 2023-07-21T18:42:42.280 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.252+0000 7ff8083f1b80 1 -- 172.21.15.39:0/1282270299 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55891ecfdad0 msgr2=0x55891ecf79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:42.280 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.252+0000 7ff8083f1b80 1 --2- 172.21.15.39:0/1282270299 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55891ecfdad0 0x55891ecf79a0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec007ce0 tx=0x7ff7ec007d10 comp rx=0 tx=0).stop 2023-07-21T18:42:42.280 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.252+0000 7ff8083f1b80 1 -- 172.21.15.39:0/1282270299 shutdown_connections 2023-07-21T18:42:42.281 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.252+0000 7ff8083f1b80 1 --2- 172.21.15.39:0/1282270299 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55891ecf22d0 0x55891ed09790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.281 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.252+0000 7ff8083f1b80 1 --2- 172.21.15.39:0/1282270299 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55891ecfdad0 0x55891ecf79a0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.281 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.252+0000 7ff8083f1b80 1 --2- 172.21.15.39:0/1282270299 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55891ecf2be0 0x55891ecfd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.281 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.252+0000 7ff8083f1b80 1 -- 172.21.15.39:0/1282270299 >> 172.21.15.39:0/1282270299 conn(0x55891ece8da0 msgr2=0x55891ecece10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:42.281 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.252+0000 7ff8083f1b80 1 -- 172.21.15.39:0/1282270299 shutdown_connections 2023-07-21T18:42:42.281 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.252+0000 7ff8083f1b80 1 -- 172.21.15.39:0/1282270299 wait complete. 2023-07-21T18:42:42.281 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.253+0000 7ff8083f1b80 1 -- start start 2023-07-21T18:42:42.282 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.253+0000 7ff8083f1b80 1 Processor -- start 2023-07-21T18:42:42.282 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.253+0000 7ff8083f1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55891ecf22d0 0x55891ed02e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:42.282 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.253+0000 7ff8083f1b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55891ecf2be0 0x55891ed03390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:42.282 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.253+0000 7ff8083f1b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55891ecfdad0 0x55891ed03910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:42.282 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.253+0000 7ff8083f1b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55891ed614a0 con 0x55891ecf2be0 2023-07-21T18:42:42.282 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.253+0000 7ff8083f1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55891ed61610 con 0x55891ecf22d0 2023-07-21T18:42:42.282 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.253+0000 7ff8083f1b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55891ed617d0 con 0x55891ecfdad0 2023-07-21T18:42:42.282 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.253+0000 7ff8083f1b80 10 client.0 _ll_register_callbacks cb 0x55891ebe2760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:42:42.283 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.253+0000 7ff7f9806700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55891ecf2be0 0x55891ed03390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:42.283 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.253+0000 7ff7f9806700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55891ecf2be0 0x55891ed03390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58330/0 (socket says 172.21.15.39:58330) 2023-07-21T18:42:42.283 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.253+0000 7ff7f9806700 1 -- 172.21.15.39:0/1800755414 learned_addr learned my addr 172.21.15.39:0/1800755414 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:42.283 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.253+0000 7ff7fa808700 1 --2- 172.21.15.39:0/1800755414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55891ecfdad0 0x55891ed03910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:42.283 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.253+0000 7ff7f9806700 1 -- 172.21.15.39:0/1800755414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55891ecfdad0 msgr2=0x55891ed03910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:42.283 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.253+0000 7ff7f9806700 1 --2- 172.21.15.39:0/1800755414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55891ecfdad0 0x55891ed03910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.283 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.253+0000 7ff7fa007700 1 --2- 172.21.15.39:0/1800755414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55891ecf22d0 0x55891ed02e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:42.284 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.253+0000 7ff7f9806700 1 -- 172.21.15.39:0/1800755414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55891ecf22d0 msgr2=0x55891ed02e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:42.284 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.254+0000 7ff7f9806700 1 --2- 172.21.15.39:0/1800755414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55891ecf22d0 0x55891ed02e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.284 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.254+0000 7ff7f9806700 1 -- 172.21.15.39:0/1800755414 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55891ea72de0 con 0x55891ecf2be0 2023-07-21T18:42:42.284 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.254+0000 7ff7fa808700 1 --2- 172.21.15.39:0/1800755414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55891ecfdad0 0x55891ed03910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:42:42.284 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.254+0000 7ff7fa007700 1 --2- 172.21.15.39:0/1800755414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55891ecf22d0 0x55891ed02e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:42:42.284 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.254+0000 7ff7f9806700 1 --2- 172.21.15.39:0/1800755414 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55891ecf2be0 0x55891ed03390 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8038ce0 tx=0x7ff7e80387a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=504b487dbbdcc77 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:42.284 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.254+0000 7ff7ddffb700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:42:42.285 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.254+0000 7ff7ddffb700 1 -- 172.21.15.39:0/1800755414 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e800b2a0 con 0x55891ecf2be0 2023-07-21T18:42:42.285 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.254+0000 7ff7ddffb700 1 -- 172.21.15.39:0/1800755414 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7ff7e803d080 con 0x55891ecf2be0 2023-07-21T18:42:42.285 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.254+0000 7ff8083f1b80 1 -- 172.21.15.39:0/1800755414 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff7ec00f040 con 0x55891ecf2be0 2023-07-21T18:42:42.285 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.254+0000 7ff7ddffb700 1 -- 172.21.15.39:0/1800755414 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7e80047e0 con 0x55891ecf2be0 2023-07-21T18:42:42.285 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.254+0000 7ff8083f1b80 10 client.15912 fetch_fsmap learned FSMap version 26 2023-07-21T18:42:42.285 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.254+0000 7ff8083f1b80 1 -- 172.21.15.39:0/1800755414 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55891ea712b0 con 0x55891ecf2be0 2023-07-21T18:42:42.285 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.254+0000 7ff7ddffb700 1 -- 172.21.15.39:0/1800755414 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7e804d080 con 0x55891ecf2be0 2023-07-21T18:42:42.285 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.254+0000 7ff7ddffb700 1 -- 172.21.15.39:0/1800755414 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff7e800f700 con 0x55891ecf2be0 2023-07-21T18:42:42.286 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.254+0000 7ff8083f1b80 10 client.15912 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T18:42:42.286 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.254+0000 7ff8083f1b80 10 client.15912 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T18:42:42.286 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.254+0000 7ff8083f1b80 20 client.15912 populate_metadata read hostname 'smithi039' 2023-07-21T18:42:42.286 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.254+0000 7ff8083f1b80 1 --2- 172.21.15.39:0/1800755414 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55891ed62c20 0x55891ed65030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:42.286 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.254+0000 7ff8083f1b80 4 client.15912 mds_command: new command op to 24536 tid=0[{"prefix": "scrub status"}] 2023-07-21T18:42:42.286 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.254+0000 7ff8083f1b80 1 -- 172.21.15.39:0/1800755414 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55891ec00050 con 0x55891ed62c20 2023-07-21T18:42:42.286 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.255+0000 7ff7fa007700 1 --2- 172.21.15.39:0/1800755414 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55891ed62c20 0x55891ed65030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:42.287 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.255+0000 7ff7fa007700 1 --2- 172.21.15.39:0/1800755414 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55891ed62c20 0x55891ed65030 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=798cefa4c160799f server_cookie=14abae7553e6e552 in_seq=0 out_seq=0 2023-07-21T18:42:42.287 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.255+0000 7ff7ddffb700 10 client.15912 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:42:42.287 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.255+0000 7ff7ddffb700 1 -- 172.21.15.39:0/1800755414 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55891ec00050 con 0x55891ed62c20 2023-07-21T18:42:42.287 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.255+0000 7ff7ddffb700 10 client.15912 handle_command_reply: tid=0 2023-07-21T18:42:42.287 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.255+0000 7ff8083f1b80 1 client.15912 shutdown 2023-07-21T18:42:42.287 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.256+0000 7ff8083f1b80 1 -- 172.21.15.39:0/1800755414 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55891ecf2be0 msgr2=0x55891ed03390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:42.287 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.256+0000 7ff8083f1b80 1 --2- 172.21.15.39:0/1800755414 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55891ecf2be0 0x55891ed03390 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8038ce0 tx=0x7ff7e80387a0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.287 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.256+0000 7ff8083f1b80 1 -- 172.21.15.39:0/1800755414 shutdown_connections 2023-07-21T18:42:42.288 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.256+0000 7ff8083f1b80 1 --2- 172.21.15.39:0/1800755414 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55891ed62c20 0x55891ed65030 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.288 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.256+0000 7ff8083f1b80 1 --2- 172.21.15.39:0/1800755414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55891ecf22d0 0x55891ed02e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.288 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.256+0000 7ff7ddffb700 0 client.15912 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:42:42.288 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.256+0000 7ff8083f1b80 1 --2- 172.21.15.39:0/1800755414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55891ecfdad0 0x55891ed03910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.288 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.256+0000 7ff8083f1b80 1 --2- 172.21.15.39:0/1800755414 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55891ecf2be0 0x55891ed03390 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.288 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.256+0000 7ff8083f1b80 1 -- 172.21.15.39:0/1800755414 >> 172.21.15.39:0/1800755414 conn(0x55891ece8da0 msgr2=0x55891ed09790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:42.288 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.256+0000 7ff8083f1b80 1 -- 172.21.15.39:0/1800755414 shutdown_connections 2023-07-21T18:42:42.289 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.256+0000 7ff8083f1b80 1 -- 172.21.15.39:0/1800755414 wait complete. 2023-07-21T18:42:42.289 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.256+0000 7ff8083f1b80 20 client.15912 trim_cache size 0 max 16384 2023-07-21T18:42:42.289 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2023-07-21T18:42:42.289 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T18:42:42.289 INFO:teuthology.orchestra.run.smithi039.stdout: "scrubs": { 2023-07-21T18:42:42.290 INFO:teuthology.orchestra.run.smithi039.stdout: "e64b5b31-b4f8-42de-af6b-50e6288c6e95": { 2023-07-21T18:42:42.290 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2023-07-21T18:42:42.290 INFO:teuthology.orchestra.run.smithi039.stdout: "tag": "e64b5b31-b4f8-42de-af6b-50e6288c6e95", 2023-07-21T18:42:42.290 INFO:teuthology.orchestra.run.smithi039.stdout: "options": "recursive,force" 2023-07-21T18:42:42.290 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T18:42:42.290 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T18:42:42.290 INFO:teuthology.orchestra.run.smithi039.stdout:} 2023-07-21T18:42:42.291 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.256+0000 7ff7fc26b700 1 -- 172.21.15.39:0/1746345002 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7ff7e00a1460 msgr2=0x7ff7e00a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:42.291 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.256+0000 7ff7fc26b700 1 --2- 172.21.15.39:0/1746345002 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7ff7e00a1460 0x7ff7e00a3910 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff7f4199440 tx=0x7ff7e8005fb0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.291 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.256+0000 7ff7fc26b700 1 -- 172.21.15.39:0/1746345002 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7f4100ca0 msgr2=0x7ff7f41981a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:42.292 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.256+0000 7ff7fc26b700 1 --2- 172.21.15.39:0/1746345002 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7f4100ca0 0x7ff7f41981a0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7ff7f000e9a0 tx=0x7ff7f000ecb0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.292 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.256+0000 7ff7fc26b700 1 -- 172.21.15.39:0/1746345002 shutdown_connections 2023-07-21T18:42:42.292 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.256+0000 7ff7fc26b700 1 --2- 172.21.15.39:0/1746345002 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7f410a070 0x7ff7f419c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.292 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.256+0000 7ff7fc26b700 1 --2- 172.21.15.39:0/1746345002 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7f4100ca0 0x7ff7f41981a0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.292 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.256+0000 7ff7fc26b700 1 --2- 172.21.15.39:0/1746345002 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7ff7e00a1460 0x7ff7e00a3910 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.292 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.256+0000 7ff7fc26b700 1 --2- 172.21.15.39:0/1746345002 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7f41015b0 0x7ff7f41986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:42.292 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.256+0000 7ff7fc26b700 1 -- 172.21.15.39:0/1746345002 >> 172.21.15.39:0/1746345002 conn(0x7ff7f40fa4b0 msgr2=0x7ff7f40fc890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:42.293 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.256+0000 7ff7fc26b700 1 -- 172.21.15.39:0/1746345002 shutdown_connections 2023-07-21T18:42:42.293 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:42:42.256+0000 7ff7fc26b700 1 -- 172.21.15.39:0/1746345002 wait complete. 2023-07-21T18:42:42.293 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:42:42 smithi039 ceph-mon[96489]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.4 KiB/s wr, 14 op/s 2023-07-21T18:42:42.294 INFO:tasks.cephfs.filesystem:scrub status for tag:e64b5b31-b4f8-42de-af6b-50e6288c6e95 - {'path': '/', 'tag': 'e64b5b31-b4f8-42de-af6b-50e6288c6e95', 'options': 'recursive,force'} 2023-07-21T18:42:42.387 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:42:42 smithi155 ceph-mon[97934]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.4 KiB/s wr, 14 op/s 2023-07-21T18:42:42.618 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:42:42 smithi193 ceph-mon[98204]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.4 KiB/s wr, 14 op/s 2023-07-21T18:42:43.387 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:42:43 smithi155 ceph-mon[97934]: scrub queued for path: / 2023-07-21T18:42:43.387 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:42:43 smithi155 ceph-mon[97934]: scrub summary: idle+waiting paths [/] 2023-07-21T18:42:43.388 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:42:43 smithi155 ceph-mon[97934]: scrub summary: active paths [/] 2023-07-21T18:42:43.388 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:42:43 smithi155 ceph-mon[97934]: scrub summary: idle+waiting paths [/] 2023-07-21T18:42:43.436 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:42:43 smithi039 ceph-mon[96489]: scrub queued for path: / 2023-07-21T18:42:43.436 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:42:43 smithi039 ceph-mon[96489]: scrub summary: idle+waiting paths [/] 2023-07-21T18:42:43.436 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:42:43 smithi039 ceph-mon[96489]: scrub summary: active paths [/] 2023-07-21T18:42:43.437 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:42:43 smithi039 ceph-mon[96489]: scrub summary: idle+waiting paths [/] 2023-07-21T18:42:43.618 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:42:43 smithi193 ceph-mon[98204]: scrub queued for path: / 2023-07-21T18:42:43.619 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:42:43 smithi193 ceph-mon[98204]: scrub summary: idle+waiting paths [/] 2023-07-21T18:42:43.619 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:42:43 smithi193 ceph-mon[98204]: scrub summary: active paths [/] 2023-07-21T18:42:43.619 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:42:43 smithi193 ceph-mon[98204]: scrub summary: idle+waiting paths [/] 2023-07-21T18:42:44.436 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:42:44 smithi039 ceph-mon[96489]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2023-07-21T18:42:44.436 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:42:44 smithi039 ceph-mon[96489]: scrub summary: idle 2023-07-21T18:42:44.618 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:42:44 smithi193 ceph-mon[98204]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2023-07-21T18:42:44.618 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:42:44 smithi193 ceph-mon[98204]: scrub summary: idle 2023-07-21T18:42:44.637 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:42:44 smithi155 ceph-mon[97934]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2023-07-21T18:42:44.637 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:42:44 smithi155 ceph-mon[97934]: scrub summary: idle 2023-07-21T18:42:45.575 INFO:tasks.workunit.client.0.smithi039.stderr:Updating files: 96% (11339/11710) Updating files: 97% (11359/11710) Updating files: 98% (11476/11710) Updating files: 99% (11593/11710) Updating files: 100% (11710/11710) Updating files: 100% (11710/11710), done. 2023-07-21T18:42:46.286 INFO:tasks.workunit.client.0.smithi039.stderr:Note: switching to '81cf785aa7781c4e08381c540f890cb9bc987ef9'. 2023-07-21T18:42:46.287 INFO:tasks.workunit.client.0.smithi039.stderr: 2023-07-21T18:42:46.287 INFO:tasks.workunit.client.0.smithi039.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2023-07-21T18:42:46.287 INFO:tasks.workunit.client.0.smithi039.stderr:changes and commit them, and you can discard any commits you make in this 2023-07-21T18:42:46.287 INFO:tasks.workunit.client.0.smithi039.stderr:state without impacting any branches by switching back to a branch. 2023-07-21T18:42:46.287 INFO:tasks.workunit.client.0.smithi039.stderr: 2023-07-21T18:42:46.287 INFO:tasks.workunit.client.0.smithi039.stderr:If you want to create a new branch to retain commits you create, you may 2023-07-21T18:42:46.288 INFO:tasks.workunit.client.0.smithi039.stderr:do so (now or later) by using -c with the switch command. Example: 2023-07-21T18:42:46.288 INFO:tasks.workunit.client.0.smithi039.stderr: 2023-07-21T18:42:46.288 INFO:tasks.workunit.client.0.smithi039.stderr: git switch -c 2023-07-21T18:42:46.288 INFO:tasks.workunit.client.0.smithi039.stderr: 2023-07-21T18:42:46.288 INFO:tasks.workunit.client.0.smithi039.stderr:Or undo this operation with: 2023-07-21T18:42:46.288 INFO:tasks.workunit.client.0.smithi039.stderr: 2023-07-21T18:42:46.288 INFO:tasks.workunit.client.0.smithi039.stderr: git switch - 2023-07-21T18:42:46.288 INFO:tasks.workunit.client.0.smithi039.stderr: 2023-07-21T18:42:46.289 INFO:tasks.workunit.client.0.smithi039.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2023-07-21T18:42:46.289 INFO:tasks.workunit.client.0.smithi039.stderr: 2023-07-21T18:42:46.289 INFO:tasks.workunit.client.0.smithi039.stderr:HEAD is now at 81cf785aa77 Merge remote-tracking branch 'temp/wip-61599' into wip-rishabh-2023Jul13-m-quota 2023-07-21T18:42:46.292 DEBUG:teuthology.orchestra.run.smithi039:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2023-07-21T18:42:46.331 INFO:tasks.workunit.client.0.smithi039.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2023-07-21T18:42:46.333 INFO:tasks.workunit.client.0.smithi039.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-07-21T18:42:46.334 INFO:tasks.workunit.client.0.smithi039.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2023-07-21T18:42:46.436 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:42:46 smithi039 ceph-mon[96489]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2023-07-21T18:42:46.618 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:42:46 smithi193 ceph-mon[98204]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2023-07-21T18:42:46.637 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:42:46 smithi155 ceph-mon[97934]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2023-07-21T18:42:46.891 INFO:tasks.workunit.client.0.smithi039.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2023-07-21T18:42:46.948 INFO:tasks.workunit.client.0.smithi039.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2023-07-21T18:42:46.987 INFO:tasks.workunit.client.0.smithi039.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-07-21T18:42:46.989 INFO:tasks.workunit.client.0.smithi039.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-07-21T18:42:46.989 INFO:tasks.workunit.client.0.smithi039.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2023-07-21T18:42:47.026 INFO:tasks.workunit.client.0.smithi039.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-07-21T18:42:47.030 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-07-21T18:42:47.031 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2023-07-21T18:42:47.088 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2023-07-21T18:42:47.089 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2023-07-21T18:42:47.090 DEBUG:teuthology.orchestra.run.smithi039:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=81cf785aa7781c4e08381c540f890cb9bc987ef9 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2023-07-21T18:42:47.152 INFO:tasks.workunit.client.0.smithi039.stderr:+ dbench 1 2023-07-21T18:42:47.157 INFO:tasks.workunit.client.0.smithi039.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2023-07-21T18:42:47.157 INFO:tasks.workunit.client.0.smithi039.stdout: 2023-07-21T18:42:47.157 INFO:tasks.workunit.client.0.smithi039.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2023-07-21T18:42:47.158 INFO:tasks.workunit.client.0.smithi039.stdout:0 of 1 processes prepared for launch 0 sec 2023-07-21T18:42:47.258 INFO:tasks.workunit.client.0.smithi039.stdout:1 of 1 processes prepared for launch 0 sec 2023-07-21T18:42:47.259 INFO:tasks.workunit.client.0.smithi039.stdout:releasing clients 2023-07-21T18:42:48.258 INFO:tasks.workunit.client.0.smithi039.stdout: 1 2926 33.48 MB/sec warmup 1 sec latency 54.344 ms 2023-07-21T18:42:48.436 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:42:48 smithi039 ceph-mon[96489]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 255 B/s wr, 13 op/s 2023-07-21T18:42:48.618 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:42:48 smithi193 ceph-mon[98204]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 255 B/s wr, 13 op/s 2023-07-21T18:42:48.637 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:42:48 smithi155 ceph-mon[97934]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 255 B/s wr, 13 op/s 2023-07-21T18:42:49.258 INFO:tasks.workunit.client.0.smithi039.stdout: 1 5385 22.73 MB/sec warmup 2 sec latency 7.852 ms 2023-07-21T18:42:50.258 INFO:tasks.workunit.client.0.smithi039.stdout: 1 7891 19.87 MB/sec warmup 3 sec latency 8.297 ms 2023-07-21T18:42:50.436 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:42:50 smithi039 ceph-mon[96489]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 9 op/s 2023-07-21T18:42:50.436 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:42:50 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:42:50.618 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:42:50 smithi193 ceph-mon[98204]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 9 op/s 2023-07-21T18:42:50.618 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:42:50 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:42:50.637 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:42:50 smithi155 ceph-mon[97934]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 9 op/s 2023-07-21T18:42:50.637 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:42:50 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:42:51.258 INFO:tasks.workunit.client.0.smithi039.stdout: 1 10870 19.50 MB/sec warmup 4 sec latency 56.389 ms 2023-07-21T18:42:51.436 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:42:51 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:42:51.436 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:42:51 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:42:51.436 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:42:51 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:42:51.618 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:42:51 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:42:51.618 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:42:51 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:42:51.619 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:42:51 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:42:51.637 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:42:51 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:42:51.637 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:42:51 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:42:51.637 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:42:51 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:42:52.258 INFO:tasks.workunit.client.0.smithi039.stdout: 1 13419 17.87 MB/sec warmup 5 sec latency 15.043 ms 2023-07-21T18:42:52.436 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:42:52 smithi039 ceph-mon[96489]: pgmap v36: 97 pgs: 97 active+clean; 5.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 391 KiB/s wr, 62 op/s 2023-07-21T18:42:52.618 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:42:52 smithi193 ceph-mon[98204]: pgmap v36: 97 pgs: 97 active+clean; 5.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 391 KiB/s wr, 62 op/s 2023-07-21T18:42:52.637 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:42:52 smithi155 ceph-mon[97934]: pgmap v36: 97 pgs: 97 active+clean; 5.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 391 KiB/s wr, 62 op/s 2023-07-21T18:42:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.674+0000 7fb6fd961700 1 -- 172.21.15.39:0/639878176 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6f80fea80 msgr2=0x7fb6f80feef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.674+0000 7fb6fd961700 1 --2- 172.21.15.39:0/639878176 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6f80fea80 0x7fb6f80feef0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8009990 tx=0x7fb6e8009ca0 comp rx=0 tx=0).stop 2023-07-21T18:42:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.674+0000 7fb6fd961700 1 -- 172.21.15.39:0/639878176 shutdown_connections 2023-07-21T18:42:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.674+0000 7fb6fd961700 1 --2- 172.21.15.39:0/639878176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6f8104a90 0x7fb6f8104e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.674+0000 7fb6fd961700 1 --2- 172.21.15.39:0/639878176 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb6f80ff430 0x7fb6f810b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.674+0000 7fb6fd961700 1 --2- 172.21.15.39:0/639878176 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6f80fea80 0x7fb6f80feef0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.674+0000 7fb6fd961700 1 -- 172.21.15.39:0/639878176 >> 172.21.15.39:0/639878176 conn(0x7fb6f80769b0 msgr2=0x7fb6f8076db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.674+0000 7fb6fd961700 1 -- 172.21.15.39:0/639878176 shutdown_connections 2023-07-21T18:42:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.674+0000 7fb6fd961700 1 -- 172.21.15.39:0/639878176 wait complete. 2023-07-21T18:42:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.675+0000 7fb6fd961700 1 Processor -- start 2023-07-21T18:42:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.675+0000 7fb6fd961700 1 -- start start 2023-07-21T18:42:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.675+0000 7fb6fd961700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6f80fea80 0x7fb6f819aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.675+0000 7fb6fd961700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6f80ff430 0x7fb6f819b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.675+0000 7fb6fd961700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb6f8104a90 0x7fb6f8194920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.675+0000 7fb6fd961700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f8194f90 con 0x7fb6f80fea80 2023-07-21T18:42:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.675+0000 7fb6fd961700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f8195100 con 0x7fb6f80ff430 2023-07-21T18:42:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.675+0000 7fb6fd961700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f81952c0 con 0x7fb6f8104a90 2023-07-21T18:42:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.675+0000 7fb6f6ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6f80fea80 0x7fb6f819aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.675+0000 7fb6f6ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6f80fea80 0x7fb6f819aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58368/0 (socket says 172.21.15.39:58368) 2023-07-21T18:42:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.675+0000 7fb6f6ffd700 1 -- 172.21.15.39:0/1687915620 learned_addr learned my addr 172.21.15.39:0/1687915620 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.675+0000 7fb6f67fc700 1 --2- 172.21.15.39:0/1687915620 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6f80ff430 0x7fb6f819b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.675+0000 7fb6f77fe700 1 --2- 172.21.15.39:0/1687915620 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb6f8104a90 0x7fb6f8194920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.676+0000 7fb6f6ffd700 1 -- 172.21.15.39:0/1687915620 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb6f8104a90 msgr2=0x7fb6f8194920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.676+0000 7fb6f6ffd700 1 --2- 172.21.15.39:0/1687915620 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb6f8104a90 0x7fb6f8194920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.676+0000 7fb6f6ffd700 1 -- 172.21.15.39:0/1687915620 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6f80ff430 msgr2=0x7fb6f819b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.676+0000 7fb6f6ffd700 1 --2- 172.21.15.39:0/1687915620 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6f80ff430 0x7fb6f819b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.676+0000 7fb6f6ffd700 1 -- 172.21.15.39:0/1687915620 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6e80095f0 con 0x7fb6f80fea80 2023-07-21T18:42:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.676+0000 7fb6f67fc700 1 --2- 172.21.15.39:0/1687915620 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6f80ff430 0x7fb6f819b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:42:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.676+0000 7fb6f6ffd700 1 --2- 172.21.15.39:0/1687915620 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6f80fea80 0x7fb6f819aac0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec00e970 tx=0x7fb6ec00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.676+0000 7fb6fc95f700 1 -- 172.21.15.39:0/1687915620 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec009790 con 0x7fb6f80fea80 2023-07-21T18:42:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.676+0000 7fb6fc95f700 1 -- 172.21.15.39:0/1687915620 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6ec00c7e0 con 0x7fb6f80fea80 2023-07-21T18:42:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.676+0000 7fb6fd961700 1 -- 172.21.15.39:0/1687915620 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6f81955a0 con 0x7fb6f80fea80 2023-07-21T18:42:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.676+0000 7fb6fc95f700 1 -- 172.21.15.39:0/1687915620 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec003c70 con 0x7fb6f80fea80 2023-07-21T18:42:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.676+0000 7fb6fd961700 1 -- 172.21.15.39:0/1687915620 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6f81a1ab0 con 0x7fb6f80fea80 2023-07-21T18:42:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.679+0000 7fb6fc95f700 1 -- 172.21.15.39:0/1687915620 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7fb6ec00cc50 con 0x7fb6f80fea80 2023-07-21T18:42:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.679+0000 7fb6fc95f700 1 --2- 172.21.15.39:0/1687915620 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fb6e00a13e0 0x7fb6e00a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.680+0000 7fb6f67fc700 1 --2- 172.21.15.39:0/1687915620 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fb6e00a13e0 0x7fb6e00a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.680+0000 7fb6fc95f700 1 -- 172.21.15.39:0/1687915620 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fb6ec014070 con 0x7fb6f80fea80 2023-07-21T18:42:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.680+0000 7fb6f67fc700 1 --2- 172.21.15.39:0/1687915620 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fb6e00a13e0 0x7fb6e00a3890 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb6f80fc910 tx=0x7fb6e8005fd0 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.680+0000 7fb709ae7b80 1 Processor -- start 2023-07-21T18:42:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.680+0000 7fb709ae7b80 1 -- start start 2023-07-21T18:42:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.680+0000 7fb709ae7b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f641e66b0 0x559f64392010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.680+0000 7fb709ae7b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f64392550 0x559f6438c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.680+0000 7fb709ae7b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f6438c790 0x559f6438cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.680+0000 7fb709ae7b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f6428fc50 con 0x559f64392550 2023-07-21T18:42:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.680+0000 7fb709ae7b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f642e29e0 con 0x559f641e66b0 2023-07-21T18:42:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.680+0000 7fb709ae7b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f642dbe70 con 0x559f6438c790 2023-07-21T18:42:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.681+0000 7fb6f6ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f641e66b0 0x559f64392010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.681+0000 7fb6f67fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f64392550 0x559f6438c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.681+0000 7fb6f6ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f641e66b0 0x559f64392010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:42740/0 (socket says 172.21.15.39:42740) 2023-07-21T18:42:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.681+0000 7fb6f6ffd700 1 -- 172.21.15.39:0/3328463958 learned_addr learned my addr 172.21.15.39:0/3328463958 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.681+0000 7fb6f77fe700 1 --2- 172.21.15.39:0/3328463958 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f6438c790 0x559f6438cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.681+0000 7fb6f6ffd700 1 -- 172.21.15.39:0/3328463958 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f6438c790 msgr2=0x559f6438cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.681+0000 7fb6f6ffd700 1 --2- 172.21.15.39:0/3328463958 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f6438c790 0x559f6438cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.681+0000 7fb6f6ffd700 1 -- 172.21.15.39:0/3328463958 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f64392550 msgr2=0x559f6438c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.681+0000 7fb6f6ffd700 1 --2- 172.21.15.39:0/3328463958 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f64392550 0x559f6438c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.681+0000 7fb6f6ffd700 1 -- 172.21.15.39:0/3328463958 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f64109d80 con 0x559f641e66b0 2023-07-21T18:42:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.681+0000 7fb6f67fc700 1 --2- 172.21.15.39:0/3328463958 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f64392550 0x559f6438c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:42:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.681+0000 7fb6f6ffd700 1 --2- 172.21.15.39:0/3328463958 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f641e66b0 0x559f64392010 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec099c90 tx=0x7fb6ec0998f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0aa40fc707b6504 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.681+0000 7fb6ddffb700 1 -- 172.21.15.39:0/3328463958 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec09dfc0 con 0x559f641e66b0 2023-07-21T18:42:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.681+0000 7fb6ddffb700 1 -- 172.21.15.39:0/3328463958 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6ec0a23e0 con 0x559f641e66b0 2023-07-21T18:42:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.681+0000 7fb6ddffb700 1 -- 172.21.15.39:0/3328463958 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec0996e0 con 0x559f641e66b0 2023-07-21T18:42:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.682+0000 7fb709ae7b80 1 -- 172.21.15.39:0/3328463958 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f641e66b0 msgr2=0x559f64392010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.682+0000 7fb709ae7b80 1 --2- 172.21.15.39:0/3328463958 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f641e66b0 0x559f64392010 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec099c90 tx=0x7fb6ec0998f0 comp rx=0 tx=0).stop 2023-07-21T18:42:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.682+0000 7fb709ae7b80 1 -- 172.21.15.39:0/3328463958 shutdown_connections 2023-07-21T18:42:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.682+0000 7fb709ae7b80 1 --2- 172.21.15.39:0/3328463958 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f641e66b0 0x559f64392010 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.682+0000 7fb709ae7b80 1 --2- 172.21.15.39:0/3328463958 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f6438c790 0x559f6438cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.682+0000 7fb709ae7b80 1 --2- 172.21.15.39:0/3328463958 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f64392550 0x559f6438c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.682+0000 7fb709ae7b80 1 -- 172.21.15.39:0/3328463958 >> 172.21.15.39:0/3328463958 conn(0x559f6437fbe0 msgr2=0x559f64384f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:52.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.682+0000 7fb709ae7b80 1 -- 172.21.15.39:0/3328463958 shutdown_connections 2023-07-21T18:42:52.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.682+0000 7fb709ae7b80 1 -- 172.21.15.39:0/3328463958 wait complete. 2023-07-21T18:42:52.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.683+0000 7fb709ae7b80 1 -- start start 2023-07-21T18:42:52.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.683+0000 7fb709ae7b80 1 Processor -- start 2023-07-21T18:42:52.710 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.683+0000 7fb709ae7b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f641e66b0 0x559f64398740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:52.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.683+0000 7fb709ae7b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f6438c790 0x559f64395a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:52.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.683+0000 7fb709ae7b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f64392550 0x559f64395fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:52.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.683+0000 7fb709ae7b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f6427ba60 con 0x559f6438c790 2023-07-21T18:42:52.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.683+0000 7fb709ae7b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f6427b760 con 0x559f64392550 2023-07-21T18:42:52.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.683+0000 7fb709ae7b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f6415f8c0 con 0x559f641e66b0 2023-07-21T18:42:52.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.683+0000 7fb709ae7b80 10 client.0 _ll_register_callbacks cb 0x559f64279700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:42:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.683+0000 7fb6f67fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f6438c790 0x559f64395a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.683+0000 7fb6f67fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f6438c790 0x559f64395a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58384/0 (socket says 172.21.15.39:58384) 2023-07-21T18:42:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.683+0000 7fb6f67fc700 1 -- 172.21.15.39:0/187876905 learned_addr learned my addr 172.21.15.39:0/187876905 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.683+0000 7fb6f77fe700 1 --2- 172.21.15.39:0/187876905 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f64392550 0x559f64395fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.683+0000 7fb6f6ffd700 1 --2- 172.21.15.39:0/187876905 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f641e66b0 0x559f64398740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.684+0000 7fb6f67fc700 1 -- 172.21.15.39:0/187876905 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f641e66b0 msgr2=0x559f64398740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.684+0000 7fb6f67fc700 1 --2- 172.21.15.39:0/187876905 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f641e66b0 0x559f64398740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.684+0000 7fb6f67fc700 1 -- 172.21.15.39:0/187876905 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f64392550 msgr2=0x559f64395fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.684+0000 7fb6f67fc700 1 --2- 172.21.15.39:0/187876905 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f64392550 0x559f64395fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.684+0000 7fb6f67fc700 1 -- 172.21.15.39:0/187876905 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6f810c1c0 con 0x559f6438c790 2023-07-21T18:42:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.684+0000 7fb6f77fe700 1 --2- 172.21.15.39:0/187876905 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f64392550 0x559f64395fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:42:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.684+0000 7fb6f67fc700 1 --2- 172.21.15.39:0/187876905 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f6438c790 0x559f64395a60 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8009990 tx=0x7fb6e801a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=580082b7330b1069 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.684+0000 7fb6d6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:42:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.684+0000 7fb6d6ffd700 1 -- 172.21.15.39:0/187876905 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e802b050 con 0x559f6438c790 2023-07-21T18:42:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.684+0000 7fb6d6ffd700 1 -- 172.21.15.39:0/187876905 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fb6e8026a10 con 0x559f6438c790 2023-07-21T18:42:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.684+0000 7fb709ae7b80 1 -- 172.21.15.39:0/187876905 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559f64235490 con 0x559f6438c790 2023-07-21T18:42:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.684+0000 7fb6d6ffd700 1 -- 172.21.15.39:0/187876905 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb6e8026bd0 con 0x559f6438c790 2023-07-21T18:42:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.684+0000 7fb709ae7b80 10 client.15924 fetch_fsmap learned FSMap version 26 2023-07-21T18:42:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.684+0000 7fb709ae7b80 1 -- 172.21.15.39:0/187876905 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7fb6f81a1ab0 con 0x559f6438c790 2023-07-21T18:42:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.684+0000 7fb6d6ffd700 1 -- 172.21.15.39:0/187876905 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559f64235490 con 0x559f6438c790 2023-07-21T18:42:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.685+0000 7fb6d6ffd700 1 -- 172.21.15.39:0/187876905 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb6e8024c80 con 0x559f6438c790 2023-07-21T18:42:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.685+0000 7fb709ae7b80 10 client.15924 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T18:42:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.685+0000 7fb709ae7b80 10 client.15924 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:42:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.685+0000 7fb709ae7b80 20 client.15924 populate_metadata read hostname 'smithi039' 2023-07-21T18:42:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.685+0000 7fb709ae7b80 1 --2- 172.21.15.39:0/187876905 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x559f643a5d90 0x559f643a81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.685+0000 7fb709ae7b80 4 client.15924 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:42:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.685+0000 7fb709ae7b80 1 -- 172.21.15.39:0/187876905 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559f642d4e50 con 0x559f643a5d90 2023-07-21T18:42:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.685+0000 7fb6f6ffd700 1 --2- 172.21.15.39:0/187876905 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x559f643a5d90 0x559f643a81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.685+0000 7fb6d6ffd700 10 client.15924 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:42:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.685+0000 7fb6f6ffd700 1 --2- 172.21.15.39:0/187876905 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x559f643a5d90 0x559f643a81a0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de177b00b6be8597 server_cookie=64b696388ba80562 in_seq=0 out_seq=0 2023-07-21T18:42:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.686+0000 7fb6d6ffd700 1 -- 172.21.15.39:0/187876905 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559f642d4e50 con 0x559f643a5d90 2023-07-21T18:42:52.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.686+0000 7fb6d6ffd700 10 client.15924 handle_command_reply: tid=0 2023-07-21T18:42:52.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.686+0000 7fb709ae7b80 1 client.15924 shutdown 2023-07-21T18:42:52.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.686+0000 7fb709ae7b80 1 -- 172.21.15.39:0/187876905 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f6438c790 msgr2=0x559f64395a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:52.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.686+0000 7fb709ae7b80 1 --2- 172.21.15.39:0/187876905 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f6438c790 0x559f64395a60 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8009990 tx=0x7fb6e801a050 comp rx=0 tx=0).stop 2023-07-21T18:42:52.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.687+0000 7fb709ae7b80 1 -- 172.21.15.39:0/187876905 shutdown_connections 2023-07-21T18:42:52.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.687+0000 7fb709ae7b80 1 --2- 172.21.15.39:0/187876905 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x559f643a5d90 0x559f643a81a0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:52.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.687+0000 7fb709ae7b80 1 --2- 172.21.15.39:0/187876905 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f64392550 0x559f64395fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.687+0000 7fb709ae7b80 1 --2- 172.21.15.39:0/187876905 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f641e66b0 0x559f64398740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.687+0000 7fb709ae7b80 1 --2- 172.21.15.39:0/187876905 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f6438c790 0x559f64395a60 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.687+0000 7fb6d6ffd700 0 client.15924 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:42:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.687+0000 7fb709ae7b80 1 -- 172.21.15.39:0/187876905 >> 172.21.15.39:0/187876905 conn(0x559f6437fbe0 msgr2=0x559f64383c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.687+0000 7fb709ae7b80 1 -- 172.21.15.39:0/187876905 shutdown_connections 2023-07-21T18:42:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.687+0000 7fb709ae7b80 1 -- 172.21.15.39:0/187876905 wait complete. 2023-07-21T18:42:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.687+0000 7fb709ae7b80 20 client.15924 trim_cache size 0 max 16384 2023-07-21T18:42:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.702+0000 7fb709ae7b80 1 Processor -- start 2023-07-21T18:42:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.702+0000 7fb709ae7b80 1 -- start start 2023-07-21T18:42:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.702+0000 7fb709ae7b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f641e66b0 0x559f6439cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.702+0000 7fb709ae7b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f6438c790 0x559f64384c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.702+0000 7fb709ae7b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f64392550 0x559f64398200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.702+0000 7fb709ae7b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f64381d00 con 0x559f64392550 2023-07-21T18:42:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.702+0000 7fb709ae7b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f6437cbb0 con 0x559f641e66b0 2023-07-21T18:42:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.702+0000 7fb709ae7b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f64385fb0 con 0x559f6438c790 2023-07-21T18:42:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.702+0000 7fb6f77fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f64392550 0x559f64398200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.702+0000 7fb6f77fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f64392550 0x559f64398200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58394/0 (socket says 172.21.15.39:58394) 2023-07-21T18:42:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.702+0000 7fb6f6ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f641e66b0 0x559f6439cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.702+0000 7fb6f77fe700 1 -- 172.21.15.39:0/1197230808 learned_addr learned my addr 172.21.15.39:0/1197230808 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.702+0000 7fb6f67fc700 1 --2- 172.21.15.39:0/1197230808 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f6438c790 0x559f64384c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.702+0000 7fb6f77fe700 1 -- 172.21.15.39:0/1197230808 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f6438c790 msgr2=0x559f64384c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.702+0000 7fb6f77fe700 1 --2- 172.21.15.39:0/1197230808 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f6438c790 0x559f64384c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.702+0000 7fb6f77fe700 1 -- 172.21.15.39:0/1197230808 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f641e66b0 msgr2=0x559f6439cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.702+0000 7fb6f77fe700 1 --2- 172.21.15.39:0/1197230808 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f641e66b0 0x559f6439cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.702+0000 7fb6f77fe700 1 -- 172.21.15.39:0/1197230808 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f640f2c00 con 0x559f64392550 2023-07-21T18:42:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.703+0000 7fb6f77fe700 1 --2- 172.21.15.39:0/1197230808 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f64392550 0x559f64398200 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fb6f0011c70 tx=0x7fb6f0009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50780698ae134ab8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.703+0000 7fb6de7fc700 1 -- 172.21.15.39:0/1197230808 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f000e040 con 0x559f64392550 2023-07-21T18:42:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.703+0000 7fb6de7fc700 1 -- 172.21.15.39:0/1197230808 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6f000f040 con 0x559f64392550 2023-07-21T18:42:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.703+0000 7fb6de7fc700 1 -- 172.21.15.39:0/1197230808 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f0004050 con 0x559f64392550 2023-07-21T18:42:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.703+0000 7fb709ae7b80 1 -- 172.21.15.39:0/1197230808 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f64392550 msgr2=0x559f64398200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.703+0000 7fb709ae7b80 1 --2- 172.21.15.39:0/1197230808 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f64392550 0x559f64398200 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fb6f0011c70 tx=0x7fb6f0009920 comp rx=0 tx=0).stop 2023-07-21T18:42:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.703+0000 7fb709ae7b80 1 -- 172.21.15.39:0/1197230808 shutdown_connections 2023-07-21T18:42:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.703+0000 7fb709ae7b80 1 --2- 172.21.15.39:0/1197230808 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f641e66b0 0x559f6439cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.703+0000 7fb709ae7b80 1 --2- 172.21.15.39:0/1197230808 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f64392550 0x559f64398200 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.703+0000 7fb709ae7b80 1 --2- 172.21.15.39:0/1197230808 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f6438c790 0x559f64384c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.703+0000 7fb709ae7b80 1 -- 172.21.15.39:0/1197230808 >> 172.21.15.39:0/1197230808 conn(0x559f6437fbe0 msgr2=0x559f6438cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.703+0000 7fb709ae7b80 1 -- 172.21.15.39:0/1197230808 shutdown_connections 2023-07-21T18:42:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.703+0000 7fb709ae7b80 1 -- 172.21.15.39:0/1197230808 wait complete. 2023-07-21T18:42:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.704+0000 7fb709ae7b80 1 -- start start 2023-07-21T18:42:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.704+0000 7fb709ae7b80 1 Processor -- start 2023-07-21T18:42:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.704+0000 7fb709ae7b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f641e66b0 0x559f6438ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:52.730 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.704+0000 7fb709ae7b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f6438c790 0x559f6438f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:52.730 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.704+0000 7fb709ae7b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f64392550 0x559f6438d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:52.730 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.704+0000 7fb709ae7b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f6438df30 con 0x559f64392550 2023-07-21T18:42:52.730 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.704+0000 7fb709ae7b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f6438e0a0 con 0x559f641e66b0 2023-07-21T18:42:52.730 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.704+0000 7fb709ae7b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f6438e260 con 0x559f6438c790 2023-07-21T18:42:52.730 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.704+0000 7fb709ae7b80 10 client.0 _ll_register_callbacks cb 0x7fb6e800f830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:42:52.730 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.704+0000 7fb6f6ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f641e66b0 0x559f6438ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:52.730 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.704+0000 7fb6f77fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f64392550 0x559f6438d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.704+0000 7fb6f77fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f64392550 0x559f6438d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58400/0 (socket says 172.21.15.39:58400) 2023-07-21T18:42:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.704+0000 7fb6f6ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f641e66b0 0x559f6438ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:42760/0 (socket says 172.21.15.39:42760) 2023-07-21T18:42:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.704+0000 7fb6f77fe700 1 -- 172.21.15.39:0/2477861126 learned_addr learned my addr 172.21.15.39:0/2477861126 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:42:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.705+0000 7fb6f67fc700 1 --2- 172.21.15.39:0/2477861126 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f6438c790 0x559f6438f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.705+0000 7fb6f77fe700 1 -- 172.21.15.39:0/2477861126 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f6438c790 msgr2=0x559f6438f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.705+0000 7fb6f77fe700 1 --2- 172.21.15.39:0/2477861126 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f6438c790 0x559f6438f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.705+0000 7fb6f77fe700 1 -- 172.21.15.39:0/2477861126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f641e66b0 msgr2=0x559f6438ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:42:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.705+0000 7fb6f77fe700 1 --2- 172.21.15.39:0/2477861126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f641e66b0 0x559f6438ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.705+0000 7fb6f77fe700 1 -- 172.21.15.39:0/2477861126 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6f00118c0 con 0x559f64392550 2023-07-21T18:42:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.705+0000 7fb6f6ffd700 1 --2- 172.21.15.39:0/2477861126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f641e66b0 0x559f6438ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:42:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.705+0000 7fb6f77fe700 1 --2- 172.21.15.39:0/2477861126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f64392550 0x559f6438d9c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fb6f000b010 tx=0x7fb6f0004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2a24a4b2ef09f1a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:42:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.705+0000 7fb6d6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:42:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.705+0000 7fb6d6ffd700 1 -- 172.21.15.39:0/2477861126 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f0023070 con 0x559f64392550 2023-07-21T18:42:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.705+0000 7fb6d6ffd700 1 -- 172.21.15.39:0/2477861126 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fb6f00093a0 con 0x559f64392550 2023-07-21T18:42:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.705+0000 7fb709ae7b80 1 -- 172.21.15.39:0/2477861126 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb6ec0a23e0 con 0x559f64392550 2023-07-21T18:42:52.733 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.705+0000 7fb6d6ffd700 1 -- 172.21.15.39:0/2477861126 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb6f000e050 con 0x559f64392550 2023-07-21T18:42:52.733 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.705+0000 7fb709ae7b80 10 client.15930 fetch_fsmap learned FSMap version 26 2023-07-21T18:42:52.733 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.705+0000 7fb709ae7b80 1 -- 172.21.15.39:0/2477861126 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x559f641301f0 con 0x559f64392550 2023-07-21T18:42:52.733 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.706+0000 7fb6d6ffd700 1 -- 172.21.15.39:0/2477861126 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6ec0a23e0 con 0x559f64392550 2023-07-21T18:42:52.733 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.706+0000 7fb6d6ffd700 1 -- 172.21.15.39:0/2477861126 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb6f002a4c0 con 0x559f64392550 2023-07-21T18:42:52.733 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.706+0000 7fb709ae7b80 10 client.15930 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T18:42:52.733 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.706+0000 7fb709ae7b80 10 client.15930 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:42:52.733 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.706+0000 7fb709ae7b80 20 client.15930 populate_metadata read hostname 'smithi039' 2023-07-21T18:42:52.734 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.706+0000 7fb709ae7b80 1 --2- 172.21.15.39:0/2477861126 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x559f643faa10 0x559f643fce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:42:52.734 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.706+0000 7fb709ae7b80 4 client.15930 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:42:52.734 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.706+0000 7fb709ae7b80 1 -- 172.21.15.39:0/2477861126 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559f64296ff0 con 0x559f643faa10 2023-07-21T18:42:52.734 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.706+0000 7fb6f6ffd700 1 --2- 172.21.15.39:0/2477861126 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x559f643faa10 0x559f643fce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:42:52.734 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.706+0000 7fb6f6ffd700 1 --2- 172.21.15.39:0/2477861126 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x559f643faa10 0x559f643fce20 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4138a265d57a2b56 server_cookie=1bfb71b9c1e16c1f in_seq=0 out_seq=0 2023-07-21T18:42:52.734 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.706+0000 7fb6d6ffd700 10 client.15930 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:42:52.813 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.792+0000 7fb6d6ffd700 1 -- 172.21.15.39:0/2477861126 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559f64296ff0 con 0x559f643faa10 2023-07-21T18:42:52.813 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.792+0000 7fb6d6ffd700 10 client.15930 handle_command_reply: tid=0 2023-07-21T18:42:52.813 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.792+0000 7fb709ae7b80 1 client.15930 shutdown 2023-07-21T18:42:52.813 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.792+0000 7fb709ae7b80 1 -- 172.21.15.39:0/2477861126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f64392550 msgr2=0x559f6438d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:52.813 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.792+0000 7fb709ae7b80 1 --2- 172.21.15.39:0/2477861126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f64392550 0x559f6438d9c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fb6f000b010 tx=0x7fb6f0004f90 comp rx=0 tx=0).stop 2023-07-21T18:42:52.813 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.792+0000 7fb709ae7b80 1 -- 172.21.15.39:0/2477861126 shutdown_connections 2023-07-21T18:42:52.814 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.792+0000 7fb709ae7b80 1 --2- 172.21.15.39:0/2477861126 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x559f643faa10 0x559f643fce20 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:52.814 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.792+0000 7fb709ae7b80 1 --2- 172.21.15.39:0/2477861126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f641e66b0 0x559f6438ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:52.814 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.792+0000 7fb6d6ffd700 0 client.15930 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:42:52.814 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.792+0000 7fb709ae7b80 1 --2- 172.21.15.39:0/2477861126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f64392550 0x559f6438d9c0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:52.814 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.792+0000 7fb709ae7b80 1 --2- 172.21.15.39:0/2477861126 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f6438c790 0x559f6438f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:52.814 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.792+0000 7fb709ae7b80 1 -- 172.21.15.39:0/2477861126 >> 172.21.15.39:0/2477861126 conn(0x559f6437fbe0 msgr2=0x559f64395a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:52.814 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.792+0000 7fb709ae7b80 1 -- 172.21.15.39:0/2477861126 shutdown_connections 2023-07-21T18:42:52.814 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.792+0000 7fb709ae7b80 1 -- 172.21.15.39:0/2477861126 wait complete. 2023-07-21T18:42:52.815 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.792+0000 7fb709ae7b80 20 client.15930 trim_cache size 0 max 16384 2023-07-21T18:42:52.815 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:42:52.815 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:42:52.815 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:42:52.815 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:42:52.816 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.793+0000 7fb6fd961700 1 -- 172.21.15.39:0/1687915620 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fb6e00a13e0 msgr2=0x7fb6e00a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:52.816 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.793+0000 7fb6fd961700 1 --2- 172.21.15.39:0/1687915620 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fb6e00a13e0 0x7fb6e00a3890 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb6f80fc910 tx=0x7fb6e8005fd0 comp rx=0 tx=0).stop 2023-07-21T18:42:52.816 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.793+0000 7fb6fd961700 1 -- 172.21.15.39:0/1687915620 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6f80fea80 msgr2=0x7fb6f819aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:42:52.816 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.793+0000 7fb6fd961700 1 --2- 172.21.15.39:0/1687915620 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6f80fea80 0x7fb6f819aac0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec00e970 tx=0x7fb6ec00ec80 comp rx=0 tx=0).stop 2023-07-21T18:42:52.817 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.793+0000 7fb6fd961700 1 -- 172.21.15.39:0/1687915620 shutdown_connections 2023-07-21T18:42:52.817 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.793+0000 7fb6fd961700 1 --2- 172.21.15.39:0/1687915620 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb6f8104a90 0x7fb6f8194920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:52.817 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.793+0000 7fb6fd961700 1 --2- 172.21.15.39:0/1687915620 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6f80fea80 0x7fb6f819aac0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:52.817 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.793+0000 7fb6fd961700 1 --2- 172.21.15.39:0/1687915620 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fb6e00a13e0 0x7fb6e00a3890 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:52.817 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.793+0000 7fb6fd961700 1 --2- 172.21.15.39:0/1687915620 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6f80ff430 0x7fb6f819b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:42:52.817 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.793+0000 7fb6fd961700 1 -- 172.21.15.39:0/1687915620 >> 172.21.15.39:0/1687915620 conn(0x7fb6f80769b0 msgr2=0x7fb6f8102720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:42:52.817 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.793+0000 7fb6fd961700 1 -- 172.21.15.39:0/1687915620 shutdown_connections 2023-07-21T18:42:52.818 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:42:52.793+0000 7fb6fd961700 1 -- 172.21.15.39:0/1687915620 wait complete. 2023-07-21T18:42:53.258 INFO:tasks.workunit.client.0.smithi039.stdout: 1 15454 16.60 MB/sec warmup 6 sec latency 33.167 ms 2023-07-21T18:42:54.259 INFO:tasks.workunit.client.0.smithi039.stdout: 1 18217 16.65 MB/sec warmup 7 sec latency 12.536 ms 2023-07-21T18:42:54.436 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:42:54 smithi039 ceph-mon[96489]: pgmap v37: 97 pgs: 97 active+clean; 5.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 391 KiB/s wr, 57 op/s 2023-07-21T18:42:54.618 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:42:54 smithi193 ceph-mon[98204]: pgmap v37: 97 pgs: 97 active+clean; 5.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 391 KiB/s wr, 57 op/s 2023-07-21T18:42:54.637 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:42:54 smithi155 ceph-mon[97934]: pgmap v37: 97 pgs: 97 active+clean; 5.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 391 KiB/s wr, 57 op/s 2023-07-21T18:42:55.259 INFO:tasks.workunit.client.0.smithi039.stdout: 1 18466 14.60 MB/sec warmup 8 sec latency 752.004 ms 2023-07-21T18:42:55.436 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:42:55 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:42:55.618 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:42:55 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:42:55.637 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:42:55 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:42:56.259 INFO:tasks.workunit.client.0.smithi039.stdout: 1 18466 12.98 MB/sec warmup 9 sec latency 1752.098 ms 2023-07-21T18:42:56.436 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:42:56 smithi039 ceph-mon[96489]: pgmap v38: 97 pgs: 97 active+clean; 5.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 391 KiB/s wr, 57 op/s 2023-07-21T18:42:56.618 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:42:56 smithi193 ceph-mon[98204]: pgmap v38: 97 pgs: 97 active+clean; 5.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 391 KiB/s wr, 57 op/s 2023-07-21T18:42:56.637 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:42:56 smithi155 ceph-mon[97934]: pgmap v38: 97 pgs: 97 active+clean; 5.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 391 KiB/s wr, 57 op/s 2023-07-21T18:42:57.259 INFO:tasks.workunit.client.0.smithi039.stdout: 1 18466 11.68 MB/sec warmup 10 sec latency 2752.243 ms 2023-07-21T18:42:58.259 INFO:tasks.workunit.client.0.smithi039.stdout: 1 18466 10.62 MB/sec warmup 11 sec latency 3752.359 ms 2023-07-21T18:42:58.436 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:42:58 smithi039 ceph-mon[96489]: pgmap v39: 97 pgs: 97 active+clean; 10 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 933 KiB/s wr, 204 op/s 2023-07-21T18:42:58.618 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:42:58 smithi193 ceph-mon[98204]: pgmap v39: 97 pgs: 97 active+clean; 10 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 933 KiB/s wr, 204 op/s 2023-07-21T18:42:58.637 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:42:58 smithi155 ceph-mon[97934]: pgmap v39: 97 pgs: 97 active+clean; 10 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 933 KiB/s wr, 204 op/s 2023-07-21T18:42:59.259 INFO:tasks.workunit.client.0.smithi039.stdout: 1 18533 9.75 MB/sec warmup 12 sec latency 4702.798 ms 2023-07-21T18:43:00.259 INFO:tasks.workunit.client.0.smithi039.stdout: 1 18722 9.03 MB/sec warmup 13 sec latency 636.314 ms 2023-07-21T18:43:00.436 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:00 smithi039 ceph-mon[96489]: pgmap v40: 97 pgs: 97 active+clean; 10 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 933 KiB/s wr, 199 op/s 2023-07-21T18:43:00.436 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:00 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:43:00.618 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:00 smithi193 ceph-mon[98204]: pgmap v40: 97 pgs: 97 active+clean; 10 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 933 KiB/s wr, 199 op/s 2023-07-21T18:43:00.618 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:00 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:43:00.636 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:00 smithi155 ceph-mon[97934]: pgmap v40: 97 pgs: 97 active+clean; 10 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 933 KiB/s wr, 199 op/s 2023-07-21T18:43:00.637 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:00 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:43:01.259 INFO:tasks.workunit.client.0.smithi039.stdout: 1 18851 8.47 MB/sec warmup 14 sec latency 1280.262 ms 2023-07-21T18:43:02.259 INFO:tasks.workunit.client.0.smithi039.stdout: 1 21472 8.96 MB/sec warmup 15 sec latency 56.343 ms 2023-07-21T18:43:02.435 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:02 smithi039 ceph-mon[96489]: pgmap v41: 97 pgs: 97 active+clean; 11 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 1015 KiB/s wr, 213 op/s 2023-07-21T18:43:02.618 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:02 smithi193 ceph-mon[98204]: pgmap v41: 97 pgs: 97 active+clean; 11 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 1015 KiB/s wr, 213 op/s 2023-07-21T18:43:02.636 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:02 smithi155 ceph-mon[97934]: pgmap v41: 97 pgs: 97 active+clean; 11 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 1015 KiB/s wr, 213 op/s 2023-07-21T18:43:03.259 INFO:tasks.workunit.client.0.smithi039.stdout: 1 22238 8.53 MB/sec warmup 16 sec latency 287.667 ms 2023-07-21T18:43:04.259 INFO:tasks.workunit.client.0.smithi039.stdout: 1 22238 8.03 MB/sec warmup 17 sec latency 1287.763 ms 2023-07-21T18:43:04.435 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:04 smithi039 ceph-mon[96489]: pgmap v42: 97 pgs: 97 active+clean; 11 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 624 KiB/s wr, 160 op/s 2023-07-21T18:43:04.618 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:04 smithi193 ceph-mon[98204]: pgmap v42: 97 pgs: 97 active+clean; 11 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 624 KiB/s wr, 160 op/s 2023-07-21T18:43:04.636 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:04 smithi155 ceph-mon[97934]: pgmap v42: 97 pgs: 97 active+clean; 11 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 624 KiB/s wr, 160 op/s 2023-07-21T18:43:05.259 INFO:tasks.workunit.client.0.smithi039.stdout: 1 22393 7.65 MB/sec warmup 18 sec latency 1535.511 ms 2023-07-21T18:43:05.937 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.917+0000 7fd7097e6700 1 -- 172.21.15.39:0/3753826976 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd704103f10 msgr2=0x7fd70410bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:05.938 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.917+0000 7fd7097e6700 1 --2- 172.21.15.39:0/3753826976 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd704103f10 0x7fd70410bae0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc00b0d0 tx=0x7fd6fc00b3e0 comp rx=0 tx=0).stop 2023-07-21T18:43:05.938 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.917+0000 7fd7097e6700 1 -- 172.21.15.39:0/3753826976 shutdown_connections 2023-07-21T18:43:05.938 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.917+0000 7fd7097e6700 1 --2- 172.21.15.39:0/3753826976 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd704103f10 0x7fd70410bae0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:05.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.917+0000 7fd7097e6700 1 --2- 172.21.15.39:0/3753826976 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd7040fedc0 0x7fd7040ff190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:05.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.917+0000 7fd7097e6700 1 --2- 172.21.15.39:0/3753826976 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7040ff760 0x7fd704103830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:05.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.917+0000 7fd7097e6700 1 -- 172.21.15.39:0/3753826976 >> 172.21.15.39:0/3753826976 conn(0x7fd7040759e0 msgr2=0x7fd704075de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:05.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.917+0000 7fd7097e6700 1 -- 172.21.15.39:0/3753826976 shutdown_connections 2023-07-21T18:43:05.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.917+0000 7fd7097e6700 1 -- 172.21.15.39:0/3753826976 wait complete. 2023-07-21T18:43:05.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.918+0000 7fd7097e6700 1 Processor -- start 2023-07-21T18:43:05.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.918+0000 7fd7097e6700 1 -- start start 2023-07-21T18:43:05.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.918+0000 7fd7097e6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7040fedc0 0x7fd704195f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:05.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.918+0000 7fd7097e6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7040ff760 0x7fd704196460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:05.942 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.918+0000 7fd7097e6700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd704103f10 0x7fd70419aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:05.942 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.918+0000 7fd7097e6700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd70419b110 con 0x7fd7040ff760 2023-07-21T18:43:05.942 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.918+0000 7fd7097e6700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd70419b280 con 0x7fd7040fedc0 2023-07-21T18:43:05.942 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.918+0000 7fd7097e6700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd70419b440 con 0x7fd704103f10 2023-07-21T18:43:05.942 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.918+0000 7fd6fbfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7040ff760 0x7fd704196460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:05.942 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.918+0000 7fd6fbfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7040ff760 0x7fd704196460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58420/0 (socket says 172.21.15.39:58420) 2023-07-21T18:43:05.942 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.918+0000 7fd6fbfff700 1 -- 172.21.15.39:0/2141957303 learned_addr learned my addr 172.21.15.39:0/2141957303 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.918+0000 7fd702ffd700 1 --2- 172.21.15.39:0/2141957303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7040fedc0 0x7fd704195f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.918+0000 7fd7037fe700 1 --2- 172.21.15.39:0/2141957303 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd704103f10 0x7fd70419aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.919+0000 7fd6fbfff700 1 -- 172.21.15.39:0/2141957303 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd704103f10 msgr2=0x7fd70419aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.919+0000 7fd6fbfff700 1 --2- 172.21.15.39:0/2141957303 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd704103f10 0x7fd70419aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.919+0000 7fd6fbfff700 1 -- 172.21.15.39:0/2141957303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7040fedc0 msgr2=0x7fd704195f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.919+0000 7fd6fbfff700 1 --2- 172.21.15.39:0/2141957303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7040fedc0 0x7fd704195f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.919+0000 7fd6fbfff700 1 -- 172.21.15.39:0/2141957303 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6fc009d00 con 0x7fd7040ff760 2023-07-21T18:43:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.919+0000 7fd702ffd700 1 --2- 172.21.15.39:0/2141957303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7040fedc0 0x7fd704195f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:43:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.919+0000 7fd7037fe700 1 --2- 172.21.15.39:0/2141957303 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd704103f10 0x7fd70419aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:43:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.919+0000 7fd6fbfff700 1 --2- 172.21.15.39:0/2141957303 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7040ff760 0x7fd704196460 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec00d950 tx=0x7fd6ec00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.919+0000 7fd700ff9700 1 -- 172.21.15.39:0/2141957303 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6ec00bb60 con 0x7fd7040ff760 2023-07-21T18:43:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.919+0000 7fd700ff9700 1 -- 172.21.15.39:0/2141957303 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd6ec00de30 con 0x7fd7040ff760 2023-07-21T18:43:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.919+0000 7fd7097e6700 1 -- 172.21.15.39:0/2141957303 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd70419b720 con 0x7fd7040ff760 2023-07-21T18:43:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.919+0000 7fd700ff9700 1 -- 172.21.15.39:0/2141957303 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6ec0054d0 con 0x7fd7040ff760 2023-07-21T18:43:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.919+0000 7fd7097e6700 1 -- 172.21.15.39:0/2141957303 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7041a2f60 con 0x7fd7040ff760 2023-07-21T18:43:05.945 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.921+0000 7fd700ff9700 1 -- 172.21.15.39:0/2141957303 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7fd6ec004750 con 0x7fd7040ff760 2023-07-21T18:43:05.948 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.922+0000 7fd700ff9700 1 --2- 172.21.15.39:0/2141957303 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fd6f00a1460 0x7fd6f00a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:05.949 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.922+0000 7fd702ffd700 1 --2- 172.21.15.39:0/2141957303 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fd6f00a1460 0x7fd6f00a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:05.949 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.922+0000 7fd700ff9700 1 -- 172.21.15.39:0/2141957303 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fd6ec0cc790 con 0x7fd7040ff760 2023-07-21T18:43:05.949 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.922+0000 7fd702ffd700 1 --2- 172.21.15.39:0/2141957303 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fd6f00a1460 0x7fd6f00a3910 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd6f4005950 tx=0x7fd6f40058e0 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:05.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.923+0000 7fd71596cb80 1 Processor -- start 2023-07-21T18:43:05.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.923+0000 7fd71596cb80 1 -- start start 2023-07-21T18:43:05.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.923+0000 7fd71596cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ed83a06b0 0x562ed8549d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:05.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.923+0000 7fd71596cb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ed854a2d0 0x562ed8544040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:05.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.923+0000 7fd71596cb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ed8544580 0x562ed85449d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:05.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.923+0000 7fd71596cb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ed8449c50 con 0x562ed8544580 2023-07-21T18:43:05.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.923+0000 7fd71596cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ed849c9e0 con 0x562ed83a06b0 2023-07-21T18:43:05.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.923+0000 7fd71596cb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ed8495e70 con 0x562ed854a2d0 2023-07-21T18:43:05.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.923+0000 7fd7037fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ed8544580 0x562ed85449d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:05.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.923+0000 7fd7037fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ed8544580 0x562ed85449d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58430/0 (socket says 172.21.15.39:58430) 2023-07-21T18:43:05.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.923+0000 7fd7037fe700 1 -- 172.21.15.39:0/455266857 learned_addr learned my addr 172.21.15.39:0/455266857 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:05.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.924+0000 7fd6fbfff700 1 --2- 172.21.15.39:0/455266857 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ed854a2d0 0x562ed8544040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:05.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.924+0000 7fd702ffd700 1 --2- 172.21.15.39:0/455266857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ed83a06b0 0x562ed8549d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:05.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.924+0000 7fd7037fe700 1 -- 172.21.15.39:0/455266857 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ed854a2d0 msgr2=0x562ed8544040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:05.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.924+0000 7fd7037fe700 1 --2- 172.21.15.39:0/455266857 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ed854a2d0 0x562ed8544040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:05.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.924+0000 7fd7037fe700 1 -- 172.21.15.39:0/455266857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ed83a06b0 msgr2=0x562ed8549d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:05.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.924+0000 7fd7037fe700 1 --2- 172.21.15.39:0/455266857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ed83a06b0 0x562ed8549d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:05.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.924+0000 7fd7037fe700 1 -- 172.21.15.39:0/455266857 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ed82c3d80 con 0x562ed8544580 2023-07-21T18:43:05.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.924+0000 7fd702ffd700 1 --2- 172.21.15.39:0/455266857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ed83a06b0 0x562ed8549d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:43:05.953 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.924+0000 7fd7037fe700 1 --2- 172.21.15.39:0/455266857 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ed8544580 0x562ed85449d0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc00cda0 tx=0x7fd6fc00cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f72f54962bfb46b1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:05.953 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.924+0000 7fd6f97fa700 1 -- 172.21.15.39:0/455266857 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc00e070 con 0x562ed8544580 2023-07-21T18:43:05.953 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.924+0000 7fd6f97fa700 1 -- 172.21.15.39:0/455266857 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd6fc004a50 con 0x562ed8544580 2023-07-21T18:43:05.953 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.924+0000 7fd6f97fa700 1 -- 172.21.15.39:0/455266857 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc005280 con 0x562ed8544580 2023-07-21T18:43:05.953 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.925+0000 7fd71596cb80 1 -- 172.21.15.39:0/455266857 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ed8544580 msgr2=0x562ed85449d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:05.953 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.925+0000 7fd71596cb80 1 --2- 172.21.15.39:0/455266857 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ed8544580 0x562ed85449d0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc00cda0 tx=0x7fd6fc00cdd0 comp rx=0 tx=0).stop 2023-07-21T18:43:05.953 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.925+0000 7fd71596cb80 1 -- 172.21.15.39:0/455266857 shutdown_connections 2023-07-21T18:43:05.953 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.925+0000 7fd71596cb80 1 --2- 172.21.15.39:0/455266857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ed83a06b0 0x562ed8549d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:05.954 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.925+0000 7fd71596cb80 1 --2- 172.21.15.39:0/455266857 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ed8544580 0x562ed85449d0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:05.954 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.925+0000 7fd71596cb80 1 --2- 172.21.15.39:0/455266857 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ed854a2d0 0x562ed8544040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:05.954 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.925+0000 7fd71596cb80 1 -- 172.21.15.39:0/455266857 >> 172.21.15.39:0/455266857 conn(0x562ed8539be0 msgr2=0x562ed853ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:05.954 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.925+0000 7fd71596cb80 1 -- 172.21.15.39:0/455266857 shutdown_connections 2023-07-21T18:43:05.954 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.925+0000 7fd71596cb80 1 -- 172.21.15.39:0/455266857 wait complete. 2023-07-21T18:43:05.954 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.925+0000 7fd71596cb80 1 -- start start 2023-07-21T18:43:05.954 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.925+0000 7fd71596cb80 1 Processor -- start 2023-07-21T18:43:05.954 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.926+0000 7fd71596cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ed83a06b0 0x562ed8550480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:05.955 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.926+0000 7fd71596cb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ed8544580 0x562ed854d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:05.955 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.926+0000 7fd71596cb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ed854a2d0 0x562ed854dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:05.955 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.926+0000 7fd71596cb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ed8435a60 con 0x562ed854a2d0 2023-07-21T18:43:05.955 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.926+0000 7fd71596cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ed8435760 con 0x562ed83a06b0 2023-07-21T18:43:05.955 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.926+0000 7fd71596cb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ed83198c0 con 0x562ed8544580 2023-07-21T18:43:05.955 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.926+0000 7fd71596cb80 10 client.0 _ll_register_callbacks cb 0x562ed8433700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:43:05.955 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.926+0000 7fd7037fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ed854a2d0 0x562ed854dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:05.956 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.926+0000 7fd7037fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ed854a2d0 0x562ed854dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58436/0 (socket says 172.21.15.39:58436) 2023-07-21T18:43:05.956 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.926+0000 7fd7037fe700 1 -- 172.21.15.39:0/1639038996 learned_addr learned my addr 172.21.15.39:0/1639038996 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:05.956 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.926+0000 7fd6fbfff700 1 --2- 172.21.15.39:0/1639038996 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ed8544580 0x562ed854d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:05.956 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.926+0000 7fd702ffd700 1 --2- 172.21.15.39:0/1639038996 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ed83a06b0 0x562ed8550480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:05.956 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.926+0000 7fd7037fe700 1 -- 172.21.15.39:0/1639038996 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ed8544580 msgr2=0x562ed854d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:05.956 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.926+0000 7fd7037fe700 1 --2- 172.21.15.39:0/1639038996 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ed8544580 0x562ed854d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:05.956 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.926+0000 7fd7037fe700 1 -- 172.21.15.39:0/1639038996 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ed83a06b0 msgr2=0x562ed8550480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:05.957 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.926+0000 7fd7037fe700 1 --2- 172.21.15.39:0/1639038996 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ed83a06b0 0x562ed8550480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:05.957 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.926+0000 7fd7037fe700 1 -- 172.21.15.39:0/1639038996 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd70410c320 con 0x562ed854a2d0 2023-07-21T18:43:05.957 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.927+0000 7fd702ffd700 1 --2- 172.21.15.39:0/1639038996 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ed83a06b0 0x562ed8550480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:43:05.957 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.927+0000 7fd7037fe700 1 --2- 172.21.15.39:0/1639038996 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ed854a2d0 0x562ed854dd00 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc00cd70 tx=0x7fd6fc009380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cce891823db9a32 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:05.957 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.927+0000 7fd6ea7fc700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:43:05.957 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.927+0000 7fd6ea7fc700 1 -- 172.21.15.39:0/1639038996 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc00e040 con 0x562ed854a2d0 2023-07-21T18:43:05.957 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.927+0000 7fd6ea7fc700 1 -- 172.21.15.39:0/1639038996 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fd6fc012680 con 0x562ed854a2d0 2023-07-21T18:43:05.957 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.927+0000 7fd71596cb80 1 -- 172.21.15.39:0/1639038996 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562ed83ef490 con 0x562ed854a2d0 2023-07-21T18:43:05.958 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.927+0000 7fd6ea7fc700 1 -- 172.21.15.39:0/1639038996 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd6fc012840 con 0x562ed854a2d0 2023-07-21T18:43:05.958 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.927+0000 7fd71596cb80 10 client.15942 fetch_fsmap learned FSMap version 28 2023-07-21T18:43:05.958 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.927+0000 7fd71596cb80 1 -- 172.21.15.39:0/1639038996 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x562ed82acc00 con 0x562ed854a2d0 2023-07-21T18:43:05.958 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.927+0000 7fd6ea7fc700 1 -- 172.21.15.39:0/1639038996 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562ed83ef490 con 0x562ed854a2d0 2023-07-21T18:43:05.958 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.927+0000 7fd6ea7fc700 1 -- 172.21.15.39:0/1639038996 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fd6fc00b100 con 0x562ed854a2d0 2023-07-21T18:43:05.958 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.927+0000 7fd71596cb80 10 client.15942 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T18:43:05.958 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.927+0000 7fd71596cb80 10 client.15942 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:43:05.958 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.927+0000 7fd71596cb80 20 client.15942 populate_metadata read hostname 'smithi039' 2023-07-21T18:43:05.959 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.928+0000 7fd71596cb80 1 --2- 172.21.15.39:0/1639038996 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x562ed855d8a0 0x562ed855fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:05.959 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.928+0000 7fd71596cb80 4 client.15942 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:43:05.959 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.928+0000 7fd71596cb80 1 -- 172.21.15.39:0/1639038996 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562ed848ee50 con 0x562ed855d8a0 2023-07-21T18:43:05.959 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.928+0000 7fd702ffd700 1 --2- 172.21.15.39:0/1639038996 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x562ed855d8a0 0x562ed855fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:05.959 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.928+0000 7fd6ea7fc700 10 client.15942 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:43:05.959 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.928+0000 7fd702ffd700 1 --2- 172.21.15.39:0/1639038996 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x562ed855d8a0 0x562ed855fcb0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cec0fd1c8567f07 server_cookie=7a02f1a4c6830e18 in_seq=0 out_seq=0 2023-07-21T18:43:05.960 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.929+0000 7fd6ea7fc700 1 -- 172.21.15.39:0/1639038996 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562ed848ee50 con 0x562ed855d8a0 2023-07-21T18:43:05.960 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.929+0000 7fd6ea7fc700 10 client.15942 handle_command_reply: tid=0 2023-07-21T18:43:05.961 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.929+0000 7fd71596cb80 1 client.15942 shutdown 2023-07-21T18:43:05.961 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.929+0000 7fd71596cb80 1 -- 172.21.15.39:0/1639038996 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ed854a2d0 msgr2=0x562ed854dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:05.961 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.929+0000 7fd71596cb80 1 --2- 172.21.15.39:0/1639038996 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ed854a2d0 0x562ed854dd00 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc00cd70 tx=0x7fd6fc009380 comp rx=0 tx=0).stop 2023-07-21T18:43:05.962 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.929+0000 7fd71596cb80 1 -- 172.21.15.39:0/1639038996 shutdown_connections 2023-07-21T18:43:05.962 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.929+0000 7fd71596cb80 1 --2- 172.21.15.39:0/1639038996 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x562ed855d8a0 0x562ed855fcb0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:05.962 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.929+0000 7fd71596cb80 1 --2- 172.21.15.39:0/1639038996 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ed83a06b0 0x562ed8550480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:05.962 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.929+0000 7fd6ea7fc700 0 client.15942 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:43:05.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.929+0000 7fd71596cb80 1 --2- 172.21.15.39:0/1639038996 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ed854a2d0 0x562ed854dd00 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:05.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.929+0000 7fd71596cb80 1 --2- 172.21.15.39:0/1639038996 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ed8544580 0x562ed854d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:05.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.929+0000 7fd71596cb80 1 -- 172.21.15.39:0/1639038996 >> 172.21.15.39:0/1639038996 conn(0x562ed8539be0 msgr2=0x562ed853dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:05.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.929+0000 7fd71596cb80 1 -- 172.21.15.39:0/1639038996 shutdown_connections 2023-07-21T18:43:05.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.929+0000 7fd71596cb80 1 -- 172.21.15.39:0/1639038996 wait complete. 2023-07-21T18:43:05.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.930+0000 7fd71596cb80 20 client.15942 trim_cache size 0 max 16384 2023-07-21T18:43:05.971 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.944+0000 7fd71596cb80 1 Processor -- start 2023-07-21T18:43:05.971 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.944+0000 7fd71596cb80 1 -- start start 2023-07-21T18:43:05.971 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.945+0000 7fd71596cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ed83a06b0 0x562ed854b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:05.972 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.945+0000 7fd71596cb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ed8544580 0x562ed853ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:05.972 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.945+0000 7fd71596cb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ed854a2d0 0x562ed854ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:05.972 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.945+0000 7fd71596cb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ed854a6a0 con 0x562ed8544580 2023-07-21T18:43:05.972 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.945+0000 7fd71596cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ed853bd00 con 0x562ed83a06b0 2023-07-21T18:43:05.973 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.945+0000 7fd71596cb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ed85406d0 con 0x562ed854a2d0 2023-07-21T18:43:05.973 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.945+0000 7fd6fbfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ed8544580 0x562ed853ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:05.973 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.945+0000 7fd6fbfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ed8544580 0x562ed853ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58442/0 (socket says 172.21.15.39:58442) 2023-07-21T18:43:05.973 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.945+0000 7fd6fbfff700 1 -- 172.21.15.39:0/325543779 learned_addr learned my addr 172.21.15.39:0/325543779 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.945+0000 7fd7037fe700 1 --2- 172.21.15.39:0/325543779 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ed854a2d0 0x562ed854ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.945+0000 7fd702ffd700 1 --2- 172.21.15.39:0/325543779 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ed83a06b0 0x562ed854b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.945+0000 7fd6fbfff700 1 -- 172.21.15.39:0/325543779 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ed854a2d0 msgr2=0x562ed854ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.945+0000 7fd6fbfff700 1 --2- 172.21.15.39:0/325543779 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ed854a2d0 0x562ed854ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.945+0000 7fd6fbfff700 1 -- 172.21.15.39:0/325543779 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ed83a06b0 msgr2=0x562ed854b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.945+0000 7fd6fbfff700 1 --2- 172.21.15.39:0/325543779 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ed83a06b0 0x562ed854b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.945+0000 7fd6fbfff700 1 -- 172.21.15.39:0/325543779 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ed85648a0 con 0x562ed8544580 2023-07-21T18:43:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.946+0000 7fd6fbfff700 1 --2- 172.21.15.39:0/325543779 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ed8544580 0x562ed853ec60 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec0c9e90 tx=0x7fd6ec0b9040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df6aa60aa7cf15d7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.946+0000 7fd6f9ffb700 1 -- 172.21.15.39:0/325543779 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6ec005630 con 0x562ed8544580 2023-07-21T18:43:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.946+0000 7fd6f9ffb700 1 -- 172.21.15.39:0/325543779 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd6ec099630 con 0x562ed8544580 2023-07-21T18:43:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.946+0000 7fd6f9ffb700 1 -- 172.21.15.39:0/325543779 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6ec098b20 con 0x562ed8544580 2023-07-21T18:43:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.946+0000 7fd71596cb80 1 -- 172.21.15.39:0/325543779 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ed8544580 msgr2=0x562ed853ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.946+0000 7fd71596cb80 1 --2- 172.21.15.39:0/325543779 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ed8544580 0x562ed853ec60 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec0c9e90 tx=0x7fd6ec0b9040 comp rx=0 tx=0).stop 2023-07-21T18:43:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.946+0000 7fd71596cb80 1 -- 172.21.15.39:0/325543779 shutdown_connections 2023-07-21T18:43:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.946+0000 7fd71596cb80 1 --2- 172.21.15.39:0/325543779 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ed83a06b0 0x562ed854b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.946+0000 7fd71596cb80 1 --2- 172.21.15.39:0/325543779 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ed854a2d0 0x562ed854ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.947+0000 7fd71596cb80 1 --2- 172.21.15.39:0/325543779 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ed8544580 0x562ed853ec60 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.947+0000 7fd71596cb80 1 -- 172.21.15.39:0/325543779 >> 172.21.15.39:0/325543779 conn(0x562ed8539be0 msgr2=0x562ed8559ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.947+0000 7fd71596cb80 1 -- 172.21.15.39:0/325543779 shutdown_connections 2023-07-21T18:43:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.947+0000 7fd71596cb80 1 -- 172.21.15.39:0/325543779 wait complete. 2023-07-21T18:43:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.947+0000 7fd71596cb80 1 -- start start 2023-07-21T18:43:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.947+0000 7fd71596cb80 1 Processor -- start 2023-07-21T18:43:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.947+0000 7fd71596cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ed83a06b0 0x562ed85aa8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.947+0000 7fd71596cb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ed8544580 0x562ed85aae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.947+0000 7fd71596cb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ed854a2d0 0x562ed8552c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.947+0000 7fd71596cb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ed8553190 con 0x562ed854a2d0 2023-07-21T18:43:05.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.947+0000 7fd71596cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ed85532d0 con 0x562ed83a06b0 2023-07-21T18:43:05.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.947+0000 7fd71596cb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ed8553490 con 0x562ed8544580 2023-07-21T18:43:05.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.947+0000 7fd71596cb80 10 client.0 _ll_register_callbacks cb 0x7fd6fc009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:43:05.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.948+0000 7fd7037fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ed854a2d0 0x562ed8552c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:05.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.948+0000 7fd7037fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ed854a2d0 0x562ed8552c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58450/0 (socket says 172.21.15.39:58450) 2023-07-21T18:43:05.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.948+0000 7fd7037fe700 1 -- 172.21.15.39:0/1284075673 learned_addr learned my addr 172.21.15.39:0/1284075673 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:05.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.948+0000 7fd702ffd700 1 --2- 172.21.15.39:0/1284075673 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ed83a06b0 0x562ed85aa8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:05.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.948+0000 7fd6fbfff700 1 --2- 172.21.15.39:0/1284075673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ed8544580 0x562ed85aae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:05.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.948+0000 7fd7037fe700 1 -- 172.21.15.39:0/1284075673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ed8544580 msgr2=0x562ed85aae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:05.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.948+0000 7fd7037fe700 1 --2- 172.21.15.39:0/1284075673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ed8544580 0x562ed85aae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:05.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.948+0000 7fd7037fe700 1 -- 172.21.15.39:0/1284075673 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ed83a06b0 msgr2=0x562ed85aa8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:05.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.948+0000 7fd7037fe700 1 --2- 172.21.15.39:0/1284075673 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ed83a06b0 0x562ed85aa8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.948+0000 7fd7037fe700 1 -- 172.21.15.39:0/1284075673 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7041a2f60 con 0x562ed854a2d0 2023-07-21T18:43:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.948+0000 7fd7037fe700 1 --2- 172.21.15.39:0/1284075673 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ed854a2d0 0x562ed8552c50 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc01bc60 tx=0x7fd6fc0099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c645d46f0c0a713 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.948+0000 7fd6ea7fc700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:43:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.948+0000 7fd6ea7fc700 1 -- 172.21.15.39:0/1284075673 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc00e050 con 0x562ed854a2d0 2023-07-21T18:43:05.984 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.948+0000 7fd6ea7fc700 1 -- 172.21.15.39:0/1284075673 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fd6fc01b430 con 0x562ed854a2d0 2023-07-21T18:43:05.984 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.948+0000 7fd71596cb80 1 -- 172.21.15.39:0/1284075673 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd6fc004a50 con 0x562ed854a2d0 2023-07-21T18:43:05.984 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.948+0000 7fd6ea7fc700 1 -- 172.21.15.39:0/1284075673 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd6fc00b2c0 con 0x562ed854a2d0 2023-07-21T18:43:05.984 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.948+0000 7fd71596cb80 10 client.15948 fetch_fsmap learned FSMap version 28 2023-07-21T18:43:05.985 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.948+0000 7fd71596cb80 1 -- 172.21.15.39:0/1284075673 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x562ed82ea1f0 con 0x562ed854a2d0 2023-07-21T18:43:05.985 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.949+0000 7fd6ea7fc700 1 -- 172.21.15.39:0/1284075673 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd6fc004a50 con 0x562ed854a2d0 2023-07-21T18:43:05.985 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.949+0000 7fd6ea7fc700 1 -- 172.21.15.39:0/1284075673 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fd6fc0244e0 con 0x562ed854a2d0 2023-07-21T18:43:05.986 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.949+0000 7fd71596cb80 10 client.15948 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T18:43:05.986 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.949+0000 7fd71596cb80 10 client.15948 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:43:05.986 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.949+0000 7fd71596cb80 20 client.15948 populate_metadata read hostname 'smithi039' 2023-07-21T18:43:05.986 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.949+0000 7fd71596cb80 1 --2- 172.21.15.39:0/1284075673 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x562ed85b2290 0x562ed85b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:05.987 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.949+0000 7fd71596cb80 4 client.15948 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:43:05.987 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.949+0000 7fd71596cb80 1 -- 172.21.15.39:0/1284075673 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562ed8450ff0 con 0x562ed85b2290 2023-07-21T18:43:05.987 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.949+0000 7fd702ffd700 1 --2- 172.21.15.39:0/1284075673 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x562ed85b2290 0x562ed85b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:05.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.949+0000 7fd6ea7fc700 10 client.15948 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:43:05.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.949+0000 7fd702ffd700 1 --2- 172.21.15.39:0/1284075673 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x562ed85b2290 0x562ed85b4670 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84b85e4f35900384 server_cookie=51c92b2f954124b8 in_seq=0 out_seq=0 2023-07-21T18:43:06.009 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.988+0000 7fd6ea7fc700 1 -- 172.21.15.39:0/1284075673 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562ed8450ff0 con 0x562ed85b2290 2023-07-21T18:43:06.009 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.988+0000 7fd6ea7fc700 10 client.15948 handle_command_reply: tid=0 2023-07-21T18:43:06.009 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.988+0000 7fd71596cb80 1 client.15948 shutdown 2023-07-21T18:43:06.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.989+0000 7fd71596cb80 1 -- 172.21.15.39:0/1284075673 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ed854a2d0 msgr2=0x562ed8552c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:06.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.989+0000 7fd71596cb80 1 --2- 172.21.15.39:0/1284075673 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ed854a2d0 0x562ed8552c50 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc01bc60 tx=0x7fd6fc0099e0 comp rx=0 tx=0).stop 2023-07-21T18:43:06.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.989+0000 7fd71596cb80 1 -- 172.21.15.39:0/1284075673 shutdown_connections 2023-07-21T18:43:06.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.989+0000 7fd71596cb80 1 --2- 172.21.15.39:0/1284075673 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x562ed85b2290 0x562ed85b4670 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:06.011 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.989+0000 7fd71596cb80 1 --2- 172.21.15.39:0/1284075673 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ed83a06b0 0x562ed85aa8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:06.011 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.989+0000 7fd71596cb80 1 --2- 172.21.15.39:0/1284075673 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ed854a2d0 0x562ed8552c50 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:06.011 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.989+0000 7fd71596cb80 1 --2- 172.21.15.39:0/1284075673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ed8544580 0x562ed85aae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:06.011 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.989+0000 7fd6ea7fc700 0 client.15948 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:43:06.011 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.989+0000 7fd71596cb80 1 -- 172.21.15.39:0/1284075673 >> 172.21.15.39:0/1284075673 conn(0x562ed8539be0 msgr2=0x562ed8559410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:06.011 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.989+0000 7fd71596cb80 1 -- 172.21.15.39:0/1284075673 shutdown_connections 2023-07-21T18:43:06.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.989+0000 7fd71596cb80 1 -- 172.21.15.39:0/1284075673 wait complete. 2023-07-21T18:43:06.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.989+0000 7fd71596cb80 20 client.15948 trim_cache size 0 max 16384 2023-07-21T18:43:06.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:43:06.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:43:06.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:43:06.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:43:06.013 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.989+0000 7fd7097e6700 1 -- 172.21.15.39:0/2141957303 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fd6f00a1460 msgr2=0x7fd6f00a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:06.013 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.989+0000 7fd7097e6700 1 --2- 172.21.15.39:0/2141957303 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fd6f00a1460 0x7fd6f00a3910 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd6f4005950 tx=0x7fd6f40058e0 comp rx=0 tx=0).stop 2023-07-21T18:43:06.013 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.989+0000 7fd7097e6700 1 -- 172.21.15.39:0/2141957303 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7040ff760 msgr2=0x7fd704196460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:06.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.989+0000 7fd7097e6700 1 --2- 172.21.15.39:0/2141957303 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7040ff760 0x7fd704196460 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec00d950 tx=0x7fd6ec00dd10 comp rx=0 tx=0).stop 2023-07-21T18:43:06.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.989+0000 7fd7097e6700 1 -- 172.21.15.39:0/2141957303 shutdown_connections 2023-07-21T18:43:06.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.989+0000 7fd7097e6700 1 --2- 172.21.15.39:0/2141957303 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fd6f00a1460 0x7fd6f00a3910 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:06.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.989+0000 7fd7097e6700 1 --2- 172.21.15.39:0/2141957303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7040fedc0 0x7fd704195f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:06.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.989+0000 7fd7097e6700 1 --2- 172.21.15.39:0/2141957303 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd704103f10 0x7fd70419aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:06.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.989+0000 7fd7097e6700 1 --2- 172.21.15.39:0/2141957303 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7040ff760 0x7fd704196460 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:06.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.990+0000 7fd7097e6700 1 -- 172.21.15.39:0/2141957303 >> 172.21.15.39:0/2141957303 conn(0x7fd7040759e0 msgr2=0x7fd704102a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:06.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.990+0000 7fd7097e6700 1 -- 172.21.15.39:0/2141957303 shutdown_connections 2023-07-21T18:43:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:05.990+0000 7fd7097e6700 1 -- 172.21.15.39:0/2141957303 wait complete. 2023-07-21T18:43:06.259 INFO:tasks.workunit.client.0.smithi039.stdout: 1 22393 7.25 MB/sec warmup 19 sec latency 1661.951 ms 2023-07-21T18:43:06.618 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:06 smithi193 ceph-mon[98204]: pgmap v43: 97 pgs: 97 active+clean; 11 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 624 KiB/s wr, 160 op/s 2023-07-21T18:43:06.636 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:06 smithi155 ceph-mon[97934]: pgmap v43: 97 pgs: 97 active+clean; 11 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 624 KiB/s wr, 160 op/s 2023-07-21T18:43:06.685 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:06 smithi039 ceph-mon[96489]: pgmap v43: 97 pgs: 97 active+clean; 11 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 624 KiB/s wr, 160 op/s 2023-07-21T18:43:07.259 INFO:tasks.workunit.client.0.smithi039.stdout: 1 23927 7.19 MB/sec warmup 20 sec latency 2112.958 ms 2023-07-21T18:43:08.259 INFO:tasks.workunit.client.0.smithi039.stdout: 1 25597 7.39 MB/sec warmup 21 sec latency 22.507 ms 2023-07-21T18:43:08.618 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:08 smithi193 ceph-mon[98204]: pgmap v44: 97 pgs: 97 active+clean; 14 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 943 KiB/s wr, 233 op/s 2023-07-21T18:43:08.636 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:08 smithi155 ceph-mon[97934]: pgmap v44: 97 pgs: 97 active+clean; 14 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 943 KiB/s wr, 233 op/s 2023-07-21T18:43:08.685 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:08 smithi039 ceph-mon[96489]: pgmap v44: 97 pgs: 97 active+clean; 14 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 943 KiB/s wr, 233 op/s 2023-07-21T18:43:09.259 INFO:tasks.workunit.client.0.smithi039.stdout: 1 25665 7.06 MB/sec warmup 22 sec latency 958.594 ms 2023-07-21T18:43:10.259 INFO:tasks.workunit.client.0.smithi039.stdout: 1 27773 7.25 MB/sec warmup 23 sec latency 1207.205 ms 2023-07-21T18:43:10.617 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:10 smithi193 ceph-mon[98204]: pgmap v45: 97 pgs: 97 active+clean; 14 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 401 KiB/s wr, 86 op/s 2023-07-21T18:43:10.636 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:10 smithi155 ceph-mon[97934]: pgmap v45: 97 pgs: 97 active+clean; 14 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 401 KiB/s wr, 86 op/s 2023-07-21T18:43:10.685 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:10 smithi039 ceph-mon[96489]: pgmap v45: 97 pgs: 97 active+clean; 14 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 401 KiB/s wr, 86 op/s 2023-07-21T18:43:11.259 INFO:tasks.workunit.client.0.smithi039.stdout: 1 29139 7.26 MB/sec warmup 24 sec latency 108.554 ms 2023-07-21T18:43:12.259 INFO:tasks.workunit.client.0.smithi039.stdout: 1 29139 6.97 MB/sec warmup 25 sec latency 1108.619 ms 2023-07-21T18:43:12.295 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T18:43:12.439 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.419+0000 7f484b105700 1 -- 172.21.15.39:0/721116016 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f48441049e0 msgr2=0x7f4844104db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:12.439 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.419+0000 7f484b105700 1 --2- 172.21.15.39:0/721116016 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f48441049e0 0x7f4844104db0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f4838009990 tx=0x7f4838009ca0 comp rx=0 tx=0).stop 2023-07-21T18:43:12.439 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.419+0000 7f484b105700 1 -- 172.21.15.39:0/721116016 shutdown_connections 2023-07-21T18:43:12.439 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.419+0000 7f484b105700 1 --2- 172.21.15.39:0/721116016 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f48440ff380 0x7f484410b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:12.440 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.419+0000 7f484b105700 1 --2- 172.21.15.39:0/721116016 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f48441049e0 0x7f4844104db0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:12.440 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.419+0000 7f484b105700 1 --2- 172.21.15.39:0/721116016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48440fe9d0 0x7f48440fee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:12.440 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.419+0000 7f484b105700 1 -- 172.21.15.39:0/721116016 >> 172.21.15.39:0/721116016 conn(0x7f48440fa4b0 msgr2=0x7f48440fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:12.440 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.419+0000 7f484b105700 1 -- 172.21.15.39:0/721116016 shutdown_connections 2023-07-21T18:43:12.440 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.419+0000 7f484b105700 1 -- 172.21.15.39:0/721116016 wait complete. 2023-07-21T18:43:12.442 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.419+0000 7f484b105700 1 Processor -- start 2023-07-21T18:43:12.442 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.419+0000 7f484b105700 1 -- start start 2023-07-21T18:43:12.442 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.420+0000 7f484b105700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48440fe9d0 0x7f484419a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:12.442 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.420+0000 7f484b105700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f48440ff380 0x7f484419ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:12.442 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.420+0000 7f484b105700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f48441049e0 0x7f48441948d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:12.442 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.420+0000 7f484b105700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4844194f40 con 0x7f48440ff380 2023-07-21T18:43:12.442 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.420+0000 7f484b105700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48441950b0 con 0x7f48440fe9d0 2023-07-21T18:43:12.443 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.420+0000 7f484b105700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4844195270 con 0x7f48441049e0 2023-07-21T18:43:12.443 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.420+0000 7f4843fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f48440ff380 0x7f484419ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:12.443 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.420+0000 7f4843fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f48440ff380 0x7f484419ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58462/0 (socket says 172.21.15.39:58462) 2023-07-21T18:43:12.443 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.420+0000 7f4843fff700 1 -- 172.21.15.39:0/211458630 learned_addr learned my addr 172.21.15.39:0/211458630 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:12.443 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.420+0000 7f4848ea1700 1 --2- 172.21.15.39:0/211458630 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48440fe9d0 0x7f484419a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:12.443 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.420+0000 7f48496a2700 1 --2- 172.21.15.39:0/211458630 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f48441049e0 0x7f48441948d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:12.443 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.420+0000 7f4843fff700 1 -- 172.21.15.39:0/211458630 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f48441049e0 msgr2=0x7f48441948d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:12.444 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.420+0000 7f4843fff700 1 --2- 172.21.15.39:0/211458630 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f48441049e0 0x7f48441948d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:12.444 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.420+0000 7f4843fff700 1 -- 172.21.15.39:0/211458630 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48440fe9d0 msgr2=0x7f484419a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:12.444 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.420+0000 7f4843fff700 1 --2- 172.21.15.39:0/211458630 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48440fe9d0 0x7f484419a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:12.444 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.420+0000 7f4843fff700 1 -- 172.21.15.39:0/211458630 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48380095f0 con 0x7f48440ff380 2023-07-21T18:43:12.444 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.420+0000 7f48496a2700 1 --2- 172.21.15.39:0/211458630 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f48441049e0 0x7f48441948d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:43:12.444 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.420+0000 7f4843fff700 1 --2- 172.21.15.39:0/211458630 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f48440ff380 0x7f484419ada0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f483000d950 tx=0x7f483000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:12.444 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.420+0000 7f4841ffb700 1 -- 172.21.15.39:0/211458630 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4830009940 con 0x7f48440ff380 2023-07-21T18:43:12.444 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.420+0000 7f4841ffb700 1 -- 172.21.15.39:0/211458630 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4830010460 con 0x7f48440ff380 2023-07-21T18:43:12.445 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.420+0000 7f484b105700 1 -- 172.21.15.39:0/211458630 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4844195550 con 0x7f48440ff380 2023-07-21T18:43:12.445 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.420+0000 7f4841ffb700 1 -- 172.21.15.39:0/211458630 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f483000f620 con 0x7f48440ff380 2023-07-21T18:43:12.445 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.421+0000 7f484b105700 1 -- 172.21.15.39:0/211458630 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48441a1850 con 0x7f48440ff380 2023-07-21T18:43:12.448 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.423+0000 7f4841ffb700 1 -- 172.21.15.39:0/211458630 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f48300108d0 con 0x7f48440ff380 2023-07-21T18:43:12.448 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.423+0000 7f4841ffb700 1 --2- 172.21.15.39:0/211458630 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f48340a13e0 0x7f48340a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:12.448 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.424+0000 7f4848ea1700 1 --2- 172.21.15.39:0/211458630 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f48340a13e0 0x7f48340a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:12.449 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.424+0000 7f4841ffb700 1 -- 172.21.15.39:0/211458630 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f48300cce20 con 0x7f48440ff380 2023-07-21T18:43:12.449 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.424+0000 7f4848ea1700 1 --2- 172.21.15.39:0/211458630 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f48340a13e0 0x7f48340a3890 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f483800b5c0 tx=0x7f4838005fd0 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:12.449 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.425+0000 7f485728bb80 1 Processor -- start 2023-07-21T18:43:12.449 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.425+0000 7f485728bb80 1 -- start start 2023-07-21T18:43:12.449 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.425+0000 7f485728bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595f6da42d0 0x5595f6da46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:12.449 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.425+0000 7f485728bb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595f6da4be0 0x5595f6daf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:12.449 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.425+0000 7f485728bb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595f6dafad0 0x5595f6db1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:12.450 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.425+0000 7f485728bb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595f6caacb0 con 0x5595f6dafad0 2023-07-21T18:43:12.450 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.425+0000 7f485728bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595f6cfda40 con 0x5595f6da42d0 2023-07-21T18:43:12.450 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.425+0000 7f485728bb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595f6cf6ed0 con 0x5595f6da4be0 2023-07-21T18:43:12.450 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.425+0000 7f48496a2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595f6dafad0 0x5595f6db1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:12.450 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.425+0000 7f48496a2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595f6dafad0 0x5595f6db1eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58472/0 (socket says 172.21.15.39:58472) 2023-07-21T18:43:12.450 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.425+0000 7f48496a2700 1 -- 172.21.15.39:0/521072827 learned_addr learned my addr 172.21.15.39:0/521072827 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:12.450 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.425+0000 7f4843fff700 1 --2- 172.21.15.39:0/521072827 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595f6da4be0 0x5595f6daf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:12.450 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.425+0000 7f4848ea1700 1 --2- 172.21.15.39:0/521072827 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595f6da42d0 0x5595f6da46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:12.451 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.425+0000 7f48496a2700 1 -- 172.21.15.39:0/521072827 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595f6da4be0 msgr2=0x5595f6daf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:12.451 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.425+0000 7f48496a2700 1 --2- 172.21.15.39:0/521072827 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595f6da4be0 0x5595f6daf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:12.451 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.425+0000 7f48496a2700 1 -- 172.21.15.39:0/521072827 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595f6da42d0 msgr2=0x5595f6da46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:12.451 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.425+0000 7f48496a2700 1 --2- 172.21.15.39:0/521072827 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595f6da42d0 0x5595f6da46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:12.451 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.425+0000 7f48496a2700 1 -- 172.21.15.39:0/521072827 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595f6b24de0 con 0x5595f6dafad0 2023-07-21T18:43:12.451 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.426+0000 7f48496a2700 1 --2- 172.21.15.39:0/521072827 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595f6dafad0 0x5595f6db1eb0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f483c009dc0 tx=0x7f483c0077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55ea8967da2062a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:12.451 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.426+0000 7f482effd700 1 -- 172.21.15.39:0/521072827 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f483c013670 con 0x5595f6dafad0 2023-07-21T18:43:12.451 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.426+0000 7f482effd700 1 -- 172.21.15.39:0/521072827 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f483c013c90 con 0x5595f6dafad0 2023-07-21T18:43:12.452 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.426+0000 7f482effd700 1 -- 172.21.15.39:0/521072827 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f483c012b40 con 0x5595f6dafad0 2023-07-21T18:43:12.452 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.426+0000 7f485728bb80 1 -- 172.21.15.39:0/521072827 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595f6dafad0 msgr2=0x5595f6db1eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:12.452 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.426+0000 7f485728bb80 1 --2- 172.21.15.39:0/521072827 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595f6dafad0 0x5595f6db1eb0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f483c009dc0 tx=0x7f483c0077e0 comp rx=0 tx=0).stop 2023-07-21T18:43:12.452 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.426+0000 7f485728bb80 1 -- 172.21.15.39:0/521072827 shutdown_connections 2023-07-21T18:43:12.452 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.426+0000 7f485728bb80 1 --2- 172.21.15.39:0/521072827 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595f6da42d0 0x5595f6da46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:12.452 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.426+0000 7f485728bb80 1 --2- 172.21.15.39:0/521072827 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595f6dafad0 0x5595f6db1eb0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:12.452 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.426+0000 7f485728bb80 1 --2- 172.21.15.39:0/521072827 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595f6da4be0 0x5595f6daf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:12.453 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.426+0000 7f485728bb80 1 -- 172.21.15.39:0/521072827 >> 172.21.15.39:0/521072827 conn(0x5595f6d9ada0 msgr2=0x5595f6da06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:12.453 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.426+0000 7f485728bb80 1 -- 172.21.15.39:0/521072827 shutdown_connections 2023-07-21T18:43:12.453 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.426+0000 7f485728bb80 1 -- 172.21.15.39:0/521072827 wait complete. 2023-07-21T18:43:12.453 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.427+0000 7f485728bb80 1 -- start start 2023-07-21T18:43:12.453 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.427+0000 7f485728bb80 1 Processor -- start 2023-07-21T18:43:12.453 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.427+0000 7f485728bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595f6da42d0 0x5595f6daf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:12.453 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.427+0000 7f485728bb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595f6da4be0 0x5595f6dac7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:12.453 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.427+0000 7f485728bb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595f6dafad0 0x5595f6dacf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:12.454 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.427+0000 7f485728bb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595f6c96ac0 con 0x5595f6dafad0 2023-07-21T18:43:12.454 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.427+0000 7f485728bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595f6c967c0 con 0x5595f6da42d0 2023-07-21T18:43:12.454 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.427+0000 7f485728bb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595f6b7a920 con 0x5595f6da4be0 2023-07-21T18:43:12.454 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.427+0000 7f485728bb80 10 client.0 _ll_register_callbacks cb 0x5595f6c94760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:43:12.454 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.427+0000 7f48496a2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595f6dafad0 0x5595f6dacf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:12.454 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.428+0000 7f48496a2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595f6dafad0 0x5595f6dacf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58478/0 (socket says 172.21.15.39:58478) 2023-07-21T18:43:12.454 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.428+0000 7f48496a2700 1 -- 172.21.15.39:0/155902714 learned_addr learned my addr 172.21.15.39:0/155902714 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:12.454 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.428+0000 7f4848ea1700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595f6da42d0 0x5595f6daf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:12.455 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.428+0000 7f4843fff700 1 --2- 172.21.15.39:0/155902714 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595f6da4be0 0x5595f6dac7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:12.455 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.428+0000 7f48496a2700 1 -- 172.21.15.39:0/155902714 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595f6da4be0 msgr2=0x5595f6dac7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:12.455 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.428+0000 7f48496a2700 1 --2- 172.21.15.39:0/155902714 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595f6da4be0 0x5595f6dac7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:12.455 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.428+0000 7f48496a2700 1 -- 172.21.15.39:0/155902714 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595f6da42d0 msgr2=0x5595f6daf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:12.455 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.428+0000 7f48496a2700 1 --2- 172.21.15.39:0/155902714 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595f6da42d0 0x5595f6daf570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:12.455 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.428+0000 7f48496a2700 1 -- 172.21.15.39:0/155902714 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f483c0099f0 con 0x5595f6dafad0 2023-07-21T18:43:12.455 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.428+0000 7f48496a2700 1 --2- 172.21.15.39:0/155902714 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595f6dafad0 0x5595f6dacf30 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f483c0126c0 tx=0x7f483c0136a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63b4a76243e3027 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:12.455 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.428+0000 7f4823fff700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:43:12.456 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.428+0000 7f4823fff700 1 -- 172.21.15.39:0/155902714 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f483c022070 con 0x5595f6dafad0 2023-07-21T18:43:12.456 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.428+0000 7f4823fff700 1 -- 172.21.15.39:0/155902714 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f483c013a20 con 0x5595f6dafad0 2023-07-21T18:43:12.456 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.428+0000 7f485728bb80 1 -- 172.21.15.39:0/155902714 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5595f6c504f0 con 0x5595f6dafad0 2023-07-21T18:43:12.456 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.428+0000 7f4823fff700 1 -- 172.21.15.39:0/155902714 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f483c0124e0 con 0x5595f6dafad0 2023-07-21T18:43:12.456 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.428+0000 7f485728bb80 10 client.15960 fetch_fsmap learned FSMap version 28 2023-07-21T18:43:12.456 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.428+0000 7f485728bb80 1 -- 172.21.15.39:0/155902714 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5595f6b0dc60 con 0x5595f6dafad0 2023-07-21T18:43:12.456 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.429+0000 7f4823fff700 1 -- 172.21.15.39:0/155902714 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5595f6c504f0 con 0x5595f6dafad0 2023-07-21T18:43:12.458 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.429+0000 7f4823fff700 1 -- 172.21.15.39:0/155902714 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f483c00e070 con 0x5595f6dafad0 2023-07-21T18:43:12.458 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.429+0000 7f485728bb80 10 client.15960 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T18:43:12.458 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.429+0000 7f485728bb80 10 client.15960 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T18:43:12.458 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.429+0000 7f485728bb80 20 client.15960 populate_metadata read hostname 'smithi039' 2023-07-21T18:43:12.458 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.429+0000 7f485728bb80 1 --2- 172.21.15.39:0/155902714 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5595f6dbf500 0x5595f6dc1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:12.459 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.429+0000 7f485728bb80 4 client.15960 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:43:12.459 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.429+0000 7f485728bb80 1 -- 172.21.15.39:0/155902714 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5595f6cefeb0 con 0x5595f6dbf500 2023-07-21T18:43:12.459 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.429+0000 7f4848ea1700 1 --2- 172.21.15.39:0/155902714 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5595f6dbf500 0x5595f6dc1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:12.459 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.429+0000 7f4823fff700 10 client.15960 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:43:12.459 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.429+0000 7f4848ea1700 1 --2- 172.21.15.39:0/155902714 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5595f6dbf500 0x5595f6dc1910 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1f59ca5202b8105 server_cookie=6ab99b64d42d0e44 in_seq=0 out_seq=0 2023-07-21T18:43:12.459 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.430+0000 7f4823fff700 1 -- 172.21.15.39:0/155902714 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5595f6cefeb0 con 0x5595f6dbf500 2023-07-21T18:43:12.459 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.430+0000 7f4823fff700 10 client.15960 handle_command_reply: tid=0 2023-07-21T18:43:12.459 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.430+0000 7f485728bb80 1 client.15960 shutdown 2023-07-21T18:43:12.460 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.430+0000 7f485728bb80 1 -- 172.21.15.39:0/155902714 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595f6dafad0 msgr2=0x5595f6dacf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:12.460 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.430+0000 7f485728bb80 1 --2- 172.21.15.39:0/155902714 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595f6dafad0 0x5595f6dacf30 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f483c0126c0 tx=0x7f483c0136a0 comp rx=0 tx=0).stop 2023-07-21T18:43:12.460 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.430+0000 7f485728bb80 1 -- 172.21.15.39:0/155902714 shutdown_connections 2023-07-21T18:43:12.460 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.430+0000 7f485728bb80 1 --2- 172.21.15.39:0/155902714 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5595f6dbf500 0x5595f6dc1910 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:12.460 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.430+0000 7f485728bb80 1 --2- 172.21.15.39:0/155902714 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595f6da42d0 0x5595f6daf570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:12.460 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.430+0000 7f485728bb80 1 --2- 172.21.15.39:0/155902714 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595f6dafad0 0x5595f6dacf30 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:12.460 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.430+0000 7f4823fff700 0 client.15960 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:43:12.461 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.430+0000 7f485728bb80 1 --2- 172.21.15.39:0/155902714 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595f6da4be0 0x5595f6dac7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:12.461 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.430+0000 7f485728bb80 1 -- 172.21.15.39:0/155902714 >> 172.21.15.39:0/155902714 conn(0x5595f6d9ada0 msgr2=0x5595f6da0230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:12.461 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.430+0000 7f485728bb80 1 -- 172.21.15.39:0/155902714 shutdown_connections 2023-07-21T18:43:12.461 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.430+0000 7f485728bb80 1 -- 172.21.15.39:0/155902714 wait complete. 2023-07-21T18:43:12.461 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.431+0000 7f485728bb80 20 client.15960 trim_cache size 0 max 16384 2023-07-21T18:43:12.468 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.447+0000 7f485728bb80 1 Processor -- start 2023-07-21T18:43:12.468 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.447+0000 7f485728bb80 1 -- start start 2023-07-21T18:43:12.468 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.447+0000 7f485728bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595f6da42d0 0x5595f6dbb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:12.469 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.448+0000 7f485728bb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595f6da4be0 0x5595f6dabd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:12.469 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.448+0000 7f485728bb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595f6dafad0 0x5595f6daf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:12.469 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.448+0000 7f485728bb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595f6d9cef0 con 0x5595f6dafad0 2023-07-21T18:43:12.469 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.448+0000 7f485728bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595f6da1d50 con 0x5595f6da42d0 2023-07-21T18:43:12.469 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.448+0000 7f485728bb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595f6d97fa0 con 0x5595f6da4be0 2023-07-21T18:43:12.469 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.448+0000 7f48496a2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595f6dafad0 0x5595f6daf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:12.470 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.448+0000 7f48496a2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595f6dafad0 0x5595f6daf0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58486/0 (socket says 172.21.15.39:58486) 2023-07-21T18:43:12.470 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.448+0000 7f48496a2700 1 -- 172.21.15.39:0/3575610763 learned_addr learned my addr 172.21.15.39:0/3575610763 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:12.470 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.448+0000 7f4843fff700 1 --2- 172.21.15.39:0/3575610763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595f6da4be0 0x5595f6dabd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:12.470 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.448+0000 7f4848ea1700 1 --2- 172.21.15.39:0/3575610763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595f6da42d0 0x5595f6dbb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:12.470 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.448+0000 7f48496a2700 1 -- 172.21.15.39:0/3575610763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595f6da4be0 msgr2=0x5595f6dabd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:12.470 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.448+0000 7f48496a2700 1 --2- 172.21.15.39:0/3575610763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595f6da4be0 0x5595f6dabd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:12.470 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.448+0000 7f48496a2700 1 -- 172.21.15.39:0/3575610763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595f6da42d0 msgr2=0x5595f6dbb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:12.471 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.448+0000 7f48496a2700 1 --2- 172.21.15.39:0/3575610763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595f6da42d0 0x5595f6dbb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:12.471 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.448+0000 7f48496a2700 1 -- 172.21.15.39:0/3575610763 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595f6b4b250 con 0x5595f6dafad0 2023-07-21T18:43:12.471 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.448+0000 7f4848ea1700 1 --2- 172.21.15.39:0/3575610763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595f6da42d0 0x5595f6dbb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:43:12.471 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.448+0000 7f48496a2700 1 --2- 172.21.15.39:0/3575610763 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595f6dafad0 0x5595f6daf0a0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f483c002230 tx=0x7f483c0060f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3ad88a4bd18b431 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:12.471 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.448+0000 7f482f7fe700 1 -- 172.21.15.39:0/3575610763 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f483c022040 con 0x5595f6dafad0 2023-07-21T18:43:12.471 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.448+0000 7f482f7fe700 1 -- 172.21.15.39:0/3575610763 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f483c012480 con 0x5595f6dafad0 2023-07-21T18:43:12.471 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.448+0000 7f482f7fe700 1 -- 172.21.15.39:0/3575610763 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f483c012bb0 con 0x5595f6dafad0 2023-07-21T18:43:12.474 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.449+0000 7f485728bb80 1 -- 172.21.15.39:0/3575610763 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595f6dafad0 msgr2=0x5595f6daf0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:12.474 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.449+0000 7f485728bb80 1 --2- 172.21.15.39:0/3575610763 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595f6dafad0 0x5595f6daf0a0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f483c002230 tx=0x7f483c0060f0 comp rx=0 tx=0).stop 2023-07-21T18:43:12.475 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.449+0000 7f485728bb80 1 -- 172.21.15.39:0/3575610763 shutdown_connections 2023-07-21T18:43:12.475 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.449+0000 7f485728bb80 1 --2- 172.21.15.39:0/3575610763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595f6da42d0 0x5595f6dbb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:12.475 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.449+0000 7f485728bb80 1 --2- 172.21.15.39:0/3575610763 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595f6dafad0 0x5595f6daf0a0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:12.475 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.449+0000 7f485728bb80 1 --2- 172.21.15.39:0/3575610763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595f6da4be0 0x5595f6dabd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:12.475 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.449+0000 7f485728bb80 1 -- 172.21.15.39:0/3575610763 >> 172.21.15.39:0/3575610763 conn(0x5595f6d9ada0 msgr2=0x5595f6d9ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:12.475 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.449+0000 7f485728bb80 1 -- 172.21.15.39:0/3575610763 shutdown_connections 2023-07-21T18:43:12.475 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.449+0000 7f485728bb80 1 -- 172.21.15.39:0/3575610763 wait complete. 2023-07-21T18:43:12.476 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.449+0000 7f485728bb80 1 -- start start 2023-07-21T18:43:12.476 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.449+0000 7f485728bb80 1 Processor -- start 2023-07-21T18:43:12.476 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.450+0000 7f485728bb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595f6da42d0 0x5595f6db6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:12.476 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.450+0000 7f485728bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595f6da4be0 0x5595f6db6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:12.476 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.450+0000 7f485728bb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595f6dafad0 0x5595f6db4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:12.476 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.450+0000 7f485728bb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595f6db4c80 con 0x5595f6dafad0 2023-07-21T18:43:12.476 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.450+0000 7f485728bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595f6db4df0 con 0x5595f6da4be0 2023-07-21T18:43:12.476 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.450+0000 7f485728bb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595f6db4fb0 con 0x5595f6da42d0 2023-07-21T18:43:12.477 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.450+0000 7f485728bb80 10 client.0 _ll_register_callbacks cb 0x7f4830096000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:43:12.477 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.450+0000 7f48496a2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595f6dafad0 0x5595f6db4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:12.477 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.450+0000 7f48496a2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595f6dafad0 0x5595f6db4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58492/0 (socket says 172.21.15.39:58492) 2023-07-21T18:43:12.477 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.450+0000 7f48496a2700 1 -- 172.21.15.39:0/868203847 learned_addr learned my addr 172.21.15.39:0/868203847 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:12.477 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.450+0000 7f4843fff700 1 --2- 172.21.15.39:0/868203847 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595f6da4be0 0x5595f6db6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:12.477 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.450+0000 7f4848ea1700 1 --2- 172.21.15.39:0/868203847 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595f6da42d0 0x5595f6db6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:12.477 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.450+0000 7f48496a2700 1 -- 172.21.15.39:0/868203847 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595f6da42d0 msgr2=0x5595f6db6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:12.477 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.450+0000 7f48496a2700 1 --2- 172.21.15.39:0/868203847 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595f6da42d0 0x5595f6db6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:12.478 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.450+0000 7f48496a2700 1 -- 172.21.15.39:0/868203847 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595f6da4be0 msgr2=0x5595f6db6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:12.478 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.450+0000 7f48496a2700 1 --2- 172.21.15.39:0/868203847 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595f6da4be0 0x5595f6db6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:12.478 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.450+0000 7f48496a2700 1 -- 172.21.15.39:0/868203847 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5595f6b0dc60 con 0x5595f6dafad0 2023-07-21T18:43:12.478 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.450+0000 7f48496a2700 1 --2- 172.21.15.39:0/868203847 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595f6dafad0 0x5595f6db4570 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f483c01c830 tx=0x7f483c01cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f22a2fcb96af5b9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:12.478 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.450+0000 7f4823fff700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:43:12.478 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.451+0000 7f4823fff700 1 -- 172.21.15.39:0/868203847 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f483c041070 con 0x5595f6dafad0 2023-07-21T18:43:12.478 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.451+0000 7f485728bb80 1 -- 172.21.15.39:0/868203847 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f483c013c90 con 0x5595f6dafad0 2023-07-21T18:43:12.478 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.451+0000 7f4823fff700 1 -- 172.21.15.39:0/868203847 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f483c0128c0 con 0x5595f6dafad0 2023-07-21T18:43:12.479 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.451+0000 7f4823fff700 1 -- 172.21.15.39:0/868203847 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f483c022040 con 0x5595f6dafad0 2023-07-21T18:43:12.479 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.451+0000 7f485728bb80 10 client.15966 fetch_fsmap learned FSMap version 28 2023-07-21T18:43:12.479 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.451+0000 7f485728bb80 1 -- 172.21.15.39:0/868203847 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5595f6b232b0 con 0x5595f6dafad0 2023-07-21T18:43:12.479 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.451+0000 7f4823fff700 1 -- 172.21.15.39:0/868203847 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f483c013c90 con 0x5595f6dafad0 2023-07-21T18:43:12.479 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.451+0000 7f4823fff700 1 -- 172.21.15.39:0/868203847 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f483c044020 con 0x5595f6dafad0 2023-07-21T18:43:12.479 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.451+0000 7f485728bb80 10 client.15966 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T18:43:12.479 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.451+0000 7f485728bb80 10 client.15966 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T18:43:12.479 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.451+0000 7f485728bb80 20 client.15966 populate_metadata read hostname 'smithi039' 2023-07-21T18:43:12.480 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.451+0000 7f485728bb80 1 --2- 172.21.15.39:0/868203847 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5595f6e14160 0x5595f6e16570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:12.480 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.451+0000 7f485728bb80 4 client.15966 mds_command: new command op to 24536 tid=0[{"prefix": "scrub status"}] 2023-07-21T18:43:12.480 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.451+0000 7f485728bb80 1 -- 172.21.15.39:0/868203847 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5595f6cb2050 con 0x5595f6e14160 2023-07-21T18:43:12.480 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.452+0000 7f4848ea1700 1 --2- 172.21.15.39:0/868203847 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5595f6e14160 0x5595f6e16570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:12.480 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.452+0000 7f4848ea1700 1 --2- 172.21.15.39:0/868203847 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5595f6e14160 0x5595f6e16570 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c32943e387ae62f0 server_cookie=f303484fecf7a288 in_seq=0 out_seq=0 2023-07-21T18:43:12.480 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.452+0000 7f4823fff700 10 client.15966 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:43:12.480 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.452+0000 7f4823fff700 1 -- 172.21.15.39:0/868203847 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5595f6cb2050 con 0x5595f6e14160 2023-07-21T18:43:12.481 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.452+0000 7f4823fff700 10 client.15966 handle_command_reply: tid=0 2023-07-21T18:43:12.481 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.452+0000 7f485728bb80 1 client.15966 shutdown 2023-07-21T18:43:12.481 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.452+0000 7f485728bb80 1 -- 172.21.15.39:0/868203847 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595f6dafad0 msgr2=0x5595f6db4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:12.481 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.452+0000 7f485728bb80 1 --2- 172.21.15.39:0/868203847 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595f6dafad0 0x5595f6db4570 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f483c01c830 tx=0x7f483c01cbd0 comp rx=0 tx=0).stop 2023-07-21T18:43:12.481 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.452+0000 7f485728bb80 1 -- 172.21.15.39:0/868203847 shutdown_connections 2023-07-21T18:43:12.481 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.452+0000 7f485728bb80 1 --2- 172.21.15.39:0/868203847 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5595f6e14160 0x5595f6e16570 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:12.481 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.452+0000 7f485728bb80 1 --2- 172.21.15.39:0/868203847 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595f6dafad0 0x5595f6db4570 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:12.482 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.452+0000 7f485728bb80 1 --2- 172.21.15.39:0/868203847 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595f6da42d0 0x5595f6db6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:12.482 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.452+0000 7f4823fff700 0 client.15966 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:43:12.482 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.452+0000 7f485728bb80 1 --2- 172.21.15.39:0/868203847 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595f6da4be0 0x5595f6db6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:12.482 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.452+0000 7f485728bb80 1 -- 172.21.15.39:0/868203847 >> 172.21.15.39:0/868203847 conn(0x5595f6d9ada0 msgr2=0x5595f6da7b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:12.482 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.452+0000 7f485728bb80 1 -- 172.21.15.39:0/868203847 shutdown_connections 2023-07-21T18:43:12.482 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.453+0000 7f485728bb80 1 -- 172.21.15.39:0/868203847 wait complete. 2023-07-21T18:43:12.482 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.453+0000 7f485728bb80 20 client.15966 trim_cache size 0 max 16384 2023-07-21T18:43:12.483 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2023-07-21T18:43:12.483 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "no active scrubs running", 2023-07-21T18:43:12.483 INFO:teuthology.orchestra.run.smithi039.stdout: "scrubs": {} 2023-07-21T18:43:12.483 INFO:teuthology.orchestra.run.smithi039.stdout:} 2023-07-21T18:43:12.484 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.453+0000 7f484b105700 1 -- 172.21.15.39:0/211458630 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f48340a13e0 msgr2=0x7f48340a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:12.484 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.453+0000 7f484b105700 1 --2- 172.21.15.39:0/211458630 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f48340a13e0 0x7f48340a3890 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f483800b5c0 tx=0x7f4838005fd0 comp rx=0 tx=0).stop 2023-07-21T18:43:12.484 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.453+0000 7f484b105700 1 -- 172.21.15.39:0/211458630 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f48440ff380 msgr2=0x7f484419ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:12.484 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.453+0000 7f484b105700 1 --2- 172.21.15.39:0/211458630 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f48440ff380 0x7f484419ada0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f483000d950 tx=0x7f483000dc60 comp rx=0 tx=0).stop 2023-07-21T18:43:12.484 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.453+0000 7f484b105700 1 -- 172.21.15.39:0/211458630 shutdown_connections 2023-07-21T18:43:12.485 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.453+0000 7f484b105700 1 --2- 172.21.15.39:0/211458630 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f48340a13e0 0x7f48340a3890 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:12.485 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.453+0000 7f484b105700 1 --2- 172.21.15.39:0/211458630 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48440fe9d0 0x7f484419a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:12.485 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.453+0000 7f484b105700 1 --2- 172.21.15.39:0/211458630 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f48441049e0 0x7f48441948d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:12.485 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.453+0000 7f484b105700 1 --2- 172.21.15.39:0/211458630 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f48440ff380 0x7f484419ada0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:12.485 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.453+0000 7f484b105700 1 -- 172.21.15.39:0/211458630 >> 172.21.15.39:0/211458630 conn(0x7f48440fa4b0 msgr2=0x7f48440fbaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:12.485 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.453+0000 7f484b105700 1 -- 172.21.15.39:0/211458630 shutdown_connections 2023-07-21T18:43:12.485 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.453+0000 7f484b105700 1 -- 172.21.15.39:0/211458630 wait complete. 2023-07-21T18:43:12.486 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:12 smithi039 ceph-mon[96489]: pgmap v46: 97 pgs: 97 active+clean; 12 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 694 KiB/s wr, 156 op/s 2023-07-21T18:43:12.486 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-21T18:43:12.487 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T18:43:12.613 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.593+0000 7efd7f604700 1 -- 172.21.15.39:0/4060841759 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd780fe9d0 msgr2=0x7efd780fee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:12.613 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.593+0000 7efd7f604700 1 --2- 172.21.15.39:0/4060841759 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd780fe9d0 0x7efd780fee40 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7efd68009990 tx=0x7efd68009ca0 comp rx=0 tx=0).stop 2023-07-21T18:43:12.613 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.593+0000 7efd7f604700 1 -- 172.21.15.39:0/4060841759 shutdown_connections 2023-07-21T18:43:12.613 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.593+0000 7efd7f604700 1 --2- 172.21.15.39:0/4060841759 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd781049e0 0x7efd78104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:12.614 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.593+0000 7efd7f604700 1 --2- 172.21.15.39:0/4060841759 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efd780ff380 0x7efd7810b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:12.614 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.593+0000 7efd7f604700 1 --2- 172.21.15.39:0/4060841759 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd780fe9d0 0x7efd780fee40 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:12.614 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.593+0000 7efd7f604700 1 -- 172.21.15.39:0/4060841759 >> 172.21.15.39:0/4060841759 conn(0x7efd780fa490 msgr2=0x7efd780fc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:12.614 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.593+0000 7efd7f604700 1 -- 172.21.15.39:0/4060841759 shutdown_connections 2023-07-21T18:43:12.614 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.593+0000 7efd7f604700 1 -- 172.21.15.39:0/4060841759 wait complete. 2023-07-21T18:43:12.615 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.593+0000 7efd7f604700 1 Processor -- start 2023-07-21T18:43:12.616 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.593+0000 7efd7f604700 1 -- start start 2023-07-21T18:43:12.616 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.594+0000 7efd7f604700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efd780fe9d0 0x7efd78197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:12.616 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.594+0000 7efd7f604700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd780ff380 0x7efd78198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:12.616 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.594+0000 7efd7f604700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd781049e0 0x7efd7819c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:12.616 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.594+0000 7efd7f604700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd7819cd10 con 0x7efd780ff380 2023-07-21T18:43:12.616 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.594+0000 7efd7f604700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd7819ce80 con 0x7efd781049e0 2023-07-21T18:43:12.617 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.594+0000 7efd7f604700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd7819d040 con 0x7efd780fe9d0 2023-07-21T18:43:12.617 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.594+0000 7efd7cb9f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd780ff380 0x7efd78198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:12.617 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.594+0000 7efd7cb9f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd780ff380 0x7efd78198270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58504/0 (socket says 172.21.15.39:58504) 2023-07-21T18:43:12.617 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.594+0000 7efd7cb9f700 1 -- 172.21.15.39:0/817293471 learned_addr learned my addr 172.21.15.39:0/817293471 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:12.617 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.594+0000 7efd7d3a0700 1 --2- 172.21.15.39:0/817293471 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efd780fe9d0 0x7efd78197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:12.617 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.594+0000 7efd7cb9f700 1 -- 172.21.15.39:0/817293471 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efd780fe9d0 msgr2=0x7efd78197d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:12.617 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.594+0000 7efd7cb9f700 1 --2- 172.21.15.39:0/817293471 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efd780fe9d0 0x7efd78197d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:12.618 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.594+0000 7efd7cb9f700 1 -- 172.21.15.39:0/817293471 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd781049e0 msgr2=0x7efd7819c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:12.618 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.594+0000 7efd7cb9f700 1 --2- 172.21.15.39:0/817293471 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd781049e0 0x7efd7819c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:12.618 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.594+0000 7efd7cb9f700 1 -- 172.21.15.39:0/817293471 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd680095f0 con 0x7efd780ff380 2023-07-21T18:43:12.618 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.594+0000 7efd7cb9f700 1 --2- 172.21.15.39:0/817293471 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd780ff380 0x7efd78198270 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7efd68000c00 tx=0x7efd6800f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:12.618 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.594+0000 7efd6e7fc700 1 -- 172.21.15.39:0/817293471 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd6801d070 con 0x7efd780ff380 2023-07-21T18:43:12.618 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.594+0000 7efd7f604700 1 -- 172.21.15.39:0/817293471 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd7819d2c0 con 0x7efd780ff380 2023-07-21T18:43:12.618 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.594+0000 7efd6e7fc700 1 -- 172.21.15.39:0/817293471 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd6800fba0 con 0x7efd780ff380 2023-07-21T18:43:12.618 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.594+0000 7efd6e7fc700 1 -- 172.21.15.39:0/817293471 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd680173f0 con 0x7efd780ff380 2023-07-21T18:43:12.619 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.594+0000 7efd7f604700 1 -- 172.21.15.39:0/817293471 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd781a4b60 con 0x7efd780ff380 2023-07-21T18:43:12.619 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:12 smithi193 ceph-mon[98204]: pgmap v46: 97 pgs: 97 active+clean; 12 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 694 KiB/s wr, 156 op/s 2023-07-21T18:43:12.620 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.596+0000 7efd7f604700 1 -- 172.21.15.39:0/817293471 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd78067300 con 0x7efd780ff380 2023-07-21T18:43:12.620 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.597+0000 7efd6e7fc700 1 -- 172.21.15.39:0/817293471 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7efd68021410 con 0x7efd780ff380 2023-07-21T18:43:12.620 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.597+0000 7efd6e7fc700 1 --2- 172.21.15.39:0/817293471 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7efd640a1430 0x7efd640a38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:12.620 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.597+0000 7efd6e7fc700 1 -- 172.21.15.39:0/817293471 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7efd680ce0d0 con 0x7efd780ff380 2023-07-21T18:43:12.620 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.597+0000 7efd7d3a0700 1 --2- 172.21.15.39:0/817293471 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7efd640a1430 0x7efd640a38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:12.620 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.598+0000 7efd7d3a0700 1 --2- 172.21.15.39:0/817293471 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7efd640a1430 0x7efd640a38e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7efd74005fd0 tx=0x7efd74005dd0 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:12.621 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.602+0000 7efd6e7fc700 1 -- 172.21.15.39:0/817293471 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7efd68097500 con 0x7efd780ff380 2023-07-21T18:43:12.636 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:12 smithi155 ceph-mon[97934]: pgmap v46: 97 pgs: 97 active+clean; 12 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 694 KiB/s wr, 156 op/s 2023-07-21T18:43:12.865 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.845+0000 7efd7f604700 1 -- 172.21.15.39:0/817293471 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efd781a4cf0 con 0x7efd780ff380 2023-07-21T18:43:12.865 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.846+0000 7efd6e7fc700 1 -- 172.21.15.39:0/817293471 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10894 (secure 0 0 0) 0x7efd6809ab20 con 0x7efd780ff380 2023-07-21T18:43:12.866 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:43:12.866 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/147318681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":147318681},{"type":"v1","addr":"172.21.15.39:6835","nonce":147318681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/1062891760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1062891760},{"type":"v1","addr":"172.21.15.39:6837","nonce":1062891760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T18:40:04.594553+0000","modified":"2023-07-21T18:42:59.152694+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24536,"mds_1":24500,"mds_2":24487,"mds_3":24464,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/3184273349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":3184273349},{"type":"v1","addr":"172.21.15.39:6839","nonce":3184273349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6841/640981220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":640981220},{"type":"v1","addr":"172.21.15.39:6841","nonce":640981220}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3106166804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3106166804},{"type":"v1","addr":"172.21.15.193:6833","nonce":3106166804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24437":{"gid":24437,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/1034755277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1034755277},{"type":"v1","addr":"172.21.15.155:6833","nonce":1034755277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/381560657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":381560657},{"type":"v1","addr":"172.21.15.193:6835","nonce":381560657}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6837/2626761843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2626761843},{"type":"v1","addr":"172.21.15.193:6837","nonce":2626761843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24464":{"gid":24464,"name":"e","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.155:6835/3854856286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3854856286},{"type":"v1","addr":"172.21.15.155:6835","nonce":3854856286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.193:6839/3148488639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3148488639},{"type":"v1","addr":"172.21.15.193:6839","nonce":3148488639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.155:6837/947670056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":947670056},{"type":"v1","addr":"172.21.15.155:6837","nonce":947670056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6839/431690342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":431690342},{"type":"v1","addr":"172.21.15.155:6839","nonce":431690342}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T18:43:12.868 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.848+0000 7efd7f604700 1 -- 172.21.15.39:0/817293471 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7efd640a1430 msgr2=0x7efd640a38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:12.868 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.848+0000 7efd7f604700 1 --2- 172.21.15.39:0/817293471 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7efd640a1430 0x7efd640a38e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7efd74005fd0 tx=0x7efd74005dd0 comp rx=0 tx=0).stop 2023-07-21T18:43:12.868 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.848+0000 7efd7f604700 1 -- 172.21.15.39:0/817293471 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd780ff380 msgr2=0x7efd78198270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:12.868 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.848+0000 7efd7f604700 1 --2- 172.21.15.39:0/817293471 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd780ff380 0x7efd78198270 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7efd68000c00 tx=0x7efd6800f510 comp rx=0 tx=0).stop 2023-07-21T18:43:12.869 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.848+0000 7efd7f604700 1 -- 172.21.15.39:0/817293471 shutdown_connections 2023-07-21T18:43:12.869 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.848+0000 7efd7f604700 1 --2- 172.21.15.39:0/817293471 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7efd640a1430 0x7efd640a38e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:12.869 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.848+0000 7efd7f604700 1 --2- 172.21.15.39:0/817293471 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd781049e0 0x7efd7819c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:12.869 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.848+0000 7efd7f604700 1 --2- 172.21.15.39:0/817293471 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efd780fe9d0 0x7efd78197d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:12.869 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.848+0000 7efd7f604700 1 --2- 172.21.15.39:0/817293471 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd780ff380 0x7efd78198270 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:12.869 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.848+0000 7efd7f604700 1 -- 172.21.15.39:0/817293471 >> 172.21.15.39:0/817293471 conn(0x7efd780fa490 msgr2=0x7efd780fbe80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:12.869 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.848+0000 7efd7f604700 1 -- 172.21.15.39:0/817293471 shutdown_connections 2023-07-21T18:43:12.869 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.848+0000 7efd7f604700 1 -- 172.21.15.39:0/817293471 wait complete. 2023-07-21T18:43:12.870 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 28 2023-07-21T18:43:12.880 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-07-21T18:43:13.008 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.987+0000 7ffb2f58e700 1 -- 172.21.15.39:0/2869770575 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffb281049e0 msgr2=0x7ffb28104db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:13.008 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.987+0000 7ffb2f58e700 1 --2- 172.21.15.39:0/2869770575 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffb281049e0 0x7ffb28104db0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7ffb24009990 tx=0x7ffb24009ca0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.008 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.987+0000 7ffb2f58e700 1 -- 172.21.15.39:0/2869770575 shutdown_connections 2023-07-21T18:43:13.008 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.987+0000 7ffb2f58e700 1 --2- 172.21.15.39:0/2869770575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffb280ff380 0x7ffb2810b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.008 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.988+0000 7ffb2f58e700 1 --2- 172.21.15.39:0/2869770575 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffb281049e0 0x7ffb28104db0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.008 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.988+0000 7ffb2f58e700 1 --2- 172.21.15.39:0/2869770575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffb280fe9d0 0x7ffb280fee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.009 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.988+0000 7ffb2f58e700 1 -- 172.21.15.39:0/2869770575 >> 172.21.15.39:0/2869770575 conn(0x7ffb280fa4b0 msgr2=0x7ffb280fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:13.009 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.988+0000 7ffb2f58e700 1 -- 172.21.15.39:0/2869770575 shutdown_connections 2023-07-21T18:43:13.009 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.988+0000 7ffb2f58e700 1 -- 172.21.15.39:0/2869770575 wait complete. 2023-07-21T18:43:13.010 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.988+0000 7ffb2f58e700 1 Processor -- start 2023-07-21T18:43:13.011 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.988+0000 7ffb2f58e700 1 -- start start 2023-07-21T18:43:13.011 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.988+0000 7ffb2f58e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffb280fe9d0 0x7ffb281005e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.011 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.988+0000 7ffb2f58e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffb280ff380 0x7ffb28100b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.011 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.988+0000 7ffb2f58e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffb281049e0 0x7ffb281a3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.011 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.988+0000 7ffb2f58e700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb28101370 con 0x7ffb280ff380 2023-07-21T18:43:13.011 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.989+0000 7ffb2f58e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb281a42b0 con 0x7ffb281049e0 2023-07-21T18:43:13.011 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.989+0000 7ffb2f58e700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb281a43f0 con 0x7ffb280fe9d0 2023-07-21T18:43:13.011 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.989+0000 7ffb2cb29700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffb280ff380 0x7ffb28100b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.012 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.989+0000 7ffb2cb29700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffb280ff380 0x7ffb28100b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58518/0 (socket says 172.21.15.39:58518) 2023-07-21T18:43:13.012 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.989+0000 7ffb2cb29700 1 -- 172.21.15.39:0/3086986544 learned_addr learned my addr 172.21.15.39:0/3086986544 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:13.012 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.989+0000 7ffb2d32a700 1 --2- 172.21.15.39:0/3086986544 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffb280fe9d0 0x7ffb281005e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.012 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.989+0000 7ffb2cb29700 1 -- 172.21.15.39:0/3086986544 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffb280fe9d0 msgr2=0x7ffb281005e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:13.012 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.989+0000 7ffb2cb29700 1 --2- 172.21.15.39:0/3086986544 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffb280fe9d0 0x7ffb281005e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.012 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.989+0000 7ffb2cb29700 1 -- 172.21.15.39:0/3086986544 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffb281049e0 msgr2=0x7ffb281a3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:13.012 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.989+0000 7ffb2db2b700 1 --2- 172.21.15.39:0/3086986544 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffb281049e0 0x7ffb281a3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.013 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.989+0000 7ffb2cb29700 1 --2- 172.21.15.39:0/3086986544 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffb281049e0 0x7ffb281a3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.013 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.989+0000 7ffb2cb29700 1 -- 172.21.15.39:0/3086986544 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb240095f0 con 0x7ffb280ff380 2023-07-21T18:43:13.013 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.989+0000 7ffb2db2b700 1 --2- 172.21.15.39:0/3086986544 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffb281049e0 0x7ffb281a3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:43:13.013 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.989+0000 7ffb2cb29700 1 --2- 172.21.15.39:0/3086986544 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffb280ff380 0x7ffb28100b20 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7ffb1800c9e0 tx=0x7ffb1800cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:13.013 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.989+0000 7ffb1e7fc700 1 -- 172.21.15.39:0/3086986544 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb18007ac0 con 0x7ffb280ff380 2023-07-21T18:43:13.013 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.989+0000 7ffb2f58e700 1 -- 172.21.15.39:0/3086986544 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb281a4650 con 0x7ffb280ff380 2023-07-21T18:43:13.013 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.989+0000 7ffb1e7fc700 1 -- 172.21.15.39:0/3086986544 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb18007c20 con 0x7ffb280ff380 2023-07-21T18:43:13.013 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.989+0000 7ffb1e7fc700 1 -- 172.21.15.39:0/3086986544 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb18018660 con 0x7ffb280ff380 2023-07-21T18:43:13.014 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.989+0000 7ffb2f58e700 1 -- 172.21.15.39:0/3086986544 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb281a4b60 con 0x7ffb280ff380 2023-07-21T18:43:13.016 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.992+0000 7ffb1e7fc700 1 -- 172.21.15.39:0/3086986544 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7ffb1800f450 con 0x7ffb280ff380 2023-07-21T18:43:13.017 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.993+0000 7ffb1e7fc700 1 --2- 172.21.15.39:0/3086986544 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7ffb140a13e0 0x7ffb140a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.017 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.993+0000 7ffb2d32a700 1 --2- 172.21.15.39:0/3086986544 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7ffb140a13e0 0x7ffb140a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.017 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.993+0000 7ffb1e7fc700 1 -- 172.21.15.39:0/3086986544 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7ffb180ccfe0 con 0x7ffb280ff380 2023-07-21T18:43:13.017 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.993+0000 7ffb2d32a700 1 --2- 172.21.15.39:0/3086986544 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7ffb140a13e0 0x7ffb140a3890 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ffb240098e0 tx=0x7ffb24009510 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:13.017 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.993+0000 7ffb3b714b80 1 Processor -- start 2023-07-21T18:43:13.017 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.994+0000 7ffb3b714b80 1 -- start start 2023-07-21T18:43:13.017 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.994+0000 7ffb3b714b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7af7182d0 0x55c7af7186a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.018 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.994+0000 7ffb3b714b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7af718be0 0x55c7af723590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.018 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.994+0000 7ffb3b714b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c7af723ad0 0x55c7af725eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.018 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.994+0000 7ffb3b714b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7af61ecb0 con 0x55c7af718be0 2023-07-21T18:43:13.018 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.994+0000 7ffb3b714b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7af671a40 con 0x55c7af7182d0 2023-07-21T18:43:13.018 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.994+0000 7ffb3b714b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7af66aed0 con 0x55c7af723ad0 2023-07-21T18:43:13.018 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.994+0000 7ffb2cb29700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7af718be0 0x55c7af723590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.018 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.994+0000 7ffb2cb29700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7af718be0 0x55c7af723590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58526/0 (socket says 172.21.15.39:58526) 2023-07-21T18:43:13.018 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.994+0000 7ffb2cb29700 1 -- 172.21.15.39:0/1740070168 learned_addr learned my addr 172.21.15.39:0/1740070168 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:13.019 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.994+0000 7ffb2db2b700 1 --2- 172.21.15.39:0/1740070168 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c7af723ad0 0x55c7af725eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.019 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.994+0000 7ffb2d32a700 1 --2- 172.21.15.39:0/1740070168 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7af7182d0 0x55c7af7186a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.019 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.994+0000 7ffb2cb29700 1 -- 172.21.15.39:0/1740070168 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c7af723ad0 msgr2=0x55c7af725eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:13.019 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.994+0000 7ffb2cb29700 1 --2- 172.21.15.39:0/1740070168 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c7af723ad0 0x55c7af725eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.019 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.994+0000 7ffb2cb29700 1 -- 172.21.15.39:0/1740070168 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7af7182d0 msgr2=0x55c7af7186a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:13.019 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.994+0000 7ffb2cb29700 1 --2- 172.21.15.39:0/1740070168 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7af7182d0 0x55c7af7186a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.019 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.994+0000 7ffb2cb29700 1 -- 172.21.15.39:0/1740070168 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c7af498de0 con 0x55c7af718be0 2023-07-21T18:43:13.020 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.994+0000 7ffb2cb29700 1 --2- 172.21.15.39:0/1740070168 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7af718be0 0x55c7af723590 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7ffb18096340 tx=0x7ffb18099f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90afee5f59d82610 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:13.020 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.995+0000 7ffb137fe700 1 -- 172.21.15.39:0/1740070168 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb180981e0 con 0x55c7af718be0 2023-07-21T18:43:13.020 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.995+0000 7ffb137fe700 1 -- 172.21.15.39:0/1740070168 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb1809a0a0 con 0x55c7af718be0 2023-07-21T18:43:13.020 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.995+0000 7ffb137fe700 1 -- 172.21.15.39:0/1740070168 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb180a2060 con 0x55c7af718be0 2023-07-21T18:43:13.020 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.995+0000 7ffb3b714b80 1 -- 172.21.15.39:0/1740070168 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7af718be0 msgr2=0x55c7af723590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:13.020 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.995+0000 7ffb3b714b80 1 --2- 172.21.15.39:0/1740070168 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7af718be0 0x55c7af723590 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7ffb18096340 tx=0x7ffb18099f10 comp rx=0 tx=0).stop 2023-07-21T18:43:13.020 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.995+0000 7ffb3b714b80 1 -- 172.21.15.39:0/1740070168 shutdown_connections 2023-07-21T18:43:13.020 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.995+0000 7ffb3b714b80 1 --2- 172.21.15.39:0/1740070168 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7af7182d0 0x55c7af7186a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.021 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.995+0000 7ffb3b714b80 1 --2- 172.21.15.39:0/1740070168 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c7af723ad0 0x55c7af725eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.021 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.995+0000 7ffb3b714b80 1 --2- 172.21.15.39:0/1740070168 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7af718be0 0x55c7af723590 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.021 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.995+0000 7ffb3b714b80 1 -- 172.21.15.39:0/1740070168 >> 172.21.15.39:0/1740070168 conn(0x55c7af70eda0 msgr2=0x55c7af7146a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:13.021 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.995+0000 7ffb3b714b80 1 -- 172.21.15.39:0/1740070168 shutdown_connections 2023-07-21T18:43:13.021 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.996+0000 7ffb3b714b80 1 -- 172.21.15.39:0/1740070168 wait complete. 2023-07-21T18:43:13.021 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.996+0000 7ffb3b714b80 1 -- start start 2023-07-21T18:43:13.021 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.996+0000 7ffb3b714b80 1 Processor -- start 2023-07-21T18:43:13.021 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.996+0000 7ffb3b714b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7af7182d0 0x55c7af723570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.022 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.996+0000 7ffb3b714b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7af718be0 0x55c7af720860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.022 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.996+0000 7ffb3b714b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c7af723ad0 0x55c7af720fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.022 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.996+0000 7ffb3b714b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7af60aac0 con 0x55c7af7182d0 2023-07-21T18:43:13.022 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.996+0000 7ffb3b714b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7af60a7c0 con 0x55c7af718be0 2023-07-21T18:43:13.022 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.996+0000 7ffb3b714b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7af4ee920 con 0x55c7af723ad0 2023-07-21T18:43:13.022 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.996+0000 7ffb3b714b80 10 client.0 _ll_register_callbacks cb 0x55c7af608760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:43:13.022 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.996+0000 7ffb2d32a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7af7182d0 0x55c7af723570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.023 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.997+0000 7ffb2d32a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7af7182d0 0x55c7af723570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58530/0 (socket says 172.21.15.39:58530) 2023-07-21T18:43:13.023 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.997+0000 7ffb2d32a700 1 -- 172.21.15.39:0/1955335595 learned_addr learned my addr 172.21.15.39:0/1955335595 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:13.023 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.997+0000 7ffb2cb29700 1 --2- 172.21.15.39:0/1955335595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7af718be0 0x55c7af720860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.023 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.997+0000 7ffb2db2b700 1 --2- 172.21.15.39:0/1955335595 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c7af723ad0 0x55c7af720fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.023 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.997+0000 7ffb2d32a700 1 -- 172.21.15.39:0/1955335595 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c7af723ad0 msgr2=0x55c7af720fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:13.023 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.997+0000 7ffb2d32a700 1 --2- 172.21.15.39:0/1955335595 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c7af723ad0 0x55c7af720fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.023 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.997+0000 7ffb2d32a700 1 -- 172.21.15.39:0/1955335595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7af718be0 msgr2=0x55c7af720860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:13.023 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.997+0000 7ffb2d32a700 1 --2- 172.21.15.39:0/1955335595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7af718be0 0x55c7af720860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.024 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.997+0000 7ffb2d32a700 1 -- 172.21.15.39:0/1955335595 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb281a4b60 con 0x55c7af7182d0 2023-07-21T18:43:13.025 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.997+0000 7ffb2d32a700 1 --2- 172.21.15.39:0/1955335595 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7af7182d0 0x55c7af723570 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7ffb280ffb10 tx=0x7ffb24009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16c9a3d945220266 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:13.026 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.997+0000 7ffb10ff9700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:43:13.026 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.997+0000 7ffb10ff9700 1 -- 172.21.15.39:0/1955335595 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb2402b070 con 0x55c7af7182d0 2023-07-21T18:43:13.026 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.997+0000 7ffb10ff9700 1 -- 172.21.15.39:0/1955335595 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7ffb24005260 con 0x55c7af7182d0 2023-07-21T18:43:13.026 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.997+0000 7ffb3b714b80 1 -- 172.21.15.39:0/1955335595 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c7af5c44f0 con 0x55c7af7182d0 2023-07-21T18:43:13.026 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.997+0000 7ffb10ff9700 1 -- 172.21.15.39:0/1955335595 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb2401d070 con 0x55c7af7182d0 2023-07-21T18:43:13.026 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.997+0000 7ffb3b714b80 10 client.15984 fetch_fsmap learned FSMap version 28 2023-07-21T18:43:13.026 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.997+0000 7ffb3b714b80 1 -- 172.21.15.39:0/1955335595 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c7af481c60 con 0x55c7af7182d0 2023-07-21T18:43:13.027 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.998+0000 7ffb10ff9700 1 -- 172.21.15.39:0/1955335595 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c7af5c44f0 con 0x55c7af7182d0 2023-07-21T18:43:13.027 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.998+0000 7ffb10ff9700 1 -- 172.21.15.39:0/1955335595 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7ffb24030420 con 0x55c7af7182d0 2023-07-21T18:43:13.027 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.998+0000 7ffb3b714b80 10 client.15984 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T18:43:13.027 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.998+0000 7ffb3b714b80 10 client.15984 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2023-07-21T18:43:13.027 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.998+0000 7ffb3b714b80 20 client.15984 populate_metadata read hostname 'smithi039' 2023-07-21T18:43:13.027 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.998+0000 7ffb3b714b80 1 --2- 172.21.15.39:0/1955335595 >> [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] conn(0x55c7af733450 0x55c7af735860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.027 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.998+0000 7ffb3b714b80 4 client.15984 mds_command: new command op to 14604 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:43:13.028 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.998+0000 7ffb3b714b80 1 -- 172.21.15.39:0/1955335595 --> [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c7af663eb0 con 0x55c7af733450 2023-07-21T18:43:13.028 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.998+0000 7ffb2cb29700 1 --2- 172.21.15.39:0/1955335595 >> [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] conn(0x55c7af733450 0x55c7af735860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.028 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.998+0000 7ffb10ff9700 10 client.15984 ms_handle_connect on v2:172.21.15.39:6840/640981220 2023-07-21T18:43:13.028 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.998+0000 7ffb2cb29700 1 --2- 172.21.15.39:0/1955335595 >> [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] conn(0x55c7af733450 0x55c7af735860 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e0b81b6a2b42e348 server_cookie=d90d88f82b043077 in_seq=0 out_seq=0 2023-07-21T18:43:13.028 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.999+0000 7ffb10ff9700 1 -- 172.21.15.39:0/1955335595 <== mds.4 v2:172.21.15.39:6840/640981220 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c7af663eb0 con 0x55c7af733450 2023-07-21T18:43:13.028 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.999+0000 7ffb10ff9700 10 client.15984 handle_command_reply: tid=0 2023-07-21T18:43:13.028 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.999+0000 7ffb3b714b80 1 client.15984 shutdown 2023-07-21T18:43:13.028 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.999+0000 7ffb3b714b80 1 -- 172.21.15.39:0/1955335595 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7af7182d0 msgr2=0x55c7af723570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:13.029 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.999+0000 7ffb3b714b80 1 --2- 172.21.15.39:0/1955335595 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7af7182d0 0x55c7af723570 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7ffb280ffb10 tx=0x7ffb24009310 comp rx=0 tx=0).stop 2023-07-21T18:43:13.029 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.999+0000 7ffb3b714b80 1 -- 172.21.15.39:0/1955335595 shutdown_connections 2023-07-21T18:43:13.029 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.999+0000 7ffb3b714b80 1 --2- 172.21.15.39:0/1955335595 >> [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] conn(0x55c7af733450 0x55c7af735860 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.029 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.999+0000 7ffb3b714b80 1 --2- 172.21.15.39:0/1955335595 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c7af723ad0 0x55c7af720fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.029 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.999+0000 7ffb3b714b80 1 --2- 172.21.15.39:0/1955335595 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7af7182d0 0x55c7af723570 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.029 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.999+0000 7ffb10ff9700 0 client.15984 ms_handle_reset on v2:172.21.15.39:6840/640981220 2023-07-21T18:43:13.029 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.999+0000 7ffb3b714b80 1 --2- 172.21.15.39:0/1955335595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7af718be0 0x55c7af720860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.029 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.999+0000 7ffb3b714b80 1 -- 172.21.15.39:0/1955335595 >> 172.21.15.39:0/1955335595 conn(0x55c7af70eda0 msgr2=0x55c7af714230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:13.030 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.999+0000 7ffb3b714b80 1 -- 172.21.15.39:0/1955335595 shutdown_connections 2023-07-21T18:43:13.030 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.999+0000 7ffb3b714b80 1 -- 172.21.15.39:0/1955335595 wait complete. 2023-07-21T18:43:13.030 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:12.999+0000 7ffb3b714b80 20 client.15984 trim_cache size 0 max 16384 2023-07-21T18:43:13.033 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.012+0000 7ffb3b714b80 1 Processor -- start 2023-07-21T18:43:13.033 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.012+0000 7ffb3b714b80 1 -- start start 2023-07-21T18:43:13.033 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.012+0000 7ffb3b714b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c7af7182d0 0x55c7af725eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.033 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.012+0000 7ffb3b714b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7af718be0 0x55c7af723110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.033 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.012+0000 7ffb3b714b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7af723ad0 0x55c7af71bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.033 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.013+0000 7ffb3b714b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7af71edc0 con 0x55c7af723ad0 2023-07-21T18:43:13.033 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.013+0000 7ffb3b714b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7af7188f0 con 0x55c7af718be0 2023-07-21T18:43:13.034 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.013+0000 7ffb3b714b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7af735a80 con 0x55c7af7182d0 2023-07-21T18:43:13.034 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.013+0000 7ffb2db2b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7af723ad0 0x55c7af71bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.034 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.013+0000 7ffb2db2b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7af723ad0 0x55c7af71bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58542/0 (socket says 172.21.15.39:58542) 2023-07-21T18:43:13.034 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.013+0000 7ffb2db2b700 1 -- 172.21.15.39:0/2513246505 learned_addr learned my addr 172.21.15.39:0/2513246505 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:13.034 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.013+0000 7ffb2d32a700 1 --2- 172.21.15.39:0/2513246505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c7af7182d0 0x55c7af725eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.034 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.013+0000 7ffb2cb29700 1 --2- 172.21.15.39:0/2513246505 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7af718be0 0x55c7af723110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.034 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.013+0000 7ffb2db2b700 1 -- 172.21.15.39:0/2513246505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c7af7182d0 msgr2=0x55c7af725eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:13.035 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.013+0000 7ffb2db2b700 1 --2- 172.21.15.39:0/2513246505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c7af7182d0 0x55c7af725eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.035 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.013+0000 7ffb2db2b700 1 -- 172.21.15.39:0/2513246505 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7af718be0 msgr2=0x55c7af723110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:13.035 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.013+0000 7ffb2db2b700 1 --2- 172.21.15.39:0/2513246505 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7af718be0 0x55c7af723110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.035 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.013+0000 7ffb2db2b700 1 -- 172.21.15.39:0/2513246505 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c7af4bf250 con 0x55c7af723ad0 2023-07-21T18:43:13.035 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.013+0000 7ffb2d32a700 1 --2- 172.21.15.39:0/2513246505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c7af7182d0 0x55c7af725eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:43:13.035 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.013+0000 7ffb2db2b700 1 --2- 172.21.15.39:0/2513246505 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7af723ad0 0x55c7af71bb70 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7ffb20009d10 tx=0x7ffb20009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8d55f391d6abde8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:13.038 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.013+0000 7ffb13fff700 1 -- 172.21.15.39:0/2513246505 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb200099f0 con 0x55c7af723ad0 2023-07-21T18:43:13.038 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.013+0000 7ffb13fff700 1 -- 172.21.15.39:0/2513246505 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb20017930 con 0x55c7af723ad0 2023-07-21T18:43:13.039 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.013+0000 7ffb13fff700 1 -- 172.21.15.39:0/2513246505 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb200169e0 con 0x55c7af723ad0 2023-07-21T18:43:13.039 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.014+0000 7ffb3b714b80 1 -- 172.21.15.39:0/2513246505 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7af723ad0 msgr2=0x55c7af71bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:13.039 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.014+0000 7ffb3b714b80 1 --2- 172.21.15.39:0/2513246505 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7af723ad0 0x55c7af71bb70 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7ffb20009d10 tx=0x7ffb20009040 comp rx=0 tx=0).stop 2023-07-21T18:43:13.039 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.014+0000 7ffb3b714b80 1 -- 172.21.15.39:0/2513246505 shutdown_connections 2023-07-21T18:43:13.039 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.014+0000 7ffb3b714b80 1 --2- 172.21.15.39:0/2513246505 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7af723ad0 0x55c7af71bb70 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.039 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.014+0000 7ffb3b714b80 1 --2- 172.21.15.39:0/2513246505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c7af7182d0 0x55c7af725eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.039 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.014+0000 7ffb3b714b80 1 --2- 172.21.15.39:0/2513246505 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7af718be0 0x55c7af723110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.039 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.014+0000 7ffb3b714b80 1 -- 172.21.15.39:0/2513246505 >> 172.21.15.39:0/2513246505 conn(0x55c7af70eda0 msgr2=0x55c7af72f780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:13.040 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.014+0000 7ffb3b714b80 1 -- 172.21.15.39:0/2513246505 shutdown_connections 2023-07-21T18:43:13.040 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.014+0000 7ffb3b714b80 1 -- 172.21.15.39:0/2513246505 wait complete. 2023-07-21T18:43:13.040 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.014+0000 7ffb3b714b80 1 -- start start 2023-07-21T18:43:13.040 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.014+0000 7ffb3b714b80 1 Processor -- start 2023-07-21T18:43:13.040 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.015+0000 7ffb3b714b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c7af7182d0 0x55c7af72a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.040 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.015+0000 7ffb3b714b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7af718be0 0x55c7af72a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.040 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.015+0000 7ffb3b714b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7af723ad0 0x55c7af728710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.041 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.015+0000 7ffb3b714b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7af728e20 con 0x55c7af723ad0 2023-07-21T18:43:13.041 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.015+0000 7ffb3b714b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7af728f90 con 0x55c7af718be0 2023-07-21T18:43:13.041 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.015+0000 7ffb3b714b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7af729150 con 0x55c7af7182d0 2023-07-21T18:43:13.041 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.015+0000 7ffb3b714b80 10 client.0 _ll_register_callbacks cb 0x55c7af608760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:43:13.041 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.015+0000 7ffb2db2b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7af723ad0 0x55c7af728710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.041 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.015+0000 7ffb2db2b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7af723ad0 0x55c7af728710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58548/0 (socket says 172.21.15.39:58548) 2023-07-21T18:43:13.041 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.015+0000 7ffb2db2b700 1 -- 172.21.15.39:0/3879080958 learned_addr learned my addr 172.21.15.39:0/3879080958 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:13.041 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.015+0000 7ffb2cb29700 1 --2- 172.21.15.39:0/3879080958 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7af718be0 0x55c7af72a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.042 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.015+0000 7ffb2d32a700 1 --2- 172.21.15.39:0/3879080958 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c7af7182d0 0x55c7af72a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.042 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.015+0000 7ffb2db2b700 1 -- 172.21.15.39:0/3879080958 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c7af7182d0 msgr2=0x55c7af72a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:13.042 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.015+0000 7ffb2db2b700 1 --2- 172.21.15.39:0/3879080958 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c7af7182d0 0x55c7af72a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.042 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.015+0000 7ffb2db2b700 1 -- 172.21.15.39:0/3879080958 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7af718be0 msgr2=0x55c7af72a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:13.042 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.015+0000 7ffb2db2b700 1 --2- 172.21.15.39:0/3879080958 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7af718be0 0x55c7af72a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.042 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.015+0000 7ffb2db2b700 1 -- 172.21.15.39:0/3879080958 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb20011040 con 0x55c7af723ad0 2023-07-21T18:43:13.042 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.015+0000 7ffb2db2b700 1 --2- 172.21.15.39:0/3879080958 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7af723ad0 0x55c7af728710 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7ffb20016480 tx=0x7ffb200177d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77b659caeaaa98b8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:13.042 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.015+0000 7ffb10ff9700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:43:13.043 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.016+0000 7ffb10ff9700 1 -- 172.21.15.39:0/3879080958 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb20004ce0 con 0x55c7af723ad0 2023-07-21T18:43:13.043 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.016+0000 7ffb10ff9700 1 -- 172.21.15.39:0/3879080958 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7ffb20004e40 con 0x55c7af723ad0 2023-07-21T18:43:13.043 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.016+0000 7ffb3b714b80 1 -- 172.21.15.39:0/3879080958 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffb1809a0a0 con 0x55c7af723ad0 2023-07-21T18:43:13.043 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.016+0000 7ffb10ff9700 1 -- 172.21.15.39:0/3879080958 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb20005070 con 0x55c7af723ad0 2023-07-21T18:43:13.043 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.016+0000 7ffb3b714b80 10 client.15990 fetch_fsmap learned FSMap version 28 2023-07-21T18:43:13.043 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.016+0000 7ffb3b714b80 1 -- 172.21.15.39:0/3879080958 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c7af4972b0 con 0x55c7af723ad0 2023-07-21T18:43:13.043 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.016+0000 7ffb10ff9700 1 -- 172.21.15.39:0/3879080958 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb1809a0a0 con 0x55c7af723ad0 2023-07-21T18:43:13.044 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.016+0000 7ffb10ff9700 1 -- 172.21.15.39:0/3879080958 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7ffb20016840 con 0x55c7af723ad0 2023-07-21T18:43:13.044 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.016+0000 7ffb3b714b80 10 client.15990 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T18:43:13.044 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.016+0000 7ffb3b714b80 10 client.15990 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2023-07-21T18:43:13.044 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.016+0000 7ffb3b714b80 20 client.15990 populate_metadata read hostname 'smithi039' 2023-07-21T18:43:13.044 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.016+0000 7ffb3b714b80 1 --2- 172.21.15.39:0/3879080958 >> [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] conn(0x55c7af7884a0 0x55c7af78a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.044 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.016+0000 7ffb3b714b80 4 client.15990 mds_command: new command op to 14604 tid=0[{"prefix": "damage ls"}] 2023-07-21T18:43:13.044 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.016+0000 7ffb3b714b80 1 -- 172.21.15.39:0/3879080958 --> [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c7af626050 con 0x55c7af7884a0 2023-07-21T18:43:13.045 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.016+0000 7ffb2d32a700 1 --2- 172.21.15.39:0/3879080958 >> [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] conn(0x55c7af7884a0 0x55c7af78a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.045 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.017+0000 7ffb2d32a700 1 --2- 172.21.15.39:0/3879080958 >> [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] conn(0x55c7af7884a0 0x55c7af78a8b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4f43ad590b45fea1 server_cookie=b5bb8e0f81ebc695 in_seq=0 out_seq=0 2023-07-21T18:43:13.045 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.017+0000 7ffb10ff9700 10 client.15990 ms_handle_connect on v2:172.21.15.39:6840/640981220 2023-07-21T18:43:13.045 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.017+0000 7ffb10ff9700 1 -- 172.21.15.39:0/3879080958 <== mds.4 v2:172.21.15.39:6840/640981220 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c7af626050 con 0x55c7af7884a0 2023-07-21T18:43:13.045 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.017+0000 7ffb10ff9700 10 client.15990 handle_command_reply: tid=0 2023-07-21T18:43:13.045 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.017+0000 7ffb3b714b80 1 client.15990 shutdown 2023-07-21T18:43:13.045 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.017+0000 7ffb3b714b80 1 -- 172.21.15.39:0/3879080958 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7af723ad0 msgr2=0x55c7af728710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:13.045 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.017+0000 7ffb3b714b80 1 --2- 172.21.15.39:0/3879080958 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7af723ad0 0x55c7af728710 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7ffb20016480 tx=0x7ffb200177d0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.046 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.017+0000 7ffb3b714b80 1 -- 172.21.15.39:0/3879080958 shutdown_connections 2023-07-21T18:43:13.046 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.017+0000 7ffb3b714b80 1 --2- 172.21.15.39:0/3879080958 >> [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] conn(0x55c7af7884a0 0x55c7af78a8b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.046 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.017+0000 7ffb3b714b80 1 --2- 172.21.15.39:0/3879080958 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7af723ad0 0x55c7af728710 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.046 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.017+0000 7ffb10ff9700 0 client.15990 ms_handle_reset on v2:172.21.15.39:6840/640981220 2023-07-21T18:43:13.046 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.017+0000 7ffb3b714b80 1 --2- 172.21.15.39:0/3879080958 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c7af7182d0 0x55c7af72a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.046 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.017+0000 7ffb3b714b80 1 --2- 172.21.15.39:0/3879080958 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7af718be0 0x55c7af72a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.046 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.017+0000 7ffb3b714b80 1 -- 172.21.15.39:0/3879080958 >> 172.21.15.39:0/3879080958 conn(0x55c7af70eda0 msgr2=0x55c7af71bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:13.046 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.017+0000 7ffb3b714b80 1 -- 172.21.15.39:0/3879080958 shutdown_connections 2023-07-21T18:43:13.047 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.017+0000 7ffb3b714b80 1 -- 172.21.15.39:0/3879080958 wait complete. 2023-07-21T18:43:13.047 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.017+0000 7ffb3b714b80 20 client.15990 trim_cache size 0 max 16384 2023-07-21T18:43:13.047 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2023-07-21T18:43:13.048 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.018+0000 7ffb2f58e700 1 -- 172.21.15.39:0/3086986544 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7ffb140a13e0 msgr2=0x7ffb140a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:13.048 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.018+0000 7ffb2f58e700 1 --2- 172.21.15.39:0/3086986544 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7ffb140a13e0 0x7ffb140a3890 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ffb240098e0 tx=0x7ffb24009510 comp rx=0 tx=0).stop 2023-07-21T18:43:13.048 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.018+0000 7ffb2f58e700 1 -- 172.21.15.39:0/3086986544 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffb280ff380 msgr2=0x7ffb28100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:13.048 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.018+0000 7ffb2f58e700 1 --2- 172.21.15.39:0/3086986544 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffb280ff380 0x7ffb28100b20 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7ffb1800c9e0 tx=0x7ffb1800cda0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.048 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.018+0000 7ffb2f58e700 1 -- 172.21.15.39:0/3086986544 shutdown_connections 2023-07-21T18:43:13.049 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.018+0000 7ffb2f58e700 1 --2- 172.21.15.39:0/3086986544 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7ffb140a13e0 0x7ffb140a3890 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.049 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.018+0000 7ffb2f58e700 1 --2- 172.21.15.39:0/3086986544 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffb281049e0 0x7ffb281a3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.049 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.018+0000 7ffb2f58e700 1 --2- 172.21.15.39:0/3086986544 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffb280fe9d0 0x7ffb281005e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.049 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.018+0000 7ffb2f58e700 1 --2- 172.21.15.39:0/3086986544 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffb280ff380 0x7ffb28100b20 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.049 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.018+0000 7ffb2f58e700 1 -- 172.21.15.39:0/3086986544 >> 172.21.15.39:0/3086986544 conn(0x7ffb280fa4b0 msgr2=0x7ffb280fbf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:13.049 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.018+0000 7ffb2f58e700 1 -- 172.21.15.39:0/3086986544 shutdown_connections 2023-07-21T18:43:13.049 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.018+0000 7ffb2f58e700 1 -- 172.21.15.39:0/3086986544 wait complete. 2023-07-21T18:43:13.050 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-07-21T18:43:13.179 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.159+0000 7f3c60757700 1 -- 172.21.15.39:0/3480953543 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c580fe9d0 msgr2=0x7f3c580fee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:13.179 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.159+0000 7f3c60757700 1 --2- 172.21.15.39:0/3480953543 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c580fe9d0 0x7f3c580fee40 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f3c48009990 tx=0x7f3c48009ca0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.179 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.159+0000 7f3c60757700 1 -- 172.21.15.39:0/3480953543 shutdown_connections 2023-07-21T18:43:13.179 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.159+0000 7f3c60757700 1 --2- 172.21.15.39:0/3480953543 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c580ff380 0x7f3c5810b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.179 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.159+0000 7f3c60757700 1 --2- 172.21.15.39:0/3480953543 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c581049e0 0x7f3c58104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.180 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.159+0000 7f3c60757700 1 --2- 172.21.15.39:0/3480953543 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c580fe9d0 0x7f3c580fee40 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.180 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.159+0000 7f3c60757700 1 -- 172.21.15.39:0/3480953543 >> 172.21.15.39:0/3480953543 conn(0x7f3c580fa4f0 msgr2=0x7f3c580fc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:13.180 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.159+0000 7f3c60757700 1 -- 172.21.15.39:0/3480953543 shutdown_connections 2023-07-21T18:43:13.180 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.159+0000 7f3c60757700 1 -- 172.21.15.39:0/3480953543 wait complete. 2023-07-21T18:43:13.182 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.159+0000 7f3c60757700 1 Processor -- start 2023-07-21T18:43:13.182 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.159+0000 7f3c60757700 1 -- start start 2023-07-21T18:43:13.182 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.160+0000 7f3c60757700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c580fe9d0 0x7f3c5819a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.182 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.160+0000 7f3c60757700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c580ff380 0x7f3c5819ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.182 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.160+0000 7f3c60757700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c581049e0 0x7f3c581948c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.182 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.160+0000 7f3c60757700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c58194f30 con 0x7f3c581049e0 2023-07-21T18:43:13.182 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.160+0000 7f3c60757700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c581950a0 con 0x7f3c580fe9d0 2023-07-21T18:43:13.183 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.160+0000 7f3c60757700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c58195260 con 0x7f3c580ff380 2023-07-21T18:43:13.183 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.160+0000 7f3c5ecf4700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c581049e0 0x7f3c581948c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.183 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.160+0000 7f3c5ecf4700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c581049e0 0x7f3c581948c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58560/0 (socket says 172.21.15.39:58560) 2023-07-21T18:43:13.183 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.160+0000 7f3c5ecf4700 1 -- 172.21.15.39:0/1918232917 learned_addr learned my addr 172.21.15.39:0/1918232917 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:13.183 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.160+0000 7f3c5e4f3700 1 --2- 172.21.15.39:0/1918232917 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c580fe9d0 0x7f3c5819a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.183 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.160+0000 7f3c5dcf2700 1 --2- 172.21.15.39:0/1918232917 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c580ff380 0x7f3c5819ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.183 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.160+0000 7f3c5ecf4700 1 -- 172.21.15.39:0/1918232917 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c580ff380 msgr2=0x7f3c5819ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:13.184 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.160+0000 7f3c5ecf4700 1 --2- 172.21.15.39:0/1918232917 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c580ff380 0x7f3c5819ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.184 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.160+0000 7f3c5ecf4700 1 -- 172.21.15.39:0/1918232917 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c580fe9d0 msgr2=0x7f3c5819a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:13.184 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.160+0000 7f3c5ecf4700 1 --2- 172.21.15.39:0/1918232917 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c580fe9d0 0x7f3c5819a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.184 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.160+0000 7f3c5ecf4700 1 -- 172.21.15.39:0/1918232917 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c480095f0 con 0x7f3c581049e0 2023-07-21T18:43:13.184 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.160+0000 7f3c5dcf2700 1 --2- 172.21.15.39:0/1918232917 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c580ff380 0x7f3c5819ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:43:13.184 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.160+0000 7f3c5ecf4700 1 --2- 172.21.15.39:0/1918232917 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c581049e0 0x7f3c581948c0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f3c50007f10 tx=0x7f3c5000f5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:13.184 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.160+0000 7f3c4f7fe700 1 -- 172.21.15.39:0/1918232917 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c5000fde0 con 0x7f3c581049e0 2023-07-21T18:43:13.184 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.160+0000 7f3c60757700 1 -- 172.21.15.39:0/1918232917 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c58195540 con 0x7f3c581049e0 2023-07-21T18:43:13.185 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.160+0000 7f3c4f7fe700 1 -- 172.21.15.39:0/1918232917 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c50004bb0 con 0x7f3c581049e0 2023-07-21T18:43:13.185 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.161+0000 7f3c4f7fe700 1 -- 172.21.15.39:0/1918232917 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c50005020 con 0x7f3c581049e0 2023-07-21T18:43:13.185 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.161+0000 7f3c60757700 1 -- 172.21.15.39:0/1918232917 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c58068a00 con 0x7f3c581049e0 2023-07-21T18:43:13.189 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.164+0000 7f3c6c8ddb80 1 Processor -- start 2023-07-21T18:43:13.189 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.164+0000 7f3c6c8ddb80 1 -- start start 2023-07-21T18:43:13.189 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.165+0000 7f3c6c8ddb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f5cd8ea2d0 0x55f5cd8ea6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.189 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.165+0000 7f3c6c8ddb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5cd8eabe0 0x55f5cd8f5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.189 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.165+0000 7f3c6c8ddb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f5cd8f5ad0 0x55f5cd8f7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.189 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.165+0000 7f3c6c8ddb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5cd7f0cb0 con 0x55f5cd8f5ad0 2023-07-21T18:43:13.190 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.165+0000 7f3c6c8ddb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5cd843a40 con 0x55f5cd8eabe0 2023-07-21T18:43:13.190 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.165+0000 7f3c6c8ddb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5cd83ced0 con 0x55f5cd8ea2d0 2023-07-21T18:43:13.190 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.165+0000 7f3c5dcf2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f5cd8f5ad0 0x55f5cd8f7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.190 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.165+0000 7f3c5dcf2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f5cd8f5ad0 0x55f5cd8f7eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58566/0 (socket says 172.21.15.39:58566) 2023-07-21T18:43:13.190 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.165+0000 7f3c5dcf2700 1 -- 172.21.15.39:0/1470904945 learned_addr learned my addr 172.21.15.39:0/1470904945 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:13.190 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.165+0000 7f3c5e4f3700 1 --2- 172.21.15.39:0/1470904945 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5cd8eabe0 0x55f5cd8f5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.190 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.165+0000 7f3c5dcf2700 1 --2- 172.21.15.39:0/1470904945 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f5cd8ea2d0 0x55f5cd8ea6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.191 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.165+0000 7f3c5dcf2700 1 -- 172.21.15.39:0/1470904945 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f5cd8ea2d0 msgr2=0x55f5cd8ea6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:13.191 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.165+0000 7f3c5dcf2700 1 --2- 172.21.15.39:0/1470904945 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f5cd8ea2d0 0x55f5cd8ea6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.191 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.165+0000 7f3c5dcf2700 1 -- 172.21.15.39:0/1470904945 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5cd8eabe0 msgr2=0x55f5cd8f5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:13.191 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.165+0000 7f3c5dcf2700 1 --2- 172.21.15.39:0/1470904945 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5cd8eabe0 0x55f5cd8f5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.191 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.165+0000 7f3c5dcf2700 1 -- 172.21.15.39:0/1470904945 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5cd66ade0 con 0x55f5cd8f5ad0 2023-07-21T18:43:13.191 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.165+0000 7f3c5dcf2700 1 --2- 172.21.15.39:0/1470904945 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f5cd8f5ad0 0x55f5cd8f7eb0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f3c48006010 tx=0x7f3c4801a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e7ee23b1e73f3c3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:13.191 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.166+0000 7f3c4cff9700 1 -- 172.21.15.39:0/1470904945 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c4800b7d0 con 0x55f5cd8f5ad0 2023-07-21T18:43:13.191 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.166+0000 7f3c4cff9700 1 -- 172.21.15.39:0/1470904945 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c4801e030 con 0x55f5cd8f5ad0 2023-07-21T18:43:13.192 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.166+0000 7f3c4cff9700 1 -- 172.21.15.39:0/1470904945 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c48005580 con 0x55f5cd8f5ad0 2023-07-21T18:43:13.192 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.166+0000 7f3c4f7fe700 1 -- 172.21.15.39:0/1918232917 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f3c50004d20 con 0x7f3c581049e0 2023-07-21T18:43:13.192 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.166+0000 7f3c4f7fe700 1 --2- 172.21.15.39:0/1918232917 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f3c440a13e0 0x7f3c440a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.192 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.166+0000 7f3c6c8ddb80 1 -- 172.21.15.39:0/1470904945 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f5cd8f5ad0 msgr2=0x55f5cd8f7eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:13.192 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.166+0000 7f3c6c8ddb80 1 --2- 172.21.15.39:0/1470904945 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f5cd8f5ad0 0x55f5cd8f7eb0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f3c48006010 tx=0x7f3c4801a040 comp rx=0 tx=0).stop 2023-07-21T18:43:13.192 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.166+0000 7f3c5e4f3700 1 --2- 172.21.15.39:0/1918232917 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f3c440a13e0 0x7f3c440a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.192 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.166+0000 7f3c4f7fe700 1 -- 172.21.15.39:0/1918232917 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f3c500cf240 con 0x7f3c581049e0 2023-07-21T18:43:13.192 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.166+0000 7f3c6c8ddb80 1 -- 172.21.15.39:0/1470904945 shutdown_connections 2023-07-21T18:43:13.193 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.166+0000 7f3c6c8ddb80 1 --2- 172.21.15.39:0/1470904945 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f5cd8f5ad0 0x55f5cd8f7eb0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.193 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.166+0000 7f3c6c8ddb80 1 --2- 172.21.15.39:0/1470904945 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f5cd8ea2d0 0x55f5cd8ea6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.193 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.166+0000 7f3c6c8ddb80 1 --2- 172.21.15.39:0/1470904945 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5cd8eabe0 0x55f5cd8f5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.193 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.166+0000 7f3c6c8ddb80 1 -- 172.21.15.39:0/1470904945 >> 172.21.15.39:0/1470904945 conn(0x55f5cd8e0da0 msgr2=0x55f5cd8e66a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:13.193 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.167+0000 7f3c6c8ddb80 1 -- 172.21.15.39:0/1470904945 shutdown_connections 2023-07-21T18:43:13.193 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.167+0000 7f3c6c8ddb80 1 -- 172.21.15.39:0/1470904945 wait complete. 2023-07-21T18:43:13.193 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.167+0000 7f3c5e4f3700 1 --2- 172.21.15.39:0/1918232917 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f3c440a13e0 0x7f3c440a3890 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3c580ffb10 tx=0x7f3c5400b410 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:13.193 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.167+0000 7f3c6c8ddb80 1 -- start start 2023-07-21T18:43:13.194 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.167+0000 7f3c6c8ddb80 1 Processor -- start 2023-07-21T18:43:13.194 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.167+0000 7f3c6c8ddb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f5cd8ea2d0 0x55f5cd8f5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.194 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.167+0000 7f3c6c8ddb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5cd8eabe0 0x55f5cd8f2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.194 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.167+0000 7f3c6c8ddb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f5cd8f5ad0 0x55f5cd8f2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.194 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.167+0000 7f3c6c8ddb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5cd7dcac0 con 0x55f5cd8ea2d0 2023-07-21T18:43:13.194 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.167+0000 7f3c6c8ddb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5cd7dc7c0 con 0x55f5cd8eabe0 2023-07-21T18:43:13.194 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.167+0000 7f3c6c8ddb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5cd6c0920 con 0x55f5cd8f5ad0 2023-07-21T18:43:13.195 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.167+0000 7f3c5e4f3700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f5cd8ea2d0 0x55f5cd8f5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.195 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.167+0000 7f3c5e4f3700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f5cd8ea2d0 0x55f5cd8f5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58570/0 (socket says 172.21.15.39:58570) 2023-07-21T18:43:13.195 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.167+0000 7f3c6c8ddb80 10 client.0 _ll_register_callbacks cb 0x55f5cd7da760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:43:13.195 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.167+0000 7f3c5e4f3700 1 -- 172.21.15.39:0/754541348 learned_addr learned my addr 172.21.15.39:0/754541348 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:13.195 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.168+0000 7f3c5dcf2700 1 --2- 172.21.15.39:0/754541348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5cd8eabe0 0x55f5cd8f2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.195 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.168+0000 7f3c5ecf4700 1 --2- 172.21.15.39:0/754541348 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f5cd8f5ad0 0x55f5cd8f2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.195 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.168+0000 7f3c5e4f3700 1 -- 172.21.15.39:0/754541348 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f5cd8f5ad0 msgr2=0x55f5cd8f2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:13.195 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.168+0000 7f3c5e4f3700 1 --2- 172.21.15.39:0/754541348 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f5cd8f5ad0 0x55f5cd8f2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.196 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.168+0000 7f3c5e4f3700 1 -- 172.21.15.39:0/754541348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5cd8eabe0 msgr2=0x55f5cd8f2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:13.196 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.168+0000 7f3c5e4f3700 1 --2- 172.21.15.39:0/754541348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5cd8eabe0 0x55f5cd8f2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.196 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.168+0000 7f3c5e4f3700 1 -- 172.21.15.39:0/754541348 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c5810c110 con 0x55f5cd8ea2d0 2023-07-21T18:43:13.196 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.168+0000 7f3c5e4f3700 1 --2- 172.21.15.39:0/754541348 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f5cd8ea2d0 0x55f5cd8f5520 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f3c54011de0 tx=0x7f3c540047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c32bc1d2ced81708 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:13.196 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.168+0000 7f3c41ffb700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:43:13.196 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.168+0000 7f3c41ffb700 1 -- 172.21.15.39:0/754541348 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c54016070 con 0x55f5cd8ea2d0 2023-07-21T18:43:13.196 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.168+0000 7f3c41ffb700 1 -- 172.21.15.39:0/754541348 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f3c54027410 con 0x55f5cd8ea2d0 2023-07-21T18:43:13.196 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.168+0000 7f3c6c8ddb80 1 -- 172.21.15.39:0/754541348 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f5cd7964f0 con 0x55f5cd8ea2d0 2023-07-21T18:43:13.197 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.168+0000 7f3c41ffb700 1 -- 172.21.15.39:0/754541348 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c54011ec0 con 0x55f5cd8ea2d0 2023-07-21T18:43:13.197 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.168+0000 7f3c6c8ddb80 10 client.16002 fetch_fsmap learned FSMap version 28 2023-07-21T18:43:13.197 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.168+0000 7f3c6c8ddb80 1 -- 172.21.15.39:0/754541348 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55f5cd653c60 con 0x55f5cd8ea2d0 2023-07-21T18:43:13.197 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.168+0000 7f3c41ffb700 1 -- 172.21.15.39:0/754541348 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c54034020 con 0x55f5cd8ea2d0 2023-07-21T18:43:13.197 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.169+0000 7f3c41ffb700 1 -- 172.21.15.39:0/754541348 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f3c540104c0 con 0x55f5cd8ea2d0 2023-07-21T18:43:13.197 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.169+0000 7f3c6c8ddb80 10 client.16002 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T18:43:13.197 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.169+0000 7f3c6c8ddb80 10 client.16002 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2023-07-21T18:43:13.197 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.169+0000 7f3c6c8ddb80 20 client.16002 populate_metadata read hostname 'smithi039' 2023-07-21T18:43:13.198 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.169+0000 7f3c6c8ddb80 1 --2- 172.21.15.39:0/754541348 >> [v2:172.21.15.155:6834/3854856286,v1:172.21.15.155:6835/3854856286] conn(0x55f5cd9054b0 0x55f5cd9078c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.198 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.169+0000 7f3c6c8ddb80 4 client.16002 mds_command: new command op to 24464 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:43:13.198 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.169+0000 7f3c6c8ddb80 1 -- 172.21.15.39:0/754541348 --> [v2:172.21.15.155:6834/3854856286,v1:172.21.15.155:6835/3854856286] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f5cd835eb0 con 0x55f5cd9054b0 2023-07-21T18:43:13.198 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.169+0000 7f3c5dcf2700 1 --2- 172.21.15.39:0/754541348 >> [v2:172.21.15.155:6834/3854856286,v1:172.21.15.155:6835/3854856286] conn(0x55f5cd9054b0 0x55f5cd9078c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.199 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.169+0000 7f3c5dcf2700 1 --2- 172.21.15.39:0/754541348 >> [v2:172.21.15.155:6834/3854856286,v1:172.21.15.155:6835/3854856286] conn(0x55f5cd9054b0 0x55f5cd9078c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=857a324c9d120412 server_cookie=caa892215784c28d in_seq=0 out_seq=0 2023-07-21T18:43:13.199 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.169+0000 7f3c41ffb700 10 client.16002 ms_handle_connect on v2:172.21.15.155:6834/3854856286 2023-07-21T18:43:13.199 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.170+0000 7f3c41ffb700 1 -- 172.21.15.39:0/754541348 <== mds.3 v2:172.21.15.155:6834/3854856286 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f5cd835eb0 con 0x55f5cd9054b0 2023-07-21T18:43:13.199 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.170+0000 7f3c41ffb700 10 client.16002 handle_command_reply: tid=0 2023-07-21T18:43:13.200 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.170+0000 7f3c6c8ddb80 1 client.16002 shutdown 2023-07-21T18:43:13.200 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.170+0000 7f3c6c8ddb80 1 -- 172.21.15.39:0/754541348 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f5cd8ea2d0 msgr2=0x55f5cd8f5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:13.200 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.170+0000 7f3c6c8ddb80 1 --2- 172.21.15.39:0/754541348 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f5cd8ea2d0 0x55f5cd8f5520 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f3c54011de0 tx=0x7f3c540047c0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.200 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.170+0000 7f3c6c8ddb80 1 -- 172.21.15.39:0/754541348 shutdown_connections 2023-07-21T18:43:13.200 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.170+0000 7f3c6c8ddb80 1 --2- 172.21.15.39:0/754541348 >> [v2:172.21.15.155:6834/3854856286,v1:172.21.15.155:6835/3854856286] conn(0x55f5cd9054b0 0x55f5cd9078c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.200 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.170+0000 7f3c6c8ddb80 1 --2- 172.21.15.39:0/754541348 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f5cd8f5ad0 0x55f5cd8f2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.200 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.170+0000 7f3c41ffb700 0 client.16002 ms_handle_reset on v2:172.21.15.155:6834/3854856286 2023-07-21T18:43:13.200 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.170+0000 7f3c6c8ddb80 1 --2- 172.21.15.39:0/754541348 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f5cd8ea2d0 0x55f5cd8f5520 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.201 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.170+0000 7f3c6c8ddb80 1 --2- 172.21.15.39:0/754541348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5cd8eabe0 0x55f5cd8f2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.201 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.170+0000 7f3c6c8ddb80 1 -- 172.21.15.39:0/754541348 >> 172.21.15.39:0/754541348 conn(0x55f5cd8e0da0 msgr2=0x55f5cd8e6290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:13.201 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.170+0000 7f3c6c8ddb80 1 -- 172.21.15.39:0/754541348 shutdown_connections 2023-07-21T18:43:13.201 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.170+0000 7f3c6c8ddb80 1 -- 172.21.15.39:0/754541348 wait complete. 2023-07-21T18:43:13.201 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.171+0000 7f3c6c8ddb80 20 client.16002 trim_cache size 0 max 16384 2023-07-21T18:43:13.204 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.184+0000 7f3c6c8ddb80 1 Processor -- start 2023-07-21T18:43:13.205 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.184+0000 7f3c6c8ddb80 1 -- start start 2023-07-21T18:43:13.205 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.184+0000 7f3c6c8ddb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f5cd8ea2d0 0x55f5cd8f50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.205 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.184+0000 7f3c6c8ddb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5cd8eabe0 0x55f5cd8f1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.205 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.184+0000 7f3c6c8ddb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f5cd8f5ad0 0x55f5cd8f2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.205 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.184+0000 7f3c6c8ddb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5cd8f37d0 con 0x55f5cd8ea2d0 2023-07-21T18:43:13.205 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.184+0000 7f3c6c8ddb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5cd8e0970 con 0x55f5cd8eabe0 2023-07-21T18:43:13.205 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.184+0000 7f3c6c8ddb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5cd8ec1a0 con 0x55f5cd8f5ad0 2023-07-21T18:43:13.206 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.184+0000 7f3c5e4f3700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f5cd8ea2d0 0x55f5cd8f50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.206 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.184+0000 7f3c5e4f3700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f5cd8ea2d0 0x55f5cd8f50c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58578/0 (socket says 172.21.15.39:58578) 2023-07-21T18:43:13.206 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.184+0000 7f3c5e4f3700 1 -- 172.21.15.39:0/1308696653 learned_addr learned my addr 172.21.15.39:0/1308696653 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:13.206 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.184+0000 7f3c5ecf4700 1 --2- 172.21.15.39:0/1308696653 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f5cd8f5ad0 0x55f5cd8f2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.206 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.184+0000 7f3c5e4f3700 1 -- 172.21.15.39:0/1308696653 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f5cd8f5ad0 msgr2=0x55f5cd8f2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:13.206 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.184+0000 7f3c5e4f3700 1 --2- 172.21.15.39:0/1308696653 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f5cd8f5ad0 0x55f5cd8f2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.206 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.184+0000 7f3c5e4f3700 1 -- 172.21.15.39:0/1308696653 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5cd8eabe0 msgr2=0x55f5cd8f1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:13.207 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.184+0000 7f3c5dcf2700 1 --2- 172.21.15.39:0/1308696653 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5cd8eabe0 0x55f5cd8f1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.207 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.184+0000 7f3c5e4f3700 1 --2- 172.21.15.39:0/1308696653 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5cd8eabe0 0x55f5cd8f1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.207 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.184+0000 7f3c5e4f3700 1 -- 172.21.15.39:0/1308696653 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5cd8e25e0 con 0x55f5cd8ea2d0 2023-07-21T18:43:13.207 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.184+0000 7f3c5ecf4700 1 --2- 172.21.15.39:0/1308696653 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f5cd8f5ad0 0x55f5cd8f2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:43:13.207 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.184+0000 7f3c5dcf2700 1 --2- 172.21.15.39:0/1308696653 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5cd8eabe0 0x55f5cd8f1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:43:13.207 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.185+0000 7f3c5e4f3700 1 --2- 172.21.15.39:0/1308696653 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f5cd8ea2d0 0x55f5cd8f50c0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f3c5400bed0 tx=0x7f3c5400bac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8509f9638f825545 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:13.207 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.185+0000 7f3c4d7fa700 1 -- 172.21.15.39:0/1308696653 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c54011ec0 con 0x55f5cd8ea2d0 2023-07-21T18:43:13.207 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.185+0000 7f3c4d7fa700 1 -- 172.21.15.39:0/1308696653 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c54034040 con 0x55f5cd8ea2d0 2023-07-21T18:43:13.208 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.185+0000 7f3c4d7fa700 1 -- 172.21.15.39:0/1308696653 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c54003680 con 0x55f5cd8ea2d0 2023-07-21T18:43:13.209 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.185+0000 7f3c6c8ddb80 1 -- 172.21.15.39:0/1308696653 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f5cd8ea2d0 msgr2=0x55f5cd8f50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:13.209 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.185+0000 7f3c6c8ddb80 1 --2- 172.21.15.39:0/1308696653 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f5cd8ea2d0 0x55f5cd8f50c0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f3c5400bed0 tx=0x7f3c5400bac0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.210 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.185+0000 7f3c6c8ddb80 1 -- 172.21.15.39:0/1308696653 shutdown_connections 2023-07-21T18:43:13.210 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.185+0000 7f3c6c8ddb80 1 --2- 172.21.15.39:0/1308696653 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f5cd8f5ad0 0x55f5cd8f2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.210 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.185+0000 7f3c6c8ddb80 1 --2- 172.21.15.39:0/1308696653 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f5cd8ea2d0 0x55f5cd8f50c0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.210 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.185+0000 7f3c6c8ddb80 1 --2- 172.21.15.39:0/1308696653 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5cd8eabe0 0x55f5cd8f1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.210 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.185+0000 7f3c6c8ddb80 1 -- 172.21.15.39:0/1308696653 >> 172.21.15.39:0/1308696653 conn(0x55f5cd8e0da0 msgr2=0x55f5cd8edb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:13.210 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.185+0000 7f3c6c8ddb80 1 -- 172.21.15.39:0/1308696653 shutdown_connections 2023-07-21T18:43:13.210 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.185+0000 7f3c6c8ddb80 1 -- 172.21.15.39:0/1308696653 wait complete. 2023-07-21T18:43:13.210 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.186+0000 7f3c6c8ddb80 1 -- start start 2023-07-21T18:43:13.211 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.186+0000 7f3c6c8ddb80 1 Processor -- start 2023-07-21T18:43:13.211 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.186+0000 7f3c6c8ddb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f5cd8ea2d0 0x55f5cd8fbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.211 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.186+0000 7f3c6c8ddb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5cd8eabe0 0x55f5cd8fc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.211 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.186+0000 7f3c6c8ddb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f5cd8f5ad0 0x55f5cd8fa490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.211 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.186+0000 7f3c6c8ddb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5cd8faa50 con 0x55f5cd8ea2d0 2023-07-21T18:43:13.211 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.186+0000 7f3c6c8ddb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5cd8fabc0 con 0x55f5cd8eabe0 2023-07-21T18:43:13.211 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.186+0000 7f3c6c8ddb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5cd8fad80 con 0x55f5cd8f5ad0 2023-07-21T18:43:13.211 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.186+0000 7f3c6c8ddb80 10 client.0 _ll_register_callbacks cb 0x7f3c54011bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:43:13.212 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.186+0000 7f3c5ecf4700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f5cd8f5ad0 0x55f5cd8fa490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.212 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.186+0000 7f3c5dcf2700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5cd8eabe0 0x55f5cd8fc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.212 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.186+0000 7f3c5ecf4700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f5cd8f5ad0 0x55f5cd8fa490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:48908/0 (socket says 172.21.15.39:48908) 2023-07-21T18:43:13.212 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.186+0000 7f3c5dcf2700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5cd8eabe0 0x55f5cd8fc3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:42950/0 (socket says 172.21.15.39:42950) 2023-07-21T18:43:13.212 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.186+0000 7f3c5ecf4700 1 -- 172.21.15.39:0/837564312 learned_addr learned my addr 172.21.15.39:0/837564312 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:13.212 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.187+0000 7f3c5e4f3700 1 --2- 172.21.15.39:0/837564312 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f5cd8ea2d0 0x55f5cd8fbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.212 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.187+0000 7f3c5ecf4700 1 -- 172.21.15.39:0/837564312 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5cd8eabe0 msgr2=0x55f5cd8fc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:13.213 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.187+0000 7f3c5ecf4700 1 --2- 172.21.15.39:0/837564312 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5cd8eabe0 0x55f5cd8fc3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.213 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.187+0000 7f3c5ecf4700 1 -- 172.21.15.39:0/837564312 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f5cd8ea2d0 msgr2=0x55f5cd8fbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:13.213 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.187+0000 7f3c5ecf4700 1 --2- 172.21.15.39:0/837564312 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f5cd8ea2d0 0x55f5cd8fbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.213 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.187+0000 7f3c5ecf4700 1 -- 172.21.15.39:0/837564312 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f5cd653c60 con 0x55f5cd8f5ad0 2023-07-21T18:43:13.213 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.187+0000 7f3c5e4f3700 1 --2- 172.21.15.39:0/837564312 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f5cd8ea2d0 0x55f5cd8fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:43:13.213 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.187+0000 7f3c5dcf2700 1 --2- 172.21.15.39:0/837564312 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5cd8eabe0 0x55f5cd8fc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:43:13.213 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.187+0000 7f3c5ecf4700 1 --2- 172.21.15.39:0/837564312 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f5cd8f5ad0 0x55f5cd8fa490 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3c5009c070 tx=0x7f3c5009c160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5559bd4497cc2172 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:13.213 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.187+0000 7f3c41ffb700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2023-07-21T18:43:13.214 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.189+0000 7f3c41ffb700 1 -- 172.21.15.39:0/837564312 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c500a4530 con 0x55f5cd8f5ad0 2023-07-21T18:43:13.214 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.189+0000 7f3c41ffb700 1 -- 172.21.15.39:0/837564312 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f3c500a40a0 con 0x55f5cd8f5ad0 2023-07-21T18:43:13.214 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.189+0000 7f3c6c8ddb80 1 -- 172.21.15.39:0/837564312 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3c4801e030 con 0x55f5cd8f5ad0 2023-07-21T18:43:13.215 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.195+0000 7f3c41ffb700 1 -- 172.21.15.39:0/837564312 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c500a4260 con 0x55f5cd8f5ad0 2023-07-21T18:43:13.215 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.195+0000 7f3c6c8ddb80 10 client.25531 fetch_fsmap learned FSMap version 28 2023-07-21T18:43:13.215 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.195+0000 7f3c6c8ddb80 1 -- 172.21.15.39:0/837564312 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55f5cd691250 con 0x55f5cd8f5ad0 2023-07-21T18:43:13.215 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.195+0000 7f3c41ffb700 1 -- 172.21.15.39:0/837564312 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c500ca080 con 0x55f5cd8f5ad0 2023-07-21T18:43:13.216 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.195+0000 7f3c41ffb700 1 -- 172.21.15.39:0/837564312 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f3c500980d0 con 0x55f5cd8f5ad0 2023-07-21T18:43:13.216 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.195+0000 7f3c6c8ddb80 10 client.25531 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T18:43:13.216 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.195+0000 7f3c6c8ddb80 10 client.25531 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2023-07-21T18:43:13.216 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.195+0000 7f3c6c8ddb80 20 client.25531 populate_metadata read hostname 'smithi039' 2023-07-21T18:43:13.216 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.195+0000 7f3c6c8ddb80 1 --2- 172.21.15.39:0/837564312 >> [v2:172.21.15.155:6834/3854856286,v1:172.21.15.155:6835/3854856286] conn(0x55f5cd9054b0 0x55f5cd960890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.216 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.195+0000 7f3c6c8ddb80 4 client.25531 mds_command: new command op to 24464 tid=0[{"prefix": "damage ls"}] 2023-07-21T18:43:13.216 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.195+0000 7f3c6c8ddb80 1 -- 172.21.15.39:0/837564312 --> [v2:172.21.15.155:6834/3854856286,v1:172.21.15.155:6835/3854856286] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f5cd7f8050 con 0x55f5cd9054b0 2023-07-21T18:43:13.217 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.196+0000 7f3c5e4f3700 1 --2- 172.21.15.39:0/837564312 >> [v2:172.21.15.155:6834/3854856286,v1:172.21.15.155:6835/3854856286] conn(0x55f5cd9054b0 0x55f5cd960890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.218 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.196+0000 7f3c5e4f3700 1 --2- 172.21.15.39:0/837564312 >> [v2:172.21.15.155:6834/3854856286,v1:172.21.15.155:6835/3854856286] conn(0x55f5cd9054b0 0x55f5cd960890 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d806cf6d6e206c3b server_cookie=cdbc7ce4ec68acb8 in_seq=0 out_seq=0 2023-07-21T18:43:13.218 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.196+0000 7f3c41ffb700 10 client.25531 ms_handle_connect on v2:172.21.15.155:6834/3854856286 2023-07-21T18:43:13.218 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.196+0000 7f3c41ffb700 1 -- 172.21.15.39:0/837564312 <== mds.3 v2:172.21.15.155:6834/3854856286 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f5cd7f8050 con 0x55f5cd9054b0 2023-07-21T18:43:13.218 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.196+0000 7f3c41ffb700 10 client.25531 handle_command_reply: tid=0 2023-07-21T18:43:13.218 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.196+0000 7f3c6c8ddb80 1 client.25531 shutdown 2023-07-21T18:43:13.218 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.196+0000 7f3c6c8ddb80 1 -- 172.21.15.39:0/837564312 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f5cd8f5ad0 msgr2=0x55f5cd8fa490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:13.218 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.196+0000 7f3c6c8ddb80 1 --2- 172.21.15.39:0/837564312 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f5cd8f5ad0 0x55f5cd8fa490 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3c5009c070 tx=0x7f3c5009c160 comp rx=0 tx=0).stop 2023-07-21T18:43:13.219 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.196+0000 7f3c6c8ddb80 1 -- 172.21.15.39:0/837564312 shutdown_connections 2023-07-21T18:43:13.219 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.196+0000 7f3c6c8ddb80 1 --2- 172.21.15.39:0/837564312 >> [v2:172.21.15.155:6834/3854856286,v1:172.21.15.155:6835/3854856286] conn(0x55f5cd9054b0 0x55f5cd960890 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.219 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.196+0000 7f3c6c8ddb80 1 --2- 172.21.15.39:0/837564312 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f5cd8f5ad0 0x55f5cd8fa490 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.219 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.196+0000 7f3c6c8ddb80 1 --2- 172.21.15.39:0/837564312 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f5cd8ea2d0 0x55f5cd8fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.219 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.196+0000 7f3c6c8ddb80 1 --2- 172.21.15.39:0/837564312 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5cd8eabe0 0x55f5cd8fc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.219 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.196+0000 7f3c41ffb700 0 client.25531 ms_handle_reset on v2:172.21.15.155:6834/3854856286 2023-07-21T18:43:13.219 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.196+0000 7f3c6c8ddb80 1 -- 172.21.15.39:0/837564312 >> 172.21.15.39:0/837564312 conn(0x55f5cd8e0da0 msgr2=0x55f5cd8ec3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:13.219 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.197+0000 7f3c6c8ddb80 1 -- 172.21.15.39:0/837564312 shutdown_connections 2023-07-21T18:43:13.220 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.197+0000 7f3c6c8ddb80 1 -- 172.21.15.39:0/837564312 wait complete. 2023-07-21T18:43:13.220 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.197+0000 7f3c6c8ddb80 20 client.25531 trim_cache size 0 max 16384 2023-07-21T18:43:13.220 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2023-07-21T18:43:13.221 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.197+0000 7f3c60757700 1 -- 172.21.15.39:0/1918232917 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f3c440a13e0 msgr2=0x7f3c440a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:13.221 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.197+0000 7f3c60757700 1 --2- 172.21.15.39:0/1918232917 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f3c440a13e0 0x7f3c440a3890 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3c580ffb10 tx=0x7f3c5400b410 comp rx=0 tx=0).stop 2023-07-21T18:43:13.221 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.197+0000 7f3c60757700 1 -- 172.21.15.39:0/1918232917 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c581049e0 msgr2=0x7f3c581948c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:13.221 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.197+0000 7f3c60757700 1 --2- 172.21.15.39:0/1918232917 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c581049e0 0x7f3c581948c0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f3c50007f10 tx=0x7f3c5000f5c0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.221 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.197+0000 7f3c60757700 1 -- 172.21.15.39:0/1918232917 shutdown_connections 2023-07-21T18:43:13.221 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.197+0000 7f3c60757700 1 --2- 172.21.15.39:0/1918232917 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f3c440a13e0 0x7f3c440a3890 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.222 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.197+0000 7f3c60757700 1 --2- 172.21.15.39:0/1918232917 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c580fe9d0 0x7f3c5819a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.222 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.197+0000 7f3c60757700 1 --2- 172.21.15.39:0/1918232917 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c581049e0 0x7f3c581948c0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.222 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.197+0000 7f3c60757700 1 --2- 172.21.15.39:0/1918232917 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c580ff380 0x7f3c5819ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.222 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.197+0000 7f3c60757700 1 -- 172.21.15.39:0/1918232917 >> 172.21.15.39:0/1918232917 conn(0x7f3c580fa4f0 msgr2=0x7f3c580fbb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:13.222 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.197+0000 7f3c60757700 1 -- 172.21.15.39:0/1918232917 shutdown_connections 2023-07-21T18:43:13.222 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.197+0000 7f3c60757700 1 -- 172.21.15.39:0/1918232917 wait complete. 2023-07-21T18:43:13.229 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-21T18:43:13.259 INFO:tasks.workunit.client.0.smithi039.stdout: 1 29139 6.70 MB/sec warmup 26 sec latency 2108.655 ms 2023-07-21T18:43:13.368 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.348+0000 7f5830011700 1 -- 172.21.15.39:0/3489989104 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58280ff3a0 msgr2=0x7f5828103870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:13.368 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.348+0000 7f5830011700 1 --2- 172.21.15.39:0/3489989104 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58280ff3a0 0x7f5828103870 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f5818009990 tx=0x7f5818009ca0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.368 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.348+0000 7f5830011700 1 -- 172.21.15.39:0/3489989104 shutdown_connections 2023-07-21T18:43:13.369 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.348+0000 7f5830011700 1 --2- 172.21.15.39:0/3489989104 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5828103e40 0x7f58281042e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.369 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.348+0000 7f5830011700 1 --2- 172.21.15.39:0/3489989104 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58280fea90 0x7f58280fee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.369 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.348+0000 7f5830011700 1 --2- 172.21.15.39:0/3489989104 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58280ff3a0 0x7f5828103870 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.369 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.348+0000 7f5830011700 1 -- 172.21.15.39:0/3489989104 >> 172.21.15.39:0/3489989104 conn(0x7f58280fa4b0 msgr2=0x7f58280fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:13.369 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.348+0000 7f5830011700 1 -- 172.21.15.39:0/3489989104 shutdown_connections 2023-07-21T18:43:13.369 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.348+0000 7f5830011700 1 -- 172.21.15.39:0/3489989104 wait complete. 2023-07-21T18:43:13.371 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.349+0000 7f5830011700 1 Processor -- start 2023-07-21T18:43:13.371 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.349+0000 7f5830011700 1 -- start start 2023-07-21T18:43:13.371 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.349+0000 7f5830011700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58280fea90 0x7f582819c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.371 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.349+0000 7f5830011700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58280ff3a0 0x7f582819c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.371 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.349+0000 7f5830011700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5828103e40 0x7f58281a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.371 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.349+0000 7f5830011700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58281a1210 con 0x7f58280ff3a0 2023-07-21T18:43:13.372 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.349+0000 7f5830011700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58281a1380 con 0x7f5828103e40 2023-07-21T18:43:13.372 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.349+0000 7f5830011700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58281a1540 con 0x7f58280fea90 2023-07-21T18:43:13.372 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.350+0000 7f582d5ac700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58280ff3a0 0x7f582819c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.372 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.350+0000 7f582ddad700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58280fea90 0x7f582819c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.372 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.350+0000 7f582d5ac700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58280ff3a0 0x7f582819c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58602/0 (socket says 172.21.15.39:58602) 2023-07-21T18:43:13.372 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.350+0000 7f582e5ae700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5828103e40 0x7f58281a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.372 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.350+0000 7f582d5ac700 1 -- 172.21.15.39:0/1452154487 learned_addr learned my addr 172.21.15.39:0/1452154487 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:13.372 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.350+0000 7f582ddad700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58280fea90 0x7f582819c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:48918/0 (socket says 172.21.15.39:48918) 2023-07-21T18:43:13.373 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.350+0000 7f582d5ac700 1 -- 172.21.15.39:0/1452154487 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58280fea90 msgr2=0x7f582819c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:13.373 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.350+0000 7f582d5ac700 1 --2- 172.21.15.39:0/1452154487 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58280fea90 0x7f582819c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.373 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.350+0000 7f582d5ac700 1 -- 172.21.15.39:0/1452154487 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5828103e40 msgr2=0x7f58281a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:13.373 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.350+0000 7f582d5ac700 1 --2- 172.21.15.39:0/1452154487 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5828103e40 0x7f58281a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.373 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.350+0000 7f582d5ac700 1 -- 172.21.15.39:0/1452154487 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58180095f0 con 0x7f58280ff3a0 2023-07-21T18:43:13.373 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.350+0000 7f582d5ac700 1 --2- 172.21.15.39:0/1452154487 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58280ff3a0 0x7f582819c770 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f5818005910 tx=0x7f5818004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:13.373 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.350+0000 7f581effd700 1 -- 172.21.15.39:0/1452154487 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581801d070 con 0x7f58280ff3a0 2023-07-21T18:43:13.374 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.350+0000 7f5830011700 1 -- 172.21.15.39:0/1452154487 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58281a17c0 con 0x7f58280ff3a0 2023-07-21T18:43:13.374 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.350+0000 7f581effd700 1 -- 172.21.15.39:0/1452154487 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5818022470 con 0x7f58280ff3a0 2023-07-21T18:43:13.374 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.350+0000 7f581effd700 1 -- 172.21.15.39:0/1452154487 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5818017650 con 0x7f58280ff3a0 2023-07-21T18:43:13.374 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.350+0000 7f5830011700 1 -- 172.21.15.39:0/1452154487 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58281a9060 con 0x7f58280ff3a0 2023-07-21T18:43:13.376 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.354+0000 7f583c197b80 1 Processor -- start 2023-07-21T18:43:13.377 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.354+0000 7f583c197b80 1 -- start start 2023-07-21T18:43:13.377 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.354+0000 7f583c197b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5561624302d0 0x5561624306a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.377 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.354+0000 7f583c197b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556162430be0 0x55616243b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.377 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.354+0000 7f583c197b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55616243bad0 0x55616243deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.377 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.354+0000 7f583c197b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x556162336cb0 con 0x5561624302d0 2023-07-21T18:43:13.377 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.354+0000 7f583c197b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556162389a40 con 0x556162430be0 2023-07-21T18:43:13.377 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.354+0000 7f583c197b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x556162382ed0 con 0x55616243bad0 2023-07-21T18:43:13.378 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.354+0000 7f582e5ae700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5561624302d0 0x5561624306a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.378 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.354+0000 7f582e5ae700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5561624302d0 0x5561624306a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58604/0 (socket says 172.21.15.39:58604) 2023-07-21T18:43:13.378 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.354+0000 7f582e5ae700 1 -- 172.21.15.39:0/877302071 learned_addr learned my addr 172.21.15.39:0/877302071 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:13.378 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.355+0000 7f582ddad700 1 --2- 172.21.15.39:0/877302071 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556162430be0 0x55616243b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.378 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.355+0000 7f582e5ae700 1 -- 172.21.15.39:0/877302071 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55616243bad0 msgr2=0x55616243deb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T18:43:13.378 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.355+0000 7f582e5ae700 1 --2- 172.21.15.39:0/877302071 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55616243bad0 0x55616243deb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.378 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.355+0000 7f582e5ae700 1 -- 172.21.15.39:0/877302071 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556162430be0 msgr2=0x55616243b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:13.378 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.355+0000 7f582e5ae700 1 --2- 172.21.15.39:0/877302071 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556162430be0 0x55616243b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.379 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.355+0000 7f582e5ae700 1 -- 172.21.15.39:0/877302071 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5561621b0de0 con 0x5561624302d0 2023-07-21T18:43:13.379 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.355+0000 7f582ddad700 1 --2- 172.21.15.39:0/877302071 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556162430be0 0x55616243b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:43:13.379 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.355+0000 7f582e5ae700 1 --2- 172.21.15.39:0/877302071 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5561624302d0 0x5561624306a0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f5820011680 tx=0x7f5820009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a751c4073507f1e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:13.379 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.355+0000 7f5813fff700 1 -- 172.21.15.39:0/877302071 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582000e040 con 0x5561624302d0 2023-07-21T18:43:13.379 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.355+0000 7f5813fff700 1 -- 172.21.15.39:0/877302071 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f582000f040 con 0x5561624302d0 2023-07-21T18:43:13.379 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.355+0000 7f5813fff700 1 -- 172.21.15.39:0/877302071 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5820004890 con 0x5561624302d0 2023-07-21T18:43:13.379 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.356+0000 7f581effd700 1 -- 172.21.15.39:0/1452154487 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f58180225e0 con 0x7f58280ff3a0 2023-07-21T18:43:13.380 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.356+0000 7f583c197b80 1 -- 172.21.15.39:0/877302071 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5561624302d0 msgr2=0x5561624306a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:13.380 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.356+0000 7f583c197b80 1 --2- 172.21.15.39:0/877302071 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5561624302d0 0x5561624306a0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f5820011680 tx=0x7f5820009920 comp rx=0 tx=0).stop 2023-07-21T18:43:13.380 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.356+0000 7f583c197b80 1 -- 172.21.15.39:0/877302071 shutdown_connections 2023-07-21T18:43:13.380 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.356+0000 7f583c197b80 1 --2- 172.21.15.39:0/877302071 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55616243bad0 0x55616243deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.380 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.356+0000 7f583c197b80 1 --2- 172.21.15.39:0/877302071 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5561624302d0 0x5561624306a0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.380 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.356+0000 7f583c197b80 1 --2- 172.21.15.39:0/877302071 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556162430be0 0x55616243b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.380 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.356+0000 7f583c197b80 1 -- 172.21.15.39:0/877302071 >> 172.21.15.39:0/877302071 conn(0x556162426da0 msgr2=0x55616242c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:13.380 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.356+0000 7f583c197b80 1 -- 172.21.15.39:0/877302071 shutdown_connections 2023-07-21T18:43:13.381 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.356+0000 7f583c197b80 1 -- 172.21.15.39:0/877302071 wait complete. 2023-07-21T18:43:13.381 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.356+0000 7f581effd700 1 --2- 172.21.15.39:0/1452154487 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f58140a12f0 0x7f58140a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.381 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.356+0000 7f582ddad700 1 --2- 172.21.15.39:0/1452154487 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f58140a12f0 0x7f58140a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.381 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.356+0000 7f581effd700 1 -- 172.21.15.39:0/1452154487 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f58180ce200 con 0x7f58280ff3a0 2023-07-21T18:43:13.381 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.357+0000 7f583c197b80 1 -- start start 2023-07-21T18:43:13.381 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.357+0000 7f583c197b80 1 Processor -- start 2023-07-21T18:43:13.381 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.357+0000 7f582ddad700 1 --2- 172.21.15.39:0/1452154487 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f58140a12f0 0x7f58140a37a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5824009d00 tx=0x7f5824009260 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:13.384 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.357+0000 7f583c197b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5561624302d0 0x55616243b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.384 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.357+0000 7f583c197b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556162430be0 0x5561624387b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.384 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.357+0000 7f583c197b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55616243bad0 0x556162438ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.384 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.357+0000 7f583c197b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x556162322ac0 con 0x55616243bad0 2023-07-21T18:43:13.384 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.357+0000 7f583c197b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561623227c0 con 0x5561624302d0 2023-07-21T18:43:13.385 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.357+0000 7f583c197b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x556162206920 con 0x556162430be0 2023-07-21T18:43:13.385 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.357+0000 7f583c197b80 10 client.0 _ll_register_callbacks cb 0x556162320760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:43:13.385 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.357+0000 7f582e5ae700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55616243bad0 0x556162438ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.385 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.357+0000 7f582e5ae700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55616243bad0 0x556162438ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58616/0 (socket says 172.21.15.39:58616) 2023-07-21T18:43:13.385 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.357+0000 7f582e5ae700 1 -- 172.21.15.39:0/997015874 learned_addr learned my addr 172.21.15.39:0/997015874 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:13.385 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.357+0000 7f582d5ac700 1 --2- 172.21.15.39:0/997015874 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556162430be0 0x5561624387b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.385 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.358+0000 7f582ddad700 1 --2- 172.21.15.39:0/997015874 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5561624302d0 0x55616243b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.386 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.358+0000 7f582e5ae700 1 -- 172.21.15.39:0/997015874 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556162430be0 msgr2=0x5561624387b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:13.386 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.358+0000 7f582e5ae700 1 --2- 172.21.15.39:0/997015874 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556162430be0 0x5561624387b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.386 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.358+0000 7f582e5ae700 1 -- 172.21.15.39:0/997015874 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5561624302d0 msgr2=0x55616243b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:13.386 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.358+0000 7f582e5ae700 1 --2- 172.21.15.39:0/997015874 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5561624302d0 0x55616243b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.386 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.358+0000 7f582e5ae700 1 -- 172.21.15.39:0/997015874 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5820011330 con 0x55616243bad0 2023-07-21T18:43:13.386 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.358+0000 7f582ddad700 1 --2- 172.21.15.39:0/997015874 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5561624302d0 0x55616243b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:43:13.386 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.358+0000 7f582e5ae700 1 --2- 172.21.15.39:0/997015874 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55616243bad0 0x556162438ef0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f582000b010 tx=0x7f5820005b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=256b4fe2bae6a8a7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:13.386 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.358+0000 7f58117fa700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:43:13.387 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.358+0000 7f58117fa700 1 -- 172.21.15.39:0/997015874 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582000e070 con 0x55616243bad0 2023-07-21T18:43:13.387 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.358+0000 7f58117fa700 1 -- 172.21.15.39:0/997015874 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f5820003dd0 con 0x55616243bad0 2023-07-21T18:43:13.387 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.358+0000 7f583c197b80 1 -- 172.21.15.39:0/997015874 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5561622dc4f0 con 0x55616243bad0 2023-07-21T18:43:13.387 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.358+0000 7f58117fa700 1 -- 172.21.15.39:0/997015874 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5820007500 con 0x55616243bad0 2023-07-21T18:43:13.387 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.358+0000 7f583c197b80 10 client.16017 fetch_fsmap learned FSMap version 28 2023-07-21T18:43:13.387 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.358+0000 7f583c197b80 1 -- 172.21.15.39:0/997015874 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x556162199c60 con 0x55616243bad0 2023-07-21T18:43:13.387 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.358+0000 7f58117fa700 1 -- 172.21.15.39:0/997015874 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5561622dc4f0 con 0x55616243bad0 2023-07-21T18:43:13.387 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.359+0000 7f58117fa700 1 -- 172.21.15.39:0/997015874 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f58200305d0 con 0x55616243bad0 2023-07-21T18:43:13.388 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.359+0000 7f583c197b80 10 client.16017 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T18:43:13.388 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.359+0000 7f583c197b80 10 client.16017 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T18:43:13.388 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.359+0000 7f583c197b80 20 client.16017 populate_metadata read hostname 'smithi039' 2023-07-21T18:43:13.388 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.359+0000 7f583c197b80 1 --2- 172.21.15.39:0/997015874 >> [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] conn(0x55616244b390 0x55616244d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.388 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.359+0000 7f583c197b80 4 client.16017 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:43:13.388 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.359+0000 7f583c197b80 1 -- 172.21.15.39:0/997015874 --> [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55616237beb0 con 0x55616244b390 2023-07-21T18:43:13.388 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.359+0000 7f582ddad700 1 --2- 172.21.15.39:0/997015874 >> [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] conn(0x55616244b390 0x55616244d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.388 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.359+0000 7f58117fa700 10 client.16017 ms_handle_connect on v2:172.21.15.193:6838/3148488639 2023-07-21T18:43:13.389 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.359+0000 7f582ddad700 1 --2- 172.21.15.39:0/997015874 >> [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] conn(0x55616244b390 0x55616244d7a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d5a6a9860aeffe60 server_cookie=ee99d284af346eae in_seq=0 out_seq=0 2023-07-21T18:43:13.389 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.360+0000 7f58117fa700 1 -- 172.21.15.39:0/997015874 <== mds.2 v2:172.21.15.193:6838/3148488639 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55616237beb0 con 0x55616244b390 2023-07-21T18:43:13.389 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.360+0000 7f58117fa700 10 client.16017 handle_command_reply: tid=0 2023-07-21T18:43:13.389 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.360+0000 7f583c197b80 1 client.16017 shutdown 2023-07-21T18:43:13.389 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.360+0000 7f583c197b80 1 -- 172.21.15.39:0/997015874 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55616243bad0 msgr2=0x556162438ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:13.389 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.360+0000 7f583c197b80 1 --2- 172.21.15.39:0/997015874 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55616243bad0 0x556162438ef0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f582000b010 tx=0x7f5820005b60 comp rx=0 tx=0).stop 2023-07-21T18:43:13.389 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.360+0000 7f583c197b80 1 -- 172.21.15.39:0/997015874 shutdown_connections 2023-07-21T18:43:13.390 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.360+0000 7f583c197b80 1 --2- 172.21.15.39:0/997015874 >> [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] conn(0x55616244b390 0x55616244d7a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.390 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.360+0000 7f583c197b80 1 --2- 172.21.15.39:0/997015874 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5561624302d0 0x55616243b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.390 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.360+0000 7f583c197b80 1 --2- 172.21.15.39:0/997015874 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55616243bad0 0x556162438ef0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.390 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.360+0000 7f583c197b80 1 --2- 172.21.15.39:0/997015874 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556162430be0 0x5561624387b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.390 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.360+0000 7f58117fa700 0 client.16017 ms_handle_reset on v2:172.21.15.193:6838/3148488639 2023-07-21T18:43:13.390 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.360+0000 7f583c197b80 1 -- 172.21.15.39:0/997015874 >> 172.21.15.39:0/997015874 conn(0x556162426da0 msgr2=0x55616242c230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:13.390 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.360+0000 7f583c197b80 1 -- 172.21.15.39:0/997015874 shutdown_connections 2023-07-21T18:43:13.390 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.360+0000 7f583c197b80 1 -- 172.21.15.39:0/997015874 wait complete. 2023-07-21T18:43:13.391 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.360+0000 7f583c197b80 20 client.16017 trim_cache size 0 max 16384 2023-07-21T18:43:13.394 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.373+0000 7f583c197b80 1 Processor -- start 2023-07-21T18:43:13.394 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.373+0000 7f583c197b80 1 -- start start 2023-07-21T18:43:13.394 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.373+0000 7f583c197b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5561624302d0 0x55616243b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.394 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.373+0000 7f583c197b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556162430be0 0x556162437f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.394 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.373+0000 7f583c197b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55616243bad0 0x5561624384b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.394 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.373+0000 7f583c197b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x556162428ec0 con 0x5561624302d0 2023-07-21T18:43:13.395 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.373+0000 7f583c197b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556162433680 con 0x55616243bad0 2023-07-21T18:43:13.395 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.373+0000 7f583c197b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55616242dc90 con 0x556162430be0 2023-07-21T18:43:13.395 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.373+0000 7f582ddad700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5561624302d0 0x55616243b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.395 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.373+0000 7f582ddad700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5561624302d0 0x55616243b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58620/0 (socket says 172.21.15.39:58620) 2023-07-21T18:43:13.395 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.374+0000 7f582ddad700 1 -- 172.21.15.39:0/946002387 learned_addr learned my addr 172.21.15.39:0/946002387 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:13.395 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.374+0000 7f582e5ae700 1 --2- 172.21.15.39:0/946002387 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55616243bad0 0x5561624384b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.395 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.374+0000 7f582d5ac700 1 --2- 172.21.15.39:0/946002387 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556162430be0 0x556162437f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.395 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.374+0000 7f582ddad700 1 -- 172.21.15.39:0/946002387 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556162430be0 msgr2=0x556162437f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:13.396 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.374+0000 7f582ddad700 1 --2- 172.21.15.39:0/946002387 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556162430be0 0x556162437f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.396 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.374+0000 7f582ddad700 1 -- 172.21.15.39:0/946002387 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55616243bad0 msgr2=0x5561624384b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:13.396 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.374+0000 7f582ddad700 1 --2- 172.21.15.39:0/946002387 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55616243bad0 0x5561624384b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.396 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.374+0000 7f582ddad700 1 -- 172.21.15.39:0/946002387 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5561621d7250 con 0x5561624302d0 2023-07-21T18:43:13.396 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.374+0000 7f582e5ae700 1 --2- 172.21.15.39:0/946002387 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55616243bad0 0x5561624384b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:43:13.396 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.374+0000 7f582ddad700 1 --2- 172.21.15.39:0/946002387 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5561624302d0 0x55616243b160 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x55616244e070 tx=0x7f5824016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43b361ab5cd60878 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:13.396 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.374+0000 7f581cff9700 1 -- 172.21.15.39:0/946002387 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5824024040 con 0x5561624302d0 2023-07-21T18:43:13.397 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.374+0000 7f581cff9700 1 -- 172.21.15.39:0/946002387 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f582402d070 con 0x5561624302d0 2023-07-21T18:43:13.397 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.374+0000 7f581cff9700 1 -- 172.21.15.39:0/946002387 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5824016070 con 0x5561624302d0 2023-07-21T18:43:13.400 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.375+0000 7f583c197b80 1 -- 172.21.15.39:0/946002387 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5561624302d0 msgr2=0x55616243b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:13.400 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.375+0000 7f583c197b80 1 --2- 172.21.15.39:0/946002387 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5561624302d0 0x55616243b160 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x55616244e070 tx=0x7f5824016000 comp rx=0 tx=0).stop 2023-07-21T18:43:13.400 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.375+0000 7f583c197b80 1 -- 172.21.15.39:0/946002387 shutdown_connections 2023-07-21T18:43:13.400 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.375+0000 7f583c197b80 1 --2- 172.21.15.39:0/946002387 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55616243bad0 0x5561624384b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.400 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.375+0000 7f583c197b80 1 --2- 172.21.15.39:0/946002387 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5561624302d0 0x55616243b160 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.400 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.375+0000 7f583c197b80 1 --2- 172.21.15.39:0/946002387 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556162430be0 0x556162437f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.401 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.375+0000 7f583c197b80 1 -- 172.21.15.39:0/946002387 >> 172.21.15.39:0/946002387 conn(0x556162426da0 msgr2=0x556162447820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:13.401 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.375+0000 7f583c197b80 1 -- 172.21.15.39:0/946002387 shutdown_connections 2023-07-21T18:43:13.401 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.375+0000 7f583c197b80 1 -- 172.21.15.39:0/946002387 wait complete. 2023-07-21T18:43:13.401 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.375+0000 7f583c197b80 1 -- start start 2023-07-21T18:43:13.401 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.375+0000 7f583c197b80 1 Processor -- start 2023-07-21T18:43:13.401 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.375+0000 7f583c197b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5561624302d0 0x556162442600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.401 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.375+0000 7f583c197b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556162430be0 0x556162440920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.401 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.375+0000 7f583c197b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55616243bad0 0x556162440ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.402 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.375+0000 7f583c197b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561624415d0 con 0x55616243bad0 2023-07-21T18:43:13.402 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.375+0000 7f583c197b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556162441740 con 0x556162430be0 2023-07-21T18:43:13.402 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.375+0000 7f583c197b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55616249f220 con 0x5561624302d0 2023-07-21T18:43:13.402 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.376+0000 7f583c197b80 10 client.0 _ll_register_callbacks cb 0x7f5820011550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:43:13.402 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.376+0000 7f582e5ae700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55616243bad0 0x556162440ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.402 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.376+0000 7f582d5ac700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556162430be0 0x556162440920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.402 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.376+0000 7f582e5ae700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55616243bad0 0x556162440ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58630/0 (socket says 172.21.15.39:58630) 2023-07-21T18:43:13.402 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.376+0000 7f582e5ae700 1 -- 172.21.15.39:0/4016550974 learned_addr learned my addr 172.21.15.39:0/4016550974 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:13.403 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.376+0000 7f582ddad700 1 --2- 172.21.15.39:0/4016550974 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5561624302d0 0x556162442600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.403 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.376+0000 7f582e5ae700 1 -- 172.21.15.39:0/4016550974 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5561624302d0 msgr2=0x556162442600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:13.403 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.376+0000 7f582e5ae700 1 --2- 172.21.15.39:0/4016550974 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5561624302d0 0x556162442600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.403 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.376+0000 7f582e5ae700 1 -- 172.21.15.39:0/4016550974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556162430be0 msgr2=0x556162440920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:13.403 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.376+0000 7f582e5ae700 1 --2- 172.21.15.39:0/4016550974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556162430be0 0x556162440920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.403 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.376+0000 7f582e5ae700 1 -- 172.21.15.39:0/4016550974 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5824029950 con 0x55616243bad0 2023-07-21T18:43:13.403 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.376+0000 7f582e5ae700 1 --2- 172.21.15.39:0/4016550974 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55616243bad0 0x556162440ec0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f5820009dd0 tx=0x7f5820009e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27649f31e08d0483 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:13.404 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.376+0000 7f58117fa700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:43:13.404 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.376+0000 7f58117fa700 1 -- 172.21.15.39:0/4016550974 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582000e070 con 0x55616243bad0 2023-07-21T18:43:13.404 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.376+0000 7f58117fa700 1 -- 172.21.15.39:0/4016550974 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f5820030e40 con 0x55616243bad0 2023-07-21T18:43:13.404 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.376+0000 7f583c197b80 1 -- 172.21.15.39:0/4016550974 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f582000f040 con 0x55616243bad0 2023-07-21T18:43:13.404 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.376+0000 7f58117fa700 1 -- 172.21.15.39:0/4016550974 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5820003fe0 con 0x55616243bad0 2023-07-21T18:43:13.404 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.376+0000 7f583c197b80 10 client.16023 fetch_fsmap learned FSMap version 28 2023-07-21T18:43:13.404 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.376+0000 7f583c197b80 1 -- 172.21.15.39:0/4016550974 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5561621af2b0 con 0x55616243bad0 2023-07-21T18:43:13.404 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.377+0000 7f58117fa700 1 -- 172.21.15.39:0/4016550974 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f582003c080 con 0x55616243bad0 2023-07-21T18:43:13.405 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.377+0000 7f58117fa700 1 -- 172.21.15.39:0/4016550974 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f58200157a0 con 0x55616243bad0 2023-07-21T18:43:13.405 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.377+0000 7f583c197b80 10 client.16023 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T18:43:13.405 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.377+0000 7f583c197b80 10 client.16023 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T18:43:13.405 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.377+0000 7f583c197b80 20 client.16023 populate_metadata read hostname 'smithi039' 2023-07-21T18:43:13.405 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.377+0000 7f583c197b80 1 --2- 172.21.15.39:0/4016550974 >> [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] conn(0x5561624a0620 0x5561624a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.405 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.377+0000 7f583c197b80 4 client.16023 mds_command: new command op to 24487 tid=0[{"prefix": "damage ls"}] 2023-07-21T18:43:13.405 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.377+0000 7f583c197b80 1 -- 172.21.15.39:0/4016550974 --> [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55616233e050 con 0x5561624a0620 2023-07-21T18:43:13.405 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.377+0000 7f582ddad700 1 --2- 172.21.15.39:0/4016550974 >> [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] conn(0x5561624a0620 0x5561624a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.406 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.377+0000 7f58117fa700 10 client.16023 ms_handle_connect on v2:172.21.15.193:6838/3148488639 2023-07-21T18:43:13.406 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.377+0000 7f582ddad700 1 --2- 172.21.15.39:0/4016550974 >> [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] conn(0x5561624a0620 0x5561624a2a30 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=aa327ea0896777aa server_cookie=d325f4dc75490650 in_seq=0 out_seq=0 2023-07-21T18:43:13.406 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.378+0000 7f58117fa700 1 -- 172.21.15.39:0/4016550974 <== mds.2 v2:172.21.15.193:6838/3148488639 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55616233e050 con 0x5561624a0620 2023-07-21T18:43:13.406 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.378+0000 7f58117fa700 10 client.16023 handle_command_reply: tid=0 2023-07-21T18:43:13.406 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.378+0000 7f583c197b80 1 client.16023 shutdown 2023-07-21T18:43:13.406 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.378+0000 7f583c197b80 1 -- 172.21.15.39:0/4016550974 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55616243bad0 msgr2=0x556162440ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:13.406 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.378+0000 7f583c197b80 1 --2- 172.21.15.39:0/4016550974 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55616243bad0 0x556162440ec0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f5820009dd0 tx=0x7f5820009e00 comp rx=0 tx=0).stop 2023-07-21T18:43:13.407 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.378+0000 7f583c197b80 1 -- 172.21.15.39:0/4016550974 shutdown_connections 2023-07-21T18:43:13.407 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.378+0000 7f583c197b80 1 --2- 172.21.15.39:0/4016550974 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55616243bad0 0x556162440ec0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.407 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.378+0000 7f583c197b80 1 --2- 172.21.15.39:0/4016550974 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5561624302d0 0x556162442600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.407 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.378+0000 7f583c197b80 1 --2- 172.21.15.39:0/4016550974 >> [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] conn(0x5561624a0620 0x5561624a2a30 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.407 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.378+0000 7f583c197b80 1 --2- 172.21.15.39:0/4016550974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556162430be0 0x556162440920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.407 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.378+0000 7f58117fa700 0 client.16023 ms_handle_reset on v2:172.21.15.193:6838/3148488639 2023-07-21T18:43:13.407 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.378+0000 7f583c197b80 1 -- 172.21.15.39:0/4016550974 >> 172.21.15.39:0/4016550974 conn(0x556162426da0 msgr2=0x55616243b160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:13.407 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.378+0000 7f583c197b80 1 -- 172.21.15.39:0/4016550974 shutdown_connections 2023-07-21T18:43:13.408 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.378+0000 7f583c197b80 1 -- 172.21.15.39:0/4016550974 wait complete. 2023-07-21T18:43:13.408 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.378+0000 7f583c197b80 20 client.16023 trim_cache size 0 max 16384 2023-07-21T18:43:13.408 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2023-07-21T18:43:13.409 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.378+0000 7f5830011700 1 -- 172.21.15.39:0/1452154487 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f58140a12f0 msgr2=0x7f58140a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:13.409 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.378+0000 7f5830011700 1 --2- 172.21.15.39:0/1452154487 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f58140a12f0 0x7f58140a37a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5824009d00 tx=0x7f5824009260 comp rx=0 tx=0).stop 2023-07-21T18:43:13.409 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.378+0000 7f5830011700 1 -- 172.21.15.39:0/1452154487 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58280ff3a0 msgr2=0x7f582819c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:13.409 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.378+0000 7f5830011700 1 --2- 172.21.15.39:0/1452154487 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58280ff3a0 0x7f582819c770 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f5818005910 tx=0x7f5818004730 comp rx=0 tx=0).stop 2023-07-21T18:43:13.409 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.379+0000 7f5830011700 1 -- 172.21.15.39:0/1452154487 shutdown_connections 2023-07-21T18:43:13.409 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.379+0000 7f5830011700 1 --2- 172.21.15.39:0/1452154487 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f58140a12f0 0x7f58140a37a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.410 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.379+0000 7f5830011700 1 --2- 172.21.15.39:0/1452154487 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5828103e40 0x7f58281a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.410 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.379+0000 7f5830011700 1 --2- 172.21.15.39:0/1452154487 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58280fea90 0x7f582819c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.410 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.379+0000 7f5830011700 1 --2- 172.21.15.39:0/1452154487 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58280ff3a0 0x7f582819c770 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.410 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.379+0000 7f5830011700 1 -- 172.21.15.39:0/1452154487 >> 172.21.15.39:0/1452154487 conn(0x7f58280fa4b0 msgr2=0x7f5828101990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:13.410 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.379+0000 7f5830011700 1 -- 172.21.15.39:0/1452154487 shutdown_connections 2023-07-21T18:43:13.410 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.379+0000 7f5830011700 1 -- 172.21.15.39:0/1452154487 wait complete. 2023-07-21T18:43:13.411 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-21T18:43:13.476 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:13 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/817293471' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:43:13.580 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.560+0000 7f741858f700 1 -- 172.21.15.39:0/4136566353 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7410103ce0 msgr2=0x7f741010b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:13.580 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.560+0000 7f741858f700 1 --2- 172.21.15.39:0/4136566353 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7410103ce0 0x7f741010b8f0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f740800b0d0 tx=0x7f740800b3e0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.580 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.560+0000 7f741858f700 1 -- 172.21.15.39:0/4136566353 shutdown_connections 2023-07-21T18:43:13.580 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.560+0000 7f741858f700 1 --2- 172.21.15.39:0/4136566353 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74100ff580 0x7f74100ff950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.581 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.560+0000 7f741858f700 1 --2- 172.21.15.39:0/4136566353 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7410103ce0 0x7f741010b8f0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.581 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.560+0000 7f741858f700 1 --2- 172.21.15.39:0/4136566353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74100fff20 0x7f7410103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.581 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.560+0000 7f741858f700 1 -- 172.21.15.39:0/4136566353 >> 172.21.15.39:0/4136566353 conn(0x7f7410076200 msgr2=0x7f7410076600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:13.581 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.560+0000 7f741858f700 1 -- 172.21.15.39:0/4136566353 shutdown_connections 2023-07-21T18:43:13.581 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.560+0000 7f741858f700 1 -- 172.21.15.39:0/4136566353 wait complete. 2023-07-21T18:43:13.583 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.561+0000 7f741858f700 1 Processor -- start 2023-07-21T18:43:13.583 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.561+0000 7f741858f700 1 -- start start 2023-07-21T18:43:13.583 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.561+0000 7f741858f700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74100ff580 0x7f741019aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.583 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.561+0000 7f741858f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74100fff20 0x7f741019af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.583 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.561+0000 7f741858f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7410103ce0 0x7f7410194a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.584 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.561+0000 7f741858f700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7410195100 con 0x7f7410103ce0 2023-07-21T18:43:13.584 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.561+0000 7f741858f700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7410195270 con 0x7f74100fff20 2023-07-21T18:43:13.584 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.561+0000 7f741858f700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7410195430 con 0x7f74100ff580 2023-07-21T18:43:13.584 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.561+0000 7f7416b2c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7410103ce0 0x7f7410194a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.584 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.561+0000 7f7416b2c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7410103ce0 0x7f7410194a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58644/0 (socket says 172.21.15.39:58644) 2023-07-21T18:43:13.584 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.561+0000 7f7416b2c700 1 -- 172.21.15.39:0/3592488002 learned_addr learned my addr 172.21.15.39:0/3592488002 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:13.584 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.561+0000 7f741632b700 1 --2- 172.21.15.39:0/3592488002 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74100ff580 0x7f741019aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.585 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.561+0000 7f7415b2a700 1 --2- 172.21.15.39:0/3592488002 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74100fff20 0x7f741019af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.585 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.562+0000 7f7416b2c700 1 -- 172.21.15.39:0/3592488002 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74100ff580 msgr2=0x7f741019aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:13.585 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.562+0000 7f7416b2c700 1 --2- 172.21.15.39:0/3592488002 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74100ff580 0x7f741019aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.585 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.562+0000 7f7416b2c700 1 -- 172.21.15.39:0/3592488002 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74100fff20 msgr2=0x7f741019af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:13.585 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.562+0000 7f7416b2c700 1 --2- 172.21.15.39:0/3592488002 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74100fff20 0x7f741019af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.585 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.562+0000 7f7416b2c700 1 -- 172.21.15.39:0/3592488002 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7408009d00 con 0x7f7410103ce0 2023-07-21T18:43:13.585 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.562+0000 7f7415b2a700 1 --2- 172.21.15.39:0/3592488002 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74100fff20 0x7f741019af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:43:13.585 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.562+0000 7f7416b2c700 1 --2- 172.21.15.39:0/3592488002 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7410103ce0 0x7f7410194a90 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f7408007870 tx=0x7f7408007950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:13.586 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.562+0000 7f74037fe700 1 -- 172.21.15.39:0/3592488002 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740800e040 con 0x7f7410103ce0 2023-07-21T18:43:13.586 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.562+0000 7f74037fe700 1 -- 172.21.15.39:0/3592488002 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f740800be30 con 0x7f7410103ce0 2023-07-21T18:43:13.586 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.562+0000 7f741858f700 1 -- 172.21.15.39:0/3592488002 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74101956b0 con 0x7f7410103ce0 2023-07-21T18:43:13.586 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.562+0000 7f74037fe700 1 -- 172.21.15.39:0/3592488002 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740801b750 con 0x7f7410103ce0 2023-07-21T18:43:13.586 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.562+0000 7f741858f700 1 -- 172.21.15.39:0/3592488002 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74101a1a10 con 0x7f7410103ce0 2023-07-21T18:43:13.588 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.565+0000 7f74037fe700 1 -- 172.21.15.39:0/3592488002 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f7408024410 con 0x7f7410103ce0 2023-07-21T18:43:13.588 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.565+0000 7f74037fe700 1 --2- 172.21.15.39:0/3592488002 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f73fc0a13e0 0x7f73fc0a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.588 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.565+0000 7f741632b700 1 --2- 172.21.15.39:0/3592488002 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f73fc0a13e0 0x7f73fc0a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.589 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.565+0000 7f74037fe700 1 -- 172.21.15.39:0/3592488002 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f74080ce230 con 0x7f7410103ce0 2023-07-21T18:43:13.589 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.566+0000 7f7424715b80 1 Processor -- start 2023-07-21T18:43:13.589 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.566+0000 7f7424715b80 1 -- start start 2023-07-21T18:43:13.589 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.566+0000 7f7424715b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55665f1622d0 0x55665f1626a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.589 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.566+0000 7f7424715b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55665f162be0 0x55665f16d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.589 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.566+0000 7f7424715b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55665f16dad0 0x55665f16feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.589 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.566+0000 7f7424715b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55665f068cb0 con 0x55665f16dad0 2023-07-21T18:43:13.590 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.566+0000 7f7424715b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55665f0bba40 con 0x55665f162be0 2023-07-21T18:43:13.590 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.566+0000 7f7424715b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55665f0b4ed0 con 0x55665f1622d0 2023-07-21T18:43:13.590 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.566+0000 7f741632b700 1 --2- 172.21.15.39:0/3592488002 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f73fc0a13e0 0x7f73fc0a3890 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7404005950 tx=0x7f74040058e0 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:13.590 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.567+0000 7f7416b2c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55665f16dad0 0x55665f16feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.590 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.567+0000 7f7416b2c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55665f16dad0 0x55665f16feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58652/0 (socket says 172.21.15.39:58652) 2023-07-21T18:43:13.590 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.567+0000 7f7416b2c700 1 -- 172.21.15.39:0/3590430704 learned_addr learned my addr 172.21.15.39:0/3590430704 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:13.590 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.567+0000 7f741632b700 1 --2- 172.21.15.39:0/3590430704 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55665f1622d0 0x55665f1626a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.591 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.567+0000 7f7415b2a700 1 --2- 172.21.15.39:0/3590430704 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55665f162be0 0x55665f16d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.591 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.567+0000 7f7416b2c700 1 -- 172.21.15.39:0/3590430704 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55665f1622d0 msgr2=0x55665f1626a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:13.591 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.567+0000 7f7416b2c700 1 --2- 172.21.15.39:0/3590430704 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55665f1622d0 0x55665f1626a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.591 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.567+0000 7f7416b2c700 1 -- 172.21.15.39:0/3590430704 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55665f162be0 msgr2=0x55665f16d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:13.591 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.567+0000 7f7416b2c700 1 --2- 172.21.15.39:0/3590430704 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55665f162be0 0x55665f16d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.591 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.567+0000 7f7416b2c700 1 -- 172.21.15.39:0/3590430704 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55665eee2de0 con 0x55665f16dad0 2023-07-21T18:43:13.591 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.567+0000 7f7416b2c700 1 --2- 172.21.15.39:0/3590430704 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55665f16dad0 0x55665f16feb0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f740809b0f0 tx=0x7f740809afb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ce54e1eb142d141 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:13.591 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.567+0000 7f7400ff9700 1 -- 172.21.15.39:0/3590430704 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740809b160 con 0x55665f16dad0 2023-07-21T18:43:13.592 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.567+0000 7f7400ff9700 1 -- 172.21.15.39:0/3590430704 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f740809f190 con 0x55665f16dad0 2023-07-21T18:43:13.592 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.567+0000 7f7400ff9700 1 -- 172.21.15.39:0/3590430704 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740809a650 con 0x55665f16dad0 2023-07-21T18:43:13.592 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.568+0000 7f7424715b80 1 -- 172.21.15.39:0/3590430704 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55665f16dad0 msgr2=0x55665f16feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:13.592 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.568+0000 7f7424715b80 1 --2- 172.21.15.39:0/3590430704 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55665f16dad0 0x55665f16feb0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f740809b0f0 tx=0x7f740809afb0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.592 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.568+0000 7f7424715b80 1 -- 172.21.15.39:0/3590430704 shutdown_connections 2023-07-21T18:43:13.592 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.568+0000 7f7424715b80 1 --2- 172.21.15.39:0/3590430704 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55665f16dad0 0x55665f16feb0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.592 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.568+0000 7f7424715b80 1 --2- 172.21.15.39:0/3590430704 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55665f1622d0 0x55665f1626a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.593 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.568+0000 7f7424715b80 1 --2- 172.21.15.39:0/3590430704 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55665f162be0 0x55665f16d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.593 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.568+0000 7f7424715b80 1 -- 172.21.15.39:0/3590430704 >> 172.21.15.39:0/3590430704 conn(0x55665f158da0 msgr2=0x55665f15e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:13.593 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.568+0000 7f7424715b80 1 -- 172.21.15.39:0/3590430704 shutdown_connections 2023-07-21T18:43:13.593 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.568+0000 7f7424715b80 1 -- 172.21.15.39:0/3590430704 wait complete. 2023-07-21T18:43:13.593 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.568+0000 7f7424715b80 1 -- start start 2023-07-21T18:43:13.593 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.569+0000 7f7424715b80 1 Processor -- start 2023-07-21T18:43:13.596 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.569+0000 7f7424715b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55665f1622d0 0x55665f16dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.596 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.569+0000 7f7424715b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55665f162be0 0x55665f16e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.596 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.569+0000 7f7424715b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55665f16dad0 0x55665f16e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.596 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.569+0000 7f7424715b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55665f054ac0 con 0x55665f162be0 2023-07-21T18:43:13.596 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.569+0000 7f7424715b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55665f0547c0 con 0x55665f16dad0 2023-07-21T18:43:13.597 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.569+0000 7f7424715b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55665ef38920 con 0x55665f1622d0 2023-07-21T18:43:13.597 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.569+0000 7f7424715b80 10 client.0 _ll_register_callbacks cb 0x55665f052760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:43:13.597 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.569+0000 7f7415b2a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55665f162be0 0x55665f16e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.597 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.569+0000 7f7415b2a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55665f162be0 0x55665f16e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58656/0 (socket says 172.21.15.39:58656) 2023-07-21T18:43:13.597 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.569+0000 7f7415b2a700 1 -- 172.21.15.39:0/1868348136 learned_addr learned my addr 172.21.15.39:0/1868348136 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:13.597 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.569+0000 7f741632b700 1 --2- 172.21.15.39:0/1868348136 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55665f1622d0 0x55665f16dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.597 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.569+0000 7f7415b2a700 1 -- 172.21.15.39:0/1868348136 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55665f1622d0 msgr2=0x55665f16dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:13.598 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.569+0000 7f7416b2c700 1 --2- 172.21.15.39:0/1868348136 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55665f16dad0 0x55665f16e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.598 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.569+0000 7f7415b2a700 1 --2- 172.21.15.39:0/1868348136 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55665f1622d0 0x55665f16dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.598 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.569+0000 7f7415b2a700 1 -- 172.21.15.39:0/1868348136 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55665f16dad0 msgr2=0x55665f16e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:13.598 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.569+0000 7f7415b2a700 1 --2- 172.21.15.39:0/1868348136 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55665f16dad0 0x55665f16e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.598 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.569+0000 7f7415b2a700 1 -- 172.21.15.39:0/1868348136 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74101a1a10 con 0x55665f162be0 2023-07-21T18:43:13.598 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.569+0000 7f7416b2c700 1 --2- 172.21.15.39:0/1868348136 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55665f16dad0 0x55665f16e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:43:13.598 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.569+0000 7f7415b2a700 1 --2- 172.21.15.39:0/1868348136 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55665f162be0 0x55665f16e3e0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f740c003930 tx=0x7f740c003c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e37f2455d28066d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:13.598 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.570+0000 7f73f9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:43:13.599 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.570+0000 7f73f9ffb700 1 -- 172.21.15.39:0/1868348136 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740c009e60 con 0x55665f162be0 2023-07-21T18:43:13.599 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.570+0000 7f73f9ffb700 1 -- 172.21.15.39:0/1868348136 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f740c01b470 con 0x55665f162be0 2023-07-21T18:43:13.599 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.570+0000 7f7424715b80 1 -- 172.21.15.39:0/1868348136 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55665f00e4f0 con 0x55665f162be0 2023-07-21T18:43:13.599 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.570+0000 7f73f9ffb700 1 -- 172.21.15.39:0/1868348136 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f740c003e80 con 0x55665f162be0 2023-07-21T18:43:13.599 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.570+0000 7f7424715b80 10 client.16035 fetch_fsmap learned FSMap version 28 2023-07-21T18:43:13.599 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.570+0000 7f7424715b80 1 -- 172.21.15.39:0/1868348136 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55665eecbc60 con 0x55665f162be0 2023-07-21T18:43:13.599 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.570+0000 7f73f9ffb700 1 -- 172.21.15.39:0/1868348136 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55665f00e4f0 con 0x55665f162be0 2023-07-21T18:43:13.599 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.570+0000 7f73f9ffb700 1 -- 172.21.15.39:0/1868348136 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f740c0247c0 con 0x55665f162be0 2023-07-21T18:43:13.600 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.570+0000 7f7424715b80 10 client.16035 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T18:43:13.600 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.570+0000 7f7424715b80 10 client.16035 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T18:43:13.600 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.570+0000 7f7424715b80 20 client.16035 populate_metadata read hostname 'smithi039' 2023-07-21T18:43:13.600 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.570+0000 7f7424715b80 1 --2- 172.21.15.39:0/1868348136 >> [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] conn(0x55665f17d140 0x55665f17f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.600 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.570+0000 7f7424715b80 4 client.16035 mds_command: new command op to 24500 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:43:13.600 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.570+0000 7f7424715b80 1 -- 172.21.15.39:0/1868348136 --> [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55665f0adeb0 con 0x55665f17d140 2023-07-21T18:43:13.600 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.571+0000 7f741632b700 1 --2- 172.21.15.39:0/1868348136 >> [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] conn(0x55665f17d140 0x55665f17f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.600 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.571+0000 7f73f9ffb700 10 client.16035 ms_handle_connect on v2:172.21.15.155:6836/947670056 2023-07-21T18:43:13.601 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.571+0000 7f741632b700 1 --2- 172.21.15.39:0/1868348136 >> [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] conn(0x55665f17d140 0x55665f17f550 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e3a56a817354c994 server_cookie=e03d00a388f991e9 in_seq=0 out_seq=0 2023-07-21T18:43:13.601 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.572+0000 7f73f9ffb700 1 -- 172.21.15.39:0/1868348136 <== mds.1 v2:172.21.15.155:6836/947670056 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55665f0adeb0 con 0x55665f17d140 2023-07-21T18:43:13.601 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.572+0000 7f73f9ffb700 10 client.16035 handle_command_reply: tid=0 2023-07-21T18:43:13.601 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.572+0000 7f7424715b80 1 client.16035 shutdown 2023-07-21T18:43:13.601 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.572+0000 7f7424715b80 1 -- 172.21.15.39:0/1868348136 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55665f162be0 msgr2=0x55665f16e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:13.601 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.572+0000 7f7424715b80 1 --2- 172.21.15.39:0/1868348136 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55665f162be0 0x55665f16e3e0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f740c003930 tx=0x7f740c003c40 comp rx=0 tx=0).stop 2023-07-21T18:43:13.601 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.572+0000 7f7424715b80 1 -- 172.21.15.39:0/1868348136 shutdown_connections 2023-07-21T18:43:13.602 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.572+0000 7f7424715b80 1 --2- 172.21.15.39:0/1868348136 >> [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] conn(0x55665f17d140 0x55665f17f550 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.602 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.572+0000 7f7424715b80 1 --2- 172.21.15.39:0/1868348136 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55665f16dad0 0x55665f16e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.602 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.572+0000 7f7424715b80 1 --2- 172.21.15.39:0/1868348136 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55665f1622d0 0x55665f16dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.602 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.572+0000 7f7424715b80 1 --2- 172.21.15.39:0/1868348136 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55665f162be0 0x55665f16e3e0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.602 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.572+0000 7f73f9ffb700 0 client.16035 ms_handle_reset on v2:172.21.15.155:6836/947670056 2023-07-21T18:43:13.602 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.572+0000 7f7424715b80 1 -- 172.21.15.39:0/1868348136 >> 172.21.15.39:0/1868348136 conn(0x55665f158da0 msgr2=0x55665f15e230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:13.602 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.572+0000 7f7424715b80 1 -- 172.21.15.39:0/1868348136 shutdown_connections 2023-07-21T18:43:13.602 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.572+0000 7f7424715b80 1 -- 172.21.15.39:0/1868348136 wait complete. 2023-07-21T18:43:13.603 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.572+0000 7f7424715b80 20 client.16035 trim_cache size 0 max 16384 2023-07-21T18:43:13.606 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.585+0000 7f7424715b80 1 Processor -- start 2023-07-21T18:43:13.606 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.585+0000 7f7424715b80 1 -- start start 2023-07-21T18:43:13.606 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.585+0000 7f7424715b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55665f1622d0 0x55665f1730a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.606 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.585+0000 7f7424715b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55665f162be0 0x55665f167400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.606 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.585+0000 7f7424715b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55665f16dad0 0x55665f178660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.606 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.585+0000 7f7424715b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55665f16d140 con 0x55665f162be0 2023-07-21T18:43:13.606 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.585+0000 7f7424715b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55665f168710 con 0x55665f1622d0 2023-07-21T18:43:13.607 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.585+0000 7f7424715b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55665f17fbb0 con 0x55665f16dad0 2023-07-21T18:43:13.607 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.585+0000 7f7415b2a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55665f162be0 0x55665f167400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.607 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.585+0000 7f7415b2a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55665f162be0 0x55665f167400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58664/0 (socket says 172.21.15.39:58664) 2023-07-21T18:43:13.607 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.585+0000 7f7415b2a700 1 -- 172.21.15.39:0/1639730725 learned_addr learned my addr 172.21.15.39:0/1639730725 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:13.607 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.586+0000 7f741632b700 1 --2- 172.21.15.39:0/1639730725 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55665f1622d0 0x55665f1730a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.607 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.586+0000 7f7416b2c700 1 --2- 172.21.15.39:0/1639730725 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55665f16dad0 0x55665f178660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.607 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.586+0000 7f7415b2a700 1 -- 172.21.15.39:0/1639730725 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55665f16dad0 msgr2=0x55665f178660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:13.607 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.586+0000 7f7415b2a700 1 --2- 172.21.15.39:0/1639730725 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55665f16dad0 0x55665f178660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.608 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.586+0000 7f7415b2a700 1 -- 172.21.15.39:0/1639730725 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55665f1622d0 msgr2=0x55665f1730a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:13.608 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.586+0000 7f7415b2a700 1 --2- 172.21.15.39:0/1639730725 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55665f1622d0 0x55665f1730a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.608 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.586+0000 7f7415b2a700 1 -- 172.21.15.39:0/1639730725 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55665ef09250 con 0x55665f162be0 2023-07-21T18:43:13.608 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.586+0000 7f741632b700 1 --2- 172.21.15.39:0/1639730725 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55665f1622d0 0x55665f1730a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:43:13.608 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.586+0000 7f7415b2a700 1 --2- 172.21.15.39:0/1639730725 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55665f162be0 0x55665f167400 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f740c000c00 tx=0x7f740c0098a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc62290f4f662f1e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:13.608 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.586+0000 7f74017fa700 1 -- 172.21.15.39:0/1639730725 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740c009e60 con 0x55665f162be0 2023-07-21T18:43:13.608 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.586+0000 7f74017fa700 1 -- 172.21.15.39:0/1639730725 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f740c024920 con 0x55665f162be0 2023-07-21T18:43:13.609 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.586+0000 7f74017fa700 1 -- 172.21.15.39:0/1639730725 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740c012be0 con 0x55665f162be0 2023-07-21T18:43:13.611 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.586+0000 7f7424715b80 1 -- 172.21.15.39:0/1639730725 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55665f162be0 msgr2=0x55665f167400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:13.612 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.587+0000 7f7424715b80 1 --2- 172.21.15.39:0/1639730725 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55665f162be0 0x55665f167400 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f740c000c00 tx=0x7f740c0098a0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.612 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.587+0000 7f7424715b80 1 -- 172.21.15.39:0/1639730725 shutdown_connections 2023-07-21T18:43:13.612 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.587+0000 7f7424715b80 1 --2- 172.21.15.39:0/1639730725 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55665f1622d0 0x55665f1730a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.612 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.587+0000 7f7424715b80 1 --2- 172.21.15.39:0/1639730725 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55665f16dad0 0x55665f178660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.612 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.587+0000 7f7424715b80 1 --2- 172.21.15.39:0/1639730725 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55665f162be0 0x55665f167400 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.612 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.587+0000 7f7424715b80 1 -- 172.21.15.39:0/1639730725 >> 172.21.15.39:0/1639730725 conn(0x55665f158da0 msgr2=0x55665f183e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:13.612 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.587+0000 7f7424715b80 1 -- 172.21.15.39:0/1639730725 shutdown_connections 2023-07-21T18:43:13.612 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.587+0000 7f7424715b80 1 -- 172.21.15.39:0/1639730725 wait complete. 2023-07-21T18:43:13.613 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.587+0000 7f7424715b80 1 -- start start 2023-07-21T18:43:13.613 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.587+0000 7f7424715b80 1 Processor -- start 2023-07-21T18:43:13.613 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.587+0000 7f7424715b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55665f1622d0 0x55665f1cc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.613 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.587+0000 7f7424715b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55665f162be0 0x55665f1cce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.613 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.587+0000 7f7424715b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55665f16dad0 0x55665f1c6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.613 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.587+0000 7f7424715b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55665f1c7570 con 0x55665f1622d0 2023-07-21T18:43:13.613 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.587+0000 7f7424715b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55665f1c76e0 con 0x55665f16dad0 2023-07-21T18:43:13.614 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.587+0000 7f7424715b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55665f1c78a0 con 0x55665f162be0 2023-07-21T18:43:13.614 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.587+0000 7f7424715b80 10 client.0 _ll_register_callbacks cb 0x7f740c004430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:43:13.614 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.587+0000 7f741632b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55665f1622d0 0x55665f1cc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.614 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.587+0000 7f741632b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55665f1622d0 0x55665f1cc940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58668/0 (socket says 172.21.15.39:58668) 2023-07-21T18:43:13.614 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.587+0000 7f741632b700 1 -- 172.21.15.39:0/2573213508 learned_addr learned my addr 172.21.15.39:0/2573213508 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:13.614 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.588+0000 7f7416b2c700 1 --2- 172.21.15.39:0/2573213508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55665f16dad0 0x55665f1c6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.614 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.588+0000 7f7415b2a700 1 --2- 172.21.15.39:0/2573213508 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55665f162be0 0x55665f1cce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.614 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.588+0000 7f741632b700 1 -- 172.21.15.39:0/2573213508 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55665f162be0 msgr2=0x55665f1cce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:13.615 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.588+0000 7f741632b700 1 --2- 172.21.15.39:0/2573213508 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55665f162be0 0x55665f1cce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.615 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.588+0000 7f741632b700 1 -- 172.21.15.39:0/2573213508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55665f16dad0 msgr2=0x55665f1c6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:13.615 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.588+0000 7f741632b700 1 --2- 172.21.15.39:0/2573213508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55665f16dad0 0x55665f1c6f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.615 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.588+0000 7f741632b700 1 -- 172.21.15.39:0/2573213508 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55665eecbc60 con 0x55665f1622d0 2023-07-21T18:43:13.615 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.588+0000 7f741632b700 1 --2- 172.21.15.39:0/2573213508 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55665f1622d0 0x55665f1cc940 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f740400ba40 tx=0x7f74040047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6af48f9d10309056 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:13.615 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.588+0000 7f73f9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:43:13.615 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.588+0000 7f73f9ffb700 1 -- 172.21.15.39:0/2573213508 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7404025050 con 0x55665f1622d0 2023-07-21T18:43:13.615 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.588+0000 7f73f9ffb700 1 -- 172.21.15.39:0/2573213508 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f7404027220 con 0x55665f1622d0 2023-07-21T18:43:13.616 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.588+0000 7f7424715b80 1 -- 172.21.15.39:0/2573213508 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f740809f190 con 0x55665f1622d0 2023-07-21T18:43:13.616 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.588+0000 7f73f9ffb700 1 -- 172.21.15.39:0/2573213508 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74040273e0 con 0x55665f1622d0 2023-07-21T18:43:13.616 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.588+0000 7f7424715b80 10 client.16041 fetch_fsmap learned FSMap version 28 2023-07-21T18:43:13.616 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.588+0000 7f7424715b80 1 -- 172.21.15.39:0/2573213508 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55665eee12b0 con 0x55665f1622d0 2023-07-21T18:43:13.616 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.589+0000 7f73f9ffb700 1 -- 172.21.15.39:0/2573213508 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f740809f190 con 0x55665f1622d0 2023-07-21T18:43:13.616 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.589+0000 7f73f9ffb700 1 -- 172.21.15.39:0/2573213508 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f74040307e0 con 0x55665f1622d0 2023-07-21T18:43:13.616 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.589+0000 7f7424715b80 10 client.16041 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T18:43:13.616 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.589+0000 7f7424715b80 10 client.16041 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T18:43:13.617 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.589+0000 7f7424715b80 20 client.16041 populate_metadata read hostname 'smithi039' 2023-07-21T18:43:13.617 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.589+0000 7f7424715b80 1 --2- 172.21.15.39:0/2573213508 >> [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] conn(0x55665f1d47a0 0x55665f1d6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.617 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.589+0000 7f7424715b80 4 client.16041 mds_command: new command op to 24500 tid=0[{"prefix": "damage ls"}] 2023-07-21T18:43:13.617 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.589+0000 7f7424715b80 1 -- 172.21.15.39:0/2573213508 --> [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55665f070050 con 0x55665f1d47a0 2023-07-21T18:43:13.617 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.589+0000 7f7415b2a700 1 --2- 172.21.15.39:0/2573213508 >> [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] conn(0x55665f1d47a0 0x55665f1d6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.617 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.589+0000 7f73f9ffb700 10 client.16041 ms_handle_connect on v2:172.21.15.155:6836/947670056 2023-07-21T18:43:13.617 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.589+0000 7f7415b2a700 1 --2- 172.21.15.39:0/2573213508 >> [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] conn(0x55665f1d47a0 0x55665f1d6bb0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9ae4ff8174f38a5c server_cookie=9265a1c153292825 in_seq=0 out_seq=0 2023-07-21T18:43:13.618 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.589+0000 7f73f9ffb700 1 -- 172.21.15.39:0/2573213508 <== mds.1 v2:172.21.15.155:6836/947670056 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55665f070050 con 0x55665f1d47a0 2023-07-21T18:43:13.618 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.589+0000 7f73f9ffb700 10 client.16041 handle_command_reply: tid=0 2023-07-21T18:43:13.618 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.589+0000 7f7424715b80 1 client.16041 shutdown 2023-07-21T18:43:13.618 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.590+0000 7f7424715b80 1 -- 172.21.15.39:0/2573213508 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55665f1622d0 msgr2=0x55665f1cc940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:13.618 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.590+0000 7f7424715b80 1 --2- 172.21.15.39:0/2573213508 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55665f1622d0 0x55665f1cc940 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f740400ba40 tx=0x7f74040047c0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.618 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.590+0000 7f7424715b80 1 -- 172.21.15.39:0/2573213508 shutdown_connections 2023-07-21T18:43:13.618 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.590+0000 7f7424715b80 1 --2- 172.21.15.39:0/2573213508 >> [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] conn(0x55665f1d47a0 0x55665f1d6bb0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.618 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.590+0000 7f7424715b80 1 --2- 172.21.15.39:0/2573213508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55665f16dad0 0x55665f1c6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.619 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.590+0000 7f7424715b80 1 --2- 172.21.15.39:0/2573213508 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55665f1622d0 0x55665f1cc940 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.619 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.590+0000 7f7424715b80 1 --2- 172.21.15.39:0/2573213508 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55665f162be0 0x55665f1cce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.619 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.590+0000 7f73f9ffb700 0 client.16041 ms_handle_reset on v2:172.21.15.155:6836/947670056 2023-07-21T18:43:13.619 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.590+0000 7f7424715b80 1 -- 172.21.15.39:0/2573213508 >> 172.21.15.39:0/2573213508 conn(0x55665f158da0 msgr2=0x55665f16feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:13.619 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.590+0000 7f7424715b80 1 -- 172.21.15.39:0/2573213508 shutdown_connections 2023-07-21T18:43:13.619 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.590+0000 7f7424715b80 1 -- 172.21.15.39:0/2573213508 wait complete. 2023-07-21T18:43:13.619 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.590+0000 7f7424715b80 20 client.16041 trim_cache size 0 max 16384 2023-07-21T18:43:13.620 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2023-07-21T18:43:13.620 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.590+0000 7f741858f700 1 -- 172.21.15.39:0/3592488002 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f73fc0a13e0 msgr2=0x7f73fc0a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:13.620 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.590+0000 7f741858f700 1 --2- 172.21.15.39:0/3592488002 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f73fc0a13e0 0x7f73fc0a3890 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7404005950 tx=0x7f74040058e0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.621 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.590+0000 7f741858f700 1 -- 172.21.15.39:0/3592488002 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7410103ce0 msgr2=0x7f7410194a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:13.621 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.590+0000 7f741858f700 1 --2- 172.21.15.39:0/3592488002 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7410103ce0 0x7f7410194a90 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f7408007870 tx=0x7f7408007950 comp rx=0 tx=0).stop 2023-07-21T18:43:13.621 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.590+0000 7f741858f700 1 -- 172.21.15.39:0/3592488002 shutdown_connections 2023-07-21T18:43:13.621 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.590+0000 7f741858f700 1 --2- 172.21.15.39:0/3592488002 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7410103ce0 0x7f7410194a90 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.621 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.590+0000 7f741858f700 1 --2- 172.21.15.39:0/3592488002 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74100ff580 0x7f741019aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.621 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.590+0000 7f741858f700 1 --2- 172.21.15.39:0/3592488002 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f73fc0a13e0 0x7f73fc0a3890 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.621 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.590+0000 7f741858f700 1 --2- 172.21.15.39:0/3592488002 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74100fff20 0x7f741019af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.622 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.590+0000 7f741858f700 1 -- 172.21.15.39:0/3592488002 >> 172.21.15.39:0/3592488002 conn(0x7f7410076200 msgr2=0x7f74100fbc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:13.622 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.590+0000 7f741858f700 1 -- 172.21.15.39:0/3592488002 shutdown_connections 2023-07-21T18:43:13.622 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.590+0000 7f741858f700 1 -- 172.21.15.39:0/3592488002 wait complete. 2023-07-21T18:43:13.622 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:13 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/817293471' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:43:13.623 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-21T18:43:13.636 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:13 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/817293471' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:43:13.792 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.771+0000 7f8f73ee9700 1 -- 172.21.15.39:0/3628270574 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f6c0fe9f0 msgr2=0x7f8f6c0fee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:13.792 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.772+0000 7f8f73ee9700 1 --2- 172.21.15.39:0/3628270574 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f6c0fe9f0 0x7f8f6c0fee60 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c009990 tx=0x7f8f5c009ca0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.792 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.772+0000 7f8f73ee9700 1 -- 172.21.15.39:0/3628270574 shutdown_connections 2023-07-21T18:43:13.792 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.772+0000 7f8f73ee9700 1 --2- 172.21.15.39:0/3628270574 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f6c104a00 0x7f8f6c104dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.792 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.772+0000 7f8f73ee9700 1 --2- 172.21.15.39:0/3628270574 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f6c0ff3a0 0x7f8f6c10b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.792 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.772+0000 7f8f73ee9700 1 --2- 172.21.15.39:0/3628270574 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f6c0fe9f0 0x7f8f6c0fee60 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.793 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.772+0000 7f8f73ee9700 1 -- 172.21.15.39:0/3628270574 >> 172.21.15.39:0/3628270574 conn(0x7f8f6c0fa4d0 msgr2=0x7f8f6c0fc8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:13.793 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.772+0000 7f8f73ee9700 1 -- 172.21.15.39:0/3628270574 shutdown_connections 2023-07-21T18:43:13.793 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.772+0000 7f8f73ee9700 1 -- 172.21.15.39:0/3628270574 wait complete. 2023-07-21T18:43:13.794 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.772+0000 7f8f73ee9700 1 Processor -- start 2023-07-21T18:43:13.795 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.772+0000 7f8f73ee9700 1 -- start start 2023-07-21T18:43:13.795 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.772+0000 7f8f73ee9700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f6c0fe9f0 0x7f8f6c19a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.795 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.772+0000 7f8f73ee9700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f6c0ff3a0 0x7f8f6c19adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.795 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.773+0000 7f8f73ee9700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f6c104a00 0x7f8f6c194920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.795 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.773+0000 7f8f73ee9700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6c194f90 con 0x7f8f6c104a00 2023-07-21T18:43:13.795 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.773+0000 7f8f73ee9700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6c195100 con 0x7f8f6c0fe9f0 2023-07-21T18:43:13.795 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.773+0000 7f8f73ee9700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6c1952c0 con 0x7f8f6c0ff3a0 2023-07-21T18:43:13.796 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.773+0000 7f8f72486700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f6c104a00 0x7f8f6c194920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.796 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.773+0000 7f8f72486700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f6c104a00 0x7f8f6c194920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58686/0 (socket says 172.21.15.39:58686) 2023-07-21T18:43:13.796 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.773+0000 7f8f72486700 1 -- 172.21.15.39:0/2185104681 learned_addr learned my addr 172.21.15.39:0/2185104681 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:13.796 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.773+0000 7f8f71484700 1 --2- 172.21.15.39:0/2185104681 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f6c0ff3a0 0x7f8f6c19adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.796 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.773+0000 7f8f71c85700 1 --2- 172.21.15.39:0/2185104681 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f6c0fe9f0 0x7f8f6c19a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.796 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.773+0000 7f8f72486700 1 -- 172.21.15.39:0/2185104681 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f6c0ff3a0 msgr2=0x7f8f6c19adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:13.796 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.773+0000 7f8f72486700 1 --2- 172.21.15.39:0/2185104681 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f6c0ff3a0 0x7f8f6c19adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.796 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.773+0000 7f8f72486700 1 -- 172.21.15.39:0/2185104681 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f6c0fe9f0 msgr2=0x7f8f6c19a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:13.797 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.773+0000 7f8f72486700 1 --2- 172.21.15.39:0/2185104681 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f6c0fe9f0 0x7f8f6c19a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.797 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.773+0000 7f8f72486700 1 -- 172.21.15.39:0/2185104681 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f5c0095f0 con 0x7f8f6c104a00 2023-07-21T18:43:13.797 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.773+0000 7f8f71484700 1 --2- 172.21.15.39:0/2185104681 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f6c0ff3a0 0x7f8f6c19adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:43:13.797 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.773+0000 7f8f72486700 1 --2- 172.21.15.39:0/2185104681 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f6c104a00 0x7f8f6c194920 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f8f64007ee0 tx=0x7f8f6400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:13.797 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.773+0000 7f8f62ffd700 1 -- 172.21.15.39:0/2185104681 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f64017070 con 0x7f8f6c104a00 2023-07-21T18:43:13.797 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.773+0000 7f8f62ffd700 1 -- 172.21.15.39:0/2185104681 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f6400f040 con 0x7f8f6c104a00 2023-07-21T18:43:13.797 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.773+0000 7f8f73ee9700 1 -- 172.21.15.39:0/2185104681 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f6c1955a0 con 0x7f8f6c104a00 2023-07-21T18:43:13.797 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.773+0000 7f8f62ffd700 1 -- 172.21.15.39:0/2185104681 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f64005740 con 0x7f8f6c104a00 2023-07-21T18:43:13.798 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.773+0000 7f8f73ee9700 1 -- 172.21.15.39:0/2185104681 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f6c1a18a0 con 0x7f8f6c104a00 2023-07-21T18:43:13.802 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.776+0000 7f8f62ffd700 1 -- 172.21.15.39:0/2185104681 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f8f640058a0 con 0x7f8f6c104a00 2023-07-21T18:43:13.802 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.776+0000 7f8f62ffd700 1 --2- 172.21.15.39:0/2185104681 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f8f580a1460 0x7f8f580a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.802 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.777+0000 7f8f71c85700 1 --2- 172.21.15.39:0/2185104681 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f8f580a1460 0x7f8f580a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.802 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.777+0000 7f8f62ffd700 1 -- 172.21.15.39:0/2185104681 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f8f640cc7c0 con 0x7f8f6c104a00 2023-07-21T18:43:13.802 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.777+0000 7f8f71c85700 1 --2- 172.21.15.39:0/2185104681 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f8f580a1460 0x7f8f580a3910 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8f68005fd0 tx=0x7f8f6800a4c0 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:13.802 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.777+0000 7f8f8006fb80 1 Processor -- start 2023-07-21T18:43:13.802 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.778+0000 7f8f8006fb80 1 -- start start 2023-07-21T18:43:13.803 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.778+0000 7f8f8006fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578a0b444e0 0x5578a0b448b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.803 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.778+0000 7f8f8006fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5578a0b44df0 0x5578a0b51b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.803 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.778+0000 7f8f8006fb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5578a0b520d0 0x5578a0b544b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.803 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.778+0000 7f8f8006fb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578a0a48cb0 con 0x5578a0b520d0 2023-07-21T18:43:13.803 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.778+0000 7f8f8006fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578a0a9ba40 con 0x5578a0b44df0 2023-07-21T18:43:13.803 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.778+0000 7f8f8006fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578a0a94ed0 con 0x5578a0b444e0 2023-07-21T18:43:13.803 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.778+0000 7f8f72486700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5578a0b520d0 0x5578a0b544b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.803 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.778+0000 7f8f72486700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5578a0b520d0 0x5578a0b544b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58694/0 (socket says 172.21.15.39:58694) 2023-07-21T18:43:13.804 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.778+0000 7f8f72486700 1 -- 172.21.15.39:0/1505714733 learned_addr learned my addr 172.21.15.39:0/1505714733 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:13.804 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.778+0000 7f8f71c85700 1 --2- 172.21.15.39:0/1505714733 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578a0b444e0 0x5578a0b448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.804 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.778+0000 7f8f71484700 1 --2- 172.21.15.39:0/1505714733 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5578a0b44df0 0x5578a0b51b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.804 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.778+0000 7f8f72486700 1 -- 172.21.15.39:0/1505714733 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578a0b444e0 msgr2=0x5578a0b448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:13.804 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.778+0000 7f8f72486700 1 --2- 172.21.15.39:0/1505714733 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578a0b444e0 0x5578a0b448b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.804 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.778+0000 7f8f72486700 1 -- 172.21.15.39:0/1505714733 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5578a0b44df0 msgr2=0x5578a0b51b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:13.804 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.778+0000 7f8f72486700 1 --2- 172.21.15.39:0/1505714733 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5578a0b44df0 0x5578a0b51b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.805 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.778+0000 7f8f72486700 1 -- 172.21.15.39:0/1505714733 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5578a08c2de0 con 0x5578a0b520d0 2023-07-21T18:43:13.805 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.779+0000 7f8f72486700 1 --2- 172.21.15.39:0/1505714733 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5578a0b520d0 0x5578a0b544b0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f8f6400d7b0 tx=0x7f8f640953a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc3d766e7d62296e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:13.805 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.779+0000 7f8f57fff700 1 -- 172.21.15.39:0/1505714733 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6409d690 con 0x5578a0b520d0 2023-07-21T18:43:13.805 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.779+0000 7f8f57fff700 1 -- 172.21.15.39:0/1505714733 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f640a1ab0 con 0x5578a0b520d0 2023-07-21T18:43:13.805 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.779+0000 7f8f57fff700 1 -- 172.21.15.39:0/1505714733 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f640998c0 con 0x5578a0b520d0 2023-07-21T18:43:13.805 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.779+0000 7f8f8006fb80 1 -- 172.21.15.39:0/1505714733 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5578a0b520d0 msgr2=0x5578a0b544b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:13.805 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.779+0000 7f8f8006fb80 1 --2- 172.21.15.39:0/1505714733 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5578a0b520d0 0x5578a0b544b0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f8f6400d7b0 tx=0x7f8f640953a0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.805 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.779+0000 7f8f8006fb80 1 -- 172.21.15.39:0/1505714733 shutdown_connections 2023-07-21T18:43:13.806 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.779+0000 7f8f8006fb80 1 --2- 172.21.15.39:0/1505714733 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5578a0b520d0 0x5578a0b544b0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.806 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.779+0000 7f8f8006fb80 1 --2- 172.21.15.39:0/1505714733 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578a0b444e0 0x5578a0b448b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.806 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.779+0000 7f8f8006fb80 1 --2- 172.21.15.39:0/1505714733 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5578a0b44df0 0x5578a0b51b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.806 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.779+0000 7f8f8006fb80 1 -- 172.21.15.39:0/1505714733 >> 172.21.15.39:0/1505714733 conn(0x5578a0b38da0 msgr2=0x5578a0b3e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:13.806 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.780+0000 7f8f8006fb80 1 -- 172.21.15.39:0/1505714733 shutdown_connections 2023-07-21T18:43:13.806 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.780+0000 7f8f8006fb80 1 -- 172.21.15.39:0/1505714733 wait complete. 2023-07-21T18:43:13.806 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.780+0000 7f8f8006fb80 1 -- start start 2023-07-21T18:43:13.806 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.780+0000 7f8f8006fb80 1 Processor -- start 2023-07-21T18:43:13.807 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.780+0000 7f8f8006fb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5578a0b444e0 0x5578a0b524a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.807 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.780+0000 7f8f8006fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578a0b44df0 0x5578a0b529e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.807 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.780+0000 7f8f8006fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5578a0b520d0 0x5578a0b5d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.807 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.780+0000 7f8f8006fb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578a0a34ac0 con 0x5578a0b444e0 2023-07-21T18:43:13.807 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.780+0000 7f8f8006fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578a0a347c0 con 0x5578a0b520d0 2023-07-21T18:43:13.807 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.780+0000 7f8f8006fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578a0918920 con 0x5578a0b44df0 2023-07-21T18:43:13.807 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.780+0000 7f8f8006fb80 10 client.0 _ll_register_callbacks cb 0x5578a0a32760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:43:13.807 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.780+0000 7f8f71c85700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5578a0b444e0 0x5578a0b524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.808 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.781+0000 7f8f71c85700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5578a0b444e0 0x5578a0b524a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58696/0 (socket says 172.21.15.39:58696) 2023-07-21T18:43:13.808 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.781+0000 7f8f71c85700 1 -- 172.21.15.39:0/2591300035 learned_addr learned my addr 172.21.15.39:0/2591300035 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:13.808 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.781+0000 7f8f72486700 1 --2- 172.21.15.39:0/2591300035 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5578a0b520d0 0x5578a0b5d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.808 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.781+0000 7f8f71484700 1 --2- 172.21.15.39:0/2591300035 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578a0b44df0 0x5578a0b529e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.808 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.781+0000 7f8f71c85700 1 -- 172.21.15.39:0/2591300035 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578a0b44df0 msgr2=0x5578a0b529e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:13.808 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.781+0000 7f8f71c85700 1 --2- 172.21.15.39:0/2591300035 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578a0b44df0 0x5578a0b529e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.808 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.781+0000 7f8f71c85700 1 -- 172.21.15.39:0/2591300035 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5578a0b520d0 msgr2=0x5578a0b5d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:13.808 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.781+0000 7f8f71c85700 1 --2- 172.21.15.39:0/2591300035 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5578a0b520d0 0x5578a0b5d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.809 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.781+0000 7f8f71c85700 1 -- 172.21.15.39:0/2591300035 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f6c1a18a0 con 0x5578a0b444e0 2023-07-21T18:43:13.809 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.781+0000 7f8f71484700 1 --2- 172.21.15.39:0/2591300035 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578a0b44df0 0x5578a0b529e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:43:13.809 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.781+0000 7f8f72486700 1 --2- 172.21.15.39:0/2591300035 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5578a0b520d0 0x5578a0b5d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:43:13.809 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.781+0000 7f8f71c85700 1 --2- 172.21.15.39:0/2591300035 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5578a0b444e0 0x5578a0b524a0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f8f68010790 tx=0x7f8f68004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3275d3c2616c74c8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:13.809 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.781+0000 7f8f557fa700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:43:13.809 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.781+0000 7f8f557fa700 1 -- 172.21.15.39:0/2591300035 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f68004440 con 0x5578a0b444e0 2023-07-21T18:43:13.809 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.781+0000 7f8f557fa700 1 -- 172.21.15.39:0/2591300035 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f8f68003680 con 0x5578a0b444e0 2023-07-21T18:43:13.810 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.781+0000 7f8f8006fb80 1 -- 172.21.15.39:0/2591300035 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5578a09ee4f0 con 0x5578a0b444e0 2023-07-21T18:43:13.810 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.781+0000 7f8f557fa700 1 -- 172.21.15.39:0/2591300035 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f68003840 con 0x5578a0b444e0 2023-07-21T18:43:13.810 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.781+0000 7f8f8006fb80 10 client.16053 fetch_fsmap learned FSMap version 28 2023-07-21T18:43:13.810 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.781+0000 7f8f8006fb80 1 -- 172.21.15.39:0/2591300035 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5578a08abc60 con 0x5578a0b444e0 2023-07-21T18:43:13.810 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.782+0000 7f8f557fa700 1 -- 172.21.15.39:0/2591300035 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5578a09ee4f0 con 0x5578a0b444e0 2023-07-21T18:43:13.810 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.782+0000 7f8f557fa700 1 -- 172.21.15.39:0/2591300035 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f8f6801dae0 con 0x5578a0b444e0 2023-07-21T18:43:13.810 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.782+0000 7f8f8006fb80 10 client.16053 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T18:43:13.810 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.782+0000 7f8f8006fb80 10 client.16053 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T18:43:13.811 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.782+0000 7f8f8006fb80 20 client.16053 populate_metadata read hostname 'smithi039' 2023-07-21T18:43:13.811 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.782+0000 7f8f8006fb80 1 --2- 172.21.15.39:0/2591300035 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5578a0b618e0 0x5578a0b63cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.811 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.782+0000 7f8f8006fb80 4 client.16053 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:43:13.811 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.782+0000 7f8f8006fb80 1 -- 172.21.15.39:0/2591300035 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5578a0a8deb0 con 0x5578a0b618e0 2023-07-21T18:43:13.812 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.782+0000 7f8f71484700 1 --2- 172.21.15.39:0/2591300035 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5578a0b618e0 0x5578a0b63cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.812 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.783+0000 7f8f557fa700 10 client.16053 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:43:13.812 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.783+0000 7f8f71484700 1 --2- 172.21.15.39:0/2591300035 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5578a0b618e0 0x5578a0b63cf0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec71b99d0adec30d server_cookie=d2fea9d3ce87fc33 in_seq=0 out_seq=0 2023-07-21T18:43:13.812 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.783+0000 7f8f557fa700 1 -- 172.21.15.39:0/2591300035 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5578a0a8deb0 con 0x5578a0b618e0 2023-07-21T18:43:13.813 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.783+0000 7f8f557fa700 10 client.16053 handle_command_reply: tid=0 2023-07-21T18:43:13.813 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.783+0000 7f8f8006fb80 1 client.16053 shutdown 2023-07-21T18:43:13.813 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.784+0000 7f8f8006fb80 1 -- 172.21.15.39:0/2591300035 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5578a0b444e0 msgr2=0x5578a0b524a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:13.813 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.784+0000 7f8f8006fb80 1 --2- 172.21.15.39:0/2591300035 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5578a0b444e0 0x5578a0b524a0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f8f68010790 tx=0x7f8f68004750 comp rx=0 tx=0).stop 2023-07-21T18:43:13.813 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.784+0000 7f8f8006fb80 1 -- 172.21.15.39:0/2591300035 shutdown_connections 2023-07-21T18:43:13.813 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.784+0000 7f8f8006fb80 1 --2- 172.21.15.39:0/2591300035 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5578a0b618e0 0x5578a0b63cf0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.813 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.784+0000 7f8f8006fb80 1 --2- 172.21.15.39:0/2591300035 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5578a0b520d0 0x5578a0b5d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.813 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.784+0000 7f8f557fa700 0 client.16053 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:43:13.814 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.784+0000 7f8f8006fb80 1 --2- 172.21.15.39:0/2591300035 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5578a0b444e0 0x5578a0b524a0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.814 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.784+0000 7f8f8006fb80 1 --2- 172.21.15.39:0/2591300035 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578a0b44df0 0x5578a0b529e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.814 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.784+0000 7f8f8006fb80 1 -- 172.21.15.39:0/2591300035 >> 172.21.15.39:0/2591300035 conn(0x5578a0b38da0 msgr2=0x5578a0b3e230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:13.814 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.784+0000 7f8f8006fb80 1 -- 172.21.15.39:0/2591300035 shutdown_connections 2023-07-21T18:43:13.814 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.784+0000 7f8f8006fb80 1 -- 172.21.15.39:0/2591300035 wait complete. 2023-07-21T18:43:13.814 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.784+0000 7f8f8006fb80 20 client.16053 trim_cache size 0 max 16384 2023-07-21T18:43:13.817 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.797+0000 7f8f8006fb80 1 Processor -- start 2023-07-21T18:43:13.817 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.797+0000 7f8f8006fb80 1 -- start start 2023-07-21T18:43:13.817 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.797+0000 7f8f8006fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5578a0b444e0 0x5578a0b524a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.817 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.797+0000 7f8f8006fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578a0b44df0 0x5578a0b529e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.817 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.797+0000 7f8f8006fb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5578a0b520d0 0x5578a0b5cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.817 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.797+0000 7f8f8006fb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578a0b38970 con 0x5578a0b520d0 2023-07-21T18:43:13.817 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.797+0000 7f8f8006fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578a0b518a0 con 0x5578a0b444e0 2023-07-21T18:43:13.818 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.797+0000 7f8f8006fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578a0b421f0 con 0x5578a0b44df0 2023-07-21T18:43:13.818 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.797+0000 7f8f72486700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5578a0b520d0 0x5578a0b5cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.818 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.797+0000 7f8f72486700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5578a0b520d0 0x5578a0b5cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58708/0 (socket says 172.21.15.39:58708) 2023-07-21T18:43:13.818 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.797+0000 7f8f72486700 1 -- 172.21.15.39:0/1204606522 learned_addr learned my addr 172.21.15.39:0/1204606522 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:13.821 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.797+0000 7f8f71484700 1 --2- 172.21.15.39:0/1204606522 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578a0b44df0 0x5578a0b529e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.821 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.797+0000 7f8f72486700 1 -- 172.21.15.39:0/1204606522 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578a0b44df0 msgr2=0x5578a0b529e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:13.821 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.797+0000 7f8f72486700 1 --2- 172.21.15.39:0/1204606522 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578a0b44df0 0x5578a0b529e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.821 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.797+0000 7f8f72486700 1 -- 172.21.15.39:0/1204606522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5578a0b444e0 msgr2=0x5578a0b524a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:13.821 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.797+0000 7f8f72486700 1 --2- 172.21.15.39:0/1204606522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5578a0b444e0 0x5578a0b524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.822 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.797+0000 7f8f72486700 1 -- 172.21.15.39:0/1204606522 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5578a08e9250 con 0x5578a0b520d0 2023-07-21T18:43:13.822 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.797+0000 7f8f72486700 1 --2- 172.21.15.39:0/1204606522 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5578a0b520d0 0x5578a0b5cc90 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f8f64009510 tx=0x7f8f64015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc5047178399683d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:13.822 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.798+0000 7f8f60ff9700 1 -- 172.21.15.39:0/1204606522 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6409d690 con 0x5578a0b520d0 2023-07-21T18:43:13.822 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.798+0000 7f8f60ff9700 1 -- 172.21.15.39:0/1204606522 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f640c3e90 con 0x5578a0b520d0 2023-07-21T18:43:13.822 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.798+0000 7f8f60ff9700 1 -- 172.21.15.39:0/1204606522 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f64098db0 con 0x5578a0b520d0 2023-07-21T18:43:13.822 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.798+0000 7f8f8006fb80 1 -- 172.21.15.39:0/1204606522 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5578a0b520d0 msgr2=0x5578a0b5cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:13.822 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.798+0000 7f8f8006fb80 1 --2- 172.21.15.39:0/1204606522 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5578a0b520d0 0x5578a0b5cc90 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f8f64009510 tx=0x7f8f64015040 comp rx=0 tx=0).stop 2023-07-21T18:43:13.823 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.798+0000 7f8f8006fb80 1 -- 172.21.15.39:0/1204606522 shutdown_connections 2023-07-21T18:43:13.823 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.798+0000 7f8f8006fb80 1 --2- 172.21.15.39:0/1204606522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5578a0b444e0 0x5578a0b524a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.823 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.798+0000 7f8f8006fb80 1 --2- 172.21.15.39:0/1204606522 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5578a0b520d0 0x5578a0b5cc90 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.823 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.798+0000 7f8f8006fb80 1 --2- 172.21.15.39:0/1204606522 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578a0b44df0 0x5578a0b529e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.823 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.798+0000 7f8f8006fb80 1 -- 172.21.15.39:0/1204606522 >> 172.21.15.39:0/1204606522 conn(0x5578a0b38da0 msgr2=0x5578a0b5dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:13.823 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.798+0000 7f8f8006fb80 1 -- 172.21.15.39:0/1204606522 shutdown_connections 2023-07-21T18:43:13.823 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.798+0000 7f8f8006fb80 1 -- 172.21.15.39:0/1204606522 wait complete. 2023-07-21T18:43:13.823 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.799+0000 7f8f8006fb80 1 -- start start 2023-07-21T18:43:13.824 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.799+0000 7f8f8006fb80 1 Processor -- start 2023-07-21T18:43:13.824 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.799+0000 7f8f8006fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578a0b444e0 0x5578a0b55a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.824 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.799+0000 7f8f8006fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5578a0b44df0 0x5578a0b55f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.824 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.799+0000 7f8f8006fb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5578a0b520d0 0x5578a0b564c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.824 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.799+0000 7f8f8006fb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578a0b57a90 con 0x5578a0b520d0 2023-07-21T18:43:13.824 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.799+0000 7f8f8006fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578a0bb5770 con 0x5578a0b44df0 2023-07-21T18:43:13.824 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.799+0000 7f8f8006fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578a0bb58b0 con 0x5578a0b444e0 2023-07-21T18:43:13.824 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.799+0000 7f8f8006fb80 10 client.0 _ll_register_callbacks cb 0x7f8f6800ed40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:43:13.825 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.799+0000 7f8f72486700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5578a0b520d0 0x5578a0b564c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.825 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.799+0000 7f8f72486700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5578a0b520d0 0x5578a0b564c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58714/0 (socket says 172.21.15.39:58714) 2023-07-21T18:43:13.825 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.799+0000 7f8f72486700 1 -- 172.21.15.39:0/1423433215 learned_addr learned my addr 172.21.15.39:0/1423433215 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:13.825 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.799+0000 7f8f71c85700 1 --2- 172.21.15.39:0/1423433215 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578a0b444e0 0x5578a0b55a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.825 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.799+0000 7f8f71484700 1 --2- 172.21.15.39:0/1423433215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5578a0b44df0 0x5578a0b55f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.825 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.800+0000 7f8f72486700 1 -- 172.21.15.39:0/1423433215 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578a0b444e0 msgr2=0x5578a0b55a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:13.825 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.800+0000 7f8f72486700 1 --2- 172.21.15.39:0/1423433215 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578a0b444e0 0x5578a0b55a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.825 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.800+0000 7f8f72486700 1 -- 172.21.15.39:0/1423433215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5578a0b44df0 msgr2=0x5578a0b55f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:13.826 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.800+0000 7f8f72486700 1 --2- 172.21.15.39:0/1423433215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5578a0b44df0 0x5578a0b55f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.826 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.800+0000 7f8f72486700 1 -- 172.21.15.39:0/1423433215 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5578a08c2de0 con 0x5578a0b520d0 2023-07-21T18:43:13.826 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.800+0000 7f8f72486700 1 --2- 172.21.15.39:0/1423433215 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5578a0b520d0 0x5578a0b564c0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f8f64095b10 tx=0x7f8f640c9d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=893353386c6e501c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:13.826 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.800+0000 7f8f557fa700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:43:13.826 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.800+0000 7f8f557fa700 1 -- 172.21.15.39:0/1423433215 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f640998c0 con 0x5578a0b520d0 2023-07-21T18:43:13.826 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.800+0000 7f8f557fa700 1 -- 172.21.15.39:0/1423433215 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f8f64099a20 con 0x5578a0b520d0 2023-07-21T18:43:13.826 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.800+0000 7f8f8006fb80 1 -- 172.21.15.39:0/1423433215 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8f640a1ab0 con 0x5578a0b520d0 2023-07-21T18:43:13.826 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.800+0000 7f8f557fa700 1 -- 172.21.15.39:0/1423433215 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f64098b50 con 0x5578a0b520d0 2023-07-21T18:43:13.827 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.800+0000 7f8f8006fb80 10 client.16059 fetch_fsmap learned FSMap version 28 2023-07-21T18:43:13.827 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.800+0000 7f8f8006fb80 1 -- 172.21.15.39:0/1423433215 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5578a08c12b0 con 0x5578a0b520d0 2023-07-21T18:43:13.827 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.800+0000 7f8f557fa700 1 -- 172.21.15.39:0/1423433215 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f640a1ab0 con 0x5578a0b520d0 2023-07-21T18:43:13.827 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.800+0000 7f8f557fa700 1 -- 172.21.15.39:0/1423433215 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f8f640ca090 con 0x5578a0b520d0 2023-07-21T18:43:13.827 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.800+0000 7f8f8006fb80 10 client.16059 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T18:43:13.827 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.800+0000 7f8f8006fb80 10 client.16059 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T18:43:13.827 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.800+0000 7f8f8006fb80 20 client.16059 populate_metadata read hostname 'smithi039' 2023-07-21T18:43:13.828 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.800+0000 7f8f8006fb80 1 --2- 172.21.15.39:0/1423433215 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5578a0bb6ab0 0x5578a0bb8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:13.828 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.800+0000 7f8f8006fb80 4 client.16059 mds_command: new command op to 24536 tid=0[{"prefix": "damage ls"}] 2023-07-21T18:43:13.828 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.801+0000 7f8f8006fb80 1 -- 172.21.15.39:0/1423433215 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5578a0a50050 con 0x5578a0bb6ab0 2023-07-21T18:43:13.828 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.801+0000 7f8f71c85700 1 --2- 172.21.15.39:0/1423433215 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5578a0bb6ab0 0x5578a0bb8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:13.828 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.801+0000 7f8f557fa700 10 client.16059 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:43:13.828 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.801+0000 7f8f71c85700 1 --2- 172.21.15.39:0/1423433215 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5578a0bb6ab0 0x5578a0bb8e90 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f27cf076872df09f server_cookie=c7ae9d63130684ee in_seq=0 out_seq=0 2023-07-21T18:43:13.829 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.802+0000 7f8f557fa700 1 -- 172.21.15.39:0/1423433215 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5578a0a50050 con 0x5578a0bb6ab0 2023-07-21T18:43:13.829 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.802+0000 7f8f557fa700 10 client.16059 handle_command_reply: tid=0 2023-07-21T18:43:13.829 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.802+0000 7f8f8006fb80 1 client.16059 shutdown 2023-07-21T18:43:13.830 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.802+0000 7f8f8006fb80 1 -- 172.21.15.39:0/1423433215 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5578a0b520d0 msgr2=0x5578a0b564c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:13.830 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.802+0000 7f8f8006fb80 1 --2- 172.21.15.39:0/1423433215 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5578a0b520d0 0x5578a0b564c0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f8f64095b10 tx=0x7f8f640c9d70 comp rx=0 tx=0).stop 2023-07-21T18:43:13.830 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.802+0000 7f8f8006fb80 1 -- 172.21.15.39:0/1423433215 shutdown_connections 2023-07-21T18:43:13.830 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.802+0000 7f8f8006fb80 1 --2- 172.21.15.39:0/1423433215 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5578a0bb6ab0 0x5578a0bb8e90 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.830 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.802+0000 7f8f8006fb80 1 --2- 172.21.15.39:0/1423433215 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5578a0b520d0 0x5578a0b564c0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.830 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.802+0000 7f8f8006fb80 1 --2- 172.21.15.39:0/1423433215 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578a0b444e0 0x5578a0b55a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.830 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.802+0000 7f8f8006fb80 1 --2- 172.21.15.39:0/1423433215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5578a0b44df0 0x5578a0b55f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.830 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.802+0000 7f8f557fa700 0 client.16059 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:43:13.831 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.802+0000 7f8f8006fb80 1 -- 172.21.15.39:0/1423433215 >> 172.21.15.39:0/1423433215 conn(0x5578a0b38da0 msgr2=0x5578a0b524a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:13.831 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.802+0000 7f8f8006fb80 1 -- 172.21.15.39:0/1423433215 shutdown_connections 2023-07-21T18:43:13.831 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.802+0000 7f8f8006fb80 1 -- 172.21.15.39:0/1423433215 wait complete. 2023-07-21T18:43:13.831 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.802+0000 7f8f8006fb80 20 client.16059 trim_cache size 0 max 16384 2023-07-21T18:43:13.831 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2023-07-21T18:43:13.832 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.802+0000 7f8f73ee9700 1 -- 172.21.15.39:0/2185104681 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f8f580a1460 msgr2=0x7f8f580a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:13.832 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.802+0000 7f8f73ee9700 1 --2- 172.21.15.39:0/2185104681 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f8f580a1460 0x7f8f580a3910 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8f68005fd0 tx=0x7f8f6800a4c0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.832 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.802+0000 7f8f73ee9700 1 -- 172.21.15.39:0/2185104681 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f6c104a00 msgr2=0x7f8f6c194920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:13.832 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.802+0000 7f8f73ee9700 1 --2- 172.21.15.39:0/2185104681 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f6c104a00 0x7f8f6c194920 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f8f64007ee0 tx=0x7f8f6400d3b0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.833 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.802+0000 7f8f73ee9700 1 -- 172.21.15.39:0/2185104681 shutdown_connections 2023-07-21T18:43:13.833 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.802+0000 7f8f73ee9700 1 --2- 172.21.15.39:0/2185104681 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f8f580a1460 0x7f8f580a3910 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.833 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.802+0000 7f8f73ee9700 1 --2- 172.21.15.39:0/2185104681 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f6c0fe9f0 0x7f8f6c19a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.833 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.802+0000 7f8f73ee9700 1 --2- 172.21.15.39:0/2185104681 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f6c104a00 0x7f8f6c194920 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.833 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.802+0000 7f8f73ee9700 1 --2- 172.21.15.39:0/2185104681 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f6c0ff3a0 0x7f8f6c19adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:13.833 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.802+0000 7f8f73ee9700 1 -- 172.21.15.39:0/2185104681 >> 172.21.15.39:0/2185104681 conn(0x7f8f6c0fa4d0 msgr2=0x7f8f6c102620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:13.833 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.803+0000 7f8f73ee9700 1 -- 172.21.15.39:0/2185104681 shutdown_connections 2023-07-21T18:43:13.833 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:13.803+0000 7f8f73ee9700 1 -- 172.21.15.39:0/2185104681 wait complete. 2023-07-21T18:43:14.260 INFO:tasks.workunit.client.0.smithi039.stdout: 1 29139 6.45 MB/sec warmup 27 sec latency 3108.752 ms 2023-07-21T18:43:14.617 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:14 smithi193 ceph-mon[98204]: pgmap v47: 97 pgs: 97 active+clean; 12 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 611 KiB/s wr, 142 op/s 2023-07-21T18:43:14.636 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:14 smithi155 ceph-mon[97934]: pgmap v47: 97 pgs: 97 active+clean; 12 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 611 KiB/s wr, 142 op/s 2023-07-21T18:43:14.685 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:14 smithi039 ceph-mon[96489]: pgmap v47: 97 pgs: 97 active+clean; 12 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 611 KiB/s wr, 142 op/s 2023-07-21T18:43:14.834 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-21T18:43:14.835 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-21T18:43:14.973 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.952+0000 7fd3845f4700 1 -- 172.21.15.39:0/420675909 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd37c0ff380 msgr2=0x7fd37c10b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:14.974 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.952+0000 7fd3845f4700 1 --2- 172.21.15.39:0/420675909 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd37c0ff380 0x7fd37c10b810 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fd37400b0a0 tx=0x7fd37400b3b0 comp rx=0 tx=0).stop 2023-07-21T18:43:14.974 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.953+0000 7fd3845f4700 1 -- 172.21.15.39:0/420675909 shutdown_connections 2023-07-21T18:43:14.974 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.953+0000 7fd3845f4700 1 --2- 172.21.15.39:0/420675909 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd37c0ff380 0x7fd37c10b810 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:14.974 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.953+0000 7fd3845f4700 1 --2- 172.21.15.39:0/420675909 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd37c1049e0 0x7fd37c104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:14.974 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.953+0000 7fd3845f4700 1 --2- 172.21.15.39:0/420675909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd37c0fe9d0 0x7fd37c0fee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:14.974 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.953+0000 7fd3845f4700 1 -- 172.21.15.39:0/420675909 >> 172.21.15.39:0/420675909 conn(0x7fd37c0fa4b0 msgr2=0x7fd37c0fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:14.975 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.953+0000 7fd3845f4700 1 -- 172.21.15.39:0/420675909 shutdown_connections 2023-07-21T18:43:14.975 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.953+0000 7fd3845f4700 1 -- 172.21.15.39:0/420675909 wait complete. 2023-07-21T18:43:14.976 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.953+0000 7fd3845f4700 1 Processor -- start 2023-07-21T18:43:14.977 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.953+0000 7fd3845f4700 1 -- start start 2023-07-21T18:43:14.977 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.954+0000 7fd3845f4700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd37c0fe9d0 0x7fd37c19b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:14.977 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.954+0000 7fd3845f4700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd37c0ff380 0x7fd37c19bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:14.977 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.954+0000 7fd3845f4700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd37c1049e0 0x7fd37c195a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:14.977 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.954+0000 7fd3845f4700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd37c1960c0 con 0x7fd37c1049e0 2023-07-21T18:43:14.977 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.954+0000 7fd3845f4700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd37c196230 con 0x7fd37c0ff380 2023-07-21T18:43:14.977 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.954+0000 7fd3845f4700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd37c1963f0 con 0x7fd37c0fe9d0 2023-07-21T18:43:14.978 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.954+0000 7fd382390700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd37c0fe9d0 0x7fd37c19b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:14.978 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.954+0000 7fd382b91700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd37c1049e0 0x7fd37c195a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:14.978 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.954+0000 7fd381b8f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd37c0ff380 0x7fd37c19bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:14.978 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.954+0000 7fd382390700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd37c0fe9d0 0x7fd37c19b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:49044/0 (socket says 172.21.15.39:49044) 2023-07-21T18:43:14.978 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.954+0000 7fd382b91700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd37c1049e0 0x7fd37c195a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58728/0 (socket says 172.21.15.39:58728) 2023-07-21T18:43:14.978 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.954+0000 7fd382b91700 1 -- 172.21.15.39:0/2584668933 learned_addr learned my addr 172.21.15.39:0/2584668933 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:14.978 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.954+0000 7fd382390700 1 -- 172.21.15.39:0/2584668933 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd37c0ff380 msgr2=0x7fd37c19bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:14.978 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.954+0000 7fd382390700 1 --2- 172.21.15.39:0/2584668933 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd37c0ff380 0x7fd37c19bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:14.979 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.954+0000 7fd382390700 1 -- 172.21.15.39:0/2584668933 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd37c1049e0 msgr2=0x7fd37c195a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:14.979 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.954+0000 7fd382390700 1 --2- 172.21.15.39:0/2584668933 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd37c1049e0 0x7fd37c195a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:14.979 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.954+0000 7fd382390700 1 -- 172.21.15.39:0/2584668933 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd374009d00 con 0x7fd37c0fe9d0 2023-07-21T18:43:14.979 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.954+0000 7fd382b91700 1 --2- 172.21.15.39:0/2584668933 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd37c1049e0 0x7fd37c195a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:43:14.979 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.954+0000 7fd381b8f700 1 --2- 172.21.15.39:0/2584668933 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd37c0ff380 0x7fd37c19bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:43:14.979 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.955+0000 7fd382390700 1 --2- 172.21.15.39:0/2584668933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd37c0fe9d0 0x7fd37c19b9e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd37800e9c0 tx=0x7fd37800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:14.979 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.955+0000 7fd3737fe700 1 -- 172.21.15.39:0/2584668933 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd378004220 con 0x7fd37c0fe9d0 2023-07-21T18:43:14.979 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.955+0000 7fd3845f4700 1 -- 172.21.15.39:0/2584668933 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd37c1966d0 con 0x7fd37c0fe9d0 2023-07-21T18:43:14.980 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.955+0000 7fd3737fe700 1 -- 172.21.15.39:0/2584668933 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd378004380 con 0x7fd37c0fe9d0 2023-07-21T18:43:14.980 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.955+0000 7fd3737fe700 1 -- 172.21.15.39:0/2584668933 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd378003be0 con 0x7fd37c0fe9d0 2023-07-21T18:43:14.980 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.955+0000 7fd3845f4700 1 -- 172.21.15.39:0/2584668933 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd37c1a29d0 con 0x7fd37c0fe9d0 2023-07-21T18:43:14.983 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.958+0000 7fd3737fe700 1 -- 172.21.15.39:0/2584668933 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7fd378003d40 con 0x7fd37c0fe9d0 2023-07-21T18:43:14.984 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.958+0000 7fd3737fe700 1 --2- 172.21.15.39:0/2584668933 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fd3680a13e0 0x7fd3680a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:14.984 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.958+0000 7fd3737fe700 1 -- 172.21.15.39:0/2584668933 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fd378014070 con 0x7fd37c0fe9d0 2023-07-21T18:43:14.984 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.958+0000 7fd381b8f700 1 --2- 172.21.15.39:0/2584668933 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fd3680a13e0 0x7fd3680a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:14.985 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.959+0000 7fd381b8f700 1 --2- 172.21.15.39:0/2584668933 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fd3680a13e0 0x7fd3680a3890 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd37c0fdd70 tx=0x7fd36c009260 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:14.985 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.959+0000 7fd39077ab80 1 Processor -- start 2023-07-21T18:43:14.985 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.959+0000 7fd39077ab80 1 -- start start 2023-07-21T18:43:14.985 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.959+0000 7fd39077ab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56033544a350 0x56033544a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:14.985 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.959+0000 7fd39077ab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56033544ac60 0x560335455610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:14.985 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.959+0000 7fd39077ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560335455b50 0x560335457f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:14.985 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.959+0000 7fd39077ab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x560335350d30 con 0x56033544a350 2023-07-21T18:43:14.985 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.959+0000 7fd39077ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603353a3ac0 con 0x560335455b50 2023-07-21T18:43:14.986 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.959+0000 7fd39077ab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x56033539cf50 con 0x56033544ac60 2023-07-21T18:43:14.986 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.960+0000 7fd382390700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56033544a350 0x56033544a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:14.986 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.960+0000 7fd382390700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56033544a350 0x56033544a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58732/0 (socket says 172.21.15.39:58732) 2023-07-21T18:43:14.986 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.960+0000 7fd382390700 1 -- 172.21.15.39:0/1085412877 learned_addr learned my addr 172.21.15.39:0/1085412877 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:14.986 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.960+0000 7fd382b91700 1 --2- 172.21.15.39:0/1085412877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560335455b50 0x560335457f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:14.986 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.960+0000 7fd382390700 1 -- 172.21.15.39:0/1085412877 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56033544ac60 msgr2=0x560335455610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:14.986 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.960+0000 7fd382390700 1 --2- 172.21.15.39:0/1085412877 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56033544ac60 0x560335455610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:14.987 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.960+0000 7fd382390700 1 -- 172.21.15.39:0/1085412877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560335455b50 msgr2=0x560335457f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:14.987 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.960+0000 7fd382390700 1 --2- 172.21.15.39:0/1085412877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560335455b50 0x560335457f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:14.987 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.960+0000 7fd382390700 1 -- 172.21.15.39:0/1085412877 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5603351cae60 con 0x56033544a350 2023-07-21T18:43:14.987 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.960+0000 7fd382390700 1 --2- 172.21.15.39:0/1085412877 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56033544a350 0x56033544a720 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fd3780a2000 tx=0x7fd378099b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98989e4eb52297e8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:14.987 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.960+0000 7fd370ff9700 1 -- 172.21.15.39:0/1085412877 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37809e110 con 0x56033544a350 2023-07-21T18:43:14.987 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.960+0000 7fd370ff9700 1 -- 172.21.15.39:0/1085412877 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd3780a2530 con 0x56033544a350 2023-07-21T18:43:14.987 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.960+0000 7fd370ff9700 1 -- 172.21.15.39:0/1085412877 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3780a20a0 con 0x56033544a350 2023-07-21T18:43:14.987 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.961+0000 7fd39077ab80 1 -- 172.21.15.39:0/1085412877 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56033544a350 msgr2=0x56033544a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:14.988 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.961+0000 7fd39077ab80 1 --2- 172.21.15.39:0/1085412877 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56033544a350 0x56033544a720 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fd3780a2000 tx=0x7fd378099b10 comp rx=0 tx=0).stop 2023-07-21T18:43:14.988 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.961+0000 7fd39077ab80 1 -- 172.21.15.39:0/1085412877 shutdown_connections 2023-07-21T18:43:14.988 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.961+0000 7fd39077ab80 1 --2- 172.21.15.39:0/1085412877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560335455b50 0x560335457f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:14.988 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.961+0000 7fd39077ab80 1 --2- 172.21.15.39:0/1085412877 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56033544a350 0x56033544a720 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:14.988 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.961+0000 7fd39077ab80 1 --2- 172.21.15.39:0/1085412877 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56033544ac60 0x560335455610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:14.988 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.961+0000 7fd39077ab80 1 -- 172.21.15.39:0/1085412877 >> 172.21.15.39:0/1085412877 conn(0x560335440e20 msgr2=0x560335446720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:14.988 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.961+0000 7fd39077ab80 1 -- 172.21.15.39:0/1085412877 shutdown_connections 2023-07-21T18:43:14.988 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.961+0000 7fd39077ab80 1 -- 172.21.15.39:0/1085412877 wait complete. 2023-07-21T18:43:14.989 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.961+0000 7fd39077ab80 1 -- start start 2023-07-21T18:43:14.989 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.961+0000 7fd39077ab80 1 Processor -- start 2023-07-21T18:43:14.989 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.962+0000 7fd39077ab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56033544a350 0x5603354555b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:14.989 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.962+0000 7fd39077ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56033544ac60 0x560335452830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:14.989 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.962+0000 7fd39077ab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560335455b50 0x560335452f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:14.989 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.962+0000 7fd39077ab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x56033533cb40 con 0x56033544a350 2023-07-21T18:43:14.989 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.962+0000 7fd39077ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56033533c840 con 0x56033544ac60 2023-07-21T18:43:14.989 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.962+0000 7fd39077ab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603352209a0 con 0x560335455b50 2023-07-21T18:43:14.990 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.962+0000 7fd39077ab80 10 client.0 _ll_register_callbacks cb 0x56033533a7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:43:14.990 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.962+0000 7fd381b8f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56033544ac60 0x560335452830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:14.990 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.962+0000 7fd382390700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56033544a350 0x5603354555b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:14.990 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.962+0000 7fd382b91700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560335455b50 0x560335452f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:14.990 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.962+0000 7fd381b8f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56033544ac60 0x560335452830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:43104/0 (socket says 172.21.15.39:43104) 2023-07-21T18:43:14.990 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.962+0000 7fd382390700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56033544a350 0x5603354555b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58738/0 (socket says 172.21.15.39:58738) 2023-07-21T18:43:14.990 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.962+0000 7fd381b8f700 1 -- 172.21.15.39:0/1851552584 learned_addr learned my addr 172.21.15.39:0/1851552584 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:14.990 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.962+0000 7fd381b8f700 1 -- 172.21.15.39:0/1851552584 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560335455b50 msgr2=0x560335452f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:14.991 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.962+0000 7fd381b8f700 1 --2- 172.21.15.39:0/1851552584 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560335455b50 0x560335452f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:14.991 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.962+0000 7fd381b8f700 1 -- 172.21.15.39:0/1851552584 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56033544a350 msgr2=0x5603354555b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:14.991 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.962+0000 7fd381b8f700 1 --2- 172.21.15.39:0/1851552584 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56033544a350 0x5603354555b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:14.991 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.962+0000 7fd381b8f700 1 -- 172.21.15.39:0/1851552584 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd37c1a29d0 con 0x56033544ac60 2023-07-21T18:43:14.991 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.962+0000 7fd381b8f700 1 --2- 172.21.15.39:0/1851552584 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56033544ac60 0x560335452830 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd36c0038e0 tx=0x7fd36c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=368f45c91620c25d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:14.991 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.962+0000 7fd365ffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-21T18:43:14.991 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.962+0000 7fd365ffb700 1 -- 172.21.15.39:0/1851552584 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36c01e800 con 0x56033544ac60 2023-07-21T18:43:14.991 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.963+0000 7fd39077ab80 1 -- 172.21.15.39:0/1851552584 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5603352f6570 con 0x56033544ac60 2023-07-21T18:43:14.992 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.963+0000 7fd365ffb700 1 -- 172.21.15.39:0/1851552584 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fd36c01e960 con 0x56033544ac60 2023-07-21T18:43:14.992 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.963+0000 7fd365ffb700 1 -- 172.21.15.39:0/1851552584 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd36c01eb20 con 0x56033544ac60 2023-07-21T18:43:14.992 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.963+0000 7fd39077ab80 10 client.25592 fetch_fsmap learned FSMap version 28 2023-07-21T18:43:14.992 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.963+0000 7fd39077ab80 1 -- 172.21.15.39:0/1851552584 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5603351b3ce0 con 0x56033544ac60 2023-07-21T18:43:14.992 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.963+0000 7fd365ffb700 1 -- 172.21.15.39:0/1851552584 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5603352f6570 con 0x56033544ac60 2023-07-21T18:43:14.992 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.963+0000 7fd365ffb700 1 -- 172.21.15.39:0/1851552584 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fd36c024020 con 0x56033544ac60 2023-07-21T18:43:14.992 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.963+0000 7fd39077ab80 10 client.25592 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T18:43:14.993 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.963+0000 7fd39077ab80 10 client.25592 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T18:43:14.993 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.963+0000 7fd39077ab80 20 client.25592 populate_metadata read hostname 'smithi039' 2023-07-21T18:43:14.993 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.963+0000 7fd39077ab80 1 --2- 172.21.15.39:0/1851552584 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x560335465520 0x560335467930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:14.993 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.963+0000 7fd39077ab80 4 client.25592 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:43:14.993 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.963+0000 7fd39077ab80 1 -- 172.21.15.39:0/1851552584 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560335395f30 con 0x560335465520 2023-07-21T18:43:14.993 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.963+0000 7fd382390700 1 --2- 172.21.15.39:0/1851552584 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x560335465520 0x560335467930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:14.993 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.964+0000 7fd382390700 1 --2- 172.21.15.39:0/1851552584 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x560335465520 0x560335467930 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b66800bbf29405b server_cookie=e47b9987cb4e742b in_seq=0 out_seq=0 2023-07-21T18:43:14.993 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.964+0000 7fd365ffb700 10 client.25592 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:43:14.994 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.965+0000 7fd365ffb700 1 -- 172.21.15.39:0/1851552584 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560335395f30 con 0x560335465520 2023-07-21T18:43:14.995 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.965+0000 7fd365ffb700 10 client.25592 handle_command_reply: tid=0 2023-07-21T18:43:14.995 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.965+0000 7fd39077ab80 1 client.25592 shutdown 2023-07-21T18:43:14.995 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.965+0000 7fd39077ab80 1 -- 172.21.15.39:0/1851552584 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56033544ac60 msgr2=0x560335452830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:14.995 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.965+0000 7fd39077ab80 1 --2- 172.21.15.39:0/1851552584 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56033544ac60 0x560335452830 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd36c0038e0 tx=0x7fd36c004750 comp rx=0 tx=0).stop 2023-07-21T18:43:14.995 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.965+0000 7fd39077ab80 1 -- 172.21.15.39:0/1851552584 shutdown_connections 2023-07-21T18:43:14.995 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.965+0000 7fd39077ab80 1 --2- 172.21.15.39:0/1851552584 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x560335465520 0x560335467930 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:14.995 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.965+0000 7fd39077ab80 1 --2- 172.21.15.39:0/1851552584 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560335455b50 0x560335452f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:14.995 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.965+0000 7fd39077ab80 1 --2- 172.21.15.39:0/1851552584 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56033544a350 0x5603354555b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:14.996 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.965+0000 7fd39077ab80 1 --2- 172.21.15.39:0/1851552584 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56033544ac60 0x560335452830 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:14.996 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.965+0000 7fd365ffb700 0 client.25592 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:43:14.996 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.965+0000 7fd39077ab80 1 -- 172.21.15.39:0/1851552584 >> 172.21.15.39:0/1851552584 conn(0x560335440e20 msgr2=0x5603354462b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:14.996 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.965+0000 7fd39077ab80 1 -- 172.21.15.39:0/1851552584 shutdown_connections 2023-07-21T18:43:14.996 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.965+0000 7fd39077ab80 1 -- 172.21.15.39:0/1851552584 wait complete. 2023-07-21T18:43:14.996 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.965+0000 7fd39077ab80 20 client.25592 trim_cache size 0 max 16384 2023-07-21T18:43:15.003 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.983+0000 7fd39077ab80 1 Processor -- start 2023-07-21T18:43:15.004 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.983+0000 7fd39077ab80 1 -- start start 2023-07-21T18:43:15.004 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.983+0000 7fd39077ab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56033544a350 0x56033544db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:15.004 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.983+0000 7fd39077ab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56033544ac60 0x56033544e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:15.004 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.983+0000 7fd39077ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560335455b50 0x560335452010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:15.004 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.983+0000 7fd39077ab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x56033544c220 con 0x56033544ac60 2023-07-21T18:43:15.004 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.983+0000 7fd39077ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603354582f0 con 0x560335455b50 2023-07-21T18:43:15.004 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.983+0000 7fd39077ab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603354558f0 con 0x56033544a350 2023-07-21T18:43:15.005 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.983+0000 7fd381b8f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56033544ac60 0x56033544e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:15.005 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.983+0000 7fd381b8f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56033544ac60 0x56033544e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58748/0 (socket says 172.21.15.39:58748) 2023-07-21T18:43:15.005 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.983+0000 7fd381b8f700 1 -- 172.21.15.39:0/2755555845 learned_addr learned my addr 172.21.15.39:0/2755555845 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:15.005 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.983+0000 7fd382b91700 1 --2- 172.21.15.39:0/2755555845 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560335455b50 0x560335452010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:15.005 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.984+0000 7fd382390700 1 --2- 172.21.15.39:0/2755555845 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56033544a350 0x56033544db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:15.005 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.984+0000 7fd381b8f700 1 -- 172.21.15.39:0/2755555845 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56033544a350 msgr2=0x56033544db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:15.005 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.984+0000 7fd381b8f700 1 --2- 172.21.15.39:0/2755555845 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56033544a350 0x56033544db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:15.006 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.984+0000 7fd381b8f700 1 -- 172.21.15.39:0/2755555845 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560335455b50 msgr2=0x560335452010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:15.006 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.984+0000 7fd381b8f700 1 --2- 172.21.15.39:0/2755555845 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560335455b50 0x560335452010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:15.006 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.984+0000 7fd381b8f700 1 -- 172.21.15.39:0/2755555845 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560335442660 con 0x56033544ac60 2023-07-21T18:43:15.006 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.984+0000 7fd381b8f700 1 --2- 172.21.15.39:0/2755555845 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56033544ac60 0x56033544e090 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fd36c000c00 tx=0x7fd36c003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=280acd0b9a0cd432 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:15.008 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.984+0000 7fd3717fa700 1 -- 172.21.15.39:0/2755555845 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36c00e8b0 con 0x56033544ac60 2023-07-21T18:43:15.009 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.984+0000 7fd3717fa700 1 -- 172.21.15.39:0/2755555845 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd36c01e600 con 0x56033544ac60 2023-07-21T18:43:15.009 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.984+0000 7fd3717fa700 1 -- 172.21.15.39:0/2755555845 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36c026640 con 0x56033544ac60 2023-07-21T18:43:15.009 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.985+0000 7fd39077ab80 1 -- 172.21.15.39:0/2755555845 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56033544ac60 msgr2=0x56033544e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:15.009 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.985+0000 7fd39077ab80 1 --2- 172.21.15.39:0/2755555845 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56033544ac60 0x56033544e090 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fd36c000c00 tx=0x7fd36c003f50 comp rx=0 tx=0).stop 2023-07-21T18:43:15.009 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.985+0000 7fd39077ab80 1 -- 172.21.15.39:0/2755555845 shutdown_connections 2023-07-21T18:43:15.009 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.985+0000 7fd39077ab80 1 --2- 172.21.15.39:0/2755555845 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560335455b50 0x560335452010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:15.009 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.985+0000 7fd39077ab80 1 --2- 172.21.15.39:0/2755555845 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56033544a350 0x56033544db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:15.009 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.985+0000 7fd39077ab80 1 --2- 172.21.15.39:0/2755555845 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56033544ac60 0x56033544e090 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:15.010 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.985+0000 7fd39077ab80 1 -- 172.21.15.39:0/2755555845 >> 172.21.15.39:0/2755555845 conn(0x560335440e20 msgr2=0x5603354551e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:15.010 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.985+0000 7fd39077ab80 1 -- 172.21.15.39:0/2755555845 shutdown_connections 2023-07-21T18:43:15.010 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.985+0000 7fd39077ab80 1 -- 172.21.15.39:0/2755555845 wait complete. 2023-07-21T18:43:15.010 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.985+0000 7fd39077ab80 1 -- start start 2023-07-21T18:43:15.010 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.985+0000 7fd39077ab80 1 Processor -- start 2023-07-21T18:43:15.010 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.985+0000 7fd39077ab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56033544a350 0x56033545b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:15.010 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.986+0000 7fd39077ab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56033544ac60 0x56033545b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:15.011 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.986+0000 7fd39077ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560335455b50 0x5603354b33f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:15.011 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.986+0000 7fd39077ab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603354b3930 con 0x56033544a350 2023-07-21T18:43:15.011 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.986+0000 7fd39077ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603354b3a70 con 0x560335455b50 2023-07-21T18:43:15.011 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.986+0000 7fd39077ab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603354b3c30 con 0x56033544ac60 2023-07-21T18:43:15.011 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.986+0000 7fd39077ab80 10 client.0 _ll_register_callbacks cb 0x56033533a7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:43:15.011 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.986+0000 7fd382390700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56033544a350 0x56033545b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:15.011 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.986+0000 7fd382390700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56033544a350 0x56033545b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58752/0 (socket says 172.21.15.39:58752) 2023-07-21T18:43:15.011 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.986+0000 7fd382390700 1 -- 172.21.15.39:0/2891017457 learned_addr learned my addr 172.21.15.39:0/2891017457 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:15.012 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.986+0000 7fd382b91700 1 --2- 172.21.15.39:0/2891017457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560335455b50 0x5603354b33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:15.012 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.986+0000 7fd381b8f700 1 --2- 172.21.15.39:0/2891017457 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56033544ac60 0x56033545b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:15.012 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.986+0000 7fd382390700 1 -- 172.21.15.39:0/2891017457 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56033544ac60 msgr2=0x56033545b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:15.012 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.986+0000 7fd382390700 1 --2- 172.21.15.39:0/2891017457 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56033544ac60 0x56033545b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:15.012 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.986+0000 7fd382390700 1 -- 172.21.15.39:0/2891017457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560335455b50 msgr2=0x5603354b33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:15.012 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.986+0000 7fd382390700 1 --2- 172.21.15.39:0/2891017457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560335455b50 0x5603354b33f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:15.012 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.986+0000 7fd382390700 1 -- 172.21.15.39:0/2891017457 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5603351b3ce0 con 0x56033544a350 2023-07-21T18:43:15.012 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.986+0000 7fd382b91700 1 --2- 172.21.15.39:0/2891017457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560335455b50 0x5603354b33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:43:15.013 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.986+0000 7fd382390700 1 --2- 172.21.15.39:0/2891017457 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56033544a350 0x56033545b480 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fd3780ca770 tx=0x7fd378016470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8444fb0308656b16 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:15.013 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.986+0000 7fd365ffb700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:43:15.013 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.986+0000 7fd365ffb700 1 -- 172.21.15.39:0/2891017457 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3780b7040 con 0x56033544a350 2023-07-21T18:43:15.013 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.986+0000 7fd365ffb700 1 -- 172.21.15.39:0/2891017457 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fd3780ca2f0 con 0x56033544a350 2023-07-21T18:43:15.013 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.986+0000 7fd39077ab80 1 -- 172.21.15.39:0/2891017457 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd3780a2530 con 0x56033544a350 2023-07-21T18:43:15.013 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.986+0000 7fd365ffb700 1 -- 172.21.15.39:0/2891017457 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd378098500 con 0x56033544a350 2023-07-21T18:43:15.013 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.987+0000 7fd39077ab80 10 client.16077 fetch_fsmap learned FSMap version 28 2023-07-21T18:43:15.014 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.987+0000 7fd39077ab80 1 -- 172.21.15.39:0/2891017457 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5603351f12d0 con 0x56033544a350 2023-07-21T18:43:15.014 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.987+0000 7fd365ffb700 1 -- 172.21.15.39:0/2891017457 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd3780a2530 con 0x56033544a350 2023-07-21T18:43:15.014 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.987+0000 7fd365ffb700 1 -- 172.21.15.39:0/2891017457 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fd3780a20a0 con 0x56033544a350 2023-07-21T18:43:15.014 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.987+0000 7fd39077ab80 10 client.16077 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T18:43:15.014 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.987+0000 7fd39077ab80 10 client.16077 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T18:43:15.014 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.987+0000 7fd39077ab80 20 client.16077 populate_metadata read hostname 'smithi039' 2023-07-21T18:43:15.014 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.987+0000 7fd39077ab80 1 --2- 172.21.15.39:0/2891017457 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5603354bb020 0x5603354bd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:15.014 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.987+0000 7fd39077ab80 4 client.16077 mds_command: new command op to 24536 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-21T18:43:15.015 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.987+0000 7fd39077ab80 1 -- 172.21.15.39:0/2891017457 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5603353580d0 con 0x5603354bb020 2023-07-21T18:43:15.015 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.987+0000 7fd381b8f700 1 --2- 172.21.15.39:0/2891017457 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5603354bb020 0x5603354bd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:15.015 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.987+0000 7fd381b8f700 1 --2- 172.21.15.39:0/2891017457 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5603354bb020 0x5603354bd430 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=564a4b1bf5ea6934 server_cookie=885f74eac46e22b6 in_seq=0 out_seq=0 2023-07-21T18:43:15.015 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.987+0000 7fd365ffb700 10 client.16077 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:43:15.016 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.989+0000 7fd365ffb700 1 -- 172.21.15.39:0/2891017457 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5603353580d0 con 0x5603354bb020 2023-07-21T18:43:15.016 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.989+0000 7fd365ffb700 10 client.16077 handle_command_reply: tid=0 2023-07-21T18:43:15.016 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.989+0000 7fd39077ab80 1 client.16077 shutdown 2023-07-21T18:43:15.016 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.989+0000 7fd39077ab80 1 -- 172.21.15.39:0/2891017457 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56033544a350 msgr2=0x56033545b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:15.017 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.989+0000 7fd39077ab80 1 --2- 172.21.15.39:0/2891017457 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56033544a350 0x56033545b480 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fd3780ca770 tx=0x7fd378016470 comp rx=0 tx=0).stop 2023-07-21T18:43:15.017 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.989+0000 7fd39077ab80 1 -- 172.21.15.39:0/2891017457 shutdown_connections 2023-07-21T18:43:15.017 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.989+0000 7fd39077ab80 1 --2- 172.21.15.39:0/2891017457 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5603354bb020 0x5603354bd430 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:15.017 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.989+0000 7fd39077ab80 1 --2- 172.21.15.39:0/2891017457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560335455b50 0x5603354b33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:15.017 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.989+0000 7fd39077ab80 1 --2- 172.21.15.39:0/2891017457 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56033544a350 0x56033545b480 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:15.017 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.989+0000 7fd39077ab80 1 --2- 172.21.15.39:0/2891017457 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56033544ac60 0x56033545b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:15.017 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.989+0000 7fd39077ab80 1 -- 172.21.15.39:0/2891017457 >> 172.21.15.39:0/2891017457 conn(0x560335440e20 msgr2=0x56033544db50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:15.017 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.989+0000 7fd365ffb700 0 client.16077 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:43:15.018 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.989+0000 7fd39077ab80 1 -- 172.21.15.39:0/2891017457 shutdown_connections 2023-07-21T18:43:15.018 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.990+0000 7fd39077ab80 1 -- 172.21.15.39:0/2891017457 wait complete. 2023-07-21T18:43:15.018 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.990+0000 7fd39077ab80 20 client.16077 trim_cache size 0 max 16384 2023-07-21T18:43:15.018 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2023-07-21T18:43:15.018 INFO:teuthology.orchestra.run.smithi039.stdout: "return_code": 0, 2023-07-21T18:43:15.018 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_tag": "c6b195a6-7fa4-4f06-83fb-ff2911723603", 2023-07-21T18:43:15.019 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": "asynchronous" 2023-07-21T18:43:15.019 INFO:teuthology.orchestra.run.smithi039.stdout:} 2023-07-21T18:43:15.019 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.990+0000 7fd3845f4700 1 -- 172.21.15.39:0/2584668933 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fd3680a13e0 msgr2=0x7fd3680a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:15.019 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.990+0000 7fd3845f4700 1 --2- 172.21.15.39:0/2584668933 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fd3680a13e0 0x7fd3680a3890 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd37c0fdd70 tx=0x7fd36c009260 comp rx=0 tx=0).stop 2023-07-21T18:43:15.020 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.990+0000 7fd3845f4700 1 -- 172.21.15.39:0/2584668933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd37c0fe9d0 msgr2=0x7fd37c19b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:15.020 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.990+0000 7fd3845f4700 1 --2- 172.21.15.39:0/2584668933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd37c0fe9d0 0x7fd37c19b9e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd37800e9c0 tx=0x7fd37800ecd0 comp rx=0 tx=0).stop 2023-07-21T18:43:15.020 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.990+0000 7fd3845f4700 1 -- 172.21.15.39:0/2584668933 shutdown_connections 2023-07-21T18:43:15.020 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.990+0000 7fd3845f4700 1 --2- 172.21.15.39:0/2584668933 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd37c1049e0 0x7fd37c195a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:15.020 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.990+0000 7fd3845f4700 1 --2- 172.21.15.39:0/2584668933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd37c0fe9d0 0x7fd37c19b9e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:15.020 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.990+0000 7fd3845f4700 1 --2- 172.21.15.39:0/2584668933 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fd3680a13e0 0x7fd3680a3890 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:15.020 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.990+0000 7fd3845f4700 1 --2- 172.21.15.39:0/2584668933 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd37c0ff380 0x7fd37c19bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:15.021 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.990+0000 7fd3845f4700 1 -- 172.21.15.39:0/2584668933 >> 172.21.15.39:0/2584668933 conn(0x7fd37c0fa4b0 msgr2=0x7fd37c102520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:15.021 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.990+0000 7fd3845f4700 1 -- 172.21.15.39:0/2584668933 shutdown_connections 2023-07-21T18:43:15.021 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:14.990+0000 7fd3845f4700 1 -- 172.21.15.39:0/2584668933 wait complete. 2023-07-21T18:43:15.022 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T18:43:15.156 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.136+0000 7f8edcec9700 1 -- 172.21.15.39:0/260564518 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ed8105c00 msgr2=0x7f8ed8105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:15.157 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.136+0000 7f8edcec9700 1 --2- 172.21.15.39:0/260564518 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ed8105c00 0x7f8ed8105fd0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc009990 tx=0x7f8ecc009ca0 comp rx=0 tx=0).stop 2023-07-21T18:43:15.157 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.136+0000 7f8edcec9700 1 -- 172.21.15.39:0/260564518 shutdown_connections 2023-07-21T18:43:15.157 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.136+0000 7f8edcec9700 1 --2- 172.21.15.39:0/260564518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ed80feb90 0x7f8ed810c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:15.157 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.136+0000 7f8edcec9700 1 --2- 172.21.15.39:0/260564518 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ed8105c00 0x7f8ed8105fd0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:15.157 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.136+0000 7f8edcec9700 1 --2- 172.21.15.39:0/260564518 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ed80fe200 0x7f8ed80fe650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:15.157 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.136+0000 7f8edcec9700 1 -- 172.21.15.39:0/260564518 >> 172.21.15.39:0/260564518 conn(0x7f8ed8076200 msgr2=0x7f8ed8076600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:15.157 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.136+0000 7f8edcec9700 1 -- 172.21.15.39:0/260564518 shutdown_connections 2023-07-21T18:43:15.158 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.136+0000 7f8edcec9700 1 -- 172.21.15.39:0/260564518 wait complete. 2023-07-21T18:43:15.159 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.137+0000 7f8edcec9700 1 Processor -- start 2023-07-21T18:43:15.159 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.137+0000 7f8edcec9700 1 -- start start 2023-07-21T18:43:15.159 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.137+0000 7f8edcec9700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ed80fe200 0x7f8ed819ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:15.159 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.137+0000 7f8edcec9700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ed80feb90 0x7f8ed819bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:15.160 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.137+0000 7f8edcec9700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ed8105c00 0x7f8ed8195ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:15.160 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.137+0000 7f8edcec9700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed8196130 con 0x7f8ed80feb90 2023-07-21T18:43:15.160 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.137+0000 7f8edcec9700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed81962a0 con 0x7f8ed80fe200 2023-07-21T18:43:15.160 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.137+0000 7f8edcec9700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed8196460 con 0x7f8ed8105c00 2023-07-21T18:43:15.160 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.138+0000 7f8ed5d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ed80feb90 0x7f8ed819bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:15.160 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.138+0000 7f8ed659c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ed80fe200 0x7f8ed819ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:15.160 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.138+0000 7f8ed5d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ed80feb90 0x7f8ed819bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58766/0 (socket says 172.21.15.39:58766) 2023-07-21T18:43:15.161 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.138+0000 7f8ed5d9b700 1 -- 172.21.15.39:0/3148543602 learned_addr learned my addr 172.21.15.39:0/3148543602 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:15.161 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.138+0000 7f8ed6d9d700 1 --2- 172.21.15.39:0/3148543602 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ed8105c00 0x7f8ed8195ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:15.161 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.138+0000 7f8ed5d9b700 1 -- 172.21.15.39:0/3148543602 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ed8105c00 msgr2=0x7f8ed8195ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:15.161 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.138+0000 7f8ed5d9b700 1 --2- 172.21.15.39:0/3148543602 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ed8105c00 0x7f8ed8195ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:15.161 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.138+0000 7f8ed5d9b700 1 -- 172.21.15.39:0/3148543602 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ed80fe200 msgr2=0x7f8ed819ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:15.161 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.138+0000 7f8ed5d9b700 1 --2- 172.21.15.39:0/3148543602 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ed80fe200 0x7f8ed819ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:15.161 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.138+0000 7f8ed5d9b700 1 -- 172.21.15.39:0/3148543602 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ecc0095f0 con 0x7f8ed80feb90 2023-07-21T18:43:15.161 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.138+0000 7f8ed5d9b700 1 --2- 172.21.15.39:0/3148543602 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ed80feb90 0x7f8ed819bf90 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f8ec800d980 tx=0x7f8ec800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:15.162 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.138+0000 7f8ec77fe700 1 -- 172.21.15.39:0/3148543602 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8004a30 con 0x7f8ed80feb90 2023-07-21T18:43:15.162 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.138+0000 7f8ec77fe700 1 -- 172.21.15.39:0/3148543602 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ec80054e0 con 0x7f8ed80feb90 2023-07-21T18:43:15.162 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.138+0000 7f8edcec9700 1 -- 172.21.15.39:0/3148543602 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ed8196740 con 0x7f8ed80feb90 2023-07-21T18:43:15.162 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.138+0000 7f8ec77fe700 1 -- 172.21.15.39:0/3148543602 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec800fbc0 con 0x7f8ed80feb90 2023-07-21T18:43:15.162 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.138+0000 7f8edcec9700 1 -- 172.21.15.39:0/3148543602 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ed81a2a40 con 0x7f8ed80feb90 2023-07-21T18:43:15.164 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.141+0000 7f8ec77fe700 1 -- 172.21.15.39:0/3148543602 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f8ec8004c20 con 0x7f8ed80feb90 2023-07-21T18:43:15.164 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.142+0000 7f8ec77fe700 1 --2- 172.21.15.39:0/3148543602 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f8ec00a13e0 0x7f8ec00a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:15.164 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.142+0000 7f8ec77fe700 1 -- 172.21.15.39:0/3148543602 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f8ec80cce00 con 0x7f8ed80feb90 2023-07-21T18:43:15.164 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.142+0000 7f8ed659c700 1 --2- 172.21.15.39:0/3148543602 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f8ec00a13e0 0x7f8ec00a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:15.164 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.143+0000 7f8ed659c700 1 --2- 172.21.15.39:0/3148543602 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f8ec00a13e0 0x7f8ec00a3890 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc00b2e0 tx=0x7f8ecc005fd0 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:15.164 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.143+0000 7f8ee904fb80 1 Processor -- start 2023-07-21T18:43:15.164 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.143+0000 7f8ee904fb80 1 -- start start 2023-07-21T18:43:15.165 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.143+0000 7f8ee904fb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecce7362d0 0x55ecce7366a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:15.165 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.143+0000 7f8ee904fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecce736be0 0x55ecce741590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:15.165 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.143+0000 7f8ee904fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ecce741ad0 0x55ecce743eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:15.165 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.143+0000 7f8ee904fb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecce63ccb0 con 0x55ecce7362d0 2023-07-21T18:43:15.165 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.143+0000 7f8ee904fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecce68fa40 con 0x55ecce736be0 2023-07-21T18:43:15.165 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.143+0000 7f8ee904fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecce688ed0 con 0x55ecce741ad0 2023-07-21T18:43:15.165 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.144+0000 7f8ed659c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecce7362d0 0x55ecce7366a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:15.165 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.144+0000 7f8ed5d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecce736be0 0x55ecce741590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:15.166 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.144+0000 7f8ed6d9d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ecce741ad0 0x55ecce743eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:15.166 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.144+0000 7f8ed659c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecce7362d0 0x55ecce7366a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58772/0 (socket says 172.21.15.39:58772) 2023-07-21T18:43:15.166 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.144+0000 7f8ed659c700 1 -- 172.21.15.39:0/1061160116 learned_addr learned my addr 172.21.15.39:0/1061160116 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:15.166 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.144+0000 7f8ed659c700 1 -- 172.21.15.39:0/1061160116 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ecce741ad0 msgr2=0x55ecce743eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:15.166 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.144+0000 7f8ed659c700 1 --2- 172.21.15.39:0/1061160116 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ecce741ad0 0x55ecce743eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:15.166 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.144+0000 7f8ed659c700 1 -- 172.21.15.39:0/1061160116 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecce736be0 msgr2=0x55ecce741590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:15.166 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.144+0000 7f8ed659c700 1 --2- 172.21.15.39:0/1061160116 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecce736be0 0x55ecce741590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:15.167 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.144+0000 7f8ed659c700 1 -- 172.21.15.39:0/1061160116 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ecce4b6de0 con 0x55ecce7362d0 2023-07-21T18:43:15.167 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.144+0000 7f8ed5d9b700 1 --2- 172.21.15.39:0/1061160116 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecce736be0 0x55ecce741590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:43:15.167 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.144+0000 7f8ed659c700 1 --2- 172.21.15.39:0/1061160116 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecce7362d0 0x55ecce7366a0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc015000 tx=0x7f8ecc020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da24ba85da13ecfc server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:15.170 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.144+0000 7f8ec4ff9700 1 -- 172.21.15.39:0/1061160116 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc02b070 con 0x55ecce7362d0 2023-07-21T18:43:15.170 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.144+0000 7f8ec4ff9700 1 -- 172.21.15.39:0/1061160116 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ecc01e040 con 0x55ecce7362d0 2023-07-21T18:43:15.170 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.144+0000 7f8ec4ff9700 1 -- 172.21.15.39:0/1061160116 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc00f6c0 con 0x55ecce7362d0 2023-07-21T18:43:15.170 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.145+0000 7f8ee904fb80 1 -- 172.21.15.39:0/1061160116 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecce7362d0 msgr2=0x55ecce7366a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:15.170 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.145+0000 7f8ee904fb80 1 --2- 172.21.15.39:0/1061160116 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecce7362d0 0x55ecce7366a0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc015000 tx=0x7f8ecc020000 comp rx=0 tx=0).stop 2023-07-21T18:43:15.171 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.145+0000 7f8ee904fb80 1 -- 172.21.15.39:0/1061160116 shutdown_connections 2023-07-21T18:43:15.171 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.145+0000 7f8ee904fb80 1 --2- 172.21.15.39:0/1061160116 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ecce741ad0 0x55ecce743eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:15.171 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.145+0000 7f8ee904fb80 1 --2- 172.21.15.39:0/1061160116 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecce7362d0 0x55ecce7366a0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:15.171 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.145+0000 7f8ee904fb80 1 --2- 172.21.15.39:0/1061160116 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecce736be0 0x55ecce741590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:15.171 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.145+0000 7f8ee904fb80 1 -- 172.21.15.39:0/1061160116 >> 172.21.15.39:0/1061160116 conn(0x55ecce72cda0 msgr2=0x55ecce7326a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:15.171 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.145+0000 7f8ee904fb80 1 -- 172.21.15.39:0/1061160116 shutdown_connections 2023-07-21T18:43:15.171 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.145+0000 7f8ee904fb80 1 -- 172.21.15.39:0/1061160116 wait complete. 2023-07-21T18:43:15.172 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.145+0000 7f8ee904fb80 1 -- start start 2023-07-21T18:43:15.172 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.146+0000 7f8ee904fb80 1 Processor -- start 2023-07-21T18:43:15.172 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.146+0000 7f8ee904fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ecce7362d0 0x55ecce7414a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:15.172 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.146+0000 7f8ee904fb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecce736be0 0x55ecce73e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:15.172 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.146+0000 7f8ee904fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecce741ad0 0x55ecce73ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:15.172 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.146+0000 7f8ee904fb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecce628ac0 con 0x55ecce736be0 2023-07-21T18:43:15.172 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.146+0000 7f8ee904fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecce6287c0 con 0x55ecce741ad0 2023-07-21T18:43:15.172 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.146+0000 7f8ee904fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecce50c920 con 0x55ecce7362d0 2023-07-21T18:43:15.173 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.146+0000 7f8ee904fb80 10 client.0 _ll_register_callbacks cb 0x55ecce626760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:43:15.173 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.146+0000 7f8ed5d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecce736be0 0x55ecce73e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:15.173 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.146+0000 7f8ed5d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecce736be0 0x55ecce73e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58780/0 (socket says 172.21.15.39:58780) 2023-07-21T18:43:15.173 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.146+0000 7f8ed5d9b700 1 -- 172.21.15.39:0/3853972120 learned_addr learned my addr 172.21.15.39:0/3853972120 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:15.173 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.146+0000 7f8ed6d9d700 1 --2- 172.21.15.39:0/3853972120 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecce741ad0 0x55ecce73ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:15.173 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.146+0000 7f8ed659c700 1 --2- 172.21.15.39:0/3853972120 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ecce7362d0 0x55ecce7414a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:15.173 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.146+0000 7f8ed5d9b700 1 -- 172.21.15.39:0/3853972120 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ecce7362d0 msgr2=0x55ecce7414a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:15.173 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.146+0000 7f8ed5d9b700 1 --2- 172.21.15.39:0/3853972120 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ecce7362d0 0x55ecce7414a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:15.174 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.146+0000 7f8ed5d9b700 1 -- 172.21.15.39:0/3853972120 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecce741ad0 msgr2=0x55ecce73ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:15.174 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.146+0000 7f8ed5d9b700 1 --2- 172.21.15.39:0/3853972120 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecce741ad0 0x55ecce73ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:15.174 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.146+0000 7f8ed5d9b700 1 -- 172.21.15.39:0/3853972120 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ed81a2a40 con 0x55ecce736be0 2023-07-21T18:43:15.174 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.146+0000 7f8ed5d9b700 1 --2- 172.21.15.39:0/3853972120 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecce736be0 0x55ecce73e790 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f8ec800d950 tx=0x7f8ec8098250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59c0042ab3a49817 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:15.174 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.146+0000 7f8ebdffb700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:43:15.174 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.147+0000 7f8ebdffb700 1 -- 172.21.15.39:0/3853972120 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec80b2410 con 0x55ecce736be0 2023-07-21T18:43:15.174 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.147+0000 7f8ebdffb700 1 -- 172.21.15.39:0/3853972120 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f8ec8009bf0 con 0x55ecce736be0 2023-07-21T18:43:15.174 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.147+0000 7f8ee904fb80 1 -- 172.21.15.39:0/3853972120 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ecce5e24f0 con 0x55ecce736be0 2023-07-21T18:43:15.175 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.147+0000 7f8ebdffb700 1 -- 172.21.15.39:0/3853972120 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ec8009db0 con 0x55ecce736be0 2023-07-21T18:43:15.175 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.147+0000 7f8ee904fb80 10 client.16089 fetch_fsmap learned FSMap version 28 2023-07-21T18:43:15.175 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.147+0000 7f8ee904fb80 1 -- 172.21.15.39:0/3853972120 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f8ed810d2b0 con 0x55ecce736be0 2023-07-21T18:43:15.175 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.147+0000 7f8ebdffb700 1 -- 172.21.15.39:0/3853972120 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ecce5e24f0 con 0x55ecce736be0 2023-07-21T18:43:15.175 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.147+0000 7f8ebdffb700 1 -- 172.21.15.39:0/3853972120 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f8ec8099290 con 0x55ecce736be0 2023-07-21T18:43:15.175 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.147+0000 7f8ee904fb80 10 client.16089 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T18:43:15.175 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.147+0000 7f8ee904fb80 10 client.16089 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T18:43:15.176 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.147+0000 7f8ee904fb80 20 client.16089 populate_metadata read hostname 'smithi039' 2023-07-21T18:43:15.176 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.147+0000 7f8ee904fb80 1 --2- 172.21.15.39:0/3853972120 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55ecce7512a0 0x55ecce7536b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:15.176 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.147+0000 7f8ee904fb80 4 client.16089 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:43:15.176 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.147+0000 7f8ee904fb80 1 -- 172.21.15.39:0/3853972120 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ecce681eb0 con 0x55ecce7512a0 2023-07-21T18:43:15.176 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.147+0000 7f8ed659c700 1 --2- 172.21.15.39:0/3853972120 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55ecce7512a0 0x55ecce7536b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:15.176 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.148+0000 7f8ed659c700 1 --2- 172.21.15.39:0/3853972120 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55ecce7512a0 0x55ecce7536b0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b691d1f5649a19ed server_cookie=df453101f217a616 in_seq=0 out_seq=0 2023-07-21T18:43:15.176 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.148+0000 7f8ebdffb700 10 client.16089 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:43:15.176 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.148+0000 7f8ebdffb700 1 -- 172.21.15.39:0/3853972120 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ecce681eb0 con 0x55ecce7512a0 2023-07-21T18:43:15.177 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.148+0000 7f8ebdffb700 10 client.16089 handle_command_reply: tid=0 2023-07-21T18:43:15.177 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.148+0000 7f8ee904fb80 1 client.16089 shutdown 2023-07-21T18:43:15.177 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.149+0000 7f8ee904fb80 1 -- 172.21.15.39:0/3853972120 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecce736be0 msgr2=0x55ecce73e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:15.177 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.149+0000 7f8ee904fb80 1 --2- 172.21.15.39:0/3853972120 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecce736be0 0x55ecce73e790 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f8ec800d950 tx=0x7f8ec8098250 comp rx=0 tx=0).stop 2023-07-21T18:43:15.177 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.149+0000 7f8ee904fb80 1 -- 172.21.15.39:0/3853972120 shutdown_connections 2023-07-21T18:43:15.177 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.149+0000 7f8ee904fb80 1 --2- 172.21.15.39:0/3853972120 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55ecce7512a0 0x55ecce7536b0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:15.177 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.149+0000 7f8ee904fb80 1 --2- 172.21.15.39:0/3853972120 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecce741ad0 0x55ecce73ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:15.177 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.149+0000 7f8ee904fb80 1 --2- 172.21.15.39:0/3853972120 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ecce7362d0 0x55ecce7414a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:15.178 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.149+0000 7f8ebdffb700 0 client.16089 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:43:15.178 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.149+0000 7f8ee904fb80 1 --2- 172.21.15.39:0/3853972120 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecce736be0 0x55ecce73e790 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:15.178 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.149+0000 7f8ee904fb80 1 -- 172.21.15.39:0/3853972120 >> 172.21.15.39:0/3853972120 conn(0x55ecce72cda0 msgr2=0x55ecce732290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:15.178 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.149+0000 7f8ee904fb80 1 -- 172.21.15.39:0/3853972120 shutdown_connections 2023-07-21T18:43:15.178 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.149+0000 7f8ee904fb80 1 -- 172.21.15.39:0/3853972120 wait complete. 2023-07-21T18:43:15.178 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.149+0000 7f8ee904fb80 20 client.16089 trim_cache size 0 max 16384 2023-07-21T18:43:15.185 INFO:journalctl@ceph.mds.a.smithi039.stdout:Jul 21 18:43:15 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-a[131175]: 2023-07-21T18:43:15.027+0000 7f76fd211700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000023f [4,head] /volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/ACCESS/FASTENER.MDB auth v282 ap=1 s=260096 n(v0 rc2023-07-21T18:43:10.410188+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15303=0-4194304@3} caps={15303=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@25},l=15303 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x562a0e91a000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000023f:[<0x1000000023e/FASTENER.MDB v120>,<0x10000000009/ACCESS v2610>,<0x10000000008/~dmtmp v40>,<0x10000000007/client0 v24>,<0x10000000006/clients v14>,<0x10000000005/tmp v14>,<0x100000001fb/client.0 v18>,<0x100000001fa/8d6c01e4-c829-4108-ba84-43703682343a v34>,<0x10000000001/sv_1 v57>,<0x10000000000/_nogroup v43>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x1000000023f:[<0x1000000023e/FASTENER.MDB v282>,<0x10000000009/ACCESS v4627>,<0x10000000008/~dmtmp v903>,<0x10000000007/client0 v59>,<0x10000000006/clients v48>,<0x10000000005/tmp v46>,<0x100000001fb/client.0 v48>,<0x100000001fa/8d6c01e4-c829-4108-ba84-43703682343a v52>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v25>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T18:43:15.186 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.166+0000 7f8ee904fb80 1 Processor -- start 2023-07-21T18:43:15.186 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.167+0000 7f8ee904fb80 1 -- start start 2023-07-21T18:43:15.188 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.167+0000 7f8ee904fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecce7362d0 0x55ecce741040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:15.188 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.167+0000 7f8ee904fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ecce736be0 0x55ecce73df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:15.188 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.167+0000 7f8ee904fb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecce741ad0 0x55ecce73e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:15.188 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.167+0000 7f8ee904fb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecce737540 con 0x55ecce741ad0 2023-07-21T18:43:15.188 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.167+0000 7f8ee904fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecce72c970 con 0x55ecce7362d0 2023-07-21T18:43:15.188 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.167+0000 7f8ee904fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecce7334d0 con 0x55ecce736be0 2023-07-21T18:43:15.189 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.167+0000 7f8ed6d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecce741ad0 0x55ecce73e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:15.189 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.167+0000 7f8ed6d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecce741ad0 0x55ecce73e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58788/0 (socket says 172.21.15.39:58788) 2023-07-21T18:43:15.189 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.167+0000 7f8ed6d9d700 1 -- 172.21.15.39:0/802278647 learned_addr learned my addr 172.21.15.39:0/802278647 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:15.189 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.167+0000 7f8ed5d9b700 1 --2- 172.21.15.39:0/802278647 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ecce736be0 0x55ecce73df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:15.189 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.167+0000 7f8ed659c700 1 --2- 172.21.15.39:0/802278647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecce7362d0 0x55ecce741040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:15.189 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.167+0000 7f8ed6d9d700 1 -- 172.21.15.39:0/802278647 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ecce736be0 msgr2=0x55ecce73df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:15.189 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.167+0000 7f8ed6d9d700 1 --2- 172.21.15.39:0/802278647 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ecce736be0 0x55ecce73df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:15.189 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.167+0000 7f8ed6d9d700 1 -- 172.21.15.39:0/802278647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecce7362d0 msgr2=0x55ecce741040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:15.190 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.167+0000 7f8ed6d9d700 1 --2- 172.21.15.39:0/802278647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecce7362d0 0x55ecce741040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:15.190 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.167+0000 7f8ed6d9d700 1 -- 172.21.15.39:0/802278647 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ecce49fc60 con 0x55ecce741ad0 2023-07-21T18:43:15.190 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.167+0000 7f8ed6d9d700 1 --2- 172.21.15.39:0/802278647 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecce741ad0 0x55ecce73e490 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f8ed00118e0 tx=0x7f8ed0009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b61accfe5accf0d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:15.190 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.168+0000 7f8ec57fa700 1 -- 172.21.15.39:0/802278647 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed000e040 con 0x55ecce741ad0 2023-07-21T18:43:15.190 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.168+0000 7f8ec57fa700 1 -- 172.21.15.39:0/802278647 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ed000f040 con 0x55ecce741ad0 2023-07-21T18:43:15.190 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.168+0000 7f8ec57fa700 1 -- 172.21.15.39:0/802278647 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed0007760 con 0x55ecce741ad0 2023-07-21T18:43:15.193 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.168+0000 7f8ee904fb80 1 -- 172.21.15.39:0/802278647 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecce741ad0 msgr2=0x55ecce73e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:15.193 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.168+0000 7f8ee904fb80 1 --2- 172.21.15.39:0/802278647 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecce741ad0 0x55ecce73e490 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f8ed00118e0 tx=0x7f8ed0009920 comp rx=0 tx=0).stop 2023-07-21T18:43:15.193 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.168+0000 7f8ee904fb80 1 -- 172.21.15.39:0/802278647 shutdown_connections 2023-07-21T18:43:15.194 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.168+0000 7f8ee904fb80 1 --2- 172.21.15.39:0/802278647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecce7362d0 0x55ecce741040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:15.194 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.168+0000 7f8ee904fb80 1 --2- 172.21.15.39:0/802278647 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecce741ad0 0x55ecce73e490 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:15.194 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.168+0000 7f8ee904fb80 1 --2- 172.21.15.39:0/802278647 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ecce736be0 0x55ecce73df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:15.194 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.168+0000 7f8ee904fb80 1 -- 172.21.15.39:0/802278647 >> 172.21.15.39:0/802278647 conn(0x55ecce72cda0 msgr2=0x55ecce739a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:15.194 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.168+0000 7f8ee904fb80 1 -- 172.21.15.39:0/802278647 shutdown_connections 2023-07-21T18:43:15.194 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.168+0000 7f8ee904fb80 1 -- 172.21.15.39:0/802278647 wait complete. 2023-07-21T18:43:15.194 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.169+0000 7f8ee904fb80 1 -- start start 2023-07-21T18:43:15.195 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.169+0000 7f8ee904fb80 1 Processor -- start 2023-07-21T18:43:15.195 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.169+0000 7f8ee904fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ecce7362d0 0x55ecce746bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:15.195 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.169+0000 7f8ee904fb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecce736be0 0x55ecce747100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:15.195 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.169+0000 7f8ee904fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecce741ad0 0x55ecce747640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:15.195 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.169+0000 7f8ee904fb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecce7a9580 con 0x55ecce736be0 2023-07-21T18:43:15.195 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.169+0000 7f8ee904fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecce7a96c0 con 0x55ecce741ad0 2023-07-21T18:43:15.195 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.169+0000 7f8ee904fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecce7a9830 con 0x55ecce7362d0 2023-07-21T18:43:15.195 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.169+0000 7f8ee904fb80 10 client.0 _ll_register_callbacks cb 0x7f8ec8099680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:43:15.196 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.169+0000 7f8ed5d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecce736be0 0x55ecce747100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:15.196 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.169+0000 7f8ed659c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ecce7362d0 0x55ecce746bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:15.196 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.169+0000 7f8ed5d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecce736be0 0x55ecce747100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58794/0 (socket says 172.21.15.39:58794) 2023-07-21T18:43:15.196 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.169+0000 7f8ed5d9b700 1 -- 172.21.15.39:0/49388677 learned_addr learned my addr 172.21.15.39:0/49388677 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:15.196 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.169+0000 7f8ed6d9d700 1 --2- 172.21.15.39:0/49388677 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecce741ad0 0x55ecce747640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:15.196 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.170+0000 7f8ed659c700 1 -- 172.21.15.39:0/49388677 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecce741ad0 msgr2=0x55ecce747640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:15.196 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.170+0000 7f8ed659c700 1 --2- 172.21.15.39:0/49388677 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecce741ad0 0x55ecce747640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:15.196 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.170+0000 7f8ed659c700 1 -- 172.21.15.39:0/49388677 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecce736be0 msgr2=0x55ecce747100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:15.197 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.170+0000 7f8ed659c700 1 --2- 172.21.15.39:0/49388677 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecce736be0 0x55ecce747100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:15.197 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.170+0000 7f8ed659c700 1 -- 172.21.15.39:0/49388677 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ed0011530 con 0x55ecce7362d0 2023-07-21T18:43:15.197 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.170+0000 7f8ed659c700 1 --2- 172.21.15.39:0/49388677 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ecce7362d0 0x55ecce746bc0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc00b6f0 tx=0x7f8ecc005a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa90181989fec004 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:15.197 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.170+0000 7f8ebdffb700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2023-07-21T18:43:15.197 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.170+0000 7f8ebdffb700 1 -- 172.21.15.39:0/49388677 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc02b040 con 0x55ecce7362d0 2023-07-21T18:43:15.197 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.170+0000 7f8ebdffb700 1 -- 172.21.15.39:0/49388677 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f8ecc03c0b0 con 0x55ecce7362d0 2023-07-21T18:43:15.197 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.170+0000 7f8ee904fb80 1 -- 172.21.15.39:0/49388677 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8ecc01e040 con 0x55ecce7362d0 2023-07-21T18:43:15.197 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.170+0000 7f8ebdffb700 1 -- 172.21.15.39:0/49388677 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ecc03c270 con 0x55ecce7362d0 2023-07-21T18:43:15.198 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.170+0000 7f8ee904fb80 10 client.25609 fetch_fsmap learned FSMap version 28 2023-07-21T18:43:15.198 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.170+0000 7f8ee904fb80 1 -- 172.21.15.39:0/49388677 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55ecce4dd250 con 0x55ecce7362d0 2023-07-21T18:43:15.198 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.170+0000 7f8ebdffb700 1 -- 172.21.15.39:0/49388677 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ecc01e040 con 0x55ecce7362d0 2023-07-21T18:43:15.198 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.171+0000 7f8ebdffb700 1 -- 172.21.15.39:0/49388677 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f8ecc04e580 con 0x55ecce7362d0 2023-07-21T18:43:15.198 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.171+0000 7f8ee904fb80 10 client.25609 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T18:43:15.198 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.171+0000 7f8ee904fb80 10 client.25609 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T18:43:15.198 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.171+0000 7f8ee904fb80 20 client.25609 populate_metadata read hostname 'smithi039' 2023-07-21T18:43:15.199 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.171+0000 7f8ee904fb80 1 --2- 172.21.15.39:0/49388677 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55ecce7aabe0 0x55ecce7acff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:15.199 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.171+0000 7f8ee904fb80 4 client.25609 mds_command: new command op to 24536 tid=0[{"prefix": "scrub status"}] 2023-07-21T18:43:15.199 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.171+0000 7f8ee904fb80 1 -- 172.21.15.39:0/49388677 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ecce644050 con 0x55ecce7aabe0 2023-07-21T18:43:15.199 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.171+0000 7f8ed5d9b700 1 --2- 172.21.15.39:0/49388677 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55ecce7aabe0 0x55ecce7acff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:15.199 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.171+0000 7f8ed5d9b700 1 --2- 172.21.15.39:0/49388677 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55ecce7aabe0 0x55ecce7acff0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fb3e13c96436fef server_cookie=d34c8a1a3ecee464 in_seq=0 out_seq=0 2023-07-21T18:43:15.199 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.171+0000 7f8ebdffb700 10 client.25609 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:43:15.199 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.171+0000 7f8ebdffb700 1 -- 172.21.15.39:0/49388677 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55ecce644050 con 0x55ecce7aabe0 2023-07-21T18:43:15.199 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.171+0000 7f8ebdffb700 10 client.25609 handle_command_reply: tid=0 2023-07-21T18:43:15.200 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.171+0000 7f8ee904fb80 1 client.25609 shutdown 2023-07-21T18:43:15.200 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.171+0000 7f8ee904fb80 1 -- 172.21.15.39:0/49388677 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ecce7362d0 msgr2=0x55ecce746bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:15.200 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.171+0000 7f8ee904fb80 1 --2- 172.21.15.39:0/49388677 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ecce7362d0 0x55ecce746bc0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc00b6f0 tx=0x7f8ecc005a90 comp rx=0 tx=0).stop 2023-07-21T18:43:15.200 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.171+0000 7f8ee904fb80 1 -- 172.21.15.39:0/49388677 shutdown_connections 2023-07-21T18:43:15.200 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.171+0000 7f8ee904fb80 1 --2- 172.21.15.39:0/49388677 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55ecce7aabe0 0x55ecce7acff0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:15.200 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.172+0000 7f8ee904fb80 1 --2- 172.21.15.39:0/49388677 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecce741ad0 0x55ecce747640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:15.200 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.172+0000 7f8ebdffb700 0 client.25609 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:43:15.200 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.172+0000 7f8ee904fb80 1 --2- 172.21.15.39:0/49388677 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ecce7362d0 0x55ecce746bc0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:15.201 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.172+0000 7f8ee904fb80 1 --2- 172.21.15.39:0/49388677 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecce736be0 0x55ecce747100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:15.201 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.172+0000 7f8ee904fb80 1 -- 172.21.15.39:0/49388677 >> 172.21.15.39:0/49388677 conn(0x55ecce72cda0 msgr2=0x55ecce738350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:15.201 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.172+0000 7f8ee904fb80 1 -- 172.21.15.39:0/49388677 shutdown_connections 2023-07-21T18:43:15.201 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.172+0000 7f8ee904fb80 1 -- 172.21.15.39:0/49388677 wait complete. 2023-07-21T18:43:15.201 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.172+0000 7f8ee904fb80 20 client.25609 trim_cache size 0 max 16384 2023-07-21T18:43:15.201 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2023-07-21T18:43:15.202 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T18:43:15.202 INFO:teuthology.orchestra.run.smithi039.stdout: "scrubs": { 2023-07-21T18:43:15.202 INFO:teuthology.orchestra.run.smithi039.stdout: "c6b195a6-7fa4-4f06-83fb-ff2911723603": { 2023-07-21T18:43:15.202 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2023-07-21T18:43:15.202 INFO:teuthology.orchestra.run.smithi039.stdout: "tag": "c6b195a6-7fa4-4f06-83fb-ff2911723603", 2023-07-21T18:43:15.202 INFO:teuthology.orchestra.run.smithi039.stdout: "options": "recursive,force" 2023-07-21T18:43:15.202 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T18:43:15.203 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T18:43:15.203 INFO:teuthology.orchestra.run.smithi039.stdout:} 2023-07-21T18:43:15.203 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.172+0000 7f8edcec9700 1 -- 172.21.15.39:0/3148543602 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f8ec00a13e0 msgr2=0x7f8ec00a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:15.203 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.172+0000 7f8edcec9700 1 --2- 172.21.15.39:0/3148543602 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f8ec00a13e0 0x7f8ec00a3890 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc00b2e0 tx=0x7f8ecc005fd0 comp rx=0 tx=0).stop 2023-07-21T18:43:15.204 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.172+0000 7f8edcec9700 1 -- 172.21.15.39:0/3148543602 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ed80feb90 msgr2=0x7f8ed819bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:15.204 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.172+0000 7f8edcec9700 1 --2- 172.21.15.39:0/3148543602 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ed80feb90 0x7f8ed819bf90 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f8ec800d980 tx=0x7f8ec800dc90 comp rx=0 tx=0).stop 2023-07-21T18:43:15.204 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.172+0000 7f8edcec9700 1 -- 172.21.15.39:0/3148543602 shutdown_connections 2023-07-21T18:43:15.204 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.172+0000 7f8edcec9700 1 --2- 172.21.15.39:0/3148543602 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f8ec00a13e0 0x7f8ec00a3890 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:15.204 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.172+0000 7f8edcec9700 1 --2- 172.21.15.39:0/3148543602 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ed80fe200 0x7f8ed819ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:15.204 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.172+0000 7f8edcec9700 1 --2- 172.21.15.39:0/3148543602 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ed8105c00 0x7f8ed8195ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:15.204 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.172+0000 7f8edcec9700 1 --2- 172.21.15.39:0/3148543602 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ed80feb90 0x7f8ed819bf90 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:15.205 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.172+0000 7f8edcec9700 1 -- 172.21.15.39:0/3148543602 >> 172.21.15.39:0/3148543602 conn(0x7f8ed8076200 msgr2=0x7f8ed80fbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:15.205 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.172+0000 7f8edcec9700 1 -- 172.21.15.39:0/3148543602 shutdown_connections 2023-07-21T18:43:15.205 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:15.172+0000 7f8edcec9700 1 -- 172.21.15.39:0/3148543602 wait complete. 2023-07-21T18:43:15.206 INFO:tasks.cephfs.filesystem:scrub status for tag:c6b195a6-7fa4-4f06-83fb-ff2911723603 - {'path': '/', 'tag': 'c6b195a6-7fa4-4f06-83fb-ff2911723603', 'options': 'recursive,force'} 2023-07-21T18:43:15.260 INFO:tasks.workunit.client.0.smithi039.stdout: 1 30908 6.49 MB/sec warmup 28 sec latency 3358.577 ms 2023-07-21T18:43:15.367 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:15 smithi193 ceph-mon[98204]: scrub summary: active paths [/] 2023-07-21T18:43:15.368 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:15 smithi193 ceph-mon[98204]: Scrub error on inode 0x1000000023f (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/ACCESS/FASTENER.MDB) see mds.a log and `damage ls` output for details 2023-07-21T18:43:15.368 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:15 smithi193 ceph-mon[98204]: scrub summary: idle+waiting paths [/] 2023-07-21T18:43:15.369 INFO:journalctl@ceph.mds.i.smithi193.stdout:Jul 21 18:43:15 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-i[124588]: 2023-07-21T18:43:15.002+0000 7f7954b5c700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000021f [3,head] /volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/COREL/ROSE.CPT auth v20 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15303=0-4194304@2} caps={15303=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@50},l=15303 | request=0 lock=0 importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b5656be680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000021f:[<0x10000000215/ROSE.CPT v20>,<0x10000000009/COREL v2662>,<0x10000000008/~dmtmp v40>,<0x10000000007/client0 v24>,<0x10000000006/clients v14>,<0x10000000005/tmp v14>,<0x100000001fb/client.0 v18>,<0x100000001fa/8d6c01e4-c829-4108-ba84-43703682343a v34>,<0x10000000001/sv_1 v57>,<0x10000000000/_nogroup v43>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x1000000021f:[<0x10000000215/ROSE.CPT v20>,<0x10000000009/COREL v4637>,<0x10000000008/~dmtmp v903>,<0x10000000007/client0 v59>,<0x10000000006/clients v48>,<0x10000000005/tmp v46>,<0x100000001fb/client.0 v48>,<0x100000001fa/8d6c01e4-c829-4108-ba84-43703682343a v52>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v25>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T18:43:15.369 INFO:journalctl@ceph.mds.i.smithi193.stdout:Jul 21 18:43:15 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-i[124588]: 2023-07-21T18:43:15.002+0000 7f7954b5c700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000021b [3,head] /volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/COREL/TEAPOT2.CPT auth v12 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15303=0-4194304@2} caps={15303=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@50},l=15303 | request=0 lock=0 importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b5656bec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000021b:[<0x10000000215/TEAPOT2.CPT v12>,<0x10000000009/COREL v2662>,<0x10000000008/~dmtmp v40>,<0x10000000007/client0 v24>,<0x10000000006/clients v14>,<0x10000000005/tmp v14>,<0x100000001fb/client.0 v18>,<0x100000001fa/8d6c01e4-c829-4108-ba84-43703682343a v34>,<0x10000000001/sv_1 v57>,<0x10000000000/_nogroup v43>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x1000000021b:[<0x10000000215/TEAPOT2.CPT v12>,<0x10000000009/COREL v4637>,<0x10000000008/~dmtmp v903>,<0x10000000007/client0 v59>,<0x10000000006/clients v48>,<0x10000000005/tmp v46>,<0x100000001fb/client.0 v48>,<0x100000001fa/8d6c01e4-c829-4108-ba84-43703682343a v52>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v25>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T18:43:15.369 INFO:journalctl@ceph.mds.i.smithi193.stdout:Jul 21 18:43:15 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-i[124588]: 2023-07-21T18:43:15.002+0000 7f7954b5c700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000220 [3,head] /volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/COREL/CORELDRW.CDT auth v22 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15303=0-4194304@2} caps={15303=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@50},l=15303 | request=0 lock=0 importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b5656bf700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000220:[<0x10000000215/CORELDRW.CDT v22>,<0x10000000009/COREL v2662>,<0x10000000008/~dmtmp v40>,<0x10000000007/client0 v24>,<0x10000000006/clients v14>,<0x10000000005/tmp v14>,<0x100000001fb/client.0 v18>,<0x100000001fa/8d6c01e4-c829-4108-ba84-43703682343a v34>,<0x10000000001/sv_1 v57>,<0x10000000000/_nogroup v43>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x10000000220:[<0x10000000215/CORELDRW.CDT v22>,<0x10000000009/COREL v4637>,<0x10000000008/~dmtmp v903>,<0x10000000007/client0 v59>,<0x10000000006/clients v48>,<0x10000000005/tmp v46>,<0x100000001fb/client.0 v48>,<0x100000001fa/8d6c01e4-c829-4108-ba84-43703682343a v52>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v25>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T18:43:15.370 INFO:journalctl@ceph.mds.i.smithi193.stdout:Jul 21 18:43:15 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-i[124588]: 2023-07-21T18:43:15.003+0000 7f7954b5c700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000021a [3,head] /volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/COREL/MAILBOX2.CPT auth v10 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15303=0-4194304@2} caps={15303=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@50},l=15303 | request=0 lock=0 importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b5656bf180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000021a:[<0x10000000215/MAILBOX2.CPT v10>,<0x10000000009/COREL v2662>,<0x10000000008/~dmtmp v40>,<0x10000000007/client0 v24>,<0x10000000006/clients v14>,<0x10000000005/tmp v14>,<0x100000001fb/client.0 v18>,<0x100000001fa/8d6c01e4-c829-4108-ba84-43703682343a v34>,<0x10000000001/sv_1 v57>,<0x10000000000/_nogroup v43>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x1000000021a:[<0x10000000215/MAILBOX2.CPT v10>,<0x10000000009/COREL v4637>,<0x10000000008/~dmtmp v903>,<0x10000000007/client0 v59>,<0x10000000006/clients v48>,<0x10000000005/tmp v46>,<0x100000001fb/client.0 v48>,<0x100000001fa/8d6c01e4-c829-4108-ba84-43703682343a v52>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v25>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T18:43:15.370 INFO:journalctl@ceph.mds.i.smithi193.stdout:Jul 21 18:43:15 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-i[124588]: 2023-07-21T18:43:15.003+0000 7f7954b5c700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000021e [3,head] /volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC2.CDR auth v18 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15303=0-4194304@2} caps={15303=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@50},l=15303 | request=0 lock=0 importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b5656fc000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000021e:[<0x10000000215/GRAPHIC2.CDR v18>,<0x10000000009/COREL v2662>,<0x10000000008/~dmtmp v40>,<0x10000000007/client0 v24>,<0x10000000006/clients v14>,<0x10000000005/tmp v14>,<0x100000001fb/client.0 v18>,<0x100000001fa/8d6c01e4-c829-4108-ba84-43703682343a v34>,<0x10000000001/sv_1 v57>,<0x10000000000/_nogroup v43>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x1000000021e:[<0x10000000215/GRAPHIC2.CDR v18>,<0x10000000009/COREL v4637>,<0x10000000008/~dmtmp v903>,<0x10000000007/client0 v59>,<0x10000000006/clients v48>,<0x10000000005/tmp v46>,<0x100000001fb/client.0 v48>,<0x100000001fa/8d6c01e4-c829-4108-ba84-43703682343a v52>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v25>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T18:43:15.370 INFO:journalctl@ceph.mds.i.smithi193.stdout:Jul 21 18:43:15 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-i[124588]: 2023-07-21T18:43:15.003+0000 7f7954b5c700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000218 [3,head] /volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/COREL/ARTISTIC.ACL auth v6 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15303=0-4194304@2} caps={15303=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@50},l=15303 | request=0 lock=0 importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b5656fc580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000218:[<0x10000000215/ARTISTIC.ACL v6>,<0x10000000009/COREL v2662>,<0x10000000008/~dmtmp v40>,<0x10000000007/client0 v24>,<0x10000000006/clients v14>,<0x10000000005/tmp v14>,<0x100000001fb/client.0 v18>,<0x100000001fa/8d6c01e4-c829-4108-ba84-43703682343a v34>,<0x10000000001/sv_1 v57>,<0x10000000000/_nogroup v43>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x10000000218:[<0x10000000215/ARTISTIC.ACL v6>,<0x10000000009/COREL v4637>,<0x10000000008/~dmtmp v903>,<0x10000000007/client0 v59>,<0x10000000006/clients v48>,<0x10000000005/tmp v46>,<0x100000001fb/client.0 v48>,<0x100000001fa/8d6c01e4-c829-4108-ba84-43703682343a v52>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v25>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T18:43:15.370 INFO:journalctl@ceph.mds.i.smithi193.stdout:Jul 21 18:43:15 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-i[124588]: 2023-07-21T18:43:15.003+0000 7f7954b5c700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000021c [3,head] /volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/COREL/SIGN2.CPT auth v14 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15303=0-4194304@2} caps={15303=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@50},l=15303 | request=0 lock=0 importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b5656fcb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000021c:[<0x10000000215/SIGN2.CPT v14>,<0x10000000009/COREL v2662>,<0x10000000008/~dmtmp v40>,<0x10000000007/client0 v24>,<0x10000000006/clients v14>,<0x10000000005/tmp v14>,<0x100000001fb/client.0 v18>,<0x100000001fa/8d6c01e4-c829-4108-ba84-43703682343a v34>,<0x10000000001/sv_1 v57>,<0x10000000000/_nogroup v43>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x1000000021c:[<0x10000000215/SIGN2.CPT v14>,<0x10000000009/COREL v4637>,<0x10000000008/~dmtmp v903>,<0x10000000007/client0 v59>,<0x10000000006/clients v48>,<0x10000000005/tmp v46>,<0x100000001fb/client.0 v48>,<0x100000001fa/8d6c01e4-c829-4108-ba84-43703682343a v52>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v25>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T18:43:15.370 INFO:journalctl@ceph.mds.i.smithi193.stdout:Jul 21 18:43:15 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-i[124588]: 2023-07-21T18:43:15.003+0000 7f7954b5c700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000217 [3,head] /volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/COREL/CDRBARS.CFG auth v4 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15303=0-4194304@2} caps={15303=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@50},l=15303 | request=0 lock=0 importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b5656fd080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000217:[<0x10000000215/CDRBARS.CFG v4>,<0x10000000009/COREL v2662>,<0x10000000008/~dmtmp v40>,<0x10000000007/client0 v24>,<0x10000000006/clients v14>,<0x10000000005/tmp v14>,<0x100000001fb/client.0 v18>,<0x100000001fa/8d6c01e4-c829-4108-ba84-43703682343a v34>,<0x10000000001/sv_1 v57>,<0x10000000000/_nogroup v43>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x10000000217:[<0x10000000215/CDRBARS.CFG v4>,<0x10000000009/COREL v4637>,<0x10000000008/~dmtmp v903>,<0x10000000007/client0 v59>,<0x10000000006/clients v48>,<0x10000000005/tmp v46>,<0x100000001fb/client.0 v48>,<0x100000001fa/8d6c01e4-c829-4108-ba84-43703682343a v52>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v25>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T18:43:15.370 INFO:journalctl@ceph.mds.i.smithi193.stdout:Jul 21 18:43:15 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-i[124588]: 2023-07-21T18:43:15.004+0000 7f7954b5c700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000021d [3,head] /volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/COREL/TRAIN.CDR auth v16 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15303=0-4194304@2} caps={15303=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@50},l=15303 | request=0 lock=0 importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b5656fd600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000021d:[<0x10000000215/TRAIN.CDR v16>,<0x10000000009/COREL v2662>,<0x10000000008/~dmtmp v40>,<0x10000000007/client0 v24>,<0x10000000006/clients v14>,<0x10000000005/tmp v14>,<0x100000001fb/client.0 v18>,<0x100000001fa/8d6c01e4-c829-4108-ba84-43703682343a v34>,<0x10000000001/sv_1 v57>,<0x10000000000/_nogroup v43>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x1000000021d:[<0x10000000215/TRAIN.CDR v16>,<0x10000000009/COREL v4637>,<0x10000000008/~dmtmp v903>,<0x10000000007/client0 v59>,<0x10000000006/clients v48>,<0x10000000005/tmp v46>,<0x100000001fb/client.0 v48>,<0x100000001fa/8d6c01e4-c829-4108-ba84-43703682343a v52>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v25>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T18:43:15.370 INFO:journalctl@ceph.mds.i.smithi193.stdout:Jul 21 18:43:15 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-i[124588]: 2023-07-21T18:43:15.004+0000 7f7954b5c700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000219 [3,head] /volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC1.CDR auth v8 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15303=0-4194304@2} caps={15303=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@50},l=15303 | request=0 lock=0 importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b5656fdb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000219:[<0x10000000215/GRAPHIC1.CDR v8>,<0x10000000009/COREL v2662>,<0x10000000008/~dmtmp v40>,<0x10000000007/client0 v24>,<0x10000000006/clients v14>,<0x10000000005/tmp v14>,<0x100000001fb/client.0 v18>,<0x100000001fa/8d6c01e4-c829-4108-ba84-43703682343a v34>,<0x10000000001/sv_1 v57>,<0x10000000000/_nogroup v43>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x10000000219:[<0x10000000215/GRAPHIC1.CDR v8>,<0x10000000009/COREL v4637>,<0x10000000008/~dmtmp v903>,<0x10000000007/client0 v59>,<0x10000000006/clients v48>,<0x10000000005/tmp v46>,<0x100000001fb/client.0 v48>,<0x100000001fa/8d6c01e4-c829-4108-ba84-43703682343a v52>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v25>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T18:43:15.371 INFO:journalctl@ceph.mds.i.smithi193.stdout:Jul 21 18:43:15 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-i[124588]: 2023-07-21T18:43:15.004+0000 7f7954b5c700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000216 [3,head] /volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/COREL/CDRROLS.CFG auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15303=0-4194304@2} caps={15303=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@50},l=15303 | request=0 lock=0 importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b5656fe100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000216:[<0x10000000215/CDRROLS.CFG v2>,<0x10000000009/COREL v2662>,<0x10000000008/~dmtmp v40>,<0x10000000007/client0 v24>,<0x10000000006/clients v14>,<0x10000000005/tmp v14>,<0x100000001fb/client.0 v18>,<0x100000001fa/8d6c01e4-c829-4108-ba84-43703682343a v34>,<0x10000000001/sv_1 v57>,<0x10000000000/_nogroup v43>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x10000000216:[<0x10000000215/CDRROLS.CFG v2>,<0x10000000009/COREL v4637>,<0x10000000008/~dmtmp v903>,<0x10000000007/client0 v59>,<0x10000000006/clients v48>,<0x10000000005/tmp v46>,<0x100000001fb/client.0 v48>,<0x100000001fa/8d6c01e4-c829-4108-ba84-43703682343a v52>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v25>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T18:43:15.371 INFO:journalctl@ceph.mds.i.smithi193.stdout:Jul 21 18:43:15 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-i[124588]: 2023-07-21T18:43:15.004+0000 7f7954b5c700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020a [3,head] /volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/WORD/BASEMACH.DOC auth v4 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15303=0-4194304@2} caps={15303=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@65},l=15303 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b5656bc580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000020a:[<0x10000000208/BASEMACH.DOC v4>,<0x10000000009/WORD v2342>,<0x10000000008/~dmtmp v40>,<0x10000000007/client0 v24>,<0x10000000006/clients v14>,<0x10000000005/tmp v14>,<0x100000001fb/client.0 v18>,<0x100000001fa/8d6c01e4-c829-4108-ba84-43703682343a v34>,<0x10000000001/sv_1 v57>,<0x10000000000/_nogroup v43>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x1000000020a:[<0x10000000208/BASEMACH.DOC v4>,<0x10000000009/WORD v4694>,<0x10000000008/~dmtmp v903>,<0x10000000007/client0 v59>,<0x10000000006/clients v48>,<0x10000000005/tmp v46>,<0x100000001fb/client.0 v48>,<0x100000001fa/8d6c01e4-c829-4108-ba84-43703682343a v52>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v25>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T18:43:15.371 INFO:journalctl@ceph.mds.i.smithi193.stdout:Jul 21 18:43:15 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-i[124588]: 2023-07-21T18:43:15.005+0000 7f7954b5c700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020b [3,head] /volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/WORD/FACTS.DOC auth v6 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15303=0-4194304@2} caps={15303=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@65},l=15303 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b5656bcb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000020b:[<0x10000000208/FACTS.DOC v6>,<0x10000000009/WORD v2342>,<0x10000000008/~dmtmp v40>,<0x10000000007/client0 v24>,<0x10000000006/clients v14>,<0x10000000005/tmp v14>,<0x100000001fb/client.0 v18>,<0x100000001fa/8d6c01e4-c829-4108-ba84-43703682343a v34>,<0x10000000001/sv_1 v57>,<0x10000000000/_nogroup v43>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x1000000020b:[<0x10000000208/FACTS.DOC v6>,<0x10000000009/WORD v4694>,<0x10000000008/~dmtmp v903>,<0x10000000007/client0 v59>,<0x10000000006/clients v48>,<0x10000000005/tmp v46>,<0x100000001fb/client.0 v48>,<0x100000001fa/8d6c01e4-c829-4108-ba84-43703682343a v52>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v25>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T18:43:15.371 INFO:journalctl@ceph.mds.i.smithi193.stdout:Jul 21 18:43:15 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-i[124588]: 2023-07-21T18:43:15.005+0000 7f7954b5c700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000209 [3,head] /volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/WORD/CHAP10.DOC auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15303=0-4194304@2} caps={15303=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@65},l=15303 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b5656bd080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000209:[<0x10000000208/CHAP10.DOC v2>,<0x10000000009/WORD v2342>,<0x10000000008/~dmtmp v40>,<0x10000000007/client0 v24>,<0x10000000006/clients v14>,<0x10000000005/tmp v14>,<0x100000001fb/client.0 v18>,<0x100000001fa/8d6c01e4-c829-4108-ba84-43703682343a v34>,<0x10000000001/sv_1 v57>,<0x10000000000/_nogroup v43>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x10000000209:[<0x10000000208/CHAP10.DOC v2>,<0x10000000009/WORD v4694>,<0x10000000008/~dmtmp v903>,<0x10000000007/client0 v59>,<0x10000000006/clients v48>,<0x10000000005/tmp v46>,<0x100000001fb/client.0 v48>,<0x100000001fa/8d6c01e4-c829-4108-ba84-43703682343a v52>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v25>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T18:43:15.371 INFO:journalctl@ceph.mds.i.smithi193.stdout:Jul 21 18:43:15 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-i[124588]: 2023-07-21T18:43:15.005+0000 7f7954b5c700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020d [3,head] /volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/EXCEL/PCMAGCD.XLS auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15303=0-4194304@2} caps={15303=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@71},l=15303 | request=0 lock=0 importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b565633700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000020d:[<0x1000000020c/PCMAGCD.XLS v2>,<0x10000000009/EXCEL v2502>,<0x10000000008/~dmtmp v40>,<0x10000000007/client0 v24>,<0x10000000006/clients v14>,<0x10000000005/tmp v14>,<0x100000001fb/client.0 v18>,<0x100000001fa/8d6c01e4-c829-4108-ba84-43703682343a v34>,<0x10000000001/sv_1 v57>,<0x10000000000/_nogroup v43>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x1000000020d:[<0x1000000020c/PCMAGCD.XLS v2>,<0x10000000009/EXCEL v4611>,<0x10000000008/~dmtmp v903>,<0x10000000007/client0 v59>,<0x10000000006/clients v48>,<0x10000000005/tmp v46>,<0x100000001fb/client.0 v48>,<0x100000001fa/8d6c01e4-c829-4108-ba84-43703682343a v52>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v25>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T18:43:15.371 INFO:journalctl@ceph.mds.i.smithi193.stdout:Jul 21 18:43:15 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-i[124588]: 2023-07-21T18:43:15.005+0000 7f7954b5c700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020e [3,head] /volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/EXCEL/SALES.XLS auth v4 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15303=0-4194304@2} caps={15303=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@71},l=15303 | request=0 lock=0 importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b5656bc000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000020e:[<0x1000000020c/SALES.XLS v4>,<0x10000000009/EXCEL v2502>,<0x10000000008/~dmtmp v40>,<0x10000000007/client0 v24>,<0x10000000006/clients v14>,<0x10000000005/tmp v14>,<0x100000001fb/client.0 v18>,<0x100000001fa/8d6c01e4-c829-4108-ba84-43703682343a v34>,<0x10000000001/sv_1 v57>,<0x10000000000/_nogroup v43>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x1000000020e:[<0x1000000020c/SALES.XLS v4>,<0x10000000009/EXCEL v4611>,<0x10000000008/~dmtmp v903>,<0x10000000007/client0 v59>,<0x10000000006/clients v48>,<0x10000000005/tmp v46>,<0x100000001fb/client.0 v48>,<0x100000001fa/8d6c01e4-c829-4108-ba84-43703682343a v52>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v25>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T18:43:15.372 INFO:journalctl@ceph.mds.i.smithi193.stdout:Jul 21 18:43:15 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-i[124588]: 2023-07-21T18:43:15.011+0000 7f7954b5c700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000226 [3,head] /volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/PM/T1.XLS auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15303=0-4194304@2} caps={15303=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@115},l=15303 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b5656bd600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000226:[<0x10000000225/T1.XLS v2>,<0x10000000009/PM v2748>,<0x10000000008/~dmtmp v40>,<0x10000000007/client0 v24>,<0x10000000006/clients v14>,<0x10000000005/tmp v14>,<0x100000001fb/client.0 v18>,<0x100000001fa/8d6c01e4-c829-4108-ba84-43703682343a v34>,<0x10000000001/sv_1 v57>,<0x10000000000/_nogroup v43>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x10000000226:[<0x10000000225/T1.XLS v2>,<0x10000000009/PM v4639>,<0x10000000008/~dmtmp v903>,<0x10000000007/client0 v59>,<0x10000000006/clients v48>,<0x10000000005/tmp v46>,<0x100000001fb/client.0 v48>,<0x100000001fa/8d6c01e4-c829-4108-ba84-43703682343a v52>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v25>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T18:43:15.372 INFO:journalctl@ceph.mds.i.smithi193.stdout:Jul 21 18:43:15 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-i[124588]: 2023-07-21T18:43:15.012+0000 7f7954b5c700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000228 [3,head] /volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/PM/MOVED.DOC auth v6 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15303=0-4194304@2} caps={15303=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@115},l=15303 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b5656be100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000228:[<0x10000000225/MOVED.DOC v6>,<0x10000000009/PM v2748>,<0x10000000008/~dmtmp v40>,<0x10000000007/client0 v24>,<0x10000000006/clients v14>,<0x10000000005/tmp v14>,<0x100000001fb/client.0 v18>,<0x100000001fa/8d6c01e4-c829-4108-ba84-43703682343a v34>,<0x10000000001/sv_1 v57>,<0x10000000000/_nogroup v43>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x10000000228:[<0x10000000225/MOVED.DOC v6>,<0x10000000009/PM v4639>,<0x10000000008/~dmtmp v903>,<0x10000000007/client0 v59>,<0x10000000006/clients v48>,<0x10000000005/tmp v46>,<0x100000001fb/client.0 v48>,<0x100000001fa/8d6c01e4-c829-4108-ba84-43703682343a v52>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v25>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T18:43:15.372 INFO:journalctl@ceph.mds.i.smithi193.stdout:Jul 21 18:43:15 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-i[124588]: 2023-07-21T18:43:15.012+0000 7f7954b5c700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000227 [3,head] /volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/PM/EVENTS.DOC auth v4 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15303=0-4194304@2} caps={15303=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@115},l=15303 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b5656bdb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000227:[<0x10000000225/EVENTS.DOC v4>,<0x10000000009/PM v2748>,<0x10000000008/~dmtmp v40>,<0x10000000007/client0 v24>,<0x10000000006/clients v14>,<0x10000000005/tmp v14>,<0x100000001fb/client.0 v18>,<0x100000001fa/8d6c01e4-c829-4108-ba84-43703682343a v34>,<0x10000000001/sv_1 v57>,<0x10000000000/_nogroup v43>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x10000000227:[<0x10000000225/EVENTS.DOC v4>,<0x10000000009/PM v4639>,<0x10000000008/~dmtmp v903>,<0x10000000007/client0 v59>,<0x10000000006/clients v48>,<0x10000000005/tmp v46>,<0x100000001fb/client.0 v48>,<0x100000001fa/8d6c01e4-c829-4108-ba84-43703682343a v52>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v25>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T18:43:15.386 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:15 smithi155 ceph-mon[97934]: scrub summary: active paths [/] 2023-07-21T18:43:15.386 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:15 smithi155 ceph-mon[97934]: Scrub error on inode 0x1000000023f (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/ACCESS/FASTENER.MDB) see mds.a log and `damage ls` output for details 2023-07-21T18:43:15.387 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:15 smithi155 ceph-mon[97934]: scrub summary: idle+waiting paths [/] 2023-07-21T18:43:15.387 INFO:journalctl@ceph.mds.h.smithi155.stdout:Jul 21 18:43:15 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-h[122931]: 2023-07-21T18:43:15.002+0000 7f5ed46a0700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000241 [3,head] /volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/SEED/SMALL.FIL auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15303=0-4194304@2} caps={15303=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@3},l=15303 | request=0 lock=0 importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x562377e27b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000241:[<0x10000000240/SMALL.FIL v2>,<0x10000000009/SEED v120>,<0x10000000008/~dmtmp v40>,<0x10000000007/client0 v24>,<0x10000000006/clients v14>,<0x10000000005/tmp v14>,<0x100000001fb/client.0 v18>,<0x100000001fa/8d6c01e4-c829-4108-ba84-43703682343a v34>,<0x10000000001/sv_1 v57>,<0x10000000000/_nogroup v43>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x10000000241:[<0x10000000240/SMALL.FIL v2>,<0x10000000009/SEED v120>,<0x10000000008/~dmtmp v903>,<0x10000000007/client0 v59>,<0x10000000006/clients v48>,<0x10000000005/tmp v46>,<0x100000001fb/client.0 v48>,<0x100000001fa/8d6c01e4-c829-4108-ba84-43703682343a v52>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v25>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T18:43:15.388 INFO:journalctl@ceph.mds.h.smithi155.stdout:Jul 21 18:43:15 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-h[122931]: 2023-07-21T18:43:15.002+0000 7f5ed46a0700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000243 [3,head] /volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/SEED/LARGE.FIL auth v6 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15303=0-4194304@2} caps={15303=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@3},l=15303 | request=0 lock=0 importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x562377e28680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000243:[<0x10000000240/LARGE.FIL v6>,<0x10000000009/SEED v120>,<0x10000000008/~dmtmp v40>,<0x10000000007/client0 v24>,<0x10000000006/clients v14>,<0x10000000005/tmp v14>,<0x100000001fb/client.0 v18>,<0x100000001fa/8d6c01e4-c829-4108-ba84-43703682343a v34>,<0x10000000001/sv_1 v57>,<0x10000000000/_nogroup v43>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x10000000243:[<0x10000000240/LARGE.FIL v6>,<0x10000000009/SEED v120>,<0x10000000008/~dmtmp v903>,<0x10000000007/client0 v59>,<0x10000000006/clients v48>,<0x10000000005/tmp v46>,<0x100000001fb/client.0 v48>,<0x100000001fa/8d6c01e4-c829-4108-ba84-43703682343a v52>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v25>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T18:43:15.388 INFO:journalctl@ceph.mds.h.smithi155.stdout:Jul 21 18:43:15 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-h[122931]: 2023-07-21T18:43:15.002+0000 7f5ed46a0700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000242 [3,head] /volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/SEED/MEDIUM.FIL auth v4 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15303=0-4194304@2} caps={15303=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@3},l=15303 | request=0 lock=0 importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x562377e28100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000242:[<0x10000000240/MEDIUM.FIL v4>,<0x10000000009/SEED v120>,<0x10000000008/~dmtmp v40>,<0x10000000007/client0 v24>,<0x10000000006/clients v14>,<0x10000000005/tmp v14>,<0x100000001fb/client.0 v18>,<0x100000001fa/8d6c01e4-c829-4108-ba84-43703682343a v34>,<0x10000000001/sv_1 v57>,<0x10000000000/_nogroup v43>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x10000000242:[<0x10000000240/MEDIUM.FIL v4>,<0x10000000009/SEED v120>,<0x10000000008/~dmtmp v903>,<0x10000000007/client0 v59>,<0x10000000006/clients v48>,<0x10000000005/tmp v46>,<0x100000001fb/client.0 v48>,<0x100000001fa/8d6c01e4-c829-4108-ba84-43703682343a v52>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v25>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T18:43:15.388 INFO:journalctl@ceph.mds.h.smithi155.stdout:Jul 21 18:43:15 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-h[122931]: 2023-07-21T18:43:15.003+0000 7f5ed46a0700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000222 [3,head] /volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/WORDPRO/BENCHS.LWP auth{0=1} v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15303=0-4194304@2} caps={15303=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@65},l=15303 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x562377c92680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000222:[<0x10000000221/BENCHS.LWP v2>,<0x10000000009/WORDPRO v2580>,<0x10000000008/~dmtmp v40>,<0x10000000007/client0 v24>,<0x10000000006/clients v14>,<0x10000000005/tmp v14>,<0x100000001fb/client.0 v18>,<0x100000001fa/8d6c01e4-c829-4108-ba84-43703682343a v34>,<0x10000000001/sv_1 v57>,<0x10000000000/_nogroup v43>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x10000000222:[<0x10000000221/BENCHS.LWP v2>,<0x10000000009/WORDPRO v4487>,<0x10000000008/~dmtmp v903>,<0x10000000007/client0 v59>,<0x10000000006/clients v48>,<0x10000000005/tmp v46>,<0x100000001fb/client.0 v48>,<0x100000001fa/8d6c01e4-c829-4108-ba84-43703682343a v52>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v25>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T18:43:15.388 INFO:journalctl@ceph.mds.h.smithi155.stdout:Jul 21 18:43:15 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-h[122931]: 2023-07-21T18:43:15.003+0000 7f5ed46a0700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000223 [3,head] /volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/WORDPRO/RESULTS.XLS auth v4 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15303=0-4194304@2} caps={15303=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@65},l=15303 | request=0 lock=0 importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x562377c92c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000223:[<0x10000000221/RESULTS.XLS v4>,<0x10000000009/WORDPRO v2580>,<0x10000000008/~dmtmp v40>,<0x10000000007/client0 v24>,<0x10000000006/clients v14>,<0x10000000005/tmp v14>,<0x100000001fb/client.0 v18>,<0x100000001fa/8d6c01e4-c829-4108-ba84-43703682343a v34>,<0x10000000001/sv_1 v57>,<0x10000000000/_nogroup v43>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x10000000223:[<0x10000000221/RESULTS.XLS v4>,<0x10000000009/WORDPRO v4487>,<0x10000000008/~dmtmp v903>,<0x10000000007/client0 v59>,<0x10000000006/clients v48>,<0x10000000005/tmp v46>,<0x100000001fb/client.0 v48>,<0x100000001fa/8d6c01e4-c829-4108-ba84-43703682343a v52>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v25>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T18:43:15.388 INFO:journalctl@ceph.mds.h.smithi155.stdout:Jul 21 18:43:15 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-h[122931]: 2023-07-21T18:43:15.003+0000 7f5ed46a0700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000224 [3,head] /volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/WORDPRO/NEWS1_1.LWP auth v6 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15303=0-4194304@2} caps={15303=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@65},l=15303 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x562377e26580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000224:[<0x10000000221/NEWS1_1.LWP v6>,<0x10000000009/WORDPRO v2580>,<0x10000000008/~dmtmp v40>,<0x10000000007/client0 v24>,<0x10000000006/clients v14>,<0x10000000005/tmp v14>,<0x100000001fb/client.0 v18>,<0x100000001fa/8d6c01e4-c829-4108-ba84-43703682343a v34>,<0x10000000001/sv_1 v57>,<0x10000000000/_nogroup v43>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x10000000224:[<0x10000000221/NEWS1_1.LWP v6>,<0x10000000009/WORDPRO v4487>,<0x10000000008/~dmtmp v903>,<0x10000000007/client0 v59>,<0x10000000006/clients v48>,<0x10000000005/tmp v46>,<0x100000001fb/client.0 v48>,<0x100000001fa/8d6c01e4-c829-4108-ba84-43703682343a v52>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v25>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T18:43:15.388 INFO:journalctl@ceph.mds.h.smithi155.stdout:Jul 21 18:43:15 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-h[122931]: 2023-07-21T18:43:15.019+0000 7f5ed46a0700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000210 [3,head] /volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/PWRPNT/PCBENCHM.PPT auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15303=0-4194304@2} caps={15303=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@69},l=15303 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x562377d85180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000210:[<0x1000000020f/PCBENCHM.PPT v2>,<0x10000000009/PWRPNT v2286>,<0x10000000008/~dmtmp v40>,<0x10000000007/client0 v24>,<0x10000000006/clients v14>,<0x10000000005/tmp v14>,<0x100000001fb/client.0 v18>,<0x100000001fa/8d6c01e4-c829-4108-ba84-43703682343a v34>,<0x10000000001/sv_1 v57>,<0x10000000000/_nogroup v43>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x10000000210:[<0x1000000020f/PCBENCHM.PPT v2>,<0x10000000009/PWRPNT v4664>,<0x10000000008/~dmtmp v903>,<0x10000000007/client0 v59>,<0x10000000006/clients v48>,<0x10000000005/tmp v46>,<0x100000001fb/client.0 v48>,<0x100000001fa/8d6c01e4-c829-4108-ba84-43703682343a v52>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v25>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T18:43:15.388 INFO:journalctl@ceph.mds.h.smithi155.stdout:Jul 21 18:43:15 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-h[122931]: 2023-07-21T18:43:15.019+0000 7f5ed46a0700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000211 [3,head] /volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/PWRPNT/ZD16.BMP auth{0=1} v4 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15303=0-4194304@2} caps={15303=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@69},l=15303 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x562377d85700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000211:[<0x1000000020f/ZD16.BMP v4>,<0x10000000009/PWRPNT v2286>,<0x10000000008/~dmtmp v40>,<0x10000000007/client0 v24>,<0x10000000006/clients v14>,<0x10000000005/tmp v14>,<0x100000001fb/client.0 v18>,<0x100000001fa/8d6c01e4-c829-4108-ba84-43703682343a v34>,<0x10000000001/sv_1 v57>,<0x10000000000/_nogroup v43>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x10000000211:[<0x1000000020f/ZD16.BMP v4>,<0x10000000009/PWRPNT v4664>,<0x10000000008/~dmtmp v903>,<0x10000000007/client0 v59>,<0x10000000006/clients v48>,<0x10000000005/tmp v46>,<0x100000001fb/client.0 v48>,<0x100000001fa/8d6c01e4-c829-4108-ba84-43703682343a v52>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v25>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T18:43:15.389 INFO:journalctl@ceph.mds.h.smithi155.stdout:Jul 21 18:43:15 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-h[122931]: 2023-07-21T18:43:15.020+0000 7f5ed46a0700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000214 [3,head] /volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/PWRPNT/TRIDOTS.POT auth v10 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15303=0-4194304@2} caps={15303=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@69},l=15303 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x562377c90580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000214:[<0x1000000020f/TRIDOTS.POT v10>,<0x10000000009/PWRPNT v2286>,<0x10000000008/~dmtmp v40>,<0x10000000007/client0 v24>,<0x10000000006/clients v14>,<0x10000000005/tmp v14>,<0x100000001fb/client.0 v18>,<0x100000001fa/8d6c01e4-c829-4108-ba84-43703682343a v34>,<0x10000000001/sv_1 v57>,<0x10000000000/_nogroup v43>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x10000000214:[<0x1000000020f/TRIDOTS.POT v10>,<0x10000000009/PWRPNT v4664>,<0x10000000008/~dmtmp v903>,<0x10000000007/client0 v59>,<0x10000000006/clients v48>,<0x10000000005/tmp v46>,<0x100000001fb/client.0 v48>,<0x100000001fa/8d6c01e4-c829-4108-ba84-43703682343a v52>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v25>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T18:43:15.389 INFO:journalctl@ceph.mds.h.smithi155.stdout:Jul 21 18:43:15 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-h[122931]: 2023-07-21T18:43:15.020+0000 7f5ed46a0700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000213 [3,head] /volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/PWRPNT/TIPS.PPT auth v8 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15303=0-4194304@2} caps={15303=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@69},l=15303 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x562377c90b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000213:[<0x1000000020f/TIPS.PPT v8>,<0x10000000009/PWRPNT v2286>,<0x10000000008/~dmtmp v40>,<0x10000000007/client0 v24>,<0x10000000006/clients v14>,<0x10000000005/tmp v14>,<0x100000001fb/client.0 v18>,<0x100000001fa/8d6c01e4-c829-4108-ba84-43703682343a v34>,<0x10000000001/sv_1 v57>,<0x10000000000/_nogroup v43>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x10000000213:[<0x1000000020f/TIPS.PPT v8>,<0x10000000009/PWRPNT v4664>,<0x10000000008/~dmtmp v903>,<0x10000000007/client0 v59>,<0x10000000006/clients v48>,<0x10000000005/tmp v46>,<0x100000001fb/client.0 v48>,<0x100000001fa/8d6c01e4-c829-4108-ba84-43703682343a v52>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v25>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T18:43:15.389 INFO:journalctl@ceph.mds.h.smithi155.stdout:Jul 21 18:43:15 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-h[122931]: 2023-07-21T18:43:15.020+0000 7f5ed46a0700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000212 [3,head] /volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/PWRPNT/PPTOOLS1.PPA auth{0=1} v6 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15303=0-4194304@2} caps={15303=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@69},l=15303 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x562377c91080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000212:[<0x1000000020f/PPTOOLS1.PPA v6>,<0x10000000009/PWRPNT v2286>,<0x10000000008/~dmtmp v40>,<0x10000000007/client0 v24>,<0x10000000006/clients v14>,<0x10000000005/tmp v14>,<0x100000001fb/client.0 v18>,<0x100000001fa/8d6c01e4-c829-4108-ba84-43703682343a v34>,<0x10000000001/sv_1 v57>,<0x10000000000/_nogroup v43>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x10000000212:[<0x1000000020f/PPTOOLS1.PPA v6>,<0x10000000009/PWRPNT v4664>,<0x10000000008/~dmtmp v903>,<0x10000000007/client0 v59>,<0x10000000006/clients v48>,<0x10000000005/tmp v46>,<0x100000001fb/client.0 v48>,<0x100000001fa/8d6c01e4-c829-4108-ba84-43703682343a v52>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v25>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T18:43:15.685 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:15 smithi039 ceph-mon[96489]: scrub summary: active paths [/] 2023-07-21T18:43:15.685 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:15 smithi039 ceph-mon[96489]: Scrub error on inode 0x1000000023f (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/ACCESS/FASTENER.MDB) see mds.a log and `damage ls` output for details 2023-07-21T18:43:15.686 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:15 smithi039 ceph-mon[96489]: scrub summary: idle+waiting paths [/] 2023-07-21T18:43:16.260 INFO:tasks.workunit.client.0.smithi039.stdout: 1 32631 6.66 MB/sec warmup 29 sec latency 29.171 ms 2023-07-21T18:43:16.618 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:16 smithi193 ceph-mon[98204]: pgmap v48: 97 pgs: 97 active+clean; 12 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 611 KiB/s wr, 142 op/s 2023-07-21T18:43:16.618 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:16 smithi193 ceph-mon[98204]: scrub queued for path: / 2023-07-21T18:43:16.618 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:16 smithi193 ceph-mon[98204]: scrub summary: idle+waiting paths [/] 2023-07-21T18:43:16.618 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:16 smithi193 ceph-mon[98204]: scrub summary: active paths [/] 2023-07-21T18:43:16.619 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:16 smithi193 ceph-mon[98204]: scrub summary: active paths [/] 2023-07-21T18:43:16.619 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:16 smithi193 ceph-mon[98204]: scrub summary: active paths [/] 2023-07-21T18:43:16.619 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:16 smithi193 ceph-mon[98204]: Scrub error on inode 0x10000000241 (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/SEED/SMALL.FIL) see mds.h log and `damage ls` output for details 2023-07-21T18:43:16.619 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:16 smithi193 ceph-mon[98204]: Scrub error on inode 0x10000000243 (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/SEED/LARGE.FIL) see mds.h log and `damage ls` output for details 2023-07-21T18:43:16.619 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:16 smithi193 ceph-mon[98204]: Scrub error on inode 0x1000000021f (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/COREL/ROSE.CPT) see mds.i log and `damage ls` output for details 2023-07-21T18:43:16.619 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:16 smithi193 ceph-mon[98204]: Scrub error on inode 0x10000000242 (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/SEED/MEDIUM.FIL) see mds.h log and `damage ls` output for details 2023-07-21T18:43:16.619 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:16 smithi193 ceph-mon[98204]: Scrub error on inode 0x10000000222 (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/WORDPRO/BENCHS.LWP) see mds.h log and `damage ls` output for details 2023-07-21T18:43:16.620 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:16 smithi193 ceph-mon[98204]: Scrub error on inode 0x10000000223 (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/WORDPRO/RESULTS.XLS) see mds.h log and `damage ls` output for details 2023-07-21T18:43:16.620 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:16 smithi193 ceph-mon[98204]: Scrub error on inode 0x10000000224 (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/WORDPRO/NEWS1_1.LWP) see mds.h log and `damage ls` output for details 2023-07-21T18:43:16.620 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:16 smithi193 ceph-mon[98204]: scrub summary: idle+waiting paths [/] 2023-07-21T18:43:16.620 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:16 smithi193 ceph-mon[98204]: Scrub error on inode 0x1000000021b (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/COREL/TEAPOT2.CPT) see mds.i log and `damage ls` output for details 2023-07-21T18:43:16.620 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:16 smithi193 ceph-mon[98204]: Scrub error on inode 0x10000000220 (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/COREL/CORELDRW.CDT) see mds.i log and `damage ls` output for details 2023-07-21T18:43:16.620 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:16 smithi193 ceph-mon[98204]: Scrub error on inode 0x1000000021a (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/COREL/MAILBOX2.CPT) see mds.i log and `damage ls` output for details 2023-07-21T18:43:16.620 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:16 smithi193 ceph-mon[98204]: Scrub error on inode 0x1000000021e (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC2.CDR) see mds.i log and `damage ls` output for details 2023-07-21T18:43:16.620 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:16 smithi193 ceph-mon[98204]: Scrub error on inode 0x10000000218 (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/COREL/ARTISTIC.ACL) see mds.i log and `damage ls` output for details 2023-07-21T18:43:16.621 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:16 smithi193 ceph-mon[98204]: Scrub error on inode 0x1000000021c (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/COREL/SIGN2.CPT) see mds.i log and `damage ls` output for details 2023-07-21T18:43:16.621 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:16 smithi193 ceph-mon[98204]: Scrub error on inode 0x10000000217 (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/COREL/CDRBARS.CFG) see mds.i log and `damage ls` output for details 2023-07-21T18:43:16.621 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:16 smithi193 ceph-mon[98204]: Scrub error on inode 0x1000000021d (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/COREL/TRAIN.CDR) see mds.i log and `damage ls` output for details 2023-07-21T18:43:16.621 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:16 smithi193 ceph-mon[98204]: Scrub error on inode 0x10000000219 (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC1.CDR) see mds.i log and `damage ls` output for details 2023-07-21T18:43:16.621 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:16 smithi193 ceph-mon[98204]: Scrub error on inode 0x10000000216 (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/COREL/CDRROLS.CFG) see mds.i log and `damage ls` output for details 2023-07-21T18:43:16.621 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:16 smithi193 ceph-mon[98204]: Scrub error on inode 0x1000000020a (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/WORD/BASEMACH.DOC) see mds.i log and `damage ls` output for details 2023-07-21T18:43:16.621 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:16 smithi193 ceph-mon[98204]: Scrub error on inode 0x1000000020b (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/WORD/FACTS.DOC) see mds.i log and `damage ls` output for details 2023-07-21T18:43:16.622 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:16 smithi193 ceph-mon[98204]: Scrub error on inode 0x10000000209 (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/WORD/CHAP10.DOC) see mds.i log and `damage ls` output for details 2023-07-21T18:43:16.622 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:16 smithi193 ceph-mon[98204]: Scrub error on inode 0x1000000020d (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/EXCEL/PCMAGCD.XLS) see mds.i log and `damage ls` output for details 2023-07-21T18:43:16.622 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:16 smithi193 ceph-mon[98204]: Scrub error on inode 0x1000000020e (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/EXCEL/SALES.XLS) see mds.i log and `damage ls` output for details 2023-07-21T18:43:16.622 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:16 smithi193 ceph-mon[98204]: scrub summary: idle+waiting paths [/] 2023-07-21T18:43:16.622 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:16 smithi193 ceph-mon[98204]: scrub summary: active paths [/] 2023-07-21T18:43:16.622 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:16 smithi193 ceph-mon[98204]: Scrub error on inode 0x10000000226 (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/PM/T1.XLS) see mds.i log and `damage ls` output for details 2023-07-21T18:43:16.622 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:16 smithi193 ceph-mon[98204]: Scrub error on inode 0x10000000228 (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/PM/MOVED.DOC) see mds.i log and `damage ls` output for details 2023-07-21T18:43:16.622 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:16 smithi193 ceph-mon[98204]: Scrub error on inode 0x10000000227 (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/PM/EVENTS.DOC) see mds.i log and `damage ls` output for details 2023-07-21T18:43:16.623 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:16 smithi193 ceph-mon[98204]: scrub summary: idle+waiting paths [/] 2023-07-21T18:43:16.623 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:16 smithi193 ceph-mon[98204]: scrub summary: active paths [/] 2023-07-21T18:43:16.623 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:16 smithi193 ceph-mon[98204]: Scrub error on inode 0x10000000210 (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/PWRPNT/PCBENCHM.PPT) see mds.h log and `damage ls` output for details 2023-07-21T18:43:16.623 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:16 smithi193 ceph-mon[98204]: Scrub error on inode 0x10000000211 (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/PWRPNT/ZD16.BMP) see mds.h log and `damage ls` output for details 2023-07-21T18:43:16.623 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:16 smithi193 ceph-mon[98204]: Scrub error on inode 0x10000000214 (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/PWRPNT/TRIDOTS.POT) see mds.h log and `damage ls` output for details 2023-07-21T18:43:16.623 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:16 smithi193 ceph-mon[98204]: Scrub error on inode 0x10000000213 (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/PWRPNT/TIPS.PPT) see mds.h log and `damage ls` output for details 2023-07-21T18:43:16.623 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:16 smithi193 ceph-mon[98204]: Scrub error on inode 0x10000000212 (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/PWRPNT/PPTOOLS1.PPA) see mds.h log and `damage ls` output for details 2023-07-21T18:43:16.623 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:16 smithi193 ceph-mon[98204]: scrub summary: idle+waiting paths [/] 2023-07-21T18:43:16.624 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:16 smithi193 ceph-mon[98204]: scrub summary: idle+waiting paths [/] 2023-07-21T18:43:16.624 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:16 smithi193 ceph-mon[98204]: mgrmap e24: x(active, since 90s), standbys: y, z 2023-07-21T18:43:16.636 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:16 smithi155 ceph-mon[97934]: pgmap v48: 97 pgs: 97 active+clean; 12 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 611 KiB/s wr, 142 op/s 2023-07-21T18:43:16.637 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:16 smithi155 ceph-mon[97934]: scrub queued for path: / 2023-07-21T18:43:16.637 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:16 smithi155 ceph-mon[97934]: scrub summary: idle+waiting paths [/] 2023-07-21T18:43:16.637 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:16 smithi155 ceph-mon[97934]: scrub summary: active paths [/] 2023-07-21T18:43:16.637 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:16 smithi155 ceph-mon[97934]: scrub summary: active paths [/] 2023-07-21T18:43:16.637 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:16 smithi155 ceph-mon[97934]: scrub summary: active paths [/] 2023-07-21T18:43:16.637 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:16 smithi155 ceph-mon[97934]: Scrub error on inode 0x10000000241 (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/SEED/SMALL.FIL) see mds.h log and `damage ls` output for details 2023-07-21T18:43:16.638 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:16 smithi155 ceph-mon[97934]: Scrub error on inode 0x10000000243 (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/SEED/LARGE.FIL) see mds.h log and `damage ls` output for details 2023-07-21T18:43:16.638 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:16 smithi155 ceph-mon[97934]: Scrub error on inode 0x1000000021f (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/COREL/ROSE.CPT) see mds.i log and `damage ls` output for details 2023-07-21T18:43:16.638 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:16 smithi155 ceph-mon[97934]: Scrub error on inode 0x10000000242 (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/SEED/MEDIUM.FIL) see mds.h log and `damage ls` output for details 2023-07-21T18:43:16.638 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:16 smithi155 ceph-mon[97934]: Scrub error on inode 0x10000000222 (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/WORDPRO/BENCHS.LWP) see mds.h log and `damage ls` output for details 2023-07-21T18:43:16.638 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:16 smithi155 ceph-mon[97934]: Scrub error on inode 0x10000000223 (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/WORDPRO/RESULTS.XLS) see mds.h log and `damage ls` output for details 2023-07-21T18:43:16.638 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:16 smithi155 ceph-mon[97934]: Scrub error on inode 0x10000000224 (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/WORDPRO/NEWS1_1.LWP) see mds.h log and `damage ls` output for details 2023-07-21T18:43:16.638 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:16 smithi155 ceph-mon[97934]: scrub summary: idle+waiting paths [/] 2023-07-21T18:43:16.638 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:16 smithi155 ceph-mon[97934]: Scrub error on inode 0x1000000021b (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/COREL/TEAPOT2.CPT) see mds.i log and `damage ls` output for details 2023-07-21T18:43:16.639 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:16 smithi155 ceph-mon[97934]: Scrub error on inode 0x10000000220 (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/COREL/CORELDRW.CDT) see mds.i log and `damage ls` output for details 2023-07-21T18:43:16.639 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:16 smithi155 ceph-mon[97934]: Scrub error on inode 0x1000000021a (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/COREL/MAILBOX2.CPT) see mds.i log and `damage ls` output for details 2023-07-21T18:43:16.639 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:16 smithi155 ceph-mon[97934]: Scrub error on inode 0x1000000021e (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC2.CDR) see mds.i log and `damage ls` output for details 2023-07-21T18:43:16.639 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:16 smithi155 ceph-mon[97934]: Scrub error on inode 0x10000000218 (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/COREL/ARTISTIC.ACL) see mds.i log and `damage ls` output for details 2023-07-21T18:43:16.639 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:16 smithi155 ceph-mon[97934]: Scrub error on inode 0x1000000021c (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/COREL/SIGN2.CPT) see mds.i log and `damage ls` output for details 2023-07-21T18:43:16.639 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:16 smithi155 ceph-mon[97934]: Scrub error on inode 0x10000000217 (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/COREL/CDRBARS.CFG) see mds.i log and `damage ls` output for details 2023-07-21T18:43:16.639 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:16 smithi155 ceph-mon[97934]: Scrub error on inode 0x1000000021d (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/COREL/TRAIN.CDR) see mds.i log and `damage ls` output for details 2023-07-21T18:43:16.640 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:16 smithi155 ceph-mon[97934]: Scrub error on inode 0x10000000219 (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC1.CDR) see mds.i log and `damage ls` output for details 2023-07-21T18:43:16.640 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:16 smithi155 ceph-mon[97934]: Scrub error on inode 0x10000000216 (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/COREL/CDRROLS.CFG) see mds.i log and `damage ls` output for details 2023-07-21T18:43:16.640 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:16 smithi155 ceph-mon[97934]: Scrub error on inode 0x1000000020a (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/WORD/BASEMACH.DOC) see mds.i log and `damage ls` output for details 2023-07-21T18:43:16.640 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:16 smithi155 ceph-mon[97934]: Scrub error on inode 0x1000000020b (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/WORD/FACTS.DOC) see mds.i log and `damage ls` output for details 2023-07-21T18:43:16.640 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:16 smithi155 ceph-mon[97934]: Scrub error on inode 0x10000000209 (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/WORD/CHAP10.DOC) see mds.i log and `damage ls` output for details 2023-07-21T18:43:16.640 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:16 smithi155 ceph-mon[97934]: Scrub error on inode 0x1000000020d (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/EXCEL/PCMAGCD.XLS) see mds.i log and `damage ls` output for details 2023-07-21T18:43:16.640 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:16 smithi155 ceph-mon[97934]: Scrub error on inode 0x1000000020e (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/EXCEL/SALES.XLS) see mds.i log and `damage ls` output for details 2023-07-21T18:43:16.640 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:16 smithi155 ceph-mon[97934]: scrub summary: idle+waiting paths [/] 2023-07-21T18:43:16.641 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:16 smithi155 ceph-mon[97934]: scrub summary: active paths [/] 2023-07-21T18:43:16.641 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:16 smithi155 ceph-mon[97934]: Scrub error on inode 0x10000000226 (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/PM/T1.XLS) see mds.i log and `damage ls` output for details 2023-07-21T18:43:16.641 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:16 smithi155 ceph-mon[97934]: Scrub error on inode 0x10000000228 (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/PM/MOVED.DOC) see mds.i log and `damage ls` output for details 2023-07-21T18:43:16.641 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:16 smithi155 ceph-mon[97934]: Scrub error on inode 0x10000000227 (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/PM/EVENTS.DOC) see mds.i log and `damage ls` output for details 2023-07-21T18:43:16.641 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:16 smithi155 ceph-mon[97934]: scrub summary: idle+waiting paths [/] 2023-07-21T18:43:16.641 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:16 smithi155 ceph-mon[97934]: scrub summary: active paths [/] 2023-07-21T18:43:16.641 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:16 smithi155 ceph-mon[97934]: Scrub error on inode 0x10000000210 (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/PWRPNT/PCBENCHM.PPT) see mds.h log and `damage ls` output for details 2023-07-21T18:43:16.641 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:16 smithi155 ceph-mon[97934]: Scrub error on inode 0x10000000211 (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/PWRPNT/ZD16.BMP) see mds.h log and `damage ls` output for details 2023-07-21T18:43:16.642 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:16 smithi155 ceph-mon[97934]: Scrub error on inode 0x10000000214 (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/PWRPNT/TRIDOTS.POT) see mds.h log and `damage ls` output for details 2023-07-21T18:43:16.642 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:16 smithi155 ceph-mon[97934]: Scrub error on inode 0x10000000213 (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/PWRPNT/TIPS.PPT) see mds.h log and `damage ls` output for details 2023-07-21T18:43:16.642 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:16 smithi155 ceph-mon[97934]: Scrub error on inode 0x10000000212 (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/PWRPNT/PPTOOLS1.PPA) see mds.h log and `damage ls` output for details 2023-07-21T18:43:16.642 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:16 smithi155 ceph-mon[97934]: scrub summary: idle+waiting paths [/] 2023-07-21T18:43:16.642 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:16 smithi155 ceph-mon[97934]: scrub summary: idle+waiting paths [/] 2023-07-21T18:43:16.642 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:16 smithi155 ceph-mon[97934]: mgrmap e24: x(active, since 90s), standbys: y, z 2023-07-21T18:43:16.685 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:16 smithi039 ceph-mon[96489]: pgmap v48: 97 pgs: 97 active+clean; 12 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 611 KiB/s wr, 142 op/s 2023-07-21T18:43:16.686 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:16 smithi039 ceph-mon[96489]: scrub queued for path: / 2023-07-21T18:43:16.686 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:16 smithi039 ceph-mon[96489]: scrub summary: idle+waiting paths [/] 2023-07-21T18:43:16.686 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:16 smithi039 ceph-mon[96489]: scrub summary: active paths [/] 2023-07-21T18:43:16.686 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:16 smithi039 ceph-mon[96489]: scrub summary: active paths [/] 2023-07-21T18:43:16.686 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:16 smithi039 ceph-mon[96489]: scrub summary: active paths [/] 2023-07-21T18:43:16.687 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:16 smithi039 ceph-mon[96489]: Scrub error on inode 0x10000000241 (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/SEED/SMALL.FIL) see mds.h log and `damage ls` output for details 2023-07-21T18:43:16.687 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:16 smithi039 ceph-mon[96489]: Scrub error on inode 0x10000000243 (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/SEED/LARGE.FIL) see mds.h log and `damage ls` output for details 2023-07-21T18:43:16.687 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:16 smithi039 ceph-mon[96489]: Scrub error on inode 0x1000000021f (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/COREL/ROSE.CPT) see mds.i log and `damage ls` output for details 2023-07-21T18:43:16.687 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:16 smithi039 ceph-mon[96489]: Scrub error on inode 0x10000000242 (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/SEED/MEDIUM.FIL) see mds.h log and `damage ls` output for details 2023-07-21T18:43:16.687 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:16 smithi039 ceph-mon[96489]: Scrub error on inode 0x10000000222 (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/WORDPRO/BENCHS.LWP) see mds.h log and `damage ls` output for details 2023-07-21T18:43:16.687 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:16 smithi039 ceph-mon[96489]: Scrub error on inode 0x10000000223 (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/WORDPRO/RESULTS.XLS) see mds.h log and `damage ls` output for details 2023-07-21T18:43:16.687 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:16 smithi039 ceph-mon[96489]: Scrub error on inode 0x10000000224 (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/WORDPRO/NEWS1_1.LWP) see mds.h log and `damage ls` output for details 2023-07-21T18:43:16.687 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:16 smithi039 ceph-mon[96489]: scrub summary: idle+waiting paths [/] 2023-07-21T18:43:16.688 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:16 smithi039 ceph-mon[96489]: Scrub error on inode 0x1000000021b (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/COREL/TEAPOT2.CPT) see mds.i log and `damage ls` output for details 2023-07-21T18:43:16.688 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:16 smithi039 ceph-mon[96489]: Scrub error on inode 0x10000000220 (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/COREL/CORELDRW.CDT) see mds.i log and `damage ls` output for details 2023-07-21T18:43:16.688 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:16 smithi039 ceph-mon[96489]: Scrub error on inode 0x1000000021a (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/COREL/MAILBOX2.CPT) see mds.i log and `damage ls` output for details 2023-07-21T18:43:16.688 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:16 smithi039 ceph-mon[96489]: Scrub error on inode 0x1000000021e (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC2.CDR) see mds.i log and `damage ls` output for details 2023-07-21T18:43:16.688 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:16 smithi039 ceph-mon[96489]: Scrub error on inode 0x10000000218 (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/COREL/ARTISTIC.ACL) see mds.i log and `damage ls` output for details 2023-07-21T18:43:16.688 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:16 smithi039 ceph-mon[96489]: Scrub error on inode 0x1000000021c (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/COREL/SIGN2.CPT) see mds.i log and `damage ls` output for details 2023-07-21T18:43:16.688 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:16 smithi039 ceph-mon[96489]: Scrub error on inode 0x10000000217 (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/COREL/CDRBARS.CFG) see mds.i log and `damage ls` output for details 2023-07-21T18:43:16.689 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:16 smithi039 ceph-mon[96489]: Scrub error on inode 0x1000000021d (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/COREL/TRAIN.CDR) see mds.i log and `damage ls` output for details 2023-07-21T18:43:16.689 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:16 smithi039 ceph-mon[96489]: Scrub error on inode 0x10000000219 (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC1.CDR) see mds.i log and `damage ls` output for details 2023-07-21T18:43:16.689 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:16 smithi039 ceph-mon[96489]: Scrub error on inode 0x10000000216 (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/COREL/CDRROLS.CFG) see mds.i log and `damage ls` output for details 2023-07-21T18:43:16.689 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:16 smithi039 ceph-mon[96489]: Scrub error on inode 0x1000000020a (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/WORD/BASEMACH.DOC) see mds.i log and `damage ls` output for details 2023-07-21T18:43:16.689 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:16 smithi039 ceph-mon[96489]: Scrub error on inode 0x1000000020b (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/WORD/FACTS.DOC) see mds.i log and `damage ls` output for details 2023-07-21T18:43:16.689 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:16 smithi039 ceph-mon[96489]: Scrub error on inode 0x10000000209 (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/WORD/CHAP10.DOC) see mds.i log and `damage ls` output for details 2023-07-21T18:43:16.689 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:16 smithi039 ceph-mon[96489]: Scrub error on inode 0x1000000020d (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/EXCEL/PCMAGCD.XLS) see mds.i log and `damage ls` output for details 2023-07-21T18:43:16.689 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:16 smithi039 ceph-mon[96489]: Scrub error on inode 0x1000000020e (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/EXCEL/SALES.XLS) see mds.i log and `damage ls` output for details 2023-07-21T18:43:16.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:16 smithi039 ceph-mon[96489]: scrub summary: idle+waiting paths [/] 2023-07-21T18:43:16.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:16 smithi039 ceph-mon[96489]: scrub summary: active paths [/] 2023-07-21T18:43:16.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:16 smithi039 ceph-mon[96489]: Scrub error on inode 0x10000000226 (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/PM/T1.XLS) see mds.i log and `damage ls` output for details 2023-07-21T18:43:16.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:16 smithi039 ceph-mon[96489]: Scrub error on inode 0x10000000228 (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/PM/MOVED.DOC) see mds.i log and `damage ls` output for details 2023-07-21T18:43:16.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:16 smithi039 ceph-mon[96489]: Scrub error on inode 0x10000000227 (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/PM/EVENTS.DOC) see mds.i log and `damage ls` output for details 2023-07-21T18:43:16.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:16 smithi039 ceph-mon[96489]: scrub summary: idle+waiting paths [/] 2023-07-21T18:43:16.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:16 smithi039 ceph-mon[96489]: scrub summary: active paths [/] 2023-07-21T18:43:16.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:16 smithi039 ceph-mon[96489]: Scrub error on inode 0x10000000210 (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/PWRPNT/PCBENCHM.PPT) see mds.h log and `damage ls` output for details 2023-07-21T18:43:16.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:16 smithi039 ceph-mon[96489]: Scrub error on inode 0x10000000211 (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/PWRPNT/ZD16.BMP) see mds.h log and `damage ls` output for details 2023-07-21T18:43:16.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:16 smithi039 ceph-mon[96489]: Scrub error on inode 0x10000000214 (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/PWRPNT/TRIDOTS.POT) see mds.h log and `damage ls` output for details 2023-07-21T18:43:16.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:16 smithi039 ceph-mon[96489]: Scrub error on inode 0x10000000213 (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/PWRPNT/TIPS.PPT) see mds.h log and `damage ls` output for details 2023-07-21T18:43:16.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:16 smithi039 ceph-mon[96489]: Scrub error on inode 0x10000000212 (/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/PWRPNT/PPTOOLS1.PPA) see mds.h log and `damage ls` output for details 2023-07-21T18:43:16.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:16 smithi039 ceph-mon[96489]: scrub summary: idle+waiting paths [/] 2023-07-21T18:43:16.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:16 smithi039 ceph-mon[96489]: scrub summary: idle+waiting paths [/] 2023-07-21T18:43:16.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:16 smithi039 ceph-mon[96489]: mgrmap e24: x(active, since 90s), standbys: y, z 2023-07-21T18:43:17.260 INFO:tasks.workunit.client.0.smithi039.stdout: 1 32631 6.44 MB/sec warmup 30 sec latency 1029.221 ms 2023-07-21T18:43:18.260 INFO:tasks.workunit.client.0.smithi039.stdout: 1 32631 6.23 MB/sec warmup 31 sec latency 2029.278 ms 2023-07-21T18:43:18.617 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:18 smithi193 ceph-mon[98204]: pgmap v49: 97 pgs: 97 active+clean; 15 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 832 KiB/s wr, 189 op/s 2023-07-21T18:43:18.618 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:18 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:43:18.636 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:18 smithi155 ceph-mon[97934]: pgmap v49: 97 pgs: 97 active+clean; 15 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 832 KiB/s wr, 189 op/s 2023-07-21T18:43:18.636 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:18 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:43:18.685 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:18 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mon-a[96465]: 2023-07-21T18:43:18.217+0000 7f094dd1e700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-07-21T18:43:18.685 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:18 smithi039 ceph-mon[96489]: pgmap v49: 97 pgs: 97 active+clean; 15 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 832 KiB/s wr, 189 op/s 2023-07-21T18:43:18.686 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:18 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:43:19.135 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.113+0000 7f7e7b040700 1 -- 172.21.15.39:0/3022422572 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e74104a30 msgr2=0x7f7e74104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:19.135 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.114+0000 7f7e7b040700 1 --2- 172.21.15.39:0/3022422572 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e74104a30 0x7f7e74104e00 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f7e68009990 tx=0x7f7e68009ca0 comp rx=0 tx=0).stop 2023-07-21T18:43:19.136 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.114+0000 7f7e7b040700 1 -- 172.21.15.39:0/3022422572 shutdown_connections 2023-07-21T18:43:19.136 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.114+0000 7f7e7b040700 1 --2- 172.21.15.39:0/3022422572 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7e740ff3e0 0x7f7e7410b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:19.136 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.114+0000 7f7e7b040700 1 --2- 172.21.15.39:0/3022422572 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e74104a30 0x7f7e74104e00 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:19.136 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.114+0000 7f7e7b040700 1 --2- 172.21.15.39:0/3022422572 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e740fea30 0x7f7e740feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:19.136 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.114+0000 7f7e7b040700 1 -- 172.21.15.39:0/3022422572 >> 172.21.15.39:0/3022422572 conn(0x7f7e74074760 msgr2=0x7f7e74074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:19.136 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.114+0000 7f7e7b040700 1 -- 172.21.15.39:0/3022422572 shutdown_connections 2023-07-21T18:43:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.114+0000 7f7e7b040700 1 -- 172.21.15.39:0/3022422572 wait complete. 2023-07-21T18:43:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.114+0000 7f7e7b040700 1 Processor -- start 2023-07-21T18:43:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.114+0000 7f7e7b040700 1 -- start start 2023-07-21T18:43:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.114+0000 7f7e7b040700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e740fea30 0x7f7e74105ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.115+0000 7f7e7b040700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e740ff3e0 0x7f7e74106020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.115+0000 7f7e7b040700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7e74104a30 0x7f7e741a3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.115+0000 7f7e7b040700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e741a4310 con 0x7f7e740ff3e0 2023-07-21T18:43:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.115+0000 7f7e7b040700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e741a4450 con 0x7f7e740fea30 2023-07-21T18:43:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.115+0000 7f7e7b040700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e741a45c0 con 0x7f7e74104a30 2023-07-21T18:43:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.115+0000 7f7e73fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e740ff3e0 0x7f7e74106020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.115+0000 7f7e73fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e740ff3e0 0x7f7e74106020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58808/0 (socket says 172.21.15.39:58808) 2023-07-21T18:43:19.140 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.115+0000 7f7e73fff700 1 -- 172.21.15.39:0/4276759168 learned_addr learned my addr 172.21.15.39:0/4276759168 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:19.140 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.115+0000 7f7e78ddc700 1 --2- 172.21.15.39:0/4276759168 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e740fea30 0x7f7e74105ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:19.140 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.115+0000 7f7e795dd700 1 --2- 172.21.15.39:0/4276759168 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7e74104a30 0x7f7e741a3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:19.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.115+0000 7f7e73fff700 1 -- 172.21.15.39:0/4276759168 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7e74104a30 msgr2=0x7f7e741a3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:19.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.115+0000 7f7e73fff700 1 --2- 172.21.15.39:0/4276759168 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7e74104a30 0x7f7e741a3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:19.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.115+0000 7f7e73fff700 1 -- 172.21.15.39:0/4276759168 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e740fea30 msgr2=0x7f7e74105ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:19.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.115+0000 7f7e73fff700 1 --2- 172.21.15.39:0/4276759168 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e740fea30 0x7f7e74105ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:19.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.115+0000 7f7e73fff700 1 -- 172.21.15.39:0/4276759168 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e680095f0 con 0x7f7e740ff3e0 2023-07-21T18:43:19.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.115+0000 7f7e78ddc700 1 --2- 172.21.15.39:0/4276759168 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e740fea30 0x7f7e74105ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:43:19.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.115+0000 7f7e73fff700 1 --2- 172.21.15.39:0/4276759168 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e740ff3e0 0x7f7e74106020 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f7e6000e9a0 tx=0x7f7e6000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:19.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.115+0000 7f7e71ffb700 1 -- 172.21.15.39:0/4276759168 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e60009790 con 0x7f7e740ff3e0 2023-07-21T18:43:19.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.115+0000 7f7e71ffb700 1 -- 172.21.15.39:0/4276759168 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e60010430 con 0x7f7e740ff3e0 2023-07-21T18:43:19.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.115+0000 7f7e7b040700 1 -- 172.21.15.39:0/4276759168 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e741a48a0 con 0x7f7e740ff3e0 2023-07-21T18:43:19.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.115+0000 7f7e71ffb700 1 -- 172.21.15.39:0/4276759168 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e60018680 con 0x7f7e740ff3e0 2023-07-21T18:43:19.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.116+0000 7f7e7b040700 1 -- 172.21.15.39:0/4276759168 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e741a4db0 con 0x7f7e740ff3e0 2023-07-21T18:43:19.151 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.118+0000 7f7e71ffb700 1 -- 172.21.15.39:0/4276759168 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f7e600105a0 con 0x7f7e740ff3e0 2023-07-21T18:43:19.151 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.119+0000 7f7e71ffb700 1 --2- 172.21.15.39:0/4276759168 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f7e640a12c0 0x7f7e640a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:19.151 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.119+0000 7f7e71ffb700 1 -- 172.21.15.39:0/4276759168 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f7e60014070 con 0x7f7e740ff3e0 2023-07-21T18:43:19.151 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.119+0000 7f7e78ddc700 1 --2- 172.21.15.39:0/4276759168 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f7e640a12c0 0x7f7e640a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:19.151 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.120+0000 7f7e78ddc700 1 --2- 172.21.15.39:0/4276759168 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f7e640a12c0 0x7f7e640a3770 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7e6800b2e0 tx=0x7f7e68005fd0 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:19.151 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.120+0000 7f7e871c6b80 1 Processor -- start 2023-07-21T18:43:19.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.120+0000 7f7e871c6b80 1 -- start start 2023-07-21T18:43:19.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.120+0000 7f7e871c6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf7f26c6b0 0x55cf7f415d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:19.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.120+0000 7f7e871c6b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf7f4162d0 0x55cf7f410040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:19.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.120+0000 7f7e871c6b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf7f410580 0x55cf7f4109d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:19.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.120+0000 7f7e871c6b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf7f315c50 con 0x55cf7f410580 2023-07-21T18:43:19.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.120+0000 7f7e871c6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf7f3689e0 con 0x55cf7f26c6b0 2023-07-21T18:43:19.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.120+0000 7f7e871c6b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf7f361e70 con 0x55cf7f4162d0 2023-07-21T18:43:19.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.120+0000 7f7e795dd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf7f410580 0x55cf7f4109d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:19.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.120+0000 7f7e795dd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf7f410580 0x55cf7f4109d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58818/0 (socket says 172.21.15.39:58818) 2023-07-21T18:43:19.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.120+0000 7f7e795dd700 1 -- 172.21.15.39:0/2482632966 learned_addr learned my addr 172.21.15.39:0/2482632966 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:19.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.120+0000 7f7e73fff700 1 --2- 172.21.15.39:0/2482632966 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf7f4162d0 0x55cf7f410040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:19.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.120+0000 7f7e78ddc700 1 --2- 172.21.15.39:0/2482632966 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf7f26c6b0 0x55cf7f415d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:19.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.120+0000 7f7e795dd700 1 -- 172.21.15.39:0/2482632966 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf7f4162d0 msgr2=0x55cf7f410040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:19.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.121+0000 7f7e795dd700 1 --2- 172.21.15.39:0/2482632966 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf7f4162d0 0x55cf7f410040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:19.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.121+0000 7f7e795dd700 1 -- 172.21.15.39:0/2482632966 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf7f26c6b0 msgr2=0x55cf7f415d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.121+0000 7f7e795dd700 1 --2- 172.21.15.39:0/2482632966 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf7f26c6b0 0x55cf7f415d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.121+0000 7f7e795dd700 1 -- 172.21.15.39:0/2482632966 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cf7f18fd80 con 0x55cf7f410580 2023-07-21T18:43:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.121+0000 7f7e78ddc700 1 --2- 172.21.15.39:0/2482632966 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf7f26c6b0 0x55cf7f415d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:43:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.121+0000 7f7e795dd700 1 --2- 172.21.15.39:0/2482632966 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf7f410580 0x55cf7f4109d0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c003fa0 tx=0x7f7e6c009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c4fba50db65a9e2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.121+0000 7f7e5effd700 1 -- 172.21.15.39:0/2482632966 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c019070 con 0x55cf7f410580 2023-07-21T18:43:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.121+0000 7f7e5effd700 1 -- 172.21.15.39:0/2482632966 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e6c00f040 con 0x55cf7f410580 2023-07-21T18:43:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.121+0000 7f7e5effd700 1 -- 172.21.15.39:0/2482632966 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c0079a0 con 0x55cf7f410580 2023-07-21T18:43:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.122+0000 7f7e871c6b80 1 -- 172.21.15.39:0/2482632966 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf7f410580 msgr2=0x55cf7f4109d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.122+0000 7f7e871c6b80 1 --2- 172.21.15.39:0/2482632966 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf7f410580 0x55cf7f4109d0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c003fa0 tx=0x7f7e6c009ba0 comp rx=0 tx=0).stop 2023-07-21T18:43:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.122+0000 7f7e871c6b80 1 -- 172.21.15.39:0/2482632966 shutdown_connections 2023-07-21T18:43:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.122+0000 7f7e871c6b80 1 --2- 172.21.15.39:0/2482632966 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf7f26c6b0 0x55cf7f415d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.122+0000 7f7e871c6b80 1 --2- 172.21.15.39:0/2482632966 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf7f410580 0x55cf7f4109d0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.122+0000 7f7e871c6b80 1 --2- 172.21.15.39:0/2482632966 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf7f4162d0 0x55cf7f410040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.122+0000 7f7e871c6b80 1 -- 172.21.15.39:0/2482632966 >> 172.21.15.39:0/2482632966 conn(0x55cf7f405be0 msgr2=0x55cf7f40af10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.122+0000 7f7e871c6b80 1 -- 172.21.15.39:0/2482632966 shutdown_connections 2023-07-21T18:43:19.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.122+0000 7f7e871c6b80 1 -- 172.21.15.39:0/2482632966 wait complete. 2023-07-21T18:43:19.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.122+0000 7f7e871c6b80 1 -- start start 2023-07-21T18:43:19.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.122+0000 7f7e871c6b80 1 Processor -- start 2023-07-21T18:43:19.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.123+0000 7f7e871c6b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf7f26c6b0 0x55cf7f41c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:19.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.123+0000 7f7e871c6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf7f410580 0x55cf7f4197d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:19.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.123+0000 7f7e871c6b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf7f4162d0 0x55cf7f419e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:19.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.123+0000 7f7e871c6b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf7f301a60 con 0x55cf7f26c6b0 2023-07-21T18:43:19.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.123+0000 7f7e871c6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf7f301760 con 0x55cf7f410580 2023-07-21T18:43:19.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.123+0000 7f7e871c6b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf7f1e58c0 con 0x55cf7f4162d0 2023-07-21T18:43:19.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.123+0000 7f7e871c6b80 10 client.0 _ll_register_callbacks cb 0x55cf7f2ff700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:43:19.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.123+0000 7f7e78ddc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf7f26c6b0 0x55cf7f41c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:19.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.123+0000 7f7e78ddc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf7f26c6b0 0x55cf7f41c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58820/0 (socket says 172.21.15.39:58820) 2023-07-21T18:43:19.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.123+0000 7f7e78ddc700 1 -- 172.21.15.39:0/1262648373 learned_addr learned my addr 172.21.15.39:0/1262648373 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:19.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.123+0000 7f7e795dd700 1 --2- 172.21.15.39:0/1262648373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf7f4162d0 0x55cf7f419e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:19.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.123+0000 7f7e73fff700 1 --2- 172.21.15.39:0/1262648373 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf7f410580 0x55cf7f4197d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:19.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.123+0000 7f7e78ddc700 1 -- 172.21.15.39:0/1262648373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf7f4162d0 msgr2=0x55cf7f419e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:19.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.123+0000 7f7e78ddc700 1 --2- 172.21.15.39:0/1262648373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf7f4162d0 0x55cf7f419e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:19.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.123+0000 7f7e78ddc700 1 -- 172.21.15.39:0/1262648373 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf7f410580 msgr2=0x55cf7f4197d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:19.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.123+0000 7f7e78ddc700 1 --2- 172.21.15.39:0/1262648373 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf7f410580 0x55cf7f4197d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:19.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.123+0000 7f7e78ddc700 1 -- 172.21.15.39:0/1262648373 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e6c003bd0 con 0x55cf7f26c6b0 2023-07-21T18:43:19.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.123+0000 7f7e795dd700 1 --2- 172.21.15.39:0/1262648373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf7f4162d0 0x55cf7f419e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:43:19.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.123+0000 7f7e78ddc700 1 --2- 172.21.15.39:0/1262648373 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf7f26c6b0 0x55cf7f41c550 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f7e68015000 tx=0x7f7e68009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48878d82549bcbd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:19.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.123+0000 7f7e53fff700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:43:19.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.124+0000 7f7e53fff700 1 -- 172.21.15.39:0/1262648373 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e68024cb0 con 0x55cf7f26c6b0 2023-07-21T18:43:19.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.124+0000 7f7e53fff700 1 -- 172.21.15.39:0/1262648373 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f7e680056b0 con 0x55cf7f26c6b0 2023-07-21T18:43:19.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.124+0000 7f7e871c6b80 1 -- 172.21.15.39:0/1262648373 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cf7f2bb490 con 0x55cf7f26c6b0 2023-07-21T18:43:19.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.124+0000 7f7e53fff700 1 -- 172.21.15.39:0/1262648373 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e68024e10 con 0x55cf7f26c6b0 2023-07-21T18:43:19.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.124+0000 7f7e871c6b80 10 client.16107 fetch_fsmap learned FSMap version 30 2023-07-21T18:43:19.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.124+0000 7f7e871c6b80 1 -- 172.21.15.39:0/1262648373 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55cf7f178c00 con 0x55cf7f26c6b0 2023-07-21T18:43:19.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.124+0000 7f7e53fff700 1 -- 172.21.15.39:0/1262648373 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e6803d020 con 0x55cf7f26c6b0 2023-07-21T18:43:19.160 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.124+0000 7f7e53fff700 1 -- 172.21.15.39:0/1262648373 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f7e68017510 con 0x55cf7f26c6b0 2023-07-21T18:43:19.160 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.124+0000 7f7e871c6b80 10 client.16107 fetch_fsmap finished waiting for FSMap version 30 2023-07-21T18:43:19.160 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.124+0000 7f7e871c6b80 10 client.16107 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:43:19.160 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.124+0000 7f7e871c6b80 20 client.16107 populate_metadata read hostname 'smithi039' 2023-07-21T18:43:19.160 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.124+0000 7f7e871c6b80 1 --2- 172.21.15.39:0/1262648373 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55cf7f429bc0 0x55cf7f42bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:19.160 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.124+0000 7f7e871c6b80 4 client.16107 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:43:19.160 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.124+0000 7f7e871c6b80 1 -- 172.21.15.39:0/1262648373 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cf7f35ae50 con 0x55cf7f429bc0 2023-07-21T18:43:19.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.125+0000 7f7e73fff700 1 --2- 172.21.15.39:0/1262648373 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55cf7f429bc0 0x55cf7f42bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:19.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.125+0000 7f7e73fff700 1 --2- 172.21.15.39:0/1262648373 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55cf7f429bc0 0x55cf7f42bfd0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f07362548cc9179 server_cookie=34db5b0d8700f3eb in_seq=0 out_seq=0 2023-07-21T18:43:19.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.125+0000 7f7e53fff700 10 client.16107 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:43:19.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.126+0000 7f7e53fff700 1 -- 172.21.15.39:0/1262648373 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55cf7f35ae50 con 0x55cf7f429bc0 2023-07-21T18:43:19.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.126+0000 7f7e53fff700 10 client.16107 handle_command_reply: tid=0 2023-07-21T18:43:19.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.126+0000 7f7e871c6b80 1 client.16107 shutdown 2023-07-21T18:43:19.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.126+0000 7f7e871c6b80 1 -- 172.21.15.39:0/1262648373 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf7f26c6b0 msgr2=0x55cf7f41c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:19.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.126+0000 7f7e871c6b80 1 --2- 172.21.15.39:0/1262648373 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf7f26c6b0 0x55cf7f41c550 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f7e68015000 tx=0x7f7e68009ca0 comp rx=0 tx=0).stop 2023-07-21T18:43:19.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.126+0000 7f7e871c6b80 1 -- 172.21.15.39:0/1262648373 shutdown_connections 2023-07-21T18:43:19.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.126+0000 7f7e871c6b80 1 --2- 172.21.15.39:0/1262648373 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55cf7f429bc0 0x55cf7f42bfd0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:19.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.126+0000 7f7e871c6b80 1 --2- 172.21.15.39:0/1262648373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf7f4162d0 0x55cf7f419e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:19.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.126+0000 7f7e871c6b80 1 --2- 172.21.15.39:0/1262648373 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf7f26c6b0 0x55cf7f41c550 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:19.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.126+0000 7f7e53fff700 0 client.16107 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:43:19.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.126+0000 7f7e871c6b80 1 --2- 172.21.15.39:0/1262648373 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf7f410580 0x55cf7f4197d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:19.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.126+0000 7f7e871c6b80 1 -- 172.21.15.39:0/1262648373 >> 172.21.15.39:0/1262648373 conn(0x55cf7f405be0 msgr2=0x55cf7f409c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:19.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.126+0000 7f7e871c6b80 1 -- 172.21.15.39:0/1262648373 shutdown_connections 2023-07-21T18:43:19.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.126+0000 7f7e871c6b80 1 -- 172.21.15.39:0/1262648373 wait complete. 2023-07-21T18:43:19.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.126+0000 7f7e871c6b80 20 client.16107 trim_cache size 0 max 16384 2023-07-21T18:43:19.166 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.141+0000 7f7e871c6b80 1 Processor -- start 2023-07-21T18:43:19.166 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.141+0000 7f7e871c6b80 1 -- start start 2023-07-21T18:43:19.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.141+0000 7f7e871c6b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf7f26c6b0 0x55cf7f420c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:19.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.141+0000 7f7e871c6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf7f410580 0x55cf7f40ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:19.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.141+0000 7f7e871c6b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf7f4162d0 0x55cf7f417010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:19.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.141+0000 7f7e871c6b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf7f410fd0 con 0x55cf7f26c6b0 2023-07-21T18:43:19.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.141+0000 7f7e871c6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf7f4159c0 con 0x55cf7f410580 2023-07-21T18:43:19.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.141+0000 7f7e871c6b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf7f417d30 con 0x55cf7f4162d0 2023-07-21T18:43:19.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.141+0000 7f7e78ddc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf7f26c6b0 0x55cf7f420c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.141+0000 7f7e78ddc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf7f26c6b0 0x55cf7f420c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58828/0 (socket says 172.21.15.39:58828) 2023-07-21T18:43:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.141+0000 7f7e78ddc700 1 -- 172.21.15.39:0/2407812320 learned_addr learned my addr 172.21.15.39:0/2407812320 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.141+0000 7f7e795dd700 1 --2- 172.21.15.39:0/2407812320 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf7f4162d0 0x55cf7f417010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.141+0000 7f7e73fff700 1 --2- 172.21.15.39:0/2407812320 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf7f410580 0x55cf7f40ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.141+0000 7f7e78ddc700 1 -- 172.21.15.39:0/2407812320 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf7f4162d0 msgr2=0x55cf7f417010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.141+0000 7f7e78ddc700 1 --2- 172.21.15.39:0/2407812320 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf7f4162d0 0x55cf7f417010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.141+0000 7f7e78ddc700 1 -- 172.21.15.39:0/2407812320 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf7f410580 msgr2=0x55cf7f40ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.141+0000 7f7e78ddc700 1 --2- 172.21.15.39:0/2407812320 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf7f410580 0x55cf7f40ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.141+0000 7f7e78ddc700 1 -- 172.21.15.39:0/2407812320 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cf7f1b61f0 con 0x55cf7f26c6b0 2023-07-21T18:43:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.142+0000 7f7e78ddc700 1 --2- 172.21.15.39:0/2407812320 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf7f26c6b0 0x55cf7f420c60 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f7e680098e0 tx=0x7f7e68005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d15e6a72d5c7057e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.142+0000 7f7e5f7fe700 1 -- 172.21.15.39:0/2407812320 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e680056b0 con 0x55cf7f26c6b0 2023-07-21T18:43:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.142+0000 7f7e5f7fe700 1 -- 172.21.15.39:0/2407812320 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e6803d030 con 0x55cf7f26c6b0 2023-07-21T18:43:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.142+0000 7f7e5f7fe700 1 -- 172.21.15.39:0/2407812320 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e68030650 con 0x55cf7f26c6b0 2023-07-21T18:43:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.142+0000 7f7e871c6b80 1 -- 172.21.15.39:0/2407812320 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf7f26c6b0 msgr2=0x55cf7f420c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.142+0000 7f7e871c6b80 1 --2- 172.21.15.39:0/2407812320 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf7f26c6b0 0x55cf7f420c60 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f7e680098e0 tx=0x7f7e68005e80 comp rx=0 tx=0).stop 2023-07-21T18:43:19.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.142+0000 7f7e871c6b80 1 -- 172.21.15.39:0/2407812320 shutdown_connections 2023-07-21T18:43:19.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.142+0000 7f7e871c6b80 1 --2- 172.21.15.39:0/2407812320 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf7f4162d0 0x55cf7f417010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:19.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.142+0000 7f7e871c6b80 1 --2- 172.21.15.39:0/2407812320 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf7f26c6b0 0x55cf7f420c60 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:19.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.142+0000 7f7e871c6b80 1 --2- 172.21.15.39:0/2407812320 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf7f410580 0x55cf7f40ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:19.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.142+0000 7f7e871c6b80 1 -- 172.21.15.39:0/2407812320 >> 172.21.15.39:0/2407812320 conn(0x55cf7f405be0 msgr2=0x55cf7f416900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:19.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.142+0000 7f7e871c6b80 1 -- 172.21.15.39:0/2407812320 shutdown_connections 2023-07-21T18:43:19.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.142+0000 7f7e871c6b80 1 -- 172.21.15.39:0/2407812320 wait complete. 2023-07-21T18:43:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.143+0000 7f7e871c6b80 1 -- start start 2023-07-21T18:43:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.143+0000 7f7e871c6b80 1 Processor -- start 2023-07-21T18:43:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.143+0000 7f7e871c6b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf7f26c6b0 0x55cf7f4128f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.143+0000 7f7e871c6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf7f410580 0x55cf7f412e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.143+0000 7f7e871c6b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf7f4162d0 0x55cf7f411790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.143+0000 7f7e871c6b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf7f413400 con 0x55cf7f26c6b0 2023-07-21T18:43:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.143+0000 7f7e871c6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf7f413570 con 0x55cf7f410580 2023-07-21T18:43:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.143+0000 7f7e871c6b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf7f411cd0 con 0x55cf7f4162d0 2023-07-21T18:43:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.143+0000 7f7e871c6b80 10 client.0 _ll_register_callbacks cb 0x7f7e6800f770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:43:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.143+0000 7f7e78ddc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf7f26c6b0 0x55cf7f4128f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.143+0000 7f7e78ddc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf7f26c6b0 0x55cf7f4128f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58834/0 (socket says 172.21.15.39:58834) 2023-07-21T18:43:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.143+0000 7f7e78ddc700 1 -- 172.21.15.39:0/3488943286 learned_addr learned my addr 172.21.15.39:0/3488943286 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.144+0000 7f7e795dd700 1 --2- 172.21.15.39:0/3488943286 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf7f4162d0 0x55cf7f411790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.144+0000 7f7e73fff700 1 --2- 172.21.15.39:0/3488943286 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf7f410580 0x55cf7f412e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.144+0000 7f7e78ddc700 1 -- 172.21.15.39:0/3488943286 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf7f4162d0 msgr2=0x55cf7f411790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:19.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.144+0000 7f7e78ddc700 1 --2- 172.21.15.39:0/3488943286 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf7f4162d0 0x55cf7f411790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:19.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.144+0000 7f7e78ddc700 1 -- 172.21.15.39:0/3488943286 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf7f410580 msgr2=0x55cf7f412e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:19.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.144+0000 7f7e78ddc700 1 --2- 172.21.15.39:0/3488943286 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf7f410580 0x55cf7f412e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:19.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.144+0000 7f7e78ddc700 1 -- 172.21.15.39:0/3488943286 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cf7f178c00 con 0x55cf7f26c6b0 2023-07-21T18:43:19.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.144+0000 7f7e795dd700 1 --2- 172.21.15.39:0/3488943286 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf7f4162d0 0x55cf7f411790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:43:19.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.144+0000 7f7e73fff700 1 --2- 172.21.15.39:0/3488943286 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf7f410580 0x55cf7f412e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:43:19.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.144+0000 7f7e78ddc700 1 --2- 172.21.15.39:0/3488943286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf7f26c6b0 0x55cf7f4128f0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f7e68010010 tx=0x7f7e68017df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5153b53557ee5db server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:19.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.144+0000 7f7e53fff700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:43:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.144+0000 7f7e53fff700 1 -- 172.21.15.39:0/3488943286 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e68042840 con 0x55cf7f26c6b0 2023-07-21T18:43:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.144+0000 7f7e53fff700 1 -- 172.21.15.39:0/3488943286 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f7e6803e080 con 0x55cf7f26c6b0 2023-07-21T18:43:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.144+0000 7f7e871c6b80 1 -- 172.21.15.39:0/3488943286 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7e6c00f040 con 0x55cf7f26c6b0 2023-07-21T18:43:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.144+0000 7f7e53fff700 1 -- 172.21.15.39:0/3488943286 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e680056b0 con 0x55cf7f26c6b0 2023-07-21T18:43:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.144+0000 7f7e871c6b80 10 client.16113 fetch_fsmap learned FSMap version 30 2023-07-21T18:43:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.144+0000 7f7e871c6b80 1 -- 172.21.15.39:0/3488943286 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55cf7f18e250 con 0x55cf7f26c6b0 2023-07-21T18:43:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.144+0000 7f7e53fff700 1 -- 172.21.15.39:0/3488943286 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e6c00f040 con 0x55cf7f26c6b0 2023-07-21T18:43:19.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.144+0000 7f7e53fff700 1 -- 172.21.15.39:0/3488943286 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f7e680548b0 con 0x55cf7f26c6b0 2023-07-21T18:43:19.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.145+0000 7f7e871c6b80 10 client.16113 fetch_fsmap finished waiting for FSMap version 30 2023-07-21T18:43:19.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.145+0000 7f7e871c6b80 10 client.16113 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:43:19.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.145+0000 7f7e871c6b80 20 client.16113 populate_metadata read hostname 'smithi039' 2023-07-21T18:43:19.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.145+0000 7f7e871c6b80 1 --2- 172.21.15.39:0/3488943286 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55cf7f47e260 0x55cf7f480670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:19.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.145+0000 7f7e871c6b80 4 client.16113 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:43:19.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.145+0000 7f7e871c6b80 1 -- 172.21.15.39:0/3488943286 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55cf7f31cff0 con 0x55cf7f47e260 2023-07-21T18:43:19.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.145+0000 7f7e73fff700 1 --2- 172.21.15.39:0/3488943286 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55cf7f47e260 0x55cf7f480670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:19.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.145+0000 7f7e53fff700 10 client.16113 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:43:19.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.145+0000 7f7e73fff700 1 --2- 172.21.15.39:0/3488943286 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55cf7f47e260 0x55cf7f480670 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=822f0d013543965e server_cookie=1e6bca9173d1636a in_seq=0 out_seq=0 2023-07-21T18:43:19.202 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.182+0000 7f7e53fff700 1 -- 172.21.15.39:0/3488943286 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55cf7f31cff0 con 0x55cf7f47e260 2023-07-21T18:43:19.202 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.182+0000 7f7e53fff700 10 client.16113 handle_command_reply: tid=0 2023-07-21T18:43:19.202 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.182+0000 7f7e871c6b80 1 client.16113 shutdown 2023-07-21T18:43:19.202 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.182+0000 7f7e871c6b80 1 -- 172.21.15.39:0/3488943286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf7f26c6b0 msgr2=0x55cf7f4128f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:19.202 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.182+0000 7f7e871c6b80 1 --2- 172.21.15.39:0/3488943286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf7f26c6b0 0x55cf7f4128f0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f7e68010010 tx=0x7f7e68017df0 comp rx=0 tx=0).stop 2023-07-21T18:43:19.202 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.182+0000 7f7e871c6b80 1 -- 172.21.15.39:0/3488943286 shutdown_connections 2023-07-21T18:43:19.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.182+0000 7f7e871c6b80 1 --2- 172.21.15.39:0/3488943286 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55cf7f47e260 0x55cf7f480670 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:19.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.182+0000 7f7e871c6b80 1 --2- 172.21.15.39:0/3488943286 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf7f4162d0 0x55cf7f411790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:19.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.182+0000 7f7e53fff700 0 client.16113 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:43:19.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.182+0000 7f7e871c6b80 1 --2- 172.21.15.39:0/3488943286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf7f26c6b0 0x55cf7f4128f0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:19.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.182+0000 7f7e871c6b80 1 --2- 172.21.15.39:0/3488943286 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf7f410580 0x55cf7f412e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:19.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.182+0000 7f7e871c6b80 1 -- 172.21.15.39:0/3488943286 >> 172.21.15.39:0/3488943286 conn(0x55cf7f405be0 msgr2=0x55cf7f417190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:19.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.182+0000 7f7e871c6b80 1 -- 172.21.15.39:0/3488943286 shutdown_connections 2023-07-21T18:43:19.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.182+0000 7f7e871c6b80 1 -- 172.21.15.39:0/3488943286 wait complete. 2023-07-21T18:43:19.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.182+0000 7f7e871c6b80 20 client.16113 trim_cache size 0 max 16384 2023-07-21T18:43:19.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:43:19.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:43:19.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:43:19.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:43:19.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.183+0000 7f7e7b040700 1 -- 172.21.15.39:0/4276759168 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f7e640a12c0 msgr2=0x7f7e640a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:19.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.183+0000 7f7e7b040700 1 --2- 172.21.15.39:0/4276759168 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f7e640a12c0 0x7f7e640a3770 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7e6800b2e0 tx=0x7f7e68005fd0 comp rx=0 tx=0).stop 2023-07-21T18:43:19.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.183+0000 7f7e7b040700 1 -- 172.21.15.39:0/4276759168 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e740ff3e0 msgr2=0x7f7e74106020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:19.206 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.183+0000 7f7e7b040700 1 --2- 172.21.15.39:0/4276759168 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e740ff3e0 0x7f7e74106020 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f7e6000e9a0 tx=0x7f7e6000ecb0 comp rx=0 tx=0).stop 2023-07-21T18:43:19.206 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.183+0000 7f7e7b040700 1 -- 172.21.15.39:0/4276759168 shutdown_connections 2023-07-21T18:43:19.206 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.183+0000 7f7e7b040700 1 --2- 172.21.15.39:0/4276759168 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f7e640a12c0 0x7f7e640a3770 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:19.206 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.183+0000 7f7e7b040700 1 --2- 172.21.15.39:0/4276759168 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e740fea30 0x7f7e74105ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:19.206 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.183+0000 7f7e7b040700 1 --2- 172.21.15.39:0/4276759168 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7e74104a30 0x7f7e741a3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:19.206 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.183+0000 7f7e7b040700 1 --2- 172.21.15.39:0/4276759168 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e740ff3e0 0x7f7e74106020 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:19.206 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.183+0000 7f7e7b040700 1 -- 172.21.15.39:0/4276759168 >> 172.21.15.39:0/4276759168 conn(0x7f7e74074760 msgr2=0x7f7e740fc5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:19.207 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.183+0000 7f7e7b040700 1 -- 172.21.15.39:0/4276759168 shutdown_connections 2023-07-21T18:43:19.207 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:19.183+0000 7f7e7b040700 1 -- 172.21.15.39:0/4276759168 wait complete. 2023-07-21T18:43:19.260 INFO:tasks.workunit.client.0.smithi039.stdout: 1 32631 6.04 MB/sec warmup 32 sec latency 3029.365 ms 2023-07-21T18:43:19.617 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:19 smithi193 ceph-mon[98204]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-07-21T18:43:19.617 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:19 smithi193 ceph-mon[98204]: mds.? [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] up:active 2023-07-21T18:43:19.618 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:19 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:43:19.636 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:19 smithi155 ceph-mon[97934]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-07-21T18:43:19.636 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:19 smithi155 ceph-mon[97934]: mds.? [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] up:active 2023-07-21T18:43:19.636 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:19 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:43:19.685 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:19 smithi039 ceph-mon[96489]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-07-21T18:43:19.685 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:19 smithi039 ceph-mon[96489]: mds.? [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] up:active 2023-07-21T18:43:19.685 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:19 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:43:20.260 INFO:tasks.workunit.client.0.smithi039.stdout: 1 34354 6.06 MB/sec warmup 33 sec latency 3294.762 ms 2023-07-21T18:43:20.617 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:20 smithi193 ceph-mon[98204]: pgmap v50: 97 pgs: 97 active+clean; 15 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 514 KiB/s wr, 117 op/s 2023-07-21T18:43:20.617 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:20 smithi193 ceph-mon[98204]: mds.? [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] up:active 2023-07-21T18:43:20.618 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:20 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:43:20.636 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:20 smithi155 ceph-mon[97934]: pgmap v50: 97 pgs: 97 active+clean; 15 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 514 KiB/s wr, 117 op/s 2023-07-21T18:43:20.636 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:20 smithi155 ceph-mon[97934]: mds.? [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] up:active 2023-07-21T18:43:20.636 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:20 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:43:20.685 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:20 smithi039 ceph-mon[96489]: pgmap v50: 97 pgs: 97 active+clean; 15 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 514 KiB/s wr, 117 op/s 2023-07-21T18:43:20.685 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:20 smithi039 ceph-mon[96489]: mds.? [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] up:active 2023-07-21T18:43:20.685 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:20 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:43:21.260 INFO:tasks.workunit.client.0.smithi039.stdout: 1 34354 5.89 MB/sec warmup 34 sec latency 1138.840 ms 2023-07-21T18:43:21.617 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:21 smithi193 ceph-mon[98204]: mds.? [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] up:active 2023-07-21T18:43:21.618 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:21 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:43:21.636 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:21 smithi155 ceph-mon[97934]: mds.? [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] up:active 2023-07-21T18:43:21.636 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:21 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:43:21.685 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:21 smithi039 ceph-mon[96489]: mds.? [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] up:active 2023-07-21T18:43:21.685 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:21 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:43:22.260 INFO:tasks.workunit.client.0.smithi039.stdout: 1 34354 5.72 MB/sec warmup 35 sec latency 2138.969 ms 2023-07-21T18:43:22.617 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:22 smithi193 ceph-mon[98204]: pgmap v51: 97 pgs: 97 active+clean; 41 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 4.4 MiB/s wr, 241 op/s 2023-07-21T18:43:22.636 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:22 smithi155 ceph-mon[97934]: pgmap v51: 97 pgs: 97 active+clean; 41 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 4.4 MiB/s wr, 241 op/s 2023-07-21T18:43:22.685 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:22 smithi039 ceph-mon[96489]: pgmap v51: 97 pgs: 97 active+clean; 41 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 4.4 MiB/s wr, 241 op/s 2023-07-21T18:43:23.260 INFO:tasks.workunit.client.0.smithi039.stdout: 1 34366 5.56 MB/sec warmup 36 sec latency 2772.342 ms 2023-07-21T18:43:23.617 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:23 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:43:23.636 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:23 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:43:23.685 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:23 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:43:24.260 INFO:tasks.workunit.client.0.smithi039.stdout: 1 34366 5.41 MB/sec warmup 37 sec latency 1352.482 ms 2023-07-21T18:43:24.617 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:24 smithi193 ceph-mon[98204]: pgmap v52: 97 pgs: 97 active+clean; 41 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 4.2 MiB/s wr, 171 op/s 2023-07-21T18:43:24.618 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:24 smithi193 ceph-mon[98204]: scrub summary: idle 2023-07-21T18:43:24.618 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:24 smithi193 ceph-mon[98204]: scrub summary: idle 2023-07-21T18:43:24.618 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:24 smithi193 ceph-mon[98204]: scrub summary: idle 2023-07-21T18:43:24.618 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:24 smithi193 ceph-mon[98204]: scrub summary: idle 2023-07-21T18:43:24.636 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:24 smithi155 ceph-mon[97934]: pgmap v52: 97 pgs: 97 active+clean; 41 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 4.2 MiB/s wr, 171 op/s 2023-07-21T18:43:24.636 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:24 smithi155 ceph-mon[97934]: scrub summary: idle 2023-07-21T18:43:24.637 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:24 smithi155 ceph-mon[97934]: scrub summary: idle 2023-07-21T18:43:24.637 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:24 smithi155 ceph-mon[97934]: scrub summary: idle 2023-07-21T18:43:24.637 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:24 smithi155 ceph-mon[97934]: scrub summary: idle 2023-07-21T18:43:24.685 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:24 smithi039 ceph-mon[96489]: pgmap v52: 97 pgs: 97 active+clean; 41 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 4.2 MiB/s wr, 171 op/s 2023-07-21T18:43:24.685 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:24 smithi039 ceph-mon[96489]: scrub summary: idle 2023-07-21T18:43:24.685 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:24 smithi039 ceph-mon[96489]: scrub summary: idle 2023-07-21T18:43:24.685 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:24 smithi039 ceph-mon[96489]: scrub summary: idle 2023-07-21T18:43:24.686 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:24 smithi039 ceph-mon[96489]: scrub summary: idle 2023-07-21T18:43:25.260 INFO:tasks.workunit.client.0.smithi039.stdout: 1 34366 5.27 MB/sec warmup 38 sec latency 2352.574 ms 2023-07-21T18:43:25.617 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:25 smithi193 ceph-mon[98204]: pgmap v53: 97 pgs: 97 active+clean; 41 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 4.2 MiB/s wr, 171 op/s 2023-07-21T18:43:25.636 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:25 smithi155 ceph-mon[97934]: pgmap v53: 97 pgs: 97 active+clean; 41 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 4.2 MiB/s wr, 171 op/s 2023-07-21T18:43:25.685 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:25 smithi039 ceph-mon[96489]: pgmap v53: 97 pgs: 97 active+clean; 41 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 4.2 MiB/s wr, 171 op/s 2023-07-21T18:43:26.260 INFO:tasks.workunit.client.0.smithi039.stdout: 1 34366 5.13 MB/sec warmup 39 sec latency 3352.686 ms 2023-07-21T18:43:26.292 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:26 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mon-a[96465]: 2023-07-21T18:43:26.007+0000 7f094dd1e700 -1 log_channel(cluster) log [ERR] : Health check update: 3 MDSs report damaged metadata (MDS_DAMAGE) 2023-07-21T18:43:26.617 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:26 smithi193 ceph-mon[98204]: Health check update: 3 MDSs report damaged metadata (MDS_DAMAGE) 2023-07-21T18:43:26.636 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:26 smithi155 ceph-mon[97934]: Health check update: 3 MDSs report damaged metadata (MDS_DAMAGE) 2023-07-21T18:43:26.685 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:26 smithi039 ceph-mon[96489]: Health check update: 3 MDSs report damaged metadata (MDS_DAMAGE) 2023-07-21T18:43:27.261 INFO:tasks.workunit.client.0.smithi039.stdout: 1 34366 5.00 MB/sec warmup 40 sec latency 4352.777 ms 2023-07-21T18:43:27.617 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:27 smithi193 ceph-mon[98204]: pgmap v54: 97 pgs: 97 active+clean; 30 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 4.2 MiB/s wr, 187 op/s 2023-07-21T18:43:27.636 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:27 smithi155 ceph-mon[97934]: pgmap v54: 97 pgs: 97 active+clean; 30 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 4.2 MiB/s wr, 187 op/s 2023-07-21T18:43:27.685 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:27 smithi039 ceph-mon[96489]: pgmap v54: 97 pgs: 97 active+clean; 30 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 4.2 MiB/s wr, 187 op/s 2023-07-21T18:43:28.261 INFO:tasks.workunit.client.0.smithi039.stdout: 1 34419 4.88 MB/sec warmup 41 sec latency 4982.501 ms 2023-07-21T18:43:29.261 INFO:tasks.workunit.client.0.smithi039.stdout: 1 34419 4.77 MB/sec warmup 42 sec latency 1300.086 ms 2023-07-21T18:43:29.617 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:29 smithi193 ceph-mon[98204]: pgmap v55: 97 pgs: 97 active+clean; 30 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 4.0 MiB/s wr, 140 op/s 2023-07-21T18:43:29.636 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:29 smithi155 ceph-mon[97934]: pgmap v55: 97 pgs: 97 active+clean; 30 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 4.0 MiB/s wr, 140 op/s 2023-07-21T18:43:29.685 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:29 smithi039 ceph-mon[96489]: pgmap v55: 97 pgs: 97 active+clean; 30 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 4.0 MiB/s wr, 140 op/s 2023-07-21T18:43:30.261 INFO:tasks.workunit.client.0.smithi039.stdout: 1 34419 4.66 MB/sec warmup 43 sec latency 2300.219 ms 2023-07-21T18:43:31.261 INFO:tasks.workunit.client.0.smithi039.stdout: 1 34419 4.55 MB/sec warmup 44 sec latency 3300.326 ms 2023-07-21T18:43:32.261 INFO:tasks.workunit.client.0.smithi039.stdout: 1 34419 4.45 MB/sec warmup 45 sec latency 4300.386 ms 2023-07-21T18:43:32.323 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.302+0000 7f182fd86700 1 -- 172.21.15.39:0/293848868 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1828072850 msgr2=0x7f1828072c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:32.324 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.302+0000 7f182fd86700 1 --2- 172.21.15.39:0/293848868 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1828072850 0x7f1828072c20 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f181c009990 tx=0x7f181c009ca0 comp rx=0 tx=0).stop 2023-07-21T18:43:32.324 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.302+0000 7f182fd86700 1 -- 172.21.15.39:0/293848868 shutdown_connections 2023-07-21T18:43:32.324 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.302+0000 7f182fd86700 1 --2- 172.21.15.39:0/293848868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f182810c3b0 0x7f182810e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:32.324 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.302+0000 7f182fd86700 1 --2- 172.21.15.39:0/293848868 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1828072850 0x7f1828072c20 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:32.324 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.302+0000 7f182fd86700 1 --2- 172.21.15.39:0/293848868 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1828073160 0x7f182810be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:32.324 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.302+0000 7f182fd86700 1 -- 172.21.15.39:0/293848868 >> 172.21.15.39:0/293848868 conn(0x7f18280fb5f0 msgr2=0x7f18280fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.302+0000 7f182fd86700 1 -- 172.21.15.39:0/293848868 shutdown_connections 2023-07-21T18:43:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.302+0000 7f182fd86700 1 -- 172.21.15.39:0/293848868 wait complete. 2023-07-21T18:43:32.326 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.303+0000 7f182fd86700 1 Processor -- start 2023-07-21T18:43:32.327 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.303+0000 7f182fd86700 1 -- start start 2023-07-21T18:43:32.327 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.303+0000 7f182fd86700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1828072850 0x7f1828197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:32.327 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.303+0000 7f182fd86700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1828073160 0x7f1828198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:32.327 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.303+0000 7f182fd86700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f182810c3b0 0x7f182819cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:32.327 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.303+0000 7f182fd86700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f182819d100 con 0x7f1828073160 2023-07-21T18:43:32.327 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.303+0000 7f182fd86700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f182819d270 con 0x7f182810c3b0 2023-07-21T18:43:32.327 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.303+0000 7f182fd86700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f182819d430 con 0x7f1828072850 2023-07-21T18:43:32.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.303+0000 7f182d321700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1828073160 0x7f1828198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:32.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.303+0000 7f182d321700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1828073160 0x7f1828198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58848/0 (socket says 172.21.15.39:58848) 2023-07-21T18:43:32.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.303+0000 7f182d321700 1 -- 172.21.15.39:0/103409632 learned_addr learned my addr 172.21.15.39:0/103409632 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:32.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.303+0000 7f182db22700 1 --2- 172.21.15.39:0/103409632 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1828072850 0x7f1828197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:32.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.304+0000 7f182e323700 1 --2- 172.21.15.39:0/103409632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f182810c3b0 0x7f182819cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:32.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.304+0000 7f182d321700 1 -- 172.21.15.39:0/103409632 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1828072850 msgr2=0x7f1828197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:32.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.304+0000 7f182d321700 1 --2- 172.21.15.39:0/103409632 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1828072850 0x7f1828197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:32.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.304+0000 7f182d321700 1 -- 172.21.15.39:0/103409632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f182810c3b0 msgr2=0x7f182819cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:32.329 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.304+0000 7f182d321700 1 --2- 172.21.15.39:0/103409632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f182810c3b0 0x7f182819cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:32.329 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.304+0000 7f182d321700 1 -- 172.21.15.39:0/103409632 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f181c0095f0 con 0x7f1828073160 2023-07-21T18:43:32.329 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.304+0000 7f182d321700 1 --2- 172.21.15.39:0/103409632 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1828073160 0x7f1828198450 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f182400d950 tx=0x7f182400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:32.329 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.304+0000 7f181affd700 1 -- 172.21.15.39:0/103409632 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1824009940 con 0x7f1828073160 2023-07-21T18:43:32.329 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.304+0000 7f181affd700 1 -- 172.21.15.39:0/103409632 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1824010460 con 0x7f1828073160 2023-07-21T18:43:32.329 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.304+0000 7f182fd86700 1 -- 172.21.15.39:0/103409632 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f182819d710 con 0x7f1828073160 2023-07-21T18:43:32.329 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.304+0000 7f181affd700 1 -- 172.21.15.39:0/103409632 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182400f620 con 0x7f1828073160 2023-07-21T18:43:32.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.304+0000 7f182fd86700 1 -- 172.21.15.39:0/103409632 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18281a4f50 con 0x7f1828073160 2023-07-21T18:43:32.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.306+0000 7f181affd700 1 -- 172.21.15.39:0/103409632 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f18240108d0 con 0x7f1828073160 2023-07-21T18:43:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.307+0000 7f181affd700 1 --2- 172.21.15.39:0/103409632 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f18140a12c0 0x7f18140a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.307+0000 7f181affd700 1 -- 172.21.15.39:0/103409632 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f18240ccb70 con 0x7f1828073160 2023-07-21T18:43:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.307+0000 7f182db22700 1 --2- 172.21.15.39:0/103409632 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f18140a12c0 0x7f18140a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.308+0000 7f182db22700 1 --2- 172.21.15.39:0/103409632 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f18140a12c0 0x7f18140a3770 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f181c00b5c0 tx=0x7f181c005fd0 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.308+0000 7f183bf0cb80 1 Processor -- start 2023-07-21T18:43:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.308+0000 7f183bf0cb80 1 -- start start 2023-07-21T18:43:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.308+0000 7f183bf0cb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557826f806b0 0x557827129d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.308+0000 7f183bf0cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55782712a2d0 0x557827124040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.308+0000 7f183bf0cb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557827124580 0x5578271249d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.308+0000 7f183bf0cb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x557827029c50 con 0x557827124580 2023-07-21T18:43:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.308+0000 7f183bf0cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55782707c9e0 con 0x55782712a2d0 2023-07-21T18:43:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.308+0000 7f183bf0cb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x557827075e70 con 0x557826f806b0 2023-07-21T18:43:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.308+0000 7f182e323700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557827124580 0x5578271249d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.308+0000 7f182e323700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557827124580 0x5578271249d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58858/0 (socket says 172.21.15.39:58858) 2023-07-21T18:43:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.308+0000 7f182e323700 1 -- 172.21.15.39:0/1717436203 learned_addr learned my addr 172.21.15.39:0/1717436203 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.308+0000 7f182db22700 1 --2- 172.21.15.39:0/1717436203 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557826f806b0 0x557827129d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.309+0000 7f182d321700 1 --2- 172.21.15.39:0/1717436203 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55782712a2d0 0x557827124040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.309+0000 7f182e323700 1 -- 172.21.15.39:0/1717436203 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557826f806b0 msgr2=0x557827129d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.309+0000 7f182e323700 1 --2- 172.21.15.39:0/1717436203 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557826f806b0 0x557827129d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.309+0000 7f182e323700 1 -- 172.21.15.39:0/1717436203 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55782712a2d0 msgr2=0x557827124040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.309+0000 7f182e323700 1 --2- 172.21.15.39:0/1717436203 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55782712a2d0 0x557827124040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.309+0000 7f182e323700 1 -- 172.21.15.39:0/1717436203 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557826ea3d80 con 0x557827124580 2023-07-21T18:43:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.309+0000 7f182e323700 1 --2- 172.21.15.39:0/1717436203 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557827124580 0x5578271249d0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f1820008470 tx=0x7f1820008780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61faa169930d51be server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.309+0000 7f1813fff700 1 -- 172.21.15.39:0/1717436203 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1820003a00 con 0x557827124580 2023-07-21T18:43:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.309+0000 7f1813fff700 1 -- 172.21.15.39:0/1717436203 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1820010040 con 0x557827124580 2023-07-21T18:43:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.309+0000 7f1813fff700 1 -- 172.21.15.39:0/1717436203 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1820012540 con 0x557827124580 2023-07-21T18:43:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.310+0000 7f183bf0cb80 1 -- 172.21.15.39:0/1717436203 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557827124580 msgr2=0x5578271249d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.310+0000 7f183bf0cb80 1 --2- 172.21.15.39:0/1717436203 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557827124580 0x5578271249d0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f1820008470 tx=0x7f1820008780 comp rx=0 tx=0).stop 2023-07-21T18:43:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.310+0000 7f183bf0cb80 1 -- 172.21.15.39:0/1717436203 shutdown_connections 2023-07-21T18:43:32.339 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.310+0000 7f183bf0cb80 1 --2- 172.21.15.39:0/1717436203 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557827124580 0x5578271249d0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:32.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.310+0000 7f183bf0cb80 1 --2- 172.21.15.39:0/1717436203 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557826f806b0 0x557827129d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:32.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.310+0000 7f183bf0cb80 1 --2- 172.21.15.39:0/1717436203 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55782712a2d0 0x557827124040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:32.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.310+0000 7f183bf0cb80 1 -- 172.21.15.39:0/1717436203 >> 172.21.15.39:0/1717436203 conn(0x557827119be0 msgr2=0x55782711ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:32.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.310+0000 7f183bf0cb80 1 -- 172.21.15.39:0/1717436203 shutdown_connections 2023-07-21T18:43:32.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.310+0000 7f183bf0cb80 1 -- 172.21.15.39:0/1717436203 wait complete. 2023-07-21T18:43:32.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.310+0000 7f183bf0cb80 1 -- start start 2023-07-21T18:43:32.341 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.310+0000 7f183bf0cb80 1 Processor -- start 2023-07-21T18:43:32.341 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.311+0000 7f183bf0cb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557826f806b0 0x557827130580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:32.341 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.311+0000 7f183bf0cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557827124580 0x55782712d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:32.341 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.311+0000 7f183bf0cb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55782712a2d0 0x55782712dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:32.341 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.311+0000 7f183bf0cb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x557827015a60 con 0x557826f806b0 2023-07-21T18:43:32.341 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.311+0000 7f183bf0cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557827015760 con 0x557827124580 2023-07-21T18:43:32.341 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.311+0000 7f183bf0cb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x557826ef98c0 con 0x55782712a2d0 2023-07-21T18:43:32.342 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.311+0000 7f183bf0cb80 10 client.0 _ll_register_callbacks cb 0x557827013700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:43:32.342 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.311+0000 7f182db22700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557826f806b0 0x557827130580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:32.342 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.311+0000 7f182db22700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557826f806b0 0x557827130580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58860/0 (socket says 172.21.15.39:58860) 2023-07-21T18:43:32.342 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.311+0000 7f182db22700 1 -- 172.21.15.39:0/4244035596 learned_addr learned my addr 172.21.15.39:0/4244035596 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:32.342 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.311+0000 7f182e323700 1 --2- 172.21.15.39:0/4244035596 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55782712a2d0 0x55782712dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:32.342 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.311+0000 7f182d321700 1 --2- 172.21.15.39:0/4244035596 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557827124580 0x55782712d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:32.342 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.311+0000 7f182db22700 1 -- 172.21.15.39:0/4244035596 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55782712a2d0 msgr2=0x55782712dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:32.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.311+0000 7f182db22700 1 --2- 172.21.15.39:0/4244035596 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55782712a2d0 0x55782712dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:32.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.311+0000 7f182db22700 1 -- 172.21.15.39:0/4244035596 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557827124580 msgr2=0x55782712d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:32.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.311+0000 7f182db22700 1 --2- 172.21.15.39:0/4244035596 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557827124580 0x55782712d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:32.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.311+0000 7f182db22700 1 -- 172.21.15.39:0/4244035596 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1820007cb0 con 0x557826f806b0 2023-07-21T18:43:32.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.312+0000 7f182db22700 1 --2- 172.21.15.39:0/4244035596 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557826f806b0 0x557827130580 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f181c0098e0 tx=0x7f181c009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23d84de10c614e1b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:32.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.312+0000 7f18117fa700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:43:32.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.312+0000 7f18117fa700 1 -- 172.21.15.39:0/4244035596 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181c02a050 con 0x557826f806b0 2023-07-21T18:43:32.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.312+0000 7f18117fa700 1 -- 172.21.15.39:0/4244035596 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f181c023660 con 0x557826f806b0 2023-07-21T18:43:32.344 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.312+0000 7f183bf0cb80 1 -- 172.21.15.39:0/4244035596 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557826fcf490 con 0x557826f806b0 2023-07-21T18:43:32.344 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.312+0000 7f18117fa700 1 -- 172.21.15.39:0/4244035596 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f181c023820 con 0x557826f806b0 2023-07-21T18:43:32.344 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.312+0000 7f183bf0cb80 10 client.16125 fetch_fsmap learned FSMap version 33 2023-07-21T18:43:32.344 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.312+0000 7f183bf0cb80 1 -- 172.21.15.39:0/4244035596 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x557826e8cc00 con 0x557826f806b0 2023-07-21T18:43:32.344 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.312+0000 7f18117fa700 1 -- 172.21.15.39:0/4244035596 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557826fcf490 con 0x557826f806b0 2023-07-21T18:43:32.344 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.312+0000 7f18117fa700 1 -- 172.21.15.39:0/4244035596 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f181c00fae0 con 0x557826f806b0 2023-07-21T18:43:32.344 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.312+0000 7f183bf0cb80 10 client.16125 fetch_fsmap finished waiting for FSMap version 33 2023-07-21T18:43:32.344 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.312+0000 7f183bf0cb80 10 client.16125 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:43:32.345 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.312+0000 7f183bf0cb80 20 client.16125 populate_metadata read hostname 'smithi039' 2023-07-21T18:43:32.345 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.312+0000 7f183bf0cb80 1 --2- 172.21.15.39:0/4244035596 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x557827146380 0x557827148790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:32.345 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.312+0000 7f183bf0cb80 4 client.16125 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:43:32.345 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.312+0000 7f183bf0cb80 1 -- 172.21.15.39:0/4244035596 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55782706ee50 con 0x557827146380 2023-07-21T18:43:32.345 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.313+0000 7f182d321700 1 --2- 172.21.15.39:0/4244035596 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x557827146380 0x557827148790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:32.345 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.313+0000 7f18117fa700 10 client.16125 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:43:32.345 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.313+0000 7f182d321700 1 --2- 172.21.15.39:0/4244035596 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x557827146380 0x557827148790 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7551f11e58140ee7 server_cookie=1ad34cdee16ab2a1 in_seq=0 out_seq=0 2023-07-21T18:43:32.346 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.314+0000 7f18117fa700 1 -- 172.21.15.39:0/4244035596 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55782706ee50 con 0x557827146380 2023-07-21T18:43:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.314+0000 7f18117fa700 10 client.16125 handle_command_reply: tid=0 2023-07-21T18:43:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.314+0000 7f183bf0cb80 1 client.16125 shutdown 2023-07-21T18:43:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.314+0000 7f183bf0cb80 1 -- 172.21.15.39:0/4244035596 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557826f806b0 msgr2=0x557827130580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.314+0000 7f183bf0cb80 1 --2- 172.21.15.39:0/4244035596 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557826f806b0 0x557827130580 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f181c0098e0 tx=0x7f181c009ca0 comp rx=0 tx=0).stop 2023-07-21T18:43:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.314+0000 7f183bf0cb80 1 -- 172.21.15.39:0/4244035596 shutdown_connections 2023-07-21T18:43:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.314+0000 7f183bf0cb80 1 --2- 172.21.15.39:0/4244035596 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x557827146380 0x557827148790 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.314+0000 7f183bf0cb80 1 --2- 172.21.15.39:0/4244035596 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55782712a2d0 0x55782712dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.314+0000 7f183bf0cb80 1 --2- 172.21.15.39:0/4244035596 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557826f806b0 0x557827130580 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.314+0000 7f183bf0cb80 1 --2- 172.21.15.39:0/4244035596 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557827124580 0x55782712d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.314+0000 7f18117fa700 0 client.16125 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:43:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.314+0000 7f183bf0cb80 1 -- 172.21.15.39:0/4244035596 >> 172.21.15.39:0/4244035596 conn(0x557827119be0 msgr2=0x55782711dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.314+0000 7f183bf0cb80 1 -- 172.21.15.39:0/4244035596 shutdown_connections 2023-07-21T18:43:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.314+0000 7f183bf0cb80 1 -- 172.21.15.39:0/4244035596 wait complete. 2023-07-21T18:43:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.314+0000 7f183bf0cb80 20 client.16125 trim_cache size 0 max 16384 2023-07-21T18:43:32.350 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.329+0000 7f183bf0cb80 1 Processor -- start 2023-07-21T18:43:32.350 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.329+0000 7f183bf0cb80 1 -- start start 2023-07-21T18:43:32.350 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.329+0000 7f183bf0cb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557826f806b0 0x55782713d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:32.350 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.329+0000 7f183bf0cb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557827124580 0x55782711ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:32.350 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.329+0000 7f183bf0cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55782712a2d0 0x55782712b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:32.350 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.329+0000 7f183bf0cb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578270ea090 con 0x557827124580 2023-07-21T18:43:32.350 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.329+0000 7f183bf0cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557827143990 con 0x55782712a2d0 2023-07-21T18:43:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.329+0000 7f183bf0cb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55782711bd50 con 0x557826f806b0 2023-07-21T18:43:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.329+0000 7f182d321700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557827124580 0x55782711ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.329+0000 7f182d321700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557827124580 0x55782711ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58870/0 (socket says 172.21.15.39:58870) 2023-07-21T18:43:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.329+0000 7f182d321700 1 -- 172.21.15.39:0/4080967514 learned_addr learned my addr 172.21.15.39:0/4080967514 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.329+0000 7f182db22700 1 --2- 172.21.15.39:0/4080967514 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557826f806b0 0x55782713d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.329+0000 7f182d321700 1 -- 172.21.15.39:0/4080967514 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557826f806b0 msgr2=0x55782713d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.329+0000 7f182d321700 1 --2- 172.21.15.39:0/4080967514 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557826f806b0 0x55782713d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.329+0000 7f182d321700 1 -- 172.21.15.39:0/4080967514 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55782712a2d0 msgr2=0x55782712b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.329+0000 7f182d321700 1 --2- 172.21.15.39:0/4080967514 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55782712a2d0 0x55782712b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.329+0000 7f182d321700 1 -- 172.21.15.39:0/4080967514 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55782714d3c0 con 0x557827124580 2023-07-21T18:43:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.330+0000 7f182d321700 1 --2- 172.21.15.39:0/4080967514 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557827124580 0x55782711ec60 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f18240966d0 tx=0x7f18240960d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0f1637b8bb2ad12 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.330+0000 7f1818ff9700 1 -- 172.21.15.39:0/4080967514 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18240b6070 con 0x557827124580 2023-07-21T18:43:32.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.330+0000 7f1818ff9700 1 -- 172.21.15.39:0/4080967514 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1824097da0 con 0x557827124580 2023-07-21T18:43:32.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.330+0000 7f1818ff9700 1 -- 172.21.15.39:0/4080967514 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182409d040 con 0x557827124580 2023-07-21T18:43:32.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.330+0000 7f183bf0cb80 1 -- 172.21.15.39:0/4080967514 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557827124580 msgr2=0x55782711ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:32.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.330+0000 7f183bf0cb80 1 --2- 172.21.15.39:0/4080967514 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557827124580 0x55782711ec60 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f18240966d0 tx=0x7f18240960d0 comp rx=0 tx=0).stop 2023-07-21T18:43:32.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.330+0000 7f183bf0cb80 1 -- 172.21.15.39:0/4080967514 shutdown_connections 2023-07-21T18:43:32.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.330+0000 7f183bf0cb80 1 --2- 172.21.15.39:0/4080967514 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55782712a2d0 0x55782712b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:32.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.330+0000 7f183bf0cb80 1 --2- 172.21.15.39:0/4080967514 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557826f806b0 0x55782713d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:32.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.330+0000 7f183bf0cb80 1 --2- 172.21.15.39:0/4080967514 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557827124580 0x55782711ec60 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:32.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.330+0000 7f183bf0cb80 1 -- 172.21.15.39:0/4080967514 >> 172.21.15.39:0/4080967514 conn(0x557827119be0 msgr2=0x55782712a900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:32.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.330+0000 7f183bf0cb80 1 -- 172.21.15.39:0/4080967514 shutdown_connections 2023-07-21T18:43:32.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.330+0000 7f183bf0cb80 1 -- 172.21.15.39:0/4080967514 wait complete. 2023-07-21T18:43:32.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.331+0000 7f183bf0cb80 1 -- start start 2023-07-21T18:43:32.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.331+0000 7f183bf0cb80 1 Processor -- start 2023-07-21T18:43:32.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.331+0000 7f183bf0cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557826f806b0 0x557827126ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.331+0000 7f183bf0cb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557827124580 0x557827125790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.331+0000 7f183bf0cb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55782712a2d0 0x557827125cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.331+0000 7f183bf0cb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x557827127420 con 0x55782712a2d0 2023-07-21T18:43:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.331+0000 7f183bf0cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557827127590 con 0x557826f806b0 2023-07-21T18:43:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.331+0000 7f183bf0cb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x557827126210 con 0x557827124580 2023-07-21T18:43:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.331+0000 7f183bf0cb80 10 client.0 _ll_register_callbacks cb 0x7f181c003ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:43:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.331+0000 7f182e323700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55782712a2d0 0x557827125cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.331+0000 7f182e323700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55782712a2d0 0x557827125cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58878/0 (socket says 172.21.15.39:58878) 2023-07-21T18:43:32.358 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.331+0000 7f182e323700 1 -- 172.21.15.39:0/1164254846 learned_addr learned my addr 172.21.15.39:0/1164254846 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:32.358 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.331+0000 7f182d321700 1 --2- 172.21.15.39:0/1164254846 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557827124580 0x557827125790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:32.358 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.332+0000 7f182db22700 1 --2- 172.21.15.39:0/1164254846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557826f806b0 0x557827126ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:32.358 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.332+0000 7f182e323700 1 -- 172.21.15.39:0/1164254846 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557827124580 msgr2=0x557827125790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:32.358 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.332+0000 7f182e323700 1 --2- 172.21.15.39:0/1164254846 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557827124580 0x557827125790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:32.358 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.332+0000 7f182e323700 1 -- 172.21.15.39:0/1164254846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557826f806b0 msgr2=0x557827126ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:32.358 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.332+0000 7f182e323700 1 --2- 172.21.15.39:0/1164254846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557826f806b0 0x557827126ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:32.358 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.332+0000 7f182e323700 1 -- 172.21.15.39:0/1164254846 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18281a4f50 con 0x55782712a2d0 2023-07-21T18:43:32.359 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.332+0000 7f182e323700 1 --2- 172.21.15.39:0/1164254846 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55782712a2d0 0x557827125cd0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f1820004630 tx=0x7f1820012730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad8335c7cc75a2fc server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:32.359 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.332+0000 7f18117fa700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:43:32.359 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.332+0000 7f18117fa700 1 -- 172.21.15.39:0/1164254846 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18200078f0 con 0x55782712a2d0 2023-07-21T18:43:32.359 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.332+0000 7f18117fa700 1 -- 172.21.15.39:0/1164254846 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f1820008e00 con 0x55782712a2d0 2023-07-21T18:43:32.359 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.332+0000 7f183bf0cb80 1 -- 172.21.15.39:0/1164254846 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1820010040 con 0x55782712a2d0 2023-07-21T18:43:32.359 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.332+0000 7f18117fa700 1 -- 172.21.15.39:0/1164254846 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1820012b60 con 0x55782712a2d0 2023-07-21T18:43:32.359 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.332+0000 7f183bf0cb80 10 client.16131 fetch_fsmap learned FSMap version 33 2023-07-21T18:43:32.359 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.332+0000 7f183bf0cb80 1 -- 172.21.15.39:0/1164254846 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x557826eca1f0 con 0x55782712a2d0 2023-07-21T18:43:32.360 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.332+0000 7f18117fa700 1 -- 172.21.15.39:0/1164254846 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1820010040 con 0x55782712a2d0 2023-07-21T18:43:32.360 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.332+0000 7f18117fa700 1 -- 172.21.15.39:0/1164254846 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f182001c050 con 0x55782712a2d0 2023-07-21T18:43:32.360 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.333+0000 7f183bf0cb80 10 client.16131 fetch_fsmap finished waiting for FSMap version 33 2023-07-21T18:43:32.360 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.333+0000 7f183bf0cb80 10 client.16131 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:43:32.360 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.333+0000 7f183bf0cb80 20 client.16131 populate_metadata read hostname 'smithi039' 2023-07-21T18:43:32.360 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.333+0000 7f183bf0cb80 1 --2- 172.21.15.39:0/1164254846 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55782719b260 0x55782719d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:32.360 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.333+0000 7f183bf0cb80 4 client.16131 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:43:32.361 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.333+0000 7f183bf0cb80 1 -- 172.21.15.39:0/1164254846 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557827030ff0 con 0x55782719b260 2023-07-21T18:43:32.361 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.333+0000 7f182db22700 1 --2- 172.21.15.39:0/1164254846 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55782719b260 0x55782719d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:32.361 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.333+0000 7f18117fa700 10 client.16131 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:43:32.361 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.333+0000 7f182db22700 1 --2- 172.21.15.39:0/1164254846 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55782719b260 0x55782719d670 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ad9b054a829db5c server_cookie=8ccbcb67fc016007 in_seq=0 out_seq=0 2023-07-21T18:43:32.368 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:32 smithi193 ceph-mon[98204]: pgmap v56: 97 pgs: 97 active+clean; 28 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 4.0 MiB/s wr, 153 op/s 2023-07-21T18:43:32.379 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.359+0000 7f18117fa700 1 -- 172.21.15.39:0/1164254846 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557827030ff0 con 0x55782719b260 2023-07-21T18:43:32.380 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.359+0000 7f18117fa700 10 client.16131 handle_command_reply: tid=0 2023-07-21T18:43:32.380 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.359+0000 7f183bf0cb80 1 client.16131 shutdown 2023-07-21T18:43:32.380 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.359+0000 7f183bf0cb80 1 -- 172.21.15.39:0/1164254846 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55782712a2d0 msgr2=0x557827125cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:32.380 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.359+0000 7f183bf0cb80 1 --2- 172.21.15.39:0/1164254846 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55782712a2d0 0x557827125cd0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f1820004630 tx=0x7f1820012730 comp rx=0 tx=0).stop 2023-07-21T18:43:32.380 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.359+0000 7f183bf0cb80 1 -- 172.21.15.39:0/1164254846 shutdown_connections 2023-07-21T18:43:32.380 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.359+0000 7f183bf0cb80 1 --2- 172.21.15.39:0/1164254846 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55782719b260 0x55782719d670 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:32.380 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.359+0000 7f183bf0cb80 1 --2- 172.21.15.39:0/1164254846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557826f806b0 0x557827126ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:32.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.359+0000 7f18117fa700 0 client.16131 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:43:32.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.359+0000 7f183bf0cb80 1 --2- 172.21.15.39:0/1164254846 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55782712a2d0 0x557827125cd0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:32.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.359+0000 7f183bf0cb80 1 --2- 172.21.15.39:0/1164254846 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557827124580 0x557827125790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:32.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.359+0000 7f183bf0cb80 1 -- 172.21.15.39:0/1164254846 >> 172.21.15.39:0/1164254846 conn(0x557827119be0 msgr2=0x55782712b190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:32.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.359+0000 7f183bf0cb80 1 -- 172.21.15.39:0/1164254846 shutdown_connections 2023-07-21T18:43:32.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.359+0000 7f183bf0cb80 1 -- 172.21.15.39:0/1164254846 wait complete. 2023-07-21T18:43:32.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.359+0000 7f183bf0cb80 20 client.16131 trim_cache size 0 max 16384 2023-07-21T18:43:32.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:43:32.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:43:32.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:43:32.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:43:32.383 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.360+0000 7f182fd86700 1 -- 172.21.15.39:0/103409632 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f18140a12c0 msgr2=0x7f18140a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:32.383 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.360+0000 7f182fd86700 1 --2- 172.21.15.39:0/103409632 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f18140a12c0 0x7f18140a3770 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f181c00b5c0 tx=0x7f181c005fd0 comp rx=0 tx=0).stop 2023-07-21T18:43:32.383 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.360+0000 7f182fd86700 1 -- 172.21.15.39:0/103409632 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1828073160 msgr2=0x7f1828198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:32.383 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.360+0000 7f182fd86700 1 --2- 172.21.15.39:0/103409632 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1828073160 0x7f1828198450 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f182400d950 tx=0x7f182400dc60 comp rx=0 tx=0).stop 2023-07-21T18:43:32.383 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.360+0000 7f182fd86700 1 -- 172.21.15.39:0/103409632 shutdown_connections 2023-07-21T18:43:32.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.360+0000 7f182fd86700 1 --2- 172.21.15.39:0/103409632 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f18140a12c0 0x7f18140a3770 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:32.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.360+0000 7f182fd86700 1 --2- 172.21.15.39:0/103409632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f182810c3b0 0x7f182819cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:32.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.360+0000 7f182fd86700 1 --2- 172.21.15.39:0/103409632 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1828072850 0x7f1828197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:32.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.360+0000 7f182fd86700 1 --2- 172.21.15.39:0/103409632 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1828073160 0x7f1828198450 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:32.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.360+0000 7f182fd86700 1 -- 172.21.15.39:0/103409632 >> 172.21.15.39:0/103409632 conn(0x7f18280fb5f0 msgr2=0x7f182810e450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:32.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.360+0000 7f182fd86700 1 -- 172.21.15.39:0/103409632 shutdown_connections 2023-07-21T18:43:32.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:32.360+0000 7f182fd86700 1 -- 172.21.15.39:0/103409632 wait complete. 2023-07-21T18:43:32.386 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:32 smithi155 ceph-mon[97934]: pgmap v56: 97 pgs: 97 active+clean; 28 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 4.0 MiB/s wr, 153 op/s 2023-07-21T18:43:32.435 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:32 smithi039 ceph-mon[96489]: pgmap v56: 97 pgs: 97 active+clean; 28 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 4.0 MiB/s wr, 153 op/s 2023-07-21T18:43:33.261 INFO:tasks.workunit.client.0.smithi039.stdout: 1 34455 4.37 MB/sec warmup 46 sec latency 4929.320 ms 2023-07-21T18:43:34.261 INFO:tasks.workunit.client.0.smithi039.stdout: 1 34455 4.27 MB/sec warmup 47 sec latency 1349.959 ms 2023-07-21T18:43:34.367 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:34 smithi193 ceph-mon[98204]: pgmap v57: 97 pgs: 97 active+clean; 28 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 71 KiB/s wr, 29 op/s 2023-07-21T18:43:34.386 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:34 smithi155 ceph-mon[97934]: pgmap v57: 97 pgs: 97 active+clean; 28 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 71 KiB/s wr, 29 op/s 2023-07-21T18:43:34.435 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:34 smithi039 ceph-mon[96489]: pgmap v57: 97 pgs: 97 active+clean; 28 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 71 KiB/s wr, 29 op/s 2023-07-21T18:43:35.261 INFO:tasks.workunit.client.0.smithi039.stdout: 1 34455 4.18 MB/sec warmup 48 sec latency 2350.089 ms 2023-07-21T18:43:36.261 INFO:tasks.workunit.client.0.smithi039.stdout: 1 34455 4.10 MB/sec warmup 49 sec latency 3350.194 ms 2023-07-21T18:43:36.367 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:36 smithi193 ceph-mon[98204]: pgmap v58: 97 pgs: 97 active+clean; 28 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 71 KiB/s wr, 29 op/s 2023-07-21T18:43:36.386 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:36 smithi155 ceph-mon[97934]: pgmap v58: 97 pgs: 97 active+clean; 28 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 71 KiB/s wr, 29 op/s 2023-07-21T18:43:36.435 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:36 smithi039 ceph-mon[96489]: pgmap v58: 97 pgs: 97 active+clean; 28 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 71 KiB/s wr, 29 op/s 2023-07-21T18:43:37.261 INFO:tasks.workunit.client.0.smithi039.stdout: 1 34455 4.02 MB/sec warmup 50 sec latency 4350.294 ms 2023-07-21T18:43:38.261 INFO:tasks.workunit.client.0.smithi039.stdout: 1 35327 4.07 MB/sec warmup 51 sec latency 4983.330 ms 2023-07-21T18:43:38.367 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:38 smithi193 ceph-mon[98204]: pgmap v59: 97 pgs: 97 active+clean; 28 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 93 KiB/s wr, 44 op/s 2023-07-21T18:43:38.388 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:38 smithi155 ceph-mon[97934]: pgmap v59: 97 pgs: 97 active+clean; 28 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 93 KiB/s wr, 44 op/s 2023-07-21T18:43:38.435 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:38 smithi039 ceph-mon[96489]: pgmap v59: 97 pgs: 97 active+clean; 28 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 93 KiB/s wr, 44 op/s 2023-07-21T18:43:39.261 INFO:tasks.workunit.client.0.smithi039.stdout: 1 36672 4.12 MB/sec warmup 52 sec latency 10.819 ms 2023-07-21T18:43:40.261 INFO:tasks.workunit.client.0.smithi039.stdout: 1 36692 4.04 MB/sec warmup 53 sec latency 984.361 ms 2023-07-21T18:43:40.367 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:40 smithi193 ceph-mon[98204]: pgmap v60: 97 pgs: 97 active+clean; 28 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 56 KiB/s wr, 28 op/s 2023-07-21T18:43:40.386 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:40 smithi155 ceph-mon[97934]: pgmap v60: 97 pgs: 97 active+clean; 28 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 56 KiB/s wr, 28 op/s 2023-07-21T18:43:40.435 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:40 smithi039 ceph-mon[96489]: pgmap v60: 97 pgs: 97 active+clean; 28 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 56 KiB/s wr, 28 op/s 2023-07-21T18:43:41.261 INFO:tasks.workunit.client.0.smithi039.stdout: 1 36692 3.97 MB/sec warmup 54 sec latency 1984.469 ms 2023-07-21T18:43:42.262 INFO:tasks.workunit.client.0.smithi039.stdout: 1 36692 3.90 MB/sec warmup 55 sec latency 2984.598 ms 2023-07-21T18:43:42.367 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:42 smithi193 ceph-mon[98204]: pgmap v61: 97 pgs: 97 active+clean; 29 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 201 KiB/s wr, 58 op/s 2023-07-21T18:43:42.385 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:42 smithi155 ceph-mon[97934]: pgmap v61: 97 pgs: 97 active+clean; 29 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 201 KiB/s wr, 58 op/s 2023-07-21T18:43:42.435 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:42 smithi039 ceph-mon[96489]: pgmap v61: 97 pgs: 97 active+clean; 29 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 201 KiB/s wr, 58 op/s 2023-07-21T18:43:43.262 INFO:tasks.workunit.client.0.smithi039.stdout: 1 37932 3.92 MB/sec warmup 56 sec latency 3612.969 ms 2023-07-21T18:43:44.262 INFO:tasks.workunit.client.0.smithi039.stdout: 1 39588 4.06 MB/sec warmup 57 sec latency 12.239 ms 2023-07-21T18:43:44.366 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:44 smithi193 ceph-mon[98204]: pgmap v62: 97 pgs: 97 active+clean; 29 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 167 KiB/s wr, 46 op/s 2023-07-21T18:43:44.385 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:44 smithi155 ceph-mon[97934]: pgmap v62: 97 pgs: 97 active+clean; 29 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 167 KiB/s wr, 46 op/s 2023-07-21T18:43:44.436 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:44 smithi039 ceph-mon[96489]: pgmap v62: 97 pgs: 97 active+clean; 29 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 167 KiB/s wr, 46 op/s 2023-07-21T18:43:45.207 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T18:43:45.262 INFO:tasks.workunit.client.0.smithi039.stdout: 1 41666 4.13 MB/sec warmup 58 sec latency 106.170 ms 2023-07-21T18:43:45.341 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.322+0000 7fac7eee5700 1 -- 172.21.15.39:0/3579732085 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac781049e0 msgr2=0x7fac78104db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:45.342 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.322+0000 7fac7eee5700 1 --2- 172.21.15.39:0/3579732085 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac781049e0 0x7fac78104db0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fac6c009990 tx=0x7fac6c009ca0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.342 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.322+0000 7fac7eee5700 1 -- 172.21.15.39:0/3579732085 shutdown_connections 2023-07-21T18:43:45.342 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.322+0000 7fac7eee5700 1 --2- 172.21.15.39:0/3579732085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac780ff380 0x7fac7810b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.342 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.322+0000 7fac7eee5700 1 --2- 172.21.15.39:0/3579732085 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac781049e0 0x7fac78104db0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.342 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.322+0000 7fac7eee5700 1 --2- 172.21.15.39:0/3579732085 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fac780fe9d0 0x7fac780fee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.342 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.322+0000 7fac7eee5700 1 -- 172.21.15.39:0/3579732085 >> 172.21.15.39:0/3579732085 conn(0x7fac780fa4b0 msgr2=0x7fac780fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:45.343 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.322+0000 7fac7eee5700 1 -- 172.21.15.39:0/3579732085 shutdown_connections 2023-07-21T18:43:45.343 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.322+0000 7fac7eee5700 1 -- 172.21.15.39:0/3579732085 wait complete. 2023-07-21T18:43:45.344 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.323+0000 7fac7eee5700 1 Processor -- start 2023-07-21T18:43:45.344 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.323+0000 7fac7eee5700 1 -- start start 2023-07-21T18:43:45.344 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.323+0000 7fac7eee5700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac780fe9d0 0x7fac781005e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:45.345 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.323+0000 7fac7eee5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac780ff380 0x7fac78100b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:45.345 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.323+0000 7fac7eee5700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fac781049e0 0x7fac781a3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:45.345 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.323+0000 7fac7eee5700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac781013c0 con 0x7fac780ff380 2023-07-21T18:43:45.345 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.323+0000 7fac7eee5700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac781a4300 con 0x7fac780fe9d0 2023-07-21T18:43:45.345 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.323+0000 7fac7eee5700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac781a4440 con 0x7fac781049e0 2023-07-21T18:43:45.345 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.323+0000 7fac77fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac780ff380 0x7fac78100b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:45.345 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.323+0000 7fac77fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac780ff380 0x7fac78100b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58890/0 (socket says 172.21.15.39:58890) 2023-07-21T18:43:45.346 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.323+0000 7fac7cc81700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac780fe9d0 0x7fac781005e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:45.346 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.323+0000 7fac77fff700 1 -- 172.21.15.39:0/1009501100 learned_addr learned my addr 172.21.15.39:0/1009501100 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:45.346 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.323+0000 7fac7d482700 1 --2- 172.21.15.39:0/1009501100 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fac781049e0 0x7fac781a3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:45.346 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.324+0000 7fac77fff700 1 -- 172.21.15.39:0/1009501100 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fac781049e0 msgr2=0x7fac781a3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:45.346 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.324+0000 7fac77fff700 1 --2- 172.21.15.39:0/1009501100 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fac781049e0 0x7fac781a3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.346 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.324+0000 7fac77fff700 1 -- 172.21.15.39:0/1009501100 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac780fe9d0 msgr2=0x7fac781005e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:45.346 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.324+0000 7fac77fff700 1 --2- 172.21.15.39:0/1009501100 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac780fe9d0 0x7fac781005e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.346 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.324+0000 7fac77fff700 1 -- 172.21.15.39:0/1009501100 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac6c0095f0 con 0x7fac780ff380 2023-07-21T18:43:45.347 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.324+0000 7fac7cc81700 1 --2- 172.21.15.39:0/1009501100 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac780fe9d0 0x7fac781005e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:43:45.347 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.324+0000 7fac77fff700 1 --2- 172.21.15.39:0/1009501100 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac780ff380 0x7fac78100b20 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fac6400e9d0 tx=0x7fac6400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:45.347 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.324+0000 7fac75ffb700 1 -- 172.21.15.39:0/1009501100 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac64009790 con 0x7fac780ff380 2023-07-21T18:43:45.347 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.324+0000 7fac7eee5700 1 -- 172.21.15.39:0/1009501100 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac781a46f0 con 0x7fac780ff380 2023-07-21T18:43:45.347 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.324+0000 7fac75ffb700 1 -- 172.21.15.39:0/1009501100 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac6400cbd0 con 0x7fac780ff380 2023-07-21T18:43:45.347 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.324+0000 7fac75ffb700 1 -- 172.21.15.39:0/1009501100 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac64018680 con 0x7fac780ff380 2023-07-21T18:43:45.347 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.324+0000 7fac7eee5700 1 -- 172.21.15.39:0/1009501100 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac781a4c00 con 0x7fac780ff380 2023-07-21T18:43:45.349 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.326+0000 7fac75ffb700 1 -- 172.21.15.39:0/1009501100 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fac6400cd40 con 0x7fac780ff380 2023-07-21T18:43:45.350 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.327+0000 7fac75ffb700 1 --2- 172.21.15.39:0/1009501100 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fac680a12c0 0x7fac680a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:45.350 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.327+0000 7fac7cc81700 1 --2- 172.21.15.39:0/1009501100 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fac680a12c0 0x7fac680a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:45.350 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.327+0000 7fac75ffb700 1 -- 172.21.15.39:0/1009501100 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fac64014070 con 0x7fac780ff380 2023-07-21T18:43:45.350 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.328+0000 7fac7cc81700 1 --2- 172.21.15.39:0/1009501100 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fac680a12c0 0x7fac680a3770 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fac6c00b2e0 tx=0x7fac6c005fd0 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:45.350 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.328+0000 7fac8b06bb80 1 Processor -- start 2023-07-21T18:43:45.350 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.328+0000 7fac8b06bb80 1 -- start start 2023-07-21T18:43:45.350 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.328+0000 7fac8b06bb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5623e43d7ae0 0x5623e43d7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:45.351 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.328+0000 7fac8b06bb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5623e43d83f0 0x5623e43e2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:45.351 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.328+0000 7fac8b06bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5623e43e32e0 0x5623e43e56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:45.351 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.328+0000 7fac8b06bb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623e42d5cb0 con 0x5623e43d7ae0 2023-07-21T18:43:45.351 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.328+0000 7fac8b06bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623e4328a40 con 0x5623e43e32e0 2023-07-21T18:43:45.351 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.328+0000 7fac8b06bb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623e4321ed0 con 0x5623e43d83f0 2023-07-21T18:43:45.351 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.329+0000 7fac7cc81700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5623e43d7ae0 0x5623e43d7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:45.351 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.329+0000 7fac7cc81700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5623e43d7ae0 0x5623e43d7eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58896/0 (socket says 172.21.15.39:58896) 2023-07-21T18:43:45.351 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.329+0000 7fac7cc81700 1 -- 172.21.15.39:0/1215840955 learned_addr learned my addr 172.21.15.39:0/1215840955 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:45.352 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.329+0000 7fac7d482700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5623e43e32e0 0x5623e43e56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:45.352 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.329+0000 7fac77fff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5623e43d83f0 0x5623e43e2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:45.352 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.329+0000 7fac7cc81700 1 -- 172.21.15.39:0/1215840955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5623e43d83f0 msgr2=0x5623e43e2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:45.352 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.329+0000 7fac7cc81700 1 --2- 172.21.15.39:0/1215840955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5623e43d83f0 0x5623e43e2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.352 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.329+0000 7fac7cc81700 1 -- 172.21.15.39:0/1215840955 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5623e43e32e0 msgr2=0x5623e43e56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:45.352 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.329+0000 7fac7cc81700 1 --2- 172.21.15.39:0/1215840955 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5623e43e32e0 0x5623e43e56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.352 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.329+0000 7fac7cc81700 1 -- 172.21.15.39:0/1215840955 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623e414fde0 con 0x5623e43d7ae0 2023-07-21T18:43:45.352 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.329+0000 7fac7cc81700 1 --2- 172.21.15.39:0/1215840955 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5623e43d7ae0 0x5623e43d7eb0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fac780ffb10 tx=0x7fac6c020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5666b23d8723243 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:45.353 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.329+0000 7fac62ffd700 1 -- 172.21.15.39:0/1215840955 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac6c02b070 con 0x5623e43d7ae0 2023-07-21T18:43:45.353 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.329+0000 7fac62ffd700 1 -- 172.21.15.39:0/1215840955 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac6c01e040 con 0x5623e43d7ae0 2023-07-21T18:43:45.353 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.329+0000 7fac62ffd700 1 -- 172.21.15.39:0/1215840955 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac6c024d30 con 0x5623e43d7ae0 2023-07-21T18:43:45.353 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.330+0000 7fac8b06bb80 1 -- 172.21.15.39:0/1215840955 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5623e43d7ae0 msgr2=0x5623e43d7eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:45.353 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.330+0000 7fac8b06bb80 1 --2- 172.21.15.39:0/1215840955 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5623e43d7ae0 0x5623e43d7eb0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fac780ffb10 tx=0x7fac6c020000 comp rx=0 tx=0).stop 2023-07-21T18:43:45.353 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.330+0000 7fac8b06bb80 1 -- 172.21.15.39:0/1215840955 shutdown_connections 2023-07-21T18:43:45.353 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.330+0000 7fac8b06bb80 1 --2- 172.21.15.39:0/1215840955 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5623e43e32e0 0x5623e43e56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.354 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.330+0000 7fac8b06bb80 1 --2- 172.21.15.39:0/1215840955 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5623e43d7ae0 0x5623e43d7eb0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.354 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.330+0000 7fac8b06bb80 1 --2- 172.21.15.39:0/1215840955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5623e43d83f0 0x5623e43e2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.354 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.330+0000 7fac8b06bb80 1 -- 172.21.15.39:0/1215840955 >> 172.21.15.39:0/1215840955 conn(0x5623e43c5da0 msgr2=0x5623e43cb6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:45.354 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.330+0000 7fac8b06bb80 1 -- 172.21.15.39:0/1215840955 shutdown_connections 2023-07-21T18:43:45.354 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.330+0000 7fac8b06bb80 1 -- 172.21.15.39:0/1215840955 wait complete. 2023-07-21T18:43:45.354 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.330+0000 7fac8b06bb80 1 -- start start 2023-07-21T18:43:45.354 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.331+0000 7fac8b06bb80 1 Processor -- start 2023-07-21T18:43:45.357 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.331+0000 7fac8b06bb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5623e43d7ae0 0x5623e43dffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:45.357 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.331+0000 7fac8b06bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5623e43d83f0 0x5623e43e0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:45.357 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.331+0000 7fac8b06bb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5623e43e32e0 0x5623e43e0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:45.357 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.331+0000 7fac8b06bb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623e42c1ac0 con 0x5623e43e32e0 2023-07-21T18:43:45.357 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.331+0000 7fac8b06bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623e42c17c0 con 0x5623e43d83f0 2023-07-21T18:43:45.358 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.331+0000 7fac8b06bb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623e41a5920 con 0x5623e43d7ae0 2023-07-21T18:43:45.358 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.331+0000 7fac8b06bb80 10 client.0 _ll_register_callbacks cb 0x5623e42bf760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:43:45.358 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.331+0000 7fac7d482700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5623e43e32e0 0x5623e43e0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:45.358 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.331+0000 7fac77fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5623e43d83f0 0x5623e43e0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:45.358 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.331+0000 7fac7cc81700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5623e43d7ae0 0x5623e43dffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:45.358 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.331+0000 7fac7d482700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5623e43e32e0 0x5623e43e0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58906/0 (socket says 172.21.15.39:58906) 2023-07-21T18:43:45.358 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.331+0000 7fac7d482700 1 -- 172.21.15.39:0/4261979108 learned_addr learned my addr 172.21.15.39:0/4261979108 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:45.358 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.331+0000 7fac77fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5623e43d83f0 0x5623e43e0520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:43268/0 (socket says 172.21.15.39:43268) 2023-07-21T18:43:45.359 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.332+0000 7fac7cc81700 1 -- 172.21.15.39:0/4261979108 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5623e43d83f0 msgr2=0x5623e43e0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:45.359 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.332+0000 7fac7cc81700 1 --2- 172.21.15.39:0/4261979108 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5623e43d83f0 0x5623e43e0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.359 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.332+0000 7fac7cc81700 1 -- 172.21.15.39:0/4261979108 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5623e43e32e0 msgr2=0x5623e43e0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:45.359 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.332+0000 7fac7cc81700 1 --2- 172.21.15.39:0/4261979108 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5623e43e32e0 0x5623e43e0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.359 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.332+0000 7fac7cc81700 1 -- 172.21.15.39:0/4261979108 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac7810c110 con 0x5623e43d7ae0 2023-07-21T18:43:45.359 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.332+0000 7fac7cc81700 1 --2- 172.21.15.39:0/4261979108 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5623e43d7ae0 0x5623e43dffe0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fac6c0098e0 tx=0x7fac6c024c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1f99b172c865d65 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:45.359 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.332+0000 7fac57fff700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2023-07-21T18:43:45.360 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.332+0000 7fac57fff700 1 -- 172.21.15.39:0/4261979108 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac6c02b040 con 0x5623e43d7ae0 2023-07-21T18:43:45.360 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.332+0000 7fac57fff700 1 -- 172.21.15.39:0/4261979108 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fac6c024410 con 0x5623e43d7ae0 2023-07-21T18:43:45.360 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.332+0000 7fac8b06bb80 1 -- 172.21.15.39:0/4261979108 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5623e427b4f0 con 0x5623e43d7ae0 2023-07-21T18:43:45.360 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.332+0000 7fac57fff700 1 -- 172.21.15.39:0/4261979108 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac6c00bc80 con 0x5623e43d7ae0 2023-07-21T18:43:45.360 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.332+0000 7fac8b06bb80 10 client.25654 fetch_fsmap learned FSMap version 33 2023-07-21T18:43:45.360 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.332+0000 7fac8b06bb80 1 -- 172.21.15.39:0/4261979108 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5623e4138c60 con 0x5623e43d7ae0 2023-07-21T18:43:45.360 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.333+0000 7fac57fff700 1 -- 172.21.15.39:0/4261979108 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5623e427b4f0 con 0x5623e43d7ae0 2023-07-21T18:43:45.360 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.333+0000 7fac57fff700 1 -- 172.21.15.39:0/4261979108 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fac6c037670 con 0x5623e43d7ae0 2023-07-21T18:43:45.361 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.333+0000 7fac8b06bb80 10 client.25654 fetch_fsmap finished waiting for FSMap version 33 2023-07-21T18:43:45.361 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.333+0000 7fac8b06bb80 10 client.25654 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T18:43:45.361 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.333+0000 7fac8b06bb80 20 client.25654 populate_metadata read hostname 'smithi039' 2023-07-21T18:43:45.361 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.333+0000 7fac8b06bb80 1 --2- 172.21.15.39:0/4261979108 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5623e43f2770 0x5623e43f4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:45.361 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.333+0000 7fac8b06bb80 4 client.25654 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:43:45.361 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.333+0000 7fac8b06bb80 1 -- 172.21.15.39:0/4261979108 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5623e431aeb0 con 0x5623e43f2770 2023-07-21T18:43:45.361 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.333+0000 7fac77fff700 1 --2- 172.21.15.39:0/4261979108 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5623e43f2770 0x5623e43f4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:45.361 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.333+0000 7fac57fff700 10 client.25654 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:43:45.362 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.333+0000 7fac77fff700 1 --2- 172.21.15.39:0/4261979108 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5623e43f2770 0x5623e43f4b50 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=157b53fb51a050e3 server_cookie=69a70a2b047ea60c in_seq=0 out_seq=0 2023-07-21T18:43:45.362 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.334+0000 7fac57fff700 1 -- 172.21.15.39:0/4261979108 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5623e431aeb0 con 0x5623e43f2770 2023-07-21T18:43:45.362 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.334+0000 7fac57fff700 10 client.25654 handle_command_reply: tid=0 2023-07-21T18:43:45.362 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.334+0000 7fac8b06bb80 1 client.25654 shutdown 2023-07-21T18:43:45.362 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.334+0000 7fac8b06bb80 1 -- 172.21.15.39:0/4261979108 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5623e43d7ae0 msgr2=0x5623e43dffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:45.362 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.334+0000 7fac8b06bb80 1 --2- 172.21.15.39:0/4261979108 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5623e43d7ae0 0x5623e43dffe0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fac6c0098e0 tx=0x7fac6c024c50 comp rx=0 tx=0).stop 2023-07-21T18:43:45.362 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.334+0000 7fac8b06bb80 1 -- 172.21.15.39:0/4261979108 shutdown_connections 2023-07-21T18:43:45.362 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.334+0000 7fac8b06bb80 1 --2- 172.21.15.39:0/4261979108 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5623e43f2770 0x5623e43f4b50 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.363 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.334+0000 7fac8b06bb80 1 --2- 172.21.15.39:0/4261979108 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5623e43e32e0 0x5623e43e0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.363 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.334+0000 7fac8b06bb80 1 --2- 172.21.15.39:0/4261979108 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5623e43d7ae0 0x5623e43dffe0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.363 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.334+0000 7fac57fff700 0 client.25654 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:43:45.363 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.334+0000 7fac8b06bb80 1 --2- 172.21.15.39:0/4261979108 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5623e43d83f0 0x5623e43e0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.363 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.334+0000 7fac8b06bb80 1 -- 172.21.15.39:0/4261979108 >> 172.21.15.39:0/4261979108 conn(0x5623e43c5da0 msgr2=0x5623e43cb230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:45.363 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.335+0000 7fac8b06bb80 1 -- 172.21.15.39:0/4261979108 shutdown_connections 2023-07-21T18:43:45.363 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.335+0000 7fac8b06bb80 1 -- 172.21.15.39:0/4261979108 wait complete. 2023-07-21T18:43:45.364 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.335+0000 7fac8b06bb80 20 client.25654 trim_cache size 0 max 16384 2023-07-21T18:43:45.372 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.352+0000 7fac8b06bb80 1 Processor -- start 2023-07-21T18:43:45.372 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.352+0000 7fac8b06bb80 1 -- start start 2023-07-21T18:43:45.372 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.352+0000 7fac8b06bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5623e43d7ae0 0x5623e43d99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:45.372 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.352+0000 7fac8b06bb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5623e43d83f0 0x5623e43e0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:45.372 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.352+0000 7fac8b06bb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5623e43e32e0 0x5623e43db240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:45.372 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.352+0000 7fac8b06bb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623e43ccd20 con 0x5623e43e32e0 2023-07-21T18:43:45.373 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.352+0000 7fac8b06bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623e43d8d50 con 0x5623e43d7ae0 2023-07-21T18:43:45.373 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.352+0000 7fac8b06bb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623e43d9ef0 con 0x5623e43d83f0 2023-07-21T18:43:45.373 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.353+0000 7fac7d482700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5623e43e32e0 0x5623e43db240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:45.373 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.353+0000 7fac7d482700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5623e43e32e0 0x5623e43db240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58914/0 (socket says 172.21.15.39:58914) 2023-07-21T18:43:45.373 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.353+0000 7fac7cc81700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5623e43d7ae0 0x5623e43d99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:45.373 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.353+0000 7fac7d482700 1 -- 172.21.15.39:0/1899093575 learned_addr learned my addr 172.21.15.39:0/1899093575 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:45.373 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.353+0000 7fac77fff700 1 --2- 172.21.15.39:0/1899093575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5623e43d83f0 0x5623e43e0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:45.374 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.353+0000 7fac7d482700 1 -- 172.21.15.39:0/1899093575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5623e43d83f0 msgr2=0x5623e43e0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:45.374 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.353+0000 7fac7d482700 1 --2- 172.21.15.39:0/1899093575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5623e43d83f0 0x5623e43e0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.374 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.353+0000 7fac7d482700 1 -- 172.21.15.39:0/1899093575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5623e43d7ae0 msgr2=0x5623e43d99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:45.374 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.353+0000 7fac7d482700 1 --2- 172.21.15.39:0/1899093575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5623e43d7ae0 0x5623e43d99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.374 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.353+0000 7fac7d482700 1 -- 172.21.15.39:0/1899093575 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623e4176250 con 0x5623e43e32e0 2023-07-21T18:43:45.374 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.353+0000 7fac7d482700 1 --2- 172.21.15.39:0/1899093575 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5623e43e32e0 0x5623e43db240 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fac70007600 tx=0x7fac7000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ba55e8f1d3121fe server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:45.377 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.353+0000 7fac637fe700 1 -- 172.21.15.39:0/1899093575 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac7000e040 con 0x5623e43e32e0 2023-07-21T18:43:45.377 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.353+0000 7fac637fe700 1 -- 172.21.15.39:0/1899093575 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac7000f040 con 0x5623e43e32e0 2023-07-21T18:43:45.377 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.353+0000 7fac637fe700 1 -- 172.21.15.39:0/1899093575 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac70017410 con 0x5623e43e32e0 2023-07-21T18:43:45.378 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.354+0000 7fac8b06bb80 1 -- 172.21.15.39:0/1899093575 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5623e43e32e0 msgr2=0x5623e43db240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:45.378 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.354+0000 7fac8b06bb80 1 --2- 172.21.15.39:0/1899093575 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5623e43e32e0 0x5623e43db240 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fac70007600 tx=0x7fac7000b040 comp rx=0 tx=0).stop 2023-07-21T18:43:45.378 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.354+0000 7fac8b06bb80 1 -- 172.21.15.39:0/1899093575 shutdown_connections 2023-07-21T18:43:45.378 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.354+0000 7fac8b06bb80 1 --2- 172.21.15.39:0/1899093575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5623e43d7ae0 0x5623e43d99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.378 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.354+0000 7fac8b06bb80 1 --2- 172.21.15.39:0/1899093575 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5623e43e32e0 0x5623e43db240 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.378 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.354+0000 7fac8b06bb80 1 --2- 172.21.15.39:0/1899093575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5623e43d83f0 0x5623e43e0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.378 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.354+0000 7fac8b06bb80 1 -- 172.21.15.39:0/1899093575 >> 172.21.15.39:0/1899093575 conn(0x5623e43c5da0 msgr2=0x5623e43eef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:45.379 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.354+0000 7fac8b06bb80 1 -- 172.21.15.39:0/1899093575 shutdown_connections 2023-07-21T18:43:45.379 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.354+0000 7fac8b06bb80 1 -- 172.21.15.39:0/1899093575 wait complete. 2023-07-21T18:43:45.379 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.354+0000 7fac8b06bb80 1 -- start start 2023-07-21T18:43:45.379 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.354+0000 7fac8b06bb80 1 Processor -- start 2023-07-21T18:43:45.379 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.355+0000 7fac8b06bb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5623e43d7ae0 0x5623e43e9b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:45.379 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.355+0000 7fac8b06bb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5623e43d83f0 0x5623e43e7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:45.379 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.355+0000 7fac8b06bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5623e43e32e0 0x5623e43e83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:45.379 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.355+0000 7fac8b06bb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623e43e8900 con 0x5623e43d83f0 2023-07-21T18:43:45.380 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.355+0000 7fac8b06bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623e43e8a70 con 0x5623e43e32e0 2023-07-21T18:43:45.380 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.355+0000 7fac8b06bb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623e43e8c30 con 0x5623e43d7ae0 2023-07-21T18:43:45.380 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.355+0000 7fac8b06bb80 10 client.0 _ll_register_callbacks cb 0x7fac6c00bde0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:43:45.380 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.355+0000 7fac77fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5623e43d83f0 0x5623e43e7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:45.380 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.355+0000 7fac7cc81700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5623e43d7ae0 0x5623e43e9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:45.380 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.355+0000 7fac77fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5623e43d83f0 0x5623e43e7e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58918/0 (socket says 172.21.15.39:58918) 2023-07-21T18:43:45.381 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.355+0000 7fac77fff700 1 -- 172.21.15.39:0/3606721522 learned_addr learned my addr 172.21.15.39:0/3606721522 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:45.381 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.355+0000 7fac7d482700 1 --2- 172.21.15.39:0/3606721522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5623e43e32e0 0x5623e43e83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:45.381 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.355+0000 7fac7cc81700 1 -- 172.21.15.39:0/3606721522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5623e43e32e0 msgr2=0x5623e43e83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:45.381 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.355+0000 7fac7cc81700 1 --2- 172.21.15.39:0/3606721522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5623e43e32e0 0x5623e43e83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.381 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.355+0000 7fac7cc81700 1 -- 172.21.15.39:0/3606721522 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5623e43d83f0 msgr2=0x5623e43e7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:45.381 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.355+0000 7fac7cc81700 1 --2- 172.21.15.39:0/3606721522 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5623e43d83f0 0x5623e43e7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.381 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.355+0000 7fac7cc81700 1 -- 172.21.15.39:0/3606721522 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac700092e0 con 0x5623e43d7ae0 2023-07-21T18:43:45.382 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.355+0000 7fac7cc81700 1 --2- 172.21.15.39:0/3606721522 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5623e43d7ae0 0x5623e43e9b50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fac6c000c00 tx=0x7fac6c024640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad5462d648fad2d7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:45.382 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.355+0000 7fac57fff700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2023-07-21T18:43:45.382 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.355+0000 7fac57fff700 1 -- 172.21.15.39:0/3606721522 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac6c02b040 con 0x5623e43d7ae0 2023-07-21T18:43:45.382 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.356+0000 7fac57fff700 1 -- 172.21.15.39:0/3606721522 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fac6c00bc00 con 0x5623e43d7ae0 2023-07-21T18:43:45.382 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.356+0000 7fac8b06bb80 1 -- 172.21.15.39:0/3606721522 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fac6c01e040 con 0x5623e43d7ae0 2023-07-21T18:43:45.382 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.356+0000 7fac57fff700 1 -- 172.21.15.39:0/3606721522 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac6c037610 con 0x5623e43d7ae0 2023-07-21T18:43:45.382 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.356+0000 7fac8b06bb80 10 client.25660 fetch_fsmap learned FSMap version 33 2023-07-21T18:43:45.382 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.356+0000 7fac8b06bb80 1 -- 172.21.15.39:0/3606721522 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5623e414e2b0 con 0x5623e43d7ae0 2023-07-21T18:43:45.383 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.356+0000 7fac57fff700 1 -- 172.21.15.39:0/3606721522 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac6c01e040 con 0x5623e43d7ae0 2023-07-21T18:43:45.383 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.356+0000 7fac57fff700 1 -- 172.21.15.39:0/3606721522 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fac6c005390 con 0x5623e43d7ae0 2023-07-21T18:43:45.383 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.356+0000 7fac8b06bb80 10 client.25660 fetch_fsmap finished waiting for FSMap version 33 2023-07-21T18:43:45.383 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.356+0000 7fac8b06bb80 10 client.25660 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T18:43:45.383 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.356+0000 7fac8b06bb80 20 client.25660 populate_metadata read hostname 'smithi039' 2023-07-21T18:43:45.383 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.356+0000 7fac8b06bb80 1 --2- 172.21.15.39:0/3606721522 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5623e4447670 0x5623e4449a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:45.383 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.356+0000 7fac8b06bb80 4 client.25660 mds_command: new command op to 24536 tid=0[{"prefix": "scrub status"}] 2023-07-21T18:43:45.383 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.356+0000 7fac8b06bb80 1 -- 172.21.15.39:0/3606721522 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5623e42dd050 con 0x5623e4447670 2023-07-21T18:43:45.384 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.356+0000 7fac77fff700 1 --2- 172.21.15.39:0/3606721522 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5623e4447670 0x5623e4449a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:45.384 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.357+0000 7fac57fff700 10 client.25660 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:43:45.384 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.357+0000 7fac77fff700 1 --2- 172.21.15.39:0/3606721522 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5623e4447670 0x5623e4449a80 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6fc7e8d99c2cce2 server_cookie=9018f868ee57e8b4 in_seq=0 out_seq=0 2023-07-21T18:43:45.384 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.357+0000 7fac57fff700 1 -- 172.21.15.39:0/3606721522 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5623e42dd050 con 0x5623e4447670 2023-07-21T18:43:45.384 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.357+0000 7fac57fff700 10 client.25660 handle_command_reply: tid=0 2023-07-21T18:43:45.384 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.357+0000 7fac8b06bb80 1 client.25660 shutdown 2023-07-21T18:43:45.384 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.357+0000 7fac8b06bb80 1 -- 172.21.15.39:0/3606721522 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5623e43d7ae0 msgr2=0x5623e43e9b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:45.385 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.357+0000 7fac8b06bb80 1 --2- 172.21.15.39:0/3606721522 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5623e43d7ae0 0x5623e43e9b50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fac6c000c00 tx=0x7fac6c024640 comp rx=0 tx=0).stop 2023-07-21T18:43:45.385 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.357+0000 7fac8b06bb80 1 -- 172.21.15.39:0/3606721522 shutdown_connections 2023-07-21T18:43:45.385 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.357+0000 7fac8b06bb80 1 --2- 172.21.15.39:0/3606721522 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5623e4447670 0x5623e4449a80 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.385 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.357+0000 7fac8b06bb80 1 --2- 172.21.15.39:0/3606721522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5623e43e32e0 0x5623e43e83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.385 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.357+0000 7fac8b06bb80 1 --2- 172.21.15.39:0/3606721522 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5623e43d7ae0 0x5623e43e9b50 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.385 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.357+0000 7fac8b06bb80 1 --2- 172.21.15.39:0/3606721522 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5623e43d83f0 0x5623e43e7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.385 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.357+0000 7fac57fff700 0 client.25660 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:43:45.385 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.357+0000 7fac8b06bb80 1 -- 172.21.15.39:0/3606721522 >> 172.21.15.39:0/3606721522 conn(0x5623e43c5da0 msgr2=0x5623e43db240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:45.386 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.357+0000 7fac8b06bb80 1 -- 172.21.15.39:0/3606721522 shutdown_connections 2023-07-21T18:43:45.386 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.357+0000 7fac8b06bb80 1 -- 172.21.15.39:0/3606721522 wait complete. 2023-07-21T18:43:45.386 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.357+0000 7fac8b06bb80 20 client.25660 trim_cache size 0 max 16384 2023-07-21T18:43:45.386 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2023-07-21T18:43:45.386 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "no active scrubs running", 2023-07-21T18:43:45.386 INFO:teuthology.orchestra.run.smithi039.stdout: "scrubs": {} 2023-07-21T18:43:45.387 INFO:teuthology.orchestra.run.smithi039.stdout:} 2023-07-21T18:43:45.387 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.358+0000 7fac7eee5700 1 -- 172.21.15.39:0/1009501100 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fac680a12c0 msgr2=0x7fac680a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:45.387 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.358+0000 7fac7eee5700 1 --2- 172.21.15.39:0/1009501100 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fac680a12c0 0x7fac680a3770 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fac6c00b2e0 tx=0x7fac6c005fd0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.388 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.358+0000 7fac7eee5700 1 -- 172.21.15.39:0/1009501100 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac780ff380 msgr2=0x7fac78100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:45.388 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.358+0000 7fac7eee5700 1 --2- 172.21.15.39:0/1009501100 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac780ff380 0x7fac78100b20 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fac6400e9d0 tx=0x7fac6400ece0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.388 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.358+0000 7fac7eee5700 1 -- 172.21.15.39:0/1009501100 shutdown_connections 2023-07-21T18:43:45.388 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.358+0000 7fac7eee5700 1 --2- 172.21.15.39:0/1009501100 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fac680a12c0 0x7fac680a3770 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.388 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.358+0000 7fac7eee5700 1 --2- 172.21.15.39:0/1009501100 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac780fe9d0 0x7fac781005e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.388 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.358+0000 7fac7eee5700 1 --2- 172.21.15.39:0/1009501100 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fac781049e0 0x7fac781a3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.388 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.358+0000 7fac7eee5700 1 --2- 172.21.15.39:0/1009501100 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac780ff380 0x7fac78100b20 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.389 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.358+0000 7fac7eee5700 1 -- 172.21.15.39:0/1009501100 >> 172.21.15.39:0/1009501100 conn(0x7fac780fa4b0 msgr2=0x7fac78102820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:45.389 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.358+0000 7fac7eee5700 1 -- 172.21.15.39:0/1009501100 shutdown_connections 2023-07-21T18:43:45.389 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.358+0000 7fac7eee5700 1 -- 172.21.15.39:0/1009501100 wait complete. 2023-07-21T18:43:45.390 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-21T18:43:45.390 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T18:43:45.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.480+0000 7fbc2fd19700 1 -- 172.21.15.39:0/3881616054 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc28104a20 msgr2=0x7fbc28104df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:45.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.480+0000 7fbc2fd19700 1 --2- 172.21.15.39:0/3881616054 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc28104a20 0x7fbc28104df0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fbc24009990 tx=0x7fbc24009ca0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.480+0000 7fbc2fd19700 1 -- 172.21.15.39:0/3881616054 shutdown_connections 2023-07-21T18:43:45.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.480+0000 7fbc2fd19700 1 --2- 172.21.15.39:0/3881616054 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc280ff3c0 0x7fbc2810b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.480+0000 7fbc2fd19700 1 --2- 172.21.15.39:0/3881616054 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc28104a20 0x7fbc28104df0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.480+0000 7fbc2fd19700 1 --2- 172.21.15.39:0/3881616054 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc280fea10 0x7fbc280fee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.480+0000 7fbc2fd19700 1 -- 172.21.15.39:0/3881616054 >> 172.21.15.39:0/3881616054 conn(0x7fbc280fa4f0 msgr2=0x7fbc280fc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:45.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.480+0000 7fbc2fd19700 1 -- 172.21.15.39:0/3881616054 shutdown_connections 2023-07-21T18:43:45.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.480+0000 7fbc2fd19700 1 -- 172.21.15.39:0/3881616054 wait complete. 2023-07-21T18:43:45.505 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.481+0000 7fbc2fd19700 1 Processor -- start 2023-07-21T18:43:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.481+0000 7fbc2fd19700 1 -- start start 2023-07-21T18:43:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.482+0000 7fbc2fd19700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc280fea10 0x7fbc28100800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.482+0000 7fbc2fd19700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc280ff3c0 0x7fbc28100d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.482+0000 7fbc2fd19700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc28104a20 0x7fbc281a3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.482+0000 7fbc2fd19700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc281a42f0 con 0x7fbc280fea10 2023-07-21T18:43:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.482+0000 7fbc2fd19700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc281a4430 con 0x7fbc28104a20 2023-07-21T18:43:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.482+0000 7fbc2fd19700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc281a45a0 con 0x7fbc280ff3c0 2023-07-21T18:43:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.482+0000 7fbc2dab5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc280fea10 0x7fbc28100800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:45.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.482+0000 7fbc2dab5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc280fea10 0x7fbc28100800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58930/0 (socket says 172.21.15.39:58930) 2023-07-21T18:43:45.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.482+0000 7fbc2dab5700 1 -- 172.21.15.39:0/1983986820 learned_addr learned my addr 172.21.15.39:0/1983986820 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:45.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.482+0000 7fbc2e2b6700 1 --2- 172.21.15.39:0/1983986820 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc28104a20 0x7fbc281a3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:45.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.482+0000 7fbc2d2b4700 1 --2- 172.21.15.39:0/1983986820 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc280ff3c0 0x7fbc28100d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:45.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.482+0000 7fbc2dab5700 1 -- 172.21.15.39:0/1983986820 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc280ff3c0 msgr2=0x7fbc28100d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:45.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.482+0000 7fbc2dab5700 1 --2- 172.21.15.39:0/1983986820 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc280ff3c0 0x7fbc28100d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.482+0000 7fbc2dab5700 1 -- 172.21.15.39:0/1983986820 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc28104a20 msgr2=0x7fbc281a3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:45.510 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.482+0000 7fbc2dab5700 1 --2- 172.21.15.39:0/1983986820 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc28104a20 0x7fbc281a3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.510 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.482+0000 7fbc2dab5700 1 -- 172.21.15.39:0/1983986820 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc240095f0 con 0x7fbc280fea10 2023-07-21T18:43:45.510 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.482+0000 7fbc2e2b6700 1 --2- 172.21.15.39:0/1983986820 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc28104a20 0x7fbc281a3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:43:45.510 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.483+0000 7fbc2dab5700 1 --2- 172.21.15.39:0/1983986820 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc280fea10 0x7fbc28100800 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fbc24000c00 tx=0x7fbc2400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:45.511 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.483+0000 7fbc1effd700 1 -- 172.21.15.39:0/1983986820 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2401d070 con 0x7fbc280fea10 2023-07-21T18:43:45.511 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.483+0000 7fbc2fd19700 1 -- 172.21.15.39:0/1983986820 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc281a4820 con 0x7fbc280fea10 2023-07-21T18:43:45.511 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.483+0000 7fbc1effd700 1 -- 172.21.15.39:0/1983986820 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc2400be40 con 0x7fbc280fea10 2023-07-21T18:43:45.511 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.483+0000 7fbc1effd700 1 -- 172.21.15.39:0/1983986820 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc240173f0 con 0x7fbc280fea10 2023-07-21T18:43:45.512 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.483+0000 7fbc2fd19700 1 -- 172.21.15.39:0/1983986820 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc281a4c40 con 0x7fbc280fea10 2023-07-21T18:43:45.512 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.486+0000 7fbc1effd700 1 -- 172.21.15.39:0/1983986820 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fbc24022470 con 0x7fbc280fea10 2023-07-21T18:43:45.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.487+0000 7fbc1effd700 1 --2- 172.21.15.39:0/1983986820 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fbc140a12c0 0x7fbc140a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:45.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.487+0000 7fbc2d2b4700 1 --2- 172.21.15.39:0/1983986820 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fbc140a12c0 0x7fbc140a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:45.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.487+0000 7fbc1effd700 1 -- 172.21.15.39:0/1983986820 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fbc240cdf50 con 0x7fbc280fea10 2023-07-21T18:43:45.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.487+0000 7fbc3be9fb80 1 Processor -- start 2023-07-21T18:43:45.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.487+0000 7fbc2d2b4700 1 --2- 172.21.15.39:0/1983986820 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fbc140a12c0 0x7fbc140a3770 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbc28102020 tx=0x7fbc18009260 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:45.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.488+0000 7fbc3be9fb80 1 -- start start 2023-07-21T18:43:45.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.488+0000 7fbc3be9fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5603c398a6b0 0x5603c3b33d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:45.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.488+0000 7fbc3be9fb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603c3b342d0 0x5603c3b2e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:45.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.488+0000 7fbc3be9fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5603c3b2e580 0x5603c3b2e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:45.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.488+0000 7fbc3be9fb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603c3a33c50 con 0x5603c3b342d0 2023-07-21T18:43:45.523 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.488+0000 7fbc3be9fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603c3a869e0 con 0x5603c398a6b0 2023-07-21T18:43:45.523 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.488+0000 7fbc3be9fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603c3a7fe70 con 0x5603c3b2e580 2023-07-21T18:43:45.523 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.488+0000 7fbc2d2b4700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603c3b342d0 0x5603c3b2e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:45.524 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.488+0000 7fbc2d2b4700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603c3b342d0 0x5603c3b2e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58940/0 (socket says 172.21.15.39:58940) 2023-07-21T18:43:45.524 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.488+0000 7fbc2d2b4700 1 -- 172.21.15.39:0/1858021353 learned_addr learned my addr 172.21.15.39:0/1858021353 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:45.524 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.488+0000 7fbc2dab5700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5603c398a6b0 0x5603c3b33d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:45.524 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.488+0000 7fbc2e2b6700 1 --2- 172.21.15.39:0/1858021353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5603c3b2e580 0x5603c3b2e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:45.525 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.488+0000 7fbc2d2b4700 1 -- 172.21.15.39:0/1858021353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5603c3b2e580 msgr2=0x5603c3b2e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:45.525 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.488+0000 7fbc2d2b4700 1 --2- 172.21.15.39:0/1858021353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5603c3b2e580 0x5603c3b2e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.525 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.489+0000 7fbc2d2b4700 1 -- 172.21.15.39:0/1858021353 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5603c398a6b0 msgr2=0x5603c3b33d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:45.526 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.489+0000 7fbc2d2b4700 1 --2- 172.21.15.39:0/1858021353 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5603c398a6b0 0x5603c3b33d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.526 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.489+0000 7fbc2d2b4700 1 -- 172.21.15.39:0/1858021353 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5603c38add80 con 0x5603c3b342d0 2023-07-21T18:43:45.526 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.489+0000 7fbc2e2b6700 1 --2- 172.21.15.39:0/1858021353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5603c3b2e580 0x5603c3b2e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:43:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.489+0000 7fbc2d2b4700 1 --2- 172.21.15.39:0/1858021353 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603c3b342d0 0x5603c3b2e040 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fbc18011d50 tx=0x7fbc18014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd9fa6dac28bc020 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.489+0000 7fbc13fff700 1 -- 172.21.15.39:0/1858021353 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc1800eb20 con 0x5603c3b342d0 2023-07-21T18:43:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.489+0000 7fbc13fff700 1 -- 172.21.15.39:0/1858021353 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc18021070 con 0x5603c3b342d0 2023-07-21T18:43:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.489+0000 7fbc13fff700 1 -- 172.21.15.39:0/1858021353 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc1801d670 con 0x5603c3b342d0 2023-07-21T18:43:45.528 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.490+0000 7fbc3be9fb80 1 -- 172.21.15.39:0/1858021353 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603c3b342d0 msgr2=0x5603c3b2e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:45.528 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.490+0000 7fbc3be9fb80 1 --2- 172.21.15.39:0/1858021353 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603c3b342d0 0x5603c3b2e040 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fbc18011d50 tx=0x7fbc18014000 comp rx=0 tx=0).stop 2023-07-21T18:43:45.528 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.490+0000 7fbc3be9fb80 1 -- 172.21.15.39:0/1858021353 shutdown_connections 2023-07-21T18:43:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.490+0000 7fbc3be9fb80 1 --2- 172.21.15.39:0/1858021353 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5603c398a6b0 0x5603c3b33d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.490+0000 7fbc3be9fb80 1 --2- 172.21.15.39:0/1858021353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5603c3b2e580 0x5603c3b2e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.490+0000 7fbc3be9fb80 1 --2- 172.21.15.39:0/1858021353 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603c3b342d0 0x5603c3b2e040 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.490+0000 7fbc3be9fb80 1 -- 172.21.15.39:0/1858021353 >> 172.21.15.39:0/1858021353 conn(0x5603c3b23be0 msgr2=0x5603c3b28f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.490+0000 7fbc3be9fb80 1 -- 172.21.15.39:0/1858021353 shutdown_connections 2023-07-21T18:43:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.490+0000 7fbc3be9fb80 1 -- 172.21.15.39:0/1858021353 wait complete. 2023-07-21T18:43:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.491+0000 7fbc3be9fb80 1 -- start start 2023-07-21T18:43:45.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.491+0000 7fbc3be9fb80 1 Processor -- start 2023-07-21T18:43:45.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.491+0000 7fbc3be9fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5603c398a6b0 0x5603c3b3a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:45.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.491+0000 7fbc3be9fb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603c3b2e580 0x5603c3b37820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:45.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.491+0000 7fbc3be9fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5603c3b342d0 0x5603c3b37ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:45.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.491+0000 7fbc3be9fb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603c3a1fa60 con 0x5603c3b2e580 2023-07-21T18:43:45.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.491+0000 7fbc3be9fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603c3a1f760 con 0x5603c3b342d0 2023-07-21T18:43:45.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.491+0000 7fbc3be9fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603c39038c0 con 0x5603c398a6b0 2023-07-21T18:43:45.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.491+0000 7fbc3be9fb80 10 client.0 _ll_register_callbacks cb 0x5603c3a1d700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:43:45.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.491+0000 7fbc2d2b4700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603c3b2e580 0x5603c3b37820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:45.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.491+0000 7fbc2dab5700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5603c398a6b0 0x5603c3b3a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:45.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.491+0000 7fbc2d2b4700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603c3b2e580 0x5603c3b37820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58946/0 (socket says 172.21.15.39:58946) 2023-07-21T18:43:45.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.491+0000 7fbc2d2b4700 1 -- 172.21.15.39:0/923828707 learned_addr learned my addr 172.21.15.39:0/923828707 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:45.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.492+0000 7fbc2e2b6700 1 --2- 172.21.15.39:0/923828707 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5603c3b342d0 0x5603c3b37ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:45.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.492+0000 7fbc2dab5700 1 -- 172.21.15.39:0/923828707 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5603c3b342d0 msgr2=0x5603c3b37ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:45.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.492+0000 7fbc2dab5700 1 --2- 172.21.15.39:0/923828707 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5603c3b342d0 0x5603c3b37ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.492+0000 7fbc2dab5700 1 -- 172.21.15.39:0/923828707 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603c3b2e580 msgr2=0x5603c3b37820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:45.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.492+0000 7fbc2dab5700 1 --2- 172.21.15.39:0/923828707 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603c3b2e580 0x5603c3b37820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.492+0000 7fbc2dab5700 1 -- 172.21.15.39:0/923828707 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc18017070 con 0x5603c398a6b0 2023-07-21T18:43:45.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.492+0000 7fbc2d2b4700 1 --2- 172.21.15.39:0/923828707 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603c3b2e580 0x5603c3b37820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:43:45.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.492+0000 7fbc2dab5700 1 --2- 172.21.15.39:0/923828707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5603c398a6b0 0x5603c3b3a5a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbc240cb6c0 tx=0x7fbc2409aab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9201b03491aa2dfc server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:45.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.492+0000 7fbc117fa700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2023-07-21T18:43:45.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.492+0000 7fbc117fa700 1 -- 172.21.15.39:0/923828707 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2409b530 con 0x5603c398a6b0 2023-07-21T18:43:45.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.492+0000 7fbc117fa700 1 -- 172.21.15.39:0/923828707 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fbc240a3490 con 0x5603c398a6b0 2023-07-21T18:43:45.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.492+0000 7fbc3be9fb80 1 -- 172.21.15.39:0/923828707 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5603c39d9490 con 0x5603c398a6b0 2023-07-21T18:43:45.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.492+0000 7fbc117fa700 1 -- 172.21.15.39:0/923828707 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc240b9320 con 0x5603c398a6b0 2023-07-21T18:43:45.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.492+0000 7fbc3be9fb80 10 client.25672 fetch_fsmap learned FSMap version 33 2023-07-21T18:43:45.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.492+0000 7fbc3be9fb80 1 -- 172.21.15.39:0/923828707 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5603c3896c00 con 0x5603c398a6b0 2023-07-21T18:43:45.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.492+0000 7fbc117fa700 1 -- 172.21.15.39:0/923828707 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5603c39d9490 con 0x5603c398a6b0 2023-07-21T18:43:45.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.493+0000 7fbc117fa700 1 -- 172.21.15.39:0/923828707 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fbc240a3090 con 0x5603c398a6b0 2023-07-21T18:43:45.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.493+0000 7fbc3be9fb80 10 client.25672 fetch_fsmap finished waiting for FSMap version 33 2023-07-21T18:43:45.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.493+0000 7fbc3be9fb80 10 client.25672 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:43:45.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.493+0000 7fbc3be9fb80 20 client.25672 populate_metadata read hostname 'smithi039' 2023-07-21T18:43:45.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.493+0000 7fbc3be9fb80 1 --2- 172.21.15.39:0/923828707 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5603c3b478b0 0x5603c3b49cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:45.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.493+0000 7fbc3be9fb80 4 client.25672 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:43:45.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.493+0000 7fbc3be9fb80 1 -- 172.21.15.39:0/923828707 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5603c3a78e50 con 0x5603c3b478b0 2023-07-21T18:43:45.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.493+0000 7fbc2d2b4700 1 --2- 172.21.15.39:0/923828707 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5603c3b478b0 0x5603c3b49cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:45.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.493+0000 7fbc117fa700 10 client.25672 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:43:45.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.493+0000 7fbc2d2b4700 1 --2- 172.21.15.39:0/923828707 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5603c3b478b0 0x5603c3b49cc0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=729be9c7e9b4a99e server_cookie=5e847a909d050877 in_seq=0 out_seq=0 2023-07-21T18:43:45.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.494+0000 7fbc117fa700 1 -- 172.21.15.39:0/923828707 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5603c3a78e50 con 0x5603c3b478b0 2023-07-21T18:43:45.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.494+0000 7fbc117fa700 10 client.25672 handle_command_reply: tid=0 2023-07-21T18:43:45.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.494+0000 7fbc3be9fb80 1 client.25672 shutdown 2023-07-21T18:43:45.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.494+0000 7fbc3be9fb80 1 -- 172.21.15.39:0/923828707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5603c398a6b0 msgr2=0x5603c3b3a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:45.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.494+0000 7fbc3be9fb80 1 --2- 172.21.15.39:0/923828707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5603c398a6b0 0x5603c3b3a5a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbc240cb6c0 tx=0x7fbc2409aab0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.538 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.494+0000 7fbc3be9fb80 1 -- 172.21.15.39:0/923828707 shutdown_connections 2023-07-21T18:43:45.538 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.494+0000 7fbc3be9fb80 1 --2- 172.21.15.39:0/923828707 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5603c3b478b0 0x5603c3b49cc0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.538 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.494+0000 7fbc3be9fb80 1 --2- 172.21.15.39:0/923828707 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5603c3b342d0 0x5603c3b37ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.538 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.494+0000 7fbc3be9fb80 1 --2- 172.21.15.39:0/923828707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5603c398a6b0 0x5603c3b3a5a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.538 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.494+0000 7fbc3be9fb80 1 --2- 172.21.15.39:0/923828707 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603c3b2e580 0x5603c3b37820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.538 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.494+0000 7fbc3be9fb80 1 -- 172.21.15.39:0/923828707 >> 172.21.15.39:0/923828707 conn(0x5603c3b23be0 msgr2=0x5603c3b27c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:45.538 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.494+0000 7fbc117fa700 0 client.25672 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:43:45.538 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.494+0000 7fbc3be9fb80 1 -- 172.21.15.39:0/923828707 shutdown_connections 2023-07-21T18:43:45.539 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.495+0000 7fbc3be9fb80 1 -- 172.21.15.39:0/923828707 wait complete. 2023-07-21T18:43:45.539 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.495+0000 7fbc3be9fb80 20 client.25672 trim_cache size 0 max 16384 2023-07-21T18:43:45.541 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.508+0000 7fee2dd61700 1 -- 172.21.15.39:0/4043505166 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee28104a90 msgr2=0x7fee28104e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:45.541 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.508+0000 7fee2dd61700 1 --2- 172.21.15.39:0/4043505166 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee28104a90 0x7fee28104e60 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fee18009990 tx=0x7fee18009ca0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.541 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.508+0000 7fee2dd61700 1 -- 172.21.15.39:0/4043505166 shutdown_connections 2023-07-21T18:43:45.541 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.508+0000 7fee2dd61700 1 --2- 172.21.15.39:0/4043505166 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee280ff430 0x7fee2810b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.541 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.508+0000 7fee2dd61700 1 --2- 172.21.15.39:0/4043505166 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee28104a90 0x7fee28104e60 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.541 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.508+0000 7fee2dd61700 1 --2- 172.21.15.39:0/4043505166 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee280fea80 0x7fee280feef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.542 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.508+0000 7fee2dd61700 1 -- 172.21.15.39:0/4043505166 >> 172.21.15.39:0/4043505166 conn(0x7fee28075c50 msgr2=0x7fee28076050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:45.542 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.508+0000 7fee2dd61700 1 -- 172.21.15.39:0/4043505166 shutdown_connections 2023-07-21T18:43:45.542 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.508+0000 7fee2dd61700 1 -- 172.21.15.39:0/4043505166 wait complete. 2023-07-21T18:43:45.542 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.509+0000 7fee2dd61700 1 Processor -- start 2023-07-21T18:43:45.542 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.509+0000 7fee2dd61700 1 -- start start 2023-07-21T18:43:45.542 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.510+0000 7fee2dd61700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee280fea80 0x7fee2819a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:45.542 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.510+0000 7fee2dd61700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee280ff430 0x7fee2819ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:45.543 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.510+0000 7fee2dd61700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee28104a90 0x7fee281948a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:45.543 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.510+0000 7fee2dd61700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee28194f10 con 0x7fee28104a90 2023-07-21T18:43:45.543 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.510+0000 7fee2dd61700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee28195080 con 0x7fee280fea80 2023-07-21T18:43:45.543 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.510+0000 7fee2dd61700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee28195240 con 0x7fee280ff430 2023-07-21T18:43:45.543 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.510+0000 7fee27fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee28104a90 0x7fee281948a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:45.543 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.510+0000 7fee26ffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee280ff430 0x7fee2819ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:45.543 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.510+0000 7fee27fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee28104a90 0x7fee281948a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58962/0 (socket says 172.21.15.39:58962) 2023-07-21T18:43:45.543 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.510+0000 7fee27fff700 1 -- 172.21.15.39:0/518618751 learned_addr learned my addr 172.21.15.39:0/518618751 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:45.544 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.510+0000 7fee277fe700 1 --2- 172.21.15.39:0/518618751 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee280fea80 0x7fee2819a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:45.544 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.510+0000 7fee26ffd700 1 -- 172.21.15.39:0/518618751 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee280fea80 msgr2=0x7fee2819a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:45.544 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.510+0000 7fee26ffd700 1 --2- 172.21.15.39:0/518618751 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee280fea80 0x7fee2819a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.544 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.510+0000 7fee26ffd700 1 -- 172.21.15.39:0/518618751 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee28104a90 msgr2=0x7fee281948a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:45.544 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.510+0000 7fee26ffd700 1 --2- 172.21.15.39:0/518618751 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee28104a90 0x7fee281948a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.544 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.510+0000 7fee26ffd700 1 -- 172.21.15.39:0/518618751 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee1c009520 con 0x7fee280ff430 2023-07-21T18:43:45.544 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.510+0000 7fee26ffd700 1 --2- 172.21.15.39:0/518618751 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee280ff430 0x7fee2819ad70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fee1c00e9f0 tx=0x7fee1c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:45.544 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.511+0000 7fee24ff9700 1 -- 172.21.15.39:0/518618751 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee1c00cb70 con 0x7fee280ff430 2023-07-21T18:43:45.545 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.511+0000 7fee24ff9700 1 -- 172.21.15.39:0/518618751 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee1c00ee90 con 0x7fee280ff430 2023-07-21T18:43:45.545 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.511+0000 7fee2dd61700 1 -- 172.21.15.39:0/518618751 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee180095f0 con 0x7fee280ff430 2023-07-21T18:43:45.545 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.511+0000 7fee24ff9700 1 -- 172.21.15.39:0/518618751 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee1c003cf0 con 0x7fee280ff430 2023-07-21T18:43:45.545 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.511+0000 7fee2dd61700 1 -- 172.21.15.39:0/518618751 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee28195710 con 0x7fee280ff430 2023-07-21T18:43:45.545 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.512+0000 7fee2dd61700 1 -- 172.21.15.39:0/518618751 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee28067750 con 0x7fee280ff430 2023-07-21T18:43:45.545 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.513+0000 7fee24ff9700 1 -- 172.21.15.39:0/518618751 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fee1c00ccd0 con 0x7fee280ff430 2023-07-21T18:43:45.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.513+0000 7fbc3be9fb80 1 Processor -- start 2023-07-21T18:43:45.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.513+0000 7fbc3be9fb80 1 -- start start 2023-07-21T18:43:45.546 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.513+0000 7fee24ff9700 1 --2- 172.21.15.39:0/518618751 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fee100a1380 0x7fee100a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:45.547 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.514+0000 7fbc3be9fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5603c3b23be0 0x5603c3b3ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:45.547 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.514+0000 7fbc3be9fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5603c398a6b0 0x5603c3b35010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:45.547 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.514+0000 7fbc3be9fb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603c3b2e580 0x5603c3b3a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:45.547 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.514+0000 7fbc3be9fb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603c3b387b0 con 0x5603c3b2e580 2023-07-21T18:43:45.547 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.514+0000 7fbc3be9fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603c3b38e00 con 0x5603c3b23be0 2023-07-21T18:43:45.547 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.514+0000 7fbc3be9fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603c3b2ab40 con 0x5603c398a6b0 2023-07-21T18:43:45.548 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.514+0000 7fee24ff9700 1 -- 172.21.15.39:0/518618751 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fee1c014070 con 0x7fee280ff430 2023-07-21T18:43:45.548 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.514+0000 7fee277fe700 1 --2- 172.21.15.39:0/518618751 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fee100a1380 0x7fee100a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:45.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.514+0000 7fbc2dab5700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5603c3b23be0 0x5603c3b3ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:45.549 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.514+0000 7fbc2e2b6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603c3b2e580 0x5603c3b3a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:45.549 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.514+0000 7fbc2e2b6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603c3b2e580 0x5603c3b3a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58968/0 (socket says 172.21.15.39:58968) 2023-07-21T18:43:45.549 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.514+0000 7fbc2dab5700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5603c3b23be0 0x5603c3b3ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:43330/0 (socket says 172.21.15.39:43330) 2023-07-21T18:43:45.549 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.514+0000 7fbc2e2b6700 1 -- 172.21.15.39:0/2731865526 learned_addr learned my addr 172.21.15.39:0/2731865526 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:45.549 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.514+0000 7fbc2dab5700 1 -- 172.21.15.39:0/2731865526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5603c398a6b0 msgr2=0x5603c3b35010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:45.549 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.514+0000 7fbc2dab5700 1 --2- 172.21.15.39:0/2731865526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5603c398a6b0 0x5603c3b35010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.549 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.514+0000 7fbc2dab5700 1 -- 172.21.15.39:0/2731865526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603c3b2e580 msgr2=0x5603c3b3a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:45.550 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.514+0000 7fbc2dab5700 1 --2- 172.21.15.39:0/2731865526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603c3b2e580 0x5603c3b3a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.550 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.514+0000 7fbc2dab5700 1 -- 172.21.15.39:0/2731865526 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5603c3b4e8f0 con 0x5603c3b23be0 2023-07-21T18:43:45.550 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.514+0000 7fee277fe700 1 --2- 172.21.15.39:0/518618751 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fee100a1380 0x7fee100a3830 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fee18006010 tx=0x7fee18005960 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.515+0000 7fbc2dab5700 1 --2- 172.21.15.39:0/2731865526 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5603c3b23be0 0x5603c3b3ec60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbc240975d0 tx=0x7fbc24099680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c215345b73ef0adf server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:45.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.515+0000 7fbc1cff9700 1 -- 172.21.15.39:0/2731865526 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc240a3490 con 0x5603c3b23be0 2023-07-21T18:43:45.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.515+0000 7fbc1cff9700 1 -- 172.21.15.39:0/2731865526 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc24017570 con 0x5603c3b23be0 2023-07-21T18:43:45.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.515+0000 7fbc1cff9700 1 -- 172.21.15.39:0/2731865526 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc240a3200 con 0x5603c3b23be0 2023-07-21T18:43:45.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.515+0000 7fbc3be9fb80 1 -- 172.21.15.39:0/2731865526 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5603c3b23be0 msgr2=0x5603c3b3ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:45.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.515+0000 7fbc3be9fb80 1 --2- 172.21.15.39:0/2731865526 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5603c3b23be0 0x5603c3b3ec60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbc240975d0 tx=0x7fbc24099680 comp rx=0 tx=0).stop 2023-07-21T18:43:45.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.515+0000 7fbc3be9fb80 1 -- 172.21.15.39:0/2731865526 shutdown_connections 2023-07-21T18:43:45.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.516+0000 7fbc3be9fb80 1 --2- 172.21.15.39:0/2731865526 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5603c3b23be0 0x5603c3b3ec60 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.516+0000 7fbc3be9fb80 1 --2- 172.21.15.39:0/2731865526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603c3b2e580 0x5603c3b3a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.516+0000 7fbc3be9fb80 1 --2- 172.21.15.39:0/2731865526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5603c398a6b0 0x5603c3b35010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.516+0000 7fbc3be9fb80 1 -- 172.21.15.39:0/2731865526 >> 172.21.15.39:0/2731865526 conn(0x5603c3b44160 msgr2=0x5603c3b34900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:45.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.516+0000 7fbc3be9fb80 1 -- 172.21.15.39:0/2731865526 shutdown_connections 2023-07-21T18:43:45.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.516+0000 7fbc3be9fb80 1 -- 172.21.15.39:0/2731865526 wait complete. 2023-07-21T18:43:45.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.516+0000 7fbc3be9fb80 1 -- start start 2023-07-21T18:43:45.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.516+0000 7fbc3be9fb80 1 Processor -- start 2023-07-21T18:43:45.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.516+0000 7fbc3be9fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5603c398a6b0 0x5603c3b2f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:45.554 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.516+0000 7fbc3be9fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5603c3b23be0 0x5603c3b2fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:45.554 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.516+0000 7fbc3be9fb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603c3b2e580 0x5603c3b30210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:45.554 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.516+0000 7fbc3be9fb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603c3b3f2a0 con 0x5603c3b2e580 2023-07-21T18:43:45.554 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.516+0000 7fbc3be9fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603c3b30750 con 0x5603c3b23be0 2023-07-21T18:43:45.554 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.516+0000 7fbc3be9fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603c3b30910 con 0x5603c398a6b0 2023-07-21T18:43:45.554 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.517+0000 7fbc3be9fb80 10 client.0 _ll_register_callbacks cb 0x7fbc2409aea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:43:45.554 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.517+0000 7fbc2d2b4700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5603c3b23be0 0x5603c3b2fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:45.554 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.517+0000 7fbc2d2b4700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5603c3b23be0 0x5603c3b2fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:43338/0 (socket says 172.21.15.39:43338) 2023-07-21T18:43:45.555 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.517+0000 7fbc2d2b4700 1 -- 172.21.15.39:0/1454913013 learned_addr learned my addr 172.21.15.39:0/1454913013 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:45.555 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.517+0000 7fbc2dab5700 1 --2- 172.21.15.39:0/1454913013 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5603c398a6b0 0x5603c3b2f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:45.555 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.517+0000 7fbc2e2b6700 1 --2- 172.21.15.39:0/1454913013 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603c3b2e580 0x5603c3b30210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:45.555 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.517+0000 7fee24ff9700 1 -- 172.21.15.39:0/518618751 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fee1c0968f0 con 0x7fee280ff430 2023-07-21T18:43:45.557 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.517+0000 7fbc2d2b4700 1 -- 172.21.15.39:0/1454913013 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5603c398a6b0 msgr2=0x5603c3b2f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:45.557 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.517+0000 7fbc2d2b4700 1 --2- 172.21.15.39:0/1454913013 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5603c398a6b0 0x5603c3b2f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.557 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.517+0000 7fbc2d2b4700 1 -- 172.21.15.39:0/1454913013 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603c3b2e580 msgr2=0x5603c3b30210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:45.557 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.517+0000 7fbc2d2b4700 1 --2- 172.21.15.39:0/1454913013 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603c3b2e580 0x5603c3b30210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.557 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.517+0000 7fbc2d2b4700 1 -- 172.21.15.39:0/1454913013 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5603c3896c00 con 0x5603c3b23be0 2023-07-21T18:43:45.557 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.517+0000 7fbc2dab5700 1 --2- 172.21.15.39:0/1454913013 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5603c398a6b0 0x5603c3b2f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:43:45.557 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.517+0000 7fbc2d2b4700 1 --2- 172.21.15.39:0/1454913013 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5603c3b23be0 0x5603c3b2fcd0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x5603c3b3ab80 tx=0x7fbc18005c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8bc55856637677c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:45.558 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.517+0000 7fbc117fa700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-21T18:43:45.558 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.517+0000 7fbc117fa700 1 -- 172.21.15.39:0/1454913013 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc1801cce0 con 0x5603c3b23be0 2023-07-21T18:43:45.558 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.517+0000 7fbc117fa700 1 -- 172.21.15.39:0/1454913013 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fbc1800eb20 con 0x5603c3b23be0 2023-07-21T18:43:45.558 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.517+0000 7fbc3be9fb80 1 -- 172.21.15.39:0/1454913013 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbc18021070 con 0x5603c3b23be0 2023-07-21T18:43:45.558 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.518+0000 7fbc117fa700 1 -- 172.21.15.39:0/1454913013 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc1800ece0 con 0x5603c3b23be0 2023-07-21T18:43:45.558 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.518+0000 7fbc3be9fb80 10 client.25682 fetch_fsmap learned FSMap version 33 2023-07-21T18:43:45.558 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.518+0000 7fbc3be9fb80 1 -- 172.21.15.39:0/1454913013 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5603c38d41f0 con 0x5603c3b23be0 2023-07-21T18:43:45.558 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.519+0000 7fbc117fa700 1 -- 172.21.15.39:0/1454913013 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc1803f020 con 0x5603c3b23be0 2023-07-21T18:43:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.519+0000 7fbc117fa700 1 -- 172.21.15.39:0/1454913013 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fbc18005030 con 0x5603c3b23be0 2023-07-21T18:43:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.519+0000 7fbc3be9fb80 10 client.25682 fetch_fsmap finished waiting for FSMap version 33 2023-07-21T18:43:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.519+0000 7fbc3be9fb80 10 client.25682 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:43:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.519+0000 7fbc3be9fb80 20 client.25682 populate_metadata read hostname 'smithi039' 2023-07-21T18:43:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.519+0000 7fbc3be9fb80 1 --2- 172.21.15.39:0/1454913013 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5603c3b342d0 0x5603c3b9e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.519+0000 7fbc3be9fb80 4 client.25682 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:43:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.519+0000 7fbc3be9fb80 1 -- 172.21.15.39:0/1454913013 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5603c3a3aff0 con 0x5603c3b342d0 2023-07-21T18:43:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.519+0000 7fbc2dab5700 1 --2- 172.21.15.39:0/1454913013 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5603c3b342d0 0x5603c3b9e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.519+0000 7fbc117fa700 10 client.25682 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:43:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.519+0000 7fbc2dab5700 1 --2- 172.21.15.39:0/1454913013 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5603c3b342d0 0x5603c3b9e0e0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93e9b32a583eed0d server_cookie=e726a78c18ecd1b5 in_seq=0 out_seq=0 2023-07-21T18:43:45.565 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.545+0000 7fbc117fa700 1 -- 172.21.15.39:0/1454913013 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5603c3a3aff0 con 0x5603c3b342d0 2023-07-21T18:43:45.565 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.545+0000 7fbc117fa700 10 client.25682 handle_command_reply: tid=0 2023-07-21T18:43:45.565 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.545+0000 7fbc3be9fb80 1 client.25682 shutdown 2023-07-21T18:43:45.565 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.546+0000 7fbc3be9fb80 1 -- 172.21.15.39:0/1454913013 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5603c3b23be0 msgr2=0x5603c3b2fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:45.565 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.546+0000 7fbc3be9fb80 1 --2- 172.21.15.39:0/1454913013 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5603c3b23be0 0x5603c3b2fcd0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x5603c3b3ab80 tx=0x7fbc18005c20 comp rx=0 tx=0).stop 2023-07-21T18:43:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.546+0000 7fbc3be9fb80 1 -- 172.21.15.39:0/1454913013 shutdown_connections 2023-07-21T18:43:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.546+0000 7fbc3be9fb80 1 --2- 172.21.15.39:0/1454913013 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5603c3b342d0 0x5603c3b9e0e0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.546+0000 7fbc3be9fb80 1 --2- 172.21.15.39:0/1454913013 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603c3b2e580 0x5603c3b30210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.546+0000 7fbc117fa700 0 client.25682 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:43:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.546+0000 7fbc3be9fb80 1 --2- 172.21.15.39:0/1454913013 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5603c398a6b0 0x5603c3b2f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.546+0000 7fbc3be9fb80 1 --2- 172.21.15.39:0/1454913013 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5603c3b23be0 0x5603c3b2fcd0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.546+0000 7fbc3be9fb80 1 -- 172.21.15.39:0/1454913013 >> 172.21.15.39:0/1454913013 conn(0x5603c3b44160 msgr2=0x5603c3b35190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.546+0000 7fbc3be9fb80 1 -- 172.21.15.39:0/1454913013 shutdown_connections 2023-07-21T18:43:45.567 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.546+0000 7fbc3be9fb80 1 -- 172.21.15.39:0/1454913013 wait complete. 2023-07-21T18:43:45.567 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.546+0000 7fbc3be9fb80 20 client.25682 trim_cache size 0 max 16384 2023-07-21T18:43:45.567 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:43:45.567 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:43:45.567 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:43:45.567 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:43:45.568 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.546+0000 7fbc2fd19700 1 -- 172.21.15.39:0/1983986820 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fbc140a12c0 msgr2=0x7fbc140a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:45.568 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.546+0000 7fbc2fd19700 1 --2- 172.21.15.39:0/1983986820 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fbc140a12c0 0x7fbc140a3770 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbc28102020 tx=0x7fbc18009260 comp rx=0 tx=0).stop 2023-07-21T18:43:45.568 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.546+0000 7fbc2fd19700 1 -- 172.21.15.39:0/1983986820 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc280fea10 msgr2=0x7fbc28100800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.546+0000 7fbc2fd19700 1 --2- 172.21.15.39:0/1983986820 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc280fea10 0x7fbc28100800 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fbc24000c00 tx=0x7fbc2400f690 comp rx=0 tx=0).stop 2023-07-21T18:43:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.546+0000 7fbc2fd19700 1 -- 172.21.15.39:0/1983986820 shutdown_connections 2023-07-21T18:43:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.547+0000 7fbc2fd19700 1 --2- 172.21.15.39:0/1983986820 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fbc140a12c0 0x7fbc140a3770 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.547+0000 7fbc2fd19700 1 --2- 172.21.15.39:0/1983986820 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc28104a20 0x7fbc281a3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.547+0000 7fbc2fd19700 1 --2- 172.21.15.39:0/1983986820 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc280fea10 0x7fbc28100800 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.547+0000 7fbc2fd19700 1 --2- 172.21.15.39:0/1983986820 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc280ff3c0 0x7fbc28100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.547+0000 7fbc2fd19700 1 -- 172.21.15.39:0/1983986820 >> 172.21.15.39:0/1983986820 conn(0x7fbc280fa4f0 msgr2=0x7fbc28068460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.547+0000 7fbc2fd19700 1 -- 172.21.15.39:0/1983986820 shutdown_connections 2023-07-21T18:43:45.570 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:45.547+0000 7fbc2fd19700 1 -- 172.21.15.39:0/1983986820 wait complete. 2023-07-21T18:43:45.799 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.780+0000 7fee2dd61700 1 -- 172.21.15.39:0/518618751 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fee28195710 con 0x7fee280ff430 2023-07-21T18:43:45.800 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.781+0000 7fee24ff9700 1 -- 172.21.15.39:0/518618751 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+10900 (secure 0 0 0) 0x7fee28195710 con 0x7fee280ff430 2023-07-21T18:43:45.800 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T18:43:45.800 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/147318681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":147318681},{"type":"v1","addr":"172.21.15.39:6835","nonce":147318681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/1062891760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1062891760},{"type":"v1","addr":"172.21.15.39:6837","nonce":1062891760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T18:40:04.594553+0000","modified":"2023-07-21T18:43:22.251010+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24536,"mds_1":24500,"mds_2":24487,"mds_3":24464,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/3184273349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":3184273349},{"type":"v1","addr":"172.21.15.39:6839","nonce":3184273349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":52,"addr":"172.21.15.39:6841/640981220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":640981220},{"type":"v1","addr":"172.21.15.39:6841","nonce":640981220}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3106166804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3106166804},{"type":"v1","addr":"172.21.15.193:6833","nonce":3106166804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24437":{"gid":24437,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/1034755277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1034755277},{"type":"v1","addr":"172.21.15.155:6833","nonce":1034755277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/381560657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":381560657},{"type":"v1","addr":"172.21.15.193:6835","nonce":381560657}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6837/2626761843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2626761843},{"type":"v1","addr":"172.21.15.193:6837","nonce":2626761843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24464":{"gid":24464,"name":"e","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.155:6835/3854856286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3854856286},{"type":"v1","addr":"172.21.15.155:6835","nonce":3854856286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":53,"addr":"172.21.15.193:6839/3148488639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3148488639},{"type":"v1","addr":"172.21.15.193:6839","nonce":3148488639}]},"join_fscid":-1,"export_targets":[0,3],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":54,"addr":"172.21.15.155:6837/947670056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":947670056},{"type":"v1","addr":"172.21.15.155:6837","nonce":947670056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6839/431690342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":431690342},{"type":"v1","addr":"172.21.15.155:6839","nonce":431690342}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T18:43:45.802 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.783+0000 7fee2dd61700 1 -- 172.21.15.39:0/518618751 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fee100a1380 msgr2=0x7fee100a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:45.802 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.783+0000 7fee2dd61700 1 --2- 172.21.15.39:0/518618751 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fee100a1380 0x7fee100a3830 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fee18006010 tx=0x7fee18005960 comp rx=0 tx=0).stop 2023-07-21T18:43:45.802 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.783+0000 7fee2dd61700 1 -- 172.21.15.39:0/518618751 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee280ff430 msgr2=0x7fee2819ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:45.802 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.783+0000 7fee2dd61700 1 --2- 172.21.15.39:0/518618751 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee280ff430 0x7fee2819ad70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fee1c00e9f0 tx=0x7fee1c00ed00 comp rx=0 tx=0).stop 2023-07-21T18:43:45.802 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.783+0000 7fee2dd61700 1 -- 172.21.15.39:0/518618751 shutdown_connections 2023-07-21T18:43:45.803 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.783+0000 7fee2dd61700 1 --2- 172.21.15.39:0/518618751 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fee100a1380 0x7fee100a3830 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.803 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.783+0000 7fee2dd61700 1 --2- 172.21.15.39:0/518618751 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee280fea80 0x7fee2819a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.803 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.783+0000 7fee2dd61700 1 --2- 172.21.15.39:0/518618751 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee28104a90 0x7fee281948a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.803 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.783+0000 7fee2dd61700 1 --2- 172.21.15.39:0/518618751 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee280ff430 0x7fee2819ad70 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.803 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.783+0000 7fee2dd61700 1 -- 172.21.15.39:0/518618751 >> 172.21.15.39:0/518618751 conn(0x7fee28075c50 msgr2=0x7fee28102780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:45.803 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.783+0000 7fee2dd61700 1 -- 172.21.15.39:0/518618751 shutdown_connections 2023-07-21T18:43:45.803 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.783+0000 7fee2dd61700 1 -- 172.21.15.39:0/518618751 wait complete. 2023-07-21T18:43:45.804 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 33 2023-07-21T18:43:45.814 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-07-21T18:43:45.942 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.923+0000 7fc09ad86700 1 -- 172.21.15.39:0/3815022165 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc09410a040 msgr2=0x7fc09410c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:45.943 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.923+0000 7fc09ad86700 1 --2- 172.21.15.39:0/3815022165 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc09410a040 0x7fc09410c4b0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fc08c00b0a0 tx=0x7fc08c00b3b0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.943 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.923+0000 7fc09ad86700 1 -- 172.21.15.39:0/3815022165 shutdown_connections 2023-07-21T18:43:45.943 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.923+0000 7fc09ad86700 1 --2- 172.21.15.39:0/3815022165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc094100d00 0x7fc0941010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.943 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.923+0000 7fc09ad86700 1 --2- 172.21.15.39:0/3815022165 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc09410a040 0x7fc09410c4b0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.943 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.923+0000 7fc09ad86700 1 --2- 172.21.15.39:0/3815022165 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc094101610 0x7fc094109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.944 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.923+0000 7fc09ad86700 1 -- 172.21.15.39:0/3815022165 >> 172.21.15.39:0/3815022165 conn(0x7fc0940fa4b0 msgr2=0x7fc0940fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:45.944 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.923+0000 7fc09ad86700 1 -- 172.21.15.39:0/3815022165 shutdown_connections 2023-07-21T18:43:45.944 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.923+0000 7fc09ad86700 1 -- 172.21.15.39:0/3815022165 wait complete. 2023-07-21T18:43:45.945 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.924+0000 7fc09ad86700 1 Processor -- start 2023-07-21T18:43:45.945 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.924+0000 7fc09ad86700 1 -- start start 2023-07-21T18:43:45.946 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.924+0000 7fc09ad86700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc094100d00 0x7fc0940fe950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:45.946 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.924+0000 7fc09ad86700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc094101610 0x7fc0940fee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:45.946 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.924+0000 7fc09ad86700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc09410a040 0x7fc0941a4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:45.946 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.924+0000 7fc09ad86700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0940ff6c0 con 0x7fc094100d00 2023-07-21T18:43:45.946 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.924+0000 7fc09ad86700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0941a45c0 con 0x7fc094101610 2023-07-21T18:43:45.946 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.924+0000 7fc09ad86700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0941a4700 con 0x7fc09410a040 2023-07-21T18:43:45.946 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.924+0000 7fc098b22700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc094100d00 0x7fc0940fe950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:45.947 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.924+0000 7fc098b22700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc094100d00 0x7fc0940fe950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58986/0 (socket says 172.21.15.39:58986) 2023-07-21T18:43:45.947 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.924+0000 7fc098b22700 1 -- 172.21.15.39:0/2407706353 learned_addr learned my addr 172.21.15.39:0/2407706353 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:45.947 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.924+0000 7fc099323700 1 --2- 172.21.15.39:0/2407706353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc09410a040 0x7fc0941a4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:45.947 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.924+0000 7fc093fff700 1 --2- 172.21.15.39:0/2407706353 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc094101610 0x7fc0940fee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:45.947 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.925+0000 7fc098b22700 1 -- 172.21.15.39:0/2407706353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc09410a040 msgr2=0x7fc0941a4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:45.947 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.925+0000 7fc098b22700 1 --2- 172.21.15.39:0/2407706353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc09410a040 0x7fc0941a4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.947 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.925+0000 7fc098b22700 1 -- 172.21.15.39:0/2407706353 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc094101610 msgr2=0x7fc0940fee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:45.947 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.925+0000 7fc098b22700 1 --2- 172.21.15.39:0/2407706353 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc094101610 0x7fc0940fee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.948 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.925+0000 7fc098b22700 1 -- 172.21.15.39:0/2407706353 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc08c009d00 con 0x7fc094100d00 2023-07-21T18:43:45.948 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.925+0000 7fc098b22700 1 --2- 172.21.15.39:0/2407706353 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc094100d00 0x7fc0940fe950 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fc08400e9a0 tx=0x7fc08400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:45.948 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.925+0000 7fc091ffb700 1 -- 172.21.15.39:0/2407706353 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc084009790 con 0x7fc094100d00 2023-07-21T18:43:45.948 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.925+0000 7fc091ffb700 1 -- 172.21.15.39:0/2407706353 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc084010430 con 0x7fc094100d00 2023-07-21T18:43:45.948 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.925+0000 7fc09ad86700 1 -- 172.21.15.39:0/2407706353 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0941a4960 con 0x7fc094100d00 2023-07-21T18:43:45.948 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.925+0000 7fc091ffb700 1 -- 172.21.15.39:0/2407706353 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc084018680 con 0x7fc094100d00 2023-07-21T18:43:45.948 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.925+0000 7fc09ad86700 1 -- 172.21.15.39:0/2407706353 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0941a4e40 con 0x7fc094100d00 2023-07-21T18:43:45.951 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.928+0000 7fc091ffb700 1 -- 172.21.15.39:0/2407706353 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fc0840105a0 con 0x7fc094100d00 2023-07-21T18:43:45.952 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.928+0000 7fc091ffb700 1 --2- 172.21.15.39:0/2407706353 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fc07c0a12c0 0x7fc07c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:45.952 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.928+0000 7fc091ffb700 1 -- 172.21.15.39:0/2407706353 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fc084014070 con 0x7fc094100d00 2023-07-21T18:43:45.952 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.928+0000 7fc093fff700 1 --2- 172.21.15.39:0/2407706353 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fc07c0a12c0 0x7fc07c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:45.952 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.929+0000 7fc0a6f0cb80 1 Processor -- start 2023-07-21T18:43:45.952 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.929+0000 7fc093fff700 1 --2- 172.21.15.39:0/2407706353 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fc07c0a12c0 0x7fc07c0a3770 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc094100140 tx=0x7fc088009260 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:45.952 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.929+0000 7fc0a6f0cb80 1 -- start start 2023-07-21T18:43:45.952 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.929+0000 7fc0a6f0cb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e92cf82d0 0x558e92cf86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:45.953 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.929+0000 7fc0a6f0cb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e92cf8be0 0x558e92d03590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:45.953 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.929+0000 7fc0a6f0cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e92d03ad0 0x558e92d05eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:45.953 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.929+0000 7fc0a6f0cb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e92bfecb0 con 0x558e92cf82d0 2023-07-21T18:43:45.953 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.929+0000 7fc0a6f0cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e92c51a40 con 0x558e92d03ad0 2023-07-21T18:43:45.953 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.929+0000 7fc0a6f0cb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e92c4aed0 con 0x558e92cf8be0 2023-07-21T18:43:45.953 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.929+0000 7fc098b22700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e92cf82d0 0x558e92cf86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:45.953 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.929+0000 7fc098b22700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e92cf82d0 0x558e92cf86a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58994/0 (socket says 172.21.15.39:58994) 2023-07-21T18:43:45.953 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.929+0000 7fc098b22700 1 -- 172.21.15.39:0/2826632000 learned_addr learned my addr 172.21.15.39:0/2826632000 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:45.954 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.929+0000 7fc093fff700 1 --2- 172.21.15.39:0/2826632000 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e92cf8be0 0x558e92d03590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:45.954 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.929+0000 7fc099323700 1 --2- 172.21.15.39:0/2826632000 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e92d03ad0 0x558e92d05eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:45.954 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.929+0000 7fc098b22700 1 -- 172.21.15.39:0/2826632000 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e92cf8be0 msgr2=0x558e92d03590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:45.954 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.929+0000 7fc098b22700 1 --2- 172.21.15.39:0/2826632000 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e92cf8be0 0x558e92d03590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.954 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.929+0000 7fc098b22700 1 -- 172.21.15.39:0/2826632000 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e92d03ad0 msgr2=0x558e92d05eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:45.954 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.929+0000 7fc098b22700 1 --2- 172.21.15.39:0/2826632000 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e92d03ad0 0x558e92d05eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.954 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.929+0000 7fc098b22700 1 -- 172.21.15.39:0/2826632000 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e92a78de0 con 0x558e92cf82d0 2023-07-21T18:43:45.954 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.930+0000 7fc098b22700 1 --2- 172.21.15.39:0/2826632000 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e92cf82d0 0x558e92cf86a0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fc0840a1ec0 tx=0x7fc084099fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8669538ac06cc5a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:45.955 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.930+0000 7fc07affd700 1 -- 172.21.15.39:0/2826632000 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0840a23f0 con 0x558e92cf82d0 2023-07-21T18:43:45.955 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.930+0000 7fc07affd700 1 -- 172.21.15.39:0/2826632000 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc084010880 con 0x558e92cf82d0 2023-07-21T18:43:45.955 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.930+0000 7fc07affd700 1 -- 172.21.15.39:0/2826632000 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0840a1f60 con 0x558e92cf82d0 2023-07-21T18:43:45.955 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.930+0000 7fc0a6f0cb80 1 -- 172.21.15.39:0/2826632000 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e92cf82d0 msgr2=0x558e92cf86a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:45.955 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.930+0000 7fc0a6f0cb80 1 --2- 172.21.15.39:0/2826632000 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e92cf82d0 0x558e92cf86a0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fc0840a1ec0 tx=0x7fc084099fa0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.955 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.930+0000 7fc0a6f0cb80 1 -- 172.21.15.39:0/2826632000 shutdown_connections 2023-07-21T18:43:45.955 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.930+0000 7fc0a6f0cb80 1 --2- 172.21.15.39:0/2826632000 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e92d03ad0 0x558e92d05eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.956 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.930+0000 7fc0a6f0cb80 1 --2- 172.21.15.39:0/2826632000 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e92cf82d0 0x558e92cf86a0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.956 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.931+0000 7fc0a6f0cb80 1 --2- 172.21.15.39:0/2826632000 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e92cf8be0 0x558e92d03590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.956 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.931+0000 7fc0a6f0cb80 1 -- 172.21.15.39:0/2826632000 >> 172.21.15.39:0/2826632000 conn(0x558e92ceeda0 msgr2=0x558e92cf46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:45.956 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.931+0000 7fc0a6f0cb80 1 -- 172.21.15.39:0/2826632000 shutdown_connections 2023-07-21T18:43:45.956 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.931+0000 7fc0a6f0cb80 1 -- 172.21.15.39:0/2826632000 wait complete. 2023-07-21T18:43:45.956 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.931+0000 7fc0a6f0cb80 1 -- start start 2023-07-21T18:43:45.956 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.931+0000 7fc0a6f0cb80 1 Processor -- start 2023-07-21T18:43:45.956 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.931+0000 7fc0a6f0cb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e92cf82d0 0x558e92d00880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:45.957 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.931+0000 7fc0a6f0cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e92cf8be0 0x558e92d00dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:45.957 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.931+0000 7fc0a6f0cb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e92d03ad0 0x558e92d01300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:45.957 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.931+0000 7fc0a6f0cb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e92beaac0 con 0x558e92d03ad0 2023-07-21T18:43:45.957 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.931+0000 7fc0a6f0cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e92bea7c0 con 0x558e92cf8be0 2023-07-21T18:43:45.957 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.931+0000 7fc0a6f0cb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e92ace920 con 0x558e92cf82d0 2023-07-21T18:43:45.957 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.932+0000 7fc0a6f0cb80 10 client.0 _ll_register_callbacks cb 0x558e92be8760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:43:45.957 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.932+0000 7fc093fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e92cf8be0 0x558e92d00dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:45.957 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.932+0000 7fc098b22700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e92cf82d0 0x558e92d00880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:45.958 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.932+0000 7fc098b22700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e92cf82d0 0x558e92d00880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:49320/0 (socket says 172.21.15.39:49320) 2023-07-21T18:43:45.958 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.932+0000 7fc099323700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e92d03ad0 0x558e92d01300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:45.958 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.932+0000 7fc093fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e92cf8be0 0x558e92d00dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:43366/0 (socket says 172.21.15.39:43366) 2023-07-21T18:43:45.958 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.932+0000 7fc098b22700 1 -- 172.21.15.39:0/1616435283 learned_addr learned my addr 172.21.15.39:0/1616435283 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:45.958 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.932+0000 7fc098b22700 1 -- 172.21.15.39:0/1616435283 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e92cf8be0 msgr2=0x558e92d00dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:45.958 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.932+0000 7fc098b22700 1 --2- 172.21.15.39:0/1616435283 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e92cf8be0 0x558e92d00dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.958 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.932+0000 7fc098b22700 1 -- 172.21.15.39:0/1616435283 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e92d03ad0 msgr2=0x558e92d01300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:45.958 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.932+0000 7fc098b22700 1 --2- 172.21.15.39:0/1616435283 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e92d03ad0 0x558e92d01300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.959 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.932+0000 7fc098b22700 1 -- 172.21.15.39:0/1616435283 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0941a4e40 con 0x558e92cf82d0 2023-07-21T18:43:45.959 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.932+0000 7fc098b22700 1 --2- 172.21.15.39:0/1616435283 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e92cf82d0 0x558e92d00880 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc084009230 tx=0x7fc084099740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5924719b6ad50e5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:45.959 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.932+0000 7fc073fff700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2023-07-21T18:43:45.959 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.932+0000 7fc073fff700 1 -- 172.21.15.39:0/1616435283 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0840a23f0 con 0x558e92cf82d0 2023-07-21T18:43:45.959 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.932+0000 7fc073fff700 1 -- 172.21.15.39:0/1616435283 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fc0840a2170 con 0x558e92cf82d0 2023-07-21T18:43:45.959 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.932+0000 7fc0a6f0cb80 1 -- 172.21.15.39:0/1616435283 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558e92ba44f0 con 0x558e92cf82d0 2023-07-21T18:43:45.959 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.932+0000 7fc073fff700 1 -- 172.21.15.39:0/1616435283 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc084094bb0 con 0x558e92cf82d0 2023-07-21T18:43:45.959 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.933+0000 7fc0a6f0cb80 10 client.25693 fetch_fsmap learned FSMap version 33 2023-07-21T18:43:45.960 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.933+0000 7fc0a6f0cb80 1 -- 172.21.15.39:0/1616435283 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x558e92a61c60 con 0x558e92cf82d0 2023-07-21T18:43:45.961 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.933+0000 7fc073fff700 1 -- 172.21.15.39:0/1616435283 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558e92ba44f0 con 0x558e92cf82d0 2023-07-21T18:43:45.961 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.933+0000 7fc073fff700 1 -- 172.21.15.39:0/1616435283 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fc084098f00 con 0x558e92cf82d0 2023-07-21T18:43:45.961 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.933+0000 7fc0a6f0cb80 10 client.25693 fetch_fsmap finished waiting for FSMap version 33 2023-07-21T18:43:45.961 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.933+0000 7fc0a6f0cb80 10 client.25693 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2023-07-21T18:43:45.961 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.933+0000 7fc0a6f0cb80 20 client.25693 populate_metadata read hostname 'smithi039' 2023-07-21T18:43:45.961 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.933+0000 7fc0a6f0cb80 1 --2- 172.21.15.39:0/1616435283 >> [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] conn(0x558e92d13160 0x558e92d15540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:45.961 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.933+0000 7fc0a6f0cb80 4 client.25693 mds_command: new command op to 14604 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:43:45.962 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.933+0000 7fc0a6f0cb80 1 -- 172.21.15.39:0/1616435283 --> [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558e92c43eb0 con 0x558e92d13160 2023-07-21T18:43:45.962 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.933+0000 7fc093fff700 1 --2- 172.21.15.39:0/1616435283 >> [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] conn(0x558e92d13160 0x558e92d15540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:45.962 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.933+0000 7fc073fff700 10 client.25693 ms_handle_connect on v2:172.21.15.39:6840/640981220 2023-07-21T18:43:45.962 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.933+0000 7fc093fff700 1 --2- 172.21.15.39:0/1616435283 >> [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] conn(0x558e92d13160 0x558e92d15540 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=333454229431003 server_cookie=c70080eac8209fe6 in_seq=0 out_seq=0 2023-07-21T18:43:45.962 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.934+0000 7fc073fff700 1 -- 172.21.15.39:0/1616435283 <== mds.4 v2:172.21.15.39:6840/640981220 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558e92c43eb0 con 0x558e92d13160 2023-07-21T18:43:45.962 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.934+0000 7fc073fff700 10 client.25693 handle_command_reply: tid=0 2023-07-21T18:43:45.962 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.934+0000 7fc0a6f0cb80 1 client.25693 shutdown 2023-07-21T18:43:45.963 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.934+0000 7fc0a6f0cb80 1 -- 172.21.15.39:0/1616435283 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e92cf82d0 msgr2=0x558e92d00880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:45.963 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.934+0000 7fc0a6f0cb80 1 --2- 172.21.15.39:0/1616435283 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e92cf82d0 0x558e92d00880 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc084009230 tx=0x7fc084099740 comp rx=0 tx=0).stop 2023-07-21T18:43:45.963 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.934+0000 7fc0a6f0cb80 1 -- 172.21.15.39:0/1616435283 shutdown_connections 2023-07-21T18:43:45.963 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.934+0000 7fc0a6f0cb80 1 --2- 172.21.15.39:0/1616435283 >> [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] conn(0x558e92d13160 0x558e92d15540 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.963 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.934+0000 7fc0a6f0cb80 1 --2- 172.21.15.39:0/1616435283 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e92d03ad0 0x558e92d01300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.963 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.934+0000 7fc0a6f0cb80 1 --2- 172.21.15.39:0/1616435283 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e92cf82d0 0x558e92d00880 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.963 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.934+0000 7fc0a6f0cb80 1 --2- 172.21.15.39:0/1616435283 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e92cf8be0 0x558e92d00dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.963 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.934+0000 7fc073fff700 0 client.25693 ms_handle_reset on v2:172.21.15.39:6840/640981220 2023-07-21T18:43:45.964 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.934+0000 7fc0a6f0cb80 1 -- 172.21.15.39:0/1616435283 >> 172.21.15.39:0/1616435283 conn(0x558e92ceeda0 msgr2=0x558e92cf4230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:45.964 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.934+0000 7fc0a6f0cb80 1 -- 172.21.15.39:0/1616435283 shutdown_connections 2023-07-21T18:43:45.964 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.934+0000 7fc0a6f0cb80 1 -- 172.21.15.39:0/1616435283 wait complete. 2023-07-21T18:43:45.964 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.935+0000 7fc0a6f0cb80 20 client.25693 trim_cache size 0 max 16384 2023-07-21T18:43:45.968 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.948+0000 7fc0a6f0cb80 1 Processor -- start 2023-07-21T18:43:45.968 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.948+0000 7fc0a6f0cb80 1 -- start start 2023-07-21T18:43:45.968 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.948+0000 7fc0a6f0cb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e92cf82d0 0x558e92d0a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:45.968 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.948+0000 7fc0a6f0cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e92cf8be0 0x558e92cfbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:45.969 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.948+0000 7fc0a6f0cb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e92d03ad0 0x558e92cfc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:45.969 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.948+0000 7fc0a6f0cb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e92cfb6a0 con 0x558e92cf82d0 2023-07-21T18:43:45.969 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.948+0000 7fc0a6f0cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e92d0aa30 con 0x558e92cf8be0 2023-07-21T18:43:45.969 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.948+0000 7fc0a6f0cb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e92d01770 con 0x558e92d03ad0 2023-07-21T18:43:45.969 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.948+0000 7fc098b22700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e92cf82d0 0x558e92d0a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:45.969 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.948+0000 7fc098b22700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e92cf82d0 0x558e92d0a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59008/0 (socket says 172.21.15.39:59008) 2023-07-21T18:43:45.969 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.948+0000 7fc098b22700 1 -- 172.21.15.39:0/1040573693 learned_addr learned my addr 172.21.15.39:0/1040573693 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:45.969 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.948+0000 7fc093fff700 1 --2- 172.21.15.39:0/1040573693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e92cf8be0 0x558e92cfbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:45.970 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.948+0000 7fc099323700 1 --2- 172.21.15.39:0/1040573693 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e92d03ad0 0x558e92cfc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:45.970 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.949+0000 7fc098b22700 1 -- 172.21.15.39:0/1040573693 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e92d03ad0 msgr2=0x558e92cfc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:45.970 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.949+0000 7fc098b22700 1 --2- 172.21.15.39:0/1040573693 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e92d03ad0 0x558e92cfc110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.970 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.949+0000 7fc098b22700 1 -- 172.21.15.39:0/1040573693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e92cf8be0 msgr2=0x558e92cfbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:45.970 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.949+0000 7fc098b22700 1 --2- 172.21.15.39:0/1040573693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e92cf8be0 0x558e92cfbbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.970 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.949+0000 7fc098b22700 1 -- 172.21.15.39:0/1040573693 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e92a9f250 con 0x558e92cf82d0 2023-07-21T18:43:45.970 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.949+0000 7fc093fff700 1 --2- 172.21.15.39:0/1040573693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e92cf8be0 0x558e92cfbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:43:45.970 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.949+0000 7fc098b22700 1 --2- 172.21.15.39:0/1040573693 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e92cf82d0 0x558e92d0a4f0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fc084099ca0 tx=0x7fc0840997b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3c88b6e9755a5a3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:45.971 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.949+0000 7fc07b7fe700 1 -- 172.21.15.39:0/1040573693 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0840962e0 con 0x558e92cf82d0 2023-07-21T18:43:45.971 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.949+0000 7fc07b7fe700 1 -- 172.21.15.39:0/1040573693 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0840a23f0 con 0x558e92cf82d0 2023-07-21T18:43:45.971 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.949+0000 7fc07b7fe700 1 -- 172.21.15.39:0/1040573693 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc084094bb0 con 0x558e92cf82d0 2023-07-21T18:43:45.974 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.950+0000 7fc0a6f0cb80 1 -- 172.21.15.39:0/1040573693 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e92cf82d0 msgr2=0x558e92d0a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:45.974 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.950+0000 7fc0a6f0cb80 1 --2- 172.21.15.39:0/1040573693 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e92cf82d0 0x558e92d0a4f0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fc084099ca0 tx=0x7fc0840997b0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.974 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.950+0000 7fc0a6f0cb80 1 -- 172.21.15.39:0/1040573693 shutdown_connections 2023-07-21T18:43:45.974 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.950+0000 7fc0a6f0cb80 1 --2- 172.21.15.39:0/1040573693 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e92d03ad0 0x558e92cfc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.974 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.950+0000 7fc0a6f0cb80 1 --2- 172.21.15.39:0/1040573693 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e92cf82d0 0x558e92d0a4f0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.975 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.950+0000 7fc0a6f0cb80 1 --2- 172.21.15.39:0/1040573693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e92cf8be0 0x558e92cfbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.975 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.950+0000 7fc0a6f0cb80 1 -- 172.21.15.39:0/1040573693 >> 172.21.15.39:0/1040573693 conn(0x558e92ceeda0 msgr2=0x558e92d01060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:45.975 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.950+0000 7fc0a6f0cb80 1 -- 172.21.15.39:0/1040573693 shutdown_connections 2023-07-21T18:43:45.975 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.950+0000 7fc0a6f0cb80 1 -- 172.21.15.39:0/1040573693 wait complete. 2023-07-21T18:43:45.975 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.950+0000 7fc0a6f0cb80 1 -- start start 2023-07-21T18:43:45.975 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.950+0000 7fc0a6f0cb80 1 Processor -- start 2023-07-21T18:43:45.975 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.951+0000 7fc0a6f0cb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e92cf82d0 0x558e92d602a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:45.975 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.951+0000 7fc0a6f0cb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e92cf8be0 0x558e92d607e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:45.976 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.951+0000 7fc0a6f0cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e92d03ad0 0x558e92d5f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:45.976 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.951+0000 7fc0a6f0cb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e92d5f5a0 con 0x558e92cf82d0 2023-07-21T18:43:45.976 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.951+0000 7fc0a6f0cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e92d5f710 con 0x558e92d03ad0 2023-07-21T18:43:45.976 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.951+0000 7fc0a6f0cb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e92d5f8d0 con 0x558e92cf8be0 2023-07-21T18:43:45.976 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.951+0000 7fc0a6f0cb80 10 client.0 _ll_register_callbacks cb 0x558e92be8760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:43:45.976 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.951+0000 7fc098b22700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e92cf82d0 0x558e92d602a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:45.976 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.951+0000 7fc098b22700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e92cf82d0 0x558e92d602a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59014/0 (socket says 172.21.15.39:59014) 2023-07-21T18:43:45.977 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.951+0000 7fc098b22700 1 -- 172.21.15.39:0/1174212415 learned_addr learned my addr 172.21.15.39:0/1174212415 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:45.977 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.951+0000 7fc093fff700 1 --2- 172.21.15.39:0/1174212415 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e92cf8be0 0x558e92d607e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:45.977 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.951+0000 7fc099323700 1 --2- 172.21.15.39:0/1174212415 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e92d03ad0 0x558e92d5f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:45.977 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.951+0000 7fc098b22700 1 -- 172.21.15.39:0/1174212415 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e92cf8be0 msgr2=0x558e92d607e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:45.977 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.951+0000 7fc098b22700 1 --2- 172.21.15.39:0/1174212415 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e92cf8be0 0x558e92d607e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.977 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.951+0000 7fc098b22700 1 -- 172.21.15.39:0/1174212415 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e92d03ad0 msgr2=0x558e92d5f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:45.977 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.951+0000 7fc098b22700 1 --2- 172.21.15.39:0/1174212415 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e92d03ad0 0x558e92d5f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.977 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.951+0000 7fc098b22700 1 -- 172.21.15.39:0/1174212415 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc088017070 con 0x558e92cf82d0 2023-07-21T18:43:45.978 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.951+0000 7fc098b22700 1 --2- 172.21.15.39:0/1174212415 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e92cf82d0 0x558e92d602a0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fc0840a1ec0 tx=0x7fc0840999d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e586a72fc07d5fcd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:45.978 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.951+0000 7fc073fff700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:43:45.978 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.951+0000 7fc073fff700 1 -- 172.21.15.39:0/1174212415 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0840962e0 con 0x558e92cf82d0 2023-07-21T18:43:45.978 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.952+0000 7fc0a6f0cb80 1 -- 172.21.15.39:0/1174212415 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc084010880 con 0x558e92cf82d0 2023-07-21T18:43:45.978 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.952+0000 7fc073fff700 1 -- 172.21.15.39:0/1174212415 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fc084099fa0 con 0x558e92cf82d0 2023-07-21T18:43:45.978 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.952+0000 7fc073fff700 1 -- 172.21.15.39:0/1174212415 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0840ca290 con 0x558e92cf82d0 2023-07-21T18:43:45.978 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.952+0000 7fc0a6f0cb80 10 client.16191 fetch_fsmap learned FSMap version 33 2023-07-21T18:43:45.978 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.952+0000 7fc0a6f0cb80 1 -- 172.21.15.39:0/1174212415 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x558e92a61c60 con 0x558e92cf82d0 2023-07-21T18:43:45.979 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.952+0000 7fc073fff700 1 -- 172.21.15.39:0/1174212415 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc08407c080 con 0x558e92cf82d0 2023-07-21T18:43:45.979 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.952+0000 7fc073fff700 1 -- 172.21.15.39:0/1174212415 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fc08409ad80 con 0x558e92cf82d0 2023-07-21T18:43:45.979 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.952+0000 7fc0a6f0cb80 10 client.16191 fetch_fsmap finished waiting for FSMap version 33 2023-07-21T18:43:45.979 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.952+0000 7fc0a6f0cb80 10 client.16191 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2023-07-21T18:43:45.979 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.952+0000 7fc0a6f0cb80 20 client.16191 populate_metadata read hostname 'smithi039' 2023-07-21T18:43:45.979 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.952+0000 7fc0a6f0cb80 1 --2- 172.21.15.39:0/1174212415 >> [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] conn(0x558e92d67d40 0x558e92d6a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:45.979 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.952+0000 7fc0a6f0cb80 4 client.16191 mds_command: new command op to 14604 tid=0[{"prefix": "damage ls"}] 2023-07-21T18:43:45.979 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.952+0000 7fc0a6f0cb80 1 -- 172.21.15.39:0/1174212415 --> [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558e92c06050 con 0x558e92d67d40 2023-07-21T18:43:45.980 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.952+0000 7fc093fff700 1 --2- 172.21.15.39:0/1174212415 >> [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] conn(0x558e92d67d40 0x558e92d6a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:45.980 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.953+0000 7fc073fff700 10 client.16191 ms_handle_connect on v2:172.21.15.39:6840/640981220 2023-07-21T18:43:45.980 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.953+0000 7fc093fff700 1 --2- 172.21.15.39:0/1174212415 >> [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] conn(0x558e92d67d40 0x558e92d6a120 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4701cc9d723b0d76 server_cookie=57a0a52808f4960b in_seq=0 out_seq=0 2023-07-21T18:43:45.980 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.953+0000 7fc073fff700 1 -- 172.21.15.39:0/1174212415 <== mds.4 v2:172.21.15.39:6840/640981220 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+241 (crc 0 0 0) 0x558e92c06050 con 0x558e92d67d40 2023-07-21T18:43:45.980 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.953+0000 7fc073fff700 10 client.16191 handle_command_reply: tid=0 2023-07-21T18:43:45.980 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.953+0000 7fc0a6f0cb80 1 client.16191 shutdown 2023-07-21T18:43:45.980 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.953+0000 7fc0a6f0cb80 1 -- 172.21.15.39:0/1174212415 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e92cf82d0 msgr2=0x558e92d602a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:45.981 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.953+0000 7fc0a6f0cb80 1 --2- 172.21.15.39:0/1174212415 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e92cf82d0 0x558e92d602a0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fc0840a1ec0 tx=0x7fc0840999d0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.981 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.953+0000 7fc0a6f0cb80 1 -- 172.21.15.39:0/1174212415 shutdown_connections 2023-07-21T18:43:45.981 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.953+0000 7fc0a6f0cb80 1 --2- 172.21.15.39:0/1174212415 >> [v2:172.21.15.39:6840/640981220,v1:172.21.15.39:6841/640981220] conn(0x558e92d67d40 0x558e92d6a120 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.981 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.953+0000 7fc0a6f0cb80 1 --2- 172.21.15.39:0/1174212415 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e92d03ad0 0x558e92d5f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.981 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.953+0000 7fc0a6f0cb80 1 --2- 172.21.15.39:0/1174212415 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e92cf82d0 0x558e92d602a0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.981 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.953+0000 7fc073fff700 0 client.16191 ms_handle_reset on v2:172.21.15.39:6840/640981220 2023-07-21T18:43:45.981 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.953+0000 7fc0a6f0cb80 1 --2- 172.21.15.39:0/1174212415 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e92cf8be0 0x558e92d607e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.981 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.953+0000 7fc0a6f0cb80 1 -- 172.21.15.39:0/1174212415 >> 172.21.15.39:0/1174212415 conn(0x558e92ceeda0 msgr2=0x558e92cfbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:45.982 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.953+0000 7fc0a6f0cb80 1 -- 172.21.15.39:0/1174212415 shutdown_connections 2023-07-21T18:43:45.982 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.953+0000 7fc0a6f0cb80 1 -- 172.21.15.39:0/1174212415 wait complete. 2023-07-21T18:43:45.982 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.953+0000 7fc0a6f0cb80 20 client.16191 trim_cache size 0 max 16384 2023-07-21T18:43:45.982 INFO:teuthology.orchestra.run.smithi039.stdout:[ 2023-07-21T18:43:45.982 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T18:43:45.982 INFO:teuthology.orchestra.run.smithi039.stdout: "damage_type": "backtrace", 2023-07-21T18:43:45.983 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 284680805, 2023-07-21T18:43:45.983 INFO:teuthology.orchestra.run.smithi039.stdout: "ino": 1099511628351, 2023-07-21T18:43:45.983 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/ACCESS/FASTENER.MDB" 2023-07-21T18:43:45.983 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T18:43:45.983 INFO:teuthology.orchestra.run.smithi039.stdout:] 2023-07-21T18:43:45.984 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.954+0000 7fc09ad86700 1 -- 172.21.15.39:0/2407706353 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fc07c0a12c0 msgr2=0x7fc07c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:45.984 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.954+0000 7fc09ad86700 1 --2- 172.21.15.39:0/2407706353 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fc07c0a12c0 0x7fc07c0a3770 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc094100140 tx=0x7fc088009260 comp rx=0 tx=0).stop 2023-07-21T18:43:45.984 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.954+0000 7fc09ad86700 1 -- 172.21.15.39:0/2407706353 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc094100d00 msgr2=0x7fc0940fe950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:45.984 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.954+0000 7fc09ad86700 1 --2- 172.21.15.39:0/2407706353 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc094100d00 0x7fc0940fe950 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fc08400e9a0 tx=0x7fc08400ecb0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.984 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.954+0000 7fc09ad86700 1 -- 172.21.15.39:0/2407706353 shutdown_connections 2023-07-21T18:43:45.984 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.954+0000 7fc09ad86700 1 --2- 172.21.15.39:0/2407706353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc09410a040 0x7fc0941a4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.985 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.954+0000 7fc09ad86700 1 --2- 172.21.15.39:0/2407706353 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc094100d00 0x7fc0940fe950 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.985 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.954+0000 7fc09ad86700 1 --2- 172.21.15.39:0/2407706353 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fc07c0a12c0 0x7fc07c0a3770 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.985 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.954+0000 7fc09ad86700 1 --2- 172.21.15.39:0/2407706353 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc094101610 0x7fc0940fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:45.985 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.954+0000 7fc09ad86700 1 -- 172.21.15.39:0/2407706353 >> 172.21.15.39:0/2407706353 conn(0x7fc0940fa4b0 msgr2=0x7fc09410c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:45.985 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.954+0000 7fc09ad86700 1 -- 172.21.15.39:0/2407706353 shutdown_connections 2023-07-21T18:43:45.985 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:45.954+0000 7fc09ad86700 1 -- 172.21.15.39:0/2407706353 wait complete. 2023-07-21T18:43:45.986 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-07-21T18:43:46.117 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.098+0000 7f772d59b700 1 -- 172.21.15.39:0/3351208672 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f77280fff20 msgr2=0x7f7728103600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:46.118 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.098+0000 7f772d59b700 1 --2- 172.21.15.39:0/3351208672 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f77280fff20 0x7f7728103600 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f77100099c0 tx=0x7f7710009cd0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.118 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.098+0000 7f772d59b700 1 -- 172.21.15.39:0/3351208672 shutdown_connections 2023-07-21T18:43:46.118 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.098+0000 7f772d59b700 1 --2- 172.21.15.39:0/3351208672 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77280ff580 0x7f77280ff950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.118 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.098+0000 7f772d59b700 1 --2- 172.21.15.39:0/3351208672 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7728103ce0 0x7f772810b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.118 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.098+0000 7f772d59b700 1 --2- 172.21.15.39:0/3351208672 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f77280fff20 0x7f7728103600 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.118 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.098+0000 7f772d59b700 1 -- 172.21.15.39:0/3351208672 >> 172.21.15.39:0/3351208672 conn(0x7f7728076200 msgr2=0x7f7728076600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:46.119 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.098+0000 7f772d59b700 1 -- 172.21.15.39:0/3351208672 shutdown_connections 2023-07-21T18:43:46.119 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.098+0000 7f772d59b700 1 -- 172.21.15.39:0/3351208672 wait complete. 2023-07-21T18:43:46.120 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.099+0000 7f772d59b700 1 Processor -- start 2023-07-21T18:43:46.120 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.099+0000 7f772d59b700 1 -- start start 2023-07-21T18:43:46.121 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.099+0000 7f772d59b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f77280ff580 0x7f7728197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.121 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.099+0000 7f772d59b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77280fff20 0x7f77281983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.121 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.099+0000 7f772d59b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7728103ce0 0x7f772819c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.121 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.099+0000 7f772d59b700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f772819ce90 con 0x7f77280ff580 2023-07-21T18:43:46.121 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.099+0000 7f772d59b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f772819d000 con 0x7f77280fff20 2023-07-21T18:43:46.121 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.099+0000 7f772d59b700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f772819d1c0 con 0x7f7728103ce0 2023-07-21T18:43:46.121 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.099+0000 7f7726ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f77280ff580 0x7f7728197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.121 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.099+0000 7f7726ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f77280ff580 0x7f7728197eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59028/0 (socket says 172.21.15.39:59028) 2023-07-21T18:43:46.122 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.099+0000 7f7726ffd700 1 -- 172.21.15.39:0/129089906 learned_addr learned my addr 172.21.15.39:0/129089906 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:46.122 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.099+0000 7f771ffff700 1 --2- 172.21.15.39:0/129089906 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77280fff20 0x7f77281983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.122 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.099+0000 7f77277fe700 1 --2- 172.21.15.39:0/129089906 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7728103ce0 0x7f772819c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.122 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.100+0000 7f7726ffd700 1 -- 172.21.15.39:0/129089906 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7728103ce0 msgr2=0x7f772819c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:46.122 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.100+0000 7f7726ffd700 1 --2- 172.21.15.39:0/129089906 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7728103ce0 0x7f772819c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.122 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.100+0000 7f7726ffd700 1 -- 172.21.15.39:0/129089906 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77280fff20 msgr2=0x7f77281983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:46.122 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.100+0000 7f7726ffd700 1 --2- 172.21.15.39:0/129089906 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77280fff20 0x7f77281983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.122 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.100+0000 7f7726ffd700 1 -- 172.21.15.39:0/129089906 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77100095f0 con 0x7f77280ff580 2023-07-21T18:43:46.123 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.100+0000 7f77277fe700 1 --2- 172.21.15.39:0/129089906 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7728103ce0 0x7f772819c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:43:46.123 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.100+0000 7f7726ffd700 1 --2- 172.21.15.39:0/129089906 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f77280ff580 0x7f7728197eb0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f771800e9a0 tx=0x7f771800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:46.123 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.100+0000 7f7724ff9700 1 -- 172.21.15.39:0/129089906 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7718009790 con 0x7f77280ff580 2023-07-21T18:43:46.123 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.100+0000 7f7724ff9700 1 -- 172.21.15.39:0/129089906 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7718010430 con 0x7f77280ff580 2023-07-21T18:43:46.123 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.100+0000 7f7724ff9700 1 -- 172.21.15.39:0/129089906 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7718018680 con 0x7f77280ff580 2023-07-21T18:43:46.123 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.100+0000 7f772d59b700 1 -- 172.21.15.39:0/129089906 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f772819d4a0 con 0x7f77280ff580 2023-07-21T18:43:46.123 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.100+0000 7f772d59b700 1 -- 172.21.15.39:0/129089906 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77281a4ce0 con 0x7f77280ff580 2023-07-21T18:43:46.126 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.102+0000 7f7724ff9700 1 -- 172.21.15.39:0/129089906 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f77180105a0 con 0x7f77280ff580 2023-07-21T18:43:46.127 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.103+0000 7f7724ff9700 1 --2- 172.21.15.39:0/129089906 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f77140a12c0 0x7f77140a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.127 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.103+0000 7f771ffff700 1 --2- 172.21.15.39:0/129089906 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f77140a12c0 0x7f77140a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.127 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.103+0000 7f7724ff9700 1 -- 172.21.15.39:0/129089906 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f7718014070 con 0x7f77280ff580 2023-07-21T18:43:46.127 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.104+0000 7f771ffff700 1 --2- 172.21.15.39:0/129089906 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f77140a12c0 0x7f77140a3770 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f771000b580 tx=0x7f7710000bc0 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:46.127 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.104+0000 7f7739721b80 1 Processor -- start 2023-07-21T18:43:46.127 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.104+0000 7f7739721b80 1 -- start start 2023-07-21T18:43:46.128 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.104+0000 7f7739721b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564c093ab2d0 0x564c093ab6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.128 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.104+0000 7f7739721b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c093abbe0 0x564c093b6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.128 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.104+0000 7f7739721b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564c093b6ad0 0x564c093b8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.128 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.104+0000 7f7739721b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c092b1cb0 con 0x564c093b6ad0 2023-07-21T18:43:46.128 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.104+0000 7f7739721b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c09304a40 con 0x564c093abbe0 2023-07-21T18:43:46.128 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.104+0000 7f7739721b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c092fded0 con 0x564c093ab2d0 2023-07-21T18:43:46.128 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.104+0000 7f7726ffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564c093ab2d0 0x564c093ab6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.128 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.104+0000 7f77277fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564c093b6ad0 0x564c093b8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.129 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.104+0000 7f77277fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564c093b6ad0 0x564c093b8eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59040/0 (socket says 172.21.15.39:59040) 2023-07-21T18:43:46.129 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.104+0000 7f77277fe700 1 -- 172.21.15.39:0/2539081850 learned_addr learned my addr 172.21.15.39:0/2539081850 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:46.129 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.105+0000 7f771ffff700 1 --2- 172.21.15.39:0/2539081850 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c093abbe0 0x564c093b6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.129 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.105+0000 7f77277fe700 1 -- 172.21.15.39:0/2539081850 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564c093ab2d0 msgr2=0x564c093ab6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:46.129 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.105+0000 7f77277fe700 1 --2- 172.21.15.39:0/2539081850 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564c093ab2d0 0x564c093ab6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.129 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.105+0000 7f77277fe700 1 -- 172.21.15.39:0/2539081850 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c093abbe0 msgr2=0x564c093b6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:46.129 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.105+0000 7f77277fe700 1 --2- 172.21.15.39:0/2539081850 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c093abbe0 0x564c093b6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.129 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.105+0000 7f77277fe700 1 -- 172.21.15.39:0/2539081850 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c0912bde0 con 0x564c093b6ad0 2023-07-21T18:43:46.130 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.105+0000 7f7726ffd700 1 --2- 172.21.15.39:0/2539081850 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564c093ab2d0 0x564c093ab6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:43:46.130 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.105+0000 7f77277fe700 1 --2- 172.21.15.39:0/2539081850 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564c093b6ad0 0x564c093b8eb0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f7720003eb0 tx=0x7f7720009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e451c17fd5a9e9b0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:46.130 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.105+0000 7f771d7fa700 1 -- 172.21.15.39:0/2539081850 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77200092d0 con 0x564c093b6ad0 2023-07-21T18:43:46.130 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.105+0000 7f771d7fa700 1 -- 172.21.15.39:0/2539081850 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f772000f040 con 0x564c093b6ad0 2023-07-21T18:43:46.130 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.105+0000 7f771d7fa700 1 -- 172.21.15.39:0/2539081850 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772000e750 con 0x564c093b6ad0 2023-07-21T18:43:46.130 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.106+0000 7f7739721b80 1 -- 172.21.15.39:0/2539081850 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564c093b6ad0 msgr2=0x564c093b8eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:46.130 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.106+0000 7f7739721b80 1 --2- 172.21.15.39:0/2539081850 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564c093b6ad0 0x564c093b8eb0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f7720003eb0 tx=0x7f7720009920 comp rx=0 tx=0).stop 2023-07-21T18:43:46.131 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.106+0000 7f7739721b80 1 -- 172.21.15.39:0/2539081850 shutdown_connections 2023-07-21T18:43:46.131 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.106+0000 7f7739721b80 1 --2- 172.21.15.39:0/2539081850 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564c093b6ad0 0x564c093b8eb0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.131 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.106+0000 7f7739721b80 1 --2- 172.21.15.39:0/2539081850 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564c093ab2d0 0x564c093ab6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.131 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.106+0000 7f7739721b80 1 --2- 172.21.15.39:0/2539081850 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c093abbe0 0x564c093b6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.131 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.106+0000 7f7739721b80 1 -- 172.21.15.39:0/2539081850 >> 172.21.15.39:0/2539081850 conn(0x564c093a1da0 msgr2=0x564c093a76a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:46.131 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.106+0000 7f7739721b80 1 -- 172.21.15.39:0/2539081850 shutdown_connections 2023-07-21T18:43:46.131 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.106+0000 7f7739721b80 1 -- 172.21.15.39:0/2539081850 wait complete. 2023-07-21T18:43:46.131 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.106+0000 7f7739721b80 1 -- start start 2023-07-21T18:43:46.132 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.106+0000 7f7739721b80 1 Processor -- start 2023-07-21T18:43:46.132 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.107+0000 7f7739721b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564c093ab2d0 0x564c093b6520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.132 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.107+0000 7f7739721b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c093abbe0 0x564c093b3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.132 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.107+0000 7f7739721b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564c093b6ad0 0x564c093b3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.132 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.107+0000 7f7739721b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c0929dac0 con 0x564c093ab2d0 2023-07-21T18:43:46.132 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.107+0000 7f7739721b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c0929d7c0 con 0x564c093abbe0 2023-07-21T18:43:46.132 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.107+0000 7f7739721b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c09181920 con 0x564c093b6ad0 2023-07-21T18:43:46.132 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.107+0000 7f7739721b80 10 client.0 _ll_register_callbacks cb 0x564c0929b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:43:46.133 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.107+0000 7f7726ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564c093ab2d0 0x564c093b6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.133 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.107+0000 7f7726ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564c093ab2d0 0x564c093b6520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59042/0 (socket says 172.21.15.39:59042) 2023-07-21T18:43:46.133 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.107+0000 7f7726ffd700 1 -- 172.21.15.39:0/2179118649 learned_addr learned my addr 172.21.15.39:0/2179118649 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:46.133 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.107+0000 7f77277fe700 1 --2- 172.21.15.39:0/2179118649 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564c093b6ad0 0x564c093b3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.133 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.107+0000 7f771ffff700 1 --2- 172.21.15.39:0/2179118649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c093abbe0 0x564c093b3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.133 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.107+0000 7f7726ffd700 1 -- 172.21.15.39:0/2179118649 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564c093b6ad0 msgr2=0x564c093b3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:46.133 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.107+0000 7f7726ffd700 1 --2- 172.21.15.39:0/2179118649 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564c093b6ad0 0x564c093b3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.134 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.107+0000 7f7726ffd700 1 -- 172.21.15.39:0/2179118649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c093abbe0 msgr2=0x564c093b3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:46.134 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.107+0000 7f7726ffd700 1 --2- 172.21.15.39:0/2179118649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c093abbe0 0x564c093b3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.134 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.107+0000 7f7726ffd700 1 -- 172.21.15.39:0/2179118649 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77281a4ce0 con 0x564c093ab2d0 2023-07-21T18:43:46.134 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.107+0000 7f77277fe700 1 --2- 172.21.15.39:0/2179118649 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564c093b6ad0 0x564c093b3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:43:46.134 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.107+0000 7f7726ffd700 1 --2- 172.21.15.39:0/2179118649 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564c093ab2d0 0x564c093b6520 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f7718099cb0 tx=0x7f771809a030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93102cd0499e082a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:46.134 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.107+0000 7f770e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:43:46.134 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.107+0000 7f770e7fc700 1 -- 172.21.15.39:0/2179118649 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77180996a0 con 0x564c093ab2d0 2023-07-21T18:43:46.134 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.107+0000 7f770e7fc700 1 -- 172.21.15.39:0/2179118649 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f77180a2400 con 0x564c093ab2d0 2023-07-21T18:43:46.135 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.107+0000 7f7739721b80 1 -- 172.21.15.39:0/2179118649 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564c092574f0 con 0x564c093ab2d0 2023-07-21T18:43:46.135 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.108+0000 7f770e7fc700 1 -- 172.21.15.39:0/2179118649 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77180a1f70 con 0x564c093ab2d0 2023-07-21T18:43:46.135 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.108+0000 7f7739721b80 10 client.16203 fetch_fsmap learned FSMap version 33 2023-07-21T18:43:46.135 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.108+0000 7f7739721b80 1 -- 172.21.15.39:0/2179118649 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7f7720003b90 con 0x564c093ab2d0 2023-07-21T18:43:46.135 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.108+0000 7f770e7fc700 1 -- 172.21.15.39:0/2179118649 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564c092574f0 con 0x564c093ab2d0 2023-07-21T18:43:46.136 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.108+0000 7f770e7fc700 1 -- 172.21.15.39:0/2179118649 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f7718098f10 con 0x564c093ab2d0 2023-07-21T18:43:46.136 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.108+0000 7f7739721b80 10 client.16203 fetch_fsmap finished waiting for FSMap version 33 2023-07-21T18:43:46.137 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.108+0000 7f7739721b80 10 client.16203 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2023-07-21T18:43:46.137 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.108+0000 7f7739721b80 20 client.16203 populate_metadata read hostname 'smithi039' 2023-07-21T18:43:46.137 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.108+0000 7f7739721b80 1 --2- 172.21.15.39:0/2179118649 >> [v2:172.21.15.155:6834/3854856286,v1:172.21.15.155:6835/3854856286] conn(0x564c093c63c0 0x564c093c87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.137 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.108+0000 7f7739721b80 4 client.16203 mds_command: new command op to 24464 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:43:46.137 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.108+0000 7f7739721b80 1 -- 172.21.15.39:0/2179118649 --> [v2:172.21.15.155:6834/3854856286,v1:172.21.15.155:6835/3854856286] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564c092f6eb0 con 0x564c093c63c0 2023-07-21T18:43:46.137 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.108+0000 7f771ffff700 1 --2- 172.21.15.39:0/2179118649 >> [v2:172.21.15.155:6834/3854856286,v1:172.21.15.155:6835/3854856286] conn(0x564c093c63c0 0x564c093c87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.137 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.109+0000 7f770e7fc700 10 client.16203 ms_handle_connect on v2:172.21.15.155:6834/3854856286 2023-07-21T18:43:46.138 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.109+0000 7f771ffff700 1 --2- 172.21.15.39:0/2179118649 >> [v2:172.21.15.155:6834/3854856286,v1:172.21.15.155:6835/3854856286] conn(0x564c093c63c0 0x564c093c87a0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=127da2b722950284 server_cookie=eba96486cd6ba63e in_seq=0 out_seq=0 2023-07-21T18:43:46.138 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.109+0000 7f770e7fc700 1 -- 172.21.15.39:0/2179118649 <== mds.3 v2:172.21.15.155:6834/3854856286 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564c092f6eb0 con 0x564c093c63c0 2023-07-21T18:43:46.138 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.109+0000 7f770e7fc700 10 client.16203 handle_command_reply: tid=0 2023-07-21T18:43:46.138 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.109+0000 7f7739721b80 1 client.16203 shutdown 2023-07-21T18:43:46.138 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.109+0000 7f7739721b80 1 -- 172.21.15.39:0/2179118649 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564c093ab2d0 msgr2=0x564c093b6520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:46.138 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.109+0000 7f7739721b80 1 --2- 172.21.15.39:0/2179118649 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564c093ab2d0 0x564c093b6520 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f7718099cb0 tx=0x7f771809a030 comp rx=0 tx=0).stop 2023-07-21T18:43:46.138 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.109+0000 7f7739721b80 1 -- 172.21.15.39:0/2179118649 shutdown_connections 2023-07-21T18:43:46.138 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.109+0000 7f7739721b80 1 --2- 172.21.15.39:0/2179118649 >> [v2:172.21.15.155:6834/3854856286,v1:172.21.15.155:6835/3854856286] conn(0x564c093c63c0 0x564c093c87a0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.139 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.109+0000 7f7739721b80 1 --2- 172.21.15.39:0/2179118649 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564c093b6ad0 0x564c093b3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.139 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.109+0000 7f7739721b80 1 --2- 172.21.15.39:0/2179118649 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564c093ab2d0 0x564c093b6520 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.139 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.109+0000 7f770e7fc700 0 client.16203 ms_handle_reset on v2:172.21.15.155:6834/3854856286 2023-07-21T18:43:46.139 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.109+0000 7f7739721b80 1 --2- 172.21.15.39:0/2179118649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c093abbe0 0x564c093b3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.139 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.109+0000 7f7739721b80 1 -- 172.21.15.39:0/2179118649 >> 172.21.15.39:0/2179118649 conn(0x564c093a1da0 msgr2=0x564c093a7290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:46.139 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.110+0000 7f7739721b80 1 -- 172.21.15.39:0/2179118649 shutdown_connections 2023-07-21T18:43:46.139 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.110+0000 7f7739721b80 1 -- 172.21.15.39:0/2179118649 wait complete. 2023-07-21T18:43:46.139 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.110+0000 7f7739721b80 20 client.16203 trim_cache size 0 max 16384 2023-07-21T18:43:46.142 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.123+0000 7f7739721b80 1 Processor -- start 2023-07-21T18:43:46.142 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.123+0000 7f7739721b80 1 -- start start 2023-07-21T18:43:46.143 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.123+0000 7f7739721b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564c093ab2d0 0x564c093b6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.143 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.123+0000 7f7739721b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c093abbe0 0x564c093b2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.144 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.123+0000 7f7739721b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564c093b6ad0 0x564c093b3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.144 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.123+0000 7f7739721b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c093a3ec0 con 0x564c093ab2d0 2023-07-21T18:43:46.144 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.124+0000 7f7739721b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c093ac540 con 0x564c093abbe0 2023-07-21T18:43:46.144 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.124+0000 7f7739721b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c093ae530 con 0x564c093b6ad0 2023-07-21T18:43:46.144 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.124+0000 7f7726ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564c093ab2d0 0x564c093b6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.144 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.124+0000 7f7726ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564c093ab2d0 0x564c093b6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59050/0 (socket says 172.21.15.39:59050) 2023-07-21T18:43:46.144 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.124+0000 7f7726ffd700 1 -- 172.21.15.39:0/1314334210 learned_addr learned my addr 172.21.15.39:0/1314334210 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:46.145 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.124+0000 7f771ffff700 1 --2- 172.21.15.39:0/1314334210 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c093abbe0 0x564c093b2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.145 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.124+0000 7f77277fe700 1 --2- 172.21.15.39:0/1314334210 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564c093b6ad0 0x564c093b3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.145 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.124+0000 7f7726ffd700 1 -- 172.21.15.39:0/1314334210 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564c093b6ad0 msgr2=0x564c093b3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:46.145 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.124+0000 7f7726ffd700 1 --2- 172.21.15.39:0/1314334210 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564c093b6ad0 0x564c093b3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.145 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.124+0000 7f7726ffd700 1 -- 172.21.15.39:0/1314334210 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c093abbe0 msgr2=0x564c093b2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:46.145 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.124+0000 7f7726ffd700 1 --2- 172.21.15.39:0/1314334210 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c093abbe0 0x564c093b2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.145 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.124+0000 7f7726ffd700 1 -- 172.21.15.39:0/1314334210 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c09114c60 con 0x564c093ab2d0 2023-07-21T18:43:46.145 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.124+0000 7f7726ffd700 1 --2- 172.21.15.39:0/1314334210 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564c093ab2d0 0x564c093b6050 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f77180964b0 tx=0x7f7718010a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7faf787bb394149 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:46.146 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.124+0000 7f771dffb700 1 -- 172.21.15.39:0/1314334210 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77180a2400 con 0x564c093ab2d0 2023-07-21T18:43:46.146 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.124+0000 7f771dffb700 1 -- 172.21.15.39:0/1314334210 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7718098f10 con 0x564c093ab2d0 2023-07-21T18:43:46.146 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.124+0000 7f771dffb700 1 -- 172.21.15.39:0/1314334210 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7718099700 con 0x564c093ab2d0 2023-07-21T18:43:46.149 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.125+0000 7f7739721b80 1 -- 172.21.15.39:0/1314334210 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564c093ab2d0 msgr2=0x564c093b6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:46.149 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.125+0000 7f7739721b80 1 --2- 172.21.15.39:0/1314334210 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564c093ab2d0 0x564c093b6050 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f77180964b0 tx=0x7f7718010a90 comp rx=0 tx=0).stop 2023-07-21T18:43:46.149 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.125+0000 7f7739721b80 1 -- 172.21.15.39:0/1314334210 shutdown_connections 2023-07-21T18:43:46.149 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.125+0000 7f7739721b80 1 --2- 172.21.15.39:0/1314334210 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564c093b6ad0 0x564c093b3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.149 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.125+0000 7f7739721b80 1 --2- 172.21.15.39:0/1314334210 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564c093ab2d0 0x564c093b6050 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.150 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.125+0000 7f7739721b80 1 --2- 172.21.15.39:0/1314334210 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c093abbe0 0x564c093b2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.150 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.125+0000 7f7739721b80 1 -- 172.21.15.39:0/1314334210 >> 172.21.15.39:0/1314334210 conn(0x564c093a1da0 msgr2=0x564c093ae9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:46.150 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.125+0000 7f7739721b80 1 -- 172.21.15.39:0/1314334210 shutdown_connections 2023-07-21T18:43:46.150 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.125+0000 7f7739721b80 1 -- 172.21.15.39:0/1314334210 wait complete. 2023-07-21T18:43:46.150 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.125+0000 7f7739721b80 1 -- start start 2023-07-21T18:43:46.150 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.126+0000 7f7739721b80 1 Processor -- start 2023-07-21T18:43:46.150 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.126+0000 7f7739721b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564c093ab2d0 0x564c093b9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.151 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.126+0000 7f7739721b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564c093abbe0 0x564c093b9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.151 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.126+0000 7f7739721b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c093b6ad0 0x564c094131e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.151 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.126+0000 7f7739721b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c09413760 con 0x564c093ab2d0 2023-07-21T18:43:46.151 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.126+0000 7f7739721b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c094138d0 con 0x564c093b6ad0 2023-07-21T18:43:46.151 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.126+0000 7f7739721b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c09413a90 con 0x564c093abbe0 2023-07-21T18:43:46.151 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.126+0000 7f7739721b80 10 client.0 _ll_register_callbacks cb 0x7f7718099910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:43:46.151 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.126+0000 7f7726ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564c093ab2d0 0x564c093b9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.151 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.126+0000 7f771ffff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564c093abbe0 0x564c093b9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.152 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.126+0000 7f77277fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c093b6ad0 0x564c094131e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.152 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.126+0000 7f771ffff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564c093abbe0 0x564c093b9c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:49378/0 (socket says 172.21.15.39:49378) 2023-07-21T18:43:46.152 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.126+0000 7f7726ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564c093ab2d0 0x564c093b9700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59056/0 (socket says 172.21.15.39:59056) 2023-07-21T18:43:46.152 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.126+0000 7f771ffff700 1 -- 172.21.15.39:0/3523011247 learned_addr learned my addr 172.21.15.39:0/3523011247 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:46.152 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.126+0000 7f77277fe700 1 -- 172.21.15.39:0/3523011247 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564c093abbe0 msgr2=0x564c093b9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:46.152 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.126+0000 7f77277fe700 1 --2- 172.21.15.39:0/3523011247 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564c093abbe0 0x564c093b9c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.152 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.126+0000 7f77277fe700 1 -- 172.21.15.39:0/3523011247 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564c093ab2d0 msgr2=0x564c093b9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:46.152 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.126+0000 7f77277fe700 1 --2- 172.21.15.39:0/3523011247 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564c093ab2d0 0x564c093b9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.153 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.126+0000 7f77277fe700 1 -- 172.21.15.39:0/3523011247 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7720003b90 con 0x564c093b6ad0 2023-07-21T18:43:46.153 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.127+0000 7f77277fe700 1 --2- 172.21.15.39:0/3523011247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c093b6ad0 0x564c094131e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7720007da0 tx=0x7f77200060e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22b217b8299eaa13 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:46.153 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.127+0000 7f770e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-21T18:43:46.153 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.127+0000 7f770e7fc700 1 -- 172.21.15.39:0/3523011247 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7720007a10 con 0x564c093b6ad0 2023-07-21T18:43:46.153 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.127+0000 7f7739721b80 1 -- 172.21.15.39:0/3523011247 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f772000f040 con 0x564c093b6ad0 2023-07-21T18:43:46.153 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.127+0000 7f770e7fc700 1 -- 172.21.15.39:0/3523011247 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f7720007b70 con 0x564c093b6ad0 2023-07-21T18:43:46.153 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.127+0000 7f770e7fc700 1 -- 172.21.15.39:0/3523011247 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f772000e4f0 con 0x564c093b6ad0 2023-07-21T18:43:46.153 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.127+0000 7f7739721b80 10 client.25718 fetch_fsmap learned FSMap version 33 2023-07-21T18:43:46.154 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.127+0000 7f7739721b80 1 -- 172.21.15.39:0/3523011247 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x564c09152250 con 0x564c093b6ad0 2023-07-21T18:43:46.154 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.127+0000 7f770e7fc700 1 -- 172.21.15.39:0/3523011247 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f772000f040 con 0x564c093b6ad0 2023-07-21T18:43:46.154 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.127+0000 7f770e7fc700 1 -- 172.21.15.39:0/3523011247 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f772002f760 con 0x564c093b6ad0 2023-07-21T18:43:46.154 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.127+0000 7f7739721b80 10 client.25718 fetch_fsmap finished waiting for FSMap version 33 2023-07-21T18:43:46.154 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.127+0000 7f7739721b80 10 client.25718 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2023-07-21T18:43:46.154 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.127+0000 7f7739721b80 20 client.25718 populate_metadata read hostname 'smithi039' 2023-07-21T18:43:46.154 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.127+0000 7f7739721b80 1 --2- 172.21.15.39:0/3523011247 >> [v2:172.21.15.155:6834/3854856286,v1:172.21.15.155:6835/3854856286] conn(0x564c0941b1c0 0x564c0941d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.154 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.127+0000 7f7739721b80 4 client.25718 mds_command: new command op to 24464 tid=0[{"prefix": "damage ls"}] 2023-07-21T18:43:46.155 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.127+0000 7f7739721b80 1 -- 172.21.15.39:0/3523011247 --> [v2:172.21.15.155:6834/3854856286,v1:172.21.15.155:6835/3854856286] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564c092b9050 con 0x564c0941b1c0 2023-07-21T18:43:46.155 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.128+0000 7f7726ffd700 1 --2- 172.21.15.39:0/3523011247 >> [v2:172.21.15.155:6834/3854856286,v1:172.21.15.155:6835/3854856286] conn(0x564c0941b1c0 0x564c0941d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.155 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.128+0000 7f770e7fc700 10 client.25718 ms_handle_connect on v2:172.21.15.155:6834/3854856286 2023-07-21T18:43:46.155 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.128+0000 7f7726ffd700 1 --2- 172.21.15.39:0/3523011247 >> [v2:172.21.15.155:6834/3854856286,v1:172.21.15.155:6835/3854856286] conn(0x564c0941b1c0 0x564c0941d5d0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=49f3a67c0ea0e64c server_cookie=e44980781a0046fd in_seq=0 out_seq=0 2023-07-21T18:43:46.155 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.128+0000 7f770e7fc700 1 -- 172.21.15.39:0/3523011247 <== mds.3 v2:172.21.15.155:6834/3854856286 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564c092b9050 con 0x564c0941b1c0 2023-07-21T18:43:46.155 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.128+0000 7f770e7fc700 10 client.25718 handle_command_reply: tid=0 2023-07-21T18:43:46.155 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.128+0000 7f7739721b80 1 client.25718 shutdown 2023-07-21T18:43:46.156 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.128+0000 7f7739721b80 1 -- 172.21.15.39:0/3523011247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c093b6ad0 msgr2=0x564c094131e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:46.156 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.128+0000 7f7739721b80 1 --2- 172.21.15.39:0/3523011247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c093b6ad0 0x564c094131e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7720007da0 tx=0x7f77200060e0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.156 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.128+0000 7f7739721b80 1 -- 172.21.15.39:0/3523011247 shutdown_connections 2023-07-21T18:43:46.156 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.128+0000 7f7739721b80 1 --2- 172.21.15.39:0/3523011247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c093b6ad0 0x564c094131e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.156 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.128+0000 7f7739721b80 1 --2- 172.21.15.39:0/3523011247 >> [v2:172.21.15.155:6834/3854856286,v1:172.21.15.155:6835/3854856286] conn(0x564c0941b1c0 0x564c0941d5d0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.156 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.128+0000 7f7739721b80 1 --2- 172.21.15.39:0/3523011247 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564c093ab2d0 0x564c093b9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.156 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.128+0000 7f7739721b80 1 --2- 172.21.15.39:0/3523011247 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564c093abbe0 0x564c093b9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.156 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.128+0000 7f770e7fc700 0 client.25718 ms_handle_reset on v2:172.21.15.155:6834/3854856286 2023-07-21T18:43:46.157 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.128+0000 7f7739721b80 1 -- 172.21.15.39:0/3523011247 >> 172.21.15.39:0/3523011247 conn(0x564c093a1da0 msgr2=0x564c093ad3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:46.157 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.128+0000 7f7739721b80 1 -- 172.21.15.39:0/3523011247 shutdown_connections 2023-07-21T18:43:46.157 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.129+0000 7f7739721b80 1 -- 172.21.15.39:0/3523011247 wait complete. 2023-07-21T18:43:46.157 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.129+0000 7f7739721b80 20 client.25718 trim_cache size 0 max 16384 2023-07-21T18:43:46.157 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2023-07-21T18:43:46.158 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.129+0000 7f772d59b700 1 -- 172.21.15.39:0/129089906 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f77140a12c0 msgr2=0x7f77140a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:46.158 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.129+0000 7f772d59b700 1 --2- 172.21.15.39:0/129089906 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f77140a12c0 0x7f77140a3770 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f771000b580 tx=0x7f7710000bc0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.158 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.129+0000 7f772d59b700 1 -- 172.21.15.39:0/129089906 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f77280ff580 msgr2=0x7f7728197eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:46.158 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.129+0000 7f772d59b700 1 --2- 172.21.15.39:0/129089906 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f77280ff580 0x7f7728197eb0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f771800e9a0 tx=0x7f771800ecb0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.158 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.129+0000 7f772d59b700 1 -- 172.21.15.39:0/129089906 shutdown_connections 2023-07-21T18:43:46.159 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.129+0000 7f772d59b700 1 --2- 172.21.15.39:0/129089906 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7728103ce0 0x7f772819c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.159 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.129+0000 7f772d59b700 1 --2- 172.21.15.39:0/129089906 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f77280ff580 0x7f7728197eb0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.159 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.129+0000 7f772d59b700 1 --2- 172.21.15.39:0/129089906 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f77140a12c0 0x7f77140a3770 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.159 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.129+0000 7f772d59b700 1 --2- 172.21.15.39:0/129089906 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77280fff20 0x7f77281983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.159 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.129+0000 7f772d59b700 1 -- 172.21.15.39:0/129089906 >> 172.21.15.39:0/129089906 conn(0x7f7728076200 msgr2=0x7f77280fc480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:46.159 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.129+0000 7f772d59b700 1 -- 172.21.15.39:0/129089906 shutdown_connections 2023-07-21T18:43:46.159 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.129+0000 7f772d59b700 1 -- 172.21.15.39:0/129089906 wait complete. 2023-07-21T18:43:46.160 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:46 smithi039 ceph-mon[96489]: pgmap v63: 97 pgs: 97 active+clean; 29 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 167 KiB/s wr, 46 op/s 2023-07-21T18:43:46.160 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:46 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/518618751' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:43:46.161 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-21T18:43:46.262 INFO:tasks.workunit.client.0.smithi039.stdout: 1 43193 4.24 MB/sec warmup 59 sec latency 19.016 ms 2023-07-21T18:43:46.293 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.274+0000 7f9d26924700 1 -- 172.21.15.39:0/1046310050 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d2010c3d0 msgr2=0x7f9d2010e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:46.293 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.274+0000 7f9d26924700 1 --2- 172.21.15.39:0/1046310050 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d2010c3d0 0x7f9d2010e7b0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f9d1800b0a0 tx=0x7f9d1800b3b0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.293 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.274+0000 7f9d26924700 1 -- 172.21.15.39:0/1046310050 shutdown_connections 2023-07-21T18:43:46.294 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.274+0000 7f9d26924700 1 --2- 172.21.15.39:0/1046310050 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d2010c3d0 0x7f9d2010e7b0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.294 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.274+0000 7f9d26924700 1 --2- 172.21.15.39:0/1046310050 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d200fe820 0x7f9d200febf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.294 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.274+0000 7f9d26924700 1 --2- 172.21.15.39:0/1046310050 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d200ff1c0 0x7f9d2010be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.294 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.274+0000 7f9d26924700 1 -- 172.21.15.39:0/1046310050 >> 172.21.15.39:0/1046310050 conn(0x7f9d200fa4b0 msgr2=0x7f9d200fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:46.294 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.274+0000 7f9d26924700 1 -- 172.21.15.39:0/1046310050 shutdown_connections 2023-07-21T18:43:46.294 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.274+0000 7f9d26924700 1 -- 172.21.15.39:0/1046310050 wait complete. 2023-07-21T18:43:46.296 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.274+0000 7f9d26924700 1 Processor -- start 2023-07-21T18:43:46.296 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.274+0000 7f9d26924700 1 -- start start 2023-07-21T18:43:46.296 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.275+0000 7f9d26924700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d200fe820 0x7f9d20197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.296 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.275+0000 7f9d26924700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d200ff1c0 0x7f9d20198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.296 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.275+0000 7f9d26924700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d2010c3d0 0x7f9d2019c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.296 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.275+0000 7f9d26924700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2019cea0 con 0x7f9d200ff1c0 2023-07-21T18:43:46.297 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.275+0000 7f9d26924700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2019d010 con 0x7f9d2010c3d0 2023-07-21T18:43:46.297 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.275+0000 7f9d26924700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2019d1d0 con 0x7f9d200fe820 2023-07-21T18:43:46.297 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.275+0000 7f9d1f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d200ff1c0 0x7f9d20198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.297 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.275+0000 7f9d1f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d200ff1c0 0x7f9d20198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59072/0 (socket says 172.21.15.39:59072) 2023-07-21T18:43:46.297 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.275+0000 7f9d1f7fe700 1 -- 172.21.15.39:0/1328186909 learned_addr learned my addr 172.21.15.39:0/1328186909 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:46.297 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.275+0000 7f9d24ec1700 1 --2- 172.21.15.39:0/1328186909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d2010c3d0 0x7f9d2019c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.297 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.275+0000 7f9d1ffff700 1 --2- 172.21.15.39:0/1328186909 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d200fe820 0x7f9d20197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.297 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.275+0000 7f9d1f7fe700 1 -- 172.21.15.39:0/1328186909 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d200fe820 msgr2=0x7f9d20197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:46.298 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.275+0000 7f9d1f7fe700 1 --2- 172.21.15.39:0/1328186909 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d200fe820 0x7f9d20197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.298 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.275+0000 7f9d1f7fe700 1 -- 172.21.15.39:0/1328186909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d2010c3d0 msgr2=0x7f9d2019c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:46.298 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.275+0000 7f9d1f7fe700 1 --2- 172.21.15.39:0/1328186909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d2010c3d0 0x7f9d2019c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.298 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.275+0000 7f9d1f7fe700 1 -- 172.21.15.39:0/1328186909 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d18009d00 con 0x7f9d200ff1c0 2023-07-21T18:43:46.298 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.275+0000 7f9d1f7fe700 1 --2- 172.21.15.39:0/1328186909 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d200ff1c0 0x7f9d20198400 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f9d1400e9a0 tx=0x7f9d1400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:46.298 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.276+0000 7f9d1d7fa700 1 -- 172.21.15.39:0/1328186909 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d14009790 con 0x7f9d200ff1c0 2023-07-21T18:43:46.298 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.276+0000 7f9d1d7fa700 1 -- 172.21.15.39:0/1328186909 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d14010430 con 0x7f9d200ff1c0 2023-07-21T18:43:46.298 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.276+0000 7f9d26924700 1 -- 172.21.15.39:0/1328186909 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d2019d4b0 con 0x7f9d200ff1c0 2023-07-21T18:43:46.299 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.276+0000 7f9d1d7fa700 1 -- 172.21.15.39:0/1328186909 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d14018680 con 0x7f9d200ff1c0 2023-07-21T18:43:46.299 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.276+0000 7f9d26924700 1 -- 172.21.15.39:0/1328186909 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d201a4cf0 con 0x7f9d200ff1c0 2023-07-21T18:43:46.301 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.278+0000 7f9d1d7fa700 1 -- 172.21.15.39:0/1328186909 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f9d140105a0 con 0x7f9d200ff1c0 2023-07-21T18:43:46.301 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.279+0000 7f9d1d7fa700 1 --2- 172.21.15.39:0/1328186909 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f9d080a12c0 0x7f9d080a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.301 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.279+0000 7f9d1ffff700 1 --2- 172.21.15.39:0/1328186909 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f9d080a12c0 0x7f9d080a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.302 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.279+0000 7f9d1d7fa700 1 -- 172.21.15.39:0/1328186909 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f9d14014070 con 0x7f9d200ff1c0 2023-07-21T18:43:46.302 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.279+0000 7f9d1ffff700 1 --2- 172.21.15.39:0/1328186909 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f9d080a12c0 0x7f9d080a3770 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9d10009c60 tx=0x7f9d10009260 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:46.302 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.280+0000 7f9d32aaab80 1 Processor -- start 2023-07-21T18:43:46.302 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.280+0000 7f9d32aaab80 1 -- start start 2023-07-21T18:43:46.302 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.280+0000 7f9d32aaab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d011322d0 0x560d011326a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.302 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.280+0000 7f9d32aaab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d01132be0 0x560d0113d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.302 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.280+0000 7f9d32aaab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d0113dad0 0x560d0113feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.303 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.280+0000 7f9d32aaab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d01038cb0 con 0x560d0113dad0 2023-07-21T18:43:46.303 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.280+0000 7f9d32aaab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d0108ba40 con 0x560d011322d0 2023-07-21T18:43:46.303 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.280+0000 7f9d32aaab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d01084ed0 con 0x560d01132be0 2023-07-21T18:43:46.303 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.280+0000 7f9d24ec1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d0113dad0 0x560d0113feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.303 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.280+0000 7f9d24ec1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d0113dad0 0x560d0113feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59082/0 (socket says 172.21.15.39:59082) 2023-07-21T18:43:46.303 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.280+0000 7f9d24ec1700 1 -- 172.21.15.39:0/781664491 learned_addr learned my addr 172.21.15.39:0/781664491 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:46.303 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.280+0000 7f9d1f7fe700 1 --2- 172.21.15.39:0/781664491 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d01132be0 0x560d0113d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.303 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.280+0000 7f9d1ffff700 1 --2- 172.21.15.39:0/781664491 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d011322d0 0x560d011326a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.304 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.281+0000 7f9d24ec1700 1 -- 172.21.15.39:0/781664491 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d01132be0 msgr2=0x560d0113d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:46.304 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.281+0000 7f9d24ec1700 1 --2- 172.21.15.39:0/781664491 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d01132be0 0x560d0113d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.304 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.281+0000 7f9d24ec1700 1 -- 172.21.15.39:0/781664491 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d011322d0 msgr2=0x560d011326a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:46.304 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.281+0000 7f9d24ec1700 1 --2- 172.21.15.39:0/781664491 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d011322d0 0x560d011326a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.304 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.281+0000 7f9d24ec1700 1 -- 172.21.15.39:0/781664491 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560d00eb2de0 con 0x560d0113dad0 2023-07-21T18:43:46.304 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.281+0000 7f9d1f7fe700 1 --2- 172.21.15.39:0/781664491 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d01132be0 0x560d0113d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:43:46.304 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.281+0000 7f9d24ec1700 1 --2- 172.21.15.39:0/781664491 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d0113dad0 0x560d0113feb0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f9d18015000 tx=0x7f9d18009c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=484ce126bd711963 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:46.304 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.281+0000 7f9d067fc700 1 -- 172.21.15.39:0/781664491 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1800e040 con 0x560d0113dad0 2023-07-21T18:43:46.305 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.281+0000 7f9d067fc700 1 -- 172.21.15.39:0/781664491 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d18003a00 con 0x560d0113dad0 2023-07-21T18:43:46.305 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.281+0000 7f9d067fc700 1 -- 172.21.15.39:0/781664491 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1801c640 con 0x560d0113dad0 2023-07-21T18:43:46.305 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.282+0000 7f9d32aaab80 1 -- 172.21.15.39:0/781664491 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d0113dad0 msgr2=0x560d0113feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:46.305 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.282+0000 7f9d32aaab80 1 --2- 172.21.15.39:0/781664491 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d0113dad0 0x560d0113feb0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f9d18015000 tx=0x7f9d18009c40 comp rx=0 tx=0).stop 2023-07-21T18:43:46.305 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.282+0000 7f9d32aaab80 1 -- 172.21.15.39:0/781664491 shutdown_connections 2023-07-21T18:43:46.305 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.282+0000 7f9d32aaab80 1 --2- 172.21.15.39:0/781664491 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d011322d0 0x560d011326a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.305 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.282+0000 7f9d32aaab80 1 --2- 172.21.15.39:0/781664491 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d0113dad0 0x560d0113feb0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.306 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.282+0000 7f9d32aaab80 1 --2- 172.21.15.39:0/781664491 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d01132be0 0x560d0113d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.306 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.282+0000 7f9d32aaab80 1 -- 172.21.15.39:0/781664491 >> 172.21.15.39:0/781664491 conn(0x560d01128da0 msgr2=0x560d0112e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:46.306 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.282+0000 7f9d32aaab80 1 -- 172.21.15.39:0/781664491 shutdown_connections 2023-07-21T18:43:46.306 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.282+0000 7f9d32aaab80 1 -- 172.21.15.39:0/781664491 wait complete. 2023-07-21T18:43:46.306 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.282+0000 7f9d32aaab80 1 -- start start 2023-07-21T18:43:46.306 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.282+0000 7f9d32aaab80 1 Processor -- start 2023-07-21T18:43:46.309 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.283+0000 7f9d32aaab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d011322d0 0x560d0113d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.309 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.283+0000 7f9d32aaab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d01132be0 0x560d0113a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.309 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.283+0000 7f9d32aaab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d0113dad0 0x560d0113ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.309 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.283+0000 7f9d32aaab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d01024ac0 con 0x560d01132be0 2023-07-21T18:43:46.309 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.283+0000 7f9d32aaab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d010247c0 con 0x560d011322d0 2023-07-21T18:43:46.309 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.283+0000 7f9d32aaab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d00f08920 con 0x560d0113dad0 2023-07-21T18:43:46.310 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.283+0000 7f9d32aaab80 10 client.0 _ll_register_callbacks cb 0x560d01022760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:43:46.310 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.283+0000 7f9d1f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d01132be0 0x560d0113a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.310 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.283+0000 7f9d1f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d01132be0 0x560d0113a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59086/0 (socket says 172.21.15.39:59086) 2023-07-21T18:43:46.310 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.283+0000 7f9d1f7fe700 1 -- 172.21.15.39:0/2130297642 learned_addr learned my addr 172.21.15.39:0/2130297642 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:46.310 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.283+0000 7f9d24ec1700 1 --2- 172.21.15.39:0/2130297642 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d0113dad0 0x560d0113ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.310 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.283+0000 7f9d1ffff700 1 --2- 172.21.15.39:0/2130297642 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d011322d0 0x560d0113d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.310 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.283+0000 7f9d1f7fe700 1 -- 172.21.15.39:0/2130297642 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d0113dad0 msgr2=0x560d0113ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:46.310 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.283+0000 7f9d1f7fe700 1 --2- 172.21.15.39:0/2130297642 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d0113dad0 0x560d0113ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.311 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.283+0000 7f9d1f7fe700 1 -- 172.21.15.39:0/2130297642 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d011322d0 msgr2=0x560d0113d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:46.311 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.283+0000 7f9d1f7fe700 1 --2- 172.21.15.39:0/2130297642 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d011322d0 0x560d0113d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.311 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.283+0000 7f9d1f7fe700 1 -- 172.21.15.39:0/2130297642 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d2010f3f0 con 0x560d01132be0 2023-07-21T18:43:46.311 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.283+0000 7f9d1f7fe700 1 --2- 172.21.15.39:0/2130297642 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d01132be0 0x560d0113a730 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f9d140caba0 tx=0x7f9d1409dfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ae24da1332482d3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:46.311 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.283+0000 7f9cff7fe700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:43:46.311 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.283+0000 7f9cff7fe700 1 -- 172.21.15.39:0/2130297642 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1409a380 con 0x560d01132be0 2023-07-21T18:43:46.311 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.283+0000 7f9cff7fe700 1 -- 172.21.15.39:0/2130297642 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f9d14099410 con 0x560d01132be0 2023-07-21T18:43:46.311 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.283+0000 7f9d32aaab80 1 -- 172.21.15.39:0/2130297642 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560d00fde4f0 con 0x560d01132be0 2023-07-21T18:43:46.312 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.284+0000 7f9cff7fe700 1 -- 172.21.15.39:0/2130297642 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d140995d0 con 0x560d01132be0 2023-07-21T18:43:46.312 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.284+0000 7f9d32aaab80 10 client.16221 fetch_fsmap learned FSMap version 33 2023-07-21T18:43:46.312 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.284+0000 7f9d32aaab80 1 -- 172.21.15.39:0/2130297642 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x560d00e9bc60 con 0x560d01132be0 2023-07-21T18:43:46.312 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.284+0000 7f9cff7fe700 1 -- 172.21.15.39:0/2130297642 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560d00fde4f0 con 0x560d01132be0 2023-07-21T18:43:46.312 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.284+0000 7f9cff7fe700 1 -- 172.21.15.39:0/2130297642 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f9d140b7fa0 con 0x560d01132be0 2023-07-21T18:43:46.312 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.284+0000 7f9d32aaab80 10 client.16221 fetch_fsmap finished waiting for FSMap version 33 2023-07-21T18:43:46.312 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.284+0000 7f9d32aaab80 10 client.16221 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T18:43:46.313 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.284+0000 7f9d32aaab80 20 client.16221 populate_metadata read hostname 'smithi039' 2023-07-21T18:43:46.313 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.284+0000 7f9d32aaab80 1 --2- 172.21.15.39:0/2130297642 >> [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] conn(0x560d0114d340 0x560d0114f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.313 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.284+0000 7f9d32aaab80 4 client.16221 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:43:46.313 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.284+0000 7f9d32aaab80 1 -- 172.21.15.39:0/2130297642 --> [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560d0107deb0 con 0x560d0114d340 2023-07-21T18:43:46.313 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.284+0000 7f9d1ffff700 1 --2- 172.21.15.39:0/2130297642 >> [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] conn(0x560d0114d340 0x560d0114f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.313 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.285+0000 7f9cff7fe700 10 client.16221 ms_handle_connect on v2:172.21.15.193:6838/3148488639 2023-07-21T18:43:46.313 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.285+0000 7f9d1ffff700 1 --2- 172.21.15.39:0/2130297642 >> [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] conn(0x560d0114d340 0x560d0114f750 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5b2601ff4c03b673 server_cookie=38720a332c300a96 in_seq=0 out_seq=0 2023-07-21T18:43:46.313 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.285+0000 7f9cff7fe700 1 -- 172.21.15.39:0/2130297642 <== mds.2 v2:172.21.15.193:6838/3148488639 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560d0107deb0 con 0x560d0114d340 2023-07-21T18:43:46.314 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.285+0000 7f9cff7fe700 10 client.16221 handle_command_reply: tid=0 2023-07-21T18:43:46.314 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.285+0000 7f9d32aaab80 1 client.16221 shutdown 2023-07-21T18:43:46.314 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.285+0000 7f9d32aaab80 1 -- 172.21.15.39:0/2130297642 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d01132be0 msgr2=0x560d0113a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:46.314 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.285+0000 7f9d32aaab80 1 --2- 172.21.15.39:0/2130297642 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d01132be0 0x560d0113a730 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f9d140caba0 tx=0x7f9d1409dfd0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.314 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.285+0000 7f9d32aaab80 1 -- 172.21.15.39:0/2130297642 shutdown_connections 2023-07-21T18:43:46.314 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.285+0000 7f9d32aaab80 1 --2- 172.21.15.39:0/2130297642 >> [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] conn(0x560d0114d340 0x560d0114f750 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.314 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.285+0000 7f9d32aaab80 1 --2- 172.21.15.39:0/2130297642 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d011322d0 0x560d0113d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.314 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.285+0000 7f9d32aaab80 1 --2- 172.21.15.39:0/2130297642 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d0113dad0 0x560d0113ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.315 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.285+0000 7f9d32aaab80 1 --2- 172.21.15.39:0/2130297642 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d01132be0 0x560d0113a730 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.315 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.285+0000 7f9d32aaab80 1 -- 172.21.15.39:0/2130297642 >> 172.21.15.39:0/2130297642 conn(0x560d01128da0 msgr2=0x560d0112e230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:46.315 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.285+0000 7f9cff7fe700 0 client.16221 ms_handle_reset on v2:172.21.15.193:6838/3148488639 2023-07-21T18:43:46.315 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.285+0000 7f9d32aaab80 1 -- 172.21.15.39:0/2130297642 shutdown_connections 2023-07-21T18:43:46.315 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.286+0000 7f9d32aaab80 1 -- 172.21.15.39:0/2130297642 wait complete. 2023-07-21T18:43:46.315 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.286+0000 7f9d32aaab80 20 client.16221 trim_cache size 0 max 16384 2023-07-21T18:43:46.319 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.299+0000 7f9d32aaab80 1 Processor -- start 2023-07-21T18:43:46.319 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.299+0000 7f9d32aaab80 1 -- start start 2023-07-21T18:43:46.319 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.299+0000 7f9d32aaab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d011322d0 0x560d01149720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.320 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.299+0000 7f9d32aaab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d01132be0 0x560d0113d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.320 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.299+0000 7f9d32aaab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d0113dad0 0x560d011359f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.320 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.300+0000 7f9d32aaab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d0112fd60 con 0x560d01132be0 2023-07-21T18:43:46.320 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.300+0000 7f9d32aaab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d01133570 con 0x560d011322d0 2023-07-21T18:43:46.320 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.300+0000 7f9d32aaab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d01149ce0 con 0x560d0113dad0 2023-07-21T18:43:46.320 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.300+0000 7f9d1f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d01132be0 0x560d0113d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.320 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.300+0000 7f9d1f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d01132be0 0x560d0113d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59094/0 (socket says 172.21.15.39:59094) 2023-07-21T18:43:46.320 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.300+0000 7f9d1f7fe700 1 -- 172.21.15.39:0/4006114204 learned_addr learned my addr 172.21.15.39:0/4006114204 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:46.321 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.300+0000 7f9d24ec1700 1 --2- 172.21.15.39:0/4006114204 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d0113dad0 0x560d011359f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.321 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.300+0000 7f9d1ffff700 1 --2- 172.21.15.39:0/4006114204 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d011322d0 0x560d01149720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.321 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.300+0000 7f9d1f7fe700 1 -- 172.21.15.39:0/4006114204 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d0113dad0 msgr2=0x560d011359f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:46.321 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.300+0000 7f9d1f7fe700 1 --2- 172.21.15.39:0/4006114204 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d0113dad0 0x560d011359f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.321 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.300+0000 7f9d1f7fe700 1 -- 172.21.15.39:0/4006114204 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d011322d0 msgr2=0x560d01149720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:46.321 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.300+0000 7f9d1f7fe700 1 --2- 172.21.15.39:0/4006114204 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d011322d0 0x560d01149720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.321 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.300+0000 7f9d1f7fe700 1 -- 172.21.15.39:0/4006114204 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560d00ed9250 con 0x560d01132be0 2023-07-21T18:43:46.322 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.300+0000 7f9d1ffff700 1 --2- 172.21.15.39:0/4006114204 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d011322d0 0x560d01149720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:43:46.322 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.300+0000 7f9d1f7fe700 1 --2- 172.21.15.39:0/4006114204 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d01132be0 0x560d0113d050 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f9d14099eb0 tx=0x7f9d1400c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a186772e7fe0c40 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:46.322 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.300+0000 7f9d06ffd700 1 -- 172.21.15.39:0/4006114204 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d140b8170 con 0x560d01132be0 2023-07-21T18:43:46.322 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.300+0000 7f9d06ffd700 1 -- 172.21.15.39:0/4006114204 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d140b82d0 con 0x560d01132be0 2023-07-21T18:43:46.322 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.300+0000 7f9d06ffd700 1 -- 172.21.15.39:0/4006114204 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d140a21b0 con 0x560d01132be0 2023-07-21T18:43:46.325 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.301+0000 7f9d32aaab80 1 -- 172.21.15.39:0/4006114204 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d01132be0 msgr2=0x560d0113d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:46.325 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.301+0000 7f9d32aaab80 1 --2- 172.21.15.39:0/4006114204 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d01132be0 0x560d0113d050 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f9d14099eb0 tx=0x7f9d1400c480 comp rx=0 tx=0).stop 2023-07-21T18:43:46.325 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.301+0000 7f9d32aaab80 1 -- 172.21.15.39:0/4006114204 shutdown_connections 2023-07-21T18:43:46.325 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.301+0000 7f9d32aaab80 1 --2- 172.21.15.39:0/4006114204 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d011322d0 0x560d01149720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.326 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.301+0000 7f9d32aaab80 1 --2- 172.21.15.39:0/4006114204 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d0113dad0 0x560d011359f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.326 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.301+0000 7f9d32aaab80 1 --2- 172.21.15.39:0/4006114204 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d01132be0 0x560d0113d050 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.326 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.301+0000 7f9d32aaab80 1 -- 172.21.15.39:0/4006114204 >> 172.21.15.39:0/4006114204 conn(0x560d01128da0 msgr2=0x560d011540b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:46.326 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.301+0000 7f9d32aaab80 1 -- 172.21.15.39:0/4006114204 shutdown_connections 2023-07-21T18:43:46.326 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.301+0000 7f9d32aaab80 1 -- 172.21.15.39:0/4006114204 wait complete. 2023-07-21T18:43:46.326 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.301+0000 7f9d32aaab80 1 -- start start 2023-07-21T18:43:46.326 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.301+0000 7f9d32aaab80 1 Processor -- start 2023-07-21T18:43:46.327 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.302+0000 7f9d32aaab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d011322d0 0x560d01141eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.327 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.302+0000 7f9d32aaab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d01132be0 0x560d011423f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.327 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.302+0000 7f9d32aaab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d0113dad0 0x560d011a0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.327 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.302+0000 7f9d32aaab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d01142bf0 con 0x560d01132be0 2023-07-21T18:43:46.327 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.302+0000 7f9d32aaab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d01142d60 con 0x560d011322d0 2023-07-21T18:43:46.327 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.302+0000 7f9d32aaab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d011a07c0 con 0x560d0113dad0 2023-07-21T18:43:46.327 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.302+0000 7f9d32aaab80 10 client.0 _ll_register_callbacks cb 0x7f9d14099f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:43:46.327 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.302+0000 7f9d1f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d01132be0 0x560d011423f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.328 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.302+0000 7f9d1f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d01132be0 0x560d011423f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59100/0 (socket says 172.21.15.39:59100) 2023-07-21T18:43:46.328 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.302+0000 7f9d1f7fe700 1 -- 172.21.15.39:0/3857428683 learned_addr learned my addr 172.21.15.39:0/3857428683 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:46.328 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.302+0000 7f9d24ec1700 1 --2- 172.21.15.39:0/3857428683 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d0113dad0 0x560d011a0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.328 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.302+0000 7f9d1ffff700 1 --2- 172.21.15.39:0/3857428683 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d011322d0 0x560d01141eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.328 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.302+0000 7f9d1f7fe700 1 -- 172.21.15.39:0/3857428683 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d0113dad0 msgr2=0x560d011a0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:46.328 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.302+0000 7f9d1f7fe700 1 --2- 172.21.15.39:0/3857428683 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d0113dad0 0x560d011a0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.328 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.302+0000 7f9d1f7fe700 1 -- 172.21.15.39:0/3857428683 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d011322d0 msgr2=0x560d01141eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:46.328 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.302+0000 7f9d1f7fe700 1 --2- 172.21.15.39:0/3857428683 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d011322d0 0x560d01141eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.329 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.302+0000 7f9d1f7fe700 1 -- 172.21.15.39:0/3857428683 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560d00e9bc60 con 0x560d01132be0 2023-07-21T18:43:46.329 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.302+0000 7f9d1ffff700 1 --2- 172.21.15.39:0/3857428683 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d011322d0 0x560d01141eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:43:46.329 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.302+0000 7f9d1f7fe700 1 --2- 172.21.15.39:0/3857428683 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d01132be0 0x560d011423f0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f9d1409e110 tx=0x7f9d140a1f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e05aa0bd3a80e63 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:46.329 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.302+0000 7f9cff7fe700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:43:46.329 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.302+0000 7f9cff7fe700 1 -- 172.21.15.39:0/3857428683 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d14099610 con 0x560d01132be0 2023-07-21T18:43:46.329 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.302+0000 7f9cff7fe700 1 -- 172.21.15.39:0/3857428683 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f9d1409a180 con 0x560d01132be0 2023-07-21T18:43:46.329 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.302+0000 7f9d32aaab80 1 -- 172.21.15.39:0/3857428683 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9d18003a00 con 0x560d01132be0 2023-07-21T18:43:46.329 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.303+0000 7f9cff7fe700 1 -- 172.21.15.39:0/3857428683 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d1409a340 con 0x560d01132be0 2023-07-21T18:43:46.330 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.303+0000 7f9d32aaab80 10 client.16227 fetch_fsmap learned FSMap version 33 2023-07-21T18:43:46.330 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.303+0000 7f9d32aaab80 1 -- 172.21.15.39:0/3857428683 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x560d00eb12b0 con 0x560d01132be0 2023-07-21T18:43:46.330 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.303+0000 7f9cff7fe700 1 -- 172.21.15.39:0/3857428683 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d18003a00 con 0x560d01132be0 2023-07-21T18:43:46.330 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.303+0000 7f9cff7fe700 1 -- 172.21.15.39:0/3857428683 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f9d1400caa0 con 0x560d01132be0 2023-07-21T18:43:46.330 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.303+0000 7f9d32aaab80 10 client.16227 fetch_fsmap finished waiting for FSMap version 33 2023-07-21T18:43:46.330 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.303+0000 7f9d32aaab80 10 client.16227 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T18:43:46.330 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.303+0000 7f9d32aaab80 20 client.16227 populate_metadata read hostname 'smithi039' 2023-07-21T18:43:46.331 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.303+0000 7f9d32aaab80 1 --2- 172.21.15.39:0/3857428683 >> [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] conn(0x560d011a1960 0x560d011a3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.331 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.303+0000 7f9d32aaab80 4 client.16227 mds_command: new command op to 24487 tid=0[{"prefix": "damage ls"}] 2023-07-21T18:43:46.331 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.303+0000 7f9d32aaab80 1 -- 172.21.15.39:0/3857428683 --> [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560d01040050 con 0x560d011a1960 2023-07-21T18:43:46.331 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.303+0000 7f9d1ffff700 1 --2- 172.21.15.39:0/3857428683 >> [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] conn(0x560d011a1960 0x560d011a3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.331 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.303+0000 7f9cff7fe700 10 client.16227 ms_handle_connect on v2:172.21.15.193:6838/3148488639 2023-07-21T18:43:46.331 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.303+0000 7f9d1ffff700 1 --2- 172.21.15.39:0/3857428683 >> [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] conn(0x560d011a1960 0x560d011a3d40 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ed952f8684755637 server_cookie=5db3f368ab947e11 in_seq=0 out_seq=0 2023-07-21T18:43:46.331 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.304+0000 7f9cff7fe700 1 -- 172.21.15.39:0/3857428683 <== mds.2 v2:172.21.15.193:6838/3148488639 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+4475 (crc 0 0 0) 0x560d01040050 con 0x560d011a1960 2023-07-21T18:43:46.331 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.304+0000 7f9cff7fe700 10 client.16227 handle_command_reply: tid=0 2023-07-21T18:43:46.332 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.304+0000 7f9d32aaab80 1 client.16227 shutdown 2023-07-21T18:43:46.332 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.304+0000 7f9d32aaab80 1 -- 172.21.15.39:0/3857428683 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d01132be0 msgr2=0x560d011423f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:46.332 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.304+0000 7f9d32aaab80 1 --2- 172.21.15.39:0/3857428683 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d01132be0 0x560d011423f0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f9d1409e110 tx=0x7f9d140a1f40 comp rx=0 tx=0).stop 2023-07-21T18:43:46.332 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.304+0000 7f9d32aaab80 1 -- 172.21.15.39:0/3857428683 shutdown_connections 2023-07-21T18:43:46.332 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.304+0000 7f9d32aaab80 1 --2- 172.21.15.39:0/3857428683 >> [v2:172.21.15.193:6838/3148488639,v1:172.21.15.193:6839/3148488639] conn(0x560d011a1960 0x560d011a3d40 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.332 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.304+0000 7f9d32aaab80 1 --2- 172.21.15.39:0/3857428683 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d011322d0 0x560d01141eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.332 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.304+0000 7f9cff7fe700 0 client.16227 ms_handle_reset on v2:172.21.15.193:6838/3148488639 2023-07-21T18:43:46.332 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.304+0000 7f9d32aaab80 1 --2- 172.21.15.39:0/3857428683 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d0113dad0 0x560d011a0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.333 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.304+0000 7f9d32aaab80 1 --2- 172.21.15.39:0/3857428683 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d01132be0 0x560d011423f0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.333 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.304+0000 7f9d32aaab80 1 -- 172.21.15.39:0/3857428683 >> 172.21.15.39:0/3857428683 conn(0x560d01128da0 msgr2=0x560d011359f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:46.333 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.304+0000 7f9d32aaab80 1 -- 172.21.15.39:0/3857428683 shutdown_connections 2023-07-21T18:43:46.333 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.304+0000 7f9d32aaab80 1 -- 172.21.15.39:0/3857428683 wait complete. 2023-07-21T18:43:46.333 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.304+0000 7f9d32aaab80 20 client.16227 trim_cache size 0 max 16384 2023-07-21T18:43:46.333 INFO:teuthology.orchestra.run.smithi039.stdout:[ 2023-07-21T18:43:46.334 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T18:43:46.334 INFO:teuthology.orchestra.run.smithi039.stdout: "damage_type": "backtrace", 2023-07-21T18:43:46.334 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 32841408, 2023-07-21T18:43:46.334 INFO:teuthology.orchestra.run.smithi039.stdout: "ino": 1099511628298, 2023-07-21T18:43:46.334 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/WORD/BASEMACH.DOC" 2023-07-21T18:43:46.334 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T18:43:46.334 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T18:43:46.335 INFO:teuthology.orchestra.run.smithi039.stdout: "damage_type": "backtrace", 2023-07-21T18:43:46.335 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 642881598, 2023-07-21T18:43:46.335 INFO:teuthology.orchestra.run.smithi039.stdout: "ino": 1099511628299, 2023-07-21T18:43:46.335 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/WORD/FACTS.DOC" 2023-07-21T18:43:46.335 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T18:43:46.335 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T18:43:46.335 INFO:teuthology.orchestra.run.smithi039.stdout: "damage_type": "backtrace", 2023-07-21T18:43:46.335 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 695121249, 2023-07-21T18:43:46.336 INFO:teuthology.orchestra.run.smithi039.stdout: "ino": 1099511628313, 2023-07-21T18:43:46.336 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC1.CDR" 2023-07-21T18:43:46.336 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T18:43:46.336 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T18:43:46.336 INFO:teuthology.orchestra.run.smithi039.stdout: "damage_type": "backtrace", 2023-07-21T18:43:46.336 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 999289580, 2023-07-21T18:43:46.336 INFO:teuthology.orchestra.run.smithi039.stdout: "ino": 1099511628315, 2023-07-21T18:43:46.336 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/COREL/TEAPOT2.CPT" 2023-07-21T18:43:46.337 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T18:43:46.337 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T18:43:46.337 INFO:teuthology.orchestra.run.smithi039.stdout: "damage_type": "backtrace", 2023-07-21T18:43:46.337 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 1470998723, 2023-07-21T18:43:46.337 INFO:teuthology.orchestra.run.smithi039.stdout: "ino": 1099511628302, 2023-07-21T18:43:46.337 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/EXCEL/SALES.XLS" 2023-07-21T18:43:46.337 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T18:43:46.337 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T18:43:46.338 INFO:teuthology.orchestra.run.smithi039.stdout: "damage_type": "backtrace", 2023-07-21T18:43:46.338 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 1538270232, 2023-07-21T18:43:46.338 INFO:teuthology.orchestra.run.smithi039.stdout: "ino": 1099511628310, 2023-07-21T18:43:46.338 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/COREL/CDRROLS.CFG" 2023-07-21T18:43:46.338 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T18:43:46.338 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T18:43:46.338 INFO:teuthology.orchestra.run.smithi039.stdout: "damage_type": "backtrace", 2023-07-21T18:43:46.338 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 1735867768, 2023-07-21T18:43:46.339 INFO:teuthology.orchestra.run.smithi039.stdout: "ino": 1099511628314, 2023-07-21T18:43:46.339 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/COREL/MAILBOX2.CPT" 2023-07-21T18:43:46.339 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T18:43:46.339 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T18:43:46.339 INFO:teuthology.orchestra.run.smithi039.stdout: "damage_type": "backtrace", 2023-07-21T18:43:46.339 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 1811568482, 2023-07-21T18:43:46.339 INFO:teuthology.orchestra.run.smithi039.stdout: "ino": 1099511628317, 2023-07-21T18:43:46.339 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/COREL/TRAIN.CDR" 2023-07-21T18:43:46.340 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T18:43:46.340 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T18:43:46.340 INFO:teuthology.orchestra.run.smithi039.stdout: "damage_type": "backtrace", 2023-07-21T18:43:46.340 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 2089966051, 2023-07-21T18:43:46.340 INFO:teuthology.orchestra.run.smithi039.stdout: "ino": 1099511628319, 2023-07-21T18:43:46.340 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/COREL/ROSE.CPT" 2023-07-21T18:43:46.340 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T18:43:46.340 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T18:43:46.341 INFO:teuthology.orchestra.run.smithi039.stdout: "damage_type": "backtrace", 2023-07-21T18:43:46.341 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 2752898913, 2023-07-21T18:43:46.341 INFO:teuthology.orchestra.run.smithi039.stdout: "ino": 1099511628327, 2023-07-21T18:43:46.341 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/PM/EVENTS.DOC" 2023-07-21T18:43:46.341 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T18:43:46.341 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T18:43:46.341 INFO:teuthology.orchestra.run.smithi039.stdout: "damage_type": "backtrace", 2023-07-21T18:43:46.341 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 2944040979, 2023-07-21T18:43:46.342 INFO:teuthology.orchestra.run.smithi039.stdout: "ino": 1099511628311, 2023-07-21T18:43:46.342 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/COREL/CDRBARS.CFG" 2023-07-21T18:43:46.342 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T18:43:46.342 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T18:43:46.342 INFO:teuthology.orchestra.run.smithi039.stdout: "damage_type": "backtrace", 2023-07-21T18:43:46.342 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 3256400975, 2023-07-21T18:43:46.342 INFO:teuthology.orchestra.run.smithi039.stdout: "ino": 1099511628326, 2023-07-21T18:43:46.343 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/PM/T1.XLS" 2023-07-21T18:43:46.343 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T18:43:46.343 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T18:43:46.343 INFO:teuthology.orchestra.run.smithi039.stdout: "damage_type": "backtrace", 2023-07-21T18:43:46.343 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 3354151126, 2023-07-21T18:43:46.343 INFO:teuthology.orchestra.run.smithi039.stdout: "ino": 1099511628320, 2023-07-21T18:43:46.343 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/COREL/CORELDRW.CDT" 2023-07-21T18:43:46.343 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T18:43:46.344 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T18:43:46.344 INFO:teuthology.orchestra.run.smithi039.stdout: "damage_type": "backtrace", 2023-07-21T18:43:46.344 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 3372412515, 2023-07-21T18:43:46.344 INFO:teuthology.orchestra.run.smithi039.stdout: "ino": 1099511628297, 2023-07-21T18:43:46.344 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/WORD/CHAP10.DOC" 2023-07-21T18:43:46.344 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T18:43:46.344 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T18:43:46.344 INFO:teuthology.orchestra.run.smithi039.stdout: "damage_type": "backtrace", 2023-07-21T18:43:46.345 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 3643633672, 2023-07-21T18:43:46.345 INFO:teuthology.orchestra.run.smithi039.stdout: "ino": 1099511628328, 2023-07-21T18:43:46.345 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/PM/MOVED.DOC" 2023-07-21T18:43:46.345 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T18:43:46.345 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T18:43:46.345 INFO:teuthology.orchestra.run.smithi039.stdout: "damage_type": "backtrace", 2023-07-21T18:43:46.345 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 3913448916, 2023-07-21T18:43:46.345 INFO:teuthology.orchestra.run.smithi039.stdout: "ino": 1099511628301, 2023-07-21T18:43:46.346 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/EXCEL/PCMAGCD.XLS" 2023-07-21T18:43:46.346 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T18:43:46.346 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T18:43:46.346 INFO:teuthology.orchestra.run.smithi039.stdout: "damage_type": "backtrace", 2023-07-21T18:43:46.346 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 3958397844, 2023-07-21T18:43:46.346 INFO:teuthology.orchestra.run.smithi039.stdout: "ino": 1099511628318, 2023-07-21T18:43:46.346 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC2.CDR" 2023-07-21T18:43:46.347 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T18:43:46.347 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T18:43:46.347 INFO:teuthology.orchestra.run.smithi039.stdout: "damage_type": "backtrace", 2023-07-21T18:43:46.347 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 4216958244, 2023-07-21T18:43:46.347 INFO:teuthology.orchestra.run.smithi039.stdout: "ino": 1099511628316, 2023-07-21T18:43:46.347 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/COREL/SIGN2.CPT" 2023-07-21T18:43:46.347 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T18:43:46.347 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T18:43:46.348 INFO:teuthology.orchestra.run.smithi039.stdout: "damage_type": "backtrace", 2023-07-21T18:43:46.348 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 4245475529, 2023-07-21T18:43:46.348 INFO:teuthology.orchestra.run.smithi039.stdout: "ino": 1099511628312, 2023-07-21T18:43:46.348 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/COREL/ARTISTIC.ACL" 2023-07-21T18:43:46.348 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T18:43:46.348 INFO:teuthology.orchestra.run.smithi039.stdout:] 2023-07-21T18:43:46.349 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.304+0000 7f9d26924700 1 -- 172.21.15.39:0/1328186909 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f9d080a12c0 msgr2=0x7f9d080a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:46.349 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.304+0000 7f9d26924700 1 --2- 172.21.15.39:0/1328186909 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f9d080a12c0 0x7f9d080a3770 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9d10009c60 tx=0x7f9d10009260 comp rx=0 tx=0).stop 2023-07-21T18:43:46.350 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.304+0000 7f9d26924700 1 -- 172.21.15.39:0/1328186909 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d200ff1c0 msgr2=0x7f9d20198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:46.350 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.304+0000 7f9d26924700 1 --2- 172.21.15.39:0/1328186909 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d200ff1c0 0x7f9d20198400 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f9d1400e9a0 tx=0x7f9d1400ecb0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.350 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.305+0000 7f9d26924700 1 -- 172.21.15.39:0/1328186909 shutdown_connections 2023-07-21T18:43:46.350 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.305+0000 7f9d26924700 1 --2- 172.21.15.39:0/1328186909 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f9d080a12c0 0x7f9d080a3770 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.350 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.305+0000 7f9d26924700 1 --2- 172.21.15.39:0/1328186909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d2010c3d0 0x7f9d2019c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.350 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.305+0000 7f9d26924700 1 --2- 172.21.15.39:0/1328186909 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d200fe820 0x7f9d20197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.350 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.305+0000 7f9d26924700 1 --2- 172.21.15.39:0/1328186909 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d200ff1c0 0x7f9d20198400 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.350 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.305+0000 7f9d26924700 1 -- 172.21.15.39:0/1328186909 >> 172.21.15.39:0/1328186909 conn(0x7f9d200fa4b0 msgr2=0x7f9d2010d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:46.351 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.305+0000 7f9d26924700 1 -- 172.21.15.39:0/1328186909 shutdown_connections 2023-07-21T18:43:46.351 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.305+0000 7f9d26924700 1 -- 172.21.15.39:0/1328186909 wait complete. 2023-07-21T18:43:46.351 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-21T18:43:46.366 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:46 smithi193 ceph-mon[98204]: pgmap v63: 97 pgs: 97 active+clean; 29 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 167 KiB/s wr, 46 op/s 2023-07-21T18:43:46.367 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:46 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/518618751' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:43:46.385 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:46 smithi155 ceph-mon[97934]: pgmap v63: 97 pgs: 97 active+clean; 29 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 167 KiB/s wr, 46 op/s 2023-07-21T18:43:46.386 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:46 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/518618751' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T18:43:46.483 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.464+0000 7f8e885aa700 1 -- 172.21.15.39:0/551193515 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e801049e0 msgr2=0x7f8e80104db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:46.484 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.464+0000 7f8e885aa700 1 --2- 172.21.15.39:0/551193515 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e801049e0 0x7f8e80104db0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f8e70009990 tx=0x7f8e70009ca0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.484 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.464+0000 7f8e885aa700 1 -- 172.21.15.39:0/551193515 shutdown_connections 2023-07-21T18:43:46.484 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.464+0000 7f8e885aa700 1 --2- 172.21.15.39:0/551193515 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e800ff380 0x7f8e8010b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.484 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.464+0000 7f8e885aa700 1 --2- 172.21.15.39:0/551193515 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e801049e0 0x7f8e80104db0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.484 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.464+0000 7f8e885aa700 1 --2- 172.21.15.39:0/551193515 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e800fe9d0 0x7f8e800fee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.484 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.464+0000 7f8e885aa700 1 -- 172.21.15.39:0/551193515 >> 172.21.15.39:0/551193515 conn(0x7f8e800fa4b0 msgr2=0x7f8e800fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:46.484 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.464+0000 7f8e885aa700 1 -- 172.21.15.39:0/551193515 shutdown_connections 2023-07-21T18:43:46.485 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.464+0000 7f8e885aa700 1 -- 172.21.15.39:0/551193515 wait complete. 2023-07-21T18:43:46.486 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.465+0000 7f8e885aa700 1 Processor -- start 2023-07-21T18:43:46.486 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.465+0000 7f8e885aa700 1 -- start start 2023-07-21T18:43:46.486 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.465+0000 7f8e885aa700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e800fe9d0 0x7f8e80194800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.487 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.465+0000 7f8e885aa700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e800ff380 0x7f8e80194d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.487 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.465+0000 7f8e885aa700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e801049e0 0x7f8e80195280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.487 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.465+0000 7f8e885aa700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e801a4040 con 0x7f8e800ff380 2023-07-21T18:43:46.487 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.465+0000 7f8e885aa700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e801a41b0 con 0x7f8e800fe9d0 2023-07-21T18:43:46.487 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.465+0000 7f8e885aa700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e801a4370 con 0x7f8e801049e0 2023-07-21T18:43:46.487 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.465+0000 7f8e85b45700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e800ff380 0x7f8e80194d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.487 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.465+0000 7f8e85b45700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e800ff380 0x7f8e80194d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59114/0 (socket says 172.21.15.39:59114) 2023-07-21T18:43:46.487 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.465+0000 7f8e85b45700 1 -- 172.21.15.39:0/2012910853 learned_addr learned my addr 172.21.15.39:0/2012910853 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:46.488 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.465+0000 7f8e86b47700 1 --2- 172.21.15.39:0/2012910853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e801049e0 0x7f8e80195280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.488 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.465+0000 7f8e86346700 1 --2- 172.21.15.39:0/2012910853 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e800fe9d0 0x7f8e80194800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.488 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.465+0000 7f8e85b45700 1 -- 172.21.15.39:0/2012910853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e801049e0 msgr2=0x7f8e80195280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:46.488 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.465+0000 7f8e85b45700 1 --2- 172.21.15.39:0/2012910853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e801049e0 0x7f8e80195280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.488 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.465+0000 7f8e85b45700 1 -- 172.21.15.39:0/2012910853 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e800fe9d0 msgr2=0x7f8e80194800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:46.488 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.465+0000 7f8e85b45700 1 --2- 172.21.15.39:0/2012910853 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e800fe9d0 0x7f8e80194800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.488 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.465+0000 7f8e85b45700 1 -- 172.21.15.39:0/2012910853 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e700095f0 con 0x7f8e800ff380 2023-07-21T18:43:46.489 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.466+0000 7f8e85b45700 1 --2- 172.21.15.39:0/2012910853 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e800ff380 0x7f8e80194d40 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c00e9a0 tx=0x7f8e7c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:46.489 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.466+0000 7f8e777fe700 1 -- 172.21.15.39:0/2012910853 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c009790 con 0x7f8e800ff380 2023-07-21T18:43:46.489 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.466+0000 7f8e777fe700 1 -- 172.21.15.39:0/2012910853 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e7c010430 con 0x7f8e800ff380 2023-07-21T18:43:46.489 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.466+0000 7f8e777fe700 1 -- 172.21.15.39:0/2012910853 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c018680 con 0x7f8e800ff380 2023-07-21T18:43:46.489 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.466+0000 7f8e885aa700 1 -- 172.21.15.39:0/2012910853 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e801a4650 con 0x7f8e800ff380 2023-07-21T18:43:46.489 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.466+0000 7f8e885aa700 1 -- 172.21.15.39:0/2012910853 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e801a4b10 con 0x7f8e800ff380 2023-07-21T18:43:46.493 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.468+0000 7f8e777fe700 1 -- 172.21.15.39:0/2012910853 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f8e7c0105a0 con 0x7f8e800ff380 2023-07-21T18:43:46.493 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.468+0000 7f8e777fe700 1 --2- 172.21.15.39:0/2012910853 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f8e6c0a12c0 0x7f8e6c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.493 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.469+0000 7f8e86346700 1 --2- 172.21.15.39:0/2012910853 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f8e6c0a12c0 0x7f8e6c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.494 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.469+0000 7f8e777fe700 1 -- 172.21.15.39:0/2012910853 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f8e7c014070 con 0x7f8e800ff380 2023-07-21T18:43:46.494 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.469+0000 7f8e86346700 1 --2- 172.21.15.39:0/2012910853 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f8e6c0a12c0 0x7f8e6c0a3770 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8e7000b2e0 tx=0x7f8e70005fd0 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:46.494 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.469+0000 7f8e94730b80 1 Processor -- start 2023-07-21T18:43:46.494 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.469+0000 7f8e94730b80 1 -- start start 2023-07-21T18:43:46.494 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.470+0000 7f8e94730b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de599692d0 0x55de599696a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.494 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.470+0000 7f8e94730b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de59969be0 0x55de59974590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.494 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.470+0000 7f8e94730b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55de59974ad0 0x55de59976eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.494 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.470+0000 7f8e94730b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de5986fcb0 con 0x55de59969be0 2023-07-21T18:43:46.495 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.470+0000 7f8e94730b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de598c2a40 con 0x55de599692d0 2023-07-21T18:43:46.495 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.470+0000 7f8e94730b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de598bbed0 con 0x55de59974ad0 2023-07-21T18:43:46.495 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.470+0000 7f8e85b45700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de59969be0 0x55de59974590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.495 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.470+0000 7f8e85b45700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de59969be0 0x55de59974590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59122/0 (socket says 172.21.15.39:59122) 2023-07-21T18:43:46.495 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.470+0000 7f8e85b45700 1 -- 172.21.15.39:0/731071835 learned_addr learned my addr 172.21.15.39:0/731071835 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:46.495 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.470+0000 7f8e86346700 1 --2- 172.21.15.39:0/731071835 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de599692d0 0x55de599696a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.495 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.470+0000 7f8e86b47700 1 --2- 172.21.15.39:0/731071835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55de59974ad0 0x55de59976eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.496 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.470+0000 7f8e85b45700 1 -- 172.21.15.39:0/731071835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55de59974ad0 msgr2=0x55de59976eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:46.496 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.470+0000 7f8e85b45700 1 --2- 172.21.15.39:0/731071835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55de59974ad0 0x55de59976eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.496 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.470+0000 7f8e85b45700 1 -- 172.21.15.39:0/731071835 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de599692d0 msgr2=0x55de599696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:46.496 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.470+0000 7f8e85b45700 1 --2- 172.21.15.39:0/731071835 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de599692d0 0x55de599696a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.496 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.470+0000 7f8e85b45700 1 -- 172.21.15.39:0/731071835 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de596e9de0 con 0x55de59969be0 2023-07-21T18:43:46.496 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.470+0000 7f8e85b45700 1 --2- 172.21.15.39:0/731071835 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de59969be0 0x55de59974590 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c0a1ed0 tx=0x7f8e7c099fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdc102801e165112 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:46.496 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.470+0000 7f8e74ff9700 1 -- 172.21.15.39:0/731071835 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c0a2400 con 0x55de59969be0 2023-07-21T18:43:46.496 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.470+0000 7f8e74ff9700 1 -- 172.21.15.39:0/731071835 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e7c010880 con 0x55de59969be0 2023-07-21T18:43:46.497 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.471+0000 7f8e74ff9700 1 -- 172.21.15.39:0/731071835 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c0a1f70 con 0x55de59969be0 2023-07-21T18:43:46.497 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.471+0000 7f8e94730b80 1 -- 172.21.15.39:0/731071835 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de59969be0 msgr2=0x55de59974590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:46.497 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.471+0000 7f8e94730b80 1 --2- 172.21.15.39:0/731071835 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de59969be0 0x55de59974590 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c0a1ed0 tx=0x7f8e7c099fb0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.497 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.471+0000 7f8e94730b80 1 -- 172.21.15.39:0/731071835 shutdown_connections 2023-07-21T18:43:46.497 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.471+0000 7f8e94730b80 1 --2- 172.21.15.39:0/731071835 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de599692d0 0x55de599696a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.497 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.471+0000 7f8e94730b80 1 --2- 172.21.15.39:0/731071835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55de59974ad0 0x55de59976eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.497 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.471+0000 7f8e94730b80 1 --2- 172.21.15.39:0/731071835 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de59969be0 0x55de59974590 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.497 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.471+0000 7f8e94730b80 1 -- 172.21.15.39:0/731071835 >> 172.21.15.39:0/731071835 conn(0x55de5995fda0 msgr2=0x55de599656a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:46.498 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.471+0000 7f8e94730b80 1 -- 172.21.15.39:0/731071835 shutdown_connections 2023-07-21T18:43:46.498 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.471+0000 7f8e94730b80 1 -- 172.21.15.39:0/731071835 wait complete. 2023-07-21T18:43:46.498 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.472+0000 7f8e94730b80 1 -- start start 2023-07-21T18:43:46.498 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.472+0000 7f8e94730b80 1 Processor -- start 2023-07-21T18:43:46.498 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.472+0000 7f8e94730b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de599692d0 0x55de59971880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.498 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.472+0000 7f8e94730b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55de59969be0 0x55de59971dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.498 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.472+0000 7f8e94730b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de59974ad0 0x55de59972300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.498 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.472+0000 7f8e94730b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de5985bac0 con 0x55de59974ad0 2023-07-21T18:43:46.499 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.472+0000 7f8e94730b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de5985b7c0 con 0x55de599692d0 2023-07-21T18:43:46.499 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.472+0000 7f8e94730b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de5973f920 con 0x55de59969be0 2023-07-21T18:43:46.499 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.472+0000 7f8e94730b80 10 client.0 _ll_register_callbacks cb 0x55de59859760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:43:46.499 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.472+0000 7f8e86b47700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de59974ad0 0x55de59972300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.499 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.472+0000 7f8e86b47700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de59974ad0 0x55de59972300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59130/0 (socket says 172.21.15.39:59130) 2023-07-21T18:43:46.499 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.472+0000 7f8e86b47700 1 -- 172.21.15.39:0/4030474520 learned_addr learned my addr 172.21.15.39:0/4030474520 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:46.499 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.472+0000 7f8e85b45700 1 --2- 172.21.15.39:0/4030474520 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55de59969be0 0x55de59971dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.500 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.472+0000 7f8e86346700 1 --2- 172.21.15.39:0/4030474520 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de599692d0 0x55de59971880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.500 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.472+0000 7f8e86b47700 1 -- 172.21.15.39:0/4030474520 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55de59969be0 msgr2=0x55de59971dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:46.500 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.472+0000 7f8e86b47700 1 --2- 172.21.15.39:0/4030474520 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55de59969be0 0x55de59971dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.500 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.473+0000 7f8e86b47700 1 -- 172.21.15.39:0/4030474520 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de599692d0 msgr2=0x55de59971880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:46.500 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.473+0000 7f8e86b47700 1 --2- 172.21.15.39:0/4030474520 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de599692d0 0x55de59971880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.500 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.473+0000 7f8e86b47700 1 -- 172.21.15.39:0/4030474520 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e801a4b10 con 0x55de59974ad0 2023-07-21T18:43:46.500 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.473+0000 7f8e85b45700 1 --2- 172.21.15.39:0/4030474520 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55de59969be0 0x55de59971dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:43:46.500 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.473+0000 7f8e86346700 1 --2- 172.21.15.39:0/4030474520 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de599692d0 0x55de59971880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:43:46.501 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.473+0000 7f8e86b47700 1 --2- 172.21.15.39:0/4030474520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de59974ad0 0x55de59972300 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f8e78009790 tx=0x7f8e78007b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a82e3b7645e7f74 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:46.501 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.473+0000 7f8e69ffb700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:43:46.501 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.473+0000 7f8e69ffb700 1 -- 172.21.15.39:0/4030474520 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7800e040 con 0x55de59974ad0 2023-07-21T18:43:46.501 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.473+0000 7f8e69ffb700 1 -- 172.21.15.39:0/4030474520 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f8e780136c0 con 0x55de59974ad0 2023-07-21T18:43:46.501 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.473+0000 7f8e94730b80 1 -- 172.21.15.39:0/4030474520 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55de598154f0 con 0x55de59974ad0 2023-07-21T18:43:46.501 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.473+0000 7f8e69ffb700 1 -- 172.21.15.39:0/4030474520 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e78013880 con 0x55de59974ad0 2023-07-21T18:43:46.501 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.473+0000 7f8e94730b80 10 client.16239 fetch_fsmap learned FSMap version 33 2023-07-21T18:43:46.501 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.473+0000 7f8e94730b80 1 -- 172.21.15.39:0/4030474520 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55de596d2c60 con 0x55de59974ad0 2023-07-21T18:43:46.502 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.473+0000 7f8e69ffb700 1 -- 172.21.15.39:0/4030474520 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55de598154f0 con 0x55de59974ad0 2023-07-21T18:43:46.502 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.473+0000 7f8e69ffb700 1 -- 172.21.15.39:0/4030474520 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f8e78028c20 con 0x55de59974ad0 2023-07-21T18:43:46.502 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.473+0000 7f8e94730b80 10 client.16239 fetch_fsmap finished waiting for FSMap version 33 2023-07-21T18:43:46.502 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.473+0000 7f8e94730b80 10 client.16239 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T18:43:46.502 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.473+0000 7f8e94730b80 20 client.16239 populate_metadata read hostname 'smithi039' 2023-07-21T18:43:46.502 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.473+0000 7f8e94730b80 1 --2- 172.21.15.39:0/4030474520 >> [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] conn(0x55de599844a0 0x55de599868b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.502 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.473+0000 7f8e94730b80 4 client.16239 mds_command: new command op to 24500 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:43:46.502 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.474+0000 7f8e94730b80 1 -- 172.21.15.39:0/4030474520 --> [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55de598b4eb0 con 0x55de599844a0 2023-07-21T18:43:46.503 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.474+0000 7f8e86346700 1 --2- 172.21.15.39:0/4030474520 >> [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] conn(0x55de599844a0 0x55de599868b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.503 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.474+0000 7f8e69ffb700 10 client.16239 ms_handle_connect on v2:172.21.15.155:6836/947670056 2023-07-21T18:43:46.503 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.474+0000 7f8e86346700 1 --2- 172.21.15.39:0/4030474520 >> [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] conn(0x55de599844a0 0x55de599868b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b788a403796ffcf6 server_cookie=56f3d3c267d91d83 in_seq=0 out_seq=0 2023-07-21T18:43:46.504 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.475+0000 7f8e69ffb700 1 -- 172.21.15.39:0/4030474520 <== mds.1 v2:172.21.15.155:6836/947670056 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55de598b4eb0 con 0x55de599844a0 2023-07-21T18:43:46.504 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.475+0000 7f8e69ffb700 10 client.16239 handle_command_reply: tid=0 2023-07-21T18:43:46.504 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.475+0000 7f8e94730b80 1 client.16239 shutdown 2023-07-21T18:43:46.504 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.475+0000 7f8e94730b80 1 -- 172.21.15.39:0/4030474520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de59974ad0 msgr2=0x55de59972300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:46.504 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.475+0000 7f8e94730b80 1 --2- 172.21.15.39:0/4030474520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de59974ad0 0x55de59972300 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f8e78009790 tx=0x7f8e78007b50 comp rx=0 tx=0).stop 2023-07-21T18:43:46.505 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.475+0000 7f8e94730b80 1 -- 172.21.15.39:0/4030474520 shutdown_connections 2023-07-21T18:43:46.505 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.475+0000 7f8e94730b80 1 --2- 172.21.15.39:0/4030474520 >> [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] conn(0x55de599844a0 0x55de599868b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.505 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.475+0000 7f8e94730b80 1 --2- 172.21.15.39:0/4030474520 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de599692d0 0x55de59971880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.505 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.475+0000 7f8e94730b80 1 --2- 172.21.15.39:0/4030474520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de59974ad0 0x55de59972300 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.505 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.475+0000 7f8e69ffb700 0 client.16239 ms_handle_reset on v2:172.21.15.155:6836/947670056 2023-07-21T18:43:46.505 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.475+0000 7f8e94730b80 1 --2- 172.21.15.39:0/4030474520 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55de59969be0 0x55de59971dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.505 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.475+0000 7f8e94730b80 1 -- 172.21.15.39:0/4030474520 >> 172.21.15.39:0/4030474520 conn(0x55de5995fda0 msgr2=0x55de59965230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:46.505 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.475+0000 7f8e94730b80 1 -- 172.21.15.39:0/4030474520 shutdown_connections 2023-07-21T18:43:46.506 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.475+0000 7f8e94730b80 1 -- 172.21.15.39:0/4030474520 wait complete. 2023-07-21T18:43:46.506 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.475+0000 7f8e94730b80 20 client.16239 trim_cache size 0 max 16384 2023-07-21T18:43:46.507 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.488+0000 7f8e94730b80 1 Processor -- start 2023-07-21T18:43:46.507 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.488+0000 7f8e94730b80 1 -- start start 2023-07-21T18:43:46.508 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.488+0000 7f8e94730b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de599692d0 0x55de5997b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.508 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.488+0000 7f8e94730b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de59969be0 0x55de59972100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.509 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.488+0000 7f8e94730b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55de59974ad0 0x55de5996eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.509 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.488+0000 7f8e94730b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de5996b1a0 con 0x55de599692d0 2023-07-21T18:43:46.509 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.488+0000 7f8e94730b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de59981c30 con 0x55de59969be0 2023-07-21T18:43:46.509 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.488+0000 7f8e94730b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de59980f00 con 0x55de59974ad0 2023-07-21T18:43:46.509 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.488+0000 7f8e86346700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de599692d0 0x55de5997b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.509 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.489+0000 7f8e86346700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de599692d0 0x55de5997b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59134/0 (socket says 172.21.15.39:59134) 2023-07-21T18:43:46.509 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.489+0000 7f8e86346700 1 -- 172.21.15.39:0/4186287581 learned_addr learned my addr 172.21.15.39:0/4186287581 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:46.510 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.489+0000 7f8e86346700 1 -- 172.21.15.39:0/4186287581 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55de59974ad0 msgr2=0x55de5996eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:46.510 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.489+0000 7f8e86346700 1 --2- 172.21.15.39:0/4186287581 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55de59974ad0 0x55de5996eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.510 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.489+0000 7f8e86346700 1 -- 172.21.15.39:0/4186287581 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de59969be0 msgr2=0x55de59972100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:46.510 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.489+0000 7f8e86346700 1 --2- 172.21.15.39:0/4186287581 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de59969be0 0x55de59972100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.510 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.489+0000 7f8e86346700 1 -- 172.21.15.39:0/4186287581 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de59710250 con 0x55de599692d0 2023-07-21T18:43:46.510 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.489+0000 7f8e86346700 1 --2- 172.21.15.39:0/4186287581 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de599692d0 0x55de5997b4e0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f8e70015000 tx=0x7f8e70020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90cb40d45cefdafe server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:46.510 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.489+0000 7f8e757fa700 1 -- 172.21.15.39:0/4186287581 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7002d040 con 0x55de599692d0 2023-07-21T18:43:46.510 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.489+0000 7f8e757fa700 1 -- 172.21.15.39:0/4186287581 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e7001e040 con 0x55de599692d0 2023-07-21T18:43:46.511 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.489+0000 7f8e757fa700 1 -- 172.21.15.39:0/4186287581 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e70004e40 con 0x55de599692d0 2023-07-21T18:43:46.513 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.490+0000 7f8e94730b80 1 -- 172.21.15.39:0/4186287581 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de599692d0 msgr2=0x55de5997b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:46.514 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.490+0000 7f8e94730b80 1 --2- 172.21.15.39:0/4186287581 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de599692d0 0x55de5997b4e0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f8e70015000 tx=0x7f8e70020000 comp rx=0 tx=0).stop 2023-07-21T18:43:46.514 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.490+0000 7f8e94730b80 1 -- 172.21.15.39:0/4186287581 shutdown_connections 2023-07-21T18:43:46.514 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.490+0000 7f8e94730b80 1 --2- 172.21.15.39:0/4186287581 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55de59974ad0 0x55de5996eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.514 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.490+0000 7f8e94730b80 1 --2- 172.21.15.39:0/4186287581 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de599692d0 0x55de5997b4e0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.514 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.490+0000 7f8e94730b80 1 --2- 172.21.15.39:0/4186287581 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de59969be0 0x55de59972100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.514 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.490+0000 7f8e94730b80 1 -- 172.21.15.39:0/4186287581 >> 172.21.15.39:0/4186287581 conn(0x55de5995fda0 msgr2=0x55de5996cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:46.515 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.490+0000 7f8e94730b80 1 -- 172.21.15.39:0/4186287581 shutdown_connections 2023-07-21T18:43:46.515 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.490+0000 7f8e94730b80 1 -- 172.21.15.39:0/4186287581 wait complete. 2023-07-21T18:43:46.515 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.490+0000 7f8e94730b80 1 -- start start 2023-07-21T18:43:46.515 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.490+0000 7f8e94730b80 1 Processor -- start 2023-07-21T18:43:46.515 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.490+0000 7f8e94730b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de599692d0 0x55de5997aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.515 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.490+0000 7f8e94730b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de59969be0 0x55de5997b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.515 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.490+0000 7f8e94730b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55de59974ad0 0x55de599794d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.515 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.490+0000 7f8e94730b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de59979bd0 con 0x55de59969be0 2023-07-21T18:43:46.516 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.490+0000 7f8e94730b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de59979d40 con 0x55de599692d0 2023-07-21T18:43:46.516 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.490+0000 7f8e94730b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de59979f00 con 0x55de59974ad0 2023-07-21T18:43:46.516 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.491+0000 7f8e94730b80 10 client.0 _ll_register_callbacks cb 0x55de59859760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:43:46.516 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.491+0000 7f8e85b45700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de59969be0 0x55de5997b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.516 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.491+0000 7f8e86346700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de599692d0 0x55de5997aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.516 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.491+0000 7f8e86346700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de599692d0 0x55de5997aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:43504/0 (socket says 172.21.15.39:43504) 2023-07-21T18:43:46.516 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.491+0000 7f8e86346700 1 -- 172.21.15.39:0/2428616564 learned_addr learned my addr 172.21.15.39:0/2428616564 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:46.516 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.491+0000 7f8e85b45700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de59969be0 0x55de5997b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59142/0 (socket says 172.21.15.39:59142) 2023-07-21T18:43:46.517 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.491+0000 7f8e86b47700 1 --2- 172.21.15.39:0/2428616564 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55de59974ad0 0x55de599794d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.517 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.491+0000 7f8e85b45700 1 -- 172.21.15.39:0/2428616564 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55de59974ad0 msgr2=0x55de599794d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:46.517 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.491+0000 7f8e85b45700 1 --2- 172.21.15.39:0/2428616564 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55de59974ad0 0x55de599794d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.517 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.491+0000 7f8e85b45700 1 -- 172.21.15.39:0/2428616564 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de599692d0 msgr2=0x55de5997aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:46.517 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.491+0000 7f8e85b45700 1 --2- 172.21.15.39:0/2428616564 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de599692d0 0x55de5997aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.517 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.491+0000 7f8e85b45700 1 -- 172.21.15.39:0/2428616564 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e8010c110 con 0x55de59969be0 2023-07-21T18:43:46.517 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.491+0000 7f8e86346700 1 --2- 172.21.15.39:0/2428616564 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de599692d0 0x55de5997aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:43:46.518 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.491+0000 7f8e85b45700 1 --2- 172.21.15.39:0/2428616564 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de59969be0 0x55de5997b440 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x55de5996b350 tx=0x7f8e7c0999c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2edbe30810f23f5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:46.518 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.491+0000 7f8e69ffb700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:43:46.518 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.491+0000 7f8e69ffb700 1 -- 172.21.15.39:0/2428616564 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c0962f0 con 0x55de59969be0 2023-07-21T18:43:46.518 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.491+0000 7f8e69ffb700 1 -- 172.21.15.39:0/2428616564 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f8e7c0a2400 con 0x55de59969be0 2023-07-21T18:43:46.518 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.491+0000 7f8e94730b80 1 -- 172.21.15.39:0/2428616564 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8e7c010880 con 0x55de59969be0 2023-07-21T18:43:46.518 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.491+0000 7f8e69ffb700 1 -- 172.21.15.39:0/2428616564 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e7c0b82c0 con 0x55de59969be0 2023-07-21T18:43:46.518 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.491+0000 7f8e94730b80 10 client.16245 fetch_fsmap learned FSMap version 33 2023-07-21T18:43:46.518 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.491+0000 7f8e94730b80 1 -- 172.21.15.39:0/2428616564 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55de596e82b0 con 0x55de59969be0 2023-07-21T18:43:46.519 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.492+0000 7f8e69ffb700 1 -- 172.21.15.39:0/2428616564 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e7c010880 con 0x55de59969be0 2023-07-21T18:43:46.519 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.492+0000 7f8e69ffb700 1 -- 172.21.15.39:0/2428616564 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f8e7c0990f0 con 0x55de59969be0 2023-07-21T18:43:46.519 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.492+0000 7f8e94730b80 10 client.16245 fetch_fsmap finished waiting for FSMap version 33 2023-07-21T18:43:46.519 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.492+0000 7f8e94730b80 10 client.16245 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T18:43:46.519 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.492+0000 7f8e94730b80 20 client.16245 populate_metadata read hostname 'smithi039' 2023-07-21T18:43:46.519 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.492+0000 7f8e94730b80 1 --2- 172.21.15.39:0/2428616564 >> [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] conn(0x55de599844a0 0x55de599db160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.519 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.492+0000 7f8e94730b80 4 client.16245 mds_command: new command op to 24500 tid=0[{"prefix": "damage ls"}] 2023-07-21T18:43:46.519 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.492+0000 7f8e94730b80 1 -- 172.21.15.39:0/2428616564 --> [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55de59877050 con 0x55de599844a0 2023-07-21T18:43:46.520 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.492+0000 7f8e86346700 1 --2- 172.21.15.39:0/2428616564 >> [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] conn(0x55de599844a0 0x55de599db160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.520 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.493+0000 7f8e69ffb700 10 client.16245 ms_handle_connect on v2:172.21.15.155:6836/947670056 2023-07-21T18:43:46.520 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.493+0000 7f8e86346700 1 --2- 172.21.15.39:0/2428616564 >> [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] conn(0x55de599844a0 0x55de599db160 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f8c6cfeb25e16de5 server_cookie=5d22cfc6bc644aa8 in_seq=0 out_seq=0 2023-07-21T18:43:46.520 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.493+0000 7f8e69ffb700 1 -- 172.21.15.39:0/2428616564 <== mds.1 v2:172.21.15.155:6836/947670056 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+2603 (crc 0 0 0) 0x55de59877050 con 0x55de599844a0 2023-07-21T18:43:46.520 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.493+0000 7f8e69ffb700 10 client.16245 handle_command_reply: tid=0 2023-07-21T18:43:46.520 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.493+0000 7f8e94730b80 1 client.16245 shutdown 2023-07-21T18:43:46.520 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.493+0000 7f8e94730b80 1 -- 172.21.15.39:0/2428616564 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de59969be0 msgr2=0x55de5997b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:46.520 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.493+0000 7f8e94730b80 1 --2- 172.21.15.39:0/2428616564 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de59969be0 0x55de5997b440 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x55de5996b350 tx=0x7f8e7c0999c0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.521 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.493+0000 7f8e94730b80 1 -- 172.21.15.39:0/2428616564 shutdown_connections 2023-07-21T18:43:46.521 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.493+0000 7f8e94730b80 1 --2- 172.21.15.39:0/2428616564 >> [v2:172.21.15.155:6836/947670056,v1:172.21.15.155:6837/947670056] conn(0x55de599844a0 0x55de599db160 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.521 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.493+0000 7f8e94730b80 1 --2- 172.21.15.39:0/2428616564 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de599692d0 0x55de5997aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.521 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.493+0000 7f8e94730b80 1 --2- 172.21.15.39:0/2428616564 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55de59974ad0 0x55de599794d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.521 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.493+0000 7f8e69ffb700 0 client.16245 ms_handle_reset on v2:172.21.15.155:6836/947670056 2023-07-21T18:43:46.521 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.493+0000 7f8e94730b80 1 --2- 172.21.15.39:0/2428616564 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de59969be0 0x55de5997b440 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.521 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.493+0000 7f8e94730b80 1 -- 172.21.15.39:0/2428616564 >> 172.21.15.39:0/2428616564 conn(0x55de5995fda0 msgr2=0x55de59970050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:46.521 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.493+0000 7f8e94730b80 1 -- 172.21.15.39:0/2428616564 shutdown_connections 2023-07-21T18:43:46.522 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.493+0000 7f8e94730b80 1 -- 172.21.15.39:0/2428616564 wait complete. 2023-07-21T18:43:46.522 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.493+0000 7f8e94730b80 20 client.16245 trim_cache size 0 max 16384 2023-07-21T18:43:46.522 INFO:teuthology.orchestra.run.smithi039.stdout:[ 2023-07-21T18:43:46.522 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T18:43:46.522 INFO:teuthology.orchestra.run.smithi039.stdout: "damage_type": "backtrace", 2023-07-21T18:43:46.523 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 84880919, 2023-07-21T18:43:46.523 INFO:teuthology.orchestra.run.smithi039.stdout: "ino": 1099511628323, 2023-07-21T18:43:46.523 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/WORDPRO/RESULTS.XLS" 2023-07-21T18:43:46.523 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T18:43:46.523 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T18:43:46.523 INFO:teuthology.orchestra.run.smithi039.stdout: "damage_type": "backtrace", 2023-07-21T18:43:46.523 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 274735116, 2023-07-21T18:43:46.523 INFO:teuthology.orchestra.run.smithi039.stdout: "ino": 1099511628306, 2023-07-21T18:43:46.524 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/PWRPNT/PPTOOLS1.PPA" 2023-07-21T18:43:46.524 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T18:43:46.524 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T18:43:46.524 INFO:teuthology.orchestra.run.smithi039.stdout: "damage_type": "backtrace", 2023-07-21T18:43:46.524 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 382208560, 2023-07-21T18:43:46.524 INFO:teuthology.orchestra.run.smithi039.stdout: "ino": 1099511628304, 2023-07-21T18:43:46.524 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/PWRPNT/PCBENCHM.PPT" 2023-07-21T18:43:46.524 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T18:43:46.525 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T18:43:46.525 INFO:teuthology.orchestra.run.smithi039.stdout: "damage_type": "backtrace", 2023-07-21T18:43:46.525 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 506218746, 2023-07-21T18:43:46.525 INFO:teuthology.orchestra.run.smithi039.stdout: "ino": 1099511628353, 2023-07-21T18:43:46.525 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/SEED/SMALL.FIL" 2023-07-21T18:43:46.525 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T18:43:46.525 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T18:43:46.526 INFO:teuthology.orchestra.run.smithi039.stdout: "damage_type": "backtrace", 2023-07-21T18:43:46.526 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 1014379023, 2023-07-21T18:43:46.526 INFO:teuthology.orchestra.run.smithi039.stdout: "ino": 1099511628324, 2023-07-21T18:43:46.526 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/WORDPRO/NEWS1_1.LWP" 2023-07-21T18:43:46.526 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T18:43:46.526 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T18:43:46.526 INFO:teuthology.orchestra.run.smithi039.stdout: "damage_type": "backtrace", 2023-07-21T18:43:46.526 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 1125690066, 2023-07-21T18:43:46.527 INFO:teuthology.orchestra.run.smithi039.stdout: "ino": 1099511628308, 2023-07-21T18:43:46.527 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/PWRPNT/TRIDOTS.POT" 2023-07-21T18:43:46.527 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T18:43:46.527 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T18:43:46.527 INFO:teuthology.orchestra.run.smithi039.stdout: "damage_type": "backtrace", 2023-07-21T18:43:46.527 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 1176633375, 2023-07-21T18:43:46.527 INFO:teuthology.orchestra.run.smithi039.stdout: "ino": 1099511628307, 2023-07-21T18:43:46.527 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/PWRPNT/TIPS.PPT" 2023-07-21T18:43:46.528 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T18:43:46.528 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T18:43:46.528 INFO:teuthology.orchestra.run.smithi039.stdout: "damage_type": "backtrace", 2023-07-21T18:43:46.528 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 1553323092, 2023-07-21T18:43:46.528 INFO:teuthology.orchestra.run.smithi039.stdout: "ino": 1099511628305, 2023-07-21T18:43:46.528 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/PWRPNT/ZD16.BMP" 2023-07-21T18:43:46.528 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T18:43:46.528 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T18:43:46.529 INFO:teuthology.orchestra.run.smithi039.stdout: "damage_type": "backtrace", 2023-07-21T18:43:46.529 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 2348645306, 2023-07-21T18:43:46.529 INFO:teuthology.orchestra.run.smithi039.stdout: "ino": 1099511628355, 2023-07-21T18:43:46.529 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/SEED/LARGE.FIL" 2023-07-21T18:43:46.529 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T18:43:46.529 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T18:43:46.529 INFO:teuthology.orchestra.run.smithi039.stdout: "damage_type": "backtrace", 2023-07-21T18:43:46.529 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 3463820566, 2023-07-21T18:43:46.530 INFO:teuthology.orchestra.run.smithi039.stdout: "ino": 1099511628322, 2023-07-21T18:43:46.530 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/WORDPRO/BENCHS.LWP" 2023-07-21T18:43:46.530 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T18:43:46.530 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T18:43:46.530 INFO:teuthology.orchestra.run.smithi039.stdout: "damage_type": "backtrace", 2023-07-21T18:43:46.530 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 4181462961, 2023-07-21T18:43:46.530 INFO:teuthology.orchestra.run.smithi039.stdout: "ino": 1099511628354, 2023-07-21T18:43:46.530 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/SEED/MEDIUM.FIL" 2023-07-21T18:43:46.531 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T18:43:46.531 INFO:teuthology.orchestra.run.smithi039.stdout:] 2023-07-21T18:43:46.532 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.493+0000 7f8e885aa700 1 -- 172.21.15.39:0/2012910853 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f8e6c0a12c0 msgr2=0x7f8e6c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:46.532 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.494+0000 7f8e885aa700 1 --2- 172.21.15.39:0/2012910853 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f8e6c0a12c0 0x7f8e6c0a3770 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8e7000b2e0 tx=0x7f8e70005fd0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.532 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.494+0000 7f8e885aa700 1 -- 172.21.15.39:0/2012910853 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e800ff380 msgr2=0x7f8e80194d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:46.532 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.494+0000 7f8e885aa700 1 --2- 172.21.15.39:0/2012910853 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e800ff380 0x7f8e80194d40 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c00e9a0 tx=0x7f8e7c00ecb0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.532 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.494+0000 7f8e885aa700 1 -- 172.21.15.39:0/2012910853 shutdown_connections 2023-07-21T18:43:46.532 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.494+0000 7f8e885aa700 1 --2- 172.21.15.39:0/2012910853 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f8e6c0a12c0 0x7f8e6c0a3770 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.533 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.494+0000 7f8e885aa700 1 --2- 172.21.15.39:0/2012910853 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e800fe9d0 0x7f8e80194800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.533 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.494+0000 7f8e885aa700 1 --2- 172.21.15.39:0/2012910853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e801049e0 0x7f8e80195280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.533 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.494+0000 7f8e885aa700 1 --2- 172.21.15.39:0/2012910853 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e800ff380 0x7f8e80194d40 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.533 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.494+0000 7f8e885aa700 1 -- 172.21.15.39:0/2012910853 >> 172.21.15.39:0/2012910853 conn(0x7f8e800fa4b0 msgr2=0x7f8e800fbb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:46.533 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.494+0000 7f8e885aa700 1 -- 172.21.15.39:0/2012910853 shutdown_connections 2023-07-21T18:43:46.533 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.494+0000 7f8e885aa700 1 -- 172.21.15.39:0/2012910853 wait complete. 2023-07-21T18:43:46.534 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-21T18:43:46.662 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.643+0000 7f7f3f2a3700 1 -- 172.21.15.39:0/754695318 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f380ff380 msgr2=0x7f7f3810b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:46.662 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.643+0000 7f7f3f2a3700 1 --2- 172.21.15.39:0/754695318 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f380ff380 0x7f7f3810b810 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f7f3000b0a0 tx=0x7f7f3000b3b0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.663 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.643+0000 7f7f3f2a3700 1 -- 172.21.15.39:0/754695318 shutdown_connections 2023-07-21T18:43:46.663 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.643+0000 7f7f3f2a3700 1 --2- 172.21.15.39:0/754695318 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f380ff380 0x7f7f3810b810 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.663 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.643+0000 7f7f3f2a3700 1 --2- 172.21.15.39:0/754695318 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7f381049e0 0x7f7f38104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.663 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.643+0000 7f7f3f2a3700 1 --2- 172.21.15.39:0/754695318 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f380fe9d0 0x7f7f380fee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.663 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.643+0000 7f7f3f2a3700 1 -- 172.21.15.39:0/754695318 >> 172.21.15.39:0/754695318 conn(0x7f7f380fa4b0 msgr2=0x7f7f380fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:46.663 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.643+0000 7f7f3f2a3700 1 -- 172.21.15.39:0/754695318 shutdown_connections 2023-07-21T18:43:46.663 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.643+0000 7f7f3f2a3700 1 -- 172.21.15.39:0/754695318 wait complete. 2023-07-21T18:43:46.665 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.644+0000 7f7f3f2a3700 1 Processor -- start 2023-07-21T18:43:46.665 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.644+0000 7f7f3f2a3700 1 -- start start 2023-07-21T18:43:46.665 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.644+0000 7f7f3f2a3700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f380fe9d0 0x7f7f3819ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.666 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.644+0000 7f7f3f2a3700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f380ff380 0x7f7f3819bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.666 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.644+0000 7f7f3f2a3700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7f381049e0 0x7f7f38195a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.666 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.644+0000 7f7f3f2a3700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f381960e0 con 0x7f7f380fe9d0 2023-07-21T18:43:46.666 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.644+0000 7f7f3f2a3700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f38196250 con 0x7f7f380ff380 2023-07-21T18:43:46.666 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.644+0000 7f7f3f2a3700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f38196410 con 0x7f7f381049e0 2023-07-21T18:43:46.666 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.644+0000 7f7f3d03f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f380fe9d0 0x7f7f3819ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.666 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.644+0000 7f7f3d03f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f380fe9d0 0x7f7f3819ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59154/0 (socket says 172.21.15.39:59154) 2023-07-21T18:43:46.667 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.644+0000 7f7f3d03f700 1 -- 172.21.15.39:0/870294424 learned_addr learned my addr 172.21.15.39:0/870294424 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:46.667 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.644+0000 7f7f3d840700 1 --2- 172.21.15.39:0/870294424 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7f381049e0 0x7f7f38195a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.667 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.644+0000 7f7f3c83e700 1 --2- 172.21.15.39:0/870294424 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f380ff380 0x7f7f3819bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.667 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.644+0000 7f7f3d03f700 1 -- 172.21.15.39:0/870294424 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7f381049e0 msgr2=0x7f7f38195a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:46.667 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.644+0000 7f7f3d03f700 1 --2- 172.21.15.39:0/870294424 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7f381049e0 0x7f7f38195a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.667 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.644+0000 7f7f3d03f700 1 -- 172.21.15.39:0/870294424 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f380ff380 msgr2=0x7f7f3819bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:46.667 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.644+0000 7f7f3d03f700 1 --2- 172.21.15.39:0/870294424 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f380ff380 0x7f7f3819bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.667 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.644+0000 7f7f3d03f700 1 -- 172.21.15.39:0/870294424 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f30009d00 con 0x7f7f380fe9d0 2023-07-21T18:43:46.668 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.644+0000 7f7f3d03f700 1 --2- 172.21.15.39:0/870294424 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f380fe9d0 0x7f7f3819ba00 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f7f2800e970 tx=0x7f7f2800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:46.668 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.645+0000 7f7f2e7fc700 1 -- 172.21.15.39:0/870294424 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f28009790 con 0x7f7f380fe9d0 2023-07-21T18:43:46.668 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.645+0000 7f7f2e7fc700 1 -- 172.21.15.39:0/870294424 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f2800c7e0 con 0x7f7f380fe9d0 2023-07-21T18:43:46.668 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.645+0000 7f7f3f2a3700 1 -- 172.21.15.39:0/870294424 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f381966f0 con 0x7f7f380fe9d0 2023-07-21T18:43:46.668 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.645+0000 7f7f2e7fc700 1 -- 172.21.15.39:0/870294424 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f28003c70 con 0x7f7f380fe9d0 2023-07-21T18:43:46.668 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.645+0000 7f7f3f2a3700 1 -- 172.21.15.39:0/870294424 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f381a29f0 con 0x7f7f380fe9d0 2023-07-21T18:43:46.672 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.647+0000 7f7f2e7fc700 1 -- 172.21.15.39:0/870294424 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f7f2800cc50 con 0x7f7f380fe9d0 2023-07-21T18:43:46.672 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.648+0000 7f7f2e7fc700 1 --2- 172.21.15.39:0/870294424 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f7f240a12c0 0x7f7f240a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.673 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.648+0000 7f7f2e7fc700 1 -- 172.21.15.39:0/870294424 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f7f28014070 con 0x7f7f380fe9d0 2023-07-21T18:43:46.673 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.648+0000 7f7f3c83e700 1 --2- 172.21.15.39:0/870294424 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f7f240a12c0 0x7f7f240a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.673 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.649+0000 7f7f3c83e700 1 --2- 172.21.15.39:0/870294424 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f7f240a12c0 0x7f7f240a3770 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7f380fdd70 tx=0x7f7f34009260 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:46.673 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.649+0000 7f7f4b429b80 1 Processor -- start 2023-07-21T18:43:46.673 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.649+0000 7f7f4b429b80 1 -- start start 2023-07-21T18:43:46.673 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.649+0000 7f7f4b429b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566ca69e2d0 0x5566ca69e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.673 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.649+0000 7f7f4b429b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566ca69ebe0 0x5566ca6a9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.673 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.649+0000 7f7f4b429b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566ca6a9ad0 0x5566ca6abeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.674 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.649+0000 7f7f4b429b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566ca5a4cb0 con 0x5566ca6a9ad0 2023-07-21T18:43:46.674 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.649+0000 7f7f4b429b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566ca5f7a40 con 0x5566ca69e2d0 2023-07-21T18:43:46.674 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.649+0000 7f7f4b429b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566ca5f0ed0 con 0x5566ca69ebe0 2023-07-21T18:43:46.674 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.649+0000 7f7f3d03f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566ca69e2d0 0x5566ca69e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.674 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.649+0000 7f7f3d840700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566ca6a9ad0 0x5566ca6abeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.674 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.649+0000 7f7f3d03f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566ca69e2d0 0x5566ca69e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:43526/0 (socket says 172.21.15.39:43526) 2023-07-21T18:43:46.674 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.649+0000 7f7f3d03f700 1 -- 172.21.15.39:0/687132594 learned_addr learned my addr 172.21.15.39:0/687132594 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:46.674 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.649+0000 7f7f3c83e700 1 --2- 172.21.15.39:0/687132594 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566ca69ebe0 0x5566ca6a9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.675 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.650+0000 7f7f3d840700 1 -- 172.21.15.39:0/687132594 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566ca69ebe0 msgr2=0x5566ca6a9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:46.675 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.650+0000 7f7f3d840700 1 --2- 172.21.15.39:0/687132594 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566ca69ebe0 0x5566ca6a9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.675 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.650+0000 7f7f3d840700 1 -- 172.21.15.39:0/687132594 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566ca69e2d0 msgr2=0x5566ca69e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:46.675 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.650+0000 7f7f3d840700 1 --2- 172.21.15.39:0/687132594 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566ca69e2d0 0x5566ca69e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.675 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.650+0000 7f7f3d840700 1 -- 172.21.15.39:0/687132594 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5566ca41ede0 con 0x5566ca6a9ad0 2023-07-21T18:43:46.675 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.650+0000 7f7f3d03f700 1 --2- 172.21.15.39:0/687132594 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566ca69e2d0 0x5566ca69e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:43:46.675 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.650+0000 7f7f3d840700 1 --2- 172.21.15.39:0/687132594 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566ca6a9ad0 0x5566ca6abeb0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f7f3819c700 tx=0x7f7f30009c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6799f311ba1733d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:46.676 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.650+0000 7f7f237fe700 1 -- 172.21.15.39:0/687132594 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3000e040 con 0x5566ca6a9ad0 2023-07-21T18:43:46.676 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.650+0000 7f7f237fe700 1 -- 172.21.15.39:0/687132594 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f30009460 con 0x5566ca6a9ad0 2023-07-21T18:43:46.676 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.650+0000 7f7f237fe700 1 -- 172.21.15.39:0/687132594 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3001c6a0 con 0x5566ca6a9ad0 2023-07-21T18:43:46.676 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.651+0000 7f7f4b429b80 1 -- 172.21.15.39:0/687132594 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566ca6a9ad0 msgr2=0x5566ca6abeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:46.676 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.651+0000 7f7f4b429b80 1 --2- 172.21.15.39:0/687132594 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566ca6a9ad0 0x5566ca6abeb0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f7f3819c700 tx=0x7f7f30009c60 comp rx=0 tx=0).stop 2023-07-21T18:43:46.676 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.651+0000 7f7f4b429b80 1 -- 172.21.15.39:0/687132594 shutdown_connections 2023-07-21T18:43:46.676 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.651+0000 7f7f4b429b80 1 --2- 172.21.15.39:0/687132594 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566ca69e2d0 0x5566ca69e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.677 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.651+0000 7f7f4b429b80 1 --2- 172.21.15.39:0/687132594 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566ca6a9ad0 0x5566ca6abeb0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.677 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.651+0000 7f7f4b429b80 1 --2- 172.21.15.39:0/687132594 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566ca69ebe0 0x5566ca6a9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.677 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.651+0000 7f7f4b429b80 1 -- 172.21.15.39:0/687132594 >> 172.21.15.39:0/687132594 conn(0x5566ca694da0 msgr2=0x5566ca69a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:46.677 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.651+0000 7f7f4b429b80 1 -- 172.21.15.39:0/687132594 shutdown_connections 2023-07-21T18:43:46.677 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.651+0000 7f7f4b429b80 1 -- 172.21.15.39:0/687132594 wait complete. 2023-07-21T18:43:46.677 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.651+0000 7f7f4b429b80 1 -- start start 2023-07-21T18:43:46.677 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.651+0000 7f7f4b429b80 1 Processor -- start 2023-07-21T18:43:46.677 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.652+0000 7f7f4b429b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566ca69e2d0 0x5566ca6a9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.678 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.652+0000 7f7f4b429b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566ca69ebe0 0x5566ca6a6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.678 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.652+0000 7f7f4b429b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566ca6a9ad0 0x5566ca6a6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.678 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.652+0000 7f7f4b429b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566ca590ac0 con 0x5566ca6a9ad0 2023-07-21T18:43:46.678 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.652+0000 7f7f4b429b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566ca5907c0 con 0x5566ca69ebe0 2023-07-21T18:43:46.678 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.652+0000 7f7f4b429b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566ca474920 con 0x5566ca69e2d0 2023-07-21T18:43:46.678 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.652+0000 7f7f4b429b80 10 client.0 _ll_register_callbacks cb 0x5566ca58e760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:43:46.678 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.652+0000 7f7f3d840700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566ca6a9ad0 0x5566ca6a6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.678 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.652+0000 7f7f3d840700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566ca6a9ad0 0x5566ca6a6e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59172/0 (socket says 172.21.15.39:59172) 2023-07-21T18:43:46.679 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.652+0000 7f7f3d840700 1 -- 172.21.15.39:0/1221635405 learned_addr learned my addr 172.21.15.39:0/1221635405 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:46.679 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.652+0000 7f7f3d03f700 1 --2- 172.21.15.39:0/1221635405 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566ca69e2d0 0x5566ca6a9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.679 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.652+0000 7f7f3c83e700 1 --2- 172.21.15.39:0/1221635405 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566ca69ebe0 0x5566ca6a6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.679 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.652+0000 7f7f3d840700 1 -- 172.21.15.39:0/1221635405 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566ca69e2d0 msgr2=0x5566ca6a9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:46.679 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.652+0000 7f7f3d840700 1 --2- 172.21.15.39:0/1221635405 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566ca69e2d0 0x5566ca6a9520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.679 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.652+0000 7f7f3d840700 1 -- 172.21.15.39:0/1221635405 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566ca69ebe0 msgr2=0x5566ca6a6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:46.679 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.652+0000 7f7f3d840700 1 --2- 172.21.15.39:0/1221635405 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566ca69ebe0 0x5566ca6a6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.680 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.652+0000 7f7f3d840700 1 -- 172.21.15.39:0/1221635405 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f381a29f0 con 0x5566ca6a9ad0 2023-07-21T18:43:46.680 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.652+0000 7f7f3d03f700 1 --2- 172.21.15.39:0/1221635405 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566ca69e2d0 0x5566ca6a9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:43:46.680 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.652+0000 7f7f3d840700 1 --2- 172.21.15.39:0/1221635405 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566ca6a9ad0 0x5566ca6a6e70 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f7f30009bb0 tx=0x7f7f3000bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=494fb99ffd09da0c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:46.680 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.652+0000 7f7f20ff9700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:43:46.680 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.652+0000 7f7f20ff9700 1 -- 172.21.15.39:0/1221635405 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3000e040 con 0x5566ca6a9ad0 2023-07-21T18:43:46.680 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.652+0000 7f7f20ff9700 1 -- 172.21.15.39:0/1221635405 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f7f30003c00 con 0x5566ca6a9ad0 2023-07-21T18:43:46.680 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.652+0000 7f7f4b429b80 1 -- 172.21.15.39:0/1221635405 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5566ca54a4f0 con 0x5566ca6a9ad0 2023-07-21T18:43:46.680 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.653+0000 7f7f20ff9700 1 -- 172.21.15.39:0/1221635405 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f30003dc0 con 0x5566ca6a9ad0 2023-07-21T18:43:46.681 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.653+0000 7f7f4b429b80 10 client.16257 fetch_fsmap learned FSMap version 33 2023-07-21T18:43:46.681 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.653+0000 7f7f4b429b80 1 -- 172.21.15.39:0/1221635405 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7f7f3810c110 con 0x5566ca6a9ad0 2023-07-21T18:43:46.681 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.653+0000 7f7f20ff9700 1 -- 172.21.15.39:0/1221635405 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5566ca54a4f0 con 0x5566ca6a9ad0 2023-07-21T18:43:46.681 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.653+0000 7f7f20ff9700 1 -- 172.21.15.39:0/1221635405 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f7f30019030 con 0x5566ca6a9ad0 2023-07-21T18:43:46.681 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.653+0000 7f7f4b429b80 10 client.16257 fetch_fsmap finished waiting for FSMap version 33 2023-07-21T18:43:46.681 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.653+0000 7f7f4b429b80 10 client.16257 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T18:43:46.681 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.653+0000 7f7f4b429b80 20 client.16257 populate_metadata read hostname 'smithi039' 2023-07-21T18:43:46.682 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.653+0000 7f7f4b429b80 1 --2- 172.21.15.39:0/1221635405 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5566ca6b9060 0x5566ca6bb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.682 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.653+0000 7f7f4b429b80 4 client.16257 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:43:46.682 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.653+0000 7f7f4b429b80 1 -- 172.21.15.39:0/1221635405 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5566ca5e9eb0 con 0x5566ca6b9060 2023-07-21T18:43:46.682 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.653+0000 7f7f3d03f700 1 --2- 172.21.15.39:0/1221635405 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5566ca6b9060 0x5566ca6bb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.682 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.653+0000 7f7f20ff9700 10 client.16257 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:43:46.682 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.653+0000 7f7f3d03f700 1 --2- 172.21.15.39:0/1221635405 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5566ca6b9060 0x5566ca6bb470 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4aaae6908963ea75 server_cookie=f004fb63ee189f3e in_seq=0 out_seq=0 2023-07-21T18:43:46.683 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.655+0000 7f7f20ff9700 1 -- 172.21.15.39:0/1221635405 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5566ca5e9eb0 con 0x5566ca6b9060 2023-07-21T18:43:46.683 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.655+0000 7f7f20ff9700 10 client.16257 handle_command_reply: tid=0 2023-07-21T18:43:46.683 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.655+0000 7f7f4b429b80 1 client.16257 shutdown 2023-07-21T18:43:46.683 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.655+0000 7f7f4b429b80 1 -- 172.21.15.39:0/1221635405 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566ca6a9ad0 msgr2=0x5566ca6a6e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:46.684 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.655+0000 7f7f4b429b80 1 --2- 172.21.15.39:0/1221635405 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566ca6a9ad0 0x5566ca6a6e70 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f7f30009bb0 tx=0x7f7f3000bce0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.684 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.655+0000 7f7f4b429b80 1 -- 172.21.15.39:0/1221635405 shutdown_connections 2023-07-21T18:43:46.684 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.655+0000 7f7f4b429b80 1 --2- 172.21.15.39:0/1221635405 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5566ca6b9060 0x5566ca6bb470 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.684 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.655+0000 7f7f4b429b80 1 --2- 172.21.15.39:0/1221635405 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566ca6a9ad0 0x5566ca6a6e70 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.684 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.655+0000 7f7f20ff9700 0 client.16257 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:43:46.684 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.655+0000 7f7f4b429b80 1 --2- 172.21.15.39:0/1221635405 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566ca69e2d0 0x5566ca6a9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.684 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.655+0000 7f7f4b429b80 1 --2- 172.21.15.39:0/1221635405 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566ca69ebe0 0x5566ca6a6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.685 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.655+0000 7f7f4b429b80 1 -- 172.21.15.39:0/1221635405 >> 172.21.15.39:0/1221635405 conn(0x5566ca694da0 msgr2=0x5566ca69a290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:46.685 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.656+0000 7f7f4b429b80 1 -- 172.21.15.39:0/1221635405 shutdown_connections 2023-07-21T18:43:46.685 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.656+0000 7f7f4b429b80 1 -- 172.21.15.39:0/1221635405 wait complete. 2023-07-21T18:43:46.685 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.656+0000 7f7f4b429b80 20 client.16257 trim_cache size 0 max 16384 2023-07-21T18:43:46.688 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.669+0000 7f7f4b429b80 1 Processor -- start 2023-07-21T18:43:46.688 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.669+0000 7f7f4b429b80 1 -- start start 2023-07-21T18:43:46.688 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.669+0000 7f7f4b429b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566ca69e2d0 0x5566ca6a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.689 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.669+0000 7f7f4b429b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566ca69ebe0 0x5566ca6a9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.689 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.669+0000 7f7f4b429b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566ca6a9ad0 0x5566ca6a5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.689 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.669+0000 7f7f4b429b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566ca69bd70 con 0x5566ca69ebe0 2023-07-21T18:43:46.689 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.669+0000 7f7f4b429b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566ca69f570 con 0x5566ca6a9ad0 2023-07-21T18:43:46.689 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.669+0000 7f7f4b429b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566ca6a7450 con 0x5566ca69e2d0 2023-07-21T18:43:46.689 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.669+0000 7f7f3c83e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566ca69ebe0 0x5566ca6a9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.689 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.669+0000 7f7f3c83e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566ca69ebe0 0x5566ca6a9050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59178/0 (socket says 172.21.15.39:59178) 2023-07-21T18:43:46.689 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.669+0000 7f7f3c83e700 1 -- 172.21.15.39:0/523624554 learned_addr learned my addr 172.21.15.39:0/523624554 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:46.690 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.669+0000 7f7f3d03f700 1 --2- 172.21.15.39:0/523624554 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566ca69e2d0 0x5566ca6a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.690 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.669+0000 7f7f3c83e700 1 -- 172.21.15.39:0/523624554 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566ca69e2d0 msgr2=0x5566ca6a5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:46.690 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.669+0000 7f7f3c83e700 1 --2- 172.21.15.39:0/523624554 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566ca69e2d0 0x5566ca6a5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.690 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.669+0000 7f7f3c83e700 1 -- 172.21.15.39:0/523624554 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566ca6a9ad0 msgr2=0x5566ca6a5ff0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T18:43:46.690 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.669+0000 7f7f3c83e700 1 --2- 172.21.15.39:0/523624554 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566ca6a9ad0 0x5566ca6a5ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.690 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.669+0000 7f7f3c83e700 1 -- 172.21.15.39:0/523624554 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5566ca407c60 con 0x5566ca69ebe0 2023-07-21T18:43:46.693 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.669+0000 7f7f3c83e700 1 --2- 172.21.15.39:0/523624554 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566ca69ebe0 0x5566ca6a9050 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f7f34011d80 tx=0x7f7f34014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1c814b96673ffbc server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:46.694 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.670+0000 7f7f23fff700 1 -- 172.21.15.39:0/523624554 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f34023070 con 0x5566ca69ebe0 2023-07-21T18:43:46.694 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.670+0000 7f7f23fff700 1 -- 172.21.15.39:0/523624554 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f34011e70 con 0x5566ca69ebe0 2023-07-21T18:43:46.694 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.670+0000 7f7f23fff700 1 -- 172.21.15.39:0/523624554 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f34027da0 con 0x5566ca69ebe0 2023-07-21T18:43:46.694 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.670+0000 7f7f4b429b80 1 -- 172.21.15.39:0/523624554 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566ca69ebe0 msgr2=0x5566ca6a9050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:46.694 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.670+0000 7f7f4b429b80 1 --2- 172.21.15.39:0/523624554 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566ca69ebe0 0x5566ca6a9050 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f7f34011d80 tx=0x7f7f34014000 comp rx=0 tx=0).stop 2023-07-21T18:43:46.694 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.670+0000 7f7f4b429b80 1 -- 172.21.15.39:0/523624554 shutdown_connections 2023-07-21T18:43:46.694 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.670+0000 7f7f4b429b80 1 --2- 172.21.15.39:0/523624554 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566ca6a9ad0 0x5566ca6a5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.695 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.670+0000 7f7f4b429b80 1 --2- 172.21.15.39:0/523624554 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566ca69e2d0 0x5566ca6a5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.695 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.670+0000 7f7f4b429b80 1 --2- 172.21.15.39:0/523624554 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566ca69ebe0 0x5566ca6a9050 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.695 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.670+0000 7f7f4b429b80 1 -- 172.21.15.39:0/523624554 >> 172.21.15.39:0/523624554 conn(0x5566ca694da0 msgr2=0x5566ca6b5780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:46.695 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.670+0000 7f7f4b429b80 1 -- 172.21.15.39:0/523624554 shutdown_connections 2023-07-21T18:43:46.695 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.670+0000 7f7f4b429b80 1 -- 172.21.15.39:0/523624554 wait complete. 2023-07-21T18:43:46.695 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.671+0000 7f7f4b429b80 1 -- start start 2023-07-21T18:43:46.695 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.671+0000 7f7f4b429b80 1 Processor -- start 2023-07-21T18:43:46.695 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.671+0000 7f7f4b429b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566ca69e2d0 0x5566ca6b01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.696 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.671+0000 7f7f4b429b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566ca69ebe0 0x5566ca6ae4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.696 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.671+0000 7f7f4b429b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566ca6a9ad0 0x5566ca6aea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.696 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.671+0000 7f7f4b429b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566ca6b0ad0 con 0x5566ca6a9ad0 2023-07-21T18:43:46.696 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.671+0000 7f7f4b429b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566ca6aef70 con 0x5566ca69ebe0 2023-07-21T18:43:46.696 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.671+0000 7f7f4b429b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566ca6af0e0 con 0x5566ca69e2d0 2023-07-21T18:43:46.696 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.671+0000 7f7f4b429b80 10 client.0 _ll_register_callbacks cb 0x7f7f28095c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:43:46.696 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.671+0000 7f7f3d840700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566ca6a9ad0 0x5566ca6aea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.697 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.671+0000 7f7f3d840700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566ca6a9ad0 0x5566ca6aea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59186/0 (socket says 172.21.15.39:59186) 2023-07-21T18:43:46.697 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.671+0000 7f7f3d840700 1 -- 172.21.15.39:0/169211518 learned_addr learned my addr 172.21.15.39:0/169211518 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:46.697 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.671+0000 7f7f3c83e700 1 --2- 172.21.15.39:0/169211518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566ca69ebe0 0x5566ca6ae4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.697 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.671+0000 7f7f3d03f700 1 --2- 172.21.15.39:0/169211518 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566ca69e2d0 0x5566ca6b01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.697 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.672+0000 7f7f3d840700 1 -- 172.21.15.39:0/169211518 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566ca69e2d0 msgr2=0x5566ca6b01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:46.697 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.672+0000 7f7f3d840700 1 --2- 172.21.15.39:0/169211518 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566ca69e2d0 0x5566ca6b01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.697 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.672+0000 7f7f3d840700 1 -- 172.21.15.39:0/169211518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566ca69ebe0 msgr2=0x5566ca6ae4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:46.698 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.672+0000 7f7f3d840700 1 --2- 172.21.15.39:0/169211518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566ca69ebe0 0x5566ca6ae4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.698 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.672+0000 7f7f3d840700 1 -- 172.21.15.39:0/169211518 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f34017070 con 0x5566ca6a9ad0 2023-07-21T18:43:46.698 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.672+0000 7f7f3d840700 1 --2- 172.21.15.39:0/169211518 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566ca6a9ad0 0x5566ca6aea30 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f7f3000b3f0 tx=0x7f7f300021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc36582fd6d96fd6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:46.698 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.672+0000 7f7f20ff9700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:43:46.698 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.672+0000 7f7f20ff9700 1 -- 172.21.15.39:0/169211518 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3000e040 con 0x5566ca6a9ad0 2023-07-21T18:43:46.698 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.672+0000 7f7f20ff9700 1 -- 172.21.15.39:0/169211518 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f7f3001ce40 con 0x5566ca6a9ad0 2023-07-21T18:43:46.698 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.672+0000 7f7f4b429b80 1 -- 172.21.15.39:0/169211518 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7f30009460 con 0x5566ca6a9ad0 2023-07-21T18:43:46.698 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.672+0000 7f7f20ff9700 1 -- 172.21.15.39:0/169211518 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f3001cce0 con 0x5566ca6a9ad0 2023-07-21T18:43:46.699 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.672+0000 7f7f4b429b80 10 client.16263 fetch_fsmap learned FSMap version 33 2023-07-21T18:43:46.699 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.672+0000 7f7f4b429b80 1 -- 172.21.15.39:0/169211518 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5566ca445250 con 0x5566ca6a9ad0 2023-07-21T18:43:46.699 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.672+0000 7f7f20ff9700 1 -- 172.21.15.39:0/169211518 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f30009460 con 0x5566ca6a9ad0 2023-07-21T18:43:46.699 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.672+0000 7f7f20ff9700 1 -- 172.21.15.39:0/169211518 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f7f30019070 con 0x5566ca6a9ad0 2023-07-21T18:43:46.699 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.672+0000 7f7f4b429b80 10 client.16263 fetch_fsmap finished waiting for FSMap version 33 2023-07-21T18:43:46.699 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.673+0000 7f7f4b429b80 10 client.16263 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T18:43:46.699 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.673+0000 7f7f4b429b80 20 client.16263 populate_metadata read hostname 'smithi039' 2023-07-21T18:43:46.699 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.673+0000 7f7f4b429b80 1 --2- 172.21.15.39:0/169211518 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5566ca70dea0 0x5566ca7102b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:46.700 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.673+0000 7f7f4b429b80 4 client.16263 mds_command: new command op to 24536 tid=0[{"prefix": "damage ls"}] 2023-07-21T18:43:46.700 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.673+0000 7f7f4b429b80 1 -- 172.21.15.39:0/169211518 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5566ca5ac050 con 0x5566ca70dea0 2023-07-21T18:43:46.700 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.673+0000 7f7f3d03f700 1 --2- 172.21.15.39:0/169211518 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5566ca70dea0 0x5566ca7102b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:46.700 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.673+0000 7f7f3d03f700 1 --2- 172.21.15.39:0/169211518 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5566ca70dea0 0x5566ca7102b0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f62633df339a2bbb server_cookie=b502cdd3c225c225 in_seq=0 out_seq=0 2023-07-21T18:43:46.700 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.673+0000 7f7f20ff9700 10 client.16263 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:43:46.700 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.673+0000 7f7f20ff9700 1 -- 172.21.15.39:0/169211518 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5566ca5ac050 con 0x5566ca70dea0 2023-07-21T18:43:46.700 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.673+0000 7f7f20ff9700 10 client.16263 handle_command_reply: tid=0 2023-07-21T18:43:46.701 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.673+0000 7f7f4b429b80 1 client.16263 shutdown 2023-07-21T18:43:46.701 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.673+0000 7f7f4b429b80 1 -- 172.21.15.39:0/169211518 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566ca6a9ad0 msgr2=0x5566ca6aea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:46.701 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.673+0000 7f7f4b429b80 1 --2- 172.21.15.39:0/169211518 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566ca6a9ad0 0x5566ca6aea30 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f7f3000b3f0 tx=0x7f7f300021d0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.701 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.673+0000 7f7f4b429b80 1 -- 172.21.15.39:0/169211518 shutdown_connections 2023-07-21T18:43:46.701 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.673+0000 7f7f4b429b80 1 --2- 172.21.15.39:0/169211518 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5566ca70dea0 0x5566ca7102b0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.701 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.673+0000 7f7f4b429b80 1 --2- 172.21.15.39:0/169211518 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566ca6a9ad0 0x5566ca6aea30 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.701 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.673+0000 7f7f4b429b80 1 --2- 172.21.15.39:0/169211518 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566ca69e2d0 0x5566ca6b01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.701 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.673+0000 7f7f20ff9700 0 client.16263 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:43:46.702 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.673+0000 7f7f4b429b80 1 --2- 172.21.15.39:0/169211518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566ca69ebe0 0x5566ca6ae4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.702 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.673+0000 7f7f4b429b80 1 -- 172.21.15.39:0/169211518 >> 172.21.15.39:0/169211518 conn(0x5566ca694da0 msgr2=0x5566ca6a5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:46.702 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.674+0000 7f7f4b429b80 1 -- 172.21.15.39:0/169211518 shutdown_connections 2023-07-21T18:43:46.702 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.674+0000 7f7f4b429b80 1 -- 172.21.15.39:0/169211518 wait complete. 2023-07-21T18:43:46.702 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.674+0000 7f7f4b429b80 20 client.16263 trim_cache size 0 max 16384 2023-07-21T18:43:46.702 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2023-07-21T18:43:46.703 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.674+0000 7f7f3f2a3700 1 -- 172.21.15.39:0/870294424 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f7f240a12c0 msgr2=0x7f7f240a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:46.704 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.674+0000 7f7f3f2a3700 1 --2- 172.21.15.39:0/870294424 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f7f240a12c0 0x7f7f240a3770 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7f380fdd70 tx=0x7f7f34009260 comp rx=0 tx=0).stop 2023-07-21T18:43:46.704 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.674+0000 7f7f3f2a3700 1 -- 172.21.15.39:0/870294424 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f380fe9d0 msgr2=0x7f7f3819ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:46.704 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.674+0000 7f7f3f2a3700 1 --2- 172.21.15.39:0/870294424 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f380fe9d0 0x7f7f3819ba00 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f7f2800e970 tx=0x7f7f2800ec80 comp rx=0 tx=0).stop 2023-07-21T18:43:46.705 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.674+0000 7f7f3f2a3700 1 -- 172.21.15.39:0/870294424 shutdown_connections 2023-07-21T18:43:46.705 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.674+0000 7f7f3f2a3700 1 --2- 172.21.15.39:0/870294424 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7f381049e0 0x7f7f38195a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.705 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.674+0000 7f7f3f2a3700 1 --2- 172.21.15.39:0/870294424 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f380fe9d0 0x7f7f3819ba00 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.706 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.674+0000 7f7f3f2a3700 1 --2- 172.21.15.39:0/870294424 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f7f240a12c0 0x7f7f240a3770 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.706 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.674+0000 7f7f3f2a3700 1 --2- 172.21.15.39:0/870294424 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f380ff380 0x7f7f3819bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:46.706 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.674+0000 7f7f3f2a3700 1 -- 172.21.15.39:0/870294424 >> 172.21.15.39:0/870294424 conn(0x7f7f380fa4b0 msgr2=0x7f7f380fbe30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:46.706 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.674+0000 7f7f3f2a3700 1 -- 172.21.15.39:0/870294424 shutdown_connections 2023-07-21T18:43:46.707 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T18:43:46.674+0000 7f7f3f2a3700 1 -- 172.21.15.39:0/870294424 wait complete. 2023-07-21T18:43:46.708 ERROR:tasks.fwd_scrub:rank 4 damaged: [{'damage_type': 'backtrace', 'id': 284680805, 'ino': 1099511628351, 'path': '/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/ACCESS/FASTENER.MDB'}] 2023-07-21T18:43:46.709 ERROR:tasks.fwd_scrub:rank 2 damaged: [{'damage_type': 'backtrace', 'id': 32841408, 'ino': 1099511628298, 'path': '/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/WORD/BASEMACH.DOC'}, {'damage_type': 'backtrace', 'id': 642881598, 'ino': 1099511628299, 'path': '/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/WORD/FACTS.DOC'}, {'damage_type': 'backtrace', 'id': 695121249, 'ino': 1099511628313, 'path': '/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC1.CDR'}, {'damage_type': 'backtrace', 'id': 999289580, 'ino': 1099511628315, 'path': '/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/COREL/TEAPOT2.CPT'}, {'damage_type': 'backtrace', 'id': 1470998723, 'ino': 1099511628302, 'path': '/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/EXCEL/SALES.XLS'}, {'damage_type': 'backtrace', 'id': 1538270232, 'ino': 1099511628310, 'path': '/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/COREL/CDRROLS.CFG'}, {'damage_type': 'backtrace', 'id': 1735867768, 'ino': 1099511628314, 'path': '/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/COREL/MAILBOX2.CPT'}, {'damage_type': 'backtrace', 'id': 1811568482, 'ino': 1099511628317, 'path': '/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/COREL/TRAIN.CDR'}, {'damage_type': 'backtrace', 'id': 2089966051, 'ino': 1099511628319, 'path': '/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/COREL/ROSE.CPT'}, {'damage_type': 'backtrace', 'id': 2752898913, 'ino': 1099511628327, 'path': '/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/PM/EVENTS.DOC'}, {'damage_type': 'backtrace', 'id': 2944040979, 'ino': 1099511628311, 'path': '/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/COREL/CDRBARS.CFG'}, {'damage_type': 'backtrace', 'id': 3256400975, 'ino': 1099511628326, 'path': '/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/PM/T1.XLS'}, {'damage_type': 'backtrace', 'id': 3354151126, 'ino': 1099511628320, 'path': '/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/COREL/CORELDRW.CDT'}, {'damage_type': 'backtrace', 'id': 3372412515, 'ino': 1099511628297, 'path': '/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/WORD/CHAP10.DOC'}, {'damage_type': 'backtrace', 'id': 3643633672, 'ino': 1099511628328, 'path': '/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/PM/MOVED.DOC'}, {'damage_type': 'backtrace', 'id': 3913448916, 'ino': 1099511628301, 'path': '/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/EXCEL/PCMAGCD.XLS'}, {'damage_type': 'backtrace', 'id': 3958397844, 'ino': 1099511628318, 'path': '/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC2.CDR'}, {'damage_type': 'backtrace', 'id': 4216958244, 'ino': 1099511628316, 'path': '/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/COREL/SIGN2.CPT'}, {'damage_type': 'backtrace', 'id': 4245475529, 'ino': 1099511628312, 'path': '/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/COREL/ARTISTIC.ACL'}] 2023-07-21T18:43:46.709 ERROR:tasks.fwd_scrub:rank 1 damaged: [{'damage_type': 'backtrace', 'id': 84880919, 'ino': 1099511628323, 'path': '/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/WORDPRO/RESULTS.XLS'}, {'damage_type': 'backtrace', 'id': 274735116, 'ino': 1099511628306, 'path': '/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/PWRPNT/PPTOOLS1.PPA'}, {'damage_type': 'backtrace', 'id': 382208560, 'ino': 1099511628304, 'path': '/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/PWRPNT/PCBENCHM.PPT'}, {'damage_type': 'backtrace', 'id': 506218746, 'ino': 1099511628353, 'path': '/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/SEED/SMALL.FIL'}, {'damage_type': 'backtrace', 'id': 1014379023, 'ino': 1099511628324, 'path': '/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/WORDPRO/NEWS1_1.LWP'}, {'damage_type': 'backtrace', 'id': 1125690066, 'ino': 1099511628308, 'path': '/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/PWRPNT/TRIDOTS.POT'}, {'damage_type': 'backtrace', 'id': 1176633375, 'ino': 1099511628307, 'path': '/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/PWRPNT/TIPS.PPT'}, {'damage_type': 'backtrace', 'id': 1553323092, 'ino': 1099511628305, 'path': '/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/PWRPNT/ZD16.BMP'}, {'damage_type': 'backtrace', 'id': 2348645306, 'ino': 1099511628355, 'path': '/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/SEED/LARGE.FIL'}, {'damage_type': 'backtrace', 'id': 3463820566, 'ino': 1099511628322, 'path': '/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/WORDPRO/BENCHS.LWP'}, {'damage_type': 'backtrace', 'id': 4181462961, 'ino': 1099511628354, 'path': '/volumes/_nogroup/sv_1/8d6c01e4-c829-4108-ba84-43703682343a/client.0/tmp/clients/client0/~dmtmp/SEED/MEDIUM.FIL'}] 2023-07-21T18:43:46.710 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_81cf785aa7781c4e08381c540f890cb9bc987ef9/qa/tasks/fwd_scrub.py", line 38, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_81cf785aa7781c4e08381c540f890cb9bc987ef9/qa/tasks/fwd_scrub.py", line 55, in do_scrub self._scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_81cf785aa7781c4e08381c540f890cb9bc987ef9/qa/tasks/fwd_scrub.py", line 79, in _scrub self._check_damage() File "/home/teuthworker/src/git.ceph.com_ceph-c_81cf785aa7781c4e08381c540f890cb9bc987ef9/qa/tasks/fwd_scrub.py", line 90, in _check_damage raise RuntimeError(f"rank damage found: {types}") RuntimeError: rank damage found: {'backtrace'} 2023-07-21T18:43:47.262 INFO:tasks.workunit.client.0.smithi039.stdout: 1 43264 4.18 MB/sec warmup 60 sec latency 965.331 ms 2023-07-21T18:43:48.262 INFO:tasks.workunit.client.0.smithi039.stdout: 1 43264 4.11 MB/sec warmup 61 sec latency 1965.380 ms 2023-07-21T18:43:48.367 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:48 smithi193 ceph-mon[98204]: pgmap v64: 97 pgs: 97 active+clean; 32 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 440 KiB/s wr, 97 op/s 2023-07-21T18:43:48.385 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:48 smithi155 ceph-mon[97934]: pgmap v64: 97 pgs: 97 active+clean; 32 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 440 KiB/s wr, 97 op/s 2023-07-21T18:43:48.434 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:48 smithi039 ceph-mon[96489]: pgmap v64: 97 pgs: 97 active+clean; 32 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 440 KiB/s wr, 97 op/s 2023-07-21T18:43:49.262 INFO:tasks.workunit.client.0.smithi039.stdout: 1 43320 4.04 MB/sec warmup 62 sec latency 2422.017 ms 2023-07-21T18:43:50.262 INFO:tasks.workunit.client.0.smithi039.stdout: 1 45174 4.10 MB/sec warmup 63 sec latency 749.076 ms 2023-07-21T18:43:50.366 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:50 smithi193 ceph-mon[98204]: pgmap v65: 97 pgs: 97 active+clean; 32 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 417 KiB/s wr, 82 op/s 2023-07-21T18:43:50.385 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:50 smithi155 ceph-mon[97934]: pgmap v65: 97 pgs: 97 active+clean; 32 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 417 KiB/s wr, 82 op/s 2023-07-21T18:43:50.434 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:50 smithi039 ceph-mon[96489]: pgmap v65: 97 pgs: 97 active+clean; 32 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 417 KiB/s wr, 82 op/s 2023-07-21T18:43:51.262 INFO:tasks.workunit.client.0.smithi039.stdout: 1 46721 4.21 MB/sec warmup 64 sec latency 15.273 ms 2023-07-21T18:43:51.366 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:51 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:43:51.385 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:51 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:43:51.434 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:51 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:43:52.262 INFO:tasks.workunit.client.0.smithi039.stdout: 1 49046 4.33 MB/sec warmup 65 sec latency 12.539 ms 2023-07-21T18:43:52.366 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:52 smithi193 ceph-mon[98204]: pgmap v66: 97 pgs: 97 active+clean; 35 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 742 KiB/s wr, 140 op/s 2023-07-21T18:43:52.367 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:52 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:43:52.367 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:52 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:43:52.367 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:52 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:43:52.385 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:52 smithi155 ceph-mon[97934]: pgmap v66: 97 pgs: 97 active+clean; 35 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 742 KiB/s wr, 140 op/s 2023-07-21T18:43:52.385 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:52 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:43:52.386 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:52 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:43:52.386 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:52 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:43:52.434 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:52 smithi039 ceph-mon[96489]: pgmap v66: 97 pgs: 97 active+clean; 35 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 742 KiB/s wr, 140 op/s 2023-07-21T18:43:52.434 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:52 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:43:52.435 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:52 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:43:52.435 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:52 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:43:53.262 INFO:tasks.workunit.client.0.smithi039.stdout: 1 50562 4.38 MB/sec warmup 66 sec latency 11.362 ms 2023-07-21T18:43:54.262 INFO:tasks.workunit.client.0.smithi039.stdout: 1 53052 4.52 MB/sec warmup 67 sec latency 20.613 ms 2023-07-21T18:43:54.366 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:54 smithi193 ceph-mon[98204]: pgmap v67: 97 pgs: 97 active+clean; 35 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 597 KiB/s wr, 109 op/s 2023-07-21T18:43:54.385 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:54 smithi155 ceph-mon[97934]: pgmap v67: 97 pgs: 97 active+clean; 35 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 597 KiB/s wr, 109 op/s 2023-07-21T18:43:54.434 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:54 smithi039 ceph-mon[96489]: pgmap v67: 97 pgs: 97 active+clean; 35 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 597 KiB/s wr, 109 op/s 2023-07-21T18:43:55.262 INFO:tasks.workunit.client.0.smithi039.stdout: 1 54421 4.56 MB/sec warmup 68 sec latency 15.530 ms 2023-07-21T18:43:56.262 INFO:tasks.workunit.client.0.smithi039.stdout: 1 57016 4.73 MB/sec warmup 69 sec latency 10.304 ms 2023-07-21T18:43:56.366 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:56 smithi193 ceph-mon[98204]: pgmap v68: 97 pgs: 97 active+clean; 35 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 597 KiB/s wr, 109 op/s 2023-07-21T18:43:56.385 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:56 smithi155 ceph-mon[97934]: pgmap v68: 97 pgs: 97 active+clean; 35 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 597 KiB/s wr, 109 op/s 2023-07-21T18:43:56.434 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:56 smithi039 ceph-mon[96489]: pgmap v68: 97 pgs: 97 active+clean; 35 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 597 KiB/s wr, 109 op/s 2023-07-21T18:43:57.262 INFO:tasks.workunit.client.0.smithi039.stdout: 1 58420 4.74 MB/sec warmup 70 sec latency 34.043 ms 2023-07-21T18:43:58.262 INFO:tasks.workunit.client.0.smithi039.stdout: 1 60678 4.87 MB/sec warmup 71 sec latency 11.025 ms 2023-07-21T18:43:58.366 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:43:58 smithi193 ceph-mon[98204]: pgmap v69: 97 pgs: 97 active+clean; 40 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 985 KiB/s rd, 1.2 MiB/s wr, 222 op/s 2023-07-21T18:43:58.385 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:43:58 smithi155 ceph-mon[97934]: pgmap v69: 97 pgs: 97 active+clean; 40 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 985 KiB/s rd, 1.2 MiB/s wr, 222 op/s 2023-07-21T18:43:58.434 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:43:58 smithi039 ceph-mon[96489]: pgmap v69: 97 pgs: 97 active+clean; 40 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 985 KiB/s rd, 1.2 MiB/s wr, 222 op/s 2023-07-21T18:43:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.663+0000 7fd755f6a700 1 -- 172.21.15.39:0/643089651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd750104a30 msgr2=0x7fd750104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.663+0000 7fd755f6a700 1 --2- 172.21.15.39:0/643089651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd750104a30 0x7fd750104e00 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fd740009990 tx=0x7fd740009ca0 comp rx=0 tx=0).stop 2023-07-21T18:43:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.663+0000 7fd755f6a700 1 -- 172.21.15.39:0/643089651 shutdown_connections 2023-07-21T18:43:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.663+0000 7fd755f6a700 1 --2- 172.21.15.39:0/643089651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7500ff3e0 0x7fd75010b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.663+0000 7fd755f6a700 1 --2- 172.21.15.39:0/643089651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd750104a30 0x7fd750104e00 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.663+0000 7fd755f6a700 1 --2- 172.21.15.39:0/643089651 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd7500fea30 0x7fd7500feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.663+0000 7fd755f6a700 1 -- 172.21.15.39:0/643089651 >> 172.21.15.39:0/643089651 conn(0x7fd750074760 msgr2=0x7fd750074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.663+0000 7fd755f6a700 1 -- 172.21.15.39:0/643089651 shutdown_connections 2023-07-21T18:43:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.663+0000 7fd755f6a700 1 -- 172.21.15.39:0/643089651 wait complete. 2023-07-21T18:43:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.664+0000 7fd755f6a700 1 Processor -- start 2023-07-21T18:43:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.665+0000 7fd755f6a700 1 -- start start 2023-07-21T18:43:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.665+0000 7fd755f6a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7500fea30 0x7fd750196b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.665+0000 7fd755f6a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7500ff3e0 0x7fd750197080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.665+0000 7fd755f6a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd750104a30 0x7fd7501a19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.665+0000 7fd755f6a700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7501a1fe0 con 0x7fd7500ff3e0 2023-07-21T18:43:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.665+0000 7fd755f6a700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7501a2150 con 0x7fd7500fea30 2023-07-21T18:43:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.665+0000 7fd755f6a700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7501a2310 con 0x7fd750104a30 2023-07-21T18:43:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.665+0000 7fd74effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7500ff3e0 0x7fd750197080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.665+0000 7fd74f7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7500fea30 0x7fd750196b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.665+0000 7fd74f7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7500fea30 0x7fd750196b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:43560/0 (socket says 172.21.15.39:43560) 2023-07-21T18:43:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.665+0000 7fd74ffff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd750104a30 0x7fd7501a19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.665+0000 7fd74f7fe700 1 -- 172.21.15.39:0/2932758207 learned_addr learned my addr 172.21.15.39:0/2932758207 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.665+0000 7fd74effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7500ff3e0 0x7fd750197080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59198/0 (socket says 172.21.15.39:59198) 2023-07-21T18:43:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.666+0000 7fd74effd700 1 -- 172.21.15.39:0/2932758207 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd750104a30 msgr2=0x7fd7501a19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.666+0000 7fd74effd700 1 --2- 172.21.15.39:0/2932758207 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd750104a30 0x7fd7501a19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.666+0000 7fd74effd700 1 -- 172.21.15.39:0/2932758207 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7500fea30 msgr2=0x7fd750196b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.666+0000 7fd74effd700 1 --2- 172.21.15.39:0/2932758207 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7500fea30 0x7fd750196b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.666+0000 7fd74effd700 1 -- 172.21.15.39:0/2932758207 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7400095f0 con 0x7fd7500ff3e0 2023-07-21T18:43:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.666+0000 7fd74f7fe700 1 --2- 172.21.15.39:0/2932758207 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7500fea30 0x7fd750196b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:43:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.666+0000 7fd74effd700 1 --2- 172.21.15.39:0/2932758207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7500ff3e0 0x7fd750197080 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fd74400d950 tx=0x7fd74400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.666+0000 7fd74cff9700 1 -- 172.21.15.39:0/2932758207 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd74400f890 con 0x7fd7500ff3e0 2023-07-21T18:43:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.666+0000 7fd755f6a700 1 -- 172.21.15.39:0/2932758207 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7501a25f0 con 0x7fd7500ff3e0 2023-07-21T18:43:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.666+0000 7fd74cff9700 1 -- 172.21.15.39:0/2932758207 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd744010460 con 0x7fd7500ff3e0 2023-07-21T18:43:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.666+0000 7fd74cff9700 1 -- 172.21.15.39:0/2932758207 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd74400e610 con 0x7fd7500ff3e0 2023-07-21T18:43:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.666+0000 7fd755f6a700 1 -- 172.21.15.39:0/2932758207 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7501a2b50 con 0x7fd7500ff3e0 2023-07-21T18:43:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.668+0000 7fd74cff9700 1 -- 172.21.15.39:0/2932758207 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fd7440108d0 con 0x7fd7500ff3e0 2023-07-21T18:43:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.669+0000 7fd74cff9700 1 --2- 172.21.15.39:0/2932758207 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fd7380a12c0 0x7fd7380a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:58.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.669+0000 7fd74cff9700 1 -- 172.21.15.39:0/2932758207 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fd7440cc970 con 0x7fd7500ff3e0 2023-07-21T18:43:58.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.669+0000 7fd74f7fe700 1 --2- 172.21.15.39:0/2932758207 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fd7380a12c0 0x7fd7380a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:58.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.669+0000 7fd74f7fe700 1 --2- 172.21.15.39:0/2932758207 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fd7380a12c0 0x7fd7380a3770 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd74000b5c0 tx=0x7fd74000bb80 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:58.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.670+0000 7fd7620f0b80 1 Processor -- start 2023-07-21T18:43:58.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.670+0000 7fd7620f0b80 1 -- start start 2023-07-21T18:43:58.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.670+0000 7fd7620f0b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562876c216b0 0x562876dc9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:58.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.670+0000 7fd7620f0b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562876dc9b80 0x562876dcddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:58.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.670+0000 7fd7620f0b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562876dc9ff0 0x562876dce300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:58.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.670+0000 7fd7620f0b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x562876ccac50 con 0x562876dc9ff0 2023-07-21T18:43:58.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.670+0000 7fd7620f0b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562876d1d9e0 con 0x562876dc9b80 2023-07-21T18:43:58.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.670+0000 7fd7620f0b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x562876d16e70 con 0x562876c216b0 2023-07-21T18:43:58.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.670+0000 7fd74ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562876dc9ff0 0x562876dce300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:58.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.670+0000 7fd74f7fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562876c216b0 0x562876dc9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:58.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.670+0000 7fd74f7fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562876c216b0 0x562876dc9640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:49524/0 (socket says 172.21.15.39:49524) 2023-07-21T18:43:58.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.670+0000 7fd74ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562876dc9ff0 0x562876dce300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59208/0 (socket says 172.21.15.39:59208) 2023-07-21T18:43:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.671+0000 7fd74effd700 1 --2- 172.21.15.39:0/644979793 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562876dc9b80 0x562876dcddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.670+0000 7fd74f7fe700 1 -- 172.21.15.39:0/644979793 learned_addr learned my addr 172.21.15.39:0/644979793 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.671+0000 7fd74f7fe700 1 -- 172.21.15.39:0/644979793 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562876dc9b80 msgr2=0x562876dcddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.671+0000 7fd74f7fe700 1 --2- 172.21.15.39:0/644979793 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562876dc9b80 0x562876dcddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.671+0000 7fd74f7fe700 1 -- 172.21.15.39:0/644979793 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562876dc9ff0 msgr2=0x562876dce300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.671+0000 7fd74f7fe700 1 --2- 172.21.15.39:0/644979793 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562876dc9ff0 0x562876dce300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.671+0000 7fd74f7fe700 1 -- 172.21.15.39:0/644979793 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562876b44d80 con 0x562876c216b0 2023-07-21T18:43:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.671+0000 7fd74ffff700 1 --2- 172.21.15.39:0/644979793 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562876dc9ff0 0x562876dce300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:43:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.671+0000 7fd74f7fe700 1 --2- 172.21.15.39:0/644979793 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562876c216b0 0x562876dc9640 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd74000bc60 tx=0x7fd740020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73c2692c7205eb59 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.671+0000 7fd735ffb700 1 -- 172.21.15.39:0/644979793 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd74002a070 con 0x562876c216b0 2023-07-21T18:43:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.671+0000 7fd735ffb700 1 -- 172.21.15.39:0/644979793 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd74001e040 con 0x562876c216b0 2023-07-21T18:43:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.671+0000 7fd735ffb700 1 -- 172.21.15.39:0/644979793 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd740023d30 con 0x562876c216b0 2023-07-21T18:43:58.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.672+0000 7fd7620f0b80 1 -- 172.21.15.39:0/644979793 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562876c216b0 msgr2=0x562876dc9640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:58.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.672+0000 7fd7620f0b80 1 --2- 172.21.15.39:0/644979793 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562876c216b0 0x562876dc9640 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd74000bc60 tx=0x7fd740020000 comp rx=0 tx=0).stop 2023-07-21T18:43:58.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.672+0000 7fd7620f0b80 1 -- 172.21.15.39:0/644979793 shutdown_connections 2023-07-21T18:43:58.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.672+0000 7fd7620f0b80 1 --2- 172.21.15.39:0/644979793 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562876dc9ff0 0x562876dce300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:58.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.672+0000 7fd7620f0b80 1 --2- 172.21.15.39:0/644979793 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562876c216b0 0x562876dc9640 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:58.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.672+0000 7fd7620f0b80 1 --2- 172.21.15.39:0/644979793 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562876dc9b80 0x562876dcddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:58.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.672+0000 7fd7620f0b80 1 -- 172.21.15.39:0/644979793 >> 172.21.15.39:0/644979793 conn(0x562876dbabe0 msgr2=0x562876dbff10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:58.698 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.672+0000 7fd7620f0b80 1 -- 172.21.15.39:0/644979793 shutdown_connections 2023-07-21T18:43:58.698 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.672+0000 7fd7620f0b80 1 -- 172.21.15.39:0/644979793 wait complete. 2023-07-21T18:43:58.698 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.673+0000 7fd7620f0b80 1 -- start start 2023-07-21T18:43:58.698 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.673+0000 7fd7620f0b80 1 Processor -- start 2023-07-21T18:43:58.698 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.673+0000 7fd7620f0b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562876c216b0 0x562876dd8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:58.698 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.673+0000 7fd7620f0b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562876dc9b80 0x562876dd8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:58.698 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.673+0000 7fd7620f0b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562876dc9ff0 0x562876dd2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:58.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.673+0000 7fd7620f0b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x562876cb6a60 con 0x562876dc9ff0 2023-07-21T18:43:58.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.673+0000 7fd7620f0b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562876cb6760 con 0x562876c216b0 2023-07-21T18:43:58.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.673+0000 7fd7620f0b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x562876b9a8c0 con 0x562876dc9b80 2023-07-21T18:43:58.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.673+0000 7fd7620f0b80 10 client.0 _ll_register_callbacks cb 0x562876cb4700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:43:58.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.673+0000 7fd74ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562876dc9ff0 0x562876dd2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:58.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.673+0000 7fd74effd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562876dc9b80 0x562876dd8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:58.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.673+0000 7fd74ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562876dc9ff0 0x562876dd2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59214/0 (socket says 172.21.15.39:59214) 2023-07-21T18:43:58.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.673+0000 7fd74ffff700 1 -- 172.21.15.39:0/1643724584 learned_addr learned my addr 172.21.15.39:0/1643724584 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.673+0000 7fd74f7fe700 1 --2- 172.21.15.39:0/1643724584 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562876c216b0 0x562876dd8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.674+0000 7fd74ffff700 1 -- 172.21.15.39:0/1643724584 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562876dc9b80 msgr2=0x562876dd8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.674+0000 7fd74ffff700 1 --2- 172.21.15.39:0/1643724584 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562876dc9b80 0x562876dd8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.674+0000 7fd74ffff700 1 -- 172.21.15.39:0/1643724584 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562876c216b0 msgr2=0x562876dd8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.674+0000 7fd74ffff700 1 --2- 172.21.15.39:0/1643724584 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562876c216b0 0x562876dd8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.674+0000 7fd74ffff700 1 -- 172.21.15.39:0/1643724584 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd748003ac0 con 0x562876dc9ff0 2023-07-21T18:43:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.674+0000 7fd74ffff700 1 --2- 172.21.15.39:0/1643724584 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562876dc9ff0 0x562876dd2cb0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fd748004660 tx=0x7fd748004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0a676bcdee41e16 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.674+0000 7fd72effd700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:43:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.674+0000 7fd72effd700 1 -- 172.21.15.39:0/1643724584 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd748007910 con 0x562876dc9ff0 2023-07-21T18:43:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.674+0000 7fd72effd700 1 -- 172.21.15.39:0/1643724584 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fd748007a70 con 0x562876dc9ff0 2023-07-21T18:43:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.674+0000 7fd7620f0b80 1 -- 172.21.15.39:0/1643724584 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562876c70490 con 0x562876dc9ff0 2023-07-21T18:43:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.674+0000 7fd72effd700 1 -- 172.21.15.39:0/1643724584 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd748007c30 con 0x562876dc9ff0 2023-07-21T18:43:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.674+0000 7fd7620f0b80 10 client.16275 fetch_fsmap learned FSMap version 33 2023-07-21T18:43:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.674+0000 7fd7620f0b80 1 -- 172.21.15.39:0/1643724584 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7fd75010c160 con 0x562876dc9ff0 2023-07-21T18:43:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.674+0000 7fd72effd700 1 -- 172.21.15.39:0/1643724584 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562876c70490 con 0x562876dc9ff0 2023-07-21T18:43:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.675+0000 7fd72effd700 1 -- 172.21.15.39:0/1643724584 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fd74801e3b0 con 0x562876dc9ff0 2023-07-21T18:43:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.675+0000 7fd7620f0b80 10 client.16275 fetch_fsmap finished waiting for FSMap version 33 2023-07-21T18:43:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.675+0000 7fd7620f0b80 10 client.16275 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:43:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.675+0000 7fd7620f0b80 20 client.16275 populate_metadata read hostname 'smithi039' 2023-07-21T18:43:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.675+0000 7fd7620f0b80 1 --2- 172.21.15.39:0/1643724584 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x562876de2fe0 0x562876de53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.675+0000 7fd7620f0b80 4 client.16275 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:43:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.675+0000 7fd7620f0b80 1 -- 172.21.15.39:0/1643724584 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562876d0fe50 con 0x562876de2fe0 2023-07-21T18:43:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.675+0000 7fd74f7fe700 1 --2- 172.21.15.39:0/1643724584 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x562876de2fe0 0x562876de53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.675+0000 7fd72effd700 10 client.16275 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:43:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.675+0000 7fd74f7fe700 1 --2- 172.21.15.39:0/1643724584 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x562876de2fe0 0x562876de53f0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1e6ecae17e037f8 server_cookie=9359bb9b82f33023 in_seq=0 out_seq=0 2023-07-21T18:43:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.676+0000 7fd72effd700 1 -- 172.21.15.39:0/1643724584 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562876d0fe50 con 0x562876de2fe0 2023-07-21T18:43:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.676+0000 7fd72effd700 10 client.16275 handle_command_reply: tid=0 2023-07-21T18:43:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.676+0000 7fd7620f0b80 1 client.16275 shutdown 2023-07-21T18:43:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.676+0000 7fd7620f0b80 1 -- 172.21.15.39:0/1643724584 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562876dc9ff0 msgr2=0x562876dd2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.676+0000 7fd7620f0b80 1 --2- 172.21.15.39:0/1643724584 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562876dc9ff0 0x562876dd2cb0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fd748004660 tx=0x7fd748004690 comp rx=0 tx=0).stop 2023-07-21T18:43:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.676+0000 7fd7620f0b80 1 -- 172.21.15.39:0/1643724584 shutdown_connections 2023-07-21T18:43:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.676+0000 7fd7620f0b80 1 --2- 172.21.15.39:0/1643724584 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x562876de2fe0 0x562876de53f0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.676+0000 7fd7620f0b80 1 --2- 172.21.15.39:0/1643724584 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562876c216b0 0x562876dd8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.676+0000 7fd72effd700 0 client.16275 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:43:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.676+0000 7fd7620f0b80 1 --2- 172.21.15.39:0/1643724584 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562876dc9ff0 0x562876dd2cb0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.676+0000 7fd7620f0b80 1 --2- 172.21.15.39:0/1643724584 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562876dc9b80 0x562876dd8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.676+0000 7fd7620f0b80 1 -- 172.21.15.39:0/1643724584 >> 172.21.15.39:0/1643724584 conn(0x562876dbabe0 msgr2=0x562876dbf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.676+0000 7fd7620f0b80 1 -- 172.21.15.39:0/1643724584 shutdown_connections 2023-07-21T18:43:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.676+0000 7fd7620f0b80 1 -- 172.21.15.39:0/1643724584 wait complete. 2023-07-21T18:43:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.676+0000 7fd7620f0b80 20 client.16275 trim_cache size 0 max 16384 2023-07-21T18:43:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.692+0000 7fd7620f0b80 1 Processor -- start 2023-07-21T18:43:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.692+0000 7fd7620f0b80 1 -- start start 2023-07-21T18:43:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.692+0000 7fd7620f0b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562876c216b0 0x562876dd07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.692+0000 7fd7620f0b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562876dc9b80 0x562876dd2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.692+0000 7fd7620f0b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562876dc9ff0 0x562876dcf380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.692+0000 7fd7620f0b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x562876dc1b50 con 0x562876dc9ff0 2023-07-21T18:43:58.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.692+0000 7fd7620f0b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562876dc0fb0 con 0x562876c216b0 2023-07-21T18:43:58.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.692+0000 7fd7620f0b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x562876dca3c0 con 0x562876dc9b80 2023-07-21T18:43:58.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.692+0000 7fd74ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562876dc9ff0 0x562876dcf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:58.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.692+0000 7fd74f7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562876c216b0 0x562876dd07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:58.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.692+0000 7fd74ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562876dc9ff0 0x562876dcf380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59222/0 (socket says 172.21.15.39:59222) 2023-07-21T18:43:58.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.692+0000 7fd74ffff700 1 -- 172.21.15.39:0/2919686624 learned_addr learned my addr 172.21.15.39:0/2919686624 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:58.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.692+0000 7fd74effd700 1 --2- 172.21.15.39:0/2919686624 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562876dc9b80 0x562876dd2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:58.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.692+0000 7fd74ffff700 1 -- 172.21.15.39:0/2919686624 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562876dc9b80 msgr2=0x562876dd2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:58.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.692+0000 7fd74ffff700 1 --2- 172.21.15.39:0/2919686624 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562876dc9b80 0x562876dd2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:58.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.692+0000 7fd74ffff700 1 -- 172.21.15.39:0/2919686624 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562876c216b0 msgr2=0x562876dd07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:58.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.692+0000 7fd74ffff700 1 --2- 172.21.15.39:0/2919686624 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562876c216b0 0x562876dd07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.692+0000 7fd74ffff700 1 -- 172.21.15.39:0/2919686624 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562876b2dc00 con 0x562876dc9ff0 2023-07-21T18:43:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.692+0000 7fd74f7fe700 1 --2- 172.21.15.39:0/2919686624 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562876c216b0 0x562876dd07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:43:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.692+0000 7fd74ffff700 1 --2- 172.21.15.39:0/2919686624 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562876dc9ff0 0x562876dcf380 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fd74801e740 tx=0x7fd748003ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fe549378cc141bf server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.693+0000 7fd7367fc700 1 -- 172.21.15.39:0/2919686624 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd74801eab0 con 0x562876dc9ff0 2023-07-21T18:43:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.693+0000 7fd7367fc700 1 -- 172.21.15.39:0/2919686624 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd74800f040 con 0x562876dc9ff0 2023-07-21T18:43:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.693+0000 7fd7367fc700 1 -- 172.21.15.39:0/2919686624 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd748007970 con 0x562876dc9ff0 2023-07-21T18:43:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.693+0000 7fd7620f0b80 1 -- 172.21.15.39:0/2919686624 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562876dc9ff0 msgr2=0x562876dcf380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.693+0000 7fd7620f0b80 1 --2- 172.21.15.39:0/2919686624 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562876dc9ff0 0x562876dcf380 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fd74801e740 tx=0x7fd748003ec0 comp rx=0 tx=0).stop 2023-07-21T18:43:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.693+0000 7fd7620f0b80 1 -- 172.21.15.39:0/2919686624 shutdown_connections 2023-07-21T18:43:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.693+0000 7fd7620f0b80 1 --2- 172.21.15.39:0/2919686624 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562876c216b0 0x562876dd07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.693+0000 7fd7620f0b80 1 --2- 172.21.15.39:0/2919686624 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562876dc9ff0 0x562876dcf380 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.693+0000 7fd7620f0b80 1 --2- 172.21.15.39:0/2919686624 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562876dc9b80 0x562876dd2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.693+0000 7fd7620f0b80 1 -- 172.21.15.39:0/2919686624 >> 172.21.15.39:0/2919686624 conn(0x562876dbabe0 msgr2=0x562876dcec70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.693+0000 7fd7620f0b80 1 -- 172.21.15.39:0/2919686624 shutdown_connections 2023-07-21T18:43:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.693+0000 7fd7620f0b80 1 -- 172.21.15.39:0/2919686624 wait complete. 2023-07-21T18:43:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.694+0000 7fd7620f0b80 1 -- start start 2023-07-21T18:43:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.694+0000 7fd7620f0b80 1 Processor -- start 2023-07-21T18:43:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.694+0000 7fd7620f0b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562876c216b0 0x562876dd6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.694+0000 7fd7620f0b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562876dc9b80 0x562876dd6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.694+0000 7fd7620f0b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562876dc9ff0 0x562876dd70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.694+0000 7fd7620f0b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x562876e3a780 con 0x562876dc9ff0 2023-07-21T18:43:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.694+0000 7fd7620f0b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562876e3a8f0 con 0x562876dc9b80 2023-07-21T18:43:58.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.694+0000 7fd7620f0b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x562876e3aab0 con 0x562876c216b0 2023-07-21T18:43:58.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.694+0000 7fd7620f0b80 10 client.0 _ll_register_callbacks cb 0x7fd748004270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:43:58.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.694+0000 7fd74f7fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562876c216b0 0x562876dd6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:58.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.694+0000 7fd74effd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562876dc9b80 0x562876dd6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:58.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.694+0000 7fd74effd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562876dc9b80 0x562876dd6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:43590/0 (socket says 172.21.15.39:43590) 2023-07-21T18:43:58.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.694+0000 7fd74f7fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562876c216b0 0x562876dd6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:49544/0 (socket says 172.21.15.39:49544) 2023-07-21T18:43:58.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.694+0000 7fd74effd700 1 -- 172.21.15.39:0/2691800003 learned_addr learned my addr 172.21.15.39:0/2691800003 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:43:58.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.695+0000 7fd74ffff700 1 --2- 172.21.15.39:0/2691800003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562876dc9ff0 0x562876dd70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:58.720 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.695+0000 7fd74effd700 1 -- 172.21.15.39:0/2691800003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562876c216b0 msgr2=0x562876dd6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:58.720 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.695+0000 7fd74effd700 1 --2- 172.21.15.39:0/2691800003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562876c216b0 0x562876dd6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:58.720 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.695+0000 7fd74effd700 1 -- 172.21.15.39:0/2691800003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562876dc9ff0 msgr2=0x562876dd70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:43:58.720 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.695+0000 7fd74effd700 1 --2- 172.21.15.39:0/2691800003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562876dc9ff0 0x562876dd70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:58.720 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.695+0000 7fd74effd700 1 -- 172.21.15.39:0/2691800003 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562876b44d80 con 0x562876dc9b80 2023-07-21T18:43:58.720 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.695+0000 7fd74f7fe700 1 --2- 172.21.15.39:0/2691800003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562876c216b0 0x562876dd6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:43:58.720 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.695+0000 7fd74effd700 1 --2- 172.21.15.39:0/2691800003 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562876dc9b80 0x562876dd6b70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd7440964d0 tx=0x7fd744099be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd7814c177587818 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:43:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.695+0000 7fd72effd700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-21T18:43:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.695+0000 7fd72effd700 1 -- 172.21.15.39:0/2691800003 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd744099ec0 con 0x562876dc9b80 2023-07-21T18:43:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.695+0000 7fd72effd700 1 -- 172.21.15.39:0/2691800003 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fd744098f50 con 0x562876dc9b80 2023-07-21T18:43:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.695+0000 7fd7620f0b80 1 -- 172.21.15.39:0/2691800003 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd74001e040 con 0x562876dc9b80 2023-07-21T18:43:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.695+0000 7fd72effd700 1 -- 172.21.15.39:0/2691800003 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7440b2260 con 0x562876dc9b80 2023-07-21T18:43:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.695+0000 7fd7620f0b80 10 client.25781 fetch_fsmap learned FSMap version 33 2023-07-21T18:43:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.695+0000 7fd7620f0b80 1 -- 172.21.15.39:0/2691800003 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7fd75010c160 con 0x562876dc9b80 2023-07-21T18:43:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.696+0000 7fd72effd700 1 -- 172.21.15.39:0/2691800003 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd74001e040 con 0x562876dc9b80 2023-07-21T18:43:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.696+0000 7fd72effd700 1 -- 172.21.15.39:0/2691800003 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fd7440989c0 con 0x562876dc9b80 2023-07-21T18:43:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.696+0000 7fd7620f0b80 10 client.25781 fetch_fsmap finished waiting for FSMap version 33 2023-07-21T18:43:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.696+0000 7fd7620f0b80 10 client.25781 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:43:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.696+0000 7fd7620f0b80 20 client.25781 populate_metadata read hostname 'smithi039' 2023-07-21T18:43:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.696+0000 7fd7620f0b80 1 --2- 172.21.15.39:0/2691800003 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x562876e3c1b0 0x562876e3e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:43:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.696+0000 7fd7620f0b80 4 client.25781 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:43:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.696+0000 7fd7620f0b80 1 -- 172.21.15.39:0/2691800003 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562876cd1ff0 con 0x562876e3c1b0 2023-07-21T18:43:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.696+0000 7fd74f7fe700 1 --2- 172.21.15.39:0/2691800003 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x562876e3c1b0 0x562876e3e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:43:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.696+0000 7fd72effd700 10 client.25781 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:43:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.696+0000 7fd74f7fe700 1 --2- 172.21.15.39:0/2691800003 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x562876e3c1b0 0x562876e3e5c0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6b3c5aeae222820 server_cookie=9f1e9bbea88e30f9 in_seq=0 out_seq=0 2023-07-21T18:43:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.733+0000 7fd72effd700 1 -- 172.21.15.39:0/2691800003 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562876cd1ff0 con 0x562876e3c1b0 2023-07-21T18:43:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.733+0000 7fd72effd700 10 client.25781 handle_command_reply: tid=0 2023-07-21T18:43:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.733+0000 7fd7620f0b80 1 client.25781 shutdown 2023-07-21T18:43:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.733+0000 7fd7620f0b80 1 -- 172.21.15.39:0/2691800003 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562876dc9b80 msgr2=0x562876dd6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:58.753 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.733+0000 7fd7620f0b80 1 --2- 172.21.15.39:0/2691800003 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562876dc9b80 0x562876dd6b70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd7440964d0 tx=0x7fd744099be0 comp rx=0 tx=0).stop 2023-07-21T18:43:58.753 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.733+0000 7fd7620f0b80 1 -- 172.21.15.39:0/2691800003 shutdown_connections 2023-07-21T18:43:58.753 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.733+0000 7fd7620f0b80 1 --2- 172.21.15.39:0/2691800003 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x562876e3c1b0 0x562876e3e5c0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:58.753 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.733+0000 7fd7620f0b80 1 --2- 172.21.15.39:0/2691800003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562876dc9ff0 0x562876dd70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:58.753 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.733+0000 7fd72effd700 0 client.25781 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:43:58.753 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.733+0000 7fd7620f0b80 1 --2- 172.21.15.39:0/2691800003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562876c216b0 0x562876dd6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:58.753 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.733+0000 7fd7620f0b80 1 --2- 172.21.15.39:0/2691800003 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562876dc9b80 0x562876dd6b70 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:58.753 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.733+0000 7fd7620f0b80 1 -- 172.21.15.39:0/2691800003 >> 172.21.15.39:0/2691800003 conn(0x562876dbabe0 msgr2=0x562876dcf500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:58.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.733+0000 7fd7620f0b80 1 -- 172.21.15.39:0/2691800003 shutdown_connections 2023-07-21T18:43:58.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.733+0000 7fd7620f0b80 1 -- 172.21.15.39:0/2691800003 wait complete. 2023-07-21T18:43:58.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.733+0000 7fd7620f0b80 20 client.25781 trim_cache size 0 max 16384 2023-07-21T18:43:58.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:43:58.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:43:58.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:43:58.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:43:58.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.734+0000 7fd755f6a700 1 -- 172.21.15.39:0/2932758207 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fd7380a12c0 msgr2=0x7fd7380a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:58.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.734+0000 7fd755f6a700 1 --2- 172.21.15.39:0/2932758207 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fd7380a12c0 0x7fd7380a3770 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd74000b5c0 tx=0x7fd74000bb80 comp rx=0 tx=0).stop 2023-07-21T18:43:58.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.734+0000 7fd755f6a700 1 -- 172.21.15.39:0/2932758207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7500ff3e0 msgr2=0x7fd750197080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:43:58.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.734+0000 7fd755f6a700 1 --2- 172.21.15.39:0/2932758207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7500ff3e0 0x7fd750197080 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fd74400d950 tx=0x7fd74400dc60 comp rx=0 tx=0).stop 2023-07-21T18:43:58.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.734+0000 7fd755f6a700 1 -- 172.21.15.39:0/2932758207 shutdown_connections 2023-07-21T18:43:58.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.734+0000 7fd755f6a700 1 --2- 172.21.15.39:0/2932758207 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fd7380a12c0 0x7fd7380a3770 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:58.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.734+0000 7fd755f6a700 1 --2- 172.21.15.39:0/2932758207 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7500fea30 0x7fd750196b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:58.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.734+0000 7fd755f6a700 1 --2- 172.21.15.39:0/2932758207 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd750104a30 0x7fd7501a19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:58.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.734+0000 7fd755f6a700 1 --2- 172.21.15.39:0/2932758207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7500ff3e0 0x7fd750197080 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:43:58.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.734+0000 7fd755f6a700 1 -- 172.21.15.39:0/2932758207 >> 172.21.15.39:0/2932758207 conn(0x7fd750074760 msgr2=0x7fd7500fd000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:43:58.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.734+0000 7fd755f6a700 1 -- 172.21.15.39:0/2932758207 shutdown_connections 2023-07-21T18:43:58.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:43:58.734+0000 7fd755f6a700 1 -- 172.21.15.39:0/2932758207 wait complete. 2023-07-21T18:43:59.262 INFO:tasks.workunit.client.0.smithi039.stdout: 1 62676 4.90 MB/sec warmup 72 sec latency 27.337 ms 2023-07-21T18:44:00.262 INFO:tasks.workunit.client.0.smithi039.stdout: 1 64308 5.00 MB/sec warmup 73 sec latency 9.882 ms 2023-07-21T18:44:00.366 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:44:00 smithi193 ceph-mon[98204]: pgmap v70: 97 pgs: 97 active+clean; 40 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 924 KiB/s wr, 170 op/s 2023-07-21T18:44:00.385 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:44:00 smithi155 ceph-mon[97934]: pgmap v70: 97 pgs: 97 active+clean; 40 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 924 KiB/s wr, 170 op/s 2023-07-21T18:44:00.434 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:44:00 smithi039 ceph-mon[96489]: pgmap v70: 97 pgs: 97 active+clean; 40 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 924 KiB/s wr, 170 op/s 2023-07-21T18:44:01.262 INFO:tasks.workunit.client.0.smithi039.stdout: 1 66157 5.02 MB/sec warmup 74 sec latency 22.143 ms 2023-07-21T18:44:02.262 INFO:tasks.workunit.client.0.smithi039.stdout: 1 67746 5.12 MB/sec warmup 75 sec latency 22.412 ms 2023-07-21T18:44:02.366 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:44:02 smithi193 ceph-mon[98204]: pgmap v71: 97 pgs: 97 active+clean; 41 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 297 op/s 2023-07-21T18:44:02.385 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:44:02 smithi155 ceph-mon[97934]: pgmap v71: 97 pgs: 97 active+clean; 41 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 297 op/s 2023-07-21T18:44:02.434 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:44:02 smithi039 ceph-mon[96489]: pgmap v71: 97 pgs: 97 active+clean; 41 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 297 op/s 2023-07-21T18:44:03.262 INFO:tasks.workunit.client.0.smithi039.stdout: 1 69821 5.15 MB/sec warmup 76 sec latency 13.811 ms 2023-07-21T18:44:04.262 INFO:tasks.workunit.client.0.smithi039.stdout: 1 71428 5.24 MB/sec warmup 77 sec latency 33.742 ms 2023-07-21T18:44:04.366 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:44:04 smithi193 ceph-mon[98204]: pgmap v72: 97 pgs: 97 active+clean; 41 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 239 op/s 2023-07-21T18:44:04.407 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:44:04 smithi155 ceph-mon[97934]: pgmap v72: 97 pgs: 97 active+clean; 41 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 239 op/s 2023-07-21T18:44:04.434 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:44:04 smithi039 ceph-mon[96489]: pgmap v72: 97 pgs: 97 active+clean; 41 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 239 op/s 2023-07-21T18:44:05.262 INFO:tasks.workunit.client.0.smithi039.stdout: 1 73777 5.31 MB/sec warmup 78 sec latency 12.642 ms 2023-07-21T18:44:06.262 INFO:tasks.workunit.client.0.smithi039.stdout: 1 75333 5.35 MB/sec warmup 79 sec latency 9.496 ms 2023-07-21T18:44:06.366 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:44:06 smithi193 ceph-mon[98204]: pgmap v73: 97 pgs: 97 active+clean; 41 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 239 op/s 2023-07-21T18:44:06.385 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:44:06 smithi155 ceph-mon[97934]: pgmap v73: 97 pgs: 97 active+clean; 41 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 239 op/s 2023-07-21T18:44:06.434 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:44:06 smithi039 ceph-mon[96489]: pgmap v73: 97 pgs: 97 active+clean; 41 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 239 op/s 2023-07-21T18:44:07.262 INFO:tasks.workunit.client.0.smithi039.stdout: 1 77751 5.45 MB/sec warmup 80 sec latency 34.846 ms 2023-07-21T18:44:08.262 INFO:tasks.workunit.client.0.smithi039.stdout: 1 79065 5.46 MB/sec warmup 81 sec latency 13.319 ms 2023-07-21T18:44:08.366 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:44:08 smithi193 ceph-mon[98204]: pgmap v74: 97 pgs: 97 active+clean; 47 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 375 op/s 2023-07-21T18:44:08.385 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:44:08 smithi155 ceph-mon[97934]: pgmap v74: 97 pgs: 97 active+clean; 47 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 375 op/s 2023-07-21T18:44:08.434 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:44:08 smithi039 ceph-mon[96489]: pgmap v74: 97 pgs: 97 active+clean; 47 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 375 op/s 2023-07-21T18:44:09.262 INFO:tasks.workunit.client.0.smithi039.stdout: 1 81295 5.55 MB/sec warmup 82 sec latency 75.847 ms 2023-07-21T18:44:10.262 INFO:tasks.workunit.client.0.smithi039.stdout: 1 82708 5.58 MB/sec warmup 83 sec latency 14.281 ms 2023-07-21T18:44:10.366 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:44:10 smithi193 ceph-mon[98204]: pgmap v75: 97 pgs: 97 active+clean; 47 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 262 op/s 2023-07-21T18:44:10.385 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:44:10 smithi155 ceph-mon[97934]: pgmap v75: 97 pgs: 97 active+clean; 47 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 262 op/s 2023-07-21T18:44:10.434 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:44:10 smithi039 ceph-mon[96489]: pgmap v75: 97 pgs: 97 active+clean; 47 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 262 op/s 2023-07-21T18:44:11.262 INFO:tasks.workunit.client.0.smithi039.stdout: 1 85329 5.70 MB/sec warmup 84 sec latency 10.243 ms 2023-07-21T18:44:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.849+0000 7f3c7f439700 1 -- 172.21.15.39:0/3040381699 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c780ffb30 msgr2=0x7f3c780fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:44:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.849+0000 7f3c7f439700 1 --2- 172.21.15.39:0/3040381699 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c780ffb30 0x7f3c780fffa0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f3c68009990 tx=0x7f3c68009ca0 comp rx=0 tx=0).stop 2023-07-21T18:44:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.849+0000 7f3c7f439700 1 -- 172.21.15.39:0/3040381699 shutdown_connections 2023-07-21T18:44:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.849+0000 7f3c7f439700 1 --2- 172.21.15.39:0/3040381699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c781004e0 0x7f3c7810c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.849+0000 7f3c7f439700 1 --2- 172.21.15.39:0/3040381699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c78105b60 0x7f3c78105f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.849+0000 7f3c7f439700 1 --2- 172.21.15.39:0/3040381699 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c780ffb30 0x7f3c780fffa0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.849+0000 7f3c7f439700 1 -- 172.21.15.39:0/3040381699 >> 172.21.15.39:0/3040381699 conn(0x7f3c780fb5f0 msgr2=0x7f3c780fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:44:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.849+0000 7f3c7f439700 1 -- 172.21.15.39:0/3040381699 shutdown_connections 2023-07-21T18:44:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.849+0000 7f3c7f439700 1 -- 172.21.15.39:0/3040381699 wait complete. 2023-07-21T18:44:11.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.850+0000 7f3c7f439700 1 Processor -- start 2023-07-21T18:44:11.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.850+0000 7f3c7f439700 1 -- start start 2023-07-21T18:44:11.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.850+0000 7f3c7f439700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c780ffb30 0x7f3c7819bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:11.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.850+0000 7f3c7f439700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c781004e0 0x7f3c7819c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:11.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.850+0000 7f3c7f439700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c78105b60 0x7f3c78195a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:11.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.850+0000 7f3c7f439700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c781960a0 con 0x7f3c780ffb30 2023-07-21T18:44:11.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.850+0000 7f3c7f439700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c78196210 con 0x7f3c78105b60 2023-07-21T18:44:11.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.850+0000 7f3c7f439700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c781963d0 con 0x7f3c781004e0 2023-07-21T18:44:11.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.850+0000 7f3c7d1d5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c780ffb30 0x7f3c7819bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:11.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.851+0000 7f3c7d1d5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c780ffb30 0x7f3c7819bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59238/0 (socket says 172.21.15.39:59238) 2023-07-21T18:44:11.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.851+0000 7f3c7d1d5700 1 -- 172.21.15.39:0/384597953 learned_addr learned my addr 172.21.15.39:0/384597953 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:44:11.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.851+0000 7f3c7d9d6700 1 --2- 172.21.15.39:0/384597953 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c78105b60 0x7f3c78195a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.851+0000 7f3c7c9d4700 1 --2- 172.21.15.39:0/384597953 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c781004e0 0x7f3c7819c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.851+0000 7f3c7d1d5700 1 -- 172.21.15.39:0/384597953 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c781004e0 msgr2=0x7f3c7819c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:44:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.851+0000 7f3c7d1d5700 1 --2- 172.21.15.39:0/384597953 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c781004e0 0x7f3c7819c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.851+0000 7f3c7d1d5700 1 -- 172.21.15.39:0/384597953 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c78105b60 msgr2=0x7f3c78195a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:44:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.851+0000 7f3c7d1d5700 1 --2- 172.21.15.39:0/384597953 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c78105b60 0x7f3c78195a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.851+0000 7f3c7d1d5700 1 -- 172.21.15.39:0/384597953 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c680095f0 con 0x7f3c780ffb30 2023-07-21T18:44:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.851+0000 7f3c7d1d5700 1 --2- 172.21.15.39:0/384597953 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c780ffb30 0x7f3c7819bbd0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f3c7400e9c0 tx=0x7f3c7400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:44:11.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.851+0000 7f3c6e7fc700 1 -- 172.21.15.39:0/384597953 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c740097b0 con 0x7f3c780ffb30 2023-07-21T18:44:11.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.851+0000 7f3c6e7fc700 1 -- 172.21.15.39:0/384597953 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c74004cf0 con 0x7f3c780ffb30 2023-07-21T18:44:11.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.851+0000 7f3c7f439700 1 -- 172.21.15.39:0/384597953 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c781966b0 con 0x7f3c780ffb30 2023-07-21T18:44:11.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.851+0000 7f3c6e7fc700 1 -- 172.21.15.39:0/384597953 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c74005600 con 0x7f3c780ffb30 2023-07-21T18:44:11.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.851+0000 7f3c7f439700 1 -- 172.21.15.39:0/384597953 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c781a2bc0 con 0x7f3c780ffb30 2023-07-21T18:44:11.877 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.854+0000 7f3c6e7fc700 1 -- 172.21.15.39:0/384597953 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f3c7400cc60 con 0x7f3c780ffb30 2023-07-21T18:44:11.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.854+0000 7f3c6e7fc700 1 --2- 172.21.15.39:0/384597953 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f3c640a1340 0x7f3c640a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:11.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.855+0000 7f3c6e7fc700 1 -- 172.21.15.39:0/384597953 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f3c74014070 con 0x7f3c780ffb30 2023-07-21T18:44:11.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.855+0000 7f3c7c9d4700 1 --2- 172.21.15.39:0/384597953 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f3c640a1340 0x7f3c640a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:11.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.855+0000 7f3c7c9d4700 1 --2- 172.21.15.39:0/384597953 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f3c640a1340 0x7f3c640a37f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3c780fef80 tx=0x7f3c6800b260 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:44:11.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.855+0000 7f3c8b5bfb80 1 Processor -- start 2023-07-21T18:44:11.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.855+0000 7f3c8b5bfb80 1 -- start start 2023-07-21T18:44:11.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.856+0000 7f3c8b5bfb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55edbf6656b0 0x55edbf813160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:11.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.856+0000 7f3c8b5bfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edbf8136a0 0x55edbf80d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:11.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.856+0000 7f3c8b5bfb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55edbf813b10 0x55edbf80d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:11.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.856+0000 7f3c8b5bfb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edbf70ec50 con 0x55edbf6656b0 2023-07-21T18:44:11.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.856+0000 7f3c8b5bfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edbf7619e0 con 0x55edbf8136a0 2023-07-21T18:44:11.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.856+0000 7f3c8b5bfb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edbf75ae70 con 0x55edbf813b10 2023-07-21T18:44:11.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.856+0000 7f3c7d1d5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55edbf6656b0 0x55edbf813160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:11.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.856+0000 7f3c7d1d5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55edbf6656b0 0x55edbf813160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59246/0 (socket says 172.21.15.39:59246) 2023-07-21T18:44:11.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.856+0000 7f3c7d1d5700 1 -- 172.21.15.39:0/716568088 learned_addr learned my addr 172.21.15.39:0/716568088 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:44:11.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.856+0000 7f3c7c9d4700 1 --2- 172.21.15.39:0/716568088 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edbf8136a0 0x55edbf80d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:11.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.856+0000 7f3c7d9d6700 1 --2- 172.21.15.39:0/716568088 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55edbf813b10 0x55edbf80d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:11.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.856+0000 7f3c7d1d5700 1 -- 172.21.15.39:0/716568088 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55edbf813b10 msgr2=0x55edbf80d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:44:11.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.856+0000 7f3c7d1d5700 1 --2- 172.21.15.39:0/716568088 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55edbf813b10 0x55edbf80d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:11.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.856+0000 7f3c7d1d5700 1 -- 172.21.15.39:0/716568088 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edbf8136a0 msgr2=0x55edbf80d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:44:11.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.856+0000 7f3c7d1d5700 1 --2- 172.21.15.39:0/716568088 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edbf8136a0 0x55edbf80d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:11.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.856+0000 7f3c7d1d5700 1 -- 172.21.15.39:0/716568088 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55edbf588d80 con 0x55edbf6656b0 2023-07-21T18:44:11.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.856+0000 7f3c7d1d5700 1 --2- 172.21.15.39:0/716568088 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55edbf6656b0 0x55edbf813160 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f3c74096540 tx=0x7f3c74099a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b8b5da88738132d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:44:11.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.856+0000 7f3c637fe700 1 -- 172.21.15.39:0/716568088 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c7409a040 con 0x55edbf6656b0 2023-07-21T18:44:11.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.856+0000 7f3c637fe700 1 -- 172.21.15.39:0/716568088 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c7409e070 con 0x55edbf6656b0 2023-07-21T18:44:11.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.856+0000 7f3c637fe700 1 -- 172.21.15.39:0/716568088 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c74099530 con 0x55edbf6656b0 2023-07-21T18:44:11.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.857+0000 7f3c8b5bfb80 1 -- 172.21.15.39:0/716568088 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55edbf6656b0 msgr2=0x55edbf813160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:44:11.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.857+0000 7f3c8b5bfb80 1 --2- 172.21.15.39:0/716568088 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55edbf6656b0 0x55edbf813160 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f3c74096540 tx=0x7f3c74099a20 comp rx=0 tx=0).stop 2023-07-21T18:44:11.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.857+0000 7f3c8b5bfb80 1 -- 172.21.15.39:0/716568088 shutdown_connections 2023-07-21T18:44:11.882 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.857+0000 7f3c8b5bfb80 1 --2- 172.21.15.39:0/716568088 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55edbf813b10 0x55edbf80d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:11.882 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.857+0000 7f3c8b5bfb80 1 --2- 172.21.15.39:0/716568088 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55edbf6656b0 0x55edbf813160 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:11.882 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.857+0000 7f3c8b5bfb80 1 --2- 172.21.15.39:0/716568088 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edbf8136a0 0x55edbf80d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:11.882 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.857+0000 7f3c8b5bfb80 1 -- 172.21.15.39:0/716568088 >> 172.21.15.39:0/716568088 conn(0x55edbf7febe0 msgr2=0x55edbf803f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:44:11.882 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.857+0000 7f3c8b5bfb80 1 -- 172.21.15.39:0/716568088 shutdown_connections 2023-07-21T18:44:11.882 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.857+0000 7f3c8b5bfb80 1 -- 172.21.15.39:0/716568088 wait complete. 2023-07-21T18:44:11.882 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.858+0000 7f3c8b5bfb80 1 -- start start 2023-07-21T18:44:11.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.858+0000 7f3c8b5bfb80 1 Processor -- start 2023-07-21T18:44:11.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.858+0000 7f3c8b5bfb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55edbf6656b0 0x55edbf819a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:11.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.858+0000 7f3c8b5bfb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55edbf8136a0 0x55edbf816d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:11.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.858+0000 7f3c8b5bfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edbf813b10 0x55edbf8174c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:11.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.858+0000 7f3c8b5bfb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edbf6faa60 con 0x55edbf6656b0 2023-07-21T18:44:11.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.858+0000 7f3c8b5bfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edbf6fa760 con 0x55edbf813b10 2023-07-21T18:44:11.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.858+0000 7f3c8b5bfb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edbf5de8c0 con 0x55edbf8136a0 2023-07-21T18:44:11.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.858+0000 7f3c8b5bfb80 10 client.0 _ll_register_callbacks cb 0x55edbf6f8700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:44:11.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.858+0000 7f3c7d1d5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55edbf6656b0 0x55edbf819a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:11.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.858+0000 7f3c7d1d5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55edbf6656b0 0x55edbf819a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59252/0 (socket says 172.21.15.39:59252) 2023-07-21T18:44:11.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.858+0000 7f3c7d9d6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edbf813b10 0x55edbf8174c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:11.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.858+0000 7f3c7d1d5700 1 -- 172.21.15.39:0/575754500 learned_addr learned my addr 172.21.15.39:0/575754500 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:44:11.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.858+0000 7f3c7c9d4700 1 --2- 172.21.15.39:0/575754500 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55edbf8136a0 0x55edbf816d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:11.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.859+0000 7f3c7d9d6700 1 -- 172.21.15.39:0/575754500 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55edbf8136a0 msgr2=0x55edbf816d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:44:11.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.859+0000 7f3c7d9d6700 1 --2- 172.21.15.39:0/575754500 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55edbf8136a0 0x55edbf816d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:11.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.859+0000 7f3c7d9d6700 1 -- 172.21.15.39:0/575754500 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55edbf6656b0 msgr2=0x55edbf819a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:44:11.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.859+0000 7f3c7d9d6700 1 --2- 172.21.15.39:0/575754500 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55edbf6656b0 0x55edbf819a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:11.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.859+0000 7f3c7d9d6700 1 -- 172.21.15.39:0/575754500 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c781a2bc0 con 0x55edbf813b10 2023-07-21T18:44:11.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.859+0000 7f3c7d1d5700 1 --2- 172.21.15.39:0/575754500 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55edbf6656b0 0x55edbf819a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:44:11.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.859+0000 7f3c7d9d6700 1 --2- 172.21.15.39:0/575754500 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edbf813b10 0x55edbf8174c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3c7819c8d0 tx=0x7f3c70014600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2952498b8d6c5282 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:44:11.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.859+0000 7f3c60ff9700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-21T18:44:11.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.859+0000 7f3c60ff9700 1 -- 172.21.15.39:0/575754500 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c7000e040 con 0x55edbf813b10 2023-07-21T18:44:11.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.859+0000 7f3c8b5bfb80 1 -- 172.21.15.39:0/575754500 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55edbf6b4490 con 0x55edbf813b10 2023-07-21T18:44:11.886 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.859+0000 7f3c60ff9700 1 -- 172.21.15.39:0/575754500 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f3c70014b30 con 0x55edbf813b10 2023-07-21T18:44:11.886 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.859+0000 7f3c60ff9700 1 -- 172.21.15.39:0/575754500 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c70012eb0 con 0x55edbf813b10 2023-07-21T18:44:11.886 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.859+0000 7f3c8b5bfb80 10 client.25793 fetch_fsmap learned FSMap version 33 2023-07-21T18:44:11.886 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.859+0000 7f3c8b5bfb80 1 -- 172.21.15.39:0/575754500 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55edbf571c00 con 0x55edbf813b10 2023-07-21T18:44:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.860+0000 7f3c60ff9700 1 -- 172.21.15.39:0/575754500 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55edbf6b4490 con 0x55edbf813b10 2023-07-21T18:44:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.860+0000 7f3c60ff9700 1 -- 172.21.15.39:0/575754500 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f3c7001c020 con 0x55edbf813b10 2023-07-21T18:44:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.860+0000 7f3c8b5bfb80 10 client.25793 fetch_fsmap finished waiting for FSMap version 33 2023-07-21T18:44:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.860+0000 7f3c8b5bfb80 10 client.25793 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:44:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.860+0000 7f3c8b5bfb80 20 client.25793 populate_metadata read hostname 'smithi039' 2023-07-21T18:44:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.860+0000 7f3c8b5bfb80 1 --2- 172.21.15.39:0/575754500 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55edbf827150 0x55edbf829560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.860+0000 7f3c8b5bfb80 4 client.25793 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:44:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.860+0000 7f3c8b5bfb80 1 -- 172.21.15.39:0/575754500 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55edbf753e50 con 0x55edbf827150 2023-07-21T18:44:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.860+0000 7f3c7d1d5700 1 --2- 172.21.15.39:0/575754500 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55edbf827150 0x55edbf829560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.860+0000 7f3c7d1d5700 1 --2- 172.21.15.39:0/575754500 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55edbf827150 0x55edbf829560 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca67a68d8c33ed82 server_cookie=f1dd7f4b422e388e in_seq=0 out_seq=0 2023-07-21T18:44:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.860+0000 7f3c60ff9700 10 client.25793 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:44:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.861+0000 7f3c60ff9700 1 -- 172.21.15.39:0/575754500 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55edbf753e50 con 0x55edbf827150 2023-07-21T18:44:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.861+0000 7f3c60ff9700 10 client.25793 handle_command_reply: tid=0 2023-07-21T18:44:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.861+0000 7f3c8b5bfb80 1 client.25793 shutdown 2023-07-21T18:44:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.861+0000 7f3c8b5bfb80 1 -- 172.21.15.39:0/575754500 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edbf813b10 msgr2=0x55edbf8174c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:44:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.861+0000 7f3c8b5bfb80 1 --2- 172.21.15.39:0/575754500 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edbf813b10 0x55edbf8174c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3c7819c8d0 tx=0x7f3c70014600 comp rx=0 tx=0).stop 2023-07-21T18:44:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.861+0000 7f3c8b5bfb80 1 -- 172.21.15.39:0/575754500 shutdown_connections 2023-07-21T18:44:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.861+0000 7f3c8b5bfb80 1 --2- 172.21.15.39:0/575754500 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55edbf827150 0x55edbf829560 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.861+0000 7f3c8b5bfb80 1 --2- 172.21.15.39:0/575754500 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edbf813b10 0x55edbf8174c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.861+0000 7f3c8b5bfb80 1 --2- 172.21.15.39:0/575754500 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55edbf6656b0 0x55edbf819a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.861+0000 7f3c8b5bfb80 1 --2- 172.21.15.39:0/575754500 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55edbf8136a0 0x55edbf816d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.861+0000 7f3c60ff9700 0 client.25793 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:44:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.861+0000 7f3c8b5bfb80 1 -- 172.21.15.39:0/575754500 >> 172.21.15.39:0/575754500 conn(0x55edbf7febe0 msgr2=0x55edbf8034e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:44:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.861+0000 7f3c8b5bfb80 1 -- 172.21.15.39:0/575754500 shutdown_connections 2023-07-21T18:44:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.861+0000 7f3c8b5bfb80 1 -- 172.21.15.39:0/575754500 wait complete. 2023-07-21T18:44:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.862+0000 7f3c8b5bfb80 20 client.25793 trim_cache size 0 max 16384 2023-07-21T18:44:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.877+0000 7f3c8b5bfb80 1 Processor -- start 2023-07-21T18:44:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.877+0000 7f3c8b5bfb80 1 -- start start 2023-07-21T18:44:11.897 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.877+0000 7f3c8b5bfb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55edbf6656b0 0x55edbf81e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:11.897 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.877+0000 7f3c8b5bfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edbf8136a0 0x55edbf823340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:11.897 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.877+0000 7f3c8b5bfb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55edbf813b10 0x55edbf803c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:11.897 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.877+0000 7f3c8b5bfb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edbf80e5f0 con 0x55edbf6656b0 2023-07-21T18:44:11.897 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.877+0000 7f3c8b5bfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edbf8180d0 con 0x55edbf8136a0 2023-07-21T18:44:11.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.877+0000 7f3c8b5bfb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edbf7fbbb0 con 0x55edbf813b10 2023-07-21T18:44:11.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.878+0000 7f3c7d1d5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55edbf6656b0 0x55edbf81e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:11.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.878+0000 7f3c7c9d4700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edbf8136a0 0x55edbf823340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:11.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.878+0000 7f3c7d9d6700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55edbf813b10 0x55edbf803c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:11.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.878+0000 7f3c7c9d4700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edbf8136a0 0x55edbf823340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:43626/0 (socket says 172.21.15.39:43626) 2023-07-21T18:44:11.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.878+0000 7f3c7c9d4700 1 -- 172.21.15.39:0/2005499465 learned_addr learned my addr 172.21.15.39:0/2005499465 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:44:11.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.878+0000 7f3c7d1d5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55edbf6656b0 0x55edbf81e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59260/0 (socket says 172.21.15.39:59260) 2023-07-21T18:44:11.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.878+0000 7f3c7d9d6700 1 -- 172.21.15.39:0/2005499465 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edbf8136a0 msgr2=0x55edbf823340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:44:11.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.878+0000 7f3c7d9d6700 1 --2- 172.21.15.39:0/2005499465 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edbf8136a0 0x55edbf823340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:11.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.878+0000 7f3c7d9d6700 1 -- 172.21.15.39:0/2005499465 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55edbf6656b0 msgr2=0x55edbf81e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:44:11.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.878+0000 7f3c7d9d6700 1 --2- 172.21.15.39:0/2005499465 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55edbf6656b0 0x55edbf81e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:11.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.878+0000 7f3c7d9d6700 1 -- 172.21.15.39:0/2005499465 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55edbf5af1f0 con 0x55edbf813b10 2023-07-21T18:44:11.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.878+0000 7f3c7d9d6700 1 --2- 172.21.15.39:0/2005499465 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55edbf813b10 0x55edbf803c60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3c7000b010 tx=0x7f3c700046a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b42515590cb034c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:44:11.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.878+0000 7f3c63fff700 1 -- 172.21.15.39:0/2005499465 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c7000e070 con 0x55edbf813b10 2023-07-21T18:44:11.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.878+0000 7f3c63fff700 1 -- 172.21.15.39:0/2005499465 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c7000f040 con 0x55edbf813b10 2023-07-21T18:44:11.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.878+0000 7f3c63fff700 1 -- 172.21.15.39:0/2005499465 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c70003a00 con 0x55edbf813b10 2023-07-21T18:44:11.902 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.879+0000 7f3c8b5bfb80 1 -- 172.21.15.39:0/2005499465 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55edbf813b10 msgr2=0x55edbf803c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:44:11.902 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.879+0000 7f3c8b5bfb80 1 --2- 172.21.15.39:0/2005499465 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55edbf813b10 0x55edbf803c60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3c7000b010 tx=0x7f3c700046a0 comp rx=0 tx=0).stop 2023-07-21T18:44:11.902 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.879+0000 7f3c8b5bfb80 1 -- 172.21.15.39:0/2005499465 shutdown_connections 2023-07-21T18:44:11.902 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.879+0000 7f3c8b5bfb80 1 --2- 172.21.15.39:0/2005499465 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55edbf813b10 0x55edbf803c60 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:11.902 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.879+0000 7f3c8b5bfb80 1 --2- 172.21.15.39:0/2005499465 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55edbf6656b0 0x55edbf81e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:11.903 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.879+0000 7f3c8b5bfb80 1 --2- 172.21.15.39:0/2005499465 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edbf8136a0 0x55edbf823340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:11.903 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.879+0000 7f3c8b5bfb80 1 -- 172.21.15.39:0/2005499465 >> 172.21.15.39:0/2005499465 conn(0x55edbf7febe0 msgr2=0x55edbf82de80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:44:11.903 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.879+0000 7f3c8b5bfb80 1 -- 172.21.15.39:0/2005499465 shutdown_connections 2023-07-21T18:44:11.903 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.879+0000 7f3c8b5bfb80 1 -- 172.21.15.39:0/2005499465 wait complete. 2023-07-21T18:44:11.903 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.879+0000 7f3c8b5bfb80 1 -- start start 2023-07-21T18:44:11.903 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.879+0000 7f3c8b5bfb80 1 Processor -- start 2023-07-21T18:44:11.903 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.880+0000 7f3c8b5bfb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55edbf6656b0 0x55edbf81d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:11.904 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.880+0000 7f3c8b5bfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edbf8136a0 0x55edbf81dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:11.904 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.880+0000 7f3c8b5bfb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55edbf813b10 0x55edbf879fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:11.904 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.880+0000 7f3c8b5bfb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edbf87a510 con 0x55edbf6656b0 2023-07-21T18:44:11.904 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.880+0000 7f3c8b5bfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edbf87a650 con 0x55edbf8136a0 2023-07-21T18:44:11.904 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.880+0000 7f3c8b5bfb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edbf87a7c0 con 0x55edbf813b10 2023-07-21T18:44:11.904 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.880+0000 7f3c8b5bfb80 10 client.0 _ll_register_callbacks cb 0x7f3c700126f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:44:11.904 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.880+0000 7f3c7d1d5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55edbf6656b0 0x55edbf81d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:11.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.880+0000 7f3c7d1d5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55edbf6656b0 0x55edbf81d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59266/0 (socket says 172.21.15.39:59266) 2023-07-21T18:44:11.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.880+0000 7f3c7d1d5700 1 -- 172.21.15.39:0/1868874916 learned_addr learned my addr 172.21.15.39:0/1868874916 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:44:11.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.880+0000 7f3c7d9d6700 1 --2- 172.21.15.39:0/1868874916 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55edbf813b10 0x55edbf879fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:11.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.880+0000 7f3c7c9d4700 1 --2- 172.21.15.39:0/1868874916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edbf8136a0 0x55edbf81dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:11.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.880+0000 7f3c7d1d5700 1 -- 172.21.15.39:0/1868874916 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55edbf813b10 msgr2=0x55edbf879fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:44:11.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.880+0000 7f3c7d1d5700 1 --2- 172.21.15.39:0/1868874916 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55edbf813b10 0x55edbf879fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:11.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.880+0000 7f3c7d1d5700 1 -- 172.21.15.39:0/1868874916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edbf8136a0 msgr2=0x55edbf81dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:44:11.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.880+0000 7f3c7d1d5700 1 --2- 172.21.15.39:0/1868874916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edbf8136a0 0x55edbf81dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.880+0000 7f3c7d1d5700 1 -- 172.21.15.39:0/1868874916 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55edbf571c00 con 0x55edbf6656b0 2023-07-21T18:44:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.880+0000 7f3c7d1d5700 1 --2- 172.21.15.39:0/1868874916 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55edbf6656b0 0x55edbf81d6e0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f3c74096ab0 tx=0x7f3c7400efa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21284bdbb477e542 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:44:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.880+0000 7f3c60ff9700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:44:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.881+0000 7f3c60ff9700 1 -- 172.21.15.39:0/1868874916 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c740b8040 con 0x55edbf6656b0 2023-07-21T18:44:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.881+0000 7f3c60ff9700 1 -- 172.21.15.39:0/1868874916 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f3c7409a040 con 0x55edbf6656b0 2023-07-21T18:44:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.881+0000 7f3c8b5bfb80 1 -- 172.21.15.39:0/1868874916 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3c7409e070 con 0x55edbf6656b0 2023-07-21T18:44:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.881+0000 7f3c60ff9700 1 -- 172.21.15.39:0/1868874916 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c74096380 con 0x55edbf6656b0 2023-07-21T18:44:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.881+0000 7f3c8b5bfb80 10 client.16299 fetch_fsmap learned FSMap version 33 2023-07-21T18:44:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.881+0000 7f3c8b5bfb80 1 -- 172.21.15.39:0/1868874916 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55edbf587250 con 0x55edbf6656b0 2023-07-21T18:44:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.881+0000 7f3c60ff9700 1 -- 172.21.15.39:0/1868874916 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c7409e070 con 0x55edbf6656b0 2023-07-21T18:44:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.881+0000 7f3c60ff9700 1 -- 172.21.15.39:0/1868874916 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f3c740a2090 con 0x55edbf6656b0 2023-07-21T18:44:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.881+0000 7f3c8b5bfb80 10 client.16299 fetch_fsmap finished waiting for FSMap version 33 2023-07-21T18:44:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.881+0000 7f3c8b5bfb80 10 client.16299 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:44:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.881+0000 7f3c8b5bfb80 20 client.16299 populate_metadata read hostname 'smithi039' 2023-07-21T18:44:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.881+0000 7f3c8b5bfb80 1 --2- 172.21.15.39:0/1868874916 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55edbf87baf0 0x55edbf87df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:11.908 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.881+0000 7f3c8b5bfb80 4 client.16299 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:44:11.908 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.881+0000 7f3c8b5bfb80 1 -- 172.21.15.39:0/1868874916 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55edbf715ff0 con 0x55edbf87baf0 2023-07-21T18:44:11.908 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.881+0000 7f3c7c9d4700 1 --2- 172.21.15.39:0/1868874916 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55edbf87baf0 0x55edbf87df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:11.908 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.882+0000 7f3c60ff9700 10 client.16299 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:44:11.908 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.882+0000 7f3c7c9d4700 1 --2- 172.21.15.39:0/1868874916 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55edbf87baf0 0x55edbf87df00 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=348c4b51fb1974e7 server_cookie=8bda2b4c913b049c in_seq=0 out_seq=0 2023-07-21T18:44:11.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.924+0000 7f3c60ff9700 1 -- 172.21.15.39:0/1868874916 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55edbf715ff0 con 0x55edbf87baf0 2023-07-21T18:44:11.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.924+0000 7f3c60ff9700 10 client.16299 handle_command_reply: tid=0 2023-07-21T18:44:11.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.924+0000 7f3c8b5bfb80 1 client.16299 shutdown 2023-07-21T18:44:11.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.924+0000 7f3c8b5bfb80 1 -- 172.21.15.39:0/1868874916 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55edbf6656b0 msgr2=0x55edbf81d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:44:11.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.924+0000 7f3c8b5bfb80 1 --2- 172.21.15.39:0/1868874916 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55edbf6656b0 0x55edbf81d6e0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f3c74096ab0 tx=0x7f3c7400efa0 comp rx=0 tx=0).stop 2023-07-21T18:44:11.944 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.924+0000 7f3c8b5bfb80 1 -- 172.21.15.39:0/1868874916 shutdown_connections 2023-07-21T18:44:11.944 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.924+0000 7f3c8b5bfb80 1 --2- 172.21.15.39:0/1868874916 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55edbf87baf0 0x55edbf87df00 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:11.944 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.924+0000 7f3c8b5bfb80 1 --2- 172.21.15.39:0/1868874916 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55edbf813b10 0x55edbf879fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:11.944 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.924+0000 7f3c60ff9700 0 client.16299 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:44:11.944 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.924+0000 7f3c8b5bfb80 1 --2- 172.21.15.39:0/1868874916 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55edbf6656b0 0x55edbf81d6e0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:11.944 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.924+0000 7f3c8b5bfb80 1 --2- 172.21.15.39:0/1868874916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edbf8136a0 0x55edbf81dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:11.944 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.924+0000 7f3c8b5bfb80 1 -- 172.21.15.39:0/1868874916 >> 172.21.15.39:0/1868874916 conn(0x55edbf7febe0 msgr2=0x55edbf823340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:44:11.944 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.924+0000 7f3c8b5bfb80 1 -- 172.21.15.39:0/1868874916 shutdown_connections 2023-07-21T18:44:11.945 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.924+0000 7f3c8b5bfb80 1 -- 172.21.15.39:0/1868874916 wait complete. 2023-07-21T18:44:11.945 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.925+0000 7f3c8b5bfb80 20 client.16299 trim_cache size 0 max 16384 2023-07-21T18:44:11.945 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:44:11.945 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:44:11.946 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:44:11.946 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:44:11.946 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.925+0000 7f3c7f439700 1 -- 172.21.15.39:0/384597953 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f3c640a1340 msgr2=0x7f3c640a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:44:11.946 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.925+0000 7f3c7f439700 1 --2- 172.21.15.39:0/384597953 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f3c640a1340 0x7f3c640a37f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3c780fef80 tx=0x7f3c6800b260 comp rx=0 tx=0).stop 2023-07-21T18:44:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.925+0000 7f3c7f439700 1 -- 172.21.15.39:0/384597953 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c780ffb30 msgr2=0x7f3c7819bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:44:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.925+0000 7f3c7f439700 1 --2- 172.21.15.39:0/384597953 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c780ffb30 0x7f3c7819bbd0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f3c7400e9c0 tx=0x7f3c7400ecd0 comp rx=0 tx=0).stop 2023-07-21T18:44:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.925+0000 7f3c7f439700 1 -- 172.21.15.39:0/384597953 shutdown_connections 2023-07-21T18:44:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.925+0000 7f3c7f439700 1 --2- 172.21.15.39:0/384597953 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f3c640a1340 0x7f3c640a37f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.925+0000 7f3c7f439700 1 --2- 172.21.15.39:0/384597953 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c78105b60 0x7f3c78195a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.925+0000 7f3c7f439700 1 --2- 172.21.15.39:0/384597953 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c780ffb30 0x7f3c7819bbd0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.925+0000 7f3c7f439700 1 --2- 172.21.15.39:0/384597953 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c781004e0 0x7f3c7819c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:11.948 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.925+0000 7f3c7f439700 1 -- 172.21.15.39:0/384597953 >> 172.21.15.39:0/384597953 conn(0x7f3c780fb5f0 msgr2=0x7f3c780fcb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:44:11.948 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.925+0000 7f3c7f439700 1 -- 172.21.15.39:0/384597953 shutdown_connections 2023-07-21T18:44:11.948 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:11.926+0000 7f3c7f439700 1 -- 172.21.15.39:0/384597953 wait complete. 2023-07-21T18:44:12.262 INFO:tasks.workunit.client.0.smithi039.stdout: 1 86578 5.71 MB/sec warmup 85 sec latency 40.897 ms 2023-07-21T18:44:12.366 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:44:12 smithi193 ceph-mon[98204]: pgmap v76: 97 pgs: 97 active+clean; 53 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 371 op/s 2023-07-21T18:44:12.385 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:44:12 smithi155 ceph-mon[97934]: pgmap v76: 97 pgs: 97 active+clean; 53 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 371 op/s 2023-07-21T18:44:12.434 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:44:12 smithi039 ceph-mon[96489]: pgmap v76: 97 pgs: 97 active+clean; 53 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 371 op/s 2023-07-21T18:44:13.262 INFO:tasks.workunit.client.0.smithi039.stdout: 1 88967 5.80 MB/sec warmup 86 sec latency 10.362 ms 2023-07-21T18:44:14.263 INFO:tasks.workunit.client.0.smithi039.stdout: 1 90387 5.80 MB/sec warmup 87 sec latency 42.308 ms 2023-07-21T18:44:14.366 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:44:14 smithi193 ceph-mon[98204]: pgmap v77: 97 pgs: 97 active+clean; 53 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 245 op/s 2023-07-21T18:44:14.385 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:44:14 smithi155 ceph-mon[97934]: pgmap v77: 97 pgs: 97 active+clean; 53 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 245 op/s 2023-07-21T18:44:14.434 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:44:14 smithi039 ceph-mon[96489]: pgmap v77: 97 pgs: 97 active+clean; 53 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 245 op/s 2023-07-21T18:44:15.263 INFO:tasks.workunit.client.0.smithi039.stdout: 1 92571 5.88 MB/sec warmup 88 sec latency 18.403 ms 2023-07-21T18:44:16.263 INFO:tasks.workunit.client.0.smithi039.stdout: 1 94617 5.90 MB/sec warmup 89 sec latency 12.037 ms 2023-07-21T18:44:16.366 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:44:16 smithi193 ceph-mon[98204]: pgmap v78: 97 pgs: 97 active+clean; 53 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 245 op/s 2023-07-21T18:44:16.385 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:44:16 smithi155 ceph-mon[97934]: pgmap v78: 97 pgs: 97 active+clean; 53 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 245 op/s 2023-07-21T18:44:16.433 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:44:16 smithi039 ceph-mon[96489]: pgmap v78: 97 pgs: 97 active+clean; 53 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 245 op/s 2023-07-21T18:44:17.263 INFO:tasks.workunit.client.0.smithi039.stdout: 1 96236 5.97 MB/sec warmup 90 sec latency 10.545 ms 2023-07-21T18:44:18.263 INFO:tasks.workunit.client.0.smithi039.stdout: 1 98375 5.99 MB/sec warmup 91 sec latency 10.007 ms 2023-07-21T18:44:18.366 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:44:18 smithi193 ceph-mon[98204]: pgmap v79: 97 pgs: 97 active+clean; 53 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 380 op/s 2023-07-21T18:44:18.385 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:44:18 smithi155 ceph-mon[97934]: pgmap v79: 97 pgs: 97 active+clean; 53 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 380 op/s 2023-07-21T18:44:18.433 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:44:18 smithi039 ceph-mon[96489]: pgmap v79: 97 pgs: 97 active+clean; 53 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 380 op/s 2023-07-21T18:44:19.263 INFO:tasks.workunit.client.0.smithi039.stdout: 1 99897 6.05 MB/sec warmup 92 sec latency 28.147 ms 2023-07-21T18:44:20.263 INFO:tasks.workunit.client.0.smithi039.stdout: 1 102279 6.12 MB/sec warmup 93 sec latency 15.874 ms 2023-07-21T18:44:20.366 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:44:20 smithi193 ceph-mon[98204]: pgmap v80: 97 pgs: 97 active+clean; 53 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 244 op/s 2023-07-21T18:44:20.385 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:44:20 smithi155 ceph-mon[97934]: pgmap v80: 97 pgs: 97 active+clean; 53 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 244 op/s 2023-07-21T18:44:20.433 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:44:20 smithi039 ceph-mon[96489]: pgmap v80: 97 pgs: 97 active+clean; 53 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 244 op/s 2023-07-21T18:44:21.263 INFO:tasks.workunit.client.0.smithi039.stdout: 1 103642 6.13 MB/sec warmup 94 sec latency 31.530 ms 2023-07-21T18:44:22.263 INFO:tasks.workunit.client.0.smithi039.stdout: 1 106271 6.21 MB/sec warmup 95 sec latency 10.642 ms 2023-07-21T18:44:22.366 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:44:22 smithi193 ceph-mon[98204]: pgmap v81: 97 pgs: 97 active+clean; 60 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 358 op/s 2023-07-21T18:44:22.384 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:44:22 smithi155 ceph-mon[97934]: pgmap v81: 97 pgs: 97 active+clean; 60 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 358 op/s 2023-07-21T18:44:22.433 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:44:22 smithi039 ceph-mon[96489]: pgmap v81: 97 pgs: 97 active+clean; 60 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 358 op/s 2023-07-21T18:44:23.263 INFO:tasks.workunit.client.0.smithi039.stdout: 1 107629 6.22 MB/sec warmup 96 sec latency 10.864 ms 2023-07-21T18:44:24.263 INFO:tasks.workunit.client.0.smithi039.stdout: 1 110209 6.32 MB/sec warmup 97 sec latency 9.599 ms 2023-07-21T18:44:24.366 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:44:24 smithi193 ceph-mon[98204]: pgmap v82: 97 pgs: 97 active+clean; 60 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 249 op/s 2023-07-21T18:44:24.384 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:44:24 smithi155 ceph-mon[97934]: pgmap v82: 97 pgs: 97 active+clean; 60 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 249 op/s 2023-07-21T18:44:24.433 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:44:24 smithi039 ceph-mon[96489]: pgmap v82: 97 pgs: 97 active+clean; 60 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 249 op/s 2023-07-21T18:44:25.058 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.038+0000 7fb101bdd700 1 -- 172.21.15.39:0/3408772841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0fc0fea30 msgr2=0x7fb0fc0feea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:44:25.058 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.038+0000 7fb101bdd700 1 --2- 172.21.15.39:0/3408772841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0fc0fea30 0x7fb0fc0feea0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0009990 tx=0x7fb0f0009ca0 comp rx=0 tx=0).stop 2023-07-21T18:44:25.059 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.038+0000 7fb101bdd700 1 -- 172.21.15.39:0/3408772841 shutdown_connections 2023-07-21T18:44:25.059 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.038+0000 7fb101bdd700 1 --2- 172.21.15.39:0/3408772841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0fc104a30 0x7fb0fc104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:25.059 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.038+0000 7fb101bdd700 1 --2- 172.21.15.39:0/3408772841 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0fc0ff3e0 0x7fb0fc10b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:25.059 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.038+0000 7fb101bdd700 1 --2- 172.21.15.39:0/3408772841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0fc0fea30 0x7fb0fc0feea0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:25.059 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.038+0000 7fb101bdd700 1 -- 172.21.15.39:0/3408772841 >> 172.21.15.39:0/3408772841 conn(0x7fb0fc074760 msgr2=0x7fb0fc074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:44:25.059 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.038+0000 7fb101bdd700 1 -- 172.21.15.39:0/3408772841 shutdown_connections 2023-07-21T18:44:25.059 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.038+0000 7fb101bdd700 1 -- 172.21.15.39:0/3408772841 wait complete. 2023-07-21T18:44:25.060 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.039+0000 7fb101bdd700 1 Processor -- start 2023-07-21T18:44:25.061 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.039+0000 7fb101bdd700 1 -- start start 2023-07-21T18:44:25.061 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.039+0000 7fb101bdd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0fc0fea30 0x7fb0fc19aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:25.062 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.040+0000 7fb101bdd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0fc0ff3e0 0x7fb0fc19af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:25.062 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.040+0000 7fb101bdd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0fc104a30 0x7fb0fc1948a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:25.062 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.040+0000 7fb101bdd700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0fc194f10 con 0x7fb0fc104a30 2023-07-21T18:44:25.062 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.040+0000 7fb101bdd700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0fc195080 con 0x7fb0fc0fea30 2023-07-21T18:44:25.062 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.040+0000 7fb101bdd700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0fc195240 con 0x7fb0fc0ff3e0 2023-07-21T18:44:25.062 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.040+0000 7fb0fb7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0fc0fea30 0x7fb0fc19aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:25.062 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.040+0000 7fb0faffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0fc0ff3e0 0x7fb0fc19af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:25.062 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.040+0000 7fb0fb7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0fc0fea30 0x7fb0fc19aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:43644/0 (socket says 172.21.15.39:43644) 2023-07-21T18:44:25.063 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.040+0000 7fb0fb7fe700 1 -- 172.21.15.39:0/504678555 learned_addr learned my addr 172.21.15.39:0/504678555 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:44:25.063 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.040+0000 7fb0fbfff700 1 --2- 172.21.15.39:0/504678555 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0fc104a30 0x7fb0fc1948a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:25.063 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.040+0000 7fb0faffd700 1 -- 172.21.15.39:0/504678555 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0fc0fea30 msgr2=0x7fb0fc19aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:44:25.063 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.040+0000 7fb0faffd700 1 --2- 172.21.15.39:0/504678555 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0fc0fea30 0x7fb0fc19aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:25.063 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.040+0000 7fb0faffd700 1 -- 172.21.15.39:0/504678555 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0fc104a30 msgr2=0x7fb0fc1948a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:44:25.063 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.040+0000 7fb0faffd700 1 --2- 172.21.15.39:0/504678555 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0fc104a30 0x7fb0fc1948a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:25.063 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.040+0000 7fb0faffd700 1 -- 172.21.15.39:0/504678555 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f00095f0 con 0x7fb0fc0ff3e0 2023-07-21T18:44:25.064 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.040+0000 7fb0fb7fe700 1 --2- 172.21.15.39:0/504678555 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0fc0fea30 0x7fb0fc19aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:44:25.064 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.040+0000 7fb0fbfff700 1 --2- 172.21.15.39:0/504678555 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0fc104a30 0x7fb0fc1948a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:44:25.064 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.040+0000 7fb0faffd700 1 --2- 172.21.15.39:0/504678555 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0fc0ff3e0 0x7fb0fc19af80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb0f00059d0 tx=0x7fb0f0005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:44:25.064 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.040+0000 7fb0f8ff9700 1 -- 172.21.15.39:0/504678555 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f001d070 con 0x7fb0fc0ff3e0 2023-07-21T18:44:25.064 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.041+0000 7fb0f8ff9700 1 -- 172.21.15.39:0/504678555 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0f0022470 con 0x7fb0fc0ff3e0 2023-07-21T18:44:25.064 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.041+0000 7fb101bdd700 1 -- 172.21.15.39:0/504678555 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0fc1954c0 con 0x7fb0fc0ff3e0 2023-07-21T18:44:25.064 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.041+0000 7fb0f8ff9700 1 -- 172.21.15.39:0/504678555 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f0017650 con 0x7fb0fc0ff3e0 2023-07-21T18:44:25.064 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.041+0000 7fb101bdd700 1 -- 172.21.15.39:0/504678555 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0fc1a1a30 con 0x7fb0fc0ff3e0 2023-07-21T18:44:25.068 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.043+0000 7fb0f8ff9700 1 -- 172.21.15.39:0/504678555 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fb0f0022890 con 0x7fb0fc0ff3e0 2023-07-21T18:44:25.068 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.043+0000 7fb0f8ff9700 1 --2- 172.21.15.39:0/504678555 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fb0e40a12c0 0x7fb0e40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:25.068 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.044+0000 7fb0f8ff9700 1 -- 172.21.15.39:0/504678555 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fb0f00ce130 con 0x7fb0fc0ff3e0 2023-07-21T18:44:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.044+0000 7fb0fb7fe700 1 --2- 172.21.15.39:0/504678555 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fb0e40a12c0 0x7fb0e40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.044+0000 7fb10dd63b80 1 Processor -- start 2023-07-21T18:44:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.044+0000 7fb10dd63b80 1 -- start start 2023-07-21T18:44:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.044+0000 7fb0fb7fe700 1 --2- 172.21.15.39:0/504678555 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fb0e40a12c0 0x7fb0e40a3770 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec005950 tx=0x7fb0ec009260 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:44:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.044+0000 7fb10dd63b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e9f708f6b0 0x55e9f723b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.044+0000 7fb10dd63b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e9f723b550 0x55e9f7235250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.044+0000 7fb10dd63b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9f7235790 0x55e9f7235be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:25.070 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.044+0000 7fb10dd63b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f7138c50 con 0x55e9f708f6b0 2023-07-21T18:44:25.070 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.044+0000 7fb10dd63b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f718b9e0 con 0x55e9f7235790 2023-07-21T18:44:25.070 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.044+0000 7fb10dd63b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f7184e70 con 0x55e9f723b550 2023-07-21T18:44:25.070 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.044+0000 7fb0fb7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e9f708f6b0 0x55e9f723b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:25.070 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.044+0000 7fb0fb7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e9f708f6b0 0x55e9f723b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59288/0 (socket says 172.21.15.39:59288) 2023-07-21T18:44:25.070 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.044+0000 7fb0fb7fe700 1 -- 172.21.15.39:0/3264157047 learned_addr learned my addr 172.21.15.39:0/3264157047 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:44:25.070 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.045+0000 7fb0fbfff700 1 --2- 172.21.15.39:0/3264157047 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9f7235790 0x55e9f7235be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:25.070 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.045+0000 7fb0faffd700 1 --2- 172.21.15.39:0/3264157047 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e9f723b550 0x55e9f7235250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.045+0000 7fb0fb7fe700 1 -- 172.21.15.39:0/3264157047 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e9f723b550 msgr2=0x55e9f7235250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:44:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.045+0000 7fb0fb7fe700 1 --2- 172.21.15.39:0/3264157047 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e9f723b550 0x55e9f7235250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.045+0000 7fb0fb7fe700 1 -- 172.21.15.39:0/3264157047 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9f7235790 msgr2=0x55e9f7235be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:44:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.045+0000 7fb0fb7fe700 1 --2- 172.21.15.39:0/3264157047 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9f7235790 0x55e9f7235be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.045+0000 7fb0fb7fe700 1 -- 172.21.15.39:0/3264157047 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9f6fb2d80 con 0x55e9f708f6b0 2023-07-21T18:44:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.045+0000 7fb0fbfff700 1 --2- 172.21.15.39:0/3264157047 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9f7235790 0x55e9f7235be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:44:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.045+0000 7fb0fb7fe700 1 --2- 172.21.15.39:0/3264157047 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e9f708f6b0 0x55e9f723b010 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec00e730 tx=0x7fb0ec014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b625c4bffab58c6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:44:25.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.045+0000 7fb0e1ffb700 1 -- 172.21.15.39:0/3264157047 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec004830 con 0x55e9f708f6b0 2023-07-21T18:44:25.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.045+0000 7fb0e1ffb700 1 -- 172.21.15.39:0/3264157047 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0ec004e50 con 0x55e9f708f6b0 2023-07-21T18:44:25.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.045+0000 7fb0e1ffb700 1 -- 172.21.15.39:0/3264157047 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec01e680 con 0x55e9f708f6b0 2023-07-21T18:44:25.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.046+0000 7fb10dd63b80 1 -- 172.21.15.39:0/3264157047 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e9f708f6b0 msgr2=0x55e9f723b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:44:25.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.046+0000 7fb10dd63b80 1 --2- 172.21.15.39:0/3264157047 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e9f708f6b0 0x55e9f723b010 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec00e730 tx=0x7fb0ec014000 comp rx=0 tx=0).stop 2023-07-21T18:44:25.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.046+0000 7fb10dd63b80 1 -- 172.21.15.39:0/3264157047 shutdown_connections 2023-07-21T18:44:25.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.046+0000 7fb10dd63b80 1 --2- 172.21.15.39:0/3264157047 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9f7235790 0x55e9f7235be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:25.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.046+0000 7fb10dd63b80 1 --2- 172.21.15.39:0/3264157047 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e9f708f6b0 0x55e9f723b010 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:25.073 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.046+0000 7fb10dd63b80 1 --2- 172.21.15.39:0/3264157047 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e9f723b550 0x55e9f7235250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:25.073 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.046+0000 7fb10dd63b80 1 -- 172.21.15.39:0/3264157047 >> 172.21.15.39:0/3264157047 conn(0x55e9f7228be0 msgr2=0x55e9f722df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:44:25.073 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.046+0000 7fb10dd63b80 1 -- 172.21.15.39:0/3264157047 shutdown_connections 2023-07-21T18:44:25.073 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.046+0000 7fb10dd63b80 1 -- 172.21.15.39:0/3264157047 wait complete. 2023-07-21T18:44:25.073 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.046+0000 7fb10dd63b80 1 -- start start 2023-07-21T18:44:25.073 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.047+0000 7fb10dd63b80 1 Processor -- start 2023-07-21T18:44:25.073 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.047+0000 7fb10dd63b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e9f708f6b0 0x55e9f72416d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:25.074 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.047+0000 7fb10dd63b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9f7235790 0x55e9f723c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:25.074 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.047+0000 7fb10dd63b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e9f723b550 0x55e9f723cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:25.074 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.047+0000 7fb10dd63b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f7124a60 con 0x55e9f708f6b0 2023-07-21T18:44:25.074 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.047+0000 7fb10dd63b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f7124760 con 0x55e9f7235790 2023-07-21T18:44:25.074 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.047+0000 7fb10dd63b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f70088c0 con 0x55e9f723b550 2023-07-21T18:44:25.074 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.047+0000 7fb10dd63b80 10 client.0 _ll_register_callbacks cb 0x55e9f7122700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:44:25.074 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.047+0000 7fb0fb7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e9f708f6b0 0x55e9f72416d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:25.074 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.047+0000 7fb0fb7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e9f708f6b0 0x55e9f72416d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59294/0 (socket says 172.21.15.39:59294) 2023-07-21T18:44:25.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.047+0000 7fb0fb7fe700 1 -- 172.21.15.39:0/1894004713 learned_addr learned my addr 172.21.15.39:0/1894004713 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:44:25.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.047+0000 7fb0faffd700 1 --2- 172.21.15.39:0/1894004713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9f7235790 0x55e9f723c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:25.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.047+0000 7fb0fbfff700 1 --2- 172.21.15.39:0/1894004713 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e9f723b550 0x55e9f723cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:25.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.047+0000 7fb0faffd700 1 -- 172.21.15.39:0/1894004713 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e9f723b550 msgr2=0x55e9f723cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:44:25.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.047+0000 7fb0faffd700 1 --2- 172.21.15.39:0/1894004713 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e9f723b550 0x55e9f723cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:25.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.047+0000 7fb0faffd700 1 -- 172.21.15.39:0/1894004713 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e9f708f6b0 msgr2=0x55e9f72416d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:44:25.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.047+0000 7fb0faffd700 1 --2- 172.21.15.39:0/1894004713 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e9f708f6b0 0x55e9f72416d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:25.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.047+0000 7fb0faffd700 1 -- 172.21.15.39:0/1894004713 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0f4009e30 con 0x55e9f7235790 2023-07-21T18:44:25.076 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.047+0000 7fb0fb7fe700 1 --2- 172.21.15.39:0/1894004713 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e9f708f6b0 0x55e9f72416d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:44:25.076 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.047+0000 7fb0fbfff700 1 --2- 172.21.15.39:0/1894004713 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e9f723b550 0x55e9f723cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:44:25.076 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.048+0000 7fb0faffd700 1 --2- 172.21.15.39:0/1894004713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9f7235790 0x55e9f723c9d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb0f00cb8a0 tx=0x7fb0f009adf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=708b1dc738752c35 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:44:25.076 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.048+0000 7fb0daffd700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-21T18:44:25.076 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.048+0000 7fb0daffd700 1 -- 172.21.15.39:0/1894004713 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f0011110 con 0x55e9f7235790 2023-07-21T18:44:25.076 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.048+0000 7fb0daffd700 1 -- 172.21.15.39:0/1894004713 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fb0f009b2b0 con 0x55e9f7235790 2023-07-21T18:44:25.076 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.048+0000 7fb10dd63b80 1 -- 172.21.15.39:0/1894004713 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e9f70de490 con 0x55e9f7235790 2023-07-21T18:44:25.077 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.048+0000 7fb0daffd700 1 -- 172.21.15.39:0/1894004713 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0f009f0d0 con 0x55e9f7235790 2023-07-21T18:44:25.077 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.048+0000 7fb10dd63b80 10 client.25811 fetch_fsmap learned FSMap version 33 2023-07-21T18:44:25.077 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.048+0000 7fb10dd63b80 1 -- 172.21.15.39:0/1894004713 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7fb0ec017070 con 0x55e9f7235790 2023-07-21T18:44:25.077 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.048+0000 7fb0daffd700 1 -- 172.21.15.39:0/1894004713 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e9f70de490 con 0x55e9f7235790 2023-07-21T18:44:25.077 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.048+0000 7fb0daffd700 1 -- 172.21.15.39:0/1894004713 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fb0f009a7a0 con 0x55e9f7235790 2023-07-21T18:44:25.077 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.048+0000 7fb10dd63b80 10 client.25811 fetch_fsmap finished waiting for FSMap version 33 2023-07-21T18:44:25.077 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.048+0000 7fb10dd63b80 10 client.25811 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:44:25.077 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.048+0000 7fb10dd63b80 20 client.25811 populate_metadata read hostname 'smithi039' 2023-07-21T18:44:25.078 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.048+0000 7fb10dd63b80 1 --2- 172.21.15.39:0/1894004713 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55e9f7250bf0 0x55e9f7253000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:25.078 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.049+0000 7fb10dd63b80 4 client.25811 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:44:25.078 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.049+0000 7fb10dd63b80 1 -- 172.21.15.39:0/1894004713 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e9f717de50 con 0x55e9f7250bf0 2023-07-21T18:44:25.078 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.049+0000 7fb0fb7fe700 1 --2- 172.21.15.39:0/1894004713 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55e9f7250bf0 0x55e9f7253000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:25.078 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.049+0000 7fb0fb7fe700 1 --2- 172.21.15.39:0/1894004713 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55e9f7250bf0 0x55e9f7253000 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65d1b6003ec24caf server_cookie=80e4c60cd37bf4de in_seq=0 out_seq=0 2023-07-21T18:44:25.078 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.049+0000 7fb0daffd700 10 client.25811 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:44:25.079 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.050+0000 7fb0daffd700 1 -- 172.21.15.39:0/1894004713 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e9f717de50 con 0x55e9f7250bf0 2023-07-21T18:44:25.079 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.050+0000 7fb0daffd700 10 client.25811 handle_command_reply: tid=0 2023-07-21T18:44:25.079 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.050+0000 7fb10dd63b80 1 client.25811 shutdown 2023-07-21T18:44:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.050+0000 7fb10dd63b80 1 -- 172.21.15.39:0/1894004713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9f7235790 msgr2=0x55e9f723c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:44:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.050+0000 7fb10dd63b80 1 --2- 172.21.15.39:0/1894004713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9f7235790 0x55e9f723c9d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb0f00cb8a0 tx=0x7fb0f009adf0 comp rx=0 tx=0).stop 2023-07-21T18:44:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.050+0000 7fb10dd63b80 1 -- 172.21.15.39:0/1894004713 shutdown_connections 2023-07-21T18:44:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.050+0000 7fb10dd63b80 1 --2- 172.21.15.39:0/1894004713 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55e9f7250bf0 0x55e9f7253000 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.050+0000 7fb10dd63b80 1 --2- 172.21.15.39:0/1894004713 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e9f723b550 0x55e9f723cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.050+0000 7fb0daffd700 0 client.25811 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:44:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.050+0000 7fb10dd63b80 1 --2- 172.21.15.39:0/1894004713 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e9f708f6b0 0x55e9f72416d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:25.081 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.050+0000 7fb10dd63b80 1 --2- 172.21.15.39:0/1894004713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9f7235790 0x55e9f723c9d0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:25.081 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.050+0000 7fb10dd63b80 1 -- 172.21.15.39:0/1894004713 >> 172.21.15.39:0/1894004713 conn(0x55e9f7228be0 msgr2=0x55e9f722d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:44:25.081 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.050+0000 7fb10dd63b80 1 -- 172.21.15.39:0/1894004713 shutdown_connections 2023-07-21T18:44:25.081 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.050+0000 7fb10dd63b80 1 -- 172.21.15.39:0/1894004713 wait complete. 2023-07-21T18:44:25.081 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.050+0000 7fb10dd63b80 20 client.25811 trim_cache size 0 max 16384 2023-07-21T18:44:25.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.065+0000 7fb10dd63b80 1 Processor -- start 2023-07-21T18:44:25.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.065+0000 7fb10dd63b80 1 -- start start 2023-07-21T18:44:25.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.065+0000 7fb10dd63b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e9f708f6b0 0x55e9f724a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:25.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.065+0000 7fb10dd63b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e9f7235790 0x55e9f724d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:25.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.065+0000 7fb10dd63b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9f723b550 0x55e9f723a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:25.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.065+0000 7fb10dd63b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f71f9090 con 0x55e9f7235790 2023-07-21T18:44:25.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.065+0000 7fb10dd63b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f7225be0 con 0x55e9f723b550 2023-07-21T18:44:25.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.065+0000 7fb10dd63b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f723d2b0 con 0x55e9f708f6b0 2023-07-21T18:44:25.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.065+0000 7fb0faffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e9f7235790 0x55e9f724d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:25.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.065+0000 7fb0fbfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9f723b550 0x55e9f723a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:25.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.065+0000 7fb0faffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e9f7235790 0x55e9f724d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59306/0 (socket says 172.21.15.39:59306) 2023-07-21T18:44:25.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.065+0000 7fb0fb7fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e9f708f6b0 0x55e9f724a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:25.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.065+0000 7fb0faffd700 1 -- 172.21.15.39:0/2271761095 learned_addr learned my addr 172.21.15.39:0/2271761095 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:44:25.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.066+0000 7fb0fbfff700 1 -- 172.21.15.39:0/2271761095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e9f708f6b0 msgr2=0x55e9f724a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:44:25.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.066+0000 7fb0fbfff700 1 --2- 172.21.15.39:0/2271761095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e9f708f6b0 0x55e9f724a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:25.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.066+0000 7fb0fbfff700 1 -- 172.21.15.39:0/2271761095 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e9f7235790 msgr2=0x55e9f724d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:44:25.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.066+0000 7fb0fbfff700 1 --2- 172.21.15.39:0/2271761095 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e9f7235790 0x55e9f724d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:25.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.066+0000 7fb0fbfff700 1 -- 172.21.15.39:0/2271761095 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9f6f9bc00 con 0x55e9f723b550 2023-07-21T18:44:25.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.066+0000 7fb0faffd700 1 --2- 172.21.15.39:0/2271761095 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e9f7235790 0x55e9f724d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:44:25.088 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.066+0000 7fb0fb7fe700 1 --2- 172.21.15.39:0/2271761095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e9f708f6b0 0x55e9f724a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:44:25.088 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.066+0000 7fb0fbfff700 1 --2- 172.21.15.39:0/2271761095 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9f723b550 0x55e9f723a5e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc19b740 tx=0x7fb0f4004380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=268ccffcb322ff54 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:44:25.088 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.066+0000 7fb0e27fc700 1 -- 172.21.15.39:0/2271761095 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f4004db0 con 0x55e9f723b550 2023-07-21T18:44:25.088 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.066+0000 7fb0e27fc700 1 -- 172.21.15.39:0/2271761095 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0f400f040 con 0x55e9f723b550 2023-07-21T18:44:25.088 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.066+0000 7fb0e27fc700 1 -- 172.21.15.39:0/2271761095 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f401f980 con 0x55e9f723b550 2023-07-21T18:44:25.090 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.067+0000 7fb10dd63b80 1 -- 172.21.15.39:0/2271761095 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9f723b550 msgr2=0x55e9f723a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:44:25.091 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.067+0000 7fb10dd63b80 1 --2- 172.21.15.39:0/2271761095 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9f723b550 0x55e9f723a5e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc19b740 tx=0x7fb0f4004380 comp rx=0 tx=0).stop 2023-07-21T18:44:25.091 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.067+0000 7fb10dd63b80 1 -- 172.21.15.39:0/2271761095 shutdown_connections 2023-07-21T18:44:25.091 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.067+0000 7fb10dd63b80 1 --2- 172.21.15.39:0/2271761095 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9f723b550 0x55e9f723a5e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:25.091 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.067+0000 7fb10dd63b80 1 --2- 172.21.15.39:0/2271761095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e9f708f6b0 0x55e9f724a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:25.091 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.067+0000 7fb10dd63b80 1 --2- 172.21.15.39:0/2271761095 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e9f7235790 0x55e9f724d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:25.091 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.067+0000 7fb10dd63b80 1 -- 172.21.15.39:0/2271761095 >> 172.21.15.39:0/2271761095 conn(0x55e9f7228be0 msgr2=0x55e9f7236e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:44:25.091 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.067+0000 7fb10dd63b80 1 -- 172.21.15.39:0/2271761095 shutdown_connections 2023-07-21T18:44:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.067+0000 7fb10dd63b80 1 -- 172.21.15.39:0/2271761095 wait complete. 2023-07-21T18:44:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.067+0000 7fb10dd63b80 1 -- start start 2023-07-21T18:44:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.067+0000 7fb10dd63b80 1 Processor -- start 2023-07-21T18:44:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.068+0000 7fb10dd63b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e9f708f6b0 0x55e9f72488d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.068+0000 7fb10dd63b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9f7235790 0x55e9f7248e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.068+0000 7fb10dd63b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e9f723b550 0x55e9f7249350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.068+0000 7fb10dd63b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f72a4a80 con 0x55e9f723b550 2023-07-21T18:44:25.093 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.068+0000 7fb10dd63b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f72a4bc0 con 0x55e9f7235790 2023-07-21T18:44:25.093 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.068+0000 7fb10dd63b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f72a4d00 con 0x55e9f708f6b0 2023-07-21T18:44:25.093 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.068+0000 7fb10dd63b80 10 client.0 _ll_register_callbacks cb 0x7fb0f0097610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:44:25.093 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.068+0000 7fb0fbfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e9f723b550 0x55e9f7249350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:25.093 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.068+0000 7fb0fb7fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e9f708f6b0 0x55e9f72488d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:25.093 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.068+0000 7fb0faffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9f7235790 0x55e9f7248e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:25.093 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.068+0000 7fb0fb7fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e9f708f6b0 0x55e9f72488d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:49628/0 (socket says 172.21.15.39:49628) 2023-07-21T18:44:25.093 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.068+0000 7fb0fbfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e9f723b550 0x55e9f7249350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59312/0 (socket says 172.21.15.39:59312) 2023-07-21T18:44:25.094 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.068+0000 7fb0fb7fe700 1 -- 172.21.15.39:0/1200748253 learned_addr learned my addr 172.21.15.39:0/1200748253 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:44:25.094 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.068+0000 7fb0fb7fe700 1 -- 172.21.15.39:0/1200748253 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9f7235790 msgr2=0x55e9f7248e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:44:25.094 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.068+0000 7fb0fb7fe700 1 --2- 172.21.15.39:0/1200748253 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9f7235790 0x55e9f7248e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:25.094 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.068+0000 7fb0fb7fe700 1 -- 172.21.15.39:0/1200748253 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e9f723b550 msgr2=0x55e9f7249350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:44:25.094 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.068+0000 7fb0fb7fe700 1 --2- 172.21.15.39:0/1200748253 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e9f723b550 0x55e9f7249350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:25.094 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.068+0000 7fb0fb7fe700 1 -- 172.21.15.39:0/1200748253 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e9f6fb2d80 con 0x55e9f708f6b0 2023-07-21T18:44:25.094 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.068+0000 7fb0faffd700 1 --2- 172.21.15.39:0/1200748253 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9f7235790 0x55e9f7248e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:44:25.094 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.068+0000 7fb0fbfff700 1 --2- 172.21.15.39:0/1200748253 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e9f723b550 0x55e9f7249350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:44:25.095 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.068+0000 7fb0fb7fe700 1 --2- 172.21.15.39:0/1200748253 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e9f708f6b0 0x55e9f72488d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x55e9f72538d0 tx=0x7fb0ec009e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ce759b49dc9a7c0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:44:25.095 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.068+0000 7fb0daffd700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2023-07-21T18:44:25.095 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.069+0000 7fb0daffd700 1 -- 172.21.15.39:0/1200748253 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec02b040 con 0x55e9f708f6b0 2023-07-21T18:44:25.095 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.069+0000 7fb0daffd700 1 -- 172.21.15.39:0/1200748253 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fb0ec004a20 con 0x55e9f708f6b0 2023-07-21T18:44:25.095 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.069+0000 7fb10dd63b80 1 -- 172.21.15.39:0/1200748253 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb0ec004e50 con 0x55e9f708f6b0 2023-07-21T18:44:25.095 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.069+0000 7fb0daffd700 1 -- 172.21.15.39:0/1200748253 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0ec004be0 con 0x55e9f708f6b0 2023-07-21T18:44:25.095 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.069+0000 7fb10dd63b80 10 client.25822 fetch_fsmap learned FSMap version 33 2023-07-21T18:44:25.095 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.069+0000 7fb10dd63b80 1 -- 172.21.15.39:0/1200748253 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7fb0ec017070 con 0x55e9f708f6b0 2023-07-21T18:44:25.096 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.069+0000 7fb0daffd700 1 -- 172.21.15.39:0/1200748253 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0ec004e50 con 0x55e9f708f6b0 2023-07-21T18:44:25.096 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.069+0000 7fb0daffd700 1 -- 172.21.15.39:0/1200748253 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fb0ec024030 con 0x55e9f708f6b0 2023-07-21T18:44:25.096 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.069+0000 7fb10dd63b80 10 client.25822 fetch_fsmap finished waiting for FSMap version 33 2023-07-21T18:44:25.096 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.069+0000 7fb10dd63b80 10 client.25822 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:44:25.096 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.069+0000 7fb10dd63b80 20 client.25822 populate_metadata read hostname 'smithi039' 2023-07-21T18:44:25.096 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.069+0000 7fb10dd63b80 1 --2- 172.21.15.39:0/1200748253 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55e9f7250bf0 0x55e9f72a8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:25.096 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.069+0000 7fb10dd63b80 4 client.25822 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:44:25.097 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.069+0000 7fb10dd63b80 1 -- 172.21.15.39:0/1200748253 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e9f713fff0 con 0x55e9f7250bf0 2023-07-21T18:44:25.097 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.069+0000 7fb0faffd700 1 --2- 172.21.15.39:0/1200748253 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55e9f7250bf0 0x55e9f72a8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:25.097 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.070+0000 7fb0daffd700 10 client.25822 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:44:25.097 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.070+0000 7fb0faffd700 1 --2- 172.21.15.39:0/1200748253 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55e9f7250bf0 0x55e9f72a8430 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c29893e2d3caff11 server_cookie=6dd32d3a4f31bb4f in_seq=0 out_seq=0 2023-07-21T18:44:25.137 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.117+0000 7fb0daffd700 1 -- 172.21.15.39:0/1200748253 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e9f713fff0 con 0x55e9f7250bf0 2023-07-21T18:44:25.137 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.117+0000 7fb0daffd700 10 client.25822 handle_command_reply: tid=0 2023-07-21T18:44:25.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.117+0000 7fb10dd63b80 1 client.25822 shutdown 2023-07-21T18:44:25.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.118+0000 7fb10dd63b80 1 -- 172.21.15.39:0/1200748253 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e9f708f6b0 msgr2=0x55e9f72488d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:44:25.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.118+0000 7fb10dd63b80 1 --2- 172.21.15.39:0/1200748253 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e9f708f6b0 0x55e9f72488d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x55e9f72538d0 tx=0x7fb0ec009e10 comp rx=0 tx=0).stop 2023-07-21T18:44:25.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.118+0000 7fb10dd63b80 1 -- 172.21.15.39:0/1200748253 shutdown_connections 2023-07-21T18:44:25.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.118+0000 7fb10dd63b80 1 --2- 172.21.15.39:0/1200748253 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55e9f7250bf0 0x55e9f72a8430 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:25.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.118+0000 7fb10dd63b80 1 --2- 172.21.15.39:0/1200748253 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e9f723b550 0x55e9f7249350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:25.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.118+0000 7fb10dd63b80 1 --2- 172.21.15.39:0/1200748253 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e9f708f6b0 0x55e9f72488d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:25.139 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.118+0000 7fb0daffd700 0 client.25822 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:44:25.139 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.118+0000 7fb10dd63b80 1 --2- 172.21.15.39:0/1200748253 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9f7235790 0x55e9f7248e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:25.139 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.118+0000 7fb10dd63b80 1 -- 172.21.15.39:0/1200748253 >> 172.21.15.39:0/1200748253 conn(0x55e9f7228be0 msgr2=0x55e9f724c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:44:25.139 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.118+0000 7fb10dd63b80 1 -- 172.21.15.39:0/1200748253 shutdown_connections 2023-07-21T18:44:25.139 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.118+0000 7fb10dd63b80 1 -- 172.21.15.39:0/1200748253 wait complete. 2023-07-21T18:44:25.139 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.118+0000 7fb10dd63b80 20 client.25822 trim_cache size 0 max 16384 2023-07-21T18:44:25.140 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:44:25.140 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:44:25.140 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:44:25.140 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:44:25.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.119+0000 7fb101bdd700 1 -- 172.21.15.39:0/504678555 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fb0e40a12c0 msgr2=0x7fb0e40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:44:25.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.119+0000 7fb101bdd700 1 --2- 172.21.15.39:0/504678555 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fb0e40a12c0 0x7fb0e40a3770 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec005950 tx=0x7fb0ec009260 comp rx=0 tx=0).stop 2023-07-21T18:44:25.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.119+0000 7fb101bdd700 1 -- 172.21.15.39:0/504678555 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0fc0ff3e0 msgr2=0x7fb0fc19af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:44:25.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.119+0000 7fb101bdd700 1 --2- 172.21.15.39:0/504678555 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0fc0ff3e0 0x7fb0fc19af80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb0f00059d0 tx=0x7fb0f0005740 comp rx=0 tx=0).stop 2023-07-21T18:44:25.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.119+0000 7fb101bdd700 1 -- 172.21.15.39:0/504678555 shutdown_connections 2023-07-21T18:44:25.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.119+0000 7fb101bdd700 1 --2- 172.21.15.39:0/504678555 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fb0e40a12c0 0x7fb0e40a3770 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:25.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.119+0000 7fb101bdd700 1 --2- 172.21.15.39:0/504678555 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0fc0fea30 0x7fb0fc19aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:25.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.119+0000 7fb101bdd700 1 --2- 172.21.15.39:0/504678555 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0fc104a30 0x7fb0fc1948a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:25.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.119+0000 7fb101bdd700 1 --2- 172.21.15.39:0/504678555 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0fc0ff3e0 0x7fb0fc19af80 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:25.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.119+0000 7fb101bdd700 1 -- 172.21.15.39:0/504678555 >> 172.21.15.39:0/504678555 conn(0x7fb0fc074760 msgr2=0x7fb0fc0fd030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:44:25.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.119+0000 7fb101bdd700 1 -- 172.21.15.39:0/504678555 shutdown_connections 2023-07-21T18:44:25.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:25.119+0000 7fb101bdd700 1 -- 172.21.15.39:0/504678555 wait complete. 2023-07-21T18:44:25.263 INFO:tasks.workunit.client.0.smithi039.stdout: 1 111426 6.31 MB/sec warmup 98 sec latency 64.293 ms 2023-07-21T18:44:26.263 INFO:tasks.workunit.client.0.smithi039.stdout: 1 113789 6.39 MB/sec warmup 99 sec latency 14.139 ms 2023-07-21T18:44:26.366 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:44:26 smithi193 ceph-mon[98204]: pgmap v83: 97 pgs: 97 active+clean; 60 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 249 op/s 2023-07-21T18:44:26.384 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:44:26 smithi155 ceph-mon[97934]: pgmap v83: 97 pgs: 97 active+clean; 60 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 249 op/s 2023-07-21T18:44:26.433 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:44:26 smithi039 ceph-mon[96489]: pgmap v83: 97 pgs: 97 active+clean; 60 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 249 op/s 2023-07-21T18:44:27.263 INFO:tasks.workunit.client.0.smithi039.stdout: 1 115903 6.40 MB/sec warmup 100 sec latency 10.227 ms 2023-07-21T18:44:28.263 INFO:tasks.workunit.client.0.smithi039.stdout: 1 117518 6.46 MB/sec warmup 101 sec latency 11.303 ms 2023-07-21T18:44:28.366 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:44:28 smithi193 ceph-mon[98204]: pgmap v84: 97 pgs: 97 active+clean; 67 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 396 op/s 2023-07-21T18:44:28.384 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:44:28 smithi155 ceph-mon[97934]: pgmap v84: 97 pgs: 97 active+clean; 67 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 396 op/s 2023-07-21T18:44:28.433 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:44:28 smithi039 ceph-mon[96489]: pgmap v84: 97 pgs: 97 active+clean; 67 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 396 op/s 2023-07-21T18:44:29.263 INFO:tasks.workunit.client.0.smithi039.stdout: 1 119661 6.47 MB/sec warmup 102 sec latency 11.766 ms 2023-07-21T18:44:30.263 INFO:tasks.workunit.client.0.smithi039.stdout: 1 121177 6.52 MB/sec warmup 103 sec latency 34.600 ms 2023-07-21T18:44:30.366 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:44:30 smithi193 ceph-mon[98204]: pgmap v85: 97 pgs: 97 active+clean; 67 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 261 op/s 2023-07-21T18:44:30.384 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:44:30 smithi155 ceph-mon[97934]: pgmap v85: 97 pgs: 97 active+clean; 67 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 261 op/s 2023-07-21T18:44:30.433 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:44:30 smithi039 ceph-mon[96489]: pgmap v85: 97 pgs: 97 active+clean; 67 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 261 op/s 2023-07-21T18:44:31.263 INFO:tasks.workunit.client.0.smithi039.stdout: 1 123215 6.53 MB/sec warmup 104 sec latency 68.404 ms 2023-07-21T18:44:32.263 INFO:tasks.workunit.client.0.smithi039.stdout: 1 124634 6.57 MB/sec warmup 105 sec latency 68.462 ms 2023-07-21T18:44:32.365 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:44:32 smithi193 ceph-mon[98204]: pgmap v86: 97 pgs: 97 active+clean; 72 MiB data, 801 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 388 op/s 2023-07-21T18:44:32.384 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:44:32 smithi155 ceph-mon[97934]: pgmap v86: 97 pgs: 97 active+clean; 72 MiB data, 801 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 388 op/s 2023-07-21T18:44:32.433 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:44:32 smithi039 ceph-mon[96489]: pgmap v86: 97 pgs: 97 active+clean; 72 MiB data, 801 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 388 op/s 2023-07-21T18:44:33.263 INFO:tasks.workunit.client.0.smithi039.stdout: 1 126945 6.62 MB/sec warmup 106 sec latency 10.819 ms 2023-07-21T18:44:34.263 INFO:tasks.workunit.client.0.smithi039.stdout: 1 128312 6.63 MB/sec warmup 107 sec latency 65.206 ms 2023-07-21T18:44:34.366 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:44:34 smithi193 ceph-mon[98204]: pgmap v87: 97 pgs: 97 active+clean; 72 MiB data, 801 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.6 MiB/s wr, 274 op/s 2023-07-21T18:44:34.384 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:44:34 smithi155 ceph-mon[97934]: pgmap v87: 97 pgs: 97 active+clean; 72 MiB data, 801 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.6 MiB/s wr, 274 op/s 2023-07-21T18:44:34.433 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:44:34 smithi039 ceph-mon[96489]: pgmap v87: 97 pgs: 97 active+clean; 72 MiB data, 801 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.6 MiB/s wr, 274 op/s 2023-07-21T18:44:35.263 INFO:tasks.workunit.client.0.smithi039.stdout: 1 130622 6.68 MB/sec warmup 108 sec latency 27.227 ms 2023-07-21T18:44:36.263 INFO:tasks.workunit.client.0.smithi039.stdout: 1 131987 6.69 MB/sec warmup 109 sec latency 24.386 ms 2023-07-21T18:44:36.365 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:44:36 smithi193 ceph-mon[98204]: pgmap v88: 97 pgs: 97 active+clean; 72 MiB data, 801 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.6 MiB/s wr, 274 op/s 2023-07-21T18:44:36.384 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:44:36 smithi155 ceph-mon[97934]: pgmap v88: 97 pgs: 97 active+clean; 72 MiB data, 801 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.6 MiB/s wr, 274 op/s 2023-07-21T18:44:36.433 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:44:36 smithi039 ceph-mon[96489]: pgmap v88: 97 pgs: 97 active+clean; 72 MiB data, 801 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.6 MiB/s wr, 274 op/s 2023-07-21T18:44:37.263 INFO:tasks.workunit.client.0.smithi039.stdout: 1 134612 6.75 MB/sec warmup 110 sec latency 10.233 ms 2023-07-21T18:44:38.252 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.234+0000 7f136c8fc700 1 -- 172.21.15.39:0/382029895 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1368104e50 msgr2=0x7f136810ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:44:38.253 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.234+0000 7f136c8fc700 1 --2- 172.21.15.39:0/382029895 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1368104e50 0x7f136810ca00 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f136000b0d0 tx=0x7f136000b3e0 comp rx=0 tx=0).stop 2023-07-21T18:44:38.253 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.234+0000 7f136c8fc700 1 -- 172.21.15.39:0/382029895 shutdown_connections 2023-07-21T18:44:38.253 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.234+0000 7f136c8fc700 1 --2- 172.21.15.39:0/382029895 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1368104e50 0x7f136810ca00 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:38.253 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.234+0000 7f136c8fc700 1 --2- 172.21.15.39:0/382029895 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1368067c10 0x7f1368067fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:38.253 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.234+0000 7f136c8fc700 1 --2- 172.21.15.39:0/382029895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13680685b0 0x7f1368104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:38.254 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.234+0000 7f136c8fc700 1 -- 172.21.15.39:0/382029895 >> 172.21.15.39:0/382029895 conn(0x7f1368074760 msgr2=0x7f1368074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:44:38.254 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.234+0000 7f136c8fc700 1 -- 172.21.15.39:0/382029895 shutdown_connections 2023-07-21T18:44:38.254 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.235+0000 7f136c8fc700 1 -- 172.21.15.39:0/382029895 wait complete. 2023-07-21T18:44:38.255 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.235+0000 7f136c8fc700 1 Processor -- start 2023-07-21T18:44:38.256 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.235+0000 7f136c8fc700 1 -- start start 2023-07-21T18:44:38.256 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.235+0000 7f136c8fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1368067c10 0x7f1368197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:38.256 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.235+0000 7f136c8fc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13680685b0 0x7f13681984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:38.257 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.235+0000 7f136c8fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1368104e50 0x7f136819cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:38.257 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.235+0000 7f136c8fc700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f136819d190 con 0x7f1368104e50 2023-07-21T18:44:38.257 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.236+0000 7f136c8fc700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f136819d300 con 0x7f1368067c10 2023-07-21T18:44:38.257 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.236+0000 7f136c8fc700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f136819d4c0 con 0x7f13680685b0 2023-07-21T18:44:38.258 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.236+0000 7f1366d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1368104e50 0x7f136819cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:38.258 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.236+0000 7f1365d9b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13680685b0 0x7f13681984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:38.258 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.236+0000 7f1366d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1368104e50 0x7f136819cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59326/0 (socket says 172.21.15.39:59326) 2023-07-21T18:44:38.258 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.236+0000 7f1366d9d700 1 -- 172.21.15.39:0/2456730729 learned_addr learned my addr 172.21.15.39:0/2456730729 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:44:38.259 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.236+0000 7f136659c700 1 --2- 172.21.15.39:0/2456730729 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1368067c10 0x7f1368197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:38.259 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.236+0000 7f1366d9d700 1 -- 172.21.15.39:0/2456730729 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13680685b0 msgr2=0x7f13681984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:44:38.259 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.236+0000 7f1366d9d700 1 --2- 172.21.15.39:0/2456730729 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13680685b0 0x7f13681984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:38.260 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.236+0000 7f1366d9d700 1 -- 172.21.15.39:0/2456730729 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1368067c10 msgr2=0x7f1368197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:44:38.260 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.236+0000 7f1366d9d700 1 --2- 172.21.15.39:0/2456730729 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1368067c10 0x7f1368197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:38.260 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.236+0000 7f1366d9d700 1 -- 172.21.15.39:0/2456730729 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1360009d00 con 0x7f1368104e50 2023-07-21T18:44:38.260 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.236+0000 7f1365d9b700 1 --2- 172.21.15.39:0/2456730729 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13680685b0 0x7f13681984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:44:38.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.236+0000 7f1366d9d700 1 --2- 172.21.15.39:0/2456730729 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1368104e50 0x7f136819cb50 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f1360007820 tx=0x7f1360007850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:44:38.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.236+0000 7f13577fe700 1 -- 172.21.15.39:0/2456730729 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136000e040 con 0x7f1368104e50 2023-07-21T18:44:38.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.236+0000 7f13577fe700 1 -- 172.21.15.39:0/2456730729 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f136000bcb0 con 0x7f1368104e50 2023-07-21T18:44:38.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.236+0000 7f13577fe700 1 -- 172.21.15.39:0/2456730729 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13600058a0 con 0x7f1368104e50 2023-07-21T18:44:38.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.236+0000 7f136c8fc700 1 -- 172.21.15.39:0/2456730729 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f136819d740 con 0x7f1368104e50 2023-07-21T18:44:38.262 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.237+0000 7f136c8fc700 1 -- 172.21.15.39:0/2456730729 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13681a4fe0 con 0x7f1368104e50 2023-07-21T18:44:38.265 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.239+0000 7f13577fe700 1 -- 172.21.15.39:0/2456730729 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f1360004a50 con 0x7f1368104e50 2023-07-21T18:44:38.265 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.240+0000 7f13577fe700 1 --2- 172.21.15.39:0/2456730729 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f13500a1340 0x7f13500a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:38.265 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.240+0000 7f136659c700 1 --2- 172.21.15.39:0/2456730729 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f13500a1340 0x7f13500a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:38.266 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.240+0000 7f13577fe700 1 -- 172.21.15.39:0/2456730729 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f13600cda20 con 0x7f1368104e50 2023-07-21T18:44:38.266 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.240+0000 7f136659c700 1 --2- 172.21.15.39:0/2456730729 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f13500a1340 0x7f13500a37f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1358009c60 tx=0x7f1358009260 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:44:38.266 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.241+0000 7f1378a82b80 1 Processor -- start 2023-07-21T18:44:38.266 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.241+0000 7f1378a82b80 1 -- start start 2023-07-21T18:44:38.266 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.241+0000 7f1378a82b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b0887a6b0 0x562b08a23d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:38.266 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.241+0000 7f1378a82b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b08a242d0 0x562b08a1e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:38.266 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.241+0000 7f1378a82b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b08a1e580 0x562b08a1e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:38.267 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.241+0000 7f1378a82b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b08923c50 con 0x562b08a242d0 2023-07-21T18:44:38.267 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.241+0000 7f1378a82b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b089769e0 con 0x562b08a1e580 2023-07-21T18:44:38.267 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.241+0000 7f1378a82b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b0896fe70 con 0x562b0887a6b0 2023-07-21T18:44:38.267 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.241+0000 7f136659c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b0887a6b0 0x562b08a23d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:38.267 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.241+0000 7f1365d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b08a242d0 0x562b08a1e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:38.267 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.241+0000 7f136659c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b0887a6b0 0x562b08a23d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:49650/0 (socket says 172.21.15.39:49650) 2023-07-21T18:44:38.267 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.241+0000 7f1365d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b08a242d0 0x562b08a1e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59332/0 (socket says 172.21.15.39:59332) 2023-07-21T18:44:38.267 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.241+0000 7f136659c700 1 -- 172.21.15.39:0/748832494 learned_addr learned my addr 172.21.15.39:0/748832494 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:44:38.268 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.241+0000 7f1366d9d700 1 --2- 172.21.15.39:0/748832494 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b08a1e580 0x562b08a1e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:38.268 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.241+0000 7f136659c700 1 -- 172.21.15.39:0/748832494 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b08a1e580 msgr2=0x562b08a1e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:44:38.268 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.241+0000 7f136659c700 1 --2- 172.21.15.39:0/748832494 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b08a1e580 0x562b08a1e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:38.268 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.241+0000 7f136659c700 1 -- 172.21.15.39:0/748832494 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b08a242d0 msgr2=0x562b08a1e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:44:38.268 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.241+0000 7f136659c700 1 --2- 172.21.15.39:0/748832494 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b08a242d0 0x562b08a1e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:38.268 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.242+0000 7f136659c700 1 -- 172.21.15.39:0/748832494 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b0879dd80 con 0x562b0887a6b0 2023-07-21T18:44:38.268 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.242+0000 7f136659c700 1 --2- 172.21.15.39:0/748832494 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b0887a6b0 0x562b08a23d90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f135800e4e0 tx=0x7f1358014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8eda31aaf27f3a81 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:44:38.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.242+0000 7f1354ff9700 1 -- 172.21.15.39:0/748832494 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135800ee00 con 0x562b0887a6b0 2023-07-21T18:44:38.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.242+0000 7f1354ff9700 1 -- 172.21.15.39:0/748832494 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1358021070 con 0x562b0887a6b0 2023-07-21T18:44:38.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.242+0000 7f1354ff9700 1 -- 172.21.15.39:0/748832494 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135801d700 con 0x562b0887a6b0 2023-07-21T18:44:38.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.243+0000 7f1378a82b80 1 -- 172.21.15.39:0/748832494 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b0887a6b0 msgr2=0x562b08a23d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:44:38.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.243+0000 7f1378a82b80 1 --2- 172.21.15.39:0/748832494 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b0887a6b0 0x562b08a23d90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f135800e4e0 tx=0x7f1358014000 comp rx=0 tx=0).stop 2023-07-21T18:44:38.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.243+0000 7f1378a82b80 1 -- 172.21.15.39:0/748832494 shutdown_connections 2023-07-21T18:44:38.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.243+0000 7f1378a82b80 1 --2- 172.21.15.39:0/748832494 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b08a1e580 0x562b08a1e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:38.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.243+0000 7f1378a82b80 1 --2- 172.21.15.39:0/748832494 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b0887a6b0 0x562b08a23d90 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:38.270 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.243+0000 7f1378a82b80 1 --2- 172.21.15.39:0/748832494 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b08a242d0 0x562b08a1e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:38.270 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.243+0000 7f1378a82b80 1 -- 172.21.15.39:0/748832494 >> 172.21.15.39:0/748832494 conn(0x562b08a13be0 msgr2=0x562b08a18f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:44:38.270 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.243+0000 7f1378a82b80 1 -- 172.21.15.39:0/748832494 shutdown_connections 2023-07-21T18:44:38.270 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.243+0000 7f1378a82b80 1 -- 172.21.15.39:0/748832494 wait complete. 2023-07-21T18:44:38.270 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.243+0000 7f1378a82b80 1 -- start start 2023-07-21T18:44:38.270 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.243+0000 7f1378a82b80 1 Processor -- start 2023-07-21T18:44:38.270 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.243+0000 7f1378a82b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b0887a6b0 0x562b08a2a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:38.270 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.243+0000 7f1378a82b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b08a1e580 0x562b08a278f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:38.271 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.244+0000 7f1378a82b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b08a242d0 0x562b08a28030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:38.271 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.244+0000 7f1378a82b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b0890fa60 con 0x562b08a1e580 2023-07-21T18:44:38.271 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.244+0000 7f1378a82b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b0890f760 con 0x562b08a242d0 2023-07-21T18:44:38.271 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.244+0000 7f1378a82b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b087f38c0 con 0x562b0887a6b0 2023-07-21T18:44:38.271 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.244+0000 7f1378a82b80 10 client.0 _ll_register_callbacks cb 0x562b0890d700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:44:38.271 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.244+0000 7f136659c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b0887a6b0 0x562b08a2a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:38.271 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.244+0000 7f136659c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b0887a6b0 0x562b08a2a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:49656/0 (socket says 172.21.15.39:49656) 2023-07-21T18:44:38.271 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.244+0000 7f136659c700 1 -- 172.21.15.39:0/1213114499 learned_addr learned my addr 172.21.15.39:0/1213114499 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:44:38.272 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.244+0000 7f1365d9b700 1 --2- 172.21.15.39:0/1213114499 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b08a1e580 0x562b08a278f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:38.272 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.244+0000 7f1366d9d700 1 --2- 172.21.15.39:0/1213114499 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b08a242d0 0x562b08a28030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:38.272 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.244+0000 7f136659c700 1 -- 172.21.15.39:0/1213114499 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b08a242d0 msgr2=0x562b08a28030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:44:38.272 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.244+0000 7f136659c700 1 --2- 172.21.15.39:0/1213114499 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b08a242d0 0x562b08a28030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:38.272 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.244+0000 7f136659c700 1 -- 172.21.15.39:0/1213114499 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b08a1e580 msgr2=0x562b08a278f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:44:38.272 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.244+0000 7f136659c700 1 --2- 172.21.15.39:0/1213114499 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b08a1e580 0x562b08a278f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:38.272 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.244+0000 7f136659c700 1 -- 172.21.15.39:0/1213114499 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1358017070 con 0x562b0887a6b0 2023-07-21T18:44:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.244+0000 7f1366d9d700 1 --2- 172.21.15.39:0/1213114499 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b08a242d0 0x562b08a28030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:44:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.244+0000 7f136659c700 1 --2- 172.21.15.39:0/1213114499 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b0887a6b0 0x562b08a2a670 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f13580097c0 tx=0x7f135800ec00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c823ca912b041eb2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:44:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.244+0000 7f134dffb700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2023-07-21T18:44:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.244+0000 7f134dffb700 1 -- 172.21.15.39:0/1213114499 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1358005250 con 0x562b0887a6b0 2023-07-21T18:44:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.244+0000 7f1378a82b80 1 -- 172.21.15.39:0/1213114499 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562b088c9490 con 0x562b0887a6b0 2023-07-21T18:44:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.244+0000 7f134dffb700 1 -- 172.21.15.39:0/1213114499 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f13580053b0 con 0x562b0887a6b0 2023-07-21T18:44:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.245+0000 7f134dffb700 1 -- 172.21.15.39:0/1213114499 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1358005570 con 0x562b0887a6b0 2023-07-21T18:44:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.245+0000 7f1378a82b80 10 client.25834 fetch_fsmap learned FSMap version 33 2023-07-21T18:44:38.274 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.245+0000 7f1378a82b80 1 -- 172.21.15.39:0/1213114499 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x562b08786c00 con 0x562b0887a6b0 2023-07-21T18:44:38.274 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.245+0000 7f134dffb700 1 -- 172.21.15.39:0/1213114499 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f135803b020 con 0x562b0887a6b0 2023-07-21T18:44:38.274 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.245+0000 7f134dffb700 1 -- 172.21.15.39:0/1213114499 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f135801c5a0 con 0x562b0887a6b0 2023-07-21T18:44:38.274 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.245+0000 7f1378a82b80 10 client.25834 fetch_fsmap finished waiting for FSMap version 33 2023-07-21T18:44:38.274 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.245+0000 7f1378a82b80 10 client.25834 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:44:38.274 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.245+0000 7f1378a82b80 20 client.25834 populate_metadata read hostname 'smithi039' 2023-07-21T18:44:38.274 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.245+0000 7f1378a82b80 1 --2- 172.21.15.39:0/1213114499 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x562b08a37c50 0x562b08a3a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:38.275 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.245+0000 7f1378a82b80 4 client.25834 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:44:38.275 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.245+0000 7f1378a82b80 1 -- 172.21.15.39:0/1213114499 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562b08968e50 con 0x562b08a37c50 2023-07-21T18:44:38.275 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.245+0000 7f1365d9b700 1 --2- 172.21.15.39:0/1213114499 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x562b08a37c50 0x562b08a3a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:38.275 INFO:tasks.workunit.client.0.smithi039.stdout: 1 135944 6.75 MB/sec warmup 111 sec latency 10.661 ms 2023-07-21T18:44:38.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.246+0000 7f134dffb700 10 client.25834 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:44:38.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.246+0000 7f1365d9b700 1 --2- 172.21.15.39:0/1213114499 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x562b08a37c50 0x562b08a3a060 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b978662839170ca5 server_cookie=c3fd4ed5c69363bd in_seq=0 out_seq=0 2023-07-21T18:44:38.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.246+0000 7f134dffb700 1 -- 172.21.15.39:0/1213114499 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562b08968e50 con 0x562b08a37c50 2023-07-21T18:44:38.277 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.246+0000 7f134dffb700 10 client.25834 handle_command_reply: tid=0 2023-07-21T18:44:38.277 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.246+0000 7f1378a82b80 1 client.25834 shutdown 2023-07-21T18:44:38.277 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.246+0000 7f1378a82b80 1 -- 172.21.15.39:0/1213114499 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b0887a6b0 msgr2=0x562b08a2a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:44:38.277 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.246+0000 7f1378a82b80 1 --2- 172.21.15.39:0/1213114499 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b0887a6b0 0x562b08a2a670 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f13580097c0 tx=0x7f135800ec00 comp rx=0 tx=0).stop 2023-07-21T18:44:38.277 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.246+0000 7f1378a82b80 1 -- 172.21.15.39:0/1213114499 shutdown_connections 2023-07-21T18:44:38.277 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.246+0000 7f1378a82b80 1 --2- 172.21.15.39:0/1213114499 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x562b08a37c50 0x562b08a3a060 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:38.277 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.246+0000 7f1378a82b80 1 --2- 172.21.15.39:0/1213114499 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b08a242d0 0x562b08a28030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:38.278 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.246+0000 7f134dffb700 0 client.25834 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:44:38.278 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.246+0000 7f1378a82b80 1 --2- 172.21.15.39:0/1213114499 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b0887a6b0 0x562b08a2a670 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:38.278 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.246+0000 7f1378a82b80 1 --2- 172.21.15.39:0/1213114499 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b08a1e580 0x562b08a278f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:38.278 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.246+0000 7f1378a82b80 1 -- 172.21.15.39:0/1213114499 >> 172.21.15.39:0/1213114499 conn(0x562b08a13be0 msgr2=0x562b08a184e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:44:38.278 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.247+0000 7f1378a82b80 1 -- 172.21.15.39:0/1213114499 shutdown_connections 2023-07-21T18:44:38.278 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.247+0000 7f1378a82b80 1 -- 172.21.15.39:0/1213114499 wait complete. 2023-07-21T18:44:38.278 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.247+0000 7f1378a82b80 20 client.25834 trim_cache size 0 max 16384 2023-07-21T18:44:38.280 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.262+0000 7f1378a82b80 1 Processor -- start 2023-07-21T18:44:38.281 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.262+0000 7f1378a82b80 1 -- start start 2023-07-21T18:44:38.281 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.262+0000 7f1378a82b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b0887a6b0 0x562b08a231c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:38.281 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.262+0000 7f1378a82b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b08a1e580 0x562b08a2a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:38.281 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.262+0000 7f1378a82b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b08a242d0 0x562b08a18c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:38.281 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.262+0000 7f1378a82b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b08a239c0 con 0x562b0887a6b0 2023-07-21T18:44:38.281 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.262+0000 7f1378a82b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b08a1a930 con 0x562b08a1e580 2023-07-21T18:44:38.281 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.262+0000 7f1378a82b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b08a226a0 con 0x562b08a242d0 2023-07-21T18:44:38.282 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.262+0000 7f136659c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b0887a6b0 0x562b08a231c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:38.282 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.262+0000 7f136659c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b0887a6b0 0x562b08a231c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59344/0 (socket says 172.21.15.39:59344) 2023-07-21T18:44:38.282 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.262+0000 7f136659c700 1 -- 172.21.15.39:0/2787190695 learned_addr learned my addr 172.21.15.39:0/2787190695 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:44:38.282 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.262+0000 7f1365d9b700 1 --2- 172.21.15.39:0/2787190695 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b08a1e580 0x562b08a2a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:38.282 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.262+0000 7f1366d9d700 1 --2- 172.21.15.39:0/2787190695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b08a242d0 0x562b08a18c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:38.282 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.263+0000 7f136659c700 1 -- 172.21.15.39:0/2787190695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b08a242d0 msgr2=0x562b08a18c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:44:38.282 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.263+0000 7f136659c700 1 --2- 172.21.15.39:0/2787190695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b08a242d0 0x562b08a18c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:38.282 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.263+0000 7f136659c700 1 -- 172.21.15.39:0/2787190695 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b08a1e580 msgr2=0x562b08a2a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:44:38.283 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.263+0000 7f136659c700 1 --2- 172.21.15.39:0/2787190695 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b08a1e580 0x562b08a2a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:38.283 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.263+0000 7f136659c700 1 -- 172.21.15.39:0/2787190695 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b08a1ec20 con 0x562b0887a6b0 2023-07-21T18:44:38.283 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.263+0000 7f136659c700 1 --2- 172.21.15.39:0/2787190695 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b0887a6b0 0x562b08a231c0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f13580264a0 tx=0x7f135800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29d08d434e92cc8e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:44:38.285 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.263+0000 7f13557fa700 1 -- 172.21.15.39:0/2787190695 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135800ec70 con 0x562b0887a6b0 2023-07-21T18:44:38.286 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.263+0000 7f13557fa700 1 -- 172.21.15.39:0/2787190695 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f135803b070 con 0x562b0887a6b0 2023-07-21T18:44:38.286 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.263+0000 7f13557fa700 1 -- 172.21.15.39:0/2787190695 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1358003900 con 0x562b0887a6b0 2023-07-21T18:44:38.286 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.264+0000 7f1378a82b80 1 -- 172.21.15.39:0/2787190695 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b0887a6b0 msgr2=0x562b08a231c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:44:38.286 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.264+0000 7f1378a82b80 1 --2- 172.21.15.39:0/2787190695 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b0887a6b0 0x562b08a231c0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f13580264a0 tx=0x7f135800ef60 comp rx=0 tx=0).stop 2023-07-21T18:44:38.286 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.264+0000 7f1378a82b80 1 -- 172.21.15.39:0/2787190695 shutdown_connections 2023-07-21T18:44:38.286 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.264+0000 7f1378a82b80 1 --2- 172.21.15.39:0/2787190695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b08a242d0 0x562b08a18c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:38.286 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.264+0000 7f1378a82b80 1 --2- 172.21.15.39:0/2787190695 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b0887a6b0 0x562b08a231c0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:38.287 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.264+0000 7f1378a82b80 1 --2- 172.21.15.39:0/2787190695 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b08a1e580 0x562b08a2a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:38.287 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.264+0000 7f1378a82b80 1 -- 172.21.15.39:0/2787190695 >> 172.21.15.39:0/2787190695 conn(0x562b08a13be0 msgr2=0x562b08a22ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:44:38.287 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.264+0000 7f1378a82b80 1 -- 172.21.15.39:0/2787190695 shutdown_connections 2023-07-21T18:44:38.287 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.264+0000 7f1378a82b80 1 -- 172.21.15.39:0/2787190695 wait complete. 2023-07-21T18:44:38.287 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.264+0000 7f1378a82b80 1 -- start start 2023-07-21T18:44:38.287 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.264+0000 7f1378a82b80 1 Processor -- start 2023-07-21T18:44:38.287 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.264+0000 7f1378a82b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b0887a6b0 0x562b08a2cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:38.288 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.264+0000 7f1378a82b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b08a1e580 0x562b08a2d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:38.288 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.264+0000 7f1378a82b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b08a242d0 0x562b08a2d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:38.288 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.264+0000 7f1378a82b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b08a8abb0 con 0x562b08a1e580 2023-07-21T18:44:38.288 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.264+0000 7f1378a82b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b08a8ad20 con 0x562b08a242d0 2023-07-21T18:44:38.288 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.264+0000 7f1378a82b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b08a8aee0 con 0x562b0887a6b0 2023-07-21T18:44:38.288 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.264+0000 7f1378a82b80 10 client.0 _ll_register_callbacks cb 0x7f135c009be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:44:38.288 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.264+0000 7f136659c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b0887a6b0 0x562b08a2cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:38.288 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.264+0000 7f136659c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b0887a6b0 0x562b08a2cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:49670/0 (socket says 172.21.15.39:49670) 2023-07-21T18:44:38.289 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.264+0000 7f1365d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b08a1e580 0x562b08a2d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:38.289 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.264+0000 7f136659c700 1 -- 172.21.15.39:0/3346682257 learned_addr learned my addr 172.21.15.39:0/3346682257 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:44:38.289 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.265+0000 7f1366d9d700 1 --2- 172.21.15.39:0/3346682257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b08a242d0 0x562b08a2d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:38.289 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.265+0000 7f136659c700 1 -- 172.21.15.39:0/3346682257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b08a242d0 msgr2=0x562b08a2d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:44:38.289 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.265+0000 7f136659c700 1 --2- 172.21.15.39:0/3346682257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b08a242d0 0x562b08a2d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:38.289 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.265+0000 7f136659c700 1 -- 172.21.15.39:0/3346682257 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b08a1e580 msgr2=0x562b08a2d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:44:38.289 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.265+0000 7f136659c700 1 --2- 172.21.15.39:0/3346682257 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b08a1e580 0x562b08a2d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:38.289 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.265+0000 7f136659c700 1 -- 172.21.15.39:0/3346682257 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562b08786c00 con 0x562b0887a6b0 2023-07-21T18:44:38.290 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.265+0000 7f1365d9b700 1 --2- 172.21.15.39:0/3346682257 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b08a1e580 0x562b08a2d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:44:38.290 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.265+0000 7f136659c700 1 --2- 172.21.15.39:0/3346682257 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b0887a6b0 0x562b08a2cc50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f135801c900 tx=0x7f1358011b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=189505655cd6dab5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:44:38.290 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.265+0000 7f134dffb700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2023-07-21T18:44:38.290 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.265+0000 7f134dffb700 1 -- 172.21.15.39:0/3346682257 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135801cb20 con 0x562b0887a6b0 2023-07-21T18:44:38.290 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.265+0000 7f134dffb700 1 -- 172.21.15.39:0/3346682257 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f135803c080 con 0x562b0887a6b0 2023-07-21T18:44:38.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.265+0000 7f1378a82b80 1 -- 172.21.15.39:0/3346682257 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1358021070 con 0x562b0887a6b0 2023-07-21T18:44:38.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.265+0000 7f134dffb700 1 -- 172.21.15.39:0/3346682257 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f135801cc80 con 0x562b0887a6b0 2023-07-21T18:44:38.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.265+0000 7f1378a82b80 10 client.25840 fetch_fsmap learned FSMap version 33 2023-07-21T18:44:38.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.265+0000 7f1378a82b80 1 -- 172.21.15.39:0/3346682257 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x562b087c41f0 con 0x562b0887a6b0 2023-07-21T18:44:38.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.266+0000 7f134dffb700 1 -- 172.21.15.39:0/3346682257 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f135804d020 con 0x562b0887a6b0 2023-07-21T18:44:38.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.266+0000 7f134dffb700 1 -- 172.21.15.39:0/3346682257 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f13580264d0 con 0x562b0887a6b0 2023-07-21T18:44:38.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.266+0000 7f1378a82b80 10 client.25840 fetch_fsmap finished waiting for FSMap version 33 2023-07-21T18:44:38.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.266+0000 7f1378a82b80 10 client.25840 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:44:38.292 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.266+0000 7f1378a82b80 20 client.25840 populate_metadata read hostname 'smithi039' 2023-07-21T18:44:38.292 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.266+0000 7f1378a82b80 1 --2- 172.21.15.39:0/3346682257 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x562b08a8c180 0x562b08a8e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:38.292 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.266+0000 7f1378a82b80 4 client.25840 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:44:38.292 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.266+0000 7f1378a82b80 1 -- 172.21.15.39:0/3346682257 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562b0892aff0 con 0x562b08a8c180 2023-07-21T18:44:38.292 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.266+0000 7f1365d9b700 1 --2- 172.21.15.39:0/3346682257 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x562b08a8c180 0x562b08a8e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:38.292 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.266+0000 7f134dffb700 10 client.25840 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:44:38.292 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.266+0000 7f1365d9b700 1 --2- 172.21.15.39:0/3346682257 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x562b08a8c180 0x562b08a8e590 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=527facd72e5b565e server_cookie=74fb5d277155e428 in_seq=0 out_seq=0 2023-07-21T18:44:38.331 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.313+0000 7f134dffb700 1 -- 172.21.15.39:0/3346682257 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562b0892aff0 con 0x562b08a8c180 2023-07-21T18:44:38.331 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.313+0000 7f134dffb700 10 client.25840 handle_command_reply: tid=0 2023-07-21T18:44:38.331 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.313+0000 7f1378a82b80 1 client.25840 shutdown 2023-07-21T18:44:38.331 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.313+0000 7f1378a82b80 1 -- 172.21.15.39:0/3346682257 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b0887a6b0 msgr2=0x562b08a2cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:44:38.331 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.313+0000 7f1378a82b80 1 --2- 172.21.15.39:0/3346682257 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b0887a6b0 0x562b08a2cc50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f135801c900 tx=0x7f1358011b30 comp rx=0 tx=0).stop 2023-07-21T18:44:38.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.313+0000 7f1378a82b80 1 -- 172.21.15.39:0/3346682257 shutdown_connections 2023-07-21T18:44:38.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.313+0000 7f1378a82b80 1 --2- 172.21.15.39:0/3346682257 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x562b08a8c180 0x562b08a8e590 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:38.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.313+0000 7f1378a82b80 1 --2- 172.21.15.39:0/3346682257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b08a242d0 0x562b08a2d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:38.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.313+0000 7f1378a82b80 1 --2- 172.21.15.39:0/3346682257 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b0887a6b0 0x562b08a2cc50 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:38.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.313+0000 7f134dffb700 0 client.25840 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:44:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.313+0000 7f1378a82b80 1 --2- 172.21.15.39:0/3346682257 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b08a1e580 0x562b08a2d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.313+0000 7f1378a82b80 1 -- 172.21.15.39:0/3346682257 >> 172.21.15.39:0/3346682257 conn(0x562b08a13be0 msgr2=0x562b08a23340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:44:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.313+0000 7f1378a82b80 1 -- 172.21.15.39:0/3346682257 shutdown_connections 2023-07-21T18:44:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.313+0000 7f1378a82b80 1 -- 172.21.15.39:0/3346682257 wait complete. 2023-07-21T18:44:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.313+0000 7f1378a82b80 20 client.25840 trim_cache size 0 max 16384 2023-07-21T18:44:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:44:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:44:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:44:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:44:38.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.314+0000 7f136c8fc700 1 -- 172.21.15.39:0/2456730729 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f13500a1340 msgr2=0x7f13500a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:44:38.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.314+0000 7f136c8fc700 1 --2- 172.21.15.39:0/2456730729 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f13500a1340 0x7f13500a37f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1358009c60 tx=0x7f1358009260 comp rx=0 tx=0).stop 2023-07-21T18:44:38.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.314+0000 7f136c8fc700 1 -- 172.21.15.39:0/2456730729 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1368104e50 msgr2=0x7f136819cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:44:38.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.314+0000 7f136c8fc700 1 --2- 172.21.15.39:0/2456730729 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1368104e50 0x7f136819cb50 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f1360007820 tx=0x7f1360007850 comp rx=0 tx=0).stop 2023-07-21T18:44:38.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.314+0000 7f136c8fc700 1 -- 172.21.15.39:0/2456730729 shutdown_connections 2023-07-21T18:44:38.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.314+0000 7f136c8fc700 1 --2- 172.21.15.39:0/2456730729 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f13500a1340 0x7f13500a37f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:38.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.314+0000 7f136c8fc700 1 --2- 172.21.15.39:0/2456730729 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1368067c10 0x7f1368197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:38.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.314+0000 7f136c8fc700 1 --2- 172.21.15.39:0/2456730729 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1368104e50 0x7f136819cb50 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:38.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.314+0000 7f136c8fc700 1 --2- 172.21.15.39:0/2456730729 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13680685b0 0x7f13681984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:38.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.314+0000 7f136c8fc700 1 -- 172.21.15.39:0/2456730729 >> 172.21.15.39:0/2456730729 conn(0x7f1368074760 msgr2=0x7f13680fc920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:44:38.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.314+0000 7f136c8fc700 1 -- 172.21.15.39:0/2456730729 shutdown_connections 2023-07-21T18:44:38.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:38.314+0000 7f136c8fc700 1 -- 172.21.15.39:0/2456730729 wait complete. 2023-07-21T18:44:38.365 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:44:38 smithi193 ceph-mon[98204]: pgmap v89: 97 pgs: 97 active+clean; 75 MiB data, 843 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.8 MiB/s wr, 401 op/s 2023-07-21T18:44:38.384 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:44:38 smithi155 ceph-mon[97934]: pgmap v89: 97 pgs: 97 active+clean; 75 MiB data, 843 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.8 MiB/s wr, 401 op/s 2023-07-21T18:44:38.433 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:44:38 smithi039 ceph-mon[96489]: pgmap v89: 97 pgs: 97 active+clean; 75 MiB data, 843 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.8 MiB/s wr, 401 op/s 2023-07-21T18:44:39.263 INFO:tasks.workunit.client.0.smithi039.stdout: 1 138308 6.83 MB/sec warmup 112 sec latency 76.379 ms 2023-07-21T18:44:40.263 INFO:tasks.workunit.client.0.smithi039.stdout: 1 139503 6.80 MB/sec warmup 113 sec latency 24.584 ms 2023-07-21T18:44:40.365 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:44:40 smithi193 ceph-mon[98204]: pgmap v90: 97 pgs: 97 active+clean; 75 MiB data, 843 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 254 op/s 2023-07-21T18:44:40.384 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:44:40 smithi155 ceph-mon[97934]: pgmap v90: 97 pgs: 97 active+clean; 75 MiB data, 843 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 254 op/s 2023-07-21T18:44:40.433 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:44:40 smithi039 ceph-mon[96489]: pgmap v90: 97 pgs: 97 active+clean; 75 MiB data, 843 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 254 op/s 2023-07-21T18:44:41.263 INFO:tasks.workunit.client.0.smithi039.stdout: 1 142033 6.88 MB/sec warmup 114 sec latency 27.641 ms 2023-07-21T18:44:42.263 INFO:tasks.workunit.client.0.smithi039.stdout: 1 143505 6.88 MB/sec warmup 115 sec latency 12.632 ms 2023-07-21T18:44:42.384 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:44:42 smithi155 ceph-mon[97934]: pgmap v91: 97 pgs: 97 active+clean; 73 MiB data, 884 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.8 MiB/s wr, 408 op/s 2023-07-21T18:44:42.433 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:44:42 smithi039 ceph-mon[96489]: pgmap v91: 97 pgs: 97 active+clean; 73 MiB data, 884 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.8 MiB/s wr, 408 op/s 2023-07-21T18:44:42.615 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:44:42 smithi193 ceph-mon[98204]: pgmap v91: 97 pgs: 97 active+clean; 73 MiB data, 884 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.8 MiB/s wr, 408 op/s 2023-07-21T18:44:43.263 INFO:tasks.workunit.client.0.smithi039.stdout: 1 145783 6.94 MB/sec warmup 116 sec latency 13.655 ms 2023-07-21T18:44:44.263 INFO:tasks.workunit.client.0.smithi039.stdout: 1 147861 6.94 MB/sec warmup 117 sec latency 11.692 ms 2023-07-21T18:44:44.384 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:44:44 smithi155 ceph-mon[97934]: pgmap v92: 97 pgs: 97 active+clean; 73 MiB data, 884 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 280 op/s 2023-07-21T18:44:44.433 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:44:44 smithi039 ceph-mon[96489]: pgmap v92: 97 pgs: 97 active+clean; 73 MiB data, 884 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 280 op/s 2023-07-21T18:44:44.615 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:44:44 smithi193 ceph-mon[98204]: pgmap v92: 97 pgs: 97 active+clean; 73 MiB data, 884 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 280 op/s 2023-07-21T18:44:45.263 INFO:tasks.workunit.client.0.smithi039.stdout: 1 149428 6.98 MB/sec warmup 118 sec latency 38.896 ms 2023-07-21T18:44:46.263 INFO:tasks.workunit.client.0.smithi039.stdout: 1 151577 6.99 MB/sec warmup 119 sec latency 16.341 ms 2023-07-21T18:44:46.384 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:44:46 smithi155 ceph-mon[97934]: pgmap v93: 97 pgs: 97 active+clean; 73 MiB data, 884 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 280 op/s 2023-07-21T18:44:46.433 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:44:46 smithi039 ceph-mon[96489]: pgmap v93: 97 pgs: 97 active+clean; 73 MiB data, 884 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 280 op/s 2023-07-21T18:44:46.615 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:44:46 smithi193 ceph-mon[98204]: pgmap v93: 97 pgs: 97 active+clean; 73 MiB data, 884 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 280 op/s 2023-07-21T18:44:48.263 INFO:tasks.workunit.client.0.smithi039.stdout: 1 155521 12.54 MB/sec execute 1 sec latency 33.608 ms 2023-07-21T18:44:48.384 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:44:48 smithi155 ceph-mon[97934]: pgmap v94: 97 pgs: 97 active+clean; 78 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.8 MiB/s wr, 409 op/s 2023-07-21T18:44:48.384 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:44:48 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:44:48.433 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:44:48 smithi039 ceph-mon[96489]: pgmap v94: 97 pgs: 97 active+clean; 78 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.8 MiB/s wr, 409 op/s 2023-07-21T18:44:48.433 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:44:48 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:44:48.615 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:44:48 smithi193 ceph-mon[98204]: pgmap v94: 97 pgs: 97 active+clean; 78 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.8 MiB/s wr, 409 op/s 2023-07-21T18:44:48.616 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:44:48 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:44:49.263 INFO:tasks.workunit.client.0.smithi039.stdout: 1 156250 8.91 MB/sec execute 2 sec latency 630.409 ms 2023-07-21T18:44:50.263 INFO:tasks.workunit.client.0.smithi039.stdout: 1 156250 5.94 MB/sec execute 3 sec latency 1630.473 ms 2023-07-21T18:44:50.384 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:44:50 smithi155 ceph-mon[97934]: pgmap v95: 97 pgs: 97 active+clean; 78 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 282 op/s 2023-07-21T18:44:50.433 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:44:50 smithi039 ceph-mon[96489]: pgmap v95: 97 pgs: 97 active+clean; 78 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 282 op/s 2023-07-21T18:44:50.615 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:44:50 smithi193 ceph-mon[98204]: pgmap v95: 97 pgs: 97 active+clean; 78 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 282 op/s 2023-07-21T18:44:51.263 INFO:tasks.workunit.client.0.smithi039.stdout: 1 156390 4.70 MB/sec execute 4 sec latency 2450.936 ms 2023-07-21T18:44:51.455 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.437+0000 7fe60a4bb700 1 -- 172.21.15.39:0/365366776 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6040ff3b0 msgr2=0x7fe6040ff780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:44:51.455 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.437+0000 7fe60a4bb700 1 --2- 172.21.15.39:0/365366776 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6040ff3b0 0x7fe6040ff780 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fe5f400b0d0 tx=0x7fe5f400b3e0 comp rx=0 tx=0).stop 2023-07-21T18:44:51.455 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.437+0000 7fe60a4bb700 1 -- 172.21.15.39:0/365366776 shutdown_connections 2023-07-21T18:44:51.455 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.437+0000 7fe60a4bb700 1 --2- 172.21.15.39:0/365366776 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe604103ed0 0x7fe60410baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:51.455 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.437+0000 7fe60a4bb700 1 --2- 172.21.15.39:0/365366776 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6040ff3b0 0x7fe6040ff780 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:51.456 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.437+0000 7fe60a4bb700 1 --2- 172.21.15.39:0/365366776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6040ffd50 0x7fe6041037f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:51.456 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.437+0000 7fe60a4bb700 1 -- 172.21.15.39:0/365366776 >> 172.21.15.39:0/365366776 conn(0x7fe604075f90 msgr2=0x7fe604076390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:44:51.456 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.437+0000 7fe60a4bb700 1 -- 172.21.15.39:0/365366776 shutdown_connections 2023-07-21T18:44:51.456 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.437+0000 7fe60a4bb700 1 -- 172.21.15.39:0/365366776 wait complete. 2023-07-21T18:44:51.457 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.438+0000 7fe60a4bb700 1 Processor -- start 2023-07-21T18:44:51.458 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.438+0000 7fe60a4bb700 1 -- start start 2023-07-21T18:44:51.458 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.438+0000 7fe60a4bb700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6040ff3b0 0x7fe604196020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:51.458 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.438+0000 7fe60a4bb700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6040ffd50 0x7fe604196560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:51.458 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.438+0000 7fe60a4bb700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe604103ed0 0x7fe60419abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:51.458 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.438+0000 7fe60a4bb700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe60419b210 con 0x7fe6040ff3b0 2023-07-21T18:44:51.458 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.438+0000 7fe60a4bb700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe60419b380 con 0x7fe6040ffd50 2023-07-21T18:44:51.459 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.438+0000 7fe60a4bb700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe60419b540 con 0x7fe604103ed0 2023-07-21T18:44:51.459 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.438+0000 7fe603fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6040ff3b0 0x7fe604196020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:51.459 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.438+0000 7fe603fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6040ff3b0 0x7fe604196020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59364/0 (socket says 172.21.15.39:59364) 2023-07-21T18:44:51.459 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.438+0000 7fe603fff700 1 -- 172.21.15.39:0/2120919039 learned_addr learned my addr 172.21.15.39:0/2120919039 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:44:51.459 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.438+0000 7fe608a58700 1 --2- 172.21.15.39:0/2120919039 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe604103ed0 0x7fe60419abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:51.459 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.438+0000 7fe6037fe700 1 --2- 172.21.15.39:0/2120919039 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6040ffd50 0x7fe604196560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:51.459 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.438+0000 7fe603fff700 1 -- 172.21.15.39:0/2120919039 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe604103ed0 msgr2=0x7fe60419abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:44:51.459 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.438+0000 7fe603fff700 1 --2- 172.21.15.39:0/2120919039 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe604103ed0 0x7fe60419abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:51.460 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.438+0000 7fe603fff700 1 -- 172.21.15.39:0/2120919039 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6040ffd50 msgr2=0x7fe604196560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:44:51.460 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.438+0000 7fe603fff700 1 --2- 172.21.15.39:0/2120919039 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6040ffd50 0x7fe604196560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:51.460 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.438+0000 7fe603fff700 1 -- 172.21.15.39:0/2120919039 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5f4009d00 con 0x7fe6040ff3b0 2023-07-21T18:44:51.460 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.439+0000 7fe603fff700 1 --2- 172.21.15.39:0/2120919039 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6040ff3b0 0x7fe604196020 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4000f80 tx=0x7fe5f4007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:44:51.460 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.439+0000 7fe6017fa700 1 -- 172.21.15.39:0/2120919039 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f400e040 con 0x7fe6040ff3b0 2023-07-21T18:44:51.460 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.439+0000 7fe6017fa700 1 -- 172.21.15.39:0/2120919039 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5f4005030 con 0x7fe6040ff3b0 2023-07-21T18:44:51.460 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.439+0000 7fe60a4bb700 1 -- 172.21.15.39:0/2120919039 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe60419b7c0 con 0x7fe6040ff3b0 2023-07-21T18:44:51.461 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.439+0000 7fe6017fa700 1 -- 172.21.15.39:0/2120919039 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f401d6d0 con 0x7fe6040ff3b0 2023-07-21T18:44:51.461 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.439+0000 7fe60a4bb700 1 -- 172.21.15.39:0/2120919039 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6041a3060 con 0x7fe6040ff3b0 2023-07-21T18:44:51.463 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.441+0000 7fe6017fa700 1 -- 172.21.15.39:0/2120919039 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fe5f40054a0 con 0x7fe6040ff3b0 2023-07-21T18:44:51.463 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.442+0000 7fe6017fa700 1 --2- 172.21.15.39:0/2120919039 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fe5f00a12c0 0x7fe5f00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:51.464 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.442+0000 7fe6017fa700 1 -- 172.21.15.39:0/2120919039 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fe5f40cdbe0 con 0x7fe6040ff3b0 2023-07-21T18:44:51.464 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.442+0000 7fe6037fe700 1 --2- 172.21.15.39:0/2120919039 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fe5f00a12c0 0x7fe5f00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:51.464 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.442+0000 7fe6037fe700 1 --2- 172.21.15.39:0/2120919039 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fe5f00a12c0 0x7fe5f00a3770 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe604197840 tx=0x7fe5ec009260 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:44:51.464 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.443+0000 7fe616641b80 1 Processor -- start 2023-07-21T18:44:51.464 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.443+0000 7fe616641b80 1 -- start start 2023-07-21T18:44:51.464 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.443+0000 7fe616641b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba3165e6b0 0x55ba3180c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:51.464 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.443+0000 7fe616641b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba3180c6a0 0x55ba31806450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:51.465 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.443+0000 7fe616641b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ba3180cb10 0x55ba318069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:51.465 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.443+0000 7fe616641b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba31707c50 con 0x55ba3180cb10 2023-07-21T18:44:51.465 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.443+0000 7fe616641b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba3175a9e0 con 0x55ba3180c6a0 2023-07-21T18:44:51.465 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.443+0000 7fe616641b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba31753e70 con 0x55ba3165e6b0 2023-07-21T18:44:51.465 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.443+0000 7fe603fff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba3165e6b0 0x55ba3180c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:51.465 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.443+0000 7fe608a58700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ba3180cb10 0x55ba318069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:51.465 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.443+0000 7fe603fff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba3165e6b0 0x55ba3180c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:49692/0 (socket says 172.21.15.39:49692) 2023-07-21T18:44:51.465 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.443+0000 7fe603fff700 1 -- 172.21.15.39:0/2573194193 learned_addr learned my addr 172.21.15.39:0/2573194193 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:44:51.466 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.443+0000 7fe6037fe700 1 --2- 172.21.15.39:0/2573194193 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba3180c6a0 0x55ba31806450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:51.466 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.443+0000 7fe603fff700 1 -- 172.21.15.39:0/2573194193 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba3180c6a0 msgr2=0x55ba31806450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:44:51.466 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.443+0000 7fe603fff700 1 --2- 172.21.15.39:0/2573194193 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba3180c6a0 0x55ba31806450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:51.466 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.443+0000 7fe603fff700 1 -- 172.21.15.39:0/2573194193 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ba3180cb10 msgr2=0x55ba318069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:44:51.466 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.443+0000 7fe603fff700 1 --2- 172.21.15.39:0/2573194193 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ba3180cb10 0x55ba318069d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:51.466 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.443+0000 7fe603fff700 1 -- 172.21.15.39:0/2573194193 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba31581d80 con 0x55ba3165e6b0 2023-07-21T18:44:51.466 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.444+0000 7fe603fff700 1 --2- 172.21.15.39:0/2573194193 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba3165e6b0 0x55ba3180c160 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe5f40977d0 tx=0x7fe5f409abb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e355b1f290d842ec server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:44:51.467 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.444+0000 7fe5fa7fc700 1 -- 172.21.15.39:0/2573194193 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f40051a0 con 0x55ba3165e6b0 2023-07-21T18:44:51.467 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.444+0000 7fe5fa7fc700 1 -- 172.21.15.39:0/2573194193 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5f409ad60 con 0x55ba3165e6b0 2023-07-21T18:44:51.467 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.444+0000 7fe5fa7fc700 1 -- 172.21.15.39:0/2573194193 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f409b200 con 0x55ba3165e6b0 2023-07-21T18:44:51.467 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.444+0000 7fe616641b80 1 -- 172.21.15.39:0/2573194193 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba3165e6b0 msgr2=0x55ba3180c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:44:51.467 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.444+0000 7fe616641b80 1 --2- 172.21.15.39:0/2573194193 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba3165e6b0 0x55ba3180c160 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe5f40977d0 tx=0x7fe5f409abb0 comp rx=0 tx=0).stop 2023-07-21T18:44:51.467 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.445+0000 7fe616641b80 1 -- 172.21.15.39:0/2573194193 shutdown_connections 2023-07-21T18:44:51.467 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.445+0000 7fe616641b80 1 --2- 172.21.15.39:0/2573194193 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ba3180cb10 0x55ba318069d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:51.467 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.445+0000 7fe616641b80 1 --2- 172.21.15.39:0/2573194193 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba3165e6b0 0x55ba3180c160 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:51.468 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.445+0000 7fe616641b80 1 --2- 172.21.15.39:0/2573194193 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba3180c6a0 0x55ba31806450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:51.468 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.445+0000 7fe616641b80 1 -- 172.21.15.39:0/2573194193 >> 172.21.15.39:0/2573194193 conn(0x55ba317f7be0 msgr2=0x55ba317fcf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:44:51.468 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.445+0000 7fe616641b80 1 -- 172.21.15.39:0/2573194193 shutdown_connections 2023-07-21T18:44:51.468 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.445+0000 7fe616641b80 1 -- 172.21.15.39:0/2573194193 wait complete. 2023-07-21T18:44:51.468 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.445+0000 7fe616641b80 1 -- start start 2023-07-21T18:44:51.468 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.445+0000 7fe616641b80 1 Processor -- start 2023-07-21T18:44:51.468 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.445+0000 7fe616641b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba3165e6b0 0x55ba31815820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.445+0000 7fe616641b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ba3180c6a0 0x55ba31815d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.445+0000 7fe616641b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba3180cb10 0x55ba3180fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.445+0000 7fe616641b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba316f3a60 con 0x55ba3180c6a0 2023-07-21T18:44:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.445+0000 7fe616641b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba316f3760 con 0x55ba3180cb10 2023-07-21T18:44:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.445+0000 7fe616641b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba315d78c0 con 0x55ba3165e6b0 2023-07-21T18:44:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.445+0000 7fe616641b80 10 client.0 _ll_register_callbacks cb 0x55ba316f1700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:44:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.446+0000 7fe6037fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ba3180c6a0 0x55ba31815d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.446+0000 7fe603fff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba3165e6b0 0x55ba31815820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:51.470 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.446+0000 7fe6037fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ba3180c6a0 0x55ba31815d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59380/0 (socket says 172.21.15.39:59380) 2023-07-21T18:44:51.470 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.446+0000 7fe6037fe700 1 -- 172.21.15.39:0/1489664328 learned_addr learned my addr 172.21.15.39:0/1489664328 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:44:51.470 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.446+0000 7fe608a58700 1 --2- 172.21.15.39:0/1489664328 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba3180cb10 0x55ba3180fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:51.470 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.446+0000 7fe603fff700 1 -- 172.21.15.39:0/1489664328 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba3180cb10 msgr2=0x55ba3180fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:44:51.470 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.446+0000 7fe603fff700 1 --2- 172.21.15.39:0/1489664328 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba3180cb10 0x55ba3180fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:51.470 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.446+0000 7fe603fff700 1 -- 172.21.15.39:0/1489664328 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ba3180c6a0 msgr2=0x55ba31815d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:44:51.470 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.446+0000 7fe603fff700 1 --2- 172.21.15.39:0/1489664328 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ba3180c6a0 0x55ba31815d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:51.470 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.446+0000 7fe603fff700 1 -- 172.21.15.39:0/1489664328 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6041a3060 con 0x55ba3165e6b0 2023-07-21T18:44:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.446+0000 7fe603fff700 1 --2- 172.21.15.39:0/1489664328 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba3165e6b0 0x55ba31815820 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe5f40a32e0 tx=0x7fe5f409a480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9217253f92c2a440 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:44:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.446+0000 7fe5e37fe700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2023-07-21T18:44:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.446+0000 7fe5e37fe700 1 -- 172.21.15.39:0/1489664328 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f40051a0 con 0x55ba3165e6b0 2023-07-21T18:44:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.446+0000 7fe5e37fe700 1 -- 172.21.15.39:0/1489664328 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fe5f409ed90 con 0x55ba3165e6b0 2023-07-21T18:44:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.446+0000 7fe616641b80 1 -- 172.21.15.39:0/1489664328 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ba316ad490 con 0x55ba3165e6b0 2023-07-21T18:44:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.446+0000 7fe5e37fe700 1 -- 172.21.15.39:0/1489664328 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe5f409b1d0 con 0x55ba3165e6b0 2023-07-21T18:44:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.446+0000 7fe616641b80 10 client.25852 fetch_fsmap learned FSMap version 34 2023-07-21T18:44:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.446+0000 7fe616641b80 1 -- 172.21.15.39:0/1489664328 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55ba3156ac00 con 0x55ba3165e6b0 2023-07-21T18:44:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.447+0000 7fe5e37fe700 1 -- 172.21.15.39:0/1489664328 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ba316ad490 con 0x55ba3165e6b0 2023-07-21T18:44:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.447+0000 7fe5e37fe700 1 -- 172.21.15.39:0/1489664328 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) v1 ==== 7490+0+0 (secure 0 0 0) 0x7fe5f40b3120 con 0x55ba3165e6b0 2023-07-21T18:44:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.447+0000 7fe616641b80 10 client.25852 fetch_fsmap finished waiting for FSMap version 34 2023-07-21T18:44:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.447+0000 7fe616641b80 10 client.25852 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:44:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.447+0000 7fe616641b80 20 client.25852 populate_metadata read hostname 'smithi039' 2023-07-21T18:44:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.447+0000 7fe616641b80 1 --2- 172.21.15.39:0/1489664328 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55ba3181f850 0x55ba31821c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.447+0000 7fe616641b80 4 client.25852 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:44:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.447+0000 7fe616641b80 1 -- 172.21.15.39:0/1489664328 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ba3174ce50 con 0x55ba3181f850 2023-07-21T18:44:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.447+0000 7fe6037fe700 1 --2- 172.21.15.39:0/1489664328 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55ba3181f850 0x55ba31821c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.447+0000 7fe5e37fe700 10 client.25852 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:44:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.447+0000 7fe6037fe700 1 --2- 172.21.15.39:0/1489664328 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55ba3181f850 0x55ba31821c60 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9c6c01b47c41424 server_cookie=bd1493cbaf96ec1c in_seq=0 out_seq=0 2023-07-21T18:44:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.448+0000 7fe5e37fe700 1 -- 172.21.15.39:0/1489664328 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ba3174ce50 con 0x55ba3181f850 2023-07-21T18:44:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.448+0000 7fe5e37fe700 10 client.25852 handle_command_reply: tid=0 2023-07-21T18:44:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.448+0000 7fe616641b80 1 client.25852 shutdown 2023-07-21T18:44:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.448+0000 7fe616641b80 1 -- 172.21.15.39:0/1489664328 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba3165e6b0 msgr2=0x55ba31815820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:44:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.448+0000 7fe616641b80 1 --2- 172.21.15.39:0/1489664328 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba3165e6b0 0x55ba31815820 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe5f40a32e0 tx=0x7fe5f409a480 comp rx=0 tx=0).stop 2023-07-21T18:44:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.448+0000 7fe616641b80 1 -- 172.21.15.39:0/1489664328 shutdown_connections 2023-07-21T18:44:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.448+0000 7fe616641b80 1 --2- 172.21.15.39:0/1489664328 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55ba3181f850 0x55ba31821c60 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.448+0000 7fe616641b80 1 --2- 172.21.15.39:0/1489664328 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba3180cb10 0x55ba3180fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.448+0000 7fe5e37fe700 0 client.25852 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:44:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.448+0000 7fe616641b80 1 --2- 172.21.15.39:0/1489664328 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba3165e6b0 0x55ba31815820 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.448+0000 7fe616641b80 1 --2- 172.21.15.39:0/1489664328 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ba3180c6a0 0x55ba31815d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.448+0000 7fe616641b80 1 -- 172.21.15.39:0/1489664328 >> 172.21.15.39:0/1489664328 conn(0x55ba317f7be0 msgr2=0x55ba317fc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:44:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.448+0000 7fe616641b80 1 -- 172.21.15.39:0/1489664328 shutdown_connections 2023-07-21T18:44:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.448+0000 7fe616641b80 1 -- 172.21.15.39:0/1489664328 wait complete. 2023-07-21T18:44:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.448+0000 7fe616641b80 20 client.25852 trim_cache size 0 max 16384 2023-07-21T18:44:51.482 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.464+0000 7fe616641b80 1 Processor -- start 2023-07-21T18:44:51.483 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.464+0000 7fe616641b80 1 -- start start 2023-07-21T18:44:51.483 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.464+0000 7fe616641b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ba3165e6b0 0x55ba3180d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:51.483 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.464+0000 7fe616641b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba3180c6a0 0x55ba317fcc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:51.483 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.464+0000 7fe616641b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba3180cb10 0x55ba3180f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:51.483 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.464+0000 7fe616641b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba317c80a0 con 0x55ba3165e6b0 2023-07-21T18:44:51.483 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.464+0000 7fe616641b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba31807620 con 0x55ba3180cb10 2023-07-21T18:44:51.483 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.464+0000 7fe616641b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba317f95f0 con 0x55ba3180c6a0 2023-07-21T18:44:51.483 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.465+0000 7fe603fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ba3165e6b0 0x55ba3180d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:51.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.465+0000 7fe603fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ba3165e6b0 0x55ba3180d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59386/0 (socket says 172.21.15.39:59386) 2023-07-21T18:44:51.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.465+0000 7fe603fff700 1 -- 172.21.15.39:0/1498983249 learned_addr learned my addr 172.21.15.39:0/1498983249 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:44:51.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.465+0000 7fe608a58700 1 --2- 172.21.15.39:0/1498983249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba3180cb10 0x55ba3180f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:51.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.465+0000 7fe6037fe700 1 --2- 172.21.15.39:0/1498983249 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba3180c6a0 0x55ba317fcc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:51.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.465+0000 7fe603fff700 1 -- 172.21.15.39:0/1498983249 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba3180c6a0 msgr2=0x55ba317fcc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:44:51.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.465+0000 7fe603fff700 1 --2- 172.21.15.39:0/1498983249 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba3180c6a0 0x55ba317fcc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:51.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.465+0000 7fe603fff700 1 -- 172.21.15.39:0/1498983249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba3180cb10 msgr2=0x55ba3180f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:44:51.485 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.465+0000 7fe603fff700 1 --2- 172.21.15.39:0/1498983249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba3180cb10 0x55ba3180f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:51.485 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.465+0000 7fe603fff700 1 -- 172.21.15.39:0/1498983249 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba315a81f0 con 0x55ba3165e6b0 2023-07-21T18:44:51.485 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.465+0000 7fe608a58700 1 --2- 172.21.15.39:0/1498983249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba3180cb10 0x55ba3180f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:44:51.485 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.465+0000 7fe603fff700 1 --2- 172.21.15.39:0/1498983249 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ba3165e6b0 0x55ba3180d7c0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fe5f40cb950 tx=0x7fe5f400b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd62494c89ef2ba8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:44:51.487 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.465+0000 7fe5faffd700 1 -- 172.21.15.39:0/1498983249 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f40051a0 con 0x55ba3165e6b0 2023-07-21T18:44:51.488 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.465+0000 7fe5faffd700 1 -- 172.21.15.39:0/1498983249 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5f40b36b0 con 0x55ba3165e6b0 2023-07-21T18:44:51.488 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.465+0000 7fe5faffd700 1 -- 172.21.15.39:0/1498983249 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f4096db0 con 0x55ba3165e6b0 2023-07-21T18:44:51.488 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.466+0000 7fe616641b80 1 -- 172.21.15.39:0/1498983249 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ba3165e6b0 msgr2=0x55ba3180d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:44:51.488 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.466+0000 7fe616641b80 1 --2- 172.21.15.39:0/1498983249 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ba3165e6b0 0x55ba3180d7c0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fe5f40cb950 tx=0x7fe5f400b3e0 comp rx=0 tx=0).stop 2023-07-21T18:44:51.488 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.466+0000 7fe616641b80 1 -- 172.21.15.39:0/1498983249 shutdown_connections 2023-07-21T18:44:51.488 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.466+0000 7fe616641b80 1 --2- 172.21.15.39:0/1498983249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba3180cb10 0x55ba3180f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:51.489 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.466+0000 7fe616641b80 1 --2- 172.21.15.39:0/1498983249 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ba3165e6b0 0x55ba3180d7c0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:51.489 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.466+0000 7fe616641b80 1 --2- 172.21.15.39:0/1498983249 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba3180c6a0 0x55ba317fcc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:51.489 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.466+0000 7fe616641b80 1 -- 172.21.15.39:0/1498983249 >> 172.21.15.39:0/1498983249 conn(0x55ba317f7be0 msgr2=0x55ba31806cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:44:51.489 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.466+0000 7fe616641b80 1 -- 172.21.15.39:0/1498983249 shutdown_connections 2023-07-21T18:44:51.489 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.466+0000 7fe616641b80 1 -- 172.21.15.39:0/1498983249 wait complete. 2023-07-21T18:44:51.489 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.466+0000 7fe616641b80 1 -- start start 2023-07-21T18:44:51.489 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.467+0000 7fe616641b80 1 Processor -- start 2023-07-21T18:44:51.489 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.467+0000 7fe616641b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ba3165e6b0 0x55ba31813240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:51.490 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.467+0000 7fe616641b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba3180c6a0 0x55ba31813780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:51.490 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.467+0000 7fe616641b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba3180cb10 0x55ba31813cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:51.490 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.467+0000 7fe616641b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba318726d0 con 0x55ba3165e6b0 2023-07-21T18:44:51.490 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.467+0000 7fe616641b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba31872840 con 0x55ba3180c6a0 2023-07-21T18:44:51.490 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.467+0000 7fe616641b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba31872a00 con 0x55ba3180cb10 2023-07-21T18:44:51.490 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.467+0000 7fe616641b80 10 client.0 _ll_register_callbacks cb 0x7fe5fc009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:44:51.490 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.467+0000 7fe603fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ba3165e6b0 0x55ba31813240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:51.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.467+0000 7fe603fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ba3165e6b0 0x55ba31813240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59392/0 (socket says 172.21.15.39:59392) 2023-07-21T18:44:51.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.467+0000 7fe603fff700 1 -- 172.21.15.39:0/2822635497 learned_addr learned my addr 172.21.15.39:0/2822635497 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:44:51.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.467+0000 7fe6037fe700 1 --2- 172.21.15.39:0/2822635497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba3180c6a0 0x55ba31813780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:51.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.467+0000 7fe608a58700 1 --2- 172.21.15.39:0/2822635497 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba3180cb10 0x55ba31813cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:51.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.467+0000 7fe603fff700 1 -- 172.21.15.39:0/2822635497 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba3180cb10 msgr2=0x55ba31813cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:44:51.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.467+0000 7fe603fff700 1 --2- 172.21.15.39:0/2822635497 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba3180cb10 0x55ba31813cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:51.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.467+0000 7fe603fff700 1 -- 172.21.15.39:0/2822635497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba3180c6a0 msgr2=0x55ba31813780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:44:51.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.467+0000 7fe603fff700 1 --2- 172.21.15.39:0/2822635497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba3180c6a0 0x55ba31813780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:51.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.467+0000 7fe603fff700 1 -- 172.21.15.39:0/2822635497 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5fc009b20 con 0x55ba3165e6b0 2023-07-21T18:44:51.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.468+0000 7fe603fff700 1 --2- 172.21.15.39:0/2822635497 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ba3165e6b0 0x55ba31813240 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fe5f40a32e0 tx=0x7fe5f409abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea02dccb1bd3344 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:44:51.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.468+0000 7fe5e37fe700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:44:51.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.468+0000 7fe5e37fe700 1 -- 172.21.15.39:0/2822635497 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f4096b50 con 0x55ba3165e6b0 2023-07-21T18:44:51.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.468+0000 7fe5e37fe700 1 -- 172.21.15.39:0/2822635497 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fe5f40a3120 con 0x55ba3165e6b0 2023-07-21T18:44:51.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.468+0000 7fe616641b80 1 -- 172.21.15.39:0/2822635497 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe5f409ad60 con 0x55ba3165e6b0 2023-07-21T18:44:51.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.468+0000 7fe5e37fe700 1 -- 172.21.15.39:0/2822635497 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe5f4096cb0 con 0x55ba3165e6b0 2023-07-21T18:44:51.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.468+0000 7fe616641b80 10 client.16353 fetch_fsmap learned FSMap version 34 2023-07-21T18:44:51.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.468+0000 7fe616641b80 1 -- 172.21.15.39:0/2822635497 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55ba3156ac00 con 0x55ba3165e6b0 2023-07-21T18:44:51.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.468+0000 7fe5e37fe700 1 -- 172.21.15.39:0/2822635497 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5f409ad60 con 0x55ba3165e6b0 2023-07-21T18:44:51.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.468+0000 7fe5e37fe700 1 -- 172.21.15.39:0/2822635497 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 34) v1 ==== 7490+0+0 (secure 0 0 0) 0x7fe5f4099cc0 con 0x55ba3165e6b0 2023-07-21T18:44:51.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.468+0000 7fe616641b80 10 client.16353 fetch_fsmap finished waiting for FSMap version 34 2023-07-21T18:44:51.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.468+0000 7fe616641b80 10 client.16353 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:44:51.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.468+0000 7fe616641b80 20 client.16353 populate_metadata read hostname 'smithi039' 2023-07-21T18:44:51.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.468+0000 7fe616641b80 1 --2- 172.21.15.39:0/2822635497 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55ba31873c40 0x55ba31876020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:44:51.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.468+0000 7fe616641b80 4 client.16353 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:44:51.494 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.468+0000 7fe616641b80 1 -- 172.21.15.39:0/2822635497 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ba3170eff0 con 0x55ba31873c40 2023-07-21T18:44:51.494 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.468+0000 7fe6037fe700 1 --2- 172.21.15.39:0/2822635497 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55ba31873c40 0x55ba31876020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:44:51.494 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.469+0000 7fe5e37fe700 10 client.16353 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:44:51.494 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.469+0000 7fe6037fe700 1 --2- 172.21.15.39:0/2822635497 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55ba31873c40 0x55ba31876020 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2da8882a70681cd server_cookie=524fccea20ee54ab in_seq=0 out_seq=0 2023-07-21T18:44:51.529 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.511+0000 7fe5e37fe700 1 -- 172.21.15.39:0/2822635497 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ba3170eff0 con 0x55ba31873c40 2023-07-21T18:44:51.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.511+0000 7fe5e37fe700 10 client.16353 handle_command_reply: tid=0 2023-07-21T18:44:51.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.511+0000 7fe616641b80 1 client.16353 shutdown 2023-07-21T18:44:51.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.512+0000 7fe616641b80 1 -- 172.21.15.39:0/2822635497 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ba3165e6b0 msgr2=0x55ba31813240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:44:51.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.512+0000 7fe616641b80 1 --2- 172.21.15.39:0/2822635497 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ba3165e6b0 0x55ba31813240 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fe5f40a32e0 tx=0x7fe5f409abd0 comp rx=0 tx=0).stop 2023-07-21T18:44:51.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.512+0000 7fe616641b80 1 -- 172.21.15.39:0/2822635497 shutdown_connections 2023-07-21T18:44:51.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.512+0000 7fe616641b80 1 --2- 172.21.15.39:0/2822635497 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55ba31873c40 0x55ba31876020 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:51.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.512+0000 7fe616641b80 1 --2- 172.21.15.39:0/2822635497 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba3180cb10 0x55ba31813cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:51.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.512+0000 7fe616641b80 1 --2- 172.21.15.39:0/2822635497 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ba3165e6b0 0x55ba31813240 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:51.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.512+0000 7fe5e37fe700 0 client.16353 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:44:51.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.512+0000 7fe616641b80 1 --2- 172.21.15.39:0/2822635497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba3180c6a0 0x55ba31813780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:51.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.512+0000 7fe616641b80 1 -- 172.21.15.39:0/2822635497 >> 172.21.15.39:0/2822635497 conn(0x55ba317f7be0 msgr2=0x55ba31801030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:44:51.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.512+0000 7fe616641b80 1 -- 172.21.15.39:0/2822635497 shutdown_connections 2023-07-21T18:44:51.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.512+0000 7fe616641b80 1 -- 172.21.15.39:0/2822635497 wait complete. 2023-07-21T18:44:51.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.512+0000 7fe616641b80 20 client.16353 trim_cache size 0 max 16384 2023-07-21T18:44:51.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:44:51.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:44:51.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:44:51.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:44:51.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.512+0000 7fe60a4bb700 1 -- 172.21.15.39:0/2120919039 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fe5f00a12c0 msgr2=0x7fe5f00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:44:51.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.512+0000 7fe60a4bb700 1 --2- 172.21.15.39:0/2120919039 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fe5f00a12c0 0x7fe5f00a3770 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe604197840 tx=0x7fe5ec009260 comp rx=0 tx=0).stop 2023-07-21T18:44:51.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.512+0000 7fe60a4bb700 1 -- 172.21.15.39:0/2120919039 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6040ff3b0 msgr2=0x7fe604196020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:44:51.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.512+0000 7fe60a4bb700 1 --2- 172.21.15.39:0/2120919039 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6040ff3b0 0x7fe604196020 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4000f80 tx=0x7fe5f4007870 comp rx=0 tx=0).stop 2023-07-21T18:44:51.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.513+0000 7fe60a4bb700 1 -- 172.21.15.39:0/2120919039 shutdown_connections 2023-07-21T18:44:51.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.513+0000 7fe60a4bb700 1 --2- 172.21.15.39:0/2120919039 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe604103ed0 0x7fe60419abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:51.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.513+0000 7fe60a4bb700 1 --2- 172.21.15.39:0/2120919039 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6040ff3b0 0x7fe604196020 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:51.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.513+0000 7fe60a4bb700 1 --2- 172.21.15.39:0/2120919039 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fe5f00a12c0 0x7fe5f00a3770 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:51.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.513+0000 7fe60a4bb700 1 --2- 172.21.15.39:0/2120919039 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6040ffd50 0x7fe604196560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:44:51.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.513+0000 7fe60a4bb700 1 -- 172.21.15.39:0/2120919039 >> 172.21.15.39:0/2120919039 conn(0x7fe604075f90 msgr2=0x7fe6040fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:44:51.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.513+0000 7fe60a4bb700 1 -- 172.21.15.39:0/2120919039 shutdown_connections 2023-07-21T18:44:51.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:44:51.513+0000 7fe60a4bb700 1 -- 172.21.15.39:0/2120919039 wait complete. 2023-07-21T18:44:52.263 INFO:tasks.workunit.client.0.smithi039.stdout: 1 157684 4.86 MB/sec execute 5 sec latency 39.684 ms 2023-07-21T18:44:52.433 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:44:52 smithi039 ceph-mon[96489]: pgmap v96: 97 pgs: 97 active+clean; 79 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.6 MiB/s wr, 382 op/s 2023-07-21T18:44:52.433 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:44:52 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:44:52.433 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:44:52 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:44:52.433 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:44:52 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:44:52.434 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:44:52 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:44:52.615 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:44:52 smithi193 ceph-mon[98204]: pgmap v96: 97 pgs: 97 active+clean; 79 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.6 MiB/s wr, 382 op/s 2023-07-21T18:44:52.615 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:44:52 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:44:52.616 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:44:52 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:44:52.616 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:44:52 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:44:52.616 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:44:52 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:44:52.634 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:44:52 smithi155 ceph-mon[97934]: pgmap v96: 97 pgs: 97 active+clean; 79 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.6 MiB/s wr, 382 op/s 2023-07-21T18:44:52.634 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:44:52 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:44:52.634 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:44:52 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:44:52.634 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:44:52 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:44:52.635 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:44:52 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:44:53.264 INFO:tasks.workunit.client.0.smithi039.stdout: 1 159645 6.10 MB/sec execute 6 sec latency 166.822 ms 2023-07-21T18:44:53.433 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:44:53 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:44:53.433 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:44:53 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:44:53.433 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:44:53 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:44:53.433 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:44:53 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:44:53.434 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:44:53 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:44:53.434 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:44:53 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:44:53.434 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:44:53 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:44:53.615 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:44:53 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:44:53.616 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:44:53 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:44:53.616 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:44:53 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:44:53.616 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:44:53 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:44:53.616 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:44:53 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:44:53.616 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:44:53 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:44:53.616 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:44:53 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:44:53.634 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:44:53 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:44:53.634 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:44:53 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:44:53.634 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:44:53 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:44:53.634 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:44:53 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:44:53.635 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:44:53 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:44:53.635 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:44:53 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:44:53.635 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:44:53 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:44:54.264 INFO:tasks.workunit.client.0.smithi039.stdout: 1 159645 5.22 MB/sec execute 7 sec latency 1166.945 ms 2023-07-21T18:44:54.433 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:44:54 smithi039 ceph-mon[96489]: pgmap v97: 97 pgs: 97 active+clean; 79 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.3 MiB/s wr, 228 op/s 2023-07-21T18:44:54.615 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:44:54 smithi193 ceph-mon[98204]: pgmap v97: 97 pgs: 97 active+clean; 79 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.3 MiB/s wr, 228 op/s 2023-07-21T18:44:54.634 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:44:54 smithi155 ceph-mon[97934]: pgmap v97: 97 pgs: 97 active+clean; 79 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.3 MiB/s wr, 228 op/s 2023-07-21T18:44:55.267 INFO:tasks.workunit.client.0.smithi039.stdout: 1 159645 4.57 MB/sec execute 8 sec latency 2169.480 ms 2023-07-21T18:44:56.266 INFO:tasks.workunit.client.0.smithi039.stdout: 1 159796 4.10 MB/sec execute 9 sec latency 2987.892 ms 2023-07-21T18:44:56.433 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:44:56 smithi039 ceph-mon[96489]: pgmap v98: 97 pgs: 97 active+clean; 79 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.3 MiB/s wr, 228 op/s 2023-07-21T18:44:56.615 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:44:56 smithi193 ceph-mon[98204]: pgmap v98: 97 pgs: 97 active+clean; 79 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.3 MiB/s wr, 228 op/s 2023-07-21T18:44:56.634 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:44:56 smithi155 ceph-mon[97934]: pgmap v98: 97 pgs: 97 active+clean; 79 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.3 MiB/s wr, 228 op/s 2023-07-21T18:44:57.266 INFO:tasks.workunit.client.0.smithi039.stdout: 1 159796 3.69 MB/sec execute 10 sec latency 1051.576 ms 2023-07-21T18:44:58.266 INFO:tasks.workunit.client.0.smithi039.stdout: 1 159796 3.36 MB/sec execute 11 sec latency 2051.694 ms 2023-07-21T18:44:58.433 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:44:58 smithi039 ceph-mon[96489]: pgmap v99: 97 pgs: 97 active+clean; 76 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 319 op/s 2023-07-21T18:44:58.615 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:44:58 smithi193 ceph-mon[98204]: pgmap v99: 97 pgs: 97 active+clean; 76 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 319 op/s 2023-07-21T18:44:58.634 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:44:58 smithi155 ceph-mon[97934]: pgmap v99: 97 pgs: 97 active+clean; 76 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 319 op/s 2023-07-21T18:44:59.267 INFO:tasks.workunit.client.0.smithi039.stdout: 1 159796 3.08 MB/sec execute 12 sec latency 3051.783 ms 2023-07-21T18:45:00.267 INFO:tasks.workunit.client.0.smithi039.stdout: 1 159796 2.84 MB/sec execute 13 sec latency 4051.835 ms 2023-07-21T18:45:00.433 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:45:00 smithi039 ceph-mon[96489]: pgmap v100: 97 pgs: 97 active+clean; 76 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 1.9 MiB/s wr, 191 op/s 2023-07-21T18:45:00.615 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:45:00 smithi193 ceph-mon[98204]: pgmap v100: 97 pgs: 97 active+clean; 76 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 1.9 MiB/s wr, 191 op/s 2023-07-21T18:45:00.634 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:45:00 smithi155 ceph-mon[97934]: pgmap v100: 97 pgs: 97 active+clean; 76 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 1.9 MiB/s wr, 191 op/s 2023-07-21T18:45:01.267 INFO:tasks.workunit.client.0.smithi039.stdout: 1 159978 2.73 MB/sec execute 14 sec latency 4836.746 ms 2023-07-21T18:45:02.267 INFO:tasks.workunit.client.0.smithi039.stdout: 1 161939 3.00 MB/sec execute 15 sec latency 21.628 ms 2023-07-21T18:45:02.433 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:45:02 smithi039 ceph-mon[96489]: pgmap v101: 97 pgs: 97 active+clean; 74 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 2.1 MiB/s wr, 215 op/s 2023-07-21T18:45:02.615 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:45:02 smithi193 ceph-mon[98204]: pgmap v101: 97 pgs: 97 active+clean; 74 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 2.1 MiB/s wr, 215 op/s 2023-07-21T18:45:02.634 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:45:02 smithi155 ceph-mon[97934]: pgmap v101: 97 pgs: 97 active+clean; 74 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 2.1 MiB/s wr, 215 op/s 2023-07-21T18:45:03.267 INFO:tasks.workunit.client.0.smithi039.stdout: 1 163257 3.49 MB/sec execute 16 sec latency 365.721 ms 2023-07-21T18:45:04.267 INFO:tasks.workunit.client.0.smithi039.stdout: 1 163257 3.29 MB/sec execute 17 sec latency 1365.831 ms 2023-07-21T18:45:04.433 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:45:04 smithi039 ceph-mon[96489]: pgmap v102: 97 pgs: 97 active+clean; 74 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 1.1 MiB/s wr, 115 op/s 2023-07-21T18:45:04.615 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:45:04 smithi193 ceph-mon[98204]: pgmap v102: 97 pgs: 97 active+clean; 74 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 1.1 MiB/s wr, 115 op/s 2023-07-21T18:45:04.634 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:45:04 smithi155 ceph-mon[97934]: pgmap v102: 97 pgs: 97 active+clean; 74 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 1.1 MiB/s wr, 115 op/s 2023-07-21T18:45:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.634+0000 7fe2bf4a2700 1 -- 172.21.15.39:0/1580729800 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2b80fe260 msgr2=0x7fe2b80fe630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:45:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.634+0000 7fe2bf4a2700 1 --2- 172.21.15.39:0/1580729800 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2b80fe260 0x7fe2b80fe630 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fe2a80099c0 tx=0x7fe2a8009cd0 comp rx=0 tx=0).stop 2023-07-21T18:45:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.635+0000 7fe2bf4a2700 1 -- 172.21.15.39:0/1580729800 shutdown_connections 2023-07-21T18:45:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.635+0000 7fe2bf4a2700 1 --2- 172.21.15.39:0/1580729800 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2b8104db0 0x7fe2b810c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.635+0000 7fe2bf4a2700 1 --2- 172.21.15.39:0/1580729800 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2b80fe260 0x7fe2b80fe630 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.635+0000 7fe2bf4a2700 1 --2- 172.21.15.39:0/1580729800 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2b80fec00 0x7fe2b81046d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.635+0000 7fe2bf4a2700 1 -- 172.21.15.39:0/1580729800 >> 172.21.15.39:0/1580729800 conn(0x7fe2b8075f90 msgr2=0x7fe2b8076390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:45:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.635+0000 7fe2bf4a2700 1 -- 172.21.15.39:0/1580729800 shutdown_connections 2023-07-21T18:45:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.635+0000 7fe2bf4a2700 1 -- 172.21.15.39:0/1580729800 wait complete. 2023-07-21T18:45:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.635+0000 7fe2bf4a2700 1 Processor -- start 2023-07-21T18:45:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.635+0000 7fe2bf4a2700 1 -- start start 2023-07-21T18:45:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.636+0000 7fe2bf4a2700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2b80fe260 0x7fe2b80ff3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.636+0000 7fe2bf4a2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2b80fec00 0x7fe2b80ff8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:04.656 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.636+0000 7fe2bf4a2700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2b8104db0 0x7fe2b81a3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:04.656 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.636+0000 7fe2bf4a2700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b81a4340 con 0x7fe2b80fec00 2023-07-21T18:45:04.656 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.636+0000 7fe2bf4a2700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b81a4480 con 0x7fe2b80fe260 2023-07-21T18:45:04.656 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.636+0000 7fe2bf4a2700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b81a45f0 con 0x7fe2b8104db0 2023-07-21T18:45:04.656 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.636+0000 7fe2bca3d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2b80fec00 0x7fe2b80ff8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:04.656 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.636+0000 7fe2bca3d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2b80fec00 0x7fe2b80ff8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59408/0 (socket says 172.21.15.39:59408) 2023-07-21T18:45:04.656 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.636+0000 7fe2bca3d700 1 -- 172.21.15.39:0/954529320 learned_addr learned my addr 172.21.15.39:0/954529320 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:45:04.656 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.636+0000 7fe2bd23e700 1 --2- 172.21.15.39:0/954529320 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2b80fe260 0x7fe2b80ff3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:04.657 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.636+0000 7fe2bda3f700 1 --2- 172.21.15.39:0/954529320 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2b8104db0 0x7fe2b81a3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:04.657 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.636+0000 7fe2bca3d700 1 -- 172.21.15.39:0/954529320 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2b8104db0 msgr2=0x7fe2b81a3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:45:04.657 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.636+0000 7fe2bca3d700 1 --2- 172.21.15.39:0/954529320 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2b8104db0 0x7fe2b81a3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:04.657 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.636+0000 7fe2bca3d700 1 -- 172.21.15.39:0/954529320 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2b80fe260 msgr2=0x7fe2b80ff3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:45:04.657 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.636+0000 7fe2bca3d700 1 --2- 172.21.15.39:0/954529320 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2b80fe260 0x7fe2b80ff3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:04.657 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.636+0000 7fe2bca3d700 1 -- 172.21.15.39:0/954529320 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2a80095f0 con 0x7fe2b80fec00 2023-07-21T18:45:04.657 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.636+0000 7fe2bda3f700 1 --2- 172.21.15.39:0/954529320 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2b8104db0 0x7fe2b81a3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:45:04.657 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.636+0000 7fe2bca3d700 1 --2- 172.21.15.39:0/954529320 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2b80fec00 0x7fe2b80ff8f0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fe2b400d950 tx=0x7fe2b400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:45:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.636+0000 7fe2ae7fc700 1 -- 172.21.15.39:0/954529320 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b4009930 con 0x7fe2b80fec00 2023-07-21T18:45:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.637+0000 7fe2ae7fc700 1 -- 172.21.15.39:0/954529320 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2b4010460 con 0x7fe2b80fec00 2023-07-21T18:45:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.637+0000 7fe2ae7fc700 1 -- 172.21.15.39:0/954529320 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b400f620 con 0x7fe2b80fec00 2023-07-21T18:45:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.637+0000 7fe2bf4a2700 1 -- 172.21.15.39:0/954529320 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2b81a48d0 con 0x7fe2b80fec00 2023-07-21T18:45:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.637+0000 7fe2bf4a2700 1 -- 172.21.15.39:0/954529320 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2b81a4de0 con 0x7fe2b80fec00 2023-07-21T18:45:04.661 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.639+0000 7fe2ae7fc700 1 -- 172.21.15.39:0/954529320 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fe2b4010860 con 0x7fe2b80fec00 2023-07-21T18:45:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.639+0000 7fe2ae7fc700 1 --2- 172.21.15.39:0/954529320 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fe2a40a1340 0x7fe2a40a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.640+0000 7fe2ae7fc700 1 -- 172.21.15.39:0/954529320 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fe2b40cc8a0 con 0x7fe2b80fec00 2023-07-21T18:45:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.640+0000 7fe2bd23e700 1 --2- 172.21.15.39:0/954529320 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fe2a40a1340 0x7fe2a40a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.640+0000 7fe2bd23e700 1 --2- 172.21.15.39:0/954529320 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fe2a40a1340 0x7fe2a40a37f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8009990 tx=0x7fe2a8003680 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:45:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.641+0000 7fe2cb628b80 1 Processor -- start 2023-07-21T18:45:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.641+0000 7fe2cb628b80 1 -- start start 2023-07-21T18:45:04.663 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.641+0000 7fe2cb628b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d66cbf26b0 0x55d66cd9bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:04.663 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.641+0000 7fe2cb628b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d66cd9c2d0 0x55d66cd96040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:04.663 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.641+0000 7fe2cb628b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d66cd96580 0x55d66cd969d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:04.663 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.641+0000 7fe2cb628b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d66cc9bc50 con 0x55d66cd9c2d0 2023-07-21T18:45:04.663 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.641+0000 7fe2cb628b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d66ccee9e0 con 0x55d66cbf26b0 2023-07-21T18:45:04.663 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.641+0000 7fe2cb628b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d66cce7e70 con 0x55d66cd96580 2023-07-21T18:45:04.663 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.641+0000 7fe2bca3d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d66cd9c2d0 0x55d66cd96040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:04.663 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.641+0000 7fe2bca3d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d66cd9c2d0 0x55d66cd96040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59416/0 (socket says 172.21.15.39:59416) 2023-07-21T18:45:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.641+0000 7fe2bca3d700 1 -- 172.21.15.39:0/1956547569 learned_addr learned my addr 172.21.15.39:0/1956547569 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:45:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.641+0000 7fe2bd23e700 1 --2- 172.21.15.39:0/1956547569 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d66cbf26b0 0x55d66cd9bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.641+0000 7fe2bda3f700 1 --2- 172.21.15.39:0/1956547569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d66cd96580 0x55d66cd969d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.641+0000 7fe2bca3d700 1 -- 172.21.15.39:0/1956547569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d66cd96580 msgr2=0x55d66cd969d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:45:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.641+0000 7fe2bca3d700 1 --2- 172.21.15.39:0/1956547569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d66cd96580 0x55d66cd969d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.641+0000 7fe2bca3d700 1 -- 172.21.15.39:0/1956547569 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d66cbf26b0 msgr2=0x55d66cd9bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:45:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.641+0000 7fe2bca3d700 1 --2- 172.21.15.39:0/1956547569 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d66cbf26b0 0x55d66cd9bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.641+0000 7fe2bca3d700 1 -- 172.21.15.39:0/1956547569 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d66cb15d80 con 0x55d66cd9c2d0 2023-07-21T18:45:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.641+0000 7fe2bda3f700 1 --2- 172.21.15.39:0/1956547569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d66cd96580 0x55d66cd969d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:45:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.641+0000 7fe2bd23e700 1 --2- 172.21.15.39:0/1956547569 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d66cbf26b0 0x55d66cd9bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:45:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.642+0000 7fe2bca3d700 1 --2- 172.21.15.39:0/1956547569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d66cd9c2d0 0x55d66cd96040 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fe2b40963a0 tx=0x7fe2b4099310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a652b49e665be1e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:45:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.642+0000 7fe2a37fe700 1 -- 172.21.15.39:0/1956547569 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b4084070 con 0x55d66cd9c2d0 2023-07-21T18:45:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.642+0000 7fe2a37fe700 1 -- 172.21.15.39:0/1956547569 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2b40a1d80 con 0x55d66cd9c2d0 2023-07-21T18:45:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.642+0000 7fe2a37fe700 1 -- 172.21.15.39:0/1956547569 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b4099080 con 0x55d66cd9c2d0 2023-07-21T18:45:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.642+0000 7fe2cb628b80 1 -- 172.21.15.39:0/1956547569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d66cd9c2d0 msgr2=0x55d66cd96040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:45:04.666 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.642+0000 7fe2cb628b80 1 --2- 172.21.15.39:0/1956547569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d66cd9c2d0 0x55d66cd96040 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fe2b40963a0 tx=0x7fe2b4099310 comp rx=0 tx=0).stop 2023-07-21T18:45:04.666 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.643+0000 7fe2cb628b80 1 -- 172.21.15.39:0/1956547569 shutdown_connections 2023-07-21T18:45:04.666 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.643+0000 7fe2cb628b80 1 --2- 172.21.15.39:0/1956547569 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d66cbf26b0 0x55d66cd9bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:04.666 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.643+0000 7fe2cb628b80 1 --2- 172.21.15.39:0/1956547569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d66cd96580 0x55d66cd969d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:04.666 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.643+0000 7fe2cb628b80 1 --2- 172.21.15.39:0/1956547569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d66cd9c2d0 0x55d66cd96040 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:04.666 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.643+0000 7fe2cb628b80 1 -- 172.21.15.39:0/1956547569 >> 172.21.15.39:0/1956547569 conn(0x55d66cd8bbe0 msgr2=0x55d66cd90f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:45:04.666 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.643+0000 7fe2cb628b80 1 -- 172.21.15.39:0/1956547569 shutdown_connections 2023-07-21T18:45:04.667 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.643+0000 7fe2cb628b80 1 -- 172.21.15.39:0/1956547569 wait complete. 2023-07-21T18:45:04.667 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.643+0000 7fe2cb628b80 1 -- start start 2023-07-21T18:45:04.667 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.643+0000 7fe2cb628b80 1 Processor -- start 2023-07-21T18:45:04.667 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.643+0000 7fe2cb628b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d66cbf26b0 0x55d66cda2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:04.667 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.643+0000 7fe2cb628b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d66cd96580 0x55d66cd9f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:04.667 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.643+0000 7fe2cb628b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d66cd9c2d0 0x55d66cda0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:04.667 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.643+0000 7fe2cb628b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d66cc87a60 con 0x55d66cd96580 2023-07-21T18:45:04.667 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.644+0000 7fe2cb628b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d66cc87760 con 0x55d66cd9c2d0 2023-07-21T18:45:04.668 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.644+0000 7fe2cb628b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d66cb6b8c0 con 0x55d66cbf26b0 2023-07-21T18:45:04.668 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.644+0000 7fe2cb628b80 10 client.0 _ll_register_callbacks cb 0x55d66cc85700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:45:04.668 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.644+0000 7fe2bca3d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d66cd96580 0x55d66cd9f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:04.668 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.644+0000 7fe2bca3d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d66cd96580 0x55d66cd9f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59422/0 (socket says 172.21.15.39:59422) 2023-07-21T18:45:04.668 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.644+0000 7fe2bca3d700 1 -- 172.21.15.39:0/1472485514 learned_addr learned my addr 172.21.15.39:0/1472485514 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:45:04.668 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.644+0000 7fe2bd23e700 1 --2- 172.21.15.39:0/1472485514 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d66cbf26b0 0x55d66cda2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:04.668 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.644+0000 7fe2bda3f700 1 --2- 172.21.15.39:0/1472485514 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d66cd9c2d0 0x55d66cda0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:04.668 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.644+0000 7fe2bca3d700 1 -- 172.21.15.39:0/1472485514 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d66cbf26b0 msgr2=0x55d66cda2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:45:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.644+0000 7fe2bca3d700 1 --2- 172.21.15.39:0/1472485514 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d66cbf26b0 0x55d66cda2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.644+0000 7fe2bca3d700 1 -- 172.21.15.39:0/1472485514 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d66cd9c2d0 msgr2=0x55d66cda0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:45:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.644+0000 7fe2bca3d700 1 --2- 172.21.15.39:0/1472485514 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d66cd9c2d0 0x55d66cda0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.644+0000 7fe2bca3d700 1 -- 172.21.15.39:0/1472485514 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2b81a4de0 con 0x55d66cd96580 2023-07-21T18:45:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.644+0000 7fe2bca3d700 1 --2- 172.21.15.39:0/1472485514 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d66cd96580 0x55d66cd9f910 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4099630 tx=0x7fe2b40c9fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b56dce82a26d4083 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:45:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.644+0000 7fe2a0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:45:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.644+0000 7fe2a0ff9700 1 -- 172.21.15.39:0/1472485514 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b4084040 con 0x55d66cd96580 2023-07-21T18:45:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.644+0000 7fe2a0ff9700 1 -- 172.21.15.39:0/1472485514 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fe2b4099b90 con 0x55d66cd96580 2023-07-21T18:45:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.644+0000 7fe2cb628b80 1 -- 172.21.15.39:0/1472485514 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d66cc41490 con 0x55d66cd96580 2023-07-21T18:45:04.671 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.645+0000 7fe2a0ff9700 1 -- 172.21.15.39:0/1472485514 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe2b4099d50 con 0x55d66cd96580 2023-07-21T18:45:04.672 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.645+0000 7fe2cb628b80 10 client.16365 fetch_fsmap learned FSMap version 35 2023-07-21T18:45:04.672 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.645+0000 7fe2cb628b80 1 -- 172.21.15.39:0/1472485514 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55d66cafec00 con 0x55d66cd96580 2023-07-21T18:45:04.672 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.645+0000 7fe2a0ff9700 1 -- 172.21.15.39:0/1472485514 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d66cc41490 con 0x55d66cd96580 2023-07-21T18:45:04.672 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.645+0000 7fe2a0ff9700 1 -- 172.21.15.39:0/1472485514 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 35) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fe2b40c8050 con 0x55d66cd96580 2023-07-21T18:45:04.672 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.645+0000 7fe2cb628b80 10 client.16365 fetch_fsmap finished waiting for FSMap version 35 2023-07-21T18:45:04.672 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.645+0000 7fe2cb628b80 10 client.16365 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:45:04.672 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.645+0000 7fe2cb628b80 20 client.16365 populate_metadata read hostname 'smithi039' 2023-07-21T18:45:04.672 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.645+0000 7fe2cb628b80 1 --2- 172.21.15.39:0/1472485514 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d66cdaf840 0x55d66cdb1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.645+0000 7fe2cb628b80 4 client.16365 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:45:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.645+0000 7fe2cb628b80 1 -- 172.21.15.39:0/1472485514 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d66cce0e50 con 0x55d66cdaf840 2023-07-21T18:45:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.645+0000 7fe2bd23e700 1 --2- 172.21.15.39:0/1472485514 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d66cdaf840 0x55d66cdb1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.646+0000 7fe2bd23e700 1 --2- 172.21.15.39:0/1472485514 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d66cdaf840 0x55d66cdb1c50 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e3475df937e2093 server_cookie=de393af485b99c9c in_seq=0 out_seq=0 2023-07-21T18:45:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.646+0000 7fe2a0ff9700 10 client.16365 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:45:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.647+0000 7fe2a0ff9700 1 -- 172.21.15.39:0/1472485514 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d66cce0e50 con 0x55d66cdaf840 2023-07-21T18:45:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.647+0000 7fe2a0ff9700 10 client.16365 handle_command_reply: tid=0 2023-07-21T18:45:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.647+0000 7fe2cb628b80 1 client.16365 shutdown 2023-07-21T18:45:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.647+0000 7fe2cb628b80 1 -- 172.21.15.39:0/1472485514 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d66cd96580 msgr2=0x55d66cd9f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:45:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.647+0000 7fe2cb628b80 1 --2- 172.21.15.39:0/1472485514 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d66cd96580 0x55d66cd9f910 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4099630 tx=0x7fe2b40c9fd0 comp rx=0 tx=0).stop 2023-07-21T18:45:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.647+0000 7fe2cb628b80 1 -- 172.21.15.39:0/1472485514 shutdown_connections 2023-07-21T18:45:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.647+0000 7fe2cb628b80 1 --2- 172.21.15.39:0/1472485514 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d66cdaf840 0x55d66cdb1c50 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.647+0000 7fe2cb628b80 1 --2- 172.21.15.39:0/1472485514 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d66cd9c2d0 0x55d66cda0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.647+0000 7fe2cb628b80 1 --2- 172.21.15.39:0/1472485514 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d66cbf26b0 0x55d66cda2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.647+0000 7fe2a0ff9700 0 client.16365 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:45:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.647+0000 7fe2cb628b80 1 --2- 172.21.15.39:0/1472485514 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d66cd96580 0x55d66cd9f910 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.647+0000 7fe2cb628b80 1 -- 172.21.15.39:0/1472485514 >> 172.21.15.39:0/1472485514 conn(0x55d66cd8bbe0 msgr2=0x55d66cd904e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:45:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.647+0000 7fe2cb628b80 1 -- 172.21.15.39:0/1472485514 shutdown_connections 2023-07-21T18:45:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.647+0000 7fe2cb628b80 1 -- 172.21.15.39:0/1472485514 wait complete. 2023-07-21T18:45:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.647+0000 7fe2cb628b80 20 client.16365 trim_cache size 0 max 16384 2023-07-21T18:45:04.680 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.662+0000 7fe2cb628b80 1 Processor -- start 2023-07-21T18:45:04.680 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.662+0000 7fe2cb628b80 1 -- start start 2023-07-21T18:45:04.680 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.662+0000 7fe2cb628b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d66cbf26b0 0x55d66cd96040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:04.680 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.662+0000 7fe2cb628b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d66cd96580 0x55d66cd90c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:04.680 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.662+0000 7fe2cb628b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d66cd9c2d0 0x55d66cd9aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:04.680 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.662+0000 7fe2cb628b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d66cd91d90 con 0x55d66cd9c2d0 2023-07-21T18:45:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.662+0000 7fe2cb628b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d66cd8dd30 con 0x55d66cd96580 2023-07-21T18:45:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.662+0000 7fe2cb628b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d66cd91f70 con 0x55d66cbf26b0 2023-07-21T18:45:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.663+0000 7fe2bda3f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d66cd9c2d0 0x55d66cd9aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.663+0000 7fe2bda3f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d66cd9c2d0 0x55d66cd9aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59432/0 (socket says 172.21.15.39:59432) 2023-07-21T18:45:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.663+0000 7fe2bda3f700 1 -- 172.21.15.39:0/1406633153 learned_addr learned my addr 172.21.15.39:0/1406633153 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:45:04.684 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.663+0000 7fe2bd23e700 1 --2- 172.21.15.39:0/1406633153 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d66cbf26b0 0x55d66cd96040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:04.684 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.663+0000 7fe2bda3f700 1 -- 172.21.15.39:0/1406633153 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d66cbf26b0 msgr2=0x55d66cd96040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:45:04.684 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.663+0000 7fe2bda3f700 1 --2- 172.21.15.39:0/1406633153 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d66cbf26b0 0x55d66cd96040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:04.685 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.663+0000 7fe2bda3f700 1 -- 172.21.15.39:0/1406633153 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d66cd96580 msgr2=0x55d66cd90c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:45:04.685 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.663+0000 7fe2bca3d700 1 --2- 172.21.15.39:0/1406633153 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d66cd96580 0x55d66cd90c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:04.685 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.663+0000 7fe2bda3f700 1 --2- 172.21.15.39:0/1406633153 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d66cd96580 0x55d66cd90c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:04.685 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.663+0000 7fe2bda3f700 1 -- 172.21.15.39:0/1406633153 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d66cb3c1f0 con 0x55d66cd9c2d0 2023-07-21T18:45:04.685 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.663+0000 7fe2bda3f700 1 --2- 172.21.15.39:0/1406633153 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d66cd9c2d0 0x55d66cd9aab0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0009e30 tx=0x7fe2b0007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40ddd9cec696ea53 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:45:04.685 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.663+0000 7fe2a3fff700 1 -- 172.21.15.39:0/1406633153 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b0017470 con 0x55d66cd9c2d0 2023-07-21T18:45:04.685 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.663+0000 7fe2a3fff700 1 -- 172.21.15.39:0/1406633153 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2b000f040 con 0x55d66cd9c2d0 2023-07-21T18:45:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.663+0000 7fe2a3fff700 1 -- 172.21.15.39:0/1406633153 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b0016840 con 0x55d66cd9c2d0 2023-07-21T18:45:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.664+0000 7fe2cb628b80 1 -- 172.21.15.39:0/1406633153 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d66cd9c2d0 msgr2=0x55d66cd9aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:45:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.664+0000 7fe2cb628b80 1 --2- 172.21.15.39:0/1406633153 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d66cd9c2d0 0x55d66cd9aab0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0009e30 tx=0x7fe2b0007700 comp rx=0 tx=0).stop 2023-07-21T18:45:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.664+0000 7fe2cb628b80 1 -- 172.21.15.39:0/1406633153 shutdown_connections 2023-07-21T18:45:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.664+0000 7fe2cb628b80 1 --2- 172.21.15.39:0/1406633153 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d66cd9c2d0 0x55d66cd9aab0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.664+0000 7fe2cb628b80 1 --2- 172.21.15.39:0/1406633153 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d66cbf26b0 0x55d66cd96040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.664+0000 7fe2cb628b80 1 --2- 172.21.15.39:0/1406633153 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d66cd96580 0x55d66cd90c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.664+0000 7fe2cb628b80 1 -- 172.21.15.39:0/1406633153 >> 172.21.15.39:0/1406633153 conn(0x55d66cd8bbe0 msgr2=0x55d66cd99b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:45:04.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.664+0000 7fe2cb628b80 1 -- 172.21.15.39:0/1406633153 shutdown_connections 2023-07-21T18:45:04.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.664+0000 7fe2cb628b80 1 -- 172.21.15.39:0/1406633153 wait complete. 2023-07-21T18:45:04.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.664+0000 7fe2cb628b80 1 -- start start 2023-07-21T18:45:04.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.664+0000 7fe2cb628b80 1 Processor -- start 2023-07-21T18:45:04.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.665+0000 7fe2cb628b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d66cbf26b0 0x55d66cdfc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:04.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.665+0000 7fe2cb628b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d66cd96580 0x55d66cdfce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:04.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.665+0000 7fe2cb628b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d66cd9c2d0 0x55d66cda4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:04.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.665+0000 7fe2cb628b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d66cda5190 con 0x55d66cd96580 2023-07-21T18:45:04.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.665+0000 7fe2cb628b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d66cda52d0 con 0x55d66cd9c2d0 2023-07-21T18:45:04.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.665+0000 7fe2cb628b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d66cda5490 con 0x55d66cbf26b0 2023-07-21T18:45:04.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.665+0000 7fe2cb628b80 10 client.0 _ll_register_callbacks cb 0x7fe2b409d070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:45:04.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.665+0000 7fe2bca3d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d66cd96580 0x55d66cdfce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:04.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.665+0000 7fe2bca3d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d66cd96580 0x55d66cdfce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59436/0 (socket says 172.21.15.39:59436) 2023-07-21T18:45:04.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.665+0000 7fe2bca3d700 1 -- 172.21.15.39:0/4208877437 learned_addr learned my addr 172.21.15.39:0/4208877437 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:45:04.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.665+0000 7fe2bd23e700 1 --2- 172.21.15.39:0/4208877437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d66cbf26b0 0x55d66cdfc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:04.689 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.665+0000 7fe2bda3f700 1 --2- 172.21.15.39:0/4208877437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d66cd9c2d0 0x55d66cda4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:04.689 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.665+0000 7fe2bca3d700 1 -- 172.21.15.39:0/4208877437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d66cbf26b0 msgr2=0x55d66cdfc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:45:04.689 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.665+0000 7fe2bca3d700 1 --2- 172.21.15.39:0/4208877437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d66cbf26b0 0x55d66cdfc8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:04.689 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.665+0000 7fe2bca3d700 1 -- 172.21.15.39:0/4208877437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d66cd9c2d0 msgr2=0x55d66cda4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:45:04.689 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.665+0000 7fe2bca3d700 1 --2- 172.21.15.39:0/4208877437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d66cd9c2d0 0x55d66cda4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:04.689 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.665+0000 7fe2bca3d700 1 -- 172.21.15.39:0/4208877437 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2b0009a60 con 0x55d66cd96580 2023-07-21T18:45:04.689 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.665+0000 7fe2bda3f700 1 --2- 172.21.15.39:0/4208877437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d66cd9c2d0 0x55d66cda4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:45:04.689 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.665+0000 7fe2bca3d700 1 --2- 172.21.15.39:0/4208877437 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d66cd96580 0x55d66cdfce30 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4005530 tx=0x7fe2b4099310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34889e80989a4bd4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:45:04.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.665+0000 7fe2a0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:45:04.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.665+0000 7fe2a0ff9700 1 -- 172.21.15.39:0/4208877437 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b4084040 con 0x55d66cd96580 2023-07-21T18:45:04.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.666+0000 7fe2a0ff9700 1 -- 172.21.15.39:0/4208877437 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fe2b4099d90 con 0x55d66cd96580 2023-07-21T18:45:04.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.666+0000 7fe2cb628b80 1 -- 172.21.15.39:0/4208877437 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe2b40a1d80 con 0x55d66cd96580 2023-07-21T18:45:04.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.666+0000 7fe2a0ff9700 1 -- 172.21.15.39:0/4208877437 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe2b4098890 con 0x55d66cd96580 2023-07-21T18:45:04.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.666+0000 7fe2cb628b80 10 client.16371 fetch_fsmap learned FSMap version 35 2023-07-21T18:45:04.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.666+0000 7fe2cb628b80 1 -- 172.21.15.39:0/4208877437 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55d66cb14250 con 0x55d66cd96580 2023-07-21T18:45:04.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.666+0000 7fe2a0ff9700 1 -- 172.21.15.39:0/4208877437 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2b40a1d80 con 0x55d66cd96580 2023-07-21T18:45:04.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.666+0000 7fe2a0ff9700 1 -- 172.21.15.39:0/4208877437 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 35) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fe2b40c8070 con 0x55d66cd96580 2023-07-21T18:45:04.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.666+0000 7fe2cb628b80 10 client.16371 fetch_fsmap finished waiting for FSMap version 35 2023-07-21T18:45:04.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.666+0000 7fe2cb628b80 10 client.16371 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:45:04.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.666+0000 7fe2cb628b80 20 client.16371 populate_metadata read hostname 'smithi039' 2023-07-21T18:45:04.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.666+0000 7fe2cb628b80 1 --2- 172.21.15.39:0/4208877437 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d66ce04190 0x55d66ce06570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:04.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.666+0000 7fe2cb628b80 4 client.16371 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:45:04.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.666+0000 7fe2cb628b80 1 -- 172.21.15.39:0/4208877437 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d66cca2ff0 con 0x55d66ce04190 2023-07-21T18:45:04.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.666+0000 7fe2bd23e700 1 --2- 172.21.15.39:0/4208877437 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d66ce04190 0x55d66ce06570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:04.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.667+0000 7fe2bd23e700 1 --2- 172.21.15.39:0/4208877437 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d66ce04190 0x55d66ce06570 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd58c75dc9b1dca6 server_cookie=f266ed9a3d5a88f1 in_seq=0 out_seq=0 2023-07-21T18:45:04.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.667+0000 7fe2a0ff9700 10 client.16371 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:45:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.705+0000 7fe2a0ff9700 1 -- 172.21.15.39:0/4208877437 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d66cca2ff0 con 0x55d66ce04190 2023-07-21T18:45:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.705+0000 7fe2a0ff9700 10 client.16371 handle_command_reply: tid=0 2023-07-21T18:45:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.705+0000 7fe2cb628b80 1 client.16371 shutdown 2023-07-21T18:45:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.705+0000 7fe2cb628b80 1 -- 172.21.15.39:0/4208877437 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d66cd96580 msgr2=0x55d66cdfce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:45:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.705+0000 7fe2cb628b80 1 --2- 172.21.15.39:0/4208877437 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d66cd96580 0x55d66cdfce30 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4005530 tx=0x7fe2b4099310 comp rx=0 tx=0).stop 2023-07-21T18:45:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.705+0000 7fe2cb628b80 1 -- 172.21.15.39:0/4208877437 shutdown_connections 2023-07-21T18:45:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.705+0000 7fe2cb628b80 1 --2- 172.21.15.39:0/4208877437 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d66ce04190 0x55d66ce06570 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:04.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.705+0000 7fe2cb628b80 1 --2- 172.21.15.39:0/4208877437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d66cd9c2d0 0x55d66cda4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:04.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.705+0000 7fe2cb628b80 1 --2- 172.21.15.39:0/4208877437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d66cbf26b0 0x55d66cdfc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:04.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.705+0000 7fe2cb628b80 1 --2- 172.21.15.39:0/4208877437 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d66cd96580 0x55d66cdfce30 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:04.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.705+0000 7fe2cb628b80 1 -- 172.21.15.39:0/4208877437 >> 172.21.15.39:0/4208877437 conn(0x55d66cd8bbe0 msgr2=0x55d66cd9aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:45:04.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.705+0000 7fe2a0ff9700 0 client.16371 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:45:04.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.705+0000 7fe2cb628b80 1 -- 172.21.15.39:0/4208877437 shutdown_connections 2023-07-21T18:45:04.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.705+0000 7fe2cb628b80 1 -- 172.21.15.39:0/4208877437 wait complete. 2023-07-21T18:45:04.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.705+0000 7fe2cb628b80 20 client.16371 trim_cache size 0 max 16384 2023-07-21T18:45:04.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:45:04.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:45:04.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:45:04.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:45:04.726 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.706+0000 7fe2bf4a2700 1 -- 172.21.15.39:0/954529320 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fe2a40a1340 msgr2=0x7fe2a40a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:45:04.726 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.706+0000 7fe2bf4a2700 1 --2- 172.21.15.39:0/954529320 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fe2a40a1340 0x7fe2a40a37f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8009990 tx=0x7fe2a8003680 comp rx=0 tx=0).stop 2023-07-21T18:45:04.726 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.706+0000 7fe2bf4a2700 1 -- 172.21.15.39:0/954529320 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2b80fec00 msgr2=0x7fe2b80ff8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:45:04.726 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.706+0000 7fe2bf4a2700 1 --2- 172.21.15.39:0/954529320 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2b80fec00 0x7fe2b80ff8f0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fe2b400d950 tx=0x7fe2b400dc60 comp rx=0 tx=0).stop 2023-07-21T18:45:04.726 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.706+0000 7fe2bf4a2700 1 -- 172.21.15.39:0/954529320 shutdown_connections 2023-07-21T18:45:04.727 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.706+0000 7fe2bf4a2700 1 --2- 172.21.15.39:0/954529320 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fe2a40a1340 0x7fe2a40a37f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:04.727 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.706+0000 7fe2bf4a2700 1 --2- 172.21.15.39:0/954529320 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2b80fe260 0x7fe2b80ff3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:04.727 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.706+0000 7fe2bf4a2700 1 --2- 172.21.15.39:0/954529320 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2b8104db0 0x7fe2b81a3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:04.727 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.706+0000 7fe2bf4a2700 1 --2- 172.21.15.39:0/954529320 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2b80fec00 0x7fe2b80ff8f0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:04.727 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.706+0000 7fe2bf4a2700 1 -- 172.21.15.39:0/954529320 >> 172.21.15.39:0/954529320 conn(0x7fe2b8075f90 msgr2=0x7fe2b80fbbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:45:04.727 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.706+0000 7fe2bf4a2700 1 -- 172.21.15.39:0/954529320 shutdown_connections 2023-07-21T18:45:04.727 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:04.706+0000 7fe2bf4a2700 1 -- 172.21.15.39:0/954529320 wait complete. 2023-07-21T18:45:05.267 INFO:tasks.workunit.client.0.smithi039.stdout: 1 163257 3.10 MB/sec execute 18 sec latency 2365.941 ms 2023-07-21T18:45:06.267 INFO:tasks.workunit.client.0.smithi039.stdout: 1 163273 2.94 MB/sec execute 19 sec latency 3182.410 ms 2023-07-21T18:45:06.433 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:45:06 smithi039 ceph-mon[96489]: pgmap v103: 97 pgs: 97 active+clean; 74 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 1.1 MiB/s wr, 115 op/s 2023-07-21T18:45:06.615 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:45:06 smithi193 ceph-mon[98204]: pgmap v103: 97 pgs: 97 active+clean; 74 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 1.1 MiB/s wr, 115 op/s 2023-07-21T18:45:06.634 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:45:06 smithi155 ceph-mon[97934]: pgmap v103: 97 pgs: 97 active+clean; 74 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 1.1 MiB/s wr, 115 op/s 2023-07-21T18:45:07.267 INFO:tasks.workunit.client.0.smithi039.stdout: 1 163273 2.79 MB/sec execute 20 sec latency 1163.015 ms 2023-07-21T18:45:08.268 INFO:tasks.workunit.client.0.smithi039.stdout: 1 163273 2.66 MB/sec execute 21 sec latency 2163.553 ms 2023-07-21T18:45:08.432 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:45:08 smithi039 ceph-mon[96489]: pgmap v104: 97 pgs: 97 active+clean; 66 MiB data, 900 MiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 2.1 MiB/s wr, 215 op/s 2023-07-21T18:45:08.615 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:45:08 smithi193 ceph-mon[98204]: pgmap v104: 97 pgs: 97 active+clean; 66 MiB data, 900 MiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 2.1 MiB/s wr, 215 op/s 2023-07-21T18:45:08.634 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:45:08 smithi155 ceph-mon[97934]: pgmap v104: 97 pgs: 97 active+clean; 66 MiB data, 900 MiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 2.1 MiB/s wr, 215 op/s 2023-07-21T18:45:09.268 INFO:tasks.workunit.client.0.smithi039.stdout: 1 163273 2.54 MB/sec execute 22 sec latency 3163.633 ms 2023-07-21T18:45:10.268 INFO:tasks.workunit.client.0.smithi039.stdout: 1 163273 2.43 MB/sec execute 23 sec latency 4163.673 ms 2023-07-21T18:45:10.433 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:45:10 smithi039 ceph-mon[96489]: pgmap v105: 97 pgs: 97 active+clean; 66 MiB data, 900 MiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 1.1 MiB/s wr, 124 op/s 2023-07-21T18:45:10.615 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:45:10 smithi193 ceph-mon[98204]: pgmap v105: 97 pgs: 97 active+clean; 66 MiB data, 900 MiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 1.1 MiB/s wr, 124 op/s 2023-07-21T18:45:10.635 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:45:10 smithi155 ceph-mon[97934]: pgmap v105: 97 pgs: 97 active+clean; 66 MiB data, 900 MiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 1.1 MiB/s wr, 124 op/s 2023-07-21T18:45:11.268 INFO:tasks.workunit.client.0.smithi039.stdout: 1 163439 2.38 MB/sec execute 24 sec latency 4976.635 ms 2023-07-21T18:45:12.268 INFO:tasks.workunit.client.0.smithi039.stdout: 1 165246 2.52 MB/sec execute 25 sec latency 18.046 ms 2023-07-21T18:45:12.432 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:45:12 smithi039 ceph-mon[96489]: pgmap v106: 97 pgs: 97 active+clean; 70 MiB data, 908 MiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 1.9 MiB/s wr, 156 op/s 2023-07-21T18:45:12.615 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:45:12 smithi193 ceph-mon[98204]: pgmap v106: 97 pgs: 97 active+clean; 70 MiB data, 908 MiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 1.9 MiB/s wr, 156 op/s 2023-07-21T18:45:12.634 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:45:12 smithi155 ceph-mon[97934]: pgmap v106: 97 pgs: 97 active+clean; 70 MiB data, 908 MiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 1.9 MiB/s wr, 156 op/s 2023-07-21T18:45:13.268 INFO:tasks.workunit.client.0.smithi039.stdout: 1 166735 2.88 MB/sec execute 26 sec latency 353.989 ms 2023-07-21T18:45:14.268 INFO:tasks.workunit.client.0.smithi039.stdout: 1 166735 2.77 MB/sec execute 27 sec latency 1354.039 ms 2023-07-21T18:45:14.433 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:45:14 smithi039 ceph-mon[96489]: pgmap v107: 97 pgs: 97 active+clean; 70 MiB data, 908 MiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 1.7 MiB/s wr, 132 op/s 2023-07-21T18:45:14.615 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:45:14 smithi193 ceph-mon[98204]: pgmap v107: 97 pgs: 97 active+clean; 70 MiB data, 908 MiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 1.7 MiB/s wr, 132 op/s 2023-07-21T18:45:14.634 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:45:14 smithi155 ceph-mon[97934]: pgmap v107: 97 pgs: 97 active+clean; 70 MiB data, 908 MiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 1.7 MiB/s wr, 132 op/s 2023-07-21T18:45:15.268 INFO:tasks.workunit.client.0.smithi039.stdout: 1 166735 2.67 MB/sec execute 28 sec latency 2354.078 ms 2023-07-21T18:45:16.268 INFO:tasks.workunit.client.0.smithi039.stdout: 1 166951 2.63 MB/sec execute 29 sec latency 3172.889 ms 2023-07-21T18:45:16.615 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:45:16 smithi193 ceph-mon[98204]: pgmap v108: 97 pgs: 97 active+clean; 70 MiB data, 908 MiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 1.7 MiB/s wr, 132 op/s 2023-07-21T18:45:16.633 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:45:16 smithi155 ceph-mon[97934]: pgmap v108: 97 pgs: 97 active+clean; 70 MiB data, 908 MiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 1.7 MiB/s wr, 132 op/s 2023-07-21T18:45:16.682 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:45:16 smithi039 ceph-mon[96489]: pgmap v108: 97 pgs: 97 active+clean; 70 MiB data, 908 MiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 1.7 MiB/s wr, 132 op/s 2023-07-21T18:45:17.268 INFO:tasks.workunit.client.0.smithi039.stdout: 1 168370 2.72 MB/sec execute 30 sec latency 22.606 ms 2023-07-21T18:45:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.817+0000 7fc615ea5700 1 -- 172.21.15.39:0/2350343099 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6100fea30 msgr2=0x7fc6100feea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:45:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.817+0000 7fc615ea5700 1 --2- 172.21.15.39:0/2350343099 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6100fea30 0x7fc6100feea0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8009990 tx=0x7fc5f8009ca0 comp rx=0 tx=0).stop 2023-07-21T18:45:17.836 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.817+0000 7fc615ea5700 1 -- 172.21.15.39:0/2350343099 shutdown_connections 2023-07-21T18:45:17.836 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.817+0000 7fc615ea5700 1 --2- 172.21.15.39:0/2350343099 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6100ff3e0 0x7fc61010b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:17.836 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.817+0000 7fc615ea5700 1 --2- 172.21.15.39:0/2350343099 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc610104a30 0x7fc610104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:17.836 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.817+0000 7fc615ea5700 1 --2- 172.21.15.39:0/2350343099 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6100fea30 0x7fc6100feea0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:17.836 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.817+0000 7fc615ea5700 1 -- 172.21.15.39:0/2350343099 >> 172.21.15.39:0/2350343099 conn(0x7fc610074760 msgr2=0x7fc610074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:45:17.837 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.817+0000 7fc615ea5700 1 -- 172.21.15.39:0/2350343099 shutdown_connections 2023-07-21T18:45:17.837 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.817+0000 7fc615ea5700 1 -- 172.21.15.39:0/2350343099 wait complete. 2023-07-21T18:45:17.838 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.818+0000 7fc615ea5700 1 Processor -- start 2023-07-21T18:45:17.839 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.818+0000 7fc615ea5700 1 -- start start 2023-07-21T18:45:17.839 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.818+0000 7fc615ea5700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc6100fea30 0x7fc61019aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:17.839 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.818+0000 7fc615ea5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6100ff3e0 0x7fc61019aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:17.839 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.818+0000 7fc615ea5700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc610104a30 0x7fc610194910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:17.839 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.818+0000 7fc615ea5700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc610194f80 con 0x7fc6100ff3e0 2023-07-21T18:45:17.839 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.818+0000 7fc615ea5700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6101950f0 con 0x7fc610104a30 2023-07-21T18:45:17.840 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.818+0000 7fc615ea5700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6101952b0 con 0x7fc6100fea30 2023-07-21T18:45:17.840 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.819+0000 7fc60effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6100ff3e0 0x7fc61019aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:17.840 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.819+0000 7fc60effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6100ff3e0 0x7fc61019aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59450/0 (socket says 172.21.15.39:59450) 2023-07-21T18:45:17.840 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.819+0000 7fc60effd700 1 -- 172.21.15.39:0/2838870126 learned_addr learned my addr 172.21.15.39:0/2838870126 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:45:17.840 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.819+0000 7fc60ffff700 1 --2- 172.21.15.39:0/2838870126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc610104a30 0x7fc610194910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:17.840 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.819+0000 7fc60f7fe700 1 --2- 172.21.15.39:0/2838870126 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc6100fea30 0x7fc61019aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:17.840 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.819+0000 7fc60effd700 1 -- 172.21.15.39:0/2838870126 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc6100fea30 msgr2=0x7fc61019aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:45:17.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.819+0000 7fc60effd700 1 --2- 172.21.15.39:0/2838870126 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc6100fea30 0x7fc61019aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:17.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.819+0000 7fc60effd700 1 -- 172.21.15.39:0/2838870126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc610104a30 msgr2=0x7fc610194910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:45:17.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.819+0000 7fc60effd700 1 --2- 172.21.15.39:0/2838870126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc610104a30 0x7fc610194910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:17.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.819+0000 7fc60effd700 1 -- 172.21.15.39:0/2838870126 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5f80095f0 con 0x7fc6100ff3e0 2023-07-21T18:45:17.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.819+0000 7fc60effd700 1 --2- 172.21.15.39:0/2838870126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6100ff3e0 0x7fc61019aff0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8005910 tx=0x7fc5f8004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:45:17.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.819+0000 7fc60cff9700 1 -- 172.21.15.39:0/2838870126 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f801d070 con 0x7fc6100ff3e0 2023-07-21T18:45:17.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.819+0000 7fc60cff9700 1 -- 172.21.15.39:0/2838870126 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5f8022470 con 0x7fc6100ff3e0 2023-07-21T18:45:17.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.819+0000 7fc615ea5700 1 -- 172.21.15.39:0/2838870126 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc610195530 con 0x7fc6100ff3e0 2023-07-21T18:45:17.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.819+0000 7fc60cff9700 1 -- 172.21.15.39:0/2838870126 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f8017650 con 0x7fc6100ff3e0 2023-07-21T18:45:17.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.819+0000 7fc615ea5700 1 -- 172.21.15.39:0/2838870126 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6101a1aa0 con 0x7fc6100ff3e0 2023-07-21T18:45:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.822+0000 7fc60cff9700 1 -- 172.21.15.39:0/2838870126 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fc5f80225e0 con 0x7fc6100ff3e0 2023-07-21T18:45:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.822+0000 7fc60cff9700 1 --2- 172.21.15.39:0/2838870126 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fc5fc0a12c0 0x7fc5fc0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.822+0000 7fc60cff9700 1 -- 172.21.15.39:0/2838870126 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fc5f80cdf10 con 0x7fc6100ff3e0 2023-07-21T18:45:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.822+0000 7fc60f7fe700 1 --2- 172.21.15.39:0/2838870126 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fc5fc0a12c0 0x7fc5fc0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.823+0000 7fc60f7fe700 1 --2- 172.21.15.39:0/2838870126 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fc5fc0a12c0 0x7fc5fc0a3770 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc6100ffba0 tx=0x7fc608009260 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:45:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.823+0000 7fc62202bb80 1 Processor -- start 2023-07-21T18:45:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.823+0000 7fc62202bb80 1 -- start start 2023-07-21T18:45:17.847 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.823+0000 7fc62202bb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ed0d8286b0 0x55ed0d9d3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:17.847 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.823+0000 7fc62202bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed0d9d44a0 0x55ed0d9ce250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:17.847 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.823+0000 7fc62202bb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ed0d9d4910 0x55ed0d9ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:17.847 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.823+0000 7fc62202bb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed0d8d1c50 con 0x55ed0d8286b0 2023-07-21T18:45:17.847 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.823+0000 7fc62202bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed0d9249e0 con 0x55ed0d9d44a0 2023-07-21T18:45:17.848 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.823+0000 7fc62202bb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed0d91de70 con 0x55ed0d9d4910 2023-07-21T18:45:17.848 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.823+0000 7fc60f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ed0d8286b0 0x55ed0d9d3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:17.848 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.823+0000 7fc60f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ed0d8286b0 0x55ed0d9d3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59456/0 (socket says 172.21.15.39:59456) 2023-07-21T18:45:17.848 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.823+0000 7fc60effd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed0d9d44a0 0x55ed0d9ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:17.848 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.823+0000 7fc60f7fe700 1 -- 172.21.15.39:0/690038304 learned_addr learned my addr 172.21.15.39:0/690038304 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:45:17.848 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.824+0000 7fc60ffff700 1 --2- 172.21.15.39:0/690038304 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ed0d9d4910 0x55ed0d9ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.824+0000 7fc60f7fe700 1 -- 172.21.15.39:0/690038304 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ed0d9d4910 msgr2=0x55ed0d9ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:45:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.824+0000 7fc60f7fe700 1 --2- 172.21.15.39:0/690038304 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ed0d9d4910 0x55ed0d9ce7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.824+0000 7fc60f7fe700 1 -- 172.21.15.39:0/690038304 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed0d9d44a0 msgr2=0x55ed0d9ce250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:45:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.824+0000 7fc60f7fe700 1 --2- 172.21.15.39:0/690038304 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed0d9d44a0 0x55ed0d9ce250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.824+0000 7fc60f7fe700 1 -- 172.21.15.39:0/690038304 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed0d74bd80 con 0x55ed0d8286b0 2023-07-21T18:45:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.824+0000 7fc60effd700 1 --2- 172.21.15.39:0/690038304 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed0d9d44a0 0x55ed0d9ce250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:45:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.824+0000 7fc60ffff700 1 --2- 172.21.15.39:0/690038304 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ed0d9d4910 0x55ed0d9ce7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:45:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.824+0000 7fc60f7fe700 1 --2- 172.21.15.39:0/690038304 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ed0d8286b0 0x55ed0d9d3f60 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fc608011d50 tx=0x7fc608014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2e51e7934d66470 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:45:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.824+0000 7fc605ffb700 1 -- 172.21.15.39:0/690038304 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc60800ed20 con 0x55ed0d8286b0 2023-07-21T18:45:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.824+0000 7fc605ffb700 1 -- 172.21.15.39:0/690038304 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc608021070 con 0x55ed0d8286b0 2023-07-21T18:45:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.824+0000 7fc605ffb700 1 -- 172.21.15.39:0/690038304 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc60801d410 con 0x55ed0d8286b0 2023-07-21T18:45:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.825+0000 7fc62202bb80 1 -- 172.21.15.39:0/690038304 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ed0d8286b0 msgr2=0x55ed0d9d3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:45:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.825+0000 7fc62202bb80 1 --2- 172.21.15.39:0/690038304 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ed0d8286b0 0x55ed0d9d3f60 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fc608011d50 tx=0x7fc608014000 comp rx=0 tx=0).stop 2023-07-21T18:45:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.825+0000 7fc62202bb80 1 -- 172.21.15.39:0/690038304 shutdown_connections 2023-07-21T18:45:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.825+0000 7fc62202bb80 1 --2- 172.21.15.39:0/690038304 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ed0d9d4910 0x55ed0d9ce7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.825+0000 7fc62202bb80 1 --2- 172.21.15.39:0/690038304 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ed0d8286b0 0x55ed0d9d3f60 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.825+0000 7fc62202bb80 1 --2- 172.21.15.39:0/690038304 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed0d9d44a0 0x55ed0d9ce250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.825+0000 7fc62202bb80 1 -- 172.21.15.39:0/690038304 >> 172.21.15.39:0/690038304 conn(0x55ed0d9c1be0 msgr2=0x55ed0d9c6f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:45:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.825+0000 7fc62202bb80 1 -- 172.21.15.39:0/690038304 shutdown_connections 2023-07-21T18:45:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.825+0000 7fc62202bb80 1 -- 172.21.15.39:0/690038304 wait complete. 2023-07-21T18:45:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.825+0000 7fc62202bb80 1 -- start start 2023-07-21T18:45:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.825+0000 7fc62202bb80 1 Processor -- start 2023-07-21T18:45:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.826+0000 7fc62202bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed0d8286b0 0x55ed0d9da740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.826+0000 7fc62202bb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ed0d9d44a0 0x55ed0d9d7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.826+0000 7fc62202bb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ed0d9d4910 0x55ed0d9d8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.826+0000 7fc62202bb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed0d8bda60 con 0x55ed0d9d4910 2023-07-21T18:45:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.826+0000 7fc62202bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed0d8bd760 con 0x55ed0d8286b0 2023-07-21T18:45:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.826+0000 7fc62202bb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed0d7a18c0 con 0x55ed0d9d44a0 2023-07-21T18:45:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.826+0000 7fc62202bb80 10 client.0 _ll_register_callbacks cb 0x55ed0d8bb700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:45:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.826+0000 7fc60f7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed0d8286b0 0x55ed0d9da740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.826+0000 7fc60effd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ed0d9d44a0 0x55ed0d9d7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.826+0000 7fc60ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ed0d9d4910 0x55ed0d9d8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.826+0000 7fc60f7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed0d8286b0 0x55ed0d9da740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:43826/0 (socket says 172.21.15.39:43826) 2023-07-21T18:45:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.826+0000 7fc60effd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ed0d9d44a0 0x55ed0d9d7a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:49784/0 (socket says 172.21.15.39:49784) 2023-07-21T18:45:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.826+0000 7fc60f7fe700 1 -- 172.21.15.39:0/998109734 learned_addr learned my addr 172.21.15.39:0/998109734 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:45:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.826+0000 7fc60f7fe700 1 -- 172.21.15.39:0/998109734 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ed0d9d44a0 msgr2=0x55ed0d9d7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:45:17.854 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.826+0000 7fc60f7fe700 1 --2- 172.21.15.39:0/998109734 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ed0d9d44a0 0x55ed0d9d7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:17.854 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.826+0000 7fc60f7fe700 1 -- 172.21.15.39:0/998109734 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ed0d9d4910 msgr2=0x55ed0d9d8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:45:17.854 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.826+0000 7fc60f7fe700 1 --2- 172.21.15.39:0/998109734 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ed0d9d4910 0x55ed0d9d8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:17.854 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.826+0000 7fc60f7fe700 1 -- 172.21.15.39:0/998109734 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6101a1aa0 con 0x55ed0d8286b0 2023-07-21T18:45:17.854 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.826+0000 7fc60f7fe700 1 --2- 172.21.15.39:0/998109734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed0d8286b0 0x55ed0d9da740 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc608011b30 tx=0x7fc60801ce00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2ba82dafae2986a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:45:17.854 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.826+0000 7fc5eeffd700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-21T18:45:17.854 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.826+0000 7fc5eeffd700 1 -- 172.21.15.39:0/998109734 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc60800ed30 con 0x55ed0d8286b0 2023-07-21T18:45:17.854 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.826+0000 7fc62202bb80 1 -- 172.21.15.39:0/998109734 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ed0d877490 con 0x55ed0d8286b0 2023-07-21T18:45:17.855 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.827+0000 7fc5eeffd700 1 -- 172.21.15.39:0/998109734 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fc60801c410 con 0x55ed0d8286b0 2023-07-21T18:45:17.855 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.827+0000 7fc5eeffd700 1 -- 172.21.15.39:0/998109734 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc60801c5d0 con 0x55ed0d8286b0 2023-07-21T18:45:17.855 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.827+0000 7fc62202bb80 10 client.25877 fetch_fsmap learned FSMap version 35 2023-07-21T18:45:17.855 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.827+0000 7fc62202bb80 1 -- 172.21.15.39:0/998109734 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x7fc608017070 con 0x55ed0d8286b0 2023-07-21T18:45:17.855 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.827+0000 7fc5eeffd700 1 -- 172.21.15.39:0/998109734 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ed0d877490 con 0x55ed0d8286b0 2023-07-21T18:45:17.855 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.827+0000 7fc5eeffd700 1 -- 172.21.15.39:0/998109734 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 35) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fc608004780 con 0x55ed0d8286b0 2023-07-21T18:45:17.855 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.827+0000 7fc62202bb80 10 client.25877 fetch_fsmap finished waiting for FSMap version 35 2023-07-21T18:45:17.856 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.827+0000 7fc62202bb80 10 client.25877 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:45:17.856 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.827+0000 7fc62202bb80 20 client.25877 populate_metadata read hostname 'smithi039' 2023-07-21T18:45:17.856 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.827+0000 7fc62202bb80 1 --2- 172.21.15.39:0/998109734 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55ed0d9e7ca0 0x55ed0d9ea0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:17.856 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.827+0000 7fc62202bb80 4 client.25877 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:45:17.856 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.827+0000 7fc62202bb80 1 -- 172.21.15.39:0/998109734 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ed0d916e50 con 0x55ed0d9e7ca0 2023-07-21T18:45:17.856 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.827+0000 7fc60effd700 1 --2- 172.21.15.39:0/998109734 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55ed0d9e7ca0 0x55ed0d9ea0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:17.856 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.828+0000 7fc5eeffd700 10 client.25877 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:45:17.856 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.828+0000 7fc60effd700 1 --2- 172.21.15.39:0/998109734 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55ed0d9e7ca0 0x55ed0d9ea0b0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a69131e8728d11e server_cookie=356ff554bce823af in_seq=0 out_seq=0 2023-07-21T18:45:17.857 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.828+0000 7fc5eeffd700 1 -- 172.21.15.39:0/998109734 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ed0d916e50 con 0x55ed0d9e7ca0 2023-07-21T18:45:17.858 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.828+0000 7fc5eeffd700 10 client.25877 handle_command_reply: tid=0 2023-07-21T18:45:17.858 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.828+0000 7fc62202bb80 1 client.25877 shutdown 2023-07-21T18:45:17.858 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.828+0000 7fc62202bb80 1 -- 172.21.15.39:0/998109734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed0d8286b0 msgr2=0x55ed0d9da740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:45:17.858 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.828+0000 7fc62202bb80 1 --2- 172.21.15.39:0/998109734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed0d8286b0 0x55ed0d9da740 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc608011b30 tx=0x7fc60801ce00 comp rx=0 tx=0).stop 2023-07-21T18:45:17.859 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.829+0000 7fc62202bb80 1 -- 172.21.15.39:0/998109734 shutdown_connections 2023-07-21T18:45:17.859 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.829+0000 7fc62202bb80 1 --2- 172.21.15.39:0/998109734 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55ed0d9e7ca0 0x55ed0d9ea0b0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:17.859 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.829+0000 7fc62202bb80 1 --2- 172.21.15.39:0/998109734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed0d8286b0 0x55ed0d9da740 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:17.859 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.829+0000 7fc62202bb80 1 --2- 172.21.15.39:0/998109734 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ed0d9d4910 0x55ed0d9d8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:17.859 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.829+0000 7fc62202bb80 1 --2- 172.21.15.39:0/998109734 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ed0d9d44a0 0x55ed0d9d7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:17.859 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.829+0000 7fc5eeffd700 0 client.25877 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:45:17.859 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.829+0000 7fc62202bb80 1 -- 172.21.15.39:0/998109734 >> 172.21.15.39:0/998109734 conn(0x55ed0d9c1be0 msgr2=0x55ed0d9c64e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:45:17.860 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.829+0000 7fc62202bb80 1 -- 172.21.15.39:0/998109734 shutdown_connections 2023-07-21T18:45:17.860 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.829+0000 7fc62202bb80 1 -- 172.21.15.39:0/998109734 wait complete. 2023-07-21T18:45:17.860 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.829+0000 7fc62202bb80 20 client.25877 trim_cache size 0 max 16384 2023-07-21T18:45:17.864 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.845+0000 7fc62202bb80 1 Processor -- start 2023-07-21T18:45:17.864 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.845+0000 7fc62202bb80 1 -- start start 2023-07-21T18:45:17.864 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.845+0000 7fc62202bb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ed0d8286b0 0x55ed0d9deed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:17.864 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.845+0000 7fc62202bb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ed0d9d44a0 0x55ed0d9d32b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:17.864 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.845+0000 7fc62202bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed0d9d4910 0x55ed0d9c6c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.845+0000 7fc62202bb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed0d9e3d90 con 0x55ed0d8286b0 2023-07-21T18:45:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.845+0000 7fc62202bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed0d9d2790 con 0x55ed0d9d4910 2023-07-21T18:45:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.845+0000 7fc62202bb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed0d9c7a00 con 0x55ed0d9d44a0 2023-07-21T18:45:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.846+0000 7fc60f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ed0d8286b0 0x55ed0d9deed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.846+0000 7fc60f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ed0d8286b0 0x55ed0d9deed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59470/0 (socket says 172.21.15.39:59470) 2023-07-21T18:45:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.846+0000 7fc60effd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ed0d9d44a0 0x55ed0d9d32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.846+0000 7fc60f7fe700 1 -- 172.21.15.39:0/706367983 learned_addr learned my addr 172.21.15.39:0/706367983 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:45:17.866 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.846+0000 7fc60ffff700 1 --2- 172.21.15.39:0/706367983 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed0d9d4910 0x55ed0d9c6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:17.866 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.846+0000 7fc60f7fe700 1 -- 172.21.15.39:0/706367983 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ed0d9d44a0 msgr2=0x55ed0d9d32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:45:17.866 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.846+0000 7fc60f7fe700 1 --2- 172.21.15.39:0/706367983 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ed0d9d44a0 0x55ed0d9d32b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:17.866 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.846+0000 7fc60f7fe700 1 -- 172.21.15.39:0/706367983 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed0d9d4910 msgr2=0x55ed0d9c6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:45:17.866 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.846+0000 7fc60f7fe700 1 --2- 172.21.15.39:0/706367983 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed0d9d4910 0x55ed0d9c6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:17.866 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.846+0000 7fc60f7fe700 1 -- 172.21.15.39:0/706367983 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed0d9eeca0 con 0x55ed0d8286b0 2023-07-21T18:45:17.866 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.846+0000 7fc60effd700 1 --2- 172.21.15.39:0/706367983 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ed0d9d44a0 0x55ed0d9d32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:45:17.869 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.846+0000 7fc60f7fe700 1 --2- 172.21.15.39:0/706367983 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ed0d8286b0 0x55ed0d9deed0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fc608016040 tx=0x7fc60801ce70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e03a7b86be33b74 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:45:17.870 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.846+0000 7fc6067fc700 1 -- 172.21.15.39:0/706367983 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc60800edb0 con 0x55ed0d8286b0 2023-07-21T18:45:17.870 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.846+0000 7fc6067fc700 1 -- 172.21.15.39:0/706367983 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc6080044e0 con 0x55ed0d8286b0 2023-07-21T18:45:17.870 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.846+0000 7fc6067fc700 1 -- 172.21.15.39:0/706367983 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc60801c700 con 0x55ed0d8286b0 2023-07-21T18:45:17.870 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.847+0000 7fc62202bb80 1 -- 172.21.15.39:0/706367983 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ed0d8286b0 msgr2=0x55ed0d9deed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:45:17.870 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.847+0000 7fc62202bb80 1 --2- 172.21.15.39:0/706367983 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ed0d8286b0 0x55ed0d9deed0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fc608016040 tx=0x7fc60801ce70 comp rx=0 tx=0).stop 2023-07-21T18:45:17.870 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.847+0000 7fc62202bb80 1 -- 172.21.15.39:0/706367983 shutdown_connections 2023-07-21T18:45:17.870 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.847+0000 7fc62202bb80 1 --2- 172.21.15.39:0/706367983 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed0d9d4910 0x55ed0d9c6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.847+0000 7fc62202bb80 1 --2- 172.21.15.39:0/706367983 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ed0d8286b0 0x55ed0d9deed0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.847+0000 7fc62202bb80 1 --2- 172.21.15.39:0/706367983 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ed0d9d44a0 0x55ed0d9d32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.847+0000 7fc62202bb80 1 -- 172.21.15.39:0/706367983 >> 172.21.15.39:0/706367983 conn(0x55ed0d9c1be0 msgr2=0x55ed0d9d2ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:45:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.847+0000 7fc62202bb80 1 -- 172.21.15.39:0/706367983 shutdown_connections 2023-07-21T18:45:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.847+0000 7fc62202bb80 1 -- 172.21.15.39:0/706367983 wait complete. 2023-07-21T18:45:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.847+0000 7fc62202bb80 1 -- start start 2023-07-21T18:45:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.848+0000 7fc62202bb80 1 Processor -- start 2023-07-21T18:45:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.848+0000 7fc62202bb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ed0d8286b0 0x55ed0da2fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:17.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.848+0000 7fc62202bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed0d9d44a0 0x55ed0da2fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:17.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.848+0000 7fc62202bb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ed0d9d4910 0x55ed0da3ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:17.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.848+0000 7fc62202bb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed0da30860 con 0x55ed0d9d4910 2023-07-21T18:45:17.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.848+0000 7fc62202bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed0da3b070 con 0x55ed0d9d44a0 2023-07-21T18:45:17.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.848+0000 7fc62202bb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed0da3b1b0 con 0x55ed0d8286b0 2023-07-21T18:45:17.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.848+0000 7fc62202bb80 10 client.0 _ll_register_callbacks cb 0x7fc5f8009290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:45:17.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.848+0000 7fc60effd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed0d9d44a0 0x55ed0da2fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.848+0000 7fc60ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ed0d9d4910 0x55ed0da3ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.848+0000 7fc60effd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed0d9d44a0 0x55ed0da2fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:43842/0 (socket says 172.21.15.39:43842) 2023-07-21T18:45:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.848+0000 7fc60effd700 1 -- 172.21.15.39:0/3875126175 learned_addr learned my addr 172.21.15.39:0/3875126175 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:45:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.848+0000 7fc60f7fe700 1 --2- 172.21.15.39:0/3875126175 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ed0d8286b0 0x55ed0da2fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.849+0000 7fc60ffff700 1 -- 172.21.15.39:0/3875126175 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ed0d8286b0 msgr2=0x55ed0da2fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:45:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.849+0000 7fc60ffff700 1 --2- 172.21.15.39:0/3875126175 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ed0d8286b0 0x55ed0da2fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.849+0000 7fc60ffff700 1 -- 172.21.15.39:0/3875126175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed0d9d44a0 msgr2=0x55ed0da2fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:45:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.849+0000 7fc60ffff700 1 --2- 172.21.15.39:0/3875126175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed0d9d44a0 0x55ed0da2fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.849+0000 7fc60ffff700 1 -- 172.21.15.39:0/3875126175 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc610195530 con 0x55ed0d9d4910 2023-07-21T18:45:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.849+0000 7fc60f7fe700 1 --2- 172.21.15.39:0/3875126175 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ed0d8286b0 0x55ed0da2fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:45:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.849+0000 7fc60effd700 1 --2- 172.21.15.39:0/3875126175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed0d9d44a0 0x55ed0da2fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:45:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.849+0000 7fc60ffff700 1 --2- 172.21.15.39:0/3875126175 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ed0d9d4910 0x55ed0da3ab30 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x55ed0d9dade0 tx=0x7fc600007bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2cc31554c4819df server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:45:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.849+0000 7fc5eeffd700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:45:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.849+0000 7fc5eeffd700 1 -- 172.21.15.39:0/3875126175 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc60000e040 con 0x55ed0d9d4910 2023-07-21T18:45:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.849+0000 7fc5eeffd700 1 -- 172.21.15.39:0/3875126175 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fc600004db0 con 0x55ed0d9d4910 2023-07-21T18:45:17.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.849+0000 7fc62202bb80 1 -- 172.21.15.39:0/3875126175 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc608021070 con 0x55ed0d9d4910 2023-07-21T18:45:17.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.849+0000 7fc5eeffd700 1 -- 172.21.15.39:0/3875126175 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc600003a00 con 0x55ed0d9d4910 2023-07-21T18:45:17.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.849+0000 7fc62202bb80 10 client.16389 fetch_fsmap learned FSMap version 35 2023-07-21T18:45:17.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.849+0000 7fc62202bb80 1 -- 172.21.15.39:0/3875126175 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x7fc608017070 con 0x55ed0d9d4910 2023-07-21T18:45:17.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.850+0000 7fc5eeffd700 1 -- 172.21.15.39:0/3875126175 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc608021070 con 0x55ed0d9d4910 2023-07-21T18:45:17.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.850+0000 7fc5eeffd700 1 -- 172.21.15.39:0/3875126175 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 35) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fc600017550 con 0x55ed0d9d4910 2023-07-21T18:45:17.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.850+0000 7fc62202bb80 10 client.16389 fetch_fsmap finished waiting for FSMap version 35 2023-07-21T18:45:17.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.850+0000 7fc62202bb80 10 client.16389 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:45:17.876 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.850+0000 7fc62202bb80 20 client.16389 populate_metadata read hostname 'smithi039' 2023-07-21T18:45:17.876 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.850+0000 7fc62202bb80 1 --2- 172.21.15.39:0/3875126175 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55ed0da3c970 0x55ed0da3ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:17.876 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.850+0000 7fc62202bb80 4 client.16389 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:45:17.876 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.850+0000 7fc62202bb80 1 -- 172.21.15.39:0/3875126175 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ed0d8d8ff0 con 0x55ed0da3c970 2023-07-21T18:45:17.876 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.850+0000 7fc60f7fe700 1 --2- 172.21.15.39:0/3875126175 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55ed0da3c970 0x55ed0da3ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:17.876 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.850+0000 7fc5eeffd700 10 client.16389 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:45:17.876 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.850+0000 7fc60f7fe700 1 --2- 172.21.15.39:0/3875126175 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55ed0da3c970 0x55ed0da3ed80 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3523d4e7324ebb3a server_cookie=cd976419e5ffa0e4 in_seq=0 out_seq=0 2023-07-21T18:45:17.903 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.884+0000 7fc5eeffd700 1 -- 172.21.15.39:0/3875126175 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ed0d8d8ff0 con 0x55ed0da3c970 2023-07-21T18:45:17.903 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.884+0000 7fc5eeffd700 10 client.16389 handle_command_reply: tid=0 2023-07-21T18:45:17.903 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.884+0000 7fc62202bb80 1 client.16389 shutdown 2023-07-21T18:45:17.903 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.884+0000 7fc62202bb80 1 -- 172.21.15.39:0/3875126175 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ed0d9d4910 msgr2=0x55ed0da3ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:45:17.903 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.884+0000 7fc62202bb80 1 --2- 172.21.15.39:0/3875126175 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ed0d9d4910 0x55ed0da3ab30 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x55ed0d9dade0 tx=0x7fc600007bc0 comp rx=0 tx=0).stop 2023-07-21T18:45:17.903 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.884+0000 7fc62202bb80 1 -- 172.21.15.39:0/3875126175 shutdown_connections 2023-07-21T18:45:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.884+0000 7fc62202bb80 1 --2- 172.21.15.39:0/3875126175 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55ed0da3c970 0x55ed0da3ed80 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.884+0000 7fc62202bb80 1 --2- 172.21.15.39:0/3875126175 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ed0d9d4910 0x55ed0da3ab30 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.884+0000 7fc62202bb80 1 --2- 172.21.15.39:0/3875126175 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ed0d8286b0 0x55ed0da2fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.884+0000 7fc5eeffd700 0 client.16389 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:45:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.884+0000 7fc62202bb80 1 --2- 172.21.15.39:0/3875126175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed0d9d44a0 0x55ed0da2fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.885+0000 7fc62202bb80 1 -- 172.21.15.39:0/3875126175 >> 172.21.15.39:0/3875126175 conn(0x55ed0d9c1be0 msgr2=0x55ed0d9cb030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:45:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.885+0000 7fc62202bb80 1 -- 172.21.15.39:0/3875126175 shutdown_connections 2023-07-21T18:45:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.885+0000 7fc62202bb80 1 -- 172.21.15.39:0/3875126175 wait complete. 2023-07-21T18:45:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.885+0000 7fc62202bb80 20 client.16389 trim_cache size 0 max 16384 2023-07-21T18:45:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:45:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:45:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:45:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:45:17.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.885+0000 7fc615ea5700 1 -- 172.21.15.39:0/2838870126 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fc5fc0a12c0 msgr2=0x7fc5fc0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:45:17.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.885+0000 7fc615ea5700 1 --2- 172.21.15.39:0/2838870126 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fc5fc0a12c0 0x7fc5fc0a3770 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc6100ffba0 tx=0x7fc608009260 comp rx=0 tx=0).stop 2023-07-21T18:45:17.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.885+0000 7fc615ea5700 1 -- 172.21.15.39:0/2838870126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6100ff3e0 msgr2=0x7fc61019aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:45:17.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.885+0000 7fc615ea5700 1 --2- 172.21.15.39:0/2838870126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6100ff3e0 0x7fc61019aff0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8005910 tx=0x7fc5f8004710 comp rx=0 tx=0).stop 2023-07-21T18:45:17.907 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.885+0000 7fc615ea5700 1 -- 172.21.15.39:0/2838870126 shutdown_connections 2023-07-21T18:45:17.907 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.885+0000 7fc615ea5700 1 --2- 172.21.15.39:0/2838870126 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fc5fc0a12c0 0x7fc5fc0a3770 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:17.907 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.885+0000 7fc615ea5700 1 --2- 172.21.15.39:0/2838870126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc610104a30 0x7fc610194910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:17.907 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.885+0000 7fc615ea5700 1 --2- 172.21.15.39:0/2838870126 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc6100fea30 0x7fc61019aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:17.907 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.885+0000 7fc615ea5700 1 --2- 172.21.15.39:0/2838870126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6100ff3e0 0x7fc61019aff0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:17.907 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.885+0000 7fc615ea5700 1 -- 172.21.15.39:0/2838870126 >> 172.21.15.39:0/2838870126 conn(0x7fc610074760 msgr2=0x7fc6100fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:45:17.907 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.885+0000 7fc615ea5700 1 -- 172.21.15.39:0/2838870126 shutdown_connections 2023-07-21T18:45:17.908 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:17.886+0000 7fc615ea5700 1 -- 172.21.15.39:0/2838870126 wait complete. 2023-07-21T18:45:18.268 INFO:tasks.workunit.client.0.smithi039.stdout: 1 170433 3.04 MB/sec execute 31 sec latency 91.394 ms 2023-07-21T18:45:18.365 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:45:18 smithi193 ceph-mon[98204]: pgmap v109: 97 pgs: 97 active+clean; 69 MiB data, 916 MiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 2.7 MiB/s wr, 203 op/s 2023-07-21T18:45:18.365 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:45:18 smithi193 ceph-mon[98204]: daemon mds.c restarted 2023-07-21T18:45:18.365 INFO:journalctl@ceph.mds.c.smithi193.stdout:Jul 21 18:45:17 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-c[124252]: ignoring --setuser ceph since I am not root 2023-07-21T18:45:18.366 INFO:journalctl@ceph.mds.c.smithi193.stdout:Jul 21 18:45:17 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-c[124252]: ignoring --setgroup ceph since I am not root 2023-07-21T18:45:18.366 INFO:journalctl@ceph.mds.c.smithi193.stdout:Jul 21 18:45:17 smithi193 ceph-mds[124276]: -- 172.21.15.193:0/2949055499 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560bfab931e0 con 0x560bfab68c00 2023-07-21T18:45:18.366 INFO:journalctl@ceph.mds.c.smithi193.stdout:Jul 21 18:45:17 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-c[124252]: starting mds.c at 2023-07-21T18:45:18.634 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:45:18 smithi155 ceph-mon[97934]: pgmap v109: 97 pgs: 97 active+clean; 69 MiB data, 916 MiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 2.7 MiB/s wr, 203 op/s 2023-07-21T18:45:18.634 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:45:18 smithi155 ceph-mon[97934]: daemon mds.c restarted 2023-07-21T18:45:18.682 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:45:18 smithi039 ceph-mon[96489]: pgmap v109: 97 pgs: 97 active+clean; 69 MiB data, 916 MiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 2.7 MiB/s wr, 203 op/s 2023-07-21T18:45:18.683 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:45:18 smithi039 ceph-mon[96489]: daemon mds.c restarted 2023-07-21T18:45:19.268 INFO:tasks.workunit.client.0.smithi039.stdout: 1 170433 2.95 MB/sec execute 32 sec latency 1091.483 ms 2023-07-21T18:45:19.615 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:45:19 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-21T18:45:19.615 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:45:19 smithi193 ceph-mon[98204]: osdmap e83: 12 total, 12 up, 12 in 2023-07-21T18:45:19.615 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:45:19 smithi193 ceph-mon[98204]: mds.? [v2:172.21.15.193:6836/4255708820,v1:172.21.15.193:6837/4255708820] up:boot 2023-07-21T18:45:19.615 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:45:19 smithi193 ceph-mon[98204]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2023-07-21T18:45:19.616 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:45:19 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:45:19.634 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:45:19 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-21T18:45:19.634 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:45:19 smithi155 ceph-mon[97934]: osdmap e83: 12 total, 12 up, 12 in 2023-07-21T18:45:19.634 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:45:19 smithi155 ceph-mon[97934]: mds.? [v2:172.21.15.193:6836/4255708820,v1:172.21.15.193:6837/4255708820] up:boot 2023-07-21T18:45:19.634 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:45:19 smithi155 ceph-mon[97934]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2023-07-21T18:45:19.634 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:45:19 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:45:19.682 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:45:19 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-21T18:45:19.683 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:45:19 smithi039 ceph-mon[96489]: osdmap e83: 12 total, 12 up, 12 in 2023-07-21T18:45:19.683 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:45:19 smithi039 ceph-mon[96489]: mds.? [v2:172.21.15.193:6836/4255708820,v1:172.21.15.193:6837/4255708820] up:boot 2023-07-21T18:45:19.683 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:45:19 smithi039 ceph-mon[96489]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2023-07-21T18:45:19.683 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:45:19 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:45:20.268 INFO:tasks.workunit.client.0.smithi039.stdout: 1 170433 2.86 MB/sec execute 33 sec latency 2091.584 ms 2023-07-21T18:45:20.615 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:45:20 smithi193 ceph-mon[98204]: pgmap v111: 97 pgs: 97 active+clean; 69 MiB data, 916 MiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 2.0 MiB/s wr, 123 op/s 2023-07-21T18:45:20.616 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:45:20 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:45:20.633 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:45:20 smithi155 ceph-mon[97934]: pgmap v111: 97 pgs: 97 active+clean; 69 MiB data, 916 MiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 2.0 MiB/s wr, 123 op/s 2023-07-21T18:45:20.634 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:45:20 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:45:20.682 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:45:20 smithi039 ceph-mon[96489]: pgmap v111: 97 pgs: 97 active+clean; 69 MiB data, 916 MiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 2.0 MiB/s wr, 123 op/s 2023-07-21T18:45:20.683 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:45:20 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:45:21.268 INFO:tasks.workunit.client.0.smithi039.stdout: 1 170585 2.80 MB/sec execute 34 sec latency 2905.686 ms 2023-07-21T18:45:22.268 INFO:tasks.workunit.client.0.smithi039.stdout: 1 172620 2.93 MB/sec execute 35 sec latency 17.826 ms 2023-07-21T18:45:22.614 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:45:22 smithi193 ceph-mon[98204]: pgmap v112: 97 pgs: 97 active+clean; 67 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 2.4 MiB/s wr, 188 op/s 2023-07-21T18:45:22.633 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:45:22 smithi155 ceph-mon[97934]: pgmap v112: 97 pgs: 97 active+clean; 67 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 2.4 MiB/s wr, 188 op/s 2023-07-21T18:45:22.682 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:45:22 smithi039 ceph-mon[96489]: pgmap v112: 97 pgs: 97 active+clean; 67 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 2.4 MiB/s wr, 188 op/s 2023-07-21T18:45:23.268 INFO:tasks.workunit.client.0.smithi039.stdout: 1 173908 3.15 MB/sec execute 36 sec latency 375.059 ms 2023-07-21T18:45:24.268 INFO:tasks.workunit.client.0.smithi039.stdout: 1 173908 3.06 MB/sec execute 37 sec latency 1375.154 ms 2023-07-21T18:45:24.615 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:45:24 smithi193 ceph-mon[98204]: pgmap v113: 97 pgs: 97 active+clean; 67 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 2.4 MiB/s wr, 188 op/s 2023-07-21T18:45:24.633 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:45:24 smithi155 ceph-mon[97934]: pgmap v113: 97 pgs: 97 active+clean; 67 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 2.4 MiB/s wr, 188 op/s 2023-07-21T18:45:24.682 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:45:24 smithi039 ceph-mon[96489]: pgmap v113: 97 pgs: 97 active+clean; 67 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 2.4 MiB/s wr, 188 op/s 2023-07-21T18:45:25.269 INFO:tasks.workunit.client.0.smithi039.stdout: 1 173908 2.98 MB/sec execute 38 sec latency 2375.269 ms 2023-07-21T18:45:25.615 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:45:25 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:45:25.633 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:45:25 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:45:25.682 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:45:25 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:45:26.268 INFO:tasks.workunit.client.0.smithi039.stdout: 1 173959 2.91 MB/sec execute 39 sec latency 3185.462 ms 2023-07-21T18:45:26.614 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:45:26 smithi193 ceph-mon[98204]: pgmap v114: 97 pgs: 97 active+clean; 67 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 2.4 MiB/s wr, 188 op/s 2023-07-21T18:45:26.633 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:45:26 smithi155 ceph-mon[97934]: pgmap v114: 97 pgs: 97 active+clean; 67 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 2.4 MiB/s wr, 188 op/s 2023-07-21T18:45:26.682 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:45:26 smithi039 ceph-mon[96489]: pgmap v114: 97 pgs: 97 active+clean; 67 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 2.4 MiB/s wr, 188 op/s 2023-07-21T18:45:27.269 INFO:tasks.workunit.client.0.smithi039.stdout: 1 173959 2.83 MB/sec execute 40 sec latency 1143.534 ms 2023-07-21T18:45:28.269 INFO:tasks.workunit.client.0.smithi039.stdout: 1 173959 2.76 MB/sec execute 41 sec latency 2143.660 ms 2023-07-21T18:45:28.615 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:45:28 smithi193 ceph-mon[98204]: pgmap v115: 97 pgs: 97 active+clean; 68 MiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 2.1 MiB/s wr, 175 op/s 2023-07-21T18:45:28.633 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:45:28 smithi155 ceph-mon[97934]: pgmap v115: 97 pgs: 97 active+clean; 68 MiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 2.1 MiB/s wr, 175 op/s 2023-07-21T18:45:28.682 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:45:28 smithi039 ceph-mon[96489]: pgmap v115: 97 pgs: 97 active+clean; 68 MiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 2.1 MiB/s wr, 175 op/s 2023-07-21T18:45:29.272 INFO:tasks.workunit.client.0.smithi039.stdout: 1 173959 2.70 MB/sec execute 42 sec latency 3143.716 ms 2023-07-21T18:45:30.269 INFO:tasks.workunit.client.0.smithi039.stdout: 1 173959 2.64 MB/sec execute 43 sec latency 4143.799 ms 2023-07-21T18:45:30.614 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:45:30 smithi193 ceph-mon[98204]: pgmap v116: 97 pgs: 97 active+clean; 68 MiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 679 KiB/s rd, 2.0 MiB/s wr, 173 op/s 2023-07-21T18:45:30.615 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:45:30 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:45:30.633 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:45:30 smithi155 ceph-mon[97934]: pgmap v116: 97 pgs: 97 active+clean; 68 MiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 679 KiB/s rd, 2.0 MiB/s wr, 173 op/s 2023-07-21T18:45:30.633 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:45:30 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:45:30.682 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:45:30 smithi039 ceph-mon[96489]: pgmap v116: 97 pgs: 97 active+clean; 68 MiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 679 KiB/s rd, 2.0 MiB/s wr, 173 op/s 2023-07-21T18:45:30.683 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:45:30 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:45:31.026 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.009+0000 7fe5cd953700 1 -- 172.21.15.39:0/4089844490 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5c80ff250 msgr2=0x7fe5c80ff6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:45:31.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.009+0000 7fe5cd953700 1 --2- 172.21.15.39:0/4089844490 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5c80ff250 0x7fe5c80ff6c0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0009990 tx=0x7fe5b0009ca0 comp rx=0 tx=0).stop 2023-07-21T18:45:31.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.009+0000 7fe5cd953700 1 -- 172.21.15.39:0/4089844490 shutdown_connections 2023-07-21T18:45:31.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.009+0000 7fe5cd953700 1 --2- 172.21.15.39:0/4089844490 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5c8068420 0x7fe5c80fed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:31.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.009+0000 7fe5cd953700 1 --2- 172.21.15.39:0/4089844490 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5c810a120 0x7fe5c810c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:31.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.009+0000 7fe5cd953700 1 --2- 172.21.15.39:0/4089844490 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5c80ff250 0x7fe5c80ff6c0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:31.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.010+0000 7fe5cd953700 1 -- 172.21.15.39:0/4089844490 >> 172.21.15.39:0/4089844490 conn(0x7fe5c80752a0 msgr2=0x7fe5c80756a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:45:31.028 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.010+0000 7fe5cd953700 1 -- 172.21.15.39:0/4089844490 shutdown_connections 2023-07-21T18:45:31.028 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.010+0000 7fe5cd953700 1 -- 172.21.15.39:0/4089844490 wait complete. 2023-07-21T18:45:31.029 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.010+0000 7fe5cd953700 1 Processor -- start 2023-07-21T18:45:31.029 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.011+0000 7fe5cd953700 1 -- start start 2023-07-21T18:45:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.011+0000 7fe5cd953700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5c8068420 0x7fe5c819bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.011+0000 7fe5cd953700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5c80ff250 0x7fe5c819c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.011+0000 7fe5cd953700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5c810a120 0x7fe5c8195a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.011+0000 7fe5cd953700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c81960e0 con 0x7fe5c8068420 2023-07-21T18:45:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.011+0000 7fe5cd953700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c8196250 con 0x7fe5c80ff250 2023-07-21T18:45:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.011+0000 7fe5cd953700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c8196410 con 0x7fe5c810a120 2023-07-21T18:45:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.011+0000 7fe5c6ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5c8068420 0x7fe5c819bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:31.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.011+0000 7fe5c6ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5c8068420 0x7fe5c819bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59490/0 (socket says 172.21.15.39:59490) 2023-07-21T18:45:31.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.011+0000 7fe5c6ffd700 1 -- 172.21.15.39:0/2753874635 learned_addr learned my addr 172.21.15.39:0/2753874635 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:45:31.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.011+0000 7fe5c67fc700 1 --2- 172.21.15.39:0/2753874635 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5c80ff250 0x7fe5c819c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:31.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.011+0000 7fe5c77fe700 1 --2- 172.21.15.39:0/2753874635 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5c810a120 0x7fe5c8195a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:31.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.012+0000 7fe5c6ffd700 1 -- 172.21.15.39:0/2753874635 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5c810a120 msgr2=0x7fe5c8195a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:45:31.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.012+0000 7fe5c6ffd700 1 --2- 172.21.15.39:0/2753874635 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5c810a120 0x7fe5c8195a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:31.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.012+0000 7fe5c6ffd700 1 -- 172.21.15.39:0/2753874635 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5c80ff250 msgr2=0x7fe5c819c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:45:31.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.012+0000 7fe5c6ffd700 1 --2- 172.21.15.39:0/2753874635 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5c80ff250 0x7fe5c819c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:31.032 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.012+0000 7fe5c6ffd700 1 -- 172.21.15.39:0/2753874635 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5b00095f0 con 0x7fe5c8068420 2023-07-21T18:45:31.032 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.012+0000 7fe5c6ffd700 1 --2- 172.21.15.39:0/2753874635 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5c8068420 0x7fe5c819bc10 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fe5b800e9a0 tx=0x7fe5b800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:45:31.032 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.012+0000 7fe5cc951700 1 -- 172.21.15.39:0/2753874635 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b8009790 con 0x7fe5c8068420 2023-07-21T18:45:31.032 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.012+0000 7fe5cd953700 1 -- 172.21.15.39:0/2753874635 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5c81966f0 con 0x7fe5c8068420 2023-07-21T18:45:31.032 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.012+0000 7fe5cc951700 1 -- 172.21.15.39:0/2753874635 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5b800cbd0 con 0x7fe5c8068420 2023-07-21T18:45:31.032 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.012+0000 7fe5cc951700 1 -- 172.21.15.39:0/2753874635 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b8018680 con 0x7fe5c8068420 2023-07-21T18:45:31.032 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.012+0000 7fe5cd953700 1 -- 172.21.15.39:0/2753874635 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5c81a2c00 con 0x7fe5c8068420 2023-07-21T18:45:31.034 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.015+0000 7fe5cc951700 1 -- 172.21.15.39:0/2753874635 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fe5b800cd40 con 0x7fe5c8068420 2023-07-21T18:45:31.034 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.015+0000 7fe5cc951700 1 --2- 172.21.15.39:0/2753874635 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fe5b40a12c0 0x7fe5b40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:31.035 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.016+0000 7fe5cc951700 1 -- 172.21.15.39:0/2753874635 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fe5b8014070 con 0x7fe5c8068420 2023-07-21T18:45:31.035 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.016+0000 7fe5c67fc700 1 --2- 172.21.15.39:0/2753874635 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fe5b40a12c0 0x7fe5b40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:31.035 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.016+0000 7fe5c67fc700 1 --2- 172.21.15.39:0/2753874635 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fe5b40a12c0 0x7fe5b40a3770 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe5c80ff0f0 tx=0x7fe5b0005fb0 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:45:31.035 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.016+0000 7fe5d9ad9b80 1 Processor -- start 2023-07-21T18:45:31.035 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.016+0000 7fe5d9ad9b80 1 -- start start 2023-07-21T18:45:31.035 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.016+0000 7fe5d9ad9b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559af534f6b0 0x559af54f8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:31.035 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.017+0000 7fe5d9ad9b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559af54f92d0 0x559af54f3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:31.035 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.017+0000 7fe5d9ad9b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559af54f3580 0x559af54f39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.017+0000 7fe5d9ad9b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x559af53f8c50 con 0x559af534f6b0 2023-07-21T18:45:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.017+0000 7fe5d9ad9b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559af544b9e0 con 0x559af54f3580 2023-07-21T18:45:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.017+0000 7fe5d9ad9b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x559af5444e70 con 0x559af54f92d0 2023-07-21T18:45:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.017+0000 7fe5c6ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559af534f6b0 0x559af54f8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.017+0000 7fe5c6ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559af534f6b0 0x559af54f8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59498/0 (socket says 172.21.15.39:59498) 2023-07-21T18:45:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.017+0000 7fe5c6ffd700 1 -- 172.21.15.39:0/1900991761 learned_addr learned my addr 172.21.15.39:0/1900991761 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:45:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.017+0000 7fe5c67fc700 1 --2- 172.21.15.39:0/1900991761 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559af54f92d0 0x559af54f3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:31.037 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.017+0000 7fe5c77fe700 1 --2- 172.21.15.39:0/1900991761 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559af54f3580 0x559af54f39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:31.037 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.017+0000 7fe5c6ffd700 1 -- 172.21.15.39:0/1900991761 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559af54f92d0 msgr2=0x559af54f3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:45:31.037 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.017+0000 7fe5c6ffd700 1 --2- 172.21.15.39:0/1900991761 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559af54f92d0 0x559af54f3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:31.037 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.017+0000 7fe5c6ffd700 1 -- 172.21.15.39:0/1900991761 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559af54f3580 msgr2=0x559af54f39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:45:31.037 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.017+0000 7fe5c6ffd700 1 --2- 172.21.15.39:0/1900991761 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559af54f3580 0x559af54f39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:31.037 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.017+0000 7fe5c6ffd700 1 -- 172.21.15.39:0/1900991761 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559af5272d80 con 0x559af534f6b0 2023-07-21T18:45:31.037 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.017+0000 7fe5c67fc700 1 --2- 172.21.15.39:0/1900991761 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559af54f92d0 0x559af54f3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:45:31.038 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.017+0000 7fe5c6ffd700 1 --2- 172.21.15.39:0/1900991761 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559af534f6b0 0x559af54f8d90 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fe5b8096ab0 tx=0x7fe5b8099a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7456c63c0887c0ec server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:45:31.038 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.017+0000 7fe5bdffb700 1 -- 172.21.15.39:0/1900991761 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b80a2490 con 0x559af534f6b0 2023-07-21T18:45:31.038 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.017+0000 7fe5bdffb700 1 -- 172.21.15.39:0/1900991761 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5b809a2a0 con 0x559af534f6b0 2023-07-21T18:45:31.038 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.017+0000 7fe5bdffb700 1 -- 172.21.15.39:0/1900991761 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b80a2260 con 0x559af534f6b0 2023-07-21T18:45:31.041 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.018+0000 7fe5d9ad9b80 1 -- 172.21.15.39:0/1900991761 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559af534f6b0 msgr2=0x559af54f8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:45:31.041 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.018+0000 7fe5d9ad9b80 1 --2- 172.21.15.39:0/1900991761 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559af534f6b0 0x559af54f8d90 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fe5b8096ab0 tx=0x7fe5b8099a20 comp rx=0 tx=0).stop 2023-07-21T18:45:31.041 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.018+0000 7fe5d9ad9b80 1 -- 172.21.15.39:0/1900991761 shutdown_connections 2023-07-21T18:45:31.042 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.018+0000 7fe5d9ad9b80 1 --2- 172.21.15.39:0/1900991761 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559af54f3580 0x559af54f39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:31.042 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.018+0000 7fe5d9ad9b80 1 --2- 172.21.15.39:0/1900991761 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559af534f6b0 0x559af54f8d90 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:31.042 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.018+0000 7fe5d9ad9b80 1 --2- 172.21.15.39:0/1900991761 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559af54f92d0 0x559af54f3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:31.042 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.018+0000 7fe5d9ad9b80 1 -- 172.21.15.39:0/1900991761 >> 172.21.15.39:0/1900991761 conn(0x559af54e8be0 msgr2=0x559af54edf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:45:31.042 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.018+0000 7fe5d9ad9b80 1 -- 172.21.15.39:0/1900991761 shutdown_connections 2023-07-21T18:45:31.042 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.018+0000 7fe5d9ad9b80 1 -- 172.21.15.39:0/1900991761 wait complete. 2023-07-21T18:45:31.042 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.018+0000 7fe5d9ad9b80 1 -- start start 2023-07-21T18:45:31.042 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.019+0000 7fe5d9ad9b80 1 Processor -- start 2023-07-21T18:45:31.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.019+0000 7fe5d9ad9b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559af534f6b0 0x559af54fc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:31.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.019+0000 7fe5d9ad9b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559af54f3580 0x559af54fce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:31.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.019+0000 7fe5d9ad9b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559af54f92d0 0x559af54fd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:31.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.019+0000 7fe5d9ad9b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x559af53e4a60 con 0x559af54f92d0 2023-07-21T18:45:31.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.019+0000 7fe5d9ad9b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559af53e4760 con 0x559af54f3580 2023-07-21T18:45:31.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.019+0000 7fe5d9ad9b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x559af52c88c0 con 0x559af534f6b0 2023-07-21T18:45:31.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.019+0000 7fe5d9ad9b80 10 client.0 _ll_register_callbacks cb 0x559af53e2700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:45:31.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.019+0000 7fe5c77fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559af54f92d0 0x559af54fd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.019+0000 7fe5c77fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559af54f92d0 0x559af54fd340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59508/0 (socket says 172.21.15.39:59508) 2023-07-21T18:45:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.019+0000 7fe5c77fe700 1 -- 172.21.15.39:0/3755626911 learned_addr learned my addr 172.21.15.39:0/3755626911 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:45:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.019+0000 7fe5c6ffd700 1 --2- 172.21.15.39:0/3755626911 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559af534f6b0 0x559af54fc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.019+0000 7fe5c67fc700 1 --2- 172.21.15.39:0/3755626911 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559af54f3580 0x559af54fce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.019+0000 7fe5c77fe700 1 -- 172.21.15.39:0/3755626911 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559af534f6b0 msgr2=0x559af54fc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:45:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.020+0000 7fe5c77fe700 1 --2- 172.21.15.39:0/3755626911 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559af534f6b0 0x559af54fc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.020+0000 7fe5c77fe700 1 -- 172.21.15.39:0/3755626911 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559af54f3580 msgr2=0x559af54fce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:45:31.045 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.020+0000 7fe5c77fe700 1 --2- 172.21.15.39:0/3755626911 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559af54f3580 0x559af54fce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:31.045 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.020+0000 7fe5c77fe700 1 -- 172.21.15.39:0/3755626911 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5c81a2c00 con 0x559af54f92d0 2023-07-21T18:45:31.045 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.020+0000 7fe5c77fe700 1 --2- 172.21.15.39:0/3755626911 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559af54f92d0 0x559af54fd340 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fe5c0009550 tx=0x7fe5c0007e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27c0b9ed1cdab195 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:45:31.045 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.020+0000 7fe5a6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:45:31.045 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.020+0000 7fe5a6ffd700 1 -- 172.21.15.39:0/3755626911 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c000e070 con 0x559af54f92d0 2023-07-21T18:45:31.045 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.020+0000 7fe5a6ffd700 1 -- 172.21.15.39:0/3755626911 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fe5c0004db0 con 0x559af54f92d0 2023-07-21T18:45:31.045 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.020+0000 7fe5d9ad9b80 1 -- 172.21.15.39:0/3755626911 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559af539e490 con 0x559af54f92d0 2023-07-21T18:45:31.045 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.020+0000 7fe5a6ffd700 1 -- 172.21.15.39:0/3755626911 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe5c0003a00 con 0x559af54f92d0 2023-07-21T18:45:31.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.020+0000 7fe5d9ad9b80 10 client.16407 fetch_fsmap learned FSMap version 40 2023-07-21T18:45:31.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.020+0000 7fe5d9ad9b80 1 -- 172.21.15.39:0/3755626911 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x559af525bc00 con 0x559af54f92d0 2023-07-21T18:45:31.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.020+0000 7fe5a6ffd700 1 -- 172.21.15.39:0/3755626911 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559af539e490 con 0x559af54f92d0 2023-07-21T18:45:31.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.020+0000 7fe5a6ffd700 1 -- 172.21.15.39:0/3755626911 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 40) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fe5c0014040 con 0x559af54f92d0 2023-07-21T18:45:31.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.020+0000 7fe5d9ad9b80 10 client.16407 fetch_fsmap finished waiting for FSMap version 40 2023-07-21T18:45:31.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.020+0000 7fe5d9ad9b80 10 client.16407 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:45:31.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.020+0000 7fe5d9ad9b80 20 client.16407 populate_metadata read hostname 'smithi039' 2023-07-21T18:45:31.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.020+0000 7fe5d9ad9b80 1 --2- 172.21.15.39:0/3755626911 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x559af550cc10 0x559af550f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:31.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.020+0000 7fe5d9ad9b80 4 client.16407 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:45:31.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.020+0000 7fe5d9ad9b80 1 -- 172.21.15.39:0/3755626911 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559af543de50 con 0x559af550cc10 2023-07-21T18:45:31.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.021+0000 7fe5c6ffd700 1 --2- 172.21.15.39:0/3755626911 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x559af550cc10 0x559af550f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:31.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.021+0000 7fe5c6ffd700 1 --2- 172.21.15.39:0/3755626911 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x559af550cc10 0x559af550f020 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d8c5017d78ddc16 server_cookie=4a1a416ddb138668 in_seq=0 out_seq=0 2023-07-21T18:45:31.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.021+0000 7fe5a6ffd700 10 client.16407 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:45:31.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.022+0000 7fe5a6ffd700 1 -- 172.21.15.39:0/3755626911 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559af543de50 con 0x559af550cc10 2023-07-21T18:45:31.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.022+0000 7fe5a6ffd700 10 client.16407 handle_command_reply: tid=0 2023-07-21T18:45:31.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.022+0000 7fe5d9ad9b80 1 client.16407 shutdown 2023-07-21T18:45:31.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.022+0000 7fe5d9ad9b80 1 -- 172.21.15.39:0/3755626911 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559af54f92d0 msgr2=0x559af54fd340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:45:31.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.022+0000 7fe5d9ad9b80 1 --2- 172.21.15.39:0/3755626911 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559af54f92d0 0x559af54fd340 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fe5c0009550 tx=0x7fe5c0007e30 comp rx=0 tx=0).stop 2023-07-21T18:45:31.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.022+0000 7fe5d9ad9b80 1 -- 172.21.15.39:0/3755626911 shutdown_connections 2023-07-21T18:45:31.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.022+0000 7fe5d9ad9b80 1 --2- 172.21.15.39:0/3755626911 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x559af550cc10 0x559af550f020 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:31.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.022+0000 7fe5d9ad9b80 1 --2- 172.21.15.39:0/3755626911 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559af54f92d0 0x559af54fd340 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:31.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.022+0000 7fe5d9ad9b80 1 --2- 172.21.15.39:0/3755626911 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559af534f6b0 0x559af54fc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:31.049 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.022+0000 7fe5d9ad9b80 1 --2- 172.21.15.39:0/3755626911 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559af54f3580 0x559af54fce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:31.049 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.022+0000 7fe5a6ffd700 0 client.16407 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:45:31.049 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.022+0000 7fe5d9ad9b80 1 -- 172.21.15.39:0/3755626911 >> 172.21.15.39:0/3755626911 conn(0x559af54e8be0 msgr2=0x559af54ed4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:45:31.049 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.022+0000 7fe5d9ad9b80 1 -- 172.21.15.39:0/3755626911 shutdown_connections 2023-07-21T18:45:31.049 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.022+0000 7fe5d9ad9b80 1 -- 172.21.15.39:0/3755626911 wait complete. 2023-07-21T18:45:31.049 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.022+0000 7fe5d9ad9b80 20 client.16407 trim_cache size 0 max 16384 2023-07-21T18:45:31.055 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.038+0000 7fe5d9ad9b80 1 Processor -- start 2023-07-21T18:45:31.055 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.038+0000 7fe5d9ad9b80 1 -- start start 2023-07-21T18:45:31.055 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.038+0000 7fe5d9ad9b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559af534f6b0 0x559af54f3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:31.055 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.038+0000 7fe5d9ad9b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559af54f3580 0x559af54f8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:31.056 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.038+0000 7fe5d9ad9b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559af54f92d0 0x559af54fc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:31.056 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.038+0000 7fe5d9ad9b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x559af54b9060 con 0x559af54f3580 2023-07-21T18:45:31.056 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.038+0000 7fe5d9ad9b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559af54ff950 con 0x559af534f6b0 2023-07-21T18:45:31.056 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.038+0000 7fe5d9ad9b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x559af54f8a10 con 0x559af54f92d0 2023-07-21T18:45:31.056 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.038+0000 7fe5c67fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559af54f3580 0x559af54f8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:31.056 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.038+0000 7fe5c67fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559af54f3580 0x559af54f8320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59514/0 (socket says 172.21.15.39:59514) 2023-07-21T18:45:31.056 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.038+0000 7fe5c67fc700 1 -- 172.21.15.39:0/2432583931 learned_addr learned my addr 172.21.15.39:0/2432583931 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:45:31.057 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.038+0000 7fe5c67fc700 1 -- 172.21.15.39:0/2432583931 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559af54f92d0 msgr2=0x559af54fc0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:45:31.057 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.038+0000 7fe5c67fc700 1 --2- 172.21.15.39:0/2432583931 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559af54f92d0 0x559af54fc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:31.057 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.038+0000 7fe5c67fc700 1 -- 172.21.15.39:0/2432583931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559af534f6b0 msgr2=0x559af54f3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:45:31.057 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.038+0000 7fe5c6ffd700 1 --2- 172.21.15.39:0/2432583931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559af534f6b0 0x559af54f3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:31.057 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.038+0000 7fe5c67fc700 1 --2- 172.21.15.39:0/2432583931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559af534f6b0 0x559af54f3040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:31.057 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.038+0000 7fe5c67fc700 1 -- 172.21.15.39:0/2432583931 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559af5513c50 con 0x559af54f3580 2023-07-21T18:45:31.059 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.039+0000 7fe5c67fc700 1 --2- 172.21.15.39:0/2432583931 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559af54f3580 0x559af54f8320 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fe5b00098e0 tx=0x7fe5b0023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17fb2dc1f705da54 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:45:31.059 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.039+0000 7fe5be7fc700 1 -- 172.21.15.39:0/2432583931 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b0004700 con 0x559af54f3580 2023-07-21T18:45:31.059 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.039+0000 7fe5be7fc700 1 -- 172.21.15.39:0/2432583931 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5b002c070 con 0x559af54f3580 2023-07-21T18:45:31.059 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.039+0000 7fe5be7fc700 1 -- 172.21.15.39:0/2432583931 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b000fc90 con 0x559af54f3580 2023-07-21T18:45:31.059 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.039+0000 7fe5d9ad9b80 1 -- 172.21.15.39:0/2432583931 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559af54f3580 msgr2=0x559af54f8320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:45:31.059 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.039+0000 7fe5d9ad9b80 1 --2- 172.21.15.39:0/2432583931 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559af54f3580 0x559af54f8320 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fe5b00098e0 tx=0x7fe5b0023000 comp rx=0 tx=0).stop 2023-07-21T18:45:31.060 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.039+0000 7fe5d9ad9b80 1 -- 172.21.15.39:0/2432583931 shutdown_connections 2023-07-21T18:45:31.060 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.039+0000 7fe5d9ad9b80 1 --2- 172.21.15.39:0/2432583931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559af534f6b0 0x559af54f3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:31.060 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.039+0000 7fe5d9ad9b80 1 --2- 172.21.15.39:0/2432583931 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559af54f92d0 0x559af54fc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:31.060 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.039+0000 7fe5d9ad9b80 1 --2- 172.21.15.39:0/2432583931 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559af54f3580 0x559af54f8320 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:31.060 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.039+0000 7fe5d9ad9b80 1 -- 172.21.15.39:0/2432583931 >> 172.21.15.39:0/2432583931 conn(0x559af54e8be0 msgr2=0x559af54f9a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:45:31.060 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.039+0000 7fe5d9ad9b80 1 -- 172.21.15.39:0/2432583931 shutdown_connections 2023-07-21T18:45:31.060 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.040+0000 7fe5d9ad9b80 1 -- 172.21.15.39:0/2432583931 wait complete. 2023-07-21T18:45:31.061 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.040+0000 7fe5d9ad9b80 1 -- start start 2023-07-21T18:45:31.061 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.040+0000 7fe5d9ad9b80 1 Processor -- start 2023-07-21T18:45:31.061 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.040+0000 7fe5d9ad9b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559af534f6b0 0x559af5503600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:31.061 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.040+0000 7fe5d9ad9b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559af54f3580 0x559af5503b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:31.061 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.040+0000 7fe5d9ad9b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559af54f92d0 0x559af55019e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:31.061 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.040+0000 7fe5d9ad9b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x559af55020d0 con 0x559af534f6b0 2023-07-21T18:45:31.061 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.040+0000 7fe5d9ad9b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559af5502240 con 0x559af54f3580 2023-07-21T18:45:31.061 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.040+0000 7fe5d9ad9b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x559af5502400 con 0x559af54f92d0 2023-07-21T18:45:31.062 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.040+0000 7fe5d9ad9b80 10 client.0 _ll_register_callbacks cb 0x7fe5c0007500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:45:31.062 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.041+0000 7fe5c6ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559af534f6b0 0x559af5503600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:31.062 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.041+0000 7fe5c67fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559af54f3580 0x559af5503b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:31.062 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.041+0000 7fe5c6ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559af534f6b0 0x559af5503600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59518/0 (socket says 172.21.15.39:59518) 2023-07-21T18:45:31.062 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.041+0000 7fe5c6ffd700 1 -- 172.21.15.39:0/893532618 learned_addr learned my addr 172.21.15.39:0/893532618 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:45:31.062 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.041+0000 7fe5c77fe700 1 --2- 172.21.15.39:0/893532618 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559af54f92d0 0x559af55019e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:31.062 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.041+0000 7fe5c6ffd700 1 -- 172.21.15.39:0/893532618 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559af54f92d0 msgr2=0x559af55019e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:45:31.062 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.041+0000 7fe5c6ffd700 1 --2- 172.21.15.39:0/893532618 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559af54f92d0 0x559af55019e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.041+0000 7fe5c6ffd700 1 -- 172.21.15.39:0/893532618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559af54f3580 msgr2=0x559af5503b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:45:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.041+0000 7fe5c6ffd700 1 --2- 172.21.15.39:0/893532618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559af54f3580 0x559af5503b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.041+0000 7fe5c6ffd700 1 -- 172.21.15.39:0/893532618 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5c810d240 con 0x559af534f6b0 2023-07-21T18:45:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.041+0000 7fe5c77fe700 1 --2- 172.21.15.39:0/893532618 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559af54f92d0 0x559af55019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:45:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.041+0000 7fe5c6ffd700 1 --2- 172.21.15.39:0/893532618 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559af534f6b0 0x559af5503600 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fe5b80cc010 tx=0x7fe5b809a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16fa21dea3a2ff09 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:45:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.041+0000 7fe5a6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:45:31.064 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.043+0000 7fe5a6ffd700 1 -- 172.21.15.39:0/893532618 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b80a2490 con 0x559af534f6b0 2023-07-21T18:45:31.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.043+0000 7fe5a6ffd700 1 -- 172.21.15.39:0/893532618 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fe5b8099530 con 0x559af534f6b0 2023-07-21T18:45:31.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.043+0000 7fe5d9ad9b80 1 -- 172.21.15.39:0/893532618 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe5b809a2a0 con 0x559af534f6b0 2023-07-21T18:45:31.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.043+0000 7fe5a6ffd700 1 -- 172.21.15.39:0/893532618 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe5b80996f0 con 0x559af534f6b0 2023-07-21T18:45:31.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.044+0000 7fe5d9ad9b80 10 client.16413 fetch_fsmap learned FSMap version 40 2023-07-21T18:45:31.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.044+0000 7fe5d9ad9b80 1 -- 172.21.15.39:0/893532618 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x559af52991f0 con 0x559af534f6b0 2023-07-21T18:45:31.066 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.044+0000 7fe5a6ffd700 1 -- 172.21.15.39:0/893532618 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5b809a2a0 con 0x559af534f6b0 2023-07-21T18:45:31.066 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.044+0000 7fe5a6ffd700 1 -- 172.21.15.39:0/893532618 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 40) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fe5b8095e30 con 0x559af534f6b0 2023-07-21T18:45:31.066 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.044+0000 7fe5d9ad9b80 10 client.16413 fetch_fsmap finished waiting for FSMap version 40 2023-07-21T18:45:31.066 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.044+0000 7fe5d9ad9b80 10 client.16413 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:45:31.066 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.044+0000 7fe5d9ad9b80 20 client.16413 populate_metadata read hostname 'smithi039' 2023-07-21T18:45:31.066 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.044+0000 7fe5d9ad9b80 1 --2- 172.21.15.39:0/893532618 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x559af55618d0 0x559af5563ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:31.067 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.044+0000 7fe5d9ad9b80 4 client.16413 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:45:31.067 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.044+0000 7fe5d9ad9b80 1 -- 172.21.15.39:0/893532618 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559af53ffff0 con 0x559af55618d0 2023-07-21T18:45:31.067 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.044+0000 7fe5c67fc700 1 --2- 172.21.15.39:0/893532618 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x559af55618d0 0x559af5563ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:31.067 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.045+0000 7fe5c67fc700 1 --2- 172.21.15.39:0/893532618 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x559af55618d0 0x559af5563ce0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=361769779dbd5dd3 server_cookie=f346be9785fe69f8 in_seq=0 out_seq=0 2023-07-21T18:45:31.067 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.045+0000 7fe5a6ffd700 10 client.16413 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:45:31.104 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.087+0000 7fe5a6ffd700 1 -- 172.21.15.39:0/893532618 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559af53ffff0 con 0x559af55618d0 2023-07-21T18:45:31.104 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.087+0000 7fe5a6ffd700 10 client.16413 handle_command_reply: tid=0 2023-07-21T18:45:31.104 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.087+0000 7fe5d9ad9b80 1 client.16413 shutdown 2023-07-21T18:45:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.087+0000 7fe5d9ad9b80 1 -- 172.21.15.39:0/893532618 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559af534f6b0 msgr2=0x559af5503600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:45:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.087+0000 7fe5d9ad9b80 1 --2- 172.21.15.39:0/893532618 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559af534f6b0 0x559af5503600 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fe5b80cc010 tx=0x7fe5b809a1c0 comp rx=0 tx=0).stop 2023-07-21T18:45:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.087+0000 7fe5d9ad9b80 1 -- 172.21.15.39:0/893532618 shutdown_connections 2023-07-21T18:45:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.087+0000 7fe5d9ad9b80 1 --2- 172.21.15.39:0/893532618 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x559af55618d0 0x559af5563ce0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.087+0000 7fe5d9ad9b80 1 --2- 172.21.15.39:0/893532618 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559af54f92d0 0x559af55019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.087+0000 7fe5d9ad9b80 1 --2- 172.21.15.39:0/893532618 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559af534f6b0 0x559af5503600 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.087+0000 7fe5d9ad9b80 1 --2- 172.21.15.39:0/893532618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559af54f3580 0x559af5503b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:31.106 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.087+0000 7fe5d9ad9b80 1 -- 172.21.15.39:0/893532618 >> 172.21.15.39:0/893532618 conn(0x559af54e8be0 msgr2=0x559af54f8190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:45:31.106 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.087+0000 7fe5a6ffd700 0 client.16413 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:45:31.106 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.087+0000 7fe5d9ad9b80 1 -- 172.21.15.39:0/893532618 shutdown_connections 2023-07-21T18:45:31.106 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.087+0000 7fe5d9ad9b80 1 -- 172.21.15.39:0/893532618 wait complete. 2023-07-21T18:45:31.106 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.087+0000 7fe5d9ad9b80 20 client.16413 trim_cache size 0 max 16384 2023-07-21T18:45:31.106 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:45:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:45:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:45:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:45:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.088+0000 7fe5cd953700 1 -- 172.21.15.39:0/2753874635 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fe5b40a12c0 msgr2=0x7fe5b40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:45:31.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.088+0000 7fe5cd953700 1 --2- 172.21.15.39:0/2753874635 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fe5b40a12c0 0x7fe5b40a3770 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe5c80ff0f0 tx=0x7fe5b0005fb0 comp rx=0 tx=0).stop 2023-07-21T18:45:31.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.088+0000 7fe5cd953700 1 -- 172.21.15.39:0/2753874635 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5c8068420 msgr2=0x7fe5c819bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:45:31.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.088+0000 7fe5cd953700 1 --2- 172.21.15.39:0/2753874635 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5c8068420 0x7fe5c819bc10 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fe5b800e9a0 tx=0x7fe5b800ecb0 comp rx=0 tx=0).stop 2023-07-21T18:45:31.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.088+0000 7fe5cd953700 1 -- 172.21.15.39:0/2753874635 shutdown_connections 2023-07-21T18:45:31.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.088+0000 7fe5cd953700 1 --2- 172.21.15.39:0/2753874635 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5c810a120 0x7fe5c8195a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:31.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.088+0000 7fe5cd953700 1 --2- 172.21.15.39:0/2753874635 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5c8068420 0x7fe5c819bc10 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:31.109 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.088+0000 7fe5cd953700 1 --2- 172.21.15.39:0/2753874635 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fe5b40a12c0 0x7fe5b40a3770 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:31.109 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.088+0000 7fe5cd953700 1 --2- 172.21.15.39:0/2753874635 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5c80ff250 0x7fe5c819c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:31.109 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.088+0000 7fe5cd953700 1 -- 172.21.15.39:0/2753874635 >> 172.21.15.39:0/2753874635 conn(0x7fe5c80752a0 msgr2=0x7fe5c8102030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:45:31.109 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.088+0000 7fe5cd953700 1 -- 172.21.15.39:0/2753874635 shutdown_connections 2023-07-21T18:45:31.109 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:31.088+0000 7fe5cd953700 1 -- 172.21.15.39:0/2753874635 wait complete. 2023-07-21T18:45:31.269 INFO:tasks.workunit.client.0.smithi039.stdout: 1 174115 2.60 MB/sec execute 44 sec latency 4948.933 ms 2023-07-21T18:45:32.269 INFO:tasks.workunit.client.0.smithi039.stdout: 1 176176 2.70 MB/sec execute 45 sec latency 9.826 ms 2023-07-21T18:45:32.614 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:45:32 smithi193 ceph-mon[98204]: pgmap v117: 97 pgs: 97 active+clean; 69 MiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 1.8 MiB/s wr, 162 op/s 2023-07-21T18:45:32.633 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:45:32 smithi155 ceph-mon[97934]: pgmap v117: 97 pgs: 97 active+clean; 69 MiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 1.8 MiB/s wr, 162 op/s 2023-07-21T18:45:32.682 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:45:32 smithi039 ceph-mon[96489]: pgmap v117: 97 pgs: 97 active+clean; 69 MiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 1.8 MiB/s wr, 162 op/s 2023-07-21T18:45:33.269 INFO:tasks.workunit.client.0.smithi039.stdout: 1 177504 2.88 MB/sec execute 46 sec latency 342.218 ms 2023-07-21T18:45:34.269 INFO:tasks.workunit.client.0.smithi039.stdout: 1 177504 2.82 MB/sec execute 47 sec latency 1342.305 ms 2023-07-21T18:45:34.614 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:45:34 smithi193 ceph-mon[98204]: pgmap v118: 97 pgs: 97 active+clean; 69 MiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 805 KiB/s wr, 76 op/s 2023-07-21T18:45:34.633 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:45:34 smithi155 ceph-mon[97934]: pgmap v118: 97 pgs: 97 active+clean; 69 MiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 805 KiB/s wr, 76 op/s 2023-07-21T18:45:34.682 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:45:34 smithi039 ceph-mon[96489]: pgmap v118: 97 pgs: 97 active+clean; 69 MiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 805 KiB/s wr, 76 op/s 2023-07-21T18:45:35.269 INFO:tasks.workunit.client.0.smithi039.stdout: 1 177504 2.76 MB/sec execute 48 sec latency 2342.395 ms 2023-07-21T18:45:36.269 INFO:tasks.workunit.client.0.smithi039.stdout: 1 177631 2.72 MB/sec execute 49 sec latency 3148.681 ms 2023-07-21T18:45:36.614 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:45:36 smithi193 ceph-mon[98204]: pgmap v119: 97 pgs: 97 active+clean; 69 MiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 805 KiB/s wr, 76 op/s 2023-07-21T18:45:36.633 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:45:36 smithi155 ceph-mon[97934]: pgmap v119: 97 pgs: 97 active+clean; 69 MiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 805 KiB/s wr, 76 op/s 2023-07-21T18:45:36.682 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:45:36 smithi039 ceph-mon[96489]: pgmap v119: 97 pgs: 97 active+clean; 69 MiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 805 KiB/s wr, 76 op/s 2023-07-21T18:45:37.269 INFO:tasks.workunit.client.0.smithi039.stdout: 1 179282 2.79 MB/sec execute 50 sec latency 12.063 ms 2023-07-21T18:45:38.269 INFO:tasks.workunit.client.0.smithi039.stdout: 1 180918 2.97 MB/sec execute 51 sec latency 345.512 ms 2023-07-21T18:45:38.614 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:45:38 smithi193 ceph-mon[98204]: pgmap v120: 97 pgs: 97 active+clean; 71 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 1.1 MiB/s wr, 129 op/s 2023-07-21T18:45:38.633 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:45:38 smithi155 ceph-mon[97934]: pgmap v120: 97 pgs: 97 active+clean; 71 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 1.1 MiB/s wr, 129 op/s 2023-07-21T18:45:38.682 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:45:38 smithi039 ceph-mon[96489]: pgmap v120: 97 pgs: 97 active+clean; 71 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 1.1 MiB/s wr, 129 op/s 2023-07-21T18:45:39.269 INFO:tasks.workunit.client.0.smithi039.stdout: 1 180918 2.91 MB/sec execute 52 sec latency 1345.621 ms 2023-07-21T18:45:40.269 INFO:tasks.workunit.client.0.smithi039.stdout: 1 180918 2.85 MB/sec execute 53 sec latency 2345.708 ms 2023-07-21T18:45:40.614 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:45:40 smithi193 ceph-mon[98204]: pgmap v121: 97 pgs: 97 active+clean; 71 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 406 KiB/s wr, 68 op/s 2023-07-21T18:45:40.615 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:45:40 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:45:40.633 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:45:40 smithi155 ceph-mon[97934]: pgmap v121: 97 pgs: 97 active+clean; 71 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 406 KiB/s wr, 68 op/s 2023-07-21T18:45:40.633 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:45:40 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:45:40.682 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:45:40 smithi039 ceph-mon[96489]: pgmap v121: 97 pgs: 97 active+clean; 71 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 406 KiB/s wr, 68 op/s 2023-07-21T18:45:40.682 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:45:40 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:45:41.269 INFO:tasks.workunit.client.0.smithi039.stdout: 1 181050 2.81 MB/sec execute 54 sec latency 3151.083 ms 2023-07-21T18:45:42.269 INFO:tasks.workunit.client.0.smithi039.stdout: 1 181050 2.76 MB/sec execute 55 sec latency 1096.004 ms 2023-07-21T18:45:42.614 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:45:42 smithi193 ceph-mon[98204]: pgmap v122: 97 pgs: 97 active+clean; 74 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 538 KiB/s rd, 908 KiB/s wr, 124 op/s 2023-07-21T18:45:42.633 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:45:42 smithi155 ceph-mon[97934]: pgmap v122: 97 pgs: 97 active+clean; 74 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 538 KiB/s rd, 908 KiB/s wr, 124 op/s 2023-07-21T18:45:42.682 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:45:42 smithi039 ceph-mon[96489]: pgmap v122: 97 pgs: 97 active+clean; 74 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 538 KiB/s rd, 908 KiB/s wr, 124 op/s 2023-07-21T18:45:43.269 INFO:tasks.workunit.client.0.smithi039.stdout: 1 181050 2.71 MB/sec execute 56 sec latency 2096.059 ms 2023-07-21T18:45:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.210+0000 7fdf519bc700 1 -- 172.21.15.39:0/282521771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf4c100db0 msgr2=0x7fdf4c101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:45:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.210+0000 7fdf519bc700 1 --2- 172.21.15.39:0/282521771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf4c100db0 0x7fdf4c101180 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c009990 tx=0x7fdf3c009ca0 comp rx=0 tx=0).stop 2023-07-21T18:45:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.211+0000 7fdf519bc700 1 -- 172.21.15.39:0/282521771 shutdown_connections 2023-07-21T18:45:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.211+0000 7fdf519bc700 1 --2- 172.21.15.39:0/282521771 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf4c10a180 0x7fdf4c10c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.211+0000 7fdf519bc700 1 --2- 172.21.15.39:0/282521771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf4c100db0 0x7fdf4c101180 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.211+0000 7fdf519bc700 1 --2- 172.21.15.39:0/282521771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf4c1016c0 0x7fdf4c109bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.211+0000 7fdf519bc700 1 -- 172.21.15.39:0/282521771 >> 172.21.15.39:0/282521771 conn(0x7fdf4c075bd0 msgr2=0x7fdf4c075fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:45:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.211+0000 7fdf519bc700 1 -- 172.21.15.39:0/282521771 shutdown_connections 2023-07-21T18:45:44.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.211+0000 7fdf519bc700 1 -- 172.21.15.39:0/282521771 wait complete. 2023-07-21T18:45:44.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.211+0000 7fdf519bc700 1 Processor -- start 2023-07-21T18:45:44.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.211+0000 7fdf519bc700 1 -- start start 2023-07-21T18:45:44.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.212+0000 7fdf519bc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf4c100db0 0x7fdf4c19bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:44.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.212+0000 7fdf519bc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf4c1016c0 0x7fdf4c19c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:44.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.212+0000 7fdf519bc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf4c10a180 0x7fdf4c195a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:44.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.212+0000 7fdf519bc700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4c196090 con 0x7fdf4c10a180 2023-07-21T18:45:44.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.212+0000 7fdf519bc700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4c196200 con 0x7fdf4c1016c0 2023-07-21T18:45:44.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.212+0000 7fdf519bc700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4c1963c0 con 0x7fdf4c100db0 2023-07-21T18:45:44.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.212+0000 7fdf4b7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf4c10a180 0x7fdf4c195a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:44.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.212+0000 7fdf4b7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf4c10a180 0x7fdf4c195a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59536/0 (socket says 172.21.15.39:59536) 2023-07-21T18:45:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.212+0000 7fdf4b7fe700 1 -- 172.21.15.39:0/483381153 learned_addr learned my addr 172.21.15.39:0/483381153 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:45:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.212+0000 7fdf4affd700 1 --2- 172.21.15.39:0/483381153 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf4c100db0 0x7fdf4c19bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.212+0000 7fdf4a7fc700 1 --2- 172.21.15.39:0/483381153 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf4c1016c0 0x7fdf4c19c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.212+0000 7fdf4b7fe700 1 -- 172.21.15.39:0/483381153 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf4c100db0 msgr2=0x7fdf4c19bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:45:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.212+0000 7fdf4b7fe700 1 --2- 172.21.15.39:0/483381153 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf4c100db0 0x7fdf4c19bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.212+0000 7fdf4b7fe700 1 -- 172.21.15.39:0/483381153 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf4c1016c0 msgr2=0x7fdf4c19c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:45:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.212+0000 7fdf4b7fe700 1 --2- 172.21.15.39:0/483381153 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf4c1016c0 0x7fdf4c19c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.212+0000 7fdf4b7fe700 1 -- 172.21.15.39:0/483381153 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf3c0095f0 con 0x7fdf4c10a180 2023-07-21T18:45:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.212+0000 7fdf4b7fe700 1 --2- 172.21.15.39:0/483381153 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf4c10a180 0x7fdf4c195a20 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fdf440094f0 tx=0x7fdf4400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:45:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.213+0000 7fdf509ba700 1 -- 172.21.15.39:0/483381153 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4400ddd0 con 0x7fdf4c10a180 2023-07-21T18:45:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.213+0000 7fdf509ba700 1 -- 172.21.15.39:0/483381153 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf4400f040 con 0x7fdf4c10a180 2023-07-21T18:45:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.213+0000 7fdf519bc700 1 -- 172.21.15.39:0/483381153 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf4c1966a0 con 0x7fdf4c10a180 2023-07-21T18:45:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.213+0000 7fdf509ba700 1 -- 172.21.15.39:0/483381153 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf44015400 con 0x7fdf4c10a180 2023-07-21T18:45:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.213+0000 7fdf519bc700 1 -- 172.21.15.39:0/483381153 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf4c1a2bb0 con 0x7fdf4c10a180 2023-07-21T18:45:44.235 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.216+0000 7fdf509ba700 1 -- 172.21.15.39:0/483381153 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fdf44004c40 con 0x7fdf4c10a180 2023-07-21T18:45:44.235 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.216+0000 7fdf509ba700 1 --2- 172.21.15.39:0/483381153 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fdf340a1340 0x7fdf340a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:44.235 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.217+0000 7fdf4affd700 1 --2- 172.21.15.39:0/483381153 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fdf340a1340 0x7fdf340a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:44.235 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.217+0000 7fdf509ba700 1 -- 172.21.15.39:0/483381153 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fdf440cc5c0 con 0x7fdf4c10a180 2023-07-21T18:45:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.217+0000 7fdf5db42b80 1 Processor -- start 2023-07-21T18:45:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.217+0000 7fdf5db42b80 1 -- start start 2023-07-21T18:45:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.217+0000 7fdf4affd700 1 --2- 172.21.15.39:0/483381153 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fdf340a1340 0x7fdf340a37f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c00b5c0 tx=0x7fdf3c005fd0 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:45:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.217+0000 7fdf5db42b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c71ddf6b0 0x557c71f88d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.217+0000 7fdf5db42b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c71f892d0 0x557c71f83040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.217+0000 7fdf5db42b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c71f83580 0x557c71f839d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.217+0000 7fdf5db42b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c71e88c50 con 0x557c71f892d0 2023-07-21T18:45:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.217+0000 7fdf5db42b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c71edb9e0 con 0x557c71f83580 2023-07-21T18:45:44.237 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.217+0000 7fdf5db42b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c71ed4e70 con 0x557c71ddf6b0 2023-07-21T18:45:44.237 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.217+0000 7fdf4a7fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c71f892d0 0x557c71f83040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:44.237 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.217+0000 7fdf4a7fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c71f892d0 0x557c71f83040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59542/0 (socket says 172.21.15.39:59542) 2023-07-21T18:45:44.237 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.217+0000 7fdf4a7fc700 1 -- 172.21.15.39:0/2547296987 learned_addr learned my addr 172.21.15.39:0/2547296987 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:45:44.237 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.218+0000 7fdf4b7fe700 1 --2- 172.21.15.39:0/2547296987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c71f83580 0x557c71f839d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:44.237 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.218+0000 7fdf4affd700 1 --2- 172.21.15.39:0/2547296987 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c71ddf6b0 0x557c71f88d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:44.237 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.218+0000 7fdf4a7fc700 1 -- 172.21.15.39:0/2547296987 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c71ddf6b0 msgr2=0x557c71f88d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:45:44.238 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.218+0000 7fdf4a7fc700 1 --2- 172.21.15.39:0/2547296987 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c71ddf6b0 0x557c71f88d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:44.238 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.218+0000 7fdf4a7fc700 1 -- 172.21.15.39:0/2547296987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c71f83580 msgr2=0x557c71f839d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:45:44.238 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.218+0000 7fdf4a7fc700 1 --2- 172.21.15.39:0/2547296987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c71f83580 0x557c71f839d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:44.238 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.218+0000 7fdf4a7fc700 1 -- 172.21.15.39:0/2547296987 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c71d02d80 con 0x557c71f892d0 2023-07-21T18:45:44.238 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.218+0000 7fdf4a7fc700 1 --2- 172.21.15.39:0/2547296987 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c71f892d0 0x557c71f83040 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fdf4000bd10 tx=0x7fdf4000f640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfff576b698ebdd4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:45:44.238 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.218+0000 7fdf31ffb700 1 -- 172.21.15.39:0/2547296987 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4000fda0 con 0x557c71f892d0 2023-07-21T18:45:44.238 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.218+0000 7fdf31ffb700 1 -- 172.21.15.39:0/2547296987 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf4000be60 con 0x557c71f892d0 2023-07-21T18:45:44.238 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.218+0000 7fdf31ffb700 1 -- 172.21.15.39:0/2547296987 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf400107f0 con 0x557c71f892d0 2023-07-21T18:45:44.241 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.219+0000 7fdf5db42b80 1 -- 172.21.15.39:0/2547296987 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c71f892d0 msgr2=0x557c71f83040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:45:44.242 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.219+0000 7fdf5db42b80 1 --2- 172.21.15.39:0/2547296987 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c71f892d0 0x557c71f83040 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fdf4000bd10 tx=0x7fdf4000f640 comp rx=0 tx=0).stop 2023-07-21T18:45:44.242 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.219+0000 7fdf5db42b80 1 -- 172.21.15.39:0/2547296987 shutdown_connections 2023-07-21T18:45:44.242 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.219+0000 7fdf5db42b80 1 --2- 172.21.15.39:0/2547296987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c71f83580 0x557c71f839d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:44.242 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.219+0000 7fdf5db42b80 1 --2- 172.21.15.39:0/2547296987 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c71ddf6b0 0x557c71f88d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:44.242 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.219+0000 7fdf5db42b80 1 --2- 172.21.15.39:0/2547296987 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c71f892d0 0x557c71f83040 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:44.242 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.219+0000 7fdf5db42b80 1 -- 172.21.15.39:0/2547296987 >> 172.21.15.39:0/2547296987 conn(0x557c71f78be0 msgr2=0x557c71f7df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:45:44.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.219+0000 7fdf5db42b80 1 -- 172.21.15.39:0/2547296987 shutdown_connections 2023-07-21T18:45:44.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.219+0000 7fdf5db42b80 1 -- 172.21.15.39:0/2547296987 wait complete. 2023-07-21T18:45:44.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.219+0000 7fdf5db42b80 1 -- start start 2023-07-21T18:45:44.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.220+0000 7fdf5db42b80 1 Processor -- start 2023-07-21T18:45:44.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.220+0000 7fdf5db42b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c71ddf6b0 0x557c71f8f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:44.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.220+0000 7fdf5db42b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c71f83580 0x557c71f8c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:44.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.220+0000 7fdf5db42b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c71f892d0 0x557c71f8ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:44.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.220+0000 7fdf5db42b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c71e74a60 con 0x557c71f892d0 2023-07-21T18:45:44.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.220+0000 7fdf5db42b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c71e74760 con 0x557c71f83580 2023-07-21T18:45:44.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.220+0000 7fdf5db42b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c71d588c0 con 0x557c71ddf6b0 2023-07-21T18:45:44.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.220+0000 7fdf5db42b80 10 client.0 _ll_register_callbacks cb 0x557c71e72700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:45:44.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.220+0000 7fdf4b7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c71f892d0 0x557c71f8ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:44.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.220+0000 7fdf4b7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c71f892d0 0x557c71f8ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59550/0 (socket says 172.21.15.39:59550) 2023-07-21T18:45:44.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.220+0000 7fdf4affd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c71ddf6b0 0x557c71f8f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:44.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.220+0000 7fdf4b7fe700 1 -- 172.21.15.39:0/654252281 learned_addr learned my addr 172.21.15.39:0/654252281 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:45:44.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.220+0000 7fdf4a7fc700 1 --2- 172.21.15.39:0/654252281 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c71f83580 0x557c71f8c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:44.245 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.220+0000 7fdf4b7fe700 1 -- 172.21.15.39:0/654252281 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c71ddf6b0 msgr2=0x557c71f8f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:45:44.245 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.220+0000 7fdf4b7fe700 1 --2- 172.21.15.39:0/654252281 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c71ddf6b0 0x557c71f8f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:44.245 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.220+0000 7fdf4b7fe700 1 -- 172.21.15.39:0/654252281 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c71f83580 msgr2=0x557c71f8c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:45:44.245 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.220+0000 7fdf4b7fe700 1 --2- 172.21.15.39:0/654252281 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c71f83580 0x557c71f8c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:44.245 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.221+0000 7fdf4b7fe700 1 -- 172.21.15.39:0/654252281 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf4000b940 con 0x557c71f892d0 2023-07-21T18:45:44.245 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.221+0000 7fdf4affd700 1 --2- 172.21.15.39:0/654252281 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c71ddf6b0 0x557c71f8f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:45:44.245 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.221+0000 7fdf4b7fe700 1 --2- 172.21.15.39:0/654252281 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c71f892d0 0x557c71f8ce80 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fdf4400d7b0 tx=0x7fdf44099300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c833d518e50ac213 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:45:44.246 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.221+0000 7fdf2affd700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:45:44.246 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.221+0000 7fdf2affd700 1 -- 172.21.15.39:0/654252281 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4409d6e0 con 0x557c71f892d0 2023-07-21T18:45:44.246 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.221+0000 7fdf2affd700 1 -- 172.21.15.39:0/654252281 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fdf440a1b00 con 0x557c71f892d0 2023-07-21T18:45:44.246 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.221+0000 7fdf5db42b80 1 -- 172.21.15.39:0/654252281 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557c71e2e490 con 0x557c71f892d0 2023-07-21T18:45:44.246 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.221+0000 7fdf2affd700 1 -- 172.21.15.39:0/654252281 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf44098ba0 con 0x557c71f892d0 2023-07-21T18:45:44.246 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.221+0000 7fdf5db42b80 10 client.16425 fetch_fsmap learned FSMap version 41 2023-07-21T18:45:44.246 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.221+0000 7fdf5db42b80 1 -- 172.21.15.39:0/654252281 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x557c71cebc00 con 0x557c71f892d0 2023-07-21T18:45:44.246 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.221+0000 7fdf2affd700 1 -- 172.21.15.39:0/654252281 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557c71e2e490 con 0x557c71f892d0 2023-07-21T18:45:44.247 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.221+0000 7fdf2affd700 1 -- 172.21.15.39:0/654252281 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 41) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fdf440b7020 con 0x557c71f892d0 2023-07-21T18:45:44.247 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.221+0000 7fdf5db42b80 10 client.16425 fetch_fsmap finished waiting for FSMap version 41 2023-07-21T18:45:44.247 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.221+0000 7fdf5db42b80 10 client.16425 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:45:44.247 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.221+0000 7fdf5db42b80 20 client.16425 populate_metadata read hostname 'smithi039' 2023-07-21T18:45:44.247 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.221+0000 7fdf5db42b80 1 --2- 172.21.15.39:0/654252281 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x557c71f9cb00 0x557c71f9ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:44.247 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.221+0000 7fdf5db42b80 4 client.16425 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:45:44.247 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.221+0000 7fdf5db42b80 1 -- 172.21.15.39:0/654252281 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557c71ecde50 con 0x557c71f9cb00 2023-07-21T18:45:44.247 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.222+0000 7fdf4affd700 1 --2- 172.21.15.39:0/654252281 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x557c71f9cb00 0x557c71f9ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.222+0000 7fdf2affd700 10 client.16425 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:45:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.222+0000 7fdf4affd700 1 --2- 172.21.15.39:0/654252281 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x557c71f9cb00 0x557c71f9ef10 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86be91adf7a47c0e server_cookie=591635b11de6bbca in_seq=0 out_seq=0 2023-07-21T18:45:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.223+0000 7fdf2affd700 1 -- 172.21.15.39:0/654252281 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557c71ecde50 con 0x557c71f9cb00 2023-07-21T18:45:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.223+0000 7fdf2affd700 10 client.16425 handle_command_reply: tid=0 2023-07-21T18:45:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.223+0000 7fdf5db42b80 1 client.16425 shutdown 2023-07-21T18:45:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.223+0000 7fdf5db42b80 1 -- 172.21.15.39:0/654252281 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c71f892d0 msgr2=0x557c71f8ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:45:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.223+0000 7fdf5db42b80 1 --2- 172.21.15.39:0/654252281 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c71f892d0 0x557c71f8ce80 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fdf4400d7b0 tx=0x7fdf44099300 comp rx=0 tx=0).stop 2023-07-21T18:45:44.249 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.223+0000 7fdf5db42b80 1 -- 172.21.15.39:0/654252281 shutdown_connections 2023-07-21T18:45:44.249 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.223+0000 7fdf5db42b80 1 --2- 172.21.15.39:0/654252281 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x557c71f9cb00 0x557c71f9ef10 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:44.249 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.223+0000 7fdf5db42b80 1 --2- 172.21.15.39:0/654252281 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c71f892d0 0x557c71f8ce80 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:44.249 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.223+0000 7fdf5db42b80 1 --2- 172.21.15.39:0/654252281 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c71ddf6b0 0x557c71f8f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:44.249 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.223+0000 7fdf5db42b80 1 --2- 172.21.15.39:0/654252281 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c71f83580 0x557c71f8c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:44.249 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.223+0000 7fdf2affd700 0 client.16425 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:45:44.249 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.223+0000 7fdf5db42b80 1 -- 172.21.15.39:0/654252281 >> 172.21.15.39:0/654252281 conn(0x557c71f78be0 msgr2=0x557c71f7d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:45:44.249 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.223+0000 7fdf5db42b80 1 -- 172.21.15.39:0/654252281 shutdown_connections 2023-07-21T18:45:44.250 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.223+0000 7fdf5db42b80 1 -- 172.21.15.39:0/654252281 wait complete. 2023-07-21T18:45:44.250 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.223+0000 7fdf5db42b80 20 client.16425 trim_cache size 0 max 16384 2023-07-21T18:45:44.255 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.238+0000 7fdf5db42b80 1 Processor -- start 2023-07-21T18:45:44.255 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.238+0000 7fdf5db42b80 1 -- start start 2023-07-21T18:45:44.257 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.239+0000 7fdf5db42b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c71ddf6b0 0x557c71f83040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:44.257 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.239+0000 7fdf5db42b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c71f83580 0x557c71f882a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:44.257 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.239+0000 7fdf5db42b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c71f892d0 0x557c71f93ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:44.257 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.239+0000 7fdf5db42b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c71f7f740 con 0x557c71f892d0 2023-07-21T18:45:44.257 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.239+0000 7fdf5db42b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c71f7fb70 con 0x557c71ddf6b0 2023-07-21T18:45:44.258 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.239+0000 7fdf5db42b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c71f8dbf0 con 0x557c71f83580 2023-07-21T18:45:44.258 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.239+0000 7fdf4b7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c71f892d0 0x557c71f93ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:44.258 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.239+0000 7fdf4b7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c71f892d0 0x557c71f93ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59558/0 (socket says 172.21.15.39:59558) 2023-07-21T18:45:44.258 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.239+0000 7fdf4b7fe700 1 -- 172.21.15.39:0/151676294 learned_addr learned my addr 172.21.15.39:0/151676294 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:45:44.258 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.239+0000 7fdf4a7fc700 1 --2- 172.21.15.39:0/151676294 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c71f83580 0x557c71f882a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:44.258 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.239+0000 7fdf4b7fe700 1 -- 172.21.15.39:0/151676294 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c71f83580 msgr2=0x557c71f882a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:45:44.258 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.239+0000 7fdf4affd700 1 --2- 172.21.15.39:0/151676294 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c71ddf6b0 0x557c71f83040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:44.258 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.239+0000 7fdf4b7fe700 1 --2- 172.21.15.39:0/151676294 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c71f83580 0x557c71f882a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:44.259 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.239+0000 7fdf4b7fe700 1 -- 172.21.15.39:0/151676294 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c71ddf6b0 msgr2=0x557c71f83040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:45:44.259 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.239+0000 7fdf4b7fe700 1 --2- 172.21.15.39:0/151676294 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c71ddf6b0 0x557c71f83040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:44.259 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.239+0000 7fdf4b7fe700 1 -- 172.21.15.39:0/151676294 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c71d291f0 con 0x557c71f892d0 2023-07-21T18:45:44.259 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.239+0000 7fdf4a7fc700 1 --2- 172.21.15.39:0/151676294 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c71f83580 0x557c71f882a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:45:44.259 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.239+0000 7fdf4affd700 1 --2- 172.21.15.39:0/151676294 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c71ddf6b0 0x557c71f83040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:45:44.259 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.239+0000 7fdf4b7fe700 1 --2- 172.21.15.39:0/151676294 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c71f892d0 0x557c71f93ca0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fdf44095bb0 tx=0x7fdf44099830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be841aa82bd6bed2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:45:44.259 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.239+0000 7fdf327fc700 1 -- 172.21.15.39:0/151676294 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf44099010 con 0x557c71f892d0 2023-07-21T18:45:44.260 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.240+0000 7fdf327fc700 1 -- 172.21.15.39:0/151676294 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf440a1b00 con 0x557c71f892d0 2023-07-21T18:45:44.260 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.240+0000 7fdf327fc700 1 -- 172.21.15.39:0/151676294 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf44098de0 con 0x557c71f892d0 2023-07-21T18:45:44.260 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.240+0000 7fdf5db42b80 1 -- 172.21.15.39:0/151676294 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c71f892d0 msgr2=0x557c71f93ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:45:44.260 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.240+0000 7fdf5db42b80 1 --2- 172.21.15.39:0/151676294 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c71f892d0 0x557c71f93ca0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fdf44095bb0 tx=0x7fdf44099830 comp rx=0 tx=0).stop 2023-07-21T18:45:44.260 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.240+0000 7fdf5db42b80 1 -- 172.21.15.39:0/151676294 shutdown_connections 2023-07-21T18:45:44.260 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.240+0000 7fdf5db42b80 1 --2- 172.21.15.39:0/151676294 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c71ddf6b0 0x557c71f83040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:44.260 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.240+0000 7fdf5db42b80 1 --2- 172.21.15.39:0/151676294 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c71f892d0 0x557c71f93ca0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:44.260 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.240+0000 7fdf5db42b80 1 --2- 172.21.15.39:0/151676294 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c71f83580 0x557c71f882a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:44.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.240+0000 7fdf5db42b80 1 -- 172.21.15.39:0/151676294 >> 172.21.15.39:0/151676294 conn(0x557c71f78be0 msgr2=0x557c71f8b000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:45:44.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.240+0000 7fdf5db42b80 1 -- 172.21.15.39:0/151676294 shutdown_connections 2023-07-21T18:45:44.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.240+0000 7fdf5db42b80 1 -- 172.21.15.39:0/151676294 wait complete. 2023-07-21T18:45:44.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.240+0000 7fdf5db42b80 1 -- start start 2023-07-21T18:45:44.263 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.241+0000 7fdf5db42b80 1 Processor -- start 2023-07-21T18:45:44.263 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.241+0000 7fdf5db42b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c71ddf6b0 0x557c71fe9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:44.263 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.241+0000 7fdf5db42b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c71f83580 0x557c71fea120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:44.263 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.241+0000 7fdf5db42b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c71f892d0 0x557c71f8fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:44.263 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.241+0000 7fdf5db42b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c71f901c0 con 0x557c71ddf6b0 2023-07-21T18:45:44.264 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.241+0000 7fdf5db42b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c71f90330 con 0x557c71f892d0 2023-07-21T18:45:44.264 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.241+0000 7fdf5db42b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c71f904f0 con 0x557c71f83580 2023-07-21T18:45:44.264 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.241+0000 7fdf5db42b80 10 client.0 _ll_register_callbacks cb 0x7fdf440994a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:45:44.264 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.241+0000 7fdf4affd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c71ddf6b0 0x557c71fe9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:44.264 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.241+0000 7fdf4affd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c71ddf6b0 0x557c71fe9be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59560/0 (socket says 172.21.15.39:59560) 2023-07-21T18:45:44.264 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.241+0000 7fdf4affd700 1 -- 172.21.15.39:0/3571563276 learned_addr learned my addr 172.21.15.39:0/3571563276 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:45:44.264 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.241+0000 7fdf4a7fc700 1 --2- 172.21.15.39:0/3571563276 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c71f83580 0x557c71fea120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:44.265 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.241+0000 7fdf4b7fe700 1 --2- 172.21.15.39:0/3571563276 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c71f892d0 0x557c71f8fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:44.265 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.242+0000 7fdf4affd700 1 -- 172.21.15.39:0/3571563276 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c71f83580 msgr2=0x557c71fea120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:45:44.265 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.242+0000 7fdf4affd700 1 --2- 172.21.15.39:0/3571563276 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c71f83580 0x557c71fea120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:44.265 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.242+0000 7fdf4affd700 1 -- 172.21.15.39:0/3571563276 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c71f892d0 msgr2=0x557c71f8fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:45:44.265 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.242+0000 7fdf4affd700 1 --2- 172.21.15.39:0/3571563276 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c71f892d0 0x557c71f8fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:44.265 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.242+0000 7fdf4affd700 1 -- 172.21.15.39:0/3571563276 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557c71cebc00 con 0x557c71ddf6b0 2023-07-21T18:45:44.265 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.242+0000 7fdf4affd700 1 --2- 172.21.15.39:0/3571563276 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c71ddf6b0 0x557c71fe9be0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c005f30 tx=0x7fdf3c020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f47470eebf76bef9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:45:44.265 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.242+0000 7fdf2affd700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:45:44.266 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.242+0000 7fdf2affd700 1 -- 172.21.15.39:0/3571563276 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c033a50 con 0x557c71ddf6b0 2023-07-21T18:45:44.266 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.242+0000 7fdf2affd700 1 -- 172.21.15.39:0/3571563276 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fdf3c033bb0 con 0x557c71ddf6b0 2023-07-21T18:45:44.266 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.242+0000 7fdf5db42b80 1 -- 172.21.15.39:0/3571563276 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdf4000be60 con 0x557c71ddf6b0 2023-07-21T18:45:44.266 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.242+0000 7fdf2affd700 1 -- 172.21.15.39:0/3571563276 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf3c033d70 con 0x557c71ddf6b0 2023-07-21T18:45:44.266 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.242+0000 7fdf5db42b80 10 client.16431 fetch_fsmap learned FSMap version 41 2023-07-21T18:45:44.266 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.242+0000 7fdf5db42b80 1 -- 172.21.15.39:0/3571563276 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x557c71d01250 con 0x557c71ddf6b0 2023-07-21T18:45:44.266 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.242+0000 7fdf2affd700 1 -- 172.21.15.39:0/3571563276 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf3c040020 con 0x557c71ddf6b0 2023-07-21T18:45:44.267 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.242+0000 7fdf2affd700 1 -- 172.21.15.39:0/3571563276 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 41) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fdf3c0239d0 con 0x557c71ddf6b0 2023-07-21T18:45:44.267 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.242+0000 7fdf5db42b80 10 client.16431 fetch_fsmap finished waiting for FSMap version 41 2023-07-21T18:45:44.267 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.242+0000 7fdf5db42b80 10 client.16431 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:45:44.267 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.242+0000 7fdf5db42b80 20 client.16431 populate_metadata read hostname 'smithi039' 2023-07-21T18:45:44.267 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.243+0000 7fdf5db42b80 1 --2- 172.21.15.39:0/3571563276 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x557c71ff18c0 0x557c71ff3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:44.267 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.243+0000 7fdf5db42b80 4 client.16431 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:45:44.267 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.243+0000 7fdf5db42b80 1 -- 172.21.15.39:0/3571563276 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557c71e8fff0 con 0x557c71ff18c0 2023-07-21T18:45:44.267 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.243+0000 7fdf4a7fc700 1 --2- 172.21.15.39:0/3571563276 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x557c71ff18c0 0x557c71ff3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:44.268 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.243+0000 7fdf4a7fc700 1 --2- 172.21.15.39:0/3571563276 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x557c71ff18c0 0x557c71ff3cd0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ca8eea9daf97e7 server_cookie=66891d794db92165 in_seq=0 out_seq=0 2023-07-21T18:45:44.268 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.243+0000 7fdf2affd700 10 client.16431 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:45:44.269 INFO:tasks.workunit.client.0.smithi039.stdout: 1 181050 2.66 MB/sec execute 57 sec latency 3096.110 ms 2023-07-21T18:45:44.290 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.273+0000 7fdf2affd700 1 -- 172.21.15.39:0/3571563276 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557c71e8fff0 con 0x557c71ff18c0 2023-07-21T18:45:44.290 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.273+0000 7fdf2affd700 10 client.16431 handle_command_reply: tid=0 2023-07-21T18:45:44.290 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.273+0000 7fdf5db42b80 1 client.16431 shutdown 2023-07-21T18:45:44.290 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.273+0000 7fdf5db42b80 1 -- 172.21.15.39:0/3571563276 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c71ddf6b0 msgr2=0x557c71fe9be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:45:44.290 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.273+0000 7fdf5db42b80 1 --2- 172.21.15.39:0/3571563276 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c71ddf6b0 0x557c71fe9be0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c005f30 tx=0x7fdf3c020000 comp rx=0 tx=0).stop 2023-07-21T18:45:44.290 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.273+0000 7fdf5db42b80 1 -- 172.21.15.39:0/3571563276 shutdown_connections 2023-07-21T18:45:44.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.273+0000 7fdf5db42b80 1 --2- 172.21.15.39:0/3571563276 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x557c71ff18c0 0x557c71ff3cd0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:44.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.273+0000 7fdf5db42b80 1 --2- 172.21.15.39:0/3571563276 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c71f892d0 0x557c71f8fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:44.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.273+0000 7fdf2affd700 0 client.16431 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:45:44.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.273+0000 7fdf5db42b80 1 --2- 172.21.15.39:0/3571563276 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c71ddf6b0 0x557c71fe9be0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:44.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.273+0000 7fdf5db42b80 1 --2- 172.21.15.39:0/3571563276 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c71f83580 0x557c71fea120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:44.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.273+0000 7fdf5db42b80 1 -- 172.21.15.39:0/3571563276 >> 172.21.15.39:0/3571563276 conn(0x557c71f78be0 msgr2=0x557c71f8a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:45:44.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.273+0000 7fdf5db42b80 1 -- 172.21.15.39:0/3571563276 shutdown_connections 2023-07-21T18:45:44.292 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.273+0000 7fdf5db42b80 1 -- 172.21.15.39:0/3571563276 wait complete. 2023-07-21T18:45:44.292 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.273+0000 7fdf5db42b80 20 client.16431 trim_cache size 0 max 16384 2023-07-21T18:45:44.292 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:45:44.292 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:45:44.292 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:45:44.293 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:45:44.293 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.274+0000 7fdf519bc700 1 -- 172.21.15.39:0/483381153 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fdf340a1340 msgr2=0x7fdf340a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:45:44.293 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.274+0000 7fdf519bc700 1 --2- 172.21.15.39:0/483381153 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fdf340a1340 0x7fdf340a37f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c00b5c0 tx=0x7fdf3c005fd0 comp rx=0 tx=0).stop 2023-07-21T18:45:44.294 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.274+0000 7fdf519bc700 1 -- 172.21.15.39:0/483381153 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf4c10a180 msgr2=0x7fdf4c195a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:45:44.294 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.274+0000 7fdf519bc700 1 --2- 172.21.15.39:0/483381153 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf4c10a180 0x7fdf4c195a20 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fdf440094f0 tx=0x7fdf4400d3b0 comp rx=0 tx=0).stop 2023-07-21T18:45:44.294 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.274+0000 7fdf519bc700 1 -- 172.21.15.39:0/483381153 shutdown_connections 2023-07-21T18:45:44.294 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.274+0000 7fdf519bc700 1 --2- 172.21.15.39:0/483381153 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf4c10a180 0x7fdf4c195a20 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:44.294 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.274+0000 7fdf519bc700 1 --2- 172.21.15.39:0/483381153 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf4c100db0 0x7fdf4c19bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:44.294 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.274+0000 7fdf519bc700 1 --2- 172.21.15.39:0/483381153 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fdf340a1340 0x7fdf340a37f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:44.294 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.274+0000 7fdf519bc700 1 --2- 172.21.15.39:0/483381153 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf4c1016c0 0x7fdf4c19c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:44.294 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.274+0000 7fdf519bc700 1 -- 172.21.15.39:0/483381153 >> 172.21.15.39:0/483381153 conn(0x7fdf4c075bd0 msgr2=0x7fdf4c101b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:45:44.295 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.274+0000 7fdf519bc700 1 -- 172.21.15.39:0/483381153 shutdown_connections 2023-07-21T18:45:44.295 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:44.274+0000 7fdf519bc700 1 -- 172.21.15.39:0/483381153 wait complete. 2023-07-21T18:45:44.614 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:45:44 smithi193 ceph-mon[98204]: pgmap v123: 97 pgs: 97 active+clean; 74 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 828 KiB/s wr, 108 op/s 2023-07-21T18:45:44.633 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:45:44 smithi155 ceph-mon[97934]: pgmap v123: 97 pgs: 97 active+clean; 74 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 828 KiB/s wr, 108 op/s 2023-07-21T18:45:44.682 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:45:44 smithi039 ceph-mon[96489]: pgmap v123: 97 pgs: 97 active+clean; 74 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 828 KiB/s wr, 108 op/s 2023-07-21T18:45:45.269 INFO:tasks.workunit.client.0.smithi039.stdout: 1 181050 2.61 MB/sec execute 58 sec latency 4096.145 ms 2023-07-21T18:45:46.269 INFO:tasks.workunit.client.0.smithi039.stdout: 1 181210 2.59 MB/sec execute 59 sec latency 4900.578 ms 2023-07-21T18:45:46.614 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:45:46 smithi193 ceph-mon[98204]: pgmap v124: 97 pgs: 97 active+clean; 74 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 828 KiB/s wr, 108 op/s 2023-07-21T18:45:46.633 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:45:46 smithi155 ceph-mon[97934]: pgmap v124: 97 pgs: 97 active+clean; 74 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 828 KiB/s wr, 108 op/s 2023-07-21T18:45:46.682 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:45:46 smithi039 ceph-mon[96489]: pgmap v124: 97 pgs: 97 active+clean; 74 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 828 KiB/s wr, 108 op/s 2023-07-21T18:45:47.269 INFO:tasks.workunit.client.0.smithi039.stdout: 1 183204 2.66 MB/sec execute 60 sec latency 19.400 ms 2023-07-21T18:45:48.269 INFO:tasks.workunit.client.0.smithi039.stdout: 1 184814 2.82 MB/sec execute 61 sec latency 42.412 ms 2023-07-21T18:45:48.614 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:45:48 smithi193 ceph-mon[98204]: pgmap v125: 97 pgs: 97 active+clean; 70 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 876 KiB/s wr, 130 op/s 2023-07-21T18:45:48.633 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:45:48 smithi155 ceph-mon[97934]: pgmap v125: 97 pgs: 97 active+clean; 70 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 876 KiB/s wr, 130 op/s 2023-07-21T18:45:48.682 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:45:48 smithi039 ceph-mon[96489]: pgmap v125: 97 pgs: 97 active+clean; 70 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 876 KiB/s wr, 130 op/s 2023-07-21T18:45:49.269 INFO:tasks.workunit.client.0.smithi039.stdout: 1 186857 2.89 MB/sec execute 62 sec latency 14.849 ms 2023-07-21T18:45:50.270 INFO:tasks.workunit.client.0.smithi039.stdout: 1 188009 3.01 MB/sec execute 63 sec latency 456.551 ms 2023-07-21T18:45:50.614 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:45:50 smithi193 ceph-mon[98204]: pgmap v126: 97 pgs: 97 active+clean; 70 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 550 KiB/s wr, 77 op/s 2023-07-21T18:45:50.633 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:45:50 smithi155 ceph-mon[97934]: pgmap v126: 97 pgs: 97 active+clean; 70 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 550 KiB/s wr, 77 op/s 2023-07-21T18:45:50.682 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:45:50 smithi039 ceph-mon[96489]: pgmap v126: 97 pgs: 97 active+clean; 70 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 550 KiB/s wr, 77 op/s 2023-07-21T18:45:51.270 INFO:tasks.workunit.client.0.smithi039.stdout: 1 188090 2.96 MB/sec execute 64 sec latency 1263.207 ms 2023-07-21T18:45:52.270 INFO:tasks.workunit.client.0.smithi039.stdout: 1 188665 2.94 MB/sec execute 65 sec latency 492.135 ms 2023-07-21T18:45:52.614 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:45:52 smithi193 ceph-mon[98204]: pgmap v127: 97 pgs: 97 active+clean; 74 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 1.3 MiB/s wr, 167 op/s 2023-07-21T18:45:52.633 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:45:52 smithi155 ceph-mon[97934]: pgmap v127: 97 pgs: 97 active+clean; 74 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 1.3 MiB/s wr, 167 op/s 2023-07-21T18:45:52.682 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:45:52 smithi039 ceph-mon[96489]: pgmap v127: 97 pgs: 97 active+clean; 74 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 1.3 MiB/s wr, 167 op/s 2023-07-21T18:45:53.270 INFO:tasks.workunit.client.0.smithi039.stdout: 1 190985 3.09 MB/sec execute 66 sec latency 15.084 ms 2023-07-21T18:45:53.614 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:45:53 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:45:53.633 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:45:53 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:45:53.682 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:45:53 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:45:54.270 INFO:tasks.workunit.client.0.smithi039.stdout: 1 192166 3.14 MB/sec execute 67 sec latency 23.810 ms 2023-07-21T18:45:54.614 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:45:54 smithi193 ceph-mon[98204]: pgmap v128: 97 pgs: 97 active+clean; 74 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 807 KiB/s wr, 112 op/s 2023-07-21T18:45:54.614 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:45:54 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:45:54.614 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:45:54 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:45:54.615 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:45:54 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:45:54.633 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:45:54 smithi155 ceph-mon[97934]: pgmap v128: 97 pgs: 97 active+clean; 74 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 807 KiB/s wr, 112 op/s 2023-07-21T18:45:54.633 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:45:54 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:45:54.633 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:45:54 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:45:54.633 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:45:54 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:45:54.682 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:45:54 smithi039 ceph-mon[96489]: pgmap v128: 97 pgs: 97 active+clean; 74 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 807 KiB/s wr, 112 op/s 2023-07-21T18:45:54.682 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:45:54 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:45:54.682 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:45:54 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:45:54.682 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:45:54 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:45:55.270 INFO:tasks.workunit.client.0.smithi039.stdout: 1 194476 3.28 MB/sec execute 68 sec latency 15.099 ms 2023-07-21T18:45:56.270 INFO:tasks.workunit.client.0.smithi039.stdout: 1 195888 3.33 MB/sec execute 69 sec latency 11.915 ms 2023-07-21T18:45:56.614 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:45:56 smithi193 ceph-mon[98204]: pgmap v129: 97 pgs: 97 active+clean; 74 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 807 KiB/s wr, 112 op/s 2023-07-21T18:45:56.633 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:45:56 smithi155 ceph-mon[97934]: pgmap v129: 97 pgs: 97 active+clean; 74 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 807 KiB/s wr, 112 op/s 2023-07-21T18:45:56.682 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:45:56 smithi039 ceph-mon[96489]: pgmap v129: 97 pgs: 97 active+clean; 74 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 807 KiB/s wr, 112 op/s 2023-07-21T18:45:57.270 INFO:tasks.workunit.client.0.smithi039.stdout: 1 198420 3.48 MB/sec execute 70 sec latency 24.067 ms 2023-07-21T18:45:57.409 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.393+0000 7fea391cd700 1 -- 172.21.15.39:0/2803305860 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea340fea30 msgr2=0x7fea340feea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:45:57.410 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.393+0000 7fea391cd700 1 --2- 172.21.15.39:0/2803305860 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea340fea30 0x7fea340feea0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fea28009990 tx=0x7fea28009ca0 comp rx=0 tx=0).stop 2023-07-21T18:45:57.410 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.393+0000 7fea391cd700 1 -- 172.21.15.39:0/2803305860 shutdown_connections 2023-07-21T18:45:57.410 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.393+0000 7fea391cd700 1 --2- 172.21.15.39:0/2803305860 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea340ff3e0 0x7fea3410b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:57.410 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.393+0000 7fea391cd700 1 --2- 172.21.15.39:0/2803305860 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea34104a30 0x7fea34104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:57.410 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.393+0000 7fea391cd700 1 --2- 172.21.15.39:0/2803305860 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea340fea30 0x7fea340feea0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:57.410 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.393+0000 7fea391cd700 1 -- 172.21.15.39:0/2803305860 >> 172.21.15.39:0/2803305860 conn(0x7fea34074760 msgr2=0x7fea34074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:45:57.411 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.393+0000 7fea391cd700 1 -- 172.21.15.39:0/2803305860 shutdown_connections 2023-07-21T18:45:57.411 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.393+0000 7fea391cd700 1 -- 172.21.15.39:0/2803305860 wait complete. 2023-07-21T18:45:57.412 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.394+0000 7fea391cd700 1 Processor -- start 2023-07-21T18:45:57.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.394+0000 7fea391cd700 1 -- start start 2023-07-21T18:45:57.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.394+0000 7fea391cd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea340fea30 0x7fea34196b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:57.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.394+0000 7fea391cd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea340ff3e0 0x7fea341970a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:57.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.394+0000 7fea391cd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea34104a30 0x7fea341a1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:57.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.394+0000 7fea391cd700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea341a2000 con 0x7fea340fea30 2023-07-21T18:45:57.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.394+0000 7fea391cd700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea341a2170 con 0x7fea340ff3e0 2023-07-21T18:45:57.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.394+0000 7fea391cd700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea341a2330 con 0x7fea34104a30 2023-07-21T18:45:57.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.395+0000 7fea32d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea340fea30 0x7fea34196b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:57.414 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.395+0000 7fea32d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea340fea30 0x7fea34196b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59572/0 (socket says 172.21.15.39:59572) 2023-07-21T18:45:57.414 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.395+0000 7fea3259c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea340ff3e0 0x7fea341970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:57.414 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.395+0000 7fea32d9d700 1 -- 172.21.15.39:0/2769628437 learned_addr learned my addr 172.21.15.39:0/2769628437 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:45:57.414 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.395+0000 7fea3359e700 1 --2- 172.21.15.39:0/2769628437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea34104a30 0x7fea341a1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:57.414 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.395+0000 7fea3259c700 1 -- 172.21.15.39:0/2769628437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea34104a30 msgr2=0x7fea341a1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:45:57.414 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.395+0000 7fea3259c700 1 --2- 172.21.15.39:0/2769628437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea34104a30 0x7fea341a1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:57.414 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.395+0000 7fea3259c700 1 -- 172.21.15.39:0/2769628437 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea340fea30 msgr2=0x7fea34196b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:45:57.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.395+0000 7fea3259c700 1 --2- 172.21.15.39:0/2769628437 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea340fea30 0x7fea34196b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:57.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.395+0000 7fea3259c700 1 -- 172.21.15.39:0/2769628437 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea280095f0 con 0x7fea340ff3e0 2023-07-21T18:45:57.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.395+0000 7fea32d9d700 1 --2- 172.21.15.39:0/2769628437 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea340fea30 0x7fea34196b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:45:57.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.395+0000 7fea3259c700 1 --2- 172.21.15.39:0/2769628437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea340ff3e0 0x7fea341970a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fea28000c00 tx=0x7fea28009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:45:57.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.395+0000 7fea1bfff700 1 -- 172.21.15.39:0/2769628437 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2801d070 con 0x7fea340ff3e0 2023-07-21T18:45:57.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.395+0000 7fea391cd700 1 -- 172.21.15.39:0/2769628437 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea341a25b0 con 0x7fea340ff3e0 2023-07-21T18:45:57.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.395+0000 7fea1bfff700 1 -- 172.21.15.39:0/2769628437 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea28022470 con 0x7fea340ff3e0 2023-07-21T18:45:57.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.395+0000 7fea1bfff700 1 -- 172.21.15.39:0/2769628437 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea28017650 con 0x7fea340ff3e0 2023-07-21T18:45:57.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.396+0000 7fea391cd700 1 -- 172.21.15.39:0/2769628437 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea341a29d0 con 0x7fea340ff3e0 2023-07-21T18:45:57.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.398+0000 7fea1bfff700 1 -- 172.21.15.39:0/2769628437 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fea28022870 con 0x7fea340ff3e0 2023-07-21T18:45:57.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.399+0000 7fea1bfff700 1 --2- 172.21.15.39:0/2769628437 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fea1c0a12c0 0x7fea1c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:57.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.399+0000 7fea32d9d700 1 --2- 172.21.15.39:0/2769628437 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fea1c0a12c0 0x7fea1c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:57.418 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.399+0000 7fea1bfff700 1 -- 172.21.15.39:0/2769628437 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fea280ce0d0 con 0x7fea340ff3e0 2023-07-21T18:45:57.418 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.400+0000 7fea45353b80 1 Processor -- start 2023-07-21T18:45:57.418 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.400+0000 7fea45353b80 1 -- start start 2023-07-21T18:45:57.418 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.400+0000 7fea32d9d700 1 --2- 172.21.15.39:0/2769628437 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fea1c0a12c0 0x7fea1c0a3770 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fea24005950 tx=0x7fea24009260 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:45:57.418 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.400+0000 7fea45353b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebe08566b0 0x55ebe09fdde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:57.418 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.400+0000 7fea45353b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ebe09fa040 0x55ebe09fa4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:57.418 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.400+0000 7fea45353b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ebe09fa9f0 0x55ebe0a0ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:57.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.400+0000 7fea45353b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebe08ffc50 con 0x55ebe09fa9f0 2023-07-21T18:45:57.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.400+0000 7fea45353b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebe09529e0 con 0x55ebe09fa040 2023-07-21T18:45:57.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.400+0000 7fea45353b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebe094be70 con 0x55ebe08566b0 2023-07-21T18:45:57.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.400+0000 7fea3359e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ebe09fa9f0 0x55ebe0a0ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:57.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.400+0000 7fea3259c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ebe09fa040 0x55ebe09fa4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:57.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.400+0000 7fea3359e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ebe09fa9f0 0x55ebe0a0ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59584/0 (socket says 172.21.15.39:59584) 2023-07-21T18:45:57.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.400+0000 7fea3359e700 1 -- 172.21.15.39:0/782729813 learned_addr learned my addr 172.21.15.39:0/782729813 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:45:57.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.400+0000 7fea32d9d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebe08566b0 0x55ebe09fdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:57.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.401+0000 7fea3259c700 1 -- 172.21.15.39:0/782729813 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebe08566b0 msgr2=0x55ebe09fdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:45:57.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.401+0000 7fea3259c700 1 --2- 172.21.15.39:0/782729813 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebe08566b0 0x55ebe09fdde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:57.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.401+0000 7fea3259c700 1 -- 172.21.15.39:0/782729813 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ebe09fa9f0 msgr2=0x55ebe0a0ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:45:57.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.401+0000 7fea3259c700 1 --2- 172.21.15.39:0/782729813 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ebe09fa9f0 0x55ebe0a0ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:57.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.401+0000 7fea3259c700 1 -- 172.21.15.39:0/782729813 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ebe0779d80 con 0x55ebe09fa040 2023-07-21T18:45:57.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.401+0000 7fea3259c700 1 --2- 172.21.15.39:0/782729813 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ebe09fa040 0x55ebe09fa4b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fea280cb840 tx=0x7fea28009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=637c7b64f522087d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:45:57.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.401+0000 7fea197fa700 1 -- 172.21.15.39:0/782729813 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2809b250 con 0x55ebe09fa040 2023-07-21T18:45:57.421 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.401+0000 7fea197fa700 1 -- 172.21.15.39:0/782729813 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea2809f280 con 0x55ebe09fa040 2023-07-21T18:45:57.421 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.401+0000 7fea197fa700 1 -- 172.21.15.39:0/782729813 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2800fd30 con 0x55ebe09fa040 2023-07-21T18:45:57.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.402+0000 7fea45353b80 1 -- 172.21.15.39:0/782729813 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ebe09fa040 msgr2=0x55ebe09fa4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:45:57.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.402+0000 7fea45353b80 1 --2- 172.21.15.39:0/782729813 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ebe09fa040 0x55ebe09fa4b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fea280cb840 tx=0x7fea28009ca0 comp rx=0 tx=0).stop 2023-07-21T18:45:57.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.402+0000 7fea45353b80 1 -- 172.21.15.39:0/782729813 shutdown_connections 2023-07-21T18:45:57.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.402+0000 7fea45353b80 1 --2- 172.21.15.39:0/782729813 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ebe09fa9f0 0x55ebe0a0ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:57.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.402+0000 7fea45353b80 1 --2- 172.21.15.39:0/782729813 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebe08566b0 0x55ebe09fdde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:57.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.402+0000 7fea45353b80 1 --2- 172.21.15.39:0/782729813 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ebe09fa040 0x55ebe09fa4b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:57.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.402+0000 7fea45353b80 1 -- 172.21.15.39:0/782729813 >> 172.21.15.39:0/782729813 conn(0x55ebe09efbe0 msgr2=0x55ebe09f4f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:45:57.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.403+0000 7fea45353b80 1 -- 172.21.15.39:0/782729813 shutdown_connections 2023-07-21T18:45:57.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.403+0000 7fea45353b80 1 -- 172.21.15.39:0/782729813 wait complete. 2023-07-21T18:45:57.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.403+0000 7fea45353b80 1 -- start start 2023-07-21T18:45:57.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.404+0000 7fea45353b80 1 Processor -- start 2023-07-21T18:45:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.404+0000 7fea45353b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebe08566b0 0x55ebe0a08380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.404+0000 7fea45353b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ebe09fa040 0x55ebe0a088c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.404+0000 7fea45353b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ebe09fa9f0 0x55ebe0a03670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.404+0000 7fea45353b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebe08eba60 con 0x55ebe09fa040 2023-07-21T18:45:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.404+0000 7fea45353b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebe08eb760 con 0x55ebe09fa9f0 2023-07-21T18:45:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.404+0000 7fea45353b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebe07cf8c0 con 0x55ebe08566b0 2023-07-21T18:45:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.404+0000 7fea45353b80 10 client.0 _ll_register_callbacks cb 0x55ebe08e9700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:45:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.404+0000 7fea32d9d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebe08566b0 0x55ebe0a08380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.404+0000 7fea32d9d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebe08566b0 0x55ebe0a08380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:49906/0 (socket says 172.21.15.39:49906) 2023-07-21T18:45:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.404+0000 7fea32d9d700 1 -- 172.21.15.39:0/2168724796 learned_addr learned my addr 172.21.15.39:0/2168724796 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:45:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.404+0000 7fea3259c700 1 --2- 172.21.15.39:0/2168724796 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ebe09fa040 0x55ebe0a088c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.404+0000 7fea3359e700 1 --2- 172.21.15.39:0/2168724796 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ebe09fa9f0 0x55ebe0a03670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.405+0000 7fea32d9d700 1 -- 172.21.15.39:0/2168724796 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ebe09fa9f0 msgr2=0x55ebe0a03670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:45:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.405+0000 7fea32d9d700 1 --2- 172.21.15.39:0/2168724796 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ebe09fa9f0 0x55ebe0a03670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.405+0000 7fea32d9d700 1 -- 172.21.15.39:0/2168724796 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ebe09fa040 msgr2=0x55ebe0a088c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:45:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.405+0000 7fea32d9d700 1 --2- 172.21.15.39:0/2168724796 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ebe09fa040 0x55ebe0a088c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.405+0000 7fea32d9d700 1 -- 172.21.15.39:0/2168724796 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea341a29d0 con 0x55ebe08566b0 2023-07-21T18:45:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.405+0000 7fea3359e700 1 --2- 172.21.15.39:0/2168724796 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ebe09fa9f0 0x55ebe0a03670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:45:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.405+0000 7fea32d9d700 1 --2- 172.21.15.39:0/2168724796 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebe08566b0 0x55ebe0a08380 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fea2400ee50 tx=0x7fea240047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b840089a06f342e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:45:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.405+0000 7fea16ff5700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2023-07-21T18:45:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.405+0000 7fea16ff5700 1 -- 172.21.15.39:0/2168724796 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea24022070 con 0x55ebe08566b0 2023-07-21T18:45:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.405+0000 7fea16ff5700 1 -- 172.21.15.39:0/2168724796 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fea2401ddc0 con 0x55ebe08566b0 2023-07-21T18:45:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.405+0000 7fea45353b80 1 -- 172.21.15.39:0/2168724796 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ebe08a5490 con 0x55ebe08566b0 2023-07-21T18:45:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.405+0000 7fea16ff5700 1 -- 172.21.15.39:0/2168724796 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea2400ee80 con 0x55ebe08566b0 2023-07-21T18:45:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.405+0000 7fea45353b80 10 client.25936 fetch_fsmap learned FSMap version 41 2023-07-21T18:45:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.405+0000 7fea45353b80 1 -- 172.21.15.39:0/2168724796 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55ebe0762c00 con 0x55ebe08566b0 2023-07-21T18:45:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.406+0000 7fea16ff5700 1 -- 172.21.15.39:0/2168724796 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ebe08a5490 con 0x55ebe08566b0 2023-07-21T18:45:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.406+0000 7fea16ff5700 1 -- 172.21.15.39:0/2168724796 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 41) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fea240055d0 con 0x55ebe08566b0 2023-07-21T18:45:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.406+0000 7fea45353b80 10 client.25936 fetch_fsmap finished waiting for FSMap version 41 2023-07-21T18:45:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.406+0000 7fea45353b80 10 client.25936 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:45:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.406+0000 7fea45353b80 20 client.25936 populate_metadata read hostname 'smithi039' 2023-07-21T18:45:57.429 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.406+0000 7fea45353b80 1 --2- 172.21.15.39:0/2168724796 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55ebe0a248a0 0x55ebe0a26cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:57.429 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.406+0000 7fea45353b80 4 client.25936 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:45:57.429 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.406+0000 7fea45353b80 1 -- 172.21.15.39:0/2168724796 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ebe0944e50 con 0x55ebe0a248a0 2023-07-21T18:45:57.429 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.406+0000 7fea3259c700 1 --2- 172.21.15.39:0/2168724796 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55ebe0a248a0 0x55ebe0a26cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:57.430 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.407+0000 7fea16ff5700 10 client.25936 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:45:57.430 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.407+0000 7fea3259c700 1 --2- 172.21.15.39:0/2168724796 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55ebe0a248a0 0x55ebe0a26cb0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=468c1ab9962df515 server_cookie=3acb70491944d15b in_seq=0 out_seq=0 2023-07-21T18:45:57.430 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.408+0000 7fea16ff5700 1 -- 172.21.15.39:0/2168724796 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ebe0944e50 con 0x55ebe0a248a0 2023-07-21T18:45:57.430 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.408+0000 7fea16ff5700 10 client.25936 handle_command_reply: tid=0 2023-07-21T18:45:57.431 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.408+0000 7fea45353b80 1 client.25936 shutdown 2023-07-21T18:45:57.431 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.408+0000 7fea45353b80 1 -- 172.21.15.39:0/2168724796 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebe08566b0 msgr2=0x55ebe0a08380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:45:57.431 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.408+0000 7fea45353b80 1 --2- 172.21.15.39:0/2168724796 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebe08566b0 0x55ebe0a08380 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fea2400ee50 tx=0x7fea240047c0 comp rx=0 tx=0).stop 2023-07-21T18:45:57.431 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.408+0000 7fea45353b80 1 -- 172.21.15.39:0/2168724796 shutdown_connections 2023-07-21T18:45:57.431 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.408+0000 7fea45353b80 1 --2- 172.21.15.39:0/2168724796 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55ebe0a248a0 0x55ebe0a26cb0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:57.431 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.408+0000 7fea45353b80 1 --2- 172.21.15.39:0/2168724796 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ebe09fa9f0 0x55ebe0a03670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:57.431 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.408+0000 7fea45353b80 1 --2- 172.21.15.39:0/2168724796 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebe08566b0 0x55ebe0a08380 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:57.431 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.408+0000 7fea16ff5700 0 client.25936 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:45:57.432 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.408+0000 7fea45353b80 1 --2- 172.21.15.39:0/2168724796 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ebe09fa040 0x55ebe0a088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:57.432 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.408+0000 7fea45353b80 1 -- 172.21.15.39:0/2168724796 >> 172.21.15.39:0/2168724796 conn(0x55ebe09efbe0 msgr2=0x55ebe09f44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:45:57.432 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.408+0000 7fea45353b80 1 -- 172.21.15.39:0/2168724796 shutdown_connections 2023-07-21T18:45:57.432 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.408+0000 7fea45353b80 1 -- 172.21.15.39:0/2168724796 wait complete. 2023-07-21T18:45:57.432 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.408+0000 7fea45353b80 20 client.25936 trim_cache size 0 max 16384 2023-07-21T18:45:57.440 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.424+0000 7fea45353b80 1 Processor -- start 2023-07-21T18:45:57.441 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.424+0000 7fea45353b80 1 -- start start 2023-07-21T18:45:57.442 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.424+0000 7fea45353b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ebe08566b0 0x55ebe0a02530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:57.442 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.424+0000 7fea45353b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebe09fa040 0x55ebe09f4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:57.442 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.424+0000 7fea45353b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ebe09fa9f0 0x55ebe0a008e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:57.442 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.424+0000 7fea45353b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebe09fe4b0 con 0x55ebe09fa9f0 2023-07-21T18:45:57.442 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.424+0000 7fea45353b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebe09f6880 con 0x55ebe08566b0 2023-07-21T18:45:57.442 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.424+0000 7fea45353b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebe0a04ce0 con 0x55ebe09fa040 2023-07-21T18:45:57.442 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.425+0000 7fea32d9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ebe08566b0 0x55ebe0a02530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:57.443 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.425+0000 7fea32d9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ebe08566b0 0x55ebe0a02530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:43958/0 (socket says 172.21.15.39:43958) 2023-07-21T18:45:57.443 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.425+0000 7fea32d9d700 1 -- 172.21.15.39:0/3555080151 learned_addr learned my addr 172.21.15.39:0/3555080151 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:45:57.443 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.425+0000 7fea3359e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ebe09fa9f0 0x55ebe0a008e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:57.443 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.425+0000 7fea3259c700 1 --2- 172.21.15.39:0/3555080151 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebe09fa040 0x55ebe09f4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:57.443 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.425+0000 7fea3359e700 1 -- 172.21.15.39:0/3555080151 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebe09fa040 msgr2=0x55ebe09f4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:45:57.443 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.425+0000 7fea3359e700 1 --2- 172.21.15.39:0/3555080151 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebe09fa040 0x55ebe09f4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:57.443 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.425+0000 7fea3359e700 1 -- 172.21.15.39:0/3555080151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ebe08566b0 msgr2=0x55ebe0a02530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:45:57.443 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.425+0000 7fea3359e700 1 --2- 172.21.15.39:0/3555080151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ebe08566b0 0x55ebe0a02530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:57.444 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.425+0000 7fea3359e700 1 -- 172.21.15.39:0/3555080151 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ebe07a01f0 con 0x55ebe09fa9f0 2023-07-21T18:45:57.444 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.425+0000 7fea32d9d700 1 --2- 172.21.15.39:0/3555080151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ebe08566b0 0x55ebe0a02530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:45:57.444 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.425+0000 7fea3359e700 1 --2- 172.21.15.39:0/3555080151 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ebe09fa9f0 0x55ebe0a008e0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fea2c007580 tx=0x7fea2c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca43cb8428c0c01e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:45:57.446 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.426+0000 7fea19ffb700 1 -- 172.21.15.39:0/3555080151 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2c00e070 con 0x55ebe09fa9f0 2023-07-21T18:45:57.446 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.426+0000 7fea19ffb700 1 -- 172.21.15.39:0/3555080151 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea2c00f040 con 0x55ebe09fa9f0 2023-07-21T18:45:57.446 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.426+0000 7fea19ffb700 1 -- 172.21.15.39:0/3555080151 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2c0043a0 con 0x55ebe09fa9f0 2023-07-21T18:45:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.427+0000 7fea45353b80 1 -- 172.21.15.39:0/3555080151 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ebe09fa9f0 msgr2=0x55ebe0a008e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:45:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.427+0000 7fea45353b80 1 --2- 172.21.15.39:0/3555080151 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ebe09fa9f0 0x55ebe0a008e0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fea2c007580 tx=0x7fea2c00b040 comp rx=0 tx=0).stop 2023-07-21T18:45:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.427+0000 7fea45353b80 1 -- 172.21.15.39:0/3555080151 shutdown_connections 2023-07-21T18:45:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.427+0000 7fea45353b80 1 --2- 172.21.15.39:0/3555080151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ebe08566b0 0x55ebe0a02530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.427+0000 7fea45353b80 1 --2- 172.21.15.39:0/3555080151 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ebe09fa9f0 0x55ebe0a008e0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.427+0000 7fea45353b80 1 --2- 172.21.15.39:0/3555080151 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebe09fa040 0x55ebe09f4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.427+0000 7fea45353b80 1 -- 172.21.15.39:0/3555080151 >> 172.21.15.39:0/3555080151 conn(0x55ebe09efbe0 msgr2=0x55ebe0a01e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:45:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.427+0000 7fea45353b80 1 -- 172.21.15.39:0/3555080151 shutdown_connections 2023-07-21T18:45:57.448 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.427+0000 7fea45353b80 1 -- 172.21.15.39:0/3555080151 wait complete. 2023-07-21T18:45:57.448 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.427+0000 7fea45353b80 1 -- start start 2023-07-21T18:45:57.448 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.427+0000 7fea45353b80 1 Processor -- start 2023-07-21T18:45:57.448 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.428+0000 7fea45353b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ebe08566b0 0x55ebe0a0df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:57.448 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.428+0000 7fea45353b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebe09fa040 0x55ebe0a0c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:57.448 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.428+0000 7fea45353b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ebe09fa9f0 0x55ebe0a0c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:57.448 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.428+0000 7fea45353b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebe0a0d000 con 0x55ebe08566b0 2023-07-21T18:45:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.428+0000 7fea45353b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebe0a78730 con 0x55ebe09fa9f0 2023-07-21T18:45:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.428+0000 7fea45353b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebe0a78870 con 0x55ebe09fa040 2023-07-21T18:45:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.428+0000 7fea45353b80 10 client.0 _ll_register_callbacks cb 0x7fea2400ea30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:45:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.428+0000 7fea3259c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebe09fa040 0x55ebe0a0c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.428+0000 7fea3259c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebe09fa040 0x55ebe0a0c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:49920/0 (socket says 172.21.15.39:49920) 2023-07-21T18:45:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.428+0000 7fea3259c700 1 -- 172.21.15.39:0/3513895547 learned_addr learned my addr 172.21.15.39:0/3513895547 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:45:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.428+0000 7fea32d9d700 1 --2- 172.21.15.39:0/3513895547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ebe08566b0 0x55ebe0a0df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.428+0000 7fea3359e700 1 --2- 172.21.15.39:0/3513895547 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ebe09fa9f0 0x55ebe0a0c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:57.450 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.428+0000 7fea3259c700 1 -- 172.21.15.39:0/3513895547 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ebe09fa9f0 msgr2=0x55ebe0a0c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:45:57.450 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.428+0000 7fea3259c700 1 --2- 172.21.15.39:0/3513895547 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ebe09fa9f0 0x55ebe0a0c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:57.450 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.428+0000 7fea3259c700 1 -- 172.21.15.39:0/3513895547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ebe08566b0 msgr2=0x55ebe0a0df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:45:57.450 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.428+0000 7fea3259c700 1 --2- 172.21.15.39:0/3513895547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ebe08566b0 0x55ebe0a0df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:57.450 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.428+0000 7fea3259c700 1 -- 172.21.15.39:0/3513895547 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ebe0762c00 con 0x55ebe09fa040 2023-07-21T18:45:57.450 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.428+0000 7fea32d9d700 1 --2- 172.21.15.39:0/3513895547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ebe08566b0 0x55ebe0a0df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:45:57.450 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.429+0000 7fea3259c700 1 --2- 172.21.15.39:0/3513895547 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebe09fa040 0x55ebe0a0c1f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fea280cbdc0 tx=0x7fea2801a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c7a158e7452e806 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:45:57.450 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.429+0000 7fea16ff5700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2023-07-21T18:45:57.451 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.429+0000 7fea16ff5700 1 -- 172.21.15.39:0/3513895547 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2809b250 con 0x55ebe09fa040 2023-07-21T18:45:57.451 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.429+0000 7fea16ff5700 1 -- 172.21.15.39:0/3513895547 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fea2800fb10 con 0x55ebe09fa040 2023-07-21T18:45:57.451 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.429+0000 7fea45353b80 1 -- 172.21.15.39:0/3513895547 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fea2809f280 con 0x55ebe09fa040 2023-07-21T18:45:57.451 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.429+0000 7fea16ff5700 1 -- 172.21.15.39:0/3513895547 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea2800fcd0 con 0x55ebe09fa040 2023-07-21T18:45:57.451 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.429+0000 7fea45353b80 10 client.25942 fetch_fsmap learned FSMap version 41 2023-07-21T18:45:57.451 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.429+0000 7fea45353b80 1 -- 172.21.15.39:0/3513895547 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7fea2c0092a0 con 0x55ebe09fa040 2023-07-21T18:45:57.451 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.429+0000 7fea16ff5700 1 -- 172.21.15.39:0/3513895547 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea2809f280 con 0x55ebe09fa040 2023-07-21T18:45:57.452 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.429+0000 7fea16ff5700 1 -- 172.21.15.39:0/3513895547 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 41) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fea2809a740 con 0x55ebe09fa040 2023-07-21T18:45:57.452 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.429+0000 7fea45353b80 10 client.25942 fetch_fsmap finished waiting for FSMap version 41 2023-07-21T18:45:57.452 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.429+0000 7fea45353b80 10 client.25942 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:45:57.452 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.429+0000 7fea45353b80 20 client.25942 populate_metadata read hostname 'smithi039' 2023-07-21T18:45:57.452 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.429+0000 7fea45353b80 1 --2- 172.21.15.39:0/3513895547 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55ebe0a799c0 0x55ebe0a7bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:45:57.452 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.429+0000 7fea45353b80 4 client.25942 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:45:57.452 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.429+0000 7fea45353b80 1 -- 172.21.15.39:0/3513895547 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ebe0906ff0 con 0x55ebe0a799c0 2023-07-21T18:45:57.452 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.430+0000 7fea32d9d700 1 --2- 172.21.15.39:0/3513895547 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55ebe0a799c0 0x55ebe0a7bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:45:57.453 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.431+0000 7fea32d9d700 1 --2- 172.21.15.39:0/3513895547 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55ebe0a799c0 0x55ebe0a7bda0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da5c91dc7d696fb6 server_cookie=7029b124c1b4defe in_seq=0 out_seq=0 2023-07-21T18:45:57.453 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.431+0000 7fea16ff5700 10 client.25942 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:45:57.488 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.472+0000 7fea16ff5700 1 -- 172.21.15.39:0/3513895547 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ebe0906ff0 con 0x55ebe0a799c0 2023-07-21T18:45:57.489 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.472+0000 7fea16ff5700 10 client.25942 handle_command_reply: tid=0 2023-07-21T18:45:57.489 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.472+0000 7fea45353b80 1 client.25942 shutdown 2023-07-21T18:45:57.489 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.472+0000 7fea45353b80 1 -- 172.21.15.39:0/3513895547 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebe09fa040 msgr2=0x55ebe0a0c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:45:57.489 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.472+0000 7fea45353b80 1 --2- 172.21.15.39:0/3513895547 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebe09fa040 0x55ebe0a0c1f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fea280cbdc0 tx=0x7fea2801a040 comp rx=0 tx=0).stop 2023-07-21T18:45:57.489 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.472+0000 7fea45353b80 1 -- 172.21.15.39:0/3513895547 shutdown_connections 2023-07-21T18:45:57.489 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.472+0000 7fea45353b80 1 --2- 172.21.15.39:0/3513895547 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55ebe0a799c0 0x55ebe0a7bda0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:57.490 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.472+0000 7fea45353b80 1 --2- 172.21.15.39:0/3513895547 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ebe09fa9f0 0x55ebe0a0c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:57.490 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.472+0000 7fea16ff5700 0 client.25942 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:45:57.490 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.472+0000 7fea45353b80 1 --2- 172.21.15.39:0/3513895547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ebe08566b0 0x55ebe0a0df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:57.490 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.472+0000 7fea45353b80 1 --2- 172.21.15.39:0/3513895547 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebe09fa040 0x55ebe0a0c1f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:57.490 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.472+0000 7fea45353b80 1 -- 172.21.15.39:0/3513895547 >> 172.21.15.39:0/3513895547 conn(0x55ebe09efbe0 msgr2=0x55ebe0a026b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:45:57.490 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.472+0000 7fea45353b80 1 -- 172.21.15.39:0/3513895547 shutdown_connections 2023-07-21T18:45:57.490 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.472+0000 7fea45353b80 1 -- 172.21.15.39:0/3513895547 wait complete. 2023-07-21T18:45:57.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.472+0000 7fea45353b80 20 client.25942 trim_cache size 0 max 16384 2023-07-21T18:45:57.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:45:57.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:45:57.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:45:57.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:45:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.473+0000 7fea391cd700 1 -- 172.21.15.39:0/2769628437 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fea1c0a12c0 msgr2=0x7fea1c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:45:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.473+0000 7fea391cd700 1 --2- 172.21.15.39:0/2769628437 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fea1c0a12c0 0x7fea1c0a3770 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fea24005950 tx=0x7fea24009260 comp rx=0 tx=0).stop 2023-07-21T18:45:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.473+0000 7fea391cd700 1 -- 172.21.15.39:0/2769628437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea340ff3e0 msgr2=0x7fea341970a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:45:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.473+0000 7fea391cd700 1 --2- 172.21.15.39:0/2769628437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea340ff3e0 0x7fea341970a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fea28000c00 tx=0x7fea28009580 comp rx=0 tx=0).stop 2023-07-21T18:45:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.473+0000 7fea391cd700 1 -- 172.21.15.39:0/2769628437 shutdown_connections 2023-07-21T18:45:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.473+0000 7fea391cd700 1 --2- 172.21.15.39:0/2769628437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea34104a30 0x7fea341a1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.473+0000 7fea391cd700 1 --2- 172.21.15.39:0/2769628437 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea340fea30 0x7fea34196b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.473+0000 7fea391cd700 1 --2- 172.21.15.39:0/2769628437 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fea1c0a12c0 0x7fea1c0a3770 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.473+0000 7fea391cd700 1 --2- 172.21.15.39:0/2769628437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea340ff3e0 0x7fea341970a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:45:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.473+0000 7fea391cd700 1 -- 172.21.15.39:0/2769628437 >> 172.21.15.39:0/2769628437 conn(0x7fea34074760 msgr2=0x7fea340fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:45:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.473+0000 7fea391cd700 1 -- 172.21.15.39:0/2769628437 shutdown_connections 2023-07-21T18:45:57.498 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:45:57.473+0000 7fea391cd700 1 -- 172.21.15.39:0/2769628437 wait complete. 2023-07-21T18:45:58.270 INFO:tasks.workunit.client.0.smithi039.stdout: 1 199541 3.51 MB/sec execute 71 sec latency 40.983 ms 2023-07-21T18:45:58.614 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:45:58 smithi193 ceph-mon[98204]: pgmap v130: 97 pgs: 97 active+clean; 78 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 1.5 MiB/s wr, 200 op/s 2023-07-21T18:45:58.633 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:45:58 smithi155 ceph-mon[97934]: pgmap v130: 97 pgs: 97 active+clean; 78 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 1.5 MiB/s wr, 200 op/s 2023-07-21T18:45:58.681 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:45:58 smithi039 ceph-mon[96489]: pgmap v130: 97 pgs: 97 active+clean; 78 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 1.5 MiB/s wr, 200 op/s 2023-07-21T18:45:59.270 INFO:tasks.workunit.client.0.smithi039.stdout: 1 201979 3.65 MB/sec execute 72 sec latency 38.006 ms 2023-07-21T18:46:00.270 INFO:tasks.workunit.client.0.smithi039.stdout: 1 203166 3.68 MB/sec execute 73 sec latency 19.162 ms 2023-07-21T18:46:00.614 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:46:00 smithi193 ceph-mon[98204]: pgmap v131: 97 pgs: 97 active+clean; 78 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 951 KiB/s rd, 1.5 MiB/s wr, 178 op/s 2023-07-21T18:46:00.633 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:46:00 smithi155 ceph-mon[97934]: pgmap v131: 97 pgs: 97 active+clean; 78 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 951 KiB/s rd, 1.5 MiB/s wr, 178 op/s 2023-07-21T18:46:00.681 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:46:00 smithi039 ceph-mon[96489]: pgmap v131: 97 pgs: 97 active+clean; 78 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 951 KiB/s rd, 1.5 MiB/s wr, 178 op/s 2023-07-21T18:46:01.270 INFO:tasks.workunit.client.0.smithi039.stdout: 1 205065 3.77 MB/sec execute 74 sec latency 24.048 ms 2023-07-21T18:46:02.270 INFO:tasks.workunit.client.0.smithi039.stdout: 1 206411 3.83 MB/sec execute 75 sec latency 11.759 ms 2023-07-21T18:46:02.614 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:46:02 smithi193 ceph-mon[98204]: pgmap v132: 97 pgs: 97 active+clean; 80 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 324 op/s 2023-07-21T18:46:02.633 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:46:02 smithi155 ceph-mon[97934]: pgmap v132: 97 pgs: 97 active+clean; 80 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 324 op/s 2023-07-21T18:46:02.682 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:46:02 smithi039 ceph-mon[96489]: pgmap v132: 97 pgs: 97 active+clean; 80 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 324 op/s 2023-07-21T18:46:03.270 INFO:tasks.workunit.client.0.smithi039.stdout: 1 208737 3.94 MB/sec execute 76 sec latency 11.959 ms 2023-07-21T18:46:04.270 INFO:tasks.workunit.client.0.smithi039.stdout: 1 210088 3.98 MB/sec execute 77 sec latency 29.263 ms 2023-07-21T18:46:04.614 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:46:04 smithi193 ceph-mon[98204]: pgmap v133: 97 pgs: 97 active+clean; 80 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 234 op/s 2023-07-21T18:46:04.632 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:46:04 smithi155 ceph-mon[97934]: pgmap v133: 97 pgs: 97 active+clean; 80 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 234 op/s 2023-07-21T18:46:04.681 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:46:04 smithi039 ceph-mon[96489]: pgmap v133: 97 pgs: 97 active+clean; 80 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 234 op/s 2023-07-21T18:46:05.270 INFO:tasks.workunit.client.0.smithi039.stdout: 1 212639 4.11 MB/sec execute 78 sec latency 12.416 ms 2023-07-21T18:46:06.270 INFO:tasks.workunit.client.0.smithi039.stdout: 1 213878 4.14 MB/sec execute 79 sec latency 13.052 ms 2023-07-21T18:46:06.614 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:46:06 smithi193 ceph-mon[98204]: pgmap v134: 97 pgs: 97 active+clean; 80 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 234 op/s 2023-07-21T18:46:06.632 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:46:06 smithi155 ceph-mon[97934]: pgmap v134: 97 pgs: 97 active+clean; 80 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 234 op/s 2023-07-21T18:46:06.681 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:46:06 smithi039 ceph-mon[96489]: pgmap v134: 97 pgs: 97 active+clean; 80 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 234 op/s 2023-07-21T18:46:07.270 INFO:tasks.workunit.client.0.smithi039.stdout: 1 216081 4.23 MB/sec execute 80 sec latency 15.423 ms 2023-07-21T18:46:08.270 INFO:tasks.workunit.client.0.smithi039.stdout: 1 217335 4.26 MB/sec execute 81 sec latency 13.908 ms 2023-07-21T18:46:08.614 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:46:08 smithi193 ceph-mon[98204]: pgmap v135: 97 pgs: 97 active+clean; 88 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 356 op/s 2023-07-21T18:46:08.633 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:46:08 smithi155 ceph-mon[97934]: pgmap v135: 97 pgs: 97 active+clean; 88 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 356 op/s 2023-07-21T18:46:08.681 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:46:08 smithi039 ceph-mon[96489]: pgmap v135: 97 pgs: 97 active+clean; 88 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 356 op/s 2023-07-21T18:46:09.270 INFO:tasks.workunit.client.0.smithi039.stdout: 1 219617 4.36 MB/sec execute 82 sec latency 13.926 ms 2023-07-21T18:46:10.270 INFO:tasks.workunit.client.0.smithi039.stdout: 1 220698 4.38 MB/sec execute 83 sec latency 47.605 ms 2023-07-21T18:46:10.615 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.596+0000 7efc1b278700 1 -- 172.21.15.39:0/773757697 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc14104a30 msgr2=0x7efc14104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:46:10.616 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.596+0000 7efc1b278700 1 --2- 172.21.15.39:0/773757697 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc14104a30 0x7efc14104e00 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7efc10009990 tx=0x7efc10009ca0 comp rx=0 tx=0).stop 2023-07-21T18:46:10.616 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.596+0000 7efc1b278700 1 -- 172.21.15.39:0/773757697 shutdown_connections 2023-07-21T18:46:10.616 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.596+0000 7efc1b278700 1 --2- 172.21.15.39:0/773757697 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc140ff3d0 0x7efc1410b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:10.616 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.596+0000 7efc1b278700 1 --2- 172.21.15.39:0/773757697 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc14104a30 0x7efc14104e00 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:10.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.596+0000 7efc1b278700 1 --2- 172.21.15.39:0/773757697 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc140fea20 0x7efc140fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:10.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.596+0000 7efc1b278700 1 -- 172.21.15.39:0/773757697 >> 172.21.15.39:0/773757697 conn(0x7efc140fa500 msgr2=0x7efc140fc910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:46:10.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.596+0000 7efc1b278700 1 -- 172.21.15.39:0/773757697 shutdown_connections 2023-07-21T18:46:10.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.596+0000 7efc1b278700 1 -- 172.21.15.39:0/773757697 wait complete. 2023-07-21T18:46:10.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.597+0000 7efc1b278700 1 Processor -- start 2023-07-21T18:46:10.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.597+0000 7efc1b278700 1 -- start start 2023-07-21T18:46:10.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.597+0000 7efc1b278700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc140fea20 0x7efc14196bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:10.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.597+0000 7efc1b278700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc140ff3d0 0x7efc14197100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:10.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.597+0000 7efc1b278700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc14104a30 0x7efc141a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:10.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.597+0000 7efc1b278700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc141a2030 con 0x7efc140ff3d0 2023-07-21T18:46:10.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.597+0000 7efc1b278700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc141a21a0 con 0x7efc14104a30 2023-07-21T18:46:10.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.597+0000 7efc1b278700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc141a2360 con 0x7efc140fea20 2023-07-21T18:46:10.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.597+0000 7efc18813700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc140ff3d0 0x7efc14197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:10.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.597+0000 7efc19014700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc140fea20 0x7efc14196bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:10.619 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.597+0000 7efc18813700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc140ff3d0 0x7efc14197100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59616/0 (socket says 172.21.15.39:59616) 2023-07-21T18:46:10.619 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.597+0000 7efc19014700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc140fea20 0x7efc14196bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:49934/0 (socket says 172.21.15.39:49934) 2023-07-21T18:46:10.619 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.597+0000 7efc18813700 1 -- 172.21.15.39:0/926341236 learned_addr learned my addr 172.21.15.39:0/926341236 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:46:10.619 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.597+0000 7efc19815700 1 --2- 172.21.15.39:0/926341236 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc14104a30 0x7efc141a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:10.619 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.597+0000 7efc18813700 1 -- 172.21.15.39:0/926341236 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc140fea20 msgr2=0x7efc14196bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:46:10.619 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.597+0000 7efc18813700 1 --2- 172.21.15.39:0/926341236 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc140fea20 0x7efc14196bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:10.619 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.597+0000 7efc18813700 1 -- 172.21.15.39:0/926341236 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc14104a30 msgr2=0x7efc141a1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:46:10.620 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.597+0000 7efc18813700 1 --2- 172.21.15.39:0/926341236 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc14104a30 0x7efc141a1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:10.620 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.597+0000 7efc18813700 1 -- 172.21.15.39:0/926341236 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc100095f0 con 0x7efc140ff3d0 2023-07-21T18:46:10.620 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.597+0000 7efc19815700 1 --2- 172.21.15.39:0/926341236 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc14104a30 0x7efc141a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:46:10.620 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.598+0000 7efc18813700 1 --2- 172.21.15.39:0/926341236 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc140ff3d0 0x7efc14197100 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7efc0400d950 tx=0x7efc0400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:46:10.620 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.598+0000 7efc0a7fc700 1 -- 172.21.15.39:0/926341236 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc0400f890 con 0x7efc140ff3d0 2023-07-21T18:46:10.620 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.598+0000 7efc0a7fc700 1 -- 172.21.15.39:0/926341236 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc04010460 con 0x7efc140ff3d0 2023-07-21T18:46:10.621 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.598+0000 7efc1b278700 1 -- 172.21.15.39:0/926341236 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc141a2640 con 0x7efc140ff3d0 2023-07-21T18:46:10.621 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.598+0000 7efc0a7fc700 1 -- 172.21.15.39:0/926341236 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc0400e610 con 0x7efc140ff3d0 2023-07-21T18:46:10.621 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.598+0000 7efc1b278700 1 -- 172.21.15.39:0/926341236 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc141a2ba0 con 0x7efc140ff3d0 2023-07-21T18:46:10.622 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:46:10 smithi193 ceph-mon[98204]: pgmap v136: 97 pgs: 97 active+clean; 88 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 267 op/s 2023-07-21T18:46:10.626 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.600+0000 7efc0a7fc700 1 -- 172.21.15.39:0/926341236 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7efc040108d0 con 0x7efc140ff3d0 2023-07-21T18:46:10.626 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.601+0000 7efc0a7fc700 1 --2- 172.21.15.39:0/926341236 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7efc000a1340 0x7efc000a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:10.626 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.601+0000 7efc0a7fc700 1 -- 172.21.15.39:0/926341236 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8483+0+0 (secure 0 0 0) 0x7efc040ccaa0 con 0x7efc140ff3d0 2023-07-21T18:46:10.627 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.601+0000 7efc19014700 1 --2- 172.21.15.39:0/926341236 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7efc000a1340 0x7efc000a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:10.627 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.602+0000 7efc19014700 1 --2- 172.21.15.39:0/926341236 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7efc000a1340 0x7efc000a37f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7efc1000b2e0 tx=0x7efc10005fd0 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:46:10.627 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.602+0000 7efc273feb80 1 Processor -- start 2023-07-21T18:46:10.627 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.602+0000 7efc273feb80 1 -- start start 2023-07-21T18:46:10.627 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.602+0000 7efc273feb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618ff9fc6b0 0x5618ffba5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:10.627 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.602+0000 7efc273feb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618ffba62d0 0x5618ffba0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:10.628 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.602+0000 7efc273feb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618ffba0580 0x5618ffba09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:10.628 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.602+0000 7efc273feb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618ffaa5c50 con 0x5618ffba62d0 2023-07-21T18:46:10.628 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.602+0000 7efc273feb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618ffaf89e0 con 0x5618ff9fc6b0 2023-07-21T18:46:10.628 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.602+0000 7efc273feb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618ffaf1e70 con 0x5618ffba0580 2023-07-21T18:46:10.628 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.602+0000 7efc18813700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618ffba62d0 0x5618ffba0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:10.628 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.602+0000 7efc18813700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618ffba62d0 0x5618ffba0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59624/0 (socket says 172.21.15.39:59624) 2023-07-21T18:46:10.628 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.602+0000 7efc18813700 1 -- 172.21.15.39:0/1536657536 learned_addr learned my addr 172.21.15.39:0/1536657536 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:46:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.602+0000 7efc19014700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618ff9fc6b0 0x5618ffba5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.602+0000 7efc19815700 1 --2- 172.21.15.39:0/1536657536 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618ffba0580 0x5618ffba09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.602+0000 7efc18813700 1 -- 172.21.15.39:0/1536657536 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618ffba0580 msgr2=0x5618ffba09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:46:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.602+0000 7efc18813700 1 --2- 172.21.15.39:0/1536657536 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618ffba0580 0x5618ffba09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.602+0000 7efc18813700 1 -- 172.21.15.39:0/1536657536 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618ff9fc6b0 msgr2=0x5618ffba5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:46:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.602+0000 7efc18813700 1 --2- 172.21.15.39:0/1536657536 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618ff9fc6b0 0x5618ffba5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:10.630 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.602+0000 7efc18813700 1 -- 172.21.15.39:0/1536657536 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5618ff91fd80 con 0x5618ffba62d0 2023-07-21T18:46:10.630 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.602+0000 7efc18813700 1 --2- 172.21.15.39:0/1536657536 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618ffba62d0 0x5618ffba0040 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7efc040a1a50 tx=0x7efc04099bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b1bd5a1945e044a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:46:10.630 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.603+0000 7efbff7fe700 1 -- 172.21.15.39:0/1536657536 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc040a1f80 con 0x5618ffba62d0 2023-07-21T18:46:10.630 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.603+0000 7efbff7fe700 1 -- 172.21.15.39:0/1536657536 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc04096240 con 0x5618ffba62d0 2023-07-21T18:46:10.630 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.603+0000 7efbff7fe700 1 -- 172.21.15.39:0/1536657536 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc040a1af0 con 0x5618ffba62d0 2023-07-21T18:46:10.630 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.603+0000 7efc273feb80 1 -- 172.21.15.39:0/1536657536 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618ffba62d0 msgr2=0x5618ffba0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:46:10.631 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.603+0000 7efc273feb80 1 --2- 172.21.15.39:0/1536657536 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618ffba62d0 0x5618ffba0040 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7efc040a1a50 tx=0x7efc04099bb0 comp rx=0 tx=0).stop 2023-07-21T18:46:10.631 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.604+0000 7efc273feb80 1 -- 172.21.15.39:0/1536657536 shutdown_connections 2023-07-21T18:46:10.631 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.604+0000 7efc273feb80 1 --2- 172.21.15.39:0/1536657536 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618ff9fc6b0 0x5618ffba5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:10.631 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.604+0000 7efc273feb80 1 --2- 172.21.15.39:0/1536657536 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618ffba0580 0x5618ffba09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:10.631 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.604+0000 7efc273feb80 1 --2- 172.21.15.39:0/1536657536 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618ffba62d0 0x5618ffba0040 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:10.631 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.604+0000 7efc273feb80 1 -- 172.21.15.39:0/1536657536 >> 172.21.15.39:0/1536657536 conn(0x5618ffb95be0 msgr2=0x5618ffb9af10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:46:10.631 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.604+0000 7efc273feb80 1 -- 172.21.15.39:0/1536657536 shutdown_connections 2023-07-21T18:46:10.632 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.604+0000 7efc273feb80 1 -- 172.21.15.39:0/1536657536 wait complete. 2023-07-21T18:46:10.632 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.604+0000 7efc273feb80 1 -- start start 2023-07-21T18:46:10.632 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.604+0000 7efc273feb80 1 Processor -- start 2023-07-21T18:46:10.632 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.604+0000 7efc273feb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618ff9fc6b0 0x5618ffbaf4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:10.632 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.604+0000 7efc273feb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618ffba0580 0x5618ffbaf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:10.632 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.604+0000 7efc273feb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618ffba62d0 0x5618ffba9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:10.632 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.604+0000 7efc273feb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618ffa91a60 con 0x5618ffba62d0 2023-07-21T18:46:10.633 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.604+0000 7efc273feb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618ffa91760 con 0x5618ff9fc6b0 2023-07-21T18:46:10.633 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.604+0000 7efc273feb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618ff9758c0 con 0x5618ffba0580 2023-07-21T18:46:10.633 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.605+0000 7efc273feb80 10 client.0 _ll_register_callbacks cb 0x5618ffa8f700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:46:10.633 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.605+0000 7efc19815700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618ffba62d0 0x5618ffba9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:10.633 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.605+0000 7efc19014700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618ff9fc6b0 0x5618ffbaf4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:10.634 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.605+0000 7efc18813700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618ffba0580 0x5618ffbaf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:10.634 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.605+0000 7efc19815700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618ffba62d0 0x5618ffba9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59632/0 (socket says 172.21.15.39:59632) 2023-07-21T18:46:10.634 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.605+0000 7efc19815700 1 -- 172.21.15.39:0/78710969 learned_addr learned my addr 172.21.15.39:0/78710969 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:46:10.634 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.605+0000 7efc19014700 1 -- 172.21.15.39:0/78710969 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618ffba0580 msgr2=0x5618ffbaf9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:46:10.634 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.605+0000 7efc19014700 1 --2- 172.21.15.39:0/78710969 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618ffba0580 0x5618ffbaf9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:10.634 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.605+0000 7efc19014700 1 -- 172.21.15.39:0/78710969 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618ffba62d0 msgr2=0x5618ffba9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:46:10.634 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.605+0000 7efc19014700 1 --2- 172.21.15.39:0/78710969 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618ffba62d0 0x5618ffba9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:10.635 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.605+0000 7efc19014700 1 -- 172.21.15.39:0/78710969 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc141a2ba0 con 0x5618ff9fc6b0 2023-07-21T18:46:10.635 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.605+0000 7efc19815700 1 --2- 172.21.15.39:0/78710969 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618ffba62d0 0x5618ffba9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:46:10.635 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.605+0000 7efc18813700 1 --2- 172.21.15.39:0/78710969 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618ffba0580 0x5618ffbaf9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:46:10.635 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.605+0000 7efc19014700 1 --2- 172.21.15.39:0/78710969 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618ff9fc6b0 0x5618ffbaf4a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efc10000c00 tx=0x7efc10020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8d7778d107976bb server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:46:10.635 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.605+0000 7efbfcff9700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-21T18:46:10.635 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.605+0000 7efbfcff9700 1 -- 172.21.15.39:0/78710969 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1002d040 con 0x5618ff9fc6b0 2023-07-21T18:46:10.635 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.605+0000 7efbfcff9700 1 -- 172.21.15.39:0/78710969 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8483+0+0 (secure 0 0 0) 0x7efc1000f460 con 0x5618ff9fc6b0 2023-07-21T18:46:10.636 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.605+0000 7efc273feb80 1 -- 172.21.15.39:0/78710969 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5618ffa4b490 con 0x5618ff9fc6b0 2023-07-21T18:46:10.636 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.606+0000 7efbfcff9700 1 -- 172.21.15.39:0/78710969 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc10005920 con 0x5618ff9fc6b0 2023-07-21T18:46:10.636 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.606+0000 7efc273feb80 10 client.25949 fetch_fsmap learned FSMap version 41 2023-07-21T18:46:10.636 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.606+0000 7efc273feb80 1 -- 172.21.15.39:0/78710969 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5618ff908c00 con 0x5618ff9fc6b0 2023-07-21T18:46:10.636 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.606+0000 7efbfcff9700 1 -- 172.21.15.39:0/78710969 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5618ffa4b490 con 0x5618ff9fc6b0 2023-07-21T18:46:10.636 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.606+0000 7efbfcff9700 1 -- 172.21.15.39:0/78710969 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 41) v1 ==== 7478+0+0 (secure 0 0 0) 0x7efc100327f0 con 0x5618ff9fc6b0 2023-07-21T18:46:10.637 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.606+0000 7efc273feb80 10 client.25949 fetch_fsmap finished waiting for FSMap version 41 2023-07-21T18:46:10.637 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.606+0000 7efc273feb80 10 client.25949 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:46:10.637 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.606+0000 7efc273feb80 20 client.25949 populate_metadata read hostname 'smithi039' 2023-07-21T18:46:10.637 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.606+0000 7efc273feb80 1 --2- 172.21.15.39:0/78710969 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5618ffbb99a0 0x5618ffbbbd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:10.637 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.606+0000 7efc273feb80 4 client.25949 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:46:10.637 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.606+0000 7efc273feb80 1 -- 172.21.15.39:0/78710969 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5618ffaeae50 con 0x5618ffbb99a0 2023-07-21T18:46:10.638 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.606+0000 7efc18813700 1 --2- 172.21.15.39:0/78710969 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5618ffbb99a0 0x5618ffbbbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:10.638 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.607+0000 7efbfcff9700 10 client.25949 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:46:10.638 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.607+0000 7efc18813700 1 --2- 172.21.15.39:0/78710969 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5618ffbb99a0 0x5618ffbbbd80 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f69d251db3775a3 server_cookie=de5a6c640b9e371f in_seq=0 out_seq=0 2023-07-21T18:46:10.638 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.608+0000 7efbfcff9700 1 -- 172.21.15.39:0/78710969 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5618ffaeae50 con 0x5618ffbb99a0 2023-07-21T18:46:10.638 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.608+0000 7efbfcff9700 10 client.25949 handle_command_reply: tid=0 2023-07-21T18:46:10.638 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.608+0000 7efc273feb80 1 client.25949 shutdown 2023-07-21T18:46:10.638 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.608+0000 7efc273feb80 1 -- 172.21.15.39:0/78710969 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618ff9fc6b0 msgr2=0x5618ffbaf4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:46:10.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.608+0000 7efc273feb80 1 --2- 172.21.15.39:0/78710969 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618ff9fc6b0 0x5618ffbaf4a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efc10000c00 tx=0x7efc10020000 comp rx=0 tx=0).stop 2023-07-21T18:46:10.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.608+0000 7efc273feb80 1 -- 172.21.15.39:0/78710969 shutdown_connections 2023-07-21T18:46:10.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.608+0000 7efc273feb80 1 --2- 172.21.15.39:0/78710969 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5618ffbb99a0 0x5618ffbbbd80 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:10.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.608+0000 7efc273feb80 1 --2- 172.21.15.39:0/78710969 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618ff9fc6b0 0x5618ffbaf4a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:10.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.608+0000 7efbfcff9700 0 client.25949 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:46:10.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.608+0000 7efc273feb80 1 --2- 172.21.15.39:0/78710969 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618ffba62d0 0x5618ffba9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:10.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.608+0000 7efc273feb80 1 --2- 172.21.15.39:0/78710969 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618ffba0580 0x5618ffbaf9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.608+0000 7efc273feb80 1 -- 172.21.15.39:0/78710969 >> 172.21.15.39:0/78710969 conn(0x5618ffb95be0 msgr2=0x5618ffb9a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:46:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.609+0000 7efc273feb80 1 -- 172.21.15.39:0/78710969 shutdown_connections 2023-07-21T18:46:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.609+0000 7efc273feb80 1 -- 172.21.15.39:0/78710969 wait complete. 2023-07-21T18:46:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.609+0000 7efc273feb80 20 client.25949 trim_cache size 0 max 16384 2023-07-21T18:46:10.641 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:46:10 smithi155 ceph-mon[97934]: pgmap v136: 97 pgs: 97 active+clean; 88 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 267 op/s 2023-07-21T18:46:10.642 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.624+0000 7efc273feb80 1 Processor -- start 2023-07-21T18:46:10.642 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.624+0000 7efc273feb80 1 -- start start 2023-07-21T18:46:10.643 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.624+0000 7efc273feb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618ff9fc6b0 0x5618ffba0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:10.643 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.624+0000 7efc273feb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618ffba0580 0x5618ffba9970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:10.643 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.624+0000 7efc273feb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618ffba62d0 0x5618ffba5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:10.643 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.624+0000 7efc273feb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618ffb9cb70 con 0x5618ff9fc6b0 2023-07-21T18:46:10.643 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.624+0000 7efc273feb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618ffb97d20 con 0x5618ffba62d0 2023-07-21T18:46:10.643 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.624+0000 7efc273feb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618ffba5a10 con 0x5618ffba0580 2023-07-21T18:46:10.643 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.624+0000 7efc19014700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618ff9fc6b0 0x5618ffba0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:10.644 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.624+0000 7efc19014700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618ff9fc6b0 0x5618ffba0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59636/0 (socket says 172.21.15.39:59636) 2023-07-21T18:46:10.644 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.624+0000 7efc19014700 1 -- 172.21.15.39:0/3345556919 learned_addr learned my addr 172.21.15.39:0/3345556919 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:46:10.644 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.624+0000 7efc18813700 1 --2- 172.21.15.39:0/3345556919 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618ffba0580 0x5618ffba9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:10.644 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.624+0000 7efc19815700 1 --2- 172.21.15.39:0/3345556919 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618ffba62d0 0x5618ffba5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:10.644 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.625+0000 7efc19014700 1 -- 172.21.15.39:0/3345556919 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618ffba0580 msgr2=0x5618ffba9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:46:10.644 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.625+0000 7efc19014700 1 --2- 172.21.15.39:0/3345556919 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618ffba0580 0x5618ffba9970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.625+0000 7efc19014700 1 -- 172.21.15.39:0/3345556919 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618ffba62d0 msgr2=0x5618ffba5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:46:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.625+0000 7efc19014700 1 --2- 172.21.15.39:0/3345556919 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618ffba62d0 0x5618ffba5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.625+0000 7efc19014700 1 -- 172.21.15.39:0/3345556919 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5618ffba0c20 con 0x5618ff9fc6b0 2023-07-21T18:46:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.625+0000 7efc19014700 1 --2- 172.21.15.39:0/3345556919 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618ff9fc6b0 0x5618ffba0040 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7efc100098e0 tx=0x7efc10005920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73a9ee1ce3f96a45 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:46:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.625+0000 7efbfffff700 1 -- 172.21.15.39:0/3345556919 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1002d040 con 0x5618ff9fc6b0 2023-07-21T18:46:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.625+0000 7efbfffff700 1 -- 172.21.15.39:0/3345556919 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc1001e040 con 0x5618ff9fc6b0 2023-07-21T18:46:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.625+0000 7efbfffff700 1 -- 172.21.15.39:0/3345556919 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1000fb20 con 0x5618ff9fc6b0 2023-07-21T18:46:10.648 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.626+0000 7efc273feb80 1 -- 172.21.15.39:0/3345556919 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618ff9fc6b0 msgr2=0x5618ffba0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:46:10.648 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.626+0000 7efc273feb80 1 --2- 172.21.15.39:0/3345556919 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618ff9fc6b0 0x5618ffba0040 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7efc100098e0 tx=0x7efc10005920 comp rx=0 tx=0).stop 2023-07-21T18:46:10.648 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.626+0000 7efc273feb80 1 -- 172.21.15.39:0/3345556919 shutdown_connections 2023-07-21T18:46:10.649 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.626+0000 7efc273feb80 1 --2- 172.21.15.39:0/3345556919 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618ffba62d0 0x5618ffba5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:10.649 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.626+0000 7efc273feb80 1 --2- 172.21.15.39:0/3345556919 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618ff9fc6b0 0x5618ffba0040 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:10.649 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.626+0000 7efc273feb80 1 --2- 172.21.15.39:0/3345556919 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618ffba0580 0x5618ffba9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:10.649 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.626+0000 7efc273feb80 1 -- 172.21.15.39:0/3345556919 >> 172.21.15.39:0/3345556919 conn(0x5618ffb95be0 msgr2=0x5618ffba4b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:46:10.649 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.626+0000 7efc273feb80 1 -- 172.21.15.39:0/3345556919 shutdown_connections 2023-07-21T18:46:10.649 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.626+0000 7efc273feb80 1 -- 172.21.15.39:0/3345556919 wait complete. 2023-07-21T18:46:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.626+0000 7efc273feb80 1 -- start start 2023-07-21T18:46:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.626+0000 7efc273feb80 1 Processor -- start 2023-07-21T18:46:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.627+0000 7efc273feb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618ff9fc6b0 0x5618ffc061b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.627+0000 7efc273feb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618ffba0580 0x5618ffc066f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.627+0000 7efc273feb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618ffba62d0 0x5618ffbaec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.627+0000 7efc273feb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618ffbaf230 con 0x5618ff9fc6b0 2023-07-21T18:46:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.627+0000 7efc273feb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618ffbaf3a0 con 0x5618ffba62d0 2023-07-21T18:46:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.627+0000 7efc273feb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618ffbaf560 con 0x5618ffba0580 2023-07-21T18:46:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.627+0000 7efc273feb80 10 client.0 _ll_register_callbacks cb 0x7efc10005350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:46:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.627+0000 7efc19014700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618ff9fc6b0 0x5618ffc061b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.627+0000 7efc19014700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618ff9fc6b0 0x5618ffc061b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59642/0 (socket says 172.21.15.39:59642) 2023-07-21T18:46:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.627+0000 7efc19815700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618ffba62d0 0x5618ffbaec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.627+0000 7efc19014700 1 -- 172.21.15.39:0/888157202 learned_addr learned my addr 172.21.15.39:0/888157202 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:46:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.627+0000 7efc18813700 1 --2- 172.21.15.39:0/888157202 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618ffba0580 0x5618ffc066f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:10.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.627+0000 7efc19014700 1 -- 172.21.15.39:0/888157202 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618ffba0580 msgr2=0x5618ffc066f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:46:10.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.627+0000 7efc19014700 1 --2- 172.21.15.39:0/888157202 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618ffba0580 0x5618ffc066f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:10.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.627+0000 7efc19014700 1 -- 172.21.15.39:0/888157202 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618ffba62d0 msgr2=0x5618ffbaec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:46:10.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.627+0000 7efc19014700 1 --2- 172.21.15.39:0/888157202 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618ffba62d0 0x5618ffbaec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:10.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.627+0000 7efc19014700 1 -- 172.21.15.39:0/888157202 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5618ff908c00 con 0x5618ff9fc6b0 2023-07-21T18:46:10.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.627+0000 7efc19014700 1 --2- 172.21.15.39:0/888157202 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618ff9fc6b0 0x5618ffc061b0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7efc10032ba0 tx=0x7efc10004f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21ac01c940a304f0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:46:10.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.627+0000 7efbfcff9700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:46:10.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.627+0000 7efbfcff9700 1 -- 172.21.15.39:0/888157202 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1002d040 con 0x5618ff9fc6b0 2023-07-21T18:46:10.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.627+0000 7efbfcff9700 1 -- 172.21.15.39:0/888157202 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8483+0+0 (secure 0 0 0) 0x7efc10032840 con 0x5618ff9fc6b0 2023-07-21T18:46:10.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.627+0000 7efc273feb80 1 -- 172.21.15.39:0/888157202 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efc04096240 con 0x5618ff9fc6b0 2023-07-21T18:46:10.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.628+0000 7efbfcff9700 1 -- 172.21.15.39:0/888157202 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc1000f7a0 con 0x5618ff9fc6b0 2023-07-21T18:46:10.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.628+0000 7efc273feb80 10 client.16467 fetch_fsmap learned FSMap version 41 2023-07-21T18:46:10.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.628+0000 7efc273feb80 1 -- 172.21.15.39:0/888157202 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5618ff9461f0 con 0x5618ff9fc6b0 2023-07-21T18:46:10.654 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.628+0000 7efbfcff9700 1 -- 172.21.15.39:0/888157202 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc04096240 con 0x5618ff9fc6b0 2023-07-21T18:46:10.654 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.628+0000 7efbfcff9700 1 -- 172.21.15.39:0/888157202 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 41) v1 ==== 7478+0+0 (secure 0 0 0) 0x7efc10041020 con 0x5618ff9fc6b0 2023-07-21T18:46:10.654 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.628+0000 7efc273feb80 10 client.16467 fetch_fsmap finished waiting for FSMap version 41 2023-07-21T18:46:10.654 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.628+0000 7efc273feb80 10 client.16467 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:46:10.654 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.628+0000 7efc273feb80 20 client.16467 populate_metadata read hostname 'smithi039' 2023-07-21T18:46:10.654 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.628+0000 7efc273feb80 1 --2- 172.21.15.39:0/888157202 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5618ffc0dae0 0x5618ffc0fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:10.654 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.628+0000 7efc273feb80 4 client.16467 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:46:10.655 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.628+0000 7efc273feb80 1 -- 172.21.15.39:0/888157202 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5618ffaacff0 con 0x5618ffc0dae0 2023-07-21T18:46:10.655 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.628+0000 7efc18813700 1 --2- 172.21.15.39:0/888157202 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5618ffc0dae0 0x5618ffc0fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:10.655 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.628+0000 7efbfcff9700 10 client.16467 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:46:10.655 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.628+0000 7efc18813700 1 --2- 172.21.15.39:0/888157202 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5618ffc0dae0 0x5618ffc0fec0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd4d76685bff89ab server_cookie=b2ea2a420b52c1ca in_seq=0 out_seq=0 2023-07-21T18:46:10.681 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:46:10 smithi039 ceph-mon[96489]: pgmap v136: 97 pgs: 97 active+clean; 88 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 267 op/s 2023-07-21T18:46:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.675+0000 7efbfcff9700 1 -- 172.21.15.39:0/888157202 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5618ffaacff0 con 0x5618ffc0dae0 2023-07-21T18:46:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.675+0000 7efbfcff9700 10 client.16467 handle_command_reply: tid=0 2023-07-21T18:46:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.675+0000 7efc273feb80 1 client.16467 shutdown 2023-07-21T18:46:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.675+0000 7efc273feb80 1 -- 172.21.15.39:0/888157202 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618ff9fc6b0 msgr2=0x5618ffc061b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:46:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.675+0000 7efc273feb80 1 --2- 172.21.15.39:0/888157202 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618ff9fc6b0 0x5618ffc061b0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7efc10032ba0 tx=0x7efc10004f10 comp rx=0 tx=0).stop 2023-07-21T18:46:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.675+0000 7efc273feb80 1 -- 172.21.15.39:0/888157202 shutdown_connections 2023-07-21T18:46:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.675+0000 7efc273feb80 1 --2- 172.21.15.39:0/888157202 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5618ffc0dae0 0x5618ffc0fec0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.675+0000 7efc273feb80 1 --2- 172.21.15.39:0/888157202 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618ffba62d0 0x5618ffbaec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.675+0000 7efc273feb80 1 --2- 172.21.15.39:0/888157202 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618ff9fc6b0 0x5618ffc061b0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.675+0000 7efbfcff9700 0 client.16467 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:46:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.675+0000 7efc273feb80 1 --2- 172.21.15.39:0/888157202 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618ffba0580 0x5618ffc066f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.675+0000 7efc273feb80 1 -- 172.21.15.39:0/888157202 >> 172.21.15.39:0/888157202 conn(0x5618ffb95be0 msgr2=0x5618ffba7ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:46:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.676+0000 7efc273feb80 1 -- 172.21.15.39:0/888157202 shutdown_connections 2023-07-21T18:46:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.676+0000 7efc273feb80 1 -- 172.21.15.39:0/888157202 wait complete. 2023-07-21T18:46:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.676+0000 7efc273feb80 20 client.16467 trim_cache size 0 max 16384 2023-07-21T18:46:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:46:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:46:10.696 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:46:10.696 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:46:10.696 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.676+0000 7efc1b278700 1 -- 172.21.15.39:0/926341236 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7efc000a1340 msgr2=0x7efc000a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:46:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.676+0000 7efc1b278700 1 --2- 172.21.15.39:0/926341236 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7efc000a1340 0x7efc000a37f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7efc1000b2e0 tx=0x7efc10005fd0 comp rx=0 tx=0).stop 2023-07-21T18:46:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.676+0000 7efc1b278700 1 -- 172.21.15.39:0/926341236 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc140ff3d0 msgr2=0x7efc14197100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:46:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.676+0000 7efc1b278700 1 --2- 172.21.15.39:0/926341236 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc140ff3d0 0x7efc14197100 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7efc0400d950 tx=0x7efc0400dc60 comp rx=0 tx=0).stop 2023-07-21T18:46:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.676+0000 7efc1b278700 1 -- 172.21.15.39:0/926341236 shutdown_connections 2023-07-21T18:46:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.676+0000 7efc1b278700 1 --2- 172.21.15.39:0/926341236 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7efc000a1340 0x7efc000a37f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.676+0000 7efc1b278700 1 --2- 172.21.15.39:0/926341236 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc14104a30 0x7efc141a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:10.698 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.676+0000 7efc1b278700 1 --2- 172.21.15.39:0/926341236 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc140fea20 0x7efc14196bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:10.698 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.676+0000 7efc1b278700 1 --2- 172.21.15.39:0/926341236 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc140ff3d0 0x7efc14197100 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:10.698 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.676+0000 7efc1b278700 1 -- 172.21.15.39:0/926341236 >> 172.21.15.39:0/926341236 conn(0x7efc140fa500 msgr2=0x7efc140fbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:46:10.698 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.676+0000 7efc1b278700 1 -- 172.21.15.39:0/926341236 shutdown_connections 2023-07-21T18:46:10.698 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:10.676+0000 7efc1b278700 1 -- 172.21.15.39:0/926341236 wait complete. 2023-07-21T18:46:11.270 INFO:tasks.workunit.client.0.smithi039.stdout: 1 222842 4.46 MB/sec execute 84 sec latency 19.334 ms 2023-07-21T18:46:12.270 INFO:tasks.workunit.client.0.smithi039.stdout: 1 224106 4.50 MB/sec execute 85 sec latency 52.630 ms 2023-07-21T18:46:12.614 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:46:12 smithi193 ceph-mon[98204]: pgmap v137: 97 pgs: 97 active+clean; 94 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 371 op/s 2023-07-21T18:46:12.632 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:46:12 smithi155 ceph-mon[97934]: pgmap v137: 97 pgs: 97 active+clean; 94 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 371 op/s 2023-07-21T18:46:12.681 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:46:12 smithi039 ceph-mon[96489]: pgmap v137: 97 pgs: 97 active+clean; 94 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 371 op/s 2023-07-21T18:46:13.270 INFO:tasks.workunit.client.0.smithi039.stdout: 1 226178 4.56 MB/sec execute 86 sec latency 13.409 ms 2023-07-21T18:46:14.270 INFO:tasks.workunit.client.0.smithi039.stdout: 1 227643 4.62 MB/sec execute 87 sec latency 13.786 ms 2023-07-21T18:46:14.614 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:46:14 smithi193 ceph-mon[98204]: pgmap v138: 97 pgs: 97 active+clean; 94 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 225 op/s 2023-07-21T18:46:14.632 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:46:14 smithi155 ceph-mon[97934]: pgmap v138: 97 pgs: 97 active+clean; 94 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 225 op/s 2023-07-21T18:46:14.681 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:46:14 smithi039 ceph-mon[96489]: pgmap v138: 97 pgs: 97 active+clean; 94 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 225 op/s 2023-07-21T18:46:15.270 INFO:tasks.workunit.client.0.smithi039.stdout: 1 229631 4.65 MB/sec execute 88 sec latency 13.815 ms 2023-07-21T18:46:15.614 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:46:15 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:46:15.614 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:46:15 smithi193 ceph-mon[98204]: pgmap v139: 97 pgs: 97 active+clean; 94 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 225 op/s 2023-07-21T18:46:15.637 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:46:15 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:46:15.637 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:46:15 smithi155 ceph-mon[97934]: pgmap v139: 97 pgs: 97 active+clean; 94 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 225 op/s 2023-07-21T18:46:15.681 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:46:15 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:46:15.682 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:46:15 smithi039 ceph-mon[96489]: pgmap v139: 97 pgs: 97 active+clean; 94 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 225 op/s 2023-07-21T18:46:16.270 INFO:tasks.workunit.client.0.smithi039.stdout: 1 231072 4.73 MB/sec execute 89 sec latency 32.929 ms 2023-07-21T18:46:17.270 INFO:tasks.workunit.client.0.smithi039.stdout: 1 233034 4.75 MB/sec execute 90 sec latency 16.748 ms 2023-07-21T18:46:18.270 INFO:tasks.workunit.client.0.smithi039.stdout: 1 234586 4.83 MB/sec execute 91 sec latency 14.966 ms 2023-07-21T18:46:18.363 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:46:18 smithi193 ceph-mon[98204]: pgmap v140: 97 pgs: 97 active+clean; 92 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 353 op/s 2023-07-21T18:46:18.382 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:46:18 smithi155 ceph-mon[97934]: pgmap v140: 97 pgs: 97 active+clean; 92 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 353 op/s 2023-07-21T18:46:18.431 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:46:18 smithi039 ceph-mon[96489]: pgmap v140: 97 pgs: 97 active+clean; 92 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 353 op/s 2023-07-21T18:46:19.270 INFO:tasks.workunit.client.0.smithi039.stdout: 1 236517 4.86 MB/sec execute 92 sec latency 17.022 ms 2023-07-21T18:46:20.270 INFO:tasks.workunit.client.0.smithi039.stdout: 1 238085 4.93 MB/sec execute 93 sec latency 14.115 ms 2023-07-21T18:46:20.363 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:46:20 smithi193 ceph-mon[98204]: pgmap v141: 97 pgs: 97 active+clean; 92 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 231 op/s 2023-07-21T18:46:20.364 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:46:20 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:46:20.382 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:46:20 smithi155 ceph-mon[97934]: pgmap v141: 97 pgs: 97 active+clean; 92 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 231 op/s 2023-07-21T18:46:20.382 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:46:20 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:46:20.431 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:46:20 smithi039 ceph-mon[96489]: pgmap v141: 97 pgs: 97 active+clean; 92 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 231 op/s 2023-07-21T18:46:20.431 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:46:20 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:46:21.270 INFO:tasks.workunit.client.0.smithi039.stdout: 1 239586 4.94 MB/sec execute 94 sec latency 24.593 ms 2023-07-21T18:46:22.270 INFO:tasks.workunit.client.0.smithi039.stdout: 1 241578 5.03 MB/sec execute 95 sec latency 12.883 ms 2023-07-21T18:46:22.363 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:46:22 smithi193 ceph-mon[98204]: pgmap v142: 97 pgs: 97 active+clean; 98 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 330 op/s 2023-07-21T18:46:22.382 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:46:22 smithi155 ceph-mon[97934]: pgmap v142: 97 pgs: 97 active+clean; 98 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 330 op/s 2023-07-21T18:46:22.431 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:46:22 smithi039 ceph-mon[96489]: pgmap v142: 97 pgs: 97 active+clean; 98 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 330 op/s 2023-07-21T18:46:23.270 INFO:tasks.workunit.client.0.smithi039.stdout: 1 243155 5.04 MB/sec execute 96 sec latency 18.664 ms 2023-07-21T18:46:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.792+0000 7f54a0e18700 1 -- 172.21.15.39:0/2477130173 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f549c0fea30 msgr2=0x7f549c0feea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:46:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.792+0000 7f54a0e18700 1 --2- 172.21.15.39:0/2477130173 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f549c0fea30 0x7f549c0feea0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f5490009990 tx=0x7f5490009ca0 comp rx=0 tx=0).stop 2023-07-21T18:46:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.792+0000 7f54a0e18700 1 -- 172.21.15.39:0/2477130173 shutdown_connections 2023-07-21T18:46:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.792+0000 7f54a0e18700 1 --2- 172.21.15.39:0/2477130173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f549c0ff3e0 0x7f549c10b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.792+0000 7f54a0e18700 1 --2- 172.21.15.39:0/2477130173 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f549c104a30 0x7f549c104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.792+0000 7f54a0e18700 1 --2- 172.21.15.39:0/2477130173 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f549c0fea30 0x7f549c0feea0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.792+0000 7f54a0e18700 1 -- 172.21.15.39:0/2477130173 >> 172.21.15.39:0/2477130173 conn(0x7f549c074760 msgr2=0x7f549c074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:46:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.792+0000 7f54a0e18700 1 -- 172.21.15.39:0/2477130173 shutdown_connections 2023-07-21T18:46:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.792+0000 7f54a0e18700 1 -- 172.21.15.39:0/2477130173 wait complete. 2023-07-21T18:46:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.792+0000 7f54a0e18700 1 Processor -- start 2023-07-21T18:46:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.792+0000 7f54a0e18700 1 -- start start 2023-07-21T18:46:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.793+0000 7f54a0e18700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f549c0fea30 0x7f549c19bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:23.812 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.793+0000 7f54a0e18700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f549c0ff3e0 0x7f549c19c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:23.812 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.793+0000 7f54a0e18700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f549c104a30 0x7f549c1959f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:23.812 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.793+0000 7f54a0e18700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549c196060 con 0x7f549c0fea30 2023-07-21T18:46:23.812 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.793+0000 7f54a0e18700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549c1961d0 con 0x7f549c0ff3e0 2023-07-21T18:46:23.812 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.793+0000 7f54a0e18700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549c196390 con 0x7f549c104a30 2023-07-21T18:46:23.812 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.793+0000 7f549a59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f549c0fea30 0x7f549c19bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:23.812 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.793+0000 7f549a59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f549c0fea30 0x7f549c19bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59656/0 (socket says 172.21.15.39:59656) 2023-07-21T18:46:23.813 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.793+0000 7f549a59c700 1 -- 172.21.15.39:0/428350284 learned_addr learned my addr 172.21.15.39:0/428350284 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:46:23.813 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.793+0000 7f5499d9b700 1 --2- 172.21.15.39:0/428350284 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f549c0ff3e0 0x7f549c19c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:23.813 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.793+0000 7f549ad9d700 1 --2- 172.21.15.39:0/428350284 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f549c104a30 0x7f549c1959f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:23.813 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.793+0000 7f549a59c700 1 -- 172.21.15.39:0/428350284 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f549c104a30 msgr2=0x7f549c1959f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:46:23.813 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.793+0000 7f549a59c700 1 --2- 172.21.15.39:0/428350284 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f549c104a30 0x7f549c1959f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:23.813 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.793+0000 7f549a59c700 1 -- 172.21.15.39:0/428350284 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f549c0ff3e0 msgr2=0x7f549c19c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:46:23.813 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.793+0000 7f549a59c700 1 --2- 172.21.15.39:0/428350284 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f549c0ff3e0 0x7f549c19c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:23.814 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.793+0000 7f549a59c700 1 -- 172.21.15.39:0/428350284 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54900095f0 con 0x7f549c0fea30 2023-07-21T18:46:23.814 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.793+0000 7f5499d9b700 1 --2- 172.21.15.39:0/428350284 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f549c0ff3e0 0x7f549c19c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:46:23.814 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.793+0000 7f549a59c700 1 --2- 172.21.15.39:0/428350284 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f549c0fea30 0x7f549c19bb90 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f548c00e970 tx=0x7f548c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:46:23.814 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.794+0000 7f548b7fe700 1 -- 172.21.15.39:0/428350284 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548c009790 con 0x7f549c0fea30 2023-07-21T18:46:23.814 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.794+0000 7f548b7fe700 1 -- 172.21.15.39:0/428350284 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f548c00c7e0 con 0x7f549c0fea30 2023-07-21T18:46:23.814 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.794+0000 7f54a0e18700 1 -- 172.21.15.39:0/428350284 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f549c196670 con 0x7f549c0fea30 2023-07-21T18:46:23.814 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.794+0000 7f548b7fe700 1 -- 172.21.15.39:0/428350284 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548c003c70 con 0x7f549c0fea30 2023-07-21T18:46:23.815 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.794+0000 7f54a0e18700 1 -- 172.21.15.39:0/428350284 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f549c1a2b80 con 0x7f549c0fea30 2023-07-21T18:46:23.819 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.796+0000 7f548b7fe700 1 -- 172.21.15.39:0/428350284 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f548c00cc50 con 0x7f549c0fea30 2023-07-21T18:46:23.820 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.797+0000 7f548b7fe700 1 --2- 172.21.15.39:0/428350284 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f54840a1340 0x7f54840a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:23.820 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.797+0000 7f5499d9b700 1 --2- 172.21.15.39:0/428350284 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f54840a1340 0x7f54840a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:23.820 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.797+0000 7f548b7fe700 1 -- 172.21.15.39:0/428350284 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f548c014070 con 0x7f549c0fea30 2023-07-21T18:46:23.820 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.797+0000 7f5499d9b700 1 --2- 172.21.15.39:0/428350284 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f54840a1340 0x7f54840a37f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f549c0fc9b0 tx=0x7f5490005fb0 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:46:23.820 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.797+0000 7f54acf9eb80 1 Processor -- start 2023-07-21T18:46:23.820 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.798+0000 7f54acf9eb80 1 -- start start 2023-07-21T18:46:23.821 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.798+0000 7f54acf9eb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f94feb06b0 0x55f950059d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:23.821 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.798+0000 7f54acf9eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f95005a2d0 0x55f950054040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:23.821 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.798+0000 7f54acf9eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f950054580 0x55f9500549d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:23.821 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.798+0000 7f54acf9eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f94ff59c50 con 0x55f95005a2d0 2023-07-21T18:46:23.821 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.798+0000 7f54acf9eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f94ffac9e0 con 0x55f950054580 2023-07-21T18:46:23.821 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.798+0000 7f54acf9eb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f94ffa5e70 con 0x55f94feb06b0 2023-07-21T18:46:23.821 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.798+0000 7f5499d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f95005a2d0 0x55f950054040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.798+0000 7f5499d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f95005a2d0 0x55f950054040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59666/0 (socket says 172.21.15.39:59666) 2023-07-21T18:46:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.798+0000 7f5499d9b700 1 -- 172.21.15.39:0/770812575 learned_addr learned my addr 172.21.15.39:0/770812575 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:46:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.798+0000 7f549a59c700 1 --2- 172.21.15.39:0/770812575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f94feb06b0 0x55f950059d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.798+0000 7f549ad9d700 1 --2- 172.21.15.39:0/770812575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f950054580 0x55f9500549d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.798+0000 7f549a59c700 1 -- 172.21.15.39:0/770812575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f950054580 msgr2=0x55f9500549d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:46:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.798+0000 7f549a59c700 1 --2- 172.21.15.39:0/770812575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f950054580 0x55f9500549d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.798+0000 7f549a59c700 1 -- 172.21.15.39:0/770812575 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f95005a2d0 msgr2=0x55f950054040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:46:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.798+0000 7f549a59c700 1 --2- 172.21.15.39:0/770812575 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f95005a2d0 0x55f950054040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.798+0000 7f549a59c700 1 -- 172.21.15.39:0/770812575 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f94fdd3d80 con 0x55f94feb06b0 2023-07-21T18:46:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.798+0000 7f549ad9d700 1 --2- 172.21.15.39:0/770812575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f950054580 0x55f9500549d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:46:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.798+0000 7f5499d9b700 1 --2- 172.21.15.39:0/770812575 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f95005a2d0 0x55f950054040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:46:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.799+0000 7f549a59c700 1 --2- 172.21.15.39:0/770812575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f94feb06b0 0x55f950059d90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f548c0ca5b0 tx=0x7f548c099890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32676ffb8034c833 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:46:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.799+0000 7f5488ff9700 1 -- 172.21.15.39:0/770812575 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548c09df60 con 0x55f94feb06b0 2023-07-21T18:46:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.799+0000 7f5488ff9700 1 -- 172.21.15.39:0/770812575 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f548c0a2380 con 0x55f94feb06b0 2023-07-21T18:46:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.799+0000 7f5488ff9700 1 -- 172.21.15.39:0/770812575 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548c099420 con 0x55f94feb06b0 2023-07-21T18:46:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.799+0000 7f54acf9eb80 1 -- 172.21.15.39:0/770812575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f94feb06b0 msgr2=0x55f950059d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:46:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.799+0000 7f54acf9eb80 1 --2- 172.21.15.39:0/770812575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f94feb06b0 0x55f950059d90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f548c0ca5b0 tx=0x7f548c099890 comp rx=0 tx=0).stop 2023-07-21T18:46:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.800+0000 7f54acf9eb80 1 -- 172.21.15.39:0/770812575 shutdown_connections 2023-07-21T18:46:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.800+0000 7f54acf9eb80 1 --2- 172.21.15.39:0/770812575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f950054580 0x55f9500549d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.800+0000 7f54acf9eb80 1 --2- 172.21.15.39:0/770812575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f94feb06b0 0x55f950059d90 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.800+0000 7f54acf9eb80 1 --2- 172.21.15.39:0/770812575 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f95005a2d0 0x55f950054040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.800+0000 7f54acf9eb80 1 -- 172.21.15.39:0/770812575 >> 172.21.15.39:0/770812575 conn(0x55f950049be0 msgr2=0x55f95004ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:46:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.800+0000 7f54acf9eb80 1 -- 172.21.15.39:0/770812575 shutdown_connections 2023-07-21T18:46:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.800+0000 7f54acf9eb80 1 -- 172.21.15.39:0/770812575 wait complete. 2023-07-21T18:46:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.800+0000 7f54acf9eb80 1 -- start start 2023-07-21T18:46:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.800+0000 7f54acf9eb80 1 Processor -- start 2023-07-21T18:46:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.800+0000 7f54acf9eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f94feb06b0 0x55f95005d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.800+0000 7f54acf9eb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f950054580 0x55f95005ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:23.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.800+0000 7f54acf9eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f95005a2d0 0x55f95005e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:23.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.800+0000 7f54acf9eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f94ff45a60 con 0x55f94feb06b0 2023-07-21T18:46:23.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.800+0000 7f54acf9eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f94ff45760 con 0x55f95005a2d0 2023-07-21T18:46:23.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.800+0000 7f54acf9eb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f94fe298c0 con 0x55f950054580 2023-07-21T18:46:23.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.800+0000 7f54acf9eb80 10 client.0 _ll_register_callbacks cb 0x55f94ff43700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:46:23.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.800+0000 7f549a59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f94feb06b0 0x55f95005d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:23.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.801+0000 7f5499d9b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f950054580 0x55f95005ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:23.827 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.801+0000 7f549ad9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f95005a2d0 0x55f95005e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:23.827 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.801+0000 7f549a59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f94feb06b0 0x55f95005d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59670/0 (socket says 172.21.15.39:59670) 2023-07-21T18:46:23.827 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.801+0000 7f549a59c700 1 -- 172.21.15.39:0/1612284303 learned_addr learned my addr 172.21.15.39:0/1612284303 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:46:23.827 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.801+0000 7f5499d9b700 1 --2- 172.21.15.39:0/1612284303 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f950054580 0x55f95005ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:49992/0 (socket says 172.21.15.39:49992) 2023-07-21T18:46:23.827 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.801+0000 7f5499d9b700 1 -- 172.21.15.39:0/1612284303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f95005a2d0 msgr2=0x55f95005e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:46:23.827 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.801+0000 7f5499d9b700 1 --2- 172.21.15.39:0/1612284303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f95005a2d0 0x55f95005e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:23.827 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.801+0000 7f5499d9b700 1 -- 172.21.15.39:0/1612284303 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f94feb06b0 msgr2=0x55f95005d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:46:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.801+0000 7f5499d9b700 1 --2- 172.21.15.39:0/1612284303 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f94feb06b0 0x55f95005d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.801+0000 7f5499d9b700 1 -- 172.21.15.39:0/1612284303 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5494011230 con 0x55f950054580 2023-07-21T18:46:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.801+0000 7f549a59c700 1 --2- 172.21.15.39:0/1612284303 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f94feb06b0 0x55f95005d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:46:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.801+0000 7f5499d9b700 1 --2- 172.21.15.39:0/1612284303 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f950054580 0x55f95005ddd0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5490000c00 tx=0x7f549001a050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86eda014999d195f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:46:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.801+0000 7f5481ffb700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2023-07-21T18:46:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.801+0000 7f5481ffb700 1 -- 172.21.15.39:0/1612284303 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549002b050 con 0x55f950054580 2023-07-21T18:46:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.801+0000 7f5481ffb700 1 -- 172.21.15.39:0/1612284303 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f5490026a10 con 0x55f950054580 2023-07-21T18:46:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.801+0000 7f54acf9eb80 1 -- 172.21.15.39:0/1612284303 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f94feff490 con 0x55f950054580 2023-07-21T18:46:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.801+0000 7f5481ffb700 1 -- 172.21.15.39:0/1612284303 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f549000fe70 con 0x55f950054580 2023-07-21T18:46:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.801+0000 7f54acf9eb80 10 client.25972 fetch_fsmap learned FSMap version 43 2023-07-21T18:46:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.801+0000 7f54acf9eb80 1 -- 172.21.15.39:0/1612284303 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x7f549c10c160 con 0x55f950054580 2023-07-21T18:46:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.802+0000 7f5481ffb700 1 -- 172.21.15.39:0/1612284303 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f94feff490 con 0x55f950054580 2023-07-21T18:46:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.802+0000 7f5481ffb700 1 -- 172.21.15.39:0/1612284303 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 43) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f5490024b70 con 0x55f950054580 2023-07-21T18:46:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.802+0000 7f54acf9eb80 10 client.25972 fetch_fsmap finished waiting for FSMap version 43 2023-07-21T18:46:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.802+0000 7f54acf9eb80 10 client.25972 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:46:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.802+0000 7f54acf9eb80 20 client.25972 populate_metadata read hostname 'smithi039' 2023-07-21T18:46:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.802+0000 7f54acf9eb80 1 --2- 172.21.15.39:0/1612284303 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55f95006dc50 0x55f950070060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.802+0000 7f54acf9eb80 4 client.25972 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:46:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.802+0000 7f54acf9eb80 1 -- 172.21.15.39:0/1612284303 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f94ff9ee50 con 0x55f95006dc50 2023-07-21T18:46:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.802+0000 7f549a59c700 1 --2- 172.21.15.39:0/1612284303 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55f95006dc50 0x55f950070060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.802+0000 7f5481ffb700 10 client.25972 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:46:23.831 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.802+0000 7f549a59c700 1 --2- 172.21.15.39:0/1612284303 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55f95006dc50 0x55f950070060 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf20d5bd8ce0d42a server_cookie=3c1f396c5b69dfd5 in_seq=0 out_seq=0 2023-07-21T18:46:23.831 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.803+0000 7f5481ffb700 1 -- 172.21.15.39:0/1612284303 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f94ff9ee50 con 0x55f95006dc50 2023-07-21T18:46:23.831 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.803+0000 7f5481ffb700 10 client.25972 handle_command_reply: tid=0 2023-07-21T18:46:23.831 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.803+0000 7f54acf9eb80 1 client.25972 shutdown 2023-07-21T18:46:23.831 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.803+0000 7f54acf9eb80 1 -- 172.21.15.39:0/1612284303 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f950054580 msgr2=0x55f95005ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:46:23.831 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.803+0000 7f54acf9eb80 1 --2- 172.21.15.39:0/1612284303 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f950054580 0x55f95005ddd0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5490000c00 tx=0x7f549001a050 comp rx=0 tx=0).stop 2023-07-21T18:46:23.832 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.803+0000 7f54acf9eb80 1 -- 172.21.15.39:0/1612284303 shutdown_connections 2023-07-21T18:46:23.832 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.803+0000 7f54acf9eb80 1 --2- 172.21.15.39:0/1612284303 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55f95006dc50 0x55f950070060 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:23.832 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.803+0000 7f54acf9eb80 1 --2- 172.21.15.39:0/1612284303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f95005a2d0 0x55f95005e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:23.832 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.803+0000 7f54acf9eb80 1 --2- 172.21.15.39:0/1612284303 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f94feb06b0 0x55f95005d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:23.832 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.803+0000 7f54acf9eb80 1 --2- 172.21.15.39:0/1612284303 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f950054580 0x55f95005ddd0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:23.832 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.803+0000 7f5481ffb700 0 client.25972 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:46:23.832 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.803+0000 7f54acf9eb80 1 -- 172.21.15.39:0/1612284303 >> 172.21.15.39:0/1612284303 conn(0x55f950049be0 msgr2=0x55f95004e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:46:23.833 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.803+0000 7f54acf9eb80 1 -- 172.21.15.39:0/1612284303 shutdown_connections 2023-07-21T18:46:23.833 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.803+0000 7f54acf9eb80 1 -- 172.21.15.39:0/1612284303 wait complete. 2023-07-21T18:46:23.833 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.803+0000 7f54acf9eb80 20 client.25972 trim_cache size 0 max 16384 2023-07-21T18:46:23.835 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.818+0000 7f54acf9eb80 1 Processor -- start 2023-07-21T18:46:23.835 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.818+0000 7f54acf9eb80 1 -- start start 2023-07-21T18:46:23.835 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.819+0000 7f54acf9eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f94feb06b0 0x55f950054040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:23.835 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.819+0000 7f54acf9eb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f950054580 0x55f950059320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:23.835 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.819+0000 7f54acf9eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f95005a2d0 0x55f95005e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:23.835 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.819+0000 7f54acf9eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f95005a6a0 con 0x55f95005a2d0 2023-07-21T18:46:23.836 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.819+0000 7f54acf9eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f95006b260 con 0x55f94feb06b0 2023-07-21T18:46:23.836 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.819+0000 7f54acf9eb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f95004fa00 con 0x55f950054580 2023-07-21T18:46:23.836 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.819+0000 7f549ad9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f95005a2d0 0x55f95005e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:23.836 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.819+0000 7f549a59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f94feb06b0 0x55f950054040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:23.836 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.819+0000 7f5499d9b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f950054580 0x55f950059320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:23.836 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.819+0000 7f549ad9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f95005a2d0 0x55f95005e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59682/0 (socket says 172.21.15.39:59682) 2023-07-21T18:46:23.837 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.819+0000 7f549a59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f94feb06b0 0x55f950054040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:44042/0 (socket says 172.21.15.39:44042) 2023-07-21T18:46:23.837 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.819+0000 7f549ad9d700 1 -- 172.21.15.39:0/4185600948 learned_addr learned my addr 172.21.15.39:0/4185600948 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:46:23.840 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.819+0000 7f549ad9d700 1 -- 172.21.15.39:0/4185600948 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f950054580 msgr2=0x55f950059320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:46:23.840 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.819+0000 7f549ad9d700 1 --2- 172.21.15.39:0/4185600948 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f950054580 0x55f950059320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:23.840 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.819+0000 7f549ad9d700 1 -- 172.21.15.39:0/4185600948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f94feb06b0 msgr2=0x55f950054040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:46:23.840 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.819+0000 7f549ad9d700 1 --2- 172.21.15.39:0/4185600948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f94feb06b0 0x55f950054040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:23.840 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.819+0000 7f549ad9d700 1 -- 172.21.15.39:0/4185600948 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f549c1a2b80 con 0x55f95005a2d0 2023-07-21T18:46:23.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.819+0000 7f549a59c700 1 --2- 172.21.15.39:0/4185600948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f94feb06b0 0x55f950054040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:46:23.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.819+0000 7f549ad9d700 1 --2- 172.21.15.39:0/4185600948 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f95005a2d0 0x55f95005e140 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f5494004040 tx=0x7f5494009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec682b3b243f74af server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:46:23.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.820+0000 7f54897fa700 1 -- 172.21.15.39:0/4185600948 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549400e040 con 0x55f95005a2d0 2023-07-21T18:46:23.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.820+0000 7f54897fa700 1 -- 172.21.15.39:0/4185600948 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f549400f040 con 0x55f95005a2d0 2023-07-21T18:46:23.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.820+0000 7f54897fa700 1 -- 172.21.15.39:0/4185600948 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5494016a70 con 0x55f95005a2d0 2023-07-21T18:46:23.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.820+0000 7f54acf9eb80 1 -- 172.21.15.39:0/4185600948 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f95005a2d0 msgr2=0x55f95005e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:46:23.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.820+0000 7f54acf9eb80 1 --2- 172.21.15.39:0/4185600948 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f95005a2d0 0x55f95005e140 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f5494004040 tx=0x7f5494009920 comp rx=0 tx=0).stop 2023-07-21T18:46:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.820+0000 7f54acf9eb80 1 -- 172.21.15.39:0/4185600948 shutdown_connections 2023-07-21T18:46:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.820+0000 7f54acf9eb80 1 --2- 172.21.15.39:0/4185600948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f94feb06b0 0x55f950054040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.820+0000 7f54acf9eb80 1 --2- 172.21.15.39:0/4185600948 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f95005a2d0 0x55f95005e140 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.820+0000 7f54acf9eb80 1 --2- 172.21.15.39:0/4185600948 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f950054580 0x55f950059320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.820+0000 7f54acf9eb80 1 -- 172.21.15.39:0/4185600948 >> 172.21.15.39:0/4185600948 conn(0x55f950049be0 msgr2=0x55f95005a900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:46:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.821+0000 7f54acf9eb80 1 -- 172.21.15.39:0/4185600948 shutdown_connections 2023-07-21T18:46:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.821+0000 7f54acf9eb80 1 -- 172.21.15.39:0/4185600948 wait complete. 2023-07-21T18:46:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.821+0000 7f54acf9eb80 1 -- start start 2023-07-21T18:46:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.821+0000 7f54acf9eb80 1 Processor -- start 2023-07-21T18:46:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.821+0000 7f54acf9eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f94feb06b0 0x55f9500baf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.821+0000 7f54acf9eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f950054580 0x55f9500bb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.821+0000 7f54acf9eb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f95005a2d0 0x55f9500b9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.821+0000 7f54acf9eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9500bba60 con 0x55f94feb06b0 2023-07-21T18:46:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.821+0000 7f54acf9eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9500bbba0 con 0x55f950054580 2023-07-21T18:46:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.821+0000 7f54acf9eb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9500b9bd0 con 0x55f95005a2d0 2023-07-21T18:46:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.821+0000 7f54acf9eb80 10 client.0 _ll_register_callbacks cb 0x7f549000f770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:46:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.821+0000 7f549a59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f94feb06b0 0x55f9500baf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.821+0000 7f5499d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f950054580 0x55f9500bb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.821+0000 7f549a59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f94feb06b0 0x55f9500baf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59684/0 (socket says 172.21.15.39:59684) 2023-07-21T18:46:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.821+0000 7f549a59c700 1 -- 172.21.15.39:0/1249229835 learned_addr learned my addr 172.21.15.39:0/1249229835 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:46:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.822+0000 7f549ad9d700 1 --2- 172.21.15.39:0/1249229835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f95005a2d0 0x55f9500b9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.822+0000 7f5499d9b700 1 -- 172.21.15.39:0/1249229835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f95005a2d0 msgr2=0x55f9500b9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:46:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.822+0000 7f5499d9b700 1 --2- 172.21.15.39:0/1249229835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f95005a2d0 0x55f9500b9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.822+0000 7f5499d9b700 1 -- 172.21.15.39:0/1249229835 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f94feb06b0 msgr2=0x55f9500baf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:46:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.822+0000 7f5499d9b700 1 --2- 172.21.15.39:0/1249229835 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f94feb06b0 0x55f9500baf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.822+0000 7f5499d9b700 1 -- 172.21.15.39:0/1249229835 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f548c0b6040 con 0x55f950054580 2023-07-21T18:46:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.822+0000 7f549a59c700 1 --2- 172.21.15.39:0/1249229835 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f94feb06b0 0x55f9500baf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:46:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.822+0000 7f549ad9d700 1 --2- 172.21.15.39:0/1249229835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f95005a2d0 0x55f9500b9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:46:23.846 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.822+0000 7f5499d9b700 1 --2- 172.21.15.39:0/1249229835 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f950054580 0x55f9500bb440 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5490026400 tx=0x7f5490026360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93054ec3b6d97302 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:46:23.846 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.822+0000 7f5481ffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-21T18:46:23.846 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.822+0000 7f5481ffb700 1 -- 172.21.15.39:0/1249229835 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549002b040 con 0x55f950054580 2023-07-21T18:46:23.846 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.822+0000 7f5481ffb700 1 -- 172.21.15.39:0/1249229835 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f549000fe70 con 0x55f950054580 2023-07-21T18:46:23.846 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.822+0000 7f54acf9eb80 1 -- 172.21.15.39:0/1249229835 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f548c0a2380 con 0x55f950054580 2023-07-21T18:46:23.846 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.822+0000 7f5481ffb700 1 -- 172.21.15.39:0/1249229835 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5490026920 con 0x55f950054580 2023-07-21T18:46:23.846 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.822+0000 7f54acf9eb80 10 client.25973 fetch_fsmap learned FSMap version 43 2023-07-21T18:46:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.822+0000 7f54acf9eb80 1 -- 172.21.15.39:0/1249229835 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x7f5494003c90 con 0x55f950054580 2023-07-21T18:46:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.823+0000 7f5481ffb700 1 -- 172.21.15.39:0/1249229835 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f548c0a2380 con 0x55f950054580 2023-07-21T18:46:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.823+0000 7f5481ffb700 1 -- 172.21.15.39:0/1249229835 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 43) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f5490030d10 con 0x55f950054580 2023-07-21T18:46:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.823+0000 7f54acf9eb80 10 client.25973 fetch_fsmap finished waiting for FSMap version 43 2023-07-21T18:46:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.823+0000 7f54acf9eb80 10 client.25973 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:46:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.823+0000 7f54acf9eb80 20 client.25973 populate_metadata read hostname 'smithi039' 2023-07-21T18:46:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.823+0000 7f54acf9eb80 1 --2- 172.21.15.39:0/1249229835 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55f9500caf30 0x55f9500cd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.823+0000 7f54acf9eb80 4 client.25973 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:46:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.823+0000 7f54acf9eb80 1 -- 172.21.15.39:0/1249229835 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f94ff60ff0 con 0x55f9500caf30 2023-07-21T18:46:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.823+0000 7f549a59c700 1 --2- 172.21.15.39:0/1249229835 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55f9500caf30 0x55f9500cd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.823+0000 7f549a59c700 1 --2- 172.21.15.39:0/1249229835 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55f9500caf30 0x55f9500cd340 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1d1be461b028c01 server_cookie=7bc170b44298b8dd in_seq=0 out_seq=0 2023-07-21T18:46:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.823+0000 7f5481ffb700 10 client.25973 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:46:23.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.867+0000 7f5481ffb700 1 -- 172.21.15.39:0/1249229835 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f94ff60ff0 con 0x55f9500caf30 2023-07-21T18:46:23.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.867+0000 7f5481ffb700 10 client.25973 handle_command_reply: tid=0 2023-07-21T18:46:23.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.867+0000 7f54acf9eb80 1 client.25973 shutdown 2023-07-21T18:46:23.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.867+0000 7f54acf9eb80 1 -- 172.21.15.39:0/1249229835 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f950054580 msgr2=0x55f9500bb440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:46:23.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.867+0000 7f54acf9eb80 1 --2- 172.21.15.39:0/1249229835 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f950054580 0x55f9500bb440 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5490026400 tx=0x7f5490026360 comp rx=0 tx=0).stop 2023-07-21T18:46:23.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.867+0000 7f54acf9eb80 1 -- 172.21.15.39:0/1249229835 shutdown_connections 2023-07-21T18:46:23.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.867+0000 7f54acf9eb80 1 --2- 172.21.15.39:0/1249229835 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55f9500caf30 0x55f9500cd340 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:23.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.867+0000 7f54acf9eb80 1 --2- 172.21.15.39:0/1249229835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f95005a2d0 0x55f9500b9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:23.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.867+0000 7f5481ffb700 0 client.25973 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:46:23.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.867+0000 7f54acf9eb80 1 --2- 172.21.15.39:0/1249229835 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f94feb06b0 0x55f9500baf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:23.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.867+0000 7f54acf9eb80 1 --2- 172.21.15.39:0/1249229835 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f950054580 0x55f9500bb440 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:23.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.867+0000 7f54acf9eb80 1 -- 172.21.15.39:0/1249229835 >> 172.21.15.39:0/1249229835 conn(0x55f950049be0 msgr2=0x55f950059190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:46:23.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.867+0000 7f54acf9eb80 1 -- 172.21.15.39:0/1249229835 shutdown_connections 2023-07-21T18:46:23.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.867+0000 7f54acf9eb80 1 -- 172.21.15.39:0/1249229835 wait complete. 2023-07-21T18:46:23.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.867+0000 7f54acf9eb80 20 client.25973 trim_cache size 0 max 16384 2023-07-21T18:46:23.886 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:46:23.886 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:46:23.886 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:46:23.886 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:46:23.887 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.869+0000 7f54a0e18700 1 -- 172.21.15.39:0/428350284 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f54840a1340 msgr2=0x7f54840a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:46:23.887 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.869+0000 7f54a0e18700 1 --2- 172.21.15.39:0/428350284 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f54840a1340 0x7f54840a37f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f549c0fc9b0 tx=0x7f5490005fb0 comp rx=0 tx=0).stop 2023-07-21T18:46:23.887 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.869+0000 7f54a0e18700 1 -- 172.21.15.39:0/428350284 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f549c0fea30 msgr2=0x7f549c19bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:46:23.887 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.869+0000 7f54a0e18700 1 --2- 172.21.15.39:0/428350284 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f549c0fea30 0x7f549c19bb90 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f548c00e970 tx=0x7f548c00ec80 comp rx=0 tx=0).stop 2023-07-21T18:46:23.887 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.870+0000 7f54a0e18700 1 -- 172.21.15.39:0/428350284 shutdown_connections 2023-07-21T18:46:23.887 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.870+0000 7f54a0e18700 1 --2- 172.21.15.39:0/428350284 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f549c104a30 0x7f549c1959f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:23.887 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.870+0000 7f54a0e18700 1 --2- 172.21.15.39:0/428350284 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f549c0fea30 0x7f549c19bb90 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:23.888 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.870+0000 7f54a0e18700 1 --2- 172.21.15.39:0/428350284 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f54840a1340 0x7f54840a37f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:23.888 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.870+0000 7f54a0e18700 1 --2- 172.21.15.39:0/428350284 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f549c0ff3e0 0x7f549c19c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:23.888 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.870+0000 7f54a0e18700 1 -- 172.21.15.39:0/428350284 >> 172.21.15.39:0/428350284 conn(0x7f549c074760 msgr2=0x7f549c067920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:46:23.888 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.870+0000 7f54a0e18700 1 -- 172.21.15.39:0/428350284 shutdown_connections 2023-07-21T18:46:23.888 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:23.870+0000 7f54a0e18700 1 -- 172.21.15.39:0/428350284 wait complete. 2023-07-21T18:46:24.270 INFO:tasks.workunit.client.0.smithi039.stdout: 1 245071 5.12 MB/sec execute 97 sec latency 35.612 ms 2023-07-21T18:46:24.363 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:46:24 smithi193 ceph-mon[98204]: pgmap v143: 97 pgs: 97 active+clean; 98 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 226 op/s 2023-07-21T18:46:24.382 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:46:24 smithi155 ceph-mon[97934]: pgmap v143: 97 pgs: 97 active+clean; 98 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 226 op/s 2023-07-21T18:46:24.431 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:46:24 smithi039 ceph-mon[96489]: pgmap v143: 97 pgs: 97 active+clean; 98 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 226 op/s 2023-07-21T18:46:25.271 INFO:tasks.workunit.client.0.smithi039.stdout: 1 246318 5.13 MB/sec execute 98 sec latency 14.632 ms 2023-07-21T18:46:26.270 INFO:tasks.workunit.client.0.smithi039.stdout: 1 248127 5.17 MB/sec execute 99 sec latency 48.139 ms 2023-07-21T18:46:26.363 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:46:26 smithi193 ceph-mon[98204]: pgmap v144: 97 pgs: 97 active+clean; 98 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 226 op/s 2023-07-21T18:46:26.382 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:46:26 smithi155 ceph-mon[97934]: pgmap v144: 97 pgs: 97 active+clean; 98 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 226 op/s 2023-07-21T18:46:26.431 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:46:26 smithi039 ceph-mon[96489]: pgmap v144: 97 pgs: 97 active+clean; 98 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 226 op/s 2023-07-21T18:46:27.270 INFO:tasks.workunit.client.0.smithi039.stdout: 1 249367 5.18 MB/sec execute 100 sec latency 12.414 ms 2023-07-21T18:46:28.271 INFO:tasks.workunit.client.0.smithi039.stdout: 1 251350 5.24 MB/sec execute 101 sec latency 47.342 ms 2023-07-21T18:46:28.364 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:46:28 smithi193 ceph-mon[98204]: pgmap v145: 97 pgs: 97 active+clean; 96 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 358 op/s 2023-07-21T18:46:28.382 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:46:28 smithi155 ceph-mon[97934]: pgmap v145: 97 pgs: 97 active+clean; 96 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 358 op/s 2023-07-21T18:46:28.431 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:46:28 smithi039 ceph-mon[96489]: pgmap v145: 97 pgs: 97 active+clean; 96 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 358 op/s 2023-07-21T18:46:29.270 INFO:tasks.workunit.client.0.smithi039.stdout: 1 252753 5.25 MB/sec execute 102 sec latency 15.757 ms 2023-07-21T18:46:30.271 INFO:tasks.workunit.client.0.smithi039.stdout: 1 254270 5.27 MB/sec execute 103 sec latency 16.287 ms 2023-07-21T18:46:30.363 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:46:30 smithi193 ceph-mon[98204]: pgmap v146: 97 pgs: 97 active+clean; 96 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 230 op/s 2023-07-21T18:46:30.382 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:46:30 smithi155 ceph-mon[97934]: pgmap v146: 97 pgs: 97 active+clean; 96 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 230 op/s 2023-07-21T18:46:30.431 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:46:30 smithi039 ceph-mon[96489]: pgmap v146: 97 pgs: 97 active+clean; 96 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 230 op/s 2023-07-21T18:46:31.270 INFO:tasks.workunit.client.0.smithi039.stdout: 1 255843 5.33 MB/sec execute 104 sec latency 12.505 ms 2023-07-21T18:46:32.271 INFO:tasks.workunit.client.0.smithi039.stdout: 1 257562 5.34 MB/sec execute 105 sec latency 17.496 ms 2023-07-21T18:46:32.363 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:46:32 smithi193 ceph-mon[98204]: pgmap v147: 97 pgs: 97 active+clean; 100 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 338 op/s 2023-07-21T18:46:32.382 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:46:32 smithi155 ceph-mon[97934]: pgmap v147: 97 pgs: 97 active+clean; 100 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 338 op/s 2023-07-21T18:46:32.431 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:46:32 smithi039 ceph-mon[96489]: pgmap v147: 97 pgs: 97 active+clean; 100 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 338 op/s 2023-07-21T18:46:33.271 INFO:tasks.workunit.client.0.smithi039.stdout: 1 259295 5.41 MB/sec execute 106 sec latency 21.863 ms 2023-07-21T18:46:34.271 INFO:tasks.workunit.client.0.smithi039.stdout: 1 260664 5.41 MB/sec execute 107 sec latency 16.501 ms 2023-07-21T18:46:34.363 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:46:34 smithi193 ceph-mon[98204]: pgmap v148: 97 pgs: 97 active+clean; 100 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 240 op/s 2023-07-21T18:46:34.382 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:46:34 smithi155 ceph-mon[97934]: pgmap v148: 97 pgs: 97 active+clean; 100 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 240 op/s 2023-07-21T18:46:34.431 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:46:34 smithi039 ceph-mon[96489]: pgmap v148: 97 pgs: 97 active+clean; 100 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 240 op/s 2023-07-21T18:46:35.271 INFO:tasks.workunit.client.0.smithi039.stdout: 1 262775 5.49 MB/sec execute 108 sec latency 15.940 ms 2023-07-21T18:46:36.271 INFO:tasks.workunit.client.0.smithi039.stdout: 1 263938 5.48 MB/sec execute 109 sec latency 59.874 ms 2023-07-21T18:46:36.363 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:46:36 smithi193 ceph-mon[98204]: pgmap v149: 97 pgs: 97 active+clean; 100 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 240 op/s 2023-07-21T18:46:36.382 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:46:36 smithi155 ceph-mon[97934]: pgmap v149: 97 pgs: 97 active+clean; 100 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 240 op/s 2023-07-21T18:46:36.431 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:46:36 smithi039 ceph-mon[96489]: pgmap v149: 97 pgs: 97 active+clean; 100 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 240 op/s 2023-07-21T18:46:36.998 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.982+0000 7f1505ac5700 1 -- 172.21.15.39:0/1503450214 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1500101700 msgr2=0x7f1500109bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:46:36.998 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.982+0000 7f1505ac5700 1 --2- 172.21.15.39:0/1503450214 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1500101700 0x7f1500109bf0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f14e8009990 tx=0x7f14e8009ca0 comp rx=0 tx=0).stop 2023-07-21T18:46:36.998 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.982+0000 7f1505ac5700 1 -- 172.21.15.39:0/1503450214 shutdown_connections 2023-07-21T18:46:36.998 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.982+0000 7f1505ac5700 1 --2- 172.21.15.39:0/1503450214 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f150010a130 0x7f150010c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:36.998 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.982+0000 7f1505ac5700 1 --2- 172.21.15.39:0/1503450214 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1500100df0 0x7f15001011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.982+0000 7f1505ac5700 1 --2- 172.21.15.39:0/1503450214 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1500101700 0x7f1500109bf0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.982+0000 7f1505ac5700 1 -- 172.21.15.39:0/1503450214 >> 172.21.15.39:0/1503450214 conn(0x7f1500075bd0 msgr2=0x7f1500075fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:46:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.982+0000 7f1505ac5700 1 -- 172.21.15.39:0/1503450214 shutdown_connections 2023-07-21T18:46:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.982+0000 7f1505ac5700 1 -- 172.21.15.39:0/1503450214 wait complete. 2023-07-21T18:46:37.000 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.982+0000 7f1505ac5700 1 Processor -- start 2023-07-21T18:46:37.000 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.982+0000 7f1505ac5700 1 -- start start 2023-07-21T18:46:37.001 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.983+0000 7f1505ac5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1500100df0 0x7f150019bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:37.001 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.983+0000 7f1505ac5700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1500101700 0x7f150019c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:37.001 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.983+0000 7f1505ac5700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f150010a130 0x7f15001959e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:37.001 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.983+0000 7f1505ac5700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1500196050 con 0x7f1500100df0 2023-07-21T18:46:37.001 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.983+0000 7f1505ac5700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15001961c0 con 0x7f150010a130 2023-07-21T18:46:37.001 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.983+0000 7f1505ac5700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1500196380 con 0x7f1500101700 2023-07-21T18:46:37.001 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.983+0000 7f14ff7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1500100df0 0x7f150019bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:37.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.983+0000 7f14ff7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1500100df0 0x7f150019bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59698/0 (socket says 172.21.15.39:59698) 2023-07-21T18:46:37.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.983+0000 7f14ff7fe700 1 -- 172.21.15.39:0/3812185378 learned_addr learned my addr 172.21.15.39:0/3812185378 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:46:37.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.983+0000 7f14f6ffd700 1 --2- 172.21.15.39:0/3812185378 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1500101700 0x7f150019c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:37.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.983+0000 7f14fffff700 1 --2- 172.21.15.39:0/3812185378 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f150010a130 0x7f15001959e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:37.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.983+0000 7f14ff7fe700 1 -- 172.21.15.39:0/3812185378 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1500101700 msgr2=0x7f150019c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:46:37.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.983+0000 7f14ff7fe700 1 --2- 172.21.15.39:0/3812185378 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1500101700 0x7f150019c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:37.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.983+0000 7f14ff7fe700 1 -- 172.21.15.39:0/3812185378 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f150010a130 msgr2=0x7f15001959e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:46:37.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.983+0000 7f14ff7fe700 1 --2- 172.21.15.39:0/3812185378 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f150010a130 0x7f15001959e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:37.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.983+0000 7f14ff7fe700 1 -- 172.21.15.39:0/3812185378 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14e80095f0 con 0x7f1500100df0 2023-07-21T18:46:37.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.983+0000 7f14ff7fe700 1 --2- 172.21.15.39:0/3812185378 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1500100df0 0x7f150019bb80 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f14f000e9a0 tx=0x7f14f000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:46:37.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.984+0000 7f14fd7fa700 1 -- 172.21.15.39:0/3812185378 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f0009790 con 0x7f1500100df0 2023-07-21T18:46:37.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.984+0000 7f1505ac5700 1 -- 172.21.15.39:0/3812185378 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1500196660 con 0x7f1500100df0 2023-07-21T18:46:37.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.984+0000 7f14fd7fa700 1 -- 172.21.15.39:0/3812185378 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f14f0010430 con 0x7f1500100df0 2023-07-21T18:46:37.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.984+0000 7f14fd7fa700 1 -- 172.21.15.39:0/3812185378 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f0018680 con 0x7f1500100df0 2023-07-21T18:46:37.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.984+0000 7f1505ac5700 1 -- 172.21.15.39:0/3812185378 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15001a2b70 con 0x7f1500100df0 2023-07-21T18:46:37.006 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.986+0000 7f14fd7fa700 1 -- 172.21.15.39:0/3812185378 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f14f00105a0 con 0x7f1500100df0 2023-07-21T18:46:37.006 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.986+0000 7f14fd7fa700 1 --2- 172.21.15.39:0/3812185378 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f14ec0a12c0 0x7f14ec0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:37.006 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.986+0000 7f14fd7fa700 1 -- 172.21.15.39:0/3812185378 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f14f0014070 con 0x7f1500100df0 2023-07-21T18:46:37.006 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.986+0000 7f14f6ffd700 1 --2- 172.21.15.39:0/3812185378 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f14ec0a12c0 0x7f14ec0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:37.007 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.987+0000 7f14f6ffd700 1 --2- 172.21.15.39:0/3812185378 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f14ec0a12c0 0x7f14ec0a3770 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f15000fcde0 tx=0x7f14e8005fd0 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:46:37.007 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.987+0000 7f1511c4bb80 1 Processor -- start 2023-07-21T18:46:37.007 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.987+0000 7f1511c4bb80 1 -- start start 2023-07-21T18:46:37.007 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.988+0000 7f1511c4bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585503086b0 0x5585504b1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:37.007 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.988+0000 7f1511c4bb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5585504b22d0 0x5585504ac040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:37.007 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.988+0000 7f1511c4bb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5585504ac580 0x5585504ac9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:37.007 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.988+0000 7f1511c4bb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585503b1c50 con 0x5585504ac580 2023-07-21T18:46:37.007 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.988+0000 7f1511c4bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585504049e0 con 0x5585503086b0 2023-07-21T18:46:37.008 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.988+0000 7f1511c4bb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585503fde70 con 0x5585504b22d0 2023-07-21T18:46:37.008 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.988+0000 7f14fffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5585504ac580 0x5585504ac9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:37.008 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.988+0000 7f14fffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5585504ac580 0x5585504ac9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59710/0 (socket says 172.21.15.39:59710) 2023-07-21T18:46:37.008 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.988+0000 7f14ff7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585503086b0 0x5585504b1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:37.008 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.988+0000 7f14f6ffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5585504b22d0 0x5585504ac040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:37.008 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.988+0000 7f14fffff700 1 -- 172.21.15.39:0/3105438433 learned_addr learned my addr 172.21.15.39:0/3105438433 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:46:37.008 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.988+0000 7f14fffff700 1 -- 172.21.15.39:0/3105438433 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5585504b22d0 msgr2=0x5585504ac040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:46:37.009 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.988+0000 7f14fffff700 1 --2- 172.21.15.39:0/3105438433 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5585504b22d0 0x5585504ac040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:37.009 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.988+0000 7f14fffff700 1 -- 172.21.15.39:0/3105438433 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585503086b0 msgr2=0x5585504b1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:46:37.009 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.988+0000 7f14fffff700 1 --2- 172.21.15.39:0/3105438433 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585503086b0 0x5585504b1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:37.009 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.988+0000 7f14fffff700 1 -- 172.21.15.39:0/3105438433 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55855022bd80 con 0x5585504ac580 2023-07-21T18:46:37.009 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.988+0000 7f14fffff700 1 --2- 172.21.15.39:0/3105438433 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5585504ac580 0x5585504ac9d0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f14f8011600 tx=0x7f14f8009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9de8b3d1b988ab8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:46:37.009 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.988+0000 7f14f5ffb700 1 -- 172.21.15.39:0/3105438433 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f800e040 con 0x5585504ac580 2023-07-21T18:46:37.009 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.989+0000 7f14f5ffb700 1 -- 172.21.15.39:0/3105438433 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f14f800f040 con 0x5585504ac580 2023-07-21T18:46:37.009 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.989+0000 7f14f5ffb700 1 -- 172.21.15.39:0/3105438433 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f8004700 con 0x5585504ac580 2023-07-21T18:46:37.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.989+0000 7f1511c4bb80 1 -- 172.21.15.39:0/3105438433 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5585504ac580 msgr2=0x5585504ac9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:46:37.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.989+0000 7f1511c4bb80 1 --2- 172.21.15.39:0/3105438433 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5585504ac580 0x5585504ac9d0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f14f8011600 tx=0x7f14f8009920 comp rx=0 tx=0).stop 2023-07-21T18:46:37.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.989+0000 7f1511c4bb80 1 -- 172.21.15.39:0/3105438433 shutdown_connections 2023-07-21T18:46:37.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.989+0000 7f1511c4bb80 1 --2- 172.21.15.39:0/3105438433 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585503086b0 0x5585504b1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:37.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.989+0000 7f1511c4bb80 1 --2- 172.21.15.39:0/3105438433 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5585504ac580 0x5585504ac9d0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:37.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.989+0000 7f1511c4bb80 1 --2- 172.21.15.39:0/3105438433 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5585504b22d0 0x5585504ac040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:37.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.989+0000 7f1511c4bb80 1 -- 172.21.15.39:0/3105438433 >> 172.21.15.39:0/3105438433 conn(0x5585504a1be0 msgr2=0x5585504a6f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:46:37.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.989+0000 7f1511c4bb80 1 -- 172.21.15.39:0/3105438433 shutdown_connections 2023-07-21T18:46:37.011 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.989+0000 7f1511c4bb80 1 -- 172.21.15.39:0/3105438433 wait complete. 2023-07-21T18:46:37.011 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.990+0000 7f1511c4bb80 1 -- start start 2023-07-21T18:46:37.011 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.990+0000 7f1511c4bb80 1 Processor -- start 2023-07-21T18:46:37.011 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.990+0000 7f1511c4bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585503086b0 0x5585504b8530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:37.011 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.990+0000 7f1511c4bb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5585504ac580 0x5585504b5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:37.011 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.990+0000 7f1511c4bb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5585504b22d0 0x5585504b5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:37.011 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.990+0000 7f1511c4bb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55855039da60 con 0x5585504ac580 2023-07-21T18:46:37.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.990+0000 7f1511c4bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55855039d760 con 0x5585503086b0 2023-07-21T18:46:37.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.990+0000 7f1511c4bb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585502818c0 con 0x5585504b22d0 2023-07-21T18:46:37.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.990+0000 7f1511c4bb80 10 client.0 _ll_register_callbacks cb 0x55855039b700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:46:37.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.990+0000 7f14f6ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5585504ac580 0x5585504b5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:37.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.990+0000 7f14ff7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585503086b0 0x5585504b8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:37.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.990+0000 7f14f6ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5585504ac580 0x5585504b5740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59714/0 (socket says 172.21.15.39:59714) 2023-07-21T18:46:37.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.990+0000 7f14f6ffd700 1 -- 172.21.15.39:0/1394596769 learned_addr learned my addr 172.21.15.39:0/1394596769 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:46:37.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.990+0000 7f14fffff700 1 --2- 172.21.15.39:0/1394596769 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5585504b22d0 0x5585504b5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:37.015 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.991+0000 7f14f6ffd700 1 -- 172.21.15.39:0/1394596769 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5585504b22d0 msgr2=0x5585504b5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:46:37.015 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.991+0000 7f14f6ffd700 1 --2- 172.21.15.39:0/1394596769 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5585504b22d0 0x5585504b5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:37.015 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.991+0000 7f14f6ffd700 1 -- 172.21.15.39:0/1394596769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585503086b0 msgr2=0x5585504b8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:46:37.015 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.991+0000 7f14f6ffd700 1 --2- 172.21.15.39:0/1394596769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585503086b0 0x5585504b8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:37.015 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.991+0000 7f14f6ffd700 1 -- 172.21.15.39:0/1394596769 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14f80112b0 con 0x5585504ac580 2023-07-21T18:46:37.015 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.991+0000 7f14ff7fe700 1 --2- 172.21.15.39:0/1394596769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585503086b0 0x5585504b8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:46:37.015 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.991+0000 7f14fffff700 1 --2- 172.21.15.39:0/1394596769 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5585504b22d0 0x5585504b5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:46:37.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.991+0000 7f14f6ffd700 1 --2- 172.21.15.39:0/1394596769 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5585504ac580 0x5585504b5740 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f14e8009990 tx=0x7f14e801a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f165c37cda7a151 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:46:37.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.991+0000 7f14deffd700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:46:37.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.991+0000 7f14deffd700 1 -- 172.21.15.39:0/1394596769 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e802b050 con 0x5585504ac580 2023-07-21T18:46:37.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.991+0000 7f14deffd700 1 -- 172.21.15.39:0/1394596769 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f14e800f460 con 0x5585504ac580 2023-07-21T18:46:37.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.991+0000 7f1511c4bb80 1 -- 172.21.15.39:0/1394596769 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558550357490 con 0x5585504ac580 2023-07-21T18:46:37.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.991+0000 7f14deffd700 1 -- 172.21.15.39:0/1394596769 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14e8005920 con 0x5585504ac580 2023-07-21T18:46:37.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.991+0000 7f1511c4bb80 10 client.16497 fetch_fsmap learned FSMap version 43 2023-07-21T18:46:37.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.991+0000 7f1511c4bb80 1 -- 172.21.15.39:0/1394596769 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x558550214c00 con 0x5585504ac580 2023-07-21T18:46:37.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.992+0000 7f14deffd700 1 -- 172.21.15.39:0/1394596769 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558550357490 con 0x5585504ac580 2023-07-21T18:46:37.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.992+0000 7f14deffd700 1 -- 172.21.15.39:0/1394596769 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 43) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f14e8024610 con 0x5585504ac580 2023-07-21T18:46:37.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.992+0000 7f1511c4bb80 10 client.16497 fetch_fsmap finished waiting for FSMap version 43 2023-07-21T18:46:37.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.992+0000 7f1511c4bb80 10 client.16497 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:46:37.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.992+0000 7f1511c4bb80 20 client.16497 populate_metadata read hostname 'smithi039' 2023-07-21T18:46:37.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.992+0000 7f1511c4bb80 1 --2- 172.21.15.39:0/1394596769 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5585504c5ba0 0x5585504c7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:37.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.992+0000 7f1511c4bb80 4 client.16497 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:46:37.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.992+0000 7f1511c4bb80 1 -- 172.21.15.39:0/1394596769 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5585503f6e50 con 0x5585504c5ba0 2023-07-21T18:46:37.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.992+0000 7f14ff7fe700 1 --2- 172.21.15.39:0/1394596769 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5585504c5ba0 0x5585504c7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:37.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.992+0000 7f14ff7fe700 1 --2- 172.21.15.39:0/1394596769 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5585504c5ba0 0x5585504c7fb0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf8c815764d08f2d server_cookie=6f072952c238e37a in_seq=0 out_seq=0 2023-07-21T18:46:37.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.992+0000 7f14deffd700 10 client.16497 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:46:37.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.993+0000 7f14deffd700 1 -- 172.21.15.39:0/1394596769 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5585503f6e50 con 0x5585504c5ba0 2023-07-21T18:46:37.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.993+0000 7f14deffd700 10 client.16497 handle_command_reply: tid=0 2023-07-21T18:46:37.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.993+0000 7f1511c4bb80 1 client.16497 shutdown 2023-07-21T18:46:37.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.993+0000 7f1511c4bb80 1 -- 172.21.15.39:0/1394596769 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5585504ac580 msgr2=0x5585504b5740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:46:37.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.993+0000 7f1511c4bb80 1 --2- 172.21.15.39:0/1394596769 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5585504ac580 0x5585504b5740 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f14e8009990 tx=0x7f14e801a050 comp rx=0 tx=0).stop 2023-07-21T18:46:37.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.993+0000 7f1511c4bb80 1 -- 172.21.15.39:0/1394596769 shutdown_connections 2023-07-21T18:46:37.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.993+0000 7f1511c4bb80 1 --2- 172.21.15.39:0/1394596769 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5585504c5ba0 0x5585504c7fb0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:37.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.993+0000 7f1511c4bb80 1 --2- 172.21.15.39:0/1394596769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585503086b0 0x5585504b8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:37.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.993+0000 7f1511c4bb80 1 --2- 172.21.15.39:0/1394596769 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5585504b22d0 0x5585504b5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:37.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.993+0000 7f14deffd700 0 client.16497 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:46:37.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.993+0000 7f1511c4bb80 1 --2- 172.21.15.39:0/1394596769 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5585504ac580 0x5585504b5740 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:37.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.993+0000 7f1511c4bb80 1 -- 172.21.15.39:0/1394596769 >> 172.21.15.39:0/1394596769 conn(0x5585504a1be0 msgr2=0x5585504a5c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:46:37.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.993+0000 7f1511c4bb80 1 -- 172.21.15.39:0/1394596769 shutdown_connections 2023-07-21T18:46:37.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.993+0000 7f1511c4bb80 1 -- 172.21.15.39:0/1394596769 wait complete. 2023-07-21T18:46:37.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:36.993+0000 7f1511c4bb80 20 client.16497 trim_cache size 0 max 16384 2023-07-21T18:46:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.009+0000 7f1511c4bb80 1 Processor -- start 2023-07-21T18:46:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.009+0000 7f1511c4bb80 1 -- start start 2023-07-21T18:46:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.009+0000 7f1511c4bb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5585503086b0 0x5585504ac040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:37.026 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.009+0000 7f1511c4bb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5585504ac580 0x5585504bcc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:37.026 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.009+0000 7f1511c4bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585504b22d0 0x5585504a6c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:37.026 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.009+0000 7f1511c4bb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585504a8b70 con 0x5585503086b0 2023-07-21T18:46:37.026 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.009+0000 7f1511c4bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585504b1a10 con 0x5585504b22d0 2023-07-21T18:46:37.026 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.009+0000 7f1511c4bb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585504c3120 con 0x5585504ac580 2023-07-21T18:46:37.026 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.009+0000 7f14ff7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5585503086b0 0x5585504ac040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:37.026 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.009+0000 7f14ff7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5585503086b0 0x5585504ac040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59720/0 (socket says 172.21.15.39:59720) 2023-07-21T18:46:37.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.009+0000 7f14ff7fe700 1 -- 172.21.15.39:0/845942815 learned_addr learned my addr 172.21.15.39:0/845942815 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:46:37.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.009+0000 7f14fffff700 1 --2- 172.21.15.39:0/845942815 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585504b22d0 0x5585504a6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:37.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.009+0000 7f14f6ffd700 1 --2- 172.21.15.39:0/845942815 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5585504ac580 0x5585504bcc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:37.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.010+0000 7f14ff7fe700 1 -- 172.21.15.39:0/845942815 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5585504ac580 msgr2=0x5585504bcc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:46:37.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.010+0000 7f14ff7fe700 1 --2- 172.21.15.39:0/845942815 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5585504ac580 0x5585504bcc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:37.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.010+0000 7f14ff7fe700 1 -- 172.21.15.39:0/845942815 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585504b22d0 msgr2=0x5585504a6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:46:37.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.010+0000 7f14ff7fe700 1 --2- 172.21.15.39:0/845942815 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585504b22d0 0x5585504a6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:37.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.010+0000 7f14ff7fe700 1 -- 172.21.15.39:0/845942815 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5585502521f0 con 0x5585503086b0 2023-07-21T18:46:37.028 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.010+0000 7f14ff7fe700 1 --2- 172.21.15.39:0/845942815 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5585503086b0 0x5585504ac040 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f14f0096490 tx=0x7f14f009d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3598e2c3ba3c92ef server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:46:37.030 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.010+0000 7f14f67fc700 1 -- 172.21.15.39:0/845942815 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f0098190 con 0x5585503086b0 2023-07-21T18:46:37.030 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.010+0000 7f14f67fc700 1 -- 172.21.15.39:0/845942815 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f14f00b1070 con 0x5585503086b0 2023-07-21T18:46:37.030 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.010+0000 7f14f67fc700 1 -- 172.21.15.39:0/845942815 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f0094ba0 con 0x5585503086b0 2023-07-21T18:46:37.030 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.011+0000 7f1511c4bb80 1 -- 172.21.15.39:0/845942815 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5585503086b0 msgr2=0x5585504ac040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:46:37.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.011+0000 7f1511c4bb80 1 --2- 172.21.15.39:0/845942815 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5585503086b0 0x5585504ac040 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f14f0096490 tx=0x7f14f009d040 comp rx=0 tx=0).stop 2023-07-21T18:46:37.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.011+0000 7f1511c4bb80 1 -- 172.21.15.39:0/845942815 shutdown_connections 2023-07-21T18:46:37.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.011+0000 7f1511c4bb80 1 --2- 172.21.15.39:0/845942815 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585504b22d0 0x5585504a6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:37.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.011+0000 7f1511c4bb80 1 --2- 172.21.15.39:0/845942815 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5585503086b0 0x5585504ac040 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:37.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.011+0000 7f1511c4bb80 1 --2- 172.21.15.39:0/845942815 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5585504ac580 0x5585504bcc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:37.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.011+0000 7f1511c4bb80 1 -- 172.21.15.39:0/845942815 >> 172.21.15.39:0/845942815 conn(0x5585504a1be0 msgr2=0x5585504c1da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:46:37.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.011+0000 7f1511c4bb80 1 -- 172.21.15.39:0/845942815 shutdown_connections 2023-07-21T18:46:37.032 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.011+0000 7f1511c4bb80 1 -- 172.21.15.39:0/845942815 wait complete. 2023-07-21T18:46:37.032 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.011+0000 7f1511c4bb80 1 -- start start 2023-07-21T18:46:37.032 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.011+0000 7f1511c4bb80 1 Processor -- start 2023-07-21T18:46:37.032 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.012+0000 7f1511c4bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585503086b0 0x5585504aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:37.032 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.012+0000 7f1511c4bb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5585504ac580 0x5585504af050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:37.032 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.012+0000 7f1511c4bb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5585504b22d0 0x5585504ad790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:37.032 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.012+0000 7f1511c4bb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585504adcd0 con 0x5585504ac580 2023-07-21T18:46:37.032 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.012+0000 7f1511c4bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585504ade40 con 0x5585503086b0 2023-07-21T18:46:37.033 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.012+0000 7f1511c4bb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585504ae000 con 0x5585504b22d0 2023-07-21T18:46:37.033 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.012+0000 7f1511c4bb80 10 client.0 _ll_register_callbacks cb 0x7f14e8005350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:46:37.033 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.012+0000 7f14f6ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5585504ac580 0x5585504af050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:37.033 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.012+0000 7f14f6ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5585504ac580 0x5585504af050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59728/0 (socket says 172.21.15.39:59728) 2023-07-21T18:46:37.033 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.012+0000 7f14f6ffd700 1 -- 172.21.15.39:0/1568941172 learned_addr learned my addr 172.21.15.39:0/1568941172 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:46:37.033 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.012+0000 7f14ff7fe700 1 --2- 172.21.15.39:0/1568941172 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585503086b0 0x5585504aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:37.033 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.012+0000 7f14fffff700 1 --2- 172.21.15.39:0/1568941172 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5585504b22d0 0x5585504ad790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:37.034 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.012+0000 7f14f6ffd700 1 -- 172.21.15.39:0/1568941172 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5585504b22d0 msgr2=0x5585504ad790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:46:37.034 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.012+0000 7f14f6ffd700 1 --2- 172.21.15.39:0/1568941172 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5585504b22d0 0x5585504ad790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:37.034 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.012+0000 7f14f6ffd700 1 -- 172.21.15.39:0/1568941172 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585503086b0 msgr2=0x5585504aeb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:46:37.034 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.012+0000 7f14f6ffd700 1 --2- 172.21.15.39:0/1568941172 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585503086b0 0x5585504aeb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:37.034 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.012+0000 7f14f6ffd700 1 -- 172.21.15.39:0/1568941172 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15001a2b70 con 0x5585504ac580 2023-07-21T18:46:37.034 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.012+0000 7f14ff7fe700 1 --2- 172.21.15.39:0/1568941172 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585503086b0 0x5585504aeb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:46:37.034 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.012+0000 7f14f6ffd700 1 --2- 172.21.15.39:0/1568941172 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5585504ac580 0x5585504af050 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f14e8005f50 tx=0x7f14e8005920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ea0dc22cb2744d7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:46:37.034 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.012+0000 7f14deffd700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:46:37.035 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.012+0000 7f14deffd700 1 -- 172.21.15.39:0/1568941172 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e802b040 con 0x5585504ac580 2023-07-21T18:46:37.035 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.012+0000 7f14deffd700 1 -- 172.21.15.39:0/1568941172 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f14e8004e50 con 0x5585504ac580 2023-07-21T18:46:37.035 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.012+0000 7f1511c4bb80 1 -- 172.21.15.39:0/1568941172 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f14f800f040 con 0x5585504ac580 2023-07-21T18:46:37.035 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.013+0000 7f14deffd700 1 -- 172.21.15.39:0/1568941172 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14e8024da0 con 0x5585504ac580 2023-07-21T18:46:37.035 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.013+0000 7f1511c4bb80 10 client.16503 fetch_fsmap learned FSMap version 43 2023-07-21T18:46:37.035 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.013+0000 7f1511c4bb80 1 -- 172.21.15.39:0/1568941172 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55855022a250 con 0x5585504ac580 2023-07-21T18:46:37.035 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.013+0000 7f14deffd700 1 -- 172.21.15.39:0/1568941172 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f14e804f020 con 0x5585504ac580 2023-07-21T18:46:37.035 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.013+0000 7f14deffd700 1 -- 172.21.15.39:0/1568941172 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 43) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f14e803a450 con 0x5585504ac580 2023-07-21T18:46:37.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.013+0000 7f1511c4bb80 10 client.16503 fetch_fsmap finished waiting for FSMap version 43 2023-07-21T18:46:37.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.013+0000 7f1511c4bb80 10 client.16503 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:46:37.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.013+0000 7f1511c4bb80 20 client.16503 populate_metadata read hostname 'smithi039' 2023-07-21T18:46:37.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.013+0000 7f1511c4bb80 1 --2- 172.21.15.39:0/1568941172 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5585504c5ba0 0x55855051c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:37.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.013+0000 7f1511c4bb80 4 client.16503 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:46:37.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.013+0000 7f1511c4bb80 1 -- 172.21.15.39:0/1568941172 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5585503b8ff0 con 0x5585504c5ba0 2023-07-21T18:46:37.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.013+0000 7f14ff7fe700 1 --2- 172.21.15.39:0/1568941172 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5585504c5ba0 0x55855051c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:37.037 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.014+0000 7f14ff7fe700 1 --2- 172.21.15.39:0/1568941172 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5585504c5ba0 0x55855051c620 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8e73c4446b14428 server_cookie=6c7ed6354448c1da in_seq=0 out_seq=0 2023-07-21T18:46:37.037 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.014+0000 7f14deffd700 10 client.16503 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:46:37.066 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.050+0000 7f14deffd700 1 -- 172.21.15.39:0/1568941172 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5585503b8ff0 con 0x5585504c5ba0 2023-07-21T18:46:37.066 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.050+0000 7f14deffd700 10 client.16503 handle_command_reply: tid=0 2023-07-21T18:46:37.066 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.050+0000 7f1511c4bb80 1 client.16503 shutdown 2023-07-21T18:46:37.066 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.050+0000 7f1511c4bb80 1 -- 172.21.15.39:0/1568941172 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5585504ac580 msgr2=0x5585504af050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:46:37.066 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.050+0000 7f1511c4bb80 1 --2- 172.21.15.39:0/1568941172 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5585504ac580 0x5585504af050 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f14e8005f50 tx=0x7f14e8005920 comp rx=0 tx=0).stop 2023-07-21T18:46:37.067 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.050+0000 7f1511c4bb80 1 -- 172.21.15.39:0/1568941172 shutdown_connections 2023-07-21T18:46:37.067 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.050+0000 7f1511c4bb80 1 --2- 172.21.15.39:0/1568941172 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5585504c5ba0 0x55855051c620 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:37.067 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.050+0000 7f1511c4bb80 1 --2- 172.21.15.39:0/1568941172 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585503086b0 0x5585504aeb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:37.067 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.050+0000 7f1511c4bb80 1 --2- 172.21.15.39:0/1568941172 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5585504b22d0 0x5585504ad790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:37.067 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.050+0000 7f1511c4bb80 1 --2- 172.21.15.39:0/1568941172 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5585504ac580 0x5585504af050 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:37.067 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.050+0000 7f1511c4bb80 1 -- 172.21.15.39:0/1568941172 >> 172.21.15.39:0/1568941172 conn(0x5585504a1be0 msgr2=0x5585504b56e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:46:37.067 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.050+0000 7f14deffd700 0 client.16503 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:46:37.067 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.050+0000 7f1511c4bb80 1 -- 172.21.15.39:0/1568941172 shutdown_connections 2023-07-21T18:46:37.068 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.050+0000 7f1511c4bb80 1 -- 172.21.15.39:0/1568941172 wait complete. 2023-07-21T18:46:37.068 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.050+0000 7f1511c4bb80 20 client.16503 trim_cache size 0 max 16384 2023-07-21T18:46:37.068 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:46:37.068 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:46:37.069 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:46:37.069 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:46:37.069 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.051+0000 7f1505ac5700 1 -- 172.21.15.39:0/3812185378 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f14ec0a12c0 msgr2=0x7f14ec0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:46:37.069 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.051+0000 7f1505ac5700 1 --2- 172.21.15.39:0/3812185378 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f14ec0a12c0 0x7f14ec0a3770 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f15000fcde0 tx=0x7f14e8005fd0 comp rx=0 tx=0).stop 2023-07-21T18:46:37.070 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.051+0000 7f1505ac5700 1 -- 172.21.15.39:0/3812185378 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1500100df0 msgr2=0x7f150019bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:46:37.070 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.051+0000 7f1505ac5700 1 --2- 172.21.15.39:0/3812185378 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1500100df0 0x7f150019bb80 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f14f000e9a0 tx=0x7f14f000ecb0 comp rx=0 tx=0).stop 2023-07-21T18:46:37.070 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.051+0000 7f1505ac5700 1 -- 172.21.15.39:0/3812185378 shutdown_connections 2023-07-21T18:46:37.070 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.051+0000 7f1505ac5700 1 --2- 172.21.15.39:0/3812185378 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f14ec0a12c0 0x7f14ec0a3770 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:37.070 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.051+0000 7f1505ac5700 1 --2- 172.21.15.39:0/3812185378 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f150010a130 0x7f15001959e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:37.070 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.051+0000 7f1505ac5700 1 --2- 172.21.15.39:0/3812185378 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1500100df0 0x7f150019bb80 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:37.070 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.051+0000 7f1505ac5700 1 --2- 172.21.15.39:0/3812185378 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1500101700 0x7f150019c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:37.071 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.051+0000 7f1505ac5700 1 -- 172.21.15.39:0/3812185378 >> 172.21.15.39:0/3812185378 conn(0x7f1500075bd0 msgr2=0x7f150010ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:46:37.071 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.051+0000 7f1505ac5700 1 -- 172.21.15.39:0/3812185378 shutdown_connections 2023-07-21T18:46:37.071 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:37.051+0000 7f1505ac5700 1 -- 172.21.15.39:0/3812185378 wait complete. 2023-07-21T18:46:37.271 INFO:tasks.workunit.client.0.smithi039.stdout: 1 265389 5.49 MB/sec execute 110 sec latency 20.834 ms 2023-07-21T18:46:38.271 INFO:tasks.workunit.client.0.smithi039.stdout: 1 266561 5.51 MB/sec execute 111 sec latency 57.928 ms 2023-07-21T18:46:38.363 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:46:38 smithi193 ceph-mon[98204]: pgmap v150: 97 pgs: 97 active+clean; 105 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.6 MiB/s wr, 353 op/s 2023-07-21T18:46:38.382 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:46:38 smithi155 ceph-mon[97934]: pgmap v150: 97 pgs: 97 active+clean; 105 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.6 MiB/s wr, 353 op/s 2023-07-21T18:46:38.431 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:46:38 smithi039 ceph-mon[96489]: pgmap v150: 97 pgs: 97 active+clean; 105 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.6 MiB/s wr, 353 op/s 2023-07-21T18:46:39.271 INFO:tasks.workunit.client.0.smithi039.stdout: 1 268399 5.52 MB/sec execute 112 sec latency 63.624 ms 2023-07-21T18:46:40.271 INFO:tasks.workunit.client.0.smithi039.stdout: 1 270001 5.58 MB/sec execute 113 sec latency 10.401 ms 2023-07-21T18:46:40.363 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:46:40 smithi193 ceph-mon[98204]: pgmap v151: 97 pgs: 97 active+clean; 105 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 221 op/s 2023-07-21T18:46:40.382 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:46:40 smithi155 ceph-mon[97934]: pgmap v151: 97 pgs: 97 active+clean; 105 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 221 op/s 2023-07-21T18:46:40.431 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:46:40 smithi039 ceph-mon[96489]: pgmap v151: 97 pgs: 97 active+clean; 105 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 221 op/s 2023-07-21T18:46:41.271 INFO:tasks.workunit.client.0.smithi039.stdout: 1 271261 5.58 MB/sec execute 114 sec latency 16.039 ms 2023-07-21T18:46:42.271 INFO:tasks.workunit.client.0.smithi039.stdout: 1 273242 5.64 MB/sec execute 115 sec latency 15.550 ms 2023-07-21T18:46:42.363 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:46:42 smithi193 ceph-mon[98204]: pgmap v152: 97 pgs: 97 active+clean; 102 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 349 op/s 2023-07-21T18:46:42.382 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:46:42 smithi155 ceph-mon[97934]: pgmap v152: 97 pgs: 97 active+clean; 102 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 349 op/s 2023-07-21T18:46:42.431 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:46:42 smithi039 ceph-mon[96489]: pgmap v152: 97 pgs: 97 active+clean; 102 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 349 op/s 2023-07-21T18:46:43.271 INFO:tasks.workunit.client.0.smithi039.stdout: 1 274335 5.62 MB/sec execute 116 sec latency 17.705 ms 2023-07-21T18:46:43.382 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:46:43 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:46:43.431 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:46:43 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:46:43.613 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:46:43 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:46:44.271 INFO:tasks.workunit.client.0.smithi039.stdout: 1 276489 5.68 MB/sec execute 117 sec latency 31.348 ms 2023-07-21T18:46:44.382 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:46:44 smithi155 ceph-mon[97934]: pgmap v153: 97 pgs: 97 active+clean; 102 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 240 op/s 2023-07-21T18:46:44.431 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:46:44 smithi039 ceph-mon[96489]: pgmap v153: 97 pgs: 97 active+clean; 102 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 240 op/s 2023-07-21T18:46:44.613 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:46:44 smithi193 ceph-mon[98204]: pgmap v153: 97 pgs: 97 active+clean; 102 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 240 op/s 2023-07-21T18:46:45.271 INFO:tasks.workunit.client.0.smithi039.stdout: 1 277672 5.68 MB/sec execute 118 sec latency 15.270 ms 2023-07-21T18:46:46.271 INFO:tasks.workunit.client.0.smithi039.stdout: 1 279090 5.68 MB/sec execute 119 sec latency 18.164 ms 2023-07-21T18:46:46.382 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:46:46 smithi155 ceph-mon[97934]: pgmap v154: 97 pgs: 97 active+clean; 102 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 240 op/s 2023-07-21T18:46:46.431 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:46:46 smithi039 ceph-mon[96489]: pgmap v154: 97 pgs: 97 active+clean; 102 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 240 op/s 2023-07-21T18:46:46.613 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:46:46 smithi193 ceph-mon[98204]: pgmap v154: 97 pgs: 97 active+clean; 102 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 240 op/s 2023-07-21T18:46:47.271 INFO:tasks.workunit.client.0.smithi039.stdout: 1 280649 5.74 MB/sec execute 120 sec latency 11.988 ms 2023-07-21T18:46:48.271 INFO:tasks.workunit.client.0.smithi039.stdout: 1 282134 5.73 MB/sec execute 121 sec latency 36.427 ms 2023-07-21T18:46:48.382 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:46:48 smithi155 ceph-mon[97934]: pgmap v155: 97 pgs: 97 active+clean; 107 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.7 MiB/s wr, 350 op/s 2023-07-21T18:46:48.431 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:46:48 smithi039 ceph-mon[96489]: pgmap v155: 97 pgs: 97 active+clean; 107 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.7 MiB/s wr, 350 op/s 2023-07-21T18:46:48.613 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:46:48 smithi193 ceph-mon[98204]: pgmap v155: 97 pgs: 97 active+clean; 107 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.7 MiB/s wr, 350 op/s 2023-07-21T18:46:49.271 INFO:tasks.workunit.client.0.smithi039.stdout: 1 284083 5.80 MB/sec execute 122 sec latency 19.916 ms 2023-07-21T18:46:50.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.167+0000 7f6084259700 1 -- 172.21.15.39:0/2806135383 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f607c0ff470 msgr2=0x7f607c10b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:46:50.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.167+0000 7f6084259700 1 --2- 172.21.15.39:0/2806135383 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f607c0ff470 0x7f607c10b900 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f607400b0a0 tx=0x7f607400b3b0 comp rx=0 tx=0).stop 2023-07-21T18:46:50.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.167+0000 7f6084259700 1 -- 172.21.15.39:0/2806135383 shutdown_connections 2023-07-21T18:46:50.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.167+0000 7f6084259700 1 --2- 172.21.15.39:0/2806135383 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f607c104ad0 0x7f607c104ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:50.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.167+0000 7f6084259700 1 --2- 172.21.15.39:0/2806135383 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f607c0ff470 0x7f607c10b900 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:50.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.167+0000 7f6084259700 1 --2- 172.21.15.39:0/2806135383 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f607c0feac0 0x7f607c0fef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:50.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.167+0000 7f6084259700 1 -- 172.21.15.39:0/2806135383 >> 172.21.15.39:0/2806135383 conn(0x7f607c075bd0 msgr2=0x7f607c075fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:46:50.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.167+0000 7f6084259700 1 -- 172.21.15.39:0/2806135383 shutdown_connections 2023-07-21T18:46:50.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.167+0000 7f6084259700 1 -- 172.21.15.39:0/2806135383 wait complete. 2023-07-21T18:46:50.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.168+0000 7f6084259700 1 Processor -- start 2023-07-21T18:46:50.186 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.168+0000 7f6084259700 1 -- start start 2023-07-21T18:46:50.186 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.168+0000 7f6084259700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f607c0feac0 0x7f607c194ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:50.186 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.168+0000 7f6084259700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f607c0ff470 0x7f607c194ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:50.186 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.168+0000 7f6084259700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f607c104ad0 0x7f607c1a3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:50.186 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.168+0000 7f6084259700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607c195740 con 0x7f607c0feac0 2023-07-21T18:46:50.186 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.168+0000 7f6084259700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607c1a4490 con 0x7f607c104ad0 2023-07-21T18:46:50.186 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.168+0000 7f6084259700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607c1a4620 con 0x7f607c0ff470 2023-07-21T18:46:50.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.168+0000 7f6081ff5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f607c0feac0 0x7f607c194ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:50.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.168+0000 7f60827f6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f607c104ad0 0x7f607c1a3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:50.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.168+0000 7f60817f4700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f607c0ff470 0x7f607c194ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:50.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.168+0000 7f6081ff5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f607c0feac0 0x7f607c194ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59740/0 (socket says 172.21.15.39:59740) 2023-07-21T18:46:50.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.168+0000 7f6081ff5700 1 -- 172.21.15.39:0/3566897409 learned_addr learned my addr 172.21.15.39:0/3566897409 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:46:50.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.169+0000 7f6081ff5700 1 -- 172.21.15.39:0/3566897409 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f607c0ff470 msgr2=0x7f607c194ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:46:50.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.169+0000 7f6081ff5700 1 --2- 172.21.15.39:0/3566897409 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f607c0ff470 0x7f607c194ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:50.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.169+0000 7f6081ff5700 1 -- 172.21.15.39:0/3566897409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f607c104ad0 msgr2=0x7f607c1a3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:46:50.188 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.169+0000 7f6081ff5700 1 --2- 172.21.15.39:0/3566897409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f607c104ad0 0x7f607c1a3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:50.188 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.169+0000 7f6081ff5700 1 -- 172.21.15.39:0/3566897409 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6078009520 con 0x7f607c0feac0 2023-07-21T18:46:50.188 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.169+0000 7f60827f6700 1 --2- 172.21.15.39:0/3566897409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f607c104ad0 0x7f607c1a3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:46:50.188 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.169+0000 7f60817f4700 1 --2- 172.21.15.39:0/3566897409 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f607c0ff470 0x7f607c194ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:46:50.188 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.169+0000 7f6081ff5700 1 --2- 172.21.15.39:0/3566897409 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f607c0feac0 0x7f607c194ab0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f607800ea50 tx=0x7f607800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:46:50.188 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.169+0000 7f6072ffd700 1 -- 172.21.15.39:0/3566897409 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607800cbd0 con 0x7f607c0feac0 2023-07-21T18:46:50.188 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.169+0000 7f6072ffd700 1 -- 172.21.15.39:0/3566897409 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f607800cd30 con 0x7f607c0feac0 2023-07-21T18:46:50.189 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.169+0000 7f6084259700 1 -- 172.21.15.39:0/3566897409 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6074009d00 con 0x7f607c0feac0 2023-07-21T18:46:50.189 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.169+0000 7f6072ffd700 1 -- 172.21.15.39:0/3566897409 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6078018680 con 0x7f607c0feac0 2023-07-21T18:46:50.189 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.169+0000 7f6084259700 1 -- 172.21.15.39:0/3566897409 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f607c1a4cd0 con 0x7f607c0feac0 2023-07-21T18:46:50.191 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.172+0000 7f6072ffd700 1 -- 172.21.15.39:0/3566897409 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f6078010430 con 0x7f607c0feac0 2023-07-21T18:46:50.191 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.172+0000 7f6072ffd700 1 --2- 172.21.15.39:0/3566897409 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f60680a1340 0x7f60680a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:50.191 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.173+0000 7f6072ffd700 1 -- 172.21.15.39:0/3566897409 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f6078014070 con 0x7f607c0feac0 2023-07-21T18:46:50.191 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.173+0000 7f60817f4700 1 --2- 172.21.15.39:0/3566897409 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f60680a1340 0x7f60680a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:50.191 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.173+0000 7f60817f4700 1 --2- 172.21.15.39:0/3566897409 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f60680a1340 0x7f60680a37f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f607c069340 tx=0x7f606c009260 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:46:50.191 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.173+0000 7f60903dfb80 1 Processor -- start 2023-07-21T18:46:50.191 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.173+0000 7f60903dfb80 1 -- start start 2023-07-21T18:46:50.192 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.174+0000 7f60903dfb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5641e1ae96b0 0x5641e1c92d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:50.192 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.174+0000 7f60903dfb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e1c932d0 0x5641e1c8d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:50.192 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.174+0000 7f60903dfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5641e1c8d580 0x5641e1c8d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:50.192 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.174+0000 7f60903dfb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641e1b92c50 con 0x5641e1ae96b0 2023-07-21T18:46:50.192 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.174+0000 7f60903dfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641e1be59e0 con 0x5641e1c8d580 2023-07-21T18:46:50.192 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.174+0000 7f60903dfb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641e1bdee70 con 0x5641e1c932d0 2023-07-21T18:46:50.192 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.174+0000 7f6081ff5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5641e1ae96b0 0x5641e1c92d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:50.193 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.174+0000 7f60817f4700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e1c932d0 0x5641e1c8d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:50.193 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.174+0000 7f6081ff5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5641e1ae96b0 0x5641e1c92d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59748/0 (socket says 172.21.15.39:59748) 2023-07-21T18:46:50.193 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.174+0000 7f60827f6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5641e1c8d580 0x5641e1c8d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:50.193 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.174+0000 7f60817f4700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e1c932d0 0x5641e1c8d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:50070/0 (socket says 172.21.15.39:50070) 2023-07-21T18:46:50.193 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.174+0000 7f6081ff5700 1 -- 172.21.15.39:0/3859816665 learned_addr learned my addr 172.21.15.39:0/3859816665 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:46:50.193 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.174+0000 7f60817f4700 1 -- 172.21.15.39:0/3859816665 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5641e1c8d580 msgr2=0x5641e1c8d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:46:50.193 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.174+0000 7f60817f4700 1 --2- 172.21.15.39:0/3859816665 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5641e1c8d580 0x5641e1c8d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:50.193 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.174+0000 7f60817f4700 1 -- 172.21.15.39:0/3859816665 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5641e1ae96b0 msgr2=0x5641e1c92d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:46:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.174+0000 7f60817f4700 1 --2- 172.21.15.39:0/3859816665 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5641e1ae96b0 0x5641e1c92d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.174+0000 7f60817f4700 1 -- 172.21.15.39:0/3859816665 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5641e1a0cd80 con 0x5641e1c932d0 2023-07-21T18:46:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.174+0000 7f60817f4700 1 --2- 172.21.15.39:0/3859816665 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e1c932d0 0x5641e1c8d040 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f606c0042f0 tx=0x7f606c014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85692b9d61ba46c5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:46:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.174+0000 7f6067fff700 1 -- 172.21.15.39:0/3859816665 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606c003d10 con 0x5641e1c932d0 2023-07-21T18:46:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.175+0000 7f6067fff700 1 -- 172.21.15.39:0/3859816665 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f606c003e70 con 0x5641e1c932d0 2023-07-21T18:46:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.175+0000 7f6067fff700 1 -- 172.21.15.39:0/3859816665 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606c01e7f0 con 0x5641e1c932d0 2023-07-21T18:46:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.175+0000 7f60903dfb80 1 -- 172.21.15.39:0/3859816665 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e1c932d0 msgr2=0x5641e1c8d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:46:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.175+0000 7f60903dfb80 1 --2- 172.21.15.39:0/3859816665 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e1c932d0 0x5641e1c8d040 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f606c0042f0 tx=0x7f606c014000 comp rx=0 tx=0).stop 2023-07-21T18:46:50.195 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.175+0000 7f60903dfb80 1 -- 172.21.15.39:0/3859816665 shutdown_connections 2023-07-21T18:46:50.195 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.175+0000 7f60903dfb80 1 --2- 172.21.15.39:0/3859816665 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5641e1c8d580 0x5641e1c8d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:50.195 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.175+0000 7f60903dfb80 1 --2- 172.21.15.39:0/3859816665 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5641e1ae96b0 0x5641e1c92d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:50.195 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.175+0000 7f60903dfb80 1 --2- 172.21.15.39:0/3859816665 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e1c932d0 0x5641e1c8d040 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:50.195 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.175+0000 7f60903dfb80 1 -- 172.21.15.39:0/3859816665 >> 172.21.15.39:0/3859816665 conn(0x5641e1c82be0 msgr2=0x5641e1c87f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:46:50.195 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.175+0000 7f60903dfb80 1 -- 172.21.15.39:0/3859816665 shutdown_connections 2023-07-21T18:46:50.195 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.175+0000 7f60903dfb80 1 -- 172.21.15.39:0/3859816665 wait complete. 2023-07-21T18:46:50.198 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.176+0000 7f60903dfb80 1 -- start start 2023-07-21T18:46:50.198 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.176+0000 7f60903dfb80 1 Processor -- start 2023-07-21T18:46:50.198 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.176+0000 7f60903dfb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5641e1ae96b0 0x5641e1c96900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:50.198 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.176+0000 7f60903dfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5641e1c8d580 0x5641e1c96e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:50.198 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.176+0000 7f60903dfb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e1c932d0 0x5641e1c97380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:50.199 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.176+0000 7f60903dfb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641e1b7ea60 con 0x5641e1ae96b0 2023-07-21T18:46:50.199 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.176+0000 7f60903dfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641e1b7e760 con 0x5641e1c8d580 2023-07-21T18:46:50.199 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.176+0000 7f60903dfb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641e1a628c0 con 0x5641e1c932d0 2023-07-21T18:46:50.199 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.176+0000 7f60903dfb80 10 client.0 _ll_register_callbacks cb 0x5641e1b7c700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:46:50.199 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.176+0000 7f6081ff5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5641e1ae96b0 0x5641e1c96900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:50.199 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.176+0000 7f6081ff5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5641e1ae96b0 0x5641e1c96900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59754/0 (socket says 172.21.15.39:59754) 2023-07-21T18:46:50.199 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.176+0000 7f6081ff5700 1 -- 172.21.15.39:0/3182240870 learned_addr learned my addr 172.21.15.39:0/3182240870 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:46:50.199 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.176+0000 7f60817f4700 1 --2- 172.21.15.39:0/3182240870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5641e1c8d580 0x5641e1c96e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:50.200 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.176+0000 7f60827f6700 1 --2- 172.21.15.39:0/3182240870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e1c932d0 0x5641e1c97380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:50.200 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.177+0000 7f6081ff5700 1 -- 172.21.15.39:0/3182240870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e1c932d0 msgr2=0x5641e1c97380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:46:50.200 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.177+0000 7f6081ff5700 1 --2- 172.21.15.39:0/3182240870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e1c932d0 0x5641e1c97380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:50.200 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.177+0000 7f6081ff5700 1 -- 172.21.15.39:0/3182240870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5641e1c8d580 msgr2=0x5641e1c96e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:46:50.200 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.177+0000 7f6081ff5700 1 --2- 172.21.15.39:0/3182240870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5641e1c8d580 0x5641e1c96e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:50.200 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.177+0000 7f6081ff5700 1 -- 172.21.15.39:0/3182240870 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f606c017070 con 0x5641e1ae96b0 2023-07-21T18:46:50.200 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.177+0000 7f6081ff5700 1 --2- 172.21.15.39:0/3182240870 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5641e1ae96b0 0x5641e1c96900 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f607809a1d0 tx=0x7f607809a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b552d96dcf4c685 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:46:50.201 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.177+0000 7f60657fa700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:46:50.201 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.177+0000 7f60657fa700 1 -- 172.21.15.39:0/3182240870 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60780a2690 con 0x5641e1ae96b0 2023-07-21T18:46:50.201 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.177+0000 7f60657fa700 1 -- 172.21.15.39:0/3182240870 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f60780a2200 con 0x5641e1ae96b0 2023-07-21T18:46:50.201 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.177+0000 7f60903dfb80 1 -- 172.21.15.39:0/3182240870 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5641e1b38490 con 0x5641e1ae96b0 2023-07-21T18:46:50.201 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.177+0000 7f60657fa700 1 -- 172.21.15.39:0/3182240870 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60780a23c0 con 0x5641e1ae96b0 2023-07-21T18:46:50.201 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.177+0000 7f60903dfb80 10 client.16515 fetch_fsmap learned FSMap version 44 2023-07-21T18:46:50.201 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.177+0000 7f60903dfb80 1 -- 172.21.15.39:0/3182240870 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x5641e19f5c00 con 0x5641e1ae96b0 2023-07-21T18:46:50.201 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.177+0000 7f60657fa700 1 -- 172.21.15.39:0/3182240870 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5641e1b38490 con 0x5641e1ae96b0 2023-07-21T18:46:50.202 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.177+0000 7f60657fa700 1 -- 172.21.15.39:0/3182240870 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 44) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f6078099280 con 0x5641e1ae96b0 2023-07-21T18:46:50.202 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.178+0000 7f60903dfb80 10 client.16515 fetch_fsmap finished waiting for FSMap version 44 2023-07-21T18:46:50.202 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.178+0000 7f60903dfb80 10 client.16515 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:46:50.202 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.178+0000 7f60903dfb80 20 client.16515 populate_metadata read hostname 'smithi039' 2023-07-21T18:46:50.202 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.178+0000 7f60903dfb80 1 --2- 172.21.15.39:0/3182240870 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5641e1ca6ca0 0x5641e1ca90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:50.202 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.178+0000 7f60903dfb80 4 client.16515 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:46:50.202 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.178+0000 7f60903dfb80 1 -- 172.21.15.39:0/3182240870 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5641e1bd7e50 con 0x5641e1ca6ca0 2023-07-21T18:46:50.202 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.178+0000 7f60817f4700 1 --2- 172.21.15.39:0/3182240870 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5641e1ca6ca0 0x5641e1ca90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:50.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.178+0000 7f60657fa700 10 client.16515 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:46:50.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.178+0000 7f60817f4700 1 --2- 172.21.15.39:0/3182240870 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5641e1ca6ca0 0x5641e1ca90b0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d05cfa0dcedc6f6 server_cookie=f3e819c6c64c14d9 in_seq=0 out_seq=0 2023-07-21T18:46:50.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.179+0000 7f60657fa700 1 -- 172.21.15.39:0/3182240870 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5641e1bd7e50 con 0x5641e1ca6ca0 2023-07-21T18:46:50.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.179+0000 7f60657fa700 10 client.16515 handle_command_reply: tid=0 2023-07-21T18:46:50.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.179+0000 7f60903dfb80 1 client.16515 shutdown 2023-07-21T18:46:50.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.180+0000 7f60903dfb80 1 -- 172.21.15.39:0/3182240870 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5641e1ae96b0 msgr2=0x5641e1c96900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:46:50.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.180+0000 7f60903dfb80 1 --2- 172.21.15.39:0/3182240870 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5641e1ae96b0 0x5641e1c96900 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f607809a1d0 tx=0x7f607809a2c0 comp rx=0 tx=0).stop 2023-07-21T18:46:50.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.180+0000 7f60903dfb80 1 -- 172.21.15.39:0/3182240870 shutdown_connections 2023-07-21T18:46:50.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.180+0000 7f60903dfb80 1 --2- 172.21.15.39:0/3182240870 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5641e1ca6ca0 0x5641e1ca90b0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:50.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.180+0000 7f60903dfb80 1 --2- 172.21.15.39:0/3182240870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e1c932d0 0x5641e1c97380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:50.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.180+0000 7f60657fa700 0 client.16515 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:46:50.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.180+0000 7f60903dfb80 1 --2- 172.21.15.39:0/3182240870 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5641e1ae96b0 0x5641e1c96900 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:50.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.180+0000 7f60903dfb80 1 --2- 172.21.15.39:0/3182240870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5641e1c8d580 0x5641e1c96e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:50.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.180+0000 7f60903dfb80 1 -- 172.21.15.39:0/3182240870 >> 172.21.15.39:0/3182240870 conn(0x5641e1c82be0 msgr2=0x5641e1c874e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:46:50.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.180+0000 7f60903dfb80 1 -- 172.21.15.39:0/3182240870 shutdown_connections 2023-07-21T18:46:50.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.180+0000 7f60903dfb80 1 -- 172.21.15.39:0/3182240870 wait complete. 2023-07-21T18:46:50.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.180+0000 7f60903dfb80 20 client.16515 trim_cache size 0 max 16384 2023-07-21T18:46:50.210 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.195+0000 7f60903dfb80 1 Processor -- start 2023-07-21T18:46:50.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.195+0000 7f60903dfb80 1 -- start start 2023-07-21T18:46:50.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.195+0000 7f60903dfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5641e1c912a0 0x5641e1c974f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:50.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.195+0000 7f60903dfb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e1c82be0 0x5641e1c92350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:50.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.195+0000 7f60903dfb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5641e1ae96b0 0x5641e1c9dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:50.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.195+0000 7f60903dfb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641e1c936a0 con 0x5641e1ae96b0 2023-07-21T18:46:50.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.195+0000 7f60903dfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641e1c84d00 con 0x5641e1c912a0 2023-07-21T18:46:50.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.195+0000 7f60903dfb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641e1ca2b30 con 0x5641e1c82be0 2023-07-21T18:46:50.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.195+0000 7f60827f6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5641e1ae96b0 0x5641e1c9dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:50.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.195+0000 7f60827f6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5641e1ae96b0 0x5641e1c9dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59766/0 (socket says 172.21.15.39:59766) 2023-07-21T18:46:50.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.195+0000 7f60827f6700 1 -- 172.21.15.39:0/1861153300 learned_addr learned my addr 172.21.15.39:0/1861153300 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:46:50.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.195+0000 7f6081ff5700 1 --2- 172.21.15.39:0/1861153300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5641e1c912a0 0x5641e1c974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:50.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.195+0000 7f60817f4700 1 --2- 172.21.15.39:0/1861153300 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e1c82be0 0x5641e1c92350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:50.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.196+0000 7f6081ff5700 1 -- 172.21.15.39:0/1861153300 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e1c82be0 msgr2=0x5641e1c92350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:46:50.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.196+0000 7f6081ff5700 1 --2- 172.21.15.39:0/1861153300 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e1c82be0 0x5641e1c92350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:50.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.196+0000 7f6081ff5700 1 -- 172.21.15.39:0/1861153300 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5641e1ae96b0 msgr2=0x5641e1c9dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:46:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.196+0000 7f6081ff5700 1 --2- 172.21.15.39:0/1861153300 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5641e1ae96b0 0x5641e1c9dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.196+0000 7f6081ff5700 1 -- 172.21.15.39:0/1861153300 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5641e1c8dc20 con 0x5641e1c912a0 2023-07-21T18:46:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.196+0000 7f60827f6700 1 --2- 172.21.15.39:0/1861153300 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5641e1ae96b0 0x5641e1c9dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:46:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.196+0000 7f6081ff5700 1 --2- 172.21.15.39:0/1861153300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5641e1c912a0 0x5641e1c974f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f607800c5b0 tx=0x7f6078010c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c174943c1390e445 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:46:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.196+0000 7f6070ff9700 1 -- 172.21.15.39:0/1861153300 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60780a2690 con 0x5641e1c912a0 2023-07-21T18:46:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.196+0000 7f6070ff9700 1 -- 172.21.15.39:0/1861153300 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60780c4cb0 con 0x5641e1c912a0 2023-07-21T18:46:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.196+0000 7f6070ff9700 1 -- 172.21.15.39:0/1861153300 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60780a2200 con 0x5641e1c912a0 2023-07-21T18:46:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.197+0000 7f60903dfb80 1 -- 172.21.15.39:0/1861153300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5641e1c912a0 msgr2=0x5641e1c974f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:46:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.197+0000 7f60903dfb80 1 --2- 172.21.15.39:0/1861153300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5641e1c912a0 0x5641e1c974f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f607800c5b0 tx=0x7f6078010c80 comp rx=0 tx=0).stop 2023-07-21T18:46:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.197+0000 7f60903dfb80 1 -- 172.21.15.39:0/1861153300 shutdown_connections 2023-07-21T18:46:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.197+0000 7f60903dfb80 1 --2- 172.21.15.39:0/1861153300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5641e1c912a0 0x5641e1c974f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.197+0000 7f60903dfb80 1 --2- 172.21.15.39:0/1861153300 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5641e1ae96b0 0x5641e1c9dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.197+0000 7f60903dfb80 1 --2- 172.21.15.39:0/1861153300 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e1c82be0 0x5641e1c92350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.197+0000 7f60903dfb80 1 -- 172.21.15.39:0/1861153300 >> 172.21.15.39:0/1861153300 conn(0x5641e1c90ed0 msgr2=0x5641e1c98210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:46:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.197+0000 7f60903dfb80 1 -- 172.21.15.39:0/1861153300 shutdown_connections 2023-07-21T18:46:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.197+0000 7f60903dfb80 1 -- 172.21.15.39:0/1861153300 wait complete. 2023-07-21T18:46:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.197+0000 7f60903dfb80 1 -- start start 2023-07-21T18:46:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.197+0000 7f60903dfb80 1 Processor -- start 2023-07-21T18:46:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.197+0000 7f60903dfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5641e1ae96b0 0x5641e1cf3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.198+0000 7f60903dfb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5641e1c82be0 0x5641e1cf4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.198+0000 7f60903dfb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e1c912a0 0x5641e1cf2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.198+0000 7f60903dfb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641e1cf30e0 con 0x5641e1c82be0 2023-07-21T18:46:50.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.198+0000 7f60903dfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641e1cf3250 con 0x5641e1ae96b0 2023-07-21T18:46:50.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.198+0000 7f60903dfb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641e1cf3410 con 0x5641e1c912a0 2023-07-21T18:46:50.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.198+0000 7f60903dfb80 10 client.0 _ll_register_callbacks cb 0x7f6078099c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:46:50.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.198+0000 7f60817f4700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5641e1c82be0 0x5641e1cf4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:50.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.198+0000 7f60817f4700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5641e1c82be0 0x5641e1cf4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59770/0 (socket says 172.21.15.39:59770) 2023-07-21T18:46:50.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.198+0000 7f60817f4700 1 -- 172.21.15.39:0/1092585925 learned_addr learned my addr 172.21.15.39:0/1092585925 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:46:50.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.198+0000 7f6081ff5700 1 --2- 172.21.15.39:0/1092585925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5641e1ae96b0 0x5641e1cf3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:50.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.198+0000 7f60827f6700 1 --2- 172.21.15.39:0/1092585925 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e1c912a0 0x5641e1cf2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:50.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.198+0000 7f60817f4700 1 -- 172.21.15.39:0/1092585925 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e1c912a0 msgr2=0x5641e1cf2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:46:50.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.198+0000 7f60817f4700 1 --2- 172.21.15.39:0/1092585925 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e1c912a0 0x5641e1cf2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:50.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.198+0000 7f60817f4700 1 -- 172.21.15.39:0/1092585925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5641e1ae96b0 msgr2=0x5641e1cf3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:46:50.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.198+0000 7f60817f4700 1 --2- 172.21.15.39:0/1092585925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5641e1ae96b0 0x5641e1cf3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:50.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.198+0000 7f60817f4700 1 -- 172.21.15.39:0/1092585925 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f607c10c200 con 0x5641e1c82be0 2023-07-21T18:46:50.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.198+0000 7f60817f4700 1 --2- 172.21.15.39:0/1092585925 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5641e1c82be0 0x5641e1cf4320 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f606c027f10 tx=0x7f606c005c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54a78901b38afe03 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:46:50.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.198+0000 7f60657fa700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:46:50.221 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.198+0000 7f60657fa700 1 -- 172.21.15.39:0/1092585925 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606c02a040 con 0x5641e1c82be0 2023-07-21T18:46:50.221 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.198+0000 7f60657fa700 1 -- 172.21.15.39:0/1092585925 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f606c003a90 con 0x5641e1c82be0 2023-07-21T18:46:50.221 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.198+0000 7f60903dfb80 1 -- 172.21.15.39:0/1092585925 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f606c003e70 con 0x5641e1c82be0 2023-07-21T18:46:50.221 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.198+0000 7f60657fa700 1 -- 172.21.15.39:0/1092585925 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f606c034b20 con 0x5641e1c82be0 2023-07-21T18:46:50.221 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.199+0000 7f60903dfb80 10 client.16521 fetch_fsmap learned FSMap version 44 2023-07-21T18:46:50.221 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.199+0000 7f60903dfb80 1 -- 172.21.15.39:0/1092585925 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x5641e19f5c00 con 0x5641e1c82be0 2023-07-21T18:46:50.221 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.199+0000 7f60657fa700 1 -- 172.21.15.39:0/1092585925 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f606c003e70 con 0x5641e1c82be0 2023-07-21T18:46:50.221 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.199+0000 7f60657fa700 1 -- 172.21.15.39:0/1092585925 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 44) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f606c024020 con 0x5641e1c82be0 2023-07-21T18:46:50.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.199+0000 7f60903dfb80 10 client.16521 fetch_fsmap finished waiting for FSMap version 44 2023-07-21T18:46:50.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.199+0000 7f60903dfb80 10 client.16521 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:46:50.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.199+0000 7f60903dfb80 20 client.16521 populate_metadata read hostname 'smithi039' 2023-07-21T18:46:50.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.199+0000 7f60903dfb80 1 --2- 172.21.15.39:0/1092585925 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5641e1c8d580 0x5641e1cfde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:46:50.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.199+0000 7f60903dfb80 4 client.16521 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:46:50.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.199+0000 7f60903dfb80 1 -- 172.21.15.39:0/1092585925 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5641e1b99ff0 con 0x5641e1c8d580 2023-07-21T18:46:50.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.199+0000 7f6081ff5700 1 --2- 172.21.15.39:0/1092585925 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5641e1c8d580 0x5641e1cfde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:46:50.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.199+0000 7f60657fa700 10 client.16521 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:46:50.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.199+0000 7f6081ff5700 1 --2- 172.21.15.39:0/1092585925 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5641e1c8d580 0x5641e1cfde50 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a93509510d1bdf2b server_cookie=ca8ed6df254199b9 in_seq=0 out_seq=0 2023-07-21T18:46:50.257 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.241+0000 7f60657fa700 1 -- 172.21.15.39:0/1092585925 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5641e1b99ff0 con 0x5641e1c8d580 2023-07-21T18:46:50.257 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.241+0000 7f60657fa700 10 client.16521 handle_command_reply: tid=0 2023-07-21T18:46:50.257 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.241+0000 7f60903dfb80 1 client.16521 shutdown 2023-07-21T18:46:50.258 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.241+0000 7f60903dfb80 1 -- 172.21.15.39:0/1092585925 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5641e1c82be0 msgr2=0x5641e1cf4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:46:50.258 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.241+0000 7f60903dfb80 1 --2- 172.21.15.39:0/1092585925 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5641e1c82be0 0x5641e1cf4320 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f606c027f10 tx=0x7f606c005c80 comp rx=0 tx=0).stop 2023-07-21T18:46:50.258 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.241+0000 7f60903dfb80 1 -- 172.21.15.39:0/1092585925 shutdown_connections 2023-07-21T18:46:50.258 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.241+0000 7f60903dfb80 1 --2- 172.21.15.39:0/1092585925 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5641e1c8d580 0x5641e1cfde50 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:50.258 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.241+0000 7f60903dfb80 1 --2- 172.21.15.39:0/1092585925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5641e1ae96b0 0x5641e1cf3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:50.258 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.241+0000 7f60903dfb80 1 --2- 172.21.15.39:0/1092585925 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e1c912a0 0x5641e1cf2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:50.258 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.241+0000 7f60903dfb80 1 --2- 172.21.15.39:0/1092585925 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5641e1c82be0 0x5641e1cf4320 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:50.259 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.241+0000 7f60903dfb80 1 -- 172.21.15.39:0/1092585925 >> 172.21.15.39:0/1092585925 conn(0x5641e1c90ed0 msgr2=0x5641e1c98aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:46:50.259 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.241+0000 7f60657fa700 0 client.16521 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:46:50.259 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.241+0000 7f60903dfb80 1 -- 172.21.15.39:0/1092585925 shutdown_connections 2023-07-21T18:46:50.259 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.241+0000 7f60903dfb80 1 -- 172.21.15.39:0/1092585925 wait complete. 2023-07-21T18:46:50.259 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.242+0000 7f60903dfb80 20 client.16521 trim_cache size 0 max 16384 2023-07-21T18:46:50.259 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:46:50.260 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:46:50.260 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:46:50.260 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:46:50.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.242+0000 7f6084259700 1 -- 172.21.15.39:0/3566897409 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f60680a1340 msgr2=0x7f60680a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:46:50.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.242+0000 7f6084259700 1 --2- 172.21.15.39:0/3566897409 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f60680a1340 0x7f60680a37f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f607c069340 tx=0x7f606c009260 comp rx=0 tx=0).stop 2023-07-21T18:46:50.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.242+0000 7f6084259700 1 -- 172.21.15.39:0/3566897409 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f607c0feac0 msgr2=0x7f607c194ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:46:50.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.242+0000 7f6084259700 1 --2- 172.21.15.39:0/3566897409 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f607c0feac0 0x7f607c194ab0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f607800ea50 tx=0x7f607800ed60 comp rx=0 tx=0).stop 2023-07-21T18:46:50.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.242+0000 7f6084259700 1 -- 172.21.15.39:0/3566897409 shutdown_connections 2023-07-21T18:46:50.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.242+0000 7f6084259700 1 --2- 172.21.15.39:0/3566897409 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f60680a1340 0x7f60680a37f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:50.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.242+0000 7f6084259700 1 --2- 172.21.15.39:0/3566897409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f607c104ad0 0x7f607c1a3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:50.262 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.242+0000 7f6084259700 1 --2- 172.21.15.39:0/3566897409 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f607c0feac0 0x7f607c194ab0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:50.262 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.242+0000 7f6084259700 1 --2- 172.21.15.39:0/3566897409 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f607c0ff470 0x7f607c194ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:46:50.262 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.242+0000 7f6084259700 1 -- 172.21.15.39:0/3566897409 >> 172.21.15.39:0/3566897409 conn(0x7f607c075bd0 msgr2=0x7f607c0fcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:46:50.262 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.242+0000 7f6084259700 1 -- 172.21.15.39:0/3566897409 shutdown_connections 2023-07-21T18:46:50.262 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:46:50.242+0000 7f6084259700 1 -- 172.21.15.39:0/3566897409 wait complete. 2023-07-21T18:46:50.271 INFO:tasks.workunit.client.0.smithi039.stdout: 1 285292 5.79 MB/sec execute 123 sec latency 26.622 ms 2023-07-21T18:46:50.382 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:46:50 smithi155 ceph-mon[97934]: pgmap v156: 97 pgs: 97 active+clean; 107 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 237 op/s 2023-07-21T18:46:50.431 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:46:50 smithi039 ceph-mon[96489]: pgmap v156: 97 pgs: 97 active+clean; 107 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 237 op/s 2023-07-21T18:46:50.613 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:46:50 smithi193 ceph-mon[98204]: pgmap v156: 97 pgs: 97 active+clean; 107 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 237 op/s 2023-07-21T18:46:51.271 INFO:tasks.workunit.client.0.smithi039.stdout: 1 286971 5.81 MB/sec execute 124 sec latency 16.907 ms 2023-07-21T18:46:52.271 INFO:tasks.workunit.client.0.smithi039.stdout: 1 288159 5.82 MB/sec execute 125 sec latency 34.368 ms 2023-07-21T18:46:52.431 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:46:52 smithi039 ceph-mon[96489]: pgmap v157: 97 pgs: 97 active+clean; 108 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.7 MiB/s wr, 361 op/s 2023-07-21T18:46:52.613 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:46:52 smithi193 ceph-mon[98204]: pgmap v157: 97 pgs: 97 active+clean; 108 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.7 MiB/s wr, 361 op/s 2023-07-21T18:46:52.632 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:46:52 smithi155 ceph-mon[97934]: pgmap v157: 97 pgs: 97 active+clean; 108 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.7 MiB/s wr, 361 op/s 2023-07-21T18:46:53.271 INFO:tasks.workunit.client.0.smithi039.stdout: 1 289723 5.82 MB/sec execute 126 sec latency 50.335 ms 2023-07-21T18:46:54.271 INFO:tasks.workunit.client.0.smithi039.stdout: 1 291251 5.87 MB/sec execute 127 sec latency 22.964 ms 2023-07-21T18:46:54.431 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:46:54 smithi039 ceph-mon[96489]: pgmap v158: 97 pgs: 97 active+clean; 108 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 2.4 MiB/s wr, 233 op/s 2023-07-21T18:46:54.431 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:46:54 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:46:54.431 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:46:54 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:46:54.613 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:46:54 smithi193 ceph-mon[98204]: pgmap v158: 97 pgs: 97 active+clean; 108 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 2.4 MiB/s wr, 233 op/s 2023-07-21T18:46:54.613 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:46:54 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:46:54.613 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:46:54 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:46:54.632 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:46:54 smithi155 ceph-mon[97934]: pgmap v158: 97 pgs: 97 active+clean; 108 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 2.4 MiB/s wr, 233 op/s 2023-07-21T18:46:54.632 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:46:54 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:46:54.632 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:46:54 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:46:55.271 INFO:tasks.workunit.client.0.smithi039.stdout: 1 292433 5.87 MB/sec execute 128 sec latency 25.835 ms 2023-07-21T18:46:55.431 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:46:55 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:46:55.431 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:46:55 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:46:55.431 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:46:55 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:46:55.613 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:46:55 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:46:55.613 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:46:55 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:46:55.613 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:46:55 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:46:55.632 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:46:55 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:46:55.632 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:46:55 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:46:55.632 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:46:55 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:46:56.271 INFO:tasks.workunit.client.0.smithi039.stdout: 1 294362 5.92 MB/sec execute 129 sec latency 17.164 ms 2023-07-21T18:46:56.430 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:46:56 smithi039 ceph-mon[96489]: pgmap v159: 97 pgs: 97 active+clean; 108 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 2.4 MiB/s wr, 233 op/s 2023-07-21T18:46:56.613 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:46:56 smithi193 ceph-mon[98204]: pgmap v159: 97 pgs: 97 active+clean; 108 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 2.4 MiB/s wr, 233 op/s 2023-07-21T18:46:56.632 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:46:56 smithi155 ceph-mon[97934]: pgmap v159: 97 pgs: 97 active+clean; 108 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 2.4 MiB/s wr, 233 op/s 2023-07-21T18:46:57.271 INFO:tasks.workunit.client.0.smithi039.stdout: 1 295493 5.90 MB/sec execute 130 sec latency 50.555 ms 2023-07-21T18:46:58.271 INFO:tasks.workunit.client.0.smithi039.stdout: 1 295493 5.85 MB/sec execute 131 sec latency 1050.631 ms 2023-07-21T18:46:58.431 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:46:58 smithi039 ceph-mon[96489]: pgmap v160: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 336 op/s 2023-07-21T18:46:58.431 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:46:58 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:46:58.613 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:46:58 smithi193 ceph-mon[98204]: pgmap v160: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 336 op/s 2023-07-21T18:46:58.613 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:46:58 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:46:58.632 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:46:58 smithi155 ceph-mon[97934]: pgmap v160: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 336 op/s 2023-07-21T18:46:58.632 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:46:58 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:46:59.271 INFO:tasks.workunit.client.0.smithi039.stdout: 1 295493 5.81 MB/sec execute 132 sec latency 2050.722 ms 2023-07-21T18:46:59.431 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:46:59 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:46:59.613 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:46:59 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:46:59.632 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:46:59 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:47:00.271 INFO:tasks.workunit.client.0.smithi039.stdout: 1 296982 5.81 MB/sec execute 133 sec latency 2294.764 ms 2023-07-21T18:47:00.431 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:47:00 smithi039 ceph-mon[96489]: pgmap v161: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 987 KiB/s rd, 2.4 MiB/s wr, 226 op/s 2023-07-21T18:47:00.613 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:47:00 smithi193 ceph-mon[98204]: pgmap v161: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 987 KiB/s rd, 2.4 MiB/s wr, 226 op/s 2023-07-21T18:47:00.632 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:47:00 smithi155 ceph-mon[97934]: pgmap v161: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 987 KiB/s rd, 2.4 MiB/s wr, 226 op/s 2023-07-21T18:47:01.271 INFO:tasks.workunit.client.0.smithi039.stdout: 1 298261 5.85 MB/sec execute 134 sec latency 23.736 ms 2023-07-21T18:47:02.271 INFO:tasks.workunit.client.0.smithi039.stdout: 1 299482 5.84 MB/sec execute 135 sec latency 18.873 ms 2023-07-21T18:47:02.430 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:47:02 smithi039 ceph-mon[96489]: pgmap v162: 97 pgs: 97 active+clean; 122 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.3 MiB/s wr, 309 op/s 2023-07-21T18:47:02.613 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:47:02 smithi193 ceph-mon[98204]: pgmap v162: 97 pgs: 97 active+clean; 122 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.3 MiB/s wr, 309 op/s 2023-07-21T18:47:02.632 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:47:02 smithi155 ceph-mon[97934]: pgmap v162: 97 pgs: 97 active+clean; 122 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.3 MiB/s wr, 309 op/s 2023-07-21T18:47:03.272 INFO:tasks.workunit.client.0.smithi039.stdout: 1 301148 5.86 MB/sec execute 136 sec latency 16.343 ms 2023-07-21T18:47:03.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.364+0000 7f62b230d700 1 -- 172.21.15.39:0/1569296525 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62ac0fe3f0 msgr2=0x7f62ac0fe7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:47:03.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.364+0000 7f62b230d700 1 --2- 172.21.15.39:0/1569296525 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62ac0fe3f0 0x7f62ac0fe7c0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f629c0099c0 tx=0x7f629c009cd0 comp rx=0 tx=0).stop 2023-07-21T18:47:03.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.364+0000 7f62b230d700 1 -- 172.21.15.39:0/1569296525 shutdown_connections 2023-07-21T18:47:03.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.364+0000 7f62b230d700 1 --2- 172.21.15.39:0/1569296525 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62ac103430 0x7f62ac10b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:03.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.364+0000 7f62b230d700 1 --2- 172.21.15.39:0/1569296525 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62ac0fe3f0 0x7f62ac0fe7c0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:03.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.364+0000 7f62b230d700 1 --2- 172.21.15.39:0/1569296525 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62ac0fed90 0x7f62ac102d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:03.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.364+0000 7f62b230d700 1 -- 172.21.15.39:0/1569296525 >> 172.21.15.39:0/1569296525 conn(0x7f62ac075f90 msgr2=0x7f62ac076390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:47:03.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.364+0000 7f62b230d700 1 -- 172.21.15.39:0/1569296525 shutdown_connections 2023-07-21T18:47:03.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.364+0000 7f62b230d700 1 -- 172.21.15.39:0/1569296525 wait complete. 2023-07-21T18:47:03.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.365+0000 7f62b230d700 1 Processor -- start 2023-07-21T18:47:03.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.365+0000 7f62b230d700 1 -- start start 2023-07-21T18:47:03.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.366+0000 7f62b230d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62ac0fe3f0 0x7f62ac196c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:03.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.366+0000 7f62b230d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62ac0fed90 0x7f62ac197140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:03.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.366+0000 7f62b230d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62ac103430 0x7f62ac1a1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:03.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.366+0000 7f62b230d700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62ac1a2070 con 0x7f62ac0fe3f0 2023-07-21T18:47:03.385 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.366+0000 7f62b230d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62ac1a21e0 con 0x7f62ac0fed90 2023-07-21T18:47:03.385 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.366+0000 7f62b230d700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62ac1a23a0 con 0x7f62ac103430 2023-07-21T18:47:03.385 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.366+0000 7f62abfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62ac0fe3f0 0x7f62ac196c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:03.385 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.366+0000 7f62ab7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62ac0fed90 0x7f62ac197140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:03.385 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.366+0000 7f62b08aa700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62ac103430 0x7f62ac1a1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:03.385 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.366+0000 7f62ab7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62ac0fed90 0x7f62ac197140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:44148/0 (socket says 172.21.15.39:44148) 2023-07-21T18:47:03.385 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.366+0000 7f62ab7fe700 1 -- 172.21.15.39:0/232279840 learned_addr learned my addr 172.21.15.39:0/232279840 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:47:03.385 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.366+0000 7f62ab7fe700 1 -- 172.21.15.39:0/232279840 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62ac103430 msgr2=0x7f62ac1a1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:47:03.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.366+0000 7f62ab7fe700 1 --2- 172.21.15.39:0/232279840 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62ac103430 0x7f62ac1a1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:03.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.366+0000 7f62ab7fe700 1 -- 172.21.15.39:0/232279840 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62ac0fe3f0 msgr2=0x7f62ac196c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:47:03.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.366+0000 7f62ab7fe700 1 --2- 172.21.15.39:0/232279840 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62ac0fe3f0 0x7f62ac196c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:03.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.366+0000 7f62ab7fe700 1 -- 172.21.15.39:0/232279840 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f629c0095f0 con 0x7f62ac0fed90 2023-07-21T18:47:03.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.366+0000 7f62abfff700 1 --2- 172.21.15.39:0/232279840 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62ac0fe3f0 0x7f62ac196c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:47:03.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.366+0000 7f62b08aa700 1 --2- 172.21.15.39:0/232279840 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62ac103430 0x7f62ac1a1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:47:03.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.366+0000 7f62ab7fe700 1 --2- 172.21.15.39:0/232279840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62ac0fed90 0x7f62ac197140 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f62a000c9b0 tx=0x7f62a000ccc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:47:03.387 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.366+0000 7f62a97fa700 1 -- 172.21.15.39:0/232279840 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a0007aa0 con 0x7f62ac0fed90 2023-07-21T18:47:03.387 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.367+0000 7f62b230d700 1 -- 172.21.15.39:0/232279840 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62ac1a2680 con 0x7f62ac0fed90 2023-07-21T18:47:03.387 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.367+0000 7f62a97fa700 1 -- 172.21.15.39:0/232279840 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f62a000ce50 con 0x7f62ac0fed90 2023-07-21T18:47:03.387 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.367+0000 7f62a97fa700 1 -- 172.21.15.39:0/232279840 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a0018420 con 0x7f62ac0fed90 2023-07-21T18:47:03.387 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.367+0000 7f62b230d700 1 -- 172.21.15.39:0/232279840 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62ac1a2b90 con 0x7f62ac0fed90 2023-07-21T18:47:03.389 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.369+0000 7f62a97fa700 1 -- 172.21.15.39:0/232279840 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f62a000f450 con 0x7f62ac0fed90 2023-07-21T18:47:03.389 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.369+0000 7f62a97fa700 1 --2- 172.21.15.39:0/232279840 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f62940a12c0 0x7f62940a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:03.390 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.370+0000 7f62abfff700 1 --2- 172.21.15.39:0/232279840 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f62940a12c0 0x7f62940a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:03.390 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.370+0000 7f62a97fa700 1 -- 172.21.15.39:0/232279840 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f62a00ccb50 con 0x7f62ac0fed90 2023-07-21T18:47:03.390 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.370+0000 7f62abfff700 1 --2- 172.21.15.39:0/232279840 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f62940a12c0 0x7f62940a3770 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f629c009990 tx=0x7f629c005fb0 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:47:03.390 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.371+0000 7f62be493b80 1 Processor -- start 2023-07-21T18:47:03.390 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.371+0000 7f62be493b80 1 -- start start 2023-07-21T18:47:03.390 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.371+0000 7f62be493b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5633b8fcd6b0 0x5633b9176d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:03.390 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.371+0000 7f62be493b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5633b91772d0 0x5633b9171040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:03.391 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.371+0000 7f62be493b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5633b9171580 0x5633b91719d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:03.391 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.371+0000 7f62be493b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633b9076c50 con 0x5633b9171580 2023-07-21T18:47:03.391 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.371+0000 7f62be493b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633b90c99e0 con 0x5633b8fcd6b0 2023-07-21T18:47:03.391 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.371+0000 7f62be493b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633b90c2e70 con 0x5633b91772d0 2023-07-21T18:47:03.391 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.371+0000 7f62b08aa700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5633b9171580 0x5633b91719d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:03.391 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.371+0000 7f62abfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5633b8fcd6b0 0x5633b9176d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:03.392 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.371+0000 7f62b08aa700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5633b9171580 0x5633b91719d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59794/0 (socket says 172.21.15.39:59794) 2023-07-21T18:47:03.392 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.371+0000 7f62b08aa700 1 -- 172.21.15.39:0/1099596112 learned_addr learned my addr 172.21.15.39:0/1099596112 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:47:03.392 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.371+0000 7f62ab7fe700 1 --2- 172.21.15.39:0/1099596112 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5633b91772d0 0x5633b9171040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:03.392 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.372+0000 7f62abfff700 1 -- 172.21.15.39:0/1099596112 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5633b91772d0 msgr2=0x5633b9171040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:47:03.392 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.372+0000 7f62abfff700 1 --2- 172.21.15.39:0/1099596112 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5633b91772d0 0x5633b9171040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:03.392 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.372+0000 7f62abfff700 1 -- 172.21.15.39:0/1099596112 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5633b9171580 msgr2=0x5633b91719d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:47:03.392 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.372+0000 7f62abfff700 1 --2- 172.21.15.39:0/1099596112 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5633b9171580 0x5633b91719d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:03.392 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.372+0000 7f62abfff700 1 -- 172.21.15.39:0/1099596112 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633b8ef0d80 con 0x5633b8fcd6b0 2023-07-21T18:47:03.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.372+0000 7f62ab7fe700 1 --2- 172.21.15.39:0/1099596112 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5633b91772d0 0x5633b9171040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:47:03.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.372+0000 7f62abfff700 1 --2- 172.21.15.39:0/1099596112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5633b8fcd6b0 0x5633b9176d90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f629c0098e0 tx=0x7f629c01f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db6ac4e78fa2d2f9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:47:03.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.372+0000 7f62927fc700 1 -- 172.21.15.39:0/1099596112 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629c029070 con 0x5633b8fcd6b0 2023-07-21T18:47:03.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.372+0000 7f62927fc700 1 -- 172.21.15.39:0/1099596112 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f629c01e040 con 0x5633b8fcd6b0 2023-07-21T18:47:03.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.372+0000 7f62927fc700 1 -- 172.21.15.39:0/1099596112 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629c00fc60 con 0x5633b8fcd6b0 2023-07-21T18:47:03.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.373+0000 7f62be493b80 1 -- 172.21.15.39:0/1099596112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5633b8fcd6b0 msgr2=0x5633b9176d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:47:03.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.373+0000 7f62be493b80 1 --2- 172.21.15.39:0/1099596112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5633b8fcd6b0 0x5633b9176d90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f629c0098e0 tx=0x7f629c01f000 comp rx=0 tx=0).stop 2023-07-21T18:47:03.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.373+0000 7f62be493b80 1 -- 172.21.15.39:0/1099596112 shutdown_connections 2023-07-21T18:47:03.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.373+0000 7f62be493b80 1 --2- 172.21.15.39:0/1099596112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5633b8fcd6b0 0x5633b9176d90 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:03.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.373+0000 7f62be493b80 1 --2- 172.21.15.39:0/1099596112 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5633b9171580 0x5633b91719d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:03.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.373+0000 7f62be493b80 1 --2- 172.21.15.39:0/1099596112 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5633b91772d0 0x5633b9171040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:03.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.373+0000 7f62be493b80 1 -- 172.21.15.39:0/1099596112 >> 172.21.15.39:0/1099596112 conn(0x5633b9166be0 msgr2=0x5633b916bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:47:03.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.373+0000 7f62be493b80 1 -- 172.21.15.39:0/1099596112 shutdown_connections 2023-07-21T18:47:03.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.373+0000 7f62be493b80 1 -- 172.21.15.39:0/1099596112 wait complete. 2023-07-21T18:47:03.397 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.373+0000 7f62be493b80 1 -- start start 2023-07-21T18:47:03.398 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.374+0000 7f62be493b80 1 Processor -- start 2023-07-21T18:47:03.398 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.374+0000 7f62be493b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5633b8fcd6b0 0x5633b9180580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:03.398 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.374+0000 7f62be493b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5633b9171580 0x5633b9180ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:03.398 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.374+0000 7f62be493b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5633b91772d0 0x5633b917a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:03.398 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.374+0000 7f62be493b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633b9062a60 con 0x5633b8fcd6b0 2023-07-21T18:47:03.398 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.374+0000 7f62be493b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633b9062760 con 0x5633b91772d0 2023-07-21T18:47:03.398 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.374+0000 7f62be493b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633b8f468c0 con 0x5633b9171580 2023-07-21T18:47:03.399 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.374+0000 7f62be493b80 10 client.0 _ll_register_callbacks cb 0x5633b9060700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:47:03.399 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.374+0000 7f62ab7fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5633b9171580 0x5633b9180ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:03.399 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.374+0000 7f62abfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5633b8fcd6b0 0x5633b9180580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:03.399 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.374+0000 7f62ab7fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5633b9171580 0x5633b9180ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:50118/0 (socket says 172.21.15.39:50118) 2023-07-21T18:47:03.399 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.374+0000 7f62abfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5633b8fcd6b0 0x5633b9180580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59796/0 (socket says 172.21.15.39:59796) 2023-07-21T18:47:03.399 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.374+0000 7f62ab7fe700 1 -- 172.21.15.39:0/3557477509 learned_addr learned my addr 172.21.15.39:0/3557477509 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:47:03.399 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.375+0000 7f62b08aa700 1 --2- 172.21.15.39:0/3557477509 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5633b91772d0 0x5633b917a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:03.400 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.375+0000 7f62ab7fe700 1 -- 172.21.15.39:0/3557477509 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5633b91772d0 msgr2=0x5633b917a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:47:03.400 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.375+0000 7f62ab7fe700 1 --2- 172.21.15.39:0/3557477509 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5633b91772d0 0x5633b917a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:03.400 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.375+0000 7f62ab7fe700 1 -- 172.21.15.39:0/3557477509 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5633b8fcd6b0 msgr2=0x5633b9180580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:47:03.400 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.375+0000 7f62ab7fe700 1 --2- 172.21.15.39:0/3557477509 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5633b8fcd6b0 0x5633b9180580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:03.400 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.375+0000 7f62ab7fe700 1 -- 172.21.15.39:0/3557477509 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62ac1a2b90 con 0x5633b9171580 2023-07-21T18:47:03.400 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.375+0000 7f62ab7fe700 1 --2- 172.21.15.39:0/3557477509 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5633b9171580 0x5633b9180ac0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f62a00ca320 tx=0x7f62a001f430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2eeae1c4416d4214 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:47:03.400 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.375+0000 7f628b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2023-07-21T18:47:03.400 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.375+0000 7f628b7fe700 1 -- 172.21.15.39:0/3557477509 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a009a0a0 con 0x5633b9171580 2023-07-21T18:47:03.401 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.375+0000 7f628b7fe700 1 -- 172.21.15.39:0/3557477509 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f62a000f910 con 0x5633b9171580 2023-07-21T18:47:03.401 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.375+0000 7f62be493b80 1 -- 172.21.15.39:0/3557477509 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5633b901c490 con 0x5633b9171580 2023-07-21T18:47:03.401 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.375+0000 7f628b7fe700 1 -- 172.21.15.39:0/3557477509 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f62a00a2090 con 0x5633b9171580 2023-07-21T18:47:03.401 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.375+0000 7f62be493b80 10 client.26020 fetch_fsmap learned FSMap version 47 2023-07-21T18:47:03.401 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.375+0000 7f62be493b80 1 -- 172.21.15.39:0/3557477509 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x7f62ac10b840 con 0x5633b9171580 2023-07-21T18:47:03.401 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.376+0000 7f628b7fe700 1 -- 172.21.15.39:0/3557477509 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f62a00c8080 con 0x5633b9171580 2023-07-21T18:47:03.401 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.376+0000 7f628b7fe700 1 -- 172.21.15.39:0/3557477509 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 47) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f62a00a1c90 con 0x5633b9171580 2023-07-21T18:47:03.402 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.376+0000 7f62be493b80 10 client.26020 fetch_fsmap finished waiting for FSMap version 47 2023-07-21T18:47:03.402 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.376+0000 7f62be493b80 10 client.26020 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:47:03.402 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.376+0000 7f62be493b80 20 client.26020 populate_metadata read hostname 'smithi039' 2023-07-21T18:47:03.402 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.376+0000 7f62be493b80 1 --2- 172.21.15.39:0/3557477509 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5633b918a9d0 0x5633b918cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:03.402 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.376+0000 7f62be493b80 4 client.26020 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:47:03.402 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.376+0000 7f62be493b80 1 -- 172.21.15.39:0/3557477509 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5633b90bbe50 con 0x5633b918a9d0 2023-07-21T18:47:03.402 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.376+0000 7f62abfff700 1 --2- 172.21.15.39:0/3557477509 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5633b918a9d0 0x5633b918cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:03.402 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.376+0000 7f62abfff700 1 --2- 172.21.15.39:0/3557477509 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5633b918a9d0 0x5633b918cde0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=245e8d9b65cbe868 server_cookie=217f222dbd3dcf4f in_seq=0 out_seq=0 2023-07-21T18:47:03.403 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.376+0000 7f628b7fe700 10 client.26020 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:47:03.403 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.377+0000 7f628b7fe700 1 -- 172.21.15.39:0/3557477509 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5633b90bbe50 con 0x5633b918a9d0 2023-07-21T18:47:03.403 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.377+0000 7f628b7fe700 10 client.26020 handle_command_reply: tid=0 2023-07-21T18:47:03.403 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.377+0000 7f62be493b80 1 client.26020 shutdown 2023-07-21T18:47:03.403 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.377+0000 7f62be493b80 1 -- 172.21.15.39:0/3557477509 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5633b9171580 msgr2=0x5633b9180ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:47:03.403 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.377+0000 7f62be493b80 1 --2- 172.21.15.39:0/3557477509 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5633b9171580 0x5633b9180ac0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f62a00ca320 tx=0x7f62a001f430 comp rx=0 tx=0).stop 2023-07-21T18:47:03.403 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.377+0000 7f62be493b80 1 -- 172.21.15.39:0/3557477509 shutdown_connections 2023-07-21T18:47:03.404 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.378+0000 7f62be493b80 1 --2- 172.21.15.39:0/3557477509 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5633b918a9d0 0x5633b918cde0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:03.404 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.378+0000 7f62be493b80 1 --2- 172.21.15.39:0/3557477509 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5633b91772d0 0x5633b917a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:03.404 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.378+0000 7f628b7fe700 0 client.26020 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:47:03.404 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.378+0000 7f62be493b80 1 --2- 172.21.15.39:0/3557477509 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5633b8fcd6b0 0x5633b9180580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:03.404 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.378+0000 7f62be493b80 1 --2- 172.21.15.39:0/3557477509 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5633b9171580 0x5633b9180ac0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:03.404 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.378+0000 7f62be493b80 1 -- 172.21.15.39:0/3557477509 >> 172.21.15.39:0/3557477509 conn(0x5633b9166be0 msgr2=0x5633b916ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:47:03.404 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.378+0000 7f62be493b80 1 -- 172.21.15.39:0/3557477509 shutdown_connections 2023-07-21T18:47:03.404 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.378+0000 7f62be493b80 1 -- 172.21.15.39:0/3557477509 wait complete. 2023-07-21T18:47:03.405 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.378+0000 7f62be493b80 20 client.26020 trim_cache size 0 max 16384 2023-07-21T18:47:03.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.399+0000 7f62be493b80 1 Processor -- start 2023-07-21T18:47:03.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.399+0000 7f62be493b80 1 -- start start 2023-07-21T18:47:03.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.399+0000 7f62be493b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5633b8fcd6b0 0x5633b9171040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:03.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.399+0000 7f62be493b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5633b9171580 0x5633b91762f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:03.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.399+0000 7f62be493b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5633b91772d0 0x5633b9178140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:03.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.399+0000 7f62be493b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633b9177810 con 0x5633b91772d0 2023-07-21T18:47:03.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.399+0000 7f62be493b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633b9176a10 con 0x5633b9171580 2023-07-21T18:47:03.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.399+0000 7f62be493b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633b917b6b0 con 0x5633b8fcd6b0 2023-07-21T18:47:03.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.399+0000 7f62b08aa700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5633b91772d0 0x5633b9178140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:03.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.399+0000 7f62b08aa700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5633b91772d0 0x5633b9178140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59808/0 (socket says 172.21.15.39:59808) 2023-07-21T18:47:03.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.399+0000 7f62b08aa700 1 -- 172.21.15.39:0/1886174782 learned_addr learned my addr 172.21.15.39:0/1886174782 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:47:03.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.399+0000 7f62ab7fe700 1 --2- 172.21.15.39:0/1886174782 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5633b9171580 0x5633b91762f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:03.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.399+0000 7f62abfff700 1 --2- 172.21.15.39:0/1886174782 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5633b8fcd6b0 0x5633b9171040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:03.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.399+0000 7f62b08aa700 1 -- 172.21.15.39:0/1886174782 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5633b8fcd6b0 msgr2=0x5633b9171040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:47:03.418 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.399+0000 7f62b08aa700 1 --2- 172.21.15.39:0/1886174782 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5633b8fcd6b0 0x5633b9171040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:03.418 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.399+0000 7f62b08aa700 1 -- 172.21.15.39:0/1886174782 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5633b9171580 msgr2=0x5633b91762f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:47:03.418 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.399+0000 7f62b08aa700 1 --2- 172.21.15.39:0/1886174782 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5633b9171580 0x5633b91762f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:03.418 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.399+0000 7f62b08aa700 1 -- 172.21.15.39:0/1886174782 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633b9168420 con 0x5633b91772d0 2023-07-21T18:47:03.418 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.400+0000 7f62b08aa700 1 --2- 172.21.15.39:0/1886174782 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5633b91772d0 0x5633b9178140 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f62a4009460 tx=0x7f62a4009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cccbbe1975561b6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:47:03.421 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.400+0000 7f6292ffd700 1 -- 172.21.15.39:0/1886174782 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a4007ea0 con 0x5633b91772d0 2023-07-21T18:47:03.421 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.400+0000 7f6292ffd700 1 -- 172.21.15.39:0/1886174782 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f62a400f040 con 0x5633b91772d0 2023-07-21T18:47:03.421 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.400+0000 7f6292ffd700 1 -- 172.21.15.39:0/1886174782 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a401f710 con 0x5633b91772d0 2023-07-21T18:47:03.421 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.400+0000 7f62be493b80 1 -- 172.21.15.39:0/1886174782 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5633b91772d0 msgr2=0x5633b9178140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:47:03.421 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.400+0000 7f62be493b80 1 --2- 172.21.15.39:0/1886174782 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5633b91772d0 0x5633b9178140 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f62a4009460 tx=0x7f62a4009920 comp rx=0 tx=0).stop 2023-07-21T18:47:03.421 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.401+0000 7f62be493b80 1 -- 172.21.15.39:0/1886174782 shutdown_connections 2023-07-21T18:47:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.401+0000 7f62be493b80 1 --2- 172.21.15.39:0/1886174782 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5633b91772d0 0x5633b9178140 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.401+0000 7f62be493b80 1 --2- 172.21.15.39:0/1886174782 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5633b8fcd6b0 0x5633b9171040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.401+0000 7f62be493b80 1 --2- 172.21.15.39:0/1886174782 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5633b9171580 0x5633b91762f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.401+0000 7f62be493b80 1 -- 172.21.15.39:0/1886174782 >> 172.21.15.39:0/1886174782 conn(0x5633b9166be0 msgr2=0x5633b9177a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:47:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.401+0000 7f62be493b80 1 -- 172.21.15.39:0/1886174782 shutdown_connections 2023-07-21T18:47:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.401+0000 7f62be493b80 1 -- 172.21.15.39:0/1886174782 wait complete. 2023-07-21T18:47:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.401+0000 7f62be493b80 1 -- start start 2023-07-21T18:47:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.401+0000 7f62be493b80 1 Processor -- start 2023-07-21T18:47:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.401+0000 7f62be493b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5633b8fcd6b0 0x5633b91d2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.401+0000 7f62be493b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5633b9171580 0x5633b91d2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.401+0000 7f62be493b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5633b91772d0 0x5633b91d78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.401+0000 7f62be493b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633b91d7e20 con 0x5633b91772d0 2023-07-21T18:47:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.401+0000 7f62be493b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633b91d7f60 con 0x5633b8fcd6b0 2023-07-21T18:47:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.401+0000 7f62be493b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633b91d8120 con 0x5633b9171580 2023-07-21T18:47:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.402+0000 7f62be493b80 10 client.0 _ll_register_callbacks cb 0x7f62a0007cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:47:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.402+0000 7f62b08aa700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5633b91772d0 0x5633b91d78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.402+0000 7f62abfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5633b8fcd6b0 0x5633b91d2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.402+0000 7f62ab7fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5633b9171580 0x5633b91d2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.402+0000 7f62b08aa700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5633b91772d0 0x5633b91d78e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59814/0 (socket says 172.21.15.39:59814) 2023-07-21T18:47:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.402+0000 7f62b08aa700 1 -- 172.21.15.39:0/2710523525 learned_addr learned my addr 172.21.15.39:0/2710523525 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:47:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.402+0000 7f62b08aa700 1 -- 172.21.15.39:0/2710523525 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5633b9171580 msgr2=0x5633b91d2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:47:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.402+0000 7f62b08aa700 1 --2- 172.21.15.39:0/2710523525 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5633b9171580 0x5633b91d2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.402+0000 7f62b08aa700 1 -- 172.21.15.39:0/2710523525 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5633b8fcd6b0 msgr2=0x5633b91d2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:47:03.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.402+0000 7f62b08aa700 1 --2- 172.21.15.39:0/2710523525 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5633b8fcd6b0 0x5633b91d2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:03.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.402+0000 7f62b08aa700 1 -- 172.21.15.39:0/2710523525 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62a4009110 con 0x5633b91772d0 2023-07-21T18:47:03.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.402+0000 7f62abfff700 1 --2- 172.21.15.39:0/2710523525 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5633b8fcd6b0 0x5633b91d2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:47:03.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.402+0000 7f62b08aa700 1 --2- 172.21.15.39:0/2710523525 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5633b91772d0 0x5633b91d78e0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f62a400ee60 tx=0x7f62a400e7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd7b1ff306f408ea server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:47:03.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.402+0000 7f628b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:47:03.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.402+0000 7f628b7fe700 1 -- 172.21.15.39:0/2710523525 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a4007ea0 con 0x5633b91772d0 2023-07-21T18:47:03.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.402+0000 7f62be493b80 1 -- 172.21.15.39:0/2710523525 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f629c01e040 con 0x5633b91772d0 2023-07-21T18:47:03.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.402+0000 7f628b7fe700 1 -- 172.21.15.39:0/2710523525 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f62a4029660 con 0x5633b91772d0 2023-07-21T18:47:03.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.403+0000 7f628b7fe700 1 -- 172.21.15.39:0/2710523525 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f62a400ec80 con 0x5633b91772d0 2023-07-21T18:47:03.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.403+0000 7f62be493b80 10 client.16539 fetch_fsmap learned FSMap version 47 2023-07-21T18:47:03.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.403+0000 7f62be493b80 1 -- 172.21.15.39:0/2710523525 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x5633b8ed9c00 con 0x5633b91772d0 2023-07-21T18:47:03.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.403+0000 7f628b7fe700 1 -- 172.21.15.39:0/2710523525 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f629c01e040 con 0x5633b91772d0 2023-07-21T18:47:03.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.403+0000 7f628b7fe700 1 -- 172.21.15.39:0/2710523525 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 47) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f62a4017410 con 0x5633b91772d0 2023-07-21T18:47:03.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.403+0000 7f62be493b80 10 client.16539 fetch_fsmap finished waiting for FSMap version 47 2023-07-21T18:47:03.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.403+0000 7f62be493b80 10 client.16539 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:47:03.427 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.403+0000 7f62be493b80 20 client.16539 populate_metadata read hostname 'smithi039' 2023-07-21T18:47:03.427 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.403+0000 7f62be493b80 1 --2- 172.21.15.39:0/2710523525 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5633b91df8a0 0x5633b91e1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:03.427 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.403+0000 7f62be493b80 4 client.16539 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:47:03.427 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.403+0000 7f62be493b80 1 -- 172.21.15.39:0/2710523525 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5633b907dff0 con 0x5633b91df8a0 2023-07-21T18:47:03.427 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.403+0000 7f62abfff700 1 --2- 172.21.15.39:0/2710523525 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5633b91df8a0 0x5633b91e1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:03.427 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.404+0000 7f62abfff700 1 --2- 172.21.15.39:0/2710523525 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5633b91df8a0 0x5633b91e1cb0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c8753e4df0dc44b server_cookie=48482209ce4603fc in_seq=0 out_seq=0 2023-07-21T18:47:03.427 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.404+0000 7f628b7fe700 10 client.16539 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:47:03.456 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.439+0000 7f628b7fe700 1 -- 172.21.15.39:0/2710523525 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5633b907dff0 con 0x5633b91df8a0 2023-07-21T18:47:03.456 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.439+0000 7f628b7fe700 10 client.16539 handle_command_reply: tid=0 2023-07-21T18:47:03.456 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.439+0000 7f62be493b80 1 client.16539 shutdown 2023-07-21T18:47:03.457 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.440+0000 7f62be493b80 1 -- 172.21.15.39:0/2710523525 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5633b91772d0 msgr2=0x5633b91d78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:47:03.457 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.440+0000 7f62be493b80 1 --2- 172.21.15.39:0/2710523525 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5633b91772d0 0x5633b91d78e0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f62a400ee60 tx=0x7f62a400e7a0 comp rx=0 tx=0).stop 2023-07-21T18:47:03.457 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.440+0000 7f62be493b80 1 -- 172.21.15.39:0/2710523525 shutdown_connections 2023-07-21T18:47:03.457 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.440+0000 7f62be493b80 1 --2- 172.21.15.39:0/2710523525 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5633b91df8a0 0x5633b91e1cb0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:03.457 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.440+0000 7f62be493b80 1 --2- 172.21.15.39:0/2710523525 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5633b8fcd6b0 0x5633b91d2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:03.458 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.440+0000 7f62be493b80 1 --2- 172.21.15.39:0/2710523525 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5633b91772d0 0x5633b91d78e0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:03.458 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.440+0000 7f62be493b80 1 --2- 172.21.15.39:0/2710523525 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5633b9171580 0x5633b91d2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:03.458 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.440+0000 7f62be493b80 1 -- 172.21.15.39:0/2710523525 >> 172.21.15.39:0/2710523525 conn(0x5633b9166be0 msgr2=0x5633b91782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:47:03.458 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.440+0000 7f628b7fe700 0 client.16539 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:47:03.458 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.440+0000 7f62be493b80 1 -- 172.21.15.39:0/2710523525 shutdown_connections 2023-07-21T18:47:03.458 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.440+0000 7f62be493b80 1 -- 172.21.15.39:0/2710523525 wait complete. 2023-07-21T18:47:03.458 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.441+0000 7f62be493b80 20 client.16539 trim_cache size 0 max 16384 2023-07-21T18:47:03.459 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:47:03.459 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:47:03.459 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:47:03.459 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:47:03.460 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.441+0000 7f62b230d700 1 -- 172.21.15.39:0/232279840 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f62940a12c0 msgr2=0x7f62940a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:47:03.460 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.441+0000 7f62b230d700 1 --2- 172.21.15.39:0/232279840 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f62940a12c0 0x7f62940a3770 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f629c009990 tx=0x7f629c005fb0 comp rx=0 tx=0).stop 2023-07-21T18:47:03.460 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.441+0000 7f62b230d700 1 -- 172.21.15.39:0/232279840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62ac0fed90 msgr2=0x7f62ac197140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:47:03.460 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.441+0000 7f62b230d700 1 --2- 172.21.15.39:0/232279840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62ac0fed90 0x7f62ac197140 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f62a000c9b0 tx=0x7f62a000ccc0 comp rx=0 tx=0).stop 2023-07-21T18:47:03.460 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.441+0000 7f62b230d700 1 -- 172.21.15.39:0/232279840 shutdown_connections 2023-07-21T18:47:03.461 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.441+0000 7f62b230d700 1 --2- 172.21.15.39:0/232279840 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62ac103430 0x7f62ac1a1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:03.461 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.441+0000 7f62b230d700 1 --2- 172.21.15.39:0/232279840 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62ac0fe3f0 0x7f62ac196c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:03.461 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.441+0000 7f62b230d700 1 --2- 172.21.15.39:0/232279840 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f62940a12c0 0x7f62940a3770 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:03.461 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.441+0000 7f62b230d700 1 --2- 172.21.15.39:0/232279840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62ac0fed90 0x7f62ac197140 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:03.461 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.441+0000 7f62b230d700 1 -- 172.21.15.39:0/232279840 >> 172.21.15.39:0/232279840 conn(0x7f62ac075f90 msgr2=0x7f62ac0fbb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:47:03.461 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.441+0000 7f62b230d700 1 -- 172.21.15.39:0/232279840 shutdown_connections 2023-07-21T18:47:03.461 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:03.441+0000 7f62b230d700 1 -- 172.21.15.39:0/232279840 wait complete. 2023-07-21T18:47:04.272 INFO:tasks.workunit.client.0.smithi039.stdout: 1 302245 5.86 MB/sec execute 137 sec latency 39.822 ms 2023-07-21T18:47:04.430 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:47:04 smithi039 ceph-mon[96489]: pgmap v163: 97 pgs: 97 active+clean; 122 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 810 KiB/s rd, 2.0 MiB/s wr, 184 op/s 2023-07-21T18:47:04.613 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:47:04 smithi193 ceph-mon[98204]: pgmap v163: 97 pgs: 97 active+clean; 122 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 810 KiB/s rd, 2.0 MiB/s wr, 184 op/s 2023-07-21T18:47:04.632 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:47:04 smithi155 ceph-mon[97934]: pgmap v163: 97 pgs: 97 active+clean; 122 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 810 KiB/s rd, 2.0 MiB/s wr, 184 op/s 2023-07-21T18:47:05.272 INFO:tasks.workunit.client.0.smithi039.stdout: 1 304125 5.88 MB/sec execute 138 sec latency 37.659 ms 2023-07-21T18:47:06.272 INFO:tasks.workunit.client.0.smithi039.stdout: 1 305626 5.91 MB/sec execute 139 sec latency 22.684 ms 2023-07-21T18:47:06.613 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:47:06 smithi193 ceph-mon[98204]: pgmap v164: 97 pgs: 97 active+clean; 122 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 810 KiB/s rd, 2.0 MiB/s wr, 184 op/s 2023-07-21T18:47:06.631 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:47:06 smithi155 ceph-mon[97934]: pgmap v164: 97 pgs: 97 active+clean; 122 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 810 KiB/s rd, 2.0 MiB/s wr, 184 op/s 2023-07-21T18:47:06.680 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:47:06 smithi039 ceph-mon[96489]: pgmap v164: 97 pgs: 97 active+clean; 122 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 810 KiB/s rd, 2.0 MiB/s wr, 184 op/s 2023-07-21T18:47:07.272 INFO:tasks.workunit.client.0.smithi039.stdout: 1 306911 5.91 MB/sec execute 140 sec latency 18.169 ms 2023-07-21T18:47:08.272 INFO:tasks.workunit.client.0.smithi039.stdout: 1 308865 5.96 MB/sec execute 141 sec latency 12.592 ms 2023-07-21T18:47:08.613 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:47:08 smithi193 ceph-mon[98204]: pgmap v165: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 326 op/s 2023-07-21T18:47:08.632 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:47:08 smithi155 ceph-mon[97934]: pgmap v165: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 326 op/s 2023-07-21T18:47:08.680 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:47:08 smithi039 ceph-mon[96489]: pgmap v165: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 326 op/s 2023-07-21T18:47:09.272 INFO:tasks.workunit.client.0.smithi039.stdout: 1 310143 5.96 MB/sec execute 142 sec latency 20.694 ms 2023-07-21T18:47:10.273 INFO:tasks.workunit.client.0.smithi039.stdout: 1 312317 6.01 MB/sec execute 143 sec latency 24.159 ms 2023-07-21T18:47:10.613 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:47:10 smithi193 ceph-mon[98204]: pgmap v166: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 2.3 MiB/s wr, 224 op/s 2023-07-21T18:47:10.631 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:47:10 smithi155 ceph-mon[97934]: pgmap v166: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 2.3 MiB/s wr, 224 op/s 2023-07-21T18:47:10.680 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:47:10 smithi039 ceph-mon[96489]: pgmap v166: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 2.3 MiB/s wr, 224 op/s 2023-07-21T18:47:11.273 INFO:tasks.workunit.client.0.smithi039.stdout: 1 313643 6.01 MB/sec execute 144 sec latency 13.679 ms 2023-07-21T18:47:12.273 INFO:tasks.workunit.client.0.smithi039.stdout: 1 315817 6.06 MB/sec execute 145 sec latency 18.854 ms 2023-07-21T18:47:12.613 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:47:12 smithi193 ceph-mon[98204]: pgmap v167: 97 pgs: 97 active+clean; 118 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 331 op/s 2023-07-21T18:47:12.632 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:47:12 smithi155 ceph-mon[97934]: pgmap v167: 97 pgs: 97 active+clean; 118 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 331 op/s 2023-07-21T18:47:12.680 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:47:12 smithi039 ceph-mon[96489]: pgmap v167: 97 pgs: 97 active+clean; 118 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 331 op/s 2023-07-21T18:47:13.273 INFO:tasks.workunit.client.0.smithi039.stdout: 1 316911 6.04 MB/sec execute 146 sec latency 15.632 ms 2023-07-21T18:47:14.273 INFO:tasks.workunit.client.0.smithi039.stdout: 1 318713 6.07 MB/sec execute 147 sec latency 41.405 ms 2023-07-21T18:47:14.613 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:47:14 smithi193 ceph-mon[98204]: pgmap v168: 97 pgs: 97 active+clean; 118 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 248 op/s 2023-07-21T18:47:14.631 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:47:14 smithi155 ceph-mon[97934]: pgmap v168: 97 pgs: 97 active+clean; 118 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 248 op/s 2023-07-21T18:47:14.680 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:47:14 smithi039 ceph-mon[96489]: pgmap v168: 97 pgs: 97 active+clean; 118 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 248 op/s 2023-07-21T18:47:15.273 INFO:tasks.workunit.client.0.smithi039.stdout: 1 320049 6.08 MB/sec execute 148 sec latency 16.306 ms 2023-07-21T18:47:16.273 INFO:tasks.workunit.client.0.smithi039.stdout: 1 321689 6.08 MB/sec execute 149 sec latency 16.439 ms 2023-07-21T18:47:16.582 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.566+0000 7f1b5a177700 1 -- 172.21.15.39:0/714793008 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b54073160 msgr2=0x7f1b5410be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:47:16.582 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.566+0000 7f1b5a177700 1 --2- 172.21.15.39:0/714793008 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b54073160 0x7f1b5410be70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1b480098f0 tx=0x7f1b48009c00 comp rx=0 tx=0).stop 2023-07-21T18:47:16.582 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.567+0000 7f1b5a177700 1 -- 172.21.15.39:0/714793008 shutdown_connections 2023-07-21T18:47:16.582 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.567+0000 7f1b5a177700 1 --2- 172.21.15.39:0/714793008 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b5410c3b0 0x7f1b5410e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:16.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.567+0000 7f1b5a177700 1 --2- 172.21.15.39:0/714793008 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1b54072850 0x7f1b54072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:16.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.567+0000 7f1b5a177700 1 --2- 172.21.15.39:0/714793008 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b54073160 0x7f1b5410be70 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:16.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.567+0000 7f1b5a177700 1 -- 172.21.15.39:0/714793008 >> 172.21.15.39:0/714793008 conn(0x7f1b540fb650 msgr2=0x7f1b540fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:47:16.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.567+0000 7f1b5a177700 1 -- 172.21.15.39:0/714793008 shutdown_connections 2023-07-21T18:47:16.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.567+0000 7f1b5a177700 1 -- 172.21.15.39:0/714793008 wait complete. 2023-07-21T18:47:16.585 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.567+0000 7f1b5a177700 1 Processor -- start 2023-07-21T18:47:16.585 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.568+0000 7f1b5a177700 1 -- start start 2023-07-21T18:47:16.585 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.568+0000 7f1b5a177700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b54072850 0x7f1b54197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:16.585 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.568+0000 7f1b5a177700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b54073160 0x7f1b54198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:16.585 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.568+0000 7f1b5a177700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1b5410c3b0 0x7f1b5419ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:16.585 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.568+0000 7f1b5a177700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5419d0b0 con 0x7f1b54072850 2023-07-21T18:47:16.585 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.568+0000 7f1b5a177700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5419d220 con 0x7f1b54073160 2023-07-21T18:47:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.568+0000 7f1b5a177700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5419d3e0 con 0x7f1b5410c3b0 2023-07-21T18:47:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.568+0000 7f1b537fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b54072850 0x7f1b54197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.568+0000 7f1b52ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b54073160 0x7f1b54198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.568+0000 7f1b537fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b54072850 0x7f1b54197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59824/0 (socket says 172.21.15.39:59824) 2023-07-21T18:47:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.568+0000 7f1b537fe700 1 -- 172.21.15.39:0/670335010 learned_addr learned my addr 172.21.15.39:0/670335010 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:47:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.568+0000 7f1b53fff700 1 --2- 172.21.15.39:0/670335010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1b5410c3b0 0x7f1b5419ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.568+0000 7f1b537fe700 1 -- 172.21.15.39:0/670335010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1b5410c3b0 msgr2=0x7f1b5419ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:47:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.568+0000 7f1b537fe700 1 --2- 172.21.15.39:0/670335010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1b5410c3b0 0x7f1b5419ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.568+0000 7f1b537fe700 1 -- 172.21.15.39:0/670335010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b54073160 msgr2=0x7f1b54198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:47:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.568+0000 7f1b537fe700 1 --2- 172.21.15.39:0/670335010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b54073160 0x7f1b54198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.568+0000 7f1b537fe700 1 -- 172.21.15.39:0/670335010 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b48009520 con 0x7f1b54072850 2023-07-21T18:47:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.569+0000 7f1b52ffd700 1 --2- 172.21.15.39:0/670335010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b54073160 0x7f1b54198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:47:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.569+0000 7f1b537fe700 1 --2- 172.21.15.39:0/670335010 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b54072850 0x7f1b54197ec0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f1b4400e9c0 tx=0x7f1b4400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:47:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.569+0000 7f1b50ff9700 1 -- 172.21.15.39:0/670335010 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b44004a30 con 0x7f1b54072850 2023-07-21T18:47:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.569+0000 7f1b50ff9700 1 -- 172.21.15.39:0/670335010 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b4400cc80 con 0x7f1b54072850 2023-07-21T18:47:16.588 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.569+0000 7f1b5a177700 1 -- 172.21.15.39:0/670335010 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b5419d6c0 con 0x7f1b54072850 2023-07-21T18:47:16.588 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.569+0000 7f1b50ff9700 1 -- 172.21.15.39:0/670335010 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b440054e0 con 0x7f1b54072850 2023-07-21T18:47:16.588 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.569+0000 7f1b5a177700 1 -- 172.21.15.39:0/670335010 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b541a4f00 con 0x7f1b54072850 2023-07-21T18:47:16.590 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.571+0000 7f1b50ff9700 1 -- 172.21.15.39:0/670335010 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f1b44004b90 con 0x7f1b54072850 2023-07-21T18:47:16.590 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.572+0000 7f1b50ff9700 1 --2- 172.21.15.39:0/670335010 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f1b3c0a12c0 0x7f1b3c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:16.590 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.572+0000 7f1b50ff9700 1 -- 172.21.15.39:0/670335010 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f1b44014070 con 0x7f1b54072850 2023-07-21T18:47:16.590 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.572+0000 7f1b52ffd700 1 --2- 172.21.15.39:0/670335010 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f1b3c0a12c0 0x7f1b3c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.572+0000 7f1b52ffd700 1 --2- 172.21.15.39:0/670335010 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f1b3c0a12c0 0x7f1b3c0a3770 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1b48009fd0 tx=0x7f1b48019040 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:47:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.573+0000 7f1b662fdb80 1 Processor -- start 2023-07-21T18:47:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.573+0000 7f1b662fdb80 1 -- start start 2023-07-21T18:47:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.573+0000 7f1b662fdb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5626dae7f6b0 0x5626db028d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.573+0000 7f1b662fdb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5626db0292d0 0x5626db023040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.573+0000 7f1b662fdb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626db023580 0x5626db0239d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.573+0000 7f1b662fdb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626daf28c50 con 0x5626dae7f6b0 2023-07-21T18:47:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.573+0000 7f1b662fdb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626daf7b9e0 con 0x5626db023580 2023-07-21T18:47:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.573+0000 7f1b662fdb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626daf74e70 con 0x5626db0292d0 2023-07-21T18:47:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.573+0000 7f1b537fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5626dae7f6b0 0x5626db028d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.573+0000 7f1b537fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5626dae7f6b0 0x5626db028d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59832/0 (socket says 172.21.15.39:59832) 2023-07-21T18:47:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.573+0000 7f1b537fe700 1 -- 172.21.15.39:0/2807762856 learned_addr learned my addr 172.21.15.39:0/2807762856 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:47:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.573+0000 7f1b52ffd700 1 --2- 172.21.15.39:0/2807762856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5626db0292d0 0x5626db023040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.573+0000 7f1b53fff700 1 --2- 172.21.15.39:0/2807762856 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626db023580 0x5626db0239d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.573+0000 7f1b537fe700 1 -- 172.21.15.39:0/2807762856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5626db0292d0 msgr2=0x5626db023040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:47:16.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.573+0000 7f1b537fe700 1 --2- 172.21.15.39:0/2807762856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5626db0292d0 0x5626db023040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:16.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.573+0000 7f1b537fe700 1 -- 172.21.15.39:0/2807762856 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626db023580 msgr2=0x5626db0239d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:47:16.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.573+0000 7f1b537fe700 1 --2- 172.21.15.39:0/2807762856 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626db023580 0x5626db0239d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:16.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.574+0000 7f1b537fe700 1 -- 172.21.15.39:0/2807762856 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626dada2d80 con 0x5626dae7f6b0 2023-07-21T18:47:16.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.574+0000 7f1b537fe700 1 --2- 172.21.15.39:0/2807762856 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5626dae7f6b0 0x5626db028d90 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f1b44096930 tx=0x7f1b44099b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3b230042017b943 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:47:16.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.574+0000 7f1b39ffb700 1 -- 172.21.15.39:0/2807762856 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b4409def0 con 0x5626dae7f6b0 2023-07-21T18:47:16.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.574+0000 7f1b39ffb700 1 -- 172.21.15.39:0/2807762856 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b440a2310 con 0x5626dae7f6b0 2023-07-21T18:47:16.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.574+0000 7f1b39ffb700 1 -- 172.21.15.39:0/2807762856 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b440993b0 con 0x5626dae7f6b0 2023-07-21T18:47:16.594 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.574+0000 7f1b662fdb80 1 -- 172.21.15.39:0/2807762856 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5626dae7f6b0 msgr2=0x5626db028d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:47:16.594 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.574+0000 7f1b662fdb80 1 --2- 172.21.15.39:0/2807762856 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5626dae7f6b0 0x5626db028d90 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f1b44096930 tx=0x7f1b44099b10 comp rx=0 tx=0).stop 2023-07-21T18:47:16.594 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.575+0000 7f1b662fdb80 1 -- 172.21.15.39:0/2807762856 shutdown_connections 2023-07-21T18:47:16.594 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.575+0000 7f1b662fdb80 1 --2- 172.21.15.39:0/2807762856 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626db023580 0x5626db0239d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:16.594 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.575+0000 7f1b662fdb80 1 --2- 172.21.15.39:0/2807762856 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5626dae7f6b0 0x5626db028d90 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:16.594 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.575+0000 7f1b662fdb80 1 --2- 172.21.15.39:0/2807762856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5626db0292d0 0x5626db023040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:16.594 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.575+0000 7f1b662fdb80 1 -- 172.21.15.39:0/2807762856 >> 172.21.15.39:0/2807762856 conn(0x5626db018be0 msgr2=0x5626db01df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:47:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.575+0000 7f1b662fdb80 1 -- 172.21.15.39:0/2807762856 shutdown_connections 2023-07-21T18:47:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.575+0000 7f1b662fdb80 1 -- 172.21.15.39:0/2807762856 wait complete. 2023-07-21T18:47:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.575+0000 7f1b662fdb80 1 -- start start 2023-07-21T18:47:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.575+0000 7f1b662fdb80 1 Processor -- start 2023-07-21T18:47:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.575+0000 7f1b662fdb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5626dae7f6b0 0x5626db02f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.575+0000 7f1b662fdb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5626db023580 0x5626db02c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.575+0000 7f1b662fdb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626db0292d0 0x5626db02ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.576+0000 7f1b662fdb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626daf14a60 con 0x5626dae7f6b0 2023-07-21T18:47:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.576+0000 7f1b662fdb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626daf14760 con 0x5626db0292d0 2023-07-21T18:47:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.576+0000 7f1b662fdb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626dadf88c0 con 0x5626db023580 2023-07-21T18:47:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.576+0000 7f1b662fdb80 10 client.0 _ll_register_callbacks cb 0x5626daf12700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:47:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.576+0000 7f1b537fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5626dae7f6b0 0x5626db02f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.576+0000 7f1b537fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5626dae7f6b0 0x5626db02f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59838/0 (socket says 172.21.15.39:59838) 2023-07-21T18:47:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.576+0000 7f1b537fe700 1 -- 172.21.15.39:0/2027044263 learned_addr learned my addr 172.21.15.39:0/2027044263 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:47:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.576+0000 7f1b52ffd700 1 --2- 172.21.15.39:0/2027044263 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5626db023580 0x5626db02c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.576+0000 7f1b53fff700 1 --2- 172.21.15.39:0/2027044263 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626db0292d0 0x5626db02ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.576+0000 7f1b537fe700 1 -- 172.21.15.39:0/2027044263 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5626db023580 msgr2=0x5626db02c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:47:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.576+0000 7f1b537fe700 1 --2- 172.21.15.39:0/2027044263 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5626db023580 0x5626db02c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.576+0000 7f1b537fe700 1 -- 172.21.15.39:0/2027044263 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626db0292d0 msgr2=0x5626db02ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:47:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.576+0000 7f1b537fe700 1 --2- 172.21.15.39:0/2027044263 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626db0292d0 0x5626db02ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.576+0000 7f1b537fe700 1 -- 172.21.15.39:0/2027044263 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b541a4f00 con 0x5626dae7f6b0 2023-07-21T18:47:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.576+0000 7f1b537fe700 1 --2- 172.21.15.39:0/2027044263 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5626dae7f6b0 0x5626db02f5b0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f1b44005950 tx=0x7f1b4409a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfb2a5cb547de126 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:47:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.576+0000 7f1b32ffd700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:47:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.576+0000 7f1b32ffd700 1 -- 172.21.15.39:0/2027044263 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b440c8070 con 0x5626dae7f6b0 2023-07-21T18:47:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.576+0000 7f1b32ffd700 1 -- 172.21.15.39:0/2027044263 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f1b4409def0 con 0x5626dae7f6b0 2023-07-21T18:47:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.576+0000 7f1b662fdb80 1 -- 172.21.15.39:0/2027044263 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5626daece490 con 0x5626dae7f6b0 2023-07-21T18:47:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.576+0000 7f1b32ffd700 1 -- 172.21.15.39:0/2027044263 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b440ca2b0 con 0x5626dae7f6b0 2023-07-21T18:47:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.576+0000 7f1b662fdb80 10 client.16551 fetch_fsmap learned FSMap version 47 2023-07-21T18:47:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.576+0000 7f1b662fdb80 1 -- 172.21.15.39:0/2027044263 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x5626dad8bc00 con 0x5626dae7f6b0 2023-07-21T18:47:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.577+0000 7f1b32ffd700 1 -- 172.21.15.39:0/2027044263 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b44065070 con 0x5626dae7f6b0 2023-07-21T18:47:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.577+0000 7f1b32ffd700 1 -- 172.21.15.39:0/2027044263 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 47) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f1b440a1e80 con 0x5626dae7f6b0 2023-07-21T18:47:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.577+0000 7f1b662fdb80 10 client.16551 fetch_fsmap finished waiting for FSMap version 47 2023-07-21T18:47:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.577+0000 7f1b662fdb80 10 client.16551 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:47:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.577+0000 7f1b662fdb80 20 client.16551 populate_metadata read hostname 'smithi039' 2023-07-21T18:47:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.577+0000 7f1b662fdb80 1 --2- 172.21.15.39:0/2027044263 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5626db03c640 0x5626db03ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.577+0000 7f1b662fdb80 4 client.16551 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:47:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.577+0000 7f1b662fdb80 1 -- 172.21.15.39:0/2027044263 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5626daf6de50 con 0x5626db03c640 2023-07-21T18:47:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.577+0000 7f1b52ffd700 1 --2- 172.21.15.39:0/2027044263 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5626db03c640 0x5626db03ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.578+0000 7f1b32ffd700 10 client.16551 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:47:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.578+0000 7f1b52ffd700 1 --2- 172.21.15.39:0/2027044263 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5626db03c640 0x5626db03ea20 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92eac10f98a91f39 server_cookie=98dae7ffb4017856 in_seq=0 out_seq=0 2023-07-21T18:47:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.578+0000 7f1b32ffd700 1 -- 172.21.15.39:0/2027044263 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5626daf6de50 con 0x5626db03c640 2023-07-21T18:47:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.578+0000 7f1b32ffd700 10 client.16551 handle_command_reply: tid=0 2023-07-21T18:47:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.578+0000 7f1b662fdb80 1 client.16551 shutdown 2023-07-21T18:47:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.578+0000 7f1b662fdb80 1 -- 172.21.15.39:0/2027044263 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5626dae7f6b0 msgr2=0x5626db02f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:47:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.578+0000 7f1b662fdb80 1 --2- 172.21.15.39:0/2027044263 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5626dae7f6b0 0x5626db02f5b0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f1b44005950 tx=0x7f1b4409a040 comp rx=0 tx=0).stop 2023-07-21T18:47:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.578+0000 7f1b662fdb80 1 -- 172.21.15.39:0/2027044263 shutdown_connections 2023-07-21T18:47:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.578+0000 7f1b662fdb80 1 --2- 172.21.15.39:0/2027044263 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5626db03c640 0x5626db03ea20 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.579+0000 7f1b662fdb80 1 --2- 172.21.15.39:0/2027044263 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626db0292d0 0x5626db02ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.579+0000 7f1b662fdb80 1 --2- 172.21.15.39:0/2027044263 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5626dae7f6b0 0x5626db02f5b0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.579+0000 7f1b32ffd700 0 client.16551 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:47:16.604 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.579+0000 7f1b662fdb80 1 --2- 172.21.15.39:0/2027044263 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5626db023580 0x5626db02c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:16.604 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.579+0000 7f1b662fdb80 1 -- 172.21.15.39:0/2027044263 >> 172.21.15.39:0/2027044263 conn(0x5626db018be0 msgr2=0x5626db01d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:47:16.604 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.579+0000 7f1b662fdb80 1 -- 172.21.15.39:0/2027044263 shutdown_connections 2023-07-21T18:47:16.604 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.579+0000 7f1b662fdb80 1 -- 172.21.15.39:0/2027044263 wait complete. 2023-07-21T18:47:16.604 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.579+0000 7f1b662fdb80 20 client.16551 trim_cache size 0 max 16384 2023-07-21T18:47:16.609 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.594+0000 7f1b662fdb80 1 Processor -- start 2023-07-21T18:47:16.609 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.594+0000 7f1b662fdb80 1 -- start start 2023-07-21T18:47:16.609 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.594+0000 7f1b662fdb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5626dae7f6b0 0x5626db023040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:16.609 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.594+0000 7f1b662fdb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5626db023580 0x5626db028130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:16.609 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.594+0000 7f1b662fdb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626db0292d0 0x5626db02f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.594+0000 7f1b662fdb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626db029110 con 0x5626dae7f6b0 2023-07-21T18:47:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.594+0000 7f1b662fdb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626db0394e0 con 0x5626db0292d0 2023-07-21T18:47:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.594+0000 7f1b662fdb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626db01eea0 con 0x5626db023580 2023-07-21T18:47:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.594+0000 7f1b537fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5626dae7f6b0 0x5626db023040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.594+0000 7f1b52ffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5626db023580 0x5626db028130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.594+0000 7f1b537fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5626dae7f6b0 0x5626db023040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59846/0 (socket says 172.21.15.39:59846) 2023-07-21T18:47:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.594+0000 7f1b537fe700 1 -- 172.21.15.39:0/2405918665 learned_addr learned my addr 172.21.15.39:0/2405918665 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:47:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.594+0000 7f1b53fff700 1 --2- 172.21.15.39:0/2405918665 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626db0292d0 0x5626db02f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.594+0000 7f1b537fe700 1 -- 172.21.15.39:0/2405918665 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5626db023580 msgr2=0x5626db028130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:47:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.594+0000 7f1b537fe700 1 --2- 172.21.15.39:0/2405918665 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5626db023580 0x5626db028130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.594+0000 7f1b537fe700 1 -- 172.21.15.39:0/2405918665 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626db0292d0 msgr2=0x5626db02f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:47:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.594+0000 7f1b537fe700 1 --2- 172.21.15.39:0/2405918665 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626db0292d0 0x5626db02f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.594+0000 7f1b537fe700 1 -- 172.21.15.39:0/2405918665 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626db023c20 con 0x5626dae7f6b0 2023-07-21T18:47:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.594+0000 7f1b52ffd700 1 --2- 172.21.15.39:0/2405918665 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5626db023580 0x5626db028130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:47:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.594+0000 7f1b537fe700 1 --2- 172.21.15.39:0/2405918665 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5626dae7f6b0 0x5626db023040 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f1b44099bc0 tx=0x7f1b440b8400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=639a956665240fe5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:47:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.595+0000 7f1b3a7fc700 1 -- 172.21.15.39:0/2405918665 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b440c8070 con 0x5626dae7f6b0 2023-07-21T18:47:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.595+0000 7f1b3a7fc700 1 -- 172.21.15.39:0/2405918665 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b44065070 con 0x5626dae7f6b0 2023-07-21T18:47:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.595+0000 7f1b3a7fc700 1 -- 172.21.15.39:0/2405918665 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b440a1e80 con 0x5626dae7f6b0 2023-07-21T18:47:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.595+0000 7f1b662fdb80 1 -- 172.21.15.39:0/2405918665 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5626dae7f6b0 msgr2=0x5626db023040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:47:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.595+0000 7f1b662fdb80 1 --2- 172.21.15.39:0/2405918665 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5626dae7f6b0 0x5626db023040 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f1b44099bc0 tx=0x7f1b440b8400 comp rx=0 tx=0).stop 2023-07-21T18:47:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.595+0000 7f1b662fdb80 1 -- 172.21.15.39:0/2405918665 shutdown_connections 2023-07-21T18:47:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.595+0000 7f1b662fdb80 1 --2- 172.21.15.39:0/2405918665 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626db0292d0 0x5626db02f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.595+0000 7f1b662fdb80 1 --2- 172.21.15.39:0/2405918665 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5626dae7f6b0 0x5626db023040 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.595+0000 7f1b662fdb80 1 --2- 172.21.15.39:0/2405918665 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5626db023580 0x5626db028130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.595+0000 7f1b662fdb80 1 -- 172.21.15.39:0/2405918665 >> 172.21.15.39:0/2405918665 conn(0x5626db018be0 msgr2=0x5626db027a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:47:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.595+0000 7f1b662fdb80 1 -- 172.21.15.39:0/2405918665 shutdown_connections 2023-07-21T18:47:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.595+0000 7f1b662fdb80 1 -- 172.21.15.39:0/2405918665 wait complete. 2023-07-21T18:47:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.595+0000 7f1b662fdb80 1 -- start start 2023-07-21T18:47:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.596+0000 7f1b662fdb80 1 Processor -- start 2023-07-21T18:47:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.596+0000 7f1b662fdb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5626dae7f6b0 0x5626db032e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.596+0000 7f1b662fdb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5626db023580 0x5626db0333d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.596+0000 7f1b662fdb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626db0292d0 0x5626db08f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.596+0000 7f1b662fdb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626db033cc0 con 0x5626dae7f6b0 2023-07-21T18:47:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.596+0000 7f1b662fdb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626db08f9a0 con 0x5626db0292d0 2023-07-21T18:47:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.596+0000 7f1b662fdb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626db08fae0 con 0x5626db023580 2023-07-21T18:47:16.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.596+0000 7f1b662fdb80 10 client.0 _ll_register_callbacks cb 0x7f1b4c009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:47:16.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.596+0000 7f1b537fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5626dae7f6b0 0x5626db032e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:16.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.596+0000 7f1b52ffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5626db023580 0x5626db0333d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:16.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.596+0000 7f1b52ffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5626db023580 0x5626db0333d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:50174/0 (socket says 172.21.15.39:50174) 2023-07-21T18:47:16.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.596+0000 7f1b52ffd700 1 -- 172.21.15.39:0/1665487925 learned_addr learned my addr 172.21.15.39:0/1665487925 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:47:16.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.596+0000 7f1b537fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5626dae7f6b0 0x5626db032e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59852/0 (socket says 172.21.15.39:59852) 2023-07-21T18:47:16.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.596+0000 7f1b53fff700 1 --2- 172.21.15.39:0/1665487925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626db0292d0 0x5626db08f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:16.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.596+0000 7f1b52ffd700 1 -- 172.21.15.39:0/1665487925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626db0292d0 msgr2=0x5626db08f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:47:16.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.596+0000 7f1b52ffd700 1 --2- 172.21.15.39:0/1665487925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626db0292d0 0x5626db08f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:16.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.596+0000 7f1b52ffd700 1 -- 172.21.15.39:0/1665487925 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5626dae7f6b0 msgr2=0x5626db032e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:47:16.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.596+0000 7f1b52ffd700 1 --2- 172.21.15.39:0/1665487925 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5626dae7f6b0 0x5626db032e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:16.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.596+0000 7f1b52ffd700 1 -- 172.21.15.39:0/1665487925 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b5410f3d0 con 0x5626db023580 2023-07-21T18:47:16.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.597+0000 7f1b52ffd700 1 --2- 172.21.15.39:0/1665487925 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5626db023580 0x5626db0333d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1b480098f0 tx=0x7f1b48009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0ffb8a9456d845e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:47:16.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.597+0000 7f1b32ffd700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2023-07-21T18:47:16.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.597+0000 7f1b32ffd700 1 -- 172.21.15.39:0/1665487925 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b48035070 con 0x5626db023580 2023-07-21T18:47:16.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.597+0000 7f1b32ffd700 1 -- 172.21.15.39:0/1665487925 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f1b480050b0 con 0x5626db023580 2023-07-21T18:47:16.619 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.597+0000 7f1b662fdb80 1 -- 172.21.15.39:0/1665487925 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1b440a2310 con 0x5626db023580 2023-07-21T18:47:16.619 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.597+0000 7f1b32ffd700 1 -- 172.21.15.39:0/1665487925 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b48005270 con 0x5626db023580 2023-07-21T18:47:16.619 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.597+0000 7f1b662fdb80 10 client.26044 fetch_fsmap learned FSMap version 47 2023-07-21T18:47:16.619 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.597+0000 7f1b662fdb80 1 -- 172.21.15.39:0/1665487925 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x7f1b44061e70 con 0x5626db023580 2023-07-21T18:47:16.619 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.597+0000 7f1b32ffd700 1 -- 172.21.15.39:0/1665487925 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b440a2310 con 0x5626db023580 2023-07-21T18:47:16.619 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.597+0000 7f1b32ffd700 1 -- 172.21.15.39:0/1665487925 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 47) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f1b48003c60 con 0x5626db023580 2023-07-21T18:47:16.619 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.597+0000 7f1b662fdb80 10 client.26044 fetch_fsmap finished waiting for FSMap version 47 2023-07-21T18:47:16.619 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.597+0000 7f1b662fdb80 10 client.26044 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:47:16.620 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.597+0000 7f1b662fdb80 20 client.26044 populate_metadata read hostname 'smithi039' 2023-07-21T18:47:16.620 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.597+0000 7f1b662fdb80 1 --2- 172.21.15.39:0/1665487925 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5626db0912d0 0x5626db0936e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:16.620 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.598+0000 7f1b662fdb80 4 client.26044 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:47:16.620 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.598+0000 7f1b662fdb80 1 -- 172.21.15.39:0/1665487925 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5626daf2fff0 con 0x5626db0912d0 2023-07-21T18:47:16.620 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.598+0000 7f1b537fe700 1 --2- 172.21.15.39:0/1665487925 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5626db0912d0 0x5626db0936e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:16.620 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.598+0000 7f1b32ffd700 10 client.26044 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:47:16.620 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.598+0000 7f1b537fe700 1 --2- 172.21.15.39:0/1665487925 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5626db0912d0 0x5626db0936e0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b841dccb7708bf84 server_cookie=c31f7c7403ccae2b in_seq=0 out_seq=0 2023-07-21T18:47:16.621 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:47:16 smithi193 ceph-mon[98204]: pgmap v169: 97 pgs: 97 active+clean; 118 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 248 op/s 2023-07-21T18:47:16.634 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:47:16 smithi155 ceph-mon[97934]: pgmap v169: 97 pgs: 97 active+clean; 118 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 248 op/s 2023-07-21T18:47:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.636+0000 7f1b32ffd700 1 -- 172.21.15.39:0/1665487925 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5626daf2fff0 con 0x5626db0912d0 2023-07-21T18:47:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.636+0000 7f1b32ffd700 10 client.26044 handle_command_reply: tid=0 2023-07-21T18:47:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.636+0000 7f1b662fdb80 1 client.26044 shutdown 2023-07-21T18:47:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.636+0000 7f1b662fdb80 1 -- 172.21.15.39:0/1665487925 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5626db023580 msgr2=0x5626db0333d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:47:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.636+0000 7f1b662fdb80 1 --2- 172.21.15.39:0/1665487925 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5626db023580 0x5626db0333d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1b480098f0 tx=0x7f1b48009c00 comp rx=0 tx=0).stop 2023-07-21T18:47:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.636+0000 7f1b662fdb80 1 -- 172.21.15.39:0/1665487925 shutdown_connections 2023-07-21T18:47:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.636+0000 7f1b662fdb80 1 --2- 172.21.15.39:0/1665487925 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5626db0912d0 0x5626db0936e0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.636+0000 7f1b662fdb80 1 --2- 172.21.15.39:0/1665487925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626db0292d0 0x5626db08f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.636+0000 7f1b662fdb80 1 --2- 172.21.15.39:0/1665487925 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5626dae7f6b0 0x5626db032e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.636+0000 7f1b32ffd700 0 client.26044 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:47:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.636+0000 7f1b662fdb80 1 --2- 172.21.15.39:0/1665487925 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5626db023580 0x5626db0333d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.636+0000 7f1b662fdb80 1 -- 172.21.15.39:0/1665487925 >> 172.21.15.39:0/1665487925 conn(0x5626db018be0 msgr2=0x5626db0282b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:47:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.636+0000 7f1b662fdb80 1 -- 172.21.15.39:0/1665487925 shutdown_connections 2023-07-21T18:47:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.636+0000 7f1b662fdb80 1 -- 172.21.15.39:0/1665487925 wait complete. 2023-07-21T18:47:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.636+0000 7f1b662fdb80 20 client.26044 trim_cache size 0 max 16384 2023-07-21T18:47:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:47:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:47:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:47:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:47:16.655 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.637+0000 7f1b5a177700 1 -- 172.21.15.39:0/670335010 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f1b3c0a12c0 msgr2=0x7f1b3c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:47:16.655 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.637+0000 7f1b5a177700 1 --2- 172.21.15.39:0/670335010 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f1b3c0a12c0 0x7f1b3c0a3770 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1b48009fd0 tx=0x7f1b48019040 comp rx=0 tx=0).stop 2023-07-21T18:47:16.655 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.637+0000 7f1b5a177700 1 -- 172.21.15.39:0/670335010 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b54072850 msgr2=0x7f1b54197ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:47:16.655 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.637+0000 7f1b5a177700 1 --2- 172.21.15.39:0/670335010 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b54072850 0x7f1b54197ec0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f1b4400e9c0 tx=0x7f1b4400ecd0 comp rx=0 tx=0).stop 2023-07-21T18:47:16.655 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.637+0000 7f1b5a177700 1 -- 172.21.15.39:0/670335010 shutdown_connections 2023-07-21T18:47:16.655 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.637+0000 7f1b5a177700 1 --2- 172.21.15.39:0/670335010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1b5410c3b0 0x7f1b5419ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:16.655 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.637+0000 7f1b5a177700 1 --2- 172.21.15.39:0/670335010 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b54072850 0x7f1b54197ec0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:16.656 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.637+0000 7f1b5a177700 1 --2- 172.21.15.39:0/670335010 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f1b3c0a12c0 0x7f1b3c0a3770 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:16.656 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.637+0000 7f1b5a177700 1 --2- 172.21.15.39:0/670335010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b54073160 0x7f1b54198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:16.656 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.637+0000 7f1b5a177700 1 -- 172.21.15.39:0/670335010 >> 172.21.15.39:0/670335010 conn(0x7f1b540fb650 msgr2=0x7f1b5410e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:47:16.656 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.637+0000 7f1b5a177700 1 -- 172.21.15.39:0/670335010 shutdown_connections 2023-07-21T18:47:16.656 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:16.637+0000 7f1b5a177700 1 -- 172.21.15.39:0/670335010 wait complete. 2023-07-21T18:47:16.680 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:47:16 smithi039 ceph-mon[96489]: pgmap v169: 97 pgs: 97 active+clean; 118 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 248 op/s 2023-07-21T18:47:17.273 INFO:tasks.workunit.client.0.smithi039.stdout: 1 323210 6.12 MB/sec execute 150 sec latency 23.584 ms 2023-07-21T18:47:18.273 INFO:tasks.workunit.client.0.smithi039.stdout: 1 324444 6.11 MB/sec execute 151 sec latency 16.977 ms 2023-07-21T18:47:18.613 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:47:18 smithi193 ceph-mon[98204]: pgmap v170: 97 pgs: 97 active+clean; 125 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 363 op/s 2023-07-21T18:47:18.632 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:47:18 smithi155 ceph-mon[97934]: pgmap v170: 97 pgs: 97 active+clean; 125 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 363 op/s 2023-07-21T18:47:18.680 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:47:18 smithi039 ceph-mon[96489]: pgmap v170: 97 pgs: 97 active+clean; 125 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 363 op/s 2023-07-21T18:47:19.273 INFO:tasks.workunit.client.0.smithi039.stdout: 1 326132 6.13 MB/sec execute 152 sec latency 46.165 ms 2023-07-21T18:47:20.273 INFO:tasks.workunit.client.0.smithi039.stdout: 1 327250 6.13 MB/sec execute 153 sec latency 48.514 ms 2023-07-21T18:47:20.613 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:47:20 smithi193 ceph-mon[98204]: pgmap v171: 97 pgs: 97 active+clean; 125 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 221 op/s 2023-07-21T18:47:20.631 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:47:20 smithi155 ceph-mon[97934]: pgmap v171: 97 pgs: 97 active+clean; 125 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 221 op/s 2023-07-21T18:47:20.680 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:47:20 smithi039 ceph-mon[96489]: pgmap v171: 97 pgs: 97 active+clean; 125 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 221 op/s 2023-07-21T18:47:21.273 INFO:tasks.workunit.client.0.smithi039.stdout: 1 329328 6.17 MB/sec execute 154 sec latency 19.053 ms 2023-07-21T18:47:22.273 INFO:tasks.workunit.client.0.smithi039.stdout: 1 330596 6.17 MB/sec execute 155 sec latency 40.381 ms 2023-07-21T18:47:22.613 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:47:22 smithi193 ceph-mon[98204]: pgmap v172: 97 pgs: 97 active+clean; 121 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 342 op/s 2023-07-21T18:47:22.632 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:47:22 smithi155 ceph-mon[97934]: pgmap v172: 97 pgs: 97 active+clean; 121 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 342 op/s 2023-07-21T18:47:22.680 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:47:22 smithi039 ceph-mon[96489]: pgmap v172: 97 pgs: 97 active+clean; 121 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 342 op/s 2023-07-21T18:47:23.273 INFO:tasks.workunit.client.0.smithi039.stdout: 1 332660 6.20 MB/sec execute 156 sec latency 18.574 ms 2023-07-21T18:47:24.273 INFO:tasks.workunit.client.0.smithi039.stdout: 1 333995 6.21 MB/sec execute 157 sec latency 35.458 ms 2023-07-21T18:47:24.613 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:47:24 smithi193 ceph-mon[98204]: pgmap v173: 97 pgs: 97 active+clean; 121 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 235 op/s 2023-07-21T18:47:24.631 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:47:24 smithi155 ceph-mon[97934]: pgmap v173: 97 pgs: 97 active+clean; 121 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 235 op/s 2023-07-21T18:47:24.680 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:47:24 smithi039 ceph-mon[96489]: pgmap v173: 97 pgs: 97 active+clean; 121 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 235 op/s 2023-07-21T18:47:25.273 INFO:tasks.workunit.client.0.smithi039.stdout: 1 335857 6.22 MB/sec execute 158 sec latency 65.514 ms 2023-07-21T18:47:26.273 INFO:tasks.workunit.client.0.smithi039.stdout: 1 337399 6.25 MB/sec execute 159 sec latency 11.498 ms 2023-07-21T18:47:26.613 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:47:26 smithi193 ceph-mon[98204]: pgmap v174: 97 pgs: 97 active+clean; 121 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 235 op/s 2023-07-21T18:47:26.631 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:47:26 smithi155 ceph-mon[97934]: pgmap v174: 97 pgs: 97 active+clean; 121 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 235 op/s 2023-07-21T18:47:26.680 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:47:26 smithi039 ceph-mon[96489]: pgmap v174: 97 pgs: 97 active+clean; 121 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 235 op/s 2023-07-21T18:47:27.273 INFO:tasks.workunit.client.0.smithi039.stdout: 1 339038 6.25 MB/sec execute 160 sec latency 28.301 ms 2023-07-21T18:47:28.273 INFO:tasks.workunit.client.0.smithi039.stdout: 1 340890 6.29 MB/sec execute 161 sec latency 10.964 ms 2023-07-21T18:47:28.612 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:47:28 smithi193 ceph-mon[98204]: pgmap v175: 97 pgs: 97 active+clean; 127 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.6 MiB/s wr, 347 op/s 2023-07-21T18:47:28.631 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:47:28 smithi155 ceph-mon[97934]: pgmap v175: 97 pgs: 97 active+clean; 127 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.6 MiB/s wr, 347 op/s 2023-07-21T18:47:28.680 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:47:28 smithi039 ceph-mon[96489]: pgmap v175: 97 pgs: 97 active+clean; 127 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.6 MiB/s wr, 347 op/s 2023-07-21T18:47:29.273 INFO:tasks.workunit.client.0.smithi039.stdout: 1 342344 6.29 MB/sec execute 162 sec latency 27.972 ms 2023-07-21T18:47:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.756+0000 7fadddba9700 1 -- 172.21.15.39:0/573602129 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fadd80fea30 msgr2=0x7fadd80feea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:47:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.756+0000 7fadddba9700 1 --2- 172.21.15.39:0/573602129 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fadd80fea30 0x7fadd80feea0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fadc8009990 tx=0x7fadc8009ca0 comp rx=0 tx=0).stop 2023-07-21T18:47:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.756+0000 7fadddba9700 1 -- 172.21.15.39:0/573602129 shutdown_connections 2023-07-21T18:47:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.756+0000 7fadddba9700 1 --2- 172.21.15.39:0/573602129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fadd80ff3e0 0x7fadd810b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.756+0000 7fadddba9700 1 --2- 172.21.15.39:0/573602129 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fadd8104a30 0x7fadd8104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.756+0000 7fadddba9700 1 --2- 172.21.15.39:0/573602129 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fadd80fea30 0x7fadd80feea0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.756+0000 7fadddba9700 1 -- 172.21.15.39:0/573602129 >> 172.21.15.39:0/573602129 conn(0x7fadd8074760 msgr2=0x7fadd8074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:47:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.756+0000 7fadddba9700 1 -- 172.21.15.39:0/573602129 shutdown_connections 2023-07-21T18:47:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.756+0000 7fadddba9700 1 -- 172.21.15.39:0/573602129 wait complete. 2023-07-21T18:47:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.757+0000 7fadddba9700 1 Processor -- start 2023-07-21T18:47:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.757+0000 7fadddba9700 1 -- start start 2023-07-21T18:47:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.757+0000 7fadddba9700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fadd80fea30 0x7fadd819ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.757+0000 7fadddba9700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fadd80ff3e0 0x7fadd819b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.757+0000 7fadddba9700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fadd8104a30 0x7fadd8194960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.757+0000 7fadddba9700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd8194fd0 con 0x7fadd8104a30 2023-07-21T18:47:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.757+0000 7fadddba9700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd8195140 con 0x7fadd80ff3e0 2023-07-21T18:47:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.757+0000 7fadddba9700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd8195300 con 0x7fadd80fea30 2023-07-21T18:47:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.758+0000 7fadd7fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fadd8104a30 0x7fadd8194960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.758+0000 7fadd7fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fadd8104a30 0x7fadd8194960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59870/0 (socket says 172.21.15.39:59870) 2023-07-21T18:47:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.758+0000 7fadd6ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fadd80ff3e0 0x7fadd819b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.758+0000 7fadd7fff700 1 -- 172.21.15.39:0/2015947683 learned_addr learned my addr 172.21.15.39:0/2015947683 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:47:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.758+0000 7fadd77fe700 1 --2- 172.21.15.39:0/2015947683 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fadd80fea30 0x7fadd819ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.758+0000 7fadd7fff700 1 -- 172.21.15.39:0/2015947683 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fadd80fea30 msgr2=0x7fadd819ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:47:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.758+0000 7fadd7fff700 1 --2- 172.21.15.39:0/2015947683 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fadd80fea30 0x7fadd819ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.758+0000 7fadd7fff700 1 -- 172.21.15.39:0/2015947683 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fadd80ff3e0 msgr2=0x7fadd819b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:47:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.758+0000 7fadd7fff700 1 --2- 172.21.15.39:0/2015947683 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fadd80ff3e0 0x7fadd819b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.758+0000 7fadd7fff700 1 -- 172.21.15.39:0/2015947683 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadc80095f0 con 0x7fadd8104a30 2023-07-21T18:47:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.758+0000 7fadd7fff700 1 --2- 172.21.15.39:0/2015947683 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fadd8104a30 0x7fadd8194960 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fadd000ef30 tx=0x7fadd000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:47:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.758+0000 7fadd4ff9700 1 -- 172.21.15.39:0/2015947683 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd0018070 con 0x7fadd8104a30 2023-07-21T18:47:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.758+0000 7fadd4ff9700 1 -- 172.21.15.39:0/2015947683 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fadd000f040 con 0x7fadd8104a30 2023-07-21T18:47:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.758+0000 7fadddba9700 1 -- 172.21.15.39:0/2015947683 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadd8195580 con 0x7fadd8104a30 2023-07-21T18:47:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.758+0000 7fadd4ff9700 1 -- 172.21.15.39:0/2015947683 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd000cdb0 con 0x7fadd8104a30 2023-07-21T18:47:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.758+0000 7fadddba9700 1 -- 172.21.15.39:0/2015947683 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadd81a1af0 con 0x7fadd8104a30 2023-07-21T18:47:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.761+0000 7fadd4ff9700 1 -- 172.21.15.39:0/2015947683 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fadd0003a00 con 0x7fadd8104a30 2023-07-21T18:47:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.761+0000 7fadd4ff9700 1 --2- 172.21.15.39:0/2015947683 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fadc40a12c0 0x7fadc40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.761+0000 7fadd4ff9700 1 -- 172.21.15.39:0/2015947683 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fadd00cc990 con 0x7fadd8104a30 2023-07-21T18:47:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.761+0000 7fadd77fe700 1 --2- 172.21.15.39:0/2015947683 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fadc40a12c0 0x7fadc40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.762+0000 7fadd77fe700 1 --2- 172.21.15.39:0/2015947683 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fadc40a12c0 0x7fadc40a3770 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fadd80ffba0 tx=0x7fadc0009260 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:47:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.762+0000 7fade9d2fb80 1 Processor -- start 2023-07-21T18:47:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.762+0000 7fade9d2fb80 1 -- start start 2023-07-21T18:47:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.762+0000 7fade9d2fb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556a685b16b0 0x556a6875ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.762+0000 7fade9d2fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a6875b2d0 0x556a68755040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.763+0000 7fade9d2fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a68755580 0x556a687559d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.763+0000 7fade9d2fb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a6865ac50 con 0x556a685b16b0 2023-07-21T18:47:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.763+0000 7fade9d2fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a686ad9e0 con 0x556a6875b2d0 2023-07-21T18:47:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.763+0000 7fade9d2fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a686a6e70 con 0x556a68755580 2023-07-21T18:47:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.763+0000 7fadd6ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a6875b2d0 0x556a68755040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.763+0000 7fadd6ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a6875b2d0 0x556a68755040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:44240/0 (socket says 172.21.15.39:44240) 2023-07-21T18:47:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.763+0000 7fadd6ffd700 1 -- 172.21.15.39:0/2480945348 learned_addr learned my addr 172.21.15.39:0/2480945348 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:47:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.763+0000 7fadd77fe700 1 --2- 172.21.15.39:0/2480945348 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556a685b16b0 0x556a6875ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.763+0000 7fadd7fff700 1 --2- 172.21.15.39:0/2480945348 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a68755580 0x556a687559d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.763+0000 7fadd6ffd700 1 -- 172.21.15.39:0/2480945348 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a68755580 msgr2=0x556a687559d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:47:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.763+0000 7fadd6ffd700 1 --2- 172.21.15.39:0/2480945348 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a68755580 0x556a687559d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.763+0000 7fadd6ffd700 1 -- 172.21.15.39:0/2480945348 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556a685b16b0 msgr2=0x556a6875ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:47:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.763+0000 7fadd6ffd700 1 --2- 172.21.15.39:0/2480945348 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556a685b16b0 0x556a6875ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.763+0000 7fadd6ffd700 1 -- 172.21.15.39:0/2480945348 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a684d4d80 con 0x556a6875b2d0 2023-07-21T18:47:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.763+0000 7fadd7fff700 1 --2- 172.21.15.39:0/2480945348 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a68755580 0x556a687559d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:47:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.763+0000 7fadd6ffd700 1 --2- 172.21.15.39:0/2480945348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a6875b2d0 0x556a68755040 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fadc8015000 tx=0x7fadc8005ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f492a04fba81cc8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:47:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.763+0000 7fadcdffb700 1 -- 172.21.15.39:0/2480945348 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc800bee0 con 0x556a6875b2d0 2023-07-21T18:47:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.763+0000 7fadcdffb700 1 -- 172.21.15.39:0/2480945348 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fadc8005500 con 0x556a6875b2d0 2023-07-21T18:47:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.764+0000 7fadcdffb700 1 -- 172.21.15.39:0/2480945348 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc8004ad0 con 0x556a6875b2d0 2023-07-21T18:47:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.764+0000 7fade9d2fb80 1 -- 172.21.15.39:0/2480945348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a6875b2d0 msgr2=0x556a68755040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:47:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.764+0000 7fade9d2fb80 1 --2- 172.21.15.39:0/2480945348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a6875b2d0 0x556a68755040 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fadc8015000 tx=0x7fadc8005ea0 comp rx=0 tx=0).stop 2023-07-21T18:47:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.764+0000 7fade9d2fb80 1 -- 172.21.15.39:0/2480945348 shutdown_connections 2023-07-21T18:47:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.764+0000 7fade9d2fb80 1 --2- 172.21.15.39:0/2480945348 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a68755580 0x556a687559d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.764+0000 7fade9d2fb80 1 --2- 172.21.15.39:0/2480945348 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556a685b16b0 0x556a6875ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.764+0000 7fade9d2fb80 1 --2- 172.21.15.39:0/2480945348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a6875b2d0 0x556a68755040 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.764+0000 7fade9d2fb80 1 -- 172.21.15.39:0/2480945348 >> 172.21.15.39:0/2480945348 conn(0x556a6874abe0 msgr2=0x556a6874ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:47:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.764+0000 7fade9d2fb80 1 -- 172.21.15.39:0/2480945348 shutdown_connections 2023-07-21T18:47:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.764+0000 7fade9d2fb80 1 -- 172.21.15.39:0/2480945348 wait complete. 2023-07-21T18:47:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.765+0000 7fade9d2fb80 1 -- start start 2023-07-21T18:47:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.765+0000 7fade9d2fb80 1 Processor -- start 2023-07-21T18:47:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.765+0000 7fade9d2fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a685b16b0 0x556a68764520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.765+0000 7fade9d2fb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556a68755580 0x556a68764a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.765+0000 7fade9d2fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a6875b2d0 0x556a6875e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.765+0000 7fade9d2fb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a68646a60 con 0x556a68755580 2023-07-21T18:47:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.765+0000 7fade9d2fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a68646760 con 0x556a685b16b0 2023-07-21T18:47:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.765+0000 7fade9d2fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a6852a8c0 con 0x556a6875b2d0 2023-07-21T18:47:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.765+0000 7fade9d2fb80 10 client.0 _ll_register_callbacks cb 0x556a68644700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:47:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.765+0000 7fadd6ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556a68755580 0x556a68764a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.766+0000 7fadd6ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556a68755580 0x556a68764a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59882/0 (socket says 172.21.15.39:59882) 2023-07-21T18:47:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.766+0000 7fadd77fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a685b16b0 0x556a68764520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.766+0000 7fadd6ffd700 1 -- 172.21.15.39:0/1100796239 learned_addr learned my addr 172.21.15.39:0/1100796239 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:47:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.766+0000 7fadd7fff700 1 --2- 172.21.15.39:0/1100796239 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a6875b2d0 0x556a6875e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.766+0000 7fadd6ffd700 1 -- 172.21.15.39:0/1100796239 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a6875b2d0 msgr2=0x556a6875e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:47:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.766+0000 7fadd6ffd700 1 --2- 172.21.15.39:0/1100796239 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a6875b2d0 0x556a6875e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.766+0000 7fadd6ffd700 1 -- 172.21.15.39:0/1100796239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a685b16b0 msgr2=0x556a68764520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:47:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.766+0000 7fadd6ffd700 1 --2- 172.21.15.39:0/1100796239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a685b16b0 0x556a68764520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.766+0000 7fadd6ffd700 1 -- 172.21.15.39:0/1100796239 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadd810c160 con 0x556a68755580 2023-07-21T18:47:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.766+0000 7fadd7fff700 1 --2- 172.21.15.39:0/1100796239 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a6875b2d0 0x556a6875e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:47:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.766+0000 7fadd6ffd700 1 --2- 172.21.15.39:0/1100796239 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556a68755580 0x556a68764a60 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fadc80098e0 tx=0x7fadc8005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13f9e6f4a76c3651 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:47:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.766+0000 7fadb6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:47:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.766+0000 7fadb6ffd700 1 -- 172.21.15.39:0/1100796239 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc802e070 con 0x556a68755580 2023-07-21T18:47:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.766+0000 7fadb6ffd700 1 -- 172.21.15.39:0/1100796239 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fadc800f670 con 0x556a68755580 2023-07-21T18:47:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.766+0000 7fade9d2fb80 1 -- 172.21.15.39:0/1100796239 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556a68600490 con 0x556a68755580 2023-07-21T18:47:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.766+0000 7fadb6ffd700 1 -- 172.21.15.39:0/1100796239 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fadc800f830 con 0x556a68755580 2023-07-21T18:47:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.766+0000 7fade9d2fb80 10 client.16569 fetch_fsmap learned FSMap version 47 2023-07-21T18:47:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.766+0000 7fade9d2fb80 1 -- 172.21.15.39:0/1100796239 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x556a684bdc00 con 0x556a68755580 2023-07-21T18:47:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.767+0000 7fadb6ffd700 1 -- 172.21.15.39:0/1100796239 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fadc803f020 con 0x556a68755580 2023-07-21T18:47:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.767+0000 7fadb6ffd700 1 -- 172.21.15.39:0/1100796239 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 47) v1 ==== 7490+0+0 (secure 0 0 0) 0x7fadc8027020 con 0x556a68755580 2023-07-21T18:47:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.767+0000 7fade9d2fb80 10 client.16569 fetch_fsmap finished waiting for FSMap version 47 2023-07-21T18:47:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.767+0000 7fade9d2fb80 10 client.16569 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:47:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.767+0000 7fade9d2fb80 20 client.16569 populate_metadata read hostname 'smithi039' 2023-07-21T18:47:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.767+0000 7fade9d2fb80 1 --2- 172.21.15.39:0/1100796239 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x556a6876e5e0 0x556a687709f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.767+0000 7fade9d2fb80 4 client.16569 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:47:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.767+0000 7fade9d2fb80 1 -- 172.21.15.39:0/1100796239 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556a6869fe50 con 0x556a6876e5e0 2023-07-21T18:47:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.767+0000 7fadd77fe700 1 --2- 172.21.15.39:0/1100796239 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x556a6876e5e0 0x556a687709f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.768+0000 7fadb6ffd700 10 client.16569 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:47:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.768+0000 7fadd77fe700 1 --2- 172.21.15.39:0/1100796239 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x556a6876e5e0 0x556a687709f0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac4e8153fae21c37 server_cookie=3b406a6c171508f1 in_seq=0 out_seq=0 2023-07-21T18:47:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.768+0000 7fadb6ffd700 1 -- 172.21.15.39:0/1100796239 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556a6869fe50 con 0x556a6876e5e0 2023-07-21T18:47:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.768+0000 7fadb6ffd700 10 client.16569 handle_command_reply: tid=0 2023-07-21T18:47:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.768+0000 7fade9d2fb80 1 client.16569 shutdown 2023-07-21T18:47:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.768+0000 7fade9d2fb80 1 -- 172.21.15.39:0/1100796239 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556a68755580 msgr2=0x556a68764a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:47:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.768+0000 7fade9d2fb80 1 --2- 172.21.15.39:0/1100796239 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556a68755580 0x556a68764a60 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fadc80098e0 tx=0x7fadc8005f10 comp rx=0 tx=0).stop 2023-07-21T18:47:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.768+0000 7fade9d2fb80 1 -- 172.21.15.39:0/1100796239 shutdown_connections 2023-07-21T18:47:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.768+0000 7fade9d2fb80 1 --2- 172.21.15.39:0/1100796239 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x556a6876e5e0 0x556a687709f0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.768+0000 7fade9d2fb80 1 --2- 172.21.15.39:0/1100796239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a685b16b0 0x556a68764520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.768+0000 7fadb6ffd700 0 client.16569 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:47:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.768+0000 7fade9d2fb80 1 --2- 172.21.15.39:0/1100796239 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a6875b2d0 0x556a6875e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.768+0000 7fade9d2fb80 1 --2- 172.21.15.39:0/1100796239 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556a68755580 0x556a68764a60 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.768+0000 7fade9d2fb80 1 -- 172.21.15.39:0/1100796239 >> 172.21.15.39:0/1100796239 conn(0x556a6874abe0 msgr2=0x556a6874ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:47:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.769+0000 7fade9d2fb80 1 -- 172.21.15.39:0/1100796239 shutdown_connections 2023-07-21T18:47:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.769+0000 7fade9d2fb80 1 -- 172.21.15.39:0/1100796239 wait complete. 2023-07-21T18:47:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.769+0000 7fade9d2fb80 20 client.16569 trim_cache size 0 max 16384 2023-07-21T18:47:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.784+0000 7fade9d2fb80 1 Processor -- start 2023-07-21T18:47:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.784+0000 7fade9d2fb80 1 -- start start 2023-07-21T18:47:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.784+0000 7fade9d2fb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556a685b16b0 0x556a68755040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.784+0000 7fade9d2fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a68755580 0x556a6875a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:29.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.784+0000 7fade9d2fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a6875b2d0 0x556a6874fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:29.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.784+0000 7fade9d2fb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a68747e40 con 0x556a685b16b0 2023-07-21T18:47:29.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.784+0000 7fade9d2fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a6869fe50 con 0x556a68755580 2023-07-21T18:47:29.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.784+0000 7fade9d2fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a6875aa10 con 0x556a6875b2d0 2023-07-21T18:47:29.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.784+0000 7fadd77fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556a685b16b0 0x556a68755040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:29.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.784+0000 7fadd77fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556a685b16b0 0x556a68755040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59888/0 (socket says 172.21.15.39:59888) 2023-07-21T18:47:29.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.784+0000 7fadd77fe700 1 -- 172.21.15.39:0/611222340 learned_addr learned my addr 172.21.15.39:0/611222340 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:47:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.784+0000 7fadd6ffd700 1 --2- 172.21.15.39:0/611222340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a68755580 0x556a6875a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.784+0000 7fadd7fff700 1 --2- 172.21.15.39:0/611222340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a6875b2d0 0x556a6874fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.785+0000 7fadd77fe700 1 -- 172.21.15.39:0/611222340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a6875b2d0 msgr2=0x556a6874fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:47:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.785+0000 7fadd77fe700 1 --2- 172.21.15.39:0/611222340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a6875b2d0 0x556a6874fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.785+0000 7fadd77fe700 1 -- 172.21.15.39:0/611222340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a68755580 msgr2=0x556a6875a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:47:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.785+0000 7fadd77fe700 1 --2- 172.21.15.39:0/611222340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a68755580 0x556a6875a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.785+0000 7fadd77fe700 1 -- 172.21.15.39:0/611222340 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a684fb1f0 con 0x556a685b16b0 2023-07-21T18:47:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.785+0000 7fadd77fe700 1 --2- 172.21.15.39:0/611222340 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556a685b16b0 0x556a68755040 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fadc0010760 tx=0x7fadc0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71217e1b573842bd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:47:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.785+0000 7fadce7fc700 1 -- 172.21.15.39:0/611222340 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc0024040 con 0x556a685b16b0 2023-07-21T18:47:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.785+0000 7fadce7fc700 1 -- 172.21.15.39:0/611222340 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fadc0015070 con 0x556a685b16b0 2023-07-21T18:47:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.785+0000 7fadce7fc700 1 -- 172.21.15.39:0/611222340 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc001f690 con 0x556a685b16b0 2023-07-21T18:47:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.786+0000 7fade9d2fb80 1 -- 172.21.15.39:0/611222340 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556a685b16b0 msgr2=0x556a68755040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:47:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.786+0000 7fade9d2fb80 1 --2- 172.21.15.39:0/611222340 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556a685b16b0 0x556a68755040 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fadc0010760 tx=0x7fadc0013000 comp rx=0 tx=0).stop 2023-07-21T18:47:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.786+0000 7fade9d2fb80 1 -- 172.21.15.39:0/611222340 shutdown_connections 2023-07-21T18:47:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.786+0000 7fade9d2fb80 1 --2- 172.21.15.39:0/611222340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a6875b2d0 0x556a6874fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.786+0000 7fade9d2fb80 1 --2- 172.21.15.39:0/611222340 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556a685b16b0 0x556a68755040 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.786+0000 7fade9d2fb80 1 --2- 172.21.15.39:0/611222340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a68755580 0x556a6875a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.786+0000 7fade9d2fb80 1 -- 172.21.15.39:0/611222340 >> 172.21.15.39:0/611222340 conn(0x556a6874abe0 msgr2=0x556a6875e740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:47:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.786+0000 7fade9d2fb80 1 -- 172.21.15.39:0/611222340 shutdown_connections 2023-07-21T18:47:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.786+0000 7fade9d2fb80 1 -- 172.21.15.39:0/611222340 wait complete. 2023-07-21T18:47:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.786+0000 7fade9d2fb80 1 -- start start 2023-07-21T18:47:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.786+0000 7fade9d2fb80 1 Processor -- start 2023-07-21T18:47:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.787+0000 7fade9d2fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a685b16b0 0x556a687ba840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.787+0000 7fade9d2fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a68755580 0x556a687bad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.787+0000 7fade9d2fb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556a6875b2d0 0x556a687bb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.787+0000 7fade9d2fb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a687c59f0 con 0x556a6875b2d0 2023-07-21T18:47:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.787+0000 7fade9d2fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a687c5b30 con 0x556a68755580 2023-07-21T18:47:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.787+0000 7fade9d2fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a687c5cf0 con 0x556a685b16b0 2023-07-21T18:47:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.787+0000 7fade9d2fb80 10 client.0 _ll_register_callbacks cb 0x7fadc0009960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:47:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.787+0000 7fadd77fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a685b16b0 0x556a687ba840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.787+0000 7fadd77fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a685b16b0 0x556a687ba840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:50214/0 (socket says 172.21.15.39:50214) 2023-07-21T18:47:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.787+0000 7fadd6ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a68755580 0x556a687bad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.787+0000 7fadd77fe700 1 -- 172.21.15.39:0/3467907388 learned_addr learned my addr 172.21.15.39:0/3467907388 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:47:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.787+0000 7fadd7fff700 1 --2- 172.21.15.39:0/3467907388 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556a6875b2d0 0x556a687bb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:29.808 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.787+0000 7fadd77fe700 1 -- 172.21.15.39:0/3467907388 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a68755580 msgr2=0x556a687bad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:47:29.808 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.787+0000 7fadd77fe700 1 --2- 172.21.15.39:0/3467907388 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a68755580 0x556a687bad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:29.808 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.787+0000 7fadd77fe700 1 -- 172.21.15.39:0/3467907388 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556a6875b2d0 msgr2=0x556a687bb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:47:29.808 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.787+0000 7fadd77fe700 1 --2- 172.21.15.39:0/3467907388 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556a6875b2d0 0x556a687bb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:29.808 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.787+0000 7fadd77fe700 1 -- 172.21.15.39:0/3467907388 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadc0011440 con 0x556a685b16b0 2023-07-21T18:47:29.808 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.787+0000 7fadd7fff700 1 --2- 172.21.15.39:0/3467907388 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556a6875b2d0 0x556a687bb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:47:29.808 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.787+0000 7fadd77fe700 1 --2- 172.21.15.39:0/3467907388 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a685b16b0 0x556a687ba840 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fadc0009a90 tx=0x7fadc0010c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=740180f9cd70f581 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:47:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.787+0000 7fadb6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2023-07-21T18:47:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.788+0000 7fadb6ffd700 1 -- 172.21.15.39:0/3467907388 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc0024040 con 0x556a685b16b0 2023-07-21T18:47:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.788+0000 7fadb6ffd700 1 -- 172.21.15.39:0/3467907388 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fadc0017070 con 0x556a685b16b0 2023-07-21T18:47:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.788+0000 7fade9d2fb80 1 -- 172.21.15.39:0/3467907388 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fadc8005500 con 0x556a685b16b0 2023-07-21T18:47:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.788+0000 7fadb6ffd700 1 -- 172.21.15.39:0/3467907388 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fadc0011c90 con 0x556a685b16b0 2023-07-21T18:47:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.788+0000 7fade9d2fb80 10 client.26062 fetch_fsmap learned FSMap version 47 2023-07-21T18:47:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.788+0000 7fade9d2fb80 1 -- 172.21.15.39:0/3467907388 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x556a684d3250 con 0x556a685b16b0 2023-07-21T18:47:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.788+0000 7fadb6ffd700 1 -- 172.21.15.39:0/3467907388 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fadc8005500 con 0x556a685b16b0 2023-07-21T18:47:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.788+0000 7fadb6ffd700 1 -- 172.21.15.39:0/3467907388 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 47) v1 ==== 7490+0+0 (secure 0 0 0) 0x7fadc001f430 con 0x556a685b16b0 2023-07-21T18:47:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.788+0000 7fade9d2fb80 10 client.26062 fetch_fsmap finished waiting for FSMap version 47 2023-07-21T18:47:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.788+0000 7fade9d2fb80 10 client.26062 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:47:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.788+0000 7fade9d2fb80 20 client.26062 populate_metadata read hostname 'smithi039' 2023-07-21T18:47:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.788+0000 7fade9d2fb80 1 --2- 172.21.15.39:0/3467907388 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x556a687c74b0 0x556a687c98c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.788+0000 7fade9d2fb80 4 client.26062 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:47:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.788+0000 7fade9d2fb80 1 -- 172.21.15.39:0/3467907388 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556a68661ff0 con 0x556a687c74b0 2023-07-21T18:47:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.789+0000 7fadd6ffd700 1 --2- 172.21.15.39:0/3467907388 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x556a687c74b0 0x556a687c98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.789+0000 7fadb6ffd700 10 client.26062 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:47:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.789+0000 7fadd6ffd700 1 --2- 172.21.15.39:0/3467907388 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x556a687c74b0 0x556a687c98c0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71ef4814e10f522f server_cookie=f7510e38f7b9174c in_seq=0 out_seq=0 2023-07-21T18:47:29.840 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.825+0000 7fadb6ffd700 1 -- 172.21.15.39:0/3467907388 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556a68661ff0 con 0x556a687c74b0 2023-07-21T18:47:29.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.825+0000 7fadb6ffd700 10 client.26062 handle_command_reply: tid=0 2023-07-21T18:47:29.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.825+0000 7fade9d2fb80 1 client.26062 shutdown 2023-07-21T18:47:29.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.825+0000 7fade9d2fb80 1 -- 172.21.15.39:0/3467907388 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a685b16b0 msgr2=0x556a687ba840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:47:29.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.825+0000 7fade9d2fb80 1 --2- 172.21.15.39:0/3467907388 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a685b16b0 0x556a687ba840 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fadc0009a90 tx=0x7fadc0010c30 comp rx=0 tx=0).stop 2023-07-21T18:47:29.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.825+0000 7fade9d2fb80 1 -- 172.21.15.39:0/3467907388 shutdown_connections 2023-07-21T18:47:29.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.825+0000 7fade9d2fb80 1 --2- 172.21.15.39:0/3467907388 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x556a687c74b0 0x556a687c98c0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:29.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.825+0000 7fade9d2fb80 1 --2- 172.21.15.39:0/3467907388 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556a6875b2d0 0x556a687bb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:29.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.825+0000 7fade9d2fb80 1 --2- 172.21.15.39:0/3467907388 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a685b16b0 0x556a687ba840 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:29.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.825+0000 7fade9d2fb80 1 --2- 172.21.15.39:0/3467907388 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a68755580 0x556a687bad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:29.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.825+0000 7fadb6ffd700 0 client.26062 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:47:29.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.825+0000 7fade9d2fb80 1 -- 172.21.15.39:0/3467907388 >> 172.21.15.39:0/3467907388 conn(0x556a6874abe0 msgr2=0x556a6875cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:47:29.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.826+0000 7fade9d2fb80 1 -- 172.21.15.39:0/3467907388 shutdown_connections 2023-07-21T18:47:29.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.826+0000 7fade9d2fb80 1 -- 172.21.15.39:0/3467907388 wait complete. 2023-07-21T18:47:29.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.826+0000 7fade9d2fb80 20 client.26062 trim_cache size 0 max 16384 2023-07-21T18:47:29.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:47:29.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:47:29.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:47:29.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:47:29.844 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.826+0000 7fadddba9700 1 -- 172.21.15.39:0/2015947683 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fadc40a12c0 msgr2=0x7fadc40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:47:29.844 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.826+0000 7fadddba9700 1 --2- 172.21.15.39:0/2015947683 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fadc40a12c0 0x7fadc40a3770 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fadd80ffba0 tx=0x7fadc0009260 comp rx=0 tx=0).stop 2023-07-21T18:47:29.844 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.826+0000 7fadddba9700 1 -- 172.21.15.39:0/2015947683 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fadd8104a30 msgr2=0x7fadd8194960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:47:29.844 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.826+0000 7fadddba9700 1 --2- 172.21.15.39:0/2015947683 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fadd8104a30 0x7fadd8194960 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fadd000ef30 tx=0x7fadd000c3a0 comp rx=0 tx=0).stop 2023-07-21T18:47:29.844 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.826+0000 7fadddba9700 1 -- 172.21.15.39:0/2015947683 shutdown_connections 2023-07-21T18:47:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.826+0000 7fadddba9700 1 --2- 172.21.15.39:0/2015947683 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fadd8104a30 0x7fadd8194960 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.826+0000 7fadddba9700 1 --2- 172.21.15.39:0/2015947683 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fadd80fea30 0x7fadd819ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.826+0000 7fadddba9700 1 --2- 172.21.15.39:0/2015947683 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fadc40a12c0 0x7fadc40a3770 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.826+0000 7fadddba9700 1 --2- 172.21.15.39:0/2015947683 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fadd80ff3e0 0x7fadd819b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.826+0000 7fadddba9700 1 -- 172.21.15.39:0/2015947683 >> 172.21.15.39:0/2015947683 conn(0x7fadd8074760 msgr2=0x7fadd80fd250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:47:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.827+0000 7fadddba9700 1 -- 172.21.15.39:0/2015947683 shutdown_connections 2023-07-21T18:47:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:29.827+0000 7fadddba9700 1 -- 172.21.15.39:0/2015947683 wait complete. 2023-07-21T18:47:30.273 INFO:tasks.workunit.client.0.smithi039.stdout: 1 344336 6.33 MB/sec execute 163 sec latency 29.117 ms 2023-07-21T18:47:30.612 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:47:30 smithi193 ceph-mon[98204]: pgmap v176: 97 pgs: 97 active+clean; 127 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 232 op/s 2023-07-21T18:47:30.631 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:47:30 smithi155 ceph-mon[97934]: pgmap v176: 97 pgs: 97 active+clean; 127 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 232 op/s 2023-07-21T18:47:30.680 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:47:30 smithi039 ceph-mon[96489]: pgmap v176: 97 pgs: 97 active+clean; 127 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 232 op/s 2023-07-21T18:47:31.273 INFO:tasks.workunit.client.0.smithi039.stdout: 1 345586 6.33 MB/sec execute 164 sec latency 20.458 ms 2023-07-21T18:47:32.273 INFO:tasks.workunit.client.0.smithi039.stdout: 1 347404 6.34 MB/sec execute 165 sec latency 16.480 ms 2023-07-21T18:47:32.612 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:47:32 smithi193 ceph-mon[98204]: pgmap v177: 97 pgs: 97 active+clean; 126 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 356 op/s 2023-07-21T18:47:32.631 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:47:32 smithi155 ceph-mon[97934]: pgmap v177: 97 pgs: 97 active+clean; 126 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 356 op/s 2023-07-21T18:47:32.680 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:47:32 smithi039 ceph-mon[96489]: pgmap v177: 97 pgs: 97 active+clean; 126 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 356 op/s 2023-07-21T18:47:33.273 INFO:tasks.workunit.client.0.smithi039.stdout: 1 348589 6.34 MB/sec execute 166 sec latency 13.109 ms 2023-07-21T18:47:34.274 INFO:tasks.workunit.client.0.smithi039.stdout: 1 350630 6.37 MB/sec execute 167 sec latency 35.744 ms 2023-07-21T18:47:34.613 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:47:34 smithi193 ceph-mon[98204]: pgmap v178: 97 pgs: 97 active+clean; 126 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 235 op/s 2023-07-21T18:47:34.631 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:47:34 smithi155 ceph-mon[97934]: pgmap v178: 97 pgs: 97 active+clean; 126 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 235 op/s 2023-07-21T18:47:34.680 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:47:34 smithi039 ceph-mon[96489]: pgmap v178: 97 pgs: 97 active+clean; 126 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 235 op/s 2023-07-21T18:47:35.273 INFO:tasks.workunit.client.0.smithi039.stdout: 1 351984 6.38 MB/sec execute 168 sec latency 14.950 ms 2023-07-21T18:47:36.275 INFO:tasks.workunit.client.0.smithi039.stdout: 1 354088 6.41 MB/sec execute 169 sec latency 19.350 ms 2023-07-21T18:47:36.612 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:47:36 smithi193 ceph-mon[98204]: pgmap v179: 97 pgs: 97 active+clean; 126 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 235 op/s 2023-07-21T18:47:36.631 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:47:36 smithi155 ceph-mon[97934]: pgmap v179: 97 pgs: 97 active+clean; 126 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 235 op/s 2023-07-21T18:47:36.680 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:47:36 smithi039 ceph-mon[96489]: pgmap v179: 97 pgs: 97 active+clean; 126 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 235 op/s 2023-07-21T18:47:37.273 INFO:tasks.workunit.client.0.smithi039.stdout: 1 355442 6.41 MB/sec execute 170 sec latency 12.691 ms 2023-07-21T18:47:38.273 INFO:tasks.workunit.client.0.smithi039.stdout: 1 357108 6.41 MB/sec execute 171 sec latency 18.552 ms 2023-07-21T18:47:38.612 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:47:38 smithi193 ceph-mon[98204]: pgmap v180: 97 pgs: 97 active+clean; 132 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 341 op/s 2023-07-21T18:47:38.631 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:47:38 smithi155 ceph-mon[97934]: pgmap v180: 97 pgs: 97 active+clean; 132 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 341 op/s 2023-07-21T18:47:38.680 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:47:38 smithi039 ceph-mon[96489]: pgmap v180: 97 pgs: 97 active+clean; 132 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 341 op/s 2023-07-21T18:47:39.273 INFO:tasks.workunit.client.0.smithi039.stdout: 1 358671 6.45 MB/sec execute 172 sec latency 27.456 ms 2023-07-21T18:47:40.273 INFO:tasks.workunit.client.0.smithi039.stdout: 1 360341 6.44 MB/sec execute 173 sec latency 23.260 ms 2023-07-21T18:47:40.612 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:47:40 smithi193 ceph-mon[98204]: pgmap v181: 97 pgs: 97 active+clean; 132 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 229 op/s 2023-07-21T18:47:40.631 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:47:40 smithi155 ceph-mon[97934]: pgmap v181: 97 pgs: 97 active+clean; 132 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 229 op/s 2023-07-21T18:47:40.680 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:47:40 smithi039 ceph-mon[96489]: pgmap v181: 97 pgs: 97 active+clean; 132 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 229 op/s 2023-07-21T18:47:41.273 INFO:tasks.workunit.client.0.smithi039.stdout: 1 362119 6.48 MB/sec execute 174 sec latency 10.646 ms 2023-07-21T18:47:42.273 INFO:tasks.workunit.client.0.smithi039.stdout: 1 363310 6.48 MB/sec execute 175 sec latency 16.869 ms 2023-07-21T18:47:42.612 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:47:42 smithi193 ceph-mon[98204]: pgmap v182: 97 pgs: 97 active+clean; 137 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 340 op/s 2023-07-21T18:47:42.631 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:47:42 smithi155 ceph-mon[97934]: pgmap v182: 97 pgs: 97 active+clean; 137 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 340 op/s 2023-07-21T18:47:42.680 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:47:42 smithi039 ceph-mon[96489]: pgmap v182: 97 pgs: 97 active+clean; 137 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 340 op/s 2023-07-21T18:47:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.944+0000 7f9312ec5700 1 -- 172.21.15.39:0/2778977548 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f930c104a30 msgr2=0x7f930c104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:47:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.944+0000 7f9312ec5700 1 --2- 172.21.15.39:0/2778977548 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f930c104a30 0x7f930c104e00 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f9300009990 tx=0x7f9300009ca0 comp rx=0 tx=0).stop 2023-07-21T18:47:42.962 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.944+0000 7f9312ec5700 1 -- 172.21.15.39:0/2778977548 shutdown_connections 2023-07-21T18:47:42.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.944+0000 7f9312ec5700 1 --2- 172.21.15.39:0/2778977548 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f930c0ff3e0 0x7f930c10b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:42.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.944+0000 7f9312ec5700 1 --2- 172.21.15.39:0/2778977548 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f930c104a30 0x7f930c104e00 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:42.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.944+0000 7f9312ec5700 1 --2- 172.21.15.39:0/2778977548 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f930c0fea30 0x7f930c0feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:42.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.944+0000 7f9312ec5700 1 -- 172.21.15.39:0/2778977548 >> 172.21.15.39:0/2778977548 conn(0x7f930c074760 msgr2=0x7f930c074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:47:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.944+0000 7f9312ec5700 1 -- 172.21.15.39:0/2778977548 shutdown_connections 2023-07-21T18:47:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.944+0000 7f9312ec5700 1 -- 172.21.15.39:0/2778977548 wait complete. 2023-07-21T18:47:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.944+0000 7f9312ec5700 1 Processor -- start 2023-07-21T18:47:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.945+0000 7f9312ec5700 1 -- start start 2023-07-21T18:47:42.965 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.945+0000 7f9312ec5700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f930c0fea30 0x7f930c194970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:42.965 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.945+0000 7f9312ec5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f930c0ff3e0 0x7f930c194eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:42.965 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.945+0000 7f9312ec5700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f930c104a30 0x7f930c1a3e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.945+0000 7f9312ec5700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f930c195620 con 0x7f930c0ff3e0 2023-07-21T18:47:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.945+0000 7f9312ec5700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f930c1a4350 con 0x7f930c0fea30 2023-07-21T18:47:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.945+0000 7f9312ec5700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f930c1a4510 con 0x7f930c104a30 2023-07-21T18:47:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.945+0000 7f930bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f930c0ff3e0 0x7f930c194eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.945+0000 7f930bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f930c0ff3e0 0x7f930c194eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59910/0 (socket says 172.21.15.39:59910) 2023-07-21T18:47:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.945+0000 7f930bfff700 1 -- 172.21.15.39:0/944073038 learned_addr learned my addr 172.21.15.39:0/944073038 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:47:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.945+0000 7f9311462700 1 --2- 172.21.15.39:0/944073038 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f930c104a30 0x7f930c1a3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.945+0000 7f9310c61700 1 --2- 172.21.15.39:0/944073038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f930c0fea30 0x7f930c194970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.945+0000 7f930bfff700 1 -- 172.21.15.39:0/944073038 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f930c104a30 msgr2=0x7f930c1a3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:47:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.945+0000 7f930bfff700 1 --2- 172.21.15.39:0/944073038 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f930c104a30 0x7f930c1a3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.945+0000 7f930bfff700 1 -- 172.21.15.39:0/944073038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f930c0fea30 msgr2=0x7f930c194970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:47:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.945+0000 7f930bfff700 1 --2- 172.21.15.39:0/944073038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f930c0fea30 0x7f930c194970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.945+0000 7f930bfff700 1 -- 172.21.15.39:0/944073038 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9304009c40 con 0x7f930c0ff3e0 2023-07-21T18:47:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.945+0000 7f930bfff700 1 --2- 172.21.15.39:0/944073038 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f930c0ff3e0 0x7f930c194eb0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f92f800e9a0 tx=0x7f92f800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:47:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.946+0000 7f9309ffb700 1 -- 172.21.15.39:0/944073038 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f8009790 con 0x7f930c0ff3e0 2023-07-21T18:47:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.946+0000 7f9309ffb700 1 -- 172.21.15.39:0/944073038 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92f800ce80 con 0x7f930c0ff3e0 2023-07-21T18:47:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.946+0000 7f9312ec5700 1 -- 172.21.15.39:0/944073038 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93000095f0 con 0x7f930c0ff3e0 2023-07-21T18:47:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.946+0000 7f9309ffb700 1 -- 172.21.15.39:0/944073038 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f8018680 con 0x7f930c0ff3e0 2023-07-21T18:47:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.946+0000 7f9312ec5700 1 -- 172.21.15.39:0/944073038 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f930c1a4bc0 con 0x7f930c0ff3e0 2023-07-21T18:47:42.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.948+0000 7f9309ffb700 1 -- 172.21.15.39:0/944073038 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f92f8010430 con 0x7f930c0ff3e0 2023-07-21T18:47:42.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.949+0000 7f9309ffb700 1 --2- 172.21.15.39:0/944073038 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f92fc0a1340 0x7f92fc0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:42.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.949+0000 7f9309ffb700 1 -- 172.21.15.39:0/944073038 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f92f8014070 con 0x7f930c0ff3e0 2023-07-21T18:47:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.949+0000 7f9310c61700 1 --2- 172.21.15.39:0/944073038 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f92fc0a1340 0x7f92fc0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.949+0000 7f9310c61700 1 --2- 172.21.15.39:0/944073038 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f92fc0a1340 0x7f92fc0a37f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f930000b2e0 tx=0x7f9300005fd0 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:47:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.949+0000 7f931f04bb80 1 Processor -- start 2023-07-21T18:47:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.950+0000 7f931f04bb80 1 -- start start 2023-07-21T18:47:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.950+0000 7f931f04bb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d25ada76b0 0x55d25af50d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.950+0000 7f931f04bb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d25af512d0 0x55d25af4b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.950+0000 7f931f04bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d25af4b580 0x55d25af4b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.950+0000 7f931f04bb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d25ae50c50 con 0x55d25af512d0 2023-07-21T18:47:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.950+0000 7f931f04bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d25aea39e0 con 0x55d25af4b580 2023-07-21T18:47:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.950+0000 7f931f04bb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d25ae9ce70 con 0x55d25ada76b0 2023-07-21T18:47:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.950+0000 7f930bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d25af512d0 0x55d25af4b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.950+0000 7f930bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d25af512d0 0x55d25af4b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59918/0 (socket says 172.21.15.39:59918) 2023-07-21T18:47:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.950+0000 7f930bfff700 1 -- 172.21.15.39:0/2156387914 learned_addr learned my addr 172.21.15.39:0/2156387914 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:47:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.950+0000 7f9311462700 1 --2- 172.21.15.39:0/2156387914 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d25af4b580 0x55d25af4b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.950+0000 7f9310c61700 1 --2- 172.21.15.39:0/2156387914 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d25ada76b0 0x55d25af50d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:42.984 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.950+0000 7f930bfff700 1 -- 172.21.15.39:0/2156387914 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d25ada76b0 msgr2=0x55d25af50d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:47:42.984 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.950+0000 7f930bfff700 1 --2- 172.21.15.39:0/2156387914 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d25ada76b0 0x55d25af50d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:42.984 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.950+0000 7f930bfff700 1 -- 172.21.15.39:0/2156387914 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d25af4b580 msgr2=0x55d25af4b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:47:42.984 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.950+0000 7f930bfff700 1 --2- 172.21.15.39:0/2156387914 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d25af4b580 0x55d25af4b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.950+0000 7f930bfff700 1 -- 172.21.15.39:0/2156387914 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d25accad80 con 0x55d25af512d0 2023-07-21T18:47:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.950+0000 7f930bfff700 1 --2- 172.21.15.39:0/2156387914 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d25af512d0 0x55d25af4b040 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f92f80a1df0 tx=0x7f92f8099b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa1f9f1f7b94f04e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:47:42.987 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.951+0000 7f92f6ffd700 1 -- 172.21.15.39:0/2156387914 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f80a2320 con 0x55d25af512d0 2023-07-21T18:47:42.987 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.951+0000 7f92f6ffd700 1 -- 172.21.15.39:0/2156387914 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92f80965e0 con 0x55d25af512d0 2023-07-21T18:47:42.987 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.951+0000 7f92f6ffd700 1 -- 172.21.15.39:0/2156387914 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f80a1e90 con 0x55d25af512d0 2023-07-21T18:47:42.987 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.951+0000 7f931f04bb80 1 -- 172.21.15.39:0/2156387914 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d25af512d0 msgr2=0x55d25af4b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:47:42.987 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.951+0000 7f931f04bb80 1 --2- 172.21.15.39:0/2156387914 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d25af512d0 0x55d25af4b040 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f92f80a1df0 tx=0x7f92f8099b40 comp rx=0 tx=0).stop 2023-07-21T18:47:42.987 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.951+0000 7f931f04bb80 1 -- 172.21.15.39:0/2156387914 shutdown_connections 2023-07-21T18:47:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.951+0000 7f931f04bb80 1 --2- 172.21.15.39:0/2156387914 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d25af4b580 0x55d25af4b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.951+0000 7f931f04bb80 1 --2- 172.21.15.39:0/2156387914 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d25ada76b0 0x55d25af50d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.951+0000 7f931f04bb80 1 --2- 172.21.15.39:0/2156387914 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d25af512d0 0x55d25af4b040 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.951+0000 7f931f04bb80 1 -- 172.21.15.39:0/2156387914 >> 172.21.15.39:0/2156387914 conn(0x55d25af40be0 msgr2=0x55d25af45f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:47:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.951+0000 7f931f04bb80 1 -- 172.21.15.39:0/2156387914 shutdown_connections 2023-07-21T18:47:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.951+0000 7f931f04bb80 1 -- 172.21.15.39:0/2156387914 wait complete. 2023-07-21T18:47:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.952+0000 7f931f04bb80 1 -- start start 2023-07-21T18:47:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.952+0000 7f931f04bb80 1 Processor -- start 2023-07-21T18:47:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.952+0000 7f931f04bb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d25ada76b0 0x55d25af57650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.952+0000 7f931f04bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d25af4b580 0x55d25af548d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.952+0000 7f931f04bb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d25af512d0 0x55d25af55010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.952+0000 7f931f04bb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d25ae3ca60 con 0x55d25af512d0 2023-07-21T18:47:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.952+0000 7f931f04bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d25ae3c760 con 0x55d25af4b580 2023-07-21T18:47:42.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.952+0000 7f931f04bb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d25ad208c0 con 0x55d25ada76b0 2023-07-21T18:47:42.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.952+0000 7f931f04bb80 10 client.0 _ll_register_callbacks cb 0x55d25ae3a700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:47:42.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.952+0000 7f9311462700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d25af512d0 0x55d25af55010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:42.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.953+0000 7f9310c61700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d25ada76b0 0x55d25af57650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:42.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.953+0000 7f9311462700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d25af512d0 0x55d25af55010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59926/0 (socket says 172.21.15.39:59926) 2023-07-21T18:47:42.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.953+0000 7f9310c61700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d25ada76b0 0x55d25af57650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:50242/0 (socket says 172.21.15.39:50242) 2023-07-21T18:47:42.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.953+0000 7f9311462700 1 -- 172.21.15.39:0/2431957869 learned_addr learned my addr 172.21.15.39:0/2431957869 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:47:42.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.953+0000 7f930bfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d25af4b580 0x55d25af548d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:42.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.953+0000 7f9311462700 1 -- 172.21.15.39:0/2431957869 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d25ada76b0 msgr2=0x55d25af57650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:47:42.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.953+0000 7f9311462700 1 --2- 172.21.15.39:0/2431957869 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d25ada76b0 0x55d25af57650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:42.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.953+0000 7f9311462700 1 -- 172.21.15.39:0/2431957869 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d25af4b580 msgr2=0x55d25af548d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:47:42.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.953+0000 7f9311462700 1 --2- 172.21.15.39:0/2431957869 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d25af4b580 0x55d25af548d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:42.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.953+0000 7f9311462700 1 -- 172.21.15.39:0/2431957869 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f930c1a4bc0 con 0x55d25af512d0 2023-07-21T18:47:42.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.953+0000 7f9311462700 1 --2- 172.21.15.39:0/2431957869 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d25af512d0 0x55d25af55010 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f9304014e80 tx=0x7f930400e630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a7f1ff9161e0295 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:47:42.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.953+0000 7f92ebfff700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:47:42.992 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.953+0000 7f92ebfff700 1 -- 172.21.15.39:0/2431957869 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930400e040 con 0x55d25af512d0 2023-07-21T18:47:42.992 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.953+0000 7f931f04bb80 1 -- 172.21.15.39:0/2431957869 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d25adf6490 con 0x55d25af512d0 2023-07-21T18:47:42.992 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.953+0000 7f92ebfff700 1 -- 172.21.15.39:0/2431957869 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f9304016400 con 0x55d25af512d0 2023-07-21T18:47:42.992 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.953+0000 7f92ebfff700 1 -- 172.21.15.39:0/2431957869 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93040165c0 con 0x55d25af512d0 2023-07-21T18:47:42.992 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.953+0000 7f931f04bb80 10 client.16587 fetch_fsmap learned FSMap version 47 2023-07-21T18:47:42.992 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.953+0000 7f931f04bb80 1 -- 172.21.15.39:0/2431957869 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x55d25acb3c00 con 0x55d25af512d0 2023-07-21T18:47:42.992 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.954+0000 7f92ebfff700 1 -- 172.21.15.39:0/2431957869 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d25adf6490 con 0x55d25af512d0 2023-07-21T18:47:42.992 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.954+0000 7f92ebfff700 1 -- 172.21.15.39:0/2431957869 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 47) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f930401c070 con 0x55d25af512d0 2023-07-21T18:47:42.993 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.954+0000 7f931f04bb80 10 client.16587 fetch_fsmap finished waiting for FSMap version 47 2023-07-21T18:47:42.993 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.954+0000 7f931f04bb80 10 client.16587 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:47:42.993 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.954+0000 7f931f04bb80 20 client.16587 populate_metadata read hostname 'smithi039' 2023-07-21T18:47:42.993 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.954+0000 7f931f04bb80 1 --2- 172.21.15.39:0/2431957869 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d25af64d30 0x55d25af67140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:42.993 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.954+0000 7f931f04bb80 4 client.16587 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:47:42.993 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.954+0000 7f931f04bb80 1 -- 172.21.15.39:0/2431957869 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d25ae95e50 con 0x55d25af64d30 2023-07-21T18:47:42.994 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.954+0000 7f9310c61700 1 --2- 172.21.15.39:0/2431957869 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d25af64d30 0x55d25af67140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:42.994 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.954+0000 7f92ebfff700 10 client.16587 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:47:42.994 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.954+0000 7f9310c61700 1 --2- 172.21.15.39:0/2431957869 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d25af64d30 0x55d25af67140 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=929e867180a2dd3 server_cookie=b2d43f7fcddb9fa0 in_seq=0 out_seq=0 2023-07-21T18:47:42.994 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.955+0000 7f92ebfff700 1 -- 172.21.15.39:0/2431957869 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d25ae95e50 con 0x55d25af64d30 2023-07-21T18:47:42.994 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.955+0000 7f92ebfff700 10 client.16587 handle_command_reply: tid=0 2023-07-21T18:47:42.994 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.955+0000 7f931f04bb80 1 client.16587 shutdown 2023-07-21T18:47:42.994 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.955+0000 7f931f04bb80 1 -- 172.21.15.39:0/2431957869 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d25af512d0 msgr2=0x55d25af55010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:47:42.995 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.955+0000 7f931f04bb80 1 --2- 172.21.15.39:0/2431957869 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d25af512d0 0x55d25af55010 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f9304014e80 tx=0x7f930400e630 comp rx=0 tx=0).stop 2023-07-21T18:47:42.995 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.955+0000 7f931f04bb80 1 -- 172.21.15.39:0/2431957869 shutdown_connections 2023-07-21T18:47:42.995 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.955+0000 7f931f04bb80 1 --2- 172.21.15.39:0/2431957869 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d25af64d30 0x55d25af67140 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:42.995 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.955+0000 7f931f04bb80 1 --2- 172.21.15.39:0/2431957869 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d25af512d0 0x55d25af55010 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:42.995 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.955+0000 7f931f04bb80 1 --2- 172.21.15.39:0/2431957869 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d25ada76b0 0x55d25af57650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:42.995 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.955+0000 7f92ebfff700 0 client.16587 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:47:42.995 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.955+0000 7f931f04bb80 1 --2- 172.21.15.39:0/2431957869 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d25af4b580 0x55d25af548d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.955+0000 7f931f04bb80 1 -- 172.21.15.39:0/2431957869 >> 172.21.15.39:0/2431957869 conn(0x55d25af40be0 msgr2=0x55d25af454e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:47:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.956+0000 7f931f04bb80 1 -- 172.21.15.39:0/2431957869 shutdown_connections 2023-07-21T18:47:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.956+0000 7f931f04bb80 1 -- 172.21.15.39:0/2431957869 wait complete. 2023-07-21T18:47:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.956+0000 7f931f04bb80 20 client.16587 trim_cache size 0 max 16384 2023-07-21T18:47:42.999 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.971+0000 7f931f04bb80 1 Processor -- start 2023-07-21T18:47:43.000 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.971+0000 7f931f04bb80 1 -- start start 2023-07-21T18:47:43.000 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.971+0000 7f931f04bb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d25ada76b0 0x55d25af4b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:43.000 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.971+0000 7f931f04bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d25af4b580 0x55d25af50340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:43.000 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.971+0000 7f931f04bb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d25af512d0 0x55d25af5a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:43.000 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.971+0000 7f931f04bb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d25af55840 con 0x55d25ada76b0 2023-07-21T18:47:43.000 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.971+0000 7f931f04bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d25af42d00 con 0x55d25af4b580 2023-07-21T18:47:43.001 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.971+0000 7f931f04bb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d25af51110 con 0x55d25af512d0 2023-07-21T18:47:43.001 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.972+0000 7f9310c61700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d25ada76b0 0x55d25af4b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:43.001 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.972+0000 7f9310c61700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d25ada76b0 0x55d25af4b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59930/0 (socket says 172.21.15.39:59930) 2023-07-21T18:47:43.001 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.972+0000 7f9310c61700 1 -- 172.21.15.39:0/1415794675 learned_addr learned my addr 172.21.15.39:0/1415794675 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:47:43.001 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.972+0000 7f930bfff700 1 --2- 172.21.15.39:0/1415794675 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d25af4b580 0x55d25af50340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:43.001 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.972+0000 7f9310c61700 1 -- 172.21.15.39:0/1415794675 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d25af512d0 msgr2=0x55d25af5a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:47:43.001 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.972+0000 7f9311462700 1 --2- 172.21.15.39:0/1415794675 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d25af512d0 0x55d25af5a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:43.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.972+0000 7f9310c61700 1 --2- 172.21.15.39:0/1415794675 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d25af512d0 0x55d25af5a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:43.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.972+0000 7f9310c61700 1 -- 172.21.15.39:0/1415794675 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d25af4b580 msgr2=0x55d25af50340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:47:43.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.972+0000 7f9310c61700 1 --2- 172.21.15.39:0/1415794675 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d25af4b580 0x55d25af50340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:43.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.972+0000 7f9310c61700 1 -- 172.21.15.39:0/1415794675 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d25acf11f0 con 0x55d25ada76b0 2023-07-21T18:47:43.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.972+0000 7f9311462700 1 --2- 172.21.15.39:0/1415794675 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d25af512d0 0x55d25af5a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:47:43.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.972+0000 7f9310c61700 1 --2- 172.21.15.39:0/1415794675 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d25ada76b0 0x55d25af4b040 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f9300005f50 tx=0x7f9300020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f145c4e672148e9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:47:43.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.972+0000 7f92f77fe700 1 -- 172.21.15.39:0/1415794675 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930002d040 con 0x55d25ada76b0 2023-07-21T18:47:43.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.972+0000 7f92f77fe700 1 -- 172.21.15.39:0/1415794675 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f930002e040 con 0x55d25ada76b0 2023-07-21T18:47:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.972+0000 7f92f77fe700 1 -- 172.21.15.39:0/1415794675 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9300034030 con 0x55d25ada76b0 2023-07-21T18:47:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.973+0000 7f931f04bb80 1 -- 172.21.15.39:0/1415794675 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d25ada76b0 msgr2=0x55d25af4b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:47:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.973+0000 7f931f04bb80 1 --2- 172.21.15.39:0/1415794675 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d25ada76b0 0x55d25af4b040 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f9300005f50 tx=0x7f9300020000 comp rx=0 tx=0).stop 2023-07-21T18:47:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.973+0000 7f931f04bb80 1 -- 172.21.15.39:0/1415794675 shutdown_connections 2023-07-21T18:47:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.973+0000 7f931f04bb80 1 --2- 172.21.15.39:0/1415794675 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d25af512d0 0x55d25af5a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.973+0000 7f931f04bb80 1 --2- 172.21.15.39:0/1415794675 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d25ada76b0 0x55d25af4b040 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.973+0000 7f931f04bb80 1 --2- 172.21.15.39:0/1415794675 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d25af4b580 0x55d25af50340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.973+0000 7f931f04bb80 1 -- 172.21.15.39:0/1415794675 >> 172.21.15.39:0/1415794675 conn(0x55d25af40be0 msgr2=0x55d25af51a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:47:43.004 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.973+0000 7f931f04bb80 1 -- 172.21.15.39:0/1415794675 shutdown_connections 2023-07-21T18:47:43.004 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.973+0000 7f931f04bb80 1 -- 172.21.15.39:0/1415794675 wait complete. 2023-07-21T18:47:43.004 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.973+0000 7f931f04bb80 1 -- start start 2023-07-21T18:47:43.004 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.973+0000 7f931f04bb80 1 Processor -- start 2023-07-21T18:47:43.004 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.974+0000 7f931f04bb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d25ada76b0 0x55d25afb2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:43.004 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.974+0000 7f931f04bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d25af4b580 0x55d25afb26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:43.004 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.974+0000 7f931f04bb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d25af512d0 0x55d25afacb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:43.005 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.974+0000 7f931f04bb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d25afad250 con 0x55d25af512d0 2023-07-21T18:47:43.005 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.974+0000 7f931f04bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d25afad3c0 con 0x55d25af4b580 2023-07-21T18:47:43.005 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.974+0000 7f931f04bb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d25afad580 con 0x55d25ada76b0 2023-07-21T18:47:43.005 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.974+0000 7f931f04bb80 10 client.0 _ll_register_callbacks cb 0x7f93040147e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:47:43.005 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.974+0000 7f930bfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d25af4b580 0x55d25afb26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:43.005 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.974+0000 7f930bfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d25af4b580 0x55d25afb26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:44302/0 (socket says 172.21.15.39:44302) 2023-07-21T18:47:43.005 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.974+0000 7f930bfff700 1 -- 172.21.15.39:0/3742029274 learned_addr learned my addr 172.21.15.39:0/3742029274 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:47:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.974+0000 7f9311462700 1 --2- 172.21.15.39:0/3742029274 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d25af512d0 0x55d25afacb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.974+0000 7f9310c61700 1 --2- 172.21.15.39:0/3742029274 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d25ada76b0 0x55d25afb2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.974+0000 7f930bfff700 1 -- 172.21.15.39:0/3742029274 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d25ada76b0 msgr2=0x55d25afb2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:47:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.974+0000 7f930bfff700 1 --2- 172.21.15.39:0/3742029274 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d25ada76b0 0x55d25afb2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.974+0000 7f930bfff700 1 -- 172.21.15.39:0/3742029274 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d25af512d0 msgr2=0x55d25afacb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:47:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.974+0000 7f930bfff700 1 --2- 172.21.15.39:0/3742029274 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d25af512d0 0x55d25afacb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.974+0000 7f930bfff700 1 -- 172.21.15.39:0/3742029274 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f930c10c160 con 0x55d25af4b580 2023-07-21T18:47:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.974+0000 7f9311462700 1 --2- 172.21.15.39:0/3742029274 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d25af512d0 0x55d25afacb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:47:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.975+0000 7f930bfff700 1 --2- 172.21.15.39:0/3742029274 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d25af4b580 0x55d25afb26a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f92f809a090 tx=0x7f92f8098480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce14a63423080ae8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:47:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.975+0000 7f92ebfff700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-21T18:47:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.975+0000 7f92ebfff700 1 -- 172.21.15.39:0/3742029274 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f8099690 con 0x55d25af4b580 2023-07-21T18:47:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.975+0000 7f92ebfff700 1 -- 172.21.15.39:0/3742029274 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f92f8010e40 con 0x55d25af4b580 2023-07-21T18:47:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.975+0000 7f931f04bb80 1 -- 172.21.15.39:0/3742029274 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f92f80965e0 con 0x55d25af4b580 2023-07-21T18:47:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.975+0000 7f92ebfff700 1 -- 172.21.15.39:0/3742029274 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92f80b82e0 con 0x55d25af4b580 2023-07-21T18:47:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.975+0000 7f931f04bb80 10 client.26081 fetch_fsmap learned FSMap version 47 2023-07-21T18:47:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.975+0000 7f931f04bb80 1 -- 172.21.15.39:0/3742029274 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x55d25acc9250 con 0x55d25af4b580 2023-07-21T18:47:43.009 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.975+0000 7f92ebfff700 1 -- 172.21.15.39:0/3742029274 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92f80965e0 con 0x55d25af4b580 2023-07-21T18:47:43.009 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.975+0000 7f92ebfff700 1 -- 172.21.15.39:0/3742029274 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 47) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f92f8094c70 con 0x55d25af4b580 2023-07-21T18:47:43.009 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.975+0000 7f931f04bb80 10 client.26081 fetch_fsmap finished waiting for FSMap version 47 2023-07-21T18:47:43.009 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.975+0000 7f931f04bb80 10 client.26081 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:47:43.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.975+0000 7f931f04bb80 20 client.26081 populate_metadata read hostname 'smithi039' 2023-07-21T18:47:43.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.975+0000 7f931f04bb80 1 --2- 172.21.15.39:0/3742029274 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d25afb9c10 0x55d25afbc020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:43.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.975+0000 7f931f04bb80 4 client.26081 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:47:43.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.975+0000 7f931f04bb80 1 -- 172.21.15.39:0/3742029274 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d25ae57ff0 con 0x55d25afb9c10 2023-07-21T18:47:43.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.975+0000 7f9310c61700 1 --2- 172.21.15.39:0/3742029274 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d25afb9c10 0x55d25afbc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:43.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.976+0000 7f92ebfff700 10 client.26081 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:47:43.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:42.976+0000 7f9310c61700 1 --2- 172.21.15.39:0/3742029274 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d25afb9c10 0x55d25afbc020 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a346ed187dd6c741 server_cookie=9f1253725c02254f in_seq=0 out_seq=0 2023-07-21T18:47:43.026 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:43.011+0000 7f92ebfff700 1 -- 172.21.15.39:0/3742029274 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d25ae57ff0 con 0x55d25afb9c10 2023-07-21T18:47:43.026 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:43.011+0000 7f92ebfff700 10 client.26081 handle_command_reply: tid=0 2023-07-21T18:47:43.026 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:43.011+0000 7f931f04bb80 1 client.26081 shutdown 2023-07-21T18:47:43.026 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:43.011+0000 7f931f04bb80 1 -- 172.21.15.39:0/3742029274 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d25af4b580 msgr2=0x55d25afb26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:47:43.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:43.011+0000 7f931f04bb80 1 --2- 172.21.15.39:0/3742029274 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d25af4b580 0x55d25afb26a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f92f809a090 tx=0x7f92f8098480 comp rx=0 tx=0).stop 2023-07-21T18:47:43.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:43.011+0000 7f931f04bb80 1 -- 172.21.15.39:0/3742029274 shutdown_connections 2023-07-21T18:47:43.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:43.011+0000 7f931f04bb80 1 --2- 172.21.15.39:0/3742029274 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d25afb9c10 0x55d25afbc020 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:43.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:43.011+0000 7f931f04bb80 1 --2- 172.21.15.39:0/3742029274 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d25af512d0 0x55d25afacb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:43.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:43.011+0000 7f931f04bb80 1 --2- 172.21.15.39:0/3742029274 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d25ada76b0 0x55d25afb2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:43.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:43.011+0000 7f931f04bb80 1 --2- 172.21.15.39:0/3742029274 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d25af4b580 0x55d25afb26a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:43.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:43.011+0000 7f931f04bb80 1 -- 172.21.15.39:0/3742029274 >> 172.21.15.39:0/3742029274 conn(0x55d25af40be0 msgr2=0x55d25af52320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:47:43.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:43.011+0000 7f92ebfff700 0 client.26081 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:47:43.028 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:43.011+0000 7f931f04bb80 1 -- 172.21.15.39:0/3742029274 shutdown_connections 2023-07-21T18:47:43.028 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:43.011+0000 7f931f04bb80 1 -- 172.21.15.39:0/3742029274 wait complete. 2023-07-21T18:47:43.028 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:43.011+0000 7f931f04bb80 20 client.26081 trim_cache size 0 max 16384 2023-07-21T18:47:43.028 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:47:43.028 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:47:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:47:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:47:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:43.012+0000 7f9312ec5700 1 -- 172.21.15.39:0/944073038 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f92fc0a1340 msgr2=0x7f92fc0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:47:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:43.012+0000 7f9312ec5700 1 --2- 172.21.15.39:0/944073038 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f92fc0a1340 0x7f92fc0a37f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f930000b2e0 tx=0x7f9300005fd0 comp rx=0 tx=0).stop 2023-07-21T18:47:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:43.012+0000 7f9312ec5700 1 -- 172.21.15.39:0/944073038 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f930c0ff3e0 msgr2=0x7f930c194eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:47:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:43.012+0000 7f9312ec5700 1 --2- 172.21.15.39:0/944073038 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f930c0ff3e0 0x7f930c194eb0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f92f800e9a0 tx=0x7f92f800ecb0 comp rx=0 tx=0).stop 2023-07-21T18:47:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:43.012+0000 7f9312ec5700 1 -- 172.21.15.39:0/944073038 shutdown_connections 2023-07-21T18:47:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:43.012+0000 7f9312ec5700 1 --2- 172.21.15.39:0/944073038 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f92fc0a1340 0x7f92fc0a37f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:43.012+0000 7f9312ec5700 1 --2- 172.21.15.39:0/944073038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f930c0fea30 0x7f930c194970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:43.012+0000 7f9312ec5700 1 --2- 172.21.15.39:0/944073038 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f930c104a30 0x7f930c1a3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:43.012+0000 7f9312ec5700 1 --2- 172.21.15.39:0/944073038 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f930c0ff3e0 0x7f930c194eb0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:43.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:43.012+0000 7f9312ec5700 1 -- 172.21.15.39:0/944073038 >> 172.21.15.39:0/944073038 conn(0x7f930c074760 msgr2=0x7f930c0fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:47:43.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:43.012+0000 7f9312ec5700 1 -- 172.21.15.39:0/944073038 shutdown_connections 2023-07-21T18:47:43.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:43.012+0000 7f9312ec5700 1 -- 172.21.15.39:0/944073038 wait complete. 2023-07-21T18:47:43.273 INFO:tasks.workunit.client.0.smithi039.stdout: 1 364789 6.48 MB/sec execute 176 sec latency 21.516 ms 2023-07-21T18:47:44.273 INFO:tasks.workunit.client.0.smithi039.stdout: 1 365945 6.48 MB/sec execute 177 sec latency 57.462 ms 2023-07-21T18:47:44.612 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:47:44 smithi193 ceph-mon[98204]: pgmap v183: 97 pgs: 97 active+clean; 137 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 216 op/s 2023-07-21T18:47:44.631 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:47:44 smithi155 ceph-mon[97934]: pgmap v183: 97 pgs: 97 active+clean; 137 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 216 op/s 2023-07-21T18:47:44.680 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:47:44 smithi039 ceph-mon[96489]: pgmap v183: 97 pgs: 97 active+clean; 137 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 216 op/s 2023-07-21T18:47:45.273 INFO:tasks.workunit.client.0.smithi039.stdout: 1 367893 6.49 MB/sec execute 178 sec latency 24.735 ms 2023-07-21T18:47:46.274 INFO:tasks.workunit.client.0.smithi039.stdout: 1 369388 6.52 MB/sec execute 179 sec latency 10.966 ms 2023-07-21T18:47:46.612 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:47:46 smithi193 ceph-mon[98204]: pgmap v184: 97 pgs: 97 active+clean; 137 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 216 op/s 2023-07-21T18:47:46.631 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:47:46 smithi155 ceph-mon[97934]: pgmap v184: 97 pgs: 97 active+clean; 137 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 216 op/s 2023-07-21T18:47:46.680 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:47:46 smithi039 ceph-mon[96489]: pgmap v184: 97 pgs: 97 active+clean; 137 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 216 op/s 2023-07-21T18:47:47.274 INFO:tasks.workunit.client.0.smithi039.stdout: 1 370691 6.51 MB/sec execute 180 sec latency 19.070 ms 2023-07-21T18:47:48.273 INFO:tasks.workunit.client.0.smithi039.stdout: 1 372569 6.54 MB/sec execute 181 sec latency 19.292 ms 2023-07-21T18:47:48.612 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:47:48 smithi193 ceph-mon[98204]: pgmap v185: 97 pgs: 97 active+clean; 138 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 335 op/s 2023-07-21T18:47:48.631 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:47:48 smithi155 ceph-mon[97934]: pgmap v185: 97 pgs: 97 active+clean; 138 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 335 op/s 2023-07-21T18:47:48.680 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:47:48 smithi039 ceph-mon[96489]: pgmap v185: 97 pgs: 97 active+clean; 138 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 335 op/s 2023-07-21T18:47:49.274 INFO:tasks.workunit.client.0.smithi039.stdout: 1 373816 6.54 MB/sec execute 182 sec latency 36.618 ms 2023-07-21T18:47:50.274 INFO:tasks.workunit.client.0.smithi039.stdout: 1 375788 6.56 MB/sec execute 183 sec latency 24.863 ms 2023-07-21T18:47:50.612 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:47:50 smithi193 ceph-mon[98204]: pgmap v186: 97 pgs: 97 active+clean; 138 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 230 op/s 2023-07-21T18:47:50.631 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:47:50 smithi155 ceph-mon[97934]: pgmap v186: 97 pgs: 97 active+clean; 138 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 230 op/s 2023-07-21T18:47:50.680 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:47:50 smithi039 ceph-mon[96489]: pgmap v186: 97 pgs: 97 active+clean; 138 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 230 op/s 2023-07-21T18:47:51.274 INFO:tasks.workunit.client.0.smithi039.stdout: 1 376964 6.55 MB/sec execute 184 sec latency 16.077 ms 2023-07-21T18:47:52.274 INFO:tasks.workunit.client.0.smithi039.stdout: 1 378409 6.55 MB/sec execute 185 sec latency 16.701 ms 2023-07-21T18:47:52.612 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:47:52 smithi193 ceph-mon[98204]: pgmap v187: 97 pgs: 97 active+clean; 142 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.6 MiB/s wr, 338 op/s 2023-07-21T18:47:52.631 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:47:52 smithi155 ceph-mon[97934]: pgmap v187: 97 pgs: 97 active+clean; 142 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.6 MiB/s wr, 338 op/s 2023-07-21T18:47:52.680 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:47:52 smithi039 ceph-mon[96489]: pgmap v187: 97 pgs: 97 active+clean; 142 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.6 MiB/s wr, 338 op/s 2023-07-21T18:47:53.274 INFO:tasks.workunit.client.0.smithi039.stdout: 1 379946 6.58 MB/sec execute 186 sec latency 20.126 ms 2023-07-21T18:47:54.274 INFO:tasks.workunit.client.0.smithi039.stdout: 1 381543 6.57 MB/sec execute 187 sec latency 28.067 ms 2023-07-21T18:47:54.612 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:47:54 smithi193 ceph-mon[98204]: pgmap v188: 97 pgs: 97 active+clean; 142 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 227 op/s 2023-07-21T18:47:54.631 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:47:54 smithi155 ceph-mon[97934]: pgmap v188: 97 pgs: 97 active+clean; 142 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 227 op/s 2023-07-21T18:47:54.680 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:47:54 smithi039 ceph-mon[96489]: pgmap v188: 97 pgs: 97 active+clean; 142 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 227 op/s 2023-07-21T18:47:55.274 INFO:tasks.workunit.client.0.smithi039.stdout: 1 383349 6.61 MB/sec execute 188 sec latency 30.375 ms 2023-07-21T18:47:55.612 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:47:55 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:47:55.631 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:47:55 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:47:55.680 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:47:55 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:47:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.129+0000 7f5297fa9700 1 -- 172.21.15.39:0/3947442250 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5290100500 msgr2=0x7f529010c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:47:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.129+0000 7f5297fa9700 1 --2- 172.21.15.39:0/3947442250 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5290100500 0x7f529010c940 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f528800b0a0 tx=0x7f528800b3b0 comp rx=0 tx=0).stop 2023-07-21T18:47:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.129+0000 7f5297fa9700 1 -- 172.21.15.39:0/3947442250 shutdown_connections 2023-07-21T18:47:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.129+0000 7f5297fa9700 1 --2- 172.21.15.39:0/3947442250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5290105b80 0x7f5290105f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.129+0000 7f5297fa9700 1 --2- 172.21.15.39:0/3947442250 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5290100500 0x7f529010c940 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.129+0000 7f5297fa9700 1 --2- 172.21.15.39:0/3947442250 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52900ffb50 0x7f52900fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.129+0000 7f5297fa9700 1 -- 172.21.15.39:0/3947442250 >> 172.21.15.39:0/3947442250 conn(0x7f52900fb650 msgr2=0x7f52900fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:47:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.130+0000 7f5297fa9700 1 -- 172.21.15.39:0/3947442250 shutdown_connections 2023-07-21T18:47:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.130+0000 7f5297fa9700 1 -- 172.21.15.39:0/3947442250 wait complete. 2023-07-21T18:47:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.130+0000 7f5297fa9700 1 Processor -- start 2023-07-21T18:47:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.130+0000 7f5297fa9700 1 -- start start 2023-07-21T18:47:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.131+0000 7f5297fa9700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52900ffb50 0x7f529019bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.131+0000 7f5297fa9700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5290100500 0x7f529019c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.131+0000 7f5297fa9700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5290105b80 0x7f52901959b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.131+0000 7f5297fa9700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5290196020 con 0x7f5290100500 2023-07-21T18:47:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.131+0000 7f5297fa9700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5290196190 con 0x7f5290105b80 2023-07-21T18:47:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.131+0000 7f5297fa9700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5290196350 con 0x7f52900ffb50 2023-07-21T18:47:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.131+0000 7f5295544700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5290100500 0x7f529019c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.131+0000 7f5295544700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5290100500 0x7f529019c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59952/0 (socket says 172.21.15.39:59952) 2023-07-21T18:47:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.131+0000 7f5295544700 1 -- 172.21.15.39:0/145710524 learned_addr learned my addr 172.21.15.39:0/145710524 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:47:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.131+0000 7f5296546700 1 --2- 172.21.15.39:0/145710524 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5290105b80 0x7f52901959b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.131+0000 7f5295d45700 1 --2- 172.21.15.39:0/145710524 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52900ffb50 0x7f529019bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.131+0000 7f5295544700 1 -- 172.21.15.39:0/145710524 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52900ffb50 msgr2=0x7f529019bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:47:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.131+0000 7f5295544700 1 --2- 172.21.15.39:0/145710524 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52900ffb50 0x7f529019bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.131+0000 7f5295544700 1 -- 172.21.15.39:0/145710524 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5290105b80 msgr2=0x7f52901959b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:47:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.131+0000 7f5295544700 1 --2- 172.21.15.39:0/145710524 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5290105b80 0x7f52901959b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.131+0000 7f5295544700 1 -- 172.21.15.39:0/145710524 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5288009d00 con 0x7f5290100500 2023-07-21T18:47:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.131+0000 7f5295d45700 1 --2- 172.21.15.39:0/145710524 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52900ffb50 0x7f529019bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:47:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.131+0000 7f5296546700 1 --2- 172.21.15.39:0/145710524 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5290105b80 0x7f52901959b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:47:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.131+0000 7f5295544700 1 --2- 172.21.15.39:0/145710524 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5290100500 0x7f529019c090 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f528000b550 tx=0x7f528000b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:47:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.132+0000 7f5286ffd700 1 -- 172.21.15.39:0/145710524 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5280011830 con 0x7f5290100500 2023-07-21T18:47:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.132+0000 7f5286ffd700 1 -- 172.21.15.39:0/145710524 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5280011e50 con 0x7f5290100500 2023-07-21T18:47:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.132+0000 7f5286ffd700 1 -- 172.21.15.39:0/145710524 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528000f2e0 con 0x7f5290100500 2023-07-21T18:47:56.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.132+0000 7f5297fa9700 1 -- 172.21.15.39:0/145710524 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5290196630 con 0x7f5290100500 2023-07-21T18:47:56.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.132+0000 7f5297fa9700 1 -- 172.21.15.39:0/145710524 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52901a2b40 con 0x7f5290100500 2023-07-21T18:47:56.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.134+0000 7f5286ffd700 1 -- 172.21.15.39:0/145710524 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f5280011990 con 0x7f5290100500 2023-07-21T18:47:56.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.134+0000 7f5286ffd700 1 --2- 172.21.15.39:0/145710524 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f527c0a1340 0x7f527c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:56.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.135+0000 7f5286ffd700 1 -- 172.21.15.39:0/145710524 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f52800cc680 con 0x7f5290100500 2023-07-21T18:47:56.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.135+0000 7f5295d45700 1 --2- 172.21.15.39:0/145710524 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f527c0a1340 0x7f527c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:56.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.135+0000 7f5295d45700 1 --2- 172.21.15.39:0/145710524 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f527c0a1340 0x7f527c0a37f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f528c009cb0 tx=0x7f528c009260 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:47:56.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.135+0000 7f52a412fb80 1 Processor -- start 2023-07-21T18:47:56.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.136+0000 7f52a412fb80 1 -- start start 2023-07-21T18:47:56.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.136+0000 7f52a412fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bd57096b0 0x555bd58b2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:56.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.136+0000 7f52a412fb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555bd58b32d0 0x555bd58ad040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:56.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.136+0000 7f52a412fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555bd58ad580 0x555bd58ad9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:56.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.136+0000 7f52a412fb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bd57b2c50 con 0x555bd58b32d0 2023-07-21T18:47:56.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.136+0000 7f52a412fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bd58059e0 con 0x555bd58ad580 2023-07-21T18:47:56.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.136+0000 7f52a412fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bd57fee70 con 0x555bd57096b0 2023-07-21T18:47:56.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.136+0000 7f5295544700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555bd58b32d0 0x555bd58ad040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:56.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.136+0000 7f5295d45700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bd57096b0 0x555bd58b2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:56.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.136+0000 7f5296546700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555bd58ad580 0x555bd58ad9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:56.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.136+0000 7f5295d45700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bd57096b0 0x555bd58b2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:50278/0 (socket says 172.21.15.39:50278) 2023-07-21T18:47:56.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.136+0000 7f5295544700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555bd58b32d0 0x555bd58ad040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59960/0 (socket says 172.21.15.39:59960) 2023-07-21T18:47:56.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.136+0000 7f5295d45700 1 -- 172.21.15.39:0/238443287 learned_addr learned my addr 172.21.15.39:0/238443287 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:47:56.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.136+0000 7f5295544700 1 -- 172.21.15.39:0/238443287 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bd57096b0 msgr2=0x555bd58b2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:47:56.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.136+0000 7f5295544700 1 --2- 172.21.15.39:0/238443287 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bd57096b0 0x555bd58b2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:56.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.136+0000 7f5295544700 1 -- 172.21.15.39:0/238443287 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555bd58ad580 msgr2=0x555bd58ad9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:47:56.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.136+0000 7f5295544700 1 --2- 172.21.15.39:0/238443287 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555bd58ad580 0x555bd58ad9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:56.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.136+0000 7f5295544700 1 -- 172.21.15.39:0/238443287 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555bd562cd80 con 0x555bd58b32d0 2023-07-21T18:47:56.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.136+0000 7f5296546700 1 --2- 172.21.15.39:0/238443287 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555bd58ad580 0x555bd58ad9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:47:56.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.136+0000 7f5295544700 1 --2- 172.21.15.39:0/238443287 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555bd58b32d0 0x555bd58ad040 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f52800c9ea0 tx=0x7f5280099270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f06b86cc8526e2b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:47:56.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.137+0000 7f527bfff700 1 -- 172.21.15.39:0/238443287 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5280099820 con 0x555bd58b32d0 2023-07-21T18:47:56.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.137+0000 7f527bfff700 1 -- 172.21.15.39:0/238443287 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f528009d850 con 0x555bd58b32d0 2023-07-21T18:47:56.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.137+0000 7f527bfff700 1 -- 172.21.15.39:0/238443287 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52800b9310 con 0x555bd58b32d0 2023-07-21T18:47:56.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.137+0000 7f52a412fb80 1 -- 172.21.15.39:0/238443287 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555bd58b32d0 msgr2=0x555bd58ad040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:47:56.160 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.137+0000 7f52a412fb80 1 --2- 172.21.15.39:0/238443287 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555bd58b32d0 0x555bd58ad040 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f52800c9ea0 tx=0x7f5280099270 comp rx=0 tx=0).stop 2023-07-21T18:47:56.160 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.137+0000 7f52a412fb80 1 -- 172.21.15.39:0/238443287 shutdown_connections 2023-07-21T18:47:56.160 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.138+0000 7f52a412fb80 1 --2- 172.21.15.39:0/238443287 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555bd58ad580 0x555bd58ad9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:56.160 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.138+0000 7f52a412fb80 1 --2- 172.21.15.39:0/238443287 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bd57096b0 0x555bd58b2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:56.160 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.138+0000 7f52a412fb80 1 --2- 172.21.15.39:0/238443287 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555bd58b32d0 0x555bd58ad040 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:56.160 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.138+0000 7f52a412fb80 1 -- 172.21.15.39:0/238443287 >> 172.21.15.39:0/238443287 conn(0x555bd58a2be0 msgr2=0x555bd58a7f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:47:56.160 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.138+0000 7f52a412fb80 1 -- 172.21.15.39:0/238443287 shutdown_connections 2023-07-21T18:47:56.160 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.138+0000 7f52a412fb80 1 -- 172.21.15.39:0/238443287 wait complete. 2023-07-21T18:47:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.138+0000 7f52a412fb80 1 -- start start 2023-07-21T18:47:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.138+0000 7f52a412fb80 1 Processor -- start 2023-07-21T18:47:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.138+0000 7f52a412fb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555bd57096b0 0x555bd58b6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.138+0000 7f52a412fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555bd58ad580 0x555bd58b6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.138+0000 7f52a412fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bd58b32d0 0x555bd58b7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.138+0000 7f52a412fb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bd579ea60 con 0x555bd57096b0 2023-07-21T18:47:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.138+0000 7f52a412fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bd579e760 con 0x555bd58ad580 2023-07-21T18:47:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.138+0000 7f52a412fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bd56828c0 con 0x555bd58b32d0 2023-07-21T18:47:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.139+0000 7f52a412fb80 10 client.0 _ll_register_callbacks cb 0x555bd579c700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:47:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.139+0000 7f5295d45700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555bd57096b0 0x555bd58b6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.139+0000 7f5295d45700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555bd57096b0 0x555bd58b6990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59964/0 (socket says 172.21.15.39:59964) 2023-07-21T18:47:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.139+0000 7f5295544700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555bd58ad580 0x555bd58b6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.139+0000 7f5295d45700 1 -- 172.21.15.39:0/491628869 learned_addr learned my addr 172.21.15.39:0/491628869 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:47:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.139+0000 7f5296546700 1 --2- 172.21.15.39:0/491628869 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bd58b32d0 0x555bd58b7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.139+0000 7f5295544700 1 -- 172.21.15.39:0/491628869 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bd58b32d0 msgr2=0x555bd58b7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:47:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.139+0000 7f5295544700 1 --2- 172.21.15.39:0/491628869 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bd58b32d0 0x555bd58b7410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.139+0000 7f5295544700 1 -- 172.21.15.39:0/491628869 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555bd57096b0 msgr2=0x555bd58b6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:47:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.139+0000 7f5295544700 1 --2- 172.21.15.39:0/491628869 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555bd57096b0 0x555bd58b6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.139+0000 7f5295544700 1 -- 172.21.15.39:0/491628869 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f529010d240 con 0x555bd58ad580 2023-07-21T18:47:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.139+0000 7f5295d45700 1 --2- 172.21.15.39:0/491628869 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555bd57096b0 0x555bd58b6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:47:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.139+0000 7f5295544700 1 --2- 172.21.15.39:0/491628869 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555bd58ad580 0x555bd58b6ed0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x555bd58ae020 tx=0x7f528000bdd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be36506e46de9bbe server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:47:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.139+0000 7f52797fa700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-21T18:47:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.139+0000 7f52797fa700 1 -- 172.21.15.39:0/491628869 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5280099820 con 0x555bd58ad580 2023-07-21T18:47:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.139+0000 7f52a412fb80 1 -- 172.21.15.39:0/491628869 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555bd5758490 con 0x555bd58ad580 2023-07-21T18:47:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.139+0000 7f52797fa700 1 -- 172.21.15.39:0/491628869 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f5280099a80 con 0x555bd58ad580 2023-07-21T18:47:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.140+0000 7f52797fa700 1 -- 172.21.15.39:0/491628869 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5280099c40 con 0x555bd58ad580 2023-07-21T18:47:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.140+0000 7f52a412fb80 10 client.26093 fetch_fsmap learned FSMap version 47 2023-07-21T18:47:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.140+0000 7f52a412fb80 1 -- 172.21.15.39:0/491628869 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x7f52901a2b40 con 0x555bd58ad580 2023-07-21T18:47:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.140+0000 7f52797fa700 1 -- 172.21.15.39:0/491628869 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52800c7070 con 0x555bd58ad580 2023-07-21T18:47:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.140+0000 7f52797fa700 1 -- 172.21.15.39:0/491628869 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 47) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f52800b9040 con 0x555bd58ad580 2023-07-21T18:47:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.140+0000 7f52a412fb80 10 client.26093 fetch_fsmap finished waiting for FSMap version 47 2023-07-21T18:47:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.140+0000 7f52a412fb80 10 client.26093 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:47:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.140+0000 7f52a412fb80 20 client.26093 populate_metadata read hostname 'smithi039' 2023-07-21T18:47:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.140+0000 7f52a412fb80 1 --2- 172.21.15.39:0/491628869 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x555bd58c6750 0x555bd58c8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.140+0000 7f52a412fb80 4 client.26093 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:47:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.140+0000 7f52a412fb80 1 -- 172.21.15.39:0/491628869 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555bd57f7e50 con 0x555bd58c6750 2023-07-21T18:47:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.140+0000 7f5295d45700 1 --2- 172.21.15.39:0/491628869 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x555bd58c6750 0x555bd58c8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:56.166 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.141+0000 7f52797fa700 10 client.26093 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:47:56.166 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.141+0000 7f5295d45700 1 --2- 172.21.15.39:0/491628869 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x555bd58c6750 0x555bd58c8b60 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6888f69dcdcb957f server_cookie=8828089cd7fce9f1 in_seq=0 out_seq=0 2023-07-21T18:47:56.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.141+0000 7f52797fa700 1 -- 172.21.15.39:0/491628869 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555bd57f7e50 con 0x555bd58c6750 2023-07-21T18:47:56.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.141+0000 7f52797fa700 10 client.26093 handle_command_reply: tid=0 2023-07-21T18:47:56.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.141+0000 7f52a412fb80 1 client.26093 shutdown 2023-07-21T18:47:56.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.142+0000 7f52a412fb80 1 -- 172.21.15.39:0/491628869 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555bd58ad580 msgr2=0x555bd58b6ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:47:56.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.142+0000 7f52a412fb80 1 --2- 172.21.15.39:0/491628869 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555bd58ad580 0x555bd58b6ed0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x555bd58ae020 tx=0x7f528000bdd0 comp rx=0 tx=0).stop 2023-07-21T18:47:56.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.142+0000 7f52a412fb80 1 -- 172.21.15.39:0/491628869 shutdown_connections 2023-07-21T18:47:56.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.142+0000 7f52a412fb80 1 --2- 172.21.15.39:0/491628869 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x555bd58c6750 0x555bd58c8b60 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:56.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.142+0000 7f52a412fb80 1 --2- 172.21.15.39:0/491628869 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bd58b32d0 0x555bd58b7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:56.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.142+0000 7f52a412fb80 1 --2- 172.21.15.39:0/491628869 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555bd57096b0 0x555bd58b6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:56.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.142+0000 7f52a412fb80 1 --2- 172.21.15.39:0/491628869 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555bd58ad580 0x555bd58b6ed0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:56.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.142+0000 7f52797fa700 0 client.26093 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:47:56.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.142+0000 7f52a412fb80 1 -- 172.21.15.39:0/491628869 >> 172.21.15.39:0/491628869 conn(0x555bd58a2be0 msgr2=0x555bd58a74e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:47:56.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.142+0000 7f52a412fb80 1 -- 172.21.15.39:0/491628869 shutdown_connections 2023-07-21T18:47:56.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.142+0000 7f52a412fb80 1 -- 172.21.15.39:0/491628869 wait complete. 2023-07-21T18:47:56.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.142+0000 7f52a412fb80 20 client.26093 trim_cache size 0 max 16384 2023-07-21T18:47:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.157+0000 7f52a412fb80 1 Processor -- start 2023-07-21T18:47:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.157+0000 7f52a412fb80 1 -- start start 2023-07-21T18:47:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.157+0000 7f52a412fb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555bd57096b0 0x555bd58ad040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.157+0000 7f52a412fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bd58ad580 0x555bd58bdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.157+0000 7f52a412fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555bd58b32d0 0x555bd58b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.157+0000 7f52a412fb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bd58b15c0 con 0x555bd57096b0 2023-07-21T18:47:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.157+0000 7f52a412fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bd58a2a80 con 0x555bd58b32d0 2023-07-21T18:47:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.157+0000 7f52a412fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bd57f7e50 con 0x555bd58ad580 2023-07-21T18:47:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.158+0000 7f5295d45700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555bd57096b0 0x555bd58ad040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.158+0000 7f5295d45700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555bd57096b0 0x555bd58ad040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59972/0 (socket says 172.21.15.39:59972) 2023-07-21T18:47:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.158+0000 7f5295d45700 1 -- 172.21.15.39:0/2492534630 learned_addr learned my addr 172.21.15.39:0/2492534630 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:47:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.158+0000 7f5296546700 1 --2- 172.21.15.39:0/2492534630 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555bd58b32d0 0x555bd58b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.158+0000 7f5295544700 1 --2- 172.21.15.39:0/2492534630 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bd58ad580 0x555bd58bdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.158+0000 7f5295d45700 1 -- 172.21.15.39:0/2492534630 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bd58ad580 msgr2=0x555bd58bdbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:47:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.158+0000 7f5295d45700 1 --2- 172.21.15.39:0/2492534630 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bd58ad580 0x555bd58bdbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:56.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.158+0000 7f5295d45700 1 -- 172.21.15.39:0/2492534630 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555bd58b32d0 msgr2=0x555bd58b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:47:56.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.158+0000 7f5295d45700 1 --2- 172.21.15.39:0/2492534630 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555bd58b32d0 0x555bd58b23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:56.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.158+0000 7f5295d45700 1 -- 172.21.15.39:0/2492534630 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555bd5615c00 con 0x555bd57096b0 2023-07-21T18:47:56.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.158+0000 7f5295d45700 1 --2- 172.21.15.39:0/2492534630 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555bd57096b0 0x555bd58ad040 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x555bd58c9400 tx=0x7f528c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eba87c03d62da905 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:47:56.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.158+0000 7f5284ff9700 1 -- 172.21.15.39:0/2492534630 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528c027040 con 0x555bd57096b0 2023-07-21T18:47:56.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.158+0000 7f5284ff9700 1 -- 172.21.15.39:0/2492534630 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f528c0294c0 con 0x555bd57096b0 2023-07-21T18:47:56.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.158+0000 7f5284ff9700 1 -- 172.21.15.39:0/2492534630 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528c012450 con 0x555bd57096b0 2023-07-21T18:47:56.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.159+0000 7f52a412fb80 1 -- 172.21.15.39:0/2492534630 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555bd57096b0 msgr2=0x555bd58ad040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:47:56.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.159+0000 7f52a412fb80 1 --2- 172.21.15.39:0/2492534630 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555bd57096b0 0x555bd58ad040 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x555bd58c9400 tx=0x7f528c017000 comp rx=0 tx=0).stop 2023-07-21T18:47:56.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.159+0000 7f52a412fb80 1 -- 172.21.15.39:0/2492534630 shutdown_connections 2023-07-21T18:47:56.180 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.159+0000 7f52a412fb80 1 --2- 172.21.15.39:0/2492534630 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555bd58b32d0 0x555bd58b23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:56.180 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.159+0000 7f52a412fb80 1 --2- 172.21.15.39:0/2492534630 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555bd57096b0 0x555bd58ad040 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:56.180 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.159+0000 7f52a412fb80 1 --2- 172.21.15.39:0/2492534630 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bd58ad580 0x555bd58bdbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:56.180 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.159+0000 7f52a412fb80 1 -- 172.21.15.39:0/2492534630 >> 172.21.15.39:0/2492534630 conn(0x555bd58a2be0 msgr2=0x555bd58b72f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:47:56.180 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.159+0000 7f52a412fb80 1 -- 172.21.15.39:0/2492534630 shutdown_connections 2023-07-21T18:47:56.180 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.159+0000 7f52a412fb80 1 -- 172.21.15.39:0/2492534630 wait complete. 2023-07-21T18:47:56.180 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.159+0000 7f52a412fb80 1 -- start start 2023-07-21T18:47:56.180 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.160+0000 7f52a412fb80 1 Processor -- start 2023-07-21T18:47:56.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.160+0000 7f52a412fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bd57096b0 0x555bd590e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:56.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.160+0000 7f52a412fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555bd58ad580 0x555bd590ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:56.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.160+0000 7f52a412fb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555bd58b32d0 0x555bd59195e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:56.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.160+0000 7f52a412fb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bd5919b20 con 0x555bd58b32d0 2023-07-21T18:47:56.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.160+0000 7f52a412fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bd5919c60 con 0x555bd58ad580 2023-07-21T18:47:56.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.160+0000 7f52a412fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bd5919e20 con 0x555bd57096b0 2023-07-21T18:47:56.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.160+0000 7f52a412fb80 10 client.0 _ll_register_callbacks cb 0x7f528c005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:47:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.160+0000 7f5296546700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555bd58b32d0 0x555bd59195e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.160+0000 7f5295d45700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bd57096b0 0x555bd590e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.160+0000 7f5296546700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555bd58b32d0 0x555bd59195e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59982/0 (socket says 172.21.15.39:59982) 2023-07-21T18:47:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.160+0000 7f5295544700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555bd58ad580 0x555bd590ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.160+0000 7f5296546700 1 -- 172.21.15.39:0/2493242798 learned_addr learned my addr 172.21.15.39:0/2493242798 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:47:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.161+0000 7f5295d45700 1 -- 172.21.15.39:0/2493242798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555bd58ad580 msgr2=0x555bd590ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:47:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.161+0000 7f5295d45700 1 --2- 172.21.15.39:0/2493242798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555bd58ad580 0x555bd590ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.161+0000 7f5295d45700 1 -- 172.21.15.39:0/2493242798 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555bd58b32d0 msgr2=0x555bd59195e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:47:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.161+0000 7f5295d45700 1 --2- 172.21.15.39:0/2493242798 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555bd58b32d0 0x555bd59195e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.161+0000 7f5295d45700 1 -- 172.21.15.39:0/2493242798 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f528c011c60 con 0x555bd57096b0 2023-07-21T18:47:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.161+0000 7f5295d45700 1 --2- 172.21.15.39:0/2493242798 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bd57096b0 0x555bd590e750 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f528c0046e0 tx=0x7f528c0057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb13783a0a293bd9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:47:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.161+0000 7f52797fa700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2023-07-21T18:47:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.161+0000 7f52797fa700 1 -- 172.21.15.39:0/2493242798 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528c027050 con 0x555bd57096b0 2023-07-21T18:47:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.161+0000 7f52a412fb80 1 -- 172.21.15.39:0/2493242798 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f528009d850 con 0x555bd57096b0 2023-07-21T18:47:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.161+0000 7f52797fa700 1 -- 172.21.15.39:0/2493242798 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f528c018070 con 0x555bd57096b0 2023-07-21T18:47:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.161+0000 7f52797fa700 1 -- 172.21.15.39:0/2493242798 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f528c029db0 con 0x555bd57096b0 2023-07-21T18:47:56.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.161+0000 7f52a412fb80 10 client.26092 fetch_fsmap learned FSMap version 47 2023-07-21T18:47:56.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.161+0000 7f52a412fb80 1 -- 172.21.15.39:0/2493242798 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x555bd56531f0 con 0x555bd57096b0 2023-07-21T18:47:56.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.162+0000 7f52797fa700 1 -- 172.21.15.39:0/2493242798 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f528009d850 con 0x555bd57096b0 2023-07-21T18:47:56.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.162+0000 7f52797fa700 1 -- 172.21.15.39:0/2493242798 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 47) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f528c033a90 con 0x555bd57096b0 2023-07-21T18:47:56.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.162+0000 7f52a412fb80 10 client.26092 fetch_fsmap finished waiting for FSMap version 47 2023-07-21T18:47:56.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.162+0000 7f52a412fb80 10 client.26092 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:47:56.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.162+0000 7f52a412fb80 20 client.26092 populate_metadata read hostname 'smithi039' 2023-07-21T18:47:56.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.162+0000 7f52a412fb80 1 --2- 172.21.15.39:0/2493242798 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x555bd591b550 0x555bd591d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:47:56.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.162+0000 7f52a412fb80 4 client.26092 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:47:56.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.162+0000 7f52a412fb80 1 -- 172.21.15.39:0/2493242798 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555bd57b9ff0 con 0x555bd591b550 2023-07-21T18:47:56.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.162+0000 7f5295544700 1 --2- 172.21.15.39:0/2493242798 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x555bd591b550 0x555bd591d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:47:56.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.163+0000 7f52797fa700 10 client.26092 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:47:56.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.163+0000 7f5295544700 1 --2- 172.21.15.39:0/2493242798 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x555bd591b550 0x555bd591d960 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad456bb19c2fb4bb server_cookie=20fc373099266aa7 in_seq=0 out_seq=0 2023-07-21T18:47:56.221 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.205+0000 7f52797fa700 1 -- 172.21.15.39:0/2493242798 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555bd57b9ff0 con 0x555bd591b550 2023-07-21T18:47:56.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.205+0000 7f52797fa700 10 client.26092 handle_command_reply: tid=0 2023-07-21T18:47:56.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.205+0000 7f52a412fb80 1 client.26092 shutdown 2023-07-21T18:47:56.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.206+0000 7f52a412fb80 1 -- 172.21.15.39:0/2493242798 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bd57096b0 msgr2=0x555bd590e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:47:56.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.206+0000 7f52a412fb80 1 --2- 172.21.15.39:0/2493242798 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bd57096b0 0x555bd590e750 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f528c0046e0 tx=0x7f528c0057f0 comp rx=0 tx=0).stop 2023-07-21T18:47:56.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.206+0000 7f52a412fb80 1 -- 172.21.15.39:0/2493242798 shutdown_connections 2023-07-21T18:47:56.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.206+0000 7f52a412fb80 1 --2- 172.21.15.39:0/2493242798 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x555bd591b550 0x555bd591d960 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.206+0000 7f52a412fb80 1 --2- 172.21.15.39:0/2493242798 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555bd58b32d0 0x555bd59195e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.206+0000 7f52a412fb80 1 --2- 172.21.15.39:0/2493242798 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bd57096b0 0x555bd590e750 secure :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f528c0046e0 tx=0x7f528c0057f0 comp rx=0 tx=0).stop 2023-07-21T18:47:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.206+0000 7f52797fa700 0 client.26092 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:47:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.206+0000 7f52a412fb80 1 --2- 172.21.15.39:0/2493242798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555bd58ad580 0x555bd590ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.206+0000 7f52a412fb80 1 -- 172.21.15.39:0/2493242798 >> 172.21.15.39:0/2493242798 conn(0x555bd58a2be0 msgr2=0x555bd58b23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:47:56.224 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.207+0000 7f52a412fb80 1 -- 172.21.15.39:0/2493242798 shutdown_connections 2023-07-21T18:47:56.224 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.207+0000 7f52a412fb80 1 -- 172.21.15.39:0/2493242798 wait complete. 2023-07-21T18:47:56.224 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.208+0000 7f52a412fb80 20 client.26092 trim_cache size 0 max 16384 2023-07-21T18:47:56.224 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:47:56.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:47:56.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:47:56.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:47:56.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.209+0000 7f5297fa9700 1 -- 172.21.15.39:0/145710524 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f527c0a1340 msgr2=0x7f527c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:47:56.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.209+0000 7f5297fa9700 1 --2- 172.21.15.39:0/145710524 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f527c0a1340 0x7f527c0a37f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f528c009cb0 tx=0x7f528c009260 comp rx=0 tx=0).stop 2023-07-21T18:47:56.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.209+0000 7f5297fa9700 1 -- 172.21.15.39:0/145710524 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5290100500 msgr2=0x7f529019c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:47:56.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.209+0000 7f5297fa9700 1 --2- 172.21.15.39:0/145710524 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5290100500 0x7f529019c090 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f528000b550 tx=0x7f528000b860 comp rx=0 tx=0).stop 2023-07-21T18:47:56.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.209+0000 7f5297fa9700 1 -- 172.21.15.39:0/145710524 shutdown_connections 2023-07-21T18:47:56.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.209+0000 7f5297fa9700 1 --2- 172.21.15.39:0/145710524 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f527c0a1340 0x7f527c0a37f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:56.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.209+0000 7f5297fa9700 1 --2- 172.21.15.39:0/145710524 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5290105b80 0x7f52901959b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:56.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.209+0000 7f5297fa9700 1 --2- 172.21.15.39:0/145710524 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52900ffb50 0x7f529019bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:56.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.209+0000 7f5297fa9700 1 --2- 172.21.15.39:0/145710524 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5290100500 0x7f529019c090 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:47:56.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.209+0000 7f5297fa9700 1 -- 172.21.15.39:0/145710524 >> 172.21.15.39:0/145710524 conn(0x7f52900fb650 msgr2=0x7f52901036e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:47:56.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.209+0000 7f5297fa9700 1 -- 172.21.15.39:0/145710524 shutdown_connections 2023-07-21T18:47:56.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:47:56.209+0000 7f5297fa9700 1 -- 172.21.15.39:0/145710524 wait complete. 2023-07-21T18:47:56.274 INFO:tasks.workunit.client.0.smithi039.stdout: 1 384362 6.59 MB/sec execute 189 sec latency 47.374 ms 2023-07-21T18:47:56.612 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:47:56 smithi193 ceph-mon[98204]: pgmap v189: 97 pgs: 97 active+clean; 142 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 227 op/s 2023-07-21T18:47:56.612 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:47:56 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:47:56.613 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:47:56 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:47:56.613 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:47:56 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:47:56.631 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:47:56 smithi155 ceph-mon[97934]: pgmap v189: 97 pgs: 97 active+clean; 142 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 227 op/s 2023-07-21T18:47:56.631 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:47:56 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:47:56.631 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:47:56 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:47:56.631 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:47:56 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:47:56.680 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:47:56 smithi039 ceph-mon[96489]: pgmap v189: 97 pgs: 97 active+clean; 142 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 227 op/s 2023-07-21T18:47:56.680 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:47:56 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:47:56.680 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:47:56 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:47:56.680 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:47:56 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:47:57.274 INFO:tasks.workunit.client.0.smithi039.stdout: 1 386303 6.61 MB/sec execute 190 sec latency 59.959 ms 2023-07-21T18:47:58.274 INFO:tasks.workunit.client.0.smithi039.stdout: 1 387296 6.61 MB/sec execute 191 sec latency 105.991 ms 2023-07-21T18:47:58.612 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:47:58 smithi193 ceph-mon[98204]: pgmap v190: 97 pgs: 97 active+clean; 147 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.6 MiB/s wr, 328 op/s 2023-07-21T18:47:58.631 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:47:58 smithi155 ceph-mon[97934]: pgmap v190: 97 pgs: 97 active+clean; 147 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.6 MiB/s wr, 328 op/s 2023-07-21T18:47:58.680 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:47:58 smithi039 ceph-mon[96489]: pgmap v190: 97 pgs: 97 active+clean; 147 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.6 MiB/s wr, 328 op/s 2023-07-21T18:47:59.274 INFO:tasks.workunit.client.0.smithi039.stdout: 1 388521 6.60 MB/sec execute 192 sec latency 155.614 ms 2023-07-21T18:48:00.274 INFO:tasks.workunit.client.0.smithi039.stdout: 1 390433 6.64 MB/sec execute 193 sec latency 32.361 ms 2023-07-21T18:48:00.612 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:48:00 smithi193 ceph-mon[98204]: pgmap v191: 97 pgs: 97 active+clean; 147 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 209 op/s 2023-07-21T18:48:00.631 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:48:00 smithi155 ceph-mon[97934]: pgmap v191: 97 pgs: 97 active+clean; 147 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 209 op/s 2023-07-21T18:48:00.680 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:48:00 smithi039 ceph-mon[96489]: pgmap v191: 97 pgs: 97 active+clean; 147 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 209 op/s 2023-07-21T18:48:01.274 INFO:tasks.workunit.client.0.smithi039.stdout: 1 390433 6.60 MB/sec execute 194 sec latency 1032.407 ms 2023-07-21T18:48:02.274 INFO:tasks.workunit.client.0.smithi039.stdout: 1 390462 6.57 MB/sec execute 195 sec latency 1470.327 ms 2023-07-21T18:48:02.612 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:48:02 smithi193 ceph-mon[98204]: pgmap v192: 97 pgs: 97 active+clean; 149 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.4 MiB/s wr, 319 op/s 2023-07-21T18:48:02.631 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:48:02 smithi155 ceph-mon[97934]: pgmap v192: 97 pgs: 97 active+clean; 149 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.4 MiB/s wr, 319 op/s 2023-07-21T18:48:02.680 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:48:02 smithi039 ceph-mon[96489]: pgmap v192: 97 pgs: 97 active+clean; 149 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.4 MiB/s wr, 319 op/s 2023-07-21T18:48:03.274 INFO:tasks.workunit.client.0.smithi039.stdout: 1 390462 6.53 MB/sec execute 196 sec latency 1492.791 ms 2023-07-21T18:48:04.274 INFO:tasks.workunit.client.0.smithi039.stdout: 1 390462 6.50 MB/sec execute 197 sec latency 2492.828 ms 2023-07-21T18:48:04.612 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:48:04 smithi193 ceph-mon[98204]: pgmap v193: 97 pgs: 97 active+clean; 149 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 986 KiB/s rd, 2.2 MiB/s wr, 211 op/s 2023-07-21T18:48:04.631 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:48:04 smithi155 ceph-mon[97934]: pgmap v193: 97 pgs: 97 active+clean; 149 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 986 KiB/s rd, 2.2 MiB/s wr, 211 op/s 2023-07-21T18:48:04.680 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:48:04 smithi039 ceph-mon[96489]: pgmap v193: 97 pgs: 97 active+clean; 149 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 986 KiB/s rd, 2.2 MiB/s wr, 211 op/s 2023-07-21T18:48:05.274 INFO:tasks.workunit.client.0.smithi039.stdout: 1 390462 6.47 MB/sec execute 198 sec latency 3492.936 ms 2023-07-21T18:48:06.274 INFO:tasks.workunit.client.0.smithi039.stdout: 1 390462 6.44 MB/sec execute 199 sec latency 4493.049 ms 2023-07-21T18:48:06.612 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:48:06 smithi193 ceph-mon[98204]: pgmap v194: 97 pgs: 97 active+clean; 149 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 986 KiB/s rd, 2.2 MiB/s wr, 211 op/s 2023-07-21T18:48:06.631 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:48:06 smithi155 ceph-mon[97934]: pgmap v194: 97 pgs: 97 active+clean; 149 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 986 KiB/s rd, 2.2 MiB/s wr, 211 op/s 2023-07-21T18:48:06.679 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:48:06 smithi039 ceph-mon[96489]: pgmap v194: 97 pgs: 97 active+clean; 149 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 986 KiB/s rd, 2.2 MiB/s wr, 211 op/s 2023-07-21T18:48:07.274 INFO:tasks.workunit.client.0.smithi039.stdout: 1 390825 6.41 MB/sec execute 200 sec latency 4935.314 ms 2023-07-21T18:48:07.612 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:48:07 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:48:07.631 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:48:07 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:48:07.680 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:48:07 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:48:08.275 INFO:tasks.workunit.client.0.smithi039.stdout: 1 390825 6.37 MB/sec execute 201 sec latency 1076.240 ms 2023-07-21T18:48:08.612 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:48:08 smithi193 ceph-mon[98204]: pgmap v195: 97 pgs: 97 active+clean; 149 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 256 op/s 2023-07-21T18:48:08.631 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:48:08 smithi155 ceph-mon[97934]: pgmap v195: 97 pgs: 97 active+clean; 149 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 256 op/s 2023-07-21T18:48:08.680 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:48:08 smithi039 ceph-mon[96489]: pgmap v195: 97 pgs: 97 active+clean; 149 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 256 op/s 2023-07-21T18:48:09.274 INFO:tasks.workunit.client.0.smithi039.stdout: 1 390825 6.34 MB/sec execute 202 sec latency 2076.298 ms 2023-07-21T18:48:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.377+0000 7efc7fac9700 1 -- 172.21.15.39:0/3149462820 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc7806d3a0 msgr2=0x7efc7811b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:48:09.392 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.377+0000 7efc7fac9700 1 --2- 172.21.15.39:0/3149462820 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc7806d3a0 0x7efc7811b470 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7efc680098f0 tx=0x7efc68009c00 comp rx=0 tx=0).stop 2023-07-21T18:48:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.379+0000 7efc7fac9700 1 -- 172.21.15.39:0/3149462820 shutdown_connections 2023-07-21T18:48:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.379+0000 7efc7fac9700 1 --2- 172.21.15.39:0/3149462820 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc7806d3a0 0x7efc7811b470 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.379+0000 7efc7fac9700 1 --2- 172.21.15.39:0/3149462820 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc7810e2a0 0x7efc7810e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.379+0000 7efc7fac9700 1 --2- 172.21.15.39:0/3149462820 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc7806c910 0x7efc7806cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.379+0000 7efc7fac9700 1 -- 172.21.15.39:0/3149462820 >> 172.21.15.39:0/3149462820 conn(0x7efc7806bd70 msgr2=0x7efc78070f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:48:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.380+0000 7efc7fac9700 1 -- 172.21.15.39:0/3149462820 shutdown_connections 2023-07-21T18:48:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.380+0000 7efc7fac9700 1 -- 172.21.15.39:0/3149462820 wait complete. 2023-07-21T18:48:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.381+0000 7efc7fac9700 1 Processor -- start 2023-07-21T18:48:09.396 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.381+0000 7efc7fac9700 1 -- start start 2023-07-21T18:48:09.396 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.381+0000 7efc7fac9700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc7806c910 0x7efc781aaac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:09.397 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.381+0000 7efc7fac9700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc7806d3a0 0x7efc781ab000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:09.397 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.381+0000 7efc7fac9700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc7810e2a0 0x7efc781a4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:09.397 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.381+0000 7efc7fac9700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc781a5060 con 0x7efc7810e2a0 2023-07-21T18:48:09.397 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.382+0000 7efc7fac9700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc781a51d0 con 0x7efc7806c910 2023-07-21T18:48:09.397 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.382+0000 7efc7fac9700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc781a5390 con 0x7efc7806d3a0 2023-07-21T18:48:09.397 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.382+0000 7efc7d865700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc7806c910 0x7efc781aaac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:09.397 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.382+0000 7efc7d064700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc7806d3a0 0x7efc781ab000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:09.397 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.382+0000 7efc7d064700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc7806d3a0 0x7efc781ab000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:50314/0 (socket says 172.21.15.39:50314) 2023-07-21T18:48:09.398 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.382+0000 7efc7d064700 1 -- 172.21.15.39:0/1198597998 learned_addr learned my addr 172.21.15.39:0/1198597998 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:48:09.398 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.382+0000 7efc7e066700 1 --2- 172.21.15.39:0/1198597998 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc7810e2a0 0x7efc781a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:09.398 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.382+0000 7efc7d865700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc7806c910 0x7efc781aaac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:44356/0 (socket says 172.21.15.39:44356) 2023-07-21T18:48:09.398 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.382+0000 7efc7e066700 1 -- 172.21.15.39:0/1198597998 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc7806d3a0 msgr2=0x7efc781ab000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:48:09.399 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.382+0000 7efc7e066700 1 --2- 172.21.15.39:0/1198597998 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc7806d3a0 0x7efc781ab000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:09.399 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.382+0000 7efc7e066700 1 -- 172.21.15.39:0/1198597998 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc7806c910 msgr2=0x7efc781aaac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:48:09.399 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.382+0000 7efc7e066700 1 --2- 172.21.15.39:0/1198597998 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc7806c910 0x7efc781aaac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:09.399 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.383+0000 7efc7e066700 1 -- 172.21.15.39:0/1198597998 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc68009520 con 0x7efc7810e2a0 2023-07-21T18:48:09.399 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.383+0000 7efc7e066700 1 --2- 172.21.15.39:0/1198597998 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc7810e2a0 0x7efc781a4890 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7efc68009810 tx=0x7efc6800fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:48:09.399 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.383+0000 7efc6effd700 1 -- 172.21.15.39:0/1198597998 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc6801c070 con 0x7efc7810e2a0 2023-07-21T18:48:09.399 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.384+0000 7efc7fac9700 1 -- 172.21.15.39:0/1198597998 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc781a5610 con 0x7efc7810e2a0 2023-07-21T18:48:09.400 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.384+0000 7efc7fac9700 1 -- 172.21.15.39:0/1198597998 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc781b1a40 con 0x7efc7810e2a0 2023-07-21T18:48:09.400 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.384+0000 7efc6effd700 1 -- 172.21.15.39:0/1198597998 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc6800baa0 con 0x7efc7810e2a0 2023-07-21T18:48:09.400 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.384+0000 7efc6effd700 1 -- 172.21.15.39:0/1198597998 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc68004930 con 0x7efc7810e2a0 2023-07-21T18:48:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.388+0000 7efc6effd700 1 -- 172.21.15.39:0/1198597998 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7efc68020070 con 0x7efc7810e2a0 2023-07-21T18:48:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.389+0000 7efc6effd700 1 --2- 172.21.15.39:0/1198597998 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7efc640a1400 0x7efc640a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.390+0000 7efc6effd700 1 -- 172.21.15.39:0/1198597998 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8571+0+0 (secure 0 0 0) 0x7efc680cd820 con 0x7efc7810e2a0 2023-07-21T18:48:09.405 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.390+0000 7efc7d865700 1 --2- 172.21.15.39:0/1198597998 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7efc640a1400 0x7efc640a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:09.405 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.390+0000 7efc8bc4fb80 1 Processor -- start 2023-07-21T18:48:09.406 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.391+0000 7efc8bc4fb80 1 -- start start 2023-07-21T18:48:09.406 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.391+0000 7efc8bc4fb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b58e9656b0 0x55b58eb0ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:09.406 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.391+0000 7efc8bc4fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b58eb0f2d0 0x55b58eb09040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:09.406 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.391+0000 7efc8bc4fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b58eb09580 0x55b58eb099d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:09.406 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.391+0000 7efc8bc4fb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b58ea0ec50 con 0x55b58e9656b0 2023-07-21T18:48:09.406 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.391+0000 7efc8bc4fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b58ea619e0 con 0x55b58eb09580 2023-07-21T18:48:09.406 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.391+0000 7efc8bc4fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b58ea5ae70 con 0x55b58eb0f2d0 2023-07-21T18:48:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.392+0000 7efc7e066700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b58eb09580 0x55b58eb099d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.392+0000 7efc7e066700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b58eb09580 0x55b58eb099d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:44366/0 (socket says 172.21.15.39:44366) 2023-07-21T18:48:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.392+0000 7efc7e066700 1 -- 172.21.15.39:0/859457232 learned_addr learned my addr 172.21.15.39:0/859457232 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:48:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.393+0000 7efc7d064700 1 --2- 172.21.15.39:0/859457232 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b58eb0f2d0 0x55b58eb09040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.394+0000 7efc7e066700 1 -- 172.21.15.39:0/859457232 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b58eb0f2d0 msgr2=0x55b58eb09040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:48:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.394+0000 7efc7e066700 1 --2- 172.21.15.39:0/859457232 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b58eb0f2d0 0x55b58eb09040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.394+0000 7efc7e066700 1 -- 172.21.15.39:0/859457232 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b58e9656b0 msgr2=0x55b58eb0ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:48:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.394+0000 7efc7e066700 1 --2- 172.21.15.39:0/859457232 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b58e9656b0 0x55b58eb0ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.394+0000 7efc7e066700 1 -- 172.21.15.39:0/859457232 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b58e888d80 con 0x55b58eb09580 2023-07-21T18:48:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.394+0000 7efc7e066700 1 --2- 172.21.15.39:0/859457232 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b58eb09580 0x55b58eb099d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7efc680a26e0 tx=0x7efc6809a6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d2d6dc6ae4ef01b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:48:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.395+0000 7efc63fff700 1 -- 172.21.15.39:0/859457232 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc6809e880 con 0x55b58eb09580 2023-07-21T18:48:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.395+0000 7efc63fff700 1 -- 172.21.15.39:0/859457232 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc680a2c10 con 0x55b58eb09580 2023-07-21T18:48:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.395+0000 7efc63fff700 1 -- 172.21.15.39:0/859457232 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc6809aab0 con 0x55b58eb09580 2023-07-21T18:48:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.396+0000 7efc8bc4fb80 1 -- 172.21.15.39:0/859457232 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b58eb09580 msgr2=0x55b58eb099d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:48:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.396+0000 7efc7d865700 1 --2- 172.21.15.39:0/1198597998 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7efc640a1400 0x7efc640a38b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7efc74005950 tx=0x7efc740106b0 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:48:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.396+0000 7efc8bc4fb80 1 --2- 172.21.15.39:0/859457232 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b58eb09580 0x55b58eb099d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7efc680a26e0 tx=0x7efc6809a6a0 comp rx=0 tx=0).stop 2023-07-21T18:48:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.396+0000 7efc8bc4fb80 1 -- 172.21.15.39:0/859457232 shutdown_connections 2023-07-21T18:48:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.396+0000 7efc8bc4fb80 1 --2- 172.21.15.39:0/859457232 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b58eb09580 0x55b58eb099d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.396+0000 7efc8bc4fb80 1 --2- 172.21.15.39:0/859457232 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b58e9656b0 0x55b58eb0ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.396+0000 7efc8bc4fb80 1 --2- 172.21.15.39:0/859457232 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b58eb0f2d0 0x55b58eb09040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.396+0000 7efc8bc4fb80 1 -- 172.21.15.39:0/859457232 >> 172.21.15.39:0/859457232 conn(0x55b58eafebe0 msgr2=0x55b58eb03f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:48:09.412 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.398+0000 7efc8bc4fb80 1 -- 172.21.15.39:0/859457232 shutdown_connections 2023-07-21T18:48:09.412 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.398+0000 7efc8bc4fb80 1 -- 172.21.15.39:0/859457232 wait complete. 2023-07-21T18:48:09.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.398+0000 7efc8bc4fb80 1 -- start start 2023-07-21T18:48:09.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.399+0000 7efc8bc4fb80 1 Processor -- start 2023-07-21T18:48:09.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.399+0000 7efc8bc4fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b58e9656b0 0x55b58eb184c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:09.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.399+0000 7efc8bc4fb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b58eb09580 0x55b58eb18a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:09.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.399+0000 7efc8bc4fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b58eb127c0 0x55b58eb12c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:09.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.399+0000 7efc8bc4fb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b58e9faa60 con 0x55b58eb09580 2023-07-21T18:48:09.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.399+0000 7efc8bc4fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b58e9fa760 con 0x55b58eb127c0 2023-07-21T18:48:09.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.399+0000 7efc8bc4fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b58e8de8c0 con 0x55b58e9656b0 2023-07-21T18:48:09.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.399+0000 7efc8bc4fb80 10 client.0 _ll_register_callbacks cb 0x55b58e9f8700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:48:09.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.401+0000 7efc7e066700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b58eb127c0 0x55b58eb12c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:09.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.401+0000 7efc7d064700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b58eb09580 0x55b58eb18a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:09.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.401+0000 7efc7d064700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b58eb09580 0x55b58eb18a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60006/0 (socket says 172.21.15.39:60006) 2023-07-21T18:48:09.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.401+0000 7efc7d064700 1 -- 172.21.15.39:0/2549064769 learned_addr learned my addr 172.21.15.39:0/2549064769 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:48:09.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.402+0000 7efc7d064700 1 -- 172.21.15.39:0/2549064769 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b58e9656b0 msgr2=0x55b58eb184c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T18:48:09.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.402+0000 7efc7d064700 1 --2- 172.21.15.39:0/2549064769 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b58e9656b0 0x55b58eb184c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:09.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.402+0000 7efc7d064700 1 -- 172.21.15.39:0/2549064769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b58eb127c0 msgr2=0x55b58eb12c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:48:09.418 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.402+0000 7efc7d064700 1 --2- 172.21.15.39:0/2549064769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b58eb127c0 0x55b58eb12c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:09.418 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.402+0000 7efc7d064700 1 -- 172.21.15.39:0/2549064769 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc781b1a40 con 0x55b58eb09580 2023-07-21T18:48:09.418 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.403+0000 7efc7d064700 1 --2- 172.21.15.39:0/2549064769 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b58eb09580 0x55b58eb18a00 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7efc70009fd0 tx=0x7efc70007c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8d062e7cedcbdf3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:48:09.418 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.403+0000 7efc617fa700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:48:09.418 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.403+0000 7efc617fa700 1 -- 172.21.15.39:0/2549064769 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc7000e020 con 0x55b58eb09580 2023-07-21T18:48:09.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.403+0000 7efc617fa700 1 -- 172.21.15.39:0/2549064769 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8571+0+0 (secure 0 0 0) 0x7efc70013b00 con 0x55b58eb09580 2023-07-21T18:48:09.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.404+0000 7efc8bc4fb80 1 -- 172.21.15.39:0/2549064769 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b58e9b4490 con 0x55b58eb09580 2023-07-21T18:48:09.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.405+0000 7efc617fa700 1 -- 172.21.15.39:0/2549064769 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc70013cc0 con 0x55b58eb09580 2023-07-21T18:48:09.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.405+0000 7efc8bc4fb80 10 client.16620 fetch_fsmap learned FSMap version 48 2023-07-21T18:48:09.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.405+0000 7efc8bc4fb80 1 -- 172.21.15.39:0/2549064769 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x55b58e871c00 con 0x55b58eb09580 2023-07-21T18:48:09.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.405+0000 7efc617fa700 1 -- 172.21.15.39:0/2549064769 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc70025030 con 0x55b58eb09580 2023-07-21T18:48:09.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.406+0000 7efc617fa700 1 -- 172.21.15.39:0/2549064769 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 48) v1 ==== 7494+0+0 (secure 0 0 0) 0x7efc7001e020 con 0x55b58eb09580 2023-07-21T18:48:09.421 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.406+0000 7efc8bc4fb80 10 client.16620 fetch_fsmap finished waiting for FSMap version 48 2023-07-21T18:48:09.422 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.406+0000 7efc8bc4fb80 10 client.16620 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:48:09.422 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.406+0000 7efc8bc4fb80 20 client.16620 populate_metadata read hostname 'smithi039' 2023-07-21T18:48:09.422 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.406+0000 7efc8bc4fb80 1 --2- 172.21.15.39:0/2549064769 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55b58eb22f70 0x55b58eb25380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:09.422 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.406+0000 7efc8bc4fb80 4 client.16620 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:48:09.422 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.406+0000 7efc8bc4fb80 1 -- 172.21.15.39:0/2549064769 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b58ea53e50 con 0x55b58eb22f70 2023-07-21T18:48:09.422 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.406+0000 7efc7d865700 1 --2- 172.21.15.39:0/2549064769 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55b58eb22f70 0x55b58eb25380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:09.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.407+0000 7efc617fa700 10 client.16620 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:48:09.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.407+0000 7efc7d865700 1 --2- 172.21.15.39:0/2549064769 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55b58eb22f70 0x55b58eb25380 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74ecee058a4b5e45 server_cookie=c82f0975ef737bf0 in_seq=0 out_seq=0 2023-07-21T18:48:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.408+0000 7efc617fa700 1 -- 172.21.15.39:0/2549064769 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b58ea53e50 con 0x55b58eb22f70 2023-07-21T18:48:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.408+0000 7efc617fa700 10 client.16620 handle_command_reply: tid=0 2023-07-21T18:48:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.408+0000 7efc8bc4fb80 1 client.16620 shutdown 2023-07-21T18:48:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.409+0000 7efc8bc4fb80 1 -- 172.21.15.39:0/2549064769 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b58eb09580 msgr2=0x55b58eb18a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:48:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.409+0000 7efc8bc4fb80 1 --2- 172.21.15.39:0/2549064769 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b58eb09580 0x55b58eb18a00 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7efc70009fd0 tx=0x7efc70007c30 comp rx=0 tx=0).stop 2023-07-21T18:48:09.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.409+0000 7efc8bc4fb80 1 -- 172.21.15.39:0/2549064769 shutdown_connections 2023-07-21T18:48:09.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.409+0000 7efc8bc4fb80 1 --2- 172.21.15.39:0/2549064769 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55b58eb22f70 0x55b58eb25380 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:09.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.409+0000 7efc8bc4fb80 1 --2- 172.21.15.39:0/2549064769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b58eb127c0 0x55b58eb12c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:09.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.409+0000 7efc8bc4fb80 1 --2- 172.21.15.39:0/2549064769 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b58e9656b0 0x55b58eb184c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:09.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.409+0000 7efc8bc4fb80 1 --2- 172.21.15.39:0/2549064769 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b58eb09580 0x55b58eb18a00 secure :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7efc70009fd0 tx=0x7efc70007c30 comp rx=0 tx=0).stop 2023-07-21T18:48:09.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.409+0000 7efc8bc4fb80 1 -- 172.21.15.39:0/2549064769 >> 172.21.15.39:0/2549064769 conn(0x55b58eafebe0 msgr2=0x55b58eb034e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:48:09.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.409+0000 7efc617fa700 0 client.16620 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:48:09.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.409+0000 7efc8bc4fb80 1 -- 172.21.15.39:0/2549064769 shutdown_connections 2023-07-21T18:48:09.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.409+0000 7efc8bc4fb80 1 -- 172.21.15.39:0/2549064769 wait complete. 2023-07-21T18:48:09.428 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.414+0000 7efc8bc4fb80 20 client.16620 trim_cache size 0 max 16384 2023-07-21T18:48:09.458 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.443+0000 7efc8bc4fb80 1 Processor -- start 2023-07-21T18:48:09.459 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.444+0000 7efc8bc4fb80 1 -- start start 2023-07-21T18:48:09.459 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.444+0000 7efc8bc4fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b58e9656b0 0x55b58eb11f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:09.459 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.444+0000 7efc8bc4fb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b58eb127c0 0x55b58eb100c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:09.459 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.444+0000 7efc8bc4fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b58eb03c60 0x55b58eb04090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:09.459 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.444+0000 7efc8bc4fb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b58eafbbb0 con 0x55b58eb127c0 2023-07-21T18:48:09.459 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.444+0000 7efc8bc4fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b58eb005f0 con 0x55b58eb03c60 2023-07-21T18:48:09.460 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.444+0000 7efc8bc4fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b58eb04fa0 con 0x55b58e9656b0 2023-07-21T18:48:09.460 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.445+0000 7efc7d865700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b58e9656b0 0x55b58eb11f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:09.460 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.445+0000 7efc7d064700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b58eb127c0 0x55b58eb100c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:09.460 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.445+0000 7efc7d865700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b58e9656b0 0x55b58eb11f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:50334/0 (socket says 172.21.15.39:50334) 2023-07-21T18:48:09.460 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.445+0000 7efc7d865700 1 -- 172.21.15.39:0/1641953990 learned_addr learned my addr 172.21.15.39:0/1641953990 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:48:09.461 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.445+0000 7efc7e066700 1 --2- 172.21.15.39:0/1641953990 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b58eb03c60 0x55b58eb04090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:09.461 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.446+0000 7efc7e066700 1 -- 172.21.15.39:0/1641953990 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b58e9656b0 msgr2=0x55b58eb11f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:48:09.461 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.446+0000 7efc7e066700 1 --2- 172.21.15.39:0/1641953990 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b58e9656b0 0x55b58eb11f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:09.461 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.446+0000 7efc7e066700 1 -- 172.21.15.39:0/1641953990 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b58eb127c0 msgr2=0x55b58eb100c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:48:09.461 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.446+0000 7efc7e066700 1 --2- 172.21.15.39:0/1641953990 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b58eb127c0 0x55b58eb100c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:09.461 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.446+0000 7efc7e066700 1 -- 172.21.15.39:0/1641953990 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b58eb1f460 con 0x55b58eb03c60 2023-07-21T18:48:09.462 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.447+0000 7efc7e066700 1 --2- 172.21.15.39:0/1641953990 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b58eb03c60 0x55b58eb04090 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7efc6809a7e0 tx=0x7efc680a0040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a60a26dab0d3c22a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:48:09.462 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.448+0000 7efc6cdac700 1 -- 172.21.15.39:0/1641953990 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc6809e880 con 0x55b58eb03c60 2023-07-21T18:48:09.462 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.448+0000 7efc6cdac700 1 -- 172.21.15.39:0/1641953990 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc680c9070 con 0x55b58eb03c60 2023-07-21T18:48:09.462 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.448+0000 7efc6cdac700 1 -- 172.21.15.39:0/1641953990 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc68099d40 con 0x55b58eb03c60 2023-07-21T18:48:09.463 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.449+0000 7efc8bc4fb80 1 -- 172.21.15.39:0/1641953990 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b58eb03c60 msgr2=0x55b58eb04090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:48:09.463 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.449+0000 7efc8bc4fb80 1 --2- 172.21.15.39:0/1641953990 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b58eb03c60 0x55b58eb04090 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7efc6809a7e0 tx=0x7efc680a0040 comp rx=0 tx=0).stop 2023-07-21T18:48:09.467 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.453+0000 7efc8bc4fb80 1 -- 172.21.15.39:0/1641953990 shutdown_connections 2023-07-21T18:48:09.467 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.453+0000 7efc8bc4fb80 1 --2- 172.21.15.39:0/1641953990 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b58eb03c60 0x55b58eb04090 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:09.468 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.453+0000 7efc8bc4fb80 1 --2- 172.21.15.39:0/1641953990 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b58e9656b0 0x55b58eb11f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:09.468 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.453+0000 7efc8bc4fb80 1 --2- 172.21.15.39:0/1641953990 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b58eb127c0 0x55b58eb100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:09.468 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.453+0000 7efc8bc4fb80 1 -- 172.21.15.39:0/1641953990 >> 172.21.15.39:0/1641953990 conn(0x55b58eafebe0 msgr2=0x55b58eb0f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:48:09.470 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.453+0000 7efc8bc4fb80 1 -- 172.21.15.39:0/1641953990 shutdown_connections 2023-07-21T18:48:09.470 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.454+0000 7efc8bc4fb80 1 -- 172.21.15.39:0/1641953990 wait complete. 2023-07-21T18:48:09.470 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.454+0000 7efc8bc4fb80 1 -- start start 2023-07-21T18:48:09.471 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.454+0000 7efc8bc4fb80 1 Processor -- start 2023-07-21T18:48:09.471 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.455+0000 7efc8bc4fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b58e9656b0 0x55b58eb16070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:09.471 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.455+0000 7efc8bc4fb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b58eb03c60 0x55b58eb165b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:09.471 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.455+0000 7efc8bc4fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b58eb127c0 0x55b58eb16af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:09.472 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.455+0000 7efc8bc4fb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b58eb75e30 con 0x55b58eb03c60 2023-07-21T18:48:09.472 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.455+0000 7efc8bc4fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b58eb75f70 con 0x55b58e9656b0 2023-07-21T18:48:09.472 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.455+0000 7efc8bc4fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b58eb870c0 con 0x55b58eb127c0 2023-07-21T18:48:09.473 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.455+0000 7efc8bc4fb80 10 client.0 _ll_register_callbacks cb 0x7efc700075c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:48:09.473 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.455+0000 7efc7d064700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b58eb03c60 0x55b58eb165b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.455+0000 7efc7d064700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b58eb03c60 0x55b58eb165b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60022/0 (socket says 172.21.15.39:60022) 2023-07-21T18:48:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.455+0000 7efc7d064700 1 -- 172.21.15.39:0/664196375 learned_addr learned my addr 172.21.15.39:0/664196375 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:48:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.455+0000 7efc7e066700 1 --2- 172.21.15.39:0/664196375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b58eb127c0 0x55b58eb16af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.456+0000 7efc7d064700 1 -- 172.21.15.39:0/664196375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b58eb127c0 msgr2=0x55b58eb16af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:48:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.456+0000 7efc7d064700 1 --2- 172.21.15.39:0/664196375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b58eb127c0 0x55b58eb16af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.456+0000 7efc7d064700 1 -- 172.21.15.39:0/664196375 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b58e9656b0 msgr2=0x55b58eb16070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:48:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.456+0000 7efc7d064700 1 --2- 172.21.15.39:0/664196375 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b58e9656b0 0x55b58eb16070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.456+0000 7efc7d064700 1 -- 172.21.15.39:0/664196375 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b58e888d80 con 0x55b58eb03c60 2023-07-21T18:48:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.456+0000 7efc7d064700 1 --2- 172.21.15.39:0/664196375 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b58eb03c60 0x55b58eb165b0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7efc70018040 tx=0x7efc70020a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4124d200ac504dc server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:48:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.457+0000 7efc617fa700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:48:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.457+0000 7efc617fa700 1 -- 172.21.15.39:0/664196375 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc7002e070 con 0x55b58eb03c60 2023-07-21T18:48:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.457+0000 7efc617fa700 1 -- 172.21.15.39:0/664196375 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8571+0+0 (secure 0 0 0) 0x7efc70003a00 con 0x55b58eb03c60 2023-07-21T18:48:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.457+0000 7efc8bc4fb80 1 -- 172.21.15.39:0/664196375 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efc680a2c10 con 0x55b58eb03c60 2023-07-21T18:48:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.458+0000 7efc617fa700 1 -- 172.21.15.39:0/664196375 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc70003be0 con 0x55b58eb03c60 2023-07-21T18:48:09.476 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.461+0000 7efc8bc4fb80 10 client.16623 fetch_fsmap learned FSMap version 48 2023-07-21T18:48:09.476 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.461+0000 7efc8bc4fb80 1 -- 172.21.15.39:0/664196375 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x55b58e8af1f0 con 0x55b58eb03c60 2023-07-21T18:48:09.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.463+0000 7efc617fa700 1 -- 172.21.15.39:0/664196375 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc680a2c10 con 0x55b58eb03c60 2023-07-21T18:48:09.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.463+0000 7efc617fa700 1 -- 172.21.15.39:0/664196375 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 48) v1 ==== 7494+0+0 (secure 0 0 0) 0x7efc70014040 con 0x55b58eb03c60 2023-07-21T18:48:09.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.463+0000 7efc8bc4fb80 10 client.16623 fetch_fsmap finished waiting for FSMap version 48 2023-07-21T18:48:09.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.463+0000 7efc8bc4fb80 10 client.16623 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:48:09.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.463+0000 7efc8bc4fb80 20 client.16623 populate_metadata read hostname 'smithi039' 2023-07-21T18:48:09.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.463+0000 7efc8bc4fb80 1 --2- 172.21.15.39:0/664196375 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55b58eb88750 0x55b58eb8ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:09.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.463+0000 7efc8bc4fb80 4 client.16623 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:48:09.479 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.463+0000 7efc8bc4fb80 1 -- 172.21.15.39:0/664196375 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b58ea15ff0 con 0x55b58eb88750 2023-07-21T18:48:09.479 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.464+0000 7efc7d865700 1 --2- 172.21.15.39:0/664196375 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55b58eb88750 0x55b58eb8ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:09.479 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.464+0000 7efc7d865700 1 --2- 172.21.15.39:0/664196375 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55b58eb88750 0x55b58eb8ab60 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e996dd4038d7763d server_cookie=486d35f0522cb843 in_seq=0 out_seq=0 2023-07-21T18:48:09.479 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.465+0000 7efc617fa700 10 client.16623 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:48:09.525 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.511+0000 7efc617fa700 1 -- 172.21.15.39:0/664196375 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b58ea15ff0 con 0x55b58eb88750 2023-07-21T18:48:09.525 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.511+0000 7efc617fa700 10 client.16623 handle_command_reply: tid=0 2023-07-21T18:48:09.525 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.511+0000 7efc8bc4fb80 1 client.16623 shutdown 2023-07-21T18:48:09.526 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.512+0000 7efc8bc4fb80 1 -- 172.21.15.39:0/664196375 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b58eb03c60 msgr2=0x55b58eb165b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:48:09.526 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.512+0000 7efc8bc4fb80 1 --2- 172.21.15.39:0/664196375 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b58eb03c60 0x55b58eb165b0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7efc70018040 tx=0x7efc70020a40 comp rx=0 tx=0).stop 2023-07-21T18:48:09.526 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.512+0000 7efc8bc4fb80 1 -- 172.21.15.39:0/664196375 shutdown_connections 2023-07-21T18:48:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.512+0000 7efc8bc4fb80 1 --2- 172.21.15.39:0/664196375 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55b58eb88750 0x55b58eb8ab60 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.512+0000 7efc8bc4fb80 1 --2- 172.21.15.39:0/664196375 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b58e9656b0 0x55b58eb16070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.512+0000 7efc8bc4fb80 1 --2- 172.21.15.39:0/664196375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b58eb127c0 0x55b58eb16af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.512+0000 7efc8bc4fb80 1 --2- 172.21.15.39:0/664196375 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b58eb03c60 0x55b58eb165b0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.512+0000 7efc8bc4fb80 1 -- 172.21.15.39:0/664196375 >> 172.21.15.39:0/664196375 conn(0x55b58eafebe0 msgr2=0x55b58eb034e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:48:09.528 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.512+0000 7efc617fa700 0 client.16623 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:48:09.528 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.512+0000 7efc8bc4fb80 1 -- 172.21.15.39:0/664196375 shutdown_connections 2023-07-21T18:48:09.528 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.512+0000 7efc8bc4fb80 1 -- 172.21.15.39:0/664196375 wait complete. 2023-07-21T18:48:09.528 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.513+0000 7efc8bc4fb80 20 client.16623 trim_cache size 0 max 16384 2023-07-21T18:48:09.528 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:48:09.529 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:48:09.529 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:48:09.529 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:48:09.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.513+0000 7efc7fac9700 1 -- 172.21.15.39:0/1198597998 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7efc640a1400 msgr2=0x7efc640a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:48:09.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.513+0000 7efc7fac9700 1 --2- 172.21.15.39:0/1198597998 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7efc640a1400 0x7efc640a38b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7efc74005950 tx=0x7efc740106b0 comp rx=0 tx=0).stop 2023-07-21T18:48:09.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.513+0000 7efc7fac9700 1 -- 172.21.15.39:0/1198597998 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc7810e2a0 msgr2=0x7efc781a4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:48:09.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.513+0000 7efc7fac9700 1 --2- 172.21.15.39:0/1198597998 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc7810e2a0 0x7efc781a4890 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7efc68009810 tx=0x7efc6800fa00 comp rx=0 tx=0).stop 2023-07-21T18:48:09.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.513+0000 7efc7fac9700 1 -- 172.21.15.39:0/1198597998 shutdown_connections 2023-07-21T18:48:09.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.513+0000 7efc7fac9700 1 --2- 172.21.15.39:0/1198597998 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7efc640a1400 0x7efc640a38b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:09.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.513+0000 7efc7fac9700 1 --2- 172.21.15.39:0/1198597998 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc7806c910 0x7efc781aaac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:09.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.513+0000 7efc7fac9700 1 --2- 172.21.15.39:0/1198597998 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc7810e2a0 0x7efc781a4890 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:09.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.513+0000 7efc7fac9700 1 --2- 172.21.15.39:0/1198597998 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc7806d3a0 0x7efc781ab000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:09.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.513+0000 7efc7fac9700 1 -- 172.21.15.39:0/1198597998 >> 172.21.15.39:0/1198597998 conn(0x7efc7806bd70 msgr2=0x7efc780707d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:48:09.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.514+0000 7efc7fac9700 1 -- 172.21.15.39:0/1198597998 shutdown_connections 2023-07-21T18:48:09.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:09.514+0000 7efc7fac9700 1 -- 172.21.15.39:0/1198597998 wait complete. 2023-07-21T18:48:10.275 INFO:tasks.workunit.client.0.smithi039.stdout: 1 390825 6.31 MB/sec execute 203 sec latency 3076.384 ms 2023-07-21T18:48:10.612 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:48:10 smithi193 ceph-mon[98204]: pgmap v196: 97 pgs: 97 active+clean; 149 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 1.9 MiB/s wr, 154 op/s 2023-07-21T18:48:10.612 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:48:10 smithi193 ceph-mon[98204]: osdmap e84: 12 total, 12 up, 12 in 2023-07-21T18:48:10.631 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:48:10 smithi155 ceph-mon[97934]: pgmap v196: 97 pgs: 97 active+clean; 149 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 1.9 MiB/s wr, 154 op/s 2023-07-21T18:48:10.631 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:48:10 smithi155 ceph-mon[97934]: osdmap e84: 12 total, 12 up, 12 in 2023-07-21T18:48:10.679 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:48:10 smithi039 ceph-mon[96489]: pgmap v196: 97 pgs: 97 active+clean; 149 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 1.9 MiB/s wr, 154 op/s 2023-07-21T18:48:10.680 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:48:10 smithi039 ceph-mon[96489]: osdmap e84: 12 total, 12 up, 12 in 2023-07-21T18:48:11.275 INFO:tasks.workunit.client.0.smithi039.stdout: 1 391132 6.28 MB/sec execute 204 sec latency 3873.303 ms 2023-07-21T18:48:12.275 INFO:tasks.workunit.client.0.smithi039.stdout: 1 393134 6.31 MB/sec execute 205 sec latency 20.799 ms 2023-07-21T18:48:12.612 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:48:12 smithi193 ceph-mon[98204]: pgmap v198: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 1.7 MiB/s wr, 110 op/s 2023-07-21T18:48:12.631 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:48:12 smithi155 ceph-mon[97934]: pgmap v198: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 1.7 MiB/s wr, 110 op/s 2023-07-21T18:48:12.679 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:48:12 smithi039 ceph-mon[96489]: pgmap v198: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 1.7 MiB/s wr, 110 op/s 2023-07-21T18:48:13.275 INFO:tasks.workunit.client.0.smithi039.stdout: 1 394431 6.31 MB/sec execute 206 sec latency 16.252 ms 2023-07-21T18:48:14.275 INFO:tasks.workunit.client.0.smithi039.stdout: 1 395641 6.31 MB/sec execute 207 sec latency 17.944 ms 2023-07-21T18:48:14.612 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:48:14 smithi193 ceph-mon[98204]: pgmap v199: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 1.7 MiB/s wr, 110 op/s 2023-07-21T18:48:14.631 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:48:14 smithi155 ceph-mon[97934]: pgmap v199: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 1.7 MiB/s wr, 110 op/s 2023-07-21T18:48:14.679 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:48:14 smithi039 ceph-mon[96489]: pgmap v199: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 1.7 MiB/s wr, 110 op/s 2023-07-21T18:48:15.275 INFO:tasks.workunit.client.0.smithi039.stdout: 1 397524 6.34 MB/sec execute 208 sec latency 31.310 ms 2023-07-21T18:48:16.275 INFO:tasks.workunit.client.0.smithi039.stdout: 1 398719 6.34 MB/sec execute 209 sec latency 25.514 ms 2023-07-21T18:48:16.612 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:48:16 smithi193 ceph-mon[98204]: pgmap v200: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 1.7 MiB/s wr, 110 op/s 2023-07-21T18:48:16.612 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:48:16 smithi193 ceph-mon[98204]: osdmap e85: 12 total, 12 up, 12 in 2023-07-21T18:48:16.630 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:48:16 smithi155 ceph-mon[97934]: pgmap v200: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 1.7 MiB/s wr, 110 op/s 2023-07-21T18:48:16.631 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:48:16 smithi155 ceph-mon[97934]: osdmap e85: 12 total, 12 up, 12 in 2023-07-21T18:48:16.679 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:48:16 smithi039 ceph-mon[96489]: pgmap v200: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 1.7 MiB/s wr, 110 op/s 2023-07-21T18:48:16.680 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:48:16 smithi039 ceph-mon[96489]: osdmap e85: 12 total, 12 up, 12 in 2023-07-21T18:48:17.275 INFO:tasks.workunit.client.0.smithi039.stdout: 1 400927 6.37 MB/sec execute 210 sec latency 22.400 ms 2023-07-21T18:48:18.275 INFO:tasks.workunit.client.0.smithi039.stdout: 1 402117 6.35 MB/sec execute 211 sec latency 12.081 ms 2023-07-21T18:48:18.612 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:48:18 smithi193 ceph-mon[98204]: pgmap v202: 97 pgs: 97 active+clean; 149 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.6 MiB/s wr, 210 op/s 2023-07-21T18:48:18.631 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:48:18 smithi155 ceph-mon[97934]: pgmap v202: 97 pgs: 97 active+clean; 149 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.6 MiB/s wr, 210 op/s 2023-07-21T18:48:18.679 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:48:18 smithi039 ceph-mon[96489]: pgmap v202: 97 pgs: 97 active+clean; 149 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.6 MiB/s wr, 210 op/s 2023-07-21T18:48:19.275 INFO:tasks.workunit.client.0.smithi039.stdout: 1 404399 6.40 MB/sec execute 212 sec latency 17.989 ms 2023-07-21T18:48:20.275 INFO:tasks.workunit.client.0.smithi039.stdout: 1 405390 6.38 MB/sec execute 213 sec latency 31.047 ms 2023-07-21T18:48:20.612 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:48:20 smithi193 ceph-mon[98204]: pgmap v203: 97 pgs: 97 active+clean; 149 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 2.3 MiB/s wr, 185 op/s 2023-07-21T18:48:20.631 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:48:20 smithi155 ceph-mon[97934]: pgmap v203: 97 pgs: 97 active+clean; 149 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 2.3 MiB/s wr, 185 op/s 2023-07-21T18:48:20.679 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:48:20 smithi039 ceph-mon[96489]: pgmap v203: 97 pgs: 97 active+clean; 149 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 2.3 MiB/s wr, 185 op/s 2023-07-21T18:48:21.275 INFO:tasks.workunit.client.0.smithi039.stdout: 1 407413 6.40 MB/sec execute 214 sec latency 24.456 ms 2023-07-21T18:48:22.275 INFO:tasks.workunit.client.0.smithi039.stdout: 1 408672 6.40 MB/sec execute 215 sec latency 14.197 ms 2023-07-21T18:48:22.612 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:48:22 smithi193 ceph-mon[98204]: pgmap v204: 97 pgs: 97 active+clean; 159 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 256 op/s 2023-07-21T18:48:22.631 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:48:22 smithi155 ceph-mon[97934]: pgmap v204: 97 pgs: 97 active+clean; 159 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 256 op/s 2023-07-21T18:48:22.667 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.652+0000 7f5c64f16700 1 -- 172.21.15.39:0/1033291590 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c600fec00 msgr2=0x7f5c601046d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:48:22.667 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.652+0000 7f5c64f16700 1 --2- 172.21.15.39:0/1033291590 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c600fec00 0x7f5c601046d0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f5c540099c0 tx=0x7f5c54009cd0 comp rx=0 tx=0).stop 2023-07-21T18:48:22.667 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.652+0000 7f5c64f16700 1 -- 172.21.15.39:0/1033291590 shutdown_connections 2023-07-21T18:48:22.668 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.652+0000 7f5c64f16700 1 --2- 172.21.15.39:0/1033291590 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c60104db0 0x7f5c6010c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:22.668 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.652+0000 7f5c64f16700 1 --2- 172.21.15.39:0/1033291590 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c600fe260 0x7f5c600fe630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:22.668 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.652+0000 7f5c64f16700 1 --2- 172.21.15.39:0/1033291590 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c600fec00 0x7f5c601046d0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:22.668 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.652+0000 7f5c64f16700 1 -- 172.21.15.39:0/1033291590 >> 172.21.15.39:0/1033291590 conn(0x7f5c60075f90 msgr2=0x7f5c60076390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:48:22.668 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.653+0000 7f5c64f16700 1 -- 172.21.15.39:0/1033291590 shutdown_connections 2023-07-21T18:48:22.668 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.653+0000 7f5c64f16700 1 -- 172.21.15.39:0/1033291590 wait complete. 2023-07-21T18:48:22.670 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.653+0000 7f5c64f16700 1 Processor -- start 2023-07-21T18:48:22.670 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.653+0000 7f5c64f16700 1 -- start start 2023-07-21T18:48:22.670 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.654+0000 7f5c64f16700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c600fe260 0x7f5c6019bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:22.670 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.654+0000 7f5c64f16700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c600fec00 0x7f5c6019c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:22.670 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.654+0000 7f5c64f16700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c60104db0 0x7f5c60195a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:22.670 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.654+0000 7f5c64f16700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c601960c0 con 0x7f5c600fe260 2023-07-21T18:48:22.671 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.654+0000 7f5c64f16700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c60196230 con 0x7f5c60104db0 2023-07-21T18:48:22.671 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.654+0000 7f5c64f16700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c601963f0 con 0x7f5c600fec00 2023-07-21T18:48:22.671 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.654+0000 7f5c5e59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c600fe260 0x7f5c6019bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:22.671 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.654+0000 7f5c5e59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c600fe260 0x7f5c6019bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60034/0 (socket says 172.21.15.39:60034) 2023-07-21T18:48:22.671 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.654+0000 7f5c5e59c700 1 -- 172.21.15.39:0/2587489448 learned_addr learned my addr 172.21.15.39:0/2587489448 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:48:22.671 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.654+0000 7f5c5ed9d700 1 --2- 172.21.15.39:0/2587489448 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c60104db0 0x7f5c60195a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:22.671 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.654+0000 7f5c5dd9b700 1 --2- 172.21.15.39:0/2587489448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c600fec00 0x7f5c6019c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:22.672 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.654+0000 7f5c5e59c700 1 -- 172.21.15.39:0/2587489448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c600fec00 msgr2=0x7f5c6019c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:48:22.672 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.654+0000 7f5c5e59c700 1 --2- 172.21.15.39:0/2587489448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c600fec00 0x7f5c6019c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:22.672 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.654+0000 7f5c5e59c700 1 -- 172.21.15.39:0/2587489448 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c60104db0 msgr2=0x7f5c60195a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:48:22.672 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.654+0000 7f5c5e59c700 1 --2- 172.21.15.39:0/2587489448 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c60104db0 0x7f5c60195a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:22.672 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.654+0000 7f5c5e59c700 1 -- 172.21.15.39:0/2587489448 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c540095f0 con 0x7f5c600fe260 2023-07-21T18:48:22.672 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.654+0000 7f5c5e59c700 1 --2- 172.21.15.39:0/2587489448 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c600fe260 0x7f5c6019bbf0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f5c5000e9a0 tx=0x7f5c5000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:48:22.672 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.654+0000 7f5c4f7fe700 1 -- 172.21.15.39:0/2587489448 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c50009790 con 0x7f5c600fe260 2023-07-21T18:48:22.673 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.654+0000 7f5c4f7fe700 1 -- 172.21.15.39:0/2587489448 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c5000cbd0 con 0x7f5c600fe260 2023-07-21T18:48:22.673 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.654+0000 7f5c64f16700 1 -- 172.21.15.39:0/2587489448 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c601966d0 con 0x7f5c600fe260 2023-07-21T18:48:22.673 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.654+0000 7f5c4f7fe700 1 -- 172.21.15.39:0/2587489448 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c50018680 con 0x7f5c600fe260 2023-07-21T18:48:22.673 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.655+0000 7f5c64f16700 1 -- 172.21.15.39:0/2587489448 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c601a2be0 con 0x7f5c600fe260 2023-07-21T18:48:22.675 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.657+0000 7f5c4f7fe700 1 -- 172.21.15.39:0/2587489448 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f5c5000cd40 con 0x7f5c600fe260 2023-07-21T18:48:22.675 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.658+0000 7f5c4f7fe700 1 --2- 172.21.15.39:0/2587489448 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f5c480a1340 0x7f5c480a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:22.675 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.658+0000 7f5c5dd9b700 1 --2- 172.21.15.39:0/2587489448 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f5c480a1340 0x7f5c480a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:22.675 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.658+0000 7f5c4f7fe700 1 -- 172.21.15.39:0/2587489448 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f5c50014070 con 0x7f5c600fe260 2023-07-21T18:48:22.675 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.658+0000 7f5c5dd9b700 1 --2- 172.21.15.39:0/2587489448 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f5c480a1340 0x7f5c480a37f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5c540098e0 tx=0x7f5c5401a040 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:48:22.675 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.658+0000 7f5c7109cb80 1 Processor -- start 2023-07-21T18:48:22.676 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.659+0000 7f5c7109cb80 1 -- start start 2023-07-21T18:48:22.676 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.659+0000 7f5c7109cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607e84606b0 0x5607e8609d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:22.676 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.659+0000 7f5c7109cb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607e860a2d0 0x5607e8604040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:22.676 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.659+0000 7f5c7109cb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607e8604580 0x5607e86049d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:22.676 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.659+0000 7f5c7109cb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607e8509c50 con 0x5607e8604580 2023-07-21T18:48:22.676 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.659+0000 7f5c7109cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607e855c9e0 con 0x5607e84606b0 2023-07-21T18:48:22.676 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.659+0000 7f5c7109cb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607e8555e70 con 0x5607e860a2d0 2023-07-21T18:48:22.677 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.659+0000 7f5c5ed9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607e8604580 0x5607e86049d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:22.677 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.659+0000 7f5c5dd9b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607e860a2d0 0x5607e8604040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:22.677 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.659+0000 7f5c5dd9b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607e860a2d0 0x5607e8604040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:50362/0 (socket says 172.21.15.39:50362) 2023-07-21T18:48:22.677 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.659+0000 7f5c5ed9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607e8604580 0x5607e86049d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60046/0 (socket says 172.21.15.39:60046) 2023-07-21T18:48:22.677 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.659+0000 7f5c5dd9b700 1 -- 172.21.15.39:0/770857426 learned_addr learned my addr 172.21.15.39:0/770857426 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:48:22.677 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.659+0000 7f5c5e59c700 1 --2- 172.21.15.39:0/770857426 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607e84606b0 0x5607e8609d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:22.677 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.659+0000 7f5c5dd9b700 1 -- 172.21.15.39:0/770857426 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607e84606b0 msgr2=0x5607e8609d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:48:22.677 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.659+0000 7f5c5dd9b700 1 --2- 172.21.15.39:0/770857426 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607e84606b0 0x5607e8609d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:22.678 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.659+0000 7f5c5dd9b700 1 -- 172.21.15.39:0/770857426 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607e8604580 msgr2=0x5607e86049d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:48:22.678 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.659+0000 7f5c5dd9b700 1 --2- 172.21.15.39:0/770857426 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607e8604580 0x5607e86049d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:22.678 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.659+0000 7f5c5dd9b700 1 -- 172.21.15.39:0/770857426 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5607e8383d80 con 0x5607e860a2d0 2023-07-21T18:48:22.678 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.660+0000 7f5c5dd9b700 1 --2- 172.21.15.39:0/770857426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607e860a2d0 0x5607e8604040 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5c54005850 tx=0x7f5c5401e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51b0744e68872495 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:48:22.678 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.660+0000 7f5c4cff9700 1 -- 172.21.15.39:0/770857426 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c54031070 con 0x5607e860a2d0 2023-07-21T18:48:22.678 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.660+0000 7f5c4cff9700 1 -- 172.21.15.39:0/770857426 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c54029040 con 0x5607e860a2d0 2023-07-21T18:48:22.678 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.660+0000 7f5c4cff9700 1 -- 172.21.15.39:0/770857426 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c5401d040 con 0x5607e860a2d0 2023-07-21T18:48:22.679 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.661+0000 7f5c7109cb80 1 -- 172.21.15.39:0/770857426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607e860a2d0 msgr2=0x5607e8604040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:48:22.679 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.661+0000 7f5c7109cb80 1 --2- 172.21.15.39:0/770857426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607e860a2d0 0x5607e8604040 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5c54005850 tx=0x7f5c5401e000 comp rx=0 tx=0).stop 2023-07-21T18:48:22.679 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.661+0000 7f5c7109cb80 1 -- 172.21.15.39:0/770857426 shutdown_connections 2023-07-21T18:48:22.679 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.661+0000 7f5c7109cb80 1 --2- 172.21.15.39:0/770857426 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607e84606b0 0x5607e8609d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:22.679 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.661+0000 7f5c7109cb80 1 --2- 172.21.15.39:0/770857426 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607e8604580 0x5607e86049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:22.679 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.661+0000 7f5c7109cb80 1 --2- 172.21.15.39:0/770857426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607e860a2d0 0x5607e8604040 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:22.679 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.661+0000 7f5c7109cb80 1 -- 172.21.15.39:0/770857426 >> 172.21.15.39:0/770857426 conn(0x5607e85f9be0 msgr2=0x5607e85fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:48:22.679 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.661+0000 7f5c7109cb80 1 -- 172.21.15.39:0/770857426 shutdown_connections 2023-07-21T18:48:22.680 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.661+0000 7f5c7109cb80 1 -- 172.21.15.39:0/770857426 wait complete. 2023-07-21T18:48:22.682 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.661+0000 7f5c7109cb80 1 -- start start 2023-07-21T18:48:22.682 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.661+0000 7f5c7109cb80 1 Processor -- start 2023-07-21T18:48:22.682 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.662+0000 7f5c7109cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607e84606b0 0x5607e86105a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:22.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.662+0000 7f5c7109cb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607e8604580 0x5607e860d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:22.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.662+0000 7f5c7109cb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607e860a2d0 0x5607e860de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:22.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.662+0000 7f5c7109cb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607e84f5a60 con 0x5607e860a2d0 2023-07-21T18:48:22.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.662+0000 7f5c7109cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607e84f5760 con 0x5607e84606b0 2023-07-21T18:48:22.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.662+0000 7f5c7109cb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607e83d98c0 con 0x5607e8604580 2023-07-21T18:48:22.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.662+0000 7f5c7109cb80 10 client.0 _ll_register_callbacks cb 0x5607e84f3700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:48:22.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.662+0000 7f5c5ed9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607e860a2d0 0x5607e860de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:22.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.662+0000 7f5c5ed9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607e860a2d0 0x5607e860de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60052/0 (socket says 172.21.15.39:60052) 2023-07-21T18:48:22.684 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.662+0000 7f5c5ed9d700 1 -- 172.21.15.39:0/4163361093 learned_addr learned my addr 172.21.15.39:0/4163361093 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:48:22.684 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.662+0000 7f5c5dd9b700 1 --2- 172.21.15.39:0/4163361093 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607e8604580 0x5607e860d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:22.684 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.662+0000 7f5c5e59c700 1 --2- 172.21.15.39:0/4163361093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607e84606b0 0x5607e86105a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:22.684 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.662+0000 7f5c5dd9b700 1 -- 172.21.15.39:0/4163361093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607e84606b0 msgr2=0x5607e86105a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:48:22.684 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.662+0000 7f5c5dd9b700 1 --2- 172.21.15.39:0/4163361093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607e84606b0 0x5607e86105a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:22.684 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.662+0000 7f5c5dd9b700 1 -- 172.21.15.39:0/4163361093 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607e860a2d0 msgr2=0x5607e860de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:48:22.684 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.662+0000 7f5c5dd9b700 1 --2- 172.21.15.39:0/4163361093 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607e860a2d0 0x5607e860de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:22.685 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.662+0000 7f5c5dd9b700 1 -- 172.21.15.39:0/4163361093 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c6010d1c0 con 0x5607e8604580 2023-07-21T18:48:22.685 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.662+0000 7f5c5ed9d700 1 --2- 172.21.15.39:0/4163361093 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607e860a2d0 0x5607e860de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:48:22.685 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.662+0000 7f5c5dd9b700 1 --2- 172.21.15.39:0/4163361093 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607e8604580 0x5607e860d820 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5c5400b580 tx=0x7f5c54005020 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eead653438eca3fd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:48:22.685 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.662+0000 7f5c45ffb700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2023-07-21T18:48:22.685 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.662+0000 7f5c45ffb700 1 -- 172.21.15.39:0/4163361093 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c54031040 con 0x5607e8604580 2023-07-21T18:48:22.685 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.663+0000 7f5c45ffb700 1 -- 172.21.15.39:0/4163361093 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f5c5400fc50 con 0x5607e8604580 2023-07-21T18:48:22.685 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.663+0000 7f5c7109cb80 1 -- 172.21.15.39:0/4163361093 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5607e84af490 con 0x5607e8604580 2023-07-21T18:48:22.685 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.663+0000 7f5c45ffb700 1 -- 172.21.15.39:0/4163361093 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c5400fe30 con 0x5607e8604580 2023-07-21T18:48:22.686 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.663+0000 7f5c7109cb80 10 client.26119 fetch_fsmap learned FSMap version 48 2023-07-21T18:48:22.686 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.663+0000 7f5c7109cb80 1 -- 172.21.15.39:0/4163361093 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x5607e836cc00 con 0x5607e8604580 2023-07-21T18:48:22.686 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.663+0000 7f5c45ffb700 1 -- 172.21.15.39:0/4163361093 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5607e84af490 con 0x5607e8604580 2023-07-21T18:48:22.686 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.663+0000 7f5c45ffb700 1 -- 172.21.15.39:0/4163361093 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 48) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f5c5402c070 con 0x5607e8604580 2023-07-21T18:48:22.686 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.663+0000 7f5c7109cb80 10 client.26119 fetch_fsmap finished waiting for FSMap version 48 2023-07-21T18:48:22.686 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.663+0000 7f5c7109cb80 10 client.26119 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:48:22.686 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.663+0000 7f5c7109cb80 20 client.26119 populate_metadata read hostname 'smithi039' 2023-07-21T18:48:22.686 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.663+0000 7f5c7109cb80 1 --2- 172.21.15.39:0/4163361093 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5607e8621b30 0x5607e8623f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:22.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.663+0000 7f5c7109cb80 4 client.26119 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:48:22.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.663+0000 7f5c7109cb80 1 -- 172.21.15.39:0/4163361093 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5607e854ee50 con 0x5607e8621b30 2023-07-21T18:48:22.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.663+0000 7f5c5e59c700 1 --2- 172.21.15.39:0/4163361093 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5607e8621b30 0x5607e8623f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:22.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.664+0000 7f5c45ffb700 10 client.26119 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:48:22.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.664+0000 7f5c5e59c700 1 --2- 172.21.15.39:0/4163361093 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5607e8621b30 0x5607e8623f40 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=793d2420552ad780 server_cookie=cb89288205639cd8 in_seq=0 out_seq=0 2023-07-21T18:48:22.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.664+0000 7f5c45ffb700 1 -- 172.21.15.39:0/4163361093 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5607e854ee50 con 0x5607e8621b30 2023-07-21T18:48:22.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.664+0000 7f5c45ffb700 10 client.26119 handle_command_reply: tid=0 2023-07-21T18:48:22.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.664+0000 7f5c7109cb80 1 client.26119 shutdown 2023-07-21T18:48:22.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.665+0000 7f5c7109cb80 1 -- 172.21.15.39:0/4163361093 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607e8604580 msgr2=0x5607e860d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:48:22.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.665+0000 7f5c7109cb80 1 --2- 172.21.15.39:0/4163361093 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607e8604580 0x5607e860d820 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5c5400b580 tx=0x7f5c54005020 comp rx=0 tx=0).stop 2023-07-21T18:48:22.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.665+0000 7f5c7109cb80 1 -- 172.21.15.39:0/4163361093 shutdown_connections 2023-07-21T18:48:22.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.665+0000 7f5c7109cb80 1 --2- 172.21.15.39:0/4163361093 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5607e8621b30 0x5607e8623f40 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:22.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.665+0000 7f5c7109cb80 1 --2- 172.21.15.39:0/4163361093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607e84606b0 0x5607e86105a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:22.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.665+0000 7f5c7109cb80 1 --2- 172.21.15.39:0/4163361093 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607e860a2d0 0x5607e860de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:22.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.665+0000 7f5c45ffb700 0 client.26119 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:48:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.665+0000 7f5c7109cb80 1 --2- 172.21.15.39:0/4163361093 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607e8604580 0x5607e860d820 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.665+0000 7f5c7109cb80 1 -- 172.21.15.39:0/4163361093 >> 172.21.15.39:0/4163361093 conn(0x5607e85f9be0 msgr2=0x5607e85fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:48:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.665+0000 7f5c7109cb80 1 -- 172.21.15.39:0/4163361093 shutdown_connections 2023-07-21T18:48:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.665+0000 7f5c7109cb80 1 -- 172.21.15.39:0/4163361093 wait complete. 2023-07-21T18:48:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.665+0000 7f5c7109cb80 20 client.26119 trim_cache size 0 max 16384 2023-07-21T18:48:22.689 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:48:22 smithi039 ceph-mon[96489]: pgmap v204: 97 pgs: 97 active+clean; 159 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 256 op/s 2023-07-21T18:48:22.695 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.680+0000 7f5c7109cb80 1 Processor -- start 2023-07-21T18:48:22.695 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.681+0000 7f5c7109cb80 1 -- start start 2023-07-21T18:48:22.696 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.681+0000 7f5c7109cb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607e84606b0 0x5607e8604040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:22.696 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.681+0000 7f5c7109cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607e8604580 0x5607e860cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:22.696 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.681+0000 7f5c7109cb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607e860a2d0 0x5607e8614c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:22.696 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.681+0000 7f5c7109cb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607e85ffa00 con 0x5607e84606b0 2023-07-21T18:48:22.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.681+0000 7f5c7109cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607e85ffd50 con 0x5607e8604580 2023-07-21T18:48:22.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.681+0000 7f5c7109cb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607e8604fd0 con 0x5607e860a2d0 2023-07-21T18:48:22.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.681+0000 7f5c5e59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607e84606b0 0x5607e8604040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:22.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.681+0000 7f5c5dd9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607e8604580 0x5607e860cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:22.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.681+0000 7f5c5dd9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607e8604580 0x5607e860cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:44422/0 (socket says 172.21.15.39:44422) 2023-07-21T18:48:22.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.681+0000 7f5c5dd9b700 1 -- 172.21.15.39:0/1720764562 learned_addr learned my addr 172.21.15.39:0/1720764562 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:48:22.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.681+0000 7f5c5ed9d700 1 --2- 172.21.15.39:0/1720764562 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607e860a2d0 0x5607e8614c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:22.698 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.681+0000 7f5c5e59c700 1 -- 172.21.15.39:0/1720764562 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607e860a2d0 msgr2=0x5607e8614c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:48:22.698 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.681+0000 7f5c5e59c700 1 --2- 172.21.15.39:0/1720764562 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607e860a2d0 0x5607e8614c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:22.698 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.681+0000 7f5c5e59c700 1 -- 172.21.15.39:0/1720764562 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607e8604580 msgr2=0x5607e860cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:48:22.698 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.681+0000 7f5c5e59c700 1 --2- 172.21.15.39:0/1720764562 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607e8604580 0x5607e860cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:22.698 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.681+0000 7f5c5e59c700 1 -- 172.21.15.39:0/1720764562 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5607e860d390 con 0x5607e84606b0 2023-07-21T18:48:22.698 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.681+0000 7f5c5dd9b700 1 --2- 172.21.15.39:0/1720764562 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607e8604580 0x5607e860cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:48:22.698 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.682+0000 7f5c5e59c700 1 --2- 172.21.15.39:0/1720764562 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607e84606b0 0x5607e8604040 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f5c500ca690 tx=0x7f5c5001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=188c1adaca3e2f23 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:48:22.698 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.682+0000 7f5c4d7fa700 1 -- 172.21.15.39:0/1720764562 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c5009e040 con 0x5607e84606b0 2023-07-21T18:48:22.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.682+0000 7f5c4d7fa700 1 -- 172.21.15.39:0/1720764562 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c500a2460 con 0x5607e84606b0 2023-07-21T18:48:22.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.682+0000 7f5c4d7fa700 1 -- 172.21.15.39:0/1720764562 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c500a1fd0 con 0x5607e84606b0 2023-07-21T18:48:22.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.682+0000 7f5c7109cb80 1 -- 172.21.15.39:0/1720764562 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607e84606b0 msgr2=0x5607e8604040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:48:22.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.682+0000 7f5c7109cb80 1 --2- 172.21.15.39:0/1720764562 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607e84606b0 0x5607e8604040 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f5c500ca690 tx=0x7f5c5001f000 comp rx=0 tx=0).stop 2023-07-21T18:48:22.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.683+0000 7f5c7109cb80 1 -- 172.21.15.39:0/1720764562 shutdown_connections 2023-07-21T18:48:22.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.683+0000 7f5c7109cb80 1 --2- 172.21.15.39:0/1720764562 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607e860a2d0 0x5607e8614c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:22.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.683+0000 7f5c7109cb80 1 --2- 172.21.15.39:0/1720764562 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607e84606b0 0x5607e8604040 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:22.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.683+0000 7f5c7109cb80 1 --2- 172.21.15.39:0/1720764562 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607e8604580 0x5607e860cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:22.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.683+0000 7f5c7109cb80 1 -- 172.21.15.39:0/1720764562 >> 172.21.15.39:0/1720764562 conn(0x5607e85f9be0 msgr2=0x5607e8607a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:48:22.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.683+0000 7f5c7109cb80 1 -- 172.21.15.39:0/1720764562 shutdown_connections 2023-07-21T18:48:22.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.683+0000 7f5c7109cb80 1 -- 172.21.15.39:0/1720764562 wait complete. 2023-07-21T18:48:22.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.683+0000 7f5c7109cb80 1 -- start start 2023-07-21T18:48:22.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.683+0000 7f5c7109cb80 1 Processor -- start 2023-07-21T18:48:22.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.683+0000 7f5c7109cb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607e84606b0 0x5607e8606f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:22.703 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.683+0000 7f5c7109cb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607e8604580 0x5607e8605790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:22.703 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.683+0000 7f5c7109cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607e860a2d0 0x5607e8605cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:22.703 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.684+0000 7f5c7109cb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607e8615210 con 0x5607e84606b0 2023-07-21T18:48:22.703 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.684+0000 7f5c7109cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607e8606210 con 0x5607e860a2d0 2023-07-21T18:48:22.703 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.684+0000 7f5c7109cb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607e86063d0 con 0x5607e8604580 2023-07-21T18:48:22.703 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.684+0000 7f5c7109cb80 10 client.0 _ll_register_callbacks cb 0x7f5c58009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:48:22.703 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.684+0000 7f5c5e59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607e84606b0 0x5607e8606f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:22.703 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.684+0000 7f5c5e59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607e84606b0 0x5607e8606f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60062/0 (socket says 172.21.15.39:60062) 2023-07-21T18:48:22.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.684+0000 7f5c5e59c700 1 -- 172.21.15.39:0/3823797762 learned_addr learned my addr 172.21.15.39:0/3823797762 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:48:22.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.684+0000 7f5c5dd9b700 1 --2- 172.21.15.39:0/3823797762 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607e8604580 0x5607e8605790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:22.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.684+0000 7f5c5ed9d700 1 --2- 172.21.15.39:0/3823797762 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607e860a2d0 0x5607e8605cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:22.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.684+0000 7f5c5e59c700 1 -- 172.21.15.39:0/3823797762 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607e8604580 msgr2=0x5607e8605790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:48:22.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.684+0000 7f5c5e59c700 1 --2- 172.21.15.39:0/3823797762 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607e8604580 0x5607e8605790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:22.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.684+0000 7f5c5e59c700 1 -- 172.21.15.39:0/3823797762 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607e860a2d0 msgr2=0x5607e8605cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:48:22.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.684+0000 7f5c5e59c700 1 --2- 172.21.15.39:0/3823797762 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607e860a2d0 0x5607e8605cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:22.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.684+0000 7f5c5e59c700 1 -- 172.21.15.39:0/3823797762 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5607e836cc00 con 0x5607e84606b0 2023-07-21T18:48:22.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.684+0000 7f5c5e59c700 1 --2- 172.21.15.39:0/3823797762 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607e84606b0 0x5607e8606f90 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f5c50012010 tx=0x7f5c5009e190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4d8dcdd1fb7a52a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:48:22.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.684+0000 7f5c45ffb700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:48:22.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.684+0000 7f5c45ffb700 1 -- 172.21.15.39:0/3823797762 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c5009a470 con 0x5607e84606b0 2023-07-21T18:48:22.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.684+0000 7f5c45ffb700 1 -- 172.21.15.39:0/3823797762 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f5c50099500 con 0x5607e84606b0 2023-07-21T18:48:22.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.684+0000 7f5c7109cb80 1 -- 172.21.15.39:0/3823797762 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5c54029040 con 0x5607e84606b0 2023-07-21T18:48:22.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.685+0000 7f5c45ffb700 1 -- 172.21.15.39:0/3823797762 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c50099700 con 0x5607e84606b0 2023-07-21T18:48:22.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.685+0000 7f5c7109cb80 10 client.16641 fetch_fsmap learned FSMap version 48 2023-07-21T18:48:22.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.685+0000 7f5c7109cb80 1 -- 172.21.15.39:0/3823797762 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x7f5c601a2be0 con 0x5607e84606b0 2023-07-21T18:48:22.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.685+0000 7f5c45ffb700 1 -- 172.21.15.39:0/3823797762 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c54029040 con 0x5607e84606b0 2023-07-21T18:48:22.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.685+0000 7f5c45ffb700 1 -- 172.21.15.39:0/3823797762 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 48) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f5c500ca020 con 0x5607e84606b0 2023-07-21T18:48:22.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.685+0000 7f5c7109cb80 10 client.16641 fetch_fsmap finished waiting for FSMap version 48 2023-07-21T18:48:22.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.685+0000 7f5c7109cb80 10 client.16641 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:48:22.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.685+0000 7f5c7109cb80 20 client.16641 populate_metadata read hostname 'smithi039' 2023-07-21T18:48:22.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.685+0000 7f5c7109cb80 1 --2- 172.21.15.39:0/3823797762 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5607e8676b20 0x5607e8678f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:22.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.685+0000 7f5c7109cb80 4 client.16641 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:48:22.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.685+0000 7f5c7109cb80 1 -- 172.21.15.39:0/3823797762 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5607e8510ff0 con 0x5607e8676b20 2023-07-21T18:48:22.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.685+0000 7f5c5dd9b700 1 --2- 172.21.15.39:0/3823797762 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5607e8676b20 0x5607e8678f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:22.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.686+0000 7f5c5dd9b700 1 --2- 172.21.15.39:0/3823797762 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5607e8676b20 0x5607e8678f30 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5ad81d88d9cbdc0 server_cookie=a0c9fdc1d740108d in_seq=0 out_seq=0 2023-07-21T18:48:22.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.686+0000 7f5c45ffb700 10 client.16641 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:48:22.772 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.758+0000 7f5c45ffb700 1 -- 172.21.15.39:0/3823797762 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5607e8510ff0 con 0x5607e8676b20 2023-07-21T18:48:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.758+0000 7f5c45ffb700 10 client.16641 handle_command_reply: tid=0 2023-07-21T18:48:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.758+0000 7f5c7109cb80 1 client.16641 shutdown 2023-07-21T18:48:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.758+0000 7f5c7109cb80 1 -- 172.21.15.39:0/3823797762 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607e84606b0 msgr2=0x5607e8606f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:48:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.758+0000 7f5c7109cb80 1 --2- 172.21.15.39:0/3823797762 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607e84606b0 0x5607e8606f90 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f5c50012010 tx=0x7f5c5009e190 comp rx=0 tx=0).stop 2023-07-21T18:48:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.758+0000 7f5c7109cb80 1 -- 172.21.15.39:0/3823797762 shutdown_connections 2023-07-21T18:48:22.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.758+0000 7f5c7109cb80 1 --2- 172.21.15.39:0/3823797762 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5607e8676b20 0x5607e8678f30 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:22.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.758+0000 7f5c7109cb80 1 --2- 172.21.15.39:0/3823797762 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607e860a2d0 0x5607e8605cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:22.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.758+0000 7f5c7109cb80 1 --2- 172.21.15.39:0/3823797762 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607e84606b0 0x5607e8606f90 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:22.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.758+0000 7f5c7109cb80 1 --2- 172.21.15.39:0/3823797762 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607e8604580 0x5607e8605790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:22.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.758+0000 7f5c7109cb80 1 -- 172.21.15.39:0/3823797762 >> 172.21.15.39:0/3823797762 conn(0x5607e85f9be0 msgr2=0x5607e860a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:48:22.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.758+0000 7f5c45ffb700 0 client.16641 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:48:22.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.758+0000 7f5c7109cb80 1 -- 172.21.15.39:0/3823797762 shutdown_connections 2023-07-21T18:48:22.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.758+0000 7f5c7109cb80 1 -- 172.21.15.39:0/3823797762 wait complete. 2023-07-21T18:48:22.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.758+0000 7f5c7109cb80 20 client.16641 trim_cache size 0 max 16384 2023-07-21T18:48:22.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:48:22.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:48:22.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:48:22.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:48:22.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.759+0000 7f5c64f16700 1 -- 172.21.15.39:0/2587489448 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f5c480a1340 msgr2=0x7f5c480a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:48:22.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.759+0000 7f5c64f16700 1 --2- 172.21.15.39:0/2587489448 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f5c480a1340 0x7f5c480a37f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5c540098e0 tx=0x7f5c5401a040 comp rx=0 tx=0).stop 2023-07-21T18:48:22.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.759+0000 7f5c64f16700 1 -- 172.21.15.39:0/2587489448 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c600fe260 msgr2=0x7f5c6019bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:48:22.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.759+0000 7f5c64f16700 1 --2- 172.21.15.39:0/2587489448 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c600fe260 0x7f5c6019bbf0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f5c5000e9a0 tx=0x7f5c5000ecb0 comp rx=0 tx=0).stop 2023-07-21T18:48:22.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.759+0000 7f5c64f16700 1 -- 172.21.15.39:0/2587489448 shutdown_connections 2023-07-21T18:48:22.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.759+0000 7f5c64f16700 1 --2- 172.21.15.39:0/2587489448 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f5c480a1340 0x7f5c480a37f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:22.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.759+0000 7f5c64f16700 1 --2- 172.21.15.39:0/2587489448 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c60104db0 0x7f5c60195a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:22.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.759+0000 7f5c64f16700 1 --2- 172.21.15.39:0/2587489448 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c600fe260 0x7f5c6019bbf0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:22.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.759+0000 7f5c64f16700 1 --2- 172.21.15.39:0/2587489448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c600fec00 0x7f5c6019c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:22.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.759+0000 7f5c64f16700 1 -- 172.21.15.39:0/2587489448 >> 172.21.15.39:0/2587489448 conn(0x7f5c60075f90 msgr2=0x7f5c600fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:48:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.759+0000 7f5c64f16700 1 -- 172.21.15.39:0/2587489448 shutdown_connections 2023-07-21T18:48:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:22.759+0000 7f5c64f16700 1 -- 172.21.15.39:0/2587489448 wait complete. 2023-07-21T18:48:23.275 INFO:tasks.workunit.client.0.smithi039.stdout: 1 410378 6.41 MB/sec execute 216 sec latency 114.434 ms 2023-07-21T18:48:24.275 INFO:tasks.workunit.client.0.smithi039.stdout: 1 411893 6.43 MB/sec execute 217 sec latency 40.684 ms 2023-07-21T18:48:24.612 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:48:24 smithi193 ceph-mon[98204]: pgmap v205: 97 pgs: 97 active+clean; 159 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 256 op/s 2023-07-21T18:48:24.630 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:48:24 smithi155 ceph-mon[97934]: pgmap v205: 97 pgs: 97 active+clean; 159 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 256 op/s 2023-07-21T18:48:24.679 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:48:24 smithi039 ceph-mon[96489]: pgmap v205: 97 pgs: 97 active+clean; 159 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 256 op/s 2023-07-21T18:48:25.275 INFO:tasks.workunit.client.0.smithi039.stdout: 1 413109 6.43 MB/sec execute 218 sec latency 38.707 ms 2023-07-21T18:48:26.275 INFO:tasks.workunit.client.0.smithi039.stdout: 1 415100 6.45 MB/sec execute 219 sec latency 24.770 ms 2023-07-21T18:48:26.612 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:48:26 smithi193 ceph-mon[98204]: pgmap v206: 97 pgs: 97 active+clean; 159 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 256 op/s 2023-07-21T18:48:26.631 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:48:26 smithi155 ceph-mon[97934]: pgmap v206: 97 pgs: 97 active+clean; 159 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 256 op/s 2023-07-21T18:48:26.679 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:48:26 smithi039 ceph-mon[96489]: pgmap v206: 97 pgs: 97 active+clean; 159 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 256 op/s 2023-07-21T18:48:27.275 INFO:tasks.workunit.client.0.smithi039.stdout: 1 416196 6.44 MB/sec execute 220 sec latency 15.666 ms 2023-07-21T18:48:27.630 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:48:27 smithi155 ceph-mon[97934]: pgmap v207: 97 pgs: 97 active+clean; 158 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.4 MiB/s wr, 285 op/s 2023-07-21T18:48:27.679 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:48:27 smithi039 ceph-mon[96489]: pgmap v207: 97 pgs: 97 active+clean; 158 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.4 MiB/s wr, 285 op/s 2023-07-21T18:48:27.862 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:48:27 smithi193 ceph-mon[98204]: pgmap v207: 97 pgs: 97 active+clean; 158 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.4 MiB/s wr, 285 op/s 2023-07-21T18:48:28.275 INFO:tasks.workunit.client.0.smithi039.stdout: 1 417918 6.45 MB/sec execute 221 sec latency 27.692 ms 2023-07-21T18:48:29.275 INFO:tasks.workunit.client.0.smithi039.stdout: 1 419245 6.46 MB/sec execute 222 sec latency 13.616 ms 2023-07-21T18:48:29.862 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:48:29 smithi193 ceph-mon[98204]: pgmap v208: 97 pgs: 97 active+clean; 158 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 244 op/s 2023-07-21T18:48:29.880 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:48:29 smithi155 ceph-mon[97934]: pgmap v208: 97 pgs: 97 active+clean; 158 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 244 op/s 2023-07-21T18:48:29.929 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:48:29 smithi039 ceph-mon[96489]: pgmap v208: 97 pgs: 97 active+clean; 158 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 244 op/s 2023-07-21T18:48:30.275 INFO:tasks.workunit.client.0.smithi039.stdout: 1 420448 6.46 MB/sec execute 223 sec latency 41.484 ms 2023-07-21T18:48:31.275 INFO:tasks.workunit.client.0.smithi039.stdout: 1 422383 6.49 MB/sec execute 224 sec latency 21.125 ms 2023-07-21T18:48:32.275 INFO:tasks.workunit.client.0.smithi039.stdout: 1 423620 6.48 MB/sec execute 225 sec latency 20.201 ms 2023-07-21T18:48:32.362 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:48:32 smithi193 ceph-mon[98204]: pgmap v209: 97 pgs: 97 active+clean; 159 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 343 op/s 2023-07-21T18:48:32.380 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:48:32 smithi155 ceph-mon[97934]: pgmap v209: 97 pgs: 97 active+clean; 159 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 343 op/s 2023-07-21T18:48:32.429 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:48:32 smithi039 ceph-mon[96489]: pgmap v209: 97 pgs: 97 active+clean; 159 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 343 op/s 2023-07-21T18:48:33.275 INFO:tasks.workunit.client.0.smithi039.stdout: 1 425113 6.49 MB/sec execute 226 sec latency 23.921 ms 2023-07-21T18:48:34.275 INFO:tasks.workunit.client.0.smithi039.stdout: 1 426464 6.49 MB/sec execute 227 sec latency 15.865 ms 2023-07-21T18:48:34.362 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:48:34 smithi193 ceph-mon[98204]: pgmap v210: 97 pgs: 97 active+clean; 159 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 222 op/s 2023-07-21T18:48:34.380 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:48:34 smithi155 ceph-mon[97934]: pgmap v210: 97 pgs: 97 active+clean; 159 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 222 op/s 2023-07-21T18:48:34.429 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:48:34 smithi039 ceph-mon[96489]: pgmap v210: 97 pgs: 97 active+clean; 159 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 222 op/s 2023-07-21T18:48:35.275 INFO:tasks.workunit.client.0.smithi039.stdout: 1 427765 6.48 MB/sec execute 228 sec latency 16.790 ms 2023-07-21T18:48:35.894 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.880+0000 7fc556ecd700 1 -- 172.21.15.39:0/2300699490 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc5500ffab0 msgr2=0x7fc55010b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:48:35.895 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.880+0000 7fc556ecd700 1 --2- 172.21.15.39:0/2300699490 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc5500ffab0 0x7fc55010b840 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fc548009fc0 tx=0x7fc54800b330 comp rx=0 tx=0).stop 2023-07-21T18:48:35.895 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.880+0000 7fc556ecd700 1 -- 172.21.15.39:0/2300699490 shutdown_connections 2023-07-21T18:48:35.895 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.880+0000 7fc556ecd700 1 --2- 172.21.15.39:0/2300699490 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5501039b0 0x7fc550103d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:35.895 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.880+0000 7fc556ecd700 1 --2- 172.21.15.39:0/2300699490 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc5500ffab0 0x7fc55010b840 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:35.895 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.880+0000 7fc556ecd700 1 --2- 172.21.15.39:0/2300699490 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5500ff100 0x7fc5500ff570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:35.895 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.880+0000 7fc556ecd700 1 -- 172.21.15.39:0/2300699490 >> 172.21.15.39:0/2300699490 conn(0x7fc550075f90 msgr2=0x7fc550076390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:48:35.895 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.880+0000 7fc556ecd700 1 -- 172.21.15.39:0/2300699490 shutdown_connections 2023-07-21T18:48:35.896 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.880+0000 7fc556ecd700 1 -- 172.21.15.39:0/2300699490 wait complete. 2023-07-21T18:48:35.897 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.881+0000 7fc556ecd700 1 Processor -- start 2023-07-21T18:48:35.897 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.881+0000 7fc556ecd700 1 -- start start 2023-07-21T18:48:35.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.881+0000 7fc556ecd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5500ff100 0x7fc550100880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:35.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.881+0000 7fc556ecd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc5500ffab0 0x7fc550100dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:35.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.881+0000 7fc556ecd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5501039b0 0x7fc5501a3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:35.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.881+0000 7fc556ecd700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5501a43b0 con 0x7fc5500ffab0 2023-07-21T18:48:35.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.881+0000 7fc556ecd700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5501a44f0 con 0x7fc5500ff100 2023-07-21T18:48:35.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.881+0000 7fc556ecd700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5501a4660 con 0x7fc5501039b0 2023-07-21T18:48:35.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.881+0000 7fc54ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc5500ffab0 0x7fc550100dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:35.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.881+0000 7fc554c69700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5500ff100 0x7fc550100880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:35.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.881+0000 7fc55546a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5501039b0 0x7fc5501a3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:35.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.881+0000 7fc54ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc5500ffab0 0x7fc550100dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60078/0 (socket says 172.21.15.39:60078) 2023-07-21T18:48:35.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.881+0000 7fc54ffff700 1 -- 172.21.15.39:0/2322114436 learned_addr learned my addr 172.21.15.39:0/2322114436 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:48:35.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.882+0000 7fc554c69700 1 -- 172.21.15.39:0/2322114436 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5501039b0 msgr2=0x7fc5501a3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:48:35.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.882+0000 7fc554c69700 1 --2- 172.21.15.39:0/2322114436 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5501039b0 0x7fc5501a3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:35.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.882+0000 7fc554c69700 1 -- 172.21.15.39:0/2322114436 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc5500ffab0 msgr2=0x7fc550100dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:48:35.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.882+0000 7fc554c69700 1 --2- 172.21.15.39:0/2322114436 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc5500ffab0 0x7fc550100dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:35.900 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.882+0000 7fc554c69700 1 -- 172.21.15.39:0/2322114436 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc548009c40 con 0x7fc5500ff100 2023-07-21T18:48:35.900 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.882+0000 7fc55546a700 1 --2- 172.21.15.39:0/2322114436 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5501039b0 0x7fc5501a3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:48:35.900 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.882+0000 7fc54ffff700 1 --2- 172.21.15.39:0/2322114436 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc5500ffab0 0x7fc550100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:48:35.900 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.882+0000 7fc554c69700 1 --2- 172.21.15.39:0/2322114436 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5500ff100 0x7fc550100880 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc54400e9c0 tx=0x7fc54400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:48:35.900 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.882+0000 7fc54dffb700 1 -- 172.21.15.39:0/2322114436 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54400c9c0 con 0x7fc5500ff100 2023-07-21T18:48:35.900 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.882+0000 7fc556ecd700 1 -- 172.21.15.39:0/2322114436 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5501a4940 con 0x7fc5500ff100 2023-07-21T18:48:35.900 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.882+0000 7fc54dffb700 1 -- 172.21.15.39:0/2322114436 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc544010430 con 0x7fc5500ff100 2023-07-21T18:48:35.901 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.882+0000 7fc54dffb700 1 -- 172.21.15.39:0/2322114436 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc544018680 con 0x7fc5500ff100 2023-07-21T18:48:35.901 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.882+0000 7fc556ecd700 1 -- 172.21.15.39:0/2322114436 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5501a4ea0 con 0x7fc5500ff100 2023-07-21T18:48:35.903 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.884+0000 7fc54dffb700 1 -- 172.21.15.39:0/2322114436 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fc544010860 con 0x7fc5500ff100 2023-07-21T18:48:35.903 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.885+0000 7fc54dffb700 1 --2- 172.21.15.39:0/2322114436 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fc5400a1340 0x7fc5400a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:35.903 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.885+0000 7fc54ffff700 1 --2- 172.21.15.39:0/2322114436 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fc5400a1340 0x7fc5400a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:35.903 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.885+0000 7fc54dffb700 1 -- 172.21.15.39:0/2322114436 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fc544014070 con 0x7fc5500ff100 2023-07-21T18:48:35.903 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.886+0000 7fc54ffff700 1 --2- 172.21.15.39:0/2322114436 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fc5400a1340 0x7fc5400a37f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc5501020a0 tx=0x7fc53c009260 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:48:35.903 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.886+0000 7fc563053b80 1 Processor -- start 2023-07-21T18:48:35.904 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.886+0000 7fc563053b80 1 -- start start 2023-07-21T18:48:35.904 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.886+0000 7fc563053b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5604961496b0 0x5604962f2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:35.904 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.886+0000 7fc563053b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604962f32d0 0x5604962ed040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:35.904 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.886+0000 7fc563053b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604962ed580 0x5604962ed9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:35.904 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.886+0000 7fc563053b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604961f2c50 con 0x5604961496b0 2023-07-21T18:48:35.904 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.886+0000 7fc563053b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604962459e0 con 0x5604962f32d0 2023-07-21T18:48:35.904 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.886+0000 7fc563053b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x56049623ee70 con 0x5604962ed580 2023-07-21T18:48:35.904 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.886+0000 7fc554c69700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5604961496b0 0x5604962f2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:35.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.887+0000 7fc55546a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604962ed580 0x5604962ed9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:35.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.887+0000 7fc554c69700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5604961496b0 0x5604962f2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60084/0 (socket says 172.21.15.39:60084) 2023-07-21T18:48:35.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.887+0000 7fc554c69700 1 -- 172.21.15.39:0/1956706933 learned_addr learned my addr 172.21.15.39:0/1956706933 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:48:35.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.887+0000 7fc54ffff700 1 --2- 172.21.15.39:0/1956706933 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604962f32d0 0x5604962ed040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:35.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.887+0000 7fc55546a700 1 -- 172.21.15.39:0/1956706933 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604962f32d0 msgr2=0x5604962ed040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:48:35.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.887+0000 7fc55546a700 1 --2- 172.21.15.39:0/1956706933 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604962f32d0 0x5604962ed040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:35.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.887+0000 7fc55546a700 1 -- 172.21.15.39:0/1956706933 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5604961496b0 msgr2=0x5604962f2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:48:35.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.887+0000 7fc55546a700 1 --2- 172.21.15.39:0/1956706933 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5604961496b0 0x5604962f2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:35.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.887+0000 7fc55546a700 1 -- 172.21.15.39:0/1956706933 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56049606cd80 con 0x5604962ed580 2023-07-21T18:48:35.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.887+0000 7fc554c69700 1 --2- 172.21.15.39:0/1956706933 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5604961496b0 0x5604962f2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:48:35.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.887+0000 7fc55546a700 1 --2- 172.21.15.39:0/1956706933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604962ed580 0x5604962ed9d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc54800b6c0 tx=0x7fc548007460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e8bf0d750b4428e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:48:35.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.887+0000 7fc53affd700 1 -- 172.21.15.39:0/1956706933 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54800e040 con 0x5604962ed580 2023-07-21T18:48:35.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.887+0000 7fc53affd700 1 -- 172.21.15.39:0/1956706933 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5480092e0 con 0x5604962ed580 2023-07-21T18:48:35.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.887+0000 7fc53affd700 1 -- 172.21.15.39:0/1956706933 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54800bca0 con 0x5604962ed580 2023-07-21T18:48:35.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.888+0000 7fc563053b80 1 -- 172.21.15.39:0/1956706933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604962ed580 msgr2=0x5604962ed9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:48:35.907 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.888+0000 7fc563053b80 1 --2- 172.21.15.39:0/1956706933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604962ed580 0x5604962ed9d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc54800b6c0 tx=0x7fc548007460 comp rx=0 tx=0).stop 2023-07-21T18:48:35.907 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.888+0000 7fc563053b80 1 -- 172.21.15.39:0/1956706933 shutdown_connections 2023-07-21T18:48:35.907 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.888+0000 7fc563053b80 1 --2- 172.21.15.39:0/1956706933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604962ed580 0x5604962ed9d0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:35.907 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.888+0000 7fc563053b80 1 --2- 172.21.15.39:0/1956706933 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5604961496b0 0x5604962f2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:35.907 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.888+0000 7fc563053b80 1 --2- 172.21.15.39:0/1956706933 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604962f32d0 0x5604962ed040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:35.907 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.888+0000 7fc563053b80 1 -- 172.21.15.39:0/1956706933 >> 172.21.15.39:0/1956706933 conn(0x5604962e2be0 msgr2=0x5604962e7f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:48:35.907 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.888+0000 7fc563053b80 1 -- 172.21.15.39:0/1956706933 shutdown_connections 2023-07-21T18:48:35.907 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.888+0000 7fc563053b80 1 -- 172.21.15.39:0/1956706933 wait complete. 2023-07-21T18:48:35.908 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.889+0000 7fc563053b80 1 -- start start 2023-07-21T18:48:35.908 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.889+0000 7fc563053b80 1 Processor -- start 2023-07-21T18:48:35.910 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.889+0000 7fc563053b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604961496b0 0x5604962f9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:35.911 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.889+0000 7fc563053b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5604962ed580 0x5604962f68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:35.911 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.889+0000 7fc563053b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604962f32d0 0x5604962f6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:35.911 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.889+0000 7fc563053b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604961dea60 con 0x5604962ed580 2023-07-21T18:48:35.911 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.889+0000 7fc563053b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604961de760 con 0x5604961496b0 2023-07-21T18:48:35.911 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.889+0000 7fc563053b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604960c28c0 con 0x5604962f32d0 2023-07-21T18:48:35.912 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.889+0000 7fc563053b80 10 client.0 _ll_register_callbacks cb 0x5604961dc700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:48:35.912 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.889+0000 7fc54ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5604962ed580 0x5604962f68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:35.912 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.889+0000 7fc554c69700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604961496b0 0x5604962f9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:35.913 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.889+0000 7fc54ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5604962ed580 0x5604962f68f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60092/0 (socket says 172.21.15.39:60092) 2023-07-21T18:48:35.913 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.889+0000 7fc54ffff700 1 -- 172.21.15.39:0/1982224204 learned_addr learned my addr 172.21.15.39:0/1982224204 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:48:35.913 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.889+0000 7fc554c69700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604961496b0 0x5604962f9560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:44454/0 (socket says 172.21.15.39:44454) 2023-07-21T18:48:35.913 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.889+0000 7fc55546a700 1 --2- 172.21.15.39:0/1982224204 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604962f32d0 0x5604962f6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:35.914 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.890+0000 7fc54ffff700 1 -- 172.21.15.39:0/1982224204 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604962f32d0 msgr2=0x5604962f6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:48:35.914 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.890+0000 7fc54ffff700 1 --2- 172.21.15.39:0/1982224204 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604962f32d0 0x5604962f6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:35.914 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.890+0000 7fc54ffff700 1 -- 172.21.15.39:0/1982224204 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604961496b0 msgr2=0x5604962f9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:48:35.915 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.890+0000 7fc54ffff700 1 --2- 172.21.15.39:0/1982224204 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604961496b0 0x5604962f9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:35.915 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.890+0000 7fc54ffff700 1 -- 172.21.15.39:0/1982224204 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5501a4ea0 con 0x5604962ed580 2023-07-21T18:48:35.915 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.890+0000 7fc54ffff700 1 --2- 172.21.15.39:0/1982224204 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5604962ed580 0x5604962f68f0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fc53c00ed10 tx=0x7fc53c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdceff8dca9f5f87 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:48:35.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.890+0000 7fc52ffff700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:48:35.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.890+0000 7fc52ffff700 1 -- 172.21.15.39:0/1982224204 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc53c022070 con 0x5604962ed580 2023-07-21T18:48:35.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.890+0000 7fc52ffff700 1 -- 172.21.15.39:0/1982224204 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fc53c01dbb0 con 0x5604962ed580 2023-07-21T18:48:35.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.890+0000 7fc563053b80 1 -- 172.21.15.39:0/1982224204 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560496198490 con 0x5604962ed580 2023-07-21T18:48:35.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.890+0000 7fc52ffff700 1 -- 172.21.15.39:0/1982224204 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc53c01dd70 con 0x5604962ed580 2023-07-21T18:48:35.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.890+0000 7fc563053b80 10 client.16653 fetch_fsmap learned FSMap version 48 2023-07-21T18:48:35.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.890+0000 7fc563053b80 1 -- 172.21.15.39:0/1982224204 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x7fc55010c0d0 con 0x5604962ed580 2023-07-21T18:48:35.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.891+0000 7fc52ffff700 1 -- 172.21.15.39:0/1982224204 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560496198490 con 0x5604962ed580 2023-07-21T18:48:35.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.891+0000 7fc52ffff700 1 -- 172.21.15.39:0/1982224204 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 48) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fc53c005580 con 0x5604962ed580 2023-07-21T18:48:35.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.891+0000 7fc563053b80 10 client.16653 fetch_fsmap finished waiting for FSMap version 48 2023-07-21T18:48:35.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.891+0000 7fc563053b80 10 client.16653 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:48:35.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.891+0000 7fc563053b80 20 client.16653 populate_metadata read hostname 'smithi039' 2023-07-21T18:48:35.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.891+0000 7fc563053b80 1 --2- 172.21.15.39:0/1982224204 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x560496306c80 0x560496309090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:35.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.891+0000 7fc563053b80 4 client.16653 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:48:35.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.891+0000 7fc563053b80 1 -- 172.21.15.39:0/1982224204 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560496237e50 con 0x560496306c80 2023-07-21T18:48:35.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.891+0000 7fc554c69700 1 --2- 172.21.15.39:0/1982224204 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x560496306c80 0x560496309090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:35.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.891+0000 7fc554c69700 1 --2- 172.21.15.39:0/1982224204 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x560496306c80 0x560496309090 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b01a59ac6389d92 server_cookie=f365869fcf40960d in_seq=0 out_seq=0 2023-07-21T18:48:35.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.891+0000 7fc52ffff700 10 client.16653 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:48:35.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.892+0000 7fc52ffff700 1 -- 172.21.15.39:0/1982224204 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560496237e50 con 0x560496306c80 2023-07-21T18:48:35.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.892+0000 7fc52ffff700 10 client.16653 handle_command_reply: tid=0 2023-07-21T18:48:35.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.892+0000 7fc563053b80 1 client.16653 shutdown 2023-07-21T18:48:35.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.892+0000 7fc563053b80 1 -- 172.21.15.39:0/1982224204 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5604962ed580 msgr2=0x5604962f68f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:48:35.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.892+0000 7fc563053b80 1 --2- 172.21.15.39:0/1982224204 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5604962ed580 0x5604962f68f0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fc53c00ed10 tx=0x7fc53c0047c0 comp rx=0 tx=0).stop 2023-07-21T18:48:35.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.892+0000 7fc563053b80 1 -- 172.21.15.39:0/1982224204 shutdown_connections 2023-07-21T18:48:35.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.892+0000 7fc563053b80 1 --2- 172.21.15.39:0/1982224204 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x560496306c80 0x560496309090 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:35.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.892+0000 7fc563053b80 1 --2- 172.21.15.39:0/1982224204 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604961496b0 0x5604962f9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:35.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.892+0000 7fc563053b80 1 --2- 172.21.15.39:0/1982224204 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604962f32d0 0x5604962f6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:35.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.892+0000 7fc563053b80 1 --2- 172.21.15.39:0/1982224204 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5604962ed580 0x5604962f68f0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:35.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.892+0000 7fc563053b80 1 -- 172.21.15.39:0/1982224204 >> 172.21.15.39:0/1982224204 conn(0x5604962e2be0 msgr2=0x5604962e6c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:48:35.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.892+0000 7fc52ffff700 0 client.16653 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:48:35.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.893+0000 7fc563053b80 1 -- 172.21.15.39:0/1982224204 shutdown_connections 2023-07-21T18:48:35.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.893+0000 7fc563053b80 1 -- 172.21.15.39:0/1982224204 wait complete. 2023-07-21T18:48:35.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.893+0000 7fc563053b80 20 client.16653 trim_cache size 0 max 16384 2023-07-21T18:48:35.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.911+0000 7fc563053b80 1 Processor -- start 2023-07-21T18:48:35.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.911+0000 7fc563053b80 1 -- start start 2023-07-21T18:48:35.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.911+0000 7fc563053b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604961496b0 0x5604962e7c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:35.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.911+0000 7fc563053b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5604962ed580 0x5604962f9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:35.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.911+0000 7fc563053b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604962f32d0 0x5604962f6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:35.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.911+0000 7fc563053b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604962f7e60 con 0x5604962ed580 2023-07-21T18:48:35.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.911+0000 7fc563053b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560496303520 con 0x5604962f32d0 2023-07-21T18:48:35.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.911+0000 7fc563053b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604962f29c0 con 0x5604961496b0 2023-07-21T18:48:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.911+0000 7fc54ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5604962ed580 0x5604962f9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.912+0000 7fc554c69700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604961496b0 0x5604962e7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.912+0000 7fc554c69700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604961496b0 0x5604962e7c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:50418/0 (socket says 172.21.15.39:50418) 2023-07-21T18:48:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.912+0000 7fc54ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5604962ed580 0x5604962f9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60100/0 (socket says 172.21.15.39:60100) 2023-07-21T18:48:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.912+0000 7fc554c69700 1 -- 172.21.15.39:0/2373200488 learned_addr learned my addr 172.21.15.39:0/2373200488 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:48:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.912+0000 7fc55546a700 1 --2- 172.21.15.39:0/2373200488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604962f32d0 0x5604962f6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.912+0000 7fc554c69700 1 -- 172.21.15.39:0/2373200488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604962f32d0 msgr2=0x5604962f6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:48:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.912+0000 7fc554c69700 1 --2- 172.21.15.39:0/2373200488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604962f32d0 0x5604962f6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.912+0000 7fc554c69700 1 -- 172.21.15.39:0/2373200488 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5604962ed580 msgr2=0x5604962f9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:48:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.912+0000 7fc554c69700 1 --2- 172.21.15.39:0/2373200488 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5604962ed580 0x5604962f9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.912+0000 7fc554c69700 1 -- 172.21.15.39:0/2373200488 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604962edc20 con 0x5604961496b0 2023-07-21T18:48:35.932 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.912+0000 7fc554c69700 1 --2- 172.21.15.39:0/2373200488 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604961496b0 0x5604962e7c60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc544099d20 tx=0x7fc544096150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad5f7646772102e8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:48:35.932 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.912+0000 7fc53b7fe700 1 -- 172.21.15.39:0/2373200488 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5440a21e0 con 0x5604961496b0 2023-07-21T18:48:35.932 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.912+0000 7fc53b7fe700 1 -- 172.21.15.39:0/2373200488 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc544098130 con 0x5604961496b0 2023-07-21T18:48:35.932 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.912+0000 7fc53b7fe700 1 -- 172.21.15.39:0/2373200488 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc544099280 con 0x5604961496b0 2023-07-21T18:48:35.932 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.913+0000 7fc563053b80 1 -- 172.21.15.39:0/2373200488 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604961496b0 msgr2=0x5604962e7c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:48:35.932 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.913+0000 7fc563053b80 1 --2- 172.21.15.39:0/2373200488 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604961496b0 0x5604962e7c60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc544099d20 tx=0x7fc544096150 comp rx=0 tx=0).stop 2023-07-21T18:48:35.932 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.913+0000 7fc563053b80 1 -- 172.21.15.39:0/2373200488 shutdown_connections 2023-07-21T18:48:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.913+0000 7fc563053b80 1 --2- 172.21.15.39:0/2373200488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604962f32d0 0x5604962f6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.913+0000 7fc563053b80 1 --2- 172.21.15.39:0/2373200488 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604961496b0 0x5604962e7c60 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.913+0000 7fc563053b80 1 --2- 172.21.15.39:0/2373200488 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5604962ed580 0x5604962f9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.913+0000 7fc563053b80 1 -- 172.21.15.39:0/2373200488 >> 172.21.15.39:0/2373200488 conn(0x5604962e2be0 msgr2=0x5604962f0b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:48:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.913+0000 7fc563053b80 1 -- 172.21.15.39:0/2373200488 shutdown_connections 2023-07-21T18:48:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.913+0000 7fc563053b80 1 -- 172.21.15.39:0/2373200488 wait complete. 2023-07-21T18:48:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.913+0000 7fc563053b80 1 -- start start 2023-07-21T18:48:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.913+0000 7fc563053b80 1 Processor -- start 2023-07-21T18:48:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.914+0000 7fc563053b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604961496b0 0x5604962fbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.914+0000 7fc563053b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604962ed580 0x5604962fc190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.914+0000 7fc563053b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5604962f32d0 0x5604962fc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.914+0000 7fc563053b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x560496359ba0 con 0x5604962f32d0 2023-07-21T18:48:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.914+0000 7fc563053b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560496359ce0 con 0x5604962ed580 2023-07-21T18:48:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.914+0000 7fc563053b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x560496359ea0 con 0x5604961496b0 2023-07-21T18:48:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.914+0000 7fc563053b80 10 client.0 _ll_register_callbacks cb 0x7fc53c00e8f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:48:35.935 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.914+0000 7fc554c69700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604961496b0 0x5604962fbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:35.935 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.914+0000 7fc554c69700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604961496b0 0x5604962fbc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:50424/0 (socket says 172.21.15.39:50424) 2023-07-21T18:48:35.935 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.914+0000 7fc554c69700 1 -- 172.21.15.39:0/3940671479 learned_addr learned my addr 172.21.15.39:0/3940671479 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:48:35.935 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.914+0000 7fc54ffff700 1 --2- 172.21.15.39:0/3940671479 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604962ed580 0x5604962fc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:35.935 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.914+0000 7fc55546a700 1 --2- 172.21.15.39:0/3940671479 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5604962f32d0 0x5604962fc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:35.935 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.914+0000 7fc554c69700 1 -- 172.21.15.39:0/3940671479 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604962ed580 msgr2=0x5604962fc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:48:35.935 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.914+0000 7fc554c69700 1 --2- 172.21.15.39:0/3940671479 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604962ed580 0x5604962fc190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:35.935 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.914+0000 7fc554c69700 1 -- 172.21.15.39:0/3940671479 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5604962f32d0 msgr2=0x5604962fc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:48:35.936 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.914+0000 7fc554c69700 1 --2- 172.21.15.39:0/3940671479 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5604962f32d0 0x5604962fc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:35.936 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.914+0000 7fc554c69700 1 -- 172.21.15.39:0/3940671479 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5501a4940 con 0x5604961496b0 2023-07-21T18:48:35.936 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.915+0000 7fc554c69700 1 --2- 172.21.15.39:0/3940671479 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604961496b0 0x5604962fbc50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc54400f010 tx=0x7fc544010bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eed6a67e6142a331 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:48:35.936 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.915+0000 7fc52ffff700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2023-07-21T18:48:35.936 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.915+0000 7fc52ffff700 1 -- 172.21.15.39:0/3940671479 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54409a1f0 con 0x5604961496b0 2023-07-21T18:48:35.936 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.915+0000 7fc52ffff700 1 -- 172.21.15.39:0/3940671479 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fc5440a21e0 con 0x5604961496b0 2023-07-21T18:48:35.936 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.915+0000 7fc563053b80 1 -- 172.21.15.39:0/3940671479 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc5480092e0 con 0x5604961496b0 2023-07-21T18:48:35.937 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.915+0000 7fc52ffff700 1 -- 172.21.15.39:0/3940671479 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc5440ca1b0 con 0x5604961496b0 2023-07-21T18:48:35.937 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.915+0000 7fc563053b80 10 client.26143 fetch_fsmap learned FSMap version 48 2023-07-21T18:48:35.937 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.915+0000 7fc563053b80 1 -- 172.21.15.39:0/3940671479 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x560496055c00 con 0x5604961496b0 2023-07-21T18:48:35.937 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.915+0000 7fc52ffff700 1 -- 172.21.15.39:0/3940671479 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5480092e0 con 0x5604961496b0 2023-07-21T18:48:35.937 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.915+0000 7fc52ffff700 1 -- 172.21.15.39:0/3940671479 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 48) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fc5440cab60 con 0x5604961496b0 2023-07-21T18:48:35.937 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.915+0000 7fc563053b80 10 client.26143 fetch_fsmap finished waiting for FSMap version 48 2023-07-21T18:48:35.937 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.915+0000 7fc563053b80 10 client.26143 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:48:35.937 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.915+0000 7fc563053b80 20 client.26143 populate_metadata read hostname 'smithi039' 2023-07-21T18:48:35.938 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.915+0000 7fc563053b80 1 --2- 172.21.15.39:0/3940671479 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x56049635b2f0 0x56049635d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:35.938 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.915+0000 7fc563053b80 4 client.26143 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:48:35.938 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.915+0000 7fc563053b80 1 -- 172.21.15.39:0/3940671479 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5604961f9ff0 con 0x56049635b2f0 2023-07-21T18:48:35.938 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.915+0000 7fc54ffff700 1 --2- 172.21.15.39:0/3940671479 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x56049635b2f0 0x56049635d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:35.938 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.916+0000 7fc52ffff700 10 client.26143 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:48:35.938 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.916+0000 7fc54ffff700 1 --2- 172.21.15.39:0/3940671479 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x56049635b2f0 0x56049635d700 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99a99c79361bd0c9 server_cookie=cfbe3d13e642495d in_seq=0 out_seq=0 2023-07-21T18:48:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.949+0000 7fc52ffff700 1 -- 172.21.15.39:0/3940671479 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5604961f9ff0 con 0x56049635b2f0 2023-07-21T18:48:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.949+0000 7fc52ffff700 10 client.26143 handle_command_reply: tid=0 2023-07-21T18:48:35.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.949+0000 7fc563053b80 1 client.26143 shutdown 2023-07-21T18:48:35.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.949+0000 7fc563053b80 1 -- 172.21.15.39:0/3940671479 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604961496b0 msgr2=0x5604962fbc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:48:35.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.949+0000 7fc563053b80 1 --2- 172.21.15.39:0/3940671479 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604961496b0 0x5604962fbc50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc54400f010 tx=0x7fc544010bc0 comp rx=0 tx=0).stop 2023-07-21T18:48:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:48:35.968 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:48:35.968 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:48:35.968 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:48:35.968 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.950+0000 7fc563053b80 1 -- 172.21.15.39:0/3940671479 shutdown_connections 2023-07-21T18:48:35.968 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.950+0000 7fc563053b80 1 --2- 172.21.15.39:0/3940671479 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x56049635b2f0 0x56049635d700 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:35.968 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.950+0000 7fc563053b80 1 --2- 172.21.15.39:0/3940671479 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5604962f32d0 0x5604962fc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.950+0000 7fc563053b80 1 --2- 172.21.15.39:0/3940671479 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604961496b0 0x5604962fbc50 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.950+0000 7fc563053b80 1 --2- 172.21.15.39:0/3940671479 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604962ed580 0x5604962fc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.950+0000 7fc563053b80 1 -- 172.21.15.39:0/3940671479 >> 172.21.15.39:0/3940671479 conn(0x5604962e2be0 msgr2=0x5604962f1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:48:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.950+0000 7fc52ffff700 0 client.26143 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:48:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.950+0000 7fc563053b80 1 -- 172.21.15.39:0/3940671479 shutdown_connections 2023-07-21T18:48:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.950+0000 7fc563053b80 1 -- 172.21.15.39:0/3940671479 wait complete. 2023-07-21T18:48:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.950+0000 7fc563053b80 20 client.26143 trim_cache size 0 max 16384 2023-07-21T18:48:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.951+0000 7fc556ecd700 1 -- 172.21.15.39:0/2322114436 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fc5400a1340 msgr2=0x7fc5400a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:48:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.951+0000 7fc556ecd700 1 --2- 172.21.15.39:0/2322114436 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fc5400a1340 0x7fc5400a37f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc5501020a0 tx=0x7fc53c009260 comp rx=0 tx=0).stop 2023-07-21T18:48:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.951+0000 7fc556ecd700 1 -- 172.21.15.39:0/2322114436 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5500ff100 msgr2=0x7fc550100880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:48:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.951+0000 7fc556ecd700 1 --2- 172.21.15.39:0/2322114436 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5500ff100 0x7fc550100880 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc54400e9c0 tx=0x7fc54400ecd0 comp rx=0 tx=0).stop 2023-07-21T18:48:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.951+0000 7fc556ecd700 1 -- 172.21.15.39:0/2322114436 shutdown_connections 2023-07-21T18:48:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.951+0000 7fc556ecd700 1 --2- 172.21.15.39:0/2322114436 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fc5400a1340 0x7fc5400a37f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.951+0000 7fc556ecd700 1 --2- 172.21.15.39:0/2322114436 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5500ff100 0x7fc550100880 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.951+0000 7fc556ecd700 1 --2- 172.21.15.39:0/2322114436 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5501039b0 0x7fc5501a3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.951+0000 7fc556ecd700 1 --2- 172.21.15.39:0/2322114436 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc5500ffab0 0x7fc550100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.951+0000 7fc556ecd700 1 -- 172.21.15.39:0/2322114436 >> 172.21.15.39:0/2322114436 conn(0x7fc550075f90 msgr2=0x7fc5500fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:48:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.952+0000 7fc556ecd700 1 -- 172.21.15.39:0/2322114436 shutdown_connections 2023-07-21T18:48:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:35.952+0000 7fc556ecd700 1 -- 172.21.15.39:0/2322114436 wait complete. 2023-07-21T18:48:36.275 INFO:tasks.workunit.client.0.smithi039.stdout: 1 429493 6.51 MB/sec execute 229 sec latency 75.746 ms 2023-07-21T18:48:36.362 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:48:36 smithi193 ceph-mon[98204]: pgmap v211: 97 pgs: 97 active+clean; 159 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 222 op/s 2023-07-21T18:48:36.380 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:48:36 smithi155 ceph-mon[97934]: pgmap v211: 97 pgs: 97 active+clean; 159 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 222 op/s 2023-07-21T18:48:36.429 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:48:36 smithi039 ceph-mon[96489]: pgmap v211: 97 pgs: 97 active+clean; 159 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 222 op/s 2023-07-21T18:48:37.275 INFO:tasks.workunit.client.0.smithi039.stdout: 1 430718 6.51 MB/sec execute 230 sec latency 15.606 ms 2023-07-21T18:48:38.275 INFO:tasks.workunit.client.0.smithi039.stdout: 1 432578 6.52 MB/sec execute 231 sec latency 17.216 ms 2023-07-21T18:48:38.361 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:48:38 smithi193 ceph-mon[98204]: pgmap v212: 97 pgs: 97 active+clean; 163 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 315 op/s 2023-07-21T18:48:38.380 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:48:38 smithi155 ceph-mon[97934]: pgmap v212: 97 pgs: 97 active+clean; 163 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 315 op/s 2023-07-21T18:48:38.429 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:48:38 smithi039 ceph-mon[96489]: pgmap v212: 97 pgs: 97 active+clean; 163 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 315 op/s 2023-07-21T18:48:39.275 INFO:tasks.workunit.client.0.smithi039.stdout: 1 433668 6.51 MB/sec execute 232 sec latency 21.452 ms 2023-07-21T18:48:40.275 INFO:tasks.workunit.client.0.smithi039.stdout: 1 435664 6.53 MB/sec execute 233 sec latency 24.048 ms 2023-07-21T18:48:40.361 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:48:40 smithi193 ceph-mon[98204]: pgmap v213: 97 pgs: 97 active+clean; 163 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 191 op/s 2023-07-21T18:48:40.362 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:48:40 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:48:40.380 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:48:40 smithi155 ceph-mon[97934]: pgmap v213: 97 pgs: 97 active+clean; 163 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 191 op/s 2023-07-21T18:48:40.380 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:48:40 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:48:40.429 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:48:40 smithi039 ceph-mon[96489]: pgmap v213: 97 pgs: 97 active+clean; 163 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 191 op/s 2023-07-21T18:48:40.429 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:48:40 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:48:41.275 INFO:tasks.workunit.client.0.smithi039.stdout: 1 436888 6.54 MB/sec execute 234 sec latency 11.252 ms 2023-07-21T18:48:42.276 INFO:tasks.workunit.client.0.smithi039.stdout: 1 438139 6.53 MB/sec execute 235 sec latency 16.470 ms 2023-07-21T18:48:42.380 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:48:42 smithi155 ceph-mon[97934]: pgmap v214: 97 pgs: 97 active+clean; 162 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 294 op/s 2023-07-21T18:48:42.429 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:48:42 smithi039 ceph-mon[96489]: pgmap v214: 97 pgs: 97 active+clean; 162 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 294 op/s 2023-07-21T18:48:42.611 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:48:42 smithi193 ceph-mon[98204]: pgmap v214: 97 pgs: 97 active+clean; 162 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 294 op/s 2023-07-21T18:48:43.276 INFO:tasks.workunit.client.0.smithi039.stdout: 1 440002 6.56 MB/sec execute 236 sec latency 16.734 ms 2023-07-21T18:48:44.275 INFO:tasks.workunit.client.0.smithi039.stdout: 1 441010 6.54 MB/sec execute 237 sec latency 18.211 ms 2023-07-21T18:48:44.380 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:48:44 smithi155 ceph-mon[97934]: pgmap v215: 97 pgs: 97 active+clean; 162 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 195 op/s 2023-07-21T18:48:44.380 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:48:44 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:48:44.429 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:48:44 smithi039 ceph-mon[96489]: pgmap v215: 97 pgs: 97 active+clean; 162 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 195 op/s 2023-07-21T18:48:44.429 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:48:44 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:48:44.611 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:48:44 smithi193 ceph-mon[98204]: pgmap v215: 97 pgs: 97 active+clean; 162 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 195 op/s 2023-07-21T18:48:44.612 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:48:44 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:48:45.276 INFO:tasks.workunit.client.0.smithi039.stdout: 1 442749 6.56 MB/sec execute 238 sec latency 28.681 ms 2023-07-21T18:48:45.429 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:48:45 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:48:45.612 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:48:45 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:48:45.630 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:48:45 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:48:46.276 INFO:tasks.workunit.client.0.smithi039.stdout: 1 443941 6.56 MB/sec execute 239 sec latency 25.061 ms 2023-07-21T18:48:46.429 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:48:46 smithi039 ceph-mon[96489]: pgmap v216: 97 pgs: 97 active+clean; 162 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 195 op/s 2023-07-21T18:48:46.611 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:48:46 smithi193 ceph-mon[98204]: pgmap v216: 97 pgs: 97 active+clean; 162 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 195 op/s 2023-07-21T18:48:46.630 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:48:46 smithi155 ceph-mon[97934]: pgmap v216: 97 pgs: 97 active+clean; 162 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 195 op/s 2023-07-21T18:48:47.276 INFO:tasks.workunit.client.0.smithi039.stdout: 1 445789 6.56 MB/sec execute 240 sec latency 25.367 ms 2023-07-21T18:48:48.276 INFO:tasks.workunit.client.0.smithi039.stdout: 1 447329 6.59 MB/sec execute 241 sec latency 15.517 ms 2023-07-21T18:48:48.429 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:48:48 smithi039 ceph-mon[96489]: pgmap v217: 97 pgs: 97 active+clean; 169 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 273 op/s 2023-07-21T18:48:48.611 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:48:48 smithi193 ceph-mon[98204]: pgmap v217: 97 pgs: 97 active+clean; 169 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 273 op/s 2023-07-21T18:48:48.630 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:48:48 smithi155 ceph-mon[97934]: pgmap v217: 97 pgs: 97 active+clean; 169 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 273 op/s 2023-07-21T18:48:49.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.068+0000 7faef0b00700 1 -- 172.21.15.39:0/4240138611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faeec104a30 msgr2=0x7faeec104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:48:49.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.068+0000 7faef0b00700 1 --2- 172.21.15.39:0/4240138611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faeec104a30 0x7faeec104e00 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7faedc009990 tx=0x7faedc009ca0 comp rx=0 tx=0).stop 2023-07-21T18:48:49.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.068+0000 7faef0b00700 1 -- 172.21.15.39:0/4240138611 shutdown_connections 2023-07-21T18:48:49.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.068+0000 7faef0b00700 1 --2- 172.21.15.39:0/4240138611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faeec0ff3e0 0x7faeec10b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:49.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.068+0000 7faef0b00700 1 --2- 172.21.15.39:0/4240138611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faeec104a30 0x7faeec104e00 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:49.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.068+0000 7faef0b00700 1 --2- 172.21.15.39:0/4240138611 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faeec0fea30 0x7faeec0feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:49.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.068+0000 7faef0b00700 1 -- 172.21.15.39:0/4240138611 >> 172.21.15.39:0/4240138611 conn(0x7faeec074760 msgr2=0x7faeec074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:48:49.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.068+0000 7faef0b00700 1 -- 172.21.15.39:0/4240138611 shutdown_connections 2023-07-21T18:48:49.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.068+0000 7faef0b00700 1 -- 172.21.15.39:0/4240138611 wait complete. 2023-07-21T18:48:49.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.068+0000 7faef0b00700 1 Processor -- start 2023-07-21T18:48:49.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.069+0000 7faef0b00700 1 -- start start 2023-07-21T18:48:49.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.069+0000 7faef0b00700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faeec0fea30 0x7faeec196c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:49.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.069+0000 7faef0b00700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faeec0ff3e0 0x7faeec1971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:49.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.069+0000 7faef0b00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faeec104a30 0x7faeec1a1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:49.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.069+0000 7faef0b00700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeec1a2100 con 0x7faeec104a30 2023-07-21T18:48:49.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.069+0000 7faef0b00700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeec1a2270 con 0x7faeec0fea30 2023-07-21T18:48:49.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.069+0000 7faef0b00700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeec1a2430 con 0x7faeec0ff3e0 2023-07-21T18:48:49.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.069+0000 7faeea59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faeec0fea30 0x7faeec196c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:49.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.069+0000 7faee3fff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faeec0ff3e0 0x7faeec1971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:49.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.069+0000 7faeead9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faeec104a30 0x7faeec1a1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:49.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.069+0000 7faeea59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faeec0fea30 0x7faeec196c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:44482/0 (socket says 172.21.15.39:44482) 2023-07-21T18:48:49.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.069+0000 7faee3fff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faeec0ff3e0 0x7faeec1971d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:50440/0 (socket says 172.21.15.39:50440) 2023-07-21T18:48:49.088 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.069+0000 7faeea59c700 1 -- 172.21.15.39:0/2253654580 learned_addr learned my addr 172.21.15.39:0/2253654580 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:48:49.088 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.069+0000 7faeea59c700 1 -- 172.21.15.39:0/2253654580 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faeec0ff3e0 msgr2=0x7faeec1971d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:48:49.088 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.069+0000 7faeea59c700 1 --2- 172.21.15.39:0/2253654580 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faeec0ff3e0 0x7faeec1971d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:49.088 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.069+0000 7faeea59c700 1 -- 172.21.15.39:0/2253654580 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faeec104a30 msgr2=0x7faeec1a1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:48:49.088 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.069+0000 7faeea59c700 1 --2- 172.21.15.39:0/2253654580 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faeec104a30 0x7faeec1a1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:49.088 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.069+0000 7faeea59c700 1 -- 172.21.15.39:0/2253654580 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faedc0095f0 con 0x7faeec0fea30 2023-07-21T18:48:49.088 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.069+0000 7faeead9d700 1 --2- 172.21.15.39:0/2253654580 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faeec104a30 0x7faeec1a1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:48:49.089 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.069+0000 7faee3fff700 1 --2- 172.21.15.39:0/2253654580 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faeec0ff3e0 0x7faeec1971d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:48:49.089 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.070+0000 7faeea59c700 1 --2- 172.21.15.39:0/2253654580 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faeec0fea30 0x7faeec196c90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7faedc000c00 tx=0x7faedc005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:48:49.089 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.070+0000 7faee37fe700 1 -- 172.21.15.39:0/2253654580 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faedc01d070 con 0x7faeec0fea30 2023-07-21T18:48:49.089 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.070+0000 7faef0b00700 1 -- 172.21.15.39:0/2253654580 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faeec1a26b0 con 0x7faeec0fea30 2023-07-21T18:48:49.089 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.070+0000 7faee37fe700 1 -- 172.21.15.39:0/2253654580 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faedc021410 con 0x7faeec0fea30 2023-07-21T18:48:49.089 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.070+0000 7faee37fe700 1 -- 172.21.15.39:0/2253654580 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faedc017650 con 0x7faeec0fea30 2023-07-21T18:48:49.089 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.070+0000 7faef0b00700 1 -- 172.21.15.39:0/2253654580 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faeec1a2ad0 con 0x7faeec0fea30 2023-07-21T18:48:49.094 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.072+0000 7faee37fe700 1 -- 172.21.15.39:0/2253654580 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7faedc021580 con 0x7faeec0fea30 2023-07-21T18:48:49.094 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.073+0000 7faee37fe700 1 --2- 172.21.15.39:0/2253654580 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7faed80a12c0 0x7faed80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:49.094 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.073+0000 7faee37fe700 1 -- 172.21.15.39:0/2253654580 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8527+0+0 (secure 0 0 0) 0x7faedc0cdfa0 con 0x7faeec0fea30 2023-07-21T18:48:49.094 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.073+0000 7faee3fff700 1 --2- 172.21.15.39:0/2253654580 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7faed80a12c0 0x7faed80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:49.095 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.073+0000 7faefcc86b80 1 Processor -- start 2023-07-21T18:48:49.095 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.073+0000 7faefcc86b80 1 -- start start 2023-07-21T18:48:49.095 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.073+0000 7faee3fff700 1 --2- 172.21.15.39:0/2253654580 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7faed80a12c0 0x7faed80a3770 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7faeec0683c0 tx=0x7faed4009260 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:48:49.095 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.074+0000 7faefcc86b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fd0badf6b0 0x55fd0bc88d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:49.095 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.074+0000 7faefcc86b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd0bc892d0 0x55fd0bc83040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:49.095 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.074+0000 7faefcc86b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd0bc83580 0x55fd0bc839d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:49.096 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.074+0000 7faefcc86b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd0bb88c50 con 0x55fd0badf6b0 2023-07-21T18:48:49.096 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.074+0000 7faefcc86b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd0bbdb9e0 con 0x55fd0bc83580 2023-07-21T18:48:49.096 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.074+0000 7faefcc86b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd0bbd4e70 con 0x55fd0bc892d0 2023-07-21T18:48:49.096 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.074+0000 7faeea59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fd0badf6b0 0x55fd0bc88d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:49.096 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.074+0000 7faeea59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fd0badf6b0 0x55fd0bc88d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60126/0 (socket says 172.21.15.39:60126) 2023-07-21T18:48:49.096 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.074+0000 7faeea59c700 1 -- 172.21.15.39:0/293231648 learned_addr learned my addr 172.21.15.39:0/293231648 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:48:49.096 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.074+0000 7faee3fff700 1 --2- 172.21.15.39:0/293231648 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd0bc892d0 0x55fd0bc83040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:49.097 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.074+0000 7faeead9d700 1 --2- 172.21.15.39:0/293231648 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd0bc83580 0x55fd0bc839d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:49.097 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.074+0000 7faeea59c700 1 -- 172.21.15.39:0/293231648 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd0bc892d0 msgr2=0x55fd0bc83040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:48:49.097 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.074+0000 7faeea59c700 1 --2- 172.21.15.39:0/293231648 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd0bc892d0 0x55fd0bc83040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:49.097 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.074+0000 7faeea59c700 1 -- 172.21.15.39:0/293231648 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd0bc83580 msgr2=0x55fd0bc839d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:48:49.097 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.074+0000 7faeea59c700 1 --2- 172.21.15.39:0/293231648 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd0bc83580 0x55fd0bc839d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:49.097 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.074+0000 7faeea59c700 1 -- 172.21.15.39:0/293231648 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd0ba02d80 con 0x55fd0badf6b0 2023-07-21T18:48:49.097 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.074+0000 7faee3fff700 1 --2- 172.21.15.39:0/293231648 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd0bc892d0 0x55fd0bc83040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:48:49.098 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.074+0000 7faeea59c700 1 --2- 172.21.15.39:0/293231648 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fd0badf6b0 0x55fd0bc88d90 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7faedc097600 tx=0x7faedc09afd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f18625cb7621d934 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:48:49.098 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.075+0000 7faee0ff9700 1 -- 172.21.15.39:0/293231648 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faedc09b100 con 0x55fd0badf6b0 2023-07-21T18:48:49.098 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.075+0000 7faee0ff9700 1 -- 172.21.15.39:0/293231648 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faedc09f130 con 0x55fd0badf6b0 2023-07-21T18:48:49.098 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.075+0000 7faee0ff9700 1 -- 172.21.15.39:0/293231648 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faedc09a7f0 con 0x55fd0badf6b0 2023-07-21T18:48:49.098 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.075+0000 7faefcc86b80 1 -- 172.21.15.39:0/293231648 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fd0badf6b0 msgr2=0x55fd0bc88d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:48:49.098 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.075+0000 7faefcc86b80 1 --2- 172.21.15.39:0/293231648 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fd0badf6b0 0x55fd0bc88d90 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7faedc097600 tx=0x7faedc09afd0 comp rx=0 tx=0).stop 2023-07-21T18:48:49.098 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.075+0000 7faefcc86b80 1 -- 172.21.15.39:0/293231648 shutdown_connections 2023-07-21T18:48:49.098 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.075+0000 7faefcc86b80 1 --2- 172.21.15.39:0/293231648 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd0bc83580 0x55fd0bc839d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:49.099 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.075+0000 7faefcc86b80 1 --2- 172.21.15.39:0/293231648 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fd0badf6b0 0x55fd0bc88d90 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:49.099 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.075+0000 7faefcc86b80 1 --2- 172.21.15.39:0/293231648 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd0bc892d0 0x55fd0bc83040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:49.099 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.075+0000 7faefcc86b80 1 -- 172.21.15.39:0/293231648 >> 172.21.15.39:0/293231648 conn(0x55fd0bc78be0 msgr2=0x55fd0bc7df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:48:49.099 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.076+0000 7faefcc86b80 1 -- 172.21.15.39:0/293231648 shutdown_connections 2023-07-21T18:48:49.099 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.076+0000 7faefcc86b80 1 -- 172.21.15.39:0/293231648 wait complete. 2023-07-21T18:48:49.099 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.076+0000 7faefcc86b80 1 -- start start 2023-07-21T18:48:49.100 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.076+0000 7faefcc86b80 1 Processor -- start 2023-07-21T18:48:49.100 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.076+0000 7faefcc86b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fd0badf6b0 0x55fd0bc922c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:49.100 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.076+0000 7faefcc86b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd0bc83580 0x55fd0bc92800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:49.100 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.076+0000 7faefcc86b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd0bc892d0 0x55fd0bc8c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:49.100 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.076+0000 7faefcc86b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd0bb74a60 con 0x55fd0badf6b0 2023-07-21T18:48:49.100 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.076+0000 7faefcc86b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd0bb74760 con 0x55fd0bc83580 2023-07-21T18:48:49.100 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.076+0000 7faefcc86b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd0ba588c0 con 0x55fd0bc892d0 2023-07-21T18:48:49.101 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.076+0000 7faefcc86b80 10 client.0 _ll_register_callbacks cb 0x55fd0bb72700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:48:49.101 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.076+0000 7faeea59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fd0badf6b0 0x55fd0bc922c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:49.101 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.076+0000 7faee3fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd0bc83580 0x55fd0bc92800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:49.101 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.076+0000 7faeea59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fd0badf6b0 0x55fd0bc922c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60132/0 (socket says 172.21.15.39:60132) 2023-07-21T18:48:49.101 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.076+0000 7faeea59c700 1 -- 172.21.15.39:0/2382021614 learned_addr learned my addr 172.21.15.39:0/2382021614 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:48:49.101 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.077+0000 7faeead9d700 1 --2- 172.21.15.39:0/2382021614 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd0bc892d0 0x55fd0bc8c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:49.101 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.077+0000 7faeea59c700 1 -- 172.21.15.39:0/2382021614 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd0bc892d0 msgr2=0x55fd0bc8c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:48:49.102 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.077+0000 7faeea59c700 1 --2- 172.21.15.39:0/2382021614 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd0bc892d0 0x55fd0bc8c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:49.102 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.077+0000 7faeea59c700 1 -- 172.21.15.39:0/2382021614 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd0bc83580 msgr2=0x55fd0bc92800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:48:49.102 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.077+0000 7faeea59c700 1 --2- 172.21.15.39:0/2382021614 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd0bc83580 0x55fd0bc92800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:49.102 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.077+0000 7faeea59c700 1 -- 172.21.15.39:0/2382021614 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faed4017070 con 0x55fd0badf6b0 2023-07-21T18:48:49.102 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.077+0000 7faee3fff700 1 --2- 172.21.15.39:0/2382021614 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd0bc83580 0x55fd0bc92800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:48:49.102 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.077+0000 7faeea59c700 1 --2- 172.21.15.39:0/2382021614 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fd0badf6b0 0x55fd0bc922c0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7faedc0cbc70 tx=0x7faedc009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60c9cca6be100727 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:48:49.102 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.077+0000 7faed1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:48:49.103 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.077+0000 7faed1ffb700 1 -- 172.21.15.39:0/2382021614 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faedc09b100 con 0x55fd0badf6b0 2023-07-21T18:48:49.103 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.077+0000 7faed1ffb700 1 -- 172.21.15.39:0/2382021614 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8527+0+0 (secure 0 0 0) 0x7faedc0cbe40 con 0x55fd0badf6b0 2023-07-21T18:48:49.103 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.077+0000 7faefcc86b80 1 -- 172.21.15.39:0/2382021614 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fd0bb2e490 con 0x55fd0badf6b0 2023-07-21T18:48:49.103 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.077+0000 7faed1ffb700 1 -- 172.21.15.39:0/2382021614 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7faedc0a34c0 con 0x55fd0badf6b0 2023-07-21T18:48:49.103 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.077+0000 7faefcc86b80 10 client.16671 fetch_fsmap learned FSMap version 51 2023-07-21T18:48:49.103 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.077+0000 7faefcc86b80 1 -- 172.21.15.39:0/2382021614 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x7faeec1a2ad0 con 0x55fd0badf6b0 2023-07-21T18:48:49.103 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.077+0000 7faed1ffb700 1 -- 172.21.15.39:0/2382021614 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faedc064080 con 0x55fd0badf6b0 2023-07-21T18:48:49.103 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.078+0000 7faed1ffb700 1 -- 172.21.15.39:0/2382021614 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 51) v1 ==== 7474+0+0 (secure 0 0 0) 0x7faedc096ef0 con 0x55fd0badf6b0 2023-07-21T18:48:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.078+0000 7faefcc86b80 10 client.16671 fetch_fsmap finished waiting for FSMap version 51 2023-07-21T18:48:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.078+0000 7faefcc86b80 10 client.16671 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:48:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.078+0000 7faefcc86b80 20 client.16671 populate_metadata read hostname 'smithi039' 2023-07-21T18:48:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.078+0000 7faefcc86b80 1 --2- 172.21.15.39:0/2382021614 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55fd0bc9c360 0x55fd0bc9e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.078+0000 7faefcc86b80 4 client.16671 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:48:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.078+0000 7faefcc86b80 1 -- 172.21.15.39:0/2382021614 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fd0bbcde50 con 0x55fd0bc9c360 2023-07-21T18:48:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.078+0000 7faee3fff700 1 --2- 172.21.15.39:0/2382021614 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55fd0bc9c360 0x55fd0bc9e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.078+0000 7faed1ffb700 10 client.16671 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:48:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.078+0000 7faee3fff700 1 --2- 172.21.15.39:0/2382021614 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55fd0bc9c360 0x55fd0bc9e770 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f7ddfcf83ea8415 server_cookie=d3265e81ab76ff02 in_seq=0 out_seq=0 2023-07-21T18:48:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.079+0000 7faed1ffb700 1 -- 172.21.15.39:0/2382021614 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55fd0bbcde50 con 0x55fd0bc9c360 2023-07-21T18:48:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.079+0000 7faed1ffb700 10 client.16671 handle_command_reply: tid=0 2023-07-21T18:48:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.079+0000 7faefcc86b80 1 client.16671 shutdown 2023-07-21T18:48:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.079+0000 7faefcc86b80 1 -- 172.21.15.39:0/2382021614 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fd0badf6b0 msgr2=0x55fd0bc922c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:48:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.079+0000 7faefcc86b80 1 --2- 172.21.15.39:0/2382021614 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fd0badf6b0 0x55fd0bc922c0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7faedc0cbc70 tx=0x7faedc009ca0 comp rx=0 tx=0).stop 2023-07-21T18:48:49.106 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.079+0000 7faefcc86b80 1 -- 172.21.15.39:0/2382021614 shutdown_connections 2023-07-21T18:48:49.106 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.079+0000 7faefcc86b80 1 --2- 172.21.15.39:0/2382021614 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55fd0bc9c360 0x55fd0bc9e770 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:49.106 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.079+0000 7faefcc86b80 1 --2- 172.21.15.39:0/2382021614 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd0bc892d0 0x55fd0bc8c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:49.106 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.079+0000 7faefcc86b80 1 --2- 172.21.15.39:0/2382021614 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fd0badf6b0 0x55fd0bc922c0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:49.106 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.079+0000 7faefcc86b80 1 --2- 172.21.15.39:0/2382021614 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd0bc83580 0x55fd0bc92800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:49.106 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.079+0000 7faed1ffb700 0 client.16671 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:48:49.106 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.079+0000 7faefcc86b80 1 -- 172.21.15.39:0/2382021614 >> 172.21.15.39:0/2382021614 conn(0x55fd0bc78be0 msgr2=0x55fd0bc7d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:48:49.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.079+0000 7faefcc86b80 1 -- 172.21.15.39:0/2382021614 shutdown_connections 2023-07-21T18:48:49.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.079+0000 7faefcc86b80 1 -- 172.21.15.39:0/2382021614 wait complete. 2023-07-21T18:48:49.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.079+0000 7faefcc86b80 20 client.16671 trim_cache size 0 max 16384 2023-07-21T18:48:49.112 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.096+0000 7faefcc86b80 1 Processor -- start 2023-07-21T18:48:49.112 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.097+0000 7faefcc86b80 1 -- start start 2023-07-21T18:48:49.112 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.097+0000 7faefcc86b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd0badf6b0 0x55fd0bc8a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:49.112 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.097+0000 7faefcc86b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fd0bc83580 0x55fd0bc8c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:49.113 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.097+0000 7faefcc86b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd0bc892d0 0x55fd0bc7dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:49.113 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.097+0000 7faefcc86b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd0bc7ebb0 con 0x55fd0bc83580 2023-07-21T18:48:49.113 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.097+0000 7faefcc86b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd0bc7f800 con 0x55fd0bc892d0 2023-07-21T18:48:49.113 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.097+0000 7faefcc86b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd0bc7fb70 con 0x55fd0badf6b0 2023-07-21T18:48:49.113 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.097+0000 7faee3fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fd0bc83580 0x55fd0bc8c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:49.113 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.097+0000 7faee3fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fd0bc83580 0x55fd0bc8c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60142/0 (socket says 172.21.15.39:60142) 2023-07-21T18:48:49.113 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.097+0000 7faee3fff700 1 -- 172.21.15.39:0/3935834628 learned_addr learned my addr 172.21.15.39:0/3935834628 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:48:49.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.097+0000 7faeea59c700 1 --2- 172.21.15.39:0/3935834628 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd0badf6b0 0x55fd0bc8a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:49.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.097+0000 7faeead9d700 1 --2- 172.21.15.39:0/3935834628 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd0bc892d0 0x55fd0bc7dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:49.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.097+0000 7faee3fff700 1 -- 172.21.15.39:0/3935834628 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd0badf6b0 msgr2=0x55fd0bc8a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:48:49.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.097+0000 7faee3fff700 1 --2- 172.21.15.39:0/3935834628 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd0badf6b0 0x55fd0bc8a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:49.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.097+0000 7faee3fff700 1 -- 172.21.15.39:0/3935834628 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd0bc892d0 msgr2=0x55fd0bc7dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:48:49.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.097+0000 7faee3fff700 1 --2- 172.21.15.39:0/3935834628 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd0bc892d0 0x55fd0bc7dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:49.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.097+0000 7faee3fff700 1 -- 172.21.15.39:0/3935834628 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd0b9ebc00 con 0x55fd0bc83580 2023-07-21T18:48:49.117 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.098+0000 7faee3fff700 1 --2- 172.21.15.39:0/3935834628 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fd0bc83580 0x55fd0bc8c430 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7faed4013010 tx=0x7faed4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eb6a092d4f604a6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:48:49.118 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.098+0000 7faee17fa700 1 -- 172.21.15.39:0/3935834628 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed4024040 con 0x55fd0bc83580 2023-07-21T18:48:49.118 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.098+0000 7faee17fa700 1 -- 172.21.15.39:0/3935834628 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faed40295b0 con 0x55fd0bc83580 2023-07-21T18:48:49.118 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.098+0000 7faee17fa700 1 -- 172.21.15.39:0/3935834628 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed401ed90 con 0x55fd0bc83580 2023-07-21T18:48:49.118 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.098+0000 7faefcc86b80 1 -- 172.21.15.39:0/3935834628 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fd0bc83580 msgr2=0x55fd0bc8c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:48:49.118 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.098+0000 7faefcc86b80 1 --2- 172.21.15.39:0/3935834628 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fd0bc83580 0x55fd0bc8c430 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7faed4013010 tx=0x7faed4014000 comp rx=0 tx=0).stop 2023-07-21T18:48:49.118 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.098+0000 7faefcc86b80 1 -- 172.21.15.39:0/3935834628 shutdown_connections 2023-07-21T18:48:49.118 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.098+0000 7faefcc86b80 1 --2- 172.21.15.39:0/3935834628 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd0bc892d0 0x55fd0bc7dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.098+0000 7faefcc86b80 1 --2- 172.21.15.39:0/3935834628 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd0badf6b0 0x55fd0bc8a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.098+0000 7faefcc86b80 1 --2- 172.21.15.39:0/3935834628 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fd0bc83580 0x55fd0bc8c430 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.098+0000 7faefcc86b80 1 -- 172.21.15.39:0/3935834628 >> 172.21.15.39:0/3935834628 conn(0x55fd0bc78be0 msgr2=0x55fd0bc87870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:48:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.099+0000 7faefcc86b80 1 -- 172.21.15.39:0/3935834628 shutdown_connections 2023-07-21T18:48:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.099+0000 7faefcc86b80 1 -- 172.21.15.39:0/3935834628 wait complete. 2023-07-21T18:48:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.099+0000 7faefcc86b80 1 -- start start 2023-07-21T18:48:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.099+0000 7faefcc86b80 1 Processor -- start 2023-07-21T18:48:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.099+0000 7faefcc86b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd0badf6b0 0x55fd0bc90450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.099+0000 7faefcc86b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd0bc83580 0x55fd0bc90990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.099+0000 7faefcc86b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fd0bc892d0 0x55fd0bcef1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.099+0000 7faefcc86b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd0bcef720 con 0x55fd0bc892d0 2023-07-21T18:48:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.099+0000 7faefcc86b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd0bcef860 con 0x55fd0bc83580 2023-07-21T18:48:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.099+0000 7faefcc86b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd0bcef9a0 con 0x55fd0badf6b0 2023-07-21T18:48:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.099+0000 7faefcc86b80 10 client.0 _ll_register_callbacks cb 0x7faed4004030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:48:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.099+0000 7faee3fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd0bc83580 0x55fd0bc90990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.099+0000 7faeead9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fd0bc892d0 0x55fd0bcef1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.099+0000 7faeea59c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd0badf6b0 0x55fd0bc90450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.099+0000 7faee3fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd0bc83580 0x55fd0bc90990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:44512/0 (socket says 172.21.15.39:44512) 2023-07-21T18:48:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.099+0000 7faee3fff700 1 -- 172.21.15.39:0/1899771850 learned_addr learned my addr 172.21.15.39:0/1899771850 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:48:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.100+0000 7faeead9d700 1 -- 172.21.15.39:0/1899771850 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd0badf6b0 msgr2=0x55fd0bc90450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:48:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.100+0000 7faeead9d700 1 --2- 172.21.15.39:0/1899771850 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd0badf6b0 0x55fd0bc90450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:49.122 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.100+0000 7faeead9d700 1 -- 172.21.15.39:0/1899771850 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd0bc83580 msgr2=0x55fd0bc90990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:48:49.122 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.100+0000 7faeead9d700 1 --2- 172.21.15.39:0/1899771850 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd0bc83580 0x55fd0bc90990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:49.122 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.100+0000 7faeead9d700 1 -- 172.21.15.39:0/1899771850 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faed40038e0 con 0x55fd0bc892d0 2023-07-21T18:48:49.122 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.100+0000 7faeea59c700 1 --2- 172.21.15.39:0/1899771850 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd0badf6b0 0x55fd0bc90450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:48:49.122 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.100+0000 7faee3fff700 1 --2- 172.21.15.39:0/1899771850 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd0bc83580 0x55fd0bc90990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:48:49.122 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.100+0000 7faeead9d700 1 --2- 172.21.15.39:0/1899771850 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fd0bc892d0 0x55fd0bcef1e0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7faee40042c0 tx=0x7faee40045d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d2d9fd567fc2c73 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:48:49.122 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.100+0000 7faed1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:48:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.100+0000 7faed1ffb700 1 -- 172.21.15.39:0/1899771850 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee4007880 con 0x55fd0bc892d0 2023-07-21T18:48:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.100+0000 7faed1ffb700 1 -- 172.21.15.39:0/1899771850 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8527+0+0 (secure 0 0 0) 0x7faee40079e0 con 0x55fd0bc892d0 2023-07-21T18:48:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.100+0000 7faefcc86b80 1 -- 172.21.15.39:0/1899771850 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faedc09f130 con 0x55fd0bc892d0 2023-07-21T18:48:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.100+0000 7faed1ffb700 1 -- 172.21.15.39:0/1899771850 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7faee4007ba0 con 0x55fd0bc892d0 2023-07-21T18:48:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.100+0000 7faefcc86b80 10 client.16677 fetch_fsmap learned FSMap version 51 2023-07-21T18:48:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.100+0000 7faefcc86b80 1 -- 172.21.15.39:0/1899771850 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x55fd0ba291f0 con 0x55fd0bc892d0 2023-07-21T18:48:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.101+0000 7faed1ffb700 1 -- 172.21.15.39:0/1899771850 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faedc09f130 con 0x55fd0bc892d0 2023-07-21T18:48:49.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.101+0000 7faed1ffb700 1 -- 172.21.15.39:0/1899771850 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 51) v1 ==== 7474+0+0 (secure 0 0 0) 0x7faee4029480 con 0x55fd0bc892d0 2023-07-21T18:48:49.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.101+0000 7faefcc86b80 10 client.16677 fetch_fsmap finished waiting for FSMap version 51 2023-07-21T18:48:49.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.101+0000 7faefcc86b80 10 client.16677 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:48:49.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.101+0000 7faefcc86b80 20 client.16677 populate_metadata read hostname 'smithi039' 2023-07-21T18:48:49.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.101+0000 7faefcc86b80 1 --2- 172.21.15.39:0/1899771850 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55fd0bcf1130 0x55fd0bcf3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:48:49.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.101+0000 7faefcc86b80 4 client.16677 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:48:49.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.101+0000 7faefcc86b80 1 -- 172.21.15.39:0/1899771850 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fd0bb8fff0 con 0x55fd0bcf1130 2023-07-21T18:48:49.125 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.101+0000 7faeea59c700 1 --2- 172.21.15.39:0/1899771850 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55fd0bcf1130 0x55fd0bcf3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:48:49.125 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.101+0000 7faed1ffb700 10 client.16677 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:48:49.125 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.101+0000 7faeea59c700 1 --2- 172.21.15.39:0/1899771850 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55fd0bcf1130 0x55fd0bcf3540 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1dfa32c1b24c527b server_cookie=24b9c2e42c6daae0 in_seq=0 out_seq=0 2023-07-21T18:48:49.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.148+0000 7faed1ffb700 1 -- 172.21.15.39:0/1899771850 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fd0bb8fff0 con 0x55fd0bcf1130 2023-07-21T18:48:49.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.148+0000 7faed1ffb700 10 client.16677 handle_command_reply: tid=0 2023-07-21T18:48:49.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.149+0000 7faefcc86b80 1 client.16677 shutdown 2023-07-21T18:48:49.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.149+0000 7faefcc86b80 1 -- 172.21.15.39:0/1899771850 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fd0bc892d0 msgr2=0x55fd0bcef1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:48:49.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.149+0000 7faefcc86b80 1 --2- 172.21.15.39:0/1899771850 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fd0bc892d0 0x55fd0bcef1e0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7faee40042c0 tx=0x7faee40045d0 comp rx=0 tx=0).stop 2023-07-21T18:48:49.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.149+0000 7faefcc86b80 1 -- 172.21.15.39:0/1899771850 shutdown_connections 2023-07-21T18:48:49.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.149+0000 7faefcc86b80 1 --2- 172.21.15.39:0/1899771850 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55fd0bcf1130 0x55fd0bcf3540 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:49.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.149+0000 7faefcc86b80 1 --2- 172.21.15.39:0/1899771850 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fd0bc892d0 0x55fd0bcef1e0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:49.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.149+0000 7faed1ffb700 0 client.16677 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:48:49.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.149+0000 7faefcc86b80 1 --2- 172.21.15.39:0/1899771850 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd0badf6b0 0x55fd0bc90450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:49.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.149+0000 7faefcc86b80 1 --2- 172.21.15.39:0/1899771850 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd0bc83580 0x55fd0bc90990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:49.166 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.149+0000 7faefcc86b80 1 -- 172.21.15.39:0/1899771850 >> 172.21.15.39:0/1899771850 conn(0x55fd0bc78be0 msgr2=0x55fd0bc87870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:48:49.166 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.149+0000 7faefcc86b80 1 -- 172.21.15.39:0/1899771850 shutdown_connections 2023-07-21T18:48:49.166 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.149+0000 7faefcc86b80 1 -- 172.21.15.39:0/1899771850 wait complete. 2023-07-21T18:48:49.166 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.149+0000 7faefcc86b80 20 client.16677 trim_cache size 0 max 16384 2023-07-21T18:48:49.166 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:48:49.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:48:49.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:48:49.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:48:49.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.149+0000 7faef0b00700 1 -- 172.21.15.39:0/2253654580 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7faed80a12c0 msgr2=0x7faed80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:48:49.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.149+0000 7faef0b00700 1 --2- 172.21.15.39:0/2253654580 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7faed80a12c0 0x7faed80a3770 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7faeec0683c0 tx=0x7faed4009260 comp rx=0 tx=0).stop 2023-07-21T18:48:49.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.150+0000 7faef0b00700 1 -- 172.21.15.39:0/2253654580 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faeec0fea30 msgr2=0x7faeec196c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:48:49.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.150+0000 7faef0b00700 1 --2- 172.21.15.39:0/2253654580 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faeec0fea30 0x7faeec196c90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7faedc000c00 tx=0x7faedc005740 comp rx=0 tx=0).stop 2023-07-21T18:48:49.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.150+0000 7faef0b00700 1 -- 172.21.15.39:0/2253654580 shutdown_connections 2023-07-21T18:48:49.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.150+0000 7faef0b00700 1 --2- 172.21.15.39:0/2253654580 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7faed80a12c0 0x7faed80a3770 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:49.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.150+0000 7faef0b00700 1 --2- 172.21.15.39:0/2253654580 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faeec0fea30 0x7faeec196c90 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.150+0000 7faef0b00700 1 --2- 172.21.15.39:0/2253654580 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faeec104a30 0x7faeec1a1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.150+0000 7faef0b00700 1 --2- 172.21.15.39:0/2253654580 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faeec0ff3e0 0x7faeec1971d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:48:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.150+0000 7faef0b00700 1 -- 172.21.15.39:0/2253654580 >> 172.21.15.39:0/2253654580 conn(0x7faeec074760 msgr2=0x7faeec102650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:48:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.150+0000 7faef0b00700 1 -- 172.21.15.39:0/2253654580 shutdown_connections 2023-07-21T18:48:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:48:49.150+0000 7faef0b00700 1 -- 172.21.15.39:0/2253654580 wait complete. 2023-07-21T18:48:49.276 INFO:tasks.workunit.client.0.smithi039.stdout: 1 448476 6.58 MB/sec execute 242 sec latency 17.390 ms 2023-07-21T18:48:50.276 INFO:tasks.workunit.client.0.smithi039.stdout: 1 450113 6.59 MB/sec execute 243 sec latency 23.256 ms 2023-07-21T18:48:50.429 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:48:50 smithi039 ceph-mon[96489]: pgmap v218: 97 pgs: 97 active+clean; 169 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 180 op/s 2023-07-21T18:48:50.611 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:48:50 smithi193 ceph-mon[98204]: pgmap v218: 97 pgs: 97 active+clean; 169 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 180 op/s 2023-07-21T18:48:50.630 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:48:50 smithi155 ceph-mon[97934]: pgmap v218: 97 pgs: 97 active+clean; 169 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 180 op/s 2023-07-21T18:48:51.276 INFO:tasks.workunit.client.0.smithi039.stdout: 1 451217 6.58 MB/sec execute 244 sec latency 29.728 ms 2023-07-21T18:48:52.276 INFO:tasks.workunit.client.0.smithi039.stdout: 1 453261 6.60 MB/sec execute 245 sec latency 22.880 ms 2023-07-21T18:48:52.429 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:48:52 smithi039 ceph-mon[96489]: pgmap v219: 97 pgs: 97 active+clean; 167 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 296 op/s 2023-07-21T18:48:52.611 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:48:52 smithi193 ceph-mon[98204]: pgmap v219: 97 pgs: 97 active+clean; 167 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 296 op/s 2023-07-21T18:48:52.630 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:48:52 smithi155 ceph-mon[97934]: pgmap v219: 97 pgs: 97 active+clean; 167 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 296 op/s 2023-07-21T18:48:53.276 INFO:tasks.workunit.client.0.smithi039.stdout: 1 454640 6.61 MB/sec execute 246 sec latency 14.681 ms 2023-07-21T18:48:54.276 INFO:tasks.workunit.client.0.smithi039.stdout: 1 455909 6.60 MB/sec execute 247 sec latency 17.918 ms 2023-07-21T18:48:54.429 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:48:54 smithi039 ceph-mon[96489]: pgmap v220: 97 pgs: 97 active+clean; 167 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 193 op/s 2023-07-21T18:48:54.611 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:48:54 smithi193 ceph-mon[98204]: pgmap v220: 97 pgs: 97 active+clean; 167 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 193 op/s 2023-07-21T18:48:54.630 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:48:54 smithi155 ceph-mon[97934]: pgmap v220: 97 pgs: 97 active+clean; 167 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 193 op/s 2023-07-21T18:48:55.276 INFO:tasks.workunit.client.0.smithi039.stdout: 1 457823 6.63 MB/sec execute 248 sec latency 29.619 ms 2023-07-21T18:48:56.276 INFO:tasks.workunit.client.0.smithi039.stdout: 1 458491 6.63 MB/sec execute 249 sec latency 76.998 ms 2023-07-21T18:48:56.429 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:48:56 smithi039 ceph-mon[96489]: pgmap v221: 97 pgs: 97 active+clean; 167 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 193 op/s 2023-07-21T18:48:56.429 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:48:56 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:48:56.611 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:48:56 smithi193 ceph-mon[98204]: pgmap v221: 97 pgs: 97 active+clean; 167 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 193 op/s 2023-07-21T18:48:56.612 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:48:56 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:48:56.630 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:48:56 smithi155 ceph-mon[97934]: pgmap v221: 97 pgs: 97 active+clean; 167 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 193 op/s 2023-07-21T18:48:56.631 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:48:56 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:48:57.276 INFO:tasks.workunit.client.0.smithi039.stdout: 1 459830 6.68 MB/sec execute 250 sec latency 26.437 ms 2023-07-21T18:48:57.429 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:48:57 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:48:57.429 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:48:57 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:48:57.429 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:48:57 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:48:57.611 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:48:57 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:48:57.612 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:48:57 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:48:57.612 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:48:57 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:48:57.630 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:48:57 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:48:57.630 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:48:57 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:48:57.631 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:48:57 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:48:58.276 INFO:tasks.workunit.client.0.smithi039.stdout: 1 461977 6.71 MB/sec execute 251 sec latency 31.822 ms 2023-07-21T18:48:58.429 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:48:58 smithi039 ceph-mon[96489]: pgmap v222: 97 pgs: 97 active+clean; 173 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 302 op/s 2023-07-21T18:48:58.611 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:48:58 smithi193 ceph-mon[98204]: pgmap v222: 97 pgs: 97 active+clean; 173 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 302 op/s 2023-07-21T18:48:58.630 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:48:58 smithi155 ceph-mon[97934]: pgmap v222: 97 pgs: 97 active+clean; 173 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 302 op/s 2023-07-21T18:48:59.276 INFO:tasks.workunit.client.0.smithi039.stdout: 1 463305 6.71 MB/sec execute 252 sec latency 26.110 ms 2023-07-21T18:48:59.679 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:48:59 smithi039 ceph-mon[96489]: pgmap v223: 97 pgs: 97 active+clean; 173 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 225 op/s 2023-07-21T18:48:59.679 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:48:59 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:48:59.861 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:48:59 smithi193 ceph-mon[98204]: pgmap v223: 97 pgs: 97 active+clean; 173 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 225 op/s 2023-07-21T18:48:59.862 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:48:59 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:48:59.880 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:48:59 smithi155 ceph-mon[97934]: pgmap v223: 97 pgs: 97 active+clean; 173 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 225 op/s 2023-07-21T18:48:59.881 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:48:59 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:49:00.276 INFO:tasks.workunit.client.0.smithi039.stdout: 1 463305 6.68 MB/sec execute 253 sec latency 1026.179 ms 2023-07-21T18:49:01.276 INFO:tasks.workunit.client.0.smithi039.stdout: 1 463305 6.65 MB/sec execute 254 sec latency 2026.270 ms 2023-07-21T18:49:01.679 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:49:01 smithi039 ceph-mon[96489]: pgmap v224: 97 pgs: 97 active+clean; 178 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 319 op/s 2023-07-21T18:49:01.679 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:49:01 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:49:01.861 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:49:01 smithi193 ceph-mon[98204]: pgmap v224: 97 pgs: 97 active+clean; 178 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 319 op/s 2023-07-21T18:49:01.862 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:49:01 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:49:01.880 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:49:01 smithi155 ceph-mon[97934]: pgmap v224: 97 pgs: 97 active+clean; 178 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 319 op/s 2023-07-21T18:49:01.880 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:49:01 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:49:02.276 INFO:tasks.workunit.client.0.smithi039.stdout: 1 463305 6.63 MB/sec execute 255 sec latency 3026.315 ms 2023-07-21T18:49:02.278 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.264+0000 7f062883e700 1 -- 172.21.15.39:0/3175319343 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0624100500 msgr2=0x7f062410c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:49:02.278 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.264+0000 7f062883e700 1 --2- 172.21.15.39:0/3175319343 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0624100500 0x7f062410c940 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f061c00b0a0 tx=0x7f061c00b3b0 comp rx=0 tx=0).stop 2023-07-21T18:49:02.278 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.264+0000 7f062883e700 1 -- 172.21.15.39:0/3175319343 shutdown_connections 2023-07-21T18:49:02.278 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.264+0000 7f062883e700 1 --2- 172.21.15.39:0/3175319343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0624105b80 0x7f0624105f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:02.278 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.264+0000 7f062883e700 1 --2- 172.21.15.39:0/3175319343 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0624100500 0x7f062410c940 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:02.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.264+0000 7f062883e700 1 --2- 172.21.15.39:0/3175319343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06240ffb50 0x7f06240fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:02.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.264+0000 7f062883e700 1 -- 172.21.15.39:0/3175319343 >> 172.21.15.39:0/3175319343 conn(0x7f06240fb650 msgr2=0x7f06240fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:49:02.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.264+0000 7f062883e700 1 -- 172.21.15.39:0/3175319343 shutdown_connections 2023-07-21T18:49:02.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.264+0000 7f062883e700 1 -- 172.21.15.39:0/3175319343 wait complete. 2023-07-21T18:49:02.281 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.265+0000 7f062883e700 1 Processor -- start 2023-07-21T18:49:02.281 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.265+0000 7f062883e700 1 -- start start 2023-07-21T18:49:02.281 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.265+0000 7f062883e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06240ffb50 0x7f06240729f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:02.281 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.265+0000 7f062883e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0624100500 0x7f0624072f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:02.281 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.265+0000 7f062883e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0624105b80 0x7f06241a3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:02.282 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.265+0000 7f062883e700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06241a43c0 con 0x7f0624100500 2023-07-21T18:49:02.282 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.265+0000 7f062883e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06241a4500 con 0x7f06240ffb50 2023-07-21T18:49:02.282 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.265+0000 7f062883e700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06241a4670 con 0x7f0624105b80 2023-07-21T18:49:02.282 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.265+0000 7f0621d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0624100500 0x7f0624072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:02.282 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.265+0000 7f0621d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0624100500 0x7f0624072f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60162/0 (socket says 172.21.15.39:60162) 2023-07-21T18:49:02.282 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.265+0000 7f0621d9b700 1 -- 172.21.15.39:0/4092151193 learned_addr learned my addr 172.21.15.39:0/4092151193 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:49:02.282 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.265+0000 7f0622d9d700 1 --2- 172.21.15.39:0/4092151193 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0624105b80 0x7f06241a3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:02.283 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.265+0000 7f062259c700 1 --2- 172.21.15.39:0/4092151193 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06240ffb50 0x7f06240729f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:02.283 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.266+0000 7f0621d9b700 1 -- 172.21.15.39:0/4092151193 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0624105b80 msgr2=0x7f06241a3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:49:02.283 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.266+0000 7f0621d9b700 1 --2- 172.21.15.39:0/4092151193 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0624105b80 0x7f06241a3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:02.283 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.266+0000 7f0621d9b700 1 -- 172.21.15.39:0/4092151193 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06240ffb50 msgr2=0x7f06240729f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:49:02.283 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.266+0000 7f0621d9b700 1 --2- 172.21.15.39:0/4092151193 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06240ffb50 0x7f06240729f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:02.283 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.266+0000 7f0621d9b700 1 -- 172.21.15.39:0/4092151193 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f061c009d00 con 0x7f0624100500 2023-07-21T18:49:02.283 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.266+0000 7f062259c700 1 --2- 172.21.15.39:0/4092151193 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06240ffb50 0x7f06240729f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:49:02.284 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.266+0000 7f0621d9b700 1 --2- 172.21.15.39:0/4092151193 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0624100500 0x7f0624072f30 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f061400d950 tx=0x7f061400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:49:02.284 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.266+0000 7f06137fe700 1 -- 172.21.15.39:0/4092151193 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0614009940 con 0x7f0624100500 2023-07-21T18:49:02.284 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.266+0000 7f062883e700 1 -- 172.21.15.39:0/4092151193 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06241a4950 con 0x7f0624100500 2023-07-21T18:49:02.284 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.266+0000 7f06137fe700 1 -- 172.21.15.39:0/4092151193 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0614010460 con 0x7f0624100500 2023-07-21T18:49:02.284 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.266+0000 7f06137fe700 1 -- 172.21.15.39:0/4092151193 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061400f620 con 0x7f0624100500 2023-07-21T18:49:02.284 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.266+0000 7f062883e700 1 -- 172.21.15.39:0/4092151193 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06241a4e60 con 0x7f0624100500 2023-07-21T18:49:02.288 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.268+0000 7f06137fe700 1 -- 172.21.15.39:0/4092151193 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f06140108d0 con 0x7f0624100500 2023-07-21T18:49:02.288 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.269+0000 7f06137fe700 1 --2- 172.21.15.39:0/4092151193 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f060c0a12c0 0x7f060c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:02.289 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.269+0000 7f06137fe700 1 -- 172.21.15.39:0/4092151193 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f06140ccba0 con 0x7f0624100500 2023-07-21T18:49:02.289 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.269+0000 7f062259c700 1 --2- 172.21.15.39:0/4092151193 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f060c0a12c0 0x7f060c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:02.289 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.269+0000 7f062259c700 1 --2- 172.21.15.39:0/4092151193 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f060c0a12c0 0x7f060c0a3770 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0618009cb0 tx=0x7f0618009260 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:49:02.289 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.270+0000 7f06349c4b80 1 Processor -- start 2023-07-21T18:49:02.289 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.270+0000 7f06349c4b80 1 -- start start 2023-07-21T18:49:02.289 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.270+0000 7f06349c4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f4bc626b0 0x557f4be0bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:02.289 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.270+0000 7f06349c4b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f4be0c2d0 0x557f4be06040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.270+0000 7f06349c4b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f4be06580 0x557f4be069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.270+0000 7f06349c4b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f4bd0bc50 con 0x557f4be06580 2023-07-21T18:49:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.270+0000 7f06349c4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f4bd5e9e0 con 0x557f4bc626b0 2023-07-21T18:49:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.270+0000 7f06349c4b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f4bd57e70 con 0x557f4be0c2d0 2023-07-21T18:49:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.270+0000 7f0622d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f4be06580 0x557f4be069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.270+0000 7f062259c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f4bc626b0 0x557f4be0bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.270+0000 7f0622d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f4be06580 0x557f4be069d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60172/0 (socket says 172.21.15.39:60172) 2023-07-21T18:49:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.270+0000 7f0622d9d700 1 -- 172.21.15.39:0/1495234418 learned_addr learned my addr 172.21.15.39:0/1495234418 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:49:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.270+0000 7f0621d9b700 1 --2- 172.21.15.39:0/1495234418 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f4be0c2d0 0x557f4be06040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.271+0000 7f062259c700 1 -- 172.21.15.39:0/1495234418 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f4be0c2d0 msgr2=0x557f4be06040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:49:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.271+0000 7f062259c700 1 --2- 172.21.15.39:0/1495234418 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f4be0c2d0 0x557f4be06040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.271+0000 7f062259c700 1 -- 172.21.15.39:0/1495234418 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f4be06580 msgr2=0x557f4be069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:49:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.271+0000 7f062259c700 1 --2- 172.21.15.39:0/1495234418 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f4be06580 0x557f4be069d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.271+0000 7f062259c700 1 -- 172.21.15.39:0/1495234418 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f4bb85d80 con 0x557f4bc626b0 2023-07-21T18:49:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.271+0000 7f0622d9d700 1 --2- 172.21.15.39:0/1495234418 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f4be06580 0x557f4be069d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:49:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.271+0000 7f0621d9b700 1 --2- 172.21.15.39:0/1495234418 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f4be0c2d0 0x557f4be06040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:49:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.271+0000 7f062259c700 1 --2- 172.21.15.39:0/1495234418 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f4bc626b0 0x557f4be0bd90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0618011f80 tx=0x7f0618014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c10019a357292893 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:49:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.271+0000 7f0610ff9700 1 -- 172.21.15.39:0/1495234418 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0618009a40 con 0x557f4bc626b0 2023-07-21T18:49:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.271+0000 7f0610ff9700 1 -- 172.21.15.39:0/1495234418 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0618003b40 con 0x557f4bc626b0 2023-07-21T18:49:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.271+0000 7f0610ff9700 1 -- 172.21.15.39:0/1495234418 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0618012be0 con 0x557f4bc626b0 2023-07-21T18:49:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.272+0000 7f06349c4b80 1 -- 172.21.15.39:0/1495234418 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f4bc626b0 msgr2=0x557f4be0bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:49:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.272+0000 7f06349c4b80 1 --2- 172.21.15.39:0/1495234418 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f4bc626b0 0x557f4be0bd90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0618011f80 tx=0x7f0618014000 comp rx=0 tx=0).stop 2023-07-21T18:49:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.272+0000 7f06349c4b80 1 -- 172.21.15.39:0/1495234418 shutdown_connections 2023-07-21T18:49:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.272+0000 7f06349c4b80 1 --2- 172.21.15.39:0/1495234418 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f4bc626b0 0x557f4be0bd90 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.272+0000 7f06349c4b80 1 --2- 172.21.15.39:0/1495234418 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f4be06580 0x557f4be069d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.272+0000 7f06349c4b80 1 --2- 172.21.15.39:0/1495234418 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f4be0c2d0 0x557f4be06040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.272+0000 7f06349c4b80 1 -- 172.21.15.39:0/1495234418 >> 172.21.15.39:0/1495234418 conn(0x557f4bdfbbe0 msgr2=0x557f4be00f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:49:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.272+0000 7f06349c4b80 1 -- 172.21.15.39:0/1495234418 shutdown_connections 2023-07-21T18:49:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.272+0000 7f06349c4b80 1 -- 172.21.15.39:0/1495234418 wait complete. 2023-07-21T18:49:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.272+0000 7f06349c4b80 1 -- start start 2023-07-21T18:49:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.272+0000 7f06349c4b80 1 Processor -- start 2023-07-21T18:49:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.272+0000 7f06349c4b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f4bc626b0 0x557f4be125e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.272+0000 7f06349c4b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f4be06580 0x557f4be0f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.272+0000 7f06349c4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f4be0c2d0 0x557f4be0ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.272+0000 7f06349c4b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f4bcf7a60 con 0x557f4be06580 2023-07-21T18:49:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.272+0000 7f06349c4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f4bcf7760 con 0x557f4be0c2d0 2023-07-21T18:49:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.272+0000 7f06349c4b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f4bbdb8c0 con 0x557f4bc626b0 2023-07-21T18:49:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.273+0000 7f06349c4b80 10 client.0 _ll_register_callbacks cb 0x557f4bcf5700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:49:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.273+0000 7f0621d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f4be06580 0x557f4be0f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.273+0000 7f0621d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f4be06580 0x557f4be0f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60176/0 (socket says 172.21.15.39:60176) 2023-07-21T18:49:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.273+0000 7f0621d9b700 1 -- 172.21.15.39:0/3417759916 learned_addr learned my addr 172.21.15.39:0/3417759916 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:49:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.273+0000 7f062259c700 1 --2- 172.21.15.39:0/3417759916 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f4bc626b0 0x557f4be125e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:02.296 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.273+0000 7f0622d9d700 1 --2- 172.21.15.39:0/3417759916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f4be0c2d0 0x557f4be0ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:02.296 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.273+0000 7f0621d9b700 1 -- 172.21.15.39:0/3417759916 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f4bc626b0 msgr2=0x557f4be125e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:49:02.296 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.273+0000 7f0621d9b700 1 --2- 172.21.15.39:0/3417759916 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f4bc626b0 0x557f4be125e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:02.296 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.273+0000 7f0621d9b700 1 -- 172.21.15.39:0/3417759916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f4be0c2d0 msgr2=0x557f4be0ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:49:02.296 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.273+0000 7f0621d9b700 1 --2- 172.21.15.39:0/3417759916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f4be0c2d0 0x557f4be0ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:02.296 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.273+0000 7f0621d9b700 1 -- 172.21.15.39:0/3417759916 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f062410d240 con 0x557f4be06580 2023-07-21T18:49:02.296 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.273+0000 7f0622d9d700 1 --2- 172.21.15.39:0/3417759916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f4be0c2d0 0x557f4be0ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:49:02.297 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.273+0000 7f0621d9b700 1 --2- 172.21.15.39:0/3417759916 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f4be06580 0x557f4be0f860 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f0614099970 tx=0x7f061409dca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea618f33ac7e81f0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:49:02.297 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.273+0000 7f0609ffb700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:49:02.297 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.273+0000 7f0609ffb700 1 -- 172.21.15.39:0/3417759916 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06140b2400 con 0x557f4be06580 2023-07-21T18:49:02.297 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.273+0000 7f0609ffb700 1 -- 172.21.15.39:0/3417759916 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f0614099ed0 con 0x557f4be06580 2023-07-21T18:49:02.297 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.274+0000 7f06349c4b80 1 -- 172.21.15.39:0/3417759916 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557f4bcb1490 con 0x557f4be06580 2023-07-21T18:49:02.297 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.274+0000 7f0609ffb700 1 -- 172.21.15.39:0/3417759916 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f061409a090 con 0x557f4be06580 2023-07-21T18:49:02.297 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.274+0000 7f06349c4b80 10 client.16689 fetch_fsmap learned FSMap version 54 2023-07-21T18:49:02.297 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.274+0000 7f06349c4b80 1 -- 172.21.15.39:0/3417759916 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) v3 -- 0x7f0618011c60 con 0x557f4be06580 2023-07-21T18:49:02.298 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.274+0000 7f0609ffb700 1 -- 172.21.15.39:0/3417759916 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557f4bcb1490 con 0x557f4be06580 2023-07-21T18:49:02.298 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.274+0000 7f0609ffb700 1 -- 172.21.15.39:0/3417759916 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 54) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f0614098bd0 con 0x557f4be06580 2023-07-21T18:49:02.298 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.274+0000 7f06349c4b80 10 client.16689 fetch_fsmap finished waiting for FSMap version 54 2023-07-21T18:49:02.298 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.274+0000 7f06349c4b80 10 client.16689 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:49:02.298 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.274+0000 7f06349c4b80 20 client.16689 populate_metadata read hostname 'smithi039' 2023-07-21T18:49:02.298 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.274+0000 7f06349c4b80 1 --2- 172.21.15.39:0/3417759916 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x557f4be1fbe0 0x557f4be21ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:02.298 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.274+0000 7f06349c4b80 4 client.16689 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:49:02.299 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.274+0000 7f06349c4b80 1 -- 172.21.15.39:0/3417759916 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557f4bd50e50 con 0x557f4be1fbe0 2023-07-21T18:49:02.299 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.274+0000 7f062259c700 1 --2- 172.21.15.39:0/3417759916 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x557f4be1fbe0 0x557f4be21ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:02.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.275+0000 7f0609ffb700 10 client.16689 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:49:02.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.275+0000 7f062259c700 1 --2- 172.21.15.39:0/3417759916 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x557f4be1fbe0 0x557f4be21ff0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21f76bfb6dc95af server_cookie=fddb156dfd4f5253 in_seq=0 out_seq=0 2023-07-21T18:49:02.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.275+0000 7f0609ffb700 1 -- 172.21.15.39:0/3417759916 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557f4bd50e50 con 0x557f4be1fbe0 2023-07-21T18:49:02.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.275+0000 7f0609ffb700 10 client.16689 handle_command_reply: tid=0 2023-07-21T18:49:02.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.276+0000 7f06349c4b80 1 client.16689 shutdown 2023-07-21T18:49:02.301 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.276+0000 7f06349c4b80 1 -- 172.21.15.39:0/3417759916 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f4be06580 msgr2=0x557f4be0f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:49:02.301 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.276+0000 7f06349c4b80 1 --2- 172.21.15.39:0/3417759916 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f4be06580 0x557f4be0f860 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f0614099970 tx=0x7f061409dca0 comp rx=0 tx=0).stop 2023-07-21T18:49:02.301 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.276+0000 7f06349c4b80 1 -- 172.21.15.39:0/3417759916 shutdown_connections 2023-07-21T18:49:02.301 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.276+0000 7f06349c4b80 1 --2- 172.21.15.39:0/3417759916 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x557f4be1fbe0 0x557f4be21ff0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:02.301 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.276+0000 7f06349c4b80 1 --2- 172.21.15.39:0/3417759916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f4be0c2d0 0x557f4be0ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:02.301 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.276+0000 7f06349c4b80 1 --2- 172.21.15.39:0/3417759916 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f4bc626b0 0x557f4be125e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:02.301 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.276+0000 7f06349c4b80 1 --2- 172.21.15.39:0/3417759916 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f4be06580 0x557f4be0f860 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:02.302 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.276+0000 7f0609ffb700 0 client.16689 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:49:02.302 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.276+0000 7f06349c4b80 1 -- 172.21.15.39:0/3417759916 >> 172.21.15.39:0/3417759916 conn(0x557f4bdfbbe0 msgr2=0x557f4bdffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:49:02.302 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.276+0000 7f06349c4b80 1 -- 172.21.15.39:0/3417759916 shutdown_connections 2023-07-21T18:49:02.302 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.276+0000 7f06349c4b80 1 -- 172.21.15.39:0/3417759916 wait complete. 2023-07-21T18:49:02.302 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.276+0000 7f06349c4b80 20 client.16689 trim_cache size 0 max 16384 2023-07-21T18:49:02.305 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.290+0000 7f06349c4b80 1 Processor -- start 2023-07-21T18:49:02.305 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.291+0000 7f06349c4b80 1 -- start start 2023-07-21T18:49:02.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.291+0000 7f06349c4b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f4bc626b0 0x557f4be06040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:02.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.291+0000 7f06349c4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f4be06580 0x557f4be16c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:02.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.291+0000 7f06349c4b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f4be0c2d0 0x557f4be0b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:02.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.291+0000 7f06349c4b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f4be10a40 con 0x557f4be0c2d0 2023-07-21T18:49:02.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.291+0000 7f06349c4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f4be10770 con 0x557f4be06580 2023-07-21T18:49:02.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.291+0000 7f06349c4b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f4be0c0b0 con 0x557f4bc626b0 2023-07-21T18:49:02.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.291+0000 7f0622d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f4be0c2d0 0x557f4be0b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:02.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.291+0000 7f0622d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f4be0c2d0 0x557f4be0b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60186/0 (socket says 172.21.15.39:60186) 2023-07-21T18:49:02.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.291+0000 7f0622d9d700 1 -- 172.21.15.39:0/3155957273 learned_addr learned my addr 172.21.15.39:0/3155957273 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:49:02.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.291+0000 7f0621d9b700 1 --2- 172.21.15.39:0/3155957273 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f4be06580 0x557f4be16c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:02.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.291+0000 7f062259c700 1 --2- 172.21.15.39:0/3155957273 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f4bc626b0 0x557f4be06040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:02.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.291+0000 7f0622d9d700 1 -- 172.21.15.39:0/3155957273 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f4bc626b0 msgr2=0x557f4be06040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:49:02.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.291+0000 7f0622d9d700 1 --2- 172.21.15.39:0/3155957273 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f4bc626b0 0x557f4be06040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:02.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.291+0000 7f0622d9d700 1 -- 172.21.15.39:0/3155957273 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f4be06580 msgr2=0x557f4be16c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:49:02.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.291+0000 7f0622d9d700 1 --2- 172.21.15.39:0/3155957273 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f4be06580 0x557f4be16c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:02.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.291+0000 7f0622d9d700 1 -- 172.21.15.39:0/3155957273 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f4bb6ec00 con 0x557f4be0c2d0 2023-07-21T18:49:02.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.291+0000 7f0621d9b700 1 --2- 172.21.15.39:0/3155957273 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f4be06580 0x557f4be16c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:49:02.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.291+0000 7f0622d9d700 1 --2- 172.21.15.39:0/3155957273 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f4be0c2d0 0x557f4be0b130 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f061c00bb60 tx=0x7f061c00bb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cee1944cbc1b684f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:49:02.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.292+0000 7f06117fa700 1 -- 172.21.15.39:0/3155957273 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061c00e040 con 0x557f4be0c2d0 2023-07-21T18:49:02.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.292+0000 7f06117fa700 1 -- 172.21.15.39:0/3155957273 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f061c021070 con 0x557f4be0c2d0 2023-07-21T18:49:02.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.292+0000 7f06117fa700 1 -- 172.21.15.39:0/3155957273 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061c0054e0 con 0x557f4be0c2d0 2023-07-21T18:49:02.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.292+0000 7f06349c4b80 1 -- 172.21.15.39:0/3155957273 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f4be0c2d0 msgr2=0x557f4be0b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:49:02.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.292+0000 7f06349c4b80 1 --2- 172.21.15.39:0/3155957273 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f4be0c2d0 0x557f4be0b130 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f061c00bb60 tx=0x7f061c00bb90 comp rx=0 tx=0).stop 2023-07-21T18:49:02.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.292+0000 7f06349c4b80 1 -- 172.21.15.39:0/3155957273 shutdown_connections 2023-07-21T18:49:02.312 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.292+0000 7f06349c4b80 1 --2- 172.21.15.39:0/3155957273 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f4be0c2d0 0x557f4be0b130 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:02.312 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.292+0000 7f06349c4b80 1 --2- 172.21.15.39:0/3155957273 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f4bc626b0 0x557f4be06040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:02.312 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.292+0000 7f06349c4b80 1 --2- 172.21.15.39:0/3155957273 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f4be06580 0x557f4be16c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:02.312 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.292+0000 7f06349c4b80 1 -- 172.21.15.39:0/3155957273 >> 172.21.15.39:0/3155957273 conn(0x557f4bdfbbe0 msgr2=0x557f4be0aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:49:02.312 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.292+0000 7f06349c4b80 1 -- 172.21.15.39:0/3155957273 shutdown_connections 2023-07-21T18:49:02.312 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.292+0000 7f06349c4b80 1 -- 172.21.15.39:0/3155957273 wait complete. 2023-07-21T18:49:02.312 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.293+0000 7f06349c4b80 1 -- start start 2023-07-21T18:49:02.313 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.293+0000 7f06349c4b80 1 Processor -- start 2023-07-21T18:49:02.313 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.293+0000 7f06349c4b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f4bc626b0 0x557f4be08c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:02.313 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.293+0000 7f06349c4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f4be06580 0x557f4be091d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:02.313 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.293+0000 7f06349c4b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f4be0c2d0 0x557f4be07860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:02.313 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.293+0000 7f06349c4b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f4be07dd0 con 0x557f4be0c2d0 2023-07-21T18:49:02.313 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.293+0000 7f06349c4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f4be07f40 con 0x557f4be06580 2023-07-21T18:49:02.313 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.293+0000 7f06349c4b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f4be08100 con 0x557f4bc626b0 2023-07-21T18:49:02.314 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.293+0000 7f06349c4b80 10 client.0 _ll_register_callbacks cb 0x7f06140995f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:49:02.314 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.293+0000 7f0622d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f4be0c2d0 0x557f4be07860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:02.314 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.293+0000 7f0622d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f4be0c2d0 0x557f4be07860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60192/0 (socket says 172.21.15.39:60192) 2023-07-21T18:49:02.314 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.293+0000 7f0622d9d700 1 -- 172.21.15.39:0/83701600 learned_addr learned my addr 172.21.15.39:0/83701600 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:49:02.314 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.293+0000 7f062259c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f4bc626b0 0x557f4be08c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:02.314 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.293+0000 7f0621d9b700 1 --2- 172.21.15.39:0/83701600 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f4be06580 0x557f4be091d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:02.314 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.293+0000 7f0622d9d700 1 -- 172.21.15.39:0/83701600 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f4bc626b0 msgr2=0x557f4be08c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:49:02.315 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.293+0000 7f0622d9d700 1 --2- 172.21.15.39:0/83701600 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f4bc626b0 0x557f4be08c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:02.315 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.294+0000 7f0622d9d700 1 -- 172.21.15.39:0/83701600 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f4be06580 msgr2=0x557f4be091d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:49:02.315 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.294+0000 7f0622d9d700 1 --2- 172.21.15.39:0/83701600 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f4be06580 0x557f4be091d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:02.315 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.294+0000 7f0622d9d700 1 -- 172.21.15.39:0/83701600 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f061c009420 con 0x557f4be0c2d0 2023-07-21T18:49:02.315 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.294+0000 7f0622d9d700 1 --2- 172.21.15.39:0/83701600 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f4be0c2d0 0x557f4be07860 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f061c007920 tx=0x7f061c00b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9beb34be6e0039e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:49:02.315 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.294+0000 7f0609ffb700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:49:02.315 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.294+0000 7f0609ffb700 1 -- 172.21.15.39:0/83701600 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061c00e040 con 0x557f4be0c2d0 2023-07-21T18:49:02.316 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.294+0000 7f0609ffb700 1 -- 172.21.15.39:0/83701600 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f061c005310 con 0x557f4be0c2d0 2023-07-21T18:49:02.316 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.294+0000 7f06349c4b80 1 -- 172.21.15.39:0/83701600 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0618003b40 con 0x557f4be0c2d0 2023-07-21T18:49:02.316 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.294+0000 7f0609ffb700 1 -- 172.21.15.39:0/83701600 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f061c003dc0 con 0x557f4be0c2d0 2023-07-21T18:49:02.316 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.294+0000 7f06349c4b80 10 client.16695 fetch_fsmap learned FSMap version 54 2023-07-21T18:49:02.316 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.294+0000 7f06349c4b80 1 -- 172.21.15.39:0/83701600 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) v3 -- 0x557f4bbac1f0 con 0x557f4be0c2d0 2023-07-21T18:49:02.316 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.294+0000 7f0609ffb700 1 -- 172.21.15.39:0/83701600 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0618003b40 con 0x557f4be0c2d0 2023-07-21T18:49:02.316 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.294+0000 7f0609ffb700 1 -- 172.21.15.39:0/83701600 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 54) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f061c00b0d0 con 0x557f4be0c2d0 2023-07-21T18:49:02.317 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.294+0000 7f06349c4b80 10 client.16695 fetch_fsmap finished waiting for FSMap version 54 2023-07-21T18:49:02.317 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.294+0000 7f06349c4b80 10 client.16695 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:49:02.317 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.294+0000 7f06349c4b80 20 client.16695 populate_metadata read hostname 'smithi039' 2023-07-21T18:49:02.317 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.294+0000 7f06349c4b80 1 --2- 172.21.15.39:0/83701600 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x557f4be74670 0x557f4be76a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:02.317 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.294+0000 7f06349c4b80 4 client.16695 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:49:02.317 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.294+0000 7f06349c4b80 1 -- 172.21.15.39:0/83701600 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557f4bd12ff0 con 0x557f4be74670 2023-07-21T18:49:02.317 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.295+0000 7f062259c700 1 --2- 172.21.15.39:0/83701600 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x557f4be74670 0x557f4be76a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:02.318 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.295+0000 7f062259c700 1 --2- 172.21.15.39:0/83701600 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x557f4be74670 0x557f4be76a50 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cab4510addb720a server_cookie=867fdf00d0160d26 in_seq=0 out_seq=0 2023-07-21T18:49:02.318 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.295+0000 7f0609ffb700 10 client.16695 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:49:02.368 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.354+0000 7f0609ffb700 1 -- 172.21.15.39:0/83701600 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557f4bd12ff0 con 0x557f4be74670 2023-07-21T18:49:02.368 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.354+0000 7f0609ffb700 10 client.16695 handle_command_reply: tid=0 2023-07-21T18:49:02.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.354+0000 7f06349c4b80 1 client.16695 shutdown 2023-07-21T18:49:02.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.354+0000 7f06349c4b80 1 -- 172.21.15.39:0/83701600 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f4be0c2d0 msgr2=0x557f4be07860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:49:02.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.354+0000 7f06349c4b80 1 --2- 172.21.15.39:0/83701600 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f4be0c2d0 0x557f4be07860 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f061c007920 tx=0x7f061c00b790 comp rx=0 tx=0).stop 2023-07-21T18:49:02.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.354+0000 7f06349c4b80 1 -- 172.21.15.39:0/83701600 shutdown_connections 2023-07-21T18:49:02.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.354+0000 7f06349c4b80 1 --2- 172.21.15.39:0/83701600 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x557f4be74670 0x557f4be76a50 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:02.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.355+0000 7f06349c4b80 1 --2- 172.21.15.39:0/83701600 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f4be0c2d0 0x557f4be07860 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:02.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.355+0000 7f0609ffb700 0 client.16695 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:49:02.370 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.355+0000 7f06349c4b80 1 --2- 172.21.15.39:0/83701600 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f4bc626b0 0x557f4be08c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:02.370 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.355+0000 7f06349c4b80 1 --2- 172.21.15.39:0/83701600 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f4be06580 0x557f4be091d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:02.370 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.355+0000 7f06349c4b80 1 -- 172.21.15.39:0/83701600 >> 172.21.15.39:0/83701600 conn(0x557f4bdfbbe0 msgr2=0x557f4be0b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:49:02.370 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.356+0000 7f06349c4b80 1 -- 172.21.15.39:0/83701600 shutdown_connections 2023-07-21T18:49:02.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.356+0000 7f06349c4b80 1 -- 172.21.15.39:0/83701600 wait complete. 2023-07-21T18:49:02.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.356+0000 7f06349c4b80 20 client.16695 trim_cache size 0 max 16384 2023-07-21T18:49:02.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:49:02.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:49:02.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:49:02.372 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:49:02.372 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.357+0000 7f062883e700 1 -- 172.21.15.39:0/4092151193 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f060c0a12c0 msgr2=0x7f060c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:49:02.372 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.357+0000 7f062883e700 1 --2- 172.21.15.39:0/4092151193 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f060c0a12c0 0x7f060c0a3770 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0618009cb0 tx=0x7f0618009260 comp rx=0 tx=0).stop 2023-07-21T18:49:02.373 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.357+0000 7f062883e700 1 -- 172.21.15.39:0/4092151193 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0624100500 msgr2=0x7f0624072f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:49:02.373 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.357+0000 7f062883e700 1 --2- 172.21.15.39:0/4092151193 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0624100500 0x7f0624072f30 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f061400d950 tx=0x7f061400dc60 comp rx=0 tx=0).stop 2023-07-21T18:49:02.373 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.357+0000 7f062883e700 1 -- 172.21.15.39:0/4092151193 shutdown_connections 2023-07-21T18:49:02.373 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.357+0000 7f062883e700 1 --2- 172.21.15.39:0/4092151193 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f060c0a12c0 0x7f060c0a3770 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:02.373 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.357+0000 7f062883e700 1 --2- 172.21.15.39:0/4092151193 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06240ffb50 0x7f06240729f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:02.373 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.357+0000 7f062883e700 1 --2- 172.21.15.39:0/4092151193 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0624105b80 0x7f06241a3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.357+0000 7f062883e700 1 --2- 172.21.15.39:0/4092151193 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0624100500 0x7f0624072f30 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.357+0000 7f062883e700 1 -- 172.21.15.39:0/4092151193 >> 172.21.15.39:0/4092151193 conn(0x7f06240fb650 msgr2=0x7f06240fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:49:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.357+0000 7f062883e700 1 -- 172.21.15.39:0/4092151193 shutdown_connections 2023-07-21T18:49:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:02.357+0000 7f062883e700 1 -- 172.21.15.39:0/4092151193 wait complete. 2023-07-21T18:49:02.679 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:49:02 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:49:02.861 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:49:02 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:49:02.880 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:49:02 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:49:03.276 INFO:tasks.workunit.client.0.smithi039.stdout: 1 464499 6.61 MB/sec execute 256 sec latency 3106.053 ms 2023-07-21T18:49:03.861 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:49:03 smithi193 ceph-mon[98204]: pgmap v225: 97 pgs: 97 active+clean; 178 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 202 op/s 2023-07-21T18:49:03.880 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:49:03 smithi155 ceph-mon[97934]: pgmap v225: 97 pgs: 97 active+clean; 178 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 202 op/s 2023-07-21T18:49:03.929 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:49:03 smithi039 ceph-mon[96489]: pgmap v225: 97 pgs: 97 active+clean; 178 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 202 op/s 2023-07-21T18:49:04.276 INFO:tasks.workunit.client.0.smithi039.stdout: 1 465863 6.63 MB/sec execute 257 sec latency 26.130 ms 2023-07-21T18:49:04.861 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:49:04 smithi193 ceph-mon[98204]: osdmap e86: 12 total, 12 up, 12 in 2023-07-21T18:49:04.861 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:49:04 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:49:04.880 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:49:04 smithi155 ceph-mon[97934]: osdmap e86: 12 total, 12 up, 12 in 2023-07-21T18:49:04.880 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:49:04 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:49:04.929 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:49:04 smithi039 ceph-mon[96489]: osdmap e86: 12 total, 12 up, 12 in 2023-07-21T18:49:04.929 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:49:04 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:49:05.276 INFO:tasks.workunit.client.0.smithi039.stdout: 1 467877 6.63 MB/sec execute 258 sec latency 15.492 ms 2023-07-21T18:49:05.861 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:49:05 smithi193 ceph-mon[98204]: pgmap v227: 97 pgs: 97 active+clean; 178 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 243 op/s 2023-07-21T18:49:05.880 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:49:05 smithi155 ceph-mon[97934]: pgmap v227: 97 pgs: 97 active+clean; 178 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 243 op/s 2023-07-21T18:49:05.929 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:49:05 smithi039 ceph-mon[96489]: pgmap v227: 97 pgs: 97 active+clean; 178 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 243 op/s 2023-07-21T18:49:06.276 INFO:tasks.workunit.client.0.smithi039.stdout: 1 469518 6.65 MB/sec execute 259 sec latency 24.312 ms 2023-07-21T18:49:07.276 INFO:tasks.workunit.client.0.smithi039.stdout: 1 471432 6.65 MB/sec execute 260 sec latency 41.249 ms 2023-07-21T18:49:07.861 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:49:07 smithi193 ceph-mon[98204]: pgmap v228: 97 pgs: 97 active+clean; 174 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.8 MiB/s wr, 264 op/s 2023-07-21T18:49:07.861 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:49:07 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:49:07.880 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:49:07 smithi155 ceph-mon[97934]: pgmap v228: 97 pgs: 97 active+clean; 174 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.8 MiB/s wr, 264 op/s 2023-07-21T18:49:07.880 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:49:07 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:49:07.929 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:49:07 smithi039 ceph-mon[96489]: pgmap v228: 97 pgs: 97 active+clean; 174 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.8 MiB/s wr, 264 op/s 2023-07-21T18:49:07.929 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:49:07 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:49:08.276 INFO:tasks.workunit.client.0.smithi039.stdout: 1 473011 6.67 MB/sec execute 261 sec latency 21.563 ms 2023-07-21T18:49:09.276 INFO:tasks.workunit.client.0.smithi039.stdout: 1 475024 6.68 MB/sec execute 262 sec latency 19.255 ms 2023-07-21T18:49:09.861 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:49:09 smithi193 ceph-mon[98204]: pgmap v229: 97 pgs: 97 active+clean; 174 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.8 MiB/s wr, 264 op/s 2023-07-21T18:49:09.880 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:49:09 smithi155 ceph-mon[97934]: pgmap v229: 97 pgs: 97 active+clean; 174 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.8 MiB/s wr, 264 op/s 2023-07-21T18:49:09.929 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:49:09 smithi039 ceph-mon[96489]: pgmap v229: 97 pgs: 97 active+clean; 174 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.8 MiB/s wr, 264 op/s 2023-07-21T18:49:10.276 INFO:tasks.workunit.client.0.smithi039.stdout: 1 476645 6.70 MB/sec execute 263 sec latency 12.580 ms 2023-07-21T18:49:11.276 INFO:tasks.workunit.client.0.smithi039.stdout: 1 478606 6.70 MB/sec execute 264 sec latency 21.270 ms 2023-07-21T18:49:12.276 INFO:tasks.workunit.client.0.smithi039.stdout: 1 480057 6.72 MB/sec execute 265 sec latency 52.880 ms 2023-07-21T18:49:12.361 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:49:12 smithi193 ceph-mon[98204]: pgmap v230: 97 pgs: 97 active+clean; 181 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 284 op/s 2023-07-21T18:49:12.361 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:49:12 smithi193 ceph-mon[98204]: osdmap e87: 12 total, 12 up, 12 in 2023-07-21T18:49:12.380 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:49:12 smithi155 ceph-mon[97934]: pgmap v230: 97 pgs: 97 active+clean; 181 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 284 op/s 2023-07-21T18:49:12.380 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:49:12 smithi155 ceph-mon[97934]: osdmap e87: 12 total, 12 up, 12 in 2023-07-21T18:49:12.429 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:49:12 smithi039 ceph-mon[96489]: pgmap v230: 97 pgs: 97 active+clean; 181 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 284 op/s 2023-07-21T18:49:12.429 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:49:12 smithi039 ceph-mon[96489]: osdmap e87: 12 total, 12 up, 12 in 2023-07-21T18:49:13.276 INFO:tasks.workunit.client.0.smithi039.stdout: 1 481868 6.71 MB/sec execute 266 sec latency 24.203 ms 2023-07-21T18:49:14.276 INFO:tasks.workunit.client.0.smithi039.stdout: 1 483586 6.74 MB/sec execute 267 sec latency 11.464 ms 2023-07-21T18:49:14.361 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:49:14 smithi193 ceph-mon[98204]: pgmap v232: 97 pgs: 97 active+clean; 181 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.3 MiB/s wr, 316 op/s 2023-07-21T18:49:14.380 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:49:14 smithi155 ceph-mon[97934]: pgmap v232: 97 pgs: 97 active+clean; 181 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.3 MiB/s wr, 316 op/s 2023-07-21T18:49:14.429 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:49:14 smithi039 ceph-mon[96489]: pgmap v232: 97 pgs: 97 active+clean; 181 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.3 MiB/s wr, 316 op/s 2023-07-21T18:49:15.277 INFO:tasks.workunit.client.0.smithi039.stdout: 1 485310 6.74 MB/sec execute 268 sec latency 16.818 ms 2023-07-21T18:49:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.473+0000 7ff8d49fd700 1 -- 172.21.15.39:0/3504114352 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8d00feae0 msgr2=0x7ff8d00fef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:49:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.473+0000 7ff8d49fd700 1 --2- 172.21.15.39:0/3504114352 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8d00feae0 0x7ff8d00fef50 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7ff8c4009990 tx=0x7ff8c4009ca0 comp rx=0 tx=0).stop 2023-07-21T18:49:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.473+0000 7ff8d49fd700 1 -- 172.21.15.39:0/3504114352 shutdown_connections 2023-07-21T18:49:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.473+0000 7ff8d49fd700 1 --2- 172.21.15.39:0/3504114352 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8d00ff490 0x7ff8d010b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:15.488 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.473+0000 7ff8d49fd700 1 --2- 172.21.15.39:0/3504114352 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8d0104af0 0x7ff8d0104ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:15.488 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.473+0000 7ff8d49fd700 1 --2- 172.21.15.39:0/3504114352 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8d00feae0 0x7ff8d00fef50 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:15.488 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.473+0000 7ff8d49fd700 1 -- 172.21.15.39:0/3504114352 >> 172.21.15.39:0/3504114352 conn(0x7ff8d0076a10 msgr2=0x7ff8d0076e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:49:15.488 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.473+0000 7ff8d49fd700 1 -- 172.21.15.39:0/3504114352 shutdown_connections 2023-07-21T18:49:15.488 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.473+0000 7ff8d49fd700 1 -- 172.21.15.39:0/3504114352 wait complete. 2023-07-21T18:49:15.490 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.474+0000 7ff8d49fd700 1 Processor -- start 2023-07-21T18:49:15.490 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.474+0000 7ff8d49fd700 1 -- start start 2023-07-21T18:49:15.490 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.474+0000 7ff8d49fd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8d00feae0 0x7ff8d0196c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:15.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.474+0000 7ff8d49fd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8d00ff490 0x7ff8d01971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:15.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.474+0000 7ff8d49fd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8d0104af0 0x7ff8d01a1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:15.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.474+0000 7ff8d49fd700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d01a20e0 con 0x7ff8d0104af0 2023-07-21T18:49:15.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.474+0000 7ff8d49fd700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d01a2250 con 0x7ff8d00ff490 2023-07-21T18:49:15.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.474+0000 7ff8d49fd700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d01a2410 con 0x7ff8d00feae0 2023-07-21T18:49:15.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.475+0000 7ff8ced9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8d0104af0 0x7ff8d01a1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:15.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.475+0000 7ff8cdd9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8d00ff490 0x7ff8d01971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:15.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.475+0000 7ff8cdd9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8d00ff490 0x7ff8d01971b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:44568/0 (socket says 172.21.15.39:44568) 2023-07-21T18:49:15.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.475+0000 7ff8ced9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8d0104af0 0x7ff8d01a1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60206/0 (socket says 172.21.15.39:60206) 2023-07-21T18:49:15.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.475+0000 7ff8cdd9b700 1 -- 172.21.15.39:0/3617187342 learned_addr learned my addr 172.21.15.39:0/3617187342 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:49:15.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.475+0000 7ff8ce59c700 1 --2- 172.21.15.39:0/3617187342 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8d00feae0 0x7ff8d0196c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:15.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.475+0000 7ff8cdd9b700 1 -- 172.21.15.39:0/3617187342 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8d00feae0 msgr2=0x7ff8d0196c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:49:15.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.475+0000 7ff8cdd9b700 1 --2- 172.21.15.39:0/3617187342 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8d00feae0 0x7ff8d0196c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:15.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.475+0000 7ff8cdd9b700 1 -- 172.21.15.39:0/3617187342 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8d0104af0 msgr2=0x7ff8d01a1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:49:15.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.475+0000 7ff8cdd9b700 1 --2- 172.21.15.39:0/3617187342 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8d0104af0 0x7ff8d01a1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:15.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.475+0000 7ff8cdd9b700 1 -- 172.21.15.39:0/3617187342 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8c0009520 con 0x7ff8d00ff490 2023-07-21T18:49:15.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.475+0000 7ff8ced9d700 1 --2- 172.21.15.39:0/3617187342 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8d0104af0 0x7ff8d01a1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:49:15.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.475+0000 7ff8cdd9b700 1 --2- 172.21.15.39:0/3617187342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8d00ff490 0x7ff8d01971b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff8c4000c00 tx=0x7ff8c4005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:49:15.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.475+0000 7ff8bf7fe700 1 -- 172.21.15.39:0/3617187342 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c401d070 con 0x7ff8d00ff490 2023-07-21T18:49:15.494 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.475+0000 7ff8d49fd700 1 -- 172.21.15.39:0/3617187342 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8c40095f0 con 0x7ff8d00ff490 2023-07-21T18:49:15.494 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.475+0000 7ff8bf7fe700 1 -- 172.21.15.39:0/3617187342 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff8c4022470 con 0x7ff8d00ff490 2023-07-21T18:49:15.494 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.475+0000 7ff8bf7fe700 1 -- 172.21.15.39:0/3617187342 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c4017650 con 0x7ff8d00ff490 2023-07-21T18:49:15.494 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.476+0000 7ff8d49fd700 1 -- 172.21.15.39:0/3617187342 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8d01a2980 con 0x7ff8d00ff490 2023-07-21T18:49:15.498 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.478+0000 7ff8bf7fe700 1 -- 172.21.15.39:0/3617187342 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7ff8c40225e0 con 0x7ff8d00ff490 2023-07-21T18:49:15.498 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.478+0000 7ff8bf7fe700 1 --2- 172.21.15.39:0/3617187342 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7ff8b80a12c0 0x7ff8b80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:15.498 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.479+0000 7ff8ce59c700 1 --2- 172.21.15.39:0/3617187342 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7ff8b80a12c0 0x7ff8b80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:15.498 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.479+0000 7ff8bf7fe700 1 -- 172.21.15.39:0/3617187342 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8527+0+0 (secure 0 0 0) 0x7ff8c40ce0e0 con 0x7ff8d00ff490 2023-07-21T18:49:15.498 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.479+0000 7ff8ce59c700 1 --2- 172.21.15.39:0/3617187342 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7ff8b80a12c0 0x7ff8b80a3770 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff8c0009d70 tx=0x7ff8c0009260 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:49:15.499 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.479+0000 7ff8e0b83b80 1 Processor -- start 2023-07-21T18:49:15.499 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.479+0000 7ff8e0b83b80 1 -- start start 2023-07-21T18:49:15.499 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.479+0000 7ff8e0b83b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631a61506b0 0x5631a62f9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:15.499 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.479+0000 7ff8e0b83b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5631a62fa2d0 0x5631a62f4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:15.499 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.480+0000 7ff8e0b83b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631a62f4580 0x5631a62f49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:15.499 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.480+0000 7ff8e0b83b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631a61f9c50 con 0x5631a62fa2d0 2023-07-21T18:49:15.499 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.480+0000 7ff8e0b83b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631a624c9e0 con 0x5631a61506b0 2023-07-21T18:49:15.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.480+0000 7ff8e0b83b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631a6245e70 con 0x5631a62f4580 2023-07-21T18:49:15.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.480+0000 7ff8cdd9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5631a62fa2d0 0x5631a62f4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:15.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.480+0000 7ff8ce59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631a61506b0 0x5631a62f9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:15.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.480+0000 7ff8ced9d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631a62f4580 0x5631a62f49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:15.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.480+0000 7ff8cdd9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5631a62fa2d0 0x5631a62f4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60212/0 (socket says 172.21.15.39:60212) 2023-07-21T18:49:15.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.480+0000 7ff8cdd9b700 1 -- 172.21.15.39:0/2557526859 learned_addr learned my addr 172.21.15.39:0/2557526859 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:49:15.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.480+0000 7ff8ce59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631a61506b0 0x5631a62f9d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:44574/0 (socket says 172.21.15.39:44574) 2023-07-21T18:49:15.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.480+0000 7ff8cdd9b700 1 -- 172.21.15.39:0/2557526859 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631a62f4580 msgr2=0x5631a62f49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:49:15.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.480+0000 7ff8cdd9b700 1 --2- 172.21.15.39:0/2557526859 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631a62f4580 0x5631a62f49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:15.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.480+0000 7ff8cdd9b700 1 -- 172.21.15.39:0/2557526859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631a61506b0 msgr2=0x5631a62f9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:49:15.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.480+0000 7ff8cdd9b700 1 --2- 172.21.15.39:0/2557526859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631a61506b0 0x5631a62f9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:15.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.480+0000 7ff8cdd9b700 1 -- 172.21.15.39:0/2557526859 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5631a6073d80 con 0x5631a62fa2d0 2023-07-21T18:49:15.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.480+0000 7ff8cdd9b700 1 --2- 172.21.15.39:0/2557526859 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5631a62fa2d0 0x5631a62f4040 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7ff8c4097740 tx=0x7ff8c409b140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c94df9ce1761fe86 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:49:15.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.480+0000 7ff8bcff9700 1 -- 172.21.15.39:0/2557526859 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c409f270 con 0x5631a62fa2d0 2023-07-21T18:49:15.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.480+0000 7ff8bcff9700 1 -- 172.21.15.39:0/2557526859 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff8c40a3600 con 0x5631a62fa2d0 2023-07-21T18:49:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.480+0000 7ff8bcff9700 1 -- 172.21.15.39:0/2557526859 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c409b4a0 con 0x5631a62fa2d0 2023-07-21T18:49:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.481+0000 7ff8e0b83b80 1 -- 172.21.15.39:0/2557526859 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5631a62fa2d0 msgr2=0x5631a62f4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:49:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.481+0000 7ff8e0b83b80 1 --2- 172.21.15.39:0/2557526859 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5631a62fa2d0 0x5631a62f4040 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7ff8c4097740 tx=0x7ff8c409b140 comp rx=0 tx=0).stop 2023-07-21T18:49:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.481+0000 7ff8e0b83b80 1 -- 172.21.15.39:0/2557526859 shutdown_connections 2023-07-21T18:49:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.481+0000 7ff8e0b83b80 1 --2- 172.21.15.39:0/2557526859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631a61506b0 0x5631a62f9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.481+0000 7ff8e0b83b80 1 --2- 172.21.15.39:0/2557526859 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631a62f4580 0x5631a62f49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.481+0000 7ff8e0b83b80 1 --2- 172.21.15.39:0/2557526859 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5631a62fa2d0 0x5631a62f4040 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.481+0000 7ff8e0b83b80 1 -- 172.21.15.39:0/2557526859 >> 172.21.15.39:0/2557526859 conn(0x5631a62e9be0 msgr2=0x5631a62eef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:49:15.503 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.481+0000 7ff8e0b83b80 1 -- 172.21.15.39:0/2557526859 shutdown_connections 2023-07-21T18:49:15.503 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.481+0000 7ff8e0b83b80 1 -- 172.21.15.39:0/2557526859 wait complete. 2023-07-21T18:49:15.503 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.482+0000 7ff8e0b83b80 1 -- start start 2023-07-21T18:49:15.503 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.482+0000 7ff8e0b83b80 1 Processor -- start 2023-07-21T18:49:15.503 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.482+0000 7ff8e0b83b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631a61506b0 0x5631a6303320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:15.503 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.482+0000 7ff8e0b83b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5631a62f4580 0x5631a6303860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:15.503 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.482+0000 7ff8e0b83b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631a62fa2d0 0x5631a62fd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:15.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.482+0000 7ff8e0b83b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631a61e5a60 con 0x5631a62f4580 2023-07-21T18:49:15.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.482+0000 7ff8e0b83b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631a61e5760 con 0x5631a62fa2d0 2023-07-21T18:49:15.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.482+0000 7ff8e0b83b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631a60c98c0 con 0x5631a61506b0 2023-07-21T18:49:15.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.482+0000 7ff8e0b83b80 10 client.0 _ll_register_callbacks cb 0x5631a61e3700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:49:15.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.482+0000 7ff8cdd9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5631a62f4580 0x5631a6303860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:15.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.482+0000 7ff8ce59c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631a61506b0 0x5631a6303320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:15.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.482+0000 7ff8cdd9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5631a62f4580 0x5631a6303860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60218/0 (socket says 172.21.15.39:60218) 2023-07-21T18:49:15.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.482+0000 7ff8ce59c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631a61506b0 0x5631a6303320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:50536/0 (socket says 172.21.15.39:50536) 2023-07-21T18:49:15.505 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.482+0000 7ff8ced9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631a62fa2d0 0x5631a62fd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:15.505 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.482+0000 7ff8cdd9b700 1 -- 172.21.15.39:0/2159268556 learned_addr learned my addr 172.21.15.39:0/2159268556 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:49:15.505 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.482+0000 7ff8cdd9b700 1 -- 172.21.15.39:0/2159268556 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631a61506b0 msgr2=0x5631a6303320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:49:15.505 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.482+0000 7ff8cdd9b700 1 --2- 172.21.15.39:0/2159268556 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631a61506b0 0x5631a6303320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:15.505 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.482+0000 7ff8cdd9b700 1 -- 172.21.15.39:0/2159268556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631a62fa2d0 msgr2=0x5631a62fd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:49:15.505 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.482+0000 7ff8cdd9b700 1 --2- 172.21.15.39:0/2159268556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631a62fa2d0 0x5631a62fd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:15.505 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.482+0000 7ff8cdd9b700 1 -- 172.21.15.39:0/2159268556 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8d01a2980 con 0x5631a62f4580 2023-07-21T18:49:15.505 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.482+0000 7ff8ce59c700 1 --2- 172.21.15.39:0/2159268556 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631a61506b0 0x5631a6303320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:49:15.506 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.483+0000 7ff8cdd9b700 1 --2- 172.21.15.39:0/2159268556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5631a62f4580 0x5631a6303860 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7ff8c40a30d0 tx=0x7ff8c401be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60afe163732e7b09 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:49:15.506 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.483+0000 7ff8b5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:49:15.506 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.483+0000 7ff8b5ffb700 1 -- 172.21.15.39:0/2159268556 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c409f270 con 0x5631a62f4580 2023-07-21T18:49:15.506 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.483+0000 7ff8b5ffb700 1 -- 172.21.15.39:0/2159268556 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8527+0+0 (secure 0 0 0) 0x7ff8c409a940 con 0x5631a62f4580 2023-07-21T18:49:15.506 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.483+0000 7ff8e0b83b80 1 -- 172.21.15.39:0/2159268556 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5631a619f490 con 0x5631a62f4580 2023-07-21T18:49:15.506 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.483+0000 7ff8b5ffb700 1 -- 172.21.15.39:0/2159268556 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8c40b9340 con 0x5631a62f4580 2023-07-21T18:49:15.506 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.483+0000 7ff8e0b83b80 10 client.16707 fetch_fsmap learned FSMap version 56 2023-07-21T18:49:15.507 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.483+0000 7ff8e0b83b80 1 -- 172.21.15.39:0/2159268556 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x5631a605cc00 con 0x5631a62f4580 2023-07-21T18:49:15.507 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.483+0000 7ff8b5ffb700 1 -- 172.21.15.39:0/2159268556 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5631a619f490 con 0x5631a62f4580 2023-07-21T18:49:15.507 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.483+0000 7ff8b5ffb700 1 -- 172.21.15.39:0/2159268556 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 56) v1 ==== 7482+0+0 (secure 0 0 0) 0x7ff8c40a3170 con 0x5631a62f4580 2023-07-21T18:49:15.507 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.483+0000 7ff8e0b83b80 10 client.16707 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T18:49:15.507 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.483+0000 7ff8e0b83b80 10 client.16707 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:49:15.507 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.483+0000 7ff8e0b83b80 20 client.16707 populate_metadata read hostname 'smithi039' 2023-07-21T18:49:15.507 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.483+0000 7ff8e0b83b80 1 --2- 172.21.15.39:0/2159268556 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5631a630d330 0x5631a630f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:15.507 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.483+0000 7ff8e0b83b80 4 client.16707 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:49:15.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.483+0000 7ff8e0b83b80 1 -- 172.21.15.39:0/2159268556 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5631a623ee50 con 0x5631a630d330 2023-07-21T18:49:15.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.484+0000 7ff8ce59c700 1 --2- 172.21.15.39:0/2159268556 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5631a630d330 0x5631a630f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:15.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.484+0000 7ff8b5ffb700 10 client.16707 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:49:15.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.484+0000 7ff8ce59c700 1 --2- 172.21.15.39:0/2159268556 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5631a630d330 0x5631a630f710 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae4631604b9332f server_cookie=7db7a214cf076245 in_seq=0 out_seq=0 2023-07-21T18:49:15.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.485+0000 7ff8b5ffb700 1 -- 172.21.15.39:0/2159268556 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5631a623ee50 con 0x5631a630d330 2023-07-21T18:49:15.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.485+0000 7ff8b5ffb700 10 client.16707 handle_command_reply: tid=0 2023-07-21T18:49:15.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.485+0000 7ff8e0b83b80 1 client.16707 shutdown 2023-07-21T18:49:15.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.485+0000 7ff8e0b83b80 1 -- 172.21.15.39:0/2159268556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5631a62f4580 msgr2=0x5631a6303860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:49:15.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.485+0000 7ff8e0b83b80 1 --2- 172.21.15.39:0/2159268556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5631a62f4580 0x5631a6303860 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7ff8c40a30d0 tx=0x7ff8c401be10 comp rx=0 tx=0).stop 2023-07-21T18:49:15.510 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.485+0000 7ff8e0b83b80 1 -- 172.21.15.39:0/2159268556 shutdown_connections 2023-07-21T18:49:15.510 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.485+0000 7ff8e0b83b80 1 --2- 172.21.15.39:0/2159268556 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5631a630d330 0x5631a630f710 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:15.510 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.485+0000 7ff8e0b83b80 1 --2- 172.21.15.39:0/2159268556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631a62fa2d0 0x5631a62fd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:15.510 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.485+0000 7ff8b5ffb700 0 client.16707 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:49:15.510 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.485+0000 7ff8e0b83b80 1 --2- 172.21.15.39:0/2159268556 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631a61506b0 0x5631a6303320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:15.510 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.485+0000 7ff8e0b83b80 1 --2- 172.21.15.39:0/2159268556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5631a62f4580 0x5631a6303860 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:15.510 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.485+0000 7ff8e0b83b80 1 -- 172.21.15.39:0/2159268556 >> 172.21.15.39:0/2159268556 conn(0x5631a62e9be0 msgr2=0x5631a62ee4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:49:15.511 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.485+0000 7ff8e0b83b80 1 -- 172.21.15.39:0/2159268556 shutdown_connections 2023-07-21T18:49:15.511 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.485+0000 7ff8e0b83b80 1 -- 172.21.15.39:0/2159268556 wait complete. 2023-07-21T18:49:15.511 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.485+0000 7ff8e0b83b80 20 client.16707 trim_cache size 0 max 16384 2023-07-21T18:49:15.514 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.500+0000 7ff8e0b83b80 1 Processor -- start 2023-07-21T18:49:15.514 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.501+0000 7ff8e0b83b80 1 -- start start 2023-07-21T18:49:15.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.501+0000 7ff8e0b83b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5631a61506b0 0x5631a62f4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:15.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.501+0000 7ff8e0b83b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631a62f4580 0x5631a62fb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:15.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.501+0000 7ff8e0b83b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631a62fa2d0 0x5631a62eec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:15.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.501+0000 7ff8e0b83b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631a623ee50 con 0x5631a61506b0 2023-07-21T18:49:15.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.501+0000 7ff8e0b83b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631a619f490 con 0x5631a62f4580 2023-07-21T18:49:15.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.501+0000 7ff8e0b83b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631a62efef0 con 0x5631a62fa2d0 2023-07-21T18:49:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.501+0000 7ff8ce59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5631a61506b0 0x5631a62f4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.501+0000 7ff8ce59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5631a61506b0 0x5631a62f4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60224/0 (socket says 172.21.15.39:60224) 2023-07-21T18:49:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.501+0000 7ff8ce59c700 1 -- 172.21.15.39:0/1396563604 learned_addr learned my addr 172.21.15.39:0/1396563604 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:49:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.501+0000 7ff8ced9d700 1 --2- 172.21.15.39:0/1396563604 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631a62fa2d0 0x5631a62eec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.501+0000 7ff8cdd9b700 1 --2- 172.21.15.39:0/1396563604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631a62f4580 0x5631a62fb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:15.519 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.501+0000 7ff8ce59c700 1 -- 172.21.15.39:0/1396563604 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631a62fa2d0 msgr2=0x5631a62eec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:49:15.519 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.501+0000 7ff8ce59c700 1 --2- 172.21.15.39:0/1396563604 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631a62fa2d0 0x5631a62eec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:15.519 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.501+0000 7ff8ce59c700 1 -- 172.21.15.39:0/1396563604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631a62f4580 msgr2=0x5631a62fb010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:49:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.501+0000 7ff8ce59c700 1 --2- 172.21.15.39:0/1396563604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631a62f4580 0x5631a62fb010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.501+0000 7ff8ce59c700 1 -- 172.21.15.39:0/1396563604 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5631a6314340 con 0x5631a61506b0 2023-07-21T18:49:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.501+0000 7ff8cdd9b700 1 --2- 172.21.15.39:0/1396563604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631a62f4580 0x5631a62fb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:49:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.501+0000 7ff8ce59c700 1 --2- 172.21.15.39:0/1396563604 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5631a61506b0 0x5631a62f4040 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7ff8c0009870 tx=0x7ff8c0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f9b8ad69b033dc2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:49:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.502+0000 7ff8bd7fa700 1 -- 172.21.15.39:0/1396563604 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c0024070 con 0x5631a61506b0 2023-07-21T18:49:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.502+0000 7ff8bd7fa700 1 -- 172.21.15.39:0/1396563604 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff8c0025040 con 0x5631a61506b0 2023-07-21T18:49:15.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.502+0000 7ff8bd7fa700 1 -- 172.21.15.39:0/1396563604 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c0028780 con 0x5631a61506b0 2023-07-21T18:49:15.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.502+0000 7ff8e0b83b80 1 -- 172.21.15.39:0/1396563604 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5631a61506b0 msgr2=0x5631a62f4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:49:15.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.502+0000 7ff8e0b83b80 1 --2- 172.21.15.39:0/1396563604 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5631a61506b0 0x5631a62f4040 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7ff8c0009870 tx=0x7ff8c0014000 comp rx=0 tx=0).stop 2023-07-21T18:49:15.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.502+0000 7ff8e0b83b80 1 -- 172.21.15.39:0/1396563604 shutdown_connections 2023-07-21T18:49:15.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.502+0000 7ff8e0b83b80 1 --2- 172.21.15.39:0/1396563604 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631a62fa2d0 0x5631a62eec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:15.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.502+0000 7ff8e0b83b80 1 --2- 172.21.15.39:0/1396563604 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5631a61506b0 0x5631a62f4040 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:15.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.502+0000 7ff8e0b83b80 1 --2- 172.21.15.39:0/1396563604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631a62f4580 0x5631a62fb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:15.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.502+0000 7ff8e0b83b80 1 -- 172.21.15.39:0/1396563604 >> 172.21.15.39:0/1396563604 conn(0x5631a62e9be0 msgr2=0x5631a62fa900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:49:15.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.502+0000 7ff8e0b83b80 1 -- 172.21.15.39:0/1396563604 shutdown_connections 2023-07-21T18:49:15.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.502+0000 7ff8e0b83b80 1 -- 172.21.15.39:0/1396563604 wait complete. 2023-07-21T18:49:15.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.503+0000 7ff8e0b83b80 1 -- start start 2023-07-21T18:49:15.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.503+0000 7ff8e0b83b80 1 Processor -- start 2023-07-21T18:49:15.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.503+0000 7ff8e0b83b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5631a61506b0 0x5631a635c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:15.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.503+0000 7ff8e0b83b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631a62f4580 0x5631a635c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:15.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.503+0000 7ff8e0b83b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631a62fa2d0 0x5631a635cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.503+0000 7ff8e0b83b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631a6300f40 con 0x5631a61506b0 2023-07-21T18:49:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.503+0000 7ff8e0b83b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631a63010b0 con 0x5631a62fa2d0 2023-07-21T18:49:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.503+0000 7ff8e0b83b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631a6301270 con 0x5631a62f4580 2023-07-21T18:49:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.503+0000 7ff8e0b83b80 10 client.0 _ll_register_callbacks cb 0x7ff8c0009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:49:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.503+0000 7ff8ce59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5631a61506b0 0x5631a635c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.503+0000 7ff8ce59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5631a61506b0 0x5631a635c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60228/0 (socket says 172.21.15.39:60228) 2023-07-21T18:49:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.503+0000 7ff8ce59c700 1 -- 172.21.15.39:0/1453931342 learned_addr learned my addr 172.21.15.39:0/1453931342 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:49:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.503+0000 7ff8cdd9b700 1 --2- 172.21.15.39:0/1453931342 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631a62f4580 0x5631a635c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:15.524 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.503+0000 7ff8ced9d700 1 --2- 172.21.15.39:0/1453931342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631a62fa2d0 0x5631a635cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:15.524 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.504+0000 7ff8ce59c700 1 -- 172.21.15.39:0/1453931342 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631a62f4580 msgr2=0x5631a635c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:49:15.524 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.504+0000 7ff8ce59c700 1 --2- 172.21.15.39:0/1453931342 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631a62f4580 0x5631a635c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:15.524 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.504+0000 7ff8ce59c700 1 -- 172.21.15.39:0/1453931342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631a62fa2d0 msgr2=0x5631a635cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:49:15.524 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.504+0000 7ff8ce59c700 1 --2- 172.21.15.39:0/1453931342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631a62fa2d0 0x5631a635cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:15.524 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.504+0000 7ff8ce59c700 1 -- 172.21.15.39:0/1453931342 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8c0017070 con 0x5631a61506b0 2023-07-21T18:49:15.524 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.504+0000 7ff8ce59c700 1 --2- 172.21.15.39:0/1453931342 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5631a61506b0 0x5631a635c1a0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7ff8c0011ab0 tx=0x7ff8c000e9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6421fda656c7abe server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:49:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.504+0000 7ff8b5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:49:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.504+0000 7ff8b5ffb700 1 -- 172.21.15.39:0/1453931342 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c0024070 con 0x5631a61506b0 2023-07-21T18:49:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.504+0000 7ff8b5ffb700 1 -- 172.21.15.39:0/1453931342 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8527+0+0 (secure 0 0 0) 0x7ff8c0003880 con 0x5631a61506b0 2023-07-21T18:49:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.504+0000 7ff8e0b83b80 1 -- 172.21.15.39:0/1453931342 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff8c40a3600 con 0x5631a61506b0 2023-07-21T18:49:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.504+0000 7ff8b5ffb700 1 -- 172.21.15.39:0/1453931342 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8c0003a80 con 0x5631a61506b0 2023-07-21T18:49:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.504+0000 7ff8e0b83b80 10 client.16713 fetch_fsmap learned FSMap version 56 2023-07-21T18:49:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.504+0000 7ff8e0b83b80 1 -- 172.21.15.39:0/1453931342 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x5631a609a1f0 con 0x5631a61506b0 2023-07-21T18:49:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.504+0000 7ff8b5ffb700 1 -- 172.21.15.39:0/1453931342 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff8c40a3600 con 0x5631a61506b0 2023-07-21T18:49:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.504+0000 7ff8b5ffb700 1 -- 172.21.15.39:0/1453931342 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 56) v1 ==== 7482+0+0 (secure 0 0 0) 0x7ff8c001e420 con 0x5631a61506b0 2023-07-21T18:49:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.504+0000 7ff8e0b83b80 10 client.16713 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T18:49:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.504+0000 7ff8e0b83b80 10 client.16713 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:49:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.505+0000 7ff8e0b83b80 20 client.16713 populate_metadata read hostname 'smithi039' 2023-07-21T18:49:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.505+0000 7ff8e0b83b80 1 --2- 172.21.15.39:0/1453931342 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5631a630d330 0x5631a6368040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.505+0000 7ff8e0b83b80 4 client.16713 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:49:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.505+0000 7ff8e0b83b80 1 -- 172.21.15.39:0/1453931342 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5631a6200ff0 con 0x5631a630d330 2023-07-21T18:49:15.527 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.505+0000 7ff8cdd9b700 1 --2- 172.21.15.39:0/1453931342 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5631a630d330 0x5631a6368040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:15.527 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.505+0000 7ff8b5ffb700 10 client.16713 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:49:15.527 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.505+0000 7ff8cdd9b700 1 --2- 172.21.15.39:0/1453931342 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5631a630d330 0x5631a6368040 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86e52d174454238f server_cookie=a398ff0f14c11c56 in_seq=0 out_seq=0 2023-07-21T18:49:15.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.566+0000 7ff8b5ffb700 1 -- 172.21.15.39:0/1453931342 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5631a6200ff0 con 0x5631a630d330 2023-07-21T18:49:15.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.566+0000 7ff8b5ffb700 10 client.16713 handle_command_reply: tid=0 2023-07-21T18:49:15.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.566+0000 7ff8e0b83b80 1 client.16713 shutdown 2023-07-21T18:49:15.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.566+0000 7ff8e0b83b80 1 -- 172.21.15.39:0/1453931342 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5631a61506b0 msgr2=0x5631a635c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:49:15.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.566+0000 7ff8e0b83b80 1 --2- 172.21.15.39:0/1453931342 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5631a61506b0 0x5631a635c1a0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7ff8c0011ab0 tx=0x7ff8c000e9b0 comp rx=0 tx=0).stop 2023-07-21T18:49:15.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.566+0000 7ff8e0b83b80 1 -- 172.21.15.39:0/1453931342 shutdown_connections 2023-07-21T18:49:15.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.566+0000 7ff8e0b83b80 1 --2- 172.21.15.39:0/1453931342 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5631a630d330 0x5631a6368040 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:15.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.566+0000 7ff8e0b83b80 1 --2- 172.21.15.39:0/1453931342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631a62fa2d0 0x5631a635cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:15.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.566+0000 7ff8e0b83b80 1 --2- 172.21.15.39:0/1453931342 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5631a61506b0 0x5631a635c1a0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:15.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.566+0000 7ff8e0b83b80 1 --2- 172.21.15.39:0/1453931342 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631a62f4580 0x5631a635c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:15.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.566+0000 7ff8e0b83b80 1 -- 172.21.15.39:0/1453931342 >> 172.21.15.39:0/1453931342 conn(0x5631a62e9be0 msgr2=0x5631a62fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:49:15.582 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.567+0000 7ff8b5ffb700 0 client.16713 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:49:15.582 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.567+0000 7ff8e0b83b80 1 -- 172.21.15.39:0/1453931342 shutdown_connections 2023-07-21T18:49:15.582 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.567+0000 7ff8e0b83b80 1 -- 172.21.15.39:0/1453931342 wait complete. 2023-07-21T18:49:15.582 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.567+0000 7ff8e0b83b80 20 client.16713 trim_cache size 0 max 16384 2023-07-21T18:49:15.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:49:15.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:49:15.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:49:15.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:49:15.584 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.567+0000 7ff8d49fd700 1 -- 172.21.15.39:0/3617187342 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7ff8b80a12c0 msgr2=0x7ff8b80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:49:15.584 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.567+0000 7ff8d49fd700 1 --2- 172.21.15.39:0/3617187342 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7ff8b80a12c0 0x7ff8b80a3770 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff8c0009d70 tx=0x7ff8c0009260 comp rx=0 tx=0).stop 2023-07-21T18:49:15.584 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.567+0000 7ff8d49fd700 1 -- 172.21.15.39:0/3617187342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8d00ff490 msgr2=0x7ff8d01971b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:49:15.584 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.567+0000 7ff8d49fd700 1 --2- 172.21.15.39:0/3617187342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8d00ff490 0x7ff8d01971b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff8c4000c00 tx=0x7ff8c4005740 comp rx=0 tx=0).stop 2023-07-21T18:49:15.585 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.568+0000 7ff8d49fd700 1 -- 172.21.15.39:0/3617187342 shutdown_connections 2023-07-21T18:49:15.585 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.568+0000 7ff8d49fd700 1 --2- 172.21.15.39:0/3617187342 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8d0104af0 0x7ff8d01a1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:15.585 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.568+0000 7ff8d49fd700 1 --2- 172.21.15.39:0/3617187342 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8d00feae0 0x7ff8d0196c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:15.585 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.568+0000 7ff8d49fd700 1 --2- 172.21.15.39:0/3617187342 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7ff8b80a12c0 0x7ff8b80a3770 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:15.585 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.568+0000 7ff8d49fd700 1 --2- 172.21.15.39:0/3617187342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8d00ff490 0x7ff8d01971b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:15.585 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.568+0000 7ff8d49fd700 1 -- 172.21.15.39:0/3617187342 >> 172.21.15.39:0/3617187342 conn(0x7ff8d0076a10 msgr2=0x7ff8d00fc580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:49:15.585 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.568+0000 7ff8d49fd700 1 -- 172.21.15.39:0/3617187342 shutdown_connections 2023-07-21T18:49:15.585 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:15.568+0000 7ff8d49fd700 1 -- 172.21.15.39:0/3617187342 wait complete. 2023-07-21T18:49:16.277 INFO:tasks.workunit.client.0.smithi039.stdout: 1 487008 6.76 MB/sec execute 269 sec latency 99.975 ms 2023-07-21T18:49:16.361 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:49:16 smithi193 ceph-mon[98204]: pgmap v233: 97 pgs: 97 active+clean; 181 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 284 op/s 2023-07-21T18:49:16.380 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:49:16 smithi155 ceph-mon[97934]: pgmap v233: 97 pgs: 97 active+clean; 181 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 284 op/s 2023-07-21T18:49:16.429 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:49:16 smithi039 ceph-mon[96489]: pgmap v233: 97 pgs: 97 active+clean; 181 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 284 op/s 2023-07-21T18:49:17.277 INFO:tasks.workunit.client.0.smithi039.stdout: 1 488429 6.76 MB/sec execute 270 sec latency 25.681 ms 2023-07-21T18:49:18.277 INFO:tasks.workunit.client.0.smithi039.stdout: 1 490570 6.78 MB/sec execute 271 sec latency 14.884 ms 2023-07-21T18:49:18.380 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:49:18 smithi155 ceph-mon[97934]: pgmap v234: 97 pgs: 97 active+clean; 188 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 260 op/s 2023-07-21T18:49:18.429 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:49:18 smithi039 ceph-mon[96489]: pgmap v234: 97 pgs: 97 active+clean; 188 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 260 op/s 2023-07-21T18:49:18.611 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:49:18 smithi193 ceph-mon[98204]: pgmap v234: 97 pgs: 97 active+clean; 188 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 260 op/s 2023-07-21T18:49:19.277 INFO:tasks.workunit.client.0.smithi039.stdout: 1 492226 6.78 MB/sec execute 272 sec latency 14.534 ms 2023-07-21T18:49:20.277 INFO:tasks.workunit.client.0.smithi039.stdout: 1 494234 6.80 MB/sec execute 273 sec latency 12.963 ms 2023-07-21T18:49:20.380 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:49:20 smithi155 ceph-mon[97934]: pgmap v235: 97 pgs: 97 active+clean; 188 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 260 op/s 2023-07-21T18:49:20.429 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:49:20 smithi039 ceph-mon[96489]: pgmap v235: 97 pgs: 97 active+clean; 188 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 260 op/s 2023-07-21T18:49:20.611 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:49:20 smithi193 ceph-mon[98204]: pgmap v235: 97 pgs: 97 active+clean; 188 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 260 op/s 2023-07-21T18:49:21.277 INFO:tasks.workunit.client.0.smithi039.stdout: 1 496206 6.80 MB/sec execute 274 sec latency 22.169 ms 2023-07-21T18:49:22.277 INFO:tasks.workunit.client.0.smithi039.stdout: 1 497770 6.82 MB/sec execute 275 sec latency 34.147 ms 2023-07-21T18:49:22.380 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:49:22 smithi155 ceph-mon[97934]: pgmap v236: 97 pgs: 97 active+clean; 186 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.1 MiB/s wr, 294 op/s 2023-07-21T18:49:22.429 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:49:22 smithi039 ceph-mon[96489]: pgmap v236: 97 pgs: 97 active+clean; 186 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.1 MiB/s wr, 294 op/s 2023-07-21T18:49:22.611 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:49:22 smithi193 ceph-mon[98204]: pgmap v236: 97 pgs: 97 active+clean; 186 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.1 MiB/s wr, 294 op/s 2023-07-21T18:49:23.277 INFO:tasks.workunit.client.0.smithi039.stdout: 1 499698 6.82 MB/sec execute 276 sec latency 14.057 ms 2023-07-21T18:49:24.277 INFO:tasks.workunit.client.0.smithi039.stdout: 1 501334 6.84 MB/sec execute 277 sec latency 12.989 ms 2023-07-21T18:49:24.380 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:49:24 smithi155 ceph-mon[97934]: pgmap v237: 97 pgs: 97 active+clean; 186 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 261 op/s 2023-07-21T18:49:24.428 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:49:24 smithi039 ceph-mon[96489]: pgmap v237: 97 pgs: 97 active+clean; 186 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 261 op/s 2023-07-21T18:49:24.611 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:49:24 smithi193 ceph-mon[98204]: pgmap v237: 97 pgs: 97 active+clean; 186 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 261 op/s 2023-07-21T18:49:25.277 INFO:tasks.workunit.client.0.smithi039.stdout: 1 503310 6.84 MB/sec execute 278 sec latency 15.252 ms 2023-07-21T18:49:26.277 INFO:tasks.workunit.client.0.smithi039.stdout: 1 504922 6.86 MB/sec execute 279 sec latency 12.693 ms 2023-07-21T18:49:26.429 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:49:26 smithi039 ceph-mon[96489]: pgmap v238: 97 pgs: 97 active+clean; 186 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 245 op/s 2023-07-21T18:49:26.611 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:49:26 smithi193 ceph-mon[98204]: pgmap v238: 97 pgs: 97 active+clean; 186 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 245 op/s 2023-07-21T18:49:26.634 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:49:26 smithi155 ceph-mon[97934]: pgmap v238: 97 pgs: 97 active+clean; 186 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 245 op/s 2023-07-21T18:49:27.277 INFO:tasks.workunit.client.0.smithi039.stdout: 1 506829 6.86 MB/sec execute 280 sec latency 59.546 ms 2023-07-21T18:49:28.277 INFO:tasks.workunit.client.0.smithi039.stdout: 1 508434 6.88 MB/sec execute 281 sec latency 17.977 ms 2023-07-21T18:49:28.428 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:49:28 smithi039 ceph-mon[96489]: pgmap v239: 97 pgs: 97 active+clean; 193 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 357 op/s 2023-07-21T18:49:28.611 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:49:28 smithi193 ceph-mon[98204]: pgmap v239: 97 pgs: 97 active+clean; 193 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 357 op/s 2023-07-21T18:49:28.630 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:49:28 smithi155 ceph-mon[97934]: pgmap v239: 97 pgs: 97 active+clean; 193 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 357 op/s 2023-07-21T18:49:28.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.687+0000 7fbfaa719700 1 -- 172.21.15.39:0/4285611266 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfa4104a30 msgr2=0x7fbfa4104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:49:28.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.687+0000 7fbfaa719700 1 --2- 172.21.15.39:0/4285611266 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfa4104a30 0x7fbfa4104e00 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fbf94009990 tx=0x7fbf94009ca0 comp rx=0 tx=0).stop 2023-07-21T18:49:28.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.687+0000 7fbfaa719700 1 -- 172.21.15.39:0/4285611266 shutdown_connections 2023-07-21T18:49:28.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.687+0000 7fbfaa719700 1 --2- 172.21.15.39:0/4285611266 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbfa40ff3e0 0x7fbfa410b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:28.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.687+0000 7fbfaa719700 1 --2- 172.21.15.39:0/4285611266 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfa4104a30 0x7fbfa4104e00 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:28.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.687+0000 7fbfaa719700 1 --2- 172.21.15.39:0/4285611266 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfa40fea30 0x7fbfa40feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:28.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.687+0000 7fbfaa719700 1 -- 172.21.15.39:0/4285611266 >> 172.21.15.39:0/4285611266 conn(0x7fbfa4074760 msgr2=0x7fbfa4074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:49:28.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.687+0000 7fbfaa719700 1 -- 172.21.15.39:0/4285611266 shutdown_connections 2023-07-21T18:49:28.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.687+0000 7fbfaa719700 1 -- 172.21.15.39:0/4285611266 wait complete. 2023-07-21T18:49:28.703 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.688+0000 7fbfaa719700 1 Processor -- start 2023-07-21T18:49:28.703 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.688+0000 7fbfaa719700 1 -- start start 2023-07-21T18:49:28.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.688+0000 7fbfaa719700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfa40fea30 0x7fbfa4197d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:28.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.688+0000 7fbfaa719700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbfa40ff3e0 0x7fbfa41982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:28.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.688+0000 7fbfaa719700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfa4104a30 0x7fbfa419c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:28.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.688+0000 7fbfaa719700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa419cf80 con 0x7fbfa40fea30 2023-07-21T18:49:28.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.688+0000 7fbfaa719700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa419d0f0 con 0x7fbfa4104a30 2023-07-21T18:49:28.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.688+0000 7fbfaa719700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa419d2b0 con 0x7fbfa40ff3e0 2023-07-21T18:49:28.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.689+0000 7fbfa3fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfa40fea30 0x7fbfa4197d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:28.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.689+0000 7fbfa3fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfa40fea30 0x7fbfa4197d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60240/0 (socket says 172.21.15.39:60240) 2023-07-21T18:49:28.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.689+0000 7fbfa3fff700 1 -- 172.21.15.39:0/3062683115 learned_addr learned my addr 172.21.15.39:0/3062683115 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:49:28.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.689+0000 7fbfa37fe700 1 --2- 172.21.15.39:0/3062683115 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbfa40ff3e0 0x7fbfa41982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:28.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.689+0000 7fbfa8cb6700 1 --2- 172.21.15.39:0/3062683115 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfa4104a30 0x7fbfa419c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:28.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.689+0000 7fbfa3fff700 1 -- 172.21.15.39:0/3062683115 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbfa40ff3e0 msgr2=0x7fbfa41982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:49:28.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.689+0000 7fbfa3fff700 1 --2- 172.21.15.39:0/3062683115 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbfa40ff3e0 0x7fbfa41982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:28.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.689+0000 7fbfa3fff700 1 -- 172.21.15.39:0/3062683115 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfa4104a30 msgr2=0x7fbfa419c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:49:28.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.689+0000 7fbfa3fff700 1 --2- 172.21.15.39:0/3062683115 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfa4104a30 0x7fbfa419c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:28.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.689+0000 7fbfa3fff700 1 -- 172.21.15.39:0/3062683115 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf940095f0 con 0x7fbfa40fea30 2023-07-21T18:49:28.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.689+0000 7fbfa3fff700 1 --2- 172.21.15.39:0/3062683115 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfa40fea30 0x7fbfa4197d90 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fbf9400b2e0 tx=0x7fbf94004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:49:28.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.689+0000 7fbfa17fa700 1 -- 172.21.15.39:0/3062683115 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9401d070 con 0x7fbfa40fea30 2023-07-21T18:49:28.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.689+0000 7fbfaa719700 1 -- 172.21.15.39:0/3062683115 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfa419d530 con 0x7fbfa40fea30 2023-07-21T18:49:28.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.689+0000 7fbfa17fa700 1 -- 172.21.15.39:0/3062683115 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf94005030 con 0x7fbfa40fea30 2023-07-21T18:49:28.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.689+0000 7fbfa17fa700 1 -- 172.21.15.39:0/3062683115 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf94004a40 con 0x7fbfa40fea30 2023-07-21T18:49:28.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.689+0000 7fbfaa719700 1 -- 172.21.15.39:0/3062683115 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfa41a4dd0 con 0x7fbfa40fea30 2023-07-21T18:49:28.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.692+0000 7fbfa17fa700 1 -- 172.21.15.39:0/3062683115 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fbf94005430 con 0x7fbfa40fea30 2023-07-21T18:49:28.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.693+0000 7fbfa17fa700 1 --2- 172.21.15.39:0/3062683115 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fbf8c0a12c0 0x7fbf8c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:28.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.693+0000 7fbfa17fa700 1 -- 172.21.15.39:0/3062683115 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fbf940cdcf0 con 0x7fbfa40fea30 2023-07-21T18:49:28.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.693+0000 7fbfa37fe700 1 --2- 172.21.15.39:0/3062683115 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fbf8c0a12c0 0x7fbf8c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:28.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.693+0000 7fbfb689fb80 1 Processor -- start 2023-07-21T18:49:28.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.693+0000 7fbfb689fb80 1 -- start start 2023-07-21T18:49:28.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.693+0000 7fbfb689fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5630c3f5b6b0 0x5630c4104d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:28.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.693+0000 7fbfb689fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5630c41052d0 0x5630c40ff040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:28.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.693+0000 7fbfb689fb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5630c40ff580 0x5630c40ff9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:28.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.693+0000 7fbfb689fb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630c4004c50 con 0x5630c40ff580 2023-07-21T18:49:28.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.693+0000 7fbfb689fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630c40579e0 con 0x5630c3f5b6b0 2023-07-21T18:49:28.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.693+0000 7fbfa37fe700 1 --2- 172.21.15.39:0/3062683115 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fbf8c0a12c0 0x7fbf8c0a3770 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fbfa41995b0 tx=0x7fbf9800b410 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:49:28.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.693+0000 7fbfb689fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630c4050e70 con 0x5630c41052d0 2023-07-21T18:49:28.710 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.693+0000 7fbfa8cb6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5630c40ff580 0x5630c40ff9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:28.710 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.694+0000 7fbfa8cb6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5630c40ff580 0x5630c40ff9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60250/0 (socket says 172.21.15.39:60250) 2023-07-21T18:49:28.710 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.694+0000 7fbfa8cb6700 1 -- 172.21.15.39:0/260900555 learned_addr learned my addr 172.21.15.39:0/260900555 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:49:28.710 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.694+0000 7fbfa37fe700 1 --2- 172.21.15.39:0/260900555 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5630c41052d0 0x5630c40ff040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:28.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.694+0000 7fbfa8cb6700 1 -- 172.21.15.39:0/260900555 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5630c41052d0 msgr2=0x5630c40ff040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:49:28.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.694+0000 7fbfa8cb6700 1 --2- 172.21.15.39:0/260900555 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5630c41052d0 0x5630c40ff040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:28.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.694+0000 7fbfa8cb6700 1 -- 172.21.15.39:0/260900555 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5630c3f5b6b0 msgr2=0x5630c4104d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:49:28.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.694+0000 7fbfa8cb6700 1 --2- 172.21.15.39:0/260900555 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5630c3f5b6b0 0x5630c4104d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:28.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.694+0000 7fbfa8cb6700 1 -- 172.21.15.39:0/260900555 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5630c3e7ed80 con 0x5630c40ff580 2023-07-21T18:49:28.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.694+0000 7fbfa8cb6700 1 --2- 172.21.15.39:0/260900555 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5630c40ff580 0x5630c40ff9d0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c011680 tx=0x7fbf9c009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b75b547abcca9d2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:49:28.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.694+0000 7fbf8a7fc700 1 -- 172.21.15.39:0/260900555 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9c00e040 con 0x5630c40ff580 2023-07-21T18:49:28.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.694+0000 7fbf8a7fc700 1 -- 172.21.15.39:0/260900555 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf9c00f040 con 0x5630c40ff580 2023-07-21T18:49:28.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.694+0000 7fbf8a7fc700 1 -- 172.21.15.39:0/260900555 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9c0048c0 con 0x5630c40ff580 2023-07-21T18:49:28.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.695+0000 7fbfb689fb80 1 -- 172.21.15.39:0/260900555 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5630c40ff580 msgr2=0x5630c40ff9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:49:28.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.695+0000 7fbfb689fb80 1 --2- 172.21.15.39:0/260900555 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5630c40ff580 0x5630c40ff9d0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c011680 tx=0x7fbf9c009920 comp rx=0 tx=0).stop 2023-07-21T18:49:28.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.696+0000 7fbfb689fb80 1 -- 172.21.15.39:0/260900555 shutdown_connections 2023-07-21T18:49:28.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.696+0000 7fbfb689fb80 1 --2- 172.21.15.39:0/260900555 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5630c3f5b6b0 0x5630c4104d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:28.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.696+0000 7fbfb689fb80 1 --2- 172.21.15.39:0/260900555 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5630c40ff580 0x5630c40ff9d0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:28.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.696+0000 7fbfb689fb80 1 --2- 172.21.15.39:0/260900555 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5630c41052d0 0x5630c40ff040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:28.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.696+0000 7fbfb689fb80 1 -- 172.21.15.39:0/260900555 >> 172.21.15.39:0/260900555 conn(0x5630c40f4be0 msgr2=0x5630c40f9f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:49:28.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.696+0000 7fbfb689fb80 1 -- 172.21.15.39:0/260900555 shutdown_connections 2023-07-21T18:49:28.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.696+0000 7fbfb689fb80 1 -- 172.21.15.39:0/260900555 wait complete. 2023-07-21T18:49:28.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.696+0000 7fbfb689fb80 1 -- start start 2023-07-21T18:49:28.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.697+0000 7fbfb689fb80 1 Processor -- start 2023-07-21T18:49:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.697+0000 7fbfb689fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5630c3f5b6b0 0x5630c41089e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:28.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.697+0000 7fbfb689fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5630c40ff580 0x5630c4108f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:28.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.697+0000 7fbfb689fb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5630c41052d0 0x5630c4109460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:28.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.697+0000 7fbfb689fb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630c3ff0a60 con 0x5630c41052d0 2023-07-21T18:49:28.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.697+0000 7fbfb689fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630c3ff0760 con 0x5630c40ff580 2023-07-21T18:49:28.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.697+0000 7fbfb689fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630c3ed48c0 con 0x5630c3f5b6b0 2023-07-21T18:49:28.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.697+0000 7fbfb689fb80 10 client.0 _ll_register_callbacks cb 0x5630c3fee700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:49:28.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.697+0000 7fbfa3fff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5630c3f5b6b0 0x5630c41089e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:28.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.697+0000 7fbfa3fff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5630c3f5b6b0 0x5630c41089e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:50574/0 (socket says 172.21.15.39:50574) 2023-07-21T18:49:28.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.697+0000 7fbfa3fff700 1 -- 172.21.15.39:0/212104389 learned_addr learned my addr 172.21.15.39:0/212104389 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:49:28.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.697+0000 7fbfa37fe700 1 --2- 172.21.15.39:0/212104389 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5630c40ff580 0x5630c4108f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:28.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.698+0000 7fbfa3fff700 1 -- 172.21.15.39:0/212104389 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5630c40ff580 msgr2=0x5630c4108f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:49:28.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.698+0000 7fbfa3fff700 1 --2- 172.21.15.39:0/212104389 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5630c40ff580 0x5630c4108f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:28.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.698+0000 7fbfa3fff700 1 -- 172.21.15.39:0/212104389 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5630c41052d0 msgr2=0x5630c4109460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:49:28.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.698+0000 7fbfa8cb6700 1 --2- 172.21.15.39:0/212104389 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5630c41052d0 0x5630c4109460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:28.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.698+0000 7fbfa3fff700 1 --2- 172.21.15.39:0/212104389 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5630c41052d0 0x5630c4109460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:28.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.698+0000 7fbfa3fff700 1 -- 172.21.15.39:0/212104389 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf9c011330 con 0x5630c3f5b6b0 2023-07-21T18:49:28.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.698+0000 7fbfa8cb6700 1 --2- 172.21.15.39:0/212104389 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5630c41052d0 0x5630c4109460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:49:28.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.698+0000 7fbfa3fff700 1 --2- 172.21.15.39:0/212104389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5630c3f5b6b0 0x5630c41089e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbf940cba40 tx=0x7fbf9409aed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d09f31b5c49b4aa server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:49:28.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.698+0000 7fbf837fe700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2023-07-21T18:49:28.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.698+0000 7fbf837fe700 1 -- 172.21.15.39:0/212104389 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9409a540 con 0x5630c3f5b6b0 2023-07-21T18:49:28.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.698+0000 7fbfb689fb80 1 -- 172.21.15.39:0/212104389 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5630c3faa490 con 0x5630c3f5b6b0 2023-07-21T18:49:28.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.698+0000 7fbf837fe700 1 -- 172.21.15.39:0/212104389 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fbf940a3210 con 0x5630c3f5b6b0 2023-07-21T18:49:28.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.699+0000 7fbf837fe700 1 -- 172.21.15.39:0/212104389 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf940a2d80 con 0x5630c3f5b6b0 2023-07-21T18:49:28.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.699+0000 7fbfb689fb80 10 client.26209 fetch_fsmap learned FSMap version 56 2023-07-21T18:49:28.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.699+0000 7fbfb689fb80 1 -- 172.21.15.39:0/212104389 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x5630c3e67c00 con 0x5630c3f5b6b0 2023-07-21T18:49:28.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.699+0000 7fbf837fe700 1 -- 172.21.15.39:0/212104389 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5630c3faa490 con 0x5630c3f5b6b0 2023-07-21T18:49:28.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.699+0000 7fbf837fe700 1 -- 172.21.15.39:0/212104389 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 56) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fbf9409d070 con 0x5630c3f5b6b0 2023-07-21T18:49:28.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.699+0000 7fbfb689fb80 10 client.26209 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T18:49:28.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.699+0000 7fbfb689fb80 10 client.26209 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:49:28.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.699+0000 7fbfb689fb80 20 client.26209 populate_metadata read hostname 'smithi039' 2023-07-21T18:49:28.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.699+0000 7fbfb689fb80 1 --2- 172.21.15.39:0/212104389 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5630c4118950 0x5630c411ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:28.720 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.699+0000 7fbfb689fb80 4 client.26209 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:49:28.720 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.699+0000 7fbfb689fb80 1 -- 172.21.15.39:0/212104389 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5630c4049e50 con 0x5630c4118950 2023-07-21T18:49:28.720 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.700+0000 7fbfa37fe700 1 --2- 172.21.15.39:0/212104389 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5630c4118950 0x5630c411ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:28.720 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.700+0000 7fbfa37fe700 1 --2- 172.21.15.39:0/212104389 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5630c4118950 0x5630c411ad30 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71125f6716230500 server_cookie=ea437c0abffefcf7 in_seq=0 out_seq=0 2023-07-21T18:49:28.720 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.700+0000 7fbf837fe700 10 client.26209 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:49:28.720 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.701+0000 7fbf837fe700 1 -- 172.21.15.39:0/212104389 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5630c4049e50 con 0x5630c4118950 2023-07-21T18:49:28.720 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.701+0000 7fbf837fe700 10 client.26209 handle_command_reply: tid=0 2023-07-21T18:49:28.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.701+0000 7fbfb689fb80 1 client.26209 shutdown 2023-07-21T18:49:28.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.701+0000 7fbfb689fb80 1 -- 172.21.15.39:0/212104389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5630c3f5b6b0 msgr2=0x5630c41089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:49:28.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.701+0000 7fbfb689fb80 1 --2- 172.21.15.39:0/212104389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5630c3f5b6b0 0x5630c41089e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbf940cba40 tx=0x7fbf9409aed0 comp rx=0 tx=0).stop 2023-07-21T18:49:28.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.701+0000 7fbfb689fb80 1 -- 172.21.15.39:0/212104389 shutdown_connections 2023-07-21T18:49:28.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.701+0000 7fbfb689fb80 1 --2- 172.21.15.39:0/212104389 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5630c4118950 0x5630c411ad30 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:28.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.701+0000 7fbfb689fb80 1 --2- 172.21.15.39:0/212104389 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5630c41052d0 0x5630c4109460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:28.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.701+0000 7fbfb689fb80 1 --2- 172.21.15.39:0/212104389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5630c3f5b6b0 0x5630c41089e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:28.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.701+0000 7fbfb689fb80 1 --2- 172.21.15.39:0/212104389 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5630c40ff580 0x5630c4108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:28.722 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.701+0000 7fbf837fe700 0 client.26209 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:49:28.722 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.701+0000 7fbfb689fb80 1 -- 172.21.15.39:0/212104389 >> 172.21.15.39:0/212104389 conn(0x5630c40f4be0 msgr2=0x5630c40f8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:49:28.722 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.702+0000 7fbfb689fb80 1 -- 172.21.15.39:0/212104389 shutdown_connections 2023-07-21T18:49:28.722 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.702+0000 7fbfb689fb80 1 -- 172.21.15.39:0/212104389 wait complete. 2023-07-21T18:49:28.722 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.702+0000 7fbfb689fb80 20 client.26209 trim_cache size 0 max 16384 2023-07-21T18:49:28.732 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.719+0000 7fbfb689fb80 1 Processor -- start 2023-07-21T18:49:28.732 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.719+0000 7fbfb689fb80 1 -- start start 2023-07-21T18:49:28.734 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.719+0000 7fbfb689fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5630c40f4be0 0x5630c40ff040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:28.734 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.719+0000 7fbfb689fb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5630c3f5b6b0 0x5630c40f9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:28.734 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.719+0000 7fbfb689fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5630c40ff580 0x5630c4109290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:28.734 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.719+0000 7fbfb689fb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630c3faa490 con 0x5630c3f5b6b0 2023-07-21T18:49:28.734 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.719+0000 7fbfb689fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630c40f1c00 con 0x5630c40f4be0 2023-07-21T18:49:28.734 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.719+0000 7fbfb689fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630c40c5060 con 0x5630c40ff580 2023-07-21T18:49:28.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.720+0000 7fbfa37fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5630c3f5b6b0 0x5630c40f9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:28.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.720+0000 7fbfa37fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5630c3f5b6b0 0x5630c40f9c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60264/0 (socket says 172.21.15.39:60264) 2023-07-21T18:49:28.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.720+0000 7fbfa37fe700 1 -- 172.21.15.39:0/331810979 learned_addr learned my addr 172.21.15.39:0/331810979 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:49:28.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.720+0000 7fbfa3fff700 1 --2- 172.21.15.39:0/331810979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5630c40f4be0 0x5630c40ff040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:28.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.720+0000 7fbfa8cb6700 1 --2- 172.21.15.39:0/331810979 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5630c40ff580 0x5630c4109290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:28.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.720+0000 7fbfa37fe700 1 -- 172.21.15.39:0/331810979 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5630c40ff580 msgr2=0x5630c4109290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:49:28.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.720+0000 7fbfa37fe700 1 --2- 172.21.15.39:0/331810979 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5630c40ff580 0x5630c4109290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.720+0000 7fbfa37fe700 1 -- 172.21.15.39:0/331810979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5630c40f4be0 msgr2=0x5630c40ff040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:49:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.720+0000 7fbfa37fe700 1 --2- 172.21.15.39:0/331810979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5630c40f4be0 0x5630c40ff040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.720+0000 7fbfa37fe700 1 -- 172.21.15.39:0/331810979 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5630c40ffc20 con 0x5630c3f5b6b0 2023-07-21T18:49:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.720+0000 7fbfa3fff700 1 --2- 172.21.15.39:0/331810979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5630c40f4be0 0x5630c40ff040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:49:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.720+0000 7fbfa37fe700 1 --2- 172.21.15.39:0/331810979 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5630c3f5b6b0 0x5630c40f9c60 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fbf98005950 tx=0x7fbf98013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ad57468fc46c51f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:49:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.720+0000 7fbf8affd700 1 -- 172.21.15.39:0/331810979 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf98033070 con 0x5630c3f5b6b0 2023-07-21T18:49:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.720+0000 7fbf8affd700 1 -- 172.21.15.39:0/331810979 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf9802a040 con 0x5630c3f5b6b0 2023-07-21T18:49:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.720+0000 7fbf8affd700 1 -- 172.21.15.39:0/331810979 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf98011ab0 con 0x5630c3f5b6b0 2023-07-21T18:49:28.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.721+0000 7fbfb689fb80 1 -- 172.21.15.39:0/331810979 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5630c3f5b6b0 msgr2=0x5630c40f9c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:49:28.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.721+0000 7fbfb689fb80 1 --2- 172.21.15.39:0/331810979 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5630c3f5b6b0 0x5630c40f9c60 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fbf98005950 tx=0x7fbf98013000 comp rx=0 tx=0).stop 2023-07-21T18:49:28.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.721+0000 7fbfb689fb80 1 -- 172.21.15.39:0/331810979 shutdown_connections 2023-07-21T18:49:28.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.721+0000 7fbfb689fb80 1 --2- 172.21.15.39:0/331810979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5630c40f4be0 0x5630c40ff040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:28.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.721+0000 7fbfb689fb80 1 --2- 172.21.15.39:0/331810979 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5630c40ff580 0x5630c4109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.721+0000 7fbfb689fb80 1 --2- 172.21.15.39:0/331810979 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5630c3f5b6b0 0x5630c40f9c60 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.721+0000 7fbfb689fb80 1 -- 172.21.15.39:0/331810979 >> 172.21.15.39:0/331810979 conn(0x5630c4102ff0 msgr2=0x5630c4107190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:49:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.721+0000 7fbfb689fb80 1 -- 172.21.15.39:0/331810979 shutdown_connections 2023-07-21T18:49:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.721+0000 7fbfb689fb80 1 -- 172.21.15.39:0/331810979 wait complete. 2023-07-21T18:49:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.721+0000 7fbfb689fb80 1 -- start start 2023-07-21T18:49:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.722+0000 7fbfb689fb80 1 Processor -- start 2023-07-21T18:49:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.722+0000 7fbfb689fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5630c3f5b6b0 0x5630c410fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.722+0000 7fbfb689fb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5630c40f4be0 0x5630c410e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.722+0000 7fbfb689fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5630c40ff580 0x5630c410e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.722+0000 7fbfb689fb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630c410ecf0 con 0x5630c40f4be0 2023-07-21T18:49:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.722+0000 7fbfb689fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630c410ee60 con 0x5630c40ff580 2023-07-21T18:49:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.722+0000 7fbfb689fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630c416c7c0 con 0x5630c3f5b6b0 2023-07-21T18:49:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.722+0000 7fbfb689fb80 10 client.0 _ll_register_callbacks cb 0x7fbf9409a990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:49:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.722+0000 7fbfa37fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5630c40f4be0 0x5630c410e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.722+0000 7fbfa37fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5630c40f4be0 0x5630c410e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60270/0 (socket says 172.21.15.39:60270) 2023-07-21T18:49:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.722+0000 7fbfa37fe700 1 -- 172.21.15.39:0/2542478395 learned_addr learned my addr 172.21.15.39:0/2542478395 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:49:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.722+0000 7fbfa3fff700 1 --2- 172.21.15.39:0/2542478395 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5630c3f5b6b0 0x5630c410fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.722+0000 7fbfa8cb6700 1 --2- 172.21.15.39:0/2542478395 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5630c40ff580 0x5630c410e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.723+0000 7fbfa37fe700 1 -- 172.21.15.39:0/2542478395 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5630c3f5b6b0 msgr2=0x5630c410fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:49:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.723+0000 7fbfa37fe700 1 --2- 172.21.15.39:0/2542478395 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5630c3f5b6b0 0x5630c410fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.723+0000 7fbfa37fe700 1 -- 172.21.15.39:0/2542478395 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5630c40ff580 msgr2=0x5630c410e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:49:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.723+0000 7fbfa37fe700 1 --2- 172.21.15.39:0/2542478395 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5630c40ff580 0x5630c410e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.723+0000 7fbfa37fe700 1 -- 172.21.15.39:0/2542478395 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5630c3e67c00 con 0x5630c40f4be0 2023-07-21T18:49:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.723+0000 7fbfa37fe700 1 --2- 172.21.15.39:0/2542478395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5630c40f4be0 0x5630c410e0a0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fbf98000c00 tx=0x7fbf98023f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be4ba04cb4e68da5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:49:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.723+0000 7fbf837fe700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:49:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.723+0000 7fbf837fe700 1 -- 172.21.15.39:0/2542478395 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf98033040 con 0x5630c40f4be0 2023-07-21T18:49:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.723+0000 7fbf837fe700 1 -- 172.21.15.39:0/2542478395 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fbf98017070 con 0x5630c40f4be0 2023-07-21T18:49:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.723+0000 7fbfb689fb80 1 -- 172.21.15.39:0/2542478395 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbf9c00f040 con 0x5630c40f4be0 2023-07-21T18:49:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.723+0000 7fbf837fe700 1 -- 172.21.15.39:0/2542478395 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf9802fb60 con 0x5630c40f4be0 2023-07-21T18:49:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.723+0000 7fbfb689fb80 10 client.16728 fetch_fsmap learned FSMap version 56 2023-07-21T18:49:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.723+0000 7fbfb689fb80 1 -- 172.21.15.39:0/2542478395 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x7fbf98023b40 con 0x5630c40f4be0 2023-07-21T18:49:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.723+0000 7fbf837fe700 1 -- 172.21.15.39:0/2542478395 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf9c00f040 con 0x5630c40f4be0 2023-07-21T18:49:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.724+0000 7fbf837fe700 1 -- 172.21.15.39:0/2542478395 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 56) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fbf980385f0 con 0x5630c40f4be0 2023-07-21T18:49:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.724+0000 7fbfb689fb80 10 client.16728 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T18:49:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.724+0000 7fbfb689fb80 10 client.16728 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:49:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.724+0000 7fbfb689fb80 20 client.16728 populate_metadata read hostname 'smithi039' 2023-07-21T18:49:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.724+0000 7fbfb689fb80 1 --2- 172.21.15.39:0/2542478395 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5630c41052d0 0x5630c416fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.724+0000 7fbfb689fb80 4 client.16728 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:49:28.745 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.724+0000 7fbfb689fb80 1 -- 172.21.15.39:0/2542478395 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5630c400bff0 con 0x5630c41052d0 2023-07-21T18:49:28.745 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.724+0000 7fbfa3fff700 1 --2- 172.21.15.39:0/2542478395 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5630c41052d0 0x5630c416fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:28.745 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.724+0000 7fbfa3fff700 1 --2- 172.21.15.39:0/2542478395 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5630c41052d0 0x5630c416fdf0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d78f66acb1151ff server_cookie=cb01af6a0b43535d in_seq=0 out_seq=0 2023-07-21T18:49:28.745 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.724+0000 7fbf837fe700 10 client.16728 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:49:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.778+0000 7fbf837fe700 1 -- 172.21.15.39:0/2542478395 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5630c400bff0 con 0x5630c41052d0 2023-07-21T18:49:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.778+0000 7fbf837fe700 10 client.16728 handle_command_reply: tid=0 2023-07-21T18:49:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.778+0000 7fbfb689fb80 1 client.16728 shutdown 2023-07-21T18:49:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.779+0000 7fbfb689fb80 1 -- 172.21.15.39:0/2542478395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5630c40f4be0 msgr2=0x5630c410e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:49:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.779+0000 7fbfb689fb80 1 --2- 172.21.15.39:0/2542478395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5630c40f4be0 0x5630c410e0a0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fbf98000c00 tx=0x7fbf98023f20 comp rx=0 tx=0).stop 2023-07-21T18:49:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.779+0000 7fbfb689fb80 1 -- 172.21.15.39:0/2542478395 shutdown_connections 2023-07-21T18:49:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.779+0000 7fbfb689fb80 1 --2- 172.21.15.39:0/2542478395 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5630c41052d0 0x5630c416fdf0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.779+0000 7fbfb689fb80 1 --2- 172.21.15.39:0/2542478395 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5630c40ff580 0x5630c410e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.779+0000 7fbfb689fb80 1 --2- 172.21.15.39:0/2542478395 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5630c3f5b6b0 0x5630c410fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.779+0000 7fbfb689fb80 1 --2- 172.21.15.39:0/2542478395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5630c40f4be0 0x5630c410e0a0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.779+0000 7fbf837fe700 0 client.16728 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:49:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.779+0000 7fbfb689fb80 1 -- 172.21.15.39:0/2542478395 >> 172.21.15.39:0/2542478395 conn(0x5630c4102ff0 msgr2=0x5630c4107a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:49:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.779+0000 7fbfb689fb80 1 -- 172.21.15.39:0/2542478395 shutdown_connections 2023-07-21T18:49:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.779+0000 7fbfb689fb80 1 -- 172.21.15.39:0/2542478395 wait complete. 2023-07-21T18:49:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.779+0000 7fbfb689fb80 20 client.16728 trim_cache size 0 max 16384 2023-07-21T18:49:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:49:28.795 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:49:28.795 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:49:28.795 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:49:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.779+0000 7fbfaa719700 1 -- 172.21.15.39:0/3062683115 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fbf8c0a12c0 msgr2=0x7fbf8c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:49:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.779+0000 7fbfaa719700 1 --2- 172.21.15.39:0/3062683115 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fbf8c0a12c0 0x7fbf8c0a3770 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fbfa41995b0 tx=0x7fbf9800b410 comp rx=0 tx=0).stop 2023-07-21T18:49:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.779+0000 7fbfaa719700 1 -- 172.21.15.39:0/3062683115 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfa40fea30 msgr2=0x7fbfa4197d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:49:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.779+0000 7fbfaa719700 1 --2- 172.21.15.39:0/3062683115 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfa40fea30 0x7fbfa4197d90 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fbf9400b2e0 tx=0x7fbf94004730 comp rx=0 tx=0).stop 2023-07-21T18:49:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.780+0000 7fbfaa719700 1 -- 172.21.15.39:0/3062683115 shutdown_connections 2023-07-21T18:49:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.780+0000 7fbfaa719700 1 --2- 172.21.15.39:0/3062683115 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fbf8c0a12c0 0x7fbf8c0a3770 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.780+0000 7fbfaa719700 1 --2- 172.21.15.39:0/3062683115 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfa4104a30 0x7fbfa419c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.780+0000 7fbfaa719700 1 --2- 172.21.15.39:0/3062683115 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfa40fea30 0x7fbfa4197d90 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.780+0000 7fbfaa719700 1 --2- 172.21.15.39:0/3062683115 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbfa40ff3e0 0x7fbfa41982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.780+0000 7fbfaa719700 1 -- 172.21.15.39:0/3062683115 >> 172.21.15.39:0/3062683115 conn(0x7fbfa4074760 msgr2=0x7fbfa4067920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:49:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.780+0000 7fbfaa719700 1 -- 172.21.15.39:0/3062683115 shutdown_connections 2023-07-21T18:49:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:28.780+0000 7fbfaa719700 1 -- 172.21.15.39:0/3062683115 wait complete. 2023-07-21T18:49:29.277 INFO:tasks.workunit.client.0.smithi039.stdout: 1 509729 6.87 MB/sec execute 282 sec latency 133.579 ms 2023-07-21T18:49:30.277 INFO:tasks.workunit.client.0.smithi039.stdout: 1 509729 6.85 MB/sec execute 283 sec latency 1133.628 ms 2023-07-21T18:49:30.429 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:49:30 smithi039 ceph-mon[96489]: pgmap v240: 97 pgs: 97 active+clean; 193 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 251 op/s 2023-07-21T18:49:30.429 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:49:30 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:49:30.611 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:49:30 smithi193 ceph-mon[98204]: pgmap v240: 97 pgs: 97 active+clean; 193 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 251 op/s 2023-07-21T18:49:30.611 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:49:30 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:49:30.630 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:49:30 smithi155 ceph-mon[97934]: pgmap v240: 97 pgs: 97 active+clean; 193 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 251 op/s 2023-07-21T18:49:30.630 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:49:30 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:49:31.277 INFO:tasks.workunit.client.0.smithi039.stdout: 1 509729 6.83 MB/sec execute 284 sec latency 2133.683 ms 2023-07-21T18:49:32.277 INFO:tasks.workunit.client.0.smithi039.stdout: 1 509729 6.80 MB/sec execute 285 sec latency 3133.773 ms 2023-07-21T18:49:32.429 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:49:32 smithi039 ceph-mon[96489]: pgmap v241: 97 pgs: 97 active+clean; 200 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.8 MiB/s wr, 375 op/s 2023-07-21T18:49:32.611 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:49:32 smithi193 ceph-mon[98204]: pgmap v241: 97 pgs: 97 active+clean; 200 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.8 MiB/s wr, 375 op/s 2023-07-21T18:49:32.630 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:49:32 smithi155 ceph-mon[97934]: pgmap v241: 97 pgs: 97 active+clean; 200 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.8 MiB/s wr, 375 op/s 2023-07-21T18:49:33.277 INFO:tasks.workunit.client.0.smithi039.stdout: 1 509729 6.78 MB/sec execute 286 sec latency 4133.862 ms 2023-07-21T18:49:34.278 INFO:tasks.workunit.client.0.smithi039.stdout: 1 509729 6.75 MB/sec execute 287 sec latency 5133.986 ms 2023-07-21T18:49:34.429 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:49:34 smithi039 ceph-mon[96489]: pgmap v242: 97 pgs: 97 active+clean; 200 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1024 KiB/s rd, 3.5 MiB/s wr, 236 op/s 2023-07-21T18:49:34.611 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:49:34 smithi193 ceph-mon[98204]: pgmap v242: 97 pgs: 97 active+clean; 200 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1024 KiB/s rd, 3.5 MiB/s wr, 236 op/s 2023-07-21T18:49:34.630 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:49:34 smithi155 ceph-mon[97934]: pgmap v242: 97 pgs: 97 active+clean; 200 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1024 KiB/s rd, 3.5 MiB/s wr, 236 op/s 2023-07-21T18:49:35.278 INFO:tasks.workunit.client.0.smithi039.stdout: 1 509729 6.73 MB/sec execute 288 sec latency 6134.083 ms 2023-07-21T18:49:36.278 INFO:tasks.workunit.client.0.smithi039.stdout: 1 509729 6.71 MB/sec execute 289 sec latency 7134.181 ms 2023-07-21T18:49:36.429 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:49:36 smithi039 ceph-mon[96489]: pgmap v243: 97 pgs: 97 active+clean; 200 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1024 KiB/s rd, 3.5 MiB/s wr, 236 op/s 2023-07-21T18:49:36.611 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:49:36 smithi193 ceph-mon[98204]: pgmap v243: 97 pgs: 97 active+clean; 200 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1024 KiB/s rd, 3.5 MiB/s wr, 236 op/s 2023-07-21T18:49:36.630 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:49:36 smithi155 ceph-mon[97934]: pgmap v243: 97 pgs: 97 active+clean; 200 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1024 KiB/s rd, 3.5 MiB/s wr, 236 op/s 2023-07-21T18:49:37.278 INFO:tasks.workunit.client.0.smithi039.stdout: 1 509729 6.68 MB/sec execute 290 sec latency 8134.283 ms 2023-07-21T18:49:38.278 INFO:tasks.workunit.client.0.smithi039.stdout: 1 509729 6.66 MB/sec execute 291 sec latency 9134.395 ms 2023-07-21T18:49:38.429 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:49:38 smithi039 ceph-mon[96489]: pgmap v244: 97 pgs: 97 active+clean; 205 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.2 MiB/s wr, 254 op/s 2023-07-21T18:49:38.611 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:49:38 smithi193 ceph-mon[98204]: pgmap v244: 97 pgs: 97 active+clean; 205 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.2 MiB/s wr, 254 op/s 2023-07-21T18:49:38.630 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:49:38 smithi155 ceph-mon[97934]: pgmap v244: 97 pgs: 97 active+clean; 205 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.2 MiB/s wr, 254 op/s 2023-07-21T18:49:39.278 INFO:tasks.workunit.client.0.smithi039.stdout: 1 509809 6.64 MB/sec execute 292 sec latency 9270.338 ms 2023-07-21T18:49:40.278 INFO:tasks.workunit.client.0.smithi039.stdout: 1 509809 6.62 MB/sec execute 293 sec latency 1838.791 ms 2023-07-21T18:49:40.429 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:49:40 smithi039 ceph-mon[96489]: pgmap v245: 97 pgs: 97 active+clean; 205 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 2.9 MiB/s wr, 142 op/s 2023-07-21T18:49:40.611 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:49:40 smithi193 ceph-mon[98204]: pgmap v245: 97 pgs: 97 active+clean; 205 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 2.9 MiB/s wr, 142 op/s 2023-07-21T18:49:40.630 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:49:40 smithi155 ceph-mon[97934]: pgmap v245: 97 pgs: 97 active+clean; 205 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 2.9 MiB/s wr, 142 op/s 2023-07-21T18:49:41.278 INFO:tasks.workunit.client.0.smithi039.stdout: 1 509809 6.59 MB/sec execute 294 sec latency 2838.871 ms 2023-07-21T18:49:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.902+0000 7f6c3cc36700 1 -- 172.21.15.39:0/1892562228 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c38105c10 msgr2=0x7f6c38105fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:49:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.902+0000 7f6c3cc36700 1 --2- 172.21.15.39:0/1892562228 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c38105c10 0x7f6c38105fe0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f6c28009990 tx=0x7f6c28009ca0 comp rx=0 tx=0).stop 2023-07-21T18:49:41.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.902+0000 7f6c3cc36700 1 -- 172.21.15.39:0/1892562228 shutdown_connections 2023-07-21T18:49:41.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.902+0000 7f6c3cc36700 1 --2- 172.21.15.39:0/1892562228 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c380fea20 0x7f6c3810c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:41.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.902+0000 7f6c3cc36700 1 --2- 172.21.15.39:0/1892562228 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c38105c10 0x7f6c38105fe0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:41.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.902+0000 7f6c3cc36700 1 --2- 172.21.15.39:0/1892562228 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c380fe070 0x7f6c380fe4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:41.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.902+0000 7f6c3cc36700 1 -- 172.21.15.39:0/1892562228 >> 172.21.15.39:0/1892562228 conn(0x7f6c38075f90 msgr2=0x7f6c38076390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:49:41.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.902+0000 7f6c3cc36700 1 -- 172.21.15.39:0/1892562228 shutdown_connections 2023-07-21T18:49:41.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.902+0000 7f6c3cc36700 1 -- 172.21.15.39:0/1892562228 wait complete. 2023-07-21T18:49:41.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.903+0000 7f6c3cc36700 1 Processor -- start 2023-07-21T18:49:41.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.903+0000 7f6c3cc36700 1 -- start start 2023-07-21T18:49:41.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.903+0000 7f6c3cc36700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c380fe070 0x7f6c38195da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:41.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.903+0000 7f6c3cc36700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c380fea20 0x7f6c381962e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:41.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.904+0000 7f6c3cc36700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c38105c10 0x7f6c3819a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:41.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.904+0000 7f6c3cc36700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3819af90 con 0x7f6c38105c10 2023-07-21T18:49:41.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.904+0000 7f6c3cc36700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3819b100 con 0x7f6c380fea20 2023-07-21T18:49:41.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.904+0000 7f6c3cc36700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3819b2c0 con 0x7f6c380fe070 2023-07-21T18:49:41.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.904+0000 7f6c36d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c38105c10 0x7f6c3819a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:41.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.904+0000 7f6c36d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c38105c10 0x7f6c3819a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60286/0 (socket says 172.21.15.39:60286) 2023-07-21T18:49:41.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.904+0000 7f6c36d9d700 1 -- 172.21.15.39:0/3815317075 learned_addr learned my addr 172.21.15.39:0/3815317075 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:49:41.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.904+0000 7f6c3659c700 1 --2- 172.21.15.39:0/3815317075 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c380fe070 0x7f6c38195da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:41.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.904+0000 7f6c2ffff700 1 --2- 172.21.15.39:0/3815317075 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c380fea20 0x7f6c381962e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:41.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.904+0000 7f6c36d9d700 1 -- 172.21.15.39:0/3815317075 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c380fe070 msgr2=0x7f6c38195da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:49:41.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.904+0000 7f6c36d9d700 1 --2- 172.21.15.39:0/3815317075 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c380fe070 0x7f6c38195da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:41.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.904+0000 7f6c36d9d700 1 -- 172.21.15.39:0/3815317075 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c380fea20 msgr2=0x7f6c381962e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:49:41.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.904+0000 7f6c36d9d700 1 --2- 172.21.15.39:0/3815317075 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c380fea20 0x7f6c381962e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:41.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.904+0000 7f6c36d9d700 1 -- 172.21.15.39:0/3815317075 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c280095f0 con 0x7f6c38105c10 2023-07-21T18:49:41.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.904+0000 7f6c2ffff700 1 --2- 172.21.15.39:0/3815317075 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c380fea20 0x7f6c381962e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:49:41.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.904+0000 7f6c36d9d700 1 --2- 172.21.15.39:0/3815317075 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c38105c10 0x7f6c3819a950 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f6c3000b970 tx=0x7f6c3000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:49:41.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.904+0000 7f6c2f7fe700 1 -- 172.21.15.39:0/3815317075 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3000c930 con 0x7f6c38105c10 2023-07-21T18:49:41.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.904+0000 7f6c2f7fe700 1 -- 172.21.15.39:0/3815317075 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c30014440 con 0x7f6c38105c10 2023-07-21T18:49:41.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.904+0000 7f6c3cc36700 1 -- 172.21.15.39:0/3815317075 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c3819b5a0 con 0x7f6c38105c10 2023-07-21T18:49:41.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.905+0000 7f6c2f7fe700 1 -- 172.21.15.39:0/3815317075 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c300075d0 con 0x7f6c38105c10 2023-07-21T18:49:41.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.905+0000 7f6c3cc36700 1 -- 172.21.15.39:0/3815317075 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c381a2de0 con 0x7f6c38105c10 2023-07-21T18:49:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.907+0000 7f6c2f7fe700 1 -- 172.21.15.39:0/3815317075 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f6c300145b0 con 0x7f6c38105c10 2023-07-21T18:49:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.907+0000 7f6c2f7fe700 1 --2- 172.21.15.39:0/3815317075 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f6c240a12c0 0x7f6c240a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.908+0000 7f6c2f7fe700 1 -- 172.21.15.39:0/3815317075 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f6c30006e40 con 0x7f6c38105c10 2023-07-21T18:49:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.908+0000 7f6c3659c700 1 --2- 172.21.15.39:0/3815317075 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f6c240a12c0 0x7f6c240a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.908+0000 7f6c48dbcb80 1 Processor -- start 2023-07-21T18:49:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.908+0000 7f6c3659c700 1 --2- 172.21.15.39:0/3815317075 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f6c240a12c0 0x7f6c240a3770 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6c2800b5c0 tx=0x7f6c28005fd0 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:49:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.908+0000 7f6c48dbcb80 1 -- start start 2023-07-21T18:49:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.909+0000 7f6c48dbcb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e25e64c6b0 0x55e25e7f5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.909+0000 7f6c48dbcb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e25e7f62d0 0x55e25e7f0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.909+0000 7f6c48dbcb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e25e7f0580 0x55e25e7f09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.909+0000 7f6c48dbcb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e25e6f5c50 con 0x55e25e7f62d0 2023-07-21T18:49:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.909+0000 7f6c48dbcb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e25e7489e0 con 0x55e25e64c6b0 2023-07-21T18:49:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.909+0000 7f6c48dbcb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e25e741e70 con 0x55e25e7f0580 2023-07-21T18:49:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.909+0000 7f6c2ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e25e7f62d0 0x55e25e7f0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.909+0000 7f6c2ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e25e7f62d0 0x55e25e7f0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60292/0 (socket says 172.21.15.39:60292) 2023-07-21T18:49:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.909+0000 7f6c2ffff700 1 -- 172.21.15.39:0/3992791481 learned_addr learned my addr 172.21.15.39:0/3992791481 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:49:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.909+0000 7f6c3659c700 1 --2- 172.21.15.39:0/3992791481 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e25e64c6b0 0x55e25e7f5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.909+0000 7f6c36d9d700 1 --2- 172.21.15.39:0/3992791481 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e25e7f0580 0x55e25e7f09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.909+0000 7f6c2ffff700 1 -- 172.21.15.39:0/3992791481 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e25e7f0580 msgr2=0x55e25e7f09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:49:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.909+0000 7f6c2ffff700 1 --2- 172.21.15.39:0/3992791481 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e25e7f0580 0x55e25e7f09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.909+0000 7f6c2ffff700 1 -- 172.21.15.39:0/3992791481 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e25e64c6b0 msgr2=0x55e25e7f5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:49:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.909+0000 7f6c2ffff700 1 --2- 172.21.15.39:0/3992791481 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e25e64c6b0 0x55e25e7f5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.909+0000 7f6c2ffff700 1 -- 172.21.15.39:0/3992791481 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e25e56fd80 con 0x55e25e7f62d0 2023-07-21T18:49:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.909+0000 7f6c3659c700 1 --2- 172.21.15.39:0/3992791481 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e25e64c6b0 0x55e25e7f5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:49:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.909+0000 7f6c36d9d700 1 --2- 172.21.15.39:0/3992791481 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e25e7f0580 0x55e25e7f09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:49:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.909+0000 7f6c2ffff700 1 --2- 172.21.15.39:0/3992791481 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e25e7f62d0 0x55e25e7f0040 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f6c20007870 tx=0x7f6c20007b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50215db63c9a9166 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:49:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.909+0000 7f6c2cff9700 1 -- 172.21.15.39:0/3992791481 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2000d670 con 0x55e25e7f62d0 2023-07-21T18:49:41.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.909+0000 7f6c2cff9700 1 -- 172.21.15.39:0/3992791481 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c200074e0 con 0x55e25e7f62d0 2023-07-21T18:49:41.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.909+0000 7f6c2cff9700 1 -- 172.21.15.39:0/3992791481 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c20013d40 con 0x55e25e7f62d0 2023-07-21T18:49:41.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.910+0000 7f6c48dbcb80 1 -- 172.21.15.39:0/3992791481 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e25e7f62d0 msgr2=0x55e25e7f0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:49:41.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.910+0000 7f6c48dbcb80 1 --2- 172.21.15.39:0/3992791481 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e25e7f62d0 0x55e25e7f0040 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f6c20007870 tx=0x7f6c20007b80 comp rx=0 tx=0).stop 2023-07-21T18:49:41.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.910+0000 7f6c48dbcb80 1 -- 172.21.15.39:0/3992791481 shutdown_connections 2023-07-21T18:49:41.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.910+0000 7f6c48dbcb80 1 --2- 172.21.15.39:0/3992791481 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e25e64c6b0 0x55e25e7f5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:41.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.910+0000 7f6c48dbcb80 1 --2- 172.21.15.39:0/3992791481 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e25e7f0580 0x55e25e7f09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:41.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.910+0000 7f6c48dbcb80 1 --2- 172.21.15.39:0/3992791481 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e25e7f62d0 0x55e25e7f0040 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.910+0000 7f6c48dbcb80 1 -- 172.21.15.39:0/3992791481 >> 172.21.15.39:0/3992791481 conn(0x55e25e7e5be0 msgr2=0x55e25e7eaf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:49:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.910+0000 7f6c48dbcb80 1 -- 172.21.15.39:0/3992791481 shutdown_connections 2023-07-21T18:49:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.910+0000 7f6c48dbcb80 1 -- 172.21.15.39:0/3992791481 wait complete. 2023-07-21T18:49:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.911+0000 7f6c48dbcb80 1 -- start start 2023-07-21T18:49:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.911+0000 7f6c48dbcb80 1 Processor -- start 2023-07-21T18:49:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.911+0000 7f6c48dbcb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e25e64c6b0 0x55e25e7fc520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.911+0000 7f6c48dbcb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e25e7f0580 0x55e25e7f9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.911+0000 7f6c48dbcb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e25e7f62d0 0x55e25e7f9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.911+0000 7f6c48dbcb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e25e6e1a60 con 0x55e25e7f0580 2023-07-21T18:49:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.911+0000 7f6c48dbcb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e25e6e1760 con 0x55e25e64c6b0 2023-07-21T18:49:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.911+0000 7f6c48dbcb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e25e5c58c0 con 0x55e25e7f62d0 2023-07-21T18:49:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.911+0000 7f6c48dbcb80 10 client.0 _ll_register_callbacks cb 0x55e25e6df700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:49:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.911+0000 7f6c2ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e25e7f0580 0x55e25e7f9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.911+0000 7f6c2ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e25e7f0580 0x55e25e7f9810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60298/0 (socket says 172.21.15.39:60298) 2023-07-21T18:49:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.911+0000 7f6c2ffff700 1 -- 172.21.15.39:0/840364096 learned_addr learned my addr 172.21.15.39:0/840364096 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:49:41.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.911+0000 7f6c36d9d700 1 --2- 172.21.15.39:0/840364096 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e25e7f62d0 0x55e25e7f9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:41.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.912+0000 7f6c3659c700 1 --2- 172.21.15.39:0/840364096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e25e64c6b0 0x55e25e7fc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:41.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.912+0000 7f6c2ffff700 1 -- 172.21.15.39:0/840364096 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e25e7f62d0 msgr2=0x55e25e7f9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:49:41.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.912+0000 7f6c2ffff700 1 --2- 172.21.15.39:0/840364096 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e25e7f62d0 0x55e25e7f9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:41.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.912+0000 7f6c2ffff700 1 -- 172.21.15.39:0/840364096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e25e64c6b0 msgr2=0x55e25e7fc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:49:41.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.912+0000 7f6c2ffff700 1 --2- 172.21.15.39:0/840364096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e25e64c6b0 0x55e25e7fc520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:41.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.912+0000 7f6c2ffff700 1 -- 172.21.15.39:0/840364096 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c2000e040 con 0x55e25e7f0580 2023-07-21T18:49:41.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.912+0000 7f6c2ffff700 1 --2- 172.21.15.39:0/840364096 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e25e7f0580 0x55e25e7f9810 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f6c20006e50 tx=0x7f6c200074c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27bb7ec56e1261a5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:49:41.935 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.912+0000 7f6c1dffb700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:49:41.935 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.912+0000 7f6c1dffb700 1 -- 172.21.15.39:0/840364096 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c20013c50 con 0x55e25e7f0580 2023-07-21T18:49:41.935 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.912+0000 7f6c1dffb700 1 -- 172.21.15.39:0/840364096 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f6c20013db0 con 0x55e25e7f0580 2023-07-21T18:49:41.935 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.912+0000 7f6c48dbcb80 1 -- 172.21.15.39:0/840364096 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e25e73ae50 con 0x55e25e7f0580 2023-07-21T18:49:41.935 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.912+0000 7f6c1dffb700 1 -- 172.21.15.39:0/840364096 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c2000d670 con 0x55e25e7f0580 2023-07-21T18:49:41.935 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.912+0000 7f6c48dbcb80 10 client.16740 fetch_fsmap learned FSMap version 57 2023-07-21T18:49:41.935 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.912+0000 7f6c48dbcb80 1 -- 172.21.15.39:0/840364096 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x55e25e558c00 con 0x55e25e7f0580 2023-07-21T18:49:41.935 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.913+0000 7f6c1dffb700 1 -- 172.21.15.39:0/840364096 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e25e73ae50 con 0x55e25e7f0580 2023-07-21T18:49:41.936 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.913+0000 7f6c1dffb700 1 -- 172.21.15.39:0/840364096 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 57) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f6c2001d8b0 con 0x55e25e7f0580 2023-07-21T18:49:41.936 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.913+0000 7f6c48dbcb80 10 client.16740 fetch_fsmap finished waiting for FSMap version 57 2023-07-21T18:49:41.936 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.913+0000 7f6c48dbcb80 10 client.16740 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:49:41.936 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.913+0000 7f6c48dbcb80 20 client.16740 populate_metadata read hostname 'smithi039' 2023-07-21T18:49:41.936 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.913+0000 7f6c48dbcb80 1 --2- 172.21.15.39:0/840364096 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55e25e809950 0x55e25e80bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:41.936 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.913+0000 7f6c48dbcb80 4 client.16740 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:49:41.936 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.913+0000 7f6c48dbcb80 1 -- 172.21.15.39:0/840364096 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e25e6fcff0 con 0x55e25e809950 2023-07-21T18:49:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.913+0000 7f6c3659c700 1 --2- 172.21.15.39:0/840364096 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55e25e809950 0x55e25e80bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.914+0000 7f6c3659c700 1 --2- 172.21.15.39:0/840364096 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55e25e809950 0x55e25e80bd60 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcf3e5762cef6ae3 server_cookie=d809db2ffe0659f7 in_seq=0 out_seq=0 2023-07-21T18:49:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.914+0000 7f6c1dffb700 10 client.16740 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:49:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.914+0000 7f6c1dffb700 1 -- 172.21.15.39:0/840364096 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e25e6fcff0 con 0x55e25e809950 2023-07-21T18:49:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.914+0000 7f6c1dffb700 10 client.16740 handle_command_reply: tid=0 2023-07-21T18:49:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.914+0000 7f6c48dbcb80 1 client.16740 shutdown 2023-07-21T18:49:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.914+0000 7f6c48dbcb80 1 -- 172.21.15.39:0/840364096 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e25e7f0580 msgr2=0x55e25e7f9810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:49:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.914+0000 7f6c48dbcb80 1 --2- 172.21.15.39:0/840364096 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e25e7f0580 0x55e25e7f9810 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f6c20006e50 tx=0x7f6c200074c0 comp rx=0 tx=0).stop 2023-07-21T18:49:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.914+0000 7f6c48dbcb80 1 -- 172.21.15.39:0/840364096 shutdown_connections 2023-07-21T18:49:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.914+0000 7f6c48dbcb80 1 --2- 172.21.15.39:0/840364096 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55e25e809950 0x55e25e80bd60 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.914+0000 7f6c48dbcb80 1 --2- 172.21.15.39:0/840364096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e25e64c6b0 0x55e25e7fc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.914+0000 7f6c48dbcb80 1 --2- 172.21.15.39:0/840364096 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e25e7f62d0 0x55e25e7f9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.914+0000 7f6c48dbcb80 1 --2- 172.21.15.39:0/840364096 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e25e7f0580 0x55e25e7f9810 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.914+0000 7f6c1dffb700 0 client.16740 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:49:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.914+0000 7f6c48dbcb80 1 -- 172.21.15.39:0/840364096 >> 172.21.15.39:0/840364096 conn(0x55e25e7e5be0 msgr2=0x55e25e7ea4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:49:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.915+0000 7f6c48dbcb80 1 -- 172.21.15.39:0/840364096 shutdown_connections 2023-07-21T18:49:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.915+0000 7f6c48dbcb80 1 -- 172.21.15.39:0/840364096 wait complete. 2023-07-21T18:49:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.915+0000 7f6c48dbcb80 20 client.16740 trim_cache size 0 max 16384 2023-07-21T18:49:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.930+0000 7f6c48dbcb80 1 Processor -- start 2023-07-21T18:49:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.930+0000 7f6c48dbcb80 1 -- start start 2023-07-21T18:49:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.930+0000 7f6c48dbcb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e25e64c6b0 0x55e25e7f0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.930+0000 7f6c48dbcb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e25e7f0580 0x55e25e7eac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.930+0000 7f6c48dbcb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e25e7f62d0 0x55e25e7f8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.930+0000 7f6c48dbcb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e25e6fcff0 con 0x55e25e64c6b0 2023-07-21T18:49:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.930+0000 7f6c48dbcb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e25e73ae50 con 0x55e25e7f62d0 2023-07-21T18:49:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.930+0000 7f6c48dbcb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e25e806520 con 0x55e25e7f0580 2023-07-21T18:49:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.930+0000 7f6c3659c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e25e64c6b0 0x55e25e7f0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.930+0000 7f6c3659c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e25e64c6b0 0x55e25e7f0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60304/0 (socket says 172.21.15.39:60304) 2023-07-21T18:49:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.930+0000 7f6c3659c700 1 -- 172.21.15.39:0/493879533 learned_addr learned my addr 172.21.15.39:0/493879533 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:49:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.930+0000 7f6c2ffff700 1 --2- 172.21.15.39:0/493879533 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e25e7f0580 0x55e25e7eac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.930+0000 7f6c36d9d700 1 --2- 172.21.15.39:0/493879533 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e25e7f62d0 0x55e25e7f8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.930+0000 7f6c3659c700 1 -- 172.21.15.39:0/493879533 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e25e7f0580 msgr2=0x55e25e7eac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:49:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.930+0000 7f6c3659c700 1 --2- 172.21.15.39:0/493879533 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e25e7f0580 0x55e25e7eac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.930+0000 7f6c3659c700 1 -- 172.21.15.39:0/493879533 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e25e7f62d0 msgr2=0x55e25e7f8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:49:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.930+0000 7f6c3659c700 1 --2- 172.21.15.39:0/493879533 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e25e7f62d0 0x55e25e7f8fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.930+0000 7f6c3659c700 1 -- 172.21.15.39:0/493879533 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e25e5961f0 con 0x55e25e64c6b0 2023-07-21T18:49:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.930+0000 7f6c2ffff700 1 --2- 172.21.15.39:0/493879533 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e25e7f0580 0x55e25e7eac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:49:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.931+0000 7f6c3659c700 1 --2- 172.21.15.39:0/493879533 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e25e64c6b0 0x55e25e7f0040 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f6c280098e0 tx=0x7f6c28020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a787c3cbe672b601 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:49:41.948 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.931+0000 7f6c2d7fa700 1 -- 172.21.15.39:0/493879533 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c28033b20 con 0x55e25e64c6b0 2023-07-21T18:49:41.948 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.931+0000 7f6c2d7fa700 1 -- 172.21.15.39:0/493879533 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c2801e040 con 0x55e25e64c6b0 2023-07-21T18:49:41.948 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.931+0000 7f6c2d7fa700 1 -- 172.21.15.39:0/493879533 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c280334e0 con 0x55e25e64c6b0 2023-07-21T18:49:41.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.931+0000 7f6c48dbcb80 1 -- 172.21.15.39:0/493879533 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e25e64c6b0 msgr2=0x55e25e7f0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:49:41.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.931+0000 7f6c48dbcb80 1 --2- 172.21.15.39:0/493879533 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e25e64c6b0 0x55e25e7f0040 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f6c280098e0 tx=0x7f6c28020000 comp rx=0 tx=0).stop 2023-07-21T18:49:41.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.931+0000 7f6c48dbcb80 1 -- 172.21.15.39:0/493879533 shutdown_connections 2023-07-21T18:49:41.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.931+0000 7f6c48dbcb80 1 --2- 172.21.15.39:0/493879533 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e25e7f62d0 0x55e25e7f8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:41.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.931+0000 7f6c48dbcb80 1 --2- 172.21.15.39:0/493879533 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e25e64c6b0 0x55e25e7f0040 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:41.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.931+0000 7f6c48dbcb80 1 --2- 172.21.15.39:0/493879533 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e25e7f0580 0x55e25e7eac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:41.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.931+0000 7f6c48dbcb80 1 -- 172.21.15.39:0/493879533 >> 172.21.15.39:0/493879533 conn(0x55e25e7e5be0 msgr2=0x55e25e7f3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:49:41.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.931+0000 7f6c48dbcb80 1 -- 172.21.15.39:0/493879533 shutdown_connections 2023-07-21T18:49:41.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.931+0000 7f6c48dbcb80 1 -- 172.21.15.39:0/493879533 wait complete. 2023-07-21T18:49:41.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.932+0000 7f6c48dbcb80 1 -- start start 2023-07-21T18:49:41.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.932+0000 7f6c48dbcb80 1 Processor -- start 2023-07-21T18:49:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.932+0000 7f6c48dbcb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e25e64c6b0 0x55e25e7fec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.932+0000 7f6c48dbcb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e25e7f0580 0x55e25e7ff190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.932+0000 7f6c48dbcb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e25e7f62d0 0x55e25e860be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.932+0000 7f6c48dbcb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e25e7ffaa0 con 0x55e25e64c6b0 2023-07-21T18:49:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.932+0000 7f6c48dbcb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e25e861120 con 0x55e25e7f62d0 2023-07-21T18:49:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.932+0000 7f6c48dbcb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e25e861260 con 0x55e25e7f0580 2023-07-21T18:49:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.932+0000 7f6c48dbcb80 10 client.0 _ll_register_callbacks cb 0x7f6c200076f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:49:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.932+0000 7f6c3659c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e25e64c6b0 0x55e25e7fec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.932+0000 7f6c3659c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e25e64c6b0 0x55e25e7fec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60310/0 (socket says 172.21.15.39:60310) 2023-07-21T18:49:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.932+0000 7f6c3659c700 1 -- 172.21.15.39:0/315222475 learned_addr learned my addr 172.21.15.39:0/315222475 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:49:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.932+0000 7f6c2ffff700 1 --2- 172.21.15.39:0/315222475 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e25e7f0580 0x55e25e7ff190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.932+0000 7f6c3659c700 1 -- 172.21.15.39:0/315222475 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e25e7f0580 msgr2=0x55e25e7ff190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:49:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.932+0000 7f6c3659c700 1 --2- 172.21.15.39:0/315222475 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e25e7f0580 0x55e25e7ff190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.932+0000 7f6c3659c700 1 -- 172.21.15.39:0/315222475 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e25e7f62d0 msgr2=0x55e25e860be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:49:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.932+0000 7f6c3659c700 1 --2- 172.21.15.39:0/315222475 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e25e7f62d0 0x55e25e860be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.932+0000 7f6c3659c700 1 -- 172.21.15.39:0/315222475 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c3810d2d0 con 0x55e25e64c6b0 2023-07-21T18:49:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.933+0000 7f6c3659c700 1 --2- 172.21.15.39:0/315222475 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e25e64c6b0 0x55e25e7fec50 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f6c2800c010 tx=0x7f6c2800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f806a6f1195d6536 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:49:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.933+0000 7f6c1dffb700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:49:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.933+0000 7f6c1dffb700 1 -- 172.21.15.39:0/315222475 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c28005650 con 0x55e25e64c6b0 2023-07-21T18:49:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.933+0000 7f6c1dffb700 1 -- 172.21.15.39:0/315222475 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f6c28033180 con 0x55e25e64c6b0 2023-07-21T18:49:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.933+0000 7f6c48dbcb80 1 -- 172.21.15.39:0/315222475 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e25e69b490 con 0x55e25e64c6b0 2023-07-21T18:49:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.933+0000 7f6c1dffb700 1 -- 172.21.15.39:0/315222475 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c28033380 con 0x55e25e64c6b0 2023-07-21T18:49:41.955 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.933+0000 7f6c48dbcb80 10 client.16746 fetch_fsmap learned FSMap version 57 2023-07-21T18:49:41.955 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.933+0000 7f6c48dbcb80 1 -- 172.21.15.39:0/315222475 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x55e25e56e250 con 0x55e25e64c6b0 2023-07-21T18:49:41.955 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.933+0000 7f6c1dffb700 1 -- 172.21.15.39:0/315222475 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e25e69b490 con 0x55e25e64c6b0 2023-07-21T18:49:41.955 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.933+0000 7f6c1dffb700 1 -- 172.21.15.39:0/315222475 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 57) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f6c28044560 con 0x55e25e64c6b0 2023-07-21T18:49:41.955 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.933+0000 7f6c48dbcb80 10 client.16746 fetch_fsmap finished waiting for FSMap version 57 2023-07-21T18:49:41.955 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.933+0000 7f6c48dbcb80 10 client.16746 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:49:41.955 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.933+0000 7f6c48dbcb80 20 client.16746 populate_metadata read hostname 'smithi039' 2023-07-21T18:49:41.955 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.933+0000 7f6c48dbcb80 1 --2- 172.21.15.39:0/315222475 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55e25e8625e0 0x55e25e8649f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:41.956 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.933+0000 7f6c48dbcb80 4 client.16746 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:49:41.956 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.933+0000 7f6c48dbcb80 1 -- 172.21.15.39:0/315222475 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e25e590f10 con 0x55e25e8625e0 2023-07-21T18:49:41.956 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.934+0000 7f6c2ffff700 1 --2- 172.21.15.39:0/315222475 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55e25e8625e0 0x55e25e8649f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:41.956 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.934+0000 7f6c1dffb700 10 client.16746 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:49:41.956 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:41.934+0000 7f6c2ffff700 1 --2- 172.21.15.39:0/315222475 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55e25e8625e0 0x55e25e8649f0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1506f70da46fa99f server_cookie=35c2d2273bdbb05d in_seq=0 out_seq=0 2023-07-21T18:49:42.278 INFO:tasks.workunit.client.0.smithi039.stdout: 1 509809 6.57 MB/sec execute 295 sec latency 3838.977 ms 2023-07-21T18:49:42.429 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:49:42 smithi039 ceph-mon[96489]: pgmap v246: 97 pgs: 97 active+clean; 205 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 616 KiB/s rd, 3.5 MiB/s wr, 175 op/s 2023-07-21T18:49:42.611 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:49:42 smithi193 ceph-mon[98204]: pgmap v246: 97 pgs: 97 active+clean; 205 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 616 KiB/s rd, 3.5 MiB/s wr, 175 op/s 2023-07-21T18:49:42.630 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:49:42 smithi155 ceph-mon[97934]: pgmap v246: 97 pgs: 97 active+clean; 205 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 616 KiB/s rd, 3.5 MiB/s wr, 175 op/s 2023-07-21T18:49:43.278 INFO:tasks.workunit.client.0.smithi039.stdout: 1 509809 6.55 MB/sec execute 296 sec latency 4839.062 ms 2023-07-21T18:49:44.278 INFO:tasks.workunit.client.0.smithi039.stdout: 1 509809 6.53 MB/sec execute 297 sec latency 5839.145 ms 2023-07-21T18:49:44.428 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:49:44 smithi039 ceph-mon[96489]: pgmap v247: 97 pgs: 97 active+clean; 205 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 1.2 MiB/s wr, 51 op/s 2023-07-21T18:49:44.611 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:49:44 smithi193 ceph-mon[98204]: pgmap v247: 97 pgs: 97 active+clean; 205 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 1.2 MiB/s wr, 51 op/s 2023-07-21T18:49:44.630 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:49:44 smithi155 ceph-mon[97934]: pgmap v247: 97 pgs: 97 active+clean; 205 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 1.2 MiB/s wr, 51 op/s 2023-07-21T18:49:45.278 INFO:tasks.workunit.client.0.smithi039.stdout: 1 509809 6.51 MB/sec execute 298 sec latency 6839.230 ms 2023-07-21T18:49:45.611 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:49:45 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:49:45.630 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:49:45 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:49:45.679 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:49:45 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:49:46.089 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:46.074+0000 7f6c1dffb700 1 -- 172.21.15.39:0/315222475 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e25e590f10 con 0x55e25e8625e0 2023-07-21T18:49:46.089 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:46.074+0000 7f6c1dffb700 10 client.16746 handle_command_reply: tid=0 2023-07-21T18:49:46.089 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:46.074+0000 7f6c48dbcb80 1 client.16746 shutdown 2023-07-21T18:49:46.089 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:46.074+0000 7f6c48dbcb80 1 -- 172.21.15.39:0/315222475 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e25e64c6b0 msgr2=0x55e25e7fec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:49:46.090 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:46.074+0000 7f6c48dbcb80 1 --2- 172.21.15.39:0/315222475 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e25e64c6b0 0x55e25e7fec50 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f6c2800c010 tx=0x7f6c2800b910 comp rx=0 tx=0).stop 2023-07-21T18:49:46.090 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:46.075+0000 7f6c48dbcb80 1 -- 172.21.15.39:0/315222475 shutdown_connections 2023-07-21T18:49:46.090 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:46.075+0000 7f6c48dbcb80 1 --2- 172.21.15.39:0/315222475 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55e25e8625e0 0x55e25e8649f0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:46.090 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:46.075+0000 7f6c48dbcb80 1 --2- 172.21.15.39:0/315222475 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e25e7f62d0 0x55e25e860be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:46.090 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:46.075+0000 7f6c48dbcb80 1 --2- 172.21.15.39:0/315222475 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e25e64c6b0 0x55e25e7fec50 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:46.090 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:46.075+0000 7f6c48dbcb80 1 --2- 172.21.15.39:0/315222475 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e25e7f0580 0x55e25e7ff190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:46.090 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:46.075+0000 7f6c1dffb700 0 client.16746 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:49:46.091 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:46.075+0000 7f6c48dbcb80 1 -- 172.21.15.39:0/315222475 >> 172.21.15.39:0/315222475 conn(0x55e25e7e5be0 msgr2=0x55e25e7fb0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:49:46.091 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:46.075+0000 7f6c48dbcb80 1 -- 172.21.15.39:0/315222475 shutdown_connections 2023-07-21T18:49:46.091 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:46.075+0000 7f6c48dbcb80 1 -- 172.21.15.39:0/315222475 wait complete. 2023-07-21T18:49:46.091 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:46.075+0000 7f6c48dbcb80 20 client.16746 trim_cache size 0 max 16384 2023-07-21T18:49:46.091 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:49:46.091 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:49:46.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:49:46.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:49:46.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:46.075+0000 7f6c3cc36700 1 -- 172.21.15.39:0/3815317075 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f6c240a12c0 msgr2=0x7f6c240a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:49:46.093 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:46.075+0000 7f6c3cc36700 1 --2- 172.21.15.39:0/3815317075 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f6c240a12c0 0x7f6c240a3770 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6c2800b5c0 tx=0x7f6c28005fd0 comp rx=0 tx=0).stop 2023-07-21T18:49:46.093 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:46.075+0000 7f6c3cc36700 1 -- 172.21.15.39:0/3815317075 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c38105c10 msgr2=0x7f6c3819a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:49:46.093 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:46.075+0000 7f6c3cc36700 1 --2- 172.21.15.39:0/3815317075 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c38105c10 0x7f6c3819a950 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f6c3000b970 tx=0x7f6c3000bc80 comp rx=0 tx=0).stop 2023-07-21T18:49:46.093 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:46.076+0000 7f6c3cc36700 1 -- 172.21.15.39:0/3815317075 shutdown_connections 2023-07-21T18:49:46.093 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:46.076+0000 7f6c3cc36700 1 --2- 172.21.15.39:0/3815317075 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c38105c10 0x7f6c3819a950 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:46.093 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:46.076+0000 7f6c3cc36700 1 --2- 172.21.15.39:0/3815317075 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c380fe070 0x7f6c38195da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:46.093 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:46.076+0000 7f6c3cc36700 1 --2- 172.21.15.39:0/3815317075 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f6c240a12c0 0x7f6c240a3770 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:46.093 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:46.076+0000 7f6c3cc36700 1 --2- 172.21.15.39:0/3815317075 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c380fea20 0x7f6c381962e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:46.094 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:46.076+0000 7f6c3cc36700 1 -- 172.21.15.39:0/3815317075 >> 172.21.15.39:0/3815317075 conn(0x7f6c38075f90 msgr2=0x7f6c380fbaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:49:46.094 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:46.076+0000 7f6c3cc36700 1 -- 172.21.15.39:0/3815317075 shutdown_connections 2023-07-21T18:49:46.094 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:46.076+0000 7f6c3cc36700 1 -- 172.21.15.39:0/3815317075 wait complete. 2023-07-21T18:49:46.278 INFO:tasks.workunit.client.0.smithi039.stdout: 1 510370 6.49 MB/sec execute 299 sec latency 7643.471 ms 2023-07-21T18:49:46.611 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:49:46 smithi193 ceph-mon[98204]: pgmap v248: 97 pgs: 97 active+clean; 205 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 1.2 MiB/s wr, 51 op/s 2023-07-21T18:49:46.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:49:46 smithi155 ceph-mon[97934]: pgmap v248: 97 pgs: 97 active+clean; 205 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 1.2 MiB/s wr, 51 op/s 2023-07-21T18:49:46.678 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:49:46 smithi039 ceph-mon[96489]: pgmap v248: 97 pgs: 97 active+clean; 205 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 1.2 MiB/s wr, 51 op/s 2023-07-21T18:49:47.278 INFO:tasks.workunit.client.0.smithi039.stdout: 1 511900 6.51 MB/sec execute 300 sec latency 47.356 ms 2023-07-21T18:49:48.278 INFO:tasks.workunit.client.0.smithi039.stdout: 1 512193 6.49 MB/sec execute 301 sec latency 633.904 ms 2023-07-21T18:49:48.611 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:49:48 smithi193 ceph-mon[98204]: pgmap v249: 97 pgs: 97 active+clean; 204 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 1.9 MiB/s wr, 88 op/s 2023-07-21T18:49:48.630 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:49:48 smithi155 ceph-mon[97934]: pgmap v249: 97 pgs: 97 active+clean; 204 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 1.9 MiB/s wr, 88 op/s 2023-07-21T18:49:48.678 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:49:48 smithi039 ceph-mon[96489]: pgmap v249: 97 pgs: 97 active+clean; 204 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 1.9 MiB/s wr, 88 op/s 2023-07-21T18:49:49.279 INFO:tasks.workunit.client.0.smithi039.stdout: 1 513272 6.48 MB/sec execute 302 sec latency 727.184 ms 2023-07-21T18:49:50.279 INFO:tasks.workunit.client.0.smithi039.stdout: 1 513272 6.46 MB/sec execute 303 sec latency 1249.696 ms 2023-07-21T18:49:50.611 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:49:50 smithi193 ceph-mon[98204]: pgmap v250: 97 pgs: 97 active+clean; 204 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 1.3 MiB/s wr, 69 op/s 2023-07-21T18:49:50.630 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:49:50 smithi155 ceph-mon[97934]: pgmap v250: 97 pgs: 97 active+clean; 204 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 1.3 MiB/s wr, 69 op/s 2023-07-21T18:49:50.678 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:49:50 smithi039 ceph-mon[96489]: pgmap v250: 97 pgs: 97 active+clean; 204 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 1.3 MiB/s wr, 69 op/s 2023-07-21T18:49:51.279 INFO:tasks.workunit.client.0.smithi039.stdout: 1 513272 6.44 MB/sec execute 304 sec latency 2249.784 ms 2023-07-21T18:49:52.279 INFO:tasks.workunit.client.0.smithi039.stdout: 1 514455 6.44 MB/sec execute 305 sec latency 2687.583 ms 2023-07-21T18:49:52.611 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:49:52 smithi193 ceph-mon[98204]: pgmap v251: 97 pgs: 97 active+clean; 207 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 2.2 MiB/s wr, 139 op/s 2023-07-21T18:49:52.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:49:52 smithi155 ceph-mon[97934]: pgmap v251: 97 pgs: 97 active+clean; 207 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 2.2 MiB/s wr, 139 op/s 2023-07-21T18:49:52.678 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:49:52 smithi039 ceph-mon[96489]: pgmap v251: 97 pgs: 97 active+clean; 207 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 2.2 MiB/s wr, 139 op/s 2023-07-21T18:49:53.279 INFO:tasks.workunit.client.0.smithi039.stdout: 1 515694 6.44 MB/sec execute 306 sec latency 20.123 ms 2023-07-21T18:49:54.279 INFO:tasks.workunit.client.0.smithi039.stdout: 1 516816 6.44 MB/sec execute 307 sec latency 324.679 ms 2023-07-21T18:49:54.611 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:49:54 smithi193 ceph-mon[98204]: pgmap v252: 97 pgs: 97 active+clean; 207 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 1.7 MiB/s wr, 106 op/s 2023-07-21T18:49:54.630 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:49:54 smithi155 ceph-mon[97934]: pgmap v252: 97 pgs: 97 active+clean; 207 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 1.7 MiB/s wr, 106 op/s 2023-07-21T18:49:54.679 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:49:54 smithi039 ceph-mon[96489]: pgmap v252: 97 pgs: 97 active+clean; 207 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 1.7 MiB/s wr, 106 op/s 2023-07-21T18:49:55.279 INFO:tasks.workunit.client.0.smithi039.stdout: 1 516816 6.42 MB/sec execute 308 sec latency 1324.742 ms 2023-07-21T18:49:56.279 INFO:tasks.workunit.client.0.smithi039.stdout: 1 516816 6.40 MB/sec execute 309 sec latency 2324.837 ms 2023-07-21T18:49:56.611 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:49:56 smithi193 ceph-mon[98204]: pgmap v253: 97 pgs: 97 active+clean; 207 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 1.7 MiB/s wr, 106 op/s 2023-07-21T18:49:56.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:49:56 smithi155 ceph-mon[97934]: pgmap v253: 97 pgs: 97 active+clean; 207 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 1.7 MiB/s wr, 106 op/s 2023-07-21T18:49:56.678 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:49:56 smithi039 ceph-mon[96489]: pgmap v253: 97 pgs: 97 active+clean; 207 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 1.7 MiB/s wr, 106 op/s 2023-07-21T18:49:57.279 INFO:tasks.workunit.client.0.smithi039.stdout: 1 517885 6.40 MB/sec execute 310 sec latency 2768.446 ms 2023-07-21T18:49:57.611 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:49:57 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:49:57.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:49:57 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:49:57.678 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:49:57 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:49:58.279 INFO:tasks.workunit.client.0.smithi039.stdout: 1 519126 6.40 MB/sec execute 311 sec latency 48.524 ms 2023-07-21T18:49:58.611 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:49:58 smithi193 ceph-mon[98204]: pgmap v254: 97 pgs: 97 active+clean; 209 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 2.7 MiB/s wr, 177 op/s 2023-07-21T18:49:58.611 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:49:58 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:49:58.611 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:49:58 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:49:58.612 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:49:58 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:49:58.630 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:49:58 smithi155 ceph-mon[97934]: pgmap v254: 97 pgs: 97 active+clean; 209 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 2.7 MiB/s wr, 177 op/s 2023-07-21T18:49:58.630 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:49:58 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:49:58.630 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:49:58 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:49:58.630 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:49:58 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:49:58.678 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:49:58 smithi039 ceph-mon[96489]: pgmap v254: 97 pgs: 97 active+clean; 209 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 2.7 MiB/s wr, 177 op/s 2023-07-21T18:49:58.679 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:49:58 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:49:58.679 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:49:58 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:49:58.679 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:49:58 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:49:59.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.191+0000 7fc9531ad700 1 -- 172.21.15.39:0/3772825125 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc94c0fea30 msgr2=0x7fc94c0feea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:49:59.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.191+0000 7fc9531ad700 1 --2- 172.21.15.39:0/3772825125 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc94c0fea30 0x7fc94c0feea0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fc938009990 tx=0x7fc938009ca0 comp rx=0 tx=0).stop 2023-07-21T18:49:59.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.191+0000 7fc9531ad700 1 -- 172.21.15.39:0/3772825125 shutdown_connections 2023-07-21T18:49:59.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.191+0000 7fc9531ad700 1 --2- 172.21.15.39:0/3772825125 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc94c0ff3e0 0x7fc94c10b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:59.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.191+0000 7fc9531ad700 1 --2- 172.21.15.39:0/3772825125 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc94c104a30 0x7fc94c104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:59.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.191+0000 7fc9531ad700 1 --2- 172.21.15.39:0/3772825125 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc94c0fea30 0x7fc94c0feea0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:59.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.191+0000 7fc9531ad700 1 -- 172.21.15.39:0/3772825125 >> 172.21.15.39:0/3772825125 conn(0x7fc94c074760 msgr2=0x7fc94c074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:49:59.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.191+0000 7fc9531ad700 1 -- 172.21.15.39:0/3772825125 shutdown_connections 2023-07-21T18:49:59.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.191+0000 7fc9531ad700 1 -- 172.21.15.39:0/3772825125 wait complete. 2023-07-21T18:49:59.207 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.192+0000 7fc9531ad700 1 Processor -- start 2023-07-21T18:49:59.207 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.192+0000 7fc9531ad700 1 -- start start 2023-07-21T18:49:59.207 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.192+0000 7fc9531ad700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc94c0fea30 0x7fc94c196ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:59.207 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.192+0000 7fc9531ad700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc94c0ff3e0 0x7fc94c1970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.192+0000 7fc9531ad700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc94c104a30 0x7fc94c1a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.192+0000 7fc9531ad700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc94c1a2010 con 0x7fc94c104a30 2023-07-21T18:49:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.192+0000 7fc9531ad700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc94c1a2180 con 0x7fc94c0fea30 2023-07-21T18:49:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.192+0000 7fc9531ad700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc94c1a2340 con 0x7fc94c0ff3e0 2023-07-21T18:49:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.192+0000 7fc95174a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc94c104a30 0x7fc94c1a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.192+0000 7fc94bfff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc94c0ff3e0 0x7fc94c1970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.192+0000 7fc950f49700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc94c0fea30 0x7fc94c196ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.192+0000 7fc95174a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc94c104a30 0x7fc94c1a1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60328/0 (socket says 172.21.15.39:60328) 2023-07-21T18:49:59.209 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.192+0000 7fc95174a700 1 -- 172.21.15.39:0/1920549482 learned_addr learned my addr 172.21.15.39:0/1920549482 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:49:59.209 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.192+0000 7fc95174a700 1 -- 172.21.15.39:0/1920549482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc94c0ff3e0 msgr2=0x7fc94c1970e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:49:59.209 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.192+0000 7fc95174a700 1 --2- 172.21.15.39:0/1920549482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc94c0ff3e0 0x7fc94c1970e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:59.209 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.192+0000 7fc95174a700 1 -- 172.21.15.39:0/1920549482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc94c0fea30 msgr2=0x7fc94c196ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:49:59.209 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.192+0000 7fc95174a700 1 --2- 172.21.15.39:0/1920549482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc94c0fea30 0x7fc94c196ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:59.209 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.192+0000 7fc95174a700 1 -- 172.21.15.39:0/1920549482 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9380095f0 con 0x7fc94c104a30 2023-07-21T18:49:59.209 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.192+0000 7fc95174a700 1 --2- 172.21.15.39:0/1920549482 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc94c104a30 0x7fc94c1a1a40 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fc94400ef30 tx=0x7fc94400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:49:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.193+0000 7fc949ffb700 1 -- 172.21.15.39:0/1920549482 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc944018070 con 0x7fc94c104a30 2023-07-21T18:49:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.193+0000 7fc949ffb700 1 -- 172.21.15.39:0/1920549482 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc94400f040 con 0x7fc94c104a30 2023-07-21T18:49:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.193+0000 7fc9531ad700 1 -- 172.21.15.39:0/1920549482 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc94c1a25c0 con 0x7fc94c104a30 2023-07-21T18:49:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.193+0000 7fc949ffb700 1 -- 172.21.15.39:0/1920549482 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94400cd00 con 0x7fc94c104a30 2023-07-21T18:49:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.193+0000 7fc9531ad700 1 -- 172.21.15.39:0/1920549482 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc94c1a2a80 con 0x7fc94c104a30 2023-07-21T18:49:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.195+0000 7fc949ffb700 1 -- 172.21.15.39:0/1920549482 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fc944003a00 con 0x7fc94c104a30 2023-07-21T18:49:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.195+0000 7fc949ffb700 1 --2- 172.21.15.39:0/1920549482 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fc93c0a12c0 0x7fc93c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.196+0000 7fc949ffb700 1 -- 172.21.15.39:0/1920549482 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fc9440cc870 con 0x7fc94c104a30 2023-07-21T18:49:59.214 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.196+0000 7fc950f49700 1 --2- 172.21.15.39:0/1920549482 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fc93c0a12c0 0x7fc93c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:59.214 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.196+0000 7fc950f49700 1 --2- 172.21.15.39:0/1920549482 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fc93c0a12c0 0x7fc93c0a3770 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc94c0ffba0 tx=0x7fc94000b410 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:49:59.214 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.196+0000 7fc95f333b80 1 Processor -- start 2023-07-21T18:49:59.214 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.196+0000 7fc95f333b80 1 -- start start 2023-07-21T18:49:59.214 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.197+0000 7fc95f333b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560c21eee6b0 0x560c22097d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:59.214 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.197+0000 7fc95f333b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560c220982d0 0x560c22092040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:59.214 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.197+0000 7fc95f333b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c22092580 0x560c220929d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:59.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.197+0000 7fc95f333b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c21f97c50 con 0x560c21eee6b0 2023-07-21T18:49:59.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.197+0000 7fc95f333b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c21fea9e0 con 0x560c220982d0 2023-07-21T18:49:59.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.197+0000 7fc95f333b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c21fe3e70 con 0x560c22092580 2023-07-21T18:49:59.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.197+0000 7fc950f49700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560c21eee6b0 0x560c22097d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:59.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.197+0000 7fc950f49700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560c21eee6b0 0x560c22097d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60332/0 (socket says 172.21.15.39:60332) 2023-07-21T18:49:59.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.197+0000 7fc950f49700 1 -- 172.21.15.39:0/3776643642 learned_addr learned my addr 172.21.15.39:0/3776643642 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:49:59.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.197+0000 7fc94bfff700 1 --2- 172.21.15.39:0/3776643642 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560c220982d0 0x560c22092040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:59.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.197+0000 7fc95174a700 1 --2- 172.21.15.39:0/3776643642 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c22092580 0x560c220929d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:59.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.197+0000 7fc950f49700 1 -- 172.21.15.39:0/3776643642 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c22092580 msgr2=0x560c220929d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:49:59.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.197+0000 7fc950f49700 1 --2- 172.21.15.39:0/3776643642 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c22092580 0x560c220929d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:59.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.197+0000 7fc950f49700 1 -- 172.21.15.39:0/3776643642 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560c220982d0 msgr2=0x560c22092040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:49:59.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.197+0000 7fc950f49700 1 --2- 172.21.15.39:0/3776643642 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560c220982d0 0x560c22092040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:59.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.197+0000 7fc950f49700 1 -- 172.21.15.39:0/3776643642 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c21e11d80 con 0x560c21eee6b0 2023-07-21T18:49:59.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.197+0000 7fc950f49700 1 --2- 172.21.15.39:0/3776643642 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560c21eee6b0 0x560c22097d90 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fc940010f70 tx=0x7fc940013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e196b6a4550a70f0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:49:59.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.197+0000 7fc936ffd700 1 -- 172.21.15.39:0/3776643642 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc940011db0 con 0x560c21eee6b0 2023-07-21T18:49:59.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.198+0000 7fc936ffd700 1 -- 172.21.15.39:0/3776643642 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc940015070 con 0x560c21eee6b0 2023-07-21T18:49:59.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.198+0000 7fc936ffd700 1 -- 172.21.15.39:0/3776643642 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94001c610 con 0x560c21eee6b0 2023-07-21T18:49:59.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.198+0000 7fc95f333b80 1 -- 172.21.15.39:0/3776643642 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560c21eee6b0 msgr2=0x560c22097d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:49:59.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.198+0000 7fc95f333b80 1 --2- 172.21.15.39:0/3776643642 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560c21eee6b0 0x560c22097d90 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fc940010f70 tx=0x7fc940013000 comp rx=0 tx=0).stop 2023-07-21T18:49:59.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.198+0000 7fc95f333b80 1 -- 172.21.15.39:0/3776643642 shutdown_connections 2023-07-21T18:49:59.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.198+0000 7fc95f333b80 1 --2- 172.21.15.39:0/3776643642 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c22092580 0x560c220929d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:59.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.198+0000 7fc95f333b80 1 --2- 172.21.15.39:0/3776643642 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560c21eee6b0 0x560c22097d90 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:59.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.198+0000 7fc95f333b80 1 --2- 172.21.15.39:0/3776643642 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560c220982d0 0x560c22092040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:59.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.198+0000 7fc95f333b80 1 -- 172.21.15.39:0/3776643642 >> 172.21.15.39:0/3776643642 conn(0x560c22087be0 msgr2=0x560c2208cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:49:59.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.198+0000 7fc95f333b80 1 -- 172.21.15.39:0/3776643642 shutdown_connections 2023-07-21T18:49:59.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.198+0000 7fc95f333b80 1 -- 172.21.15.39:0/3776643642 wait complete. 2023-07-21T18:49:59.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.199+0000 7fc95f333b80 1 -- start start 2023-07-21T18:49:59.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.199+0000 7fc95f333b80 1 Processor -- start 2023-07-21T18:49:59.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.199+0000 7fc95f333b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560c21eee6b0 0x560c2209b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:59.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.199+0000 7fc95f333b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560c22092580 0x560c2209be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:59.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.199+0000 7fc95f333b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c220982d0 0x560c2209c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:59.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.199+0000 7fc95f333b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c21f83a60 con 0x560c22092580 2023-07-21T18:49:59.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.199+0000 7fc95f333b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c21f83760 con 0x560c21eee6b0 2023-07-21T18:49:59.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.199+0000 7fc95f333b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c21e678c0 con 0x560c220982d0 2023-07-21T18:49:59.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.199+0000 7fc95f333b80 10 client.0 _ll_register_callbacks cb 0x560c21f81700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:49:59.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.199+0000 7fc950f49700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560c21eee6b0 0x560c2209b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:59.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.199+0000 7fc94bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560c22092580 0x560c2209be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:59.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.199+0000 7fc950f49700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560c21eee6b0 0x560c2209b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:44702/0 (socket says 172.21.15.39:44702) 2023-07-21T18:49:59.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.199+0000 7fc950f49700 1 -- 172.21.15.39:0/3103901079 learned_addr learned my addr 172.21.15.39:0/3103901079 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:49:59.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.199+0000 7fc95174a700 1 --2- 172.21.15.39:0/3103901079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c220982d0 0x560c2209c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:59.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.200+0000 7fc950f49700 1 -- 172.21.15.39:0/3103901079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c220982d0 msgr2=0x560c2209c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:49:59.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.200+0000 7fc950f49700 1 --2- 172.21.15.39:0/3103901079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c220982d0 0x560c2209c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:59.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.200+0000 7fc950f49700 1 -- 172.21.15.39:0/3103901079 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560c22092580 msgr2=0x560c2209be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:49:59.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.200+0000 7fc950f49700 1 --2- 172.21.15.39:0/3103901079 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560c22092580 0x560c2209be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:59.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.200+0000 7fc950f49700 1 -- 172.21.15.39:0/3103901079 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc940010c50 con 0x560c21eee6b0 2023-07-21T18:49:59.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.200+0000 7fc94bfff700 1 --2- 172.21.15.39:0/3103901079 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560c22092580 0x560c2209be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:49:59.221 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.200+0000 7fc950f49700 1 --2- 172.21.15.39:0/3103901079 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560c21eee6b0 0x560c2209b910 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc940010a40 tx=0x7fc940011dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb99da687541769 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:49:59.221 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.200+0000 7fc92bfff700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-21T18:49:59.221 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.200+0000 7fc92bfff700 1 -- 172.21.15.39:0/3103901079 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94002c070 con 0x560c21eee6b0 2023-07-21T18:49:59.221 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.200+0000 7fc92bfff700 1 -- 172.21.15.39:0/3103901079 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fc940017070 con 0x560c21eee6b0 2023-07-21T18:49:59.221 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.200+0000 7fc95f333b80 1 -- 172.21.15.39:0/3103901079 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560c21f3d490 con 0x560c21eee6b0 2023-07-21T18:49:59.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.203+0000 7fc92bfff700 1 -- 172.21.15.39:0/3103901079 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc94001e630 con 0x560c21eee6b0 2023-07-21T18:49:59.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.203+0000 7fc95f333b80 10 client.26237 fetch_fsmap learned FSMap version 58 2023-07-21T18:49:59.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.203+0000 7fc95f333b80 1 -- 172.21.15.39:0/3103901079 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x560c21dfac00 con 0x560c21eee6b0 2023-07-21T18:49:59.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.203+0000 7fc92bfff700 1 -- 172.21.15.39:0/3103901079 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc94003e020 con 0x560c21eee6b0 2023-07-21T18:49:59.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.203+0000 7fc92bfff700 1 -- 172.21.15.39:0/3103901079 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 58) v1 ==== 7498+0+0 (secure 0 0 0) 0x7fc940026bb0 con 0x560c21eee6b0 2023-07-21T18:49:59.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.203+0000 7fc95f333b80 10 client.26237 fetch_fsmap finished waiting for FSMap version 58 2023-07-21T18:49:59.224 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.203+0000 7fc95f333b80 10 client.26237 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:49:59.224 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.203+0000 7fc95f333b80 20 client.26237 populate_metadata read hostname 'smithi039' 2023-07-21T18:49:59.224 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.203+0000 7fc95f333b80 1 --2- 172.21.15.39:0/3103901079 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x560c220abaa0 0x560c220adeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:59.224 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.203+0000 7fc95f333b80 4 client.26237 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:49:59.224 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.203+0000 7fc95f333b80 1 -- 172.21.15.39:0/3103901079 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560c21fdce50 con 0x560c220abaa0 2023-07-21T18:49:59.224 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.204+0000 7fc94bfff700 1 --2- 172.21.15.39:0/3103901079 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x560c220abaa0 0x560c220adeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:59.224 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.204+0000 7fc92bfff700 10 client.26237 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:49:59.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.204+0000 7fc94bfff700 1 --2- 172.21.15.39:0/3103901079 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x560c220abaa0 0x560c220adeb0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a854c126076d84c server_cookie=698e8d5af58c21d3 in_seq=0 out_seq=0 2023-07-21T18:49:59.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.204+0000 7fc92bfff700 1 -- 172.21.15.39:0/3103901079 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560c21fdce50 con 0x560c220abaa0 2023-07-21T18:49:59.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.204+0000 7fc92bfff700 10 client.26237 handle_command_reply: tid=0 2023-07-21T18:49:59.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.205+0000 7fc95f333b80 1 client.26237 shutdown 2023-07-21T18:49:59.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.205+0000 7fc95f333b80 1 -- 172.21.15.39:0/3103901079 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560c21eee6b0 msgr2=0x560c2209b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:49:59.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.205+0000 7fc95f333b80 1 --2- 172.21.15.39:0/3103901079 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560c21eee6b0 0x560c2209b910 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc940010a40 tx=0x7fc940011dc0 comp rx=0 tx=0).stop 2023-07-21T18:49:59.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.205+0000 7fc95f333b80 1 -- 172.21.15.39:0/3103901079 shutdown_connections 2023-07-21T18:49:59.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.205+0000 7fc95f333b80 1 --2- 172.21.15.39:0/3103901079 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x560c220abaa0 0x560c220adeb0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:59.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.205+0000 7fc95f333b80 1 --2- 172.21.15.39:0/3103901079 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560c21eee6b0 0x560c2209b910 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:59.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.205+0000 7fc95f333b80 1 --2- 172.21.15.39:0/3103901079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c220982d0 0x560c2209c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:59.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.205+0000 7fc95f333b80 1 --2- 172.21.15.39:0/3103901079 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560c22092580 0x560c2209be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:59.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.205+0000 7fc92bfff700 0 client.26237 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:49:59.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.205+0000 7fc95f333b80 1 -- 172.21.15.39:0/3103901079 >> 172.21.15.39:0/3103901079 conn(0x560c22087be0 msgr2=0x560c2208bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:49:59.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.205+0000 7fc95f333b80 1 -- 172.21.15.39:0/3103901079 shutdown_connections 2023-07-21T18:49:59.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.205+0000 7fc95f333b80 1 -- 172.21.15.39:0/3103901079 wait complete. 2023-07-21T18:49:59.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.205+0000 7fc95f333b80 20 client.26237 trim_cache size 0 max 16384 2023-07-21T18:49:59.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.220+0000 7fc95f333b80 1 Processor -- start 2023-07-21T18:49:59.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.220+0000 7fc95f333b80 1 -- start start 2023-07-21T18:49:59.234 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.220+0000 7fc95f333b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560c21eee6b0 0x560c22092040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:59.234 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.220+0000 7fc95f333b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c22092580 0x560c220a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:59.235 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.220+0000 7fc95f333b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560c220982d0 0x560c220971e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:59.235 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.220+0000 7fc95f333b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c2208ebb0 con 0x560c220982d0 2023-07-21T18:49:59.235 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.220+0000 7fc95f333b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c22089620 con 0x560c21eee6b0 2023-07-21T18:49:59.235 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.220+0000 7fc95f333b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c220986d0 con 0x560c22092580 2023-07-21T18:49:59.235 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.220+0000 7fc95174a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560c220982d0 0x560c220971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:59.235 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.220+0000 7fc95174a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560c220982d0 0x560c220971e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60350/0 (socket says 172.21.15.39:60350) 2023-07-21T18:49:59.235 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.220+0000 7fc95174a700 1 -- 172.21.15.39:0/1272363266 learned_addr learned my addr 172.21.15.39:0/1272363266 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:49:59.236 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.220+0000 7fc950f49700 1 --2- 172.21.15.39:0/1272363266 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560c21eee6b0 0x560c22092040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:59.236 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.220+0000 7fc94bfff700 1 --2- 172.21.15.39:0/1272363266 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c22092580 0x560c220a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:59.236 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.220+0000 7fc95174a700 1 -- 172.21.15.39:0/1272363266 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c22092580 msgr2=0x560c220a2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:49:59.236 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.220+0000 7fc95174a700 1 --2- 172.21.15.39:0/1272363266 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c22092580 0x560c220a2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:59.236 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.220+0000 7fc95174a700 1 -- 172.21.15.39:0/1272363266 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560c21eee6b0 msgr2=0x560c22092040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:49:59.236 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.220+0000 7fc95174a700 1 --2- 172.21.15.39:0/1272363266 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560c21eee6b0 0x560c22092040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:59.236 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.220+0000 7fc95174a700 1 -- 172.21.15.39:0/1272363266 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c21e381f0 con 0x560c220982d0 2023-07-21T18:49:59.237 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.221+0000 7fc95174a700 1 --2- 172.21.15.39:0/1272363266 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560c220982d0 0x560c220971e0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fc944096440 tx=0x7fc94409f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c84f4cdc7b5930c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:49:59.237 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.221+0000 7fc9377fe700 1 -- 172.21.15.39:0/1272363266 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94400ce60 con 0x560c220982d0 2023-07-21T18:49:59.237 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.221+0000 7fc9377fe700 1 -- 172.21.15.39:0/1272363266 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc9440999d0 con 0x560c220982d0 2023-07-21T18:49:59.237 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.221+0000 7fc9377fe700 1 -- 172.21.15.39:0/1272363266 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc944099c30 con 0x560c220982d0 2023-07-21T18:49:59.237 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.221+0000 7fc95f333b80 1 -- 172.21.15.39:0/1272363266 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560c220982d0 msgr2=0x560c220971e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:49:59.237 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.221+0000 7fc95f333b80 1 --2- 172.21.15.39:0/1272363266 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560c220982d0 0x560c220971e0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fc944096440 tx=0x7fc94409f040 comp rx=0 tx=0).stop 2023-07-21T18:49:59.237 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.221+0000 7fc95f333b80 1 -- 172.21.15.39:0/1272363266 shutdown_connections 2023-07-21T18:49:59.237 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.221+0000 7fc95f333b80 1 --2- 172.21.15.39:0/1272363266 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560c21eee6b0 0x560c22092040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:59.238 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.221+0000 7fc95f333b80 1 --2- 172.21.15.39:0/1272363266 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560c220982d0 0x560c220971e0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:59.238 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.221+0000 7fc95f333b80 1 --2- 172.21.15.39:0/1272363266 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c22092580 0x560c220a2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:59.238 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.221+0000 7fc95f333b80 1 -- 172.21.15.39:0/1272363266 >> 172.21.15.39:0/1272363266 conn(0x560c22087be0 msgr2=0x560c22096ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:49:59.238 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.221+0000 7fc95f333b80 1 -- 172.21.15.39:0/1272363266 shutdown_connections 2023-07-21T18:49:59.238 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.221+0000 7fc95f333b80 1 -- 172.21.15.39:0/1272363266 wait complete. 2023-07-21T18:49:59.240 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.222+0000 7fc95f333b80 1 -- start start 2023-07-21T18:49:59.240 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.222+0000 7fc95f333b80 1 Processor -- start 2023-07-21T18:49:59.241 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.222+0000 7fc95f333b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560c21eee6b0 0x560c220f3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:59.241 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.222+0000 7fc95f333b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560c22092580 0x560c220f4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:59.241 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.222+0000 7fc95f333b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c220982d0 0x560c220fe970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:59.241 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.222+0000 7fc95f333b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c220feeb0 con 0x560c22092580 2023-07-21T18:49:59.241 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.222+0000 7fc95f333b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c220feff0 con 0x560c21eee6b0 2023-07-21T18:49:59.241 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.222+0000 7fc95f333b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c220ff160 con 0x560c220982d0 2023-07-21T18:49:59.241 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.222+0000 7fc95f333b80 10 client.0 _ll_register_callbacks cb 0x7fc9440097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:49:59.242 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.223+0000 7fc94bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560c22092580 0x560c220f4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:59.242 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.223+0000 7fc94bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560c22092580 0x560c220f4020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60354/0 (socket says 172.21.15.39:60354) 2023-07-21T18:49:59.242 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.223+0000 7fc94bfff700 1 -- 172.21.15.39:0/2241638188 learned_addr learned my addr 172.21.15.39:0/2241638188 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:49:59.242 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.223+0000 7fc950f49700 1 --2- 172.21.15.39:0/2241638188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560c21eee6b0 0x560c220f3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:59.242 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.223+0000 7fc95174a700 1 --2- 172.21.15.39:0/2241638188 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c220982d0 0x560c220fe970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:59.242 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.223+0000 7fc94bfff700 1 -- 172.21.15.39:0/2241638188 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c220982d0 msgr2=0x560c220fe970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:49:59.242 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.223+0000 7fc94bfff700 1 --2- 172.21.15.39:0/2241638188 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c220982d0 0x560c220fe970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:59.242 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.223+0000 7fc94bfff700 1 -- 172.21.15.39:0/2241638188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560c21eee6b0 msgr2=0x560c220f3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:49:59.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.223+0000 7fc94bfff700 1 --2- 172.21.15.39:0/2241638188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560c21eee6b0 0x560c220f3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:59.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.223+0000 7fc94bfff700 1 -- 172.21.15.39:0/2241638188 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc94c1a2a80 con 0x560c22092580 2023-07-21T18:49:59.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.223+0000 7fc94bfff700 1 --2- 172.21.15.39:0/2241638188 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560c22092580 0x560c220f4020 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fc9380098e0 tx=0x7fc93800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eb83d7ec8ba76ef server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:49:59.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.223+0000 7fc92bfff700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:49:59.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.223+0000 7fc92bfff700 1 -- 172.21.15.39:0/2241638188 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93801d040 con 0x560c22092580 2023-07-21T18:49:59.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.223+0000 7fc92bfff700 1 -- 172.21.15.39:0/2241638188 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fc938029e10 con 0x560c22092580 2023-07-21T18:49:59.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.223+0000 7fc95f333b80 1 -- 172.21.15.39:0/2241638188 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc940015070 con 0x560c22092580 2023-07-21T18:49:59.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.223+0000 7fc92bfff700 1 -- 172.21.15.39:0/2241638188 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc9380292c0 con 0x560c22092580 2023-07-21T18:49:59.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.223+0000 7fc95f333b80 10 client.16764 fetch_fsmap learned FSMap version 58 2023-07-21T18:49:59.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.223+0000 7fc95f333b80 1 -- 172.21.15.39:0/2241638188 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x560c21e10250 con 0x560c22092580 2023-07-21T18:49:59.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.223+0000 7fc92bfff700 1 -- 172.21.15.39:0/2241638188 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc940015070 con 0x560c22092580 2023-07-21T18:49:59.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.224+0000 7fc92bfff700 1 -- 172.21.15.39:0/2241638188 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 58) v1 ==== 7498+0+0 (secure 0 0 0) 0x7fc938004620 con 0x560c22092580 2023-07-21T18:49:59.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.224+0000 7fc95f333b80 10 client.16764 fetch_fsmap finished waiting for FSMap version 58 2023-07-21T18:49:59.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.224+0000 7fc95f333b80 10 client.16764 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:49:59.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.224+0000 7fc95f333b80 20 client.16764 populate_metadata read hostname 'smithi039' 2023-07-21T18:49:59.245 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.224+0000 7fc95f333b80 1 --2- 172.21.15.39:0/2241638188 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x560c221009d0 0x560c22102de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:49:59.245 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.224+0000 7fc95f333b80 4 client.16764 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:49:59.245 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.224+0000 7fc95f333b80 1 -- 172.21.15.39:0/2241638188 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560c21f9eff0 con 0x560c221009d0 2023-07-21T18:49:59.245 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.224+0000 7fc950f49700 1 --2- 172.21.15.39:0/2241638188 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x560c221009d0 0x560c22102de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:49:59.245 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.224+0000 7fc92bfff700 10 client.16764 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:49:59.245 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.224+0000 7fc950f49700 1 --2- 172.21.15.39:0/2241638188 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x560c221009d0 0x560c22102de0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76fbf10d512f58c server_cookie=5e93daf97fcc603b in_seq=0 out_seq=0 2023-07-21T18:49:59.271 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.258+0000 7fc92bfff700 1 -- 172.21.15.39:0/2241638188 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560c21f9eff0 con 0x560c221009d0 2023-07-21T18:49:59.271 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.258+0000 7fc92bfff700 10 client.16764 handle_command_reply: tid=0 2023-07-21T18:49:59.271 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.258+0000 7fc95f333b80 1 client.16764 shutdown 2023-07-21T18:49:59.271 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.258+0000 7fc95f333b80 1 -- 172.21.15.39:0/2241638188 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560c22092580 msgr2=0x560c220f4020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:49:59.272 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.258+0000 7fc95f333b80 1 --2- 172.21.15.39:0/2241638188 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560c22092580 0x560c220f4020 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fc9380098e0 tx=0x7fc93800bf10 comp rx=0 tx=0).stop 2023-07-21T18:49:59.272 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.259+0000 7fc95f333b80 1 -- 172.21.15.39:0/2241638188 shutdown_connections 2023-07-21T18:49:59.272 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.259+0000 7fc95f333b80 1 --2- 172.21.15.39:0/2241638188 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x560c221009d0 0x560c22102de0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:59.273 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.259+0000 7fc95f333b80 1 --2- 172.21.15.39:0/2241638188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560c21eee6b0 0x560c220f3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:59.273 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.259+0000 7fc95f333b80 1 --2- 172.21.15.39:0/2241638188 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c220982d0 0x560c220fe970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:59.273 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.259+0000 7fc92bfff700 0 client.16764 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:49:59.273 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.259+0000 7fc95f333b80 1 --2- 172.21.15.39:0/2241638188 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560c22092580 0x560c220f4020 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:59.273 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.259+0000 7fc95f333b80 1 -- 172.21.15.39:0/2241638188 >> 172.21.15.39:0/2241638188 conn(0x560c22087be0 msgr2=0x560c22097360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:49:59.273 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.259+0000 7fc95f333b80 1 -- 172.21.15.39:0/2241638188 shutdown_connections 2023-07-21T18:49:59.273 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.259+0000 7fc95f333b80 1 -- 172.21.15.39:0/2241638188 wait complete. 2023-07-21T18:49:59.274 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.259+0000 7fc95f333b80 20 client.16764 trim_cache size 0 max 16384 2023-07-21T18:49:59.274 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:49:59.274 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:49:59.274 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:49:59.274 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:49:59.275 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.260+0000 7fc9531ad700 1 -- 172.21.15.39:0/1920549482 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fc93c0a12c0 msgr2=0x7fc93c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:49:59.275 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.260+0000 7fc9531ad700 1 --2- 172.21.15.39:0/1920549482 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fc93c0a12c0 0x7fc93c0a3770 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc94c0ffba0 tx=0x7fc94000b410 comp rx=0 tx=0).stop 2023-07-21T18:49:59.275 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.260+0000 7fc9531ad700 1 -- 172.21.15.39:0/1920549482 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc94c104a30 msgr2=0x7fc94c1a1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:49:59.275 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.260+0000 7fc9531ad700 1 --2- 172.21.15.39:0/1920549482 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc94c104a30 0x7fc94c1a1a40 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fc94400ef30 tx=0x7fc94400c3a0 comp rx=0 tx=0).stop 2023-07-21T18:49:59.275 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.260+0000 7fc9531ad700 1 -- 172.21.15.39:0/1920549482 shutdown_connections 2023-07-21T18:49:59.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.260+0000 7fc9531ad700 1 --2- 172.21.15.39:0/1920549482 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fc93c0a12c0 0x7fc93c0a3770 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:59.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.260+0000 7fc9531ad700 1 --2- 172.21.15.39:0/1920549482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc94c0fea30 0x7fc94c196ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:59.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.260+0000 7fc9531ad700 1 --2- 172.21.15.39:0/1920549482 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc94c104a30 0x7fc94c1a1a40 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:59.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.260+0000 7fc9531ad700 1 --2- 172.21.15.39:0/1920549482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc94c0ff3e0 0x7fc94c1970e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:49:59.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.260+0000 7fc9531ad700 1 -- 172.21.15.39:0/1920549482 >> 172.21.15.39:0/1920549482 conn(0x7fc94c074760 msgr2=0x7fc94c0fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:49:59.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.260+0000 7fc9531ad700 1 -- 172.21.15.39:0/1920549482 shutdown_connections 2023-07-21T18:49:59.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:49:59.260+0000 7fc9531ad700 1 -- 172.21.15.39:0/1920549482 wait complete. 2023-07-21T18:49:59.279 INFO:tasks.workunit.client.0.smithi039.stdout: 1 520365 6.40 MB/sec execute 312 sec latency 223.233 ms 2023-07-21T18:50:00.279 INFO:tasks.workunit.client.0.smithi039.stdout: 1 520365 6.38 MB/sec execute 313 sec latency 1223.310 ms 2023-07-21T18:50:00.428 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:50:00 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mon-a[96465]: 2023-07-21T18:49:59.999+0000 7f094dd1e700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 3 MDSs report damaged metadata 2023-07-21T18:50:00.429 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:50:00 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mon-a[96465]: 2023-07-21T18:49:59.999+0000 7f094dd1e700 -1 log_channel(cluster) log [ERR] : [ERR] MDS_DAMAGE: 3 MDSs report damaged metadata 2023-07-21T18:50:00.429 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:50:00 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mon-a[96465]: 2023-07-21T18:49:59.999+0000 7f094dd1e700 -1 log_channel(cluster) log [ERR] : mds.a(mds.4): Metadata damage detected 2023-07-21T18:50:00.429 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:50:00 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mon-a[96465]: 2023-07-21T18:49:59.999+0000 7f094dd1e700 -1 log_channel(cluster) log [ERR] : mds.i(mds.2): Metadata damage detected 2023-07-21T18:50:00.429 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:50:00 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mon-a[96465]: 2023-07-21T18:49:59.999+0000 7f094dd1e700 -1 log_channel(cluster) log [ERR] : mds.h(mds.1): Metadata damage detected 2023-07-21T18:50:00.429 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:50:00 smithi039 ceph-mon[96489]: pgmap v255: 97 pgs: 97 active+clean; 209 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 2.0 MiB/s wr, 140 op/s 2023-07-21T18:50:00.429 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:50:00 smithi039 ceph-mon[96489]: Health detail: HEALTH_ERR 3 MDSs report damaged metadata 2023-07-21T18:50:00.430 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:50:00 smithi039 ceph-mon[96489]: [ERR] MDS_DAMAGE: 3 MDSs report damaged metadata 2023-07-21T18:50:00.430 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:50:00 smithi039 ceph-mon[96489]: mds.a(mds.4): Metadata damage detected 2023-07-21T18:50:00.430 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:50:00 smithi039 ceph-mon[96489]: mds.i(mds.2): Metadata damage detected 2023-07-21T18:50:00.430 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:50:00 smithi039 ceph-mon[96489]: mds.h(mds.1): Metadata damage detected 2023-07-21T18:50:00.611 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:50:00 smithi193 ceph-mon[98204]: pgmap v255: 97 pgs: 97 active+clean; 209 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 2.0 MiB/s wr, 140 op/s 2023-07-21T18:50:00.611 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:50:00 smithi193 ceph-mon[98204]: Health detail: HEALTH_ERR 3 MDSs report damaged metadata 2023-07-21T18:50:00.611 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:50:00 smithi193 ceph-mon[98204]: [ERR] MDS_DAMAGE: 3 MDSs report damaged metadata 2023-07-21T18:50:00.611 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:50:00 smithi193 ceph-mon[98204]: mds.a(mds.4): Metadata damage detected 2023-07-21T18:50:00.611 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:50:00 smithi193 ceph-mon[98204]: mds.i(mds.2): Metadata damage detected 2023-07-21T18:50:00.612 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:50:00 smithi193 ceph-mon[98204]: mds.h(mds.1): Metadata damage detected 2023-07-21T18:50:00.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:50:00 smithi155 ceph-mon[97934]: pgmap v255: 97 pgs: 97 active+clean; 209 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 2.0 MiB/s wr, 140 op/s 2023-07-21T18:50:00.630 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:50:00 smithi155 ceph-mon[97934]: Health detail: HEALTH_ERR 3 MDSs report damaged metadata 2023-07-21T18:50:00.630 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:50:00 smithi155 ceph-mon[97934]: [ERR] MDS_DAMAGE: 3 MDSs report damaged metadata 2023-07-21T18:50:00.630 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:50:00 smithi155 ceph-mon[97934]: mds.a(mds.4): Metadata damage detected 2023-07-21T18:50:00.630 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:50:00 smithi155 ceph-mon[97934]: mds.i(mds.2): Metadata damage detected 2023-07-21T18:50:00.630 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:50:00 smithi155 ceph-mon[97934]: mds.h(mds.1): Metadata damage detected 2023-07-21T18:50:01.279 INFO:tasks.workunit.client.0.smithi039.stdout: 1 520365 6.36 MB/sec execute 314 sec latency 2223.403 ms 2023-07-21T18:50:02.279 INFO:tasks.workunit.client.0.smithi039.stdout: 1 520963 6.34 MB/sec execute 315 sec latency 2657.729 ms 2023-07-21T18:50:02.611 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:50:02 smithi193 ceph-mon[98204]: pgmap v256: 97 pgs: 97 active+clean; 203 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 2.9 MiB/s wr, 222 op/s 2023-07-21T18:50:02.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:50:02 smithi155 ceph-mon[97934]: pgmap v256: 97 pgs: 97 active+clean; 203 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 2.9 MiB/s wr, 222 op/s 2023-07-21T18:50:02.678 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:50:02 smithi039 ceph-mon[96489]: pgmap v256: 97 pgs: 97 active+clean; 203 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 2.9 MiB/s wr, 222 op/s 2023-07-21T18:50:03.279 INFO:tasks.workunit.client.0.smithi039.stdout: 1 522494 6.36 MB/sec execute 316 sec latency 22.874 ms 2023-07-21T18:50:04.279 INFO:tasks.workunit.client.0.smithi039.stdout: 1 523805 6.36 MB/sec execute 317 sec latency 28.539 ms 2023-07-21T18:50:04.611 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:50:04 smithi193 ceph-mon[98204]: pgmap v257: 97 pgs: 97 active+clean; 203 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 1.9 MiB/s wr, 152 op/s 2023-07-21T18:50:04.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:50:04 smithi155 ceph-mon[97934]: pgmap v257: 97 pgs: 97 active+clean; 203 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 1.9 MiB/s wr, 152 op/s 2023-07-21T18:50:04.678 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:50:04 smithi039 ceph-mon[96489]: pgmap v257: 97 pgs: 97 active+clean; 203 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 1.9 MiB/s wr, 152 op/s 2023-07-21T18:50:05.280 INFO:tasks.workunit.client.0.smithi039.stdout: 1 523906 6.34 MB/sec execute 318 sec latency 767.504 ms 2023-07-21T18:50:05.611 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:50:05 smithi193 ceph-mon[98204]: osdmap e88: 12 total, 12 up, 12 in 2023-07-21T18:50:05.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:50:05 smithi155 ceph-mon[97934]: osdmap e88: 12 total, 12 up, 12 in 2023-07-21T18:50:05.678 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:50:05 smithi039 ceph-mon[96489]: osdmap e88: 12 total, 12 up, 12 in 2023-07-21T18:50:06.280 INFO:tasks.workunit.client.0.smithi039.stdout: 1 524506 6.32 MB/sec execute 319 sec latency 1564.257 ms 2023-07-21T18:50:06.611 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:50:06 smithi193 ceph-mon[98204]: pgmap v259: 97 pgs: 97 active+clean; 203 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 2.3 MiB/s wr, 183 op/s 2023-07-21T18:50:06.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:50:06 smithi155 ceph-mon[97934]: pgmap v259: 97 pgs: 97 active+clean; 203 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 2.3 MiB/s wr, 183 op/s 2023-07-21T18:50:06.678 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:50:06 smithi039 ceph-mon[96489]: pgmap v259: 97 pgs: 97 active+clean; 203 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 2.3 MiB/s wr, 183 op/s 2023-07-21T18:50:07.280 INFO:tasks.workunit.client.0.smithi039.stdout: 1 526120 6.34 MB/sec execute 320 sec latency 11.003 ms 2023-07-21T18:50:08.280 INFO:tasks.workunit.client.0.smithi039.stdout: 1 527283 6.34 MB/sec execute 321 sec latency 32.688 ms 2023-07-21T18:50:08.611 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:50:08 smithi193 ceph-mon[98204]: pgmap v260: 97 pgs: 97 active+clean; 206 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 2.0 MiB/s wr, 204 op/s 2023-07-21T18:50:08.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:50:08 smithi155 ceph-mon[97934]: pgmap v260: 97 pgs: 97 active+clean; 206 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 2.0 MiB/s wr, 204 op/s 2023-07-21T18:50:08.678 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:50:08 smithi039 ceph-mon[96489]: pgmap v260: 97 pgs: 97 active+clean; 206 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 2.0 MiB/s wr, 204 op/s 2023-07-21T18:50:09.280 INFO:tasks.workunit.client.0.smithi039.stdout: 1 527456 6.32 MB/sec execute 322 sec latency 810.234 ms 2023-07-21T18:50:10.280 INFO:tasks.workunit.client.0.smithi039.stdout: 1 527456 6.30 MB/sec execute 323 sec latency 1810.337 ms 2023-07-21T18:50:10.611 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:50:10 smithi193 ceph-mon[98204]: pgmap v261: 97 pgs: 97 active+clean; 206 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 2.0 MiB/s wr, 204 op/s 2023-07-21T18:50:10.630 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:50:10 smithi155 ceph-mon[97934]: pgmap v261: 97 pgs: 97 active+clean; 206 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 2.0 MiB/s wr, 204 op/s 2023-07-21T18:50:10.678 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:50:10 smithi039 ceph-mon[96489]: pgmap v261: 97 pgs: 97 active+clean; 206 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 2.0 MiB/s wr, 204 op/s 2023-07-21T18:50:11.280 INFO:tasks.workunit.client.0.smithi039.stdout: 1 527456 6.28 MB/sec execute 324 sec latency 2810.422 ms 2023-07-21T18:50:12.280 INFO:tasks.workunit.client.0.smithi039.stdout: 1 528322 6.27 MB/sec execute 325 sec latency 3243.242 ms 2023-07-21T18:50:12.379 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:50:12 smithi155 ceph-mon[97934]: pgmap v262: 97 pgs: 97 active+clean; 210 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 1.3 MiB/s wr, 162 op/s 2023-07-21T18:50:12.380 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:50:12 smithi155 ceph-mon[97934]: osdmap e89: 12 total, 12 up, 12 in 2023-07-21T18:50:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.376+0000 7fd4abbf8700 1 -- 172.21.15.39:0/4288636654 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4a4067c10 msgr2=0x7fd4a4067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:50:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.376+0000 7fd4abbf8700 1 --2- 172.21.15.39:0/4288636654 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4a4067c10 0x7fd4a4067fe0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fd4a00099c0 tx=0x7fd4a0009cd0 comp rx=0 tx=0).stop 2023-07-21T18:50:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.376+0000 7fd4abbf8700 1 -- 172.21.15.39:0/4288636654 shutdown_connections 2023-07-21T18:50:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.376+0000 7fd4abbf8700 1 --2- 172.21.15.39:0/4288636654 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd4a4104e50 0x7fd4a410ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.376+0000 7fd4abbf8700 1 --2- 172.21.15.39:0/4288636654 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4a4067c10 0x7fd4a4067fe0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.376+0000 7fd4abbf8700 1 --2- 172.21.15.39:0/4288636654 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4a40685b0 0x7fd4a4104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.376+0000 7fd4abbf8700 1 -- 172.21.15.39:0/4288636654 >> 172.21.15.39:0/4288636654 conn(0x7fd4a4074760 msgr2=0x7fd4a4074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:50:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.376+0000 7fd4abbf8700 1 -- 172.21.15.39:0/4288636654 shutdown_connections 2023-07-21T18:50:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.376+0000 7fd4abbf8700 1 -- 172.21.15.39:0/4288636654 wait complete. 2023-07-21T18:50:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.377+0000 7fd4abbf8700 1 Processor -- start 2023-07-21T18:50:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.377+0000 7fd4abbf8700 1 -- start start 2023-07-21T18:50:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.377+0000 7fd4abbf8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4a4067c10 0x7fd4a4195cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.377+0000 7fd4abbf8700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4a40685b0 0x7fd4a4196210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.377+0000 7fd4abbf8700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd4a4104e50 0x7fd4a41997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.377+0000 7fd4abbf8700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a4199dd0 con 0x7fd4a4067c10 2023-07-21T18:50:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.377+0000 7fd4abbf8700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a4199f40 con 0x7fd4a40685b0 2023-07-21T18:50:12.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.377+0000 7fd4abbf8700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a419a100 con 0x7fd4a4104e50 2023-07-21T18:50:12.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.377+0000 7fd4a9994700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4a4067c10 0x7fd4a4195cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:12.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.377+0000 7fd4a9994700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4a4067c10 0x7fd4a4195cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60366/0 (socket says 172.21.15.39:60366) 2023-07-21T18:50:12.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.377+0000 7fd4a9994700 1 -- 172.21.15.39:0/979860859 learned_addr learned my addr 172.21.15.39:0/979860859 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:50:12.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.377+0000 7fd4aa195700 1 --2- 172.21.15.39:0/979860859 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd4a4104e50 0x7fd4a41997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:12.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.377+0000 7fd4a9193700 1 --2- 172.21.15.39:0/979860859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4a40685b0 0x7fd4a4196210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:12.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.377+0000 7fd4a9994700 1 -- 172.21.15.39:0/979860859 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd4a4104e50 msgr2=0x7fd4a41997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:50:12.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.377+0000 7fd4a9994700 1 --2- 172.21.15.39:0/979860859 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd4a4104e50 0x7fd4a41997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:12.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.377+0000 7fd4a9994700 1 -- 172.21.15.39:0/979860859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4a40685b0 msgr2=0x7fd4a4196210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:50:12.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.377+0000 7fd4a9994700 1 --2- 172.21.15.39:0/979860859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4a40685b0 0x7fd4a4196210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:12.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.377+0000 7fd4a9994700 1 -- 172.21.15.39:0/979860859 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4a00095f0 con 0x7fd4a4067c10 2023-07-21T18:50:12.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.378+0000 7fd4a9994700 1 --2- 172.21.15.39:0/979860859 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4a4067c10 0x7fd4a4195cd0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fd4a00098e0 tx=0x7fd4a000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:50:12.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.378+0000 7fd49affd700 1 -- 172.21.15.39:0/979860859 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a001d070 con 0x7fd4a4067c10 2023-07-21T18:50:12.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.378+0000 7fd4abbf8700 1 -- 172.21.15.39:0/979860859 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4a419a380 con 0x7fd4a4067c10 2023-07-21T18:50:12.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.378+0000 7fd49affd700 1 -- 172.21.15.39:0/979860859 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd4a0022470 con 0x7fd4a4067c10 2023-07-21T18:50:12.395 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.378+0000 7fd49affd700 1 -- 172.21.15.39:0/979860859 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a0017650 con 0x7fd4a4067c10 2023-07-21T18:50:12.395 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.378+0000 7fd4abbf8700 1 -- 172.21.15.39:0/979860859 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4a419a650 con 0x7fd4a4067c10 2023-07-21T18:50:12.396 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.381+0000 7fd49affd700 1 -- 172.21.15.39:0/979860859 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fd4a0022850 con 0x7fd4a4067c10 2023-07-21T18:50:12.397 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.381+0000 7fd49affd700 1 --2- 172.21.15.39:0/979860859 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fd4900a12c0 0x7fd4900a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:12.397 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.381+0000 7fd49affd700 1 -- 172.21.15.39:0/979860859 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fd4a00cd900 con 0x7fd4a4067c10 2023-07-21T18:50:12.397 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.382+0000 7fd4a9193700 1 --2- 172.21.15.39:0/979860859 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fd4900a12c0 0x7fd4900a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:12.397 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.382+0000 7fd4b7d7eb80 1 Processor -- start 2023-07-21T18:50:12.397 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.382+0000 7fd4a9193700 1 --2- 172.21.15.39:0/979860859 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fd4900a12c0 0x7fd4900a3770 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd4a41974b0 tx=0x7fd494009260 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:50:12.398 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.382+0000 7fd4b7d7eb80 1 -- start start 2023-07-21T18:50:12.398 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.382+0000 7fd4b7d7eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f048cfd6b0 0x55f048ea6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:12.398 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.382+0000 7fd4b7d7eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f048ea72d0 0x55f048ea1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:12.398 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.382+0000 7fd4b7d7eb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f048ea1580 0x55f048ea19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:12.398 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.382+0000 7fd4b7d7eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f048da6c50 con 0x55f048cfd6b0 2023-07-21T18:50:12.398 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.382+0000 7fd4b7d7eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f048df99e0 con 0x55f048ea72d0 2023-07-21T18:50:12.398 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.382+0000 7fd4b7d7eb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f048df2e70 con 0x55f048ea1580 2023-07-21T18:50:12.398 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.383+0000 7fd4a9994700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f048cfd6b0 0x55f048ea6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:12.399 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.383+0000 7fd4a9193700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f048ea72d0 0x55f048ea1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:12.399 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.383+0000 7fd4a9193700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f048ea72d0 0x55f048ea1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:44740/0 (socket says 172.21.15.39:44740) 2023-07-21T18:50:12.399 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.383+0000 7fd4a9193700 1 -- 172.21.15.39:0/2270414825 learned_addr learned my addr 172.21.15.39:0/2270414825 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:50:12.399 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.383+0000 7fd4a9994700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f048cfd6b0 0x55f048ea6d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60374/0 (socket says 172.21.15.39:60374) 2023-07-21T18:50:12.399 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.383+0000 7fd4aa195700 1 --2- 172.21.15.39:0/2270414825 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f048ea1580 0x55f048ea19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:12.399 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.383+0000 7fd4a9193700 1 -- 172.21.15.39:0/2270414825 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f048ea1580 msgr2=0x55f048ea19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:50:12.400 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.383+0000 7fd4a9193700 1 --2- 172.21.15.39:0/2270414825 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f048ea1580 0x55f048ea19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:12.400 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.383+0000 7fd4a9193700 1 -- 172.21.15.39:0/2270414825 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f048cfd6b0 msgr2=0x55f048ea6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:50:12.400 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.383+0000 7fd4a9193700 1 --2- 172.21.15.39:0/2270414825 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f048cfd6b0 0x55f048ea6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:12.400 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.383+0000 7fd4a9193700 1 -- 172.21.15.39:0/2270414825 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f048c20d80 con 0x55f048ea72d0 2023-07-21T18:50:12.400 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.383+0000 7fd4a9994700 1 --2- 172.21.15.39:0/2270414825 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f048cfd6b0 0x55f048ea6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:50:12.401 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.383+0000 7fd4a9193700 1 --2- 172.21.15.39:0/2270414825 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f048ea72d0 0x55f048ea1040 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd494011ca0 tx=0x7fd494014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13763f02a26cd3d6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:50:12.401 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.383+0000 7fd48ffff700 1 -- 172.21.15.39:0/2270414825 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd494004220 con 0x55f048ea72d0 2023-07-21T18:50:12.401 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.383+0000 7fd48ffff700 1 -- 172.21.15.39:0/2270414825 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd494021070 con 0x55f048ea72d0 2023-07-21T18:50:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.383+0000 7fd48ffff700 1 -- 172.21.15.39:0/2270414825 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd494003bb0 con 0x55f048ea72d0 2023-07-21T18:50:12.407 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.384+0000 7fd4b7d7eb80 1 -- 172.21.15.39:0/2270414825 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f048ea72d0 msgr2=0x55f048ea1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:50:12.408 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.384+0000 7fd4b7d7eb80 1 --2- 172.21.15.39:0/2270414825 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f048ea72d0 0x55f048ea1040 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd494011ca0 tx=0x7fd494014000 comp rx=0 tx=0).stop 2023-07-21T18:50:12.408 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.384+0000 7fd4b7d7eb80 1 -- 172.21.15.39:0/2270414825 shutdown_connections 2023-07-21T18:50:12.408 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.384+0000 7fd4b7d7eb80 1 --2- 172.21.15.39:0/2270414825 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f048ea1580 0x55f048ea19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:12.408 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.384+0000 7fd4b7d7eb80 1 --2- 172.21.15.39:0/2270414825 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f048cfd6b0 0x55f048ea6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:12.409 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.384+0000 7fd4b7d7eb80 1 --2- 172.21.15.39:0/2270414825 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f048ea72d0 0x55f048ea1040 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:12.409 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.384+0000 7fd4b7d7eb80 1 -- 172.21.15.39:0/2270414825 >> 172.21.15.39:0/2270414825 conn(0x55f048e96be0 msgr2=0x55f048e9bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:50:12.409 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.384+0000 7fd4b7d7eb80 1 -- 172.21.15.39:0/2270414825 shutdown_connections 2023-07-21T18:50:12.409 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.384+0000 7fd4b7d7eb80 1 -- 172.21.15.39:0/2270414825 wait complete. 2023-07-21T18:50:12.409 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.385+0000 7fd4b7d7eb80 1 -- start start 2023-07-21T18:50:12.410 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.385+0000 7fd4b7d7eb80 1 Processor -- start 2023-07-21T18:50:12.410 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.385+0000 7fd4b7d7eb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f048cfd6b0 0x55f048eaa940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:12.410 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.385+0000 7fd4b7d7eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f048ea1580 0x55f048eaae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:12.410 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.385+0000 7fd4b7d7eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f048ea72d0 0x55f048eab3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:12.411 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.385+0000 7fd4b7d7eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f048d92a60 con 0x55f048ea72d0 2023-07-21T18:50:12.411 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.385+0000 7fd4b7d7eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f048d92760 con 0x55f048ea1580 2023-07-21T18:50:12.411 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.385+0000 7fd4b7d7eb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f048c768c0 con 0x55f048cfd6b0 2023-07-21T18:50:12.412 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.385+0000 7fd4b7d7eb80 10 client.0 _ll_register_callbacks cb 0x55f048d90700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:50:12.412 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.385+0000 7fd4a9994700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f048cfd6b0 0x55f048eaa940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:12.412 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.385+0000 7fd4a9193700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f048ea1580 0x55f048eaae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:12.412 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.385+0000 7fd4a9994700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f048cfd6b0 0x55f048eaa940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:50700/0 (socket says 172.21.15.39:50700) 2023-07-21T18:50:12.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.385+0000 7fd4a9994700 1 -- 172.21.15.39:0/1823845709 learned_addr learned my addr 172.21.15.39:0/1823845709 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:50:12.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.385+0000 7fd4aa195700 1 --2- 172.21.15.39:0/1823845709 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f048ea72d0 0x55f048eab3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:12.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.386+0000 7fd4a9994700 1 -- 172.21.15.39:0/1823845709 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f048ea1580 msgr2=0x55f048eaae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:50:12.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.386+0000 7fd4a9994700 1 --2- 172.21.15.39:0/1823845709 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f048ea1580 0x55f048eaae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:12.414 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.386+0000 7fd4a9994700 1 -- 172.21.15.39:0/1823845709 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f048ea72d0 msgr2=0x55f048eab3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:50:12.414 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.386+0000 7fd4a9994700 1 --2- 172.21.15.39:0/1823845709 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f048ea72d0 0x55f048eab3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:12.414 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.386+0000 7fd4a9994700 1 -- 172.21.15.39:0/1823845709 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4a419a650 con 0x55f048cfd6b0 2023-07-21T18:50:12.414 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.386+0000 7fd4a9193700 1 --2- 172.21.15.39:0/1823845709 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f048ea1580 0x55f048eaae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:50:12.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.386+0000 7fd4aa195700 1 --2- 172.21.15.39:0/1823845709 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f048ea72d0 0x55f048eab3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:50:12.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.386+0000 7fd4a9994700 1 --2- 172.21.15.39:0/1823845709 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f048cfd6b0 0x55f048eaa940 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0000c00 tx=0x7fd4a009a4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90334dd0efa8d0df server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:50:12.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.386+0000 7fd48d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2023-07-21T18:50:12.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.386+0000 7fd48d7fa700 1 -- 172.21.15.39:0/1823845709 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a009a200 con 0x55f048cfd6b0 2023-07-21T18:50:12.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.386+0000 7fd48d7fa700 1 -- 172.21.15.39:0/1823845709 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fd4a009eb40 con 0x55f048cfd6b0 2023-07-21T18:50:12.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.386+0000 7fd4b7d7eb80 1 -- 172.21.15.39:0/1823845709 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f048d4c490 con 0x55f048cfd6b0 2023-07-21T18:50:12.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.386+0000 7fd48d7fa700 1 -- 172.21.15.39:0/1823845709 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4a00a2ed0 con 0x55f048cfd6b0 2023-07-21T18:50:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.386+0000 7fd4b7d7eb80 10 client.26257 fetch_fsmap learned FSMap version 58 2023-07-21T18:50:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.386+0000 7fd4b7d7eb80 1 -- 172.21.15.39:0/1823845709 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x7fd494017070 con 0x55f048cfd6b0 2023-07-21T18:50:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.386+0000 7fd48d7fa700 1 -- 172.21.15.39:0/1823845709 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd4a00b7080 con 0x55f048cfd6b0 2023-07-21T18:50:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.387+0000 7fd48d7fa700 1 -- 172.21.15.39:0/1823845709 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 58) v1 ==== 7498+0+0 (secure 0 0 0) 0x7fd4a000fbd0 con 0x55f048cfd6b0 2023-07-21T18:50:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.387+0000 7fd4b7d7eb80 10 client.26257 fetch_fsmap finished waiting for FSMap version 58 2023-07-21T18:50:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.387+0000 7fd4b7d7eb80 10 client.26257 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:50:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.387+0000 7fd4b7d7eb80 20 client.26257 populate_metadata read hostname 'smithi039' 2023-07-21T18:50:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.387+0000 7fd4b7d7eb80 1 --2- 172.21.15.39:0/1823845709 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55f048eba870 0x55f048ebcc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.387+0000 7fd4b7d7eb80 4 client.26257 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:50:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.387+0000 7fd4b7d7eb80 1 -- 172.21.15.39:0/1823845709 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f048debe50 con 0x55f048eba870 2023-07-21T18:50:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.387+0000 7fd4a9193700 1 --2- 172.21.15.39:0/1823845709 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55f048eba870 0x55f048ebcc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:12.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.387+0000 7fd48d7fa700 10 client.26257 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:50:12.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.387+0000 7fd4a9193700 1 --2- 172.21.15.39:0/1823845709 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55f048eba870 0x55f048ebcc50 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=646ee55ec76f5814 server_cookie=a3b4b481ca9d270b in_seq=0 out_seq=0 2023-07-21T18:50:12.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.388+0000 7fd48d7fa700 1 -- 172.21.15.39:0/1823845709 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f048debe50 con 0x55f048eba870 2023-07-21T18:50:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.388+0000 7fd48d7fa700 10 client.26257 handle_command_reply: tid=0 2023-07-21T18:50:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.388+0000 7fd4b7d7eb80 1 client.26257 shutdown 2023-07-21T18:50:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.388+0000 7fd4b7d7eb80 1 -- 172.21.15.39:0/1823845709 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f048cfd6b0 msgr2=0x55f048eaa940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:50:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.388+0000 7fd4b7d7eb80 1 --2- 172.21.15.39:0/1823845709 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f048cfd6b0 0x55f048eaa940 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0000c00 tx=0x7fd4a009a4f0 comp rx=0 tx=0).stop 2023-07-21T18:50:12.422 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.388+0000 7fd4b7d7eb80 1 -- 172.21.15.39:0/1823845709 shutdown_connections 2023-07-21T18:50:12.422 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.388+0000 7fd4b7d7eb80 1 --2- 172.21.15.39:0/1823845709 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55f048eba870 0x55f048ebcc50 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:12.422 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.388+0000 7fd4b7d7eb80 1 --2- 172.21.15.39:0/1823845709 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f048ea72d0 0x55f048eab3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:12.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.388+0000 7fd4b7d7eb80 1 --2- 172.21.15.39:0/1823845709 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f048cfd6b0 0x55f048eaa940 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:12.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.388+0000 7fd4b7d7eb80 1 --2- 172.21.15.39:0/1823845709 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f048ea1580 0x55f048eaae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:12.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.388+0000 7fd48d7fa700 0 client.26257 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:50:12.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.388+0000 7fd4b7d7eb80 1 -- 172.21.15.39:0/1823845709 >> 172.21.15.39:0/1823845709 conn(0x55f048e96be0 msgr2=0x55f048e9ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:50:12.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.388+0000 7fd4b7d7eb80 1 -- 172.21.15.39:0/1823845709 shutdown_connections 2023-07-21T18:50:12.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.388+0000 7fd4b7d7eb80 1 -- 172.21.15.39:0/1823845709 wait complete. 2023-07-21T18:50:12.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.388+0000 7fd4b7d7eb80 20 client.26257 trim_cache size 0 max 16384 2023-07-21T18:50:12.427 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.404+0000 7fd4b7d7eb80 1 Processor -- start 2023-07-21T18:50:12.427 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.404+0000 7fd4b7d7eb80 1 -- start start 2023-07-21T18:50:12.428 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.404+0000 7fd4b7d7eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f048cfd6b0 0x55f048ea1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:12.428 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.404+0000 7fd4b7d7eb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f048ea1580 0x55f048eb1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:12.428 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.404+0000 7fd4b7d7eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f048ea72d0 0x55f048eab1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:12.428 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.404+0000 7fd4b7d7eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f048e98620 con 0x55f048ea72d0 2023-07-21T18:50:12.428 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.404+0000 7fd4b7d7eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f048ea56c0 con 0x55f048cfd6b0 2023-07-21T18:50:12.428 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.404+0000 7fd4b7d7eb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f048eb21a0 con 0x55f048ea1580 2023-07-21T18:50:12.428 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.404+0000 7fd4aa195700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f048ea72d0 0x55f048eab1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:12.429 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.404+0000 7fd4a9994700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f048cfd6b0 0x55f048ea1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:12.429 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.404+0000 7fd4aa195700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f048ea72d0 0x55f048eab1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60392/0 (socket says 172.21.15.39:60392) 2023-07-21T18:50:12.429 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.404+0000 7fd4aa195700 1 -- 172.21.15.39:0/2730128763 learned_addr learned my addr 172.21.15.39:0/2730128763 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:50:12.429 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.404+0000 7fd4a9994700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f048cfd6b0 0x55f048ea1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:44752/0 (socket says 172.21.15.39:44752) 2023-07-21T18:50:12.429 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.404+0000 7fd4a9193700 1 --2- 172.21.15.39:0/2730128763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f048ea1580 0x55f048eb1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:12.429 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.404+0000 7fd4aa195700 1 -- 172.21.15.39:0/2730128763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f048ea1580 msgr2=0x55f048eb1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:50:12.430 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.404+0000 7fd4aa195700 1 --2- 172.21.15.39:0/2730128763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f048ea1580 0x55f048eb1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:12.430 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.404+0000 7fd4aa195700 1 -- 172.21.15.39:0/2730128763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f048cfd6b0 msgr2=0x55f048ea1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:50:12.430 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.404+0000 7fd4aa195700 1 --2- 172.21.15.39:0/2730128763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f048cfd6b0 0x55f048ea1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:12.430 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.404+0000 7fd4aa195700 1 -- 172.21.15.39:0/2730128763 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f048c09c00 con 0x55f048ea72d0 2023-07-21T18:50:12.430 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.404+0000 7fd4a9994700 1 --2- 172.21.15.39:0/2730128763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f048cfd6b0 0x55f048ea1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:50:12.430 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.404+0000 7fd4a9193700 1 --2- 172.21.15.39:0/2730128763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f048ea1580 0x55f048eb1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:50:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.404+0000 7fd4aa195700 1 --2- 172.21.15.39:0/2730128763 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f048ea72d0 0x55f048eab1f0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fd49c0079c0 tx=0x7fd49c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c148579063bb565f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:50:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.405+0000 7fd498dac700 1 -- 172.21.15.39:0/2730128763 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd49c00e040 con 0x55f048ea72d0 2023-07-21T18:50:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.405+0000 7fd498dac700 1 -- 172.21.15.39:0/2730128763 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd49c00f040 con 0x55f048ea72d0 2023-07-21T18:50:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.405+0000 7fd498dac700 1 -- 172.21.15.39:0/2730128763 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd49c004280 con 0x55f048ea72d0 2023-07-21T18:50:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.405+0000 7fd4b7d7eb80 1 -- 172.21.15.39:0/2730128763 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f048ea72d0 msgr2=0x55f048eab1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:50:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.405+0000 7fd4b7d7eb80 1 --2- 172.21.15.39:0/2730128763 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f048ea72d0 0x55f048eab1f0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fd49c0079c0 tx=0x7fd49c00b040 comp rx=0 tx=0).stop 2023-07-21T18:50:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.405+0000 7fd4b7d7eb80 1 -- 172.21.15.39:0/2730128763 shutdown_connections 2023-07-21T18:50:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.405+0000 7fd4b7d7eb80 1 --2- 172.21.15.39:0/2730128763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f048cfd6b0 0x55f048ea1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.405+0000 7fd4b7d7eb80 1 --2- 172.21.15.39:0/2730128763 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f048ea72d0 0x55f048eab1f0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.405+0000 7fd4b7d7eb80 1 --2- 172.21.15.39:0/2730128763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f048ea1580 0x55f048eb1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.405+0000 7fd4b7d7eb80 1 -- 172.21.15.39:0/2730128763 >> 172.21.15.39:0/2730128763 conn(0x55f048e96be0 msgr2=0x55f048ea6360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:50:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.405+0000 7fd4b7d7eb80 1 -- 172.21.15.39:0/2730128763 shutdown_connections 2023-07-21T18:50:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.405+0000 7fd4b7d7eb80 1 -- 172.21.15.39:0/2730128763 wait complete. 2023-07-21T18:50:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.406+0000 7fd4b7d7eb80 1 -- start start 2023-07-21T18:50:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.406+0000 7fd4b7d7eb80 1 Processor -- start 2023-07-21T18:50:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.406+0000 7fd4b7d7eb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f048cfd6b0 0x55f048eb1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.406+0000 7fd4b7d7eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f048ea1580 0x55f048eafb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.406+0000 7fd4b7d7eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f048ea72d0 0x55f048eb00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.406+0000 7fd4b7d7eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f048eb05f0 con 0x55f048ea1580 2023-07-21T18:50:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.406+0000 7fd4b7d7eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f048eb0760 con 0x55f048ea72d0 2023-07-21T18:50:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.406+0000 7fd4b7d7eb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f048eb08d0 con 0x55f048cfd6b0 2023-07-21T18:50:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.406+0000 7fd4b7d7eb80 10 client.0 _ll_register_callbacks cb 0x7fd4a009a9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:50:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.406+0000 7fd4a9193700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f048ea1580 0x55f048eafb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.406+0000 7fd4a9994700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f048cfd6b0 0x55f048eb1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.406+0000 7fd4a9994700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f048cfd6b0 0x55f048eb1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:50714/0 (socket says 172.21.15.39:50714) 2023-07-21T18:50:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.406+0000 7fd4a9193700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f048ea1580 0x55f048eafb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60396/0 (socket says 172.21.15.39:60396) 2023-07-21T18:50:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.406+0000 7fd4a9994700 1 -- 172.21.15.39:0/2194874328 learned_addr learned my addr 172.21.15.39:0/2194874328 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:50:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.406+0000 7fd4aa195700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f048ea72d0 0x55f048eb00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:12.435 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.407+0000 7fd4a9193700 1 -- 172.21.15.39:0/2194874328 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f048cfd6b0 msgr2=0x55f048eb1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:50:12.435 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.407+0000 7fd4a9193700 1 --2- 172.21.15.39:0/2194874328 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f048cfd6b0 0x55f048eb1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:12.435 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.407+0000 7fd4a9193700 1 -- 172.21.15.39:0/2194874328 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f048ea72d0 msgr2=0x55f048eb00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:50:12.435 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.407+0000 7fd4a9193700 1 --2- 172.21.15.39:0/2194874328 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f048ea72d0 0x55f048eb00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:12.435 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.407+0000 7fd4a9193700 1 -- 172.21.15.39:0/2194874328 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd494017070 con 0x55f048ea1580 2023-07-21T18:50:12.435 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.407+0000 7fd4a9994700 1 --2- 172.21.15.39:0/2194874328 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f048cfd6b0 0x55f048eb1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:50:12.435 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.407+0000 7fd4aa195700 1 --2- 172.21.15.39:0/2194874328 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f048ea72d0 0x55f048eb00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:50:12.436 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.407+0000 7fd4a9193700 1 --2- 172.21.15.39:0/2194874328 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f048ea1580 0x55f048eafb70 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fd4940097c0 tx=0x7fd494005c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f83b70ca2bea0acc server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:50:12.436 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.407+0000 7fd48d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:50:12.436 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.407+0000 7fd48d7fa700 1 -- 172.21.15.39:0/2194874328 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd49402a040 con 0x55f048ea1580 2023-07-21T18:50:12.436 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.407+0000 7fd48d7fa700 1 -- 172.21.15.39:0/2194874328 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fd494032a40 con 0x55f048ea1580 2023-07-21T18:50:12.436 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.407+0000 7fd4b7d7eb80 1 -- 172.21.15.39:0/2194874328 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd494021070 con 0x55f048ea1580 2023-07-21T18:50:12.436 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.407+0000 7fd48d7fa700 1 -- 172.21.15.39:0/2194874328 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd49400ea40 con 0x55f048ea1580 2023-07-21T18:50:12.436 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.407+0000 7fd4b7d7eb80 10 client.16782 fetch_fsmap learned FSMap version 58 2023-07-21T18:50:12.437 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.407+0000 7fd4b7d7eb80 1 -- 172.21.15.39:0/2194874328 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x7fd49c007670 con 0x55f048ea1580 2023-07-21T18:50:12.437 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.407+0000 7fd48d7fa700 1 -- 172.21.15.39:0/2194874328 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd494021070 con 0x55f048ea1580 2023-07-21T18:50:12.437 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.407+0000 7fd48d7fa700 1 -- 172.21.15.39:0/2194874328 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 58) v1 ==== 7498+0+0 (secure 0 0 0) 0x7fd4940265a0 con 0x55f048ea1580 2023-07-21T18:50:12.437 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.408+0000 7fd4b7d7eb80 10 client.16782 fetch_fsmap finished waiting for FSMap version 58 2023-07-21T18:50:12.437 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.408+0000 7fd4b7d7eb80 10 client.16782 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:50:12.437 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.408+0000 7fd4b7d7eb80 20 client.16782 populate_metadata read hostname 'smithi039' 2023-07-21T18:50:12.437 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.408+0000 7fd4b7d7eb80 1 --2- 172.21.15.39:0/2194874328 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55f048f0fbb0 0x55f048f11fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:12.437 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.408+0000 7fd4b7d7eb80 4 client.16782 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:50:12.438 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.408+0000 7fd4b7d7eb80 1 -- 172.21.15.39:0/2194874328 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f048dadff0 con 0x55f048f0fbb0 2023-07-21T18:50:12.438 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.408+0000 7fd4a9994700 1 --2- 172.21.15.39:0/2194874328 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55f048f0fbb0 0x55f048f11fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:12.438 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.408+0000 7fd48d7fa700 10 client.16782 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:50:12.438 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.408+0000 7fd4a9994700 1 --2- 172.21.15.39:0/2194874328 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55f048f0fbb0 0x55f048f11fc0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e61510c749b2384 server_cookie=8497a7f46ecafdba in_seq=0 out_seq=0 2023-07-21T18:50:12.438 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:50:12 smithi039 ceph-mon[96489]: pgmap v262: 97 pgs: 97 active+clean; 210 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 1.3 MiB/s wr, 162 op/s 2023-07-21T18:50:12.439 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:50:12 smithi039 ceph-mon[96489]: osdmap e89: 12 total, 12 up, 12 in 2023-07-21T18:50:12.451 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.438+0000 7fd48d7fa700 1 -- 172.21.15.39:0/2194874328 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f048dadff0 con 0x55f048f0fbb0 2023-07-21T18:50:12.452 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.438+0000 7fd48d7fa700 10 client.16782 handle_command_reply: tid=0 2023-07-21T18:50:12.452 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.438+0000 7fd4b7d7eb80 1 client.16782 shutdown 2023-07-21T18:50:12.452 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.438+0000 7fd4b7d7eb80 1 -- 172.21.15.39:0/2194874328 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f048ea1580 msgr2=0x55f048eafb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:50:12.452 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.438+0000 7fd4b7d7eb80 1 --2- 172.21.15.39:0/2194874328 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f048ea1580 0x55f048eafb70 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fd4940097c0 tx=0x7fd494005c20 comp rx=0 tx=0).stop 2023-07-21T18:50:12.452 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.438+0000 7fd4b7d7eb80 1 -- 172.21.15.39:0/2194874328 shutdown_connections 2023-07-21T18:50:12.452 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.438+0000 7fd4b7d7eb80 1 --2- 172.21.15.39:0/2194874328 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55f048f0fbb0 0x55f048f11fc0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.438+0000 7fd4b7d7eb80 1 --2- 172.21.15.39:0/2194874328 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f048ea72d0 0x55f048eb00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.438+0000 7fd4b7d7eb80 1 --2- 172.21.15.39:0/2194874328 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f048cfd6b0 0x55f048eb1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.438+0000 7fd4b7d7eb80 1 --2- 172.21.15.39:0/2194874328 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f048ea1580 0x55f048eafb70 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.438+0000 7fd4b7d7eb80 1 -- 172.21.15.39:0/2194874328 >> 172.21.15.39:0/2194874328 conn(0x55f048e96be0 msgr2=0x55f048eaaeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:50:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.438+0000 7fd48d7fa700 0 client.16782 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:50:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.438+0000 7fd4b7d7eb80 1 -- 172.21.15.39:0/2194874328 shutdown_connections 2023-07-21T18:50:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.439+0000 7fd4b7d7eb80 1 -- 172.21.15.39:0/2194874328 wait complete. 2023-07-21T18:50:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.439+0000 7fd4b7d7eb80 20 client.16782 trim_cache size 0 max 16384 2023-07-21T18:50:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:50:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:50:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:50:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:50:12.455 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.439+0000 7fd4abbf8700 1 -- 172.21.15.39:0/979860859 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fd4900a12c0 msgr2=0x7fd4900a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:50:12.455 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.439+0000 7fd4abbf8700 1 --2- 172.21.15.39:0/979860859 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fd4900a12c0 0x7fd4900a3770 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd4a41974b0 tx=0x7fd494009260 comp rx=0 tx=0).stop 2023-07-21T18:50:12.456 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.439+0000 7fd4abbf8700 1 -- 172.21.15.39:0/979860859 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4a4067c10 msgr2=0x7fd4a4195cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:50:12.456 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.439+0000 7fd4abbf8700 1 --2- 172.21.15.39:0/979860859 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4a4067c10 0x7fd4a4195cd0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fd4a00098e0 tx=0x7fd4a000f690 comp rx=0 tx=0).stop 2023-07-21T18:50:12.456 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.439+0000 7fd4abbf8700 1 -- 172.21.15.39:0/979860859 shutdown_connections 2023-07-21T18:50:12.456 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.439+0000 7fd4abbf8700 1 --2- 172.21.15.39:0/979860859 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd4a4104e50 0x7fd4a41997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:12.456 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.439+0000 7fd4abbf8700 1 --2- 172.21.15.39:0/979860859 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4a4067c10 0x7fd4a4195cd0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:12.456 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.439+0000 7fd4abbf8700 1 --2- 172.21.15.39:0/979860859 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fd4900a12c0 0x7fd4900a3770 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:12.457 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.439+0000 7fd4abbf8700 1 --2- 172.21.15.39:0/979860859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4a40685b0 0x7fd4a4196210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:12.457 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.439+0000 7fd4abbf8700 1 -- 172.21.15.39:0/979860859 >> 172.21.15.39:0/979860859 conn(0x7fd4a4074760 msgr2=0x7fd4a40fc5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:50:12.457 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.439+0000 7fd4abbf8700 1 -- 172.21.15.39:0/979860859 shutdown_connections 2023-07-21T18:50:12.457 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:12.439+0000 7fd4abbf8700 1 -- 172.21.15.39:0/979860859 wait complete. 2023-07-21T18:50:12.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:50:12 smithi193 ceph-mon[98204]: pgmap v262: 97 pgs: 97 active+clean; 210 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 1.3 MiB/s wr, 162 op/s 2023-07-21T18:50:12.611 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:50:12 smithi193 ceph-mon[98204]: osdmap e89: 12 total, 12 up, 12 in 2023-07-21T18:50:13.280 INFO:tasks.workunit.client.0.smithi039.stdout: 1 529755 6.28 MB/sec execute 326 sec latency 33.131 ms 2023-07-21T18:50:14.280 INFO:tasks.workunit.client.0.smithi039.stdout: 1 530999 6.28 MB/sec execute 327 sec latency 215.593 ms 2023-07-21T18:50:14.379 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:50:14 smithi155 ceph-mon[97934]: pgmap v264: 97 pgs: 97 active+clean; 210 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 875 KiB/s rd, 1.6 MiB/s wr, 199 op/s 2023-07-21T18:50:14.428 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:50:14 smithi039 ceph-mon[96489]: pgmap v264: 97 pgs: 97 active+clean; 210 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 875 KiB/s rd, 1.6 MiB/s wr, 199 op/s 2023-07-21T18:50:14.611 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:50:14 smithi193 ceph-mon[98204]: pgmap v264: 97 pgs: 97 active+clean; 210 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 875 KiB/s rd, 1.6 MiB/s wr, 199 op/s 2023-07-21T18:50:15.280 INFO:tasks.workunit.client.0.smithi039.stdout: 1 530999 6.26 MB/sec execute 328 sec latency 1215.683 ms 2023-07-21T18:50:16.280 INFO:tasks.workunit.client.0.smithi039.stdout: 1 530999 6.24 MB/sec execute 329 sec latency 2215.792 ms 2023-07-21T18:50:16.428 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:50:16 smithi039 ceph-mon[96489]: pgmap v265: 97 pgs: 97 active+clean; 210 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 1.3 MiB/s wr, 162 op/s 2023-07-21T18:50:16.611 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:50:16 smithi193 ceph-mon[98204]: pgmap v265: 97 pgs: 97 active+clean; 210 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 1.3 MiB/s wr, 162 op/s 2023-07-21T18:50:16.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:50:16 smithi155 ceph-mon[97934]: pgmap v265: 97 pgs: 97 active+clean; 210 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 1.3 MiB/s wr, 162 op/s 2023-07-21T18:50:17.280 INFO:tasks.workunit.client.0.smithi039.stdout: 1 532263 6.24 MB/sec execute 330 sec latency 2653.774 ms 2023-07-21T18:50:18.280 INFO:tasks.workunit.client.0.smithi039.stdout: 1 533564 6.25 MB/sec execute 331 sec latency 15.751 ms 2023-07-21T18:50:18.428 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:50:18 smithi039 ceph-mon[96489]: pgmap v266: 97 pgs: 97 active+clean; 210 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 986 KiB/s wr, 125 op/s 2023-07-21T18:50:18.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:50:18 smithi193 ceph-mon[98204]: pgmap v266: 97 pgs: 97 active+clean; 210 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 986 KiB/s wr, 125 op/s 2023-07-21T18:50:18.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:50:18 smithi155 ceph-mon[97934]: pgmap v266: 97 pgs: 97 active+clean; 210 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 986 KiB/s wr, 125 op/s 2023-07-21T18:50:19.281 INFO:tasks.workunit.client.0.smithi039.stdout: 1 534579 6.24 MB/sec execute 332 sec latency 378.892 ms 2023-07-21T18:50:20.281 INFO:tasks.workunit.client.0.smithi039.stdout: 1 534579 6.22 MB/sec execute 333 sec latency 1378.949 ms 2023-07-21T18:50:20.428 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:50:20 smithi039 ceph-mon[96489]: pgmap v267: 97 pgs: 97 active+clean; 210 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 986 KiB/s wr, 125 op/s 2023-07-21T18:50:20.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:50:20 smithi193 ceph-mon[98204]: pgmap v267: 97 pgs: 97 active+clean; 210 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 986 KiB/s wr, 125 op/s 2023-07-21T18:50:20.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:50:20 smithi155 ceph-mon[97934]: pgmap v267: 97 pgs: 97 active+clean; 210 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 986 KiB/s wr, 125 op/s 2023-07-21T18:50:21.281 INFO:tasks.workunit.client.0.smithi039.stdout: 1 534579 6.21 MB/sec execute 334 sec latency 2379.010 ms 2023-07-21T18:50:22.281 INFO:tasks.workunit.client.0.smithi039.stdout: 1 535808 6.21 MB/sec execute 335 sec latency 2811.105 ms 2023-07-21T18:50:22.428 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:50:22 smithi039 ceph-mon[96489]: pgmap v268: 97 pgs: 97 active+clean; 213 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 1.2 MiB/s wr, 133 op/s 2023-07-21T18:50:22.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:50:22 smithi193 ceph-mon[98204]: pgmap v268: 97 pgs: 97 active+clean; 213 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 1.2 MiB/s wr, 133 op/s 2023-07-21T18:50:22.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:50:22 smithi155 ceph-mon[97934]: pgmap v268: 97 pgs: 97 active+clean; 213 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 1.2 MiB/s wr, 133 op/s 2023-07-21T18:50:23.281 INFO:tasks.workunit.client.0.smithi039.stdout: 1 537050 6.21 MB/sec execute 336 sec latency 21.167 ms 2023-07-21T18:50:24.281 INFO:tasks.workunit.client.0.smithi039.stdout: 1 538122 6.21 MB/sec execute 337 sec latency 349.324 ms 2023-07-21T18:50:24.428 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:50:24 smithi039 ceph-mon[96489]: pgmap v269: 97 pgs: 97 active+clean; 213 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 1.1 MiB/s wr, 118 op/s 2023-07-21T18:50:24.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:50:24 smithi193 ceph-mon[98204]: pgmap v269: 97 pgs: 97 active+clean; 213 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 1.1 MiB/s wr, 118 op/s 2023-07-21T18:50:24.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:50:24 smithi155 ceph-mon[97934]: pgmap v269: 97 pgs: 97 active+clean; 213 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 1.1 MiB/s wr, 118 op/s 2023-07-21T18:50:25.281 INFO:tasks.workunit.client.0.smithi039.stdout: 1 538122 6.19 MB/sec execute 338 sec latency 1349.445 ms 2023-07-21T18:50:25.575 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.561+0000 7fcaa0dd6700 1 -- 172.21.15.39:0/1257653573 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca9c0ff3e0 msgr2=0x7fca9c10b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:50:25.575 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.561+0000 7fcaa0dd6700 1 --2- 172.21.15.39:0/1257653573 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca9c0ff3e0 0x7fca9c10b860 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fca9400b0a0 tx=0x7fca9400b3b0 comp rx=0 tx=0).stop 2023-07-21T18:50:25.576 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.561+0000 7fcaa0dd6700 1 -- 172.21.15.39:0/1257653573 shutdown_connections 2023-07-21T18:50:25.576 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.561+0000 7fcaa0dd6700 1 --2- 172.21.15.39:0/1257653573 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca9c0ff3e0 0x7fca9c10b860 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:25.576 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.561+0000 7fcaa0dd6700 1 --2- 172.21.15.39:0/1257653573 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fca9c104a30 0x7fca9c104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:25.576 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.561+0000 7fcaa0dd6700 1 --2- 172.21.15.39:0/1257653573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca9c0fea30 0x7fca9c0feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:25.576 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.561+0000 7fcaa0dd6700 1 -- 172.21.15.39:0/1257653573 >> 172.21.15.39:0/1257653573 conn(0x7fca9c074760 msgr2=0x7fca9c074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:50:25.576 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.561+0000 7fcaa0dd6700 1 -- 172.21.15.39:0/1257653573 shutdown_connections 2023-07-21T18:50:25.576 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.561+0000 7fcaa0dd6700 1 -- 172.21.15.39:0/1257653573 wait complete. 2023-07-21T18:50:25.578 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.562+0000 7fcaa0dd6700 1 Processor -- start 2023-07-21T18:50:25.578 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.562+0000 7fcaa0dd6700 1 -- start start 2023-07-21T18:50:25.578 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.562+0000 7fcaa0dd6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca9c0fea30 0x7fca9c19bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:25.578 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.562+0000 7fcaa0dd6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca9c0ff3e0 0x7fca9c19c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:25.578 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.562+0000 7fcaa0dd6700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fca9c104a30 0x7fca9c195ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.562+0000 7fcaa0dd6700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9c196120 con 0x7fca9c0fea30 2023-07-21T18:50:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.562+0000 7fcaa0dd6700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9c196290 con 0x7fca9c0ff3e0 2023-07-21T18:50:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.562+0000 7fcaa0dd6700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9c196450 con 0x7fca9c104a30 2023-07-21T18:50:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.562+0000 7fca9a59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca9c0fea30 0x7fca9c19bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.562+0000 7fca9a59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca9c0fea30 0x7fca9c19bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60408/0 (socket says 172.21.15.39:60408) 2023-07-21T18:50:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.562+0000 7fca9a59c700 1 -- 172.21.15.39:0/4229648951 learned_addr learned my addr 172.21.15.39:0/4229648951 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:50:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.563+0000 7fca93fff700 1 --2- 172.21.15.39:0/4229648951 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca9c0ff3e0 0x7fca9c19c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.563+0000 7fca9ad9d700 1 --2- 172.21.15.39:0/4229648951 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fca9c104a30 0x7fca9c195ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:25.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.563+0000 7fca9a59c700 1 -- 172.21.15.39:0/4229648951 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fca9c104a30 msgr2=0x7fca9c195ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:50:25.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.563+0000 7fca9a59c700 1 --2- 172.21.15.39:0/4229648951 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fca9c104a30 0x7fca9c195ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:25.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.563+0000 7fca9a59c700 1 -- 172.21.15.39:0/4229648951 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca9c0ff3e0 msgr2=0x7fca9c19c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:50:25.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.563+0000 7fca9a59c700 1 --2- 172.21.15.39:0/4229648951 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca9c0ff3e0 0x7fca9c19c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:25.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.563+0000 7fca9a59c700 1 -- 172.21.15.39:0/4229648951 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca94009d00 con 0x7fca9c0fea30 2023-07-21T18:50:25.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.563+0000 7fca9a59c700 1 --2- 172.21.15.39:0/4229648951 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca9c0fea30 0x7fca9c19bc50 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fca8c00d980 tx=0x7fca8c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:50:25.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.563+0000 7fca937fe700 1 -- 172.21.15.39:0/4229648951 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8c004020 con 0x7fca9c0fea30 2023-07-21T18:50:25.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.563+0000 7fcaa0dd6700 1 -- 172.21.15.39:0/4229648951 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca9c196730 con 0x7fca9c0fea30 2023-07-21T18:50:25.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.563+0000 7fca937fe700 1 -- 172.21.15.39:0/4229648951 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca8c00f690 con 0x7fca9c0fea30 2023-07-21T18:50:25.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.563+0000 7fca937fe700 1 -- 172.21.15.39:0/4229648951 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8c003a60 con 0x7fca9c0fea30 2023-07-21T18:50:25.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.563+0000 7fcaa0dd6700 1 -- 172.21.15.39:0/4229648951 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca9c1a2c40 con 0x7fca9c0fea30 2023-07-21T18:50:25.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.566+0000 7fca937fe700 1 -- 172.21.15.39:0/4229648951 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fca8c00fb00 con 0x7fca9c0fea30 2023-07-21T18:50:25.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.566+0000 7fca937fe700 1 --2- 172.21.15.39:0/4229648951 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fca880a12c0 0x7fca880a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:25.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.566+0000 7fca937fe700 1 -- 172.21.15.39:0/4229648951 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fca8c0cc910 con 0x7fca9c0fea30 2023-07-21T18:50:25.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.566+0000 7fca93fff700 1 --2- 172.21.15.39:0/4229648951 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fca880a12c0 0x7fca880a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:25.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.567+0000 7fca93fff700 1 --2- 172.21.15.39:0/4229648951 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fca880a12c0 0x7fca880a3770 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fca9c0fca20 tx=0x7fca84009260 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:50:25.584 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.567+0000 7fcaacf5cb80 1 Processor -- start 2023-07-21T18:50:25.584 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.567+0000 7fcaacf5cb80 1 -- start start 2023-07-21T18:50:25.584 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.567+0000 7fcaacf5cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3aa8306b0 0x55d3aa9d9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:25.584 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.567+0000 7fcaacf5cb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3aa9da2d0 0x55d3aa9d4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:25.584 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.567+0000 7fcaacf5cb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d3aa9d4580 0x55d3aa9d49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:25.584 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.567+0000 7fcaacf5cb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3aa8d9c50 con 0x55d3aa9da2d0 2023-07-21T18:50:25.584 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.567+0000 7fcaacf5cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3aa92c9e0 con 0x55d3aa8306b0 2023-07-21T18:50:25.585 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.567+0000 7fcaacf5cb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3aa925e70 con 0x55d3aa9d4580 2023-07-21T18:50:25.585 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.568+0000 7fca93fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3aa9da2d0 0x55d3aa9d4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:25.585 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.568+0000 7fca93fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3aa9da2d0 0x55d3aa9d4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60418/0 (socket says 172.21.15.39:60418) 2023-07-21T18:50:25.585 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.568+0000 7fca93fff700 1 -- 172.21.15.39:0/1247179712 learned_addr learned my addr 172.21.15.39:0/1247179712 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:50:25.585 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.568+0000 7fca9a59c700 1 --2- 172.21.15.39:0/1247179712 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3aa8306b0 0x55d3aa9d9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:25.585 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.568+0000 7fca9ad9d700 1 --2- 172.21.15.39:0/1247179712 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d3aa9d4580 0x55d3aa9d49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:25.585 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.568+0000 7fca93fff700 1 -- 172.21.15.39:0/1247179712 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d3aa9d4580 msgr2=0x55d3aa9d49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:50:25.585 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.568+0000 7fca93fff700 1 --2- 172.21.15.39:0/1247179712 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d3aa9d4580 0x55d3aa9d49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:25.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.568+0000 7fca93fff700 1 -- 172.21.15.39:0/1247179712 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3aa8306b0 msgr2=0x55d3aa9d9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:50:25.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.568+0000 7fca93fff700 1 --2- 172.21.15.39:0/1247179712 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3aa8306b0 0x55d3aa9d9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:25.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.568+0000 7fca93fff700 1 -- 172.21.15.39:0/1247179712 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3aa753d80 con 0x55d3aa9da2d0 2023-07-21T18:50:25.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.568+0000 7fca9a59c700 1 --2- 172.21.15.39:0/1247179712 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3aa8306b0 0x55d3aa9d9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:50:25.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.568+0000 7fca93fff700 1 --2- 172.21.15.39:0/1247179712 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3aa9da2d0 0x55d3aa9d4040 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fca84011d50 tx=0x7fca84014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0fa170aa217cb42 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:50:25.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.568+0000 7fca90ff9700 1 -- 172.21.15.39:0/1247179712 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8400ebd0 con 0x55d3aa9da2d0 2023-07-21T18:50:25.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.568+0000 7fca90ff9700 1 -- 172.21.15.39:0/1247179712 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca84021070 con 0x55d3aa9da2d0 2023-07-21T18:50:25.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.568+0000 7fca90ff9700 1 -- 172.21.15.39:0/1247179712 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8401d670 con 0x55d3aa9da2d0 2023-07-21T18:50:25.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.569+0000 7fcaacf5cb80 1 -- 172.21.15.39:0/1247179712 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3aa9da2d0 msgr2=0x55d3aa9d4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:50:25.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.569+0000 7fcaacf5cb80 1 --2- 172.21.15.39:0/1247179712 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3aa9da2d0 0x55d3aa9d4040 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fca84011d50 tx=0x7fca84014000 comp rx=0 tx=0).stop 2023-07-21T18:50:25.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.569+0000 7fcaacf5cb80 1 -- 172.21.15.39:0/1247179712 shutdown_connections 2023-07-21T18:50:25.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.569+0000 7fcaacf5cb80 1 --2- 172.21.15.39:0/1247179712 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3aa8306b0 0x55d3aa9d9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:25.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.569+0000 7fcaacf5cb80 1 --2- 172.21.15.39:0/1247179712 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d3aa9d4580 0x55d3aa9d49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:25.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.569+0000 7fcaacf5cb80 1 --2- 172.21.15.39:0/1247179712 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3aa9da2d0 0x55d3aa9d4040 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:25.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.569+0000 7fcaacf5cb80 1 -- 172.21.15.39:0/1247179712 >> 172.21.15.39:0/1247179712 conn(0x55d3aa9c9be0 msgr2=0x55d3aa9cef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:50:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.569+0000 7fcaacf5cb80 1 -- 172.21.15.39:0/1247179712 shutdown_connections 2023-07-21T18:50:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.569+0000 7fcaacf5cb80 1 -- 172.21.15.39:0/1247179712 wait complete. 2023-07-21T18:50:25.590 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.569+0000 7fcaacf5cb80 1 -- start start 2023-07-21T18:50:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.570+0000 7fcaacf5cb80 1 Processor -- start 2023-07-21T18:50:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.570+0000 7fcaacf5cb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d3aa8306b0 0x55d3aa9dd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.570+0000 7fcaacf5cb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3aa9d4580 0x55d3aa9dde20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.570+0000 7fcaacf5cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3aa9da2d0 0x55d3aa9de360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.570+0000 7fcaacf5cb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3aa8c5a60 con 0x55d3aa9d4580 2023-07-21T18:50:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.570+0000 7fcaacf5cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3aa8c5760 con 0x55d3aa9da2d0 2023-07-21T18:50:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.570+0000 7fcaacf5cb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3aa7a98c0 con 0x55d3aa8306b0 2023-07-21T18:50:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.570+0000 7fcaacf5cb80 10 client.0 _ll_register_callbacks cb 0x55d3aa8c3700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:50:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.570+0000 7fca93fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3aa9d4580 0x55d3aa9dde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.570+0000 7fca93fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3aa9d4580 0x55d3aa9dde20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60424/0 (socket says 172.21.15.39:60424) 2023-07-21T18:50:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.570+0000 7fca93fff700 1 -- 172.21.15.39:0/1164782766 learned_addr learned my addr 172.21.15.39:0/1164782766 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:50:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.570+0000 7fca9a59c700 1 --2- 172.21.15.39:0/1164782766 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d3aa8306b0 0x55d3aa9dd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.570+0000 7fca9ad9d700 1 --2- 172.21.15.39:0/1164782766 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3aa9da2d0 0x55d3aa9de360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.570+0000 7fca93fff700 1 -- 172.21.15.39:0/1164782766 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d3aa8306b0 msgr2=0x55d3aa9dd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:50:25.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.570+0000 7fca93fff700 1 --2- 172.21.15.39:0/1164782766 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d3aa8306b0 0x55d3aa9dd8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:25.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.570+0000 7fca93fff700 1 -- 172.21.15.39:0/1164782766 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3aa9da2d0 msgr2=0x55d3aa9de360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:50:25.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.570+0000 7fca93fff700 1 --2- 172.21.15.39:0/1164782766 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3aa9da2d0 0x55d3aa9de360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:25.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.570+0000 7fca93fff700 1 -- 172.21.15.39:0/1164782766 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca84017070 con 0x55d3aa9d4580 2023-07-21T18:50:25.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.570+0000 7fca9ad9d700 1 --2- 172.21.15.39:0/1164782766 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3aa9da2d0 0x55d3aa9de360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:50:25.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.571+0000 7fca93fff700 1 --2- 172.21.15.39:0/1164782766 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3aa9d4580 0x55d3aa9dde20 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fca8400ea20 tx=0x7fca8401cfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21b8b96b464a0d1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:50:25.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.571+0000 7fca81ffb700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:50:25.594 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.571+0000 7fca81ffb700 1 -- 172.21.15.39:0/1164782766 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca84005250 con 0x55d3aa9d4580 2023-07-21T18:50:25.594 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.571+0000 7fca81ffb700 1 -- 172.21.15.39:0/1164782766 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fca840053b0 con 0x55d3aa9d4580 2023-07-21T18:50:25.594 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.571+0000 7fcaacf5cb80 1 -- 172.21.15.39:0/1164782766 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d3aa87f490 con 0x55d3aa9d4580 2023-07-21T18:50:25.594 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.571+0000 7fca81ffb700 1 -- 172.21.15.39:0/1164782766 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca84005570 con 0x55d3aa9d4580 2023-07-21T18:50:25.594 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.571+0000 7fcaacf5cb80 10 client.16794 fetch_fsmap learned FSMap version 58 2023-07-21T18:50:25.594 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.571+0000 7fcaacf5cb80 1 -- 172.21.15.39:0/1164782766 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x55d3aa73cc00 con 0x55d3aa9d4580 2023-07-21T18:50:25.594 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.571+0000 7fca81ffb700 1 -- 172.21.15.39:0/1164782766 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d3aa87f490 con 0x55d3aa9d4580 2023-07-21T18:50:25.594 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.571+0000 7fca81ffb700 1 -- 172.21.15.39:0/1164782766 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 58) v1 ==== 7498+0+0 (secure 0 0 0) 0x7fca840039e0 con 0x55d3aa9d4580 2023-07-21T18:50:25.595 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.571+0000 7fcaacf5cb80 10 client.16794 fetch_fsmap finished waiting for FSMap version 58 2023-07-21T18:50:25.595 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.571+0000 7fcaacf5cb80 10 client.16794 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:50:25.595 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.571+0000 7fcaacf5cb80 20 client.16794 populate_metadata read hostname 'smithi039' 2023-07-21T18:50:25.595 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.571+0000 7fcaacf5cb80 1 --2- 172.21.15.39:0/1164782766 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d3aa9edc30 0x55d3aa9f0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:25.595 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.571+0000 7fcaacf5cb80 4 client.16794 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:50:25.595 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.571+0000 7fcaacf5cb80 1 -- 172.21.15.39:0/1164782766 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d3aa91ee50 con 0x55d3aa9edc30 2023-07-21T18:50:25.595 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.572+0000 7fca9a59c700 1 --2- 172.21.15.39:0/1164782766 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d3aa9edc30 0x55d3aa9f0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:25.595 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.572+0000 7fca9a59c700 1 --2- 172.21.15.39:0/1164782766 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d3aa9edc30 0x55d3aa9f0040 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70a797dfd32b428d server_cookie=ffe003099852d77e in_seq=0 out_seq=0 2023-07-21T18:50:25.596 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.572+0000 7fca81ffb700 10 client.16794 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:50:25.596 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.573+0000 7fca81ffb700 1 -- 172.21.15.39:0/1164782766 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d3aa91ee50 con 0x55d3aa9edc30 2023-07-21T18:50:25.596 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.573+0000 7fca81ffb700 10 client.16794 handle_command_reply: tid=0 2023-07-21T18:50:25.596 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.573+0000 7fcaacf5cb80 1 client.16794 shutdown 2023-07-21T18:50:25.596 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.573+0000 7fcaacf5cb80 1 -- 172.21.15.39:0/1164782766 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3aa9d4580 msgr2=0x55d3aa9dde20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:50:25.596 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.573+0000 7fcaacf5cb80 1 --2- 172.21.15.39:0/1164782766 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3aa9d4580 0x55d3aa9dde20 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fca8400ea20 tx=0x7fca8401cfd0 comp rx=0 tx=0).stop 2023-07-21T18:50:25.596 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.573+0000 7fcaacf5cb80 1 -- 172.21.15.39:0/1164782766 shutdown_connections 2023-07-21T18:50:25.597 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.573+0000 7fcaacf5cb80 1 --2- 172.21.15.39:0/1164782766 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d3aa9edc30 0x55d3aa9f0040 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:25.597 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.573+0000 7fcaacf5cb80 1 --2- 172.21.15.39:0/1164782766 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3aa9da2d0 0x55d3aa9de360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:25.597 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.573+0000 7fca81ffb700 0 client.16794 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:50:25.597 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.573+0000 7fcaacf5cb80 1 --2- 172.21.15.39:0/1164782766 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d3aa8306b0 0x55d3aa9dd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:25.597 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.573+0000 7fcaacf5cb80 1 --2- 172.21.15.39:0/1164782766 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3aa9d4580 0x55d3aa9dde20 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:25.597 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.573+0000 7fcaacf5cb80 1 -- 172.21.15.39:0/1164782766 >> 172.21.15.39:0/1164782766 conn(0x55d3aa9c9be0 msgr2=0x55d3aa9ce4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:50:25.597 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.573+0000 7fcaacf5cb80 1 -- 172.21.15.39:0/1164782766 shutdown_connections 2023-07-21T18:50:25.597 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.573+0000 7fcaacf5cb80 1 -- 172.21.15.39:0/1164782766 wait complete. 2023-07-21T18:50:25.598 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.573+0000 7fcaacf5cb80 20 client.16794 trim_cache size 0 max 16384 2023-07-21T18:50:25.602 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.588+0000 7fcaacf5cb80 1 Processor -- start 2023-07-21T18:50:25.602 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.588+0000 7fcaacf5cb80 1 -- start start 2023-07-21T18:50:25.602 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.588+0000 7fcaacf5cb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3aa8306b0 0x55d3aa9d4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:25.602 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.588+0000 7fcaacf5cb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d3aa9d4580 0x55d3aa9e4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:25.602 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.588+0000 7fcaacf5cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3aa9da2d0 0x55d3aa9cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:25.602 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.588+0000 7fcaacf5cb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3aa9d99e0 con 0x55d3aa8306b0 2023-07-21T18:50:25.603 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.588+0000 7fcaacf5cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3aa9cffb0 con 0x55d3aa9da2d0 2023-07-21T18:50:25.603 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.588+0000 7fcaacf5cb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3aa9cb5f0 con 0x55d3aa9d4580 2023-07-21T18:50:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.588+0000 7fca9a59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3aa8306b0 0x55d3aa9d4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:25.606 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.588+0000 7fca9a59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3aa8306b0 0x55d3aa9d4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60430/0 (socket says 172.21.15.39:60430) 2023-07-21T18:50:25.606 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.588+0000 7fca9a59c700 1 -- 172.21.15.39:0/1887624061 learned_addr learned my addr 172.21.15.39:0/1887624061 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:50:25.606 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.589+0000 7fca9a59c700 1 -- 172.21.15.39:0/1887624061 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d3aa9d4580 msgr2=0x55d3aa9e4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:50:25.606 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.589+0000 7fca93fff700 1 --2- 172.21.15.39:0/1887624061 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d3aa9d4580 0x55d3aa9e4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:25.606 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.589+0000 7fca9a59c700 1 --2- 172.21.15.39:0/1887624061 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d3aa9d4580 0x55d3aa9e4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:25.606 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.589+0000 7fca9a59c700 1 -- 172.21.15.39:0/1887624061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3aa9da2d0 msgr2=0x55d3aa9cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:50:25.606 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.589+0000 7fca9a59c700 1 --2- 172.21.15.39:0/1887624061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3aa9da2d0 0x55d3aa9cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.589+0000 7fca9a59c700 1 -- 172.21.15.39:0/1887624061 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3aa9dd450 con 0x55d3aa8306b0 2023-07-21T18:50:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.589+0000 7fca9a59c700 1 --2- 172.21.15.39:0/1887624061 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3aa8306b0 0x55d3aa9d4040 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fca8c0ca110 tx=0x7fca8c099640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40ae62d76efde017 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:50:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.589+0000 7fca917fa700 1 -- 172.21.15.39:0/1887624061 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8c097be0 con 0x55d3aa8306b0 2023-07-21T18:50:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.589+0000 7fca917fa700 1 -- 172.21.15.39:0/1887624061 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca8c0960f0 con 0x55d3aa8306b0 2023-07-21T18:50:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.589+0000 7fca917fa700 1 -- 172.21.15.39:0/1887624061 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8c099e80 con 0x55d3aa8306b0 2023-07-21T18:50:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.589+0000 7fcaacf5cb80 1 -- 172.21.15.39:0/1887624061 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3aa8306b0 msgr2=0x55d3aa9d4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:50:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.589+0000 7fcaacf5cb80 1 --2- 172.21.15.39:0/1887624061 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3aa8306b0 0x55d3aa9d4040 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fca8c0ca110 tx=0x7fca8c099640 comp rx=0 tx=0).stop 2023-07-21T18:50:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.590+0000 7fcaacf5cb80 1 -- 172.21.15.39:0/1887624061 shutdown_connections 2023-07-21T18:50:25.608 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.590+0000 7fcaacf5cb80 1 --2- 172.21.15.39:0/1887624061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3aa9da2d0 0x55d3aa9cec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:25.608 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.590+0000 7fcaacf5cb80 1 --2- 172.21.15.39:0/1887624061 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3aa8306b0 0x55d3aa9d4040 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:25.608 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.590+0000 7fcaacf5cb80 1 --2- 172.21.15.39:0/1887624061 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d3aa9d4580 0x55d3aa9e4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:25.608 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.590+0000 7fcaacf5cb80 1 -- 172.21.15.39:0/1887624061 >> 172.21.15.39:0/1887624061 conn(0x55d3aa9c9be0 msgr2=0x55d3aa9d7b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:50:25.608 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.590+0000 7fcaacf5cb80 1 -- 172.21.15.39:0/1887624061 shutdown_connections 2023-07-21T18:50:25.608 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.590+0000 7fcaacf5cb80 1 -- 172.21.15.39:0/1887624061 wait complete. 2023-07-21T18:50:25.608 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.590+0000 7fcaacf5cb80 1 -- start start 2023-07-21T18:50:25.609 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.590+0000 7fcaacf5cb80 1 Processor -- start 2023-07-21T18:50:25.609 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.590+0000 7fcaacf5cb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3aa8306b0 0x55d3aa9d6a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:25.609 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.590+0000 7fcaacf5cb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d3aa9d4580 0x55d3aa9d6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:25.609 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.590+0000 7fcaacf5cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3aa9da2d0 0x55d3aa9d5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:25.609 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.590+0000 7fcaacf5cb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3aa9d7590 con 0x55d3aa8306b0 2023-07-21T18:50:25.609 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.590+0000 7fcaacf5cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3aa9d5da0 con 0x55d3aa9da2d0 2023-07-21T18:50:25.609 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.590+0000 7fcaacf5cb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3aa9d5f60 con 0x55d3aa9d4580 2023-07-21T18:50:25.609 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.590+0000 7fcaacf5cb80 10 client.0 _ll_register_callbacks cb 0x7fca84011c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:50:25.610 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.590+0000 7fca9a59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3aa8306b0 0x55d3aa9d6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:25.610 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.590+0000 7fca9a59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3aa8306b0 0x55d3aa9d6a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60436/0 (socket says 172.21.15.39:60436) 2023-07-21T18:50:25.610 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.590+0000 7fca9a59c700 1 -- 172.21.15.39:0/2756921070 learned_addr learned my addr 172.21.15.39:0/2756921070 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:50:25.610 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.590+0000 7fca93fff700 1 --2- 172.21.15.39:0/2756921070 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d3aa9d4580 0x55d3aa9d6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:25.610 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.591+0000 7fca9ad9d700 1 --2- 172.21.15.39:0/2756921070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3aa9da2d0 0x55d3aa9d5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:25.610 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.591+0000 7fca9a59c700 1 -- 172.21.15.39:0/2756921070 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d3aa9d4580 msgr2=0x55d3aa9d6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:50:25.610 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.591+0000 7fca9a59c700 1 --2- 172.21.15.39:0/2756921070 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d3aa9d4580 0x55d3aa9d6f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:25.610 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.591+0000 7fca9a59c700 1 -- 172.21.15.39:0/2756921070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3aa9da2d0 msgr2=0x55d3aa9d5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:50:25.611 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.591+0000 7fca9a59c700 1 --2- 172.21.15.39:0/2756921070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3aa9da2d0 0x55d3aa9d5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:25.611 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.591+0000 7fca9a59c700 1 -- 172.21.15.39:0/2756921070 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca9c1a2c40 con 0x55d3aa8306b0 2023-07-21T18:50:25.611 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.591+0000 7fca93fff700 1 --2- 172.21.15.39:0/2756921070 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d3aa9d4580 0x55d3aa9d6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:50:25.611 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.591+0000 7fca9ad9d700 1 --2- 172.21.15.39:0/2756921070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3aa9da2d0 0x55d3aa9d5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:50:25.611 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.591+0000 7fca9a59c700 1 --2- 172.21.15.39:0/2756921070 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3aa8306b0 0x55d3aa9d6a30 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fca8c00be80 tx=0x7fca8c097be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c859bbefd53174d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:50:25.611 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.591+0000 7fca81ffb700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:50:25.611 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.591+0000 7fca81ffb700 1 -- 172.21.15.39:0/2756921070 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8c097e40 con 0x55d3aa8306b0 2023-07-21T18:50:25.612 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.591+0000 7fca81ffb700 1 -- 172.21.15.39:0/2756921070 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fca8c098ed0 con 0x55d3aa8306b0 2023-07-21T18:50:25.612 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.591+0000 7fcaacf5cb80 1 -- 172.21.15.39:0/2756921070 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fca84021070 con 0x55d3aa8306b0 2023-07-21T18:50:25.612 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.591+0000 7fca81ffb700 1 -- 172.21.15.39:0/2756921070 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca8c099090 con 0x55d3aa8306b0 2023-07-21T18:50:25.612 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.591+0000 7fcaacf5cb80 10 client.16800 fetch_fsmap learned FSMap version 58 2023-07-21T18:50:25.612 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.591+0000 7fcaacf5cb80 1 -- 172.21.15.39:0/2756921070 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x55d3aa77a1f0 con 0x55d3aa8306b0 2023-07-21T18:50:25.612 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.591+0000 7fca81ffb700 1 -- 172.21.15.39:0/2756921070 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca84021070 con 0x55d3aa8306b0 2023-07-21T18:50:25.612 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.591+0000 7fca81ffb700 1 -- 172.21.15.39:0/2756921070 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 58) v1 ==== 7498+0+0 (secure 0 0 0) 0x7fca8c003bc0 con 0x55d3aa8306b0 2023-07-21T18:50:25.612 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.591+0000 7fcaacf5cb80 10 client.16800 fetch_fsmap finished waiting for FSMap version 58 2023-07-21T18:50:25.613 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.591+0000 7fcaacf5cb80 10 client.16800 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:50:25.613 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.592+0000 7fcaacf5cb80 20 client.16800 populate_metadata read hostname 'smithi039' 2023-07-21T18:50:25.613 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.592+0000 7fcaacf5cb80 1 --2- 172.21.15.39:0/2756921070 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d3aaa42600 0x55d3aaa44a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:25.613 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.592+0000 7fcaacf5cb80 4 client.16800 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:50:25.613 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.592+0000 7fcaacf5cb80 1 -- 172.21.15.39:0/2756921070 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d3aa8e0ff0 con 0x55d3aaa42600 2023-07-21T18:50:25.613 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.592+0000 7fca93fff700 1 --2- 172.21.15.39:0/2756921070 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d3aaa42600 0x55d3aaa44a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:25.614 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.592+0000 7fca81ffb700 10 client.16800 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:50:25.614 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.592+0000 7fca93fff700 1 --2- 172.21.15.39:0/2756921070 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d3aaa42600 0x55d3aaa44a10 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5ab75e4c0e70abc server_cookie=285cb03a98f98d1c in_seq=0 out_seq=0 2023-07-21T18:50:25.647 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.633+0000 7fca81ffb700 1 -- 172.21.15.39:0/2756921070 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d3aa8e0ff0 con 0x55d3aaa42600 2023-07-21T18:50:25.647 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.633+0000 7fca81ffb700 10 client.16800 handle_command_reply: tid=0 2023-07-21T18:50:25.647 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.633+0000 7fcaacf5cb80 1 client.16800 shutdown 2023-07-21T18:50:25.647 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.633+0000 7fcaacf5cb80 1 -- 172.21.15.39:0/2756921070 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3aa8306b0 msgr2=0x55d3aa9d6a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:50:25.648 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.633+0000 7fcaacf5cb80 1 --2- 172.21.15.39:0/2756921070 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3aa8306b0 0x55d3aa9d6a30 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fca8c00be80 tx=0x7fca8c097be0 comp rx=0 tx=0).stop 2023-07-21T18:50:25.648 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.633+0000 7fcaacf5cb80 1 -- 172.21.15.39:0/2756921070 shutdown_connections 2023-07-21T18:50:25.648 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.633+0000 7fcaacf5cb80 1 --2- 172.21.15.39:0/2756921070 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d3aaa42600 0x55d3aaa44a10 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:25.648 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.633+0000 7fcaacf5cb80 1 --2- 172.21.15.39:0/2756921070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3aa9da2d0 0x55d3aa9d5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:25.648 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.633+0000 7fca81ffb700 0 client.16800 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:50:25.648 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.633+0000 7fcaacf5cb80 1 --2- 172.21.15.39:0/2756921070 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3aa8306b0 0x55d3aa9d6a30 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:25.648 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.633+0000 7fcaacf5cb80 1 --2- 172.21.15.39:0/2756921070 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d3aa9d4580 0x55d3aa9d6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:25.649 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.633+0000 7fcaacf5cb80 1 -- 172.21.15.39:0/2756921070 >> 172.21.15.39:0/2756921070 conn(0x55d3aa9c9be0 msgr2=0x55d3aa9d8ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:50:25.649 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.633+0000 7fcaacf5cb80 1 -- 172.21.15.39:0/2756921070 shutdown_connections 2023-07-21T18:50:25.649 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.633+0000 7fcaacf5cb80 1 -- 172.21.15.39:0/2756921070 wait complete. 2023-07-21T18:50:25.649 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.633+0000 7fcaacf5cb80 20 client.16800 trim_cache size 0 max 16384 2023-07-21T18:50:25.649 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:50:25.649 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:50:25.650 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:50:25.650 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:50:25.650 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.634+0000 7fcaa0dd6700 1 -- 172.21.15.39:0/4229648951 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fca880a12c0 msgr2=0x7fca880a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:50:25.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.634+0000 7fcaa0dd6700 1 --2- 172.21.15.39:0/4229648951 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fca880a12c0 0x7fca880a3770 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fca9c0fca20 tx=0x7fca84009260 comp rx=0 tx=0).stop 2023-07-21T18:50:25.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.634+0000 7fcaa0dd6700 1 -- 172.21.15.39:0/4229648951 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca9c0fea30 msgr2=0x7fca9c19bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:50:25.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.634+0000 7fcaa0dd6700 1 --2- 172.21.15.39:0/4229648951 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca9c0fea30 0x7fca9c19bc50 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fca8c00d980 tx=0x7fca8c00dc90 comp rx=0 tx=0).stop 2023-07-21T18:50:25.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.634+0000 7fcaa0dd6700 1 -- 172.21.15.39:0/4229648951 shutdown_connections 2023-07-21T18:50:25.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.634+0000 7fcaa0dd6700 1 --2- 172.21.15.39:0/4229648951 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fca9c104a30 0x7fca9c195ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:25.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.634+0000 7fcaa0dd6700 1 --2- 172.21.15.39:0/4229648951 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca9c0fea30 0x7fca9c19bc50 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:25.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.634+0000 7fcaa0dd6700 1 --2- 172.21.15.39:0/4229648951 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fca880a12c0 0x7fca880a3770 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:25.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.634+0000 7fcaa0dd6700 1 --2- 172.21.15.39:0/4229648951 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca9c0ff3e0 0x7fca9c19c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:25.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.634+0000 7fcaa0dd6700 1 -- 172.21.15.39:0/4229648951 >> 172.21.15.39:0/4229648951 conn(0x7fca9c074760 msgr2=0x7fca9c0fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:50:25.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.634+0000 7fcaa0dd6700 1 -- 172.21.15.39:0/4229648951 shutdown_connections 2023-07-21T18:50:25.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:25.634+0000 7fcaa0dd6700 1 -- 172.21.15.39:0/4229648951 wait complete. 2023-07-21T18:50:26.281 INFO:tasks.workunit.client.0.smithi039.stdout: 1 538754 6.17 MB/sec execute 339 sec latency 2146.352 ms 2023-07-21T18:50:26.428 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:50:26 smithi039 ceph-mon[96489]: pgmap v270: 97 pgs: 97 active+clean; 213 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 1.0 MiB/s wr, 111 op/s 2023-07-21T18:50:26.611 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:50:26 smithi193 ceph-mon[98204]: pgmap v270: 97 pgs: 97 active+clean; 213 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 1.0 MiB/s wr, 111 op/s 2023-07-21T18:50:26.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:50:26 smithi155 ceph-mon[97934]: pgmap v270: 97 pgs: 97 active+clean; 213 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 1.0 MiB/s wr, 111 op/s 2023-07-21T18:50:27.281 INFO:tasks.workunit.client.0.smithi039.stdout: 1 540315 6.19 MB/sec execute 340 sec latency 13.055 ms 2023-07-21T18:50:28.281 INFO:tasks.workunit.client.0.smithi039.stdout: 1 541665 6.19 MB/sec execute 341 sec latency 33.322 ms 2023-07-21T18:50:28.428 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:50:28 smithi039 ceph-mon[96489]: pgmap v271: 97 pgs: 97 active+clean; 217 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 1.6 MiB/s wr, 164 op/s 2023-07-21T18:50:28.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:50:28 smithi193 ceph-mon[98204]: pgmap v271: 97 pgs: 97 active+clean; 217 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 1.6 MiB/s wr, 164 op/s 2023-07-21T18:50:28.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:50:28 smithi155 ceph-mon[97934]: pgmap v271: 97 pgs: 97 active+clean; 217 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 1.6 MiB/s wr, 164 op/s 2023-07-21T18:50:29.281 INFO:tasks.workunit.client.0.smithi039.stdout: 1 541671 6.17 MB/sec execute 342 sec latency 997.083 ms 2023-07-21T18:50:30.281 INFO:tasks.workunit.client.0.smithi039.stdout: 1 541671 6.15 MB/sec execute 343 sec latency 1997.153 ms 2023-07-21T18:50:30.428 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:50:30 smithi039 ceph-mon[96489]: pgmap v272: 97 pgs: 97 active+clean; 217 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 1.1 MiB/s wr, 107 op/s 2023-07-21T18:50:30.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:50:30 smithi193 ceph-mon[98204]: pgmap v272: 97 pgs: 97 active+clean; 217 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 1.1 MiB/s wr, 107 op/s 2023-07-21T18:50:30.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:50:30 smithi155 ceph-mon[97934]: pgmap v272: 97 pgs: 97 active+clean; 217 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 1.1 MiB/s wr, 107 op/s 2023-07-21T18:50:31.281 INFO:tasks.workunit.client.0.smithi039.stdout: 1 541671 6.14 MB/sec execute 344 sec latency 2997.201 ms 2023-07-21T18:50:32.281 INFO:tasks.workunit.client.0.smithi039.stdout: 1 542898 6.14 MB/sec execute 345 sec latency 3432.548 ms 2023-07-21T18:50:32.428 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:50:32 smithi039 ceph-mon[96489]: pgmap v273: 97 pgs: 97 active+clean; 216 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 1.7 MiB/s wr, 171 op/s 2023-07-21T18:50:32.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:50:32 smithi193 ceph-mon[98204]: pgmap v273: 97 pgs: 97 active+clean; 216 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 1.7 MiB/s wr, 171 op/s 2023-07-21T18:50:32.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:50:32 smithi155 ceph-mon[97934]: pgmap v273: 97 pgs: 97 active+clean; 216 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 1.7 MiB/s wr, 171 op/s 2023-07-21T18:50:33.281 INFO:tasks.workunit.client.0.smithi039.stdout: 1 544272 6.14 MB/sec execute 346 sec latency 19.404 ms 2023-07-21T18:50:34.281 INFO:tasks.workunit.client.0.smithi039.stdout: 1 545212 6.14 MB/sec execute 347 sec latency 256.308 ms 2023-07-21T18:50:34.428 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:50:34 smithi039 ceph-mon[96489]: pgmap v274: 97 pgs: 97 active+clean; 216 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 1.1 MiB/s wr, 117 op/s 2023-07-21T18:50:34.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:50:34 smithi193 ceph-mon[98204]: pgmap v274: 97 pgs: 97 active+clean; 216 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 1.1 MiB/s wr, 117 op/s 2023-07-21T18:50:34.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:50:34 smithi155 ceph-mon[97934]: pgmap v274: 97 pgs: 97 active+clean; 216 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 1.1 MiB/s wr, 117 op/s 2023-07-21T18:50:35.281 INFO:tasks.workunit.client.0.smithi039.stdout: 1 545212 6.12 MB/sec execute 348 sec latency 1256.372 ms 2023-07-21T18:50:36.281 INFO:tasks.workunit.client.0.smithi039.stdout: 1 545212 6.10 MB/sec execute 349 sec latency 2256.451 ms 2023-07-21T18:50:36.428 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:50:36 smithi039 ceph-mon[96489]: pgmap v275: 97 pgs: 97 active+clean; 216 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 1.1 MiB/s wr, 117 op/s 2023-07-21T18:50:36.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:50:36 smithi193 ceph-mon[98204]: pgmap v275: 97 pgs: 97 active+clean; 216 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 1.1 MiB/s wr, 117 op/s 2023-07-21T18:50:36.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:50:36 smithi155 ceph-mon[97934]: pgmap v275: 97 pgs: 97 active+clean; 216 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 1.1 MiB/s wr, 117 op/s 2023-07-21T18:50:37.281 INFO:tasks.workunit.client.0.smithi039.stdout: 1 546450 6.11 MB/sec execute 350 sec latency 2688.479 ms 2023-07-21T18:50:38.281 INFO:tasks.workunit.client.0.smithi039.stdout: 1 547859 6.11 MB/sec execute 351 sec latency 11.679 ms 2023-07-21T18:50:38.428 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:50:38 smithi039 ceph-mon[96489]: pgmap v276: 97 pgs: 97 active+clean; 220 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 1.8 MiB/s wr, 173 op/s 2023-07-21T18:50:38.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:50:38 smithi193 ceph-mon[98204]: pgmap v276: 97 pgs: 97 active+clean; 220 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 1.8 MiB/s wr, 173 op/s 2023-07-21T18:50:38.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:50:38 smithi155 ceph-mon[97934]: pgmap v276: 97 pgs: 97 active+clean; 220 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 1.8 MiB/s wr, 173 op/s 2023-07-21T18:50:38.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.763+0000 7f969a5d3700 1 -- 172.21.15.39:0/1428011668 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9694104a30 msgr2=0x7f9694104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:50:38.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.763+0000 7f969a5d3700 1 --2- 172.21.15.39:0/1428011668 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9694104a30 0x7f9694104e00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9684009990 tx=0x7f9684009ca0 comp rx=0 tx=0).stop 2023-07-21T18:50:38.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.763+0000 7f969a5d3700 1 -- 172.21.15.39:0/1428011668 shutdown_connections 2023-07-21T18:50:38.778 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.763+0000 7f969a5d3700 1 --2- 172.21.15.39:0/1428011668 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9694104a30 0x7f9694104e00 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:38.778 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.763+0000 7f969a5d3700 1 --2- 172.21.15.39:0/1428011668 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96940ff3e0 0x7f969410b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:38.778 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.763+0000 7f969a5d3700 1 --2- 172.21.15.39:0/1428011668 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96940fea30 0x7f96940feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:38.778 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.763+0000 7f969a5d3700 1 -- 172.21.15.39:0/1428011668 >> 172.21.15.39:0/1428011668 conn(0x7f9694074760 msgr2=0x7f9694074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:50:38.778 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.763+0000 7f969a5d3700 1 -- 172.21.15.39:0/1428011668 shutdown_connections 2023-07-21T18:50:38.778 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.763+0000 7f969a5d3700 1 -- 172.21.15.39:0/1428011668 wait complete. 2023-07-21T18:50:38.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.764+0000 7f969a5d3700 1 Processor -- start 2023-07-21T18:50:38.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.764+0000 7f969a5d3700 1 -- start start 2023-07-21T18:50:38.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.764+0000 7f969a5d3700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96940fea30 0x7f969410b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:38.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.764+0000 7f969a5d3700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96940ff3e0 0x7f96941063e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:38.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.764+0000 7f969a5d3700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9694104a30 0x7f9694106920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:38.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.764+0000 7f969a5d3700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96941070c0 con 0x7f9694104a30 2023-07-21T18:50:38.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.764+0000 7f969a5d3700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9694107230 con 0x7f96940ff3e0 2023-07-21T18:50:38.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.764+0000 7f969a5d3700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96941a8990 con 0x7f96940fea30 2023-07-21T18:50:38.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.764+0000 7f9698b70700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9694104a30 0x7f9694106920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:38.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.764+0000 7f96937fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96940ff3e0 0x7f96941063e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:38.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.764+0000 7f9698b70700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9694104a30 0x7f9694106920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60454/0 (socket says 172.21.15.39:60454) 2023-07-21T18:50:38.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.764+0000 7f9698b70700 1 -- 172.21.15.39:0/4265323819 learned_addr learned my addr 172.21.15.39:0/4265323819 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:50:38.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.765+0000 7f9693fff700 1 --2- 172.21.15.39:0/4265323819 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96940fea30 0x7f969410b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:38.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.765+0000 7f9698b70700 1 -- 172.21.15.39:0/4265323819 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96940fea30 msgr2=0x7f969410b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:50:38.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.765+0000 7f9698b70700 1 --2- 172.21.15.39:0/4265323819 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96940fea30 0x7f969410b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:38.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.765+0000 7f9698b70700 1 -- 172.21.15.39:0/4265323819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96940ff3e0 msgr2=0x7f96941063e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:50:38.785 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.765+0000 7f9698b70700 1 --2- 172.21.15.39:0/4265323819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96940ff3e0 0x7f96941063e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:38.785 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.765+0000 7f9698b70700 1 -- 172.21.15.39:0/4265323819 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9688009520 con 0x7f9694104a30 2023-07-21T18:50:38.785 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.765+0000 7f96937fe700 1 --2- 172.21.15.39:0/4265323819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96940ff3e0 0x7f96941063e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:50:38.785 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.765+0000 7f9698b70700 1 --2- 172.21.15.39:0/4265323819 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9694104a30 0x7f9694106920 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f968c00c370 tx=0x7f968c00c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:50:38.785 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.765+0000 7f96917fa700 1 -- 172.21.15.39:0/4265323819 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f968c017070 con 0x7f9694104a30 2023-07-21T18:50:38.786 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.765+0000 7f96917fa700 1 -- 172.21.15.39:0/4265323819 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f968c00f040 con 0x7f9694104a30 2023-07-21T18:50:38.786 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.765+0000 7f969a5d3700 1 -- 172.21.15.39:0/4265323819 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f968c009c40 con 0x7f9694104a30 2023-07-21T18:50:38.786 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.765+0000 7f96917fa700 1 -- 172.21.15.39:0/4265323819 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f968c00e050 con 0x7f9694104a30 2023-07-21T18:50:38.786 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.765+0000 7f969a5d3700 1 -- 172.21.15.39:0/4265323819 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96840095f0 con 0x7f9694104a30 2023-07-21T18:50:38.786 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.767+0000 7f96917fa700 1 -- 172.21.15.39:0/4265323819 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f968c00cb60 con 0x7f9694104a30 2023-07-21T18:50:38.793 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.768+0000 7f96917fa700 1 --2- 172.21.15.39:0/4265323819 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f967c0a12c0 0x7f967c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:38.793 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.768+0000 7f9693fff700 1 --2- 172.21.15.39:0/4265323819 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f967c0a12c0 0x7f967c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:38.794 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.768+0000 7f96917fa700 1 -- 172.21.15.39:0/4265323819 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f968c0cc6a0 con 0x7f9694104a30 2023-07-21T18:50:38.794 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.768+0000 7f9693fff700 1 --2- 172.21.15.39:0/4265323819 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f967c0a12c0 0x7f967c0a3770 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f968400b2e0 tx=0x7f9684005fd0 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:50:38.794 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.769+0000 7f96a6759b80 1 Processor -- start 2023-07-21T18:50:38.794 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.769+0000 7f96a6759b80 1 -- start start 2023-07-21T18:50:38.794 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.770+0000 7f96a6759b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f89e4e6b0 0x558f89ff7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:38.794 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.770+0000 7f96a6759b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558f89ff82d0 0x558f89ff2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:38.794 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.770+0000 7f96a6759b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f89ff2580 0x558f89ff29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:38.795 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.770+0000 7f96a6759b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f89ef7c50 con 0x558f89ff2580 2023-07-21T18:50:38.795 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.770+0000 7f96a6759b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f89f4a9e0 con 0x558f89ff82d0 2023-07-21T18:50:38.795 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.770+0000 7f96a6759b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f89f43e70 con 0x558f89e4e6b0 2023-07-21T18:50:38.795 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.770+0000 7f9698b70700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f89ff2580 0x558f89ff29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:38.795 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.770+0000 7f9698b70700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f89ff2580 0x558f89ff29d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60462/0 (socket says 172.21.15.39:60462) 2023-07-21T18:50:38.795 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.770+0000 7f9698b70700 1 -- 172.21.15.39:0/452154692 learned_addr learned my addr 172.21.15.39:0/452154692 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:50:38.795 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.770+0000 7f96937fe700 1 --2- 172.21.15.39:0/452154692 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558f89ff82d0 0x558f89ff2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.770+0000 7f9693fff700 1 --2- 172.21.15.39:0/452154692 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f89e4e6b0 0x558f89ff7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.770+0000 7f9698b70700 1 -- 172.21.15.39:0/452154692 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f89e4e6b0 msgr2=0x558f89ff7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:50:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.770+0000 7f9698b70700 1 --2- 172.21.15.39:0/452154692 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f89e4e6b0 0x558f89ff7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.770+0000 7f9698b70700 1 -- 172.21.15.39:0/452154692 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558f89ff82d0 msgr2=0x558f89ff2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:50:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.770+0000 7f9698b70700 1 --2- 172.21.15.39:0/452154692 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558f89ff82d0 0x558f89ff2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.770+0000 7f9698b70700 1 -- 172.21.15.39:0/452154692 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f89d71d80 con 0x558f89ff2580 2023-07-21T18:50:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.770+0000 7f9698b70700 1 --2- 172.21.15.39:0/452154692 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f89ff2580 0x558f89ff29d0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f968c0090a0 tx=0x7f968c0992b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cc171536ce92dd8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:50:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.770+0000 7f967a7fc700 1 -- 172.21.15.39:0/452154692 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f968c099770 con 0x558f89ff2580 2023-07-21T18:50:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.770+0000 7f967a7fc700 1 -- 172.21.15.39:0/452154692 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f968c09d7a0 con 0x558f89ff2580 2023-07-21T18:50:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.770+0000 7f967a7fc700 1 -- 172.21.15.39:0/452154692 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f968c0192c0 con 0x558f89ff2580 2023-07-21T18:50:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.771+0000 7f96a6759b80 1 -- 172.21.15.39:0/452154692 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f89ff2580 msgr2=0x558f89ff29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:50:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.771+0000 7f96a6759b80 1 --2- 172.21.15.39:0/452154692 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f89ff2580 0x558f89ff29d0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f968c0090a0 tx=0x7f968c0992b0 comp rx=0 tx=0).stop 2023-07-21T18:50:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.771+0000 7f96a6759b80 1 -- 172.21.15.39:0/452154692 shutdown_connections 2023-07-21T18:50:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.771+0000 7f96a6759b80 1 --2- 172.21.15.39:0/452154692 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f89ff2580 0x558f89ff29d0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:38.798 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.771+0000 7f96a6759b80 1 --2- 172.21.15.39:0/452154692 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f89e4e6b0 0x558f89ff7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:38.798 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.771+0000 7f96a6759b80 1 --2- 172.21.15.39:0/452154692 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558f89ff82d0 0x558f89ff2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:38.798 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.771+0000 7f96a6759b80 1 -- 172.21.15.39:0/452154692 >> 172.21.15.39:0/452154692 conn(0x558f89fe7be0 msgr2=0x558f89fecf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:50:38.798 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.771+0000 7f96a6759b80 1 -- 172.21.15.39:0/452154692 shutdown_connections 2023-07-21T18:50:38.798 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.771+0000 7f96a6759b80 1 -- 172.21.15.39:0/452154692 wait complete. 2023-07-21T18:50:38.798 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.772+0000 7f96a6759b80 1 -- start start 2023-07-21T18:50:38.798 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.772+0000 7f96a6759b80 1 Processor -- start 2023-07-21T18:50:38.798 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.772+0000 7f96a6759b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558f89e4e6b0 0x558f89ffb950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:38.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.772+0000 7f96a6759b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f89ff2580 0x558f89ffbe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:38.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.772+0000 7f96a6759b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f89ff82d0 0x558f89ffc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:38.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.772+0000 7f96a6759b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f89ee3a60 con 0x558f89ff2580 2023-07-21T18:50:38.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.772+0000 7f96a6759b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f89ee3760 con 0x558f89e4e6b0 2023-07-21T18:50:38.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.772+0000 7f96a6759b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f89dc78c0 con 0x558f89ff82d0 2023-07-21T18:50:38.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.772+0000 7f96a6759b80 10 client.0 _ll_register_callbacks cb 0x558f89ee1700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:50:38.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.772+0000 7f96937fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f89ff2580 0x558f89ffbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.772+0000 7f96937fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f89ff2580 0x558f89ffbe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60466/0 (socket says 172.21.15.39:60466) 2023-07-21T18:50:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.772+0000 7f96937fe700 1 -- 172.21.15.39:0/4145090366 learned_addr learned my addr 172.21.15.39:0/4145090366 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:50:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.772+0000 7f9693fff700 1 --2- 172.21.15.39:0/4145090366 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558f89e4e6b0 0x558f89ffb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.772+0000 7f9698b70700 1 --2- 172.21.15.39:0/4145090366 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f89ff82d0 0x558f89ffc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.773+0000 7f96937fe700 1 -- 172.21.15.39:0/4145090366 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f89ff82d0 msgr2=0x558f89ffc3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:50:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.773+0000 7f96937fe700 1 --2- 172.21.15.39:0/4145090366 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f89ff82d0 0x558f89ffc3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.773+0000 7f96937fe700 1 -- 172.21.15.39:0/4145090366 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558f89e4e6b0 msgr2=0x558f89ffb950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:50:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.773+0000 7f96937fe700 1 --2- 172.21.15.39:0/4145090366 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558f89e4e6b0 0x558f89ffb950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.773+0000 7f96937fe700 1 -- 172.21.15.39:0/4145090366 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96840095f0 con 0x558f89ff2580 2023-07-21T18:50:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.773+0000 7f9698b70700 1 --2- 172.21.15.39:0/4145090366 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f89ff82d0 0x558f89ffc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:50:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.773+0000 7f96937fe700 1 --2- 172.21.15.39:0/4145090366 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f89ff2580 0x558f89ffbe90 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f9688012cf0 tx=0x7f968800f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d735cbaebcdd519 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:50:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.773+0000 7f96737fe700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:50:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.773+0000 7f96737fe700 1 -- 172.21.15.39:0/4145090366 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9688017070 con 0x558f89ff2580 2023-07-21T18:50:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.773+0000 7f96737fe700 1 -- 172.21.15.39:0/4145090366 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f968800f9c0 con 0x558f89ff2580 2023-07-21T18:50:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.773+0000 7f96a6759b80 1 -- 172.21.15.39:0/4145090366 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558f89e9d490 con 0x558f89ff2580 2023-07-21T18:50:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.773+0000 7f96737fe700 1 -- 172.21.15.39:0/4145090366 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f968800fb80 con 0x558f89ff2580 2023-07-21T18:50:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.773+0000 7f96a6759b80 10 client.16812 fetch_fsmap learned FSMap version 58 2023-07-21T18:50:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.773+0000 7f96a6759b80 1 -- 172.21.15.39:0/4145090366 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x558f89d5ac00 con 0x558f89ff2580 2023-07-21T18:50:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.774+0000 7f96737fe700 1 -- 172.21.15.39:0/4145090366 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558f89e9d490 con 0x558f89ff2580 2023-07-21T18:50:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.774+0000 7f96737fe700 1 -- 172.21.15.39:0/4145090366 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 58) v1 ==== 7498+0+0 (secure 0 0 0) 0x7f96880039c0 con 0x558f89ff2580 2023-07-21T18:50:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.774+0000 7f96a6759b80 10 client.16812 fetch_fsmap finished waiting for FSMap version 58 2023-07-21T18:50:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.774+0000 7f96a6759b80 10 client.16812 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:50:38.803 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.774+0000 7f96a6759b80 20 client.16812 populate_metadata read hostname 'smithi039' 2023-07-21T18:50:38.803 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.774+0000 7f96a6759b80 1 --2- 172.21.15.39:0/4145090366 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x558f8a00bd50 0x558f8a00e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:38.803 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.774+0000 7f96a6759b80 4 client.16812 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:50:38.803 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.774+0000 7f96a6759b80 1 -- 172.21.15.39:0/4145090366 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558f89f3ce50 con 0x558f8a00bd50 2023-07-21T18:50:38.803 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.774+0000 7f9693fff700 1 --2- 172.21.15.39:0/4145090366 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x558f8a00bd50 0x558f8a00e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:38.803 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.774+0000 7f9693fff700 1 --2- 172.21.15.39:0/4145090366 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x558f8a00bd50 0x558f8a00e160 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8403c1741516dc3 server_cookie=66014b9e2613668c in_seq=0 out_seq=0 2023-07-21T18:50:38.803 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.774+0000 7f96737fe700 10 client.16812 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:50:38.803 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.775+0000 7f96737fe700 1 -- 172.21.15.39:0/4145090366 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558f89f3ce50 con 0x558f8a00bd50 2023-07-21T18:50:38.804 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.775+0000 7f96737fe700 10 client.16812 handle_command_reply: tid=0 2023-07-21T18:50:38.804 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.775+0000 7f96a6759b80 1 client.16812 shutdown 2023-07-21T18:50:38.804 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.775+0000 7f96a6759b80 1 -- 172.21.15.39:0/4145090366 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f89ff2580 msgr2=0x558f89ffbe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:50:38.804 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.775+0000 7f96a6759b80 1 --2- 172.21.15.39:0/4145090366 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f89ff2580 0x558f89ffbe90 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f9688012cf0 tx=0x7f968800f3c0 comp rx=0 tx=0).stop 2023-07-21T18:50:38.804 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.775+0000 7f96a6759b80 1 -- 172.21.15.39:0/4145090366 shutdown_connections 2023-07-21T18:50:38.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.775+0000 7f96a6759b80 1 --2- 172.21.15.39:0/4145090366 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x558f8a00bd50 0x558f8a00e160 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:38.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.775+0000 7f96a6759b80 1 --2- 172.21.15.39:0/4145090366 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558f89e4e6b0 0x558f89ffb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:38.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.775+0000 7f96a6759b80 1 --2- 172.21.15.39:0/4145090366 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f89ff82d0 0x558f89ffc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:38.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.775+0000 7f96a6759b80 1 --2- 172.21.15.39:0/4145090366 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f89ff2580 0x558f89ffbe90 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:38.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.775+0000 7f96737fe700 0 client.16812 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:50:38.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.775+0000 7f96a6759b80 1 -- 172.21.15.39:0/4145090366 >> 172.21.15.39:0/4145090366 conn(0x558f89fe7be0 msgr2=0x558f89fec4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:50:38.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.775+0000 7f96a6759b80 1 -- 172.21.15.39:0/4145090366 shutdown_connections 2023-07-21T18:50:38.806 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.775+0000 7f96a6759b80 1 -- 172.21.15.39:0/4145090366 wait complete. 2023-07-21T18:50:38.806 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.775+0000 7f96a6759b80 20 client.16812 trim_cache size 0 max 16384 2023-07-21T18:50:38.809 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.791+0000 7f96a6759b80 1 Processor -- start 2023-07-21T18:50:38.810 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.791+0000 7f96a6759b80 1 -- start start 2023-07-21T18:50:38.810 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.791+0000 7f96a6759b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f89e4e6b0 0x558f89ff2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:38.810 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.791+0000 7f96a6759b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558f89ff2580 0x558f89fecc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:38.810 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.791+0000 7f96a6759b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f89ff82d0 0x558f89ffd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:38.810 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.791+0000 7f96a6759b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f89fedea0 con 0x558f89e4e6b0 2023-07-21T18:50:38.810 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.791+0000 7f96a6759b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f89ff9eb0 con 0x558f89ff2580 2023-07-21T18:50:38.810 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.791+0000 7f96a6759b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f89ff66e0 con 0x558f89ff82d0 2023-07-21T18:50:38.811 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.791+0000 7f9693fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f89e4e6b0 0x558f89ff2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:38.811 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.791+0000 7f9693fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f89e4e6b0 0x558f89ff2040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60472/0 (socket says 172.21.15.39:60472) 2023-07-21T18:50:38.811 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.791+0000 7f9693fff700 1 -- 172.21.15.39:0/1039630399 learned_addr learned my addr 172.21.15.39:0/1039630399 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:50:38.811 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.791+0000 7f96937fe700 1 --2- 172.21.15.39:0/1039630399 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558f89ff2580 0x558f89fecc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:38.811 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.791+0000 7f9698b70700 1 --2- 172.21.15.39:0/1039630399 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f89ff82d0 0x558f89ffd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:38.811 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.791+0000 7f9693fff700 1 -- 172.21.15.39:0/1039630399 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f89ff82d0 msgr2=0x558f89ffd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:50:38.811 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.791+0000 7f9693fff700 1 --2- 172.21.15.39:0/1039630399 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f89ff82d0 0x558f89ffd260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:38.811 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.791+0000 7f9693fff700 1 -- 172.21.15.39:0/1039630399 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558f89ff2580 msgr2=0x558f89fecc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:50:38.812 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.791+0000 7f9693fff700 1 --2- 172.21.15.39:0/1039630399 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558f89ff2580 0x558f89fecc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:38.812 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.791+0000 7f9693fff700 1 -- 172.21.15.39:0/1039630399 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f89d981f0 con 0x558f89e4e6b0 2023-07-21T18:50:38.812 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.791+0000 7f9698b70700 1 --2- 172.21.15.39:0/1039630399 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f89ff82d0 0x558f89ffd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:50:38.812 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.792+0000 7f9693fff700 1 --2- 172.21.15.39:0/1039630399 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f89e4e6b0 0x558f89ff2040 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f9684009990 tx=0x7f9684023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1b673991be7684b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:50:38.812 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.792+0000 7f967affd700 1 -- 172.21.15.39:0/1039630399 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9684034a50 con 0x558f89e4e6b0 2023-07-21T18:50:38.812 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.792+0000 7f967affd700 1 -- 172.21.15.39:0/1039630399 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f968402c040 con 0x558f89e4e6b0 2023-07-21T18:50:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.792+0000 7f967affd700 1 -- 172.21.15.39:0/1039630399 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9684004760 con 0x558f89e4e6b0 2023-07-21T18:50:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.793+0000 7f96a6759b80 1 -- 172.21.15.39:0/1039630399 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f89e4e6b0 msgr2=0x558f89ff2040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:50:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.793+0000 7f96a6759b80 1 --2- 172.21.15.39:0/1039630399 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f89e4e6b0 0x558f89ff2040 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f9684009990 tx=0x7f9684023000 comp rx=0 tx=0).stop 2023-07-21T18:50:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.793+0000 7f96a6759b80 1 -- 172.21.15.39:0/1039630399 shutdown_connections 2023-07-21T18:50:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.793+0000 7f96a6759b80 1 --2- 172.21.15.39:0/1039630399 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f89ff82d0 0x558f89ffd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.793+0000 7f96a6759b80 1 --2- 172.21.15.39:0/1039630399 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f89e4e6b0 0x558f89ff2040 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.793+0000 7f96a6759b80 1 --2- 172.21.15.39:0/1039630399 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558f89ff2580 0x558f89fecc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.793+0000 7f96a6759b80 1 -- 172.21.15.39:0/1039630399 >> 172.21.15.39:0/1039630399 conn(0x558f89fe7be0 msgr2=0x558f89ff5b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:50:38.814 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.793+0000 7f96a6759b80 1 -- 172.21.15.39:0/1039630399 shutdown_connections 2023-07-21T18:50:38.814 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.793+0000 7f96a6759b80 1 -- 172.21.15.39:0/1039630399 wait complete. 2023-07-21T18:50:38.814 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.793+0000 7f96a6759b80 1 -- start start 2023-07-21T18:50:38.814 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.793+0000 7f96a6759b80 1 Processor -- start 2023-07-21T18:50:38.814 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.794+0000 7f96a6759b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f89e4e6b0 0x558f8a0587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:38.814 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.794+0000 7f96a6759b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f89ff2580 0x558f8a058d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:38.814 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.794+0000 7f96a6759b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558f89ff82d0 0x558f8a0022f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:38.814 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.794+0000 7f96a6759b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f8a002900 con 0x558f89e4e6b0 2023-07-21T18:50:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.794+0000 7f96a6759b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f8a002a70 con 0x558f89ff82d0 2023-07-21T18:50:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.794+0000 7f96a6759b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f8a002c30 con 0x558f89ff2580 2023-07-21T18:50:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.794+0000 7f96a6759b80 10 client.0 _ll_register_callbacks cb 0x7f96880127f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:50:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.794+0000 7f9693fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f89e4e6b0 0x558f8a0587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.794+0000 7f9693fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f89e4e6b0 0x558f8a0587d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60478/0 (socket says 172.21.15.39:60478) 2023-07-21T18:50:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.794+0000 7f96937fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f89ff2580 0x558f8a058d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.794+0000 7f9693fff700 1 -- 172.21.15.39:0/2242172073 learned_addr learned my addr 172.21.15.39:0/2242172073 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:50:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.794+0000 7f9698b70700 1 --2- 172.21.15.39:0/2242172073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558f89ff82d0 0x558f8a0022f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.794+0000 7f96937fe700 1 -- 172.21.15.39:0/2242172073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558f89ff82d0 msgr2=0x558f8a0022f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:50:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.794+0000 7f96937fe700 1 --2- 172.21.15.39:0/2242172073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558f89ff82d0 0x558f8a0022f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.794+0000 7f96937fe700 1 -- 172.21.15.39:0/2242172073 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f89e4e6b0 msgr2=0x558f8a0587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:50:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.794+0000 7f96937fe700 1 --2- 172.21.15.39:0/2242172073 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f89e4e6b0 0x558f8a0587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.794+0000 7f96937fe700 1 -- 172.21.15.39:0/2242172073 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f969410c160 con 0x558f89ff2580 2023-07-21T18:50:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.794+0000 7f9693fff700 1 --2- 172.21.15.39:0/2242172073 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f89e4e6b0 0x558f8a0587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:50:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.794+0000 7f96937fe700 1 --2- 172.21.15.39:0/2242172073 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f89ff2580 0x558f8a058d10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9688003e20 tx=0x7f96880092d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d3ba05e164e2d76 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:50:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.794+0000 7f96737fe700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2023-07-21T18:50:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.795+0000 7f96737fe700 1 -- 172.21.15.39:0/2242172073 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9688017030 con 0x558f89ff2580 2023-07-21T18:50:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.795+0000 7f96737fe700 1 -- 172.21.15.39:0/2242172073 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f968800fbc0 con 0x558f89ff2580 2023-07-21T18:50:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.795+0000 7f96a6759b80 1 -- 172.21.15.39:0/2242172073 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f968c09d7a0 con 0x558f89ff2580 2023-07-21T18:50:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.795+0000 7f96737fe700 1 -- 172.21.15.39:0/2242172073 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9688009340 con 0x558f89ff2580 2023-07-21T18:50:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.795+0000 7f96a6759b80 10 client.26296 fetch_fsmap learned FSMap version 58 2023-07-21T18:50:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.795+0000 7f96a6759b80 1 -- 172.21.15.39:0/2242172073 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x558f89d70250 con 0x558f89ff2580 2023-07-21T18:50:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.795+0000 7f96737fe700 1 -- 172.21.15.39:0/2242172073 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f968803a020 con 0x558f89ff2580 2023-07-21T18:50:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.795+0000 7f96737fe700 1 -- 172.21.15.39:0/2242172073 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 58) v1 ==== 7498+0+0 (secure 0 0 0) 0x7f96880107f0 con 0x558f89ff2580 2023-07-21T18:50:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.795+0000 7f96a6759b80 10 client.26296 fetch_fsmap finished waiting for FSMap version 58 2023-07-21T18:50:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.795+0000 7f96a6759b80 10 client.26296 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:50:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.795+0000 7f96a6759b80 20 client.26296 populate_metadata read hostname 'smithi039' 2023-07-21T18:50:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.795+0000 7f96a6759b80 1 --2- 172.21.15.39:0/2242172073 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x558f8a060620 0x558f8a062a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.795+0000 7f96a6759b80 4 client.26296 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:50:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.795+0000 7f96a6759b80 1 -- 172.21.15.39:0/2242172073 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558f89efeff0 con 0x558f8a060620 2023-07-21T18:50:38.819 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.795+0000 7f9693fff700 1 --2- 172.21.15.39:0/2242172073 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x558f8a060620 0x558f8a062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:38.819 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.796+0000 7f96737fe700 10 client.26296 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:50:38.819 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.796+0000 7f9693fff700 1 --2- 172.21.15.39:0/2242172073 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x558f8a060620 0x558f8a062a30 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4502d79a67517c4 server_cookie=a2b7c959152b72d3 in_seq=0 out_seq=0 2023-07-21T18:50:38.840 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.826+0000 7f96737fe700 1 -- 172.21.15.39:0/2242172073 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558f89efeff0 con 0x558f8a060620 2023-07-21T18:50:38.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.826+0000 7f96737fe700 10 client.26296 handle_command_reply: tid=0 2023-07-21T18:50:38.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.826+0000 7f96a6759b80 1 client.26296 shutdown 2023-07-21T18:50:38.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.826+0000 7f96a6759b80 1 -- 172.21.15.39:0/2242172073 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f89ff2580 msgr2=0x558f8a058d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:50:38.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.826+0000 7f96a6759b80 1 --2- 172.21.15.39:0/2242172073 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f89ff2580 0x558f8a058d10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9688003e20 tx=0x7f96880092d0 comp rx=0 tx=0).stop 2023-07-21T18:50:38.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.827+0000 7f96a6759b80 1 -- 172.21.15.39:0/2242172073 shutdown_connections 2023-07-21T18:50:38.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.827+0000 7f96a6759b80 1 --2- 172.21.15.39:0/2242172073 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x558f8a060620 0x558f8a062a30 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:38.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.827+0000 7f96a6759b80 1 --2- 172.21.15.39:0/2242172073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558f89ff82d0 0x558f8a0022f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:38.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.827+0000 7f96a6759b80 1 --2- 172.21.15.39:0/2242172073 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f89e4e6b0 0x558f8a0587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:38.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.827+0000 7f96a6759b80 1 --2- 172.21.15.39:0/2242172073 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558f89ff2580 0x558f8a058d10 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:38.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.827+0000 7f96737fe700 0 client.26296 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:50:38.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.827+0000 7f96a6759b80 1 -- 172.21.15.39:0/2242172073 >> 172.21.15.39:0/2242172073 conn(0x558f89fe7be0 msgr2=0x558f89ff8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:50:38.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.827+0000 7f96a6759b80 1 -- 172.21.15.39:0/2242172073 shutdown_connections 2023-07-21T18:50:38.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.827+0000 7f96a6759b80 1 -- 172.21.15.39:0/2242172073 wait complete. 2023-07-21T18:50:38.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.827+0000 7f96a6759b80 20 client.26296 trim_cache size 0 max 16384 2023-07-21T18:50:38.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:50:38.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:50:38.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:50:38.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:50:38.844 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.827+0000 7f969a5d3700 1 -- 172.21.15.39:0/4265323819 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f967c0a12c0 msgr2=0x7f967c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:50:38.844 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.827+0000 7f969a5d3700 1 --2- 172.21.15.39:0/4265323819 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f967c0a12c0 0x7f967c0a3770 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f968400b2e0 tx=0x7f9684005fd0 comp rx=0 tx=0).stop 2023-07-21T18:50:38.844 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.827+0000 7f969a5d3700 1 -- 172.21.15.39:0/4265323819 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9694104a30 msgr2=0x7f9694106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:50:38.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.827+0000 7f969a5d3700 1 --2- 172.21.15.39:0/4265323819 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9694104a30 0x7f9694106920 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f968c00c370 tx=0x7f968c00c680 comp rx=0 tx=0).stop 2023-07-21T18:50:38.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.827+0000 7f969a5d3700 1 -- 172.21.15.39:0/4265323819 shutdown_connections 2023-07-21T18:50:38.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.827+0000 7f969a5d3700 1 --2- 172.21.15.39:0/4265323819 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9694104a30 0x7f9694106920 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:38.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.827+0000 7f969a5d3700 1 --2- 172.21.15.39:0/4265323819 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96940fea30 0x7f969410b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:38.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.827+0000 7f969a5d3700 1 --2- 172.21.15.39:0/4265323819 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f967c0a12c0 0x7f967c0a3770 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:38.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.827+0000 7f969a5d3700 1 --2- 172.21.15.39:0/4265323819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96940ff3e0 0x7f96941063e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:38.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.827+0000 7f969a5d3700 1 -- 172.21.15.39:0/4265323819 >> 172.21.15.39:0/4265323819 conn(0x7f9694074760 msgr2=0x7f96940fd030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:50:38.846 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.827+0000 7f969a5d3700 1 -- 172.21.15.39:0/4265323819 shutdown_connections 2023-07-21T18:50:38.846 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:38.827+0000 7f969a5d3700 1 -- 172.21.15.39:0/4265323819 wait complete. 2023-07-21T18:50:39.282 INFO:tasks.workunit.client.0.smithi039.stdout: 1 548762 6.10 MB/sec execute 352 sec latency 238.572 ms 2023-07-21T18:50:40.282 INFO:tasks.workunit.client.0.smithi039.stdout: 1 548762 6.09 MB/sec execute 353 sec latency 1238.668 ms 2023-07-21T18:50:40.428 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:50:40 smithi039 ceph-mon[96489]: pgmap v277: 97 pgs: 97 active+clean; 220 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 1.2 MiB/s wr, 120 op/s 2023-07-21T18:50:40.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:50:40 smithi193 ceph-mon[98204]: pgmap v277: 97 pgs: 97 active+clean; 220 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 1.2 MiB/s wr, 120 op/s 2023-07-21T18:50:40.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:50:40 smithi155 ceph-mon[97934]: pgmap v277: 97 pgs: 97 active+clean; 220 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 1.2 MiB/s wr, 120 op/s 2023-07-21T18:50:41.282 INFO:tasks.workunit.client.0.smithi039.stdout: 1 548762 6.07 MB/sec execute 354 sec latency 2238.789 ms 2023-07-21T18:50:42.282 INFO:tasks.workunit.client.0.smithi039.stdout: 1 549998 6.07 MB/sec execute 355 sec latency 2673.813 ms 2023-07-21T18:50:42.428 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:50:42 smithi039 ceph-mon[96489]: pgmap v278: 97 pgs: 97 active+clean; 219 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 1.9 MiB/s wr, 187 op/s 2023-07-21T18:50:42.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:50:42 smithi193 ceph-mon[98204]: pgmap v278: 97 pgs: 97 active+clean; 219 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 1.9 MiB/s wr, 187 op/s 2023-07-21T18:50:42.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:50:42 smithi155 ceph-mon[97934]: pgmap v278: 97 pgs: 97 active+clean; 219 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 1.9 MiB/s wr, 187 op/s 2023-07-21T18:50:43.282 INFO:tasks.workunit.client.0.smithi039.stdout: 1 551351 6.08 MB/sec execute 356 sec latency 15.292 ms 2023-07-21T18:50:44.282 INFO:tasks.workunit.client.0.smithi039.stdout: 1 552305 6.07 MB/sec execute 357 sec latency 435.552 ms 2023-07-21T18:50:44.428 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:50:44 smithi039 ceph-mon[96489]: pgmap v279: 97 pgs: 97 active+clean; 219 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 1.3 MiB/s wr, 123 op/s 2023-07-21T18:50:44.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:50:44 smithi193 ceph-mon[98204]: pgmap v279: 97 pgs: 97 active+clean; 219 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 1.3 MiB/s wr, 123 op/s 2023-07-21T18:50:44.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:50:44 smithi155 ceph-mon[97934]: pgmap v279: 97 pgs: 97 active+clean; 219 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 1.3 MiB/s wr, 123 op/s 2023-07-21T18:50:45.282 INFO:tasks.workunit.client.0.smithi039.stdout: 1 552305 6.06 MB/sec execute 358 sec latency 1435.655 ms 2023-07-21T18:50:46.282 INFO:tasks.workunit.client.0.smithi039.stdout: 1 552927 6.04 MB/sec execute 359 sec latency 2235.356 ms 2023-07-21T18:50:46.428 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:50:46 smithi039 ceph-mon[96489]: pgmap v280: 97 pgs: 97 active+clean; 219 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 1.3 MiB/s wr, 123 op/s 2023-07-21T18:50:46.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:50:46 smithi193 ceph-mon[98204]: pgmap v280: 97 pgs: 97 active+clean; 219 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 1.3 MiB/s wr, 123 op/s 2023-07-21T18:50:46.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:50:46 smithi155 ceph-mon[97934]: pgmap v280: 97 pgs: 97 active+clean; 219 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 1.3 MiB/s wr, 123 op/s 2023-07-21T18:50:47.282 INFO:tasks.workunit.client.0.smithi039.stdout: 1 554534 6.06 MB/sec execute 360 sec latency 11.939 ms 2023-07-21T18:50:48.282 INFO:tasks.workunit.client.0.smithi039.stdout: 1 555754 6.06 MB/sec execute 361 sec latency 18.906 ms 2023-07-21T18:50:48.428 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:50:48 smithi039 ceph-mon[96489]: pgmap v281: 97 pgs: 97 active+clean; 222 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 1.9 MiB/s wr, 178 op/s 2023-07-21T18:50:48.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:50:48 smithi193 ceph-mon[98204]: pgmap v281: 97 pgs: 97 active+clean; 222 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 1.9 MiB/s wr, 178 op/s 2023-07-21T18:50:48.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:50:48 smithi155 ceph-mon[97934]: pgmap v281: 97 pgs: 97 active+clean; 222 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 1.9 MiB/s wr, 178 op/s 2023-07-21T18:50:49.282 INFO:tasks.workunit.client.0.smithi039.stdout: 1 555852 6.04 MB/sec execute 362 sec latency 892.117 ms 2023-07-21T18:50:49.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:50:49 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:50:49.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:50:49 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:50:49.678 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:50:49 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:50:50.282 INFO:tasks.workunit.client.0.smithi039.stdout: 1 555852 6.03 MB/sec execute 363 sec latency 1892.199 ms 2023-07-21T18:50:50.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:50:50 smithi193 ceph-mon[98204]: pgmap v282: 97 pgs: 97 active+clean; 222 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 1.3 MiB/s wr, 122 op/s 2023-07-21T18:50:50.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:50:50 smithi155 ceph-mon[97934]: pgmap v282: 97 pgs: 97 active+clean; 222 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 1.3 MiB/s wr, 122 op/s 2023-07-21T18:50:50.678 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:50:50 smithi039 ceph-mon[96489]: pgmap v282: 97 pgs: 97 active+clean; 222 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 1.3 MiB/s wr, 122 op/s 2023-07-21T18:50:51.282 INFO:tasks.workunit.client.0.smithi039.stdout: 1 555852 6.01 MB/sec execute 364 sec latency 2892.312 ms 2023-07-21T18:50:51.959 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.946+0000 7facfa055700 1 -- 172.21.15.39:0/1316229732 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7facf40ff3e0 msgr2=0x7facf410b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:50:51.959 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.946+0000 7facfa055700 1 --2- 172.21.15.39:0/1316229732 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7facf40ff3e0 0x7facf410b860 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7facec00b0a0 tx=0x7facec00b3b0 comp rx=0 tx=0).stop 2023-07-21T18:50:51.959 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.946+0000 7facfa055700 1 -- 172.21.15.39:0/1316229732 shutdown_connections 2023-07-21T18:50:51.959 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.946+0000 7facfa055700 1 --2- 172.21.15.39:0/1316229732 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7facf4104a30 0x7facf4104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:51.960 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.946+0000 7facfa055700 1 --2- 172.21.15.39:0/1316229732 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7facf40ff3e0 0x7facf410b860 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:51.960 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.946+0000 7facfa055700 1 --2- 172.21.15.39:0/1316229732 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7facf40fea30 0x7facf40feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:51.960 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.946+0000 7facfa055700 1 -- 172.21.15.39:0/1316229732 >> 172.21.15.39:0/1316229732 conn(0x7facf4074760 msgr2=0x7facf4074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:50:51.960 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.946+0000 7facfa055700 1 -- 172.21.15.39:0/1316229732 shutdown_connections 2023-07-21T18:50:51.960 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.946+0000 7facfa055700 1 -- 172.21.15.39:0/1316229732 wait complete. 2023-07-21T18:50:51.962 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.947+0000 7facfa055700 1 Processor -- start 2023-07-21T18:50:51.962 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.947+0000 7facfa055700 1 -- start start 2023-07-21T18:50:51.962 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.947+0000 7facfa055700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7facf40fea30 0x7facf4196ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:51.962 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.947+0000 7facfa055700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7facf40ff3e0 0x7facf41971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:51.962 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.947+0000 7facfa055700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7facf4104a30 0x7facf41a1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:51.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.947+0000 7facfa055700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf41a2110 con 0x7facf40ff3e0 2023-07-21T18:50:51.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.947+0000 7facfa055700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf41a2280 con 0x7facf4104a30 2023-07-21T18:50:51.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.947+0000 7facfa055700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf41a2440 con 0x7facf40fea30 2023-07-21T18:50:51.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.948+0000 7facf37fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7facf40fea30 0x7facf4196ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:51.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.948+0000 7facf2ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7facf40ff3e0 0x7facf41971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:51.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.948+0000 7facf37fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7facf40fea30 0x7facf4196ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:50812/0 (socket says 172.21.15.39:50812) 2023-07-21T18:50:51.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.948+0000 7facf37fe700 1 -- 172.21.15.39:0/947695233 learned_addr learned my addr 172.21.15.39:0/947695233 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:50:51.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.948+0000 7facf3fff700 1 --2- 172.21.15.39:0/947695233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7facf4104a30 0x7facf41a1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:51.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.948+0000 7facf37fe700 1 -- 172.21.15.39:0/947695233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7facf4104a30 msgr2=0x7facf41a1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:50:51.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.948+0000 7facf37fe700 1 --2- 172.21.15.39:0/947695233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7facf4104a30 0x7facf41a1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:51.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.948+0000 7facf37fe700 1 -- 172.21.15.39:0/947695233 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7facf40ff3e0 msgr2=0x7facf41971e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:50:51.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.948+0000 7facf37fe700 1 --2- 172.21.15.39:0/947695233 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7facf40ff3e0 0x7facf41971e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:51.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.948+0000 7facf37fe700 1 -- 172.21.15.39:0/947695233 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facec009d00 con 0x7facf40fea30 2023-07-21T18:50:51.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.948+0000 7facf2ffd700 1 --2- 172.21.15.39:0/947695233 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7facf40ff3e0 0x7facf41971e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:50:51.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.948+0000 7facf37fe700 1 --2- 172.21.15.39:0/947695233 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7facf40fea30 0x7facf4196ca0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7face800e970 tx=0x7face800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:50:51.965 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.948+0000 7facf0ff9700 1 -- 172.21.15.39:0/947695233 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face8009790 con 0x7facf40fea30 2023-07-21T18:50:51.965 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.948+0000 7facfa055700 1 -- 172.21.15.39:0/947695233 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facf41a2720 con 0x7facf40fea30 2023-07-21T18:50:51.965 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.948+0000 7facf0ff9700 1 -- 172.21.15.39:0/947695233 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7face800c7e0 con 0x7facf40fea30 2023-07-21T18:50:51.965 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.948+0000 7facf0ff9700 1 -- 172.21.15.39:0/947695233 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face8003c70 con 0x7facf40fea30 2023-07-21T18:50:51.965 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.948+0000 7facfa055700 1 -- 172.21.15.39:0/947695233 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facf41a2c80 con 0x7facf40fea30 2023-07-21T18:50:51.967 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.951+0000 7facf0ff9700 1 -- 172.21.15.39:0/947695233 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7face800cc50 con 0x7facf40fea30 2023-07-21T18:50:51.967 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.952+0000 7facf0ff9700 1 --2- 172.21.15.39:0/947695233 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7facdc0a12c0 0x7facdc0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:51.967 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.952+0000 7facf0ff9700 1 -- 172.21.15.39:0/947695233 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8527+0+0 (secure 0 0 0) 0x7face8014070 con 0x7facf40fea30 2023-07-21T18:50:51.967 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.952+0000 7facf2ffd700 1 --2- 172.21.15.39:0/947695233 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7facdc0a12c0 0x7facdc0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.952+0000 7facf2ffd700 1 --2- 172.21.15.39:0/947695233 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7facdc0a12c0 0x7facdc0a3770 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7facf40689e0 tx=0x7face4009260 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:50:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.953+0000 7fad061dbb80 1 Processor -- start 2023-07-21T18:50:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.953+0000 7fad061dbb80 1 -- start start 2023-07-21T18:50:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.953+0000 7fad061dbb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564febafe6b0 0x564febca7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.953+0000 7fad061dbb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564febca82d0 0x564febca2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.953+0000 7fad061dbb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564febca2580 0x564febca29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.953+0000 7fad061dbb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x564febba7c50 con 0x564febca82d0 2023-07-21T18:50:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.953+0000 7fad061dbb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564febbfa9e0 con 0x564febafe6b0 2023-07-21T18:50:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.953+0000 7fad061dbb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x564febbf3e70 con 0x564febca2580 2023-07-21T18:50:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.953+0000 7facf2ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564febca82d0 0x564febca2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.953+0000 7facf37fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564febafe6b0 0x564febca7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.953+0000 7facf37fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564febafe6b0 0x564febca7d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:44864/0 (socket says 172.21.15.39:44864) 2023-07-21T18:50:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.953+0000 7facf3fff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564febca2580 0x564febca29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.953+0000 7facf2ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564febca82d0 0x564febca2040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60502/0 (socket says 172.21.15.39:60502) 2023-07-21T18:50:51.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.953+0000 7facf37fe700 1 -- 172.21.15.39:0/1026804310 learned_addr learned my addr 172.21.15.39:0/1026804310 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:50:51.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.953+0000 7facf37fe700 1 -- 172.21.15.39:0/1026804310 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564febca2580 msgr2=0x564febca29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:50:51.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.953+0000 7facf37fe700 1 --2- 172.21.15.39:0/1026804310 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564febca2580 0x564febca29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:51.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.953+0000 7facf37fe700 1 -- 172.21.15.39:0/1026804310 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564febca82d0 msgr2=0x564febca2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:50:51.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.953+0000 7facf37fe700 1 --2- 172.21.15.39:0/1026804310 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564febca82d0 0x564febca2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:51.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.953+0000 7facf37fe700 1 -- 172.21.15.39:0/1026804310 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564feba21d80 con 0x564febafe6b0 2023-07-21T18:50:51.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.953+0000 7facf2ffd700 1 --2- 172.21.15.39:0/1026804310 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564febca82d0 0x564febca2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:50:51.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.954+0000 7facf37fe700 1 --2- 172.21.15.39:0/1026804310 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564febafe6b0 0x564febca7d90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7face80a1ea0 tx=0x7face80999d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2466ba7f9721cffe server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:50:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.954+0000 7facd9ffb700 1 -- 172.21.15.39:0/1026804310 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face809dfb0 con 0x564febafe6b0 2023-07-21T18:50:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.954+0000 7facd9ffb700 1 -- 172.21.15.39:0/1026804310 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7face80a23d0 con 0x564febafe6b0 2023-07-21T18:50:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.954+0000 7facd9ffb700 1 -- 172.21.15.39:0/1026804310 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face809a440 con 0x564febafe6b0 2023-07-21T18:50:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.954+0000 7fad061dbb80 1 -- 172.21.15.39:0/1026804310 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564febafe6b0 msgr2=0x564febca7d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:50:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.954+0000 7fad061dbb80 1 --2- 172.21.15.39:0/1026804310 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564febafe6b0 0x564febca7d90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7face80a1ea0 tx=0x7face80999d0 comp rx=0 tx=0).stop 2023-07-21T18:50:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.955+0000 7fad061dbb80 1 -- 172.21.15.39:0/1026804310 shutdown_connections 2023-07-21T18:50:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.955+0000 7fad061dbb80 1 --2- 172.21.15.39:0/1026804310 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564febafe6b0 0x564febca7d90 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.955+0000 7fad061dbb80 1 --2- 172.21.15.39:0/1026804310 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564febca2580 0x564febca29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.955+0000 7fad061dbb80 1 --2- 172.21.15.39:0/1026804310 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564febca82d0 0x564febca2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.955+0000 7fad061dbb80 1 -- 172.21.15.39:0/1026804310 >> 172.21.15.39:0/1026804310 conn(0x564febc97be0 msgr2=0x564febc9cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:50:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.955+0000 7fad061dbb80 1 -- 172.21.15.39:0/1026804310 shutdown_connections 2023-07-21T18:50:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.955+0000 7fad061dbb80 1 -- 172.21.15.39:0/1026804310 wait complete. 2023-07-21T18:50:51.975 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.955+0000 7fad061dbb80 1 -- start start 2023-07-21T18:50:51.975 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.955+0000 7fad061dbb80 1 Processor -- start 2023-07-21T18:50:51.975 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.956+0000 7fad061dbb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564febafe6b0 0x564febcae5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:51.975 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.956+0000 7fad061dbb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564febca2580 0x564febcab840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:51.975 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.956+0000 7fad061dbb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564febca82d0 0x564febcabee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.956+0000 7fad061dbb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x564febb93a60 con 0x564febca82d0 2023-07-21T18:50:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.956+0000 7fad061dbb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564febb93760 con 0x564febafe6b0 2023-07-21T18:50:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.956+0000 7fad061dbb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x564feba778c0 con 0x564febca2580 2023-07-21T18:50:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.956+0000 7fad061dbb80 10 client.0 _ll_register_callbacks cb 0x564febb91700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:50:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.956+0000 7facf3fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564febca82d0 0x564febcabee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.956+0000 7facf37fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564febafe6b0 0x564febcae5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.956+0000 7facf3fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564febca82d0 0x564febcabee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60508/0 (socket says 172.21.15.39:60508) 2023-07-21T18:50:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.956+0000 7facf3fff700 1 -- 172.21.15.39:0/428555113 learned_addr learned my addr 172.21.15.39:0/428555113 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:50:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.956+0000 7facf37fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564febafe6b0 0x564febcae5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:44870/0 (socket says 172.21.15.39:44870) 2023-07-21T18:50:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.956+0000 7facf2ffd700 1 --2- 172.21.15.39:0/428555113 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564febca2580 0x564febcab840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.956+0000 7facf37fe700 1 -- 172.21.15.39:0/428555113 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564febca2580 msgr2=0x564febcab840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:50:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.956+0000 7facf37fe700 1 --2- 172.21.15.39:0/428555113 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564febca2580 0x564febcab840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.956+0000 7facf37fe700 1 -- 172.21.15.39:0/428555113 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564febca82d0 msgr2=0x564febcabee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:50:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.956+0000 7facf37fe700 1 --2- 172.21.15.39:0/428555113 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564febca82d0 0x564febcabee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.956+0000 7facf37fe700 1 -- 172.21.15.39:0/428555113 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7face4017070 con 0x564febafe6b0 2023-07-21T18:50:51.978 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.956+0000 7facf3fff700 1 --2- 172.21.15.39:0/428555113 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564febca82d0 0x564febcabee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:50:51.978 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.956+0000 7facf37fe700 1 --2- 172.21.15.39:0/428555113 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564febafe6b0 0x564febcae5c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7face80a1ea0 tx=0x7face8003680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=932437ea11174a1d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:50:51.978 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.956+0000 7facd2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-21T18:50:51.978 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.957+0000 7facd2ffd700 1 -- 172.21.15.39:0/428555113 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face80c8070 con 0x564febafe6b0 2023-07-21T18:50:51.978 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.957+0000 7facd2ffd700 1 -- 172.21.15.39:0/428555113 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8527+0+0 (secure 0 0 0) 0x7face8098320 con 0x564febafe6b0 2023-07-21T18:50:51.978 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.957+0000 7fad061dbb80 1 -- 172.21.15.39:0/428555113 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564febb4d490 con 0x564febafe6b0 2023-07-21T18:50:51.978 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.957+0000 7facd2ffd700 1 -- 172.21.15.39:0/428555113 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7face809e030 con 0x564febafe6b0 2023-07-21T18:50:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.957+0000 7fad061dbb80 10 client.26300 fetch_fsmap learned FSMap version 59 2023-07-21T18:50:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.957+0000 7fad061dbb80 1 -- 172.21.15.39:0/428555113 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x7facf41a2c80 con 0x564febafe6b0 2023-07-21T18:50:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.957+0000 7facd2ffd700 1 -- 172.21.15.39:0/428555113 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564febb4d490 con 0x564febafe6b0 2023-07-21T18:50:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.957+0000 7facd2ffd700 1 -- 172.21.15.39:0/428555113 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 59) v1 ==== 7494+0+0 (secure 0 0 0) 0x7face80a1f40 con 0x564febafe6b0 2023-07-21T18:50:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.957+0000 7fad061dbb80 10 client.26300 fetch_fsmap finished waiting for FSMap version 59 2023-07-21T18:50:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.957+0000 7fad061dbb80 10 client.26300 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:50:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.957+0000 7fad061dbb80 20 client.26300 populate_metadata read hostname 'smithi039' 2023-07-21T18:50:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.957+0000 7fad061dbb80 1 --2- 172.21.15.39:0/428555113 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x564febcbb8b0 0x564febcbdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:51.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.957+0000 7fad061dbb80 4 client.26300 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:50:51.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.957+0000 7fad061dbb80 1 -- 172.21.15.39:0/428555113 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564febbece50 con 0x564febcbb8b0 2023-07-21T18:50:51.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.957+0000 7facf2ffd700 1 --2- 172.21.15.39:0/428555113 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x564febcbb8b0 0x564febcbdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:51.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.958+0000 7facd2ffd700 10 client.26300 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:50:51.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.958+0000 7facf2ffd700 1 --2- 172.21.15.39:0/428555113 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x564febcbb8b0 0x564febcbdcc0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f26758aa7442ab91 server_cookie=35828156bfe9a256 in_seq=0 out_seq=0 2023-07-21T18:50:51.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.959+0000 7facd2ffd700 1 -- 172.21.15.39:0/428555113 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564febbece50 con 0x564febcbb8b0 2023-07-21T18:50:51.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.959+0000 7facd2ffd700 10 client.26300 handle_command_reply: tid=0 2023-07-21T18:50:51.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.959+0000 7fad061dbb80 1 client.26300 shutdown 2023-07-21T18:50:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.959+0000 7fad061dbb80 1 -- 172.21.15.39:0/428555113 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564febafe6b0 msgr2=0x564febcae5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:50:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.959+0000 7fad061dbb80 1 --2- 172.21.15.39:0/428555113 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564febafe6b0 0x564febcae5c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7face80a1ea0 tx=0x7face8003680 comp rx=0 tx=0).stop 2023-07-21T18:50:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.959+0000 7fad061dbb80 1 -- 172.21.15.39:0/428555113 shutdown_connections 2023-07-21T18:50:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.959+0000 7fad061dbb80 1 --2- 172.21.15.39:0/428555113 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x564febcbb8b0 0x564febcbdcc0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.959+0000 7fad061dbb80 1 --2- 172.21.15.39:0/428555113 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564febafe6b0 0x564febcae5c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.959+0000 7facd2ffd700 0 client.26300 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:50:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.959+0000 7fad061dbb80 1 --2- 172.21.15.39:0/428555113 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564febca82d0 0x564febcabee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:51.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.959+0000 7fad061dbb80 1 --2- 172.21.15.39:0/428555113 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564febca2580 0x564febcab840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:51.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.959+0000 7fad061dbb80 1 -- 172.21.15.39:0/428555113 >> 172.21.15.39:0/428555113 conn(0x564febc97be0 msgr2=0x564febc9bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:50:51.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.959+0000 7fad061dbb80 1 -- 172.21.15.39:0/428555113 shutdown_connections 2023-07-21T18:50:51.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.959+0000 7fad061dbb80 1 -- 172.21.15.39:0/428555113 wait complete. 2023-07-21T18:50:51.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.960+0000 7fad061dbb80 20 client.26300 trim_cache size 0 max 16384 2023-07-21T18:50:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.976+0000 7fad061dbb80 1 Processor -- start 2023-07-21T18:50:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.976+0000 7fad061dbb80 1 -- start start 2023-07-21T18:50:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.976+0000 7fad061dbb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564febafe6b0 0x564febcb2c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.976+0000 7fad061dbb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564febca2580 0x564febca70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.976+0000 7fad061dbb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564febca82d0 0x564febcae160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.976+0000 7fad061dbb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x564febcace20 con 0x564febafe6b0 2023-07-21T18:50:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.976+0000 7fad061dbb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564febc9da00 con 0x564febca82d0 2023-07-21T18:50:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.976+0000 7fad061dbb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x564febca86a0 con 0x564febca2580 2023-07-21T18:50:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.976+0000 7facf37fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564febafe6b0 0x564febcb2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.976+0000 7facf37fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564febafe6b0 0x564febcb2c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60514/0 (socket says 172.21.15.39:60514) 2023-07-21T18:50:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.976+0000 7facf37fe700 1 -- 172.21.15.39:0/3657297677 learned_addr learned my addr 172.21.15.39:0/3657297677 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:50:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.976+0000 7facf3fff700 1 --2- 172.21.15.39:0/3657297677 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564febca82d0 0x564febcae160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.976+0000 7facf2ffd700 1 --2- 172.21.15.39:0/3657297677 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564febca2580 0x564febca70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.976+0000 7facf37fe700 1 -- 172.21.15.39:0/3657297677 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564febca2580 msgr2=0x564febca70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:50:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.976+0000 7facf37fe700 1 --2- 172.21.15.39:0/3657297677 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564febca2580 0x564febca70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.976+0000 7facf37fe700 1 -- 172.21.15.39:0/3657297677 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564febca82d0 msgr2=0x564febcae160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:50:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.976+0000 7facf37fe700 1 --2- 172.21.15.39:0/3657297677 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564febca82d0 0x564febcae160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.976+0000 7facf37fe700 1 -- 172.21.15.39:0/3657297677 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564febcc28b0 con 0x564febafe6b0 2023-07-21T18:50:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.976+0000 7facf37fe700 1 --2- 172.21.15.39:0/3657297677 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564febafe6b0 0x564febcb2c60 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7face8000c00 tx=0x7face8096910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f83df6fa774f64ae server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:50:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.977+0000 7facda7fc700 1 -- 172.21.15.39:0/3657297677 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face80c8070 con 0x564febafe6b0 2023-07-21T18:50:51.995 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.977+0000 7facda7fc700 1 -- 172.21.15.39:0/3657297677 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7face8003dd0 con 0x564febafe6b0 2023-07-21T18:50:51.995 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.977+0000 7facda7fc700 1 -- 172.21.15.39:0/3657297677 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face80a1f40 con 0x564febafe6b0 2023-07-21T18:50:51.997 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.977+0000 7fad061dbb80 1 -- 172.21.15.39:0/3657297677 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564febafe6b0 msgr2=0x564febcb2c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:50:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.977+0000 7fad061dbb80 1 --2- 172.21.15.39:0/3657297677 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564febafe6b0 0x564febcb2c60 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7face8000c00 tx=0x7face8096910 comp rx=0 tx=0).stop 2023-07-21T18:50:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.977+0000 7fad061dbb80 1 -- 172.21.15.39:0/3657297677 shutdown_connections 2023-07-21T18:50:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.977+0000 7fad061dbb80 1 --2- 172.21.15.39:0/3657297677 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564febca82d0 0x564febcae160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.977+0000 7fad061dbb80 1 --2- 172.21.15.39:0/3657297677 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564febafe6b0 0x564febcb2c60 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.977+0000 7fad061dbb80 1 --2- 172.21.15.39:0/3657297677 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564febca2580 0x564febca70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.977+0000 7fad061dbb80 1 -- 172.21.15.39:0/3657297677 >> 172.21.15.39:0/3657297677 conn(0x564febc97be0 msgr2=0x564febca69d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:50:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.977+0000 7fad061dbb80 1 -- 172.21.15.39:0/3657297677 shutdown_connections 2023-07-21T18:50:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.977+0000 7fad061dbb80 1 -- 172.21.15.39:0/3657297677 wait complete. 2023-07-21T18:50:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.978+0000 7fad061dbb80 1 -- start start 2023-07-21T18:50:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.978+0000 7fad061dbb80 1 Processor -- start 2023-07-21T18:50:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.978+0000 7fad061dbb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564febca2580 0x564febca3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.978+0000 7fad061dbb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564febca82d0 0x564febca3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.978+0000 7fad061dbb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564febca4210 0x564febca4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.978+0000 7fad061dbb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x564febcb3200 con 0x564febca4210 2023-07-21T18:50:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.978+0000 7fad061dbb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564febca4bc0 con 0x564febca2580 2023-07-21T18:50:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.978+0000 7fad061dbb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x564febca4d80 con 0x564febca82d0 2023-07-21T18:50:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.978+0000 7fad061dbb80 10 client.0 _ll_register_callbacks cb 0x7face4004030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:50:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.978+0000 7facf3fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564febca4210 0x564febca4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:52.001 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.978+0000 7facf37fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564febca2580 0x564febca3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:52.001 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.978+0000 7facf3fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564febca4210 0x564febca4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60524/0 (socket says 172.21.15.39:60524) 2023-07-21T18:50:52.001 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.978+0000 7facf3fff700 1 -- 172.21.15.39:0/1128433300 learned_addr learned my addr 172.21.15.39:0/1128433300 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:50:52.001 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.978+0000 7facf2ffd700 1 --2- 172.21.15.39:0/1128433300 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564febca82d0 0x564febca3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:52.001 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.979+0000 7facf37fe700 1 -- 172.21.15.39:0/1128433300 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564febca82d0 msgr2=0x564febca3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:50:52.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.979+0000 7facf37fe700 1 --2- 172.21.15.39:0/1128433300 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564febca82d0 0x564febca3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:52.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.979+0000 7facf37fe700 1 -- 172.21.15.39:0/1128433300 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564febca4210 msgr2=0x564febca4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:50:52.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.979+0000 7facf37fe700 1 --2- 172.21.15.39:0/1128433300 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564febca4210 0x564febca4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:52.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.979+0000 7facf37fe700 1 -- 172.21.15.39:0/1128433300 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facf41a2c80 con 0x564febca2580 2023-07-21T18:50:52.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.979+0000 7facf3fff700 1 --2- 172.21.15.39:0/1128433300 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564febca4210 0x564febca4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:50:52.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.979+0000 7facf37fe700 1 --2- 172.21.15.39:0/1128433300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564febca2580 0x564febca3790 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x564febca3120 tx=0x7face80962c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b519d1380f88e924 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:50:52.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.979+0000 7facd2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-21T18:50:52.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.979+0000 7facd2ffd700 1 -- 172.21.15.39:0/1128433300 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face80c8020 con 0x564febca2580 2023-07-21T18:50:52.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.979+0000 7facd2ffd700 1 -- 172.21.15.39:0/1128433300 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8527+0+0 (secure 0 0 0) 0x7face809dfb0 con 0x564febca2580 2023-07-21T18:50:52.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.979+0000 7fad061dbb80 1 -- 172.21.15.39:0/1128433300 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7face80a23d0 con 0x564febca2580 2023-07-21T18:50:52.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.979+0000 7facd2ffd700 1 -- 172.21.15.39:0/1128433300 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7face80ca2b0 con 0x564febca2580 2023-07-21T18:50:52.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.979+0000 7fad061dbb80 10 client.26306 fetch_fsmap learned FSMap version 59 2023-07-21T18:50:52.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.979+0000 7fad061dbb80 1 -- 172.21.15.39:0/1128433300 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x564feba0ac00 con 0x564febca2580 2023-07-21T18:50:52.004 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.980+0000 7facd2ffd700 1 -- 172.21.15.39:0/1128433300 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7face80a23d0 con 0x564febca2580 2023-07-21T18:50:52.004 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.980+0000 7facd2ffd700 1 -- 172.21.15.39:0/1128433300 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 59) v1 ==== 7494+0+0 (secure 0 0 0) 0x7face807e020 con 0x564febca2580 2023-07-21T18:50:52.004 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.980+0000 7fad061dbb80 10 client.26306 fetch_fsmap finished waiting for FSMap version 59 2023-07-21T18:50:52.004 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.980+0000 7fad061dbb80 10 client.26306 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:50:52.004 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.980+0000 7fad061dbb80 20 client.26306 populate_metadata read hostname 'smithi039' 2023-07-21T18:50:52.004 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.980+0000 7fad061dbb80 1 --2- 172.21.15.39:0/1128433300 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x564febd0fe90 0x564febd12270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:50:52.004 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.980+0000 7fad061dbb80 4 client.26306 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:50:52.005 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.980+0000 7fad061dbb80 1 -- 172.21.15.39:0/1128433300 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564febbaeff0 con 0x564febd0fe90 2023-07-21T18:50:52.005 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.980+0000 7facf2ffd700 1 --2- 172.21.15.39:0/1128433300 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x564febd0fe90 0x564febd12270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:50:52.005 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.980+0000 7facd2ffd700 10 client.26306 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:50:52.005 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:51.980+0000 7facf2ffd700 1 --2- 172.21.15.39:0/1128433300 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x564febd0fe90 0x564febd12270 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=361b833038167852 server_cookie=68dd6d6e1f029f9b in_seq=0 out_seq=0 2023-07-21T18:50:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:52.017+0000 7facd2ffd700 1 -- 172.21.15.39:0/1128433300 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564febbaeff0 con 0x564febd0fe90 2023-07-21T18:50:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:52.017+0000 7facd2ffd700 10 client.26306 handle_command_reply: tid=0 2023-07-21T18:50:52.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:52.017+0000 7fad061dbb80 1 client.26306 shutdown 2023-07-21T18:50:52.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:52.017+0000 7fad061dbb80 1 -- 172.21.15.39:0/1128433300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564febca2580 msgr2=0x564febca3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:50:52.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:52.017+0000 7fad061dbb80 1 --2- 172.21.15.39:0/1128433300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564febca2580 0x564febca3790 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x564febca3120 tx=0x7face80962c0 comp rx=0 tx=0).stop 2023-07-21T18:50:52.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:52.018+0000 7fad061dbb80 1 -- 172.21.15.39:0/1128433300 shutdown_connections 2023-07-21T18:50:52.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:52.018+0000 7fad061dbb80 1 --2- 172.21.15.39:0/1128433300 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x564febd0fe90 0x564febd12270 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:52.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:52.018+0000 7fad061dbb80 1 --2- 172.21.15.39:0/1128433300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564febca2580 0x564febca3790 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:52.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:52.018+0000 7facd2ffd700 0 client.26306 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:50:52.032 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:52.018+0000 7fad061dbb80 1 --2- 172.21.15.39:0/1128433300 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564febca4210 0x564febca4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:52.032 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:52.018+0000 7fad061dbb80 1 --2- 172.21.15.39:0/1128433300 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564febca82d0 0x564febca3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:52.032 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:52.018+0000 7fad061dbb80 1 -- 172.21.15.39:0/1128433300 >> 172.21.15.39:0/1128433300 conn(0x564febc97be0 msgr2=0x564febca7260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:50:52.032 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:52.018+0000 7fad061dbb80 1 -- 172.21.15.39:0/1128433300 shutdown_connections 2023-07-21T18:50:52.032 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:52.018+0000 7fad061dbb80 1 -- 172.21.15.39:0/1128433300 wait complete. 2023-07-21T18:50:52.032 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:52.018+0000 7fad061dbb80 20 client.26306 trim_cache size 0 max 16384 2023-07-21T18:50:52.033 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:50:52.033 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:50:52.033 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:50:52.033 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:50:52.034 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:52.018+0000 7facfa055700 1 -- 172.21.15.39:0/947695233 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7facdc0a12c0 msgr2=0x7facdc0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:50:52.034 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:52.018+0000 7facfa055700 1 --2- 172.21.15.39:0/947695233 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7facdc0a12c0 0x7facdc0a3770 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7facf40689e0 tx=0x7face4009260 comp rx=0 tx=0).stop 2023-07-21T18:50:52.034 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:52.018+0000 7facfa055700 1 -- 172.21.15.39:0/947695233 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7facf40fea30 msgr2=0x7facf4196ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:50:52.034 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:52.018+0000 7facfa055700 1 --2- 172.21.15.39:0/947695233 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7facf40fea30 0x7facf4196ca0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7face800e970 tx=0x7face800ec80 comp rx=0 tx=0).stop 2023-07-21T18:50:52.034 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:52.018+0000 7facfa055700 1 -- 172.21.15.39:0/947695233 shutdown_connections 2023-07-21T18:50:52.034 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:52.018+0000 7facfa055700 1 --2- 172.21.15.39:0/947695233 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7facdc0a12c0 0x7facdc0a3770 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:52.035 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:52.018+0000 7facfa055700 1 --2- 172.21.15.39:0/947695233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7facf4104a30 0x7facf41a1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:52.035 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:52.018+0000 7facfa055700 1 --2- 172.21.15.39:0/947695233 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7facf40fea30 0x7facf4196ca0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:52.035 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:52.018+0000 7facfa055700 1 --2- 172.21.15.39:0/947695233 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7facf40ff3e0 0x7facf41971e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:50:52.035 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:52.018+0000 7facfa055700 1 -- 172.21.15.39:0/947695233 >> 172.21.15.39:0/947695233 conn(0x7facf4074760 msgr2=0x7facf41058a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:50:52.035 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:52.019+0000 7facfa055700 1 -- 172.21.15.39:0/947695233 shutdown_connections 2023-07-21T18:50:52.035 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:50:52.019+0000 7facfa055700 1 -- 172.21.15.39:0/947695233 wait complete. 2023-07-21T18:50:52.282 INFO:tasks.workunit.client.0.smithi039.stdout: 1 557039 6.01 MB/sec execute 365 sec latency 3328.738 ms 2023-07-21T18:50:52.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:50:52 smithi193 ceph-mon[98204]: pgmap v283: 97 pgs: 97 active+clean; 225 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 919 KiB/s rd, 1.8 MiB/s wr, 179 op/s 2023-07-21T18:50:52.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:50:52 smithi155 ceph-mon[97934]: pgmap v283: 97 pgs: 97 active+clean; 225 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 919 KiB/s rd, 1.8 MiB/s wr, 179 op/s 2023-07-21T18:50:52.678 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:50:52 smithi039 ceph-mon[96489]: pgmap v283: 97 pgs: 97 active+clean; 225 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 919 KiB/s rd, 1.8 MiB/s wr, 179 op/s 2023-07-21T18:50:53.282 INFO:tasks.workunit.client.0.smithi039.stdout: 1 558372 6.01 MB/sec execute 366 sec latency 11.472 ms 2023-07-21T18:50:54.282 INFO:tasks.workunit.client.0.smithi039.stdout: 1 559397 6.01 MB/sec execute 367 sec latency 366.840 ms 2023-07-21T18:50:54.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:50:54 smithi193 ceph-mon[98204]: pgmap v284: 97 pgs: 97 active+clean; 225 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 1.2 MiB/s wr, 112 op/s 2023-07-21T18:50:54.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:50:54 smithi155 ceph-mon[97934]: pgmap v284: 97 pgs: 97 active+clean; 225 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 1.2 MiB/s wr, 112 op/s 2023-07-21T18:50:54.678 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:50:54 smithi039 ceph-mon[96489]: pgmap v284: 97 pgs: 97 active+clean; 225 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 1.2 MiB/s wr, 112 op/s 2023-07-21T18:50:55.283 INFO:tasks.workunit.client.0.smithi039.stdout: 1 559397 6.00 MB/sec execute 368 sec latency 1366.957 ms 2023-07-21T18:50:56.283 INFO:tasks.workunit.client.0.smithi039.stdout: 1 559397 5.98 MB/sec execute 369 sec latency 2367.069 ms 2023-07-21T18:50:56.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:50:56 smithi193 ceph-mon[98204]: pgmap v285: 97 pgs: 97 active+clean; 225 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 1.2 MiB/s wr, 112 op/s 2023-07-21T18:50:56.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:50:56 smithi155 ceph-mon[97934]: pgmap v285: 97 pgs: 97 active+clean; 225 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 1.2 MiB/s wr, 112 op/s 2023-07-21T18:50:56.678 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:50:56 smithi039 ceph-mon[96489]: pgmap v285: 97 pgs: 97 active+clean; 225 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 1.2 MiB/s wr, 112 op/s 2023-07-21T18:50:57.283 INFO:tasks.workunit.client.0.smithi039.stdout: 1 560619 5.98 MB/sec execute 370 sec latency 2799.092 ms 2023-07-21T18:50:58.283 INFO:tasks.workunit.client.0.smithi039.stdout: 1 561824 5.98 MB/sec execute 371 sec latency 15.353 ms 2023-07-21T18:50:58.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:50:58 smithi193 ceph-mon[98204]: pgmap v286: 97 pgs: 97 active+clean; 224 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 922 KiB/s rd, 1.8 MiB/s wr, 179 op/s 2023-07-21T18:50:58.611 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:50:58 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:50:58.611 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:50:58 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:50:58.611 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:50:58 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:50:58.611 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:50:58 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:50:58.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:50:58 smithi155 ceph-mon[97934]: pgmap v286: 97 pgs: 97 active+clean; 224 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 922 KiB/s rd, 1.8 MiB/s wr, 179 op/s 2023-07-21T18:50:58.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:50:58 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:50:58.630 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:50:58 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:50:58.630 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:50:58 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:50:58.630 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:50:58 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:50:58.678 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:50:58 smithi039 ceph-mon[96489]: pgmap v286: 97 pgs: 97 active+clean; 224 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 922 KiB/s rd, 1.8 MiB/s wr, 179 op/s 2023-07-21T18:50:58.679 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:50:58 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:50:58.679 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:50:58 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:50:58.679 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:50:58 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:50:58.679 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:50:58 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:50:59.283 INFO:tasks.workunit.client.0.smithi039.stdout: 1 562946 5.98 MB/sec execute 372 sec latency 94.933 ms 2023-07-21T18:51:00.283 INFO:tasks.workunit.client.0.smithi039.stdout: 1 562946 5.97 MB/sec execute 373 sec latency 1095.030 ms 2023-07-21T18:51:00.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:51:00 smithi193 ceph-mon[98204]: pgmap v287: 97 pgs: 97 active+clean; 224 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 1.2 MiB/s wr, 124 op/s 2023-07-21T18:51:00.630 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:51:00 smithi155 ceph-mon[97934]: pgmap v287: 97 pgs: 97 active+clean; 224 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 1.2 MiB/s wr, 124 op/s 2023-07-21T18:51:00.678 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:51:00 smithi039 ceph-mon[96489]: pgmap v287: 97 pgs: 97 active+clean; 224 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 1.2 MiB/s wr, 124 op/s 2023-07-21T18:51:01.283 INFO:tasks.workunit.client.0.smithi039.stdout: 1 562946 5.95 MB/sec execute 374 sec latency 2095.179 ms 2023-07-21T18:51:02.284 INFO:tasks.workunit.client.0.smithi039.stdout: 1 563546 5.94 MB/sec execute 375 sec latency 2527.912 ms 2023-07-21T18:51:02.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:51:02 smithi193 ceph-mon[98204]: pgmap v288: 97 pgs: 97 active+clean; 227 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 1.8 MiB/s wr, 182 op/s 2023-07-21T18:51:02.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:51:02 smithi155 ceph-mon[97934]: pgmap v288: 97 pgs: 97 active+clean; 227 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 1.8 MiB/s wr, 182 op/s 2023-07-21T18:51:02.678 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:51:02 smithi039 ceph-mon[96489]: pgmap v288: 97 pgs: 97 active+clean; 227 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 1.8 MiB/s wr, 182 op/s 2023-07-21T18:51:03.283 INFO:tasks.workunit.client.0.smithi039.stdout: 1 565095 5.95 MB/sec execute 376 sec latency 25.050 ms 2023-07-21T18:51:04.283 INFO:tasks.workunit.client.0.smithi039.stdout: 1 566266 5.95 MB/sec execute 377 sec latency 21.677 ms 2023-07-21T18:51:04.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:51:04 smithi193 ceph-mon[98204]: pgmap v289: 97 pgs: 97 active+clean; 227 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 1.2 MiB/s wr, 124 op/s 2023-07-21T18:51:04.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:51:04 smithi155 ceph-mon[97934]: pgmap v289: 97 pgs: 97 active+clean; 227 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 1.2 MiB/s wr, 124 op/s 2023-07-21T18:51:04.678 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:51:04 smithi039 ceph-mon[96489]: pgmap v289: 97 pgs: 97 active+clean; 227 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 1.2 MiB/s wr, 124 op/s 2023-07-21T18:51:05.147 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.132+0000 7fb207382700 1 -- 172.21.15.39:0/107536962 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2000685b0 msgr2=0x7fb200104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:51:05.147 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.132+0000 7fb207382700 1 --2- 172.21.15.39:0/107536962 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2000685b0 0x7fb200104770 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fb1f00099c0 tx=0x7fb1f0009cd0 comp rx=0 tx=0).stop 2023-07-21T18:51:05.147 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.132+0000 7fb207382700 1 -- 172.21.15.39:0/107536962 shutdown_connections 2023-07-21T18:51:05.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.132+0000 7fb207382700 1 --2- 172.21.15.39:0/107536962 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb200104e50 0x7fb20010ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:05.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.132+0000 7fb207382700 1 --2- 172.21.15.39:0/107536962 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb200067c10 0x7fb200067fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:05.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.132+0000 7fb207382700 1 --2- 172.21.15.39:0/107536962 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2000685b0 0x7fb200104770 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:05.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.132+0000 7fb207382700 1 -- 172.21.15.39:0/107536962 >> 172.21.15.39:0/107536962 conn(0x7fb200074760 msgr2=0x7fb200074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:51:05.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.132+0000 7fb207382700 1 -- 172.21.15.39:0/107536962 shutdown_connections 2023-07-21T18:51:05.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.132+0000 7fb207382700 1 -- 172.21.15.39:0/107536962 wait complete. 2023-07-21T18:51:05.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.133+0000 7fb207382700 1 Processor -- start 2023-07-21T18:51:05.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.133+0000 7fb207382700 1 -- start start 2023-07-21T18:51:05.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.133+0000 7fb207382700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb200067c10 0x7fb2000fec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:05.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.133+0000 7fb207382700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2000685b0 0x7fb2000ff180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:05.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.133+0000 7fb207382700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb200104e50 0x7fb2001a3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:05.150 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.133+0000 7fb207382700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2001a42e0 con 0x7fb200067c10 2023-07-21T18:51:05.150 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.133+0000 7fb207382700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2001a4420 con 0x7fb200104e50 2023-07-21T18:51:05.150 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.133+0000 7fb207382700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2001a4590 con 0x7fb2000685b0 2023-07-21T18:51:05.150 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.133+0000 7fb20511e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb200067c10 0x7fb2000fec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:05.150 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.133+0000 7fb20511e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb200067c10 0x7fb2000fec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60534/0 (socket says 172.21.15.39:60534) 2023-07-21T18:51:05.151 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.133+0000 7fb20511e700 1 -- 172.21.15.39:0/566574526 learned_addr learned my addr 172.21.15.39:0/566574526 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:51:05.151 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.133+0000 7fb20491d700 1 --2- 172.21.15.39:0/566574526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2000685b0 0x7fb2000ff180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:05.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.133+0000 7fb20591f700 1 --2- 172.21.15.39:0/566574526 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb200104e50 0x7fb2001a3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:05.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.133+0000 7fb20511e700 1 -- 172.21.15.39:0/566574526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2000685b0 msgr2=0x7fb2000ff180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:51:05.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.133+0000 7fb20511e700 1 --2- 172.21.15.39:0/566574526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2000685b0 0x7fb2000ff180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:05.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.134+0000 7fb20511e700 1 -- 172.21.15.39:0/566574526 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb200104e50 msgr2=0x7fb2001a3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:51:05.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.134+0000 7fb20511e700 1 --2- 172.21.15.39:0/566574526 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb200104e50 0x7fb2001a3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:05.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.134+0000 7fb20511e700 1 -- 172.21.15.39:0/566574526 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1f00095f0 con 0x7fb200067c10 2023-07-21T18:51:05.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.134+0000 7fb20591f700 1 --2- 172.21.15.39:0/566574526 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb200104e50 0x7fb2001a3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:51:05.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.134+0000 7fb20511e700 1 --2- 172.21.15.39:0/566574526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb200067c10 0x7fb2000fec40 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc00e9a0 tx=0x7fb1fc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:51:05.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.134+0000 7fb1f67fc700 1 -- 172.21.15.39:0/566574526 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1fc009790 con 0x7fb200067c10 2023-07-21T18:51:05.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.134+0000 7fb207382700 1 -- 172.21.15.39:0/566574526 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2001a4870 con 0x7fb200067c10 2023-07-21T18:51:05.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.134+0000 7fb1f67fc700 1 -- 172.21.15.39:0/566574526 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb1fc00cbd0 con 0x7fb200067c10 2023-07-21T18:51:05.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.134+0000 7fb1f67fc700 1 -- 172.21.15.39:0/566574526 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1fc018680 con 0x7fb200067c10 2023-07-21T18:51:05.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.134+0000 7fb207382700 1 -- 172.21.15.39:0/566574526 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2001a4d50 con 0x7fb200067c10 2023-07-21T18:51:05.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.136+0000 7fb1f67fc700 1 -- 172.21.15.39:0/566574526 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fb1fc00cd40 con 0x7fb200067c10 2023-07-21T18:51:05.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.137+0000 7fb1f67fc700 1 --2- 172.21.15.39:0/566574526 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fb1ec0a12c0 0x7fb1ec0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.137+0000 7fb20491d700 1 --2- 172.21.15.39:0/566574526 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fb1ec0a12c0 0x7fb1ec0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.137+0000 7fb1f67fc700 1 -- 172.21.15.39:0/566574526 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fb1fc014070 con 0x7fb200067c10 2023-07-21T18:51:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.137+0000 7fb20491d700 1 --2- 172.21.15.39:0/566574526 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fb1ec0a12c0 0x7fb1ec0a3770 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb1f00098e0 tx=0x7fb1f0000bc0 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:51:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.138+0000 7fb213508b80 1 Processor -- start 2023-07-21T18:51:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.138+0000 7fb213508b80 1 -- start start 2023-07-21T18:51:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.138+0000 7fb213508b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55df4952a6b0 0x55df496d3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.138+0000 7fb213508b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55df496d42d0 0x55df496ce040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.138+0000 7fb213508b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55df496ce580 0x55df496ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.138+0000 7fb213508b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df495d3c50 con 0x55df496ce580 2023-07-21T18:51:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.138+0000 7fb213508b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df496269e0 con 0x55df496d42d0 2023-07-21T18:51:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.138+0000 7fb213508b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df4961fe70 con 0x55df4952a6b0 2023-07-21T18:51:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.138+0000 7fb20591f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55df496ce580 0x55df496ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.138+0000 7fb20591f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55df496ce580 0x55df496ce9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60546/0 (socket says 172.21.15.39:60546) 2023-07-21T18:51:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.138+0000 7fb20591f700 1 -- 172.21.15.39:0/758996740 learned_addr learned my addr 172.21.15.39:0/758996740 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:51:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.138+0000 7fb20511e700 1 --2- 172.21.15.39:0/758996740 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55df4952a6b0 0x55df496d3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.139+0000 7fb20491d700 1 --2- 172.21.15.39:0/758996740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55df496d42d0 0x55df496ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.139+0000 7fb20591f700 1 -- 172.21.15.39:0/758996740 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55df4952a6b0 msgr2=0x55df496d3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:51:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.139+0000 7fb20591f700 1 --2- 172.21.15.39:0/758996740 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55df4952a6b0 0x55df496d3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.139+0000 7fb20591f700 1 -- 172.21.15.39:0/758996740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55df496d42d0 msgr2=0x55df496ce040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:51:05.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.139+0000 7fb20591f700 1 --2- 172.21.15.39:0/758996740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55df496d42d0 0x55df496ce040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:05.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.139+0000 7fb20591f700 1 -- 172.21.15.39:0/758996740 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df4944dd80 con 0x55df496ce580 2023-07-21T18:51:05.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.139+0000 7fb20591f700 1 --2- 172.21.15.39:0/758996740 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55df496ce580 0x55df496ce9d0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8009e60 tx=0x7fb1f80077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e69aa864cce3d93a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:51:05.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.139+0000 7fb1eb7fe700 1 -- 172.21.15.39:0/758996740 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f8014610 con 0x55df496ce580 2023-07-21T18:51:05.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.139+0000 7fb1eb7fe700 1 -- 172.21.15.39:0/758996740 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb1f800f050 con 0x55df496ce580 2023-07-21T18:51:05.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.139+0000 7fb1eb7fe700 1 -- 172.21.15.39:0/758996740 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f8013990 con 0x55df496ce580 2023-07-21T18:51:05.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.140+0000 7fb213508b80 1 -- 172.21.15.39:0/758996740 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55df496ce580 msgr2=0x55df496ce9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:51:05.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.140+0000 7fb213508b80 1 --2- 172.21.15.39:0/758996740 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55df496ce580 0x55df496ce9d0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8009e60 tx=0x7fb1f80077e0 comp rx=0 tx=0).stop 2023-07-21T18:51:05.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.140+0000 7fb213508b80 1 -- 172.21.15.39:0/758996740 shutdown_connections 2023-07-21T18:51:05.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.140+0000 7fb213508b80 1 --2- 172.21.15.39:0/758996740 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55df496ce580 0x55df496ce9d0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:05.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.140+0000 7fb213508b80 1 --2- 172.21.15.39:0/758996740 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55df4952a6b0 0x55df496d3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:05.166 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.140+0000 7fb213508b80 1 --2- 172.21.15.39:0/758996740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55df496d42d0 0x55df496ce040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:05.166 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.140+0000 7fb213508b80 1 -- 172.21.15.39:0/758996740 >> 172.21.15.39:0/758996740 conn(0x55df496c3be0 msgr2=0x55df496c8f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:51:05.166 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.140+0000 7fb213508b80 1 -- 172.21.15.39:0/758996740 shutdown_connections 2023-07-21T18:51:05.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.140+0000 7fb213508b80 1 -- 172.21.15.39:0/758996740 wait complete. 2023-07-21T18:51:05.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.140+0000 7fb213508b80 1 -- start start 2023-07-21T18:51:05.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.140+0000 7fb213508b80 1 Processor -- start 2023-07-21T18:51:05.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.141+0000 7fb213508b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55df4952a6b0 0x55df496da580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:05.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.141+0000 7fb213508b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55df496ce580 0x55df496d7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:05.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.141+0000 7fb213508b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55df496d42d0 0x55df496d7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:05.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.141+0000 7fb213508b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df495bfa60 con 0x55df496ce580 2023-07-21T18:51:05.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.141+0000 7fb213508b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df495bf760 con 0x55df496d42d0 2023-07-21T18:51:05.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.141+0000 7fb213508b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df494a38c0 con 0x55df4952a6b0 2023-07-21T18:51:05.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.141+0000 7fb213508b80 10 client.0 _ll_register_callbacks cb 0x55df495bd700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:51:05.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.141+0000 7fb20491d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55df496ce580 0x55df496d7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:05.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.141+0000 7fb20491d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55df496ce580 0x55df496d7790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60550/0 (socket says 172.21.15.39:60550) 2023-07-21T18:51:05.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.141+0000 7fb20491d700 1 -- 172.21.15.39:0/4114366398 learned_addr learned my addr 172.21.15.39:0/4114366398 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:51:05.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.141+0000 7fb20511e700 1 --2- 172.21.15.39:0/4114366398 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55df4952a6b0 0x55df496da580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:05.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.141+0000 7fb20591f700 1 --2- 172.21.15.39:0/4114366398 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55df496d42d0 0x55df496d7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:05.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.141+0000 7fb20491d700 1 -- 172.21.15.39:0/4114366398 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55df4952a6b0 msgr2=0x55df496da580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:51:05.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.141+0000 7fb20491d700 1 --2- 172.21.15.39:0/4114366398 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55df4952a6b0 0x55df496da580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:05.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.141+0000 7fb20491d700 1 -- 172.21.15.39:0/4114366398 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55df496d42d0 msgr2=0x55df496d7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:51:05.172 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.141+0000 7fb20491d700 1 --2- 172.21.15.39:0/4114366398 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55df496d42d0 0x55df496d7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:05.172 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.141+0000 7fb20491d700 1 -- 172.21.15.39:0/4114366398 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1f8009a90 con 0x55df496ce580 2023-07-21T18:51:05.172 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.141+0000 7fb20491d700 1 --2- 172.21.15.39:0/4114366398 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55df496ce580 0x55df496d7790 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fb1f00036c0 tx=0x7fb1f0009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5a31f565a32c7f3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:51:05.172 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.141+0000 7fb1e8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:51:05.172 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.141+0000 7fb1e8ff9700 1 -- 172.21.15.39:0/4114366398 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f002c070 con 0x55df496ce580 2023-07-21T18:51:05.172 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.141+0000 7fb1e8ff9700 1 -- 172.21.15.39:0/4114366398 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fb1f0004260 con 0x55df496ce580 2023-07-21T18:51:05.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.141+0000 7fb213508b80 1 -- 172.21.15.39:0/4114366398 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55df49579490 con 0x55df496ce580 2023-07-21T18:51:05.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.142+0000 7fb1e8ff9700 1 -- 172.21.15.39:0/4114366398 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb1f0003ad0 con 0x55df496ce580 2023-07-21T18:51:05.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.142+0000 7fb213508b80 10 client.16848 fetch_fsmap learned FSMap version 59 2023-07-21T18:51:05.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.142+0000 7fb213508b80 1 -- 172.21.15.39:0/4114366398 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x55df49436c00 con 0x55df496ce580 2023-07-21T18:51:05.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.142+0000 7fb1e8ff9700 1 -- 172.21.15.39:0/4114366398 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55df49579490 con 0x55df496ce580 2023-07-21T18:51:05.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.142+0000 7fb1e8ff9700 1 -- 172.21.15.39:0/4114366398 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 59) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fb1f003acf0 con 0x55df496ce580 2023-07-21T18:51:05.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.142+0000 7fb213508b80 10 client.16848 fetch_fsmap finished waiting for FSMap version 59 2023-07-21T18:51:05.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.142+0000 7fb213508b80 10 client.16848 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:51:05.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.142+0000 7fb213508b80 20 client.16848 populate_metadata read hostname 'smithi039' 2023-07-21T18:51:05.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.142+0000 7fb213508b80 1 --2- 172.21.15.39:0/4114366398 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55df496e7b50 0x55df496e9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:05.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.142+0000 7fb213508b80 4 client.16848 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:51:05.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.142+0000 7fb213508b80 1 -- 172.21.15.39:0/4114366398 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55df49618e50 con 0x55df496e7b50 2023-07-21T18:51:05.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.142+0000 7fb20511e700 1 --2- 172.21.15.39:0/4114366398 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55df496e7b50 0x55df496e9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:05.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.143+0000 7fb1e8ff9700 10 client.16848 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:51:05.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.143+0000 7fb20511e700 1 --2- 172.21.15.39:0/4114366398 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55df496e7b50 0x55df496e9f60 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c7db0546d918afc server_cookie=9613dbdb8b41456f in_seq=0 out_seq=0 2023-07-21T18:51:05.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.143+0000 7fb1e8ff9700 1 -- 172.21.15.39:0/4114366398 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55df49618e50 con 0x55df496e7b50 2023-07-21T18:51:05.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.143+0000 7fb1e8ff9700 10 client.16848 handle_command_reply: tid=0 2023-07-21T18:51:05.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.143+0000 7fb213508b80 1 client.16848 shutdown 2023-07-21T18:51:05.177 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.143+0000 7fb213508b80 1 -- 172.21.15.39:0/4114366398 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55df496ce580 msgr2=0x55df496d7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:51:05.177 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.143+0000 7fb213508b80 1 --2- 172.21.15.39:0/4114366398 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55df496ce580 0x55df496d7790 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fb1f00036c0 tx=0x7fb1f0009310 comp rx=0 tx=0).stop 2023-07-21T18:51:05.177 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.143+0000 7fb213508b80 1 -- 172.21.15.39:0/4114366398 shutdown_connections 2023-07-21T18:51:05.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.143+0000 7fb213508b80 1 --2- 172.21.15.39:0/4114366398 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55df496e7b50 0x55df496e9f60 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:05.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.143+0000 7fb213508b80 1 --2- 172.21.15.39:0/4114366398 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55df496d42d0 0x55df496d7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:05.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.143+0000 7fb1e8ff9700 0 client.16848 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:51:05.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.143+0000 7fb213508b80 1 --2- 172.21.15.39:0/4114366398 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55df4952a6b0 0x55df496da580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:05.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.143+0000 7fb213508b80 1 --2- 172.21.15.39:0/4114366398 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55df496ce580 0x55df496d7790 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:05.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.143+0000 7fb213508b80 1 -- 172.21.15.39:0/4114366398 >> 172.21.15.39:0/4114366398 conn(0x55df496c3be0 msgr2=0x55df496c7c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:51:05.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.144+0000 7fb213508b80 1 -- 172.21.15.39:0/4114366398 shutdown_connections 2023-07-21T18:51:05.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.144+0000 7fb213508b80 1 -- 172.21.15.39:0/4114366398 wait complete. 2023-07-21T18:51:05.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.144+0000 7fb213508b80 20 client.16848 trim_cache size 0 max 16384 2023-07-21T18:51:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.158+0000 7fb213508b80 1 Processor -- start 2023-07-21T18:51:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.158+0000 7fb213508b80 1 -- start start 2023-07-21T18:51:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.159+0000 7fb213508b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55df496c3be0 0x55df496ce040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.159+0000 7fb213508b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55df4952a6b0 0x55df496dec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.159+0000 7fb213508b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55df496ce580 0x55df496d6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.159+0000 7fb213508b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df496c5d00 con 0x55df4952a6b0 2023-07-21T18:51:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.159+0000 7fb213508b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df496c9b70 con 0x55df496c3be0 2023-07-21T18:51:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.159+0000 7fb213508b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df496d8b80 con 0x55df496ce580 2023-07-21T18:51:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.159+0000 7fb20491d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55df4952a6b0 0x55df496dec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.159+0000 7fb20491d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55df4952a6b0 0x55df496dec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60558/0 (socket says 172.21.15.39:60558) 2023-07-21T18:51:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.159+0000 7fb20491d700 1 -- 172.21.15.39:0/1073679679 learned_addr learned my addr 172.21.15.39:0/1073679679 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:51:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.159+0000 7fb20511e700 1 --2- 172.21.15.39:0/1073679679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55df496c3be0 0x55df496ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.159+0000 7fb20591f700 1 --2- 172.21.15.39:0/1073679679 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55df496ce580 0x55df496d6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.159+0000 7fb20491d700 1 -- 172.21.15.39:0/1073679679 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55df496ce580 msgr2=0x55df496d6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:51:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.159+0000 7fb20491d700 1 --2- 172.21.15.39:0/1073679679 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55df496ce580 0x55df496d6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.159+0000 7fb20491d700 1 -- 172.21.15.39:0/1073679679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55df496c3be0 msgr2=0x55df496ce040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:51:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.159+0000 7fb20491d700 1 --2- 172.21.15.39:0/1073679679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55df496c3be0 0x55df496ce040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.159+0000 7fb20491d700 1 -- 172.21.15.39:0/1073679679 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df496eeb50 con 0x55df4952a6b0 2023-07-21T18:51:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.159+0000 7fb20491d700 1 --2- 172.21.15.39:0/1073679679 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55df4952a6b0 0x55df496dec60 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fb1f00095a0 tx=0x7fb1f0003ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4236120acf07adc server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:51:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.159+0000 7fb1ebfff700 1 -- 172.21.15.39:0/1073679679 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f002c070 con 0x55df4952a6b0 2023-07-21T18:51:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.159+0000 7fb1ebfff700 1 -- 172.21.15.39:0/1073679679 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb1f002d040 con 0x55df4952a6b0 2023-07-21T18:51:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.159+0000 7fb1ebfff700 1 -- 172.21.15.39:0/1073679679 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f000fd60 con 0x55df4952a6b0 2023-07-21T18:51:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.160+0000 7fb213508b80 1 -- 172.21.15.39:0/1073679679 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55df4952a6b0 msgr2=0x55df496dec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:51:05.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.160+0000 7fb213508b80 1 --2- 172.21.15.39:0/1073679679 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55df4952a6b0 0x55df496dec60 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fb1f00095a0 tx=0x7fb1f0003ad0 comp rx=0 tx=0).stop 2023-07-21T18:51:05.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.160+0000 7fb213508b80 1 -- 172.21.15.39:0/1073679679 shutdown_connections 2023-07-21T18:51:05.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.160+0000 7fb213508b80 1 --2- 172.21.15.39:0/1073679679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55df496c3be0 0x55df496ce040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:05.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.160+0000 7fb213508b80 1 --2- 172.21.15.39:0/1073679679 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55df496ce580 0x55df496d6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:05.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.160+0000 7fb213508b80 1 --2- 172.21.15.39:0/1073679679 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55df4952a6b0 0x55df496dec60 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:05.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.160+0000 7fb213508b80 1 -- 172.21.15.39:0/1073679679 >> 172.21.15.39:0/1073679679 conn(0x55df496e4100 msgr2=0x55df496d5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:51:05.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.160+0000 7fb213508b80 1 -- 172.21.15.39:0/1073679679 shutdown_connections 2023-07-21T18:51:05.188 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.160+0000 7fb213508b80 1 -- 172.21.15.39:0/1073679679 wait complete. 2023-07-21T18:51:05.188 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.160+0000 7fb213508b80 1 -- start start 2023-07-21T18:51:05.188 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.160+0000 7fb213508b80 1 Processor -- start 2023-07-21T18:51:05.188 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.161+0000 7fb213508b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55df4952a6b0 0x55df496d0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:05.188 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.161+0000 7fb213508b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55df496c3be0 0x55df496d1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:05.188 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.161+0000 7fb213508b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55df496ce580 0x55df496cf790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:05.189 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.161+0000 7fb213508b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df496cfd10 con 0x55df496ce580 2023-07-21T18:51:05.189 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.161+0000 7fb213508b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df496cfe80 con 0x55df4952a6b0 2023-07-21T18:51:05.189 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.161+0000 7fb213508b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df496d0040 con 0x55df496c3be0 2023-07-21T18:51:05.189 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.161+0000 7fb213508b80 10 client.0 _ll_register_callbacks cb 0x7fb1f000b790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:51:05.189 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.161+0000 7fb20591f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55df496ce580 0x55df496cf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:05.189 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.161+0000 7fb20491d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55df496c3be0 0x55df496d1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:05.189 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.161+0000 7fb20591f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55df496ce580 0x55df496cf790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60566/0 (socket says 172.21.15.39:60566) 2023-07-21T18:51:05.190 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.161+0000 7fb20591f700 1 -- 172.21.15.39:0/2639260906 learned_addr learned my addr 172.21.15.39:0/2639260906 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:51:05.190 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.161+0000 7fb20511e700 1 --2- 172.21.15.39:0/2639260906 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55df4952a6b0 0x55df496d0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:05.190 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.161+0000 7fb20491d700 1 -- 172.21.15.39:0/2639260906 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55df4952a6b0 msgr2=0x55df496d0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:51:05.190 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.161+0000 7fb20491d700 1 --2- 172.21.15.39:0/2639260906 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55df4952a6b0 0x55df496d0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:05.190 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.161+0000 7fb20491d700 1 -- 172.21.15.39:0/2639260906 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55df496ce580 msgr2=0x55df496cf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:51:05.190 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.161+0000 7fb20491d700 1 --2- 172.21.15.39:0/2639260906 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55df496ce580 0x55df496cf790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:05.190 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.161+0000 7fb20491d700 1 -- 172.21.15.39:0/2639260906 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55df49436c00 con 0x55df496c3be0 2023-07-21T18:51:05.190 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.161+0000 7fb20591f700 1 --2- 172.21.15.39:0/2639260906 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55df496ce580 0x55df496cf790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:51:05.191 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.162+0000 7fb20491d700 1 --2- 172.21.15.39:0/2639260906 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55df496c3be0 0x55df496d1140 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x55df496d8200 tx=0x7fb1f00047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8a95c4c2b32f049 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:51:05.191 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.162+0000 7fb1e8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2023-07-21T18:51:05.191 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.162+0000 7fb1e8ff9700 1 -- 172.21.15.39:0/2639260906 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f002c070 con 0x55df496c3be0 2023-07-21T18:51:05.191 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.162+0000 7fb213508b80 1 -- 172.21.15.39:0/2639260906 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb1f800f050 con 0x55df496c3be0 2023-07-21T18:51:05.191 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.162+0000 7fb1e8ff9700 1 -- 172.21.15.39:0/2639260906 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fb1f000f890 con 0x55df496c3be0 2023-07-21T18:51:05.191 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.162+0000 7fb1e8ff9700 1 -- 172.21.15.39:0/2639260906 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb1f000fa50 con 0x55df496c3be0 2023-07-21T18:51:05.191 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.162+0000 7fb213508b80 10 client.26332 fetch_fsmap learned FSMap version 59 2023-07-21T18:51:05.191 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.162+0000 7fb213508b80 1 -- 172.21.15.39:0/2639260906 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x55df494741f0 con 0x55df496c3be0 2023-07-21T18:51:05.192 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.162+0000 7fb1e8ff9700 1 -- 172.21.15.39:0/2639260906 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb1f800f050 con 0x55df496c3be0 2023-07-21T18:51:05.192 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.162+0000 7fb1e8ff9700 1 -- 172.21.15.39:0/2639260906 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 59) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fb1f0031460 con 0x55df496c3be0 2023-07-21T18:51:05.192 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.162+0000 7fb213508b80 10 client.26332 fetch_fsmap finished waiting for FSMap version 59 2023-07-21T18:51:05.192 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.162+0000 7fb213508b80 10 client.26332 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:51:05.192 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.162+0000 7fb213508b80 20 client.26332 populate_metadata read hostname 'smithi039' 2023-07-21T18:51:05.192 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.162+0000 7fb213508b80 1 --2- 172.21.15.39:0/2639260906 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55df496d42d0 0x55df4973e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:05.192 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.162+0000 7fb213508b80 4 client.26332 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:51:05.193 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.162+0000 7fb213508b80 1 -- 172.21.15.39:0/2639260906 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55df495daff0 con 0x55df496d42d0 2023-07-21T18:51:05.193 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.163+0000 7fb20511e700 1 --2- 172.21.15.39:0/2639260906 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55df496d42d0 0x55df4973e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:05.193 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.163+0000 7fb1e8ff9700 10 client.26332 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:51:05.193 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.163+0000 7fb20511e700 1 --2- 172.21.15.39:0/2639260906 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55df496d42d0 0x55df4973e550 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d564d7564974d46 server_cookie=4c5e73f9e8a7c933 in_seq=0 out_seq=0 2023-07-21T18:51:05.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.199+0000 7fb1e8ff9700 1 -- 172.21.15.39:0/2639260906 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55df495daff0 con 0x55df496d42d0 2023-07-21T18:51:05.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.199+0000 7fb1e8ff9700 10 client.26332 handle_command_reply: tid=0 2023-07-21T18:51:05.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.199+0000 7fb213508b80 1 client.26332 shutdown 2023-07-21T18:51:05.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.199+0000 7fb213508b80 1 -- 172.21.15.39:0/2639260906 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55df496c3be0 msgr2=0x55df496d1140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:51:05.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.199+0000 7fb213508b80 1 --2- 172.21.15.39:0/2639260906 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55df496c3be0 0x55df496d1140 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x55df496d8200 tx=0x7fb1f00047b0 comp rx=0 tx=0).stop 2023-07-21T18:51:05.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.199+0000 7fb213508b80 1 -- 172.21.15.39:0/2639260906 shutdown_connections 2023-07-21T18:51:05.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.199+0000 7fb213508b80 1 --2- 172.21.15.39:0/2639260906 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55df496d42d0 0x55df4973e550 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:05.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.199+0000 7fb213508b80 1 --2- 172.21.15.39:0/2639260906 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55df4952a6b0 0x55df496d0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:05.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.199+0000 7fb213508b80 1 --2- 172.21.15.39:0/2639260906 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55df496ce580 0x55df496cf790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:05.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.199+0000 7fb213508b80 1 --2- 172.21.15.39:0/2639260906 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55df496c3be0 0x55df496d1140 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:05.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.199+0000 7fb1e8ff9700 0 client.26332 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:51:05.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.199+0000 7fb213508b80 1 -- 172.21.15.39:0/2639260906 >> 172.21.15.39:0/2639260906 conn(0x55df496e4100 msgr2=0x55df496d5190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:51:05.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.199+0000 7fb213508b80 1 -- 172.21.15.39:0/2639260906 shutdown_connections 2023-07-21T18:51:05.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.199+0000 7fb213508b80 1 -- 172.21.15.39:0/2639260906 wait complete. 2023-07-21T18:51:05.214 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.199+0000 7fb213508b80 20 client.26332 trim_cache size 0 max 16384 2023-07-21T18:51:05.214 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:51:05.214 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:51:05.214 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:51:05.214 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:51:05.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.200+0000 7fb207382700 1 -- 172.21.15.39:0/566574526 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fb1ec0a12c0 msgr2=0x7fb1ec0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:51:05.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.200+0000 7fb207382700 1 --2- 172.21.15.39:0/566574526 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fb1ec0a12c0 0x7fb1ec0a3770 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb1f00098e0 tx=0x7fb1f0000bc0 comp rx=0 tx=0).stop 2023-07-21T18:51:05.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.200+0000 7fb207382700 1 -- 172.21.15.39:0/566574526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb200067c10 msgr2=0x7fb2000fec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:51:05.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.200+0000 7fb207382700 1 --2- 172.21.15.39:0/566574526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb200067c10 0x7fb2000fec40 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc00e9a0 tx=0x7fb1fc00ecb0 comp rx=0 tx=0).stop 2023-07-21T18:51:05.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.200+0000 7fb207382700 1 -- 172.21.15.39:0/566574526 shutdown_connections 2023-07-21T18:51:05.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.200+0000 7fb207382700 1 --2- 172.21.15.39:0/566574526 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fb1ec0a12c0 0x7fb1ec0a3770 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:05.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.200+0000 7fb207382700 1 --2- 172.21.15.39:0/566574526 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb200104e50 0x7fb2001a3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:05.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.200+0000 7fb207382700 1 --2- 172.21.15.39:0/566574526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb200067c10 0x7fb2000fec40 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:05.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.200+0000 7fb207382700 1 --2- 172.21.15.39:0/566574526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2000685b0 0x7fb2000ff180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:05.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.200+0000 7fb207382700 1 -- 172.21.15.39:0/566574526 >> 172.21.15.39:0/566574526 conn(0x7fb200074760 msgr2=0x7fb200103740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:51:05.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.200+0000 7fb207382700 1 -- 172.21.15.39:0/566574526 shutdown_connections 2023-07-21T18:51:05.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:05.200+0000 7fb207382700 1 -- 172.21.15.39:0/566574526 wait complete. 2023-07-21T18:51:05.283 INFO:tasks.workunit.client.0.smithi039.stdout: 1 566486 5.94 MB/sec execute 378 sec latency 622.225 ms 2023-07-21T18:51:05.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:51:05 smithi193 ceph-mon[98204]: osdmap e90: 12 total, 12 up, 12 in 2023-07-21T18:51:05.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:51:05 smithi155 ceph-mon[97934]: osdmap e90: 12 total, 12 up, 12 in 2023-07-21T18:51:05.678 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:51:05 smithi039 ceph-mon[96489]: osdmap e90: 12 total, 12 up, 12 in 2023-07-21T18:51:06.283 INFO:tasks.workunit.client.0.smithi039.stdout: 1 566857 5.92 MB/sec execute 379 sec latency 1418.008 ms 2023-07-21T18:51:06.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:51:06 smithi193 ceph-mon[98204]: pgmap v291: 97 pgs: 97 active+clean; 227 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 1.4 MiB/s wr, 149 op/s 2023-07-21T18:51:06.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:51:06 smithi155 ceph-mon[97934]: pgmap v291: 97 pgs: 97 active+clean; 227 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 1.4 MiB/s wr, 149 op/s 2023-07-21T18:51:06.678 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:51:06 smithi039 ceph-mon[96489]: pgmap v291: 97 pgs: 97 active+clean; 227 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 1.4 MiB/s wr, 149 op/s 2023-07-21T18:51:07.283 INFO:tasks.workunit.client.0.smithi039.stdout: 1 568629 5.94 MB/sec execute 380 sec latency 13.900 ms 2023-07-21T18:51:07.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:51:07 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:51:07.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:51:07 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:51:07.678 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:51:07 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:51:08.284 INFO:tasks.workunit.client.0.smithi039.stdout: 1 569863 5.94 MB/sec execute 381 sec latency 21.429 ms 2023-07-21T18:51:08.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:51:08 smithi193 ceph-mon[98204]: pgmap v292: 97 pgs: 97 active+clean; 225 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 1.7 MiB/s wr, 187 op/s 2023-07-21T18:51:08.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:51:08 smithi155 ceph-mon[97934]: pgmap v292: 97 pgs: 97 active+clean; 225 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 1.7 MiB/s wr, 187 op/s 2023-07-21T18:51:08.678 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:51:08 smithi039 ceph-mon[96489]: pgmap v292: 97 pgs: 97 active+clean; 225 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 1.7 MiB/s wr, 187 op/s 2023-07-21T18:51:09.284 INFO:tasks.workunit.client.0.smithi039.stdout: 1 570036 5.93 MB/sec execute 382 sec latency 801.760 ms 2023-07-21T18:51:10.284 INFO:tasks.workunit.client.0.smithi039.stdout: 1 570036 5.91 MB/sec execute 383 sec latency 1801.816 ms 2023-07-21T18:51:10.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:51:10 smithi193 ceph-mon[98204]: pgmap v293: 97 pgs: 97 active+clean; 225 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 1.7 MiB/s wr, 187 op/s 2023-07-21T18:51:10.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:51:10 smithi155 ceph-mon[97934]: pgmap v293: 97 pgs: 97 active+clean; 225 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 1.7 MiB/s wr, 187 op/s 2023-07-21T18:51:10.678 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:51:10 smithi039 ceph-mon[96489]: pgmap v293: 97 pgs: 97 active+clean; 225 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 1.7 MiB/s wr, 187 op/s 2023-07-21T18:51:11.284 INFO:tasks.workunit.client.0.smithi039.stdout: 1 570036 5.90 MB/sec execute 384 sec latency 2801.928 ms 2023-07-21T18:51:12.284 INFO:tasks.workunit.client.0.smithi039.stdout: 1 571271 5.90 MB/sec execute 385 sec latency 3234.997 ms 2023-07-21T18:51:12.428 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:51:12 smithi039 ceph-mon[96489]: pgmap v294: 97 pgs: 97 active+clean; 228 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 1.7 MiB/s wr, 184 op/s 2023-07-21T18:51:12.428 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:51:12 smithi039 ceph-mon[96489]: osdmap e91: 12 total, 12 up, 12 in 2023-07-21T18:51:12.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:51:12 smithi193 ceph-mon[98204]: pgmap v294: 97 pgs: 97 active+clean; 228 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 1.7 MiB/s wr, 184 op/s 2023-07-21T18:51:12.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:51:12 smithi193 ceph-mon[98204]: osdmap e91: 12 total, 12 up, 12 in 2023-07-21T18:51:12.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:51:12 smithi155 ceph-mon[97934]: pgmap v294: 97 pgs: 97 active+clean; 228 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 1.7 MiB/s wr, 184 op/s 2023-07-21T18:51:12.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:51:12 smithi155 ceph-mon[97934]: osdmap e91: 12 total, 12 up, 12 in 2023-07-21T18:51:13.284 INFO:tasks.workunit.client.0.smithi039.stdout: 1 572610 5.90 MB/sec execute 386 sec latency 16.947 ms 2023-07-21T18:51:14.284 INFO:tasks.workunit.client.0.smithi039.stdout: 1 573579 5.90 MB/sec execute 387 sec latency 372.943 ms 2023-07-21T18:51:14.428 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:51:14 smithi039 ceph-mon[96489]: pgmap v296: 97 pgs: 97 active+clean; 228 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 951 KiB/s rd, 2.1 MiB/s wr, 224 op/s 2023-07-21T18:51:14.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:51:14 smithi193 ceph-mon[98204]: pgmap v296: 97 pgs: 97 active+clean; 228 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 951 KiB/s rd, 2.1 MiB/s wr, 224 op/s 2023-07-21T18:51:14.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:51:14 smithi155 ceph-mon[97934]: pgmap v296: 97 pgs: 97 active+clean; 228 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 951 KiB/s rd, 2.1 MiB/s wr, 224 op/s 2023-07-21T18:51:15.284 INFO:tasks.workunit.client.0.smithi039.stdout: 1 573579 5.88 MB/sec execute 388 sec latency 1372.995 ms 2023-07-21T18:51:16.284 INFO:tasks.workunit.client.0.smithi039.stdout: 1 573579 5.87 MB/sec execute 389 sec latency 2373.052 ms 2023-07-21T18:51:16.428 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:51:16 smithi039 ceph-mon[96489]: pgmap v297: 97 pgs: 97 active+clean; 228 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 1.7 MiB/s wr, 184 op/s 2023-07-21T18:51:16.428 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:51:16 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:51:16.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:51:16 smithi193 ceph-mon[98204]: pgmap v297: 97 pgs: 97 active+clean; 228 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 1.7 MiB/s wr, 184 op/s 2023-07-21T18:51:16.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:51:16 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:51:16.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:51:16 smithi155 ceph-mon[97934]: pgmap v297: 97 pgs: 97 active+clean; 228 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 1.7 MiB/s wr, 184 op/s 2023-07-21T18:51:16.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:51:16 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:51:17.284 INFO:tasks.workunit.client.0.smithi039.stdout: 1 574786 5.87 MB/sec execute 390 sec latency 2808.804 ms 2023-07-21T18:51:18.284 INFO:tasks.workunit.client.0.smithi039.stdout: 1 576042 5.87 MB/sec execute 391 sec latency 16.077 ms 2023-07-21T18:51:18.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.318+0000 7f26c6d25700 1 -- 172.21.15.39:0/734286395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26c0104a30 msgr2=0x7f26c0104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:51:18.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.318+0000 7f26c6d25700 1 --2- 172.21.15.39:0/734286395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26c0104a30 0x7f26c0104e00 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f26b4009990 tx=0x7f26b4009ca0 comp rx=0 tx=0).stop 2023-07-21T18:51:18.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.319+0000 7f26c6d25700 1 -- 172.21.15.39:0/734286395 shutdown_connections 2023-07-21T18:51:18.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.319+0000 7f26c6d25700 1 --2- 172.21.15.39:0/734286395 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26c00ff3e0 0x7f26c010b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:18.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.319+0000 7f26c6d25700 1 --2- 172.21.15.39:0/734286395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26c0104a30 0x7f26c0104e00 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:18.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.319+0000 7f26c6d25700 1 --2- 172.21.15.39:0/734286395 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26c00fea30 0x7f26c00feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:18.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.319+0000 7f26c6d25700 1 -- 172.21.15.39:0/734286395 >> 172.21.15.39:0/734286395 conn(0x7f26c0074760 msgr2=0x7f26c0074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:51:18.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.319+0000 7f26c6d25700 1 -- 172.21.15.39:0/734286395 shutdown_connections 2023-07-21T18:51:18.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.319+0000 7f26c6d25700 1 -- 172.21.15.39:0/734286395 wait complete. 2023-07-21T18:51:18.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.320+0000 7f26c6d25700 1 Processor -- start 2023-07-21T18:51:18.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.320+0000 7f26c6d25700 1 -- start start 2023-07-21T18:51:18.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.320+0000 7f26c6d25700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26c00fea30 0x7f26c019bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:18.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.320+0000 7f26c6d25700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26c00ff3e0 0x7f26c019c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:18.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.320+0000 7f26c6d25700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26c0104a30 0x7f26c01959b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:18.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.320+0000 7f26c6d25700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26c0196020 con 0x7f26c0104a30 2023-07-21T18:51:18.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.320+0000 7f26c6d25700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26c0196190 con 0x7f26c00ff3e0 2023-07-21T18:51:18.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.320+0000 7f26c6d25700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26c0196350 con 0x7f26c00fea30 2023-07-21T18:51:18.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.320+0000 7f26c52c2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26c0104a30 0x7f26c01959b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:18.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.320+0000 7f26c4ac1700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26c00fea30 0x7f26c019bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:18.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.320+0000 7f26c4ac1700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26c00fea30 0x7f26c019bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:50896/0 (socket says 172.21.15.39:50896) 2023-07-21T18:51:18.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.320+0000 7f26c52c2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26c0104a30 0x7f26c01959b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60580/0 (socket says 172.21.15.39:60580) 2023-07-21T18:51:18.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.320+0000 7f26c4ac1700 1 -- 172.21.15.39:0/2593122540 learned_addr learned my addr 172.21.15.39:0/2593122540 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:51:18.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.320+0000 7f26bffff700 1 --2- 172.21.15.39:0/2593122540 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26c00ff3e0 0x7f26c019c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:18.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.321+0000 7f26c4ac1700 1 -- 172.21.15.39:0/2593122540 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26c00ff3e0 msgr2=0x7f26c019c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:51:18.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.321+0000 7f26c4ac1700 1 --2- 172.21.15.39:0/2593122540 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26c00ff3e0 0x7f26c019c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:18.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.321+0000 7f26c4ac1700 1 -- 172.21.15.39:0/2593122540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26c0104a30 msgr2=0x7f26c01959b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:51:18.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.321+0000 7f26c4ac1700 1 --2- 172.21.15.39:0/2593122540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26c0104a30 0x7f26c01959b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:18.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.321+0000 7f26c4ac1700 1 -- 172.21.15.39:0/2593122540 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26b40095f0 con 0x7f26c00fea30 2023-07-21T18:51:18.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.321+0000 7f26c52c2700 1 --2- 172.21.15.39:0/2593122540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26c0104a30 0x7f26c01959b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:51:18.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.321+0000 7f26c4ac1700 1 --2- 172.21.15.39:0/2593122540 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26c00fea30 0x7f26c019bb50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f26b4000c00 tx=0x7f26b4009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:51:18.338 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.321+0000 7f26bdffb700 1 -- 172.21.15.39:0/2593122540 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b401d070 con 0x7f26c00fea30 2023-07-21T18:51:18.338 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.321+0000 7f26c6d25700 1 -- 172.21.15.39:0/2593122540 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26c01965d0 con 0x7f26c00fea30 2023-07-21T18:51:18.338 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.321+0000 7f26bdffb700 1 -- 172.21.15.39:0/2593122540 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26b4022470 con 0x7f26c00fea30 2023-07-21T18:51:18.338 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.321+0000 7f26bdffb700 1 -- 172.21.15.39:0/2593122540 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b4017630 con 0x7f26c00fea30 2023-07-21T18:51:18.338 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.321+0000 7f26c6d25700 1 -- 172.21.15.39:0/2593122540 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26c01a2b40 con 0x7f26c00fea30 2023-07-21T18:51:18.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.324+0000 7f26bdffb700 1 -- 172.21.15.39:0/2593122540 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f26b40228e0 con 0x7f26c00fea30 2023-07-21T18:51:18.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.324+0000 7f26bdffb700 1 --2- 172.21.15.39:0/2593122540 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f26b00a12c0 0x7f26b00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:18.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.324+0000 7f26bdffb700 1 -- 172.21.15.39:0/2593122540 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f26b40cde90 con 0x7f26c00fea30 2023-07-21T18:51:18.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.324+0000 7f26bffff700 1 --2- 172.21.15.39:0/2593122540 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f26b00a12c0 0x7f26b00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:18.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.325+0000 7f26bffff700 1 --2- 172.21.15.39:0/2593122540 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f26b00a12c0 0x7f26b00a3770 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f26c00fcb30 tx=0x7f26ac009260 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:51:18.341 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.325+0000 7f26d2eabb80 1 Processor -- start 2023-07-21T18:51:18.341 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.325+0000 7f26d2eabb80 1 -- start start 2023-07-21T18:51:18.341 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.325+0000 7f26d2eabb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5624663046b0 0x5624664add90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:18.341 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.325+0000 7f26d2eabb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624664ae2d0 0x5624664a8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:18.341 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.325+0000 7f26d2eabb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624664a8580 0x5624664a89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:18.341 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.325+0000 7f26d2eabb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624663adc50 con 0x5624663046b0 2023-07-21T18:51:18.341 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.325+0000 7f26d2eabb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624664009e0 con 0x5624664ae2d0 2023-07-21T18:51:18.341 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.325+0000 7f26d2eabb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624663f9e70 con 0x5624664a8580 2023-07-21T18:51:18.342 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.326+0000 7f26c4ac1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5624663046b0 0x5624664add90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:18.342 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.326+0000 7f26c4ac1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5624663046b0 0x5624664add90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60584/0 (socket says 172.21.15.39:60584) 2023-07-21T18:51:18.342 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.326+0000 7f26c4ac1700 1 -- 172.21.15.39:0/3042804882 learned_addr learned my addr 172.21.15.39:0/3042804882 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:51:18.342 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.326+0000 7f26bffff700 1 --2- 172.21.15.39:0/3042804882 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624664ae2d0 0x5624664a8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:18.342 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.326+0000 7f26c52c2700 1 --2- 172.21.15.39:0/3042804882 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624664a8580 0x5624664a89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:18.342 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.326+0000 7f26c4ac1700 1 -- 172.21.15.39:0/3042804882 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624664a8580 msgr2=0x5624664a89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:51:18.342 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.326+0000 7f26c4ac1700 1 --2- 172.21.15.39:0/3042804882 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624664a8580 0x5624664a89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:18.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.326+0000 7f26c4ac1700 1 -- 172.21.15.39:0/3042804882 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624664ae2d0 msgr2=0x5624664a8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:51:18.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.326+0000 7f26c4ac1700 1 --2- 172.21.15.39:0/3042804882 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624664ae2d0 0x5624664a8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:18.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.326+0000 7f26c4ac1700 1 -- 172.21.15.39:0/3042804882 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562466227d80 con 0x5624663046b0 2023-07-21T18:51:18.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.326+0000 7f26c4ac1700 1 --2- 172.21.15.39:0/3042804882 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5624663046b0 0x5624664add90 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f26b400b2e0 tx=0x7f26b409ab30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b5f6d237e707d8e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:51:18.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.326+0000 7f26aaffd700 1 -- 172.21.15.39:0/3042804882 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b409f020 con 0x5624663046b0 2023-07-21T18:51:18.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.326+0000 7f26aaffd700 1 -- 172.21.15.39:0/3042804882 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26b40a33b0 con 0x5624663046b0 2023-07-21T18:51:18.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.326+0000 7f26aaffd700 1 -- 172.21.15.39:0/3042804882 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b409a720 con 0x5624663046b0 2023-07-21T18:51:18.346 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.327+0000 7f26d2eabb80 1 -- 172.21.15.39:0/3042804882 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5624663046b0 msgr2=0x5624664add90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:51:18.347 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.327+0000 7f26d2eabb80 1 --2- 172.21.15.39:0/3042804882 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5624663046b0 0x5624664add90 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f26b400b2e0 tx=0x7f26b409ab30 comp rx=0 tx=0).stop 2023-07-21T18:51:18.347 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.327+0000 7f26d2eabb80 1 -- 172.21.15.39:0/3042804882 shutdown_connections 2023-07-21T18:51:18.347 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.327+0000 7f26d2eabb80 1 --2- 172.21.15.39:0/3042804882 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624664a8580 0x5624664a89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:18.347 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.327+0000 7f26d2eabb80 1 --2- 172.21.15.39:0/3042804882 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5624663046b0 0x5624664add90 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:18.347 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.327+0000 7f26d2eabb80 1 --2- 172.21.15.39:0/3042804882 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624664ae2d0 0x5624664a8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:18.347 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.327+0000 7f26d2eabb80 1 -- 172.21.15.39:0/3042804882 >> 172.21.15.39:0/3042804882 conn(0x56246649dbe0 msgr2=0x5624664a2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:51:18.348 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.327+0000 7f26d2eabb80 1 -- 172.21.15.39:0/3042804882 shutdown_connections 2023-07-21T18:51:18.348 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.327+0000 7f26d2eabb80 1 -- 172.21.15.39:0/3042804882 wait complete. 2023-07-21T18:51:18.348 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.327+0000 7f26d2eabb80 1 -- start start 2023-07-21T18:51:18.348 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.328+0000 7f26d2eabb80 1 Processor -- start 2023-07-21T18:51:18.348 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.328+0000 7f26d2eabb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624663046b0 0x5624664b7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:18.348 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.328+0000 7f26d2eabb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5624664a8580 0x5624664b7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:18.348 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.328+0000 7f26d2eabb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624664ae2d0 0x5624664b15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:18.348 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.328+0000 7f26d2eabb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x562466399a60 con 0x5624664a8580 2023-07-21T18:51:18.349 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.328+0000 7f26d2eabb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562466399760 con 0x5624663046b0 2023-07-21T18:51:18.349 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.328+0000 7f26d2eabb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x56246627d8c0 con 0x5624664ae2d0 2023-07-21T18:51:18.349 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.328+0000 7f26d2eabb80 10 client.0 _ll_register_callbacks cb 0x562466397700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:51:18.349 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.328+0000 7f26bffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5624664a8580 0x5624664b7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:18.349 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.328+0000 7f26bffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5624664a8580 0x5624664b7890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60592/0 (socket says 172.21.15.39:60592) 2023-07-21T18:51:18.349 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.328+0000 7f26c52c2700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624664ae2d0 0x5624664b15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:18.349 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.328+0000 7f26bffff700 1 -- 172.21.15.39:0/4093426444 learned_addr learned my addr 172.21.15.39:0/4093426444 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:51:18.350 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.328+0000 7f26c4ac1700 1 --2- 172.21.15.39:0/4093426444 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624663046b0 0x5624664b7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:18.350 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.328+0000 7f26bffff700 1 -- 172.21.15.39:0/4093426444 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624664ae2d0 msgr2=0x5624664b15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:51:18.350 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.328+0000 7f26bffff700 1 --2- 172.21.15.39:0/4093426444 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624664ae2d0 0x5624664b15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:18.350 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.328+0000 7f26bffff700 1 -- 172.21.15.39:0/4093426444 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624663046b0 msgr2=0x5624664b7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:51:18.350 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.328+0000 7f26bffff700 1 --2- 172.21.15.39:0/4093426444 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624663046b0 0x5624664b7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:18.350 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.328+0000 7f26bffff700 1 -- 172.21.15.39:0/4093426444 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26c01a2b40 con 0x5624664a8580 2023-07-21T18:51:18.350 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.328+0000 7f26c4ac1700 1 --2- 172.21.15.39:0/4093426444 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624663046b0 0x5624664b7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:51:18.350 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.328+0000 7f26c52c2700 1 --2- 172.21.15.39:0/4093426444 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624664ae2d0 0x5624664b15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:51:18.351 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.329+0000 7f26bffff700 1 --2- 172.21.15.39:0/4093426444 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5624664a8580 0x5624664b7890 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f26ac004580 tx=0x7f26ac0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bd70e83f899128c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:51:18.351 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.329+0000 7f269ffff700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:51:18.351 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.329+0000 7f269ffff700 1 -- 172.21.15.39:0/4093426444 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ac022070 con 0x5624664a8580 2023-07-21T18:51:18.351 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.329+0000 7f269ffff700 1 -- 172.21.15.39:0/4093426444 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f26ac01d410 con 0x5624664a8580 2023-07-21T18:51:18.351 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.329+0000 7f26d2eabb80 1 -- 172.21.15.39:0/4093426444 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562466353490 con 0x5624664a8580 2023-07-21T18:51:18.351 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.329+0000 7f269ffff700 1 -- 172.21.15.39:0/4093426444 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26ac0045b0 con 0x5624664a8580 2023-07-21T18:51:18.351 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.329+0000 7f26d2eabb80 10 client.16866 fetch_fsmap learned FSMap version 61 2023-07-21T18:51:18.351 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.329+0000 7f26d2eabb80 1 -- 172.21.15.39:0/4093426444 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x562466210c00 con 0x5624664a8580 2023-07-21T18:51:18.352 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.329+0000 7f269ffff700 1 -- 172.21.15.39:0/4093426444 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26ac033080 con 0x5624664a8580 2023-07-21T18:51:18.352 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.329+0000 7f269ffff700 1 -- 172.21.15.39:0/4093426444 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 61) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f26ac02d030 con 0x5624664a8580 2023-07-21T18:51:18.352 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.329+0000 7f26d2eabb80 10 client.16866 fetch_fsmap finished waiting for FSMap version 61 2023-07-21T18:51:18.352 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.329+0000 7f26d2eabb80 10 client.16866 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:51:18.352 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.330+0000 7f26d2eabb80 20 client.16866 populate_metadata read hostname 'smithi039' 2023-07-21T18:51:18.352 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.330+0000 7f26d2eabb80 1 --2- 172.21.15.39:0/4093426444 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5624664c19f0 0x5624664c3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:18.352 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.330+0000 7f26d2eabb80 4 client.16866 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:51:18.353 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.330+0000 7f26d2eabb80 1 -- 172.21.15.39:0/4093426444 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5624663f2e50 con 0x5624664c19f0 2023-07-21T18:51:18.353 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.330+0000 7f26c4ac1700 1 --2- 172.21.15.39:0/4093426444 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5624664c19f0 0x5624664c3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:18.353 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.330+0000 7f26c4ac1700 1 --2- 172.21.15.39:0/4093426444 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5624664c19f0 0x5624664c3e00 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7671ce347998d94 server_cookie=14d60dc30eeb1a64 in_seq=0 out_seq=0 2023-07-21T18:51:18.353 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.330+0000 7f269ffff700 10 client.16866 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:51:18.353 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.331+0000 7f269ffff700 1 -- 172.21.15.39:0/4093426444 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5624663f2e50 con 0x5624664c19f0 2023-07-21T18:51:18.353 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.331+0000 7f269ffff700 10 client.16866 handle_command_reply: tid=0 2023-07-21T18:51:18.353 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.331+0000 7f26d2eabb80 1 client.16866 shutdown 2023-07-21T18:51:18.353 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.331+0000 7f26d2eabb80 1 -- 172.21.15.39:0/4093426444 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5624664a8580 msgr2=0x5624664b7890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:51:18.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.331+0000 7f26d2eabb80 1 --2- 172.21.15.39:0/4093426444 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5624664a8580 0x5624664b7890 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f26ac004580 tx=0x7f26ac0047c0 comp rx=0 tx=0).stop 2023-07-21T18:51:18.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.331+0000 7f26d2eabb80 1 -- 172.21.15.39:0/4093426444 shutdown_connections 2023-07-21T18:51:18.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.331+0000 7f26d2eabb80 1 --2- 172.21.15.39:0/4093426444 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5624664c19f0 0x5624664c3e00 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:18.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.331+0000 7f26d2eabb80 1 --2- 172.21.15.39:0/4093426444 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624663046b0 0x5624664b7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:18.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.331+0000 7f26d2eabb80 1 --2- 172.21.15.39:0/4093426444 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624664ae2d0 0x5624664b15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:18.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.331+0000 7f269ffff700 0 client.16866 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:51:18.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.331+0000 7f26d2eabb80 1 --2- 172.21.15.39:0/4093426444 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5624664a8580 0x5624664b7890 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:18.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.331+0000 7f26d2eabb80 1 -- 172.21.15.39:0/4093426444 >> 172.21.15.39:0/4093426444 conn(0x56246649dbe0 msgr2=0x5624664a24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:51:18.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.331+0000 7f26d2eabb80 1 -- 172.21.15.39:0/4093426444 shutdown_connections 2023-07-21T18:51:18.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.331+0000 7f26d2eabb80 1 -- 172.21.15.39:0/4093426444 wait complete. 2023-07-21T18:51:18.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.332+0000 7f26d2eabb80 20 client.16866 trim_cache size 0 max 16384 2023-07-21T18:51:18.361 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.348+0000 7f26d2eabb80 1 Processor -- start 2023-07-21T18:51:18.362 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.348+0000 7f26d2eabb80 1 -- start start 2023-07-21T18:51:18.363 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.349+0000 7f26d2eabb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5624663046b0 0x5624664a8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:18.363 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.349+0000 7f26d2eabb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624664a8580 0x5624664af130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:18.363 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.349+0000 7f26d2eabb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624664ae2d0 0x5624664a2c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:18.363 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.349+0000 7f26d2eabb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624664beed0 con 0x5624663046b0 2023-07-21T18:51:18.363 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.349+0000 7f26d2eabb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624664b2bb0 con 0x5624664ae2d0 2023-07-21T18:51:18.364 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.349+0000 7f26d2eabb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624664b2730 con 0x5624664a8580 2023-07-21T18:51:18.364 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.349+0000 7f26c4ac1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5624663046b0 0x5624664a8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:18.364 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.349+0000 7f26bffff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624664a8580 0x5624664af130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:18.364 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.349+0000 7f26c52c2700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624664ae2d0 0x5624664a2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:18.364 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.349+0000 7f26bffff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624664a8580 0x5624664af130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:50920/0 (socket says 172.21.15.39:50920) 2023-07-21T18:51:18.364 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.349+0000 7f26c4ac1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5624663046b0 0x5624664a8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60598/0 (socket says 172.21.15.39:60598) 2023-07-21T18:51:18.364 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.349+0000 7f26c52c2700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624664ae2d0 0x5624664a2c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:44966/0 (socket says 172.21.15.39:44966) 2023-07-21T18:51:18.365 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.349+0000 7f26bffff700 1 -- 172.21.15.39:0/3937353655 learned_addr learned my addr 172.21.15.39:0/3937353655 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:51:18.365 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.349+0000 7f26bffff700 1 -- 172.21.15.39:0/3937353655 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624664ae2d0 msgr2=0x5624664a2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:51:18.365 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.349+0000 7f26bffff700 1 --2- 172.21.15.39:0/3937353655 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624664ae2d0 0x5624664a2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:18.365 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.349+0000 7f26bffff700 1 -- 172.21.15.39:0/3937353655 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5624663046b0 msgr2=0x5624664a8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:51:18.365 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.349+0000 7f26bffff700 1 --2- 172.21.15.39:0/3937353655 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5624663046b0 0x5624664a8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:18.365 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.349+0000 7f26bffff700 1 -- 172.21.15.39:0/3937353655 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5624664b1110 con 0x5624664a8580 2023-07-21T18:51:18.365 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.349+0000 7f26c4ac1700 1 --2- 172.21.15.39:0/3937353655 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5624663046b0 0x5624664a8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:51:18.365 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.349+0000 7f26bffff700 1 --2- 172.21.15.39:0/3937353655 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624664a8580 0x5624664af130 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f26ac009b20 tx=0x7f26ac003c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32208f91a6e1ab46 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:51:18.366 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.349+0000 7f26ab7fe700 1 -- 172.21.15.39:0/3937353655 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ac036070 con 0x5624664a8580 2023-07-21T18:51:18.366 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.350+0000 7f26ab7fe700 1 -- 172.21.15.39:0/3937353655 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562466353490 con 0x5624664a8580 2023-07-21T18:51:18.366 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.350+0000 7f26ab7fe700 1 -- 172.21.15.39:0/3937353655 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ac022030 con 0x5624664a8580 2023-07-21T18:51:18.368 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.350+0000 7f26d2eabb80 1 -- 172.21.15.39:0/3937353655 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624664a8580 msgr2=0x5624664af130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:51:18.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.350+0000 7f26d2eabb80 1 --2- 172.21.15.39:0/3937353655 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624664a8580 0x5624664af130 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f26ac009b20 tx=0x7f26ac003c40 comp rx=0 tx=0).stop 2023-07-21T18:51:18.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.350+0000 7f26d2eabb80 1 -- 172.21.15.39:0/3937353655 shutdown_connections 2023-07-21T18:51:18.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.350+0000 7f26d2eabb80 1 --2- 172.21.15.39:0/3937353655 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624664ae2d0 0x5624664a2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:18.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.350+0000 7f26d2eabb80 1 --2- 172.21.15.39:0/3937353655 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5624663046b0 0x5624664a8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:18.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.350+0000 7f26d2eabb80 1 --2- 172.21.15.39:0/3937353655 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624664a8580 0x5624664af130 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:18.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.350+0000 7f26d2eabb80 1 -- 172.21.15.39:0/3937353655 >> 172.21.15.39:0/3937353655 conn(0x56246649dbe0 msgr2=0x5624664c8720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:51:18.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.350+0000 7f26d2eabb80 1 -- 172.21.15.39:0/3937353655 shutdown_connections 2023-07-21T18:51:18.370 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.350+0000 7f26d2eabb80 1 -- 172.21.15.39:0/3937353655 wait complete. 2023-07-21T18:51:18.370 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.351+0000 7f26d2eabb80 1 -- start start 2023-07-21T18:51:18.370 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.351+0000 7f26d2eabb80 1 Processor -- start 2023-07-21T18:51:18.370 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.351+0000 7f26d2eabb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624663046b0 0x5624664b7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:18.370 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.351+0000 7f26d2eabb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624664a8580 0x5624664b7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:18.370 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.351+0000 7f26d2eabb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5624664ae2d0 0x5624664b54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:18.370 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.351+0000 7f26d2eabb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624664b5ab0 con 0x5624664ae2d0 2023-07-21T18:51:18.370 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.351+0000 7f26d2eabb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624664b5c20 con 0x5624663046b0 2023-07-21T18:51:18.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.351+0000 7f26d2eabb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624664b5de0 con 0x5624664a8580 2023-07-21T18:51:18.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.351+0000 7f26d2eabb80 10 client.0 _ll_register_callbacks cb 0x7f26ac0043e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:51:18.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.351+0000 7f26c4ac1700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624663046b0 0x5624664b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:18.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.351+0000 7f26c52c2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5624664ae2d0 0x5624664b54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:18.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.351+0000 7f26c4ac1700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624663046b0 0x5624664b7040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:44968/0 (socket says 172.21.15.39:44968) 2023-07-21T18:51:18.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.351+0000 7f26c4ac1700 1 -- 172.21.15.39:0/1750737623 learned_addr learned my addr 172.21.15.39:0/1750737623 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:51:18.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.351+0000 7f26bffff700 1 --2- 172.21.15.39:0/1750737623 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624664a8580 0x5624664b7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:18.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.352+0000 7f26c52c2700 1 -- 172.21.15.39:0/1750737623 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624664a8580 msgr2=0x5624664b7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:51:18.372 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.352+0000 7f26c52c2700 1 --2- 172.21.15.39:0/1750737623 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624664a8580 0x5624664b7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:18.372 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.352+0000 7f26c52c2700 1 -- 172.21.15.39:0/1750737623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624663046b0 msgr2=0x5624664b7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:51:18.372 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.352+0000 7f26c52c2700 1 --2- 172.21.15.39:0/1750737623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624663046b0 0x5624664b7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:18.372 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.352+0000 7f26c52c2700 1 -- 172.21.15.39:0/1750737623 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562466227d80 con 0x5624664ae2d0 2023-07-21T18:51:18.372 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.352+0000 7f26c4ac1700 1 --2- 172.21.15.39:0/1750737623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624663046b0 0x5624664b7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:51:18.372 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.352+0000 7f26c52c2700 1 --2- 172.21.15.39:0/1750737623 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5624664ae2d0 0x5624664b54e0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f26b80094f0 tx=0x7f26b8004630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b89d4787d9140fdf server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:51:18.372 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.352+0000 7f269ffff700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:51:18.373 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.352+0000 7f269ffff700 1 -- 172.21.15.39:0/1750737623 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b8004db0 con 0x5624664ae2d0 2023-07-21T18:51:18.373 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.352+0000 7f269ffff700 1 -- 172.21.15.39:0/1750737623 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f26b8007740 con 0x5624664ae2d0 2023-07-21T18:51:18.373 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.352+0000 7f26d2eabb80 1 -- 172.21.15.39:0/1750737623 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f26b40a33b0 con 0x5624664ae2d0 2023-07-21T18:51:18.373 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.352+0000 7f269ffff700 1 -- 172.21.15.39:0/1750737623 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26b8007900 con 0x5624664ae2d0 2023-07-21T18:51:18.373 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.352+0000 7f26d2eabb80 10 client.16872 fetch_fsmap learned FSMap version 61 2023-07-21T18:51:18.373 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.352+0000 7f26d2eabb80 1 -- 172.21.15.39:0/1750737623 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x562466210c00 con 0x5624664ae2d0 2023-07-21T18:51:18.373 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.352+0000 7f269ffff700 1 -- 172.21.15.39:0/1750737623 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26b40a33b0 con 0x5624664ae2d0 2023-07-21T18:51:18.373 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.353+0000 7f269ffff700 1 -- 172.21.15.39:0/1750737623 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 61) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f26b800ecf0 con 0x5624664ae2d0 2023-07-21T18:51:18.374 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.353+0000 7f26d2eabb80 10 client.16872 fetch_fsmap finished waiting for FSMap version 61 2023-07-21T18:51:18.374 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.353+0000 7f26d2eabb80 10 client.16872 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:51:18.374 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.353+0000 7f26d2eabb80 20 client.16872 populate_metadata read hostname 'smithi039' 2023-07-21T18:51:18.374 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.353+0000 7f26d2eabb80 1 --2- 172.21.15.39:0/1750737623 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x562466516b60 0x562466518f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:18.374 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.353+0000 7f26d2eabb80 4 client.16872 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:51:18.374 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.353+0000 7f26d2eabb80 1 -- 172.21.15.39:0/1750737623 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5624663b4ff0 con 0x562466516b60 2023-07-21T18:51:18.374 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.353+0000 7f26c4ac1700 1 --2- 172.21.15.39:0/1750737623 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x562466516b60 0x562466518f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:18.374 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.353+0000 7f269ffff700 10 client.16872 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:51:18.375 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.353+0000 7f26c4ac1700 1 --2- 172.21.15.39:0/1750737623 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x562466516b60 0x562466518f70 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfec550369d42f91 server_cookie=712ff5314256f128 in_seq=0 out_seq=0 2023-07-21T18:51:18.398 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.385+0000 7f269ffff700 1 -- 172.21.15.39:0/1750737623 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5624663b4ff0 con 0x562466516b60 2023-07-21T18:51:18.399 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.385+0000 7f269ffff700 10 client.16872 handle_command_reply: tid=0 2023-07-21T18:51:18.399 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.385+0000 7f26d2eabb80 1 client.16872 shutdown 2023-07-21T18:51:18.399 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.385+0000 7f26d2eabb80 1 -- 172.21.15.39:0/1750737623 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5624664ae2d0 msgr2=0x5624664b54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:51:18.399 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.385+0000 7f26d2eabb80 1 --2- 172.21.15.39:0/1750737623 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5624664ae2d0 0x5624664b54e0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f26b80094f0 tx=0x7f26b8004630 comp rx=0 tx=0).stop 2023-07-21T18:51:18.399 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.385+0000 7f26d2eabb80 1 -- 172.21.15.39:0/1750737623 shutdown_connections 2023-07-21T18:51:18.399 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.385+0000 7f26d2eabb80 1 --2- 172.21.15.39:0/1750737623 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x562466516b60 0x562466518f70 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:18.400 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.385+0000 7f26d2eabb80 1 --2- 172.21.15.39:0/1750737623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624663046b0 0x5624664b7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:18.400 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.385+0000 7f26d2eabb80 1 --2- 172.21.15.39:0/1750737623 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5624664ae2d0 0x5624664b54e0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:18.400 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.385+0000 7f269ffff700 0 client.16872 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:51:18.400 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.385+0000 7f26d2eabb80 1 --2- 172.21.15.39:0/1750737623 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624664a8580 0x5624664b7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:18.400 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.385+0000 7f26d2eabb80 1 -- 172.21.15.39:0/1750737623 >> 172.21.15.39:0/1750737623 conn(0x56246649dbe0 msgr2=0x5624664ac900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:51:18.400 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.385+0000 7f26d2eabb80 1 -- 172.21.15.39:0/1750737623 shutdown_connections 2023-07-21T18:51:18.400 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.385+0000 7f26d2eabb80 1 -- 172.21.15.39:0/1750737623 wait complete. 2023-07-21T18:51:18.400 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.385+0000 7f26d2eabb80 20 client.16872 trim_cache size 0 max 16384 2023-07-21T18:51:18.401 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:51:18.401 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:51:18.401 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:51:18.401 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:51:18.402 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.386+0000 7f26c6d25700 1 -- 172.21.15.39:0/2593122540 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f26b00a12c0 msgr2=0x7f26b00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:51:18.402 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.386+0000 7f26c6d25700 1 --2- 172.21.15.39:0/2593122540 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f26b00a12c0 0x7f26b00a3770 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f26c00fcb30 tx=0x7f26ac009260 comp rx=0 tx=0).stop 2023-07-21T18:51:18.402 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.386+0000 7f26c6d25700 1 -- 172.21.15.39:0/2593122540 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26c00fea30 msgr2=0x7f26c019bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:51:18.403 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.386+0000 7f26c6d25700 1 --2- 172.21.15.39:0/2593122540 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26c00fea30 0x7f26c019bb50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f26b4000c00 tx=0x7f26b4009580 comp rx=0 tx=0).stop 2023-07-21T18:51:18.403 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.386+0000 7f26c6d25700 1 -- 172.21.15.39:0/2593122540 shutdown_connections 2023-07-21T18:51:18.403 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.386+0000 7f26c6d25700 1 --2- 172.21.15.39:0/2593122540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26c0104a30 0x7f26c01959b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:18.403 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.386+0000 7f26c6d25700 1 --2- 172.21.15.39:0/2593122540 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26c00fea30 0x7f26c019bb50 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:18.403 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.386+0000 7f26c6d25700 1 --2- 172.21.15.39:0/2593122540 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f26b00a12c0 0x7f26b00a3770 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:18.403 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.386+0000 7f26c6d25700 1 --2- 172.21.15.39:0/2593122540 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26c00ff3e0 0x7f26c019c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:18.403 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.386+0000 7f26c6d25700 1 -- 172.21.15.39:0/2593122540 >> 172.21.15.39:0/2593122540 conn(0x7f26c0074760 msgr2=0x7f26c00fd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:51:18.403 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.386+0000 7f26c6d25700 1 -- 172.21.15.39:0/2593122540 shutdown_connections 2023-07-21T18:51:18.404 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:18.386+0000 7f26c6d25700 1 -- 172.21.15.39:0/2593122540 wait complete. 2023-07-21T18:51:18.427 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:51:18 smithi039 ceph-mon[96489]: pgmap v298: 97 pgs: 97 active+clean; 232 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 1.4 MiB/s wr, 136 op/s 2023-07-21T18:51:18.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:51:18 smithi193 ceph-mon[98204]: pgmap v298: 97 pgs: 97 active+clean; 232 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 1.4 MiB/s wr, 136 op/s 2023-07-21T18:51:18.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:51:18 smithi155 ceph-mon[97934]: pgmap v298: 97 pgs: 97 active+clean; 232 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 1.4 MiB/s wr, 136 op/s 2023-07-21T18:51:19.284 INFO:tasks.workunit.client.0.smithi039.stdout: 1 577126 5.87 MB/sec execute 392 sec latency 285.223 ms 2023-07-21T18:51:20.284 INFO:tasks.workunit.client.0.smithi039.stdout: 1 577126 5.86 MB/sec execute 393 sec latency 1285.321 ms 2023-07-21T18:51:20.428 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:51:20 smithi039 ceph-mon[96489]: pgmap v299: 97 pgs: 97 active+clean; 232 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 1.4 MiB/s wr, 136 op/s 2023-07-21T18:51:20.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:51:20 smithi193 ceph-mon[98204]: pgmap v299: 97 pgs: 97 active+clean; 232 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 1.4 MiB/s wr, 136 op/s 2023-07-21T18:51:20.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:51:20 smithi155 ceph-mon[97934]: pgmap v299: 97 pgs: 97 active+clean; 232 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 1.4 MiB/s wr, 136 op/s 2023-07-21T18:51:21.284 INFO:tasks.workunit.client.0.smithi039.stdout: 1 577126 5.84 MB/sec execute 394 sec latency 2285.420 ms 2023-07-21T18:51:22.284 INFO:tasks.workunit.client.0.smithi039.stdout: 1 578335 5.84 MB/sec execute 395 sec latency 2717.630 ms 2023-07-21T18:51:22.427 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:51:22 smithi039 ceph-mon[96489]: pgmap v300: 97 pgs: 97 active+clean; 231 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 1.6 MiB/s wr, 154 op/s 2023-07-21T18:51:22.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:51:22 smithi193 ceph-mon[98204]: pgmap v300: 97 pgs: 97 active+clean; 231 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 1.6 MiB/s wr, 154 op/s 2023-07-21T18:51:22.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:51:22 smithi155 ceph-mon[97934]: pgmap v300: 97 pgs: 97 active+clean; 231 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 1.6 MiB/s wr, 154 op/s 2023-07-21T18:51:23.284 INFO:tasks.workunit.client.0.smithi039.stdout: 1 579597 5.85 MB/sec execute 396 sec latency 12.326 ms 2023-07-21T18:51:24.285 INFO:tasks.workunit.client.0.smithi039.stdout: 1 580669 5.85 MB/sec execute 397 sec latency 151.184 ms 2023-07-21T18:51:24.428 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:51:24 smithi039 ceph-mon[96489]: pgmap v301: 97 pgs: 97 active+clean; 231 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 1.4 MiB/s wr, 136 op/s 2023-07-21T18:51:24.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:51:24 smithi193 ceph-mon[98204]: pgmap v301: 97 pgs: 97 active+clean; 231 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 1.4 MiB/s wr, 136 op/s 2023-07-21T18:51:24.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:51:24 smithi155 ceph-mon[97934]: pgmap v301: 97 pgs: 97 active+clean; 231 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 1.4 MiB/s wr, 136 op/s 2023-07-21T18:51:25.284 INFO:tasks.workunit.client.0.smithi039.stdout: 1 580669 5.83 MB/sec execute 398 sec latency 1151.270 ms 2023-07-21T18:51:26.285 INFO:tasks.workunit.client.0.smithi039.stdout: 1 581040 5.82 MB/sec execute 399 sec latency 1947.761 ms 2023-07-21T18:51:26.427 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:51:26 smithi039 ceph-mon[96489]: pgmap v302: 97 pgs: 97 active+clean; 231 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 1.3 MiB/s wr, 128 op/s 2023-07-21T18:51:26.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:51:26 smithi193 ceph-mon[98204]: pgmap v302: 97 pgs: 97 active+clean; 231 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 1.3 MiB/s wr, 128 op/s 2023-07-21T18:51:26.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:51:26 smithi155 ceph-mon[97934]: pgmap v302: 97 pgs: 97 active+clean; 231 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 1.3 MiB/s wr, 128 op/s 2023-07-21T18:51:27.285 INFO:tasks.workunit.client.0.smithi039.stdout: 1 582769 5.84 MB/sec execute 400 sec latency 13.617 ms 2023-07-21T18:51:27.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:51:27 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:51:27.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:51:27 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:51:27.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:51:27 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:51:28.285 INFO:tasks.workunit.client.0.smithi039.stdout: 1 583982 5.84 MB/sec execute 401 sec latency 18.921 ms 2023-07-21T18:51:28.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:51:28 smithi193 ceph-mon[98204]: pgmap v303: 97 pgs: 97 active+clean; 234 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 1.9 MiB/s wr, 184 op/s 2023-07-21T18:51:28.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:51:28 smithi155 ceph-mon[97934]: pgmap v303: 97 pgs: 97 active+clean; 234 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 1.9 MiB/s wr, 184 op/s 2023-07-21T18:51:28.678 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:51:28 smithi039 ceph-mon[96489]: pgmap v303: 97 pgs: 97 active+clean; 234 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 1.9 MiB/s wr, 184 op/s 2023-07-21T18:51:29.285 INFO:tasks.workunit.client.0.smithi039.stdout: 1 584222 5.82 MB/sec execute 402 sec latency 693.256 ms 2023-07-21T18:51:30.286 INFO:tasks.workunit.client.0.smithi039.stdout: 1 584222 5.81 MB/sec execute 403 sec latency 1693.346 ms 2023-07-21T18:51:30.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:51:30 smithi193 ceph-mon[98204]: pgmap v304: 97 pgs: 97 active+clean; 234 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 1.3 MiB/s wr, 125 op/s 2023-07-21T18:51:30.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:51:30 smithi155 ceph-mon[97934]: pgmap v304: 97 pgs: 97 active+clean; 234 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 1.3 MiB/s wr, 125 op/s 2023-07-21T18:51:30.678 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:51:30 smithi039 ceph-mon[96489]: pgmap v304: 97 pgs: 97 active+clean; 234 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 1.3 MiB/s wr, 125 op/s 2023-07-21T18:51:31.285 INFO:tasks.workunit.client.0.smithi039.stdout: 1 584222 5.79 MB/sec execute 404 sec latency 2693.401 ms 2023-07-21T18:51:31.524 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.511+0000 7ff1a6f43700 1 -- 172.21.15.39:0/4095284706 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff1a0104a30 msgr2=0x7ff1a0104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:51:31.524 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.511+0000 7ff1a6f43700 1 --2- 172.21.15.39:0/4095284706 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff1a0104a30 0x7ff1a0104e00 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7ff19c009990 tx=0x7ff19c009ca0 comp rx=0 tx=0).stop 2023-07-21T18:51:31.524 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.511+0000 7ff1a6f43700 1 -- 172.21.15.39:0/4095284706 shutdown_connections 2023-07-21T18:51:31.525 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.511+0000 7ff1a6f43700 1 --2- 172.21.15.39:0/4095284706 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff1a00ff3e0 0x7ff1a010b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:31.525 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.511+0000 7ff1a6f43700 1 --2- 172.21.15.39:0/4095284706 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff1a0104a30 0x7ff1a0104e00 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:31.525 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.511+0000 7ff1a6f43700 1 --2- 172.21.15.39:0/4095284706 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1a00fea30 0x7ff1a00feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:31.525 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.511+0000 7ff1a6f43700 1 -- 172.21.15.39:0/4095284706 >> 172.21.15.39:0/4095284706 conn(0x7ff1a0074760 msgr2=0x7ff1a0074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:51:31.525 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.511+0000 7ff1a6f43700 1 -- 172.21.15.39:0/4095284706 shutdown_connections 2023-07-21T18:51:31.525 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.511+0000 7ff1a6f43700 1 -- 172.21.15.39:0/4095284706 wait complete. 2023-07-21T18:51:31.527 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.511+0000 7ff1a6f43700 1 Processor -- start 2023-07-21T18:51:31.527 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.511+0000 7ff1a6f43700 1 -- start start 2023-07-21T18:51:31.527 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.512+0000 7ff1a6f43700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff1a00fea30 0x7ff1a019bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:31.527 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.512+0000 7ff1a6f43700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1a00ff3e0 0x7ff1a019c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:31.527 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.512+0000 7ff1a6f43700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff1a0104a30 0x7ff1a0195a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:31.527 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.512+0000 7ff1a6f43700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a0196080 con 0x7ff1a0104a30 2023-07-21T18:51:31.527 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.512+0000 7ff1a6f43700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a01961f0 con 0x7ff1a00ff3e0 2023-07-21T18:51:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.512+0000 7ff1a6f43700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a01963b0 con 0x7ff1a00fea30 2023-07-21T18:51:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.512+0000 7ff1a54e0700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff1a0104a30 0x7ff1a0195a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.512+0000 7ff1a54e0700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff1a0104a30 0x7ff1a0195a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60622/0 (socket says 172.21.15.39:60622) 2023-07-21T18:51:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.512+0000 7ff1a4cdf700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff1a00fea30 0x7ff1a019bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.512+0000 7ff1a54e0700 1 -- 172.21.15.39:0/216592363 learned_addr learned my addr 172.21.15.39:0/216592363 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:51:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.512+0000 7ff197fff700 1 --2- 172.21.15.39:0/216592363 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1a00ff3e0 0x7ff1a019c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.512+0000 7ff1a54e0700 1 -- 172.21.15.39:0/216592363 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff1a00fea30 msgr2=0x7ff1a019bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:51:31.529 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.512+0000 7ff1a54e0700 1 --2- 172.21.15.39:0/216592363 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff1a00fea30 0x7ff1a019bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:31.529 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.512+0000 7ff1a54e0700 1 -- 172.21.15.39:0/216592363 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1a00ff3e0 msgr2=0x7ff1a019c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:51:31.529 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.512+0000 7ff1a54e0700 1 --2- 172.21.15.39:0/216592363 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1a00ff3e0 0x7ff1a019c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:31.529 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.512+0000 7ff1a54e0700 1 -- 172.21.15.39:0/216592363 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff19c0095f0 con 0x7ff1a0104a30 2023-07-21T18:51:31.529 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.512+0000 7ff1a54e0700 1 --2- 172.21.15.39:0/216592363 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff1a0104a30 0x7ff1a0195a10 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7ff19800ef30 tx=0x7ff19800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:51:31.529 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.512+0000 7ff195ffb700 1 -- 172.21.15.39:0/216592363 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff198018070 con 0x7ff1a0104a30 2023-07-21T18:51:31.529 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.512+0000 7ff1a6f43700 1 -- 172.21.15.39:0/216592363 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1a0196630 con 0x7ff1a0104a30 2023-07-21T18:51:31.529 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.512+0000 7ff195ffb700 1 -- 172.21.15.39:0/216592363 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff19800f040 con 0x7ff1a0104a30 2023-07-21T18:51:31.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.512+0000 7ff195ffb700 1 -- 172.21.15.39:0/216592363 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19800cd00 con 0x7ff1a0104a30 2023-07-21T18:51:31.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.513+0000 7ff1a6f43700 1 -- 172.21.15.39:0/216592363 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1a01a2b50 con 0x7ff1a0104a30 2023-07-21T18:51:31.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.515+0000 7ff195ffb700 1 -- 172.21.15.39:0/216592363 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7ff198003a00 con 0x7ff1a0104a30 2023-07-21T18:51:31.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.515+0000 7ff195ffb700 1 --2- 172.21.15.39:0/216592363 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7ff1880a12c0 0x7ff1880a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:31.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.515+0000 7ff195ffb700 1 -- 172.21.15.39:0/216592363 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8527+0+0 (secure 0 0 0) 0x7ff1980cc870 con 0x7ff1a0104a30 2023-07-21T18:51:31.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.515+0000 7ff1a4cdf700 1 --2- 172.21.15.39:0/216592363 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7ff1880a12c0 0x7ff1880a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:31.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.516+0000 7ff1a4cdf700 1 --2- 172.21.15.39:0/216592363 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7ff1880a12c0 0x7ff1880a3770 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff19c00b2e0 tx=0x7ff19c005fd0 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:51:31.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.516+0000 7ff1b30c9b80 1 Processor -- start 2023-07-21T18:51:31.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.516+0000 7ff1b30c9b80 1 -- start start 2023-07-21T18:51:31.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.517+0000 7ff1b30c9b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d7a0dd96b0 0x55d7a0f82d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:31.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.517+0000 7ff1b30c9b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d7a0f832d0 0x55d7a0f7d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:31.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.517+0000 7ff1b30c9b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7a0f7d580 0x55d7a0f7d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:31.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.517+0000 7ff1b30c9b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7a0e82c50 con 0x55d7a0f832d0 2023-07-21T18:51:31.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.517+0000 7ff1b30c9b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7a0ed59e0 con 0x55d7a0dd96b0 2023-07-21T18:51:31.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.517+0000 7ff1b30c9b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7a0ecee70 con 0x55d7a0f7d580 2023-07-21T18:51:31.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.517+0000 7ff197fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d7a0f832d0 0x55d7a0f7d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:31.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.517+0000 7ff197fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d7a0f832d0 0x55d7a0f7d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60628/0 (socket says 172.21.15.39:60628) 2023-07-21T18:51:31.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.517+0000 7ff197fff700 1 -- 172.21.15.39:0/2572230209 learned_addr learned my addr 172.21.15.39:0/2572230209 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:51:31.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.517+0000 7ff1a4cdf700 1 --2- 172.21.15.39:0/2572230209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d7a0dd96b0 0x55d7a0f82d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:31.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.517+0000 7ff1a54e0700 1 --2- 172.21.15.39:0/2572230209 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7a0f7d580 0x55d7a0f7d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:31.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.517+0000 7ff197fff700 1 -- 172.21.15.39:0/2572230209 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7a0f7d580 msgr2=0x55d7a0f7d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:51:31.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.517+0000 7ff197fff700 1 --2- 172.21.15.39:0/2572230209 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7a0f7d580 0x55d7a0f7d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:31.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.517+0000 7ff197fff700 1 -- 172.21.15.39:0/2572230209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d7a0dd96b0 msgr2=0x55d7a0f82d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:51:31.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.517+0000 7ff197fff700 1 --2- 172.21.15.39:0/2572230209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d7a0dd96b0 0x55d7a0f82d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:31.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.517+0000 7ff197fff700 1 -- 172.21.15.39:0/2572230209 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7a0cfcd80 con 0x55d7a0f832d0 2023-07-21T18:51:31.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.517+0000 7ff197fff700 1 --2- 172.21.15.39:0/2572230209 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d7a0f832d0 0x55d7a0f7d040 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7ff19000bd30 tx=0x7ff1900106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27971fc8c6f3755d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:51:31.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.518+0000 7ff186ffd700 1 -- 172.21.15.39:0/2572230209 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff190010e10 con 0x55d7a0f832d0 2023-07-21T18:51:31.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.518+0000 7ff186ffd700 1 -- 172.21.15.39:0/2572230209 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff190015070 con 0x55d7a0f832d0 2023-07-21T18:51:31.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.518+0000 7ff186ffd700 1 -- 172.21.15.39:0/2572230209 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff190003e00 con 0x55d7a0f832d0 2023-07-21T18:51:31.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.518+0000 7ff1b30c9b80 1 -- 172.21.15.39:0/2572230209 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d7a0f832d0 msgr2=0x55d7a0f7d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:51:31.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.518+0000 7ff1b30c9b80 1 --2- 172.21.15.39:0/2572230209 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d7a0f832d0 0x55d7a0f7d040 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7ff19000bd30 tx=0x7ff1900106b0 comp rx=0 tx=0).stop 2023-07-21T18:51:31.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.518+0000 7ff1b30c9b80 1 -- 172.21.15.39:0/2572230209 shutdown_connections 2023-07-21T18:51:31.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.518+0000 7ff1b30c9b80 1 --2- 172.21.15.39:0/2572230209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d7a0dd96b0 0x55d7a0f82d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:31.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.518+0000 7ff1b30c9b80 1 --2- 172.21.15.39:0/2572230209 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7a0f7d580 0x55d7a0f7d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:31.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.518+0000 7ff1b30c9b80 1 --2- 172.21.15.39:0/2572230209 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d7a0f832d0 0x55d7a0f7d040 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:31.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.518+0000 7ff1b30c9b80 1 -- 172.21.15.39:0/2572230209 >> 172.21.15.39:0/2572230209 conn(0x55d7a0f72be0 msgr2=0x55d7a0f77f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:51:31.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.518+0000 7ff1b30c9b80 1 -- 172.21.15.39:0/2572230209 shutdown_connections 2023-07-21T18:51:31.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.518+0000 7ff1b30c9b80 1 -- 172.21.15.39:0/2572230209 wait complete. 2023-07-21T18:51:31.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.519+0000 7ff1b30c9b80 1 -- start start 2023-07-21T18:51:31.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.519+0000 7ff1b30c9b80 1 Processor -- start 2023-07-21T18:51:31.539 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.520+0000 7ff1b30c9b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d7a0dd96b0 0x55d7a0f86920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:31.540 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.520+0000 7ff1b30c9b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d7a0f7d580 0x55d7a0f86e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:31.540 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.520+0000 7ff1b30c9b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7a0f832d0 0x55d7a0f873a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:31.540 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.520+0000 7ff1b30c9b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7a0e6ea60 con 0x55d7a0dd96b0 2023-07-21T18:51:31.540 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.520+0000 7ff1b30c9b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7a0e6e760 con 0x55d7a0f7d580 2023-07-21T18:51:31.540 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.520+0000 7ff1b30c9b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7a0d528c0 con 0x55d7a0f832d0 2023-07-21T18:51:31.540 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.520+0000 7ff1a4cdf700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d7a0dd96b0 0x55d7a0f86920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:31.540 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.520+0000 7ff1b30c9b80 10 client.0 _ll_register_callbacks cb 0x55d7a0e6c700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:51:31.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.520+0000 7ff1a4cdf700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d7a0dd96b0 0x55d7a0f86920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60632/0 (socket says 172.21.15.39:60632) 2023-07-21T18:51:31.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.520+0000 7ff1a4cdf700 1 -- 172.21.15.39:0/1592482615 learned_addr learned my addr 172.21.15.39:0/1592482615 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:51:31.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.520+0000 7ff197fff700 1 --2- 172.21.15.39:0/1592482615 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d7a0f7d580 0x55d7a0f86e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:31.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.520+0000 7ff1a54e0700 1 --2- 172.21.15.39:0/1592482615 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7a0f832d0 0x55d7a0f873a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:31.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.520+0000 7ff1a4cdf700 1 -- 172.21.15.39:0/1592482615 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7a0f832d0 msgr2=0x55d7a0f873a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:51:31.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.520+0000 7ff1a4cdf700 1 --2- 172.21.15.39:0/1592482615 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7a0f832d0 0x55d7a0f873a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:31.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.520+0000 7ff1a4cdf700 1 -- 172.21.15.39:0/1592482615 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d7a0f7d580 msgr2=0x55d7a0f86e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:51:31.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.520+0000 7ff1a4cdf700 1 --2- 172.21.15.39:0/1592482615 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d7a0f7d580 0x55d7a0f86e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:31.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.520+0000 7ff1a4cdf700 1 -- 172.21.15.39:0/1592482615 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff19000b960 con 0x55d7a0dd96b0 2023-07-21T18:51:31.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.520+0000 7ff1a4cdf700 1 --2- 172.21.15.39:0/1592482615 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d7a0dd96b0 0x55d7a0f86920 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7ff19c009990 tx=0x7ff19c01a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f2fe54b662f4226 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:51:31.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.520+0000 7ff17ffff700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:51:31.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.520+0000 7ff17ffff700 1 -- 172.21.15.39:0/1592482615 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19c02b050 con 0x55d7a0dd96b0 2023-07-21T18:51:31.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.520+0000 7ff17ffff700 1 -- 172.21.15.39:0/1592482615 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8527+0+0 (secure 0 0 0) 0x7ff19c00f460 con 0x55d7a0dd96b0 2023-07-21T18:51:31.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.520+0000 7ff1b30c9b80 1 -- 172.21.15.39:0/1592482615 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d7a0e28490 con 0x55d7a0dd96b0 2023-07-21T18:51:31.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.521+0000 7ff17ffff700 1 -- 172.21.15.39:0/1592482615 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff19c005920 con 0x55d7a0dd96b0 2023-07-21T18:51:31.543 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.521+0000 7ff1b30c9b80 10 client.16884 fetch_fsmap learned FSMap version 62 2023-07-21T18:51:31.543 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.521+0000 7ff1b30c9b80 1 -- 172.21.15.39:0/1592482615 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) v3 -- 0x55d7a0ce5c00 con 0x55d7a0dd96b0 2023-07-21T18:51:31.543 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.521+0000 7ff17ffff700 1 -- 172.21.15.39:0/1592482615 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d7a0e28490 con 0x55d7a0dd96b0 2023-07-21T18:51:31.543 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.521+0000 7ff17ffff700 1 -- 172.21.15.39:0/1592482615 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 62) v1 ==== 7478+0+0 (secure 0 0 0) 0x7ff19c024790 con 0x55d7a0dd96b0 2023-07-21T18:51:31.543 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.521+0000 7ff1b30c9b80 10 client.16884 fetch_fsmap finished waiting for FSMap version 62 2023-07-21T18:51:31.543 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.521+0000 7ff1b30c9b80 10 client.16884 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:51:31.543 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.521+0000 7ff1b30c9b80 20 client.16884 populate_metadata read hostname 'smithi039' 2023-07-21T18:51:31.543 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.521+0000 7ff1b30c9b80 1 --2- 172.21.15.39:0/1592482615 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d7a0f96cf0 0x55d7a0f99100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:31.544 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.521+0000 7ff1b30c9b80 4 client.16884 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:51:31.544 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.521+0000 7ff1b30c9b80 1 -- 172.21.15.39:0/1592482615 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d7a0ec7e50 con 0x55d7a0f96cf0 2023-07-21T18:51:31.544 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.521+0000 7ff197fff700 1 --2- 172.21.15.39:0/1592482615 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d7a0f96cf0 0x55d7a0f99100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:31.544 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.522+0000 7ff197fff700 1 --2- 172.21.15.39:0/1592482615 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d7a0f96cf0 0x55d7a0f99100 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70869a1d67c760b2 server_cookie=fde42145a0fc46cd in_seq=0 out_seq=0 2023-07-21T18:51:31.544 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.522+0000 7ff17ffff700 10 client.16884 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:51:31.544 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.523+0000 7ff17ffff700 1 -- 172.21.15.39:0/1592482615 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d7a0ec7e50 con 0x55d7a0f96cf0 2023-07-21T18:51:31.544 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.523+0000 7ff17ffff700 10 client.16884 handle_command_reply: tid=0 2023-07-21T18:51:31.544 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.523+0000 7ff1b30c9b80 1 client.16884 shutdown 2023-07-21T18:51:31.545 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.523+0000 7ff1b30c9b80 1 -- 172.21.15.39:0/1592482615 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d7a0dd96b0 msgr2=0x55d7a0f86920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:51:31.545 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.523+0000 7ff1b30c9b80 1 --2- 172.21.15.39:0/1592482615 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d7a0dd96b0 0x55d7a0f86920 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7ff19c009990 tx=0x7ff19c01a050 comp rx=0 tx=0).stop 2023-07-21T18:51:31.545 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.523+0000 7ff1b30c9b80 1 -- 172.21.15.39:0/1592482615 shutdown_connections 2023-07-21T18:51:31.545 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.523+0000 7ff1b30c9b80 1 --2- 172.21.15.39:0/1592482615 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d7a0f96cf0 0x55d7a0f99100 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:31.545 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.523+0000 7ff1b30c9b80 1 --2- 172.21.15.39:0/1592482615 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7a0f832d0 0x55d7a0f873a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:31.545 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.523+0000 7ff1b30c9b80 1 --2- 172.21.15.39:0/1592482615 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d7a0dd96b0 0x55d7a0f86920 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:31.545 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.523+0000 7ff1b30c9b80 1 --2- 172.21.15.39:0/1592482615 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d7a0f7d580 0x55d7a0f86e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:31.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.523+0000 7ff17ffff700 0 client.16884 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:51:31.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.523+0000 7ff1b30c9b80 1 -- 172.21.15.39:0/1592482615 >> 172.21.15.39:0/1592482615 conn(0x55d7a0f72be0 msgr2=0x55d7a0f774e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:51:31.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.523+0000 7ff1b30c9b80 1 -- 172.21.15.39:0/1592482615 shutdown_connections 2023-07-21T18:51:31.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.523+0000 7ff1b30c9b80 1 -- 172.21.15.39:0/1592482615 wait complete. 2023-07-21T18:51:31.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.523+0000 7ff1b30c9b80 20 client.16884 trim_cache size 0 max 16384 2023-07-21T18:51:31.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.538+0000 7ff1b30c9b80 1 Processor -- start 2023-07-21T18:51:31.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.538+0000 7ff1b30c9b80 1 -- start start 2023-07-21T18:51:31.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.538+0000 7ff1b30c9b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d7a0dd96b0 0x55d7a0f7d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:31.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.538+0000 7ff1b30c9b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d7a0f7d580 0x55d7a0f77c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:31.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.538+0000 7ff1b30c9b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7a0f832d0 0x55d7a0f83900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:31.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.538+0000 7ff1b30c9b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7a0f836a0 con 0x55d7a0f7d580 2023-07-21T18:51:31.554 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.538+0000 7ff1b30c9b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7a0f79860 con 0x55d7a0dd96b0 2023-07-21T18:51:31.554 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.538+0000 7ff1b30c9b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7a0f899d0 con 0x55d7a0f832d0 2023-07-21T18:51:31.554 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.539+0000 7ff197fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d7a0f7d580 0x55d7a0f77c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:31.554 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.539+0000 7ff197fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d7a0f7d580 0x55d7a0f77c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60642/0 (socket says 172.21.15.39:60642) 2023-07-21T18:51:31.554 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.539+0000 7ff197fff700 1 -- 172.21.15.39:0/2379039935 learned_addr learned my addr 172.21.15.39:0/2379039935 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:51:31.554 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.539+0000 7ff1a4cdf700 1 --2- 172.21.15.39:0/2379039935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d7a0dd96b0 0x55d7a0f7d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:31.554 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.539+0000 7ff1a54e0700 1 --2- 172.21.15.39:0/2379039935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7a0f832d0 0x55d7a0f83900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:31.555 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.539+0000 7ff197fff700 1 -- 172.21.15.39:0/2379039935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7a0f832d0 msgr2=0x55d7a0f83900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:51:31.555 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.539+0000 7ff197fff700 1 --2- 172.21.15.39:0/2379039935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7a0f832d0 0x55d7a0f83900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:31.555 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.539+0000 7ff197fff700 1 -- 172.21.15.39:0/2379039935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d7a0dd96b0 msgr2=0x55d7a0f7d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:51:31.555 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.539+0000 7ff197fff700 1 --2- 172.21.15.39:0/2379039935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d7a0dd96b0 0x55d7a0f7d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:31.555 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.539+0000 7ff197fff700 1 -- 172.21.15.39:0/2379039935 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7a0d231f0 con 0x55d7a0f7d580 2023-07-21T18:51:31.555 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.539+0000 7ff1a4cdf700 1 --2- 172.21.15.39:0/2379039935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d7a0dd96b0 0x55d7a0f7d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:51:31.555 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.539+0000 7ff197fff700 1 --2- 172.21.15.39:0/2379039935 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d7a0f7d580 0x55d7a0f77c60 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7ff19000bd30 tx=0x7ff19000b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dcb2157cec974b7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:51:31.555 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.539+0000 7ff1877fe700 1 -- 172.21.15.39:0/2379039935 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff190020040 con 0x55d7a0f7d580 2023-07-21T18:51:31.556 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.539+0000 7ff1877fe700 1 -- 172.21.15.39:0/2379039935 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff190021040 con 0x55d7a0f7d580 2023-07-21T18:51:31.556 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.539+0000 7ff1877fe700 1 -- 172.21.15.39:0/2379039935 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff190003b80 con 0x55d7a0f7d580 2023-07-21T18:51:31.558 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.540+0000 7ff1b30c9b80 1 -- 172.21.15.39:0/2379039935 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d7a0f7d580 msgr2=0x55d7a0f77c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:51:31.558 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.540+0000 7ff1b30c9b80 1 --2- 172.21.15.39:0/2379039935 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d7a0f7d580 0x55d7a0f77c60 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7ff19000bd30 tx=0x7ff19000b630 comp rx=0 tx=0).stop 2023-07-21T18:51:31.558 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.540+0000 7ff1b30c9b80 1 -- 172.21.15.39:0/2379039935 shutdown_connections 2023-07-21T18:51:31.559 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.540+0000 7ff1b30c9b80 1 --2- 172.21.15.39:0/2379039935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d7a0dd96b0 0x55d7a0f7d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:31.559 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.540+0000 7ff1b30c9b80 1 --2- 172.21.15.39:0/2379039935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7a0f832d0 0x55d7a0f83900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:31.559 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.540+0000 7ff1b30c9b80 1 --2- 172.21.15.39:0/2379039935 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d7a0f7d580 0x55d7a0f77c60 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:31.559 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.540+0000 7ff1b30c9b80 1 -- 172.21.15.39:0/2379039935 >> 172.21.15.39:0/2379039935 conn(0x55d7a0f72be0 msgr2=0x55d7a0f80b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:51:31.559 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.540+0000 7ff1b30c9b80 1 -- 172.21.15.39:0/2379039935 shutdown_connections 2023-07-21T18:51:31.559 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.540+0000 7ff1b30c9b80 1 -- 172.21.15.39:0/2379039935 wait complete. 2023-07-21T18:51:31.559 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.540+0000 7ff1b30c9b80 1 -- start start 2023-07-21T18:51:31.560 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.540+0000 7ff1b30c9b80 1 Processor -- start 2023-07-21T18:51:31.560 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.541+0000 7ff1b30c9b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d7a0dd96b0 0x55d7a0fe3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:31.560 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.541+0000 7ff1b30c9b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d7a0f7d580 0x55d7a0fe3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:31.560 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.541+0000 7ff1b30c9b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7a0f832d0 0x55d7a0f8d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:31.560 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.541+0000 7ff1b30c9b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7a0f8d8f0 con 0x55d7a0dd96b0 2023-07-21T18:51:31.560 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.541+0000 7ff1b30c9b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7a0f8da60 con 0x55d7a0f7d580 2023-07-21T18:51:31.560 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.541+0000 7ff1b30c9b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7a0f8dc20 con 0x55d7a0f832d0 2023-07-21T18:51:31.560 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.541+0000 7ff1b30c9b80 10 client.0 _ll_register_callbacks cb 0x7ff19c005350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:51:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.541+0000 7ff1a4cdf700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d7a0dd96b0 0x55d7a0fe3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.541+0000 7ff1a4cdf700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d7a0dd96b0 0x55d7a0fe3720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60646/0 (socket says 172.21.15.39:60646) 2023-07-21T18:51:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.541+0000 7ff1a4cdf700 1 -- 172.21.15.39:0/2339286714 learned_addr learned my addr 172.21.15.39:0/2339286714 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:51:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.541+0000 7ff197fff700 1 --2- 172.21.15.39:0/2339286714 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d7a0f7d580 0x55d7a0fe3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.541+0000 7ff1a54e0700 1 --2- 172.21.15.39:0/2339286714 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7a0f832d0 0x55d7a0f8d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.541+0000 7ff1a4cdf700 1 -- 172.21.15.39:0/2339286714 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7a0f832d0 msgr2=0x55d7a0f8d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:51:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.541+0000 7ff1a4cdf700 1 --2- 172.21.15.39:0/2339286714 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7a0f832d0 0x55d7a0f8d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.541+0000 7ff1a4cdf700 1 -- 172.21.15.39:0/2339286714 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d7a0f7d580 msgr2=0x55d7a0fe3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:51:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.541+0000 7ff1a4cdf700 1 --2- 172.21.15.39:0/2339286714 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d7a0f7d580 0x55d7a0fe3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.541+0000 7ff1a4cdf700 1 -- 172.21.15.39:0/2339286714 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d7a0cfcd80 con 0x55d7a0dd96b0 2023-07-21T18:51:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.541+0000 7ff1a4cdf700 1 --2- 172.21.15.39:0/2339286714 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d7a0dd96b0 0x55d7a0fe3720 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7ff19c024af0 tx=0x7ff19c00bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddaa9c4d61504ac4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:51:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.541+0000 7ff17ffff700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:51:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.542+0000 7ff17ffff700 1 -- 172.21.15.39:0/2339286714 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19c02b040 con 0x55d7a0dd96b0 2023-07-21T18:51:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.542+0000 7ff17ffff700 1 -- 172.21.15.39:0/2339286714 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8527+0+0 (secure 0 0 0) 0x7ff19c0248a0 con 0x55d7a0dd96b0 2023-07-21T18:51:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.542+0000 7ff1b30c9b80 1 -- 172.21.15.39:0/2339286714 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff190015070 con 0x55d7a0dd96b0 2023-07-21T18:51:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.542+0000 7ff17ffff700 1 -- 172.21.15.39:0/2339286714 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff19c00f460 con 0x55d7a0dd96b0 2023-07-21T18:51:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.542+0000 7ff1b30c9b80 10 client.16890 fetch_fsmap learned FSMap version 62 2023-07-21T18:51:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.542+0000 7ff1b30c9b80 1 -- 172.21.15.39:0/2339286714 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) v3 -- 0x55d7a0cfb250 con 0x55d7a0dd96b0 2023-07-21T18:51:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.542+0000 7ff17ffff700 1 -- 172.21.15.39:0/2339286714 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff19c04f020 con 0x55d7a0dd96b0 2023-07-21T18:51:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.542+0000 7ff17ffff700 1 -- 172.21.15.39:0/2339286714 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 62) v1 ==== 7478+0+0 (secure 0 0 0) 0x7ff19c03a840 con 0x55d7a0dd96b0 2023-07-21T18:51:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.542+0000 7ff1b30c9b80 10 client.16890 fetch_fsmap finished waiting for FSMap version 62 2023-07-21T18:51:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.542+0000 7ff1b30c9b80 10 client.16890 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:51:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.542+0000 7ff1b30c9b80 20 client.16890 populate_metadata read hostname 'smithi039' 2023-07-21T18:51:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.542+0000 7ff1b30c9b80 1 --2- 172.21.15.39:0/2339286714 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d7a0feb250 0x55d7a0fed660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.542+0000 7ff1b30c9b80 4 client.16890 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:51:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.542+0000 7ff1b30c9b80 1 -- 172.21.15.39:0/2339286714 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d7a0e89ff0 con 0x55d7a0feb250 2023-07-21T18:51:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.542+0000 7ff197fff700 1 --2- 172.21.15.39:0/2339286714 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d7a0feb250 0x55d7a0fed660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.543+0000 7ff17ffff700 10 client.16890 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:51:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.543+0000 7ff197fff700 1 --2- 172.21.15.39:0/2339286714 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d7a0feb250 0x55d7a0fed660 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac34b878c1a785c server_cookie=998edef57f667c85 in_seq=0 out_seq=0 2023-07-21T18:51:31.595 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.581+0000 7ff17ffff700 1 -- 172.21.15.39:0/2339286714 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d7a0e89ff0 con 0x55d7a0feb250 2023-07-21T18:51:31.595 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.581+0000 7ff17ffff700 10 client.16890 handle_command_reply: tid=0 2023-07-21T18:51:31.596 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.581+0000 7ff1b30c9b80 1 client.16890 shutdown 2023-07-21T18:51:31.596 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.582+0000 7ff1b30c9b80 1 -- 172.21.15.39:0/2339286714 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d7a0dd96b0 msgr2=0x55d7a0fe3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:51:31.596 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.582+0000 7ff1b30c9b80 1 --2- 172.21.15.39:0/2339286714 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d7a0dd96b0 0x55d7a0fe3720 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7ff19c024af0 tx=0x7ff19c00bd40 comp rx=0 tx=0).stop 2023-07-21T18:51:31.596 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.582+0000 7ff1b30c9b80 1 -- 172.21.15.39:0/2339286714 shutdown_connections 2023-07-21T18:51:31.596 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.582+0000 7ff1b30c9b80 1 --2- 172.21.15.39:0/2339286714 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d7a0feb250 0x55d7a0fed660 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:31.596 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.582+0000 7ff1b30c9b80 1 --2- 172.21.15.39:0/2339286714 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7a0f832d0 0x55d7a0f8d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:31.596 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.582+0000 7ff17ffff700 0 client.16890 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:51:31.597 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.582+0000 7ff1b30c9b80 1 --2- 172.21.15.39:0/2339286714 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d7a0dd96b0 0x55d7a0fe3720 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:31.597 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.582+0000 7ff1b30c9b80 1 --2- 172.21.15.39:0/2339286714 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d7a0f7d580 0x55d7a0fe3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:31.597 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.582+0000 7ff1b30c9b80 1 -- 172.21.15.39:0/2339286714 >> 172.21.15.39:0/2339286714 conn(0x55d7a0f72be0 msgr2=0x55d7a0f81af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:51:31.597 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.582+0000 7ff1b30c9b80 1 -- 172.21.15.39:0/2339286714 shutdown_connections 2023-07-21T18:51:31.597 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.582+0000 7ff1b30c9b80 1 -- 172.21.15.39:0/2339286714 wait complete. 2023-07-21T18:51:31.597 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.582+0000 7ff1b30c9b80 20 client.16890 trim_cache size 0 max 16384 2023-07-21T18:51:31.597 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:51:31.598 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:51:31.598 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:51:31.598 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:51:31.599 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.583+0000 7ff1a6f43700 1 -- 172.21.15.39:0/216592363 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7ff1880a12c0 msgr2=0x7ff1880a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:51:31.599 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.583+0000 7ff1a6f43700 1 --2- 172.21.15.39:0/216592363 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7ff1880a12c0 0x7ff1880a3770 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff19c00b2e0 tx=0x7ff19c005fd0 comp rx=0 tx=0).stop 2023-07-21T18:51:31.599 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.583+0000 7ff1a6f43700 1 -- 172.21.15.39:0/216592363 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff1a0104a30 msgr2=0x7ff1a0195a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:51:31.599 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.583+0000 7ff1a6f43700 1 --2- 172.21.15.39:0/216592363 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff1a0104a30 0x7ff1a0195a10 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7ff19800ef30 tx=0x7ff19800c3a0 comp rx=0 tx=0).stop 2023-07-21T18:51:31.599 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.583+0000 7ff1a6f43700 1 -- 172.21.15.39:0/216592363 shutdown_connections 2023-07-21T18:51:31.599 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.583+0000 7ff1a6f43700 1 --2- 172.21.15.39:0/216592363 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff1a0104a30 0x7ff1a0195a10 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.583+0000 7ff1a6f43700 1 --2- 172.21.15.39:0/216592363 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff1a00fea30 0x7ff1a019bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.583+0000 7ff1a6f43700 1 --2- 172.21.15.39:0/216592363 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7ff1880a12c0 0x7ff1880a3770 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.583+0000 7ff1a6f43700 1 --2- 172.21.15.39:0/216592363 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1a00ff3e0 0x7ff1a019c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.583+0000 7ff1a6f43700 1 -- 172.21.15.39:0/216592363 >> 172.21.15.39:0/216592363 conn(0x7ff1a0074760 msgr2=0x7ff1a0067900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:51:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.583+0000 7ff1a6f43700 1 -- 172.21.15.39:0/216592363 shutdown_connections 2023-07-21T18:51:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:31.583+0000 7ff1a6f43700 1 -- 172.21.15.39:0/216592363 wait complete. 2023-07-21T18:51:32.285 INFO:tasks.workunit.client.0.smithi039.stdout: 1 585449 5.80 MB/sec execute 405 sec latency 3124.082 ms 2023-07-21T18:51:32.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:51:32 smithi193 ceph-mon[98204]: pgmap v305: 97 pgs: 97 active+clean; 239 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 1.9 MiB/s wr, 182 op/s 2023-07-21T18:51:32.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:51:32 smithi155 ceph-mon[97934]: pgmap v305: 97 pgs: 97 active+clean; 239 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 1.9 MiB/s wr, 182 op/s 2023-07-21T18:51:32.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:51:32 smithi039 ceph-mon[96489]: pgmap v305: 97 pgs: 97 active+clean; 239 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 1.9 MiB/s wr, 182 op/s 2023-07-21T18:51:33.285 INFO:tasks.workunit.client.0.smithi039.stdout: 1 586747 5.80 MB/sec execute 406 sec latency 16.946 ms 2023-07-21T18:51:34.285 INFO:tasks.workunit.client.0.smithi039.stdout: 1 587763 5.80 MB/sec execute 407 sec latency 184.148 ms 2023-07-21T18:51:34.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:51:34 smithi193 ceph-mon[98204]: pgmap v306: 97 pgs: 97 active+clean; 239 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 1.1 MiB/s wr, 112 op/s 2023-07-21T18:51:34.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:51:34 smithi155 ceph-mon[97934]: pgmap v306: 97 pgs: 97 active+clean; 239 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 1.1 MiB/s wr, 112 op/s 2023-07-21T18:51:34.678 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:51:34 smithi039 ceph-mon[96489]: pgmap v306: 97 pgs: 97 active+clean; 239 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 1.1 MiB/s wr, 112 op/s 2023-07-21T18:51:35.286 INFO:tasks.workunit.client.0.smithi039.stdout: 1 587763 5.78 MB/sec execute 408 sec latency 1184.196 ms 2023-07-21T18:51:36.285 INFO:tasks.workunit.client.0.smithi039.stdout: 1 587763 5.77 MB/sec execute 409 sec latency 2184.248 ms 2023-07-21T18:51:36.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:51:36 smithi193 ceph-mon[98204]: pgmap v307: 97 pgs: 97 active+clean; 239 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 1.1 MiB/s wr, 112 op/s 2023-07-21T18:51:36.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:51:36 smithi155 ceph-mon[97934]: pgmap v307: 97 pgs: 97 active+clean; 239 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 1.1 MiB/s wr, 112 op/s 2023-07-21T18:51:36.678 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:51:36 smithi039 ceph-mon[96489]: pgmap v307: 97 pgs: 97 active+clean; 239 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 1.1 MiB/s wr, 112 op/s 2023-07-21T18:51:37.285 INFO:tasks.workunit.client.0.smithi039.stdout: 1 588988 5.77 MB/sec execute 410 sec latency 2620.719 ms 2023-07-21T18:51:38.285 INFO:tasks.workunit.client.0.smithi039.stdout: 1 590297 5.77 MB/sec execute 411 sec latency 14.067 ms 2023-07-21T18:51:38.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:51:38 smithi193 ceph-mon[98204]: pgmap v308: 97 pgs: 97 active+clean; 239 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 843 KiB/s rd, 1.8 MiB/s wr, 173 op/s 2023-07-21T18:51:38.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:51:38 smithi155 ceph-mon[97934]: pgmap v308: 97 pgs: 97 active+clean; 239 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 843 KiB/s rd, 1.8 MiB/s wr, 173 op/s 2023-07-21T18:51:38.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:51:38 smithi039 ceph-mon[96489]: pgmap v308: 97 pgs: 97 active+clean; 239 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 843 KiB/s rd, 1.8 MiB/s wr, 173 op/s 2023-07-21T18:51:39.285 INFO:tasks.workunit.client.0.smithi039.stdout: 1 591312 5.77 MB/sec execute 412 sec latency 200.283 ms 2023-07-21T18:51:40.285 INFO:tasks.workunit.client.0.smithi039.stdout: 1 591312 5.76 MB/sec execute 413 sec latency 1200.336 ms 2023-07-21T18:51:40.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:51:40 smithi193 ceph-mon[98204]: pgmap v309: 97 pgs: 97 active+clean; 239 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 1.2 MiB/s wr, 117 op/s 2023-07-21T18:51:40.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:51:40 smithi155 ceph-mon[97934]: pgmap v309: 97 pgs: 97 active+clean; 239 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 1.2 MiB/s wr, 117 op/s 2023-07-21T18:51:40.678 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:51:40 smithi039 ceph-mon[96489]: pgmap v309: 97 pgs: 97 active+clean; 239 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 1.2 MiB/s wr, 117 op/s 2023-07-21T18:51:41.285 INFO:tasks.workunit.client.0.smithi039.stdout: 1 591312 5.74 MB/sec execute 414 sec latency 2200.445 ms 2023-07-21T18:51:42.285 INFO:tasks.workunit.client.0.smithi039.stdout: 1 592540 5.75 MB/sec execute 415 sec latency 2633.967 ms 2023-07-21T18:51:42.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:51:42 smithi193 ceph-mon[98204]: pgmap v310: 97 pgs: 97 active+clean; 242 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 836 KiB/s rd, 1.8 MiB/s wr, 175 op/s 2023-07-21T18:51:42.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:51:42 smithi155 ceph-mon[97934]: pgmap v310: 97 pgs: 97 active+clean; 242 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 836 KiB/s rd, 1.8 MiB/s wr, 175 op/s 2023-07-21T18:51:42.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:51:42 smithi039 ceph-mon[96489]: pgmap v310: 97 pgs: 97 active+clean; 242 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 836 KiB/s rd, 1.8 MiB/s wr, 175 op/s 2023-07-21T18:51:43.285 INFO:tasks.workunit.client.0.smithi039.stdout: 1 593886 5.75 MB/sec execute 416 sec latency 11.457 ms 2023-07-21T18:51:44.286 INFO:tasks.workunit.client.0.smithi039.stdout: 1 594856 5.75 MB/sec execute 417 sec latency 441.822 ms 2023-07-21T18:51:44.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:51:44 smithi193 ceph-mon[98204]: pgmap v311: 97 pgs: 97 active+clean; 242 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2023-07-21T18:51:44.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:51:44 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:51:44.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:51:44 smithi155 ceph-mon[97934]: pgmap v311: 97 pgs: 97 active+clean; 242 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2023-07-21T18:51:44.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:51:44 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:51:44.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:51:44 smithi039 ceph-mon[96489]: pgmap v311: 97 pgs: 97 active+clean; 242 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2023-07-21T18:51:44.678 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:51:44 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:51:44.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.698+0000 7feab6a95700 1 -- 172.21.15.39:0/2599157281 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feab0105b80 msgr2=0x7feab0105f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:51:44.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.698+0000 7feab6a95700 1 --2- 172.21.15.39:0/2599157281 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feab0105b80 0x7feab0105f50 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7feaa800b0a0 tx=0x7feaa800b3b0 comp rx=0 tx=0).stop 2023-07-21T18:51:44.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.698+0000 7feab6a95700 1 -- 172.21.15.39:0/2599157281 shutdown_connections 2023-07-21T18:51:44.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.698+0000 7feab6a95700 1 --2- 172.21.15.39:0/2599157281 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feab0100500 0x7feab010c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:44.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.698+0000 7feab6a95700 1 --2- 172.21.15.39:0/2599157281 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feab0105b80 0x7feab0105f50 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:44.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.698+0000 7feab6a95700 1 --2- 172.21.15.39:0/2599157281 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feab00ffb50 0x7feab00fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:44.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.698+0000 7feab6a95700 1 -- 172.21.15.39:0/2599157281 >> 172.21.15.39:0/2599157281 conn(0x7feab00fb650 msgr2=0x7feab00fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:51:44.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.698+0000 7feab6a95700 1 -- 172.21.15.39:0/2599157281 shutdown_connections 2023-07-21T18:51:44.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.698+0000 7feab6a95700 1 -- 172.21.15.39:0/2599157281 wait complete. 2023-07-21T18:51:44.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.699+0000 7feab6a95700 1 Processor -- start 2023-07-21T18:51:44.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.699+0000 7feab6a95700 1 -- start start 2023-07-21T18:51:44.714 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.699+0000 7feab6a95700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feab00ffb50 0x7feab00729f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:44.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.699+0000 7feab6a95700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feab0100500 0x7feab0072f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:44.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.699+0000 7feab6a95700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feab0105b80 0x7feab01a3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:44.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.699+0000 7feab6a95700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab01a4370 con 0x7feab00ffb50 2023-07-21T18:51:44.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.699+0000 7feab6a95700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab01a44b0 con 0x7feab0100500 2023-07-21T18:51:44.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.699+0000 7feab6a95700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab01a4620 con 0x7feab0105b80 2023-07-21T18:51:44.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.699+0000 7feab4831700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feab00ffb50 0x7feab00729f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:44.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.699+0000 7feab4831700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feab00ffb50 0x7feab00729f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60660/0 (socket says 172.21.15.39:60660) 2023-07-21T18:51:44.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.699+0000 7feab4831700 1 -- 172.21.15.39:0/3555988882 learned_addr learned my addr 172.21.15.39:0/3555988882 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:51:44.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.699+0000 7feab5032700 1 --2- 172.21.15.39:0/3555988882 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feab0105b80 0x7feab01a3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:44.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.700+0000 7feaaffff700 1 --2- 172.21.15.39:0/3555988882 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feab0100500 0x7feab0072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:44.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.700+0000 7feab4831700 1 -- 172.21.15.39:0/3555988882 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feab0105b80 msgr2=0x7feab01a3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:51:44.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.700+0000 7feab4831700 1 --2- 172.21.15.39:0/3555988882 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feab0105b80 0x7feab01a3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:44.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.700+0000 7feab4831700 1 -- 172.21.15.39:0/3555988882 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feab0100500 msgr2=0x7feab0072f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:51:44.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.700+0000 7feab4831700 1 --2- 172.21.15.39:0/3555988882 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feab0100500 0x7feab0072f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:44.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.700+0000 7feab4831700 1 -- 172.21.15.39:0/3555988882 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaa8009d00 con 0x7feab00ffb50 2023-07-21T18:51:44.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.700+0000 7feaaffff700 1 --2- 172.21.15.39:0/3555988882 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feab0100500 0x7feab0072f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:51:44.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.700+0000 7feab4831700 1 --2- 172.21.15.39:0/3555988882 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feab00ffb50 0x7feab00729f0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7feaa8006070 tx=0x7feaa80079b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:51:44.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.700+0000 7feaadffb700 1 -- 172.21.15.39:0/3555988882 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa800e040 con 0x7feab00ffb50 2023-07-21T18:51:44.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.700+0000 7feab6a95700 1 -- 172.21.15.39:0/3555988882 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feab01a48a0 con 0x7feab00ffb50 2023-07-21T18:51:44.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.700+0000 7feaadffb700 1 -- 172.21.15.39:0/3555988882 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feaa8004de0 con 0x7feab00ffb50 2023-07-21T18:51:44.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.700+0000 7feaadffb700 1 -- 172.21.15.39:0/3555988882 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa801d6e0 con 0x7feab00ffb50 2023-07-21T18:51:44.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.700+0000 7feab6a95700 1 -- 172.21.15.39:0/3555988882 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feab01a4d20 con 0x7feab00ffb50 2023-07-21T18:51:44.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.702+0000 7feaadffb700 1 -- 172.21.15.39:0/3555988882 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7feaa8003a00 con 0x7feab00ffb50 2023-07-21T18:51:44.722 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.703+0000 7feaadffb700 1 --2- 172.21.15.39:0/3555988882 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fea9c0a12c0 0x7fea9c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:44.722 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.703+0000 7feaadffb700 1 -- 172.21.15.39:0/3555988882 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8527+0+0 (secure 0 0 0) 0x7feaa80ce370 con 0x7feab00ffb50 2023-07-21T18:51:44.722 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.703+0000 7feaaffff700 1 --2- 172.21.15.39:0/3555988882 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fea9c0a12c0 0x7fea9c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:44.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.703+0000 7feaaffff700 1 --2- 172.21.15.39:0/3555988882 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fea9c0a12c0 0x7fea9c0a3770 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fea98005fd0 tx=0x7fea98005e30 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:51:44.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.704+0000 7feac2c1bb80 1 Processor -- start 2023-07-21T18:51:44.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.704+0000 7feac2c1bb80 1 -- start start 2023-07-21T18:51:44.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.704+0000 7feac2c1bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d89f6286b0 0x55d89f7d1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:44.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.704+0000 7feac2c1bb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d89f7d22d0 0x55d89f7cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:44.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.704+0000 7feac2c1bb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d89f7cc580 0x55d89f7cc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:44.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.704+0000 7feac2c1bb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d89f6d1c50 con 0x55d89f7cc580 2023-07-21T18:51:44.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.704+0000 7feac2c1bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d89f7249e0 con 0x55d89f6286b0 2023-07-21T18:51:44.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.704+0000 7feac2c1bb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d89f71de70 con 0x55d89f7d22d0 2023-07-21T18:51:44.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.704+0000 7feab5032700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d89f7cc580 0x55d89f7cc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:44.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.704+0000 7feab5032700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d89f7cc580 0x55d89f7cc9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60672/0 (socket says 172.21.15.39:60672) 2023-07-21T18:51:44.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.704+0000 7feab5032700 1 -- 172.21.15.39:0/4051980481 learned_addr learned my addr 172.21.15.39:0/4051980481 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:51:44.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.704+0000 7feaaffff700 1 --2- 172.21.15.39:0/4051980481 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d89f7d22d0 0x55d89f7cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:44.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.704+0000 7feab4831700 1 --2- 172.21.15.39:0/4051980481 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d89f6286b0 0x55d89f7d1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:44.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.704+0000 7feab5032700 1 -- 172.21.15.39:0/4051980481 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d89f7d22d0 msgr2=0x55d89f7cc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:51:44.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.704+0000 7feab5032700 1 --2- 172.21.15.39:0/4051980481 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d89f7d22d0 0x55d89f7cc040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:44.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.704+0000 7feab5032700 1 -- 172.21.15.39:0/4051980481 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d89f6286b0 msgr2=0x55d89f7d1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:51:44.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.704+0000 7feab5032700 1 --2- 172.21.15.39:0/4051980481 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d89f6286b0 0x55d89f7d1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:44.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.704+0000 7feab5032700 1 -- 172.21.15.39:0/4051980481 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d89f54bd80 con 0x55d89f7cc580 2023-07-21T18:51:44.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.705+0000 7feab5032700 1 --2- 172.21.15.39:0/4051980481 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d89f7cc580 0x55d89f7cc9d0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7feaa000b6c0 tx=0x7feaa000b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3a4bf5ebb8fb3ec server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:51:44.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.705+0000 7feaa6ffd700 1 -- 172.21.15.39:0/4051980481 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa0004a30 con 0x55d89f7cc580 2023-07-21T18:51:44.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.705+0000 7feaa6ffd700 1 -- 172.21.15.39:0/4051980481 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feaa00054e0 con 0x55d89f7cc580 2023-07-21T18:51:44.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.705+0000 7feaa6ffd700 1 -- 172.21.15.39:0/4051980481 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa0003ba0 con 0x55d89f7cc580 2023-07-21T18:51:44.726 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.705+0000 7feac2c1bb80 1 -- 172.21.15.39:0/4051980481 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d89f7cc580 msgr2=0x55d89f7cc9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:51:44.726 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.705+0000 7feac2c1bb80 1 --2- 172.21.15.39:0/4051980481 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d89f7cc580 0x55d89f7cc9d0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7feaa000b6c0 tx=0x7feaa000b9d0 comp rx=0 tx=0).stop 2023-07-21T18:51:44.726 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.705+0000 7feac2c1bb80 1 -- 172.21.15.39:0/4051980481 shutdown_connections 2023-07-21T18:51:44.726 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.705+0000 7feac2c1bb80 1 --2- 172.21.15.39:0/4051980481 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d89f6286b0 0x55d89f7d1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:44.726 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.705+0000 7feac2c1bb80 1 --2- 172.21.15.39:0/4051980481 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d89f7cc580 0x55d89f7cc9d0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:44.726 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.705+0000 7feac2c1bb80 1 --2- 172.21.15.39:0/4051980481 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d89f7d22d0 0x55d89f7cc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:44.726 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.705+0000 7feac2c1bb80 1 -- 172.21.15.39:0/4051980481 >> 172.21.15.39:0/4051980481 conn(0x55d89f7c1be0 msgr2=0x55d89f7c6f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:51:44.727 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.706+0000 7feac2c1bb80 1 -- 172.21.15.39:0/4051980481 shutdown_connections 2023-07-21T18:51:44.727 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.706+0000 7feac2c1bb80 1 -- 172.21.15.39:0/4051980481 wait complete. 2023-07-21T18:51:44.727 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.706+0000 7feac2c1bb80 1 -- start start 2023-07-21T18:51:44.727 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.706+0000 7feac2c1bb80 1 Processor -- start 2023-07-21T18:51:44.727 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.707+0000 7feac2c1bb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d89f6286b0 0x55d89f7d8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:44.727 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.707+0000 7feac2c1bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d89f7cc580 0x55d89f7d5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:44.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.707+0000 7feac2c1bb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d89f7d22d0 0x55d89f7d5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:44.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.707+0000 7feac2c1bb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d89f6bda60 con 0x55d89f7d22d0 2023-07-21T18:51:44.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.707+0000 7feac2c1bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d89f6bd760 con 0x55d89f7cc580 2023-07-21T18:51:44.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.707+0000 7feac2c1bb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d89f5a18c0 con 0x55d89f6286b0 2023-07-21T18:51:44.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.707+0000 7feac2c1bb80 10 client.0 _ll_register_callbacks cb 0x55d89f6bb700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:51:44.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.707+0000 7feab5032700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d89f7d22d0 0x55d89f7d5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:44.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.707+0000 7feab5032700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d89f7d22d0 0x55d89f7d5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60678/0 (socket says 172.21.15.39:60678) 2023-07-21T18:51:44.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.707+0000 7feab5032700 1 -- 172.21.15.39:0/1159449789 learned_addr learned my addr 172.21.15.39:0/1159449789 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:51:44.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.707+0000 7feab4831700 1 --2- 172.21.15.39:0/1159449789 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d89f6286b0 0x55d89f7d8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:44.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.707+0000 7feaaffff700 1 --2- 172.21.15.39:0/1159449789 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d89f7cc580 0x55d89f7d5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:44.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.707+0000 7feab5032700 1 -- 172.21.15.39:0/1159449789 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d89f6286b0 msgr2=0x55d89f7d8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:51:44.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.707+0000 7feab5032700 1 --2- 172.21.15.39:0/1159449789 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d89f6286b0 0x55d89f7d8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:44.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.707+0000 7feab5032700 1 -- 172.21.15.39:0/1159449789 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d89f7cc580 msgr2=0x55d89f7d5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:51:44.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.707+0000 7feab5032700 1 --2- 172.21.15.39:0/1159449789 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d89f7cc580 0x55d89f7d5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:44.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.707+0000 7feab5032700 1 -- 172.21.15.39:0/1159449789 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaa0004350 con 0x55d89f7d22d0 2023-07-21T18:51:44.730 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.707+0000 7feab5032700 1 --2- 172.21.15.39:0/1159449789 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d89f7d22d0 0x55d89f7d5d20 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7feaa0005d90 tx=0x7feaa0004df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37b3829475b678dc server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:51:44.730 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.707+0000 7fea8ffff700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:51:44.730 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.707+0000 7fea8ffff700 1 -- 172.21.15.39:0/1159449789 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa001f070 con 0x55d89f7d22d0 2023-07-21T18:51:44.730 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.707+0000 7feac2c1bb80 1 -- 172.21.15.39:0/1159449789 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d89f677490 con 0x55d89f7d22d0 2023-07-21T18:51:44.731 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.707+0000 7fea8ffff700 1 -- 172.21.15.39:0/1159449789 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8527+0+0 (secure 0 0 0) 0x7feaa0004750 con 0x55d89f7d22d0 2023-07-21T18:51:44.731 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.708+0000 7fea8ffff700 1 -- 172.21.15.39:0/1159449789 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7feaa0004910 con 0x55d89f7d22d0 2023-07-21T18:51:44.731 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.708+0000 7feac2c1bb80 10 client.16902 fetch_fsmap learned FSMap version 63 2023-07-21T18:51:44.731 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.708+0000 7feac2c1bb80 1 -- 172.21.15.39:0/1159449789 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) v3 -- 0x55d89f534c00 con 0x55d89f7d22d0 2023-07-21T18:51:44.731 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.708+0000 7fea8ffff700 1 -- 172.21.15.39:0/1159449789 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feaa0030020 con 0x55d89f7d22d0 2023-07-21T18:51:44.731 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.708+0000 7fea8ffff700 1 -- 172.21.15.39:0/1159449789 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 63) v1 ==== 7482+0+0 (secure 0 0 0) 0x7feaa00114d0 con 0x55d89f7d22d0 2023-07-21T18:51:44.731 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.708+0000 7feac2c1bb80 10 client.16902 fetch_fsmap finished waiting for FSMap version 63 2023-07-21T18:51:44.731 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.708+0000 7feac2c1bb80 10 client.16902 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:51:44.732 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.708+0000 7feac2c1bb80 20 client.16902 populate_metadata read hostname 'smithi039' 2023-07-21T18:51:44.732 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.708+0000 7feac2c1bb80 1 --2- 172.21.15.39:0/1159449789 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d89f7e5b00 0x55d89f7e7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:44.732 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.708+0000 7feac2c1bb80 4 client.16902 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:51:44.732 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.708+0000 7feac2c1bb80 1 -- 172.21.15.39:0/1159449789 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d89f716e50 con 0x55d89f7e5b00 2023-07-21T18:51:44.733 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.708+0000 7feab4831700 1 --2- 172.21.15.39:0/1159449789 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d89f7e5b00 0x55d89f7e7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:44.733 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.709+0000 7fea8ffff700 10 client.16902 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:51:44.733 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.709+0000 7feab4831700 1 --2- 172.21.15.39:0/1159449789 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d89f7e5b00 0x55d89f7e7f10 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a61ae953492fa290 server_cookie=9ffecde42305e136 in_seq=0 out_seq=0 2023-07-21T18:51:44.733 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.709+0000 7fea8ffff700 1 -- 172.21.15.39:0/1159449789 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d89f716e50 con 0x55d89f7e5b00 2023-07-21T18:51:44.733 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.709+0000 7fea8ffff700 10 client.16902 handle_command_reply: tid=0 2023-07-21T18:51:44.733 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.709+0000 7feac2c1bb80 1 client.16902 shutdown 2023-07-21T18:51:44.733 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.709+0000 7feac2c1bb80 1 -- 172.21.15.39:0/1159449789 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d89f7d22d0 msgr2=0x55d89f7d5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:51:44.734 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.709+0000 7feac2c1bb80 1 --2- 172.21.15.39:0/1159449789 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d89f7d22d0 0x55d89f7d5d20 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7feaa0005d90 tx=0x7feaa0004df0 comp rx=0 tx=0).stop 2023-07-21T18:51:44.734 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.710+0000 7feac2c1bb80 1 -- 172.21.15.39:0/1159449789 shutdown_connections 2023-07-21T18:51:44.734 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.710+0000 7feac2c1bb80 1 --2- 172.21.15.39:0/1159449789 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d89f7e5b00 0x55d89f7e7f10 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:44.734 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.710+0000 7feac2c1bb80 1 --2- 172.21.15.39:0/1159449789 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d89f7d22d0 0x55d89f7d5d20 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:44.734 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.710+0000 7feac2c1bb80 1 --2- 172.21.15.39:0/1159449789 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d89f6286b0 0x55d89f7d8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:44.734 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.710+0000 7fea8ffff700 0 client.16902 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:51:44.734 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.710+0000 7feac2c1bb80 1 --2- 172.21.15.39:0/1159449789 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d89f7cc580 0x55d89f7d5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.710+0000 7feac2c1bb80 1 -- 172.21.15.39:0/1159449789 >> 172.21.15.39:0/1159449789 conn(0x55d89f7c1be0 msgr2=0x55d89f7c5c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:51:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.710+0000 7feac2c1bb80 1 -- 172.21.15.39:0/1159449789 shutdown_connections 2023-07-21T18:51:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.710+0000 7feac2c1bb80 1 -- 172.21.15.39:0/1159449789 wait complete. 2023-07-21T18:51:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.710+0000 7feac2c1bb80 20 client.16902 trim_cache size 0 max 16384 2023-07-21T18:51:44.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.725+0000 7feac2c1bb80 1 Processor -- start 2023-07-21T18:51:44.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.725+0000 7feac2c1bb80 1 -- start start 2023-07-21T18:51:44.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.725+0000 7feac2c1bb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d89f6286b0 0x55d89f7dcca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:44.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.725+0000 7feac2c1bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d89f7cc580 0x55d89f7d4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:44.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.725+0000 7feac2c1bb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d89f7d22d0 0x55d89f7d3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:44.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.725+0000 7feac2c1bb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d89f7d6300 con 0x55d89f6286b0 2023-07-21T18:51:44.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.725+0000 7feac2c1bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d89f7c35f0 con 0x55d89f7cc580 2023-07-21T18:51:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.725+0000 7feac2c1bb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d89f7d26a0 con 0x55d89f7d22d0 2023-07-21T18:51:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.725+0000 7feab4831700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d89f6286b0 0x55d89f7dcca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.725+0000 7feab4831700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d89f6286b0 0x55d89f7dcca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60682/0 (socket says 172.21.15.39:60682) 2023-07-21T18:51:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.725+0000 7feab4831700 1 -- 172.21.15.39:0/3978592187 learned_addr learned my addr 172.21.15.39:0/3978592187 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:51:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.725+0000 7feaaffff700 1 --2- 172.21.15.39:0/3978592187 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d89f7cc580 0x55d89f7d4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.725+0000 7feab5032700 1 --2- 172.21.15.39:0/3978592187 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d89f7d22d0 0x55d89f7d3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.725+0000 7feab4831700 1 -- 172.21.15.39:0/3978592187 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d89f7d22d0 msgr2=0x55d89f7d3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:51:44.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.725+0000 7feab4831700 1 --2- 172.21.15.39:0/3978592187 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d89f7d22d0 0x55d89f7d3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:44.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.725+0000 7feab4831700 1 -- 172.21.15.39:0/3978592187 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d89f7cc580 msgr2=0x55d89f7d4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:51:44.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.725+0000 7feab4831700 1 --2- 172.21.15.39:0/3978592187 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d89f7cc580 0x55d89f7d4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:44.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.725+0000 7feab4831700 1 -- 172.21.15.39:0/3978592187 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d89f5721f0 con 0x55d89f6286b0 2023-07-21T18:51:44.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.725+0000 7feab5032700 1 --2- 172.21.15.39:0/3978592187 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d89f7d22d0 0x55d89f7d3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:51:44.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.725+0000 7feaaffff700 1 --2- 172.21.15.39:0/3978592187 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d89f7cc580 0x55d89f7d4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:51:44.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.726+0000 7feab4831700 1 --2- 172.21.15.39:0/3978592187 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d89f6286b0 0x55d89f7dcca0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7feaa8097970 tx=0x7feaa8086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69bb6827767a069e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:51:44.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.726+0000 7feaa77fe700 1 -- 172.21.15.39:0/3978592187 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa80b1070 con 0x55d89f6286b0 2023-07-21T18:51:44.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.726+0000 7feaa77fe700 1 -- 172.21.15.39:0/3978592187 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feaa809f4a0 con 0x55d89f6286b0 2023-07-21T18:51:44.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.726+0000 7feaa77fe700 1 -- 172.21.15.39:0/3978592187 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa80cbce0 con 0x55d89f6286b0 2023-07-21T18:51:44.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.726+0000 7feac2c1bb80 1 -- 172.21.15.39:0/3978592187 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d89f6286b0 msgr2=0x55d89f7dcca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:51:44.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.726+0000 7feac2c1bb80 1 --2- 172.21.15.39:0/3978592187 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d89f6286b0 0x55d89f7dcca0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7feaa8097970 tx=0x7feaa8086040 comp rx=0 tx=0).stop 2023-07-21T18:51:44.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.726+0000 7feac2c1bb80 1 -- 172.21.15.39:0/3978592187 shutdown_connections 2023-07-21T18:51:44.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.726+0000 7feac2c1bb80 1 --2- 172.21.15.39:0/3978592187 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d89f7d22d0 0x55d89f7d3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:44.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.726+0000 7feac2c1bb80 1 --2- 172.21.15.39:0/3978592187 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d89f6286b0 0x55d89f7dcca0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:44.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.726+0000 7feac2c1bb80 1 --2- 172.21.15.39:0/3978592187 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d89f7cc580 0x55d89f7d4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:44.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.726+0000 7feac2c1bb80 1 -- 172.21.15.39:0/3978592187 >> 172.21.15.39:0/3978592187 conn(0x55d89f7c1be0 msgr2=0x55d89f7d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:51:44.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.726+0000 7feac2c1bb80 1 -- 172.21.15.39:0/3978592187 shutdown_connections 2023-07-21T18:51:44.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.726+0000 7feac2c1bb80 1 -- 172.21.15.39:0/3978592187 wait complete. 2023-07-21T18:51:44.746 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.727+0000 7feac2c1bb80 1 -- start start 2023-07-21T18:51:44.746 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.727+0000 7feac2c1bb80 1 Processor -- start 2023-07-21T18:51:44.746 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.727+0000 7feac2c1bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d89f6286b0 0x55d89f7d8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:44.746 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.727+0000 7feac2c1bb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d89f7cc580 0x55d89f7d93b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:44.747 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.727+0000 7feac2c1bb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d89f7d22d0 0x55d89f8389b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:44.747 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.727+0000 7feac2c1bb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d89f838ef0 con 0x55d89f7d22d0 2023-07-21T18:51:44.747 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.727+0000 7feac2c1bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d89f839030 con 0x55d89f6286b0 2023-07-21T18:51:44.747 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.727+0000 7feac2c1bb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d89f8391c0 con 0x55d89f7cc580 2023-07-21T18:51:44.748 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.727+0000 7feac2c1bb80 10 client.0 _ll_register_callbacks cb 0x7feaa8009b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:51:44.748 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.727+0000 7feab5032700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d89f7d22d0 0x55d89f8389b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:44.748 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.727+0000 7feab5032700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d89f7d22d0 0x55d89f8389b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60692/0 (socket says 172.21.15.39:60692) 2023-07-21T18:51:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.727+0000 7feab5032700 1 -- 172.21.15.39:0/2306313522 learned_addr learned my addr 172.21.15.39:0/2306313522 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:51:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.728+0000 7feab4831700 1 --2- 172.21.15.39:0/2306313522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d89f6286b0 0x55d89f7d8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.728+0000 7feaaffff700 1 --2- 172.21.15.39:0/2306313522 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d89f7cc580 0x55d89f7d93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.728+0000 7feab5032700 1 -- 172.21.15.39:0/2306313522 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d89f7cc580 msgr2=0x55d89f7d93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:51:44.750 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.728+0000 7feab5032700 1 --2- 172.21.15.39:0/2306313522 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d89f7cc580 0x55d89f7d93b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:44.750 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.728+0000 7feab5032700 1 -- 172.21.15.39:0/2306313522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d89f6286b0 msgr2=0x55d89f7d8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:51:44.750 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.728+0000 7feab5032700 1 --2- 172.21.15.39:0/2306313522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d89f6286b0 0x55d89f7d8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:44.751 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.728+0000 7feab5032700 1 -- 172.21.15.39:0/2306313522 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feab01a4d20 con 0x55d89f7d22d0 2023-07-21T18:51:44.751 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.728+0000 7feaaffff700 1 --2- 172.21.15.39:0/2306313522 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d89f7cc580 0x55d89f7d93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:51:44.751 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.728+0000 7feab5032700 1 --2- 172.21.15.39:0/2306313522 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d89f7d22d0 0x55d89f8389b0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7feaa0005d20 tx=0x7feaa000ddf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d72493f4e84c90fe server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:51:44.752 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.728+0000 7fea8ffff700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:51:44.752 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.728+0000 7fea8ffff700 1 -- 172.21.15.39:0/2306313522 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa001f070 con 0x55d89f7d22d0 2023-07-21T18:51:44.752 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.728+0000 7fea8ffff700 1 -- 172.21.15.39:0/2306313522 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8527+0+0 (secure 0 0 0) 0x7feaa0000e40 con 0x55d89f7d22d0 2023-07-21T18:51:44.752 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.728+0000 7feac2c1bb80 1 -- 172.21.15.39:0/2306313522 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feaa00054e0 con 0x55d89f7d22d0 2023-07-21T18:51:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.728+0000 7fea8ffff700 1 -- 172.21.15.39:0/2306313522 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7feaa0005ee0 con 0x55d89f7d22d0 2023-07-21T18:51:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.728+0000 7feac2c1bb80 10 client.16908 fetch_fsmap learned FSMap version 63 2023-07-21T18:51:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.728+0000 7feac2c1bb80 1 -- 172.21.15.39:0/2306313522 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) v3 -- 0x55d89f54a250 con 0x55d89f7d22d0 2023-07-21T18:51:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.728+0000 7fea8ffff700 1 -- 172.21.15.39:0/2306313522 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feaa00054e0 con 0x55d89f7d22d0 2023-07-21T18:51:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.729+0000 7fea8ffff700 1 -- 172.21.15.39:0/2306313522 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 63) v1 ==== 7482+0+0 (secure 0 0 0) 0x7feaa0004740 con 0x55d89f7d22d0 2023-07-21T18:51:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.729+0000 7feac2c1bb80 10 client.16908 fetch_fsmap finished waiting for FSMap version 63 2023-07-21T18:51:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.729+0000 7feac2c1bb80 10 client.16908 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:51:44.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.729+0000 7feac2c1bb80 20 client.16908 populate_metadata read hostname 'smithi039' 2023-07-21T18:51:44.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.729+0000 7feac2c1bb80 1 --2- 172.21.15.39:0/2306313522 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d89f7e5b00 0x55d89f83c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:44.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.729+0000 7feac2c1bb80 4 client.16908 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:51:44.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.729+0000 7feac2c1bb80 1 -- 172.21.15.39:0/2306313522 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d89f6d8ff0 con 0x55d89f7e5b00 2023-07-21T18:51:44.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.729+0000 7feab4831700 1 --2- 172.21.15.39:0/2306313522 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d89f7e5b00 0x55d89f83c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:44.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.729+0000 7feab4831700 1 --2- 172.21.15.39:0/2306313522 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d89f7e5b00 0x55d89f83c6d0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ebab321eba39598 server_cookie=7b111a6b53786c0e in_seq=0 out_seq=0 2023-07-21T18:51:44.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.729+0000 7fea8ffff700 10 client.16908 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:51:44.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.759+0000 7fea8ffff700 1 -- 172.21.15.39:0/2306313522 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d89f6d8ff0 con 0x55d89f7e5b00 2023-07-21T18:51:44.772 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.759+0000 7fea8ffff700 10 client.16908 handle_command_reply: tid=0 2023-07-21T18:51:44.772 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.759+0000 7feac2c1bb80 1 client.16908 shutdown 2023-07-21T18:51:44.772 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.759+0000 7feac2c1bb80 1 -- 172.21.15.39:0/2306313522 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d89f7d22d0 msgr2=0x55d89f8389b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:51:44.772 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.759+0000 7feac2c1bb80 1 --2- 172.21.15.39:0/2306313522 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d89f7d22d0 0x55d89f8389b0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7feaa0005d20 tx=0x7feaa000ddf0 comp rx=0 tx=0).stop 2023-07-21T18:51:44.772 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.759+0000 7feac2c1bb80 1 -- 172.21.15.39:0/2306313522 shutdown_connections 2023-07-21T18:51:44.773 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.759+0000 7feac2c1bb80 1 --2- 172.21.15.39:0/2306313522 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d89f7e5b00 0x55d89f83c6d0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:44.773 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.759+0000 7feac2c1bb80 1 --2- 172.21.15.39:0/2306313522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d89f6286b0 0x55d89f7d8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:44.773 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.759+0000 7feac2c1bb80 1 --2- 172.21.15.39:0/2306313522 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d89f7d22d0 0x55d89f8389b0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:44.773 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.759+0000 7fea8ffff700 0 client.16908 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:51:44.773 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.759+0000 7feac2c1bb80 1 --2- 172.21.15.39:0/2306313522 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d89f7cc580 0x55d89f7d93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:44.773 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.759+0000 7feac2c1bb80 1 -- 172.21.15.39:0/2306313522 >> 172.21.15.39:0/2306313522 conn(0x55d89f7c1be0 msgr2=0x55d89f7d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:51:44.773 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.759+0000 7feac2c1bb80 1 -- 172.21.15.39:0/2306313522 shutdown_connections 2023-07-21T18:51:44.773 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.759+0000 7feac2c1bb80 1 -- 172.21.15.39:0/2306313522 wait complete. 2023-07-21T18:51:44.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.760+0000 7feac2c1bb80 20 client.16908 trim_cache size 0 max 16384 2023-07-21T18:51:44.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:51:44.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:51:44.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:51:44.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:51:44.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.760+0000 7feab6a95700 1 -- 172.21.15.39:0/3555988882 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fea9c0a12c0 msgr2=0x7fea9c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:51:44.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.760+0000 7feab6a95700 1 --2- 172.21.15.39:0/3555988882 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fea9c0a12c0 0x7fea9c0a3770 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fea98005fd0 tx=0x7fea98005e30 comp rx=0 tx=0).stop 2023-07-21T18:51:44.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.760+0000 7feab6a95700 1 -- 172.21.15.39:0/3555988882 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feab00ffb50 msgr2=0x7feab00729f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:51:44.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.760+0000 7feab6a95700 1 --2- 172.21.15.39:0/3555988882 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feab00ffb50 0x7feab00729f0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7feaa8006070 tx=0x7feaa80079b0 comp rx=0 tx=0).stop 2023-07-21T18:51:44.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.760+0000 7feab6a95700 1 -- 172.21.15.39:0/3555988882 shutdown_connections 2023-07-21T18:51:44.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.760+0000 7feab6a95700 1 --2- 172.21.15.39:0/3555988882 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feab0105b80 0x7feab01a3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:44.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.760+0000 7feab6a95700 1 --2- 172.21.15.39:0/3555988882 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feab00ffb50 0x7feab00729f0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:44.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.760+0000 7feab6a95700 1 --2- 172.21.15.39:0/3555988882 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fea9c0a12c0 0x7fea9c0a3770 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:44.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.760+0000 7feab6a95700 1 --2- 172.21.15.39:0/3555988882 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feab0100500 0x7feab0072f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:44.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.760+0000 7feab6a95700 1 -- 172.21.15.39:0/3555988882 >> 172.21.15.39:0/3555988882 conn(0x7feab00fb650 msgr2=0x7feab00fcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:51:44.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.760+0000 7feab6a95700 1 -- 172.21.15.39:0/3555988882 shutdown_connections 2023-07-21T18:51:44.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:44.760+0000 7feab6a95700 1 -- 172.21.15.39:0/3555988882 wait complete. 2023-07-21T18:51:45.285 INFO:tasks.workunit.client.0.smithi039.stdout: 1 594856 5.74 MB/sec execute 418 sec latency 1441.941 ms 2023-07-21T18:51:46.288 INFO:tasks.workunit.client.0.smithi039.stdout: 1 595465 5.73 MB/sec execute 419 sec latency 2238.407 ms 2023-07-21T18:51:46.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:51:46 smithi193 ceph-mon[98204]: pgmap v312: 97 pgs: 97 active+clean; 242 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2023-07-21T18:51:46.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:51:46 smithi155 ceph-mon[97934]: pgmap v312: 97 pgs: 97 active+clean; 242 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2023-07-21T18:51:46.678 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:51:46 smithi039 ceph-mon[96489]: pgmap v312: 97 pgs: 97 active+clean; 242 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2023-07-21T18:51:47.286 INFO:tasks.workunit.client.0.smithi039.stdout: 1 597013 5.74 MB/sec execute 420 sec latency 19.813 ms 2023-07-21T18:51:48.286 INFO:tasks.workunit.client.0.smithi039.stdout: 1 598220 5.74 MB/sec execute 421 sec latency 18.710 ms 2023-07-21T18:51:48.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:51:48 smithi193 ceph-mon[98204]: pgmap v313: 97 pgs: 97 active+clean; 237 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 848 KiB/s rd, 1.8 MiB/s wr, 184 op/s 2023-07-21T18:51:48.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:51:48 smithi155 ceph-mon[97934]: pgmap v313: 97 pgs: 97 active+clean; 237 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 848 KiB/s rd, 1.8 MiB/s wr, 184 op/s 2023-07-21T18:51:48.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:51:48 smithi039 ceph-mon[96489]: pgmap v313: 97 pgs: 97 active+clean; 237 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 848 KiB/s rd, 1.8 MiB/s wr, 184 op/s 2023-07-21T18:51:49.286 INFO:tasks.workunit.client.0.smithi039.stdout: 1 598403 5.73 MB/sec execute 422 sec latency 798.032 ms 2023-07-21T18:51:50.286 INFO:tasks.workunit.client.0.smithi039.stdout: 1 598403 5.71 MB/sec execute 423 sec latency 1798.099 ms 2023-07-21T18:51:50.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:51:50 smithi193 ceph-mon[98204]: pgmap v314: 97 pgs: 97 active+clean; 237 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 1.2 MiB/s wr, 122 op/s 2023-07-21T18:51:50.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:51:50 smithi155 ceph-mon[97934]: pgmap v314: 97 pgs: 97 active+clean; 237 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 1.2 MiB/s wr, 122 op/s 2023-07-21T18:51:50.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:51:50 smithi039 ceph-mon[96489]: pgmap v314: 97 pgs: 97 active+clean; 237 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 1.2 MiB/s wr, 122 op/s 2023-07-21T18:51:51.286 INFO:tasks.workunit.client.0.smithi039.stdout: 1 598403 5.70 MB/sec execute 424 sec latency 2798.185 ms 2023-07-21T18:51:52.286 INFO:tasks.workunit.client.0.smithi039.stdout: 1 599627 5.70 MB/sec execute 425 sec latency 3230.412 ms 2023-07-21T18:51:52.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:51:52 smithi193 ceph-mon[98204]: pgmap v315: 97 pgs: 97 active+clean; 240 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 838 KiB/s rd, 1.7 MiB/s wr, 178 op/s 2023-07-21T18:51:52.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:51:52 smithi155 ceph-mon[97934]: pgmap v315: 97 pgs: 97 active+clean; 240 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 838 KiB/s rd, 1.7 MiB/s wr, 178 op/s 2023-07-21T18:51:52.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:51:52 smithi039 ceph-mon[96489]: pgmap v315: 97 pgs: 97 active+clean; 240 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 838 KiB/s rd, 1.7 MiB/s wr, 178 op/s 2023-07-21T18:51:53.286 INFO:tasks.workunit.client.0.smithi039.stdout: 1 600965 5.71 MB/sec execute 426 sec latency 13.591 ms 2023-07-21T18:51:54.286 INFO:tasks.workunit.client.0.smithi039.stdout: 1 601947 5.70 MB/sec execute 427 sec latency 411.104 ms 2023-07-21T18:51:54.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:51:54 smithi193 ceph-mon[98204]: pgmap v316: 97 pgs: 97 active+clean; 240 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 1.2 MiB/s wr, 120 op/s 2023-07-21T18:51:54.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:51:54 smithi155 ceph-mon[97934]: pgmap v316: 97 pgs: 97 active+clean; 240 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 1.2 MiB/s wr, 120 op/s 2023-07-21T18:51:54.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:51:54 smithi039 ceph-mon[96489]: pgmap v316: 97 pgs: 97 active+clean; 240 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 1.2 MiB/s wr, 120 op/s 2023-07-21T18:51:55.286 INFO:tasks.workunit.client.0.smithi039.stdout: 1 601947 5.69 MB/sec execute 428 sec latency 1411.182 ms 2023-07-21T18:51:56.286 INFO:tasks.workunit.client.0.smithi039.stdout: 1 601947 5.68 MB/sec execute 429 sec latency 2411.294 ms 2023-07-21T18:51:56.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:51:56 smithi193 ceph-mon[98204]: pgmap v317: 97 pgs: 97 active+clean; 240 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 1.2 MiB/s wr, 120 op/s 2023-07-21T18:51:56.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:51:56 smithi155 ceph-mon[97934]: pgmap v317: 97 pgs: 97 active+clean; 240 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 1.2 MiB/s wr, 120 op/s 2023-07-21T18:51:56.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:51:56 smithi039 ceph-mon[96489]: pgmap v317: 97 pgs: 97 active+clean; 240 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 1.2 MiB/s wr, 120 op/s 2023-07-21T18:51:57.286 INFO:tasks.workunit.client.0.smithi039.stdout: 1 603169 5.68 MB/sec execute 430 sec latency 2845.893 ms 2023-07-21T18:51:57.903 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.891+0000 7fb5bdcac700 1 -- 172.21.15.39:0/3708514746 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5b8104a30 msgr2=0x7fb5b8104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:51:57.904 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.891+0000 7fb5bdcac700 1 --2- 172.21.15.39:0/3708514746 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5b8104a30 0x7fb5b8104e00 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fb5a8009990 tx=0x7fb5a8009ca0 comp rx=0 tx=0).stop 2023-07-21T18:51:57.904 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.891+0000 7fb5bdcac700 1 -- 172.21.15.39:0/3708514746 shutdown_connections 2023-07-21T18:51:57.904 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.891+0000 7fb5bdcac700 1 --2- 172.21.15.39:0/3708514746 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb5b80ff3e0 0x7fb5b810b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:57.904 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.891+0000 7fb5bdcac700 1 --2- 172.21.15.39:0/3708514746 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5b8104a30 0x7fb5b8104e00 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:57.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.891+0000 7fb5bdcac700 1 --2- 172.21.15.39:0/3708514746 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5b80fea30 0x7fb5b80feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:57.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.891+0000 7fb5bdcac700 1 -- 172.21.15.39:0/3708514746 >> 172.21.15.39:0/3708514746 conn(0x7fb5b8074760 msgr2=0x7fb5b8074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:51:57.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.891+0000 7fb5bdcac700 1 -- 172.21.15.39:0/3708514746 shutdown_connections 2023-07-21T18:51:57.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.891+0000 7fb5bdcac700 1 -- 172.21.15.39:0/3708514746 wait complete. 2023-07-21T18:51:57.907 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.892+0000 7fb5bdcac700 1 Processor -- start 2023-07-21T18:51:57.908 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.892+0000 7fb5bdcac700 1 -- start start 2023-07-21T18:51:57.908 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.892+0000 7fb5bdcac700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5b80fea30 0x7fb5b8196bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:57.908 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.892+0000 7fb5bdcac700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5b80ff3e0 0x7fb5b8197130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:57.908 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.892+0000 7fb5bdcac700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb5b8104a30 0x7fb5b81a1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:57.908 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.892+0000 7fb5bdcac700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b81a2060 con 0x7fb5b80ff3e0 2023-07-21T18:51:57.908 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.892+0000 7fb5bdcac700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b81a21d0 con 0x7fb5b8104a30 2023-07-21T18:51:57.909 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.892+0000 7fb5bdcac700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b81a2390 con 0x7fb5b80fea30 2023-07-21T18:51:57.909 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.892+0000 7fb5b77fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5b80fea30 0x7fb5b8196bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:57.909 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.892+0000 7fb5b6ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5b80ff3e0 0x7fb5b8197130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:57.909 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.892+0000 7fb5b77fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5b80fea30 0x7fb5b8196bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:51022/0 (socket says 172.21.15.39:51022) 2023-07-21T18:51:57.909 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.892+0000 7fb5b77fe700 1 -- 172.21.15.39:0/2635229919 learned_addr learned my addr 172.21.15.39:0/2635229919 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:51:57.909 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.893+0000 7fb5b7fff700 1 --2- 172.21.15.39:0/2635229919 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb5b8104a30 0x7fb5b81a1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:57.909 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.893+0000 7fb5b77fe700 1 -- 172.21.15.39:0/2635229919 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb5b8104a30 msgr2=0x7fb5b81a1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:51:57.910 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.893+0000 7fb5b77fe700 1 --2- 172.21.15.39:0/2635229919 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb5b8104a30 0x7fb5b81a1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:57.910 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.893+0000 7fb5b77fe700 1 -- 172.21.15.39:0/2635229919 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5b80ff3e0 msgr2=0x7fb5b8197130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:51:57.910 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.893+0000 7fb5b77fe700 1 --2- 172.21.15.39:0/2635229919 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5b80ff3e0 0x7fb5b8197130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:57.910 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.893+0000 7fb5b77fe700 1 -- 172.21.15.39:0/2635229919 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5a80095f0 con 0x7fb5b80fea30 2023-07-21T18:51:57.910 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.893+0000 7fb5b6ffd700 1 --2- 172.21.15.39:0/2635229919 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5b80ff3e0 0x7fb5b8197130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:51:57.910 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.893+0000 7fb5b77fe700 1 --2- 172.21.15.39:0/2635229919 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5b80fea30 0x7fb5b8196bf0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb5a8004740 tx=0x7fb5a8009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:51:57.910 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.893+0000 7fb5b4ff9700 1 -- 172.21.15.39:0/2635229919 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a801d070 con 0x7fb5b80fea30 2023-07-21T18:51:57.910 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.893+0000 7fb5bdcac700 1 -- 172.21.15.39:0/2635229919 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5b81a2610 con 0x7fb5b80fea30 2023-07-21T18:51:57.911 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.893+0000 7fb5b4ff9700 1 -- 172.21.15.39:0/2635229919 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb5a8004830 con 0x7fb5b80fea30 2023-07-21T18:51:57.911 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.893+0000 7fb5b4ff9700 1 -- 172.21.15.39:0/2635229919 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a80052c0 con 0x7fb5b80fea30 2023-07-21T18:51:57.911 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.893+0000 7fb5bdcac700 1 -- 172.21.15.39:0/2635229919 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5b81a29d0 con 0x7fb5b80fea30 2023-07-21T18:51:57.911 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.896+0000 7fb5b4ff9700 1 -- 172.21.15.39:0/2635229919 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fb5a8005480 con 0x7fb5b80fea30 2023-07-21T18:51:57.915 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.896+0000 7fb5b4ff9700 1 --2- 172.21.15.39:0/2635229919 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fb5a00a12c0 0x7fb5a00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:57.915 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.896+0000 7fb5b6ffd700 1 --2- 172.21.15.39:0/2635229919 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fb5a00a12c0 0x7fb5a00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:57.915 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.896+0000 7fb5b4ff9700 1 -- 172.21.15.39:0/2635229919 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fb5a80cdd70 con 0x7fb5b80fea30 2023-07-21T18:51:57.915 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.897+0000 7fb5c9e32b80 1 Processor -- start 2023-07-21T18:51:57.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.897+0000 7fb5b6ffd700 1 --2- 172.21.15.39:0/2635229919 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fb5a00a12c0 0x7fb5a00a3770 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb5b80683c0 tx=0x7fb5ac009260 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:51:57.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.897+0000 7fb5c9e32b80 1 -- start start 2023-07-21T18:51:57.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.897+0000 7fb5c9e32b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e89cdc56b0 0x55e89cf6ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:57.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.897+0000 7fb5c9e32b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e89cf6f2d0 0x55e89cf69040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:57.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.897+0000 7fb5c9e32b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e89cf69580 0x55e89cf699d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:57.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.897+0000 7fb5c9e32b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e89ce6ec50 con 0x55e89cf6f2d0 2023-07-21T18:51:57.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.897+0000 7fb5c9e32b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e89cec19e0 con 0x55e89cdc56b0 2023-07-21T18:51:57.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.897+0000 7fb5c9e32b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e89cebae70 con 0x55e89cf69580 2023-07-21T18:51:57.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.897+0000 7fb5b6ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e89cf6f2d0 0x55e89cf69040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:57.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.897+0000 7fb5b77fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e89cdc56b0 0x55e89cf6ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:57.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.897+0000 7fb5b77fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e89cdc56b0 0x55e89cf6ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:45074/0 (socket says 172.21.15.39:45074) 2023-07-21T18:51:57.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.897+0000 7fb5b6ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e89cf6f2d0 0x55e89cf69040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60712/0 (socket says 172.21.15.39:60712) 2023-07-21T18:51:57.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.897+0000 7fb5b77fe700 1 -- 172.21.15.39:0/3164593456 learned_addr learned my addr 172.21.15.39:0/3164593456 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:51:57.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.897+0000 7fb5b7fff700 1 --2- 172.21.15.39:0/3164593456 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e89cf69580 0x55e89cf699d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:57.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.898+0000 7fb5b77fe700 1 -- 172.21.15.39:0/3164593456 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e89cf69580 msgr2=0x55e89cf699d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:51:57.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.898+0000 7fb5b77fe700 1 --2- 172.21.15.39:0/3164593456 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e89cf69580 0x55e89cf699d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:57.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.898+0000 7fb5b77fe700 1 -- 172.21.15.39:0/3164593456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e89cf6f2d0 msgr2=0x55e89cf69040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:51:57.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.898+0000 7fb5b77fe700 1 --2- 172.21.15.39:0/3164593456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e89cf6f2d0 0x55e89cf69040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:57.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.898+0000 7fb5b77fe700 1 -- 172.21.15.39:0/3164593456 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e89cce8d80 con 0x55e89cdc56b0 2023-07-21T18:51:57.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.898+0000 7fb5b77fe700 1 --2- 172.21.15.39:0/3164593456 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e89cdc56b0 0x55e89cf6ed90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb5a809ae60 tx=0x7fb5a80057b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36ed8e71708ffcf3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:51:57.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.898+0000 7fb59dffb700 1 -- 172.21.15.39:0/3164593456 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a80a3290 con 0x55e89cdc56b0 2023-07-21T18:51:57.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.898+0000 7fb59dffb700 1 -- 172.21.15.39:0/3164593456 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb5a809b130 con 0x55e89cdc56b0 2023-07-21T18:51:57.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.898+0000 7fb59dffb700 1 -- 172.21.15.39:0/3164593456 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a80a3000 con 0x55e89cdc56b0 2023-07-21T18:51:57.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.899+0000 7fb5c9e32b80 1 -- 172.21.15.39:0/3164593456 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e89cdc56b0 msgr2=0x55e89cf6ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:51:57.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.899+0000 7fb5c9e32b80 1 --2- 172.21.15.39:0/3164593456 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e89cdc56b0 0x55e89cf6ed90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb5a809ae60 tx=0x7fb5a80057b0 comp rx=0 tx=0).stop 2023-07-21T18:51:57.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.899+0000 7fb5c9e32b80 1 -- 172.21.15.39:0/3164593456 shutdown_connections 2023-07-21T18:51:57.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.899+0000 7fb5c9e32b80 1 --2- 172.21.15.39:0/3164593456 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e89cdc56b0 0x55e89cf6ed90 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:57.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.899+0000 7fb5c9e32b80 1 --2- 172.21.15.39:0/3164593456 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e89cf69580 0x55e89cf699d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:57.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.899+0000 7fb5c9e32b80 1 --2- 172.21.15.39:0/3164593456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e89cf6f2d0 0x55e89cf69040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:57.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.899+0000 7fb5c9e32b80 1 -- 172.21.15.39:0/3164593456 >> 172.21.15.39:0/3164593456 conn(0x55e89cf5ebe0 msgr2=0x55e89cf63f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:51:57.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.899+0000 7fb5c9e32b80 1 -- 172.21.15.39:0/3164593456 shutdown_connections 2023-07-21T18:51:57.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.899+0000 7fb5c9e32b80 1 -- 172.21.15.39:0/3164593456 wait complete. 2023-07-21T18:51:57.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.899+0000 7fb5c9e32b80 1 -- start start 2023-07-21T18:51:57.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.899+0000 7fb5c9e32b80 1 Processor -- start 2023-07-21T18:51:57.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.900+0000 7fb5c9e32b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e89cdc56b0 0x55e89cf78250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:57.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.900+0000 7fb5c9e32b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e89cf69580 0x55e89cf78790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:57.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.900+0000 7fb5c9e32b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e89cf6f2d0 0x55e89cf724e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:57.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.900+0000 7fb5c9e32b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e89ce5aa60 con 0x55e89cf6f2d0 2023-07-21T18:51:57.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.900+0000 7fb5c9e32b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e89ce5a760 con 0x55e89cdc56b0 2023-07-21T18:51:57.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.900+0000 7fb5c9e32b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e89cd3e8c0 con 0x55e89cf69580 2023-07-21T18:51:57.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.900+0000 7fb5c9e32b80 10 client.0 _ll_register_callbacks cb 0x55e89ce58700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:51:57.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.900+0000 7fb5b7fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e89cf6f2d0 0x55e89cf724e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:57.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.900+0000 7fb5b77fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e89cdc56b0 0x55e89cf78250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:57.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.900+0000 7fb5b6ffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e89cf69580 0x55e89cf78790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:57.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.900+0000 7fb5b77fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e89cdc56b0 0x55e89cf78250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:45080/0 (socket says 172.21.15.39:45080) 2023-07-21T18:51:57.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.900+0000 7fb5b6ffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e89cf69580 0x55e89cf78790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:51038/0 (socket says 172.21.15.39:51038) 2023-07-21T18:51:57.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.900+0000 7fb5b77fe700 1 -- 172.21.15.39:0/3538916586 learned_addr learned my addr 172.21.15.39:0/3538916586 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:51:57.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.900+0000 7fb5b77fe700 1 -- 172.21.15.39:0/3538916586 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e89cf69580 msgr2=0x55e89cf78790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:51:57.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.900+0000 7fb5b77fe700 1 --2- 172.21.15.39:0/3538916586 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e89cf69580 0x55e89cf78790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:57.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.900+0000 7fb5b77fe700 1 -- 172.21.15.39:0/3538916586 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e89cf6f2d0 msgr2=0x55e89cf724e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:51:57.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.900+0000 7fb5b77fe700 1 --2- 172.21.15.39:0/3538916586 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e89cf6f2d0 0x55e89cf724e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:57.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.900+0000 7fb5b77fe700 1 -- 172.21.15.39:0/3538916586 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5a80a0040 con 0x55e89cdc56b0 2023-07-21T18:51:57.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.900+0000 7fb5b77fe700 1 --2- 172.21.15.39:0/3538916586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e89cdc56b0 0x55e89cf78250 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb5a80ce050 tx=0x7fb5a8005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d9ab2a13820a734 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:51:57.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.900+0000 7fb596ffd700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-21T18:51:57.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.900+0000 7fb596ffd700 1 -- 172.21.15.39:0/3538916586 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a80a3290 con 0x55e89cdc56b0 2023-07-21T18:51:57.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.901+0000 7fb596ffd700 1 -- 172.21.15.39:0/3538916586 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fb5a80a3010 con 0x55e89cdc56b0 2023-07-21T18:51:57.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.901+0000 7fb5c9e32b80 1 -- 172.21.15.39:0/3538916586 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e89ce14490 con 0x55e89cdc56b0 2023-07-21T18:51:57.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.901+0000 7fb596ffd700 1 -- 172.21.15.39:0/3538916586 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb5a80cb310 con 0x55e89cdc56b0 2023-07-21T18:51:57.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.901+0000 7fb5c9e32b80 10 client.26387 fetch_fsmap learned FSMap version 63 2023-07-21T18:51:57.924 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.901+0000 7fb5c9e32b80 1 -- 172.21.15.39:0/3538916586 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) v3 -- 0x55e89ccd1c00 con 0x55e89cdc56b0 2023-07-21T18:51:57.924 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.901+0000 7fb596ffd700 1 -- 172.21.15.39:0/3538916586 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e89ce14490 con 0x55e89cdc56b0 2023-07-21T18:51:57.924 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.901+0000 7fb596ffd700 1 -- 172.21.15.39:0/3538916586 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 63) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fb5a8095c60 con 0x55e89cdc56b0 2023-07-21T18:51:57.924 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.901+0000 7fb5c9e32b80 10 client.26387 fetch_fsmap finished waiting for FSMap version 63 2023-07-21T18:51:57.924 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.901+0000 7fb5c9e32b80 10 client.26387 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:51:57.924 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.901+0000 7fb5c9e32b80 20 client.26387 populate_metadata read hostname 'smithi039' 2023-07-21T18:51:57.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.901+0000 7fb5c9e32b80 1 --2- 172.21.15.39:0/3538916586 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55e89cf823c0 0x55e89cf847d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:57.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.901+0000 7fb5c9e32b80 4 client.26387 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:51:57.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.901+0000 7fb5c9e32b80 1 -- 172.21.15.39:0/3538916586 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e89ceb3e50 con 0x55e89cf823c0 2023-07-21T18:51:57.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.901+0000 7fb5b6ffd700 1 --2- 172.21.15.39:0/3538916586 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55e89cf823c0 0x55e89cf847d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:57.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.902+0000 7fb596ffd700 10 client.26387 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:51:57.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.902+0000 7fb5b6ffd700 1 --2- 172.21.15.39:0/3538916586 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55e89cf823c0 0x55e89cf847d0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77d9b0488a23d1a3 server_cookie=10e0bbb3e103b0ff in_seq=0 out_seq=0 2023-07-21T18:51:57.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.902+0000 7fb596ffd700 1 -- 172.21.15.39:0/3538916586 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e89ceb3e50 con 0x55e89cf823c0 2023-07-21T18:51:57.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.902+0000 7fb596ffd700 10 client.26387 handle_command_reply: tid=0 2023-07-21T18:51:57.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.902+0000 7fb5c9e32b80 1 client.26387 shutdown 2023-07-21T18:51:57.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.903+0000 7fb5c9e32b80 1 -- 172.21.15.39:0/3538916586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e89cdc56b0 msgr2=0x55e89cf78250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:51:57.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.903+0000 7fb5c9e32b80 1 --2- 172.21.15.39:0/3538916586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e89cdc56b0 0x55e89cf78250 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb5a80ce050 tx=0x7fb5a8005c00 comp rx=0 tx=0).stop 2023-07-21T18:51:57.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.903+0000 7fb5c9e32b80 1 -- 172.21.15.39:0/3538916586 shutdown_connections 2023-07-21T18:51:57.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.903+0000 7fb5c9e32b80 1 --2- 172.21.15.39:0/3538916586 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55e89cf823c0 0x55e89cf847d0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:57.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.903+0000 7fb5c9e32b80 1 --2- 172.21.15.39:0/3538916586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e89cdc56b0 0x55e89cf78250 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:57.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.903+0000 7fb596ffd700 0 client.26387 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:51:57.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.903+0000 7fb5c9e32b80 1 --2- 172.21.15.39:0/3538916586 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e89cf6f2d0 0x55e89cf724e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:57.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.903+0000 7fb5c9e32b80 1 --2- 172.21.15.39:0/3538916586 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e89cf69580 0x55e89cf78790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:57.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.903+0000 7fb5c9e32b80 1 -- 172.21.15.39:0/3538916586 >> 172.21.15.39:0/3538916586 conn(0x55e89cf5ebe0 msgr2=0x55e89cf634e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:51:57.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.903+0000 7fb5c9e32b80 1 -- 172.21.15.39:0/3538916586 shutdown_connections 2023-07-21T18:51:57.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.903+0000 7fb5c9e32b80 1 -- 172.21.15.39:0/3538916586 wait complete. 2023-07-21T18:51:57.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.903+0000 7fb5c9e32b80 20 client.26387 trim_cache size 0 max 16384 2023-07-21T18:51:57.932 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.919+0000 7fb5c9e32b80 1 Processor -- start 2023-07-21T18:51:57.932 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.919+0000 7fb5c9e32b80 1 -- start start 2023-07-21T18:51:57.932 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.919+0000 7fb5c9e32b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e89cdc56b0 0x55e89cf69040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:57.932 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.919+0000 7fb5c9e32b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e89cf69580 0x55e89cf72610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:57.932 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.919+0000 7fb5c9e32b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e89cf6f2d0 0x55e89cf6ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:57.933 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.919+0000 7fb5c9e32b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e89cf65b70 con 0x55e89cf69580 2023-07-21T18:51:57.933 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.919+0000 7fb5c9e32b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e89cf6f6a0 con 0x55e89cdc56b0 2023-07-21T18:51:57.933 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.919+0000 7fb5c9e32b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e89cf60d00 con 0x55e89cf6f2d0 2023-07-21T18:51:57.933 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.920+0000 7fb5b6ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e89cf69580 0x55e89cf72610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:57.933 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.920+0000 7fb5b77fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e89cdc56b0 0x55e89cf69040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:57.933 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.920+0000 7fb5b6ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e89cf69580 0x55e89cf72610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60726/0 (socket says 172.21.15.39:60726) 2023-07-21T18:51:57.933 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.920+0000 7fb5b6ffd700 1 -- 172.21.15.39:0/1629342992 learned_addr learned my addr 172.21.15.39:0/1629342992 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:51:57.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.920+0000 7fb5b77fe700 1 --2- 172.21.15.39:0/1629342992 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e89cdc56b0 0x55e89cf69040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:45088/0 (socket says 172.21.15.39:45088) 2023-07-21T18:51:57.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.920+0000 7fb5b7fff700 1 --2- 172.21.15.39:0/1629342992 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e89cf6f2d0 0x55e89cf6ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:57.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.920+0000 7fb5b77fe700 1 -- 172.21.15.39:0/1629342992 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e89cf6f2d0 msgr2=0x55e89cf6ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:51:57.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.920+0000 7fb5b77fe700 1 --2- 172.21.15.39:0/1629342992 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e89cf6f2d0 0x55e89cf6ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:57.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.920+0000 7fb5b77fe700 1 -- 172.21.15.39:0/1629342992 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e89cf69580 msgr2=0x55e89cf72610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:51:57.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.920+0000 7fb5b77fe700 1 --2- 172.21.15.39:0/1629342992 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e89cf69580 0x55e89cf72610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:57.935 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.920+0000 7fb5b77fe700 1 -- 172.21.15.39:0/1629342992 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e89cf893c0 con 0x55e89cdc56b0 2023-07-21T18:51:57.937 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.920+0000 7fb5b77fe700 1 --2- 172.21.15.39:0/1629342992 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e89cdc56b0 0x55e89cf69040 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb5a809abd0 tx=0x7fb5a80c9fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb40ef76fb527e60 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:51:57.938 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.920+0000 7fb59e7fc700 1 -- 172.21.15.39:0/1629342992 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a8096d90 con 0x55e89cdc56b0 2023-07-21T18:51:57.938 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.920+0000 7fb59e7fc700 1 -- 172.21.15.39:0/1629342992 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb5a80a3290 con 0x55e89cdc56b0 2023-07-21T18:51:57.938 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.920+0000 7fb59e7fc700 1 -- 172.21.15.39:0/1629342992 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a80ba040 con 0x55e89cdc56b0 2023-07-21T18:51:57.938 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.921+0000 7fb5c9e32b80 1 -- 172.21.15.39:0/1629342992 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e89cdc56b0 msgr2=0x55e89cf69040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:51:57.938 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.921+0000 7fb5c9e32b80 1 --2- 172.21.15.39:0/1629342992 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e89cdc56b0 0x55e89cf69040 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb5a809abd0 tx=0x7fb5a80c9fa0 comp rx=0 tx=0).stop 2023-07-21T18:51:57.938 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.921+0000 7fb5c9e32b80 1 -- 172.21.15.39:0/1629342992 shutdown_connections 2023-07-21T18:51:57.938 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.921+0000 7fb5c9e32b80 1 --2- 172.21.15.39:0/1629342992 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e89cdc56b0 0x55e89cf69040 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:57.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.921+0000 7fb5c9e32b80 1 --2- 172.21.15.39:0/1629342992 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e89cf6f2d0 0x55e89cf6ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:57.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.921+0000 7fb5c9e32b80 1 --2- 172.21.15.39:0/1629342992 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e89cf69580 0x55e89cf72610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:57.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.921+0000 7fb5c9e32b80 1 -- 172.21.15.39:0/1629342992 >> 172.21.15.39:0/1629342992 conn(0x55e89cf5ebe0 msgr2=0x55e89cf6f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:51:57.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.921+0000 7fb5c9e32b80 1 -- 172.21.15.39:0/1629342992 shutdown_connections 2023-07-21T18:51:57.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.921+0000 7fb5c9e32b80 1 -- 172.21.15.39:0/1629342992 wait complete. 2023-07-21T18:51:57.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.922+0000 7fb5c9e32b80 1 -- start start 2023-07-21T18:51:57.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.922+0000 7fb5c9e32b80 1 Processor -- start 2023-07-21T18:51:57.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.922+0000 7fb5c9e32b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e89cdc56b0 0x55e89cf75c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:57.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.922+0000 7fb5c9e32b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e89cf69580 0x55e89cf761b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:57.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.922+0000 7fb5c9e32b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e89cf6f2d0 0x55e89cf766f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:57.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.922+0000 7fb5c9e32b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e89cfd9690 con 0x55e89cdc56b0 2023-07-21T18:51:57.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.922+0000 7fb5c9e32b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e89cfd9800 con 0x55e89cf6f2d0 2023-07-21T18:51:57.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.922+0000 7fb5c9e32b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e89cfd99c0 con 0x55e89cf69580 2023-07-21T18:51:57.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.922+0000 7fb5c9e32b80 10 client.0 _ll_register_callbacks cb 0x7fb5ac004030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:51:57.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.922+0000 7fb5b77fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e89cdc56b0 0x55e89cf75c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:57.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.922+0000 7fb5b77fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e89cdc56b0 0x55e89cf75c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60730/0 (socket says 172.21.15.39:60730) 2023-07-21T18:51:57.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.922+0000 7fb5b77fe700 1 -- 172.21.15.39:0/2367170408 learned_addr learned my addr 172.21.15.39:0/2367170408 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:51:57.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.922+0000 7fb5b6ffd700 1 --2- 172.21.15.39:0/2367170408 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e89cf69580 0x55e89cf761b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:57.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.922+0000 7fb5b7fff700 1 --2- 172.21.15.39:0/2367170408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e89cf6f2d0 0x55e89cf766f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:57.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.923+0000 7fb5b77fe700 1 -- 172.21.15.39:0/2367170408 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e89cf69580 msgr2=0x55e89cf761b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:51:57.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.923+0000 7fb5b77fe700 1 --2- 172.21.15.39:0/2367170408 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e89cf69580 0x55e89cf761b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:57.942 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.923+0000 7fb5b77fe700 1 -- 172.21.15.39:0/2367170408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e89cf6f2d0 msgr2=0x55e89cf766f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:51:57.942 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.923+0000 7fb5b77fe700 1 --2- 172.21.15.39:0/2367170408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e89cf6f2d0 0x55e89cf766f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:57.942 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.923+0000 7fb5b77fe700 1 -- 172.21.15.39:0/2367170408 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e89ccd1c00 con 0x55e89cdc56b0 2023-07-21T18:51:57.942 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.923+0000 7fb5b77fe700 1 --2- 172.21.15.39:0/2367170408 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e89cdc56b0 0x55e89cf75c70 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fb5a8099220 tx=0x7fb5a80c9f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7013c7b384ea4f98 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:51:57.942 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.923+0000 7fb596ffd700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:51:57.942 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.923+0000 7fb596ffd700 1 -- 172.21.15.39:0/2367170408 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a80c9070 con 0x55e89cdc56b0 2023-07-21T18:51:57.942 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.923+0000 7fb596ffd700 1 -- 172.21.15.39:0/2367170408 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fb5a80ba040 con 0x55e89cdc56b0 2023-07-21T18:51:57.942 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.923+0000 7fb5c9e32b80 1 -- 172.21.15.39:0/2367170408 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb5a809b130 con 0x55e89cdc56b0 2023-07-21T18:51:57.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.923+0000 7fb596ffd700 1 -- 172.21.15.39:0/2367170408 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb5a80ba240 con 0x55e89cdc56b0 2023-07-21T18:51:57.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.923+0000 7fb5c9e32b80 10 client.16926 fetch_fsmap learned FSMap version 63 2023-07-21T18:51:57.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.923+0000 7fb5c9e32b80 1 -- 172.21.15.39:0/2367170408 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) v3 -- 0x55e89cd0f1f0 con 0x55e89cdc56b0 2023-07-21T18:51:57.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.923+0000 7fb596ffd700 1 -- 172.21.15.39:0/2367170408 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb5a809b130 con 0x55e89cdc56b0 2023-07-21T18:51:57.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.923+0000 7fb596ffd700 1 -- 172.21.15.39:0/2367170408 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 63) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fb5a8095c50 con 0x55e89cdc56b0 2023-07-21T18:51:57.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.923+0000 7fb5c9e32b80 10 client.16926 fetch_fsmap finished waiting for FSMap version 63 2023-07-21T18:51:57.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.923+0000 7fb5c9e32b80 10 client.16926 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:51:57.944 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.923+0000 7fb5c9e32b80 20 client.16926 populate_metadata read hostname 'smithi039' 2023-07-21T18:51:57.944 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.923+0000 7fb5c9e32b80 1 --2- 172.21.15.39:0/2367170408 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55e89cfdabe0 0x55e89cfdcfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:51:57.944 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.923+0000 7fb5c9e32b80 4 client.16926 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:51:57.944 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.923+0000 7fb5c9e32b80 1 -- 172.21.15.39:0/2367170408 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e89ce75ff0 con 0x55e89cfdabe0 2023-07-21T18:51:57.944 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.924+0000 7fb5b6ffd700 1 --2- 172.21.15.39:0/2367170408 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55e89cfdabe0 0x55e89cfdcfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:51:57.944 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.924+0000 7fb596ffd700 10 client.16926 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:51:57.944 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.924+0000 7fb5b6ffd700 1 --2- 172.21.15.39:0/2367170408 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55e89cfdabe0 0x55e89cfdcfc0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ff1de9820d2d46a server_cookie=3764a6dd847c50a5 in_seq=0 out_seq=0 2023-07-21T18:51:57.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.964+0000 7fb596ffd700 1 -- 172.21.15.39:0/2367170408 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e89ce75ff0 con 0x55e89cfdabe0 2023-07-21T18:51:57.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.964+0000 7fb596ffd700 10 client.16926 handle_command_reply: tid=0 2023-07-21T18:51:57.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.964+0000 7fb5c9e32b80 1 client.16926 shutdown 2023-07-21T18:51:57.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.964+0000 7fb5c9e32b80 1 -- 172.21.15.39:0/2367170408 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e89cdc56b0 msgr2=0x55e89cf75c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:51:57.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.964+0000 7fb5c9e32b80 1 --2- 172.21.15.39:0/2367170408 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e89cdc56b0 0x55e89cf75c70 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fb5a8099220 tx=0x7fb5a80c9f00 comp rx=0 tx=0).stop 2023-07-21T18:51:57.978 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.964+0000 7fb5c9e32b80 1 -- 172.21.15.39:0/2367170408 shutdown_connections 2023-07-21T18:51:57.978 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.964+0000 7fb5c9e32b80 1 --2- 172.21.15.39:0/2367170408 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55e89cfdabe0 0x55e89cfdcfc0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:57.978 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.964+0000 7fb5c9e32b80 1 --2- 172.21.15.39:0/2367170408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e89cf6f2d0 0x55e89cf766f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:57.978 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.964+0000 7fb596ffd700 0 client.16926 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:51:57.978 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.964+0000 7fb5c9e32b80 1 --2- 172.21.15.39:0/2367170408 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e89cdc56b0 0x55e89cf75c70 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:57.978 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.964+0000 7fb5c9e32b80 1 --2- 172.21.15.39:0/2367170408 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e89cf69580 0x55e89cf761b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:57.978 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.964+0000 7fb5c9e32b80 1 -- 172.21.15.39:0/2367170408 >> 172.21.15.39:0/2367170408 conn(0x55e89cf5ebe0 msgr2=0x55e89cf6ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:51:57.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.965+0000 7fb5c9e32b80 1 -- 172.21.15.39:0/2367170408 shutdown_connections 2023-07-21T18:51:57.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.965+0000 7fb5c9e32b80 1 -- 172.21.15.39:0/2367170408 wait complete. 2023-07-21T18:51:57.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.965+0000 7fb5c9e32b80 20 client.16926 trim_cache size 0 max 16384 2023-07-21T18:51:57.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:51:57.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:51:57.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:51:57.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:51:57.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.965+0000 7fb5bdcac700 1 -- 172.21.15.39:0/2635229919 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fb5a00a12c0 msgr2=0x7fb5a00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:51:57.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.965+0000 7fb5bdcac700 1 --2- 172.21.15.39:0/2635229919 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fb5a00a12c0 0x7fb5a00a3770 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb5b80683c0 tx=0x7fb5ac009260 comp rx=0 tx=0).stop 2023-07-21T18:51:57.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.965+0000 7fb5bdcac700 1 -- 172.21.15.39:0/2635229919 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5b80fea30 msgr2=0x7fb5b8196bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:51:57.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.965+0000 7fb5bdcac700 1 --2- 172.21.15.39:0/2635229919 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5b80fea30 0x7fb5b8196bf0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb5a8004740 tx=0x7fb5a8009580 comp rx=0 tx=0).stop 2023-07-21T18:51:57.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.965+0000 7fb5bdcac700 1 -- 172.21.15.39:0/2635229919 shutdown_connections 2023-07-21T18:51:57.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.965+0000 7fb5bdcac700 1 --2- 172.21.15.39:0/2635229919 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fb5a00a12c0 0x7fb5a00a3770 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:57.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.965+0000 7fb5bdcac700 1 --2- 172.21.15.39:0/2635229919 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb5b8104a30 0x7fb5b81a1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:57.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.965+0000 7fb5bdcac700 1 --2- 172.21.15.39:0/2635229919 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5b80fea30 0x7fb5b8196bf0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:57.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.965+0000 7fb5bdcac700 1 --2- 172.21.15.39:0/2635229919 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5b80ff3e0 0x7fb5b8197130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:51:57.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.965+0000 7fb5bdcac700 1 -- 172.21.15.39:0/2635229919 >> 172.21.15.39:0/2635229919 conn(0x7fb5b8074760 msgr2=0x7fb5b80fd030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:51:57.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.965+0000 7fb5bdcac700 1 -- 172.21.15.39:0/2635229919 shutdown_connections 2023-07-21T18:51:57.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:51:57.965+0000 7fb5bdcac700 1 -- 172.21.15.39:0/2635229919 wait complete. 2023-07-21T18:51:58.286 INFO:tasks.workunit.client.0.smithi039.stdout: 1 604264 5.68 MB/sec execute 431 sec latency 56.086 ms 2023-07-21T18:51:58.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:51:58 smithi193 ceph-mon[98204]: pgmap v318: 97 pgs: 97 active+clean; 243 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 1.7 MiB/s wr, 177 op/s 2023-07-21T18:51:58.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:51:58 smithi155 ceph-mon[97934]: pgmap v318: 97 pgs: 97 active+clean; 243 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 1.7 MiB/s wr, 177 op/s 2023-07-21T18:51:58.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:51:58 smithi039 ceph-mon[96489]: pgmap v318: 97 pgs: 97 active+clean; 243 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 1.7 MiB/s wr, 177 op/s 2023-07-21T18:51:59.286 INFO:tasks.workunit.client.0.smithi039.stdout: 1 605495 5.68 MB/sec execute 432 sec latency 138.094 ms 2023-07-21T18:51:59.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:51:59 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:51:59.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:51:59 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:51:59.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:51:59 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:51:59.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:51:59 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:51:59.678 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:51:59 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:51:59.678 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:51:59 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:52:00.286 INFO:tasks.workunit.client.0.smithi039.stdout: 1 605495 5.67 MB/sec execute 433 sec latency 1138.187 ms 2023-07-21T18:52:00.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:00 smithi193 ceph-mon[98204]: pgmap v319: 97 pgs: 97 active+clean; 243 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 538 KiB/s rd, 1.1 MiB/s wr, 112 op/s 2023-07-21T18:52:00.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:00 smithi155 ceph-mon[97934]: pgmap v319: 97 pgs: 97 active+clean; 243 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 538 KiB/s rd, 1.1 MiB/s wr, 112 op/s 2023-07-21T18:52:00.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:00 smithi039 ceph-mon[96489]: pgmap v319: 97 pgs: 97 active+clean; 243 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 538 KiB/s rd, 1.1 MiB/s wr, 112 op/s 2023-07-21T18:52:01.286 INFO:tasks.workunit.client.0.smithi039.stdout: 1 605495 5.66 MB/sec execute 434 sec latency 2138.222 ms 2023-07-21T18:52:02.286 INFO:tasks.workunit.client.0.smithi039.stdout: 1 606096 5.65 MB/sec execute 435 sec latency 2566.804 ms 2023-07-21T18:52:02.360 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:01 smithi193 ceph-mon[98204]: pgmap v320: 97 pgs: 97 active+clean; 243 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 957 KiB/s rd, 1.9 MiB/s wr, 180 op/s 2023-07-21T18:52:02.360 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:01 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:52:02.360 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:01 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:52:02.360 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:01 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:52:02.361 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:01 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:52:02.361 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:01 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:52:02.361 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:01 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:52:02.361 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:01 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.361 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:01 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.361 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:01 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.361 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:01 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.362 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:01 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.362 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:01 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.362 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:01 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.362 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:01 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.362 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:01 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.362 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:01 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.362 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:01 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.362 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:01 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.363 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:01 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.363 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:01 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.363 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:01 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.363 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:01 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.363 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:01 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.363 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:01 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.363 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:01 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.363 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:01 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.364 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:01 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.364 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:01 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.364 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:01 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.364 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:01 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.364 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:01 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:52:02.364 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:01 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:52:02.364 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:01 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:52:02.379 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:01 smithi155 ceph-mon[97934]: pgmap v320: 97 pgs: 97 active+clean; 243 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 957 KiB/s rd, 1.9 MiB/s wr, 180 op/s 2023-07-21T18:52:02.379 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:01 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:52:02.379 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:01 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:52:02.379 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:01 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:52:02.379 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:01 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:52:02.380 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:01 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:52:02.380 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:01 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:52:02.380 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:01 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.380 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:01 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.380 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:01 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.380 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:01 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.380 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:01 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.381 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:01 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.381 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:01 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.381 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:01 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.381 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:01 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.381 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:01 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.381 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:01 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.381 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:01 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.381 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:01 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.382 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:01 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.382 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:01 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.382 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:01 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.382 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:01 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.382 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:01 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.382 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:01 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.382 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:01 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.382 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:01 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.383 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:01 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.383 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:01 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.383 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:01 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.383 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:01 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:52:02.383 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:01 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:52:02.383 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:01 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:52:02.428 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:01 smithi039 ceph-mon[96489]: pgmap v320: 97 pgs: 97 active+clean; 243 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 957 KiB/s rd, 1.9 MiB/s wr, 180 op/s 2023-07-21T18:52:02.428 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:01 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:52:02.428 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:01 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:52:02.428 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:01 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:52:02.428 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:01 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:52:02.428 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:01 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:52:02.429 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:01 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:52:02.429 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:01 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.429 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:01 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.429 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:01 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.429 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:01 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.429 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:01 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.429 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:01 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.430 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:01 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.430 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:01 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.430 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:01 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.430 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:01 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.430 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:01 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.430 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:01 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.430 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:01 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.430 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:01 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.431 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:01 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.431 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:01 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.431 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:01 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.431 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:01 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.431 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:01 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.431 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:01 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.431 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:01 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.432 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:01 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.432 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:01 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.432 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:01 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:52:02.432 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:01 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:52:02.432 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:01 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:52:02.432 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:01 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:52:03.286 INFO:tasks.workunit.client.0.smithi039.stdout: 1 607617 5.66 MB/sec execute 436 sec latency 23.680 ms 2023-07-21T18:52:03.360 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:02 smithi193 ceph-mon[98204]: Adjusting osd_memory_target on smithi193 to 196.2M 2023-07-21T18:52:03.360 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:02 smithi193 ceph-mon[98204]: Unable to set osd_memory_target on smithi193 to 205773926: error parsing value: Value '205773926' is below minimum 939524096 2023-07-21T18:52:03.360 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:02 smithi193 ceph-mon[98204]: Adjusting osd_memory_target on smithi155 to 196.2M 2023-07-21T18:52:03.361 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:02 smithi193 ceph-mon[98204]: Adjusting osd_memory_target on smithi039 to 196.2M 2023-07-21T18:52:03.361 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:02 smithi193 ceph-mon[98204]: Unable to set osd_memory_target on smithi155 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-07-21T18:52:03.361 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:02 smithi193 ceph-mon[98204]: Unable to set osd_memory_target on smithi039 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-07-21T18:52:03.378 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:02 smithi155 ceph-mon[97934]: Adjusting osd_memory_target on smithi193 to 196.2M 2023-07-21T18:52:03.379 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:02 smithi155 ceph-mon[97934]: Unable to set osd_memory_target on smithi193 to 205773926: error parsing value: Value '205773926' is below minimum 939524096 2023-07-21T18:52:03.379 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:02 smithi155 ceph-mon[97934]: Adjusting osd_memory_target on smithi155 to 196.2M 2023-07-21T18:52:03.379 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:02 smithi155 ceph-mon[97934]: Adjusting osd_memory_target on smithi039 to 196.2M 2023-07-21T18:52:03.379 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:02 smithi155 ceph-mon[97934]: Unable to set osd_memory_target on smithi155 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-07-21T18:52:03.379 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:02 smithi155 ceph-mon[97934]: Unable to set osd_memory_target on smithi039 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-07-21T18:52:03.427 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:02 smithi039 ceph-mon[96489]: Adjusting osd_memory_target on smithi193 to 196.2M 2023-07-21T18:52:03.428 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:02 smithi039 ceph-mon[96489]: Unable to set osd_memory_target on smithi193 to 205773926: error parsing value: Value '205773926' is below minimum 939524096 2023-07-21T18:52:03.428 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:02 smithi039 ceph-mon[96489]: Adjusting osd_memory_target on smithi155 to 196.2M 2023-07-21T18:52:03.428 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:02 smithi039 ceph-mon[96489]: Adjusting osd_memory_target on smithi039 to 196.2M 2023-07-21T18:52:03.428 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:02 smithi039 ceph-mon[96489]: Unable to set osd_memory_target on smithi155 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-07-21T18:52:03.428 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:02 smithi039 ceph-mon[96489]: Unable to set osd_memory_target on smithi039 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-07-21T18:52:04.287 INFO:tasks.workunit.client.0.smithi039.stdout: 1 608754 5.66 MB/sec execute 437 sec latency 25.462 ms 2023-07-21T18:52:04.360 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:03 smithi193 ceph-mon[98204]: pgmap v321: 97 pgs: 97 active+clean; 243 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 1.3 MiB/s wr, 124 op/s 2023-07-21T18:52:04.360 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:03 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:52:04.379 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:03 smithi155 ceph-mon[97934]: pgmap v321: 97 pgs: 97 active+clean; 243 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 1.3 MiB/s wr, 124 op/s 2023-07-21T18:52:04.379 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:03 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:52:04.427 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:03 smithi039 ceph-mon[96489]: pgmap v321: 97 pgs: 97 active+clean; 243 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 1.3 MiB/s wr, 124 op/s 2023-07-21T18:52:04.428 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:03 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:52:05.286 INFO:tasks.workunit.client.0.smithi039.stdout: 1 609036 5.65 MB/sec execute 438 sec latency 775.004 ms 2023-07-21T18:52:05.360 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:05 smithi193 ceph-mon[98204]: osdmap e92: 12 total, 12 up, 12 in 2023-07-21T18:52:05.360 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:05 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:52:05.378 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:04 smithi155 ceph-mon[97934]: osdmap e92: 12 total, 12 up, 12 in 2023-07-21T18:52:05.379 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:04 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:52:05.427 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:04 smithi039 ceph-mon[96489]: osdmap e92: 12 total, 12 up, 12 in 2023-07-21T18:52:05.428 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:04 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:52:06.286 INFO:tasks.workunit.client.0.smithi039.stdout: 1 609604 5.64 MB/sec execute 439 sec latency 1569.085 ms 2023-07-21T18:52:06.359 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:06 smithi193 ceph-mon[98204]: pgmap v323: 97 pgs: 97 active+clean; 243 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 1.5 MiB/s wr, 149 op/s 2023-07-21T18:52:06.378 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:06 smithi155 ceph-mon[97934]: pgmap v323: 97 pgs: 97 active+clean; 243 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 1.5 MiB/s wr, 149 op/s 2023-07-21T18:52:06.427 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:06 smithi039 ceph-mon[96489]: pgmap v323: 97 pgs: 97 active+clean; 243 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 1.5 MiB/s wr, 149 op/s 2023-07-21T18:52:07.286 INFO:tasks.workunit.client.0.smithi039.stdout: 1 611195 5.65 MB/sec execute 440 sec latency 14.123 ms 2023-07-21T18:52:08.287 INFO:tasks.workunit.client.0.smithi039.stdout: 1 612543 5.65 MB/sec execute 441 sec latency 20.689 ms 2023-07-21T18:52:08.427 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:08 smithi039 ceph-mon[96489]: pgmap v324: 97 pgs: 97 active+clean; 243 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 910 KiB/s rd, 1.5 MiB/s wr, 184 op/s 2023-07-21T18:52:08.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:08 smithi193 ceph-mon[98204]: pgmap v324: 97 pgs: 97 active+clean; 243 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 910 KiB/s rd, 1.5 MiB/s wr, 184 op/s 2023-07-21T18:52:08.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:08 smithi155 ceph-mon[97934]: pgmap v324: 97 pgs: 97 active+clean; 243 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 910 KiB/s rd, 1.5 MiB/s wr, 184 op/s 2023-07-21T18:52:09.287 INFO:tasks.workunit.client.0.smithi039.stdout: 1 612586 5.64 MB/sec execute 442 sec latency 963.826 ms 2023-07-21T18:52:10.287 INFO:tasks.workunit.client.0.smithi039.stdout: 1 612586 5.63 MB/sec execute 443 sec latency 1963.921 ms 2023-07-21T18:52:10.427 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:10 smithi039 ceph-mon[96489]: pgmap v325: 97 pgs: 97 active+clean; 243 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 910 KiB/s rd, 1.5 MiB/s wr, 184 op/s 2023-07-21T18:52:10.428 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:10 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:52:10.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:10 smithi193 ceph-mon[98204]: pgmap v325: 97 pgs: 97 active+clean; 243 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 910 KiB/s rd, 1.5 MiB/s wr, 184 op/s 2023-07-21T18:52:10.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:10 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:52:10.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:10 smithi155 ceph-mon[97934]: pgmap v325: 97 pgs: 97 active+clean; 243 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 910 KiB/s rd, 1.5 MiB/s wr, 184 op/s 2023-07-21T18:52:10.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:10 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:52:11.104 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.088+0000 7f561d959700 1 -- 172.21.15.39:0/1809990895 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56181015e0 msgr2=0x7f5618109cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:52:11.104 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.088+0000 7f561d959700 1 --2- 172.21.15.39:0/1809990895 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56181015e0 0x7f5618109cb0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f560c009990 tx=0x7f560c009ca0 comp rx=0 tx=0).stop 2023-07-21T18:52:11.105 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.088+0000 7f561d959700 1 -- 172.21.15.39:0/1809990895 shutdown_connections 2023-07-21T18:52:11.105 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.088+0000 7f561d959700 1 --2- 172.21.15.39:0/1809990895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f561810a1f0 0x7f561810c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:11.105 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.088+0000 7f561d959700 1 --2- 172.21.15.39:0/1809990895 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5618100cd0 0x7f56181010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:11.105 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.088+0000 7f561d959700 1 --2- 172.21.15.39:0/1809990895 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56181015e0 0x7f5618109cb0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:11.106 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.088+0000 7f561d959700 1 -- 172.21.15.39:0/1809990895 >> 172.21.15.39:0/1809990895 conn(0x7f5618075980 msgr2=0x7f5618075d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:52:11.106 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.088+0000 7f561d959700 1 -- 172.21.15.39:0/1809990895 shutdown_connections 2023-07-21T18:52:11.106 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.088+0000 7f561d959700 1 -- 172.21.15.39:0/1809990895 wait complete. 2023-07-21T18:52:11.106 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.089+0000 7f561d959700 1 Processor -- start 2023-07-21T18:52:11.106 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.089+0000 7f561d959700 1 -- start start 2023-07-21T18:52:11.106 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.089+0000 7f561d959700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5618100cd0 0x7f56180fece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:11.106 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.089+0000 7f561d959700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56181015e0 0x7f56180ff220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:11.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.089+0000 7f561d959700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f561810a1f0 0x7f56181a3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:11.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.089+0000 7f561d959700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56181a4410 con 0x7f5618100cd0 2023-07-21T18:52:11.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.089+0000 7f561d959700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56181a4550 con 0x7f561810a1f0 2023-07-21T18:52:11.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.089+0000 7f561d959700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56181a46c0 con 0x7f56181015e0 2023-07-21T18:52:11.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.090+0000 7f5616ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5618100cd0 0x7f56180fece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:11.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.090+0000 7f5616ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5618100cd0 0x7f56180fece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60742/0 (socket says 172.21.15.39:60742) 2023-07-21T18:52:11.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.090+0000 7f5616ffd700 1 -- 172.21.15.39:0/3882815888 learned_addr learned my addr 172.21.15.39:0/3882815888 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:52:11.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.090+0000 7f56167fc700 1 --2- 172.21.15.39:0/3882815888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56181015e0 0x7f56180ff220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:11.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.090+0000 7f56177fe700 1 --2- 172.21.15.39:0/3882815888 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f561810a1f0 0x7f56181a3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:11.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.090+0000 7f5616ffd700 1 -- 172.21.15.39:0/3882815888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56181015e0 msgr2=0x7f56180ff220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:52:11.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.090+0000 7f5616ffd700 1 --2- 172.21.15.39:0/3882815888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56181015e0 0x7f56180ff220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:11.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.090+0000 7f5616ffd700 1 -- 172.21.15.39:0/3882815888 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f561810a1f0 msgr2=0x7f56181a3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:52:11.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.090+0000 7f5616ffd700 1 --2- 172.21.15.39:0/3882815888 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f561810a1f0 0x7f56181a3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:11.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.090+0000 7f5616ffd700 1 -- 172.21.15.39:0/3882815888 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f560c0095f0 con 0x7f5618100cd0 2023-07-21T18:52:11.109 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.090+0000 7f56177fe700 1 --2- 172.21.15.39:0/3882815888 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f561810a1f0 0x7f56181a3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:52:11.109 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.090+0000 7f5616ffd700 1 --2- 172.21.15.39:0/3882815888 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5618100cd0 0x7f56180fece0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f560800e9c0 tx=0x7f560800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:52:11.109 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.090+0000 7f561c957700 1 -- 172.21.15.39:0/3882815888 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56080097b0 con 0x7f5618100cd0 2023-07-21T18:52:11.109 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.090+0000 7f561c957700 1 -- 172.21.15.39:0/3882815888 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5608004cf0 con 0x7f5618100cd0 2023-07-21T18:52:11.109 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.090+0000 7f561d959700 1 -- 172.21.15.39:0/3882815888 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56181a49a0 con 0x7f5618100cd0 2023-07-21T18:52:11.109 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.090+0000 7f561c957700 1 -- 172.21.15.39:0/3882815888 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5608005600 con 0x7f5618100cd0 2023-07-21T18:52:11.109 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.090+0000 7f561d959700 1 -- 172.21.15.39:0/3882815888 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56181a4e60 con 0x7f5618100cd0 2023-07-21T18:52:11.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.093+0000 7f561c957700 1 -- 172.21.15.39:0/3882815888 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f560800cc60 con 0x7f5618100cd0 2023-07-21T18:52:11.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.093+0000 7f561c957700 1 --2- 172.21.15.39:0/3882815888 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f56000a12c0 0x7f56000a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:11.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.093+0000 7f56167fc700 1 --2- 172.21.15.39:0/3882815888 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f56000a12c0 0x7f56000a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:11.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.093+0000 7f561c957700 1 -- 172.21.15.39:0/3882815888 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f5608014070 con 0x7f5618100cd0 2023-07-21T18:52:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.094+0000 7f56167fc700 1 --2- 172.21.15.39:0/3882815888 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f56000a12c0 0x7f56000a3770 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5618100500 tx=0x7f560c005fd0 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:52:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.094+0000 7f5629adfb80 1 Processor -- start 2023-07-21T18:52:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.094+0000 7f5629adfb80 1 -- start start 2023-07-21T18:52:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.094+0000 7f5629adfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56175b2b86b0 0x56175b461d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.094+0000 7f5629adfb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56175b4622d0 0x56175b45c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.094+0000 7f5629adfb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56175b45c580 0x56175b45c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.094+0000 7f5629adfb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x56175b361c50 con 0x56175b4622d0 2023-07-21T18:52:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.094+0000 7f5629adfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56175b3b49e0 con 0x56175b2b86b0 2023-07-21T18:52:11.116 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.094+0000 7f5629adfb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x56175b3ade70 con 0x56175b45c580 2023-07-21T18:52:11.116 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.094+0000 7f56167fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56175b4622d0 0x56175b45c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:11.116 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.095+0000 7f56167fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56175b4622d0 0x56175b45c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60752/0 (socket says 172.21.15.39:60752) 2023-07-21T18:52:11.116 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.095+0000 7f56167fc700 1 -- 172.21.15.39:0/2882106864 learned_addr learned my addr 172.21.15.39:0/2882106864 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:52:11.116 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.095+0000 7f5616ffd700 1 --2- 172.21.15.39:0/2882106864 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56175b2b86b0 0x56175b461d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:11.116 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.095+0000 7f56177fe700 1 --2- 172.21.15.39:0/2882106864 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56175b45c580 0x56175b45c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:11.116 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.095+0000 7f56167fc700 1 -- 172.21.15.39:0/2882106864 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56175b45c580 msgr2=0x56175b45c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:52:11.117 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.095+0000 7f56167fc700 1 --2- 172.21.15.39:0/2882106864 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56175b45c580 0x56175b45c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:11.117 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.095+0000 7f56167fc700 1 -- 172.21.15.39:0/2882106864 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56175b2b86b0 msgr2=0x56175b461d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:52:11.117 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.095+0000 7f56167fc700 1 --2- 172.21.15.39:0/2882106864 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56175b2b86b0 0x56175b461d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:11.117 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.095+0000 7f56167fc700 1 -- 172.21.15.39:0/2882106864 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56175b1dbd80 con 0x56175b4622d0 2023-07-21T18:52:11.117 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.095+0000 7f56167fc700 1 --2- 172.21.15.39:0/2882106864 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56175b4622d0 0x56175b45c040 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f560c009990 tx=0x7f560c020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f92ec290f51842a6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:52:11.117 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.095+0000 7f55fdffb700 1 -- 172.21.15.39:0/2882106864 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f560c02b070 con 0x56175b4622d0 2023-07-21T18:52:11.117 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.095+0000 7f55fdffb700 1 -- 172.21.15.39:0/2882106864 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f560c01e040 con 0x56175b4622d0 2023-07-21T18:52:11.118 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.095+0000 7f55fdffb700 1 -- 172.21.15.39:0/2882106864 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f560c00f660 con 0x56175b4622d0 2023-07-21T18:52:11.118 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.096+0000 7f5629adfb80 1 -- 172.21.15.39:0/2882106864 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56175b4622d0 msgr2=0x56175b45c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:52:11.118 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.096+0000 7f5629adfb80 1 --2- 172.21.15.39:0/2882106864 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56175b4622d0 0x56175b45c040 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f560c009990 tx=0x7f560c020000 comp rx=0 tx=0).stop 2023-07-21T18:52:11.118 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.096+0000 7f5629adfb80 1 -- 172.21.15.39:0/2882106864 shutdown_connections 2023-07-21T18:52:11.118 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.096+0000 7f5629adfb80 1 --2- 172.21.15.39:0/2882106864 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56175b2b86b0 0x56175b461d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:11.118 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.096+0000 7f5629adfb80 1 --2- 172.21.15.39:0/2882106864 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56175b45c580 0x56175b45c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:11.118 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.096+0000 7f5629adfb80 1 --2- 172.21.15.39:0/2882106864 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56175b4622d0 0x56175b45c040 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:11.118 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.096+0000 7f5629adfb80 1 -- 172.21.15.39:0/2882106864 >> 172.21.15.39:0/2882106864 conn(0x56175b451be0 msgr2=0x56175b456f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:52:11.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.096+0000 7f5629adfb80 1 -- 172.21.15.39:0/2882106864 shutdown_connections 2023-07-21T18:52:11.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.096+0000 7f5629adfb80 1 -- 172.21.15.39:0/2882106864 wait complete. 2023-07-21T18:52:11.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.096+0000 7f5629adfb80 1 -- start start 2023-07-21T18:52:11.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.097+0000 7f5629adfb80 1 Processor -- start 2023-07-21T18:52:11.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.097+0000 7f5629adfb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56175b2b86b0 0x56175b468640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:11.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.097+0000 7f5629adfb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56175b45c580 0x56175b4658c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:11.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.097+0000 7f5629adfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56175b4622d0 0x56175b466000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.097+0000 7f5629adfb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x56175b34da60 con 0x56175b2b86b0 2023-07-21T18:52:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.097+0000 7f5629adfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56175b34d760 con 0x56175b4622d0 2023-07-21T18:52:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.097+0000 7f5629adfb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x56175b2318c0 con 0x56175b45c580 2023-07-21T18:52:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.097+0000 7f5629adfb80 10 client.0 _ll_register_callbacks cb 0x56175b34b700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:52:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.097+0000 7f5616ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56175b2b86b0 0x56175b468640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.097+0000 7f5616ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56175b2b86b0 0x56175b468640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60756/0 (socket says 172.21.15.39:60756) 2023-07-21T18:52:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.097+0000 7f5616ffd700 1 -- 172.21.15.39:0/133200184 learned_addr learned my addr 172.21.15.39:0/133200184 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:52:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.097+0000 7f56167fc700 1 --2- 172.21.15.39:0/133200184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56175b45c580 0x56175b4658c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.097+0000 7f56177fe700 1 --2- 172.21.15.39:0/133200184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56175b4622d0 0x56175b466000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.097+0000 7f5616ffd700 1 -- 172.21.15.39:0/133200184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56175b45c580 msgr2=0x56175b4658c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:52:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.097+0000 7f5616ffd700 1 --2- 172.21.15.39:0/133200184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56175b45c580 0x56175b4658c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.097+0000 7f5616ffd700 1 -- 172.21.15.39:0/133200184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56175b4622d0 msgr2=0x56175b466000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:52:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.097+0000 7f5616ffd700 1 --2- 172.21.15.39:0/133200184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56175b4622d0 0x56175b466000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.097+0000 7f5616ffd700 1 -- 172.21.15.39:0/133200184 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f561810d2d0 con 0x56175b2b86b0 2023-07-21T18:52:11.122 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.098+0000 7f5616ffd700 1 --2- 172.21.15.39:0/133200184 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56175b2b86b0 0x56175b468640 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f56080ca7d0 tx=0x7f5608004f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=557f65bae7e3775a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:52:11.122 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.098+0000 7f55f6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:52:11.122 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.098+0000 7f55f6ffd700 1 -- 172.21.15.39:0/133200184 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f560809a550 con 0x56175b2b86b0 2023-07-21T18:52:11.122 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.098+0000 7f55f6ffd700 1 -- 172.21.15.39:0/133200184 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f56080a2540 con 0x56175b2b86b0 2023-07-21T18:52:11.122 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.098+0000 7f5629adfb80 1 -- 172.21.15.39:0/133200184 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56175b307490 con 0x56175b2b86b0 2023-07-21T18:52:11.122 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.098+0000 7f55f6ffd700 1 -- 172.21.15.39:0/133200184 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56080995e0 con 0x56175b2b86b0 2023-07-21T18:52:11.122 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.098+0000 7f5629adfb80 10 client.16938 fetch_fsmap learned FSMap version 67 2023-07-21T18:52:11.123 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.098+0000 7f5629adfb80 1 -- 172.21.15.39:0/133200184 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) v3 -- 0x56175b1c4c00 con 0x56175b2b86b0 2023-07-21T18:52:11.123 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.098+0000 7f55f6ffd700 1 -- 172.21.15.39:0/133200184 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56175b307490 con 0x56175b2b86b0 2023-07-21T18:52:11.123 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.098+0000 7f55f6ffd700 1 -- 172.21.15.39:0/133200184 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 67) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f56080b8250 con 0x56175b2b86b0 2023-07-21T18:52:11.123 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.098+0000 7f5629adfb80 10 client.16938 fetch_fsmap finished waiting for FSMap version 67 2023-07-21T18:52:11.123 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.098+0000 7f5629adfb80 10 client.16938 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:52:11.123 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.098+0000 7f5629adfb80 20 client.16938 populate_metadata read hostname 'smithi039' 2023-07-21T18:52:11.123 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.098+0000 7f5629adfb80 1 --2- 172.21.15.39:0/133200184 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x56175b475900 0x56175b477d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:11.123 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.098+0000 7f5629adfb80 4 client.16938 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:52:11.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.098+0000 7f5629adfb80 1 -- 172.21.15.39:0/133200184 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56175b3a6e50 con 0x56175b475900 2023-07-21T18:52:11.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.099+0000 7f56167fc700 1 --2- 172.21.15.39:0/133200184 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x56175b475900 0x56175b477d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:11.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.099+0000 7f55f6ffd700 10 client.16938 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:52:11.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.099+0000 7f56167fc700 1 --2- 172.21.15.39:0/133200184 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x56175b475900 0x56175b477d10 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a0627aa4c152a23 server_cookie=b7576a6fbabedda1 in_seq=0 out_seq=0 2023-07-21T18:52:11.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.100+0000 7f55f6ffd700 1 -- 172.21.15.39:0/133200184 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56175b3a6e50 con 0x56175b475900 2023-07-21T18:52:11.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.100+0000 7f55f6ffd700 10 client.16938 handle_command_reply: tid=0 2023-07-21T18:52:11.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.100+0000 7f5629adfb80 1 client.16938 shutdown 2023-07-21T18:52:11.125 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.100+0000 7f5629adfb80 1 -- 172.21.15.39:0/133200184 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56175b2b86b0 msgr2=0x56175b468640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:52:11.125 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.100+0000 7f5629adfb80 1 --2- 172.21.15.39:0/133200184 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56175b2b86b0 0x56175b468640 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f56080ca7d0 tx=0x7f5608004f50 comp rx=0 tx=0).stop 2023-07-21T18:52:11.125 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.100+0000 7f5629adfb80 1 -- 172.21.15.39:0/133200184 shutdown_connections 2023-07-21T18:52:11.125 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.100+0000 7f5629adfb80 1 --2- 172.21.15.39:0/133200184 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x56175b475900 0x56175b477d10 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:11.125 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.100+0000 7f5629adfb80 1 --2- 172.21.15.39:0/133200184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56175b4622d0 0x56175b466000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:11.125 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.100+0000 7f55f6ffd700 0 client.16938 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:52:11.125 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.100+0000 7f5629adfb80 1 --2- 172.21.15.39:0/133200184 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56175b2b86b0 0x56175b468640 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:11.125 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.100+0000 7f5629adfb80 1 --2- 172.21.15.39:0/133200184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56175b45c580 0x56175b4658c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:11.126 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.100+0000 7f5629adfb80 1 -- 172.21.15.39:0/133200184 >> 172.21.15.39:0/133200184 conn(0x56175b451be0 msgr2=0x56175b455c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:52:11.126 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.100+0000 7f5629adfb80 1 -- 172.21.15.39:0/133200184 shutdown_connections 2023-07-21T18:52:11.126 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.100+0000 7f5629adfb80 1 -- 172.21.15.39:0/133200184 wait complete. 2023-07-21T18:52:11.126 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.100+0000 7f5629adfb80 20 client.16938 trim_cache size 0 max 16384 2023-07-21T18:52:11.129 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.115+0000 7f5629adfb80 1 Processor -- start 2023-07-21T18:52:11.129 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.115+0000 7f5629adfb80 1 -- start start 2023-07-21T18:52:11.129 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.115+0000 7f5629adfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56175b2b86b0 0x56175b45c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:11.129 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.115+0000 7f5629adfb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56175b45c580 0x56175b465870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:11.129 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.115+0000 7f5629adfb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56175b4622d0 0x56175b456c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:11.130 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.115+0000 7f5629adfb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x56175b457d50 con 0x56175b45c580 2023-07-21T18:52:11.130 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.115+0000 7f5629adfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56175b457b20 con 0x56175b2b86b0 2023-07-21T18:52:11.130 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.115+0000 7f5629adfb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x56175b461990 con 0x56175b4622d0 2023-07-21T18:52:11.130 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.116+0000 7f56167fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56175b45c580 0x56175b465870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:11.130 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.116+0000 7f56167fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56175b45c580 0x56175b465870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60766/0 (socket says 172.21.15.39:60766) 2023-07-21T18:52:11.130 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.116+0000 7f56167fc700 1 -- 172.21.15.39:0/889328344 learned_addr learned my addr 172.21.15.39:0/889328344 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:52:11.130 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.116+0000 7f5616ffd700 1 --2- 172.21.15.39:0/889328344 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56175b2b86b0 0x56175b45c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:11.131 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.116+0000 7f56177fe700 1 --2- 172.21.15.39:0/889328344 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56175b4622d0 0x56175b456c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:11.131 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.116+0000 7f56167fc700 1 -- 172.21.15.39:0/889328344 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56175b4622d0 msgr2=0x56175b456c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:52:11.131 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.116+0000 7f56167fc700 1 --2- 172.21.15.39:0/889328344 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56175b4622d0 0x56175b456c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:11.131 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.116+0000 7f56167fc700 1 -- 172.21.15.39:0/889328344 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56175b2b86b0 msgr2=0x56175b45c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:52:11.131 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.116+0000 7f56167fc700 1 --2- 172.21.15.39:0/889328344 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56175b2b86b0 0x56175b45c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:11.131 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.116+0000 7f56167fc700 1 -- 172.21.15.39:0/889328344 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56175b47c940 con 0x56175b45c580 2023-07-21T18:52:11.131 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.116+0000 7f5616ffd700 1 --2- 172.21.15.39:0/889328344 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56175b2b86b0 0x56175b45c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:52:11.131 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.116+0000 7f56167fc700 1 --2- 172.21.15.39:0/889328344 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56175b45c580 0x56175b465870 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f560c009990 tx=0x7f560c01a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39943536fc134542 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:52:11.132 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.116+0000 7f55fe7fc700 1 -- 172.21.15.39:0/889328344 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f560c02b040 con 0x56175b45c580 2023-07-21T18:52:11.132 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.116+0000 7f55fe7fc700 1 -- 172.21.15.39:0/889328344 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f560c00bb70 con 0x56175b45c580 2023-07-21T18:52:11.132 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.116+0000 7f55fe7fc700 1 -- 172.21.15.39:0/889328344 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f560c004800 con 0x56175b45c580 2023-07-21T18:52:11.134 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.117+0000 7f5629adfb80 1 -- 172.21.15.39:0/889328344 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56175b45c580 msgr2=0x56175b465870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:52:11.134 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.117+0000 7f5629adfb80 1 --2- 172.21.15.39:0/889328344 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56175b45c580 0x56175b465870 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f560c009990 tx=0x7f560c01a040 comp rx=0 tx=0).stop 2023-07-21T18:52:11.135 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.117+0000 7f5629adfb80 1 -- 172.21.15.39:0/889328344 shutdown_connections 2023-07-21T18:52:11.135 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.117+0000 7f5629adfb80 1 --2- 172.21.15.39:0/889328344 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56175b2b86b0 0x56175b45c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:11.135 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.117+0000 7f5629adfb80 1 --2- 172.21.15.39:0/889328344 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56175b4622d0 0x56175b456c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:11.135 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.117+0000 7f5629adfb80 1 --2- 172.21.15.39:0/889328344 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56175b45c580 0x56175b465870 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:11.135 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.117+0000 7f5629adfb80 1 -- 172.21.15.39:0/889328344 >> 172.21.15.39:0/889328344 conn(0x56175b451be0 msgr2=0x56175b465160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:52:11.135 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.117+0000 7f5629adfb80 1 -- 172.21.15.39:0/889328344 shutdown_connections 2023-07-21T18:52:11.135 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.117+0000 7f5629adfb80 1 -- 172.21.15.39:0/889328344 wait complete. 2023-07-21T18:52:11.135 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.117+0000 7f5629adfb80 1 -- start start 2023-07-21T18:52:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.117+0000 7f5629adfb80 1 Processor -- start 2023-07-21T18:52:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.118+0000 7f5629adfb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56175b2b86b0 0x56175b46abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.118+0000 7f5629adfb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56175b45c580 0x56175b46b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.118+0000 7f5629adfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56175b4622d0 0x56175b46b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.118+0000 7f5629adfb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x56175b4c8990 con 0x56175b2b86b0 2023-07-21T18:52:11.137 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.118+0000 7f5629adfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56175b4c8b00 con 0x56175b4622d0 2023-07-21T18:52:11.137 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.118+0000 7f5629adfb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x56175b4c8cc0 con 0x56175b45c580 2023-07-21T18:52:11.137 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.118+0000 7f5629adfb80 10 client.0 _ll_register_callbacks cb 0x7f5608099d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:52:11.137 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.118+0000 7f5616ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56175b2b86b0 0x56175b46abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:11.137 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.118+0000 7f5616ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56175b2b86b0 0x56175b46abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60770/0 (socket says 172.21.15.39:60770) 2023-07-21T18:52:11.137 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.118+0000 7f5616ffd700 1 -- 172.21.15.39:0/1274825336 learned_addr learned my addr 172.21.15.39:0/1274825336 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:52:11.137 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.118+0000 7f56167fc700 1 --2- 172.21.15.39:0/1274825336 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56175b45c580 0x56175b46b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:11.137 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.118+0000 7f56177fe700 1 --2- 172.21.15.39:0/1274825336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56175b4622d0 0x56175b46b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:11.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.118+0000 7f5616ffd700 1 -- 172.21.15.39:0/1274825336 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56175b45c580 msgr2=0x56175b46b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:52:11.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.118+0000 7f5616ffd700 1 --2- 172.21.15.39:0/1274825336 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56175b45c580 0x56175b46b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:11.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.118+0000 7f5616ffd700 1 -- 172.21.15.39:0/1274825336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56175b4622d0 msgr2=0x56175b46b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:52:11.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.118+0000 7f5616ffd700 1 --2- 172.21.15.39:0/1274825336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56175b4622d0 0x56175b46b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:11.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.118+0000 7f5616ffd700 1 -- 172.21.15.39:0/1274825336 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56175b1dbd80 con 0x56175b2b86b0 2023-07-21T18:52:11.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.118+0000 7f56177fe700 1 --2- 172.21.15.39:0/1274825336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56175b4622d0 0x56175b46b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:52:11.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.118+0000 7f5616ffd700 1 --2- 172.21.15.39:0/1274825336 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56175b2b86b0 0x56175b46abc0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f56080965f0 tx=0x7f560800c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6950e001d4033934 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:52:11.139 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.118+0000 7f55f6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:52:11.139 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.119+0000 7f55f6ffd700 1 -- 172.21.15.39:0/1274825336 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5608099ec0 con 0x56175b2b86b0 2023-07-21T18:52:11.139 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.119+0000 7f55f6ffd700 1 -- 172.21.15.39:0/1274825336 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f5608098e50 con 0x56175b2b86b0 2023-07-21T18:52:11.139 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.119+0000 7f5629adfb80 1 -- 172.21.15.39:0/1274825336 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f560c01e040 con 0x56175b2b86b0 2023-07-21T18:52:11.139 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.119+0000 7f55f6ffd700 1 -- 172.21.15.39:0/1274825336 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56080a2540 con 0x56175b2b86b0 2023-07-21T18:52:11.139 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.119+0000 7f5629adfb80 10 client.16944 fetch_fsmap learned FSMap version 67 2023-07-21T18:52:11.139 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.119+0000 7f5629adfb80 1 -- 172.21.15.39:0/1274825336 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) v3 -- 0x56175b2021f0 con 0x56175b2b86b0 2023-07-21T18:52:11.139 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.119+0000 7f55f6ffd700 1 -- 172.21.15.39:0/1274825336 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f560c01e040 con 0x56175b2b86b0 2023-07-21T18:52:11.140 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.119+0000 7f55f6ffd700 1 -- 172.21.15.39:0/1274825336 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 67) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f560809a350 con 0x56175b2b86b0 2023-07-21T18:52:11.140 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.119+0000 7f5629adfb80 10 client.16944 fetch_fsmap finished waiting for FSMap version 67 2023-07-21T18:52:11.140 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.119+0000 7f5629adfb80 10 client.16944 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:52:11.140 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.119+0000 7f5629adfb80 20 client.16944 populate_metadata read hostname 'smithi039' 2023-07-21T18:52:11.140 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.119+0000 7f5629adfb80 1 --2- 172.21.15.39:0/1274825336 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x56175b4c9f20 0x56175b4cc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:11.140 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.119+0000 7f5629adfb80 4 client.16944 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:52:11.140 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.119+0000 7f5629adfb80 1 -- 172.21.15.39:0/1274825336 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56175b368ff0 con 0x56175b4c9f20 2023-07-21T18:52:11.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.120+0000 7f56167fc700 1 --2- 172.21.15.39:0/1274825336 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x56175b4c9f20 0x56175b4cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:11.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.120+0000 7f56167fc700 1 --2- 172.21.15.39:0/1274825336 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x56175b4c9f20 0x56175b4cc300 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d06c4bc5aa3c09a server_cookie=695ea1c2e2233fb2 in_seq=0 out_seq=0 2023-07-21T18:52:11.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.120+0000 7f55f6ffd700 10 client.16944 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:52:11.177 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.163+0000 7f55f6ffd700 1 -- 172.21.15.39:0/1274825336 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56175b368ff0 con 0x56175b4c9f20 2023-07-21T18:52:11.177 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.163+0000 7f55f6ffd700 10 client.16944 handle_command_reply: tid=0 2023-07-21T18:52:11.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.163+0000 7f5629adfb80 1 client.16944 shutdown 2023-07-21T18:52:11.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.163+0000 7f5629adfb80 1 -- 172.21.15.39:0/1274825336 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56175b2b86b0 msgr2=0x56175b46abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:52:11.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.163+0000 7f5629adfb80 1 --2- 172.21.15.39:0/1274825336 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56175b2b86b0 0x56175b46abc0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f56080965f0 tx=0x7f560800c6e0 comp rx=0 tx=0).stop 2023-07-21T18:52:11.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.163+0000 7f5629adfb80 1 -- 172.21.15.39:0/1274825336 shutdown_connections 2023-07-21T18:52:11.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.163+0000 7f5629adfb80 1 --2- 172.21.15.39:0/1274825336 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x56175b4c9f20 0x56175b4cc300 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:11.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.163+0000 7f5629adfb80 1 --2- 172.21.15.39:0/1274825336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56175b4622d0 0x56175b46b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:11.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.163+0000 7f5629adfb80 1 --2- 172.21.15.39:0/1274825336 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56175b2b86b0 0x56175b46abc0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:11.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.163+0000 7f55f6ffd700 0 client.16944 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:52:11.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.163+0000 7f5629adfb80 1 --2- 172.21.15.39:0/1274825336 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56175b45c580 0x56175b46b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:11.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.163+0000 7f5629adfb80 1 -- 172.21.15.39:0/1274825336 >> 172.21.15.39:0/1274825336 conn(0x56175b451be0 msgr2=0x56175b462a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:52:11.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.163+0000 7f5629adfb80 1 -- 172.21.15.39:0/1274825336 shutdown_connections 2023-07-21T18:52:11.180 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.163+0000 7f5629adfb80 1 -- 172.21.15.39:0/1274825336 wait complete. 2023-07-21T18:52:11.180 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.164+0000 7f5629adfb80 20 client.16944 trim_cache size 0 max 16384 2023-07-21T18:52:11.180 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:52:11.180 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:52:11.180 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:52:11.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:52:11.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.164+0000 7f561d959700 1 -- 172.21.15.39:0/3882815888 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f56000a12c0 msgr2=0x7f56000a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:52:11.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.164+0000 7f561d959700 1 --2- 172.21.15.39:0/3882815888 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f56000a12c0 0x7f56000a3770 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5618100500 tx=0x7f560c005fd0 comp rx=0 tx=0).stop 2023-07-21T18:52:11.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.164+0000 7f561d959700 1 -- 172.21.15.39:0/3882815888 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5618100cd0 msgr2=0x7f56180fece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:52:11.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.164+0000 7f561d959700 1 --2- 172.21.15.39:0/3882815888 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5618100cd0 0x7f56180fece0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f560800e9c0 tx=0x7f560800ecd0 comp rx=0 tx=0).stop 2023-07-21T18:52:11.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.164+0000 7f561d959700 1 -- 172.21.15.39:0/3882815888 shutdown_connections 2023-07-21T18:52:11.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.164+0000 7f561d959700 1 --2- 172.21.15.39:0/3882815888 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f56000a12c0 0x7f56000a3770 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:11.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.164+0000 7f561d959700 1 --2- 172.21.15.39:0/3882815888 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f561810a1f0 0x7f56181a3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:11.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.164+0000 7f561d959700 1 --2- 172.21.15.39:0/3882815888 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5618100cd0 0x7f56180fece0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:11.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.164+0000 7f561d959700 1 --2- 172.21.15.39:0/3882815888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56181015e0 0x7f56180ff220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:11.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.164+0000 7f561d959700 1 -- 172.21.15.39:0/3882815888 >> 172.21.15.39:0/3882815888 conn(0x7f5618075980 msgr2=0x7f56180fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:52:11.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.164+0000 7f561d959700 1 -- 172.21.15.39:0/3882815888 shutdown_connections 2023-07-21T18:52:11.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:11.164+0000 7f561d959700 1 -- 172.21.15.39:0/3882815888 wait complete. 2023-07-21T18:52:11.287 INFO:tasks.workunit.client.0.smithi039.stdout: 1 612586 5.62 MB/sec execute 444 sec latency 2963.990 ms 2023-07-21T18:52:12.287 INFO:tasks.workunit.client.0.smithi039.stdout: 1 613807 5.62 MB/sec execute 445 sec latency 3395.852 ms 2023-07-21T18:52:12.427 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:12 smithi039 ceph-mon[96489]: pgmap v326: 97 pgs: 97 active+clean; 246 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 1.3 MiB/s wr, 173 op/s 2023-07-21T18:52:12.427 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:12 smithi039 ceph-mon[96489]: osdmap e93: 12 total, 12 up, 12 in 2023-07-21T18:52:12.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:12 smithi193 ceph-mon[98204]: pgmap v326: 97 pgs: 97 active+clean; 246 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 1.3 MiB/s wr, 173 op/s 2023-07-21T18:52:12.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:12 smithi193 ceph-mon[98204]: osdmap e93: 12 total, 12 up, 12 in 2023-07-21T18:52:12.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:12 smithi155 ceph-mon[97934]: pgmap v326: 97 pgs: 97 active+clean; 246 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 1.3 MiB/s wr, 173 op/s 2023-07-21T18:52:12.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:12 smithi155 ceph-mon[97934]: osdmap e93: 12 total, 12 up, 12 in 2023-07-21T18:52:13.287 INFO:tasks.workunit.client.0.smithi039.stdout: 1 615115 5.62 MB/sec execute 446 sec latency 16.178 ms 2023-07-21T18:52:14.287 INFO:tasks.workunit.client.0.smithi039.stdout: 1 616129 5.62 MB/sec execute 447 sec latency 186.542 ms 2023-07-21T18:52:14.427 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:14 smithi039 ceph-mon[96489]: pgmap v328: 97 pgs: 97 active+clean; 246 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 930 KiB/s rd, 1.6 MiB/s wr, 205 op/s 2023-07-21T18:52:14.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:14 smithi193 ceph-mon[98204]: pgmap v328: 97 pgs: 97 active+clean; 246 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 930 KiB/s rd, 1.6 MiB/s wr, 205 op/s 2023-07-21T18:52:14.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:14 smithi155 ceph-mon[97934]: pgmap v328: 97 pgs: 97 active+clean; 246 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 930 KiB/s rd, 1.6 MiB/s wr, 205 op/s 2023-07-21T18:52:15.287 INFO:tasks.workunit.client.0.smithi039.stdout: 1 616129 5.61 MB/sec execute 448 sec latency 1186.672 ms 2023-07-21T18:52:15.427 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:15 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:52:15.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:15 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:52:15.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:15 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:52:16.287 INFO:tasks.workunit.client.0.smithi039.stdout: 1 616129 5.60 MB/sec execute 449 sec latency 2186.794 ms 2023-07-21T18:52:16.427 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:16 smithi039 ceph-mon[96489]: pgmap v329: 97 pgs: 97 active+clean; 246 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 1.3 MiB/s wr, 173 op/s 2023-07-21T18:52:16.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:16 smithi193 ceph-mon[98204]: pgmap v329: 97 pgs: 97 active+clean; 246 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 1.3 MiB/s wr, 173 op/s 2023-07-21T18:52:16.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:16 smithi155 ceph-mon[97934]: pgmap v329: 97 pgs: 97 active+clean; 246 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 1.3 MiB/s wr, 173 op/s 2023-07-21T18:52:17.288 INFO:tasks.workunit.client.0.smithi039.stdout: 1 617351 5.60 MB/sec execute 450 sec latency 2622.288 ms 2023-07-21T18:52:18.287 INFO:tasks.workunit.client.0.smithi039.stdout: 1 618672 5.60 MB/sec execute 451 sec latency 14.526 ms 2023-07-21T18:52:18.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:18 smithi193 ceph-mon[98204]: pgmap v330: 97 pgs: 97 active+clean; 247 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 1.5 MiB/s wr, 149 op/s 2023-07-21T18:52:18.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:18 smithi155 ceph-mon[97934]: pgmap v330: 97 pgs: 97 active+clean; 247 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 1.5 MiB/s wr, 149 op/s 2023-07-21T18:52:18.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:18 smithi039 ceph-mon[96489]: pgmap v330: 97 pgs: 97 active+clean; 247 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 1.5 MiB/s wr, 149 op/s 2023-07-21T18:52:19.287 INFO:tasks.workunit.client.0.smithi039.stdout: 1 619675 5.60 MB/sec execute 452 sec latency 208.574 ms 2023-07-21T18:52:20.288 INFO:tasks.workunit.client.0.smithi039.stdout: 1 619675 5.59 MB/sec execute 453 sec latency 1208.643 ms 2023-07-21T18:52:20.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:20 smithi193 ceph-mon[98204]: pgmap v331: 97 pgs: 97 active+clean; 247 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 1.5 MiB/s wr, 149 op/s 2023-07-21T18:52:20.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:20 smithi155 ceph-mon[97934]: pgmap v331: 97 pgs: 97 active+clean; 247 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 1.5 MiB/s wr, 149 op/s 2023-07-21T18:52:20.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:20 smithi039 ceph-mon[96489]: pgmap v331: 97 pgs: 97 active+clean; 247 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 1.5 MiB/s wr, 149 op/s 2023-07-21T18:52:21.288 INFO:tasks.workunit.client.0.smithi039.stdout: 1 619675 5.58 MB/sec execute 454 sec latency 2208.734 ms 2023-07-21T18:52:22.288 INFO:tasks.workunit.client.0.smithi039.stdout: 1 620881 5.58 MB/sec execute 455 sec latency 2648.269 ms 2023-07-21T18:52:22.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:22 smithi193 ceph-mon[98204]: pgmap v332: 97 pgs: 97 active+clean; 250 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 1.5 MiB/s wr, 150 op/s 2023-07-21T18:52:22.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:22 smithi155 ceph-mon[97934]: pgmap v332: 97 pgs: 97 active+clean; 250 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 1.5 MiB/s wr, 150 op/s 2023-07-21T18:52:22.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:22 smithi039 ceph-mon[96489]: pgmap v332: 97 pgs: 97 active+clean; 250 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 1.5 MiB/s wr, 150 op/s 2023-07-21T18:52:23.287 INFO:tasks.workunit.client.0.smithi039.stdout: 1 622189 5.58 MB/sec execute 456 sec latency 14.372 ms 2023-07-21T18:52:24.288 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.275+0000 7f8da93d9700 1 -- 172.21.15.39:0/4293637928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8da40fe260 msgr2=0x7f8da40fe630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:52:24.288 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.275+0000 7f8da93d9700 1 --2- 172.21.15.39:0/4293637928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8da40fe260 0x7f8da40fe630 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f8d980099c0 tx=0x7f8d98009cd0 comp rx=0 tx=0).stop 2023-07-21T18:52:24.288 INFO:tasks.workunit.client.0.smithi039.stdout: 1 623219 5.58 MB/sec execute 457 sec latency 203.120 ms 2023-07-21T18:52:24.290 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.275+0000 7f8da93d9700 1 -- 172.21.15.39:0/4293637928 shutdown_connections 2023-07-21T18:52:24.290 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.275+0000 7f8da93d9700 1 --2- 172.21.15.39:0/4293637928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8da4104db0 0x7f8da410c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:24.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.275+0000 7f8da93d9700 1 --2- 172.21.15.39:0/4293637928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8da40fe260 0x7f8da40fe630 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:24.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.275+0000 7f8da93d9700 1 --2- 172.21.15.39:0/4293637928 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8da40fec00 0x7f8da41046d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:24.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.275+0000 7f8da93d9700 1 -- 172.21.15.39:0/4293637928 >> 172.21.15.39:0/4293637928 conn(0x7f8da4075f90 msgr2=0x7f8da4076390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:52:24.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.275+0000 7f8da93d9700 1 -- 172.21.15.39:0/4293637928 shutdown_connections 2023-07-21T18:52:24.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.275+0000 7f8da93d9700 1 -- 172.21.15.39:0/4293637928 wait complete. 2023-07-21T18:52:24.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.276+0000 7f8da93d9700 1 Processor -- start 2023-07-21T18:52:24.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.276+0000 7f8da93d9700 1 -- start start 2023-07-21T18:52:24.292 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.276+0000 7f8da93d9700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8da40fe260 0x7f8da419bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:24.292 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.276+0000 7f8da93d9700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8da40fec00 0x7f8da419c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:24.292 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.276+0000 7f8da93d9700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8da4104db0 0x7f8da4195aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:24.292 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.276+0000 7f8da93d9700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da4196110 con 0x7f8da4104db0 2023-07-21T18:52:24.292 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.276+0000 7f8da93d9700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da4196280 con 0x7f8da40fe260 2023-07-21T18:52:24.292 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.276+0000 7f8da93d9700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da4196440 con 0x7f8da40fec00 2023-07-21T18:52:24.292 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.276+0000 7f8da37fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8da4104db0 0x7f8da4195aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:24.292 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.276+0000 7f8da37fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8da4104db0 0x7f8da4195aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60788/0 (socket says 172.21.15.39:60788) 2023-07-21T18:52:24.293 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.276+0000 7f8da37fe700 1 -- 172.21.15.39:0/1425062277 learned_addr learned my addr 172.21.15.39:0/1425062277 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:52:24.293 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.276+0000 7f8da27fc700 1 --2- 172.21.15.39:0/1425062277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8da40fec00 0x7f8da419c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:24.293 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.276+0000 7f8da2ffd700 1 --2- 172.21.15.39:0/1425062277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8da40fe260 0x7f8da419bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:24.293 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.276+0000 7f8da37fe700 1 -- 172.21.15.39:0/1425062277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8da40fec00 msgr2=0x7f8da419c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:52:24.293 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.276+0000 7f8da37fe700 1 --2- 172.21.15.39:0/1425062277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8da40fec00 0x7f8da419c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:24.293 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.276+0000 7f8da37fe700 1 -- 172.21.15.39:0/1425062277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8da40fe260 msgr2=0x7f8da419bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:52:24.293 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.276+0000 7f8da37fe700 1 --2- 172.21.15.39:0/1425062277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8da40fe260 0x7f8da419bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:24.294 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.276+0000 7f8da37fe700 1 -- 172.21.15.39:0/1425062277 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d980095f0 con 0x7f8da4104db0 2023-07-21T18:52:24.294 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.276+0000 7f8da2ffd700 1 --2- 172.21.15.39:0/1425062277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8da40fe260 0x7f8da419bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:52:24.294 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.277+0000 7f8da37fe700 1 --2- 172.21.15.39:0/1425062277 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8da4104db0 0x7f8da4195aa0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c00ef30 tx=0x7f8d9c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:52:24.294 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.277+0000 7f8d8bfff700 1 -- 172.21.15.39:0/1425062277 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d9c018070 con 0x7f8da4104db0 2023-07-21T18:52:24.294 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.277+0000 7f8da93d9700 1 -- 172.21.15.39:0/1425062277 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8da41966c0 con 0x7f8da4104db0 2023-07-21T18:52:24.294 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.277+0000 7f8d8bfff700 1 -- 172.21.15.39:0/1425062277 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d9c00f040 con 0x7f8da4104db0 2023-07-21T18:52:24.294 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.277+0000 7f8d8bfff700 1 -- 172.21.15.39:0/1425062277 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d9c00cd00 con 0x7f8da4104db0 2023-07-21T18:52:24.294 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.277+0000 7f8da93d9700 1 -- 172.21.15.39:0/1425062277 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8da41a2c30 con 0x7f8da4104db0 2023-07-21T18:52:24.299 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.279+0000 7f8d8bfff700 1 -- 172.21.15.39:0/1425062277 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f8d9c003a00 con 0x7f8da4104db0 2023-07-21T18:52:24.299 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.279+0000 7f8d8bfff700 1 --2- 172.21.15.39:0/1425062277 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f8d8c0a12c0 0x7f8d8c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:24.299 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.280+0000 7f8da2ffd700 1 --2- 172.21.15.39:0/1425062277 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f8d8c0a12c0 0x7f8d8c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:24.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.280+0000 7f8d8bfff700 1 -- 172.21.15.39:0/1425062277 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f8d9c0cc790 con 0x7f8da4104db0 2023-07-21T18:52:24.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.280+0000 7f8da2ffd700 1 --2- 172.21.15.39:0/1425062277 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f8d8c0a12c0 0x7f8d8c0a3770 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8d98009990 tx=0x7f8d9801a040 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:52:24.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.280+0000 7f8db555fb80 1 Processor -- start 2023-07-21T18:52:24.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.280+0000 7f8db555fb80 1 -- start start 2023-07-21T18:52:24.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.281+0000 7f8db555fb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5561825716b0 0x55618271ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:24.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.281+0000 7f8db555fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55618271b2d0 0x556182715040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:24.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.281+0000 7f8db555fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556182715580 0x5561827159d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:24.301 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.281+0000 7f8db555fb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55618261ac50 con 0x5561825716b0 2023-07-21T18:52:24.301 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.281+0000 7f8db555fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55618266d9e0 con 0x556182715580 2023-07-21T18:52:24.301 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.281+0000 7f8db555fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x556182666e70 con 0x55618271b2d0 2023-07-21T18:52:24.301 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.281+0000 7f8da2ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5561825716b0 0x55618271ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:24.301 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.281+0000 7f8da2ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5561825716b0 0x55618271ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60792/0 (socket says 172.21.15.39:60792) 2023-07-21T18:52:24.301 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.281+0000 7f8da2ffd700 1 -- 172.21.15.39:0/2934676106 learned_addr learned my addr 172.21.15.39:0/2934676106 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:52:24.301 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.281+0000 7f8da37fe700 1 --2- 172.21.15.39:0/2934676106 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556182715580 0x5561827159d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:24.301 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.281+0000 7f8da27fc700 1 --2- 172.21.15.39:0/2934676106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55618271b2d0 0x556182715040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:24.302 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.281+0000 7f8da37fe700 1 -- 172.21.15.39:0/2934676106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55618271b2d0 msgr2=0x556182715040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:52:24.302 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.281+0000 7f8da37fe700 1 --2- 172.21.15.39:0/2934676106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55618271b2d0 0x556182715040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:24.302 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.281+0000 7f8da37fe700 1 -- 172.21.15.39:0/2934676106 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5561825716b0 msgr2=0x55618271ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:52:24.302 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.281+0000 7f8da37fe700 1 --2- 172.21.15.39:0/2934676106 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5561825716b0 0x55618271ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:24.302 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.281+0000 7f8da37fe700 1 -- 172.21.15.39:0/2934676106 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556182494d80 con 0x556182715580 2023-07-21T18:52:24.302 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.281+0000 7f8da2ffd700 1 --2- 172.21.15.39:0/2934676106 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5561825716b0 0x55618271ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:52:24.302 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.281+0000 7f8da37fe700 1 --2- 172.21.15.39:0/2934676106 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556182715580 0x5561827159d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c0c9ee0 tx=0x7f8d9c095d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41b87474e2ea912f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:52:24.303 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.281+0000 7f8d897fa700 1 -- 172.21.15.39:0/2934676106 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d9c00ce60 con 0x556182715580 2023-07-21T18:52:24.303 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.281+0000 7f8d897fa700 1 -- 172.21.15.39:0/2934676106 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d9c0b7e90 con 0x556182715580 2023-07-21T18:52:24.303 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.281+0000 7f8d897fa700 1 -- 172.21.15.39:0/2934676106 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d9c099b50 con 0x556182715580 2023-07-21T18:52:24.303 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.282+0000 7f8db555fb80 1 -- 172.21.15.39:0/2934676106 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556182715580 msgr2=0x5561827159d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:52:24.303 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.282+0000 7f8db555fb80 1 --2- 172.21.15.39:0/2934676106 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556182715580 0x5561827159d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c0c9ee0 tx=0x7f8d9c095d40 comp rx=0 tx=0).stop 2023-07-21T18:52:24.303 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.282+0000 7f8db555fb80 1 -- 172.21.15.39:0/2934676106 shutdown_connections 2023-07-21T18:52:24.303 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.282+0000 7f8db555fb80 1 --2- 172.21.15.39:0/2934676106 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556182715580 0x5561827159d0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:24.303 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.282+0000 7f8db555fb80 1 --2- 172.21.15.39:0/2934676106 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5561825716b0 0x55618271ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:24.304 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.282+0000 7f8db555fb80 1 --2- 172.21.15.39:0/2934676106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55618271b2d0 0x556182715040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:24.304 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.282+0000 7f8db555fb80 1 -- 172.21.15.39:0/2934676106 >> 172.21.15.39:0/2934676106 conn(0x55618270abe0 msgr2=0x55618270ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:52:24.304 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.282+0000 7f8db555fb80 1 -- 172.21.15.39:0/2934676106 shutdown_connections 2023-07-21T18:52:24.304 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.282+0000 7f8db555fb80 1 -- 172.21.15.39:0/2934676106 wait complete. 2023-07-21T18:52:24.304 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.282+0000 7f8db555fb80 1 -- start start 2023-07-21T18:52:24.304 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.283+0000 7f8db555fb80 1 Processor -- start 2023-07-21T18:52:24.305 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.283+0000 7f8db555fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5561825716b0 0x55618271e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:24.305 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.283+0000 7f8db555fb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556182715580 0x55618271ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:24.305 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.283+0000 7f8db555fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55618271b2d0 0x55618271f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:24.305 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.283+0000 7f8db555fb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x556182606a60 con 0x556182715580 2023-07-21T18:52:24.305 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.283+0000 7f8db555fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556182606760 con 0x5561825716b0 2023-07-21T18:52:24.305 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.283+0000 7f8db555fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561824ea8c0 con 0x55618271b2d0 2023-07-21T18:52:24.305 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.283+0000 7f8db555fb80 10 client.0 _ll_register_callbacks cb 0x556182604700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:52:24.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.283+0000 7f8da27fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556182715580 0x55618271ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:24.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.283+0000 7f8da27fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556182715580 0x55618271ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60800/0 (socket says 172.21.15.39:60800) 2023-07-21T18:52:24.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.283+0000 7f8da27fc700 1 -- 172.21.15.39:0/2792881820 learned_addr learned my addr 172.21.15.39:0/2792881820 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:52:24.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.283+0000 7f8da2ffd700 1 --2- 172.21.15.39:0/2792881820 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5561825716b0 0x55618271e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:24.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.283+0000 7f8da37fe700 1 --2- 172.21.15.39:0/2792881820 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55618271b2d0 0x55618271f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:24.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.283+0000 7f8da27fc700 1 -- 172.21.15.39:0/2792881820 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55618271b2d0 msgr2=0x55618271f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:52:24.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.283+0000 7f8da27fc700 1 --2- 172.21.15.39:0/2792881820 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55618271b2d0 0x55618271f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:24.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.283+0000 7f8da27fc700 1 -- 172.21.15.39:0/2792881820 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5561825716b0 msgr2=0x55618271e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:52:24.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.283+0000 7f8da27fc700 1 --2- 172.21.15.39:0/2792881820 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5561825716b0 0x55618271e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:24.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.283+0000 7f8da27fc700 1 -- 172.21.15.39:0/2792881820 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8da410d1c0 con 0x556182715580 2023-07-21T18:52:24.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.283+0000 7f8da37fe700 1 --2- 172.21.15.39:0/2792881820 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55618271b2d0 0x55618271f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:52:24.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.283+0000 7f8da27fc700 1 --2- 172.21.15.39:0/2792881820 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556182715580 0x55618271ee90 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f8d9400b980 tx=0x7f8d9400bc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d7cd0f3e501a26e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:52:24.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.283+0000 7f8d86ff5700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:52:24.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.284+0000 7f8d86ff5700 1 -- 172.21.15.39:0/2792881820 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d94004270 con 0x556182715580 2023-07-21T18:52:24.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.284+0000 7f8d86ff5700 1 -- 172.21.15.39:0/2792881820 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f8d9401b440 con 0x556182715580 2023-07-21T18:52:24.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.284+0000 7f8db555fb80 1 -- 172.21.15.39:0/2792881820 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5561825c0490 con 0x556182715580 2023-07-21T18:52:24.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.284+0000 7f8d86ff5700 1 -- 172.21.15.39:0/2792881820 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d9401b600 con 0x556182715580 2023-07-21T18:52:24.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.284+0000 7f8db555fb80 10 client.16956 fetch_fsmap learned FSMap version 68 2023-07-21T18:52:24.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.284+0000 7f8db555fb80 1 -- 172.21.15.39:0/2792881820 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) v3 -- 0x7f8da41a2c30 con 0x556182715580 2023-07-21T18:52:24.309 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.284+0000 7f8d86ff5700 1 -- 172.21.15.39:0/2792881820 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5561825c0490 con 0x556182715580 2023-07-21T18:52:24.309 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.284+0000 7f8d86ff5700 1 -- 172.21.15.39:0/2792881820 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 68) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f8d94011900 con 0x556182715580 2023-07-21T18:52:24.309 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.284+0000 7f8db555fb80 10 client.16956 fetch_fsmap finished waiting for FSMap version 68 2023-07-21T18:52:24.309 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.284+0000 7f8db555fb80 10 client.16956 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:52:24.309 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.284+0000 7f8db555fb80 20 client.16956 populate_metadata read hostname 'smithi039' 2023-07-21T18:52:24.309 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.284+0000 7f8db555fb80 1 --2- 172.21.15.39:0/2792881820 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55618272eca0 0x5561827310b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:24.309 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.284+0000 7f8db555fb80 4 client.16956 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:52:24.309 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.284+0000 7f8db555fb80 1 -- 172.21.15.39:0/2792881820 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55618265fe50 con 0x55618272eca0 2023-07-21T18:52:24.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.284+0000 7f8da2ffd700 1 --2- 172.21.15.39:0/2792881820 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55618272eca0 0x5561827310b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:24.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.285+0000 7f8da2ffd700 1 --2- 172.21.15.39:0/2792881820 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55618272eca0 0x5561827310b0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fcb10853a28d21 server_cookie=29b47f51eb5471c8 in_seq=0 out_seq=0 2023-07-21T18:52:24.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.285+0000 7f8d86ff5700 10 client.16956 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:52:24.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.285+0000 7f8d86ff5700 1 -- 172.21.15.39:0/2792881820 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55618265fe50 con 0x55618272eca0 2023-07-21T18:52:24.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.285+0000 7f8d86ff5700 10 client.16956 handle_command_reply: tid=0 2023-07-21T18:52:24.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.285+0000 7f8db555fb80 1 client.16956 shutdown 2023-07-21T18:52:24.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.285+0000 7f8db555fb80 1 -- 172.21.15.39:0/2792881820 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556182715580 msgr2=0x55618271ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:52:24.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.285+0000 7f8db555fb80 1 --2- 172.21.15.39:0/2792881820 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556182715580 0x55618271ee90 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f8d9400b980 tx=0x7f8d9400bc90 comp rx=0 tx=0).stop 2023-07-21T18:52:24.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.286+0000 7f8db555fb80 1 -- 172.21.15.39:0/2792881820 shutdown_connections 2023-07-21T18:52:24.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.286+0000 7f8db555fb80 1 --2- 172.21.15.39:0/2792881820 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55618272eca0 0x5561827310b0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:24.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.286+0000 7f8db555fb80 1 --2- 172.21.15.39:0/2792881820 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5561825716b0 0x55618271e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:24.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.286+0000 7f8db555fb80 1 --2- 172.21.15.39:0/2792881820 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55618271b2d0 0x55618271f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:24.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.286+0000 7f8d86ff5700 0 client.16956 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:52:24.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.286+0000 7f8db555fb80 1 --2- 172.21.15.39:0/2792881820 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556182715580 0x55618271ee90 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:24.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.286+0000 7f8db555fb80 1 -- 172.21.15.39:0/2792881820 >> 172.21.15.39:0/2792881820 conn(0x55618270abe0 msgr2=0x55618270f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:52:24.312 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.286+0000 7f8db555fb80 1 -- 172.21.15.39:0/2792881820 shutdown_connections 2023-07-21T18:52:24.312 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.286+0000 7f8db555fb80 1 -- 172.21.15.39:0/2792881820 wait complete. 2023-07-21T18:52:24.312 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.286+0000 7f8db555fb80 20 client.16956 trim_cache size 0 max 16384 2023-07-21T18:52:24.314 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.300+0000 7f8db555fb80 1 Processor -- start 2023-07-21T18:52:24.314 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.300+0000 7f8db555fb80 1 -- start start 2023-07-21T18:52:24.314 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.300+0000 7f8db555fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5561825716b0 0x556182715040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:24.314 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.300+0000 7f8db555fb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556182715580 0x55618271a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:24.315 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.300+0000 7f8db555fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55618271b2d0 0x556182725c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:24.315 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.300+0000 7f8db555fb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55618271aa10 con 0x556182715580 2023-07-21T18:52:24.315 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.300+0000 7f8db555fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556182711740 con 0x5561825716b0 2023-07-21T18:52:24.315 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.300+0000 7f8db555fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55618271c910 con 0x55618271b2d0 2023-07-21T18:52:24.315 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.301+0000 7f8da27fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556182715580 0x55618271a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:24.316 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.301+0000 7f8da27fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556182715580 0x55618271a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60808/0 (socket says 172.21.15.39:60808) 2023-07-21T18:52:24.316 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.301+0000 7f8da27fc700 1 -- 172.21.15.39:0/4075274506 learned_addr learned my addr 172.21.15.39:0/4075274506 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:52:24.316 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.301+0000 7f8da37fe700 1 --2- 172.21.15.39:0/4075274506 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55618271b2d0 0x556182725c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:24.316 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.301+0000 7f8da2ffd700 1 --2- 172.21.15.39:0/4075274506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5561825716b0 0x556182715040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:24.316 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.301+0000 7f8da27fc700 1 -- 172.21.15.39:0/4075274506 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55618271b2d0 msgr2=0x556182725c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:52:24.316 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.301+0000 7f8da27fc700 1 --2- 172.21.15.39:0/4075274506 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55618271b2d0 0x556182725c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:24.316 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.301+0000 7f8da27fc700 1 -- 172.21.15.39:0/4075274506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5561825716b0 msgr2=0x556182715040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:52:24.317 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.301+0000 7f8da27fc700 1 --2- 172.21.15.39:0/4075274506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5561825716b0 0x556182715040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:24.317 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.301+0000 7f8da27fc700 1 -- 172.21.15.39:0/4075274506 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55618247dc00 con 0x556182715580 2023-07-21T18:52:24.317 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.301+0000 7f8da27fc700 1 --2- 172.21.15.39:0/4075274506 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556182715580 0x55618271a240 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f8d94011c30 tx=0x7f8d94005980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6eb0b10469dbf27 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:52:24.317 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.301+0000 7f8d89ffb700 1 -- 172.21.15.39:0/4075274506 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d94003960 con 0x556182715580 2023-07-21T18:52:24.317 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.301+0000 7f8d89ffb700 1 -- 172.21.15.39:0/4075274506 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d94011900 con 0x556182715580 2023-07-21T18:52:24.317 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.301+0000 7f8d89ffb700 1 -- 172.21.15.39:0/4075274506 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d940046e0 con 0x556182715580 2023-07-21T18:52:24.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.302+0000 7f8db555fb80 1 -- 172.21.15.39:0/4075274506 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556182715580 msgr2=0x55618271a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:52:24.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.302+0000 7f8db555fb80 1 --2- 172.21.15.39:0/4075274506 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556182715580 0x55618271a240 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f8d94011c30 tx=0x7f8d94005980 comp rx=0 tx=0).stop 2023-07-21T18:52:24.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.302+0000 7f8db555fb80 1 -- 172.21.15.39:0/4075274506 shutdown_connections 2023-07-21T18:52:24.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.302+0000 7f8db555fb80 1 --2- 172.21.15.39:0/4075274506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5561825716b0 0x556182715040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:24.322 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.302+0000 7f8db555fb80 1 --2- 172.21.15.39:0/4075274506 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55618271b2d0 0x556182725c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:24.322 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.302+0000 7f8db555fb80 1 --2- 172.21.15.39:0/4075274506 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556182715580 0x55618271a240 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:24.322 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.302+0000 7f8db555fb80 1 -- 172.21.15.39:0/4075274506 >> 172.21.15.39:0/4075274506 conn(0x55618270abe0 msgr2=0x556182719b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:52:24.322 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.302+0000 7f8db555fb80 1 -- 172.21.15.39:0/4075274506 shutdown_connections 2023-07-21T18:52:24.323 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.302+0000 7f8db555fb80 1 -- 172.21.15.39:0/4075274506 wait complete. 2023-07-21T18:52:24.323 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.302+0000 7f8db555fb80 1 -- start start 2023-07-21T18:52:24.323 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.302+0000 7f8db555fb80 1 Processor -- start 2023-07-21T18:52:24.323 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.302+0000 7f8db555fb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5561825716b0 0x5561827259b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:24.324 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.302+0000 7f8db555fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556182715580 0x556182723cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:24.324 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.302+0000 7f8db555fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55618271b2d0 0x5561827241f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:24.324 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.302+0000 7f8db555fb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x556182726280 con 0x5561825716b0 2023-07-21T18:52:24.324 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.302+0000 7f8db555fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556182724730 con 0x556182715580 2023-07-21T18:52:24.325 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.302+0000 7f8db555fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561827248f0 con 0x55618271b2d0 2023-07-21T18:52:24.325 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.303+0000 7f8db555fb80 10 client.0 _ll_register_callbacks cb 0x7f8d94004460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:52:24.325 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.303+0000 7f8da2ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5561825716b0 0x5561827259b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:24.326 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.303+0000 7f8da2ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5561825716b0 0x5561827259b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60812/0 (socket says 172.21.15.39:60812) 2023-07-21T18:52:24.326 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.303+0000 7f8da2ffd700 1 -- 172.21.15.39:0/3496370162 learned_addr learned my addr 172.21.15.39:0/3496370162 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:52:24.326 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.303+0000 7f8da37fe700 1 --2- 172.21.15.39:0/3496370162 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55618271b2d0 0x5561827241f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:24.326 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.303+0000 7f8da27fc700 1 --2- 172.21.15.39:0/3496370162 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556182715580 0x556182723cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:24.327 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.303+0000 7f8da2ffd700 1 -- 172.21.15.39:0/3496370162 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55618271b2d0 msgr2=0x5561827241f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:52:24.327 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.303+0000 7f8da2ffd700 1 --2- 172.21.15.39:0/3496370162 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55618271b2d0 0x5561827241f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:24.327 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.303+0000 7f8da2ffd700 1 -- 172.21.15.39:0/3496370162 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556182715580 msgr2=0x556182723cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:52:24.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.303+0000 7f8da2ffd700 1 --2- 172.21.15.39:0/3496370162 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556182715580 0x556182723cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:24.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.303+0000 7f8da2ffd700 1 -- 172.21.15.39:0/3496370162 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8da41a2c30 con 0x5561825716b0 2023-07-21T18:52:24.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.303+0000 7f8da27fc700 1 --2- 172.21.15.39:0/3496370162 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556182715580 0x556182723cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:52:24.329 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.303+0000 7f8da2ffd700 1 --2- 172.21.15.39:0/3496370162 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5561825716b0 0x5561827259b0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f8d9800be70 tx=0x7f8d98009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=637371f4d2b0bc6e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:52:24.329 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.303+0000 7f8d86ff5700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:52:24.329 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.303+0000 7f8d86ff5700 1 -- 172.21.15.39:0/3496370162 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d9802a030 con 0x5561825716b0 2023-07-21T18:52:24.329 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.303+0000 7f8d86ff5700 1 -- 172.21.15.39:0/3496370162 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f8d98004d30 con 0x5561825716b0 2023-07-21T18:52:24.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.303+0000 7f8db555fb80 1 -- 172.21.15.39:0/3496370162 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8d9c0b7e90 con 0x5561825716b0 2023-07-21T18:52:24.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.303+0000 7f8d86ff5700 1 -- 172.21.15.39:0/3496370162 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d98034600 con 0x5561825716b0 2023-07-21T18:52:24.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.303+0000 7f8db555fb80 10 client.16962 fetch_fsmap learned FSMap version 68 2023-07-21T18:52:24.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.303+0000 7f8db555fb80 1 -- 172.21.15.39:0/3496370162 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) v3 -- 0x5561824bb1f0 con 0x5561825716b0 2023-07-21T18:52:24.331 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.304+0000 7f8d86ff5700 1 -- 172.21.15.39:0/3496370162 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d9c0b7e90 con 0x5561825716b0 2023-07-21T18:52:24.331 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.304+0000 7f8d86ff5700 1 -- 172.21.15.39:0/3496370162 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 68) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f8d9803d6d0 con 0x5561825716b0 2023-07-21T18:52:24.331 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.304+0000 7f8db555fb80 10 client.16962 fetch_fsmap finished waiting for FSMap version 68 2023-07-21T18:52:24.331 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.304+0000 7f8db555fb80 10 client.16962 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:52:24.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.304+0000 7f8db555fb80 20 client.16962 populate_metadata read hostname 'smithi039' 2023-07-21T18:52:24.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.304+0000 7f8db555fb80 1 --2- 172.21.15.39:0/3496370162 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x556182783e50 0x556182786260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:24.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.304+0000 7f8db555fb80 4 client.16962 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:52:24.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.304+0000 7f8db555fb80 1 -- 172.21.15.39:0/3496370162 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556182621ff0 con 0x556182783e50 2023-07-21T18:52:24.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.304+0000 7f8da27fc700 1 --2- 172.21.15.39:0/3496370162 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x556182783e50 0x556182786260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:24.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.304+0000 7f8d86ff5700 10 client.16962 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:52:24.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.304+0000 7f8da27fc700 1 --2- 172.21.15.39:0/3496370162 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x556182783e50 0x556182786260 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eaff39bd88566c76 server_cookie=2363e937db397824 in_seq=0 out_seq=0 2023-07-21T18:52:24.352 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.339+0000 7f8d86ff5700 1 -- 172.21.15.39:0/3496370162 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556182621ff0 con 0x556182783e50 2023-07-21T18:52:24.352 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.339+0000 7f8d86ff5700 10 client.16962 handle_command_reply: tid=0 2023-07-21T18:52:24.352 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.339+0000 7f8db555fb80 1 client.16962 shutdown 2023-07-21T18:52:24.352 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.339+0000 7f8db555fb80 1 -- 172.21.15.39:0/3496370162 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5561825716b0 msgr2=0x5561827259b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:52:24.353 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.339+0000 7f8db555fb80 1 --2- 172.21.15.39:0/3496370162 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5561825716b0 0x5561827259b0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f8d9800be70 tx=0x7f8d98009310 comp rx=0 tx=0).stop 2023-07-21T18:52:24.353 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.339+0000 7f8db555fb80 1 -- 172.21.15.39:0/3496370162 shutdown_connections 2023-07-21T18:52:24.353 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.339+0000 7f8db555fb80 1 --2- 172.21.15.39:0/3496370162 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x556182783e50 0x556182786260 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:24.353 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.339+0000 7f8db555fb80 1 --2- 172.21.15.39:0/3496370162 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55618271b2d0 0x5561827241f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:24.353 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.339+0000 7f8db555fb80 1 --2- 172.21.15.39:0/3496370162 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5561825716b0 0x5561827259b0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:24.353 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.339+0000 7f8db555fb80 1 --2- 172.21.15.39:0/3496370162 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556182715580 0x556182723cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.339+0000 7f8d86ff5700 0 client.16962 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:52:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.339+0000 7f8db555fb80 1 -- 172.21.15.39:0/3496370162 >> 172.21.15.39:0/3496370162 conn(0x55618270abe0 msgr2=0x55618271b900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:52:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.339+0000 7f8db555fb80 1 -- 172.21.15.39:0/3496370162 shutdown_connections 2023-07-21T18:52:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.339+0000 7f8db555fb80 1 -- 172.21.15.39:0/3496370162 wait complete. 2023-07-21T18:52:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.339+0000 7f8db555fb80 20 client.16962 trim_cache size 0 max 16384 2023-07-21T18:52:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:52:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:52:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:52:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:52:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.340+0000 7f8da93d9700 1 -- 172.21.15.39:0/1425062277 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f8d8c0a12c0 msgr2=0x7f8d8c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:52:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.340+0000 7f8da93d9700 1 --2- 172.21.15.39:0/1425062277 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f8d8c0a12c0 0x7f8d8c0a3770 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8d98009990 tx=0x7f8d9801a040 comp rx=0 tx=0).stop 2023-07-21T18:52:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.340+0000 7f8da93d9700 1 -- 172.21.15.39:0/1425062277 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8da4104db0 msgr2=0x7f8da4195aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:52:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.340+0000 7f8da93d9700 1 --2- 172.21.15.39:0/1425062277 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8da4104db0 0x7f8da4195aa0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c00ef30 tx=0x7f8d9c00c3a0 comp rx=0 tx=0).stop 2023-07-21T18:52:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.340+0000 7f8da93d9700 1 -- 172.21.15.39:0/1425062277 shutdown_connections 2023-07-21T18:52:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.340+0000 7f8da93d9700 1 --2- 172.21.15.39:0/1425062277 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f8d8c0a12c0 0x7f8d8c0a3770 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.340+0000 7f8da93d9700 1 --2- 172.21.15.39:0/1425062277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8da40fe260 0x7f8da419bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.340+0000 7f8da93d9700 1 --2- 172.21.15.39:0/1425062277 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8da4104db0 0x7f8da4195aa0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.340+0000 7f8da93d9700 1 --2- 172.21.15.39:0/1425062277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8da40fec00 0x7f8da419c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.340+0000 7f8da93d9700 1 -- 172.21.15.39:0/1425062277 >> 172.21.15.39:0/1425062277 conn(0x7f8da4075f90 msgr2=0x7f8da41036b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:52:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.340+0000 7f8da93d9700 1 -- 172.21.15.39:0/1425062277 shutdown_connections 2023-07-21T18:52:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:24.340+0000 7f8da93d9700 1 -- 172.21.15.39:0/1425062277 wait complete. 2023-07-21T18:52:24.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:24 smithi193 ceph-mon[98204]: pgmap v333: 97 pgs: 97 active+clean; 250 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 1.3 MiB/s wr, 133 op/s 2023-07-21T18:52:24.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:24 smithi155 ceph-mon[97934]: pgmap v333: 97 pgs: 97 active+clean; 250 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 1.3 MiB/s wr, 133 op/s 2023-07-21T18:52:24.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:24 smithi039 ceph-mon[96489]: pgmap v333: 97 pgs: 97 active+clean; 250 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 1.3 MiB/s wr, 133 op/s 2023-07-21T18:52:25.288 INFO:tasks.workunit.client.0.smithi039.stdout: 1 623219 5.57 MB/sec execute 458 sec latency 1203.197 ms 2023-07-21T18:52:26.288 INFO:tasks.workunit.client.0.smithi039.stdout: 1 623823 5.56 MB/sec execute 459 sec latency 2005.892 ms 2023-07-21T18:52:26.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:26 smithi193 ceph-mon[98204]: pgmap v334: 97 pgs: 97 active+clean; 250 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 1.3 MiB/s wr, 125 op/s 2023-07-21T18:52:26.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:26 smithi155 ceph-mon[97934]: pgmap v334: 97 pgs: 97 active+clean; 250 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 1.3 MiB/s wr, 125 op/s 2023-07-21T18:52:26.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:26 smithi039 ceph-mon[96489]: pgmap v334: 97 pgs: 97 active+clean; 250 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 1.3 MiB/s wr, 125 op/s 2023-07-21T18:52:27.288 INFO:tasks.workunit.client.0.smithi039.stdout: 1 625379 5.57 MB/sec execute 460 sec latency 19.103 ms 2023-07-21T18:52:28.288 INFO:tasks.workunit.client.0.smithi039.stdout: 1 626595 5.57 MB/sec execute 461 sec latency 19.120 ms 2023-07-21T18:52:28.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:28 smithi193 ceph-mon[98204]: pgmap v335: 97 pgs: 97 active+clean; 250 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 2.0 MiB/s wr, 192 op/s 2023-07-21T18:52:28.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:28 smithi155 ceph-mon[97934]: pgmap v335: 97 pgs: 97 active+clean; 250 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 2.0 MiB/s wr, 192 op/s 2023-07-21T18:52:28.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:28 smithi039 ceph-mon[96489]: pgmap v335: 97 pgs: 97 active+clean; 250 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 2.0 MiB/s wr, 192 op/s 2023-07-21T18:52:29.288 INFO:tasks.workunit.client.0.smithi039.stdout: 1 626768 5.56 MB/sec execute 462 sec latency 817.881 ms 2023-07-21T18:52:30.288 INFO:tasks.workunit.client.0.smithi039.stdout: 1 626768 5.55 MB/sec execute 463 sec latency 1818.012 ms 2023-07-21T18:52:30.616 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:30 smithi193 ceph-mon[98204]: pgmap v336: 97 pgs: 97 active+clean; 250 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 1.3 MiB/s wr, 126 op/s 2023-07-21T18:52:30.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:30 smithi155 ceph-mon[97934]: pgmap v336: 97 pgs: 97 active+clean; 250 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 1.3 MiB/s wr, 126 op/s 2023-07-21T18:52:30.678 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:30 smithi039 ceph-mon[96489]: pgmap v336: 97 pgs: 97 active+clean; 250 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 1.3 MiB/s wr, 126 op/s 2023-07-21T18:52:31.288 INFO:tasks.workunit.client.0.smithi039.stdout: 1 626768 5.54 MB/sec execute 464 sec latency 2818.127 ms 2023-07-21T18:52:32.288 INFO:tasks.workunit.client.0.smithi039.stdout: 1 627975 5.54 MB/sec execute 465 sec latency 3252.011 ms 2023-07-21T18:52:32.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:32 smithi193 ceph-mon[98204]: pgmap v337: 97 pgs: 97 active+clean; 253 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 1.9 MiB/s wr, 182 op/s 2023-07-21T18:52:32.630 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:32 smithi155 ceph-mon[97934]: pgmap v337: 97 pgs: 97 active+clean; 253 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 1.9 MiB/s wr, 182 op/s 2023-07-21T18:52:32.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:32 smithi039 ceph-mon[96489]: pgmap v337: 97 pgs: 97 active+clean; 253 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 1.9 MiB/s wr, 182 op/s 2023-07-21T18:52:33.288 INFO:tasks.workunit.client.0.smithi039.stdout: 1 629278 5.54 MB/sec execute 466 sec latency 15.846 ms 2023-07-21T18:52:34.288 INFO:tasks.workunit.client.0.smithi039.stdout: 1 630309 5.54 MB/sec execute 467 sec latency 213.445 ms 2023-07-21T18:52:34.611 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:34 smithi193 ceph-mon[98204]: pgmap v338: 97 pgs: 97 active+clean; 253 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 1.3 MiB/s wr, 123 op/s 2023-07-21T18:52:34.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:34 smithi155 ceph-mon[97934]: pgmap v338: 97 pgs: 97 active+clean; 253 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 1.3 MiB/s wr, 123 op/s 2023-07-21T18:52:34.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:34 smithi039 ceph-mon[96489]: pgmap v338: 97 pgs: 97 active+clean; 253 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 1.3 MiB/s wr, 123 op/s 2023-07-21T18:52:35.288 INFO:tasks.workunit.client.0.smithi039.stdout: 1 630309 5.53 MB/sec execute 468 sec latency 1213.483 ms 2023-07-21T18:52:36.289 INFO:tasks.workunit.client.0.smithi039.stdout: 1 630309 5.52 MB/sec execute 469 sec latency 2213.573 ms 2023-07-21T18:52:36.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:36 smithi193 ceph-mon[98204]: pgmap v339: 97 pgs: 97 active+clean; 253 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 1.3 MiB/s wr, 123 op/s 2023-07-21T18:52:36.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:36 smithi155 ceph-mon[97934]: pgmap v339: 97 pgs: 97 active+clean; 253 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 1.3 MiB/s wr, 123 op/s 2023-07-21T18:52:36.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:36 smithi039 ceph-mon[96489]: pgmap v339: 97 pgs: 97 active+clean; 253 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 1.3 MiB/s wr, 123 op/s 2023-07-21T18:52:37.288 INFO:tasks.workunit.client.0.smithi039.stdout: 1 631512 5.52 MB/sec execute 470 sec latency 2648.174 ms 2023-07-21T18:52:37.471 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.459+0000 7f7ba5acc700 1 -- 172.21.15.39:0/1129704786 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ba00ff3c0 msgr2=0x7f7ba010b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:52:37.472 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.459+0000 7f7ba5acc700 1 --2- 172.21.15.39:0/1129704786 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ba00ff3c0 0x7f7ba010b850 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f7b9800b0a0 tx=0x7f7b9800b3b0 comp rx=0 tx=0).stop 2023-07-21T18:52:37.472 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.459+0000 7f7ba5acc700 1 -- 172.21.15.39:0/1129704786 shutdown_connections 2023-07-21T18:52:37.472 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.459+0000 7f7ba5acc700 1 --2- 172.21.15.39:0/1129704786 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ba0104a20 0x7f7ba0104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:37.472 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.459+0000 7f7ba5acc700 1 --2- 172.21.15.39:0/1129704786 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ba00ff3c0 0x7f7ba010b850 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:37.472 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.459+0000 7f7ba5acc700 1 --2- 172.21.15.39:0/1129704786 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ba00fea10 0x7f7ba00fee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:37.472 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.459+0000 7f7ba5acc700 1 -- 172.21.15.39:0/1129704786 >> 172.21.15.39:0/1129704786 conn(0x7f7ba00fa4d0 msgr2=0x7f7ba00fc8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:52:37.473 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.459+0000 7f7ba5acc700 1 -- 172.21.15.39:0/1129704786 shutdown_connections 2023-07-21T18:52:37.473 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.459+0000 7f7ba5acc700 1 -- 172.21.15.39:0/1129704786 wait complete. 2023-07-21T18:52:37.473 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.460+0000 7f7ba5acc700 1 Processor -- start 2023-07-21T18:52:37.475 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.460+0000 7f7ba5acc700 1 -- start start 2023-07-21T18:52:37.475 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.460+0000 7f7ba5acc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ba00fea10 0x7f7ba0194ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:37.475 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.460+0000 7f7ba5acc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ba00ff3c0 0x7f7ba0195010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:37.475 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.460+0000 7f7ba5acc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ba0104a20 0x7f7ba01a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:37.475 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.460+0000 7f7ba5acc700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba0195760 con 0x7f7ba0104a20 2023-07-21T18:52:37.475 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.460+0000 7f7ba5acc700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba01a44b0 con 0x7f7ba00fea10 2023-07-21T18:52:37.476 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.460+0000 7f7ba5acc700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba01a4640 con 0x7f7ba00ff3c0 2023-07-21T18:52:37.476 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.460+0000 7f7b9ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ba0104a20 0x7f7ba01a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:37.476 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.460+0000 7f7b9f7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ba00fea10 0x7f7ba0194ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:37.476 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.460+0000 7f7b9effd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ba00ff3c0 0x7f7ba0195010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:37.476 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.460+0000 7f7b9ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ba0104a20 0x7f7ba01a3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60830/0 (socket says 172.21.15.39:60830) 2023-07-21T18:52:37.476 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.460+0000 7f7b9ffff700 1 -- 172.21.15.39:0/22233252 learned_addr learned my addr 172.21.15.39:0/22233252 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:52:37.477 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.461+0000 7f7b9f7fe700 1 -- 172.21.15.39:0/22233252 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ba00ff3c0 msgr2=0x7f7ba0195010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:52:37.477 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.461+0000 7f7b9f7fe700 1 --2- 172.21.15.39:0/22233252 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ba00ff3c0 0x7f7ba0195010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:37.477 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.461+0000 7f7b9f7fe700 1 -- 172.21.15.39:0/22233252 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ba0104a20 msgr2=0x7f7ba01a3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:52:37.477 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.461+0000 7f7b9f7fe700 1 --2- 172.21.15.39:0/22233252 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ba0104a20 0x7f7ba01a3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:37.477 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.461+0000 7f7b9f7fe700 1 -- 172.21.15.39:0/22233252 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b98009d00 con 0x7f7ba00fea10 2023-07-21T18:52:37.477 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.461+0000 7f7b9effd700 1 --2- 172.21.15.39:0/22233252 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ba00ff3c0 0x7f7ba0195010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:52:37.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.461+0000 7f7b9ffff700 1 --2- 172.21.15.39:0/22233252 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ba0104a20 0x7f7ba01a3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:52:37.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.461+0000 7f7b9f7fe700 1 --2- 172.21.15.39:0/22233252 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ba00fea10 0x7f7ba0194ad0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7b9400e9c0 tx=0x7f7b9400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:52:37.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.461+0000 7f7b9cff9700 1 -- 172.21.15.39:0/22233252 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b94004220 con 0x7f7ba00fea10 2023-07-21T18:52:37.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.461+0000 7f7ba5acc700 1 -- 172.21.15.39:0/22233252 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ba01a4920 con 0x7f7ba00fea10 2023-07-21T18:52:37.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.461+0000 7f7b9cff9700 1 -- 172.21.15.39:0/22233252 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b94004380 con 0x7f7ba00fea10 2023-07-21T18:52:37.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.461+0000 7f7b9cff9700 1 -- 172.21.15.39:0/22233252 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b94003be0 con 0x7f7ba00fea10 2023-07-21T18:52:37.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.461+0000 7f7ba5acc700 1 -- 172.21.15.39:0/22233252 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ba01a4e80 con 0x7f7ba00fea10 2023-07-21T18:52:37.483 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.463+0000 7f7b9cff9700 1 -- 172.21.15.39:0/22233252 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f7b94003d40 con 0x7f7ba00fea10 2023-07-21T18:52:37.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.464+0000 7f7b9cff9700 1 --2- 172.21.15.39:0/22233252 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f7b880a12c0 0x7f7b880a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:37.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.464+0000 7f7b9effd700 1 --2- 172.21.15.39:0/22233252 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f7b880a12c0 0x7f7b880a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:37.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.464+0000 7f7b9cff9700 1 -- 172.21.15.39:0/22233252 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f7b94014070 con 0x7f7ba00fea10 2023-07-21T18:52:37.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.465+0000 7f7b9effd700 1 --2- 172.21.15.39:0/22233252 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f7b880a12c0 0x7f7b880a3770 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7ba0068fd0 tx=0x7f7b90009260 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:52:37.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.465+0000 7f7bb1c52b80 1 Processor -- start 2023-07-21T18:52:37.485 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.465+0000 7f7bb1c52b80 1 -- start start 2023-07-21T18:52:37.485 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.465+0000 7f7bb1c52b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555dae47d6b0 0x555dae626d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:37.485 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.465+0000 7f7bb1c52b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555dae6272d0 0x555dae621040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:37.486 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.465+0000 7f7bb1c52b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555dae621580 0x555dae6219d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:37.486 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.465+0000 7f7bb1c52b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dae526c50 con 0x555dae621580 2023-07-21T18:52:37.486 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.465+0000 7f7bb1c52b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dae5799e0 con 0x555dae47d6b0 2023-07-21T18:52:37.486 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.465+0000 7f7bb1c52b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dae572e70 con 0x555dae6272d0 2023-07-21T18:52:37.487 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.465+0000 7f7b9f7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555dae47d6b0 0x555dae626d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:37.487 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.465+0000 7f7b9f7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555dae47d6b0 0x555dae626d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:45198/0 (socket says 172.21.15.39:45198) 2023-07-21T18:52:37.488 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.465+0000 7f7b9f7fe700 1 -- 172.21.15.39:0/327189340 learned_addr learned my addr 172.21.15.39:0/327189340 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:52:37.488 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.465+0000 7f7b9effd700 1 --2- 172.21.15.39:0/327189340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555dae6272d0 0x555dae621040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:37.489 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.465+0000 7f7b9ffff700 1 --2- 172.21.15.39:0/327189340 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555dae621580 0x555dae6219d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:37.489 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.466+0000 7f7b9f7fe700 1 -- 172.21.15.39:0/327189340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555dae6272d0 msgr2=0x555dae621040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:52:37.489 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.466+0000 7f7b9f7fe700 1 --2- 172.21.15.39:0/327189340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555dae6272d0 0x555dae621040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:37.490 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.466+0000 7f7b9f7fe700 1 -- 172.21.15.39:0/327189340 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555dae621580 msgr2=0x555dae6219d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:52:37.490 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.466+0000 7f7b9f7fe700 1 --2- 172.21.15.39:0/327189340 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555dae621580 0x555dae6219d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:37.490 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.466+0000 7f7b9f7fe700 1 -- 172.21.15.39:0/327189340 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555dae3a0d80 con 0x555dae47d6b0 2023-07-21T18:52:37.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.466+0000 7f7b9f7fe700 1 --2- 172.21.15.39:0/327189340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555dae47d6b0 0x555dae626d90 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7b94096aa0 tx=0x7f7b94099a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d93b27aa106e88c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:52:37.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.466+0000 7f7b85ffb700 1 -- 172.21.15.39:0/327189340 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9409e060 con 0x555dae47d6b0 2023-07-21T18:52:37.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.466+0000 7f7b85ffb700 1 -- 172.21.15.39:0/327189340 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b940a2480 con 0x555dae47d6b0 2023-07-21T18:52:37.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.466+0000 7f7b85ffb700 1 -- 172.21.15.39:0/327189340 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9409a520 con 0x555dae47d6b0 2023-07-21T18:52:37.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.467+0000 7f7bb1c52b80 1 -- 172.21.15.39:0/327189340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555dae47d6b0 msgr2=0x555dae626d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:52:37.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.467+0000 7f7bb1c52b80 1 --2- 172.21.15.39:0/327189340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555dae47d6b0 0x555dae626d90 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7b94096aa0 tx=0x7f7b94099a80 comp rx=0 tx=0).stop 2023-07-21T18:52:37.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.467+0000 7f7bb1c52b80 1 -- 172.21.15.39:0/327189340 shutdown_connections 2023-07-21T18:52:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.467+0000 7f7bb1c52b80 1 --2- 172.21.15.39:0/327189340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555dae47d6b0 0x555dae626d90 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.467+0000 7f7bb1c52b80 1 --2- 172.21.15.39:0/327189340 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555dae621580 0x555dae6219d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.467+0000 7f7bb1c52b80 1 --2- 172.21.15.39:0/327189340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555dae6272d0 0x555dae621040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.467+0000 7f7bb1c52b80 1 -- 172.21.15.39:0/327189340 >> 172.21.15.39:0/327189340 conn(0x555dae616be0 msgr2=0x555dae61bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:52:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.467+0000 7f7bb1c52b80 1 -- 172.21.15.39:0/327189340 shutdown_connections 2023-07-21T18:52:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.467+0000 7f7bb1c52b80 1 -- 172.21.15.39:0/327189340 wait complete. 2023-07-21T18:52:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.467+0000 7f7bb1c52b80 1 -- start start 2023-07-21T18:52:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.467+0000 7f7bb1c52b80 1 Processor -- start 2023-07-21T18:52:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.468+0000 7f7bb1c52b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555dae47d6b0 0x555dae62d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.468+0000 7f7bb1c52b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555dae621580 0x555dae62a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.468+0000 7f7bb1c52b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555dae6272d0 0x555dae62b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:37.496 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.468+0000 7f7bb1c52b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dae512a60 con 0x555dae621580 2023-07-21T18:52:37.496 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.468+0000 7f7bb1c52b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dae512760 con 0x555dae47d6b0 2023-07-21T18:52:37.496 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.468+0000 7f7bb1c52b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dae3f68c0 con 0x555dae6272d0 2023-07-21T18:52:37.496 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.468+0000 7f7bb1c52b80 10 client.0 _ll_register_callbacks cb 0x555dae510700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:52:37.496 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.468+0000 7f7b9effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555dae621580 0x555dae62a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:37.497 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.468+0000 7f7b9effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555dae621580 0x555dae62a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60842/0 (socket says 172.21.15.39:60842) 2023-07-21T18:52:37.497 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.468+0000 7f7b9effd700 1 -- 172.21.15.39:0/1839398928 learned_addr learned my addr 172.21.15.39:0/1839398928 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:52:37.497 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.468+0000 7f7b9f7fe700 1 --2- 172.21.15.39:0/1839398928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555dae47d6b0 0x555dae62d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:37.497 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.468+0000 7f7b9ffff700 1 --2- 172.21.15.39:0/1839398928 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555dae6272d0 0x555dae62b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:37.497 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.468+0000 7f7b9effd700 1 -- 172.21.15.39:0/1839398928 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555dae6272d0 msgr2=0x555dae62b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:52:37.497 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.468+0000 7f7b9effd700 1 --2- 172.21.15.39:0/1839398928 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555dae6272d0 0x555dae62b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:37.497 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.468+0000 7f7b9effd700 1 -- 172.21.15.39:0/1839398928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555dae47d6b0 msgr2=0x555dae62d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:52:37.498 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.468+0000 7f7b9effd700 1 --2- 172.21.15.39:0/1839398928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555dae47d6b0 0x555dae62d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:37.498 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.468+0000 7f7b9effd700 1 -- 172.21.15.39:0/1839398928 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ba01a4e80 con 0x555dae621580 2023-07-21T18:52:37.498 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.468+0000 7f7b9ffff700 1 --2- 172.21.15.39:0/1839398928 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555dae6272d0 0x555dae62b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:52:37.498 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.468+0000 7f7b9effd700 1 --2- 172.21.15.39:0/1839398928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555dae621580 0x555dae62a920 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f7b900038c0 tx=0x7f7b900047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f2d818e2c7474dc server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:52:37.498 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.468+0000 7f7b7effd700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:52:37.498 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.469+0000 7f7b7effd700 1 -- 172.21.15.39:0/1839398928 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9001fa50 con 0x555dae621580 2023-07-21T18:52:37.498 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.469+0000 7f7b7effd700 1 -- 172.21.15.39:0/1839398928 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f7b9001fbb0 con 0x555dae621580 2023-07-21T18:52:37.499 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.469+0000 7f7bb1c52b80 1 -- 172.21.15.39:0/1839398928 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555dae4cc490 con 0x555dae621580 2023-07-21T18:52:37.499 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.469+0000 7f7b7effd700 1 -- 172.21.15.39:0/1839398928 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b9001fd70 con 0x555dae621580 2023-07-21T18:52:37.499 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.469+0000 7f7bb1c52b80 10 client.16974 fetch_fsmap learned FSMap version 68 2023-07-21T18:52:37.499 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.469+0000 7f7bb1c52b80 1 -- 172.21.15.39:0/1839398928 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) v3 -- 0x555dae389c00 con 0x555dae621580 2023-07-21T18:52:37.499 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.469+0000 7f7b7effd700 1 -- 172.21.15.39:0/1839398928 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b90034080 con 0x555dae621580 2023-07-21T18:52:37.499 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.469+0000 7f7b7effd700 1 -- 172.21.15.39:0/1839398928 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 68) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f7b9002e030 con 0x555dae621580 2023-07-21T18:52:37.499 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.469+0000 7f7bb1c52b80 10 client.16974 fetch_fsmap finished waiting for FSMap version 68 2023-07-21T18:52:37.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.469+0000 7f7bb1c52b80 10 client.16974 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:52:37.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.469+0000 7f7bb1c52b80 20 client.16974 populate_metadata read hostname 'smithi039' 2023-07-21T18:52:37.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.469+0000 7f7bb1c52b80 1 --2- 172.21.15.39:0/1839398928 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x555dae63acd0 0x555dae63d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:37.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.469+0000 7f7bb1c52b80 4 client.16974 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:52:37.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.469+0000 7f7bb1c52b80 1 -- 172.21.15.39:0/1839398928 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555dae56be50 con 0x555dae63acd0 2023-07-21T18:52:37.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.470+0000 7f7b9f7fe700 1 --2- 172.21.15.39:0/1839398928 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x555dae63acd0 0x555dae63d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:37.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.470+0000 7f7b7effd700 10 client.16974 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:52:37.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.470+0000 7f7b9f7fe700 1 --2- 172.21.15.39:0/1839398928 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x555dae63acd0 0x555dae63d0e0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35f16802200461c server_cookie=ea6ed6f8fc0f5195 in_seq=0 out_seq=0 2023-07-21T18:52:37.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.471+0000 7f7b7effd700 1 -- 172.21.15.39:0/1839398928 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555dae56be50 con 0x555dae63acd0 2023-07-21T18:52:37.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.471+0000 7f7b7effd700 10 client.16974 handle_command_reply: tid=0 2023-07-21T18:52:37.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.471+0000 7f7bb1c52b80 1 client.16974 shutdown 2023-07-21T18:52:37.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.471+0000 7f7bb1c52b80 1 -- 172.21.15.39:0/1839398928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555dae621580 msgr2=0x555dae62a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:52:37.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.471+0000 7f7bb1c52b80 1 --2- 172.21.15.39:0/1839398928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555dae621580 0x555dae62a920 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f7b900038c0 tx=0x7f7b900047c0 comp rx=0 tx=0).stop 2023-07-21T18:52:37.503 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.471+0000 7f7bb1c52b80 1 -- 172.21.15.39:0/1839398928 shutdown_connections 2023-07-21T18:52:37.503 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.471+0000 7f7bb1c52b80 1 --2- 172.21.15.39:0/1839398928 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x555dae63acd0 0x555dae63d0e0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:37.503 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.471+0000 7f7bb1c52b80 1 --2- 172.21.15.39:0/1839398928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555dae47d6b0 0x555dae62d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:37.503 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.471+0000 7f7bb1c52b80 1 --2- 172.21.15.39:0/1839398928 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555dae6272d0 0x555dae62b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:37.503 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.471+0000 7f7bb1c52b80 1 --2- 172.21.15.39:0/1839398928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555dae621580 0x555dae62a920 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:37.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.471+0000 7f7b7effd700 0 client.16974 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:52:37.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.471+0000 7f7bb1c52b80 1 -- 172.21.15.39:0/1839398928 >> 172.21.15.39:0/1839398928 conn(0x555dae616be0 msgr2=0x555dae61b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:52:37.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.471+0000 7f7bb1c52b80 1 -- 172.21.15.39:0/1839398928 shutdown_connections 2023-07-21T18:52:37.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.471+0000 7f7bb1c52b80 1 -- 172.21.15.39:0/1839398928 wait complete. 2023-07-21T18:52:37.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.471+0000 7f7bb1c52b80 20 client.16974 trim_cache size 0 max 16384 2023-07-21T18:52:37.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.487+0000 7f7bb1c52b80 1 Processor -- start 2023-07-21T18:52:37.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.487+0000 7f7bb1c52b80 1 -- start start 2023-07-21T18:52:37.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.487+0000 7f7bb1c52b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555dae616be0 0x555dae621040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:37.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.487+0000 7f7bb1c52b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555dae47d6b0 0x555dae626330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:37.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.487+0000 7f7bb1c52b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555dae621580 0x555dae62d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:37.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.487+0000 7f7bb1c52b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dae61d010 con 0x555dae47d6b0 2023-07-21T18:52:37.510 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.487+0000 7f7bb1c52b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dae626a10 con 0x555dae616be0 2023-07-21T18:52:37.510 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.487+0000 7f7bb1c52b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dae6185f0 con 0x555dae621580 2023-07-21T18:52:37.510 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.487+0000 7f7b9effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555dae47d6b0 0x555dae626330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:37.510 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.487+0000 7f7b9f7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555dae616be0 0x555dae621040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:37.510 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.487+0000 7f7b9effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555dae47d6b0 0x555dae626330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60850/0 (socket says 172.21.15.39:60850) 2023-07-21T18:52:37.510 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.487+0000 7f7b9effd700 1 -- 172.21.15.39:0/3943189781 learned_addr learned my addr 172.21.15.39:0/3943189781 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:52:37.510 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.487+0000 7f7b9ffff700 1 --2- 172.21.15.39:0/3943189781 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555dae621580 0x555dae62d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:37.511 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.487+0000 7f7b9effd700 1 -- 172.21.15.39:0/3943189781 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555dae621580 msgr2=0x555dae62d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:52:37.511 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.487+0000 7f7b9effd700 1 --2- 172.21.15.39:0/3943189781 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555dae621580 0x555dae62d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:37.511 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.487+0000 7f7b9effd700 1 -- 172.21.15.39:0/3943189781 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555dae616be0 msgr2=0x555dae621040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:52:37.511 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.487+0000 7f7b9effd700 1 --2- 172.21.15.39:0/3943189781 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555dae616be0 0x555dae621040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:37.511 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.487+0000 7f7b9effd700 1 -- 172.21.15.39:0/3943189781 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555dae641d10 con 0x555dae47d6b0 2023-07-21T18:52:37.512 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.487+0000 7f7b9f7fe700 1 --2- 172.21.15.39:0/3943189781 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555dae616be0 0x555dae621040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:52:37.512 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.488+0000 7f7b9effd700 1 --2- 172.21.15.39:0/3943189781 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555dae47d6b0 0x555dae626330 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f7b900097c0 tx=0x7f7b90004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80f7fbb72d7fb7bb server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:52:37.512 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.488+0000 7f7b867fc700 1 -- 172.21.15.39:0/3943189781 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b90030840 con 0x555dae47d6b0 2023-07-21T18:52:37.512 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.488+0000 7f7b867fc700 1 -- 172.21.15.39:0/3943189781 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555dae4cc490 con 0x555dae47d6b0 2023-07-21T18:52:37.512 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.488+0000 7f7b867fc700 1 -- 172.21.15.39:0/3943189781 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b900286a0 con 0x555dae47d6b0 2023-07-21T18:52:37.513 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.489+0000 7f7bb1c52b80 1 -- 172.21.15.39:0/3943189781 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555dae47d6b0 msgr2=0x555dae626330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:52:37.513 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.489+0000 7f7bb1c52b80 1 --2- 172.21.15.39:0/3943189781 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555dae47d6b0 0x555dae626330 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f7b900097c0 tx=0x7f7b90004750 comp rx=0 tx=0).stop 2023-07-21T18:52:37.513 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.489+0000 7f7bb1c52b80 1 -- 172.21.15.39:0/3943189781 shutdown_connections 2023-07-21T18:52:37.513 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.489+0000 7f7bb1c52b80 1 --2- 172.21.15.39:0/3943189781 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555dae616be0 0x555dae621040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:37.514 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.489+0000 7f7bb1c52b80 1 --2- 172.21.15.39:0/3943189781 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555dae621580 0x555dae62d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:37.514 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.489+0000 7f7bb1c52b80 1 --2- 172.21.15.39:0/3943189781 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555dae47d6b0 0x555dae626330 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:37.514 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.489+0000 7f7bb1c52b80 1 -- 172.21.15.39:0/3943189781 >> 172.21.15.39:0/3943189781 conn(0x555dae637160 msgr2=0x555dae627900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:52:37.514 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.489+0000 7f7bb1c52b80 1 -- 172.21.15.39:0/3943189781 shutdown_connections 2023-07-21T18:52:37.514 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.489+0000 7f7bb1c52b80 1 -- 172.21.15.39:0/3943189781 wait complete. 2023-07-21T18:52:37.514 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.489+0000 7f7bb1c52b80 1 -- start start 2023-07-21T18:52:37.514 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.489+0000 7f7bb1c52b80 1 Processor -- start 2023-07-21T18:52:37.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.490+0000 7f7bb1c52b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555dae47d6b0 0x555dae631e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:37.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.490+0000 7f7bb1c52b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555dae616be0 0x555dae622cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:37.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.490+0000 7f7bb1c52b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555dae621580 0x555dae623270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:37.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.490+0000 7f7bb1c52b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dae623940 con 0x555dae47d6b0 2023-07-21T18:52:37.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.490+0000 7f7bb1c52b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dae623ab0 con 0x555dae621580 2023-07-21T18:52:37.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.490+0000 7f7bb1c52b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dae68eb90 con 0x555dae616be0 2023-07-21T18:52:37.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.490+0000 7f7bb1c52b80 10 client.0 _ll_register_callbacks cb 0x7f7b90003750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:52:37.516 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.490+0000 7f7b9f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555dae47d6b0 0x555dae631e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:37.516 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.490+0000 7f7b9f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555dae47d6b0 0x555dae631e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60854/0 (socket says 172.21.15.39:60854) 2023-07-21T18:52:37.516 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.490+0000 7f7b9f7fe700 1 -- 172.21.15.39:0/3515449992 learned_addr learned my addr 172.21.15.39:0/3515449992 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:52:37.516 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.490+0000 7f7b9ffff700 1 --2- 172.21.15.39:0/3515449992 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555dae621580 0x555dae623270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:37.516 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.490+0000 7f7b9effd700 1 --2- 172.21.15.39:0/3515449992 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555dae616be0 0x555dae622cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:37.516 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.490+0000 7f7b9f7fe700 1 -- 172.21.15.39:0/3515449992 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555dae616be0 msgr2=0x555dae622cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:52:37.517 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.490+0000 7f7b9f7fe700 1 --2- 172.21.15.39:0/3515449992 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555dae616be0 0x555dae622cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:37.517 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.490+0000 7f7b9f7fe700 1 -- 172.21.15.39:0/3515449992 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555dae621580 msgr2=0x555dae623270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:52:37.517 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.490+0000 7f7b9f7fe700 1 --2- 172.21.15.39:0/3515449992 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555dae621580 0x555dae623270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:37.517 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.490+0000 7f7b9f7fe700 1 -- 172.21.15.39:0/3515449992 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555dae3a0d80 con 0x555dae47d6b0 2023-07-21T18:52:37.517 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.490+0000 7f7b9f7fe700 1 --2- 172.21.15.39:0/3515449992 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555dae47d6b0 0x555dae631e60 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f7b94098380 tx=0x7f7b94099f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fea7a73e670f2aea server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:52:37.517 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.490+0000 7f7b7effd700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:52:37.517 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.490+0000 7f7b7effd700 1 -- 172.21.15.39:0/3515449992 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b94098490 con 0x555dae47d6b0 2023-07-21T18:52:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.490+0000 7f7b7effd700 1 -- 172.21.15.39:0/3515449992 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f7b940b8030 con 0x555dae47d6b0 2023-07-21T18:52:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.490+0000 7f7bb1c52b80 1 -- 172.21.15.39:0/3515449992 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7b940a2480 con 0x555dae47d6b0 2023-07-21T18:52:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.491+0000 7f7b7effd700 1 -- 172.21.15.39:0/3515449992 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b9405fea0 con 0x555dae47d6b0 2023-07-21T18:52:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.491+0000 7f7bb1c52b80 10 client.16980 fetch_fsmap learned FSMap version 68 2023-07-21T18:52:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.491+0000 7f7bb1c52b80 1 -- 172.21.15.39:0/3515449992 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) v3 -- 0x555dae389c00 con 0x555dae47d6b0 2023-07-21T18:52:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.491+0000 7f7b7effd700 1 -- 172.21.15.39:0/3515449992 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b940a2480 con 0x555dae47d6b0 2023-07-21T18:52:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.491+0000 7f7b7effd700 1 -- 172.21.15.39:0/3515449992 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 68) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f7b940ca020 con 0x555dae47d6b0 2023-07-21T18:52:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.491+0000 7f7bb1c52b80 10 client.16980 fetch_fsmap finished waiting for FSMap version 68 2023-07-21T18:52:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.491+0000 7f7bb1c52b80 10 client.16980 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:52:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.491+0000 7f7bb1c52b80 20 client.16980 populate_metadata read hostname 'smithi039' 2023-07-21T18:52:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.491+0000 7f7bb1c52b80 1 --2- 172.21.15.39:0/3515449992 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x555dae6272d0 0x555dae691e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.491+0000 7f7bb1c52b80 4 client.16980 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:52:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.491+0000 7f7bb1c52b80 1 -- 172.21.15.39:0/3515449992 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555dae52dff0 con 0x555dae6272d0 2023-07-21T18:52:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.491+0000 7f7b9effd700 1 --2- 172.21.15.39:0/3515449992 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x555dae6272d0 0x555dae691e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:37.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.492+0000 7f7b7effd700 10 client.16980 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:52:37.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.492+0000 7f7b9effd700 1 --2- 172.21.15.39:0/3515449992 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x555dae6272d0 0x555dae691e90 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e9391c1fa4021bf server_cookie=417844027791f854 in_seq=0 out_seq=0 2023-07-21T18:52:37.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.524+0000 7f7b7effd700 1 -- 172.21.15.39:0/3515449992 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555dae52dff0 con 0x555dae6272d0 2023-07-21T18:52:37.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.524+0000 7f7b7effd700 10 client.16980 handle_command_reply: tid=0 2023-07-21T18:52:37.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.524+0000 7f7bb1c52b80 1 client.16980 shutdown 2023-07-21T18:52:37.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.524+0000 7f7bb1c52b80 1 -- 172.21.15.39:0/3515449992 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555dae47d6b0 msgr2=0x555dae631e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:52:37.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.525+0000 7f7bb1c52b80 1 --2- 172.21.15.39:0/3515449992 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555dae47d6b0 0x555dae631e60 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f7b94098380 tx=0x7f7b94099f30 comp rx=0 tx=0).stop 2023-07-21T18:52:37.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.525+0000 7f7bb1c52b80 1 -- 172.21.15.39:0/3515449992 shutdown_connections 2023-07-21T18:52:37.538 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.525+0000 7f7bb1c52b80 1 --2- 172.21.15.39:0/3515449992 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x555dae6272d0 0x555dae691e90 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:37.538 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.525+0000 7f7bb1c52b80 1 --2- 172.21.15.39:0/3515449992 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555dae621580 0x555dae623270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:37.538 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.525+0000 7f7bb1c52b80 1 --2- 172.21.15.39:0/3515449992 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555dae47d6b0 0x555dae631e60 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:37.538 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.525+0000 7f7bb1c52b80 1 --2- 172.21.15.39:0/3515449992 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555dae616be0 0x555dae622cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:37.538 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.525+0000 7f7bb1c52b80 1 -- 172.21.15.39:0/3515449992 >> 172.21.15.39:0/3515449992 conn(0x555dae637160 msgr2=0x555dae6261a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:52:37.538 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.525+0000 7f7b7effd700 0 client.16980 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:52:37.538 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.525+0000 7f7bb1c52b80 1 -- 172.21.15.39:0/3515449992 shutdown_connections 2023-07-21T18:52:37.539 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.525+0000 7f7bb1c52b80 1 -- 172.21.15.39:0/3515449992 wait complete. 2023-07-21T18:52:37.539 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.525+0000 7f7bb1c52b80 20 client.16980 trim_cache size 0 max 16384 2023-07-21T18:52:37.539 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:52:37.539 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:52:37.539 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:52:37.540 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:52:37.540 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.525+0000 7f7ba5acc700 1 -- 172.21.15.39:0/22233252 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f7b880a12c0 msgr2=0x7f7b880a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:52:37.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.525+0000 7f7ba5acc700 1 --2- 172.21.15.39:0/22233252 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f7b880a12c0 0x7f7b880a3770 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7ba0068fd0 tx=0x7f7b90009260 comp rx=0 tx=0).stop 2023-07-21T18:52:37.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.525+0000 7f7ba5acc700 1 -- 172.21.15.39:0/22233252 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ba00fea10 msgr2=0x7f7ba0194ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:52:37.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.525+0000 7f7ba5acc700 1 --2- 172.21.15.39:0/22233252 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ba00fea10 0x7f7ba0194ad0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7b9400e9c0 tx=0x7f7b9400ecd0 comp rx=0 tx=0).stop 2023-07-21T18:52:37.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.526+0000 7f7ba5acc700 1 -- 172.21.15.39:0/22233252 shutdown_connections 2023-07-21T18:52:37.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.526+0000 7f7ba5acc700 1 --2- 172.21.15.39:0/22233252 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f7b880a12c0 0x7f7b880a3770 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:37.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.526+0000 7f7ba5acc700 1 --2- 172.21.15.39:0/22233252 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ba00fea10 0x7f7ba0194ad0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:37.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.526+0000 7f7ba5acc700 1 --2- 172.21.15.39:0/22233252 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ba0104a20 0x7f7ba01a3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:37.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.526+0000 7f7ba5acc700 1 --2- 172.21.15.39:0/22233252 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ba00ff3c0 0x7f7ba0195010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:37.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.526+0000 7f7ba5acc700 1 -- 172.21.15.39:0/22233252 >> 172.21.15.39:0/22233252 conn(0x7f7ba00fa4d0 msgr2=0x7f7ba0105890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:52:37.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.526+0000 7f7ba5acc700 1 -- 172.21.15.39:0/22233252 shutdown_connections 2023-07-21T18:52:37.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:37.526+0000 7f7ba5acc700 1 -- 172.21.15.39:0/22233252 wait complete. 2023-07-21T18:52:38.288 INFO:tasks.workunit.client.0.smithi039.stdout: 1 632760 5.53 MB/sec execute 471 sec latency 13.258 ms 2023-07-21T18:52:38.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:38 smithi193 ceph-mon[98204]: pgmap v340: 97 pgs: 97 active+clean; 256 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 1.9 MiB/s wr, 180 op/s 2023-07-21T18:52:38.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:38 smithi155 ceph-mon[97934]: pgmap v340: 97 pgs: 97 active+clean; 256 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 1.9 MiB/s wr, 180 op/s 2023-07-21T18:52:38.678 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:38 smithi039 ceph-mon[96489]: pgmap v340: 97 pgs: 97 active+clean; 256 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 1.9 MiB/s wr, 180 op/s 2023-07-21T18:52:39.288 INFO:tasks.workunit.client.0.smithi039.stdout: 1 633858 5.53 MB/sec execute 472 sec latency 135.709 ms 2023-07-21T18:52:40.289 INFO:tasks.workunit.client.0.smithi039.stdout: 1 633858 5.51 MB/sec execute 473 sec latency 1135.743 ms 2023-07-21T18:52:40.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:40 smithi193 ceph-mon[98204]: pgmap v341: 97 pgs: 97 active+clean; 256 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 1.2 MiB/s wr, 112 op/s 2023-07-21T18:52:40.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:40 smithi155 ceph-mon[97934]: pgmap v341: 97 pgs: 97 active+clean; 256 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 1.2 MiB/s wr, 112 op/s 2023-07-21T18:52:40.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:40 smithi039 ceph-mon[96489]: pgmap v341: 97 pgs: 97 active+clean; 256 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 1.2 MiB/s wr, 112 op/s 2023-07-21T18:52:41.289 INFO:tasks.workunit.client.0.smithi039.stdout: 1 633858 5.50 MB/sec execute 474 sec latency 2135.826 ms 2023-07-21T18:52:42.289 INFO:tasks.workunit.client.0.smithi039.stdout: 1 635059 5.50 MB/sec execute 475 sec latency 2568.759 ms 2023-07-21T18:52:42.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:42 smithi193 ceph-mon[98204]: pgmap v342: 97 pgs: 97 active+clean; 256 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 1.8 MiB/s wr, 176 op/s 2023-07-21T18:52:42.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:42 smithi155 ceph-mon[97934]: pgmap v342: 97 pgs: 97 active+clean; 256 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 1.8 MiB/s wr, 176 op/s 2023-07-21T18:52:42.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:42 smithi039 ceph-mon[96489]: pgmap v342: 97 pgs: 97 active+clean; 256 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 1.8 MiB/s wr, 176 op/s 2023-07-21T18:52:43.289 INFO:tasks.workunit.client.0.smithi039.stdout: 1 636308 5.51 MB/sec execute 476 sec latency 17.405 ms 2023-07-21T18:52:44.289 INFO:tasks.workunit.client.0.smithi039.stdout: 1 637402 5.51 MB/sec execute 477 sec latency 157.654 ms 2023-07-21T18:52:44.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:44 smithi193 ceph-mon[98204]: pgmap v343: 97 pgs: 97 active+clean; 256 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 1.2 MiB/s wr, 120 op/s 2023-07-21T18:52:44.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:44 smithi155 ceph-mon[97934]: pgmap v343: 97 pgs: 97 active+clean; 256 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 1.2 MiB/s wr, 120 op/s 2023-07-21T18:52:44.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:44 smithi039 ceph-mon[96489]: pgmap v343: 97 pgs: 97 active+clean; 256 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 1.2 MiB/s wr, 120 op/s 2023-07-21T18:52:45.289 INFO:tasks.workunit.client.0.smithi039.stdout: 1 637402 5.50 MB/sec execute 478 sec latency 1157.748 ms 2023-07-21T18:52:46.289 INFO:tasks.workunit.client.0.smithi039.stdout: 1 637583 5.48 MB/sec execute 479 sec latency 1949.607 ms 2023-07-21T18:52:46.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:46 smithi193 ceph-mon[98204]: pgmap v344: 97 pgs: 97 active+clean; 256 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 1.2 MiB/s wr, 120 op/s 2023-07-21T18:52:46.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:46 smithi155 ceph-mon[97934]: pgmap v344: 97 pgs: 97 active+clean; 256 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 1.2 MiB/s wr, 120 op/s 2023-07-21T18:52:46.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:46 smithi039 ceph-mon[96489]: pgmap v344: 97 pgs: 97 active+clean; 256 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 1.2 MiB/s wr, 120 op/s 2023-07-21T18:52:47.289 INFO:tasks.workunit.client.0.smithi039.stdout: 1 639466 5.50 MB/sec execute 480 sec latency 33.380 ms 2023-07-21T18:52:48.289 INFO:tasks.workunit.client.0.smithi039.stdout: 1 640691 5.50 MB/sec execute 481 sec latency 19.850 ms 2023-07-21T18:52:48.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:48 smithi193 ceph-mon[98204]: pgmap v345: 97 pgs: 97 active+clean; 259 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 877 KiB/s rd, 1.8 MiB/s wr, 177 op/s 2023-07-21T18:52:48.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:48 smithi155 ceph-mon[97934]: pgmap v345: 97 pgs: 97 active+clean; 259 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 877 KiB/s rd, 1.8 MiB/s wr, 177 op/s 2023-07-21T18:52:48.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:48 smithi039 ceph-mon[96489]: pgmap v345: 97 pgs: 97 active+clean; 259 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 877 KiB/s rd, 1.8 MiB/s wr, 177 op/s 2023-07-21T18:52:49.289 INFO:tasks.workunit.client.0.smithi039.stdout: 1 640948 5.49 MB/sec execute 482 sec latency 824.688 ms 2023-07-21T18:52:50.289 INFO:tasks.workunit.client.0.smithi039.stdout: 1 640948 5.48 MB/sec execute 483 sec latency 1824.797 ms 2023-07-21T18:52:50.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:50 smithi193 ceph-mon[98204]: pgmap v346: 97 pgs: 97 active+clean; 259 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 1.2 MiB/s wr, 120 op/s 2023-07-21T18:52:50.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:50 smithi155 ceph-mon[97934]: pgmap v346: 97 pgs: 97 active+clean; 259 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 1.2 MiB/s wr, 120 op/s 2023-07-21T18:52:50.663 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.649+0000 7f972f0f6700 1 -- 172.21.15.39:0/2625376814 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97280fea30 msgr2=0x7f97280feea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:52:50.663 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.649+0000 7f972f0f6700 1 --2- 172.21.15.39:0/2625376814 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97280fea30 0x7f97280feea0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f9718009990 tx=0x7f9718009ca0 comp rx=0 tx=0).stop 2023-07-21T18:52:50.664 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.649+0000 7f972f0f6700 1 -- 172.21.15.39:0/2625376814 shutdown_connections 2023-07-21T18:52:50.664 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.649+0000 7f972f0f6700 1 --2- 172.21.15.39:0/2625376814 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f97280ff3e0 0x7f972810b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:50.664 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.649+0000 7f972f0f6700 1 --2- 172.21.15.39:0/2625376814 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9728104a30 0x7f9728104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:50.664 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.649+0000 7f972f0f6700 1 --2- 172.21.15.39:0/2625376814 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97280fea30 0x7f97280feea0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:50.664 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.649+0000 7f972f0f6700 1 -- 172.21.15.39:0/2625376814 >> 172.21.15.39:0/2625376814 conn(0x7f9728074760 msgr2=0x7f9728074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:52:50.664 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.649+0000 7f972f0f6700 1 -- 172.21.15.39:0/2625376814 shutdown_connections 2023-07-21T18:52:50.665 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.649+0000 7f972f0f6700 1 -- 172.21.15.39:0/2625376814 wait complete. 2023-07-21T18:52:50.665 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.650+0000 7f972f0f6700 1 Processor -- start 2023-07-21T18:52:50.665 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.650+0000 7f972f0f6700 1 -- start start 2023-07-21T18:52:50.665 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.650+0000 7f972f0f6700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f97280fea30 0x7f972819bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:50.665 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.650+0000 7f972f0f6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97280ff3e0 0x7f972819c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:50.665 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.650+0000 7f972f0f6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9728104a30 0x7f9728195930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:50.666 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.650+0000 7f972f0f6700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9728195fa0 con 0x7f97280ff3e0 2023-07-21T18:52:50.666 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.650+0000 7f972f0f6700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9728196110 con 0x7f9728104a30 2023-07-21T18:52:50.666 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.650+0000 7f972f0f6700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97281962d0 con 0x7f97280fea30 2023-07-21T18:52:50.666 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.650+0000 7f971ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97280ff3e0 0x7f972819c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:50.666 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.650+0000 7f971ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97280ff3e0 0x7f972819c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60868/0 (socket says 172.21.15.39:60868) 2023-07-21T18:52:50.666 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.650+0000 7f971ffff700 1 -- 172.21.15.39:0/3967186796 learned_addr learned my addr 172.21.15.39:0/3967186796 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:52:50.667 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.651+0000 7f972ce92700 1 --2- 172.21.15.39:0/3967186796 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f97280fea30 0x7f972819bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:50.667 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.651+0000 7f972d693700 1 --2- 172.21.15.39:0/3967186796 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9728104a30 0x7f9728195930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:50.667 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.651+0000 7f971ffff700 1 -- 172.21.15.39:0/3967186796 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f97280fea30 msgr2=0x7f972819bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:52:50.667 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.651+0000 7f971ffff700 1 --2- 172.21.15.39:0/3967186796 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f97280fea30 0x7f972819bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:50.667 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.651+0000 7f971ffff700 1 -- 172.21.15.39:0/3967186796 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9728104a30 msgr2=0x7f9728195930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:52:50.667 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.651+0000 7f971ffff700 1 --2- 172.21.15.39:0/3967186796 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9728104a30 0x7f9728195930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:50.667 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.651+0000 7f971ffff700 1 -- 172.21.15.39:0/3967186796 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97180095f0 con 0x7f97280ff3e0 2023-07-21T18:52:50.668 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.651+0000 7f972d693700 1 --2- 172.21.15.39:0/3967186796 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9728104a30 0x7f9728195930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:52:50.668 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.651+0000 7f971ffff700 1 --2- 172.21.15.39:0/3967186796 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97280ff3e0 0x7f972819c010 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f97180095c0 tx=0x7f971800f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:52:50.668 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.651+0000 7f971dffb700 1 -- 172.21.15.39:0/3967186796 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f971801d070 con 0x7f97280ff3e0 2023-07-21T18:52:50.668 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.651+0000 7f971dffb700 1 -- 172.21.15.39:0/3967186796 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f971800fb50 con 0x7f97280ff3e0 2023-07-21T18:52:50.669 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.651+0000 7f972f0f6700 1 -- 172.21.15.39:0/3967186796 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9728196550 con 0x7f97280ff3e0 2023-07-21T18:52:50.669 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.651+0000 7f971dffb700 1 -- 172.21.15.39:0/3967186796 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97180173f0 con 0x7f97280ff3e0 2023-07-21T18:52:50.669 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.651+0000 7f972f0f6700 1 -- 172.21.15.39:0/3967186796 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97281a2ac0 con 0x7f97280ff3e0 2023-07-21T18:52:50.674 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.654+0000 7f971dffb700 1 -- 172.21.15.39:0/3967186796 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f971800fcc0 con 0x7f97280ff3e0 2023-07-21T18:52:50.675 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.655+0000 7f971dffb700 1 --2- 172.21.15.39:0/3967186796 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f97100a12c0 0x7f97100a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:50.675 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.655+0000 7f971dffb700 1 -- 172.21.15.39:0/3967186796 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f97180cde20 con 0x7f97280ff3e0 2023-07-21T18:52:50.675 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.655+0000 7f972ce92700 1 --2- 172.21.15.39:0/3967186796 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f97100a12c0 0x7f97100a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:50.675 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.656+0000 7f973b27cb80 1 Processor -- start 2023-07-21T18:52:50.676 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.656+0000 7f973b27cb80 1 -- start start 2023-07-21T18:52:50.676 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.656+0000 7f972ce92700 1 --2- 172.21.15.39:0/3967186796 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f97100a12c0 0x7f97100a3770 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f97280ffba0 tx=0x7f972400b410 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:52:50.676 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.656+0000 7f973b27cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562574aa76b0 0x562574c50d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:50.676 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.656+0000 7f973b27cb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562574c512d0 0x562574c4b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:50.676 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.656+0000 7f973b27cb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562574c4b580 0x562574c4b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:50.676 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.656+0000 7f973b27cb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x562574b50c50 con 0x562574c512d0 2023-07-21T18:52:50.677 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.656+0000 7f973b27cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562574ba39e0 con 0x562574aa76b0 2023-07-21T18:52:50.677 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.656+0000 7f973b27cb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x562574b9ce70 con 0x562574c4b580 2023-07-21T18:52:50.677 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.656+0000 7f971ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562574c512d0 0x562574c4b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:50.677 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.656+0000 7f971ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562574c512d0 0x562574c4b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60876/0 (socket says 172.21.15.39:60876) 2023-07-21T18:52:50.677 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.656+0000 7f971ffff700 1 -- 172.21.15.39:0/1210810906 learned_addr learned my addr 172.21.15.39:0/1210810906 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:52:50.678 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.656+0000 7f972d693700 1 --2- 172.21.15.39:0/1210810906 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562574c4b580 0x562574c4b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:50.678 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.656+0000 7f972ce92700 1 --2- 172.21.15.39:0/1210810906 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562574aa76b0 0x562574c50d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:50.678 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.656+0000 7f971ffff700 1 -- 172.21.15.39:0/1210810906 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562574c4b580 msgr2=0x562574c4b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:52:50.678 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.656+0000 7f971ffff700 1 --2- 172.21.15.39:0/1210810906 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562574c4b580 0x562574c4b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:50.678 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.656+0000 7f971ffff700 1 -- 172.21.15.39:0/1210810906 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562574aa76b0 msgr2=0x562574c50d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:52:50.678 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.656+0000 7f971ffff700 1 --2- 172.21.15.39:0/1210810906 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562574aa76b0 0x562574c50d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:50.678 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.656+0000 7f971ffff700 1 -- 172.21.15.39:0/1210810906 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625749cad80 con 0x562574c512d0 2023-07-21T18:52:50.678 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.657+0000 7f971ffff700 1 --2- 172.21.15.39:0/1210810906 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562574c512d0 0x562574c4b040 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f97180a2db0 tx=0x7f9718009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7d72e81810323d2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:52:50.679 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.657+0000 7f970effd700 1 -- 172.21.15.39:0/1210810906 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f971809ef50 con 0x562574c512d0 2023-07-21T18:52:50.679 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.657+0000 7f970effd700 1 -- 172.21.15.39:0/1210810906 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97180a32e0 con 0x562574c512d0 2023-07-21T18:52:50.679 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.657+0000 7f970effd700 1 -- 172.21.15.39:0/1210810906 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97180a2e50 con 0x562574c512d0 2023-07-21T18:52:50.679 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.657+0000 7f973b27cb80 1 -- 172.21.15.39:0/1210810906 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562574c512d0 msgr2=0x562574c4b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:52:50.679 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.657+0000 7f973b27cb80 1 --2- 172.21.15.39:0/1210810906 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562574c512d0 0x562574c4b040 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f97180a2db0 tx=0x7f9718009ca0 comp rx=0 tx=0).stop 2023-07-21T18:52:50.679 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.657+0000 7f973b27cb80 1 -- 172.21.15.39:0/1210810906 shutdown_connections 2023-07-21T18:52:50.679 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.657+0000 7f973b27cb80 1 --2- 172.21.15.39:0/1210810906 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562574aa76b0 0x562574c50d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:50.680 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.657+0000 7f973b27cb80 1 --2- 172.21.15.39:0/1210810906 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562574c4b580 0x562574c4b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:50.680 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.658+0000 7f973b27cb80 1 --2- 172.21.15.39:0/1210810906 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562574c512d0 0x562574c4b040 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:50.680 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.658+0000 7f973b27cb80 1 -- 172.21.15.39:0/1210810906 >> 172.21.15.39:0/1210810906 conn(0x562574c40be0 msgr2=0x562574c45f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:52:50.680 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.658+0000 7f973b27cb80 1 -- 172.21.15.39:0/1210810906 shutdown_connections 2023-07-21T18:52:50.680 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.658+0000 7f973b27cb80 1 -- 172.21.15.39:0/1210810906 wait complete. 2023-07-21T18:52:50.680 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.658+0000 7f973b27cb80 1 -- start start 2023-07-21T18:52:50.681 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.658+0000 7f973b27cb80 1 Processor -- start 2023-07-21T18:52:50.681 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.659+0000 7f973b27cb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562574aa76b0 0x562574c5a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:50.681 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.659+0000 7f973b27cb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562574c4b580 0x562574c5a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:50.681 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.659+0000 7f973b27cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562574c512d0 0x562574c546b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:50.681 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.659+0000 7f973b27cb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x562574b3ca60 con 0x562574c4b580 2023-07-21T18:52:50.681 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.659+0000 7f973b27cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562574b3c760 con 0x562574c512d0 2023-07-21T18:52:50.681 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.659+0000 7f973b27cb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x562574a208c0 con 0x562574aa76b0 2023-07-21T18:52:50.682 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.659+0000 7f973b27cb80 10 client.0 _ll_register_callbacks cb 0x562574b3a700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:52:50.682 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.659+0000 7f971ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562574c4b580 0x562574c5a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:50.682 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.659+0000 7f971ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562574c4b580 0x562574c5a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60882/0 (socket says 172.21.15.39:60882) 2023-07-21T18:52:50.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.659+0000 7f971ffff700 1 -- 172.21.15.39:0/4273030267 learned_addr learned my addr 172.21.15.39:0/4273030267 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:52:50.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.659+0000 7f972ce92700 1 --2- 172.21.15.39:0/4273030267 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562574aa76b0 0x562574c5a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:50.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.659+0000 7f972d693700 1 --2- 172.21.15.39:0/4273030267 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562574c512d0 0x562574c546b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:50.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.659+0000 7f971ffff700 1 -- 172.21.15.39:0/4273030267 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562574aa76b0 msgr2=0x562574c5a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:52:50.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.659+0000 7f971ffff700 1 --2- 172.21.15.39:0/4273030267 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562574aa76b0 0x562574c5a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:50.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.659+0000 7f971ffff700 1 -- 172.21.15.39:0/4273030267 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562574c512d0 msgr2=0x562574c546b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:52:50.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.659+0000 7f971ffff700 1 --2- 172.21.15.39:0/4273030267 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562574c512d0 0x562574c546b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:50.684 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.659+0000 7f971ffff700 1 -- 172.21.15.39:0/4273030267 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97281a2ac0 con 0x562574c4b580 2023-07-21T18:52:50.684 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.659+0000 7f971ffff700 1 --2- 172.21.15.39:0/4273030267 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562574c4b580 0x562574c5a930 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f9718000c00 tx=0x7f971809a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a248342bdb996d6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:52:50.684 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.659+0000 7f9707fff700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:52:50.684 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.659+0000 7f9707fff700 1 -- 172.21.15.39:0/4273030267 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f971809ef50 con 0x562574c4b580 2023-07-21T18:52:50.684 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.659+0000 7f9707fff700 1 -- 172.21.15.39:0/4273030267 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f97180992c0 con 0x562574c4b580 2023-07-21T18:52:50.685 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.659+0000 7f973b27cb80 1 -- 172.21.15.39:0/4273030267 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562574af6490 con 0x562574c4b580 2023-07-21T18:52:50.685 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.660+0000 7f9707fff700 1 -- 172.21.15.39:0/4273030267 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97180b9250 con 0x562574c4b580 2023-07-21T18:52:50.686 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.660+0000 7f973b27cb80 10 client.16992 fetch_fsmap learned FSMap version 68 2023-07-21T18:52:50.686 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.660+0000 7f973b27cb80 1 -- 172.21.15.39:0/4273030267 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) v3 -- 0x5625749b3c00 con 0x562574c4b580 2023-07-21T18:52:50.686 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.660+0000 7f9707fff700 1 -- 172.21.15.39:0/4273030267 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562574af6490 con 0x562574c4b580 2023-07-21T18:52:50.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.660+0000 7f9707fff700 1 -- 172.21.15.39:0/4273030267 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 68) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f9718099e80 con 0x562574c4b580 2023-07-21T18:52:50.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.660+0000 7f973b27cb80 10 client.16992 fetch_fsmap finished waiting for FSMap version 68 2023-07-21T18:52:50.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.660+0000 7f973b27cb80 10 client.16992 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:52:50.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.660+0000 7f973b27cb80 20 client.16992 populate_metadata read hostname 'smithi039' 2023-07-21T18:52:50.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.660+0000 7f973b27cb80 1 --2- 172.21.15.39:0/4273030267 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x562574c64370 0x562574c66750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:50.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.660+0000 7f973b27cb80 4 client.16992 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:52:50.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.660+0000 7f973b27cb80 1 -- 172.21.15.39:0/4273030267 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562574b95e50 con 0x562574c64370 2023-07-21T18:52:50.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.660+0000 7f972ce92700 1 --2- 172.21.15.39:0/4273030267 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x562574c64370 0x562574c66750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:50.689 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.661+0000 7f972ce92700 1 --2- 172.21.15.39:0/4273030267 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x562574c64370 0x562574c66750 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d78f4d4aa3fd2648 server_cookie=1cf87cc087aaf083 in_seq=0 out_seq=0 2023-07-21T18:52:50.689 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.661+0000 7f9707fff700 10 client.16992 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:52:50.689 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.661+0000 7f9707fff700 1 -- 172.21.15.39:0/4273030267 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562574b95e50 con 0x562574c64370 2023-07-21T18:52:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.661+0000 7f9707fff700 10 client.16992 handle_command_reply: tid=0 2023-07-21T18:52:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.661+0000 7f973b27cb80 1 client.16992 shutdown 2023-07-21T18:52:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.662+0000 7f973b27cb80 1 -- 172.21.15.39:0/4273030267 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562574c4b580 msgr2=0x562574c5a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:52:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.662+0000 7f973b27cb80 1 --2- 172.21.15.39:0/4273030267 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562574c4b580 0x562574c5a930 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f9718000c00 tx=0x7f971809a640 comp rx=0 tx=0).stop 2023-07-21T18:52:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.662+0000 7f973b27cb80 1 -- 172.21.15.39:0/4273030267 shutdown_connections 2023-07-21T18:52:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.662+0000 7f973b27cb80 1 --2- 172.21.15.39:0/4273030267 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x562574c64370 0x562574c66750 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.662+0000 7f973b27cb80 1 --2- 172.21.15.39:0/4273030267 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562574c512d0 0x562574c546b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.662+0000 7f9707fff700 0 client.16992 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:52:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.662+0000 7f973b27cb80 1 --2- 172.21.15.39:0/4273030267 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562574aa76b0 0x562574c5a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.662+0000 7f973b27cb80 1 --2- 172.21.15.39:0/4273030267 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562574c4b580 0x562574c5a930 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.662+0000 7f973b27cb80 1 -- 172.21.15.39:0/4273030267 >> 172.21.15.39:0/4273030267 conn(0x562574c40be0 msgr2=0x562574c454e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:52:50.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.662+0000 7f973b27cb80 1 -- 172.21.15.39:0/4273030267 shutdown_connections 2023-07-21T18:52:50.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.662+0000 7f973b27cb80 1 -- 172.21.15.39:0/4273030267 wait complete. 2023-07-21T18:52:50.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.662+0000 7f973b27cb80 20 client.16992 trim_cache size 0 max 16384 2023-07-21T18:52:50.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:50 smithi039 ceph-mon[96489]: pgmap v346: 97 pgs: 97 active+clean; 259 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 1.2 MiB/s wr, 120 op/s 2023-07-21T18:52:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.677+0000 7f973b27cb80 1 Processor -- start 2023-07-21T18:52:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.677+0000 7f973b27cb80 1 -- start start 2023-07-21T18:52:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.677+0000 7f973b27cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562574aa76b0 0x562574c502e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:50.698 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.677+0000 7f973b27cb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562574c4b580 0x562574c45c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:50.698 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.677+0000 7f973b27cb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562574c512d0 0x562574c59f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:50.698 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.677+0000 7f973b27cb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x562574c54b10 con 0x562574c4b580 2023-07-21T18:52:50.698 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.677+0000 7f973b27cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562574c42d00 con 0x562574aa76b0 2023-07-21T18:52:50.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.677+0000 7f973b27cb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x562574c55c20 con 0x562574c512d0 2023-07-21T18:52:50.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.677+0000 7f971ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562574c4b580 0x562574c45c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:50.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.677+0000 7f971ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562574c4b580 0x562574c45c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60890/0 (socket says 172.21.15.39:60890) 2023-07-21T18:52:50.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.677+0000 7f971ffff700 1 -- 172.21.15.39:0/2585351358 learned_addr learned my addr 172.21.15.39:0/2585351358 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:52:50.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.677+0000 7f972d693700 1 --2- 172.21.15.39:0/2585351358 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562574c512d0 0x562574c59f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:50.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.677+0000 7f972ce92700 1 --2- 172.21.15.39:0/2585351358 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562574aa76b0 0x562574c502e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:50.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.677+0000 7f971ffff700 1 -- 172.21.15.39:0/2585351358 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562574c512d0 msgr2=0x562574c59f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:52:50.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.677+0000 7f971ffff700 1 --2- 172.21.15.39:0/2585351358 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562574c512d0 0x562574c59f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:50.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.677+0000 7f971ffff700 1 -- 172.21.15.39:0/2585351358 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562574aa76b0 msgr2=0x562574c502e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:52:50.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.677+0000 7f971ffff700 1 --2- 172.21.15.39:0/2585351358 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562574aa76b0 0x562574c502e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:50.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.677+0000 7f971ffff700 1 -- 172.21.15.39:0/2585351358 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625749f11f0 con 0x562574c4b580 2023-07-21T18:52:50.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.678+0000 7f971ffff700 1 --2- 172.21.15.39:0/2585351358 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562574c4b580 0x562574c45c60 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f97180cba90 tx=0x7f971809b0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf35faf9e50b4b97 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:52:50.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.678+0000 7f970f7fe700 1 -- 172.21.15.39:0/2585351358 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f971809ef50 con 0x562574c4b580 2023-07-21T18:52:50.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.678+0000 7f970f7fe700 1 -- 172.21.15.39:0/2585351358 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97180b9250 con 0x562574c4b580 2023-07-21T18:52:50.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.678+0000 7f970f7fe700 1 -- 172.21.15.39:0/2585351358 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97180aecb0 con 0x562574c4b580 2023-07-21T18:52:50.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.678+0000 7f973b27cb80 1 -- 172.21.15.39:0/2585351358 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562574c4b580 msgr2=0x562574c45c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:52:50.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.678+0000 7f973b27cb80 1 --2- 172.21.15.39:0/2585351358 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562574c4b580 0x562574c45c60 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f97180cba90 tx=0x7f971809b0a0 comp rx=0 tx=0).stop 2023-07-21T18:52:50.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.678+0000 7f973b27cb80 1 -- 172.21.15.39:0/2585351358 shutdown_connections 2023-07-21T18:52:50.703 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.678+0000 7f973b27cb80 1 --2- 172.21.15.39:0/2585351358 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562574aa76b0 0x562574c502e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:50.703 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.678+0000 7f973b27cb80 1 --2- 172.21.15.39:0/2585351358 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562574c512d0 0x562574c59f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:50.703 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.678+0000 7f973b27cb80 1 --2- 172.21.15.39:0/2585351358 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562574c4b580 0x562574c45c60 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:50.703 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.678+0000 7f973b27cb80 1 -- 172.21.15.39:0/2585351358 >> 172.21.15.39:0/2585351358 conn(0x562574c40be0 msgr2=0x562574c4b950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:52:50.703 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.679+0000 7f973b27cb80 1 -- 172.21.15.39:0/2585351358 shutdown_connections 2023-07-21T18:52:50.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.679+0000 7f973b27cb80 1 -- 172.21.15.39:0/2585351358 wait complete. 2023-07-21T18:52:50.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.679+0000 7f973b27cb80 1 -- start start 2023-07-21T18:52:50.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.679+0000 7f973b27cb80 1 Processor -- start 2023-07-21T18:52:50.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.680+0000 7f973b27cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562574aa76b0 0x562574c57f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:50.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.680+0000 7f973b27cb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562574c4b580 0x562574c58470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:50.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.680+0000 7f973b27cb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562574c512d0 0x562574c589b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:50.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.680+0000 7f973b27cb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x562574cb72c0 con 0x562574c512d0 2023-07-21T18:52:50.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.680+0000 7f973b27cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562574cb7430 con 0x562574aa76b0 2023-07-21T18:52:50.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.680+0000 7f973b27cb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x562574cb75f0 con 0x562574c4b580 2023-07-21T18:52:50.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.680+0000 7f973b27cb80 10 client.0 _ll_register_callbacks cb 0x7f9720007a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:52:50.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.680+0000 7f972d693700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562574c512d0 0x562574c589b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.680+0000 7f972d693700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562574c512d0 0x562574c589b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60898/0 (socket says 172.21.15.39:60898) 2023-07-21T18:52:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.680+0000 7f972d693700 1 -- 172.21.15.39:0/1105689643 learned_addr learned my addr 172.21.15.39:0/1105689643 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:52:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.680+0000 7f972ce92700 1 --2- 172.21.15.39:0/1105689643 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562574aa76b0 0x562574c57f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.680+0000 7f971ffff700 1 --2- 172.21.15.39:0/1105689643 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562574c4b580 0x562574c58470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.680+0000 7f972d693700 1 -- 172.21.15.39:0/1105689643 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562574c4b580 msgr2=0x562574c58470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:52:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.680+0000 7f972d693700 1 --2- 172.21.15.39:0/1105689643 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562574c4b580 0x562574c58470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.680+0000 7f972d693700 1 -- 172.21.15.39:0/1105689643 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562574aa76b0 msgr2=0x562574c57f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:52:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.680+0000 7f972d693700 1 --2- 172.21.15.39:0/1105689643 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562574aa76b0 0x562574c57f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:50.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.680+0000 7f972d693700 1 -- 172.21.15.39:0/1105689643 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5625749b3c00 con 0x562574c512d0 2023-07-21T18:52:50.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.680+0000 7f972d693700 1 --2- 172.21.15.39:0/1105689643 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562574c512d0 0x562574c589b0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f9720015da0 tx=0x7f97200135b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fab30bd016d0ef10 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:52:50.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.680+0000 7f9707fff700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:52:50.710 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.680+0000 7f9707fff700 1 -- 172.21.15.39:0/1105689643 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9720007e80 con 0x562574c512d0 2023-07-21T18:52:50.710 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.680+0000 7f9707fff700 1 -- 172.21.15.39:0/1105689643 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f9720013a90 con 0x562574c512d0 2023-07-21T18:52:50.710 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.680+0000 7f973b27cb80 1 -- 172.21.15.39:0/1105689643 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f97180a32e0 con 0x562574c512d0 2023-07-21T18:52:50.710 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.681+0000 7f9707fff700 1 -- 172.21.15.39:0/1105689643 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9720013c90 con 0x562574c512d0 2023-07-21T18:52:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.681+0000 7f973b27cb80 10 client.16998 fetch_fsmap learned FSMap version 68 2023-07-21T18:52:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.681+0000 7f973b27cb80 1 -- 172.21.15.39:0/1105689643 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) v3 -- 0x5625749c9250 con 0x562574c512d0 2023-07-21T18:52:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.681+0000 7f9707fff700 1 -- 172.21.15.39:0/1105689643 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97180a32e0 con 0x562574c512d0 2023-07-21T18:52:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.681+0000 7f9707fff700 1 -- 172.21.15.39:0/1105689643 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 68) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f9720021430 con 0x562574c512d0 2023-07-21T18:52:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.681+0000 7f973b27cb80 10 client.16998 fetch_fsmap finished waiting for FSMap version 68 2023-07-21T18:52:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.681+0000 7f973b27cb80 10 client.16998 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:52:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.681+0000 7f973b27cb80 20 client.16998 populate_metadata read hostname 'smithi039' 2023-07-21T18:52:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.681+0000 7f973b27cb80 1 --2- 172.21.15.39:0/1105689643 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x562574cb8de0 0x562574cbb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:52:50.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.681+0000 7f973b27cb80 4 client.16998 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:52:50.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.681+0000 7f973b27cb80 1 -- 172.21.15.39:0/1105689643 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562574b57ff0 con 0x562574cb8de0 2023-07-21T18:52:50.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.681+0000 7f972ce92700 1 --2- 172.21.15.39:0/1105689643 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x562574cb8de0 0x562574cbb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:52:50.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.682+0000 7f9707fff700 10 client.16998 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:52:50.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.682+0000 7f972ce92700 1 --2- 172.21.15.39:0/1105689643 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x562574cb8de0 0x562574cbb1f0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aedd6a8cd989b451 server_cookie=e64e5b842b7ebe87 in_seq=0 out_seq=0 2023-07-21T18:52:50.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.722+0000 7f9707fff700 1 -- 172.21.15.39:0/1105689643 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562574b57ff0 con 0x562574cb8de0 2023-07-21T18:52:50.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.722+0000 7f9707fff700 10 client.16998 handle_command_reply: tid=0 2023-07-21T18:52:50.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.723+0000 7f973b27cb80 1 client.16998 shutdown 2023-07-21T18:52:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.723+0000 7f973b27cb80 1 -- 172.21.15.39:0/1105689643 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562574c512d0 msgr2=0x562574c589b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:52:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.723+0000 7f973b27cb80 1 --2- 172.21.15.39:0/1105689643 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562574c512d0 0x562574c589b0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f9720015da0 tx=0x7f97200135b0 comp rx=0 tx=0).stop 2023-07-21T18:52:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.723+0000 7f973b27cb80 1 -- 172.21.15.39:0/1105689643 shutdown_connections 2023-07-21T18:52:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.723+0000 7f973b27cb80 1 --2- 172.21.15.39:0/1105689643 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x562574cb8de0 0x562574cbb1f0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.723+0000 7f973b27cb80 1 --2- 172.21.15.39:0/1105689643 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562574aa76b0 0x562574c57f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.723+0000 7f973b27cb80 1 --2- 172.21.15.39:0/1105689643 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562574c512d0 0x562574c589b0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.723+0000 7f9707fff700 0 client.16998 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:52:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.723+0000 7f973b27cb80 1 --2- 172.21.15.39:0/1105689643 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562574c4b580 0x562574c58470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.723+0000 7f973b27cb80 1 -- 172.21.15.39:0/1105689643 >> 172.21.15.39:0/1105689643 conn(0x562574c40be0 msgr2=0x562574c52fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:52:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.723+0000 7f973b27cb80 1 -- 172.21.15.39:0/1105689643 shutdown_connections 2023-07-21T18:52:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.723+0000 7f973b27cb80 1 -- 172.21.15.39:0/1105689643 wait complete. 2023-07-21T18:52:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.723+0000 7f973b27cb80 20 client.16998 trim_cache size 0 max 16384 2023-07-21T18:52:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:52:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:52:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:52:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:52:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.724+0000 7f972f0f6700 1 -- 172.21.15.39:0/3967186796 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f97100a12c0 msgr2=0x7f97100a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:52:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.724+0000 7f972f0f6700 1 --2- 172.21.15.39:0/3967186796 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f97100a12c0 0x7f97100a3770 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f97280ffba0 tx=0x7f972400b410 comp rx=0 tx=0).stop 2023-07-21T18:52:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.724+0000 7f972f0f6700 1 -- 172.21.15.39:0/3967186796 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97280ff3e0 msgr2=0x7f972819c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:52:50.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.724+0000 7f972f0f6700 1 --2- 172.21.15.39:0/3967186796 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97280ff3e0 0x7f972819c010 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f97180095c0 tx=0x7f971800f710 comp rx=0 tx=0).stop 2023-07-21T18:52:50.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.724+0000 7f972f0f6700 1 -- 172.21.15.39:0/3967186796 shutdown_connections 2023-07-21T18:52:50.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.724+0000 7f972f0f6700 1 --2- 172.21.15.39:0/3967186796 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f97100a12c0 0x7f97100a3770 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:50.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.724+0000 7f972f0f6700 1 --2- 172.21.15.39:0/3967186796 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9728104a30 0x7f9728195930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:50.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.724+0000 7f972f0f6700 1 --2- 172.21.15.39:0/3967186796 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f97280fea30 0x7f972819bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:50.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.724+0000 7f972f0f6700 1 --2- 172.21.15.39:0/3967186796 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97280ff3e0 0x7f972819c010 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:52:50.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.724+0000 7f972f0f6700 1 -- 172.21.15.39:0/3967186796 >> 172.21.15.39:0/3967186796 conn(0x7f9728074760 msgr2=0x7f9728067920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:52:50.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.724+0000 7f972f0f6700 1 -- 172.21.15.39:0/3967186796 shutdown_connections 2023-07-21T18:52:50.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:52:50.724+0000 7f972f0f6700 1 -- 172.21.15.39:0/3967186796 wait complete. 2023-07-21T18:52:51.289 INFO:tasks.workunit.client.0.smithi039.stdout: 1 640948 5.47 MB/sec execute 484 sec latency 2824.892 ms 2023-07-21T18:52:52.289 INFO:tasks.workunit.client.0.smithi039.stdout: 1 642134 5.47 MB/sec execute 485 sec latency 3254.677 ms 2023-07-21T18:52:52.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:52 smithi193 ceph-mon[98204]: pgmap v347: 97 pgs: 97 active+clean; 262 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 1.9 MiB/s wr, 177 op/s 2023-07-21T18:52:52.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:52 smithi155 ceph-mon[97934]: pgmap v347: 97 pgs: 97 active+clean; 262 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 1.9 MiB/s wr, 177 op/s 2023-07-21T18:52:52.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:52 smithi039 ceph-mon[96489]: pgmap v347: 97 pgs: 97 active+clean; 262 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 1.9 MiB/s wr, 177 op/s 2023-07-21T18:52:53.298 INFO:tasks.workunit.client.0.smithi039.stdout: 1 643392 5.47 MB/sec execute 486 sec latency 12.331 ms 2023-07-21T18:52:54.289 INFO:tasks.workunit.client.0.smithi039.stdout: 1 644492 5.47 MB/sec execute 487 sec latency 113.603 ms 2023-07-21T18:52:54.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:54 smithi193 ceph-mon[98204]: pgmap v348: 97 pgs: 97 active+clean; 262 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 1.3 MiB/s wr, 113 op/s 2023-07-21T18:52:54.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:54 smithi155 ceph-mon[97934]: pgmap v348: 97 pgs: 97 active+clean; 262 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 1.3 MiB/s wr, 113 op/s 2023-07-21T18:52:54.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:54 smithi039 ceph-mon[96489]: pgmap v348: 97 pgs: 97 active+clean; 262 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 1.3 MiB/s wr, 113 op/s 2023-07-21T18:52:55.290 INFO:tasks.workunit.client.0.smithi039.stdout: 1 644492 5.46 MB/sec execute 488 sec latency 1113.670 ms 2023-07-21T18:52:56.290 INFO:tasks.workunit.client.0.smithi039.stdout: 1 644492 5.45 MB/sec execute 489 sec latency 2113.790 ms 2023-07-21T18:52:56.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:56 smithi193 ceph-mon[98204]: pgmap v349: 97 pgs: 97 active+clean; 262 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 1.3 MiB/s wr, 113 op/s 2023-07-21T18:52:56.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:56 smithi155 ceph-mon[97934]: pgmap v349: 97 pgs: 97 active+clean; 262 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 1.3 MiB/s wr, 113 op/s 2023-07-21T18:52:56.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:56 smithi039 ceph-mon[96489]: pgmap v349: 97 pgs: 97 active+clean; 262 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 1.3 MiB/s wr, 113 op/s 2023-07-21T18:52:57.290 INFO:tasks.workunit.client.0.smithi039.stdout: 1 645677 5.45 MB/sec execute 490 sec latency 2546.675 ms 2023-07-21T18:52:58.290 INFO:tasks.workunit.client.0.smithi039.stdout: 1 646901 5.46 MB/sec execute 491 sec latency 16.800 ms 2023-07-21T18:52:58.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:52:58 smithi193 ceph-mon[98204]: pgmap v350: 97 pgs: 97 active+clean; 261 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 1.9 MiB/s wr, 176 op/s 2023-07-21T18:52:58.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:52:58 smithi155 ceph-mon[97934]: pgmap v350: 97 pgs: 97 active+clean; 261 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 1.9 MiB/s wr, 176 op/s 2023-07-21T18:52:58.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:52:58 smithi039 ceph-mon[96489]: pgmap v350: 97 pgs: 97 active+clean; 261 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 1.9 MiB/s wr, 176 op/s 2023-07-21T18:52:59.290 INFO:tasks.workunit.client.0.smithi039.stdout: 1 648042 5.46 MB/sec execute 492 sec latency 242.544 ms 2023-07-21T18:53:00.290 INFO:tasks.workunit.client.0.smithi039.stdout: 1 648042 5.44 MB/sec execute 493 sec latency 1242.622 ms 2023-07-21T18:53:00.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:53:00 smithi193 ceph-mon[98204]: pgmap v351: 97 pgs: 97 active+clean; 261 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 1.3 MiB/s wr, 120 op/s 2023-07-21T18:53:00.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:53:00 smithi155 ceph-mon[97934]: pgmap v351: 97 pgs: 97 active+clean; 261 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 1.3 MiB/s wr, 120 op/s 2023-07-21T18:53:00.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:53:00 smithi039 ceph-mon[96489]: pgmap v351: 97 pgs: 97 active+clean; 261 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 1.3 MiB/s wr, 120 op/s 2023-07-21T18:53:01.290 INFO:tasks.workunit.client.0.smithi039.stdout: 1 648042 5.43 MB/sec execute 494 sec latency 2242.726 ms 2023-07-21T18:53:02.290 INFO:tasks.workunit.client.0.smithi039.stdout: 1 648641 5.43 MB/sec execute 495 sec latency 2672.601 ms 2023-07-21T18:53:02.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:53:02 smithi193 ceph-mon[98204]: pgmap v352: 97 pgs: 97 active+clean; 264 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 932 KiB/s rd, 2.0 MiB/s wr, 182 op/s 2023-07-21T18:53:02.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:53:02 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:53:02.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:53:02 smithi155 ceph-mon[97934]: pgmap v352: 97 pgs: 97 active+clean; 264 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 932 KiB/s rd, 2.0 MiB/s wr, 182 op/s 2023-07-21T18:53:02.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:53:02 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:53:02.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:53:02 smithi039 ceph-mon[96489]: pgmap v352: 97 pgs: 97 active+clean; 264 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 932 KiB/s rd, 2.0 MiB/s wr, 182 op/s 2023-07-21T18:53:02.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:53:02 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:53:03.290 INFO:tasks.workunit.client.0.smithi039.stdout: 1 650120 5.44 MB/sec execute 496 sec latency 24.244 ms 2023-07-21T18:53:03.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:53:03 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:53:03.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:53:03 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:53:03.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:53:03 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:53:03.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:53:03 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:53:03.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:53:03 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:53:03.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:53:03 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:53:03.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:53:03 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:53:03.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:53:03 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:53:03.678 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:53:03 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:53:03.859 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.846+0000 7f831e141700 1 -- 172.21.15.39:0/3709509979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83180ff760 msgr2=0x7f8318103830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:53:03.859 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.846+0000 7f831e141700 1 --2- 172.21.15.39:0/3709509979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83180ff760 0x7f8318103830 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f830c009990 tx=0x7f830c009ca0 comp rx=0 tx=0).stop 2023-07-21T18:53:03.859 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.846+0000 7f831e141700 1 -- 172.21.15.39:0/3709509979 shutdown_connections 2023-07-21T18:53:03.859 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.846+0000 7f831e141700 1 --2- 172.21.15.39:0/3709509979 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8318103f10 0x7f831810bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:03.860 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.846+0000 7f831e141700 1 --2- 172.21.15.39:0/3709509979 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83180fedc0 0x7f83180ff190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:03.860 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.846+0000 7f831e141700 1 --2- 172.21.15.39:0/3709509979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83180ff760 0x7f8318103830 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:03.860 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.846+0000 7f831e141700 1 -- 172.21.15.39:0/3709509979 >> 172.21.15.39:0/3709509979 conn(0x7f83180759e0 msgr2=0x7f8318075de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:53:03.860 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.846+0000 7f831e141700 1 -- 172.21.15.39:0/3709509979 shutdown_connections 2023-07-21T18:53:03.860 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.846+0000 7f831e141700 1 -- 172.21.15.39:0/3709509979 wait complete. 2023-07-21T18:53:03.861 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.847+0000 7f831e141700 1 Processor -- start 2023-07-21T18:53:03.862 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.848+0000 7f831e141700 1 -- start start 2023-07-21T18:53:03.862 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.848+0000 7f831e141700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83180fedc0 0x7f8318194d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:03.862 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.848+0000 7f831e141700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83180ff760 0x7f83181952c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:03.862 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.848+0000 7f831e141700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8318103f10 0x7f8318199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:03.862 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.848+0000 7f831e141700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8318199f70 con 0x7f83180ff760 2023-07-21T18:53:03.862 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.848+0000 7f831e141700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f831819a0e0 con 0x7f8318103f10 2023-07-21T18:53:03.862 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.848+0000 7f831e141700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f831819a2a0 con 0x7f83180fedc0 2023-07-21T18:53:03.863 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.848+0000 7f8316ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83180ff760 0x7f83181952c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:03.863 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.848+0000 7f83177fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83180fedc0 0x7f8318194d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:03.863 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.849+0000 7f8316ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83180ff760 0x7f83181952c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60910/0 (socket says 172.21.15.39:60910) 2023-07-21T18:53:03.863 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.849+0000 7f83177fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83180fedc0 0x7f8318194d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:51228/0 (socket says 172.21.15.39:51228) 2023-07-21T18:53:03.863 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.849+0000 7f8316ffd700 1 -- 172.21.15.39:0/1767686917 learned_addr learned my addr 172.21.15.39:0/1767686917 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:53:03.863 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.849+0000 7f8317fff700 1 --2- 172.21.15.39:0/1767686917 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8318103f10 0x7f8318199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:03.863 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.849+0000 7f8316ffd700 1 -- 172.21.15.39:0/1767686917 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83180fedc0 msgr2=0x7f8318194d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:53:03.864 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.849+0000 7f8316ffd700 1 --2- 172.21.15.39:0/1767686917 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83180fedc0 0x7f8318194d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:03.864 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.849+0000 7f8316ffd700 1 -- 172.21.15.39:0/1767686917 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8318103f10 msgr2=0x7f8318199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:53:03.864 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.849+0000 7f8316ffd700 1 --2- 172.21.15.39:0/1767686917 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8318103f10 0x7f8318199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:03.864 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.849+0000 7f8316ffd700 1 -- 172.21.15.39:0/1767686917 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8310009c40 con 0x7f83180ff760 2023-07-21T18:53:03.864 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.849+0000 7f83177fe700 1 --2- 172.21.15.39:0/1767686917 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83180fedc0 0x7f8318194d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:53:03.864 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.849+0000 7f8316ffd700 1 --2- 172.21.15.39:0/1767686917 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83180ff760 0x7f83181952c0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f830c004710 tx=0x7f830c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:53:03.865 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.849+0000 7f8314ff9700 1 -- 172.21.15.39:0/1767686917 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f830c01d070 con 0x7f83180ff760 2023-07-21T18:53:03.865 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.849+0000 7f8314ff9700 1 -- 172.21.15.39:0/1767686917 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f830c021410 con 0x7f83180ff760 2023-07-21T18:53:03.865 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.850+0000 7f831e141700 1 -- 172.21.15.39:0/1767686917 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f830c0095f0 con 0x7f83180ff760 2023-07-21T18:53:03.865 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.850+0000 7f8314ff9700 1 -- 172.21.15.39:0/1767686917 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f830c017650 con 0x7f83180ff760 2023-07-21T18:53:03.865 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.850+0000 7f831e141700 1 -- 172.21.15.39:0/1767686917 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8318068850 con 0x7f83180ff760 2023-07-21T18:53:03.866 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.854+0000 7f8314ff9700 1 -- 172.21.15.39:0/1767686917 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f830c021580 con 0x7f83180ff760 2023-07-21T18:53:03.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.855+0000 7f832a2c7b80 1 Processor -- start 2023-07-21T18:53:03.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.855+0000 7f832a2c7b80 1 -- start start 2023-07-21T18:53:03.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.855+0000 7f8314ff9700 1 --2- 172.21.15.39:0/1767686917 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f83000a1340 0x7f83000a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:03.869 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.855+0000 7f832a2c7b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561a676506b0 0x561a677f9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:03.869 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.855+0000 7f832a2c7b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561a677fa2d0 0x561a677f4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:03.869 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.855+0000 7f832a2c7b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a677f4580 0x561a677f49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:03.869 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.855+0000 7f832a2c7b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a676f9c50 con 0x561a676506b0 2023-07-21T18:53:03.869 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.855+0000 7f832a2c7b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a6774c9e0 con 0x561a677fa2d0 2023-07-21T18:53:03.869 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.855+0000 7f832a2c7b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a67745e70 con 0x561a677f4580 2023-07-21T18:53:03.869 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.855+0000 7f83177fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561a676506b0 0x561a677f9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:03.870 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.855+0000 7f8314ff9700 1 -- 172.21.15.39:0/1767686917 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f830c0cdf00 con 0x7f83180ff760 2023-07-21T18:53:03.870 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.855+0000 7f83177fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561a676506b0 0x561a677f9d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60916/0 (socket says 172.21.15.39:60916) 2023-07-21T18:53:03.870 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.855+0000 7f83177fe700 1 -- 172.21.15.39:0/1646185512 learned_addr learned my addr 172.21.15.39:0/1646185512 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:53:03.870 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.856+0000 7f8317fff700 1 --2- 172.21.15.39:0/1646185512 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a677f4580 0x561a677f49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:03.870 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.856+0000 7f8317fff700 1 --2- 172.21.15.39:0/1767686917 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f83000a1340 0x7f83000a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:03.870 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.856+0000 7f83177fe700 1 -- 172.21.15.39:0/1646185512 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a677f4580 msgr2=0x561a677f49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:53:03.870 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.856+0000 7f83177fe700 1 --2- 172.21.15.39:0/1646185512 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a677f4580 0x561a677f49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:03.870 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.856+0000 7f83177fe700 1 -- 172.21.15.39:0/1646185512 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561a677fa2d0 msgr2=0x561a677f4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:53:03.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.856+0000 7f83177fe700 1 --2- 172.21.15.39:0/1646185512 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561a677fa2d0 0x561a677f4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:03.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.856+0000 7f83177fe700 1 -- 172.21.15.39:0/1646185512 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a67573d80 con 0x561a676506b0 2023-07-21T18:53:03.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.856+0000 7f8317fff700 1 --2- 172.21.15.39:0/1646185512 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a677f4580 0x561a677f49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:53:03.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.856+0000 7f83177fe700 1 --2- 172.21.15.39:0/1646185512 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561a676506b0 0x561a677f9d90 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f83080115f0 tx=0x7f8308011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a29bfaf2d28f631 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:53:03.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.856+0000 7f8317fff700 1 --2- 172.21.15.39:0/1767686917 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f83000a1340 0x7f83000a37f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8310009fd0 tx=0x7f831000b040 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:53:03.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.856+0000 7f82fdffb700 1 -- 172.21.15.39:0/1646185512 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f830800f680 con 0x561a676506b0 2023-07-21T18:53:03.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.856+0000 7f82fdffb700 1 -- 172.21.15.39:0/1646185512 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f830800fca0 con 0x561a676506b0 2023-07-21T18:53:03.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.856+0000 7f82fdffb700 1 -- 172.21.15.39:0/1646185512 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8308019d30 con 0x561a676506b0 2023-07-21T18:53:03.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.857+0000 7f832a2c7b80 1 -- 172.21.15.39:0/1646185512 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561a676506b0 msgr2=0x561a677f9d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:53:03.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.857+0000 7f832a2c7b80 1 --2- 172.21.15.39:0/1646185512 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561a676506b0 0x561a677f9d90 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f83080115f0 tx=0x7f8308011900 comp rx=0 tx=0).stop 2023-07-21T18:53:03.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.858+0000 7f832a2c7b80 1 -- 172.21.15.39:0/1646185512 shutdown_connections 2023-07-21T18:53:03.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.858+0000 7f832a2c7b80 1 --2- 172.21.15.39:0/1646185512 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a677f4580 0x561a677f49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:03.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.858+0000 7f832a2c7b80 1 --2- 172.21.15.39:0/1646185512 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561a676506b0 0x561a677f9d90 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:03.876 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.858+0000 7f832a2c7b80 1 --2- 172.21.15.39:0/1646185512 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561a677fa2d0 0x561a677f4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:03.876 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.858+0000 7f832a2c7b80 1 -- 172.21.15.39:0/1646185512 >> 172.21.15.39:0/1646185512 conn(0x561a677e9be0 msgr2=0x561a677eef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:53:03.876 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.858+0000 7f832a2c7b80 1 -- 172.21.15.39:0/1646185512 shutdown_connections 2023-07-21T18:53:03.876 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.858+0000 7f832a2c7b80 1 -- 172.21.15.39:0/1646185512 wait complete. 2023-07-21T18:53:03.876 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.858+0000 7f832a2c7b80 1 -- start start 2023-07-21T18:53:03.876 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.859+0000 7f832a2c7b80 1 Processor -- start 2023-07-21T18:53:03.877 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.859+0000 7f832a2c7b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a676506b0 0x561a677fd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:03.877 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.859+0000 7f832a2c7b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561a677f4580 0x561a677fdde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:03.877 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.859+0000 7f832a2c7b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561a677fa2d0 0x561a677fe320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:03.877 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.859+0000 7f832a2c7b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a676e5a60 con 0x561a677f4580 2023-07-21T18:53:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.859+0000 7f832a2c7b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a676e5760 con 0x561a677fa2d0 2023-07-21T18:53:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.859+0000 7f832a2c7b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a675c98c0 con 0x561a676506b0 2023-07-21T18:53:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.859+0000 7f832a2c7b80 10 client.0 _ll_register_callbacks cb 0x561a676e3700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:53:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.859+0000 7f8316ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561a677f4580 0x561a677fdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.859+0000 7f83177fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a676506b0 0x561a677fd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.859+0000 7f8316ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561a677f4580 0x561a677fdde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60924/0 (socket says 172.21.15.39:60924) 2023-07-21T18:53:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.859+0000 7f8316ffd700 1 -- 172.21.15.39:0/1928173249 learned_addr learned my addr 172.21.15.39:0/1928173249 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:53:03.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.859+0000 7f8317fff700 1 --2- 172.21.15.39:0/1928173249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561a677fa2d0 0x561a677fe320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:03.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.859+0000 7f8316ffd700 1 -- 172.21.15.39:0/1928173249 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a676506b0 msgr2=0x561a677fd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:53:03.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.859+0000 7f8316ffd700 1 --2- 172.21.15.39:0/1928173249 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a676506b0 0x561a677fd8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:03.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.859+0000 7f8316ffd700 1 -- 172.21.15.39:0/1928173249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561a677fa2d0 msgr2=0x561a677fe320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:53:03.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.859+0000 7f8316ffd700 1 --2- 172.21.15.39:0/1928173249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561a677fa2d0 0x561a677fe320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:03.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.859+0000 7f8316ffd700 1 -- 172.21.15.39:0/1928173249 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8308009dd0 con 0x561a677f4580 2023-07-21T18:53:03.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.859+0000 7f8316ffd700 1 --2- 172.21.15.39:0/1928173249 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561a677f4580 0x561a677fdde0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f830c0cb6b0 tx=0x7f830c09ae30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cc4736357d6ab04 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:53:03.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.859+0000 7f82f6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:53:03.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.860+0000 7f82f6ffd700 1 -- 172.21.15.39:0/1928173249 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f830c09b4c0 con 0x561a677f4580 2023-07-21T18:53:03.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.860+0000 7f82f6ffd700 1 -- 172.21.15.39:0/1928173249 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f830c09b060 con 0x561a677f4580 2023-07-21T18:53:03.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.860+0000 7f832a2c7b80 1 -- 172.21.15.39:0/1928173249 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561a6769f490 con 0x561a677f4580 2023-07-21T18:53:03.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.860+0000 7f82f6ffd700 1 -- 172.21.15.39:0/1928173249 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f830c09f090 con 0x561a677f4580 2023-07-21T18:53:03.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.860+0000 7f832a2c7b80 10 client.17010 fetch_fsmap learned FSMap version 68 2023-07-21T18:53:03.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.860+0000 7f832a2c7b80 1 -- 172.21.15.39:0/1928173249 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) v3 -- 0x561a6755cc00 con 0x561a677f4580 2023-07-21T18:53:03.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.860+0000 7f82f6ffd700 1 -- 172.21.15.39:0/1928173249 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f830c0b7080 con 0x561a677f4580 2023-07-21T18:53:03.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.860+0000 7f82f6ffd700 1 -- 172.21.15.39:0/1928173249 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 68) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f830c0a3020 con 0x561a677f4580 2023-07-21T18:53:03.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.860+0000 7f832a2c7b80 10 client.17010 fetch_fsmap finished waiting for FSMap version 68 2023-07-21T18:53:03.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.860+0000 7f832a2c7b80 10 client.17010 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:53:03.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.860+0000 7f832a2c7b80 20 client.17010 populate_metadata read hostname 'smithi039' 2023-07-21T18:53:03.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.860+0000 7f832a2c7b80 1 --2- 172.21.15.39:0/1928173249 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x561a67816010 0x561a678183f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:03.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.860+0000 7f832a2c7b80 4 client.17010 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:53:03.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.860+0000 7f832a2c7b80 1 -- 172.21.15.39:0/1928173249 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561a6773ee50 con 0x561a67816010 2023-07-21T18:53:03.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.862+0000 7f83177fe700 1 --2- 172.21.15.39:0/1928173249 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x561a67816010 0x561a678183f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:03.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.863+0000 7f82f6ffd700 10 client.17010 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:53:03.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.863+0000 7f83177fe700 1 --2- 172.21.15.39:0/1928173249 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x561a67816010 0x561a678183f0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56b94e2ee4940624 server_cookie=b3bb93c4f0d8c84 in_seq=0 out_seq=0 2023-07-21T18:53:03.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.865+0000 7f82f6ffd700 1 -- 172.21.15.39:0/1928173249 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561a6773ee50 con 0x561a67816010 2023-07-21T18:53:03.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.865+0000 7f82f6ffd700 10 client.17010 handle_command_reply: tid=0 2023-07-21T18:53:03.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.865+0000 7f832a2c7b80 1 client.17010 shutdown 2023-07-21T18:53:03.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.865+0000 7f832a2c7b80 1 -- 172.21.15.39:0/1928173249 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561a677f4580 msgr2=0x561a677fdde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:53:03.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.865+0000 7f832a2c7b80 1 --2- 172.21.15.39:0/1928173249 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561a677f4580 0x561a677fdde0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f830c0cb6b0 tx=0x7f830c09ae30 comp rx=0 tx=0).stop 2023-07-21T18:53:03.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.865+0000 7f832a2c7b80 1 -- 172.21.15.39:0/1928173249 shutdown_connections 2023-07-21T18:53:03.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.865+0000 7f832a2c7b80 1 --2- 172.21.15.39:0/1928173249 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x561a67816010 0x561a678183f0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:03.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.865+0000 7f832a2c7b80 1 --2- 172.21.15.39:0/1928173249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561a677fa2d0 0x561a677fe320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:03.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.865+0000 7f832a2c7b80 1 --2- 172.21.15.39:0/1928173249 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a676506b0 0x561a677fd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:03.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.865+0000 7f832a2c7b80 1 --2- 172.21.15.39:0/1928173249 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561a677f4580 0x561a677fdde0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:03.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.865+0000 7f832a2c7b80 1 -- 172.21.15.39:0/1928173249 >> 172.21.15.39:0/1928173249 conn(0x561a677e9be0 msgr2=0x561a677ee4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:53:03.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.865+0000 7f82f6ffd700 0 client.17010 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:53:03.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.865+0000 7f832a2c7b80 1 -- 172.21.15.39:0/1928173249 shutdown_connections 2023-07-21T18:53:03.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.865+0000 7f832a2c7b80 1 -- 172.21.15.39:0/1928173249 wait complete. 2023-07-21T18:53:03.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.865+0000 7f832a2c7b80 20 client.17010 trim_cache size 0 max 16384 2023-07-21T18:53:03.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.886+0000 7f832a2c7b80 1 Processor -- start 2023-07-21T18:53:03.900 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.886+0000 7f832a2c7b80 1 -- start start 2023-07-21T18:53:03.900 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.886+0000 7f832a2c7b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561a676506b0 0x561a677f4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:03.900 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.886+0000 7f832a2c7b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a677f4580 0x561a6780d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:03.900 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.886+0000 7f832a2c7b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561a677fa2d0 0x561a677f9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:03.900 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.886+0000 7f832a2c7b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a677fa6a0 con 0x561a677fa2d0 2023-07-21T18:53:03.900 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.886+0000 7f832a2c7b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a677ebd00 con 0x561a676506b0 2023-07-21T18:53:03.900 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.886+0000 7f832a2c7b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a677fa110 con 0x561a677f4580 2023-07-21T18:53:03.901 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.886+0000 7f8317fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561a677fa2d0 0x561a677f9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:03.901 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.886+0000 7f8316ffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a677f4580 0x561a6780d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:03.901 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.886+0000 7f8317fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561a677fa2d0 0x561a677f9160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60934/0 (socket says 172.21.15.39:60934) 2023-07-21T18:53:03.901 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.886+0000 7f8316ffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a677f4580 0x561a6780d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:51252/0 (socket says 172.21.15.39:51252) 2023-07-21T18:53:03.901 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.886+0000 7f8317fff700 1 -- 172.21.15.39:0/1030823343 learned_addr learned my addr 172.21.15.39:0/1030823343 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:53:03.901 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.886+0000 7f83177fe700 1 --2- 172.21.15.39:0/1030823343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561a676506b0 0x561a677f4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:03.901 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.887+0000 7f8317fff700 1 -- 172.21.15.39:0/1030823343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a677f4580 msgr2=0x561a6780d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:53:03.902 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.887+0000 7f8317fff700 1 --2- 172.21.15.39:0/1030823343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a677f4580 0x561a6780d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:03.902 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.887+0000 7f8317fff700 1 -- 172.21.15.39:0/1030823343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561a676506b0 msgr2=0x561a677f4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:53:03.902 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.887+0000 7f8317fff700 1 --2- 172.21.15.39:0/1030823343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561a676506b0 0x561a677f4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:03.902 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.887+0000 7f8317fff700 1 -- 172.21.15.39:0/1030823343 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a6759a1f0 con 0x561a677fa2d0 2023-07-21T18:53:03.902 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.887+0000 7f8316ffd700 1 --2- 172.21.15.39:0/1030823343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a677f4580 0x561a6780d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:53:03.902 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.887+0000 7f8317fff700 1 --2- 172.21.15.39:0/1030823343 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561a677fa2d0 0x561a677f9160 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f83100188d0 tx=0x7f8310009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2bdf6aec02854a8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:53:03.902 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.887+0000 7f82fe7fc700 1 -- 172.21.15.39:0/1030823343 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831002a070 con 0x561a677fa2d0 2023-07-21T18:53:03.902 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.887+0000 7f82fe7fc700 1 -- 172.21.15.39:0/1030823343 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8310021040 con 0x561a677fa2d0 2023-07-21T18:53:03.903 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.887+0000 7f82fe7fc700 1 -- 172.21.15.39:0/1030823343 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8310020050 con 0x561a677fa2d0 2023-07-21T18:53:03.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.888+0000 7f832a2c7b80 1 -- 172.21.15.39:0/1030823343 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561a677fa2d0 msgr2=0x561a677f9160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:53:03.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.888+0000 7f832a2c7b80 1 --2- 172.21.15.39:0/1030823343 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561a677fa2d0 0x561a677f9160 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f83100188d0 tx=0x7f8310009920 comp rx=0 tx=0).stop 2023-07-21T18:53:03.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.888+0000 7f832a2c7b80 1 -- 172.21.15.39:0/1030823343 shutdown_connections 2023-07-21T18:53:03.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.888+0000 7f832a2c7b80 1 --2- 172.21.15.39:0/1030823343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561a676506b0 0x561a677f4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:03.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.888+0000 7f832a2c7b80 1 --2- 172.21.15.39:0/1030823343 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561a677fa2d0 0x561a677f9160 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:03.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.888+0000 7f832a2c7b80 1 --2- 172.21.15.39:0/1030823343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a677f4580 0x561a6780d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:03.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.888+0000 7f832a2c7b80 1 -- 172.21.15.39:0/1030823343 >> 172.21.15.39:0/1030823343 conn(0x561a677e9be0 msgr2=0x561a677f8a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:53:03.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.888+0000 7f832a2c7b80 1 -- 172.21.15.39:0/1030823343 shutdown_connections 2023-07-21T18:53:03.907 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.888+0000 7f832a2c7b80 1 -- 172.21.15.39:0/1030823343 wait complete. 2023-07-21T18:53:03.907 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.888+0000 7f832a2c7b80 1 -- start start 2023-07-21T18:53:03.907 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.888+0000 7f832a2c7b80 1 Processor -- start 2023-07-21T18:53:03.907 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.889+0000 7f832a2c7b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561a676506b0 0x561a6780b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:03.908 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.889+0000 7f832a2c7b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a677f4580 0x561a6780bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:03.908 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.889+0000 7f832a2c7b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561a677fa2d0 0x561a67862f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:03.908 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.889+0000 7f832a2c7b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a67863490 con 0x561a677fa2d0 2023-07-21T18:53:03.908 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.889+0000 7f832a2c7b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a67863600 con 0x561a676506b0 2023-07-21T18:53:03.909 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.889+0000 7f832a2c7b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a678637c0 con 0x561a677f4580 2023-07-21T18:53:03.909 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.889+0000 7f832a2c7b80 10 client.0 _ll_register_callbacks cb 0x561a676e3700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:53:03.909 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.889+0000 7f8316ffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a677f4580 0x561a6780bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:03.910 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.889+0000 7f8317fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561a677fa2d0 0x561a67862f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:03.910 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.889+0000 7f83177fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561a676506b0 0x561a6780b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:03.910 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.889+0000 7f8316ffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a677f4580 0x561a6780bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:51258/0 (socket says 172.21.15.39:51258) 2023-07-21T18:53:03.910 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.889+0000 7f8316ffd700 1 -- 172.21.15.39:0/1364085352 learned_addr learned my addr 172.21.15.39:0/1364085352 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:53:03.911 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.889+0000 7f8316ffd700 1 -- 172.21.15.39:0/1364085352 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561a676506b0 msgr2=0x561a6780b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:53:03.911 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.889+0000 7f8316ffd700 1 --2- 172.21.15.39:0/1364085352 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561a676506b0 0x561a6780b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:03.911 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.889+0000 7f8316ffd700 1 -- 172.21.15.39:0/1364085352 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561a677fa2d0 msgr2=0x561a67862f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:53:03.911 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.889+0000 7f8316ffd700 1 --2- 172.21.15.39:0/1364085352 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561a677fa2d0 0x561a67862f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:03.912 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.889+0000 7f8316ffd700 1 -- 172.21.15.39:0/1364085352 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561a6755cc00 con 0x561a677f4580 2023-07-21T18:53:03.912 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.889+0000 7f8316ffd700 1 --2- 172.21.15.39:0/1364085352 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a677f4580 0x561a6780bb90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f830c021830 tx=0x7f830c097450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39f692cb6e655014 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:53:03.912 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.889+0000 7f82f6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2023-07-21T18:53:03.913 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.890+0000 7f82f6ffd700 1 -- 172.21.15.39:0/1364085352 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f830c096f40 con 0x561a677f4580 2023-07-21T18:53:03.913 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.890+0000 7f82f6ffd700 1 -- 172.21.15.39:0/1364085352 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f830c09b060 con 0x561a677f4580 2023-07-21T18:53:03.913 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.890+0000 7f832a2c7b80 1 -- 172.21.15.39:0/1364085352 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f830800fca0 con 0x561a677f4580 2023-07-21T18:53:03.914 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.890+0000 7f82f6ffd700 1 -- 172.21.15.39:0/1364085352 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f830c09f090 con 0x561a677f4580 2023-07-21T18:53:03.914 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.890+0000 7f832a2c7b80 10 client.26488 fetch_fsmap learned FSMap version 68 2023-07-21T18:53:03.914 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.890+0000 7f832a2c7b80 1 -- 172.21.15.39:0/1364085352 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) v3 -- 0x7f83100186f0 con 0x561a677f4580 2023-07-21T18:53:03.915 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.890+0000 7f82f6ffd700 1 -- 172.21.15.39:0/1364085352 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f830c064080 con 0x561a677f4580 2023-07-21T18:53:03.915 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.890+0000 7f82f6ffd700 1 -- 172.21.15.39:0/1364085352 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 68) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f830c09b2c0 con 0x561a677f4580 2023-07-21T18:53:03.915 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.890+0000 7f832a2c7b80 10 client.26488 fetch_fsmap finished waiting for FSMap version 68 2023-07-21T18:53:03.915 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.890+0000 7f832a2c7b80 10 client.26488 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:53:03.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.890+0000 7f832a2c7b80 20 client.26488 populate_metadata read hostname 'smithi039' 2023-07-21T18:53:03.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.890+0000 7f832a2c7b80 1 --2- 172.21.15.39:0/1364085352 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x561a6786ab50 0x561a6786cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:03.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.890+0000 7f832a2c7b80 4 client.26488 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:53:03.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.890+0000 7f832a2c7b80 1 -- 172.21.15.39:0/1364085352 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561a67700ff0 con 0x561a6786ab50 2023-07-21T18:53:03.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.890+0000 7f83177fe700 1 --2- 172.21.15.39:0/1364085352 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x561a6786ab50 0x561a6786cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:03.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.891+0000 7f82f6ffd700 10 client.26488 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:53:03.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.891+0000 7f83177fe700 1 --2- 172.21.15.39:0/1364085352 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x561a6786ab50 0x561a6786cf60 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e146ba254fb676e8 server_cookie=724ee7956b522f36 in_seq=0 out_seq=0 2023-07-21T18:53:03.947 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.934+0000 7f82f6ffd700 1 -- 172.21.15.39:0/1364085352 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561a67700ff0 con 0x561a6786ab50 2023-07-21T18:53:03.947 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.934+0000 7f82f6ffd700 10 client.26488 handle_command_reply: tid=0 2023-07-21T18:53:03.947 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.934+0000 7f832a2c7b80 1 client.26488 shutdown 2023-07-21T18:53:03.948 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.934+0000 7f832a2c7b80 1 -- 172.21.15.39:0/1364085352 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a677f4580 msgr2=0x561a6780bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:53:03.948 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.934+0000 7f832a2c7b80 1 --2- 172.21.15.39:0/1364085352 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a677f4580 0x561a6780bb90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f830c021830 tx=0x7f830c097450 comp rx=0 tx=0).stop 2023-07-21T18:53:03.948 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.934+0000 7f832a2c7b80 1 -- 172.21.15.39:0/1364085352 shutdown_connections 2023-07-21T18:53:03.948 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.934+0000 7f832a2c7b80 1 --2- 172.21.15.39:0/1364085352 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x561a6786ab50 0x561a6786cf60 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:03.948 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.934+0000 7f832a2c7b80 1 --2- 172.21.15.39:0/1364085352 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561a676506b0 0x561a6780b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:03.948 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.934+0000 7f82f6ffd700 0 client.26488 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:53:03.948 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.934+0000 7f832a2c7b80 1 --2- 172.21.15.39:0/1364085352 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561a677fa2d0 0x561a67862f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:03.949 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.934+0000 7f832a2c7b80 1 --2- 172.21.15.39:0/1364085352 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a677f4580 0x561a6780bb90 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:03.949 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.934+0000 7f832a2c7b80 1 -- 172.21.15.39:0/1364085352 >> 172.21.15.39:0/1364085352 conn(0x561a677e9be0 msgr2=0x561a677fc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:53:03.949 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.934+0000 7f832a2c7b80 1 -- 172.21.15.39:0/1364085352 shutdown_connections 2023-07-21T18:53:03.949 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.934+0000 7f832a2c7b80 1 -- 172.21.15.39:0/1364085352 wait complete. 2023-07-21T18:53:03.949 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.934+0000 7f832a2c7b80 20 client.26488 trim_cache size 0 max 16384 2023-07-21T18:53:03.949 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:53:03.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:53:03.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:53:03.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:53:03.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.935+0000 7f831e141700 1 -- 172.21.15.39:0/1767686917 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f83000a1340 msgr2=0x7f83000a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:53:03.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.935+0000 7f831e141700 1 --2- 172.21.15.39:0/1767686917 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f83000a1340 0x7f83000a37f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8310009fd0 tx=0x7f831000b040 comp rx=0 tx=0).stop 2023-07-21T18:53:03.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.935+0000 7f831e141700 1 -- 172.21.15.39:0/1767686917 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83180ff760 msgr2=0x7f83181952c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:53:03.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.935+0000 7f831e141700 1 --2- 172.21.15.39:0/1767686917 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83180ff760 0x7f83181952c0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f830c004710 tx=0x7f830c009580 comp rx=0 tx=0).stop 2023-07-21T18:53:03.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.935+0000 7f831e141700 1 -- 172.21.15.39:0/1767686917 shutdown_connections 2023-07-21T18:53:03.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.935+0000 7f831e141700 1 --2- 172.21.15.39:0/1767686917 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f83000a1340 0x7f83000a37f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:03.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.935+0000 7f831e141700 1 --2- 172.21.15.39:0/1767686917 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8318103f10 0x7f8318199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:03.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.935+0000 7f831e141700 1 --2- 172.21.15.39:0/1767686917 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83180fedc0 0x7f8318194d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:03.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.935+0000 7f831e141700 1 --2- 172.21.15.39:0/1767686917 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83180ff760 0x7f83181952c0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:03.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.935+0000 7f831e141700 1 -- 172.21.15.39:0/1767686917 >> 172.21.15.39:0/1767686917 conn(0x7f83180759e0 msgr2=0x7f83180fc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:53:03.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.935+0000 7f831e141700 1 -- 172.21.15.39:0/1767686917 shutdown_connections 2023-07-21T18:53:03.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:03.935+0000 7f831e141700 1 -- 172.21.15.39:0/1767686917 wait complete. 2023-07-21T18:53:04.290 INFO:tasks.workunit.client.0.smithi039.stdout: 1 651149 5.43 MB/sec execute 497 sec latency 23.963 ms 2023-07-21T18:53:04.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:53:04 smithi193 ceph-mon[98204]: pgmap v353: 97 pgs: 97 active+clean; 264 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 1.3 MiB/s wr, 125 op/s 2023-07-21T18:53:04.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:53:04 smithi155 ceph-mon[97934]: pgmap v353: 97 pgs: 97 active+clean; 264 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 1.3 MiB/s wr, 125 op/s 2023-07-21T18:53:04.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:53:04 smithi039 ceph-mon[96489]: pgmap v353: 97 pgs: 97 active+clean; 264 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 1.3 MiB/s wr, 125 op/s 2023-07-21T18:53:05.290 INFO:tasks.workunit.client.0.smithi039.stdout: 1 651583 5.43 MB/sec execute 498 sec latency 655.918 ms 2023-07-21T18:53:05.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:53:05 smithi193 ceph-mon[98204]: osdmap e94: 12 total, 12 up, 12 in 2023-07-21T18:53:05.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:53:05 smithi155 ceph-mon[97934]: osdmap e94: 12 total, 12 up, 12 in 2023-07-21T18:53:05.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:53:05 smithi039 ceph-mon[96489]: osdmap e94: 12 total, 12 up, 12 in 2023-07-21T18:53:06.290 INFO:tasks.workunit.client.0.smithi039.stdout: 1 652183 5.42 MB/sec execute 499 sec latency 1448.053 ms 2023-07-21T18:53:06.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:53:06 smithi193 ceph-mon[98204]: pgmap v355: 97 pgs: 97 active+clean; 264 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 1.5 MiB/s wr, 151 op/s 2023-07-21T18:53:06.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:53:06 smithi155 ceph-mon[97934]: pgmap v355: 97 pgs: 97 active+clean; 264 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 1.5 MiB/s wr, 151 op/s 2023-07-21T18:53:06.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:53:06 smithi039 ceph-mon[96489]: pgmap v355: 97 pgs: 97 active+clean; 264 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 1.5 MiB/s wr, 151 op/s 2023-07-21T18:53:07.290 INFO:tasks.workunit.client.0.smithi039.stdout: 1 653732 5.43 MB/sec execute 500 sec latency 17.977 ms 2023-07-21T18:53:08.290 INFO:tasks.workunit.client.0.smithi039.stdout: 1 654999 5.43 MB/sec execute 501 sec latency 18.872 ms 2023-07-21T18:53:08.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:53:08 smithi193 ceph-mon[98204]: pgmap v356: 97 pgs: 97 active+clean; 263 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 1.6 MiB/s wr, 190 op/s 2023-07-21T18:53:08.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:53:08 smithi155 ceph-mon[97934]: pgmap v356: 97 pgs: 97 active+clean; 263 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 1.6 MiB/s wr, 190 op/s 2023-07-21T18:53:08.680 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:53:08 smithi039 ceph-mon[96489]: pgmap v356: 97 pgs: 97 active+clean; 263 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 1.6 MiB/s wr, 190 op/s 2023-07-21T18:53:09.290 INFO:tasks.workunit.client.0.smithi039.stdout: 1 655132 5.42 MB/sec execute 502 sec latency 925.356 ms 2023-07-21T18:53:10.291 INFO:tasks.workunit.client.0.smithi039.stdout: 1 655132 5.41 MB/sec execute 503 sec latency 1925.406 ms 2023-07-21T18:53:10.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:53:10 smithi193 ceph-mon[98204]: pgmap v357: 97 pgs: 97 active+clean; 263 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 1.6 MiB/s wr, 190 op/s 2023-07-21T18:53:10.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:53:10 smithi155 ceph-mon[97934]: pgmap v357: 97 pgs: 97 active+clean; 263 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 1.6 MiB/s wr, 190 op/s 2023-07-21T18:53:10.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:53:10 smithi039 ceph-mon[96489]: pgmap v357: 97 pgs: 97 active+clean; 263 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 1.6 MiB/s wr, 190 op/s 2023-07-21T18:53:11.290 INFO:tasks.workunit.client.0.smithi039.stdout: 1 655132 5.40 MB/sec execute 504 sec latency 2925.499 ms 2023-07-21T18:53:12.290 INFO:tasks.workunit.client.0.smithi039.stdout: 1 656316 5.40 MB/sec execute 505 sec latency 3356.332 ms 2023-07-21T18:53:12.427 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:53:12 smithi039 ceph-mon[96489]: pgmap v358: 97 pgs: 97 active+clean; 266 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 1.6 MiB/s wr, 186 op/s 2023-07-21T18:53:12.427 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:53:12 smithi039 ceph-mon[96489]: osdmap e95: 12 total, 12 up, 12 in 2023-07-21T18:53:12.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:53:12 smithi193 ceph-mon[98204]: pgmap v358: 97 pgs: 97 active+clean; 266 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 1.6 MiB/s wr, 186 op/s 2023-07-21T18:53:12.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:53:12 smithi193 ceph-mon[98204]: osdmap e95: 12 total, 12 up, 12 in 2023-07-21T18:53:12.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:53:12 smithi155 ceph-mon[97934]: pgmap v358: 97 pgs: 97 active+clean; 266 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 1.6 MiB/s wr, 186 op/s 2023-07-21T18:53:12.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:53:12 smithi155 ceph-mon[97934]: osdmap e95: 12 total, 12 up, 12 in 2023-07-21T18:53:13.291 INFO:tasks.workunit.client.0.smithi039.stdout: 1 657562 5.41 MB/sec execute 506 sec latency 14.053 ms 2023-07-21T18:53:14.291 INFO:tasks.workunit.client.0.smithi039.stdout: 1 658675 5.41 MB/sec execute 507 sec latency 209.020 ms 2023-07-21T18:53:14.427 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:53:14 smithi039 ceph-mon[96489]: pgmap v360: 97 pgs: 97 active+clean; 266 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 933 KiB/s rd, 1.9 MiB/s wr, 227 op/s 2023-07-21T18:53:14.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:53:14 smithi193 ceph-mon[98204]: pgmap v360: 97 pgs: 97 active+clean; 266 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 933 KiB/s rd, 1.9 MiB/s wr, 227 op/s 2023-07-21T18:53:14.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:53:14 smithi155 ceph-mon[97934]: pgmap v360: 97 pgs: 97 active+clean; 266 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 933 KiB/s rd, 1.9 MiB/s wr, 227 op/s 2023-07-21T18:53:15.291 INFO:tasks.workunit.client.0.smithi039.stdout: 1 658675 5.40 MB/sec execute 508 sec latency 1209.074 ms 2023-07-21T18:53:16.291 INFO:tasks.workunit.client.0.smithi039.stdout: 1 658675 5.39 MB/sec execute 509 sec latency 2209.178 ms 2023-07-21T18:53:16.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:53:16 smithi193 ceph-mon[98204]: pgmap v361: 97 pgs: 97 active+clean; 266 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 1.6 MiB/s wr, 186 op/s 2023-07-21T18:53:16.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:53:16 smithi155 ceph-mon[97934]: pgmap v361: 97 pgs: 97 active+clean; 266 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 1.6 MiB/s wr, 186 op/s 2023-07-21T18:53:16.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:53:16 smithi039 ceph-mon[96489]: pgmap v361: 97 pgs: 97 active+clean; 266 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 1.6 MiB/s wr, 186 op/s 2023-07-21T18:53:17.074 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.061+0000 7ff3530e8700 1 -- 172.21.15.39:0/2025610093 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff34c10c540 msgr2=0x7ff34c10e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:53:17.074 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.061+0000 7ff3530e8700 1 --2- 172.21.15.39:0/2025610093 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff34c10c540 0x7ff34c10e920 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7ff34400b0a0 tx=0x7ff34400b3b0 comp rx=0 tx=0).stop 2023-07-21T18:53:17.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.061+0000 7ff3530e8700 1 -- 172.21.15.39:0/2025610093 shutdown_connections 2023-07-21T18:53:17.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.061+0000 7ff3530e8700 1 --2- 172.21.15.39:0/2025610093 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff34c10c540 0x7ff34c10e920 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:17.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.061+0000 7ff3530e8700 1 --2- 172.21.15.39:0/2025610093 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff34c0679a0 0x7ff34c067d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:17.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.061+0000 7ff3530e8700 1 --2- 172.21.15.39:0/2025610093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff34c0682b0 0x7ff34c10c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:17.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.061+0000 7ff3530e8700 1 -- 172.21.15.39:0/2025610093 >> 172.21.15.39:0/2025610093 conn(0x7ff34c074760 msgr2=0x7ff34c074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:53:17.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.061+0000 7ff3530e8700 1 -- 172.21.15.39:0/2025610093 shutdown_connections 2023-07-21T18:53:17.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.061+0000 7ff3530e8700 1 -- 172.21.15.39:0/2025610093 wait complete. 2023-07-21T18:53:17.076 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.062+0000 7ff3530e8700 1 Processor -- start 2023-07-21T18:53:17.076 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.062+0000 7ff3530e8700 1 -- start start 2023-07-21T18:53:17.077 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.062+0000 7ff3530e8700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff34c0679a0 0x7ff34c197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:17.077 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.062+0000 7ff3530e8700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff34c0682b0 0x7ff34c198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:17.078 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.062+0000 7ff3530e8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff34c10c540 0x7ff34c19cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:17.078 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.062+0000 7ff3530e8700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff34c19d110 con 0x7ff34c10c540 2023-07-21T18:53:17.078 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.062+0000 7ff3530e8700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff34c19d280 con 0x7ff34c0682b0 2023-07-21T18:53:17.078 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.062+0000 7ff3530e8700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff34c19d440 con 0x7ff34c0679a0 2023-07-21T18:53:17.078 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.062+0000 7ff351685700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff34c10c540 0x7ff34c19cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:17.078 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.062+0000 7ff351685700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff34c10c540 0x7ff34c19cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60954/0 (socket says 172.21.15.39:60954) 2023-07-21T18:53:17.078 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.062+0000 7ff351685700 1 -- 172.21.15.39:0/3593897569 learned_addr learned my addr 172.21.15.39:0/3593897569 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:53:17.079 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.062+0000 7ff350e84700 1 --2- 172.21.15.39:0/3593897569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff34c0679a0 0x7ff34c197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:17.079 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.063+0000 7ff34bfff700 1 --2- 172.21.15.39:0/3593897569 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff34c0682b0 0x7ff34c198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:17.079 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.063+0000 7ff351685700 1 -- 172.21.15.39:0/3593897569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff34c0679a0 msgr2=0x7ff34c197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:53:17.079 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.063+0000 7ff351685700 1 --2- 172.21.15.39:0/3593897569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff34c0679a0 0x7ff34c197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:17.079 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.063+0000 7ff351685700 1 -- 172.21.15.39:0/3593897569 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff34c0682b0 msgr2=0x7ff34c198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:53:17.079 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.063+0000 7ff351685700 1 --2- 172.21.15.39:0/3593897569 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff34c0682b0 0x7ff34c198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:17.079 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.063+0000 7ff351685700 1 -- 172.21.15.39:0/3593897569 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff344009d00 con 0x7ff34c10c540 2023-07-21T18:53:17.080 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.063+0000 7ff351685700 1 --2- 172.21.15.39:0/3593897569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff34c10c540 0x7ff34c19cad0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7ff344009fd0 tx=0x7ff34400bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:53:17.080 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.063+0000 7ff349ffb700 1 -- 172.21.15.39:0/3593897569 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34400e040 con 0x7ff34c10c540 2023-07-21T18:53:17.080 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.063+0000 7ff349ffb700 1 -- 172.21.15.39:0/3593897569 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff3440094a0 con 0x7ff34c10c540 2023-07-21T18:53:17.080 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.063+0000 7ff3530e8700 1 -- 172.21.15.39:0/3593897569 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff34c19d6c0 con 0x7ff34c10c540 2023-07-21T18:53:17.080 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.063+0000 7ff349ffb700 1 -- 172.21.15.39:0/3593897569 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34401b460 con 0x7ff34c10c540 2023-07-21T18:53:17.080 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.063+0000 7ff3530e8700 1 -- 172.21.15.39:0/3593897569 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff34c19d940 con 0x7ff34c10c540 2023-07-21T18:53:17.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.066+0000 7ff349ffb700 1 -- 172.21.15.39:0/3593897569 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7ff34401d470 con 0x7ff34c10c540 2023-07-21T18:53:17.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.067+0000 7ff349ffb700 1 --2- 172.21.15.39:0/3593897569 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7ff3380a12c0 0x7ff3380a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:17.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.067+0000 7ff349ffb700 1 -- 172.21.15.39:0/3593897569 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8527+0+0 (secure 0 0 0) 0x7ff3440cdc40 con 0x7ff34c10c540 2023-07-21T18:53:17.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.067+0000 7ff350e84700 1 --2- 172.21.15.39:0/3593897569 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7ff3380a12c0 0x7ff3380a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:17.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.067+0000 7ff350e84700 1 --2- 172.21.15.39:0/3593897569 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7ff3380a12c0 0x7ff3380a3770 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff334009c60 tx=0x7ff334009260 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:53:17.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.067+0000 7ff35f26eb80 1 Processor -- start 2023-07-21T18:53:17.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.067+0000 7ff35f26eb80 1 -- start start 2023-07-21T18:53:17.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.067+0000 7ff35f26eb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf474a76b0 0x55cf47650d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:17.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.067+0000 7ff35f26eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf476512d0 0x55cf4764b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:17.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.068+0000 7ff35f26eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf4764b580 0x55cf4764b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:17.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.068+0000 7ff35f26eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf47550c50 con 0x55cf4764b580 2023-07-21T18:53:17.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.068+0000 7ff35f26eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf475a39e0 con 0x55cf476512d0 2023-07-21T18:53:17.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.068+0000 7ff35f26eb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf4759ce70 con 0x55cf474a76b0 2023-07-21T18:53:17.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.068+0000 7ff351685700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf4764b580 0x55cf4764b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:17.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.068+0000 7ff34bfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf476512d0 0x55cf4764b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:17.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.068+0000 7ff351685700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf4764b580 0x55cf4764b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60962/0 (socket says 172.21.15.39:60962) 2023-07-21T18:53:17.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.068+0000 7ff351685700 1 -- 172.21.15.39:0/2926795736 learned_addr learned my addr 172.21.15.39:0/2926795736 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:53:17.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.068+0000 7ff350e84700 1 --2- 172.21.15.39:0/2926795736 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf474a76b0 0x55cf47650d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:17.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.068+0000 7ff351685700 1 -- 172.21.15.39:0/2926795736 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf474a76b0 msgr2=0x55cf47650d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:53:17.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.068+0000 7ff351685700 1 --2- 172.21.15.39:0/2926795736 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf474a76b0 0x55cf47650d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:17.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.068+0000 7ff351685700 1 -- 172.21.15.39:0/2926795736 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf476512d0 msgr2=0x55cf4764b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:53:17.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.068+0000 7ff351685700 1 --2- 172.21.15.39:0/2926795736 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf476512d0 0x55cf4764b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:17.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.068+0000 7ff351685700 1 -- 172.21.15.39:0/2926795736 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cf473cad80 con 0x55cf4764b580 2023-07-21T18:53:17.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.068+0000 7ff34bfff700 1 --2- 172.21.15.39:0/2926795736 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf476512d0 0x55cf4764b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:53:17.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.068+0000 7ff351685700 1 --2- 172.21.15.39:0/2926795736 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf4764b580 0x55cf4764b9d0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7ff344097810 tx=0x7ff34401db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8813d175f9ec453c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:53:17.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.068+0000 7ff342ffd700 1 -- 172.21.15.39:0/2926795736 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff344003e10 con 0x55cf4764b580 2023-07-21T18:53:17.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.068+0000 7ff342ffd700 1 -- 172.21.15.39:0/2926795736 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff34409ada0 con 0x55cf4764b580 2023-07-21T18:53:17.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.068+0000 7ff342ffd700 1 -- 172.21.15.39:0/2926795736 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34401d980 con 0x55cf4764b580 2023-07-21T18:53:17.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.069+0000 7ff35f26eb80 1 -- 172.21.15.39:0/2926795736 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf4764b580 msgr2=0x55cf4764b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:53:17.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.069+0000 7ff35f26eb80 1 --2- 172.21.15.39:0/2926795736 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf4764b580 0x55cf4764b9d0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7ff344097810 tx=0x7ff34401db10 comp rx=0 tx=0).stop 2023-07-21T18:53:17.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.069+0000 7ff35f26eb80 1 -- 172.21.15.39:0/2926795736 shutdown_connections 2023-07-21T18:53:17.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.069+0000 7ff35f26eb80 1 --2- 172.21.15.39:0/2926795736 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf4764b580 0x55cf4764b9d0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:17.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.069+0000 7ff35f26eb80 1 --2- 172.21.15.39:0/2926795736 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf474a76b0 0x55cf47650d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:17.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.069+0000 7ff35f26eb80 1 --2- 172.21.15.39:0/2926795736 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf476512d0 0x55cf4764b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:17.088 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.069+0000 7ff35f26eb80 1 -- 172.21.15.39:0/2926795736 >> 172.21.15.39:0/2926795736 conn(0x55cf47640be0 msgr2=0x55cf47645f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:53:17.088 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.070+0000 7ff35f26eb80 1 -- 172.21.15.39:0/2926795736 shutdown_connections 2023-07-21T18:53:17.088 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.070+0000 7ff35f26eb80 1 -- 172.21.15.39:0/2926795736 wait complete. 2023-07-21T18:53:17.088 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.070+0000 7ff35f26eb80 1 -- start start 2023-07-21T18:53:17.088 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.070+0000 7ff35f26eb80 1 Processor -- start 2023-07-21T18:53:17.088 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.070+0000 7ff35f26eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf474a76b0 0x55cf4765a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:17.091 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.070+0000 7ff35f26eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf4764b580 0x55cf4765a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:17.091 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.071+0000 7ff35f26eb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf476512d0 0x55cf47654780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:17.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.071+0000 7ff35f26eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf4753ca60 con 0x55cf474a76b0 2023-07-21T18:53:17.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.071+0000 7ff35f26eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf4753c760 con 0x55cf4764b580 2023-07-21T18:53:17.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.071+0000 7ff35f26eb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf474208c0 con 0x55cf476512d0 2023-07-21T18:53:17.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.071+0000 7ff35f26eb80 10 client.0 _ll_register_callbacks cb 0x55cf4753a700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:53:17.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.071+0000 7ff350e84700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf474a76b0 0x55cf4765a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:17.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.071+0000 7ff350e84700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf474a76b0 0x55cf4765a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60964/0 (socket says 172.21.15.39:60964) 2023-07-21T18:53:17.094 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.071+0000 7ff350e84700 1 -- 172.21.15.39:0/1166676738 learned_addr learned my addr 172.21.15.39:0/1166676738 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:53:17.094 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.071+0000 7ff34bfff700 1 --2- 172.21.15.39:0/1166676738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf4764b580 0x55cf4765a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:17.095 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.071+0000 7ff351685700 1 --2- 172.21.15.39:0/1166676738 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf476512d0 0x55cf47654780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:17.095 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.071+0000 7ff34bfff700 1 -- 172.21.15.39:0/1166676738 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf476512d0 msgr2=0x55cf47654780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:53:17.095 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.071+0000 7ff34bfff700 1 --2- 172.21.15.39:0/1166676738 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf476512d0 0x55cf47654780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:17.095 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.071+0000 7ff34bfff700 1 -- 172.21.15.39:0/1166676738 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf474a76b0 msgr2=0x55cf4765a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:53:17.095 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.071+0000 7ff34bfff700 1 --2- 172.21.15.39:0/1166676738 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf474a76b0 0x55cf4765a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:17.095 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.071+0000 7ff34bfff700 1 -- 172.21.15.39:0/1166676738 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff33c0098e0 con 0x55cf4764b580 2023-07-21T18:53:17.095 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.071+0000 7ff350e84700 1 --2- 172.21.15.39:0/1166676738 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf474a76b0 0x55cf4765a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:53:17.095 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.071+0000 7ff34bfff700 1 --2- 172.21.15.39:0/1166676738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf4764b580 0x55cf4765a9c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff34c199740 tx=0x7ff33c012c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba72c9672240fadc server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:53:17.096 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.071+0000 7ff32bfff700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-21T18:53:17.096 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.072+0000 7ff32bfff700 1 -- 172.21.15.39:0/1166676738 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33c017070 con 0x55cf4764b580 2023-07-21T18:53:17.096 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.072+0000 7ff35f26eb80 1 -- 172.21.15.39:0/1166676738 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cf474f6490 con 0x55cf4764b580 2023-07-21T18:53:17.096 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.072+0000 7ff32bfff700 1 -- 172.21.15.39:0/1166676738 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8527+0+0 (secure 0 0 0) 0x7ff33c004030 con 0x55cf4764b580 2023-07-21T18:53:17.096 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.072+0000 7ff32bfff700 1 -- 172.21.15.39:0/1166676738 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff33c0041f0 con 0x55cf4764b580 2023-07-21T18:53:17.096 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.072+0000 7ff35f26eb80 10 client.26489 fetch_fsmap learned FSMap version 68 2023-07-21T18:53:17.096 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.072+0000 7ff35f26eb80 1 -- 172.21.15.39:0/1166676738 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) v3 -- 0x7ff34402c070 con 0x55cf4764b580 2023-07-21T18:53:17.097 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.072+0000 7ff32bfff700 1 -- 172.21.15.39:0/1166676738 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cf474f6490 con 0x55cf4764b580 2023-07-21T18:53:17.097 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.072+0000 7ff32bfff700 1 -- 172.21.15.39:0/1166676738 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 68) v1 ==== 7474+0+0 (secure 0 0 0) 0x7ff33c00f5a0 con 0x55cf4764b580 2023-07-21T18:53:17.097 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.072+0000 7ff35f26eb80 10 client.26489 fetch_fsmap finished waiting for FSMap version 68 2023-07-21T18:53:17.097 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.072+0000 7ff35f26eb80 10 client.26489 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:53:17.097 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.072+0000 7ff35f26eb80 20 client.26489 populate_metadata read hostname 'smithi039' 2023-07-21T18:53:17.097 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.072+0000 7ff35f26eb80 1 --2- 172.21.15.39:0/1166676738 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55cf4766d140 0x55cf4766f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:17.097 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.072+0000 7ff35f26eb80 4 client.26489 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:53:17.097 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.072+0000 7ff35f26eb80 1 -- 172.21.15.39:0/1166676738 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cf47595e50 con 0x55cf4766d140 2023-07-21T18:53:17.098 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.073+0000 7ff350e84700 1 --2- 172.21.15.39:0/1166676738 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55cf4766d140 0x55cf4766f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:17.098 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.073+0000 7ff350e84700 1 --2- 172.21.15.39:0/1166676738 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55cf4766d140 0x55cf4766f550 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=768581dc940d210e server_cookie=8c83a4b0fb5689d7 in_seq=0 out_seq=0 2023-07-21T18:53:17.098 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.073+0000 7ff32bfff700 10 client.26489 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:53:17.098 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.075+0000 7ff32bfff700 1 -- 172.21.15.39:0/1166676738 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55cf47595e50 con 0x55cf4766d140 2023-07-21T18:53:17.098 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.075+0000 7ff32bfff700 10 client.26489 handle_command_reply: tid=0 2023-07-21T18:53:17.098 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.075+0000 7ff35f26eb80 1 client.26489 shutdown 2023-07-21T18:53:17.098 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.075+0000 7ff35f26eb80 1 -- 172.21.15.39:0/1166676738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf4764b580 msgr2=0x55cf4765a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:53:17.099 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.075+0000 7ff35f26eb80 1 --2- 172.21.15.39:0/1166676738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf4764b580 0x55cf4765a9c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff34c199740 tx=0x7ff33c012c30 comp rx=0 tx=0).stop 2023-07-21T18:53:17.099 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.075+0000 7ff35f26eb80 1 -- 172.21.15.39:0/1166676738 shutdown_connections 2023-07-21T18:53:17.099 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.075+0000 7ff35f26eb80 1 --2- 172.21.15.39:0/1166676738 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55cf4766d140 0x55cf4766f550 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:17.099 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.075+0000 7ff35f26eb80 1 --2- 172.21.15.39:0/1166676738 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf476512d0 0x55cf47654780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:17.099 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.075+0000 7ff32bfff700 0 client.26489 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:53:17.099 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.075+0000 7ff35f26eb80 1 --2- 172.21.15.39:0/1166676738 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf474a76b0 0x55cf4765a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:17.099 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.075+0000 7ff35f26eb80 1 --2- 172.21.15.39:0/1166676738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf4764b580 0x55cf4765a9c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:17.099 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.075+0000 7ff35f26eb80 1 -- 172.21.15.39:0/1166676738 >> 172.21.15.39:0/1166676738 conn(0x55cf47640be0 msgr2=0x55cf476454e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:53:17.100 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.075+0000 7ff35f26eb80 1 -- 172.21.15.39:0/1166676738 shutdown_connections 2023-07-21T18:53:17.100 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.075+0000 7ff35f26eb80 1 -- 172.21.15.39:0/1166676738 wait complete. 2023-07-21T18:53:17.100 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.076+0000 7ff35f26eb80 20 client.26489 trim_cache size 0 max 16384 2023-07-21T18:53:17.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.092+0000 7ff35f26eb80 1 Processor -- start 2023-07-21T18:53:17.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.092+0000 7ff35f26eb80 1 -- start start 2023-07-21T18:53:17.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.093+0000 7ff35f26eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf474a76b0 0x55cf4764b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:17.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.093+0000 7ff35f26eb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf4764b580 0x55cf476548b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:17.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.093+0000 7ff35f26eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf476512d0 0x55cf476501c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:17.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.093+0000 7ff35f26eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf47595e80 con 0x55cf474a76b0 2023-07-21T18:53:17.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.093+0000 7ff35f26eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf47646d80 con 0x55cf476512d0 2023-07-21T18:53:17.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.093+0000 7ff35f26eb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf47650a60 con 0x55cf4764b580 2023-07-21T18:53:17.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.093+0000 7ff350e84700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf474a76b0 0x55cf4764b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:17.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.093+0000 7ff350e84700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf474a76b0 0x55cf4764b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60972/0 (socket says 172.21.15.39:60972) 2023-07-21T18:53:17.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.093+0000 7ff350e84700 1 -- 172.21.15.39:0/3999464993 learned_addr learned my addr 172.21.15.39:0/3999464993 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:53:17.109 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.093+0000 7ff34bfff700 1 --2- 172.21.15.39:0/3999464993 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf4764b580 0x55cf476548b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:17.109 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.093+0000 7ff351685700 1 --2- 172.21.15.39:0/3999464993 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf476512d0 0x55cf476501c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:17.109 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.093+0000 7ff34bfff700 1 -- 172.21.15.39:0/3999464993 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf476512d0 msgr2=0x55cf476501c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:53:17.109 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.093+0000 7ff34bfff700 1 --2- 172.21.15.39:0/3999464993 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf476512d0 0x55cf476501c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:17.109 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.093+0000 7ff34bfff700 1 -- 172.21.15.39:0/3999464993 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf474a76b0 msgr2=0x55cf4764b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:53:17.109 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.093+0000 7ff34bfff700 1 --2- 172.21.15.39:0/3999464993 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf474a76b0 0x55cf4764b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:17.109 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.093+0000 7ff34bfff700 1 -- 172.21.15.39:0/3999464993 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cf4764bc40 con 0x55cf4764b580 2023-07-21T18:53:17.109 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.093+0000 7ff350e84700 1 --2- 172.21.15.39:0/3999464993 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf474a76b0 0x55cf4764b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:53:17.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.093+0000 7ff34bfff700 1 --2- 172.21.15.39:0/3999464993 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf4764b580 0x55cf476548b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff33c000c00 tx=0x7ff33c00f3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf4fe3e7cebafe34 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:53:17.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.094+0000 7ff3437fe700 1 -- 172.21.15.39:0/3999464993 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33c02a070 con 0x55cf4764b580 2023-07-21T18:53:17.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.094+0000 7ff3437fe700 1 -- 172.21.15.39:0/3999464993 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff33c00fde0 con 0x55cf4764b580 2023-07-21T18:53:17.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.094+0000 7ff3437fe700 1 -- 172.21.15.39:0/3999464993 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33c017040 con 0x55cf4764b580 2023-07-21T18:53:17.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.094+0000 7ff35f26eb80 1 -- 172.21.15.39:0/3999464993 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf4764b580 msgr2=0x55cf476548b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:53:17.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.094+0000 7ff35f26eb80 1 --2- 172.21.15.39:0/3999464993 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf4764b580 0x55cf476548b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff33c000c00 tx=0x7ff33c00f3c0 comp rx=0 tx=0).stop 2023-07-21T18:53:17.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.094+0000 7ff35f26eb80 1 -- 172.21.15.39:0/3999464993 shutdown_connections 2023-07-21T18:53:17.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.094+0000 7ff35f26eb80 1 --2- 172.21.15.39:0/3999464993 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf476512d0 0x55cf476501c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:17.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.094+0000 7ff35f26eb80 1 --2- 172.21.15.39:0/3999464993 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf474a76b0 0x55cf4764b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:17.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.094+0000 7ff35f26eb80 1 --2- 172.21.15.39:0/3999464993 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf4764b580 0x55cf476548b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:17.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.094+0000 7ff35f26eb80 1 -- 172.21.15.39:0/3999464993 >> 172.21.15.39:0/3999464993 conn(0x55cf47640be0 msgr2=0x55cf4764fab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:53:17.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.094+0000 7ff35f26eb80 1 -- 172.21.15.39:0/3999464993 shutdown_connections 2023-07-21T18:53:17.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.094+0000 7ff35f26eb80 1 -- 172.21.15.39:0/3999464993 wait complete. 2023-07-21T18:53:17.112 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.095+0000 7ff35f26eb80 1 -- start start 2023-07-21T18:53:17.112 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.095+0000 7ff35f26eb80 1 Processor -- start 2023-07-21T18:53:17.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.095+0000 7ff35f26eb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf474a76b0 0x55cf476b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:17.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.095+0000 7ff35f26eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf4764b580 0x55cf476b54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:17.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.095+0000 7ff35f26eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf476512d0 0x55cf476b5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:17.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.095+0000 7ff35f26eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf476c00f0 con 0x55cf476512d0 2023-07-21T18:53:17.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.095+0000 7ff35f26eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf476c0230 con 0x55cf4764b580 2023-07-21T18:53:17.115 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.095+0000 7ff35f26eb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf476c03f0 con 0x55cf474a76b0 2023-07-21T18:53:17.115 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.095+0000 7ff35f26eb80 10 client.0 _ll_register_callbacks cb 0x7ff33c012450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:53:17.115 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.095+0000 7ff351685700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf476512d0 0x55cf476b5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:17.115 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.095+0000 7ff351685700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf476512d0 0x55cf476b5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60980/0 (socket says 172.21.15.39:60980) 2023-07-21T18:53:17.115 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.095+0000 7ff351685700 1 -- 172.21.15.39:0/2812004174 learned_addr learned my addr 172.21.15.39:0/2812004174 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:53:17.115 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.095+0000 7ff34bfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf4764b580 0x55cf476b54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:17.115 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.095+0000 7ff350e84700 1 --2- 172.21.15.39:0/2812004174 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf474a76b0 0x55cf476b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:17.116 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.096+0000 7ff34bfff700 1 -- 172.21.15.39:0/2812004174 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf474a76b0 msgr2=0x55cf476b4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:53:17.116 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.096+0000 7ff34bfff700 1 --2- 172.21.15.39:0/2812004174 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf474a76b0 0x55cf476b4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:17.116 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.096+0000 7ff34bfff700 1 -- 172.21.15.39:0/2812004174 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf476512d0 msgr2=0x55cf476b5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:53:17.116 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.096+0000 7ff34bfff700 1 --2- 172.21.15.39:0/2812004174 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf476512d0 0x55cf476b5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:17.116 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.096+0000 7ff34bfff700 1 -- 172.21.15.39:0/2812004174 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff334017070 con 0x55cf4764b580 2023-07-21T18:53:17.116 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.096+0000 7ff350e84700 1 --2- 172.21.15.39:0/2812004174 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf474a76b0 0x55cf476b4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:53:17.116 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.096+0000 7ff351685700 1 --2- 172.21.15.39:0/2812004174 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf476512d0 0x55cf476b5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:53:17.117 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.096+0000 7ff34bfff700 1 --2- 172.21.15.39:0/2812004174 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf4764b580 0x55cf476b54d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff33c0126a0 tx=0x7ff33c005750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb312d7d6681bfb1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:53:17.117 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.096+0000 7ff32bfff700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-21T18:53:17.117 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.096+0000 7ff32bfff700 1 -- 172.21.15.39:0/2812004174 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33c02a070 con 0x55cf4764b580 2023-07-21T18:53:17.117 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.096+0000 7ff35f26eb80 1 -- 172.21.15.39:0/2812004174 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff34409ada0 con 0x55cf4764b580 2023-07-21T18:53:17.117 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.096+0000 7ff32bfff700 1 -- 172.21.15.39:0/2812004174 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8527+0+0 (secure 0 0 0) 0x7ff33c003680 con 0x55cf4764b580 2023-07-21T18:53:17.117 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.096+0000 7ff32bfff700 1 -- 172.21.15.39:0/2812004174 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff33c017030 con 0x55cf4764b580 2023-07-21T18:53:17.117 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.096+0000 7ff35f26eb80 10 client.26495 fetch_fsmap learned FSMap version 68 2023-07-21T18:53:17.117 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.096+0000 7ff35f26eb80 1 -- 172.21.15.39:0/2812004174 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) v3 -- 0x7ff34402c070 con 0x55cf4764b580 2023-07-21T18:53:17.118 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.097+0000 7ff32bfff700 1 -- 172.21.15.39:0/2812004174 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff33c03b080 con 0x55cf4764b580 2023-07-21T18:53:17.118 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.097+0000 7ff32bfff700 1 -- 172.21.15.39:0/2812004174 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 68) v1 ==== 7474+0+0 (secure 0 0 0) 0x7ff33c025890 con 0x55cf4764b580 2023-07-21T18:53:17.118 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.097+0000 7ff35f26eb80 10 client.26495 fetch_fsmap finished waiting for FSMap version 68 2023-07-21T18:53:17.118 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.097+0000 7ff35f26eb80 10 client.26495 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:53:17.118 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.097+0000 7ff35f26eb80 20 client.26495 populate_metadata read hostname 'smithi039' 2023-07-21T18:53:17.118 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.097+0000 7ff35f26eb80 1 --2- 172.21.15.39:0/2812004174 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55cf476c1810 0x55cf476c3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:17.118 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.097+0000 7ff35f26eb80 4 client.26495 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:53:17.118 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.097+0000 7ff35f26eb80 1 -- 172.21.15.39:0/2812004174 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55cf47557ff0 con 0x55cf476c1810 2023-07-21T18:53:17.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.097+0000 7ff350e84700 1 --2- 172.21.15.39:0/2812004174 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55cf476c1810 0x55cf476c3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:17.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.097+0000 7ff350e84700 1 --2- 172.21.15.39:0/2812004174 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55cf476c1810 0x55cf476c3bf0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29ff8c5be8b254f9 server_cookie=a573c0780ef5c197 in_seq=0 out_seq=0 2023-07-21T18:53:17.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.097+0000 7ff32bfff700 10 client.26495 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:53:17.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.136+0000 7ff32bfff700 1 -- 172.21.15.39:0/2812004174 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55cf47557ff0 con 0x55cf476c1810 2023-07-21T18:53:17.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.136+0000 7ff32bfff700 10 client.26495 handle_command_reply: tid=0 2023-07-21T18:53:17.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.136+0000 7ff35f26eb80 1 client.26495 shutdown 2023-07-21T18:53:17.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.136+0000 7ff35f26eb80 1 -- 172.21.15.39:0/2812004174 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf4764b580 msgr2=0x55cf476b54d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:53:17.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.136+0000 7ff35f26eb80 1 --2- 172.21.15.39:0/2812004174 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf4764b580 0x55cf476b54d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff33c0126a0 tx=0x7ff33c005750 comp rx=0 tx=0).stop 2023-07-21T18:53:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.136+0000 7ff35f26eb80 1 -- 172.21.15.39:0/2812004174 shutdown_connections 2023-07-21T18:53:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.136+0000 7ff35f26eb80 1 --2- 172.21.15.39:0/2812004174 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55cf476c1810 0x55cf476c3bf0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.136+0000 7ff35f26eb80 1 --2- 172.21.15.39:0/2812004174 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf476512d0 0x55cf476b5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.136+0000 7ff32bfff700 0 client.26495 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:53:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.136+0000 7ff35f26eb80 1 --2- 172.21.15.39:0/2812004174 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf474a76b0 0x55cf476b4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.136+0000 7ff35f26eb80 1 --2- 172.21.15.39:0/2812004174 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf4764b580 0x55cf476b54d0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.136+0000 7ff35f26eb80 1 -- 172.21.15.39:0/2812004174 >> 172.21.15.39:0/2812004174 conn(0x55cf47640be0 msgr2=0x55cf47650340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:53:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.136+0000 7ff35f26eb80 1 -- 172.21.15.39:0/2812004174 shutdown_connections 2023-07-21T18:53:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.136+0000 7ff35f26eb80 1 -- 172.21.15.39:0/2812004174 wait complete. 2023-07-21T18:53:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.136+0000 7ff35f26eb80 20 client.26495 trim_cache size 0 max 16384 2023-07-21T18:53:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:53:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:53:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:53:17.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:53:17.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.137+0000 7ff3530e8700 1 -- 172.21.15.39:0/3593897569 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7ff3380a12c0 msgr2=0x7ff3380a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:53:17.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.137+0000 7ff3530e8700 1 --2- 172.21.15.39:0/3593897569 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7ff3380a12c0 0x7ff3380a3770 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff334009c60 tx=0x7ff334009260 comp rx=0 tx=0).stop 2023-07-21T18:53:17.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.137+0000 7ff3530e8700 1 -- 172.21.15.39:0/3593897569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff34c10c540 msgr2=0x7ff34c19cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:53:17.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.137+0000 7ff3530e8700 1 --2- 172.21.15.39:0/3593897569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff34c10c540 0x7ff34c19cad0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7ff344009fd0 tx=0x7ff34400bce0 comp rx=0 tx=0).stop 2023-07-21T18:53:17.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.137+0000 7ff3530e8700 1 -- 172.21.15.39:0/3593897569 shutdown_connections 2023-07-21T18:53:17.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.137+0000 7ff3530e8700 1 --2- 172.21.15.39:0/3593897569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff34c10c540 0x7ff34c19cad0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:17.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.137+0000 7ff3530e8700 1 --2- 172.21.15.39:0/3593897569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff34c0679a0 0x7ff34c197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:17.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.137+0000 7ff3530e8700 1 --2- 172.21.15.39:0/3593897569 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7ff3380a12c0 0x7ff3380a3770 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:17.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.137+0000 7ff3530e8700 1 --2- 172.21.15.39:0/3593897569 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff34c0682b0 0x7ff34c198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:17.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.137+0000 7ff3530e8700 1 -- 172.21.15.39:0/3593897569 >> 172.21.15.39:0/3593897569 conn(0x7ff34c074760 msgr2=0x7ff34c0fc5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:53:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.137+0000 7ff3530e8700 1 -- 172.21.15.39:0/3593897569 shutdown_connections 2023-07-21T18:53:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:17.137+0000 7ff3530e8700 1 -- 172.21.15.39:0/3593897569 wait complete. 2023-07-21T18:53:17.291 INFO:tasks.workunit.client.0.smithi039.stdout: 1 659591 5.38 MB/sec execute 510 sec latency 2638.032 ms 2023-07-21T18:53:18.291 INFO:tasks.workunit.client.0.smithi039.stdout: 1 661036 5.39 MB/sec execute 511 sec latency 17.862 ms 2023-07-21T18:53:18.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:53:18 smithi193 ceph-mon[98204]: pgmap v362: 97 pgs: 97 active+clean; 270 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 1.4 MiB/s wr, 140 op/s 2023-07-21T18:53:18.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:53:18 smithi155 ceph-mon[97934]: pgmap v362: 97 pgs: 97 active+clean; 270 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 1.4 MiB/s wr, 140 op/s 2023-07-21T18:53:18.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:53:18 smithi039 ceph-mon[96489]: pgmap v362: 97 pgs: 97 active+clean; 270 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 1.4 MiB/s wr, 140 op/s 2023-07-21T18:53:19.291 INFO:tasks.workunit.client.0.smithi039.stdout: 1 662222 5.39 MB/sec execute 512 sec latency 211.891 ms 2023-07-21T18:53:20.291 INFO:tasks.workunit.client.0.smithi039.stdout: 1 662222 5.38 MB/sec execute 513 sec latency 1211.947 ms 2023-07-21T18:53:20.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:53:20 smithi193 ceph-mon[98204]: pgmap v363: 97 pgs: 97 active+clean; 270 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 1.4 MiB/s wr, 140 op/s 2023-07-21T18:53:20.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:53:20 smithi155 ceph-mon[97934]: pgmap v363: 97 pgs: 97 active+clean; 270 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 1.4 MiB/s wr, 140 op/s 2023-07-21T18:53:20.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:53:20 smithi039 ceph-mon[96489]: pgmap v363: 97 pgs: 97 active+clean; 270 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 1.4 MiB/s wr, 140 op/s 2023-07-21T18:53:21.291 INFO:tasks.workunit.client.0.smithi039.stdout: 1 662222 5.37 MB/sec execute 514 sec latency 2212.020 ms 2023-07-21T18:53:22.291 INFO:tasks.workunit.client.0.smithi039.stdout: 1 662999 5.37 MB/sec execute 515 sec latency 2641.406 ms 2023-07-21T18:53:22.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:53:22 smithi193 ceph-mon[98204]: pgmap v364: 97 pgs: 97 active+clean; 270 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 1.8 MiB/s wr, 153 op/s 2023-07-21T18:53:22.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:53:22 smithi155 ceph-mon[97934]: pgmap v364: 97 pgs: 97 active+clean; 270 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 1.8 MiB/s wr, 153 op/s 2023-07-21T18:53:22.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:53:22 smithi039 ceph-mon[96489]: pgmap v364: 97 pgs: 97 active+clean; 270 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 1.8 MiB/s wr, 153 op/s 2023-07-21T18:53:23.291 INFO:tasks.workunit.client.0.smithi039.stdout: 1 664500 5.38 MB/sec execute 516 sec latency 39.956 ms 2023-07-21T18:53:24.291 INFO:tasks.workunit.client.0.smithi039.stdout: 1 665765 5.38 MB/sec execute 517 sec latency 150.564 ms 2023-07-21T18:53:24.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:53:24 smithi193 ceph-mon[98204]: pgmap v365: 97 pgs: 97 active+clean; 270 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 1.6 MiB/s wr, 136 op/s 2023-07-21T18:53:24.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:53:24 smithi155 ceph-mon[97934]: pgmap v365: 97 pgs: 97 active+clean; 270 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 1.6 MiB/s wr, 136 op/s 2023-07-21T18:53:24.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:53:24 smithi039 ceph-mon[96489]: pgmap v365: 97 pgs: 97 active+clean; 270 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 1.6 MiB/s wr, 136 op/s 2023-07-21T18:53:25.291 INFO:tasks.workunit.client.0.smithi039.stdout: 1 665765 5.37 MB/sec execute 518 sec latency 1150.615 ms 2023-07-21T18:53:26.291 INFO:tasks.workunit.client.0.smithi039.stdout: 1 666370 5.36 MB/sec execute 519 sec latency 1946.519 ms 2023-07-21T18:53:26.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:53:26 smithi193 ceph-mon[98204]: pgmap v366: 97 pgs: 97 active+clean; 270 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 1.5 MiB/s wr, 127 op/s 2023-07-21T18:53:26.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:53:26 smithi155 ceph-mon[97934]: pgmap v366: 97 pgs: 97 active+clean; 270 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 1.5 MiB/s wr, 127 op/s 2023-07-21T18:53:26.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:53:26 smithi039 ceph-mon[96489]: pgmap v366: 97 pgs: 97 active+clean; 270 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 1.5 MiB/s wr, 127 op/s 2023-07-21T18:53:27.291 INFO:tasks.workunit.client.0.smithi039.stdout: 1 667933 5.37 MB/sec execute 520 sec latency 22.462 ms 2023-07-21T18:53:28.291 INFO:tasks.workunit.client.0.smithi039.stdout: 1 669271 5.37 MB/sec execute 521 sec latency 23.785 ms 2023-07-21T18:53:28.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:53:28 smithi193 ceph-mon[98204]: pgmap v367: 97 pgs: 97 active+clean; 273 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 930 KiB/s rd, 1.9 MiB/s wr, 183 op/s 2023-07-21T18:53:28.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:53:28 smithi155 ceph-mon[97934]: pgmap v367: 97 pgs: 97 active+clean; 273 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 930 KiB/s rd, 1.9 MiB/s wr, 183 op/s 2023-07-21T18:53:28.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:53:28 smithi039 ceph-mon[96489]: pgmap v367: 97 pgs: 97 active+clean; 273 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 930 KiB/s rd, 1.9 MiB/s wr, 183 op/s 2023-07-21T18:53:29.291 INFO:tasks.workunit.client.0.smithi039.stdout: 1 669314 5.36 MB/sec execute 522 sec latency 961.516 ms 2023-07-21T18:53:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.264+0000 7f5948bee700 1 -- 172.21.15.39:0/910574466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59440ff3e0 msgr2=0x7f594410b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:53:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.264+0000 7f5948bee700 1 --2- 172.21.15.39:0/910574466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59440ff3e0 0x7f594410b860 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f593c00b0a0 tx=0x7f593c00b3b0 comp rx=0 tx=0).stop 2023-07-21T18:53:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.264+0000 7f5948bee700 1 -- 172.21.15.39:0/910574466 shutdown_connections 2023-07-21T18:53:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.264+0000 7f5948bee700 1 --2- 172.21.15.39:0/910574466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59440ff3e0 0x7f594410b860 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.264+0000 7f5948bee700 1 --2- 172.21.15.39:0/910574466 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5944104a30 0x7f5944104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.264+0000 7f5948bee700 1 --2- 172.21.15.39:0/910574466 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59440fea30 0x7f59440feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.264+0000 7f5948bee700 1 -- 172.21.15.39:0/910574466 >> 172.21.15.39:0/910574466 conn(0x7f5944074760 msgr2=0x7f5944074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:53:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.264+0000 7f5948bee700 1 -- 172.21.15.39:0/910574466 shutdown_connections 2023-07-21T18:53:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.264+0000 7f5948bee700 1 -- 172.21.15.39:0/910574466 wait complete. 2023-07-21T18:53:30.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.265+0000 7f5948bee700 1 Processor -- start 2023-07-21T18:53:30.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.265+0000 7f5948bee700 1 -- start start 2023-07-21T18:53:30.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.265+0000 7f5948bee700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f59440fea30 0x7f5944197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:30.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.265+0000 7f5948bee700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59440ff3e0 0x7f5944198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:30.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.265+0000 7f5948bee700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5944104a30 0x7f594419ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:30.280 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.265+0000 7f5948bee700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f594419d0d0 con 0x7f59440ff3e0 2023-07-21T18:53:30.280 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.265+0000 7f5948bee700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f594419d240 con 0x7f5944104a30 2023-07-21T18:53:30.280 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.265+0000 7f5948bee700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f594419d400 con 0x7f59440fea30 2023-07-21T18:53:30.280 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.266+0000 7f593bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59440ff3e0 0x7f5944198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:30.280 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.266+0000 7f593bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59440ff3e0 0x7f5944198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60994/0 (socket says 172.21.15.39:60994) 2023-07-21T18:53:30.280 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.266+0000 7f593bfff700 1 -- 172.21.15.39:0/2643269762 learned_addr learned my addr 172.21.15.39:0/2643269762 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:53:30.281 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.266+0000 7f594259c700 1 --2- 172.21.15.39:0/2643269762 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f59440fea30 0x7f5944197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:30.281 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.266+0000 7f5942d9d700 1 --2- 172.21.15.39:0/2643269762 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5944104a30 0x7f594419ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:30.281 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.266+0000 7f593bfff700 1 -- 172.21.15.39:0/2643269762 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f59440fea30 msgr2=0x7f5944197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:53:30.281 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.266+0000 7f593bfff700 1 --2- 172.21.15.39:0/2643269762 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f59440fea30 0x7f5944197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:30.281 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.266+0000 7f593bfff700 1 -- 172.21.15.39:0/2643269762 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5944104a30 msgr2=0x7f594419ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:53:30.281 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.266+0000 7f593bfff700 1 --2- 172.21.15.39:0/2643269762 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5944104a30 0x7f594419ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:30.281 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.266+0000 7f593bfff700 1 -- 172.21.15.39:0/2643269762 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f593c009d00 con 0x7f59440ff3e0 2023-07-21T18:53:30.282 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.266+0000 7f593bfff700 1 --2- 172.21.15.39:0/2643269762 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59440ff3e0 0x7f5944198420 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f592c00e9a0 tx=0x7f592c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:53:30.282 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.266+0000 7f593b7fe700 1 -- 172.21.15.39:0/2643269762 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f592c009790 con 0x7f59440ff3e0 2023-07-21T18:53:30.282 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.266+0000 7f593b7fe700 1 -- 172.21.15.39:0/2643269762 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f592c010430 con 0x7f59440ff3e0 2023-07-21T18:53:30.282 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.266+0000 7f5948bee700 1 -- 172.21.15.39:0/2643269762 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f594419d6e0 con 0x7f59440ff3e0 2023-07-21T18:53:30.282 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.266+0000 7f593b7fe700 1 -- 172.21.15.39:0/2643269762 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f592c018680 con 0x7f59440ff3e0 2023-07-21T18:53:30.282 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.266+0000 7f5948bee700 1 -- 172.21.15.39:0/2643269762 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59441a4f20 con 0x7f59440ff3e0 2023-07-21T18:53:30.287 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.269+0000 7f593b7fe700 1 -- 172.21.15.39:0/2643269762 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f592c0105a0 con 0x7f59440ff3e0 2023-07-21T18:53:30.287 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.269+0000 7f593b7fe700 1 --2- 172.21.15.39:0/2643269762 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f59300a12c0 0x7f59300a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:30.287 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.270+0000 7f594259c700 1 --2- 172.21.15.39:0/2643269762 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f59300a12c0 0x7f59300a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:30.287 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.270+0000 7f593b7fe700 1 -- 172.21.15.39:0/2643269762 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f592c014070 con 0x7f59440ff3e0 2023-07-21T18:53:30.288 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.270+0000 7f594259c700 1 --2- 172.21.15.39:0/2643269762 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f59300a12c0 0x7f59300a3770 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f59440ffba0 tx=0x7f593400b410 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:53:30.288 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.270+0000 7f5954d74b80 1 Processor -- start 2023-07-21T18:53:30.288 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.271+0000 7f5954d74b80 1 -- start start 2023-07-21T18:53:30.288 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.271+0000 7f5954d74b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610868536b0 0x5610869fcd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:30.288 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.271+0000 7f5954d74b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5610869fd2d0 0x5610869f7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:30.288 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.271+0000 7f5954d74b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5610869f7580 0x5610869f79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:30.289 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.271+0000 7f5954d74b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610868fcc50 con 0x5610869f7580 2023-07-21T18:53:30.289 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.271+0000 7f5954d74b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56108694f9e0 con 0x5610869fd2d0 2023-07-21T18:53:30.289 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.271+0000 7f5954d74b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x561086948e70 con 0x5610868536b0 2023-07-21T18:53:30.289 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.271+0000 7f5942d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5610869f7580 0x5610869f79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:30.289 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.271+0000 7f5942d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5610869f7580 0x5610869f79d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:32772/0 (socket says 172.21.15.39:32772) 2023-07-21T18:53:30.289 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.271+0000 7f5942d9d700 1 -- 172.21.15.39:0/2769161674 learned_addr learned my addr 172.21.15.39:0/2769161674 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:53:30.289 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.271+0000 7f594259c700 1 --2- 172.21.15.39:0/2769161674 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610868536b0 0x5610869fcd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:30.290 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.271+0000 7f593bfff700 1 --2- 172.21.15.39:0/2769161674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5610869fd2d0 0x5610869f7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:30.290 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.271+0000 7f5942d9d700 1 -- 172.21.15.39:0/2769161674 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610868536b0 msgr2=0x5610869fcd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:53:30.290 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.271+0000 7f5942d9d700 1 --2- 172.21.15.39:0/2769161674 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610868536b0 0x5610869fcd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:30.290 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.271+0000 7f5942d9d700 1 -- 172.21.15.39:0/2769161674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5610869fd2d0 msgr2=0x5610869f7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:53:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.271+0000 7f5942d9d700 1 --2- 172.21.15.39:0/2769161674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5610869fd2d0 0x5610869f7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.271+0000 7f5942d9d700 1 -- 172.21.15.39:0/2769161674 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561086776d80 con 0x5610869f7580 2023-07-21T18:53:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.272+0000 7f5942d9d700 1 --2- 172.21.15.39:0/2769161674 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5610869f7580 0x5610869f79d0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f593c015000 tx=0x7f593c009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f31226c28d188730 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:53:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.272+0000 7f5938ff9700 1 -- 172.21.15.39:0/2769161674 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593c00e040 con 0x5610869f7580 2023-07-21T18:53:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.272+0000 7f5938ff9700 1 -- 172.21.15.39:0/2769161674 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f593c00bcb0 con 0x5610869f7580 2023-07-21T18:53:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.272+0000 7f5938ff9700 1 -- 172.21.15.39:0/2769161674 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593c01c6a0 con 0x5610869f7580 2023-07-21T18:53:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.272+0000 7f5954d74b80 1 -- 172.21.15.39:0/2769161674 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5610869f7580 msgr2=0x5610869f79d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:53:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.272+0000 7f5954d74b80 1 --2- 172.21.15.39:0/2769161674 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5610869f7580 0x5610869f79d0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f593c015000 tx=0x7f593c009360 comp rx=0 tx=0).stop 2023-07-21T18:53:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.273+0000 7f5954d74b80 1 -- 172.21.15.39:0/2769161674 shutdown_connections 2023-07-21T18:53:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.273+0000 7f5954d74b80 1 --2- 172.21.15.39:0/2769161674 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5610869f7580 0x5610869f79d0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.273+0000 7f5954d74b80 1 --2- 172.21.15.39:0/2769161674 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610868536b0 0x5610869fcd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.273+0000 7f5954d74b80 1 --2- 172.21.15.39:0/2769161674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5610869fd2d0 0x5610869f7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.273+0000 7f5954d74b80 1 -- 172.21.15.39:0/2769161674 >> 172.21.15.39:0/2769161674 conn(0x5610869ecbe0 msgr2=0x5610869f1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:53:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.273+0000 7f5954d74b80 1 -- 172.21.15.39:0/2769161674 shutdown_connections 2023-07-21T18:53:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.273+0000 7f5954d74b80 1 -- 172.21.15.39:0/2769161674 wait complete. 2023-07-21T18:53:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.273+0000 7f5954d74b80 1 -- start start 2023-07-21T18:53:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.273+0000 7f5954d74b80 1 Processor -- start 2023-07-21T18:53:30.294 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.273+0000 7f5954d74b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5610868536b0 0x561086a03580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:30.294 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.273+0000 7f5954d74b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5610869f7580 0x561086a00800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:30.294 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.273+0000 7f5954d74b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610869fd2d0 0x561086a00e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:30.294 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.273+0000 7f5954d74b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610868e8a60 con 0x5610869f7580 2023-07-21T18:53:30.294 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.273+0000 7f5954d74b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610868e8760 con 0x5610868536b0 2023-07-21T18:53:30.294 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.273+0000 7f5954d74b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610867cc8c0 con 0x5610869fd2d0 2023-07-21T18:53:30.294 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.273+0000 7f5954d74b80 10 client.0 _ll_register_callbacks cb 0x5610868e6700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:53:30.295 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.274+0000 7f593bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5610869f7580 0x561086a00800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:30.295 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.274+0000 7f593bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5610869f7580 0x561086a00800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:32776/0 (socket says 172.21.15.39:32776) 2023-07-21T18:53:30.295 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.274+0000 7f593bfff700 1 -- 172.21.15.39:0/1248222776 learned_addr learned my addr 172.21.15.39:0/1248222776 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:53:30.295 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.274+0000 7f594259c700 1 --2- 172.21.15.39:0/1248222776 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5610868536b0 0x561086a03580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:30.295 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.274+0000 7f5942d9d700 1 --2- 172.21.15.39:0/1248222776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610869fd2d0 0x561086a00e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:30.295 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.274+0000 7f593bfff700 1 -- 172.21.15.39:0/1248222776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610869fd2d0 msgr2=0x561086a00e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:53:30.295 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.274+0000 7f593bfff700 1 --2- 172.21.15.39:0/1248222776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610869fd2d0 0x561086a00e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.274+0000 7f593bfff700 1 -- 172.21.15.39:0/1248222776 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5610868536b0 msgr2=0x561086a03580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:53:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.274+0000 7f593bfff700 1 --2- 172.21.15.39:0/1248222776 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5610868536b0 0x561086a03580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.274+0000 7f593bfff700 1 -- 172.21.15.39:0/1248222776 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f594410c160 con 0x5610869f7580 2023-07-21T18:53:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.274+0000 7f594259c700 1 --2- 172.21.15.39:0/1248222776 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5610868536b0 0x561086a03580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:53:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.274+0000 7f593bfff700 1 --2- 172.21.15.39:0/1248222776 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5610869f7580 0x561086a00800 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f592c096a20 tx=0x7f592c09dfe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68c796cc6bddf056 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:53:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.274+0000 7f5929ffb700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:53:30.297 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.274+0000 7f5929ffb700 1 -- 172.21.15.39:0/1248222776 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f592c0996a0 con 0x5610869f7580 2023-07-21T18:53:30.297 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.274+0000 7f5954d74b80 1 -- 172.21.15.39:0/1248222776 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5610868a2490 con 0x5610869f7580 2023-07-21T18:53:30.297 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.274+0000 7f5929ffb700 1 -- 172.21.15.39:0/1248222776 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f592c0a1f70 con 0x5610869f7580 2023-07-21T18:53:30.297 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.274+0000 7f5929ffb700 1 -- 172.21.15.39:0/1248222776 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f592c0a2130 con 0x5610869f7580 2023-07-21T18:53:30.297 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.275+0000 7f5954d74b80 10 client.17046 fetch_fsmap learned FSMap version 68 2023-07-21T18:53:30.297 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.275+0000 7f5954d74b80 1 -- 172.21.15.39:0/1248222776 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) v3 -- 0x56108675fc00 con 0x5610869f7580 2023-07-21T18:53:30.297 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.275+0000 7f5929ffb700 1 -- 172.21.15.39:0/1248222776 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f592c0b8030 con 0x5610869f7580 2023-07-21T18:53:30.298 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.275+0000 7f5929ffb700 1 -- 172.21.15.39:0/1248222776 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 68) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f592c098f10 con 0x5610869f7580 2023-07-21T18:53:30.298 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.275+0000 7f5954d74b80 10 client.17046 fetch_fsmap finished waiting for FSMap version 68 2023-07-21T18:53:30.298 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.275+0000 7f5954d74b80 10 client.17046 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:53:30.298 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.275+0000 7f5954d74b80 20 client.17046 populate_metadata read hostname 'smithi039' 2023-07-21T18:53:30.298 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.275+0000 7f5954d74b80 1 --2- 172.21.15.39:0/1248222776 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x561086a10a20 0x561086a12e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:30.298 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.275+0000 7f5954d74b80 4 client.17046 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:53:30.298 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.275+0000 7f5954d74b80 1 -- 172.21.15.39:0/1248222776 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561086941e50 con 0x561086a10a20 2023-07-21T18:53:30.299 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.275+0000 7f594259c700 1 --2- 172.21.15.39:0/1248222776 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x561086a10a20 0x561086a12e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:30.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.276+0000 7f594259c700 1 --2- 172.21.15.39:0/1248222776 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x561086a10a20 0x561086a12e30 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5630fa60fa49620c server_cookie=634224ba89ebbb78 in_seq=0 out_seq=0 2023-07-21T18:53:30.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.276+0000 7f5929ffb700 10 client.17046 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:53:30.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.276+0000 7f5929ffb700 1 -- 172.21.15.39:0/1248222776 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561086941e50 con 0x561086a10a20 2023-07-21T18:53:30.301 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.276+0000 7f5929ffb700 10 client.17046 handle_command_reply: tid=0 2023-07-21T18:53:30.301 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.276+0000 7f5954d74b80 1 client.17046 shutdown 2023-07-21T18:53:30.301 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.277+0000 7f5954d74b80 1 -- 172.21.15.39:0/1248222776 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5610869f7580 msgr2=0x561086a00800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:53:30.301 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.277+0000 7f5954d74b80 1 --2- 172.21.15.39:0/1248222776 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5610869f7580 0x561086a00800 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f592c096a20 tx=0x7f592c09dfe0 comp rx=0 tx=0).stop 2023-07-21T18:53:30.301 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.277+0000 7f5954d74b80 1 -- 172.21.15.39:0/1248222776 shutdown_connections 2023-07-21T18:53:30.302 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.277+0000 7f5954d74b80 1 --2- 172.21.15.39:0/1248222776 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x561086a10a20 0x561086a12e30 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:30.302 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.277+0000 7f5954d74b80 1 --2- 172.21.15.39:0/1248222776 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5610868536b0 0x561086a03580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:30.302 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.277+0000 7f5954d74b80 1 --2- 172.21.15.39:0/1248222776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610869fd2d0 0x561086a00e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:30.302 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.277+0000 7f5954d74b80 1 --2- 172.21.15.39:0/1248222776 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5610869f7580 0x561086a00800 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:30.302 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.277+0000 7f5929ffb700 0 client.17046 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:53:30.302 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.277+0000 7f5954d74b80 1 -- 172.21.15.39:0/1248222776 >> 172.21.15.39:0/1248222776 conn(0x5610869ecbe0 msgr2=0x5610869f0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:53:30.302 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.277+0000 7f5954d74b80 1 -- 172.21.15.39:0/1248222776 shutdown_connections 2023-07-21T18:53:30.303 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.277+0000 7f5954d74b80 1 -- 172.21.15.39:0/1248222776 wait complete. 2023-07-21T18:53:30.303 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.277+0000 7f5954d74b80 20 client.17046 trim_cache size 0 max 16384 2023-07-21T18:53:30.303 INFO:tasks.workunit.client.0.smithi039.stdout: 1 669314 5.35 MB/sec execute 523 sec latency 1961.569 ms 2023-07-21T18:53:30.305 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.292+0000 7f5954d74b80 1 Processor -- start 2023-07-21T18:53:30.305 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.292+0000 7f5954d74b80 1 -- start start 2023-07-21T18:53:30.305 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.292+0000 7f5954d74b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5610868536b0 0x5610869fe1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:30.305 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.293+0000 7f5954d74b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610869f7580 0x561086a03120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:30.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.293+0000 7f5954d74b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5610869fd2d0 0x5610869f1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:30.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.293+0000 7f5954d74b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x561086a01de0 con 0x5610869fd2d0 2023-07-21T18:53:30.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.293+0000 7f5954d74b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610869fc9d0 con 0x5610868536b0 2023-07-21T18:53:30.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.293+0000 7f5954d74b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x561086a01760 con 0x5610869f7580 2023-07-21T18:53:30.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.293+0000 7f5942d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5610869fd2d0 0x5610869f1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:30.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.293+0000 7f5942d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5610869fd2d0 0x5610869f1c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:32786/0 (socket says 172.21.15.39:32786) 2023-07-21T18:53:30.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.293+0000 7f5942d9d700 1 -- 172.21.15.39:0/676668361 learned_addr learned my addr 172.21.15.39:0/676668361 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:53:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.293+0000 7f594259c700 1 --2- 172.21.15.39:0/676668361 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5610868536b0 0x5610869fe1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.293+0000 7f593bfff700 1 --2- 172.21.15.39:0/676668361 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610869f7580 0x561086a03120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.293+0000 7f5942d9d700 1 -- 172.21.15.39:0/676668361 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610869f7580 msgr2=0x561086a03120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:53:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.293+0000 7f5942d9d700 1 --2- 172.21.15.39:0/676668361 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610869f7580 0x561086a03120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.293+0000 7f5942d9d700 1 -- 172.21.15.39:0/676668361 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5610868536b0 msgr2=0x5610869fe1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:53:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.293+0000 7f5942d9d700 1 --2- 172.21.15.39:0/676668361 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5610868536b0 0x5610869fe1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.293+0000 7f5942d9d700 1 -- 172.21.15.39:0/676668361 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561086a17a60 con 0x5610869fd2d0 2023-07-21T18:53:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.293+0000 7f5942d9d700 1 --2- 172.21.15.39:0/676668361 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5610869fd2d0 0x5610869f1c60 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f593c021000 tx=0x7f593c006000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5435a27b49a7872d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:53:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.293+0000 7f59397fa700 1 -- 172.21.15.39:0/676668361 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593c00e040 con 0x5610869fd2d0 2023-07-21T18:53:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.293+0000 7f59397fa700 1 -- 172.21.15.39:0/676668361 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f593c0074a0 con 0x5610869fd2d0 2023-07-21T18:53:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.293+0000 7f59397fa700 1 -- 172.21.15.39:0/676668361 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593c005470 con 0x5610869fd2d0 2023-07-21T18:53:30.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.294+0000 7f5954d74b80 1 -- 172.21.15.39:0/676668361 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5610869fd2d0 msgr2=0x5610869f1c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:53:30.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.294+0000 7f5954d74b80 1 --2- 172.21.15.39:0/676668361 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5610869fd2d0 0x5610869f1c60 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f593c021000 tx=0x7f593c006000 comp rx=0 tx=0).stop 2023-07-21T18:53:30.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.294+0000 7f5954d74b80 1 -- 172.21.15.39:0/676668361 shutdown_connections 2023-07-21T18:53:30.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.294+0000 7f5954d74b80 1 --2- 172.21.15.39:0/676668361 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5610868536b0 0x5610869fe1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:30.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.294+0000 7f5954d74b80 1 --2- 172.21.15.39:0/676668361 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5610869fd2d0 0x5610869f1c60 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:30.312 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.294+0000 7f5954d74b80 1 --2- 172.21.15.39:0/676668361 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610869f7580 0x561086a03120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:30.312 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.294+0000 7f5954d74b80 1 -- 172.21.15.39:0/676668361 >> 172.21.15.39:0/676668361 conn(0x5610869ecbe0 msgr2=0x5610869fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:53:30.312 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.294+0000 7f5954d74b80 1 -- 172.21.15.39:0/676668361 shutdown_connections 2023-07-21T18:53:30.312 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.294+0000 7f5954d74b80 1 -- 172.21.15.39:0/676668361 wait complete. 2023-07-21T18:53:30.312 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.294+0000 7f5954d74b80 1 -- start start 2023-07-21T18:53:30.312 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.295+0000 7f5954d74b80 1 Processor -- start 2023-07-21T18:53:30.313 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.295+0000 7f5954d74b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5610868536b0 0x561086a05b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:30.313 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.295+0000 7f5954d74b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610869f7580 0x561086a060b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:30.313 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.295+0000 7f5954d74b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5610869fd2d0 0x561086a065f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:30.313 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.295+0000 7f5954d74b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x561086a63a10 con 0x5610869fd2d0 2023-07-21T18:53:30.313 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.295+0000 7f5954d74b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561086a63b80 con 0x5610868536b0 2023-07-21T18:53:30.313 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.295+0000 7f5954d74b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x561086a63d40 con 0x5610869f7580 2023-07-21T18:53:30.313 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.295+0000 7f5954d74b80 10 client.0 _ll_register_callbacks cb 0x7f592c099910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:53:30.314 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.295+0000 7f5942d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5610869fd2d0 0x561086a065f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:30.314 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.295+0000 7f5942d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5610869fd2d0 0x561086a065f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:32792/0 (socket says 172.21.15.39:32792) 2023-07-21T18:53:30.315 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.295+0000 7f5942d9d700 1 -- 172.21.15.39:0/1429738977 learned_addr learned my addr 172.21.15.39:0/1429738977 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:53:30.315 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.295+0000 7f593bfff700 1 --2- 172.21.15.39:0/1429738977 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610869f7580 0x561086a060b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:30.315 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.295+0000 7f594259c700 1 --2- 172.21.15.39:0/1429738977 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5610868536b0 0x561086a05b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:30.315 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.295+0000 7f5942d9d700 1 -- 172.21.15.39:0/1429738977 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610869f7580 msgr2=0x561086a060b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:53:30.315 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.295+0000 7f5942d9d700 1 --2- 172.21.15.39:0/1429738977 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610869f7580 0x561086a060b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:30.315 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.295+0000 7f5942d9d700 1 -- 172.21.15.39:0/1429738977 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5610868536b0 msgr2=0x561086a05b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:53:30.315 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.295+0000 7f5942d9d700 1 --2- 172.21.15.39:0/1429738977 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5610868536b0 0x561086a05b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.295+0000 7f5942d9d700 1 -- 172.21.15.39:0/1429738977 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561086776d80 con 0x5610869fd2d0 2023-07-21T18:53:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.295+0000 7f5942d9d700 1 --2- 172.21.15.39:0/1429738977 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5610869fd2d0 0x561086a065f0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f593c00d010 tx=0x7f593c0078c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec5bd9b1d2f6c809 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:53:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.295+0000 7f5929ffb700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:53:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.296+0000 7f5929ffb700 1 -- 172.21.15.39:0/1429738977 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593c004ea0 con 0x5610869fd2d0 2023-07-21T18:53:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.296+0000 7f5929ffb700 1 -- 172.21.15.39:0/1429738977 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f593c005050 con 0x5610869fd2d0 2023-07-21T18:53:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.296+0000 7f5954d74b80 1 -- 172.21.15.39:0/1429738977 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f593c00bcb0 con 0x5610869fd2d0 2023-07-21T18:53:30.317 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.296+0000 7f5929ffb700 1 -- 172.21.15.39:0/1429738977 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f593c005210 con 0x5610869fd2d0 2023-07-21T18:53:30.317 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.296+0000 7f5954d74b80 10 client.17052 fetch_fsmap learned FSMap version 68 2023-07-21T18:53:30.317 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.296+0000 7f5954d74b80 1 -- 172.21.15.39:0/1429738977 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) v3 -- 0x56108679d1f0 con 0x5610869fd2d0 2023-07-21T18:53:30.317 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.296+0000 7f5929ffb700 1 -- 172.21.15.39:0/1429738977 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f593c00bcb0 con 0x5610869fd2d0 2023-07-21T18:53:30.317 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.296+0000 7f5929ffb700 1 -- 172.21.15.39:0/1429738977 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 68) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f593c019030 con 0x5610869fd2d0 2023-07-21T18:53:30.317 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.296+0000 7f5954d74b80 10 client.17052 fetch_fsmap finished waiting for FSMap version 68 2023-07-21T18:53:30.317 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.296+0000 7f5954d74b80 10 client.17052 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:53:30.318 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.296+0000 7f5954d74b80 20 client.17052 populate_metadata read hostname 'smithi039' 2023-07-21T18:53:30.318 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.296+0000 7f5954d74b80 1 --2- 172.21.15.39:0/1429738977 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x561086a65040 0x561086a67420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:30.318 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.296+0000 7f5954d74b80 4 client.17052 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:53:30.318 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.296+0000 7f5954d74b80 1 -- 172.21.15.39:0/1429738977 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561086903ff0 con 0x561086a65040 2023-07-21T18:53:30.318 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.296+0000 7f594259c700 1 --2- 172.21.15.39:0/1429738977 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x561086a65040 0x561086a67420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:30.318 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.297+0000 7f5929ffb700 10 client.17052 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:53:30.318 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.297+0000 7f594259c700 1 --2- 172.21.15.39:0/1429738977 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x561086a65040 0x561086a67420 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0ad3fb4f9b16fdc server_cookie=1db2a5cb007d7cb in_seq=0 out_seq=0 2023-07-21T18:53:30.349 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.338+0000 7f5929ffb700 1 -- 172.21.15.39:0/1429738977 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561086903ff0 con 0x561086a65040 2023-07-21T18:53:30.350 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.338+0000 7f5929ffb700 10 client.17052 handle_command_reply: tid=0 2023-07-21T18:53:30.350 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.338+0000 7f5954d74b80 1 client.17052 shutdown 2023-07-21T18:53:30.350 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.338+0000 7f5954d74b80 1 -- 172.21.15.39:0/1429738977 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5610869fd2d0 msgr2=0x561086a065f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:53:30.350 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.338+0000 7f5954d74b80 1 --2- 172.21.15.39:0/1429738977 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5610869fd2d0 0x561086a065f0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f593c00d010 tx=0x7f593c0078c0 comp rx=0 tx=0).stop 2023-07-21T18:53:30.351 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.338+0000 7f5954d74b80 1 -- 172.21.15.39:0/1429738977 shutdown_connections 2023-07-21T18:53:30.351 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.338+0000 7f5954d74b80 1 --2- 172.21.15.39:0/1429738977 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x561086a65040 0x561086a67420 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:30.351 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.338+0000 7f5954d74b80 1 --2- 172.21.15.39:0/1429738977 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5610868536b0 0x561086a05b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:30.351 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.338+0000 7f5954d74b80 1 --2- 172.21.15.39:0/1429738977 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5610869fd2d0 0x561086a065f0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:30.351 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.338+0000 7f5954d74b80 1 --2- 172.21.15.39:0/1429738977 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610869f7580 0x561086a060b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:30.352 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.338+0000 7f5954d74b80 1 -- 172.21.15.39:0/1429738977 >> 172.21.15.39:0/1429738977 conn(0x5610869ecbe0 msgr2=0x5610869fe320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:53:30.352 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.338+0000 7f5929ffb700 0 client.17052 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:53:30.352 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.339+0000 7f5954d74b80 1 -- 172.21.15.39:0/1429738977 shutdown_connections 2023-07-21T18:53:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.339+0000 7f5954d74b80 1 -- 172.21.15.39:0/1429738977 wait complete. 2023-07-21T18:53:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.339+0000 7f5954d74b80 20 client.17052 trim_cache size 0 max 16384 2023-07-21T18:53:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:53:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:53:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:53:30.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:53:30.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.339+0000 7f5948bee700 1 -- 172.21.15.39:0/2643269762 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f59300a12c0 msgr2=0x7f59300a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:53:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.339+0000 7f5948bee700 1 --2- 172.21.15.39:0/2643269762 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f59300a12c0 0x7f59300a3770 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f59440ffba0 tx=0x7f593400b410 comp rx=0 tx=0).stop 2023-07-21T18:53:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.339+0000 7f5948bee700 1 -- 172.21.15.39:0/2643269762 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59440ff3e0 msgr2=0x7f5944198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:53:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.339+0000 7f5948bee700 1 --2- 172.21.15.39:0/2643269762 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59440ff3e0 0x7f5944198420 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f592c00e9a0 tx=0x7f592c00ecb0 comp rx=0 tx=0).stop 2023-07-21T18:53:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.340+0000 7f5948bee700 1 -- 172.21.15.39:0/2643269762 shutdown_connections 2023-07-21T18:53:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.340+0000 7f5948bee700 1 --2- 172.21.15.39:0/2643269762 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f59300a12c0 0x7f59300a3770 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.340+0000 7f5948bee700 1 --2- 172.21.15.39:0/2643269762 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5944104a30 0x7f594419ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:30.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.340+0000 7f5948bee700 1 --2- 172.21.15.39:0/2643269762 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f59440fea30 0x7f5944197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:30.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.340+0000 7f5948bee700 1 --2- 172.21.15.39:0/2643269762 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59440ff3e0 0x7f5944198420 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:30.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.340+0000 7f5948bee700 1 -- 172.21.15.39:0/2643269762 >> 172.21.15.39:0/2643269762 conn(0x7f5944074760 msgr2=0x7f59440fd460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:53:30.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.340+0000 7f5948bee700 1 -- 172.21.15.39:0/2643269762 shutdown_connections 2023-07-21T18:53:30.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:30.340+0000 7f5948bee700 1 -- 172.21.15.39:0/2643269762 wait complete. 2023-07-21T18:53:30.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:53:30 smithi193 ceph-mon[98204]: pgmap v368: 97 pgs: 97 active+clean; 273 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 1.3 MiB/s wr, 124 op/s 2023-07-21T18:53:30.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:53:30 smithi155 ceph-mon[97934]: pgmap v368: 97 pgs: 97 active+clean; 273 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 1.3 MiB/s wr, 124 op/s 2023-07-21T18:53:30.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:53:30 smithi039 ceph-mon[96489]: pgmap v368: 97 pgs: 97 active+clean; 273 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 1.3 MiB/s wr, 124 op/s 2023-07-21T18:53:31.292 INFO:tasks.workunit.client.0.smithi039.stdout: 1 669314 5.34 MB/sec execute 524 sec latency 2961.672 ms 2023-07-21T18:53:32.292 INFO:tasks.workunit.client.0.smithi039.stdout: 1 670536 5.35 MB/sec execute 525 sec latency 3391.230 ms 2023-07-21T18:53:32.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:53:32 smithi193 ceph-mon[98204]: pgmap v369: 97 pgs: 97 active+clean; 277 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 2.1 MiB/s wr, 193 op/s 2023-07-21T18:53:32.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:53:32 smithi155 ceph-mon[97934]: pgmap v369: 97 pgs: 97 active+clean; 277 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 2.1 MiB/s wr, 193 op/s 2023-07-21T18:53:32.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:53:32 smithi039 ceph-mon[96489]: pgmap v369: 97 pgs: 97 active+clean; 277 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 2.1 MiB/s wr, 193 op/s 2023-07-21T18:53:33.292 INFO:tasks.workunit.client.0.smithi039.stdout: 1 671713 5.35 MB/sec execute 526 sec latency 20.852 ms 2023-07-21T18:53:34.292 INFO:tasks.workunit.client.0.smithi039.stdout: 1 672856 5.35 MB/sec execute 527 sec latency 250.037 ms 2023-07-21T18:53:34.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:53:34 smithi193 ceph-mon[98204]: pgmap v370: 97 pgs: 97 active+clean; 277 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 1.2 MiB/s wr, 124 op/s 2023-07-21T18:53:34.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:53:34 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:53:34.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:53:34 smithi155 ceph-mon[97934]: pgmap v370: 97 pgs: 97 active+clean; 277 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 1.2 MiB/s wr, 124 op/s 2023-07-21T18:53:34.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:53:34 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:53:34.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:53:34 smithi039 ceph-mon[96489]: pgmap v370: 97 pgs: 97 active+clean; 277 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 1.2 MiB/s wr, 124 op/s 2023-07-21T18:53:34.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:53:34 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:53:35.292 INFO:tasks.workunit.client.0.smithi039.stdout: 1 672856 5.34 MB/sec execute 528 sec latency 1250.096 ms 2023-07-21T18:53:36.292 INFO:tasks.workunit.client.0.smithi039.stdout: 1 672856 5.33 MB/sec execute 529 sec latency 2250.138 ms 2023-07-21T18:53:36.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:53:36 smithi193 ceph-mon[98204]: pgmap v371: 97 pgs: 97 active+clean; 277 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 1.2 MiB/s wr, 124 op/s 2023-07-21T18:53:36.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:53:36 smithi155 ceph-mon[97934]: pgmap v371: 97 pgs: 97 active+clean; 277 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 1.2 MiB/s wr, 124 op/s 2023-07-21T18:53:36.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:53:36 smithi039 ceph-mon[96489]: pgmap v371: 97 pgs: 97 active+clean; 277 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 1.2 MiB/s wr, 124 op/s 2023-07-21T18:53:37.292 INFO:tasks.workunit.client.0.smithi039.stdout: 1 674118 5.33 MB/sec execute 530 sec latency 2677.696 ms 2023-07-21T18:53:38.292 INFO:tasks.workunit.client.0.smithi039.stdout: 1 675301 5.33 MB/sec execute 531 sec latency 18.853 ms 2023-07-21T18:53:38.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:53:38 smithi193 ceph-mon[98204]: pgmap v372: 97 pgs: 97 active+clean; 272 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 1.7 MiB/s wr, 177 op/s 2023-07-21T18:53:38.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:53:38 smithi155 ceph-mon[97934]: pgmap v372: 97 pgs: 97 active+clean; 272 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 1.7 MiB/s wr, 177 op/s 2023-07-21T18:53:38.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:53:38 smithi039 ceph-mon[96489]: pgmap v372: 97 pgs: 97 active+clean; 272 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 1.7 MiB/s wr, 177 op/s 2023-07-21T18:53:39.292 INFO:tasks.workunit.client.0.smithi039.stdout: 1 676458 5.33 MB/sec execute 532 sec latency 214.085 ms 2023-07-21T18:53:40.292 INFO:tasks.workunit.client.0.smithi039.stdout: 1 676458 5.32 MB/sec execute 533 sec latency 1214.197 ms 2023-07-21T18:53:40.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:53:40 smithi193 ceph-mon[98204]: pgmap v373: 97 pgs: 97 active+clean; 272 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 1.3 MiB/s wr, 121 op/s 2023-07-21T18:53:40.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:53:40 smithi155 ceph-mon[97934]: pgmap v373: 97 pgs: 97 active+clean; 272 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 1.3 MiB/s wr, 121 op/s 2023-07-21T18:53:40.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:53:40 smithi039 ceph-mon[96489]: pgmap v373: 97 pgs: 97 active+clean; 272 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 1.3 MiB/s wr, 121 op/s 2023-07-21T18:53:41.292 INFO:tasks.workunit.client.0.smithi039.stdout: 1 676458 5.31 MB/sec execute 534 sec latency 2214.294 ms 2023-07-21T18:53:42.292 INFO:tasks.workunit.client.0.smithi039.stdout: 1 677681 5.32 MB/sec execute 535 sec latency 2640.444 ms 2023-07-21T18:53:42.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:53:42 smithi193 ceph-mon[98204]: pgmap v374: 97 pgs: 97 active+clean; 275 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 932 KiB/s rd, 2.1 MiB/s wr, 182 op/s 2023-07-21T18:53:42.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:53:42 smithi155 ceph-mon[97934]: pgmap v374: 97 pgs: 97 active+clean; 275 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 932 KiB/s rd, 2.1 MiB/s wr, 182 op/s 2023-07-21T18:53:42.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:53:42 smithi039 ceph-mon[96489]: pgmap v374: 97 pgs: 97 active+clean; 275 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 932 KiB/s rd, 2.1 MiB/s wr, 182 op/s 2023-07-21T18:53:43.292 INFO:tasks.workunit.client.0.smithi039.stdout: 1 678918 5.32 MB/sec execute 536 sec latency 29.488 ms 2023-07-21T18:53:43.469 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.457+0000 7f953c1e0700 1 -- 172.21.15.39:0/3894248380 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95340ff3e0 msgr2=0x7f953410b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:53:43.469 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.457+0000 7f953c1e0700 1 --2- 172.21.15.39:0/3894248380 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95340ff3e0 0x7f953410b860 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f952c00b0a0 tx=0x7f952c00b3b0 comp rx=0 tx=0).stop 2023-07-21T18:53:43.469 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.457+0000 7f953c1e0700 1 -- 172.21.15.39:0/3894248380 shutdown_connections 2023-07-21T18:53:43.469 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.457+0000 7f953c1e0700 1 --2- 172.21.15.39:0/3894248380 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95340ff3e0 0x7f953410b860 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:43.470 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.457+0000 7f953c1e0700 1 --2- 172.21.15.39:0/3894248380 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9534104a30 0x7f9534104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:43.470 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.457+0000 7f953c1e0700 1 --2- 172.21.15.39:0/3894248380 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95340fea30 0x7f95340feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:43.470 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.457+0000 7f953c1e0700 1 -- 172.21.15.39:0/3894248380 >> 172.21.15.39:0/3894248380 conn(0x7f9534074760 msgr2=0x7f9534074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:53:43.470 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.457+0000 7f953c1e0700 1 -- 172.21.15.39:0/3894248380 shutdown_connections 2023-07-21T18:53:43.470 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.457+0000 7f953c1e0700 1 -- 172.21.15.39:0/3894248380 wait complete. 2023-07-21T18:53:43.472 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.458+0000 7f953c1e0700 1 Processor -- start 2023-07-21T18:53:43.472 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.458+0000 7f953c1e0700 1 -- start start 2023-07-21T18:53:43.472 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.458+0000 7f953c1e0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95340fea30 0x7f9534100810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:43.472 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.458+0000 7f953c1e0700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95340ff3e0 0x7f9534100d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:43.472 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.458+0000 7f953c1e0700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9534104a30 0x7f95341a3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:43.472 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.458+0000 7f953c1e0700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95341a43d0 con 0x7f9534104a30 2023-07-21T18:53:43.473 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.458+0000 7f953c1e0700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95341a4510 con 0x7f95340fea30 2023-07-21T18:53:43.473 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.458+0000 7f953c1e0700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95341a4680 con 0x7f95340ff3e0 2023-07-21T18:53:43.473 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.458+0000 7f953a77d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9534104a30 0x7f95341a3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:43.473 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.458+0000 7f953a77d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9534104a30 0x7f95341a3e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:32806/0 (socket says 172.21.15.39:32806) 2023-07-21T18:53:43.473 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.458+0000 7f953a77d700 1 -- 172.21.15.39:0/1214890868 learned_addr learned my addr 172.21.15.39:0/1214890868 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:53:43.473 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.458+0000 7f953977b700 1 --2- 172.21.15.39:0/1214890868 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95340ff3e0 0x7f9534100d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:43.473 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.458+0000 7f9539f7c700 1 --2- 172.21.15.39:0/1214890868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95340fea30 0x7f9534100810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:43.474 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.459+0000 7f953a77d700 1 -- 172.21.15.39:0/1214890868 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95340ff3e0 msgr2=0x7f9534100d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:53:43.474 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.459+0000 7f953a77d700 1 --2- 172.21.15.39:0/1214890868 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95340ff3e0 0x7f9534100d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:43.474 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.459+0000 7f953a77d700 1 -- 172.21.15.39:0/1214890868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95340fea30 msgr2=0x7f9534100810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:53:43.474 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.459+0000 7f953a77d700 1 --2- 172.21.15.39:0/1214890868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95340fea30 0x7f9534100810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:43.474 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.459+0000 7f953a77d700 1 -- 172.21.15.39:0/1214890868 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f952c009d00 con 0x7f9534104a30 2023-07-21T18:53:43.474 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.459+0000 7f953a77d700 1 --2- 172.21.15.39:0/1214890868 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9534104a30 0x7f95341a3e90 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f952c00b780 tx=0x7f952c00bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:53:43.474 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.459+0000 7f952affd700 1 -- 172.21.15.39:0/1214890868 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f952c00e040 con 0x7f9534104a30 2023-07-21T18:53:43.474 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.459+0000 7f953c1e0700 1 -- 172.21.15.39:0/1214890868 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95341a4900 con 0x7f9534104a30 2023-07-21T18:53:43.475 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.459+0000 7f952affd700 1 -- 172.21.15.39:0/1214890868 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f952c0094a0 con 0x7f9534104a30 2023-07-21T18:53:43.475 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.459+0000 7f952affd700 1 -- 172.21.15.39:0/1214890868 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f952c0052a0 con 0x7f9534104a30 2023-07-21T18:53:43.475 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.459+0000 7f953c1e0700 1 -- 172.21.15.39:0/1214890868 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95341a4dd0 con 0x7f9534104a30 2023-07-21T18:53:43.477 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.461+0000 7f952affd700 1 -- 172.21.15.39:0/1214890868 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f952c01d470 con 0x7f9534104a30 2023-07-21T18:53:43.477 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.462+0000 7f952affd700 1 --2- 172.21.15.39:0/1214890868 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f95200a12c0 0x7f95200a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:43.477 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.462+0000 7f952affd700 1 -- 172.21.15.39:0/1214890868 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f952c0ce200 con 0x7f9534104a30 2023-07-21T18:53:43.477 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.463+0000 7f9548366b80 1 Processor -- start 2023-07-21T18:53:43.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.463+0000 7f9539f7c700 1 --2- 172.21.15.39:0/1214890868 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f95200a12c0 0x7f95200a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:43.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.463+0000 7f9548366b80 1 -- start start 2023-07-21T18:53:43.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.463+0000 7f9548366b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566d13866b0 0x5566d152fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:43.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.463+0000 7f9548366b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566d15302d0 0x5566d152a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:43.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.463+0000 7f9548366b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566d152a580 0x5566d152a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:43.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.463+0000 7f9548366b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566d142fc50 con 0x5566d13866b0 2023-07-21T18:53:43.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.463+0000 7f9548366b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566d14829e0 con 0x5566d152a580 2023-07-21T18:53:43.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.463+0000 7f9548366b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566d147be70 con 0x5566d15302d0 2023-07-21T18:53:43.479 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.463+0000 7f9539f7c700 1 --2- 172.21.15.39:0/1214890868 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f95200a12c0 0x7f95200a3770 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f95340ffba0 tx=0x7f953000b410 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:53:43.479 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.463+0000 7f953977b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566d15302d0 0x5566d152a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:43.479 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.463+0000 7f9539f7c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566d13866b0 0x5566d152fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:43.479 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.463+0000 7f953977b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566d15302d0 0x5566d152a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:51364/0 (socket says 172.21.15.39:51364) 2023-07-21T18:53:43.479 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.463+0000 7f953977b700 1 -- 172.21.15.39:0/222438981 learned_addr learned my addr 172.21.15.39:0/222438981 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:53:43.479 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.463+0000 7f953a77d700 1 --2- 172.21.15.39:0/222438981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566d152a580 0x5566d152a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:43.479 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.464+0000 7f953977b700 1 -- 172.21.15.39:0/222438981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566d152a580 msgr2=0x5566d152a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:53:43.480 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.464+0000 7f953977b700 1 --2- 172.21.15.39:0/222438981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566d152a580 0x5566d152a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:43.480 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.464+0000 7f953977b700 1 -- 172.21.15.39:0/222438981 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566d13866b0 msgr2=0x5566d152fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:53:43.480 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.464+0000 7f953977b700 1 --2- 172.21.15.39:0/222438981 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566d13866b0 0x5566d152fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:43.480 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.464+0000 7f953977b700 1 -- 172.21.15.39:0/222438981 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5566d12a9d80 con 0x5566d15302d0 2023-07-21T18:53:43.480 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.464+0000 7f9539f7c700 1 --2- 172.21.15.39:0/222438981 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566d13866b0 0x5566d152fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:53:43.480 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.464+0000 7f953977b700 1 --2- 172.21.15.39:0/222438981 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566d15302d0 0x5566d152a040 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9524009d20 tx=0x7f9524011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e235a914590bd5f3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:53:43.480 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.464+0000 7f951ffff700 1 -- 172.21.15.39:0/222438981 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9524019630 con 0x5566d15302d0 2023-07-21T18:53:43.480 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.464+0000 7f951ffff700 1 -- 172.21.15.39:0/222438981 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9524019c50 con 0x5566d15302d0 2023-07-21T18:53:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.464+0000 7f951ffff700 1 -- 172.21.15.39:0/222438981 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f952400eb90 con 0x5566d15302d0 2023-07-21T18:53:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.465+0000 7f9548366b80 1 -- 172.21.15.39:0/222438981 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566d15302d0 msgr2=0x5566d152a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:53:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.465+0000 7f9548366b80 1 --2- 172.21.15.39:0/222438981 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566d15302d0 0x5566d152a040 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9524009d20 tx=0x7f9524011720 comp rx=0 tx=0).stop 2023-07-21T18:53:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.465+0000 7f9548366b80 1 -- 172.21.15.39:0/222438981 shutdown_connections 2023-07-21T18:53:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.465+0000 7f9548366b80 1 --2- 172.21.15.39:0/222438981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566d152a580 0x5566d152a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.465+0000 7f9548366b80 1 --2- 172.21.15.39:0/222438981 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566d13866b0 0x5566d152fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.465+0000 7f9548366b80 1 --2- 172.21.15.39:0/222438981 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566d15302d0 0x5566d152a040 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.465+0000 7f9548366b80 1 -- 172.21.15.39:0/222438981 >> 172.21.15.39:0/222438981 conn(0x5566d151fbe0 msgr2=0x5566d1524f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:53:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.465+0000 7f9548366b80 1 -- 172.21.15.39:0/222438981 shutdown_connections 2023-07-21T18:53:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.465+0000 7f9548366b80 1 -- 172.21.15.39:0/222438981 wait complete. 2023-07-21T18:53:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.465+0000 7f9548366b80 1 -- start start 2023-07-21T18:53:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.466+0000 7f9548366b80 1 Processor -- start 2023-07-21T18:53:43.485 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.466+0000 7f9548366b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566d13866b0 0x5566d15365e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:43.485 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.466+0000 7f9548366b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566d152a580 0x5566d1533860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:43.485 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.466+0000 7f9548366b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566d15302d0 0x5566d1533fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:43.485 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.466+0000 7f9548366b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566d141ba60 con 0x5566d13866b0 2023-07-21T18:53:43.485 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.466+0000 7f9548366b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566d141b760 con 0x5566d15302d0 2023-07-21T18:53:43.485 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.466+0000 7f9548366b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566d12ff8c0 con 0x5566d152a580 2023-07-21T18:53:43.485 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.466+0000 7f9548366b80 10 client.0 _ll_register_callbacks cb 0x5566d1419700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:53:43.486 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.466+0000 7f9539f7c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566d13866b0 0x5566d15365e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:43.486 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.466+0000 7f9539f7c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566d13866b0 0x5566d15365e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:32816/0 (socket says 172.21.15.39:32816) 2023-07-21T18:53:43.486 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.466+0000 7f9539f7c700 1 -- 172.21.15.39:0/754167607 learned_addr learned my addr 172.21.15.39:0/754167607 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:53:43.486 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.466+0000 7f953977b700 1 --2- 172.21.15.39:0/754167607 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566d152a580 0x5566d1533860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:43.486 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.466+0000 7f953a77d700 1 --2- 172.21.15.39:0/754167607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566d15302d0 0x5566d1533fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:43.486 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.467+0000 7f9539f7c700 1 -- 172.21.15.39:0/754167607 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566d152a580 msgr2=0x5566d1533860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:53:43.486 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.467+0000 7f9539f7c700 1 --2- 172.21.15.39:0/754167607 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566d152a580 0x5566d1533860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:43.487 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.467+0000 7f9539f7c700 1 -- 172.21.15.39:0/754167607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566d15302d0 msgr2=0x5566d1533fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:53:43.487 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.467+0000 7f9539f7c700 1 --2- 172.21.15.39:0/754167607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566d15302d0 0x5566d1533fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:43.487 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.467+0000 7f9539f7c700 1 -- 172.21.15.39:0/754167607 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9530010c50 con 0x5566d13866b0 2023-07-21T18:53:43.487 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.467+0000 7f953a77d700 1 --2- 172.21.15.39:0/754167607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566d15302d0 0x5566d1533fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:53:43.487 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.467+0000 7f9539f7c700 1 --2- 172.21.15.39:0/754167607 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566d13866b0 0x5566d15365e0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f9530011860 tx=0x7f95300047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1b6f2a85d10cfa1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:53:43.487 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.467+0000 7f951d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:53:43.487 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.467+0000 7f951d7fa700 1 -- 172.21.15.39:0/754167607 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9530004220 con 0x5566d13866b0 2023-07-21T18:53:43.487 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.467+0000 7f951d7fa700 1 -- 172.21.15.39:0/754167607 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f9530017070 con 0x5566d13866b0 2023-07-21T18:53:43.488 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.467+0000 7f9548366b80 1 -- 172.21.15.39:0/754167607 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5566d13d5490 con 0x5566d13866b0 2023-07-21T18:53:43.488 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.467+0000 7f951d7fa700 1 -- 172.21.15.39:0/754167607 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9530004380 con 0x5566d13866b0 2023-07-21T18:53:43.488 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.467+0000 7f9548366b80 10 client.17064 fetch_fsmap learned FSMap version 69 2023-07-21T18:53:43.488 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.467+0000 7f9548366b80 1 -- 172.21.15.39:0/754167607 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) v3 -- 0x7f9524009980 con 0x5566d13866b0 2023-07-21T18:53:43.488 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.468+0000 7f951d7fa700 1 -- 172.21.15.39:0/754167607 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5566d13d5490 con 0x5566d13866b0 2023-07-21T18:53:43.488 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.468+0000 7f951d7fa700 1 -- 172.21.15.39:0/754167607 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 69) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f953001e8a0 con 0x5566d13866b0 2023-07-21T18:53:43.488 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.468+0000 7f9548366b80 10 client.17064 fetch_fsmap finished waiting for FSMap version 69 2023-07-21T18:53:43.489 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.468+0000 7f9548366b80 10 client.17064 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:53:43.489 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.468+0000 7f9548366b80 20 client.17064 populate_metadata read hostname 'smithi039' 2023-07-21T18:53:43.489 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.468+0000 7f9548366b80 1 --2- 172.21.15.39:0/754167607 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5566d1543bb0 0x5566d1545fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:43.489 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.468+0000 7f9548366b80 4 client.17064 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:53:43.489 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.468+0000 7f9548366b80 1 -- 172.21.15.39:0/754167607 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5566d1474e50 con 0x5566d1543bb0 2023-07-21T18:53:43.489 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.468+0000 7f953977b700 1 --2- 172.21.15.39:0/754167607 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5566d1543bb0 0x5566d1545fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:43.489 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.468+0000 7f951d7fa700 10 client.17064 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:53:43.489 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.468+0000 7f953977b700 1 --2- 172.21.15.39:0/754167607 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5566d1543bb0 0x5566d1545fc0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e0cad0fac3a9db2 server_cookie=bd1f6e6d409ac356 in_seq=0 out_seq=0 2023-07-21T18:53:43.490 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.469+0000 7f951d7fa700 1 -- 172.21.15.39:0/754167607 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5566d1474e50 con 0x5566d1543bb0 2023-07-21T18:53:43.490 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.469+0000 7f951d7fa700 10 client.17064 handle_command_reply: tid=0 2023-07-21T18:53:43.490 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.469+0000 7f9548366b80 1 client.17064 shutdown 2023-07-21T18:53:43.490 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.469+0000 7f9548366b80 1 -- 172.21.15.39:0/754167607 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566d13866b0 msgr2=0x5566d15365e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:53:43.490 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.469+0000 7f9548366b80 1 --2- 172.21.15.39:0/754167607 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566d13866b0 0x5566d15365e0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f9530011860 tx=0x7f95300047c0 comp rx=0 tx=0).stop 2023-07-21T18:53:43.490 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.469+0000 7f9548366b80 1 -- 172.21.15.39:0/754167607 shutdown_connections 2023-07-21T18:53:43.490 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.469+0000 7f9548366b80 1 --2- 172.21.15.39:0/754167607 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5566d1543bb0 0x5566d1545fc0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:43.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.469+0000 7f9548366b80 1 --2- 172.21.15.39:0/754167607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566d15302d0 0x5566d1533fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:43.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.469+0000 7f9548366b80 1 --2- 172.21.15.39:0/754167607 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566d13866b0 0x5566d15365e0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:43.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.469+0000 7f9548366b80 1 --2- 172.21.15.39:0/754167607 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566d152a580 0x5566d1533860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:43.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.469+0000 7f9548366b80 1 -- 172.21.15.39:0/754167607 >> 172.21.15.39:0/754167607 conn(0x5566d151fbe0 msgr2=0x5566d15244e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:53:43.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.469+0000 7f951d7fa700 0 client.17064 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:53:43.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.469+0000 7f9548366b80 1 -- 172.21.15.39:0/754167607 shutdown_connections 2023-07-21T18:53:43.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.469+0000 7f9548366b80 1 -- 172.21.15.39:0/754167607 wait complete. 2023-07-21T18:53:43.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.469+0000 7f9548366b80 20 client.17064 trim_cache size 0 max 16384 2023-07-21T18:53:43.495 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.484+0000 7f9548366b80 1 Processor -- start 2023-07-21T18:53:43.496 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.484+0000 7f9548366b80 1 -- start start 2023-07-21T18:53:43.496 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.484+0000 7f9548366b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566d151fbe0 0x5566d152a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:43.496 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.484+0000 7f9548366b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566d13866b0 0x5566d152f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:43.496 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.484+0000 7f9548366b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566d152a580 0x5566d1536180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:43.496 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.484+0000 7f9548366b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566d15405d0 con 0x5566d152a580 2023-07-21T18:53:43.496 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.484+0000 7f9548366b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566d1534e40 con 0x5566d13866b0 2023-07-21T18:53:43.496 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.484+0000 7f9548366b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566d1526990 con 0x5566d151fbe0 2023-07-21T18:53:43.499 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.485+0000 7f953a77d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566d152a580 0x5566d1536180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.485+0000 7f953a77d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566d152a580 0x5566d1536180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:32828/0 (socket says 172.21.15.39:32828) 2023-07-21T18:53:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.485+0000 7f953a77d700 1 -- 172.21.15.39:0/2357545529 learned_addr learned my addr 172.21.15.39:0/2357545529 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:53:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.485+0000 7f9539f7c700 1 --2- 172.21.15.39:0/2357545529 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566d151fbe0 0x5566d152a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.485+0000 7f953977b700 1 --2- 172.21.15.39:0/2357545529 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566d13866b0 0x5566d152f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.485+0000 7f953a77d700 1 -- 172.21.15.39:0/2357545529 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566d151fbe0 msgr2=0x5566d152a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:53:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.485+0000 7f953a77d700 1 --2- 172.21.15.39:0/2357545529 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566d151fbe0 0x5566d152a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:43.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.485+0000 7f953a77d700 1 -- 172.21.15.39:0/2357545529 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566d13866b0 msgr2=0x5566d152f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:53:43.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.485+0000 7f953a77d700 1 --2- 172.21.15.39:0/2357545529 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566d13866b0 0x5566d152f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:43.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.485+0000 7f953a77d700 1 -- 172.21.15.39:0/2357545529 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5566d152ac20 con 0x5566d152a580 2023-07-21T18:53:43.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.485+0000 7f953a77d700 1 --2- 172.21.15.39:0/2357545529 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566d152a580 0x5566d1536180 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f952c0cbf50 tx=0x7f952c086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbd57cd48a995f3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:53:43.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.485+0000 7f9528ff9700 1 -- 172.21.15.39:0/2357545529 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f952c09acc0 con 0x5566d152a580 2023-07-21T18:53:43.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.485+0000 7f9528ff9700 1 -- 172.21.15.39:0/2357545529 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f952c0ce070 con 0x5566d152a580 2023-07-21T18:53:43.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.485+0000 7f9528ff9700 1 -- 172.21.15.39:0/2357545529 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f952c09b5c0 con 0x5566d152a580 2023-07-21T18:53:43.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.486+0000 7f9548366b80 1 -- 172.21.15.39:0/2357545529 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566d152a580 msgr2=0x5566d1536180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:53:43.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.486+0000 7f9548366b80 1 --2- 172.21.15.39:0/2357545529 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566d152a580 0x5566d1536180 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f952c0cbf50 tx=0x7f952c086040 comp rx=0 tx=0).stop 2023-07-21T18:53:43.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.486+0000 7f9548366b80 1 -- 172.21.15.39:0/2357545529 shutdown_connections 2023-07-21T18:53:43.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.486+0000 7f9548366b80 1 --2- 172.21.15.39:0/2357545529 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566d152a580 0x5566d1536180 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:43.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.486+0000 7f9548366b80 1 --2- 172.21.15.39:0/2357545529 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566d151fbe0 0x5566d152a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:43.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.486+0000 7f9548366b80 1 --2- 172.21.15.39:0/2357545529 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566d13866b0 0x5566d152f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:43.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.486+0000 7f9548366b80 1 -- 172.21.15.39:0/2357545529 >> 172.21.15.39:0/2357545529 conn(0x5566d1540200 msgr2=0x5566d152ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:53:43.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.486+0000 7f9548366b80 1 -- 172.21.15.39:0/2357545529 shutdown_connections 2023-07-21T18:53:43.503 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.486+0000 7f9548366b80 1 -- 172.21.15.39:0/2357545529 wait complete. 2023-07-21T18:53:43.503 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.486+0000 7f9548366b80 1 -- start start 2023-07-21T18:53:43.503 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.486+0000 7f9548366b80 1 Processor -- start 2023-07-21T18:53:43.503 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.487+0000 7f9548366b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566d13866b0 0x5566d1538c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:43.503 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.487+0000 7f9548366b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566d151fbe0 0x5566d1539190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:43.503 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.487+0000 7f9548366b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566d152a580 0x5566d15396d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:43.503 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.487+0000 7f9548366b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566d153b2b0 con 0x5566d151fbe0 2023-07-21T18:53:43.503 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.487+0000 7f9548366b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566d1596a60 con 0x5566d152a580 2023-07-21T18:53:43.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.487+0000 7f9548366b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566d1596c20 con 0x5566d13866b0 2023-07-21T18:53:43.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.487+0000 7f9548366b80 10 client.0 _ll_register_callbacks cb 0x7f9530011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:53:43.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.487+0000 7f9539f7c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566d13866b0 0x5566d1538c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:43.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.487+0000 7f953977b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566d151fbe0 0x5566d1539190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:43.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.487+0000 7f9539f7c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566d13866b0 0x5566d1538c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:51382/0 (socket says 172.21.15.39:51382) 2023-07-21T18:53:43.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.487+0000 7f9539f7c700 1 -- 172.21.15.39:0/1113028763 learned_addr learned my addr 172.21.15.39:0/1113028763 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:53:43.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.487+0000 7f953a77d700 1 --2- 172.21.15.39:0/1113028763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566d152a580 0x5566d15396d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:43.505 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.487+0000 7f9539f7c700 1 -- 172.21.15.39:0/1113028763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566d152a580 msgr2=0x5566d15396d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:53:43.505 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.487+0000 7f9539f7c700 1 --2- 172.21.15.39:0/1113028763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566d152a580 0x5566d15396d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:43.505 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.487+0000 7f9539f7c700 1 -- 172.21.15.39:0/1113028763 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566d151fbe0 msgr2=0x5566d1539190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:53:43.505 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.487+0000 7f9539f7c700 1 --2- 172.21.15.39:0/1113028763 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566d151fbe0 0x5566d1539190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:43.505 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.487+0000 7f9539f7c700 1 -- 172.21.15.39:0/1113028763 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95341a4dd0 con 0x5566d13866b0 2023-07-21T18:53:43.505 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.487+0000 7f953977b700 1 --2- 172.21.15.39:0/1113028763 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566d151fbe0 0x5566d1539190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:53:43.505 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.487+0000 7f9539f7c700 1 --2- 172.21.15.39:0/1113028763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566d13866b0 0x5566d1538c50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f953000b780 tx=0x7f9530031790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f88d3f0f201cfa12 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:53:43.506 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.487+0000 7f951d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2023-07-21T18:53:43.506 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.488+0000 7f951d7fa700 1 -- 172.21.15.39:0/1113028763 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9530031d90 con 0x5566d13866b0 2023-07-21T18:53:43.506 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.488+0000 7f951d7fa700 1 -- 172.21.15.39:0/1113028763 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f9530035070 con 0x5566d13866b0 2023-07-21T18:53:43.506 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.488+0000 7f9548366b80 1 -- 172.21.15.39:0/1113028763 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9524019c50 con 0x5566d13866b0 2023-07-21T18:53:43.506 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.488+0000 7f951d7fa700 1 -- 172.21.15.39:0/1113028763 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9530010de0 con 0x5566d13866b0 2023-07-21T18:53:43.506 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.488+0000 7f9548366b80 10 client.26542 fetch_fsmap learned FSMap version 69 2023-07-21T18:53:43.506 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.488+0000 7f9548366b80 1 -- 172.21.15.39:0/1113028763 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) v3 -- 0x5566d1292c00 con 0x5566d13866b0 2023-07-21T18:53:43.506 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.488+0000 7f951d7fa700 1 -- 172.21.15.39:0/1113028763 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9524019c50 con 0x5566d13866b0 2023-07-21T18:53:43.507 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.488+0000 7f951d7fa700 1 -- 172.21.15.39:0/1113028763 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 69) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9530003680 con 0x5566d13866b0 2023-07-21T18:53:43.507 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.488+0000 7f9548366b80 10 client.26542 fetch_fsmap finished waiting for FSMap version 69 2023-07-21T18:53:43.507 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.488+0000 7f9548366b80 10 client.26542 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:53:43.507 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.488+0000 7f9548366b80 20 client.26542 populate_metadata read hostname 'smithi039' 2023-07-21T18:53:43.507 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.488+0000 7f9548366b80 1 --2- 172.21.15.39:0/1113028763 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5566d15302d0 0x5566d159a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:43.507 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.488+0000 7f9548366b80 4 client.26542 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:53:43.507 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.488+0000 7f9548366b80 1 -- 172.21.15.39:0/1113028763 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5566d1436ff0 con 0x5566d15302d0 2023-07-21T18:53:43.507 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.488+0000 7f953977b700 1 --2- 172.21.15.39:0/1113028763 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5566d15302d0 0x5566d159a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:43.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.489+0000 7f953977b700 1 --2- 172.21.15.39:0/1113028763 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5566d15302d0 0x5566d159a390 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8510ccc48fa61256 server_cookie=ffe25556c91c08c2 in_seq=0 out_seq=0 2023-07-21T18:53:43.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.489+0000 7f951d7fa700 10 client.26542 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:53:43.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.518+0000 7f951d7fa700 1 -- 172.21.15.39:0/1113028763 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5566d1436ff0 con 0x5566d15302d0 2023-07-21T18:53:43.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.518+0000 7f951d7fa700 10 client.26542 handle_command_reply: tid=0 2023-07-21T18:53:43.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.518+0000 7f9548366b80 1 client.26542 shutdown 2023-07-21T18:53:43.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.518+0000 7f9548366b80 1 -- 172.21.15.39:0/1113028763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566d13866b0 msgr2=0x5566d1538c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:53:43.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.518+0000 7f9548366b80 1 --2- 172.21.15.39:0/1113028763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566d13866b0 0x5566d1538c50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f953000b780 tx=0x7f9530031790 comp rx=0 tx=0).stop 2023-07-21T18:53:43.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.519+0000 7f9548366b80 1 -- 172.21.15.39:0/1113028763 shutdown_connections 2023-07-21T18:53:43.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.519+0000 7f9548366b80 1 --2- 172.21.15.39:0/1113028763 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5566d15302d0 0x5566d159a390 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:43.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.519+0000 7f9548366b80 1 --2- 172.21.15.39:0/1113028763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566d152a580 0x5566d15396d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:43.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.519+0000 7f9548366b80 1 --2- 172.21.15.39:0/1113028763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566d13866b0 0x5566d1538c50 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:43.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.519+0000 7f9548366b80 1 --2- 172.21.15.39:0/1113028763 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566d151fbe0 0x5566d1539190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:43.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.519+0000 7f951d7fa700 0 client.26542 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:53:43.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.519+0000 7f9548366b80 1 -- 172.21.15.39:0/1113028763 >> 172.21.15.39:0/1113028763 conn(0x5566d1540200 msgr2=0x5566d152f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:53:43.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.519+0000 7f9548366b80 1 -- 172.21.15.39:0/1113028763 shutdown_connections 2023-07-21T18:53:43.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.519+0000 7f9548366b80 1 -- 172.21.15.39:0/1113028763 wait complete. 2023-07-21T18:53:43.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.519+0000 7f9548366b80 20 client.26542 trim_cache size 0 max 16384 2023-07-21T18:53:43.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:53:43.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:53:43.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:53:43.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:53:43.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.519+0000 7f953c1e0700 1 -- 172.21.15.39:0/1214890868 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f95200a12c0 msgr2=0x7f95200a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:53:43.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.519+0000 7f953c1e0700 1 --2- 172.21.15.39:0/1214890868 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f95200a12c0 0x7f95200a3770 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f95340ffba0 tx=0x7f953000b410 comp rx=0 tx=0).stop 2023-07-21T18:53:43.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.519+0000 7f953c1e0700 1 -- 172.21.15.39:0/1214890868 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9534104a30 msgr2=0x7f95341a3e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:53:43.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.519+0000 7f953c1e0700 1 --2- 172.21.15.39:0/1214890868 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9534104a30 0x7f95341a3e90 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f952c00b780 tx=0x7f952c00bd60 comp rx=0 tx=0).stop 2023-07-21T18:53:43.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.519+0000 7f953c1e0700 1 -- 172.21.15.39:0/1214890868 shutdown_connections 2023-07-21T18:53:43.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.519+0000 7f953c1e0700 1 --2- 172.21.15.39:0/1214890868 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f95200a12c0 0x7f95200a3770 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:43.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.519+0000 7f953c1e0700 1 --2- 172.21.15.39:0/1214890868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95340fea30 0x7f9534100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:43.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.519+0000 7f953c1e0700 1 --2- 172.21.15.39:0/1214890868 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9534104a30 0x7f95341a3e90 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.519+0000 7f953c1e0700 1 --2- 172.21.15.39:0/1214890868 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95340ff3e0 0x7f9534100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.519+0000 7f953c1e0700 1 -- 172.21.15.39:0/1214890868 >> 172.21.15.39:0/1214890868 conn(0x7f9534074760 msgr2=0x7f95340fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:53:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.520+0000 7f953c1e0700 1 -- 172.21.15.39:0/1214890868 shutdown_connections 2023-07-21T18:53:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:43.520+0000 7f953c1e0700 1 -- 172.21.15.39:0/1214890868 wait complete. 2023-07-21T18:53:44.292 INFO:tasks.workunit.client.0.smithi039.stdout: 1 680001 5.32 MB/sec execute 537 sec latency 323.977 ms 2023-07-21T18:53:44.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:53:44 smithi193 ceph-mon[98204]: pgmap v375: 97 pgs: 97 active+clean; 275 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 1.3 MiB/s wr, 113 op/s 2023-07-21T18:53:44.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:53:44 smithi155 ceph-mon[97934]: pgmap v375: 97 pgs: 97 active+clean; 275 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 1.3 MiB/s wr, 113 op/s 2023-07-21T18:53:44.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:53:44 smithi039 ceph-mon[96489]: pgmap v375: 97 pgs: 97 active+clean; 275 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 1.3 MiB/s wr, 113 op/s 2023-07-21T18:53:45.292 INFO:tasks.workunit.client.0.smithi039.stdout: 1 680001 5.31 MB/sec execute 538 sec latency 1324.071 ms 2023-07-21T18:53:46.293 INFO:tasks.workunit.client.0.smithi039.stdout: 1 680637 5.30 MB/sec execute 539 sec latency 2112.914 ms 2023-07-21T18:53:46.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:53:46 smithi193 ceph-mon[98204]: pgmap v376: 97 pgs: 97 active+clean; 275 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 1.3 MiB/s wr, 113 op/s 2023-07-21T18:53:46.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:53:46 smithi155 ceph-mon[97934]: pgmap v376: 97 pgs: 97 active+clean; 275 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 1.3 MiB/s wr, 113 op/s 2023-07-21T18:53:46.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:53:46 smithi039 ceph-mon[96489]: pgmap v376: 97 pgs: 97 active+clean; 275 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 1.3 MiB/s wr, 113 op/s 2023-07-21T18:53:47.293 INFO:tasks.workunit.client.0.smithi039.stdout: 1 682237 5.32 MB/sec execute 540 sec latency 10.636 ms 2023-07-21T18:53:48.293 INFO:tasks.workunit.client.0.smithi039.stdout: 1 683549 5.32 MB/sec execute 541 sec latency 35.771 ms 2023-07-21T18:53:48.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:53:48 smithi193 ceph-mon[98204]: pgmap v377: 97 pgs: 97 active+clean; 275 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 929 KiB/s rd, 1.9 MiB/s wr, 177 op/s 2023-07-21T18:53:48.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:53:48 smithi155 ceph-mon[97934]: pgmap v377: 97 pgs: 97 active+clean; 275 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 929 KiB/s rd, 1.9 MiB/s wr, 177 op/s 2023-07-21T18:53:48.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:53:48 smithi039 ceph-mon[96489]: pgmap v377: 97 pgs: 97 active+clean; 275 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 929 KiB/s rd, 1.9 MiB/s wr, 177 op/s 2023-07-21T18:53:49.293 INFO:tasks.workunit.client.0.smithi039.stdout: 1 683549 5.31 MB/sec execute 542 sec latency 1028.460 ms 2023-07-21T18:53:50.293 INFO:tasks.workunit.client.0.smithi039.stdout: 1 683549 5.30 MB/sec execute 543 sec latency 2028.584 ms 2023-07-21T18:53:50.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:53:50 smithi193 ceph-mon[98204]: pgmap v378: 97 pgs: 97 active+clean; 275 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 1.4 MiB/s wr, 123 op/s 2023-07-21T18:53:50.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:53:50 smithi155 ceph-mon[97934]: pgmap v378: 97 pgs: 97 active+clean; 275 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 1.4 MiB/s wr, 123 op/s 2023-07-21T18:53:50.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:53:50 smithi039 ceph-mon[96489]: pgmap v378: 97 pgs: 97 active+clean; 275 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 1.4 MiB/s wr, 123 op/s 2023-07-21T18:53:51.293 INFO:tasks.workunit.client.0.smithi039.stdout: 1 683549 5.29 MB/sec execute 544 sec latency 3028.675 ms 2023-07-21T18:53:52.293 INFO:tasks.workunit.client.0.smithi039.stdout: 1 684830 5.29 MB/sec execute 545 sec latency 3452.177 ms 2023-07-21T18:53:52.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:53:52 smithi193 ceph-mon[98204]: pgmap v379: 97 pgs: 97 active+clean; 278 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 2.1 MiB/s wr, 179 op/s 2023-07-21T18:53:52.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:53:52 smithi155 ceph-mon[97934]: pgmap v379: 97 pgs: 97 active+clean; 278 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 2.1 MiB/s wr, 179 op/s 2023-07-21T18:53:52.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:53:52 smithi039 ceph-mon[96489]: pgmap v379: 97 pgs: 97 active+clean; 278 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 2.1 MiB/s wr, 179 op/s 2023-07-21T18:53:53.293 INFO:tasks.workunit.client.0.smithi039.stdout: 1 686116 5.29 MB/sec execute 546 sec latency 25.204 ms 2023-07-21T18:53:54.293 INFO:tasks.workunit.client.0.smithi039.stdout: 1 687092 5.29 MB/sec execute 547 sec latency 381.701 ms 2023-07-21T18:53:54.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:53:54 smithi193 ceph-mon[98204]: pgmap v380: 97 pgs: 97 active+clean; 278 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 1.3 MiB/s wr, 119 op/s 2023-07-21T18:53:54.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:53:54 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:53:54.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:53:54 smithi155 ceph-mon[97934]: pgmap v380: 97 pgs: 97 active+clean; 278 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 1.3 MiB/s wr, 119 op/s 2023-07-21T18:53:54.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:53:54 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:53:54.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:53:54 smithi039 ceph-mon[96489]: pgmap v380: 97 pgs: 97 active+clean; 278 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 1.3 MiB/s wr, 119 op/s 2023-07-21T18:53:54.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:53:54 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:53:55.293 INFO:tasks.workunit.client.0.smithi039.stdout: 1 687092 5.28 MB/sec execute 548 sec latency 1381.792 ms 2023-07-21T18:53:56.293 INFO:tasks.workunit.client.0.smithi039.stdout: 1 687092 5.27 MB/sec execute 549 sec latency 2381.875 ms 2023-07-21T18:53:56.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:53:56 smithi193 ceph-mon[98204]: pgmap v381: 97 pgs: 97 active+clean; 278 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 1.3 MiB/s wr, 119 op/s 2023-07-21T18:53:56.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:53:56 smithi155 ceph-mon[97934]: pgmap v381: 97 pgs: 97 active+clean; 278 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 1.3 MiB/s wr, 119 op/s 2023-07-21T18:53:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.641+0000 7f9a21ccc700 1 -- 172.21.15.39:0/700992357 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a1c104e50 msgr2=0x7f9a1c10ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:53:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.641+0000 7f9a21ccc700 1 --2- 172.21.15.39:0/700992357 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a1c104e50 0x7f9a1c10ca00 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f9a140099c0 tx=0x7f9a14009cd0 comp rx=0 tx=0).stop 2023-07-21T18:53:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.641+0000 7f9a21ccc700 1 -- 172.21.15.39:0/700992357 shutdown_connections 2023-07-21T18:53:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.641+0000 7f9a21ccc700 1 --2- 172.21.15.39:0/700992357 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a1c104e50 0x7f9a1c10ca00 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:56.655 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.641+0000 7f9a21ccc700 1 --2- 172.21.15.39:0/700992357 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a1c067c10 0x7f9a1c067fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:56.655 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.641+0000 7f9a21ccc700 1 --2- 172.21.15.39:0/700992357 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a1c0685b0 0x7f9a1c104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:56.655 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.641+0000 7f9a21ccc700 1 -- 172.21.15.39:0/700992357 >> 172.21.15.39:0/700992357 conn(0x7f9a1c074760 msgr2=0x7f9a1c074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:53:56.655 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.642+0000 7f9a21ccc700 1 -- 172.21.15.39:0/700992357 shutdown_connections 2023-07-21T18:53:56.655 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.642+0000 7f9a21ccc700 1 -- 172.21.15.39:0/700992357 wait complete. 2023-07-21T18:53:56.657 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.642+0000 7f9a21ccc700 1 Processor -- start 2023-07-21T18:53:56.657 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.643+0000 7f9a21ccc700 1 -- start start 2023-07-21T18:53:56.657 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.643+0000 7f9a21ccc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a1c067c10 0x7f9a1c19bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:56.657 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.643+0000 7f9a21ccc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a1c0685b0 0x7f9a1c19c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:56.657 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.643+0000 7f9a21ccc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a1c104e50 0x7f9a1c195980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:56.658 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.643+0000 7f9a21ccc700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1c195ff0 con 0x7f9a1c0685b0 2023-07-21T18:53:56.658 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.643+0000 7f9a21ccc700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1c196160 con 0x7f9a1c067c10 2023-07-21T18:53:56.658 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.643+0000 7f9a21ccc700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1c196320 con 0x7f9a1c104e50 2023-07-21T18:53:56.658 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.643+0000 7f9a1affd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a1c0685b0 0x7f9a1c19c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:56.658 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.643+0000 7f9a1affd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a1c0685b0 0x7f9a1c19c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:32846/0 (socket says 172.21.15.39:32846) 2023-07-21T18:53:56.658 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.643+0000 7f9a1affd700 1 -- 172.21.15.39:0/2697304131 learned_addr learned my addr 172.21.15.39:0/2697304131 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:53:56.658 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.643+0000 7f9a1bfff700 1 --2- 172.21.15.39:0/2697304131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a1c104e50 0x7f9a1c195980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:56.659 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.643+0000 7f9a1b7fe700 1 --2- 172.21.15.39:0/2697304131 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a1c067c10 0x7f9a1c19bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:56.659 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.643+0000 7f9a1affd700 1 -- 172.21.15.39:0/2697304131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a1c104e50 msgr2=0x7f9a1c195980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:53:56.659 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.644+0000 7f9a1affd700 1 --2- 172.21.15.39:0/2697304131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a1c104e50 0x7f9a1c195980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:56.659 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.644+0000 7f9a1affd700 1 -- 172.21.15.39:0/2697304131 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a1c067c10 msgr2=0x7f9a1c19bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:53:56.659 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.644+0000 7f9a1affd700 1 --2- 172.21.15.39:0/2697304131 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a1c067c10 0x7f9a1c19bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:56.659 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.644+0000 7f9a1affd700 1 -- 172.21.15.39:0/2697304131 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a140095f0 con 0x7f9a1c0685b0 2023-07-21T18:53:56.659 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.644+0000 7f9a1b7fe700 1 --2- 172.21.15.39:0/2697304131 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a1c067c10 0x7f9a1c19bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:53:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.644+0000 7f9a1affd700 1 --2- 172.21.15.39:0/2697304131 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a1c0685b0 0x7f9a1c19c060 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f9a1000d950 tx=0x7f9a1000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:53:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.644+0000 7f9a18ff9700 1 -- 172.21.15.39:0/2697304131 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a10009930 con 0x7f9a1c0685b0 2023-07-21T18:53:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.644+0000 7f9a18ff9700 1 -- 172.21.15.39:0/2697304131 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a10010460 con 0x7f9a1c0685b0 2023-07-21T18:53:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.644+0000 7f9a21ccc700 1 -- 172.21.15.39:0/2697304131 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a1c196600 con 0x7f9a1c0685b0 2023-07-21T18:53:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.644+0000 7f9a18ff9700 1 -- 172.21.15.39:0/2697304131 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1000f620 con 0x7f9a1c0685b0 2023-07-21T18:53:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.644+0000 7f9a21ccc700 1 -- 172.21.15.39:0/2697304131 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a1c1a2b10 con 0x7f9a1c0685b0 2023-07-21T18:53:56.662 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.647+0000 7f9a18ff9700 1 -- 172.21.15.39:0/2697304131 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f9a10010860 con 0x7f9a1c0685b0 2023-07-21T18:53:56.662 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.648+0000 7f9a18ff9700 1 --2- 172.21.15.39:0/2697304131 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f9a040a12c0 0x7f9a040a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:56.662 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.648+0000 7f9a18ff9700 1 -- 172.21.15.39:0/2697304131 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f9a100cc430 con 0x7f9a1c0685b0 2023-07-21T18:53:56.662 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.648+0000 7f9a1b7fe700 1 --2- 172.21.15.39:0/2697304131 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f9a040a12c0 0x7f9a040a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:56.662 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.648+0000 7f9a2de52b80 1 Processor -- start 2023-07-21T18:53:56.663 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.648+0000 7f9a2de52b80 1 -- start start 2023-07-21T18:53:56.663 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.649+0000 7f9a2de52b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b42b9436b0 0x55b42baecd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:56.663 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.649+0000 7f9a2de52b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b42baed2d0 0x55b42bae7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:56.663 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.649+0000 7f9a2de52b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b42bae7580 0x55b42bae79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:56.663 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.649+0000 7f9a2de52b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b42b9ecc50 con 0x55b42bae7580 2023-07-21T18:53:56.663 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.649+0000 7f9a2de52b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b42ba3f9e0 con 0x55b42b9436b0 2023-07-21T18:53:56.663 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.649+0000 7f9a2de52b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b42ba38e70 con 0x55b42baed2d0 2023-07-21T18:53:56.664 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.649+0000 7f9a1b7fe700 1 --2- 172.21.15.39:0/2697304131 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f9a040a12c0 0x7f9a040a3770 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c00d340 tx=0x7f9a0c009040 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:53:56.664 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.649+0000 7f9a1bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b42bae7580 0x55b42bae79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:56.664 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.649+0000 7f9a1bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b42bae7580 0x55b42bae79d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:32856/0 (socket says 172.21.15.39:32856) 2023-07-21T18:53:56.664 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.649+0000 7f9a1bfff700 1 -- 172.21.15.39:0/1245682919 learned_addr learned my addr 172.21.15.39:0/1245682919 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:53:56.664 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.649+0000 7f9a1b7fe700 1 --2- 172.21.15.39:0/1245682919 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b42b9436b0 0x55b42baecd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:56.664 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.649+0000 7f9a1affd700 1 --2- 172.21.15.39:0/1245682919 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b42baed2d0 0x55b42bae7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:56.664 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.649+0000 7f9a1bfff700 1 -- 172.21.15.39:0/1245682919 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b42baed2d0 msgr2=0x55b42bae7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:53:56.665 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.649+0000 7f9a1bfff700 1 --2- 172.21.15.39:0/1245682919 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b42baed2d0 0x55b42bae7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:56.665 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.649+0000 7f9a1bfff700 1 -- 172.21.15.39:0/1245682919 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b42b9436b0 msgr2=0x55b42baecd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:53:56.665 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.649+0000 7f9a1bfff700 1 --2- 172.21.15.39:0/1245682919 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b42b9436b0 0x55b42baecd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:56.665 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.649+0000 7f9a1bfff700 1 -- 172.21.15.39:0/1245682919 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b42b866d80 con 0x55b42bae7580 2023-07-21T18:53:56.665 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.649+0000 7f9a1bfff700 1 --2- 172.21.15.39:0/1245682919 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b42bae7580 0x55b42bae79d0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f9a140099c0 tx=0x7f9a14005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9532bdfe17e64a01 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:53:56.665 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.650+0000 7f9a01ffb700 1 -- 172.21.15.39:0/1245682919 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1401d070 con 0x55b42bae7580 2023-07-21T18:53:56.665 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.650+0000 7f9a01ffb700 1 -- 172.21.15.39:0/1245682919 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a1401e040 con 0x55b42bae7580 2023-07-21T18:53:56.665 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.650+0000 7f9a01ffb700 1 -- 172.21.15.39:0/1245682919 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a14003b20 con 0x55b42bae7580 2023-07-21T18:53:56.667 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.650+0000 7f9a2de52b80 1 -- 172.21.15.39:0/1245682919 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b42bae7580 msgr2=0x55b42bae79d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:53:56.667 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.650+0000 7f9a2de52b80 1 --2- 172.21.15.39:0/1245682919 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b42bae7580 0x55b42bae79d0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f9a140099c0 tx=0x7f9a14005bc0 comp rx=0 tx=0).stop 2023-07-21T18:53:56.668 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.650+0000 7f9a2de52b80 1 -- 172.21.15.39:0/1245682919 shutdown_connections 2023-07-21T18:53:56.668 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.650+0000 7f9a2de52b80 1 --2- 172.21.15.39:0/1245682919 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b42b9436b0 0x55b42baecd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:56.668 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.650+0000 7f9a2de52b80 1 --2- 172.21.15.39:0/1245682919 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b42bae7580 0x55b42bae79d0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:56.668 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.650+0000 7f9a2de52b80 1 --2- 172.21.15.39:0/1245682919 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b42baed2d0 0x55b42bae7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:56.668 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.650+0000 7f9a2de52b80 1 -- 172.21.15.39:0/1245682919 >> 172.21.15.39:0/1245682919 conn(0x55b42badcbe0 msgr2=0x55b42bae1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:53:56.668 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.650+0000 7f9a2de52b80 1 -- 172.21.15.39:0/1245682919 shutdown_connections 2023-07-21T18:53:56.668 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.650+0000 7f9a2de52b80 1 -- 172.21.15.39:0/1245682919 wait complete. 2023-07-21T18:53:56.669 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.651+0000 7f9a2de52b80 1 -- start start 2023-07-21T18:53:56.669 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.651+0000 7f9a2de52b80 1 Processor -- start 2023-07-21T18:53:56.669 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.651+0000 7f9a2de52b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b42b9436b0 0x55b42baf3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:56.669 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.651+0000 7f9a2de52b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b42bae7580 0x55b42baf0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:56.669 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.651+0000 7f9a2de52b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b42baed2d0 0x55b42baf0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:56.669 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.651+0000 7f9a2de52b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b42b9d8a60 con 0x55b42bae7580 2023-07-21T18:53:56.669 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.651+0000 7f9a2de52b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b42b9d8760 con 0x55b42b9436b0 2023-07-21T18:53:56.670 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.651+0000 7f9a2de52b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b42b8bc8c0 con 0x55b42baed2d0 2023-07-21T18:53:56.670 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.651+0000 7f9a2de52b80 10 client.0 _ll_register_callbacks cb 0x55b42b9d6700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:53:56.670 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.651+0000 7f9a1affd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b42bae7580 0x55b42baf0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:56.670 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.651+0000 7f9a1affd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b42bae7580 0x55b42baf0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:32860/0 (socket says 172.21.15.39:32860) 2023-07-21T18:53:56.670 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.651+0000 7f9a1affd700 1 -- 172.21.15.39:0/208367151 learned_addr learned my addr 172.21.15.39:0/208367151 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:53:56.670 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.652+0000 7f9a1b7fe700 1 --2- 172.21.15.39:0/208367151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b42b9436b0 0x55b42baf3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:56.670 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.652+0000 7f9a1bfff700 1 --2- 172.21.15.39:0/208367151 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b42baed2d0 0x55b42baf0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:56.670 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.652+0000 7f9a1affd700 1 -- 172.21.15.39:0/208367151 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b42baed2d0 msgr2=0x55b42baf0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:53:56.671 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.652+0000 7f9a1affd700 1 --2- 172.21.15.39:0/208367151 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b42baed2d0 0x55b42baf0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:56.671 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.652+0000 7f9a1affd700 1 -- 172.21.15.39:0/208367151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b42b9436b0 msgr2=0x55b42baf3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:53:56.671 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.652+0000 7f9a1affd700 1 --2- 172.21.15.39:0/208367151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b42b9436b0 0x55b42baf3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:56.671 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.652+0000 7f9a1affd700 1 -- 172.21.15.39:0/208367151 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a1c10d210 con 0x55b42bae7580 2023-07-21T18:53:56.671 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.652+0000 7f9a1affd700 1 --2- 172.21.15.39:0/208367151 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b42bae7580 0x55b42baf0880 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f9a10009520 tx=0x7f9a10098ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b7a786ebd0ca91 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:53:56.671 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.652+0000 7f99faffd700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:53:56.671 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.652+0000 7f99faffd700 1 -- 172.21.15.39:0/208367151 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a100a18f0 con 0x55b42bae7580 2023-07-21T18:53:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.652+0000 7f99faffd700 1 -- 172.21.15.39:0/208367151 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f9a1009d0d0 con 0x55b42bae7580 2023-07-21T18:53:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.652+0000 7f9a2de52b80 1 -- 172.21.15.39:0/208367151 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b42b992490 con 0x55b42bae7580 2023-07-21T18:53:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.652+0000 7f99faffd700 1 -- 172.21.15.39:0/208367151 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a1009d290 con 0x55b42bae7580 2023-07-21T18:53:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.652+0000 7f9a2de52b80 10 client.17082 fetch_fsmap learned FSMap version 70 2023-07-21T18:53:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.652+0000 7f9a2de52b80 1 -- 172.21.15.39:0/208367151 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x55b42b84fc00 con 0x55b42bae7580 2023-07-21T18:53:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.653+0000 7f99faffd700 1 -- 172.21.15.39:0/208367151 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b42b992490 con 0x55b42bae7580 2023-07-21T18:53:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.653+0000 7f99faffd700 1 -- 172.21.15.39:0/208367151 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 70) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f9a10099820 con 0x55b42bae7580 2023-07-21T18:53:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.653+0000 7f9a2de52b80 10 client.17082 fetch_fsmap finished waiting for FSMap version 70 2023-07-21T18:53:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.653+0000 7f9a2de52b80 10 client.17082 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:53:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.653+0000 7f9a2de52b80 20 client.17082 populate_metadata read hostname 'smithi039' 2023-07-21T18:53:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.653+0000 7f9a2de52b80 1 --2- 172.21.15.39:0/208367151 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55b42bb00a80 0x55b42bb02e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.653+0000 7f9a2de52b80 4 client.17082 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:53:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.653+0000 7f9a2de52b80 1 -- 172.21.15.39:0/208367151 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b42ba31e50 con 0x55b42bb00a80 2023-07-21T18:53:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.653+0000 7f9a1b7fe700 1 --2- 172.21.15.39:0/208367151 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55b42bb00a80 0x55b42bb02e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.654+0000 7f9a1b7fe700 1 --2- 172.21.15.39:0/208367151 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55b42bb00a80 0x55b42bb02e90 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f715da8c9b2ae1b2 server_cookie=eeacf087fd16e645 in_seq=0 out_seq=0 2023-07-21T18:53:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.654+0000 7f99faffd700 10 client.17082 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:53:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.654+0000 7f99faffd700 1 -- 172.21.15.39:0/208367151 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b42ba31e50 con 0x55b42bb00a80 2023-07-21T18:53:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.654+0000 7f99faffd700 10 client.17082 handle_command_reply: tid=0 2023-07-21T18:53:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.654+0000 7f9a2de52b80 1 client.17082 shutdown 2023-07-21T18:53:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.654+0000 7f9a2de52b80 1 -- 172.21.15.39:0/208367151 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b42bae7580 msgr2=0x55b42baf0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:53:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.654+0000 7f9a2de52b80 1 --2- 172.21.15.39:0/208367151 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b42bae7580 0x55b42baf0880 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f9a10009520 tx=0x7f9a10098ed0 comp rx=0 tx=0).stop 2023-07-21T18:53:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.654+0000 7f9a2de52b80 1 -- 172.21.15.39:0/208367151 shutdown_connections 2023-07-21T18:53:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.654+0000 7f9a2de52b80 1 --2- 172.21.15.39:0/208367151 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55b42bb00a80 0x55b42bb02e90 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.654+0000 7f9a2de52b80 1 --2- 172.21.15.39:0/208367151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b42b9436b0 0x55b42baf3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.654+0000 7f9a2de52b80 1 --2- 172.21.15.39:0/208367151 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b42baed2d0 0x55b42baf0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.654+0000 7f9a2de52b80 1 --2- 172.21.15.39:0/208367151 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b42bae7580 0x55b42baf0880 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.654+0000 7f99faffd700 0 client.17082 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:53:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.654+0000 7f9a2de52b80 1 -- 172.21.15.39:0/208367151 >> 172.21.15.39:0/208367151 conn(0x55b42badcbe0 msgr2=0x55b42bae0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:53:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.654+0000 7f9a2de52b80 1 -- 172.21.15.39:0/208367151 shutdown_connections 2023-07-21T18:53:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.654+0000 7f9a2de52b80 1 -- 172.21.15.39:0/208367151 wait complete. 2023-07-21T18:53:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.655+0000 7f9a2de52b80 20 client.17082 trim_cache size 0 max 16384 2023-07-21T18:53:56.678 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:53:56 smithi039 ceph-mon[96489]: pgmap v381: 97 pgs: 97 active+clean; 278 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 1.3 MiB/s wr, 119 op/s 2023-07-21T18:53:56.682 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.669+0000 7f9a2de52b80 1 Processor -- start 2023-07-21T18:53:56.682 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.669+0000 7f9a2de52b80 1 -- start start 2023-07-21T18:53:56.682 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.669+0000 7f9a2de52b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b42baeb200 0x55b42baf07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:56.682 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.669+0000 7f9a2de52b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b42badcbe0 0x55b42baec2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:56.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.670+0000 7f9a2de52b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b42b9436b0 0x55b42baf61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:56.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.670+0000 7f9a2de52b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b42bae36f0 con 0x55b42b9436b0 2023-07-21T18:53:56.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.670+0000 7f9a2de52b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b42bae3b50 con 0x55b42baeb200 2023-07-21T18:53:56.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.670+0000 7f9a2de52b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b42baec9f0 con 0x55b42badcbe0 2023-07-21T18:53:56.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.670+0000 7f9a1bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b42b9436b0 0x55b42baf61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:56.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.670+0000 7f9a1bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b42b9436b0 0x55b42baf61c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:32870/0 (socket says 172.21.15.39:32870) 2023-07-21T18:53:56.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.670+0000 7f9a1bfff700 1 -- 172.21.15.39:0/3672546912 learned_addr learned my addr 172.21.15.39:0/3672546912 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:53:56.684 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.670+0000 7f9a1b7fe700 1 --2- 172.21.15.39:0/3672546912 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b42baeb200 0x55b42baf07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:56.684 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.670+0000 7f9a1affd700 1 --2- 172.21.15.39:0/3672546912 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b42badcbe0 0x55b42baec2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:56.684 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.670+0000 7f9a1bfff700 1 -- 172.21.15.39:0/3672546912 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b42badcbe0 msgr2=0x55b42baec2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:53:56.684 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.670+0000 7f9a1bfff700 1 --2- 172.21.15.39:0/3672546912 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b42badcbe0 0x55b42baec2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:56.684 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.670+0000 7f9a1bfff700 1 -- 172.21.15.39:0/3672546912 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b42baeb200 msgr2=0x55b42baf07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:53:56.684 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.670+0000 7f9a1bfff700 1 --2- 172.21.15.39:0/3672546912 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b42baeb200 0x55b42baf07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:56.684 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.670+0000 7f9a1bfff700 1 -- 172.21.15.39:0/3672546912 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b42bb07ac0 con 0x55b42b9436b0 2023-07-21T18:53:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.670+0000 7f9a1affd700 1 --2- 172.21.15.39:0/3672546912 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b42badcbe0 0x55b42baec2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:53:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.670+0000 7f9a1bfff700 1 --2- 172.21.15.39:0/3672546912 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b42b9436b0 0x55b42baf61c0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f9a14009f10 tx=0x7f9a14009f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dca7832980af315 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:53:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.670+0000 7f9a027fc700 1 -- 172.21.15.39:0/3672546912 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1401d040 con 0x55b42b9436b0 2023-07-21T18:53:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.670+0000 7f9a027fc700 1 -- 172.21.15.39:0/3672546912 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a14031070 con 0x55b42b9436b0 2023-07-21T18:53:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.670+0000 7f9a027fc700 1 -- 172.21.15.39:0/3672546912 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a14003b20 con 0x55b42b9436b0 2023-07-21T18:53:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.671+0000 7f9a2de52b80 1 -- 172.21.15.39:0/3672546912 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b42b9436b0 msgr2=0x55b42baf61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:53:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.671+0000 7f9a2de52b80 1 --2- 172.21.15.39:0/3672546912 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b42b9436b0 0x55b42baf61c0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f9a14009f10 tx=0x7f9a14009f40 comp rx=0 tx=0).stop 2023-07-21T18:53:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.671+0000 7f9a2de52b80 1 -- 172.21.15.39:0/3672546912 shutdown_connections 2023-07-21T18:53:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.671+0000 7f9a2de52b80 1 --2- 172.21.15.39:0/3672546912 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b42baeb200 0x55b42baf07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.671+0000 7f9a2de52b80 1 --2- 172.21.15.39:0/3672546912 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b42b9436b0 0x55b42baf61c0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.671+0000 7f9a2de52b80 1 --2- 172.21.15.39:0/3672546912 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b42badcbe0 0x55b42baec2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.671+0000 7f9a2de52b80 1 -- 172.21.15.39:0/3672546912 >> 172.21.15.39:0/3672546912 conn(0x55b42baeae30 msgr2=0x55b42baeee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:53:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.671+0000 7f9a2de52b80 1 -- 172.21.15.39:0/3672546912 shutdown_connections 2023-07-21T18:53:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.671+0000 7f9a2de52b80 1 -- 172.21.15.39:0/3672546912 wait complete. 2023-07-21T18:53:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.671+0000 7f9a2de52b80 1 -- start start 2023-07-21T18:53:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.672+0000 7f9a2de52b80 1 Processor -- start 2023-07-21T18:53:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.672+0000 7f9a2de52b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b42b9436b0 0x55b42bb48ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.672+0000 7f9a2de52b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b42badcbe0 0x55b42bb49000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.672+0000 7f9a2de52b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b42baeb200 0x55b42bb4cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.672+0000 7f9a2de52b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b42bb4d150 con 0x55b42badcbe0 2023-07-21T18:53:56.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.672+0000 7f9a2de52b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b42bb4d2c0 con 0x55b42baeb200 2023-07-21T18:53:56.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.672+0000 7f9a2de52b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b42bb4d480 con 0x55b42b9436b0 2023-07-21T18:53:56.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.672+0000 7f9a2de52b80 10 client.0 _ll_register_callbacks cb 0x7f9a10099360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:53:56.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.672+0000 7f9a1affd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b42badcbe0 0x55b42bb49000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:56.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.672+0000 7f9a1b7fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b42b9436b0 0x55b42bb48ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:56.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.672+0000 7f9a1b7fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b42b9436b0 0x55b42bb48ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:51424/0 (socket says 172.21.15.39:51424) 2023-07-21T18:53:56.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.672+0000 7f9a1affd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b42badcbe0 0x55b42bb49000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:32874/0 (socket says 172.21.15.39:32874) 2023-07-21T18:53:56.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.672+0000 7f9a1b7fe700 1 -- 172.21.15.39:0/1397733708 learned_addr learned my addr 172.21.15.39:0/1397733708 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:53:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.672+0000 7f9a1bfff700 1 --2- 172.21.15.39:0/1397733708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b42baeb200 0x55b42bb4cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.672+0000 7f9a1b7fe700 1 -- 172.21.15.39:0/1397733708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b42baeb200 msgr2=0x55b42bb4cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:53:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.672+0000 7f9a1b7fe700 1 --2- 172.21.15.39:0/1397733708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b42baeb200 0x55b42bb4cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.672+0000 7f9a1b7fe700 1 -- 172.21.15.39:0/1397733708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b42badcbe0 msgr2=0x55b42bb49000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:53:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.672+0000 7f9a1b7fe700 1 --2- 172.21.15.39:0/1397733708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b42badcbe0 0x55b42bb49000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.672+0000 7f9a1b7fe700 1 -- 172.21.15.39:0/1397733708 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b42b866d80 con 0x55b42b9436b0 2023-07-21T18:53:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.673+0000 7f9a1b7fe700 1 --2- 172.21.15.39:0/1397733708 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b42b9436b0 0x55b42bb48ac0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x55b42bb037a0 tx=0x7f9a0c010000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1ad8e81ec1b1f25 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:53:56.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.673+0000 7f99faffd700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2023-07-21T18:53:56.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.673+0000 7f99faffd700 1 -- 172.21.15.39:0/1397733708 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c02c070 con 0x55b42b9436b0 2023-07-21T18:53:56.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.673+0000 7f99faffd700 1 -- 172.21.15.39:0/1397733708 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f9a0c0226e0 con 0x55b42b9436b0 2023-07-21T18:53:56.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.673+0000 7f9a2de52b80 1 -- 172.21.15.39:0/1397733708 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9a1401e040 con 0x55b42b9436b0 2023-07-21T18:53:56.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.673+0000 7f99faffd700 1 -- 172.21.15.39:0/1397733708 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a0c0228a0 con 0x55b42b9436b0 2023-07-21T18:53:56.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.673+0000 7f9a2de52b80 10 client.26554 fetch_fsmap learned FSMap version 70 2023-07-21T18:53:56.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.673+0000 7f9a2de52b80 1 -- 172.21.15.39:0/1397733708 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x55b42b88d1f0 con 0x55b42b9436b0 2023-07-21T18:53:56.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.673+0000 7f99faffd700 1 -- 172.21.15.39:0/1397733708 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a1401e040 con 0x55b42b9436b0 2023-07-21T18:53:56.693 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.673+0000 7f99faffd700 1 -- 172.21.15.39:0/1397733708 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 70) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f9a0c030b20 con 0x55b42b9436b0 2023-07-21T18:53:56.693 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.673+0000 7f9a2de52b80 10 client.26554 fetch_fsmap finished waiting for FSMap version 70 2023-07-21T18:53:56.693 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.673+0000 7f9a2de52b80 10 client.26554 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:53:56.693 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.674+0000 7f9a2de52b80 20 client.26554 populate_metadata read hostname 'smithi039' 2023-07-21T18:53:56.693 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.674+0000 7f9a2de52b80 1 --2- 172.21.15.39:0/1397733708 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55b42bae7580 0x55b42bb57e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:53:56.693 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.674+0000 7f9a2de52b80 4 client.26554 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:53:56.693 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.674+0000 7f9a2de52b80 1 -- 172.21.15.39:0/1397733708 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b42b9f3ff0 con 0x55b42bae7580 2023-07-21T18:53:56.694 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.674+0000 7f9a1affd700 1 --2- 172.21.15.39:0/1397733708 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55b42bae7580 0x55b42bb57e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:53:56.694 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.674+0000 7f99faffd700 10 client.26554 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:53:56.694 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.674+0000 7f9a1affd700 1 --2- 172.21.15.39:0/1397733708 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55b42bae7580 0x55b42bb57e70 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24c9cd97e68a9d53 server_cookie=fbb5abe2397c6b4d in_seq=0 out_seq=0 2023-07-21T18:53:56.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.716+0000 7f99faffd700 1 -- 172.21.15.39:0/1397733708 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b42b9f3ff0 con 0x55b42bae7580 2023-07-21T18:53:56.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.716+0000 7f99faffd700 10 client.26554 handle_command_reply: tid=0 2023-07-21T18:53:56.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.716+0000 7f9a2de52b80 1 client.26554 shutdown 2023-07-21T18:53:56.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.716+0000 7f9a2de52b80 1 -- 172.21.15.39:0/1397733708 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b42b9436b0 msgr2=0x55b42bb48ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:53:56.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.716+0000 7f9a2de52b80 1 --2- 172.21.15.39:0/1397733708 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b42b9436b0 0x55b42bb48ac0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x55b42bb037a0 tx=0x7f9a0c010000 comp rx=0 tx=0).stop 2023-07-21T18:53:56.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.716+0000 7f9a2de52b80 1 -- 172.21.15.39:0/1397733708 shutdown_connections 2023-07-21T18:53:56.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.716+0000 7f9a2de52b80 1 --2- 172.21.15.39:0/1397733708 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55b42bae7580 0x55b42bb57e70 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:56.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.716+0000 7f9a2de52b80 1 --2- 172.21.15.39:0/1397733708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b42baeb200 0x55b42bb4cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.716+0000 7f9a2de52b80 1 --2- 172.21.15.39:0/1397733708 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b42b9436b0 0x55b42bb48ac0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.716+0000 7f99faffd700 0 client.26554 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:53:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.716+0000 7f9a2de52b80 1 --2- 172.21.15.39:0/1397733708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b42badcbe0 0x55b42bb49000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.716+0000 7f9a2de52b80 1 -- 172.21.15.39:0/1397733708 >> 172.21.15.39:0/1397733708 conn(0x55b42baeae30 msgr2=0x55b42baeee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:53:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.716+0000 7f9a2de52b80 1 -- 172.21.15.39:0/1397733708 shutdown_connections 2023-07-21T18:53:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.716+0000 7f9a2de52b80 1 -- 172.21.15.39:0/1397733708 wait complete. 2023-07-21T18:53:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.716+0000 7f9a2de52b80 20 client.26554 trim_cache size 0 max 16384 2023-07-21T18:53:56.731 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:53:56.731 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:53:56.731 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:53:56.731 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:53:56.732 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.717+0000 7f9a21ccc700 1 -- 172.21.15.39:0/2697304131 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f9a040a12c0 msgr2=0x7f9a040a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:53:56.732 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.717+0000 7f9a21ccc700 1 --2- 172.21.15.39:0/2697304131 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f9a040a12c0 0x7f9a040a3770 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c00d340 tx=0x7f9a0c009040 comp rx=0 tx=0).stop 2023-07-21T18:53:56.732 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.717+0000 7f9a21ccc700 1 -- 172.21.15.39:0/2697304131 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a1c0685b0 msgr2=0x7f9a1c19c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:53:56.732 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.717+0000 7f9a21ccc700 1 --2- 172.21.15.39:0/2697304131 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a1c0685b0 0x7f9a1c19c060 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f9a1000d950 tx=0x7f9a1000dc60 comp rx=0 tx=0).stop 2023-07-21T18:53:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.717+0000 7f9a21ccc700 1 -- 172.21.15.39:0/2697304131 shutdown_connections 2023-07-21T18:53:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.717+0000 7f9a21ccc700 1 --2- 172.21.15.39:0/2697304131 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f9a040a12c0 0x7f9a040a3770 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.717+0000 7f9a21ccc700 1 --2- 172.21.15.39:0/2697304131 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a1c067c10 0x7f9a1c19bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.717+0000 7f9a21ccc700 1 --2- 172.21.15.39:0/2697304131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a1c104e50 0x7f9a1c195980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.717+0000 7f9a21ccc700 1 --2- 172.21.15.39:0/2697304131 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a1c0685b0 0x7f9a1c19c060 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:53:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.717+0000 7f9a21ccc700 1 -- 172.21.15.39:0/2697304131 >> 172.21.15.39:0/2697304131 conn(0x7f9a1c074760 msgr2=0x7f9a1c0fc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:53:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.717+0000 7f9a21ccc700 1 -- 172.21.15.39:0/2697304131 shutdown_connections 2023-07-21T18:53:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:53:56.717+0000 7f9a21ccc700 1 -- 172.21.15.39:0/2697304131 wait complete. 2023-07-21T18:53:57.293 INFO:tasks.workunit.client.0.smithi039.stdout: 1 688329 5.28 MB/sec execute 550 sec latency 2807.670 ms 2023-07-21T18:53:58.293 INFO:tasks.workunit.client.0.smithi039.stdout: 1 689568 5.28 MB/sec execute 551 sec latency 22.669 ms 2023-07-21T18:53:58.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:53:58 smithi193 ceph-mon[98204]: pgmap v382: 97 pgs: 97 active+clean; 281 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 952 KiB/s rd, 1.9 MiB/s wr, 174 op/s 2023-07-21T18:53:58.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:53:58 smithi155 ceph-mon[97934]: pgmap v382: 97 pgs: 97 active+clean; 281 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 952 KiB/s rd, 1.9 MiB/s wr, 174 op/s 2023-07-21T18:53:58.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:53:58 smithi039 ceph-mon[96489]: pgmap v382: 97 pgs: 97 active+clean; 281 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 952 KiB/s rd, 1.9 MiB/s wr, 174 op/s 2023-07-21T18:53:59.293 INFO:tasks.workunit.client.0.smithi039.stdout: 1 690640 5.28 MB/sec execute 552 sec latency 319.344 ms 2023-07-21T18:54:00.294 INFO:tasks.workunit.client.0.smithi039.stdout: 1 690640 5.27 MB/sec execute 553 sec latency 1319.457 ms 2023-07-21T18:54:00.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:54:00 smithi193 ceph-mon[98204]: pgmap v383: 97 pgs: 97 active+clean; 281 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 610 KiB/s rd, 1.3 MiB/s wr, 111 op/s 2023-07-21T18:54:00.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:54:00 smithi155 ceph-mon[97934]: pgmap v383: 97 pgs: 97 active+clean; 281 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 610 KiB/s rd, 1.3 MiB/s wr, 111 op/s 2023-07-21T18:54:00.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:54:00 smithi039 ceph-mon[96489]: pgmap v383: 97 pgs: 97 active+clean; 281 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 610 KiB/s rd, 1.3 MiB/s wr, 111 op/s 2023-07-21T18:54:01.294 INFO:tasks.workunit.client.0.smithi039.stdout: 1 690640 5.26 MB/sec execute 554 sec latency 2319.543 ms 2023-07-21T18:54:02.294 INFO:tasks.workunit.client.0.smithi039.stdout: 1 691240 5.25 MB/sec execute 555 sec latency 2742.899 ms 2023-07-21T18:54:02.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:54:02 smithi193 ceph-mon[98204]: pgmap v384: 97 pgs: 97 active+clean; 281 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 901 KiB/s rd, 1.9 MiB/s wr, 176 op/s 2023-07-21T18:54:02.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:54:02 smithi155 ceph-mon[97934]: pgmap v384: 97 pgs: 97 active+clean; 281 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 901 KiB/s rd, 1.9 MiB/s wr, 176 op/s 2023-07-21T18:54:02.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:54:02 smithi039 ceph-mon[96489]: pgmap v384: 97 pgs: 97 active+clean; 281 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 901 KiB/s rd, 1.9 MiB/s wr, 176 op/s 2023-07-21T18:54:03.294 INFO:tasks.workunit.client.0.smithi039.stdout: 1 692735 5.27 MB/sec execute 556 sec latency 24.717 ms 2023-07-21T18:54:03.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:54:03 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:54:03.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:54:03 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:54:03.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:54:03 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:54:04.294 INFO:tasks.workunit.client.0.smithi039.stdout: 1 693892 5.27 MB/sec execute 557 sec latency 24.237 ms 2023-07-21T18:54:04.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:54:04 smithi193 ceph-mon[98204]: pgmap v385: 97 pgs: 97 active+clean; 281 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2023-07-21T18:54:04.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:54:04 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:54:04.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:54:04 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:54:04.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:54:04 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:54:04.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:54:04 smithi155 ceph-mon[97934]: pgmap v385: 97 pgs: 97 active+clean; 281 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2023-07-21T18:54:04.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:54:04 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:54:04.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:54:04 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:54:04.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:54:04 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:54:04.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:54:04 smithi039 ceph-mon[96489]: pgmap v385: 97 pgs: 97 active+clean; 281 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2023-07-21T18:54:04.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:54:04 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:54:04.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:54:04 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:54:04.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:54:04 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:54:05.294 INFO:tasks.workunit.client.0.smithi039.stdout: 1 694183 5.26 MB/sec execute 558 sec latency 578.139 ms 2023-07-21T18:54:05.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:54:05 smithi193 ceph-mon[98204]: osdmap e96: 12 total, 12 up, 12 in 2023-07-21T18:54:05.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:54:05 smithi155 ceph-mon[97934]: osdmap e96: 12 total, 12 up, 12 in 2023-07-21T18:54:05.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:54:05 smithi039 ceph-mon[96489]: osdmap e96: 12 total, 12 up, 12 in 2023-07-21T18:54:06.294 INFO:tasks.workunit.client.0.smithi039.stdout: 1 694813 5.25 MB/sec execute 559 sec latency 1367.539 ms 2023-07-21T18:54:06.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:54:06 smithi193 ceph-mon[98204]: pgmap v387: 97 pgs: 97 active+clean; 281 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 1.5 MiB/s wr, 143 op/s 2023-07-21T18:54:06.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:54:06 smithi155 ceph-mon[97934]: pgmap v387: 97 pgs: 97 active+clean; 281 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 1.5 MiB/s wr, 143 op/s 2023-07-21T18:54:06.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:54:06 smithi039 ceph-mon[96489]: pgmap v387: 97 pgs: 97 active+clean; 281 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 1.5 MiB/s wr, 143 op/s 2023-07-21T18:54:07.294 INFO:tasks.workunit.client.0.smithi039.stdout: 1 696349 5.26 MB/sec execute 560 sec latency 13.457 ms 2023-07-21T18:54:08.294 INFO:tasks.workunit.client.0.smithi039.stdout: 1 697535 5.26 MB/sec execute 561 sec latency 29.142 ms 2023-07-21T18:54:08.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:54:08 smithi193 ceph-mon[98204]: pgmap v388: 97 pgs: 97 active+clean; 263 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 1.6 MiB/s wr, 184 op/s 2023-07-21T18:54:08.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:54:08 smithi155 ceph-mon[97934]: pgmap v388: 97 pgs: 97 active+clean; 263 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 1.6 MiB/s wr, 184 op/s 2023-07-21T18:54:08.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:54:08 smithi039 ceph-mon[96489]: pgmap v388: 97 pgs: 97 active+clean; 263 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 1.6 MiB/s wr, 184 op/s 2023-07-21T18:54:09.294 INFO:tasks.workunit.client.0.smithi039.stdout: 1 697732 5.25 MB/sec execute 562 sec latency 743.418 ms 2023-07-21T18:54:09.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:54:09 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:54:09.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:54:09 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:54:09.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:54:09 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:54:09.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.854+0000 7fbbc625f700 1 -- 172.21.15.39:0/3487309787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbc0105b80 msgr2=0x7fbbc0105f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:54:09.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.854+0000 7fbbc625f700 1 --2- 172.21.15.39:0/3487309787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbc0105b80 0x7fbbc0105f50 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fbbb4009990 tx=0x7fbbb4009ca0 comp rx=0 tx=0).stop 2023-07-21T18:54:09.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.854+0000 7fbbc625f700 1 -- 172.21.15.39:0/3487309787 shutdown_connections 2023-07-21T18:54:09.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.854+0000 7fbbc625f700 1 --2- 172.21.15.39:0/3487309787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbbc0100500 0x7fbbc010c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:09.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.854+0000 7fbbc625f700 1 --2- 172.21.15.39:0/3487309787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbc0105b80 0x7fbbc0105f50 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:09.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.854+0000 7fbbc625f700 1 --2- 172.21.15.39:0/3487309787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbbc00ffb50 0x7fbbc00fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:09.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.854+0000 7fbbc625f700 1 -- 172.21.15.39:0/3487309787 >> 172.21.15.39:0/3487309787 conn(0x7fbbc00fb5f0 msgr2=0x7fbbc00fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:54:09.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.855+0000 7fbbc625f700 1 -- 172.21.15.39:0/3487309787 shutdown_connections 2023-07-21T18:54:09.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.855+0000 7fbbc625f700 1 -- 172.21.15.39:0/3487309787 wait complete. 2023-07-21T18:54:09.870 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.855+0000 7fbbc625f700 1 Processor -- start 2023-07-21T18:54:09.870 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.855+0000 7fbbc625f700 1 -- start start 2023-07-21T18:54:09.870 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.856+0000 7fbbc625f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbbc00ffb50 0x7fbbc019cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:09.870 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.856+0000 7fbbc625f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbc0100500 0x7fbbc019d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:09.870 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.856+0000 7fbbc625f700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbbc0105b80 0x7fbbc0197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:09.870 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.856+0000 7fbbc625f700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc01983f0 con 0x7fbbc0100500 2023-07-21T18:54:09.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.856+0000 7fbbc625f700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc0198560 con 0x7fbbc00ffb50 2023-07-21T18:54:09.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.856+0000 7fbbc625f700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc0198720 con 0x7fbbc0105b80 2023-07-21T18:54:09.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.856+0000 7fbbbeffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbc0100500 0x7fbbc019d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:09.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.856+0000 7fbbbeffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbc0100500 0x7fbbc019d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:32886/0 (socket says 172.21.15.39:32886) 2023-07-21T18:54:09.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.856+0000 7fbbbeffd700 1 -- 172.21.15.39:0/2480741153 learned_addr learned my addr 172.21.15.39:0/2480741153 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:54:09.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.856+0000 7fbbbffff700 1 --2- 172.21.15.39:0/2480741153 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbbc0105b80 0x7fbbc0197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:09.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.856+0000 7fbbbf7fe700 1 --2- 172.21.15.39:0/2480741153 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbbc00ffb50 0x7fbbc019cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:09.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.856+0000 7fbbbeffd700 1 -- 172.21.15.39:0/2480741153 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbbc0105b80 msgr2=0x7fbbc0197c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:54:09.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.856+0000 7fbbbeffd700 1 --2- 172.21.15.39:0/2480741153 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbbc0105b80 0x7fbbc0197c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:09.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.856+0000 7fbbbeffd700 1 -- 172.21.15.39:0/2480741153 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbbc00ffb50 msgr2=0x7fbbc019cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:54:09.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.856+0000 7fbbbeffd700 1 --2- 172.21.15.39:0/2480741153 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbbc00ffb50 0x7fbbc019cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:09.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.856+0000 7fbbbeffd700 1 -- 172.21.15.39:0/2480741153 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbb40095f0 con 0x7fbbc0100500 2023-07-21T18:54:09.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.856+0000 7fbbbf7fe700 1 --2- 172.21.15.39:0/2480741153 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbbc00ffb50 0x7fbbc019cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:54:09.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.856+0000 7fbbbeffd700 1 --2- 172.21.15.39:0/2480741153 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbc0100500 0x7fbbc019d190 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fbbb000d950 tx=0x7fbbb000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:54:09.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.856+0000 7fbbbcff9700 1 -- 172.21.15.39:0/2480741153 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb0009940 con 0x7fbbc0100500 2023-07-21T18:54:09.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.857+0000 7fbbbcff9700 1 -- 172.21.15.39:0/2480741153 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbbb0010460 con 0x7fbbc0100500 2023-07-21T18:54:09.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.857+0000 7fbbc625f700 1 -- 172.21.15.39:0/2480741153 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbc0198a00 con 0x7fbbc0100500 2023-07-21T18:54:09.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.857+0000 7fbbbcff9700 1 -- 172.21.15.39:0/2480741153 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb000f620 con 0x7fbbc0100500 2023-07-21T18:54:09.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.857+0000 7fbbc625f700 1 -- 172.21.15.39:0/2480741153 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbc01a4da0 con 0x7fbbc0100500 2023-07-21T18:54:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.859+0000 7fbbbcff9700 1 -- 172.21.15.39:0/2480741153 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fbbb00108d0 con 0x7fbbc0100500 2023-07-21T18:54:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.860+0000 7fbbbcff9700 1 --2- 172.21.15.39:0/2480741153 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fbba80a12c0 0x7fbba80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.860+0000 7fbbbf7fe700 1 --2- 172.21.15.39:0/2480741153 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fbba80a12c0 0x7fbba80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.860+0000 7fbbbcff9700 1 -- 172.21.15.39:0/2480741153 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fbbb00ccbd0 con 0x7fbbc0100500 2023-07-21T18:54:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.860+0000 7fbbbf7fe700 1 --2- 172.21.15.39:0/2480741153 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fbba80a12c0 0x7fbba80a3770 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbbb4006010 tx=0x7fbbb400b540 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:54:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.861+0000 7fbbd23e5b80 1 Processor -- start 2023-07-21T18:54:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.861+0000 7fbbd23e5b80 1 -- start start 2023-07-21T18:54:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.861+0000 7fbbd23e5b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561af3c716b0 0x561af3e1ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.861+0000 7fbbd23e5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af3e1b2d0 0x561af3e15040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.861+0000 7fbbd23e5b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561af3e15580 0x561af3e159d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:09.877 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.861+0000 7fbbd23e5b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x561af3d1ac50 con 0x561af3c716b0 2023-07-21T18:54:09.877 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.861+0000 7fbbd23e5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561af3d6d9e0 con 0x561af3e1b2d0 2023-07-21T18:54:09.877 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.861+0000 7fbbd23e5b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x561af3d66e70 con 0x561af3e15580 2023-07-21T18:54:09.877 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.861+0000 7fbbbf7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561af3c716b0 0x561af3e1ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:09.877 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.861+0000 7fbbbf7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561af3c716b0 0x561af3e1ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:32892/0 (socket says 172.21.15.39:32892) 2023-07-21T18:54:09.877 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.861+0000 7fbbbf7fe700 1 -- 172.21.15.39:0/4244789503 learned_addr learned my addr 172.21.15.39:0/4244789503 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:54:09.877 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.861+0000 7fbbbffff700 1 --2- 172.21.15.39:0/4244789503 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561af3e15580 0x561af3e159d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:09.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.861+0000 7fbbbeffd700 1 --2- 172.21.15.39:0/4244789503 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af3e1b2d0 0x561af3e15040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:09.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.861+0000 7fbbbf7fe700 1 -- 172.21.15.39:0/4244789503 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561af3e15580 msgr2=0x561af3e159d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:54:09.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.861+0000 7fbbbf7fe700 1 --2- 172.21.15.39:0/4244789503 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561af3e15580 0x561af3e159d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:09.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.861+0000 7fbbbf7fe700 1 -- 172.21.15.39:0/4244789503 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af3e1b2d0 msgr2=0x561af3e15040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:54:09.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.861+0000 7fbbbf7fe700 1 --2- 172.21.15.39:0/4244789503 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af3e1b2d0 0x561af3e15040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:09.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.861+0000 7fbbbf7fe700 1 -- 172.21.15.39:0/4244789503 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561af3b94d80 con 0x561af3c716b0 2023-07-21T18:54:09.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.861+0000 7fbbbffff700 1 --2- 172.21.15.39:0/4244789503 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561af3e15580 0x561af3e159d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:54:09.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.862+0000 7fbbbf7fe700 1 --2- 172.21.15.39:0/4244789503 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561af3c716b0 0x561af3e1ad90 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fbbb400b900 tx=0x7fbbb401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c323dd0ec27faa58 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:54:09.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.862+0000 7fbba5ffb700 1 -- 172.21.15.39:0/4244789503 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb402b070 con 0x561af3c716b0 2023-07-21T18:54:09.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.862+0000 7fbba5ffb700 1 -- 172.21.15.39:0/4244789503 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbbb401e040 con 0x561af3c716b0 2023-07-21T18:54:09.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.862+0000 7fbba5ffb700 1 -- 172.21.15.39:0/4244789503 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb400fcf0 con 0x561af3c716b0 2023-07-21T18:54:09.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.862+0000 7fbbd23e5b80 1 -- 172.21.15.39:0/4244789503 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561af3c716b0 msgr2=0x561af3e1ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:54:09.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.862+0000 7fbbd23e5b80 1 --2- 172.21.15.39:0/4244789503 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561af3c716b0 0x561af3e1ad90 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fbbb400b900 tx=0x7fbbb401f000 comp rx=0 tx=0).stop 2023-07-21T18:54:09.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.862+0000 7fbbd23e5b80 1 -- 172.21.15.39:0/4244789503 shutdown_connections 2023-07-21T18:54:09.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.862+0000 7fbbd23e5b80 1 --2- 172.21.15.39:0/4244789503 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561af3e15580 0x561af3e159d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:09.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.862+0000 7fbbd23e5b80 1 --2- 172.21.15.39:0/4244789503 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561af3c716b0 0x561af3e1ad90 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:09.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.862+0000 7fbbd23e5b80 1 --2- 172.21.15.39:0/4244789503 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af3e1b2d0 0x561af3e15040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:09.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.862+0000 7fbbd23e5b80 1 -- 172.21.15.39:0/4244789503 >> 172.21.15.39:0/4244789503 conn(0x561af3e0abe0 msgr2=0x561af3e0ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:54:09.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.863+0000 7fbbd23e5b80 1 -- 172.21.15.39:0/4244789503 shutdown_connections 2023-07-21T18:54:09.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.863+0000 7fbbd23e5b80 1 -- 172.21.15.39:0/4244789503 wait complete. 2023-07-21T18:54:09.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.863+0000 7fbbd23e5b80 1 -- start start 2023-07-21T18:54:09.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.863+0000 7fbbd23e5b80 1 Processor -- start 2023-07-21T18:54:09.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.863+0000 7fbbd23e5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af3c716b0 0x561af3e21530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:09.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.863+0000 7fbbd23e5b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561af3e15580 0x561af3e1e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:09.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.863+0000 7fbbd23e5b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561af3e1b2d0 0x561af3e1ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:09.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.864+0000 7fbbd23e5b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x561af3d06a60 con 0x561af3e1b2d0 2023-07-21T18:54:09.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.864+0000 7fbbd23e5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561af3d06760 con 0x561af3c716b0 2023-07-21T18:54:09.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.864+0000 7fbbd23e5b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x561af3bea8c0 con 0x561af3e15580 2023-07-21T18:54:09.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.864+0000 7fbbd23e5b80 10 client.0 _ll_register_callbacks cb 0x561af3d04700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:54:09.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.864+0000 7fbbbffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561af3e1b2d0 0x561af3e1ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:09.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.864+0000 7fbbbf7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af3c716b0 0x561af3e21530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:09.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.864+0000 7fbbbf7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af3c716b0 0x561af3e21530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:45494/0 (socket says 172.21.15.39:45494) 2023-07-21T18:54:09.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.864+0000 7fbbbffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561af3e1b2d0 0x561af3e1ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:32902/0 (socket says 172.21.15.39:32902) 2023-07-21T18:54:09.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.864+0000 7fbbbf7fe700 1 -- 172.21.15.39:0/2141354935 learned_addr learned my addr 172.21.15.39:0/2141354935 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:54:09.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.864+0000 7fbbbeffd700 1 --2- 172.21.15.39:0/2141354935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561af3e15580 0x561af3e1e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:09.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.864+0000 7fbbbffff700 1 -- 172.21.15.39:0/2141354935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561af3e15580 msgr2=0x561af3e1e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:54:09.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.864+0000 7fbbbffff700 1 --2- 172.21.15.39:0/2141354935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561af3e15580 0x561af3e1e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:09.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.864+0000 7fbbbffff700 1 -- 172.21.15.39:0/2141354935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af3c716b0 msgr2=0x561af3e21530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:54:09.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.864+0000 7fbbbffff700 1 --2- 172.21.15.39:0/2141354935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af3c716b0 0x561af3e21530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:09.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.864+0000 7fbbbffff700 1 -- 172.21.15.39:0/2141354935 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbc010d240 con 0x561af3e1b2d0 2023-07-21T18:54:09.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.864+0000 7fbbbf7fe700 1 --2- 172.21.15.39:0/2141354935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af3c716b0 0x561af3e21530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:54:09.886 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.864+0000 7fbbbeffd700 1 --2- 172.21.15.39:0/2141354935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561af3e15580 0x561af3e1e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:54:09.886 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.864+0000 7fbbbffff700 1 --2- 172.21.15.39:0/2141354935 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561af3e1b2d0 0x561af3e1ee80 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fbbb8009fd0 tx=0x7fbbb8004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12b47105bdff1293 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:54:09.886 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.864+0000 7fbb9effd700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:54:09.886 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.864+0000 7fbb9effd700 1 -- 172.21.15.39:0/2141354935 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb80079d0 con 0x561af3e1b2d0 2023-07-21T18:54:09.886 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.864+0000 7fbb9effd700 1 -- 172.21.15.39:0/2141354935 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fbbb8007b30 con 0x561af3e1b2d0 2023-07-21T18:54:09.886 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.864+0000 7fbbd23e5b80 1 -- 172.21.15.39:0/2141354935 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561af3cc0490 con 0x561af3e1b2d0 2023-07-21T18:54:09.886 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.865+0000 7fbb9effd700 1 -- 172.21.15.39:0/2141354935 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbbb8007cf0 con 0x561af3e1b2d0 2023-07-21T18:54:09.886 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.865+0000 7fbbd23e5b80 10 client.17100 fetch_fsmap learned FSMap version 71 2023-07-21T18:54:09.887 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.865+0000 7fbbd23e5b80 1 -- 172.21.15.39:0/2141354935 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) v3 -- 0x561af3b7dc00 con 0x561af3e1b2d0 2023-07-21T18:54:09.887 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.865+0000 7fbb9effd700 1 -- 172.21.15.39:0/2141354935 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561af3cc0490 con 0x561af3e1b2d0 2023-07-21T18:54:09.887 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.865+0000 7fbb9effd700 1 -- 172.21.15.39:0/2141354935 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 71) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbbb8014040 con 0x561af3e1b2d0 2023-07-21T18:54:09.887 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.865+0000 7fbbd23e5b80 10 client.17100 fetch_fsmap finished waiting for FSMap version 71 2023-07-21T18:54:09.887 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.865+0000 7fbbd23e5b80 10 client.17100 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:54:09.887 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.865+0000 7fbbd23e5b80 20 client.17100 populate_metadata read hostname 'smithi039' 2023-07-21T18:54:09.887 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.865+0000 7fbbd23e5b80 1 --2- 172.21.15.39:0/2141354935 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x561af3e2eb90 0x561af3e30fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:09.888 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.865+0000 7fbbd23e5b80 4 client.17100 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:54:09.888 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.865+0000 7fbbd23e5b80 1 -- 172.21.15.39:0/2141354935 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561af3d5fe50 con 0x561af3e2eb90 2023-07-21T18:54:09.888 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.865+0000 7fbbbf7fe700 1 --2- 172.21.15.39:0/2141354935 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x561af3e2eb90 0x561af3e30fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:09.888 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.866+0000 7fbb9effd700 10 client.17100 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:54:09.888 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.866+0000 7fbbbf7fe700 1 --2- 172.21.15.39:0/2141354935 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x561af3e2eb90 0x561af3e30fa0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b445cec2d4fe5379 server_cookie=bda66b87fb247d09 in_seq=0 out_seq=0 2023-07-21T18:54:09.888 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.866+0000 7fbb9effd700 1 -- 172.21.15.39:0/2141354935 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561af3d5fe50 con 0x561af3e2eb90 2023-07-21T18:54:09.888 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.866+0000 7fbb9effd700 10 client.17100 handle_command_reply: tid=0 2023-07-21T18:54:09.888 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.866+0000 7fbbd23e5b80 1 client.17100 shutdown 2023-07-21T18:54:09.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.867+0000 7fbbd23e5b80 1 -- 172.21.15.39:0/2141354935 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561af3e1b2d0 msgr2=0x561af3e1ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:54:09.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.867+0000 7fbbd23e5b80 1 --2- 172.21.15.39:0/2141354935 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561af3e1b2d0 0x561af3e1ee80 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fbbb8009fd0 tx=0x7fbbb8004320 comp rx=0 tx=0).stop 2023-07-21T18:54:09.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.867+0000 7fbbd23e5b80 1 -- 172.21.15.39:0/2141354935 shutdown_connections 2023-07-21T18:54:09.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.867+0000 7fbbd23e5b80 1 --2- 172.21.15.39:0/2141354935 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x561af3e2eb90 0x561af3e30fa0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:09.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.867+0000 7fbbd23e5b80 1 --2- 172.21.15.39:0/2141354935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af3c716b0 0x561af3e21530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:09.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.867+0000 7fbb9effd700 0 client.17100 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:54:09.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.867+0000 7fbbd23e5b80 1 --2- 172.21.15.39:0/2141354935 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561af3e1b2d0 0x561af3e1ee80 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:09.890 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.867+0000 7fbbd23e5b80 1 --2- 172.21.15.39:0/2141354935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561af3e15580 0x561af3e1e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:09.890 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.867+0000 7fbbd23e5b80 1 -- 172.21.15.39:0/2141354935 >> 172.21.15.39:0/2141354935 conn(0x561af3e0abe0 msgr2=0x561af3e0f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:54:09.890 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.867+0000 7fbbd23e5b80 1 -- 172.21.15.39:0/2141354935 shutdown_connections 2023-07-21T18:54:09.890 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.867+0000 7fbbd23e5b80 1 -- 172.21.15.39:0/2141354935 wait complete. 2023-07-21T18:54:09.890 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.867+0000 7fbbd23e5b80 20 client.17100 trim_cache size 0 max 16384 2023-07-21T18:54:09.896 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.883+0000 7fbbd23e5b80 1 Processor -- start 2023-07-21T18:54:09.896 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.883+0000 7fbbd23e5b80 1 -- start start 2023-07-21T18:54:09.896 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.883+0000 7fbbd23e5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af3c716b0 0x561af3e1a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:09.896 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.883+0000 7fbbd23e5b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561af3e15580 0x561af3e0fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:09.896 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.883+0000 7fbbd23e5b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561af3e1b2d0 0x561af3e21060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:09.896 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.883+0000 7fbbd23e5b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x561af3e0cd00 con 0x561af3e1b2d0 2023-07-21T18:54:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.884+0000 7fbbd23e5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561af3e07bb0 con 0x561af3c716b0 2023-07-21T18:54:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.884+0000 7fbbd23e5b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x561af3e1a9c0 con 0x561af3e15580 2023-07-21T18:54:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.884+0000 7fbbbf7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af3c716b0 0x561af3e1a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.884+0000 7fbbbf7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af3c716b0 0x561af3e1a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:45502/0 (socket says 172.21.15.39:45502) 2023-07-21T18:54:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.884+0000 7fbbbf7fe700 1 -- 172.21.15.39:0/179076066 learned_addr learned my addr 172.21.15.39:0/179076066 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:54:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.884+0000 7fbbbffff700 1 --2- 172.21.15.39:0/179076066 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561af3e1b2d0 0x561af3e21060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.884+0000 7fbbbffff700 1 -- 172.21.15.39:0/179076066 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561af3e15580 msgr2=0x561af3e0fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:54:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.884+0000 7fbbbeffd700 1 --2- 172.21.15.39:0/179076066 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561af3e15580 0x561af3e0fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.884+0000 7fbbbffff700 1 --2- 172.21.15.39:0/179076066 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561af3e15580 0x561af3e0fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.885+0000 7fbbbffff700 1 -- 172.21.15.39:0/179076066 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af3c716b0 msgr2=0x561af3e1a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:54:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.885+0000 7fbbbffff700 1 --2- 172.21.15.39:0/179076066 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af3c716b0 0x561af3e1a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.885+0000 7fbbbffff700 1 -- 172.21.15.39:0/179076066 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561af3bbb1f0 con 0x561af3e1b2d0 2023-07-21T18:54:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.885+0000 7fbbbeffd700 1 --2- 172.21.15.39:0/179076066 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561af3e15580 0x561af3e0fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:54:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.885+0000 7fbbbffff700 1 --2- 172.21.15.39:0/179076066 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561af3e1b2d0 0x561af3e21060 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fbbb8016cb0 tx=0x7fbbb8004800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5774f3662083e86 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:54:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.885+0000 7fbba67fc700 1 -- 172.21.15.39:0/179076066 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb8004ab0 con 0x561af3e1b2d0 2023-07-21T18:54:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.885+0000 7fbba67fc700 1 -- 172.21.15.39:0/179076066 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbbb800f040 con 0x561af3e1b2d0 2023-07-21T18:54:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.885+0000 7fbba67fc700 1 -- 172.21.15.39:0/179076066 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb8007d70 con 0x561af3e1b2d0 2023-07-21T18:54:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.886+0000 7fbbd23e5b80 1 -- 172.21.15.39:0/179076066 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561af3e1b2d0 msgr2=0x561af3e21060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:54:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.886+0000 7fbbd23e5b80 1 --2- 172.21.15.39:0/179076066 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561af3e1b2d0 0x561af3e21060 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fbbb8016cb0 tx=0x7fbbb8004800 comp rx=0 tx=0).stop 2023-07-21T18:54:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.886+0000 7fbbd23e5b80 1 -- 172.21.15.39:0/179076066 shutdown_connections 2023-07-21T18:54:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.886+0000 7fbbd23e5b80 1 --2- 172.21.15.39:0/179076066 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af3c716b0 0x561af3e1a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.886+0000 7fbbd23e5b80 1 --2- 172.21.15.39:0/179076066 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561af3e1b2d0 0x561af3e21060 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:09.901 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.886+0000 7fbbd23e5b80 1 --2- 172.21.15.39:0/179076066 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561af3e15580 0x561af3e0fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:09.901 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.886+0000 7fbbd23e5b80 1 -- 172.21.15.39:0/179076066 >> 172.21.15.39:0/179076066 conn(0x561af3e0abe0 msgr2=0x561af3e19a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:54:09.902 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.887+0000 7fbbd23e5b80 1 -- 172.21.15.39:0/179076066 shutdown_connections 2023-07-21T18:54:09.902 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.887+0000 7fbbd23e5b80 1 -- 172.21.15.39:0/179076066 wait complete. 2023-07-21T18:54:09.902 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.887+0000 7fbbd23e5b80 1 -- start start 2023-07-21T18:54:09.903 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.887+0000 7fbbd23e5b80 1 Processor -- start 2023-07-21T18:54:09.903 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.888+0000 7fbbd23e5b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561af3c716b0 0x561af3e23e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:09.903 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.888+0000 7fbbd23e5b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561af3e15580 0x561af3e24380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:09.903 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.888+0000 7fbbd23e5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af3e1b2d0 0x561af3e8a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:09.903 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.888+0000 7fbbd23e5b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x561af3e24ac0 con 0x561af3e15580 2023-07-21T18:54:09.903 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.888+0000 7fbbd23e5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561af3e8a760 con 0x561af3e1b2d0 2023-07-21T18:54:09.903 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.888+0000 7fbbd23e5b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x561af3e8a8a0 con 0x561af3c716b0 2023-07-21T18:54:09.903 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.888+0000 7fbbd23e5b80 10 client.0 _ll_register_callbacks cb 0x7fbbb8003ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:54:09.904 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.888+0000 7fbbbf7fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561af3c716b0 0x561af3e23e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:09.904 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.888+0000 7fbbbf7fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561af3c716b0 0x561af3e23e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:51464/0 (socket says 172.21.15.39:51464) 2023-07-21T18:54:09.904 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.888+0000 7fbbbf7fe700 1 -- 172.21.15.39:0/3799590987 learned_addr learned my addr 172.21.15.39:0/3799590987 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:54:09.904 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.889+0000 7fbbbf7fe700 1 -- 172.21.15.39:0/3799590987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af3e1b2d0 msgr2=0x561af3e8a220 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T18:54:09.904 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.889+0000 7fbbbf7fe700 1 --2- 172.21.15.39:0/3799590987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af3e1b2d0 0x561af3e8a220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:09.904 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.889+0000 7fbbbf7fe700 1 -- 172.21.15.39:0/3799590987 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561af3e15580 msgr2=0x561af3e24380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:54:09.904 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.889+0000 7fbbbf7fe700 1 --2- 172.21.15.39:0/3799590987 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561af3e15580 0x561af3e24380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:09.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.889+0000 7fbbbf7fe700 1 -- 172.21.15.39:0/3799590987 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561af3b7dc00 con 0x561af3c716b0 2023-07-21T18:54:09.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.889+0000 7fbbbf7fe700 1 --2- 172.21.15.39:0/3799590987 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561af3c716b0 0x561af3e23e40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbbb4009350 tx=0x7fbbb400f890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a960d15781432d3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:54:09.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.889+0000 7fbb9effd700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2023-07-21T18:54:09.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.890+0000 7fbb9effd700 1 -- 172.21.15.39:0/3799590987 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb402b040 con 0x561af3c716b0 2023-07-21T18:54:09.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.890+0000 7fbb9effd700 1 -- 172.21.15.39:0/3799590987 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fbbb40373e0 con 0x561af3c716b0 2023-07-21T18:54:09.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.890+0000 7fbbd23e5b80 1 -- 172.21.15.39:0/3799590987 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbbb401e040 con 0x561af3c716b0 2023-07-21T18:54:09.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.890+0000 7fbb9effd700 1 -- 172.21.15.39:0/3799590987 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbbb40375a0 con 0x561af3c716b0 2023-07-21T18:54:09.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.890+0000 7fbbd23e5b80 10 client.26569 fetch_fsmap learned FSMap version 71 2023-07-21T18:54:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.890+0000 7fbbd23e5b80 1 -- 172.21.15.39:0/3799590987 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) v3 -- 0x561af3b93250 con 0x561af3c716b0 2023-07-21T18:54:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.891+0000 7fbb9effd700 1 -- 172.21.15.39:0/3799590987 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbbb401e040 con 0x561af3c716b0 2023-07-21T18:54:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.891+0000 7fbb9effd700 1 -- 172.21.15.39:0/3799590987 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 71) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbbb403f070 con 0x561af3c716b0 2023-07-21T18:54:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.891+0000 7fbbd23e5b80 10 client.26569 fetch_fsmap finished waiting for FSMap version 71 2023-07-21T18:54:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.891+0000 7fbbd23e5b80 10 client.26569 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:54:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.891+0000 7fbbd23e5b80 20 client.26569 populate_metadata read hostname 'smithi039' 2023-07-21T18:54:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.891+0000 7fbbd23e5b80 1 --2- 172.21.15.39:0/3799590987 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x561af3e8ba30 0x561af3e8de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.891+0000 7fbbd23e5b80 4 client.26569 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:54:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.891+0000 7fbbd23e5b80 1 -- 172.21.15.39:0/3799590987 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561af3d21ff0 con 0x561af3e8ba30 2023-07-21T18:54:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.891+0000 7fbbbeffd700 1 --2- 172.21.15.39:0/3799590987 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x561af3e8ba30 0x561af3e8de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.892+0000 7fbbbeffd700 1 --2- 172.21.15.39:0/3799590987 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x561af3e8ba30 0x561af3e8de10 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e59cfb19b3f34c6 server_cookie=3039c3d37fe88a67 in_seq=0 out_seq=0 2023-07-21T18:54:09.908 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.892+0000 7fbb9effd700 10 client.26569 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:54:09.954 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.940+0000 7fbb9effd700 1 -- 172.21.15.39:0/3799590987 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561af3d21ff0 con 0x561af3e8ba30 2023-07-21T18:54:09.954 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.940+0000 7fbb9effd700 10 client.26569 handle_command_reply: tid=0 2023-07-21T18:54:09.954 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.940+0000 7fbbd23e5b80 1 client.26569 shutdown 2023-07-21T18:54:09.954 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.941+0000 7fbbd23e5b80 1 -- 172.21.15.39:0/3799590987 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561af3c716b0 msgr2=0x561af3e23e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:54:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.941+0000 7fbbd23e5b80 1 --2- 172.21.15.39:0/3799590987 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561af3c716b0 0x561af3e23e40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbbb4009350 tx=0x7fbbb400f890 comp rx=0 tx=0).stop 2023-07-21T18:54:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.941+0000 7fbbd23e5b80 1 -- 172.21.15.39:0/3799590987 shutdown_connections 2023-07-21T18:54:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.941+0000 7fbbd23e5b80 1 --2- 172.21.15.39:0/3799590987 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x561af3e8ba30 0x561af3e8de10 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:09.956 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.941+0000 7fbbd23e5b80 1 --2- 172.21.15.39:0/3799590987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af3e1b2d0 0x561af3e8a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:09.956 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.941+0000 7fbbd23e5b80 1 --2- 172.21.15.39:0/3799590987 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561af3c716b0 0x561af3e23e40 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:09.956 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.941+0000 7fbb9effd700 0 client.26569 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:54:09.956 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.941+0000 7fbbd23e5b80 1 --2- 172.21.15.39:0/3799590987 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561af3e15580 0x561af3e24380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:09.957 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.941+0000 7fbbd23e5b80 1 -- 172.21.15.39:0/3799590987 >> 172.21.15.39:0/3799590987 conn(0x561af3e0abe0 msgr2=0x561af3e1a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:54:09.957 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.941+0000 7fbbd23e5b80 1 -- 172.21.15.39:0/3799590987 shutdown_connections 2023-07-21T18:54:09.957 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.941+0000 7fbbd23e5b80 1 -- 172.21.15.39:0/3799590987 wait complete. 2023-07-21T18:54:09.957 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.941+0000 7fbbd23e5b80 20 client.26569 trim_cache size 0 max 16384 2023-07-21T18:54:09.957 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:54:09.958 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:54:09.958 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:54:09.958 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:54:09.958 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.941+0000 7fbbc625f700 1 -- 172.21.15.39:0/2480741153 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fbba80a12c0 msgr2=0x7fbba80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:54:09.959 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.941+0000 7fbbc625f700 1 --2- 172.21.15.39:0/2480741153 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fbba80a12c0 0x7fbba80a3770 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbbb4006010 tx=0x7fbbb400b540 comp rx=0 tx=0).stop 2023-07-21T18:54:09.959 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.941+0000 7fbbc625f700 1 -- 172.21.15.39:0/2480741153 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbc0100500 msgr2=0x7fbbc019d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:54:09.959 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.941+0000 7fbbc625f700 1 --2- 172.21.15.39:0/2480741153 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbc0100500 0x7fbbc019d190 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fbbb000d950 tx=0x7fbbb000dc60 comp rx=0 tx=0).stop 2023-07-21T18:54:09.959 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.942+0000 7fbbc625f700 1 -- 172.21.15.39:0/2480741153 shutdown_connections 2023-07-21T18:54:09.959 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.942+0000 7fbbc625f700 1 --2- 172.21.15.39:0/2480741153 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fbba80a12c0 0x7fbba80a3770 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:09.959 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.942+0000 7fbbc625f700 1 --2- 172.21.15.39:0/2480741153 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbbc00ffb50 0x7fbbc019cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:09.960 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.942+0000 7fbbc625f700 1 --2- 172.21.15.39:0/2480741153 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbbc0105b80 0x7fbbc0197c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:09.960 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.942+0000 7fbbc625f700 1 --2- 172.21.15.39:0/2480741153 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbc0100500 0x7fbbc019d190 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:09.960 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.942+0000 7fbbc625f700 1 -- 172.21.15.39:0/2480741153 >> 172.21.15.39:0/2480741153 conn(0x7fbbc00fb5f0 msgr2=0x7fbbc00fd900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:54:09.960 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.942+0000 7fbbc625f700 1 -- 172.21.15.39:0/2480741153 shutdown_connections 2023-07-21T18:54:09.960 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:09.942+0000 7fbbc625f700 1 -- 172.21.15.39:0/2480741153 wait complete. 2023-07-21T18:54:10.294 INFO:tasks.workunit.client.0.smithi039.stdout: 1 697732 5.24 MB/sec execute 563 sec latency 1743.505 ms 2023-07-21T18:54:10.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:54:10 smithi193 ceph-mon[98204]: pgmap v389: 97 pgs: 97 active+clean; 263 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 1.6 MiB/s wr, 184 op/s 2023-07-21T18:54:10.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:54:10 smithi155 ceph-mon[97934]: pgmap v389: 97 pgs: 97 active+clean; 263 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 1.6 MiB/s wr, 184 op/s 2023-07-21T18:54:10.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:54:10 smithi039 ceph-mon[96489]: pgmap v389: 97 pgs: 97 active+clean; 263 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 1.6 MiB/s wr, 184 op/s 2023-07-21T18:54:11.294 INFO:tasks.workunit.client.0.smithi039.stdout: 1 697732 5.23 MB/sec execute 564 sec latency 2743.605 ms 2023-07-21T18:54:12.294 INFO:tasks.workunit.client.0.smithi039.stdout: 1 698939 5.24 MB/sec execute 565 sec latency 3172.814 ms 2023-07-21T18:54:12.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:54:12 smithi193 ceph-mon[98204]: pgmap v390: 97 pgs: 97 active+clean; 263 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 1.6 MiB/s wr, 186 op/s 2023-07-21T18:54:12.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:54:12 smithi193 ceph-mon[98204]: osdmap e97: 12 total, 12 up, 12 in 2023-07-21T18:54:12.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:54:12 smithi155 ceph-mon[97934]: pgmap v390: 97 pgs: 97 active+clean; 263 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 1.6 MiB/s wr, 186 op/s 2023-07-21T18:54:12.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:54:12 smithi155 ceph-mon[97934]: osdmap e97: 12 total, 12 up, 12 in 2023-07-21T18:54:12.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:54:12 smithi039 ceph-mon[96489]: pgmap v390: 97 pgs: 97 active+clean; 263 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 1.6 MiB/s wr, 186 op/s 2023-07-21T18:54:12.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:54:12 smithi039 ceph-mon[96489]: osdmap e97: 12 total, 12 up, 12 in 2023-07-21T18:54:13.294 INFO:tasks.workunit.client.0.smithi039.stdout: 1 700191 5.24 MB/sec execute 566 sec latency 16.224 ms 2023-07-21T18:54:14.294 INFO:tasks.workunit.client.0.smithi039.stdout: 1 701275 5.24 MB/sec execute 567 sec latency 288.390 ms 2023-07-21T18:54:14.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:54:14 smithi193 ceph-mon[98204]: pgmap v392: 97 pgs: 97 active+clean; 263 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 959 KiB/s rd, 2.0 MiB/s wr, 227 op/s 2023-07-21T18:54:14.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:54:14 smithi155 ceph-mon[97934]: pgmap v392: 97 pgs: 97 active+clean; 263 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 959 KiB/s rd, 2.0 MiB/s wr, 227 op/s 2023-07-21T18:54:14.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:54:14 smithi039 ceph-mon[96489]: pgmap v392: 97 pgs: 97 active+clean; 263 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 959 KiB/s rd, 2.0 MiB/s wr, 227 op/s 2023-07-21T18:54:15.294 INFO:tasks.workunit.client.0.smithi039.stdout: 1 701275 5.23 MB/sec execute 568 sec latency 1288.424 ms 2023-07-21T18:54:16.294 INFO:tasks.workunit.client.0.smithi039.stdout: 1 701275 5.22 MB/sec execute 569 sec latency 2288.511 ms 2023-07-21T18:54:16.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:54:16 smithi193 ceph-mon[98204]: pgmap v393: 97 pgs: 97 active+clean; 263 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 1.6 MiB/s wr, 186 op/s 2023-07-21T18:54:16.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:54:16 smithi155 ceph-mon[97934]: pgmap v393: 97 pgs: 97 active+clean; 263 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 1.6 MiB/s wr, 186 op/s 2023-07-21T18:54:16.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:54:16 smithi039 ceph-mon[96489]: pgmap v393: 97 pgs: 97 active+clean; 263 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 1.6 MiB/s wr, 186 op/s 2023-07-21T18:54:17.294 INFO:tasks.workunit.client.0.smithi039.stdout: 1 702481 5.22 MB/sec execute 570 sec latency 2715.726 ms 2023-07-21T18:54:18.294 INFO:tasks.workunit.client.0.smithi039.stdout: 1 703698 5.23 MB/sec execute 571 sec latency 13.615 ms 2023-07-21T18:54:18.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:54:18 smithi193 ceph-mon[98204]: pgmap v394: 97 pgs: 97 active+clean; 266 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 1.5 MiB/s wr, 148 op/s 2023-07-21T18:54:18.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:54:18 smithi155 ceph-mon[97934]: pgmap v394: 97 pgs: 97 active+clean; 266 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 1.5 MiB/s wr, 148 op/s 2023-07-21T18:54:18.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:54:18 smithi039 ceph-mon[96489]: pgmap v394: 97 pgs: 97 active+clean; 266 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 1.5 MiB/s wr, 148 op/s 2023-07-21T18:54:19.295 INFO:tasks.workunit.client.0.smithi039.stdout: 1 704822 5.23 MB/sec execute 572 sec latency 213.625 ms 2023-07-21T18:54:20.295 INFO:tasks.workunit.client.0.smithi039.stdout: 1 704822 5.22 MB/sec execute 573 sec latency 1213.675 ms 2023-07-21T18:54:20.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:54:20 smithi193 ceph-mon[98204]: pgmap v395: 97 pgs: 97 active+clean; 266 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 1.5 MiB/s wr, 148 op/s 2023-07-21T18:54:20.729 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:54:20 smithi039 ceph-mon[96489]: pgmap v395: 97 pgs: 97 active+clean; 266 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 1.5 MiB/s wr, 148 op/s 2023-07-21T18:54:20.730 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:54:20 smithi155 ceph-mon[97934]: pgmap v395: 97 pgs: 97 active+clean; 266 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 1.5 MiB/s wr, 148 op/s 2023-07-21T18:54:21.295 INFO:tasks.workunit.client.0.smithi039.stdout: 1 704822 5.21 MB/sec execute 574 sec latency 2213.784 ms 2023-07-21T18:54:22.295 INFO:tasks.workunit.client.0.smithi039.stdout: 1 705581 5.20 MB/sec execute 575 sec latency 2636.960 ms 2023-07-21T18:54:22.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:54:22 smithi193 ceph-mon[98204]: pgmap v396: 97 pgs: 97 active+clean; 271 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 857 KiB/s rd, 1.6 MiB/s wr, 145 op/s 2023-07-21T18:54:22.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:54:22 smithi155 ceph-mon[97934]: pgmap v396: 97 pgs: 97 active+clean; 271 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 857 KiB/s rd, 1.6 MiB/s wr, 145 op/s 2023-07-21T18:54:22.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:54:22 smithi039 ceph-mon[96489]: pgmap v396: 97 pgs: 97 active+clean; 271 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 857 KiB/s rd, 1.6 MiB/s wr, 145 op/s 2023-07-21T18:54:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.060+0000 7f86bd6e8700 1 -- 172.21.15.39:0/197472822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86b80fe9a0 msgr2=0x7f86b81036b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:54:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.060+0000 7f86bd6e8700 1 --2- 172.21.15.39:0/197472822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86b80fe9a0 0x7f86b81036b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f86b0009f90 tx=0x7f86b000b330 comp rx=0 tx=0).stop 2023-07-21T18:54:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.060+0000 7f86bd6e8700 1 -- 172.21.15.39:0/197472822 shutdown_connections 2023-07-21T18:54:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.060+0000 7f86bd6e8700 1 --2- 172.21.15.39:0/197472822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86b80fe9a0 0x7f86b81036b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.060+0000 7f86bd6e8700 1 --2- 172.21.15.39:0/197472822 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f86b8104a50 0x7f86b8104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.060+0000 7f86bd6e8700 1 --2- 172.21.15.39:0/197472822 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f86b80fdff0 0x7f86b80fe460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.060+0000 7f86bd6e8700 1 -- 172.21.15.39:0/197472822 >> 172.21.15.39:0/197472822 conn(0x7f86b8075f90 msgr2=0x7f86b8076390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:54:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.061+0000 7f86bd6e8700 1 -- 172.21.15.39:0/197472822 shutdown_connections 2023-07-21T18:54:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.061+0000 7f86bd6e8700 1 -- 172.21.15.39:0/197472822 wait complete. 2023-07-21T18:54:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.061+0000 7f86bd6e8700 1 Processor -- start 2023-07-21T18:54:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.061+0000 7f86bd6e8700 1 -- start start 2023-07-21T18:54:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.061+0000 7f86bd6e8700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f86b80fdff0 0x7f86b8199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.062+0000 7f86bd6e8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f86b80fe9a0 0x7f86b8199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.062+0000 7f86bd6e8700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86b8104a50 0x7f86b81938b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.062+0000 7f86bd6e8700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b8193f20 con 0x7f86b80fe9a0 2023-07-21T18:54:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.062+0000 7f86bd6e8700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b8194090 con 0x7f86b8104a50 2023-07-21T18:54:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.062+0000 7f86bd6e8700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b8194250 con 0x7f86b80fdff0 2023-07-21T18:54:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.062+0000 7f86b6ffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f86b80fdff0 0x7f86b8199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.062+0000 7f86b67fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f86b80fe9a0 0x7f86b8199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.062+0000 7f86b6ffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f86b80fdff0 0x7f86b8199a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:51478/0 (socket says 172.21.15.39:51478) 2023-07-21T18:54:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.062+0000 7f86b6ffd700 1 -- 172.21.15.39:0/3776222261 learned_addr learned my addr 172.21.15.39:0/3776222261 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:54:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.062+0000 7f86b77fe700 1 --2- 172.21.15.39:0/3776222261 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86b8104a50 0x7f86b81938b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.062+0000 7f86b6ffd700 1 -- 172.21.15.39:0/3776222261 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86b8104a50 msgr2=0x7f86b81938b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:54:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.062+0000 7f86b6ffd700 1 --2- 172.21.15.39:0/3776222261 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86b8104a50 0x7f86b81938b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.062+0000 7f86b6ffd700 1 -- 172.21.15.39:0/3776222261 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f86b80fe9a0 msgr2=0x7f86b8199f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:54:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.062+0000 7f86b6ffd700 1 --2- 172.21.15.39:0/3776222261 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f86b80fe9a0 0x7f86b8199f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.062+0000 7f86b6ffd700 1 -- 172.21.15.39:0/3776222261 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86ac009520 con 0x7f86b80fdff0 2023-07-21T18:54:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.062+0000 7f86b6ffd700 1 --2- 172.21.15.39:0/3776222261 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f86b80fdff0 0x7f86b8199a50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f86a801fac0 tx=0x7f86a801fdd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:54:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.062+0000 7f869ffff700 1 -- 172.21.15.39:0/3776222261 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a801d9e0 con 0x7f86b80fdff0 2023-07-21T18:54:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.062+0000 7f86bd6e8700 1 -- 172.21.15.39:0/3776222261 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86b0009c40 con 0x7f86b80fdff0 2023-07-21T18:54:23.078 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.062+0000 7f869ffff700 1 -- 172.21.15.39:0/3776222261 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86a8004020 con 0x7f86b80fdff0 2023-07-21T18:54:23.078 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.062+0000 7f869ffff700 1 -- 172.21.15.39:0/3776222261 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a8003c20 con 0x7f86b80fdff0 2023-07-21T18:54:23.078 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.063+0000 7f86bd6e8700 1 -- 172.21.15.39:0/3776222261 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86b8194720 con 0x7f86b80fdff0 2023-07-21T18:54:23.081 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.065+0000 7f869ffff700 1 -- 172.21.15.39:0/3776222261 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f86a8004190 con 0x7f86b80fdff0 2023-07-21T18:54:23.081 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.066+0000 7f86c986eb80 1 Processor -- start 2023-07-21T18:54:23.081 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.066+0000 7f86c986eb80 1 -- start start 2023-07-21T18:54:23.081 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.066+0000 7f869ffff700 1 --2- 172.21.15.39:0/3776222261 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f86a00a12c0 0x7f86a00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:23.081 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.066+0000 7f86c986eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecf85ec6b0 0x55ecf8795d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.066+0000 7f86c986eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecf87962d0 0x55ecf8790040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.066+0000 7f86c986eb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ecf8790580 0x55ecf87909d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.066+0000 7f86c986eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecf8695c50 con 0x55ecf87962d0 2023-07-21T18:54:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.066+0000 7f86c986eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecf86e89e0 con 0x55ecf85ec6b0 2023-07-21T18:54:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.066+0000 7f86c986eb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecf86e1e70 con 0x55ecf8790580 2023-07-21T18:54:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.066+0000 7f869ffff700 1 -- 172.21.15.39:0/3776222261 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f86a8025070 con 0x7f86b80fdff0 2023-07-21T18:54:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.066+0000 7f86b77fe700 1 --2- 172.21.15.39:0/3776222261 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f86a00a12c0 0x7f86a00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.067+0000 7f86b67fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecf87962d0 0x55ecf8790040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.067+0000 7f86b77fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ecf8790580 0x55ecf87909d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.067+0000 7f86b67fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecf87962d0 0x55ecf8790040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:32934/0 (socket says 172.21.15.39:32934) 2023-07-21T18:54:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.067+0000 7f86b67fc700 1 -- 172.21.15.39:0/2472373440 learned_addr learned my addr 172.21.15.39:0/2472373440 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:54:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.067+0000 7f86b77fe700 1 -- 172.21.15.39:0/2472373440 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecf85ec6b0 msgr2=0x55ecf8795d90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T18:54:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.067+0000 7f86b77fe700 1 --2- 172.21.15.39:0/2472373440 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecf85ec6b0 0x55ecf8795d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.067+0000 7f86b77fe700 1 -- 172.21.15.39:0/2472373440 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecf87962d0 msgr2=0x55ecf8790040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:54:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.067+0000 7f86b77fe700 1 --2- 172.21.15.39:0/2472373440 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecf87962d0 0x55ecf8790040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.067+0000 7f86b77fe700 1 -- 172.21.15.39:0/2472373440 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ecf850fd80 con 0x55ecf8790580 2023-07-21T18:54:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.067+0000 7f86b77fe700 1 --2- 172.21.15.39:0/3776222261 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f86a00a12c0 0x7f86a00a3770 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f86b00093d0 tx=0x7f86b0007420 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:54:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.067+0000 7f86b77fe700 1 --2- 172.21.15.39:0/2472373440 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ecf8790580 0x55ecf87909d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f86b0009790 tx=0x7f86b0026000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27bc818972c77a4e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:54:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.067+0000 7f869d7fa700 1 -- 172.21.15.39:0/2472373440 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b0026070 con 0x55ecf8790580 2023-07-21T18:54:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.067+0000 7f869d7fa700 1 -- 172.21.15.39:0/2472373440 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86b0008e70 con 0x55ecf8790580 2023-07-21T18:54:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.067+0000 7f869d7fa700 1 -- 172.21.15.39:0/2472373440 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b000e040 con 0x55ecf8790580 2023-07-21T18:54:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.068+0000 7f86c986eb80 1 -- 172.21.15.39:0/2472373440 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ecf8790580 msgr2=0x55ecf87909d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:54:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.068+0000 7f86c986eb80 1 --2- 172.21.15.39:0/2472373440 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ecf8790580 0x55ecf87909d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f86b0009790 tx=0x7f86b0026000 comp rx=0 tx=0).stop 2023-07-21T18:54:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.068+0000 7f86c986eb80 1 -- 172.21.15.39:0/2472373440 shutdown_connections 2023-07-21T18:54:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.068+0000 7f86c986eb80 1 --2- 172.21.15.39:0/2472373440 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecf85ec6b0 0x55ecf8795d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.068+0000 7f86c986eb80 1 --2- 172.21.15.39:0/2472373440 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ecf8790580 0x55ecf87909d0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.068+0000 7f86c986eb80 1 --2- 172.21.15.39:0/2472373440 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecf87962d0 0x55ecf8790040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.068+0000 7f86c986eb80 1 -- 172.21.15.39:0/2472373440 >> 172.21.15.39:0/2472373440 conn(0x55ecf8785be0 msgr2=0x55ecf878af10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:54:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.068+0000 7f86c986eb80 1 -- 172.21.15.39:0/2472373440 shutdown_connections 2023-07-21T18:54:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.068+0000 7f86c986eb80 1 -- 172.21.15.39:0/2472373440 wait complete. 2023-07-21T18:54:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.068+0000 7f86c986eb80 1 -- start start 2023-07-21T18:54:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.069+0000 7f86c986eb80 1 Processor -- start 2023-07-21T18:54:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.069+0000 7f86c986eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecf85ec6b0 0x55ecf87999e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.069+0000 7f86c986eb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ecf8790580 0x55ecf8799f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.069+0000 7f86c986eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecf87962d0 0x55ecf879a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.069+0000 7f86c986eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecf8681a60 con 0x55ecf87962d0 2023-07-21T18:54:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.069+0000 7f86c986eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecf8681760 con 0x55ecf85ec6b0 2023-07-21T18:54:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.069+0000 7f86c986eb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecf85658c0 con 0x55ecf8790580 2023-07-21T18:54:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.069+0000 7f86c986eb80 10 client.0 _ll_register_callbacks cb 0x55ecf867f700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:54:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.069+0000 7f86b77fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecf87962d0 0x55ecf879a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.069+0000 7f86b6ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecf85ec6b0 0x55ecf87999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.069+0000 7f86b77fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecf87962d0 0x55ecf879a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:32944/0 (socket says 172.21.15.39:32944) 2023-07-21T18:54:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.069+0000 7f86b77fe700 1 -- 172.21.15.39:0/3067477230 learned_addr learned my addr 172.21.15.39:0/3067477230 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:54:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.069+0000 7f86b67fc700 1 --2- 172.21.15.39:0/3067477230 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ecf8790580 0x55ecf8799f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.069+0000 7f86b77fe700 1 -- 172.21.15.39:0/3067477230 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ecf8790580 msgr2=0x55ecf8799f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:54:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.069+0000 7f86b77fe700 1 --2- 172.21.15.39:0/3067477230 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ecf8790580 0x55ecf8799f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:23.088 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.069+0000 7f86b77fe700 1 -- 172.21.15.39:0/3067477230 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecf85ec6b0 msgr2=0x55ecf87999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:54:23.088 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.069+0000 7f86b77fe700 1 --2- 172.21.15.39:0/3067477230 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecf85ec6b0 0x55ecf87999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:23.088 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.069+0000 7f86b77fe700 1 -- 172.21.15.39:0/3067477230 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86b8103f70 con 0x55ecf87962d0 2023-07-21T18:54:23.088 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.069+0000 7f86b6ffd700 1 --2- 172.21.15.39:0/3067477230 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecf85ec6b0 0x55ecf87999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:54:23.088 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.070+0000 7f86b77fe700 1 --2- 172.21.15.39:0/3067477230 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecf87962d0 0x55ecf879a460 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f86b0009040 tx=0x7f86b00216c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4987824918d8561f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:54:23.088 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.070+0000 7f869aff5700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:54:23.088 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.070+0000 7f869aff5700 1 -- 172.21.15.39:0/3067477230 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b000e040 con 0x55ecf87962d0 2023-07-21T18:54:23.088 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.070+0000 7f86c986eb80 1 -- 172.21.15.39:0/3067477230 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ecf863b490 con 0x55ecf87962d0 2023-07-21T18:54:23.089 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.070+0000 7f869aff5700 1 -- 172.21.15.39:0/3067477230 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f86b000bbf0 con 0x55ecf87962d0 2023-07-21T18:54:23.089 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.070+0000 7f869aff5700 1 -- 172.21.15.39:0/3067477230 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86b000bdb0 con 0x55ecf87962d0 2023-07-21T18:54:23.089 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.070+0000 7f86c986eb80 10 client.17115 fetch_fsmap learned FSMap version 71 2023-07-21T18:54:23.089 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.070+0000 7f86c986eb80 1 -- 172.21.15.39:0/3067477230 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) v3 -- 0x55ecf84f8c00 con 0x55ecf87962d0 2023-07-21T18:54:23.090 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.070+0000 7f869aff5700 1 -- 172.21.15.39:0/3067477230 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86b0040020 con 0x55ecf87962d0 2023-07-21T18:54:23.091 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.070+0000 7f869aff5700 1 -- 172.21.15.39:0/3067477230 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 71) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f86b00334e0 con 0x55ecf87962d0 2023-07-21T18:54:23.091 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.070+0000 7f86c986eb80 10 client.17115 fetch_fsmap finished waiting for FSMap version 71 2023-07-21T18:54:23.091 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.070+0000 7f86c986eb80 10 client.17115 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:54:23.091 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.070+0000 7f86c986eb80 20 client.17115 populate_metadata read hostname 'smithi039' 2023-07-21T18:54:23.091 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.070+0000 7f86c986eb80 1 --2- 172.21.15.39:0/3067477230 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55ecf87a98b0 0x55ecf87abc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:23.091 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.070+0000 7f86c986eb80 4 client.17115 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:54:23.091 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.070+0000 7f86c986eb80 1 -- 172.21.15.39:0/3067477230 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ecf86dae50 con 0x55ecf87a98b0 2023-07-21T18:54:23.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.071+0000 7f86b6ffd700 1 --2- 172.21.15.39:0/3067477230 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55ecf87a98b0 0x55ecf87abc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:23.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.071+0000 7f869aff5700 10 client.17115 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:54:23.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.071+0000 7f86b6ffd700 1 --2- 172.21.15.39:0/3067477230 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55ecf87a98b0 0x55ecf87abc90 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e350660bd9791d20 server_cookie=4fb044b03c1b4336 in_seq=0 out_seq=0 2023-07-21T18:54:23.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.072+0000 7f869aff5700 1 -- 172.21.15.39:0/3067477230 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ecf86dae50 con 0x55ecf87a98b0 2023-07-21T18:54:23.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.072+0000 7f869aff5700 10 client.17115 handle_command_reply: tid=0 2023-07-21T18:54:23.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.072+0000 7f86c986eb80 1 client.17115 shutdown 2023-07-21T18:54:23.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.072+0000 7f86c986eb80 1 -- 172.21.15.39:0/3067477230 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecf87962d0 msgr2=0x55ecf879a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:54:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.072+0000 7f86c986eb80 1 --2- 172.21.15.39:0/3067477230 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecf87962d0 0x55ecf879a460 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f86b0009040 tx=0x7f86b00216c0 comp rx=0 tx=0).stop 2023-07-21T18:54:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.072+0000 7f86c986eb80 1 -- 172.21.15.39:0/3067477230 shutdown_connections 2023-07-21T18:54:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.072+0000 7f86c986eb80 1 --2- 172.21.15.39:0/3067477230 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55ecf87a98b0 0x55ecf87abc90 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.072+0000 7f86c986eb80 1 --2- 172.21.15.39:0/3067477230 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecf85ec6b0 0x55ecf87999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.072+0000 7f86c986eb80 1 --2- 172.21.15.39:0/3067477230 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecf87962d0 0x55ecf879a460 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.072+0000 7f86c986eb80 1 --2- 172.21.15.39:0/3067477230 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ecf8790580 0x55ecf8799f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.072+0000 7f869aff5700 0 client.17115 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:54:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.072+0000 7f86c986eb80 1 -- 172.21.15.39:0/3067477230 >> 172.21.15.39:0/3067477230 conn(0x55ecf8785be0 msgr2=0x55ecf878a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:54:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.072+0000 7f86c986eb80 1 -- 172.21.15.39:0/3067477230 shutdown_connections 2023-07-21T18:54:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.072+0000 7f86c986eb80 1 -- 172.21.15.39:0/3067477230 wait complete. 2023-07-21T18:54:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.072+0000 7f86c986eb80 20 client.17115 trim_cache size 0 max 16384 2023-07-21T18:54:23.100 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.089+0000 7f86c986eb80 1 Processor -- start 2023-07-21T18:54:23.100 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.089+0000 7f86c986eb80 1 -- start start 2023-07-21T18:54:23.100 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.089+0000 7f86c986eb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ecf85ec6b0 0x55ecf879c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:23.101 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.089+0000 7f86c986eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecf8790580 0x55ecf8798c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:23.101 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.089+0000 7f86c986eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecf87962d0 0x55ecf8797360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:23.101 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.089+0000 7f86c986eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecf8795a10 con 0x55ecf8790580 2023-07-21T18:54:23.101 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.089+0000 7f86c986eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecf87966a0 con 0x55ecf87962d0 2023-07-21T18:54:23.101 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.089+0000 7f86c986eb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecf8787d00 con 0x55ecf85ec6b0 2023-07-21T18:54:23.101 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.089+0000 7f86b67fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecf8790580 0x55ecf8798c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:23.101 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.089+0000 7f86b67fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecf8790580 0x55ecf8798c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:32950/0 (socket says 172.21.15.39:32950) 2023-07-21T18:54:23.102 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.089+0000 7f86b67fc700 1 -- 172.21.15.39:0/405589657 learned_addr learned my addr 172.21.15.39:0/405589657 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:54:23.102 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.089+0000 7f86b77fe700 1 --2- 172.21.15.39:0/405589657 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecf87962d0 0x55ecf8797360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:23.102 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.089+0000 7f86b6ffd700 1 --2- 172.21.15.39:0/405589657 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ecf85ec6b0 0x55ecf879c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:23.102 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.089+0000 7f86b67fc700 1 -- 172.21.15.39:0/405589657 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ecf85ec6b0 msgr2=0x55ecf879c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:54:23.102 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.089+0000 7f86b67fc700 1 --2- 172.21.15.39:0/405589657 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ecf85ec6b0 0x55ecf879c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:23.102 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.089+0000 7f86b67fc700 1 -- 172.21.15.39:0/405589657 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecf87962d0 msgr2=0x55ecf8797360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:54:23.102 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.089+0000 7f86b67fc700 1 --2- 172.21.15.39:0/405589657 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecf87962d0 0x55ecf8797360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:23.102 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.089+0000 7f86b67fc700 1 -- 172.21.15.39:0/405589657 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ecf85361f0 con 0x55ecf8790580 2023-07-21T18:54:23.105 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.090+0000 7f86b67fc700 1 --2- 172.21.15.39:0/405589657 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecf8790580 0x55ecf8798c70 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f86ac009f00 tx=0x7f86ac009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac708775eddb482e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:54:23.105 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.090+0000 7f869dffb700 1 -- 172.21.15.39:0/405589657 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86ac017050 con 0x55ecf8790580 2023-07-21T18:54:23.105 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.090+0000 7f869dffb700 1 -- 172.21.15.39:0/405589657 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86ac018040 con 0x55ecf8790580 2023-07-21T18:54:23.106 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.090+0000 7f869dffb700 1 -- 172.21.15.39:0/405589657 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86ac003b40 con 0x55ecf8790580 2023-07-21T18:54:23.106 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.091+0000 7f86c986eb80 1 -- 172.21.15.39:0/405589657 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecf8790580 msgr2=0x55ecf8798c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:54:23.106 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.091+0000 7f86c986eb80 1 --2- 172.21.15.39:0/405589657 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecf8790580 0x55ecf8798c70 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f86ac009f00 tx=0x7f86ac009260 comp rx=0 tx=0).stop 2023-07-21T18:54:23.106 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.092+0000 7f86c986eb80 1 -- 172.21.15.39:0/405589657 shutdown_connections 2023-07-21T18:54:23.106 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.092+0000 7f86c986eb80 1 --2- 172.21.15.39:0/405589657 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecf87962d0 0x55ecf8797360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:23.106 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.092+0000 7f86c986eb80 1 --2- 172.21.15.39:0/405589657 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ecf85ec6b0 0x55ecf879c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:23.106 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.092+0000 7f86c986eb80 1 --2- 172.21.15.39:0/405589657 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecf8790580 0x55ecf8798c70 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:23.106 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.092+0000 7f86c986eb80 1 -- 172.21.15.39:0/405589657 >> 172.21.15.39:0/405589657 conn(0x55ecf8785be0 msgr2=0x55ecf87951f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:54:23.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.092+0000 7f86c986eb80 1 -- 172.21.15.39:0/405589657 shutdown_connections 2023-07-21T18:54:23.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.092+0000 7f86c986eb80 1 -- 172.21.15.39:0/405589657 wait complete. 2023-07-21T18:54:23.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.092+0000 7f86c986eb80 1 -- start start 2023-07-21T18:54:23.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.092+0000 7f86c986eb80 1 Processor -- start 2023-07-21T18:54:23.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.092+0000 7f86c986eb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ecf85ec6b0 0x55ecf87a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:23.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.092+0000 7f86c986eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecf8790580 0x55ecf879f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:23.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.093+0000 7f86c986eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecf87962d0 0x55ecf879f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:23.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.093+0000 7f86c986eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecf879ff20 con 0x55ecf8790580 2023-07-21T18:54:23.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.093+0000 7f86c986eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecf87a0090 con 0x55ecf87962d0 2023-07-21T18:54:23.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.093+0000 7f86c986eb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecf87fd720 con 0x55ecf85ec6b0 2023-07-21T18:54:23.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.093+0000 7f86c986eb80 10 client.0 _ll_register_callbacks cb 0x7f86ac0099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:54:23.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.093+0000 7f86b67fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecf8790580 0x55ecf879f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:23.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.093+0000 7f86b67fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecf8790580 0x55ecf879f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:32956/0 (socket says 172.21.15.39:32956) 2023-07-21T18:54:23.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.093+0000 7f86b67fc700 1 -- 172.21.15.39:0/1981753932 learned_addr learned my addr 172.21.15.39:0/1981753932 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:54:23.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.093+0000 7f86b6ffd700 1 --2- 172.21.15.39:0/1981753932 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ecf85ec6b0 0x55ecf87a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.093+0000 7f86b77fe700 1 --2- 172.21.15.39:0/1981753932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecf87962d0 0x55ecf879f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.093+0000 7f86b67fc700 1 -- 172.21.15.39:0/1981753932 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ecf85ec6b0 msgr2=0x55ecf87a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:54:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.093+0000 7f86b67fc700 1 --2- 172.21.15.39:0/1981753932 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ecf85ec6b0 0x55ecf87a0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.093+0000 7f86b67fc700 1 -- 172.21.15.39:0/1981753932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecf87962d0 msgr2=0x55ecf879f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:54:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.093+0000 7f86b67fc700 1 --2- 172.21.15.39:0/1981753932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecf87962d0 0x55ecf879f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.093+0000 7f86b67fc700 1 -- 172.21.15.39:0/1981753932 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86ac009be0 con 0x55ecf8790580 2023-07-21T18:54:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.093+0000 7f86b67fc700 1 --2- 172.21.15.39:0/1981753932 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecf8790580 0x55ecf879f2b0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f86ac012420 tx=0x7f86ac012c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=414c3b9d628bdbf5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:54:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.093+0000 7f869aff5700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:54:23.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.093+0000 7f869aff5700 1 -- 172.21.15.39:0/1981753932 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86ac012e20 con 0x55ecf8790580 2023-07-21T18:54:23.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.093+0000 7f869aff5700 1 -- 172.21.15.39:0/1981753932 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f86ac004750 con 0x55ecf8790580 2023-07-21T18:54:23.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.093+0000 7f86c986eb80 1 -- 172.21.15.39:0/1981753932 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f86b0008e70 con 0x55ecf8790580 2023-07-21T18:54:23.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.094+0000 7f869aff5700 1 -- 172.21.15.39:0/1981753932 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86ac004910 con 0x55ecf8790580 2023-07-21T18:54:23.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.094+0000 7f86c986eb80 10 client.17121 fetch_fsmap learned FSMap version 71 2023-07-21T18:54:23.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.094+0000 7f86c986eb80 1 -- 172.21.15.39:0/1981753932 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) v3 -- 0x55ecf850e250 con 0x55ecf8790580 2023-07-21T18:54:23.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.094+0000 7f869aff5700 1 -- 172.21.15.39:0/1981753932 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86b0008e70 con 0x55ecf8790580 2023-07-21T18:54:23.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.094+0000 7f869aff5700 1 -- 172.21.15.39:0/1981753932 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 71) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f86ac01dc10 con 0x55ecf8790580 2023-07-21T18:54:23.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.094+0000 7f86c986eb80 10 client.17121 fetch_fsmap finished waiting for FSMap version 71 2023-07-21T18:54:23.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.094+0000 7f86c986eb80 10 client.17121 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:54:23.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.094+0000 7f86c986eb80 20 client.17121 populate_metadata read hostname 'smithi039' 2023-07-21T18:54:23.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.094+0000 7f86c986eb80 1 --2- 172.21.15.39:0/1981753932 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55ecf87fecf0 0x55ecf8801100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:23.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.094+0000 7f86c986eb80 4 client.17121 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:54:23.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.094+0000 7f86c986eb80 1 -- 172.21.15.39:0/1981753932 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ecf869cff0 con 0x55ecf87fecf0 2023-07-21T18:54:23.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.094+0000 7f86b6ffd700 1 --2- 172.21.15.39:0/1981753932 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55ecf87fecf0 0x55ecf8801100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:23.112 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.095+0000 7f869aff5700 10 client.17121 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:54:23.112 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.095+0000 7f86b6ffd700 1 --2- 172.21.15.39:0/1981753932 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55ecf87fecf0 0x55ecf8801100 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8329e2fa5943dc59 server_cookie=1206600578fae63b in_seq=0 out_seq=0 2023-07-21T18:54:23.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.133+0000 7f869aff5700 1 -- 172.21.15.39:0/1981753932 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ecf869cff0 con 0x55ecf87fecf0 2023-07-21T18:54:23.145 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.133+0000 7f869aff5700 10 client.17121 handle_command_reply: tid=0 2023-07-21T18:54:23.145 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.133+0000 7f86c986eb80 1 client.17121 shutdown 2023-07-21T18:54:23.145 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.133+0000 7f86c986eb80 1 -- 172.21.15.39:0/1981753932 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecf8790580 msgr2=0x55ecf879f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:54:23.145 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.133+0000 7f86c986eb80 1 --2- 172.21.15.39:0/1981753932 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecf8790580 0x55ecf879f2b0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f86ac012420 tx=0x7f86ac012c70 comp rx=0 tx=0).stop 2023-07-21T18:54:23.145 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.133+0000 7f86c986eb80 1 -- 172.21.15.39:0/1981753932 shutdown_connections 2023-07-21T18:54:23.145 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.133+0000 7f86c986eb80 1 --2- 172.21.15.39:0/1981753932 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55ecf87fecf0 0x55ecf8801100 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:23.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.133+0000 7f86c986eb80 1 --2- 172.21.15.39:0/1981753932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecf87962d0 0x55ecf879f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:23.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.133+0000 7f86c986eb80 1 --2- 172.21.15.39:0/1981753932 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ecf85ec6b0 0x55ecf87a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:23.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.133+0000 7f869aff5700 0 client.17121 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:54:23.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.133+0000 7f86c986eb80 1 --2- 172.21.15.39:0/1981753932 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecf8790580 0x55ecf879f2b0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:23.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.133+0000 7f86c986eb80 1 -- 172.21.15.39:0/1981753932 >> 172.21.15.39:0/1981753932 conn(0x55ecf8785be0 msgr2=0x55ecf878f030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:54:23.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.133+0000 7f86c986eb80 1 -- 172.21.15.39:0/1981753932 shutdown_connections 2023-07-21T18:54:23.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.133+0000 7f86c986eb80 1 -- 172.21.15.39:0/1981753932 wait complete. 2023-07-21T18:54:23.147 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.133+0000 7f86c986eb80 20 client.17121 trim_cache size 0 max 16384 2023-07-21T18:54:23.147 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:54:23.147 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:54:23.147 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:54:23.147 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:54:23.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.134+0000 7f86bd6e8700 1 -- 172.21.15.39:0/3776222261 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f86a00a12c0 msgr2=0x7f86a00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:54:23.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.134+0000 7f86bd6e8700 1 --2- 172.21.15.39:0/3776222261 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f86a00a12c0 0x7f86a00a3770 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f86b00093d0 tx=0x7f86b0007420 comp rx=0 tx=0).stop 2023-07-21T18:54:23.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.134+0000 7f86bd6e8700 1 -- 172.21.15.39:0/3776222261 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f86b80fdff0 msgr2=0x7f86b8199a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:54:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.134+0000 7f86bd6e8700 1 --2- 172.21.15.39:0/3776222261 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f86b80fdff0 0x7f86b8199a50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f86a801fac0 tx=0x7f86a801fdd0 comp rx=0 tx=0).stop 2023-07-21T18:54:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.134+0000 7f86bd6e8700 1 -- 172.21.15.39:0/3776222261 shutdown_connections 2023-07-21T18:54:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.134+0000 7f86bd6e8700 1 --2- 172.21.15.39:0/3776222261 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f86a00a12c0 0x7f86a00a3770 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.134+0000 7f86bd6e8700 1 --2- 172.21.15.39:0/3776222261 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86b8104a50 0x7f86b81938b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.134+0000 7f86bd6e8700 1 --2- 172.21.15.39:0/3776222261 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f86b80fdff0 0x7f86b8199a50 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.134+0000 7f86bd6e8700 1 --2- 172.21.15.39:0/3776222261 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f86b80fe9a0 0x7f86b8199f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.134+0000 7f86bd6e8700 1 -- 172.21.15.39:0/3776222261 >> 172.21.15.39:0/3776222261 conn(0x7f86b8075f90 msgr2=0x7f86b8101680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:54:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.134+0000 7f86bd6e8700 1 -- 172.21.15.39:0/3776222261 shutdown_connections 2023-07-21T18:54:23.150 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:23.134+0000 7f86bd6e8700 1 -- 172.21.15.39:0/3776222261 wait complete. 2023-07-21T18:54:23.295 INFO:tasks.workunit.client.0.smithi039.stdout: 1 707014 5.22 MB/sec execute 576 sec latency 47.166 ms 2023-07-21T18:54:24.295 INFO:tasks.workunit.client.0.smithi039.stdout: 1 708313 5.22 MB/sec execute 577 sec latency 24.622 ms 2023-07-21T18:54:24.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:54:24 smithi193 ceph-mon[98204]: pgmap v397: 97 pgs: 97 active+clean; 271 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 1.5 MiB/s wr, 129 op/s 2023-07-21T18:54:24.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:54:24 smithi155 ceph-mon[97934]: pgmap v397: 97 pgs: 97 active+clean; 271 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 1.5 MiB/s wr, 129 op/s 2023-07-21T18:54:24.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:54:24 smithi039 ceph-mon[96489]: pgmap v397: 97 pgs: 97 active+clean; 271 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 1.5 MiB/s wr, 129 op/s 2023-07-21T18:54:25.295 INFO:tasks.workunit.client.0.smithi039.stdout: 1 708365 5.21 MB/sec execute 578 sec latency 949.482 ms 2023-07-21T18:54:26.295 INFO:tasks.workunit.client.0.smithi039.stdout: 1 708969 5.20 MB/sec execute 579 sec latency 1737.299 ms 2023-07-21T18:54:26.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:54:26 smithi193 ceph-mon[98204]: pgmap v398: 97 pgs: 97 active+clean; 271 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 1.4 MiB/s wr, 121 op/s 2023-07-21T18:54:26.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:54:26 smithi155 ceph-mon[97934]: pgmap v398: 97 pgs: 97 active+clean; 271 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 1.4 MiB/s wr, 121 op/s 2023-07-21T18:54:26.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:54:26 smithi039 ceph-mon[96489]: pgmap v398: 97 pgs: 97 active+clean; 271 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 1.4 MiB/s wr, 121 op/s 2023-07-21T18:54:27.295 INFO:tasks.workunit.client.0.smithi039.stdout: 1 710438 5.21 MB/sec execute 580 sec latency 72.908 ms 2023-07-21T18:54:28.295 INFO:tasks.workunit.client.0.smithi039.stdout: 1 711633 5.21 MB/sec execute 581 sec latency 21.984 ms 2023-07-21T18:54:28.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:54:28 smithi193 ceph-mon[98204]: pgmap v399: 97 pgs: 97 active+clean; 270 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 185 op/s 2023-07-21T18:54:28.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:54:28 smithi155 ceph-mon[97934]: pgmap v399: 97 pgs: 97 active+clean; 270 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 185 op/s 2023-07-21T18:54:28.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:54:28 smithi039 ceph-mon[96489]: pgmap v399: 97 pgs: 97 active+clean; 270 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 185 op/s 2023-07-21T18:54:29.295 INFO:tasks.workunit.client.0.smithi039.stdout: 1 711916 5.20 MB/sec execute 582 sec latency 778.698 ms 2023-07-21T18:54:30.295 INFO:tasks.workunit.client.0.smithi039.stdout: 1 711916 5.19 MB/sec execute 583 sec latency 1778.779 ms 2023-07-21T18:54:30.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:54:30 smithi193 ceph-mon[98204]: pgmap v400: 97 pgs: 97 active+clean; 270 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 1.3 MiB/s wr, 127 op/s 2023-07-21T18:54:30.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:54:30 smithi155 ceph-mon[97934]: pgmap v400: 97 pgs: 97 active+clean; 270 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 1.3 MiB/s wr, 127 op/s 2023-07-21T18:54:30.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:54:30 smithi039 ceph-mon[96489]: pgmap v400: 97 pgs: 97 active+clean; 270 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 1.3 MiB/s wr, 127 op/s 2023-07-21T18:54:31.295 INFO:tasks.workunit.client.0.smithi039.stdout: 1 711916 5.19 MB/sec execute 584 sec latency 2778.863 ms 2023-07-21T18:54:32.295 INFO:tasks.workunit.client.0.smithi039.stdout: 1 713101 5.19 MB/sec execute 585 sec latency 3201.038 ms 2023-07-21T18:54:32.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:54:32 smithi193 ceph-mon[98204]: pgmap v401: 97 pgs: 97 active+clean; 274 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 1.9 MiB/s wr, 182 op/s 2023-07-21T18:54:32.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:54:32 smithi155 ceph-mon[97934]: pgmap v401: 97 pgs: 97 active+clean; 274 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 1.9 MiB/s wr, 182 op/s 2023-07-21T18:54:32.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:54:32 smithi039 ceph-mon[96489]: pgmap v401: 97 pgs: 97 active+clean; 274 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 1.9 MiB/s wr, 182 op/s 2023-07-21T18:54:33.295 INFO:tasks.workunit.client.0.smithi039.stdout: 1 714357 5.19 MB/sec execute 586 sec latency 14.131 ms 2023-07-21T18:54:34.296 INFO:tasks.workunit.client.0.smithi039.stdout: 1 715457 5.19 MB/sec execute 587 sec latency 240.821 ms 2023-07-21T18:54:34.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:54:34 smithi193 ceph-mon[98204]: pgmap v402: 97 pgs: 97 active+clean; 274 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 1.2 MiB/s wr, 118 op/s 2023-07-21T18:54:34.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:54:34 smithi155 ceph-mon[97934]: pgmap v402: 97 pgs: 97 active+clean; 274 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 1.2 MiB/s wr, 118 op/s 2023-07-21T18:54:34.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:54:34 smithi039 ceph-mon[96489]: pgmap v402: 97 pgs: 97 active+clean; 274 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 1.2 MiB/s wr, 118 op/s 2023-07-21T18:54:35.296 INFO:tasks.workunit.client.0.smithi039.stdout: 1 715457 5.18 MB/sec execute 588 sec latency 1240.934 ms 2023-07-21T18:54:36.271 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.260+0000 7f6b56626700 1 -- 172.21.15.39:0/4120106099 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b500ff3b0 msgr2=0x7f6b500ff780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:54:36.271 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.260+0000 7f6b56626700 1 --2- 172.21.15.39:0/4120106099 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b500ff3b0 0x7f6b500ff780 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f6b400099c0 tx=0x7f6b40009cd0 comp rx=0 tx=0).stop 2023-07-21T18:54:36.272 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.260+0000 7f6b56626700 1 -- 172.21.15.39:0/4120106099 shutdown_connections 2023-07-21T18:54:36.272 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.260+0000 7f6b56626700 1 --2- 172.21.15.39:0/4120106099 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b50103ed0 0x7f6b5010baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:36.272 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.260+0000 7f6b56626700 1 --2- 172.21.15.39:0/4120106099 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b500ff3b0 0x7f6b500ff780 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:36.272 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.260+0000 7f6b56626700 1 --2- 172.21.15.39:0/4120106099 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b500ffd50 0x7f6b501037f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:36.272 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.260+0000 7f6b56626700 1 -- 172.21.15.39:0/4120106099 >> 172.21.15.39:0/4120106099 conn(0x7f6b50075f90 msgr2=0x7f6b50076390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:54:36.272 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.260+0000 7f6b56626700 1 -- 172.21.15.39:0/4120106099 shutdown_connections 2023-07-21T18:54:36.272 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.260+0000 7f6b56626700 1 -- 172.21.15.39:0/4120106099 wait complete. 2023-07-21T18:54:36.274 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.261+0000 7f6b56626700 1 Processor -- start 2023-07-21T18:54:36.274 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.261+0000 7f6b56626700 1 -- start start 2023-07-21T18:54:36.275 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.261+0000 7f6b56626700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b500ff3b0 0x7f6b50198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:36.275 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.261+0000 7f6b56626700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b500ffd50 0x7f6b50198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:36.275 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.261+0000 7f6b56626700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b50103ed0 0x7f6b5019cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:36.275 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.261+0000 7f6b56626700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b5019d1f0 con 0x7f6b50103ed0 2023-07-21T18:54:36.275 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.261+0000 7f6b56626700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b5019d360 con 0x7f6b500ffd50 2023-07-21T18:54:36.275 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.261+0000 7f6b56626700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b5019d520 con 0x7f6b500ff3b0 2023-07-21T18:54:36.275 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.261+0000 7f6b54bc3700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b50103ed0 0x7f6b5019cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.261+0000 7f6b54bc3700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b50103ed0 0x7f6b5019cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:32972/0 (socket says 172.21.15.39:32972) 2023-07-21T18:54:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.261+0000 7f6b54bc3700 1 -- 172.21.15.39:0/2985421423 learned_addr learned my addr 172.21.15.39:0/2985421423 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:54:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.261+0000 7f6b4ffff700 1 --2- 172.21.15.39:0/2985421423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b500ff3b0 0x7f6b50198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.261+0000 7f6b4f7fe700 1 --2- 172.21.15.39:0/2985421423 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b500ffd50 0x7f6b50198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.262+0000 7f6b54bc3700 1 -- 172.21.15.39:0/2985421423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b500ff3b0 msgr2=0x7f6b50198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:54:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.262+0000 7f6b54bc3700 1 --2- 172.21.15.39:0/2985421423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b500ff3b0 0x7f6b50198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.262+0000 7f6b54bc3700 1 -- 172.21.15.39:0/2985421423 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b500ffd50 msgr2=0x7f6b50198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:54:36.277 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.262+0000 7f6b54bc3700 1 --2- 172.21.15.39:0/2985421423 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b500ffd50 0x7f6b50198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:36.277 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.262+0000 7f6b54bc3700 1 -- 172.21.15.39:0/2985421423 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b400095f0 con 0x7f6b50103ed0 2023-07-21T18:54:36.277 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.262+0000 7f6b54bc3700 1 --2- 172.21.15.39:0/2985421423 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b50103ed0 0x7f6b5019cbb0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f6b48007d70 tx=0x7f6b4800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:54:36.277 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.262+0000 7f6b4d7fa700 1 -- 172.21.15.39:0/2985421423 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b4800ac90 con 0x7f6b50103ed0 2023-07-21T18:54:36.277 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.262+0000 7f6b4d7fa700 1 -- 172.21.15.39:0/2985421423 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b4800adf0 con 0x7f6b50103ed0 2023-07-21T18:54:36.278 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.262+0000 7f6b56626700 1 -- 172.21.15.39:0/2985421423 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b5019d800 con 0x7f6b50103ed0 2023-07-21T18:54:36.278 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.262+0000 7f6b4d7fa700 1 -- 172.21.15.39:0/2985421423 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b480115e0 con 0x7f6b50103ed0 2023-07-21T18:54:36.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.262+0000 7f6b56626700 1 -- 172.21.15.39:0/2985421423 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b501a5040 con 0x7f6b50103ed0 2023-07-21T18:54:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.264+0000 7f6b4d7fa700 1 -- 172.21.15.39:0/2985421423 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f6b48012420 con 0x7f6b50103ed0 2023-07-21T18:54:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.265+0000 7f6b4d7fa700 1 --2- 172.21.15.39:0/2985421423 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f6b380a12c0 0x7f6b380a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.265+0000 7f6b4ffff700 1 --2- 172.21.15.39:0/2985421423 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f6b380a12c0 0x7f6b380a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.265+0000 7f6b4d7fa700 1 -- 172.21.15.39:0/2985421423 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f6b480ccd30 con 0x7f6b50103ed0 2023-07-21T18:54:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.266+0000 7f6b4ffff700 1 --2- 172.21.15.39:0/2985421423 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f6b380a12c0 0x7f6b380a3770 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6b40000c00 tx=0x7f6b40003680 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:54:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.266+0000 7f6b627acb80 1 Processor -- start 2023-07-21T18:54:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.266+0000 7f6b627acb80 1 -- start start 2023-07-21T18:54:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.266+0000 7f6b627acb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594591b76b0 0x559459360d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.266+0000 7f6b627acb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594593612d0 0x55945935b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.266+0000 7f6b627acb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55945935b580 0x55945935b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.266+0000 7f6b627acb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x559459260c50 con 0x55945935b580 2023-07-21T18:54:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.266+0000 7f6b627acb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594592b39e0 con 0x5594593612d0 2023-07-21T18:54:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.267+0000 7f6b627acb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594592ace70 con 0x5594591b76b0 2023-07-21T18:54:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.267+0000 7f6b54bc3700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55945935b580 0x55945935b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.267+0000 7f6b54bc3700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55945935b580 0x55945935b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:32980/0 (socket says 172.21.15.39:32980) 2023-07-21T18:54:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.267+0000 7f6b54bc3700 1 -- 172.21.15.39:0/313847538 learned_addr learned my addr 172.21.15.39:0/313847538 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:54:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.267+0000 7f6b4f7fe700 1 --2- 172.21.15.39:0/313847538 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594593612d0 0x55945935b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.267+0000 7f6b4ffff700 1 --2- 172.21.15.39:0/313847538 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594591b76b0 0x559459360d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.267+0000 7f6b54bc3700 1 -- 172.21.15.39:0/313847538 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594591b76b0 msgr2=0x559459360d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:54:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.267+0000 7f6b54bc3700 1 --2- 172.21.15.39:0/313847538 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594591b76b0 0x559459360d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.267+0000 7f6b54bc3700 1 -- 172.21.15.39:0/313847538 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594593612d0 msgr2=0x55945935b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:54:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.267+0000 7f6b54bc3700 1 --2- 172.21.15.39:0/313847538 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594593612d0 0x55945935b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.267+0000 7f6b54bc3700 1 -- 172.21.15.39:0/313847538 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5594590dad80 con 0x55945935b580 2023-07-21T18:54:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.267+0000 7f6b54bc3700 1 --2- 172.21.15.39:0/313847538 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55945935b580 0x55945935b9d0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f6b48096410 tx=0x7f6b48099a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8100a2e61d8d185a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:54:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.267+0000 7f6b367fc700 1 -- 172.21.15.39:0/313847538 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b48019310 con 0x55945935b580 2023-07-21T18:54:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.267+0000 7f6b367fc700 1 -- 172.21.15.39:0/313847538 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b48099f10 con 0x55945935b580 2023-07-21T18:54:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.267+0000 7f6b367fc700 1 -- 172.21.15.39:0/313847538 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b48099660 con 0x55945935b580 2023-07-21T18:54:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.268+0000 7f6b627acb80 1 -- 172.21.15.39:0/313847538 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55945935b580 msgr2=0x55945935b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:54:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.268+0000 7f6b627acb80 1 --2- 172.21.15.39:0/313847538 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55945935b580 0x55945935b9d0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f6b48096410 tx=0x7f6b48099a50 comp rx=0 tx=0).stop 2023-07-21T18:54:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.268+0000 7f6b627acb80 1 -- 172.21.15.39:0/313847538 shutdown_connections 2023-07-21T18:54:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.268+0000 7f6b627acb80 1 --2- 172.21.15.39:0/313847538 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55945935b580 0x55945935b9d0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.268+0000 7f6b627acb80 1 --2- 172.21.15.39:0/313847538 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594591b76b0 0x559459360d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.268+0000 7f6b627acb80 1 --2- 172.21.15.39:0/313847538 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594593612d0 0x55945935b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.268+0000 7f6b627acb80 1 -- 172.21.15.39:0/313847538 >> 172.21.15.39:0/313847538 conn(0x559459350be0 msgr2=0x559459355f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:54:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.268+0000 7f6b627acb80 1 -- 172.21.15.39:0/313847538 shutdown_connections 2023-07-21T18:54:36.288 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.268+0000 7f6b627acb80 1 -- 172.21.15.39:0/313847538 wait complete. 2023-07-21T18:54:36.288 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.269+0000 7f6b627acb80 1 -- start start 2023-07-21T18:54:36.288 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.269+0000 7f6b627acb80 1 Processor -- start 2023-07-21T18:54:36.288 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.269+0000 7f6b627acb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594591b76b0 0x559459367630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:36.288 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.269+0000 7f6b627acb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55945935b580 0x559459364920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:36.288 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.269+0000 7f6b627acb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594593612d0 0x559459365010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:36.288 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.269+0000 7f6b627acb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55945924ca60 con 0x55945935b580 2023-07-21T18:54:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.269+0000 7f6b627acb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55945924c760 con 0x5594593612d0 2023-07-21T18:54:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.269+0000 7f6b627acb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594591308c0 con 0x5594591b76b0 2023-07-21T18:54:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.269+0000 7f6b627acb80 10 client.0 _ll_register_callbacks cb 0x55945924a700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:54:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.269+0000 7f6b4f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55945935b580 0x559459364920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.269+0000 7f6b4ffff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594591b76b0 0x559459367630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.269+0000 7f6b4f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55945935b580 0x559459364920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:32984/0 (socket says 172.21.15.39:32984) 2023-07-21T18:54:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.269+0000 7f6b4f7fe700 1 -- 172.21.15.39:0/1271884512 learned_addr learned my addr 172.21.15.39:0/1271884512 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:54:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.269+0000 7f6b54bc3700 1 --2- 172.21.15.39:0/1271884512 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594593612d0 0x559459365010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:36.290 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.270+0000 7f6b4f7fe700 1 -- 172.21.15.39:0/1271884512 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594591b76b0 msgr2=0x559459367630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:54:36.290 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.270+0000 7f6b4f7fe700 1 --2- 172.21.15.39:0/1271884512 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594591b76b0 0x559459367630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:36.290 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.270+0000 7f6b4f7fe700 1 -- 172.21.15.39:0/1271884512 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594593612d0 msgr2=0x559459365010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:54:36.290 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.270+0000 7f6b4f7fe700 1 --2- 172.21.15.39:0/1271884512 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594593612d0 0x559459365010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:36.290 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.270+0000 7f6b4f7fe700 1 -- 172.21.15.39:0/1271884512 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b501a5040 con 0x55945935b580 2023-07-21T18:54:36.290 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.270+0000 7f6b4ffff700 1 --2- 172.21.15.39:0/1271884512 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594591b76b0 0x559459367630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:54:36.290 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.270+0000 7f6b4f7fe700 1 --2- 172.21.15.39:0/1271884512 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55945935b580 0x559459364920 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f6b50100120 tx=0x7f6b4400bba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7e218e73a1d27c3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:54:36.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.270+0000 7f6b2f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:54:36.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.270+0000 7f6b2f7fe700 1 -- 172.21.15.39:0/1271884512 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b44016070 con 0x55945935b580 2023-07-21T18:54:36.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.270+0000 7f6b2f7fe700 1 -- 172.21.15.39:0/1271884512 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f6b44003a50 con 0x55945935b580 2023-07-21T18:54:36.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.270+0000 7f6b627acb80 1 -- 172.21.15.39:0/1271884512 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559459206490 con 0x55945935b580 2023-07-21T18:54:36.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.270+0000 7f6b2f7fe700 1 -- 172.21.15.39:0/1271884512 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b44003c10 con 0x55945935b580 2023-07-21T18:54:36.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.270+0000 7f6b627acb80 10 client.17133 fetch_fsmap learned FSMap version 71 2023-07-21T18:54:36.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.270+0000 7f6b627acb80 1 -- 172.21.15.39:0/1271884512 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) v3 -- 0x5594590c3c00 con 0x55945935b580 2023-07-21T18:54:36.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.270+0000 7f6b2f7fe700 1 -- 172.21.15.39:0/1271884512 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559459206490 con 0x55945935b580 2023-07-21T18:54:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.271+0000 7f6b2f7fe700 1 -- 172.21.15.39:0/1271884512 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 71) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6b440052f0 con 0x55945935b580 2023-07-21T18:54:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.271+0000 7f6b627acb80 10 client.17133 fetch_fsmap finished waiting for FSMap version 71 2023-07-21T18:54:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.271+0000 7f6b627acb80 10 client.17133 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:54:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.271+0000 7f6b627acb80 20 client.17133 populate_metadata read hostname 'smithi039' 2023-07-21T18:54:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.271+0000 7f6b627acb80 1 --2- 172.21.15.39:0/1271884512 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x559459374b20 0x559459376f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.271+0000 7f6b627acb80 4 client.17133 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:54:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.271+0000 7f6b627acb80 1 -- 172.21.15.39:0/1271884512 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5594592a5e50 con 0x559459374b20 2023-07-21T18:54:36.293 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.271+0000 7f6b4ffff700 1 --2- 172.21.15.39:0/1271884512 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x559459374b20 0x559459376f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:36.293 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.272+0000 7f6b2f7fe700 10 client.17133 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:54:36.293 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.272+0000 7f6b4ffff700 1 --2- 172.21.15.39:0/1271884512 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x559459374b20 0x559459376f30 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=538f28bee93ce085 server_cookie=41d4128c8ccc8c3c in_seq=0 out_seq=0 2023-07-21T18:54:36.294 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.272+0000 7f6b2f7fe700 1 -- 172.21.15.39:0/1271884512 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5594592a5e50 con 0x559459374b20 2023-07-21T18:54:36.294 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.272+0000 7f6b2f7fe700 10 client.17133 handle_command_reply: tid=0 2023-07-21T18:54:36.294 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.272+0000 7f6b627acb80 1 client.17133 shutdown 2023-07-21T18:54:36.294 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.273+0000 7f6b627acb80 1 -- 172.21.15.39:0/1271884512 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55945935b580 msgr2=0x559459364920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:54:36.294 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.273+0000 7f6b627acb80 1 --2- 172.21.15.39:0/1271884512 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55945935b580 0x559459364920 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f6b50100120 tx=0x7f6b4400bba0 comp rx=0 tx=0).stop 2023-07-21T18:54:36.295 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.273+0000 7f6b627acb80 1 -- 172.21.15.39:0/1271884512 shutdown_connections 2023-07-21T18:54:36.295 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.273+0000 7f6b627acb80 1 --2- 172.21.15.39:0/1271884512 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x559459374b20 0x559459376f30 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:36.295 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.273+0000 7f6b627acb80 1 --2- 172.21.15.39:0/1271884512 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594593612d0 0x559459365010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:36.295 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.273+0000 7f6b627acb80 1 --2- 172.21.15.39:0/1271884512 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594591b76b0 0x559459367630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:36.295 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.273+0000 7f6b2f7fe700 0 client.17133 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:54:36.295 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.273+0000 7f6b627acb80 1 --2- 172.21.15.39:0/1271884512 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55945935b580 0x559459364920 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:36.295 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.273+0000 7f6b627acb80 1 -- 172.21.15.39:0/1271884512 >> 172.21.15.39:0/1271884512 conn(0x559459350be0 msgr2=0x5594593554e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:54:36.295 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.273+0000 7f6b627acb80 1 -- 172.21.15.39:0/1271884512 shutdown_connections 2023-07-21T18:54:36.296 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.273+0000 7f6b627acb80 1 -- 172.21.15.39:0/1271884512 wait complete. 2023-07-21T18:54:36.296 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.273+0000 7f6b627acb80 20 client.17133 trim_cache size 0 max 16384 2023-07-21T18:54:36.296 INFO:tasks.workunit.client.0.smithi039.stdout: 1 715457 5.17 MB/sec execute 589 sec latency 2240.990 ms 2023-07-21T18:54:36.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.288+0000 7f6b627acb80 1 Processor -- start 2023-07-21T18:54:36.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.288+0000 7f6b627acb80 1 -- start start 2023-07-21T18:54:36.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.289+0000 7f6b627acb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594591b76b0 0x55945936a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:36.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.289+0000 7f6b627acb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55945935b580 0x5594593601d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.289+0000 7f6b627acb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594593612d0 0x559459355c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.289+0000 7f6b627acb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594593609f0 con 0x55945935b580 2023-07-21T18:54:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.289+0000 7f6b627acb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559459365e90 con 0x5594593612d0 2023-07-21T18:54:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.289+0000 7f6b627acb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55945935f6b0 con 0x5594591b76b0 2023-07-21T18:54:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.289+0000 7f6b4f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55945935b580 0x5594593601d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.289+0000 7f6b4f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55945935b580 0x5594593601d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:32992/0 (socket says 172.21.15.39:32992) 2023-07-21T18:54:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.289+0000 7f6b4f7fe700 1 -- 172.21.15.39:0/2042258402 learned_addr learned my addr 172.21.15.39:0/2042258402 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:54:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.289+0000 7f6b54bc3700 1 --2- 172.21.15.39:0/2042258402 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594593612d0 0x559459355c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.289+0000 7f6b4ffff700 1 --2- 172.21.15.39:0/2042258402 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594591b76b0 0x55945936a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.289+0000 7f6b4f7fe700 1 -- 172.21.15.39:0/2042258402 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594591b76b0 msgr2=0x55945936a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:54:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.289+0000 7f6b4f7fe700 1 --2- 172.21.15.39:0/2042258402 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594591b76b0 0x55945936a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.289+0000 7f6b4f7fe700 1 -- 172.21.15.39:0/2042258402 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594593612d0 msgr2=0x559459355c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:54:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.289+0000 7f6b4f7fe700 1 --2- 172.21.15.39:0/2042258402 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594593612d0 0x559459355c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.289+0000 7f6b4f7fe700 1 -- 172.21.15.39:0/2042258402 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5594591011f0 con 0x55945935b580 2023-07-21T18:54:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.289+0000 7f6b4f7fe700 1 --2- 172.21.15.39:0/2042258402 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55945935b580 0x5594593601d0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f6b44004250 tx=0x7f6b44005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a12e607cd2ca4c02 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:54:36.305 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.289+0000 7f6b36ffd700 1 -- 172.21.15.39:0/2042258402 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b44016040 con 0x55945935b580 2023-07-21T18:54:36.305 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.289+0000 7f6b36ffd700 1 -- 172.21.15.39:0/2042258402 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b44024760 con 0x55945935b580 2023-07-21T18:54:36.305 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.289+0000 7f6b36ffd700 1 -- 172.21.15.39:0/2042258402 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b4401b6d0 con 0x55945935b580 2023-07-21T18:54:36.305 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.290+0000 7f6b627acb80 1 -- 172.21.15.39:0/2042258402 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55945935b580 msgr2=0x5594593601d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:54:36.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.290+0000 7f6b627acb80 1 --2- 172.21.15.39:0/2042258402 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55945935b580 0x5594593601d0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f6b44004250 tx=0x7f6b44005ce0 comp rx=0 tx=0).stop 2023-07-21T18:54:36.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.290+0000 7f6b627acb80 1 -- 172.21.15.39:0/2042258402 shutdown_connections 2023-07-21T18:54:36.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.290+0000 7f6b627acb80 1 --2- 172.21.15.39:0/2042258402 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594593612d0 0x559459355c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:36.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.290+0000 7f6b627acb80 1 --2- 172.21.15.39:0/2042258402 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594591b76b0 0x55945936a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:36.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.290+0000 7f6b627acb80 1 --2- 172.21.15.39:0/2042258402 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55945935b580 0x5594593601d0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:36.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.290+0000 7f6b627acb80 1 -- 172.21.15.39:0/2042258402 >> 172.21.15.39:0/2042258402 conn(0x559459350be0 msgr2=0x55945935fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:54:36.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.290+0000 7f6b627acb80 1 -- 172.21.15.39:0/2042258402 shutdown_connections 2023-07-21T18:54:36.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.290+0000 7f6b627acb80 1 -- 172.21.15.39:0/2042258402 wait complete. 2023-07-21T18:54:36.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.290+0000 7f6b627acb80 1 -- start start 2023-07-21T18:54:36.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.291+0000 7f6b627acb80 1 Processor -- start 2023-07-21T18:54:36.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.291+0000 7f6b627acb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5594591b76b0 0x5594593bcb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:36.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.291+0000 7f6b627acb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55945935b580 0x5594593bd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:36.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.291+0000 7f6b627acb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594593612d0 0x5594593c19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:36.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.291+0000 7f6b627acb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594593c1f00 con 0x5594591b76b0 2023-07-21T18:54:36.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.291+0000 7f6b627acb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594593c2040 con 0x55945935b580 2023-07-21T18:54:36.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.291+0000 7f6b627acb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594593c2200 con 0x5594593612d0 2023-07-21T18:54:36.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.291+0000 7f6b627acb80 10 client.0 _ll_register_callbacks cb 0x7f6b44004330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:54:36.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.291+0000 7f6b4ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5594591b76b0 0x5594593bcb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:36.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.291+0000 7f6b4ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5594591b76b0 0x5594593bcb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:32996/0 (socket says 172.21.15.39:32996) 2023-07-21T18:54:36.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.291+0000 7f6b4ffff700 1 -- 172.21.15.39:0/117879065 learned_addr learned my addr 172.21.15.39:0/117879065 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:54:36.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.291+0000 7f6b54bc3700 1 --2- 172.21.15.39:0/117879065 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594593612d0 0x5594593c19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:36.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.291+0000 7f6b4f7fe700 1 --2- 172.21.15.39:0/117879065 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55945935b580 0x5594593bd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:36.309 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.291+0000 7f6b4ffff700 1 -- 172.21.15.39:0/117879065 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594593612d0 msgr2=0x5594593c19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:54:36.309 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.291+0000 7f6b4ffff700 1 --2- 172.21.15.39:0/117879065 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594593612d0 0x5594593c19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:36.309 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.291+0000 7f6b4ffff700 1 -- 172.21.15.39:0/117879065 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55945935b580 msgr2=0x5594593bd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:54:36.309 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.291+0000 7f6b4ffff700 1 --2- 172.21.15.39:0/117879065 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55945935b580 0x5594593bd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:36.309 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.291+0000 7f6b4ffff700 1 -- 172.21.15.39:0/117879065 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5594590c3c00 con 0x5594591b76b0 2023-07-21T18:54:36.309 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.291+0000 7f6b4f7fe700 1 --2- 172.21.15.39:0/117879065 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55945935b580 0x5594593bd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:54:36.309 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.292+0000 7f6b4ffff700 1 --2- 172.21.15.39:0/117879065 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5594591b76b0 0x5594593bcb60 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f6b40005f50 tx=0x7f6b40009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dec4c25114593b0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:54:36.309 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.292+0000 7f6b2f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:54:36.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.292+0000 7f6b2f7fe700 1 -- 172.21.15.39:0/117879065 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b4000bec0 con 0x5594591b76b0 2023-07-21T18:54:36.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.292+0000 7f6b627acb80 1 -- 172.21.15.39:0/117879065 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6b48099f10 con 0x5594591b76b0 2023-07-21T18:54:36.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.292+0000 7f6b2f7fe700 1 -- 172.21.15.39:0/117879065 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f6b40036b50 con 0x5594591b76b0 2023-07-21T18:54:36.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.292+0000 7f6b2f7fe700 1 -- 172.21.15.39:0/117879065 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b40036d10 con 0x5594591b76b0 2023-07-21T18:54:36.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.292+0000 7f6b627acb80 10 client.17139 fetch_fsmap learned FSMap version 71 2023-07-21T18:54:36.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.292+0000 7f6b627acb80 1 -- 172.21.15.39:0/117879065 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) v3 -- 0x5594590d9250 con 0x5594591b76b0 2023-07-21T18:54:36.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.292+0000 7f6b2f7fe700 1 -- 172.21.15.39:0/117879065 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b48099f10 con 0x5594591b76b0 2023-07-21T18:54:36.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.292+0000 7f6b2f7fe700 1 -- 172.21.15.39:0/117879065 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 71) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6b40004500 con 0x5594591b76b0 2023-07-21T18:54:36.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.292+0000 7f6b627acb80 10 client.17139 fetch_fsmap finished waiting for FSMap version 71 2023-07-21T18:54:36.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.292+0000 7f6b627acb80 10 client.17139 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:54:36.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.292+0000 7f6b627acb80 20 client.17139 populate_metadata read hostname 'smithi039' 2023-07-21T18:54:36.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.292+0000 7f6b627acb80 1 --2- 172.21.15.39:0/117879065 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5594593c99b0 0x5594593cbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:36.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.292+0000 7f6b627acb80 4 client.17139 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:54:36.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.292+0000 7f6b627acb80 1 -- 172.21.15.39:0/117879065 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559459267ff0 con 0x5594593c99b0 2023-07-21T18:54:36.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.293+0000 7f6b4f7fe700 1 --2- 172.21.15.39:0/117879065 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5594593c99b0 0x5594593cbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:36.312 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.293+0000 7f6b4f7fe700 1 --2- 172.21.15.39:0/117879065 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5594593c99b0 0x5594593cbdc0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ecbb54244a3de60 server_cookie=114e05476d417092 in_seq=0 out_seq=0 2023-07-21T18:54:36.312 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.293+0000 7f6b2f7fe700 10 client.17139 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:54:36.342 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.331+0000 7f6b2f7fe700 1 -- 172.21.15.39:0/117879065 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559459267ff0 con 0x5594593c99b0 2023-07-21T18:54:36.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.331+0000 7f6b2f7fe700 10 client.17139 handle_command_reply: tid=0 2023-07-21T18:54:36.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.331+0000 7f6b627acb80 1 client.17139 shutdown 2023-07-21T18:54:36.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.331+0000 7f6b627acb80 1 -- 172.21.15.39:0/117879065 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5594591b76b0 msgr2=0x5594593bcb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:54:36.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.331+0000 7f6b627acb80 1 --2- 172.21.15.39:0/117879065 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5594591b76b0 0x5594593bcb60 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f6b40005f50 tx=0x7f6b40009cd0 comp rx=0 tx=0).stop 2023-07-21T18:54:36.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.331+0000 7f6b627acb80 1 -- 172.21.15.39:0/117879065 shutdown_connections 2023-07-21T18:54:36.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.331+0000 7f6b627acb80 1 --2- 172.21.15.39:0/117879065 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5594593c99b0 0x5594593cbdc0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:36.344 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.331+0000 7f6b627acb80 1 --2- 172.21.15.39:0/117879065 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594593612d0 0x5594593c19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:36.344 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.331+0000 7f6b627acb80 1 --2- 172.21.15.39:0/117879065 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5594591b76b0 0x5594593bcb60 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:36.344 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.331+0000 7f6b2f7fe700 0 client.17139 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:54:36.344 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.331+0000 7f6b627acb80 1 --2- 172.21.15.39:0/117879065 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55945935b580 0x5594593bd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:36.344 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.331+0000 7f6b627acb80 1 -- 172.21.15.39:0/117879065 >> 172.21.15.39:0/117879065 conn(0x559459350be0 msgr2=0x559459361a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:54:36.344 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.332+0000 7f6b627acb80 1 -- 172.21.15.39:0/117879065 shutdown_connections 2023-07-21T18:54:36.344 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.332+0000 7f6b627acb80 1 -- 172.21.15.39:0/117879065 wait complete. 2023-07-21T18:54:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.332+0000 7f6b627acb80 20 client.17139 trim_cache size 0 max 16384 2023-07-21T18:54:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:54:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:54:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:54:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:54:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.332+0000 7f6b56626700 1 -- 172.21.15.39:0/2985421423 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f6b380a12c0 msgr2=0x7f6b380a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:54:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.332+0000 7f6b56626700 1 --2- 172.21.15.39:0/2985421423 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f6b380a12c0 0x7f6b380a3770 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6b40000c00 tx=0x7f6b40003680 comp rx=0 tx=0).stop 2023-07-21T18:54:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.332+0000 7f6b56626700 1 -- 172.21.15.39:0/2985421423 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b50103ed0 msgr2=0x7f6b5019cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:54:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.332+0000 7f6b56626700 1 --2- 172.21.15.39:0/2985421423 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b50103ed0 0x7f6b5019cbb0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f6b48007d70 tx=0x7f6b4800a340 comp rx=0 tx=0).stop 2023-07-21T18:54:36.347 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.332+0000 7f6b56626700 1 -- 172.21.15.39:0/2985421423 shutdown_connections 2023-07-21T18:54:36.347 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.332+0000 7f6b56626700 1 --2- 172.21.15.39:0/2985421423 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b50103ed0 0x7f6b5019cbb0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:36.347 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.332+0000 7f6b56626700 1 --2- 172.21.15.39:0/2985421423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b500ff3b0 0x7f6b50198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:36.347 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.332+0000 7f6b56626700 1 --2- 172.21.15.39:0/2985421423 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f6b380a12c0 0x7f6b380a3770 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:36.347 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.332+0000 7f6b56626700 1 --2- 172.21.15.39:0/2985421423 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b500ffd50 0x7f6b50198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:36.347 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.332+0000 7f6b56626700 1 -- 172.21.15.39:0/2985421423 >> 172.21.15.39:0/2985421423 conn(0x7f6b50075f90 msgr2=0x7f6b500fc270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:54:36.347 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.332+0000 7f6b56626700 1 -- 172.21.15.39:0/2985421423 shutdown_connections 2023-07-21T18:54:36.348 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:36.332+0000 7f6b56626700 1 -- 172.21.15.39:0/2985421423 wait complete. 2023-07-21T18:54:36.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:54:36 smithi193 ceph-mon[98204]: pgmap v403: 97 pgs: 97 active+clean; 274 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 1.2 MiB/s wr, 118 op/s 2023-07-21T18:54:36.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:54:36 smithi155 ceph-mon[97934]: pgmap v403: 97 pgs: 97 active+clean; 274 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 1.2 MiB/s wr, 118 op/s 2023-07-21T18:54:36.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:54:36 smithi039 ceph-mon[96489]: pgmap v403: 97 pgs: 97 active+clean; 274 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 1.2 MiB/s wr, 118 op/s 2023-07-21T18:54:37.296 INFO:tasks.workunit.client.0.smithi039.stdout: 1 716666 5.18 MB/sec execute 590 sec latency 2663.541 ms 2023-07-21T18:54:38.296 INFO:tasks.workunit.client.0.smithi039.stdout: 1 717906 5.18 MB/sec execute 591 sec latency 14.157 ms 2023-07-21T18:54:38.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:54:38 smithi193 ceph-mon[98204]: pgmap v404: 97 pgs: 97 active+clean; 277 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 919 KiB/s rd, 1.9 MiB/s wr, 174 op/s 2023-07-21T18:54:38.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:54:38 smithi155 ceph-mon[97934]: pgmap v404: 97 pgs: 97 active+clean; 277 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 919 KiB/s rd, 1.9 MiB/s wr, 174 op/s 2023-07-21T18:54:38.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:54:38 smithi039 ceph-mon[96489]: pgmap v404: 97 pgs: 97 active+clean; 277 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 919 KiB/s rd, 1.9 MiB/s wr, 174 op/s 2023-07-21T18:54:39.296 INFO:tasks.workunit.client.0.smithi039.stdout: 1 719005 5.18 MB/sec execute 592 sec latency 170.180 ms 2023-07-21T18:54:40.296 INFO:tasks.workunit.client.0.smithi039.stdout: 1 719005 5.17 MB/sec execute 593 sec latency 1170.266 ms 2023-07-21T18:54:40.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:54:40 smithi193 ceph-mon[98204]: pgmap v405: 97 pgs: 97 active+clean; 277 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 1.3 MiB/s wr, 110 op/s 2023-07-21T18:54:40.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:54:40 smithi155 ceph-mon[97934]: pgmap v405: 97 pgs: 97 active+clean; 277 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 1.3 MiB/s wr, 110 op/s 2023-07-21T18:54:40.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:54:40 smithi039 ceph-mon[96489]: pgmap v405: 97 pgs: 97 active+clean; 277 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 1.3 MiB/s wr, 110 op/s 2023-07-21T18:54:41.296 INFO:tasks.workunit.client.0.smithi039.stdout: 1 719005 5.16 MB/sec execute 594 sec latency 2170.404 ms 2023-07-21T18:54:42.296 INFO:tasks.workunit.client.0.smithi039.stdout: 1 720228 5.17 MB/sec execute 595 sec latency 2596.801 ms 2023-07-21T18:54:42.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:54:42 smithi193 ceph-mon[98204]: pgmap v406: 97 pgs: 97 active+clean; 276 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 1.9 MiB/s wr, 178 op/s 2023-07-21T18:54:42.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:54:42 smithi155 ceph-mon[97934]: pgmap v406: 97 pgs: 97 active+clean; 276 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 1.9 MiB/s wr, 178 op/s 2023-07-21T18:54:42.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:54:42 smithi039 ceph-mon[96489]: pgmap v406: 97 pgs: 97 active+clean; 276 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 1.9 MiB/s wr, 178 op/s 2023-07-21T18:54:43.296 INFO:tasks.workunit.client.0.smithi039.stdout: 1 721476 5.17 MB/sec execute 596 sec latency 22.017 ms 2023-07-21T18:54:44.296 INFO:tasks.workunit.client.0.smithi039.stdout: 1 722548 5.17 MB/sec execute 597 sec latency 322.642 ms 2023-07-21T18:54:44.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:54:44 smithi193 ceph-mon[98204]: pgmap v407: 97 pgs: 97 active+clean; 276 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 1.3 MiB/s wr, 123 op/s 2023-07-21T18:54:44.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:54:44 smithi155 ceph-mon[97934]: pgmap v407: 97 pgs: 97 active+clean; 276 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 1.3 MiB/s wr, 123 op/s 2023-07-21T18:54:44.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:54:44 smithi039 ceph-mon[96489]: pgmap v407: 97 pgs: 97 active+clean; 276 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 1.3 MiB/s wr, 123 op/s 2023-07-21T18:54:45.296 INFO:tasks.workunit.client.0.smithi039.stdout: 1 722548 5.16 MB/sec execute 598 sec latency 1322.706 ms 2023-07-21T18:54:46.296 INFO:tasks.workunit.client.0.smithi039.stdout: 1 723177 5.15 MB/sec execute 599 sec latency 2113.012 ms 2023-07-21T18:54:46.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:54:46 smithi193 ceph-mon[98204]: pgmap v408: 97 pgs: 97 active+clean; 276 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 1.3 MiB/s wr, 123 op/s 2023-07-21T18:54:46.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:54:46 smithi155 ceph-mon[97934]: pgmap v408: 97 pgs: 97 active+clean; 276 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 1.3 MiB/s wr, 123 op/s 2023-07-21T18:54:46.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:54:46 smithi039 ceph-mon[96489]: pgmap v408: 97 pgs: 97 active+clean; 276 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 1.3 MiB/s wr, 123 op/s 2023-07-21T18:54:47.296 INFO:tasks.workunit.client.0.smithi039.stdout: 1 cleanup 600 sec 2023-07-21T18:54:47.547 INFO:tasks.workunit.client.0.smithi039.stdout: 0 cleanup 600 sec 2023-07-21T18:54:47.547 INFO:tasks.workunit.client.0.smithi039.stdout: 2023-07-21T18:54:47.548 INFO:tasks.workunit.client.0.smithi039.stdout: Operation Count AvgLat MaxLat 2023-07-21T18:54:47.548 INFO:tasks.workunit.client.0.smithi039.stdout: ---------------------------------------- 2023-07-21T18:54:47.548 INFO:tasks.workunit.client.0.smithi039.stdout: NTCreateX 98779 1.838 9270.332 2023-07-21T18:54:47.548 INFO:tasks.workunit.client.0.smithi039.stdout: Close 72585 0.002 0.100 2023-07-21T18:54:47.548 INFO:tasks.workunit.client.0.smithi039.stdout: Rename 4187 43.790 3452.162 2023-07-21T18:54:47.548 INFO:tasks.workunit.client.0.smithi039.stdout: Unlink 19940 2.571 7643.461 2023-07-21T18:54:47.549 INFO:tasks.workunit.client.0.smithi039.stdout: Deltree 2 129.009 181.028 2023-07-21T18:54:47.549 INFO:tasks.workunit.client.0.smithi039.stdout: Mkdir 1 0.004 0.004 2023-07-21T18:54:47.549 INFO:tasks.workunit.client.0.smithi039.stdout: Qpathinfo 89566 1.641 4976.622 2023-07-21T18:54:47.549 INFO:tasks.workunit.client.0.smithi039.stdout: Qfileinfo 15688 0.001 0.027 2023-07-21T18:54:47.549 INFO:tasks.workunit.client.0.smithi039.stdout: Qfsinfo 16426 0.131 11.678 2023-07-21T18:54:47.549 INFO:tasks.workunit.client.0.smithi039.stdout: Sfileinfo 8054 0.036 4.080 2023-07-21T18:54:47.549 INFO:tasks.workunit.client.0.smithi039.stdout: Find 34631 0.224 727.174 2023-07-21T18:54:47.549 INFO:tasks.workunit.client.0.smithi039.stdout: WriteX 49195 0.072 30.466 2023-07-21T18:54:47.550 INFO:tasks.workunit.client.0.smithi039.stdout: ReadX 154992 0.010 7.311 2023-07-21T18:54:47.550 INFO:tasks.workunit.client.0.smithi039.stdout: LockX 322 0.754 3.883 2023-07-21T18:54:47.550 INFO:tasks.workunit.client.0.smithi039.stdout: UnlockX 322 0.695 2.475 2023-07-21T18:54:47.550 INFO:tasks.workunit.client.0.smithi039.stdout: Flush 6929 2.812 41.481 2023-07-21T18:54:47.550 INFO:tasks.workunit.client.0.smithi039.stdout: 2023-07-21T18:54:47.550 INFO:tasks.workunit.client.0.smithi039.stdout:Throughput 5.15443 MB/sec 1 clients 1 procs max_latency=9270.338 ms 2023-07-21T18:54:47.551 INFO:tasks.workunit.client.0.smithi039.stderr:+ dbench 10 2023-07-21T18:54:47.551 INFO:tasks.workunit.client.0.smithi039.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2023-07-21T18:54:47.551 INFO:tasks.workunit.client.0.smithi039.stdout: 2023-07-21T18:54:47.551 INFO:tasks.workunit.client.0.smithi039.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2023-07-21T18:54:47.551 INFO:tasks.workunit.client.0.smithi039.stdout:8 of 10 processes prepared for launch 0 sec 2023-07-21T18:54:47.649 INFO:tasks.workunit.client.0.smithi039.stdout:10 of 10 processes prepared for launch 0 sec 2023-07-21T18:54:47.649 INFO:tasks.workunit.client.0.smithi039.stdout:releasing clients 2023-07-21T18:54:48.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:54:48 smithi193 ceph-mon[98204]: pgmap v409: 97 pgs: 97 active+clean; 280 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 937 KiB/s rd, 1.9 MiB/s wr, 178 op/s 2023-07-21T18:54:48.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:54:48 smithi155 ceph-mon[97934]: pgmap v409: 97 pgs: 97 active+clean; 280 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 937 KiB/s rd, 1.9 MiB/s wr, 178 op/s 2023-07-21T18:54:48.649 INFO:tasks.workunit.client.0.smithi039.stdout: 10 308 113.29 MB/sec warmup 1 sec latency 46.737 ms 2023-07-21T18:54:48.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:54:48 smithi039 ceph-mon[96489]: pgmap v409: 97 pgs: 97 active+clean; 280 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 937 KiB/s rd, 1.9 MiB/s wr, 178 op/s 2023-07-21T18:54:49.472 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.460+0000 7fea06a5b700 1 -- 172.21.15.39:0/782471012 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea00100500 msgr2=0x7fea0010c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:54:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.460+0000 7fea06a5b700 1 --2- 172.21.15.39:0/782471012 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea00100500 0x7fea0010c940 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fe9f800b0a0 tx=0x7fe9f800b3b0 comp rx=0 tx=0).stop 2023-07-21T18:54:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.460+0000 7fea06a5b700 1 -- 172.21.15.39:0/782471012 shutdown_connections 2023-07-21T18:54:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.460+0000 7fea06a5b700 1 --2- 172.21.15.39:0/782471012 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea00100500 0x7fea0010c940 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.460+0000 7fea06a5b700 1 --2- 172.21.15.39:0/782471012 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea00105b80 0x7fea00105f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.460+0000 7fea06a5b700 1 --2- 172.21.15.39:0/782471012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea000ffb50 0x7fea000fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.460+0000 7fea06a5b700 1 -- 172.21.15.39:0/782471012 >> 172.21.15.39:0/782471012 conn(0x7fea000fb5f0 msgr2=0x7fea000fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:54:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.460+0000 7fea06a5b700 1 -- 172.21.15.39:0/782471012 shutdown_connections 2023-07-21T18:54:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.460+0000 7fea06a5b700 1 -- 172.21.15.39:0/782471012 wait complete. 2023-07-21T18:54:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.461+0000 7fea06a5b700 1 Processor -- start 2023-07-21T18:54:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.461+0000 7fea06a5b700 1 -- start start 2023-07-21T18:54:49.476 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.462+0000 7fea06a5b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea000ffb50 0x7fea00195d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:49.476 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.462+0000 7fea06a5b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea00100500 0x7fea001962d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:49.476 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.462+0000 7fea06a5b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea00105b80 0x7fea0019a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:49.476 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.462+0000 7fea06a5b700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea0019af80 con 0x7fea000ffb50 2023-07-21T18:54:49.476 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.462+0000 7fea06a5b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea0019b0f0 con 0x7fea00105b80 2023-07-21T18:54:49.476 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.462+0000 7fea06a5b700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea0019b2b0 con 0x7fea00100500 2023-07-21T18:54:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.462+0000 7fe9fffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea000ffb50 0x7fea00195d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.462+0000 7fe9fffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea000ffb50 0x7fea00195d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33010/0 (socket says 172.21.15.39:33010) 2023-07-21T18:54:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.462+0000 7fe9fffff700 1 -- 172.21.15.39:0/805152597 learned_addr learned my addr 172.21.15.39:0/805152597 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:54:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.462+0000 7fe9f7fff700 1 --2- 172.21.15.39:0/805152597 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea00100500 0x7fea001962d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.462+0000 7fea04ff8700 1 --2- 172.21.15.39:0/805152597 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea00105b80 0x7fea0019a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.462+0000 7fe9f7fff700 1 -- 172.21.15.39:0/805152597 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea00105b80 msgr2=0x7fea0019a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:54:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.462+0000 7fe9f7fff700 1 --2- 172.21.15.39:0/805152597 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea00105b80 0x7fea0019a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.462+0000 7fe9f7fff700 1 -- 172.21.15.39:0/805152597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea000ffb50 msgr2=0x7fea00195d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:54:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.462+0000 7fe9f7fff700 1 --2- 172.21.15.39:0/805152597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea000ffb50 0x7fea00195d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.462+0000 7fe9f7fff700 1 -- 172.21.15.39:0/805152597 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9e8009520 con 0x7fea00100500 2023-07-21T18:54:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.462+0000 7fe9fffff700 1 --2- 172.21.15.39:0/805152597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea000ffb50 0x7fea00195d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:54:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.462+0000 7fe9f7fff700 1 --2- 172.21.15.39:0/805152597 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea00100500 0x7fea001962d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe9e800e9f0 tx=0x7fe9e800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:54:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.463+0000 7fe9fdffb700 1 -- 172.21.15.39:0/805152597 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e800cb70 con 0x7fea00100500 2023-07-21T18:54:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.463+0000 7fea06a5b700 1 -- 172.21.15.39:0/805152597 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9f8009d00 con 0x7fea00100500 2023-07-21T18:54:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.463+0000 7fe9fdffb700 1 -- 172.21.15.39:0/805152597 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9e800ee90 con 0x7fea00100500 2023-07-21T18:54:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.463+0000 7fe9fdffb700 1 -- 172.21.15.39:0/805152597 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e8003cf0 con 0x7fea00100500 2023-07-21T18:54:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.463+0000 7fea06a5b700 1 -- 172.21.15.39:0/805152597 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea001a2bf0 con 0x7fea00100500 2023-07-21T18:54:49.480 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.465+0000 7fe9fdffb700 1 -- 172.21.15.39:0/805152597 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fe9e800ccd0 con 0x7fea00100500 2023-07-21T18:54:49.481 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.466+0000 7fe9fdffb700 1 --2- 172.21.15.39:0/805152597 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fe9ec0a12c0 0x7fe9ec0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:49.481 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.466+0000 7fe9fdffb700 1 -- 172.21.15.39:0/805152597 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fe9e8014070 con 0x7fea00100500 2023-07-21T18:54:49.481 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.466+0000 7fe9fffff700 1 --2- 172.21.15.39:0/805152597 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fe9ec0a12c0 0x7fe9ec0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:49.481 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.467+0000 7fe9fffff700 1 --2- 172.21.15.39:0/805152597 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fe9ec0a12c0 0x7fe9ec0a3770 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0005950 tx=0x7fe9f000a400 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:54:49.481 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.467+0000 7fea12be1b80 1 Processor -- start 2023-07-21T18:54:49.481 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.467+0000 7fea12be1b80 1 -- start start 2023-07-21T18:54:49.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.467+0000 7fea12be1b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e5f87b6b0 0x558e5fa24d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:49.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.467+0000 7fea12be1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e5fa252d0 0x558e5fa1f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:49.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.467+0000 7fea12be1b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e5fa1f580 0x558e5fa1f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:49.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.467+0000 7fea12be1b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e5f924c50 con 0x558e5f87b6b0 2023-07-21T18:54:49.485 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.467+0000 7fea12be1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e5f9779e0 con 0x558e5fa252d0 2023-07-21T18:54:49.485 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.467+0000 7fea12be1b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e5f970e70 con 0x558e5fa1f580 2023-07-21T18:54:49.485 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.468+0000 7fe9fffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e5f87b6b0 0x558e5fa24d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:49.485 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.468+0000 7fe9f7fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e5fa252d0 0x558e5fa1f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:49.485 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.468+0000 7fe9f7fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e5fa252d0 0x558e5fa1f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:45616/0 (socket says 172.21.15.39:45616) 2023-07-21T18:54:49.485 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.468+0000 7fe9f7fff700 1 -- 172.21.15.39:0/3894670036 learned_addr learned my addr 172.21.15.39:0/3894670036 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:54:49.485 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.468+0000 7fea04ff8700 1 --2- 172.21.15.39:0/3894670036 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e5fa1f580 0x558e5fa1f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:49.486 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.468+0000 7fe9f7fff700 1 -- 172.21.15.39:0/3894670036 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e5fa1f580 msgr2=0x558e5fa1f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:54:49.486 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.468+0000 7fe9f7fff700 1 --2- 172.21.15.39:0/3894670036 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e5fa1f580 0x558e5fa1f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:49.486 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.468+0000 7fe9f7fff700 1 -- 172.21.15.39:0/3894670036 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e5f87b6b0 msgr2=0x558e5fa24d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:54:49.486 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.468+0000 7fe9f7fff700 1 --2- 172.21.15.39:0/3894670036 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e5f87b6b0 0x558e5fa24d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:49.486 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.468+0000 7fe9f7fff700 1 -- 172.21.15.39:0/3894670036 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e5f79ed80 con 0x558e5fa252d0 2023-07-21T18:54:49.486 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.468+0000 7fe9f7fff700 1 --2- 172.21.15.39:0/3894670036 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e5fa252d0 0x558e5fa1f040 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe9e80a1fc0 tx=0x7fe9e8099a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20eba6f61a8eca2d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:54:49.486 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.468+0000 7fe9f67fc700 1 -- 172.21.15.39:0/3894670036 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e809e0d0 con 0x558e5fa252d0 2023-07-21T18:54:49.486 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.468+0000 7fe9f67fc700 1 -- 172.21.15.39:0/3894670036 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9e80a24f0 con 0x558e5fa252d0 2023-07-21T18:54:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.468+0000 7fe9f67fc700 1 -- 172.21.15.39:0/3894670036 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e80a2060 con 0x558e5fa252d0 2023-07-21T18:54:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.469+0000 7fea12be1b80 1 -- 172.21.15.39:0/3894670036 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e5fa252d0 msgr2=0x558e5fa1f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:54:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.469+0000 7fea12be1b80 1 --2- 172.21.15.39:0/3894670036 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e5fa252d0 0x558e5fa1f040 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe9e80a1fc0 tx=0x7fe9e8099a80 comp rx=0 tx=0).stop 2023-07-21T18:54:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.469+0000 7fea12be1b80 1 -- 172.21.15.39:0/3894670036 shutdown_connections 2023-07-21T18:54:49.490 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.469+0000 7fea12be1b80 1 --2- 172.21.15.39:0/3894670036 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e5fa1f580 0x558e5fa1f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:49.490 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.469+0000 7fea12be1b80 1 --2- 172.21.15.39:0/3894670036 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e5f87b6b0 0x558e5fa24d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:49.490 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.469+0000 7fea12be1b80 1 --2- 172.21.15.39:0/3894670036 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e5fa252d0 0x558e5fa1f040 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:49.490 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.469+0000 7fea12be1b80 1 -- 172.21.15.39:0/3894670036 >> 172.21.15.39:0/3894670036 conn(0x558e5fa14be0 msgr2=0x558e5fa19f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:54:49.490 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.469+0000 7fea12be1b80 1 -- 172.21.15.39:0/3894670036 shutdown_connections 2023-07-21T18:54:49.490 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.469+0000 7fea12be1b80 1 -- 172.21.15.39:0/3894670036 wait complete. 2023-07-21T18:54:49.490 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.470+0000 7fea12be1b80 1 -- start start 2023-07-21T18:54:49.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.470+0000 7fea12be1b80 1 Processor -- start 2023-07-21T18:54:49.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.470+0000 7fea12be1b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e5f87b6b0 0x558e5fa2b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:49.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.470+0000 7fea12be1b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e5fa1f580 0x558e5fa288a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:49.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.470+0000 7fea12be1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e5fa252d0 0x558e5fa28f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:49.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.470+0000 7fea12be1b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e5f910a60 con 0x558e5fa1f580 2023-07-21T18:54:49.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.470+0000 7fea12be1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e5f910760 con 0x558e5fa252d0 2023-07-21T18:54:49.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.470+0000 7fea12be1b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e5f7f48c0 con 0x558e5f87b6b0 2023-07-21T18:54:49.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.471+0000 7fea12be1b80 10 client.0 _ll_register_callbacks cb 0x558e5f90e700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:54:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.471+0000 7fea04ff8700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e5fa252d0 0x558e5fa28f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.471+0000 7fe9f7fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e5fa1f580 0x558e5fa288a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.471+0000 7fe9f7fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e5fa1f580 0x558e5fa288a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33024/0 (socket says 172.21.15.39:33024) 2023-07-21T18:54:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.471+0000 7fe9f7fff700 1 -- 172.21.15.39:0/2851552904 learned_addr learned my addr 172.21.15.39:0/2851552904 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:54:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.471+0000 7fea04ff8700 1 --2- 172.21.15.39:0/2851552904 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e5fa252d0 0x558e5fa28f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:45622/0 (socket says 172.21.15.39:45622) 2023-07-21T18:54:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.471+0000 7fe9fffff700 1 --2- 172.21.15.39:0/2851552904 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e5f87b6b0 0x558e5fa2b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.471+0000 7fea04ff8700 1 -- 172.21.15.39:0/2851552904 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e5f87b6b0 msgr2=0x558e5fa2b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:54:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.471+0000 7fea04ff8700 1 --2- 172.21.15.39:0/2851552904 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e5f87b6b0 0x558e5fa2b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.471+0000 7fea04ff8700 1 -- 172.21.15.39:0/2851552904 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e5fa1f580 msgr2=0x558e5fa288a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:54:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.471+0000 7fea04ff8700 1 --2- 172.21.15.39:0/2851552904 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e5fa1f580 0x558e5fa288a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.471+0000 7fea04ff8700 1 -- 172.21.15.39:0/2851552904 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea001a2bf0 con 0x558e5fa252d0 2023-07-21T18:54:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.471+0000 7fe9fffff700 1 --2- 172.21.15.39:0/2851552904 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e5f87b6b0 0x558e5fa2b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:54:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.471+0000 7fea04ff8700 1 --2- 172.21.15.39:0/2851552904 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e5fa252d0 0x558e5fa28f90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe9f80062a0 tx=0x7fe9f8007710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71ce5f4e006e3387 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:54:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.471+0000 7fe9df7fe700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-21T18:54:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.472+0000 7fe9df7fe700 1 -- 172.21.15.39:0/2851552904 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f800e040 con 0x558e5fa252d0 2023-07-21T18:54:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.472+0000 7fe9df7fe700 1 -- 172.21.15.39:0/2851552904 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fe9f8003e10 con 0x558e5fa252d0 2023-07-21T18:54:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.472+0000 7fea12be1b80 1 -- 172.21.15.39:0/2851552904 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558e5f8ca490 con 0x558e5fa252d0 2023-07-21T18:54:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.472+0000 7fe9df7fe700 1 -- 172.21.15.39:0/2851552904 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9f8007930 con 0x558e5fa252d0 2023-07-21T18:54:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.472+0000 7fea12be1b80 10 client.26603 fetch_fsmap learned FSMap version 72 2023-07-21T18:54:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.472+0000 7fea12be1b80 1 -- 172.21.15.39:0/2851552904 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x558e5f787c00 con 0x558e5fa252d0 2023-07-21T18:54:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.472+0000 7fe9df7fe700 1 -- 172.21.15.39:0/2851552904 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558e5f8ca490 con 0x558e5fa252d0 2023-07-21T18:54:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.472+0000 7fe9df7fe700 1 -- 172.21.15.39:0/2851552904 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 72) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fe9f800b0d0 con 0x558e5fa252d0 2023-07-21T18:54:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.472+0000 7fea12be1b80 10 client.26603 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T18:54:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.472+0000 7fea12be1b80 10 client.26603 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:54:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.472+0000 7fea12be1b80 20 client.26603 populate_metadata read hostname 'smithi039' 2023-07-21T18:54:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.472+0000 7fea12be1b80 1 --2- 172.21.15.39:0/2851552904 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x558e5fa38c20 0x558e5fa3b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.472+0000 7fea12be1b80 4 client.26603 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:54:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.472+0000 7fea12be1b80 1 -- 172.21.15.39:0/2851552904 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558e5f969e50 con 0x558e5fa38c20 2023-07-21T18:54:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.472+0000 7fe9fffff700 1 --2- 172.21.15.39:0/2851552904 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x558e5fa38c20 0x558e5fa3b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.473+0000 7fe9fffff700 1 --2- 172.21.15.39:0/2851552904 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x558e5fa38c20 0x558e5fa3b030 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2dd91074d42b5326 server_cookie=aa3a32e27cdf58c6 in_seq=0 out_seq=0 2023-07-21T18:54:49.496 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.473+0000 7fe9df7fe700 10 client.26603 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:54:49.496 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.475+0000 7fe9df7fe700 1 -- 172.21.15.39:0/2851552904 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558e5f969e50 con 0x558e5fa38c20 2023-07-21T18:54:49.496 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.475+0000 7fe9df7fe700 10 client.26603 handle_command_reply: tid=0 2023-07-21T18:54:49.496 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.475+0000 7fea12be1b80 1 client.26603 shutdown 2023-07-21T18:54:49.496 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.475+0000 7fea12be1b80 1 -- 172.21.15.39:0/2851552904 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e5fa252d0 msgr2=0x558e5fa28f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:54:49.496 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.475+0000 7fea12be1b80 1 --2- 172.21.15.39:0/2851552904 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e5fa252d0 0x558e5fa28f90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe9f80062a0 tx=0x7fe9f8007710 comp rx=0 tx=0).stop 2023-07-21T18:54:49.496 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.475+0000 7fea12be1b80 1 -- 172.21.15.39:0/2851552904 shutdown_connections 2023-07-21T18:54:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.475+0000 7fea12be1b80 1 --2- 172.21.15.39:0/2851552904 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x558e5fa38c20 0x558e5fa3b030 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.475+0000 7fea12be1b80 1 --2- 172.21.15.39:0/2851552904 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e5fa252d0 0x558e5fa28f90 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.475+0000 7fea12be1b80 1 --2- 172.21.15.39:0/2851552904 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e5f87b6b0 0x558e5fa2b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.475+0000 7fe9df7fe700 0 client.26603 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:54:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.475+0000 7fea12be1b80 1 --2- 172.21.15.39:0/2851552904 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e5fa1f580 0x558e5fa288a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.475+0000 7fea12be1b80 1 -- 172.21.15.39:0/2851552904 >> 172.21.15.39:0/2851552904 conn(0x558e5fa14be0 msgr2=0x558e5fa194e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:54:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.475+0000 7fea12be1b80 1 -- 172.21.15.39:0/2851552904 shutdown_connections 2023-07-21T18:54:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.475+0000 7fea12be1b80 1 -- 172.21.15.39:0/2851552904 wait complete. 2023-07-21T18:54:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.476+0000 7fea12be1b80 20 client.26603 trim_cache size 0 max 16384 2023-07-21T18:54:49.507 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.495+0000 7fea12be1b80 1 Processor -- start 2023-07-21T18:54:49.507 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.495+0000 7fea12be1b80 1 -- start start 2023-07-21T18:54:49.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.495+0000 7fea12be1b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e5f87b6b0 0x558e5fa1f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:49.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.495+0000 7fea12be1b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e5fa1f580 0x558e5fa2fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:49.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.495+0000 7fea12be1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e5fa252d0 0x558e5fa24160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:49.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.495+0000 7fea12be1b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e5fa16d50 con 0x558e5f87b6b0 2023-07-21T18:54:49.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.495+0000 7fea12be1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e5fa29cc0 con 0x558e5fa252d0 2023-07-21T18:54:49.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.495+0000 7fea12be1b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e5fa26e30 con 0x558e5fa1f580 2023-07-21T18:54:49.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.495+0000 7fe9fffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e5f87b6b0 0x558e5fa1f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:49.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.495+0000 7fe9f7fff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e5fa1f580 0x558e5fa2fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:49.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.495+0000 7fe9fffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e5f87b6b0 0x558e5fa1f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33032/0 (socket says 172.21.15.39:33032) 2023-07-21T18:54:49.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.495+0000 7fe9fffff700 1 -- 172.21.15.39:0/1012418529 learned_addr learned my addr 172.21.15.39:0/1012418529 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:54:49.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.496+0000 7fea04ff8700 1 --2- 172.21.15.39:0/1012418529 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e5fa252d0 0x558e5fa24160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.496+0000 7fe9fffff700 1 -- 172.21.15.39:0/1012418529 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e5fa1f580 msgr2=0x558e5fa2fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:54:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.496+0000 7fe9fffff700 1 --2- 172.21.15.39:0/1012418529 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e5fa1f580 0x558e5fa2fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.496+0000 7fe9fffff700 1 -- 172.21.15.39:0/1012418529 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e5fa252d0 msgr2=0x558e5fa24160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:54:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.496+0000 7fe9fffff700 1 --2- 172.21.15.39:0/1012418529 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e5fa252d0 0x558e5fa24160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.496+0000 7fe9fffff700 1 -- 172.21.15.39:0/1012418529 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e5f7c51f0 con 0x558e5f87b6b0 2023-07-21T18:54:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.496+0000 7fe9fffff700 1 --2- 172.21.15.39:0/1012418529 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e5f87b6b0 0x558e5fa1f040 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x558e5fa3b940 tx=0x7fe9f0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dba05c088b7665ba server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:54:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.496+0000 7fe9f6ffd700 1 -- 172.21.15.39:0/1012418529 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f0022040 con 0x558e5f87b6b0 2023-07-21T18:54:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.496+0000 7fe9f6ffd700 1 -- 172.21.15.39:0/1012418529 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9f0014070 con 0x558e5f87b6b0 2023-07-21T18:54:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.496+0000 7fe9f6ffd700 1 -- 172.21.15.39:0/1012418529 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f0027e90 con 0x558e5f87b6b0 2023-07-21T18:54:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.497+0000 7fea12be1b80 1 -- 172.21.15.39:0/1012418529 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e5f87b6b0 msgr2=0x558e5fa1f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:54:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.497+0000 7fea12be1b80 1 --2- 172.21.15.39:0/1012418529 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e5f87b6b0 0x558e5fa1f040 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x558e5fa3b940 tx=0x7fe9f0012000 comp rx=0 tx=0).stop 2023-07-21T18:54:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.497+0000 7fea12be1b80 1 -- 172.21.15.39:0/1012418529 shutdown_connections 2023-07-21T18:54:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.497+0000 7fea12be1b80 1 --2- 172.21.15.39:0/1012418529 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e5fa252d0 0x558e5fa24160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.497+0000 7fea12be1b80 1 --2- 172.21.15.39:0/1012418529 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e5f87b6b0 0x558e5fa1f040 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.497+0000 7fea12be1b80 1 --2- 172.21.15.39:0/1012418529 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e5fa1f580 0x558e5fa2fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.497+0000 7fea12be1b80 1 -- 172.21.15.39:0/1012418529 >> 172.21.15.39:0/1012418529 conn(0x558e5fa14be0 msgr2=0x558e5fa23a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:54:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.497+0000 7fea12be1b80 1 -- 172.21.15.39:0/1012418529 shutdown_connections 2023-07-21T18:54:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.497+0000 7fea12be1b80 1 -- 172.21.15.39:0/1012418529 wait complete. 2023-07-21T18:54:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.498+0000 7fea12be1b80 1 -- start start 2023-07-21T18:54:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.498+0000 7fea12be1b80 1 Processor -- start 2023-07-21T18:54:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.498+0000 7fea12be1b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e5f87b6b0 0x558e5fa2fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.498+0000 7fea12be1b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e5fa1f580 0x558e5fa2c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.498+0000 7fea12be1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e5fa252d0 0x558e5fa2c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.498+0000 7fea12be1b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e5fa2cef0 con 0x558e5f87b6b0 2023-07-21T18:54:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.498+0000 7fea12be1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e5fa2d060 con 0x558e5fa252d0 2023-07-21T18:54:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.498+0000 7fea12be1b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e5fa8cae0 con 0x558e5fa1f580 2023-07-21T18:54:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.498+0000 7fea12be1b80 10 client.0 _ll_register_callbacks cb 0x7fe9f800bdc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:54:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.498+0000 7fe9fffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e5f87b6b0 0x558e5fa2fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.498+0000 7fe9f7fff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e5fa1f580 0x558e5fa2c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.498+0000 7fe9fffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e5f87b6b0 0x558e5fa2fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33038/0 (socket says 172.21.15.39:33038) 2023-07-21T18:54:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.498+0000 7fe9f7fff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e5fa1f580 0x558e5fa2c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:51592/0 (socket says 172.21.15.39:51592) 2023-07-21T18:54:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.498+0000 7fe9fffff700 1 -- 172.21.15.39:0/478442350 learned_addr learned my addr 172.21.15.39:0/478442350 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:54:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.498+0000 7fea04ff8700 1 --2- 172.21.15.39:0/478442350 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e5fa252d0 0x558e5fa2c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.499+0000 7fe9f7fff700 1 -- 172.21.15.39:0/478442350 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e5fa252d0 msgr2=0x558e5fa2c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:54:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.499+0000 7fe9f7fff700 1 --2- 172.21.15.39:0/478442350 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e5fa252d0 0x558e5fa2c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.499+0000 7fe9f7fff700 1 -- 172.21.15.39:0/478442350 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e5f87b6b0 msgr2=0x558e5fa2fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:54:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.499+0000 7fe9f7fff700 1 --2- 172.21.15.39:0/478442350 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e5f87b6b0 0x558e5fa2fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.499+0000 7fe9f7fff700 1 -- 172.21.15.39:0/478442350 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9f001d040 con 0x558e5fa1f580 2023-07-21T18:54:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.499+0000 7fe9f7fff700 1 --2- 172.21.15.39:0/478442350 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e5fa1f580 0x558e5fa2c240 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe9e80983f0 tx=0x7fe9e8003780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=881dc9e416ee5f12 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:54:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.499+0000 7fe9df7fe700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2023-07-21T18:54:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.499+0000 7fe9df7fe700 1 -- 172.21.15.39:0/478442350 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e809e0d0 con 0x558e5fa1f580 2023-07-21T18:54:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.499+0000 7fe9df7fe700 1 -- 172.21.15.39:0/478442350 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fe9e8099590 con 0x558e5fa1f580 2023-07-21T18:54:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.499+0000 7fea12be1b80 1 -- 172.21.15.39:0/478442350 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe9e80a24f0 con 0x558e5fa1f580 2023-07-21T18:54:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.499+0000 7fe9df7fe700 1 -- 172.21.15.39:0/478442350 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9e8099750 con 0x558e5fa1f580 2023-07-21T18:54:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.499+0000 7fea12be1b80 10 client.26620 fetch_fsmap learned FSMap version 72 2023-07-21T18:54:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.499+0000 7fea12be1b80 1 -- 172.21.15.39:0/478442350 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x558e5f79d250 con 0x558e5fa1f580 2023-07-21T18:54:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.499+0000 7fe9df7fe700 1 -- 172.21.15.39:0/478442350 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9e80a24f0 con 0x558e5fa1f580 2023-07-21T18:54:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.500+0000 7fe9df7fe700 1 -- 172.21.15.39:0/478442350 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 72) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fe9e8099000 con 0x558e5fa1f580 2023-07-21T18:54:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.500+0000 7fea12be1b80 10 client.26620 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T18:54:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.500+0000 7fea12be1b80 10 client.26620 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:54:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.500+0000 7fea12be1b80 20 client.26620 populate_metadata read hostname 'smithi039' 2023-07-21T18:54:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.500+0000 7fea12be1b80 1 --2- 172.21.15.39:0/478442350 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x558e5fa8dde0 0x558e5fa901f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:54:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.500+0000 7fea12be1b80 4 client.26620 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:54:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.500+0000 7fea12be1b80 1 -- 172.21.15.39:0/478442350 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558e5f92bff0 con 0x558e5fa8dde0 2023-07-21T18:54:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.500+0000 7fe9fffff700 1 --2- 172.21.15.39:0/478442350 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x558e5fa8dde0 0x558e5fa901f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:54:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.500+0000 7fe9fffff700 1 --2- 172.21.15.39:0/478442350 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x558e5fa8dde0 0x558e5fa901f0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2840ac2cbaffbbc server_cookie=dec027e4b7f92c2c in_seq=0 out_seq=0 2023-07-21T18:54:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.500+0000 7fe9df7fe700 10 client.26620 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:54:49.611 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:54:49 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:54:49.633 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:54:49 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:54:49.652 INFO:tasks.workunit.client.0.smithi039.stdout: 10 552 94.75 MB/sec warmup 2 sec latency 173.101 ms 2023-07-21T18:54:49.678 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:54:49 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:54:49.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.759+0000 7fe9df7fe700 1 -- 172.21.15.39:0/478442350 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558e5f92bff0 con 0x558e5fa8dde0 2023-07-21T18:54:49.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.759+0000 7fe9df7fe700 10 client.26620 handle_command_reply: tid=0 2023-07-21T18:54:49.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.759+0000 7fea12be1b80 1 client.26620 shutdown 2023-07-21T18:54:49.772 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.759+0000 7fea12be1b80 1 -- 172.21.15.39:0/478442350 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e5fa1f580 msgr2=0x558e5fa2c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:54:49.772 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.759+0000 7fea12be1b80 1 --2- 172.21.15.39:0/478442350 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e5fa1f580 0x558e5fa2c240 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe9e80983f0 tx=0x7fe9e8003780 comp rx=0 tx=0).stop 2023-07-21T18:54:49.772 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.759+0000 7fea12be1b80 1 -- 172.21.15.39:0/478442350 shutdown_connections 2023-07-21T18:54:49.772 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.759+0000 7fea12be1b80 1 --2- 172.21.15.39:0/478442350 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x558e5fa8dde0 0x558e5fa901f0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:49.772 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.759+0000 7fea12be1b80 1 --2- 172.21.15.39:0/478442350 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e5fa252d0 0x558e5fa2c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:49.773 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.759+0000 7fea12be1b80 1 --2- 172.21.15.39:0/478442350 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e5f87b6b0 0x558e5fa2fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:49.773 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.759+0000 7fe9df7fe700 0 client.26620 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:54:49.773 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.759+0000 7fea12be1b80 1 --2- 172.21.15.39:0/478442350 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e5fa1f580 0x558e5fa2c240 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:49.773 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.759+0000 7fea12be1b80 1 -- 172.21.15.39:0/478442350 >> 172.21.15.39:0/478442350 conn(0x558e5fa14be0 msgr2=0x558e5fa2aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:54:49.773 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.759+0000 7fea12be1b80 1 -- 172.21.15.39:0/478442350 shutdown_connections 2023-07-21T18:54:49.773 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.759+0000 7fea12be1b80 1 -- 172.21.15.39:0/478442350 wait complete. 2023-07-21T18:54:49.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.760+0000 7fea12be1b80 20 client.26620 trim_cache size 0 max 16384 2023-07-21T18:54:49.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:54:49.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:54:49.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:54:49.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:54:49.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.760+0000 7fea06a5b700 1 -- 172.21.15.39:0/805152597 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fe9ec0a12c0 msgr2=0x7fe9ec0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:54:49.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.760+0000 7fea06a5b700 1 --2- 172.21.15.39:0/805152597 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fe9ec0a12c0 0x7fe9ec0a3770 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0005950 tx=0x7fe9f000a400 comp rx=0 tx=0).stop 2023-07-21T18:54:49.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.760+0000 7fea06a5b700 1 -- 172.21.15.39:0/805152597 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea00100500 msgr2=0x7fea001962d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:54:49.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.760+0000 7fea06a5b700 1 --2- 172.21.15.39:0/805152597 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea00100500 0x7fea001962d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe9e800e9f0 tx=0x7fe9e800ed00 comp rx=0 tx=0).stop 2023-07-21T18:54:49.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.760+0000 7fea06a5b700 1 -- 172.21.15.39:0/805152597 shutdown_connections 2023-07-21T18:54:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.760+0000 7fea06a5b700 1 --2- 172.21.15.39:0/805152597 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fe9ec0a12c0 0x7fe9ec0a3770 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.760+0000 7fea06a5b700 1 --2- 172.21.15.39:0/805152597 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea00105b80 0x7fea0019a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.760+0000 7fea06a5b700 1 --2- 172.21.15.39:0/805152597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea000ffb50 0x7fea00195d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.760+0000 7fea06a5b700 1 --2- 172.21.15.39:0/805152597 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea00100500 0x7fea001962d0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:54:49.778 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.760+0000 7fea06a5b700 1 -- 172.21.15.39:0/805152597 >> 172.21.15.39:0/805152597 conn(0x7fea000fb5f0 msgr2=0x7fea00103670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:54:49.778 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.760+0000 7fea06a5b700 1 -- 172.21.15.39:0/805152597 shutdown_connections 2023-07-21T18:54:49.778 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:54:49.760+0000 7fea06a5b700 1 -- 172.21.15.39:0/805152597 wait complete. 2023-07-21T18:54:50.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:54:50 smithi193 ceph-mon[98204]: pgmap v410: 97 pgs: 97 active+clean; 280 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 1.3 MiB/s wr, 122 op/s 2023-07-21T18:54:50.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:54:50 smithi155 ceph-mon[97934]: pgmap v410: 97 pgs: 97 active+clean; 280 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 1.3 MiB/s wr, 122 op/s 2023-07-21T18:54:50.652 INFO:tasks.workunit.client.0.smithi039.stdout: 10 899 71.09 MB/sec warmup 3 sec latency 279.397 ms 2023-07-21T18:54:50.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:54:50 smithi039 ceph-mon[96489]: pgmap v410: 97 pgs: 97 active+clean; 280 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 1.3 MiB/s wr, 122 op/s 2023-07-21T18:54:51.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:54:51 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:54:51.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:54:51 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:54:51.652 INFO:tasks.workunit.client.0.smithi039.stdout: 10 1406 63.84 MB/sec warmup 4 sec latency 814.813 ms 2023-07-21T18:54:51.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:54:51 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:54:52.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:54:52 smithi193 ceph-mon[98204]: pgmap v411: 97 pgs: 97 active+clean; 279 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 352 op/s 2023-07-21T18:54:52.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:54:52 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:54:52.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:54:52 smithi155 ceph-mon[97934]: pgmap v411: 97 pgs: 97 active+clean; 279 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 352 op/s 2023-07-21T18:54:52.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:54:52 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:54:52.652 INFO:tasks.workunit.client.0.smithi039.stdout: 10 2196 54.43 MB/sec warmup 5 sec latency 1814.857 ms 2023-07-21T18:54:52.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:54:52 smithi039 ceph-mon[96489]: pgmap v411: 97 pgs: 97 active+clean; 279 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 352 op/s 2023-07-21T18:54:52.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:54:52 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:54:53.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:54:53 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:54:53.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:54:53 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:54:53.652 INFO:tasks.workunit.client.0.smithi039.stdout: 10 2639 52.41 MB/sec warmup 6 sec latency 2814.895 ms 2023-07-21T18:54:53.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:54:53 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:54:54.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:54:54 smithi193 ceph-mon[98204]: pgmap v412: 97 pgs: 97 active+clean; 279 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 1.8 MiB/s wr, 284 op/s 2023-07-21T18:54:54.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:54:54 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:54:54.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:54:54 smithi155 ceph-mon[97934]: pgmap v412: 97 pgs: 97 active+clean; 279 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 1.8 MiB/s wr, 284 op/s 2023-07-21T18:54:54.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:54:54 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:54:54.653 INFO:tasks.workunit.client.0.smithi039.stdout: 10 2992 47.68 MB/sec warmup 7 sec latency 3814.943 ms 2023-07-21T18:54:54.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:54:54 smithi039 ceph-mon[96489]: pgmap v412: 97 pgs: 97 active+clean; 279 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 1.8 MiB/s wr, 284 op/s 2023-07-21T18:54:54.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:54:54 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:54:55.653 INFO:tasks.workunit.client.0.smithi039.stdout: 10 3116 42.87 MB/sec warmup 8 sec latency 4357.709 ms 2023-07-21T18:54:56.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:54:56 smithi193 ceph-mon[98204]: pgmap v413: 97 pgs: 97 active+clean; 279 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 1.8 MiB/s wr, 284 op/s 2023-07-21T18:54:56.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:54:56 smithi155 ceph-mon[97934]: pgmap v413: 97 pgs: 97 active+clean; 279 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 1.8 MiB/s wr, 284 op/s 2023-07-21T18:54:56.653 INFO:tasks.workunit.client.0.smithi039.stdout: 10 3416 41.34 MB/sec warmup 9 sec latency 131.472 ms 2023-07-21T18:54:56.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:54:56 smithi039 ceph-mon[96489]: pgmap v413: 97 pgs: 97 active+clean; 279 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 1.8 MiB/s wr, 284 op/s 2023-07-21T18:54:57.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:54:57 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:54:57.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:54:57 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:54:57.653 INFO:tasks.workunit.client.0.smithi039.stdout: 10 3598 38.41 MB/sec warmup 10 sec latency 105.022 ms 2023-07-21T18:54:57.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:54:57 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:54:58.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:54:58 smithi193 ceph-mon[98204]: pgmap v414: 97 pgs: 97 active+clean; 514 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 23 MiB/s wr, 464 op/s 2023-07-21T18:54:58.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:54:58 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:54:58.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:54:58 smithi155 ceph-mon[97934]: pgmap v414: 97 pgs: 97 active+clean; 514 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 23 MiB/s wr, 464 op/s 2023-07-21T18:54:58.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:54:58 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:54:58.653 INFO:tasks.workunit.client.0.smithi039.stdout: 10 3740 35.15 MB/sec warmup 11 sec latency 155.315 ms 2023-07-21T18:54:58.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:54:58 smithi039 ceph-mon[96489]: pgmap v414: 97 pgs: 97 active+clean; 514 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 23 MiB/s wr, 464 op/s 2023-07-21T18:54:58.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:54:58 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:54:59.653 INFO:tasks.workunit.client.0.smithi039.stdout: 10 3920 32.28 MB/sec warmup 12 sec latency 130.253 ms 2023-07-21T18:55:00.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:55:00 smithi193 ceph-mon[98204]: pgmap v415: 97 pgs: 97 active+clean; 514 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 22 MiB/s wr, 409 op/s 2023-07-21T18:55:00.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:55:00 smithi155 ceph-mon[97934]: pgmap v415: 97 pgs: 97 active+clean; 514 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 22 MiB/s wr, 409 op/s 2023-07-21T18:55:00.653 INFO:tasks.workunit.client.0.smithi039.stdout: 10 4192 30.33 MB/sec warmup 13 sec latency 160.038 ms 2023-07-21T18:55:00.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:55:00 smithi039 ceph-mon[96489]: pgmap v415: 97 pgs: 97 active+clean; 514 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 22 MiB/s wr, 409 op/s 2023-07-21T18:55:01.653 INFO:tasks.workunit.client.0.smithi039.stdout: 10 4440 28.98 MB/sec warmup 14 sec latency 125.983 ms 2023-07-21T18:55:02.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:55:02 smithi193 ceph-mon[98204]: pgmap v416: 97 pgs: 97 active+clean; 508 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 23 MiB/s wr, 527 op/s 2023-07-21T18:55:02.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:55:02 smithi155 ceph-mon[97934]: pgmap v416: 97 pgs: 97 active+clean; 508 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 23 MiB/s wr, 527 op/s 2023-07-21T18:55:02.653 INFO:tasks.workunit.client.0.smithi039.stdout: 10 4814 28.38 MB/sec warmup 15 sec latency 74.114 ms 2023-07-21T18:55:02.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:55:02 smithi039 ceph-mon[96489]: pgmap v416: 97 pgs: 97 active+clean; 508 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 23 MiB/s wr, 527 op/s 2023-07-21T18:55:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.877+0000 7ff46e5d9700 1 -- 172.21.15.39:0/2079275587 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4680ff400 msgr2=0x7ff46810b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:55:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.877+0000 7ff46e5d9700 1 --2- 172.21.15.39:0/2079275587 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4680ff400 0x7ff46810b890 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7ff46000b0a0 tx=0x7ff46000b3b0 comp rx=0 tx=0).stop 2023-07-21T18:55:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.877+0000 7ff46e5d9700 1 -- 172.21.15.39:0/2079275587 shutdown_connections 2023-07-21T18:55:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.877+0000 7ff46e5d9700 1 --2- 172.21.15.39:0/2079275587 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff468104a60 0x7ff468104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.877+0000 7ff46e5d9700 1 --2- 172.21.15.39:0/2079275587 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4680ff400 0x7ff46810b890 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.877+0000 7ff46e5d9700 1 --2- 172.21.15.39:0/2079275587 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4680fea50 0x7ff4680feec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.877+0000 7ff46e5d9700 1 -- 172.21.15.39:0/2079275587 >> 172.21.15.39:0/2079275587 conn(0x7ff4680fa570 msgr2=0x7ff4680fc980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:55:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.877+0000 7ff46e5d9700 1 -- 172.21.15.39:0/2079275587 shutdown_connections 2023-07-21T18:55:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.877+0000 7ff46e5d9700 1 -- 172.21.15.39:0/2079275587 wait complete. 2023-07-21T18:55:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.878+0000 7ff46e5d9700 1 Processor -- start 2023-07-21T18:55:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.878+0000 7ff46e5d9700 1 -- start start 2023-07-21T18:55:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.878+0000 7ff46e5d9700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4680fea50 0x7ff468196cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.878+0000 7ff46e5d9700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4680ff400 0x7ff4681971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.878+0000 7ff46e5d9700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff468104a60 0x7ff4681a1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.878+0000 7ff46e5d9700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4681a2120 con 0x7ff468104a60 2023-07-21T18:55:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.878+0000 7ff46e5d9700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4681a2290 con 0x7ff4680fea50 2023-07-21T18:55:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.878+0000 7ff46e5d9700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4681a2450 con 0x7ff4680ff400 2023-07-21T18:55:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.879+0000 7ff46cb76700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff468104a60 0x7ff4681a1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.879+0000 7ff467fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4680fea50 0x7ff468196cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.879+0000 7ff4677fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4680ff400 0x7ff4681971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:02.894 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.879+0000 7ff46cb76700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff468104a60 0x7ff4681a1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33056/0 (socket says 172.21.15.39:33056) 2023-07-21T18:55:02.894 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.879+0000 7ff46cb76700 1 -- 172.21.15.39:0/3346132973 learned_addr learned my addr 172.21.15.39:0/3346132973 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:55:02.894 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.879+0000 7ff4677fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4680ff400 0x7ff4681971f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:51606/0 (socket says 172.21.15.39:51606) 2023-07-21T18:55:02.894 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.879+0000 7ff467fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4680fea50 0x7ff468196cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:45648/0 (socket says 172.21.15.39:45648) 2023-07-21T18:55:02.894 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.879+0000 7ff46cb76700 1 -- 172.21.15.39:0/3346132973 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4680ff400 msgr2=0x7ff4681971f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:55:02.894 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.879+0000 7ff46cb76700 1 --2- 172.21.15.39:0/3346132973 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4680ff400 0x7ff4681971f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:02.894 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.879+0000 7ff46cb76700 1 -- 172.21.15.39:0/3346132973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4680fea50 msgr2=0x7ff468196cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:55:02.895 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.879+0000 7ff46cb76700 1 --2- 172.21.15.39:0/3346132973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4680fea50 0x7ff468196cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:02.895 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.879+0000 7ff46cb76700 1 -- 172.21.15.39:0/3346132973 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff460009d00 con 0x7ff468104a60 2023-07-21T18:55:02.895 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.879+0000 7ff46cb76700 1 --2- 172.21.15.39:0/3346132973 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff468104a60 0x7ff4681a1b50 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7ff46000b780 tx=0x7ff460007990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:55:02.895 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.879+0000 7ff4657fa700 1 -- 172.21.15.39:0/3346132973 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46000e040 con 0x7ff468104a60 2023-07-21T18:55:02.895 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.879+0000 7ff4657fa700 1 -- 172.21.15.39:0/3346132973 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff46001d470 con 0x7ff468104a60 2023-07-21T18:55:02.895 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.879+0000 7ff46e5d9700 1 -- 172.21.15.39:0/3346132973 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4681a26d0 con 0x7ff468104a60 2023-07-21T18:55:02.895 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.879+0000 7ff4657fa700 1 -- 172.21.15.39:0/3346132973 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4600052a0 con 0x7ff468104a60 2023-07-21T18:55:02.895 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.879+0000 7ff46e5d9700 1 -- 172.21.15.39:0/3346132973 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4681a2b00 con 0x7ff468104a60 2023-07-21T18:55:02.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.881+0000 7ff4657fa700 1 -- 172.21.15.39:0/3346132973 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7ff46001d5e0 con 0x7ff468104a60 2023-07-21T18:55:02.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.882+0000 7ff4657fa700 1 --2- 172.21.15.39:0/3346132973 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7ff4500a12c0 0x7ff4500a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:02.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.882+0000 7ff467fff700 1 --2- 172.21.15.39:0/3346132973 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7ff4500a12c0 0x7ff4500a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:02.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.882+0000 7ff4657fa700 1 -- 172.21.15.39:0/3346132973 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8527+0+0 (secure 0 0 0) 0x7ff4600ce2a0 con 0x7ff468104a60 2023-07-21T18:55:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.882+0000 7ff467fff700 1 --2- 172.21.15.39:0/3346132973 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7ff4500a12c0 0x7ff4500a3770 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff4680ffb90 tx=0x7ff458009260 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:55:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.883+0000 7ff47a75fb80 1 Processor -- start 2023-07-21T18:55:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.883+0000 7ff47a75fb80 1 -- start start 2023-07-21T18:55:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.883+0000 7ff47a75fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618e1fea6b0 0x5618e2193d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.883+0000 7ff47a75fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618e21942d0 0x5618e218e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.883+0000 7ff47a75fb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618e218e580 0x5618e218e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.883+0000 7ff47a75fb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618e2093c50 con 0x5618e218e580 2023-07-21T18:55:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.883+0000 7ff47a75fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618e20e69e0 con 0x5618e21942d0 2023-07-21T18:55:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.883+0000 7ff47a75fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618e20dfe70 con 0x5618e1fea6b0 2023-07-21T18:55:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.883+0000 7ff46cb76700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618e218e580 0x5618e218e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.883+0000 7ff4677fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618e21942d0 0x5618e218e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.883+0000 7ff467fff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618e1fea6b0 0x5618e2193d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.883+0000 7ff46cb76700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618e218e580 0x5618e218e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33064/0 (socket says 172.21.15.39:33064) 2023-07-21T18:55:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.883+0000 7ff4677fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618e21942d0 0x5618e218e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:45658/0 (socket says 172.21.15.39:45658) 2023-07-21T18:55:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.883+0000 7ff467fff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618e1fea6b0 0x5618e2193d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:51612/0 (socket says 172.21.15.39:51612) 2023-07-21T18:55:02.902 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.883+0000 7ff46cb76700 1 -- 172.21.15.39:0/63092523 learned_addr learned my addr 172.21.15.39:0/63092523 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:55:02.902 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.884+0000 7ff46cb76700 1 -- 172.21.15.39:0/63092523 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618e1fea6b0 msgr2=0x5618e2193d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:55:02.902 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.884+0000 7ff46cb76700 1 --2- 172.21.15.39:0/63092523 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618e1fea6b0 0x5618e2193d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:02.902 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.884+0000 7ff46cb76700 1 -- 172.21.15.39:0/63092523 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618e21942d0 msgr2=0x5618e218e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:55:02.902 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.884+0000 7ff46cb76700 1 --2- 172.21.15.39:0/63092523 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618e21942d0 0x5618e218e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:02.902 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.884+0000 7ff46cb76700 1 -- 172.21.15.39:0/63092523 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5618e1f0dd80 con 0x5618e218e580 2023-07-21T18:55:02.902 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.884+0000 7ff4677fe700 1 --2- 172.21.15.39:0/63092523 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618e21942d0 0x5618e218e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:55:02.903 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.884+0000 7ff46cb76700 1 --2- 172.21.15.39:0/63092523 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618e218e580 0x5618e218e9d0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7ff460009fd0 tx=0x7ff46001dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff11c2bc51cc97c3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:55:02.903 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.884+0000 7ff44e7fc700 1 -- 172.21.15.39:0/63092523 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46009b3a0 con 0x5618e218e580 2023-07-21T18:55:02.903 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.884+0000 7ff44e7fc700 1 -- 172.21.15.39:0/63092523 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4600ce070 con 0x5618e218e580 2023-07-21T18:55:02.903 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.884+0000 7ff44e7fc700 1 -- 172.21.15.39:0/63092523 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff460097ab0 con 0x5618e218e580 2023-07-21T18:55:02.903 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.885+0000 7ff47a75fb80 1 -- 172.21.15.39:0/63092523 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618e218e580 msgr2=0x5618e218e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:55:02.903 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.885+0000 7ff47a75fb80 1 --2- 172.21.15.39:0/63092523 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618e218e580 0x5618e218e9d0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7ff460009fd0 tx=0x7ff46001dc90 comp rx=0 tx=0).stop 2023-07-21T18:55:02.903 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.885+0000 7ff47a75fb80 1 -- 172.21.15.39:0/63092523 shutdown_connections 2023-07-21T18:55:02.903 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.885+0000 7ff47a75fb80 1 --2- 172.21.15.39:0/63092523 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618e218e580 0x5618e218e9d0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.885+0000 7ff47a75fb80 1 --2- 172.21.15.39:0/63092523 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618e1fea6b0 0x5618e2193d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.885+0000 7ff47a75fb80 1 --2- 172.21.15.39:0/63092523 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618e21942d0 0x5618e218e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.885+0000 7ff47a75fb80 1 -- 172.21.15.39:0/63092523 >> 172.21.15.39:0/63092523 conn(0x5618e2183be0 msgr2=0x5618e2188f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:55:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.885+0000 7ff47a75fb80 1 -- 172.21.15.39:0/63092523 shutdown_connections 2023-07-21T18:55:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.885+0000 7ff47a75fb80 1 -- 172.21.15.39:0/63092523 wait complete. 2023-07-21T18:55:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.885+0000 7ff47a75fb80 1 -- start start 2023-07-21T18:55:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.885+0000 7ff47a75fb80 1 Processor -- start 2023-07-21T18:55:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.886+0000 7ff47a75fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618e1fea6b0 0x5618e219d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.886+0000 7ff47a75fb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618e218e580 0x5618e219da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.886+0000 7ff47a75fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618e21942d0 0x5618e21977f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.886+0000 7ff47a75fb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618e207fa60 con 0x5618e218e580 2023-07-21T18:55:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.886+0000 7ff47a75fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618e207f760 con 0x5618e1fea6b0 2023-07-21T18:55:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.886+0000 7ff47a75fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618e1f638c0 con 0x5618e21942d0 2023-07-21T18:55:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.886+0000 7ff47a75fb80 10 client.0 _ll_register_callbacks cb 0x5618e207d700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:55:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.886+0000 7ff4677fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618e218e580 0x5618e219da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.886+0000 7ff467fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618e1fea6b0 0x5618e219d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.886+0000 7ff467fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618e1fea6b0 0x5618e219d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:45662/0 (socket says 172.21.15.39:45662) 2023-07-21T18:55:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.886+0000 7ff4677fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618e218e580 0x5618e219da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33068/0 (socket says 172.21.15.39:33068) 2023-07-21T18:55:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.886+0000 7ff467fff700 1 -- 172.21.15.39:0/2988193302 learned_addr learned my addr 172.21.15.39:0/2988193302 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:55:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.886+0000 7ff46cb76700 1 --2- 172.21.15.39:0/2988193302 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618e21942d0 0x5618e21977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.886+0000 7ff4677fe700 1 -- 172.21.15.39:0/2988193302 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618e21942d0 msgr2=0x5618e21977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:55:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.886+0000 7ff4677fe700 1 --2- 172.21.15.39:0/2988193302 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618e21942d0 0x5618e21977f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.886+0000 7ff4677fe700 1 -- 172.21.15.39:0/2988193302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618e1fea6b0 msgr2=0x5618e219d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:55:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.886+0000 7ff4677fe700 1 --2- 172.21.15.39:0/2988193302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618e1fea6b0 0x5618e219d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.886+0000 7ff4677fe700 1 -- 172.21.15.39:0/2988193302 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff45c0098c0 con 0x5618e218e580 2023-07-21T18:55:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.886+0000 7ff4677fe700 1 --2- 172.21.15.39:0/2988193302 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618e218e580 0x5618e219da30 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7ff468068ea0 tx=0x7ff45c012c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a2ef8537aa91da server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:55:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.886+0000 7ff4477fe700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:55:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.887+0000 7ff4477fe700 1 -- 172.21.15.39:0/2988193302 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45c004250 con 0x5618e218e580 2023-07-21T18:55:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.887+0000 7ff47a75fb80 1 -- 172.21.15.39:0/2988193302 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5618e2039490 con 0x5618e218e580 2023-07-21T18:55:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.887+0000 7ff4477fe700 1 -- 172.21.15.39:0/2988193302 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8527+0+0 (secure 0 0 0) 0x7ff45c0043b0 con 0x5618e218e580 2023-07-21T18:55:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.887+0000 7ff4477fe700 1 -- 172.21.15.39:0/2988193302 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff45c004570 con 0x5618e218e580 2023-07-21T18:55:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.887+0000 7ff47a75fb80 10 client.17169 fetch_fsmap learned FSMap version 78 2023-07-21T18:55:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.887+0000 7ff47a75fb80 1 -- 172.21.15.39:0/2988193302 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x7ff4681a2b00 con 0x5618e218e580 2023-07-21T18:55:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.887+0000 7ff4477fe700 1 -- 172.21.15.39:0/2988193302 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5618e2039490 con 0x5618e218e580 2023-07-21T18:55:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.887+0000 7ff4477fe700 1 -- 172.21.15.39:0/2988193302 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 78) v1 ==== 7506+0+0 (secure 0 0 0) 0x7ff45c003b70 con 0x5618e218e580 2023-07-21T18:55:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.887+0000 7ff47a75fb80 10 client.17169 fetch_fsmap finished waiting for FSMap version 78 2023-07-21T18:55:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.887+0000 7ff47a75fb80 10 client.17169 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:55:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.887+0000 7ff47a75fb80 20 client.17169 populate_metadata read hostname 'smithi039' 2023-07-21T18:55:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.887+0000 7ff47a75fb80 1 --2- 172.21.15.39:0/2988193302 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5618e21a7a40 0x5618e21a9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.887+0000 7ff47a75fb80 4 client.17169 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:55:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.887+0000 7ff47a75fb80 1 -- 172.21.15.39:0/2988193302 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5618e20d8e50 con 0x5618e21a7a40 2023-07-21T18:55:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.888+0000 7ff467fff700 1 --2- 172.21.15.39:0/2988193302 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5618e21a7a40 0x5618e21a9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.889+0000 7ff467fff700 1 --2- 172.21.15.39:0/2988193302 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5618e21a7a40 0x5618e21a9e50 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c60103a1c12a0cc6 server_cookie=5c48cecceee01384 in_seq=0 out_seq=0 2023-07-21T18:55:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.889+0000 7ff4477fe700 10 client.17169 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:55:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.891+0000 7ff4477fe700 1 -- 172.21.15.39:0/2988193302 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5618e20d8e50 con 0x5618e21a7a40 2023-07-21T18:55:02.911 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.891+0000 7ff4477fe700 10 client.17169 handle_command_reply: tid=0 2023-07-21T18:55:02.911 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.891+0000 7ff47a75fb80 1 client.17169 shutdown 2023-07-21T18:55:02.911 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.891+0000 7ff47a75fb80 1 -- 172.21.15.39:0/2988193302 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618e218e580 msgr2=0x5618e219da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:55:02.911 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.891+0000 7ff47a75fb80 1 --2- 172.21.15.39:0/2988193302 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618e218e580 0x5618e219da30 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7ff468068ea0 tx=0x7ff45c012c30 comp rx=0 tx=0).stop 2023-07-21T18:55:02.911 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.891+0000 7ff47a75fb80 1 -- 172.21.15.39:0/2988193302 shutdown_connections 2023-07-21T18:55:02.911 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.891+0000 7ff47a75fb80 1 --2- 172.21.15.39:0/2988193302 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5618e21a7a40 0x5618e21a9e50 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:02.911 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.891+0000 7ff47a75fb80 1 --2- 172.21.15.39:0/2988193302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618e1fea6b0 0x5618e219d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:02.912 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.891+0000 7ff47a75fb80 1 --2- 172.21.15.39:0/2988193302 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618e21942d0 0x5618e21977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:02.912 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.891+0000 7ff4477fe700 0 client.17169 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:55:02.912 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.891+0000 7ff47a75fb80 1 --2- 172.21.15.39:0/2988193302 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618e218e580 0x5618e219da30 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:02.912 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.891+0000 7ff47a75fb80 1 -- 172.21.15.39:0/2988193302 >> 172.21.15.39:0/2988193302 conn(0x5618e2183be0 msgr2=0x5618e21884e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:55:02.912 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.892+0000 7ff47a75fb80 1 -- 172.21.15.39:0/2988193302 shutdown_connections 2023-07-21T18:55:02.912 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.892+0000 7ff47a75fb80 1 -- 172.21.15.39:0/2988193302 wait complete. 2023-07-21T18:55:02.912 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.892+0000 7ff47a75fb80 20 client.17169 trim_cache size 0 max 16384 2023-07-21T18:55:02.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.907+0000 7ff47a75fb80 1 Processor -- start 2023-07-21T18:55:02.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.907+0000 7ff47a75fb80 1 -- start start 2023-07-21T18:55:02.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.907+0000 7ff47a75fb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618e1fea6b0 0x5618e218e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:02.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.907+0000 7ff47a75fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618e218e580 0x5618e2197710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:02.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.907+0000 7ff47a75fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618e21942d0 0x5618e2188c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:02.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.907+0000 7ff47a75fb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618e20d8e50 con 0x5618e1fea6b0 2023-07-21T18:55:02.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.907+0000 7ff47a75fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618e2039490 con 0x5618e21942d0 2023-07-21T18:55:02.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.907+0000 7ff47a75fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618e218a5b0 con 0x5618e218e580 2023-07-21T18:55:02.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.907+0000 7ff467fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618e1fea6b0 0x5618e218e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:02.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.907+0000 7ff4677fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618e218e580 0x5618e2197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:02.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.907+0000 7ff46cb76700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618e21942d0 0x5618e2188c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:02.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.907+0000 7ff4677fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618e218e580 0x5618e2197710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:51626/0 (socket says 172.21.15.39:51626) 2023-07-21T18:55:02.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.907+0000 7ff467fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618e1fea6b0 0x5618e218e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33078/0 (socket says 172.21.15.39:33078) 2023-07-21T18:55:02.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.907+0000 7ff4677fe700 1 -- 172.21.15.39:0/3016506712 learned_addr learned my addr 172.21.15.39:0/3016506712 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:55:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.908+0000 7ff4677fe700 1 -- 172.21.15.39:0/3016506712 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618e21942d0 msgr2=0x5618e2188c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:55:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.908+0000 7ff4677fe700 1 --2- 172.21.15.39:0/3016506712 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618e21942d0 0x5618e2188c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.908+0000 7ff4677fe700 1 -- 172.21.15.39:0/3016506712 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618e1fea6b0 msgr2=0x5618e218e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:55:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.908+0000 7ff4677fe700 1 --2- 172.21.15.39:0/3016506712 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618e1fea6b0 0x5618e218e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.908+0000 7ff4677fe700 1 -- 172.21.15.39:0/3016506712 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5618e218ee90 con 0x5618e218e580 2023-07-21T18:55:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.908+0000 7ff467fff700 1 --2- 172.21.15.39:0/3016506712 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618e1fea6b0 0x5618e218e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:55:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.908+0000 7ff4677fe700 1 --2- 172.21.15.39:0/3016506712 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618e218e580 0x5618e2197710 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff45c009340 tx=0x7ff45c019570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9767b33c54b42bd6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:55:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.908+0000 7ff44effd700 1 -- 172.21.15.39:0/3016506712 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45c0050f0 con 0x5618e218e580 2023-07-21T18:55:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.908+0000 7ff44effd700 1 -- 172.21.15.39:0/3016506712 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff45c01d070 con 0x5618e218e580 2023-07-21T18:55:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.908+0000 7ff44effd700 1 -- 172.21.15.39:0/3016506712 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45c003cc0 con 0x5618e218e580 2023-07-21T18:55:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.909+0000 7ff47a75fb80 1 -- 172.21.15.39:0/3016506712 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618e218e580 msgr2=0x5618e2197710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:55:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.909+0000 7ff47a75fb80 1 --2- 172.21.15.39:0/3016506712 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618e218e580 0x5618e2197710 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff45c009340 tx=0x7ff45c019570 comp rx=0 tx=0).stop 2023-07-21T18:55:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.909+0000 7ff47a75fb80 1 -- 172.21.15.39:0/3016506712 shutdown_connections 2023-07-21T18:55:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.909+0000 7ff47a75fb80 1 --2- 172.21.15.39:0/3016506712 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618e21942d0 0x5618e2188c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.909+0000 7ff47a75fb80 1 --2- 172.21.15.39:0/3016506712 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618e1fea6b0 0x5618e218e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.909+0000 7ff47a75fb80 1 --2- 172.21.15.39:0/3016506712 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618e218e580 0x5618e2197710 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:02.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.909+0000 7ff47a75fb80 1 -- 172.21.15.39:0/3016506712 >> 172.21.15.39:0/3016506712 conn(0x5618e2183be0 msgr2=0x5618e21933b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:55:02.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.909+0000 7ff47a75fb80 1 -- 172.21.15.39:0/3016506712 shutdown_connections 2023-07-21T18:55:02.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.909+0000 7ff47a75fb80 1 -- 172.21.15.39:0/3016506712 wait complete. 2023-07-21T18:55:02.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.909+0000 7ff47a75fb80 1 -- start start 2023-07-21T18:55:02.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.909+0000 7ff47a75fb80 1 Processor -- start 2023-07-21T18:55:02.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.910+0000 7ff47a75fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618e1fea6b0 0x5618e21fd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:02.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.910+0000 7ff47a75fb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618e218e580 0x5618e21fd990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.910+0000 7ff47a75fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618e21942d0 0x5618e21f80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.910+0000 7ff47a75fb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618e21f8670 con 0x5618e218e580 2023-07-21T18:55:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.910+0000 7ff47a75fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618e21f87e0 con 0x5618e1fea6b0 2023-07-21T18:55:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.910+0000 7ff47a75fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618e21f89a0 con 0x5618e21942d0 2023-07-21T18:55:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.910+0000 7ff47a75fb80 10 client.0 _ll_register_callbacks cb 0x7ff45c012450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:55:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.910+0000 7ff4677fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618e218e580 0x5618e21fd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.910+0000 7ff4677fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618e218e580 0x5618e21fd990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33082/0 (socket says 172.21.15.39:33082) 2023-07-21T18:55:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.910+0000 7ff4677fe700 1 -- 172.21.15.39:0/1561363118 learned_addr learned my addr 172.21.15.39:0/1561363118 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:55:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.910+0000 7ff467fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618e1fea6b0 0x5618e21fd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.910+0000 7ff46cb76700 1 --2- 172.21.15.39:0/1561363118 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618e21942d0 0x5618e21f80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.910+0000 7ff4677fe700 1 -- 172.21.15.39:0/1561363118 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618e21942d0 msgr2=0x5618e21f80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:55:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.910+0000 7ff4677fe700 1 --2- 172.21.15.39:0/1561363118 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618e21942d0 0x5618e21f80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.910+0000 7ff4677fe700 1 -- 172.21.15.39:0/1561363118 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618e1fea6b0 msgr2=0x5618e21fd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:55:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.910+0000 7ff4677fe700 1 --2- 172.21.15.39:0/1561363118 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618e1fea6b0 0x5618e21fd450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.910+0000 7ff4677fe700 1 -- 172.21.15.39:0/1561363118 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff458017070 con 0x5618e218e580 2023-07-21T18:55:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.910+0000 7ff46cb76700 1 --2- 172.21.15.39:0/1561363118 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618e21942d0 0x5618e21f80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:55:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.910+0000 7ff4677fe700 1 --2- 172.21.15.39:0/1561363118 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618e218e580 0x5618e21fd990 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7ff45c012920 tx=0x7ff45c009370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac4f54d6675b0771 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:55:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.910+0000 7ff4477fe700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:55:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.910+0000 7ff4477fe700 1 -- 172.21.15.39:0/1561363118 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45c003c30 con 0x5618e218e580 2023-07-21T18:55:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.911+0000 7ff4477fe700 1 -- 172.21.15.39:0/1561363118 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8527+0+0 (secure 0 0 0) 0x7ff45c019900 con 0x5618e218e580 2023-07-21T18:55:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.911+0000 7ff47a75fb80 1 -- 172.21.15.39:0/1561363118 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff4600ce070 con 0x5618e218e580 2023-07-21T18:55:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.911+0000 7ff4477fe700 1 -- 172.21.15.39:0/1561363118 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff45c003680 con 0x5618e218e580 2023-07-21T18:55:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.911+0000 7ff47a75fb80 10 client.17175 fetch_fsmap learned FSMap version 78 2023-07-21T18:55:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.911+0000 7ff47a75fb80 1 -- 172.21.15.39:0/1561363118 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x7ff4681a2b00 con 0x5618e218e580 2023-07-21T18:55:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.911+0000 7ff4477fe700 1 -- 172.21.15.39:0/1561363118 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4600ce070 con 0x5618e218e580 2023-07-21T18:55:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.911+0000 7ff4477fe700 1 -- 172.21.15.39:0/1561363118 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 78) v1 ==== 7506+0+0 (secure 0 0 0) 0x7ff45c018030 con 0x5618e218e580 2023-07-21T18:55:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.911+0000 7ff47a75fb80 10 client.17175 fetch_fsmap finished waiting for FSMap version 78 2023-07-21T18:55:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.911+0000 7ff47a75fb80 10 client.17175 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:55:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.911+0000 7ff47a75fb80 20 client.17175 populate_metadata read hostname 'smithi039' 2023-07-21T18:55:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.911+0000 7ff47a75fb80 1 --2- 172.21.15.39:0/1561363118 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5618e2204e00 0x5618e2207210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:02.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.911+0000 7ff47a75fb80 4 client.17175 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:55:02.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.911+0000 7ff47a75fb80 1 -- 172.21.15.39:0/1561363118 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5618e209aff0 con 0x5618e2204e00 2023-07-21T18:55:02.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.911+0000 7ff467fff700 1 --2- 172.21.15.39:0/1561363118 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5618e2204e00 0x5618e2207210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:02.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.912+0000 7ff467fff700 1 --2- 172.21.15.39:0/1561363118 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5618e2204e00 0x5618e2207210 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25835e7a1152670b server_cookie=6b98842625b75b98 in_seq=0 out_seq=0 2023-07-21T18:55:02.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:02.912+0000 7ff4477fe700 10 client.17175 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:55:03.242 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:03.230+0000 7ff4477fe700 1 -- 172.21.15.39:0/1561363118 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5618e209aff0 con 0x5618e2204e00 2023-07-21T18:55:03.242 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:03.230+0000 7ff4477fe700 10 client.17175 handle_command_reply: tid=0 2023-07-21T18:55:03.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:03.230+0000 7ff47a75fb80 1 client.17175 shutdown 2023-07-21T18:55:03.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:03.231+0000 7ff47a75fb80 1 -- 172.21.15.39:0/1561363118 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618e218e580 msgr2=0x5618e21fd990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:55:03.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:03.231+0000 7ff47a75fb80 1 --2- 172.21.15.39:0/1561363118 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618e218e580 0x5618e21fd990 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7ff45c012920 tx=0x7ff45c009370 comp rx=0 tx=0).stop 2023-07-21T18:55:03.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:03.232+0000 7ff47a75fb80 1 -- 172.21.15.39:0/1561363118 shutdown_connections 2023-07-21T18:55:03.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:03.232+0000 7ff47a75fb80 1 --2- 172.21.15.39:0/1561363118 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5618e2204e00 0x5618e2207210 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:03.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:03.232+0000 7ff47a75fb80 1 --2- 172.21.15.39:0/1561363118 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618e1fea6b0 0x5618e21fd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:03.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:03.232+0000 7ff47a75fb80 1 --2- 172.21.15.39:0/1561363118 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618e21942d0 0x5618e21f80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:03.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:03.232+0000 7ff47a75fb80 1 --2- 172.21.15.39:0/1561363118 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618e218e580 0x5618e21fd990 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:03.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:03.232+0000 7ff47a75fb80 1 -- 172.21.15.39:0/1561363118 >> 172.21.15.39:0/1561363118 conn(0x5618e2183be0 msgr2=0x5618e21933b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:55:03.245 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:03.232+0000 7ff4477fe700 0 client.17175 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:55:03.245 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:03.232+0000 7ff47a75fb80 1 -- 172.21.15.39:0/1561363118 shutdown_connections 2023-07-21T18:55:03.245 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:03.232+0000 7ff47a75fb80 1 -- 172.21.15.39:0/1561363118 wait complete. 2023-07-21T18:55:03.245 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:03.232+0000 7ff47a75fb80 20 client.17175 trim_cache size 0 max 16384 2023-07-21T18:55:03.245 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:55:03.246 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:55:03.246 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:55:03.246 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:55:03.246 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:03.232+0000 7ff46e5d9700 1 -- 172.21.15.39:0/3346132973 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7ff4500a12c0 msgr2=0x7ff4500a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:55:03.247 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:03.232+0000 7ff46e5d9700 1 --2- 172.21.15.39:0/3346132973 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7ff4500a12c0 0x7ff4500a3770 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff4680ffb90 tx=0x7ff458009260 comp rx=0 tx=0).stop 2023-07-21T18:55:03.247 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:03.232+0000 7ff46e5d9700 1 -- 172.21.15.39:0/3346132973 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff468104a60 msgr2=0x7ff4681a1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:55:03.247 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:03.232+0000 7ff46e5d9700 1 --2- 172.21.15.39:0/3346132973 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff468104a60 0x7ff4681a1b50 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7ff46000b780 tx=0x7ff460007990 comp rx=0 tx=0).stop 2023-07-21T18:55:03.247 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:03.233+0000 7ff46e5d9700 1 -- 172.21.15.39:0/3346132973 shutdown_connections 2023-07-21T18:55:03.248 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:03.233+0000 7ff46e5d9700 1 --2- 172.21.15.39:0/3346132973 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7ff4500a12c0 0x7ff4500a3770 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:03.248 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:03.233+0000 7ff46e5d9700 1 --2- 172.21.15.39:0/3346132973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4680fea50 0x7ff468196cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:03.248 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:03.233+0000 7ff46e5d9700 1 --2- 172.21.15.39:0/3346132973 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff468104a60 0x7ff4681a1b50 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:03.248 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:03.233+0000 7ff46e5d9700 1 --2- 172.21.15.39:0/3346132973 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4680ff400 0x7ff4681971f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:03.248 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:03.233+0000 7ff46e5d9700 1 -- 172.21.15.39:0/3346132973 >> 172.21.15.39:0/3346132973 conn(0x7ff4680fa570 msgr2=0x7ff468105860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:55:03.248 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:03.233+0000 7ff46e5d9700 1 -- 172.21.15.39:0/3346132973 shutdown_connections 2023-07-21T18:55:03.248 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:03.233+0000 7ff46e5d9700 1 -- 172.21.15.39:0/3346132973 wait complete. 2023-07-21T18:55:03.653 INFO:tasks.workunit.client.0.smithi039.stdout: 10 5179 28.15 MB/sec warmup 16 sec latency 198.658 ms 2023-07-21T18:55:03.859 INFO:journalctl@ceph.mds.c.smithi193.stdout:Jul 21 18:55:03 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-c[124252]: ignoring --setuser ceph since I am not root 2023-07-21T18:55:03.859 INFO:journalctl@ceph.mds.c.smithi193.stdout:Jul 21 18:55:03 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-c[124252]: ignoring --setgroup ceph since I am not root 2023-07-21T18:55:03.859 INFO:journalctl@ceph.mds.c.smithi193.stdout:Jul 21 18:55:03 smithi193 ceph-mds[124276]: -- 172.21.15.193:0/3300796117 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5602b8d971e0 con 0x5602b8d6d000 2023-07-21T18:55:03.859 INFO:journalctl@ceph.mds.c.smithi193.stdout:Jul 21 18:55:03 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-c[124252]: starting mds.c at 2023-07-21T18:55:04.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:55:04 smithi193 ceph-mon[98204]: pgmap v417: 97 pgs: 97 active+clean; 508 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 22 MiB/s wr, 298 op/s 2023-07-21T18:55:04.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:55:04 smithi193 ceph-mon[98204]: daemon mds.c restarted 2023-07-21T18:55:04.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:55:04 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:55:04.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:55:04 smithi193 ceph-mon[98204]: osdmap e98: 12 total, 12 up, 12 in 2023-07-21T18:55:04.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:55:04 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:55:04.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:55:04 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:55:04.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:55:04 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:55:04.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:55:04 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:55:04.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:55:04 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:55:04.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:55:04 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:55:04.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:55:04 smithi155 ceph-mon[97934]: pgmap v417: 97 pgs: 97 active+clean; 508 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 22 MiB/s wr, 298 op/s 2023-07-21T18:55:04.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:55:04 smithi155 ceph-mon[97934]: daemon mds.c restarted 2023-07-21T18:55:04.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:55:04 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:55:04.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:55:04 smithi155 ceph-mon[97934]: osdmap e98: 12 total, 12 up, 12 in 2023-07-21T18:55:04.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:55:04 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:55:04.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:55:04 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:55:04.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:55:04 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:55:04.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:55:04 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:55:04.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:55:04 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:55:04.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:55:04 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:55:04.653 INFO:tasks.workunit.client.0.smithi039.stdout: 10 5673 27.31 MB/sec warmup 17 sec latency 164.768 ms 2023-07-21T18:55:04.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:55:04 smithi039 ceph-mon[96489]: pgmap v417: 97 pgs: 97 active+clean; 508 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 22 MiB/s wr, 298 op/s 2023-07-21T18:55:04.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:55:04 smithi039 ceph-mon[96489]: daemon mds.c restarted 2023-07-21T18:55:04.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:55:04 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:55:04.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:55:04 smithi039 ceph-mon[96489]: osdmap e98: 12 total, 12 up, 12 in 2023-07-21T18:55:04.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:55:04 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:55:04.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:55:04 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:55:04.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:55:04 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:55:04.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:55:04 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:55:04.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:55:04 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:55:04.678 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:55:04 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:55:05.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:55:05 smithi193 ceph-mon[98204]: mds.? [v2:172.21.15.193:6836/2617962167,v1:172.21.15.193:6837/2617962167] up:boot 2023-07-21T18:55:05.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:55:05 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-21T18:55:05.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:55:05 smithi193 ceph-mon[98204]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2023-07-21T18:55:05.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:55:05 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:55:05.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:55:05 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:55:05.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:55:05 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:55:05.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:55:05 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:55:05.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:55:05 smithi155 ceph-mon[97934]: mds.? [v2:172.21.15.193:6836/2617962167,v1:172.21.15.193:6837/2617962167] up:boot 2023-07-21T18:55:05.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:55:05 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-21T18:55:05.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:55:05 smithi155 ceph-mon[97934]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2023-07-21T18:55:05.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:55:05 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:55:05.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:55:05 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:55:05.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:55:05 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:55:05.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:55:05 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:55:05.653 INFO:tasks.workunit.client.0.smithi039.stdout: 10 6041 27.17 MB/sec warmup 18 sec latency 58.193 ms 2023-07-21T18:55:05.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:55:05 smithi039 ceph-mon[96489]: mds.? [v2:172.21.15.193:6836/2617962167,v1:172.21.15.193:6837/2617962167] up:boot 2023-07-21T18:55:05.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:55:05 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-21T18:55:05.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:55:05 smithi039 ceph-mon[96489]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2023-07-21T18:55:05.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:55:05 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:55:05.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:55:05 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:55:05.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:55:05 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:55:05.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:55:05 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:55:06.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:55:06 smithi193 ceph-mon[98204]: pgmap v419: 97 pgs: 97 active+clean; 508 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 26 MiB/s wr, 358 op/s 2023-07-21T18:55:06.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:55:06 smithi155 ceph-mon[97934]: pgmap v419: 97 pgs: 97 active+clean; 508 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 26 MiB/s wr, 358 op/s 2023-07-21T18:55:06.653 INFO:tasks.workunit.client.0.smithi039.stdout: 10 6467 27.67 MB/sec warmup 19 sec latency 72.087 ms 2023-07-21T18:55:06.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:55:06 smithi039 ceph-mon[96489]: pgmap v419: 97 pgs: 97 active+clean; 508 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 26 MiB/s wr, 358 op/s 2023-07-21T18:55:07.653 INFO:tasks.workunit.client.0.smithi039.stdout: 10 6813 27.68 MB/sec warmup 20 sec latency 89.663 ms 2023-07-21T18:55:08.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:55:08 smithi193 ceph-mon[98204]: pgmap v420: 97 pgs: 97 active+clean; 484 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.2 MiB/s wr, 661 op/s 2023-07-21T18:55:08.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:55:08 smithi155 ceph-mon[97934]: pgmap v420: 97 pgs: 97 active+clean; 484 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.2 MiB/s wr, 661 op/s 2023-07-21T18:55:08.653 INFO:tasks.workunit.client.0.smithi039.stdout: 10 7039 27.05 MB/sec warmup 21 sec latency 88.057 ms 2023-07-21T18:55:08.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:55:08 smithi039 ceph-mon[96489]: pgmap v420: 97 pgs: 97 active+clean; 484 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.2 MiB/s wr, 661 op/s 2023-07-21T18:55:09.653 INFO:tasks.workunit.client.0.smithi039.stdout: 10 7217 26.22 MB/sec warmup 22 sec latency 105.952 ms 2023-07-21T18:55:10.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:55:10 smithi193 ceph-mon[98204]: pgmap v421: 97 pgs: 97 active+clean; 484 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.2 MiB/s wr, 661 op/s 2023-07-21T18:55:10.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:55:10 smithi155 ceph-mon[97934]: pgmap v421: 97 pgs: 97 active+clean; 484 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.2 MiB/s wr, 661 op/s 2023-07-21T18:55:10.653 INFO:tasks.workunit.client.0.smithi039.stdout: 10 7374 25.14 MB/sec warmup 23 sec latency 120.693 ms 2023-07-21T18:55:10.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:55:10 smithi039 ceph-mon[96489]: pgmap v421: 97 pgs: 97 active+clean; 484 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.2 MiB/s wr, 661 op/s 2023-07-21T18:55:11.653 INFO:tasks.workunit.client.0.smithi039.stdout: 10 7563 24.23 MB/sec warmup 24 sec latency 247.505 ms 2023-07-21T18:55:12.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:55:12 smithi193 ceph-mon[98204]: pgmap v422: 97 pgs: 97 active+clean; 492 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.2 MiB/s wr, 626 op/s 2023-07-21T18:55:12.611 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:55:12 smithi193 ceph-mon[98204]: osdmap e99: 12 total, 12 up, 12 in 2023-07-21T18:55:12.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:55:12 smithi155 ceph-mon[97934]: pgmap v422: 97 pgs: 97 active+clean; 492 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.2 MiB/s wr, 626 op/s 2023-07-21T18:55:12.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:55:12 smithi155 ceph-mon[97934]: osdmap e99: 12 total, 12 up, 12 in 2023-07-21T18:55:12.653 INFO:tasks.workunit.client.0.smithi039.stdout: 10 7888 23.76 MB/sec warmup 25 sec latency 107.293 ms 2023-07-21T18:55:12.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:55:12 smithi039 ceph-mon[96489]: pgmap v422: 97 pgs: 97 active+clean; 492 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.2 MiB/s wr, 626 op/s 2023-07-21T18:55:12.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:55:12 smithi039 ceph-mon[96489]: osdmap e99: 12 total, 12 up, 12 in 2023-07-21T18:55:13.653 INFO:tasks.workunit.client.0.smithi039.stdout: 10 8279 23.73 MB/sec warmup 26 sec latency 116.757 ms 2023-07-21T18:55:14.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:55:14 smithi193 ceph-mon[98204]: pgmap v424: 97 pgs: 97 active+clean; 492 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.8 MiB/s wr, 751 op/s 2023-07-21T18:55:14.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:55:14 smithi155 ceph-mon[97934]: pgmap v424: 97 pgs: 97 active+clean; 492 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.8 MiB/s wr, 751 op/s 2023-07-21T18:55:14.653 INFO:tasks.workunit.client.0.smithi039.stdout: 10 8818 23.59 MB/sec warmup 27 sec latency 447.986 ms 2023-07-21T18:55:14.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:55:14 smithi039 ceph-mon[96489]: pgmap v424: 97 pgs: 97 active+clean; 492 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.8 MiB/s wr, 751 op/s 2023-07-21T18:55:15.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:55:15 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:55:15.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:55:15 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:55:15.653 INFO:tasks.workunit.client.0.smithi039.stdout: 10 9359 23.89 MB/sec warmup 28 sec latency 1448.029 ms 2023-07-21T18:55:15.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:55:15 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:55:16.362 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.351+0000 7f0d6d072700 1 -- 172.21.15.39:0/679935324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d68104a30 msgr2=0x7f0d68104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:55:16.362 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.351+0000 7f0d6d072700 1 --2- 172.21.15.39:0/679935324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d68104a30 0x7f0d68104e00 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f0d58009990 tx=0x7f0d58009ca0 comp rx=0 tx=0).stop 2023-07-21T18:55:16.362 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.351+0000 7f0d6d072700 1 -- 172.21.15.39:0/679935324 shutdown_connections 2023-07-21T18:55:16.363 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.351+0000 7f0d6d072700 1 --2- 172.21.15.39:0/679935324 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d680ff3e0 0x7f0d6810b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:16.363 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.351+0000 7f0d6d072700 1 --2- 172.21.15.39:0/679935324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d68104a30 0x7f0d68104e00 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:16.363 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.351+0000 7f0d6d072700 1 --2- 172.21.15.39:0/679935324 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d680fea30 0x7f0d680feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:16.363 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.351+0000 7f0d6d072700 1 -- 172.21.15.39:0/679935324 >> 172.21.15.39:0/679935324 conn(0x7f0d68074760 msgr2=0x7f0d68074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:55:16.363 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.351+0000 7f0d6d072700 1 -- 172.21.15.39:0/679935324 shutdown_connections 2023-07-21T18:55:16.363 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.351+0000 7f0d6d072700 1 -- 172.21.15.39:0/679935324 wait complete. 2023-07-21T18:55:16.365 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.352+0000 7f0d6d072700 1 Processor -- start 2023-07-21T18:55:16.365 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.352+0000 7f0d6d072700 1 -- start start 2023-07-21T18:55:16.365 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.352+0000 7f0d6d072700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d680fea30 0x7f0d6819aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:16.365 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.352+0000 7f0d6d072700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d680ff3e0 0x7f0d6819b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:16.365 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.352+0000 7f0d6d072700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d68104a30 0x7f0d68194930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:16.365 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.352+0000 7f0d6d072700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d68194fa0 con 0x7f0d68104a30 2023-07-21T18:55:16.366 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.352+0000 7f0d6d072700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d68195110 con 0x7f0d680ff3e0 2023-07-21T18:55:16.366 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.352+0000 7f0d6d072700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d681952d0 con 0x7f0d680fea30 2023-07-21T18:55:16.366 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.352+0000 7f0d66d9d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d680fea30 0x7f0d6819aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:16.366 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.352+0000 7f0d6759e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d68104a30 0x7f0d68194930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:16.366 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.352+0000 7f0d6659c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d680ff3e0 0x7f0d6819b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:16.366 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.352+0000 7f0d66d9d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d680fea30 0x7f0d6819aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:51648/0 (socket says 172.21.15.39:51648) 2023-07-21T18:55:16.366 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.352+0000 7f0d66d9d700 1 -- 172.21.15.39:0/840789386 learned_addr learned my addr 172.21.15.39:0/840789386 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:55:16.367 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.353+0000 7f0d6759e700 1 -- 172.21.15.39:0/840789386 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d680fea30 msgr2=0x7f0d6819aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:55:16.367 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.353+0000 7f0d6759e700 1 --2- 172.21.15.39:0/840789386 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d680fea30 0x7f0d6819aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:16.367 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.353+0000 7f0d6759e700 1 -- 172.21.15.39:0/840789386 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d680ff3e0 msgr2=0x7f0d6819b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:55:16.367 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.353+0000 7f0d6759e700 1 --2- 172.21.15.39:0/840789386 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d680ff3e0 0x7f0d6819b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:16.367 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.353+0000 7f0d6759e700 1 -- 172.21.15.39:0/840789386 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d580095f0 con 0x7f0d68104a30 2023-07-21T18:55:16.367 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.353+0000 7f0d66d9d700 1 --2- 172.21.15.39:0/840789386 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d680fea30 0x7f0d6819aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:55:16.367 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.353+0000 7f0d6759e700 1 --2- 172.21.15.39:0/840789386 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d68104a30 0x7f0d68194930 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f0d6000efc0 tx=0x7f0d6000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:55:16.368 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.353+0000 7f0d4ffff700 1 -- 172.21.15.39:0/840789386 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d60018070 con 0x7f0d68104a30 2023-07-21T18:55:16.368 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.353+0000 7f0d6d072700 1 -- 172.21.15.39:0/840789386 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d68195550 con 0x7f0d68104a30 2023-07-21T18:55:16.368 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.353+0000 7f0d4ffff700 1 -- 172.21.15.39:0/840789386 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d6000f040 con 0x7f0d68104a30 2023-07-21T18:55:16.368 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.353+0000 7f0d4ffff700 1 -- 172.21.15.39:0/840789386 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d6000cd40 con 0x7f0d68104a30 2023-07-21T18:55:16.368 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.353+0000 7f0d6d072700 1 -- 172.21.15.39:0/840789386 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d681a1ac0 con 0x7f0d68104a30 2023-07-21T18:55:16.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.355+0000 7f0d4ffff700 1 -- 172.21.15.39:0/840789386 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f0d60003a00 con 0x7f0d68104a30 2023-07-21T18:55:16.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.356+0000 7f0d4ffff700 1 --2- 172.21.15.39:0/840789386 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f0d500a12c0 0x7f0d500a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:16.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.356+0000 7f0d4ffff700 1 -- 172.21.15.39:0/840789386 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f0d600cc890 con 0x7f0d68104a30 2023-07-21T18:55:16.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.356+0000 7f0d66d9d700 1 --2- 172.21.15.39:0/840789386 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f0d500a12c0 0x7f0d500a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:16.372 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.356+0000 7f0d66d9d700 1 --2- 172.21.15.39:0/840789386 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f0d500a12c0 0x7f0d500a3770 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0d5800b2e0 tx=0x7f0d58005fd0 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:55:16.372 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.357+0000 7f0d791f8b80 1 Processor -- start 2023-07-21T18:55:16.372 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.357+0000 7f0d791f8b80 1 -- start start 2023-07-21T18:55:16.372 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.357+0000 7f0d791f8b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eab13726b0 0x55eab1516230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:16.372 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.357+0000 7f0d791f8b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eab1516770 0x55eab15231e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:16.372 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.357+0000 7f0d791f8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eab1516be0 0x55eab1523720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:16.372 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.357+0000 7f0d791f8b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eab141bc50 con 0x55eab13726b0 2023-07-21T18:55:16.373 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.357+0000 7f0d791f8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eab146e9e0 con 0x55eab1516be0 2023-07-21T18:55:16.373 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.357+0000 7f0d791f8b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eab1467e70 con 0x55eab1516770 2023-07-21T18:55:16.373 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.357+0000 7f0d66d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eab13726b0 0x55eab1516230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:16.373 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.357+0000 7f0d6659c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eab1516770 0x55eab15231e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:16.373 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.357+0000 7f0d66d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eab13726b0 0x55eab1516230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33104/0 (socket says 172.21.15.39:33104) 2023-07-21T18:55:16.373 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.357+0000 7f0d66d9d700 1 -- 172.21.15.39:0/4010804316 learned_addr learned my addr 172.21.15.39:0/4010804316 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:55:16.373 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.357+0000 7f0d6759e700 1 --2- 172.21.15.39:0/4010804316 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eab1516be0 0x55eab1523720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:16.373 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.358+0000 7f0d66d9d700 1 -- 172.21.15.39:0/4010804316 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eab1516770 msgr2=0x55eab15231e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:55:16.374 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.358+0000 7f0d66d9d700 1 --2- 172.21.15.39:0/4010804316 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eab1516770 0x55eab15231e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:16.374 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.358+0000 7f0d66d9d700 1 -- 172.21.15.39:0/4010804316 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eab1516be0 msgr2=0x55eab1523720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:55:16.374 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.358+0000 7f0d66d9d700 1 --2- 172.21.15.39:0/4010804316 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eab1516be0 0x55eab1523720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:16.374 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.358+0000 7f0d66d9d700 1 -- 172.21.15.39:0/4010804316 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eab1295d80 con 0x55eab13726b0 2023-07-21T18:55:16.374 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.358+0000 7f0d6659c700 1 --2- 172.21.15.39:0/4010804316 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eab1516770 0x55eab15231e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:55:16.374 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.358+0000 7f0d66d9d700 1 --2- 172.21.15.39:0/4010804316 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eab13726b0 0x55eab1516230 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f0d5800bf80 tx=0x7f0d58020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec839ee5385ee1ef server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:55:16.374 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.358+0000 7f0d4d7fa700 1 -- 172.21.15.39:0/4010804316 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d5802b070 con 0x55eab13726b0 2023-07-21T18:55:16.374 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.358+0000 7f0d4d7fa700 1 -- 172.21.15.39:0/4010804316 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d5801e040 con 0x55eab13726b0 2023-07-21T18:55:16.375 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.358+0000 7f0d4d7fa700 1 -- 172.21.15.39:0/4010804316 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d5802f400 con 0x55eab13726b0 2023-07-21T18:55:16.375 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.359+0000 7f0d791f8b80 1 -- 172.21.15.39:0/4010804316 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eab13726b0 msgr2=0x55eab1516230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:55:16.375 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.359+0000 7f0d791f8b80 1 --2- 172.21.15.39:0/4010804316 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eab13726b0 0x55eab1516230 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f0d5800bf80 tx=0x7f0d58020000 comp rx=0 tx=0).stop 2023-07-21T18:55:16.375 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.359+0000 7f0d791f8b80 1 -- 172.21.15.39:0/4010804316 shutdown_connections 2023-07-21T18:55:16.375 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.359+0000 7f0d791f8b80 1 --2- 172.21.15.39:0/4010804316 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eab1516be0 0x55eab1523720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:16.375 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.359+0000 7f0d791f8b80 1 --2- 172.21.15.39:0/4010804316 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eab13726b0 0x55eab1516230 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:16.375 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.359+0000 7f0d791f8b80 1 --2- 172.21.15.39:0/4010804316 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eab1516770 0x55eab15231e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:16.376 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.359+0000 7f0d791f8b80 1 -- 172.21.15.39:0/4010804316 >> 172.21.15.39:0/4010804316 conn(0x55eab150bbe0 msgr2=0x55eab15110c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:55:16.376 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.359+0000 7f0d791f8b80 1 -- 172.21.15.39:0/4010804316 shutdown_connections 2023-07-21T18:55:16.376 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.359+0000 7f0d791f8b80 1 -- 172.21.15.39:0/4010804316 wait complete. 2023-07-21T18:55:16.376 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.359+0000 7f0d791f8b80 1 -- start start 2023-07-21T18:55:16.376 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.359+0000 7f0d791f8b80 1 Processor -- start 2023-07-21T18:55:16.376 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.360+0000 7f0d791f8b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eab13726b0 0x55eab152de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:16.376 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.360+0000 7f0d791f8b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eab1516770 0x55eab152e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:16.376 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.360+0000 7f0d791f8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eab1516be0 0x55eab1528120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:16.377 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.360+0000 7f0d791f8b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eab1407a60 con 0x55eab13726b0 2023-07-21T18:55:16.377 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.360+0000 7f0d791f8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eab1407760 con 0x55eab1516be0 2023-07-21T18:55:16.377 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.360+0000 7f0d791f8b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eab12eb8c0 con 0x55eab1516770 2023-07-21T18:55:16.377 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.360+0000 7f0d66d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eab13726b0 0x55eab152de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:16.377 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.360+0000 7f0d791f8b80 10 client.0 _ll_register_callbacks cb 0x55eab1405700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:55:16.377 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.360+0000 7f0d66d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eab13726b0 0x55eab152de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33110/0 (socket says 172.21.15.39:33110) 2023-07-21T18:55:16.377 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.360+0000 7f0d66d9d700 1 -- 172.21.15.39:0/302261158 learned_addr learned my addr 172.21.15.39:0/302261158 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:55:16.378 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.360+0000 7f0d6659c700 1 --2- 172.21.15.39:0/302261158 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eab1516770 0x55eab152e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:16.378 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.360+0000 7f0d66d9d700 1 -- 172.21.15.39:0/302261158 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eab1516770 msgr2=0x55eab152e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:55:16.378 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.360+0000 7f0d66d9d700 1 --2- 172.21.15.39:0/302261158 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eab1516770 0x55eab152e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:16.378 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.360+0000 7f0d66d9d700 1 -- 172.21.15.39:0/302261158 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eab1516be0 msgr2=0x55eab1528120 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T18:55:16.378 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.360+0000 7f0d66d9d700 1 --2- 172.21.15.39:0/302261158 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eab1516be0 0x55eab1528120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:16.378 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.360+0000 7f0d66d9d700 1 -- 172.21.15.39:0/302261158 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d5c0098e0 con 0x55eab13726b0 2023-07-21T18:55:16.378 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.360+0000 7f0d66d9d700 1 --2- 172.21.15.39:0/302261158 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eab13726b0 0x55eab152de20 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f0d5800fa20 tx=0x7f0d58005a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af704d5fae10fd40 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:55:16.378 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.360+0000 7f0d4aff5700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:55:16.380 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.361+0000 7f0d4aff5700 1 -- 172.21.15.39:0/302261158 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d5802b040 con 0x55eab13726b0 2023-07-21T18:55:16.380 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.361+0000 7f0d4aff5700 1 -- 172.21.15.39:0/302261158 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f0d5802f6c0 con 0x55eab13726b0 2023-07-21T18:55:16.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.361+0000 7f0d791f8b80 1 -- 172.21.15.39:0/302261158 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eab13c1490 con 0x55eab13726b0 2023-07-21T18:55:16.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.361+0000 7f0d4aff5700 1 -- 172.21.15.39:0/302261158 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d5802f8a0 con 0x55eab13726b0 2023-07-21T18:55:16.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.361+0000 7f0d791f8b80 10 client.17193 fetch_fsmap learned FSMap version 81 2023-07-21T18:55:16.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.361+0000 7f0d791f8b80 1 -- 172.21.15.39:0/302261158 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) v3 -- 0x7f0d6810c160 con 0x55eab13726b0 2023-07-21T18:55:16.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.361+0000 7f0d4aff5700 1 -- 172.21.15.39:0/302261158 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d58045020 con 0x55eab13726b0 2023-07-21T18:55:16.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.361+0000 7f0d4aff5700 1 -- 172.21.15.39:0/302261158 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 81) v1 ==== 7510+0+0 (secure 0 0 0) 0x7f0d58017a60 con 0x55eab13726b0 2023-07-21T18:55:16.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.361+0000 7f0d791f8b80 10 client.17193 fetch_fsmap finished waiting for FSMap version 81 2023-07-21T18:55:16.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.361+0000 7f0d791f8b80 10 client.17193 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:55:16.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.361+0000 7f0d791f8b80 20 client.17193 populate_metadata read hostname 'smithi039' 2023-07-21T18:55:16.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.361+0000 7f0d791f8b80 1 --2- 172.21.15.39:0/302261158 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55eab1537f00 0x55eab153a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:16.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.361+0000 7f0d791f8b80 4 client.17193 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:55:16.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.361+0000 7f0d791f8b80 1 -- 172.21.15.39:0/302261158 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eab1460e50 con 0x55eab1537f00 2023-07-21T18:55:16.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.361+0000 7f0d6659c700 1 --2- 172.21.15.39:0/302261158 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55eab1537f00 0x55eab153a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:16.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.362+0000 7f0d6659c700 1 --2- 172.21.15.39:0/302261158 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55eab1537f00 0x55eab153a310 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecc3447edc828eb5 server_cookie=e75384c49b18adb8 in_seq=0 out_seq=0 2023-07-21T18:55:16.383 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.362+0000 7f0d4aff5700 10 client.17193 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:55:16.383 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.364+0000 7f0d4aff5700 1 -- 172.21.15.39:0/302261158 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55eab1460e50 con 0x55eab1537f00 2023-07-21T18:55:16.383 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.364+0000 7f0d4aff5700 10 client.17193 handle_command_reply: tid=0 2023-07-21T18:55:16.383 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.364+0000 7f0d791f8b80 1 client.17193 shutdown 2023-07-21T18:55:16.383 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.364+0000 7f0d791f8b80 1 -- 172.21.15.39:0/302261158 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eab13726b0 msgr2=0x55eab152de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:55:16.383 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.364+0000 7f0d791f8b80 1 --2- 172.21.15.39:0/302261158 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eab13726b0 0x55eab152de20 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f0d5800fa20 tx=0x7f0d58005a90 comp rx=0 tx=0).stop 2023-07-21T18:55:16.383 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.364+0000 7f0d791f8b80 1 -- 172.21.15.39:0/302261158 shutdown_connections 2023-07-21T18:55:16.383 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.364+0000 7f0d791f8b80 1 --2- 172.21.15.39:0/302261158 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55eab1537f00 0x55eab153a310 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:16.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.364+0000 7f0d791f8b80 1 --2- 172.21.15.39:0/302261158 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eab1516be0 0x55eab1528120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:16.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.364+0000 7f0d791f8b80 1 --2- 172.21.15.39:0/302261158 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eab13726b0 0x55eab152de20 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:16.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.364+0000 7f0d4aff5700 0 client.17193 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:55:16.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.364+0000 7f0d791f8b80 1 --2- 172.21.15.39:0/302261158 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eab1516770 0x55eab152e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:16.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.364+0000 7f0d791f8b80 1 -- 172.21.15.39:0/302261158 >> 172.21.15.39:0/302261158 conn(0x55eab150bbe0 msgr2=0x55eab15104e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:55:16.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.364+0000 7f0d791f8b80 1 -- 172.21.15.39:0/302261158 shutdown_connections 2023-07-21T18:55:16.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.364+0000 7f0d791f8b80 1 -- 172.21.15.39:0/302261158 wait complete. 2023-07-21T18:55:16.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.364+0000 7f0d791f8b80 20 client.17193 trim_cache size 0 max 16384 2023-07-21T18:55:16.391 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.380+0000 7f0d791f8b80 1 Processor -- start 2023-07-21T18:55:16.391 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.380+0000 7f0d791f8b80 1 -- start start 2023-07-21T18:55:16.392 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.380+0000 7f0d791f8b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eab13726b0 0x55eab15260d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:16.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.380+0000 7f0d791f8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eab1516770 0x55eab1528250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:16.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.380+0000 7f0d791f8b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eab1516be0 0x55eab1523490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:16.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.380+0000 7f0d791f8b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eab1511e30 con 0x55eab13726b0 2023-07-21T18:55:16.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.380+0000 7f0d791f8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eab153a970 con 0x55eab1516770 2023-07-21T18:55:16.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.380+0000 7f0d791f8b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eab1526680 con 0x55eab1516be0 2023-07-21T18:55:16.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.380+0000 7f0d66d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eab13726b0 0x55eab15260d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:16.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.381+0000 7f0d66d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eab13726b0 0x55eab15260d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33118/0 (socket says 172.21.15.39:33118) 2023-07-21T18:55:16.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.381+0000 7f0d66d9d700 1 -- 172.21.15.39:0/1992993188 learned_addr learned my addr 172.21.15.39:0/1992993188 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:55:16.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.381+0000 7f0d6659c700 1 --2- 172.21.15.39:0/1992993188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eab1516770 0x55eab1528250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:16.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.381+0000 7f0d6759e700 1 --2- 172.21.15.39:0/1992993188 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eab1516be0 0x55eab1523490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:16.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.381+0000 7f0d66d9d700 1 -- 172.21.15.39:0/1992993188 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eab1516be0 msgr2=0x55eab1523490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:55:16.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.381+0000 7f0d66d9d700 1 --2- 172.21.15.39:0/1992993188 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eab1516be0 0x55eab1523490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:16.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.381+0000 7f0d66d9d700 1 -- 172.21.15.39:0/1992993188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eab1516770 msgr2=0x55eab1528250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:55:16.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.381+0000 7f0d66d9d700 1 --2- 172.21.15.39:0/1992993188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eab1516770 0x55eab1528250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:16.395 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.381+0000 7f0d66d9d700 1 -- 172.21.15.39:0/1992993188 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eab153ef00 con 0x55eab13726b0 2023-07-21T18:55:16.395 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.381+0000 7f0d6659c700 1 --2- 172.21.15.39:0/1992993188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eab1516770 0x55eab1528250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:55:16.395 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.381+0000 7f0d66d9d700 1 --2- 172.21.15.39:0/1992993188 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eab13726b0 0x55eab15260d0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f0d58017dc0 tx=0x7f0d58017f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e24028ce3b70efc2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:55:16.395 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.381+0000 7f0d4dffb700 1 -- 172.21.15.39:0/1992993188 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d5802b040 con 0x55eab13726b0 2023-07-21T18:55:16.395 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.381+0000 7f0d4dffb700 1 -- 172.21.15.39:0/1992993188 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d58045030 con 0x55eab13726b0 2023-07-21T18:55:16.395 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.381+0000 7f0d4dffb700 1 -- 172.21.15.39:0/1992993188 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d5802f880 con 0x55eab13726b0 2023-07-21T18:55:16.395 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.382+0000 7f0d791f8b80 1 -- 172.21.15.39:0/1992993188 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eab13726b0 msgr2=0x55eab15260d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:55:16.396 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.382+0000 7f0d791f8b80 1 --2- 172.21.15.39:0/1992993188 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eab13726b0 0x55eab15260d0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f0d58017dc0 tx=0x7f0d58017f90 comp rx=0 tx=0).stop 2023-07-21T18:55:16.396 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.382+0000 7f0d791f8b80 1 -- 172.21.15.39:0/1992993188 shutdown_connections 2023-07-21T18:55:16.396 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.382+0000 7f0d791f8b80 1 --2- 172.21.15.39:0/1992993188 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eab1516be0 0x55eab1523490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:16.396 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.382+0000 7f0d791f8b80 1 --2- 172.21.15.39:0/1992993188 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eab13726b0 0x55eab15260d0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:16.396 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.382+0000 7f0d791f8b80 1 --2- 172.21.15.39:0/1992993188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eab1516770 0x55eab1528250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:16.396 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.382+0000 7f0d791f8b80 1 -- 172.21.15.39:0/1992993188 >> 172.21.15.39:0/1992993188 conn(0x55eab150bbe0 msgr2=0x55eab1522d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:55:16.396 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.382+0000 7f0d791f8b80 1 -- 172.21.15.39:0/1992993188 shutdown_connections 2023-07-21T18:55:16.396 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.382+0000 7f0d791f8b80 1 -- 172.21.15.39:0/1992993188 wait complete. 2023-07-21T18:55:16.398 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.382+0000 7f0d791f8b80 1 -- start start 2023-07-21T18:55:16.399 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.382+0000 7f0d791f8b80 1 Processor -- start 2023-07-21T18:55:16.399 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.383+0000 7f0d791f8b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eab13726b0 0x55eab152c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:16.399 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.383+0000 7f0d791f8b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eab1516770 0x55eab152c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:16.399 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.383+0000 7f0d791f8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eab1516be0 0x55eab1584d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:16.399 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.383+0000 7f0d791f8b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eab1585290 con 0x55eab1516770 2023-07-21T18:55:16.399 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.383+0000 7f0d791f8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eab1585400 con 0x55eab1516be0 2023-07-21T18:55:16.399 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.383+0000 7f0d791f8b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eab15855c0 con 0x55eab13726b0 2023-07-21T18:55:16.400 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.383+0000 7f0d791f8b80 10 client.0 _ll_register_callbacks cb 0x7f0d60009370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:55:16.400 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.383+0000 7f0d6659c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eab1516770 0x55eab152c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:16.400 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.383+0000 7f0d6759e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eab1516be0 0x55eab1584d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:16.400 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.383+0000 7f0d6659c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eab1516770 0x55eab152c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33126/0 (socket says 172.21.15.39:33126) 2023-07-21T18:55:16.400 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.383+0000 7f0d6759e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eab1516be0 0x55eab1584d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:45724/0 (socket says 172.21.15.39:45724) 2023-07-21T18:55:16.400 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.383+0000 7f0d66d9d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eab13726b0 0x55eab152c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:16.400 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.383+0000 7f0d6659c700 1 -- 172.21.15.39:0/800521612 learned_addr learned my addr 172.21.15.39:0/800521612 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:55:16.400 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.383+0000 7f0d6659c700 1 -- 172.21.15.39:0/800521612 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eab13726b0 msgr2=0x55eab152c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:55:16.401 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.383+0000 7f0d6659c700 1 --2- 172.21.15.39:0/800521612 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eab13726b0 0x55eab152c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:16.401 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.383+0000 7f0d6659c700 1 -- 172.21.15.39:0/800521612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eab1516be0 msgr2=0x55eab1584d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:55:16.401 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.383+0000 7f0d6659c700 1 --2- 172.21.15.39:0/800521612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eab1516be0 0x55eab1584d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:16.401 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.383+0000 7f0d6659c700 1 -- 172.21.15.39:0/800521612 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d5c004040 con 0x55eab1516770 2023-07-21T18:55:16.401 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.383+0000 7f0d66d9d700 1 --2- 172.21.15.39:0/800521612 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eab13726b0 0x55eab152c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:55:16.401 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.383+0000 7f0d6759e700 1 --2- 172.21.15.39:0/800521612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eab1516be0 0x55eab1584d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:55:16.401 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.383+0000 7f0d6659c700 1 --2- 172.21.15.39:0/800521612 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eab1516770 0x55eab152c5a0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x55eab153abe0 tx=0x7f0d5c005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f4ee6b36cdef957 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:55:16.402 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.383+0000 7f0d4aff5700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:55:16.402 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.384+0000 7f0d4aff5700 1 -- 172.21.15.39:0/800521612 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d5c017070 con 0x55eab1516770 2023-07-21T18:55:16.402 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.384+0000 7f0d4aff5700 1 -- 172.21.15.39:0/800521612 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f0d5c0122e0 con 0x55eab1516770 2023-07-21T18:55:16.402 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.384+0000 7f0d791f8b80 1 -- 172.21.15.39:0/800521612 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0d5801e040 con 0x55eab1516770 2023-07-21T18:55:16.402 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.384+0000 7f0d4aff5700 1 -- 172.21.15.39:0/800521612 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d5c021600 con 0x55eab1516770 2023-07-21T18:55:16.402 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.384+0000 7f0d791f8b80 10 client.17199 fetch_fsmap learned FSMap version 81 2023-07-21T18:55:16.402 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.384+0000 7f0d791f8b80 1 -- 172.21.15.39:0/800521612 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) v3 -- 0x7f0d6810c160 con 0x55eab1516770 2023-07-21T18:55:16.402 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.384+0000 7f0d4aff5700 1 -- 172.21.15.39:0/800521612 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d5801e040 con 0x55eab1516770 2023-07-21T18:55:16.403 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.384+0000 7f0d4aff5700 1 -- 172.21.15.39:0/800521612 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 81) v1 ==== 7510+0+0 (secure 0 0 0) 0x7f0d5c012530 con 0x55eab1516770 2023-07-21T18:55:16.403 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.384+0000 7f0d791f8b80 10 client.17199 fetch_fsmap finished waiting for FSMap version 81 2023-07-21T18:55:16.403 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.384+0000 7f0d791f8b80 10 client.17199 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:55:16.403 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.384+0000 7f0d791f8b80 20 client.17199 populate_metadata read hostname 'smithi039' 2023-07-21T18:55:16.403 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.384+0000 7f0d791f8b80 1 --2- 172.21.15.39:0/800521612 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55eab158ce70 0x55eab158f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:16.403 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.384+0000 7f0d791f8b80 4 client.17199 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:55:16.403 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.384+0000 7f0d791f8b80 1 -- 172.21.15.39:0/800521612 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55eab1422ff0 con 0x55eab158ce70 2023-07-21T18:55:16.403 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.384+0000 7f0d66d9d700 1 --2- 172.21.15.39:0/800521612 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55eab158ce70 0x55eab158f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:16.404 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.385+0000 7f0d4aff5700 10 client.17199 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:55:16.404 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:16.385+0000 7f0d66d9d700 1 --2- 172.21.15.39:0/800521612 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55eab158ce70 0x55eab158f280 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28e16539a7f45c5a server_cookie=ce33dc94a4e1246d in_seq=0 out_seq=0 2023-07-21T18:55:16.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:55:16 smithi193 ceph-mon[98204]: pgmap v425: 97 pgs: 97 active+clean; 492 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.2 MiB/s wr, 626 op/s 2023-07-21T18:55:16.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:55:16 smithi155 ceph-mon[97934]: pgmap v425: 97 pgs: 97 active+clean; 492 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.2 MiB/s wr, 626 op/s 2023-07-21T18:55:16.653 INFO:tasks.workunit.client.0.smithi039.stdout: 10 9684 23.98 MB/sec warmup 29 sec latency 2448.082 ms 2023-07-21T18:55:16.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:55:16 smithi039 ceph-mon[96489]: pgmap v425: 97 pgs: 97 active+clean; 492 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.2 MiB/s wr, 626 op/s 2023-07-21T18:55:17.653 INFO:tasks.workunit.client.0.smithi039.stdout: 10 10018 24.13 MB/sec warmup 30 sec latency 3448.114 ms 2023-07-21T18:55:18.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:55:18 smithi193 ceph-mon[98204]: pgmap v426: 97 pgs: 97 active+clean; 504 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 238 op/s 2023-07-21T18:55:18.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:55:18 smithi155 ceph-mon[97934]: pgmap v426: 97 pgs: 97 active+clean; 504 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 238 op/s 2023-07-21T18:55:18.653 INFO:tasks.workunit.client.0.smithi039.stdout: 10 10172 23.63 MB/sec warmup 31 sec latency 4448.163 ms 2023-07-21T18:55:18.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:55:18 smithi039 ceph-mon[96489]: pgmap v426: 97 pgs: 97 active+clean; 504 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 238 op/s 2023-07-21T18:55:19.653 INFO:tasks.workunit.client.0.smithi039.stdout: 10 10354 23.07 MB/sec warmup 32 sec latency 5448.216 ms 2023-07-21T18:55:20.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:55:20 smithi193 ceph-mon[98204]: pgmap v427: 97 pgs: 97 active+clean; 504 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 238 op/s 2023-07-21T18:55:20.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:55:20 smithi155 ceph-mon[97934]: pgmap v427: 97 pgs: 97 active+clean; 504 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 238 op/s 2023-07-21T18:55:20.653 INFO:tasks.workunit.client.0.smithi039.stdout: 10 10561 22.55 MB/sec warmup 33 sec latency 6448.298 ms 2023-07-21T18:55:20.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:55:20 smithi039 ceph-mon[96489]: pgmap v427: 97 pgs: 97 active+clean; 504 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 238 op/s 2023-07-21T18:55:21.654 INFO:tasks.workunit.client.0.smithi039.stdout: 10 10893 22.31 MB/sec warmup 34 sec latency 7448.329 ms 2023-07-21T18:55:22.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:55:22 smithi193 ceph-mon[98204]: pgmap v428: 97 pgs: 97 active+clean; 512 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 435 op/s 2023-07-21T18:55:22.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:55:22 smithi155 ceph-mon[97934]: pgmap v428: 97 pgs: 97 active+clean; 512 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 435 op/s 2023-07-21T18:55:22.653 INFO:tasks.workunit.client.0.smithi039.stdout: 10 11420 22.27 MB/sec warmup 35 sec latency 8448.358 ms 2023-07-21T18:55:22.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:55:22 smithi039 ceph-mon[96489]: pgmap v428: 97 pgs: 97 active+clean; 512 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 435 op/s 2023-07-21T18:55:23.512 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:23.501+0000 7f0d4aff5700 1 -- 172.21.15.39:0/800521612 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55eab1422ff0 con 0x55eab158ce70 2023-07-21T18:55:23.513 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:23.501+0000 7f0d4aff5700 10 client.17199 handle_command_reply: tid=0 2023-07-21T18:55:23.513 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:23.501+0000 7f0d791f8b80 1 client.17199 shutdown 2023-07-21T18:55:23.513 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:23.502+0000 7f0d791f8b80 1 -- 172.21.15.39:0/800521612 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eab1516770 msgr2=0x55eab152c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:55:23.514 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:23.502+0000 7f0d791f8b80 1 --2- 172.21.15.39:0/800521612 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eab1516770 0x55eab152c5a0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x55eab153abe0 tx=0x7f0d5c005e40 comp rx=0 tx=0).stop 2023-07-21T18:55:23.514 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:23.502+0000 7f0d791f8b80 1 -- 172.21.15.39:0/800521612 shutdown_connections 2023-07-21T18:55:23.514 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:23.502+0000 7f0d791f8b80 1 --2- 172.21.15.39:0/800521612 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55eab158ce70 0x55eab158f280 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:23.514 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:23.502+0000 7f0d791f8b80 1 --2- 172.21.15.39:0/800521612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eab1516be0 0x55eab1584d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:23.514 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:23.502+0000 7f0d791f8b80 1 --2- 172.21.15.39:0/800521612 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eab13726b0 0x55eab152c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:23.514 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:23.502+0000 7f0d791f8b80 1 --2- 172.21.15.39:0/800521612 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eab1516770 0x55eab152c5a0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:23.514 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:23.502+0000 7f0d791f8b80 1 -- 172.21.15.39:0/800521612 >> 172.21.15.39:0/800521612 conn(0x55eab150bbe0 msgr2=0x55eab15260d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:55:23.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:23.502+0000 7f0d4aff5700 0 client.17199 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:55:23.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:23.502+0000 7f0d791f8b80 1 -- 172.21.15.39:0/800521612 shutdown_connections 2023-07-21T18:55:23.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:23.502+0000 7f0d791f8b80 1 -- 172.21.15.39:0/800521612 wait complete. 2023-07-21T18:55:23.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:23.502+0000 7f0d791f8b80 20 client.17199 trim_cache size 0 max 16384 2023-07-21T18:55:23.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:55:23.516 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:55:23.516 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:55:23.516 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:55:23.516 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:23.503+0000 7f0d6d072700 1 -- 172.21.15.39:0/840789386 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f0d500a12c0 msgr2=0x7f0d500a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:55:23.517 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:23.503+0000 7f0d6d072700 1 --2- 172.21.15.39:0/840789386 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f0d500a12c0 0x7f0d500a3770 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0d5800b2e0 tx=0x7f0d58005fd0 comp rx=0 tx=0).stop 2023-07-21T18:55:23.517 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:23.503+0000 7f0d6d072700 1 -- 172.21.15.39:0/840789386 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d68104a30 msgr2=0x7f0d68194930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:55:23.517 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:23.503+0000 7f0d6d072700 1 --2- 172.21.15.39:0/840789386 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d68104a30 0x7f0d68194930 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f0d6000efc0 tx=0x7f0d6000c3a0 comp rx=0 tx=0).stop 2023-07-21T18:55:23.517 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:23.503+0000 7f0d6d072700 1 -- 172.21.15.39:0/840789386 shutdown_connections 2023-07-21T18:55:23.517 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:23.503+0000 7f0d6d072700 1 --2- 172.21.15.39:0/840789386 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d68104a30 0x7f0d68194930 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:23.517 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:23.503+0000 7f0d6d072700 1 --2- 172.21.15.39:0/840789386 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d680fea30 0x7f0d6819aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:23.517 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:23.503+0000 7f0d6d072700 1 --2- 172.21.15.39:0/840789386 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f0d500a12c0 0x7f0d500a3770 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:23.518 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:23.503+0000 7f0d6d072700 1 --2- 172.21.15.39:0/840789386 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d680ff3e0 0x7f0d6819b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:23.518 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:23.503+0000 7f0d6d072700 1 -- 172.21.15.39:0/840789386 >> 172.21.15.39:0/840789386 conn(0x7f0d68074760 msgr2=0x7f0d68102650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:55:23.518 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:23.504+0000 7f0d6d072700 1 -- 172.21.15.39:0/840789386 shutdown_connections 2023-07-21T18:55:23.518 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:23.504+0000 7f0d6d072700 1 -- 172.21.15.39:0/840789386 wait complete. 2023-07-21T18:55:23.653 INFO:tasks.workunit.client.0.smithi039.stdout: 10 11823 22.23 MB/sec warmup 36 sec latency 9399.990 ms 2023-07-21T18:55:24.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:55:24 smithi193 ceph-mon[98204]: pgmap v429: 97 pgs: 97 active+clean; 512 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 385 op/s 2023-07-21T18:55:24.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:55:24 smithi155 ceph-mon[97934]: pgmap v429: 97 pgs: 97 active+clean; 512 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 385 op/s 2023-07-21T18:55:24.653 INFO:tasks.workunit.client.0.smithi039.stdout: 10 12516 23.00 MB/sec warmup 37 sec latency 69.201 ms 2023-07-21T18:55:24.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:55:24 smithi039 ceph-mon[96489]: pgmap v429: 97 pgs: 97 active+clean; 512 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 385 op/s 2023-07-21T18:55:25.654 INFO:tasks.workunit.client.0.smithi039.stdout: 10 13000 23.40 MB/sec warmup 38 sec latency 69.338 ms 2023-07-21T18:55:26.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:55:26 smithi193 ceph-mon[98204]: pgmap v430: 97 pgs: 97 active+clean; 512 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 362 op/s 2023-07-21T18:55:26.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:55:26 smithi155 ceph-mon[97934]: pgmap v430: 97 pgs: 97 active+clean; 512 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 362 op/s 2023-07-21T18:55:26.654 INFO:tasks.workunit.client.0.smithi039.stdout: 10 13266 23.11 MB/sec warmup 39 sec latency 94.624 ms 2023-07-21T18:55:26.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:55:26 smithi039 ceph-mon[96489]: pgmap v430: 97 pgs: 97 active+clean; 512 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 362 op/s 2023-07-21T18:55:27.654 INFO:tasks.workunit.client.0.smithi039.stdout: 10 13593 22.65 MB/sec warmup 40 sec latency 67.137 ms 2023-07-21T18:55:28.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:55:28 smithi193 ceph-mon[98204]: pgmap v431: 97 pgs: 97 active+clean; 510 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 487 op/s 2023-07-21T18:55:28.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:55:28 smithi155 ceph-mon[97934]: pgmap v431: 97 pgs: 97 active+clean; 510 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 487 op/s 2023-07-21T18:55:28.654 INFO:tasks.workunit.client.0.smithi039.stdout: 10 14027 22.63 MB/sec warmup 41 sec latency 127.389 ms 2023-07-21T18:55:28.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:55:28 smithi039 ceph-mon[96489]: pgmap v431: 97 pgs: 97 active+clean; 510 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 487 op/s 2023-07-21T18:55:29.654 INFO:tasks.workunit.client.0.smithi039.stdout: 10 14774 22.89 MB/sec warmup 42 sec latency 151.354 ms 2023-07-21T18:55:30.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:55:30 smithi193 ceph-mon[98204]: pgmap v432: 97 pgs: 97 active+clean; 510 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 377 op/s 2023-07-21T18:55:30.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:55:30 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:55:30.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:55:30 smithi155 ceph-mon[97934]: pgmap v432: 97 pgs: 97 active+clean; 510 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 377 op/s 2023-07-21T18:55:30.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:55:30 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:55:30.654 INFO:tasks.workunit.client.0.smithi039.stdout: 10 15711 23.72 MB/sec warmup 43 sec latency 110.535 ms 2023-07-21T18:55:30.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:55:30 smithi039 ceph-mon[96489]: pgmap v432: 97 pgs: 97 active+clean; 510 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 377 op/s 2023-07-21T18:55:30.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:55:30 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:55:31.654 INFO:tasks.workunit.client.0.smithi039.stdout: 10 16422 24.32 MB/sec warmup 44 sec latency 57.252 ms 2023-07-21T18:55:32.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:55:32 smithi193 ceph-mon[98204]: pgmap v433: 97 pgs: 97 active+clean; 542 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.0 MiB/s wr, 555 op/s 2023-07-21T18:55:32.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:55:32 smithi155 ceph-mon[97934]: pgmap v433: 97 pgs: 97 active+clean; 542 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.0 MiB/s wr, 555 op/s 2023-07-21T18:55:32.654 INFO:tasks.workunit.client.0.smithi039.stdout: 10 16986 24.31 MB/sec warmup 45 sec latency 65.368 ms 2023-07-21T18:55:32.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:55:32 smithi039 ceph-mon[96489]: pgmap v433: 97 pgs: 97 active+clean; 542 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.0 MiB/s wr, 555 op/s 2023-07-21T18:55:33.654 INFO:tasks.workunit.client.0.smithi039.stdout: 10 17707 24.40 MB/sec warmup 46 sec latency 96.983 ms 2023-07-21T18:55:34.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:55:34 smithi193 ceph-mon[98204]: pgmap v434: 97 pgs: 97 active+clean; 542 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.1 MiB/s wr, 301 op/s 2023-07-21T18:55:34.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:55:34 smithi155 ceph-mon[97934]: pgmap v434: 97 pgs: 97 active+clean; 542 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.1 MiB/s wr, 301 op/s 2023-07-21T18:55:34.655 INFO:tasks.workunit.client.0.smithi039.stdout: 10 18646 25.03 MB/sec warmup 47 sec latency 79.880 ms 2023-07-21T18:55:34.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:55:34 smithi039 ceph-mon[96489]: pgmap v434: 97 pgs: 97 active+clean; 542 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.1 MiB/s wr, 301 op/s 2023-07-21T18:55:35.654 INFO:tasks.workunit.client.0.smithi039.stdout: 10 19353 25.37 MB/sec warmup 48 sec latency 437.056 ms 2023-07-21T18:55:36.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:55:36 smithi193 ceph-mon[98204]: pgmap v435: 97 pgs: 97 active+clean; 542 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.1 MiB/s wr, 301 op/s 2023-07-21T18:55:36.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:55:36 smithi155 ceph-mon[97934]: pgmap v435: 97 pgs: 97 active+clean; 542 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.1 MiB/s wr, 301 op/s 2023-07-21T18:55:36.654 INFO:tasks.workunit.client.0.smithi039.stdout: 10 19950 25.54 MB/sec warmup 49 sec latency 1437.090 ms 2023-07-21T18:55:36.655 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.643+0000 7fab1f33a700 1 -- 172.21.15.39:0/10480545 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab181039b0 msgr2=0x7fab18103d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:55:36.655 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.643+0000 7fab1f33a700 1 --2- 172.21.15.39:0/10480545 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab181039b0 0x7fab18103d80 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fab140099c0 tx=0x7fab14009cd0 comp rx=0 tx=0).stop 2023-07-21T18:55:36.655 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.644+0000 7fab1f33a700 1 -- 172.21.15.39:0/10480545 shutdown_connections 2023-07-21T18:55:36.655 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.644+0000 7fab1f33a700 1 --2- 172.21.15.39:0/10480545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab180ffab0 0x7fab1810b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:36.656 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.644+0000 7fab1f33a700 1 --2- 172.21.15.39:0/10480545 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab181039b0 0x7fab18103d80 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:36.656 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.644+0000 7fab1f33a700 1 --2- 172.21.15.39:0/10480545 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fab180ff100 0x7fab180ff570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:36.656 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.644+0000 7fab1f33a700 1 -- 172.21.15.39:0/10480545 >> 172.21.15.39:0/10480545 conn(0x7fab18075f90 msgr2=0x7fab18076390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:55:36.656 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.645+0000 7fab1f33a700 1 -- 172.21.15.39:0/10480545 shutdown_connections 2023-07-21T18:55:36.657 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.645+0000 7fab1f33a700 1 -- 172.21.15.39:0/10480545 wait complete. 2023-07-21T18:55:36.657 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.646+0000 7fab1f33a700 1 Processor -- start 2023-07-21T18:55:36.657 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.646+0000 7fab1f33a700 1 -- start start 2023-07-21T18:55:36.658 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.646+0000 7fab1f33a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab180ff100 0x7fab18199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:36.658 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.646+0000 7fab1f33a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab180ffab0 0x7fab18199e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:36.658 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.646+0000 7fab1f33a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fab181039b0 0x7fab18193790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:36.658 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.646+0000 7fab1f33a700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab18193e00 con 0x7fab180ffab0 2023-07-21T18:55:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.646+0000 7fab1f33a700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab18193f70 con 0x7fab180ff100 2023-07-21T18:55:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.646+0000 7fab1f33a700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab18194130 con 0x7fab181039b0 2023-07-21T18:55:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.647+0000 7fab1c8d5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab180ffab0 0x7fab18199e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.647+0000 7fab1d0d6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab180ff100 0x7fab18199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.647+0000 7fab1c8d5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab180ffab0 0x7fab18199e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33140/0 (socket says 172.21.15.39:33140) 2023-07-21T18:55:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.647+0000 7fab1c8d5700 1 -- 172.21.15.39:0/3776640679 learned_addr learned my addr 172.21.15.39:0/3776640679 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:55:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.647+0000 7fab1d8d7700 1 --2- 172.21.15.39:0/3776640679 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fab181039b0 0x7fab18193790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:36.660 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.647+0000 7fab1c8d5700 1 -- 172.21.15.39:0/3776640679 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fab181039b0 msgr2=0x7fab18193790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:55:36.660 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.647+0000 7fab1c8d5700 1 --2- 172.21.15.39:0/3776640679 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fab181039b0 0x7fab18193790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:36.660 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.647+0000 7fab1c8d5700 1 -- 172.21.15.39:0/3776640679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab180ff100 msgr2=0x7fab18199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:55:36.660 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.647+0000 7fab1c8d5700 1 --2- 172.21.15.39:0/3776640679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab180ff100 0x7fab18199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:36.660 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.647+0000 7fab1c8d5700 1 -- 172.21.15.39:0/3776640679 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab140095f0 con 0x7fab180ffab0 2023-07-21T18:55:36.660 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.647+0000 7fab1c8d5700 1 --2- 172.21.15.39:0/3776640679 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab180ffab0 0x7fab18199e70 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fab0800d950 tx=0x7fab0800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:55:36.661 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.648+0000 7fab0e7fc700 1 -- 172.21.15.39:0/3776640679 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0800f890 con 0x7fab180ffab0 2023-07-21T18:55:36.661 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.648+0000 7fab0e7fc700 1 -- 172.21.15.39:0/3776640679 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab08010460 con 0x7fab180ffab0 2023-07-21T18:55:36.661 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.648+0000 7fab1f33a700 1 -- 172.21.15.39:0/3776640679 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab18194410 con 0x7fab180ffab0 2023-07-21T18:55:36.661 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.648+0000 7fab0e7fc700 1 -- 172.21.15.39:0/3776640679 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0800e610 con 0x7fab180ffab0 2023-07-21T18:55:36.661 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.648+0000 7fab1f33a700 1 -- 172.21.15.39:0/3776640679 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab181a0920 con 0x7fab180ffab0 2023-07-21T18:55:36.662 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.651+0000 7fab0e7fc700 1 -- 172.21.15.39:0/3776640679 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fab080108d0 con 0x7fab180ffab0 2023-07-21T18:55:36.662 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.651+0000 7fab0e7fc700 1 --2- 172.21.15.39:0/3776640679 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fab040a12b0 0x7fab040a3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:36.662 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.651+0000 7fab0e7fc700 1 -- 172.21.15.39:0/3776640679 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fab080ccb00 con 0x7fab180ffab0 2023-07-21T18:55:36.663 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.652+0000 7fab1d0d6700 1 --2- 172.21.15.39:0/3776640679 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fab040a12b0 0x7fab040a3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:36.663 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.652+0000 7fab1d0d6700 1 --2- 172.21.15.39:0/3776640679 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fab040a12b0 0x7fab040a3760 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fab14009990 tx=0x7fab14005fb0 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:55:36.664 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.653+0000 7fab2b4c0b80 1 Processor -- start 2023-07-21T18:55:36.664 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.653+0000 7fab2b4c0b80 1 -- start start 2023-07-21T18:55:36.664 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.653+0000 7fab2b4c0b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bbad0f6b0 0x564bbaeb8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:36.664 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.653+0000 7fab2b4c0b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564bbaeb92d0 0x564bbaeb3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:36.664 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.653+0000 7fab2b4c0b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bbaeb3580 0x564bbaeb39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:36.665 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.653+0000 7fab2b4c0b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bbadb8c50 con 0x564bbaeb3580 2023-07-21T18:55:36.665 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.653+0000 7fab2b4c0b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bbae0b9e0 con 0x564bbad0f6b0 2023-07-21T18:55:36.665 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.653+0000 7fab2b4c0b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bbae04e70 con 0x564bbaeb92d0 2023-07-21T18:55:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.654+0000 7fab1c8d5700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564bbaeb92d0 0x564bbaeb3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.654+0000 7fab1d0d6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bbad0f6b0 0x564bbaeb8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:36.667 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.654+0000 7fab1d0d6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bbad0f6b0 0x564bbaeb8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:45742/0 (socket says 172.21.15.39:45742) 2023-07-21T18:55:36.667 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.654+0000 7fab1c8d5700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564bbaeb92d0 0x564bbaeb3040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:51700/0 (socket says 172.21.15.39:51700) 2023-07-21T18:55:36.667 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.654+0000 7fab1c8d5700 1 -- 172.21.15.39:0/1250598701 learned_addr learned my addr 172.21.15.39:0/1250598701 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:55:36.667 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.654+0000 7fab1d8d7700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bbaeb3580 0x564bbaeb39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:36.667 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.654+0000 7fab1c8d5700 1 -- 172.21.15.39:0/1250598701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bbad0f6b0 msgr2=0x564bbaeb8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:55:36.667 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.654+0000 7fab1c8d5700 1 --2- 172.21.15.39:0/1250598701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bbad0f6b0 0x564bbaeb8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:36.667 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.654+0000 7fab1c8d5700 1 -- 172.21.15.39:0/1250598701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bbaeb3580 msgr2=0x564bbaeb39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:55:36.667 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.654+0000 7fab1c8d5700 1 --2- 172.21.15.39:0/1250598701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bbaeb3580 0x564bbaeb39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:36.668 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.654+0000 7fab1c8d5700 1 -- 172.21.15.39:0/1250598701 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564bbac32d80 con 0x564bbaeb92d0 2023-07-21T18:55:36.668 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.654+0000 7fab1c8d5700 1 --2- 172.21.15.39:0/1250598701 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564bbaeb92d0 0x564bbaeb3040 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fab080960d0 tx=0x7fab08099c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=211213484f181008 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:55:36.668 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.654+0000 7fab037fe700 1 -- 172.21.15.39:0/1250598701 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab080962e0 con 0x564bbaeb92d0 2023-07-21T18:55:36.668 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.654+0000 7fab037fe700 1 -- 172.21.15.39:0/1250598701 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab08096440 con 0x564bbaeb92d0 2023-07-21T18:55:36.668 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.654+0000 7fab037fe700 1 -- 172.21.15.39:0/1250598701 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab080a1b90 con 0x564bbaeb92d0 2023-07-21T18:55:36.668 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.655+0000 7fab2b4c0b80 1 -- 172.21.15.39:0/1250598701 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564bbaeb92d0 msgr2=0x564bbaeb3040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:55:36.668 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.655+0000 7fab2b4c0b80 1 --2- 172.21.15.39:0/1250598701 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564bbaeb92d0 0x564bbaeb3040 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fab080960d0 tx=0x7fab08099c50 comp rx=0 tx=0).stop 2023-07-21T18:55:36.669 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.655+0000 7fab2b4c0b80 1 -- 172.21.15.39:0/1250598701 shutdown_connections 2023-07-21T18:55:36.669 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.655+0000 7fab2b4c0b80 1 --2- 172.21.15.39:0/1250598701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bbad0f6b0 0x564bbaeb8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:36.669 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.655+0000 7fab2b4c0b80 1 --2- 172.21.15.39:0/1250598701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bbaeb3580 0x564bbaeb39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:36.669 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.655+0000 7fab2b4c0b80 1 --2- 172.21.15.39:0/1250598701 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564bbaeb92d0 0x564bbaeb3040 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:36.669 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.655+0000 7fab2b4c0b80 1 -- 172.21.15.39:0/1250598701 >> 172.21.15.39:0/1250598701 conn(0x564bbaea8be0 msgr2=0x564bbaeadf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:55:36.669 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.655+0000 7fab2b4c0b80 1 -- 172.21.15.39:0/1250598701 shutdown_connections 2023-07-21T18:55:36.670 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.655+0000 7fab2b4c0b80 1 -- 172.21.15.39:0/1250598701 wait complete. 2023-07-21T18:55:36.670 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.656+0000 7fab2b4c0b80 1 -- start start 2023-07-21T18:55:36.670 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.656+0000 7fab2b4c0b80 1 Processor -- start 2023-07-21T18:55:36.672 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.656+0000 7fab2b4c0b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bbad0f6b0 0x564bbaec2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:36.672 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.656+0000 7fab2b4c0b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bbaeb3580 0x564bbaec2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:36.672 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.656+0000 7fab2b4c0b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564bbaeb92d0 0x564bbaebc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:36.672 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.656+0000 7fab2b4c0b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bbada4a60 con 0x564bbad0f6b0 2023-07-21T18:55:36.673 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.656+0000 7fab2b4c0b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bbada4760 con 0x564bbaeb3580 2023-07-21T18:55:36.673 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.656+0000 7fab2b4c0b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bbac888c0 con 0x564bbaeb92d0 2023-07-21T18:55:36.673 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.656+0000 7fab2b4c0b80 10 client.0 _ll_register_callbacks cb 0x564bbada2700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:55:36.673 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.656+0000 7fab1c8d5700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bbaeb3580 0x564bbaec2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:36.673 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.656+0000 7fab1d0d6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bbad0f6b0 0x564bbaec2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:36.673 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.656+0000 7fab1d8d7700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564bbaeb92d0 0x564bbaebc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:36.673 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.656+0000 7fab1c8d5700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bbaeb3580 0x564bbaec2ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:45750/0 (socket says 172.21.15.39:45750) 2023-07-21T18:55:36.673 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.656+0000 7fab1c8d5700 1 -- 172.21.15.39:0/3447456706 learned_addr learned my addr 172.21.15.39:0/3447456706 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:55:36.674 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.656+0000 7fab1d0d6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bbad0f6b0 0x564bbaec2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33152/0 (socket says 172.21.15.39:33152) 2023-07-21T18:55:36.674 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.657+0000 7fab1c8d5700 1 -- 172.21.15.39:0/3447456706 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564bbaeb92d0 msgr2=0x564bbaebc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:55:36.674 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.657+0000 7fab1c8d5700 1 --2- 172.21.15.39:0/3447456706 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564bbaeb92d0 0x564bbaebc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:36.674 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.657+0000 7fab1c8d5700 1 -- 172.21.15.39:0/3447456706 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bbad0f6b0 msgr2=0x564bbaec2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:55:36.674 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.657+0000 7fab1c8d5700 1 --2- 172.21.15.39:0/3447456706 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bbad0f6b0 0x564bbaec2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:36.674 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.657+0000 7fab1c8d5700 1 -- 172.21.15.39:0/3447456706 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab181a0920 con 0x564bbaeb3580 2023-07-21T18:55:36.674 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.657+0000 7fab1d8d7700 1 --2- 172.21.15.39:0/3447456706 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564bbaeb92d0 0x564bbaebc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:55:36.675 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.657+0000 7fab1d0d6700 1 --2- 172.21.15.39:0/3447456706 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bbad0f6b0 0x564bbaec2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:55:36.675 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.657+0000 7fab1c8d5700 1 --2- 172.21.15.39:0/3447456706 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bbaeb3580 0x564bbaec2ba0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fab080a1af0 tx=0x7fab0809a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b17ef499c5e045c6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:55:36.675 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.657+0000 7fab00ff9700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-21T18:55:36.675 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.657+0000 7fab00ff9700 1 -- 172.21.15.39:0/3447456706 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab080962e0 con 0x564bbaeb3580 2023-07-21T18:55:36.675 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.657+0000 7fab2b4c0b80 1 -- 172.21.15.39:0/3447456706 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564bbad5e490 con 0x564bbaeb3580 2023-07-21T18:55:36.675 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.658+0000 7fab00ff9700 1 -- 172.21.15.39:0/3447456706 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fab080990c0 con 0x564bbaeb3580 2023-07-21T18:55:36.675 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.658+0000 7fab00ff9700 1 -- 172.21.15.39:0/3447456706 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab080b23b0 con 0x564bbaeb3580 2023-07-21T18:55:36.675 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.658+0000 7fab2b4c0b80 10 client.26660 fetch_fsmap learned FSMap version 82 2023-07-21T18:55:36.676 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.658+0000 7fab2b4c0b80 1 -- 172.21.15.39:0/3447456706 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=4+}) v3 -- 0x564bbac1bc00 con 0x564bbaeb3580 2023-07-21T18:55:36.676 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.659+0000 7fab00ff9700 1 -- 172.21.15.39:0/3447456706 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564bbad5e490 con 0x564bbaeb3580 2023-07-21T18:55:36.676 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.659+0000 7fab00ff9700 1 -- 172.21.15.39:0/3447456706 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 82) v1 ==== 7522+0+0 (secure 0 0 0) 0x7fab08098b30 con 0x564bbaeb3580 2023-07-21T18:55:36.676 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.659+0000 7fab2b4c0b80 10 client.26660 fetch_fsmap finished waiting for FSMap version 82 2023-07-21T18:55:36.676 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.659+0000 7fab2b4c0b80 10 client.26660 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:55:36.676 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.659+0000 7fab2b4c0b80 20 client.26660 populate_metadata read hostname 'smithi039' 2023-07-21T18:55:36.676 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.659+0000 7fab2b4c0b80 1 --2- 172.21.15.39:0/3447456706 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x564bbaecc780 0x564bbaeceb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:36.677 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.659+0000 7fab2b4c0b80 4 client.26660 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:55:36.677 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.659+0000 7fab2b4c0b80 1 -- 172.21.15.39:0/3447456706 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564bbadfde50 con 0x564bbaecc780 2023-07-21T18:55:36.677 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.660+0000 7fab1d0d6700 1 --2- 172.21.15.39:0/3447456706 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x564bbaecc780 0x564bbaeceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:36.677 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.661+0000 7fab1d0d6700 1 --2- 172.21.15.39:0/3447456706 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x564bbaecc780 0x564bbaeceb90 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8525f3de68c56c4a server_cookie=b644e519f1294fab in_seq=0 out_seq=0 2023-07-21T18:55:36.677 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.661+0000 7fab00ff9700 10 client.26660 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:55:36.678 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.663+0000 7fab00ff9700 1 -- 172.21.15.39:0/3447456706 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564bbadfde50 con 0x564bbaecc780 2023-07-21T18:55:36.678 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.664+0000 7fab00ff9700 10 client.26660 handle_command_reply: tid=0 2023-07-21T18:55:36.678 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.664+0000 7fab2b4c0b80 1 client.26660 shutdown 2023-07-21T18:55:36.678 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.664+0000 7fab2b4c0b80 1 -- 172.21.15.39:0/3447456706 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bbaeb3580 msgr2=0x564bbaec2ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:55:36.679 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.664+0000 7fab2b4c0b80 1 --2- 172.21.15.39:0/3447456706 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bbaeb3580 0x564bbaec2ba0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fab080a1af0 tx=0x7fab0809a060 comp rx=0 tx=0).stop 2023-07-21T18:55:36.679 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.664+0000 7fab2b4c0b80 1 -- 172.21.15.39:0/3447456706 shutdown_connections 2023-07-21T18:55:36.679 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.664+0000 7fab2b4c0b80 1 --2- 172.21.15.39:0/3447456706 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x564bbaecc780 0x564bbaeceb90 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:36.679 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.664+0000 7fab2b4c0b80 1 --2- 172.21.15.39:0/3447456706 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564bbaeb92d0 0x564bbaebc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:36.679 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.664+0000 7fab00ff9700 0 client.26660 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:55:36.679 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.664+0000 7fab2b4c0b80 1 --2- 172.21.15.39:0/3447456706 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bbad0f6b0 0x564bbaec2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:36.679 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.664+0000 7fab2b4c0b80 1 --2- 172.21.15.39:0/3447456706 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bbaeb3580 0x564bbaec2ba0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:36.679 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.664+0000 7fab2b4c0b80 1 -- 172.21.15.39:0/3447456706 >> 172.21.15.39:0/3447456706 conn(0x564bbaea8be0 msgr2=0x564bbaead4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:55:36.680 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.664+0000 7fab2b4c0b80 1 -- 172.21.15.39:0/3447456706 shutdown_connections 2023-07-21T18:55:36.680 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.664+0000 7fab2b4c0b80 1 -- 172.21.15.39:0/3447456706 wait complete. 2023-07-21T18:55:36.680 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.665+0000 7fab2b4c0b80 20 client.26660 trim_cache size 0 max 16384 2023-07-21T18:55:36.680 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:55:36 smithi039 ceph-mon[96489]: pgmap v435: 97 pgs: 97 active+clean; 542 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.1 MiB/s wr, 301 op/s 2023-07-21T18:55:36.698 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.687+0000 7fab2b4c0b80 1 Processor -- start 2023-07-21T18:55:36.698 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.687+0000 7fab2b4c0b80 1 -- start start 2023-07-21T18:55:36.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.688+0000 7fab2b4c0b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bbad0f6b0 0x564bbaebbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:36.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.688+0000 7fab2b4c0b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564bbaeb3580 0x564bbaeb83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:36.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.688+0000 7fab2b4c0b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bbaeb92d0 0x564bbaeadc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:36.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.688+0000 7fab2b4c0b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bbaeb8a10 con 0x564bbaeb92d0 2023-07-21T18:55:36.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.688+0000 7fab2b4c0b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bbaeb8fe0 con 0x564bbad0f6b0 2023-07-21T18:55:36.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.688+0000 7fab2b4c0b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bbaeaad00 con 0x564bbaeb3580 2023-07-21T18:55:36.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.688+0000 7fab1d0d6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bbad0f6b0 0x564bbaebbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:36.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.688+0000 7fab1d0d6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bbad0f6b0 0x564bbaebbbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:45756/0 (socket says 172.21.15.39:45756) 2023-07-21T18:55:36.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.688+0000 7fab1c8d5700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564bbaeb3580 0x564bbaeb83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:36.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.688+0000 7fab1d0d6700 1 -- 172.21.15.39:0/50023649 learned_addr learned my addr 172.21.15.39:0/50023649 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:55:36.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.688+0000 7fab1d8d7700 1 --2- 172.21.15.39:0/50023649 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bbaeb92d0 0x564bbaeadc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:36.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.688+0000 7fab1d0d6700 1 -- 172.21.15.39:0/50023649 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564bbaeb3580 msgr2=0x564bbaeb83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:55:36.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.688+0000 7fab1d0d6700 1 --2- 172.21.15.39:0/50023649 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564bbaeb3580 0x564bbaeb83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:36.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.688+0000 7fab1d0d6700 1 -- 172.21.15.39:0/50023649 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bbaeb92d0 msgr2=0x564bbaeadc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:55:36.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.688+0000 7fab1d0d6700 1 --2- 172.21.15.39:0/50023649 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bbaeb92d0 0x564bbaeadc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:36.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.688+0000 7fab1d0d6700 1 -- 172.21.15.39:0/50023649 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564bbac591f0 con 0x564bbad0f6b0 2023-07-21T18:55:36.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.689+0000 7fab1c8d5700 1 --2- 172.21.15.39:0/50023649 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564bbaeb3580 0x564bbaeb83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:55:36.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.689+0000 7fab1d0d6700 1 --2- 172.21.15.39:0/50023649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bbad0f6b0 0x564bbaebbbf0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fab14005040 tx=0x7fab14025000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efa3c6a294005a6b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:55:36.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.689+0000 7fab03fff700 1 -- 172.21.15.39:0/50023649 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1401bea0 con 0x564bbad0f6b0 2023-07-21T18:55:36.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.689+0000 7fab03fff700 1 -- 172.21.15.39:0/50023649 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab14022040 con 0x564bbad0f6b0 2023-07-21T18:55:36.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.689+0000 7fab03fff700 1 -- 172.21.15.39:0/50023649 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab14003a30 con 0x564bbad0f6b0 2023-07-21T18:55:36.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.690+0000 7fab2b4c0b80 1 -- 172.21.15.39:0/50023649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bbad0f6b0 msgr2=0x564bbaebbbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:55:36.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.690+0000 7fab2b4c0b80 1 --2- 172.21.15.39:0/50023649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bbad0f6b0 0x564bbaebbbf0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fab14005040 tx=0x7fab14025000 comp rx=0 tx=0).stop 2023-07-21T18:55:36.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.690+0000 7fab2b4c0b80 1 -- 172.21.15.39:0/50023649 shutdown_connections 2023-07-21T18:55:36.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.690+0000 7fab2b4c0b80 1 --2- 172.21.15.39:0/50023649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bbad0f6b0 0x564bbaebbbf0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:36.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.690+0000 7fab2b4c0b80 1 --2- 172.21.15.39:0/50023649 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bbaeb92d0 0x564bbaeadc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.690+0000 7fab2b4c0b80 1 --2- 172.21.15.39:0/50023649 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564bbaeb3580 0x564bbaeb83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.690+0000 7fab2b4c0b80 1 -- 172.21.15.39:0/50023649 >> 172.21.15.39:0/50023649 conn(0x564bbaea8be0 msgr2=0x564bbaebc880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:55:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.690+0000 7fab2b4c0b80 1 -- 172.21.15.39:0/50023649 shutdown_connections 2023-07-21T18:55:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.690+0000 7fab2b4c0b80 1 -- 172.21.15.39:0/50023649 wait complete. 2023-07-21T18:55:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.690+0000 7fab2b4c0b80 1 -- start start 2023-07-21T18:55:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.691+0000 7fab2b4c0b80 1 Processor -- start 2023-07-21T18:55:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.691+0000 7fab2b4c0b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bbad0f6b0 0x564bbaf14590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:36.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.691+0000 7fab2b4c0b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bbaeb3580 0x564bbaf14ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:36.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.691+0000 7fab2b4c0b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564bbaeb92d0 0x564bbaf1f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:36.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.691+0000 7fab2b4c0b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bbaf152a0 con 0x564bbad0f6b0 2023-07-21T18:55:36.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.691+0000 7fab2b4c0b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bbaf15410 con 0x564bbaeb3580 2023-07-21T18:55:36.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.691+0000 7fab2b4c0b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bbaf1fb80 con 0x564bbaeb92d0 2023-07-21T18:55:36.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.691+0000 7fab2b4c0b80 10 client.0 _ll_register_callbacks cb 0x7fab10009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:55:36.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.691+0000 7fab1d0d6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bbad0f6b0 0x564bbaf14590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:36.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.691+0000 7fab1d0d6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bbad0f6b0 0x564bbaf14590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33166/0 (socket says 172.21.15.39:33166) 2023-07-21T18:55:36.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.691+0000 7fab1d0d6700 1 -- 172.21.15.39:0/1330512905 learned_addr learned my addr 172.21.15.39:0/1330512905 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:55:36.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.691+0000 7fab1c8d5700 1 --2- 172.21.15.39:0/1330512905 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bbaeb3580 0x564bbaf14ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:36.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.691+0000 7fab1d8d7700 1 --2- 172.21.15.39:0/1330512905 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564bbaeb92d0 0x564bbaf1f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:36.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.692+0000 7fab1d0d6700 1 -- 172.21.15.39:0/1330512905 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564bbaeb92d0 msgr2=0x564bbaf1f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:55:36.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.692+0000 7fab1d0d6700 1 --2- 172.21.15.39:0/1330512905 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564bbaeb92d0 0x564bbaf1f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:36.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.692+0000 7fab1d0d6700 1 -- 172.21.15.39:0/1330512905 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bbaeb3580 msgr2=0x564bbaf14ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:55:36.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.692+0000 7fab1d0d6700 1 --2- 172.21.15.39:0/1330512905 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bbaeb3580 0x564bbaf14ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:36.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.692+0000 7fab1d0d6700 1 -- 172.21.15.39:0/1330512905 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564bbac1bc00 con 0x564bbad0f6b0 2023-07-21T18:55:36.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.692+0000 7fab1d8d7700 1 --2- 172.21.15.39:0/1330512905 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564bbaeb92d0 0x564bbaf1f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:55:36.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.692+0000 7fab1c8d5700 1 --2- 172.21.15.39:0/1330512905 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bbaeb3580 0x564bbaf14ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:55:36.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.692+0000 7fab1d0d6700 1 --2- 172.21.15.39:0/1330512905 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bbad0f6b0 0x564bbaf14590 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fab140036c0 tx=0x7fab1401bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb86ebf49fc215e7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:55:36.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.692+0000 7fab00ff9700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:55:36.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.692+0000 7fab00ff9700 1 -- 172.21.15.39:0/1330512905 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab14003be0 con 0x564bbad0f6b0 2023-07-21T18:55:36.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.692+0000 7fab00ff9700 1 -- 172.21.15.39:0/1330512905 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fab1402f070 con 0x564bbad0f6b0 2023-07-21T18:55:36.710 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.692+0000 7fab2b4c0b80 1 -- 172.21.15.39:0/1330512905 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fab08096440 con 0x564bbad0f6b0 2023-07-21T18:55:36.710 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.692+0000 7fab00ff9700 1 -- 172.21.15.39:0/1330512905 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab14003d40 con 0x564bbad0f6b0 2023-07-21T18:55:36.710 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.693+0000 7fab2b4c0b80 10 client.17217 fetch_fsmap learned FSMap version 82 2023-07-21T18:55:36.710 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.693+0000 7fab2b4c0b80 1 -- 172.21.15.39:0/1330512905 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=4+}) v3 -- 0x7fab1810c0d0 con 0x564bbad0f6b0 2023-07-21T18:55:36.710 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.693+0000 7fab00ff9700 1 -- 172.21.15.39:0/1330512905 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab14048020 con 0x564bbad0f6b0 2023-07-21T18:55:36.710 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.693+0000 7fab00ff9700 1 -- 172.21.15.39:0/1330512905 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 82) v1 ==== 7522+0+0 (secure 0 0 0) 0x7fab14031870 con 0x564bbad0f6b0 2023-07-21T18:55:36.710 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.693+0000 7fab2b4c0b80 10 client.17217 fetch_fsmap finished waiting for FSMap version 82 2023-07-21T18:55:36.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.693+0000 7fab2b4c0b80 10 client.17217 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:55:36.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.693+0000 7fab2b4c0b80 20 client.17217 populate_metadata read hostname 'smithi039' 2023-07-21T18:55:36.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.693+0000 7fab2b4c0b80 1 --2- 172.21.15.39:0/1330512905 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x564bbaf20f00 0x564bbaf23310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:36.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.693+0000 7fab2b4c0b80 4 client.17217 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:55:36.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.693+0000 7fab2b4c0b80 1 -- 172.21.15.39:0/1330512905 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564bbadbfff0 con 0x564bbaf20f00 2023-07-21T18:55:36.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.693+0000 7fab1c8d5700 1 --2- 172.21.15.39:0/1330512905 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x564bbaf20f00 0x564bbaf23310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:36.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.694+0000 7fab00ff9700 10 client.17217 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:55:36.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.694+0000 7fab1c8d5700 1 --2- 172.21.15.39:0/1330512905 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x564bbaf20f00 0x564bbaf23310 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=320eee469c3faa86 server_cookie=dc8196e00dadaf6c in_seq=0 out_seq=0 2023-07-21T18:55:36.959 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.948+0000 7fab00ff9700 1 -- 172.21.15.39:0/1330512905 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564bbadbfff0 con 0x564bbaf20f00 2023-07-21T18:55:36.959 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.948+0000 7fab00ff9700 10 client.17217 handle_command_reply: tid=0 2023-07-21T18:55:36.960 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.948+0000 7fab2b4c0b80 1 client.17217 shutdown 2023-07-21T18:55:36.960 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.948+0000 7fab2b4c0b80 1 -- 172.21.15.39:0/1330512905 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bbad0f6b0 msgr2=0x564bbaf14590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:55:36.960 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.948+0000 7fab2b4c0b80 1 --2- 172.21.15.39:0/1330512905 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bbad0f6b0 0x564bbaf14590 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fab140036c0 tx=0x7fab1401bf50 comp rx=0 tx=0).stop 2023-07-21T18:55:36.960 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.948+0000 7fab2b4c0b80 1 -- 172.21.15.39:0/1330512905 shutdown_connections 2023-07-21T18:55:36.960 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.948+0000 7fab2b4c0b80 1 --2- 172.21.15.39:0/1330512905 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x564bbaf20f00 0x564bbaf23310 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:36.960 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.948+0000 7fab2b4c0b80 1 --2- 172.21.15.39:0/1330512905 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564bbaeb92d0 0x564bbaf1f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:36.961 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.948+0000 7fab2b4c0b80 1 --2- 172.21.15.39:0/1330512905 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bbad0f6b0 0x564bbaf14590 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:36.961 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.948+0000 7fab2b4c0b80 1 --2- 172.21.15.39:0/1330512905 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bbaeb3580 0x564bbaf14ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:36.961 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.948+0000 7fab00ff9700 0 client.17217 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:55:36.961 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.948+0000 7fab2b4c0b80 1 -- 172.21.15.39:0/1330512905 >> 172.21.15.39:0/1330512905 conn(0x564bbaea8be0 msgr2=0x564bbaeb9a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:55:36.961 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.948+0000 7fab2b4c0b80 1 -- 172.21.15.39:0/1330512905 shutdown_connections 2023-07-21T18:55:36.961 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.948+0000 7fab2b4c0b80 1 -- 172.21.15.39:0/1330512905 wait complete. 2023-07-21T18:55:36.961 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.949+0000 7fab2b4c0b80 20 client.17217 trim_cache size 0 max 16384 2023-07-21T18:55:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:55:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:55:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:55:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:55:36.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.949+0000 7fab1f33a700 1 -- 172.21.15.39:0/3776640679 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fab040a12b0 msgr2=0x7fab040a3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:55:36.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.949+0000 7fab1f33a700 1 --2- 172.21.15.39:0/3776640679 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fab040a12b0 0x7fab040a3760 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fab14009990 tx=0x7fab14005fb0 comp rx=0 tx=0).stop 2023-07-21T18:55:36.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.949+0000 7fab1f33a700 1 -- 172.21.15.39:0/3776640679 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab180ffab0 msgr2=0x7fab18199e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:55:36.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.949+0000 7fab1f33a700 1 --2- 172.21.15.39:0/3776640679 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab180ffab0 0x7fab18199e70 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fab0800d950 tx=0x7fab0800dc60 comp rx=0 tx=0).stop 2023-07-21T18:55:36.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.949+0000 7fab1f33a700 1 -- 172.21.15.39:0/3776640679 shutdown_connections 2023-07-21T18:55:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.949+0000 7fab1f33a700 1 --2- 172.21.15.39:0/3776640679 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fab040a12b0 0x7fab040a3760 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.949+0000 7fab1f33a700 1 --2- 172.21.15.39:0/3776640679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab180ff100 0x7fab18199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.949+0000 7fab1f33a700 1 --2- 172.21.15.39:0/3776640679 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fab181039b0 0x7fab18193790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.949+0000 7fab1f33a700 1 --2- 172.21.15.39:0/3776640679 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab180ffab0 0x7fab18199e70 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.950+0000 7fab1f33a700 1 -- 172.21.15.39:0/3776640679 >> 172.21.15.39:0/3776640679 conn(0x7fab18075f90 msgr2=0x7fab180fbaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:55:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.950+0000 7fab1f33a700 1 -- 172.21.15.39:0/3776640679 shutdown_connections 2023-07-21T18:55:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:36.950+0000 7fab1f33a700 1 -- 172.21.15.39:0/3776640679 wait complete. 2023-07-21T18:55:37.654 INFO:tasks.workunit.client.0.smithi039.stdout: 10 20578 25.68 MB/sec warmup 50 sec latency 2437.124 ms 2023-07-21T18:55:38.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:55:38 smithi193 ceph-mon[98204]: pgmap v436: 97 pgs: 97 active+clean; 568 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 7.1 MiB/s wr, 579 op/s 2023-07-21T18:55:38.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:55:38 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:55:38.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:55:38 smithi155 ceph-mon[97934]: pgmap v436: 97 pgs: 97 active+clean; 568 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 7.1 MiB/s wr, 579 op/s 2023-07-21T18:55:38.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:55:38 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:55:38.654 INFO:tasks.workunit.client.0.smithi039.stdout: 10 21313 26.07 MB/sec warmup 51 sec latency 3437.159 ms 2023-07-21T18:55:38.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:55:38 smithi039 ceph-mon[96489]: pgmap v436: 97 pgs: 97 active+clean; 568 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 7.1 MiB/s wr, 579 op/s 2023-07-21T18:55:38.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:55:38 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:55:39.654 INFO:tasks.workunit.client.0.smithi039.stdout: 10 21873 26.06 MB/sec warmup 52 sec latency 4437.192 ms 2023-07-21T18:55:40.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:55:40 smithi193 ceph-mon[98204]: pgmap v437: 97 pgs: 97 active+clean; 568 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.8 MiB/s wr, 455 op/s 2023-07-21T18:55:40.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:55:40 smithi155 ceph-mon[97934]: pgmap v437: 97 pgs: 97 active+clean; 568 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.8 MiB/s wr, 455 op/s 2023-07-21T18:55:40.654 INFO:tasks.workunit.client.0.smithi039.stdout: 10 22508 26.01 MB/sec warmup 53 sec latency 5437.226 ms 2023-07-21T18:55:40.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:55:40 smithi039 ceph-mon[96489]: pgmap v437: 97 pgs: 97 active+clean; 568 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.8 MiB/s wr, 455 op/s 2023-07-21T18:55:41.655 INFO:tasks.workunit.client.0.smithi039.stdout: 10 23529 26.60 MB/sec warmup 54 sec latency 6437.266 ms 2023-07-21T18:55:42.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:55:42 smithi193 ceph-mon[98204]: pgmap v438: 97 pgs: 97 active+clean; 570 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 8.6 MiB/s wr, 812 op/s 2023-07-21T18:55:42.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:55:42 smithi155 ceph-mon[97934]: pgmap v438: 97 pgs: 97 active+clean; 570 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 8.6 MiB/s wr, 812 op/s 2023-07-21T18:55:42.654 INFO:tasks.workunit.client.0.smithi039.stdout: 10 24289 26.98 MB/sec warmup 55 sec latency 6560.199 ms 2023-07-21T18:55:42.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:55:42 smithi039 ceph-mon[96489]: pgmap v438: 97 pgs: 97 active+clean; 570 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 8.6 MiB/s wr, 812 op/s 2023-07-21T18:55:43.654 INFO:tasks.workunit.client.0.smithi039.stdout: 10 24952 27.14 MB/sec warmup 56 sec latency 75.094 ms 2023-07-21T18:55:44.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:55:44 smithi193 ceph-mon[98204]: pgmap v439: 97 pgs: 97 active+clean; 570 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.7 MiB/s wr, 634 op/s 2023-07-21T18:55:44.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:55:44 smithi155 ceph-mon[97934]: pgmap v439: 97 pgs: 97 active+clean; 570 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.7 MiB/s wr, 634 op/s 2023-07-21T18:55:44.654 INFO:tasks.workunit.client.0.smithi039.stdout: 10 25625 27.32 MB/sec warmup 57 sec latency 64.421 ms 2023-07-21T18:55:44.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:55:44 smithi039 ceph-mon[96489]: pgmap v439: 97 pgs: 97 active+clean; 570 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.7 MiB/s wr, 634 op/s 2023-07-21T18:55:45.654 INFO:tasks.workunit.client.0.smithi039.stdout: 10 26372 27.40 MB/sec warmup 58 sec latency 103.537 ms 2023-07-21T18:55:46.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:55:46 smithi193 ceph-mon[98204]: pgmap v440: 97 pgs: 97 active+clean; 570 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.7 MiB/s wr, 634 op/s 2023-07-21T18:55:46.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:55:46 smithi155 ceph-mon[97934]: pgmap v440: 97 pgs: 97 active+clean; 570 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.7 MiB/s wr, 634 op/s 2023-07-21T18:55:46.654 INFO:tasks.workunit.client.0.smithi039.stdout: 10 27009 27.54 MB/sec warmup 59 sec latency 115.817 ms 2023-07-21T18:55:46.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:55:46 smithi039 ceph-mon[96489]: pgmap v440: 97 pgs: 97 active+clean; 570 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.7 MiB/s wr, 634 op/s 2023-07-21T18:55:47.654 INFO:tasks.workunit.client.0.smithi039.stdout: 10 27834 27.80 MB/sec warmup 60 sec latency 61.957 ms 2023-07-21T18:55:48.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:55:48 smithi193 ceph-mon[98204]: pgmap v441: 97 pgs: 97 active+clean; 592 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 10 MiB/s wr, 909 op/s 2023-07-21T18:55:48.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:55:48 smithi155 ceph-mon[97934]: pgmap v441: 97 pgs: 97 active+clean; 592 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 10 MiB/s wr, 909 op/s 2023-07-21T18:55:48.654 INFO:tasks.workunit.client.0.smithi039.stdout: 10 28601 28.06 MB/sec warmup 61 sec latency 277.032 ms 2023-07-21T18:55:48.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:55:48 smithi039 ceph-mon[96489]: pgmap v441: 97 pgs: 97 active+clean; 592 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 10 MiB/s wr, 909 op/s 2023-07-21T18:55:49.654 INFO:tasks.workunit.client.0.smithi039.stdout: 10 29203 28.17 MB/sec warmup 62 sec latency 1277.072 ms 2023-07-21T18:55:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.094+0000 7fe71572c700 1 -- 172.21.15.39:0/1706880680 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7100fe550 msgr2=0x7fe7100fe920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:55:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.094+0000 7fe71572c700 1 --2- 172.21.15.39:0/1706880680 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7100fe550 0x7fe7100fe920 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fe700009990 tx=0x7fe700009ca0 comp rx=0 tx=0).stop 2023-07-21T18:55:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.094+0000 7fe71572c700 1 -- 172.21.15.39:0/1706880680 shutdown_connections 2023-07-21T18:55:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.094+0000 7fe71572c700 1 --2- 172.21.15.39:0/1706880680 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe710103470 0x7fe71010b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.094+0000 7fe71572c700 1 --2- 172.21.15.39:0/1706880680 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7100fe550 0x7fe7100fe920 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.094+0000 7fe71572c700 1 --2- 172.21.15.39:0/1706880680 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7100feef0 0x7fe710102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.094+0000 7fe71572c700 1 -- 172.21.15.39:0/1706880680 >> 172.21.15.39:0/1706880680 conn(0x7fe710075060 msgr2=0x7fe710075460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:55:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.094+0000 7fe71572c700 1 -- 172.21.15.39:0/1706880680 shutdown_connections 2023-07-21T18:55:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.094+0000 7fe71572c700 1 -- 172.21.15.39:0/1706880680 wait complete. 2023-07-21T18:55:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.094+0000 7fe71572c700 1 Processor -- start 2023-07-21T18:55:50.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.095+0000 7fe71572c700 1 -- start start 2023-07-21T18:55:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.095+0000 7fe71572c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7100fe550 0x7fe710199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.095+0000 7fe71572c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe7100feef0 0x7fe710199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.095+0000 7fe71572c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe710103470 0x7fe710193880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.095+0000 7fe71572c700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe710193ef0 con 0x7fe710103470 2023-07-21T18:55:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.095+0000 7fe71572c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe710194060 con 0x7fe7100fe550 2023-07-21T18:55:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.095+0000 7fe71572c700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe710194220 con 0x7fe7100feef0 2023-07-21T18:55:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.095+0000 7fe70effd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7100fe550 0x7fe710199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.095+0000 7fe70effd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7100fe550 0x7fe710199a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:45776/0 (socket says 172.21.15.39:45776) 2023-07-21T18:55:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.095+0000 7fe70effd700 1 -- 172.21.15.39:0/1869685499 learned_addr learned my addr 172.21.15.39:0/1869685499 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:55:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.095+0000 7fe70f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe710103470 0x7fe710193880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.095+0000 7fe70e7fc700 1 --2- 172.21.15.39:0/1869685499 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe7100feef0 0x7fe710199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.096+0000 7fe70effd700 1 -- 172.21.15.39:0/1869685499 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe7100feef0 msgr2=0x7fe710199f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:55:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.096+0000 7fe70effd700 1 --2- 172.21.15.39:0/1869685499 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe7100feef0 0x7fe710199f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.096+0000 7fe70effd700 1 -- 172.21.15.39:0/1869685499 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe710103470 msgr2=0x7fe710193880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:55:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.096+0000 7fe70effd700 1 --2- 172.21.15.39:0/1869685499 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe710103470 0x7fe710193880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.096+0000 7fe70effd700 1 -- 172.21.15.39:0/1869685499 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe704009520 con 0x7fe7100fe550 2023-07-21T18:55:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.096+0000 7fe70f7fe700 1 --2- 172.21.15.39:0/1869685499 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe710103470 0x7fe710193880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:55:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.096+0000 7fe70effd700 1 --2- 172.21.15.39:0/1869685499 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7100fe550 0x7fe710199a20 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe700000c00 tx=0x7fe700004790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:55:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.096+0000 7fe6f7fff700 1 -- 172.21.15.39:0/1869685499 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70001d070 con 0x7fe7100fe550 2023-07-21T18:55:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.096+0000 7fe71572c700 1 -- 172.21.15.39:0/1869685499 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7000095f0 con 0x7fe7100fe550 2023-07-21T18:55:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.096+0000 7fe71572c700 1 -- 172.21.15.39:0/1869685499 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7101a0830 con 0x7fe7100fe550 2023-07-21T18:55:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.096+0000 7fe6f7fff700 1 -- 172.21.15.39:0/1869685499 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe700021410 con 0x7fe7100fe550 2023-07-21T18:55:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.096+0000 7fe6f7fff700 1 -- 172.21.15.39:0/1869685499 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe700021ea0 con 0x7fe7100fe550 2023-07-21T18:55:50.113 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.099+0000 7fe6f7fff700 1 -- 172.21.15.39:0/1869685499 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fe700021580 con 0x7fe7100fe550 2023-07-21T18:55:50.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.100+0000 7fe6f7fff700 1 --2- 172.21.15.39:0/1869685499 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fe6f80a12c0 0x7fe6f80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:50.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.100+0000 7fe6f7fff700 1 -- 172.21.15.39:0/1869685499 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fe7000cddb0 con 0x7fe7100fe550 2023-07-21T18:55:50.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.100+0000 7fe70e7fc700 1 --2- 172.21.15.39:0/1869685499 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fe6f80a12c0 0x7fe6f80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:50.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.100+0000 7fe7218b2b80 1 Processor -- start 2023-07-21T18:55:50.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.100+0000 7fe7218b2b80 1 -- start start 2023-07-21T18:55:50.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.100+0000 7fe70e7fc700 1 --2- 172.21.15.39:0/1869685499 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fe6f80a12c0 0x7fe6f80a3770 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe710103060 tx=0x7fe704009260 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:55:50.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.100+0000 7fe7218b2b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d035eaf6b0 0x55d036058d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:50.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.100+0000 7fe7218b2b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d0360592d0 0x55d036053040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:50.115 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.100+0000 7fe7218b2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d036053580 0x55d0360539d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:50.115 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.100+0000 7fe7218b2b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d035f58c50 con 0x55d035eaf6b0 2023-07-21T18:55:50.115 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.100+0000 7fe7218b2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d035fab9e0 con 0x55d036053580 2023-07-21T18:55:50.115 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.100+0000 7fe7218b2b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d035fa4e70 con 0x55d0360592d0 2023-07-21T18:55:50.115 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.101+0000 7fe70e7fc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d0360592d0 0x55d036053040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:50.115 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.101+0000 7fe70effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d035eaf6b0 0x55d036058d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:50.115 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.101+0000 7fe70e7fc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d0360592d0 0x55d036053040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:51740/0 (socket says 172.21.15.39:51740) 2023-07-21T18:55:50.116 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.101+0000 7fe70e7fc700 1 -- 172.21.15.39:0/688631016 learned_addr learned my addr 172.21.15.39:0/688631016 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:55:50.116 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.101+0000 7fe70f7fe700 1 --2- 172.21.15.39:0/688631016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d036053580 0x55d0360539d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:50.116 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.101+0000 7fe70e7fc700 1 -- 172.21.15.39:0/688631016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d036053580 msgr2=0x55d0360539d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:55:50.116 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.101+0000 7fe70e7fc700 1 --2- 172.21.15.39:0/688631016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d036053580 0x55d0360539d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:50.116 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.101+0000 7fe70e7fc700 1 -- 172.21.15.39:0/688631016 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d035eaf6b0 msgr2=0x55d036058d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:55:50.116 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.101+0000 7fe70e7fc700 1 --2- 172.21.15.39:0/688631016 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d035eaf6b0 0x55d036058d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:50.116 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.101+0000 7fe70e7fc700 1 -- 172.21.15.39:0/688631016 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d035dd2d80 con 0x55d0360592d0 2023-07-21T18:55:50.116 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.101+0000 7fe70f7fe700 1 --2- 172.21.15.39:0/688631016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d036053580 0x55d0360539d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:55:50.117 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.101+0000 7fe70e7fc700 1 --2- 172.21.15.39:0/688631016 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d0360592d0 0x55d036053040 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe70400e5d0 tx=0x7fe704014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=467319c943789df6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:55:50.117 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.101+0000 7fe6f57fa700 1 -- 172.21.15.39:0/688631016 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe704004020 con 0x55d0360592d0 2023-07-21T18:55:50.117 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.101+0000 7fe6f57fa700 1 -- 172.21.15.39:0/688631016 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe704021070 con 0x55d0360592d0 2023-07-21T18:55:50.117 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.101+0000 7fe6f57fa700 1 -- 172.21.15.39:0/688631016 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7040266e0 con 0x55d0360592d0 2023-07-21T18:55:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.102+0000 7fe7218b2b80 1 -- 172.21.15.39:0/688631016 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d0360592d0 msgr2=0x55d036053040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:55:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.102+0000 7fe7218b2b80 1 --2- 172.21.15.39:0/688631016 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d0360592d0 0x55d036053040 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe70400e5d0 tx=0x7fe704014000 comp rx=0 tx=0).stop 2023-07-21T18:55:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.102+0000 7fe7218b2b80 1 -- 172.21.15.39:0/688631016 shutdown_connections 2023-07-21T18:55:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.102+0000 7fe7218b2b80 1 --2- 172.21.15.39:0/688631016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d036053580 0x55d0360539d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.102+0000 7fe7218b2b80 1 --2- 172.21.15.39:0/688631016 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d035eaf6b0 0x55d036058d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.102+0000 7fe7218b2b80 1 --2- 172.21.15.39:0/688631016 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d0360592d0 0x55d036053040 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.102+0000 7fe7218b2b80 1 -- 172.21.15.39:0/688631016 >> 172.21.15.39:0/688631016 conn(0x55d036048be0 msgr2=0x55d03604df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:55:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.102+0000 7fe7218b2b80 1 -- 172.21.15.39:0/688631016 shutdown_connections 2023-07-21T18:55:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.102+0000 7fe7218b2b80 1 -- 172.21.15.39:0/688631016 wait complete. 2023-07-21T18:55:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.103+0000 7fe7218b2b80 1 -- start start 2023-07-21T18:55:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.103+0000 7fe7218b2b80 1 Processor -- start 2023-07-21T18:55:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.103+0000 7fe7218b2b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d035eaf6b0 0x55d03605f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.103+0000 7fe7218b2b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d036053580 0x55d03605c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:50.122 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.103+0000 7fe7218b2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d0360592d0 0x55d03605cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:50.122 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.103+0000 7fe7218b2b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d035f44a60 con 0x55d035eaf6b0 2023-07-21T18:55:50.122 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.103+0000 7fe7218b2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d035f44760 con 0x55d0360592d0 2023-07-21T18:55:50.122 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.103+0000 7fe7218b2b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d035e288c0 con 0x55d036053580 2023-07-21T18:55:50.122 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.103+0000 7fe7218b2b80 10 client.0 _ll_register_callbacks cb 0x55d035f42700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:55:50.122 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.103+0000 7fe70effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d035eaf6b0 0x55d03605f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:50.122 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.103+0000 7fe70effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d035eaf6b0 0x55d03605f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33194/0 (socket says 172.21.15.39:33194) 2023-07-21T18:55:50.123 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.104+0000 7fe70effd700 1 -- 172.21.15.39:0/2447484054 learned_addr learned my addr 172.21.15.39:0/2447484054 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:55:50.123 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.104+0000 7fe70f7fe700 1 --2- 172.21.15.39:0/2447484054 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d0360592d0 0x55d03605cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:50.123 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.104+0000 7fe70e7fc700 1 --2- 172.21.15.39:0/2447484054 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d036053580 0x55d03605c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:50.123 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.104+0000 7fe70f7fe700 1 -- 172.21.15.39:0/2447484054 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d036053580 msgr2=0x55d03605c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:55:50.123 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.104+0000 7fe70f7fe700 1 --2- 172.21.15.39:0/2447484054 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d036053580 0x55d03605c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:50.123 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.104+0000 7fe70f7fe700 1 -- 172.21.15.39:0/2447484054 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d035eaf6b0 msgr2=0x55d03605f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:55:50.123 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.104+0000 7fe70f7fe700 1 --2- 172.21.15.39:0/2447484054 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d035eaf6b0 0x55d03605f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:50.123 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.104+0000 7fe70f7fe700 1 -- 172.21.15.39:0/2447484054 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe704017070 con 0x55d0360592d0 2023-07-21T18:55:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.104+0000 7fe70e7fc700 1 --2- 172.21.15.39:0/2447484054 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d036053580 0x55d03605c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:55:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.104+0000 7fe70f7fe700 1 --2- 172.21.15.39:0/2447484054 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d0360592d0 0x55d03605cef0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe70800dfd0 tx=0x7fe70800fab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78cd7bc411bcd6d0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:55:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.104+0000 7fe6f2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-21T18:55:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.105+0000 7fe6f2ff5700 1 -- 172.21.15.39:0/2447484054 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe708012040 con 0x55d0360592d0 2023-07-21T18:55:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.105+0000 7fe6f2ff5700 1 -- 172.21.15.39:0/2447484054 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fe708003c00 con 0x55d0360592d0 2023-07-21T18:55:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.105+0000 7fe7218b2b80 1 -- 172.21.15.39:0/2447484054 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d035efe490 con 0x55d0360592d0 2023-07-21T18:55:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.105+0000 7fe6f2ff5700 1 -- 172.21.15.39:0/2447484054 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe708003dc0 con 0x55d0360592d0 2023-07-21T18:55:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.105+0000 7fe7218b2b80 10 client.26678 fetch_fsmap learned FSMap version 83 2023-07-21T18:55:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.105+0000 7fe7218b2b80 1 -- 172.21.15.39:0/2447484054 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x55d035dbbc00 con 0x55d0360592d0 2023-07-21T18:55:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.105+0000 7fe6f2ff5700 1 -- 172.21.15.39:0/2447484054 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe708025020 con 0x55d0360592d0 2023-07-21T18:55:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.105+0000 7fe6f2ff5700 1 -- 172.21.15.39:0/2447484054 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 83) v1 ==== 7526+0+0 (secure 0 0 0) 0x7fe7080046e0 con 0x55d0360592d0 2023-07-21T18:55:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.105+0000 7fe7218b2b80 10 client.26678 fetch_fsmap finished waiting for FSMap version 83 2023-07-21T18:55:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.105+0000 7fe7218b2b80 10 client.26678 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:55:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.105+0000 7fe7218b2b80 20 client.26678 populate_metadata read hostname 'smithi039' 2023-07-21T18:55:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.105+0000 7fe7218b2b80 1 --2- 172.21.15.39:0/2447484054 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d03606cb60 0x55d03606ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.105+0000 7fe7218b2b80 4 client.26678 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:55:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.105+0000 7fe7218b2b80 1 -- 172.21.15.39:0/2447484054 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d035f9de50 con 0x55d03606cb60 2023-07-21T18:55:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.106+0000 7fe70effd700 1 --2- 172.21.15.39:0/2447484054 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d03606cb60 0x55d03606ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.106+0000 7fe70effd700 1 --2- 172.21.15.39:0/2447484054 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d03606cb60 0x55d03606ef70 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2391f06af636e7b7 server_cookie=1f8b6052f094b300 in_seq=0 out_seq=0 2023-07-21T18:55:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.106+0000 7fe6f2ff5700 10 client.26678 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:55:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.107+0000 7fe6f2ff5700 1 -- 172.21.15.39:0/2447484054 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d035f9de50 con 0x55d03606cb60 2023-07-21T18:55:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.107+0000 7fe6f2ff5700 10 client.26678 handle_command_reply: tid=0 2023-07-21T18:55:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.107+0000 7fe7218b2b80 1 client.26678 shutdown 2023-07-21T18:55:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.107+0000 7fe7218b2b80 1 -- 172.21.15.39:0/2447484054 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d0360592d0 msgr2=0x55d03605cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:55:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.107+0000 7fe7218b2b80 1 --2- 172.21.15.39:0/2447484054 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d0360592d0 0x55d03605cef0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe70800dfd0 tx=0x7fe70800fab0 comp rx=0 tx=0).stop 2023-07-21T18:55:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.107+0000 7fe7218b2b80 1 -- 172.21.15.39:0/2447484054 shutdown_connections 2023-07-21T18:55:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.107+0000 7fe7218b2b80 1 --2- 172.21.15.39:0/2447484054 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d03606cb60 0x55d03606ef70 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.107+0000 7fe7218b2b80 1 --2- 172.21.15.39:0/2447484054 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d0360592d0 0x55d03605cef0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.107+0000 7fe7218b2b80 1 --2- 172.21.15.39:0/2447484054 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d035eaf6b0 0x55d03605f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.107+0000 7fe6f2ff5700 0 client.26678 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:55:50.128 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.107+0000 7fe7218b2b80 1 --2- 172.21.15.39:0/2447484054 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d036053580 0x55d03605c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:50.128 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.107+0000 7fe7218b2b80 1 -- 172.21.15.39:0/2447484054 >> 172.21.15.39:0/2447484054 conn(0x55d036048be0 msgr2=0x55d03604cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:55:50.128 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.107+0000 7fe7218b2b80 1 -- 172.21.15.39:0/2447484054 shutdown_connections 2023-07-21T18:55:50.128 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.107+0000 7fe7218b2b80 1 -- 172.21.15.39:0/2447484054 wait complete. 2023-07-21T18:55:50.128 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.107+0000 7fe7218b2b80 20 client.26678 trim_cache size 0 max 16384 2023-07-21T18:55:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.125+0000 7fe7218b2b80 1 Processor -- start 2023-07-21T18:55:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.125+0000 7fe7218b2b80 1 -- start start 2023-07-21T18:55:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.126+0000 7fe7218b2b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d035eaf6b0 0x55d036053040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.126+0000 7fe7218b2b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d036053580 0x55d0360738d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.126+0000 7fe7218b2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d0360592d0 0x55d036058070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.126+0000 7fe7218b2b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d036057460 con 0x55d036053580 2023-07-21T18:55:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.126+0000 7fe7218b2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d03605dbc0 con 0x55d0360592d0 2023-07-21T18:55:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.126+0000 7fe7218b2b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d03605d7d0 con 0x55d035eaf6b0 2023-07-21T18:55:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.126+0000 7fe70e7fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d036053580 0x55d0360738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.126+0000 7fe70e7fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d036053580 0x55d0360738d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33204/0 (socket says 172.21.15.39:33204) 2023-07-21T18:55:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.126+0000 7fe70e7fc700 1 -- 172.21.15.39:0/4033441497 learned_addr learned my addr 172.21.15.39:0/4033441497 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:55:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.126+0000 7fe70effd700 1 --2- 172.21.15.39:0/4033441497 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d035eaf6b0 0x55d036053040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.126+0000 7fe70f7fe700 1 --2- 172.21.15.39:0/4033441497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d0360592d0 0x55d036058070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:50.140 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.126+0000 7fe70e7fc700 1 -- 172.21.15.39:0/4033441497 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d035eaf6b0 msgr2=0x55d036053040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:55:50.140 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.126+0000 7fe70e7fc700 1 --2- 172.21.15.39:0/4033441497 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d035eaf6b0 0x55d036053040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:50.140 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.126+0000 7fe70e7fc700 1 -- 172.21.15.39:0/4033441497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d0360592d0 msgr2=0x55d036058070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:55:50.140 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.126+0000 7fe70e7fc700 1 --2- 172.21.15.39:0/4033441497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d0360592d0 0x55d036058070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:50.140 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.126+0000 7fe70e7fc700 1 -- 172.21.15.39:0/4033441497 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d036053c20 con 0x55d036053580 2023-07-21T18:55:50.140 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.126+0000 7fe70effd700 1 --2- 172.21.15.39:0/4033441497 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d035eaf6b0 0x55d036053040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:55:50.140 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.126+0000 7fe70e7fc700 1 --2- 172.21.15.39:0/4033441497 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d036053580 0x55d0360738d0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x55d03604fb40 tx=0x7fe704011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c11f8086f1fc30e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:55:50.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.127+0000 7fe6f5ffb700 1 -- 172.21.15.39:0/4033441497 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70402b070 con 0x55d036053580 2023-07-21T18:55:50.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.127+0000 7fe6f5ffb700 1 -- 172.21.15.39:0/4033441497 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe70400ee70 con 0x55d036053580 2023-07-21T18:55:50.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.127+0000 7fe6f5ffb700 1 -- 172.21.15.39:0/4033441497 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe704003e40 con 0x55d036053580 2023-07-21T18:55:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.127+0000 7fe7218b2b80 1 -- 172.21.15.39:0/4033441497 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d036053580 msgr2=0x55d0360738d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:55:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.127+0000 7fe7218b2b80 1 --2- 172.21.15.39:0/4033441497 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d036053580 0x55d0360738d0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x55d03604fb40 tx=0x7fe704011ba0 comp rx=0 tx=0).stop 2023-07-21T18:55:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.127+0000 7fe7218b2b80 1 -- 172.21.15.39:0/4033441497 shutdown_connections 2023-07-21T18:55:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.127+0000 7fe7218b2b80 1 --2- 172.21.15.39:0/4033441497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d0360592d0 0x55d036058070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.127+0000 7fe7218b2b80 1 --2- 172.21.15.39:0/4033441497 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d035eaf6b0 0x55d036053040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.127+0000 7fe7218b2b80 1 --2- 172.21.15.39:0/4033441497 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d036053580 0x55d0360738d0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.128+0000 7fe7218b2b80 1 -- 172.21.15.39:0/4033441497 >> 172.21.15.39:0/4033441497 conn(0x55d036048be0 msgr2=0x55d036068550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:55:50.145 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.128+0000 7fe7218b2b80 1 -- 172.21.15.39:0/4033441497 shutdown_connections 2023-07-21T18:55:50.145 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.128+0000 7fe7218b2b80 1 -- 172.21.15.39:0/4033441497 wait complete. 2023-07-21T18:55:50.145 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.128+0000 7fe7218b2b80 1 -- start start 2023-07-21T18:55:50.145 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.128+0000 7fe7218b2b80 1 Processor -- start 2023-07-21T18:55:50.145 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.128+0000 7fe7218b2b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d035eaf6b0 0x55d036055130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:50.145 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.128+0000 7fe7218b2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d036053580 0x55d036055670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:50.145 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.128+0000 7fe7218b2b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d0360592d0 0x55d0360bfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:50.145 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.128+0000 7fe7218b2b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d036055ea0 con 0x55d035eaf6b0 2023-07-21T18:55:50.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.128+0000 7fe7218b2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0360c01f0 con 0x55d036053580 2023-07-21T18:55:50.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.128+0000 7fe7218b2b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0360c0330 con 0x55d0360592d0 2023-07-21T18:55:50.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.128+0000 7fe7218b2b80 10 client.0 _ll_register_callbacks cb 0x7fe70800bea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:55:50.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.128+0000 7fe70effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d035eaf6b0 0x55d036055130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:50.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.129+0000 7fe70effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d035eaf6b0 0x55d036055130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33208/0 (socket says 172.21.15.39:33208) 2023-07-21T18:55:50.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.129+0000 7fe70f7fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d0360592d0 0x55d0360bfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:50.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.129+0000 7fe70e7fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d036053580 0x55d036055670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:50.147 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.129+0000 7fe70effd700 1 -- 172.21.15.39:0/3449427600 learned_addr learned my addr 172.21.15.39:0/3449427600 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:55:50.147 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.129+0000 7fe70effd700 1 -- 172.21.15.39:0/3449427600 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d0360592d0 msgr2=0x55d0360bfcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:55:50.147 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.129+0000 7fe70effd700 1 --2- 172.21.15.39:0/3449427600 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d0360592d0 0x55d0360bfcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:50.147 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.129+0000 7fe70effd700 1 -- 172.21.15.39:0/3449427600 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d036053580 msgr2=0x55d036055670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:55:50.147 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.129+0000 7fe70effd700 1 --2- 172.21.15.39:0/3449427600 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d036053580 0x55d036055670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:50.147 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.129+0000 7fe70effd700 1 -- 172.21.15.39:0/3449427600 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7101a0830 con 0x55d035eaf6b0 2023-07-21T18:55:50.147 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.129+0000 7fe70e7fc700 1 --2- 172.21.15.39:0/3449427600 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d036053580 0x55d036055670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:55:50.147 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.129+0000 7fe70effd700 1 --2- 172.21.15.39:0/3449427600 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d035eaf6b0 0x55d036055130 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fe700097a40 tx=0x7fe700023040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b10361f7e61efc73 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:55:50.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.129+0000 7fe6f2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:55:50.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.129+0000 7fe6f2ff5700 1 -- 172.21.15.39:0/3449427600 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7000a3390 con 0x55d035eaf6b0 2023-07-21T18:55:50.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.129+0000 7fe6f2ff5700 1 -- 172.21.15.39:0/3449427600 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fe7000b3040 con 0x55d035eaf6b0 2023-07-21T18:55:50.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.129+0000 7fe7218b2b80 1 -- 172.21.15.39:0/3449427600 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe704021070 con 0x55d035eaf6b0 2023-07-21T18:55:50.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.129+0000 7fe6f2ff5700 1 -- 172.21.15.39:0/3449427600 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe70009afd0 con 0x55d035eaf6b0 2023-07-21T18:55:50.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.129+0000 7fe7218b2b80 10 client.17235 fetch_fsmap learned FSMap version 83 2023-07-21T18:55:50.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.129+0000 7fe7218b2b80 1 -- 172.21.15.39:0/3449427600 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x55d035dd2d80 con 0x55d035eaf6b0 2023-07-21T18:55:50.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.130+0000 7fe6f2ff5700 1 -- 172.21.15.39:0/3449427600 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe704021070 con 0x55d035eaf6b0 2023-07-21T18:55:50.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.130+0000 7fe6f2ff5700 1 -- 172.21.15.39:0/3449427600 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 83) v1 ==== 7526+0+0 (secure 0 0 0) 0x7fe70009b230 con 0x55d035eaf6b0 2023-07-21T18:55:50.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.130+0000 7fe7218b2b80 10 client.17235 fetch_fsmap finished waiting for FSMap version 83 2023-07-21T18:55:50.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.130+0000 7fe7218b2b80 10 client.17235 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:55:50.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.130+0000 7fe7218b2b80 20 client.17235 populate_metadata read hostname 'smithi039' 2023-07-21T18:55:50.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.130+0000 7fe7218b2b80 1 --2- 172.21.15.39:0/3449427600 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d03606cb60 0x55d0360c3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:55:50.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.130+0000 7fe7218b2b80 4 client.17235 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:55:50.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.130+0000 7fe7218b2b80 1 -- 172.21.15.39:0/3449427600 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d035f5fff0 con 0x55d03606cb60 2023-07-21T18:55:50.150 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.130+0000 7fe70e7fc700 1 --2- 172.21.15.39:0/3449427600 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d03606cb60 0x55d0360c3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:55:50.150 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.130+0000 7fe6f2ff5700 10 client.17235 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:55:50.150 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.130+0000 7fe70e7fc700 1 --2- 172.21.15.39:0/3449427600 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d03606cb60 0x55d0360c3740 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=572ede66af5c5ecd server_cookie=25630ce17e39c755 in_seq=0 out_seq=0 2023-07-21T18:55:50.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.357+0000 7fe6f2ff5700 1 -- 172.21.15.39:0/3449427600 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d035f5fff0 con 0x55d03606cb60 2023-07-21T18:55:50.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.357+0000 7fe6f2ff5700 10 client.17235 handle_command_reply: tid=0 2023-07-21T18:55:50.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.357+0000 7fe7218b2b80 1 client.17235 shutdown 2023-07-21T18:55:50.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.357+0000 7fe7218b2b80 1 -- 172.21.15.39:0/3449427600 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d035eaf6b0 msgr2=0x55d036055130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:55:50.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.357+0000 7fe7218b2b80 1 --2- 172.21.15.39:0/3449427600 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d035eaf6b0 0x55d036055130 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fe700097a40 tx=0x7fe700023040 comp rx=0 tx=0).stop 2023-07-21T18:55:50.370 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.357+0000 7fe7218b2b80 1 -- 172.21.15.39:0/3449427600 shutdown_connections 2023-07-21T18:55:50.370 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.357+0000 7fe7218b2b80 1 --2- 172.21.15.39:0/3449427600 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d03606cb60 0x55d0360c3740 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:50.370 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.357+0000 7fe7218b2b80 1 --2- 172.21.15.39:0/3449427600 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d0360592d0 0x55d0360bfcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:50.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.357+0000 7fe7218b2b80 1 --2- 172.21.15.39:0/3449427600 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d035eaf6b0 0x55d036055130 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:50.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.357+0000 7fe7218b2b80 1 --2- 172.21.15.39:0/3449427600 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d036053580 0x55d036055670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:50.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.357+0000 7fe7218b2b80 1 -- 172.21.15.39:0/3449427600 >> 172.21.15.39:0/3449427600 conn(0x55d036048be0 msgr2=0x55d036068550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:55:50.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.357+0000 7fe6f2ff5700 0 client.17235 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:55:50.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.357+0000 7fe7218b2b80 1 -- 172.21.15.39:0/3449427600 shutdown_connections 2023-07-21T18:55:50.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.357+0000 7fe7218b2b80 1 -- 172.21.15.39:0/3449427600 wait complete. 2023-07-21T18:55:50.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.358+0000 7fe7218b2b80 20 client.17235 trim_cache size 0 max 16384 2023-07-21T18:55:50.372 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:55:50.372 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:55:50.372 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:55:50.372 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:55:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.358+0000 7fe71572c700 1 -- 172.21.15.39:0/1869685499 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fe6f80a12c0 msgr2=0x7fe6f80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:55:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.358+0000 7fe71572c700 1 --2- 172.21.15.39:0/1869685499 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fe6f80a12c0 0x7fe6f80a3770 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe710103060 tx=0x7fe704009260 comp rx=0 tx=0).stop 2023-07-21T18:55:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.358+0000 7fe71572c700 1 -- 172.21.15.39:0/1869685499 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7100fe550 msgr2=0x7fe710199a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:55:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.358+0000 7fe71572c700 1 --2- 172.21.15.39:0/1869685499 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7100fe550 0x7fe710199a20 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe700000c00 tx=0x7fe700004790 comp rx=0 tx=0).stop 2023-07-21T18:55:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.358+0000 7fe71572c700 1 -- 172.21.15.39:0/1869685499 shutdown_connections 2023-07-21T18:55:50.374 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.358+0000 7fe71572c700 1 --2- 172.21.15.39:0/1869685499 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fe6f80a12c0 0x7fe6f80a3770 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:50.374 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.358+0000 7fe71572c700 1 --2- 172.21.15.39:0/1869685499 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7100fe550 0x7fe710199a20 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:50.374 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.358+0000 7fe71572c700 1 --2- 172.21.15.39:0/1869685499 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe710103470 0x7fe710193880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:50.374 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.358+0000 7fe71572c700 1 --2- 172.21.15.39:0/1869685499 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe7100feef0 0x7fe710199f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:55:50.374 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.358+0000 7fe71572c700 1 -- 172.21.15.39:0/1869685499 >> 172.21.15.39:0/1869685499 conn(0x7fe710075060 msgr2=0x7fe7100fbd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:55:50.374 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.359+0000 7fe71572c700 1 -- 172.21.15.39:0/1869685499 shutdown_connections 2023-07-21T18:55:50.374 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:55:50.359+0000 7fe71572c700 1 -- 172.21.15.39:0/1869685499 wait complete. 2023-07-21T18:55:50.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:55:50 smithi193 ceph-mon[98204]: pgmap v442: 97 pgs: 97 active+clean; 592 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 7.4 MiB/s wr, 631 op/s 2023-07-21T18:55:50.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:55:50 smithi155 ceph-mon[97934]: pgmap v442: 97 pgs: 97 active+clean; 592 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 7.4 MiB/s wr, 631 op/s 2023-07-21T18:55:50.655 INFO:tasks.workunit.client.0.smithi039.stdout: 10 29784 28.24 MB/sec warmup 63 sec latency 2277.101 ms 2023-07-21T18:55:50.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:55:50 smithi039 ceph-mon[96489]: pgmap v442: 97 pgs: 97 active+clean; 592 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 7.4 MiB/s wr, 631 op/s 2023-07-21T18:55:51.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:55:51 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:55:51.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:55:51 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:55:51.654 INFO:tasks.workunit.client.0.smithi039.stdout: 10 30417 28.30 MB/sec warmup 64 sec latency 3277.134 ms 2023-07-21T18:55:51.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:55:51 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:55:52.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:55:52 smithi193 ceph-mon[98204]: pgmap v443: 97 pgs: 97 active+clean; 613 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 10 MiB/s wr, 945 op/s 2023-07-21T18:55:52.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:55:52 smithi155 ceph-mon[97934]: pgmap v443: 97 pgs: 97 active+clean; 613 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 10 MiB/s wr, 945 op/s 2023-07-21T18:55:52.654 INFO:tasks.workunit.client.0.smithi039.stdout: 10 31157 28.44 MB/sec warmup 65 sec latency 4277.171 ms 2023-07-21T18:55:52.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:55:52 smithi039 ceph-mon[96489]: pgmap v443: 97 pgs: 97 active+clean; 613 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 10 MiB/s wr, 945 op/s 2023-07-21T18:55:53.655 INFO:tasks.workunit.client.0.smithi039.stdout: 10 31940 28.65 MB/sec warmup 66 sec latency 5277.209 ms 2023-07-21T18:55:54.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:55:54 smithi193 ceph-mon[98204]: pgmap v444: 97 pgs: 97 active+clean; 613 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 7.6 MiB/s wr, 588 op/s 2023-07-21T18:55:54.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:55:54 smithi155 ceph-mon[97934]: pgmap v444: 97 pgs: 97 active+clean; 613 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 7.6 MiB/s wr, 588 op/s 2023-07-21T18:55:54.654 INFO:tasks.workunit.client.0.smithi039.stdout: 10 32638 28.82 MB/sec warmup 67 sec latency 6240.385 ms 2023-07-21T18:55:54.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:55:54 smithi039 ceph-mon[96489]: pgmap v444: 97 pgs: 97 active+clean; 613 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 7.6 MiB/s wr, 588 op/s 2023-07-21T18:55:55.655 INFO:tasks.workunit.client.0.smithi039.stdout: 10 33266 28.87 MB/sec warmup 68 sec latency 181.299 ms 2023-07-21T18:55:56.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:55:56 smithi193 ceph-mon[98204]: pgmap v445: 97 pgs: 97 active+clean; 613 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 7.6 MiB/s wr, 588 op/s 2023-07-21T18:55:56.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:55:56 smithi155 ceph-mon[97934]: pgmap v445: 97 pgs: 97 active+clean; 613 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 7.6 MiB/s wr, 588 op/s 2023-07-21T18:55:56.655 INFO:tasks.workunit.client.0.smithi039.stdout: 10 33835 28.95 MB/sec warmup 69 sec latency 215.517 ms 2023-07-21T18:55:56.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:55:56 smithi039 ceph-mon[96489]: pgmap v445: 97 pgs: 97 active+clean; 613 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 7.6 MiB/s wr, 588 op/s 2023-07-21T18:55:57.655 INFO:tasks.workunit.client.0.smithi039.stdout: 10 34667 29.11 MB/sec warmup 70 sec latency 82.536 ms 2023-07-21T18:55:58.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:55:58 smithi193 ceph-mon[98204]: pgmap v446: 97 pgs: 97 active+clean; 638 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 11 MiB/s wr, 864 op/s 2023-07-21T18:55:58.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:55:58 smithi155 ceph-mon[97934]: pgmap v446: 97 pgs: 97 active+clean; 638 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 11 MiB/s wr, 864 op/s 2023-07-21T18:55:58.655 INFO:tasks.workunit.client.0.smithi039.stdout: 10 35529 29.39 MB/sec warmup 71 sec latency 136.075 ms 2023-07-21T18:55:58.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:55:58 smithi039 ceph-mon[96489]: pgmap v446: 97 pgs: 97 active+clean; 638 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 11 MiB/s wr, 864 op/s 2023-07-21T18:55:59.655 INFO:tasks.workunit.client.0.smithi039.stdout: 10 36294 29.62 MB/sec warmup 72 sec latency 86.773 ms 2023-07-21T18:56:00.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:56:00 smithi193 ceph-mon[98204]: pgmap v447: 97 pgs: 97 active+clean; 638 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 6.8 MiB/s wr, 589 op/s 2023-07-21T18:56:00.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:56:00 smithi155 ceph-mon[97934]: pgmap v447: 97 pgs: 97 active+clean; 638 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 6.8 MiB/s wr, 589 op/s 2023-07-21T18:56:00.655 INFO:tasks.workunit.client.0.smithi039.stdout: 10 37083 29.73 MB/sec warmup 73 sec latency 45.384 ms 2023-07-21T18:56:00.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:56:00 smithi039 ceph-mon[96489]: pgmap v447: 97 pgs: 97 active+clean; 638 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 6.8 MiB/s wr, 589 op/s 2023-07-21T18:56:01.655 INFO:tasks.workunit.client.0.smithi039.stdout: 10 37222 29.47 MB/sec warmup 74 sec latency 802.277 ms 2023-07-21T18:56:02.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:56:02 smithi193 ceph-mon[98204]: pgmap v448: 97 pgs: 97 active+clean; 661 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 11 MiB/s wr, 916 op/s 2023-07-21T18:56:02.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:56:02 smithi155 ceph-mon[97934]: pgmap v448: 97 pgs: 97 active+clean; 661 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 11 MiB/s wr, 916 op/s 2023-07-21T18:56:02.655 INFO:tasks.workunit.client.0.smithi039.stdout: 10 37292 29.10 MB/sec warmup 75 sec latency 884.230 ms 2023-07-21T18:56:02.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:56:02 smithi039 ceph-mon[96489]: pgmap v448: 97 pgs: 97 active+clean; 661 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 11 MiB/s wr, 916 op/s 2023-07-21T18:56:03.527 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.516+0000 7f1fd3426700 1 -- 172.21.15.39:0/4044608464 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1fcc067c10 msgr2=0x7f1fcc067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:56:03.527 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.516+0000 7f1fd3426700 1 --2- 172.21.15.39:0/4044608464 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1fcc067c10 0x7f1fcc067fe0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1fc8009990 tx=0x7f1fc8009ca0 comp rx=0 tx=0).stop 2023-07-21T18:56:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.516+0000 7f1fd3426700 1 -- 172.21.15.39:0/4044608464 shutdown_connections 2023-07-21T18:56:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.516+0000 7f1fd3426700 1 --2- 172.21.15.39:0/4044608464 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1fcc104e50 0x7f1fcc10ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.516+0000 7f1fd3426700 1 --2- 172.21.15.39:0/4044608464 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1fcc067c10 0x7f1fcc067fe0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.516+0000 7f1fd3426700 1 --2- 172.21.15.39:0/4044608464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fcc0685b0 0x7f1fcc104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.516+0000 7f1fd3426700 1 -- 172.21.15.39:0/4044608464 >> 172.21.15.39:0/4044608464 conn(0x7f1fcc074760 msgr2=0x7f1fcc074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:56:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.516+0000 7f1fd3426700 1 -- 172.21.15.39:0/4044608464 shutdown_connections 2023-07-21T18:56:03.529 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.516+0000 7f1fd3426700 1 -- 172.21.15.39:0/4044608464 wait complete. 2023-07-21T18:56:03.529 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.517+0000 7f1fd3426700 1 Processor -- start 2023-07-21T18:56:03.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.517+0000 7f1fd3426700 1 -- start start 2023-07-21T18:56:03.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.518+0000 7f1fd3426700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1fcc067c10 0x7f1fcc19bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:03.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.518+0000 7f1fd3426700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1fcc0685b0 0x7f1fcc19c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:03.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.518+0000 7f1fd3426700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fcc104e50 0x7f1fcc195a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:03.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.518+0000 7f1fd3426700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc1960d0 con 0x7f1fcc0685b0 2023-07-21T18:56:03.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.518+0000 7f1fd3426700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc196240 con 0x7f1fcc104e50 2023-07-21T18:56:03.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.518+0000 7f1fd3426700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc196400 con 0x7f1fcc067c10 2023-07-21T18:56:03.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.518+0000 7f1fd11c2700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1fcc067c10 0x7f1fcc19bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:03.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.518+0000 7f1fd11c2700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1fcc067c10 0x7f1fcc19bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:51774/0 (socket says 172.21.15.39:51774) 2023-07-21T18:56:03.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.518+0000 7f1fd11c2700 1 -- 172.21.15.39:0/3277520874 learned_addr learned my addr 172.21.15.39:0/3277520874 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:56:03.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.518+0000 7f1fd19c3700 1 --2- 172.21.15.39:0/3277520874 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fcc104e50 0x7f1fcc195a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:03.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.518+0000 7f1fd11c2700 1 -- 172.21.15.39:0/3277520874 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fcc104e50 msgr2=0x7f1fcc195a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:56:03.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.518+0000 7f1fd11c2700 1 --2- 172.21.15.39:0/3277520874 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fcc104e50 0x7f1fcc195a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:03.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.518+0000 7f1fd11c2700 1 -- 172.21.15.39:0/3277520874 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1fcc0685b0 msgr2=0x7f1fcc19c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:56:03.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.518+0000 7f1fd11c2700 1 --2- 172.21.15.39:0/3277520874 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1fcc0685b0 0x7f1fcc19c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:03.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.518+0000 7f1fd11c2700 1 -- 172.21.15.39:0/3277520874 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fc4009c40 con 0x7f1fcc067c10 2023-07-21T18:56:03.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.518+0000 7f1fd19c3700 1 --2- 172.21.15.39:0/3277520874 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fcc104e50 0x7f1fcc195a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:56:03.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.519+0000 7f1fd11c2700 1 --2- 172.21.15.39:0/3277520874 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1fcc067c10 0x7f1fcc19bc00 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1fc8005910 tx=0x7f1fc80047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:56:03.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.519+0000 7f1fc27fc700 1 -- 172.21.15.39:0/3277520874 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc801d070 con 0x7f1fcc067c10 2023-07-21T18:56:03.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.519+0000 7f1fd3426700 1 -- 172.21.15.39:0/3277520874 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fc80095f0 con 0x7f1fcc067c10 2023-07-21T18:56:03.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.519+0000 7f1fd3426700 1 -- 172.21.15.39:0/3277520874 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fcc0fe990 con 0x7f1fcc067c10 2023-07-21T18:56:03.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.519+0000 7f1fc27fc700 1 -- 172.21.15.39:0/3277520874 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1fc8022470 con 0x7f1fcc067c10 2023-07-21T18:56:03.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.520+0000 7f1fc27fc700 1 -- 172.21.15.39:0/3277520874 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc8017650 con 0x7f1fcc067c10 2023-07-21T18:56:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.524+0000 7f1fc27fc700 1 -- 172.21.15.39:0/3277520874 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f1fc800fb10 con 0x7f1fcc067c10 2023-07-21T18:56:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.524+0000 7f1fdf5acb80 1 Processor -- start 2023-07-21T18:56:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.524+0000 7f1fdf5acb80 1 -- start start 2023-07-21T18:56:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.525+0000 7f1fdf5acb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4780316b0 0x55d4781dad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.525+0000 7f1fdf5acb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4781db2d0 0x55d4781d5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.525+0000 7f1fdf5acb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d4781d5580 0x55d4781d59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.525+0000 7f1fdf5acb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4780dac50 con 0x55d4781db2d0 2023-07-21T18:56:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.525+0000 7f1fdf5acb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d47812d9e0 con 0x55d4780316b0 2023-07-21T18:56:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.525+0000 7f1fdf5acb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d478126e70 con 0x55d4781d5580 2023-07-21T18:56:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.525+0000 7f1fc27fc700 1 --2- 172.21.15.39:0/3277520874 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f1fb80a12c0 0x7f1fb80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.525+0000 7f1fc27fc700 1 -- 172.21.15.39:0/3277520874 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f1fc80cdf90 con 0x7f1fcc067c10 2023-07-21T18:56:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.526+0000 7f1fd09c1700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d4781d5580 0x55d4781d59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.526+0000 7f1fd09c1700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d4781d5580 0x55d4781d59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:51784/0 (socket says 172.21.15.39:51784) 2023-07-21T18:56:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.526+0000 7f1fd09c1700 1 -- 172.21.15.39:0/290148505 learned_addr learned my addr 172.21.15.39:0/290148505 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:56:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.526+0000 7f1fd11c2700 1 --2- 172.21.15.39:0/290148505 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4781db2d0 0x55d4781d5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.526+0000 7f1fd09c1700 1 -- 172.21.15.39:0/290148505 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4780316b0 msgr2=0x55d4781dad90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T18:56:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.526+0000 7f1fd09c1700 1 --2- 172.21.15.39:0/290148505 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4780316b0 0x55d4781dad90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.526+0000 7f1fd09c1700 1 -- 172.21.15.39:0/290148505 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4781db2d0 msgr2=0x55d4781d5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:56:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.526+0000 7f1fd09c1700 1 --2- 172.21.15.39:0/290148505 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4781db2d0 0x55d4781d5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.526+0000 7f1fd09c1700 1 -- 172.21.15.39:0/290148505 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d477f54d80 con 0x55d4781d5580 2023-07-21T18:56:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.526+0000 7f1fd09c1700 1 --2- 172.21.15.39:0/290148505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d4781d5580 0x55d4781d59d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc009c10 tx=0x7f1fbc009f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aabf39b435d0865d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:56:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.527+0000 7f1fd19c3700 1 --2- 172.21.15.39:0/3277520874 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f1fb80a12c0 0x7f1fb80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:03.540 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.527+0000 7f1fb77fe700 1 -- 172.21.15.39:0/290148505 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc00ebb0 con 0x55d4781d5580 2023-07-21T18:56:03.540 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.527+0000 7f1fb77fe700 1 -- 172.21.15.39:0/290148505 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1fbc0044e0 con 0x55d4781d5580 2023-07-21T18:56:03.540 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.527+0000 7f1fb77fe700 1 -- 172.21.15.39:0/290148505 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc005280 con 0x55d4781d5580 2023-07-21T18:56:03.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.528+0000 7f1fd19c3700 1 --2- 172.21.15.39:0/3277520874 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f1fb80a12c0 0x7f1fb80a3770 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4009c10 tx=0x7f1fc400b040 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:56:03.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.528+0000 7f1fdf5acb80 1 -- 172.21.15.39:0/290148505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d4781d5580 msgr2=0x55d4781d59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:56:03.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.528+0000 7f1fdf5acb80 1 --2- 172.21.15.39:0/290148505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d4781d5580 0x55d4781d59d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc009c10 tx=0x7f1fbc009f20 comp rx=0 tx=0).stop 2023-07-21T18:56:03.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.529+0000 7f1fdf5acb80 1 -- 172.21.15.39:0/290148505 shutdown_connections 2023-07-21T18:56:03.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.529+0000 7f1fdf5acb80 1 --2- 172.21.15.39:0/290148505 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4780316b0 0x55d4781dad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:03.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.529+0000 7f1fdf5acb80 1 --2- 172.21.15.39:0/290148505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d4781d5580 0x55d4781d59d0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:03.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.529+0000 7f1fdf5acb80 1 --2- 172.21.15.39:0/290148505 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4781db2d0 0x55d4781d5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:03.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.529+0000 7f1fdf5acb80 1 -- 172.21.15.39:0/290148505 >> 172.21.15.39:0/290148505 conn(0x55d4781cabe0 msgr2=0x55d4781cff10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:56:03.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.530+0000 7f1fdf5acb80 1 -- 172.21.15.39:0/290148505 shutdown_connections 2023-07-21T18:56:03.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.530+0000 7f1fdf5acb80 1 -- 172.21.15.39:0/290148505 wait complete. 2023-07-21T18:56:03.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.530+0000 7f1fdf5acb80 1 -- start start 2023-07-21T18:56:03.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.530+0000 7f1fdf5acb80 1 Processor -- start 2023-07-21T18:56:03.544 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.531+0000 7f1fdf5acb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4780316b0 0x55d4781e14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:03.544 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.531+0000 7f1fdf5acb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d4781d5580 0x55d4781de790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:03.544 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.531+0000 7f1fdf5acb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4781db2d0 0x55d4781ded10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:03.544 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.531+0000 7f1fdf5acb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4780c6a60 con 0x55d4781db2d0 2023-07-21T18:56:03.544 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.531+0000 7f1fdf5acb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4780c6760 con 0x55d4780316b0 2023-07-21T18:56:03.544 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.531+0000 7f1fdf5acb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d477faa8c0 con 0x55d4781d5580 2023-07-21T18:56:03.545 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.531+0000 7f1fdf5acb80 10 client.0 _ll_register_callbacks cb 0x55d4780c4700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:56:03.545 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.531+0000 7f1fd19c3700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4781db2d0 0x55d4781ded10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:03.545 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.531+0000 7f1fd11c2700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4780316b0 0x55d4781e14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:03.545 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.531+0000 7f1fd11c2700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4780316b0 0x55d4781e14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:45832/0 (socket says 172.21.15.39:45832) 2023-07-21T18:56:03.545 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.531+0000 7f1fd19c3700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4781db2d0 0x55d4781ded10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33238/0 (socket says 172.21.15.39:33238) 2023-07-21T18:56:03.545 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.531+0000 7f1fd11c2700 1 -- 172.21.15.39:0/1823915606 learned_addr learned my addr 172.21.15.39:0/1823915606 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:56:03.545 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.532+0000 7f1fd19c3700 1 -- 172.21.15.39:0/1823915606 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d4781d5580 msgr2=0x55d4781de790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T18:56:03.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.532+0000 7f1fd19c3700 1 --2- 172.21.15.39:0/1823915606 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d4781d5580 0x55d4781de790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:03.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.532+0000 7f1fd19c3700 1 -- 172.21.15.39:0/1823915606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4780316b0 msgr2=0x55d4781e14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:56:03.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.532+0000 7f1fd19c3700 1 --2- 172.21.15.39:0/1823915606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4780316b0 0x55d4781e14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:03.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.532+0000 7f1fd19c3700 1 -- 172.21.15.39:0/1823915606 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fbc0098c0 con 0x55d4781db2d0 2023-07-21T18:56:03.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.532+0000 7f1fd19c3700 1 --2- 172.21.15.39:0/1823915606 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4781db2d0 0x55d4781ded10 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc19c900 tx=0x7f1fc4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=289c7e8bd2006f6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:56:03.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.532+0000 7f1fb4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:56:03.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.532+0000 7f1fb4ff9700 1 -- 172.21.15.39:0/1823915606 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc400e050 con 0x55d4781db2d0 2023-07-21T18:56:03.547 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.532+0000 7f1fb4ff9700 1 -- 172.21.15.39:0/1823915606 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f1fc40091f0 con 0x55d4781db2d0 2023-07-21T18:56:03.547 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.532+0000 7f1fdf5acb80 1 -- 172.21.15.39:0/1823915606 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d478080490 con 0x55d4781db2d0 2023-07-21T18:56:03.547 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.532+0000 7f1fb4ff9700 1 -- 172.21.15.39:0/1823915606 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1fc4016e20 con 0x55d4781db2d0 2023-07-21T18:56:03.547 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.532+0000 7f1fdf5acb80 10 client.17244 fetch_fsmap learned FSMap version 84 2023-07-21T18:56:03.547 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.532+0000 7f1fdf5acb80 1 -- 172.21.15.39:0/1823915606 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) v3 -- 0x55d477f3dc00 con 0x55d4781db2d0 2023-07-21T18:56:03.547 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.533+0000 7f1fb4ff9700 1 -- 172.21.15.39:0/1823915606 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d478080490 con 0x55d4781db2d0 2023-07-21T18:56:03.547 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.533+0000 7f1fb4ff9700 1 -- 172.21.15.39:0/1823915606 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 84) v1 ==== 7538+0+0 (secure 0 0 0) 0x7f1fc402a710 con 0x55d4781db2d0 2023-07-21T18:56:03.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.533+0000 7f1fdf5acb80 10 client.17244 fetch_fsmap finished waiting for FSMap version 84 2023-07-21T18:56:03.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.533+0000 7f1fdf5acb80 10 client.17244 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:56:03.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.533+0000 7f1fdf5acb80 20 client.17244 populate_metadata read hostname 'smithi039' 2023-07-21T18:56:03.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.533+0000 7f1fdf5acb80 1 --2- 172.21.15.39:0/1823915606 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d4781eed40 0x55d4781f1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:03.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.533+0000 7f1fdf5acb80 4 client.17244 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:56:03.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.533+0000 7f1fdf5acb80 1 -- 172.21.15.39:0/1823915606 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d47811fe50 con 0x55d4781eed40 2023-07-21T18:56:03.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.533+0000 7f1fd11c2700 1 --2- 172.21.15.39:0/1823915606 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d4781eed40 0x55d4781f1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.534+0000 7f1fb4ff9700 10 client.17244 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:56:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.534+0000 7f1fd11c2700 1 --2- 172.21.15.39:0/1823915606 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d4781eed40 0x55d4781f1150 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46d8a036549b40e0 server_cookie=7fc789bd8b2bd63d in_seq=0 out_seq=0 2023-07-21T18:56:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.538+0000 7f1fb4ff9700 1 -- 172.21.15.39:0/1823915606 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d47811fe50 con 0x55d4781eed40 2023-07-21T18:56:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.538+0000 7f1fb4ff9700 10 client.17244 handle_command_reply: tid=0 2023-07-21T18:56:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.538+0000 7f1fdf5acb80 1 client.17244 shutdown 2023-07-21T18:56:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.538+0000 7f1fdf5acb80 1 -- 172.21.15.39:0/1823915606 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4781db2d0 msgr2=0x55d4781ded10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:56:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.538+0000 7f1fdf5acb80 1 --2- 172.21.15.39:0/1823915606 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4781db2d0 0x55d4781ded10 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc19c900 tx=0x7f1fc4004ad0 comp rx=0 tx=0).stop 2023-07-21T18:56:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.538+0000 7f1fdf5acb80 1 -- 172.21.15.39:0/1823915606 shutdown_connections 2023-07-21T18:56:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.538+0000 7f1fdf5acb80 1 --2- 172.21.15.39:0/1823915606 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d4781eed40 0x55d4781f1150 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.538+0000 7f1fdf5acb80 1 --2- 172.21.15.39:0/1823915606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4780316b0 0x55d4781e14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.538+0000 7f1fb4ff9700 0 client.17244 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:56:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.538+0000 7f1fdf5acb80 1 --2- 172.21.15.39:0/1823915606 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4781db2d0 0x55d4781ded10 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.538+0000 7f1fdf5acb80 1 --2- 172.21.15.39:0/1823915606 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d4781d5580 0x55d4781de790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.538+0000 7f1fdf5acb80 1 -- 172.21.15.39:0/1823915606 >> 172.21.15.39:0/1823915606 conn(0x55d4781cabe0 msgr2=0x55d4781cec50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:56:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.538+0000 7f1fdf5acb80 1 -- 172.21.15.39:0/1823915606 shutdown_connections 2023-07-21T18:56:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.539+0000 7f1fdf5acb80 1 -- 172.21.15.39:0/1823915606 wait complete. 2023-07-21T18:56:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.539+0000 7f1fdf5acb80 20 client.17244 trim_cache size 0 max 16384 2023-07-21T18:56:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.562+0000 7f1fdf5acb80 1 Processor -- start 2023-07-21T18:56:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.562+0000 7f1fdf5acb80 1 -- start start 2023-07-21T18:56:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.562+0000 7f1fdf5acb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4780316b0 0x55d4781d5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.562+0000 7f1fdf5acb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4781d5580 0x55d4781e5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.562+0000 7f1fdf5acb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d4781db2d0 0x55d4781cfc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:03.574 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.562+0000 7f1fdf5acb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d478080490 con 0x55d4780316b0 2023-07-21T18:56:03.574 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.562+0000 7f1fdf5acb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4781c7bb0 con 0x55d4781d5580 2023-07-21T18:56:03.574 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.562+0000 7f1fdf5acb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4781dccf0 con 0x55d4781db2d0 2023-07-21T18:56:03.575 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.562+0000 7f1fd11c2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4780316b0 0x55d4781d5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:03.575 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.563+0000 7f1fd11c2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4780316b0 0x55d4781d5040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33242/0 (socket says 172.21.15.39:33242) 2023-07-21T18:56:03.575 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.563+0000 7f1fd11c2700 1 -- 172.21.15.39:0/2899430786 learned_addr learned my addr 172.21.15.39:0/2899430786 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:56:03.576 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.563+0000 7f1fd19c3700 1 --2- 172.21.15.39:0/2899430786 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d4781db2d0 0x55d4781cfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:03.576 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.563+0000 7f1fd09c1700 1 --2- 172.21.15.39:0/2899430786 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4781d5580 0x55d4781e5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:03.576 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.563+0000 7f1fd11c2700 1 -- 172.21.15.39:0/2899430786 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d4781db2d0 msgr2=0x55d4781cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:56:03.576 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.563+0000 7f1fd11c2700 1 --2- 172.21.15.39:0/2899430786 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d4781db2d0 0x55d4781cfc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:03.576 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.563+0000 7f1fd11c2700 1 -- 172.21.15.39:0/2899430786 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4781d5580 msgr2=0x55d4781e5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:56:03.576 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.563+0000 7f1fd11c2700 1 --2- 172.21.15.39:0/2899430786 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4781d5580 0x55d4781e5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:03.576 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.563+0000 7f1fd11c2700 1 -- 172.21.15.39:0/2899430786 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d4781f5d80 con 0x55d4780316b0 2023-07-21T18:56:03.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.563+0000 7f1fd19c3700 1 --2- 172.21.15.39:0/2899430786 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d4781db2d0 0x55d4781cfc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:56:03.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.563+0000 7f1fd09c1700 1 --2- 172.21.15.39:0/2899430786 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4781d5580 0x55d4781e5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:56:03.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.563+0000 7f1fd11c2700 1 --2- 172.21.15.39:0/2899430786 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4780316b0 0x55d4781d5040 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f1fc809adf0 tx=0x7f1fc8009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bac14046dfc3b647 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:56:03.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.563+0000 7f1fb7fff700 1 -- 172.21.15.39:0/2899430786 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc80a34b0 con 0x55d4780316b0 2023-07-21T18:56:03.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.563+0000 7f1fb7fff700 1 -- 172.21.15.39:0/2899430786 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1fc800fdc0 con 0x55d4780316b0 2023-07-21T18:56:03.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.564+0000 7f1fb7fff700 1 -- 172.21.15.39:0/2899430786 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc809a5e0 con 0x55d4780316b0 2023-07-21T18:56:03.578 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.564+0000 7f1fdf5acb80 1 -- 172.21.15.39:0/2899430786 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4780316b0 msgr2=0x55d4781d5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:56:03.578 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.564+0000 7f1fdf5acb80 1 --2- 172.21.15.39:0/2899430786 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4780316b0 0x55d4781d5040 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f1fc809adf0 tx=0x7f1fc8009ca0 comp rx=0 tx=0).stop 2023-07-21T18:56:03.578 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.565+0000 7f1fdf5acb80 1 -- 172.21.15.39:0/2899430786 shutdown_connections 2023-07-21T18:56:03.578 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.565+0000 7f1fdf5acb80 1 --2- 172.21.15.39:0/2899430786 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d4781db2d0 0x55d4781cfc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:03.578 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.565+0000 7f1fdf5acb80 1 --2- 172.21.15.39:0/2899430786 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4780316b0 0x55d4781d5040 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:03.578 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.565+0000 7f1fdf5acb80 1 --2- 172.21.15.39:0/2899430786 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4781d5580 0x55d4781e5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.565+0000 7f1fdf5acb80 1 -- 172.21.15.39:0/2899430786 >> 172.21.15.39:0/2899430786 conn(0x55d4781cabe0 msgr2=0x55d4781d8990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:56:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.565+0000 7f1fdf5acb80 1 -- 172.21.15.39:0/2899430786 shutdown_connections 2023-07-21T18:56:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.565+0000 7f1fdf5acb80 1 -- 172.21.15.39:0/2899430786 wait complete. 2023-07-21T18:56:03.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.565+0000 7f1fdf5acb80 1 -- start start 2023-07-21T18:56:03.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.565+0000 7f1fdf5acb80 1 Processor -- start 2023-07-21T18:56:03.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.566+0000 7f1fdf5acb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d4780316b0 0x55d4781e3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:03.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.566+0000 7f1fdf5acb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4781d5580 0x55d4781e4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:03.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.566+0000 7f1fdf5acb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4781db2d0 0x55d4781e46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:03.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.566+0000 7f1fdf5acb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d478241c40 con 0x55d4781db2d0 2023-07-21T18:56:03.582 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.566+0000 7f1fdf5acb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d478241db0 con 0x55d4781d5580 2023-07-21T18:56:03.582 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.566+0000 7f1fdf5acb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d478241f70 con 0x55d4780316b0 2023-07-21T18:56:03.582 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.566+0000 7f1fdf5acb80 10 client.0 _ll_register_callbacks cb 0x55d4780c4700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:56:03.582 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.566+0000 7f1fd19c3700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4781db2d0 0x55d4781e46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:03.582 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.566+0000 7f1fd11c2700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d4780316b0 0x55d4781e3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:03.582 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.566+0000 7f1fd19c3700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4781db2d0 0x55d4781e46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33252/0 (socket says 172.21.15.39:33252) 2023-07-21T18:56:03.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.566+0000 7f1fd11c2700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d4780316b0 0x55d4781e3c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:51800/0 (socket says 172.21.15.39:51800) 2023-07-21T18:56:03.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.566+0000 7f1fd19c3700 1 -- 172.21.15.39:0/2255598881 learned_addr learned my addr 172.21.15.39:0/2255598881 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:56:03.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.566+0000 7f1fd09c1700 1 --2- 172.21.15.39:0/2255598881 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4781d5580 0x55d4781e4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:03.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.567+0000 7f1fd19c3700 1 -- 172.21.15.39:0/2255598881 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d4780316b0 msgr2=0x55d4781e3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:56:03.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.567+0000 7f1fd19c3700 1 --2- 172.21.15.39:0/2255598881 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d4780316b0 0x55d4781e3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:03.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.567+0000 7f1fd19c3700 1 -- 172.21.15.39:0/2255598881 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4781d5580 msgr2=0x55d4781e4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:56:03.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.567+0000 7f1fd19c3700 1 --2- 172.21.15.39:0/2255598881 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4781d5580 0x55d4781e4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.567+0000 7f1fd19c3700 1 -- 172.21.15.39:0/2255598881 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fcc0fe990 con 0x55d4781db2d0 2023-07-21T18:56:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.567+0000 7f1fd19c3700 1 --2- 172.21.15.39:0/2255598881 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4781db2d0 0x55d4781e46d0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f1fc402aa40 tx=0x7f1fc4009eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbe4de876e0b7e2f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:56:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.567+0000 7f1fb4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:56:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.567+0000 7f1fb4ff9700 1 -- 172.21.15.39:0/2255598881 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc400e070 con 0x55d4781db2d0 2023-07-21T18:56:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.567+0000 7f1fb4ff9700 1 -- 172.21.15.39:0/2255598881 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f1fc4004b40 con 0x55d4781db2d0 2023-07-21T18:56:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.567+0000 7f1fdf5acb80 1 -- 172.21.15.39:0/2255598881 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1fbc0044e0 con 0x55d4781db2d0 2023-07-21T18:56:03.585 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.567+0000 7f1fb4ff9700 1 -- 172.21.15.39:0/2255598881 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1fc4016a60 con 0x55d4781db2d0 2023-07-21T18:56:03.585 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.567+0000 7f1fdf5acb80 10 client.17250 fetch_fsmap learned FSMap version 84 2023-07-21T18:56:03.585 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.567+0000 7f1fdf5acb80 1 -- 172.21.15.39:0/2255598881 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) v3 -- 0x55d477f7b1f0 con 0x55d4781db2d0 2023-07-21T18:56:03.585 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.568+0000 7f1fb4ff9700 1 -- 172.21.15.39:0/2255598881 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1fbc0044e0 con 0x55d4781db2d0 2023-07-21T18:56:03.585 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.568+0000 7f1fb4ff9700 1 -- 172.21.15.39:0/2255598881 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 84) v1 ==== 7538+0+0 (secure 0 0 0) 0x7f1fc4005430 con 0x55d4781db2d0 2023-07-21T18:56:03.585 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.568+0000 7f1fdf5acb80 10 client.17250 fetch_fsmap finished waiting for FSMap version 84 2023-07-21T18:56:03.585 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.568+0000 7f1fdf5acb80 10 client.17250 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:56:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.568+0000 7f1fdf5acb80 20 client.17250 populate_metadata read hostname 'smithi039' 2023-07-21T18:56:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.568+0000 7f1fdf5acb80 1 --2- 172.21.15.39:0/2255598881 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d4781eed40 0x55d4782456d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.568+0000 7f1fdf5acb80 4 client.17250 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:56:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.568+0000 7f1fdf5acb80 1 -- 172.21.15.39:0/2255598881 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d4780e1ff0 con 0x55d4781eed40 2023-07-21T18:56:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.568+0000 7f1fd11c2700 1 --2- 172.21.15.39:0/2255598881 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d4781eed40 0x55d4782456d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.569+0000 7f1fb4ff9700 10 client.17250 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:56:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.569+0000 7f1fd11c2700 1 --2- 172.21.15.39:0/2255598881 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d4781eed40 0x55d4782456d0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56605ee9c86e8cd7 server_cookie=4ab8b2011c7b4a88 in_seq=0 out_seq=0 2023-07-21T18:56:03.661 INFO:tasks.workunit.client.0.smithi039.stdout: 10 37499 28.94 MB/sec warmup 76 sec latency 1604.727 ms 2023-07-21T18:56:03.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.727+0000 7f1fb4ff9700 1 -- 172.21.15.39:0/2255598881 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d4780e1ff0 con 0x55d4781eed40 2023-07-21T18:56:03.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.727+0000 7f1fb4ff9700 10 client.17250 handle_command_reply: tid=0 2023-07-21T18:56:03.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.727+0000 7f1fdf5acb80 1 client.17250 shutdown 2023-07-21T18:56:03.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.727+0000 7f1fdf5acb80 1 -- 172.21.15.39:0/2255598881 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4781db2d0 msgr2=0x55d4781e46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:56:03.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.727+0000 7f1fdf5acb80 1 --2- 172.21.15.39:0/2255598881 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4781db2d0 0x55d4781e46d0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f1fc402aa40 tx=0x7f1fc4009eb0 comp rx=0 tx=0).stop 2023-07-21T18:56:03.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.727+0000 7f1fdf5acb80 1 -- 172.21.15.39:0/2255598881 shutdown_connections 2023-07-21T18:56:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.727+0000 7f1fdf5acb80 1 --2- 172.21.15.39:0/2255598881 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d4781eed40 0x55d4782456d0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.727+0000 7f1fdf5acb80 1 --2- 172.21.15.39:0/2255598881 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4781db2d0 0x55d4781e46d0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.727+0000 7f1fb4ff9700 0 client.17250 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:56:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.727+0000 7f1fdf5acb80 1 --2- 172.21.15.39:0/2255598881 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d4780316b0 0x55d4781e3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.727+0000 7f1fdf5acb80 1 --2- 172.21.15.39:0/2255598881 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4781d5580 0x55d4781e4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.727+0000 7f1fdf5acb80 1 -- 172.21.15.39:0/2255598881 >> 172.21.15.39:0/2255598881 conn(0x55d4781cabe0 msgr2=0x55d4781e0030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:56:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.727+0000 7f1fdf5acb80 1 -- 172.21.15.39:0/2255598881 shutdown_connections 2023-07-21T18:56:03.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.727+0000 7f1fdf5acb80 1 -- 172.21.15.39:0/2255598881 wait complete. 2023-07-21T18:56:03.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.727+0000 7f1fdf5acb80 20 client.17250 trim_cache size 0 max 16384 2023-07-21T18:56:03.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:56:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:56:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:56:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:56:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.728+0000 7f1fd3426700 1 -- 172.21.15.39:0/3277520874 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f1fb80a12c0 msgr2=0x7f1fb80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:56:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.728+0000 7f1fd3426700 1 --2- 172.21.15.39:0/3277520874 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f1fb80a12c0 0x7f1fb80a3770 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4009c10 tx=0x7f1fc400b040 comp rx=0 tx=0).stop 2023-07-21T18:56:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.728+0000 7f1fd3426700 1 -- 172.21.15.39:0/3277520874 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1fcc067c10 msgr2=0x7f1fcc19bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:56:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.728+0000 7f1fd3426700 1 --2- 172.21.15.39:0/3277520874 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1fcc067c10 0x7f1fcc19bc00 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1fc8005910 tx=0x7f1fc80047c0 comp rx=0 tx=0).stop 2023-07-21T18:56:03.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.728+0000 7f1fd3426700 1 -- 172.21.15.39:0/3277520874 shutdown_connections 2023-07-21T18:56:03.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.728+0000 7f1fd3426700 1 --2- 172.21.15.39:0/3277520874 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f1fb80a12c0 0x7f1fb80a3770 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:03.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.728+0000 7f1fd3426700 1 --2- 172.21.15.39:0/3277520874 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fcc104e50 0x7f1fcc195a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:03.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.728+0000 7f1fd3426700 1 --2- 172.21.15.39:0/3277520874 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1fcc067c10 0x7f1fcc19bc00 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:03.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.728+0000 7f1fd3426700 1 --2- 172.21.15.39:0/3277520874 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1fcc0685b0 0x7f1fcc19c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:03.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.728+0000 7f1fd3426700 1 -- 172.21.15.39:0/3277520874 >> 172.21.15.39:0/3277520874 conn(0x7f1fcc074760 msgr2=0x7f1fcc0fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:56:03.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.728+0000 7f1fd3426700 1 -- 172.21.15.39:0/3277520874 shutdown_connections 2023-07-21T18:56:03.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:03.728+0000 7f1fd3426700 1 -- 172.21.15.39:0/3277520874 wait complete. 2023-07-21T18:56:04.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:56:04 smithi193 ceph-mon[98204]: pgmap v449: 97 pgs: 97 active+clean; 661 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 8.2 MiB/s wr, 602 op/s 2023-07-21T18:56:04.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:56:04 smithi155 ceph-mon[97934]: pgmap v449: 97 pgs: 97 active+clean; 661 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 8.2 MiB/s wr, 602 op/s 2023-07-21T18:56:04.661 INFO:tasks.workunit.client.0.smithi039.stdout: 10 38182 28.95 MB/sec warmup 77 sec latency 107.418 ms 2023-07-21T18:56:04.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:56:04 smithi039 ceph-mon[96489]: pgmap v449: 97 pgs: 97 active+clean; 661 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 8.2 MiB/s wr, 602 op/s 2023-07-21T18:56:05.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:56:05 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:56:05.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:56:05 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:56:05.662 INFO:tasks.workunit.client.0.smithi039.stdout: 10 38858 29.07 MB/sec warmup 78 sec latency 49.692 ms 2023-07-21T18:56:05.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:56:05 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:56:06.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:56:06 smithi193 ceph-mon[98204]: pgmap v450: 97 pgs: 97 active+clean; 661 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 8.2 MiB/s wr, 602 op/s 2023-07-21T18:56:06.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:56:06 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:56:06.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:56:06 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:56:06.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:56:06 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:56:06.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:56:06 smithi155 ceph-mon[97934]: pgmap v450: 97 pgs: 97 active+clean; 661 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 8.2 MiB/s wr, 602 op/s 2023-07-21T18:56:06.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:56:06 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:56:06.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:56:06 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:56:06.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:56:06 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:56:06.662 INFO:tasks.workunit.client.0.smithi039.stdout: 10 39085 28.84 MB/sec warmup 79 sec latency 693.071 ms 2023-07-21T18:56:06.680 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:56:06 smithi039 ceph-mon[96489]: pgmap v450: 97 pgs: 97 active+clean; 661 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 8.2 MiB/s wr, 602 op/s 2023-07-21T18:56:06.680 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:56:06 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:56:06.681 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:56:06 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:56:06.681 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:56:06 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:56:07.662 INFO:tasks.workunit.client.0.smithi039.stdout: 10 39145 28.51 MB/sec warmup 80 sec latency 1596.093 ms 2023-07-21T18:56:08.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:56:08 smithi193 ceph-mon[98204]: pgmap v451: 97 pgs: 97 active+clean; 665 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 12 MiB/s wr, 911 op/s 2023-07-21T18:56:08.631 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:56:08 smithi155 ceph-mon[97934]: pgmap v451: 97 pgs: 97 active+clean; 665 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 12 MiB/s wr, 911 op/s 2023-07-21T18:56:08.662 INFO:tasks.workunit.client.0.smithi039.stdout: 10 39603 28.43 MB/sec warmup 81 sec latency 432.912 ms 2023-07-21T18:56:08.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:56:08 smithi039 ceph-mon[96489]: pgmap v451: 97 pgs: 97 active+clean; 665 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 12 MiB/s wr, 911 op/s 2023-07-21T18:56:09.662 INFO:tasks.workunit.client.0.smithi039.stdout: 10 40198 28.43 MB/sec warmup 82 sec latency 86.942 ms 2023-07-21T18:56:10.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:56:10 smithi155 ceph-mon[97934]: pgmap v452: 97 pgs: 97 active+clean; 665 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 8.7 MiB/s wr, 635 op/s 2023-07-21T18:56:10.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:56:10 smithi155 ceph-mon[97934]: osdmap e100: 12 total, 12 up, 12 in 2023-07-21T18:56:10.662 INFO:tasks.workunit.client.0.smithi039.stdout: 10 40936 28.73 MB/sec warmup 83 sec latency 96.844 ms 2023-07-21T18:56:10.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:56:10 smithi039 ceph-mon[96489]: pgmap v452: 97 pgs: 97 active+clean; 665 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 8.7 MiB/s wr, 635 op/s 2023-07-21T18:56:10.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:56:10 smithi039 ceph-mon[96489]: osdmap e100: 12 total, 12 up, 12 in 2023-07-21T18:56:10.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:56:10 smithi193 ceph-mon[98204]: pgmap v452: 97 pgs: 97 active+clean; 665 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 8.7 MiB/s wr, 635 op/s 2023-07-21T18:56:10.859 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:56:10 smithi193 ceph-mon[98204]: osdmap e100: 12 total, 12 up, 12 in 2023-07-21T18:56:11.662 INFO:tasks.workunit.client.0.smithi039.stdout: 10 41586 28.85 MB/sec warmup 84 sec latency 58.457 ms 2023-07-21T18:56:12.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:56:12 smithi155 ceph-mon[97934]: pgmap v454: 97 pgs: 97 active+clean; 693 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 27 MiB/s wr, 1.20k op/s 2023-07-21T18:56:12.662 INFO:tasks.workunit.client.0.smithi039.stdout: 10 42128 28.73 MB/sec warmup 85 sec latency 66.669 ms 2023-07-21T18:56:12.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:56:12 smithi039 ceph-mon[96489]: pgmap v454: 97 pgs: 97 active+clean; 693 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 27 MiB/s wr, 1.20k op/s 2023-07-21T18:56:12.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:56:12 smithi193 ceph-mon[98204]: pgmap v454: 97 pgs: 97 active+clean; 693 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 27 MiB/s wr, 1.20k op/s 2023-07-21T18:56:13.662 INFO:tasks.workunit.client.0.smithi039.stdout: 10 42950 28.85 MB/sec warmup 86 sec latency 62.822 ms 2023-07-21T18:56:14.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:56:14 smithi155 ceph-mon[97934]: pgmap v455: 97 pgs: 97 active+clean; 693 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 27 MiB/s wr, 1.20k op/s 2023-07-21T18:56:14.662 INFO:tasks.workunit.client.0.smithi039.stdout: 10 43921 29.14 MB/sec warmup 87 sec latency 73.834 ms 2023-07-21T18:56:14.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:56:14 smithi039 ceph-mon[96489]: pgmap v455: 97 pgs: 97 active+clean; 693 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 27 MiB/s wr, 1.20k op/s 2023-07-21T18:56:14.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:56:14 smithi193 ceph-mon[98204]: pgmap v455: 97 pgs: 97 active+clean; 693 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 27 MiB/s wr, 1.20k op/s 2023-07-21T18:56:15.662 INFO:tasks.workunit.client.0.smithi039.stdout: 10 44828 29.50 MB/sec warmup 88 sec latency 35.794 ms 2023-07-21T18:56:16.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:56:16 smithi155 ceph-mon[97934]: pgmap v456: 97 pgs: 97 active+clean; 693 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 27 MiB/s wr, 1.20k op/s 2023-07-21T18:56:16.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:56:16 smithi155 ceph-mon[97934]: osdmap e101: 12 total, 12 up, 12 in 2023-07-21T18:56:16.662 INFO:tasks.workunit.client.0.smithi039.stdout: 10 45392 29.49 MB/sec warmup 89 sec latency 91.866 ms 2023-07-21T18:56:16.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:56:16 smithi039 ceph-mon[96489]: pgmap v456: 97 pgs: 97 active+clean; 693 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 27 MiB/s wr, 1.20k op/s 2023-07-21T18:56:16.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:56:16 smithi039 ceph-mon[96489]: osdmap e101: 12 total, 12 up, 12 in 2023-07-21T18:56:16.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:56:16 smithi193 ceph-mon[98204]: pgmap v456: 97 pgs: 97 active+clean; 693 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 27 MiB/s wr, 1.20k op/s 2023-07-21T18:56:16.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:56:16 smithi193 ceph-mon[98204]: osdmap e101: 12 total, 12 up, 12 in 2023-07-21T18:56:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.859+0000 7fb1f5d58700 1 -- 172.21.15.39:0/3341533506 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1f00ff3e0 msgr2=0x7fb1f010b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:56:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.859+0000 7fb1f5d58700 1 --2- 172.21.15.39:0/3341533506 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1f00ff3e0 0x7fb1f010b870 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fb1e800b0a0 tx=0x7fb1e800b3b0 comp rx=0 tx=0).stop 2023-07-21T18:56:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.859+0000 7fb1f5d58700 1 -- 172.21.15.39:0/3341533506 shutdown_connections 2023-07-21T18:56:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.859+0000 7fb1f5d58700 1 --2- 172.21.15.39:0/3341533506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1f0104a40 0x7fb1f0104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:16.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.859+0000 7fb1f5d58700 1 --2- 172.21.15.39:0/3341533506 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1f00ff3e0 0x7fb1f010b870 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:16.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.859+0000 7fb1f5d58700 1 --2- 172.21.15.39:0/3341533506 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb1f00fea30 0x7fb1f00feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:16.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.859+0000 7fb1f5d58700 1 -- 172.21.15.39:0/3341533506 >> 172.21.15.39:0/3341533506 conn(0x7fb1f00fa550 msgr2=0x7fb1f00fc960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:56:16.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.859+0000 7fb1f5d58700 1 -- 172.21.15.39:0/3341533506 shutdown_connections 2023-07-21T18:56:16.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.859+0000 7fb1f5d58700 1 -- 172.21.15.39:0/3341533506 wait complete. 2023-07-21T18:56:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.860+0000 7fb1f5d58700 1 Processor -- start 2023-07-21T18:56:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.860+0000 7fb1f5d58700 1 -- start start 2023-07-21T18:56:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.861+0000 7fb1f5d58700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb1f00fea30 0x7fb1f0197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.861+0000 7fb1f5d58700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1f00ff3e0 0x7fb1f01983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.861+0000 7fb1f5d58700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1f0104a40 0x7fb1f019ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.861+0000 7fb1f5d58700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f019d050 con 0x7fb1f00ff3e0 2023-07-21T18:56:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.861+0000 7fb1f5d58700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f019d1c0 con 0x7fb1f0104a40 2023-07-21T18:56:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.861+0000 7fb1f5d58700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f019d380 con 0x7fb1f00fea30 2023-07-21T18:56:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.861+0000 7fb1eeffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1f00ff3e0 0x7fb1f01983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.861+0000 7fb1ef7fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb1f00fea30 0x7fb1f0197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.861+0000 7fb1ef7fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb1f00fea30 0x7fb1f0197e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:51814/0 (socket says 172.21.15.39:51814) 2023-07-21T18:56:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.861+0000 7fb1eeffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1f00ff3e0 0x7fb1f01983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33264/0 (socket says 172.21.15.39:33264) 2023-07-21T18:56:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.861+0000 7fb1ef7fe700 1 -- 172.21.15.39:0/4017030894 learned_addr learned my addr 172.21.15.39:0/4017030894 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:56:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.861+0000 7fb1effff700 1 --2- 172.21.15.39:0/4017030894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1f0104a40 0x7fb1f019ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.861+0000 7fb1ef7fe700 1 -- 172.21.15.39:0/4017030894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1f0104a40 msgr2=0x7fb1f019ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:56:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.861+0000 7fb1ef7fe700 1 --2- 172.21.15.39:0/4017030894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1f0104a40 0x7fb1f019ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.861+0000 7fb1ef7fe700 1 -- 172.21.15.39:0/4017030894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1f00ff3e0 msgr2=0x7fb1f01983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:56:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.861+0000 7fb1ef7fe700 1 --2- 172.21.15.39:0/4017030894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1f00ff3e0 0x7fb1f01983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.861+0000 7fb1ef7fe700 1 -- 172.21.15.39:0/4017030894 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1e0009520 con 0x7fb1f00fea30 2023-07-21T18:56:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.862+0000 7fb1ef7fe700 1 --2- 172.21.15.39:0/4017030894 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb1f00fea30 0x7fb1f0197e60 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb1e000ea50 tx=0x7fb1e000ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:56:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.862+0000 7fb1ecff9700 1 -- 172.21.15.39:0/4017030894 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e000cbd0 con 0x7fb1f00fea30 2023-07-21T18:56:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.862+0000 7fb1ecff9700 1 -- 172.21.15.39:0/4017030894 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb1e000cd30 con 0x7fb1f00fea30 2023-07-21T18:56:16.876 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.862+0000 7fb1f5d58700 1 -- 172.21.15.39:0/4017030894 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1e8009d00 con 0x7fb1f00fea30 2023-07-21T18:56:16.876 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.862+0000 7fb1ecff9700 1 -- 172.21.15.39:0/4017030894 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e0018680 con 0x7fb1f00fea30 2023-07-21T18:56:16.876 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.862+0000 7fb1f5d58700 1 -- 172.21.15.39:0/4017030894 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1f01a4cc0 con 0x7fb1f00fea30 2023-07-21T18:56:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.865+0000 7fb1ecff9700 1 -- 172.21.15.39:0/4017030894 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fb1e0010430 con 0x7fb1f00fea30 2023-07-21T18:56:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.865+0000 7fb201edeb80 1 Processor -- start 2023-07-21T18:56:16.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.866+0000 7fb201edeb80 1 -- start start 2023-07-21T18:56:16.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.866+0000 7fb1ecff9700 1 --2- 172.21.15.39:0/4017030894 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fb1d80a12b0 0x7fb1d80a3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:16.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.866+0000 7fb201edeb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571fa2996b0 0x5571fa44b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:16.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.866+0000 7fb1effff700 1 --2- 172.21.15.39:0/4017030894 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fb1d80a12b0 0x7fb1d80a3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:16.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.866+0000 7fb201edeb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5571fa44ba80 0x5571fa445850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:16.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.866+0000 7fb1ecff9700 1 -- 172.21.15.39:0/4017030894 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fb1e0014070 con 0x7fb1f00fea30 2023-07-21T18:56:16.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.866+0000 7fb201edeb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571fa44bed0 0x5571fa445dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.866+0000 7fb201edeb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571fa342c50 con 0x5571fa44ba80 2023-07-21T18:56:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.866+0000 7fb201edeb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571fa3959e0 con 0x5571fa2996b0 2023-07-21T18:56:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.866+0000 7fb201edeb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571fa38ee70 con 0x5571fa44bed0 2023-07-21T18:56:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.866+0000 7fb1eeffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5571fa44ba80 0x5571fa445850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.866+0000 7fb1ef7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571fa2996b0 0x5571fa44b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.866+0000 7fb1effff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571fa44bed0 0x5571fa445dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.866+0000 7fb1eeffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5571fa44ba80 0x5571fa445850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33270/0 (socket says 172.21.15.39:33270) 2023-07-21T18:56:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.866+0000 7fb1eeffd700 1 -- 172.21.15.39:0/2310319373 learned_addr learned my addr 172.21.15.39:0/2310319373 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:56:16.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.866+0000 7fb1ef7fe700 1 -- 172.21.15.39:0/2310319373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571fa44bed0 msgr2=0x5571fa445dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:56:16.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.866+0000 7fb1ef7fe700 1 --2- 172.21.15.39:0/2310319373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571fa44bed0 0x5571fa445dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:16.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.866+0000 7fb1ef7fe700 1 -- 172.21.15.39:0/2310319373 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5571fa44ba80 msgr2=0x5571fa445850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:56:16.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.866+0000 7fb1ef7fe700 1 --2- 172.21.15.39:0/2310319373 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5571fa44ba80 0x5571fa445850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:16.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.866+0000 7fb1ef7fe700 1 -- 172.21.15.39:0/2310319373 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5571fa1bcd80 con 0x5571fa2996b0 2023-07-21T18:56:16.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.867+0000 7fb1effff700 1 --2- 172.21.15.39:0/4017030894 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fb1d80a12b0 0x7fb1d80a3760 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb1e8009cd0 tx=0x7fb1e800bd00 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:56:16.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.867+0000 7fb1ef7fe700 1 --2- 172.21.15.39:0/2310319373 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571fa2996b0 0x5571fa44b540 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb1e00ca900 tx=0x7fb1e0099cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e684488857386 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:56:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.867+0000 7fb1d5ffb700 1 -- 172.21.15.39:0/2310319373 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e009a280 con 0x5571fa2996b0 2023-07-21T18:56:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.867+0000 7fb1d5ffb700 1 -- 172.21.15.39:0/2310319373 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb1e0094e90 con 0x5571fa2996b0 2023-07-21T18:56:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.867+0000 7fb1d5ffb700 1 -- 172.21.15.39:0/2310319373 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e00999d0 con 0x5571fa2996b0 2023-07-21T18:56:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.868+0000 7fb201edeb80 1 -- 172.21.15.39:0/2310319373 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571fa2996b0 msgr2=0x5571fa44b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:56:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.868+0000 7fb201edeb80 1 --2- 172.21.15.39:0/2310319373 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571fa2996b0 0x5571fa44b540 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb1e00ca900 tx=0x7fb1e0099cd0 comp rx=0 tx=0).stop 2023-07-21T18:56:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.868+0000 7fb201edeb80 1 -- 172.21.15.39:0/2310319373 shutdown_connections 2023-07-21T18:56:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.868+0000 7fb201edeb80 1 --2- 172.21.15.39:0/2310319373 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571fa2996b0 0x5571fa44b540 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.868+0000 7fb201edeb80 1 --2- 172.21.15.39:0/2310319373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571fa44bed0 0x5571fa445dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.868+0000 7fb201edeb80 1 --2- 172.21.15.39:0/2310319373 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5571fa44ba80 0x5571fa445850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.868+0000 7fb201edeb80 1 -- 172.21.15.39:0/2310319373 >> 172.21.15.39:0/2310319373 conn(0x5571fa432be0 msgr2=0x5571fa437f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:56:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.868+0000 7fb201edeb80 1 -- 172.21.15.39:0/2310319373 shutdown_connections 2023-07-21T18:56:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.868+0000 7fb201edeb80 1 -- 172.21.15.39:0/2310319373 wait complete. 2023-07-21T18:56:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.869+0000 7fb201edeb80 1 -- start start 2023-07-21T18:56:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.869+0000 7fb201edeb80 1 Processor -- start 2023-07-21T18:56:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.870+0000 7fb201edeb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571fa2996b0 0x5571fa451df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.870+0000 7fb201edeb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571fa44ba80 0x5571fa44f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.870+0000 7fb201edeb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5571fa44bed0 0x5571fa44f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.870+0000 7fb201edeb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571fa32ea60 con 0x5571fa44bed0 2023-07-21T18:56:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.870+0000 7fb201edeb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571fa32e760 con 0x5571fa44ba80 2023-07-21T18:56:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.870+0000 7fb201edeb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571fa2128c0 con 0x5571fa2996b0 2023-07-21T18:56:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.870+0000 7fb201edeb80 10 client.0 _ll_register_callbacks cb 0x5571fa32c700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:56:16.886 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.870+0000 7fb1ef7fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571fa2996b0 0x5571fa451df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:16.886 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.870+0000 7fb1eeffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571fa44ba80 0x5571fa44f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:16.886 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.870+0000 7fb1ef7fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571fa2996b0 0x5571fa451df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:51828/0 (socket says 172.21.15.39:51828) 2023-07-21T18:56:16.886 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.870+0000 7fb1ef7fe700 1 -- 172.21.15.39:0/1402390419 learned_addr learned my addr 172.21.15.39:0/1402390419 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:56:16.886 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.870+0000 7fb1effff700 1 --2- 172.21.15.39:0/1402390419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5571fa44bed0 0x5571fa44f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:16.886 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.870+0000 7fb1ef7fe700 1 -- 172.21.15.39:0/1402390419 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571fa44ba80 msgr2=0x5571fa44f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:56:16.886 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.870+0000 7fb1ef7fe700 1 --2- 172.21.15.39:0/1402390419 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571fa44ba80 0x5571fa44f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:16.887 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.870+0000 7fb1ef7fe700 1 -- 172.21.15.39:0/1402390419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5571fa44bed0 msgr2=0x5571fa44f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:56:16.887 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.870+0000 7fb1ef7fe700 1 --2- 172.21.15.39:0/1402390419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5571fa44bed0 0x5571fa44f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:16.887 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.870+0000 7fb1ef7fe700 1 -- 172.21.15.39:0/1402390419 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1f01a4cc0 con 0x5571fa2996b0 2023-07-21T18:56:16.887 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.870+0000 7fb1eeffd700 1 --2- 172.21.15.39:0/1402390419 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571fa44ba80 0x5571fa44f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:56:16.887 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.871+0000 7fb1ef7fe700 1 --2- 172.21.15.39:0/1402390419 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571fa2996b0 0x5571fa451df0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb1e00850b0 tx=0x7fb1e009a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1c96651f3964d42 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:56:16.887 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.871+0000 7fb1ceffd700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2023-07-21T18:56:16.887 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.871+0000 7fb1ceffd700 1 -- 172.21.15.39:0/1402390419 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e009e2b0 con 0x5571fa2996b0 2023-07-21T18:56:16.887 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.871+0000 7fb1ceffd700 1 -- 172.21.15.39:0/1402390419 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fb1e00a26d0 con 0x5571fa2996b0 2023-07-21T18:56:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.871+0000 7fb201edeb80 1 -- 172.21.15.39:0/1402390419 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5571fa2e8490 con 0x5571fa2996b0 2023-07-21T18:56:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.871+0000 7fb1ceffd700 1 -- 172.21.15.39:0/1402390419 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb1e009a280 con 0x5571fa2996b0 2023-07-21T18:56:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.871+0000 7fb201edeb80 10 client.26725 fetch_fsmap learned FSMap version 84 2023-07-21T18:56:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.871+0000 7fb201edeb80 1 -- 172.21.15.39:0/1402390419 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) v3 -- 0x5571fa1a5c00 con 0x5571fa2996b0 2023-07-21T18:56:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.871+0000 7fb1ceffd700 1 -- 172.21.15.39:0/1402390419 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb1e00c8020 con 0x5571fa2996b0 2023-07-21T18:56:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.872+0000 7fb1ceffd700 1 -- 172.21.15.39:0/1402390419 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 84) v1 ==== 7538+0+0 (secure 0 0 0) 0x7fb1e001ec90 con 0x5571fa2996b0 2023-07-21T18:56:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.872+0000 7fb201edeb80 10 client.26725 fetch_fsmap finished waiting for FSMap version 84 2023-07-21T18:56:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.872+0000 7fb201edeb80 10 client.26725 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:56:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.872+0000 7fb201edeb80 20 client.26725 populate_metadata read hostname 'smithi039' 2023-07-21T18:56:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.872+0000 7fb201edeb80 1 --2- 172.21.15.39:0/1402390419 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5571fa45ef20 0x5571fa461330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.872+0000 7fb201edeb80 4 client.26725 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:56:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.872+0000 7fb201edeb80 1 -- 172.21.15.39:0/1402390419 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5571fa387e50 con 0x5571fa45ef20 2023-07-21T18:56:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.872+0000 7fb1eeffd700 1 --2- 172.21.15.39:0/1402390419 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5571fa45ef20 0x5571fa461330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.872+0000 7fb1eeffd700 1 --2- 172.21.15.39:0/1402390419 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5571fa45ef20 0x5571fa461330 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c83cd9e8051c12f2 server_cookie=7f6caee872ed53a8 in_seq=0 out_seq=0 2023-07-21T18:56:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.872+0000 7fb1ceffd700 10 client.26725 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:56:16.890 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.873+0000 7fb1ceffd700 1 -- 172.21.15.39:0/1402390419 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5571fa387e50 con 0x5571fa45ef20 2023-07-21T18:56:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.873+0000 7fb1ceffd700 10 client.26725 handle_command_reply: tid=0 2023-07-21T18:56:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.873+0000 7fb201edeb80 1 client.26725 shutdown 2023-07-21T18:56:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.874+0000 7fb201edeb80 1 -- 172.21.15.39:0/1402390419 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571fa2996b0 msgr2=0x5571fa451df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:56:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.874+0000 7fb201edeb80 1 --2- 172.21.15.39:0/1402390419 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571fa2996b0 0x5571fa451df0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb1e00850b0 tx=0x7fb1e009a710 comp rx=0 tx=0).stop 2023-07-21T18:56:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.874+0000 7fb201edeb80 1 -- 172.21.15.39:0/1402390419 shutdown_connections 2023-07-21T18:56:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.874+0000 7fb201edeb80 1 --2- 172.21.15.39:0/1402390419 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5571fa45ef20 0x5571fa461330 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.874+0000 7fb201edeb80 1 --2- 172.21.15.39:0/1402390419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5571fa44bed0 0x5571fa44f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.874+0000 7fb201edeb80 1 --2- 172.21.15.39:0/1402390419 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571fa2996b0 0x5571fa451df0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.874+0000 7fb201edeb80 1 --2- 172.21.15.39:0/1402390419 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571fa44ba80 0x5571fa44f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.874+0000 7fb201edeb80 1 -- 172.21.15.39:0/1402390419 >> 172.21.15.39:0/1402390419 conn(0x5571fa432be0 msgr2=0x5571fa4374e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:56:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.874+0000 7fb1ceffd700 0 client.26725 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:56:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.874+0000 7fb201edeb80 1 -- 172.21.15.39:0/1402390419 shutdown_connections 2023-07-21T18:56:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.874+0000 7fb201edeb80 1 -- 172.21.15.39:0/1402390419 wait complete. 2023-07-21T18:56:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.874+0000 7fb201edeb80 20 client.26725 trim_cache size 0 max 16384 2023-07-21T18:56:16.913 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.902+0000 7fb201edeb80 1 Processor -- start 2023-07-21T18:56:16.913 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.902+0000 7fb201edeb80 1 -- start start 2023-07-21T18:56:16.914 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.902+0000 7fb201edeb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5571fa2996b0 0x5571fa451990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:16.914 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.902+0000 7fb201edeb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571fa44ba80 0x5571fa437c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:16.914 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.902+0000 7fb201edeb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571fa44bed0 0x5571fa44a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:16.914 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.902+0000 7fb201edeb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571fa434d00 con 0x5571fa2996b0 2023-07-21T18:56:16.915 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.902+0000 7fb201edeb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571fa44b190 con 0x5571fa44bed0 2023-07-21T18:56:16.915 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.902+0000 7fb201edeb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571fa4463d0 con 0x5571fa44ba80 2023-07-21T18:56:16.915 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.902+0000 7fb1ef7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5571fa2996b0 0x5571fa451990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:16.915 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.902+0000 7fb1ef7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5571fa2996b0 0x5571fa451990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33284/0 (socket says 172.21.15.39:33284) 2023-07-21T18:56:16.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.902+0000 7fb1ef7fe700 1 -- 172.21.15.39:0/1890885141 learned_addr learned my addr 172.21.15.39:0/1890885141 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:56:16.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.902+0000 7fb1eeffd700 1 --2- 172.21.15.39:0/1890885141 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571fa44ba80 0x5571fa437c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:16.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.903+0000 7fb1effff700 1 --2- 172.21.15.39:0/1890885141 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571fa44bed0 0x5571fa44a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:16.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.903+0000 7fb1ef7fe700 1 -- 172.21.15.39:0/1890885141 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571fa44ba80 msgr2=0x5571fa437c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:56:16.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.903+0000 7fb1ef7fe700 1 --2- 172.21.15.39:0/1890885141 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571fa44ba80 0x5571fa437c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:16.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.903+0000 7fb1ef7fe700 1 -- 172.21.15.39:0/1890885141 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571fa44bed0 msgr2=0x5571fa44a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:56:16.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.903+0000 7fb1ef7fe700 1 --2- 172.21.15.39:0/1890885141 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571fa44bed0 0x5571fa44a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:16.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.903+0000 7fb1ef7fe700 1 -- 172.21.15.39:0/1890885141 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5571fa1e31f0 con 0x5571fa2996b0 2023-07-21T18:56:16.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.903+0000 7fb1ef7fe700 1 --2- 172.21.15.39:0/1890885141 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5571fa2996b0 0x5571fa451990 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0099f80 tx=0x7fb1e0010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b6ec2e19bbf5fb7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:56:16.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.903+0000 7fb1d67fc700 1 -- 172.21.15.39:0/1890885141 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e009a780 con 0x5571fa2996b0 2023-07-21T18:56:16.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.903+0000 7fb1d67fc700 1 -- 172.21.15.39:0/1890885141 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb1e00c8030 con 0x5571fa2996b0 2023-07-21T18:56:16.924 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.903+0000 7fb1d67fc700 1 -- 172.21.15.39:0/1890885141 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e001edf0 con 0x5571fa2996b0 2023-07-21T18:56:16.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.904+0000 7fb201edeb80 1 -- 172.21.15.39:0/1890885141 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5571fa2996b0 msgr2=0x5571fa451990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:56:16.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.904+0000 7fb201edeb80 1 --2- 172.21.15.39:0/1890885141 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5571fa2996b0 0x5571fa451990 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0099f80 tx=0x7fb1e0010c80 comp rx=0 tx=0).stop 2023-07-21T18:56:16.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.904+0000 7fb201edeb80 1 -- 172.21.15.39:0/1890885141 shutdown_connections 2023-07-21T18:56:16.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.904+0000 7fb201edeb80 1 --2- 172.21.15.39:0/1890885141 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571fa44bed0 0x5571fa44a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:16.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.904+0000 7fb201edeb80 1 --2- 172.21.15.39:0/1890885141 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5571fa2996b0 0x5571fa451990 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:16.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.904+0000 7fb201edeb80 1 --2- 172.21.15.39:0/1890885141 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571fa44ba80 0x5571fa437c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:16.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.904+0000 7fb201edeb80 1 -- 172.21.15.39:0/1890885141 >> 172.21.15.39:0/1890885141 conn(0x5571fa432be0 msgr2=0x5571fa449240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:56:16.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.904+0000 7fb201edeb80 1 -- 172.21.15.39:0/1890885141 shutdown_connections 2023-07-21T18:56:16.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.904+0000 7fb201edeb80 1 -- 172.21.15.39:0/1890885141 wait complete. 2023-07-21T18:56:16.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.905+0000 7fb201edeb80 1 -- start start 2023-07-21T18:56:16.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.905+0000 7fb201edeb80 1 Processor -- start 2023-07-21T18:56:16.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.905+0000 7fb201edeb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571fa2996b0 0x5571fa4566a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:16.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.905+0000 7fb201edeb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571fa44ba80 0x5571fa454490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:16.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.905+0000 7fb201edeb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5571fa44bed0 0x5571fa4549f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:16.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.905+0000 7fb201edeb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571fa4550c0 con 0x5571fa44bed0 2023-07-21T18:56:16.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.905+0000 7fb201edeb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571fa455230 con 0x5571fa44ba80 2023-07-21T18:56:16.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.905+0000 7fb201edeb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571fa4b1de0 con 0x5571fa2996b0 2023-07-21T18:56:16.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.906+0000 7fb201edeb80 10 client.0 _ll_register_callbacks cb 0x7fb1e4009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:56:16.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.906+0000 7fb1ef7fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571fa2996b0 0x5571fa4566a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:16.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.906+0000 7fb1ef7fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571fa2996b0 0x5571fa4566a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:51844/0 (socket says 172.21.15.39:51844) 2023-07-21T18:56:16.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.906+0000 7fb1ef7fe700 1 -- 172.21.15.39:0/1332795618 learned_addr learned my addr 172.21.15.39:0/1332795618 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:56:16.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.907+0000 7fb1effff700 1 --2- 172.21.15.39:0/1332795618 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5571fa44bed0 0x5571fa4549f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:16.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.907+0000 7fb1eeffd700 1 --2- 172.21.15.39:0/1332795618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571fa44ba80 0x5571fa454490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:16.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.907+0000 7fb1eeffd700 1 -- 172.21.15.39:0/1332795618 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571fa2996b0 msgr2=0x5571fa4566a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:56:16.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.907+0000 7fb1eeffd700 1 --2- 172.21.15.39:0/1332795618 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571fa2996b0 0x5571fa4566a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:16.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.907+0000 7fb1eeffd700 1 -- 172.21.15.39:0/1332795618 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5571fa44bed0 msgr2=0x5571fa4549f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:56:16.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.907+0000 7fb1eeffd700 1 --2- 172.21.15.39:0/1332795618 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5571fa44bed0 0x5571fa4549f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:16.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.907+0000 7fb1eeffd700 1 -- 172.21.15.39:0/1332795618 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5571fa1a5c00 con 0x5571fa44ba80 2023-07-21T18:56:16.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.907+0000 7fb1effff700 1 --2- 172.21.15.39:0/1332795618 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5571fa44bed0 0x5571fa4549f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:56:16.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.908+0000 7fb1eeffd700 1 --2- 172.21.15.39:0/1332795618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571fa44ba80 0x5571fa454490 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x5571fa42ff50 tx=0x7fb1e4012320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5ebdbe187b55f9a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:56:16.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.908+0000 7fb1ceffd700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-21T18:56:16.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.908+0000 7fb1ceffd700 1 -- 172.21.15.39:0/1332795618 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e4017070 con 0x5571fa44ba80 2023-07-21T18:56:16.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.908+0000 7fb201edeb80 1 -- 172.21.15.39:0/1332795618 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb1e0094e90 con 0x5571fa44ba80 2023-07-21T18:56:16.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.908+0000 7fb1ceffd700 1 -- 172.21.15.39:0/1332795618 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fb1e4012b80 con 0x5571fa44ba80 2023-07-21T18:56:16.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.909+0000 7fb1ceffd700 1 -- 172.21.15.39:0/1332795618 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb1e4012d40 con 0x5571fa44ba80 2023-07-21T18:56:16.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.909+0000 7fb201edeb80 10 client.26717 fetch_fsmap learned FSMap version 84 2023-07-21T18:56:16.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.909+0000 7fb201edeb80 1 -- 172.21.15.39:0/1332795618 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) v3 -- 0x5571fa1bb250 con 0x5571fa44ba80 2023-07-21T18:56:16.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.909+0000 7fb1ceffd700 1 -- 172.21.15.39:0/1332795618 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb1e0094e90 con 0x5571fa44ba80 2023-07-21T18:56:16.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.909+0000 7fb1ceffd700 1 -- 172.21.15.39:0/1332795618 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 84) v1 ==== 7538+0+0 (secure 0 0 0) 0x7fb1e40054d0 con 0x5571fa44ba80 2023-07-21T18:56:16.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.909+0000 7fb201edeb80 10 client.26717 fetch_fsmap finished waiting for FSMap version 84 2023-07-21T18:56:16.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.909+0000 7fb201edeb80 10 client.26717 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:56:16.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.909+0000 7fb201edeb80 20 client.26717 populate_metadata read hostname 'smithi039' 2023-07-21T18:56:16.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.909+0000 7fb201edeb80 1 --2- 172.21.15.39:0/1332795618 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5571fa4b3560 0x5571fa4b5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:16.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.909+0000 7fb201edeb80 4 client.26717 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:56:16.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.909+0000 7fb201edeb80 1 -- 172.21.15.39:0/1332795618 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5571fa349ff0 con 0x5571fa4b3560 2023-07-21T18:56:16.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.909+0000 7fb1ef7fe700 1 --2- 172.21.15.39:0/1332795618 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5571fa4b3560 0x5571fa4b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:16.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.910+0000 7fb1ceffd700 10 client.26717 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:56:16.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:16.910+0000 7fb1ef7fe700 1 --2- 172.21.15.39:0/1332795618 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5571fa4b3560 0x5571fa4b5970 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5db8a33a931fae3 server_cookie=52eb85d57d924bde in_seq=0 out_seq=0 2023-07-21T18:56:17.134 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:17.124+0000 7fb1ceffd700 1 -- 172.21.15.39:0/1332795618 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5571fa349ff0 con 0x5571fa4b3560 2023-07-21T18:56:17.135 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:17.124+0000 7fb1ceffd700 10 client.26717 handle_command_reply: tid=0 2023-07-21T18:56:17.135 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:17.124+0000 7fb201edeb80 1 client.26717 shutdown 2023-07-21T18:56:17.136 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:17.125+0000 7fb201edeb80 1 -- 172.21.15.39:0/1332795618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571fa44ba80 msgr2=0x5571fa454490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:56:17.136 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:17.125+0000 7fb201edeb80 1 --2- 172.21.15.39:0/1332795618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571fa44ba80 0x5571fa454490 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x5571fa42ff50 tx=0x7fb1e4012320 comp rx=0 tx=0).stop 2023-07-21T18:56:17.137 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:17.125+0000 7fb201edeb80 1 -- 172.21.15.39:0/1332795618 shutdown_connections 2023-07-21T18:56:17.137 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:17.125+0000 7fb201edeb80 1 --2- 172.21.15.39:0/1332795618 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5571fa4b3560 0x5571fa4b5970 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:17.137 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:17.125+0000 7fb201edeb80 1 --2- 172.21.15.39:0/1332795618 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5571fa44bed0 0x5571fa4549f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:17.137 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:17.125+0000 7fb201edeb80 1 --2- 172.21.15.39:0/1332795618 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571fa2996b0 0x5571fa4566a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:17.137 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:17.125+0000 7fb201edeb80 1 --2- 172.21.15.39:0/1332795618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571fa44ba80 0x5571fa454490 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:17.137 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:17.125+0000 7fb201edeb80 1 -- 172.21.15.39:0/1332795618 >> 172.21.15.39:0/1332795618 conn(0x5571fa432be0 msgr2=0x5571fa43c030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:56:17.137 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:17.125+0000 7fb1ceffd700 0 client.26717 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:56:17.137 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:17.125+0000 7fb201edeb80 1 -- 172.21.15.39:0/1332795618 shutdown_connections 2023-07-21T18:56:17.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:17.125+0000 7fb201edeb80 1 -- 172.21.15.39:0/1332795618 wait complete. 2023-07-21T18:56:17.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:17.126+0000 7fb201edeb80 20 client.26717 trim_cache size 0 max 16384 2023-07-21T18:56:17.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:56:17.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:56:17.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:56:17.139 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:56:17.139 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:17.126+0000 7fb1f5d58700 1 -- 172.21.15.39:0/4017030894 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fb1d80a12b0 msgr2=0x7fb1d80a3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:56:17.139 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:17.126+0000 7fb1f5d58700 1 --2- 172.21.15.39:0/4017030894 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fb1d80a12b0 0x7fb1d80a3760 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb1e8009cd0 tx=0x7fb1e800bd00 comp rx=0 tx=0).stop 2023-07-21T18:56:17.140 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:17.126+0000 7fb1f5d58700 1 -- 172.21.15.39:0/4017030894 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb1f00fea30 msgr2=0x7fb1f0197e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:56:17.140 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:17.126+0000 7fb1f5d58700 1 --2- 172.21.15.39:0/4017030894 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb1f00fea30 0x7fb1f0197e60 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb1e000ea50 tx=0x7fb1e000ed60 comp rx=0 tx=0).stop 2023-07-21T18:56:17.140 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:17.127+0000 7fb1f5d58700 1 -- 172.21.15.39:0/4017030894 shutdown_connections 2023-07-21T18:56:17.140 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:17.127+0000 7fb1f5d58700 1 --2- 172.21.15.39:0/4017030894 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fb1d80a12b0 0x7fb1d80a3760 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:17.140 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:17.127+0000 7fb1f5d58700 1 --2- 172.21.15.39:0/4017030894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1f0104a40 0x7fb1f019ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:17.140 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:17.127+0000 7fb1f5d58700 1 --2- 172.21.15.39:0/4017030894 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb1f00fea30 0x7fb1f0197e60 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:17.140 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:17.127+0000 7fb1f5d58700 1 --2- 172.21.15.39:0/4017030894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1f00ff3e0 0x7fb1f01983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:17.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:17.127+0000 7fb1f5d58700 1 -- 172.21.15.39:0/4017030894 >> 172.21.15.39:0/4017030894 conn(0x7fb1f00fa550 msgr2=0x7fb1f0105840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:56:17.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:17.128+0000 7fb1f5d58700 1 -- 172.21.15.39:0/4017030894 shutdown_connections 2023-07-21T18:56:17.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:17.128+0000 7fb1f5d58700 1 -- 172.21.15.39:0/4017030894 wait complete. 2023-07-21T18:56:17.608 INFO:journalctl@ceph.mds.c.smithi193.stdout:Jul 21 18:56:17 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-c[124252]: ignoring --setuser ceph since I am not root 2023-07-21T18:56:17.609 INFO:journalctl@ceph.mds.c.smithi193.stdout:Jul 21 18:56:17 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-c[124252]: ignoring --setgroup ceph since I am not root 2023-07-21T18:56:17.609 INFO:journalctl@ceph.mds.c.smithi193.stdout:Jul 21 18:56:17 smithi193 ceph-mds[124276]: -- 172.21.15.193:0/3160224398 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ea96a271e0 con 0x55ea969fd000 2023-07-21T18:56:17.609 INFO:journalctl@ceph.mds.c.smithi193.stdout:Jul 21 18:56:17 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-c[124252]: starting mds.c at 2023-07-21T18:56:17.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:56:17 smithi193 ceph-mon[98204]: daemon mds.c restarted 2023-07-21T18:56:17.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:56:17 smithi155 ceph-mon[97934]: daemon mds.c restarted 2023-07-21T18:56:17.662 INFO:tasks.workunit.client.0.smithi039.stdout: 10 45792 29.34 MB/sec warmup 90 sec latency 190.924 ms 2023-07-21T18:56:17.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:56:17 smithi039 ceph-mon[96489]: daemon mds.c restarted 2023-07-21T18:56:18.662 INFO:tasks.workunit.client.0.smithi039.stdout: 10 46599 29.51 MB/sec warmup 91 sec latency 66.258 ms 2023-07-21T18:56:18.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:56:18 smithi039 ceph-mon[96489]: pgmap v458: 97 pgs: 97 active+clean; 718 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 32 MiB/s wr, 1.45k op/s 2023-07-21T18:56:18.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:56:18 smithi039 ceph-mon[96489]: osdmap e102: 12 total, 12 up, 12 in 2023-07-21T18:56:18.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:56:18 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-21T18:56:18.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:56:18 smithi039 ceph-mon[96489]: mds.? [v2:172.21.15.193:6836/4135544527,v1:172.21.15.193:6837/4135544527] up:boot 2023-07-21T18:56:18.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:56:18 smithi039 ceph-mon[96489]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2023-07-21T18:56:18.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:56:18 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:56:18.859 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:56:18 smithi193 ceph-mon[98204]: pgmap v458: 97 pgs: 97 active+clean; 718 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 32 MiB/s wr, 1.45k op/s 2023-07-21T18:56:18.859 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:56:18 smithi193 ceph-mon[98204]: osdmap e102: 12 total, 12 up, 12 in 2023-07-21T18:56:18.859 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:56:18 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-21T18:56:18.859 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:56:18 smithi193 ceph-mon[98204]: mds.? [v2:172.21.15.193:6836/4135544527,v1:172.21.15.193:6837/4135544527] up:boot 2023-07-21T18:56:18.859 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:56:18 smithi193 ceph-mon[98204]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2023-07-21T18:56:18.859 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:56:18 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:56:18.878 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:56:18 smithi155 ceph-mon[97934]: pgmap v458: 97 pgs: 97 active+clean; 718 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 32 MiB/s wr, 1.45k op/s 2023-07-21T18:56:18.878 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:56:18 smithi155 ceph-mon[97934]: osdmap e102: 12 total, 12 up, 12 in 2023-07-21T18:56:18.878 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:56:18 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-21T18:56:18.879 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:56:18 smithi155 ceph-mon[97934]: mds.? [v2:172.21.15.193:6836/4135544527,v1:172.21.15.193:6837/4135544527] up:boot 2023-07-21T18:56:18.879 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:56:18 smithi155 ceph-mon[97934]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2023-07-21T18:56:18.879 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:56:18 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:56:19.664 INFO:tasks.workunit.client.0.smithi039.stdout: 10 47502 29.61 MB/sec warmup 92 sec latency 62.759 ms 2023-07-21T18:56:20.664 INFO:tasks.workunit.client.0.smithi039.stdout: 10 48405 30.01 MB/sec warmup 93 sec latency 49.375 ms 2023-07-21T18:56:20.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:56:20 smithi039 ceph-mon[96489]: pgmap v460: 97 pgs: 97 active+clean; 718 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.1 MiB/s wr, 412 op/s 2023-07-21T18:56:20.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:56:20 smithi193 ceph-mon[98204]: pgmap v460: 97 pgs: 97 active+clean; 718 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.1 MiB/s wr, 412 op/s 2023-07-21T18:56:20.878 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:56:20 smithi155 ceph-mon[97934]: pgmap v460: 97 pgs: 97 active+clean; 718 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.1 MiB/s wr, 412 op/s 2023-07-21T18:56:21.664 INFO:tasks.workunit.client.0.smithi039.stdout: 10 49057 29.97 MB/sec warmup 94 sec latency 105.521 ms 2023-07-21T18:56:22.664 INFO:tasks.workunit.client.0.smithi039.stdout: 10 49711 30.00 MB/sec warmup 95 sec latency 75.222 ms 2023-07-21T18:56:22.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:56:22 smithi039 ceph-mon[96489]: pgmap v461: 97 pgs: 97 active+clean; 731 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 9.4 MiB/s wr, 931 op/s 2023-07-21T18:56:22.859 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:56:22 smithi193 ceph-mon[98204]: pgmap v461: 97 pgs: 97 active+clean; 731 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 9.4 MiB/s wr, 931 op/s 2023-07-21T18:56:22.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:56:22 smithi155 ceph-mon[97934]: pgmap v461: 97 pgs: 97 active+clean; 731 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 9.4 MiB/s wr, 931 op/s 2023-07-21T18:56:23.664 INFO:tasks.workunit.client.0.smithi039.stdout: 10 50667 30.21 MB/sec warmup 96 sec latency 52.533 ms 2023-07-21T18:56:24.664 INFO:tasks.workunit.client.0.smithi039.stdout: 10 51417 30.28 MB/sec warmup 97 sec latency 136.232 ms 2023-07-21T18:56:24.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:56:24 smithi039 ceph-mon[96489]: pgmap v462: 97 pgs: 97 active+clean; 731 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 9.4 MiB/s wr, 931 op/s 2023-07-21T18:56:24.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:56:24 smithi193 ceph-mon[98204]: pgmap v462: 97 pgs: 97 active+clean; 731 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 9.4 MiB/s wr, 931 op/s 2023-07-21T18:56:24.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:56:24 smithi155 ceph-mon[97934]: pgmap v462: 97 pgs: 97 active+clean; 731 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 9.4 MiB/s wr, 931 op/s 2023-07-21T18:56:25.664 INFO:tasks.workunit.client.0.smithi039.stdout: 10 52324 30.55 MB/sec warmup 98 sec latency 54.731 ms 2023-07-21T18:56:26.664 INFO:tasks.workunit.client.0.smithi039.stdout: 10 53010 30.66 MB/sec warmup 99 sec latency 49.980 ms 2023-07-21T18:56:26.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:56:26 smithi039 ceph-mon[96489]: pgmap v463: 97 pgs: 97 active+clean; 731 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.1 MiB/s wr, 501 op/s 2023-07-21T18:56:26.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:56:26 smithi193 ceph-mon[98204]: pgmap v463: 97 pgs: 97 active+clean; 731 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.1 MiB/s wr, 501 op/s 2023-07-21T18:56:26.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:56:26 smithi155 ceph-mon[97934]: pgmap v463: 97 pgs: 97 active+clean; 731 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.1 MiB/s wr, 501 op/s 2023-07-21T18:56:27.664 INFO:tasks.workunit.client.0.smithi039.stdout: 10 53716 30.67 MB/sec warmup 100 sec latency 93.038 ms 2023-07-21T18:56:28.664 INFO:tasks.workunit.client.0.smithi039.stdout: 10 54465 30.72 MB/sec warmup 101 sec latency 78.508 ms 2023-07-21T18:56:28.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:56:28 smithi039 ceph-mon[96489]: pgmap v464: 97 pgs: 97 active+clean; 761 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 7.9 MiB/s wr, 745 op/s 2023-07-21T18:56:28.859 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:56:28 smithi193 ceph-mon[98204]: pgmap v464: 97 pgs: 97 active+clean; 761 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 7.9 MiB/s wr, 745 op/s 2023-07-21T18:56:28.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:56:28 smithi155 ceph-mon[97934]: pgmap v464: 97 pgs: 97 active+clean; 761 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 7.9 MiB/s wr, 745 op/s 2023-07-21T18:56:29.664 INFO:tasks.workunit.client.0.smithi039.stdout: 10 55463 31.03 MB/sec warmup 102 sec latency 52.915 ms 2023-07-21T18:56:30.288 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.277+0000 7f9401e4f700 1 -- 172.21.15.39:0/2302957831 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f93fc0feac0 msgr2=0x7f93fc0fef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:56:30.288 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.277+0000 7f9401e4f700 1 --2- 172.21.15.39:0/2302957831 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f93fc0feac0 0x7f93fc0fef30 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f93f00098f0 tx=0x7f93f0009c00 comp rx=0 tx=0).stop 2023-07-21T18:56:30.288 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.277+0000 7f9401e4f700 1 -- 172.21.15.39:0/2302957831 shutdown_connections 2023-07-21T18:56:30.288 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.277+0000 7f9401e4f700 1 --2- 172.21.15.39:0/2302957831 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f93fc0ff470 0x7f93fc10b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:30.288 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.277+0000 7f9401e4f700 1 --2- 172.21.15.39:0/2302957831 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93fc104ad0 0x7f93fc104ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:30.289 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.277+0000 7f9401e4f700 1 --2- 172.21.15.39:0/2302957831 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f93fc0feac0 0x7f93fc0fef30 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:30.289 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.277+0000 7f9401e4f700 1 -- 172.21.15.39:0/2302957831 >> 172.21.15.39:0/2302957831 conn(0x7f93fc075bd0 msgr2=0x7f93fc075fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:56:30.289 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.277+0000 7f9401e4f700 1 -- 172.21.15.39:0/2302957831 shutdown_connections 2023-07-21T18:56:30.289 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.277+0000 7f9401e4f700 1 -- 172.21.15.39:0/2302957831 wait complete. 2023-07-21T18:56:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.278+0000 7f9401e4f700 1 Processor -- start 2023-07-21T18:56:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.278+0000 7f9401e4f700 1 -- start start 2023-07-21T18:56:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.278+0000 7f9401e4f700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f93fc0feac0 0x7f93fc196c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.279+0000 7f9401e4f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f93fc0ff470 0x7f93fc197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.279+0000 7f9401e4f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93fc104ad0 0x7f93fc1a1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.279+0000 7f9401e4f700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93fc1a20a0 con 0x7f93fc104ad0 2023-07-21T18:56:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.279+0000 7f9401e4f700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93fc1a2210 con 0x7f93fc0ff470 2023-07-21T18:56:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.279+0000 7f9401e4f700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93fc194840 con 0x7f93fc0feac0 2023-07-21T18:56:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.279+0000 7f93fbfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93fc104ad0 0x7f93fc1a1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.279+0000 7f93fb7fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f93fc0feac0 0x7f93fc196c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.279+0000 7f93faffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f93fc0ff470 0x7f93fc197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.279+0000 7f93fbfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93fc104ad0 0x7f93fc1a1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33306/0 (socket says 172.21.15.39:33306) 2023-07-21T18:56:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.279+0000 7f93fbfff700 1 -- 172.21.15.39:0/3398760814 learned_addr learned my addr 172.21.15.39:0/3398760814 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:56:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.279+0000 7f93fbfff700 1 -- 172.21.15.39:0/3398760814 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f93fc0feac0 msgr2=0x7f93fc196c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:56:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.279+0000 7f93fbfff700 1 --2- 172.21.15.39:0/3398760814 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f93fc0feac0 0x7f93fc196c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.279+0000 7f93fbfff700 1 -- 172.21.15.39:0/3398760814 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f93fc0ff470 msgr2=0x7f93fc197170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:56:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.279+0000 7f93fbfff700 1 --2- 172.21.15.39:0/3398760814 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f93fc0ff470 0x7f93fc197170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.279+0000 7f93fbfff700 1 -- 172.21.15.39:0/3398760814 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93ec0095f0 con 0x7f93fc104ad0 2023-07-21T18:56:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.279+0000 7f93faffd700 1 --2- 172.21.15.39:0/3398760814 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f93fc0ff470 0x7f93fc197170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:56:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.279+0000 7f93fb7fe700 1 --2- 172.21.15.39:0/3398760814 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f93fc0feac0 0x7f93fc196c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:56:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.279+0000 7f93fbfff700 1 --2- 172.21.15.39:0/3398760814 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93fc104ad0 0x7f93fc1a1ad0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f93f400ef30 tx=0x7f93f400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:56:30.294 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.280+0000 7f93f8ff9700 1 -- 172.21.15.39:0/3398760814 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f4018070 con 0x7f93fc104ad0 2023-07-21T18:56:30.294 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.280+0000 7f93f8ff9700 1 -- 172.21.15.39:0/3398760814 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93f400f040 con 0x7f93fc104ad0 2023-07-21T18:56:30.294 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.280+0000 7f9401e4f700 1 -- 172.21.15.39:0/3398760814 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93f0009520 con 0x7f93fc104ad0 2023-07-21T18:56:30.294 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.280+0000 7f93f8ff9700 1 -- 172.21.15.39:0/3398760814 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f400cd00 con 0x7f93fc104ad0 2023-07-21T18:56:30.294 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.280+0000 7f9401e4f700 1 -- 172.21.15.39:0/3398760814 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93fc194dc0 con 0x7f93fc104ad0 2023-07-21T18:56:30.295 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.283+0000 7f93f8ff9700 1 -- 172.21.15.39:0/3398760814 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f93f4003a00 con 0x7f93fc104ad0 2023-07-21T18:56:30.295 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.284+0000 7f93f8ff9700 1 --2- 172.21.15.39:0/3398760814 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f93e40a12c0 0x7f93e40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:30.295 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.284+0000 7f940dfd5b80 1 Processor -- start 2023-07-21T18:56:30.295 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.284+0000 7f93f8ff9700 1 -- 172.21.15.39:0/3398760814 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f93f40cc880 con 0x7f93fc104ad0 2023-07-21T18:56:30.295 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.284+0000 7f93fb7fe700 1 --2- 172.21.15.39:0/3398760814 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f93e40a12c0 0x7f93e40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.284+0000 7f940dfd5b80 1 -- start start 2023-07-21T18:56:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.285+0000 7f940dfd5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556abe0b96b0 0x556abe262d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.285+0000 7f940dfd5b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556abe2632d0 0x556abe25d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.285+0000 7f940dfd5b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556abe25d580 0x556abe25d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.285+0000 7f940dfd5b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x556abe162c50 con 0x556abe2632d0 2023-07-21T18:56:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.285+0000 7f940dfd5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556abe1b59e0 con 0x556abe0b96b0 2023-07-21T18:56:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.285+0000 7f940dfd5b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x556abe1aee70 con 0x556abe25d580 2023-07-21T18:56:30.298 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.285+0000 7f93faffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556abe2632d0 0x556abe25d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:30.298 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.285+0000 7f93fb7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556abe0b96b0 0x556abe262d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:30.298 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.285+0000 7f93faffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556abe2632d0 0x556abe25d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33314/0 (socket says 172.21.15.39:33314) 2023-07-21T18:56:30.298 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.285+0000 7f93fb7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556abe0b96b0 0x556abe262d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:45908/0 (socket says 172.21.15.39:45908) 2023-07-21T18:56:30.299 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.285+0000 7f93faffd700 1 -- 172.21.15.39:0/1888601797 learned_addr learned my addr 172.21.15.39:0/1888601797 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:56:30.299 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.285+0000 7f93fbfff700 1 --2- 172.21.15.39:0/1888601797 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556abe25d580 0x556abe25d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:30.299 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.285+0000 7f93fb7fe700 1 --2- 172.21.15.39:0/3398760814 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f93e40a12c0 0x7f93e40a3770 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f93fc0ffc00 tx=0x7f93ec009310 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:56:30.299 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.285+0000 7f93faffd700 1 -- 172.21.15.39:0/1888601797 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556abe25d580 msgr2=0x556abe25d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:56:30.299 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.285+0000 7f93faffd700 1 --2- 172.21.15.39:0/1888601797 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556abe25d580 0x556abe25d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:30.299 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.285+0000 7f93faffd700 1 -- 172.21.15.39:0/1888601797 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556abe0b96b0 msgr2=0x556abe262d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:56:30.299 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.285+0000 7f93faffd700 1 --2- 172.21.15.39:0/1888601797 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556abe0b96b0 0x556abe262d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:30.299 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.285+0000 7f93faffd700 1 -- 172.21.15.39:0/1888601797 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556abdfdcd80 con 0x556abe2632d0 2023-07-21T18:56:30.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.285+0000 7f93fbfff700 1 --2- 172.21.15.39:0/1888601797 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556abe25d580 0x556abe25d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:56:30.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.285+0000 7f93fb7fe700 1 --2- 172.21.15.39:0/1888601797 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556abe0b96b0 0x556abe262d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:56:30.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.285+0000 7f93faffd700 1 --2- 172.21.15.39:0/1888601797 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556abe2632d0 0x556abe25d040 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f93f0000c00 tx=0x7f93f0005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=528262be12392027 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:56:30.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.286+0000 7f93e1ffb700 1 -- 172.21.15.39:0/1888601797 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f001c050 con 0x556abe2632d0 2023-07-21T18:56:30.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.286+0000 7f93e1ffb700 1 -- 172.21.15.39:0/1888601797 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93f001d040 con 0x556abe2632d0 2023-07-21T18:56:30.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.286+0000 7f93e1ffb700 1 -- 172.21.15.39:0/1888601797 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f00052a0 con 0x556abe2632d0 2023-07-21T18:56:30.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.287+0000 7f940dfd5b80 1 -- 172.21.15.39:0/1888601797 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556abe2632d0 msgr2=0x556abe25d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:56:30.301 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.287+0000 7f940dfd5b80 1 --2- 172.21.15.39:0/1888601797 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556abe2632d0 0x556abe25d040 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f93f0000c00 tx=0x7f93f0005dc0 comp rx=0 tx=0).stop 2023-07-21T18:56:30.301 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.287+0000 7f940dfd5b80 1 -- 172.21.15.39:0/1888601797 shutdown_connections 2023-07-21T18:56:30.301 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.287+0000 7f940dfd5b80 1 --2- 172.21.15.39:0/1888601797 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556abe0b96b0 0x556abe262d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:30.301 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.287+0000 7f940dfd5b80 1 --2- 172.21.15.39:0/1888601797 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556abe25d580 0x556abe25d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:30.301 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.287+0000 7f940dfd5b80 1 --2- 172.21.15.39:0/1888601797 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556abe2632d0 0x556abe25d040 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:30.301 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.287+0000 7f940dfd5b80 1 -- 172.21.15.39:0/1888601797 >> 172.21.15.39:0/1888601797 conn(0x556abe252be0 msgr2=0x556abe257f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:56:30.301 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.287+0000 7f940dfd5b80 1 -- 172.21.15.39:0/1888601797 shutdown_connections 2023-07-21T18:56:30.301 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.287+0000 7f940dfd5b80 1 -- 172.21.15.39:0/1888601797 wait complete. 2023-07-21T18:56:30.302 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.287+0000 7f940dfd5b80 1 -- start start 2023-07-21T18:56:30.302 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.288+0000 7f940dfd5b80 1 Processor -- start 2023-07-21T18:56:30.304 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.288+0000 7f940dfd5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556abe0b96b0 0x556abe2695d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:30.304 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.288+0000 7f940dfd5b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556abe25d580 0x556abe266880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:30.304 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.288+0000 7f940dfd5b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556abe2632d0 0x556abe266eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:30.305 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.288+0000 7f940dfd5b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x556abe14ea60 con 0x556abe2632d0 2023-07-21T18:56:30.305 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.288+0000 7f940dfd5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556abe14e760 con 0x556abe0b96b0 2023-07-21T18:56:30.305 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.288+0000 7f940dfd5b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x556abe0328c0 con 0x556abe25d580 2023-07-21T18:56:30.305 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.288+0000 7f940dfd5b80 10 client.0 _ll_register_callbacks cb 0x556abe14c700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:56:30.305 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.288+0000 7f93faffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556abe25d580 0x556abe266880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:30.305 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.288+0000 7f93faffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556abe25d580 0x556abe266880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:51872/0 (socket says 172.21.15.39:51872) 2023-07-21T18:56:30.305 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.288+0000 7f93faffd700 1 -- 172.21.15.39:0/1593066135 learned_addr learned my addr 172.21.15.39:0/1593066135 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:56:30.305 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.288+0000 7f93fb7fe700 1 --2- 172.21.15.39:0/1593066135 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556abe0b96b0 0x556abe2695d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:30.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.288+0000 7f93fbfff700 1 --2- 172.21.15.39:0/1593066135 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556abe2632d0 0x556abe266eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:30.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.289+0000 7f93fb7fe700 1 -- 172.21.15.39:0/1593066135 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556abe25d580 msgr2=0x556abe266880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:56:30.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.289+0000 7f93fb7fe700 1 --2- 172.21.15.39:0/1593066135 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556abe25d580 0x556abe266880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:30.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.289+0000 7f93fb7fe700 1 -- 172.21.15.39:0/1593066135 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556abe2632d0 msgr2=0x556abe266eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:56:30.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.289+0000 7f93fb7fe700 1 --2- 172.21.15.39:0/1593066135 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556abe2632d0 0x556abe266eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:30.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.289+0000 7f93fb7fe700 1 -- 172.21.15.39:0/1593066135 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93ec017050 con 0x556abe0b96b0 2023-07-21T18:56:30.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.289+0000 7f93fb7fe700 1 --2- 172.21.15.39:0/1593066135 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556abe0b96b0 0x556abe2695d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f93ec005fd0 tx=0x7f93ec0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5051a7b90072100 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:56:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.289+0000 7f93daffd700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-21T18:56:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.289+0000 7f93daffd700 1 -- 172.21.15.39:0/1593066135 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ec021070 con 0x556abe0b96b0 2023-07-21T18:56:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.289+0000 7f93daffd700 1 -- 172.21.15.39:0/1593066135 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f93ec0116b0 con 0x556abe0b96b0 2023-07-21T18:56:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.289+0000 7f940dfd5b80 1 -- 172.21.15.39:0/1593066135 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556abe108490 con 0x556abe0b96b0 2023-07-21T18:56:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.289+0000 7f93daffd700 1 -- 172.21.15.39:0/1593066135 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93ec0118b0 con 0x556abe0b96b0 2023-07-21T18:56:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.289+0000 7f940dfd5b80 10 client.26732 fetch_fsmap learned FSMap version 86 2023-07-21T18:56:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.289+0000 7f940dfd5b80 1 -- 172.21.15.39:0/1593066135 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) v3 -- 0x7f93fc194dc0 con 0x556abe0b96b0 2023-07-21T18:56:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.290+0000 7f93daffd700 1 -- 172.21.15.39:0/1593066135 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556abe108490 con 0x556abe0b96b0 2023-07-21T18:56:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.290+0000 7f93daffd700 1 -- 172.21.15.39:0/1593066135 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 86) v1 ==== 7538+0+0 (secure 0 0 0) 0x7f93ec005020 con 0x556abe0b96b0 2023-07-21T18:56:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.290+0000 7f940dfd5b80 10 client.26732 fetch_fsmap finished waiting for FSMap version 86 2023-07-21T18:56:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.290+0000 7f940dfd5b80 10 client.26732 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:56:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.290+0000 7f940dfd5b80 20 client.26732 populate_metadata read hostname 'smithi039' 2023-07-21T18:56:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.290+0000 7f940dfd5b80 1 --2- 172.21.15.39:0/1593066135 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x556abe276ca0 0x556abe2790b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.290+0000 7f940dfd5b80 4 client.26732 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:56:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.290+0000 7f940dfd5b80 1 -- 172.21.15.39:0/1593066135 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556abe1a7e50 con 0x556abe276ca0 2023-07-21T18:56:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.290+0000 7f93faffd700 1 --2- 172.21.15.39:0/1593066135 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x556abe276ca0 0x556abe2790b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:30.309 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.291+0000 7f93daffd700 10 client.26732 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:56:30.309 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.291+0000 7f93faffd700 1 --2- 172.21.15.39:0/1593066135 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x556abe276ca0 0x556abe2790b0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e08596b6c2c0acb1 server_cookie=43455ebdf189a6e0 in_seq=0 out_seq=0 2023-07-21T18:56:30.309 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.291+0000 7f93daffd700 1 -- 172.21.15.39:0/1593066135 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556abe1a7e50 con 0x556abe276ca0 2023-07-21T18:56:30.309 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.291+0000 7f93daffd700 10 client.26732 handle_command_reply: tid=0 2023-07-21T18:56:30.309 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.291+0000 7f940dfd5b80 1 client.26732 shutdown 2023-07-21T18:56:30.309 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.292+0000 7f940dfd5b80 1 -- 172.21.15.39:0/1593066135 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556abe0b96b0 msgr2=0x556abe2695d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:56:30.309 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.292+0000 7f940dfd5b80 1 --2- 172.21.15.39:0/1593066135 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556abe0b96b0 0x556abe2695d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f93ec005fd0 tx=0x7f93ec0047c0 comp rx=0 tx=0).stop 2023-07-21T18:56:30.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.292+0000 7f940dfd5b80 1 -- 172.21.15.39:0/1593066135 shutdown_connections 2023-07-21T18:56:30.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.292+0000 7f940dfd5b80 1 --2- 172.21.15.39:0/1593066135 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x556abe276ca0 0x556abe2790b0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:30.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.292+0000 7f940dfd5b80 1 --2- 172.21.15.39:0/1593066135 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556abe0b96b0 0x556abe2695d0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:30.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.292+0000 7f940dfd5b80 1 --2- 172.21.15.39:0/1593066135 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556abe2632d0 0x556abe266eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:30.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.292+0000 7f940dfd5b80 1 --2- 172.21.15.39:0/1593066135 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556abe25d580 0x556abe266880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:30.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.292+0000 7f93daffd700 0 client.26732 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:56:30.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.292+0000 7f940dfd5b80 1 -- 172.21.15.39:0/1593066135 >> 172.21.15.39:0/1593066135 conn(0x556abe252be0 msgr2=0x556abe256c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:56:30.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.292+0000 7f940dfd5b80 1 -- 172.21.15.39:0/1593066135 shutdown_connections 2023-07-21T18:56:30.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.292+0000 7f940dfd5b80 1 -- 172.21.15.39:0/1593066135 wait complete. 2023-07-21T18:56:30.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.292+0000 7f940dfd5b80 20 client.26732 trim_cache size 0 max 16384 2023-07-21T18:56:30.331 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.320+0000 7f940dfd5b80 1 Processor -- start 2023-07-21T18:56:30.331 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.320+0000 7f940dfd5b80 1 -- start start 2023-07-21T18:56:30.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.320+0000 7f940dfd5b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556abe0b96b0 0x556abe25d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:30.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.320+0000 7f940dfd5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556abe25d580 0x556abe2622f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:30.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.321+0000 7f940dfd5b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556abe2632d0 0x556abe26dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:30.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.321+0000 7f940dfd5b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x556abe262a40 con 0x556abe0b96b0 2023-07-21T18:56:30.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.321+0000 7f940dfd5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556abe2674e0 con 0x556abe25d580 2023-07-21T18:56:30.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.321+0000 7f940dfd5b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x556abe258a30 con 0x556abe2632d0 2023-07-21T18:56:30.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.321+0000 7f93fb7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556abe0b96b0 0x556abe25d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:30.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.321+0000 7f93faffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556abe25d580 0x556abe2622f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:30.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.321+0000 7f93fb7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556abe0b96b0 0x556abe25d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33328/0 (socket says 172.21.15.39:33328) 2023-07-21T18:56:30.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.321+0000 7f93fb7fe700 1 -- 172.21.15.39:0/3658153914 learned_addr learned my addr 172.21.15.39:0/3658153914 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:56:30.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.321+0000 7f93fbfff700 1 --2- 172.21.15.39:0/3658153914 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556abe2632d0 0x556abe26dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:30.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.321+0000 7f93faffd700 1 -- 172.21.15.39:0/3658153914 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556abe2632d0 msgr2=0x556abe26dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:56:30.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.321+0000 7f93faffd700 1 --2- 172.21.15.39:0/3658153914 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556abe2632d0 0x556abe26dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:30.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.321+0000 7f93faffd700 1 -- 172.21.15.39:0/3658153914 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556abe0b96b0 msgr2=0x556abe25d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:56:30.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.321+0000 7f93faffd700 1 --2- 172.21.15.39:0/3658153914 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556abe0b96b0 0x556abe25d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:30.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.321+0000 7f93faffd700 1 -- 172.21.15.39:0/3658153914 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93fc10c200 con 0x556abe25d580 2023-07-21T18:56:30.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.321+0000 7f93fbfff700 1 --2- 172.21.15.39:0/3658153914 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556abe2632d0 0x556abe26dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:56:30.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.321+0000 7f93fb7fe700 1 --2- 172.21.15.39:0/3658153914 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556abe0b96b0 0x556abe25d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:56:30.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.321+0000 7f93faffd700 1 --2- 172.21.15.39:0/3658153914 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556abe25d580 0x556abe2622f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x556abe279980 tx=0x7f93f0009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=344b19dd8bf28b77 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:56:30.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.322+0000 7f93e27fc700 1 -- 172.21.15.39:0/3658153914 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f001c040 con 0x556abe25d580 2023-07-21T18:56:30.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.322+0000 7f93e27fc700 1 -- 172.21.15.39:0/3658153914 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93f0005570 con 0x556abe25d580 2023-07-21T18:56:30.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.322+0000 7f93e27fc700 1 -- 172.21.15.39:0/3658153914 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f0003680 con 0x556abe25d580 2023-07-21T18:56:30.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.323+0000 7f940dfd5b80 1 -- 172.21.15.39:0/3658153914 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556abe25d580 msgr2=0x556abe2622f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:56:30.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.323+0000 7f940dfd5b80 1 --2- 172.21.15.39:0/3658153914 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556abe25d580 0x556abe2622f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x556abe279980 tx=0x7f93f0009c00 comp rx=0 tx=0).stop 2023-07-21T18:56:30.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.323+0000 7f940dfd5b80 1 -- 172.21.15.39:0/3658153914 shutdown_connections 2023-07-21T18:56:30.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.323+0000 7f940dfd5b80 1 --2- 172.21.15.39:0/3658153914 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556abe2632d0 0x556abe26dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:30.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.323+0000 7f940dfd5b80 1 --2- 172.21.15.39:0/3658153914 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556abe0b96b0 0x556abe25d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:30.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.323+0000 7f940dfd5b80 1 --2- 172.21.15.39:0/3658153914 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556abe25d580 0x556abe2622f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:30.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.323+0000 7f940dfd5b80 1 -- 172.21.15.39:0/3658153914 >> 172.21.15.39:0/3658153914 conn(0x556abe252be0 msgr2=0x556abe263ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:56:30.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.324+0000 7f940dfd5b80 1 -- 172.21.15.39:0/3658153914 shutdown_connections 2023-07-21T18:56:30.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.324+0000 7f940dfd5b80 1 -- 172.21.15.39:0/3658153914 wait complete. 2023-07-21T18:56:30.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.325+0000 7f940dfd5b80 1 -- start start 2023-07-21T18:56:30.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.325+0000 7f940dfd5b80 1 Processor -- start 2023-07-21T18:56:30.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.325+0000 7f940dfd5b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556abe0b96b0 0x556abe26bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:30.338 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.325+0000 7f940dfd5b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556abe25d580 0x556abe26c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:30.338 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.325+0000 7f940dfd5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556abe2632d0 0x556abe26c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:30.338 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.325+0000 7f940dfd5b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x556abe2c9bc0 con 0x556abe25d580 2023-07-21T18:56:30.338 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.325+0000 7f940dfd5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556abe2c9d00 con 0x556abe2632d0 2023-07-21T18:56:30.338 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.325+0000 7f940dfd5b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x556abe2c9ec0 con 0x556abe0b96b0 2023-07-21T18:56:30.339 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.325+0000 7f940dfd5b80 10 client.0 _ll_register_callbacks cb 0x7f93ec0125d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:56:30.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.326+0000 7f93faffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556abe25d580 0x556abe26c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:30.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.326+0000 7f93faffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556abe25d580 0x556abe26c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33334/0 (socket says 172.21.15.39:33334) 2023-07-21T18:56:30.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.326+0000 7f93faffd700 1 -- 172.21.15.39:0/2681170626 learned_addr learned my addr 172.21.15.39:0/2681170626 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:56:30.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.326+0000 7f93fb7fe700 1 --2- 172.21.15.39:0/2681170626 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556abe0b96b0 0x556abe26bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:30.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.326+0000 7f93faffd700 1 -- 172.21.15.39:0/2681170626 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556abe0b96b0 msgr2=0x556abe26bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:56:30.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.326+0000 7f93faffd700 1 --2- 172.21.15.39:0/2681170626 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556abe0b96b0 0x556abe26bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:30.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.326+0000 7f93faffd700 1 -- 172.21.15.39:0/2681170626 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556abe2632d0 msgr2=0x556abe26c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:56:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.326+0000 7f93faffd700 1 --2- 172.21.15.39:0/2681170626 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556abe2632d0 0x556abe26c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.326+0000 7f93faffd700 1 -- 172.21.15.39:0/2681170626 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93fc194dc0 con 0x556abe25d580 2023-07-21T18:56:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.326+0000 7f93faffd700 1 --2- 172.21.15.39:0/2681170626 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556abe25d580 0x556abe26c190 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f93f0000c00 tx=0x7f93f00059e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb5de95cd7936cfa server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:56:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.326+0000 7f93daffd700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:56:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.326+0000 7f93daffd700 1 -- 172.21.15.39:0/2681170626 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f001c030 con 0x556abe25d580 2023-07-21T18:56:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.327+0000 7f93daffd700 1 -- 172.21.15.39:0/2681170626 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f93f00052b0 con 0x556abe25d580 2023-07-21T18:56:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.327+0000 7f940dfd5b80 1 -- 172.21.15.39:0/2681170626 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f93f001d040 con 0x556abe25d580 2023-07-21T18:56:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.327+0000 7f93daffd700 1 -- 172.21.15.39:0/2681170626 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93f000b560 con 0x556abe25d580 2023-07-21T18:56:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.327+0000 7f940dfd5b80 10 client.17292 fetch_fsmap learned FSMap version 86 2023-07-21T18:56:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.327+0000 7f940dfd5b80 1 -- 172.21.15.39:0/2681170626 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) v3 -- 0x556abdfdcd80 con 0x556abe25d580 2023-07-21T18:56:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.327+0000 7f93daffd700 1 -- 172.21.15.39:0/2681170626 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93f001d040 con 0x556abe25d580 2023-07-21T18:56:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.327+0000 7f93daffd700 1 -- 172.21.15.39:0/2681170626 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 86) v1 ==== 7538+0+0 (secure 0 0 0) 0x7f93f0009c70 con 0x556abe25d580 2023-07-21T18:56:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.327+0000 7f940dfd5b80 10 client.17292 fetch_fsmap finished waiting for FSMap version 86 2023-07-21T18:56:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.327+0000 7f940dfd5b80 10 client.17292 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:56:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.327+0000 7f940dfd5b80 20 client.17292 populate_metadata read hostname 'smithi039' 2023-07-21T18:56:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.327+0000 7f940dfd5b80 1 --2- 172.21.15.39:0/2681170626 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x556abe2cb220 0x556abe2cd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.327+0000 7f940dfd5b80 4 client.17292 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:56:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.327+0000 7f940dfd5b80 1 -- 172.21.15.39:0/2681170626 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556abe169ff0 con 0x556abe2cb220 2023-07-21T18:56:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.328+0000 7f93fb7fe700 1 --2- 172.21.15.39:0/2681170626 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x556abe2cb220 0x556abe2cd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.328+0000 7f93fb7fe700 1 --2- 172.21.15.39:0/2681170626 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x556abe2cb220 0x556abe2cd630 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5df142779cf75c9 server_cookie=1f2f09671569100f in_seq=0 out_seq=0 2023-07-21T18:56:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.328+0000 7f93daffd700 10 client.17292 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:56:30.499 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.488+0000 7f93daffd700 1 -- 172.21.15.39:0/2681170626 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556abe169ff0 con 0x556abe2cb220 2023-07-21T18:56:30.499 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.488+0000 7f93daffd700 10 client.17292 handle_command_reply: tid=0 2023-07-21T18:56:30.499 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.488+0000 7f940dfd5b80 1 client.17292 shutdown 2023-07-21T18:56:30.499 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.488+0000 7f940dfd5b80 1 -- 172.21.15.39:0/2681170626 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556abe25d580 msgr2=0x556abe26c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:56:30.499 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.488+0000 7f940dfd5b80 1 --2- 172.21.15.39:0/2681170626 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556abe25d580 0x556abe26c190 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f93f0000c00 tx=0x7f93f00059e0 comp rx=0 tx=0).stop 2023-07-21T18:56:30.499 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.488+0000 7f940dfd5b80 1 -- 172.21.15.39:0/2681170626 shutdown_connections 2023-07-21T18:56:30.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.488+0000 7f940dfd5b80 1 --2- 172.21.15.39:0/2681170626 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x556abe2cb220 0x556abe2cd630 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:30.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.488+0000 7f940dfd5b80 1 --2- 172.21.15.39:0/2681170626 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556abe2632d0 0x556abe26c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:30.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.488+0000 7f940dfd5b80 1 --2- 172.21.15.39:0/2681170626 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556abe0b96b0 0x556abe26bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:30.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.488+0000 7f940dfd5b80 1 --2- 172.21.15.39:0/2681170626 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556abe25d580 0x556abe26c190 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:30.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.488+0000 7f93daffd700 0 client.17292 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:56:30.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.488+0000 7f940dfd5b80 1 -- 172.21.15.39:0/2681170626 >> 172.21.15.39:0/2681170626 conn(0x556abe252be0 msgr2=0x556abe263ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:56:30.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.488+0000 7f940dfd5b80 1 -- 172.21.15.39:0/2681170626 shutdown_connections 2023-07-21T18:56:30.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.488+0000 7f940dfd5b80 1 -- 172.21.15.39:0/2681170626 wait complete. 2023-07-21T18:56:30.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.488+0000 7f940dfd5b80 20 client.17292 trim_cache size 0 max 16384 2023-07-21T18:56:30.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:56:30.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:56:30.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:56:30.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:56:30.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.489+0000 7f9401e4f700 1 -- 172.21.15.39:0/3398760814 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f93e40a12c0 msgr2=0x7f93e40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:56:30.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.489+0000 7f9401e4f700 1 --2- 172.21.15.39:0/3398760814 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f93e40a12c0 0x7f93e40a3770 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f93fc0ffc00 tx=0x7f93ec009310 comp rx=0 tx=0).stop 2023-07-21T18:56:30.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.489+0000 7f9401e4f700 1 -- 172.21.15.39:0/3398760814 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93fc104ad0 msgr2=0x7f93fc1a1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:56:30.503 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.489+0000 7f9401e4f700 1 --2- 172.21.15.39:0/3398760814 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93fc104ad0 0x7f93fc1a1ad0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f93f400ef30 tx=0x7f93f400c3a0 comp rx=0 tx=0).stop 2023-07-21T18:56:30.503 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.489+0000 7f9401e4f700 1 -- 172.21.15.39:0/3398760814 shutdown_connections 2023-07-21T18:56:30.503 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.489+0000 7f9401e4f700 1 --2- 172.21.15.39:0/3398760814 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93fc104ad0 0x7f93fc1a1ad0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:30.503 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.489+0000 7f9401e4f700 1 --2- 172.21.15.39:0/3398760814 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f93fc0feac0 0x7f93fc196c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:30.503 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.489+0000 7f9401e4f700 1 --2- 172.21.15.39:0/3398760814 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f93e40a12c0 0x7f93e40a3770 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:30.503 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.489+0000 7f9401e4f700 1 --2- 172.21.15.39:0/3398760814 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f93fc0ff470 0x7f93fc197170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:30.503 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.489+0000 7f9401e4f700 1 -- 172.21.15.39:0/3398760814 >> 172.21.15.39:0/3398760814 conn(0x7f93fc075bd0 msgr2=0x7f93fc0fcde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:56:30.503 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.489+0000 7f9401e4f700 1 -- 172.21.15.39:0/3398760814 shutdown_connections 2023-07-21T18:56:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:30.489+0000 7f9401e4f700 1 -- 172.21.15.39:0/3398760814 wait complete. 2023-07-21T18:56:30.664 INFO:tasks.workunit.client.0.smithi039.stdout: 10 56129 31.10 MB/sec warmup 103 sec latency 137.990 ms 2023-07-21T18:56:30.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:56:30 smithi039 ceph-mon[96489]: pgmap v465: 97 pgs: 97 active+clean; 761 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 7.1 MiB/s wr, 669 op/s 2023-07-21T18:56:30.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:56:30 smithi193 ceph-mon[98204]: pgmap v465: 97 pgs: 97 active+clean; 761 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 7.1 MiB/s wr, 669 op/s 2023-07-21T18:56:30.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:56:30 smithi155 ceph-mon[97934]: pgmap v465: 97 pgs: 97 active+clean; 761 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 7.1 MiB/s wr, 669 op/s 2023-07-21T18:56:31.664 INFO:tasks.workunit.client.0.smithi039.stdout: 10 56781 31.15 MB/sec warmup 104 sec latency 66.420 ms 2023-07-21T18:56:32.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:56:32 smithi193 ceph-mon[98204]: pgmap v466: 97 pgs: 97 active+clean; 790 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 9.2 MiB/s wr, 892 op/s 2023-07-21T18:56:32.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:56:32 smithi155 ceph-mon[97934]: pgmap v466: 97 pgs: 97 active+clean; 790 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 9.2 MiB/s wr, 892 op/s 2023-07-21T18:56:32.664 INFO:tasks.workunit.client.0.smithi039.stdout: 10 57499 31.11 MB/sec warmup 105 sec latency 60.214 ms 2023-07-21T18:56:32.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:56:32 smithi039 ceph-mon[96489]: pgmap v466: 97 pgs: 97 active+clean; 790 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 9.2 MiB/s wr, 892 op/s 2023-07-21T18:56:33.664 INFO:tasks.workunit.client.0.smithi039.stdout: 10 58377 31.26 MB/sec warmup 106 sec latency 74.430 ms 2023-07-21T18:56:34.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:56:34 smithi193 ceph-mon[98204]: pgmap v467: 97 pgs: 97 active+clean; 790 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.8 MiB/s wr, 546 op/s 2023-07-21T18:56:34.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:56:34 smithi155 ceph-mon[97934]: pgmap v467: 97 pgs: 97 active+clean; 790 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.8 MiB/s wr, 546 op/s 2023-07-21T18:56:34.664 INFO:tasks.workunit.client.0.smithi039.stdout: 10 59001 31.40 MB/sec warmup 107 sec latency 289.716 ms 2023-07-21T18:56:34.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:56:34 smithi039 ceph-mon[96489]: pgmap v467: 97 pgs: 97 active+clean; 790 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.8 MiB/s wr, 546 op/s 2023-07-21T18:56:35.664 INFO:tasks.workunit.client.0.smithi039.stdout: 10 59723 31.41 MB/sec warmup 108 sec latency 50.246 ms 2023-07-21T18:56:36.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:56:36 smithi193 ceph-mon[98204]: pgmap v468: 97 pgs: 97 active+clean; 790 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.8 MiB/s wr, 546 op/s 2023-07-21T18:56:36.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:56:36 smithi155 ceph-mon[97934]: pgmap v468: 97 pgs: 97 active+clean; 790 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.8 MiB/s wr, 546 op/s 2023-07-21T18:56:36.665 INFO:tasks.workunit.client.0.smithi039.stdout: 10 60448 31.50 MB/sec warmup 109 sec latency 60.693 ms 2023-07-21T18:56:36.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:56:36 smithi039 ceph-mon[96489]: pgmap v468: 97 pgs: 97 active+clean; 790 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.8 MiB/s wr, 546 op/s 2023-07-21T18:56:37.665 INFO:tasks.workunit.client.0.smithi039.stdout: 10 60499 31.22 MB/sec warmup 110 sec latency 966.267 ms 2023-07-21T18:56:38.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:56:38 smithi193 ceph-mon[98204]: pgmap v469: 97 pgs: 97 active+clean; 817 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 9.0 MiB/s wr, 873 op/s 2023-07-21T18:56:38.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:56:38 smithi155 ceph-mon[97934]: pgmap v469: 97 pgs: 97 active+clean; 817 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 9.0 MiB/s wr, 873 op/s 2023-07-21T18:56:38.665 INFO:tasks.workunit.client.0.smithi039.stdout: 10 60790 31.04 MB/sec warmup 111 sec latency 976.466 ms 2023-07-21T18:56:38.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:56:38 smithi039 ceph-mon[96489]: pgmap v469: 97 pgs: 97 active+clean; 817 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 9.0 MiB/s wr, 873 op/s 2023-07-21T18:56:39.665 INFO:tasks.workunit.client.0.smithi039.stdout: 10 61268 30.99 MB/sec warmup 112 sec latency 121.221 ms 2023-07-21T18:56:40.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:56:40 smithi193 ceph-mon[98204]: pgmap v470: 97 pgs: 97 active+clean; 817 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.9 MiB/s wr, 598 op/s 2023-07-21T18:56:40.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:56:40 smithi155 ceph-mon[97934]: pgmap v470: 97 pgs: 97 active+clean; 817 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.9 MiB/s wr, 598 op/s 2023-07-21T18:56:40.665 INFO:tasks.workunit.client.0.smithi039.stdout: 10 62046 31.18 MB/sec warmup 113 sec latency 66.620 ms 2023-07-21T18:56:40.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:56:40 smithi039 ceph-mon[96489]: pgmap v470: 97 pgs: 97 active+clean; 817 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.9 MiB/s wr, 598 op/s 2023-07-21T18:56:41.665 INFO:tasks.workunit.client.0.smithi039.stdout: 10 62718 31.18 MB/sec warmup 114 sec latency 62.991 ms 2023-07-21T18:56:42.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:56:42 smithi193 ceph-mon[98204]: pgmap v471: 97 pgs: 97 active+clean; 824 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 24 MiB/s wr, 1.28k op/s 2023-07-21T18:56:42.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:56:42 smithi155 ceph-mon[97934]: pgmap v471: 97 pgs: 97 active+clean; 824 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 24 MiB/s wr, 1.28k op/s 2023-07-21T18:56:42.665 INFO:tasks.workunit.client.0.smithi039.stdout: 10 63420 31.19 MB/sec warmup 115 sec latency 107.113 ms 2023-07-21T18:56:42.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:56:42 smithi039 ceph-mon[96489]: pgmap v471: 97 pgs: 97 active+clean; 824 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 24 MiB/s wr, 1.28k op/s 2023-07-21T18:56:43.657 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.646+0000 7f8cee405700 1 -- 172.21.15.39:0/672635621 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ce8104990 msgr2=0x7f8ce8104d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:56:43.658 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.646+0000 7f8cee405700 1 --2- 172.21.15.39:0/672635621 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ce8104990 0x7f8ce8104d60 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f8cd8009990 tx=0x7f8cd8009ca0 comp rx=0 tx=0).stop 2023-07-21T18:56:43.658 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.647+0000 7f8cee405700 1 -- 172.21.15.39:0/672635621 shutdown_connections 2023-07-21T18:56:43.659 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.647+0000 7f8cee405700 1 --2- 172.21.15.39:0/672635621 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ce80ff330 0x7f8ce810b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:43.659 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.647+0000 7f8cee405700 1 --2- 172.21.15.39:0/672635621 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ce8104990 0x7f8ce8104d60 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:43.659 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.647+0000 7f8cee405700 1 --2- 172.21.15.39:0/672635621 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ce80fe980 0x7f8ce80fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:43.659 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.647+0000 7f8cee405700 1 -- 172.21.15.39:0/672635621 >> 172.21.15.39:0/672635621 conn(0x7f8ce80fa440 msgr2=0x7f8ce80fc850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:56:43.659 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.647+0000 7f8cee405700 1 -- 172.21.15.39:0/672635621 shutdown_connections 2023-07-21T18:56:43.659 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.647+0000 7f8cee405700 1 -- 172.21.15.39:0/672635621 wait complete. 2023-07-21T18:56:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.647+0000 7f8cee405700 1 Processor -- start 2023-07-21T18:56:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.648+0000 7f8cee405700 1 -- start start 2023-07-21T18:56:43.661 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.648+0000 7f8cee405700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ce80fe980 0x7f8ce81a1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:43.661 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.648+0000 7f8cee405700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ce80ff330 0x7f8ce81a24d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:43.661 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.648+0000 7f8cee405700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ce8104990 0x7f8ce819c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:43.661 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.648+0000 7f8cee405700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce819cea0 con 0x7f8ce80fe980 2023-07-21T18:56:43.661 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.648+0000 7f8cee405700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce819d010 con 0x7f8ce80ff330 2023-07-21T18:56:43.661 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.648+0000 7f8cee405700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce819d1d0 con 0x7f8ce8104990 2023-07-21T18:56:43.661 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.649+0000 7f8cec9a2700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ce8104990 0x7f8ce819c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.649+0000 7f8ce77fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ce80ff330 0x7f8ce81a24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.649+0000 7f8cec9a2700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ce8104990 0x7f8ce819c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:51900/0 (socket says 172.21.15.39:51900) 2023-07-21T18:56:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.649+0000 7f8cec9a2700 1 -- 172.21.15.39:0/1319307901 learned_addr learned my addr 172.21.15.39:0/1319307901 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:56:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.649+0000 7f8ce77fe700 1 --2- 172.21.15.39:0/1319307901 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ce80ff330 0x7f8ce81a24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:45942/0 (socket says 172.21.15.39:45942) 2023-07-21T18:56:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.649+0000 7f8ce7fff700 1 --2- 172.21.15.39:0/1319307901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ce80fe980 0x7f8ce81a1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.649+0000 7f8cec9a2700 1 -- 172.21.15.39:0/1319307901 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ce80ff330 msgr2=0x7f8ce81a24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:56:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.649+0000 7f8cec9a2700 1 --2- 172.21.15.39:0/1319307901 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ce80ff330 0x7f8ce81a24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.649+0000 7f8cec9a2700 1 -- 172.21.15.39:0/1319307901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ce80fe980 msgr2=0x7f8ce81a1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:56:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.649+0000 7f8cec9a2700 1 --2- 172.21.15.39:0/1319307901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ce80fe980 0x7f8ce81a1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.649+0000 7f8cec9a2700 1 -- 172.21.15.39:0/1319307901 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cd80095f0 con 0x7f8ce8104990 2023-07-21T18:56:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.649+0000 7f8cec9a2700 1 --2- 172.21.15.39:0/1319307901 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ce8104990 0x7f8ce819c7c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8ce000ef30 tx=0x7f8ce000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:56:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.649+0000 7f8ce57fa700 1 -- 172.21.15.39:0/1319307901 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce0018070 con 0x7f8ce8104990 2023-07-21T18:56:43.664 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.649+0000 7f8cee405700 1 -- 172.21.15.39:0/1319307901 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ce819d450 con 0x7f8ce8104990 2023-07-21T18:56:43.664 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.650+0000 7f8ce57fa700 1 -- 172.21.15.39:0/1319307901 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ce000f040 con 0x7f8ce8104990 2023-07-21T18:56:43.664 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.650+0000 7f8ce57fa700 1 -- 172.21.15.39:0/1319307901 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce000cd00 con 0x7f8ce8104990 2023-07-21T18:56:43.664 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.650+0000 7f8cee405700 1 -- 172.21.15.39:0/1319307901 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ce806a110 con 0x7f8ce8104990 2023-07-21T18:56:43.665 INFO:tasks.workunit.client.0.smithi039.stdout: 10 64363 31.33 MB/sec warmup 116 sec latency 52.797 ms 2023-07-21T18:56:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.655+0000 7f8cfa58bb80 1 Processor -- start 2023-07-21T18:56:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.655+0000 7f8ce57fa700 1 -- 172.21.15.39:0/1319307901 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f8ce0003a00 con 0x7f8ce8104990 2023-07-21T18:56:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.655+0000 7f8cfa58bb80 1 -- start start 2023-07-21T18:56:43.668 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.655+0000 7f8cfa58bb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563b8b5836b0 0x563b8b72cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:43.668 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.655+0000 7f8cfa58bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563b8b72d2d0 0x563b8b727040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:43.668 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.655+0000 7f8cfa58bb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b8b727580 0x563b8b7279d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:43.668 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.655+0000 7f8cfa58bb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b8b62cc50 con 0x563b8b5836b0 2023-07-21T18:56:43.668 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.655+0000 7f8cfa58bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b8b67f9e0 con 0x563b8b72d2d0 2023-07-21T18:56:43.668 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.655+0000 7f8cfa58bb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b8b678e70 con 0x563b8b727580 2023-07-21T18:56:43.669 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.656+0000 7f8ce77fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563b8b5836b0 0x563b8b72cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:43.669 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.656+0000 7f8ce77fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563b8b5836b0 0x563b8b72cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33350/0 (socket says 172.21.15.39:33350) 2023-07-21T18:56:43.669 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.656+0000 7f8ce77fe700 1 -- 172.21.15.39:0/1542156374 learned_addr learned my addr 172.21.15.39:0/1542156374 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:56:43.669 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.656+0000 7f8ce7fff700 1 --2- 172.21.15.39:0/1542156374 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563b8b72d2d0 0x563b8b727040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:43.669 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.656+0000 7f8ce77fe700 1 --2- 172.21.15.39:0/1542156374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b8b727580 0x563b8b7279d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:43.669 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.656+0000 7f8ce57fa700 1 --2- 172.21.15.39:0/1319307901 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f8cd40a12c0 0x7f8cd40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:43.669 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.656+0000 7f8ce77fe700 1 -- 172.21.15.39:0/1542156374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b8b727580 msgr2=0x563b8b7279d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:56:43.669 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.656+0000 7f8ce77fe700 1 --2- 172.21.15.39:0/1542156374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b8b727580 0x563b8b7279d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:43.670 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.656+0000 7f8ce77fe700 1 -- 172.21.15.39:0/1542156374 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563b8b72d2d0 msgr2=0x563b8b727040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:56:43.670 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.656+0000 7f8ce77fe700 1 --2- 172.21.15.39:0/1542156374 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563b8b72d2d0 0x563b8b727040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:43.670 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.656+0000 7f8ce77fe700 1 -- 172.21.15.39:0/1542156374 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b8b4a6d80 con 0x563b8b5836b0 2023-07-21T18:56:43.670 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.656+0000 7f8cec9a2700 1 --2- 172.21.15.39:0/1319307901 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f8cd40a12c0 0x7f8cd40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:43.673 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.656+0000 7f8ce57fa700 1 -- 172.21.15.39:0/1319307901 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f8ce00cc820 con 0x7f8ce8104990 2023-07-21T18:56:43.673 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.656+0000 7f8ce77fe700 1 --2- 172.21.15.39:0/1542156374 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563b8b5836b0 0x563b8b72cd90 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f8cd00177d0 tx=0x7f8cd0009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e01d24d6b24a2186 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:56:43.673 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.656+0000 7f8cde7fc700 1 -- 172.21.15.39:0/1542156374 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd0018450 con 0x563b8b5836b0 2023-07-21T18:56:43.673 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.657+0000 7f8cde7fc700 1 -- 172.21.15.39:0/1542156374 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8cd0018a70 con 0x563b8b5836b0 2023-07-21T18:56:43.673 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.657+0000 7f8cde7fc700 1 -- 172.21.15.39:0/1542156374 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd00169e0 con 0x563b8b5836b0 2023-07-21T18:56:43.673 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.657+0000 7f8cfa58bb80 1 -- 172.21.15.39:0/1542156374 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563b8b5836b0 msgr2=0x563b8b72cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:56:43.674 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.657+0000 7f8cec9a2700 1 --2- 172.21.15.39:0/1319307901 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f8cd40a12c0 0x7f8cd40a3770 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8ce00996a0 tx=0x7f8ce0004dc0 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:56:43.674 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.657+0000 7f8cfa58bb80 1 --2- 172.21.15.39:0/1542156374 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563b8b5836b0 0x563b8b72cd90 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f8cd00177d0 tx=0x7f8cd0009580 comp rx=0 tx=0).stop 2023-07-21T18:56:43.674 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.658+0000 7f8cfa58bb80 1 -- 172.21.15.39:0/1542156374 shutdown_connections 2023-07-21T18:56:43.674 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.658+0000 7f8cfa58bb80 1 --2- 172.21.15.39:0/1542156374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b8b727580 0x563b8b7279d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:43.675 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.658+0000 7f8cfa58bb80 1 --2- 172.21.15.39:0/1542156374 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563b8b5836b0 0x563b8b72cd90 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:43.675 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.658+0000 7f8cfa58bb80 1 --2- 172.21.15.39:0/1542156374 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563b8b72d2d0 0x563b8b727040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:43.675 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.658+0000 7f8cfa58bb80 1 -- 172.21.15.39:0/1542156374 >> 172.21.15.39:0/1542156374 conn(0x563b8b71cbe0 msgr2=0x563b8b721f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:56:43.675 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.658+0000 7f8cfa58bb80 1 -- 172.21.15.39:0/1542156374 shutdown_connections 2023-07-21T18:56:43.675 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.658+0000 7f8cfa58bb80 1 -- 172.21.15.39:0/1542156374 wait complete. 2023-07-21T18:56:43.675 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.658+0000 7f8cfa58bb80 1 -- start start 2023-07-21T18:56:43.675 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.658+0000 7f8cfa58bb80 1 Processor -- start 2023-07-21T18:56:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.658+0000 7f8cfa58bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563b8b5836b0 0x563b8b7308d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.659+0000 7f8cfa58bb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563b8b727580 0x563b8b730e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.659+0000 7f8cfa58bb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b8b72d2d0 0x563b8b731350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.659+0000 7f8cfa58bb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b8b618a60 con 0x563b8b727580 2023-07-21T18:56:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.659+0000 7f8cfa58bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b8b618760 con 0x563b8b5836b0 2023-07-21T18:56:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.659+0000 7f8cfa58bb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b8b4fc8c0 con 0x563b8b72d2d0 2023-07-21T18:56:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.659+0000 7f8cfa58bb80 10 client.0 _ll_register_callbacks cb 0x563b8b616700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:56:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.659+0000 7f8ce7fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563b8b727580 0x563b8b730e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.659+0000 7f8ce77fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b8b72d2d0 0x563b8b731350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.659+0000 7f8ce7fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563b8b727580 0x563b8b730e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33360/0 (socket says 172.21.15.39:33360) 2023-07-21T18:56:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.659+0000 7f8ce7fff700 1 -- 172.21.15.39:0/3381871648 learned_addr learned my addr 172.21.15.39:0/3381871648 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:56:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.659+0000 7f8ce77fe700 1 --2- 172.21.15.39:0/3381871648 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563b8b5836b0 0x563b8b7308d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.659+0000 7f8ce7fff700 1 -- 172.21.15.39:0/3381871648 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b8b72d2d0 msgr2=0x563b8b731350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:56:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.659+0000 7f8ce7fff700 1 --2- 172.21.15.39:0/3381871648 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b8b72d2d0 0x563b8b731350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.659+0000 7f8ce7fff700 1 -- 172.21.15.39:0/3381871648 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563b8b5836b0 msgr2=0x563b8b7308d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:56:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.659+0000 7f8ce7fff700 1 --2- 172.21.15.39:0/3381871648 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563b8b5836b0 0x563b8b7308d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.659+0000 7f8ce7fff700 1 -- 172.21.15.39:0/3381871648 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cd0017480 con 0x563b8b727580 2023-07-21T18:56:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.659+0000 7f8ce77fe700 1 --2- 172.21.15.39:0/3381871648 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b8b72d2d0 0x563b8b731350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:56:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.659+0000 7f8ce7fff700 1 --2- 172.21.15.39:0/3381871648 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563b8b727580 0x563b8b730e10 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f8ce80ffac0 tx=0x7f8cd8005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2c493dba47ec419 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:56:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.659+0000 7f8cc77fe700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:56:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.659+0000 7f8cc77fe700 1 -- 172.21.15.39:0/3381871648 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd801d070 con 0x563b8b727580 2023-07-21T18:56:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.659+0000 7f8cc77fe700 1 -- 172.21.15.39:0/3381871648 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f8cd800be50 con 0x563b8b727580 2023-07-21T18:56:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.660+0000 7f8cfa58bb80 1 -- 172.21.15.39:0/3381871648 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563b8b5d2490 con 0x563b8b727580 2023-07-21T18:56:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.660+0000 7f8cc77fe700 1 -- 172.21.15.39:0/3381871648 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8cd8004700 con 0x563b8b727580 2023-07-21T18:56:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.660+0000 7f8cfa58bb80 10 client.17304 fetch_fsmap learned FSMap version 86 2023-07-21T18:56:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.660+0000 7f8cfa58bb80 1 -- 172.21.15.39:0/3381871648 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) v3 -- 0x563b8b48fc00 con 0x563b8b727580 2023-07-21T18:56:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.660+0000 7f8cc77fe700 1 -- 172.21.15.39:0/3381871648 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563b8b5d2490 con 0x563b8b727580 2023-07-21T18:56:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.660+0000 7f8cc77fe700 1 -- 172.21.15.39:0/3381871648 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 86) v1 ==== 7538+0+0 (secure 0 0 0) 0x7f8cd800f960 con 0x563b8b727580 2023-07-21T18:56:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.660+0000 7f8cfa58bb80 10 client.17304 fetch_fsmap finished waiting for FSMap version 86 2023-07-21T18:56:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.660+0000 7f8cfa58bb80 10 client.17304 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:56:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.660+0000 7f8cfa58bb80 20 client.17304 populate_metadata read hostname 'smithi039' 2023-07-21T18:56:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.660+0000 7f8cfa58bb80 1 --2- 172.21.15.39:0/3381871648 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x563b8b740c60 0x563b8b743070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.660+0000 7f8cfa58bb80 4 client.17304 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:56:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.660+0000 7f8cfa58bb80 1 -- 172.21.15.39:0/3381871648 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563b8b671e50 con 0x563b8b740c60 2023-07-21T18:56:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.661+0000 7f8ce77fe700 1 --2- 172.21.15.39:0/3381871648 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x563b8b740c60 0x563b8b743070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.662+0000 7f8cc77fe700 10 client.17304 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:56:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.662+0000 7f8ce77fe700 1 --2- 172.21.15.39:0/3381871648 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x563b8b740c60 0x563b8b743070 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be76686725a9f4fa server_cookie=9a18df8cc5982f5e in_seq=0 out_seq=0 2023-07-21T18:56:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.663+0000 7f8cc77fe700 1 -- 172.21.15.39:0/3381871648 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563b8b671e50 con 0x563b8b740c60 2023-07-21T18:56:43.682 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.663+0000 7f8cc77fe700 10 client.17304 handle_command_reply: tid=0 2023-07-21T18:56:43.682 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.663+0000 7f8cfa58bb80 1 client.17304 shutdown 2023-07-21T18:56:43.682 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.663+0000 7f8cfa58bb80 1 -- 172.21.15.39:0/3381871648 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563b8b727580 msgr2=0x563b8b730e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:56:43.682 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.663+0000 7f8cfa58bb80 1 --2- 172.21.15.39:0/3381871648 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563b8b727580 0x563b8b730e10 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f8ce80ffac0 tx=0x7f8cd8005800 comp rx=0 tx=0).stop 2023-07-21T18:56:43.682 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.663+0000 7f8cfa58bb80 1 -- 172.21.15.39:0/3381871648 shutdown_connections 2023-07-21T18:56:43.682 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.663+0000 7f8cfa58bb80 1 --2- 172.21.15.39:0/3381871648 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x563b8b740c60 0x563b8b743070 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:43.682 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.663+0000 7f8cfa58bb80 1 --2- 172.21.15.39:0/3381871648 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563b8b5836b0 0x563b8b7308d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:43.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.663+0000 7f8cc77fe700 0 client.17304 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:56:43.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.663+0000 7f8cfa58bb80 1 --2- 172.21.15.39:0/3381871648 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b8b72d2d0 0x563b8b731350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:43.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.663+0000 7f8cfa58bb80 1 --2- 172.21.15.39:0/3381871648 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563b8b727580 0x563b8b730e10 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:43.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.663+0000 7f8cfa58bb80 1 -- 172.21.15.39:0/3381871648 >> 172.21.15.39:0/3381871648 conn(0x563b8b71cbe0 msgr2=0x563b8b7214e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:56:43.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.663+0000 7f8cfa58bb80 1 -- 172.21.15.39:0/3381871648 shutdown_connections 2023-07-21T18:56:43.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.663+0000 7f8cfa58bb80 1 -- 172.21.15.39:0/3381871648 wait complete. 2023-07-21T18:56:43.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.663+0000 7f8cfa58bb80 20 client.17304 trim_cache size 0 max 16384 2023-07-21T18:56:43.694 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.683+0000 7f8cfa58bb80 1 Processor -- start 2023-07-21T18:56:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.683+0000 7f8cfa58bb80 1 -- start start 2023-07-21T18:56:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.683+0000 7f8cfa58bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563b8b5836b0 0x563b8b727040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.683+0000 7f8cfa58bb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563b8b727580 0x563b8b737be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.683+0000 7f8cfa58bb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b8b72d2d0 0x563b8b72c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.683+0000 7f8cfa58bb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b8b71ed00 con 0x563b8b727580 2023-07-21T18:56:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.683+0000 7f8cfa58bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b8b719bb0 con 0x563b8b5836b0 2023-07-21T18:56:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.683+0000 7f8cfa58bb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b8b722fb0 con 0x563b8b72d2d0 2023-07-21T18:56:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.683+0000 7f8ce7fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563b8b727580 0x563b8b737be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.683+0000 7f8ce7fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563b8b727580 0x563b8b737be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33368/0 (socket says 172.21.15.39:33368) 2023-07-21T18:56:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.683+0000 7f8ce7fff700 1 -- 172.21.15.39:0/2409402584 learned_addr learned my addr 172.21.15.39:0/2409402584 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:56:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.683+0000 7f8ce77fe700 1 --2- 172.21.15.39:0/2409402584 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563b8b5836b0 0x563b8b727040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.683+0000 7f8ce77fe700 1 --2- 172.21.15.39:0/2409402584 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b8b72d2d0 0x563b8b72c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.684+0000 7f8ce7fff700 1 -- 172.21.15.39:0/2409402584 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b8b72d2d0 msgr2=0x563b8b72c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:56:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.684+0000 7f8ce7fff700 1 --2- 172.21.15.39:0/2409402584 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b8b72d2d0 0x563b8b72c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.684+0000 7f8ce7fff700 1 -- 172.21.15.39:0/2409402584 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563b8b5836b0 msgr2=0x563b8b727040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:56:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.684+0000 7f8ce7fff700 1 --2- 172.21.15.39:0/2409402584 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563b8b5836b0 0x563b8b727040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.684+0000 7f8ce7fff700 1 -- 172.21.15.39:0/2409402584 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b8b4cd1f0 con 0x563b8b727580 2023-07-21T18:56:43.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.684+0000 7f8ce7fff700 1 --2- 172.21.15.39:0/2409402584 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563b8b727580 0x563b8b737be0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f8cd800fd40 tx=0x7f8cd8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc60915a1e4ec634 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:56:43.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.684+0000 7f8cdeffd700 1 -- 172.21.15.39:0/2409402584 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd801d070 con 0x563b8b727580 2023-07-21T18:56:43.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.684+0000 7f8cdeffd700 1 -- 172.21.15.39:0/2409402584 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8cd803b070 con 0x563b8b727580 2023-07-21T18:56:43.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.684+0000 7f8cdeffd700 1 -- 172.21.15.39:0/2409402584 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd80249b0 con 0x563b8b727580 2023-07-21T18:56:43.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.685+0000 7f8cfa58bb80 1 -- 172.21.15.39:0/2409402584 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563b8b727580 msgr2=0x563b8b737be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:56:43.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.685+0000 7f8cfa58bb80 1 --2- 172.21.15.39:0/2409402584 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563b8b727580 0x563b8b737be0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f8cd800fd40 tx=0x7f8cd8005fd0 comp rx=0 tx=0).stop 2023-07-21T18:56:43.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.685+0000 7f8cfa58bb80 1 -- 172.21.15.39:0/2409402584 shutdown_connections 2023-07-21T18:56:43.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.685+0000 7f8cfa58bb80 1 --2- 172.21.15.39:0/2409402584 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563b8b5836b0 0x563b8b727040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:43.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.685+0000 7f8cfa58bb80 1 --2- 172.21.15.39:0/2409402584 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b8b72d2d0 0x563b8b72c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:43.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.685+0000 7f8cfa58bb80 1 --2- 172.21.15.39:0/2409402584 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563b8b727580 0x563b8b737be0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:43.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.685+0000 7f8cfa58bb80 1 -- 172.21.15.39:0/2409402584 >> 172.21.15.39:0/2409402584 conn(0x563b8b71cbe0 msgr2=0x563b8b72ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:56:43.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.685+0000 7f8cfa58bb80 1 -- 172.21.15.39:0/2409402584 shutdown_connections 2023-07-21T18:56:43.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.685+0000 7f8cfa58bb80 1 -- 172.21.15.39:0/2409402584 wait complete. 2023-07-21T18:56:43.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.686+0000 7f8cfa58bb80 1 -- start start 2023-07-21T18:56:43.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.686+0000 7f8cfa58bb80 1 Processor -- start 2023-07-21T18:56:43.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.686+0000 7f8cfa58bb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563b8b5836b0 0x563b8b78dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:43.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.686+0000 7f8cfa58bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563b8b727580 0x563b8b78e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:43.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.686+0000 7f8cfa58bb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b8b72d2d0 0x563b8b788a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.686+0000 7f8cfa58bb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b8b78e710 con 0x563b8b5836b0 2023-07-21T18:56:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.686+0000 7f8cfa58bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b8b78e880 con 0x563b8b727580 2023-07-21T18:56:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.686+0000 7f8cfa58bb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b8b788fb0 con 0x563b8b72d2d0 2023-07-21T18:56:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.686+0000 7f8cfa58bb80 10 client.0 _ll_register_callbacks cb 0x7f8cd8005350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:56:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.686+0000 7f8ce77fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563b8b5836b0 0x563b8b78dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.686+0000 7f8ce77fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563b8b5836b0 0x563b8b78dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33372/0 (socket says 172.21.15.39:33372) 2023-07-21T18:56:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.686+0000 7f8ce77fe700 1 -- 172.21.15.39:0/959465159 learned_addr learned my addr 172.21.15.39:0/959465159 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:56:43.703 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.686+0000 7f8ce77fe700 1 --2- 172.21.15.39:0/959465159 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b8b72d2d0 0x563b8b788a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:43.703 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.687+0000 7f8ce77fe700 1 -- 172.21.15.39:0/959465159 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563b8b727580 msgr2=0x563b8b78e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:56:43.703 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.687+0000 7f8ce77fe700 1 --2- 172.21.15.39:0/959465159 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563b8b727580 0x563b8b78e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:43.703 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.687+0000 7f8ce77fe700 1 -- 172.21.15.39:0/959465159 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563b8b5836b0 msgr2=0x563b8b78dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:56:43.703 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.687+0000 7f8ce77fe700 1 --2- 172.21.15.39:0/959465159 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563b8b5836b0 0x563b8b78dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:43.703 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.687+0000 7f8ce77fe700 1 -- 172.21.15.39:0/959465159 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563b8b48fc00 con 0x563b8b72d2d0 2023-07-21T18:56:43.703 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.687+0000 7f8ce77fe700 1 --2- 172.21.15.39:0/959465159 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b8b72d2d0 0x563b8b788a70 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8cd0006fd0 tx=0x7f8cd0027040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a99492420beb8f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:56:43.703 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.687+0000 7f8cc77fe700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2023-07-21T18:56:43.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.687+0000 7f8cc77fe700 1 -- 172.21.15.39:0/959465159 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd002b070 con 0x563b8b72d2d0 2023-07-21T18:56:43.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.687+0000 7f8cc77fe700 1 -- 172.21.15.39:0/959465159 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f8cd0018660 con 0x563b8b72d2d0 2023-07-21T18:56:43.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.688+0000 7f8cfa58bb80 1 -- 172.21.15.39:0/959465159 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8cd0018a70 con 0x563b8b72d2d0 2023-07-21T18:56:43.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.688+0000 7f8cc77fe700 1 -- 172.21.15.39:0/959465159 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8cd0033070 con 0x563b8b72d2d0 2023-07-21T18:56:43.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.688+0000 7f8cfa58bb80 10 client.26770 fetch_fsmap learned FSMap version 86 2023-07-21T18:56:43.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.688+0000 7f8cfa58bb80 1 -- 172.21.15.39:0/959465159 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) v3 -- 0x563b8b4a5250 con 0x563b8b72d2d0 2023-07-21T18:56:43.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.688+0000 7f8cc77fe700 1 -- 172.21.15.39:0/959465159 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8cd0018a70 con 0x563b8b72d2d0 2023-07-21T18:56:43.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.689+0000 7f8cc77fe700 1 -- 172.21.15.39:0/959465159 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 86) v1 ==== 7538+0+0 (secure 0 0 0) 0x7f8cd002e870 con 0x563b8b72d2d0 2023-07-21T18:56:43.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.689+0000 7f8cfa58bb80 10 client.26770 fetch_fsmap finished waiting for FSMap version 86 2023-07-21T18:56:43.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.689+0000 7f8cfa58bb80 10 client.26770 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:56:43.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.689+0000 7f8cfa58bb80 20 client.26770 populate_metadata read hostname 'smithi039' 2023-07-21T18:56:43.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.689+0000 7f8cfa58bb80 1 --2- 172.21.15.39:0/959465159 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x563b8b7a66b0 0x563b8b7a8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:43.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.689+0000 7f8cfa58bb80 4 client.26770 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:56:43.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.689+0000 7f8cfa58bb80 1 -- 172.21.15.39:0/959465159 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563b8b633ff0 con 0x563b8b7a66b0 2023-07-21T18:56:43.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.690+0000 7f8ce7fff700 1 --2- 172.21.15.39:0/959465159 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x563b8b7a66b0 0x563b8b7a8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:43.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.690+0000 7f8ce7fff700 1 --2- 172.21.15.39:0/959465159 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x563b8b7a66b0 0x563b8b7a8ac0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1508c9d39406393 server_cookie=ca05dec966b6e276 in_seq=0 out_seq=0 2023-07-21T18:56:43.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.690+0000 7f8cc77fe700 10 client.26770 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:56:43.829 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.816+0000 7f8cc77fe700 1 -- 172.21.15.39:0/959465159 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563b8b633ff0 con 0x563b8b7a66b0 2023-07-21T18:56:43.829 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.816+0000 7f8cc77fe700 10 client.26770 handle_command_reply: tid=0 2023-07-21T18:56:43.830 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.816+0000 7f8cfa58bb80 1 client.26770 shutdown 2023-07-21T18:56:43.830 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.816+0000 7f8cfa58bb80 1 -- 172.21.15.39:0/959465159 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b8b72d2d0 msgr2=0x563b8b788a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:56:43.830 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.816+0000 7f8cfa58bb80 1 --2- 172.21.15.39:0/959465159 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b8b72d2d0 0x563b8b788a70 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8cd0006fd0 tx=0x7f8cd0027040 comp rx=0 tx=0).stop 2023-07-21T18:56:43.830 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.816+0000 7f8cfa58bb80 1 -- 172.21.15.39:0/959465159 shutdown_connections 2023-07-21T18:56:43.831 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.816+0000 7f8cfa58bb80 1 --2- 172.21.15.39:0/959465159 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x563b8b7a66b0 0x563b8b7a8ac0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:43.831 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.816+0000 7f8cfa58bb80 1 --2- 172.21.15.39:0/959465159 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b8b72d2d0 0x563b8b788a70 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:43.831 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.816+0000 7f8cfa58bb80 1 --2- 172.21.15.39:0/959465159 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563b8b5836b0 0x563b8b78dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:43.831 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.816+0000 7f8cfa58bb80 1 --2- 172.21.15.39:0/959465159 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563b8b727580 0x563b8b78e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:43.831 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.816+0000 7f8cfa58bb80 1 -- 172.21.15.39:0/959465159 >> 172.21.15.39:0/959465159 conn(0x563b8b71cbe0 msgr2=0x563b8b72e320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:56:43.832 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.816+0000 7f8cc77fe700 0 client.26770 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:56:43.832 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.817+0000 7f8cfa58bb80 1 -- 172.21.15.39:0/959465159 shutdown_connections 2023-07-21T18:56:43.832 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.817+0000 7f8cfa58bb80 1 -- 172.21.15.39:0/959465159 wait complete. 2023-07-21T18:56:43.832 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.817+0000 7f8cfa58bb80 20 client.26770 trim_cache size 0 max 16384 2023-07-21T18:56:43.833 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:56:43.833 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:56:43.833 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:56:43.833 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:56:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.817+0000 7f8cee405700 1 -- 172.21.15.39:0/1319307901 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f8cd40a12c0 msgr2=0x7f8cd40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:56:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.817+0000 7f8cee405700 1 --2- 172.21.15.39:0/1319307901 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f8cd40a12c0 0x7f8cd40a3770 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8ce00996a0 tx=0x7f8ce0004dc0 comp rx=0 tx=0).stop 2023-07-21T18:56:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.817+0000 7f8cee405700 1 -- 172.21.15.39:0/1319307901 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ce8104990 msgr2=0x7f8ce819c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:56:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.817+0000 7f8cee405700 1 --2- 172.21.15.39:0/1319307901 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ce8104990 0x7f8ce819c7c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8ce000ef30 tx=0x7f8ce000c3a0 comp rx=0 tx=0).stop 2023-07-21T18:56:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.818+0000 7f8cee405700 1 -- 172.21.15.39:0/1319307901 shutdown_connections 2023-07-21T18:56:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.818+0000 7f8cee405700 1 --2- 172.21.15.39:0/1319307901 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ce8104990 0x7f8ce819c7c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.818+0000 7f8cee405700 1 --2- 172.21.15.39:0/1319307901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ce80fe980 0x7f8ce81a1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.818+0000 7f8cee405700 1 --2- 172.21.15.39:0/1319307901 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f8cd40a12c0 0x7f8cd40a3770 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.818+0000 7f8cee405700 1 --2- 172.21.15.39:0/1319307901 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ce80ff330 0x7f8ce81a24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.818+0000 7f8cee405700 1 -- 172.21.15.39:0/1319307901 >> 172.21.15.39:0/1319307901 conn(0x7f8ce80fa440 msgr2=0x7f8ce81025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:56:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.818+0000 7f8cee405700 1 -- 172.21.15.39:0/1319307901 shutdown_connections 2023-07-21T18:56:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:43.818+0000 7f8cee405700 1 -- 172.21.15.39:0/1319307901 wait complete. 2023-07-21T18:56:44.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:56:44 smithi193 ceph-mon[98204]: pgmap v472: 97 pgs: 97 active+clean; 824 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 21 MiB/s wr, 1.01k op/s 2023-07-21T18:56:44.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:56:44 smithi155 ceph-mon[97934]: pgmap v472: 97 pgs: 97 active+clean; 824 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 21 MiB/s wr, 1.01k op/s 2023-07-21T18:56:44.665 INFO:tasks.workunit.client.0.smithi039.stdout: 10 65170 31.53 MB/sec warmup 117 sec latency 73.119 ms 2023-07-21T18:56:44.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:56:44 smithi039 ceph-mon[96489]: pgmap v472: 97 pgs: 97 active+clean; 824 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 21 MiB/s wr, 1.01k op/s 2023-07-21T18:56:45.665 INFO:tasks.workunit.client.0.smithi039.stdout: 10 65800 31.58 MB/sec warmup 118 sec latency 50.121 ms 2023-07-21T18:56:46.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:56:46 smithi193 ceph-mon[98204]: pgmap v473: 97 pgs: 97 active+clean; 824 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 21 MiB/s wr, 1.01k op/s 2023-07-21T18:56:46.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:56:46 smithi155 ceph-mon[97934]: pgmap v473: 97 pgs: 97 active+clean; 824 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 21 MiB/s wr, 1.01k op/s 2023-07-21T18:56:46.665 INFO:tasks.workunit.client.0.smithi039.stdout: 10 66453 31.50 MB/sec warmup 119 sec latency 55.563 ms 2023-07-21T18:56:46.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:56:46 smithi039 ceph-mon[96489]: pgmap v473: 97 pgs: 97 active+clean; 824 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 21 MiB/s wr, 1.01k op/s 2023-07-21T18:56:48.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:56:48 smithi193 ceph-mon[98204]: pgmap v474: 97 pgs: 97 active+clean; 838 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 24 MiB/s wr, 1.32k op/s 2023-07-21T18:56:48.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:56:48 smithi155 ceph-mon[97934]: pgmap v474: 97 pgs: 97 active+clean; 838 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 24 MiB/s wr, 1.32k op/s 2023-07-21T18:56:48.665 INFO:tasks.workunit.client.0.smithi039.stdout: 10 68525 75.74 MB/sec execute 1 sec latency 56.197 ms 2023-07-21T18:56:48.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:56:48 smithi039 ceph-mon[96489]: pgmap v474: 97 pgs: 97 active+clean; 838 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 24 MiB/s wr, 1.32k op/s 2023-07-21T18:56:49.665 INFO:tasks.workunit.client.0.smithi039.stdout: 10 69142 54.50 MB/sec execute 2 sec latency 46.379 ms 2023-07-21T18:56:50.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:56:50 smithi193 ceph-mon[98204]: pgmap v475: 97 pgs: 97 active+clean; 838 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 20 MiB/s wr, 998 op/s 2023-07-21T18:56:50.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:56:50 smithi155 ceph-mon[97934]: pgmap v475: 97 pgs: 97 active+clean; 838 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 20 MiB/s wr, 998 op/s 2023-07-21T18:56:50.665 INFO:tasks.workunit.client.0.smithi039.stdout: 10 69703 45.82 MB/sec execute 3 sec latency 55.911 ms 2023-07-21T18:56:50.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:56:50 smithi039 ceph-mon[96489]: pgmap v475: 97 pgs: 97 active+clean; 838 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 20 MiB/s wr, 998 op/s 2023-07-21T18:56:51.665 INFO:tasks.workunit.client.0.smithi039.stdout: 10 70566 43.32 MB/sec execute 4 sec latency 47.002 ms 2023-07-21T18:56:52.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:56:52 smithi193 ceph-mon[98204]: pgmap v476: 97 pgs: 97 active+clean; 868 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 23 MiB/s wr, 1.27k op/s 2023-07-21T18:56:52.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:56:52 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:56:52.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:56:52 smithi155 ceph-mon[97934]: pgmap v476: 97 pgs: 97 active+clean; 868 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 23 MiB/s wr, 1.27k op/s 2023-07-21T18:56:52.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:56:52 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:56:52.665 INFO:tasks.workunit.client.0.smithi039.stdout: 10 71517 46.45 MB/sec execute 5 sec latency 44.158 ms 2023-07-21T18:56:52.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:56:52 smithi039 ceph-mon[96489]: pgmap v476: 97 pgs: 97 active+clean; 868 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 23 MiB/s wr, 1.27k op/s 2023-07-21T18:56:52.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:56:52 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:56:53.665 INFO:tasks.workunit.client.0.smithi039.stdout: 10 72277 45.51 MB/sec execute 6 sec latency 45.641 ms 2023-07-21T18:56:54.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:56:54 smithi193 ceph-mon[98204]: pgmap v477: 97 pgs: 97 active+clean; 868 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.4 MiB/s wr, 581 op/s 2023-07-21T18:56:54.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:56:54 smithi155 ceph-mon[97934]: pgmap v477: 97 pgs: 97 active+clean; 868 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.4 MiB/s wr, 581 op/s 2023-07-21T18:56:54.665 INFO:tasks.workunit.client.0.smithi039.stdout: 10 72911 44.20 MB/sec execute 7 sec latency 56.843 ms 2023-07-21T18:56:54.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:56:54 smithi039 ceph-mon[96489]: pgmap v477: 97 pgs: 97 active+clean; 868 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.4 MiB/s wr, 581 op/s 2023-07-21T18:56:55.665 INFO:tasks.workunit.client.0.smithi039.stdout: 10 73804 43.93 MB/sec execute 8 sec latency 80.129 ms 2023-07-21T18:56:56.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:56:56 smithi193 ceph-mon[98204]: pgmap v478: 97 pgs: 97 active+clean; 868 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.4 MiB/s wr, 581 op/s 2023-07-21T18:56:56.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:56:56 smithi155 ceph-mon[97934]: pgmap v478: 97 pgs: 97 active+clean; 868 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.4 MiB/s wr, 581 op/s 2023-07-21T18:56:56.665 INFO:tasks.workunit.client.0.smithi039.stdout: 10 74435 43.56 MB/sec execute 9 sec latency 197.855 ms 2023-07-21T18:56:56.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:56:56 smithi039 ceph-mon[96489]: pgmap v478: 97 pgs: 97 active+clean; 868 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.4 MiB/s wr, 581 op/s 2023-07-21T18:56:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.968+0000 7f4035a6a700 1 -- 172.21.15.39:0/962875282 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40300ff490 msgr2=0x7f403010b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:56:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.968+0000 7f4035a6a700 1 --2- 172.21.15.39:0/962875282 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40300ff490 0x7f403010b920 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f4020009990 tx=0x7f4020009ca0 comp rx=0 tx=0).stop 2023-07-21T18:56:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.968+0000 7f4035a6a700 1 -- 172.21.15.39:0/962875282 shutdown_connections 2023-07-21T18:56:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.968+0000 7f4035a6a700 1 --2- 172.21.15.39:0/962875282 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40300ff490 0x7f403010b920 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.968+0000 7f4035a6a700 1 --2- 172.21.15.39:0/962875282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4030104af0 0x7f4030104ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.968+0000 7f4035a6a700 1 --2- 172.21.15.39:0/962875282 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40300feae0 0x7f40300fef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.968+0000 7f4035a6a700 1 -- 172.21.15.39:0/962875282 >> 172.21.15.39:0/962875282 conn(0x7f4030076a10 msgr2=0x7f4030076e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:56:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.969+0000 7f4035a6a700 1 -- 172.21.15.39:0/962875282 shutdown_connections 2023-07-21T18:56:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.969+0000 7f4035a6a700 1 -- 172.21.15.39:0/962875282 wait complete. 2023-07-21T18:56:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.969+0000 7f4035a6a700 1 Processor -- start 2023-07-21T18:56:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.970+0000 7f4035a6a700 1 -- start start 2023-07-21T18:56:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.970+0000 7f4035a6a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40300feae0 0x7f403019bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.970+0000 7f4035a6a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40300ff490 0x7f403019c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.970+0000 7f4035a6a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4030104af0 0x7f4030195a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.970+0000 7f4035a6a700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40301960d0 con 0x7f4030104af0 2023-07-21T18:56:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.970+0000 7f4035a6a700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4030196240 con 0x7f40300ff490 2023-07-21T18:56:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.970+0000 7f4035a6a700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4030196400 con 0x7f40300feae0 2023-07-21T18:56:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.970+0000 7f402ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4030104af0 0x7f4030195a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.970+0000 7f402ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4030104af0 0x7f4030195a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33406/0 (socket says 172.21.15.39:33406) 2023-07-21T18:56:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.970+0000 7f402ffff700 1 -- 172.21.15.39:0/1860524633 learned_addr learned my addr 172.21.15.39:0/1860524633 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:56:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.970+0000 7f402effd700 1 --2- 172.21.15.39:0/1860524633 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40300ff490 0x7f403019c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.971+0000 7f402ffff700 1 -- 172.21.15.39:0/1860524633 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40300feae0 msgr2=0x7f403019bc00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-21T18:56:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.971+0000 7f402ffff700 1 --2- 172.21.15.39:0/1860524633 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40300feae0 0x7f403019bc00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.971+0000 7f402ffff700 1 -- 172.21.15.39:0/1860524633 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40300ff490 msgr2=0x7f403019c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:56:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.971+0000 7f402ffff700 1 --2- 172.21.15.39:0/1860524633 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40300ff490 0x7f403019c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.971+0000 7f402ffff700 1 -- 172.21.15.39:0/1860524633 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40200095f0 con 0x7f4030104af0 2023-07-21T18:56:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.971+0000 7f402ffff700 1 --2- 172.21.15.39:0/1860524633 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4030104af0 0x7f4030195a60 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f4020003b90 tx=0x7f40200037c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:56:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.971+0000 7f402cff9700 1 -- 172.21.15.39:0/1860524633 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402001d070 con 0x7f4030104af0 2023-07-21T18:56:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.971+0000 7f402cff9700 1 -- 172.21.15.39:0/1860524633 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4020003e60 con 0x7f4030104af0 2023-07-21T18:56:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.971+0000 7f4035a6a700 1 -- 172.21.15.39:0/1860524633 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4030196680 con 0x7f4030104af0 2023-07-21T18:56:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.971+0000 7f402cff9700 1 -- 172.21.15.39:0/1860524633 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402000fea0 con 0x7f4030104af0 2023-07-21T18:56:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.971+0000 7f4035a6a700 1 -- 172.21.15.39:0/1860524633 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f403006a110 con 0x7f4030104af0 2023-07-21T18:56:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.974+0000 7f402cff9700 1 -- 172.21.15.39:0/1860524633 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f402002b440 con 0x7f4030104af0 2023-07-21T18:56:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.975+0000 7f402cff9700 1 --2- 172.21.15.39:0/1860524633 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f40180a12c0 0x7f40180a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.975+0000 7f402f7fe700 1 --2- 172.21.15.39:0/1860524633 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f40180a12c0 0x7f40180a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:56.986 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.975+0000 7f402cff9700 1 -- 172.21.15.39:0/1860524633 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f40200cde40 con 0x7f4030104af0 2023-07-21T18:56:56.986 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.976+0000 7f402f7fe700 1 --2- 172.21.15.39:0/1860524633 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f40180a12c0 0x7f40180a3770 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f402800abb0 tx=0x7f4028009250 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:56:56.987 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.976+0000 7f4041bf0b80 1 Processor -- start 2023-07-21T18:56:56.987 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.976+0000 7f4041bf0b80 1 -- start start 2023-07-21T18:56:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.977+0000 7f4041bf0b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9b5d146b0 0x55c9b5ebdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.977+0000 7f4041bf0b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c9b5ebe2d0 0x55c9b5eb8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.977+0000 7f4041bf0b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9b5eb8580 0x55c9b5eb89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.977+0000 7f4041bf0b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9b5dbdc50 con 0x55c9b5ebe2d0 2023-07-21T18:56:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.977+0000 7f4041bf0b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9b5e109e0 con 0x55c9b5d146b0 2023-07-21T18:56:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.977+0000 7f4041bf0b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9b5e09e70 con 0x55c9b5eb8580 2023-07-21T18:56:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.977+0000 7f402f7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9b5d146b0 0x55c9b5ebdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.977+0000 7f402effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c9b5ebe2d0 0x55c9b5eb8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.977+0000 7f402f7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9b5d146b0 0x55c9b5ebdd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:46008/0 (socket says 172.21.15.39:46008) 2023-07-21T18:56:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.977+0000 7f402effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c9b5ebe2d0 0x55c9b5eb8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33414/0 (socket says 172.21.15.39:33414) 2023-07-21T18:56:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.977+0000 7f402effd700 1 -- 172.21.15.39:0/3770290089 learned_addr learned my addr 172.21.15.39:0/3770290089 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:56:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.977+0000 7f402ffff700 1 --2- 172.21.15.39:0/3770290089 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9b5eb8580 0x55c9b5eb89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.977+0000 7f402effd700 1 -- 172.21.15.39:0/3770290089 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9b5eb8580 msgr2=0x55c9b5eb89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:56:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.977+0000 7f402effd700 1 --2- 172.21.15.39:0/3770290089 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9b5eb8580 0x55c9b5eb89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.978+0000 7f402effd700 1 -- 172.21.15.39:0/3770290089 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9b5d146b0 msgr2=0x55c9b5ebdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:56:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.978+0000 7f402effd700 1 --2- 172.21.15.39:0/3770290089 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9b5d146b0 0x55c9b5ebdd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.978+0000 7f402effd700 1 -- 172.21.15.39:0/3770290089 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c9b5c37d80 con 0x55c9b5ebe2d0 2023-07-21T18:56:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.978+0000 7f402f7fe700 1 --2- 172.21.15.39:0/3770290089 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9b5d146b0 0x55c9b5ebdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:56:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.978+0000 7f402effd700 1 --2- 172.21.15.39:0/3770290089 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c9b5ebe2d0 0x55c9b5eb8040 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f402400ad10 tx=0x7f402400f6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8bd506e06e4f89b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:56:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.978+0000 7f4015ffb700 1 -- 172.21.15.39:0/3770290089 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402400ae60 con 0x55c9b5ebe2d0 2023-07-21T18:56:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.978+0000 7f4015ffb700 1 -- 172.21.15.39:0/3770290089 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40240044f0 con 0x55c9b5ebe2d0 2023-07-21T18:56:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.978+0000 7f4015ffb700 1 -- 172.21.15.39:0/3770290089 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4024018690 con 0x55c9b5ebe2d0 2023-07-21T18:56:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.979+0000 7f4041bf0b80 1 -- 172.21.15.39:0/3770290089 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c9b5ebe2d0 msgr2=0x55c9b5eb8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:56:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.979+0000 7f4041bf0b80 1 --2- 172.21.15.39:0/3770290089 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c9b5ebe2d0 0x55c9b5eb8040 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f402400ad10 tx=0x7f402400f6a0 comp rx=0 tx=0).stop 2023-07-21T18:56:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.979+0000 7f4041bf0b80 1 -- 172.21.15.39:0/3770290089 shutdown_connections 2023-07-21T18:56:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.979+0000 7f4041bf0b80 1 --2- 172.21.15.39:0/3770290089 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9b5d146b0 0x55c9b5ebdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.979+0000 7f4041bf0b80 1 --2- 172.21.15.39:0/3770290089 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9b5eb8580 0x55c9b5eb89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.979+0000 7f4041bf0b80 1 --2- 172.21.15.39:0/3770290089 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c9b5ebe2d0 0x55c9b5eb8040 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.979+0000 7f4041bf0b80 1 -- 172.21.15.39:0/3770290089 >> 172.21.15.39:0/3770290089 conn(0x55c9b5eadbe0 msgr2=0x55c9b5eb2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:56:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.979+0000 7f4041bf0b80 1 -- 172.21.15.39:0/3770290089 shutdown_connections 2023-07-21T18:56:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.979+0000 7f4041bf0b80 1 -- 172.21.15.39:0/3770290089 wait complete. 2023-07-21T18:56:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.979+0000 7f4041bf0b80 1 -- start start 2023-07-21T18:56:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.980+0000 7f4041bf0b80 1 Processor -- start 2023-07-21T18:56:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.980+0000 7f4041bf0b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c9b5d146b0 0x55c9b5ec45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.980+0000 7f4041bf0b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9b5eb8580 0x55c9b5ec18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.980+0000 7f4041bf0b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9b5ebe2d0 0x55c9b5ec2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.980+0000 7f4041bf0b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9b5da9a60 con 0x55c9b5d146b0 2023-07-21T18:56:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.980+0000 7f4041bf0b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9b5da9760 con 0x55c9b5eb8580 2023-07-21T18:56:56.996 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.980+0000 7f4041bf0b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9b5c8d8c0 con 0x55c9b5ebe2d0 2023-07-21T18:56:56.996 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.980+0000 7f4041bf0b80 10 client.0 _ll_register_callbacks cb 0x55c9b5da7700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:56:56.996 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.980+0000 7f402f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c9b5d146b0 0x55c9b5ec45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:56.996 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.980+0000 7f402f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c9b5d146b0 0x55c9b5ec45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33418/0 (socket says 172.21.15.39:33418) 2023-07-21T18:56:56.996 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.980+0000 7f402f7fe700 1 -- 172.21.15.39:0/955982874 learned_addr learned my addr 172.21.15.39:0/955982874 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:56:56.996 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.980+0000 7f402ffff700 1 --2- 172.21.15.39:0/955982874 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9b5ebe2d0 0x55c9b5ec2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:56.996 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.980+0000 7f402effd700 1 --2- 172.21.15.39:0/955982874 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9b5eb8580 0x55c9b5ec18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:56.997 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.981+0000 7f402ffff700 1 -- 172.21.15.39:0/955982874 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9b5eb8580 msgr2=0x55c9b5ec18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:56:56.997 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.981+0000 7f402ffff700 1 --2- 172.21.15.39:0/955982874 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9b5eb8580 0x55c9b5ec18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:56.997 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.981+0000 7f402ffff700 1 -- 172.21.15.39:0/955982874 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c9b5d146b0 msgr2=0x55c9b5ec45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:56:56.997 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.981+0000 7f402ffff700 1 --2- 172.21.15.39:0/955982874 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c9b5d146b0 0x55c9b5ec45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:56.997 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.981+0000 7f402ffff700 1 -- 172.21.15.39:0/955982874 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4028012070 con 0x55c9b5ebe2d0 2023-07-21T18:56:56.997 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.981+0000 7f402f7fe700 1 --2- 172.21.15.39:0/955982874 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c9b5d146b0 0x55c9b5ec45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:56:56.997 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.981+0000 7f402effd700 1 --2- 172.21.15.39:0/955982874 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9b5eb8580 0x55c9b5ec18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:56:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.981+0000 7f402ffff700 1 --2- 172.21.15.39:0/955982874 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9b5ebe2d0 0x55c9b5ec2010 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f40200cb5b0 tx=0x7f4020020370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e153c68500db2b0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:56:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.981+0000 7f400effd700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2023-07-21T18:56:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.981+0000 7f400effd700 1 -- 172.21.15.39:0/955982874 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40200a3380 con 0x55c9b5ebe2d0 2023-07-21T18:56:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.981+0000 7f4041bf0b80 1 -- 172.21.15.39:0/955982874 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c9b5d63490 con 0x55c9b5ebe2d0 2023-07-21T18:56:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.981+0000 7f400effd700 1 -- 172.21.15.39:0/955982874 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f402009a4b0 con 0x55c9b5ebe2d0 2023-07-21T18:56:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.981+0000 7f400effd700 1 -- 172.21.15.39:0/955982874 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40200b92e0 con 0x55c9b5ebe2d0 2023-07-21T18:56:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.981+0000 7f4041bf0b80 10 client.26776 fetch_fsmap learned FSMap version 87 2023-07-21T18:56:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.981+0000 7f4041bf0b80 1 -- 172.21.15.39:0/955982874 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=4+}) v3 -- 0x7f402400a970 con 0x55c9b5ebe2d0 2023-07-21T18:56:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.981+0000 7f400effd700 1 -- 172.21.15.39:0/955982874 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c9b5d63490 con 0x55c9b5ebe2d0 2023-07-21T18:56:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.982+0000 7f400effd700 1 -- 172.21.15.39:0/955982874 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 87) v1 ==== 7530+0+0 (secure 0 0 0) 0x7f40200a2ef0 con 0x55c9b5ebe2d0 2023-07-21T18:56:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.982+0000 7f4041bf0b80 10 client.26776 fetch_fsmap finished waiting for FSMap version 87 2023-07-21T18:56:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.982+0000 7f4041bf0b80 10 client.26776 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:56:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.982+0000 7f4041bf0b80 20 client.26776 populate_metadata read hostname 'smithi039' 2023-07-21T18:56:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.982+0000 7f4041bf0b80 1 --2- 172.21.15.39:0/955982874 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55c9b5ed19e0 0x55c9b5ed3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.982+0000 7f4041bf0b80 4 client.26776 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:56:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.982+0000 7f4041bf0b80 1 -- 172.21.15.39:0/955982874 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c9b5e02e50 con 0x55c9b5ed19e0 2023-07-21T18:56:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.982+0000 7f402f7fe700 1 --2- 172.21.15.39:0/955982874 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55c9b5ed19e0 0x55c9b5ed3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.983+0000 7f402f7fe700 1 --2- 172.21.15.39:0/955982874 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55c9b5ed19e0 0x55c9b5ed3df0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=970594cd82c4925b server_cookie=3ef06757b660ab4c in_seq=0 out_seq=0 2023-07-21T18:56:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.983+0000 7f400effd700 10 client.26776 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:56:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.984+0000 7f400effd700 1 -- 172.21.15.39:0/955982874 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c9b5e02e50 con 0x55c9b5ed19e0 2023-07-21T18:56:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.984+0000 7f400effd700 10 client.26776 handle_command_reply: tid=0 2023-07-21T18:56:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.984+0000 7f4041bf0b80 1 client.26776 shutdown 2023-07-21T18:56:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.985+0000 7f4041bf0b80 1 -- 172.21.15.39:0/955982874 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9b5ebe2d0 msgr2=0x55c9b5ec2010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:56:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.985+0000 7f4041bf0b80 1 --2- 172.21.15.39:0/955982874 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9b5ebe2d0 0x55c9b5ec2010 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f40200cb5b0 tx=0x7f4020020370 comp rx=0 tx=0).stop 2023-07-21T18:56:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.985+0000 7f4041bf0b80 1 -- 172.21.15.39:0/955982874 shutdown_connections 2023-07-21T18:56:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.985+0000 7f4041bf0b80 1 --2- 172.21.15.39:0/955982874 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55c9b5ed19e0 0x55c9b5ed3df0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.985+0000 7f4041bf0b80 1 --2- 172.21.15.39:0/955982874 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9b5ebe2d0 0x55c9b5ec2010 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.985+0000 7f400effd700 0 client.26776 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:56:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.985+0000 7f4041bf0b80 1 --2- 172.21.15.39:0/955982874 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c9b5d146b0 0x55c9b5ec45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.985+0000 7f4041bf0b80 1 --2- 172.21.15.39:0/955982874 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9b5eb8580 0x55c9b5ec18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.985+0000 7f4041bf0b80 1 -- 172.21.15.39:0/955982874 >> 172.21.15.39:0/955982874 conn(0x55c9b5eadbe0 msgr2=0x55c9b5eb1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:56:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.985+0000 7f4041bf0b80 1 -- 172.21.15.39:0/955982874 shutdown_connections 2023-07-21T18:56:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.985+0000 7f4041bf0b80 1 -- 172.21.15.39:0/955982874 wait complete. 2023-07-21T18:56:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:56.985+0000 7f4041bf0b80 20 client.26776 trim_cache size 0 max 16384 2023-07-21T18:56:57.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.006+0000 7f4041bf0b80 1 Processor -- start 2023-07-21T18:56:57.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.006+0000 7f4041bf0b80 1 -- start start 2023-07-21T18:56:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.007+0000 7f4041bf0b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9b5d146b0 0x55c9b5eb8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.007+0000 7f4041bf0b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c9b5eb8580 0x55c9b5ec8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.007+0000 7f4041bf0b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9b5ebe2d0 0x55c9b5eb2c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.007+0000 7f4041bf0b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9b5ec2e40 con 0x55c9b5eb8580 2023-07-21T18:56:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.007+0000 7f4041bf0b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9b5ec2610 con 0x55c9b5ebe2d0 2023-07-21T18:56:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.007+0000 7f4041bf0b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9b5ebd9c0 con 0x55c9b5d146b0 2023-07-21T18:56:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.007+0000 7f402effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c9b5eb8580 0x55c9b5ec8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.007+0000 7f402effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c9b5eb8580 0x55c9b5ec8ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33426/0 (socket says 172.21.15.39:33426) 2023-07-21T18:56:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.007+0000 7f402effd700 1 -- 172.21.15.39:0/499218222 learned_addr learned my addr 172.21.15.39:0/499218222 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:56:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.007+0000 7f402f7fe700 1 --2- 172.21.15.39:0/499218222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9b5d146b0 0x55c9b5eb8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.007+0000 7f402ffff700 1 --2- 172.21.15.39:0/499218222 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9b5ebe2d0 0x55c9b5eb2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.007+0000 7f402effd700 1 -- 172.21.15.39:0/499218222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9b5d146b0 msgr2=0x55c9b5eb8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:56:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.007+0000 7f402effd700 1 --2- 172.21.15.39:0/499218222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9b5d146b0 0x55c9b5eb8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.007+0000 7f402effd700 1 -- 172.21.15.39:0/499218222 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9b5ebe2d0 msgr2=0x55c9b5eb2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:56:57.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.007+0000 7f402effd700 1 --2- 172.21.15.39:0/499218222 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9b5ebe2d0 0x55c9b5eb2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:57.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.007+0000 7f402effd700 1 -- 172.21.15.39:0/499218222 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c9b5c20c00 con 0x55c9b5eb8580 2023-07-21T18:56:57.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.007+0000 7f402effd700 1 --2- 172.21.15.39:0/499218222 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c9b5eb8580 0x55c9b5ec8ca0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f4024019000 tx=0x7f402400aec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d82cc7f9e74a2a7f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:56:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.008+0000 7f40167fc700 1 -- 172.21.15.39:0/499218222 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4024005e70 con 0x55c9b5eb8580 2023-07-21T18:56:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.008+0000 7f40167fc700 1 -- 172.21.15.39:0/499218222 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f402400a3f0 con 0x55c9b5eb8580 2023-07-21T18:56:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.008+0000 7f40167fc700 1 -- 172.21.15.39:0/499218222 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40240057d0 con 0x55c9b5eb8580 2023-07-21T18:56:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.009+0000 7f4041bf0b80 1 -- 172.21.15.39:0/499218222 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c9b5eb8580 msgr2=0x55c9b5ec8ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:56:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.009+0000 7f4041bf0b80 1 --2- 172.21.15.39:0/499218222 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c9b5eb8580 0x55c9b5ec8ca0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f4024019000 tx=0x7f402400aec0 comp rx=0 tx=0).stop 2023-07-21T18:56:57.022 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.010+0000 7f4041bf0b80 1 -- 172.21.15.39:0/499218222 shutdown_connections 2023-07-21T18:56:57.022 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.010+0000 7f4041bf0b80 1 --2- 172.21.15.39:0/499218222 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9b5ebe2d0 0x55c9b5eb2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:57.022 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.010+0000 7f4041bf0b80 1 --2- 172.21.15.39:0/499218222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9b5d146b0 0x55c9b5eb8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:57.022 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.010+0000 7f4041bf0b80 1 --2- 172.21.15.39:0/499218222 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c9b5eb8580 0x55c9b5ec8ca0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:57.022 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.010+0000 7f4041bf0b80 1 -- 172.21.15.39:0/499218222 >> 172.21.15.39:0/499218222 conn(0x55c9b5eadbe0 msgr2=0x55c9b5ebbb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:56:57.022 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.011+0000 7f4041bf0b80 1 -- 172.21.15.39:0/499218222 shutdown_connections 2023-07-21T18:56:57.022 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.011+0000 7f4041bf0b80 1 -- 172.21.15.39:0/499218222 wait complete. 2023-07-21T18:56:57.025 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.011+0000 7f4041bf0b80 1 -- start start 2023-07-21T18:56:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.011+0000 7f4041bf0b80 1 Processor -- start 2023-07-21T18:56:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.012+0000 7f4041bf0b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9b5d146b0 0x55c9b5ebab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.012+0000 7f4041bf0b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c9b5eb8580 0x55c9b5ebb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.012+0000 7f4041bf0b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9b5ebe2d0 0x55c9b5eb9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.012+0000 7f4041bf0b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9b5eb9da0 con 0x55c9b5eb8580 2023-07-21T18:56:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.012+0000 7f4041bf0b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9b5eb9f10 con 0x55c9b5d146b0 2023-07-21T18:56:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.012+0000 7f4041bf0b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9b5eba0d0 con 0x55c9b5ebe2d0 2023-07-21T18:56:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.012+0000 7f4041bf0b80 10 client.0 _ll_register_callbacks cb 0x7f402009a920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:56:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.012+0000 7f402f7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9b5d146b0 0x55c9b5ebab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.012+0000 7f402ffff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9b5ebe2d0 0x55c9b5eb9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.012+0000 7f402effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c9b5eb8580 0x55c9b5ebb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.012+0000 7f402f7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9b5d146b0 0x55c9b5ebab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:46028/0 (socket says 172.21.15.39:46028) 2023-07-21T18:56:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.012+0000 7f402ffff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9b5ebe2d0 0x55c9b5eb9860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:51986/0 (socket says 172.21.15.39:51986) 2023-07-21T18:56:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.012+0000 7f402f7fe700 1 -- 172.21.15.39:0/1507351028 learned_addr learned my addr 172.21.15.39:0/1507351028 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:56:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.012+0000 7f402ffff700 1 -- 172.21.15.39:0/1507351028 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9b5d146b0 msgr2=0x55c9b5ebab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:56:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.012+0000 7f402ffff700 1 --2- 172.21.15.39:0/1507351028 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9b5d146b0 0x55c9b5ebab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.012+0000 7f402ffff700 1 -- 172.21.15.39:0/1507351028 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c9b5eb8580 msgr2=0x55c9b5ebb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:56:57.029 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.012+0000 7f402ffff700 1 --2- 172.21.15.39:0/1507351028 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c9b5eb8580 0x55c9b5ebb040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:57.029 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.012+0000 7f402ffff700 1 -- 172.21.15.39:0/1507351028 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c9b5c37d80 con 0x55c9b5ebe2d0 2023-07-21T18:56:57.029 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.012+0000 7f402f7fe700 1 --2- 172.21.15.39:0/1507351028 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9b5d146b0 0x55c9b5ebab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:56:57.029 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.012+0000 7f402effd700 1 --2- 172.21.15.39:0/1507351028 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c9b5eb8580 0x55c9b5ebb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:56:57.029 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.013+0000 7f402ffff700 1 --2- 172.21.15.39:0/1507351028 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9b5ebe2d0 0x55c9b5eb9860 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f402009af50 tx=0x7f402000b670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6c6af20210f5ba2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:56:57.029 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.013+0000 7f400effd700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2023-07-21T18:56:57.029 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.013+0000 7f400effd700 1 -- 172.21.15.39:0/1507351028 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402009ad90 con 0x55c9b5ebe2d0 2023-07-21T18:56:57.029 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.013+0000 7f400effd700 1 -- 172.21.15.39:0/1507351028 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f40200a3380 con 0x55c9b5ebe2d0 2023-07-21T18:56:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.013+0000 7f4041bf0b80 1 -- 172.21.15.39:0/1507351028 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f40240044f0 con 0x55c9b5ebe2d0 2023-07-21T18:56:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.013+0000 7f400effd700 1 -- 172.21.15.39:0/1507351028 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40200cb310 con 0x55c9b5ebe2d0 2023-07-21T18:56:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.013+0000 7f4041bf0b80 10 client.26782 fetch_fsmap learned FSMap version 87 2023-07-21T18:56:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.013+0000 7f4041bf0b80 1 -- 172.21.15.39:0/1507351028 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=4+}) v3 -- 0x55c9b5c5e1f0 con 0x55c9b5ebe2d0 2023-07-21T18:56:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.013+0000 7f400effd700 1 -- 172.21.15.39:0/1507351028 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40240044f0 con 0x55c9b5ebe2d0 2023-07-21T18:56:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.013+0000 7f400effd700 1 -- 172.21.15.39:0/1507351028 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 87) v1 ==== 7530+0+0 (secure 0 0 0) 0x7f40200054b0 con 0x55c9b5ebe2d0 2023-07-21T18:56:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.013+0000 7f4041bf0b80 10 client.26782 fetch_fsmap finished waiting for FSMap version 87 2023-07-21T18:56:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.013+0000 7f4041bf0b80 10 client.26782 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:56:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.013+0000 7f4041bf0b80 20 client.26782 populate_metadata read hostname 'smithi039' 2023-07-21T18:56:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.013+0000 7f4041bf0b80 1 --2- 172.21.15.39:0/1507351028 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55c9b5f261b0 0x55c9b5f285c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:56:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.013+0000 7f4041bf0b80 4 client.26782 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:56:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.014+0000 7f4041bf0b80 1 -- 172.21.15.39:0/1507351028 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c9b5dc4ff0 con 0x55c9b5f261b0 2023-07-21T18:56:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.014+0000 7f402f7fe700 1 --2- 172.21.15.39:0/1507351028 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55c9b5f261b0 0x55c9b5f285c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:56:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.014+0000 7f400effd700 10 client.26782 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:56:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.014+0000 7f402f7fe700 1 --2- 172.21.15.39:0/1507351028 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55c9b5f261b0 0x55c9b5f285c0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=408d400df09cb536 server_cookie=dd864355bd47ec94 in_seq=0 out_seq=0 2023-07-21T18:56:57.195 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.184+0000 7f400effd700 1 -- 172.21.15.39:0/1507351028 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c9b5dc4ff0 con 0x55c9b5f261b0 2023-07-21T18:56:57.195 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.184+0000 7f400effd700 10 client.26782 handle_command_reply: tid=0 2023-07-21T18:56:57.195 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.184+0000 7f4041bf0b80 1 client.26782 shutdown 2023-07-21T18:56:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.184+0000 7f4041bf0b80 1 -- 172.21.15.39:0/1507351028 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9b5ebe2d0 msgr2=0x55c9b5eb9860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:56:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.184+0000 7f4041bf0b80 1 --2- 172.21.15.39:0/1507351028 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9b5ebe2d0 0x55c9b5eb9860 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f402009af50 tx=0x7f402000b670 comp rx=0 tx=0).stop 2023-07-21T18:56:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.184+0000 7f4041bf0b80 1 -- 172.21.15.39:0/1507351028 shutdown_connections 2023-07-21T18:56:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.184+0000 7f4041bf0b80 1 --2- 172.21.15.39:0/1507351028 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55c9b5f261b0 0x55c9b5f285c0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.184+0000 7f4041bf0b80 1 --2- 172.21.15.39:0/1507351028 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9b5d146b0 0x55c9b5ebab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.184+0000 7f4041bf0b80 1 --2- 172.21.15.39:0/1507351028 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9b5ebe2d0 0x55c9b5eb9860 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.184+0000 7f400effd700 0 client.26782 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:56:57.197 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.184+0000 7f4041bf0b80 1 --2- 172.21.15.39:0/1507351028 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c9b5eb8580 0x55c9b5ebb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:57.197 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.184+0000 7f4041bf0b80 1 -- 172.21.15.39:0/1507351028 >> 172.21.15.39:0/1507351028 conn(0x55c9b5eadbe0 msgr2=0x55c9b5ebcab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:56:57.197 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.185+0000 7f4041bf0b80 1 -- 172.21.15.39:0/1507351028 shutdown_connections 2023-07-21T18:56:57.197 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.185+0000 7f4041bf0b80 1 -- 172.21.15.39:0/1507351028 wait complete. 2023-07-21T18:56:57.197 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.185+0000 7f4041bf0b80 20 client.26782 trim_cache size 0 max 16384 2023-07-21T18:56:57.198 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:56:57.198 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:56:57.198 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:56:57.198 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:56:57.199 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.185+0000 7f4035a6a700 1 -- 172.21.15.39:0/1860524633 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f40180a12c0 msgr2=0x7f40180a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:56:57.199 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.185+0000 7f4035a6a700 1 --2- 172.21.15.39:0/1860524633 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f40180a12c0 0x7f40180a3770 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f402800abb0 tx=0x7f4028009250 comp rx=0 tx=0).stop 2023-07-21T18:56:57.199 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.185+0000 7f4035a6a700 1 -- 172.21.15.39:0/1860524633 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4030104af0 msgr2=0x7f4030195a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:56:57.200 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.185+0000 7f4035a6a700 1 --2- 172.21.15.39:0/1860524633 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4030104af0 0x7f4030195a60 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f4020003b90 tx=0x7f40200037c0 comp rx=0 tx=0).stop 2023-07-21T18:56:57.200 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.186+0000 7f4035a6a700 1 -- 172.21.15.39:0/1860524633 shutdown_connections 2023-07-21T18:56:57.200 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.186+0000 7f4035a6a700 1 --2- 172.21.15.39:0/1860524633 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4030104af0 0x7f4030195a60 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:57.200 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.186+0000 7f4035a6a700 1 --2- 172.21.15.39:0/1860524633 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40300feae0 0x7f403019bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:57.200 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.186+0000 7f4035a6a700 1 --2- 172.21.15.39:0/1860524633 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f40180a12c0 0x7f40180a3770 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:57.200 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.186+0000 7f4035a6a700 1 --2- 172.21.15.39:0/1860524633 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40300ff490 0x7f403019c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:56:57.200 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.186+0000 7f4035a6a700 1 -- 172.21.15.39:0/1860524633 >> 172.21.15.39:0/1860524633 conn(0x7f4030076a10 msgr2=0x7f4030068380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:56:57.200 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.186+0000 7f4035a6a700 1 -- 172.21.15.39:0/1860524633 shutdown_connections 2023-07-21T18:56:57.201 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:56:57.186+0000 7f4035a6a700 1 -- 172.21.15.39:0/1860524633 wait complete. 2023-07-21T18:56:57.665 INFO:tasks.workunit.client.0.smithi039.stdout: 10 75112 42.76 MB/sec execute 10 sec latency 108.580 ms 2023-07-21T18:56:58.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:56:58 smithi193 ceph-mon[98204]: pgmap v479: 97 pgs: 97 active+clean; 898 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 8.1 MiB/s wr, 858 op/s 2023-07-21T18:56:58.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:56:58 smithi155 ceph-mon[97934]: pgmap v479: 97 pgs: 97 active+clean; 898 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 8.1 MiB/s wr, 858 op/s 2023-07-21T18:56:58.665 INFO:tasks.workunit.client.0.smithi039.stdout: 10 75785 42.02 MB/sec execute 11 sec latency 70.491 ms 2023-07-21T18:56:58.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:56:58 smithi039 ceph-mon[96489]: pgmap v479: 97 pgs: 97 active+clean; 898 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 8.1 MiB/s wr, 858 op/s 2023-07-21T18:56:59.665 INFO:tasks.workunit.client.0.smithi039.stdout: 10 76580 41.52 MB/sec execute 12 sec latency 51.280 ms 2023-07-21T18:57:00.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:57:00 smithi193 ceph-mon[98204]: pgmap v480: 97 pgs: 97 active+clean; 898 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.5 MiB/s wr, 547 op/s 2023-07-21T18:57:00.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:57:00 smithi155 ceph-mon[97934]: pgmap v480: 97 pgs: 97 active+clean; 898 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.5 MiB/s wr, 547 op/s 2023-07-21T18:57:00.665 INFO:tasks.workunit.client.0.smithi039.stdout: 10 77476 42.29 MB/sec execute 13 sec latency 68.029 ms 2023-07-21T18:57:00.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:57:00 smithi039 ceph-mon[96489]: pgmap v480: 97 pgs: 97 active+clean; 898 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.5 MiB/s wr, 547 op/s 2023-07-21T18:57:01.665 INFO:tasks.workunit.client.0.smithi039.stdout: 10 77604 40.17 MB/sec execute 14 sec latency 841.513 ms 2023-07-21T18:57:02.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:57:02 smithi193 ceph-mon[98204]: pgmap v481: 97 pgs: 97 active+clean; 920 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 8.6 MiB/s wr, 878 op/s 2023-07-21T18:57:02.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:57:02 smithi155 ceph-mon[97934]: pgmap v481: 97 pgs: 97 active+clean; 920 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 8.6 MiB/s wr, 878 op/s 2023-07-21T18:57:02.666 INFO:tasks.workunit.client.0.smithi039.stdout: 10 77604 37.49 MB/sec execute 15 sec latency 1841.624 ms 2023-07-21T18:57:02.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:57:02 smithi039 ceph-mon[96489]: pgmap v481: 97 pgs: 97 active+clean; 920 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 8.6 MiB/s wr, 878 op/s 2023-07-21T18:57:03.666 INFO:tasks.workunit.client.0.smithi039.stdout: 10 77875 36.32 MB/sec execute 16 sec latency 2087.193 ms 2023-07-21T18:57:04.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:57:04 smithi193 ceph-mon[98204]: pgmap v482: 97 pgs: 97 active+clean; 920 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.7 MiB/s wr, 608 op/s 2023-07-21T18:57:04.630 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:57:04 smithi155 ceph-mon[97934]: pgmap v482: 97 pgs: 97 active+clean; 920 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.7 MiB/s wr, 608 op/s 2023-07-21T18:57:04.666 INFO:tasks.workunit.client.0.smithi039.stdout: 10 78328 34.86 MB/sec execute 17 sec latency 168.846 ms 2023-07-21T18:57:04.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:57:04 smithi039 ceph-mon[96489]: pgmap v482: 97 pgs: 97 active+clean; 920 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.7 MiB/s wr, 608 op/s 2023-07-21T18:57:05.666 INFO:tasks.workunit.client.0.smithi039.stdout: 10 78802 34.57 MB/sec execute 18 sec latency 84.253 ms 2023-07-21T18:57:06.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:57:06 smithi193 ceph-mon[98204]: pgmap v483: 97 pgs: 97 active+clean; 920 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.7 MiB/s wr, 608 op/s 2023-07-21T18:57:06.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:57:06 smithi155 ceph-mon[97934]: pgmap v483: 97 pgs: 97 active+clean; 920 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.7 MiB/s wr, 608 op/s 2023-07-21T18:57:06.666 INFO:tasks.workunit.client.0.smithi039.stdout: 10 79579 34.91 MB/sec execute 19 sec latency 69.645 ms 2023-07-21T18:57:06.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:57:06 smithi039 ceph-mon[96489]: pgmap v483: 97 pgs: 97 active+clean; 920 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.7 MiB/s wr, 608 op/s 2023-07-21T18:57:07.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:57:07 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:57:07.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:57:07 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:57:07.666 INFO:tasks.workunit.client.0.smithi039.stdout: 10 80352 35.32 MB/sec execute 20 sec latency 48.217 ms 2023-07-21T18:57:07.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:57:07 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:57:08.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:57:08 smithi193 ceph-mon[98204]: pgmap v484: 97 pgs: 97 active+clean; 935 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.9 MiB/s wr, 937 op/s 2023-07-21T18:57:08.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:57:08 smithi155 ceph-mon[97934]: pgmap v484: 97 pgs: 97 active+clean; 935 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.9 MiB/s wr, 937 op/s 2023-07-21T18:57:08.666 INFO:tasks.workunit.client.0.smithi039.stdout: 10 81041 35.56 MB/sec execute 21 sec latency 49.066 ms 2023-07-21T18:57:08.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:57:08 smithi039 ceph-mon[96489]: pgmap v484: 97 pgs: 97 active+clean; 935 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.9 MiB/s wr, 937 op/s 2023-07-21T18:57:09.666 INFO:tasks.workunit.client.0.smithi039.stdout: 10 81612 35.28 MB/sec execute 22 sec latency 125.370 ms 2023-07-21T18:57:10.372 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.362+0000 7f8900919700 1 -- 172.21.15.39:0/311447328 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88fc0ff3e0 msgr2=0x7f88fc10b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:10.373 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.362+0000 7f8900919700 1 --2- 172.21.15.39:0/311447328 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88fc0ff3e0 0x7f88fc10b870 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f88f400b0a0 tx=0x7f88f400b3b0 comp rx=0 tx=0).stop 2023-07-21T18:57:10.373 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.362+0000 7f8900919700 1 -- 172.21.15.39:0/311447328 shutdown_connections 2023-07-21T18:57:10.373 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.362+0000 7f8900919700 1 --2- 172.21.15.39:0/311447328 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88fc104a40 0x7f88fc104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:10.373 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.362+0000 7f8900919700 1 --2- 172.21.15.39:0/311447328 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88fc0ff3e0 0x7f88fc10b870 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:10.373 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.362+0000 7f8900919700 1 --2- 172.21.15.39:0/311447328 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f88fc0fea30 0x7f88fc0feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:10.373 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.362+0000 7f8900919700 1 -- 172.21.15.39:0/311447328 >> 172.21.15.39:0/311447328 conn(0x7f88fc0fa4f0 msgr2=0x7f88fc0fc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:57:10.373 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.362+0000 7f8900919700 1 -- 172.21.15.39:0/311447328 shutdown_connections 2023-07-21T18:57:10.374 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.362+0000 7f8900919700 1 -- 172.21.15.39:0/311447328 wait complete. 2023-07-21T18:57:10.375 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.363+0000 7f8900919700 1 Processor -- start 2023-07-21T18:57:10.375 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.363+0000 7f8900919700 1 -- start start 2023-07-21T18:57:10.376 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.363+0000 7f8900919700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88fc0fea30 0x7f88fc197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:10.376 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.363+0000 7f8900919700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f88fc0ff3e0 0x7f88fc198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:10.376 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.363+0000 7f8900919700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88fc104a40 0x7f88fc19cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:10.376 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.363+0000 7f8900919700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88fc19d110 con 0x7f88fc0fea30 2023-07-21T18:57:10.376 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.363+0000 7f8900919700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88fc19d280 con 0x7f88fc104a40 2023-07-21T18:57:10.376 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.363+0000 7f8900919700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88fc19d440 con 0x7f88fc0ff3e0 2023-07-21T18:57:10.377 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.363+0000 7f88fa59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88fc0fea30 0x7f88fc197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:10.377 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.363+0000 7f88f9d9b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f88fc0ff3e0 0x7f88fc198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:10.377 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.363+0000 7f88fa59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88fc0fea30 0x7f88fc197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33448/0 (socket says 172.21.15.39:33448) 2023-07-21T18:57:10.377 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.363+0000 7f88fa59c700 1 -- 172.21.15.39:0/2285285474 learned_addr learned my addr 172.21.15.39:0/2285285474 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:57:10.377 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.363+0000 7f88fad9d700 1 --2- 172.21.15.39:0/2285285474 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88fc104a40 0x7f88fc19cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:10.377 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.363+0000 7f88fa59c700 1 -- 172.21.15.39:0/2285285474 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f88fc0ff3e0 msgr2=0x7f88fc198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:10.378 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.363+0000 7f88fa59c700 1 --2- 172.21.15.39:0/2285285474 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f88fc0ff3e0 0x7f88fc198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:10.378 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.364+0000 7f88fa59c700 1 -- 172.21.15.39:0/2285285474 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88fc104a40 msgr2=0x7f88fc19cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:10.378 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.364+0000 7f88fa59c700 1 --2- 172.21.15.39:0/2285285474 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88fc104a40 0x7f88fc19cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:10.378 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.364+0000 7f88fa59c700 1 -- 172.21.15.39:0/2285285474 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88f4009d00 con 0x7f88fc0fea30 2023-07-21T18:57:10.378 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.364+0000 7f88fad9d700 1 --2- 172.21.15.39:0/2285285474 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88fc104a40 0x7f88fc19cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:57:10.378 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.364+0000 7f88fa59c700 1 --2- 172.21.15.39:0/2285285474 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88fc0fea30 0x7f88fc197f20 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f88ec00b500 tx=0x7f88ec00b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:10.379 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.364+0000 7f88eb7fe700 1 -- 172.21.15.39:0/2285285474 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ec011830 con 0x7f88fc0fea30 2023-07-21T18:57:10.379 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.364+0000 7f88eb7fe700 1 -- 172.21.15.39:0/2285285474 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f88ec011e50 con 0x7f88fc0fea30 2023-07-21T18:57:10.379 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.364+0000 7f8900919700 1 -- 172.21.15.39:0/2285285474 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88fc19d720 con 0x7f88fc0fea30 2023-07-21T18:57:10.380 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.364+0000 7f88eb7fe700 1 -- 172.21.15.39:0/2285285474 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ec00f2e0 con 0x7f88fc0fea30 2023-07-21T18:57:10.380 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.364+0000 7f8900919700 1 -- 172.21.15.39:0/2285285474 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88fc1a4fe0 con 0x7f88fc0fea30 2023-07-21T18:57:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.366+0000 7f88eb7fe700 1 -- 172.21.15.39:0/2285285474 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f88ec010580 con 0x7f88fc0fea30 2023-07-21T18:57:10.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.367+0000 7f88eb7fe700 1 --2- 172.21.15.39:0/2285285474 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f88e40a1340 0x7f88e40a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:10.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.367+0000 7f88eb7fe700 1 -- 172.21.15.39:0/2285285474 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f88ec0cc620 con 0x7f88fc0fea30 2023-07-21T18:57:10.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.367+0000 7f88f9d9b700 1 --2- 172.21.15.39:0/2285285474 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f88e40a1340 0x7f88e40a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:10.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.368+0000 7f88f9d9b700 1 --2- 172.21.15.39:0/2285285474 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f88e40a1340 0x7f88e40a37f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f88fc199740 tx=0x7f88f0009260 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:10.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.368+0000 7f890ca9fb80 1 Processor -- start 2023-07-21T18:57:10.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.368+0000 7f890ca9fb80 1 -- start start 2023-07-21T18:57:10.387 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.368+0000 7f890ca9fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56174d3386b0 0x56174d4e1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:10.387 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.368+0000 7f890ca9fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56174d4e22d0 0x56174d4dc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:10.387 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.368+0000 7f890ca9fb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56174d4dc580 0x56174d4dc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:10.387 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.368+0000 7f890ca9fb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x56174d3e1c50 con 0x56174d4dc580 2023-07-21T18:57:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.368+0000 7f890ca9fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56174d4349e0 con 0x56174d4e22d0 2023-07-21T18:57:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.368+0000 7f890ca9fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x56174d42de70 con 0x56174d3386b0 2023-07-21T18:57:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.368+0000 7f88fa59c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56174d3386b0 0x56174d4e1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.368+0000 7f88fa59c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56174d3386b0 0x56174d4e1d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:52008/0 (socket says 172.21.15.39:52008) 2023-07-21T18:57:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.368+0000 7f88fa59c700 1 -- 172.21.15.39:0/1397434865 learned_addr learned my addr 172.21.15.39:0/1397434865 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:57:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.368+0000 7f88f9d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56174d4e22d0 0x56174d4dc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.369+0000 7f88fad9d700 1 --2- 172.21.15.39:0/1397434865 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56174d4dc580 0x56174d4dc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:10.389 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.369+0000 7f88fa59c700 1 -- 172.21.15.39:0/1397434865 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56174d4e22d0 msgr2=0x56174d4dc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:57:10.389 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.369+0000 7f88fa59c700 1 --2- 172.21.15.39:0/1397434865 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56174d4e22d0 0x56174d4dc040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:10.389 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.369+0000 7f88fa59c700 1 -- 172.21.15.39:0/1397434865 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56174d4dc580 msgr2=0x56174d4dc9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:57:10.389 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.369+0000 7f88fa59c700 1 --2- 172.21.15.39:0/1397434865 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56174d4dc580 0x56174d4dc9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:10.389 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.369+0000 7f88fa59c700 1 -- 172.21.15.39:0/1397434865 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56174d25bd80 con 0x56174d3386b0 2023-07-21T18:57:10.389 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.369+0000 7f88fa59c700 1 --2- 172.21.15.39:0/1397434865 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56174d3386b0 0x56174d4e1d90 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f88ec0994d0 tx=0x7f88ec099310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5fbd01cb64c6b5e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:10.389 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.369+0000 7f88e8ff9700 1 -- 172.21.15.39:0/1397434865 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ec0997d0 con 0x56174d3386b0 2023-07-21T18:57:10.390 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.369+0000 7f88e8ff9700 1 -- 172.21.15.39:0/1397434865 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f88ec09d800 con 0x56174d3386b0 2023-07-21T18:57:10.390 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.369+0000 7f88e8ff9700 1 -- 172.21.15.39:0/1397434865 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ec099c30 con 0x56174d3386b0 2023-07-21T18:57:10.390 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.370+0000 7f890ca9fb80 1 -- 172.21.15.39:0/1397434865 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56174d3386b0 msgr2=0x56174d4e1d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:10.390 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.370+0000 7f890ca9fb80 1 --2- 172.21.15.39:0/1397434865 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56174d3386b0 0x56174d4e1d90 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f88ec0994d0 tx=0x7f88ec099310 comp rx=0 tx=0).stop 2023-07-21T18:57:10.390 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.370+0000 7f890ca9fb80 1 -- 172.21.15.39:0/1397434865 shutdown_connections 2023-07-21T18:57:10.390 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.370+0000 7f890ca9fb80 1 --2- 172.21.15.39:0/1397434865 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56174d4dc580 0x56174d4dc9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:10.390 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.370+0000 7f890ca9fb80 1 --2- 172.21.15.39:0/1397434865 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56174d3386b0 0x56174d4e1d90 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:10.391 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.370+0000 7f890ca9fb80 1 --2- 172.21.15.39:0/1397434865 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56174d4e22d0 0x56174d4dc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:10.391 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.370+0000 7f890ca9fb80 1 -- 172.21.15.39:0/1397434865 >> 172.21.15.39:0/1397434865 conn(0x56174d4d1be0 msgr2=0x56174d4d6f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:57:10.391 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.370+0000 7f890ca9fb80 1 -- 172.21.15.39:0/1397434865 shutdown_connections 2023-07-21T18:57:10.391 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.370+0000 7f890ca9fb80 1 -- 172.21.15.39:0/1397434865 wait complete. 2023-07-21T18:57:10.391 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.370+0000 7f890ca9fb80 1 -- start start 2023-07-21T18:57:10.391 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.371+0000 7f890ca9fb80 1 Processor -- start 2023-07-21T18:57:10.392 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.371+0000 7f890ca9fb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56174d3386b0 0x56174d4e85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:10.392 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.371+0000 7f890ca9fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56174d4dc580 0x56174d4e5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:10.392 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.371+0000 7f890ca9fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56174d4e22d0 0x56174d4e5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:10.392 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.371+0000 7f890ca9fb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x56174d3cda60 con 0x56174d3386b0 2023-07-21T18:57:10.392 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.371+0000 7f890ca9fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56174d3cd760 con 0x56174d4e22d0 2023-07-21T18:57:10.392 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.371+0000 7f890ca9fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x56174d2b18c0 con 0x56174d4dc580 2023-07-21T18:57:10.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.371+0000 7f890ca9fb80 10 client.0 _ll_register_callbacks cb 0x56174d3cb700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:57:10.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.371+0000 7f88fa59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56174d3386b0 0x56174d4e85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:10.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.371+0000 7f88fa59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56174d3386b0 0x56174d4e85a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33462/0 (socket says 172.21.15.39:33462) 2023-07-21T18:57:10.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.371+0000 7f88fa59c700 1 -- 172.21.15.39:0/2369188184 learned_addr learned my addr 172.21.15.39:0/2369188184 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:57:10.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.371+0000 7f88f9d9b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56174d4dc580 0x56174d4e5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:10.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.371+0000 7f88fad9d700 1 --2- 172.21.15.39:0/2369188184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56174d4e22d0 0x56174d4e5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:10.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.371+0000 7f88fa59c700 1 -- 172.21.15.39:0/2369188184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56174d4dc580 msgr2=0x56174d4e5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:57:10.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.371+0000 7f88fa59c700 1 --2- 172.21.15.39:0/2369188184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56174d4dc580 0x56174d4e5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:10.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.371+0000 7f88fa59c700 1 -- 172.21.15.39:0/2369188184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56174d4e22d0 msgr2=0x56174d4e5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:57:10.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.371+0000 7f88fa59c700 1 --2- 172.21.15.39:0/2369188184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56174d4e22d0 0x56174d4e5ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:10.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.371+0000 7f88fa59c700 1 -- 172.21.15.39:0/2369188184 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88fc1a4fe0 con 0x56174d3386b0 2023-07-21T18:57:10.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.371+0000 7f88f9d9b700 1 --2- 172.21.15.39:0/2369188184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56174d4dc580 0x56174d4e5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:57:10.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.372+0000 7f88fa59c700 1 --2- 172.21.15.39:0/2369188184 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56174d3386b0 0x56174d4e85a0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f88ec082fd0 tx=0x7f88ec0c9df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95d7c8c5bc1fc07b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:10.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.372+0000 7f88e1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:57:10.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.372+0000 7f88e1ffb700 1 -- 172.21.15.39:0/2369188184 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ec0997d0 con 0x56174d3386b0 2023-07-21T18:57:10.395 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.372+0000 7f88e1ffb700 1 -- 172.21.15.39:0/2369188184 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f88ec0b9250 con 0x56174d3386b0 2023-07-21T18:57:10.395 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.372+0000 7f890ca9fb80 1 -- 172.21.15.39:0/2369188184 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56174d387490 con 0x56174d3386b0 2023-07-21T18:57:10.395 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.372+0000 7f88e1ffb700 1 -- 172.21.15.39:0/2369188184 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f88ec098cc0 con 0x56174d3386b0 2023-07-21T18:57:10.395 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.372+0000 7f890ca9fb80 10 client.17340 fetch_fsmap learned FSMap version 87 2023-07-21T18:57:10.395 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.372+0000 7f890ca9fb80 1 -- 172.21.15.39:0/2369188184 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=4+}) v3 -- 0x56174d244c00 con 0x56174d3386b0 2023-07-21T18:57:10.395 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.372+0000 7f88e1ffb700 1 -- 172.21.15.39:0/2369188184 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56174d387490 con 0x56174d3386b0 2023-07-21T18:57:10.395 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.372+0000 7f88e1ffb700 1 -- 172.21.15.39:0/2369188184 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 87) v1 ==== 7530+0+0 (secure 0 0 0) 0x7f88ec0a1870 con 0x56174d3386b0 2023-07-21T18:57:10.395 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.372+0000 7f890ca9fb80 10 client.17340 fetch_fsmap finished waiting for FSMap version 87 2023-07-21T18:57:10.396 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.372+0000 7f890ca9fb80 10 client.17340 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:57:10.396 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.372+0000 7f890ca9fb80 20 client.17340 populate_metadata read hostname 'smithi039' 2023-07-21T18:57:10.396 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.372+0000 7f890ca9fb80 1 --2- 172.21.15.39:0/2369188184 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x56174d4f57e0 0x56174d4f7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:10.396 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.372+0000 7f890ca9fb80 4 client.17340 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:57:10.396 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.372+0000 7f890ca9fb80 1 -- 172.21.15.39:0/2369188184 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56174d426e50 con 0x56174d4f57e0 2023-07-21T18:57:10.396 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.373+0000 7f88f9d9b700 1 --2- 172.21.15.39:0/2369188184 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x56174d4f57e0 0x56174d4f7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:10.397 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.373+0000 7f88f9d9b700 1 --2- 172.21.15.39:0/2369188184 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x56174d4f57e0 0x56174d4f7bf0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41100d8985928f95 server_cookie=697fcfbd6641b9bd in_seq=0 out_seq=0 2023-07-21T18:57:10.397 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.373+0000 7f88e1ffb700 10 client.17340 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:57:10.397 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.375+0000 7f88e1ffb700 1 -- 172.21.15.39:0/2369188184 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56174d426e50 con 0x56174d4f57e0 2023-07-21T18:57:10.397 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.375+0000 7f88e1ffb700 10 client.17340 handle_command_reply: tid=0 2023-07-21T18:57:10.397 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.375+0000 7f890ca9fb80 1 client.17340 shutdown 2023-07-21T18:57:10.398 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.375+0000 7f890ca9fb80 1 -- 172.21.15.39:0/2369188184 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56174d3386b0 msgr2=0x56174d4e85a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:10.398 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.375+0000 7f890ca9fb80 1 --2- 172.21.15.39:0/2369188184 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56174d3386b0 0x56174d4e85a0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f88ec082fd0 tx=0x7f88ec0c9df0 comp rx=0 tx=0).stop 2023-07-21T18:57:10.398 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.376+0000 7f890ca9fb80 1 -- 172.21.15.39:0/2369188184 shutdown_connections 2023-07-21T18:57:10.399 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.376+0000 7f890ca9fb80 1 --2- 172.21.15.39:0/2369188184 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x56174d4f57e0 0x56174d4f7bf0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:10.399 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.376+0000 7f890ca9fb80 1 --2- 172.21.15.39:0/2369188184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56174d4e22d0 0x56174d4e5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:10.399 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.376+0000 7f890ca9fb80 1 --2- 172.21.15.39:0/2369188184 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56174d3386b0 0x56174d4e85a0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:10.399 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.376+0000 7f88e1ffb700 0 client.17340 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:57:10.399 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.376+0000 7f890ca9fb80 1 --2- 172.21.15.39:0/2369188184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56174d4dc580 0x56174d4e5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:10.399 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.376+0000 7f890ca9fb80 1 -- 172.21.15.39:0/2369188184 >> 172.21.15.39:0/2369188184 conn(0x56174d4d1be0 msgr2=0x56174d4d64e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:57:10.399 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.376+0000 7f890ca9fb80 1 -- 172.21.15.39:0/2369188184 shutdown_connections 2023-07-21T18:57:10.399 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.376+0000 7f890ca9fb80 1 -- 172.21.15.39:0/2369188184 wait complete. 2023-07-21T18:57:10.400 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.376+0000 7f890ca9fb80 20 client.17340 trim_cache size 0 max 16384 2023-07-21T18:57:10.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.403+0000 7f890ca9fb80 1 Processor -- start 2023-07-21T18:57:10.414 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.403+0000 7f890ca9fb80 1 -- start start 2023-07-21T18:57:10.414 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.403+0000 7f890ca9fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56174d3386b0 0x56174d4dc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:10.414 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.403+0000 7f890ca9fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56174d4dc580 0x56174d4e1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:10.414 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.403+0000 7f890ca9fb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56174d4e22d0 0x56174d4d6c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:10.414 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.403+0000 7f890ca9fb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x56174d4a2060 con 0x56174d4e22d0 2023-07-21T18:57:10.414 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.403+0000 7f890ca9fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56174d4d7d50 con 0x56174d4dc580 2023-07-21T18:57:10.414 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.403+0000 7f890ca9fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x56174d4d3d00 con 0x56174d3386b0 2023-07-21T18:57:10.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.403+0000 7f88fad9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56174d4e22d0 0x56174d4d6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:10.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.403+0000 7f88fad9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56174d4e22d0 0x56174d4d6c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33474/0 (socket says 172.21.15.39:33474) 2023-07-21T18:57:10.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.403+0000 7f88fad9d700 1 -- 172.21.15.39:0/3389488840 learned_addr learned my addr 172.21.15.39:0/3389488840 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:57:10.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.404+0000 7f88fad9d700 1 -- 172.21.15.39:0/3389488840 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56174d3386b0 msgr2=0x56174d4dc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:57:10.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.404+0000 7f88fad9d700 1 --2- 172.21.15.39:0/3389488840 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56174d3386b0 0x56174d4dc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:10.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.404+0000 7f88fad9d700 1 -- 172.21.15.39:0/3389488840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56174d4dc580 msgr2=0x56174d4e1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:57:10.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.404+0000 7f88fad9d700 1 --2- 172.21.15.39:0/3389488840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56174d4dc580 0x56174d4e1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:10.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.404+0000 7f88fad9d700 1 -- 172.21.15.39:0/3389488840 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56174d2821f0 con 0x56174d4e22d0 2023-07-21T18:57:10.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.404+0000 7f88fad9d700 1 --2- 172.21.15.39:0/3389488840 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56174d4e22d0 0x56174d4d6c60 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f88f40094a0 tx=0x7f88f40094d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8c1bb0a0aba0030 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:10.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.404+0000 7f88e97fa700 1 -- 172.21.15.39:0/3389488840 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f400e040 con 0x56174d4e22d0 2023-07-21T18:57:10.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.404+0000 7f88e97fa700 1 -- 172.21.15.39:0/3389488840 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f88f400be70 con 0x56174d4e22d0 2023-07-21T18:57:10.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.404+0000 7f88e97fa700 1 -- 172.21.15.39:0/3389488840 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f40054a0 con 0x56174d4e22d0 2023-07-21T18:57:10.418 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.405+0000 7f890ca9fb80 1 -- 172.21.15.39:0/3389488840 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56174d4e22d0 msgr2=0x56174d4d6c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:10.418 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.405+0000 7f890ca9fb80 1 --2- 172.21.15.39:0/3389488840 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56174d4e22d0 0x56174d4d6c60 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f88f40094a0 tx=0x7f88f40094d0 comp rx=0 tx=0).stop 2023-07-21T18:57:10.418 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.405+0000 7f890ca9fb80 1 -- 172.21.15.39:0/3389488840 shutdown_connections 2023-07-21T18:57:10.418 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.405+0000 7f890ca9fb80 1 --2- 172.21.15.39:0/3389488840 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56174d4e22d0 0x56174d4d6c60 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:10.418 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.405+0000 7f890ca9fb80 1 --2- 172.21.15.39:0/3389488840 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56174d3386b0 0x56174d4dc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:10.418 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.405+0000 7f890ca9fb80 1 --2- 172.21.15.39:0/3389488840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56174d4dc580 0x56174d4e1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:10.418 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.405+0000 7f890ca9fb80 1 -- 172.21.15.39:0/3389488840 >> 172.21.15.39:0/3389488840 conn(0x56174d4d1be0 msgr2=0x56174d4e2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:57:10.418 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.405+0000 7f890ca9fb80 1 -- 172.21.15.39:0/3389488840 shutdown_connections 2023-07-21T18:57:10.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.405+0000 7f890ca9fb80 1 -- 172.21.15.39:0/3389488840 wait complete. 2023-07-21T18:57:10.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.406+0000 7f890ca9fb80 1 -- start start 2023-07-21T18:57:10.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.406+0000 7f890ca9fb80 1 Processor -- start 2023-07-21T18:57:10.421 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.406+0000 7f890ca9fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56174d3386b0 0x56174d4deed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:10.421 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.406+0000 7f890ca9fb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56174d4dc580 0x56174d4dd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:10.421 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.406+0000 7f890ca9fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56174d4e22d0 0x56174d4ddcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:10.421 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.406+0000 7f890ca9fb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x56174d4df550 con 0x56174d4dc580 2023-07-21T18:57:10.421 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.406+0000 7f890ca9fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56174d4de210 con 0x56174d4e22d0 2023-07-21T18:57:10.421 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.406+0000 7f890ca9fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x56174d4de3d0 con 0x56174d3386b0 2023-07-21T18:57:10.422 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.406+0000 7f890ca9fb80 10 client.0 _ll_register_callbacks cb 0x7f88f4009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:57:10.422 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.407+0000 7f88fad9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56174d4e22d0 0x56174d4ddcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:10.422 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.407+0000 7f88fad9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56174d4e22d0 0x56174d4ddcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:46072/0 (socket says 172.21.15.39:46072) 2023-07-21T18:57:10.422 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.407+0000 7f88fad9d700 1 -- 172.21.15.39:0/1531179216 learned_addr learned my addr 172.21.15.39:0/1531179216 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:57:10.422 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.407+0000 7f88f9d9b700 1 --2- 172.21.15.39:0/1531179216 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56174d4dc580 0x56174d4dd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:10.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.407+0000 7f88fa59c700 1 --2- 172.21.15.39:0/1531179216 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56174d3386b0 0x56174d4deed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:10.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.407+0000 7f88f9d9b700 1 -- 172.21.15.39:0/1531179216 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56174d3386b0 msgr2=0x56174d4deed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:57:10.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.407+0000 7f88f9d9b700 1 --2- 172.21.15.39:0/1531179216 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56174d3386b0 0x56174d4deed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:10.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.407+0000 7f88f9d9b700 1 -- 172.21.15.39:0/1531179216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56174d4e22d0 msgr2=0x56174d4ddcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:57:10.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.407+0000 7f88f9d9b700 1 --2- 172.21.15.39:0/1531179216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56174d4e22d0 0x56174d4ddcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:10.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.407+0000 7f88f9d9b700 1 -- 172.21.15.39:0/1531179216 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88fc068890 con 0x56174d4dc580 2023-07-21T18:57:10.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.407+0000 7f88fad9d700 1 --2- 172.21.15.39:0/1531179216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56174d4e22d0 0x56174d4ddcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:57:10.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.407+0000 7f88e1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:57:10.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.407+0000 7f88f9d9b700 1 --2- 172.21.15.39:0/1531179216 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56174d4dc580 0x56174d4dd790 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x56174d4e8b80 tx=0x7f88f0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94b88c49ed24cf7c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:10.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.408+0000 7f88e1ffb700 1 -- 172.21.15.39:0/1531179216 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f0024040 con 0x56174d4dc580 2023-07-21T18:57:10.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.408+0000 7f88e1ffb700 1 -- 172.21.15.39:0/1531179216 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f88f0003880 con 0x56174d4dc580 2023-07-21T18:57:10.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.408+0000 7f890ca9fb80 1 -- 172.21.15.39:0/1531179216 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f88ec09d800 con 0x56174d4dc580 2023-07-21T18:57:10.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.408+0000 7f88e1ffb700 1 -- 172.21.15.39:0/1531179216 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f88f0004570 con 0x56174d4dc580 2023-07-21T18:57:10.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.408+0000 7f890ca9fb80 10 client.17346 fetch_fsmap learned FSMap version 87 2023-07-21T18:57:10.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.408+0000 7f890ca9fb80 1 -- 172.21.15.39:0/1531179216 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=4+}) v3 -- 0x56174d25a250 con 0x56174d4dc580 2023-07-21T18:57:10.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.408+0000 7f88e1ffb700 1 -- 172.21.15.39:0/1531179216 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f88ec09d800 con 0x56174d4dc580 2023-07-21T18:57:10.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.408+0000 7f88e1ffb700 1 -- 172.21.15.39:0/1531179216 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 87) v1 ==== 7530+0+0 (secure 0 0 0) 0x7f88f001eac0 con 0x56174d4dc580 2023-07-21T18:57:10.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.408+0000 7f890ca9fb80 10 client.17346 fetch_fsmap finished waiting for FSMap version 87 2023-07-21T18:57:10.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.408+0000 7f890ca9fb80 10 client.17346 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:57:10.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.408+0000 7f890ca9fb80 20 client.17346 populate_metadata read hostname 'smithi039' 2023-07-21T18:57:10.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.408+0000 7f890ca9fb80 1 --2- 172.21.15.39:0/1531179216 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x56174d54a9f0 0x56174d54ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:10.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.408+0000 7f890ca9fb80 4 client.17346 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:57:10.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.408+0000 7f890ca9fb80 1 -- 172.21.15.39:0/1531179216 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56174d3e8ff0 con 0x56174d54a9f0 2023-07-21T18:57:10.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.409+0000 7f88fa59c700 1 --2- 172.21.15.39:0/1531179216 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x56174d54a9f0 0x56174d54ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:10.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.409+0000 7f88e1ffb700 10 client.17346 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:57:10.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.409+0000 7f88fa59c700 1 --2- 172.21.15.39:0/1531179216 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x56174d54a9f0 0x56174d54ce00 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3520e8133245c9ef server_cookie=ac5ea006bf405360 in_seq=0 out_seq=0 2023-07-21T18:57:10.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:57:10 smithi193 ceph-mon[98204]: pgmap v485: 97 pgs: 97 active+clean; 935 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.2 MiB/s wr, 660 op/s 2023-07-21T18:57:10.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:57:10 smithi193 ceph-mon[98204]: osdmap e103: 12 total, 12 up, 12 in 2023-07-21T18:57:10.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:57:10 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:57:10.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:57:10 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:57:10.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:57:10 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:57:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.603+0000 7f88e1ffb700 1 -- 172.21.15.39:0/1531179216 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56174d3e8ff0 con 0x56174d54a9f0 2023-07-21T18:57:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.603+0000 7f88e1ffb700 10 client.17346 handle_command_reply: tid=0 2023-07-21T18:57:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.603+0000 7f890ca9fb80 1 client.17346 shutdown 2023-07-21T18:57:10.615 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.605+0000 7f890ca9fb80 1 -- 172.21.15.39:0/1531179216 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56174d4dc580 msgr2=0x56174d4dd790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:10.615 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.605+0000 7f890ca9fb80 1 --2- 172.21.15.39:0/1531179216 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56174d4dc580 0x56174d4dd790 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x56174d4e8b80 tx=0x7f88f0004750 comp rx=0 tx=0).stop 2023-07-21T18:57:10.616 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.605+0000 7f890ca9fb80 1 -- 172.21.15.39:0/1531179216 shutdown_connections 2023-07-21T18:57:10.616 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.605+0000 7f890ca9fb80 1 --2- 172.21.15.39:0/1531179216 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x56174d54a9f0 0x56174d54ce00 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:10.616 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.605+0000 7f890ca9fb80 1 --2- 172.21.15.39:0/1531179216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56174d4e22d0 0x56174d4ddcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:10.616 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.605+0000 7f890ca9fb80 1 --2- 172.21.15.39:0/1531179216 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56174d3386b0 0x56174d4deed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:10.616 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.605+0000 7f88e1ffb700 0 client.17346 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:57:10.616 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.605+0000 7f890ca9fb80 1 --2- 172.21.15.39:0/1531179216 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56174d4dc580 0x56174d4dd790 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:10.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.605+0000 7f890ca9fb80 1 -- 172.21.15.39:0/1531179216 >> 172.21.15.39:0/1531179216 conn(0x56174d4d1be0 msgr2=0x56174d4e10e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:57:10.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.605+0000 7f890ca9fb80 1 -- 172.21.15.39:0/1531179216 shutdown_connections 2023-07-21T18:57:10.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.605+0000 7f890ca9fb80 1 -- 172.21.15.39:0/1531179216 wait complete. 2023-07-21T18:57:10.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.605+0000 7f890ca9fb80 20 client.17346 trim_cache size 0 max 16384 2023-07-21T18:57:10.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:57:10.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:57:10.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:57:10.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:57:10.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.606+0000 7f8900919700 1 -- 172.21.15.39:0/2285285474 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f88e40a1340 msgr2=0x7f88e40a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:10.619 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.606+0000 7f8900919700 1 --2- 172.21.15.39:0/2285285474 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f88e40a1340 0x7f88e40a37f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f88fc199740 tx=0x7f88f0009260 comp rx=0 tx=0).stop 2023-07-21T18:57:10.619 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.606+0000 7f8900919700 1 -- 172.21.15.39:0/2285285474 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88fc0fea30 msgr2=0x7f88fc197f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:10.619 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.606+0000 7f8900919700 1 --2- 172.21.15.39:0/2285285474 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88fc0fea30 0x7f88fc197f20 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f88ec00b500 tx=0x7f88ec00b810 comp rx=0 tx=0).stop 2023-07-21T18:57:10.619 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.606+0000 7f8900919700 1 -- 172.21.15.39:0/2285285474 shutdown_connections 2023-07-21T18:57:10.619 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.606+0000 7f8900919700 1 --2- 172.21.15.39:0/2285285474 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f88e40a1340 0x7f88e40a37f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:10.619 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.606+0000 7f8900919700 1 --2- 172.21.15.39:0/2285285474 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88fc104a40 0x7f88fc19cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:10.620 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.606+0000 7f8900919700 1 --2- 172.21.15.39:0/2285285474 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88fc0fea30 0x7f88fc197f20 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:10.620 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.606+0000 7f8900919700 1 --2- 172.21.15.39:0/2285285474 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f88fc0ff3e0 0x7f88fc198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:10.620 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.606+0000 7f8900919700 1 -- 172.21.15.39:0/2285285474 >> 172.21.15.39:0/2285285474 conn(0x7f88fc0fa4f0 msgr2=0x7f88fc102580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:57:10.620 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.606+0000 7f8900919700 1 -- 172.21.15.39:0/2285285474 shutdown_connections 2023-07-21T18:57:10.620 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:10.606+0000 7f8900919700 1 -- 172.21.15.39:0/2285285474 wait complete. 2023-07-21T18:57:10.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:57:10 smithi155 ceph-mon[97934]: pgmap v485: 97 pgs: 97 active+clean; 935 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.2 MiB/s wr, 660 op/s 2023-07-21T18:57:10.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:57:10 smithi155 ceph-mon[97934]: osdmap e103: 12 total, 12 up, 12 in 2023-07-21T18:57:10.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:57:10 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:57:10.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:57:10 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:57:10.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:57:10 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:57:10.666 INFO:tasks.workunit.client.0.smithi039.stdout: 10 82006 34.40 MB/sec execute 23 sec latency 118.289 ms 2023-07-21T18:57:10.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:57:10 smithi039 ceph-mon[96489]: pgmap v485: 97 pgs: 97 active+clean; 935 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.2 MiB/s wr, 660 op/s 2023-07-21T18:57:10.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:57:10 smithi039 ceph-mon[96489]: osdmap e103: 12 total, 12 up, 12 in 2023-07-21T18:57:10.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:57:10 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:57:10.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:57:10 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:57:10.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:57:10 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:57:11.666 INFO:tasks.workunit.client.0.smithi039.stdout: 10 82919 35.08 MB/sec execute 24 sec latency 44.600 ms 2023-07-21T18:57:12.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:57:12 smithi193 ceph-mon[98204]: pgmap v487: 97 pgs: 2 active+clean+snaptrim, 95 active+clean; 963 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.8 MiB/s wr, 694 op/s 2023-07-21T18:57:12.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:57:12 smithi155 ceph-mon[97934]: pgmap v487: 97 pgs: 2 active+clean+snaptrim, 95 active+clean; 963 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.8 MiB/s wr, 694 op/s 2023-07-21T18:57:12.671 INFO:tasks.workunit.client.0.smithi039.stdout: 10 83621 35.53 MB/sec execute 25 sec latency 142.694 ms 2023-07-21T18:57:12.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:57:12 smithi039 ceph-mon[96489]: pgmap v487: 97 pgs: 2 active+clean+snaptrim, 95 active+clean; 963 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.8 MiB/s wr, 694 op/s 2023-07-21T18:57:13.672 INFO:tasks.workunit.client.0.smithi039.stdout: 10 83632 34.17 MB/sec execute 26 sec latency 1016.876 ms 2023-07-21T18:57:14.672 INFO:tasks.workunit.client.0.smithi039.stdout: 10 83772 33.20 MB/sec execute 27 sec latency 1073.915 ms 2023-07-21T18:57:14.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:57:14 smithi039 ceph-mon[96489]: pgmap v488: 97 pgs: 2 active+clean+snaptrim, 95 active+clean; 963 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.8 MiB/s wr, 694 op/s 2023-07-21T18:57:14.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:57:14 smithi193 ceph-mon[98204]: pgmap v488: 97 pgs: 2 active+clean+snaptrim, 95 active+clean; 963 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.8 MiB/s wr, 694 op/s 2023-07-21T18:57:14.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:57:14 smithi155 ceph-mon[97934]: pgmap v488: 97 pgs: 2 active+clean+snaptrim, 95 active+clean; 963 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.8 MiB/s wr, 694 op/s 2023-07-21T18:57:15.672 INFO:tasks.workunit.client.0.smithi039.stdout: 10 84130 32.47 MB/sec execute 28 sec latency 2064.254 ms 2023-07-21T18:57:16.672 INFO:tasks.workunit.client.0.smithi039.stdout: 10 84748 32.31 MB/sec execute 29 sec latency 3064.292 ms 2023-07-21T18:57:16.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:57:16 smithi039 ceph-mon[96489]: pgmap v489: 97 pgs: 2 active+clean+snaptrim, 95 active+clean; 963 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.8 MiB/s wr, 694 op/s 2023-07-21T18:57:16.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:57:16 smithi193 ceph-mon[98204]: pgmap v489: 97 pgs: 2 active+clean+snaptrim, 95 active+clean; 963 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.8 MiB/s wr, 694 op/s 2023-07-21T18:57:16.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:57:16 smithi155 ceph-mon[97934]: pgmap v489: 97 pgs: 2 active+clean+snaptrim, 95 active+clean; 963 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.8 MiB/s wr, 694 op/s 2023-07-21T18:57:17.672 INFO:tasks.workunit.client.0.smithi039.stdout: 10 85510 32.84 MB/sec execute 30 sec latency 4064.334 ms 2023-07-21T18:57:17.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:57:17 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:57:17.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:57:17 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:57:17.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:57:17 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:57:18.673 INFO:tasks.workunit.client.0.smithi039.stdout: 10 86179 32.72 MB/sec execute 31 sec latency 5064.381 ms 2023-07-21T18:57:18.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:57:18 smithi039 ceph-mon[96489]: pgmap v490: 97 pgs: 97 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 26 MiB/s wr, 1.13k op/s 2023-07-21T18:57:18.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:57:18 smithi193 ceph-mon[98204]: pgmap v490: 97 pgs: 97 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 26 MiB/s wr, 1.13k op/s 2023-07-21T18:57:18.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:57:18 smithi155 ceph-mon[97934]: pgmap v490: 97 pgs: 97 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 26 MiB/s wr, 1.13k op/s 2023-07-21T18:57:19.673 INFO:tasks.workunit.client.0.smithi039.stdout: 10 86899 33.14 MB/sec execute 32 sec latency 6064.429 ms 2023-07-21T18:57:19.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:57:19 smithi039 ceph-mon[96489]: pgmap v491: 97 pgs: 97 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 26 MiB/s wr, 1.13k op/s 2023-07-21T18:57:19.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:57:19 smithi193 ceph-mon[98204]: pgmap v491: 97 pgs: 97 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 26 MiB/s wr, 1.13k op/s 2023-07-21T18:57:19.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:57:19 smithi155 ceph-mon[97934]: pgmap v491: 97 pgs: 97 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 26 MiB/s wr, 1.13k op/s 2023-07-21T18:57:20.673 INFO:tasks.workunit.client.0.smithi039.stdout: 10 87577 33.19 MB/sec execute 33 sec latency 7064.469 ms 2023-07-21T18:57:21.673 INFO:tasks.workunit.client.0.smithi039.stdout: 10 88451 33.40 MB/sec execute 34 sec latency 8064.508 ms 2023-07-21T18:57:22.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:57:22 smithi193 ceph-mon[98204]: pgmap v492: 97 pgs: 97 active+clean; 1012 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 27 MiB/s wr, 1.35k op/s 2023-07-21T18:57:22.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:57:22 smithi193 ceph-mon[98204]: osdmap e104: 12 total, 12 up, 12 in 2023-07-21T18:57:22.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:57:22 smithi155 ceph-mon[97934]: pgmap v492: 97 pgs: 97 active+clean; 1012 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 27 MiB/s wr, 1.35k op/s 2023-07-21T18:57:22.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:57:22 smithi155 ceph-mon[97934]: osdmap e104: 12 total, 12 up, 12 in 2023-07-21T18:57:22.673 INFO:tasks.workunit.client.0.smithi039.stdout: 10 89132 33.84 MB/sec execute 35 sec latency 9064.547 ms 2023-07-21T18:57:22.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:57:22 smithi039 ceph-mon[96489]: pgmap v492: 97 pgs: 97 active+clean; 1012 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 27 MiB/s wr, 1.35k op/s 2023-07-21T18:57:22.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:57:22 smithi039 ceph-mon[96489]: osdmap e104: 12 total, 12 up, 12 in 2023-07-21T18:57:23.673 INFO:tasks.workunit.client.0.smithi039.stdout: 10 89591 33.49 MB/sec execute 36 sec latency 9875.601 ms 2023-07-21T18:57:23.770 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.758+0000 7f2cb088a700 1 -- 172.21.15.39:0/2733320646 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2cac0685b0 msgr2=0x7f2cac104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:23.770 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.758+0000 7f2cb088a700 1 --2- 172.21.15.39:0/2733320646 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2cac0685b0 0x7f2cac104770 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2ca00098f0 tx=0x7f2ca0009c00 comp rx=0 tx=0).stop 2023-07-21T18:57:23.770 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.758+0000 7f2cb088a700 1 -- 172.21.15.39:0/2733320646 shutdown_connections 2023-07-21T18:57:23.770 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.758+0000 7f2cb088a700 1 --2- 172.21.15.39:0/2733320646 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cac104e50 0x7f2cac10ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:23.770 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.758+0000 7f2cb088a700 1 --2- 172.21.15.39:0/2733320646 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2cac067c10 0x7f2cac067fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:23.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.758+0000 7f2cb088a700 1 --2- 172.21.15.39:0/2733320646 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2cac0685b0 0x7f2cac104770 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:23.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.758+0000 7f2cb088a700 1 -- 172.21.15.39:0/2733320646 >> 172.21.15.39:0/2733320646 conn(0x7f2cac074760 msgr2=0x7f2cac074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:57:23.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.758+0000 7f2cb088a700 1 -- 172.21.15.39:0/2733320646 shutdown_connections 2023-07-21T18:57:23.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.758+0000 7f2cb088a700 1 -- 172.21.15.39:0/2733320646 wait complete. 2023-07-21T18:57:23.772 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.759+0000 7f2cb088a700 1 Processor -- start 2023-07-21T18:57:23.773 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.759+0000 7f2cb088a700 1 -- start start 2023-07-21T18:57:23.773 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.759+0000 7f2cb088a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2cac067c10 0x7f2cac1999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:23.773 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.759+0000 7f2cb088a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cac0685b0 0x7f2cac199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:23.773 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.759+0000 7f2cb088a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2cac104e50 0x7f2cac193840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:23.773 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.759+0000 7f2cb088a700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cac193eb0 con 0x7f2cac067c10 2023-07-21T18:57:23.773 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.759+0000 7f2cb088a700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cac194020 con 0x7f2cac0685b0 2023-07-21T18:57:23.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.759+0000 7f2cb088a700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cac1941e0 con 0x7f2cac104e50 2023-07-21T18:57:23.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.760+0000 7f2caa59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2cac067c10 0x7f2cac1999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:23.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.760+0000 7f2caa59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2cac067c10 0x7f2cac1999e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33490/0 (socket says 172.21.15.39:33490) 2023-07-21T18:57:23.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.760+0000 7f2caa59c700 1 -- 172.21.15.39:0/201711219 learned_addr learned my addr 172.21.15.39:0/201711219 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:57:23.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.760+0000 7f2caad9d700 1 --2- 172.21.15.39:0/201711219 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2cac104e50 0x7f2cac193840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:23.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.760+0000 7f2ca9d9b700 1 --2- 172.21.15.39:0/201711219 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cac0685b0 0x7f2cac199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:23.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.760+0000 7f2caad9d700 1 -- 172.21.15.39:0/201711219 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cac0685b0 msgr2=0x7f2cac199f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:23.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.760+0000 7f2caad9d700 1 --2- 172.21.15.39:0/201711219 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cac0685b0 0x7f2cac199f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:23.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.760+0000 7f2caad9d700 1 -- 172.21.15.39:0/201711219 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2cac067c10 msgr2=0x7f2cac1999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:23.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.760+0000 7f2caad9d700 1 --2- 172.21.15.39:0/201711219 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2cac067c10 0x7f2cac1999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:23.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.760+0000 7f2caad9d700 1 -- 172.21.15.39:0/201711219 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ca0009520 con 0x7f2cac104e50 2023-07-21T18:57:23.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.760+0000 7f2caad9d700 1 --2- 172.21.15.39:0/201711219 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2cac104e50 0x7f2cac193840 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2ca40123c0 tx=0x7f2ca40126d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:23.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.760+0000 7f2c9b7fe700 1 -- 172.21.15.39:0/201711219 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca401c070 con 0x7f2cac104e50 2023-07-21T18:57:23.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.761+0000 7f2cb088a700 1 -- 172.21.15.39:0/201711219 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cac194460 con 0x7f2cac104e50 2023-07-21T18:57:23.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.761+0000 7f2c9b7fe700 1 -- 172.21.15.39:0/201711219 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ca4013040 con 0x7f2cac104e50 2023-07-21T18:57:23.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.761+0000 7f2c9b7fe700 1 -- 172.21.15.39:0/201711219 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca400bdd0 con 0x7f2cac104e50 2023-07-21T18:57:23.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.761+0000 7f2cb088a700 1 -- 172.21.15.39:0/201711219 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cac1a09d0 con 0x7f2cac104e50 2023-07-21T18:57:23.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.763+0000 7f2c9b7fe700 1 -- 172.21.15.39:0/201711219 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f2ca4004ad0 con 0x7f2cac104e50 2023-07-21T18:57:23.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.764+0000 7f2c9b7fe700 1 --2- 172.21.15.39:0/201711219 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f2c940a12c0 0x7f2c940a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:23.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.764+0000 7f2c9b7fe700 1 -- 172.21.15.39:0/201711219 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f2ca40d0c50 con 0x7f2cac104e50 2023-07-21T18:57:23.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.764+0000 7f2caa59c700 1 --2- 172.21.15.39:0/201711219 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f2c940a12c0 0x7f2c940a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:23.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.765+0000 7f2caa59c700 1 --2- 172.21.15.39:0/201711219 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f2c940a12c0 0x7f2c940a3770 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c005950 tx=0x7f2c9c009310 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:23.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.765+0000 7f2cbca10b80 1 Processor -- start 2023-07-21T18:57:23.778 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.766+0000 7f2cbca10b80 1 -- start start 2023-07-21T18:57:23.779 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.766+0000 7f2cbca10b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617bf1956b0 0x5617bf33ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:23.779 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.766+0000 7f2cbca10b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617bf33f2d0 0x5617bf339040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:23.779 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.766+0000 7f2cbca10b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5617bf339580 0x5617bf3399d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:23.779 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.766+0000 7f2cbca10b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617bf23ec50 con 0x5617bf339580 2023-07-21T18:57:23.779 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.766+0000 7f2cbca10b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617bf2919e0 con 0x5617bf1956b0 2023-07-21T18:57:23.779 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.766+0000 7f2cbca10b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617bf28ae70 con 0x5617bf33f2d0 2023-07-21T18:57:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.766+0000 7f2caad9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5617bf339580 0x5617bf3399d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.766+0000 7f2caa59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617bf1956b0 0x5617bf33ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.766+0000 7f2caa59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617bf1956b0 0x5617bf33ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:46094/0 (socket says 172.21.15.39:46094) 2023-07-21T18:57:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.766+0000 7f2caad9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5617bf339580 0x5617bf3399d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33500/0 (socket says 172.21.15.39:33500) 2023-07-21T18:57:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.766+0000 7f2caa59c700 1 -- 172.21.15.39:0/1111818544 learned_addr learned my addr 172.21.15.39:0/1111818544 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:57:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.766+0000 7f2ca9d9b700 1 --2- 172.21.15.39:0/1111818544 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617bf33f2d0 0x5617bf339040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.766+0000 7f2caa59c700 1 -- 172.21.15.39:0/1111818544 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617bf33f2d0 msgr2=0x5617bf339040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:57:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.766+0000 7f2caa59c700 1 --2- 172.21.15.39:0/1111818544 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617bf33f2d0 0x5617bf339040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.766+0000 7f2caa59c700 1 -- 172.21.15.39:0/1111818544 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5617bf339580 msgr2=0x5617bf3399d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:57:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.767+0000 7f2caa59c700 1 --2- 172.21.15.39:0/1111818544 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5617bf339580 0x5617bf3399d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.767+0000 7f2caa59c700 1 -- 172.21.15.39:0/1111818544 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617bf0b8d80 con 0x5617bf1956b0 2023-07-21T18:57:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.767+0000 7f2caa59c700 1 --2- 172.21.15.39:0/1111818544 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617bf1956b0 0x5617bf33ed90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c004260 tx=0x7f2c9c014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=550ad3c81f81eda4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.767+0000 7f2c98ff9700 1 -- 172.21.15.39:0/1111818544 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c003d10 con 0x5617bf1956b0 2023-07-21T18:57:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.767+0000 7f2c98ff9700 1 -- 172.21.15.39:0/1111818544 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c9c003e70 con 0x5617bf1956b0 2023-07-21T18:57:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.767+0000 7f2c98ff9700 1 -- 172.21.15.39:0/1111818544 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c01d610 con 0x5617bf1956b0 2023-07-21T18:57:23.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.768+0000 7f2cbca10b80 1 -- 172.21.15.39:0/1111818544 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617bf1956b0 msgr2=0x5617bf33ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:23.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.768+0000 7f2cbca10b80 1 --2- 172.21.15.39:0/1111818544 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617bf1956b0 0x5617bf33ed90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c004260 tx=0x7f2c9c014000 comp rx=0 tx=0).stop 2023-07-21T18:57:23.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.768+0000 7f2cbca10b80 1 -- 172.21.15.39:0/1111818544 shutdown_connections 2023-07-21T18:57:23.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.768+0000 7f2cbca10b80 1 --2- 172.21.15.39:0/1111818544 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617bf1956b0 0x5617bf33ed90 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:23.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.768+0000 7f2cbca10b80 1 --2- 172.21.15.39:0/1111818544 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5617bf339580 0x5617bf3399d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:23.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.768+0000 7f2cbca10b80 1 --2- 172.21.15.39:0/1111818544 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617bf33f2d0 0x5617bf339040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:23.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.768+0000 7f2cbca10b80 1 -- 172.21.15.39:0/1111818544 >> 172.21.15.39:0/1111818544 conn(0x5617bf32ebe0 msgr2=0x5617bf333f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:57:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.768+0000 7f2cbca10b80 1 -- 172.21.15.39:0/1111818544 shutdown_connections 2023-07-21T18:57:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.768+0000 7f2cbca10b80 1 -- 172.21.15.39:0/1111818544 wait complete. 2023-07-21T18:57:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.769+0000 7f2cbca10b80 1 -- start start 2023-07-21T18:57:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.769+0000 7f2cbca10b80 1 Processor -- start 2023-07-21T18:57:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.769+0000 7f2cbca10b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617bf1956b0 0x5617bf345640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.769+0000 7f2cbca10b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617bf339580 0x5617bf3428c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.769+0000 7f2cbca10b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5617bf33f2d0 0x5617bf343000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:23.786 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.769+0000 7f2cbca10b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617bf22aa60 con 0x5617bf33f2d0 2023-07-21T18:57:23.786 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.769+0000 7f2cbca10b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617bf22a760 con 0x5617bf1956b0 2023-07-21T18:57:23.786 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.769+0000 7f2cbca10b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617bf10e8c0 con 0x5617bf339580 2023-07-21T18:57:23.786 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.770+0000 7f2cbca10b80 10 client.0 _ll_register_callbacks cb 0x5617bf228700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:57:23.786 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.770+0000 7f2ca9d9b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617bf339580 0x5617bf3428c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:23.786 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.770+0000 7f2ca9d9b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617bf339580 0x5617bf3428c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:52056/0 (socket says 172.21.15.39:52056) 2023-07-21T18:57:23.786 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.770+0000 7f2ca9d9b700 1 -- 172.21.15.39:0/3924425215 learned_addr learned my addr 172.21.15.39:0/3924425215 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:57:23.786 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.770+0000 7f2caa59c700 1 --2- 172.21.15.39:0/3924425215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617bf1956b0 0x5617bf345640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:23.787 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.770+0000 7f2caad9d700 1 --2- 172.21.15.39:0/3924425215 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5617bf33f2d0 0x5617bf343000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:23.787 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.770+0000 7f2ca9d9b700 1 -- 172.21.15.39:0/3924425215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617bf1956b0 msgr2=0x5617bf345640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:57:23.787 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.770+0000 7f2ca9d9b700 1 --2- 172.21.15.39:0/3924425215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617bf1956b0 0x5617bf345640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:23.787 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.770+0000 7f2ca9d9b700 1 -- 172.21.15.39:0/3924425215 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5617bf33f2d0 msgr2=0x5617bf343000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:57:23.787 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.770+0000 7f2ca9d9b700 1 --2- 172.21.15.39:0/3924425215 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5617bf33f2d0 0x5617bf343000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:23.787 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.770+0000 7f2ca9d9b700 1 -- 172.21.15.39:0/3924425215 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c9c017070 con 0x5617bf339580 2023-07-21T18:57:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.770+0000 7f2ca9d9b700 1 --2- 172.21.15.39:0/3924425215 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617bf339580 0x5617bf3428c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0009810 tx=0x7f2ca0005920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88d5d329f8dfe00b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.771+0000 7f2c91ffb700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2023-07-21T18:57:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.771+0000 7f2c91ffb700 1 -- 172.21.15.39:0/3924425215 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca0027070 con 0x5617bf339580 2023-07-21T18:57:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.771+0000 7f2cbca10b80 1 -- 172.21.15.39:0/3924425215 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5617bf1e4490 con 0x5617bf339580 2023-07-21T18:57:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.771+0000 7f2c91ffb700 1 -- 172.21.15.39:0/3924425215 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f2ca000b770 con 0x5617bf339580 2023-07-21T18:57:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.771+0000 7f2c91ffb700 1 -- 172.21.15.39:0/3924425215 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ca001c050 con 0x5617bf339580 2023-07-21T18:57:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.771+0000 7f2cbca10b80 10 client.26809 fetch_fsmap learned FSMap version 88 2023-07-21T18:57:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.771+0000 7f2cbca10b80 1 -- 172.21.15.39:0/3924425215 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=4+}) v3 -- 0x5617bf0a1c00 con 0x5617bf339580 2023-07-21T18:57:23.789 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.771+0000 7f2c91ffb700 1 -- 172.21.15.39:0/3924425215 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5617bf1e4490 con 0x5617bf339580 2023-07-21T18:57:23.789 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.771+0000 7f2c91ffb700 1 -- 172.21.15.39:0/3924425215 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 88) v1 ==== 7526+0+0 (secure 0 0 0) 0x7f2ca002a020 con 0x5617bf339580 2023-07-21T18:57:23.789 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.772+0000 7f2cbca10b80 10 client.26809 fetch_fsmap finished waiting for FSMap version 88 2023-07-21T18:57:23.789 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.772+0000 7f2cbca10b80 10 client.26809 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:57:23.789 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.772+0000 7f2cbca10b80 20 client.26809 populate_metadata read hostname 'smithi039' 2023-07-21T18:57:23.789 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.772+0000 7f2cbca10b80 1 --2- 172.21.15.39:0/3924425215 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5617bf352cd0 0x5617bf3550e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:23.789 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.772+0000 7f2cbca10b80 4 client.26809 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:57:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.772+0000 7f2cbca10b80 1 -- 172.21.15.39:0/3924425215 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5617bf283e50 con 0x5617bf352cd0 2023-07-21T18:57:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.772+0000 7f2caa59c700 1 --2- 172.21.15.39:0/3924425215 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5617bf352cd0 0x5617bf3550e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.772+0000 7f2c91ffb700 10 client.26809 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:57:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.772+0000 7f2caa59c700 1 --2- 172.21.15.39:0/3924425215 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5617bf352cd0 0x5617bf3550e0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b9daeb60bfdfd5d server_cookie=744c7f2f4333d1c5 in_seq=0 out_seq=0 2023-07-21T18:57:23.791 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.775+0000 7f2c91ffb700 1 -- 172.21.15.39:0/3924425215 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5617bf283e50 con 0x5617bf352cd0 2023-07-21T18:57:23.791 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.775+0000 7f2c91ffb700 10 client.26809 handle_command_reply: tid=0 2023-07-21T18:57:23.791 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.775+0000 7f2cbca10b80 1 client.26809 shutdown 2023-07-21T18:57:23.791 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.776+0000 7f2cbca10b80 1 -- 172.21.15.39:0/3924425215 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617bf339580 msgr2=0x5617bf3428c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.776+0000 7f2cbca10b80 1 --2- 172.21.15.39:0/3924425215 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617bf339580 0x5617bf3428c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0009810 tx=0x7f2ca0005920 comp rx=0 tx=0).stop 2023-07-21T18:57:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.776+0000 7f2cbca10b80 1 -- 172.21.15.39:0/3924425215 shutdown_connections 2023-07-21T18:57:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.776+0000 7f2cbca10b80 1 --2- 172.21.15.39:0/3924425215 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5617bf352cd0 0x5617bf3550e0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.776+0000 7f2cbca10b80 1 --2- 172.21.15.39:0/3924425215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617bf1956b0 0x5617bf345640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.776+0000 7f2cbca10b80 1 --2- 172.21.15.39:0/3924425215 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5617bf33f2d0 0x5617bf343000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.776+0000 7f2cbca10b80 1 --2- 172.21.15.39:0/3924425215 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617bf339580 0x5617bf3428c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.776+0000 7f2cbca10b80 1 -- 172.21.15.39:0/3924425215 >> 172.21.15.39:0/3924425215 conn(0x5617bf32ebe0 msgr2=0x5617bf332c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:57:23.793 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.776+0000 7f2c91ffb700 0 client.26809 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:57:23.793 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.776+0000 7f2cbca10b80 1 -- 172.21.15.39:0/3924425215 shutdown_connections 2023-07-21T18:57:23.793 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.776+0000 7f2cbca10b80 1 -- 172.21.15.39:0/3924425215 wait complete. 2023-07-21T18:57:23.793 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.776+0000 7f2cbca10b80 20 client.26809 trim_cache size 0 max 16384 2023-07-21T18:57:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.795+0000 7f2cbca10b80 1 Processor -- start 2023-07-21T18:57:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.795+0000 7f2cbca10b80 1 -- start start 2023-07-21T18:57:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.796+0000 7f2cbca10b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5617bf1956b0 0x5617bf339040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.796+0000 7f2cbca10b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617bf339580 0x5617bf349c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.796+0000 7f2cbca10b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617bf33f2d0 0x5617bf333c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.796+0000 7f2cbca10b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617bf339fd0 con 0x5617bf1956b0 2023-07-21T18:57:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.796+0000 7f2cbca10b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617bf340e20 con 0x5617bf33f2d0 2023-07-21T18:57:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.796+0000 7f2cbca10b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617bf33d620 con 0x5617bf339580 2023-07-21T18:57:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.796+0000 7f2caa59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5617bf1956b0 0x5617bf339040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.796+0000 7f2ca9d9b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617bf339580 0x5617bf349c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.796+0000 7f2caa59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5617bf1956b0 0x5617bf339040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33512/0 (socket says 172.21.15.39:33512) 2023-07-21T18:57:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.796+0000 7f2caa59c700 1 -- 172.21.15.39:0/3123159190 learned_addr learned my addr 172.21.15.39:0/3123159190 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:57:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.796+0000 7f2caad9d700 1 --2- 172.21.15.39:0/3123159190 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617bf33f2d0 0x5617bf333c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:23.812 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.796+0000 7f2caa59c700 1 -- 172.21.15.39:0/3123159190 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617bf339580 msgr2=0x5617bf349c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:57:23.812 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.796+0000 7f2caa59c700 1 --2- 172.21.15.39:0/3123159190 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617bf339580 0x5617bf349c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:23.812 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.796+0000 7f2caa59c700 1 -- 172.21.15.39:0/3123159190 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617bf33f2d0 msgr2=0x5617bf333c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:57:23.812 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.796+0000 7f2caa59c700 1 --2- 172.21.15.39:0/3123159190 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617bf33f2d0 0x5617bf333c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:23.812 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.796+0000 7f2caa59c700 1 -- 172.21.15.39:0/3123159190 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617bf330420 con 0x5617bf1956b0 2023-07-21T18:57:23.812 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.796+0000 7f2ca9d9b700 1 --2- 172.21.15.39:0/3123159190 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617bf339580 0x5617bf349c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:57:23.813 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.796+0000 7f2caa59c700 1 --2- 172.21.15.39:0/3123159190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5617bf1956b0 0x5617bf339040 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c009fd0 tx=0x7f2c9c003a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ae57026aeb8cab4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:23.813 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.797+0000 7f2c997fa700 1 -- 172.21.15.39:0/3123159190 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c02c040 con 0x5617bf1956b0 2023-07-21T18:57:23.813 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.797+0000 7f2c997fa700 1 -- 172.21.15.39:0/3123159190 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c9c02d040 con 0x5617bf1956b0 2023-07-21T18:57:23.813 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.797+0000 7f2c997fa700 1 -- 172.21.15.39:0/3123159190 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c01e420 con 0x5617bf1956b0 2023-07-21T18:57:23.813 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.797+0000 7f2cbca10b80 1 -- 172.21.15.39:0/3123159190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5617bf1956b0 msgr2=0x5617bf339040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:23.813 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.797+0000 7f2cbca10b80 1 --2- 172.21.15.39:0/3123159190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5617bf1956b0 0x5617bf339040 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c009fd0 tx=0x7f2c9c003a10 comp rx=0 tx=0).stop 2023-07-21T18:57:23.813 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.798+0000 7f2cbca10b80 1 -- 172.21.15.39:0/3123159190 shutdown_connections 2023-07-21T18:57:23.814 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.798+0000 7f2cbca10b80 1 --2- 172.21.15.39:0/3123159190 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617bf33f2d0 0x5617bf333c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:23.814 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.798+0000 7f2cbca10b80 1 --2- 172.21.15.39:0/3123159190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5617bf1956b0 0x5617bf339040 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:23.814 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.798+0000 7f2cbca10b80 1 --2- 172.21.15.39:0/3123159190 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617bf339580 0x5617bf349c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:23.814 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.798+0000 7f2cbca10b80 1 -- 172.21.15.39:0/3123159190 >> 172.21.15.39:0/3123159190 conn(0x5617bf32ebe0 msgr2=0x5617bf34e410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:57:23.814 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.798+0000 7f2cbca10b80 1 -- 172.21.15.39:0/3123159190 shutdown_connections 2023-07-21T18:57:23.814 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.798+0000 7f2cbca10b80 1 -- 172.21.15.39:0/3123159190 wait complete. 2023-07-21T18:57:23.814 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.798+0000 7f2cbca10b80 1 -- start start 2023-07-21T18:57:23.814 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.798+0000 7f2cbca10b80 1 Processor -- start 2023-07-21T18:57:23.815 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.798+0000 7f2cbca10b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617bf1956b0 0x5617bf33ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:23.815 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.798+0000 7f2cbca10b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617bf339580 0x5617bf33bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:23.815 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.798+0000 7f2cbca10b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5617bf33f2d0 0x5617bf33a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:23.815 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.798+0000 7f2cbca10b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617bf34a1b0 con 0x5617bf33f2d0 2023-07-21T18:57:23.815 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.798+0000 7f2cbca10b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617bf33ada0 con 0x5617bf339580 2023-07-21T18:57:23.815 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.798+0000 7f2cbca10b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617bf33af60 con 0x5617bf1956b0 2023-07-21T18:57:23.815 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.798+0000 7f2cbca10b80 10 client.0 _ll_register_callbacks cb 0x7f2ca0009740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:57:23.816 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.798+0000 7f2caa59c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617bf1956b0 0x5617bf33ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:23.816 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.799+0000 7f2caa59c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617bf1956b0 0x5617bf33ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:52070/0 (socket says 172.21.15.39:52070) 2023-07-21T18:57:23.816 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.799+0000 7f2caa59c700 1 -- 172.21.15.39:0/1424253168 learned_addr learned my addr 172.21.15.39:0/1424253168 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:57:23.816 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.799+0000 7f2caad9d700 1 --2- 172.21.15.39:0/1424253168 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5617bf33f2d0 0x5617bf33a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:23.816 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.799+0000 7f2ca9d9b700 1 --2- 172.21.15.39:0/1424253168 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617bf339580 0x5617bf33bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:23.816 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.799+0000 7f2caa59c700 1 -- 172.21.15.39:0/1424253168 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617bf339580 msgr2=0x5617bf33bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:57:23.816 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.799+0000 7f2caa59c700 1 --2- 172.21.15.39:0/1424253168 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617bf339580 0x5617bf33bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:23.816 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.799+0000 7f2caa59c700 1 -- 172.21.15.39:0/1424253168 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5617bf33f2d0 msgr2=0x5617bf33a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:57:23.817 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.799+0000 7f2caa59c700 1 --2- 172.21.15.39:0/1424253168 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5617bf33f2d0 0x5617bf33a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:23.817 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.799+0000 7f2caa59c700 1 -- 172.21.15.39:0/1424253168 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5617bf0a1c00 con 0x5617bf1956b0 2023-07-21T18:57:23.817 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.799+0000 7f2caad9d700 1 --2- 172.21.15.39:0/1424253168 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5617bf33f2d0 0x5617bf33a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:57:23.817 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.799+0000 7f2caa59c700 1 --2- 172.21.15.39:0/1424253168 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617bf1956b0 0x5617bf33ba30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c003ac0 tx=0x7f2c9c003cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dad4e3f23232ae4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:23.817 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.799+0000 7f2c91ffb700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2023-07-21T18:57:23.817 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.799+0000 7f2c91ffb700 1 -- 172.21.15.39:0/1424253168 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c033bd0 con 0x5617bf1956b0 2023-07-21T18:57:23.817 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.799+0000 7f2cbca10b80 1 -- 172.21.15.39:0/1424253168 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2c9c003e70 con 0x5617bf1956b0 2023-07-21T18:57:23.818 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.799+0000 7f2c91ffb700 1 -- 172.21.15.39:0/1424253168 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f2c9c033d30 con 0x5617bf1956b0 2023-07-21T18:57:23.818 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.799+0000 7f2c91ffb700 1 -- 172.21.15.39:0/1424253168 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c9c033040 con 0x5617bf1956b0 2023-07-21T18:57:23.818 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.800+0000 7f2cbca10b80 10 client.26815 fetch_fsmap learned FSMap version 88 2023-07-21T18:57:23.818 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.800+0000 7f2cbca10b80 1 -- 172.21.15.39:0/1424253168 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=4+}) v3 -- 0x5617bf0b8d80 con 0x5617bf1956b0 2023-07-21T18:57:23.818 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.800+0000 7f2c91ffb700 1 -- 172.21.15.39:0/1424253168 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c9c003e70 con 0x5617bf1956b0 2023-07-21T18:57:23.818 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.800+0000 7f2c91ffb700 1 -- 172.21.15.39:0/1424253168 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 88) v1 ==== 7526+0+0 (secure 0 0 0) 0x7f2c9c024020 con 0x5617bf1956b0 2023-07-21T18:57:23.818 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.800+0000 7f2cbca10b80 10 client.26815 fetch_fsmap finished waiting for FSMap version 88 2023-07-21T18:57:23.818 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.800+0000 7f2cbca10b80 10 client.26815 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:57:23.819 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.800+0000 7f2cbca10b80 20 client.26815 populate_metadata read hostname 'smithi039' 2023-07-21T18:57:23.819 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.800+0000 7f2cbca10b80 1 --2- 172.21.15.39:0/1424253168 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5617bf3a75d0 0x5617bf3a99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:23.819 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.800+0000 7f2cbca10b80 4 client.26815 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:57:23.819 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.800+0000 7f2cbca10b80 1 -- 172.21.15.39:0/1424253168 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5617bf245ff0 con 0x5617bf3a75d0 2023-07-21T18:57:23.819 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.800+0000 7f2ca9d9b700 1 --2- 172.21.15.39:0/1424253168 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5617bf3a75d0 0x5617bf3a99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:23.819 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.801+0000 7f2c91ffb700 10 client.26815 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:57:23.819 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:23.801+0000 7f2ca9d9b700 1 --2- 172.21.15.39:0/1424253168 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5617bf3a75d0 0x5617bf3a99e0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c43b69b620480a62 server_cookie=7eebab2e48ba1160 in_seq=0 out_seq=0 2023-07-21T18:57:24.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:57:24 smithi193 ceph-mon[98204]: pgmap v494: 97 pgs: 97 active+clean; 1012 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 26 MiB/s wr, 1.21k op/s 2023-07-21T18:57:24.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:57:24 smithi155 ceph-mon[97934]: pgmap v494: 97 pgs: 97 active+clean; 1012 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 26 MiB/s wr, 1.21k op/s 2023-07-21T18:57:24.673 INFO:tasks.workunit.client.0.smithi039.stdout: 10 90316 33.59 MB/sec execute 37 sec latency 1031.780 ms 2023-07-21T18:57:24.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:57:24 smithi039 ceph-mon[96489]: pgmap v494: 97 pgs: 97 active+clean; 1012 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 26 MiB/s wr, 1.21k op/s 2023-07-21T18:57:25.673 INFO:tasks.workunit.client.0.smithi039.stdout: 10 91135 33.93 MB/sec execute 38 sec latency 2031.817 ms 2023-07-21T18:57:26.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:57:26 smithi193 ceph-mon[98204]: pgmap v495: 97 pgs: 97 active+clean; 1012 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 26 MiB/s wr, 1.21k op/s 2023-07-21T18:57:26.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:57:26 smithi155 ceph-mon[97934]: pgmap v495: 97 pgs: 97 active+clean; 1012 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 26 MiB/s wr, 1.21k op/s 2023-07-21T18:57:26.673 INFO:tasks.workunit.client.0.smithi039.stdout: 10 91643 33.83 MB/sec execute 39 sec latency 3031.847 ms 2023-07-21T18:57:26.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:57:26 smithi039 ceph-mon[96489]: pgmap v495: 97 pgs: 97 active+clean; 1012 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 26 MiB/s wr, 1.21k op/s 2023-07-21T18:57:27.673 INFO:tasks.workunit.client.0.smithi039.stdout: 10 92128 33.64 MB/sec execute 40 sec latency 4031.883 ms 2023-07-21T18:57:28.437 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:28.424+0000 7f2c91ffb700 1 -- 172.21.15.39:0/1424253168 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5617bf245ff0 con 0x5617bf3a75d0 2023-07-21T18:57:28.437 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:28.424+0000 7f2c91ffb700 10 client.26815 handle_command_reply: tid=0 2023-07-21T18:57:28.437 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:28.424+0000 7f2cbca10b80 1 client.26815 shutdown 2023-07-21T18:57:28.437 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:28.425+0000 7f2cbca10b80 1 -- 172.21.15.39:0/1424253168 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617bf1956b0 msgr2=0x5617bf33ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:28.437 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:28.425+0000 7f2cbca10b80 1 --2- 172.21.15.39:0/1424253168 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617bf1956b0 0x5617bf33ba30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c003ac0 tx=0x7f2c9c003cd0 comp rx=0 tx=0).stop 2023-07-21T18:57:28.438 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:28.425+0000 7f2cbca10b80 1 -- 172.21.15.39:0/1424253168 shutdown_connections 2023-07-21T18:57:28.438 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:28.425+0000 7f2cbca10b80 1 --2- 172.21.15.39:0/1424253168 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5617bf3a75d0 0x5617bf3a99e0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:28.438 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:28.425+0000 7f2cbca10b80 1 --2- 172.21.15.39:0/1424253168 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5617bf33f2d0 0x5617bf33a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:28.438 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:28.425+0000 7f2cbca10b80 1 --2- 172.21.15.39:0/1424253168 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617bf1956b0 0x5617bf33ba30 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:28.438 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:28.425+0000 7f2cbca10b80 1 --2- 172.21.15.39:0/1424253168 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617bf339580 0x5617bf33bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:28.438 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:28.425+0000 7f2cbca10b80 1 -- 172.21.15.39:0/1424253168 >> 172.21.15.39:0/1424253168 conn(0x5617bf32ebe0 msgr2=0x5617bf33ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:57:28.438 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:28.425+0000 7f2c91ffb700 0 client.26815 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:57:28.439 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:28.425+0000 7f2cbca10b80 1 -- 172.21.15.39:0/1424253168 shutdown_connections 2023-07-21T18:57:28.439 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:28.425+0000 7f2cbca10b80 1 -- 172.21.15.39:0/1424253168 wait complete. 2023-07-21T18:57:28.439 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:28.425+0000 7f2cbca10b80 20 client.26815 trim_cache size 0 max 16384 2023-07-21T18:57:28.439 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:57:28.440 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:57:28.440 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:57:28.440 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:57:28.440 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:28.426+0000 7f2cb088a700 1 -- 172.21.15.39:0/201711219 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f2c940a12c0 msgr2=0x7f2c940a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:28.441 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:28.426+0000 7f2cb088a700 1 --2- 172.21.15.39:0/201711219 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f2c940a12c0 0x7f2c940a3770 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c005950 tx=0x7f2c9c009310 comp rx=0 tx=0).stop 2023-07-21T18:57:28.441 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:28.426+0000 7f2cb088a700 1 -- 172.21.15.39:0/201711219 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2cac104e50 msgr2=0x7f2cac193840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:28.441 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:28.426+0000 7f2cb088a700 1 --2- 172.21.15.39:0/201711219 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2cac104e50 0x7f2cac193840 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2ca40123c0 tx=0x7f2ca40126d0 comp rx=0 tx=0).stop 2023-07-21T18:57:28.441 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:28.426+0000 7f2cb088a700 1 -- 172.21.15.39:0/201711219 shutdown_connections 2023-07-21T18:57:28.441 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:28.426+0000 7f2cb088a700 1 --2- 172.21.15.39:0/201711219 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2cac104e50 0x7f2cac193840 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:28.441 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:28.426+0000 7f2cb088a700 1 --2- 172.21.15.39:0/201711219 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2cac067c10 0x7f2cac1999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:28.442 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:28.426+0000 7f2cb088a700 1 --2- 172.21.15.39:0/201711219 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f2c940a12c0 0x7f2c940a3770 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:28.442 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:28.426+0000 7f2cb088a700 1 --2- 172.21.15.39:0/201711219 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cac0685b0 0x7f2cac199f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:28.442 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:28.426+0000 7f2cb088a700 1 -- 172.21.15.39:0/201711219 >> 172.21.15.39:0/201711219 conn(0x7f2cac074760 msgr2=0x7f2cac0fc620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:57:28.442 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:28.426+0000 7f2cb088a700 1 -- 172.21.15.39:0/201711219 shutdown_connections 2023-07-21T18:57:28.442 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:28.426+0000 7f2cb088a700 1 -- 172.21.15.39:0/201711219 wait complete. 2023-07-21T18:57:28.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:57:28 smithi193 ceph-mon[98204]: pgmap v496: 97 pgs: 97 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.4 MiB/s wr, 708 op/s 2023-07-21T18:57:28.630 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:57:28 smithi155 ceph-mon[97934]: pgmap v496: 97 pgs: 97 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.4 MiB/s wr, 708 op/s 2023-07-21T18:57:28.673 INFO:tasks.workunit.client.0.smithi039.stdout: 10 92759 33.53 MB/sec execute 41 sec latency 4310.658 ms 2023-07-21T18:57:28.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:57:28 smithi039 ceph-mon[96489]: pgmap v496: 97 pgs: 97 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.4 MiB/s wr, 708 op/s 2023-07-21T18:57:29.673 INFO:tasks.workunit.client.0.smithi039.stdout: 10 93660 33.77 MB/sec execute 42 sec latency 49.233 ms 2023-07-21T18:57:30.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:57:30 smithi193 ceph-mon[98204]: pgmap v497: 97 pgs: 97 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.4 MiB/s wr, 708 op/s 2023-07-21T18:57:30.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:57:30 smithi155 ceph-mon[97934]: pgmap v497: 97 pgs: 97 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.4 MiB/s wr, 708 op/s 2023-07-21T18:57:30.673 INFO:tasks.workunit.client.0.smithi039.stdout: 10 94468 34.23 MB/sec execute 43 sec latency 50.155 ms 2023-07-21T18:57:30.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:57:30 smithi039 ceph-mon[96489]: pgmap v497: 97 pgs: 97 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.4 MiB/s wr, 708 op/s 2023-07-21T18:57:31.673 INFO:tasks.workunit.client.0.smithi039.stdout: 10 95096 34.21 MB/sec execute 44 sec latency 57.867 ms 2023-07-21T18:57:32.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:57:32 smithi193 ceph-mon[98204]: pgmap v498: 97 pgs: 97 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.6 MiB/s wr, 626 op/s 2023-07-21T18:57:32.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:57:32 smithi155 ceph-mon[97934]: pgmap v498: 97 pgs: 97 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.6 MiB/s wr, 626 op/s 2023-07-21T18:57:32.674 INFO:tasks.workunit.client.0.smithi039.stdout: 10 95759 34.14 MB/sec execute 45 sec latency 66.730 ms 2023-07-21T18:57:32.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:57:32 smithi039 ceph-mon[96489]: pgmap v498: 97 pgs: 97 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.6 MiB/s wr, 626 op/s 2023-07-21T18:57:33.673 INFO:tasks.workunit.client.0.smithi039.stdout: 10 96578 34.28 MB/sec execute 46 sec latency 73.709 ms 2023-07-21T18:57:34.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:57:34 smithi193 ceph-mon[98204]: pgmap v499: 97 pgs: 97 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 6.7 MiB/s wr, 555 op/s 2023-07-21T18:57:34.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:57:34 smithi155 ceph-mon[97934]: pgmap v499: 97 pgs: 97 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 6.7 MiB/s wr, 555 op/s 2023-07-21T18:57:34.673 INFO:tasks.workunit.client.0.smithi039.stdout: 10 97423 34.67 MB/sec execute 47 sec latency 58.582 ms 2023-07-21T18:57:34.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:57:34 smithi039 ceph-mon[96489]: pgmap v499: 97 pgs: 97 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 6.7 MiB/s wr, 555 op/s 2023-07-21T18:57:35.673 INFO:tasks.workunit.client.0.smithi039.stdout: 10 98056 34.76 MB/sec execute 48 sec latency 188.642 ms 2023-07-21T18:57:36.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:57:36 smithi193 ceph-mon[98204]: pgmap v500: 97 pgs: 97 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 6.3 MiB/s wr, 521 op/s 2023-07-21T18:57:36.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:57:36 smithi155 ceph-mon[97934]: pgmap v500: 97 pgs: 97 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 6.3 MiB/s wr, 521 op/s 2023-07-21T18:57:36.673 INFO:tasks.workunit.client.0.smithi039.stdout: 10 98675 34.59 MB/sec execute 49 sec latency 49.348 ms 2023-07-21T18:57:36.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:57:36 smithi039 ceph-mon[96489]: pgmap v500: 97 pgs: 97 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 6.3 MiB/s wr, 521 op/s 2023-07-21T18:57:37.673 INFO:tasks.workunit.client.0.smithi039.stdout: 10 99353 34.57 MB/sec execute 50 sec latency 58.157 ms 2023-07-21T18:57:38.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:57:38 smithi193 ceph-mon[98204]: pgmap v501: 97 pgs: 97 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 9.1 MiB/s wr, 772 op/s 2023-07-21T18:57:38.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:57:38 smithi155 ceph-mon[97934]: pgmap v501: 97 pgs: 97 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 9.1 MiB/s wr, 772 op/s 2023-07-21T18:57:38.673 INFO:tasks.workunit.client.0.smithi039.stdout: 10 100170 34.80 MB/sec execute 51 sec latency 47.675 ms 2023-07-21T18:57:38.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:57:38 smithi039 ceph-mon[96489]: pgmap v501: 97 pgs: 97 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 9.1 MiB/s wr, 772 op/s 2023-07-21T18:57:39.673 INFO:tasks.workunit.client.0.smithi039.stdout: 10 100992 34.94 MB/sec execute 52 sec latency 68.933 ms 2023-07-21T18:57:40.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:57:40 smithi193 ceph-mon[98204]: pgmap v502: 97 pgs: 97 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.8 MiB/s wr, 497 op/s 2023-07-21T18:57:40.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:57:40 smithi155 ceph-mon[97934]: pgmap v502: 97 pgs: 97 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.8 MiB/s wr, 497 op/s 2023-07-21T18:57:40.673 INFO:tasks.workunit.client.0.smithi039.stdout: 10 101596 34.95 MB/sec execute 53 sec latency 63.446 ms 2023-07-21T18:57:40.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:57:40 smithi039 ceph-mon[96489]: pgmap v502: 97 pgs: 97 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.8 MiB/s wr, 497 op/s 2023-07-21T18:57:41.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.575+0000 7f7bd30ae700 1 -- 172.21.15.39:0/383650767 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7bcc104a30 msgr2=0x7f7bcc104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:41.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.575+0000 7f7bd30ae700 1 --2- 172.21.15.39:0/383650767 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7bcc104a30 0x7f7bcc104e00 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0009990 tx=0x7f7bc0009ca0 comp rx=0 tx=0).stop 2023-07-21T18:57:41.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.575+0000 7f7bd30ae700 1 -- 172.21.15.39:0/383650767 shutdown_connections 2023-07-21T18:57:41.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.575+0000 7f7bd30ae700 1 --2- 172.21.15.39:0/383650767 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7bcc0ff3e0 0x7f7bcc10b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:41.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.575+0000 7f7bd30ae700 1 --2- 172.21.15.39:0/383650767 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7bcc104a30 0x7f7bcc104e00 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:41.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.575+0000 7f7bd30ae700 1 --2- 172.21.15.39:0/383650767 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7bcc0fea30 0x7f7bcc0feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:41.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.575+0000 7f7bd30ae700 1 -- 172.21.15.39:0/383650767 >> 172.21.15.39:0/383650767 conn(0x7f7bcc074760 msgr2=0x7f7bcc074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:57:41.588 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.575+0000 7f7bd30ae700 1 -- 172.21.15.39:0/383650767 shutdown_connections 2023-07-21T18:57:41.588 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.575+0000 7f7bd30ae700 1 -- 172.21.15.39:0/383650767 wait complete. 2023-07-21T18:57:41.590 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.576+0000 7f7bd30ae700 1 Processor -- start 2023-07-21T18:57:41.591 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.576+0000 7f7bd30ae700 1 -- start start 2023-07-21T18:57:41.591 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.576+0000 7f7bd30ae700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7bcc0fea30 0x7f7bcc1985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:41.591 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.576+0000 7f7bd30ae700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7bcc0ff3e0 0x7f7bcc198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:41.591 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.576+0000 7f7bd30ae700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7bcc104a30 0x7f7bcc193920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:41.591 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.576+0000 7f7bd30ae700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bcc193e60 con 0x7f7bcc0fea30 2023-07-21T18:57:41.591 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.576+0000 7f7bd30ae700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bcc193fd0 con 0x7f7bcc104a30 2023-07-21T18:57:41.591 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.576+0000 7f7bd30ae700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bcc194190 con 0x7f7bcc0ff3e0 2023-07-21T18:57:41.592 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.576+0000 7f7bcbfff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7bcc0ff3e0 0x7f7bcc198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:41.592 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.576+0000 7f7bd0e4a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7bcc0fea30 0x7f7bcc1985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:41.592 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.577+0000 7f7bcbfff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7bcc0ff3e0 0x7f7bcc198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:52084/0 (socket says 172.21.15.39:52084) 2023-07-21T18:57:41.592 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.577+0000 7f7bcbfff700 1 -- 172.21.15.39:0/3931896770 learned_addr learned my addr 172.21.15.39:0/3931896770 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:57:41.592 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.577+0000 7f7bd0e4a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7bcc0fea30 0x7f7bcc1985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33530/0 (socket says 172.21.15.39:33530) 2023-07-21T18:57:41.592 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.577+0000 7f7bd164b700 1 --2- 172.21.15.39:0/3931896770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7bcc104a30 0x7f7bcc193920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:41.592 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.577+0000 7f7bd0e4a700 1 -- 172.21.15.39:0/3931896770 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7bcc0ff3e0 msgr2=0x7f7bcc198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:41.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.577+0000 7f7bd0e4a700 1 --2- 172.21.15.39:0/3931896770 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7bcc0ff3e0 0x7f7bcc198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:41.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.577+0000 7f7bd0e4a700 1 -- 172.21.15.39:0/3931896770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7bcc104a30 msgr2=0x7f7bcc193920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:41.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.577+0000 7f7bd0e4a700 1 --2- 172.21.15.39:0/3931896770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7bcc104a30 0x7f7bcc193920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:41.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.577+0000 7f7bd0e4a700 1 -- 172.21.15.39:0/3931896770 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb800d930 con 0x7f7bcc0fea30 2023-07-21T18:57:41.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.577+0000 7f7bd0e4a700 1 --2- 172.21.15.39:0/3931896770 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7bcc0fea30 0x7f7bcc1985a0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0000c00 tx=0x7f7bc0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:41.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.578+0000 7f7bc9ffb700 1 -- 172.21.15.39:0/3931896770 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc001d070 con 0x7f7bcc0fea30 2023-07-21T18:57:41.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.578+0000 7f7bc9ffb700 1 -- 172.21.15.39:0/3931896770 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bc0005030 con 0x7f7bcc0fea30 2023-07-21T18:57:41.594 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.578+0000 7f7bd30ae700 1 -- 172.21.15.39:0/3931896770 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bc00095f0 con 0x7f7bcc0fea30 2023-07-21T18:57:41.594 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.578+0000 7f7bc9ffb700 1 -- 172.21.15.39:0/3931896770 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc0004ab0 con 0x7f7bcc0fea30 2023-07-21T18:57:41.594 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.578+0000 7f7bd30ae700 1 -- 172.21.15.39:0/3931896770 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bcc1945f0 con 0x7f7bcc0fea30 2023-07-21T18:57:41.595 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.581+0000 7f7bc9ffb700 1 -- 172.21.15.39:0/3931896770 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f7bc0004c90 con 0x7f7bcc0fea30 2023-07-21T18:57:41.596 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.582+0000 7f7bc9ffb700 1 --2- 172.21.15.39:0/3931896770 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f7bbc0e52d0 0x7f7bbc0e7780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:41.596 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.582+0000 7f7bcbfff700 1 --2- 172.21.15.39:0/3931896770 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f7bbc0e52d0 0x7f7bbc0e7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:41.596 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.582+0000 7f7bc9ffb700 1 -- 172.21.15.39:0/3931896770 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f7bc00cddd0 con 0x7f7bcc0fea30 2023-07-21T18:57:41.596 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.582+0000 7f7bdf234b80 1 Processor -- start 2023-07-21T18:57:41.596 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.583+0000 7f7bdf234b80 1 -- start start 2023-07-21T18:57:41.596 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.583+0000 7f7bdf234b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b2574a6b0 0x562b258f3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:41.597 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.583+0000 7f7bdf234b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b258f42d0 0x562b258ee040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:41.597 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.583+0000 7f7bdf234b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b258ee580 0x562b258ee9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:41.597 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.583+0000 7f7bcbfff700 1 --2- 172.21.15.39:0/3931896770 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f7bbc0e52d0 0x7f7bbc0e7780 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc195210 tx=0x7f7bb800f040 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:41.597 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.583+0000 7f7bdf234b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b257f3c50 con 0x562b2574a6b0 2023-07-21T18:57:41.597 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.583+0000 7f7bdf234b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b258469e0 con 0x562b258f42d0 2023-07-21T18:57:41.597 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.583+0000 7f7bdf234b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b2583fe70 con 0x562b258ee580 2023-07-21T18:57:41.597 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.583+0000 7f7bd0e4a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b2574a6b0 0x562b258f3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:41.598 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.583+0000 7f7bd0e4a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b2574a6b0 0x562b258f3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33538/0 (socket says 172.21.15.39:33538) 2023-07-21T18:57:41.598 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.583+0000 7f7bd0e4a700 1 -- 172.21.15.39:0/3919290489 learned_addr learned my addr 172.21.15.39:0/3919290489 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:57:41.598 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.583+0000 7f7bd164b700 1 --2- 172.21.15.39:0/3919290489 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b258ee580 0x562b258ee9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:41.598 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.583+0000 7f7bcbfff700 1 --2- 172.21.15.39:0/3919290489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b258f42d0 0x562b258ee040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:41.598 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.583+0000 7f7bd0e4a700 1 -- 172.21.15.39:0/3919290489 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b258ee580 msgr2=0x562b258ee9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:57:41.598 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.583+0000 7f7bd0e4a700 1 --2- 172.21.15.39:0/3919290489 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b258ee580 0x562b258ee9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:41.598 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.583+0000 7f7bd0e4a700 1 -- 172.21.15.39:0/3919290489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b258f42d0 msgr2=0x562b258ee040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:57:41.598 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.583+0000 7f7bd0e4a700 1 --2- 172.21.15.39:0/3919290489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b258f42d0 0x562b258ee040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.583+0000 7f7bd0e4a700 1 -- 172.21.15.39:0/3919290489 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b2566dd80 con 0x562b2574a6b0 2023-07-21T18:57:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.584+0000 7f7bd0e4a700 1 --2- 172.21.15.39:0/3919290489 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b2574a6b0 0x562b258f3d90 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0097430 tx=0x7f7bc009a910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ca05e6f76af0bce server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.584+0000 7f7bb6ffd700 1 -- 172.21.15.39:0/3919290489 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc00a32f0 con 0x562b2574a6b0 2023-07-21T18:57:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.584+0000 7f7bb6ffd700 1 -- 172.21.15.39:0/3919290489 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bc00992d0 con 0x562b2574a6b0 2023-07-21T18:57:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.584+0000 7f7bb6ffd700 1 -- 172.21.15.39:0/3919290489 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc00a2e60 con 0x562b2574a6b0 2023-07-21T18:57:41.602 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.585+0000 7f7bdf234b80 1 -- 172.21.15.39:0/3919290489 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b2574a6b0 msgr2=0x562b258f3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:41.602 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.585+0000 7f7bdf234b80 1 --2- 172.21.15.39:0/3919290489 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b2574a6b0 0x562b258f3d90 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0097430 tx=0x7f7bc009a910 comp rx=0 tx=0).stop 2023-07-21T18:57:41.602 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.585+0000 7f7bdf234b80 1 -- 172.21.15.39:0/3919290489 shutdown_connections 2023-07-21T18:57:41.602 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.585+0000 7f7bdf234b80 1 --2- 172.21.15.39:0/3919290489 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b258ee580 0x562b258ee9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:41.603 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.585+0000 7f7bdf234b80 1 --2- 172.21.15.39:0/3919290489 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b2574a6b0 0x562b258f3d90 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:41.603 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.585+0000 7f7bdf234b80 1 --2- 172.21.15.39:0/3919290489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b258f42d0 0x562b258ee040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:41.603 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.585+0000 7f7bdf234b80 1 -- 172.21.15.39:0/3919290489 >> 172.21.15.39:0/3919290489 conn(0x562b258e3be0 msgr2=0x562b258e8f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:57:41.603 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.585+0000 7f7bdf234b80 1 -- 172.21.15.39:0/3919290489 shutdown_connections 2023-07-21T18:57:41.603 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.585+0000 7f7bdf234b80 1 -- 172.21.15.39:0/3919290489 wait complete. 2023-07-21T18:57:41.603 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.585+0000 7f7bdf234b80 1 -- start start 2023-07-21T18:57:41.603 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.586+0000 7f7bdf234b80 1 Processor -- start 2023-07-21T18:57:41.603 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.586+0000 7f7bdf234b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b2574a6b0 0x562b258fd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:41.604 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.586+0000 7f7bdf234b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b258ee580 0x562b258fd950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:41.604 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.586+0000 7f7bdf234b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b258f42d0 0x562b258f7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:41.604 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.586+0000 7f7bdf234b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b257dfa60 con 0x562b258ee580 2023-07-21T18:57:41.604 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.586+0000 7f7bdf234b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b257df760 con 0x562b258f42d0 2023-07-21T18:57:41.604 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.586+0000 7f7bdf234b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b256c38c0 con 0x562b2574a6b0 2023-07-21T18:57:41.604 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.586+0000 7f7bdf234b80 10 client.0 _ll_register_callbacks cb 0x562b257dd700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:57:41.604 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.586+0000 7f7bcbfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b258ee580 0x562b258fd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:41.605 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.586+0000 7f7bcbfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b258ee580 0x562b258fd950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33546/0 (socket says 172.21.15.39:33546) 2023-07-21T18:57:41.605 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.586+0000 7f7bcbfff700 1 -- 172.21.15.39:0/2681617332 learned_addr learned my addr 172.21.15.39:0/2681617332 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:57:41.605 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.586+0000 7f7bd0e4a700 1 --2- 172.21.15.39:0/2681617332 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b2574a6b0 0x562b258fd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:41.605 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.586+0000 7f7bd164b700 1 --2- 172.21.15.39:0/2681617332 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b258f42d0 0x562b258f7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:41.605 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.587+0000 7f7bd0e4a700 1 -- 172.21.15.39:0/2681617332 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b258f42d0 msgr2=0x562b258f7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:57:41.605 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.587+0000 7f7bd0e4a700 1 --2- 172.21.15.39:0/2681617332 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b258f42d0 0x562b258f7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:41.605 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.587+0000 7f7bd0e4a700 1 -- 172.21.15.39:0/2681617332 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b258ee580 msgr2=0x562b258fd950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:57:41.605 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.587+0000 7f7bd0e4a700 1 --2- 172.21.15.39:0/2681617332 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b258ee580 0x562b258fd950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:41.606 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.587+0000 7f7bd0e4a700 1 -- 172.21.15.39:0/2681617332 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bcc1945f0 con 0x562b2574a6b0 2023-07-21T18:57:41.606 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.587+0000 7f7bd0e4a700 1 --2- 172.21.15.39:0/2681617332 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b2574a6b0 0x562b258fd410 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x562b258ef020 tx=0x7f7bc0009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc1e0d524183df80 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:41.606 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.587+0000 7f7babfff700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2023-07-21T18:57:41.606 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.587+0000 7f7babfff700 1 -- 172.21.15.39:0/2681617332 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc00a32f0 con 0x562b2574a6b0 2023-07-21T18:57:41.606 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.587+0000 7f7babfff700 1 -- 172.21.15.39:0/2681617332 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f7bc00b91b0 con 0x562b2574a6b0 2023-07-21T18:57:41.606 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.587+0000 7f7bdf234b80 1 -- 172.21.15.39:0/2681617332 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562b25799490 con 0x562b2574a6b0 2023-07-21T18:57:41.606 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.587+0000 7f7babfff700 1 -- 172.21.15.39:0/2681617332 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7bc00a2e60 con 0x562b2574a6b0 2023-07-21T18:57:41.606 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.587+0000 7f7bdf234b80 10 client.26827 fetch_fsmap learned FSMap version 88 2023-07-21T18:57:41.607 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.587+0000 7f7bdf234b80 1 -- 172.21.15.39:0/2681617332 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=4+}) v3 -- 0x562b25656c00 con 0x562b2574a6b0 2023-07-21T18:57:41.607 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.587+0000 7f7babfff700 1 -- 172.21.15.39:0/2681617332 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bc00cb020 con 0x562b2574a6b0 2023-07-21T18:57:41.607 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.588+0000 7f7babfff700 1 -- 172.21.15.39:0/2681617332 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 88) v1 ==== 7526+0+0 (secure 0 0 0) 0x7f7bc0096d20 con 0x562b2574a6b0 2023-07-21T18:57:41.607 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.588+0000 7f7bdf234b80 10 client.26827 fetch_fsmap finished waiting for FSMap version 88 2023-07-21T18:57:41.607 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.588+0000 7f7bdf234b80 10 client.26827 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:57:41.607 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.588+0000 7f7bdf234b80 20 client.26827 populate_metadata read hostname 'smithi039' 2023-07-21T18:57:41.607 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.588+0000 7f7bdf234b80 1 --2- 172.21.15.39:0/2681617332 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x562b259074e0 0x562b259098f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:41.608 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.588+0000 7f7bdf234b80 4 client.26827 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:57:41.608 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.588+0000 7f7bdf234b80 1 -- 172.21.15.39:0/2681617332 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562b25838e50 con 0x562b259074e0 2023-07-21T18:57:41.608 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.588+0000 7f7bcbfff700 1 --2- 172.21.15.39:0/2681617332 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x562b259074e0 0x562b259098f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:41.608 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.588+0000 7f7babfff700 10 client.26827 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:57:41.608 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.588+0000 7f7bcbfff700 1 --2- 172.21.15.39:0/2681617332 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x562b259074e0 0x562b259098f0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=512095261fbee8cc server_cookie=8cb6e93cc63bbcde in_seq=0 out_seq=0 2023-07-21T18:57:41.608 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.590+0000 7f7babfff700 1 -- 172.21.15.39:0/2681617332 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562b25838e50 con 0x562b259074e0 2023-07-21T18:57:41.608 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.590+0000 7f7babfff700 10 client.26827 handle_command_reply: tid=0 2023-07-21T18:57:41.608 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.590+0000 7f7bdf234b80 1 client.26827 shutdown 2023-07-21T18:57:41.609 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.590+0000 7f7bdf234b80 1 -- 172.21.15.39:0/2681617332 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b2574a6b0 msgr2=0x562b258fd410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:41.609 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.590+0000 7f7bdf234b80 1 --2- 172.21.15.39:0/2681617332 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b2574a6b0 0x562b258fd410 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x562b258ef020 tx=0x7f7bc0009310 comp rx=0 tx=0).stop 2023-07-21T18:57:41.609 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.590+0000 7f7bdf234b80 1 -- 172.21.15.39:0/2681617332 shutdown_connections 2023-07-21T18:57:41.609 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.590+0000 7f7bdf234b80 1 --2- 172.21.15.39:0/2681617332 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x562b259074e0 0x562b259098f0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:41.609 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.590+0000 7f7bdf234b80 1 --2- 172.21.15.39:0/2681617332 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b258f42d0 0x562b258f7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:41.609 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.590+0000 7f7babfff700 0 client.26827 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:57:41.609 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.590+0000 7f7bdf234b80 1 --2- 172.21.15.39:0/2681617332 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b2574a6b0 0x562b258fd410 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:41.609 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.590+0000 7f7bdf234b80 1 --2- 172.21.15.39:0/2681617332 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b258ee580 0x562b258fd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:41.610 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.590+0000 7f7bdf234b80 1 -- 172.21.15.39:0/2681617332 >> 172.21.15.39:0/2681617332 conn(0x562b258e3be0 msgr2=0x562b258e84e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:57:41.610 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.591+0000 7f7bdf234b80 1 -- 172.21.15.39:0/2681617332 shutdown_connections 2023-07-21T18:57:41.610 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.591+0000 7f7bdf234b80 1 -- 172.21.15.39:0/2681617332 wait complete. 2023-07-21T18:57:41.610 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.591+0000 7f7bdf234b80 20 client.26827 trim_cache size 0 max 16384 2023-07-21T18:57:41.624 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.613+0000 7f7bdf234b80 1 Processor -- start 2023-07-21T18:57:41.625 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.613+0000 7f7bdf234b80 1 -- start start 2023-07-21T18:57:41.625 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.613+0000 7f7bdf234b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b258e3be0 0x562b258ee040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:41.625 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.613+0000 7f7bdf234b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b2574a6b0 0x562b258f5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:41.626 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.613+0000 7f7bdf234b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b258ee580 0x562b258fcfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:41.626 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.613+0000 7f7bdf234b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b258f8660 con 0x562b258ee580 2023-07-21T18:57:41.626 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.613+0000 7f7bdf234b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b258e9d50 con 0x562b258e3be0 2023-07-21T18:57:41.626 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.613+0000 7f7bdf234b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b258f8c40 con 0x562b2574a6b0 2023-07-21T18:57:41.626 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.614+0000 7f7bd164b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b258ee580 0x562b258fcfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:41.626 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.614+0000 7f7bd0e4a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b258e3be0 0x562b258ee040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:41.626 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.614+0000 7f7bd164b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b258ee580 0x562b258fcfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33556/0 (socket says 172.21.15.39:33556) 2023-07-21T18:57:41.626 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.614+0000 7f7bd0e4a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b258e3be0 0x562b258ee040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:46148/0 (socket says 172.21.15.39:46148) 2023-07-21T18:57:41.627 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.614+0000 7f7bd164b700 1 -- 172.21.15.39:0/1799563049 learned_addr learned my addr 172.21.15.39:0/1799563049 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:57:41.627 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.614+0000 7f7bcbfff700 1 --2- 172.21.15.39:0/1799563049 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b2574a6b0 0x562b258f5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:41.627 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.614+0000 7f7bd164b700 1 -- 172.21.15.39:0/1799563049 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b2574a6b0 msgr2=0x562b258f5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:57:41.627 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.614+0000 7f7bd164b700 1 --2- 172.21.15.39:0/1799563049 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b2574a6b0 0x562b258f5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:41.627 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.614+0000 7f7bd164b700 1 -- 172.21.15.39:0/1799563049 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b258e3be0 msgr2=0x562b258ee040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:57:41.627 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.614+0000 7f7bd164b700 1 --2- 172.21.15.39:0/1799563049 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b258e3be0 0x562b258ee040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:41.627 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.614+0000 7f7bd164b700 1 -- 172.21.15.39:0/1799563049 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b258e5420 con 0x562b258ee580 2023-07-21T18:57:41.629 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.614+0000 7f7bd164b700 1 --2- 172.21.15.39:0/1799563049 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b258ee580 0x562b258fcfa0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f7bc4003f60 tx=0x7f7bc4009460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=519a6ea71d44cb93 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:41.630 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.615+0000 7f7bb77fe700 1 -- 172.21.15.39:0/1799563049 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc4010040 con 0x562b258ee580 2023-07-21T18:57:41.630 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.615+0000 7f7bb77fe700 1 -- 172.21.15.39:0/1799563049 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bc4009bd0 con 0x562b258ee580 2023-07-21T18:57:41.630 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.615+0000 7f7bb77fe700 1 -- 172.21.15.39:0/1799563049 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc401f710 con 0x562b258ee580 2023-07-21T18:57:41.630 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.616+0000 7f7bdf234b80 1 -- 172.21.15.39:0/1799563049 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b258ee580 msgr2=0x562b258fcfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:41.630 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.616+0000 7f7bdf234b80 1 --2- 172.21.15.39:0/1799563049 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b258ee580 0x562b258fcfa0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f7bc4003f60 tx=0x7f7bc4009460 comp rx=0 tx=0).stop 2023-07-21T18:57:41.630 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.616+0000 7f7bdf234b80 1 -- 172.21.15.39:0/1799563049 shutdown_connections 2023-07-21T18:57:41.630 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.616+0000 7f7bdf234b80 1 --2- 172.21.15.39:0/1799563049 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b258e3be0 0x562b258ee040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:41.631 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.616+0000 7f7bdf234b80 1 --2- 172.21.15.39:0/1799563049 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b258ee580 0x562b258fcfa0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:41.631 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.616+0000 7f7bdf234b80 1 --2- 172.21.15.39:0/1799563049 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b2574a6b0 0x562b258f5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:41.631 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.616+0000 7f7bdf234b80 1 -- 172.21.15.39:0/1799563049 >> 172.21.15.39:0/1799563049 conn(0x562b258f8290 msgr2=0x562b258f30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:57:41.631 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.616+0000 7f7bdf234b80 1 -- 172.21.15.39:0/1799563049 shutdown_connections 2023-07-21T18:57:41.631 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.616+0000 7f7bdf234b80 1 -- 172.21.15.39:0/1799563049 wait complete. 2023-07-21T18:57:41.631 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.616+0000 7f7bdf234b80 1 -- start start 2023-07-21T18:57:41.631 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.616+0000 7f7bdf234b80 1 Processor -- start 2023-07-21T18:57:41.631 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.617+0000 7f7bdf234b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b2574a6b0 0x562b25954b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:41.632 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.617+0000 7f7bdf234b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b258e3be0 0x562b259550a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:41.632 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.617+0000 7f7bdf234b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b258ee580 0x562b2594eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:41.632 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.617+0000 7f7bdf234b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b2594f4a0 con 0x562b258e3be0 2023-07-21T18:57:41.632 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.617+0000 7f7bdf234b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b2594f610 con 0x562b2574a6b0 2023-07-21T18:57:41.632 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.617+0000 7f7bdf234b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b2594f7d0 con 0x562b258ee580 2023-07-21T18:57:41.632 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.617+0000 7f7bdf234b80 10 client.0 _ll_register_callbacks cb 0x7f7bb800dee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:57:41.632 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.617+0000 7f7bcbfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b258e3be0 0x562b259550a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:41.633 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.617+0000 7f7bd0e4a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b2574a6b0 0x562b25954b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:41.633 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.617+0000 7f7bcbfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b258e3be0 0x562b259550a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33560/0 (socket says 172.21.15.39:33560) 2023-07-21T18:57:41.633 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.617+0000 7f7bcbfff700 1 -- 172.21.15.39:0/318702795 learned_addr learned my addr 172.21.15.39:0/318702795 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:57:41.633 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.617+0000 7f7bd0e4a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b2574a6b0 0x562b25954b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:46154/0 (socket says 172.21.15.39:46154) 2023-07-21T18:57:41.633 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.617+0000 7f7bd164b700 1 --2- 172.21.15.39:0/318702795 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b258ee580 0x562b2594eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:41.633 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.617+0000 7f7bd0e4a700 1 -- 172.21.15.39:0/318702795 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b258ee580 msgr2=0x562b2594eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:57:41.633 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.617+0000 7f7bd0e4a700 1 --2- 172.21.15.39:0/318702795 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b258ee580 0x562b2594eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:41.634 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.617+0000 7f7bd0e4a700 1 -- 172.21.15.39:0/318702795 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b258e3be0 msgr2=0x562b259550a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:57:41.634 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.617+0000 7f7bd0e4a700 1 --2- 172.21.15.39:0/318702795 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b258e3be0 0x562b259550a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:41.634 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.618+0000 7f7bd0e4a700 1 -- 172.21.15.39:0/318702795 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bc4003c10 con 0x562b2574a6b0 2023-07-21T18:57:41.634 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.618+0000 7f7bcbfff700 1 --2- 172.21.15.39:0/318702795 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b258e3be0 0x562b259550a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:57:41.634 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.618+0000 7f7bd0e4a700 1 --2- 172.21.15.39:0/318702795 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b2574a6b0 0x562b25954b60 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7bc009b0f0 tx=0x7f7bc009a910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f83576961b26da18 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:41.634 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.618+0000 7f7babfff700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-21T18:57:41.634 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.618+0000 7f7babfff700 1 -- 172.21.15.39:0/318702795 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc0097270 con 0x562b2574a6b0 2023-07-21T18:57:41.634 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.618+0000 7f7babfff700 1 -- 172.21.15.39:0/318702795 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f7bc0099cb0 con 0x562b2574a6b0 2023-07-21T18:57:41.635 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.618+0000 7f7bdf234b80 1 -- 172.21.15.39:0/318702795 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7bc00992d0 con 0x562b2574a6b0 2023-07-21T18:57:41.635 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.618+0000 7f7babfff700 1 -- 172.21.15.39:0/318702795 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7bc00a32f0 con 0x562b2574a6b0 2023-07-21T18:57:41.635 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.618+0000 7f7bdf234b80 10 client.26807 fetch_fsmap learned FSMap version 88 2023-07-21T18:57:41.635 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.618+0000 7f7bdf234b80 1 -- 172.21.15.39:0/318702795 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=4+}) v3 -- 0x562b256941f0 con 0x562b2574a6b0 2023-07-21T18:57:41.635 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.619+0000 7f7babfff700 1 -- 172.21.15.39:0/318702795 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bc00992d0 con 0x562b2574a6b0 2023-07-21T18:57:41.636 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.619+0000 7f7babfff700 1 -- 172.21.15.39:0/318702795 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 88) v1 ==== 7526+0+0 (secure 0 0 0) 0x7f7bc000fad0 con 0x562b2574a6b0 2023-07-21T18:57:41.636 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.619+0000 7f7bdf234b80 10 client.26807 fetch_fsmap finished waiting for FSMap version 88 2023-07-21T18:57:41.636 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.619+0000 7f7bdf234b80 10 client.26807 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:57:41.636 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.619+0000 7f7bdf234b80 20 client.26807 populate_metadata read hostname 'smithi039' 2023-07-21T18:57:41.637 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.619+0000 7f7bdf234b80 1 --2- 172.21.15.39:0/318702795 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x562b258f42d0 0x562b2595e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:41.637 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.619+0000 7f7bdf234b80 4 client.26807 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:57:41.637 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.619+0000 7f7bdf234b80 1 -- 172.21.15.39:0/318702795 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562b257faff0 con 0x562b258f42d0 2023-07-21T18:57:41.637 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.619+0000 7f7bcbfff700 1 --2- 172.21.15.39:0/318702795 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x562b258f42d0 0x562b2595e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:41.637 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.619+0000 7f7babfff700 10 client.26807 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:57:41.637 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.619+0000 7f7bcbfff700 1 --2- 172.21.15.39:0/318702795 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x562b258f42d0 0x562b2595e4c0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c751b2526c0755f server_cookie=65d70628f850738f in_seq=0 out_seq=0 2023-07-21T18:57:41.673 INFO:tasks.workunit.client.0.smithi039.stdout: 10 102337 35.03 MB/sec execute 54 sec latency 57.471 ms 2023-07-21T18:57:41.822 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.810+0000 7f7babfff700 1 -- 172.21.15.39:0/318702795 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562b257faff0 con 0x562b258f42d0 2023-07-21T18:57:41.823 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.810+0000 7f7babfff700 10 client.26807 handle_command_reply: tid=0 2023-07-21T18:57:41.823 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.810+0000 7f7bdf234b80 1 client.26807 shutdown 2023-07-21T18:57:41.823 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.811+0000 7f7bdf234b80 1 -- 172.21.15.39:0/318702795 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b2574a6b0 msgr2=0x562b25954b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:41.823 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.811+0000 7f7bdf234b80 1 --2- 172.21.15.39:0/318702795 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b2574a6b0 0x562b25954b60 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7bc009b0f0 tx=0x7f7bc009a910 comp rx=0 tx=0).stop 2023-07-21T18:57:41.823 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.811+0000 7f7bdf234b80 1 -- 172.21.15.39:0/318702795 shutdown_connections 2023-07-21T18:57:41.823 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.811+0000 7f7bdf234b80 1 --2- 172.21.15.39:0/318702795 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x562b258f42d0 0x562b2595e4c0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:41.823 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.811+0000 7f7bdf234b80 1 --2- 172.21.15.39:0/318702795 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b2574a6b0 0x562b25954b60 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:41.824 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.811+0000 7f7bdf234b80 1 --2- 172.21.15.39:0/318702795 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b258ee580 0x562b2594eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:41.824 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.811+0000 7f7bdf234b80 1 --2- 172.21.15.39:0/318702795 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562b258e3be0 0x562b259550a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:41.824 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.811+0000 7f7bdf234b80 1 -- 172.21.15.39:0/318702795 >> 172.21.15.39:0/318702795 conn(0x562b258f8290 msgr2=0x562b258f5130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:57:41.824 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.811+0000 7f7babfff700 0 client.26807 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:57:41.824 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.811+0000 7f7bdf234b80 1 -- 172.21.15.39:0/318702795 shutdown_connections 2023-07-21T18:57:41.824 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.811+0000 7f7bdf234b80 1 -- 172.21.15.39:0/318702795 wait complete. 2023-07-21T18:57:41.824 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.811+0000 7f7bdf234b80 20 client.26807 trim_cache size 0 max 16384 2023-07-21T18:57:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:57:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:57:41.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:57:41.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:57:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.812+0000 7f7bd30ae700 1 -- 172.21.15.39:0/3931896770 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f7bbc0e52d0 msgr2=0x7f7bbc0e7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.812+0000 7f7bd30ae700 1 --2- 172.21.15.39:0/3931896770 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f7bbc0e52d0 0x7f7bbc0e7780 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc195210 tx=0x7f7bb800f040 comp rx=0 tx=0).stop 2023-07-21T18:57:41.828 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.812+0000 7f7bd30ae700 1 -- 172.21.15.39:0/3931896770 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7bcc0fea30 msgr2=0x7f7bcc1985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:41.828 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.812+0000 7f7bd30ae700 1 --2- 172.21.15.39:0/3931896770 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7bcc0fea30 0x7f7bcc1985a0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0000c00 tx=0x7f7bc0004730 comp rx=0 tx=0).stop 2023-07-21T18:57:41.828 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.812+0000 7f7bd30ae700 1 -- 172.21.15.39:0/3931896770 shutdown_connections 2023-07-21T18:57:41.828 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.812+0000 7f7bd30ae700 1 --2- 172.21.15.39:0/3931896770 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f7bbc0e52d0 0x7f7bbc0e7780 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:41.828 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.812+0000 7f7bd30ae700 1 --2- 172.21.15.39:0/3931896770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7bcc104a30 0x7f7bcc193920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:41.828 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.812+0000 7f7bd30ae700 1 --2- 172.21.15.39:0/3931896770 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7bcc0fea30 0x7f7bcc1985a0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:41.829 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.812+0000 7f7bd30ae700 1 --2- 172.21.15.39:0/3931896770 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7bcc0ff3e0 0x7f7bcc198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:41.829 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.812+0000 7f7bd30ae700 1 -- 172.21.15.39:0/3931896770 >> 172.21.15.39:0/3931896770 conn(0x7f7bcc074760 msgr2=0x7f7bcc0fcff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:57:41.829 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.812+0000 7f7bd30ae700 1 -- 172.21.15.39:0/3931896770 shutdown_connections 2023-07-21T18:57:41.829 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:41.812+0000 7f7bd30ae700 1 -- 172.21.15.39:0/3931896770 wait complete. 2023-07-21T18:57:42.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:57:42 smithi193 ceph-mon[98204]: pgmap v503: 97 pgs: 97 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 8.5 MiB/s wr, 739 op/s 2023-07-21T18:57:42.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:57:42 smithi155 ceph-mon[97934]: pgmap v503: 97 pgs: 97 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 8.5 MiB/s wr, 739 op/s 2023-07-21T18:57:42.673 INFO:tasks.workunit.client.0.smithi039.stdout: 10 103075 35.10 MB/sec execute 55 sec latency 166.465 ms 2023-07-21T18:57:42.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:57:42 smithi039 ceph-mon[96489]: pgmap v503: 97 pgs: 97 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 8.5 MiB/s wr, 739 op/s 2023-07-21T18:57:43.673 INFO:tasks.workunit.client.0.smithi039.stdout: 10 103785 35.32 MB/sec execute 56 sec latency 60.737 ms 2023-07-21T18:57:44.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:57:44 smithi193 ceph-mon[98204]: pgmap v504: 97 pgs: 97 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.6 MiB/s wr, 491 op/s 2023-07-21T18:57:44.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:57:44 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:57:44.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:57:44 smithi155 ceph-mon[97934]: pgmap v504: 97 pgs: 97 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.6 MiB/s wr, 491 op/s 2023-07-21T18:57:44.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:57:44 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:57:44.673 INFO:tasks.workunit.client.0.smithi039.stdout: 10 104407 35.12 MB/sec execute 57 sec latency 51.753 ms 2023-07-21T18:57:44.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:57:44 smithi039 ceph-mon[96489]: pgmap v504: 97 pgs: 97 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.6 MiB/s wr, 491 op/s 2023-07-21T18:57:44.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:57:44 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:57:45.674 INFO:tasks.workunit.client.0.smithi039.stdout: 10 105130 35.18 MB/sec execute 58 sec latency 82.636 ms 2023-07-21T18:57:46.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:57:46 smithi193 ceph-mon[98204]: pgmap v505: 97 pgs: 97 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.6 MiB/s wr, 491 op/s 2023-07-21T18:57:46.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:57:46 smithi155 ceph-mon[97934]: pgmap v505: 97 pgs: 97 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.6 MiB/s wr, 491 op/s 2023-07-21T18:57:46.674 INFO:tasks.workunit.client.0.smithi039.stdout: 10 106065 35.40 MB/sec execute 59 sec latency 37.807 ms 2023-07-21T18:57:46.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:57:46 smithi039 ceph-mon[96489]: pgmap v505: 97 pgs: 97 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.6 MiB/s wr, 491 op/s 2023-07-21T18:57:47.675 INFO:tasks.workunit.client.0.smithi039.stdout: 10 106889 35.75 MB/sec execute 60 sec latency 72.883 ms 2023-07-21T18:57:48.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:57:48 smithi193 ceph-mon[98204]: pgmap v506: 97 pgs: 97 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 8.4 MiB/s wr, 814 op/s 2023-07-21T18:57:48.630 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:57:48 smithi155 ceph-mon[97934]: pgmap v506: 97 pgs: 97 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 8.4 MiB/s wr, 814 op/s 2023-07-21T18:57:48.675 INFO:tasks.workunit.client.0.smithi039.stdout: 10 107319 35.53 MB/sec execute 61 sec latency 299.258 ms 2023-07-21T18:57:48.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:57:48 smithi039 ceph-mon[96489]: pgmap v506: 97 pgs: 97 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 8.4 MiB/s wr, 814 op/s 2023-07-21T18:57:49.675 INFO:tasks.workunit.client.0.smithi039.stdout: 10 107338 34.96 MB/sec execute 62 sec latency 1261.717 ms 2023-07-21T18:57:50.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:57:50 smithi193 ceph-mon[98204]: pgmap v507: 97 pgs: 97 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.7 MiB/s wr, 564 op/s 2023-07-21T18:57:50.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:57:50 smithi155 ceph-mon[97934]: pgmap v507: 97 pgs: 97 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.7 MiB/s wr, 564 op/s 2023-07-21T18:57:50.675 INFO:tasks.workunit.client.0.smithi039.stdout: 10 107560 34.54 MB/sec execute 63 sec latency 1518.037 ms 2023-07-21T18:57:50.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:57:50 smithi039 ceph-mon[96489]: pgmap v507: 97 pgs: 97 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.7 MiB/s wr, 564 op/s 2023-07-21T18:57:51.675 INFO:tasks.workunit.client.0.smithi039.stdout: 10 107842 34.17 MB/sec execute 64 sec latency 206.085 ms 2023-07-21T18:57:52.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:57:52 smithi193 ceph-mon[98204]: pgmap v508: 97 pgs: 97 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 25 MiB/s wr, 1.05k op/s 2023-07-21T18:57:52.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:57:52 smithi155 ceph-mon[97934]: pgmap v508: 97 pgs: 97 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 25 MiB/s wr, 1.05k op/s 2023-07-21T18:57:52.675 INFO:tasks.workunit.client.0.smithi039.stdout: 10 108580 34.21 MB/sec execute 65 sec latency 64.575 ms 2023-07-21T18:57:52.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:57:52 smithi039 ceph-mon[96489]: pgmap v508: 97 pgs: 97 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 25 MiB/s wr, 1.05k op/s 2023-07-21T18:57:53.675 INFO:tasks.workunit.client.0.smithi039.stdout: 10 109380 34.39 MB/sec execute 66 sec latency 63.847 ms 2023-07-21T18:57:54.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:57:54 smithi193 ceph-mon[98204]: pgmap v509: 97 pgs: 97 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 23 MiB/s wr, 807 op/s 2023-07-21T18:57:54.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:57:54 smithi155 ceph-mon[97934]: pgmap v509: 97 pgs: 97 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 23 MiB/s wr, 807 op/s 2023-07-21T18:57:54.675 INFO:tasks.workunit.client.0.smithi039.stdout: 10 110123 34.51 MB/sec execute 67 sec latency 76.730 ms 2023-07-21T18:57:54.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:57:54 smithi039 ceph-mon[96489]: pgmap v509: 97 pgs: 97 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 23 MiB/s wr, 807 op/s 2023-07-21T18:57:54.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.971+0000 7fcbc1049700 1 -- 172.21.15.39:0/3209195911 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcbbc0ff4b0 msgr2=0x7fcbbc103980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:54.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.971+0000 7fcbc1049700 1 --2- 172.21.15.39:0/3209195911 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcbbc0ff4b0 0x7fcbbc103980 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fcba4009990 tx=0x7fcba4009ca0 comp rx=0 tx=0).stop 2023-07-21T18:57:54.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.972+0000 7fcbc1049700 1 -- 172.21.15.39:0/3209195911 shutdown_connections 2023-07-21T18:57:54.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.972+0000 7fcbc1049700 1 --2- 172.21.15.39:0/3209195911 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcbbc103ec0 0x7fcbbc104340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:54.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.972+0000 7fcbc1049700 1 --2- 172.21.15.39:0/3209195911 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcbbc0feba0 0x7fcbbc0fef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.972+0000 7fcbc1049700 1 --2- 172.21.15.39:0/3209195911 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcbbc0ff4b0 0x7fcbbc103980 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.972+0000 7fcbc1049700 1 -- 172.21.15.39:0/3209195911 >> 172.21.15.39:0/3209195911 conn(0x7fcbbc075c20 msgr2=0x7fcbbc076020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:57:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.972+0000 7fcbc1049700 1 -- 172.21.15.39:0/3209195911 shutdown_connections 2023-07-21T18:57:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.972+0000 7fcbc1049700 1 -- 172.21.15.39:0/3209195911 wait complete. 2023-07-21T18:57:54.986 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.973+0000 7fcbc1049700 1 Processor -- start 2023-07-21T18:57:54.986 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.973+0000 7fcbc1049700 1 -- start start 2023-07-21T18:57:54.986 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.973+0000 7fcbc1049700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcbbc0feba0 0x7fcbbc19c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:54.987 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.973+0000 7fcbc1049700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcbbc0ff4b0 0x7fcbbc19c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:54.987 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.973+0000 7fcbc1049700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcbbc103ec0 0x7fcbbc1a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:54.987 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.973+0000 7fcbc1049700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbbc1a1450 con 0x7fcbbc103ec0 2023-07-21T18:57:54.987 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.973+0000 7fcbc1049700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbbc1a15c0 con 0x7fcbbc0ff4b0 2023-07-21T18:57:54.987 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.973+0000 7fcbc1049700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbbc1a1780 con 0x7fcbbc0feba0 2023-07-21T18:57:54.987 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.974+0000 7fcbba59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcbbc0ff4b0 0x7fcbbc19c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:54.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.974+0000 7fcbba59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcbbc0ff4b0 0x7fcbbc19c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:46168/0 (socket says 172.21.15.39:46168) 2023-07-21T18:57:54.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.974+0000 7fcbba59c700 1 -- 172.21.15.39:0/635047184 learned_addr learned my addr 172.21.15.39:0/635047184 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:57:54.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.974+0000 7fcbbb59e700 1 --2- 172.21.15.39:0/635047184 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcbbc103ec0 0x7fcbbc1a0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:54.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.974+0000 7fcbbad9d700 1 --2- 172.21.15.39:0/635047184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcbbc0feba0 0x7fcbbc19c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:54.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.974+0000 7fcbbb59e700 1 -- 172.21.15.39:0/635047184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcbbc0feba0 msgr2=0x7fcbbc19c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:54.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.974+0000 7fcbbb59e700 1 --2- 172.21.15.39:0/635047184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcbbc0feba0 0x7fcbbc19c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:54.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.974+0000 7fcbbb59e700 1 -- 172.21.15.39:0/635047184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcbbc0ff4b0 msgr2=0x7fcbbc19c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:54.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.974+0000 7fcbbb59e700 1 --2- 172.21.15.39:0/635047184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcbbc0ff4b0 0x7fcbbc19c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:54.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.974+0000 7fcbbb59e700 1 -- 172.21.15.39:0/635047184 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbb4009520 con 0x7fcbbc103ec0 2023-07-21T18:57:54.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.974+0000 7fcbbb59e700 1 --2- 172.21.15.39:0/635047184 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcbbc103ec0 0x7fcbbc1a0e10 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fcbac00efc0 tx=0x7fcbac00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:54.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.975+0000 7fcbb3fff700 1 -- 172.21.15.39:0/635047184 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbac018070 con 0x7fcbbc103ec0 2023-07-21T18:57:54.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.975+0000 7fcbc1049700 1 -- 172.21.15.39:0/635047184 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcba40095f0 con 0x7fcbbc103ec0 2023-07-21T18:57:54.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.975+0000 7fcbc1049700 1 -- 172.21.15.39:0/635047184 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbbc1a90c0 con 0x7fcbbc103ec0 2023-07-21T18:57:54.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.975+0000 7fcbb3fff700 1 -- 172.21.15.39:0/635047184 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcbac00f040 con 0x7fcbbc103ec0 2023-07-21T18:57:54.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.975+0000 7fcbb3fff700 1 -- 172.21.15.39:0/635047184 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbac003cf0 con 0x7fcbbc103ec0 2023-07-21T18:57:54.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.979+0000 7fcbb3fff700 1 -- 172.21.15.39:0/635047184 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fcbac0092e0 con 0x7fcbbc103ec0 2023-07-21T18:57:54.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.979+0000 7fcbcd1cfb80 1 Processor -- start 2023-07-21T18:57:54.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.979+0000 7fcbcd1cfb80 1 -- start start 2023-07-21T18:57:54.992 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.979+0000 7fcbb3fff700 1 --2- 172.21.15.39:0/635047184 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fcba80a1340 0x7fcba80a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:54.992 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.980+0000 7fcbb3fff700 1 -- 172.21.15.39:0/635047184 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fcbac0cc7c0 con 0x7fcbbc103ec0 2023-07-21T18:57:54.992 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.980+0000 7fcbcd1cfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bdeb0166b0 0x55bdeb1bfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:54.992 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.980+0000 7fcbcd1cfb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bdeb1c02d0 0x55bdeb1ba040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:54.992 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.980+0000 7fcbcd1cfb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bdeb1ba580 0x55bdeb1ba9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:54.992 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.980+0000 7fcbcd1cfb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdeb0bfc50 con 0x55bdeb1ba580 2023-07-21T18:57:54.992 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.980+0000 7fcbcd1cfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdeb1129e0 con 0x55bdeb0166b0 2023-07-21T18:57:54.993 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.980+0000 7fcbcd1cfb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdeb10be70 con 0x55bdeb1c02d0 2023-07-21T18:57:54.993 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.980+0000 7fcbbb59e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bdeb1ba580 0x55bdeb1ba9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:54.993 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.980+0000 7fcbba59c700 1 --2- 172.21.15.39:0/635047184 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fcba80a1340 0x7fcba80a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:54.993 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.980+0000 7fcbbb59e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bdeb1ba580 0x55bdeb1ba9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33580/0 (socket says 172.21.15.39:33580) 2023-07-21T18:57:54.993 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.980+0000 7fcbbb59e700 1 -- 172.21.15.39:0/212636648 learned_addr learned my addr 172.21.15.39:0/212636648 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:57:54.993 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.980+0000 7fcbbad9d700 1 --2- 172.21.15.39:0/212636648 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bdeb0166b0 0x55bdeb1bfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:54.993 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.980+0000 7fcbba59c700 1 --2- 172.21.15.39:0/212636648 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bdeb1c02d0 0x55bdeb1ba040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:54.993 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.981+0000 7fcbbb59e700 1 -- 172.21.15.39:0/212636648 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bdeb1c02d0 msgr2=0x55bdeb1ba040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:57:54.994 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.981+0000 7fcbbb59e700 1 --2- 172.21.15.39:0/212636648 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bdeb1c02d0 0x55bdeb1ba040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:54.994 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.981+0000 7fcbbb59e700 1 -- 172.21.15.39:0/212636648 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bdeb0166b0 msgr2=0x55bdeb1bfd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:57:54.994 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.981+0000 7fcbbb59e700 1 --2- 172.21.15.39:0/212636648 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bdeb0166b0 0x55bdeb1bfd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:54.994 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.981+0000 7fcbbb59e700 1 -- 172.21.15.39:0/212636648 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bdeaf39d80 con 0x55bdeb1ba580 2023-07-21T18:57:54.994 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.981+0000 7fcbba59c700 1 --2- 172.21.15.39:0/635047184 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fcba80a1340 0x7fcba80a37f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc19da80 tx=0x7fcba400b690 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:54.994 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.981+0000 7fcbbb59e700 1 --2- 172.21.15.39:0/212636648 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bdeb1ba580 0x55bdeb1ba9d0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fcbac0998b0 tx=0x7fcbac0123c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6be64ffa0baa895 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:54.997 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.981+0000 7fcbb17fa700 1 -- 172.21.15.39:0/212636648 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbac0a1ce0 con 0x55bdeb1ba580 2023-07-21T18:57:54.997 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.981+0000 7fcbb17fa700 1 -- 172.21.15.39:0/212636648 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcbac004ad0 con 0x55bdeb1ba580 2023-07-21T18:57:54.997 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.981+0000 7fcbb17fa700 1 -- 172.21.15.39:0/212636648 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbac099070 con 0x55bdeb1ba580 2023-07-21T18:57:54.998 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.982+0000 7fcbcd1cfb80 1 -- 172.21.15.39:0/212636648 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bdeb1ba580 msgr2=0x55bdeb1ba9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:54.998 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.982+0000 7fcbcd1cfb80 1 --2- 172.21.15.39:0/212636648 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bdeb1ba580 0x55bdeb1ba9d0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fcbac0998b0 tx=0x7fcbac0123c0 comp rx=0 tx=0).stop 2023-07-21T18:57:54.998 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.982+0000 7fcbcd1cfb80 1 -- 172.21.15.39:0/212636648 shutdown_connections 2023-07-21T18:57:54.998 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.982+0000 7fcbcd1cfb80 1 --2- 172.21.15.39:0/212636648 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bdeb0166b0 0x55bdeb1bfd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:54.998 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.982+0000 7fcbcd1cfb80 1 --2- 172.21.15.39:0/212636648 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bdeb1ba580 0x55bdeb1ba9d0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:54.998 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.982+0000 7fcbcd1cfb80 1 --2- 172.21.15.39:0/212636648 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bdeb1c02d0 0x55bdeb1ba040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:54.999 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.982+0000 7fcbcd1cfb80 1 -- 172.21.15.39:0/212636648 >> 172.21.15.39:0/212636648 conn(0x55bdeb1afbe0 msgr2=0x55bdeb1b4f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:57:54.999 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.982+0000 7fcbcd1cfb80 1 -- 172.21.15.39:0/212636648 shutdown_connections 2023-07-21T18:57:54.999 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.982+0000 7fcbcd1cfb80 1 -- 172.21.15.39:0/212636648 wait complete. 2023-07-21T18:57:54.999 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.983+0000 7fcbcd1cfb80 1 -- start start 2023-07-21T18:57:54.999 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.983+0000 7fcbcd1cfb80 1 Processor -- start 2023-07-21T18:57:54.999 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.983+0000 7fcbcd1cfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bdeb0166b0 0x55bdeb1c95d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:55.000 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.983+0000 7fcbcd1cfb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bdeb1ba580 0x55bdeb1c9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:55.000 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.983+0000 7fcbcd1cfb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bdeb1c02d0 0x55bdeb1c3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:55.000 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.983+0000 7fcbcd1cfb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdeb0aba60 con 0x55bdeb1ba580 2023-07-21T18:57:55.000 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.983+0000 7fcbcd1cfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdeb0ab760 con 0x55bdeb0166b0 2023-07-21T18:57:55.000 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.983+0000 7fcbcd1cfb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdeaf8f8c0 con 0x55bdeb1c02d0 2023-07-21T18:57:55.000 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.983+0000 7fcbcd1cfb80 10 client.0 _ll_register_callbacks cb 0x55bdeb0a9700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:57:55.000 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.983+0000 7fcbba59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bdeb1ba580 0x55bdeb1c9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:55.000 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.983+0000 7fcbbad9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bdeb0166b0 0x55bdeb1c95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:55.001 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.983+0000 7fcbba59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bdeb1ba580 0x55bdeb1c9b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33586/0 (socket says 172.21.15.39:33586) 2023-07-21T18:57:55.001 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.983+0000 7fcbba59c700 1 -- 172.21.15.39:0/738814603 learned_addr learned my addr 172.21.15.39:0/738814603 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:57:55.001 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.983+0000 7fcbbb59e700 1 --2- 172.21.15.39:0/738814603 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bdeb1c02d0 0x55bdeb1c3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:55.001 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.984+0000 7fcbba59c700 1 -- 172.21.15.39:0/738814603 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bdeb1c02d0 msgr2=0x55bdeb1c3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:57:55.001 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.984+0000 7fcbba59c700 1 --2- 172.21.15.39:0/738814603 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bdeb1c02d0 0x55bdeb1c3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:55.001 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.984+0000 7fcbba59c700 1 -- 172.21.15.39:0/738814603 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bdeb0166b0 msgr2=0x55bdeb1c95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:57:55.001 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.984+0000 7fcbba59c700 1 --2- 172.21.15.39:0/738814603 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bdeb0166b0 0x55bdeb1c95d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:55.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.984+0000 7fcbba59c700 1 -- 172.21.15.39:0/738814603 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbbc1a90c0 con 0x55bdeb1ba580 2023-07-21T18:57:55.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.984+0000 7fcbba59c700 1 --2- 172.21.15.39:0/738814603 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bdeb1ba580 0x55bdeb1c9b10 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fcba4011040 tx=0x7fcba400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ff5d64a69b969a7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:55.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.984+0000 7fcba27fc700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:57:55.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.984+0000 7fcba27fc700 1 -- 172.21.15.39:0/738814603 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba4029df0 con 0x55bdeb1ba580 2023-07-21T18:57:55.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.984+0000 7fcbcd1cfb80 1 -- 172.21.15.39:0/738814603 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bdeb065490 con 0x55bdeb1ba580 2023-07-21T18:57:55.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.984+0000 7fcba27fc700 1 -- 172.21.15.39:0/738814603 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fcba400f460 con 0x55bdeb1ba580 2023-07-21T18:57:55.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.984+0000 7fcba27fc700 1 -- 172.21.15.39:0/738814603 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcba400f620 con 0x55bdeb1ba580 2023-07-21T18:57:55.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.984+0000 7fcbcd1cfb80 10 client.17394 fetch_fsmap learned FSMap version 89 2023-07-21T18:57:55.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.984+0000 7fcbcd1cfb80 1 -- 172.21.15.39:0/738814603 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=4+}) v3 -- 0x55bdeaf22c00 con 0x55bdeb1ba580 2023-07-21T18:57:55.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.985+0000 7fcba27fc700 1 -- 172.21.15.39:0/738814603 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcba4040020 con 0x55bdeb1ba580 2023-07-21T18:57:55.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.985+0000 7fcba27fc700 1 -- 172.21.15.39:0/738814603 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 89) v1 ==== 7522+0+0 (secure 0 0 0) 0x7fcba403c8c0 con 0x55bdeb1ba580 2023-07-21T18:57:55.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.985+0000 7fcbcd1cfb80 10 client.17394 fetch_fsmap finished waiting for FSMap version 89 2023-07-21T18:57:55.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.985+0000 7fcbcd1cfb80 10 client.17394 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:57:55.004 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.985+0000 7fcbcd1cfb80 20 client.17394 populate_metadata read hostname 'smithi039' 2023-07-21T18:57:55.004 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.985+0000 7fcbcd1cfb80 1 --2- 172.21.15.39:0/738814603 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55bdeb1d3c20 0x55bdeb1d6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:55.004 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.985+0000 7fcbcd1cfb80 4 client.17394 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:57:55.004 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.985+0000 7fcbcd1cfb80 1 -- 172.21.15.39:0/738814603 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bdeb104e50 con 0x55bdeb1d3c20 2023-07-21T18:57:55.004 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.986+0000 7fcbbad9d700 1 --2- 172.21.15.39:0/738814603 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55bdeb1d3c20 0x55bdeb1d6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:55.004 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.987+0000 7fcbbad9d700 1 --2- 172.21.15.39:0/738814603 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55bdeb1d3c20 0x55bdeb1d6030 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=622d2072f8a7e731 server_cookie=11755c709a3a9d94 in_seq=0 out_seq=0 2023-07-21T18:57:55.005 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.987+0000 7fcba27fc700 10 client.17394 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:57:55.006 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.988+0000 7fcba27fc700 1 -- 172.21.15.39:0/738814603 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55bdeb104e50 con 0x55bdeb1d3c20 2023-07-21T18:57:55.006 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.988+0000 7fcba27fc700 10 client.17394 handle_command_reply: tid=0 2023-07-21T18:57:55.006 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.988+0000 7fcbcd1cfb80 1 client.17394 shutdown 2023-07-21T18:57:55.006 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.988+0000 7fcbcd1cfb80 1 -- 172.21.15.39:0/738814603 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bdeb1ba580 msgr2=0x55bdeb1c9b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:55.006 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.988+0000 7fcbcd1cfb80 1 --2- 172.21.15.39:0/738814603 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bdeb1ba580 0x55bdeb1c9b10 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fcba4011040 tx=0x7fcba400b260 comp rx=0 tx=0).stop 2023-07-21T18:57:55.007 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.988+0000 7fcbcd1cfb80 1 -- 172.21.15.39:0/738814603 shutdown_connections 2023-07-21T18:57:55.007 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.988+0000 7fcbcd1cfb80 1 --2- 172.21.15.39:0/738814603 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55bdeb1d3c20 0x55bdeb1d6030 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:55.007 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.988+0000 7fcbcd1cfb80 1 --2- 172.21.15.39:0/738814603 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bdeb0166b0 0x55bdeb1c95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:55.007 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.988+0000 7fcbcd1cfb80 1 --2- 172.21.15.39:0/738814603 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bdeb1c02d0 0x55bdeb1c3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:55.007 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.988+0000 7fcbcd1cfb80 1 --2- 172.21.15.39:0/738814603 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bdeb1ba580 0x55bdeb1c9b10 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:55.007 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.988+0000 7fcbcd1cfb80 1 -- 172.21.15.39:0/738814603 >> 172.21.15.39:0/738814603 conn(0x55bdeb1afbe0 msgr2=0x55bdeb1b44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:57:55.007 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.988+0000 7fcba27fc700 0 client.17394 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:57:55.007 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.988+0000 7fcbcd1cfb80 1 -- 172.21.15.39:0/738814603 shutdown_connections 2023-07-21T18:57:55.008 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.988+0000 7fcbcd1cfb80 1 -- 172.21.15.39:0/738814603 wait complete. 2023-07-21T18:57:55.008 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:54.988+0000 7fcbcd1cfb80 20 client.17394 trim_cache size 0 max 16384 2023-07-21T18:57:55.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.006+0000 7fcbcd1cfb80 1 Processor -- start 2023-07-21T18:57:55.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.006+0000 7fcbcd1cfb80 1 -- start start 2023-07-21T18:57:55.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.006+0000 7fcbcd1cfb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bdeb0166b0 0x55bdeb1ba040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:55.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.006+0000 7fcbcd1cfb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bdeb1ba580 0x55bdeb1c1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:55.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.006+0000 7fcbcd1cfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bdeb1c02d0 0x55bdeb1c1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:55.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.006+0000 7fcbcd1cfb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdeb1be3e0 con 0x55bdeb0166b0 2023-07-21T18:57:55.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.006+0000 7fcbcd1cfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdeb1c1bc0 con 0x55bdeb1c02d0 2023-07-21T18:57:55.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.006+0000 7fcbcd1cfb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdeb1b6740 con 0x55bdeb1ba580 2023-07-21T18:57:55.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.006+0000 7fcbbad9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bdeb0166b0 0x55bdeb1ba040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.006+0000 7fcbbad9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bdeb0166b0 0x55bdeb1ba040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33592/0 (socket says 172.21.15.39:33592) 2023-07-21T18:57:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.006+0000 7fcbbad9d700 1 -- 172.21.15.39:0/2929562729 learned_addr learned my addr 172.21.15.39:0/2929562729 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:57:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.006+0000 7fcbba59c700 1 --2- 172.21.15.39:0/2929562729 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bdeb1ba580 0x55bdeb1c1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.006+0000 7fcbbb59e700 1 --2- 172.21.15.39:0/2929562729 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bdeb1c02d0 0x55bdeb1c1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.007+0000 7fcbbad9d700 1 -- 172.21.15.39:0/2929562729 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bdeb1ba580 msgr2=0x55bdeb1c1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:57:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.007+0000 7fcbbad9d700 1 --2- 172.21.15.39:0/2929562729 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bdeb1ba580 0x55bdeb1c1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.007+0000 7fcbbad9d700 1 -- 172.21.15.39:0/2929562729 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bdeb1c02d0 msgr2=0x55bdeb1c1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:57:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.007+0000 7fcbbad9d700 1 --2- 172.21.15.39:0/2929562729 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bdeb1c02d0 0x55bdeb1c1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.007+0000 7fcbbad9d700 1 -- 172.21.15.39:0/2929562729 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bdeaf601f0 con 0x55bdeb0166b0 2023-07-21T18:57:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.007+0000 7fcbbad9d700 1 --2- 172.21.15.39:0/2929562729 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bdeb0166b0 0x55bdeb1ba040 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fcbb400a010 tx=0x7fcbb4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3abd449fd4d80cf server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.007+0000 7fcbb1ffb700 1 -- 172.21.15.39:0/2929562729 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb401f270 con 0x55bdeb0166b0 2023-07-21T18:57:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.007+0000 7fcbb1ffb700 1 -- 172.21.15.39:0/2929562729 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcbb4018040 con 0x55bdeb0166b0 2023-07-21T18:57:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.007+0000 7fcbb1ffb700 1 -- 172.21.15.39:0/2929562729 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb4011a30 con 0x55bdeb0166b0 2023-07-21T18:57:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.008+0000 7fcbcd1cfb80 1 -- 172.21.15.39:0/2929562729 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bdeb0166b0 msgr2=0x55bdeb1ba040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.008+0000 7fcbcd1cfb80 1 --2- 172.21.15.39:0/2929562729 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bdeb0166b0 0x55bdeb1ba040 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fcbb400a010 tx=0x7fcbb4009260 comp rx=0 tx=0).stop 2023-07-21T18:57:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.008+0000 7fcbcd1cfb80 1 -- 172.21.15.39:0/2929562729 shutdown_connections 2023-07-21T18:57:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.008+0000 7fcbcd1cfb80 1 --2- 172.21.15.39:0/2929562729 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bdeb1c02d0 0x55bdeb1c1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.008+0000 7fcbcd1cfb80 1 --2- 172.21.15.39:0/2929562729 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bdeb0166b0 0x55bdeb1ba040 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.008+0000 7fcbcd1cfb80 1 --2- 172.21.15.39:0/2929562729 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bdeb1ba580 0x55bdeb1c1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.008+0000 7fcbcd1cfb80 1 -- 172.21.15.39:0/2929562729 >> 172.21.15.39:0/2929562729 conn(0x55bdeb1afbe0 msgr2=0x55bdeb1c0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:57:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.008+0000 7fcbcd1cfb80 1 -- 172.21.15.39:0/2929562729 shutdown_connections 2023-07-21T18:57:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.008+0000 7fcbcd1cfb80 1 -- 172.21.15.39:0/2929562729 wait complete. 2023-07-21T18:57:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.009+0000 7fcbcd1cfb80 1 -- start start 2023-07-21T18:57:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.009+0000 7fcbcd1cfb80 1 Processor -- start 2023-07-21T18:57:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.009+0000 7fcbcd1cfb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bdeb0166b0 0x55bdeb1c77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.009+0000 7fcbcd1cfb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bdeb1ba580 0x55bdeb1c7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.009+0000 7fcbcd1cfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bdeb1c02d0 0x55bdeb226aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.009+0000 7fcbcd1cfb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdeb226fe0 con 0x55bdeb1ba580 2023-07-21T18:57:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.009+0000 7fcbcd1cfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdeb227120 con 0x55bdeb1c02d0 2023-07-21T18:57:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.009+0000 7fcbcd1cfb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdeb227290 con 0x55bdeb0166b0 2023-07-21T18:57:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.009+0000 7fcbcd1cfb80 10 client.0 _ll_register_callbacks cb 0x7fcba4005c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:57:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.009+0000 7fcbbad9d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bdeb0166b0 0x55bdeb1c77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.009+0000 7fcbbad9d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bdeb0166b0 0x55bdeb1c77c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:52150/0 (socket says 172.21.15.39:52150) 2023-07-21T18:57:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.009+0000 7fcbbad9d700 1 -- 172.21.15.39:0/2508214507 learned_addr learned my addr 172.21.15.39:0/2508214507 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:57:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.009+0000 7fcbba59c700 1 --2- 172.21.15.39:0/2508214507 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bdeb1ba580 0x55bdeb1c7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.009+0000 7fcbbb59e700 1 --2- 172.21.15.39:0/2508214507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bdeb1c02d0 0x55bdeb226aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:55.026 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.010+0000 7fcbba59c700 1 -- 172.21.15.39:0/2508214507 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bdeb0166b0 msgr2=0x55bdeb1c77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:57:55.026 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.010+0000 7fcbba59c700 1 --2- 172.21.15.39:0/2508214507 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bdeb0166b0 0x55bdeb1c77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:55.026 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.010+0000 7fcbba59c700 1 -- 172.21.15.39:0/2508214507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bdeb1c02d0 msgr2=0x55bdeb226aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:57:55.026 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.010+0000 7fcbba59c700 1 --2- 172.21.15.39:0/2508214507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bdeb1c02d0 0x55bdeb226aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:55.026 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.010+0000 7fcbba59c700 1 -- 172.21.15.39:0/2508214507 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbb4009c40 con 0x55bdeb1ba580 2023-07-21T18:57:55.026 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.010+0000 7fcbba59c700 1 --2- 172.21.15.39:0/2508214507 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bdeb1ba580 0x55bdeb1c7d00 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fcba400b920 tx=0x7fcba400beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=639db7b9009c74e7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:55.026 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.010+0000 7fcba27fc700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:57:55.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.010+0000 7fcba27fc700 1 -- 172.21.15.39:0/2508214507 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba403c660 con 0x55bdeb1ba580 2023-07-21T18:57:55.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.010+0000 7fcba27fc700 1 -- 172.21.15.39:0/2508214507 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fcba403ce40 con 0x55bdeb1ba580 2023-07-21T18:57:55.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.010+0000 7fcbcd1cfb80 1 -- 172.21.15.39:0/2508214507 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcbac004ad0 con 0x55bdeb1ba580 2023-07-21T18:57:55.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.010+0000 7fcba27fc700 1 -- 172.21.15.39:0/2508214507 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcba403c7c0 con 0x55bdeb1ba580 2023-07-21T18:57:55.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.010+0000 7fcbcd1cfb80 10 client.17400 fetch_fsmap learned FSMap version 89 2023-07-21T18:57:55.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.010+0000 7fcbcd1cfb80 1 -- 172.21.15.39:0/2508214507 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=4+}) v3 -- 0x55bdeaf38250 con 0x55bdeb1ba580 2023-07-21T18:57:55.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.010+0000 7fcba27fc700 1 -- 172.21.15.39:0/2508214507 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcbac004ad0 con 0x55bdeb1ba580 2023-07-21T18:57:55.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.011+0000 7fcba27fc700 1 -- 172.21.15.39:0/2508214507 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 89) v1 ==== 7522+0+0 (secure 0 0 0) 0x7fcba404e480 con 0x55bdeb1ba580 2023-07-21T18:57:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.011+0000 7fcbcd1cfb80 10 client.17400 fetch_fsmap finished waiting for FSMap version 89 2023-07-21T18:57:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.011+0000 7fcbcd1cfb80 10 client.17400 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:57:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.011+0000 7fcbcd1cfb80 20 client.17400 populate_metadata read hostname 'smithi039' 2023-07-21T18:57:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.011+0000 7fcbcd1cfb80 1 --2- 172.21.15.39:0/2508214507 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55bdeb228640 0x55bdeb22aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.011+0000 7fcbcd1cfb80 4 client.17400 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:57:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.011+0000 7fcbcd1cfb80 1 -- 172.21.15.39:0/2508214507 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55bdeb0c6ff0 con 0x55bdeb228640 2023-07-21T18:57:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.011+0000 7fcbbad9d700 1 --2- 172.21.15.39:0/2508214507 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55bdeb228640 0x55bdeb22aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:55.029 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.011+0000 7fcbbad9d700 1 --2- 172.21.15.39:0/2508214507 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55bdeb228640 0x55bdeb22aa50 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=919213f3d4720165 server_cookie=a88f5a7614bd4233 in_seq=0 out_seq=0 2023-07-21T18:57:55.029 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.011+0000 7fcba27fc700 10 client.17400 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:57:55.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.173+0000 7fcba27fc700 1 -- 172.21.15.39:0/2508214507 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55bdeb0c6ff0 con 0x55bdeb228640 2023-07-21T18:57:55.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.173+0000 7fcba27fc700 10 client.17400 handle_command_reply: tid=0 2023-07-21T18:57:55.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.173+0000 7fcbcd1cfb80 1 client.17400 shutdown 2023-07-21T18:57:55.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.174+0000 7fcbcd1cfb80 1 -- 172.21.15.39:0/2508214507 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bdeb1ba580 msgr2=0x55bdeb1c7d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:55.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.174+0000 7fcbcd1cfb80 1 --2- 172.21.15.39:0/2508214507 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bdeb1ba580 0x55bdeb1c7d00 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fcba400b920 tx=0x7fcba400beb0 comp rx=0 tx=0).stop 2023-07-21T18:57:55.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.174+0000 7fcbcd1cfb80 1 -- 172.21.15.39:0/2508214507 shutdown_connections 2023-07-21T18:57:55.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.174+0000 7fcbcd1cfb80 1 --2- 172.21.15.39:0/2508214507 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55bdeb228640 0x55bdeb22aa50 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:55.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.174+0000 7fcbcd1cfb80 1 --2- 172.21.15.39:0/2508214507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bdeb1c02d0 0x55bdeb226aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:55.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.174+0000 7fcbcd1cfb80 1 --2- 172.21.15.39:0/2508214507 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bdeb0166b0 0x55bdeb1c77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:55.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.174+0000 7fcbcd1cfb80 1 --2- 172.21.15.39:0/2508214507 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bdeb1ba580 0x55bdeb1c7d00 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:55.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.174+0000 7fcbcd1cfb80 1 -- 172.21.15.39:0/2508214507 >> 172.21.15.39:0/2508214507 conn(0x55bdeb1afbe0 msgr2=0x55bdeb1c12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:57:55.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.174+0000 7fcba27fc700 0 client.17400 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:57:55.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.174+0000 7fcbcd1cfb80 1 -- 172.21.15.39:0/2508214507 shutdown_connections 2023-07-21T18:57:55.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.174+0000 7fcbcd1cfb80 1 -- 172.21.15.39:0/2508214507 wait complete. 2023-07-21T18:57:55.189 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.179+0000 7fcbcd1cfb80 20 client.17400 trim_cache size 0 max 16384 2023-07-21T18:57:55.189 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:57:55.190 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:57:55.190 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:57:55.190 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:57:55.190 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.180+0000 7fcbc1049700 1 -- 172.21.15.39:0/635047184 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fcba80a1340 msgr2=0x7fcba80a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:55.191 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.180+0000 7fcbc1049700 1 --2- 172.21.15.39:0/635047184 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fcba80a1340 0x7fcba80a37f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc19da80 tx=0x7fcba400b690 comp rx=0 tx=0).stop 2023-07-21T18:57:55.191 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.180+0000 7fcbc1049700 1 -- 172.21.15.39:0/635047184 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcbbc103ec0 msgr2=0x7fcbbc1a0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:55.191 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.180+0000 7fcbc1049700 1 --2- 172.21.15.39:0/635047184 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcbbc103ec0 0x7fcbbc1a0e10 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fcbac00efc0 tx=0x7fcbac00c3a0 comp rx=0 tx=0).stop 2023-07-21T18:57:55.191 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.180+0000 7fcbc1049700 1 -- 172.21.15.39:0/635047184 shutdown_connections 2023-07-21T18:57:55.191 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.180+0000 7fcbc1049700 1 --2- 172.21.15.39:0/635047184 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcbbc103ec0 0x7fcbbc1a0e10 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:55.191 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.180+0000 7fcbc1049700 1 --2- 172.21.15.39:0/635047184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcbbc0feba0 0x7fcbbc19c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:55.191 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.180+0000 7fcbc1049700 1 --2- 172.21.15.39:0/635047184 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fcba80a1340 0x7fcba80a37f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:55.192 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.180+0000 7fcbc1049700 1 --2- 172.21.15.39:0/635047184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcbbc0ff4b0 0x7fcbbc19c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:55.192 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.180+0000 7fcbc1049700 1 -- 172.21.15.39:0/635047184 >> 172.21.15.39:0/635047184 conn(0x7fcbbc075c20 msgr2=0x7fcbbc101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:57:55.192 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.182+0000 7fcbc1049700 1 -- 172.21.15.39:0/635047184 shutdown_connections 2023-07-21T18:57:55.192 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:57:55.182+0000 7fcbc1049700 1 -- 172.21.15.39:0/635047184 wait complete. 2023-07-21T18:57:55.675 INFO:tasks.workunit.client.0.smithi039.stdout: 10 110708 34.45 MB/sec execute 68 sec latency 116.247 ms 2023-07-21T18:57:56.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:57:56 smithi193 ceph-mon[98204]: pgmap v510: 97 pgs: 97 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 23 MiB/s wr, 807 op/s 2023-07-21T18:57:56.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:57:56 smithi155 ceph-mon[97934]: pgmap v510: 97 pgs: 97 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 23 MiB/s wr, 807 op/s 2023-07-21T18:57:56.675 INFO:tasks.workunit.client.0.smithi039.stdout: 10 111335 34.49 MB/sec execute 69 sec latency 76.968 ms 2023-07-21T18:57:56.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:57:56 smithi039 ceph-mon[96489]: pgmap v510: 97 pgs: 97 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 23 MiB/s wr, 807 op/s 2023-07-21T18:57:57.675 INFO:tasks.workunit.client.0.smithi039.stdout: 10 112093 34.53 MB/sec execute 70 sec latency 56.061 ms 2023-07-21T18:57:58.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:57:58 smithi193 ceph-mon[98204]: pgmap v511: 97 pgs: 97 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 25 MiB/s wr, 1.32k op/s 2023-07-21T18:57:58.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:57:58 smithi155 ceph-mon[97934]: pgmap v511: 97 pgs: 97 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 25 MiB/s wr, 1.32k op/s 2023-07-21T18:57:58.675 INFO:tasks.workunit.client.0.smithi039.stdout: 10 112779 34.60 MB/sec execute 71 sec latency 73.446 ms 2023-07-21T18:57:58.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:57:58 smithi039 ceph-mon[96489]: pgmap v511: 97 pgs: 97 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 25 MiB/s wr, 1.32k op/s 2023-07-21T18:57:59.675 INFO:tasks.workunit.client.0.smithi039.stdout: 10 113477 34.65 MB/sec execute 72 sec latency 47.342 ms 2023-07-21T18:58:00.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:58:00 smithi193 ceph-mon[98204]: pgmap v512: 97 pgs: 97 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 22 MiB/s wr, 993 op/s 2023-07-21T18:58:00.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:58:00 smithi155 ceph-mon[97934]: pgmap v512: 97 pgs: 97 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 22 MiB/s wr, 993 op/s 2023-07-21T18:58:00.675 INFO:tasks.workunit.client.0.smithi039.stdout: 10 113769 34.37 MB/sec execute 73 sec latency 643.703 ms 2023-07-21T18:58:00.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:58:00 smithi039 ceph-mon[96489]: pgmap v512: 97 pgs: 97 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 22 MiB/s wr, 993 op/s 2023-07-21T18:58:01.675 INFO:tasks.workunit.client.0.smithi039.stdout: 10 113840 33.92 MB/sec execute 74 sec latency 1096.404 ms 2023-07-21T18:58:02.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:58:02 smithi193 ceph-mon[98204]: pgmap v513: 97 pgs: 97 active+clean; 1.2 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 25 MiB/s wr, 1.24k op/s 2023-07-21T18:58:02.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:58:02 smithi155 ceph-mon[97934]: pgmap v513: 97 pgs: 97 active+clean; 1.2 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 25 MiB/s wr, 1.24k op/s 2023-07-21T18:58:02.675 INFO:tasks.workunit.client.0.smithi039.stdout: 10 113840 33.47 MB/sec execute 75 sec latency 1431.459 ms 2023-07-21T18:58:02.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:58:02 smithi039 ceph-mon[96489]: pgmap v513: 97 pgs: 97 active+clean; 1.2 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 25 MiB/s wr, 1.24k op/s 2023-07-21T18:58:03.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:58:03 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:58:03.675 INFO:tasks.workunit.client.0.smithi039.stdout: 10 113946 33.05 MB/sec execute 76 sec latency 2215.006 ms 2023-07-21T18:58:03.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:58:03 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:58:03.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:58:03 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:58:04.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:58:04 smithi155 ceph-mon[97934]: pgmap v514: 97 pgs: 97 active+clean; 1.2 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 4.9 MiB/s wr, 754 op/s 2023-07-21T18:58:04.675 INFO:tasks.workunit.client.0.smithi039.stdout: 10 114394 32.89 MB/sec execute 77 sec latency 105.284 ms 2023-07-21T18:58:04.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:58:04 smithi039 ceph-mon[96489]: pgmap v514: 97 pgs: 97 active+clean; 1.2 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 4.9 MiB/s wr, 754 op/s 2023-07-21T18:58:04.859 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:58:04 smithi193 ceph-mon[98204]: pgmap v514: 97 pgs: 97 active+clean; 1.2 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 4.9 MiB/s wr, 754 op/s 2023-07-21T18:58:05.675 INFO:tasks.workunit.client.0.smithi039.stdout: 10 114831 32.91 MB/sec execute 78 sec latency 115.490 ms 2023-07-21T18:58:06.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:58:06 smithi155 ceph-mon[97934]: pgmap v515: 97 pgs: 97 active+clean; 1.2 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 4.9 MiB/s wr, 754 op/s 2023-07-21T18:58:06.675 INFO:tasks.workunit.client.0.smithi039.stdout: 10 115529 32.98 MB/sec execute 79 sec latency 88.009 ms 2023-07-21T18:58:06.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:58:06 smithi039 ceph-mon[96489]: pgmap v515: 97 pgs: 97 active+clean; 1.2 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 4.9 MiB/s wr, 754 op/s 2023-07-21T18:58:06.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:58:06 smithi193 ceph-mon[98204]: pgmap v515: 97 pgs: 97 active+clean; 1.2 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 4.9 MiB/s wr, 754 op/s 2023-07-21T18:58:07.675 INFO:tasks.workunit.client.0.smithi039.stdout: 10 116124 32.94 MB/sec execute 80 sec latency 424.070 ms 2023-07-21T18:58:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.350+0000 7f3070d88700 1 -- 172.21.15.39:0/2843937388 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f306c104e20 msgr2=0x7f306c10ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.350+0000 7f3070d88700 1 --2- 172.21.15.39:0/2843937388 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f306c104e20 0x7f306c10ca30 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f306400b0d0 tx=0x7f306400b3e0 comp rx=0 tx=0).stop 2023-07-21T18:58:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.350+0000 7f3070d88700 1 -- 172.21.15.39:0/2843937388 shutdown_connections 2023-07-21T18:58:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.350+0000 7f3070d88700 1 --2- 172.21.15.39:0/2843937388 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f306c104e20 0x7f306c10ca30 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.350+0000 7f3070d88700 1 --2- 172.21.15.39:0/2843937388 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f306c0fe550 0x7f306c0fe920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:08.363 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.350+0000 7f3070d88700 1 --2- 172.21.15.39:0/2843937388 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f306c0feef0 0x7f306c104740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:08.363 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.350+0000 7f3070d88700 1 -- 172.21.15.39:0/2843937388 >> 172.21.15.39:0/2843937388 conn(0x7f306c075060 msgr2=0x7f306c075460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:58:08.363 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.352+0000 7f3070d88700 1 -- 172.21.15.39:0/2843937388 shutdown_connections 2023-07-21T18:58:08.363 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.352+0000 7f3070d88700 1 -- 172.21.15.39:0/2843937388 wait complete. 2023-07-21T18:58:08.364 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.353+0000 7f3070d88700 1 Processor -- start 2023-07-21T18:58:08.364 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.353+0000 7f3070d88700 1 -- start start 2023-07-21T18:58:08.364 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.353+0000 7f3070d88700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f306c0fe550 0x7f306c10c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:08.364 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.353+0000 7f3070d88700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f306c0feef0 0x7f306c1093d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:08.364 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.353+0000 7f3070d88700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f306c104e20 0x7f306c109910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:08.364 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.353+0000 7f3070d88700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f306c109f10 con 0x7f306c104e20 2023-07-21T18:58:08.365 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.353+0000 7f3070d88700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f306c10a080 con 0x7f306c0fe550 2023-07-21T18:58:08.365 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.353+0000 7f3070d88700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f306c10a240 con 0x7f306c0feef0 2023-07-21T18:58:08.365 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.353+0000 7f3069d9b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f306c0feef0 0x7f306c1093d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.353+0000 7f3069d9b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f306c0feef0 0x7f306c1093d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:52166/0 (socket says 172.21.15.39:52166) 2023-07-21T18:58:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.353+0000 7f3069d9b700 1 -- 172.21.15.39:0/1540821251 learned_addr learned my addr 172.21.15.39:0/1540821251 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:58:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.353+0000 7f306ad9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f306c104e20 0x7f306c109910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.354+0000 7f3069d9b700 1 -- 172.21.15.39:0/1540821251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f306c0fe550 msgr2=0x7f306c10c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.354+0000 7f3069d9b700 1 --2- 172.21.15.39:0/1540821251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f306c0fe550 0x7f306c10c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.354+0000 7f3069d9b700 1 -- 172.21.15.39:0/1540821251 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f306c104e20 msgr2=0x7f306c109910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.354+0000 7f3069d9b700 1 --2- 172.21.15.39:0/1540821251 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f306c104e20 0x7f306c109910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.354+0000 7f3069d9b700 1 -- 172.21.15.39:0/1540821251 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3064009d00 con 0x7f306c0feef0 2023-07-21T18:58:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.354+0000 7f3069d9b700 1 --2- 172.21.15.39:0/1540821251 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f306c0feef0 0x7f306c1093d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f306000e940 tx=0x7f306000ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:58:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.354+0000 7f305b7fe700 1 -- 172.21.15.39:0/1540821251 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3060009730 con 0x7f306c0feef0 2023-07-21T18:58:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.354+0000 7f305b7fe700 1 -- 172.21.15.39:0/1540821251 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f306000ee90 con 0x7f306c0feef0 2023-07-21T18:58:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.354+0000 7f3070d88700 1 -- 172.21.15.39:0/1540821251 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f306c19e6b0 con 0x7f306c0feef0 2023-07-21T18:58:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.354+0000 7f305b7fe700 1 -- 172.21.15.39:0/1540821251 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3060010ea0 con 0x7f306c0feef0 2023-07-21T18:58:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.354+0000 7f3070d88700 1 -- 172.21.15.39:0/1540821251 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f306c19ebe0 con 0x7f306c0feef0 2023-07-21T18:58:08.368 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.357+0000 7f305b7fe700 1 -- 172.21.15.39:0/1540821251 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f306000cbd0 con 0x7f306c0feef0 2023-07-21T18:58:08.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.358+0000 7f305b7fe700 1 --2- 172.21.15.39:0/1540821251 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f30540a1340 0x7f30540a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:08.370 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.359+0000 7f305b7fe700 1 -- 172.21.15.39:0/1540821251 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f3060014070 con 0x7f306c0feef0 2023-07-21T18:58:08.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.360+0000 7f307cf0eb80 1 Processor -- start 2023-07-21T18:58:08.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.360+0000 7f306a59c700 1 --2- 172.21.15.39:0/1540821251 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f30540a1340 0x7f30540a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:08.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.360+0000 7f306a59c700 1 --2- 172.21.15.39:0/1540821251 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f30540a1340 0x7f30540a37f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f305c005950 tx=0x7f305c0058e0 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:58:08.373 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.361+0000 7f307cf0eb80 1 -- start start 2023-07-21T18:58:08.373 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.361+0000 7f307cf0eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b9fca4c6b0 0x55b9fcbf5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:08.374 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.361+0000 7f307cf0eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9fcbf62d0 0x55b9fcbf0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:08.374 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.361+0000 7f307cf0eb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b9fcbf0580 0x55b9fcbf09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:08.374 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.361+0000 7f307cf0eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9fcaf5c50 con 0x55b9fca4c6b0 2023-07-21T18:58:08.374 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.361+0000 7f307cf0eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9fcb489e0 con 0x55b9fcbf62d0 2023-07-21T18:58:08.375 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.361+0000 7f307cf0eb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9fcb41e70 con 0x55b9fcbf0580 2023-07-21T18:58:08.375 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.362+0000 7f306a59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b9fca4c6b0 0x55b9fcbf5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:08.375 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.362+0000 7f306a59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b9fca4c6b0 0x55b9fcbf5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33620/0 (socket says 172.21.15.39:33620) 2023-07-21T18:58:08.375 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.362+0000 7f3069d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9fcbf62d0 0x55b9fcbf0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:08.376 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.362+0000 7f306a59c700 1 -- 172.21.15.39:0/3999417841 learned_addr learned my addr 172.21.15.39:0/3999417841 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:58:08.376 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.362+0000 7f306ad9d700 1 --2- 172.21.15.39:0/3999417841 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b9fcbf0580 0x55b9fcbf09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:08.376 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.362+0000 7f306a59c700 1 -- 172.21.15.39:0/3999417841 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b9fcbf0580 msgr2=0x55b9fcbf09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:58:08.376 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.362+0000 7f306a59c700 1 --2- 172.21.15.39:0/3999417841 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b9fcbf0580 0x55b9fcbf09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:08.376 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.362+0000 7f306a59c700 1 -- 172.21.15.39:0/3999417841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9fcbf62d0 msgr2=0x55b9fcbf0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:58:08.377 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.362+0000 7f306a59c700 1 --2- 172.21.15.39:0/3999417841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9fcbf62d0 0x55b9fcbf0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:08.377 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.362+0000 7f306a59c700 1 -- 172.21.15.39:0/3999417841 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b9fc96fd80 con 0x55b9fca4c6b0 2023-07-21T18:58:08.377 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.362+0000 7f306a59c700 1 --2- 172.21.15.39:0/3999417841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b9fca4c6b0 0x55b9fcbf5d90 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f305c00ebc0 tx=0x7f305c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33ef7f361154202e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:58:08.377 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.362+0000 7f3058ff9700 1 -- 172.21.15.39:0/3999417841 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f305c010ac0 con 0x55b9fca4c6b0 2023-07-21T18:58:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.363+0000 7f3058ff9700 1 -- 172.21.15.39:0/3999417841 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f305c009820 con 0x55b9fca4c6b0 2023-07-21T18:58:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.363+0000 7f3058ff9700 1 -- 172.21.15.39:0/3999417841 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f305c01c460 con 0x55b9fca4c6b0 2023-07-21T18:58:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.363+0000 7f307cf0eb80 1 -- 172.21.15.39:0/3999417841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b9fca4c6b0 msgr2=0x55b9fcbf5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.363+0000 7f307cf0eb80 1 --2- 172.21.15.39:0/3999417841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b9fca4c6b0 0x55b9fcbf5d90 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f305c00ebc0 tx=0x7f305c011000 comp rx=0 tx=0).stop 2023-07-21T18:58:08.380 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.363+0000 7f307cf0eb80 1 -- 172.21.15.39:0/3999417841 shutdown_connections 2023-07-21T18:58:08.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.363+0000 7f307cf0eb80 1 --2- 172.21.15.39:0/3999417841 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b9fcbf0580 0x55b9fcbf09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:08.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.363+0000 7f307cf0eb80 1 --2- 172.21.15.39:0/3999417841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b9fca4c6b0 0x55b9fcbf5d90 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:08.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.363+0000 7f307cf0eb80 1 --2- 172.21.15.39:0/3999417841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9fcbf62d0 0x55b9fcbf0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:08.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.363+0000 7f307cf0eb80 1 -- 172.21.15.39:0/3999417841 >> 172.21.15.39:0/3999417841 conn(0x55b9fcbe5be0 msgr2=0x55b9fcbeaf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:58:08.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.364+0000 7f307cf0eb80 1 -- 172.21.15.39:0/3999417841 shutdown_connections 2023-07-21T18:58:08.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.364+0000 7f307cf0eb80 1 -- 172.21.15.39:0/3999417841 wait complete. 2023-07-21T18:58:08.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.365+0000 7f307cf0eb80 1 -- start start 2023-07-21T18:58:08.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.365+0000 7f307cf0eb80 1 Processor -- start 2023-07-21T18:58:08.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.365+0000 7f307cf0eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9fca4c6b0 0x55b9fcbf9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:08.383 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.365+0000 7f307cf0eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b9fcbf0580 0x55b9fcbf9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:08.383 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.365+0000 7f307cf0eb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b9fcbf62d0 0x55b9fcbfa310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:08.383 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.365+0000 7f307cf0eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9fcae1a60 con 0x55b9fcbf0580 2023-07-21T18:58:08.383 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.365+0000 7f307cf0eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9fcae1760 con 0x55b9fca4c6b0 2023-07-21T18:58:08.383 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.365+0000 7f307cf0eb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9fc9c58c0 con 0x55b9fcbf62d0 2023-07-21T18:58:08.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.365+0000 7f307cf0eb80 10 client.0 _ll_register_callbacks cb 0x55b9fcadf700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:58:08.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.366+0000 7f306a59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9fca4c6b0 0x55b9fcbf9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:08.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.366+0000 7f306a59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9fca4c6b0 0x55b9fcbf9890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:46224/0 (socket says 172.21.15.39:46224) 2023-07-21T18:58:08.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.366+0000 7f306a59c700 1 -- 172.21.15.39:0/684935374 learned_addr learned my addr 172.21.15.39:0/684935374 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:58:08.385 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.366+0000 7f306ad9d700 1 --2- 172.21.15.39:0/684935374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b9fcbf62d0 0x55b9fcbfa310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:08.385 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.366+0000 7f306a59c700 1 -- 172.21.15.39:0/684935374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b9fcbf62d0 msgr2=0x55b9fcbfa310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:58:08.385 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.366+0000 7f306a59c700 1 --2- 172.21.15.39:0/684935374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b9fcbf62d0 0x55b9fcbfa310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:08.385 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.366+0000 7f306a59c700 1 -- 172.21.15.39:0/684935374 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b9fcbf0580 msgr2=0x55b9fcbf9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:58:08.385 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.366+0000 7f306a59c700 1 --2- 172.21.15.39:0/684935374 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b9fcbf0580 0x55b9fcbf9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:08.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.366+0000 7f306a59c700 1 -- 172.21.15.39:0/684935374 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f305c00e870 con 0x55b9fca4c6b0 2023-07-21T18:58:08.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.366+0000 7f306a59c700 1 --2- 172.21.15.39:0/684935374 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9fca4c6b0 0x55b9fcbf9890 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f305c009a50 tx=0x7f305c025ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddcd54632f8cc344 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:58:08.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.366+0000 7f3051ffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-21T18:58:08.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.366+0000 7f3051ffb700 1 -- 172.21.15.39:0/684935374 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f305c02b070 con 0x55b9fca4c6b0 2023-07-21T18:58:08.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.367+0000 7f3051ffb700 1 -- 172.21.15.39:0/684935374 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f305c010a80 con 0x55b9fca4c6b0 2023-07-21T18:58:08.387 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.367+0000 7f307cf0eb80 1 -- 172.21.15.39:0/684935374 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b9fcb3ae50 con 0x55b9fca4c6b0 2023-07-21T18:58:08.387 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.367+0000 7f3051ffb700 1 -- 172.21.15.39:0/684935374 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f305c010c40 con 0x55b9fca4c6b0 2023-07-21T18:58:08.387 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.367+0000 7f307cf0eb80 10 client.26834 fetch_fsmap learned FSMap version 91 2023-07-21T18:58:08.387 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.367+0000 7f307cf0eb80 1 -- 172.21.15.39:0/684935374 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=4+}) v3 -- 0x55b9fc958c00 con 0x55b9fca4c6b0 2023-07-21T18:58:08.387 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.368+0000 7f3051ffb700 1 -- 172.21.15.39:0/684935374 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f305c03e020 con 0x55b9fca4c6b0 2023-07-21T18:58:08.388 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.368+0000 7f3051ffb700 1 -- 172.21.15.39:0/684935374 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 91) v1 ==== 7510+0+0 (secure 0 0 0) 0x7f305c018070 con 0x55b9fca4c6b0 2023-07-21T18:58:08.388 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.368+0000 7f307cf0eb80 10 client.26834 fetch_fsmap finished waiting for FSMap version 91 2023-07-21T18:58:08.388 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.368+0000 7f307cf0eb80 10 client.26834 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:58:08.388 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.368+0000 7f307cf0eb80 20 client.26834 populate_metadata read hostname 'smithi039' 2023-07-21T18:58:08.388 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.368+0000 7f307cf0eb80 1 --2- 172.21.15.39:0/684935374 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55b9fcc2bb10 0x55b9fcc2df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:08.389 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.368+0000 7f307cf0eb80 4 client.26834 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:58:08.389 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.368+0000 7f307cf0eb80 1 -- 172.21.15.39:0/684935374 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b9fcafcff0 con 0x55b9fcc2bb10 2023-07-21T18:58:08.389 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.368+0000 7f3069d9b700 1 --2- 172.21.15.39:0/684935374 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55b9fcc2bb10 0x55b9fcc2df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:08.389 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.369+0000 7f3051ffb700 10 client.26834 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:58:08.389 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.369+0000 7f3069d9b700 1 --2- 172.21.15.39:0/684935374 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55b9fcc2bb10 0x55b9fcc2df60 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7494a8d7c58be99 server_cookie=1ee10dd71723a81f in_seq=0 out_seq=0 2023-07-21T18:58:08.391 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.371+0000 7f3051ffb700 1 -- 172.21.15.39:0/684935374 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b9fcafcff0 con 0x55b9fcc2bb10 2023-07-21T18:58:08.391 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.371+0000 7f3051ffb700 10 client.26834 handle_command_reply: tid=0 2023-07-21T18:58:08.391 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.371+0000 7f307cf0eb80 1 client.26834 shutdown 2023-07-21T18:58:08.391 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.371+0000 7f307cf0eb80 1 -- 172.21.15.39:0/684935374 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9fca4c6b0 msgr2=0x55b9fcbf9890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:08.392 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.371+0000 7f307cf0eb80 1 --2- 172.21.15.39:0/684935374 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9fca4c6b0 0x55b9fcbf9890 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f305c009a50 tx=0x7f305c025ae0 comp rx=0 tx=0).stop 2023-07-21T18:58:08.392 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.371+0000 7f307cf0eb80 1 -- 172.21.15.39:0/684935374 shutdown_connections 2023-07-21T18:58:08.392 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.371+0000 7f307cf0eb80 1 --2- 172.21.15.39:0/684935374 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55b9fcc2bb10 0x55b9fcc2df60 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:08.392 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.371+0000 7f307cf0eb80 1 --2- 172.21.15.39:0/684935374 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9fca4c6b0 0x55b9fcbf9890 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.371+0000 7f3051ffb700 0 client.26834 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:58:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.371+0000 7f307cf0eb80 1 --2- 172.21.15.39:0/684935374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b9fcbf62d0 0x55b9fcbfa310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.371+0000 7f307cf0eb80 1 --2- 172.21.15.39:0/684935374 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b9fcbf0580 0x55b9fcbf9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.371+0000 7f307cf0eb80 1 -- 172.21.15.39:0/684935374 >> 172.21.15.39:0/684935374 conn(0x55b9fcbe5be0 msgr2=0x55b9fcbea4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:58:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.371+0000 7f307cf0eb80 1 -- 172.21.15.39:0/684935374 shutdown_connections 2023-07-21T18:58:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.372+0000 7f307cf0eb80 1 -- 172.21.15.39:0/684935374 wait complete. 2023-07-21T18:58:08.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.372+0000 7f307cf0eb80 20 client.26834 trim_cache size 0 max 16384 2023-07-21T18:58:08.410 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.399+0000 7f307cf0eb80 1 Processor -- start 2023-07-21T18:58:08.410 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.399+0000 7f307cf0eb80 1 -- start start 2023-07-21T18:58:08.411 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.399+0000 7f307cf0eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b9fca4c6b0 0x55b9fcbf0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:08.411 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.399+0000 7f307cf0eb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b9fcbf0580 0x55b9fcbf8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:08.411 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.400+0000 7f307cf0eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9fcbf62d0 0x55b9fcc00c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:08.411 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.400+0000 7f307cf0eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9fcbf1060 con 0x55b9fca4c6b0 2023-07-21T18:58:08.412 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.400+0000 7f307cf0eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9fcbf7940 con 0x55b9fcbf62d0 2023-07-21T18:58:08.412 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.400+0000 7f307cf0eb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9fcbf46b0 con 0x55b9fcbf0580 2023-07-21T18:58:08.412 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.400+0000 7f3069d9b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b9fcbf0580 0x55b9fcbf8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:08.412 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.400+0000 7f3069d9b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b9fcbf0580 0x55b9fcbf8b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:52188/0 (socket says 172.21.15.39:52188) 2023-07-21T18:58:08.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.400+0000 7f3069d9b700 1 -- 172.21.15.39:0/806532741 learned_addr learned my addr 172.21.15.39:0/806532741 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:58:08.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.400+0000 7f306ad9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9fcbf62d0 0x55b9fcc00c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:08.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.400+0000 7f306a59c700 1 --2- 172.21.15.39:0/806532741 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b9fca4c6b0 0x55b9fcbf0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:08.418 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.400+0000 7f306ad9d700 1 -- 172.21.15.39:0/806532741 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b9fcbf0580 msgr2=0x55b9fcbf8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:58:08.418 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.400+0000 7f306ad9d700 1 --2- 172.21.15.39:0/806532741 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b9fcbf0580 0x55b9fcbf8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:08.418 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.400+0000 7f306ad9d700 1 -- 172.21.15.39:0/806532741 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b9fca4c6b0 msgr2=0x55b9fcbf0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:58:08.418 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.400+0000 7f306ad9d700 1 --2- 172.21.15.39:0/806532741 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b9fca4c6b0 0x55b9fcbf0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:08.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.400+0000 7f306ad9d700 1 -- 172.21.15.39:0/806532741 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b9fc9961f0 con 0x55b9fcbf62d0 2023-07-21T18:58:08.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.401+0000 7f306ad9d700 1 --2- 172.21.15.39:0/806532741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9fcbf62d0 0x55b9fcc00c60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3064009fd0 tx=0x7f30640092e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c3469d67951c746 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:58:08.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.401+0000 7f30597fa700 1 -- 172.21.15.39:0/806532741 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3064021070 con 0x55b9fcbf62d0 2023-07-21T18:58:08.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.401+0000 7f30597fa700 1 -- 172.21.15.39:0/806532741 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f30640093d0 con 0x55b9fcbf62d0 2023-07-21T18:58:08.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.401+0000 7f30597fa700 1 -- 172.21.15.39:0/806532741 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306400e040 con 0x55b9fcbf62d0 2023-07-21T18:58:08.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.402+0000 7f307cf0eb80 1 -- 172.21.15.39:0/806532741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9fcbf62d0 msgr2=0x55b9fcc00c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:08.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.402+0000 7f307cf0eb80 1 --2- 172.21.15.39:0/806532741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9fcbf62d0 0x55b9fcc00c60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3064009fd0 tx=0x7f30640092e0 comp rx=0 tx=0).stop 2023-07-21T18:58:08.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.402+0000 7f307cf0eb80 1 -- 172.21.15.39:0/806532741 shutdown_connections 2023-07-21T18:58:08.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.402+0000 7f307cf0eb80 1 --2- 172.21.15.39:0/806532741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9fcbf62d0 0x55b9fcc00c60 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:08.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.402+0000 7f307cf0eb80 1 --2- 172.21.15.39:0/806532741 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b9fca4c6b0 0x55b9fcbf0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:08.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.402+0000 7f307cf0eb80 1 --2- 172.21.15.39:0/806532741 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b9fcbf0580 0x55b9fcbf8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:08.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.402+0000 7f307cf0eb80 1 -- 172.21.15.39:0/806532741 >> 172.21.15.39:0/806532741 conn(0x55b9fcbe5be0 msgr2=0x55b9fcbf4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:58:08.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.402+0000 7f307cf0eb80 1 -- 172.21.15.39:0/806532741 shutdown_connections 2023-07-21T18:58:08.421 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.402+0000 7f307cf0eb80 1 -- 172.21.15.39:0/806532741 wait complete. 2023-07-21T18:58:08.421 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.402+0000 7f307cf0eb80 1 -- start start 2023-07-21T18:58:08.421 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.403+0000 7f307cf0eb80 1 Processor -- start 2023-07-21T18:58:08.421 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.403+0000 7f307cf0eb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b9fca4c6b0 0x55b9fcc00390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:08.421 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.403+0000 7f307cf0eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b9fcbf0580 0x55b9fcc008d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:08.421 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.403+0000 7f307cf0eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9fcbf62d0 0x55b9fcbfe750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:08.421 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.403+0000 7f307cf0eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9fcbfedd0 con 0x55b9fcbf0580 2023-07-21T18:58:08.421 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.403+0000 7f307cf0eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9fcbfef40 con 0x55b9fcbf62d0 2023-07-21T18:58:08.422 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.403+0000 7f307cf0eb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9fcbff100 con 0x55b9fca4c6b0 2023-07-21T18:58:08.422 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.403+0000 7f307cf0eb80 10 client.0 _ll_register_callbacks cb 0x7f30600962b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:58:08.422 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.403+0000 7f306ad9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9fcbf62d0 0x55b9fcbfe750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:08.422 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.403+0000 7f306ad9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9fcbf62d0 0x55b9fcbfe750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:46238/0 (socket says 172.21.15.39:46238) 2023-07-21T18:58:08.422 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.403+0000 7f306ad9d700 1 -- 172.21.15.39:0/242531750 learned_addr learned my addr 172.21.15.39:0/242531750 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:58:08.422 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.404+0000 7f3069d9b700 1 --2- 172.21.15.39:0/242531750 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b9fcbf0580 0x55b9fcc008d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:08.422 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.404+0000 7f306ad9d700 1 -- 172.21.15.39:0/242531750 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b9fca4c6b0 msgr2=0x55b9fcc00390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:58:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.404+0000 7f306ad9d700 1 --2- 172.21.15.39:0/242531750 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b9fca4c6b0 0x55b9fcc00390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.404+0000 7f306ad9d700 1 -- 172.21.15.39:0/242531750 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b9fcbf0580 msgr2=0x55b9fcc008d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:58:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.404+0000 7f306ad9d700 1 --2- 172.21.15.39:0/242531750 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b9fcbf0580 0x55b9fcc008d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.404+0000 7f306ad9d700 1 -- 172.21.15.39:0/242531750 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f306c10d270 con 0x55b9fcbf62d0 2023-07-21T18:58:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.404+0000 7f306ad9d700 1 --2- 172.21.15.39:0/242531750 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9fcbf62d0 0x55b9fcbfe750 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f306400b010 tx=0x7f3064007a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20662ae9257e46e2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:58:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.404+0000 7f3051ffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-21T18:58:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.404+0000 7f3051ffb700 1 -- 172.21.15.39:0/242531750 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3064021030 con 0x55b9fcbf62d0 2023-07-21T18:58:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.404+0000 7f307cf0eb80 1 -- 172.21.15.39:0/242531750 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b9fca9b490 con 0x55b9fcbf62d0 2023-07-21T18:58:08.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.404+0000 7f3051ffb700 1 -- 172.21.15.39:0/242531750 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f3064022070 con 0x55b9fcbf62d0 2023-07-21T18:58:08.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.405+0000 7f3051ffb700 1 -- 172.21.15.39:0/242531750 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f306400e040 con 0x55b9fcbf62d0 2023-07-21T18:58:08.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.405+0000 7f307cf0eb80 10 client.26840 fetch_fsmap learned FSMap version 91 2023-07-21T18:58:08.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.405+0000 7f307cf0eb80 1 -- 172.21.15.39:0/242531750 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=4+}) v3 -- 0x55b9fc96e250 con 0x55b9fcbf62d0 2023-07-21T18:58:08.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.405+0000 7f3051ffb700 1 -- 172.21.15.39:0/242531750 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b9fca9b490 con 0x55b9fcbf62d0 2023-07-21T18:58:08.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.405+0000 7f3051ffb700 1 -- 172.21.15.39:0/242531750 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 91) v1 ==== 7510+0+0 (secure 0 0 0) 0x7f3064035070 con 0x55b9fcbf62d0 2023-07-21T18:58:08.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.405+0000 7f307cf0eb80 10 client.26840 fetch_fsmap finished waiting for FSMap version 91 2023-07-21T18:58:08.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.405+0000 7f307cf0eb80 10 client.26840 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:58:08.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.405+0000 7f307cf0eb80 20 client.26840 populate_metadata read hostname 'smithi039' 2023-07-21T18:58:08.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.405+0000 7f307cf0eb80 1 --2- 172.21.15.39:0/242531750 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55b9fcc808d0 0x55b9fcc82ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:08.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.405+0000 7f307cf0eb80 4 client.26840 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:58:08.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.405+0000 7f307cf0eb80 1 -- 172.21.15.39:0/242531750 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b9fc990f10 con 0x55b9fcc808d0 2023-07-21T18:58:08.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.405+0000 7f306a59c700 1 --2- 172.21.15.39:0/242531750 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55b9fcc808d0 0x55b9fcc82ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:08.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.406+0000 7f306a59c700 1 --2- 172.21.15.39:0/242531750 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55b9fcc808d0 0x55b9fcc82ce0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b4eb9d5eddaf9fe server_cookie=9ecef5e9cfac4215 in_seq=0 out_seq=0 2023-07-21T18:58:08.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.406+0000 7f3051ffb700 10 client.26840 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:58:08.571 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.561+0000 7f3051ffb700 1 -- 172.21.15.39:0/242531750 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b9fc990f10 con 0x55b9fcc808d0 2023-07-21T18:58:08.571 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.561+0000 7f3051ffb700 10 client.26840 handle_command_reply: tid=0 2023-07-21T18:58:08.571 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.561+0000 7f307cf0eb80 1 client.26840 shutdown 2023-07-21T18:58:08.573 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.562+0000 7f307cf0eb80 1 -- 172.21.15.39:0/242531750 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9fcbf62d0 msgr2=0x55b9fcbfe750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:08.573 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.562+0000 7f307cf0eb80 1 --2- 172.21.15.39:0/242531750 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9fcbf62d0 0x55b9fcbfe750 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f306400b010 tx=0x7f3064007a40 comp rx=0 tx=0).stop 2023-07-21T18:58:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.563+0000 7f307cf0eb80 1 -- 172.21.15.39:0/242531750 shutdown_connections 2023-07-21T18:58:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.563+0000 7f307cf0eb80 1 --2- 172.21.15.39:0/242531750 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55b9fcc808d0 0x55b9fcc82ce0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.563+0000 7f307cf0eb80 1 --2- 172.21.15.39:0/242531750 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9fcbf62d0 0x55b9fcbfe750 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.563+0000 7f307cf0eb80 1 --2- 172.21.15.39:0/242531750 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b9fca4c6b0 0x55b9fcc00390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.563+0000 7f307cf0eb80 1 --2- 172.21.15.39:0/242531750 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b9fcbf0580 0x55b9fcc008d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:58:08.575 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:58:08.575 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:58:08.575 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:58:08.576 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.563+0000 7f307cf0eb80 1 -- 172.21.15.39:0/242531750 >> 172.21.15.39:0/242531750 conn(0x55b9fcbe5be0 msgr2=0x55b9fcbf9b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:58:08.576 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.563+0000 7f3051ffb700 0 client.26840 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:58:08.576 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.563+0000 7f307cf0eb80 1 -- 172.21.15.39:0/242531750 shutdown_connections 2023-07-21T18:58:08.576 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.563+0000 7f307cf0eb80 1 -- 172.21.15.39:0/242531750 wait complete. 2023-07-21T18:58:08.576 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.563+0000 7f307cf0eb80 20 client.26840 trim_cache size 0 max 16384 2023-07-21T18:58:08.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.564+0000 7f3070d88700 1 -- 172.21.15.39:0/1540821251 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f30540a1340 msgr2=0x7f30540a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:08.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.564+0000 7f3070d88700 1 --2- 172.21.15.39:0/1540821251 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f30540a1340 0x7f30540a37f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f305c005950 tx=0x7f305c0058e0 comp rx=0 tx=0).stop 2023-07-21T18:58:08.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.564+0000 7f3070d88700 1 -- 172.21.15.39:0/1540821251 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f306c0feef0 msgr2=0x7f306c1093d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:08.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.564+0000 7f3070d88700 1 --2- 172.21.15.39:0/1540821251 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f306c0feef0 0x7f306c1093d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f306000e940 tx=0x7f306000ec50 comp rx=0 tx=0).stop 2023-07-21T18:58:08.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.564+0000 7f3070d88700 1 -- 172.21.15.39:0/1540821251 shutdown_connections 2023-07-21T18:58:08.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.564+0000 7f3070d88700 1 --2- 172.21.15.39:0/1540821251 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f30540a1340 0x7f30540a37f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:08.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.564+0000 7f3070d88700 1 --2- 172.21.15.39:0/1540821251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f306c0fe550 0x7f306c10c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:08.578 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.564+0000 7f3070d88700 1 --2- 172.21.15.39:0/1540821251 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f306c104e20 0x7f306c109910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:08.578 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.564+0000 7f3070d88700 1 --2- 172.21.15.39:0/1540821251 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f306c0feef0 0x7f306c1093d0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:08.578 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.564+0000 7f3070d88700 1 -- 172.21.15.39:0/1540821251 >> 172.21.15.39:0/1540821251 conn(0x7f306c075060 msgr2=0x7f306c0fbd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:58:08.578 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.564+0000 7f3070d88700 1 -- 172.21.15.39:0/1540821251 shutdown_connections 2023-07-21T18:58:08.578 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:08.564+0000 7f3070d88700 1 -- 172.21.15.39:0/1540821251 wait complete. 2023-07-21T18:58:08.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:58:08 smithi039 ceph-mon[96489]: pgmap v516: 97 pgs: 97 active+clean; 1.2 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.0 MiB/s wr, 1.02k op/s 2023-07-21T18:58:08.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:58:08 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:58:08.676 INFO:tasks.workunit.client.0.smithi039.stdout: 10 116703 33.02 MB/sec execute 81 sec latency 1424.316 ms 2023-07-21T18:58:08.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:58:08 smithi193 ceph-mon[98204]: pgmap v516: 97 pgs: 97 active+clean; 1.2 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.0 MiB/s wr, 1.02k op/s 2023-07-21T18:58:08.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:58:08 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:58:08.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:58:08 smithi155 ceph-mon[97934]: pgmap v516: 97 pgs: 97 active+clean; 1.2 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.0 MiB/s wr, 1.02k op/s 2023-07-21T18:58:08.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:58:08 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:58:09.676 INFO:tasks.workunit.client.0.smithi039.stdout: 10 117143 32.81 MB/sec execute 82 sec latency 2424.467 ms 2023-07-21T18:58:10.676 INFO:tasks.workunit.client.0.smithi039.stdout: 10 117389 32.59 MB/sec execute 83 sec latency 3424.506 ms 2023-07-21T18:58:10.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:58:10 smithi039 ceph-mon[96489]: pgmap v517: 97 pgs: 97 active+clean; 1.2 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.6 MiB/s wr, 512 op/s 2023-07-21T18:58:10.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:58:10 smithi039 ceph-mon[96489]: osdmap e105: 12 total, 12 up, 12 in 2023-07-21T18:58:10.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:58:10 smithi193 ceph-mon[98204]: pgmap v517: 97 pgs: 97 active+clean; 1.2 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.6 MiB/s wr, 512 op/s 2023-07-21T18:58:10.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:58:10 smithi193 ceph-mon[98204]: osdmap e105: 12 total, 12 up, 12 in 2023-07-21T18:58:10.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:58:10 smithi155 ceph-mon[97934]: pgmap v517: 97 pgs: 97 active+clean; 1.2 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.6 MiB/s wr, 512 op/s 2023-07-21T18:58:10.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:58:10 smithi155 ceph-mon[97934]: osdmap e105: 12 total, 12 up, 12 in 2023-07-21T18:58:11.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:58:11 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:58:11.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:58:11 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:58:11.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:58:11 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:58:11.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:58:11 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:58:11.676 INFO:tasks.workunit.client.0.smithi039.stdout: 10 117861 32.60 MB/sec execute 84 sec latency 4424.537 ms 2023-07-21T18:58:11.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:58:11 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:58:11.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:58:11 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:58:11.859 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:58:11 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:58:11.859 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:58:11 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:58:11.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:58:11 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:58:11.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:58:11 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:58:11.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:58:11 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:58:11.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:58:11 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:58:12.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:58:12 smithi039 ceph-mon[96489]: pgmap v519: 97 pgs: 12 active+clean+snaptrim, 59 active+clean, 26 active+clean+snaptrim_wait; 1.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.9 MiB/s wr, 670 op/s 2023-07-21T18:58:12.676 INFO:tasks.workunit.client.0.smithi039.stdout: 10 118391 32.46 MB/sec execute 85 sec latency 4518.720 ms 2023-07-21T18:58:12.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:58:12 smithi193 ceph-mon[98204]: pgmap v519: 97 pgs: 12 active+clean+snaptrim, 59 active+clean, 26 active+clean+snaptrim_wait; 1.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.9 MiB/s wr, 670 op/s 2023-07-21T18:58:12.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:58:12 smithi155 ceph-mon[97934]: pgmap v519: 97 pgs: 12 active+clean+snaptrim, 59 active+clean, 26 active+clean+snaptrim_wait; 1.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.9 MiB/s wr, 670 op/s 2023-07-21T18:58:13.676 INFO:tasks.workunit.client.0.smithi039.stdout: 10 118885 32.41 MB/sec execute 86 sec latency 4628.217 ms 2023-07-21T18:58:14.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:58:14 smithi039 ceph-mon[96489]: pgmap v520: 97 pgs: 12 active+clean+snaptrim, 59 active+clean, 26 active+clean+snaptrim_wait; 1.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.9 MiB/s wr, 670 op/s 2023-07-21T18:58:14.676 INFO:tasks.workunit.client.0.smithi039.stdout: 10 119376 32.40 MB/sec execute 87 sec latency 5628.257 ms 2023-07-21T18:58:14.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:58:14 smithi193 ceph-mon[98204]: pgmap v520: 97 pgs: 12 active+clean+snaptrim, 59 active+clean, 26 active+clean+snaptrim_wait; 1.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.9 MiB/s wr, 670 op/s 2023-07-21T18:58:14.878 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:58:14 smithi155 ceph-mon[97934]: pgmap v520: 97 pgs: 12 active+clean+snaptrim, 59 active+clean, 26 active+clean+snaptrim_wait; 1.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.9 MiB/s wr, 670 op/s 2023-07-21T18:58:15.676 INFO:tasks.workunit.client.0.smithi039.stdout: 10 119743 32.21 MB/sec execute 88 sec latency 6628.502 ms 2023-07-21T18:58:16.676 INFO:tasks.workunit.client.0.smithi039.stdout: 10 120229 32.06 MB/sec execute 89 sec latency 7628.541 ms 2023-07-21T18:58:16.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:58:16 smithi039 ceph-mon[96489]: pgmap v521: 97 pgs: 12 active+clean+snaptrim, 59 active+clean, 26 active+clean+snaptrim_wait; 1.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.9 MiB/s wr, 670 op/s 2023-07-21T18:58:16.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:58:16 smithi193 ceph-mon[98204]: pgmap v521: 97 pgs: 12 active+clean+snaptrim, 59 active+clean, 26 active+clean+snaptrim_wait; 1.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.9 MiB/s wr, 670 op/s 2023-07-21T18:58:16.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:58:16 smithi155 ceph-mon[97934]: pgmap v521: 97 pgs: 12 active+clean+snaptrim, 59 active+clean, 26 active+clean+snaptrim_wait; 1.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.9 MiB/s wr, 670 op/s 2023-07-21T18:58:17.676 INFO:tasks.workunit.client.0.smithi039.stdout: 10 120895 32.13 MB/sec execute 90 sec latency 8628.577 ms 2023-07-21T18:58:18.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:58:18 smithi039 ceph-mon[96489]: pgmap v522: 97 pgs: 97 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 6.0 MiB/s wr, 579 op/s 2023-07-21T18:58:18.676 INFO:tasks.workunit.client.0.smithi039.stdout: 10 121493 32.23 MB/sec execute 91 sec latency 9628.610 ms 2023-07-21T18:58:18.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:58:18 smithi193 ceph-mon[98204]: pgmap v522: 97 pgs: 97 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 6.0 MiB/s wr, 579 op/s 2023-07-21T18:58:18.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:58:18 smithi155 ceph-mon[97934]: pgmap v522: 97 pgs: 97 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 6.0 MiB/s wr, 579 op/s 2023-07-21T18:58:19.676 INFO:tasks.workunit.client.0.smithi039.stdout: 10 122026 32.14 MB/sec execute 92 sec latency 10617.178 ms 2023-07-21T18:58:20.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:58:20 smithi039 ceph-mon[96489]: pgmap v523: 97 pgs: 97 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 6.0 MiB/s wr, 579 op/s 2023-07-21T18:58:20.676 INFO:tasks.workunit.client.0.smithi039.stdout: 10 122627 32.05 MB/sec execute 93 sec latency 96.068 ms 2023-07-21T18:58:20.859 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:58:20 smithi193 ceph-mon[98204]: pgmap v523: 97 pgs: 97 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 6.0 MiB/s wr, 579 op/s 2023-07-21T18:58:20.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:58:20 smithi155 ceph-mon[97934]: pgmap v523: 97 pgs: 97 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 6.0 MiB/s wr, 579 op/s 2023-07-21T18:58:21.676 INFO:tasks.workunit.client.0.smithi039.stdout: 10 123361 32.23 MB/sec execute 94 sec latency 64.079 ms 2023-07-21T18:58:21.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.712+0000 7f8ed9282700 1 -- 172.21.15.39:0/947240042 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ed406cb80 msgr2=0x7f8ed406cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:21.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.712+0000 7f8ed9282700 1 --2- 172.21.15.39:0/947240042 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ed406cb80 0x7f8ed406cf50 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f8ec40099c0 tx=0x7f8ec4009cd0 comp rx=0 tx=0).stop 2023-07-21T18:58:21.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.712+0000 7f8ed9282700 1 -- 172.21.15.39:0/947240042 shutdown_connections 2023-07-21T18:58:21.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.712+0000 7f8ed9282700 1 --2- 172.21.15.39:0/947240042 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ed410c4d0 0x7f8ed41140a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:21.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.712+0000 7f8ed9282700 1 --2- 172.21.15.39:0/947240042 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ed406cb80 0x7f8ed406cf50 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:21.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.712+0000 7f8ed9282700 1 --2- 172.21.15.39:0/947240042 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ed406d520 0x7f8ed410bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:21.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.712+0000 7f8ed9282700 1 -- 172.21.15.39:0/947240042 >> 172.21.15.39:0/947240042 conn(0x7f8ed406bd70 msgr2=0x7f8ed4070f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:58:21.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.713+0000 7f8ed9282700 1 -- 172.21.15.39:0/947240042 shutdown_connections 2023-07-21T18:58:21.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.713+0000 7f8ed9282700 1 -- 172.21.15.39:0/947240042 wait complete. 2023-07-21T18:58:21.726 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.713+0000 7f8ed9282700 1 Processor -- start 2023-07-21T18:58:21.727 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.714+0000 7f8ed9282700 1 -- start start 2023-07-21T18:58:21.727 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.714+0000 7f8ed9282700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ed406cb80 0x7f8ed41a3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:21.727 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.714+0000 7f8ed9282700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ed406d520 0x7f8ed41a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:21.727 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.714+0000 7f8ed9282700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ed410c4d0 0x7f8ed419c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:21.727 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.714+0000 7f8ed9282700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed419c8c0 con 0x7f8ed410c4d0 2023-07-21T18:58:21.727 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.714+0000 7f8ed9282700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed419ca30 con 0x7f8ed406cb80 2023-07-21T18:58:21.727 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.714+0000 7f8ed9282700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed419cbf0 con 0x7f8ed406d520 2023-07-21T18:58:21.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.714+0000 7f8ed27fc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ed406d520 0x7f8ed41a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:21.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.714+0000 7f8ed37fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ed410c4d0 0x7f8ed419c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:21.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.714+0000 7f8ed27fc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ed406d520 0x7f8ed41a3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:52208/0 (socket says 172.21.15.39:52208) 2023-07-21T18:58:21.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.714+0000 7f8ed27fc700 1 -- 172.21.15.39:0/3431281314 learned_addr learned my addr 172.21.15.39:0/3431281314 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:58:21.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.715+0000 7f8ed37fe700 1 -- 172.21.15.39:0/3431281314 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ed406d520 msgr2=0x7f8ed41a3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:21.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.715+0000 7f8ed37fe700 1 --2- 172.21.15.39:0/3431281314 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ed406d520 0x7f8ed41a3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:21.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.715+0000 7f8ed37fe700 1 -- 172.21.15.39:0/3431281314 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ed406cb80 msgr2=0x7f8ed41a3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:21.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.715+0000 7f8ed37fe700 1 --2- 172.21.15.39:0/3431281314 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ed406cb80 0x7f8ed41a3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:21.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.715+0000 7f8ed37fe700 1 -- 172.21.15.39:0/3431281314 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ec40095f0 con 0x7f8ed410c4d0 2023-07-21T18:58:21.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.715+0000 7f8ed37fe700 1 --2- 172.21.15.39:0/3431281314 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ed410c4d0 0x7f8ed419c110 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc007f10 tx=0x7f8ecc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:58:21.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.715+0000 7f8ebbfff700 1 -- 172.21.15.39:0/3431281314 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc00ddd0 con 0x7f8ed410c4d0 2023-07-21T18:58:21.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.715+0000 7f8ed9282700 1 -- 172.21.15.39:0/3431281314 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ed419ced0 con 0x7f8ed410c4d0 2023-07-21T18:58:21.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.716+0000 7f8ed9282700 1 -- 172.21.15.39:0/3431281314 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ed41a1190 con 0x7f8ed410c4d0 2023-07-21T18:58:21.731 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.720+0000 7f8ee5408b80 1 Processor -- start 2023-07-21T18:58:21.732 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.721+0000 7f8ebbfff700 1 -- 172.21.15.39:0/3431281314 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ecc00f040 con 0x7f8ed410c4d0 2023-07-21T18:58:21.732 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.721+0000 7f8ebbfff700 1 -- 172.21.15.39:0/3431281314 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc005850 con 0x7f8ed410c4d0 2023-07-21T18:58:21.732 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.721+0000 7f8ee5408b80 1 -- start start 2023-07-21T18:58:21.733 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.722+0000 7f8ebbfff700 1 -- 172.21.15.39:0/3431281314 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f8ecc015400 con 0x7f8ed410c4d0 2023-07-21T18:58:21.733 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.722+0000 7f8ee5408b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595f66916b0 0x5595f683ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:21.733 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.722+0000 7f8ee5408b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595f683b2d0 0x5595f6835040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:21.733 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.722+0000 7f8ee5408b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595f6835580 0x5595f68359d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:21.734 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.722+0000 7f8ee5408b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595f673ac50 con 0x5595f66916b0 2023-07-21T18:58:21.734 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.722+0000 7f8ee5408b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595f678d9e0 con 0x5595f683b2d0 2023-07-21T18:58:21.734 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.722+0000 7f8ee5408b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595f6786e70 con 0x5595f6835580 2023-07-21T18:58:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.722+0000 7f8ed27fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595f66916b0 0x5595f683ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.722+0000 7f8ed27fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595f66916b0 0x5595f683ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33660/0 (socket says 172.21.15.39:33660) 2023-07-21T18:58:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.722+0000 7f8ed27fc700 1 -- 172.21.15.39:0/3524366148 learned_addr learned my addr 172.21.15.39:0/3524366148 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:58:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.723+0000 7f8ed27fc700 1 --2- 172.21.15.39:0/3524366148 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595f6835580 0x5595f68359d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.723+0000 7f8ebbfff700 1 --2- 172.21.15.39:0/3431281314 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f8ebc0a1470 0x7f8ebc0a3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.723+0000 7f8ed27fc700 1 -- 172.21.15.39:0/3524366148 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595f6835580 msgr2=0x5595f68359d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:58:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.723+0000 7f8ebbfff700 1 -- 172.21.15.39:0/3431281314 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f8ecc0cd5e0 con 0x7f8ed410c4d0 2023-07-21T18:58:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.723+0000 7f8ed27fc700 1 --2- 172.21.15.39:0/3524366148 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595f6835580 0x5595f68359d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.723+0000 7f8ed27fc700 1 -- 172.21.15.39:0/3524366148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595f683b2d0 msgr2=0x5595f6835040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T18:58:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.723+0000 7f8ed27fc700 1 --2- 172.21.15.39:0/3524366148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595f683b2d0 0x5595f6835040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.723+0000 7f8ed27fc700 1 -- 172.21.15.39:0/3524366148 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595f65b4d80 con 0x5595f66916b0 2023-07-21T18:58:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.723+0000 7f8ed27fc700 1 --2- 172.21.15.39:0/3524366148 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595f66916b0 0x5595f683ad90 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8006b30 tx=0x7f8ec80095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b4cf9d1281883d8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:58:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.724+0000 7f8ed37fe700 1 --2- 172.21.15.39:0/3431281314 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f8ebc0a1470 0x7f8ebc0a3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.724+0000 7f8ed37fe700 1 --2- 172.21.15.39:0/3431281314 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f8ebc0a1470 0x7f8ebc0a3920 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc0cadb0 tx=0x7f8ecc009510 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:58:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.725+0000 7f8eb97fa700 1 -- 172.21.15.39:0/3524366148 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec80185f0 con 0x5595f66916b0 2023-07-21T18:58:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.725+0000 7f8eb97fa700 1 -- 172.21.15.39:0/3524366148 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ec8009b10 con 0x5595f66916b0 2023-07-21T18:58:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.725+0000 7f8eb97fa700 1 -- 172.21.15.39:0/3524366148 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8017920 con 0x5595f66916b0 2023-07-21T18:58:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.726+0000 7f8ee5408b80 1 -- 172.21.15.39:0/3524366148 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595f66916b0 msgr2=0x5595f683ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.726+0000 7f8ee5408b80 1 --2- 172.21.15.39:0/3524366148 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595f66916b0 0x5595f683ad90 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8006b30 tx=0x7f8ec80095a0 comp rx=0 tx=0).stop 2023-07-21T18:58:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.726+0000 7f8ee5408b80 1 -- 172.21.15.39:0/3524366148 shutdown_connections 2023-07-21T18:58:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.726+0000 7f8ee5408b80 1 --2- 172.21.15.39:0/3524366148 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595f6835580 0x5595f68359d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.726+0000 7f8ee5408b80 1 --2- 172.21.15.39:0/3524366148 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595f66916b0 0x5595f683ad90 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.726+0000 7f8ee5408b80 1 --2- 172.21.15.39:0/3524366148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595f683b2d0 0x5595f6835040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.726+0000 7f8ee5408b80 1 -- 172.21.15.39:0/3524366148 >> 172.21.15.39:0/3524366148 conn(0x5595f682abe0 msgr2=0x5595f682ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:58:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.726+0000 7f8ee5408b80 1 -- 172.21.15.39:0/3524366148 shutdown_connections 2023-07-21T18:58:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.726+0000 7f8ee5408b80 1 -- 172.21.15.39:0/3524366148 wait complete. 2023-07-21T18:58:21.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.727+0000 7f8ee5408b80 1 -- start start 2023-07-21T18:58:21.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.727+0000 7f8ee5408b80 1 Processor -- start 2023-07-21T18:58:21.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.727+0000 7f8ee5408b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595f66916b0 0x5595f6841620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:21.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.727+0000 7f8ee5408b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595f6835580 0x5595f683e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:21.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.727+0000 7f8ee5408b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595f683b2d0 0x5595f683ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:21.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.727+0000 7f8ee5408b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595f6726a60 con 0x5595f683b2d0 2023-07-21T18:58:21.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.727+0000 7f8ee5408b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595f6726760 con 0x5595f6835580 2023-07-21T18:58:21.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.727+0000 7f8ee5408b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595f660a8c0 con 0x5595f66916b0 2023-07-21T18:58:21.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.727+0000 7f8ee5408b80 10 client.0 _ll_register_callbacks cb 0x5595f6724700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:58:21.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.728+0000 7f8ed27fc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595f66916b0 0x5595f6841620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:21.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.728+0000 7f8ed27fc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595f66916b0 0x5595f6841620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:52218/0 (socket says 172.21.15.39:52218) 2023-07-21T18:58:21.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.728+0000 7f8ed27fc700 1 -- 172.21.15.39:0/3180744585 learned_addr learned my addr 172.21.15.39:0/3180744585 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:58:21.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.728+0000 7f8ed27fc700 1 -- 172.21.15.39:0/3180744585 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595f6835580 msgr2=0x5595f683e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:58:21.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.728+0000 7f8ed27fc700 1 --2- 172.21.15.39:0/3180744585 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595f6835580 0x5595f683e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:21.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.728+0000 7f8ed27fc700 1 -- 172.21.15.39:0/3180744585 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595f683b2d0 msgr2=0x5595f683ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:58:21.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.728+0000 7f8ed27fc700 1 --2- 172.21.15.39:0/3180744585 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595f683b2d0 0x5595f683ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:21.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.728+0000 7f8ed27fc700 1 -- 172.21.15.39:0/3180744585 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ec8015450 con 0x5595f66916b0 2023-07-21T18:58:21.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.728+0000 7f8ed27fc700 1 --2- 172.21.15.39:0/3180744585 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595f66916b0 0x5595f6841620 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8009240 tx=0x7f8ec8018f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=585d6f5c5affe77a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:58:21.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.728+0000 7f8eb6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2023-07-21T18:58:21.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.728+0000 7f8eb6ff5700 1 -- 172.21.15.39:0/3180744585 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8017720 con 0x5595f66916b0 2023-07-21T18:58:21.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.729+0000 7f8eb6ff5700 1 -- 172.21.15.39:0/3180744585 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f8ec8017880 con 0x5595f66916b0 2023-07-21T18:58:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.729+0000 7f8ee5408b80 1 -- 172.21.15.39:0/3180744585 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5595f66e0490 con 0x5595f66916b0 2023-07-21T18:58:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.729+0000 7f8eb6ff5700 1 -- 172.21.15.39:0/3180744585 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ec8017a40 con 0x5595f66916b0 2023-07-21T18:58:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.729+0000 7f8ee5408b80 10 client.26875 fetch_fsmap learned FSMap version 91 2023-07-21T18:58:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.729+0000 7f8ee5408b80 1 -- 172.21.15.39:0/3180744585 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=4+}) v3 -- 0x5595f659dc00 con 0x5595f66916b0 2023-07-21T18:58:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.730+0000 7f8eb6ff5700 1 -- 172.21.15.39:0/3180744585 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5595f66e0490 con 0x5595f66916b0 2023-07-21T18:58:21.745 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.730+0000 7f8eb6ff5700 1 -- 172.21.15.39:0/3180744585 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 91) v1 ==== 7510+0+0 (secure 0 0 0) 0x7f8ec8036bd0 con 0x5595f66916b0 2023-07-21T18:58:21.745 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.730+0000 7f8ee5408b80 10 client.26875 fetch_fsmap finished waiting for FSMap version 91 2023-07-21T18:58:21.745 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.730+0000 7f8ee5408b80 10 client.26875 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:58:21.746 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.730+0000 7f8ee5408b80 20 client.26875 populate_metadata read hostname 'smithi039' 2023-07-21T18:58:21.746 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.730+0000 7f8ee5408b80 1 --2- 172.21.15.39:0/3180744585 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5595f684eaf0 0x5595f6850f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:21.746 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.730+0000 7f8ee5408b80 4 client.26875 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:58:21.746 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.730+0000 7f8ee5408b80 1 -- 172.21.15.39:0/3180744585 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5595f677fe50 con 0x5595f684eaf0 2023-07-21T18:58:21.746 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.730+0000 7f8ed2ffd700 1 --2- 172.21.15.39:0/3180744585 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5595f684eaf0 0x5595f6850f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:21.746 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.731+0000 7f8eb6ff5700 10 client.26875 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:58:21.746 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.731+0000 7f8ed2ffd700 1 --2- 172.21.15.39:0/3180744585 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5595f684eaf0 0x5595f6850f00 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=374333375ca6c95 server_cookie=bf6fa315f6bc37ea in_seq=0 out_seq=0 2023-07-21T18:58:21.747 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.732+0000 7f8eb6ff5700 1 -- 172.21.15.39:0/3180744585 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5595f677fe50 con 0x5595f684eaf0 2023-07-21T18:58:21.747 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.732+0000 7f8eb6ff5700 10 client.26875 handle_command_reply: tid=0 2023-07-21T18:58:21.747 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.732+0000 7f8ee5408b80 1 client.26875 shutdown 2023-07-21T18:58:21.747 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.732+0000 7f8ee5408b80 1 -- 172.21.15.39:0/3180744585 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595f66916b0 msgr2=0x5595f6841620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:21.747 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.733+0000 7f8ee5408b80 1 --2- 172.21.15.39:0/3180744585 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595f66916b0 0x5595f6841620 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8009240 tx=0x7f8ec8018f10 comp rx=0 tx=0).stop 2023-07-21T18:58:21.747 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.733+0000 7f8ee5408b80 1 -- 172.21.15.39:0/3180744585 shutdown_connections 2023-07-21T18:58:21.747 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.733+0000 7f8ee5408b80 1 --2- 172.21.15.39:0/3180744585 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5595f684eaf0 0x5595f6850f00 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.733+0000 7f8ee5408b80 1 --2- 172.21.15.39:0/3180744585 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595f683b2d0 0x5595f683ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.733+0000 7f8ee5408b80 1 --2- 172.21.15.39:0/3180744585 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595f66916b0 0x5595f6841620 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.733+0000 7f8ee5408b80 1 --2- 172.21.15.39:0/3180744585 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595f6835580 0x5595f683e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.733+0000 7f8ee5408b80 1 -- 172.21.15.39:0/3180744585 >> 172.21.15.39:0/3180744585 conn(0x5595f682abe0 msgr2=0x5595f682ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:58:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.733+0000 7f8eb6ff5700 0 client.26875 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:58:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.733+0000 7f8ee5408b80 1 -- 172.21.15.39:0/3180744585 shutdown_connections 2023-07-21T18:58:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.733+0000 7f8ee5408b80 1 -- 172.21.15.39:0/3180744585 wait complete. 2023-07-21T18:58:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.733+0000 7f8ee5408b80 20 client.26875 trim_cache size 0 max 16384 2023-07-21T18:58:21.765 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.752+0000 7f8ee5408b80 1 Processor -- start 2023-07-21T18:58:21.765 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.753+0000 7f8ee5408b80 1 -- start start 2023-07-21T18:58:21.765 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.753+0000 7f8ee5408b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595f682abe0 0x5595f6835040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:21.765 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.753+0000 7f8ee5408b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595f66916b0 0x5595f683e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:21.766 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.753+0000 7f8ee5408b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595f6835580 0x5595f682fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:21.766 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.753+0000 7f8ee5408b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595f6831b70 con 0x5595f66916b0 2023-07-21T18:58:21.766 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.753+0000 7f8ee5408b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595f6830b20 con 0x5595f6835580 2023-07-21T18:58:21.766 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.753+0000 7f8ee5408b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595f683aa10 con 0x5595f682abe0 2023-07-21T18:58:21.766 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.753+0000 7f8ed2ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595f66916b0 0x5595f683e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:21.766 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.753+0000 7f8ed2ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595f66916b0 0x5595f683e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33674/0 (socket says 172.21.15.39:33674) 2023-07-21T18:58:21.766 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.753+0000 7f8ed27fc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595f682abe0 0x5595f6835040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:21.766 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.753+0000 7f8ed2ffd700 1 -- 172.21.15.39:0/1459848642 learned_addr learned my addr 172.21.15.39:0/1459848642 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:58:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.753+0000 7f8ed27fc700 1 --2- 172.21.15.39:0/1459848642 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595f6835580 0x5595f682fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.753+0000 7f8ed2ffd700 1 -- 172.21.15.39:0/1459848642 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595f682abe0 msgr2=0x5595f6835040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:58:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.753+0000 7f8ed2ffd700 1 --2- 172.21.15.39:0/1459848642 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595f682abe0 0x5595f6835040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.753+0000 7f8ed2ffd700 1 -- 172.21.15.39:0/1459848642 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595f6835580 msgr2=0x5595f682fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:58:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.753+0000 7f8ed2ffd700 1 --2- 172.21.15.39:0/1459848642 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595f6835580 0x5595f682fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.754+0000 7f8ed2ffd700 1 -- 172.21.15.39:0/1459848642 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595f6855b30 con 0x5595f66916b0 2023-07-21T18:58:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.754+0000 7f8ed2ffd700 1 --2- 172.21.15.39:0/1459848642 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595f66916b0 0x5595f683e850 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f8ec40099c0 tx=0x7f8ec4005a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca5be32683cfbcb0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:58:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.754+0000 7f8eb9ffb700 1 -- 172.21.15.39:0/1459848642 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec401d030 con 0x5595f66916b0 2023-07-21T18:58:21.770 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.754+0000 7f8eb9ffb700 1 -- 172.21.15.39:0/1459848642 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ec401e040 con 0x5595f66916b0 2023-07-21T18:58:21.770 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.754+0000 7f8eb9ffb700 1 -- 172.21.15.39:0/1459848642 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec4021780 con 0x5595f66916b0 2023-07-21T18:58:21.770 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.755+0000 7f8ee5408b80 1 -- 172.21.15.39:0/1459848642 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595f66916b0 msgr2=0x5595f683e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:21.770 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.755+0000 7f8ee5408b80 1 --2- 172.21.15.39:0/1459848642 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595f66916b0 0x5595f683e850 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f8ec40099c0 tx=0x7f8ec4005a90 comp rx=0 tx=0).stop 2023-07-21T18:58:21.770 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.755+0000 7f8ee5408b80 1 -- 172.21.15.39:0/1459848642 shutdown_connections 2023-07-21T18:58:21.770 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.755+0000 7f8ee5408b80 1 --2- 172.21.15.39:0/1459848642 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595f6835580 0x5595f682fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.755+0000 7f8ee5408b80 1 --2- 172.21.15.39:0/1459848642 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595f682abe0 0x5595f6835040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.755+0000 7f8ee5408b80 1 --2- 172.21.15.39:0/1459848642 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595f66916b0 0x5595f683e850 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.755+0000 7f8ee5408b80 1 -- 172.21.15.39:0/1459848642 >> 172.21.15.39:0/1459848642 conn(0x5595f684a770 msgr2=0x5595f683e140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:58:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.755+0000 7f8ee5408b80 1 -- 172.21.15.39:0/1459848642 shutdown_connections 2023-07-21T18:58:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.755+0000 7f8ee5408b80 1 -- 172.21.15.39:0/1459848642 wait complete. 2023-07-21T18:58:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.755+0000 7f8ee5408b80 1 -- start start 2023-07-21T18:58:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.756+0000 7f8ee5408b80 1 Processor -- start 2023-07-21T18:58:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.756+0000 7f8ee5408b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595f66916b0 0x5595f6843c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:21.772 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.756+0000 7f8ee5408b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595f682abe0 0x5595f6844190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:21.772 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.756+0000 7f8ee5408b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595f6835580 0x5595f68a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:21.772 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.756+0000 7f8ee5408b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595f6844aa0 con 0x5595f682abe0 2023-07-21T18:58:21.772 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.756+0000 7f8ee5408b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595f68a1ef0 con 0x5595f6835580 2023-07-21T18:58:21.772 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.756+0000 7f8ee5408b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595f68a2030 con 0x5595f66916b0 2023-07-21T18:58:21.772 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.756+0000 7f8ee5408b80 10 client.0 _ll_register_callbacks cb 0x7f8ec8009dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:58:21.772 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.756+0000 7f8ed2ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595f682abe0 0x5595f6844190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:21.773 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.756+0000 7f8ed2ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595f682abe0 0x5595f6844190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33680/0 (socket says 172.21.15.39:33680) 2023-07-21T18:58:21.773 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.756+0000 7f8ed2ffd700 1 -- 172.21.15.39:0/1844485884 learned_addr learned my addr 172.21.15.39:0/1844485884 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:58:21.773 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.757+0000 7f8ed2ffd700 1 -- 172.21.15.39:0/1844485884 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595f66916b0 msgr2=0x5595f6843c50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T18:58:21.773 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.757+0000 7f8ed2ffd700 1 --2- 172.21.15.39:0/1844485884 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595f66916b0 0x5595f6843c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:21.773 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.757+0000 7f8ed2ffd700 1 -- 172.21.15.39:0/1844485884 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595f6835580 msgr2=0x5595f68a19b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T18:58:21.773 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.757+0000 7f8ed2ffd700 1 --2- 172.21.15.39:0/1844485884 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595f6835580 0x5595f68a19b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:21.773 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.757+0000 7f8ed2ffd700 1 -- 172.21.15.39:0/1844485884 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ed41148e0 con 0x5595f682abe0 2023-07-21T18:58:21.773 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.757+0000 7f8ed2ffd700 1 --2- 172.21.15.39:0/1844485884 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595f682abe0 0x5595f6844190 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4024040 tx=0x7f8ec4005180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d89074d6858ef822 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:58:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.757+0000 7f8eb6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:58:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.757+0000 7f8eb6ff5700 1 -- 172.21.15.39:0/1844485884 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec401d040 con 0x5595f682abe0 2023-07-21T18:58:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.757+0000 7f8eb6ff5700 1 -- 172.21.15.39:0/1844485884 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f8ec400b830 con 0x5595f682abe0 2023-07-21T18:58:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.757+0000 7f8ee5408b80 1 -- 172.21.15.39:0/1844485884 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8ec8009b10 con 0x5595f682abe0 2023-07-21T18:58:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.757+0000 7f8eb6ff5700 1 -- 172.21.15.39:0/1844485884 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ec400b9f0 con 0x5595f682abe0 2023-07-21T18:58:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.758+0000 7f8ee5408b80 10 client.17430 fetch_fsmap learned FSMap version 91 2023-07-21T18:58:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.758+0000 7f8ee5408b80 1 -- 172.21.15.39:0/1844485884 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=4+}) v3 -- 0x5595f65db1f0 con 0x5595f682abe0 2023-07-21T18:58:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.758+0000 7f8eb6ff5700 1 -- 172.21.15.39:0/1844485884 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ec403c020 con 0x5595f682abe0 2023-07-21T18:58:21.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.758+0000 7f8eb6ff5700 1 -- 172.21.15.39:0/1844485884 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 91) v1 ==== 7510+0+0 (secure 0 0 0) 0x7f8ec4004500 con 0x5595f682abe0 2023-07-21T18:58:21.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.758+0000 7f8ee5408b80 10 client.17430 fetch_fsmap finished waiting for FSMap version 91 2023-07-21T18:58:21.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.758+0000 7f8ee5408b80 10 client.17430 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:58:21.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.758+0000 7f8ee5408b80 20 client.17430 populate_metadata read hostname 'smithi039' 2023-07-21T18:58:21.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.758+0000 7f8ee5408b80 1 --2- 172.21.15.39:0/1844485884 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5595f683b2d0 0x5595f68a5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:21.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.758+0000 7f8ee5408b80 4 client.17430 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:58:21.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.758+0000 7f8ee5408b80 1 -- 172.21.15.39:0/1844485884 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5595f6741ff0 con 0x5595f683b2d0 2023-07-21T18:58:21.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.762+0000 7f8ed27fc700 1 --2- 172.21.15.39:0/1844485884 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5595f683b2d0 0x5595f68a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:21.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.763+0000 7f8ed27fc700 1 --2- 172.21.15.39:0/1844485884 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5595f683b2d0 0x5595f68a5410 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79a0a80dde71830d server_cookie=6f732779247e0e8a in_seq=0 out_seq=0 2023-07-21T18:58:21.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.763+0000 7f8eb6ff5700 10 client.17430 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:58:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.978+0000 7f8eb6ff5700 1 -- 172.21.15.39:0/1844485884 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5595f6741ff0 con 0x5595f683b2d0 2023-07-21T18:58:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.978+0000 7f8eb6ff5700 10 client.17430 handle_command_reply: tid=0 2023-07-21T18:58:21.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.978+0000 7f8ee5408b80 1 client.17430 shutdown 2023-07-21T18:58:21.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.979+0000 7f8ee5408b80 1 -- 172.21.15.39:0/1844485884 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595f682abe0 msgr2=0x5595f6844190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:21.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.979+0000 7f8ee5408b80 1 --2- 172.21.15.39:0/1844485884 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595f682abe0 0x5595f6844190 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4024040 tx=0x7f8ec4005180 comp rx=0 tx=0).stop 2023-07-21T18:58:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.980+0000 7f8ee5408b80 1 -- 172.21.15.39:0/1844485884 shutdown_connections 2023-07-21T18:58:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.980+0000 7f8ee5408b80 1 --2- 172.21.15.39:0/1844485884 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5595f683b2d0 0x5595f68a5410 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.980+0000 7f8ee5408b80 1 --2- 172.21.15.39:0/1844485884 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595f6835580 0x5595f68a19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.980+0000 7f8ee5408b80 1 --2- 172.21.15.39:0/1844485884 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5595f66916b0 0x5595f6843c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.980+0000 7f8ee5408b80 1 --2- 172.21.15.39:0/1844485884 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595f682abe0 0x5595f6844190 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.980+0000 7f8ee5408b80 1 -- 172.21.15.39:0/1844485884 >> 172.21.15.39:0/1844485884 conn(0x5595f684a770 msgr2=0x5595f6839a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:58:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.980+0000 7f8eb6ff5700 0 client.17430 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:58:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.981+0000 7f8ee5408b80 1 -- 172.21.15.39:0/1844485884 shutdown_connections 2023-07-21T18:58:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.981+0000 7f8ee5408b80 1 -- 172.21.15.39:0/1844485884 wait complete. 2023-07-21T18:58:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.981+0000 7f8ee5408b80 20 client.17430 trim_cache size 0 max 16384 2023-07-21T18:58:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:58:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:58:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:58:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:58:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.981+0000 7f8ed9282700 1 -- 172.21.15.39:0/3431281314 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f8ebc0a1470 msgr2=0x7f8ebc0a3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.981+0000 7f8ed9282700 1 --2- 172.21.15.39:0/3431281314 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f8ebc0a1470 0x7f8ebc0a3920 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc0cadb0 tx=0x7f8ecc009510 comp rx=0 tx=0).stop 2023-07-21T18:58:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.981+0000 7f8ed9282700 1 -- 172.21.15.39:0/3431281314 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ed410c4d0 msgr2=0x7f8ed419c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.981+0000 7f8ed9282700 1 --2- 172.21.15.39:0/3431281314 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ed410c4d0 0x7f8ed419c110 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc007f10 tx=0x7f8ecc00d3b0 comp rx=0 tx=0).stop 2023-07-21T18:58:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.982+0000 7f8ed9282700 1 -- 172.21.15.39:0/3431281314 shutdown_connections 2023-07-21T18:58:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.982+0000 7f8ed9282700 1 --2- 172.21.15.39:0/3431281314 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f8ebc0a1470 0x7f8ebc0a3920 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.982+0000 7f8ed9282700 1 --2- 172.21.15.39:0/3431281314 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ed406cb80 0x7f8ed41a3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.982+0000 7f8ed9282700 1 --2- 172.21.15.39:0/3431281314 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ed410c4d0 0x7f8ed419c110 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:21.996 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.982+0000 7f8ed9282700 1 --2- 172.21.15.39:0/3431281314 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ed406d520 0x7f8ed41a3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:21.996 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.982+0000 7f8ed9282700 1 -- 172.21.15.39:0/3431281314 >> 172.21.15.39:0/3431281314 conn(0x7f8ed406bd70 msgr2=0x7f8ed410ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:58:21.996 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.982+0000 7f8ed9282700 1 -- 172.21.15.39:0/3431281314 shutdown_connections 2023-07-21T18:58:21.996 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:21.982+0000 7f8ed9282700 1 -- 172.21.15.39:0/3431281314 wait complete. 2023-07-21T18:58:22.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:58:22 smithi193 ceph-mon[98204]: pgmap v524: 97 pgs: 97 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 8.7 MiB/s wr, 770 op/s 2023-07-21T18:58:22.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:58:22 smithi193 ceph-mon[98204]: osdmap e106: 12 total, 12 up, 12 in 2023-07-21T18:58:22.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:58:22 smithi155 ceph-mon[97934]: pgmap v524: 97 pgs: 97 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 8.7 MiB/s wr, 770 op/s 2023-07-21T18:58:22.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:58:22 smithi155 ceph-mon[97934]: osdmap e106: 12 total, 12 up, 12 in 2023-07-21T18:58:22.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:58:22 smithi039 ceph-mon[96489]: pgmap v524: 97 pgs: 97 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 8.7 MiB/s wr, 770 op/s 2023-07-21T18:58:22.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:58:22 smithi039 ceph-mon[96489]: osdmap e106: 12 total, 12 up, 12 in 2023-07-21T18:58:22.676 INFO:tasks.workunit.client.0.smithi039.stdout: 10 124011 32.19 MB/sec execute 95 sec latency 192.956 ms 2023-07-21T18:58:23.676 INFO:tasks.workunit.client.0.smithi039.stdout: 10 124713 32.33 MB/sec execute 96 sec latency 57.795 ms 2023-07-21T18:58:23.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:58:23 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:58:23.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:58:23 smithi193 ceph-mon[98204]: pgmap v526: 97 pgs: 97 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.4 MiB/s wr, 509 op/s 2023-07-21T18:58:23.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:58:23 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:58:23.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:58:23 smithi155 ceph-mon[97934]: pgmap v526: 97 pgs: 97 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.4 MiB/s wr, 509 op/s 2023-07-21T18:58:23.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:58:23 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:58:23.926 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:58:23 smithi039 ceph-mon[96489]: pgmap v526: 97 pgs: 97 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.4 MiB/s wr, 509 op/s 2023-07-21T18:58:24.677 INFO:tasks.workunit.client.0.smithi039.stdout: 10 124957 32.10 MB/sec execute 97 sec latency 459.321 ms 2023-07-21T18:58:25.677 INFO:tasks.workunit.client.0.smithi039.stdout: 10 125037 31.86 MB/sec execute 98 sec latency 1236.021 ms 2023-07-21T18:58:25.927 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:58:25 smithi039 ceph-mon[96489]: pgmap v527: 97 pgs: 97 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.4 MiB/s wr, 509 op/s 2023-07-21T18:58:26.110 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:58:25 smithi193 ceph-mon[98204]: pgmap v527: 97 pgs: 97 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.4 MiB/s wr, 509 op/s 2023-07-21T18:58:26.128 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:58:25 smithi155 ceph-mon[97934]: pgmap v527: 97 pgs: 97 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.4 MiB/s wr, 509 op/s 2023-07-21T18:58:26.677 INFO:tasks.workunit.client.0.smithi039.stdout: 10 125359 31.72 MB/sec execute 99 sec latency 300.768 ms 2023-07-21T18:58:27.677 INFO:tasks.workunit.client.0.smithi039.stdout: 10 125829 31.57 MB/sec execute 100 sec latency 81.014 ms 2023-07-21T18:58:28.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:58:28 smithi193 ceph-mon[98204]: pgmap v528: 97 pgs: 97 active+clean; 1.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 22 MiB/s wr, 833 op/s 2023-07-21T18:58:28.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:58:28 smithi155 ceph-mon[97934]: pgmap v528: 97 pgs: 97 active+clean; 1.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 22 MiB/s wr, 833 op/s 2023-07-21T18:58:28.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:58:28 smithi039 ceph-mon[96489]: pgmap v528: 97 pgs: 97 active+clean; 1.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 22 MiB/s wr, 833 op/s 2023-07-21T18:58:28.676 INFO:tasks.workunit.client.0.smithi039.stdout: 10 126568 31.67 MB/sec execute 101 sec latency 84.297 ms 2023-07-21T18:58:29.677 INFO:tasks.workunit.client.0.smithi039.stdout: 10 127293 31.71 MB/sec execute 102 sec latency 92.069 ms 2023-07-21T18:58:30.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:58:30 smithi193 ceph-mon[98204]: pgmap v529: 97 pgs: 97 active+clean; 1.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 22 MiB/s wr, 833 op/s 2023-07-21T18:58:30.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:58:30 smithi155 ceph-mon[97934]: pgmap v529: 97 pgs: 97 active+clean; 1.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 22 MiB/s wr, 833 op/s 2023-07-21T18:58:30.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:58:30 smithi039 ceph-mon[96489]: pgmap v529: 97 pgs: 97 active+clean; 1.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 22 MiB/s wr, 833 op/s 2023-07-21T18:58:30.677 INFO:tasks.workunit.client.0.smithi039.stdout: 10 128048 31.81 MB/sec execute 103 sec latency 53.923 ms 2023-07-21T18:58:31.677 INFO:tasks.workunit.client.0.smithi039.stdout: 10 128667 31.89 MB/sec execute 104 sec latency 120.268 ms 2023-07-21T18:58:32.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:58:32 smithi193 ceph-mon[98204]: pgmap v530: 97 pgs: 97 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 22 MiB/s wr, 1.05k op/s 2023-07-21T18:58:32.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:58:32 smithi155 ceph-mon[97934]: pgmap v530: 97 pgs: 97 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 22 MiB/s wr, 1.05k op/s 2023-07-21T18:58:32.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:58:32 smithi039 ceph-mon[96489]: pgmap v530: 97 pgs: 97 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 22 MiB/s wr, 1.05k op/s 2023-07-21T18:58:32.677 INFO:tasks.workunit.client.0.smithi039.stdout: 10 129217 31.81 MB/sec execute 105 sec latency 137.640 ms 2023-07-21T18:58:33.677 INFO:tasks.workunit.client.0.smithi039.stdout: 10 129974 31.94 MB/sec execute 106 sec latency 55.785 ms 2023-07-21T18:58:34.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:58:34 smithi193 ceph-mon[98204]: pgmap v531: 97 pgs: 97 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 19 MiB/s wr, 928 op/s 2023-07-21T18:58:34.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:58:34 smithi155 ceph-mon[97934]: pgmap v531: 97 pgs: 97 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 19 MiB/s wr, 928 op/s 2023-07-21T18:58:34.677 INFO:tasks.workunit.client.0.smithi039.stdout: 10 130618 31.96 MB/sec execute 107 sec latency 110.671 ms 2023-07-21T18:58:34.679 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:58:34 smithi039 ceph-mon[96489]: pgmap v531: 97 pgs: 97 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 19 MiB/s wr, 928 op/s 2023-07-21T18:58:35.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.144+0000 7f4f48b08700 1 -- 172.21.15.39:0/3934196637 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f4406ce50 msgr2=0x7f4f4406d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:35.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.144+0000 7f4f48b08700 1 --2- 172.21.15.39:0/3934196637 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f4406ce50 0x7f4f4406d2c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4f340098f0 tx=0x7f4f34009c00 comp rx=0 tx=0).stop 2023-07-21T18:58:35.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.144+0000 7f4f48b08700 1 -- 172.21.15.39:0/3934196637 shutdown_connections 2023-07-21T18:58:35.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.144+0000 7f4f48b08700 1 --2- 172.21.15.39:0/3934196637 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f44114c80 0x7f4f44117060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:35.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.144+0000 7f4f48b08700 1 --2- 172.21.15.39:0/3934196637 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f4410a1b0 0x7f4f4406c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:35.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.144+0000 7f4f48b08700 1 --2- 172.21.15.39:0/3934196637 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f4406ce50 0x7f4f4406d2c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:35.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.144+0000 7f4f48b08700 1 -- 172.21.15.39:0/3934196637 >> 172.21.15.39:0/3934196637 conn(0x7f4f4406bd70 msgr2=0x7f4f44070f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:58:35.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.144+0000 7f4f48b08700 1 -- 172.21.15.39:0/3934196637 shutdown_connections 2023-07-21T18:58:35.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.144+0000 7f4f48b08700 1 -- 172.21.15.39:0/3934196637 wait complete. 2023-07-21T18:58:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.145+0000 7f4f48b08700 1 Processor -- start 2023-07-21T18:58:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.145+0000 7f4f48b08700 1 -- start start 2023-07-21T18:58:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.145+0000 7f4f48b08700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f4406ce50 0x7f4f4410a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.146+0000 7f4f48b08700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f4410a1b0 0x7f4f4410ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.146+0000 7f4f48b08700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f44114c80 0x7f4f441b0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.146+0000 7f4f48b08700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4410b6a0 con 0x7f4f44114c80 2023-07-21T18:58:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.146+0000 7f4f48b08700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f441b1160 con 0x7f4f4410a1b0 2023-07-21T18:58:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.146+0000 7f4f48b08700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f441b12a0 con 0x7f4f4406ce50 2023-07-21T18:58:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.146+0000 7f4f41d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f4410a1b0 0x7f4f4410ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.146+0000 7f4f41d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f4410a1b0 0x7f4f4410ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:46286/0 (socket says 172.21.15.39:46286) 2023-07-21T18:58:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.146+0000 7f4f41d9b700 1 -- 172.21.15.39:0/3734964054 learned_addr learned my addr 172.21.15.39:0/3734964054 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:58:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.146+0000 7f4f42d9d700 1 --2- 172.21.15.39:0/3734964054 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f44114c80 0x7f4f441b0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:35.160 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.146+0000 7f4f4259c700 1 --2- 172.21.15.39:0/3734964054 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f4406ce50 0x7f4f4410a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:35.160 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.146+0000 7f4f41d9b700 1 -- 172.21.15.39:0/3734964054 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f4406ce50 msgr2=0x7f4f4410a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:35.160 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.146+0000 7f4f41d9b700 1 --2- 172.21.15.39:0/3734964054 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f4406ce50 0x7f4f4410a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:35.160 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.146+0000 7f4f41d9b700 1 -- 172.21.15.39:0/3734964054 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f44114c80 msgr2=0x7f4f441b0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:35.160 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.146+0000 7f4f41d9b700 1 --2- 172.21.15.39:0/3734964054 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f44114c80 0x7f4f441b0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:35.160 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.146+0000 7f4f41d9b700 1 -- 172.21.15.39:0/3734964054 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f34009520 con 0x7f4f4410a1b0 2023-07-21T18:58:35.160 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.146+0000 7f4f41d9b700 1 --2- 172.21.15.39:0/3734964054 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f4410a1b0 0x7f4f4410ae40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4f34009230 tx=0x7f4f3400fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:58:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.147+0000 7f4f337fe700 1 -- 172.21.15.39:0/3734964054 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f3401c070 con 0x7f4f4410a1b0 2023-07-21T18:58:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.147+0000 7f4f48b08700 1 -- 172.21.15.39:0/3734964054 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f441b14a0 con 0x7f4f4410a1b0 2023-07-21T18:58:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.147+0000 7f4f337fe700 1 -- 172.21.15.39:0/3734964054 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f340058e0 con 0x7f4f4410a1b0 2023-07-21T18:58:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.147+0000 7f4f337fe700 1 -- 172.21.15.39:0/3734964054 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f340038e0 con 0x7f4f4410a1b0 2023-07-21T18:58:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.147+0000 7f4f48b08700 1 -- 172.21.15.39:0/3734964054 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f441b1950 con 0x7f4f4410a1b0 2023-07-21T18:58:35.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.149+0000 7f4f337fe700 1 -- 172.21.15.39:0/3734964054 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f4f34020030 con 0x7f4f4410a1b0 2023-07-21T18:58:35.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.150+0000 7f4f337fe700 1 --2- 172.21.15.39:0/3734964054 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f4f2c0a12c0 0x7f4f2c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:35.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.150+0000 7f4f337fe700 1 -- 172.21.15.39:0/3734964054 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f4f340cdaf0 con 0x7f4f4410a1b0 2023-07-21T18:58:35.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.150+0000 7f4f4259c700 1 --2- 172.21.15.39:0/3734964054 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f4f2c0a12c0 0x7f4f2c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:35.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.151+0000 7f4f4259c700 1 --2- 172.21.15.39:0/3734964054 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f4f2c0a12c0 0x7f4f2c0a3770 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4f38005fd0 tx=0x7f4f38009310 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:58:35.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.152+0000 7f4f54c8eb80 1 Processor -- start 2023-07-21T18:58:35.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.152+0000 7f4f54c8eb80 1 -- start start 2023-07-21T18:58:35.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.152+0000 7f4f54c8eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620635eb6b0 0x562063794d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:35.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.152+0000 7f4f54c8eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620637952d0 0x56206378f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:35.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.152+0000 7f4f54c8eb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56206378f580 0x56206378f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:35.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.152+0000 7f4f54c8eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x562063694c50 con 0x5620637952d0 2023-07-21T18:58:35.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.152+0000 7f4f54c8eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620636e79e0 con 0x5620635eb6b0 2023-07-21T18:58:35.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.152+0000 7f4f54c8eb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620636e0e70 con 0x56206378f580 2023-07-21T18:58:35.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.152+0000 7f4f41d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620637952d0 0x56206378f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:35.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.152+0000 7f4f41d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620637952d0 0x56206378f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33700/0 (socket says 172.21.15.39:33700) 2023-07-21T18:58:35.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.152+0000 7f4f41d9b700 1 -- 172.21.15.39:0/1699907431 learned_addr learned my addr 172.21.15.39:0/1699907431 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:58:35.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.152+0000 7f4f4259c700 1 --2- 172.21.15.39:0/1699907431 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620635eb6b0 0x562063794d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:35.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.152+0000 7f4f42d9d700 1 --2- 172.21.15.39:0/1699907431 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56206378f580 0x56206378f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:35.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.153+0000 7f4f41d9b700 1 -- 172.21.15.39:0/1699907431 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56206378f580 msgr2=0x56206378f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:58:35.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.153+0000 7f4f41d9b700 1 --2- 172.21.15.39:0/1699907431 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56206378f580 0x56206378f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:35.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.153+0000 7f4f41d9b700 1 -- 172.21.15.39:0/1699907431 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620635eb6b0 msgr2=0x562063794d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:58:35.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.153+0000 7f4f41d9b700 1 --2- 172.21.15.39:0/1699907431 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620635eb6b0 0x562063794d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:35.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.153+0000 7f4f41d9b700 1 -- 172.21.15.39:0/1699907431 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56206350ed80 con 0x5620637952d0 2023-07-21T18:58:35.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.153+0000 7f4f4259c700 1 --2- 172.21.15.39:0/1699907431 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620635eb6b0 0x562063794d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:58:35.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.153+0000 7f4f41d9b700 1 --2- 172.21.15.39:0/1699907431 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620637952d0 0x56206378f040 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f4f34097150 tx=0x7f4f34005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbfad2cdb063278f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:58:35.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.153+0000 7f4f30ff9700 1 -- 172.21.15.39:0/1699907431 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f340a3010 con 0x5620637952d0 2023-07-21T18:58:35.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.153+0000 7f4f30ff9700 1 -- 172.21.15.39:0/1699907431 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f340cb4c0 con 0x5620637952d0 2023-07-21T18:58:35.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.153+0000 7f4f30ff9700 1 -- 172.21.15.39:0/1699907431 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f3409b110 con 0x5620637952d0 2023-07-21T18:58:35.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.154+0000 7f4f54c8eb80 1 -- 172.21.15.39:0/1699907431 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620637952d0 msgr2=0x56206378f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:35.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.154+0000 7f4f54c8eb80 1 --2- 172.21.15.39:0/1699907431 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620637952d0 0x56206378f040 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f4f34097150 tx=0x7f4f34005040 comp rx=0 tx=0).stop 2023-07-21T18:58:35.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.154+0000 7f4f54c8eb80 1 -- 172.21.15.39:0/1699907431 shutdown_connections 2023-07-21T18:58:35.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.154+0000 7f4f54c8eb80 1 --2- 172.21.15.39:0/1699907431 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620635eb6b0 0x562063794d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:35.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.154+0000 7f4f54c8eb80 1 --2- 172.21.15.39:0/1699907431 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56206378f580 0x56206378f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:35.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.154+0000 7f4f54c8eb80 1 --2- 172.21.15.39:0/1699907431 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620637952d0 0x56206378f040 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:35.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.154+0000 7f4f54c8eb80 1 -- 172.21.15.39:0/1699907431 >> 172.21.15.39:0/1699907431 conn(0x562063784be0 msgr2=0x562063789f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:58:35.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.155+0000 7f4f54c8eb80 1 -- 172.21.15.39:0/1699907431 shutdown_connections 2023-07-21T18:58:35.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.155+0000 7f4f54c8eb80 1 -- 172.21.15.39:0/1699907431 wait complete. 2023-07-21T18:58:35.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.155+0000 7f4f54c8eb80 1 -- start start 2023-07-21T18:58:35.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.155+0000 7f4f54c8eb80 1 Processor -- start 2023-07-21T18:58:35.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.155+0000 7f4f54c8eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620635eb6b0 0x56206379e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:35.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.155+0000 7f4f54c8eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56206378f580 0x56206379e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:35.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.156+0000 7f4f54c8eb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620637952d0 0x562063798590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:35.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.156+0000 7f4f54c8eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x562063680a60 con 0x56206378f580 2023-07-21T18:58:35.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.156+0000 7f4f54c8eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562063680760 con 0x5620635eb6b0 2023-07-21T18:58:35.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.156+0000 7f4f54c8eb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620635648c0 con 0x5620637952d0 2023-07-21T18:58:35.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.156+0000 7f4f54c8eb80 10 client.0 _ll_register_callbacks cb 0x56206367e700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:58:35.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.156+0000 7f4f4259c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620635eb6b0 0x56206379e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:35.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.156+0000 7f4f4259c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620635eb6b0 0x56206379e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:46300/0 (socket says 172.21.15.39:46300) 2023-07-21T18:58:35.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.156+0000 7f4f4259c700 1 -- 172.21.15.39:0/1650140148 learned_addr learned my addr 172.21.15.39:0/1650140148 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:58:35.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.156+0000 7f4f42d9d700 1 --2- 172.21.15.39:0/1650140148 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620637952d0 0x562063798590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.156+0000 7f4f4259c700 1 -- 172.21.15.39:0/1650140148 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620637952d0 msgr2=0x562063798590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:58:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.156+0000 7f4f41d9b700 1 --2- 172.21.15.39:0/1650140148 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56206378f580 0x56206379e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.156+0000 7f4f4259c700 1 --2- 172.21.15.39:0/1650140148 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620637952d0 0x562063798590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.156+0000 7f4f4259c700 1 -- 172.21.15.39:0/1650140148 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56206378f580 msgr2=0x56206379e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:58:35.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.156+0000 7f4f4259c700 1 --2- 172.21.15.39:0/1650140148 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56206378f580 0x56206379e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:35.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.156+0000 7f4f4259c700 1 -- 172.21.15.39:0/1650140148 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f38017070 con 0x5620635eb6b0 2023-07-21T18:58:35.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.156+0000 7f4f42d9d700 1 --2- 172.21.15.39:0/1650140148 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620637952d0 0x562063798590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:58:35.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.156+0000 7f4f41d9b700 1 --2- 172.21.15.39:0/1650140148 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56206378f580 0x56206379e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:58:35.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.157+0000 7f4f4259c700 1 --2- 172.21.15.39:0/1650140148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620635eb6b0 0x56206379e300 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4f38009fd0 tx=0x7f4f380047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af6a506d1be5c7a1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:58:35.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.157+0000 7f4f29ffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-21T18:58:35.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.157+0000 7f4f29ffb700 1 -- 172.21.15.39:0/1650140148 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f3801fa90 con 0x5620635eb6b0 2023-07-21T18:58:35.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.157+0000 7f4f29ffb700 1 -- 172.21.15.39:0/1650140148 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f4f3801fbf0 con 0x5620635eb6b0 2023-07-21T18:58:35.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.157+0000 7f4f54c8eb80 1 -- 172.21.15.39:0/1650140148 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56206363a490 con 0x5620635eb6b0 2023-07-21T18:58:35.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.157+0000 7f4f29ffb700 1 -- 172.21.15.39:0/1650140148 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f3801fdf0 con 0x5620635eb6b0 2023-07-21T18:58:35.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.157+0000 7f4f54c8eb80 10 client.26858 fetch_fsmap learned FSMap version 92 2023-07-21T18:58:35.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.157+0000 7f4f54c8eb80 1 -- 172.21.15.39:0/1650140148 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=4+}) v3 -- 0x7f4f441b1950 con 0x5620635eb6b0 2023-07-21T18:58:35.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.157+0000 7f4f29ffb700 1 -- 172.21.15.39:0/1650140148 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f38033080 con 0x5620635eb6b0 2023-07-21T18:58:35.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.157+0000 7f4f29ffb700 1 -- 172.21.15.39:0/1650140148 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 92) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f4f3802d030 con 0x5620635eb6b0 2023-07-21T18:58:35.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.158+0000 7f4f54c8eb80 10 client.26858 fetch_fsmap finished waiting for FSMap version 92 2023-07-21T18:58:35.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.158+0000 7f4f54c8eb80 10 client.26858 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:58:35.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.158+0000 7f4f54c8eb80 20 client.26858 populate_metadata read hostname 'smithi039' 2023-07-21T18:58:35.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.158+0000 7f4f54c8eb80 1 --2- 172.21.15.39:0/1650140148 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5620637a8980 0x5620637aad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:35.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.158+0000 7f4f54c8eb80 4 client.26858 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:58:35.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.158+0000 7f4f54c8eb80 1 -- 172.21.15.39:0/1650140148 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5620636d9e50 con 0x5620637a8980 2023-07-21T18:58:35.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.158+0000 7f4f41d9b700 1 --2- 172.21.15.39:0/1650140148 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5620637a8980 0x5620637aad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:35.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.158+0000 7f4f29ffb700 10 client.26858 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:58:35.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.158+0000 7f4f41d9b700 1 --2- 172.21.15.39:0/1650140148 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5620637a8980 0x5620637aad90 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=486b7025e9732dde server_cookie=34a23c6d3bd23971 in_seq=0 out_seq=0 2023-07-21T18:58:35.177 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.161+0000 7f4f29ffb700 1 -- 172.21.15.39:0/1650140148 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5620636d9e50 con 0x5620637a8980 2023-07-21T18:58:35.177 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.161+0000 7f4f29ffb700 10 client.26858 handle_command_reply: tid=0 2023-07-21T18:58:35.177 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.161+0000 7f4f54c8eb80 1 client.26858 shutdown 2023-07-21T18:58:35.177 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.162+0000 7f4f54c8eb80 1 -- 172.21.15.39:0/1650140148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620635eb6b0 msgr2=0x56206379e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:35.177 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.162+0000 7f4f54c8eb80 1 --2- 172.21.15.39:0/1650140148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620635eb6b0 0x56206379e300 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4f38009fd0 tx=0x7f4f380047c0 comp rx=0 tx=0).stop 2023-07-21T18:58:35.177 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.162+0000 7f4f54c8eb80 1 -- 172.21.15.39:0/1650140148 shutdown_connections 2023-07-21T18:58:35.177 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.162+0000 7f4f54c8eb80 1 --2- 172.21.15.39:0/1650140148 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5620637a8980 0x5620637aad90 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:35.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.162+0000 7f4f54c8eb80 1 --2- 172.21.15.39:0/1650140148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620635eb6b0 0x56206379e300 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:35.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.162+0000 7f4f29ffb700 0 client.26858 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:58:35.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.162+0000 7f4f54c8eb80 1 --2- 172.21.15.39:0/1650140148 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620637952d0 0x562063798590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:35.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.162+0000 7f4f54c8eb80 1 --2- 172.21.15.39:0/1650140148 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56206378f580 0x56206379e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:35.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.162+0000 7f4f54c8eb80 1 -- 172.21.15.39:0/1650140148 >> 172.21.15.39:0/1650140148 conn(0x562063784be0 msgr2=0x5620637894e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:58:35.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.162+0000 7f4f54c8eb80 1 -- 172.21.15.39:0/1650140148 shutdown_connections 2023-07-21T18:58:35.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.162+0000 7f4f54c8eb80 1 -- 172.21.15.39:0/1650140148 wait complete. 2023-07-21T18:58:35.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.162+0000 7f4f54c8eb80 20 client.26858 trim_cache size 0 max 16384 2023-07-21T18:58:35.194 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.182+0000 7f4f54c8eb80 1 Processor -- start 2023-07-21T18:58:35.194 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.182+0000 7f4f54c8eb80 1 -- start start 2023-07-21T18:58:35.194 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.183+0000 7f4f54c8eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620635eb6b0 0x56206378f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:35.194 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.183+0000 7f4f54c8eb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56206378f580 0x562063796190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:35.195 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.183+0000 7f4f54c8eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620637952d0 0x562063794060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:35.195 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.183+0000 7f4f54c8eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x562063786d00 con 0x5620635eb6b0 2023-07-21T18:58:35.195 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.183+0000 7f4f54c8eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562063781bb0 con 0x5620637952d0 2023-07-21T18:58:35.195 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.183+0000 7f4f54c8eb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x562063796b30 con 0x56206378f580 2023-07-21T18:58:35.195 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.183+0000 7f4f42d9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620637952d0 0x562063794060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:35.195 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.183+0000 7f4f42d9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620637952d0 0x562063794060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:46308/0 (socket says 172.21.15.39:46308) 2023-07-21T18:58:35.195 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.183+0000 7f4f42d9d700 1 -- 172.21.15.39:0/3522832246 learned_addr learned my addr 172.21.15.39:0/3522832246 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:58:35.195 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.183+0000 7f4f41d9b700 1 --2- 172.21.15.39:0/3522832246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56206378f580 0x562063796190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:35.197 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.183+0000 7f4f42d9d700 1 -- 172.21.15.39:0/3522832246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56206378f580 msgr2=0x562063796190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:58:35.197 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.183+0000 7f4f42d9d700 1 --2- 172.21.15.39:0/3522832246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56206378f580 0x562063796190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:35.197 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.183+0000 7f4f42d9d700 1 -- 172.21.15.39:0/3522832246 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620635eb6b0 msgr2=0x56206378f040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T18:58:35.197 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.183+0000 7f4f42d9d700 1 --2- 172.21.15.39:0/3522832246 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620635eb6b0 0x56206378f040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:35.197 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.183+0000 7f4f42d9d700 1 -- 172.21.15.39:0/3522832246 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56206378fc20 con 0x5620637952d0 2023-07-21T18:58:35.197 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.183+0000 7f4f41d9b700 1 --2- 172.21.15.39:0/3522832246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56206378f580 0x562063796190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:58:35.198 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.184+0000 7f4f42d9d700 1 --2- 172.21.15.39:0/3522832246 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620637952d0 0x562063794060 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c0094e0 tx=0x7f4f3c009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c1864637a6e4a82 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:58:35.198 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.184+0000 7f4f317fa700 1 -- 172.21.15.39:0/3522832246 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c00e040 con 0x5620637952d0 2023-07-21T18:58:35.198 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.184+0000 7f4f317fa700 1 -- 172.21.15.39:0/3522832246 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f3c00f040 con 0x5620637952d0 2023-07-21T18:58:35.198 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.184+0000 7f4f317fa700 1 -- 172.21.15.39:0/3522832246 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c020c80 con 0x5620637952d0 2023-07-21T18:58:35.198 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.185+0000 7f4f54c8eb80 1 -- 172.21.15.39:0/3522832246 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620637952d0 msgr2=0x562063794060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:35.198 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.185+0000 7f4f54c8eb80 1 --2- 172.21.15.39:0/3522832246 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620637952d0 0x562063794060 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c0094e0 tx=0x7f4f3c009920 comp rx=0 tx=0).stop 2023-07-21T18:58:35.198 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.185+0000 7f4f54c8eb80 1 -- 172.21.15.39:0/3522832246 shutdown_connections 2023-07-21T18:58:35.198 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.185+0000 7f4f54c8eb80 1 --2- 172.21.15.39:0/3522832246 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620637952d0 0x562063794060 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:35.199 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.185+0000 7f4f54c8eb80 1 --2- 172.21.15.39:0/3522832246 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620635eb6b0 0x56206378f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:35.199 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.185+0000 7f4f54c8eb80 1 --2- 172.21.15.39:0/3522832246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56206378f580 0x562063796190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:35.199 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.185+0000 7f4f54c8eb80 1 -- 172.21.15.39:0/3522832246 >> 172.21.15.39:0/3522832246 conn(0x562063784be0 msgr2=0x562063793950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:58:35.199 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.185+0000 7f4f54c8eb80 1 -- 172.21.15.39:0/3522832246 shutdown_connections 2023-07-21T18:58:35.199 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.185+0000 7f4f54c8eb80 1 -- 172.21.15.39:0/3522832246 wait complete. 2023-07-21T18:58:35.199 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.186+0000 7f4f54c8eb80 1 -- start start 2023-07-21T18:58:35.199 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.186+0000 7f4f54c8eb80 1 Processor -- start 2023-07-21T18:58:35.201 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.186+0000 7f4f54c8eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620635eb6b0 0x56206379c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:35.201 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.186+0000 7f4f54c8eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56206378f580 0x56206379cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:35.201 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.186+0000 7f4f54c8eb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620637952d0 0x5620637f5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:35.201 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.186+0000 7f4f54c8eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620637f5d80 con 0x56206378f580 2023-07-21T18:58:35.202 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.186+0000 7f4f54c8eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620637f5ef0 con 0x5620635eb6b0 2023-07-21T18:58:35.202 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.186+0000 7f4f54c8eb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620637f60b0 con 0x5620637952d0 2023-07-21T18:58:35.202 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.186+0000 7f4f54c8eb80 10 client.0 _ll_register_callbacks cb 0x7f4f38004520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:58:35.202 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.187+0000 7f4f4259c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620635eb6b0 0x56206379c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:35.202 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.187+0000 7f4f4259c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620635eb6b0 0x56206379c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:46312/0 (socket says 172.21.15.39:46312) 2023-07-21T18:58:35.202 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.187+0000 7f4f4259c700 1 -- 172.21.15.39:0/4174284560 learned_addr learned my addr 172.21.15.39:0/4174284560 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:58:35.202 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.187+0000 7f4f4259c700 1 -- 172.21.15.39:0/4174284560 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620637952d0 msgr2=0x5620637f5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:58:35.202 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.187+0000 7f4f4259c700 1 --2- 172.21.15.39:0/4174284560 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620637952d0 0x5620637f5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:35.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.187+0000 7f4f4259c700 1 -- 172.21.15.39:0/4174284560 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56206378f580 msgr2=0x56206379cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:58:35.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.187+0000 7f4f4259c700 1 --2- 172.21.15.39:0/4174284560 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56206378f580 0x56206379cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:35.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.187+0000 7f4f4259c700 1 -- 172.21.15.39:0/4174284560 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56206350ed80 con 0x5620635eb6b0 2023-07-21T18:58:35.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.187+0000 7f4f4259c700 1 --2- 172.21.15.39:0/4174284560 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620635eb6b0 0x56206379c680 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4f38000c00 tx=0x7f4f38009ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b89db8aa93196b58 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:58:35.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.187+0000 7f4f29ffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-21T18:58:35.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.188+0000 7f4f29ffb700 1 -- 172.21.15.39:0/4174284560 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f3802f610 con 0x5620635eb6b0 2023-07-21T18:58:35.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.188+0000 7f4f29ffb700 1 -- 172.21.15.39:0/4174284560 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f4f3802f770 con 0x5620635eb6b0 2023-07-21T18:58:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.188+0000 7f4f54c8eb80 1 -- 172.21.15.39:0/4174284560 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4f340cb4c0 con 0x5620635eb6b0 2023-07-21T18:58:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.188+0000 7f4f29ffb700 1 -- 172.21.15.39:0/4174284560 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f3802f930 con 0x5620635eb6b0 2023-07-21T18:58:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.188+0000 7f4f54c8eb80 10 client.26864 fetch_fsmap learned FSMap version 92 2023-07-21T18:58:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.188+0000 7f4f54c8eb80 1 -- 172.21.15.39:0/4174284560 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=4+}) v3 -- 0x7f4f3c009190 con 0x5620635eb6b0 2023-07-21T18:58:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.188+0000 7f4f29ffb700 1 -- 172.21.15.39:0/4174284560 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f340cb4c0 con 0x5620635eb6b0 2023-07-21T18:58:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.188+0000 7f4f29ffb700 1 -- 172.21.15.39:0/4174284560 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 92) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f4f3803ba00 con 0x5620635eb6b0 2023-07-21T18:58:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.188+0000 7f4f54c8eb80 10 client.26864 fetch_fsmap finished waiting for FSMap version 92 2023-07-21T18:58:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.188+0000 7f4f54c8eb80 10 client.26864 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:58:35.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.188+0000 7f4f54c8eb80 20 client.26864 populate_metadata read hostname 'smithi039' 2023-07-21T18:58:35.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.188+0000 7f4f54c8eb80 1 --2- 172.21.15.39:0/4174284560 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5620637fd980 0x5620637ffdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:35.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.188+0000 7f4f54c8eb80 4 client.26864 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:58:35.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.188+0000 7f4f54c8eb80 1 -- 172.21.15.39:0/4174284560 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56206369bff0 con 0x5620637fd980 2023-07-21T18:58:35.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.189+0000 7f4f41d9b700 1 --2- 172.21.15.39:0/4174284560 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5620637fd980 0x5620637ffdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:35.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.189+0000 7f4f29ffb700 10 client.26864 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:58:35.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:35.189+0000 7f4f41d9b700 1 --2- 172.21.15.39:0/4174284560 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5620637fd980 0x5620637ffdb0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=623d10972525bda9 server_cookie=28e22c020de66bd0 in_seq=0 out_seq=0 2023-07-21T18:58:35.677 INFO:tasks.workunit.client.0.smithi039.stdout: 10 131212 31.93 MB/sec execute 108 sec latency 276.532 ms 2023-07-21T18:58:36.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:58:36 smithi193 ceph-mon[98204]: pgmap v532: 97 pgs: 97 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 18 MiB/s wr, 873 op/s 2023-07-21T18:58:36.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:58:36 smithi155 ceph-mon[97934]: pgmap v532: 97 pgs: 97 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 18 MiB/s wr, 873 op/s 2023-07-21T18:58:36.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:58:36 smithi039 ceph-mon[96489]: pgmap v532: 97 pgs: 97 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 18 MiB/s wr, 873 op/s 2023-07-21T18:58:36.677 INFO:tasks.workunit.client.0.smithi039.stdout: 10 131769 31.92 MB/sec execute 109 sec latency 1276.586 ms 2023-07-21T18:58:37.677 INFO:tasks.workunit.client.0.smithi039.stdout: 10 132304 31.95 MB/sec execute 110 sec latency 2276.634 ms 2023-07-21T18:58:38.564 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:38.554+0000 7f4f29ffb700 1 -- 172.21.15.39:0/4174284560 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56206369bff0 con 0x5620637fd980 2023-07-21T18:58:38.565 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:38.554+0000 7f4f29ffb700 10 client.26864 handle_command_reply: tid=0 2023-07-21T18:58:38.565 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:38.554+0000 7f4f54c8eb80 1 client.26864 shutdown 2023-07-21T18:58:38.565 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:38.554+0000 7f4f54c8eb80 1 -- 172.21.15.39:0/4174284560 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620635eb6b0 msgr2=0x56206379c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:38.565 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:38.554+0000 7f4f54c8eb80 1 --2- 172.21.15.39:0/4174284560 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620635eb6b0 0x56206379c680 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4f38000c00 tx=0x7f4f38009ea0 comp rx=0 tx=0).stop 2023-07-21T18:58:38.565 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:38.554+0000 7f4f54c8eb80 1 -- 172.21.15.39:0/4174284560 shutdown_connections 2023-07-21T18:58:38.565 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:38.554+0000 7f4f54c8eb80 1 --2- 172.21.15.39:0/4174284560 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5620637fd980 0x5620637ffdb0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:38.566 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:38.554+0000 7f4f54c8eb80 1 --2- 172.21.15.39:0/4174284560 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620635eb6b0 0x56206379c680 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:38.566 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:38.554+0000 7f4f29ffb700 0 client.26864 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:58:38.566 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:38.554+0000 7f4f54c8eb80 1 --2- 172.21.15.39:0/4174284560 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620637952d0 0x5620637f5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:38.566 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:38.554+0000 7f4f54c8eb80 1 --2- 172.21.15.39:0/4174284560 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56206378f580 0x56206379cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:38.566 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:38.554+0000 7f4f54c8eb80 1 -- 172.21.15.39:0/4174284560 >> 172.21.15.39:0/4174284560 conn(0x562063784be0 msgr2=0x562063796000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:58:38.566 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:38.554+0000 7f4f54c8eb80 1 -- 172.21.15.39:0/4174284560 shutdown_connections 2023-07-21T18:58:38.566 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:38.554+0000 7f4f54c8eb80 1 -- 172.21.15.39:0/4174284560 wait complete. 2023-07-21T18:58:38.567 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:38.554+0000 7f4f54c8eb80 20 client.26864 trim_cache size 0 max 16384 2023-07-21T18:58:38.567 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:58:38.567 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:58:38.567 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:58:38.567 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:58:38.568 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:38.555+0000 7f4f48b08700 1 -- 172.21.15.39:0/3734964054 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f4f2c0a12c0 msgr2=0x7f4f2c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:38.568 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:38.555+0000 7f4f48b08700 1 --2- 172.21.15.39:0/3734964054 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f4f2c0a12c0 0x7f4f2c0a3770 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4f38005fd0 tx=0x7f4f38009310 comp rx=0 tx=0).stop 2023-07-21T18:58:38.568 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:38.555+0000 7f4f48b08700 1 -- 172.21.15.39:0/3734964054 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f4410a1b0 msgr2=0x7f4f4410ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:38.568 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:38.555+0000 7f4f48b08700 1 --2- 172.21.15.39:0/3734964054 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f4410a1b0 0x7f4f4410ae40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4f34009230 tx=0x7f4f3400fa00 comp rx=0 tx=0).stop 2023-07-21T18:58:38.569 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:38.555+0000 7f4f48b08700 1 -- 172.21.15.39:0/3734964054 shutdown_connections 2023-07-21T18:58:38.569 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:38.555+0000 7f4f48b08700 1 --2- 172.21.15.39:0/3734964054 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f44114c80 0x7f4f441b0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:38.569 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:38.555+0000 7f4f48b08700 1 --2- 172.21.15.39:0/3734964054 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f4406ce50 0x7f4f4410a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:38.569 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:38.555+0000 7f4f48b08700 1 --2- 172.21.15.39:0/3734964054 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f4f2c0a12c0 0x7f4f2c0a3770 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:38.569 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:38.555+0000 7f4f48b08700 1 --2- 172.21.15.39:0/3734964054 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f4410a1b0 0x7f4f4410ae40 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:38.569 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:38.555+0000 7f4f48b08700 1 -- 172.21.15.39:0/3734964054 >> 172.21.15.39:0/3734964054 conn(0x7f4f4406bd70 msgr2=0x7f4f44115370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:58:38.569 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:38.555+0000 7f4f48b08700 1 -- 172.21.15.39:0/3734964054 shutdown_connections 2023-07-21T18:58:38.569 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:38.555+0000 7f4f48b08700 1 -- 172.21.15.39:0/3734964054 wait complete. 2023-07-21T18:58:38.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:58:38 smithi193 ceph-mon[98204]: pgmap v533: 97 pgs: 97 active+clean; 1.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 21 MiB/s wr, 1.15k op/s 2023-07-21T18:58:38.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:58:38 smithi155 ceph-mon[97934]: pgmap v533: 97 pgs: 97 active+clean; 1.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 21 MiB/s wr, 1.15k op/s 2023-07-21T18:58:38.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:58:38 smithi039 ceph-mon[96489]: pgmap v533: 97 pgs: 97 active+clean; 1.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 21 MiB/s wr, 1.15k op/s 2023-07-21T18:58:38.677 INFO:tasks.workunit.client.0.smithi039.stdout: 10 132855 31.83 MB/sec execute 111 sec latency 3152.674 ms 2023-07-21T18:58:39.677 INFO:tasks.workunit.client.0.smithi039.stdout: 10 133554 31.88 MB/sec execute 112 sec latency 74.610 ms 2023-07-21T18:58:40.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:58:40 smithi193 ceph-mon[98204]: pgmap v534: 97 pgs: 97 active+clean; 1.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 5.7 MiB/s wr, 687 op/s 2023-07-21T18:58:40.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:58:40 smithi155 ceph-mon[97934]: pgmap v534: 97 pgs: 97 active+clean; 1.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 5.7 MiB/s wr, 687 op/s 2023-07-21T18:58:40.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:58:40 smithi039 ceph-mon[96489]: pgmap v534: 97 pgs: 97 active+clean; 1.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 5.7 MiB/s wr, 687 op/s 2023-07-21T18:58:40.677 INFO:tasks.workunit.client.0.smithi039.stdout: 10 134228 32.01 MB/sec execute 113 sec latency 104.480 ms 2023-07-21T18:58:41.677 INFO:tasks.workunit.client.0.smithi039.stdout: 10 134845 31.92 MB/sec execute 114 sec latency 78.631 ms 2023-07-21T18:58:42.607 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:58:42 smithi193 ceph-mon[98204]: pgmap v535: 97 pgs: 97 active+clean; 1.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 9.0 MiB/s wr, 933 op/s 2023-07-21T18:58:42.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:58:42 smithi155 ceph-mon[97934]: pgmap v535: 97 pgs: 97 active+clean; 1.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 9.0 MiB/s wr, 933 op/s 2023-07-21T18:58:42.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:58:42 smithi039 ceph-mon[96489]: pgmap v535: 97 pgs: 97 active+clean; 1.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 9.0 MiB/s wr, 933 op/s 2023-07-21T18:58:42.677 INFO:tasks.workunit.client.0.smithi039.stdout: 10 135575 32.04 MB/sec execute 115 sec latency 80.597 ms 2023-07-21T18:58:43.677 INFO:tasks.workunit.client.0.smithi039.stdout: 10 136305 32.14 MB/sec execute 116 sec latency 50.276 ms 2023-07-21T18:58:44.607 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:58:44 smithi193 ceph-mon[98204]: pgmap v536: 97 pgs: 97 active+clean; 1.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 6.3 MiB/s wr, 521 op/s 2023-07-21T18:58:44.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:58:44 smithi155 ceph-mon[97934]: pgmap v536: 97 pgs: 97 active+clean; 1.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 6.3 MiB/s wr, 521 op/s 2023-07-21T18:58:44.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:58:44 smithi039 ceph-mon[96489]: pgmap v536: 97 pgs: 97 active+clean; 1.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 6.3 MiB/s wr, 521 op/s 2023-07-21T18:58:44.677 INFO:tasks.workunit.client.0.smithi039.stdout: 10 137042 32.17 MB/sec execute 117 sec latency 54.852 ms 2023-07-21T18:58:45.677 INFO:tasks.workunit.client.0.smithi039.stdout: 10 137602 32.19 MB/sec execute 118 sec latency 77.057 ms 2023-07-21T18:58:46.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:58:46 smithi193 ceph-mon[98204]: pgmap v537: 97 pgs: 97 active+clean; 1.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 6.3 MiB/s wr, 521 op/s 2023-07-21T18:58:46.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:58:46 smithi155 ceph-mon[97934]: pgmap v537: 97 pgs: 97 active+clean; 1.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 6.3 MiB/s wr, 521 op/s 2023-07-21T18:58:46.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:58:46 smithi039 ceph-mon[96489]: pgmap v537: 97 pgs: 97 active+clean; 1.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 6.3 MiB/s wr, 521 op/s 2023-07-21T18:58:46.677 INFO:tasks.workunit.client.0.smithi039.stdout: 10 137995 32.05 MB/sec execute 119 sec latency 151.161 ms 2023-07-21T18:58:47.677 INFO:tasks.workunit.client.0.smithi039.stdout: 10 138662 32.04 MB/sec execute 120 sec latency 98.535 ms 2023-07-21T18:58:48.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:58:48 smithi193 ceph-mon[98204]: pgmap v538: 97 pgs: 97 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 9.2 MiB/s wr, 753 op/s 2023-07-21T18:58:48.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:58:48 smithi155 ceph-mon[97934]: pgmap v538: 97 pgs: 97 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 9.2 MiB/s wr, 753 op/s 2023-07-21T18:58:48.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:58:48 smithi039 ceph-mon[96489]: pgmap v538: 97 pgs: 97 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 9.2 MiB/s wr, 753 op/s 2023-07-21T18:58:48.677 INFO:tasks.workunit.client.0.smithi039.stdout: 10 139491 32.22 MB/sec execute 121 sec latency 61.929 ms 2023-07-21T18:58:49.677 INFO:tasks.workunit.client.0.smithi039.stdout: 10 140219 32.29 MB/sec execute 122 sec latency 52.666 ms 2023-07-21T18:58:50.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:58:50 smithi193 ceph-mon[98204]: pgmap v539: 97 pgs: 97 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 6.2 MiB/s wr, 478 op/s 2023-07-21T18:58:50.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:58:50 smithi155 ceph-mon[97934]: pgmap v539: 97 pgs: 97 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 6.2 MiB/s wr, 478 op/s 2023-07-21T18:58:50.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:58:50 smithi039 ceph-mon[96489]: pgmap v539: 97 pgs: 97 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 6.2 MiB/s wr, 478 op/s 2023-07-21T18:58:50.677 INFO:tasks.workunit.client.0.smithi039.stdout: 10 140797 32.28 MB/sec execute 123 sec latency 65.699 ms 2023-07-21T18:58:51.677 INFO:tasks.workunit.client.0.smithi039.stdout: 10 141248 32.19 MB/sec execute 124 sec latency 135.331 ms 2023-07-21T18:58:51.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.705+0000 7f5e5199c700 1 -- 172.21.15.39:0/2766240819 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e4c104a20 msgr2=0x7f5e4c104df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:51.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.705+0000 7f5e5199c700 1 --2- 172.21.15.39:0/2766240819 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e4c104a20 0x7f5e4c104df0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5e34009990 tx=0x7f5e34009ca0 comp rx=0 tx=0).stop 2023-07-21T18:58:51.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.705+0000 7f5e5199c700 1 -- 172.21.15.39:0/2766240819 shutdown_connections 2023-07-21T18:58:51.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.705+0000 7f5e5199c700 1 --2- 172.21.15.39:0/2766240819 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5e4c0ff3c0 0x7f5e4c10b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:51.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.705+0000 7f5e5199c700 1 --2- 172.21.15.39:0/2766240819 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e4c104a20 0x7f5e4c104df0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:51.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.705+0000 7f5e5199c700 1 --2- 172.21.15.39:0/2766240819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e4c0fea10 0x7f5e4c0fee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:51.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.705+0000 7f5e5199c700 1 -- 172.21.15.39:0/2766240819 >> 172.21.15.39:0/2766240819 conn(0x7f5e4c0fa4d0 msgr2=0x7f5e4c0fc8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:58:51.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.705+0000 7f5e5199c700 1 -- 172.21.15.39:0/2766240819 shutdown_connections 2023-07-21T18:58:51.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.705+0000 7f5e5199c700 1 -- 172.21.15.39:0/2766240819 wait complete. 2023-07-21T18:58:51.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.706+0000 7f5e5199c700 1 Processor -- start 2023-07-21T18:58:51.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.706+0000 7f5e5199c700 1 -- start start 2023-07-21T18:58:51.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.706+0000 7f5e5199c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e4c0fea10 0x7f5e4c10c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:51.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.706+0000 7f5e5199c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e4c0ff3c0 0x7f5e4c10c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:51.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.706+0000 7f5e5199c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5e4c104a20 0x7f5e4c1a3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:51.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.706+0000 7f5e5199c700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4c1a42a0 con 0x7f5e4c104a20 2023-07-21T18:58:51.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.706+0000 7f5e5199c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4c1a43e0 con 0x7f5e4c0fea10 2023-07-21T18:58:51.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.706+0000 7f5e5199c700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4c1a4550 con 0x7f5e4c0ff3c0 2023-07-21T18:58:51.720 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.707+0000 7f5e4affd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e4c0fea10 0x7f5e4c10c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:51.720 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.707+0000 7f5e4b7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5e4c104a20 0x7f5e4c1a3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:51.720 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.707+0000 7f5e4affd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e4c0fea10 0x7f5e4c10c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:46326/0 (socket says 172.21.15.39:46326) 2023-07-21T18:58:51.720 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.707+0000 7f5e4affd700 1 -- 172.21.15.39:0/43616126 learned_addr learned my addr 172.21.15.39:0/43616126 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:58:51.720 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.707+0000 7f5e4a7fc700 1 --2- 172.21.15.39:0/43616126 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e4c0ff3c0 0x7f5e4c10c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:51.720 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.707+0000 7f5e4affd700 1 -- 172.21.15.39:0/43616126 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e4c0ff3c0 msgr2=0x7f5e4c10c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:51.720 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.707+0000 7f5e4affd700 1 --2- 172.21.15.39:0/43616126 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e4c0ff3c0 0x7f5e4c10c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:51.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.707+0000 7f5e4affd700 1 -- 172.21.15.39:0/43616126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5e4c104a20 msgr2=0x7f5e4c1a3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:51.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.707+0000 7f5e4affd700 1 --2- 172.21.15.39:0/43616126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5e4c104a20 0x7f5e4c1a3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:51.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.707+0000 7f5e4affd700 1 -- 172.21.15.39:0/43616126 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e340095f0 con 0x7f5e4c0fea10 2023-07-21T18:58:51.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.707+0000 7f5e4b7fe700 1 --2- 172.21.15.39:0/43616126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5e4c104a20 0x7f5e4c1a3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:58:51.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.707+0000 7f5e4affd700 1 --2- 172.21.15.39:0/43616126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e4c0fea10 0x7f5e4c10c2c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5e34004760 tx=0x7f5e34009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:58:51.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.708+0000 7f5e5099a700 1 -- 172.21.15.39:0/43616126 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3401d070 con 0x7f5e4c0fea10 2023-07-21T18:58:51.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.708+0000 7f5e5099a700 1 -- 172.21.15.39:0/43616126 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e34022470 con 0x7f5e4c0fea10 2023-07-21T18:58:51.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.708+0000 7f5e5099a700 1 -- 172.21.15.39:0/43616126 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e34017650 con 0x7f5e4c0fea10 2023-07-21T18:58:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.708+0000 7f5e5199c700 1 -- 172.21.15.39:0/43616126 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e4c1a47d0 con 0x7f5e4c0fea10 2023-07-21T18:58:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.708+0000 7f5e5199c700 1 -- 172.21.15.39:0/43616126 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e4c1a4bf0 con 0x7f5e4c0fea10 2023-07-21T18:58:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.711+0000 7f5e5099a700 1 -- 172.21.15.39:0/43616126 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f5e340225e0 con 0x7f5e4c0fea10 2023-07-21T18:58:51.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.713+0000 7f5e5db22b80 1 Processor -- start 2023-07-21T18:58:51.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.713+0000 7f5e5099a700 1 --2- 172.21.15.39:0/43616126 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f5e380a1340 0x7f5e380a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:51.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.714+0000 7f5e5099a700 1 -- 172.21.15.39:0/43616126 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f5e340cdec0 con 0x7f5e4c0fea10 2023-07-21T18:58:51.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.714+0000 7f5e5db22b80 1 -- start start 2023-07-21T18:58:51.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.714+0000 7f5e4b7fe700 1 --2- 172.21.15.39:0/43616126 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f5e380a1340 0x7f5e380a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:51.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.714+0000 7f5e5db22b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9945bd6b0 0x55a994766d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:51.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.714+0000 7f5e5db22b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9947672d0 0x55a994761040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:51.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.714+0000 7f5e5db22b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a994761580 0x55a9947619d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:51.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.714+0000 7f5e5db22b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a994666c50 con 0x55a994761580 2023-07-21T18:58:51.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.714+0000 7f5e5db22b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9946b99e0 con 0x55a9945bd6b0 2023-07-21T18:58:51.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.714+0000 7f5e5db22b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9946b2e70 con 0x55a9947672d0 2023-07-21T18:58:51.726 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.715+0000 7f5e4a7fc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9947672d0 0x55a994761040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:51.726 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.715+0000 7f5e4a7fc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9947672d0 0x55a994761040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:52292/0 (socket says 172.21.15.39:52292) 2023-07-21T18:58:51.726 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.715+0000 7f5e4a7fc700 1 -- 172.21.15.39:0/3079347437 learned_addr learned my addr 172.21.15.39:0/3079347437 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:58:51.726 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.715+0000 7f5e4b7fe700 1 --2- 172.21.15.39:0/43616126 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f5e380a1340 0x7f5e380a37f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5e44003e30 tx=0x7f5e4400b040 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:58:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.715+0000 7f5e4a7fc700 1 -- 172.21.15.39:0/3079347437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9945bd6b0 msgr2=0x55a994766d90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T18:58:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.715+0000 7f5e4a7fc700 1 --2- 172.21.15.39:0/3079347437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9945bd6b0 0x55a994766d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.715+0000 7f5e4a7fc700 1 -- 172.21.15.39:0/3079347437 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a994761580 msgr2=0x55a9947619d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:58:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.715+0000 7f5e4a7fc700 1 --2- 172.21.15.39:0/3079347437 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a994761580 0x55a9947619d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.715+0000 7f5e4a7fc700 1 -- 172.21.15.39:0/3079347437 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9944e0d80 con 0x55a9947672d0 2023-07-21T18:58:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.715+0000 7f5e4a7fc700 1 --2- 172.21.15.39:0/3079347437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9947672d0 0x55a994761040 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c009d20 tx=0x7f5e3c011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5f1d7f3396d833e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:58:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.716+0000 7f5e41ffb700 1 -- 172.21.15.39:0/3079347437 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c004a30 con 0x55a9947672d0 2023-07-21T18:58:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.716+0000 7f5e41ffb700 1 -- 172.21.15.39:0/3079347437 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e3c0054e0 con 0x55a9947672d0 2023-07-21T18:58:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.716+0000 7f5e41ffb700 1 -- 172.21.15.39:0/3079347437 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c019840 con 0x55a9947672d0 2023-07-21T18:58:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.717+0000 7f5e5db22b80 1 -- 172.21.15.39:0/3079347437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9947672d0 msgr2=0x55a994761040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.717+0000 7f5e5db22b80 1 --2- 172.21.15.39:0/3079347437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9947672d0 0x55a994761040 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c009d20 tx=0x7f5e3c011720 comp rx=0 tx=0).stop 2023-07-21T18:58:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.717+0000 7f5e5db22b80 1 -- 172.21.15.39:0/3079347437 shutdown_connections 2023-07-21T18:58:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.717+0000 7f5e5db22b80 1 --2- 172.21.15.39:0/3079347437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9945bd6b0 0x55a994766d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.717+0000 7f5e5db22b80 1 --2- 172.21.15.39:0/3079347437 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a994761580 0x55a9947619d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.717+0000 7f5e5db22b80 1 --2- 172.21.15.39:0/3079347437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9947672d0 0x55a994761040 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.717+0000 7f5e5db22b80 1 -- 172.21.15.39:0/3079347437 >> 172.21.15.39:0/3079347437 conn(0x55a994756be0 msgr2=0x55a99475bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:58:51.730 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.717+0000 7f5e5db22b80 1 -- 172.21.15.39:0/3079347437 shutdown_connections 2023-07-21T18:58:51.730 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.717+0000 7f5e5db22b80 1 -- 172.21.15.39:0/3079347437 wait complete. 2023-07-21T18:58:51.730 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.718+0000 7f5e5db22b80 1 -- start start 2023-07-21T18:58:51.730 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.718+0000 7f5e5db22b80 1 Processor -- start 2023-07-21T18:58:51.730 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.718+0000 7f5e5db22b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9945bd6b0 0x55a99476a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:51.731 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.719+0000 7f5e5db22b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a994761580 0x55a99476aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:51.731 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.719+0000 7f5e5db22b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a9947672d0 0x55a99476b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:51.731 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.719+0000 7f5e5db22b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a994652a60 con 0x55a9947672d0 2023-07-21T18:58:51.731 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.719+0000 7f5e5db22b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a994652760 con 0x55a994761580 2023-07-21T18:58:51.732 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.719+0000 7f5e5db22b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9945368c0 con 0x55a9945bd6b0 2023-07-21T18:58:51.732 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.719+0000 7f5e5db22b80 10 client.0 _ll_register_callbacks cb 0x55a994650700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:58:51.732 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.719+0000 7f5e4affd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9945bd6b0 0x55a99476a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:51.732 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.719+0000 7f5e4affd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9945bd6b0 0x55a99476a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:52296/0 (socket says 172.21.15.39:52296) 2023-07-21T18:58:51.732 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.719+0000 7f5e4affd700 1 -- 172.21.15.39:0/3664380791 learned_addr learned my addr 172.21.15.39:0/3664380791 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:58:51.732 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.719+0000 7f5e4a7fc700 1 --2- 172.21.15.39:0/3664380791 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a994761580 0x55a99476aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:51.732 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.719+0000 7f5e4affd700 1 -- 172.21.15.39:0/3664380791 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a994761580 msgr2=0x55a99476aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:58:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.719+0000 7f5e4affd700 1 --2- 172.21.15.39:0/3664380791 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a994761580 0x55a99476aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.719+0000 7f5e4affd700 1 -- 172.21.15.39:0/3664380791 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a9947672d0 msgr2=0x55a99476b3e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T18:58:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.719+0000 7f5e4affd700 1 --2- 172.21.15.39:0/3664380791 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a9947672d0 0x55a99476b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.719+0000 7f5e4affd700 1 -- 172.21.15.39:0/3664380791 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e3c009980 con 0x55a9945bd6b0 2023-07-21T18:58:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.719+0000 7f5e4affd700 1 --2- 172.21.15.39:0/3664380791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9945bd6b0 0x55a99476a960 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5e340974c0 tx=0x7f5e3409ab80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2a7c3763d878c53 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:58:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.719+0000 7f5e2affd700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2023-07-21T18:58:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.720+0000 7f5e2affd700 1 -- 172.21.15.39:0/3664380791 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3409eff0 con 0x55a9945bd6b0 2023-07-21T18:58:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.720+0000 7f5e2affd700 1 -- 172.21.15.39:0/3664380791 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f5e340a3380 con 0x55a9945bd6b0 2023-07-21T18:58:51.734 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.720+0000 7f5e5db22b80 1 -- 172.21.15.39:0/3664380791 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a99460c490 con 0x55a9945bd6b0 2023-07-21T18:58:51.734 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.723+0000 7f5e2affd700 1 -- 172.21.15.39:0/3664380791 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e3409a4b0 con 0x55a9945bd6b0 2023-07-21T18:58:51.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.723+0000 7f5e5db22b80 10 client.26905 fetch_fsmap learned FSMap version 92 2023-07-21T18:58:51.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.723+0000 7f5e5db22b80 1 -- 172.21.15.39:0/3664380791 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=4+}) v3 -- 0x55a9944c9c00 con 0x55a9945bd6b0 2023-07-21T18:58:51.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.723+0000 7f5e2affd700 1 -- 172.21.15.39:0/3664380791 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a99460c490 con 0x55a9945bd6b0 2023-07-21T18:58:51.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.723+0000 7f5e2affd700 1 -- 172.21.15.39:0/3664380791 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 92) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f5e340a2ef0 con 0x55a9945bd6b0 2023-07-21T18:58:51.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.723+0000 7f5e5db22b80 10 client.26905 fetch_fsmap finished waiting for FSMap version 92 2023-07-21T18:58:51.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.723+0000 7f5e5db22b80 10 client.26905 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:58:51.736 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.723+0000 7f5e5db22b80 20 client.26905 populate_metadata read hostname 'smithi039' 2023-07-21T18:58:51.736 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.723+0000 7f5e5db22b80 1 --2- 172.21.15.39:0/3664380791 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55a99477a8b0 0x55a99477ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:51.736 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.723+0000 7f5e5db22b80 4 client.26905 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:58:51.736 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.723+0000 7f5e5db22b80 1 -- 172.21.15.39:0/3664380791 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a9946abe50 con 0x55a99477a8b0 2023-07-21T18:58:51.736 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.726+0000 7f5e4a7fc700 1 --2- 172.21.15.39:0/3664380791 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55a99477a8b0 0x55a99477ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:51.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.726+0000 7f5e4a7fc700 1 --2- 172.21.15.39:0/3664380791 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55a99477a8b0 0x55a99477ccc0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6bd541894c69766 server_cookie=feab8711e3b71891 in_seq=0 out_seq=0 2023-07-21T18:58:51.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.726+0000 7f5e2affd700 10 client.26905 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:58:51.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.727+0000 7f5e2affd700 1 -- 172.21.15.39:0/3664380791 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a9946abe50 con 0x55a99477a8b0 2023-07-21T18:58:51.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.727+0000 7f5e2affd700 10 client.26905 handle_command_reply: tid=0 2023-07-21T18:58:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.727+0000 7f5e5db22b80 1 client.26905 shutdown 2023-07-21T18:58:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.728+0000 7f5e5db22b80 1 -- 172.21.15.39:0/3664380791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9945bd6b0 msgr2=0x55a99476a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.728+0000 7f5e5db22b80 1 --2- 172.21.15.39:0/3664380791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9945bd6b0 0x55a99476a960 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5e340974c0 tx=0x7f5e3409ab80 comp rx=0 tx=0).stop 2023-07-21T18:58:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.728+0000 7f5e5db22b80 1 -- 172.21.15.39:0/3664380791 shutdown_connections 2023-07-21T18:58:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.728+0000 7f5e5db22b80 1 --2- 172.21.15.39:0/3664380791 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55a99477a8b0 0x55a99477ccc0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.728+0000 7f5e5db22b80 1 --2- 172.21.15.39:0/3664380791 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a9947672d0 0x55a99476b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.728+0000 7f5e5db22b80 1 --2- 172.21.15.39:0/3664380791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9945bd6b0 0x55a99476a960 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.728+0000 7f5e2affd700 0 client.26905 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:58:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.728+0000 7f5e5db22b80 1 --2- 172.21.15.39:0/3664380791 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a994761580 0x55a99476aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.728+0000 7f5e5db22b80 1 -- 172.21.15.39:0/3664380791 >> 172.21.15.39:0/3664380791 conn(0x55a994756be0 msgr2=0x55a99475ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:58:51.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.728+0000 7f5e5db22b80 1 -- 172.21.15.39:0/3664380791 shutdown_connections 2023-07-21T18:58:51.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.728+0000 7f5e5db22b80 1 -- 172.21.15.39:0/3664380791 wait complete. 2023-07-21T18:58:51.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.728+0000 7f5e5db22b80 20 client.26905 trim_cache size 0 max 16384 2023-07-21T18:58:51.761 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.751+0000 7f5e5db22b80 1 Processor -- start 2023-07-21T18:58:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.752+0000 7f5e5db22b80 1 -- start start 2023-07-21T18:58:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.752+0000 7f5e5db22b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a994756be0 0x55a994761040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.752+0000 7f5e5db22b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a9945bd6b0 0x55a994771c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.752+0000 7f5e5db22b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a994761580 0x55a9947661f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.752+0000 7f5e5db22b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a994766a10 con 0x55a9945bd6b0 2023-07-21T18:58:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.752+0000 7f5e5db22b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9947676a0 con 0x55a994761580 2023-07-21T18:58:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.752+0000 7f5e5db22b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a99476bf40 con 0x55a994756be0 2023-07-21T18:58:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.752+0000 7f5e4b7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a994761580 0x55a9947661f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.752+0000 7f5e4affd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a994756be0 0x55a994761040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.752+0000 7f5e4b7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a994761580 0x55a9947661f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:46348/0 (socket says 172.21.15.39:46348) 2023-07-21T18:58:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.752+0000 7f5e4affd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a994756be0 0x55a994761040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:52302/0 (socket says 172.21.15.39:52302) 2023-07-21T18:58:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.752+0000 7f5e4b7fe700 1 -- 172.21.15.39:0/2494478024 learned_addr learned my addr 172.21.15.39:0/2494478024 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:58:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.752+0000 7f5e4b7fe700 1 -- 172.21.15.39:0/2494478024 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a994756be0 msgr2=0x55a994761040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:58:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.753+0000 7f5e4b7fe700 1 --2- 172.21.15.39:0/2494478024 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a994756be0 0x55a994761040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.753+0000 7f5e4b7fe700 1 -- 172.21.15.39:0/2494478024 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a9945bd6b0 msgr2=0x55a994771c70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T18:58:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.753+0000 7f5e4b7fe700 1 --2- 172.21.15.39:0/2494478024 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a9945bd6b0 0x55a994771c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.753+0000 7f5e4b7fe700 1 -- 172.21.15.39:0/2494478024 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9945071f0 con 0x55a994761580 2023-07-21T18:58:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.753+0000 7f5e4b7fe700 1 --2- 172.21.15.39:0/2494478024 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a994761580 0x55a9947661f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5e44003cf0 tx=0x7f5e44004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6e21095ec0c97 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:58:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.753+0000 7f5e427fc700 1 -- 172.21.15.39:0/2494478024 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4400ee90 con 0x55a994761580 2023-07-21T18:58:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.753+0000 7f5e427fc700 1 -- 172.21.15.39:0/2494478024 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e4400f040 con 0x55a994761580 2023-07-21T18:58:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.753+0000 7f5e427fc700 1 -- 172.21.15.39:0/2494478024 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e44022820 con 0x55a994761580 2023-07-21T18:58:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.754+0000 7f5e5db22b80 1 -- 172.21.15.39:0/2494478024 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a994761580 msgr2=0x55a9947661f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.754+0000 7f5e5db22b80 1 --2- 172.21.15.39:0/2494478024 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a994761580 0x55a9947661f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5e44003cf0 tx=0x7f5e44004b40 comp rx=0 tx=0).stop 2023-07-21T18:58:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.754+0000 7f5e5db22b80 1 -- 172.21.15.39:0/2494478024 shutdown_connections 2023-07-21T18:58:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.754+0000 7f5e5db22b80 1 --2- 172.21.15.39:0/2494478024 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a994761580 0x55a9947661f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.754+0000 7f5e5db22b80 1 --2- 172.21.15.39:0/2494478024 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a994756be0 0x55a994761040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.754+0000 7f5e5db22b80 1 --2- 172.21.15.39:0/2494478024 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a9945bd6b0 0x55a994771c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.754+0000 7f5e5db22b80 1 -- 172.21.15.39:0/2494478024 >> 172.21.15.39:0/2494478024 conn(0x55a994777290 msgr2=0x55a994765ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:58:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.754+0000 7f5e5db22b80 1 -- 172.21.15.39:0/2494478024 shutdown_connections 2023-07-21T18:58:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.754+0000 7f5e5db22b80 1 -- 172.21.15.39:0/2494478024 wait complete. 2023-07-21T18:58:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.755+0000 7f5e5db22b80 1 -- start start 2023-07-21T18:58:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.755+0000 7f5e5db22b80 1 Processor -- start 2023-07-21T18:58:51.768 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.755+0000 7f5e5db22b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9945bd6b0 0x55a9947cb100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:51.768 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.755+0000 7f5e5db22b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a994756be0 0x55a9947cb640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:51.769 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.755+0000 7f5e5db22b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a994761580 0x55a9947cf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:51.769 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.755+0000 7f5e5db22b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9947cf5a0 con 0x55a994761580 2023-07-21T18:58:51.769 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.755+0000 7f5e5db22b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9947cf710 con 0x55a9945bd6b0 2023-07-21T18:58:51.769 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.755+0000 7f5e5db22b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9947cf8d0 con 0x55a994756be0 2023-07-21T18:58:51.769 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.755+0000 7f5e5db22b80 10 client.0 _ll_register_callbacks cb 0x55a994650700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:58:51.769 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.756+0000 7f5e4a7fc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a994756be0 0x55a9947cb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:51.769 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.756+0000 7f5e4a7fc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a994756be0 0x55a9947cb640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:52308/0 (socket says 172.21.15.39:52308) 2023-07-21T18:58:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.756+0000 7f5e4a7fc700 1 -- 172.21.15.39:0/3792743595 learned_addr learned my addr 172.21.15.39:0/3792743595 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:58:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.756+0000 7f5e4affd700 1 --2- 172.21.15.39:0/3792743595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9945bd6b0 0x55a9947cb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.757+0000 7f5e4affd700 1 -- 172.21.15.39:0/3792743595 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a994756be0 msgr2=0x55a9947cb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:58:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.757+0000 7f5e4affd700 1 --2- 172.21.15.39:0/3792743595 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a994756be0 0x55a9947cb640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.757+0000 7f5e4affd700 1 -- 172.21.15.39:0/3792743595 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a994761580 msgr2=0x55a9947cf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:58:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.757+0000 7f5e4affd700 1 --2- 172.21.15.39:0/3792743595 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a994761580 0x55a9947cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.757+0000 7f5e4affd700 1 -- 172.21.15.39:0/3792743595 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e440047c0 con 0x55a9945bd6b0 2023-07-21T18:58:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.757+0000 7f5e4affd700 1 --2- 172.21.15.39:0/3792743595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9945bd6b0 0x55a9947cb100 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5e34022890 tx=0x7f5e3409aec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=748cde27c6686f5c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:58:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.757+0000 7f5e2affd700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-21T18:58:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.757+0000 7f5e2affd700 1 -- 172.21.15.39:0/3792743595 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e340a3380 con 0x55a9945bd6b0 2023-07-21T18:58:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.757+0000 7f5e2affd700 1 -- 172.21.15.39:0/3792743595 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f5e3409eff0 con 0x55a9945bd6b0 2023-07-21T18:58:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.757+0000 7f5e5db22b80 1 -- 172.21.15.39:0/3792743595 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5e3c0054e0 con 0x55a9945bd6b0 2023-07-21T18:58:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.759+0000 7f5e2affd700 1 -- 172.21.15.39:0/3792743595 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e340cb300 con 0x55a9945bd6b0 2023-07-21T18:58:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.759+0000 7f5e5db22b80 10 client.26879 fetch_fsmap learned FSMap version 92 2023-07-21T18:58:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.759+0000 7f5e5db22b80 1 -- 172.21.15.39:0/3792743595 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=4+}) v3 -- 0x55a9944df250 con 0x55a9945bd6b0 2023-07-21T18:58:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.761+0000 7f5e2affd700 1 -- 172.21.15.39:0/3792743595 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e3c0054e0 con 0x55a9945bd6b0 2023-07-21T18:58:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.762+0000 7f5e2affd700 1 -- 172.21.15.39:0/3792743595 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 92) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f5e340cb020 con 0x55a9945bd6b0 2023-07-21T18:58:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.762+0000 7f5e5db22b80 10 client.26879 fetch_fsmap finished waiting for FSMap version 92 2023-07-21T18:58:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.762+0000 7f5e5db22b80 10 client.26879 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:58:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.762+0000 7f5e5db22b80 20 client.26879 populate_metadata read hostname 'smithi039' 2023-07-21T18:58:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.762+0000 7f5e5db22b80 1 --2- 172.21.15.39:0/3792743595 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55a9947672d0 0x55a9947d9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.762+0000 7f5e5db22b80 4 client.26879 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:58:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.762+0000 7f5e5db22b80 1 -- 172.21.15.39:0/3792743595 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a99466dff0 con 0x55a9947672d0 2023-07-21T18:58:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.763+0000 7f5e4a7fc700 1 --2- 172.21.15.39:0/3792743595 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55a9947672d0 0x55a9947d9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.765+0000 7f5e4a7fc700 1 --2- 172.21.15.39:0/3792743595 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55a9947672d0 0x55a9947d9e40 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70c509166c9cdfec server_cookie=8063e35bba39da30 in_seq=0 out_seq=0 2023-07-21T18:58:51.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.765+0000 7f5e2affd700 10 client.26879 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:58:51.973 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.962+0000 7f5e2affd700 1 -- 172.21.15.39:0/3792743595 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a99466dff0 con 0x55a9947672d0 2023-07-21T18:58:51.973 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.963+0000 7f5e2affd700 10 client.26879 handle_command_reply: tid=0 2023-07-21T18:58:51.973 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.963+0000 7f5e5db22b80 1 client.26879 shutdown 2023-07-21T18:58:51.973 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.963+0000 7f5e5db22b80 1 -- 172.21.15.39:0/3792743595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9945bd6b0 msgr2=0x55a9947cb100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:51.974 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.963+0000 7f5e5db22b80 1 --2- 172.21.15.39:0/3792743595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9945bd6b0 0x55a9947cb100 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5e34022890 tx=0x7f5e3409aec0 comp rx=0 tx=0).stop 2023-07-21T18:58:51.974 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.963+0000 7f5e5db22b80 1 -- 172.21.15.39:0/3792743595 shutdown_connections 2023-07-21T18:58:51.975 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.963+0000 7f5e5db22b80 1 --2- 172.21.15.39:0/3792743595 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55a9947672d0 0x55a9947d9e40 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:51.975 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.963+0000 7f5e5db22b80 1 --2- 172.21.15.39:0/3792743595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9945bd6b0 0x55a9947cb100 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:51.975 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.963+0000 7f5e2affd700 0 client.26879 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:58:51.975 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.963+0000 7f5e5db22b80 1 --2- 172.21.15.39:0/3792743595 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a994761580 0x55a9947cf030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:51.975 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.963+0000 7f5e5db22b80 1 --2- 172.21.15.39:0/3792743595 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a994756be0 0x55a9947cb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:51.975 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.963+0000 7f5e5db22b80 1 -- 172.21.15.39:0/3792743595 >> 172.21.15.39:0/3792743595 conn(0x55a994777290 msgr2=0x55a9947678a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:58:51.975 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.963+0000 7f5e5db22b80 1 -- 172.21.15.39:0/3792743595 shutdown_connections 2023-07-21T18:58:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.963+0000 7f5e5db22b80 1 -- 172.21.15.39:0/3792743595 wait complete. 2023-07-21T18:58:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.964+0000 7f5e5db22b80 20 client.26879 trim_cache size 0 max 16384 2023-07-21T18:58:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:58:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:58:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:58:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:58:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.964+0000 7f5e5199c700 1 -- 172.21.15.39:0/43616126 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f5e380a1340 msgr2=0x7f5e380a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.964+0000 7f5e5199c700 1 --2- 172.21.15.39:0/43616126 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f5e380a1340 0x7f5e380a37f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5e44003e30 tx=0x7f5e4400b040 comp rx=0 tx=0).stop 2023-07-21T18:58:51.978 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.964+0000 7f5e5199c700 1 -- 172.21.15.39:0/43616126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e4c0fea10 msgr2=0x7f5e4c10c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:51.978 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.964+0000 7f5e5199c700 1 --2- 172.21.15.39:0/43616126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e4c0fea10 0x7f5e4c10c2c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5e34004760 tx=0x7f5e34009580 comp rx=0 tx=0).stop 2023-07-21T18:58:51.978 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.964+0000 7f5e5199c700 1 -- 172.21.15.39:0/43616126 shutdown_connections 2023-07-21T18:58:51.978 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.964+0000 7f5e5199c700 1 --2- 172.21.15.39:0/43616126 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f5e380a1340 0x7f5e380a37f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:51.978 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.964+0000 7f5e5199c700 1 --2- 172.21.15.39:0/43616126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e4c0fea10 0x7f5e4c10c2c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:51.978 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.964+0000 7f5e5199c700 1 --2- 172.21.15.39:0/43616126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5e4c104a20 0x7f5e4c1a3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:51.978 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.964+0000 7f5e5199c700 1 --2- 172.21.15.39:0/43616126 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e4c0ff3c0 0x7f5e4c10c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:51.978 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.964+0000 7f5e5199c700 1 -- 172.21.15.39:0/43616126 >> 172.21.15.39:0/43616126 conn(0x7f5e4c0fa4d0 msgr2=0x7f5e4c0fba60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:58:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.965+0000 7f5e5199c700 1 -- 172.21.15.39:0/43616126 shutdown_connections 2023-07-21T18:58:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:58:51.965+0000 7f5e5199c700 1 -- 172.21.15.39:0/43616126 wait complete. 2023-07-21T18:58:52.607 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:58:52 smithi193 ceph-mon[98204]: pgmap v540: 97 pgs: 97 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 9.2 MiB/s wr, 701 op/s 2023-07-21T18:58:52.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:58:52 smithi155 ceph-mon[97934]: pgmap v540: 97 pgs: 97 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 9.2 MiB/s wr, 701 op/s 2023-07-21T18:58:52.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:58:52 smithi039 ceph-mon[96489]: pgmap v540: 97 pgs: 97 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 9.2 MiB/s wr, 701 op/s 2023-07-21T18:58:52.677 INFO:tasks.workunit.client.0.smithi039.stdout: 10 141712 32.10 MB/sec execute 125 sec latency 242.922 ms 2023-07-21T18:58:53.678 INFO:tasks.workunit.client.0.smithi039.stdout: 10 142462 32.12 MB/sec execute 126 sec latency 99.183 ms 2023-07-21T18:58:54.607 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:58:54 smithi193 ceph-mon[98204]: pgmap v541: 97 pgs: 97 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.9 MiB/s wr, 455 op/s 2023-07-21T18:58:54.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:58:54 smithi155 ceph-mon[97934]: pgmap v541: 97 pgs: 97 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.9 MiB/s wr, 455 op/s 2023-07-21T18:58:54.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:58:54 smithi039 ceph-mon[96489]: pgmap v541: 97 pgs: 97 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.9 MiB/s wr, 455 op/s 2023-07-21T18:58:54.677 INFO:tasks.workunit.client.0.smithi039.stdout: 10 143319 32.33 MB/sec execute 127 sec latency 41.023 ms 2023-07-21T18:58:55.678 INFO:tasks.workunit.client.0.smithi039.stdout: 10 143550 32.17 MB/sec execute 128 sec latency 731.625 ms 2023-07-21T18:58:56.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:58:56 smithi193 ceph-mon[98204]: pgmap v542: 97 pgs: 97 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.9 MiB/s wr, 455 op/s 2023-07-21T18:58:56.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:58:56 smithi155 ceph-mon[97934]: pgmap v542: 97 pgs: 97 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.9 MiB/s wr, 455 op/s 2023-07-21T18:58:56.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:58:56 smithi039 ceph-mon[96489]: pgmap v542: 97 pgs: 97 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.9 MiB/s wr, 455 op/s 2023-07-21T18:58:56.678 INFO:tasks.workunit.client.0.smithi039.stdout: 10 143690 31.97 MB/sec execute 129 sec latency 959.377 ms 2023-07-21T18:58:57.678 INFO:tasks.workunit.client.0.smithi039.stdout: 10 143937 31.84 MB/sec execute 130 sec latency 191.054 ms 2023-07-21T18:58:58.607 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:58:58 smithi193 ceph-mon[98204]: pgmap v543: 97 pgs: 97 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 19 MiB/s wr, 908 op/s 2023-07-21T18:58:58.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:58:58 smithi155 ceph-mon[97934]: pgmap v543: 97 pgs: 97 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 19 MiB/s wr, 908 op/s 2023-07-21T18:58:58.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:58:58 smithi039 ceph-mon[96489]: pgmap v543: 97 pgs: 97 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 19 MiB/s wr, 908 op/s 2023-07-21T18:58:58.678 INFO:tasks.workunit.client.0.smithi039.stdout: 10 144429 31.83 MB/sec execute 131 sec latency 86.931 ms 2023-07-21T18:58:59.678 INFO:tasks.workunit.client.0.smithi039.stdout: 10 144921 31.75 MB/sec execute 132 sec latency 95.497 ms 2023-07-21T18:59:00.607 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:59:00 smithi193 ceph-mon[98204]: pgmap v544: 97 pgs: 97 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 16 MiB/s wr, 676 op/s 2023-07-21T18:59:00.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:59:00 smithi155 ceph-mon[97934]: pgmap v544: 97 pgs: 97 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 16 MiB/s wr, 676 op/s 2023-07-21T18:59:00.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:59:00 smithi039 ceph-mon[96489]: pgmap v544: 97 pgs: 97 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 16 MiB/s wr, 676 op/s 2023-07-21T18:59:00.678 INFO:tasks.workunit.client.0.smithi039.stdout: 10 145515 31.68 MB/sec execute 133 sec latency 88.595 ms 2023-07-21T18:59:01.678 INFO:tasks.workunit.client.0.smithi039.stdout: 10 145878 31.56 MB/sec execute 134 sec latency 199.347 ms 2023-07-21T18:59:02.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:59:02 smithi193 ceph-mon[98204]: pgmap v545: 97 pgs: 97 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 18 MiB/s wr, 825 op/s 2023-07-21T18:59:02.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:59:02 smithi155 ceph-mon[97934]: pgmap v545: 97 pgs: 97 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 18 MiB/s wr, 825 op/s 2023-07-21T18:59:02.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:59:02 smithi039 ceph-mon[96489]: pgmap v545: 97 pgs: 97 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 18 MiB/s wr, 825 op/s 2023-07-21T18:59:02.678 INFO:tasks.workunit.client.0.smithi039.stdout: 10 146166 31.48 MB/sec execute 135 sec latency 199.979 ms 2023-07-21T18:59:03.678 INFO:tasks.workunit.client.0.smithi039.stdout: 10 146930 31.58 MB/sec execute 136 sec latency 72.901 ms 2023-07-21T18:59:04.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:59:04 smithi193 ceph-mon[98204]: pgmap v546: 97 pgs: 97 active+clean; 1.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 19 MiB/s wr, 902 op/s 2023-07-21T18:59:04.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:59:04 smithi155 ceph-mon[97934]: pgmap v546: 97 pgs: 97 active+clean; 1.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 19 MiB/s wr, 902 op/s 2023-07-21T18:59:04.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:59:04 smithi039 ceph-mon[96489]: pgmap v546: 97 pgs: 97 active+clean; 1.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 19 MiB/s wr, 902 op/s 2023-07-21T18:59:04.678 INFO:tasks.workunit.client.0.smithi039.stdout: 10 147499 31.63 MB/sec execute 137 sec latency 68.762 ms 2023-07-21T18:59:05.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.143+0000 7f8130018700 1 -- 172.21.15.39:0/4061172404 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8128106340 msgr2=0x7f81281067b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:59:05.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.143+0000 7f8130018700 1 --2- 172.21.15.39:0/4061172404 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8128106340 0x7f81281067b0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f8118009990 tx=0x7f8118009ca0 comp rx=0 tx=0).stop 2023-07-21T18:59:05.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.145+0000 7f8130018700 1 -- 172.21.15.39:0/4061172404 shutdown_connections 2023-07-21T18:59:05.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.145+0000 7f8130018700 1 --2- 172.21.15.39:0/4061172404 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f812810c1c0 0x7f812810c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:05.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.145+0000 7f8130018700 1 --2- 172.21.15.39:0/4061172404 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8128106cf0 0x7f8128112f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:05.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.145+0000 7f8130018700 1 --2- 172.21.15.39:0/4061172404 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8128106340 0x7f81281067b0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:05.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.145+0000 7f8130018700 1 -- 172.21.15.39:0/4061172404 >> 172.21.15.39:0/4061172404 conn(0x7f8128075a20 msgr2=0x7f8128077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:59:05.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.146+0000 7f8130018700 1 -- 172.21.15.39:0/4061172404 shutdown_connections 2023-07-21T18:59:05.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.146+0000 7f8130018700 1 -- 172.21.15.39:0/4061172404 wait complete. 2023-07-21T18:59:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.147+0000 7f8130018700 1 Processor -- start 2023-07-21T18:59:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.147+0000 7f8130018700 1 -- start start 2023-07-21T18:59:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.147+0000 7f8130018700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8128106340 0x7f8128080810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.147+0000 7f8130018700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8128106cf0 0x7f8128080d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.147+0000 7f8130018700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f812810c1c0 0x7f8128079610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.147+0000 7f8130018700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8128079de0 con 0x7f8128106340 2023-07-21T18:59:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.147+0000 7f8130018700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8128079f50 con 0x7f8128106cf0 2023-07-21T18:59:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.147+0000 7f8130018700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f812807a110 con 0x7f812810c1c0 2023-07-21T18:59:05.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.149+0000 7f812ddb4700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8128106340 0x7f8128080810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:05.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.149+0000 7f812ddb4700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8128106340 0x7f8128080810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33768/0 (socket says 172.21.15.39:33768) 2023-07-21T18:59:05.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.149+0000 7f812ddb4700 1 -- 172.21.15.39:0/1555449244 learned_addr learned my addr 172.21.15.39:0/1555449244 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:59:05.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.149+0000 7f812ddb4700 1 -- 172.21.15.39:0/1555449244 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f812810c1c0 msgr2=0x7f8128079610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:59:05.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.149+0000 7f812ddb4700 1 --2- 172.21.15.39:0/1555449244 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f812810c1c0 0x7f8128079610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:05.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.149+0000 7f812ddb4700 1 -- 172.21.15.39:0/1555449244 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8128106cf0 msgr2=0x7f8128080d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:59:05.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.149+0000 7f812ddb4700 1 --2- 172.21.15.39:0/1555449244 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8128106cf0 0x7f8128080d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:05.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.149+0000 7f812ddb4700 1 -- 172.21.15.39:0/1555449244 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81180095f0 con 0x7f8128106340 2023-07-21T18:59:05.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.150+0000 7f812ddb4700 1 --2- 172.21.15.39:0/1555449244 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8128106340 0x7f8128080810 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f812400d950 tx=0x7f812400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:59:05.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.150+0000 7f811effd700 1 -- 172.21.15.39:0/1555449244 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8124009b40 con 0x7f8128106340 2023-07-21T18:59:05.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.150+0000 7f811effd700 1 -- 172.21.15.39:0/1555449244 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8124009ca0 con 0x7f8128106340 2023-07-21T18:59:05.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.150+0000 7f811effd700 1 -- 172.21.15.39:0/1555449244 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812400f620 con 0x7f8128106340 2023-07-21T18:59:05.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.151+0000 7f8130018700 1 -- 172.21.15.39:0/1555449244 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f812807a390 con 0x7f8128106340 2023-07-21T18:59:05.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.151+0000 7f8130018700 1 -- 172.21.15.39:0/1555449244 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f812807e580 con 0x7f8128106340 2023-07-21T18:59:05.166 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.156+0000 7f811effd700 1 -- 172.21.15.39:0/1555449244 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f8124010460 con 0x7f8128106340 2023-07-21T18:59:05.166 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.156+0000 7f813c19eb80 1 Processor -- start 2023-07-21T18:59:05.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.157+0000 7f813c19eb80 1 -- start start 2023-07-21T18:59:05.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.157+0000 7f811effd700 1 --2- 172.21.15.39:0/1555449244 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f81140a1450 0x7f81140a3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:05.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.157+0000 7f813c19eb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a6fdf466b0 0x55a6fe0efd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:05.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.157+0000 7f813c19eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a6fe0f02d0 0x55a6fe0ea040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:05.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.157+0000 7f813c19eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6fe0ea580 0x55a6fe0ea9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:05.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.157+0000 7f813c19eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6fdfefc50 con 0x55a6fe0f02d0 2023-07-21T18:59:05.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.157+0000 7f813c19eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6fe0429e0 con 0x55a6fe0ea580 2023-07-21T18:59:05.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.157+0000 7f813c19eb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6fe03be70 con 0x55a6fdf466b0 2023-07-21T18:59:05.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.157+0000 7f811effd700 1 -- 172.21.15.39:0/1555449244 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f81240ccc30 con 0x7f8128106340 2023-07-21T18:59:05.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.157+0000 7f812d5b3700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a6fe0f02d0 0x55a6fe0ea040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:05.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.158+0000 7f812d5b3700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a6fe0f02d0 0x55a6fe0ea040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33774/0 (socket says 172.21.15.39:33774) 2023-07-21T18:59:05.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.158+0000 7f812d5b3700 1 -- 172.21.15.39:0/2698123895 learned_addr learned my addr 172.21.15.39:0/2698123895 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:59:05.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.158+0000 7f812e5b5700 1 --2- 172.21.15.39:0/2698123895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6fe0ea580 0x55a6fe0ea9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:05.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.158+0000 7f812d5b3700 1 -- 172.21.15.39:0/2698123895 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a6fdf466b0 msgr2=0x55a6fe0efd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:59:05.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.158+0000 7f812d5b3700 1 --2- 172.21.15.39:0/2698123895 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a6fdf466b0 0x55a6fe0efd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:05.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.158+0000 7f812d5b3700 1 -- 172.21.15.39:0/2698123895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6fe0ea580 msgr2=0x55a6fe0ea9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:59:05.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.158+0000 7f812d5b3700 1 --2- 172.21.15.39:0/2698123895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6fe0ea580 0x55a6fe0ea9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:05.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.158+0000 7f812d5b3700 1 -- 172.21.15.39:0/2698123895 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6fde69d80 con 0x55a6fe0f02d0 2023-07-21T18:59:05.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.158+0000 7f812d5b3700 1 --2- 172.21.15.39:0/2698123895 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a6fe0f02d0 0x55a6fe0ea040 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f8118005b40 tx=0x7f8118005c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cb0a8a2515c979d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:59:05.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.159+0000 7f812e5b5700 1 --2- 172.21.15.39:0/1555449244 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f81140a1450 0x7f81140a3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:05.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.160+0000 7f8113fff700 1 -- 172.21.15.39:0/2698123895 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811801d070 con 0x55a6fe0f02d0 2023-07-21T18:59:05.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.160+0000 7f8113fff700 1 -- 172.21.15.39:0/2698123895 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8118003680 con 0x55a6fe0f02d0 2023-07-21T18:59:05.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.160+0000 7f8113fff700 1 -- 172.21.15.39:0/2698123895 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8118005320 con 0x55a6fe0f02d0 2023-07-21T18:59:05.172 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.161+0000 7f812e5b5700 1 --2- 172.21.15.39:0/1555449244 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f81140a1450 0x7f81140a3900 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8120009c30 tx=0x7f8120006ce0 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:59:05.172 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.161+0000 7f813c19eb80 1 -- 172.21.15.39:0/2698123895 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a6fe0f02d0 msgr2=0x55a6fe0ea040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:59:05.172 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.161+0000 7f813c19eb80 1 --2- 172.21.15.39:0/2698123895 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a6fe0f02d0 0x55a6fe0ea040 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f8118005b40 tx=0x7f8118005c40 comp rx=0 tx=0).stop 2023-07-21T18:59:05.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.161+0000 7f813c19eb80 1 -- 172.21.15.39:0/2698123895 shutdown_connections 2023-07-21T18:59:05.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.161+0000 7f813c19eb80 1 --2- 172.21.15.39:0/2698123895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6fe0ea580 0x55a6fe0ea9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:05.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.161+0000 7f813c19eb80 1 --2- 172.21.15.39:0/2698123895 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a6fdf466b0 0x55a6fe0efd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:05.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.161+0000 7f813c19eb80 1 --2- 172.21.15.39:0/2698123895 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a6fe0f02d0 0x55a6fe0ea040 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:05.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.161+0000 7f813c19eb80 1 -- 172.21.15.39:0/2698123895 >> 172.21.15.39:0/2698123895 conn(0x55a6fe0dfbe0 msgr2=0x55a6fe0e4f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:59:05.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.161+0000 7f813c19eb80 1 -- 172.21.15.39:0/2698123895 shutdown_connections 2023-07-21T18:59:05.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.161+0000 7f813c19eb80 1 -- 172.21.15.39:0/2698123895 wait complete. 2023-07-21T18:59:05.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.162+0000 7f813c19eb80 1 -- start start 2023-07-21T18:59:05.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.162+0000 7f813c19eb80 1 Processor -- start 2023-07-21T18:59:05.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.163+0000 7f813c19eb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a6fdf466b0 0x55a6fe0f65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:05.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.163+0000 7f813c19eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a6fe0ea580 0x55a6fe0f3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:05.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.163+0000 7f813c19eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6fe0f02d0 0x55a6fe0f3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:05.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.163+0000 7f813c19eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6fdfdba60 con 0x55a6fe0ea580 2023-07-21T18:59:05.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.163+0000 7f813c19eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6fdfdb760 con 0x55a6fe0f02d0 2023-07-21T18:59:05.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.163+0000 7f813c19eb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6fdebf8c0 con 0x55a6fdf466b0 2023-07-21T18:59:05.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.163+0000 7f813c19eb80 10 client.0 _ll_register_callbacks cb 0x55a6fdfd9700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:59:05.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.163+0000 7f812d5b3700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a6fe0ea580 0x55a6fe0f3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:05.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.163+0000 7f812d5b3700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a6fe0ea580 0x55a6fe0f3860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33782/0 (socket says 172.21.15.39:33782) 2023-07-21T18:59:05.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.163+0000 7f812d5b3700 1 -- 172.21.15.39:0/1057383064 learned_addr learned my addr 172.21.15.39:0/1057383064 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:59:05.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.163+0000 7f812d5b3700 1 -- 172.21.15.39:0/1057383064 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a6fdf466b0 msgr2=0x55a6fe0f65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:59:05.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.163+0000 7f812d5b3700 1 --2- 172.21.15.39:0/1057383064 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a6fdf466b0 0x55a6fe0f65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:05.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.163+0000 7f812d5b3700 1 -- 172.21.15.39:0/1057383064 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6fe0f02d0 msgr2=0x55a6fe0f3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:59:05.177 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.163+0000 7f812d5b3700 1 --2- 172.21.15.39:0/1057383064 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6fe0f02d0 0x55a6fe0f3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:05.177 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.163+0000 7f812d5b3700 1 -- 172.21.15.39:0/1057383064 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8128113840 con 0x55a6fe0ea580 2023-07-21T18:59:05.177 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.164+0000 7f812d5b3700 1 --2- 172.21.15.39:0/1057383064 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a6fe0ea580 0x55a6fe0f3860 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f8118003de0 tx=0x7f8118003ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=393d0448c4d5561d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:59:05.177 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.164+0000 7f81117fa700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:59:05.177 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.164+0000 7f81117fa700 1 -- 172.21.15.39:0/1057383064 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811801d070 con 0x55a6fe0ea580 2023-07-21T18:59:05.177 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.164+0000 7f81117fa700 1 -- 172.21.15.39:0/1057383064 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f81180052c0 con 0x55a6fe0ea580 2023-07-21T18:59:05.177 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.164+0000 7f813c19eb80 1 -- 172.21.15.39:0/1057383064 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a6fe034e50 con 0x55a6fe0ea580 2023-07-21T18:59:05.177 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.164+0000 7f81117fa700 1 -- 172.21.15.39:0/1057383064 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8118005480 con 0x55a6fe0ea580 2023-07-21T18:59:05.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.164+0000 7f813c19eb80 10 client.17457 fetch_fsmap learned FSMap version 92 2023-07-21T18:59:05.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.164+0000 7f813c19eb80 1 -- 172.21.15.39:0/1057383064 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=4+}) v3 -- 0x55a6fde52c00 con 0x55a6fe0ea580 2023-07-21T18:59:05.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.164+0000 7f81117fa700 1 -- 172.21.15.39:0/1057383064 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a6fe034e50 con 0x55a6fe0ea580 2023-07-21T18:59:05.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.165+0000 7f81117fa700 1 -- 172.21.15.39:0/1057383064 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 92) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f811802b450 con 0x55a6fe0ea580 2023-07-21T18:59:05.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.165+0000 7f813c19eb80 10 client.17457 fetch_fsmap finished waiting for FSMap version 92 2023-07-21T18:59:05.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.165+0000 7f813c19eb80 10 client.17457 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:59:05.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.165+0000 7f813c19eb80 20 client.17457 populate_metadata read hostname 'smithi039' 2023-07-21T18:59:05.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.165+0000 7f813c19eb80 1 --2- 172.21.15.39:0/1057383064 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55a6fe103840 0x55a6fe105c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:05.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.165+0000 7f813c19eb80 4 client.17457 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:59:05.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.165+0000 7f813c19eb80 1 -- 172.21.15.39:0/1057383064 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a6fdff6ff0 con 0x55a6fe103840 2023-07-21T18:59:05.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.166+0000 7f812ddb4700 1 --2- 172.21.15.39:0/1057383064 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55a6fe103840 0x55a6fe105c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:05.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.171+0000 7f81117fa700 10 client.17457 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:59:05.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.171+0000 7f812ddb4700 1 --2- 172.21.15.39:0/1057383064 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55a6fe103840 0x55a6fe105c50 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa920e832ba94fac server_cookie=7b5bb63a16c46331 in_seq=0 out_seq=0 2023-07-21T18:59:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.175+0000 7f81117fa700 1 -- 172.21.15.39:0/1057383064 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a6fdff6ff0 con 0x55a6fe103840 2023-07-21T18:59:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.175+0000 7f81117fa700 10 client.17457 handle_command_reply: tid=0 2023-07-21T18:59:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.175+0000 7f813c19eb80 1 client.17457 shutdown 2023-07-21T18:59:05.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.177+0000 7f813c19eb80 1 -- 172.21.15.39:0/1057383064 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a6fe0ea580 msgr2=0x55a6fe0f3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:59:05.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.177+0000 7f813c19eb80 1 --2- 172.21.15.39:0/1057383064 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a6fe0ea580 0x55a6fe0f3860 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f8118003de0 tx=0x7f8118003ef0 comp rx=0 tx=0).stop 2023-07-21T18:59:05.188 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.177+0000 7f813c19eb80 1 -- 172.21.15.39:0/1057383064 shutdown_connections 2023-07-21T18:59:05.188 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.177+0000 7f813c19eb80 1 --2- 172.21.15.39:0/1057383064 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55a6fe103840 0x55a6fe105c50 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:05.188 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.177+0000 7f813c19eb80 1 --2- 172.21.15.39:0/1057383064 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6fe0f02d0 0x55a6fe0f3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:05.188 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.177+0000 7f813c19eb80 1 --2- 172.21.15.39:0/1057383064 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a6fdf466b0 0x55a6fe0f65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:05.188 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.177+0000 7f813c19eb80 1 --2- 172.21.15.39:0/1057383064 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a6fe0ea580 0x55a6fe0f3860 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:05.189 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.177+0000 7f813c19eb80 1 -- 172.21.15.39:0/1057383064 >> 172.21.15.39:0/1057383064 conn(0x55a6fe0dfbe0 msgr2=0x55a6fe0e44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:59:05.189 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.178+0000 7f81117fa700 0 client.17457 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:59:05.189 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.178+0000 7f813c19eb80 1 -- 172.21.15.39:0/1057383064 shutdown_connections 2023-07-21T18:59:05.189 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.178+0000 7f813c19eb80 1 -- 172.21.15.39:0/1057383064 wait complete. 2023-07-21T18:59:05.189 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.178+0000 7f813c19eb80 20 client.17457 trim_cache size 0 max 16384 2023-07-21T18:59:05.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.205+0000 7f813c19eb80 1 Processor -- start 2023-07-21T18:59:05.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.205+0000 7f813c19eb80 1 -- start start 2023-07-21T18:59:05.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.205+0000 7f813c19eb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a6fdf466b0 0x55a6fe0f6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:05.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.205+0000 7f813c19eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6fe0ea580 0x55a6fe0e4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:05.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.205+0000 7f813c19eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a6fe0f02d0 0x55a6fe0f3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:05.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.205+0000 7f813c19eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6fe0f47f0 con 0x55a6fe0f02d0 2023-07-21T18:59:05.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.205+0000 7f813c19eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6fe100520 con 0x55a6fe0ea580 2023-07-21T18:59:05.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.205+0000 7f813c19eb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6fe0f1dc0 con 0x55a6fdf466b0 2023-07-21T18:59:05.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.206+0000 7f812e5b5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a6fe0f02d0 0x55a6fe0f3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:05.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.206+0000 7f812e5b5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a6fe0f02d0 0x55a6fe0f3020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33790/0 (socket says 172.21.15.39:33790) 2023-07-21T18:59:05.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.206+0000 7f812e5b5700 1 -- 172.21.15.39:0/2627788790 learned_addr learned my addr 172.21.15.39:0/2627788790 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:59:05.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.206+0000 7f812e5b5700 1 -- 172.21.15.39:0/2627788790 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a6fdf466b0 msgr2=0x55a6fe0f6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:59:05.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.206+0000 7f812e5b5700 1 --2- 172.21.15.39:0/2627788790 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a6fdf466b0 0x55a6fe0f6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:05.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.206+0000 7f812e5b5700 1 -- 172.21.15.39:0/2627788790 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6fe0ea580 msgr2=0x55a6fe0e4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:59:05.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.206+0000 7f812e5b5700 1 --2- 172.21.15.39:0/2627788790 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6fe0ea580 0x55a6fe0e4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:05.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.206+0000 7f812e5b5700 1 -- 172.21.15.39:0/2627788790 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6fe10a840 con 0x55a6fe0f02d0 2023-07-21T18:59:05.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.206+0000 7f812e5b5700 1 --2- 172.21.15.39:0/2627788790 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a6fe0f02d0 0x55a6fe0f3020 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f812000fac0 tx=0x7f8120006d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=196b367135da2d09 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:59:05.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.207+0000 7f811cff9700 1 -- 172.21.15.39:0/2627788790 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8120024070 con 0x55a6fe0f02d0 2023-07-21T18:59:05.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.207+0000 7f811cff9700 1 -- 172.21.15.39:0/2627788790 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f812000fe80 con 0x55a6fe0f02d0 2023-07-21T18:59:05.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.207+0000 7f811cff9700 1 -- 172.21.15.39:0/2627788790 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812000f040 con 0x55a6fe0f02d0 2023-07-21T18:59:05.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.208+0000 7f813c19eb80 1 -- 172.21.15.39:0/2627788790 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a6fe0f02d0 msgr2=0x55a6fe0f3020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:59:05.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.208+0000 7f813c19eb80 1 --2- 172.21.15.39:0/2627788790 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a6fe0f02d0 0x55a6fe0f3020 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f812000fac0 tx=0x7f8120006d50 comp rx=0 tx=0).stop 2023-07-21T18:59:05.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.213+0000 7f813c19eb80 1 -- 172.21.15.39:0/2627788790 shutdown_connections 2023-07-21T18:59:05.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.213+0000 7f813c19eb80 1 --2- 172.21.15.39:0/2627788790 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a6fe0f02d0 0x55a6fe0f3020 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:05.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.213+0000 7f813c19eb80 1 --2- 172.21.15.39:0/2627788790 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a6fdf466b0 0x55a6fe0f6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:05.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.213+0000 7f813c19eb80 1 --2- 172.21.15.39:0/2627788790 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6fe0ea580 0x55a6fe0e4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:05.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.213+0000 7f813c19eb80 1 -- 172.21.15.39:0/2627788790 >> 172.21.15.39:0/2627788790 conn(0x55a6fe0dfbe0 msgr2=0x55a6fe0edb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:59:05.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.213+0000 7f813c19eb80 1 -- 172.21.15.39:0/2627788790 shutdown_connections 2023-07-21T18:59:05.224 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.213+0000 7f813c19eb80 1 -- 172.21.15.39:0/2627788790 wait complete. 2023-07-21T18:59:05.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.213+0000 7f813c19eb80 1 -- start start 2023-07-21T18:59:05.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.213+0000 7f813c19eb80 1 Processor -- start 2023-07-21T18:59:05.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.214+0000 7f813c19eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6fdf466b0 0x55a6fe0f8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:05.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.214+0000 7f813c19eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a6fe0ea580 0x55a6fe0f9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:05.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.214+0000 7f813c19eb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a6fe0f02d0 0x55a6fe0f96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:05.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.214+0000 7f813c19eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6fe1567d0 con 0x55a6fe0ea580 2023-07-21T18:59:05.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.214+0000 7f813c19eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6fe156940 con 0x55a6fdf466b0 2023-07-21T18:59:05.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.214+0000 7f813c19eb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6fe156b00 con 0x55a6fe0f02d0 2023-07-21T18:59:05.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.214+0000 7f813c19eb80 10 client.0 _ll_register_callbacks cb 0x7f8120009430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:59:05.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.214+0000 7f812d5b3700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a6fe0ea580 0x55a6fe0f9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:05.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.214+0000 7f812d5b3700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a6fe0ea580 0x55a6fe0f9190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33796/0 (socket says 172.21.15.39:33796) 2023-07-21T18:59:05.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.214+0000 7f812d5b3700 1 -- 172.21.15.39:0/2183535536 learned_addr learned my addr 172.21.15.39:0/2183535536 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:59:05.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.214+0000 7f812ddb4700 1 --2- 172.21.15.39:0/2183535536 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6fdf466b0 0x55a6fe0f8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:05.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.215+0000 7f812d5b3700 1 -- 172.21.15.39:0/2183535536 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a6fe0f02d0 msgr2=0x55a6fe0f96d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T18:59:05.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.215+0000 7f812d5b3700 1 --2- 172.21.15.39:0/2183535536 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a6fe0f02d0 0x55a6fe0f96d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:05.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.215+0000 7f812d5b3700 1 -- 172.21.15.39:0/2183535536 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6fdf466b0 msgr2=0x55a6fe0f8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:59:05.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.215+0000 7f812d5b3700 1 --2- 172.21.15.39:0/2183535536 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6fdf466b0 0x55a6fe0f8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:05.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.215+0000 7f812d5b3700 1 -- 172.21.15.39:0/2183535536 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f812000f7a0 con 0x55a6fe0ea580 2023-07-21T18:59:05.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.215+0000 7f812d5b3700 1 --2- 172.21.15.39:0/2183535536 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a6fe0ea580 0x55a6fe0f9190 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f8118003e10 tx=0x7f8118009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b96c770c1a4e807 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:59:05.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.215+0000 7f81117fa700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:59:05.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.215+0000 7f81117fa700 1 -- 172.21.15.39:0/2183535536 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811801d070 con 0x55a6fe0ea580 2023-07-21T18:59:05.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.215+0000 7f81117fa700 1 -- 172.21.15.39:0/2183535536 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f8118000e40 con 0x55a6fe0ea580 2023-07-21T18:59:05.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.215+0000 7f813c19eb80 1 -- 172.21.15.39:0/2183535536 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a6fdf95490 con 0x55a6fe0ea580 2023-07-21T18:59:05.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.216+0000 7f81117fa700 1 -- 172.21.15.39:0/2183535536 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8118005bc0 con 0x55a6fe0ea580 2023-07-21T18:59:05.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.216+0000 7f813c19eb80 10 client.17463 fetch_fsmap learned FSMap version 92 2023-07-21T18:59:05.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.216+0000 7f813c19eb80 1 -- 172.21.15.39:0/2183535536 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=4+}) v3 -- 0x55a6fde901f0 con 0x55a6fe0ea580 2023-07-21T18:59:05.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.216+0000 7f81117fa700 1 -- 172.21.15.39:0/2183535536 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a6fdf95490 con 0x55a6fe0ea580 2023-07-21T18:59:05.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.216+0000 7f81117fa700 1 -- 172.21.15.39:0/2183535536 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 92) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f81180325e0 con 0x55a6fe0ea580 2023-07-21T18:59:05.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.216+0000 7f813c19eb80 10 client.17463 fetch_fsmap finished waiting for FSMap version 92 2023-07-21T18:59:05.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.216+0000 7f813c19eb80 10 client.17463 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:59:05.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.216+0000 7f813c19eb80 20 client.17463 populate_metadata read hostname 'smithi039' 2023-07-21T18:59:05.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.216+0000 7f813c19eb80 1 --2- 172.21.15.39:0/2183535536 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55a6fe157f10 0x55a6fe15a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:05.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.216+0000 7f813c19eb80 4 client.17463 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:59:05.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.216+0000 7f813c19eb80 1 -- 172.21.15.39:0/2183535536 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a6fde8af10 con 0x55a6fe157f10 2023-07-21T18:59:05.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.216+0000 7f812ddb4700 1 --2- 172.21.15.39:0/2183535536 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55a6fe157f10 0x55a6fe15a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:05.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.217+0000 7f81117fa700 10 client.17463 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:59:05.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.217+0000 7f812ddb4700 1 --2- 172.21.15.39:0/2183535536 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55a6fe157f10 0x55a6fe15a320 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c8f6354414869e1 server_cookie=b1ff4211129a8faf in_seq=0 out_seq=0 2023-07-21T18:59:05.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.482+0000 7f81117fa700 1 -- 172.21.15.39:0/2183535536 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a6fde8af10 con 0x55a6fe157f10 2023-07-21T18:59:05.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.482+0000 7f81117fa700 10 client.17463 handle_command_reply: tid=0 2023-07-21T18:59:05.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.482+0000 7f813c19eb80 1 client.17463 shutdown 2023-07-21T18:59:05.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.482+0000 7f813c19eb80 1 -- 172.21.15.39:0/2183535536 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a6fe0ea580 msgr2=0x55a6fe0f9190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:59:05.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.482+0000 7f813c19eb80 1 --2- 172.21.15.39:0/2183535536 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a6fe0ea580 0x55a6fe0f9190 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f8118003e10 tx=0x7f8118009fd0 comp rx=0 tx=0).stop 2023-07-21T18:59:05.494 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.484+0000 7f813c19eb80 1 -- 172.21.15.39:0/2183535536 shutdown_connections 2023-07-21T18:59:05.494 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.484+0000 7f813c19eb80 1 --2- 172.21.15.39:0/2183535536 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55a6fe157f10 0x55a6fe15a320 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:05.494 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.484+0000 7f813c19eb80 1 --2- 172.21.15.39:0/2183535536 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6fdf466b0 0x55a6fe0f8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:05.494 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.484+0000 7f813c19eb80 1 --2- 172.21.15.39:0/2183535536 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a6fe0f02d0 0x55a6fe0f96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:05.494 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.484+0000 7f813c19eb80 1 --2- 172.21.15.39:0/2183535536 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a6fe0ea580 0x55a6fe0f9190 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:05.494 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.484+0000 7f813c19eb80 1 -- 172.21.15.39:0/2183535536 >> 172.21.15.39:0/2183535536 conn(0x55a6fe0dfbe0 msgr2=0x55a6fe0eea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:59:05.495 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.484+0000 7f81117fa700 0 client.17463 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:59:05.495 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.484+0000 7f813c19eb80 1 -- 172.21.15.39:0/2183535536 shutdown_connections 2023-07-21T18:59:05.495 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.484+0000 7f813c19eb80 1 -- 172.21.15.39:0/2183535536 wait complete. 2023-07-21T18:59:05.495 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.484+0000 7f813c19eb80 20 client.17463 trim_cache size 0 max 16384 2023-07-21T18:59:05.496 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:59:05.496 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:59:05.496 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:59:05.496 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:59:05.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.490+0000 7f8130018700 1 -- 172.21.15.39:0/1555449244 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f81140a1450 msgr2=0x7f81140a3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:59:05.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.490+0000 7f8130018700 1 --2- 172.21.15.39:0/1555449244 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f81140a1450 0x7f81140a3900 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8120009c30 tx=0x7f8120006ce0 comp rx=0 tx=0).stop 2023-07-21T18:59:05.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.490+0000 7f8130018700 1 -- 172.21.15.39:0/1555449244 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8128106340 msgr2=0x7f8128080810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:59:05.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.490+0000 7f8130018700 1 --2- 172.21.15.39:0/1555449244 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8128106340 0x7f8128080810 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f812400d950 tx=0x7f812400dd10 comp rx=0 tx=0).stop 2023-07-21T18:59:05.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.490+0000 7f8130018700 1 -- 172.21.15.39:0/1555449244 shutdown_connections 2023-07-21T18:59:05.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.490+0000 7f8130018700 1 --2- 172.21.15.39:0/1555449244 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f812810c1c0 0x7f8128079610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:05.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.490+0000 7f8130018700 1 --2- 172.21.15.39:0/1555449244 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8128106340 0x7f8128080810 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:05.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.490+0000 7f8130018700 1 --2- 172.21.15.39:0/1555449244 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f81140a1450 0x7f81140a3900 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:05.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.490+0000 7f8130018700 1 --2- 172.21.15.39:0/1555449244 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8128106cf0 0x7f8128080d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:05.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.490+0000 7f8130018700 1 -- 172.21.15.39:0/1555449244 >> 172.21.15.39:0/1555449244 conn(0x7f8128075a20 msgr2=0x7f812810cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:59:05.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.490+0000 7f8130018700 1 -- 172.21.15.39:0/1555449244 shutdown_connections 2023-07-21T18:59:05.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:05.490+0000 7f8130018700 1 -- 172.21.15.39:0/1555449244 wait complete. 2023-07-21T18:59:05.681 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:59:05 smithi039 ceph-mon[96489]: osdmap e107: 12 total, 12 up, 12 in 2023-07-21T18:59:05.681 INFO:tasks.workunit.client.0.smithi039.stdout: 10 147757 31.51 MB/sec execute 138 sec latency 192.360 ms 2023-07-21T18:59:05.859 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:59:05 smithi193 ceph-mon[98204]: osdmap e107: 12 total, 12 up, 12 in 2023-07-21T18:59:05.878 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:59:05 smithi155 ceph-mon[97934]: osdmap e107: 12 total, 12 up, 12 in 2023-07-21T18:59:06.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:59:06 smithi039 ceph-mon[96489]: pgmap v548: 97 pgs: 97 active+clean; 1.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 23 MiB/s wr, 1.08k op/s 2023-07-21T18:59:06.678 INFO:tasks.workunit.client.0.smithi039.stdout: 10 148066 31.39 MB/sec execute 139 sec latency 89.033 ms 2023-07-21T18:59:06.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:59:06 smithi193 ceph-mon[98204]: pgmap v548: 97 pgs: 97 active+clean; 1.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 23 MiB/s wr, 1.08k op/s 2023-07-21T18:59:06.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:59:06 smithi155 ceph-mon[97934]: pgmap v548: 97 pgs: 97 active+clean; 1.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 23 MiB/s wr, 1.08k op/s 2023-07-21T18:59:07.678 INFO:tasks.workunit.client.0.smithi039.stdout: 10 148539 31.28 MB/sec execute 140 sec latency 111.390 ms 2023-07-21T18:59:08.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:59:08 smithi039 ceph-mon[96489]: pgmap v549: 97 pgs: 39 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 38 active+clean; 1.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 11 MiB/s wr, 1.00k op/s 2023-07-21T18:59:08.678 INFO:tasks.workunit.client.0.smithi039.stdout: 10 149083 31.27 MB/sec execute 141 sec latency 163.245 ms 2023-07-21T18:59:08.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:59:08 smithi193 ceph-mon[98204]: pgmap v549: 97 pgs: 39 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 38 active+clean; 1.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 11 MiB/s wr, 1.00k op/s 2023-07-21T18:59:08.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:59:08 smithi155 ceph-mon[97934]: pgmap v549: 97 pgs: 39 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 38 active+clean; 1.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 11 MiB/s wr, 1.00k op/s 2023-07-21T18:59:09.678 INFO:tasks.workunit.client.0.smithi039.stdout: 10 149863 31.36 MB/sec execute 142 sec latency 72.857 ms 2023-07-21T18:59:10.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:59:10 smithi039 ceph-mon[96489]: pgmap v550: 97 pgs: 39 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 38 active+clean; 1.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 11 MiB/s wr, 1.00k op/s 2023-07-21T18:59:10.678 INFO:tasks.workunit.client.0.smithi039.stdout: 10 150631 31.42 MB/sec execute 143 sec latency 45.700 ms 2023-07-21T18:59:10.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:59:10 smithi193 ceph-mon[98204]: pgmap v550: 97 pgs: 39 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 38 active+clean; 1.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 11 MiB/s wr, 1.00k op/s 2023-07-21T18:59:10.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:59:10 smithi155 ceph-mon[97934]: pgmap v550: 97 pgs: 39 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 38 active+clean; 1.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 11 MiB/s wr, 1.00k op/s 2023-07-21T18:59:11.680 INFO:tasks.workunit.client.0.smithi039.stdout: 10 151258 31.49 MB/sec execute 144 sec latency 106.190 ms 2023-07-21T18:59:12.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:59:12 smithi039 ceph-mon[96489]: pgmap v551: 97 pgs: 30 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 55 active+clean; 1.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 9.4 MiB/s wr, 942 op/s 2023-07-21T18:59:12.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:59:12 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:59:12.678 INFO:tasks.workunit.client.0.smithi039.stdout: 10 151743 31.46 MB/sec execute 145 sec latency 118.370 ms 2023-07-21T18:59:12.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:59:12 smithi193 ceph-mon[98204]: pgmap v551: 97 pgs: 30 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 55 active+clean; 1.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 9.4 MiB/s wr, 942 op/s 2023-07-21T18:59:12.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:59:12 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:59:12.909 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:59:12 smithi155 ceph-mon[97934]: pgmap v551: 97 pgs: 30 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 55 active+clean; 1.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 9.4 MiB/s wr, 942 op/s 2023-07-21T18:59:12.909 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:59:12 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:59:13.679 INFO:tasks.workunit.client.0.smithi039.stdout: 10 152195 31.35 MB/sec execute 146 sec latency 190.267 ms 2023-07-21T18:59:14.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:59:14 smithi039 ceph-mon[96489]: pgmap v552: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.1 MiB/s wr, 716 op/s 2023-07-21T18:59:14.679 INFO:tasks.workunit.client.0.smithi039.stdout: 10 152776 31.38 MB/sec execute 147 sec latency 93.201 ms 2023-07-21T18:59:14.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:59:14 smithi193 ceph-mon[98204]: pgmap v552: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.1 MiB/s wr, 716 op/s 2023-07-21T18:59:14.901 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:59:14 smithi155 ceph-mon[97934]: pgmap v552: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.1 MiB/s wr, 716 op/s 2023-07-21T18:59:15.678 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:59:15 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:59:15.678 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:59:15 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:59:15.678 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:59:15 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:59:15.680 INFO:tasks.workunit.client.0.smithi039.stdout: 10 153313 31.36 MB/sec execute 148 sec latency 259.520 ms 2023-07-21T18:59:15.860 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:59:15 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:59:15.861 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:59:15 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:59:15.861 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:59:15 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:59:15.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:59:15 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:59:15.878 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:59:15 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:59:15.878 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:59:15 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T18:59:16.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:59:16 smithi039 ceph-mon[96489]: pgmap v553: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 6.9 MiB/s wr, 701 op/s 2023-07-21T18:59:16.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:59:16 smithi039 ceph-mon[96489]: osdmap e108: 12 total, 12 up, 12 in 2023-07-21T18:59:16.682 INFO:tasks.workunit.client.0.smithi039.stdout: 10 153614 31.25 MB/sec execute 149 sec latency 715.190 ms 2023-07-21T18:59:16.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:59:16 smithi193 ceph-mon[98204]: pgmap v553: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 6.9 MiB/s wr, 701 op/s 2023-07-21T18:59:16.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:59:16 smithi193 ceph-mon[98204]: osdmap e108: 12 total, 12 up, 12 in 2023-07-21T18:59:16.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:59:16 smithi155 ceph-mon[97934]: pgmap v553: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 6.9 MiB/s wr, 701 op/s 2023-07-21T18:59:16.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:59:16 smithi155 ceph-mon[97934]: osdmap e108: 12 total, 12 up, 12 in 2023-07-21T18:59:17.682 INFO:tasks.workunit.client.0.smithi039.stdout: 10 153979 31.18 MB/sec execute 150 sec latency 128.080 ms 2023-07-21T18:59:18.654 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.642+0000 7f3b1207b700 1 -- 172.21.15.39:0/3061123974 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b0c0fea30 msgr2=0x7f3b0c0feea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:59:18.654 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.642+0000 7f3b1207b700 1 --2- 172.21.15.39:0/3061123974 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b0c0fea30 0x7f3b0c0feea0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f3b00009990 tx=0x7f3b00009ca0 comp rx=0 tx=0).stop 2023-07-21T18:59:18.654 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.642+0000 7f3b1207b700 1 -- 172.21.15.39:0/3061123974 shutdown_connections 2023-07-21T18:59:18.655 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.642+0000 7f3b1207b700 1 --2- 172.21.15.39:0/3061123974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b0c104a40 0x7f3b0c104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:18.655 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.642+0000 7f3b1207b700 1 --2- 172.21.15.39:0/3061123974 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b0c0ff3e0 0x7f3b0c10b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:18.655 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.642+0000 7f3b1207b700 1 --2- 172.21.15.39:0/3061123974 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b0c0fea30 0x7f3b0c0feea0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:18.655 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.642+0000 7f3b1207b700 1 -- 172.21.15.39:0/3061123974 >> 172.21.15.39:0/3061123974 conn(0x7f3b0c0fa4f0 msgr2=0x7f3b0c0fc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:59:18.655 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.642+0000 7f3b1207b700 1 -- 172.21.15.39:0/3061123974 shutdown_connections 2023-07-21T18:59:18.655 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.642+0000 7f3b1207b700 1 -- 172.21.15.39:0/3061123974 wait complete. 2023-07-21T18:59:18.656 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.643+0000 7f3b1207b700 1 Processor -- start 2023-07-21T18:59:18.657 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.643+0000 7f3b1207b700 1 -- start start 2023-07-21T18:59:18.657 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.643+0000 7f3b1207b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b0c0fea30 0x7f3b0c199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:18.657 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.643+0000 7f3b1207b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b0c0ff3e0 0x7f3b0c199e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:18.658 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.643+0000 7f3b1207b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b0c104a40 0x7f3b0c193760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:18.658 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.644+0000 7f3b1207b700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0c193dd0 con 0x7f3b0c104a40 2023-07-21T18:59:18.658 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.644+0000 7f3b1207b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0c193f40 con 0x7f3b0c0fea30 2023-07-21T18:59:18.658 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.644+0000 7f3b1207b700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0c194100 con 0x7f3b0c0ff3e0 2023-07-21T18:59:18.658 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.644+0000 7f3b0affd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b0c0ff3e0 0x7f3b0c199e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:18.658 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.644+0000 7f3b0affd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b0c0ff3e0 0x7f3b0c199e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:52362/0 (socket says 172.21.15.39:52362) 2023-07-21T18:59:18.658 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.644+0000 7f3b0bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b0c104a40 0x7f3b0c193760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:18.659 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.644+0000 7f3b0affd700 1 -- 172.21.15.39:0/304942403 learned_addr learned my addr 172.21.15.39:0/304942403 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:59:18.659 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.644+0000 7f3b0b7fe700 1 --2- 172.21.15.39:0/304942403 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b0c0fea30 0x7f3b0c199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:18.659 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.644+0000 7f3b0bfff700 1 -- 172.21.15.39:0/304942403 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b0c0ff3e0 msgr2=0x7f3b0c199e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:59:18.659 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.644+0000 7f3b0bfff700 1 --2- 172.21.15.39:0/304942403 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b0c0ff3e0 0x7f3b0c199e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:18.659 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.644+0000 7f3b0bfff700 1 -- 172.21.15.39:0/304942403 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b0c0fea30 msgr2=0x7f3b0c199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:59:18.659 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.644+0000 7f3b0bfff700 1 --2- 172.21.15.39:0/304942403 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b0c0fea30 0x7f3b0c199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:18.660 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.644+0000 7f3b0bfff700 1 -- 172.21.15.39:0/304942403 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b000095f0 con 0x7f3b0c104a40 2023-07-21T18:59:18.660 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.644+0000 7f3b0b7fe700 1 --2- 172.21.15.39:0/304942403 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b0c0fea30 0x7f3b0c199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:59:18.660 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.644+0000 7f3b0affd700 1 --2- 172.21.15.39:0/304942403 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b0c0ff3e0 0x7f3b0c199e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:59:18.660 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.644+0000 7f3b0bfff700 1 --2- 172.21.15.39:0/304942403 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b0c104a40 0x7f3b0c193760 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f3b0400efc0 tx=0x7f3b0400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:59:18.660 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.645+0000 7f3b08ff9700 1 -- 172.21.15.39:0/304942403 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b04018070 con 0x7f3b0c104a40 2023-07-21T18:59:18.660 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.645+0000 7f3b08ff9700 1 -- 172.21.15.39:0/304942403 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b0400f040 con 0x7f3b0c104a40 2023-07-21T18:59:18.660 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.645+0000 7f3b1207b700 1 -- 172.21.15.39:0/304942403 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b0c194380 con 0x7f3b0c104a40 2023-07-21T18:59:18.660 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.645+0000 7f3b08ff9700 1 -- 172.21.15.39:0/304942403 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b0400cdf0 con 0x7f3b0c104a40 2023-07-21T18:59:18.661 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.645+0000 7f3b1207b700 1 -- 172.21.15.39:0/304942403 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b0c1a08f0 con 0x7f3b0c104a40 2023-07-21T18:59:18.664 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.647+0000 7f3b08ff9700 1 -- 172.21.15.39:0/304942403 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f3b04003a00 con 0x7f3b0c104a40 2023-07-21T18:59:18.664 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.648+0000 7f3b08ff9700 1 --2- 172.21.15.39:0/304942403 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f3af40a12c0 0x7f3af40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:18.664 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.649+0000 7f3b0b7fe700 1 --2- 172.21.15.39:0/304942403 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f3af40a12c0 0x7f3af40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:18.664 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.649+0000 7f3b08ff9700 1 -- 172.21.15.39:0/304942403 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f3b040cc9a0 con 0x7f3b0c104a40 2023-07-21T18:59:18.664 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.649+0000 7f3b0b7fe700 1 --2- 172.21.15.39:0/304942403 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f3af40a12c0 0x7f3af40a3770 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c0ffb70 tx=0x7f3afc00f040 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:59:18.664 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.649+0000 7f3b1e201b80 1 Processor -- start 2023-07-21T18:59:18.665 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.649+0000 7f3b1e201b80 1 -- start start 2023-07-21T18:59:18.665 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.650+0000 7f3b1e201b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f77cc46b0 0x561f77e6dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:18.665 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.650+0000 7f3b1e201b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561f77e6e2d0 0x561f77e68040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:18.665 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.650+0000 7f3b1e201b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f77e68580 0x561f77e689d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:18.665 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.650+0000 7f3b1e201b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f77d6dc50 con 0x561f77cc46b0 2023-07-21T18:59:18.665 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.650+0000 7f3b1e201b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f77dc09e0 con 0x561f77e68580 2023-07-21T18:59:18.665 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.650+0000 7f3b1e201b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f77db9e70 con 0x561f77e6e2d0 2023-07-21T18:59:18.665 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.650+0000 7f3b0b7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f77cc46b0 0x561f77e6dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:18.666 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.650+0000 7f3b0b7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f77cc46b0 0x561f77e6dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33816/0 (socket says 172.21.15.39:33816) 2023-07-21T18:59:18.666 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.650+0000 7f3b0b7fe700 1 -- 172.21.15.39:0/3292450427 learned_addr learned my addr 172.21.15.39:0/3292450427 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:59:18.666 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.650+0000 7f3b0affd700 1 --2- 172.21.15.39:0/3292450427 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561f77e6e2d0 0x561f77e68040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:18.666 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.650+0000 7f3b0bfff700 1 --2- 172.21.15.39:0/3292450427 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f77e68580 0x561f77e689d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:18.666 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.650+0000 7f3b0b7fe700 1 -- 172.21.15.39:0/3292450427 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561f77e6e2d0 msgr2=0x561f77e68040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:59:18.666 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.650+0000 7f3b0b7fe700 1 --2- 172.21.15.39:0/3292450427 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561f77e6e2d0 0x561f77e68040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:18.666 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.650+0000 7f3b0b7fe700 1 -- 172.21.15.39:0/3292450427 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f77e68580 msgr2=0x561f77e689d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:59:18.667 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.650+0000 7f3b0b7fe700 1 --2- 172.21.15.39:0/3292450427 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f77e68580 0x561f77e689d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:18.667 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.650+0000 7f3b0b7fe700 1 -- 172.21.15.39:0/3292450427 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f77be7d80 con 0x561f77cc46b0 2023-07-21T18:59:18.667 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.650+0000 7f3b0bfff700 1 --2- 172.21.15.39:0/3292450427 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f77e68580 0x561f77e689d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:59:18.667 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.650+0000 7f3b0b7fe700 1 --2- 172.21.15.39:0/3292450427 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f77cc46b0 0x561f77e6dd90 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f3afc009d70 tx=0x7f3afc013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdb46ee94aecef7d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:59:18.667 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.650+0000 7f3af1ffb700 1 -- 172.21.15.39:0/3292450427 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3afc01f070 con 0x561f77cc46b0 2023-07-21T18:59:18.667 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.650+0000 7f3af1ffb700 1 -- 172.21.15.39:0/3292450427 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3afc009da0 con 0x561f77cc46b0 2023-07-21T18:59:18.667 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.650+0000 7f3af1ffb700 1 -- 172.21.15.39:0/3292450427 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3afc012040 con 0x561f77cc46b0 2023-07-21T18:59:18.667 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.651+0000 7f3b1e201b80 1 -- 172.21.15.39:0/3292450427 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f77cc46b0 msgr2=0x561f77e6dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:59:18.668 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.651+0000 7f3b1e201b80 1 --2- 172.21.15.39:0/3292450427 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f77cc46b0 0x561f77e6dd90 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f3afc009d70 tx=0x7f3afc013000 comp rx=0 tx=0).stop 2023-07-21T18:59:18.668 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.651+0000 7f3b1e201b80 1 -- 172.21.15.39:0/3292450427 shutdown_connections 2023-07-21T18:59:18.668 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.651+0000 7f3b1e201b80 1 --2- 172.21.15.39:0/3292450427 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f77e68580 0x561f77e689d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:18.668 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.651+0000 7f3b1e201b80 1 --2- 172.21.15.39:0/3292450427 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f77cc46b0 0x561f77e6dd90 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:18.668 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.651+0000 7f3b1e201b80 1 --2- 172.21.15.39:0/3292450427 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561f77e6e2d0 0x561f77e68040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:18.668 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.651+0000 7f3b1e201b80 1 -- 172.21.15.39:0/3292450427 >> 172.21.15.39:0/3292450427 conn(0x561f77e5dbe0 msgr2=0x561f77e62f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:59:18.668 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.651+0000 7f3b1e201b80 1 -- 172.21.15.39:0/3292450427 shutdown_connections 2023-07-21T18:59:18.668 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.651+0000 7f3b1e201b80 1 -- 172.21.15.39:0/3292450427 wait complete. 2023-07-21T18:59:18.669 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.652+0000 7f3b1e201b80 1 -- start start 2023-07-21T18:59:18.669 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.652+0000 7f3b1e201b80 1 Processor -- start 2023-07-21T18:59:18.669 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.652+0000 7f3b1e201b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561f77cc46b0 0x561f77e71900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:18.669 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.652+0000 7f3b1e201b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f77e68580 0x561f77e71e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:18.669 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.652+0000 7f3b1e201b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f77e6e2d0 0x561f77e72380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:18.669 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.652+0000 7f3b1e201b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f77d59a60 con 0x561f77e6e2d0 2023-07-21T18:59:18.669 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.652+0000 7f3b1e201b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f77d59760 con 0x561f77e68580 2023-07-21T18:59:18.670 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.652+0000 7f3b1e201b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f77c3d8c0 con 0x561f77cc46b0 2023-07-21T18:59:18.670 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.652+0000 7f3b1e201b80 10 client.0 _ll_register_callbacks cb 0x561f77d57700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:59:18.670 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.652+0000 7f3b0b7fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561f77cc46b0 0x561f77e71900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:18.670 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.653+0000 7f3b0b7fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561f77cc46b0 0x561f77e71900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:52374/0 (socket says 172.21.15.39:52374) 2023-07-21T18:59:18.670 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.653+0000 7f3b0b7fe700 1 -- 172.21.15.39:0/3564108075 learned_addr learned my addr 172.21.15.39:0/3564108075 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:59:18.672 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.653+0000 7f3b0bfff700 1 --2- 172.21.15.39:0/3564108075 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f77e6e2d0 0x561f77e72380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:18.672 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.653+0000 7f3b0b7fe700 1 -- 172.21.15.39:0/3564108075 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f77e68580 msgr2=0x561f77e71e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:59:18.672 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.653+0000 7f3b0b7fe700 1 --2- 172.21.15.39:0/3564108075 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f77e68580 0x561f77e71e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:18.672 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.653+0000 7f3b0b7fe700 1 -- 172.21.15.39:0/3564108075 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f77e6e2d0 msgr2=0x561f77e72380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:59:18.672 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.653+0000 7f3b0b7fe700 1 --2- 172.21.15.39:0/3564108075 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f77e6e2d0 0x561f77e72380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.653+0000 7f3b0b7fe700 1 -- 172.21.15.39:0/3564108075 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3afc009a50 con 0x561f77cc46b0 2023-07-21T18:59:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.653+0000 7f3b0bfff700 1 --2- 172.21.15.39:0/3564108075 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f77e6e2d0 0x561f77e72380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:59:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.653+0000 7f3b0b7fe700 1 --2- 172.21.15.39:0/3564108075 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561f77cc46b0 0x561f77e71900 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3afc009340 tx=0x7f3afc0050f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6ea4282ffc3afce server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:59:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.653+0000 7f3aeaffd700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2023-07-21T18:59:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.653+0000 7f3aeaffd700 1 -- 172.21.15.39:0/3564108075 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3afc01f040 con 0x561f77cc46b0 2023-07-21T18:59:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.653+0000 7f3aeaffd700 1 -- 172.21.15.39:0/3564108075 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f3afc0046d0 con 0x561f77cc46b0 2023-07-21T18:59:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.653+0000 7f3b1e201b80 1 -- 172.21.15.39:0/3564108075 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561f77d13490 con 0x561f77cc46b0 2023-07-21T18:59:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.654+0000 7f3aeaffd700 1 -- 172.21.15.39:0/3564108075 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3afc012040 con 0x561f77cc46b0 2023-07-21T18:59:18.674 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.654+0000 7f3b1e201b80 10 client.26923 fetch_fsmap learned FSMap version 93 2023-07-21T18:59:18.674 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.654+0000 7f3b1e201b80 1 -- 172.21.15.39:0/3564108075 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=93,monmap=4+}) v3 -- 0x561f77bd0c00 con 0x561f77cc46b0 2023-07-21T18:59:18.674 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.654+0000 7f3aeaffd700 1 -- 172.21.15.39:0/3564108075 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561f77d13490 con 0x561f77cc46b0 2023-07-21T18:59:18.674 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.654+0000 7f3aeaffd700 1 -- 172.21.15.39:0/3564108075 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 93) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f3afc02a070 con 0x561f77cc46b0 2023-07-21T18:59:18.674 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.654+0000 7f3b1e201b80 10 client.26923 fetch_fsmap finished waiting for FSMap version 93 2023-07-21T18:59:18.674 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.654+0000 7f3b1e201b80 10 client.26923 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:59:18.674 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.654+0000 7f3b1e201b80 20 client.26923 populate_metadata read hostname 'smithi039' 2023-07-21T18:59:18.674 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.654+0000 7f3b1e201b80 1 --2- 172.21.15.39:0/3564108075 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x561f77e81bf0 0x561f77e84000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:18.675 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.654+0000 7f3b1e201b80 4 client.26923 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:59:18.675 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.654+0000 7f3b1e201b80 1 -- 172.21.15.39:0/3564108075 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561f77db2e50 con 0x561f77e81bf0 2023-07-21T18:59:18.675 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.655+0000 7f3b0affd700 1 --2- 172.21.15.39:0/3564108075 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x561f77e81bf0 0x561f77e84000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:18.675 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.655+0000 7f3b0affd700 1 --2- 172.21.15.39:0/3564108075 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x561f77e81bf0 0x561f77e84000 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=213a3419637c7f9b server_cookie=546fcaad9d08fd9 in_seq=0 out_seq=0 2023-07-21T18:59:18.675 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.655+0000 7f3aeaffd700 10 client.26923 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:59:18.675 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.657+0000 7f3aeaffd700 1 -- 172.21.15.39:0/3564108075 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561f77db2e50 con 0x561f77e81bf0 2023-07-21T18:59:18.675 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.657+0000 7f3aeaffd700 10 client.26923 handle_command_reply: tid=0 2023-07-21T18:59:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.657+0000 7f3b1e201b80 1 client.26923 shutdown 2023-07-21T18:59:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.657+0000 7f3b1e201b80 1 -- 172.21.15.39:0/3564108075 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561f77cc46b0 msgr2=0x561f77e71900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:59:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.657+0000 7f3b1e201b80 1 --2- 172.21.15.39:0/3564108075 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561f77cc46b0 0x561f77e71900 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3afc009340 tx=0x7f3afc0050f0 comp rx=0 tx=0).stop 2023-07-21T18:59:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.657+0000 7f3b1e201b80 1 -- 172.21.15.39:0/3564108075 shutdown_connections 2023-07-21T18:59:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.657+0000 7f3b1e201b80 1 --2- 172.21.15.39:0/3564108075 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x561f77e81bf0 0x561f77e84000 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.657+0000 7f3b1e201b80 1 --2- 172.21.15.39:0/3564108075 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f77e6e2d0 0x561f77e72380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:18.677 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.657+0000 7f3aeaffd700 0 client.26923 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:59:18.677 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.657+0000 7f3b1e201b80 1 --2- 172.21.15.39:0/3564108075 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561f77cc46b0 0x561f77e71900 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:18.677 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.657+0000 7f3b1e201b80 1 --2- 172.21.15.39:0/3564108075 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f77e68580 0x561f77e71e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:18.677 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.657+0000 7f3b1e201b80 1 -- 172.21.15.39:0/3564108075 >> 172.21.15.39:0/3564108075 conn(0x561f77e5dbe0 msgr2=0x561f77e624e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:59:18.677 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.657+0000 7f3b1e201b80 1 -- 172.21.15.39:0/3564108075 shutdown_connections 2023-07-21T18:59:18.677 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.657+0000 7f3b1e201b80 1 -- 172.21.15.39:0/3564108075 wait complete. 2023-07-21T18:59:18.677 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.657+0000 7f3b1e201b80 20 client.26923 trim_cache size 0 max 16384 2023-07-21T18:59:18.678 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:59:18 smithi039 ceph-mon[96489]: pgmap v555: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 13 MiB/s wr, 612 op/s 2023-07-21T18:59:18.682 INFO:tasks.workunit.client.0.smithi039.stdout: 10 154582 31.19 MB/sec execute 151 sec latency 90.895 ms 2023-07-21T18:59:18.689 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.678+0000 7f3b1e201b80 1 Processor -- start 2023-07-21T18:59:18.689 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.678+0000 7f3b1e201b80 1 -- start start 2023-07-21T18:59:18.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.679+0000 7f3b1e201b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f77e6c2a0 0x561f77e68040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:18.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.679+0000 7f3b1e201b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f77e5dbe0 0x561f77e724f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:18.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.679+0000 7f3b1e201b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561f77cc46b0 0x561f77e6d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:18.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.679+0000 7f3b1e201b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f77e6dfe0 con 0x561f77e6c2a0 2023-07-21T18:59:18.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.679+0000 7f3b1e201b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f77e64b70 con 0x561f77e5dbe0 2023-07-21T18:59:18.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.679+0000 7f3b1e201b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f77e6da10 con 0x561f77cc46b0 2023-07-21T18:59:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.679+0000 7f3b0b7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f77e6c2a0 0x561f77e68040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.679+0000 7f3b0b7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f77e6c2a0 0x561f77e68040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33830/0 (socket says 172.21.15.39:33830) 2023-07-21T18:59:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.679+0000 7f3b0b7fe700 1 -- 172.21.15.39:0/3800659822 learned_addr learned my addr 172.21.15.39:0/3800659822 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:59:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.679+0000 7f3b0bfff700 1 --2- 172.21.15.39:0/3800659822 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561f77cc46b0 0x561f77e6d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.679+0000 7f3b0affd700 1 --2- 172.21.15.39:0/3800659822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f77e5dbe0 0x561f77e724f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.679+0000 7f3b0b7fe700 1 -- 172.21.15.39:0/3800659822 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561f77cc46b0 msgr2=0x561f77e6d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:59:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.679+0000 7f3b0b7fe700 1 --2- 172.21.15.39:0/3800659822 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561f77cc46b0 0x561f77e6d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.679+0000 7f3b0b7fe700 1 -- 172.21.15.39:0/3800659822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f77e5dbe0 msgr2=0x561f77e724f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:59:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.679+0000 7f3b0b7fe700 1 --2- 172.21.15.39:0/3800659822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f77e5dbe0 0x561f77e724f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.679+0000 7f3b0b7fe700 1 -- 172.21.15.39:0/3800659822 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f77e68c20 con 0x561f77e6c2a0 2023-07-21T18:59:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.679+0000 7f3b0b7fe700 1 --2- 172.21.15.39:0/3800659822 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f77e6c2a0 0x561f77e68040 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f3afc005cd0 tx=0x7f3afc005210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1ea8b1992fd9945 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:59:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.680+0000 7f3af27fc700 1 -- 172.21.15.39:0/3800659822 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3afc012070 con 0x561f77e6c2a0 2023-07-21T18:59:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.680+0000 7f3af27fc700 1 -- 172.21.15.39:0/3800659822 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3afc00d350 con 0x561f77e6c2a0 2023-07-21T18:59:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.680+0000 7f3af27fc700 1 -- 172.21.15.39:0/3800659822 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3afc01f040 con 0x561f77e6c2a0 2023-07-21T18:59:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.680+0000 7f3b1e201b80 1 -- 172.21.15.39:0/3800659822 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f77e6c2a0 msgr2=0x561f77e68040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:59:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.680+0000 7f3b1e201b80 1 --2- 172.21.15.39:0/3800659822 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f77e6c2a0 0x561f77e68040 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f3afc005cd0 tx=0x7f3afc005210 comp rx=0 tx=0).stop 2023-07-21T18:59:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.681+0000 7f3b1e201b80 1 -- 172.21.15.39:0/3800659822 shutdown_connections 2023-07-21T18:59:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.681+0000 7f3b1e201b80 1 --2- 172.21.15.39:0/3800659822 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561f77cc46b0 0x561f77e6d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.681+0000 7f3b1e201b80 1 --2- 172.21.15.39:0/3800659822 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f77e6c2a0 0x561f77e68040 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.681+0000 7f3b1e201b80 1 --2- 172.21.15.39:0/3800659822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f77e5dbe0 0x561f77e724f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.681+0000 7f3b1e201b80 1 -- 172.21.15.39:0/3800659822 >> 172.21.15.39:0/3800659822 conn(0x561f77e6bed0 msgr2=0x561f77e73210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:59:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.681+0000 7f3b1e201b80 1 -- 172.21.15.39:0/3800659822 shutdown_connections 2023-07-21T18:59:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.681+0000 7f3b1e201b80 1 -- 172.21.15.39:0/3800659822 wait complete. 2023-07-21T18:59:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.681+0000 7f3b1e201b80 1 -- start start 2023-07-21T18:59:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.681+0000 7f3b1e201b80 1 Processor -- start 2023-07-21T18:59:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.681+0000 7f3b1e201b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f77cc46b0 0x561f77ecdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.681+0000 7f3b1e201b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561f77e5dbe0 0x561f77ece290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.681+0000 7f3b1e201b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f77e6c2a0 0x561f77eceb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.681+0000 7f3b1e201b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f77ecf0d0 con 0x561f77e6c2a0 2023-07-21T18:59:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.681+0000 7f3b1e201b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f77ecf240 con 0x561f77cc46b0 2023-07-21T18:59:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.681+0000 7f3b1e201b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f77ecf400 con 0x561f77e5dbe0 2023-07-21T18:59:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.682+0000 7f3b1e201b80 10 client.0 _ll_register_callbacks cb 0x561f77d57700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:59:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.682+0000 7f3b0bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f77e6c2a0 0x561f77eceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.682+0000 7f3b0bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f77e6c2a0 0x561f77eceb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33840/0 (socket says 172.21.15.39:33840) 2023-07-21T18:59:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.682+0000 7f3b0bfff700 1 -- 172.21.15.39:0/47132559 learned_addr learned my addr 172.21.15.39:0/47132559 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:59:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.682+0000 7f3b0b7fe700 1 --2- 172.21.15.39:0/47132559 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f77cc46b0 0x561f77ecdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.682+0000 7f3b0affd700 1 --2- 172.21.15.39:0/47132559 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561f77e5dbe0 0x561f77ece290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.682+0000 7f3b0bfff700 1 -- 172.21.15.39:0/47132559 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561f77e5dbe0 msgr2=0x561f77ece290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:59:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.682+0000 7f3b0bfff700 1 --2- 172.21.15.39:0/47132559 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561f77e5dbe0 0x561f77ece290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.682+0000 7f3b0bfff700 1 -- 172.21.15.39:0/47132559 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f77cc46b0 msgr2=0x561f77ecdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:59:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.682+0000 7f3b0bfff700 1 --2- 172.21.15.39:0/47132559 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f77cc46b0 0x561f77ecdd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.682+0000 7f3b0bfff700 1 -- 172.21.15.39:0/47132559 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561f77bd0c00 con 0x561f77e6c2a0 2023-07-21T18:59:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.682+0000 7f3b0affd700 1 --2- 172.21.15.39:0/47132559 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561f77e5dbe0 0x561f77ece290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:59:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.682+0000 7f3b0bfff700 1 --2- 172.21.15.39:0/47132559 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f77e6c2a0 0x561f77eceb00 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f3b04099800 tx=0x7f3b040994e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c388338f37d6e42 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:59:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.682+0000 7f3aeaffd700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:59:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.682+0000 7f3aeaffd700 1 -- 172.21.15.39:0/47132559 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b0409db30 con 0x561f77e6c2a0 2023-07-21T18:59:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.682+0000 7f3aeaffd700 1 -- 172.21.15.39:0/47132559 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f3b040a1ec0 con 0x561f77e6c2a0 2023-07-21T18:59:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.683+0000 7f3b1e201b80 1 -- 172.21.15.39:0/47132559 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3afc009da0 con 0x561f77e6c2a0 2023-07-21T18:59:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.683+0000 7f3aeaffd700 1 -- 172.21.15.39:0/47132559 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b040b2120 con 0x561f77e6c2a0 2023-07-21T18:59:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.683+0000 7f3b1e201b80 10 client.17478 fetch_fsmap learned FSMap version 93 2023-07-21T18:59:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.683+0000 7f3b1e201b80 1 -- 172.21.15.39:0/47132559 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=93,monmap=4+}) v3 -- 0x561f77c0e1f0 con 0x561f77e6c2a0 2023-07-21T18:59:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.683+0000 7f3aeaffd700 1 -- 172.21.15.39:0/47132559 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3afc009da0 con 0x561f77e6c2a0 2023-07-21T18:59:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.683+0000 7f3aeaffd700 1 -- 172.21.15.39:0/47132559 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 93) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f3b04098a60 con 0x561f77e6c2a0 2023-07-21T18:59:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.683+0000 7f3b1e201b80 10 client.17478 fetch_fsmap finished waiting for FSMap version 93 2023-07-21T18:59:18.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.683+0000 7f3b1e201b80 10 client.17478 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:59:18.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.683+0000 7f3b1e201b80 20 client.17478 populate_metadata read hostname 'smithi039' 2023-07-21T18:59:18.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.683+0000 7f3b1e201b80 1 --2- 172.21.15.39:0/47132559 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x561f77e68580 0x561f77ed8d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:18.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.683+0000 7f3b1e201b80 4 client.17478 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:59:18.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.683+0000 7f3b1e201b80 1 -- 172.21.15.39:0/47132559 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561f77d74ff0 con 0x561f77e68580 2023-07-21T18:59:18.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.683+0000 7f3b0b7fe700 1 --2- 172.21.15.39:0/47132559 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x561f77e68580 0x561f77ed8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:18.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.684+0000 7f3aeaffd700 10 client.17478 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:59:18.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.684+0000 7f3b0b7fe700 1 --2- 172.21.15.39:0/47132559 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x561f77e68580 0x561f77ed8d60 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da872cf5d71851c1 server_cookie=78354ecb06bc6b3a in_seq=0 out_seq=0 2023-07-21T18:59:18.827 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.815+0000 7f3aeaffd700 1 -- 172.21.15.39:0/47132559 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561f77d74ff0 con 0x561f77e68580 2023-07-21T18:59:18.827 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.815+0000 7f3aeaffd700 10 client.17478 handle_command_reply: tid=0 2023-07-21T18:59:18.827 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.815+0000 7f3b1e201b80 1 client.17478 shutdown 2023-07-21T18:59:18.827 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.815+0000 7f3b1e201b80 1 -- 172.21.15.39:0/47132559 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f77e6c2a0 msgr2=0x561f77eceb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:59:18.828 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.815+0000 7f3b1e201b80 1 --2- 172.21.15.39:0/47132559 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f77e6c2a0 0x561f77eceb00 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f3b04099800 tx=0x7f3b040994e0 comp rx=0 tx=0).stop 2023-07-21T18:59:18.828 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.815+0000 7f3b1e201b80 1 -- 172.21.15.39:0/47132559 shutdown_connections 2023-07-21T18:59:18.828 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.815+0000 7f3b1e201b80 1 --2- 172.21.15.39:0/47132559 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x561f77e68580 0x561f77ed8d60 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:18.828 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.815+0000 7f3b1e201b80 1 --2- 172.21.15.39:0/47132559 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f77cc46b0 0x561f77ecdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:18.828 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.815+0000 7f3b1e201b80 1 --2- 172.21.15.39:0/47132559 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f77e6c2a0 0x561f77eceb00 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:18.829 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.815+0000 7f3b1e201b80 1 --2- 172.21.15.39:0/47132559 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561f77e5dbe0 0x561f77ece290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:18.829 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.815+0000 7f3b1e201b80 1 -- 172.21.15.39:0/47132559 >> 172.21.15.39:0/47132559 conn(0x561f77e6bed0 msgr2=0x561f77e6d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:59:18.829 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.815+0000 7f3aeaffd700 0 client.17478 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:59:18.829 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.815+0000 7f3b1e201b80 1 -- 172.21.15.39:0/47132559 shutdown_connections 2023-07-21T18:59:18.829 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.815+0000 7f3b1e201b80 1 -- 172.21.15.39:0/47132559 wait complete. 2023-07-21T18:59:18.829 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.815+0000 7f3b1e201b80 20 client.17478 trim_cache size 0 max 16384 2023-07-21T18:59:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:59:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:59:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:59:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:59:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.816+0000 7f3b1207b700 1 -- 172.21.15.39:0/304942403 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f3af40a12c0 msgr2=0x7f3af40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:59:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.816+0000 7f3b1207b700 1 --2- 172.21.15.39:0/304942403 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f3af40a12c0 0x7f3af40a3770 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c0ffb70 tx=0x7f3afc00f040 comp rx=0 tx=0).stop 2023-07-21T18:59:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.816+0000 7f3b1207b700 1 -- 172.21.15.39:0/304942403 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b0c104a40 msgr2=0x7f3b0c193760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:59:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.816+0000 7f3b1207b700 1 --2- 172.21.15.39:0/304942403 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b0c104a40 0x7f3b0c193760 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f3b0400efc0 tx=0x7f3b0400c3a0 comp rx=0 tx=0).stop 2023-07-21T18:59:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.816+0000 7f3b1207b700 1 -- 172.21.15.39:0/304942403 shutdown_connections 2023-07-21T18:59:18.833 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.816+0000 7f3b1207b700 1 --2- 172.21.15.39:0/304942403 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f3af40a12c0 0x7f3af40a3770 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:18.833 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.816+0000 7f3b1207b700 1 --2- 172.21.15.39:0/304942403 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b0c0fea30 0x7f3b0c199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:18.833 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.816+0000 7f3b1207b700 1 --2- 172.21.15.39:0/304942403 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b0c104a40 0x7f3b0c193760 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:18.833 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.816+0000 7f3b1207b700 1 --2- 172.21.15.39:0/304942403 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b0c0ff3e0 0x7f3b0c199e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:18.833 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.816+0000 7f3b1207b700 1 -- 172.21.15.39:0/304942403 >> 172.21.15.39:0/304942403 conn(0x7f3b0c0fa4f0 msgr2=0x7f3b0c105840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:59:18.833 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.819+0000 7f3b1207b700 1 -- 172.21.15.39:0/304942403 shutdown_connections 2023-07-21T18:59:18.833 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:18.819+0000 7f3b1207b700 1 -- 172.21.15.39:0/304942403 wait complete. 2023-07-21T18:59:18.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:59:18 smithi193 ceph-mon[98204]: pgmap v555: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 13 MiB/s wr, 612 op/s 2023-07-21T18:59:18.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:59:18 smithi155 ceph-mon[97934]: pgmap v555: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 13 MiB/s wr, 612 op/s 2023-07-21T18:59:19.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:59:19 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:59:19.682 INFO:tasks.workunit.client.0.smithi039.stdout: 10 155087 31.18 MB/sec execute 152 sec latency 97.356 ms 2023-07-21T18:59:19.863 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:59:19 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:59:19.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:59:19 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:59:20.682 INFO:tasks.workunit.client.0.smithi039.stdout: 10 155680 31.15 MB/sec execute 153 sec latency 95.927 ms 2023-07-21T18:59:20.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:59:20 smithi193 ceph-mon[98204]: pgmap v556: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 13 MiB/s wr, 612 op/s 2023-07-21T18:59:20.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:59:20 smithi155 ceph-mon[97934]: pgmap v556: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 13 MiB/s wr, 612 op/s 2023-07-21T18:59:20.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:59:20 smithi039 ceph-mon[96489]: pgmap v556: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 13 MiB/s wr, 612 op/s 2023-07-21T18:59:21.682 INFO:tasks.workunit.client.0.smithi039.stdout: 10 156294 31.21 MB/sec execute 154 sec latency 77.532 ms 2023-07-21T18:59:22.682 INFO:tasks.workunit.client.0.smithi039.stdout: 10 156963 31.23 MB/sec execute 155 sec latency 94.522 ms 2023-07-21T18:59:22.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:59:22 smithi193 ceph-mon[98204]: pgmap v557: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 12 MiB/s wr, 535 op/s 2023-07-21T18:59:22.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:59:22 smithi155 ceph-mon[97934]: pgmap v557: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 12 MiB/s wr, 535 op/s 2023-07-21T18:59:22.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:59:22 smithi039 ceph-mon[96489]: pgmap v557: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 12 MiB/s wr, 535 op/s 2023-07-21T18:59:23.682 INFO:tasks.workunit.client.0.smithi039.stdout: 10 157613 31.25 MB/sec execute 156 sec latency 65.036 ms 2023-07-21T18:59:24.682 INFO:tasks.workunit.client.0.smithi039.stdout: 10 158127 31.23 MB/sec execute 157 sec latency 88.444 ms 2023-07-21T18:59:24.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:59:24 smithi193 ceph-mon[98204]: pgmap v558: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 16 MiB/s wr, 793 op/s 2023-07-21T18:59:24.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:59:24 smithi155 ceph-mon[97934]: pgmap v558: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 16 MiB/s wr, 793 op/s 2023-07-21T18:59:24.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:59:24 smithi039 ceph-mon[96489]: pgmap v558: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 16 MiB/s wr, 793 op/s 2023-07-21T18:59:25.683 INFO:tasks.workunit.client.0.smithi039.stdout: 10 158682 31.21 MB/sec execute 158 sec latency 142.701 ms 2023-07-21T18:59:26.683 INFO:tasks.workunit.client.0.smithi039.stdout: 10 159251 31.17 MB/sec execute 159 sec latency 123.480 ms 2023-07-21T18:59:26.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:59:26 smithi193 ceph-mon[98204]: pgmap v559: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 16 MiB/s wr, 793 op/s 2023-07-21T18:59:26.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:59:26 smithi155 ceph-mon[97934]: pgmap v559: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 16 MiB/s wr, 793 op/s 2023-07-21T18:59:26.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:59:26 smithi039 ceph-mon[96489]: pgmap v559: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 16 MiB/s wr, 793 op/s 2023-07-21T18:59:27.683 INFO:tasks.workunit.client.0.smithi039.stdout: 10 159876 31.20 MB/sec execute 160 sec latency 254.044 ms 2023-07-21T18:59:27.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:59:27 smithi193 ceph-mon[98204]: pgmap v560: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 10 MiB/s wr, 683 op/s 2023-07-21T18:59:27.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:59:27 smithi155 ceph-mon[97934]: pgmap v560: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 10 MiB/s wr, 683 op/s 2023-07-21T18:59:27.928 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:59:27 smithi039 ceph-mon[96489]: pgmap v560: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 10 MiB/s wr, 683 op/s 2023-07-21T18:59:28.683 INFO:tasks.workunit.client.0.smithi039.stdout: 10 160575 31.29 MB/sec execute 161 sec latency 119.732 ms 2023-07-21T18:59:29.683 INFO:tasks.workunit.client.0.smithi039.stdout: 10 161149 31.26 MB/sec execute 162 sec latency 95.702 ms 2023-07-21T18:59:29.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:59:29 smithi039 ceph-mon[96489]: pgmap v561: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 8.6 MiB/s wr, 586 op/s 2023-07-21T18:59:30.108 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:59:29 smithi193 ceph-mon[98204]: pgmap v561: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 8.6 MiB/s wr, 586 op/s 2023-07-21T18:59:30.126 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:59:29 smithi155 ceph-mon[97934]: pgmap v561: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 8.6 MiB/s wr, 586 op/s 2023-07-21T18:59:30.683 INFO:tasks.workunit.client.0.smithi039.stdout: 10 161631 31.21 MB/sec execute 163 sec latency 124.993 ms 2023-07-21T18:59:31.683 INFO:tasks.workunit.client.0.smithi039.stdout: 10 162227 31.25 MB/sec execute 164 sec latency 72.868 ms 2023-07-21T18:59:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.969+0000 7fa35e544700 1 -- 172.21.15.39:0/2697716116 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa35810c4d0 msgr2=0x7fa3581140a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:59:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.970+0000 7fa35e544700 1 --2- 172.21.15.39:0/2697716116 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa35810c4d0 0x7fa3581140a0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fa35000b0d0 tx=0x7fa35000b3e0 comp rx=0 tx=0).stop 2023-07-21T18:59:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.970+0000 7fa35e544700 1 -- 172.21.15.39:0/2697716116 shutdown_connections 2023-07-21T18:59:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.970+0000 7fa35e544700 1 --2- 172.21.15.39:0/2697716116 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa35810c4d0 0x7fa3581140a0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.970+0000 7fa35e544700 1 --2- 172.21.15.39:0/2697716116 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa35806cb80 0x7fa35806cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.970+0000 7fa35e544700 1 --2- 172.21.15.39:0/2697716116 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa35806d520 0x7fa35810bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.970+0000 7fa35e544700 1 -- 172.21.15.39:0/2697716116 >> 172.21.15.39:0/2697716116 conn(0x7fa35806bd70 msgr2=0x7fa358070f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:59:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.970+0000 7fa35e544700 1 -- 172.21.15.39:0/2697716116 shutdown_connections 2023-07-21T18:59:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.970+0000 7fa35e544700 1 -- 172.21.15.39:0/2697716116 wait complete. 2023-07-21T18:59:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.971+0000 7fa35e544700 1 Processor -- start 2023-07-21T18:59:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.971+0000 7fa35e544700 1 -- start start 2023-07-21T18:59:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.971+0000 7fa35e544700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa35806cb80 0x7fa35819c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.971+0000 7fa35e544700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa35806d520 0x7fa35819c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.971+0000 7fa35e544700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa35810c4d0 0x7fa3581a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.971+0000 7fa35e544700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3581a1440 con 0x7fa35806d520 2023-07-21T18:59:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.971+0000 7fa35e544700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3581a15b0 con 0x7fa35810c4d0 2023-07-21T18:59:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.971+0000 7fa35e544700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3581a1770 con 0x7fa35806cb80 2023-07-21T18:59:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.971+0000 7fa3577fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa35806d520 0x7fa35819c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.971+0000 7fa3577fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa35806d520 0x7fa35819c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33852/0 (socket says 172.21.15.39:33852) 2023-07-21T18:59:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.971+0000 7fa3577fe700 1 -- 172.21.15.39:0/3622448011 learned_addr learned my addr 172.21.15.39:0/3622448011 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:59:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.971+0000 7fa357fff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa35806cb80 0x7fa35819c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.972+0000 7fa3577fe700 1 -- 172.21.15.39:0/3622448011 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa35806cb80 msgr2=0x7fa35819c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:59:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.972+0000 7fa3577fe700 1 --2- 172.21.15.39:0/3622448011 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa35806cb80 0x7fa35819c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.972+0000 7fa3577fe700 1 -- 172.21.15.39:0/3622448011 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa35810c4d0 msgr2=0x7fa3581a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:59:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.972+0000 7fa3577fe700 1 --2- 172.21.15.39:0/3622448011 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa35810c4d0 0x7fa3581a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.972+0000 7fa3577fe700 1 -- 172.21.15.39:0/3622448011 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa350009d00 con 0x7fa35806d520 2023-07-21T18:59:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.972+0000 7fa3577fe700 1 --2- 172.21.15.39:0/3622448011 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa35806d520 0x7fa35819c790 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fa34c00b4d0 tx=0x7fa34c00b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:59:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.972+0000 7fa3557fa700 1 -- 172.21.15.39:0/3622448011 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34c011830 con 0x7fa35806d520 2023-07-21T18:59:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.972+0000 7fa3557fa700 1 -- 172.21.15.39:0/3622448011 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa34c011e50 con 0x7fa35806d520 2023-07-21T18:59:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.972+0000 7fa35e544700 1 -- 172.21.15.39:0/3622448011 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3581a1a50 con 0x7fa35806d520 2023-07-21T18:59:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.972+0000 7fa3557fa700 1 -- 172.21.15.39:0/3622448011 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34c00f2e0 con 0x7fa35806d520 2023-07-21T18:59:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.973+0000 7fa35e544700 1 -- 172.21.15.39:0/3622448011 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3581a9290 con 0x7fa35806d520 2023-07-21T18:59:31.987 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.975+0000 7fa3557fa700 1 -- 172.21.15.39:0/3622448011 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fa34c011990 con 0x7fa35806d520 2023-07-21T18:59:31.987 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.976+0000 7fa3557fa700 1 --2- 172.21.15.39:0/3622448011 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fa3400a34d0 0x7fa3400a5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:31.987 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.976+0000 7fa3557fa700 1 -- 172.21.15.39:0/3622448011 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fa34c0cc610 con 0x7fa35806d520 2023-07-21T18:59:31.987 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.977+0000 7fa357fff700 1 --2- 172.21.15.39:0/3622448011 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fa3400a34d0 0x7fa3400a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.977+0000 7fa357fff700 1 --2- 172.21.15.39:0/3622448011 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fa3400a34d0 0x7fa3400a5980 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa348009c40 tx=0x7fa348009260 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:59:31.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.978+0000 7fa36a6cab80 1 Processor -- start 2023-07-21T18:59:31.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.978+0000 7fa36a6cab80 1 -- start start 2023-07-21T18:59:31.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.978+0000 7fa36a6cab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5586cb7896b0 0x5586cb932d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:31.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.978+0000 7fa36a6cab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586cb9332d0 0x5586cb92d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.978+0000 7fa36a6cab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5586cb92d580 0x5586cb92d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.978+0000 7fa36a6cab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586cb832c50 con 0x5586cb92d580 2023-07-21T18:59:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.978+0000 7fa36a6cab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586cb8859e0 con 0x5586cb9332d0 2023-07-21T18:59:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.978+0000 7fa36a6cab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586cb87ee70 con 0x5586cb7896b0 2023-07-21T18:59:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.978+0000 7fa357fff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5586cb7896b0 0x5586cb932d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.978+0000 7fa35cae1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5586cb92d580 0x5586cb92d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.978+0000 7fa357fff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5586cb7896b0 0x5586cb932d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:52410/0 (socket says 172.21.15.39:52410) 2023-07-21T18:59:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.978+0000 7fa357fff700 1 -- 172.21.15.39:0/3392784024 learned_addr learned my addr 172.21.15.39:0/3392784024 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:59:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.978+0000 7fa3577fe700 1 --2- 172.21.15.39:0/3392784024 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586cb9332d0 0x5586cb92d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.979+0000 7fa3577fe700 1 -- 172.21.15.39:0/3392784024 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5586cb7896b0 msgr2=0x5586cb932d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:59:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.979+0000 7fa3577fe700 1 --2- 172.21.15.39:0/3392784024 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5586cb7896b0 0x5586cb932d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.979+0000 7fa3577fe700 1 -- 172.21.15.39:0/3392784024 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5586cb92d580 msgr2=0x5586cb92d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:59:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.979+0000 7fa3577fe700 1 --2- 172.21.15.39:0/3392784024 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5586cb92d580 0x5586cb92d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.979+0000 7fa3577fe700 1 -- 172.21.15.39:0/3392784024 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5586cb6acd80 con 0x5586cb9332d0 2023-07-21T18:59:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.979+0000 7fa3577fe700 1 --2- 172.21.15.39:0/3392784024 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586cb9332d0 0x5586cb92d040 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa34c0c9e70 tx=0x7fa34c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f67d25174a34465 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:59:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.979+0000 7fa33e7fc700 1 -- 172.21.15.39:0/3392784024 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34c09d820 con 0x5586cb9332d0 2023-07-21T18:59:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.979+0000 7fa33e7fc700 1 -- 172.21.15.39:0/3392784024 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa34c0a1c40 con 0x5586cb9332d0 2023-07-21T18:59:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.979+0000 7fa33e7fc700 1 -- 172.21.15.39:0/3392784024 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34c0b92f0 con 0x5586cb9332d0 2023-07-21T18:59:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.980+0000 7fa36a6cab80 1 -- 172.21.15.39:0/3392784024 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586cb9332d0 msgr2=0x5586cb92d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:59:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.980+0000 7fa36a6cab80 1 --2- 172.21.15.39:0/3392784024 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586cb9332d0 0x5586cb92d040 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa34c0c9e70 tx=0x7fa34c004750 comp rx=0 tx=0).stop 2023-07-21T18:59:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.981+0000 7fa36a6cab80 1 -- 172.21.15.39:0/3392784024 shutdown_connections 2023-07-21T18:59:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.981+0000 7fa36a6cab80 1 --2- 172.21.15.39:0/3392784024 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5586cb92d580 0x5586cb92d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.981+0000 7fa36a6cab80 1 --2- 172.21.15.39:0/3392784024 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5586cb7896b0 0x5586cb932d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.981+0000 7fa36a6cab80 1 --2- 172.21.15.39:0/3392784024 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586cb9332d0 0x5586cb92d040 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.981+0000 7fa36a6cab80 1 -- 172.21.15.39:0/3392784024 >> 172.21.15.39:0/3392784024 conn(0x5586cb922be0 msgr2=0x5586cb927f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:59:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.981+0000 7fa36a6cab80 1 -- 172.21.15.39:0/3392784024 shutdown_connections 2023-07-21T18:59:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.981+0000 7fa36a6cab80 1 -- 172.21.15.39:0/3392784024 wait complete. 2023-07-21T18:59:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.981+0000 7fa36a6cab80 1 -- start start 2023-07-21T18:59:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.982+0000 7fa36a6cab80 1 Processor -- start 2023-07-21T18:59:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.982+0000 7fa36a6cab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5586cb7896b0 0x5586cb939650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.982+0000 7fa36a6cab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586cb92d580 0x5586cb936940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.982+0000 7fa36a6cab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5586cb9332d0 0x5586cb937080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.982+0000 7fa36a6cab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586cb81ea60 con 0x5586cb7896b0 2023-07-21T18:59:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.982+0000 7fa36a6cab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586cb81e760 con 0x5586cb92d580 2023-07-21T18:59:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.982+0000 7fa36a6cab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586cb7028c0 con 0x5586cb9332d0 2023-07-21T18:59:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.982+0000 7fa36a6cab80 10 client.0 _ll_register_callbacks cb 0x5586cb81c700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:59:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.982+0000 7fa357fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5586cb7896b0 0x5586cb939650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.982+0000 7fa357fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5586cb7896b0 0x5586cb939650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33864/0 (socket says 172.21.15.39:33864) 2023-07-21T18:59:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.982+0000 7fa357fff700 1 -- 172.21.15.39:0/174957011 learned_addr learned my addr 172.21.15.39:0/174957011 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:59:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.982+0000 7fa35cae1700 1 --2- 172.21.15.39:0/174957011 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5586cb9332d0 0x5586cb937080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.983+0000 7fa357fff700 1 -- 172.21.15.39:0/174957011 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5586cb9332d0 msgr2=0x5586cb937080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:59:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.983+0000 7fa357fff700 1 --2- 172.21.15.39:0/174957011 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5586cb9332d0 0x5586cb937080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.983+0000 7fa357fff700 1 -- 172.21.15.39:0/174957011 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586cb92d580 msgr2=0x5586cb936940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:59:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.983+0000 7fa357fff700 1 --2- 172.21.15.39:0/174957011 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586cb92d580 0x5586cb936940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.983+0000 7fa357fff700 1 -- 172.21.15.39:0/174957011 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3581a9290 con 0x5586cb7896b0 2023-07-21T18:59:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.983+0000 7fa357fff700 1 --2- 172.21.15.39:0/174957011 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5586cb7896b0 0x5586cb939650 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fa348004490 tx=0x7fa3480047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e2cb302b887bf7a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:59:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.983+0000 7fa3377fe700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:59:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.983+0000 7fa3377fe700 1 -- 172.21.15.39:0/174957011 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa348003ea0 con 0x5586cb7896b0 2023-07-21T18:59:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.983+0000 7fa3377fe700 1 -- 172.21.15.39:0/174957011 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fa348028670 con 0x5586cb7896b0 2023-07-21T18:59:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.983+0000 7fa36a6cab80 1 -- 172.21.15.39:0/174957011 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5586cb7d8490 con 0x5586cb7896b0 2023-07-21T18:59:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.984+0000 7fa3377fe700 1 -- 172.21.15.39:0/174957011 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa348011a40 con 0x5586cb7896b0 2023-07-21T18:59:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.984+0000 7fa36a6cab80 10 client.17490 fetch_fsmap learned FSMap version 93 2023-07-21T18:59:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.984+0000 7fa36a6cab80 1 -- 172.21.15.39:0/174957011 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=93,monmap=4+}) v3 -- 0x5586cb695c00 con 0x5586cb7896b0 2023-07-21T18:59:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.984+0000 7fa3377fe700 1 -- 172.21.15.39:0/174957011 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5586cb7d8490 con 0x5586cb7896b0 2023-07-21T18:59:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.984+0000 7fa3377fe700 1 -- 172.21.15.39:0/174957011 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 93) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fa34801f640 con 0x5586cb7896b0 2023-07-21T18:59:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.984+0000 7fa36a6cab80 10 client.17490 fetch_fsmap finished waiting for FSMap version 93 2023-07-21T18:59:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.984+0000 7fa36a6cab80 10 client.17490 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:59:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.984+0000 7fa36a6cab80 20 client.17490 populate_metadata read hostname 'smithi039' 2023-07-21T18:59:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.984+0000 7fa36a6cab80 1 --2- 172.21.15.39:0/174957011 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5586cb946c90 0x5586cb9490a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.984+0000 7fa36a6cab80 4 client.17490 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:59:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.984+0000 7fa36a6cab80 1 -- 172.21.15.39:0/174957011 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5586cb877e50 con 0x5586cb946c90 2023-07-21T18:59:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.986+0000 7fa3577fe700 1 --2- 172.21.15.39:0/174957011 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5586cb946c90 0x5586cb9490a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.986+0000 7fa3377fe700 10 client.17490 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:59:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.986+0000 7fa3577fe700 1 --2- 172.21.15.39:0/174957011 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5586cb946c90 0x5586cb9490a0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37a40fc5097f1a51 server_cookie=35dcd7d514491d1e in_seq=0 out_seq=0 2023-07-21T18:59:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.987+0000 7fa3377fe700 1 -- 172.21.15.39:0/174957011 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5586cb877e50 con 0x5586cb946c90 2023-07-21T18:59:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.987+0000 7fa3377fe700 10 client.17490 handle_command_reply: tid=0 2023-07-21T18:59:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.987+0000 7fa36a6cab80 1 client.17490 shutdown 2023-07-21T18:59:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.987+0000 7fa36a6cab80 1 -- 172.21.15.39:0/174957011 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5586cb7896b0 msgr2=0x5586cb939650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:59:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.987+0000 7fa36a6cab80 1 --2- 172.21.15.39:0/174957011 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5586cb7896b0 0x5586cb939650 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fa348004490 tx=0x7fa3480047c0 comp rx=0 tx=0).stop 2023-07-21T18:59:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.988+0000 7fa36a6cab80 1 -- 172.21.15.39:0/174957011 shutdown_connections 2023-07-21T18:59:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.988+0000 7fa36a6cab80 1 --2- 172.21.15.39:0/174957011 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5586cb946c90 0x5586cb9490a0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.988+0000 7fa36a6cab80 1 --2- 172.21.15.39:0/174957011 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5586cb9332d0 0x5586cb937080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.988+0000 7fa36a6cab80 1 --2- 172.21.15.39:0/174957011 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5586cb7896b0 0x5586cb939650 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.988+0000 7fa36a6cab80 1 --2- 172.21.15.39:0/174957011 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586cb92d580 0x5586cb936940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.988+0000 7fa3377fe700 0 client.17490 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:59:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.988+0000 7fa36a6cab80 1 -- 172.21.15.39:0/174957011 >> 172.21.15.39:0/174957011 conn(0x5586cb922be0 msgr2=0x5586cb926c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:59:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.988+0000 7fa36a6cab80 1 -- 172.21.15.39:0/174957011 shutdown_connections 2023-07-21T18:59:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.988+0000 7fa36a6cab80 1 -- 172.21.15.39:0/174957011 wait complete. 2023-07-21T18:59:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:31.988+0000 7fa36a6cab80 20 client.17490 trim_cache size 0 max 16384 2023-07-21T18:59:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.013+0000 7fa36a6cab80 1 Processor -- start 2023-07-21T18:59:32.025 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.013+0000 7fa36a6cab80 1 -- start start 2023-07-21T18:59:32.025 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.013+0000 7fa36a6cab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5586cb7896b0 0x5586cb9321d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:32.025 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.013+0000 7fa36a6cab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586cb92d580 0x5586cb927c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:32.025 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.013+0000 7fa36a6cab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5586cb9332d0 0x5586cb9391f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:32.025 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.013+0000 7fa36a6cab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586cb924d00 con 0x5586cb9332d0 2023-07-21T18:59:32.026 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.013+0000 7fa36a6cab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586cb9329f0 con 0x5586cb92d580 2023-07-21T18:59:32.026 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.013+0000 7fa36a6cab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586cb91fbb0 con 0x5586cb7896b0 2023-07-21T18:59:32.026 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.014+0000 7fa35cae1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5586cb9332d0 0x5586cb9391f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:32.026 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.014+0000 7fa35cae1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5586cb9332d0 0x5586cb9391f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33876/0 (socket says 172.21.15.39:33876) 2023-07-21T18:59:32.026 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.014+0000 7fa35cae1700 1 -- 172.21.15.39:0/3960101122 learned_addr learned my addr 172.21.15.39:0/3960101122 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:59:32.026 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.014+0000 7fa357fff700 1 --2- 172.21.15.39:0/3960101122 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5586cb7896b0 0x5586cb9321d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:32.026 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.014+0000 7fa3577fe700 1 --2- 172.21.15.39:0/3960101122 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586cb92d580 0x5586cb927c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:32.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.014+0000 7fa35cae1700 1 -- 172.21.15.39:0/3960101122 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5586cb7896b0 msgr2=0x5586cb9321d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:59:32.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.014+0000 7fa35cae1700 1 --2- 172.21.15.39:0/3960101122 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5586cb7896b0 0x5586cb9321d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:32.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.014+0000 7fa35cae1700 1 -- 172.21.15.39:0/3960101122 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586cb92d580 msgr2=0x5586cb927c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:59:32.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.014+0000 7fa35cae1700 1 --2- 172.21.15.39:0/3960101122 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586cb92d580 0x5586cb927c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:32.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.014+0000 7fa35cae1700 1 -- 172.21.15.39:0/3960101122 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5586cb92dc20 con 0x5586cb9332d0 2023-07-21T18:59:32.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.014+0000 7fa35cae1700 1 --2- 172.21.15.39:0/3960101122 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5586cb9332d0 0x5586cb9391f0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fa3500095f0 tx=0x7fa350008d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=176a89a7493cda97 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:59:32.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.014+0000 7fa33effd700 1 -- 172.21.15.39:0/3960101122 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35000e070 con 0x5586cb9332d0 2023-07-21T18:59:32.028 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.014+0000 7fa33effd700 1 -- 172.21.15.39:0/3960101122 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa350008eb0 con 0x5586cb9332d0 2023-07-21T18:59:32.028 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.014+0000 7fa33effd700 1 -- 172.21.15.39:0/3960101122 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3500050b0 con 0x5586cb9332d0 2023-07-21T18:59:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.015+0000 7fa36a6cab80 1 -- 172.21.15.39:0/3960101122 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5586cb9332d0 msgr2=0x5586cb9391f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:59:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.015+0000 7fa36a6cab80 1 --2- 172.21.15.39:0/3960101122 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5586cb9332d0 0x5586cb9391f0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fa3500095f0 tx=0x7fa350008d10 comp rx=0 tx=0).stop 2023-07-21T18:59:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.015+0000 7fa36a6cab80 1 -- 172.21.15.39:0/3960101122 shutdown_connections 2023-07-21T18:59:32.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.015+0000 7fa36a6cab80 1 --2- 172.21.15.39:0/3960101122 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5586cb9332d0 0x5586cb9391f0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:32.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.015+0000 7fa36a6cab80 1 --2- 172.21.15.39:0/3960101122 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5586cb7896b0 0x5586cb9321d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:32.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.015+0000 7fa36a6cab80 1 --2- 172.21.15.39:0/3960101122 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586cb92d580 0x5586cb927c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:32.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.015+0000 7fa36a6cab80 1 -- 172.21.15.39:0/3960101122 >> 172.21.15.39:0/3960101122 conn(0x5586cb922be0 msgr2=0x5586cb931ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:59:32.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.015+0000 7fa36a6cab80 1 -- 172.21.15.39:0/3960101122 shutdown_connections 2023-07-21T18:59:32.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.015+0000 7fa36a6cab80 1 -- 172.21.15.39:0/3960101122 wait complete. 2023-07-21T18:59:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.016+0000 7fa36a6cab80 1 -- start start 2023-07-21T18:59:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.016+0000 7fa36a6cab80 1 Processor -- start 2023-07-21T18:59:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.016+0000 7fa36a6cab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5586cb7896b0 0x5586cb93be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.016+0000 7fa36a6cab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586cb92d580 0x5586cb93c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.016+0000 7fa36a6cab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5586cb9332d0 0x5586cb993b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.016+0000 7fa36a6cab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586cb9940d0 con 0x5586cb7896b0 2023-07-21T18:59:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.016+0000 7fa36a6cab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586cb994240 con 0x5586cb92d580 2023-07-21T18:59:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.016+0000 7fa36a6cab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586cb994400 con 0x5586cb9332d0 2023-07-21T18:59:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.016+0000 7fa36a6cab80 10 client.0 _ll_register_callbacks cb 0x5586cb81c700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:59:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.016+0000 7fa357fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5586cb7896b0 0x5586cb93be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.016+0000 7fa357fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5586cb7896b0 0x5586cb93be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33878/0 (socket says 172.21.15.39:33878) 2023-07-21T18:59:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.016+0000 7fa357fff700 1 -- 172.21.15.39:0/1985317140 learned_addr learned my addr 172.21.15.39:0/1985317140 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:59:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.016+0000 7fa35cae1700 1 --2- 172.21.15.39:0/1985317140 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5586cb9332d0 0x5586cb993b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.017+0000 7fa3577fe700 1 --2- 172.21.15.39:0/1985317140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586cb92d580 0x5586cb93c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.017+0000 7fa35cae1700 1 -- 172.21.15.39:0/1985317140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586cb92d580 msgr2=0x5586cb93c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:59:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.017+0000 7fa35cae1700 1 --2- 172.21.15.39:0/1985317140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586cb92d580 0x5586cb93c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.017+0000 7fa35cae1700 1 -- 172.21.15.39:0/1985317140 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5586cb7896b0 msgr2=0x5586cb93be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:59:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.017+0000 7fa35cae1700 1 --2- 172.21.15.39:0/1985317140 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5586cb7896b0 0x5586cb93be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.017+0000 7fa35cae1700 1 -- 172.21.15.39:0/1985317140 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3581148e0 con 0x5586cb9332d0 2023-07-21T18:59:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.017+0000 7fa35cae1700 1 --2- 172.21.15.39:0/1985317140 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5586cb9332d0 0x5586cb993b90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa35000b0d0 tx=0x7fa350007930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63852a5e692e077f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:59:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.017+0000 7fa3377fe700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2023-07-21T18:59:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.017+0000 7fa3377fe700 1 -- 172.21.15.39:0/1985317140 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35000e040 con 0x5586cb9332d0 2023-07-21T18:59:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.017+0000 7fa36a6cab80 1 -- 172.21.15.39:0/1985317140 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa34c0a1c40 con 0x5586cb9332d0 2023-07-21T18:59:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.017+0000 7fa3377fe700 1 -- 172.21.15.39:0/1985317140 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fa350003e10 con 0x5586cb9332d0 2023-07-21T18:59:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.018+0000 7fa3377fe700 1 -- 172.21.15.39:0/1985317140 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa350006160 con 0x5586cb9332d0 2023-07-21T18:59:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.018+0000 7fa36a6cab80 10 client.26947 fetch_fsmap learned FSMap version 93 2023-07-21T18:59:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.018+0000 7fa36a6cab80 1 -- 172.21.15.39:0/1985317140 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=93,monmap=4+}) v3 -- 0x5586cb6d31f0 con 0x5586cb9332d0 2023-07-21T18:59:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.018+0000 7fa3377fe700 1 -- 172.21.15.39:0/1985317140 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa35003a080 con 0x5586cb9332d0 2023-07-21T18:59:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.018+0000 7fa3377fe700 1 -- 172.21.15.39:0/1985317140 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 93) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fa350034030 con 0x5586cb9332d0 2023-07-21T18:59:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.018+0000 7fa36a6cab80 10 client.26947 fetch_fsmap finished waiting for FSMap version 93 2023-07-21T18:59:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.018+0000 7fa36a6cab80 10 client.26947 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:59:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.018+0000 7fa36a6cab80 20 client.26947 populate_metadata read hostname 'smithi039' 2023-07-21T18:59:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.018+0000 7fa36a6cab80 1 --2- 172.21.15.39:0/1985317140 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5586cb99b840 0x5586cb99dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.018+0000 7fa36a6cab80 4 client.26947 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:59:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.018+0000 7fa36a6cab80 1 -- 172.21.15.39:0/1985317140 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5586cb839ff0 con 0x5586cb99b840 2023-07-21T18:59:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.019+0000 7fa357fff700 1 --2- 172.21.15.39:0/1985317140 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5586cb99b840 0x5586cb99dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.020+0000 7fa357fff700 1 --2- 172.21.15.39:0/1985317140 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5586cb99b840 0x5586cb99dc50 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bee8f8ca66aa5b0d server_cookie=7be3a8bbcf28c1ab in_seq=0 out_seq=0 2023-07-21T18:59:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.020+0000 7fa3377fe700 10 client.26947 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:59:32.246 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.236+0000 7fa3377fe700 1 -- 172.21.15.39:0/1985317140 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5586cb839ff0 con 0x5586cb99b840 2023-07-21T18:59:32.247 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.236+0000 7fa3377fe700 10 client.26947 handle_command_reply: tid=0 2023-07-21T18:59:32.247 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.236+0000 7fa36a6cab80 1 client.26947 shutdown 2023-07-21T18:59:32.247 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.236+0000 7fa36a6cab80 1 -- 172.21.15.39:0/1985317140 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5586cb9332d0 msgr2=0x5586cb993b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:59:32.247 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.236+0000 7fa36a6cab80 1 --2- 172.21.15.39:0/1985317140 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5586cb9332d0 0x5586cb993b90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa35000b0d0 tx=0x7fa350007930 comp rx=0 tx=0).stop 2023-07-21T18:59:32.247 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.236+0000 7fa36a6cab80 1 -- 172.21.15.39:0/1985317140 shutdown_connections 2023-07-21T18:59:32.248 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.236+0000 7fa36a6cab80 1 --2- 172.21.15.39:0/1985317140 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5586cb99b840 0x5586cb99dc50 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:32.248 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.236+0000 7fa36a6cab80 1 --2- 172.21.15.39:0/1985317140 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5586cb9332d0 0x5586cb993b90 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:32.248 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.236+0000 7fa36a6cab80 1 --2- 172.21.15.39:0/1985317140 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5586cb7896b0 0x5586cb93be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:32.248 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.236+0000 7fa36a6cab80 1 --2- 172.21.15.39:0/1985317140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5586cb92d580 0x5586cb93c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:32.248 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.236+0000 7fa36a6cab80 1 -- 172.21.15.39:0/1985317140 >> 172.21.15.39:0/1985317140 conn(0x5586cb922be0 msgr2=0x5586cb933900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:59:32.248 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.236+0000 7fa3377fe700 0 client.26947 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:59:32.248 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.236+0000 7fa36a6cab80 1 -- 172.21.15.39:0/1985317140 shutdown_connections 2023-07-21T18:59:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.236+0000 7fa36a6cab80 1 -- 172.21.15.39:0/1985317140 wait complete. 2023-07-21T18:59:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.236+0000 7fa36a6cab80 20 client.26947 trim_cache size 0 max 16384 2023-07-21T18:59:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:59:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:59:32.250 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:59:32.250 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:59:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.237+0000 7fa35e544700 1 -- 172.21.15.39:0/3622448011 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fa3400a34d0 msgr2=0x7fa3400a5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:59:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.237+0000 7fa35e544700 1 --2- 172.21.15.39:0/3622448011 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fa3400a34d0 0x7fa3400a5980 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa348009c40 tx=0x7fa348009260 comp rx=0 tx=0).stop 2023-07-21T18:59:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.237+0000 7fa35e544700 1 -- 172.21.15.39:0/3622448011 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa35806d520 msgr2=0x7fa35819c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:59:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.237+0000 7fa35e544700 1 --2- 172.21.15.39:0/3622448011 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa35806d520 0x7fa35819c790 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fa34c00b4d0 tx=0x7fa34c00b7e0 comp rx=0 tx=0).stop 2023-07-21T18:59:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.237+0000 7fa35e544700 1 -- 172.21.15.39:0/3622448011 shutdown_connections 2023-07-21T18:59:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.237+0000 7fa35e544700 1 --2- 172.21.15.39:0/3622448011 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fa3400a34d0 0x7fa3400a5980 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.237+0000 7fa35e544700 1 --2- 172.21.15.39:0/3622448011 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa35810c4d0 0x7fa3581a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.237+0000 7fa35e544700 1 --2- 172.21.15.39:0/3622448011 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa35806cb80 0x7fa35819c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.237+0000 7fa35e544700 1 --2- 172.21.15.39:0/3622448011 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa35806d520 0x7fa35819c790 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.237+0000 7fa35e544700 1 -- 172.21.15.39:0/3622448011 >> 172.21.15.39:0/3622448011 conn(0x7fa35806bd70 msgr2=0x7fa35810add0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:59:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.237+0000 7fa35e544700 1 -- 172.21.15.39:0/3622448011 shutdown_connections 2023-07-21T18:59:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:32.237+0000 7fa35e544700 1 -- 172.21.15.39:0/3622448011 wait complete. 2023-07-21T18:59:32.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:59:32 smithi193 ceph-mon[98204]: pgmap v562: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 9.0 MiB/s wr, 614 op/s 2023-07-21T18:59:32.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:59:32 smithi155 ceph-mon[97934]: pgmap v562: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 9.0 MiB/s wr, 614 op/s 2023-07-21T18:59:32.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:59:32 smithi039 ceph-mon[96489]: pgmap v562: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 9.0 MiB/s wr, 614 op/s 2023-07-21T18:59:32.683 INFO:tasks.workunit.client.0.smithi039.stdout: 10 162802 31.20 MB/sec execute 165 sec latency 163.518 ms 2023-07-21T18:59:33.683 INFO:tasks.workunit.client.0.smithi039.stdout: 10 163546 31.26 MB/sec execute 166 sec latency 90.099 ms 2023-07-21T18:59:34.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:59:34 smithi193 ceph-mon[98204]: pgmap v563: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 12 MiB/s wr, 778 op/s 2023-07-21T18:59:34.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:59:34 smithi155 ceph-mon[97934]: pgmap v563: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 12 MiB/s wr, 778 op/s 2023-07-21T18:59:34.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:59:34 smithi039 ceph-mon[96489]: pgmap v563: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 12 MiB/s wr, 778 op/s 2023-07-21T18:59:34.683 INFO:tasks.workunit.client.0.smithi039.stdout: 10 164300 31.34 MB/sec execute 167 sec latency 69.072 ms 2023-07-21T18:59:35.607 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:59:35 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:59:35.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:59:35 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:59:35.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:59:35 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:59:35.683 INFO:tasks.workunit.client.0.smithi039.stdout: 10 164727 31.28 MB/sec execute 168 sec latency 145.331 ms 2023-07-21T18:59:36.607 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:59:36 smithi193 ceph-mon[98204]: pgmap v564: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 7.0 MiB/s wr, 450 op/s 2023-07-21T18:59:36.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:59:36 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:59:36.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:59:36 smithi155 ceph-mon[97934]: pgmap v564: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 7.0 MiB/s wr, 450 op/s 2023-07-21T18:59:36.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:59:36 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:59:36.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:59:36 smithi039 ceph-mon[96489]: pgmap v564: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 7.0 MiB/s wr, 450 op/s 2023-07-21T18:59:36.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:59:36 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:59:36.683 INFO:tasks.workunit.client.0.smithi039.stdout: 10 165392 31.35 MB/sec execute 169 sec latency 95.489 ms 2023-07-21T18:59:37.683 INFO:tasks.workunit.client.0.smithi039.stdout: 10 165987 31.31 MB/sec execute 170 sec latency 77.634 ms 2023-07-21T18:59:38.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:59:38 smithi193 ceph-mon[98204]: pgmap v565: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 10 MiB/s wr, 725 op/s 2023-07-21T18:59:38.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:59:38 smithi155 ceph-mon[97934]: pgmap v565: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 10 MiB/s wr, 725 op/s 2023-07-21T18:59:38.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:59:38 smithi039 ceph-mon[96489]: pgmap v565: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 10 MiB/s wr, 725 op/s 2023-07-21T18:59:38.683 INFO:tasks.workunit.client.0.smithi039.stdout: 10 166646 31.34 MB/sec execute 171 sec latency 72.383 ms 2023-07-21T18:59:39.683 INFO:tasks.workunit.client.0.smithi039.stdout: 10 167338 31.40 MB/sec execute 172 sec latency 58.825 ms 2023-07-21T18:59:40.607 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:59:40 smithi193 ceph-mon[98204]: pgmap v566: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 7.0 MiB/s wr, 499 op/s 2023-07-21T18:59:40.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:59:40 smithi155 ceph-mon[97934]: pgmap v566: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 7.0 MiB/s wr, 499 op/s 2023-07-21T18:59:40.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:59:40 smithi039 ceph-mon[96489]: pgmap v566: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 7.0 MiB/s wr, 499 op/s 2023-07-21T18:59:40.683 INFO:tasks.workunit.client.0.smithi039.stdout: 10 168049 31.41 MB/sec execute 173 sec latency 60.332 ms 2023-07-21T18:59:41.683 INFO:tasks.workunit.client.0.smithi039.stdout: 10 168732 31.48 MB/sec execute 174 sec latency 81.516 ms 2023-07-21T18:59:42.607 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:59:42 smithi193 ceph-mon[98204]: pgmap v567: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 7.1 MiB/s wr, 505 op/s 2023-07-21T18:59:42.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:59:42 smithi155 ceph-mon[97934]: pgmap v567: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 7.1 MiB/s wr, 505 op/s 2023-07-21T18:59:42.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:59:42 smithi039 ceph-mon[96489]: pgmap v567: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 7.1 MiB/s wr, 505 op/s 2023-07-21T18:59:42.683 INFO:tasks.workunit.client.0.smithi039.stdout: 10 169330 31.46 MB/sec execute 175 sec latency 73.094 ms 2023-07-21T18:59:43.684 INFO:tasks.workunit.client.0.smithi039.stdout: 10 169933 31.46 MB/sec execute 176 sec latency 84.930 ms 2023-07-21T18:59:44.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:59:44 smithi193 ceph-mon[98204]: pgmap v568: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 9.7 MiB/s wr, 694 op/s 2023-07-21T18:59:44.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:59:44 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:59:44.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:59:44 smithi155 ceph-mon[97934]: pgmap v568: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 9.7 MiB/s wr, 694 op/s 2023-07-21T18:59:44.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:59:44 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:59:44.678 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:59:44 smithi039 ceph-mon[96489]: pgmap v568: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 9.7 MiB/s wr, 694 op/s 2023-07-21T18:59:44.679 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:59:44 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T18:59:44.683 INFO:tasks.workunit.client.0.smithi039.stdout: 10 170618 31.52 MB/sec execute 177 sec latency 63.075 ms 2023-07-21T18:59:45.401 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.391+0000 7f4e3cc62700 1 -- 172.21.15.39:0/2323395866 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e380fea30 msgr2=0x7f4e380feea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:59:45.402 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.391+0000 7f4e3cc62700 1 --2- 172.21.15.39:0/2323395866 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e380fea30 0x7f4e380feea0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c009990 tx=0x7f4e2c009ca0 comp rx=0 tx=0).stop 2023-07-21T18:59:45.402 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.392+0000 7f4e3cc62700 1 -- 172.21.15.39:0/2323395866 shutdown_connections 2023-07-21T18:59:45.402 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.392+0000 7f4e3cc62700 1 --2- 172.21.15.39:0/2323395866 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e38104a30 0x7f4e38104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:45.402 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.392+0000 7f4e3cc62700 1 --2- 172.21.15.39:0/2323395866 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e380ff3e0 0x7f4e3810b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:45.402 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.392+0000 7f4e3cc62700 1 --2- 172.21.15.39:0/2323395866 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e380fea30 0x7f4e380feea0 secure :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c009990 tx=0x7f4e2c009ca0 comp rx=0 tx=0).stop 2023-07-21T18:59:45.403 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.392+0000 7f4e3cc62700 1 -- 172.21.15.39:0/2323395866 >> 172.21.15.39:0/2323395866 conn(0x7f4e38074760 msgr2=0x7f4e38074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:59:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.397+0000 7f4e3cc62700 1 -- 172.21.15.39:0/2323395866 shutdown_connections 2023-07-21T18:59:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.397+0000 7f4e3cc62700 1 -- 172.21.15.39:0/2323395866 wait complete. 2023-07-21T18:59:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.398+0000 7f4e3cc62700 1 Processor -- start 2023-07-21T18:59:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.398+0000 7f4e3cc62700 1 -- start start 2023-07-21T18:59:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.398+0000 7f4e3cc62700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e380ff3e0 0x7f4e38195e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.398+0000 7f4e3cc62700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e38104a30 0x7f4e381963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.398+0000 7f4e3cc62700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e3819aa10 0x7f4e3819ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.398+0000 7f4e3cc62700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e3819b480 con 0x7f4e380ff3e0 2023-07-21T18:59:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.398+0000 7f4e3cc62700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e3819b5f0 con 0x7f4e3819aa10 2023-07-21T18:59:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.398+0000 7f4e3cc62700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e3819b7b0 con 0x7f4e38104a30 2023-07-21T18:59:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.398+0000 7f4e3659c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e380ff3e0 0x7f4e38195e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.398+0000 7f4e3659c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e380ff3e0 0x7f4e38195e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33892/0 (socket says 172.21.15.39:33892) 2023-07-21T18:59:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.398+0000 7f4e3659c700 1 -- 172.21.15.39:0/771988223 learned_addr learned my addr 172.21.15.39:0/771988223 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:59:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.398+0000 7f4e35d9b700 1 --2- 172.21.15.39:0/771988223 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e38104a30 0x7f4e381963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:45.414 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.398+0000 7f4e36d9d700 1 --2- 172.21.15.39:0/771988223 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e3819aa10 0x7f4e3819ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:45.414 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.399+0000 7f4e3659c700 1 -- 172.21.15.39:0/771988223 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e38104a30 msgr2=0x7f4e381963a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:59:45.414 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.399+0000 7f4e3659c700 1 --2- 172.21.15.39:0/771988223 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e38104a30 0x7f4e381963a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:45.414 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.399+0000 7f4e3659c700 1 -- 172.21.15.39:0/771988223 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e3819aa10 msgr2=0x7f4e3819ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:59:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.399+0000 7f4e3659c700 1 --2- 172.21.15.39:0/771988223 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e3819aa10 0x7f4e3819ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.399+0000 7f4e3659c700 1 -- 172.21.15.39:0/771988223 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e2c0095f0 con 0x7f4e380ff3e0 2023-07-21T18:59:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.399+0000 7f4e35d9b700 1 --2- 172.21.15.39:0/771988223 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e38104a30 0x7f4e381963a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:59:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.399+0000 7f4e3659c700 1 --2- 172.21.15.39:0/771988223 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e380ff3e0 0x7f4e38195e60 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f4e2800e970 tx=0x7f4e2800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:59:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.399+0000 7f4e277fe700 1 -- 172.21.15.39:0/771988223 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e28009790 con 0x7f4e380ff3e0 2023-07-21T18:59:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.399+0000 7f4e3cc62700 1 -- 172.21.15.39:0/771988223 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e381a2d20 con 0x7f4e380ff3e0 2023-07-21T18:59:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.399+0000 7f4e277fe700 1 -- 172.21.15.39:0/771988223 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e2800eec0 con 0x7f4e380ff3e0 2023-07-21T18:59:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.399+0000 7f4e277fe700 1 -- 172.21.15.39:0/771988223 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e28003cf0 con 0x7f4e380ff3e0 2023-07-21T18:59:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.399+0000 7f4e3cc62700 1 -- 172.21.15.39:0/771988223 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e381a31b0 con 0x7f4e380ff3e0 2023-07-21T18:59:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.402+0000 7f4e277fe700 1 -- 172.21.15.39:0/771988223 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f4e2800ccd0 con 0x7f4e380ff3e0 2023-07-21T18:59:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.403+0000 7f4e277fe700 1 --2- 172.21.15.39:0/771988223 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f4e200a12c0 0x7f4e200a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.403+0000 7f4e48de8b80 1 Processor -- start 2023-07-21T18:59:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.403+0000 7f4e277fe700 1 -- 172.21.15.39:0/771988223 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f4e28014070 con 0x7f4e380ff3e0 2023-07-21T18:59:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.403+0000 7f4e48de8b80 1 -- start start 2023-07-21T18:59:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.403+0000 7f4e35d9b700 1 --2- 172.21.15.39:0/771988223 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f4e200a12c0 0x7f4e200a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.404+0000 7f4e48de8b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a8588f66b0 0x55a858a9fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.404+0000 7f4e48de8b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a858aa02d0 0x55a858a9a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.404+0000 7f4e48de8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a858a9a580 0x55a858a9a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.404+0000 7f4e48de8b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a85899fc50 con 0x55a8588f66b0 2023-07-21T18:59:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.404+0000 7f4e48de8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8589f29e0 con 0x55a858a9a580 2023-07-21T18:59:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.404+0000 7f4e48de8b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8589ebe70 con 0x55a858aa02d0 2023-07-21T18:59:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.404+0000 7f4e35d9b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a858aa02d0 0x55a858a9a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.404+0000 7f4e35d9b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a858aa02d0 0x55a858a9a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:52454/0 (socket says 172.21.15.39:52454) 2023-07-21T18:59:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.404+0000 7f4e35d9b700 1 -- 172.21.15.39:0/842994101 learned_addr learned my addr 172.21.15.39:0/842994101 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:59:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.404+0000 7f4e36d9d700 1 --2- 172.21.15.39:0/842994101 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a858a9a580 0x55a858a9a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.404+0000 7f4e35d9b700 1 --2- 172.21.15.39:0/771988223 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f4e200a12c0 0x7f4e200a3770 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c000c00 tx=0x7f4e2c005fd0 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:59:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.404+0000 7f4e35d9b700 1 -- 172.21.15.39:0/842994101 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a858a9a580 msgr2=0x55a858a9a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:59:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.404+0000 7f4e35d9b700 1 --2- 172.21.15.39:0/842994101 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a858a9a580 0x55a858a9a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.404+0000 7f4e35d9b700 1 -- 172.21.15.39:0/842994101 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a8588f66b0 msgr2=0x55a858a9fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:59:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.404+0000 7f4e35d9b700 1 --2- 172.21.15.39:0/842994101 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a8588f66b0 0x55a858a9fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.404+0000 7f4e35d9b700 1 -- 172.21.15.39:0/842994101 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a858819d80 con 0x55a858aa02d0 2023-07-21T18:59:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.404+0000 7f4e35d9b700 1 --2- 172.21.15.39:0/842994101 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a858aa02d0 0x55a858a9a040 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4e380ffb20 tx=0x7f4e2c024000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c7a64a9fce5cea8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:59:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.405+0000 7f4e24ff9700 1 -- 172.21.15.39:0/842994101 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e2c024070 con 0x55a858aa02d0 2023-07-21T18:59:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.405+0000 7f4e24ff9700 1 -- 172.21.15.39:0/842994101 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e2c004800 con 0x55a858aa02d0 2023-07-21T18:59:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.405+0000 7f4e24ff9700 1 -- 172.21.15.39:0/842994101 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e2c005220 con 0x55a858aa02d0 2023-07-21T18:59:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.406+0000 7f4e48de8b80 1 -- 172.21.15.39:0/842994101 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a858aa02d0 msgr2=0x55a858a9a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:59:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.406+0000 7f4e48de8b80 1 --2- 172.21.15.39:0/842994101 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a858aa02d0 0x55a858a9a040 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4e380ffb20 tx=0x7f4e2c024000 comp rx=0 tx=0).stop 2023-07-21T18:59:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.406+0000 7f4e48de8b80 1 -- 172.21.15.39:0/842994101 shutdown_connections 2023-07-21T18:59:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.406+0000 7f4e48de8b80 1 --2- 172.21.15.39:0/842994101 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a858a9a580 0x55a858a9a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.406+0000 7f4e48de8b80 1 --2- 172.21.15.39:0/842994101 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a8588f66b0 0x55a858a9fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.406+0000 7f4e48de8b80 1 --2- 172.21.15.39:0/842994101 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a858aa02d0 0x55a858a9a040 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.406+0000 7f4e48de8b80 1 -- 172.21.15.39:0/842994101 >> 172.21.15.39:0/842994101 conn(0x55a858a8fbe0 msgr2=0x55a858a94f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:59:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.406+0000 7f4e48de8b80 1 -- 172.21.15.39:0/842994101 shutdown_connections 2023-07-21T18:59:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.406+0000 7f4e48de8b80 1 -- 172.21.15.39:0/842994101 wait complete. 2023-07-21T18:59:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.406+0000 7f4e48de8b80 1 -- start start 2023-07-21T18:59:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.407+0000 7f4e48de8b80 1 Processor -- start 2023-07-21T18:59:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.407+0000 7f4e48de8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a8588f66b0 0x55a858aa3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.407+0000 7f4e48de8b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a858a9a580 0x55a858aa3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.407+0000 7f4e48de8b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a858aa02d0 0x55a858aa43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.407+0000 7f4e48de8b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a85898ba60 con 0x55a858a9a580 2023-07-21T18:59:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.407+0000 7f4e48de8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a85898b760 con 0x55a8588f66b0 2023-07-21T18:59:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.407+0000 7f4e48de8b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a85886f8c0 con 0x55a858aa02d0 2023-07-21T18:59:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.407+0000 7f4e48de8b80 10 client.0 _ll_register_callbacks cb 0x55a858989700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:59:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.407+0000 7f4e35d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a858a9a580 0x55a858aa3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.407+0000 7f4e35d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a858a9a580 0x55a858aa3ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33908/0 (socket says 172.21.15.39:33908) 2023-07-21T18:59:45.427 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.407+0000 7f4e3659c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a8588f66b0 0x55a858aa3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:45.427 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.407+0000 7f4e35d9b700 1 -- 172.21.15.39:0/11553736 learned_addr learned my addr 172.21.15.39:0/11553736 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:59:45.427 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.407+0000 7f4e36d9d700 1 --2- 172.21.15.39:0/11553736 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a858aa02d0 0x55a858aa43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:45.427 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.407+0000 7f4e35d9b700 1 -- 172.21.15.39:0/11553736 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a858aa02d0 msgr2=0x55a858aa43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:59:45.427 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.407+0000 7f4e35d9b700 1 --2- 172.21.15.39:0/11553736 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a858aa02d0 0x55a858aa43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.407+0000 7f4e35d9b700 1 -- 172.21.15.39:0/11553736 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a8588f66b0 msgr2=0x55a858aa3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:59:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.407+0000 7f4e35d9b700 1 --2- 172.21.15.39:0/11553736 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a8588f66b0 0x55a858aa3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.407+0000 7f4e35d9b700 1 -- 172.21.15.39:0/11553736 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e3810c160 con 0x55a858a9a580 2023-07-21T18:59:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.408+0000 7f4e35d9b700 1 --2- 172.21.15.39:0/11553736 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a858a9a580 0x55a858aa3ea0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c009350 tx=0x7f4e2c01fc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b2520ec3896a27d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:59:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.408+0000 7f4e1dffb700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T18:59:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.408+0000 7f4e1dffb700 1 -- 172.21.15.39:0/11553736 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e2c036870 con 0x55a858a9a580 2023-07-21T18:59:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.408+0000 7f4e48de8b80 1 -- 172.21.15.39:0/11553736 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a858945490 con 0x55a858a9a580 2023-07-21T18:59:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.408+0000 7f4e1dffb700 1 -- 172.21.15.39:0/11553736 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f4e2c0369d0 con 0x55a858a9a580 2023-07-21T18:59:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.408+0000 7f4e1dffb700 1 -- 172.21.15.39:0/11553736 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e2c036b90 con 0x55a858a9a580 2023-07-21T18:59:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.408+0000 7f4e48de8b80 10 client.17505 fetch_fsmap learned FSMap version 96 2023-07-21T18:59:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.408+0000 7f4e48de8b80 1 -- 172.21.15.39:0/11553736 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=4+}) v3 -- 0x55a858802c00 con 0x55a858a9a580 2023-07-21T18:59:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.408+0000 7f4e1dffb700 1 -- 172.21.15.39:0/11553736 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a858945490 con 0x55a858a9a580 2023-07-21T18:59:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.409+0000 7f4e1dffb700 1 -- 172.21.15.39:0/11553736 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 96) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4e2c02c420 con 0x55a858a9a580 2023-07-21T18:59:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.409+0000 7f4e48de8b80 10 client.17505 fetch_fsmap finished waiting for FSMap version 96 2023-07-21T18:59:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.409+0000 7f4e48de8b80 10 client.17505 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:59:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.409+0000 7f4e48de8b80 20 client.17505 populate_metadata read hostname 'smithi039' 2023-07-21T18:59:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.409+0000 7f4e48de8b80 1 --2- 172.21.15.39:0/11553736 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55a858ab3790 0x55a858ab5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.409+0000 7f4e48de8b80 4 client.17505 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:59:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.409+0000 7f4e48de8b80 1 -- 172.21.15.39:0/11553736 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a8589e4e50 con 0x55a858ab3790 2023-07-21T18:59:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.409+0000 7f4e3659c700 1 --2- 172.21.15.39:0/11553736 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55a858ab3790 0x55a858ab5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.410+0000 7f4e1dffb700 10 client.17505 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:59:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.410+0000 7f4e3659c700 1 --2- 172.21.15.39:0/11553736 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55a858ab3790 0x55a858ab5ba0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aae05ac8b35193ec server_cookie=cfb36e9fdb7d3fca in_seq=0 out_seq=0 2023-07-21T18:59:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.411+0000 7f4e1dffb700 1 -- 172.21.15.39:0/11553736 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a8589e4e50 con 0x55a858ab3790 2023-07-21T18:59:45.432 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.411+0000 7f4e1dffb700 10 client.17505 handle_command_reply: tid=0 2023-07-21T18:59:45.432 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.411+0000 7f4e48de8b80 1 client.17505 shutdown 2023-07-21T18:59:45.432 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.411+0000 7f4e48de8b80 1 -- 172.21.15.39:0/11553736 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a858a9a580 msgr2=0x55a858aa3ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:59:45.432 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.411+0000 7f4e48de8b80 1 --2- 172.21.15.39:0/11553736 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a858a9a580 0x55a858aa3ea0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c009350 tx=0x7f4e2c01fc50 comp rx=0 tx=0).stop 2023-07-21T18:59:45.432 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.411+0000 7f4e48de8b80 1 -- 172.21.15.39:0/11553736 shutdown_connections 2023-07-21T18:59:45.433 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.411+0000 7f4e48de8b80 1 --2- 172.21.15.39:0/11553736 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55a858ab3790 0x55a858ab5ba0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:45.433 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.411+0000 7f4e48de8b80 1 --2- 172.21.15.39:0/11553736 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a8588f66b0 0x55a858aa3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:45.433 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.411+0000 7f4e48de8b80 1 --2- 172.21.15.39:0/11553736 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a858aa02d0 0x55a858aa43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:45.433 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.411+0000 7f4e48de8b80 1 --2- 172.21.15.39:0/11553736 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a858a9a580 0x55a858aa3ea0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:45.433 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.411+0000 7f4e48de8b80 1 -- 172.21.15.39:0/11553736 >> 172.21.15.39:0/11553736 conn(0x55a858a8fbe0 msgr2=0x55a858a93c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:59:45.433 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.411+0000 7f4e1dffb700 0 client.17505 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:59:45.433 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.411+0000 7f4e48de8b80 1 -- 172.21.15.39:0/11553736 shutdown_connections 2023-07-21T18:59:45.433 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.411+0000 7f4e48de8b80 1 -- 172.21.15.39:0/11553736 wait complete. 2023-07-21T18:59:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.412+0000 7f4e48de8b80 20 client.17505 trim_cache size 0 max 16384 2023-07-21T18:59:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.432+0000 7f4e48de8b80 1 Processor -- start 2023-07-21T18:59:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.433+0000 7f4e48de8b80 1 -- start start 2023-07-21T18:59:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.433+0000 7f4e48de8b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a8588f66b0 0x55a858a9a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.433+0000 7f4e48de8b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a858a9a580 0x55a858aaac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.433+0000 7f4e48de8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a858aa02d0 0x55a858aa4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.433+0000 7f4e48de8b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a858aa6980 con 0x55a8588f66b0 2023-07-21T18:59:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.433+0000 7f4e48de8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a858aa06a0 con 0x55a858aa02d0 2023-07-21T18:59:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.433+0000 7f4e48de8b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a858a8cbb0 con 0x55a858a9a580 2023-07-21T18:59:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.433+0000 7f4e36d9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a858aa02d0 0x55a858aa4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.433+0000 7f4e36d9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a858aa02d0 0x55a858aa4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:46510/0 (socket says 172.21.15.39:46510) 2023-07-21T18:59:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.433+0000 7f4e36d9d700 1 -- 172.21.15.39:0/1936226482 learned_addr learned my addr 172.21.15.39:0/1936226482 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:59:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.433+0000 7f4e35d9b700 1 --2- 172.21.15.39:0/1936226482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a858a9a580 0x55a858aaac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.434+0000 7f4e36d9d700 1 -- 172.21.15.39:0/1936226482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a858a9a580 msgr2=0x55a858aaac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:59:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.434+0000 7f4e36d9d700 1 --2- 172.21.15.39:0/1936226482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a858a9a580 0x55a858aaac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.434+0000 7f4e36d9d700 1 -- 172.21.15.39:0/1936226482 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a8588f66b0 msgr2=0x55a858a9a040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T18:59:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.434+0000 7f4e36d9d700 1 --2- 172.21.15.39:0/1936226482 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a8588f66b0 0x55a858a9a040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.434+0000 7f4e36d9d700 1 -- 172.21.15.39:0/1936226482 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8588401f0 con 0x55a858aa02d0 2023-07-21T18:59:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.434+0000 7f4e35d9b700 1 --2- 172.21.15.39:0/1936226482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a858a9a580 0x55a858aaac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:59:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.434+0000 7f4e36d9d700 1 --2- 172.21.15.39:0/1936226482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a858aa02d0 0x55a858aa4260 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4e30008610 tx=0x7f4e3000b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34f1ac61f912b3ef server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:59:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.434+0000 7f4e257fa700 1 -- 172.21.15.39:0/1936226482 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3000e040 con 0x55a858aa02d0 2023-07-21T18:59:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.434+0000 7f4e257fa700 1 -- 172.21.15.39:0/1936226482 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e3000f040 con 0x55a858aa02d0 2023-07-21T18:59:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.434+0000 7f4e257fa700 1 -- 172.21.15.39:0/1936226482 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3001f850 con 0x55a858aa02d0 2023-07-21T18:59:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.435+0000 7f4e48de8b80 1 -- 172.21.15.39:0/1936226482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a858aa02d0 msgr2=0x55a858aa4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:59:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.435+0000 7f4e48de8b80 1 --2- 172.21.15.39:0/1936226482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a858aa02d0 0x55a858aa4260 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4e30008610 tx=0x7f4e3000b040 comp rx=0 tx=0).stop 2023-07-21T18:59:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.435+0000 7f4e48de8b80 1 -- 172.21.15.39:0/1936226482 shutdown_connections 2023-07-21T18:59:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.435+0000 7f4e48de8b80 1 --2- 172.21.15.39:0/1936226482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a858aa02d0 0x55a858aa4260 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.435+0000 7f4e48de8b80 1 --2- 172.21.15.39:0/1936226482 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a8588f66b0 0x55a858a9a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.435+0000 7f4e48de8b80 1 --2- 172.21.15.39:0/1936226482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a858a9a580 0x55a858aaac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.435+0000 7f4e48de8b80 1 -- 172.21.15.39:0/1936226482 >> 172.21.15.39:0/1936226482 conn(0x55a858a8fbe0 msgr2=0x55a858a9eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:59:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.435+0000 7f4e48de8b80 1 -- 172.21.15.39:0/1936226482 shutdown_connections 2023-07-21T18:59:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.435+0000 7f4e48de8b80 1 -- 172.21.15.39:0/1936226482 wait complete. 2023-07-21T18:59:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.436+0000 7f4e48de8b80 1 -- start start 2023-07-21T18:59:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.436+0000 7f4e48de8b80 1 Processor -- start 2023-07-21T18:59:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.436+0000 7f4e48de8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a8588f66b0 0x55a858aa8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.436+0000 7f4e48de8b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a858a9a580 0x55a858aa8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.436+0000 7f4e48de8b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a858aa02d0 0x55a858b06610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.436+0000 7f4e48de8b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a858aa9330 con 0x55a858a9a580 2023-07-21T18:59:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.436+0000 7f4e48de8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a858aa94a0 con 0x55a8588f66b0 2023-07-21T18:59:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.436+0000 7f4e48de8b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a858b06b80 con 0x55a858aa02d0 2023-07-21T18:59:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.436+0000 7f4e48de8b80 10 client.0 _ll_register_callbacks cb 0x7f4e30009d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:59:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.436+0000 7f4e35d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a858a9a580 0x55a858aa8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.436+0000 7f4e3659c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a8588f66b0 0x55a858aa8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.436+0000 7f4e36d9d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a858aa02d0 0x55a858b06610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.436+0000 7f4e35d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a858a9a580 0x55a858aa8b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33918/0 (socket says 172.21.15.39:33918) 2023-07-21T18:59:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.436+0000 7f4e35d9b700 1 -- 172.21.15.39:0/420243650 learned_addr learned my addr 172.21.15.39:0/420243650 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:59:45.451 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.436+0000 7f4e3659c700 1 --2- 172.21.15.39:0/420243650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a8588f66b0 0x55a858aa8630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:46514/0 (socket says 172.21.15.39:46514) 2023-07-21T18:59:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.437+0000 7f4e36d9d700 1 -- 172.21.15.39:0/420243650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a8588f66b0 msgr2=0x55a858aa8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:59:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.437+0000 7f4e36d9d700 1 --2- 172.21.15.39:0/420243650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a8588f66b0 0x55a858aa8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.437+0000 7f4e36d9d700 1 -- 172.21.15.39:0/420243650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a858a9a580 msgr2=0x55a858aa8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:59:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.437+0000 7f4e36d9d700 1 --2- 172.21.15.39:0/420243650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a858a9a580 0x55a858aa8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:45.453 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.437+0000 7f4e36d9d700 1 -- 172.21.15.39:0/420243650 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e300093d0 con 0x55a858aa02d0 2023-07-21T18:59:45.453 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.437+0000 7f4e35d9b700 1 --2- 172.21.15.39:0/420243650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a858a9a580 0x55a858aa8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:59:45.453 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.437+0000 7f4e3659c700 1 --2- 172.21.15.39:0/420243650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a8588f66b0 0x55a858aa8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T18:59:45.453 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.437+0000 7f4e36d9d700 1 --2- 172.21.15.39:0/420243650 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a858aa02d0 0x55a858b06610 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4e300089d0 tx=0x7f4e30017b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1384aedf89d740c7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:59:45.453 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.437+0000 7f4e1dffb700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2023-07-21T18:59:45.453 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.437+0000 7f4e1dffb700 1 -- 172.21.15.39:0/420243650 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e30023070 con 0x55a858aa02d0 2023-07-21T18:59:45.453 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.437+0000 7f4e1dffb700 1 -- 172.21.15.39:0/420243650 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f4e30004ce0 con 0x55a858aa02d0 2023-07-21T18:59:45.454 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.437+0000 7f4e48de8b80 1 -- 172.21.15.39:0/420243650 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4e2c004800 con 0x55a858aa02d0 2023-07-21T18:59:45.454 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.438+0000 7f4e1dffb700 1 -- 172.21.15.39:0/420243650 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e3000e040 con 0x55a858aa02d0 2023-07-21T18:59:45.454 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.438+0000 7f4e48de8b80 10 client.26962 fetch_fsmap learned FSMap version 96 2023-07-21T18:59:45.454 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.438+0000 7f4e48de8b80 1 -- 172.21.15.39:0/420243650 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=4+}) v3 -- 0x55a858818250 con 0x55a858aa02d0 2023-07-21T18:59:45.454 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.438+0000 7f4e1dffb700 1 -- 172.21.15.39:0/420243650 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e30034080 con 0x55a858aa02d0 2023-07-21T18:59:45.454 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.438+0000 7f4e1dffb700 1 -- 172.21.15.39:0/420243650 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 96) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4e3001d020 con 0x55a858aa02d0 2023-07-21T18:59:45.454 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.438+0000 7f4e48de8b80 10 client.26962 fetch_fsmap finished waiting for FSMap version 96 2023-07-21T18:59:45.454 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.438+0000 7f4e48de8b80 10 client.26962 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:59:45.455 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.438+0000 7f4e48de8b80 20 client.26962 populate_metadata read hostname 'smithi039' 2023-07-21T18:59:45.455 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.438+0000 7f4e48de8b80 1 --2- 172.21.15.39:0/420243650 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55a858b080a0 0x55a858b084a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:45.455 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.438+0000 7f4e48de8b80 4 client.26962 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:59:45.455 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.438+0000 7f4e48de8b80 1 -- 172.21.15.39:0/420243650 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a8589a6ff0 con 0x55a858b080a0 2023-07-21T18:59:45.455 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.438+0000 7f4e3659c700 1 --2- 172.21.15.39:0/420243650 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55a858b080a0 0x55a858b084a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:45.455 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.439+0000 7f4e3659c700 1 --2- 172.21.15.39:0/420243650 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55a858b080a0 0x55a858b084a0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1200b1fb8f49869 server_cookie=725343d827cae1f3 in_seq=0 out_seq=0 2023-07-21T18:59:45.455 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.439+0000 7f4e1dffb700 10 client.26962 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:59:45.649 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.639+0000 7f4e1dffb700 1 -- 172.21.15.39:0/420243650 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a8589a6ff0 con 0x55a858b080a0 2023-07-21T18:59:45.650 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.639+0000 7f4e1dffb700 10 client.26962 handle_command_reply: tid=0 2023-07-21T18:59:45.650 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.639+0000 7f4e48de8b80 1 client.26962 shutdown 2023-07-21T18:59:45.650 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.639+0000 7f4e48de8b80 1 -- 172.21.15.39:0/420243650 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a858aa02d0 msgr2=0x55a858b06610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:59:45.650 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.639+0000 7f4e48de8b80 1 --2- 172.21.15.39:0/420243650 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a858aa02d0 0x55a858b06610 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4e300089d0 tx=0x7f4e30017b40 comp rx=0 tx=0).stop 2023-07-21T18:59:45.650 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.639+0000 7f4e48de8b80 1 -- 172.21.15.39:0/420243650 shutdown_connections 2023-07-21T18:59:45.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.639+0000 7f4e48de8b80 1 --2- 172.21.15.39:0/420243650 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55a858b080a0 0x55a858b084a0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:45.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.639+0000 7f4e48de8b80 1 --2- 172.21.15.39:0/420243650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a8588f66b0 0x55a858aa8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:45.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.639+0000 7f4e48de8b80 1 --2- 172.21.15.39:0/420243650 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a858aa02d0 0x55a858b06610 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:45.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.639+0000 7f4e1dffb700 0 client.26962 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:59:45.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.639+0000 7f4e48de8b80 1 --2- 172.21.15.39:0/420243650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a858a9a580 0x55a858aa8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:45.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.639+0000 7f4e48de8b80 1 -- 172.21.15.39:0/420243650 >> 172.21.15.39:0/420243650 conn(0x55a858a8fbe0 msgr2=0x55a858a9f3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:59:45.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.639+0000 7f4e48de8b80 1 -- 172.21.15.39:0/420243650 shutdown_connections 2023-07-21T18:59:45.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.639+0000 7f4e48de8b80 1 -- 172.21.15.39:0/420243650 wait complete. 2023-07-21T18:59:45.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.639+0000 7f4e48de8b80 20 client.26962 trim_cache size 0 max 16384 2023-07-21T18:59:45.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:59:45.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:59:45.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:59:45.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:59:45.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.640+0000 7f4e3cc62700 1 -- 172.21.15.39:0/771988223 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f4e200a12c0 msgr2=0x7f4e200a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:59:45.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.640+0000 7f4e3cc62700 1 --2- 172.21.15.39:0/771988223 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f4e200a12c0 0x7f4e200a3770 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c000c00 tx=0x7f4e2c005fd0 comp rx=0 tx=0).stop 2023-07-21T18:59:45.654 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.640+0000 7f4e3cc62700 1 -- 172.21.15.39:0/771988223 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e380ff3e0 msgr2=0x7f4e38195e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:59:45.654 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.640+0000 7f4e3cc62700 1 --2- 172.21.15.39:0/771988223 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e380ff3e0 0x7f4e38195e60 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f4e2800e970 tx=0x7f4e2800ec80 comp rx=0 tx=0).stop 2023-07-21T18:59:45.654 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.640+0000 7f4e3cc62700 1 -- 172.21.15.39:0/771988223 shutdown_connections 2023-07-21T18:59:45.654 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.640+0000 7f4e3cc62700 1 --2- 172.21.15.39:0/771988223 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f4e200a12c0 0x7f4e200a3770 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:45.654 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.640+0000 7f4e3cc62700 1 --2- 172.21.15.39:0/771988223 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e3819aa10 0x7f4e3819ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:45.654 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.640+0000 7f4e3cc62700 1 --2- 172.21.15.39:0/771988223 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e380ff3e0 0x7f4e38195e60 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:45.654 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.640+0000 7f4e3cc62700 1 --2- 172.21.15.39:0/771988223 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e38104a30 0x7f4e381963a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:45.655 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.640+0000 7f4e3cc62700 1 -- 172.21.15.39:0/771988223 >> 172.21.15.39:0/771988223 conn(0x7f4e38074760 msgr2=0x7f4e380fd170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:59:45.655 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.640+0000 7f4e3cc62700 1 -- 172.21.15.39:0/771988223 shutdown_connections 2023-07-21T18:59:45.655 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:45.640+0000 7f4e3cc62700 1 -- 172.21.15.39:0/771988223 wait complete. 2023-07-21T18:59:45.683 INFO:tasks.workunit.client.0.smithi039.stdout: 10 171169 31.44 MB/sec execute 178 sec latency 140.354 ms 2023-07-21T18:59:46.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:59:46 smithi193 ceph-mon[98204]: pgmap v569: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.3 MiB/s wr, 496 op/s 2023-07-21T18:59:46.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:59:46 smithi155 ceph-mon[97934]: pgmap v569: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.3 MiB/s wr, 496 op/s 2023-07-21T18:59:46.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:59:46 smithi039 ceph-mon[96489]: pgmap v569: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.3 MiB/s wr, 496 op/s 2023-07-21T18:59:46.683 INFO:tasks.workunit.client.0.smithi039.stdout: 10 171762 31.46 MB/sec execute 179 sec latency 85.685 ms 2023-07-21T18:59:47.685 INFO:tasks.workunit.client.0.smithi039.stdout: 10 172287 31.48 MB/sec execute 180 sec latency 265.149 ms 2023-07-21T18:59:48.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:59:48 smithi193 ceph-mon[98204]: pgmap v570: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 9.2 MiB/s wr, 766 op/s 2023-07-21T18:59:48.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:59:48 smithi155 ceph-mon[97934]: pgmap v570: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 9.2 MiB/s wr, 766 op/s 2023-07-21T18:59:48.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:59:48 smithi039 ceph-mon[96489]: pgmap v570: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 9.2 MiB/s wr, 766 op/s 2023-07-21T18:59:48.685 INFO:tasks.workunit.client.0.smithi039.stdout: 10 172934 31.45 MB/sec execute 181 sec latency 70.576 ms 2023-07-21T18:59:49.685 INFO:tasks.workunit.client.0.smithi039.stdout: 10 173557 31.52 MB/sec execute 182 sec latency 75.405 ms 2023-07-21T18:59:50.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:59:50 smithi193 ceph-mon[98204]: pgmap v571: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.0 MiB/s wr, 492 op/s 2023-07-21T18:59:50.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:59:50 smithi155 ceph-mon[97934]: pgmap v571: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.0 MiB/s wr, 492 op/s 2023-07-21T18:59:50.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:59:50 smithi039 ceph-mon[96489]: pgmap v571: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.0 MiB/s wr, 492 op/s 2023-07-21T18:59:50.685 INFO:tasks.workunit.client.0.smithi039.stdout: 10 174121 31.50 MB/sec execute 183 sec latency 109.377 ms 2023-07-21T18:59:51.686 INFO:tasks.workunit.client.0.smithi039.stdout: 10 174450 31.42 MB/sec execute 184 sec latency 278.182 ms 2023-07-21T18:59:52.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:59:52 smithi193 ceph-mon[98204]: pgmap v572: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.5 MiB/s wr, 527 op/s 2023-07-21T18:59:52.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:59:52 smithi155 ceph-mon[97934]: pgmap v572: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.5 MiB/s wr, 527 op/s 2023-07-21T18:59:52.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:59:52 smithi039 ceph-mon[96489]: pgmap v572: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.5 MiB/s wr, 527 op/s 2023-07-21T18:59:52.686 INFO:tasks.workunit.client.0.smithi039.stdout: 10 174940 31.37 MB/sec execute 185 sec latency 143.004 ms 2023-07-21T18:59:53.686 INFO:tasks.workunit.client.0.smithi039.stdout: 10 175526 31.37 MB/sec execute 186 sec latency 71.206 ms 2023-07-21T18:59:54.607 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:59:54 smithi193 ceph-mon[98204]: pgmap v573: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 9.4 MiB/s wr, 715 op/s 2023-07-21T18:59:54.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:59:54 smithi155 ceph-mon[97934]: pgmap v573: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 9.4 MiB/s wr, 715 op/s 2023-07-21T18:59:54.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:59:54 smithi039 ceph-mon[96489]: pgmap v573: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 9.4 MiB/s wr, 715 op/s 2023-07-21T18:59:54.686 INFO:tasks.workunit.client.0.smithi039.stdout: 10 176238 31.41 MB/sec execute 187 sec latency 121.749 ms 2023-07-21T18:59:55.686 INFO:tasks.workunit.client.0.smithi039.stdout: 10 176909 31.49 MB/sec execute 188 sec latency 58.915 ms 2023-07-21T18:59:56.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:59:56 smithi193 ceph-mon[98204]: pgmap v574: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.4 MiB/s wr, 498 op/s 2023-07-21T18:59:56.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:59:56 smithi155 ceph-mon[97934]: pgmap v574: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.4 MiB/s wr, 498 op/s 2023-07-21T18:59:56.678 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:59:56 smithi039 ceph-mon[96489]: pgmap v574: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.4 MiB/s wr, 498 op/s 2023-07-21T18:59:56.690 INFO:tasks.workunit.client.0.smithi039.stdout: 10 177291 31.40 MB/sec execute 189 sec latency 308.995 ms 2023-07-21T18:59:57.688 INFO:tasks.workunit.client.0.smithi039.stdout: 10 177298 31.24 MB/sec execute 190 sec latency 1254.561 ms 2023-07-21T18:59:58.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 18:59:58 smithi039 ceph-mon[96489]: pgmap v575: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 12 MiB/s wr, 772 op/s 2023-07-21T18:59:58.690 INFO:tasks.workunit.client.0.smithi039.stdout: 10 177492 31.14 MB/sec execute 191 sec latency 1642.373 ms 2023-07-21T18:59:58.831 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.821+0000 7f40c4435700 1 -- 172.21.15.39:0/1207361623 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40bc10c4d0 msgr2=0x7f40bc107510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:59:58.831 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.821+0000 7f40c4435700 1 --2- 172.21.15.39:0/1207361623 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40bc10c4d0 0x7f40bc107510 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f40ac009990 tx=0x7f40ac009ca0 comp rx=0 tx=0).stop 2023-07-21T18:59:58.831 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.821+0000 7f40c4435700 1 -- 172.21.15.39:0/1207361623 shutdown_connections 2023-07-21T18:59:58.832 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.821+0000 7f40c4435700 1 --2- 172.21.15.39:0/1207361623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40bc06cb80 0x7f40bc06cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:58.832 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.821+0000 7f40c4435700 1 --2- 172.21.15.39:0/1207361623 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40bc10c4d0 0x7f40bc107510 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:58.832 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.821+0000 7f40c4435700 1 --2- 172.21.15.39:0/1207361623 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40bc06d520 0x7f40bc10bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:58.832 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.821+0000 7f40c4435700 1 -- 172.21.15.39:0/1207361623 >> 172.21.15.39:0/1207361623 conn(0x7f40bc06bd70 msgr2=0x7f40bc070f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:59:58.832 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.821+0000 7f40c4435700 1 -- 172.21.15.39:0/1207361623 shutdown_connections 2023-07-21T18:59:58.840 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.830+0000 7f40c4435700 1 -- 172.21.15.39:0/1207361623 wait complete. 2023-07-21T18:59:58.840 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.831+0000 7f40c4435700 1 Processor -- start 2023-07-21T18:59:58.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.831+0000 7f40c4435700 1 -- start start 2023-07-21T18:59:58.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.831+0000 7f40c4435700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40bc06cb80 0x7f40bc119d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:58.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.831+0000 7f40c4435700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40bc06d520 0x7f40bc116b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:58.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.831+0000 7f40c4435700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40bc10c4d0 0x7f40bc1170b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:58.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.831+0000 7f40c4435700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40bc1176b0 con 0x7f40bc06d520 2023-07-21T18:59:58.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.831+0000 7f40c4435700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40bc117820 con 0x7f40bc10c4d0 2023-07-21T18:59:58.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.831+0000 7f40c4435700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40bc1179e0 con 0x7f40bc06cb80 2023-07-21T18:59:58.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.832+0000 7f40c21d1700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40bc06cb80 0x7f40bc119d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:58.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.832+0000 7f40c21d1700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40bc06cb80 0x7f40bc119d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:52484/0 (socket says 172.21.15.39:52484) 2023-07-21T18:59:58.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.832+0000 7f40c21d1700 1 -- 172.21.15.39:0/1251431795 learned_addr learned my addr 172.21.15.39:0/1251431795 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:59:58.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.833+0000 7f40c19d0700 1 --2- 172.21.15.39:0/1251431795 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40bc06d520 0x7f40bc116b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:58.844 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.833+0000 7f40c21d1700 1 -- 172.21.15.39:0/1251431795 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40bc10c4d0 msgr2=0x7f40bc1170b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:59:58.844 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.833+0000 7f40c21d1700 1 --2- 172.21.15.39:0/1251431795 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40bc10c4d0 0x7f40bc1170b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:58.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.833+0000 7f40c21d1700 1 -- 172.21.15.39:0/1251431795 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40bc06d520 msgr2=0x7f40bc116b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:59:58.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.833+0000 7f40c21d1700 1 --2- 172.21.15.39:0/1251431795 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40bc06d520 0x7f40bc116b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:58.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.833+0000 7f40c21d1700 1 -- 172.21.15.39:0/1251431795 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40ac0095f0 con 0x7f40bc06cb80 2023-07-21T18:59:58.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.833+0000 7f40c19d0700 1 --2- 172.21.15.39:0/1251431795 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40bc06d520 0x7f40bc116b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:59:58.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.833+0000 7f40c21d1700 1 --2- 172.21.15.39:0/1251431795 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40bc06cb80 0x7f40bc119d70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f40b800d950 tx=0x7f40b800dd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:59:58.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.834+0000 7f40b37fe700 1 -- 172.21.15.39:0/1251431795 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b8009b40 con 0x7f40bc06cb80 2023-07-21T18:59:58.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.834+0000 7f40b37fe700 1 -- 172.21.15.39:0/1251431795 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40b8009ca0 con 0x7f40bc06cb80 2023-07-21T18:59:58.846 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.834+0000 7f40b37fe700 1 -- 172.21.15.39:0/1251431795 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b800f620 con 0x7f40bc06cb80 2023-07-21T18:59:58.846 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.834+0000 7f40c4435700 1 -- 172.21.15.39:0/1251431795 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40bc1b1490 con 0x7f40bc06cb80 2023-07-21T18:59:58.846 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.834+0000 7f40c4435700 1 -- 172.21.15.39:0/1251431795 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40bc1b18c0 con 0x7f40bc06cb80 2023-07-21T18:59:58.849 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.839+0000 7f40d05bbb80 1 Processor -- start 2023-07-21T18:59:58.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 18:59:58 smithi193 ceph-mon[98204]: pgmap v575: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 12 MiB/s wr, 772 op/s 2023-07-21T18:59:58.860 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.843+0000 7f40d05bbb80 1 -- start start 2023-07-21T18:59:58.861 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.843+0000 7f40d05bbb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db299676b0 0x55db29b10d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:58.861 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.843+0000 7f40d05bbb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55db29b112d0 0x55db29b0b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:58.861 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.844+0000 7f40d05bbb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55db29b0b580 0x55db29b0b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:58.861 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.844+0000 7f40d05bbb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db29a10c50 con 0x55db299676b0 2023-07-21T18:59:58.861 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.844+0000 7f40d05bbb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db29a639e0 con 0x55db29b112d0 2023-07-21T18:59:58.861 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.844+0000 7f40d05bbb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db29a5ce70 con 0x55db29b0b580 2023-07-21T18:59:58.862 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.849+0000 7f40c29d2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db299676b0 0x55db29b10d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:58.862 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.849+0000 7f40c29d2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db299676b0 0x55db29b10d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33940/0 (socket says 172.21.15.39:33940) 2023-07-21T18:59:58.862 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.849+0000 7f40c29d2700 1 -- 172.21.15.39:0/2440929327 learned_addr learned my addr 172.21.15.39:0/2440929327 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:59:58.862 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.849+0000 7f40c19d0700 1 --2- 172.21.15.39:0/2440929327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55db29b0b580 0x55db29b0b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:58.862 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.849+0000 7f40b37fe700 1 -- 172.21.15.39:0/1251431795 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f40b8010460 con 0x7f40bc06cb80 2023-07-21T18:59:58.862 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.849+0000 7f40c29d2700 1 -- 172.21.15.39:0/2440929327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55db29b0b580 msgr2=0x55db29b0b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:59:58.863 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.849+0000 7f40c29d2700 1 --2- 172.21.15.39:0/2440929327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55db29b0b580 0x55db29b0b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:58.863 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.850+0000 7f40c29d2700 1 -- 172.21.15.39:0/2440929327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55db29b112d0 msgr2=0x55db29b0b040 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T18:59:58.863 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.850+0000 7f40c29d2700 1 --2- 172.21.15.39:0/2440929327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55db29b112d0 0x55db29b0b040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:58.863 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.850+0000 7f40c29d2700 1 -- 172.21.15.39:0/2440929327 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db2988ad80 con 0x55db299676b0 2023-07-21T18:59:58.863 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.850+0000 7f40c29d2700 1 --2- 172.21.15.39:0/2440929327 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db299676b0 0x55db29b10d90 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f40ac0098e0 tx=0x7f40ac004500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9630111982a76378 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:59:58.863 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.850+0000 7f40b37fe700 1 --2- 172.21.15.39:0/1251431795 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f40a80c3390 0x7f40a80c5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:58.864 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.850+0000 7f40b37fe700 1 -- 172.21.15.39:0/1251431795 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f40b80ccab0 con 0x7f40bc06cb80 2023-07-21T18:59:58.864 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.850+0000 7f40b0dac700 1 -- 172.21.15.39:0/2440929327 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ac01d070 con 0x55db299676b0 2023-07-21T18:59:58.864 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.850+0000 7f40b0dac700 1 -- 172.21.15.39:0/2440929327 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40ac005030 con 0x55db299676b0 2023-07-21T18:59:58.864 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.850+0000 7f40b0dac700 1 -- 172.21.15.39:0/2440929327 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ac0049a0 con 0x55db299676b0 2023-07-21T18:59:58.865 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.852+0000 7f40d05bbb80 1 -- 172.21.15.39:0/2440929327 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db299676b0 msgr2=0x55db29b10d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:59:58.865 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.852+0000 7f40d05bbb80 1 --2- 172.21.15.39:0/2440929327 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db299676b0 0x55db29b10d90 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f40ac0098e0 tx=0x7f40ac004500 comp rx=0 tx=0).stop 2023-07-21T18:59:58.866 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.852+0000 7f40c21d1700 1 --2- 172.21.15.39:0/1251431795 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f40a80c3390 0x7f40a80c5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:58.866 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.852+0000 7f40d05bbb80 1 -- 172.21.15.39:0/2440929327 shutdown_connections 2023-07-21T18:59:58.866 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.852+0000 7f40d05bbb80 1 --2- 172.21.15.39:0/2440929327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55db29b0b580 0x55db29b0b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:58.866 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.852+0000 7f40d05bbb80 1 --2- 172.21.15.39:0/2440929327 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db299676b0 0x55db29b10d90 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:58.866 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.852+0000 7f40d05bbb80 1 --2- 172.21.15.39:0/2440929327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55db29b112d0 0x55db29b0b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:58.866 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.852+0000 7f40d05bbb80 1 -- 172.21.15.39:0/2440929327 >> 172.21.15.39:0/2440929327 conn(0x55db29b00be0 msgr2=0x55db29b05f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:59:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.853+0000 7f40d05bbb80 1 -- 172.21.15.39:0/2440929327 shutdown_connections 2023-07-21T18:59:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.853+0000 7f40c21d1700 1 --2- 172.21.15.39:0/1251431795 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f40a80c3390 0x7f40a80c5840 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x55db29b07b40 tx=0x7f40b8005910 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:59:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.853+0000 7f40d05bbb80 1 -- 172.21.15.39:0/2440929327 wait complete. 2023-07-21T18:59:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.853+0000 7f40d05bbb80 1 -- start start 2023-07-21T18:59:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.854+0000 7f40d05bbb80 1 Processor -- start 2023-07-21T18:59:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.854+0000 7f40d05bbb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db299676b0 0x55db29b17610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.854+0000 7f40d05bbb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55db29b0b580 0x55db29b14900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.854+0000 7f40d05bbb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55db29b112d0 0x55db29b15040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.854+0000 7f40d05bbb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db299fca60 con 0x55db299676b0 2023-07-21T18:59:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.854+0000 7f40d05bbb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db299fc760 con 0x55db29b112d0 2023-07-21T18:59:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.854+0000 7f40d05bbb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db298e08c0 con 0x55db29b0b580 2023-07-21T18:59:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.854+0000 7f40d05bbb80 10 client.0 _ll_register_callbacks cb 0x55db299fa700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:59:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.855+0000 7f40c19d0700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55db29b0b580 0x55db29b14900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.855+0000 7f40c19d0700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55db29b0b580 0x55db29b14900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:52498/0 (socket says 172.21.15.39:52498) 2023-07-21T18:59:58.869 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.855+0000 7f40c19d0700 1 -- 172.21.15.39:0/2730486971 learned_addr learned my addr 172.21.15.39:0/2730486971 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:59:58.870 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.856+0000 7f40c19d0700 1 -- 172.21.15.39:0/2730486971 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55db29b112d0 msgr2=0x55db29b15040 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T18:59:58.870 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.856+0000 7f40c19d0700 1 --2- 172.21.15.39:0/2730486971 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55db29b112d0 0x55db29b15040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:58.870 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.856+0000 7f40c19d0700 1 -- 172.21.15.39:0/2730486971 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db299676b0 msgr2=0x55db29b17610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:59:58.870 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.856+0000 7f40c19d0700 1 --2- 172.21.15.39:0/2730486971 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db299676b0 0x55db29b17610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:58.870 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.856+0000 7f40c19d0700 1 -- 172.21.15.39:0/2730486971 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40bc107dd0 con 0x55db29b0b580 2023-07-21T18:59:58.870 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.859+0000 7f40c19d0700 1 --2- 172.21.15.39:0/2730486971 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55db29b0b580 0x55db29b14900 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f40b4007ab0 tx=0x7f40b4007e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17150a74706efc81 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:59:58.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.859+0000 7f40a5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2023-07-21T18:59:58.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.859+0000 7f40a5ffb700 1 -- 172.21.15.39:0/2730486971 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b4003e10 con 0x55db29b0b580 2023-07-21T18:59:58.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.859+0000 7f40a5ffb700 1 -- 172.21.15.39:0/2730486971 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f40b4004050 con 0x55db29b0b580 2023-07-21T18:59:58.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.860+0000 7f40d05bbb80 1 -- 172.21.15.39:0/2730486971 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55db299b6490 con 0x55db29b0b580 2023-07-21T18:59:58.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.860+0000 7f40a5ffb700 1 -- 172.21.15.39:0/2730486971 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40b4004210 con 0x55db29b0b580 2023-07-21T18:59:58.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.860+0000 7f40d05bbb80 10 client.26974 fetch_fsmap learned FSMap version 96 2023-07-21T18:59:58.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.860+0000 7f40d05bbb80 1 -- 172.21.15.39:0/2730486971 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=4+}) v3 -- 0x55db29873c00 con 0x55db29b0b580 2023-07-21T18:59:58.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.860+0000 7f40a5ffb700 1 -- 172.21.15.39:0/2730486971 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55db299b6490 con 0x55db29b0b580 2023-07-21T18:59:58.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.864+0000 7f40a5ffb700 1 -- 172.21.15.39:0/2730486971 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 96) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f40b401f070 con 0x55db29b0b580 2023-07-21T18:59:58.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.864+0000 7f40d05bbb80 10 client.26974 fetch_fsmap finished waiting for FSMap version 96 2023-07-21T18:59:58.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.864+0000 7f40d05bbb80 10 client.26974 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:59:58.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.864+0000 7f40d05bbb80 20 client.26974 populate_metadata read hostname 'smithi039' 2023-07-21T18:59:58.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.864+0000 7f40d05bbb80 1 --2- 172.21.15.39:0/2730486971 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55db29b24bb0 0x55db29b26fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:58.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.864+0000 7f40d05bbb80 4 client.26974 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T18:59:58.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.864+0000 7f40d05bbb80 1 -- 172.21.15.39:0/2730486971 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55db29a55e50 con 0x55db29b24bb0 2023-07-21T18:59:58.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 18:59:58 smithi155 ceph-mon[97934]: pgmap v575: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 12 MiB/s wr, 772 op/s 2023-07-21T18:59:58.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.866+0000 7f40c29d2700 1 --2- 172.21.15.39:0/2730486971 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55db29b24bb0 0x55db29b26fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:58.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.868+0000 7f40c29d2700 1 --2- 172.21.15.39:0/2730486971 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55db29b24bb0 0x55db29b26fc0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b354a86c28c1497 server_cookie=e1bf5280a8895457 in_seq=0 out_seq=0 2023-07-21T18:59:58.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.870+0000 7f40a5ffb700 10 client.26974 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:59:58.882 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.870+0000 7f40a5ffb700 1 -- 172.21.15.39:0/2730486971 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55db29a55e50 con 0x55db29b24bb0 2023-07-21T18:59:58.882 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.870+0000 7f40a5ffb700 10 client.26974 handle_command_reply: tid=0 2023-07-21T18:59:58.882 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.870+0000 7f40d05bbb80 1 client.26974 shutdown 2023-07-21T18:59:58.882 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.871+0000 7f40d05bbb80 1 -- 172.21.15.39:0/2730486971 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55db29b0b580 msgr2=0x55db29b14900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:59:58.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.871+0000 7f40d05bbb80 1 --2- 172.21.15.39:0/2730486971 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55db29b0b580 0x55db29b14900 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f40b4007ab0 tx=0x7f40b4007e70 comp rx=0 tx=0).stop 2023-07-21T18:59:58.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.871+0000 7f40d05bbb80 1 -- 172.21.15.39:0/2730486971 shutdown_connections 2023-07-21T18:59:58.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.871+0000 7f40d05bbb80 1 --2- 172.21.15.39:0/2730486971 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55db29b24bb0 0x55db29b26fc0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:58.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.871+0000 7f40d05bbb80 1 --2- 172.21.15.39:0/2730486971 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55db29b112d0 0x55db29b15040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:58.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.871+0000 7f40d05bbb80 1 --2- 172.21.15.39:0/2730486971 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db299676b0 0x55db29b17610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:58.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.871+0000 7f40d05bbb80 1 --2- 172.21.15.39:0/2730486971 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55db29b0b580 0x55db29b14900 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:58.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.871+0000 7f40d05bbb80 1 -- 172.21.15.39:0/2730486971 >> 172.21.15.39:0/2730486971 conn(0x55db29b00be0 msgr2=0x55db29b054e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:59:58.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.871+0000 7f40a5ffb700 0 client.26974 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:59:58.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.872+0000 7f40d05bbb80 1 -- 172.21.15.39:0/2730486971 shutdown_connections 2023-07-21T18:59:58.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.872+0000 7f40d05bbb80 1 -- 172.21.15.39:0/2730486971 wait complete. 2023-07-21T18:59:58.886 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.876+0000 7f40d05bbb80 20 client.26974 trim_cache size 0 max 16384 2023-07-21T18:59:58.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.906+0000 7f40d05bbb80 1 Processor -- start 2023-07-21T18:59:58.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.906+0000 7f40d05bbb80 1 -- start start 2023-07-21T18:59:58.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.907+0000 7f40d05bbb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db29b00be0 0x55db29b1a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:58.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.907+0000 7f40d05bbb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55db299676b0 0x55db29b10180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:58.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.907+0000 7f40d05bbb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55db29b0b580 0x55db29b05c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:58.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.907+0000 7f40d05bbb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db29b079c0 con 0x55db29b00be0 2023-07-21T18:59:58.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.907+0000 7f40d05bbb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db29b025f0 con 0x55db29b0b580 2023-07-21T18:59:58.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.907+0000 7f40d05bbb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db29b15800 con 0x55db299676b0 2023-07-21T18:59:58.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.907+0000 7f40c19d0700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55db299676b0 0x55db29b10180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:58.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.907+0000 7f40c29d2700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55db29b0b580 0x55db29b05c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:58.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.907+0000 7f40c19d0700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55db299676b0 0x55db29b10180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:52510/0 (socket says 172.21.15.39:52510) 2023-07-21T18:59:58.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.907+0000 7f40c19d0700 1 -- 172.21.15.39:0/1966541255 learned_addr learned my addr 172.21.15.39:0/1966541255 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:59:58.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.907+0000 7f40c19d0700 1 -- 172.21.15.39:0/1966541255 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55db29b0b580 msgr2=0x55db29b05c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:59:58.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.907+0000 7f40c19d0700 1 --2- 172.21.15.39:0/1966541255 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55db29b0b580 0x55db29b05c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:58.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.907+0000 7f40c19d0700 1 -- 172.21.15.39:0/1966541255 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db29b00be0 msgr2=0x55db29b1a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:59:58.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.907+0000 7f40c19d0700 1 --2- 172.21.15.39:0/1966541255 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db29b00be0 0x55db29b1a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:58.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.907+0000 7f40c19d0700 1 -- 172.21.15.39:0/1966541255 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db298b11f0 con 0x55db299676b0 2023-07-21T18:59:58.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.908+0000 7f40c19d0700 1 --2- 172.21.15.39:0/1966541255 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55db299676b0 0x55db29b10180 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f40b40113b0 tx=0x7f40b4007f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8323a41da54adb16 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:59:58.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.908+0000 7f40b15ad700 1 -- 172.21.15.39:0/1966541255 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b4006000 con 0x55db299676b0 2023-07-21T18:59:58.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.908+0000 7f40b15ad700 1 -- 172.21.15.39:0/1966541255 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40b4013040 con 0x55db299676b0 2023-07-21T18:59:58.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.908+0000 7f40b15ad700 1 -- 172.21.15.39:0/1966541255 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b4017ea0 con 0x55db299676b0 2023-07-21T18:59:58.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.909+0000 7f40d05bbb80 1 -- 172.21.15.39:0/1966541255 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55db299676b0 msgr2=0x55db29b10180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:59:58.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.909+0000 7f40d05bbb80 1 --2- 172.21.15.39:0/1966541255 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55db299676b0 0x55db29b10180 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f40b40113b0 tx=0x7f40b4007f90 comp rx=0 tx=0).stop 2023-07-21T18:59:58.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.911+0000 7f40d05bbb80 1 -- 172.21.15.39:0/1966541255 shutdown_connections 2023-07-21T18:59:58.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.911+0000 7f40d05bbb80 1 --2- 172.21.15.39:0/1966541255 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55db29b0b580 0x55db29b05c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:58.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.911+0000 7f40d05bbb80 1 --2- 172.21.15.39:0/1966541255 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db29b00be0 0x55db29b1a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:58.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.911+0000 7f40d05bbb80 1 --2- 172.21.15.39:0/1966541255 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55db299676b0 0x55db29b10180 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:58.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.911+0000 7f40d05bbb80 1 -- 172.21.15.39:0/1966541255 >> 172.21.15.39:0/1966541255 conn(0x55db29b151b0 msgr2=0x55db29b0fa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:59:58.924 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.913+0000 7f40d05bbb80 1 -- 172.21.15.39:0/1966541255 shutdown_connections 2023-07-21T18:59:58.924 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.914+0000 7f40d05bbb80 1 -- 172.21.15.39:0/1966541255 wait complete. 2023-07-21T18:59:58.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.914+0000 7f40d05bbb80 1 -- start start 2023-07-21T18:59:58.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.914+0000 7f40d05bbb80 1 Processor -- start 2023-07-21T18:59:58.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.915+0000 7f40d05bbb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55db299676b0 0x55db29b6ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:58.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.915+0000 7f40d05bbb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55db29b00be0 0x55db29b6cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:58.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.915+0000 7f40d05bbb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db29b0b580 0x55db29b77a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:58.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.915+0000 7f40d05bbb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db29b6d810 con 0x55db29b0b580 2023-07-21T18:59:58.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.915+0000 7f40d05bbb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db29b77fc0 con 0x55db29b00be0 2023-07-21T18:59:58.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.915+0000 7f40d05bbb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db29b78100 con 0x55db299676b0 2023-07-21T18:59:58.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.915+0000 7f40d05bbb80 10 client.0 _ll_register_callbacks cb 0x7f40b4009450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T18:59:58.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.920+0000 7f40c29d2700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55db299676b0 0x55db29b6ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:58.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.920+0000 7f40c29d2700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55db299676b0 0x55db29b6ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:52512/0 (socket says 172.21.15.39:52512) 2023-07-21T18:59:58.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.920+0000 7f40c29d2700 1 -- 172.21.15.39:0/3633331129 learned_addr learned my addr 172.21.15.39:0/3633331129 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T18:59:58.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.920+0000 7f40c29d2700 1 --2- 172.21.15.39:0/3633331129 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db29b0b580 0x55db29b77a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:58.932 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.920+0000 7f40c19d0700 1 --2- 172.21.15.39:0/3633331129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55db29b00be0 0x55db29b6cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:58.932 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.920+0000 7f40c29d2700 1 -- 172.21.15.39:0/3633331129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55db29b00be0 msgr2=0x55db29b6cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:59:58.932 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.920+0000 7f40c29d2700 1 --2- 172.21.15.39:0/3633331129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55db29b00be0 0x55db29b6cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:58.932 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.920+0000 7f40c29d2700 1 -- 172.21.15.39:0/3633331129 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db29b0b580 msgr2=0x55db29b77a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T18:59:58.932 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.920+0000 7f40c29d2700 1 --2- 172.21.15.39:0/3633331129 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db29b0b580 0x55db29b77a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:58.932 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.920+0000 7f40c29d2700 1 -- 172.21.15.39:0/3633331129 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55db29873c00 con 0x55db299676b0 2023-07-21T18:59:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.920+0000 7f40c29d2700 1 --2- 172.21.15.39:0/3633331129 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55db299676b0 0x55db29b6ca00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f40ac006010 tx=0x7f40ac0057b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5c8ecd19d4714d6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:59:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.921+0000 7f40a5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2023-07-21T18:59:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.921+0000 7f40a5ffb700 1 -- 172.21.15.39:0/3633331129 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ac01d040 con 0x55db299676b0 2023-07-21T18:59:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.921+0000 7f40a5ffb700 1 -- 172.21.15.39:0/3633331129 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f40ac02b550 con 0x55db299676b0 2023-07-21T18:59:58.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.921+0000 7f40d05bbb80 1 -- 172.21.15.39:0/3633331129 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f40ac005030 con 0x55db299676b0 2023-07-21T18:59:58.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.922+0000 7f40a5ffb700 1 -- 172.21.15.39:0/3633331129 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40ac0051a0 con 0x55db299676b0 2023-07-21T18:59:58.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.925+0000 7f40d05bbb80 10 client.26980 fetch_fsmap learned FSMap version 96 2023-07-21T18:59:58.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.925+0000 7f40d05bbb80 1 -- 172.21.15.39:0/3633331129 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=4+}) v3 -- 0x55db29889250 con 0x55db299676b0 2023-07-21T18:59:58.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.926+0000 7f40a5ffb700 1 -- 172.21.15.39:0/3633331129 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40ac005030 con 0x55db299676b0 2023-07-21T18:59:58.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.926+0000 7f40a5ffb700 1 -- 172.21.15.39:0/3633331129 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 96) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f40ac0045f0 con 0x55db299676b0 2023-07-21T18:59:58.942 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.927+0000 7f40d05bbb80 10 client.26980 fetch_fsmap finished waiting for FSMap version 96 2023-07-21T18:59:58.942 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.927+0000 7f40d05bbb80 10 client.26980 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T18:59:58.942 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.927+0000 7f40d05bbb80 20 client.26980 populate_metadata read hostname 'smithi039' 2023-07-21T18:59:58.942 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.927+0000 7f40d05bbb80 1 --2- 172.21.15.39:0/3633331129 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55db29b112d0 0x55db29b7b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:58.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.927+0000 7f40d05bbb80 4 client.26980 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T18:59:58.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.927+0000 7f40d05bbb80 1 -- 172.21.15.39:0/3633331129 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55db29a17ff0 con 0x55db29b112d0 2023-07-21T18:59:58.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.928+0000 7f40c19d0700 1 --2- 172.21.15.39:0/3633331129 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55db29b112d0 0x55db29b7b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:58.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.930+0000 7f40a5ffb700 10 client.26980 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T18:59:58.944 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:58.930+0000 7f40c19d0700 1 --2- 172.21.15.39:0/3633331129 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55db29b112d0 0x55db29b7b3c0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b22a3518196164e1 server_cookie=4efc69d8ff21fb58 in_seq=0 out_seq=0 2023-07-21T18:59:59.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:59.128+0000 7f40a5ffb700 1 -- 172.21.15.39:0/3633331129 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55db29a17ff0 con 0x55db29b112d0 2023-07-21T18:59:59.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:59.128+0000 7f40a5ffb700 10 client.26980 handle_command_reply: tid=0 2023-07-21T18:59:59.139 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:59.128+0000 7f40d05bbb80 1 client.26980 shutdown 2023-07-21T18:59:59.139 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:59.130+0000 7f40d05bbb80 1 -- 172.21.15.39:0/3633331129 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55db299676b0 msgr2=0x55db29b6ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:59:59.139 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:59.130+0000 7f40d05bbb80 1 --2- 172.21.15.39:0/3633331129 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55db299676b0 0x55db29b6ca00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f40ac006010 tx=0x7f40ac0057b0 comp rx=0 tx=0).stop 2023-07-21T18:59:59.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:59.132+0000 7f40d05bbb80 1 -- 172.21.15.39:0/3633331129 shutdown_connections 2023-07-21T18:59:59.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:59.132+0000 7f40d05bbb80 1 --2- 172.21.15.39:0/3633331129 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55db29b112d0 0x55db29b7b3c0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:59.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:59.132+0000 7f40d05bbb80 1 --2- 172.21.15.39:0/3633331129 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db29b0b580 0x55db29b77a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:59.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:59.132+0000 7f40d05bbb80 1 --2- 172.21.15.39:0/3633331129 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55db299676b0 0x55db29b6ca00 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:59.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:59.132+0000 7f40d05bbb80 1 --2- 172.21.15.39:0/3633331129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55db29b00be0 0x55db29b6cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:59.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:59.132+0000 7f40d05bbb80 1 -- 172.21.15.39:0/3633331129 >> 172.21.15.39:0/3633331129 conn(0x55db29b151b0 msgr2=0x55db29b10300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:59:59.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:59.133+0000 7f40a5ffb700 0 client.26980 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T18:59:59.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:59.133+0000 7f40d05bbb80 1 -- 172.21.15.39:0/3633331129 shutdown_connections 2023-07-21T18:59:59.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:59.133+0000 7f40d05bbb80 1 -- 172.21.15.39:0/3633331129 wait complete. 2023-07-21T18:59:59.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:59.136+0000 7f40d05bbb80 20 client.26980 trim_cache size 0 max 16384 2023-07-21T18:59:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T18:59:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T18:59:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T18:59:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T18:59:59.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:59.141+0000 7f40c4435700 1 -- 172.21.15.39:0/1251431795 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f40a80c3390 msgr2=0x7f40a80c5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:59:59.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:59.141+0000 7f40c4435700 1 --2- 172.21.15.39:0/1251431795 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f40a80c3390 0x7f40a80c5840 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x55db29b07b40 tx=0x7f40b8005910 comp rx=0 tx=0).stop 2023-07-21T18:59:59.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:59.141+0000 7f40c4435700 1 -- 172.21.15.39:0/1251431795 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40bc06cb80 msgr2=0x7f40bc119d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:59:59.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:59.141+0000 7f40c4435700 1 --2- 172.21.15.39:0/1251431795 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40bc06cb80 0x7f40bc119d70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f40b800d950 tx=0x7f40b800dd10 comp rx=0 tx=0).stop 2023-07-21T18:59:59.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:59.141+0000 7f40c4435700 1 -- 172.21.15.39:0/1251431795 shutdown_connections 2023-07-21T18:59:59.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:59.141+0000 7f40c4435700 1 --2- 172.21.15.39:0/1251431795 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f40a80c3390 0x7f40a80c5840 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:59.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:59.141+0000 7f40c4435700 1 --2- 172.21.15.39:0/1251431795 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40bc10c4d0 0x7f40bc1170b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:59.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:59.141+0000 7f40c4435700 1 --2- 172.21.15.39:0/1251431795 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40bc06cb80 0x7f40bc119d70 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:59.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:59.141+0000 7f40c4435700 1 --2- 172.21.15.39:0/1251431795 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40bc06d520 0x7f40bc116b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:59.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:59.141+0000 7f40c4435700 1 -- 172.21.15.39:0/1251431795 >> 172.21.15.39:0/1251431795 conn(0x7f40bc06bd70 msgr2=0x7f40bc109fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:59:59.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:59.141+0000 7f40c4435700 1 -- 172.21.15.39:0/1251431795 shutdown_connections 2023-07-21T18:59:59.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T18:59:59.141+0000 7f40c4435700 1 -- 172.21.15.39:0/1251431795 wait complete. 2023-07-21T18:59:59.690 INFO:tasks.workunit.client.0.smithi039.stdout: 10 177730 31.01 MB/sec execute 192 sec latency 177.994 ms 2023-07-21T19:00:00.357 INFO:journalctl@ceph.mds.c.smithi193.stdout:Jul 21 18:59:59 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-c[124252]: ignoring --setuser ceph since I am not root 2023-07-21T19:00:00.358 INFO:journalctl@ceph.mds.c.smithi193.stdout:Jul 21 18:59:59 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-c[124252]: ignoring --setgroup ceph since I am not root 2023-07-21T19:00:00.358 INFO:journalctl@ceph.mds.c.smithi193.stdout:Jul 21 18:59:59 smithi193 ceph-mds[124276]: -- 172.21.15.193:0/3969924527 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a137e7f1e0 con 0x55a137e54c00 2023-07-21T19:00:00.358 INFO:journalctl@ceph.mds.c.smithi193.stdout:Jul 21 18:59:59 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-c[124252]: starting mds.c at 2023-07-21T19:00:00.382 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:00:00 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mon-a[96465]: 2023-07-21T19:00:00.001+0000 7f094dd1e700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 3 MDSs report damaged metadata 2023-07-21T19:00:00.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:00:00 smithi039 ceph-mon[96489]: pgmap v576: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 8.8 MiB/s wr, 501 op/s 2023-07-21T19:00:00.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:00:00 smithi039 ceph-mon[96489]: daemon mds.c restarted 2023-07-21T19:00:00.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:00:00 smithi039 ceph-mon[96489]: overall HEALTH_ERR 3 MDSs report damaged metadata 2023-07-21T19:00:00.690 INFO:tasks.workunit.client.0.smithi039.stdout: 10 178145 30.97 MB/sec execute 193 sec latency 118.454 ms 2023-07-21T19:00:00.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:00:00 smithi193 ceph-mon[98204]: pgmap v576: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 8.8 MiB/s wr, 501 op/s 2023-07-21T19:00:00.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:00:00 smithi193 ceph-mon[98204]: daemon mds.c restarted 2023-07-21T19:00:00.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:00:00 smithi193 ceph-mon[98204]: overall HEALTH_ERR 3 MDSs report damaged metadata 2023-07-21T19:00:00.879 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:00:00 smithi155 ceph-mon[97934]: pgmap v576: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 8.8 MiB/s wr, 501 op/s 2023-07-21T19:00:00.880 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:00:00 smithi155 ceph-mon[97934]: daemon mds.c restarted 2023-07-21T19:00:00.880 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:00:00 smithi155 ceph-mon[97934]: overall HEALTH_ERR 3 MDSs report damaged metadata 2023-07-21T19:00:01.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:00:01 smithi039 ceph-mon[96489]: osdmap e109: 12 total, 12 up, 12 in 2023-07-21T19:00:01.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:00:01 smithi039 ceph-mon[96489]: mds.? [v2:172.21.15.193:6836/4598510,v1:172.21.15.193:6837/4598510] up:boot 2023-07-21T19:00:01.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:00:01 smithi039 ceph-mon[96489]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2023-07-21T19:00:01.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:00:01 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-21T19:00:01.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:00:01 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T19:00:01.690 INFO:tasks.workunit.client.0.smithi039.stdout: 10 178626 30.95 MB/sec execute 194 sec latency 564.924 ms 2023-07-21T19:00:01.859 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:00:01 smithi193 ceph-mon[98204]: osdmap e109: 12 total, 12 up, 12 in 2023-07-21T19:00:01.860 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:00:01 smithi193 ceph-mon[98204]: mds.? [v2:172.21.15.193:6836/4598510,v1:172.21.15.193:6837/4598510] up:boot 2023-07-21T19:00:01.860 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:00:01 smithi193 ceph-mon[98204]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2023-07-21T19:00:01.860 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:00:01 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-21T19:00:01.860 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:00:01 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T19:00:01.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:00:01 smithi155 ceph-mon[97934]: osdmap e109: 12 total, 12 up, 12 in 2023-07-21T19:00:01.878 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:00:01 smithi155 ceph-mon[97934]: mds.? [v2:172.21.15.193:6836/4598510,v1:172.21.15.193:6837/4598510] up:boot 2023-07-21T19:00:01.878 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:00:01 smithi155 ceph-mon[97934]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2023-07-21T19:00:01.878 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:00:01 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-21T19:00:01.878 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:00:01 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T19:00:02.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:00:02 smithi039 ceph-mon[96489]: pgmap v578: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 11 MiB/s wr, 610 op/s 2023-07-21T19:00:02.690 INFO:tasks.workunit.client.0.smithi039.stdout: 10 179069 30.90 MB/sec execute 195 sec latency 1307.476 ms 2023-07-21T19:00:02.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:00:02 smithi193 ceph-mon[98204]: pgmap v578: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 11 MiB/s wr, 610 op/s 2023-07-21T19:00:02.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:00:02 smithi155 ceph-mon[97934]: pgmap v578: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 11 MiB/s wr, 610 op/s 2023-07-21T19:00:03.691 INFO:tasks.workunit.client.0.smithi039.stdout: 10 179549 30.91 MB/sec execute 196 sec latency 265.385 ms 2023-07-21T19:00:04.680 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:00:04 smithi039 ceph-mon[96489]: pgmap v579: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 28 MiB/s wr, 1.10k op/s 2023-07-21T19:00:04.680 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:00:04 smithi039 ceph-mon[96489]: osdmap e110: 12 total, 12 up, 12 in 2023-07-21T19:00:04.691 INFO:tasks.workunit.client.0.smithi039.stdout: 10 179716 30.81 MB/sec execute 197 sec latency 573.492 ms 2023-07-21T19:00:04.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:00:04 smithi193 ceph-mon[98204]: pgmap v579: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 28 MiB/s wr, 1.10k op/s 2023-07-21T19:00:04.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:00:04 smithi193 ceph-mon[98204]: osdmap e110: 12 total, 12 up, 12 in 2023-07-21T19:00:04.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:00:04 smithi155 ceph-mon[97934]: pgmap v579: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 28 MiB/s wr, 1.10k op/s 2023-07-21T19:00:04.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:00:04 smithi155 ceph-mon[97934]: osdmap e110: 12 total, 12 up, 12 in 2023-07-21T19:00:05.691 INFO:tasks.workunit.client.0.smithi039.stdout: 10 180008 30.73 MB/sec execute 198 sec latency 439.704 ms 2023-07-21T19:00:06.692 INFO:tasks.workunit.client.0.smithi039.stdout: 10 180445 30.67 MB/sec execute 199 sec latency 118.603 ms 2023-07-21T19:00:06.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:00:06 smithi193 ceph-mon[98204]: pgmap v581: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 27 MiB/s wr, 969 op/s 2023-07-21T19:00:06.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:00:06 smithi155 ceph-mon[97934]: pgmap v581: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 27 MiB/s wr, 969 op/s 2023-07-21T19:00:06.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:00:06 smithi039 ceph-mon[96489]: pgmap v581: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 27 MiB/s wr, 969 op/s 2023-07-21T19:00:07.692 INFO:tasks.workunit.client.0.smithi039.stdout: 10 180921 30.63 MB/sec execute 200 sec latency 135.685 ms 2023-07-21T19:00:08.692 INFO:tasks.workunit.client.0.smithi039.stdout: 10 181639 30.67 MB/sec execute 201 sec latency 65.597 ms 2023-07-21T19:00:08.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:00:08 smithi193 ceph-mon[98204]: pgmap v582: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 30 MiB/s wr, 1.40k op/s 2023-07-21T19:00:08.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:00:08 smithi155 ceph-mon[97934]: pgmap v582: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 30 MiB/s wr, 1.40k op/s 2023-07-21T19:00:08.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:00:08 smithi039 ceph-mon[96489]: pgmap v582: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 30 MiB/s wr, 1.40k op/s 2023-07-21T19:00:09.692 INFO:tasks.workunit.client.0.smithi039.stdout: 10 182200 30.67 MB/sec execute 202 sec latency 146.134 ms 2023-07-21T19:00:10.692 INFO:tasks.workunit.client.0.smithi039.stdout: 10 182879 30.72 MB/sec execute 203 sec latency 65.783 ms 2023-07-21T19:00:10.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:00:10 smithi193 ceph-mon[98204]: pgmap v583: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 30 MiB/s wr, 1.36k op/s 2023-07-21T19:00:10.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:00:10 smithi155 ceph-mon[97934]: pgmap v583: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 30 MiB/s wr, 1.36k op/s 2023-07-21T19:00:10.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:00:10 smithi039 ceph-mon[96489]: pgmap v583: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 30 MiB/s wr, 1.36k op/s 2023-07-21T19:00:11.692 INFO:tasks.workunit.client.0.smithi039.stdout: 10 183480 30.74 MB/sec execute 204 sec latency 68.509 ms 2023-07-21T19:00:12.324 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.313+0000 7f6ea5179700 1 -- 172.21.15.39:0/3349499831 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ea0103ed0 msgr2=0x7f6ea010baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:00:12.325 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.313+0000 7f6ea5179700 1 --2- 172.21.15.39:0/3349499831 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ea0103ed0 0x7f6ea010baa0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f6e9800b0d0 tx=0x7f6e9800b3e0 comp rx=0 tx=0).stop 2023-07-21T19:00:12.325 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.313+0000 7f6ea5179700 1 -- 172.21.15.39:0/3349499831 shutdown_connections 2023-07-21T19:00:12.325 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.313+0000 7f6ea5179700 1 --2- 172.21.15.39:0/3349499831 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ea00ff3b0 0x7f6ea00ff780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:12.325 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.313+0000 7f6ea5179700 1 --2- 172.21.15.39:0/3349499831 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ea0103ed0 0x7f6ea010baa0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:12.326 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.313+0000 7f6ea5179700 1 --2- 172.21.15.39:0/3349499831 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ea00ffd50 0x7f6ea01037f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:12.326 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.313+0000 7f6ea5179700 1 -- 172.21.15.39:0/3349499831 >> 172.21.15.39:0/3349499831 conn(0x7f6ea0075f90 msgr2=0x7f6ea0076390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:00:12.326 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.313+0000 7f6ea5179700 1 -- 172.21.15.39:0/3349499831 shutdown_connections 2023-07-21T19:00:12.326 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.313+0000 7f6ea5179700 1 -- 172.21.15.39:0/3349499831 wait complete. 2023-07-21T19:00:12.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.314+0000 7f6ea5179700 1 Processor -- start 2023-07-21T19:00:12.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.315+0000 7f6ea5179700 1 -- start start 2023-07-21T19:00:12.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.315+0000 7f6ea5179700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ea00ff3b0 0x7f6ea0196f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:12.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.315+0000 7f6ea5179700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ea00ffd50 0x7f6ea01974b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:12.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.315+0000 7f6ea5179700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ea0103ed0 0x7f6ea01a1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:12.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.315+0000 7f6ea5179700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea01a2410 con 0x7f6ea00ffd50 2023-07-21T19:00:12.329 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.315+0000 7f6ea5179700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea01a2580 con 0x7f6ea0103ed0 2023-07-21T19:00:12.329 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.315+0000 7f6ea5179700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea01a2740 con 0x7f6ea00ff3b0 2023-07-21T19:00:12.329 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.315+0000 7f6e9ed9d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ea00ff3b0 0x7f6ea0196f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:12.329 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.315+0000 7f6e9ed9d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ea00ff3b0 0x7f6ea0196f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:52526/0 (socket says 172.21.15.39:52526) 2023-07-21T19:00:12.329 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.315+0000 7f6e9ed9d700 1 -- 172.21.15.39:0/408123003 learned_addr learned my addr 172.21.15.39:0/408123003 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:00:12.329 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.315+0000 7f6e9e59c700 1 --2- 172.21.15.39:0/408123003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ea00ffd50 0x7f6ea01974b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:12.329 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.315+0000 7f6e9f59e700 1 --2- 172.21.15.39:0/408123003 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ea0103ed0 0x7f6ea01a1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:12.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.315+0000 7f6e9ed9d700 1 -- 172.21.15.39:0/408123003 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ea0103ed0 msgr2=0x7f6ea01a1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:00:12.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.315+0000 7f6e9ed9d700 1 --2- 172.21.15.39:0/408123003 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ea0103ed0 0x7f6ea01a1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:12.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.315+0000 7f6e9ed9d700 1 -- 172.21.15.39:0/408123003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ea00ffd50 msgr2=0x7f6ea01974b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:00:12.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.316+0000 7f6e9ed9d700 1 --2- 172.21.15.39:0/408123003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ea00ffd50 0x7f6ea01974b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:12.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.316+0000 7f6e9ed9d700 1 -- 172.21.15.39:0/408123003 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e98009d00 con 0x7f6ea00ff3b0 2023-07-21T19:00:12.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.316+0000 7f6e9e59c700 1 --2- 172.21.15.39:0/408123003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ea00ffd50 0x7f6ea01974b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:00:12.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.316+0000 7f6e9ed9d700 1 --2- 172.21.15.39:0/408123003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ea00ff3b0 0x7f6ea0196f70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6e9000e9a0 tx=0x7f6e9000ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:00:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.316+0000 7f6e87fff700 1 -- 172.21.15.39:0/408123003 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e90009790 con 0x7f6ea00ff3b0 2023-07-21T19:00:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.316+0000 7f6e87fff700 1 -- 172.21.15.39:0/408123003 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e9000cbd0 con 0x7f6ea00ff3b0 2023-07-21T19:00:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.316+0000 7f6ea5179700 1 -- 172.21.15.39:0/408123003 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ea01a2a20 con 0x7f6ea00ff3b0 2023-07-21T19:00:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.316+0000 7f6ea5179700 1 -- 172.21.15.39:0/408123003 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ea01a2ee0 con 0x7f6ea00ff3b0 2023-07-21T19:00:12.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.316+0000 7f6e87fff700 1 -- 172.21.15.39:0/408123003 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e90018680 con 0x7f6ea00ff3b0 2023-07-21T19:00:12.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.320+0000 7f6e87fff700 1 -- 172.21.15.39:0/408123003 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f6e9000cd40 con 0x7f6ea00ff3b0 2023-07-21T19:00:12.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.321+0000 7f6e87fff700 1 --2- 172.21.15.39:0/408123003 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f6e880a12c0 0x7f6e880a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:12.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.321+0000 7f6e87fff700 1 -- 172.21.15.39:0/408123003 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f6e90014070 con 0x7f6ea00ff3b0 2023-07-21T19:00:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.321+0000 7f6eb12ffb80 1 Processor -- start 2023-07-21T19:00:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.321+0000 7f6eb12ffb80 1 -- start start 2023-07-21T19:00:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.322+0000 7f6e9e59c700 1 --2- 172.21.15.39:0/408123003 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f6e880a12c0 0x7f6e880a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.322+0000 7f6eb12ffb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b4611b76b0 0x55b46135ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.322+0000 7f6eb12ffb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b46135b040 0x55b46135b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.322+0000 7f6eb12ffb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b46135b9f0 0x55b46136fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.322+0000 7f6eb12ffb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b461260c50 con 0x55b46135b040 2023-07-21T19:00:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.322+0000 7f6eb12ffb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4612b39e0 con 0x55b46135b9f0 2023-07-21T19:00:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.322+0000 7f6eb12ffb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4612ace70 con 0x55b4611b76b0 2023-07-21T19:00:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.322+0000 7f6e9e59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b46135b040 0x55b46135b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.322+0000 7f6e9ed9d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b4611b76b0 0x55b46135ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.322+0000 7f6e9e59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b46135b040 0x55b46135b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33984/0 (socket says 172.21.15.39:33984) 2023-07-21T19:00:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.322+0000 7f6e9e59c700 1 -- 172.21.15.39:0/2044509694 learned_addr learned my addr 172.21.15.39:0/2044509694 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:00:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.322+0000 7f6e9f59e700 1 --2- 172.21.15.39:0/2044509694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b46135b9f0 0x55b46136fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.322+0000 7f6e9e59c700 1 --2- 172.21.15.39:0/408123003 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f6e880a12c0 0x7f6e880a3770 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6ea00feb50 tx=0x7f6e940106b0 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:00:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.322+0000 7f6e9ed9d700 1 -- 172.21.15.39:0/2044509694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b46135b9f0 msgr2=0x55b46136fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:00:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.322+0000 7f6e9ed9d700 1 --2- 172.21.15.39:0/2044509694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b46135b9f0 0x55b46136fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.322+0000 7f6e9ed9d700 1 -- 172.21.15.39:0/2044509694 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b46135b040 msgr2=0x55b46135b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:00:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.322+0000 7f6e9ed9d700 1 --2- 172.21.15.39:0/2044509694 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b46135b040 0x55b46135b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.322+0000 7f6e9ed9d700 1 -- 172.21.15.39:0/2044509694 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b4610dad80 con 0x55b4611b76b0 2023-07-21T19:00:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.323+0000 7f6e9ed9d700 1 --2- 172.21.15.39:0/2044509694 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b4611b76b0 0x55b46135ede0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6e900ca8b0 tx=0x7f6e90099ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79d8cc9aa9603b09 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:00:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.323+0000 7f6e857fa700 1 -- 172.21.15.39:0/2044509694 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e9009e260 con 0x55b4611b76b0 2023-07-21T19:00:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.323+0000 7f6e857fa700 1 -- 172.21.15.39:0/2044509694 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e900a2680 con 0x55b4611b76b0 2023-07-21T19:00:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.323+0000 7f6e857fa700 1 -- 172.21.15.39:0/2044509694 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e90099920 con 0x55b4611b76b0 2023-07-21T19:00:12.339 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.324+0000 7f6eb12ffb80 1 -- 172.21.15.39:0/2044509694 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b4611b76b0 msgr2=0x55b46135ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:00:12.339 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.324+0000 7f6eb12ffb80 1 --2- 172.21.15.39:0/2044509694 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b4611b76b0 0x55b46135ede0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6e900ca8b0 tx=0x7f6e90099ea0 comp rx=0 tx=0).stop 2023-07-21T19:00:12.339 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.324+0000 7f6eb12ffb80 1 -- 172.21.15.39:0/2044509694 shutdown_connections 2023-07-21T19:00:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.324+0000 7f6eb12ffb80 1 --2- 172.21.15.39:0/2044509694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b46135b9f0 0x55b46136fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.324+0000 7f6eb12ffb80 1 --2- 172.21.15.39:0/2044509694 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b4611b76b0 0x55b46135ede0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.324+0000 7f6eb12ffb80 1 --2- 172.21.15.39:0/2044509694 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b46135b040 0x55b46135b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.324+0000 7f6eb12ffb80 1 -- 172.21.15.39:0/2044509694 >> 172.21.15.39:0/2044509694 conn(0x55b461350be0 msgr2=0x55b461355f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:00:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.324+0000 7f6eb12ffb80 1 -- 172.21.15.39:0/2044509694 shutdown_connections 2023-07-21T19:00:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.324+0000 7f6eb12ffb80 1 -- 172.21.15.39:0/2044509694 wait complete. 2023-07-21T19:00:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.325+0000 7f6eb12ffb80 1 -- start start 2023-07-21T19:00:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.325+0000 7f6eb12ffb80 1 Processor -- start 2023-07-21T19:00:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.325+0000 7f6eb12ffb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b4611b76b0 0x55b4613696a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.325+0000 7f6eb12ffb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b46135b040 0x55b461369be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.325+0000 7f6eb12ffb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b46135b9f0 0x55b46136f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.325+0000 7f6eb12ffb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b46124ca60 con 0x55b46135b9f0 2023-07-21T19:00:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.325+0000 7f6eb12ffb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b46124c760 con 0x55b4611b76b0 2023-07-21T19:00:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.325+0000 7f6eb12ffb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4611308c0 con 0x55b46135b040 2023-07-21T19:00:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.325+0000 7f6eb12ffb80 10 client.0 _ll_register_callbacks cb 0x55b46124a700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:00:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.325+0000 7f6e9ed9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b4611b76b0 0x55b4613696a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.325+0000 7f6e9ed9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b4611b76b0 0x55b4613696a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:46584/0 (socket says 172.21.15.39:46584) 2023-07-21T19:00:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.325+0000 7f6e9ed9d700 1 -- 172.21.15.39:0/3338323789 learned_addr learned my addr 172.21.15.39:0/3338323789 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:00:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.325+0000 7f6e9f59e700 1 --2- 172.21.15.39:0/3338323789 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b46135b9f0 0x55b46136f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.325+0000 7f6e9e59c700 1 --2- 172.21.15.39:0/3338323789 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b46135b040 0x55b461369be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:12.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.326+0000 7f6e9ed9d700 1 -- 172.21.15.39:0/3338323789 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b46135b040 msgr2=0x55b461369be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:00:12.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.326+0000 7f6e9ed9d700 1 --2- 172.21.15.39:0/3338323789 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b46135b040 0x55b461369be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:12.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.326+0000 7f6e9ed9d700 1 -- 172.21.15.39:0/3338323789 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b46135b9f0 msgr2=0x55b46136f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:00:12.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.326+0000 7f6e9ed9d700 1 --2- 172.21.15.39:0/3338323789 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b46135b9f0 0x55b46136f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:12.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.326+0000 7f6e9ed9d700 1 -- 172.21.15.39:0/3338323789 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ea01a2ee0 con 0x55b4611b76b0 2023-07-21T19:00:12.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.326+0000 7f6e9ed9d700 1 --2- 172.21.15.39:0/3338323789 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b4611b76b0 0x55b4613696a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6e900ca960 tx=0x7f6e9009a3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=affced364c8236f7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:00:12.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.326+0000 7f6e82ff5700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-21T19:00:12.344 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.326+0000 7f6e82ff5700 1 -- 172.21.15.39:0/3338323789 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e9009a6a0 con 0x55b4611b76b0 2023-07-21T19:00:12.344 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.326+0000 7f6e82ff5700 1 -- 172.21.15.39:0/3338323789 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f6e90094e40 con 0x55b4611b76b0 2023-07-21T19:00:12.344 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.326+0000 7f6eb12ffb80 1 -- 172.21.15.39:0/3338323789 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b461206490 con 0x55b4611b76b0 2023-07-21T19:00:12.344 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.326+0000 7f6e82ff5700 1 -- 172.21.15.39:0/3338323789 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=98) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e900a21f0 con 0x55b4611b76b0 2023-07-21T19:00:12.344 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.326+0000 7f6eb12ffb80 10 client.26948 fetch_fsmap learned FSMap version 98 2023-07-21T19:00:12.344 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.327+0000 7f6eb12ffb80 1 -- 172.21.15.39:0/3338323789 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=98,monmap=4+}) v3 -- 0x55b4610c3c00 con 0x55b4611b76b0 2023-07-21T19:00:12.344 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.327+0000 7f6e82ff5700 1 -- 172.21.15.39:0/3338323789 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b461206490 con 0x55b4611b76b0 2023-07-21T19:00:12.345 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.327+0000 7f6e82ff5700 1 -- 172.21.15.39:0/3338323789 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 98) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f6e900b8020 con 0x55b4611b76b0 2023-07-21T19:00:12.345 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.327+0000 7f6eb12ffb80 10 client.26948 fetch_fsmap finished waiting for FSMap version 98 2023-07-21T19:00:12.345 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.327+0000 7f6eb12ffb80 10 client.26948 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:00:12.345 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.327+0000 7f6eb12ffb80 20 client.26948 populate_metadata read hostname 'smithi039' 2023-07-21T19:00:12.345 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.327+0000 7f6eb12ffb80 1 --2- 172.21.15.39:0/3338323789 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55b461396a10 0x55b461398df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:12.345 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.327+0000 7f6eb12ffb80 4 client.26948 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:00:12.346 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.327+0000 7f6eb12ffb80 1 -- 172.21.15.39:0/3338323789 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b4612a5e50 con 0x55b461396a10 2023-07-21T19:00:12.346 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.327+0000 7f6e9e59c700 1 --2- 172.21.15.39:0/3338323789 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55b461396a10 0x55b461398df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:12.346 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.328+0000 7f6e82ff5700 10 client.26948 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T19:00:12.346 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.328+0000 7f6e9e59c700 1 --2- 172.21.15.39:0/3338323789 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55b461396a10 0x55b461398df0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=778dd93aa52bbeef server_cookie=1250858f7f9f5e6b in_seq=0 out_seq=0 2023-07-21T19:00:12.347 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.330+0000 7f6e82ff5700 1 -- 172.21.15.39:0/3338323789 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b4612a5e50 con 0x55b461396a10 2023-07-21T19:00:12.347 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.330+0000 7f6e82ff5700 10 client.26948 handle_command_reply: tid=0 2023-07-21T19:00:12.347 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.330+0000 7f6eb12ffb80 1 client.26948 shutdown 2023-07-21T19:00:12.347 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.331+0000 7f6eb12ffb80 1 -- 172.21.15.39:0/3338323789 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b4611b76b0 msgr2=0x55b4613696a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:00:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.331+0000 7f6eb12ffb80 1 --2- 172.21.15.39:0/3338323789 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b4611b76b0 0x55b4613696a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6e900ca960 tx=0x7f6e9009a3b0 comp rx=0 tx=0).stop 2023-07-21T19:00:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.331+0000 7f6eb12ffb80 1 -- 172.21.15.39:0/3338323789 shutdown_connections 2023-07-21T19:00:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.331+0000 7f6eb12ffb80 1 --2- 172.21.15.39:0/3338323789 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55b461396a10 0x55b461398df0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.331+0000 7f6eb12ffb80 1 --2- 172.21.15.39:0/3338323789 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b4611b76b0 0x55b4613696a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.331+0000 7f6e82ff5700 0 client.26948 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T19:00:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.331+0000 7f6eb12ffb80 1 --2- 172.21.15.39:0/3338323789 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b46135b9f0 0x55b46136f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.331+0000 7f6eb12ffb80 1 --2- 172.21.15.39:0/3338323789 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b46135b040 0x55b461369be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.331+0000 7f6eb12ffb80 1 -- 172.21.15.39:0/3338323789 >> 172.21.15.39:0/3338323789 conn(0x55b461350be0 msgr2=0x55b4613554e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:00:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.331+0000 7f6eb12ffb80 1 -- 172.21.15.39:0/3338323789 shutdown_connections 2023-07-21T19:00:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.331+0000 7f6eb12ffb80 1 -- 172.21.15.39:0/3338323789 wait complete. 2023-07-21T19:00:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.331+0000 7f6eb12ffb80 20 client.26948 trim_cache size 0 max 16384 2023-07-21T19:00:12.362 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.350+0000 7f6eb12ffb80 1 Processor -- start 2023-07-21T19:00:12.362 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.350+0000 7f6eb12ffb80 1 -- start start 2023-07-21T19:00:12.362 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.351+0000 7f6eb12ffb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b4611b76b0 0x55b461364940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.351+0000 7f6eb12ffb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b46135b040 0x55b46135ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.351+0000 7f6eb12ffb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b46135b9f0 0x55b461369250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.351+0000 7f6eb12ffb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b461370020 con 0x55b46135b9f0 2023-07-21T19:00:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.351+0000 7f6eb12ffb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4612a5e50 con 0x55b46135b040 2023-07-21T19:00:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.351+0000 7f6eb12ffb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b461206490 con 0x55b4611b76b0 2023-07-21T19:00:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.351+0000 7f6e9ed9d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b4611b76b0 0x55b461364940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:12.364 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.351+0000 7f6e9f59e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b46135b9f0 0x55b461369250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:12.364 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.351+0000 7f6e9ed9d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b4611b76b0 0x55b461364940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:52548/0 (socket says 172.21.15.39:52548) 2023-07-21T19:00:12.364 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.351+0000 7f6e9ed9d700 1 -- 172.21.15.39:0/1288951073 learned_addr learned my addr 172.21.15.39:0/1288951073 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:00:12.364 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.351+0000 7f6e9e59c700 1 --2- 172.21.15.39:0/1288951073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b46135b040 0x55b46135ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.351+0000 7f6e9ed9d700 1 -- 172.21.15.39:0/1288951073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b46135b040 msgr2=0x55b46135ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:00:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.351+0000 7f6e9ed9d700 1 --2- 172.21.15.39:0/1288951073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b46135b040 0x55b46135ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.351+0000 7f6e9ed9d700 1 -- 172.21.15.39:0/1288951073 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b46135b9f0 msgr2=0x55b461369250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:00:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.351+0000 7f6e9ed9d700 1 --2- 172.21.15.39:0/1288951073 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b46135b9f0 0x55b461369250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:12.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.351+0000 7f6e9ed9d700 1 -- 172.21.15.39:0/1288951073 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b4611011f0 con 0x55b4611b76b0 2023-07-21T19:00:12.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.351+0000 7f6e9f59e700 1 --2- 172.21.15.39:0/1288951073 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b46135b9f0 0x55b461369250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:00:12.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.352+0000 7f6e9ed9d700 1 --2- 172.21.15.39:0/1288951073 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b4611b76b0 0x55b461364940 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6e9000f010 tx=0x7f6e90099a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5517c629c3482406 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:00:12.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.352+0000 7f6e85ffb700 1 -- 172.21.15.39:0/1288951073 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e9001f340 con 0x55b4611b76b0 2023-07-21T19:00:12.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.352+0000 7f6e85ffb700 1 -- 172.21.15.39:0/1288951073 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e90099270 con 0x55b4611b76b0 2023-07-21T19:00:12.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.352+0000 7f6e85ffb700 1 -- 172.21.15.39:0/1288951073 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e900a2470 con 0x55b4611b76b0 2023-07-21T19:00:12.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.353+0000 7f6eb12ffb80 1 -- 172.21.15.39:0/1288951073 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b4611b76b0 msgr2=0x55b461364940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:00:12.370 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.353+0000 7f6eb12ffb80 1 --2- 172.21.15.39:0/1288951073 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b4611b76b0 0x55b461364940 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6e9000f010 tx=0x7f6e90099a80 comp rx=0 tx=0).stop 2023-07-21T19:00:12.370 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.353+0000 7f6eb12ffb80 1 -- 172.21.15.39:0/1288951073 shutdown_connections 2023-07-21T19:00:12.370 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.353+0000 7f6eb12ffb80 1 --2- 172.21.15.39:0/1288951073 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b46135b9f0 0x55b461369250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:12.370 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.353+0000 7f6eb12ffb80 1 --2- 172.21.15.39:0/1288951073 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b4611b76b0 0x55b461364940 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:12.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.353+0000 7f6eb12ffb80 1 --2- 172.21.15.39:0/1288951073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b46135b040 0x55b46135ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:12.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.353+0000 7f6eb12ffb80 1 -- 172.21.15.39:0/1288951073 >> 172.21.15.39:0/1288951073 conn(0x55b461350be0 msgr2=0x55b461364230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:00:12.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.353+0000 7f6eb12ffb80 1 -- 172.21.15.39:0/1288951073 shutdown_connections 2023-07-21T19:00:12.372 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.353+0000 7f6eb12ffb80 1 -- 172.21.15.39:0/1288951073 wait complete. 2023-07-21T19:00:12.372 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.353+0000 7f6eb12ffb80 1 -- start start 2023-07-21T19:00:12.372 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.353+0000 7f6eb12ffb80 1 Processor -- start 2023-07-21T19:00:12.372 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.353+0000 7f6eb12ffb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4611b76b0 0x55b46136c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:12.373 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.353+0000 7f6eb12ffb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b46135b040 0x55b46136ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:12.373 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.353+0000 7f6eb12ffb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b46135b9f0 0x55b46136d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:12.373 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.353+0000 7f6eb12ffb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4613e9530 con 0x55b4611b76b0 2023-07-21T19:00:12.373 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.353+0000 7f6eb12ffb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4613e9670 con 0x55b46135b9f0 2023-07-21T19:00:12.373 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.353+0000 7f6eb12ffb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4613e97b0 con 0x55b46135b040 2023-07-21T19:00:12.373 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.354+0000 7f6e9ed9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4611b76b0 0x55b46136c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:12.373 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.354+0000 7f6e9e59c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b46135b040 0x55b46136ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:12.373 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.354+0000 7f6eb12ffb80 10 client.0 _ll_register_callbacks cb 0x7f6e94009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:00:12.374 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.354+0000 7f6e9ed9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4611b76b0 0x55b46136c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34002/0 (socket says 172.21.15.39:34002) 2023-07-21T19:00:12.374 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.354+0000 7f6e9e59c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b46135b040 0x55b46136ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:52556/0 (socket says 172.21.15.39:52556) 2023-07-21T19:00:12.374 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.354+0000 7f6e9ed9d700 1 -- 172.21.15.39:0/3164787371 learned_addr learned my addr 172.21.15.39:0/3164787371 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:00:12.374 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.354+0000 7f6e9f59e700 1 --2- 172.21.15.39:0/3164787371 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b46135b9f0 0x55b46136d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:12.374 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.354+0000 7f6e9ed9d700 1 -- 172.21.15.39:0/3164787371 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b46135b040 msgr2=0x55b46136ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:00:12.374 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.354+0000 7f6e9ed9d700 1 --2- 172.21.15.39:0/3164787371 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b46135b040 0x55b46136ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:12.374 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.354+0000 7f6e9ed9d700 1 -- 172.21.15.39:0/3164787371 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b46135b9f0 msgr2=0x55b46136d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:00:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.354+0000 7f6e9ed9d700 1 --2- 172.21.15.39:0/3164787371 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b46135b9f0 0x55b46136d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.354+0000 7f6e9ed9d700 1 -- 172.21.15.39:0/3164787371 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ea010c2e0 con 0x55b4611b76b0 2023-07-21T19:00:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.354+0000 7f6e9e59c700 1 --2- 172.21.15.39:0/3164787371 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b46135b040 0x55b46136ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:00:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.354+0000 7f6e82ff5700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T19:00:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.354+0000 7f6e9ed9d700 1 --2- 172.21.15.39:0/3164787371 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4611b76b0 0x55b46136c950 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f6e9000c5b0 tx=0x7f6e90099e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe0c4849b8a77b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:00:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.354+0000 7f6e82ff5700 1 -- 172.21.15.39:0/3164787371 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e9001f340 con 0x55b4611b76b0 2023-07-21T19:00:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.354+0000 7f6e82ff5700 1 -- 172.21.15.39:0/3164787371 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f6e90094e40 con 0x55b4611b76b0 2023-07-21T19:00:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.354+0000 7f6eb12ffb80 1 -- 172.21.15.39:0/3164787371 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6e900a2680 con 0x55b4611b76b0 2023-07-21T19:00:12.376 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.355+0000 7f6e82ff5700 1 -- 172.21.15.39:0/3164787371 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=98) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e900985d0 con 0x55b4611b76b0 2023-07-21T19:00:12.376 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.355+0000 7f6eb12ffb80 10 client.17541 fetch_fsmap learned FSMap version 98 2023-07-21T19:00:12.376 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.355+0000 7f6eb12ffb80 1 -- 172.21.15.39:0/3164787371 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=98,monmap=4+}) v3 -- 0x55b4610c3c00 con 0x55b4611b76b0 2023-07-21T19:00:12.376 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.355+0000 7f6e82ff5700 1 -- 172.21.15.39:0/3164787371 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e900a2680 con 0x55b4611b76b0 2023-07-21T19:00:12.377 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.356+0000 7f6e82ff5700 1 -- 172.21.15.39:0/3164787371 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 98) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f6e900b8040 con 0x55b4611b76b0 2023-07-21T19:00:12.377 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.356+0000 7f6eb12ffb80 10 client.17541 fetch_fsmap finished waiting for FSMap version 98 2023-07-21T19:00:12.377 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.356+0000 7f6eb12ffb80 10 client.17541 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:00:12.377 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.356+0000 7f6eb12ffb80 20 client.17541 populate_metadata read hostname 'smithi039' 2023-07-21T19:00:12.377 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.356+0000 7f6eb12ffb80 1 --2- 172.21.15.39:0/3164787371 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55b461396a10 0x55b4613ec9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:12.377 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.356+0000 7f6eb12ffb80 4 client.17541 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:00:12.377 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.356+0000 7f6eb12ffb80 1 -- 172.21.15.39:0/3164787371 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b461267ff0 con 0x55b461396a10 2023-07-21T19:00:12.378 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.356+0000 7f6e9e59c700 1 --2- 172.21.15.39:0/3164787371 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55b461396a10 0x55b4613ec9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:12.378 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.356+0000 7f6e82ff5700 10 client.17541 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T19:00:12.378 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.356+0000 7f6e9e59c700 1 --2- 172.21.15.39:0/3164787371 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55b461396a10 0x55b4613ec9e0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c49916f72b04628c server_cookie=a7c4a3111e4b8f31 in_seq=0 out_seq=0 2023-07-21T19:00:12.576 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.565+0000 7f6e82ff5700 1 -- 172.21.15.39:0/3164787371 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b461267ff0 con 0x55b461396a10 2023-07-21T19:00:12.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.565+0000 7f6e82ff5700 10 client.17541 handle_command_reply: tid=0 2023-07-21T19:00:12.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.565+0000 7f6eb12ffb80 1 client.17541 shutdown 2023-07-21T19:00:12.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.565+0000 7f6eb12ffb80 1 -- 172.21.15.39:0/3164787371 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4611b76b0 msgr2=0x55b46136c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:00:12.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.565+0000 7f6eb12ffb80 1 --2- 172.21.15.39:0/3164787371 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4611b76b0 0x55b46136c950 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f6e9000c5b0 tx=0x7f6e90099e60 comp rx=0 tx=0).stop 2023-07-21T19:00:12.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.565+0000 7f6eb12ffb80 1 -- 172.21.15.39:0/3164787371 shutdown_connections 2023-07-21T19:00:12.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.565+0000 7f6eb12ffb80 1 --2- 172.21.15.39:0/3164787371 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55b461396a10 0x55b4613ec9e0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:12.578 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.565+0000 7f6eb12ffb80 1 --2- 172.21.15.39:0/3164787371 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b46135b9f0 0x55b46136d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:12.578 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.565+0000 7f6eb12ffb80 1 --2- 172.21.15.39:0/3164787371 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4611b76b0 0x55b46136c950 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:12.578 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.565+0000 7f6eb12ffb80 1 --2- 172.21.15.39:0/3164787371 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b46135b040 0x55b46136ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:12.578 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.565+0000 7f6eb12ffb80 1 -- 172.21.15.39:0/3164787371 >> 172.21.15.39:0/3164787371 conn(0x55b461350be0 msgr2=0x55b461363140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:00:12.578 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.565+0000 7f6e82ff5700 0 client.17541 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T19:00:12.578 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.565+0000 7f6eb12ffb80 1 -- 172.21.15.39:0/3164787371 shutdown_connections 2023-07-21T19:00:12.578 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.565+0000 7f6eb12ffb80 1 -- 172.21.15.39:0/3164787371 wait complete. 2023-07-21T19:00:12.579 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.565+0000 7f6eb12ffb80 20 client.17541 trim_cache size 0 max 16384 2023-07-21T19:00:12.579 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T19:00:12.579 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T19:00:12.579 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T19:00:12.579 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T19:00:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.566+0000 7f6ea5179700 1 -- 172.21.15.39:0/408123003 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f6e880a12c0 msgr2=0x7f6e880a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:00:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.566+0000 7f6ea5179700 1 --2- 172.21.15.39:0/408123003 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f6e880a12c0 0x7f6e880a3770 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6ea00feb50 tx=0x7f6e940106b0 comp rx=0 tx=0).stop 2023-07-21T19:00:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.566+0000 7f6ea5179700 1 -- 172.21.15.39:0/408123003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ea00ff3b0 msgr2=0x7f6ea0196f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:00:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.566+0000 7f6ea5179700 1 --2- 172.21.15.39:0/408123003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ea00ff3b0 0x7f6ea0196f70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6e9000e9a0 tx=0x7f6e9000ed60 comp rx=0 tx=0).stop 2023-07-21T19:00:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.566+0000 7f6ea5179700 1 -- 172.21.15.39:0/408123003 shutdown_connections 2023-07-21T19:00:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.566+0000 7f6ea5179700 1 --2- 172.21.15.39:0/408123003 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f6e880a12c0 0x7f6e880a3770 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.566+0000 7f6ea5179700 1 --2- 172.21.15.39:0/408123003 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ea0103ed0 0x7f6ea01a1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.566+0000 7f6ea5179700 1 --2- 172.21.15.39:0/408123003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ea00ff3b0 0x7f6ea0196f70 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.566+0000 7f6ea5179700 1 --2- 172.21.15.39:0/408123003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ea00ffd50 0x7f6ea01974b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.566+0000 7f6ea5179700 1 -- 172.21.15.39:0/408123003 >> 172.21.15.39:0/408123003 conn(0x7f6ea0075f90 msgr2=0x7f6ea00fbae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:00:12.582 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.566+0000 7f6ea5179700 1 -- 172.21.15.39:0/408123003 shutdown_connections 2023-07-21T19:00:12.582 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:12.566+0000 7f6ea5179700 1 -- 172.21.15.39:0/408123003 wait complete. 2023-07-21T19:00:12.607 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:00:12 smithi193 ceph-mon[98204]: pgmap v584: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 23 MiB/s wr, 1.08k op/s 2023-07-21T19:00:12.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:00:12 smithi193 ceph-mon[98204]: osdmap e111: 12 total, 12 up, 12 in 2023-07-21T19:00:12.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:00:12 smithi155 ceph-mon[97934]: pgmap v584: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 23 MiB/s wr, 1.08k op/s 2023-07-21T19:00:12.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:00:12 smithi155 ceph-mon[97934]: osdmap e111: 12 total, 12 up, 12 in 2023-07-21T19:00:12.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:00:12 smithi039 ceph-mon[96489]: pgmap v584: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 23 MiB/s wr, 1.08k op/s 2023-07-21T19:00:12.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:00:12 smithi039 ceph-mon[96489]: osdmap e111: 12 total, 12 up, 12 in 2023-07-21T19:00:12.692 INFO:tasks.workunit.client.0.smithi039.stdout: 10 183925 30.69 MB/sec execute 205 sec latency 208.194 ms 2023-07-21T19:00:13.692 INFO:tasks.workunit.client.0.smithi039.stdout: 10 184438 30.68 MB/sec execute 206 sec latency 86.010 ms 2023-07-21T19:00:14.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:00:14 smithi193 ceph-mon[98204]: pgmap v586: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.5 MiB/s wr, 665 op/s 2023-07-21T19:00:14.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:00:14 smithi155 ceph-mon[97934]: pgmap v586: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.5 MiB/s wr, 665 op/s 2023-07-21T19:00:14.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:00:14 smithi039 ceph-mon[96489]: pgmap v586: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.5 MiB/s wr, 665 op/s 2023-07-21T19:00:14.692 INFO:tasks.workunit.client.0.smithi039.stdout: 10 185026 30.65 MB/sec execute 207 sec latency 166.999 ms 2023-07-21T19:00:15.692 INFO:tasks.workunit.client.0.smithi039.stdout: 10 185773 30.72 MB/sec execute 208 sec latency 70.147 ms 2023-07-21T19:00:16.607 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:00:16 smithi193 ceph-mon[98204]: pgmap v587: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 6.9 MiB/s wr, 611 op/s 2023-07-21T19:00:16.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:00:16 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:00:16.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:00:16 smithi155 ceph-mon[97934]: pgmap v587: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 6.9 MiB/s wr, 611 op/s 2023-07-21T19:00:16.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:00:16 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:00:16.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:00:16 smithi039 ceph-mon[96489]: pgmap v587: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 6.9 MiB/s wr, 611 op/s 2023-07-21T19:00:16.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:00:16 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:00:16.692 INFO:tasks.workunit.client.0.smithi039.stdout: 10 186374 30.72 MB/sec execute 209 sec latency 97.230 ms 2023-07-21T19:00:17.692 INFO:tasks.workunit.client.0.smithi039.stdout: 10 186956 30.74 MB/sec execute 210 sec latency 72.359 ms 2023-07-21T19:00:18.607 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:00:18 smithi193 ceph-mon[98204]: pgmap v588: 97 pgs: 97 active+clean; 1.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 8.2 MiB/s wr, 601 op/s 2023-07-21T19:00:18.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:00:18 smithi155 ceph-mon[97934]: pgmap v588: 97 pgs: 97 active+clean; 1.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 8.2 MiB/s wr, 601 op/s 2023-07-21T19:00:18.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:00:18 smithi039 ceph-mon[96489]: pgmap v588: 97 pgs: 97 active+clean; 1.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 8.2 MiB/s wr, 601 op/s 2023-07-21T19:00:18.692 INFO:tasks.workunit.client.0.smithi039.stdout: 10 187609 30.76 MB/sec execute 211 sec latency 70.728 ms 2023-07-21T19:00:19.692 INFO:tasks.workunit.client.0.smithi039.stdout: 10 188076 30.76 MB/sec execute 212 sec latency 126.759 ms 2023-07-21T19:00:20.314 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:00:20 smithi155 ceph-mon[97934]: pgmap v589: 97 pgs: 97 active+clean; 1.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 8.2 MiB/s wr, 601 op/s 2023-07-21T19:00:20.607 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:00:20 smithi193 ceph-mon[98204]: pgmap v589: 97 pgs: 97 active+clean; 1.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 8.2 MiB/s wr, 601 op/s 2023-07-21T19:00:20.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:00:20 smithi039 ceph-mon[96489]: pgmap v589: 97 pgs: 97 active+clean; 1.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 8.2 MiB/s wr, 601 op/s 2023-07-21T19:00:20.692 INFO:tasks.workunit.client.0.smithi039.stdout: 10 188623 30.72 MB/sec execute 213 sec latency 106.220 ms 2023-07-21T19:00:21.607 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:00:21 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:00:21.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:00:21 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:00:21.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:00:21 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T19:00:21.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:00:21 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:00:21.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:00:21 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:00:21.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:00:21 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T19:00:21.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:00:21 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:00:21.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:00:21 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:00:21.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:00:21 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T19:00:21.692 INFO:tasks.workunit.client.0.smithi039.stdout: 10 189251 30.73 MB/sec execute 214 sec latency 67.050 ms 2023-07-21T19:00:22.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:00:22 smithi039 ceph-mon[96489]: pgmap v590: 97 pgs: 97 active+clean; 1.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 8.2 MiB/s wr, 596 op/s 2023-07-21T19:00:22.692 INFO:tasks.workunit.client.0.smithi039.stdout: 10 189916 30.77 MB/sec execute 215 sec latency 111.552 ms 2023-07-21T19:00:22.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:00:22 smithi193 ceph-mon[98204]: pgmap v590: 97 pgs: 97 active+clean; 1.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 8.2 MiB/s wr, 596 op/s 2023-07-21T19:00:22.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:00:22 smithi155 ceph-mon[97934]: pgmap v590: 97 pgs: 97 active+clean; 1.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 8.2 MiB/s wr, 596 op/s 2023-07-21T19:00:23.692 INFO:tasks.workunit.client.0.smithi039.stdout: 10 190501 30.74 MB/sec execute 216 sec latency 84.488 ms 2023-07-21T19:00:24.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:00:24 smithi039 ceph-mon[96489]: pgmap v591: 97 pgs: 97 active+clean; 1.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.4 MiB/s wr, 525 op/s 2023-07-21T19:00:24.692 INFO:tasks.workunit.client.0.smithi039.stdout: 10 191146 30.79 MB/sec execute 217 sec latency 72.453 ms 2023-07-21T19:00:24.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:00:24 smithi193 ceph-mon[98204]: pgmap v591: 97 pgs: 97 active+clean; 1.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.4 MiB/s wr, 525 op/s 2023-07-21T19:00:24.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:00:24 smithi155 ceph-mon[97934]: pgmap v591: 97 pgs: 97 active+clean; 1.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.4 MiB/s wr, 525 op/s 2023-07-21T19:00:25.693 INFO:tasks.workunit.client.0.smithi039.stdout: 10 191756 30.81 MB/sec execute 218 sec latency 82.397 ms 2023-07-21T19:00:25.731 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.721+0000 7faf3971f700 1 -- 172.21.15.39:0/1661032935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf340ffdc0 msgr2=0x7faf34100190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:00:25.731 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.721+0000 7faf3971f700 1 --2- 172.21.15.39:0/1661032935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf340ffdc0 0x7faf34100190 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7faf240099c0 tx=0x7faf24009cd0 comp rx=0 tx=0).stop 2023-07-21T19:00:25.731 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.721+0000 7faf3971f700 1 -- 172.21.15.39:0/1661032935 shutdown_connections 2023-07-21T19:00:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.721+0000 7faf3971f700 1 --2- 172.21.15.39:0/1661032935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf34104e00 0x7faf3410c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.721+0000 7faf3971f700 1 --2- 172.21.15.39:0/1661032935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf340ffdc0 0x7faf34100190 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.721+0000 7faf3971f700 1 --2- 172.21.15.39:0/1661032935 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf34100760 0x7faf34104720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.721+0000 7faf3971f700 1 -- 172.21.15.39:0/1661032935 >> 172.21.15.39:0/1661032935 conn(0x7faf340fb650 msgr2=0x7faf340fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:00:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.722+0000 7faf3971f700 1 -- 172.21.15.39:0/1661032935 shutdown_connections 2023-07-21T19:00:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.723+0000 7faf3971f700 1 -- 172.21.15.39:0/1661032935 wait complete. 2023-07-21T19:00:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.723+0000 7faf3971f700 1 Processor -- start 2023-07-21T19:00:25.734 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.724+0000 7faf3971f700 1 -- start start 2023-07-21T19:00:25.734 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.724+0000 7faf3971f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf340ffdc0 0x7faf3410b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:25.734 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.724+0000 7faf3971f700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf34100760 0x7faf3410bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:25.734 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.724+0000 7faf3971f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf34104e00 0x7faf34199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:25.734 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.724+0000 7faf3971f700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf34199e60 con 0x7faf340ffdc0 2023-07-21T19:00:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.724+0000 7faf3971f700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf34199fd0 con 0x7faf34104e00 2023-07-21T19:00:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.724+0000 7faf3971f700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3419a190 con 0x7faf34100760 2023-07-21T19:00:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.724+0000 7faf32ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf340ffdc0 0x7faf3410b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.724+0000 7faf32ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf340ffdc0 0x7faf3410b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34016/0 (socket says 172.21.15.39:34016) 2023-07-21T19:00:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.724+0000 7faf327fc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf34100760 0x7faf3410bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.724+0000 7faf32ffd700 1 -- 172.21.15.39:0/1108157492 learned_addr learned my addr 172.21.15.39:0/1108157492 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:00:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.724+0000 7faf337fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf34104e00 0x7faf34199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.725+0000 7faf337fe700 1 -- 172.21.15.39:0/1108157492 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf34100760 msgr2=0x7faf3410bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:00:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.725+0000 7faf337fe700 1 --2- 172.21.15.39:0/1108157492 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf34100760 0x7faf3410bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.725+0000 7faf337fe700 1 -- 172.21.15.39:0/1108157492 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf340ffdc0 msgr2=0x7faf3410b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:00:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.725+0000 7faf337fe700 1 --2- 172.21.15.39:0/1108157492 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf340ffdc0 0x7faf3410b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.725+0000 7faf337fe700 1 -- 172.21.15.39:0/1108157492 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf240095f0 con 0x7faf34104e00 2023-07-21T19:00:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.725+0000 7faf32ffd700 1 --2- 172.21.15.39:0/1108157492 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf340ffdc0 0x7faf3410b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:00:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.725+0000 7faf327fc700 1 --2- 172.21.15.39:0/1108157492 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf34100760 0x7faf3410bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:00:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.725+0000 7faf337fe700 1 --2- 172.21.15.39:0/1108157492 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf34104e00 0x7faf34199860 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7faf2c00efc0 tx=0x7faf2c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:00:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.725+0000 7faf1bfff700 1 -- 172.21.15.39:0/1108157492 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2c018070 con 0x7faf34104e00 2023-07-21T19:00:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.725+0000 7faf1bfff700 1 -- 172.21.15.39:0/1108157492 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf2c00f040 con 0x7faf34104e00 2023-07-21T19:00:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.725+0000 7faf3971f700 1 -- 172.21.15.39:0/1108157492 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf3419a410 con 0x7faf34104e00 2023-07-21T19:00:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.725+0000 7faf1bfff700 1 -- 172.21.15.39:0/1108157492 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2c00cd40 con 0x7faf34104e00 2023-07-21T19:00:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.725+0000 7faf3971f700 1 -- 172.21.15.39:0/1108157492 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf34196970 con 0x7faf34104e00 2023-07-21T19:00:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.728+0000 7faf1bfff700 1 -- 172.21.15.39:0/1108157492 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7faf2c003a00 con 0x7faf34104e00 2023-07-21T19:00:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.729+0000 7faf1bfff700 1 --2- 172.21.15.39:0/1108157492 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7faf1c0a12c0 0x7faf1c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.729+0000 7faf32ffd700 1 --2- 172.21.15.39:0/1108157492 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7faf1c0a12c0 0x7faf1c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.729+0000 7faf1bfff700 1 -- 172.21.15.39:0/1108157492 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 8527+0+0 (secure 0 0 0) 0x7faf2c0cc850 con 0x7faf34104e00 2023-07-21T19:00:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.729+0000 7faf458a5b80 1 Processor -- start 2023-07-21T19:00:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.729+0000 7faf458a5b80 1 -- start start 2023-07-21T19:00:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.730+0000 7faf458a5b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bf420b26b0 0x55bf4225bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.730+0000 7faf458a5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bf4225c2d0 0x55bf42256040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.730+0000 7faf458a5b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf42256580 0x55bf422569d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.730+0000 7faf458a5b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf4215bc50 con 0x55bf420b26b0 2023-07-21T19:00:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.730+0000 7faf458a5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf421ae9e0 con 0x55bf4225c2d0 2023-07-21T19:00:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.730+0000 7faf458a5b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf421a7e70 con 0x55bf42256580 2023-07-21T19:00:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.730+0000 7faf327fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bf4225c2d0 0x55bf42256040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.730+0000 7faf327fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bf4225c2d0 0x55bf42256040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:46620/0 (socket says 172.21.15.39:46620) 2023-07-21T19:00:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.730+0000 7faf327fc700 1 -- 172.21.15.39:0/2497036392 learned_addr learned my addr 172.21.15.39:0/2497036392 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:00:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.730+0000 7faf32ffd700 1 --2- 172.21.15.39:0/1108157492 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7faf1c0a12c0 0x7faf1c0a3770 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7faf24004800 tx=0x7faf2401a040 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:00:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.730+0000 7faf337fe700 1 --2- 172.21.15.39:0/2497036392 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf42256580 0x55bf422569d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.730+0000 7faf327fc700 1 -- 172.21.15.39:0/2497036392 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf42256580 msgr2=0x55bf422569d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:00:25.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.730+0000 7faf327fc700 1 --2- 172.21.15.39:0/2497036392 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf42256580 0x55bf422569d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:25.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.730+0000 7faf327fc700 1 -- 172.21.15.39:0/2497036392 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bf420b26b0 msgr2=0x55bf4225bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:00:25.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.730+0000 7faf327fc700 1 --2- 172.21.15.39:0/2497036392 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bf420b26b0 0x55bf4225bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:25.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.730+0000 7faf327fc700 1 -- 172.21.15.39:0/2497036392 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf41fd5d80 con 0x55bf4225c2d0 2023-07-21T19:00:25.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.731+0000 7faf337fe700 1 --2- 172.21.15.39:0/2497036392 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf42256580 0x55bf422569d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:00:25.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.731+0000 7faf327fc700 1 --2- 172.21.15.39:0/2497036392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bf4225c2d0 0x55bf42256040 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7faf280114f0 tx=0x7faf28011800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eee34cbc5dfc6b2a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:00:25.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.731+0000 7faf197fa700 1 -- 172.21.15.39:0/2497036392 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf28004230 con 0x55bf4225c2d0 2023-07-21T19:00:25.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.731+0000 7faf197fa700 1 -- 172.21.15.39:0/2497036392 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf28004390 con 0x55bf4225c2d0 2023-07-21T19:00:25.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.731+0000 7faf197fa700 1 -- 172.21.15.39:0/2497036392 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2800e890 con 0x55bf4225c2d0 2023-07-21T19:00:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.732+0000 7faf458a5b80 1 -- 172.21.15.39:0/2497036392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bf4225c2d0 msgr2=0x55bf42256040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:00:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.732+0000 7faf458a5b80 1 --2- 172.21.15.39:0/2497036392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bf4225c2d0 0x55bf42256040 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7faf280114f0 tx=0x7faf28011800 comp rx=0 tx=0).stop 2023-07-21T19:00:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.732+0000 7faf458a5b80 1 -- 172.21.15.39:0/2497036392 shutdown_connections 2023-07-21T19:00:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.732+0000 7faf458a5b80 1 --2- 172.21.15.39:0/2497036392 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf42256580 0x55bf422569d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.732+0000 7faf458a5b80 1 --2- 172.21.15.39:0/2497036392 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bf420b26b0 0x55bf4225bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.732+0000 7faf458a5b80 1 --2- 172.21.15.39:0/2497036392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bf4225c2d0 0x55bf42256040 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.732+0000 7faf458a5b80 1 -- 172.21.15.39:0/2497036392 >> 172.21.15.39:0/2497036392 conn(0x55bf4224bbe0 msgr2=0x55bf42250f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:00:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.732+0000 7faf458a5b80 1 -- 172.21.15.39:0/2497036392 shutdown_connections 2023-07-21T19:00:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.732+0000 7faf458a5b80 1 -- 172.21.15.39:0/2497036392 wait complete. 2023-07-21T19:00:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.732+0000 7faf458a5b80 1 -- start start 2023-07-21T19:00:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.732+0000 7faf458a5b80 1 Processor -- start 2023-07-21T19:00:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.733+0000 7faf458a5b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf420b26b0 0x55bf42262650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.733+0000 7faf458a5b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bf42256580 0x55bf4225f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.733+0000 7faf458a5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bf4225c2d0 0x55bf4225ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.733+0000 7faf458a5b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf42147a60 con 0x55bf42256580 2023-07-21T19:00:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.733+0000 7faf458a5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf42147760 con 0x55bf4225c2d0 2023-07-21T19:00:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.733+0000 7faf458a5b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf4202b8c0 con 0x55bf420b26b0 2023-07-21T19:00:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.733+0000 7faf458a5b80 10 client.0 _ll_register_callbacks cb 0x55bf42145700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:00:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.733+0000 7faf32ffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf420b26b0 0x55bf42262650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.733+0000 7faf337fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bf4225c2d0 0x55bf4225ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.733+0000 7faf32ffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf420b26b0 0x55bf42262650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:52582/0 (socket says 172.21.15.39:52582) 2023-07-21T19:00:25.751 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.733+0000 7faf327fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bf42256580 0x55bf4225f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:25.751 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.733+0000 7faf32ffd700 1 -- 172.21.15.39:0/3055978415 learned_addr learned my addr 172.21.15.39:0/3055978415 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:00:25.751 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.733+0000 7faf32ffd700 1 -- 172.21.15.39:0/3055978415 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bf4225c2d0 msgr2=0x55bf4225ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:00:25.751 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.733+0000 7faf32ffd700 1 --2- 172.21.15.39:0/3055978415 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bf4225c2d0 0x55bf4225ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:25.751 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.733+0000 7faf32ffd700 1 -- 172.21.15.39:0/3055978415 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bf42256580 msgr2=0x55bf4225f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:00:25.751 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.733+0000 7faf32ffd700 1 --2- 172.21.15.39:0/3055978415 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bf42256580 0x55bf4225f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.733+0000 7faf32ffd700 1 -- 172.21.15.39:0/3055978415 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf28009cd0 con 0x55bf420b26b0 2023-07-21T19:00:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.734+0000 7faf32ffd700 1 --2- 172.21.15.39:0/3055978415 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf420b26b0 0x55bf42262650 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7faf24024770 tx=0x7faf24005fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=557e9ef881f47ba0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:00:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.734+0000 7faf16ff5700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2023-07-21T19:00:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.734+0000 7faf16ff5700 1 -- 172.21.15.39:0/3055978415 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2400f460 con 0x55bf420b26b0 2023-07-21T19:00:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.734+0000 7faf458a5b80 1 -- 172.21.15.39:0/3055978415 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bf42101490 con 0x55bf420b26b0 2023-07-21T19:00:25.753 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.734+0000 7faf16ff5700 1 -- 172.21.15.39:0/3055978415 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8527+0+0 (secure 0 0 0) 0x7faf2400f5c0 con 0x55bf420b26b0 2023-07-21T19:00:25.753 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.734+0000 7faf16ff5700 1 -- 172.21.15.39:0/3055978415 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=98) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf2400f780 con 0x55bf420b26b0 2023-07-21T19:00:25.753 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.734+0000 7faf458a5b80 10 client.27013 fetch_fsmap learned FSMap version 98 2023-07-21T19:00:25.753 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.734+0000 7faf458a5b80 1 -- 172.21.15.39:0/3055978415 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=98,monmap=4+}) v3 -- 0x55bf41fbec00 con 0x55bf420b26b0 2023-07-21T19:00:25.753 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.734+0000 7faf16ff5700 1 -- 172.21.15.39:0/3055978415 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf2403b020 con 0x55bf420b26b0 2023-07-21T19:00:25.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.735+0000 7faf16ff5700 1 -- 172.21.15.39:0/3055978415 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 98) v1 ==== 7466+0+0 (secure 0 0 0) 0x7faf2402d070 con 0x55bf420b26b0 2023-07-21T19:00:25.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.735+0000 7faf458a5b80 10 client.27013 fetch_fsmap finished waiting for FSMap version 98 2023-07-21T19:00:25.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.735+0000 7faf458a5b80 10 client.27013 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:00:25.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.735+0000 7faf458a5b80 20 client.27013 populate_metadata read hostname 'smithi039' 2023-07-21T19:00:25.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.735+0000 7faf458a5b80 1 --2- 172.21.15.39:0/3055978415 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55bf4226fb30 0x55bf42271f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:25.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.735+0000 7faf458a5b80 4 client.27013 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:00:25.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.735+0000 7faf458a5b80 1 -- 172.21.15.39:0/3055978415 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bf421a0e50 con 0x55bf4226fb30 2023-07-21T19:00:25.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.735+0000 7faf327fc700 1 --2- 172.21.15.39:0/3055978415 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55bf4226fb30 0x55bf42271f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:25.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.736+0000 7faf327fc700 1 --2- 172.21.15.39:0/3055978415 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55bf4226fb30 0x55bf42271f40 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=496dad90d9866f2f server_cookie=a6d57414741cc3bf in_seq=0 out_seq=0 2023-07-21T19:00:25.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.736+0000 7faf16ff5700 10 client.27013 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T19:00:25.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.737+0000 7faf16ff5700 1 -- 172.21.15.39:0/3055978415 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55bf421a0e50 con 0x55bf4226fb30 2023-07-21T19:00:25.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.737+0000 7faf16ff5700 10 client.27013 handle_command_reply: tid=0 2023-07-21T19:00:25.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.737+0000 7faf458a5b80 1 client.27013 shutdown 2023-07-21T19:00:25.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.737+0000 7faf458a5b80 1 -- 172.21.15.39:0/3055978415 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf420b26b0 msgr2=0x55bf42262650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:00:25.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.737+0000 7faf458a5b80 1 --2- 172.21.15.39:0/3055978415 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf420b26b0 0x55bf42262650 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7faf24024770 tx=0x7faf24005fb0 comp rx=0 tx=0).stop 2023-07-21T19:00:25.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.738+0000 7faf458a5b80 1 -- 172.21.15.39:0/3055978415 shutdown_connections 2023-07-21T19:00:25.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.738+0000 7faf458a5b80 1 --2- 172.21.15.39:0/3055978415 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55bf4226fb30 0x55bf42271f40 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:25.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.738+0000 7faf458a5b80 1 --2- 172.21.15.39:0/3055978415 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bf4225c2d0 0x55bf4225ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:25.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.738+0000 7faf16ff5700 0 client.27013 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T19:00:25.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.738+0000 7faf458a5b80 1 --2- 172.21.15.39:0/3055978415 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf420b26b0 0x55bf42262650 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:25.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.738+0000 7faf458a5b80 1 --2- 172.21.15.39:0/3055978415 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bf42256580 0x55bf4225f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.738+0000 7faf458a5b80 1 -- 172.21.15.39:0/3055978415 >> 172.21.15.39:0/3055978415 conn(0x55bf4224bbe0 msgr2=0x55bf4224fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:00:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.738+0000 7faf458a5b80 1 -- 172.21.15.39:0/3055978415 shutdown_connections 2023-07-21T19:00:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.738+0000 7faf458a5b80 1 -- 172.21.15.39:0/3055978415 wait complete. 2023-07-21T19:00:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.739+0000 7faf458a5b80 20 client.27013 trim_cache size 0 max 16384 2023-07-21T19:00:25.773 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.762+0000 7faf458a5b80 1 Processor -- start 2023-07-21T19:00:25.773 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.762+0000 7faf458a5b80 1 -- start start 2023-07-21T19:00:25.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.763+0000 7faf458a5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bf420b26b0 0x55bf42256040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:25.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.763+0000 7faf458a5b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bf42256580 0x55bf4225f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:25.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.763+0000 7faf458a5b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf4225c2d0 0x55bf42266c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:25.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.763+0000 7faf458a5b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf4225c6a0 con 0x55bf42256580 2023-07-21T19:00:25.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.763+0000 7faf458a5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf42251a00 con 0x55bf420b26b0 2023-07-21T19:00:25.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.763+0000 7faf458a5b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf4224d5f0 con 0x55bf4225c2d0 2023-07-21T19:00:25.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.763+0000 7faf327fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bf42256580 0x55bf4225f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:25.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.763+0000 7faf327fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bf42256580 0x55bf4225f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34040/0 (socket says 172.21.15.39:34040) 2023-07-21T19:00:25.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.763+0000 7faf327fc700 1 -- 172.21.15.39:0/2633875376 learned_addr learned my addr 172.21.15.39:0/2633875376 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:00:25.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.763+0000 7faf337fe700 1 --2- 172.21.15.39:0/2633875376 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf4225c2d0 0x55bf42266c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:25.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.763+0000 7faf327fc700 1 -- 172.21.15.39:0/2633875376 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf4225c2d0 msgr2=0x55bf42266c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:00:25.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.763+0000 7faf327fc700 1 --2- 172.21.15.39:0/2633875376 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf4225c2d0 0x55bf42266c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:25.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.763+0000 7faf327fc700 1 -- 172.21.15.39:0/2633875376 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bf420b26b0 msgr2=0x55bf42256040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:00:25.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.763+0000 7faf327fc700 1 --2- 172.21.15.39:0/2633875376 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bf420b26b0 0x55bf42256040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:25.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.763+0000 7faf327fc700 1 -- 172.21.15.39:0/2633875376 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf41ffc1f0 con 0x55bf42256580 2023-07-21T19:00:25.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.763+0000 7faf327fc700 1 --2- 172.21.15.39:0/2633875376 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bf42256580 0x55bf4225f810 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x55bf42252b40 tx=0x7faf28011800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2d1f2687fbe3fe9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:00:25.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.764+0000 7faf19ffb700 1 -- 172.21.15.39:0/2633875376 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2802b570 con 0x55bf42256580 2023-07-21T19:00:25.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.764+0000 7faf19ffb700 1 -- 172.21.15.39:0/2633875376 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf2802bcb0 con 0x55bf42256580 2023-07-21T19:00:25.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.764+0000 7faf19ffb700 1 -- 172.21.15.39:0/2633875376 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf28005600 con 0x55bf42256580 2023-07-21T19:00:25.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.765+0000 7faf458a5b80 1 -- 172.21.15.39:0/2633875376 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bf42256580 msgr2=0x55bf4225f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:00:25.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.765+0000 7faf458a5b80 1 --2- 172.21.15.39:0/2633875376 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bf42256580 0x55bf4225f810 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x55bf42252b40 tx=0x7faf28011800 comp rx=0 tx=0).stop 2023-07-21T19:00:25.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.766+0000 7faf458a5b80 1 -- 172.21.15.39:0/2633875376 shutdown_connections 2023-07-21T19:00:25.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.766+0000 7faf458a5b80 1 --2- 172.21.15.39:0/2633875376 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bf420b26b0 0x55bf42256040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.766+0000 7faf458a5b80 1 --2- 172.21.15.39:0/2633875376 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf4225c2d0 0x55bf42266c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.766+0000 7faf458a5b80 1 --2- 172.21.15.39:0/2633875376 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bf42256580 0x55bf4225f810 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.766+0000 7faf458a5b80 1 -- 172.21.15.39:0/2633875376 >> 172.21.15.39:0/2633875376 conn(0x55bf4224bbe0 msgr2=0x55bf4225f100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:00:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.766+0000 7faf458a5b80 1 -- 172.21.15.39:0/2633875376 shutdown_connections 2023-07-21T19:00:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.766+0000 7faf458a5b80 1 -- 172.21.15.39:0/2633875376 wait complete. 2023-07-21T19:00:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.766+0000 7faf458a5b80 1 -- start start 2023-07-21T19:00:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.766+0000 7faf458a5b80 1 Processor -- start 2023-07-21T19:00:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.767+0000 7faf458a5b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf420b26b0 0x55bf422bce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.767+0000 7faf458a5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bf42256580 0x55bf422bd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.767+0000 7faf458a5b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bf4225c2d0 0x55bf422b7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.767+0000 7faf458a5b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf422b7f20 con 0x55bf4225c2d0 2023-07-21T19:00:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.767+0000 7faf458a5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf422b8090 con 0x55bf42256580 2023-07-21T19:00:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.767+0000 7faf458a5b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf422b8250 con 0x55bf420b26b0 2023-07-21T19:00:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.767+0000 7faf458a5b80 10 client.0 _ll_register_callbacks cb 0x7faf2400bee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:00:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.767+0000 7faf32ffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf420b26b0 0x55bf422bce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.767+0000 7faf32ffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf420b26b0 0x55bf422bce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:52596/0 (socket says 172.21.15.39:52596) 2023-07-21T19:00:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.767+0000 7faf32ffd700 1 -- 172.21.15.39:0/1273100566 learned_addr learned my addr 172.21.15.39:0/1273100566 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:00:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.767+0000 7faf327fc700 1 --2- 172.21.15.39:0/1273100566 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bf42256580 0x55bf422bd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.767+0000 7faf337fe700 1 --2- 172.21.15.39:0/1273100566 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bf4225c2d0 0x55bf422b7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.767+0000 7faf32ffd700 1 -- 172.21.15.39:0/1273100566 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bf42256580 msgr2=0x55bf422bd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:00:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.767+0000 7faf32ffd700 1 --2- 172.21.15.39:0/1273100566 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bf42256580 0x55bf422bd3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.767+0000 7faf32ffd700 1 -- 172.21.15.39:0/1273100566 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bf4225c2d0 msgr2=0x55bf422b7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:00:25.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.767+0000 7faf32ffd700 1 --2- 172.21.15.39:0/1273100566 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bf4225c2d0 0x55bf422b7940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:25.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.767+0000 7faf32ffd700 1 -- 172.21.15.39:0/1273100566 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bf41fd5d80 con 0x55bf420b26b0 2023-07-21T19:00:25.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.767+0000 7faf327fc700 1 --2- 172.21.15.39:0/1273100566 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bf42256580 0x55bf422bd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:00:25.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.768+0000 7faf337fe700 1 --2- 172.21.15.39:0/1273100566 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bf4225c2d0 0x55bf422b7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:00:25.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.768+0000 7faf32ffd700 1 --2- 172.21.15.39:0/1273100566 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf420b26b0 0x55bf422bce90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7faf2402ea70 tx=0x7faf240059e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50e4cb88f64b4a2d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:00:25.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.768+0000 7faf16ff5700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2023-07-21T19:00:25.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.768+0000 7faf16ff5700 1 -- 172.21.15.39:0/1273100566 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf24017950 con 0x55bf420b26b0 2023-07-21T19:00:25.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.768+0000 7faf16ff5700 1 -- 172.21.15.39:0/1273100566 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8527+0+0 (secure 0 0 0) 0x7faf2403c080 con 0x55bf420b26b0 2023-07-21T19:00:25.785 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.768+0000 7faf458a5b80 1 -- 172.21.15.39:0/1273100566 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faf28004390 con 0x55bf420b26b0 2023-07-21T19:00:25.785 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.769+0000 7faf16ff5700 1 -- 172.21.15.39:0/1273100566 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=98) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf24017ab0 con 0x55bf420b26b0 2023-07-21T19:00:25.785 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.769+0000 7faf458a5b80 10 client.27019 fetch_fsmap learned FSMap version 98 2023-07-21T19:00:25.785 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.769+0000 7faf458a5b80 1 -- 172.21.15.39:0/1273100566 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=98,monmap=4+}) v3 -- 0x55bf41fd4250 con 0x55bf420b26b0 2023-07-21T19:00:25.785 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.769+0000 7faf16ff5700 1 -- 172.21.15.39:0/1273100566 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf28004390 con 0x55bf420b26b0 2023-07-21T19:00:25.785 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.769+0000 7faf16ff5700 1 -- 172.21.15.39:0/1273100566 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 98) v1 ==== 7466+0+0 (secure 0 0 0) 0x7faf24021070 con 0x55bf420b26b0 2023-07-21T19:00:25.786 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.769+0000 7faf458a5b80 10 client.27019 fetch_fsmap finished waiting for FSMap version 98 2023-07-21T19:00:25.786 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.769+0000 7faf458a5b80 10 client.27019 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:00:25.786 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.769+0000 7faf458a5b80 20 client.27019 populate_metadata read hostname 'smithi039' 2023-07-21T19:00:25.786 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.769+0000 7faf458a5b80 1 --2- 172.21.15.39:0/1273100566 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55bf422c48f0 0x55bf422c6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:25.786 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.769+0000 7faf458a5b80 4 client.27019 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:00:25.787 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.769+0000 7faf458a5b80 1 -- 172.21.15.39:0/1273100566 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55bf42162ff0 con 0x55bf422c48f0 2023-07-21T19:00:25.787 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.769+0000 7faf327fc700 1 --2- 172.21.15.39:0/1273100566 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55bf422c48f0 0x55bf422c6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:25.788 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.772+0000 7faf327fc700 1 --2- 172.21.15.39:0/1273100566 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55bf422c48f0 0x55bf422c6d00 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=896dfdcd02ad9cc5 server_cookie=4082dc4b1c8f5773 in_seq=0 out_seq=0 2023-07-21T19:00:25.788 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:25.774+0000 7faf16ff5700 10 client.27019 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T19:00:26.037 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:26.027+0000 7faf16ff5700 1 -- 172.21.15.39:0/1273100566 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55bf42162ff0 con 0x55bf422c48f0 2023-07-21T19:00:26.037 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:26.027+0000 7faf16ff5700 10 client.27019 handle_command_reply: tid=0 2023-07-21T19:00:26.038 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:26.027+0000 7faf458a5b80 1 client.27019 shutdown 2023-07-21T19:00:26.038 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:26.029+0000 7faf458a5b80 1 -- 172.21.15.39:0/1273100566 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf420b26b0 msgr2=0x55bf422bce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:00:26.038 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:26.029+0000 7faf458a5b80 1 --2- 172.21.15.39:0/1273100566 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf420b26b0 0x55bf422bce90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7faf2402ea70 tx=0x7faf240059e0 comp rx=0 tx=0).stop 2023-07-21T19:00:26.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:26.033+0000 7faf458a5b80 1 -- 172.21.15.39:0/1273100566 shutdown_connections 2023-07-21T19:00:26.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:26.033+0000 7faf458a5b80 1 --2- 172.21.15.39:0/1273100566 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55bf422c48f0 0x55bf422c6d00 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:26.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:26.033+0000 7faf458a5b80 1 --2- 172.21.15.39:0/1273100566 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bf4225c2d0 0x55bf422b7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:26.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:26.033+0000 7faf458a5b80 1 --2- 172.21.15.39:0/1273100566 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf420b26b0 0x55bf422bce90 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:26.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:26.033+0000 7faf458a5b80 1 --2- 172.21.15.39:0/1273100566 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bf42256580 0x55bf422bd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:26.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:26.033+0000 7faf458a5b80 1 -- 172.21.15.39:0/1273100566 >> 172.21.15.39:0/1273100566 conn(0x55bf4224bbe0 msgr2=0x55bf4225aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:00:26.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:26.033+0000 7faf16ff5700 0 client.27019 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T19:00:26.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:26.033+0000 7faf458a5b80 1 -- 172.21.15.39:0/1273100566 shutdown_connections 2023-07-21T19:00:26.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:26.033+0000 7faf458a5b80 1 -- 172.21.15.39:0/1273100566 wait complete. 2023-07-21T19:00:26.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:26.033+0000 7faf458a5b80 20 client.27019 trim_cache size 0 max 16384 2023-07-21T19:00:26.045 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T19:00:26.045 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T19:00:26.045 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T19:00:26.045 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T19:00:26.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:26.035+0000 7faf3971f700 1 -- 172.21.15.39:0/1108157492 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7faf1c0a12c0 msgr2=0x7faf1c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:00:26.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:26.035+0000 7faf3971f700 1 --2- 172.21.15.39:0/1108157492 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7faf1c0a12c0 0x7faf1c0a3770 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7faf24004800 tx=0x7faf2401a040 comp rx=0 tx=0).stop 2023-07-21T19:00:26.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:26.035+0000 7faf3971f700 1 -- 172.21.15.39:0/1108157492 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf34104e00 msgr2=0x7faf34199860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:00:26.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:26.035+0000 7faf3971f700 1 --2- 172.21.15.39:0/1108157492 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf34104e00 0x7faf34199860 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7faf2c00efc0 tx=0x7faf2c00c3a0 comp rx=0 tx=0).stop 2023-07-21T19:00:26.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:26.035+0000 7faf3971f700 1 -- 172.21.15.39:0/1108157492 shutdown_connections 2023-07-21T19:00:26.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:26.035+0000 7faf3971f700 1 --2- 172.21.15.39:0/1108157492 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7faf1c0a12c0 0x7faf1c0a3770 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:26.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:26.035+0000 7faf3971f700 1 --2- 172.21.15.39:0/1108157492 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf34104e00 0x7faf34199860 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:26.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:26.035+0000 7faf3971f700 1 --2- 172.21.15.39:0/1108157492 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf340ffdc0 0x7faf3410b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:26.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:26.035+0000 7faf3971f700 1 --2- 172.21.15.39:0/1108157492 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf34100760 0x7faf3410bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:26.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:26.035+0000 7faf3971f700 1 -- 172.21.15.39:0/1108157492 >> 172.21.15.39:0/1108157492 conn(0x7faf340fb650 msgr2=0x7faf340fd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:00:26.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:26.035+0000 7faf3971f700 1 -- 172.21.15.39:0/1108157492 shutdown_connections 2023-07-21T19:00:26.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:26.035+0000 7faf3971f700 1 -- 172.21.15.39:0/1108157492 wait complete. 2023-07-21T19:00:26.693 INFO:tasks.workunit.client.0.smithi039.stdout: 10 192314 30.78 MB/sec execute 219 sec latency 206.657 ms 2023-07-21T19:00:26.859 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:00:26 smithi193 ceph-mon[98204]: pgmap v592: 97 pgs: 97 active+clean; 1.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 6.9 MiB/s wr, 494 op/s 2023-07-21T19:00:26.878 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:00:26 smithi155 ceph-mon[97934]: pgmap v592: 97 pgs: 97 active+clean; 1.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 6.9 MiB/s wr, 494 op/s 2023-07-21T19:00:26.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:00:26 smithi039 ceph-mon[96489]: pgmap v592: 97 pgs: 97 active+clean; 1.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 6.9 MiB/s wr, 494 op/s 2023-07-21T19:00:27.693 INFO:tasks.workunit.client.0.smithi039.stdout: 10 192955 30.84 MB/sec execute 220 sec latency 78.361 ms 2023-07-21T19:00:28.693 INFO:tasks.workunit.client.0.smithi039.stdout: 10 193530 30.80 MB/sec execute 221 sec latency 81.467 ms 2023-07-21T19:00:28.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:00:28 smithi193 ceph-mon[98204]: pgmap v593: 97 pgs: 97 active+clean; 1.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 9.9 MiB/s wr, 729 op/s 2023-07-21T19:00:28.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:00:28 smithi155 ceph-mon[97934]: pgmap v593: 97 pgs: 97 active+clean; 1.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 9.9 MiB/s wr, 729 op/s 2023-07-21T19:00:28.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:00:28 smithi039 ceph-mon[96489]: pgmap v593: 97 pgs: 97 active+clean; 1.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 9.9 MiB/s wr, 729 op/s 2023-07-21T19:00:29.693 INFO:tasks.workunit.client.0.smithi039.stdout: 10 194154 30.83 MB/sec execute 222 sec latency 65.215 ms 2023-07-21T19:00:30.693 INFO:tasks.workunit.client.0.smithi039.stdout: 10 194872 30.84 MB/sec execute 223 sec latency 67.667 ms 2023-07-21T19:00:30.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:00:30 smithi193 ceph-mon[98204]: pgmap v594: 97 pgs: 97 active+clean; 1.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.5 MiB/s wr, 451 op/s 2023-07-21T19:00:30.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:00:30 smithi155 ceph-mon[97934]: pgmap v594: 97 pgs: 97 active+clean; 1.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.5 MiB/s wr, 451 op/s 2023-07-21T19:00:30.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:00:30 smithi039 ceph-mon[96489]: pgmap v594: 97 pgs: 97 active+clean; 1.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.5 MiB/s wr, 451 op/s 2023-07-21T19:00:31.693 INFO:tasks.workunit.client.0.smithi039.stdout: 10 195490 30.87 MB/sec execute 224 sec latency 87.146 ms 2023-07-21T19:00:32.693 INFO:tasks.workunit.client.0.smithi039.stdout: 10 196093 30.90 MB/sec execute 225 sec latency 161.396 ms 2023-07-21T19:00:32.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:00:32 smithi193 ceph-mon[98204]: pgmap v595: 97 pgs: 97 active+clean; 1.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 6.9 MiB/s wr, 485 op/s 2023-07-21T19:00:32.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:00:32 smithi155 ceph-mon[97934]: pgmap v595: 97 pgs: 97 active+clean; 1.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 6.9 MiB/s wr, 485 op/s 2023-07-21T19:00:32.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:00:32 smithi039 ceph-mon[96489]: pgmap v595: 97 pgs: 97 active+clean; 1.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 6.9 MiB/s wr, 485 op/s 2023-07-21T19:00:33.693 INFO:tasks.workunit.client.0.smithi039.stdout: 10 196596 30.87 MB/sec execute 226 sec latency 137.183 ms 2023-07-21T19:00:34.693 INFO:tasks.workunit.client.0.smithi039.stdout: 10 197126 30.86 MB/sec execute 227 sec latency 63.315 ms 2023-07-21T19:00:34.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:00:34 smithi193 ceph-mon[98204]: pgmap v596: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 9.8 MiB/s wr, 702 op/s 2023-07-21T19:00:34.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:00:34 smithi155 ceph-mon[97934]: pgmap v596: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 9.8 MiB/s wr, 702 op/s 2023-07-21T19:00:34.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:00:34 smithi039 ceph-mon[96489]: pgmap v596: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 9.8 MiB/s wr, 702 op/s 2023-07-21T19:00:35.693 INFO:tasks.workunit.client.0.smithi039.stdout: 10 197706 30.83 MB/sec execute 228 sec latency 89.473 ms 2023-07-21T19:00:36.693 INFO:tasks.workunit.client.0.smithi039.stdout: 10 198319 30.86 MB/sec execute 229 sec latency 102.138 ms 2023-07-21T19:00:36.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:00:36 smithi193 ceph-mon[98204]: pgmap v597: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.3 MiB/s wr, 492 op/s 2023-07-21T19:00:36.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:00:36 smithi155 ceph-mon[97934]: pgmap v597: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.3 MiB/s wr, 492 op/s 2023-07-21T19:00:36.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:00:36 smithi039 ceph-mon[96489]: pgmap v597: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.3 MiB/s wr, 492 op/s 2023-07-21T19:00:37.693 INFO:tasks.workunit.client.0.smithi039.stdout: 10 198815 30.84 MB/sec execute 230 sec latency 349.083 ms 2023-07-21T19:00:38.693 INFO:tasks.workunit.client.0.smithi039.stdout: 10 198831 30.71 MB/sec execute 231 sec latency 1139.431 ms 2023-07-21T19:00:38.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:00:38 smithi193 ceph-mon[98204]: pgmap v598: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 11 MiB/s wr, 744 op/s 2023-07-21T19:00:38.878 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:00:38 smithi155 ceph-mon[97934]: pgmap v598: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 11 MiB/s wr, 744 op/s 2023-07-21T19:00:38.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:00:38 smithi039 ceph-mon[96489]: pgmap v598: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 11 MiB/s wr, 744 op/s 2023-07-21T19:00:39.237 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.227+0000 7fd1f81cb700 1 -- 172.21.15.39:0/988045831 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1f00febc0 msgr2=0x7fd1f00fef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:00:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.227+0000 7fd1f81cb700 1 --2- 172.21.15.39:0/988045831 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1f00febc0 0x7fd1f00fef90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd1e001a9d0 tx=0x7fd1e001ace0 comp rx=0 tx=0).stop 2023-07-21T19:00:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.228+0000 7fd1f81cb700 1 -- 172.21.15.39:0/988045831 shutdown_connections 2023-07-21T19:00:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.228+0000 7fd1f81cb700 1 --2- 172.21.15.39:0/988045831 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1f0103fc0 0x7fd1f0104460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.228+0000 7fd1f81cb700 1 --2- 172.21.15.39:0/988045831 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1f00febc0 0x7fd1f00fef90 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.228+0000 7fd1f81cb700 1 --2- 172.21.15.39:0/988045831 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd1f00ff4d0 0x7fd1f01039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.228+0000 7fd1f81cb700 1 -- 172.21.15.39:0/988045831 >> 172.21.15.39:0/988045831 conn(0x7fd1f0076a10 msgr2=0x7fd1f0076e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:00:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.230+0000 7fd1f81cb700 1 -- 172.21.15.39:0/988045831 shutdown_connections 2023-07-21T19:00:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.230+0000 7fd1f81cb700 1 -- 172.21.15.39:0/988045831 wait complete. 2023-07-21T19:00:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.231+0000 7fd1f81cb700 1 Processor -- start 2023-07-21T19:00:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.231+0000 7fd1f81cb700 1 -- start start 2023-07-21T19:00:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.231+0000 7fd1f81cb700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1f00febc0 0x7fd1f0199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.231+0000 7fd1f81cb700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1f00ff4d0 0x7fd1f019a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.231+0000 7fd1f81cb700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd1f0103fc0 0x7fd1f010e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.231+0000 7fd1f81cb700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f010f0e0 con 0x7fd1f00febc0 2023-07-21T19:00:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.231+0000 7fd1f81cb700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f010f250 con 0x7fd1f0103fc0 2023-07-21T19:00:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.231+0000 7fd1f81cb700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f010f410 con 0x7fd1f00ff4d0 2023-07-21T19:00:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.231+0000 7fd1f6768700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd1f0103fc0 0x7fd1f010e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:39.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.231+0000 7fd1f6768700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd1f0103fc0 0x7fd1f010e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:46656/0 (socket says 172.21.15.39:46656) 2023-07-21T19:00:39.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.231+0000 7fd1f6768700 1 -- 172.21.15.39:0/3550531698 learned_addr learned my addr 172.21.15.39:0/3550531698 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:00:39.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.232+0000 7fd1f6768700 1 -- 172.21.15.39:0/3550531698 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1f00ff4d0 msgr2=0x7fd1f019a160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-21T19:00:39.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.232+0000 7fd1f6768700 1 --2- 172.21.15.39:0/3550531698 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1f00ff4d0 0x7fd1f019a160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:39.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.232+0000 7fd1f6768700 1 -- 172.21.15.39:0/3550531698 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1f00febc0 msgr2=0x7fd1f0199c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:00:39.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.232+0000 7fd1f6768700 1 --2- 172.21.15.39:0/3550531698 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1f00febc0 0x7fd1f0199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:39.245 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.232+0000 7fd1f6768700 1 -- 172.21.15.39:0/3550531698 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1e001a600 con 0x7fd1f0103fc0 2023-07-21T19:00:39.245 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.232+0000 7fd1f6768700 1 --2- 172.21.15.39:0/3550531698 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd1f0103fc0 0x7fd1f010e910 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd1e802d9e0 tx=0x7fd1e802dda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:00:39.245 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.232+0000 7fd1e6ffd700 1 -- 172.21.15.39:0/3550531698 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e802e950 con 0x7fd1f0103fc0 2023-07-21T19:00:39.245 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.232+0000 7fd1e6ffd700 1 -- 172.21.15.39:0/3550531698 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1e8036440 con 0x7fd1f0103fc0 2023-07-21T19:00:39.245 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.232+0000 7fd1e6ffd700 1 -- 172.21.15.39:0/3550531698 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e80295d0 con 0x7fd1f0103fc0 2023-07-21T19:00:39.246 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.233+0000 7fd1f81cb700 1 -- 172.21.15.39:0/3550531698 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1f010f6f0 con 0x7fd1f0103fc0 2023-07-21T19:00:39.246 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.233+0000 7fd1f81cb700 1 -- 172.21.15.39:0/3550531698 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1f01a0ba0 con 0x7fd1f0103fc0 2023-07-21T19:00:39.248 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.237+0000 7fd1e6ffd700 1 -- 172.21.15.39:0/3550531698 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fd1e8036870 con 0x7fd1f0103fc0 2023-07-21T19:00:39.248 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.238+0000 7fd1e6ffd700 1 --2- 172.21.15.39:0/3550531698 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fd1dc0a1340 0x7fd1dc0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:39.248 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.238+0000 7fd1e6ffd700 1 -- 172.21.15.39:0/3550531698 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fd1e80ee0b0 con 0x7fd1f0103fc0 2023-07-21T19:00:39.248 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.238+0000 7fd1f5f67700 1 --2- 172.21.15.39:0/3550531698 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fd1dc0a1340 0x7fd1dc0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:39.249 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.238+0000 7fd204351b80 1 Processor -- start 2023-07-21T19:00:39.250 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.238+0000 7fd1f5f67700 1 --2- 172.21.15.39:0/3550531698 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fd1dc0a1340 0x7fd1dc0a37f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd1e001a8f0 tx=0x7fd1e0000bc0 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:00:39.250 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.239+0000 7fd204351b80 1 -- start start 2023-07-21T19:00:39.250 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.239+0000 7fd204351b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c9fdd6a6b0 0x55c9fdf13d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:39.250 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.239+0000 7fd204351b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9fdf142d0 0x55c9fdf0e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:39.250 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.239+0000 7fd204351b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9fdf0e580 0x55c9fdf0e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:39.250 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.239+0000 7fd204351b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9fde13c50 con 0x55c9fdd6a6b0 2023-07-21T19:00:39.250 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.239+0000 7fd204351b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9fde669e0 con 0x55c9fdf142d0 2023-07-21T19:00:39.251 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.239+0000 7fd204351b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9fde5fe70 con 0x55c9fdf0e580 2023-07-21T19:00:39.251 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.239+0000 7fd1f5766700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9fdf142d0 0x55c9fdf0e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:39.251 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.239+0000 7fd1f5f67700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c9fdd6a6b0 0x55c9fdf13d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:39.251 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.239+0000 7fd1f5766700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9fdf142d0 0x55c9fdf0e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:46664/0 (socket says 172.21.15.39:46664) 2023-07-21T19:00:39.251 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.239+0000 7fd1f5766700 1 -- 172.21.15.39:0/1743363471 learned_addr learned my addr 172.21.15.39:0/1743363471 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:00:39.251 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.239+0000 7fd1f5f67700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c9fdd6a6b0 0x55c9fdf13d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34064/0 (socket says 172.21.15.39:34064) 2023-07-21T19:00:39.251 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.239+0000 7fd1f6768700 1 --2- 172.21.15.39:0/1743363471 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9fdf0e580 0x55c9fdf0e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:39.252 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.240+0000 7fd1f5766700 1 -- 172.21.15.39:0/1743363471 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9fdf0e580 msgr2=0x55c9fdf0e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:00:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.240+0000 7fd1f5766700 1 --2- 172.21.15.39:0/1743363471 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9fdf0e580 0x55c9fdf0e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.240+0000 7fd1f5766700 1 -- 172.21.15.39:0/1743363471 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c9fdd6a6b0 msgr2=0x55c9fdf13d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:00:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.240+0000 7fd1f5766700 1 --2- 172.21.15.39:0/1743363471 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c9fdd6a6b0 0x55c9fdf13d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.240+0000 7fd1f5766700 1 -- 172.21.15.39:0/1743363471 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c9fdc8dd80 con 0x55c9fdf142d0 2023-07-21T19:00:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.240+0000 7fd1f5766700 1 --2- 172.21.15.39:0/1743363471 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9fdf142d0 0x55c9fdf0e040 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec00ba90 tx=0x7fd1ec00bda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa6ee0a62105f6a6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:00:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.241+0000 7fd1dbfff700 1 -- 172.21.15.39:0/1743363471 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec004a30 con 0x55c9fdf142d0 2023-07-21T19:00:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.241+0000 7fd1dbfff700 1 -- 172.21.15.39:0/1743363471 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1ec00b570 con 0x55c9fdf142d0 2023-07-21T19:00:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.241+0000 7fd1dbfff700 1 -- 172.21.15.39:0/1743363471 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec0054e0 con 0x55c9fdf142d0 2023-07-21T19:00:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.242+0000 7fd204351b80 1 -- 172.21.15.39:0/1743363471 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9fdf142d0 msgr2=0x55c9fdf0e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:00:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.242+0000 7fd204351b80 1 --2- 172.21.15.39:0/1743363471 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9fdf142d0 0x55c9fdf0e040 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec00ba90 tx=0x7fd1ec00bda0 comp rx=0 tx=0).stop 2023-07-21T19:00:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.242+0000 7fd204351b80 1 -- 172.21.15.39:0/1743363471 shutdown_connections 2023-07-21T19:00:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.242+0000 7fd204351b80 1 --2- 172.21.15.39:0/1743363471 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9fdf0e580 0x55c9fdf0e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.242+0000 7fd204351b80 1 --2- 172.21.15.39:0/1743363471 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c9fdd6a6b0 0x55c9fdf13d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.242+0000 7fd204351b80 1 --2- 172.21.15.39:0/1743363471 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9fdf142d0 0x55c9fdf0e040 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.242+0000 7fd204351b80 1 -- 172.21.15.39:0/1743363471 >> 172.21.15.39:0/1743363471 conn(0x55c9fdf03be0 msgr2=0x55c9fdf08f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:00:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.242+0000 7fd204351b80 1 -- 172.21.15.39:0/1743363471 shutdown_connections 2023-07-21T19:00:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.242+0000 7fd204351b80 1 -- 172.21.15.39:0/1743363471 wait complete. 2023-07-21T19:00:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.243+0000 7fd204351b80 1 -- start start 2023-07-21T19:00:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.243+0000 7fd204351b80 1 Processor -- start 2023-07-21T19:00:39.256 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.243+0000 7fd204351b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c9fdd6a6b0 0x55c9fdf1a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:39.256 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.243+0000 7fd204351b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9fdf0e580 0x55c9fdf17860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:39.256 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.243+0000 7fd204351b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9fdf142d0 0x55c9fdf17fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:39.256 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.243+0000 7fd204351b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9fddffa60 con 0x55c9fdd6a6b0 2023-07-21T19:00:39.257 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.243+0000 7fd204351b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9fddff760 con 0x55c9fdf142d0 2023-07-21T19:00:39.257 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.243+0000 7fd204351b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9fdce38c0 con 0x55c9fdf0e580 2023-07-21T19:00:39.257 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.243+0000 7fd204351b80 10 client.0 _ll_register_callbacks cb 0x55c9fddfd700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:00:39.257 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.246+0000 7fd1f5766700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9fdf0e580 0x55c9fdf17860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:39.257 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.246+0000 7fd1f6768700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9fdf142d0 0x55c9fdf17fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:39.257 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.246+0000 7fd1f6768700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9fdf142d0 0x55c9fdf17fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:46670/0 (socket says 172.21.15.39:46670) 2023-07-21T19:00:39.257 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.246+0000 7fd1f6768700 1 -- 172.21.15.39:0/1400058945 learned_addr learned my addr 172.21.15.39:0/1400058945 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:00:39.258 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.248+0000 7fd1f5766700 1 -- 172.21.15.39:0/1400058945 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9fdf142d0 msgr2=0x55c9fdf17fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:00:39.258 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.248+0000 7fd1f5766700 1 --2- 172.21.15.39:0/1400058945 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9fdf142d0 0x55c9fdf17fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:39.258 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.248+0000 7fd1f5766700 1 -- 172.21.15.39:0/1400058945 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c9fdd6a6b0 msgr2=0x55c9fdf1a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:00:39.259 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.248+0000 7fd1f5766700 1 --2- 172.21.15.39:0/1400058945 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c9fdd6a6b0 0x55c9fdf1a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:39.259 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.248+0000 7fd1f5766700 1 -- 172.21.15.39:0/1400058945 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1ec00b6f0 con 0x55c9fdf0e580 2023-07-21T19:00:39.259 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.248+0000 7fd1f5766700 1 --2- 172.21.15.39:0/1400058945 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9fdf0e580 0x55c9fdf17860 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec00d5c0 tx=0x7fd1ec005440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ccfa8bf976373e5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:00:39.259 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.248+0000 7fd1d97fa700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2023-07-21T19:00:39.259 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.248+0000 7fd1d97fa700 1 -- 172.21.15.39:0/1400058945 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec01f070 con 0x55c9fdf0e580 2023-07-21T19:00:39.259 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.248+0000 7fd1d97fa700 1 -- 172.21.15.39:0/1400058945 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fd1ec00d7d0 con 0x55c9fdf0e580 2023-07-21T19:00:39.260 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.249+0000 7fd204351b80 1 -- 172.21.15.39:0/1400058945 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c9fde58e50 con 0x55c9fdf0e580 2023-07-21T19:00:39.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.249+0000 7fd1d97fa700 1 -- 172.21.15.39:0/1400058945 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=98) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1ec00b570 con 0x55c9fdf0e580 2023-07-21T19:00:39.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.249+0000 7fd204351b80 10 client.27028 fetch_fsmap learned FSMap version 98 2023-07-21T19:00:39.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.249+0000 7fd204351b80 1 -- 172.21.15.39:0/1400058945 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=98,monmap=4+}) v3 -- 0x55c9fdc76c00 con 0x55c9fdf0e580 2023-07-21T19:00:39.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.250+0000 7fd1d97fa700 1 -- 172.21.15.39:0/1400058945 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c9fde58e50 con 0x55c9fdf0e580 2023-07-21T19:00:39.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.250+0000 7fd1d97fa700 1 -- 172.21.15.39:0/1400058945 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 98) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fd1ec02b070 con 0x55c9fdf0e580 2023-07-21T19:00:39.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.250+0000 7fd204351b80 10 client.27028 fetch_fsmap finished waiting for FSMap version 98 2023-07-21T19:00:39.262 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.250+0000 7fd204351b80 10 client.27028 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:00:39.262 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.250+0000 7fd204351b80 20 client.27028 populate_metadata read hostname 'smithi039' 2023-07-21T19:00:39.262 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.250+0000 7fd204351b80 1 --2- 172.21.15.39:0/1400058945 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55c9fdf27b20 0x55c9fdf29f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:39.262 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.250+0000 7fd204351b80 4 client.27028 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:00:39.262 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.250+0000 7fd204351b80 1 -- 172.21.15.39:0/1400058945 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c9fde1aff0 con 0x55c9fdf27b20 2023-07-21T19:00:39.263 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.252+0000 7fd1f5f67700 1 --2- 172.21.15.39:0/1400058945 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55c9fdf27b20 0x55c9fdf29f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:39.266 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.256+0000 7fd1d97fa700 10 client.27028 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T19:00:39.266 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.256+0000 7fd1f5f67700 1 --2- 172.21.15.39:0/1400058945 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55c9fdf27b20 0x55c9fdf29f50 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6f188927ab84149 server_cookie=a6ba30adf8bdd482 in_seq=0 out_seq=0 2023-07-21T19:00:39.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.259+0000 7fd1d97fa700 1 -- 172.21.15.39:0/1400058945 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c9fde1aff0 con 0x55c9fdf27b20 2023-07-21T19:00:39.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.259+0000 7fd1d97fa700 10 client.27028 handle_command_reply: tid=0 2023-07-21T19:00:39.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.259+0000 7fd204351b80 1 client.27028 shutdown 2023-07-21T19:00:39.271 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.261+0000 7fd204351b80 1 -- 172.21.15.39:0/1400058945 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9fdf0e580 msgr2=0x55c9fdf17860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:00:39.271 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.261+0000 7fd204351b80 1 --2- 172.21.15.39:0/1400058945 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9fdf0e580 0x55c9fdf17860 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec00d5c0 tx=0x7fd1ec005440 comp rx=0 tx=0).stop 2023-07-21T19:00:39.272 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.261+0000 7fd204351b80 1 -- 172.21.15.39:0/1400058945 shutdown_connections 2023-07-21T19:00:39.272 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.261+0000 7fd204351b80 1 --2- 172.21.15.39:0/1400058945 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55c9fdf27b20 0x55c9fdf29f50 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:39.272 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.261+0000 7fd204351b80 1 --2- 172.21.15.39:0/1400058945 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9fdf142d0 0x55c9fdf17fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:39.272 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.261+0000 7fd204351b80 1 --2- 172.21.15.39:0/1400058945 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c9fdd6a6b0 0x55c9fdf1a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:39.272 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.261+0000 7fd204351b80 1 --2- 172.21.15.39:0/1400058945 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9fdf0e580 0x55c9fdf17860 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:39.272 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.261+0000 7fd204351b80 1 -- 172.21.15.39:0/1400058945 >> 172.21.15.39:0/1400058945 conn(0x55c9fdf03be0 msgr2=0x55c9fdf07c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:00:39.272 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.261+0000 7fd1d97fa700 0 client.27028 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T19:00:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.261+0000 7fd204351b80 1 -- 172.21.15.39:0/1400058945 shutdown_connections 2023-07-21T19:00:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.261+0000 7fd204351b80 1 -- 172.21.15.39:0/1400058945 wait complete. 2023-07-21T19:00:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.262+0000 7fd204351b80 20 client.27028 trim_cache size 0 max 16384 2023-07-21T19:00:39.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.288+0000 7fd204351b80 1 Processor -- start 2023-07-21T19:00:39.301 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.289+0000 7fd204351b80 1 -- start start 2023-07-21T19:00:39.301 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.289+0000 7fd204351b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9fdd6a6b0 0x55c9fdf0e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:39.301 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.289+0000 7fd204351b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c9fdf0e580 0x55c9fdf1ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:39.301 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.289+0000 7fd204351b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9fdf142d0 0x55c9fdf1a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:39.301 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.289+0000 7fd204351b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9fdf055f0 con 0x55c9fdf0e580 2023-07-21T19:00:39.302 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.289+0000 7fd204351b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9fdf0a800 con 0x55c9fdf142d0 2023-07-21T19:00:39.302 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.289+0000 7fd204351b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9fdf182e0 con 0x55c9fdd6a6b0 2023-07-21T19:00:39.302 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.289+0000 7fd1f5f67700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9fdd6a6b0 0x55c9fdf0e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:39.302 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.289+0000 7fd1f5f67700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9fdd6a6b0 0x55c9fdf0e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:52630/0 (socket says 172.21.15.39:52630) 2023-07-21T19:00:39.302 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.289+0000 7fd1f5f67700 1 -- 172.21.15.39:0/1921318135 learned_addr learned my addr 172.21.15.39:0/1921318135 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:00:39.302 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.289+0000 7fd1f6768700 1 --2- 172.21.15.39:0/1921318135 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9fdf142d0 0x55c9fdf1a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:39.302 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.290+0000 7fd1f5f67700 1 -- 172.21.15.39:0/1921318135 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9fdf142d0 msgr2=0x55c9fdf1a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:00:39.303 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.290+0000 7fd1f5f67700 1 --2- 172.21.15.39:0/1921318135 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9fdf142d0 0x55c9fdf1a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:39.303 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.290+0000 7fd1f5f67700 1 -- 172.21.15.39:0/1921318135 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c9fdf0e580 msgr2=0x55c9fdf1ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:00:39.303 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.290+0000 7fd1f5f67700 1 --2- 172.21.15.39:0/1921318135 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c9fdf0e580 0x55c9fdf1ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:39.303 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.290+0000 7fd1f5f67700 1 -- 172.21.15.39:0/1921318135 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c9fdcb41f0 con 0x55c9fdd6a6b0 2023-07-21T19:00:39.303 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.290+0000 7fd1f5f67700 1 --2- 172.21.15.39:0/1921318135 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9fdd6a6b0 0x55c9fdf0e040 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd1e001a9d0 tx=0x7fd1e000e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64a6360625b25f38 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:00:39.303 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.290+0000 7fd1e4ff9700 1 -- 172.21.15.39:0/1921318135 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e00056d0 con 0x55c9fdd6a6b0 2023-07-21T19:00:39.303 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.290+0000 7fd1e4ff9700 1 -- 172.21.15.39:0/1921318135 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1e0003e00 con 0x55c9fdd6a6b0 2023-07-21T19:00:39.304 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.290+0000 7fd1e4ff9700 1 -- 172.21.15.39:0/1921318135 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e0030dc0 con 0x55c9fdd6a6b0 2023-07-21T19:00:39.305 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.291+0000 7fd204351b80 1 -- 172.21.15.39:0/1921318135 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9fdd6a6b0 msgr2=0x55c9fdf0e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:00:39.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.291+0000 7fd204351b80 1 --2- 172.21.15.39:0/1921318135 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9fdd6a6b0 0x55c9fdf0e040 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd1e001a9d0 tx=0x7fd1e000e000 comp rx=0 tx=0).stop 2023-07-21T19:00:39.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.291+0000 7fd204351b80 1 -- 172.21.15.39:0/1921318135 shutdown_connections 2023-07-21T19:00:39.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.291+0000 7fd204351b80 1 --2- 172.21.15.39:0/1921318135 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9fdf142d0 0x55c9fdf1a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:39.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.291+0000 7fd204351b80 1 --2- 172.21.15.39:0/1921318135 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9fdd6a6b0 0x55c9fdf0e040 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:39.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.291+0000 7fd204351b80 1 --2- 172.21.15.39:0/1921318135 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c9fdf0e580 0x55c9fdf1ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:39.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.291+0000 7fd204351b80 1 -- 172.21.15.39:0/1921318135 >> 172.21.15.39:0/1921318135 conn(0x55c9fdf03be0 msgr2=0x55c9fdf11ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:00:39.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.292+0000 7fd204351b80 1 -- 172.21.15.39:0/1921318135 shutdown_connections 2023-07-21T19:00:39.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.292+0000 7fd204351b80 1 -- 172.21.15.39:0/1921318135 wait complete. 2023-07-21T19:00:39.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.292+0000 7fd204351b80 1 -- start start 2023-07-21T19:00:39.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.292+0000 7fd204351b80 1 Processor -- start 2023-07-21T19:00:39.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.293+0000 7fd204351b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9fdd6a6b0 0x55c9fdf1de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:39.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.293+0000 7fd204351b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c9fdf0e580 0x55c9fdf1e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.293+0000 7fd204351b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9fdf142d0 0x55c9fdf7aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.293+0000 7fd204351b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9fdf1eca0 con 0x55c9fdf0e580 2023-07-21T19:00:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.293+0000 7fd204351b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9fdf7af80 con 0x55c9fdf142d0 2023-07-21T19:00:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.293+0000 7fd204351b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9fdf7b0c0 con 0x55c9fdd6a6b0 2023-07-21T19:00:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.293+0000 7fd204351b80 10 client.0 _ll_register_callbacks cb 0x7fd1e0005e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:00:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.293+0000 7fd1f5766700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c9fdf0e580 0x55c9fdf1e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.294+0000 7fd1f5766700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c9fdf0e580 0x55c9fdf1e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34086/0 (socket says 172.21.15.39:34086) 2023-07-21T19:00:39.309 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.294+0000 7fd1f5766700 1 -- 172.21.15.39:0/2414151301 learned_addr learned my addr 172.21.15.39:0/2414151301 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:00:39.309 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.294+0000 7fd1f5f67700 1 --2- 172.21.15.39:0/2414151301 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9fdd6a6b0 0x55c9fdf1de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:39.309 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.294+0000 7fd1f5766700 1 -- 172.21.15.39:0/2414151301 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9fdd6a6b0 msgr2=0x55c9fdf1de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:00:39.309 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.294+0000 7fd1f5766700 1 --2- 172.21.15.39:0/2414151301 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9fdd6a6b0 0x55c9fdf1de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:39.309 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.294+0000 7fd1f5766700 1 -- 172.21.15.39:0/2414151301 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9fdf142d0 msgr2=0x55c9fdf7aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:00:39.309 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.294+0000 7fd1f5766700 1 --2- 172.21.15.39:0/2414151301 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9fdf142d0 0x55c9fdf7aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:39.309 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.294+0000 7fd1f5766700 1 -- 172.21.15.39:0/2414151301 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1f01138f0 con 0x55c9fdf0e580 2023-07-21T19:00:39.309 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.294+0000 7fd1f5766700 1 --2- 172.21.15.39:0/2414151301 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c9fdf0e580 0x55c9fdf1e3d0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x55c9fdf18f10 tx=0x7fd1ec003d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90dd26650a952855 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:00:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.294+0000 7fd1d97fa700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T19:00:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.294+0000 7fd1d97fa700 1 -- 172.21.15.39:0/2414151301 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec01f070 con 0x55c9fdf0e580 2023-07-21T19:00:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.295+0000 7fd1d97fa700 1 -- 172.21.15.39:0/2414151301 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fd1ec0104b0 con 0x55c9fdf0e580 2023-07-21T19:00:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.295+0000 7fd204351b80 1 -- 172.21.15.39:0/2414151301 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c9fddb9490 con 0x55c9fdf0e580 2023-07-21T19:00:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.295+0000 7fd1d97fa700 1 -- 172.21.15.39:0/2414151301 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=98) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1ec004f90 con 0x55c9fdf0e580 2023-07-21T19:00:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.295+0000 7fd204351b80 10 client.17562 fetch_fsmap learned FSMap version 98 2023-07-21T19:00:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.295+0000 7fd204351b80 1 -- 172.21.15.39:0/2414151301 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=98,monmap=4+}) v3 -- 0x55c9fdc8c250 con 0x55c9fdf0e580 2023-07-21T19:00:39.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.295+0000 7fd1d97fa700 1 -- 172.21.15.39:0/2414151301 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c9fddb9490 con 0x55c9fdf0e580 2023-07-21T19:00:39.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.296+0000 7fd1d97fa700 1 -- 172.21.15.39:0/2414151301 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 98) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fd1ec00d2e0 con 0x55c9fdf0e580 2023-07-21T19:00:39.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.296+0000 7fd204351b80 10 client.17562 fetch_fsmap finished waiting for FSMap version 98 2023-07-21T19:00:39.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.296+0000 7fd204351b80 10 client.17562 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:00:39.312 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.296+0000 7fd204351b80 20 client.17562 populate_metadata read hostname 'smithi039' 2023-07-21T19:00:39.312 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.296+0000 7fd204351b80 1 --2- 172.21.15.39:0/2414151301 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55c9fdf7c340 0x55c9fdf7e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:39.312 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.296+0000 7fd204351b80 4 client.17562 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:00:39.312 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.296+0000 7fd204351b80 1 -- 172.21.15.39:0/2414151301 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c9fdcaef10 con 0x55c9fdf7c340 2023-07-21T19:00:39.312 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.298+0000 7fd1f5f67700 1 --2- 172.21.15.39:0/2414151301 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55c9fdf7c340 0x55c9fdf7e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:39.322 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.309+0000 7fd1d97fa700 10 client.17562 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T19:00:39.322 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.309+0000 7fd1f5f67700 1 --2- 172.21.15.39:0/2414151301 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55c9fdf7c340 0x55c9fdf7e750 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9be2e6fe2bf77ae server_cookie=79c3c2bb5f3e01d8 in_seq=0 out_seq=0 2023-07-21T19:00:39.620 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.608+0000 7fd1d97fa700 1 -- 172.21.15.39:0/2414151301 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c9fdcaef10 con 0x55c9fdf7c340 2023-07-21T19:00:39.621 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.608+0000 7fd1d97fa700 10 client.17562 handle_command_reply: tid=0 2023-07-21T19:00:39.621 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.608+0000 7fd204351b80 1 client.17562 shutdown 2023-07-21T19:00:39.621 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.608+0000 7fd204351b80 1 -- 172.21.15.39:0/2414151301 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c9fdf0e580 msgr2=0x55c9fdf1e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:00:39.621 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.608+0000 7fd204351b80 1 --2- 172.21.15.39:0/2414151301 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c9fdf0e580 0x55c9fdf1e3d0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x55c9fdf18f10 tx=0x7fd1ec003d60 comp rx=0 tx=0).stop 2023-07-21T19:00:39.621 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.608+0000 7fd204351b80 1 -- 172.21.15.39:0/2414151301 shutdown_connections 2023-07-21T19:00:39.622 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.608+0000 7fd204351b80 1 --2- 172.21.15.39:0/2414151301 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55c9fdf7c340 0x55c9fdf7e750 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:39.622 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.608+0000 7fd204351b80 1 --2- 172.21.15.39:0/2414151301 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9fdf142d0 0x55c9fdf7aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:39.622 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.608+0000 7fd204351b80 1 --2- 172.21.15.39:0/2414151301 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9fdd6a6b0 0x55c9fdf1de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:39.622 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.608+0000 7fd204351b80 1 --2- 172.21.15.39:0/2414151301 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c9fdf0e580 0x55c9fdf1e3d0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:39.622 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.608+0000 7fd204351b80 1 -- 172.21.15.39:0/2414151301 >> 172.21.15.39:0/2414151301 conn(0x55c9fdf03be0 msgr2=0x55c9fdf17100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:00:39.622 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.608+0000 7fd1d97fa700 0 client.17562 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T19:00:39.622 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.608+0000 7fd204351b80 1 -- 172.21.15.39:0/2414151301 shutdown_connections 2023-07-21T19:00:39.623 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.608+0000 7fd204351b80 1 -- 172.21.15.39:0/2414151301 wait complete. 2023-07-21T19:00:39.623 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.610+0000 7fd204351b80 20 client.17562 trim_cache size 0 max 16384 2023-07-21T19:00:39.623 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T19:00:39.623 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T19:00:39.624 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T19:00:39.624 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T19:00:39.624 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.611+0000 7fd1f81cb700 1 -- 172.21.15.39:0/3550531698 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fd1dc0a1340 msgr2=0x7fd1dc0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:00:39.624 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.611+0000 7fd1f81cb700 1 --2- 172.21.15.39:0/3550531698 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fd1dc0a1340 0x7fd1dc0a37f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd1e001a8f0 tx=0x7fd1e0000bc0 comp rx=0 tx=0).stop 2023-07-21T19:00:39.624 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.611+0000 7fd1f81cb700 1 -- 172.21.15.39:0/3550531698 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd1f0103fc0 msgr2=0x7fd1f010e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:00:39.624 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.611+0000 7fd1f81cb700 1 --2- 172.21.15.39:0/3550531698 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd1f0103fc0 0x7fd1f010e910 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd1e802d9e0 tx=0x7fd1e802dda0 comp rx=0 tx=0).stop 2023-07-21T19:00:39.625 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.615+0000 7fd1f81cb700 1 -- 172.21.15.39:0/3550531698 shutdown_connections 2023-07-21T19:00:39.625 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.615+0000 7fd1f81cb700 1 --2- 172.21.15.39:0/3550531698 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fd1dc0a1340 0x7fd1dc0a37f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:39.625 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.615+0000 7fd1f81cb700 1 --2- 172.21.15.39:0/3550531698 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd1f0103fc0 0x7fd1f010e910 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:39.626 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.615+0000 7fd1f81cb700 1 --2- 172.21.15.39:0/3550531698 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1f00febc0 0x7fd1f0199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:39.626 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.615+0000 7fd1f81cb700 1 --2- 172.21.15.39:0/3550531698 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1f00ff4d0 0x7fd1f019a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:39.626 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.615+0000 7fd1f81cb700 1 -- 172.21.15.39:0/3550531698 >> 172.21.15.39:0/3550531698 conn(0x7fd1f0076a10 msgr2=0x7fd1f0102a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:00:39.627 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.617+0000 7fd1f81cb700 1 -- 172.21.15.39:0/3550531698 shutdown_connections 2023-07-21T19:00:39.627 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:39.617+0000 7fd1f81cb700 1 -- 172.21.15.39:0/3550531698 wait complete. 2023-07-21T19:00:39.694 INFO:tasks.workunit.client.0.smithi039.stdout: 10 199050 30.63 MB/sec execute 232 sec latency 843.803 ms 2023-07-21T19:00:39.860 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:00:39 smithi193 ceph-mon[98204]: pgmap v599: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 8.0 MiB/s wr, 509 op/s 2023-07-21T19:00:39.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:00:39 smithi155 ceph-mon[97934]: pgmap v599: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 8.0 MiB/s wr, 509 op/s 2023-07-21T19:00:39.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:00:39 smithi039 ceph-mon[96489]: pgmap v599: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 8.0 MiB/s wr, 509 op/s 2023-07-21T19:00:40.357 INFO:journalctl@ceph.mds.c.smithi193.stdout:Jul 21 19:00:40 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-c[124252]: ignoring --setuser ceph since I am not root 2023-07-21T19:00:40.358 INFO:journalctl@ceph.mds.c.smithi193.stdout:Jul 21 19:00:40 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-c[124252]: ignoring --setgroup ceph since I am not root 2023-07-21T19:00:40.358 INFO:journalctl@ceph.mds.c.smithi193.stdout:Jul 21 19:00:40 smithi193 ceph-mds[124276]: -- 172.21.15.193:0/3155579347 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a2191db1e0 con 0x55a2191b1000 2023-07-21T19:00:40.358 INFO:journalctl@ceph.mds.c.smithi193.stdout:Jul 21 19:00:40 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-c[124252]: starting mds.c at 2023-07-21T19:00:40.694 INFO:tasks.workunit.client.0.smithi039.stdout: 10 199303 30.55 MB/sec execute 233 sec latency 178.677 ms 2023-07-21T19:00:40.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:00:40 smithi193 ceph-mon[98204]: daemon mds.c restarted 2023-07-21T19:00:40.879 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:00:40 smithi155 ceph-mon[97934]: daemon mds.c restarted 2023-07-21T19:00:40.927 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:00:40 smithi039 ceph-mon[96489]: daemon mds.c restarted 2023-07-21T19:00:41.694 INFO:tasks.workunit.client.0.smithi039.stdout: 10 199811 30.54 MB/sec execute 234 sec latency 130.617 ms 2023-07-21T19:00:41.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:00:41 smithi193 ceph-mon[98204]: osdmap e112: 12 total, 12 up, 12 in 2023-07-21T19:00:41.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:00:41 smithi193 ceph-mon[98204]: mds.? [v2:172.21.15.193:6836/2158259657,v1:172.21.15.193:6837/2158259657] up:boot 2023-07-21T19:00:41.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:00:41 smithi193 ceph-mon[98204]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2023-07-21T19:00:41.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:00:41 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-21T19:00:41.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:00:41 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T19:00:41.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:00:41 smithi193 ceph-mon[98204]: pgmap v601: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 11 MiB/s wr, 634 op/s 2023-07-21T19:00:41.880 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:00:41 smithi155 ceph-mon[97934]: osdmap e112: 12 total, 12 up, 12 in 2023-07-21T19:00:41.880 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:00:41 smithi155 ceph-mon[97934]: mds.? [v2:172.21.15.193:6836/2158259657,v1:172.21.15.193:6837/2158259657] up:boot 2023-07-21T19:00:41.880 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:00:41 smithi155 ceph-mon[97934]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2023-07-21T19:00:41.880 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:00:41 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-21T19:00:41.881 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:00:41 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T19:00:41.881 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:00:41 smithi155 ceph-mon[97934]: pgmap v601: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 11 MiB/s wr, 634 op/s 2023-07-21T19:00:41.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:00:41 smithi039 ceph-mon[96489]: osdmap e112: 12 total, 12 up, 12 in 2023-07-21T19:00:41.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:00:41 smithi039 ceph-mon[96489]: mds.? [v2:172.21.15.193:6836/2158259657,v1:172.21.15.193:6837/2158259657] up:boot 2023-07-21T19:00:41.926 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:00:41 smithi039 ceph-mon[96489]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2023-07-21T19:00:41.926 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:00:41 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-21T19:00:41.926 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:00:41 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T19:00:41.927 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:00:41 smithi039 ceph-mon[96489]: pgmap v601: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 11 MiB/s wr, 634 op/s 2023-07-21T19:00:42.694 INFO:tasks.workunit.client.0.smithi039.stdout: 10 200469 30.57 MB/sec execute 235 sec latency 68.350 ms 2023-07-21T19:00:43.694 INFO:tasks.workunit.client.0.smithi039.stdout: 10 201018 30.57 MB/sec execute 236 sec latency 76.325 ms 2023-07-21T19:00:43.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:00:43 smithi039 ceph-mon[96489]: pgmap v602: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 26 MiB/s wr, 1.05k op/s 2023-07-21T19:00:44.107 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:00:43 smithi193 ceph-mon[98204]: pgmap v602: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 26 MiB/s wr, 1.05k op/s 2023-07-21T19:00:44.126 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:00:43 smithi155 ceph-mon[97934]: pgmap v602: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 26 MiB/s wr, 1.05k op/s 2023-07-21T19:00:44.694 INFO:tasks.workunit.client.0.smithi039.stdout: 10 201650 30.56 MB/sec execute 237 sec latency 68.743 ms 2023-07-21T19:00:45.694 INFO:tasks.workunit.client.0.smithi039.stdout: 10 202273 30.59 MB/sec execute 238 sec latency 82.430 ms 2023-07-21T19:00:45.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:00:45 smithi039 ceph-mon[96489]: pgmap v603: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 26 MiB/s wr, 1.05k op/s 2023-07-21T19:00:46.107 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:00:45 smithi193 ceph-mon[98204]: pgmap v603: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 26 MiB/s wr, 1.05k op/s 2023-07-21T19:00:46.126 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:00:45 smithi155 ceph-mon[97934]: pgmap v603: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 26 MiB/s wr, 1.05k op/s 2023-07-21T19:00:46.694 INFO:tasks.workunit.client.0.smithi039.stdout: 10 202800 30.59 MB/sec execute 239 sec latency 158.384 ms 2023-07-21T19:00:47.694 INFO:tasks.workunit.client.0.smithi039.stdout: 10 203353 30.55 MB/sec execute 240 sec latency 96.439 ms 2023-07-21T19:00:48.607 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:00:48 smithi193 ceph-mon[98204]: pgmap v604: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 25 MiB/s wr, 1.04k op/s 2023-07-21T19:00:48.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:00:48 smithi155 ceph-mon[97934]: pgmap v604: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 25 MiB/s wr, 1.04k op/s 2023-07-21T19:00:48.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:00:48 smithi039 ceph-mon[96489]: pgmap v604: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 25 MiB/s wr, 1.04k op/s 2023-07-21T19:00:48.694 INFO:tasks.workunit.client.0.smithi039.stdout: 10 204083 30.62 MB/sec execute 241 sec latency 57.756 ms 2023-07-21T19:00:49.694 INFO:tasks.workunit.client.0.smithi039.stdout: 10 204780 30.65 MB/sec execute 242 sec latency 63.601 ms 2023-07-21T19:00:50.607 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:00:50 smithi193 ceph-mon[98204]: pgmap v605: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 25 MiB/s wr, 1.04k op/s 2023-07-21T19:00:50.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:00:50 smithi155 ceph-mon[97934]: pgmap v605: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 25 MiB/s wr, 1.04k op/s 2023-07-21T19:00:50.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:00:50 smithi039 ceph-mon[96489]: pgmap v605: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 25 MiB/s wr, 1.04k op/s 2023-07-21T19:00:50.695 INFO:tasks.workunit.client.0.smithi039.stdout: 10 205422 30.65 MB/sec execute 243 sec latency 67.243 ms 2023-07-21T19:00:51.695 INFO:tasks.workunit.client.0.smithi039.stdout: 10 205975 30.67 MB/sec execute 244 sec latency 62.890 ms 2023-07-21T19:00:52.607 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:00:52 smithi193 ceph-mon[98204]: pgmap v606: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 23 MiB/s wr, 990 op/s 2023-07-21T19:00:52.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:00:52 smithi155 ceph-mon[97934]: pgmap v606: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 23 MiB/s wr, 990 op/s 2023-07-21T19:00:52.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:00:52 smithi039 ceph-mon[96489]: pgmap v606: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 23 MiB/s wr, 990 op/s 2023-07-21T19:00:52.695 INFO:tasks.workunit.client.0.smithi039.stdout: 10 206561 30.67 MB/sec execute 245 sec latency 84.542 ms 2023-07-21T19:00:52.772 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.762+0000 7f5e3a876700 1 -- 172.21.15.39:0/3189301198 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5e341034d0 msgr2=0x7f5e3410b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:00:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.762+0000 7f5e3a876700 1 --2- 172.21.15.39:0/3189301198 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5e341034d0 0x7f5e3410b040 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c00b0d0 tx=0x7f5e2c00b3e0 comp rx=0 tx=0).stop 2023-07-21T19:00:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.762+0000 7f5e3a876700 1 -- 172.21.15.39:0/3189301198 shutdown_connections 2023-07-21T19:00:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.762+0000 7f5e3a876700 1 --2- 172.21.15.39:0/3189301198 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e340fef80 0x7f5e340ff350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.762+0000 7f5e3a876700 1 --2- 172.21.15.39:0/3189301198 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5e341034d0 0x7f5e3410b040 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.762+0000 7f5e3a876700 1 --2- 172.21.15.39:0/3189301198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e340ff920 0x7f5e34102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.762+0000 7f5e3a876700 1 -- 172.21.15.39:0/3189301198 >> 172.21.15.39:0/3189301198 conn(0x7f5e340752a0 msgr2=0x7f5e340756a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:00:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.762+0000 7f5e3a876700 1 -- 172.21.15.39:0/3189301198 shutdown_connections 2023-07-21T19:00:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.762+0000 7f5e3a876700 1 -- 172.21.15.39:0/3189301198 wait complete. 2023-07-21T19:00:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.763+0000 7f5e3a876700 1 Processor -- start 2023-07-21T19:00:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.763+0000 7f5e3a876700 1 -- start start 2023-07-21T19:00:52.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.763+0000 7f5e3a876700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e340fef80 0x7f5e34100070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:52.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.763+0000 7f5e3a876700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5e340ff920 0x7f5e341005b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:52.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.763+0000 7f5e3a876700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e341034d0 0x7f5e341a3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:52.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.763+0000 7f5e3a876700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e341a4450 con 0x7f5e340ff920 2023-07-21T19:00:52.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.763+0000 7f5e3a876700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e341a4590 con 0x7f5e340fef80 2023-07-21T19:00:52.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.763+0000 7f5e3a876700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e341a4700 con 0x7f5e341034d0 2023-07-21T19:00:52.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.763+0000 7f5e337fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5e340ff920 0x7f5e341005b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:52.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.764+0000 7f5e33fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e340fef80 0x7f5e34100070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:52.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.764+0000 7f5e337fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5e340ff920 0x7f5e341005b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34100/0 (socket says 172.21.15.39:34100) 2023-07-21T19:00:52.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.764+0000 7f5e337fe700 1 -- 172.21.15.39:0/2871568523 learned_addr learned my addr 172.21.15.39:0/2871568523 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:00:52.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.764+0000 7f5e38e13700 1 --2- 172.21.15.39:0/2871568523 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e341034d0 0x7f5e341a3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:52.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.764+0000 7f5e337fe700 1 -- 172.21.15.39:0/2871568523 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e341034d0 msgr2=0x7f5e341a3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:00:52.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.764+0000 7f5e337fe700 1 --2- 172.21.15.39:0/2871568523 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e341034d0 0x7f5e341a3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:52.778 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.764+0000 7f5e337fe700 1 -- 172.21.15.39:0/2871568523 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e340fef80 msgr2=0x7f5e34100070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:00:52.778 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.764+0000 7f5e337fe700 1 --2- 172.21.15.39:0/2871568523 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e340fef80 0x7f5e34100070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:52.778 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.764+0000 7f5e337fe700 1 -- 172.21.15.39:0/2871568523 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e28009520 con 0x7f5e340ff920 2023-07-21T19:00:52.778 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.764+0000 7f5e33fff700 1 --2- 172.21.15.39:0/2871568523 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e340fef80 0x7f5e34100070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:00:52.778 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.764+0000 7f5e337fe700 1 --2- 172.21.15.39:0/2871568523 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5e340ff920 0x7f5e341005b0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f5e28009fd0 tx=0x7f5e2800ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:00:52.778 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.764+0000 7f5e317fa700 1 -- 172.21.15.39:0/2871568523 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2800cbd0 con 0x7f5e340ff920 2023-07-21T19:00:52.778 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.764+0000 7f5e317fa700 1 -- 172.21.15.39:0/2871568523 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e2800eea0 con 0x7f5e340ff920 2023-07-21T19:00:52.779 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.764+0000 7f5e3a876700 1 -- 172.21.15.39:0/2871568523 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e2c009d00 con 0x7f5e340ff920 2023-07-21T19:00:52.779 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.764+0000 7f5e317fa700 1 -- 172.21.15.39:0/2871568523 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e28003cf0 con 0x7f5e340ff920 2023-07-21T19:00:52.779 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.764+0000 7f5e3a876700 1 -- 172.21.15.39:0/2871568523 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e341a4d60 con 0x7f5e340ff920 2023-07-21T19:00:52.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.767+0000 7f5e317fa700 1 -- 172.21.15.39:0/2871568523 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f5e28003eb0 con 0x7f5e340ff920 2023-07-21T19:00:52.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.767+0000 7f5e317fa700 1 --2- 172.21.15.39:0/2871568523 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f5e1c0a12c0 0x7f5e1c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:52.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.768+0000 7f5e317fa700 1 -- 172.21.15.39:0/2871568523 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f5e28014070 con 0x7f5e340ff920 2023-07-21T19:00:52.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.768+0000 7f5e33fff700 1 --2- 172.21.15.39:0/2871568523 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f5e1c0a12c0 0x7f5e1c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:52.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.768+0000 7f5e33fff700 1 --2- 172.21.15.39:0/2871568523 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f5e1c0a12c0 0x7f5e1c0a3770 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5e24011490 tx=0x7f5e24009260 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:00:52.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.769+0000 7f5e469fcb80 1 Processor -- start 2023-07-21T19:00:52.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.769+0000 7f5e469fcb80 1 -- start start 2023-07-21T19:00:52.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.769+0000 7f5e469fcb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7d303a6b0 0x55f7d31e3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:52.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.769+0000 7f5e469fcb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f7d31e42d0 0x55f7d31de040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:52.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.769+0000 7f5e469fcb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7d31de580 0x55f7d31de9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:52.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.769+0000 7f5e469fcb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7d30e3c50 con 0x55f7d31e42d0 2023-07-21T19:00:52.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.769+0000 7f5e469fcb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7d31369e0 con 0x55f7d303a6b0 2023-07-21T19:00:52.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.769+0000 7f5e469fcb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7d312fe70 con 0x55f7d31de580 2023-07-21T19:00:52.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.769+0000 7f5e33fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7d303a6b0 0x55f7d31e3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:52.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.769+0000 7f5e38e13700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7d31de580 0x55f7d31de9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:52.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.769+0000 7f5e33fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7d303a6b0 0x55f7d31e3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:46702/0 (socket says 172.21.15.39:46702) 2023-07-21T19:00:52.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.769+0000 7f5e33fff700 1 -- 172.21.15.39:0/4214277836 learned_addr learned my addr 172.21.15.39:0/4214277836 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:00:52.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.770+0000 7f5e337fe700 1 --2- 172.21.15.39:0/4214277836 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f7d31e42d0 0x55f7d31de040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:52.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.770+0000 7f5e38e13700 1 -- 172.21.15.39:0/4214277836 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7d303a6b0 msgr2=0x55f7d31e3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:00:52.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.770+0000 7f5e38e13700 1 --2- 172.21.15.39:0/4214277836 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7d303a6b0 0x55f7d31e3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:52.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.770+0000 7f5e38e13700 1 -- 172.21.15.39:0/4214277836 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f7d31e42d0 msgr2=0x55f7d31de040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:00:52.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.770+0000 7f5e38e13700 1 --2- 172.21.15.39:0/4214277836 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f7d31e42d0 0x55f7d31de040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:52.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.770+0000 7f5e38e13700 1 -- 172.21.15.39:0/4214277836 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f7d2f5dd80 con 0x55f7d31de580 2023-07-21T19:00:52.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.770+0000 7f5e38e13700 1 --2- 172.21.15.39:0/4214277836 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7d31de580 0x55f7d31de9d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c00b770 tx=0x7f5e2c009c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bf919e7ee45902f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:00:52.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.770+0000 7f5e1a7fc700 1 -- 172.21.15.39:0/4214277836 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c00e040 con 0x55f7d31de580 2023-07-21T19:00:52.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.771+0000 7f5e1a7fc700 1 -- 172.21.15.39:0/4214277836 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e2c0060a0 con 0x55f7d31de580 2023-07-21T19:00:52.785 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.771+0000 7f5e1a7fc700 1 -- 172.21.15.39:0/4214277836 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c003dc0 con 0x55f7d31de580 2023-07-21T19:00:52.787 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.772+0000 7f5e469fcb80 1 -- 172.21.15.39:0/4214277836 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7d31de580 msgr2=0x55f7d31de9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:00:52.787 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.772+0000 7f5e469fcb80 1 --2- 172.21.15.39:0/4214277836 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7d31de580 0x55f7d31de9d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c00b770 tx=0x7f5e2c009c60 comp rx=0 tx=0).stop 2023-07-21T19:00:52.787 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.772+0000 7f5e469fcb80 1 -- 172.21.15.39:0/4214277836 shutdown_connections 2023-07-21T19:00:52.787 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.772+0000 7f5e469fcb80 1 --2- 172.21.15.39:0/4214277836 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7d303a6b0 0x55f7d31e3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:52.787 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.772+0000 7f5e469fcb80 1 --2- 172.21.15.39:0/4214277836 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7d31de580 0x55f7d31de9d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:52.787 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.772+0000 7f5e469fcb80 1 --2- 172.21.15.39:0/4214277836 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f7d31e42d0 0x55f7d31de040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:52.787 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.772+0000 7f5e469fcb80 1 -- 172.21.15.39:0/4214277836 >> 172.21.15.39:0/4214277836 conn(0x55f7d31d3be0 msgr2=0x55f7d31d8f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:00:52.788 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.772+0000 7f5e469fcb80 1 -- 172.21.15.39:0/4214277836 shutdown_connections 2023-07-21T19:00:52.788 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.772+0000 7f5e469fcb80 1 -- 172.21.15.39:0/4214277836 wait complete. 2023-07-21T19:00:52.788 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.772+0000 7f5e469fcb80 1 -- start start 2023-07-21T19:00:52.788 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.773+0000 7f5e469fcb80 1 Processor -- start 2023-07-21T19:00:52.788 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.773+0000 7f5e469fcb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7d303a6b0 0x55f7d31ea570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:52.788 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.773+0000 7f5e469fcb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7d31de580 0x55f7d31e77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:52.788 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.773+0000 7f5e469fcb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f7d31e42d0 0x55f7d31e7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:52.789 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.773+0000 7f5e469fcb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7d30cfa60 con 0x55f7d31e42d0 2023-07-21T19:00:52.789 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.773+0000 7f5e469fcb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7d30cf760 con 0x55f7d31de580 2023-07-21T19:00:52.789 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.773+0000 7f5e469fcb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7d2fb38c0 con 0x55f7d303a6b0 2023-07-21T19:00:52.789 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.773+0000 7f5e469fcb80 10 client.0 _ll_register_callbacks cb 0x55f7d30cd700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:00:52.789 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.773+0000 7f5e33fff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7d303a6b0 0x55f7d31ea570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:52.789 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.773+0000 7f5e33fff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7d303a6b0 0x55f7d31ea570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:52664/0 (socket says 172.21.15.39:52664) 2023-07-21T19:00:52.789 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.773+0000 7f5e33fff700 1 -- 172.21.15.39:0/145483668 learned_addr learned my addr 172.21.15.39:0/145483668 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:00:52.789 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.773+0000 7f5e337fe700 1 --2- 172.21.15.39:0/145483668 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7d31de580 0x55f7d31e77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:52.790 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.773+0000 7f5e38e13700 1 --2- 172.21.15.39:0/145483668 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f7d31e42d0 0x55f7d31e7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:52.790 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.774+0000 7f5e33fff700 1 -- 172.21.15.39:0/145483668 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7d31de580 msgr2=0x55f7d31e77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:00:52.790 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.774+0000 7f5e33fff700 1 --2- 172.21.15.39:0/145483668 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7d31de580 0x55f7d31e77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:52.790 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.774+0000 7f5e33fff700 1 -- 172.21.15.39:0/145483668 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f7d31e42d0 msgr2=0x55f7d31e7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:00:52.790 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.774+0000 7f5e33fff700 1 --2- 172.21.15.39:0/145483668 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f7d31e42d0 0x55f7d31e7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:52.790 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.774+0000 7f5e33fff700 1 -- 172.21.15.39:0/145483668 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e34067fc0 con 0x55f7d303a6b0 2023-07-21T19:00:52.790 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.774+0000 7f5e33fff700 1 --2- 172.21.15.39:0/145483668 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7d303a6b0 0x55f7d31ea570 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5e240039f0 tx=0x7f5e240047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51846fb4557301e4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:00:52.790 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.774+0000 7f5e137fe700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2023-07-21T19:00:52.791 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.774+0000 7f5e137fe700 1 -- 172.21.15.39:0/145483668 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2400e970 con 0x55f7d303a6b0 2023-07-21T19:00:52.791 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.774+0000 7f5e137fe700 1 -- 172.21.15.39:0/145483668 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f5e2400ead0 con 0x55f7d303a6b0 2023-07-21T19:00:52.791 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.774+0000 7f5e469fcb80 1 -- 172.21.15.39:0/145483668 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f7d3089490 con 0x55f7d303a6b0 2023-07-21T19:00:52.791 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.775+0000 7f5e137fe700 1 -- 172.21.15.39:0/145483668 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=100) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e2400ec90 con 0x55f7d303a6b0 2023-07-21T19:00:52.791 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.775+0000 7f5e469fcb80 10 client.27052 fetch_fsmap learned FSMap version 100 2023-07-21T19:00:52.791 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.775+0000 7f5e469fcb80 1 -- 172.21.15.39:0/145483668 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=100,monmap=4+}) v3 -- 0x55f7d2f46c00 con 0x55f7d303a6b0 2023-07-21T19:00:52.791 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.775+0000 7f5e137fe700 1 -- 172.21.15.39:0/145483668 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f7d3089490 con 0x55f7d303a6b0 2023-07-21T19:00:52.792 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.775+0000 7f5e137fe700 1 -- 172.21.15.39:0/145483668 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 100) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f5e24031580 con 0x55f7d303a6b0 2023-07-21T19:00:52.792 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.775+0000 7f5e469fcb80 10 client.27052 fetch_fsmap finished waiting for FSMap version 100 2023-07-21T19:00:52.792 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.775+0000 7f5e469fcb80 10 client.27052 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:00:52.792 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.775+0000 7f5e469fcb80 20 client.27052 populate_metadata read hostname 'smithi039' 2023-07-21T19:00:52.792 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.775+0000 7f5e469fcb80 1 --2- 172.21.15.39:0/145483668 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55f7d31f7a40 0x55f7d31f9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:52.792 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.775+0000 7f5e469fcb80 4 client.27052 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:00:52.792 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.775+0000 7f5e469fcb80 1 -- 172.21.15.39:0/145483668 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f7d3128e50 con 0x55f7d31f7a40 2023-07-21T19:00:52.793 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.781+0000 7f5e337fe700 1 --2- 172.21.15.39:0/145483668 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55f7d31f7a40 0x55f7d31f9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:52.793 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.782+0000 7f5e137fe700 10 client.27052 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T19:00:52.793 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.782+0000 7f5e337fe700 1 --2- 172.21.15.39:0/145483668 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55f7d31f7a40 0x55f7d31f9e50 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fda92ff766775a5f server_cookie=84dc016efc568ae9 in_seq=0 out_seq=0 2023-07-21T19:00:52.802 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.792+0000 7f5e137fe700 1 -- 172.21.15.39:0/145483668 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f7d3128e50 con 0x55f7d31f7a40 2023-07-21T19:00:52.802 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.792+0000 7f5e137fe700 10 client.27052 handle_command_reply: tid=0 2023-07-21T19:00:52.802 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.792+0000 7f5e469fcb80 1 client.27052 shutdown 2023-07-21T19:00:52.802 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.792+0000 7f5e469fcb80 1 -- 172.21.15.39:0/145483668 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7d303a6b0 msgr2=0x55f7d31ea570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:00:52.803 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.792+0000 7f5e469fcb80 1 --2- 172.21.15.39:0/145483668 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7d303a6b0 0x55f7d31ea570 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5e240039f0 tx=0x7f5e240047c0 comp rx=0 tx=0).stop 2023-07-21T19:00:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.794+0000 7f5e469fcb80 1 -- 172.21.15.39:0/145483668 shutdown_connections 2023-07-21T19:00:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.794+0000 7f5e469fcb80 1 --2- 172.21.15.39:0/145483668 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55f7d31f7a40 0x55f7d31f9e50 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.794+0000 7f5e469fcb80 1 --2- 172.21.15.39:0/145483668 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f7d31e42d0 0x55f7d31e7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.794+0000 7f5e469fcb80 1 --2- 172.21.15.39:0/145483668 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7d303a6b0 0x55f7d31ea570 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:52.806 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.794+0000 7f5e137fe700 0 client.27052 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T19:00:52.806 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.794+0000 7f5e469fcb80 1 --2- 172.21.15.39:0/145483668 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7d31de580 0x55f7d31e77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:52.806 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.794+0000 7f5e469fcb80 1 -- 172.21.15.39:0/145483668 >> 172.21.15.39:0/145483668 conn(0x55f7d31d3be0 msgr2=0x55f7d31d7c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:00:52.806 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.794+0000 7f5e469fcb80 1 -- 172.21.15.39:0/145483668 shutdown_connections 2023-07-21T19:00:52.806 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.794+0000 7f5e469fcb80 1 -- 172.21.15.39:0/145483668 wait complete. 2023-07-21T19:00:52.806 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.794+0000 7f5e469fcb80 20 client.27052 trim_cache size 0 max 16384 2023-07-21T19:00:52.831 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.819+0000 7f5e469fcb80 1 Processor -- start 2023-07-21T19:00:52.831 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.819+0000 7f5e469fcb80 1 -- start start 2023-07-21T19:00:52.832 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.819+0000 7f5e469fcb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f7d303a6b0 0x55f7d31eeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:52.832 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.819+0000 7f5e469fcb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7d31de580 0x55f7d31e51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:52.832 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.819+0000 7f5e469fcb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7d31e42d0 0x55f7d31ea110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:52.833 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.819+0000 7f5e469fcb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7d31f51f0 con 0x55f7d303a6b0 2023-07-21T19:00:52.833 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.819+0000 7f5e469fcb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7d31e2390 con 0x55f7d31de580 2023-07-21T19:00:52.833 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.819+0000 7f5e469fcb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7d31e39d0 con 0x55f7d31e42d0 2023-07-21T19:00:52.834 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.820+0000 7f5e38e13700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7d31e42d0 0x55f7d31ea110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:52.834 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.820+0000 7f5e33fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f7d303a6b0 0x55f7d31eeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:52.834 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.820+0000 7f5e33fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f7d303a6b0 0x55f7d31eeb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34120/0 (socket says 172.21.15.39:34120) 2023-07-21T19:00:52.834 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.820+0000 7f5e38e13700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7d31e42d0 0x55f7d31ea110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:52674/0 (socket says 172.21.15.39:52674) 2023-07-21T19:00:52.835 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.820+0000 7f5e33fff700 1 -- 172.21.15.39:0/118602591 learned_addr learned my addr 172.21.15.39:0/118602591 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:00:52.835 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.820+0000 7f5e33fff700 1 -- 172.21.15.39:0/118602591 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7d31e42d0 msgr2=0x55f7d31ea110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:00:52.835 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.820+0000 7f5e33fff700 1 --2- 172.21.15.39:0/118602591 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7d31e42d0 0x55f7d31ea110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:52.836 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.820+0000 7f5e33fff700 1 -- 172.21.15.39:0/118602591 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7d31de580 msgr2=0x55f7d31e51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:00:52.836 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.820+0000 7f5e33fff700 1 --2- 172.21.15.39:0/118602591 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7d31de580 0x55f7d31e51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:52.836 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.820+0000 7f5e33fff700 1 -- 172.21.15.39:0/118602591 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f7d31fea80 con 0x55f7d303a6b0 2023-07-21T19:00:52.836 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.820+0000 7f5e33fff700 1 --2- 172.21.15.39:0/118602591 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f7d303a6b0 0x55f7d31eeb70 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f5e24011be0 tx=0x7f5e24005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cb9058b7b536288 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:00:52.837 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.820+0000 7f5e38e13700 1 --2- 172.21.15.39:0/118602591 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7d31e42d0 0x55f7d31ea110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:00:52.837 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.821+0000 7f5e1affd700 1 -- 172.21.15.39:0/118602591 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e24031660 con 0x55f7d303a6b0 2023-07-21T19:00:52.837 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.821+0000 7f5e1affd700 1 -- 172.21.15.39:0/118602591 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e24031b80 con 0x55f7d303a6b0 2023-07-21T19:00:52.837 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.821+0000 7f5e1affd700 1 -- 172.21.15.39:0/118602591 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2400e9d0 con 0x55f7d303a6b0 2023-07-21T19:00:52.839 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.822+0000 7f5e469fcb80 1 -- 172.21.15.39:0/118602591 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f7d303a6b0 msgr2=0x55f7d31eeb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:00:52.840 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.822+0000 7f5e469fcb80 1 --2- 172.21.15.39:0/118602591 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f7d303a6b0 0x55f7d31eeb70 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f5e24011be0 tx=0x7f5e24005a00 comp rx=0 tx=0).stop 2023-07-21T19:00:52.840 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.822+0000 7f5e469fcb80 1 -- 172.21.15.39:0/118602591 shutdown_connections 2023-07-21T19:00:52.840 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.822+0000 7f5e469fcb80 1 --2- 172.21.15.39:0/118602591 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7d31e42d0 0x55f7d31ea110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:52.840 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.822+0000 7f5e469fcb80 1 --2- 172.21.15.39:0/118602591 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f7d303a6b0 0x55f7d31eeb70 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:52.840 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.822+0000 7f5e469fcb80 1 --2- 172.21.15.39:0/118602591 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7d31de580 0x55f7d31e51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:52.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.822+0000 7f5e469fcb80 1 -- 172.21.15.39:0/118602591 >> 172.21.15.39:0/118602591 conn(0x55f7d31d3be0 msgr2=0x55f7d31e4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:00:52.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.823+0000 7f5e469fcb80 1 -- 172.21.15.39:0/118602591 shutdown_connections 2023-07-21T19:00:52.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.823+0000 7f5e469fcb80 1 -- 172.21.15.39:0/118602591 wait complete. 2023-07-21T19:00:52.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.823+0000 7f5e469fcb80 1 -- start start 2023-07-21T19:00:52.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.824+0000 7f5e469fcb80 1 Processor -- start 2023-07-21T19:00:52.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.824+0000 7f5e469fcb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f7d303a6b0 0x55f7d31eeb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:52.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.824+0000 7f5e469fcb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7d31e42d0 0x55f7d31df860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:52.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.824+0000 7f5e469fcb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7d31dfda0 0x55f7d31e0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:52.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.824+0000 7f5e469fcb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7d31ef0c0 con 0x55f7d303a6b0 2023-07-21T19:00:52.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.824+0000 7f5e469fcb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7d324b900 con 0x55f7d31dfda0 2023-07-21T19:00:52.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.824+0000 7f5e469fcb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7d324bac0 con 0x55f7d31e42d0 2023-07-21T19:00:52.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.824+0000 7f5e469fcb80 10 client.0 _ll_register_callbacks cb 0x7f5e24004660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:00:52.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.825+0000 7f5e38e13700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7d31dfda0 0x55f7d31e0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:52.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.825+0000 7f5e38e13700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7d31dfda0 0x55f7d31e0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:46722/0 (socket says 172.21.15.39:46722) 2023-07-21T19:00:52.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.825+0000 7f5e38e13700 1 -- 172.21.15.39:0/1019002702 learned_addr learned my addr 172.21.15.39:0/1019002702 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:00:52.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.825+0000 7f5e337fe700 1 --2- 172.21.15.39:0/1019002702 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7d31e42d0 0x55f7d31df860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:52.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.825+0000 7f5e38e13700 1 -- 172.21.15.39:0/1019002702 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7d31e42d0 msgr2=0x55f7d31df860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:00:52.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.825+0000 7f5e38e13700 1 --2- 172.21.15.39:0/1019002702 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7d31e42d0 0x55f7d31df860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:52.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.825+0000 7f5e38e13700 1 -- 172.21.15.39:0/1019002702 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f7d303a6b0 msgr2=0x55f7d31eeb80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T19:00:52.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.825+0000 7f5e38e13700 1 --2- 172.21.15.39:0/1019002702 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f7d303a6b0 0x55f7d31eeb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:52.844 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.825+0000 7f5e38e13700 1 -- 172.21.15.39:0/1019002702 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f7d2f46c00 con 0x55f7d31dfda0 2023-07-21T19:00:52.844 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.825+0000 7f5e337fe700 1 --2- 172.21.15.39:0/1019002702 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7d31e42d0 0x55f7d31df860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:00:52.844 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.825+0000 7f5e38e13700 1 --2- 172.21.15.39:0/1019002702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7d31dfda0 0x55f7d31e0210 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c009cd0 tx=0x7f5e2c0076d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46b6496cb8450dfe server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:00:52.844 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.825+0000 7f5e137fe700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-21T19:00:52.844 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.826+0000 7f5e137fe700 1 -- 172.21.15.39:0/1019002702 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c00e040 con 0x55f7d31dfda0 2023-07-21T19:00:52.844 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.826+0000 7f5e137fe700 1 -- 172.21.15.39:0/1019002702 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f5e2c02a070 con 0x55f7d31dfda0 2023-07-21T19:00:52.844 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.826+0000 7f5e469fcb80 1 -- 172.21.15.39:0/1019002702 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5e2c0060a0 con 0x55f7d31dfda0 2023-07-21T19:00:52.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.826+0000 7f5e137fe700 1 -- 172.21.15.39:0/1019002702 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=100) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e2c0078f0 con 0x55f7d31dfda0 2023-07-21T19:00:52.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.827+0000 7f5e469fcb80 10 client.27005 fetch_fsmap learned FSMap version 100 2023-07-21T19:00:52.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.827+0000 7f5e469fcb80 1 -- 172.21.15.39:0/1019002702 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=100,monmap=4+}) v3 -- 0x55f7d2f841f0 con 0x55f7d31dfda0 2023-07-21T19:00:52.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.827+0000 7f5e137fe700 1 -- 172.21.15.39:0/1019002702 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e2c0060a0 con 0x55f7d31dfda0 2023-07-21T19:00:52.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.827+0000 7f5e137fe700 1 -- 172.21.15.39:0/1019002702 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 100) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f5e2c00b100 con 0x55f7d31dfda0 2023-07-21T19:00:52.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.827+0000 7f5e469fcb80 10 client.27005 fetch_fsmap finished waiting for FSMap version 100 2023-07-21T19:00:52.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.827+0000 7f5e469fcb80 10 client.27005 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:00:52.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.827+0000 7f5e469fcb80 20 client.27005 populate_metadata read hostname 'smithi039' 2023-07-21T19:00:52.846 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.827+0000 7f5e469fcb80 1 --2- 172.21.15.39:0/1019002702 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55f7d325dc00 0x55f7d3260010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:52.846 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.827+0000 7f5e469fcb80 4 client.27005 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:00:52.846 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.827+0000 7f5e469fcb80 1 -- 172.21.15.39:0/1019002702 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f7d30eaff0 con 0x55f7d325dc00 2023-07-21T19:00:52.846 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.832+0000 7f5e33fff700 1 --2- 172.21.15.39:0/1019002702 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55f7d325dc00 0x55f7d3260010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:52.847 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.832+0000 7f5e137fe700 10 client.27005 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T19:00:52.847 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:52.832+0000 7f5e33fff700 1 --2- 172.21.15.39:0/1019002702 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55f7d325dc00 0x55f7d3260010 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3068c7101c276f94 server_cookie=8cb34f136d4e3201 in_seq=0 out_seq=0 2023-07-21T19:00:53.151 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:53.141+0000 7f5e137fe700 1 -- 172.21.15.39:0/1019002702 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f7d30eaff0 con 0x55f7d325dc00 2023-07-21T19:00:53.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:53.141+0000 7f5e137fe700 10 client.27005 handle_command_reply: tid=0 2023-07-21T19:00:53.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:53.141+0000 7f5e469fcb80 1 client.27005 shutdown 2023-07-21T19:00:53.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:53.142+0000 7f5e469fcb80 1 -- 172.21.15.39:0/1019002702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7d31dfda0 msgr2=0x55f7d31e0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:00:53.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:53.142+0000 7f5e469fcb80 1 --2- 172.21.15.39:0/1019002702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7d31dfda0 0x55f7d31e0210 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c009cd0 tx=0x7f5e2c0076d0 comp rx=0 tx=0).stop 2023-07-21T19:00:53.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:53.143+0000 7f5e469fcb80 1 -- 172.21.15.39:0/1019002702 shutdown_connections 2023-07-21T19:00:53.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:53.143+0000 7f5e469fcb80 1 --2- 172.21.15.39:0/1019002702 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55f7d325dc00 0x55f7d3260010 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:53.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:53.143+0000 7f5e469fcb80 1 --2- 172.21.15.39:0/1019002702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7d31dfda0 0x55f7d31e0210 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:53.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:53.143+0000 7f5e469fcb80 1 --2- 172.21.15.39:0/1019002702 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f7d303a6b0 0x55f7d31eeb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:53.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:53.143+0000 7f5e469fcb80 1 --2- 172.21.15.39:0/1019002702 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7d31e42d0 0x55f7d31df860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:53.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:53.143+0000 7f5e469fcb80 1 -- 172.21.15.39:0/1019002702 >> 172.21.15.39:0/1019002702 conn(0x55f7d31d3be0 msgr2=0x55f7d31e5320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:00:53.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:53.143+0000 7f5e137fe700 0 client.27005 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T19:00:53.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:53.143+0000 7f5e469fcb80 1 -- 172.21.15.39:0/1019002702 shutdown_connections 2023-07-21T19:00:53.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:53.144+0000 7f5e469fcb80 1 -- 172.21.15.39:0/1019002702 wait complete. 2023-07-21T19:00:53.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:53.144+0000 7f5e469fcb80 20 client.27005 trim_cache size 0 max 16384 2023-07-21T19:00:53.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T19:00:53.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T19:00:53.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T19:00:53.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T19:00:53.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:53.147+0000 7f5e3a876700 1 -- 172.21.15.39:0/2871568523 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f5e1c0a12c0 msgr2=0x7f5e1c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:00:53.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:53.147+0000 7f5e3a876700 1 --2- 172.21.15.39:0/2871568523 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f5e1c0a12c0 0x7f5e1c0a3770 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5e24011490 tx=0x7f5e24009260 comp rx=0 tx=0).stop 2023-07-21T19:00:53.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:53.147+0000 7f5e3a876700 1 -- 172.21.15.39:0/2871568523 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5e340ff920 msgr2=0x7f5e341005b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:00:53.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:53.147+0000 7f5e3a876700 1 --2- 172.21.15.39:0/2871568523 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5e340ff920 0x7f5e341005b0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f5e28009fd0 tx=0x7f5e2800ed10 comp rx=0 tx=0).stop 2023-07-21T19:00:53.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:53.147+0000 7f5e3a876700 1 -- 172.21.15.39:0/2871568523 shutdown_connections 2023-07-21T19:00:53.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:53.147+0000 7f5e3a876700 1 --2- 172.21.15.39:0/2871568523 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f5e1c0a12c0 0x7f5e1c0a3770 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:53.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:53.147+0000 7f5e3a876700 1 --2- 172.21.15.39:0/2871568523 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e340fef80 0x7f5e34100070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:53.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:53.147+0000 7f5e3a876700 1 --2- 172.21.15.39:0/2871568523 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e341034d0 0x7f5e341a3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:53.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:53.147+0000 7f5e3a876700 1 --2- 172.21.15.39:0/2871568523 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5e340ff920 0x7f5e341005b0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:53.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:53.147+0000 7f5e3a876700 1 -- 172.21.15.39:0/2871568523 >> 172.21.15.39:0/2871568523 conn(0x7f5e340752a0 msgr2=0x7f5e3410b880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:00:53.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:53.147+0000 7f5e3a876700 1 -- 172.21.15.39:0/2871568523 shutdown_connections 2023-07-21T19:00:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:00:53.147+0000 7f5e3a876700 1 -- 172.21.15.39:0/2871568523 wait complete. 2023-07-21T19:00:53.695 INFO:tasks.workunit.client.0.smithi039.stdout: 10 206945 30.60 MB/sec execute 246 sec latency 246.026 ms 2023-07-21T19:00:54.607 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:00:54 smithi193 ceph-mon[98204]: pgmap v607: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 23 MiB/s wr, 1.04k op/s 2023-07-21T19:00:54.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:00:54 smithi155 ceph-mon[97934]: pgmap v607: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 23 MiB/s wr, 1.04k op/s 2023-07-21T19:00:54.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:00:54 smithi039 ceph-mon[96489]: pgmap v607: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 23 MiB/s wr, 1.04k op/s 2023-07-21T19:00:54.695 INFO:tasks.workunit.client.0.smithi039.stdout: 10 207680 30.62 MB/sec execute 247 sec latency 66.104 ms 2023-07-21T19:00:55.695 INFO:tasks.workunit.client.0.smithi039.stdout: 10 208414 30.70 MB/sec execute 248 sec latency 64.775 ms 2023-07-21T19:00:56.607 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:00:56 smithi193 ceph-mon[98204]: pgmap v608: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 7.0 MiB/s wr, 464 op/s 2023-07-21T19:00:56.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:00:56 smithi155 ceph-mon[97934]: pgmap v608: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 7.0 MiB/s wr, 464 op/s 2023-07-21T19:00:56.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:00:56 smithi039 ceph-mon[96489]: pgmap v608: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 7.0 MiB/s wr, 464 op/s 2023-07-21T19:00:56.695 INFO:tasks.workunit.client.0.smithi039.stdout: 10 209018 30.72 MB/sec execute 249 sec latency 81.155 ms 2023-07-21T19:00:57.695 INFO:tasks.workunit.client.0.smithi039.stdout: 10 209607 30.72 MB/sec execute 250 sec latency 59.720 ms 2023-07-21T19:00:58.607 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:00:58 smithi193 ceph-mon[98204]: pgmap v609: 97 pgs: 97 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 11 MiB/s wr, 738 op/s 2023-07-21T19:00:58.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:00:58 smithi155 ceph-mon[97934]: pgmap v609: 97 pgs: 97 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 11 MiB/s wr, 738 op/s 2023-07-21T19:00:58.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:00:58 smithi039 ceph-mon[96489]: pgmap v609: 97 pgs: 97 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 11 MiB/s wr, 738 op/s 2023-07-21T19:00:58.695 INFO:tasks.workunit.client.0.smithi039.stdout: 10 210132 30.69 MB/sec execute 251 sec latency 93.711 ms 2023-07-21T19:00:59.695 INFO:tasks.workunit.client.0.smithi039.stdout: 10 210813 30.70 MB/sec execute 252 sec latency 58.631 ms 2023-07-21T19:01:00.607 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:01:00 smithi193 ceph-mon[98204]: pgmap v610: 97 pgs: 97 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.1 MiB/s wr, 498 op/s 2023-07-21T19:01:00.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:01:00 smithi155 ceph-mon[97934]: pgmap v610: 97 pgs: 97 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.1 MiB/s wr, 498 op/s 2023-07-21T19:01:00.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:01:00 smithi039 ceph-mon[96489]: pgmap v610: 97 pgs: 97 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.1 MiB/s wr, 498 op/s 2023-07-21T19:01:00.695 INFO:tasks.workunit.client.0.smithi039.stdout: 10 211461 30.73 MB/sec execute 253 sec latency 88.295 ms 2023-07-21T19:01:01.695 INFO:tasks.workunit.client.0.smithi039.stdout: 10 211846 30.65 MB/sec execute 254 sec latency 142.678 ms 2023-07-21T19:01:02.354 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:01:02 smithi193 ceph-mon[98204]: pgmap v611: 97 pgs: 97 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.4 MiB/s wr, 527 op/s 2023-07-21T19:01:02.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:01:02 smithi155 ceph-mon[97934]: pgmap v611: 97 pgs: 97 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.4 MiB/s wr, 527 op/s 2023-07-21T19:01:02.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:01:02 smithi039 ceph-mon[96489]: pgmap v611: 97 pgs: 97 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.4 MiB/s wr, 527 op/s 2023-07-21T19:01:02.695 INFO:tasks.workunit.client.0.smithi039.stdout: 10 211990 30.59 MB/sec execute 255 sec latency 212.257 ms 2023-07-21T19:01:03.695 INFO:tasks.workunit.client.0.smithi039.stdout: 10 212383 30.58 MB/sec execute 256 sec latency 193.146 ms 2023-07-21T19:01:04.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:01:04 smithi193 ceph-mon[98204]: pgmap v612: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 9.9 MiB/s wr, 694 op/s 2023-07-21T19:01:04.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:01:04 smithi155 ceph-mon[97934]: pgmap v612: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 9.9 MiB/s wr, 694 op/s 2023-07-21T19:01:04.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:01:04 smithi039 ceph-mon[96489]: pgmap v612: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 9.9 MiB/s wr, 694 op/s 2023-07-21T19:01:04.695 INFO:tasks.workunit.client.0.smithi039.stdout: 10 212867 30.56 MB/sec execute 257 sec latency 293.675 ms 2023-07-21T19:01:05.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:01:05 smithi039 ceph-mon[96489]: osdmap e113: 12 total, 12 up, 12 in 2023-07-21T19:01:05.695 INFO:tasks.workunit.client.0.smithi039.stdout: 10 213020 30.47 MB/sec execute 258 sec latency 452.633 ms 2023-07-21T19:01:05.859 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:01:05 smithi193 ceph-mon[98204]: osdmap e113: 12 total, 12 up, 12 in 2023-07-21T19:01:05.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:01:05 smithi155 ceph-mon[97934]: osdmap e113: 12 total, 12 up, 12 in 2023-07-21T19:01:06.352 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.340+0000 7fb08243b700 1 -- 172.21.15.39:0/2913032317 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb07c067fa0 msgr2=0x7fb07c1043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:01:06.353 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.340+0000 7fb08243b700 1 --2- 172.21.15.39:0/2913032317 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb07c067fa0 0x7fb07c1043d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb06c009990 tx=0x7fb06c009ca0 comp rx=0 tx=0).stop 2023-07-21T19:01:06.353 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.340+0000 7fb08243b700 1 -- 172.21.15.39:0/2913032317 shutdown_connections 2023-07-21T19:01:06.353 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.340+0000 7fb08243b700 1 --2- 172.21.15.39:0/2913032317 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb07c0fed20 0x7fb07c0ff1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:06.353 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.340+0000 7fb08243b700 1 --2- 172.21.15.39:0/2913032317 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb07c067fa0 0x7fb07c1043d0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:06.353 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.340+0000 7fb08243b700 1 --2- 172.21.15.39:0/2913032317 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb07c068400 0x7fb07c104910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:06.353 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.341+0000 7fb08243b700 1 -- 172.21.15.39:0/2913032317 >> 172.21.15.39:0/2913032317 conn(0x7fb07c075270 msgr2=0x7fb07c075670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:01:06.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.344+0000 7fb08243b700 1 -- 172.21.15.39:0/2913032317 shutdown_connections 2023-07-21T19:01:06.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.344+0000 7fb08243b700 1 -- 172.21.15.39:0/2913032317 wait complete. 2023-07-21T19:01:06.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.344+0000 7fb08243b700 1 Processor -- start 2023-07-21T19:01:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.345+0000 7fb08243b700 1 -- start start 2023-07-21T19:01:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.345+0000 7fb08243b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb07c067fa0 0x7fb07c10e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.345+0000 7fb08243b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb07c068400 0x7fb07c10ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.345+0000 7fb08243b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb07c0fed20 0x7fb07c1a8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.345+0000 7fb08243b700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07c10f2a0 con 0x7fb07c067fa0 2023-07-21T19:01:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.345+0000 7fb08243b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07c197d30 con 0x7fb07c068400 2023-07-21T19:01:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.345+0000 7fb08243b700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07c197ef0 con 0x7fb07c0fed20 2023-07-21T19:01:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.345+0000 7fb0809d8700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb07c0fed20 0x7fb07c1a8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:01:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.345+0000 7fb0809d8700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb07c0fed20 0x7fb07c1a8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:52690/0 (socket says 172.21.15.39:52690) 2023-07-21T19:01:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.345+0000 7fb0809d8700 1 -- 172.21.15.39:0/2018889166 learned_addr learned my addr 172.21.15.39:0/2018889166 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:01:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.345+0000 7fb07b7fe700 1 --2- 172.21.15.39:0/2018889166 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb07c068400 0x7fb07c10ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:01:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.346+0000 7fb0809d8700 1 -- 172.21.15.39:0/2018889166 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb07c068400 msgr2=0x7fb07c10ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:01:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.346+0000 7fb0809d8700 1 --2- 172.21.15.39:0/2018889166 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb07c068400 0x7fb07c10ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:06.359 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.346+0000 7fb0809d8700 1 -- 172.21.15.39:0/2018889166 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb07c067fa0 msgr2=0x7fb07c10e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:01:06.359 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.346+0000 7fb0809d8700 1 --2- 172.21.15.39:0/2018889166 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb07c067fa0 0x7fb07c10e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:06.359 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.346+0000 7fb0809d8700 1 -- 172.21.15.39:0/2018889166 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb06c0095f0 con 0x7fb07c0fed20 2023-07-21T19:01:06.359 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.346+0000 7fb0809d8700 1 --2- 172.21.15.39:0/2018889166 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb07c0fed20 0x7fb07c1a8590 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb07400b970 tx=0x7fb07400bc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:01:06.359 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.347+0000 7fb0797fa700 1 -- 172.21.15.39:0/2018889166 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb07400c930 con 0x7fb07c0fed20 2023-07-21T19:01:06.360 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.347+0000 7fb0797fa700 1 -- 172.21.15.39:0/2018889166 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb074014440 con 0x7fb07c0fed20 2023-07-21T19:01:06.360 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.347+0000 7fb0797fa700 1 -- 172.21.15.39:0/2018889166 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0740075b0 con 0x7fb07c0fed20 2023-07-21T19:01:06.360 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.348+0000 7fb08243b700 1 -- 172.21.15.39:0/2018889166 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb07c1981d0 con 0x7fb07c0fed20 2023-07-21T19:01:06.360 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.348+0000 7fb08243b700 1 -- 172.21.15.39:0/2018889166 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb07c198660 con 0x7fb07c0fed20 2023-07-21T19:01:06.365 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.354+0000 7fb0797fa700 1 -- 172.21.15.39:0/2018889166 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fb0740145b0 con 0x7fb07c0fed20 2023-07-21T19:01:06.366 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.355+0000 7fb0797fa700 1 --2- 172.21.15.39:0/2018889166 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fb0640a14d0 0x7fb0640a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:06.366 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.355+0000 7fb0797fa700 1 -- 172.21.15.39:0/2018889166 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fb0740cbe70 con 0x7fb07c0fed20 2023-07-21T19:01:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.355+0000 7fb07bfff700 1 --2- 172.21.15.39:0/2018889166 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fb0640a14d0 0x7fb0640a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:01:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.356+0000 7fb07bfff700 1 --2- 172.21.15.39:0/2018889166 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fb0640a14d0 0x7fb0640a3980 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb06c0098e0 tx=0x7fb06c005fb0 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:01:06.368 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.357+0000 7fb08e5c1b80 1 Processor -- start 2023-07-21T19:01:06.368 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.357+0000 7fb08e5c1b80 1 -- start start 2023-07-21T19:01:06.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.358+0000 7fb08e5c1b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f28af66b0 0x562f28c9fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:06.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.358+0000 7fb08e5c1b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f28ca02d0 0x562f28c9a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:06.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.358+0000 7fb08e5c1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562f28c9a580 0x562f28c9a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:06.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.358+0000 7fb08e5c1b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f28b9fc50 con 0x562f28ca02d0 2023-07-21T19:01:06.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.358+0000 7fb08e5c1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f28bf29e0 con 0x562f28c9a580 2023-07-21T19:01:06.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.358+0000 7fb08e5c1b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f28bebe70 con 0x562f28af66b0 2023-07-21T19:01:06.370 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.358+0000 7fb0809d8700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562f28c9a580 0x562f28c9a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:01:06.370 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.358+0000 7fb0809d8700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562f28c9a580 0x562f28c9a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:46738/0 (socket says 172.21.15.39:46738) 2023-07-21T19:01:06.370 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.358+0000 7fb0809d8700 1 -- 172.21.15.39:0/670482233 learned_addr learned my addr 172.21.15.39:0/670482233 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:01:06.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.358+0000 7fb07bfff700 1 --2- 172.21.15.39:0/670482233 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f28af66b0 0x562f28c9fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:01:06.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.358+0000 7fb0809d8700 1 -- 172.21.15.39:0/670482233 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f28af66b0 msgr2=0x562f28c9fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:01:06.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.358+0000 7fb0809d8700 1 --2- 172.21.15.39:0/670482233 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f28af66b0 0x562f28c9fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:06.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.358+0000 7fb0809d8700 1 -- 172.21.15.39:0/670482233 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f28ca02d0 msgr2=0x562f28c9a040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T19:01:06.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.359+0000 7fb0809d8700 1 --2- 172.21.15.39:0/670482233 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f28ca02d0 0x562f28c9a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:06.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.359+0000 7fb0809d8700 1 -- 172.21.15.39:0/670482233 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f28a19d80 con 0x562f28c9a580 2023-07-21T19:01:06.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.359+0000 7fb0809d8700 1 --2- 172.21.15.39:0/670482233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562f28c9a580 0x562f28c9a9d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb0740954c0 tx=0x7fb0740989f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=636315876ea9fc36 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:01:06.374 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.360+0000 7fb0627fc700 1 -- 172.21.15.39:0/670482233 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0740a1410 con 0x562f28c9a580 2023-07-21T19:01:06.375 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.360+0000 7fb0627fc700 1 -- 172.21.15.39:0/670482233 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0740148b0 con 0x562f28c9a580 2023-07-21T19:01:06.375 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.360+0000 7fb0627fc700 1 -- 172.21.15.39:0/670482233 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb074093bd0 con 0x562f28c9a580 2023-07-21T19:01:06.375 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.361+0000 7fb08e5c1b80 1 -- 172.21.15.39:0/670482233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562f28c9a580 msgr2=0x562f28c9a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:01:06.375 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.361+0000 7fb08e5c1b80 1 --2- 172.21.15.39:0/670482233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562f28c9a580 0x562f28c9a9d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb0740954c0 tx=0x7fb0740989f0 comp rx=0 tx=0).stop 2023-07-21T19:01:06.376 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.363+0000 7fb08e5c1b80 1 -- 172.21.15.39:0/670482233 shutdown_connections 2023-07-21T19:01:06.376 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.363+0000 7fb08e5c1b80 1 --2- 172.21.15.39:0/670482233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562f28c9a580 0x562f28c9a9d0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:06.376 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.363+0000 7fb08e5c1b80 1 --2- 172.21.15.39:0/670482233 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f28af66b0 0x562f28c9fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:06.377 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.363+0000 7fb08e5c1b80 1 --2- 172.21.15.39:0/670482233 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f28ca02d0 0x562f28c9a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:06.377 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.363+0000 7fb08e5c1b80 1 -- 172.21.15.39:0/670482233 >> 172.21.15.39:0/670482233 conn(0x562f28c8fbe0 msgr2=0x562f28c94f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:01:06.378 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.366+0000 7fb08e5c1b80 1 -- 172.21.15.39:0/670482233 shutdown_connections 2023-07-21T19:01:06.379 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.366+0000 7fb08e5c1b80 1 -- 172.21.15.39:0/670482233 wait complete. 2023-07-21T19:01:06.379 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.367+0000 7fb08e5c1b80 1 -- start start 2023-07-21T19:01:06.379 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.367+0000 7fb08e5c1b80 1 Processor -- start 2023-07-21T19:01:06.380 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.367+0000 7fb08e5c1b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f28af66b0 0x562f28ca3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:06.380 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.368+0000 7fb08e5c1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562f28c9a580 0x562f28ca3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:06.380 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.368+0000 7fb08e5c1b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f28ca02d0 0x562f28ca43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:06.380 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.368+0000 7fb08e5c1b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f28b8ba60 con 0x562f28ca02d0 2023-07-21T19:01:06.380 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.368+0000 7fb08e5c1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f28b8b760 con 0x562f28c9a580 2023-07-21T19:01:06.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.368+0000 7fb08e5c1b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f28a6f8c0 con 0x562f28af66b0 2023-07-21T19:01:06.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.368+0000 7fb08e5c1b80 10 client.0 _ll_register_callbacks cb 0x562f28b89700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:01:06.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.369+0000 7fb07b7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562f28c9a580 0x562f28ca3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:01:06.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.369+0000 7fb0809d8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f28ca02d0 0x562f28ca43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:01:06.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.369+0000 7fb0809d8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f28ca02d0 0x562f28ca43a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34150/0 (socket says 172.21.15.39:34150) 2023-07-21T19:01:06.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.369+0000 7fb0809d8700 1 -- 172.21.15.39:0/3329911859 learned_addr learned my addr 172.21.15.39:0/3329911859 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:01:06.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.370+0000 7fb0809d8700 1 -- 172.21.15.39:0/3329911859 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f28af66b0 msgr2=0x562f28ca3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:01:06.383 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.370+0000 7fb0809d8700 1 --2- 172.21.15.39:0/3329911859 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f28af66b0 0x562f28ca3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:06.383 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.370+0000 7fb0809d8700 1 -- 172.21.15.39:0/3329911859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562f28c9a580 msgr2=0x562f28ca3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:01:06.383 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.370+0000 7fb0809d8700 1 --2- 172.21.15.39:0/3329911859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562f28c9a580 0x562f28ca3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:06.383 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.370+0000 7fb0809d8700 1 -- 172.21.15.39:0/3329911859 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb07c198660 con 0x562f28ca02d0 2023-07-21T19:01:06.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.370+0000 7fb0809d8700 1 --2- 172.21.15.39:0/3329911859 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f28ca02d0 0x562f28ca43a0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fb074098cc0 tx=0x7fb074093eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6bbe9a05ae2270f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:01:06.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.370+0000 7fb05b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T19:01:06.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.370+0000 7fb05b7fe700 1 -- 172.21.15.39:0/3329911859 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0740986c0 con 0x562f28ca02d0 2023-07-21T19:01:06.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.370+0000 7fb05b7fe700 1 -- 172.21.15.39:0/3329911859 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fb0740c9340 con 0x562f28ca02d0 2023-07-21T19:01:06.385 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.370+0000 7fb08e5c1b80 1 -- 172.21.15.39:0/3329911859 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562f28b45490 con 0x562f28ca02d0 2023-07-21T19:01:06.387 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.376+0000 7fb05b7fe700 1 -- 172.21.15.39:0/3329911859 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=100) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0740a0f80 con 0x562f28ca02d0 2023-07-21T19:01:06.390 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.379+0000 7fb08e5c1b80 10 client.17583 fetch_fsmap learned FSMap version 100 2023-07-21T19:01:06.390 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.379+0000 7fb08e5c1b80 1 -- 172.21.15.39:0/3329911859 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=100,monmap=4+}) v3 -- 0x562f28a02c00 con 0x562f28ca02d0 2023-07-21T19:01:06.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.382+0000 7fb05b7fe700 1 -- 172.21.15.39:0/3329911859 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562f28b45490 con 0x562f28ca02d0 2023-07-21T19:01:06.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.382+0000 7fb05b7fe700 1 -- 172.21.15.39:0/3329911859 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 100) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fb074097f20 con 0x562f28ca02d0 2023-07-21T19:01:06.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.382+0000 7fb08e5c1b80 10 client.17583 fetch_fsmap finished waiting for FSMap version 100 2023-07-21T19:01:06.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.382+0000 7fb08e5c1b80 10 client.17583 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:01:06.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.382+0000 7fb08e5c1b80 20 client.17583 populate_metadata read hostname 'smithi039' 2023-07-21T19:01:06.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.382+0000 7fb08e5c1b80 1 --2- 172.21.15.39:0/3329911859 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x562f28cb3920 0x562f28cb5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:06.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.382+0000 7fb08e5c1b80 4 client.17583 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:01:06.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.382+0000 7fb08e5c1b80 1 -- 172.21.15.39:0/3329911859 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562f28be4e50 con 0x562f28cb3920 2023-07-21T19:01:06.399 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.389+0000 7fb07bfff700 1 --2- 172.21.15.39:0/3329911859 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x562f28cb3920 0x562f28cb5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:01:06.400 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.389+0000 7fb07bfff700 1 --2- 172.21.15.39:0/3329911859 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x562f28cb3920 0x562f28cb5d30 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92dabe9ce2c7d290 server_cookie=aaf4830909be3826 in_seq=0 out_seq=0 2023-07-21T19:01:06.401 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.389+0000 7fb05b7fe700 10 client.17583 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T19:01:06.403 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.390+0000 7fb05b7fe700 1 -- 172.21.15.39:0/3329911859 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562f28be4e50 con 0x562f28cb3920 2023-07-21T19:01:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.390+0000 7fb05b7fe700 10 client.17583 handle_command_reply: tid=0 2023-07-21T19:01:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.390+0000 7fb08e5c1b80 1 client.17583 shutdown 2023-07-21T19:01:06.409 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.396+0000 7fb08e5c1b80 1 -- 172.21.15.39:0/3329911859 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f28ca02d0 msgr2=0x562f28ca43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:01:06.409 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.396+0000 7fb08e5c1b80 1 --2- 172.21.15.39:0/3329911859 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f28ca02d0 0x562f28ca43a0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fb074098cc0 tx=0x7fb074093eb0 comp rx=0 tx=0).stop 2023-07-21T19:01:06.409 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.396+0000 7fb08e5c1b80 1 -- 172.21.15.39:0/3329911859 shutdown_connections 2023-07-21T19:01:06.409 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.396+0000 7fb08e5c1b80 1 --2- 172.21.15.39:0/3329911859 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x562f28cb3920 0x562f28cb5d30 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:06.409 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.396+0000 7fb08e5c1b80 1 --2- 172.21.15.39:0/3329911859 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f28ca02d0 0x562f28ca43a0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:06.410 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.396+0000 7fb08e5c1b80 1 --2- 172.21.15.39:0/3329911859 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f28af66b0 0x562f28ca3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:06.410 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.396+0000 7fb08e5c1b80 1 --2- 172.21.15.39:0/3329911859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562f28c9a580 0x562f28ca3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:06.410 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.396+0000 7fb08e5c1b80 1 -- 172.21.15.39:0/3329911859 >> 172.21.15.39:0/3329911859 conn(0x562f28c8fbe0 msgr2=0x562f28c944e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:01:06.410 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.396+0000 7fb05b7fe700 0 client.17583 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T19:01:06.411 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.396+0000 7fb08e5c1b80 1 -- 172.21.15.39:0/3329911859 shutdown_connections 2023-07-21T19:01:06.411 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.396+0000 7fb08e5c1b80 1 -- 172.21.15.39:0/3329911859 wait complete. 2023-07-21T19:01:06.411 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.396+0000 7fb08e5c1b80 20 client.17583 trim_cache size 0 max 16384 2023-07-21T19:01:06.444 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.428+0000 7fb08e5c1b80 1 Processor -- start 2023-07-21T19:01:06.444 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.429+0000 7fb08e5c1b80 1 -- start start 2023-07-21T19:01:06.444 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.429+0000 7fb08e5c1b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f28af66b0 0x562f28c9a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:06.445 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.429+0000 7fb08e5c1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562f28c9a580 0x562f28caac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:06.445 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.429+0000 7fb08e5c1b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f28ca02d0 0x562f28c9f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:06.445 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.429+0000 7fb08e5c1b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f28c95d50 con 0x562f28af66b0 2023-07-21T19:01:06.446 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.429+0000 7fb08e5c1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f28c60060 con 0x562f28c9a580 2023-07-21T19:01:06.446 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.429+0000 7fb08e5c1b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f28c915f0 con 0x562f28ca02d0 2023-07-21T19:01:06.447 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.429+0000 7fb07bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f28af66b0 0x562f28c9a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:01:06.447 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.429+0000 7fb07bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f28af66b0 0x562f28c9a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34156/0 (socket says 172.21.15.39:34156) 2023-07-21T19:01:06.447 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.429+0000 7fb07bfff700 1 -- 172.21.15.39:0/4196058920 learned_addr learned my addr 172.21.15.39:0/4196058920 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:01:06.447 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.429+0000 7fb0809d8700 1 --2- 172.21.15.39:0/4196058920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f28ca02d0 0x562f28c9f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:01:06.448 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.430+0000 7fb07bfff700 1 -- 172.21.15.39:0/4196058920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f28ca02d0 msgr2=0x562f28c9f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:01:06.448 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.430+0000 7fb07bfff700 1 --2- 172.21.15.39:0/4196058920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f28ca02d0 0x562f28c9f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:06.448 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.430+0000 7fb07bfff700 1 -- 172.21.15.39:0/4196058920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562f28c9a580 msgr2=0x562f28caac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:01:06.448 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.430+0000 7fb07bfff700 1 --2- 172.21.15.39:0/4196058920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562f28c9a580 0x562f28caac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:06.448 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.430+0000 7fb07bfff700 1 -- 172.21.15.39:0/4196058920 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f28a401f0 con 0x562f28af66b0 2023-07-21T19:01:06.448 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.430+0000 7fb07bfff700 1 --2- 172.21.15.39:0/4196058920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f28af66b0 0x562f28c9a040 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fb06c009990 tx=0x7fb06c01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7245154e620ba35 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:01:06.449 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.431+0000 7fb062ffd700 1 -- 172.21.15.39:0/4196058920 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb06c029030 con 0x562f28af66b0 2023-07-21T19:01:06.450 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.431+0000 7fb062ffd700 1 -- 172.21.15.39:0/4196058920 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb06c02a040 con 0x562f28af66b0 2023-07-21T19:01:06.450 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.431+0000 7fb062ffd700 1 -- 172.21.15.39:0/4196058920 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb06c01d070 con 0x562f28af66b0 2023-07-21T19:01:06.450 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.432+0000 7fb08e5c1b80 1 -- 172.21.15.39:0/4196058920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f28af66b0 msgr2=0x562f28c9a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:01:06.450 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.432+0000 7fb08e5c1b80 1 --2- 172.21.15.39:0/4196058920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f28af66b0 0x562f28c9a040 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fb06c009990 tx=0x7fb06c01d000 comp rx=0 tx=0).stop 2023-07-21T19:01:06.450 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.433+0000 7fb08e5c1b80 1 -- 172.21.15.39:0/4196058920 shutdown_connections 2023-07-21T19:01:06.451 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.433+0000 7fb08e5c1b80 1 --2- 172.21.15.39:0/4196058920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f28ca02d0 0x562f28c9f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:06.451 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.433+0000 7fb08e5c1b80 1 --2- 172.21.15.39:0/4196058920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f28af66b0 0x562f28c9a040 secure :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fb06c009990 tx=0x7fb06c01d000 comp rx=0 tx=0).stop 2023-07-21T19:01:06.451 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.433+0000 7fb08e5c1b80 1 --2- 172.21.15.39:0/4196058920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562f28c9a580 0x562f28caac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:06.451 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.433+0000 7fb08e5c1b80 1 -- 172.21.15.39:0/4196058920 >> 172.21.15.39:0/4196058920 conn(0x562f28c8fbe0 msgr2=0x562f28c9eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:01:06.452 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.436+0000 7fb08e5c1b80 1 -- 172.21.15.39:0/4196058920 shutdown_connections 2023-07-21T19:01:06.452 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.436+0000 7fb08e5c1b80 1 -- 172.21.15.39:0/4196058920 wait complete. 2023-07-21T19:01:06.452 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.437+0000 7fb08e5c1b80 1 -- start start 2023-07-21T19:01:06.452 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.437+0000 7fb08e5c1b80 1 Processor -- start 2023-07-21T19:01:06.452 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.437+0000 7fb08e5c1b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f28c9a580 0x562f28caa8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:06.452 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.437+0000 7fb08e5c1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562f28ca02d0 0x562f28caae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:06.452 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.437+0000 7fb08e5c1b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f28ca8cf0 0x562f28ca9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:06.453 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.437+0000 7fb08e5c1b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f28ca9870 con 0x562f28c9a580 2023-07-21T19:01:06.453 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.437+0000 7fb08e5c1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f28ca99e0 con 0x562f28ca02d0 2023-07-21T19:01:06.453 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.437+0000 7fb08e5c1b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f28d077f0 con 0x562f28ca8cf0 2023-07-21T19:01:06.453 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.437+0000 7fb08e5c1b80 10 client.0 _ll_register_callbacks cb 0x562f28b89700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:01:06.453 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.438+0000 7fb0809d8700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f28ca8cf0 0x562f28ca9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:01:06.453 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.438+0000 7fb0809d8700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f28ca8cf0 0x562f28ca9160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:52714/0 (socket says 172.21.15.39:52714) 2023-07-21T19:01:06.453 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.438+0000 7fb0809d8700 1 -- 172.21.15.39:0/3642178278 learned_addr learned my addr 172.21.15.39:0/3642178278 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:01:06.453 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.439+0000 7fb0809d8700 1 -- 172.21.15.39:0/3642178278 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562f28ca02d0 msgr2=0x562f28caae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:01:06.454 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.439+0000 7fb0809d8700 1 --2- 172.21.15.39:0/3642178278 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562f28ca02d0 0x562f28caae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:06.454 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.439+0000 7fb0809d8700 1 -- 172.21.15.39:0/3642178278 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f28c9a580 msgr2=0x562f28caa8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:01:06.454 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.439+0000 7fb0809d8700 1 --2- 172.21.15.39:0/3642178278 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f28c9a580 0x562f28caa8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:06.454 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.439+0000 7fb0809d8700 1 -- 172.21.15.39:0/3642178278 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb07c105110 con 0x562f28ca8cf0 2023-07-21T19:01:06.455 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.442+0000 7fb0809d8700 1 --2- 172.21.15.39:0/3642178278 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f28ca8cf0 0x562f28ca9160 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb07400bda0 tx=0x7fb0740989f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce2bee9193cace87 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:01:06.456 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.444+0000 7fb05b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2023-07-21T19:01:06.456 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.444+0000 7fb05b7fe700 1 -- 172.21.15.39:0/3642178278 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0740a0f80 con 0x562f28ca8cf0 2023-07-21T19:01:06.456 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.445+0000 7fb05b7fe700 1 -- 172.21.15.39:0/3642178278 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fb0740a10e0 con 0x562f28ca8cf0 2023-07-21T19:01:06.457 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.445+0000 7fb08e5c1b80 1 -- 172.21.15.39:0/3642178278 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb0740148b0 con 0x562f28ca8cf0 2023-07-21T19:01:06.457 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.445+0000 7fb05b7fe700 1 -- 172.21.15.39:0/3642178278 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=100) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0740b7120 con 0x562f28ca8cf0 2023-07-21T19:01:06.457 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.445+0000 7fb08e5c1b80 10 client.27073 fetch_fsmap learned FSMap version 100 2023-07-21T19:01:06.457 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.445+0000 7fb08e5c1b80 1 -- 172.21.15.39:0/3642178278 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=100,monmap=4+}) v3 -- 0x562f28a18250 con 0x562f28ca8cf0 2023-07-21T19:01:06.457 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.445+0000 7fb05b7fe700 1 -- 172.21.15.39:0/3642178278 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0740148b0 con 0x562f28ca8cf0 2023-07-21T19:01:06.457 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.445+0000 7fb05b7fe700 1 -- 172.21.15.39:0/3642178278 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 100) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fb074099da0 con 0x562f28ca8cf0 2023-07-21T19:01:06.457 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.445+0000 7fb08e5c1b80 10 client.27073 fetch_fsmap finished waiting for FSMap version 100 2023-07-21T19:01:06.458 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.445+0000 7fb08e5c1b80 10 client.27073 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:01:06.458 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.445+0000 7fb08e5c1b80 20 client.27073 populate_metadata read hostname 'smithi039' 2023-07-21T19:01:06.458 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.445+0000 7fb08e5c1b80 1 --2- 172.21.15.39:0/3642178278 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x562f28d08980 0x562f28d0ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:06.458 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.445+0000 7fb08e5c1b80 4 client.27073 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:01:06.458 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.445+0000 7fb08e5c1b80 1 -- 172.21.15.39:0/3642178278 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562f28ba6ff0 con 0x562f28d08980 2023-07-21T19:01:06.459 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.448+0000 7fb07bfff700 1 --2- 172.21.15.39:0/3642178278 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x562f28d08980 0x562f28d0ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:01:06.468 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.449+0000 7fb07bfff700 1 --2- 172.21.15.39:0/3642178278 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x562f28d08980 0x562f28d0ad60 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5ad24eb7df3e5b5 server_cookie=c784bd52995815 in_seq=0 out_seq=0 2023-07-21T19:01:06.468 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.449+0000 7fb05b7fe700 10 client.27073 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T19:01:06.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:01:06 smithi039 ceph-mon[96489]: pgmap v614: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.1 MiB/s wr, 584 op/s 2023-07-21T19:01:06.695 INFO:tasks.workunit.client.0.smithi039.stdout: 10 213270 30.39 MB/sec execute 259 sec latency 260.972 ms 2023-07-21T19:01:06.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.698+0000 7fb05b7fe700 1 -- 172.21.15.39:0/3642178278 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562f28ba6ff0 con 0x562f28d08980 2023-07-21T19:01:06.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.698+0000 7fb05b7fe700 10 client.27073 handle_command_reply: tid=0 2023-07-21T19:01:06.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.699+0000 7fb08e5c1b80 1 client.27073 shutdown 2023-07-21T19:01:06.710 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.699+0000 7fb08e5c1b80 1 -- 172.21.15.39:0/3642178278 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f28ca8cf0 msgr2=0x562f28ca9160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:01:06.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.699+0000 7fb08e5c1b80 1 --2- 172.21.15.39:0/3642178278 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f28ca8cf0 0x562f28ca9160 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb07400bda0 tx=0x7fb0740989f0 comp rx=0 tx=0).stop 2023-07-21T19:01:06.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.700+0000 7fb08e5c1b80 1 -- 172.21.15.39:0/3642178278 shutdown_connections 2023-07-21T19:01:06.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.700+0000 7fb08e5c1b80 1 --2- 172.21.15.39:0/3642178278 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x562f28d08980 0x562f28d0ad60 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:06.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.700+0000 7fb08e5c1b80 1 --2- 172.21.15.39:0/3642178278 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f28ca8cf0 0x562f28ca9160 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:06.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.700+0000 7fb08e5c1b80 1 --2- 172.21.15.39:0/3642178278 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f28c9a580 0x562f28caa8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:06.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.700+0000 7fb08e5c1b80 1 --2- 172.21.15.39:0/3642178278 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562f28ca02d0 0x562f28caae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.700+0000 7fb08e5c1b80 1 -- 172.21.15.39:0/3642178278 >> 172.21.15.39:0/3642178278 conn(0x562f28c8fbe0 msgr2=0x562f28ca2980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:01:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.700+0000 7fb05b7fe700 0 client.27073 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T19:01:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.700+0000 7fb08e5c1b80 1 -- 172.21.15.39:0/3642178278 shutdown_connections 2023-07-21T19:01:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.702+0000 7fb08e5c1b80 1 -- 172.21.15.39:0/3642178278 wait complete. 2023-07-21T19:01:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.702+0000 7fb08e5c1b80 20 client.27073 trim_cache size 0 max 16384 2023-07-21T19:01:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T19:01:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T19:01:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T19:01:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T19:01:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.703+0000 7fb08243b700 1 -- 172.21.15.39:0/2018889166 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fb0640a14d0 msgr2=0x7fb0640a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:01:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.703+0000 7fb08243b700 1 --2- 172.21.15.39:0/2018889166 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fb0640a14d0 0x7fb0640a3980 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb06c0098e0 tx=0x7fb06c005fb0 comp rx=0 tx=0).stop 2023-07-21T19:01:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.703+0000 7fb08243b700 1 -- 172.21.15.39:0/2018889166 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb07c0fed20 msgr2=0x7fb07c1a8590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:01:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.703+0000 7fb08243b700 1 --2- 172.21.15.39:0/2018889166 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb07c0fed20 0x7fb07c1a8590 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb07400b970 tx=0x7fb07400bc80 comp rx=0 tx=0).stop 2023-07-21T19:01:06.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.704+0000 7fb08243b700 1 -- 172.21.15.39:0/2018889166 shutdown_connections 2023-07-21T19:01:06.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.704+0000 7fb08243b700 1 --2- 172.21.15.39:0/2018889166 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb07c0fed20 0x7fb07c1a8590 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:06.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.704+0000 7fb08243b700 1 --2- 172.21.15.39:0/2018889166 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb07c067fa0 0x7fb07c10e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:06.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.704+0000 7fb08243b700 1 --2- 172.21.15.39:0/2018889166 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fb0640a14d0 0x7fb0640a3980 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:06.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.704+0000 7fb08243b700 1 --2- 172.21.15.39:0/2018889166 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb07c068400 0x7fb07c10ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:06.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.704+0000 7fb08243b700 1 -- 172.21.15.39:0/2018889166 >> 172.21.15.39:0/2018889166 conn(0x7fb07c075270 msgr2=0x7fb07c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:01:06.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.704+0000 7fb08243b700 1 -- 172.21.15.39:0/2018889166 shutdown_connections 2023-07-21T19:01:06.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:06.704+0000 7fb08243b700 1 -- 172.21.15.39:0/2018889166 wait complete. 2023-07-21T19:01:06.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:01:06 smithi193 ceph-mon[98204]: pgmap v614: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.1 MiB/s wr, 584 op/s 2023-07-21T19:01:06.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:01:06 smithi155 ceph-mon[97934]: pgmap v614: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.1 MiB/s wr, 584 op/s 2023-07-21T19:01:07.695 INFO:tasks.workunit.client.0.smithi039.stdout: 10 213898 30.43 MB/sec execute 260 sec latency 92.124 ms 2023-07-21T19:01:08.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:01:08 smithi039 ceph-mon[96489]: pgmap v615: 97 pgs: 17 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 45 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 6.7 MiB/s wr, 654 op/s 2023-07-21T19:01:08.695 INFO:tasks.workunit.client.0.smithi039.stdout: 10 214524 30.41 MB/sec execute 261 sec latency 57.957 ms 2023-07-21T19:01:08.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:01:08 smithi193 ceph-mon[98204]: pgmap v615: 97 pgs: 17 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 45 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 6.7 MiB/s wr, 654 op/s 2023-07-21T19:01:08.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:01:08 smithi155 ceph-mon[97934]: pgmap v615: 97 pgs: 17 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 45 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 6.7 MiB/s wr, 654 op/s 2023-07-21T19:01:09.695 INFO:tasks.workunit.client.0.smithi039.stdout: 10 215048 30.41 MB/sec execute 262 sec latency 91.826 ms 2023-07-21T19:01:10.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:01:10 smithi039 ceph-mon[96489]: pgmap v616: 97 pgs: 17 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 45 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 6.7 MiB/s wr, 654 op/s 2023-07-21T19:01:10.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:01:10 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T19:01:10.695 INFO:tasks.workunit.client.0.smithi039.stdout: 10 215586 30.42 MB/sec execute 263 sec latency 76.554 ms 2023-07-21T19:01:10.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:01:10 smithi193 ceph-mon[98204]: pgmap v616: 97 pgs: 17 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 45 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 6.7 MiB/s wr, 654 op/s 2023-07-21T19:01:10.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:01:10 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T19:01:10.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:01:10 smithi155 ceph-mon[97934]: pgmap v616: 97 pgs: 17 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 45 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 6.7 MiB/s wr, 654 op/s 2023-07-21T19:01:10.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:01:10 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T19:01:11.695 INFO:tasks.workunit.client.0.smithi039.stdout: 10 216180 30.41 MB/sec execute 264 sec latency 110.235 ms 2023-07-21T19:01:12.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:01:12 smithi039 ceph-mon[96489]: pgmap v617: 97 pgs: 15 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 50 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 6.6 MiB/s wr, 660 op/s 2023-07-21T19:01:12.695 INFO:tasks.workunit.client.0.smithi039.stdout: 10 216909 30.45 MB/sec execute 265 sec latency 109.312 ms 2023-07-21T19:01:12.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:01:12 smithi193 ceph-mon[98204]: pgmap v617: 97 pgs: 15 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 50 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 6.6 MiB/s wr, 660 op/s 2023-07-21T19:01:12.878 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:01:12 smithi155 ceph-mon[97934]: pgmap v617: 97 pgs: 15 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 50 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 6.6 MiB/s wr, 660 op/s 2023-07-21T19:01:13.697 INFO:tasks.workunit.client.0.smithi039.stdout: 10 217146 30.39 MB/sec execute 266 sec latency 526.921 ms 2023-07-21T19:01:14.697 INFO:tasks.workunit.client.0.smithi039.stdout: 10 217165 30.28 MB/sec execute 267 sec latency 1319.382 ms 2023-07-21T19:01:14.859 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:01:14 smithi193 ceph-mon[98204]: pgmap v618: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 6.6 MiB/s wr, 714 op/s 2023-07-21T19:01:14.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:01:14 smithi155 ceph-mon[97934]: pgmap v618: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 6.6 MiB/s wr, 714 op/s 2023-07-21T19:01:14.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:01:14 smithi039 ceph-mon[96489]: pgmap v618: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 6.6 MiB/s wr, 714 op/s 2023-07-21T19:01:15.697 INFO:tasks.workunit.client.0.smithi039.stdout: 10 217344 30.19 MB/sec execute 268 sec latency 525.325 ms 2023-07-21T19:01:16.697 INFO:tasks.workunit.client.0.smithi039.stdout: 10 217688 30.15 MB/sec execute 269 sec latency 145.636 ms 2023-07-21T19:01:16.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:01:16 smithi193 ceph-mon[98204]: pgmap v619: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.5 MiB/s wr, 696 op/s 2023-07-21T19:01:16.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:01:16 smithi193 ceph-mon[98204]: osdmap e114: 12 total, 12 up, 12 in 2023-07-21T19:01:16.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:01:16 smithi155 ceph-mon[97934]: pgmap v619: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.5 MiB/s wr, 696 op/s 2023-07-21T19:01:16.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:01:16 smithi155 ceph-mon[97934]: osdmap e114: 12 total, 12 up, 12 in 2023-07-21T19:01:16.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:01:16 smithi039 ceph-mon[96489]: pgmap v619: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.5 MiB/s wr, 696 op/s 2023-07-21T19:01:16.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:01:16 smithi039 ceph-mon[96489]: osdmap e114: 12 total, 12 up, 12 in 2023-07-21T19:01:17.697 INFO:tasks.workunit.client.0.smithi039.stdout: 10 218366 30.19 MB/sec execute 270 sec latency 54.212 ms 2023-07-21T19:01:18.697 INFO:tasks.workunit.client.0.smithi039.stdout: 10 218960 30.20 MB/sec execute 271 sec latency 76.003 ms 2023-07-21T19:01:18.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:01:18 smithi193 ceph-mon[98204]: pgmap v621: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 23 MiB/s wr, 894 op/s 2023-07-21T19:01:18.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:01:18 smithi155 ceph-mon[97934]: pgmap v621: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 23 MiB/s wr, 894 op/s 2023-07-21T19:01:18.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:01:18 smithi039 ceph-mon[96489]: pgmap v621: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 23 MiB/s wr, 894 op/s 2023-07-21T19:01:19.697 INFO:tasks.workunit.client.0.smithi039.stdout: 10 219439 30.17 MB/sec execute 272 sec latency 87.214 ms 2023-07-21T19:01:19.859 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.849+0000 7fbdf7918700 1 -- 172.21.15.39:0/3131945536 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbdf00ff4d0 msgr2=0x7fbdf01039a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:01:19.859 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.849+0000 7fbdf7918700 1 --2- 172.21.15.39:0/3131945536 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbdf00ff4d0 0x7fbdf01039a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbde4009990 tx=0x7fbde4009ca0 comp rx=0 tx=0).stop 2023-07-21T19:01:19.859 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.849+0000 7fbdf7918700 1 -- 172.21.15.39:0/3131945536 shutdown_connections 2023-07-21T19:01:19.860 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.849+0000 7fbdf7918700 1 --2- 172.21.15.39:0/3131945536 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbdf0103fc0 0x7fbdf0104460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:19.860 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.849+0000 7fbdf7918700 1 --2- 172.21.15.39:0/3131945536 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbdf00febc0 0x7fbdf00fef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:19.860 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.849+0000 7fbdf7918700 1 --2- 172.21.15.39:0/3131945536 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbdf00ff4d0 0x7fbdf01039a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:19.860 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.849+0000 7fbdf7918700 1 -- 172.21.15.39:0/3131945536 >> 172.21.15.39:0/3131945536 conn(0x7fbdf0076a10 msgr2=0x7fbdf0076e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:01:19.860 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.849+0000 7fbdf7918700 1 -- 172.21.15.39:0/3131945536 shutdown_connections 2023-07-21T19:01:19.860 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.849+0000 7fbdf7918700 1 -- 172.21.15.39:0/3131945536 wait complete. 2023-07-21T19:01:19.862 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.850+0000 7fbdf7918700 1 Processor -- start 2023-07-21T19:01:19.862 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.850+0000 7fbdf7918700 1 -- start start 2023-07-21T19:01:19.862 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.850+0000 7fbdf7918700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbdf00febc0 0x7fbdf019c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:19.862 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.850+0000 7fbdf7918700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbdf00ff4d0 0x7fbdf019c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:19.862 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.850+0000 7fbdf7918700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbdf0103fc0 0x7fbdf01a0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:19.862 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.850+0000 7fbdf7918700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf01a1460 con 0x7fbdf00ff4d0 2023-07-21T19:01:19.863 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.850+0000 7fbdf7918700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf01a15d0 con 0x7fbdf0103fc0 2023-07-21T19:01:19.863 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.850+0000 7fbdf7918700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf01a1790 con 0x7fbdf00febc0 2023-07-21T19:01:19.863 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.850+0000 7fbdf4eb3700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbdf00ff4d0 0x7fbdf019c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:01:19.863 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.850+0000 7fbdf56b4700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbdf00febc0 0x7fbdf019c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:01:19.863 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.850+0000 7fbdf4eb3700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbdf00ff4d0 0x7fbdf019c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34176/0 (socket says 172.21.15.39:34176) 2023-07-21T19:01:19.863 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.850+0000 7fbdf56b4700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbdf00febc0 0x7fbdf019c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:52726/0 (socket says 172.21.15.39:52726) 2023-07-21T19:01:19.863 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.850+0000 7fbdf4eb3700 1 -- 172.21.15.39:0/3253260558 learned_addr learned my addr 172.21.15.39:0/3253260558 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:01:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.851+0000 7fbdf5eb5700 1 --2- 172.21.15.39:0/3253260558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbdf0103fc0 0x7fbdf01a0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:01:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.851+0000 7fbdf4eb3700 1 -- 172.21.15.39:0/3253260558 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbdf00febc0 msgr2=0x7fbdf019c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:01:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.851+0000 7fbdf4eb3700 1 --2- 172.21.15.39:0/3253260558 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbdf00febc0 0x7fbdf019c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.851+0000 7fbdf4eb3700 1 -- 172.21.15.39:0/3253260558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbdf0103fc0 msgr2=0x7fbdf01a0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:01:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.851+0000 7fbdf4eb3700 1 --2- 172.21.15.39:0/3253260558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbdf0103fc0 0x7fbdf01a0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.851+0000 7fbdf4eb3700 1 -- 172.21.15.39:0/3253260558 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdec009520 con 0x7fbdf00ff4d0 2023-07-21T19:01:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.851+0000 7fbdf56b4700 1 --2- 172.21.15.39:0/3253260558 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbdf00febc0 0x7fbdf019c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:01:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.851+0000 7fbdf4eb3700 1 --2- 172.21.15.39:0/3253260558 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbdf00ff4d0 0x7fbdf019c7b0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fbde4000c00 tx=0x7fbde4005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:01:19.865 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.851+0000 7fbde27fc700 1 -- 172.21.15.39:0/3253260558 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde401d070 con 0x7fbdf00ff4d0 2023-07-21T19:01:19.865 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.851+0000 7fbde27fc700 1 -- 172.21.15.39:0/3253260558 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbde4022470 con 0x7fbdf00ff4d0 2023-07-21T19:01:19.865 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.851+0000 7fbdf7918700 1 -- 172.21.15.39:0/3253260558 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbde40095f0 con 0x7fbdf00ff4d0 2023-07-21T19:01:19.865 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.851+0000 7fbde27fc700 1 -- 172.21.15.39:0/3253260558 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde4017650 con 0x7fbdf00ff4d0 2023-07-21T19:01:19.865 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.852+0000 7fbdf7918700 1 -- 172.21.15.39:0/3253260558 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdf01a90d0 con 0x7fbdf00ff4d0 2023-07-21T19:01:19.866 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.855+0000 7fbde27fc700 1 -- 172.21.15.39:0/3253260558 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fbde40225e0 con 0x7fbdf00ff4d0 2023-07-21T19:01:19.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.856+0000 7fbde27fc700 1 --2- 172.21.15.39:0/3253260558 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fbddc0a1340 0x7fbddc0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:19.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.856+0000 7fbe03a9eb80 1 Processor -- start 2023-07-21T19:01:19.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.856+0000 7fbe03a9eb80 1 -- start start 2023-07-21T19:01:19.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.856+0000 7fbde27fc700 1 -- 172.21.15.39:0/3253260558 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fbde40cdff0 con 0x7fbdf00ff4d0 2023-07-21T19:01:19.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.856+0000 7fbe03a9eb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f1d8cc6b0 0x562f1da75d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:19.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.856+0000 7fbe03a9eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f1da762d0 0x562f1da70040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.856+0000 7fbe03a9eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562f1da70580 0x562f1da709d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.856+0000 7fbe03a9eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f1d975c50 con 0x562f1da762d0 2023-07-21T19:01:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.856+0000 7fbe03a9eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f1d9c89e0 con 0x562f1da70580 2023-07-21T19:01:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.856+0000 7fbe03a9eb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f1d9c1e70 con 0x562f1d8cc6b0 2023-07-21T19:01:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.856+0000 7fbdf56b4700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f1d8cc6b0 0x562f1da75d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:01:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.857+0000 7fbdf56b4700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f1d8cc6b0 0x562f1da75d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:52734/0 (socket says 172.21.15.39:52734) 2023-07-21T19:01:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.857+0000 7fbdf56b4700 1 -- 172.21.15.39:0/344029365 learned_addr learned my addr 172.21.15.39:0/344029365 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:01:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.857+0000 7fbdf4eb3700 1 --2- 172.21.15.39:0/344029365 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f1da762d0 0x562f1da70040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:01:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.857+0000 7fbdf56b4700 1 --2- 172.21.15.39:0/3253260558 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fbddc0a1340 0x7fbddc0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:01:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.857+0000 7fbdf5eb5700 1 --2- 172.21.15.39:0/344029365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562f1da70580 0x562f1da709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:01:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.857+0000 7fbdf4eb3700 1 -- 172.21.15.39:0/344029365 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f1d8cc6b0 msgr2=0x562f1da75d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:01:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.857+0000 7fbdf4eb3700 1 --2- 172.21.15.39:0/344029365 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f1d8cc6b0 0x562f1da75d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.857+0000 7fbdf4eb3700 1 -- 172.21.15.39:0/344029365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562f1da70580 msgr2=0x562f1da709d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:01:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.857+0000 7fbdf4eb3700 1 --2- 172.21.15.39:0/344029365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562f1da70580 0x562f1da709d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.857+0000 7fbdf4eb3700 1 -- 172.21.15.39:0/344029365 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f1d7efd80 con 0x562f1da762d0 2023-07-21T19:01:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.857+0000 7fbdf4eb3700 1 --2- 172.21.15.39:0/344029365 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f1da762d0 0x562f1da70040 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fbde40cb740 tx=0x7fbde409ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b64fd1ad8b6bb07 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:01:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.857+0000 7fbdf56b4700 1 --2- 172.21.15.39:0/3253260558 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fbddc0a1340 0x7fbddc0a37f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fbdec009fd0 tx=0x7fbdec009260 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:01:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.858+0000 7fbddb7fe700 1 -- 172.21.15.39:0/344029365 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde409f180 con 0x562f1da762d0 2023-07-21T19:01:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.858+0000 7fbddb7fe700 1 -- 172.21.15.39:0/344029365 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbde40cbe90 con 0x562f1da762d0 2023-07-21T19:01:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.858+0000 7fbddb7fe700 1 -- 172.21.15.39:0/344029365 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde409a640 con 0x562f1da762d0 2023-07-21T19:01:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.858+0000 7fbe03a9eb80 1 -- 172.21.15.39:0/344029365 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f1da762d0 msgr2=0x562f1da70040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:01:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.858+0000 7fbe03a9eb80 1 --2- 172.21.15.39:0/344029365 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f1da762d0 0x562f1da70040 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fbde40cb740 tx=0x7fbde409ad60 comp rx=0 tx=0).stop 2023-07-21T19:01:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.860+0000 7fbe03a9eb80 1 -- 172.21.15.39:0/344029365 shutdown_connections 2023-07-21T19:01:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.860+0000 7fbe03a9eb80 1 --2- 172.21.15.39:0/344029365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562f1da70580 0x562f1da709d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.860+0000 7fbe03a9eb80 1 --2- 172.21.15.39:0/344029365 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f1d8cc6b0 0x562f1da75d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.860+0000 7fbe03a9eb80 1 --2- 172.21.15.39:0/344029365 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f1da762d0 0x562f1da70040 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.860+0000 7fbe03a9eb80 1 -- 172.21.15.39:0/344029365 >> 172.21.15.39:0/344029365 conn(0x562f1da65be0 msgr2=0x562f1da6af10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:01:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.860+0000 7fbe03a9eb80 1 -- 172.21.15.39:0/344029365 shutdown_connections 2023-07-21T19:01:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.861+0000 7fbe03a9eb80 1 -- 172.21.15.39:0/344029365 wait complete. 2023-07-21T19:01:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.861+0000 7fbe03a9eb80 1 -- start start 2023-07-21T19:01:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.862+0000 7fbe03a9eb80 1 Processor -- start 2023-07-21T19:01:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.862+0000 7fbe03a9eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f1d8cc6b0 0x562f1da7f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.862+0000 7fbe03a9eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562f1da70580 0x562f1da7f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.862+0000 7fbe03a9eb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f1da762d0 0x562f1da79710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.862+0000 7fbe03a9eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f1d961a60 con 0x562f1d8cc6b0 2023-07-21T19:01:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.862+0000 7fbe03a9eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f1d961760 con 0x562f1da70580 2023-07-21T19:01:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.862+0000 7fbe03a9eb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f1d8458c0 con 0x562f1da762d0 2023-07-21T19:01:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.862+0000 7fbe03a9eb80 10 client.0 _ll_register_callbacks cb 0x562f1d95f700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:01:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.862+0000 7fbdf5eb5700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f1da762d0 0x562f1da79710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:01:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.863+0000 7fbdf5eb5700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f1da762d0 0x562f1da79710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:52742/0 (socket says 172.21.15.39:52742) 2023-07-21T19:01:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.863+0000 7fbdf5eb5700 1 -- 172.21.15.39:0/974396524 learned_addr learned my addr 172.21.15.39:0/974396524 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:01:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.863+0000 7fbdf5eb5700 1 -- 172.21.15.39:0/974396524 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562f1da70580 msgr2=0x562f1da7f950 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T19:01:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.863+0000 7fbdf5eb5700 1 --2- 172.21.15.39:0/974396524 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562f1da70580 0x562f1da7f950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.863+0000 7fbdf5eb5700 1 -- 172.21.15.39:0/974396524 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f1d8cc6b0 msgr2=0x562f1da7f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:01:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.863+0000 7fbdf5eb5700 1 --2- 172.21.15.39:0/974396524 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f1d8cc6b0 0x562f1da7f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.863+0000 7fbdf5eb5700 1 -- 172.21.15.39:0/974396524 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdf01a90d0 con 0x562f1da762d0 2023-07-21T19:01:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.863+0000 7fbdf5eb5700 1 --2- 172.21.15.39:0/974396524 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f1da762d0 0x562f1da79710 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbde8012e40 tx=0x7fbde8014600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15a8992e091f292a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:01:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.863+0000 7fbdd8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2023-07-21T19:01:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.863+0000 7fbdd8ff9700 1 -- 172.21.15.39:0/974396524 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde800e040 con 0x562f1da762d0 2023-07-21T19:01:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.863+0000 7fbdd8ff9700 1 -- 172.21.15.39:0/974396524 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fbde8014e60 con 0x562f1da762d0 2023-07-21T19:01:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.863+0000 7fbe03a9eb80 1 -- 172.21.15.39:0/974396524 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562f1d91b490 con 0x562f1da762d0 2023-07-21T19:01:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.864+0000 7fbdd8ff9700 1 -- 172.21.15.39:0/974396524 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=101) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbde80092d0 con 0x562f1da762d0 2023-07-21T19:01:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.864+0000 7fbe03a9eb80 10 client.27085 fetch_fsmap learned FSMap version 101 2023-07-21T19:01:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.864+0000 7fbe03a9eb80 1 -- 172.21.15.39:0/974396524 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=101,monmap=4+}) v3 -- 0x562f1d7d8c00 con 0x562f1da762d0 2023-07-21T19:01:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.864+0000 7fbdd8ff9700 1 -- 172.21.15.39:0/974396524 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562f1d91b490 con 0x562f1da762d0 2023-07-21T19:01:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.864+0000 7fbdd8ff9700 1 -- 172.21.15.39:0/974396524 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 101) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbde801c020 con 0x562f1da762d0 2023-07-21T19:01:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.864+0000 7fbe03a9eb80 10 client.27085 fetch_fsmap finished waiting for FSMap version 101 2023-07-21T19:01:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.864+0000 7fbe03a9eb80 10 client.27085 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:01:19.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.864+0000 7fbe03a9eb80 20 client.27085 populate_metadata read hostname 'smithi039' 2023-07-21T19:01:19.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.864+0000 7fbe03a9eb80 1 --2- 172.21.15.39:0/974396524 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x562f1da89990 0x562f1da8bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:19.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.864+0000 7fbe03a9eb80 4 client.27085 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:01:19.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.864+0000 7fbe03a9eb80 1 -- 172.21.15.39:0/974396524 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562f1d9bae50 con 0x562f1da89990 2023-07-21T19:01:19.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.865+0000 7fbdf56b4700 1 --2- 172.21.15.39:0/974396524 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x562f1da89990 0x562f1da8bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:01:19.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.865+0000 7fbdd8ff9700 10 client.27085 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T19:01:19.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.865+0000 7fbdf56b4700 1 --2- 172.21.15.39:0/974396524 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x562f1da89990 0x562f1da8bde0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6187aad11da1ffb3 server_cookie=3e4aabe2378da91 in_seq=0 out_seq=0 2023-07-21T19:01:19.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.866+0000 7fbdd8ff9700 1 -- 172.21.15.39:0/974396524 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562f1d9bae50 con 0x562f1da89990 2023-07-21T19:01:19.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.866+0000 7fbdd8ff9700 10 client.27085 handle_command_reply: tid=0 2023-07-21T19:01:19.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.866+0000 7fbe03a9eb80 1 client.27085 shutdown 2023-07-21T19:01:19.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.866+0000 7fbe03a9eb80 1 -- 172.21.15.39:0/974396524 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f1da762d0 msgr2=0x562f1da79710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:01:19.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.866+0000 7fbe03a9eb80 1 --2- 172.21.15.39:0/974396524 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f1da762d0 0x562f1da79710 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbde8012e40 tx=0x7fbde8014600 comp rx=0 tx=0).stop 2023-07-21T19:01:19.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.867+0000 7fbe03a9eb80 1 -- 172.21.15.39:0/974396524 shutdown_connections 2023-07-21T19:01:19.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.867+0000 7fbe03a9eb80 1 --2- 172.21.15.39:0/974396524 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x562f1da89990 0x562f1da8bde0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:19.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.867+0000 7fbe03a9eb80 1 --2- 172.21.15.39:0/974396524 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f1da762d0 0x562f1da79710 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.867+0000 7fbe03a9eb80 1 --2- 172.21.15.39:0/974396524 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f1d8cc6b0 0x562f1da7f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.867+0000 7fbdd8ff9700 0 client.27085 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T19:01:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.867+0000 7fbe03a9eb80 1 --2- 172.21.15.39:0/974396524 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562f1da70580 0x562f1da7f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.867+0000 7fbe03a9eb80 1 -- 172.21.15.39:0/974396524 >> 172.21.15.39:0/974396524 conn(0x562f1da65be0 msgr2=0x562f1da6a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:01:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.867+0000 7fbe03a9eb80 1 -- 172.21.15.39:0/974396524 shutdown_connections 2023-07-21T19:01:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.867+0000 7fbe03a9eb80 1 -- 172.21.15.39:0/974396524 wait complete. 2023-07-21T19:01:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.867+0000 7fbe03a9eb80 20 client.27085 trim_cache size 0 max 16384 2023-07-21T19:01:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.888+0000 7fbe03a9eb80 1 Processor -- start 2023-07-21T19:01:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.889+0000 7fbe03a9eb80 1 -- start start 2023-07-21T19:01:19.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.889+0000 7fbe03a9eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f1d8cc6b0 0x562f1da70040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:19.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.889+0000 7fbe03a9eb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f1da70580 0x562f1da75320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:19.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.889+0000 7fbe03a9eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562f1da762d0 0x562f1da6ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.889+0000 7fbe03a9eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f1da6cb70 con 0x562f1d8cc6b0 2023-07-21T19:01:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.889+0000 7fbe03a9eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f1da87010 con 0x562f1da762d0 2023-07-21T19:01:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.889+0000 7fbe03a9eb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f1da75a10 con 0x562f1da70580 2023-07-21T19:01:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.889+0000 7fbdf4eb3700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f1da70580 0x562f1da75320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:01:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.889+0000 7fbdf4eb3700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f1da70580 0x562f1da75320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:52746/0 (socket says 172.21.15.39:52746) 2023-07-21T19:01:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.889+0000 7fbdf4eb3700 1 -- 172.21.15.39:0/2812186453 learned_addr learned my addr 172.21.15.39:0/2812186453 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:01:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.890+0000 7fbdf4eb3700 1 -- 172.21.15.39:0/2812186453 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562f1da762d0 msgr2=0x562f1da6ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:01:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.890+0000 7fbdf4eb3700 1 --2- 172.21.15.39:0/2812186453 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562f1da762d0 0x562f1da6ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.890+0000 7fbdf4eb3700 1 -- 172.21.15.39:0/2812186453 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f1d8cc6b0 msgr2=0x562f1da70040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:01:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.890+0000 7fbdf4eb3700 1 --2- 172.21.15.39:0/2812186453 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f1d8cc6b0 0x562f1da70040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.890+0000 7fbdf4eb3700 1 -- 172.21.15.39:0/2812186453 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f1d8161f0 con 0x562f1da70580 2023-07-21T19:01:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.890+0000 7fbdf4eb3700 1 --2- 172.21.15.39:0/2812186453 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f1da70580 0x562f1da75320 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbde40cbd40 tx=0x7fbde4009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b218db79b381617e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:01:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.890+0000 7fbddbfff700 1 -- 172.21.15.39:0/2812186453 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde40a3510 con 0x562f1da70580 2023-07-21T19:01:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.890+0000 7fbddbfff700 1 -- 172.21.15.39:0/2812186453 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbde409f180 con 0x562f1da70580 2023-07-21T19:01:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.890+0000 7fbddbfff700 1 -- 172.21.15.39:0/2812186453 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde409b610 con 0x562f1da70580 2023-07-21T19:01:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.891+0000 7fbe03a9eb80 1 -- 172.21.15.39:0/2812186453 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f1da70580 msgr2=0x562f1da75320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:01:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.891+0000 7fbe03a9eb80 1 --2- 172.21.15.39:0/2812186453 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f1da70580 0x562f1da75320 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbde40cbd40 tx=0x7fbde4009ca0 comp rx=0 tx=0).stop 2023-07-21T19:01:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.891+0000 7fbe03a9eb80 1 -- 172.21.15.39:0/2812186453 shutdown_connections 2023-07-21T19:01:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.891+0000 7fbe03a9eb80 1 --2- 172.21.15.39:0/2812186453 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562f1da762d0 0x562f1da6ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.891+0000 7fbe03a9eb80 1 --2- 172.21.15.39:0/2812186453 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f1d8cc6b0 0x562f1da70040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.891+0000 7fbe03a9eb80 1 --2- 172.21.15.39:0/2812186453 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f1da70580 0x562f1da75320 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.891+0000 7fbe03a9eb80 1 -- 172.21.15.39:0/2812186453 >> 172.21.15.39:0/2812186453 conn(0x562f1da65be0 msgr2=0x562f1da79630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:01:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.891+0000 7fbe03a9eb80 1 -- 172.21.15.39:0/2812186453 shutdown_connections 2023-07-21T19:01:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.891+0000 7fbe03a9eb80 1 -- 172.21.15.39:0/2812186453 wait complete. 2023-07-21T19:01:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.892+0000 7fbe03a9eb80 1 -- start start 2023-07-21T19:01:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.892+0000 7fbe03a9eb80 1 Processor -- start 2023-07-21T19:01:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.892+0000 7fbe03a9eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562f1d8cc6b0 0x562f1dad5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.892+0000 7fbe03a9eb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f1da70580 0x562f1dad60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:19.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.892+0000 7fbe03a9eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f1da762d0 0x562f1dad6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:19.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.892+0000 7fbe03a9eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f1dad7000 con 0x562f1da762d0 2023-07-21T19:01:19.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.892+0000 7fbe03a9eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f1dad7170 con 0x562f1d8cc6b0 2023-07-21T19:01:19.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.892+0000 7fbe03a9eb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f1dad7330 con 0x562f1da70580 2023-07-21T19:01:19.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.892+0000 7fbe03a9eb80 10 client.0 _ll_register_callbacks cb 0x7fbde8012770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:01:19.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.892+0000 7fbdf5eb5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f1da762d0 0x562f1dad6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:01:19.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.892+0000 7fbdf5eb5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f1da762d0 0x562f1dad6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34202/0 (socket says 172.21.15.39:34202) 2023-07-21T19:01:19.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.893+0000 7fbdf5eb5700 1 -- 172.21.15.39:0/3344690400 learned_addr learned my addr 172.21.15.39:0/3344690400 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:01:19.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.893+0000 7fbdf4eb3700 1 --2- 172.21.15.39:0/3344690400 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f1da70580 0x562f1dad60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:01:19.907 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.893+0000 7fbdf5eb5700 1 -- 172.21.15.39:0/3344690400 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f1da70580 msgr2=0x562f1dad60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:01:19.907 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.893+0000 7fbdf5eb5700 1 --2- 172.21.15.39:0/3344690400 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f1da70580 0x562f1dad60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:19.907 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.893+0000 7fbdf5eb5700 1 -- 172.21.15.39:0/3344690400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562f1d8cc6b0 msgr2=0x562f1dad5bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T19:01:19.907 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.893+0000 7fbdf5eb5700 1 --2- 172.21.15.39:0/3344690400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562f1d8cc6b0 0x562f1dad5bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.893+0000 7fbdf5eb5700 1 -- 172.21.15.39:0/3344690400 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562f1d7efd80 con 0x562f1da762d0 2023-07-21T19:01:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.893+0000 7fbdf4eb3700 1 --2- 172.21.15.39:0/3344690400 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f1da70580 0x562f1dad60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:01:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.893+0000 7fbdf5eb5700 1 --2- 172.21.15.39:0/3344690400 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f1da762d0 0x562f1dad6920 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fbde8012b60 tx=0x7fbde8009380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ef726dc6c567fd7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:01:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.893+0000 7fbdd8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T19:01:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.893+0000 7fbdd8ff9700 1 -- 172.21.15.39:0/3344690400 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde800e070 con 0x562f1da762d0 2023-07-21T19:01:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.893+0000 7fbdd8ff9700 1 -- 172.21.15.39:0/3344690400 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fbde80078c0 con 0x562f1da762d0 2023-07-21T19:01:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.893+0000 7fbe03a9eb80 1 -- 172.21.15.39:0/3344690400 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbde40cbe90 con 0x562f1da762d0 2023-07-21T19:01:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.894+0000 7fbdd8ff9700 1 -- 172.21.15.39:0/3344690400 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=101) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbde8009ce0 con 0x562f1da762d0 2023-07-21T19:01:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.894+0000 7fbe03a9eb80 10 client.17598 fetch_fsmap learned FSMap version 101 2023-07-21T19:01:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.894+0000 7fbe03a9eb80 1 -- 172.21.15.39:0/3344690400 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=101,monmap=4+}) v3 -- 0x562f1d7ee250 con 0x562f1da762d0 2023-07-21T19:01:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.894+0000 7fbdd8ff9700 1 -- 172.21.15.39:0/3344690400 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbde40cbe90 con 0x562f1da762d0 2023-07-21T19:01:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.894+0000 7fbdd8ff9700 1 -- 172.21.15.39:0/3344690400 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 101) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbde8035070 con 0x562f1da762d0 2023-07-21T19:01:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.894+0000 7fbe03a9eb80 10 client.17598 fetch_fsmap finished waiting for FSMap version 101 2023-07-21T19:01:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.894+0000 7fbe03a9eb80 10 client.17598 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:01:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.894+0000 7fbe03a9eb80 20 client.17598 populate_metadata read hostname 'smithi039' 2023-07-21T19:01:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.894+0000 7fbe03a9eb80 1 --2- 172.21.15.39:0/3344690400 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x562f1dadeac0 0x562f1dae0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.894+0000 7fbe03a9eb80 4 client.17598 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:01:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.894+0000 7fbe03a9eb80 1 -- 172.21.15.39:0/3344690400 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562f1d97cff0 con 0x562f1dadeac0 2023-07-21T19:01:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.894+0000 7fbdf56b4700 1 --2- 172.21.15.39:0/3344690400 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x562f1dadeac0 0x562f1dae0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:01:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.895+0000 7fbdf56b4700 1 --2- 172.21.15.39:0/3344690400 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x562f1dadeac0 0x562f1dae0ed0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89a81b0c50714e36 server_cookie=b0ab4fb23b4fe1dc in_seq=0 out_seq=0 2023-07-21T19:01:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:19.895+0000 7fbdd8ff9700 10 client.17598 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T19:01:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:20.053+0000 7fbdd8ff9700 1 -- 172.21.15.39:0/3344690400 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562f1d97cff0 con 0x562f1dadeac0 2023-07-21T19:01:20.064 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:20.053+0000 7fbdd8ff9700 10 client.17598 handle_command_reply: tid=0 2023-07-21T19:01:20.064 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:20.053+0000 7fbe03a9eb80 1 client.17598 shutdown 2023-07-21T19:01:20.064 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:20.053+0000 7fbe03a9eb80 1 -- 172.21.15.39:0/3344690400 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f1da762d0 msgr2=0x562f1dad6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:01:20.064 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:20.053+0000 7fbe03a9eb80 1 --2- 172.21.15.39:0/3344690400 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f1da762d0 0x562f1dad6920 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fbde8012b60 tx=0x7fbde8009380 comp rx=0 tx=0).stop 2023-07-21T19:01:20.064 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:20.053+0000 7fbe03a9eb80 1 -- 172.21.15.39:0/3344690400 shutdown_connections 2023-07-21T19:01:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:20.053+0000 7fbe03a9eb80 1 --2- 172.21.15.39:0/3344690400 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x562f1dadeac0 0x562f1dae0ed0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:20.053+0000 7fbe03a9eb80 1 --2- 172.21.15.39:0/3344690400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562f1d8cc6b0 0x562f1dad5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:20.053+0000 7fbe03a9eb80 1 --2- 172.21.15.39:0/3344690400 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f1da762d0 0x562f1dad6920 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:20.053+0000 7fbe03a9eb80 1 --2- 172.21.15.39:0/3344690400 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f1da70580 0x562f1dad60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:20.053+0000 7fbe03a9eb80 1 -- 172.21.15.39:0/3344690400 >> 172.21.15.39:0/3344690400 conn(0x562f1da65be0 msgr2=0x562f1da77ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:01:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:20.054+0000 7fbdd8ff9700 0 client.17598 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T19:01:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:20.054+0000 7fbe03a9eb80 1 -- 172.21.15.39:0/3344690400 shutdown_connections 2023-07-21T19:01:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:20.054+0000 7fbe03a9eb80 1 -- 172.21.15.39:0/3344690400 wait complete. 2023-07-21T19:01:20.066 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:20.054+0000 7fbe03a9eb80 20 client.17598 trim_cache size 0 max 16384 2023-07-21T19:01:20.066 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T19:01:20.066 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T19:01:20.066 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T19:01:20.066 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T19:01:20.067 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:20.054+0000 7fbdf7918700 1 -- 172.21.15.39:0/3253260558 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fbddc0a1340 msgr2=0x7fbddc0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:01:20.067 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:20.054+0000 7fbdf7918700 1 --2- 172.21.15.39:0/3253260558 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fbddc0a1340 0x7fbddc0a37f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fbdec009fd0 tx=0x7fbdec009260 comp rx=0 tx=0).stop 2023-07-21T19:01:20.067 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:20.054+0000 7fbdf7918700 1 -- 172.21.15.39:0/3253260558 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbdf00ff4d0 msgr2=0x7fbdf019c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:01:20.067 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:20.054+0000 7fbdf7918700 1 --2- 172.21.15.39:0/3253260558 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbdf00ff4d0 0x7fbdf019c7b0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fbde4000c00 tx=0x7fbde4005740 comp rx=0 tx=0).stop 2023-07-21T19:01:20.068 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:20.054+0000 7fbdf7918700 1 -- 172.21.15.39:0/3253260558 shutdown_connections 2023-07-21T19:01:20.068 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:20.054+0000 7fbdf7918700 1 --2- 172.21.15.39:0/3253260558 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fbddc0a1340 0x7fbddc0a37f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:20.068 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:20.055+0000 7fbdf7918700 1 --2- 172.21.15.39:0/3253260558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbdf0103fc0 0x7fbdf01a0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:20.068 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:20.055+0000 7fbdf7918700 1 --2- 172.21.15.39:0/3253260558 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbdf00febc0 0x7fbdf019c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:20.068 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:20.055+0000 7fbdf7918700 1 --2- 172.21.15.39:0/3253260558 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbdf00ff4d0 0x7fbdf019c7b0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:20.068 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:20.055+0000 7fbdf7918700 1 -- 172.21.15.39:0/3253260558 >> 172.21.15.39:0/3253260558 conn(0x7fbdf0076a10 msgr2=0x7fbdf01019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:01:20.068 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:20.055+0000 7fbdf7918700 1 -- 172.21.15.39:0/3253260558 shutdown_connections 2023-07-21T19:01:20.069 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:20.055+0000 7fbdf7918700 1 -- 172.21.15.39:0/3253260558 wait complete. 2023-07-21T19:01:20.697 INFO:tasks.workunit.client.0.smithi039.stdout: 10 219932 30.15 MB/sec execute 273 sec latency 172.287 ms 2023-07-21T19:01:20.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:01:20 smithi193 ceph-mon[98204]: pgmap v622: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 23 MiB/s wr, 894 op/s 2023-07-21T19:01:20.879 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:01:20 smithi155 ceph-mon[97934]: pgmap v622: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 23 MiB/s wr, 894 op/s 2023-07-21T19:01:20.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:01:20 smithi039 ceph-mon[96489]: pgmap v622: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 23 MiB/s wr, 894 op/s 2023-07-21T19:01:21.697 INFO:tasks.workunit.client.0.smithi039.stdout: 10 220558 30.15 MB/sec execute 274 sec latency 106.432 ms 2023-07-21T19:01:21.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:01:21 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:01:21.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:01:21 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:01:21.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:01:21 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:01:22.698 INFO:tasks.workunit.client.0.smithi039.stdout: 10 221204 30.18 MB/sec execute 275 sec latency 62.307 ms 2023-07-21T19:01:22.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:01:22 smithi193 ceph-mon[98204]: pgmap v623: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 24 MiB/s wr, 903 op/s 2023-07-21T19:01:22.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:01:22 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:01:22.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:01:22 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:01:22.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:01:22 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T19:01:22.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:01:22 smithi155 ceph-mon[97934]: pgmap v623: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 24 MiB/s wr, 903 op/s 2023-07-21T19:01:22.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:01:22 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:01:22.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:01:22 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:01:22.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:01:22 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T19:01:22.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:01:22 smithi039 ceph-mon[96489]: pgmap v623: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 24 MiB/s wr, 903 op/s 2023-07-21T19:01:22.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:01:22 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:01:22.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:01:22 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:01:22.926 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:01:22 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T19:01:23.698 INFO:tasks.workunit.client.0.smithi039.stdout: 10 221911 30.24 MB/sec execute 276 sec latency 65.821 ms 2023-07-21T19:01:24.697 INFO:tasks.workunit.client.0.smithi039.stdout: 10 222403 30.22 MB/sec execute 277 sec latency 124.877 ms 2023-07-21T19:01:24.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:01:24 smithi193 ceph-mon[98204]: pgmap v624: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 24 MiB/s wr, 974 op/s 2023-07-21T19:01:24.878 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:01:24 smithi155 ceph-mon[97934]: pgmap v624: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 24 MiB/s wr, 974 op/s 2023-07-21T19:01:24.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:01:24 smithi039 ceph-mon[96489]: pgmap v624: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 24 MiB/s wr, 974 op/s 2023-07-21T19:01:25.698 INFO:tasks.workunit.client.0.smithi039.stdout: 10 222921 30.20 MB/sec execute 278 sec latency 132.959 ms 2023-07-21T19:01:26.698 INFO:tasks.workunit.client.0.smithi039.stdout: 10 223410 30.18 MB/sec execute 279 sec latency 105.754 ms 2023-07-21T19:01:26.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:01:26 smithi193 ceph-mon[98204]: pgmap v625: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 24 MiB/s wr, 974 op/s 2023-07-21T19:01:26.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:01:26 smithi155 ceph-mon[97934]: pgmap v625: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 24 MiB/s wr, 974 op/s 2023-07-21T19:01:26.927 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:01:26 smithi039 ceph-mon[96489]: pgmap v625: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 24 MiB/s wr, 974 op/s 2023-07-21T19:01:27.698 INFO:tasks.workunit.client.0.smithi039.stdout: 10 224111 30.19 MB/sec execute 280 sec latency 147.002 ms 2023-07-21T19:01:28.698 INFO:tasks.workunit.client.0.smithi039.stdout: 10 224765 30.20 MB/sec execute 281 sec latency 137.400 ms 2023-07-21T19:01:28.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:01:28 smithi193 ceph-mon[98204]: pgmap v626: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.6 MiB/s wr, 624 op/s 2023-07-21T19:01:28.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:01:28 smithi155 ceph-mon[97934]: pgmap v626: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.6 MiB/s wr, 624 op/s 2023-07-21T19:01:28.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:01:28 smithi039 ceph-mon[96489]: pgmap v626: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.6 MiB/s wr, 624 op/s 2023-07-21T19:01:29.698 INFO:tasks.workunit.client.0.smithi039.stdout: 10 225470 30.25 MB/sec execute 282 sec latency 72.408 ms 2023-07-21T19:01:29.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:01:29 smithi193 ceph-mon[98204]: pgmap v627: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.6 MiB/s wr, 536 op/s 2023-07-21T19:01:29.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:01:29 smithi155 ceph-mon[97934]: pgmap v627: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.6 MiB/s wr, 536 op/s 2023-07-21T19:01:29.926 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:01:29 smithi039 ceph-mon[96489]: pgmap v627: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.6 MiB/s wr, 536 op/s 2023-07-21T19:01:30.698 INFO:tasks.workunit.client.0.smithi039.stdout: 10 226035 30.27 MB/sec execute 283 sec latency 74.403 ms 2023-07-21T19:01:31.698 INFO:tasks.workunit.client.0.smithi039.stdout: 10 226504 30.24 MB/sec execute 284 sec latency 122.686 ms 2023-07-21T19:01:32.607 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:01:32 smithi193 ceph-mon[98204]: pgmap v628: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.9 MiB/s wr, 565 op/s 2023-07-21T19:01:32.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:01:32 smithi155 ceph-mon[97934]: pgmap v628: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.9 MiB/s wr, 565 op/s 2023-07-21T19:01:32.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:01:32 smithi039 ceph-mon[96489]: pgmap v628: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.9 MiB/s wr, 565 op/s 2023-07-21T19:01:32.698 INFO:tasks.workunit.client.0.smithi039.stdout: 10 227041 30.23 MB/sec execute 285 sec latency 94.189 ms 2023-07-21T19:01:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.193+0000 7fbea1afd700 1 -- 172.21.15.39:0/3570711545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe9c104ad0 msgr2=0x7fbe9c104ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:01:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.193+0000 7fbea1afd700 1 --2- 172.21.15.39:0/3570711545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe9c104ad0 0x7fbe9c104ea0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c0098f0 tx=0x7fbe8c009c00 comp rx=0 tx=0).stop 2023-07-21T19:01:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.194+0000 7fbea1afd700 1 -- 172.21.15.39:0/3570711545 shutdown_connections 2023-07-21T19:01:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.194+0000 7fbea1afd700 1 --2- 172.21.15.39:0/3570711545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe9c104ad0 0x7fbe9c104ea0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.194+0000 7fbea1afd700 1 --2- 172.21.15.39:0/3570711545 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe9c0ff470 0x7fbe9c10b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.194+0000 7fbea1afd700 1 --2- 172.21.15.39:0/3570711545 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe9c0feac0 0x7fbe9c0fef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.194+0000 7fbea1afd700 1 -- 172.21.15.39:0/3570711545 >> 172.21.15.39:0/3570711545 conn(0x7fbe9c075c10 msgr2=0x7fbe9c076010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:01:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.194+0000 7fbea1afd700 1 -- 172.21.15.39:0/3570711545 shutdown_connections 2023-07-21T19:01:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.194+0000 7fbea1afd700 1 -- 172.21.15.39:0/3570711545 wait complete. 2023-07-21T19:01:33.206 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.194+0000 7fbea1afd700 1 Processor -- start 2023-07-21T19:01:33.207 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.194+0000 7fbea1afd700 1 -- start start 2023-07-21T19:01:33.207 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.195+0000 7fbea1afd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe9c0feac0 0x7fbe9c197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:33.207 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.195+0000 7fbea1afd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe9c0ff470 0x7fbe9c198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:33.207 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.195+0000 7fbea1afd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe9c104ad0 0x7fbe9c19caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:33.207 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.195+0000 7fbea1afd700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9c19d0e0 con 0x7fbe9c104ad0 2023-07-21T19:01:33.207 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.195+0000 7fbea1afd700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9c19d250 con 0x7fbe9c0feac0 2023-07-21T19:01:33.207 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.195+0000 7fbea1afd700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9c19d410 con 0x7fbe9c0ff470 2023-07-21T19:01:33.208 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.195+0000 7fbe9b7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe9c0feac0 0x7fbe9c197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:01:33.208 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.195+0000 7fbe9bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe9c104ad0 0x7fbe9c19caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:01:33.208 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.195+0000 7fbe9b7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe9c0feac0 0x7fbe9c197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:46810/0 (socket says 172.21.15.39:46810) 2023-07-21T19:01:33.208 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.195+0000 7fbe9b7fe700 1 -- 172.21.15.39:0/3923639128 learned_addr learned my addr 172.21.15.39:0/3923639128 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:01:33.208 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.195+0000 7fbe9affd700 1 --2- 172.21.15.39:0/3923639128 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe9c0ff470 0x7fbe9c198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:01:33.208 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.195+0000 7fbe9bfff700 1 -- 172.21.15.39:0/3923639128 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe9c0ff470 msgr2=0x7fbe9c198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:01:33.208 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.195+0000 7fbe9bfff700 1 --2- 172.21.15.39:0/3923639128 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe9c0ff470 0x7fbe9c198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.195+0000 7fbe9bfff700 1 -- 172.21.15.39:0/3923639128 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe9c0feac0 msgr2=0x7fbe9c197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:01:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.195+0000 7fbe9bfff700 1 --2- 172.21.15.39:0/3923639128 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe9c0feac0 0x7fbe9c197ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.195+0000 7fbe9bfff700 1 -- 172.21.15.39:0/3923639128 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe8c009520 con 0x7fbe9c104ad0 2023-07-21T19:01:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.195+0000 7fbe9bfff700 1 --2- 172.21.15.39:0/3923639128 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe9c104ad0 0x7fbe9c19caa0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fbe9400efc0 tx=0x7fbe9400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:01:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.196+0000 7fbe98ff9700 1 -- 172.21.15.39:0/3923639128 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe94018070 con 0x7fbe9c104ad0 2023-07-21T19:01:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.196+0000 7fbea1afd700 1 -- 172.21.15.39:0/3923639128 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe9c19d690 con 0x7fbe9c104ad0 2023-07-21T19:01:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.196+0000 7fbea1afd700 1 -- 172.21.15.39:0/3923639128 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe9c1a4f30 con 0x7fbe9c104ad0 2023-07-21T19:01:33.210 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.196+0000 7fbe98ff9700 1 -- 172.21.15.39:0/3923639128 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe9400f040 con 0x7fbe9c104ad0 2023-07-21T19:01:33.210 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.196+0000 7fbe98ff9700 1 -- 172.21.15.39:0/3923639128 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9400cbe0 con 0x7fbe9c104ad0 2023-07-21T19:01:33.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.199+0000 7fbe98ff9700 1 -- 172.21.15.39:0/3923639128 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fbe94004ad0 con 0x7fbe9c104ad0 2023-07-21T19:01:33.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.199+0000 7fbe98ff9700 1 --2- 172.21.15.39:0/3923639128 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fbe840a12c0 0x7fbe840a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:33.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.199+0000 7fbe98ff9700 1 -- 172.21.15.39:0/3923639128 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fbe940cc720 con 0x7fbe9c104ad0 2023-07-21T19:01:33.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.199+0000 7fbe9b7fe700 1 --2- 172.21.15.39:0/3923639128 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fbe840a12c0 0x7fbe840a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:01:33.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.200+0000 7fbeadc83b80 1 Processor -- start 2023-07-21T19:01:33.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.200+0000 7fbe9b7fe700 1 --2- 172.21.15.39:0/3923639128 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fbe840a12c0 0x7fbe840a3770 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c00c010 tx=0x7fbe8c019040 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:01:33.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.200+0000 7fbeadc83b80 1 -- start start 2023-07-21T19:01:33.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.200+0000 7fbeadc83b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559ee58b26b0 0x559ee5a5bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:33.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.200+0000 7fbeadc83b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ee5a5c2d0 0x559ee5a56040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:33.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.200+0000 7fbeadc83b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ee5a56580 0x559ee5a569d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:33.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.200+0000 7fbeadc83b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ee595bc50 con 0x559ee58b26b0 2023-07-21T19:01:33.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.200+0000 7fbeadc83b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ee59ae9e0 con 0x559ee5a56580 2023-07-21T19:01:33.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.200+0000 7fbeadc83b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ee59a7e70 con 0x559ee5a5c2d0 2023-07-21T19:01:33.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.200+0000 7fbe9b7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559ee58b26b0 0x559ee5a5bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:01:33.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.200+0000 7fbe9b7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559ee58b26b0 0x559ee5a5bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34222/0 (socket says 172.21.15.39:34222) 2023-07-21T19:01:33.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.200+0000 7fbe9b7fe700 1 -- 172.21.15.39:0/2284241457 learned_addr learned my addr 172.21.15.39:0/2284241457 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:01:33.214 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.201+0000 7fbe9affd700 1 --2- 172.21.15.39:0/2284241457 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ee5a5c2d0 0x559ee5a56040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:01:33.214 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.201+0000 7fbe9affd700 1 -- 172.21.15.39:0/2284241457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ee5a56580 msgr2=0x559ee5a569d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T19:01:33.214 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.201+0000 7fbe9affd700 1 --2- 172.21.15.39:0/2284241457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ee5a56580 0x559ee5a569d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:33.214 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.201+0000 7fbe9affd700 1 -- 172.21.15.39:0/2284241457 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559ee58b26b0 msgr2=0x559ee5a5bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:01:33.214 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.201+0000 7fbe9affd700 1 --2- 172.21.15.39:0/2284241457 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559ee58b26b0 0x559ee5a5bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:33.214 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.201+0000 7fbe9affd700 1 -- 172.21.15.39:0/2284241457 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ee57d5d80 con 0x559ee5a5c2d0 2023-07-21T19:01:33.214 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.201+0000 7fbe9affd700 1 --2- 172.21.15.39:0/2284241457 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ee5a5c2d0 0x559ee5a56040 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbe900099f0 tx=0x7fbe90009d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6b4f3f4c9275a5d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:01:33.214 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.201+0000 7fbe81ffb700 1 -- 172.21.15.39:0/2284241457 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe90009e90 con 0x559ee5a5c2d0 2023-07-21T19:01:33.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.201+0000 7fbe81ffb700 1 -- 172.21.15.39:0/2284241457 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe9000ed90 con 0x559ee5a5c2d0 2023-07-21T19:01:33.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.201+0000 7fbe81ffb700 1 -- 172.21.15.39:0/2284241457 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe900123d0 con 0x559ee5a5c2d0 2023-07-21T19:01:33.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.202+0000 7fbeadc83b80 1 -- 172.21.15.39:0/2284241457 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ee5a5c2d0 msgr2=0x559ee5a56040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:01:33.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.202+0000 7fbeadc83b80 1 --2- 172.21.15.39:0/2284241457 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ee5a5c2d0 0x559ee5a56040 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbe900099f0 tx=0x7fbe90009d00 comp rx=0 tx=0).stop 2023-07-21T19:01:33.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.202+0000 7fbeadc83b80 1 -- 172.21.15.39:0/2284241457 shutdown_connections 2023-07-21T19:01:33.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.202+0000 7fbeadc83b80 1 --2- 172.21.15.39:0/2284241457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ee5a56580 0x559ee5a569d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:33.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.202+0000 7fbeadc83b80 1 --2- 172.21.15.39:0/2284241457 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559ee58b26b0 0x559ee5a5bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:33.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.202+0000 7fbeadc83b80 1 --2- 172.21.15.39:0/2284241457 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ee5a5c2d0 0x559ee5a56040 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:33.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.202+0000 7fbeadc83b80 1 -- 172.21.15.39:0/2284241457 >> 172.21.15.39:0/2284241457 conn(0x559ee5a4bbe0 msgr2=0x559ee5a50f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:01:33.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.203+0000 7fbeadc83b80 1 -- 172.21.15.39:0/2284241457 shutdown_connections 2023-07-21T19:01:33.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.203+0000 7fbeadc83b80 1 -- 172.21.15.39:0/2284241457 wait complete. 2023-07-21T19:01:33.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.203+0000 7fbeadc83b80 1 -- start start 2023-07-21T19:01:33.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.203+0000 7fbeadc83b80 1 Processor -- start 2023-07-21T19:01:33.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.204+0000 7fbeadc83b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ee58b26b0 0x559ee5a625f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:33.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.204+0000 7fbeadc83b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ee5a56580 0x559ee5a5f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:33.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.204+0000 7fbeadc83b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559ee5a5c2d0 0x559ee5a5ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:33.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.204+0000 7fbeadc83b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ee5947a60 con 0x559ee5a5c2d0 2023-07-21T19:01:33.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.204+0000 7fbeadc83b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ee5947760 con 0x559ee5a56580 2023-07-21T19:01:33.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.204+0000 7fbeadc83b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ee582b8c0 con 0x559ee58b26b0 2023-07-21T19:01:33.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.204+0000 7fbeadc83b80 10 client.0 _ll_register_callbacks cb 0x559ee5945700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:01:33.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.204+0000 7fbe9b7fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ee58b26b0 0x559ee5a625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:01:33.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.204+0000 7fbe9b7fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ee58b26b0 0x559ee5a625f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:52780/0 (socket says 172.21.15.39:52780) 2023-07-21T19:01:33.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.204+0000 7fbe9b7fe700 1 -- 172.21.15.39:0/3814099750 learned_addr learned my addr 172.21.15.39:0/3814099750 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:01:33.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.204+0000 7fbe9bfff700 1 --2- 172.21.15.39:0/3814099750 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559ee5a5c2d0 0x559ee5a5ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:01:33.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.204+0000 7fbe9affd700 1 --2- 172.21.15.39:0/3814099750 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ee5a56580 0x559ee5a5f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:01:33.221 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.204+0000 7fbe9bfff700 1 -- 172.21.15.39:0/3814099750 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ee58b26b0 msgr2=0x559ee5a625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:01:33.221 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.204+0000 7fbe9bfff700 1 --2- 172.21.15.39:0/3814099750 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ee58b26b0 0x559ee5a625f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:33.221 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.204+0000 7fbe9bfff700 1 -- 172.21.15.39:0/3814099750 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ee5a56580 msgr2=0x559ee5a5f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:01:33.221 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.204+0000 7fbe9bfff700 1 --2- 172.21.15.39:0/3814099750 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ee5a56580 0x559ee5a5f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:33.221 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.205+0000 7fbe9bfff700 1 -- 172.21.15.39:0/3814099750 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe90009650 con 0x559ee5a5c2d0 2023-07-21T19:01:33.221 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.205+0000 7fbe9bfff700 1 --2- 172.21.15.39:0/3814099750 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559ee5a5c2d0 0x559ee5a5ff40 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fbe940c9e70 tx=0x7fbe94099900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeb4eb3714ace9e3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:01:33.221 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.206+0000 7fbe7affd700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T19:01:33.221 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.206+0000 7fbe7affd700 1 -- 172.21.15.39:0/3814099750 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe940b90d0 con 0x559ee5a5c2d0 2023-07-21T19:01:33.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.206+0000 7fbe7affd700 1 -- 172.21.15.39:0/3814099750 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fbe940b9230 con 0x559ee5a5c2d0 2023-07-21T19:01:33.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.206+0000 7fbeadc83b80 1 -- 172.21.15.39:0/3814099750 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559ee5901490 con 0x559ee5a5c2d0 2023-07-21T19:01:33.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.206+0000 7fbe7affd700 1 -- 172.21.15.39:0/3814099750 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=101) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe94098d70 con 0x559ee5a5c2d0 2023-07-21T19:01:33.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.206+0000 7fbeadc83b80 10 client.17607 fetch_fsmap learned FSMap version 101 2023-07-21T19:01:33.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.206+0000 7fbeadc83b80 1 -- 172.21.15.39:0/3814099750 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=101,monmap=4+}) v3 -- 0x559ee57bec00 con 0x559ee5a5c2d0 2023-07-21T19:01:33.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.206+0000 7fbe7affd700 1 -- 172.21.15.39:0/3814099750 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559ee5901490 con 0x559ee5a5c2d0 2023-07-21T19:01:33.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.206+0000 7fbe7affd700 1 -- 172.21.15.39:0/3814099750 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 101) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbe94095670 con 0x559ee5a5c2d0 2023-07-21T19:01:33.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.206+0000 7fbeadc83b80 10 client.17607 fetch_fsmap finished waiting for FSMap version 101 2023-07-21T19:01:33.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.206+0000 7fbeadc83b80 10 client.17607 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:01:33.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.206+0000 7fbeadc83b80 20 client.17607 populate_metadata read hostname 'smithi039' 2023-07-21T19:01:33.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.206+0000 7fbeadc83b80 1 --2- 172.21.15.39:0/3814099750 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x559ee5a6fb30 0x559ee5a71f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:33.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.206+0000 7fbeadc83b80 4 client.17607 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:01:33.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.206+0000 7fbeadc83b80 1 -- 172.21.15.39:0/3814099750 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559ee59a0e50 con 0x559ee5a6fb30 2023-07-21T19:01:33.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.207+0000 7fbe9b7fe700 1 --2- 172.21.15.39:0/3814099750 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x559ee5a6fb30 0x559ee5a71f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:01:33.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.207+0000 7fbe9b7fe700 1 --2- 172.21.15.39:0/3814099750 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x559ee5a6fb30 0x559ee5a71f40 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5007a5e30c4d3bc server_cookie=e4ef359b15978cc5 in_seq=0 out_seq=0 2023-07-21T19:01:33.224 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.207+0000 7fbe7affd700 10 client.17607 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T19:01:33.224 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.209+0000 7fbe7affd700 1 -- 172.21.15.39:0/3814099750 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559ee59a0e50 con 0x559ee5a6fb30 2023-07-21T19:01:33.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.209+0000 7fbe7affd700 10 client.17607 handle_command_reply: tid=0 2023-07-21T19:01:33.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.210+0000 7fbeadc83b80 1 client.17607 shutdown 2023-07-21T19:01:33.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.210+0000 7fbeadc83b80 1 -- 172.21.15.39:0/3814099750 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559ee5a5c2d0 msgr2=0x559ee5a5ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:01:33.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.210+0000 7fbeadc83b80 1 --2- 172.21.15.39:0/3814099750 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559ee5a5c2d0 0x559ee5a5ff40 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fbe940c9e70 tx=0x7fbe94099900 comp rx=0 tx=0).stop 2023-07-21T19:01:33.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.210+0000 7fbeadc83b80 1 -- 172.21.15.39:0/3814099750 shutdown_connections 2023-07-21T19:01:33.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.210+0000 7fbeadc83b80 1 --2- 172.21.15.39:0/3814099750 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x559ee5a6fb30 0x559ee5a71f40 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:33.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.210+0000 7fbeadc83b80 1 --2- 172.21.15.39:0/3814099750 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559ee5a5c2d0 0x559ee5a5ff40 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:33.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.210+0000 7fbeadc83b80 1 --2- 172.21.15.39:0/3814099750 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ee58b26b0 0x559ee5a625f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:33.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.210+0000 7fbeadc83b80 1 --2- 172.21.15.39:0/3814099750 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ee5a56580 0x559ee5a5f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:33.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.210+0000 7fbeadc83b80 1 -- 172.21.15.39:0/3814099750 >> 172.21.15.39:0/3814099750 conn(0x559ee5a4bbe0 msgr2=0x559ee5a4fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:01:33.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.210+0000 7fbe7affd700 0 client.17607 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T19:01:33.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.210+0000 7fbeadc83b80 1 -- 172.21.15.39:0/3814099750 shutdown_connections 2023-07-21T19:01:33.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.210+0000 7fbeadc83b80 1 -- 172.21.15.39:0/3814099750 wait complete. 2023-07-21T19:01:33.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.210+0000 7fbeadc83b80 20 client.17607 trim_cache size 0 max 16384 2023-07-21T19:01:33.242 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.232+0000 7fbeadc83b80 1 Processor -- start 2023-07-21T19:01:33.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.232+0000 7fbeadc83b80 1 -- start start 2023-07-21T19:01:33.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.233+0000 7fbeadc83b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ee58b26b0 0x559ee5a56040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:33.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.233+0000 7fbeadc83b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ee5a56580 0x559ee5a5f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:33.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.233+0000 7fbeadc83b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559ee5a5c2d0 0x559ee5a66ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:33.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.233+0000 7fbeadc83b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ee5a5a390 con 0x559ee5a5c2d0 2023-07-21T19:01:33.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.233+0000 7fbeadc83b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ee5a5c6a0 con 0x559ee5a56580 2023-07-21T19:01:33.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.233+0000 7fbeadc83b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ee5a51ef0 con 0x559ee58b26b0 2023-07-21T19:01:33.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.233+0000 7fbe9bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559ee5a5c2d0 0x559ee5a66ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:01:33.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.233+0000 7fbe9bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559ee5a5c2d0 0x559ee5a66ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34236/0 (socket says 172.21.15.39:34236) 2023-07-21T19:01:33.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.233+0000 7fbe9bfff700 1 -- 172.21.15.39:0/2991574717 learned_addr learned my addr 172.21.15.39:0/2991574717 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:01:33.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.233+0000 7fbe9b7fe700 1 --2- 172.21.15.39:0/2991574717 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ee58b26b0 0x559ee5a56040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:01:33.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.233+0000 7fbe9affd700 1 --2- 172.21.15.39:0/2991574717 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ee5a56580 0x559ee5a5f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:01:33.245 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.233+0000 7fbe9b7fe700 1 -- 172.21.15.39:0/2991574717 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ee5a56580 msgr2=0x559ee5a5f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:01:33.245 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.233+0000 7fbe9b7fe700 1 --2- 172.21.15.39:0/2991574717 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ee5a56580 0x559ee5a5f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:33.246 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.233+0000 7fbe9b7fe700 1 -- 172.21.15.39:0/2991574717 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559ee5a5c2d0 msgr2=0x559ee5a66ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:01:33.246 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.233+0000 7fbe9b7fe700 1 --2- 172.21.15.39:0/2991574717 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559ee5a5c2d0 0x559ee5a66ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:33.246 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.233+0000 7fbe9b7fe700 1 -- 172.21.15.39:0/2991574717 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ee57fc1f0 con 0x559ee58b26b0 2023-07-21T19:01:33.246 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.233+0000 7fbe9affd700 1 --2- 172.21.15.39:0/2991574717 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ee5a56580 0x559ee5a5f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:01:33.246 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.234+0000 7fbe9b7fe700 1 --2- 172.21.15.39:0/2991574717 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ee58b26b0 0x559ee5a56040 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fbe94097bd0 tx=0x7fbe8c01e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e0c92db3ec936d6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:01:33.246 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.234+0000 7fbe827fc700 1 -- 172.21.15.39:0/2991574717 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c02b040 con 0x559ee58b26b0 2023-07-21T19:01:33.246 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.234+0000 7fbe827fc700 1 -- 172.21.15.39:0/2991574717 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe8c01d040 con 0x559ee58b26b0 2023-07-21T19:01:33.247 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.234+0000 7fbe827fc700 1 -- 172.21.15.39:0/2991574717 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c0325e0 con 0x559ee58b26b0 2023-07-21T19:01:33.247 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.235+0000 7fbeadc83b80 1 -- 172.21.15.39:0/2991574717 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ee58b26b0 msgr2=0x559ee5a56040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:01:33.247 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.235+0000 7fbeadc83b80 1 --2- 172.21.15.39:0/2991574717 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ee58b26b0 0x559ee5a56040 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fbe94097bd0 tx=0x7fbe8c01e000 comp rx=0 tx=0).stop 2023-07-21T19:01:33.247 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.235+0000 7fbeadc83b80 1 -- 172.21.15.39:0/2991574717 shutdown_connections 2023-07-21T19:01:33.247 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.235+0000 7fbeadc83b80 1 --2- 172.21.15.39:0/2991574717 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559ee5a5c2d0 0x559ee5a66ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:33.247 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.235+0000 7fbeadc83b80 1 --2- 172.21.15.39:0/2991574717 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ee58b26b0 0x559ee5a56040 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:33.247 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.235+0000 7fbeadc83b80 1 --2- 172.21.15.39:0/2991574717 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ee5a56580 0x559ee5a5f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:33.247 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.235+0000 7fbeadc83b80 1 -- 172.21.15.39:0/2991574717 >> 172.21.15.39:0/2991574717 conn(0x559ee5a4bbe0 msgr2=0x559ee5a5f0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:01:33.248 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.235+0000 7fbeadc83b80 1 -- 172.21.15.39:0/2991574717 shutdown_connections 2023-07-21T19:01:33.248 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.235+0000 7fbeadc83b80 1 -- 172.21.15.39:0/2991574717 wait complete. 2023-07-21T19:01:33.248 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.236+0000 7fbeadc83b80 1 -- start start 2023-07-21T19:01:33.248 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.236+0000 7fbeadc83b80 1 Processor -- start 2023-07-21T19:01:33.250 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.236+0000 7fbeadc83b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ee58b26b0 0x559ee5a58b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:33.250 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.236+0000 7fbeadc83b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559ee5a56580 0x559ee5a59040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:33.250 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.236+0000 7fbeadc83b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ee5a5c2d0 0x559ee5a57860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:33.250 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.236+0000 7fbeadc83b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ee5a57da0 con 0x559ee5a56580 2023-07-21T19:01:33.250 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.236+0000 7fbeadc83b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ee5a57f10 con 0x559ee58b26b0 2023-07-21T19:01:33.250 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.236+0000 7fbeadc83b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ee5a580d0 con 0x559ee5a5c2d0 2023-07-21T19:01:33.251 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.236+0000 7fbeadc83b80 10 client.0 _ll_register_callbacks cb 0x7fbe940991e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:01:33.251 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.236+0000 7fbe9affd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559ee5a56580 0x559ee5a59040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:01:33.251 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.236+0000 7fbe9bfff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ee5a5c2d0 0x559ee5a57860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:01:33.251 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.236+0000 7fbe9affd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559ee5a56580 0x559ee5a59040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34244/0 (socket says 172.21.15.39:34244) 2023-07-21T19:01:33.251 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.236+0000 7fbe9affd700 1 -- 172.21.15.39:0/2857926728 learned_addr learned my addr 172.21.15.39:0/2857926728 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:01:33.251 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.236+0000 7fbe9b7fe700 1 --2- 172.21.15.39:0/2857926728 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ee58b26b0 0x559ee5a58b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:01:33.251 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.237+0000 7fbe9affd700 1 -- 172.21.15.39:0/2857926728 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ee5a5c2d0 msgr2=0x559ee5a57860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:01:33.251 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.237+0000 7fbe9affd700 1 --2- 172.21.15.39:0/2857926728 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ee5a5c2d0 0x559ee5a57860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:33.252 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.237+0000 7fbe9affd700 1 -- 172.21.15.39:0/2857926728 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ee58b26b0 msgr2=0x559ee5a58b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:01:33.252 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.237+0000 7fbe9affd700 1 --2- 172.21.15.39:0/2857926728 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ee58b26b0 0x559ee5a58b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:33.252 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.237+0000 7fbe9affd700 1 -- 172.21.15.39:0/2857926728 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559ee57d5d80 con 0x559ee5a56580 2023-07-21T19:01:33.252 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.237+0000 7fbe9b7fe700 1 --2- 172.21.15.39:0/2857926728 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ee58b26b0 0x559ee5a58b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:01:33.252 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.237+0000 7fbe9bfff700 1 --2- 172.21.15.39:0/2857926728 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ee5a5c2d0 0x559ee5a57860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:01:33.252 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.237+0000 7fbe9affd700 1 --2- 172.21.15.39:0/2857926728 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559ee5a56580 0x559ee5a59040 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fbe900099f0 tx=0x7fbe90013d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc668ae62ebb8cb2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:01:33.252 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.237+0000 7fbe7affd700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T19:01:33.253 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.237+0000 7fbe7affd700 1 -- 172.21.15.39:0/2857926728 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe90022070 con 0x559ee5a56580 2023-07-21T19:01:33.253 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.237+0000 7fbe7affd700 1 -- 172.21.15.39:0/2857926728 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fbe90013e00 con 0x559ee5a56580 2023-07-21T19:01:33.253 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.237+0000 7fbeadc83b80 1 -- 172.21.15.39:0/2857926728 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbe9000ed90 con 0x559ee5a56580 2023-07-21T19:01:33.253 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.238+0000 7fbe7affd700 1 -- 172.21.15.39:0/2857926728 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=101) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe90013680 con 0x559ee5a56580 2023-07-21T19:01:33.253 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.238+0000 7fbeadc83b80 10 client.17613 fetch_fsmap learned FSMap version 101 2023-07-21T19:01:33.253 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.238+0000 7fbeadc83b80 1 -- 172.21.15.39:0/2857926728 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=101,monmap=4+}) v3 -- 0x7fbe9c10c200 con 0x559ee5a56580 2023-07-21T19:01:33.253 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.238+0000 7fbe7affd700 1 -- 172.21.15.39:0/2857926728 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe90033020 con 0x559ee5a56580 2023-07-21T19:01:33.253 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.238+0000 7fbe7affd700 1 -- 172.21.15.39:0/2857926728 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 101) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbe90011070 con 0x559ee5a56580 2023-07-21T19:01:33.254 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.238+0000 7fbeadc83b80 10 client.17613 fetch_fsmap finished waiting for FSMap version 101 2023-07-21T19:01:33.254 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.238+0000 7fbeadc83b80 10 client.17613 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:01:33.254 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.238+0000 7fbeadc83b80 20 client.17613 populate_metadata read hostname 'smithi039' 2023-07-21T19:01:33.254 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.238+0000 7fbeadc83b80 1 --2- 172.21.15.39:0/2857926728 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x559ee5ac47d0 0x559ee5ac6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:33.254 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.238+0000 7fbeadc83b80 4 client.17613 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:01:33.254 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.238+0000 7fbeadc83b80 1 -- 172.21.15.39:0/2857926728 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559ee5962ff0 con 0x559ee5ac47d0 2023-07-21T19:01:33.254 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.239+0000 7fbe9b7fe700 1 --2- 172.21.15.39:0/2857926728 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x559ee5ac47d0 0x559ee5ac6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:01:33.254 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.239+0000 7fbe9b7fe700 1 --2- 172.21.15.39:0/2857926728 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x559ee5ac47d0 0x559ee5ac6be0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87b24b0675f05a1d server_cookie=f983b663066b33b1 in_seq=0 out_seq=0 2023-07-21T19:01:33.255 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.239+0000 7fbe7affd700 10 client.17613 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T19:01:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.518+0000 7fbe7affd700 1 -- 172.21.15.39:0/2857926728 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559ee5962ff0 con 0x559ee5ac47d0 2023-07-21T19:01:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.519+0000 7fbe7affd700 10 client.17613 handle_command_reply: tid=0 2023-07-21T19:01:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.519+0000 7fbeadc83b80 1 client.17613 shutdown 2023-07-21T19:01:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.519+0000 7fbeadc83b80 1 -- 172.21.15.39:0/2857926728 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559ee5a56580 msgr2=0x559ee5a59040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:01:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.519+0000 7fbeadc83b80 1 --2- 172.21.15.39:0/2857926728 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559ee5a56580 0x559ee5a59040 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fbe900099f0 tx=0x7fbe90013d10 comp rx=0 tx=0).stop 2023-07-21T19:01:33.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.519+0000 7fbeadc83b80 1 -- 172.21.15.39:0/2857926728 shutdown_connections 2023-07-21T19:01:33.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.519+0000 7fbeadc83b80 1 --2- 172.21.15.39:0/2857926728 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x559ee5ac47d0 0x559ee5ac6be0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:33.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.519+0000 7fbeadc83b80 1 --2- 172.21.15.39:0/2857926728 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ee58b26b0 0x559ee5a58b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:33.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.519+0000 7fbeadc83b80 1 --2- 172.21.15.39:0/2857926728 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ee5a5c2d0 0x559ee5a57860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:33.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T19:01:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T19:01:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T19:01:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T19:01:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.519+0000 7fbeadc83b80 1 --2- 172.21.15.39:0/2857926728 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559ee5a56580 0x559ee5a59040 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.519+0000 7fbeadc83b80 1 -- 172.21.15.39:0/2857926728 >> 172.21.15.39:0/2857926728 conn(0x559ee5a4bbe0 msgr2=0x559ee5a5c900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:01:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.519+0000 7fbe7affd700 0 client.17613 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T19:01:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.519+0000 7fbeadc83b80 1 -- 172.21.15.39:0/2857926728 shutdown_connections 2023-07-21T19:01:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.519+0000 7fbeadc83b80 1 -- 172.21.15.39:0/2857926728 wait complete. 2023-07-21T19:01:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.519+0000 7fbeadc83b80 20 client.17613 trim_cache size 0 max 16384 2023-07-21T19:01:33.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.524+0000 7fbea1afd700 1 -- 172.21.15.39:0/3923639128 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fbe840a12c0 msgr2=0x7fbe840a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:01:33.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.524+0000 7fbea1afd700 1 --2- 172.21.15.39:0/3923639128 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fbe840a12c0 0x7fbe840a3770 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c00c010 tx=0x7fbe8c019040 comp rx=0 tx=0).stop 2023-07-21T19:01:33.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.524+0000 7fbea1afd700 1 -- 172.21.15.39:0/3923639128 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe9c104ad0 msgr2=0x7fbe9c19caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:01:33.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.524+0000 7fbea1afd700 1 --2- 172.21.15.39:0/3923639128 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe9c104ad0 0x7fbe9c19caa0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fbe9400efc0 tx=0x7fbe9400c3a0 comp rx=0 tx=0).stop 2023-07-21T19:01:33.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.526+0000 7fbea1afd700 1 -- 172.21.15.39:0/3923639128 shutdown_connections 2023-07-21T19:01:33.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.526+0000 7fbea1afd700 1 --2- 172.21.15.39:0/3923639128 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fbe840a12c0 0x7fbe840a3770 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:33.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.526+0000 7fbea1afd700 1 --2- 172.21.15.39:0/3923639128 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe9c0feac0 0x7fbe9c197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:33.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.526+0000 7fbea1afd700 1 --2- 172.21.15.39:0/3923639128 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe9c104ad0 0x7fbe9c19caa0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:33.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.526+0000 7fbea1afd700 1 --2- 172.21.15.39:0/3923639128 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe9c0ff470 0x7fbe9c198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:33.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.526+0000 7fbea1afd700 1 -- 172.21.15.39:0/3923639128 >> 172.21.15.39:0/3923639128 conn(0x7fbe9c075c10 msgr2=0x7fbe9c068850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:01:33.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.526+0000 7fbea1afd700 1 -- 172.21.15.39:0/3923639128 shutdown_connections 2023-07-21T19:01:33.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:33.528+0000 7fbea1afd700 1 -- 172.21.15.39:0/3923639128 wait complete. 2023-07-21T19:01:33.698 INFO:tasks.workunit.client.0.smithi039.stdout: 10 227645 30.21 MB/sec execute 286 sec latency 185.262 ms 2023-07-21T19:01:34.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:01:34 smithi193 ceph-mon[98204]: pgmap v629: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 9.8 MiB/s wr, 716 op/s 2023-07-21T19:01:34.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:01:34 smithi155 ceph-mon[97934]: pgmap v629: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 9.8 MiB/s wr, 716 op/s 2023-07-21T19:01:34.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:01:34 smithi039 ceph-mon[96489]: pgmap v629: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 9.8 MiB/s wr, 716 op/s 2023-07-21T19:01:34.698 INFO:tasks.workunit.client.0.smithi039.stdout: 10 228437 30.27 MB/sec execute 287 sec latency 63.564 ms 2023-07-21T19:01:35.698 INFO:tasks.workunit.client.0.smithi039.stdout: 10 229106 30.33 MB/sec execute 288 sec latency 59.039 ms 2023-07-21T19:01:36.607 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:01:36 smithi193 ceph-mon[98204]: pgmap v630: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 7.0 MiB/s wr, 428 op/s 2023-07-21T19:01:36.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:01:36 smithi155 ceph-mon[97934]: pgmap v630: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 7.0 MiB/s wr, 428 op/s 2023-07-21T19:01:36.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:01:36 smithi039 ceph-mon[96489]: pgmap v630: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 7.0 MiB/s wr, 428 op/s 2023-07-21T19:01:36.698 INFO:tasks.workunit.client.0.smithi039.stdout: 10 229618 30.31 MB/sec execute 289 sec latency 69.404 ms 2023-07-21T19:01:37.698 INFO:tasks.workunit.client.0.smithi039.stdout: 10 230086 30.28 MB/sec execute 290 sec latency 146.302 ms 2023-07-21T19:01:38.607 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:01:38 smithi193 ceph-mon[98204]: pgmap v631: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 10 MiB/s wr, 693 op/s 2023-07-21T19:01:38.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:01:38 smithi155 ceph-mon[97934]: pgmap v631: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 10 MiB/s wr, 693 op/s 2023-07-21T19:01:38.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:01:38 smithi039 ceph-mon[96489]: pgmap v631: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 10 MiB/s wr, 693 op/s 2023-07-21T19:01:38.698 INFO:tasks.workunit.client.0.smithi039.stdout: 10 230780 30.27 MB/sec execute 291 sec latency 108.636 ms 2023-07-21T19:01:39.698 INFO:tasks.workunit.client.0.smithi039.stdout: 10 231511 30.33 MB/sec execute 292 sec latency 91.397 ms 2023-07-21T19:01:40.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:01:40 smithi155 ceph-mon[97934]: pgmap v632: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 7.1 MiB/s wr, 486 op/s 2023-07-21T19:01:40.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:01:40 smithi039 ceph-mon[96489]: pgmap v632: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 7.1 MiB/s wr, 486 op/s 2023-07-21T19:01:40.698 INFO:tasks.workunit.client.0.smithi039.stdout: 10 232201 30.34 MB/sec execute 293 sec latency 83.759 ms 2023-07-21T19:01:40.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:01:40 smithi193 ceph-mon[98204]: pgmap v632: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 7.1 MiB/s wr, 486 op/s 2023-07-21T19:01:41.698 INFO:tasks.workunit.client.0.smithi039.stdout: 10 232824 30.38 MB/sec execute 294 sec latency 60.881 ms 2023-07-21T19:01:42.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:01:42 smithi155 ceph-mon[97934]: pgmap v633: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 7.1 MiB/s wr, 495 op/s 2023-07-21T19:01:42.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:01:42 smithi039 ceph-mon[96489]: pgmap v633: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 7.1 MiB/s wr, 495 op/s 2023-07-21T19:01:42.698 INFO:tasks.workunit.client.0.smithi039.stdout: 10 233196 30.35 MB/sec execute 295 sec latency 160.298 ms 2023-07-21T19:01:42.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:01:42 smithi193 ceph-mon[98204]: pgmap v633: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 7.1 MiB/s wr, 495 op/s 2023-07-21T19:01:43.698 INFO:tasks.workunit.client.0.smithi039.stdout: 10 233763 30.30 MB/sec execute 296 sec latency 63.599 ms 2023-07-21T19:01:44.680 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:01:44 smithi039 ceph-mon[96489]: pgmap v634: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 10 MiB/s wr, 669 op/s 2023-07-21T19:01:44.699 INFO:tasks.workunit.client.0.smithi039.stdout: 10 233936 30.24 MB/sec execute 297 sec latency 817.712 ms 2023-07-21T19:01:44.859 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:01:44 smithi193 ceph-mon[98204]: pgmap v634: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 10 MiB/s wr, 669 op/s 2023-07-21T19:01:44.887 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:01:44 smithi155 ceph-mon[97934]: pgmap v634: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 10 MiB/s wr, 669 op/s 2023-07-21T19:01:45.699 INFO:tasks.workunit.client.0.smithi039.stdout: 10 234052 30.15 MB/sec execute 298 sec latency 1081.919 ms 2023-07-21T19:01:46.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:01:46 smithi039 ceph-mon[96489]: pgmap v635: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.0 MiB/s wr, 476 op/s 2023-07-21T19:01:46.699 INFO:tasks.workunit.client.0.smithi039.stdout: 10 234366 30.13 MB/sec execute 299 sec latency 267.328 ms 2023-07-21T19:01:46.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.709+0000 7f2c549ea700 1 -- 172.21.15.39:0/39048756 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c5006d3a0 msgr2=0x7f2c5011b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:01:46.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.709+0000 7f2c549ea700 1 --2- 172.21.15.39:0/39048756 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c5006d3a0 0x7f2c5011b470 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f2c48009f90 tx=0x7f2c4800b330 comp rx=0 tx=0).stop 2023-07-21T19:01:46.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.710+0000 7f2c549ea700 1 -- 172.21.15.39:0/39048756 shutdown_connections 2023-07-21T19:01:46.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.710+0000 7f2c549ea700 1 --2- 172.21.15.39:0/39048756 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c5010e2a0 0x7f2c5010e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:46.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.710+0000 7f2c549ea700 1 --2- 172.21.15.39:0/39048756 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c5006d3a0 0x7f2c5011b470 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:46.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.710+0000 7f2c549ea700 1 --2- 172.21.15.39:0/39048756 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c5006c910 0x7f2c5006cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:46.722 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.710+0000 7f2c549ea700 1 -- 172.21.15.39:0/39048756 >> 172.21.15.39:0/39048756 conn(0x7f2c5006bd70 msgr2=0x7f2c50070f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:01:46.722 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.710+0000 7f2c549ea700 1 -- 172.21.15.39:0/39048756 shutdown_connections 2023-07-21T19:01:46.722 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.710+0000 7f2c549ea700 1 -- 172.21.15.39:0/39048756 wait complete. 2023-07-21T19:01:46.722 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.711+0000 7f2c549ea700 1 Processor -- start 2023-07-21T19:01:46.722 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.711+0000 7f2c549ea700 1 -- start start 2023-07-21T19:01:46.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.711+0000 7f2c549ea700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c5006c910 0x7f2c5011aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:46.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.711+0000 7f2c549ea700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c5006d3a0 0x7f2c5011b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:46.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.711+0000 7f2c549ea700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c5010e2a0 0x7f2c50114dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:46.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.711+0000 7f2c549ea700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c50115590 con 0x7f2c5006d3a0 2023-07-21T19:01:46.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.711+0000 7f2c549ea700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c50115700 con 0x7f2c5010e2a0 2023-07-21T19:01:46.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.711+0000 7f2c549ea700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c501158c0 con 0x7f2c5006c910 2023-07-21T19:01:46.727 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.712+0000 7f2c45d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c5006d3a0 0x7f2c5011b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:01:46.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.712+0000 7f2c45d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c5006d3a0 0x7f2c5011b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34256/0 (socket says 172.21.15.39:34256) 2023-07-21T19:01:46.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.712+0000 7f2c45d9b700 1 -- 172.21.15.39:0/698068138 learned_addr learned my addr 172.21.15.39:0/698068138 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:01:46.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.712+0000 7f2c4e59c700 1 --2- 172.21.15.39:0/698068138 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c5006c910 0x7f2c5011aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:01:46.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.713+0000 7f2c4e59c700 1 -- 172.21.15.39:0/698068138 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c5010e2a0 msgr2=0x7f2c50114dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:01:46.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.713+0000 7f2c4e59c700 1 --2- 172.21.15.39:0/698068138 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c5010e2a0 0x7f2c50114dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:46.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.713+0000 7f2c4e59c700 1 -- 172.21.15.39:0/698068138 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c5006d3a0 msgr2=0x7f2c5011b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:01:46.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.713+0000 7f2c4e59c700 1 --2- 172.21.15.39:0/698068138 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c5006d3a0 0x7f2c5011b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:46.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.713+0000 7f2c4e59c700 1 -- 172.21.15.39:0/698068138 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c48009c40 con 0x7f2c5006c910 2023-07-21T19:01:46.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.718+0000 7f2c4e59c700 1 --2- 172.21.15.39:0/698068138 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c5006c910 0x7f2c5011aea0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2c3800b450 tx=0x7f2c3800b760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:01:46.730 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.718+0000 7f2c47fff700 1 -- 172.21.15.39:0/698068138 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c38011830 con 0x7f2c5006c910 2023-07-21T19:01:46.730 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.718+0000 7f2c47fff700 1 -- 172.21.15.39:0/698068138 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c38011e50 con 0x7f2c5006c910 2023-07-21T19:01:46.730 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.718+0000 7f2c47fff700 1 -- 172.21.15.39:0/698068138 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c38010d60 con 0x7f2c5006c910 2023-07-21T19:01:46.730 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.719+0000 7f2c549ea700 1 -- 172.21.15.39:0/698068138 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c50124110 con 0x7f2c5006c910 2023-07-21T19:01:46.730 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.719+0000 7f2c549ea700 1 -- 172.21.15.39:0/698068138 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c501245d0 con 0x7f2c5006c910 2023-07-21T19:01:46.736 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.726+0000 7f2c60b70b80 1 Processor -- start 2023-07-21T19:01:46.736 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.726+0000 7f2c60b70b80 1 -- start start 2023-07-21T19:01:46.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.726+0000 7f2c60b70b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56414c7d46b0 0x56414c97dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:46.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.726+0000 7f2c60b70b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414c97e2d0 0x56414c978040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:46.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.727+0000 7f2c60b70b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56414c978580 0x56414c9789d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:46.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.727+0000 7f2c60b70b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x56414c87dc50 con 0x56414c7d46b0 2023-07-21T19:01:46.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.727+0000 7f2c60b70b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56414c8d09e0 con 0x56414c97e2d0 2023-07-21T19:01:46.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.727+0000 7f2c60b70b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x56414c8c9e70 con 0x56414c978580 2023-07-21T19:01:46.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.727+0000 7f2c4ed9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56414c7d46b0 0x56414c97dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:01:46.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.727+0000 7f2c4ed9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56414c7d46b0 0x56414c97dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34262/0 (socket says 172.21.15.39:34262) 2023-07-21T19:01:46.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.727+0000 7f2c4ed9d700 1 -- 172.21.15.39:0/2952280024 learned_addr learned my addr 172.21.15.39:0/2952280024 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:01:46.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.728+0000 7f2c4ed9d700 1 -- 172.21.15.39:0/2952280024 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56414c978580 msgr2=0x56414c9789d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T19:01:46.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.728+0000 7f2c4ed9d700 1 --2- 172.21.15.39:0/2952280024 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56414c978580 0x56414c9789d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:46.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.728+0000 7f2c4ed9d700 1 -- 172.21.15.39:0/2952280024 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414c97e2d0 msgr2=0x56414c978040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T19:01:46.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.728+0000 7f2c4ed9d700 1 --2- 172.21.15.39:0/2952280024 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414c97e2d0 0x56414c978040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:46.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.728+0000 7f2c4ed9d700 1 -- 172.21.15.39:0/2952280024 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56414c6f7d80 con 0x56414c7d46b0 2023-07-21T19:01:46.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.728+0000 7f2c4ed9d700 1 --2- 172.21.15.39:0/2952280024 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56414c7d46b0 0x56414c97dd90 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f2c48008d20 tx=0x7f2c48008d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc24d2d6da8c590a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:01:46.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.730+0000 7f2c47fff700 1 -- 172.21.15.39:0/698068138 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f2c38010380 con 0x7f2c5006c910 2023-07-21T19:01:46.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.730+0000 7f2c47fff700 1 --2- 172.21.15.39:0/698068138 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f2c3c0a1400 0x7f2c3c0a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:46.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.731+0000 7f2c47fff700 1 -- 172.21.15.39:0/698068138 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f2c380cc4b0 con 0x7f2c5006c910 2023-07-21T19:01:46.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.733+0000 7f2c44d99700 1 -- 172.21.15.39:0/2952280024 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4800e040 con 0x56414c7d46b0 2023-07-21T19:01:46.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.733+0000 7f2c44d99700 1 -- 172.21.15.39:0/2952280024 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c48003c00 con 0x56414c7d46b0 2023-07-21T19:01:46.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.733+0000 7f2c44d99700 1 -- 172.21.15.39:0/2952280024 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4801d6d0 con 0x56414c7d46b0 2023-07-21T19:01:46.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.734+0000 7f2c60b70b80 1 -- 172.21.15.39:0/2952280024 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56414c7d46b0 msgr2=0x56414c97dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:01:46.745 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.734+0000 7f2c60b70b80 1 --2- 172.21.15.39:0/2952280024 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56414c7d46b0 0x56414c97dd90 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f2c48008d20 tx=0x7f2c48008d50 comp rx=0 tx=0).stop 2023-07-21T19:01:46.745 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.734+0000 7f2c60b70b80 1 -- 172.21.15.39:0/2952280024 shutdown_connections 2023-07-21T19:01:46.745 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.734+0000 7f2c60b70b80 1 --2- 172.21.15.39:0/2952280024 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56414c978580 0x56414c9789d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:46.746 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.734+0000 7f2c60b70b80 1 --2- 172.21.15.39:0/2952280024 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56414c7d46b0 0x56414c97dd90 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:46.746 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.734+0000 7f2c60b70b80 1 --2- 172.21.15.39:0/2952280024 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414c97e2d0 0x56414c978040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:46.746 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.734+0000 7f2c60b70b80 1 -- 172.21.15.39:0/2952280024 >> 172.21.15.39:0/2952280024 conn(0x56414c96dbe0 msgr2=0x56414c972f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:01:46.746 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.736+0000 7f2c4e59c700 1 --2- 172.21.15.39:0/698068138 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f2c3c0a1400 0x7f2c3c0a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:01:46.747 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.738+0000 7f2c60b70b80 1 -- 172.21.15.39:0/2952280024 shutdown_connections 2023-07-21T19:01:46.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.740+0000 7f2c60b70b80 1 -- 172.21.15.39:0/2952280024 wait complete. 2023-07-21T19:01:46.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.741+0000 7f2c60b70b80 1 -- start start 2023-07-21T19:01:46.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.741+0000 7f2c60b70b80 1 Processor -- start 2023-07-21T19:01:46.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.741+0000 7f2c60b70b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56414c7d46b0 0x56414c984630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:46.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.741+0000 7f2c60b70b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56414c978580 0x56414c981920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:46.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.741+0000 7f2c60b70b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414c97e2d0 0x56414c982010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.741+0000 7f2c60b70b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x56414c869a60 con 0x56414c978580 2023-07-21T19:01:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.741+0000 7f2c60b70b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56414c869760 con 0x56414c97e2d0 2023-07-21T19:01:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.741+0000 7f2c60b70b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x56414c74d8c0 con 0x56414c7d46b0 2023-07-21T19:01:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.741+0000 7f2c60b70b80 10 client.0 _ll_register_callbacks cb 0x56414c867700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:01:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.742+0000 7f2c4e59c700 1 --2- 172.21.15.39:0/698068138 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f2c3c0a1400 0x7f2c3c0a38b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2c38099330 tx=0x7f2c38004860 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:01:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.743+0000 7f2c45d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56414c978580 0x56414c981920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:01:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.744+0000 7f2c4ed9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414c97e2d0 0x56414c982010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:01:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.744+0000 7f2c45d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56414c978580 0x56414c981920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34268/0 (socket says 172.21.15.39:34268) 2023-07-21T19:01:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.744+0000 7f2c4ed9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414c97e2d0 0x56414c982010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:46866/0 (socket says 172.21.15.39:46866) 2023-07-21T19:01:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.744+0000 7f2c45d9b700 1 -- 172.21.15.39:0/2005878641 learned_addr learned my addr 172.21.15.39:0/2005878641 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:01:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.744+0000 7f2c4ed9d700 1 --2- 172.21.15.39:0/2005878641 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56414c7d46b0 0x56414c984630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:01:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.744+0000 7f2c45d9b700 1 -- 172.21.15.39:0/2005878641 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56414c7d46b0 msgr2=0x56414c984630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:01:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.744+0000 7f2c45d9b700 1 --2- 172.21.15.39:0/2005878641 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56414c7d46b0 0x56414c984630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.744+0000 7f2c45d9b700 1 -- 172.21.15.39:0/2005878641 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414c97e2d0 msgr2=0x56414c982010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:01:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.744+0000 7f2c45d9b700 1 --2- 172.21.15.39:0/2005878641 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414c97e2d0 0x56414c982010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.744+0000 7f2c45d9b700 1 -- 172.21.15.39:0/2005878641 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c5011bfb0 con 0x56414c978580 2023-07-21T19:01:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.744+0000 7f2c45d9b700 1 --2- 172.21.15.39:0/2005878641 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56414c978580 0x56414c981920 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f2c400116f0 tx=0x7f2c40011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=922c582df1f368c7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:01:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.744+0000 7f2c35ffb700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T19:01:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.744+0000 7f2c35ffb700 1 -- 172.21.15.39:0/2005878641 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c40004520 con 0x56414c978580 2023-07-21T19:01:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.745+0000 7f2c35ffb700 1 -- 172.21.15.39:0/2005878641 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f2c40003680 con 0x56414c978580 2023-07-21T19:01:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.745+0000 7f2c60b70b80 1 -- 172.21.15.39:0/2005878641 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56414c823490 con 0x56414c978580 2023-07-21T19:01:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.745+0000 7f2c35ffb700 1 -- 172.21.15.39:0/2005878641 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=101) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c40003840 con 0x56414c978580 2023-07-21T19:01:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.746+0000 7f2c60b70b80 10 client.17625 fetch_fsmap learned FSMap version 101 2023-07-21T19:01:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.746+0000 7f2c60b70b80 1 -- 172.21.15.39:0/2005878641 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=101,monmap=4+}) v3 -- 0x56414c6e0c00 con 0x56414c978580 2023-07-21T19:01:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.746+0000 7f2c35ffb700 1 -- 172.21.15.39:0/2005878641 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56414c823490 con 0x56414c978580 2023-07-21T19:01:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.746+0000 7f2c35ffb700 1 -- 172.21.15.39:0/2005878641 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 101) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2c40021070 con 0x56414c978580 2023-07-21T19:01:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.746+0000 7f2c60b70b80 10 client.17625 fetch_fsmap finished waiting for FSMap version 101 2023-07-21T19:01:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.746+0000 7f2c60b70b80 10 client.17625 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:01:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.746+0000 7f2c60b70b80 20 client.17625 populate_metadata read hostname 'smithi039' 2023-07-21T19:01:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.746+0000 7f2c60b70b80 1 --2- 172.21.15.39:0/2005878641 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x56414c9b3bc0 0x56414c9b5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.746+0000 7f2c60b70b80 4 client.17625 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:01:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.746+0000 7f2c60b70b80 1 -- 172.21.15.39:0/2005878641 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56414c8c2e50 con 0x56414c9b3bc0 2023-07-21T19:01:46.767 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.757+0000 7f2c4ed9d700 1 --2- 172.21.15.39:0/2005878641 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x56414c9b3bc0 0x56414c9b5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:01:46.770 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.758+0000 7f2c35ffb700 10 client.17625 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T19:01:46.770 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.758+0000 7f2c4ed9d700 1 --2- 172.21.15.39:0/2005878641 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x56414c9b3bc0 0x56414c9b5fd0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=121eae3f70093118 server_cookie=9070289ea5373904 in_seq=0 out_seq=0 2023-07-21T19:01:46.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.761+0000 7f2c35ffb700 1 -- 172.21.15.39:0/2005878641 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56414c8c2e50 con 0x56414c9b3bc0 2023-07-21T19:01:46.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.761+0000 7f2c35ffb700 10 client.17625 handle_command_reply: tid=0 2023-07-21T19:01:46.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.761+0000 7f2c60b70b80 1 client.17625 shutdown 2023-07-21T19:01:46.772 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.761+0000 7f2c60b70b80 1 -- 172.21.15.39:0/2005878641 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56414c978580 msgr2=0x56414c981920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:01:46.772 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.762+0000 7f2c60b70b80 1 --2- 172.21.15.39:0/2005878641 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56414c978580 0x56414c981920 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f2c400116f0 tx=0x7f2c40011a00 comp rx=0 tx=0).stop 2023-07-21T19:01:46.773 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.763+0000 7f2c60b70b80 1 -- 172.21.15.39:0/2005878641 shutdown_connections 2023-07-21T19:01:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.763+0000 7f2c60b70b80 1 --2- 172.21.15.39:0/2005878641 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x56414c9b3bc0 0x56414c9b5fd0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.763+0000 7f2c60b70b80 1 --2- 172.21.15.39:0/2005878641 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414c97e2d0 0x56414c982010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.763+0000 7f2c60b70b80 1 --2- 172.21.15.39:0/2005878641 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56414c7d46b0 0x56414c984630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.763+0000 7f2c60b70b80 1 --2- 172.21.15.39:0/2005878641 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56414c978580 0x56414c981920 secure :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f2c400116f0 tx=0x7f2c40011a00 comp rx=0 tx=0).stop 2023-07-21T19:01:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.763+0000 7f2c35ffb700 0 client.17625 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T19:01:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.763+0000 7f2c60b70b80 1 -- 172.21.15.39:0/2005878641 >> 172.21.15.39:0/2005878641 conn(0x56414c96dbe0 msgr2=0x56414c9724e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:01:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.763+0000 7f2c60b70b80 1 -- 172.21.15.39:0/2005878641 shutdown_connections 2023-07-21T19:01:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.763+0000 7f2c60b70b80 1 -- 172.21.15.39:0/2005878641 wait complete. 2023-07-21T19:01:46.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.763+0000 7f2c60b70b80 20 client.17625 trim_cache size 0 max 16384 2023-07-21T19:01:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.790+0000 7f2c60b70b80 1 Processor -- start 2023-07-21T19:01:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.790+0000 7f2c60b70b80 1 -- start start 2023-07-21T19:01:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.790+0000 7f2c60b70b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56414c7d46b0 0x56414c978040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.790+0000 7f2c60b70b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414c97e2d0 0x56414c97d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.790+0000 7f2c60b70b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56414c97c1f0 0x56414c9838d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.790+0000 7f2c60b70b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x56414c96abb0 con 0x56414c97c1f0 2023-07-21T19:01:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.790+0000 7f2c60b70b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56414c97c6b0 con 0x56414c97e2d0 2023-07-21T19:01:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.790+0000 7f2c60b70b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x56414c9b6630 con 0x56414c7d46b0 2023-07-21T19:01:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.790+0000 7f2c4ed9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56414c97c1f0 0x56414c9838d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:01:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.790+0000 7f2c45d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414c97e2d0 0x56414c97d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:01:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.790+0000 7f2c4ed9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56414c97c1f0 0x56414c9838d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34278/0 (socket says 172.21.15.39:34278) 2023-07-21T19:01:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.790+0000 7f2c4ed9d700 1 -- 172.21.15.39:0/2804033280 learned_addr learned my addr 172.21.15.39:0/2804033280 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:01:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.790+0000 7f2c4ed9d700 1 --2- 172.21.15.39:0/2804033280 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56414c7d46b0 0x56414c978040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:01:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.791+0000 7f2c45d9b700 1 -- 172.21.15.39:0/2804033280 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56414c7d46b0 msgr2=0x56414c978040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:01:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.791+0000 7f2c45d9b700 1 --2- 172.21.15.39:0/2804033280 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56414c7d46b0 0x56414c978040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.791+0000 7f2c45d9b700 1 -- 172.21.15.39:0/2804033280 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56414c97c1f0 msgr2=0x56414c9838d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:01:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.791+0000 7f2c45d9b700 1 --2- 172.21.15.39:0/2804033280 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56414c97c1f0 0x56414c9838d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.791+0000 7f2c45d9b700 1 -- 172.21.15.39:0/2804033280 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56414c71e1f0 con 0x56414c97e2d0 2023-07-21T19:01:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.791+0000 7f2c4ed9d700 1 --2- 172.21.15.39:0/2804033280 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56414c97c1f0 0x56414c9838d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:01:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.791+0000 7f2c45d9b700 1 --2- 172.21.15.39:0/2804033280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414c97e2d0 0x56414c97d350 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x56414c9829b0 tx=0x7f2c40019e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10e994e26ab543f9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:01:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.791+0000 7f2c4559a700 1 -- 172.21.15.39:0/2804033280 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c40009400 con 0x56414c97e2d0 2023-07-21T19:01:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.791+0000 7f2c4559a700 1 -- 172.21.15.39:0/2804033280 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c4000e8c0 con 0x56414c97e2d0 2023-07-21T19:01:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.791+0000 7f2c4559a700 1 -- 172.21.15.39:0/2804033280 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c40019630 con 0x56414c97e2d0 2023-07-21T19:01:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.792+0000 7f2c60b70b80 1 -- 172.21.15.39:0/2804033280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414c97e2d0 msgr2=0x56414c97d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:01:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.792+0000 7f2c60b70b80 1 --2- 172.21.15.39:0/2804033280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414c97e2d0 0x56414c97d350 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x56414c9829b0 tx=0x7f2c40019e90 comp rx=0 tx=0).stop 2023-07-21T19:01:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.792+0000 7f2c60b70b80 1 -- 172.21.15.39:0/2804033280 shutdown_connections 2023-07-21T19:01:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.792+0000 7f2c60b70b80 1 --2- 172.21.15.39:0/2804033280 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56414c97c1f0 0x56414c9838d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.792+0000 7f2c60b70b80 1 --2- 172.21.15.39:0/2804033280 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56414c7d46b0 0x56414c978040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.792+0000 7f2c60b70b80 1 --2- 172.21.15.39:0/2804033280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414c97e2d0 0x56414c97d350 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.792+0000 7f2c60b70b80 1 -- 172.21.15.39:0/2804033280 >> 172.21.15.39:0/2804033280 conn(0x56414c96dbe0 msgr2=0x56414c9831c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:01:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.792+0000 7f2c60b70b80 1 -- 172.21.15.39:0/2804033280 shutdown_connections 2023-07-21T19:01:46.807 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.792+0000 7f2c60b70b80 1 -- 172.21.15.39:0/2804033280 wait complete. 2023-07-21T19:01:46.807 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.793+0000 7f2c60b70b80 1 -- start start 2023-07-21T19:01:46.807 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.793+0000 7f2c60b70b80 1 Processor -- start 2023-07-21T19:01:46.807 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.793+0000 7f2c60b70b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414c7d46b0 0x56414c9883d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:46.807 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.793+0000 7f2c60b70b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56414c97c1f0 0x56414c986690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:46.807 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.793+0000 7f2c60b70b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56414c97e2d0 0x56414c986cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:46.807 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.793+0000 7f2c60b70b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x56414c987360 con 0x56414c97e2d0 2023-07-21T19:01:46.808 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.793+0000 7f2c60b70b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56414c9874d0 con 0x56414c7d46b0 2023-07-21T19:01:46.808 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.793+0000 7f2c60b70b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x56414ca07a50 con 0x56414c97c1f0 2023-07-21T19:01:46.808 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.793+0000 7f2c60b70b80 10 client.0 _ll_register_callbacks cb 0x7f2c40009dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:01:46.808 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.795+0000 7f2c4ed9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414c7d46b0 0x56414c9883d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:01:46.808 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.795+0000 7f2c4ed9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414c7d46b0 0x56414c9883d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:46876/0 (socket says 172.21.15.39:46876) 2023-07-21T19:01:46.809 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.795+0000 7f2c4ed9d700 1 -- 172.21.15.39:0/3582727027 learned_addr learned my addr 172.21.15.39:0/3582727027 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:01:46.809 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.795+0000 7f2c4ed9d700 1 -- 172.21.15.39:0/3582727027 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56414c97c1f0 msgr2=0x56414c986690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:01:46.809 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.795+0000 7f2c4ed9d700 1 --2- 172.21.15.39:0/3582727027 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56414c97c1f0 0x56414c986690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:46.809 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.795+0000 7f2c4ed9d700 1 -- 172.21.15.39:0/3582727027 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56414c97e2d0 msgr2=0x56414c986cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:01:46.809 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.795+0000 7f2c4ed9d700 1 --2- 172.21.15.39:0/3582727027 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56414c97e2d0 0x56414c986cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:46.809 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.795+0000 7f2c4ed9d700 1 -- 172.21.15.39:0/3582727027 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56414c6f7d80 con 0x56414c7d46b0 2023-07-21T19:01:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.804+0000 7f2c4ed9d700 1 --2- 172.21.15.39:0/3582727027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414c7d46b0 0x56414c9883d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2c48007630 tx=0x7f2c48003b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1c6f2ef51d5dd57 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:01:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.804+0000 7f2c35ffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-21T19:01:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.804+0000 7f2c35ffb700 1 -- 172.21.15.39:0/3582727027 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4800e040 con 0x56414c7d46b0 2023-07-21T19:01:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.804+0000 7f2c35ffb700 1 -- 172.21.15.39:0/3582727027 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f2c4802fe40 con 0x56414c7d46b0 2023-07-21T19:01:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.804+0000 7f2c60b70b80 1 -- 172.21.15.39:0/3582727027 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2c48003c00 con 0x56414c7d46b0 2023-07-21T19:01:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.805+0000 7f2c35ffb700 1 -- 172.21.15.39:0/3582727027 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=101) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c4800bb90 con 0x56414c7d46b0 2023-07-21T19:01:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.805+0000 7f2c60b70b80 10 client.27044 fetch_fsmap learned FSMap version 101 2023-07-21T19:01:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.805+0000 7f2c60b70b80 1 -- 172.21.15.39:0/3582727027 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=101,monmap=4+}) v3 -- 0x56414c6e0c00 con 0x56414c7d46b0 2023-07-21T19:01:46.816 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.805+0000 7f2c35ffb700 1 -- 172.21.15.39:0/3582727027 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c48003c00 con 0x56414c7d46b0 2023-07-21T19:01:46.816 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.805+0000 7f2c35ffb700 1 -- 172.21.15.39:0/3582727027 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 101) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2c4800b3a0 con 0x56414c7d46b0 2023-07-21T19:01:46.816 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.805+0000 7f2c60b70b80 10 client.27044 fetch_fsmap finished waiting for FSMap version 101 2023-07-21T19:01:46.816 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.805+0000 7f2c60b70b80 10 client.27044 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:01:46.817 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.805+0000 7f2c60b70b80 20 client.27044 populate_metadata read hostname 'smithi039' 2023-07-21T19:01:46.817 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.805+0000 7f2c60b70b80 1 --2- 172.21.15.39:0/3582727027 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x56414ca08e40 0x56414ca0b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:46.817 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.805+0000 7f2c60b70b80 4 client.27044 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:01:46.817 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.805+0000 7f2c60b70b80 1 -- 172.21.15.39:0/3582727027 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56414c884ff0 con 0x56414ca08e40 2023-07-21T19:01:46.817 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.806+0000 7f2c45d9b700 1 --2- 172.21.15.39:0/3582727027 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x56414ca08e40 0x56414ca0b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:01:46.822 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.812+0000 7f2c35ffb700 10 client.27044 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T19:01:46.822 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:46.812+0000 7f2c45d9b700 1 --2- 172.21.15.39:0/3582727027 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x56414ca08e40 0x56414ca0b250 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c015960f920aa480 server_cookie=4350ab2fbcec15e0 in_seq=0 out_seq=0 2023-07-21T19:01:46.862 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:01:46 smithi193 ceph-mon[98204]: pgmap v635: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.0 MiB/s wr, 476 op/s 2023-07-21T19:01:46.880 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:01:46 smithi155 ceph-mon[97934]: pgmap v635: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.0 MiB/s wr, 476 op/s 2023-07-21T19:01:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:47.064+0000 7f2c35ffb700 1 -- 172.21.15.39:0/3582727027 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56414c884ff0 con 0x56414ca08e40 2023-07-21T19:01:47.076 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:47.064+0000 7f2c35ffb700 10 client.27044 handle_command_reply: tid=0 2023-07-21T19:01:47.076 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:47.064+0000 7f2c60b70b80 1 client.27044 shutdown 2023-07-21T19:01:47.076 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:47.065+0000 7f2c60b70b80 1 -- 172.21.15.39:0/3582727027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414c7d46b0 msgr2=0x56414c9883d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:01:47.076 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:47.065+0000 7f2c60b70b80 1 --2- 172.21.15.39:0/3582727027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414c7d46b0 0x56414c9883d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2c48007630 tx=0x7f2c48003b50 comp rx=0 tx=0).stop 2023-07-21T19:01:47.076 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:47.065+0000 7f2c60b70b80 1 -- 172.21.15.39:0/3582727027 shutdown_connections 2023-07-21T19:01:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:47.065+0000 7f2c60b70b80 1 --2- 172.21.15.39:0/3582727027 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x56414ca08e40 0x56414ca0b250 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:47.065+0000 7f2c60b70b80 1 --2- 172.21.15.39:0/3582727027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414c7d46b0 0x56414c9883d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:47.065+0000 7f2c60b70b80 1 --2- 172.21.15.39:0/3582727027 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56414c97e2d0 0x56414c986cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:47.065+0000 7f2c60b70b80 1 --2- 172.21.15.39:0/3582727027 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56414c97c1f0 0x56414c986690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:47.065+0000 7f2c35ffb700 0 client.27044 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T19:01:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:47.065+0000 7f2c60b70b80 1 -- 172.21.15.39:0/3582727027 >> 172.21.15.39:0/3582727027 conn(0x56414c96dbe0 msgr2=0x56414c98d2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:01:47.080 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:47.070+0000 7f2c60b70b80 1 -- 172.21.15.39:0/3582727027 shutdown_connections 2023-07-21T19:01:47.081 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:47.070+0000 7f2c60b70b80 1 -- 172.21.15.39:0/3582727027 wait complete. 2023-07-21T19:01:47.082 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:47.072+0000 7f2c60b70b80 20 client.27044 trim_cache size 0 max 16384 2023-07-21T19:01:47.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T19:01:47.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T19:01:47.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T19:01:47.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T19:01:47.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:47.074+0000 7f2c549ea700 1 -- 172.21.15.39:0/698068138 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f2c3c0a1400 msgr2=0x7f2c3c0a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:01:47.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:47.074+0000 7f2c549ea700 1 --2- 172.21.15.39:0/698068138 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f2c3c0a1400 0x7f2c3c0a38b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2c38099330 tx=0x7f2c38004860 comp rx=0 tx=0).stop 2023-07-21T19:01:47.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:47.074+0000 7f2c549ea700 1 -- 172.21.15.39:0/698068138 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c5006c910 msgr2=0x7f2c5011aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:01:47.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:47.074+0000 7f2c549ea700 1 --2- 172.21.15.39:0/698068138 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c5006c910 0x7f2c5011aea0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2c3800b450 tx=0x7f2c3800b760 comp rx=0 tx=0).stop 2023-07-21T19:01:47.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:47.074+0000 7f2c549ea700 1 -- 172.21.15.39:0/698068138 shutdown_connections 2023-07-21T19:01:47.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:47.074+0000 7f2c549ea700 1 --2- 172.21.15.39:0/698068138 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f2c3c0a1400 0x7f2c3c0a38b0 secure :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2c38099330 tx=0x7f2c38004860 comp rx=0 tx=0).stop 2023-07-21T19:01:47.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:47.074+0000 7f2c549ea700 1 --2- 172.21.15.39:0/698068138 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c5010e2a0 0x7f2c50114dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:47.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:47.074+0000 7f2c549ea700 1 --2- 172.21.15.39:0/698068138 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c5006c910 0x7f2c5011aea0 secure :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2c3800b450 tx=0x7f2c3800b760 comp rx=0 tx=0).stop 2023-07-21T19:01:47.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:47.074+0000 7f2c549ea700 1 --2- 172.21.15.39:0/698068138 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c5006d3a0 0x7f2c5011b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:47.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:47.074+0000 7f2c549ea700 1 -- 172.21.15.39:0/698068138 >> 172.21.15.39:0/698068138 conn(0x7f2c5006bd70 msgr2=0x7f2c5006f110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:01:47.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:47.074+0000 7f2c549ea700 1 -- 172.21.15.39:0/698068138 shutdown_connections 2023-07-21T19:01:47.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:01:47.074+0000 7f2c549ea700 1 -- 172.21.15.39:0/698068138 wait complete. 2023-07-21T19:01:47.699 INFO:tasks.workunit.client.0.smithi039.stdout: 10 234777 30.10 MB/sec execute 300 sec latency 242.467 ms 2023-07-21T19:01:48.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:01:48 smithi039 ceph-mon[96489]: pgmap v636: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 23 MiB/s wr, 855 op/s 2023-07-21T19:01:48.699 INFO:tasks.workunit.client.0.smithi039.stdout: 10 235462 30.12 MB/sec execute 301 sec latency 103.561 ms 2023-07-21T19:01:48.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:01:48 smithi193 ceph-mon[98204]: pgmap v636: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 23 MiB/s wr, 855 op/s 2023-07-21T19:01:48.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:01:48 smithi155 ceph-mon[97934]: pgmap v636: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 23 MiB/s wr, 855 op/s 2023-07-21T19:01:49.699 INFO:tasks.workunit.client.0.smithi039.stdout: 10 236048 30.16 MB/sec execute 302 sec latency 78.589 ms 2023-07-21T19:01:50.699 INFO:tasks.workunit.client.0.smithi039.stdout: 10 236569 30.14 MB/sec execute 303 sec latency 108.730 ms 2023-07-21T19:01:50.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:01:50 smithi193 ceph-mon[98204]: pgmap v637: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 19 MiB/s wr, 590 op/s 2023-07-21T19:01:50.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:01:50 smithi155 ceph-mon[97934]: pgmap v637: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 19 MiB/s wr, 590 op/s 2023-07-21T19:01:50.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:01:50 smithi039 ceph-mon[96489]: pgmap v637: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 19 MiB/s wr, 590 op/s 2023-07-21T19:01:51.699 INFO:tasks.workunit.client.0.smithi039.stdout: 10 237051 30.11 MB/sec execute 304 sec latency 107.297 ms 2023-07-21T19:01:52.699 INFO:tasks.workunit.client.0.smithi039.stdout: 10 237660 30.12 MB/sec execute 305 sec latency 91.691 ms 2023-07-21T19:01:52.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:01:52 smithi193 ceph-mon[98204]: pgmap v638: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 19 MiB/s wr, 617 op/s 2023-07-21T19:01:52.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:01:52 smithi155 ceph-mon[97934]: pgmap v638: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 19 MiB/s wr, 617 op/s 2023-07-21T19:01:52.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:01:52 smithi039 ceph-mon[96489]: pgmap v638: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 19 MiB/s wr, 617 op/s 2023-07-21T19:01:53.699 INFO:tasks.workunit.client.0.smithi039.stdout: 10 238311 30.13 MB/sec execute 306 sec latency 104.641 ms 2023-07-21T19:01:54.699 INFO:tasks.workunit.client.0.smithi039.stdout: 10 238821 30.14 MB/sec execute 307 sec latency 174.173 ms 2023-07-21T19:01:54.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:01:54 smithi193 ceph-mon[98204]: pgmap v639: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 22 MiB/s wr, 936 op/s 2023-07-21T19:01:54.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:01:54 smithi155 ceph-mon[97934]: pgmap v639: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 22 MiB/s wr, 936 op/s 2023-07-21T19:01:54.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:01:54 smithi039 ceph-mon[96489]: pgmap v639: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 22 MiB/s wr, 936 op/s 2023-07-21T19:01:55.699 INFO:tasks.workunit.client.0.smithi039.stdout: 10 239418 30.13 MB/sec execute 308 sec latency 90.652 ms 2023-07-21T19:01:56.699 INFO:tasks.workunit.client.0.smithi039.stdout: 10 240062 30.15 MB/sec execute 309 sec latency 78.102 ms 2023-07-21T19:01:56.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:01:56 smithi193 ceph-mon[98204]: pgmap v640: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 18 MiB/s wr, 734 op/s 2023-07-21T19:01:56.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:01:56 smithi155 ceph-mon[97934]: pgmap v640: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 18 MiB/s wr, 734 op/s 2023-07-21T19:01:56.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:01:56 smithi039 ceph-mon[96489]: pgmap v640: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 18 MiB/s wr, 734 op/s 2023-07-21T19:01:57.699 INFO:tasks.workunit.client.0.smithi039.stdout: 10 240783 30.18 MB/sec execute 310 sec latency 60.547 ms 2023-07-21T19:01:58.699 INFO:tasks.workunit.client.0.smithi039.stdout: 10 241366 30.18 MB/sec execute 311 sec latency 77.553 ms 2023-07-21T19:01:58.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:01:58 smithi193 ceph-mon[98204]: pgmap v641: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 22 MiB/s wr, 943 op/s 2023-07-21T19:01:58.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:01:58 smithi155 ceph-mon[97934]: pgmap v641: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 22 MiB/s wr, 943 op/s 2023-07-21T19:01:58.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:01:58 smithi039 ceph-mon[96489]: pgmap v641: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 22 MiB/s wr, 943 op/s 2023-07-21T19:01:59.700 INFO:tasks.workunit.client.0.smithi039.stdout: 10 241885 30.18 MB/sec execute 312 sec latency 141.558 ms 2023-07-21T19:02:00.265 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.254+0000 7f830513c700 1 -- 172.21.15.39:0/3237723182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8300104a20 msgr2=0x7f8300104df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:02:00.265 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.254+0000 7f830513c700 1 --2- 172.21.15.39:0/3237723182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8300104a20 0x7f8300104df0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f82e8009990 tx=0x7f82e8009ca0 comp rx=0 tx=0).stop 2023-07-21T19:02:00.266 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.255+0000 7f830513c700 1 -- 172.21.15.39:0/3237723182 shutdown_connections 2023-07-21T19:02:00.266 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.255+0000 7f830513c700 1 --2- 172.21.15.39:0/3237723182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8300104a20 0x7f8300104df0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:00.266 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.255+0000 7f830513c700 1 --2- 172.21.15.39:0/3237723182 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83000ff3d0 0x7f830010b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:00.267 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.255+0000 7f830513c700 1 --2- 172.21.15.39:0/3237723182 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83000fea20 0x7f83000fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:00.267 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.255+0000 7f830513c700 1 -- 172.21.15.39:0/3237723182 >> 172.21.15.39:0/3237723182 conn(0x7f83000769b0 msgr2=0x7f8300076db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:02:00.268 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.257+0000 7f830513c700 1 -- 172.21.15.39:0/3237723182 shutdown_connections 2023-07-21T19:02:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.258+0000 7f830513c700 1 -- 172.21.15.39:0/3237723182 wait complete. 2023-07-21T19:02:00.270 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.259+0000 7f830513c700 1 Processor -- start 2023-07-21T19:02:00.271 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.260+0000 7f830513c700 1 -- start start 2023-07-21T19:02:00.271 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.260+0000 7f830513c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83000fea20 0x7f830010bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:00.271 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.260+0000 7f830513c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83000ff3d0 0x7f830010c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:00.271 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.260+0000 7f830513c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8300104a20 0x7f830010ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:00.271 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.260+0000 7f830513c700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83001acca0 con 0x7f83000ff3d0 2023-07-21T19:02:00.271 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.260+0000 7f830513c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83001ace10 con 0x7f8300104a20 2023-07-21T19:02:00.272 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.260+0000 7f830513c700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83001acfd0 con 0x7f83000fea20 2023-07-21T19:02:00.273 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.263+0000 7f82fed9d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83000fea20 0x7f830010bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:00.274 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.263+0000 7f82fed9d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83000fea20 0x7f830010bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:52846/0 (socket says 172.21.15.39:52846) 2023-07-21T19:02:00.274 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.263+0000 7f82fed9d700 1 -- 172.21.15.39:0/1769958076 learned_addr learned my addr 172.21.15.39:0/1769958076 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:02:00.274 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.263+0000 7f82fe59c700 1 --2- 172.21.15.39:0/1769958076 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83000ff3d0 0x7f830010c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:00.275 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.263+0000 7f82fed9d700 1 -- 172.21.15.39:0/1769958076 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8300104a20 msgr2=0x7f830010ca20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-21T19:02:00.275 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.263+0000 7f82fed9d700 1 --2- 172.21.15.39:0/1769958076 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8300104a20 0x7f830010ca20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:00.275 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.263+0000 7f82fed9d700 1 -- 172.21.15.39:0/1769958076 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83000ff3d0 msgr2=0x7f830010c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:02:00.275 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.263+0000 7f82fed9d700 1 --2- 172.21.15.39:0/1769958076 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83000ff3d0 0x7f830010c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:00.275 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.263+0000 7f82fed9d700 1 -- 172.21.15.39:0/1769958076 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e80095f0 con 0x7f83000fea20 2023-07-21T19:02:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.263+0000 7f82fe59c700 1 --2- 172.21.15.39:0/1769958076 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83000ff3d0 0x7f830010c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:02:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.263+0000 7f82fed9d700 1 --2- 172.21.15.39:0/1769958076 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83000fea20 0x7f830010bfa0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f82e8000c00 tx=0x7f82e8003c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:02:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.264+0000 7f82f7fff700 1 -- 172.21.15.39:0/1769958076 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e801d070 con 0x7f83000fea20 2023-07-21T19:02:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.264+0000 7f82f7fff700 1 -- 172.21.15.39:0/1769958076 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82e800f460 con 0x7f83000fea20 2023-07-21T19:02:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.264+0000 7f830513c700 1 -- 172.21.15.39:0/1769958076 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83001ad250 con 0x7f83000fea20 2023-07-21T19:02:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.265+0000 7f830513c700 1 -- 172.21.15.39:0/1769958076 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83001ad670 con 0x7f83000fea20 2023-07-21T19:02:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.265+0000 7f82f7fff700 1 -- 172.21.15.39:0/1769958076 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e8017650 con 0x7f83000fea20 2023-07-21T19:02:00.280 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.269+0000 7f82f7fff700 1 -- 172.21.15.39:0/1769958076 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f82e80177b0 con 0x7f83000fea20 2023-07-21T19:02:00.281 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.269+0000 7f82f7fff700 1 --2- 172.21.15.39:0/1769958076 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f82ec0a1410 0x7f82ec0a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:00.281 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.270+0000 7f82f7fff700 1 -- 172.21.15.39:0/1769958076 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f82e80cedf0 con 0x7f83000fea20 2023-07-21T19:02:00.281 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.270+0000 7f82fe59c700 1 --2- 172.21.15.39:0/1769958076 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f82ec0a1410 0x7f82ec0a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:00.282 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.270+0000 7f82fe59c700 1 --2- 172.21.15.39:0/1769958076 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f82ec0a1410 0x7f82ec0a38c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f830010d9a0 tx=0x7f82f00073a0 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:02:00.283 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.271+0000 7f83112c2b80 1 Processor -- start 2023-07-21T19:02:00.283 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.271+0000 7f83112c2b80 1 -- start start 2023-07-21T19:02:00.283 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.271+0000 7f83112c2b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b67977f6b0 0x55b6799315a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:00.283 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.271+0000 7f83112c2b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b679931ae0 0x55b67992b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:00.284 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.271+0000 7f83112c2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b67992bd90 0x55b67992c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:00.284 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.271+0000 7f83112c2b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b679828c50 con 0x55b679931ae0 2023-07-21T19:02:00.284 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.271+0000 7f83112c2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b67987b9e0 con 0x55b67992bd90 2023-07-21T19:02:00.285 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.271+0000 7f83112c2b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b679874e70 con 0x55b67977f6b0 2023-07-21T19:02:00.285 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.272+0000 7f82fed9d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b67977f6b0 0x55b6799315a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:00.285 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.272+0000 7f82fed9d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b67977f6b0 0x55b6799315a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:52850/0 (socket says 172.21.15.39:52850) 2023-07-21T19:02:00.286 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.272+0000 7f82fed9d700 1 -- 172.21.15.39:0/1023057110 learned_addr learned my addr 172.21.15.39:0/1023057110 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:02:00.286 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.272+0000 7f82fe59c700 1 --2- 172.21.15.39:0/1023057110 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b679931ae0 0x55b67992b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:00.286 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.272+0000 7f82ff59e700 1 --2- 172.21.15.39:0/1023057110 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b67992bd90 0x55b67992c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:00.286 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.272+0000 7f82fed9d700 1 -- 172.21.15.39:0/1023057110 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b67992bd90 msgr2=0x55b67992c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:02:00.287 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.272+0000 7f82fed9d700 1 --2- 172.21.15.39:0/1023057110 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b67992bd90 0x55b67992c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:00.287 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.272+0000 7f82fed9d700 1 -- 172.21.15.39:0/1023057110 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b679931ae0 msgr2=0x55b67992b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:02:00.287 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.272+0000 7f82fed9d700 1 --2- 172.21.15.39:0/1023057110 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b679931ae0 0x55b67992b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:00.288 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.272+0000 7f82fed9d700 1 -- 172.21.15.39:0/1023057110 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b6796a2d80 con 0x55b67977f6b0 2023-07-21T19:02:00.292 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.272+0000 7f82ff59e700 1 --2- 172.21.15.39:0/1023057110 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b67992bd90 0x55b67992c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:02:00.293 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.272+0000 7f82fed9d700 1 --2- 172.21.15.39:0/1023057110 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b67977f6b0 0x55b6799315a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f82e8005f50 tx=0x7f82e809b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b6a749ba6734fef server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:02:00.293 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.272+0000 7f82f57fa700 1 -- 172.21.15.39:0/1023057110 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e809ff80 con 0x55b67977f6b0 2023-07-21T19:02:00.293 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.273+0000 7f82f57fa700 1 -- 172.21.15.39:0/1023057110 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82e80a4310 con 0x55b67977f6b0 2023-07-21T19:02:00.294 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.273+0000 7f82f57fa700 1 -- 172.21.15.39:0/1023057110 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e809b440 con 0x55b67977f6b0 2023-07-21T19:02:00.294 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.274+0000 7f83112c2b80 1 -- 172.21.15.39:0/1023057110 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b67977f6b0 msgr2=0x55b6799315a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:02:00.294 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.274+0000 7f83112c2b80 1 --2- 172.21.15.39:0/1023057110 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b67977f6b0 0x55b6799315a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f82e8005f50 tx=0x7f82e809b8b0 comp rx=0 tx=0).stop 2023-07-21T19:02:00.295 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.277+0000 7f83112c2b80 1 -- 172.21.15.39:0/1023057110 shutdown_connections 2023-07-21T19:02:00.295 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.277+0000 7f83112c2b80 1 --2- 172.21.15.39:0/1023057110 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b67992bd90 0x55b67992c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:00.295 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.277+0000 7f83112c2b80 1 --2- 172.21.15.39:0/1023057110 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b67977f6b0 0x55b6799315a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:00.295 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.277+0000 7f83112c2b80 1 --2- 172.21.15.39:0/1023057110 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b679931ae0 0x55b67992b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:00.295 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.277+0000 7f83112c2b80 1 -- 172.21.15.39:0/1023057110 >> 172.21.15.39:0/1023057110 conn(0x55b679918be0 msgr2=0x55b67991df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:02:00.296 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.277+0000 7f83112c2b80 1 -- 172.21.15.39:0/1023057110 shutdown_connections 2023-07-21T19:02:00.296 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.277+0000 7f83112c2b80 1 -- 172.21.15.39:0/1023057110 wait complete. 2023-07-21T19:02:00.296 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.277+0000 7f83112c2b80 1 -- start start 2023-07-21T19:02:00.296 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.278+0000 7f83112c2b80 1 Processor -- start 2023-07-21T19:02:00.296 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.278+0000 7f83112c2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b67977f6b0 0x55b67993ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:00.297 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.278+0000 7f83112c2b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b67992bd90 0x55b67993b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:00.297 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.278+0000 7f83112c2b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b679931ae0 0x55b679934e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:00.297 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.278+0000 7f83112c2b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b679814a60 con 0x55b679931ae0 2023-07-21T19:02:00.297 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.278+0000 7f83112c2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b679814760 con 0x55b67977f6b0 2023-07-21T19:02:00.297 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.278+0000 7f83112c2b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6796f88c0 con 0x55b67992bd90 2023-07-21T19:02:00.297 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.278+0000 7f83112c2b80 10 client.0 _ll_register_callbacks cb 0x55b679812700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:02:00.297 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.278+0000 7f82fed9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b67977f6b0 0x55b67993ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:00.298 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.278+0000 7f82fed9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b67977f6b0 0x55b67993ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:46900/0 (socket says 172.21.15.39:46900) 2023-07-21T19:02:00.298 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.278+0000 7f82fed9d700 1 -- 172.21.15.39:0/3681342969 learned_addr learned my addr 172.21.15.39:0/3681342969 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:02:00.298 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.278+0000 7f82fe59c700 1 --2- 172.21.15.39:0/3681342969 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b67992bd90 0x55b67993b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:00.298 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.279+0000 7f82fe59c700 1 -- 172.21.15.39:0/3681342969 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b67977f6b0 msgr2=0x55b67993ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:02:00.298 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.279+0000 7f82fe59c700 1 --2- 172.21.15.39:0/3681342969 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b67977f6b0 0x55b67993ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:00.298 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.279+0000 7f82fe59c700 1 -- 172.21.15.39:0/3681342969 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b679931ae0 msgr2=0x55b679934e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:02:00.298 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.279+0000 7f82fe59c700 1 --2- 172.21.15.39:0/3681342969 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b679931ae0 0x55b679934e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:00.299 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.279+0000 7f82fe59c700 1 -- 172.21.15.39:0/3681342969 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83001ad670 con 0x55b67992bd90 2023-07-21T19:02:00.299 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.279+0000 7f82fed9d700 1 --2- 172.21.15.39:0/3681342969 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b67977f6b0 0x55b67993ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:02:00.299 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.279+0000 7f82fe59c700 1 --2- 172.21.15.39:0/3681342969 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b67992bd90 0x55b67993b0b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f82f0007780 tx=0x7f82f0004b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=780153d9de4c8e5b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:02:00.299 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.279+0000 7f82e67fc700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2023-07-21T19:02:00.299 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.279+0000 7f82e67fc700 1 -- 172.21.15.39:0/3681342969 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f000ccf0 con 0x55b67992bd90 2023-07-21T19:02:00.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.279+0000 7f83112c2b80 1 -- 172.21.15.39:0/3681342969 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b6797ce490 con 0x55b67992bd90 2023-07-21T19:02:00.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.279+0000 7f82e67fc700 1 -- 172.21.15.39:0/3681342969 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f82f0008040 con 0x55b67992bd90 2023-07-21T19:02:00.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.280+0000 7f82e67fc700 1 -- 172.21.15.39:0/3681342969 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=101) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82f000ce50 con 0x55b67992bd90 2023-07-21T19:02:00.301 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.280+0000 7f83112c2b80 10 client.27130 fetch_fsmap learned FSMap version 101 2023-07-21T19:02:00.301 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.280+0000 7f83112c2b80 1 -- 172.21.15.39:0/3681342969 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=101,monmap=4+}) v3 -- 0x55b67968bc00 con 0x55b67992bd90 2023-07-21T19:02:00.301 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.280+0000 7f82e67fc700 1 -- 172.21.15.39:0/3681342969 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b6797ce490 con 0x55b67992bd90 2023-07-21T19:02:00.301 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.280+0000 7f82e67fc700 1 -- 172.21.15.39:0/3681342969 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 101) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f82f0023030 con 0x55b67992bd90 2023-07-21T19:02:00.302 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.280+0000 7f83112c2b80 10 client.27130 fetch_fsmap finished waiting for FSMap version 101 2023-07-21T19:02:00.302 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.280+0000 7f83112c2b80 10 client.27130 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:02:00.302 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.280+0000 7f83112c2b80 20 client.27130 populate_metadata read hostname 'smithi039' 2023-07-21T19:02:00.303 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.280+0000 7f83112c2b80 1 --2- 172.21.15.39:0/3681342969 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55b679945110 0x55b679947520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:00.303 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.280+0000 7f83112c2b80 4 client.27130 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:02:00.303 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.280+0000 7f83112c2b80 1 -- 172.21.15.39:0/3681342969 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b67986de50 con 0x55b679945110 2023-07-21T19:02:00.303 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.280+0000 7f82fed9d700 1 --2- 172.21.15.39:0/3681342969 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55b679945110 0x55b679947520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:00.304 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.281+0000 7f82e67fc700 10 client.27130 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T19:02:00.304 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.281+0000 7f82fed9d700 1 --2- 172.21.15.39:0/3681342969 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55b679945110 0x55b679947520 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d64a242e61f5ad3d server_cookie=a5a5f7914fab7a04 in_seq=0 out_seq=0 2023-07-21T19:02:00.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.284+0000 7f82e67fc700 1 -- 172.21.15.39:0/3681342969 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b67986de50 con 0x55b679945110 2023-07-21T19:02:00.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.284+0000 7f82e67fc700 10 client.27130 handle_command_reply: tid=0 2023-07-21T19:02:00.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.284+0000 7f83112c2b80 1 client.27130 shutdown 2023-07-21T19:02:00.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.284+0000 7f83112c2b80 1 -- 172.21.15.39:0/3681342969 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b67992bd90 msgr2=0x55b67993b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:02:00.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.284+0000 7f83112c2b80 1 --2- 172.21.15.39:0/3681342969 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b67992bd90 0x55b67993b0b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f82f0007780 tx=0x7f82f0004b80 comp rx=0 tx=0).stop 2023-07-21T19:02:00.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.284+0000 7f83112c2b80 1 -- 172.21.15.39:0/3681342969 shutdown_connections 2023-07-21T19:02:00.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.284+0000 7f83112c2b80 1 --2- 172.21.15.39:0/3681342969 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55b679945110 0x55b679947520 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:00.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.284+0000 7f83112c2b80 1 --2- 172.21.15.39:0/3681342969 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b67977f6b0 0x55b67993ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:00.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.284+0000 7f83112c2b80 1 --2- 172.21.15.39:0/3681342969 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b679931ae0 0x55b679934e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:00.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.284+0000 7f83112c2b80 1 --2- 172.21.15.39:0/3681342969 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b67992bd90 0x55b67993b0b0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:00.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.284+0000 7f83112c2b80 1 -- 172.21.15.39:0/3681342969 >> 172.21.15.39:0/3681342969 conn(0x55b679918be0 msgr2=0x55b67991d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:02:00.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.284+0000 7f82e67fc700 0 client.27130 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T19:02:00.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.284+0000 7f83112c2b80 1 -- 172.21.15.39:0/3681342969 shutdown_connections 2023-07-21T19:02:00.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.284+0000 7f83112c2b80 1 -- 172.21.15.39:0/3681342969 wait complete. 2023-07-21T19:02:00.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.284+0000 7f83112c2b80 20 client.27130 trim_cache size 0 max 16384 2023-07-21T19:02:00.324 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.311+0000 7f83112c2b80 1 Processor -- start 2023-07-21T19:02:00.325 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.311+0000 7f83112c2b80 1 -- start start 2023-07-21T19:02:00.325 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.312+0000 7f83112c2b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b67977f6b0 0x55b67992b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:00.325 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.312+0000 7f83112c2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b67992bd90 0x55b679934f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:00.325 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.312+0000 7f83112c2b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b679931ae0 0x55b679932820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:00.326 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.312+0000 7f83112c2b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b679936250 con 0x55b679931ae0 2023-07-21T19:02:00.326 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.312+0000 7f83112c2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b679915bb0 con 0x55b67992bd90 2023-07-21T19:02:00.326 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.312+0000 7f83112c2b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b67991a5f0 con 0x55b67977f6b0 2023-07-21T19:02:00.326 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.312+0000 7f82fed9d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b67977f6b0 0x55b67992b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:00.326 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.312+0000 7f82ff59e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b679931ae0 0x55b679932820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:00.326 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.312+0000 7f82fed9d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b67977f6b0 0x55b67992b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:52864/0 (socket says 172.21.15.39:52864) 2023-07-21T19:02:00.327 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.312+0000 7f82fed9d700 1 -- 172.21.15.39:0/4155057142 learned_addr learned my addr 172.21.15.39:0/4155057142 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:02:00.327 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.312+0000 7f82fe59c700 1 --2- 172.21.15.39:0/4155057142 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b67992bd90 0x55b679934f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:00.327 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.312+0000 7f82fed9d700 1 -- 172.21.15.39:0/4155057142 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b67992bd90 msgr2=0x55b679934f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:02:00.327 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.312+0000 7f82fed9d700 1 --2- 172.21.15.39:0/4155057142 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b67992bd90 0x55b679934f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:00.327 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.312+0000 7f82fed9d700 1 -- 172.21.15.39:0/4155057142 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b679931ae0 msgr2=0x55b679932820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:02:00.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.312+0000 7f82fed9d700 1 --2- 172.21.15.39:0/4155057142 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b679931ae0 0x55b679932820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:00.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.312+0000 7f82fed9d700 1 -- 172.21.15.39:0/4155057142 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b6796c91f0 con 0x55b67977f6b0 2023-07-21T19:02:00.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.312+0000 7f82ff59e700 1 --2- 172.21.15.39:0/4155057142 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b679931ae0 0x55b679932820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:02:00.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.313+0000 7f82fed9d700 1 --2- 172.21.15.39:0/4155057142 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b67977f6b0 0x55b67992b850 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f82e809bee0 tx=0x7f82e8005ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6e0d4e0e9ef74d1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:02:00.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.313+0000 7f82f5ffb700 1 -- 172.21.15.39:0/4155057142 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e809a2f0 con 0x55b67977f6b0 2023-07-21T19:02:00.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.313+0000 7f82f5ffb700 1 -- 172.21.15.39:0/4155057142 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82e809b440 con 0x55b67977f6b0 2023-07-21T19:02:00.329 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.313+0000 7f82f5ffb700 1 -- 172.21.15.39:0/4155057142 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e80a3e80 con 0x55b67977f6b0 2023-07-21T19:02:00.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.314+0000 7f83112c2b80 1 -- 172.21.15.39:0/4155057142 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b67977f6b0 msgr2=0x55b67992b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:02:00.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.314+0000 7f83112c2b80 1 --2- 172.21.15.39:0/4155057142 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b67977f6b0 0x55b67992b850 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f82e809bee0 tx=0x7f82e8005ad0 comp rx=0 tx=0).stop 2023-07-21T19:02:00.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.314+0000 7f83112c2b80 1 -- 172.21.15.39:0/4155057142 shutdown_connections 2023-07-21T19:02:00.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.314+0000 7f83112c2b80 1 --2- 172.21.15.39:0/4155057142 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b679931ae0 0x55b679932820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:00.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.314+0000 7f83112c2b80 1 --2- 172.21.15.39:0/4155057142 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b67977f6b0 0x55b67992b850 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:00.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.314+0000 7f83112c2b80 1 --2- 172.21.15.39:0/4155057142 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b67992bd90 0x55b679934f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:00.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.314+0000 7f83112c2b80 1 -- 172.21.15.39:0/4155057142 >> 172.21.15.39:0/4155057142 conn(0x55b679918be0 msgr2=0x55b679932110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:02:00.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.314+0000 7f83112c2b80 1 -- 172.21.15.39:0/4155057142 shutdown_connections 2023-07-21T19:02:00.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.314+0000 7f83112c2b80 1 -- 172.21.15.39:0/4155057142 wait complete. 2023-07-21T19:02:00.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.314+0000 7f83112c2b80 1 -- start start 2023-07-21T19:02:00.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.314+0000 7f83112c2b80 1 Processor -- start 2023-07-21T19:02:00.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.315+0000 7f83112c2b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b67977f6b0 0x55b67993af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:00.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.315+0000 7f83112c2b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b67992bd90 0x55b6799384b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:00.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.315+0000 7f83112c2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b679931ae0 0x55b679938a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:00.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.315+0000 7f83112c2b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b679939190 con 0x55b67992bd90 2023-07-21T19:02:00.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.315+0000 7f83112c2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b679939300 con 0x55b679931ae0 2023-07-21T19:02:00.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.315+0000 7f83112c2b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b679997f90 con 0x55b67977f6b0 2023-07-21T19:02:00.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.315+0000 7f83112c2b80 10 client.0 _ll_register_callbacks cb 0x7f82f000f7c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:02:00.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.315+0000 7f82fe59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b67992bd90 0x55b6799384b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:00.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.315+0000 7f82fed9d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b67977f6b0 0x55b67993af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:00.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.315+0000 7f82fed9d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b67977f6b0 0x55b67993af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:52870/0 (socket says 172.21.15.39:52870) 2023-07-21T19:02:00.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.315+0000 7f82fed9d700 1 -- 172.21.15.39:0/923415345 learned_addr learned my addr 172.21.15.39:0/923415345 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:02:00.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.315+0000 7f82ff59e700 1 --2- 172.21.15.39:0/923415345 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b679931ae0 0x55b679938a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:00.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.315+0000 7f82fe59c700 1 -- 172.21.15.39:0/923415345 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b67977f6b0 msgr2=0x55b67993af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:02:00.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.315+0000 7f82fe59c700 1 --2- 172.21.15.39:0/923415345 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b67977f6b0 0x55b67993af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:00.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.315+0000 7f82fe59c700 1 -- 172.21.15.39:0/923415345 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b679931ae0 msgr2=0x55b679938a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:02:00.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.315+0000 7f82fe59c700 1 --2- 172.21.15.39:0/923415345 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b679931ae0 0x55b679938a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:00.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.315+0000 7f82fe59c700 1 -- 172.21.15.39:0/923415345 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82f800e040 con 0x55b67992bd90 2023-07-21T19:02:00.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.315+0000 7f82fed9d700 1 --2- 172.21.15.39:0/923415345 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b67977f6b0 0x55b67993af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:02:00.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.315+0000 7f82fe59c700 1 --2- 172.21.15.39:0/923415345 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b67992bd90 0x55b6799384b0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f82f0004540 tx=0x7f82f000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a072395ed462a295 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:02:00.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.315+0000 7f82e67fc700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T19:02:00.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.315+0000 7f82e67fc700 1 -- 172.21.15.39:0/923415345 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f000ec70 con 0x55b67992bd90 2023-07-21T19:02:00.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.315+0000 7f82e67fc700 1 -- 172.21.15.39:0/923415345 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f82f000edd0 con 0x55b67992bd90 2023-07-21T19:02:00.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.315+0000 7f83112c2b80 1 -- 172.21.15.39:0/923415345 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f82e80a4310 con 0x55b67992bd90 2023-07-21T19:02:00.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.316+0000 7f82e67fc700 1 -- 172.21.15.39:0/923415345 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=101) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82f0007b00 con 0x55b67992bd90 2023-07-21T19:02:00.338 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.316+0000 7f83112c2b80 10 client.17637 fetch_fsmap learned FSMap version 101 2023-07-21T19:02:00.338 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.316+0000 7f83112c2b80 1 -- 172.21.15.39:0/923415345 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=101,monmap=4+}) v3 -- 0x55b6796a2d80 con 0x55b67992bd90 2023-07-21T19:02:00.338 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.316+0000 7f82e67fc700 1 -- 172.21.15.39:0/923415345 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82f0044080 con 0x55b67992bd90 2023-07-21T19:02:00.338 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.316+0000 7f82e67fc700 1 -- 172.21.15.39:0/923415345 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 101) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f82f000e380 con 0x55b67992bd90 2023-07-21T19:02:00.338 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.316+0000 7f83112c2b80 10 client.17637 fetch_fsmap finished waiting for FSMap version 101 2023-07-21T19:02:00.338 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.316+0000 7f83112c2b80 10 client.17637 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:02:00.338 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.316+0000 7f83112c2b80 20 client.17637 populate_metadata read hostname 'smithi039' 2023-07-21T19:02:00.339 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.316+0000 7f83112c2b80 1 --2- 172.21.15.39:0/923415345 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55b679999520 0x55b67999b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:00.339 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.316+0000 7f83112c2b80 4 client.17637 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:02:00.339 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.316+0000 7f83112c2b80 1 -- 172.21.15.39:0/923415345 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b67982fff0 con 0x55b679999520 2023-07-21T19:02:00.339 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.316+0000 7f82fed9d700 1 --2- 172.21.15.39:0/923415345 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55b679999520 0x55b67999b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:00.339 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.317+0000 7f82fed9d700 1 --2- 172.21.15.39:0/923415345 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55b679999520 0x55b67999b930 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=758e93c0f0d4399e server_cookie=c4b425ef0b7f9e58 in_seq=0 out_seq=0 2023-07-21T19:02:00.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.317+0000 7f82e67fc700 10 client.17637 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T19:02:00.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.520+0000 7f82e67fc700 1 -- 172.21.15.39:0/923415345 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b67982fff0 con 0x55b679999520 2023-07-21T19:02:00.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.520+0000 7f82e67fc700 10 client.17637 handle_command_reply: tid=0 2023-07-21T19:02:00.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.520+0000 7f83112c2b80 1 client.17637 shutdown 2023-07-21T19:02:00.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.521+0000 7f83112c2b80 1 -- 172.21.15.39:0/923415345 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b67992bd90 msgr2=0x55b6799384b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:02:00.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.521+0000 7f83112c2b80 1 --2- 172.21.15.39:0/923415345 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b67992bd90 0x55b6799384b0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f82f0004540 tx=0x7f82f000cd70 comp rx=0 tx=0).stop 2023-07-21T19:02:00.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.521+0000 7f83112c2b80 1 -- 172.21.15.39:0/923415345 shutdown_connections 2023-07-21T19:02:00.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.521+0000 7f83112c2b80 1 --2- 172.21.15.39:0/923415345 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55b679999520 0x55b67999b930 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:00.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.521+0000 7f83112c2b80 1 --2- 172.21.15.39:0/923415345 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b679931ae0 0x55b679938a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:00.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.521+0000 7f83112c2b80 1 --2- 172.21.15.39:0/923415345 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b67977f6b0 0x55b67993af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:00.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.521+0000 7f83112c2b80 1 --2- 172.21.15.39:0/923415345 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b67992bd90 0x55b6799384b0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:00.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.521+0000 7f83112c2b80 1 -- 172.21.15.39:0/923415345 >> 172.21.15.39:0/923415345 conn(0x55b679918be0 msgr2=0x55b679934e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:02:00.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.521+0000 7f82e67fc700 0 client.17637 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T19:02:00.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.521+0000 7f83112c2b80 1 -- 172.21.15.39:0/923415345 shutdown_connections 2023-07-21T19:02:00.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.521+0000 7f83112c2b80 1 -- 172.21.15.39:0/923415345 wait complete. 2023-07-21T19:02:00.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.521+0000 7f83112c2b80 20 client.17637 trim_cache size 0 max 16384 2023-07-21T19:02:00.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T19:02:00.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T19:02:00.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T19:02:00.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T19:02:00.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.522+0000 7f830513c700 1 -- 172.21.15.39:0/1769958076 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f82ec0a1410 msgr2=0x7f82ec0a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:02:00.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.522+0000 7f830513c700 1 --2- 172.21.15.39:0/1769958076 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f82ec0a1410 0x7f82ec0a38c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f830010d9a0 tx=0x7f82f00073a0 comp rx=0 tx=0).stop 2023-07-21T19:02:00.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.522+0000 7f830513c700 1 -- 172.21.15.39:0/1769958076 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83000fea20 msgr2=0x7f830010bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:02:00.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.522+0000 7f830513c700 1 --2- 172.21.15.39:0/1769958076 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83000fea20 0x7f830010bfa0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f82e8000c00 tx=0x7f82e8003c40 comp rx=0 tx=0).stop 2023-07-21T19:02:00.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.522+0000 7f830513c700 1 -- 172.21.15.39:0/1769958076 shutdown_connections 2023-07-21T19:02:00.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.522+0000 7f830513c700 1 --2- 172.21.15.39:0/1769958076 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f82ec0a1410 0x7f82ec0a38c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:00.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.522+0000 7f830513c700 1 --2- 172.21.15.39:0/1769958076 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8300104a20 0x7f830010ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:00.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.522+0000 7f830513c700 1 --2- 172.21.15.39:0/1769958076 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83000fea20 0x7f830010bfa0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:00.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.522+0000 7f830513c700 1 --2- 172.21.15.39:0/1769958076 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83000ff3d0 0x7f830010c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:00.538 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.522+0000 7f830513c700 1 -- 172.21.15.39:0/1769958076 >> 172.21.15.39:0/1769958076 conn(0x7f83000769b0 msgr2=0x7f83000fd140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:02:00.538 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.522+0000 7f830513c700 1 -- 172.21.15.39:0/1769958076 shutdown_connections 2023-07-21T19:02:00.538 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:00.522+0000 7f830513c700 1 -- 172.21.15.39:0/1769958076 wait complete. 2023-07-21T19:02:00.700 INFO:tasks.workunit.client.0.smithi039.stdout: 10 242251 30.16 MB/sec execute 313 sec latency 280.654 ms 2023-07-21T19:02:00.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:00 smithi193 ceph-mon[98204]: pgmap v642: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 6.0 MiB/s wr, 564 op/s 2023-07-21T19:02:00.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:00 smithi155 ceph-mon[97934]: pgmap v642: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 6.0 MiB/s wr, 564 op/s 2023-07-21T19:02:00.927 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:00 smithi039 ceph-mon[96489]: pgmap v642: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 6.0 MiB/s wr, 564 op/s 2023-07-21T19:02:01.700 INFO:tasks.workunit.client.0.smithi039.stdout: 10 242559 30.09 MB/sec execute 314 sec latency 148.343 ms 2023-07-21T19:02:02.700 INFO:tasks.workunit.client.0.smithi039.stdout: 10 242739 30.02 MB/sec execute 315 sec latency 176.731 ms 2023-07-21T19:02:02.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:02 smithi193 ceph-mon[98204]: pgmap v643: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.3 MiB/s wr, 601 op/s 2023-07-21T19:02:02.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:02 smithi155 ceph-mon[97934]: pgmap v643: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.3 MiB/s wr, 601 op/s 2023-07-21T19:02:02.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:02 smithi039 ceph-mon[96489]: pgmap v643: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.3 MiB/s wr, 601 op/s 2023-07-21T19:02:03.700 INFO:tasks.workunit.client.0.smithi039.stdout: 10 243067 29.98 MB/sec execute 316 sec latency 225.328 ms 2023-07-21T19:02:04.701 INFO:tasks.workunit.client.0.smithi039.stdout: 10 243241 29.91 MB/sec execute 317 sec latency 438.943 ms 2023-07-21T19:02:04.859 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:04 smithi193 ceph-mon[98204]: pgmap v644: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 9.4 MiB/s wr, 811 op/s 2023-07-21T19:02:04.859 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:04 smithi193 ceph-mon[98204]: osdmap e115: 12 total, 12 up, 12 in 2023-07-21T19:02:04.879 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:04 smithi155 ceph-mon[97934]: pgmap v644: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 9.4 MiB/s wr, 811 op/s 2023-07-21T19:02:04.879 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:04 smithi155 ceph-mon[97934]: osdmap e115: 12 total, 12 up, 12 in 2023-07-21T19:02:04.930 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:04 smithi039 ceph-mon[96489]: pgmap v644: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 9.4 MiB/s wr, 811 op/s 2023-07-21T19:02:04.931 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:04 smithi039 ceph-mon[96489]: osdmap e115: 12 total, 12 up, 12 in 2023-07-21T19:02:05.701 INFO:tasks.workunit.client.0.smithi039.stdout: 10 243667 29.90 MB/sec execute 318 sec latency 129.279 ms 2023-07-21T19:02:05.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:05 smithi193 ceph-mon[98204]: pgmap v646: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 8.1 MiB/s wr, 579 op/s 2023-07-21T19:02:05.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:05 smithi155 ceph-mon[97934]: pgmap v646: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 8.1 MiB/s wr, 579 op/s 2023-07-21T19:02:05.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:05 smithi039 ceph-mon[96489]: pgmap v646: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 8.1 MiB/s wr, 579 op/s 2023-07-21T19:02:06.701 INFO:tasks.workunit.client.0.smithi039.stdout: 10 244423 29.94 MB/sec execute 319 sec latency 68.204 ms 2023-07-21T19:02:07.701 INFO:tasks.workunit.client.0.smithi039.stdout: 10 245100 29.98 MB/sec execute 320 sec latency 62.501 ms 2023-07-21T19:02:08.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:08 smithi193 ceph-mon[98204]: pgmap v647: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 7.3 MiB/s wr, 680 op/s 2023-07-21T19:02:08.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:08 smithi155 ceph-mon[97934]: pgmap v647: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 7.3 MiB/s wr, 680 op/s 2023-07-21T19:02:08.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:08 smithi039 ceph-mon[96489]: pgmap v647: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 7.3 MiB/s wr, 680 op/s 2023-07-21T19:02:08.701 INFO:tasks.workunit.client.0.smithi039.stdout: 10 245586 29.97 MB/sec execute 321 sec latency 132.729 ms 2023-07-21T19:02:09.701 INFO:tasks.workunit.client.0.smithi039.stdout: 10 246033 29.94 MB/sec execute 322 sec latency 79.380 ms 2023-07-21T19:02:10.607 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:10 smithi193 ceph-mon[98204]: pgmap v648: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 7.3 MiB/s wr, 680 op/s 2023-07-21T19:02:10.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:10 smithi155 ceph-mon[97934]: pgmap v648: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 7.3 MiB/s wr, 680 op/s 2023-07-21T19:02:10.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:10 smithi039 ceph-mon[96489]: pgmap v648: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 7.3 MiB/s wr, 680 op/s 2023-07-21T19:02:10.701 INFO:tasks.workunit.client.0.smithi039.stdout: 10 246555 29.91 MB/sec execute 323 sec latency 80.022 ms 2023-07-21T19:02:11.702 INFO:tasks.workunit.client.0.smithi039.stdout: 10 247319 29.93 MB/sec execute 324 sec latency 77.364 ms 2023-07-21T19:02:12.607 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:12 smithi193 ceph-mon[98204]: pgmap v649: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 7.7 MiB/s wr, 678 op/s 2023-07-21T19:02:12.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:12 smithi193 ceph-mon[98204]: osdmap e116: 12 total, 12 up, 12 in 2023-07-21T19:02:12.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:12 smithi155 ceph-mon[97934]: pgmap v649: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 7.7 MiB/s wr, 678 op/s 2023-07-21T19:02:12.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:12 smithi155 ceph-mon[97934]: osdmap e116: 12 total, 12 up, 12 in 2023-07-21T19:02:12.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:12 smithi039 ceph-mon[96489]: pgmap v649: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 7.7 MiB/s wr, 678 op/s 2023-07-21T19:02:12.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:12 smithi039 ceph-mon[96489]: osdmap e116: 12 total, 12 up, 12 in 2023-07-21T19:02:12.702 INFO:tasks.workunit.client.0.smithi039.stdout: 10 248115 30.00 MB/sec execute 325 sec latency 89.272 ms 2023-07-21T19:02:13.703 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.691+0000 7f1357276700 1 -- 172.21.15.39:0/1560917809 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13500ff3e0 msgr2=0x7f135010b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:02:13.703 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.691+0000 7f1357276700 1 --2- 172.21.15.39:0/1560917809 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13500ff3e0 0x7f135010b860 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f1348009f90 tx=0x7f134800b330 comp rx=0 tx=0).stop 2023-07-21T19:02:13.703 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.691+0000 7f1357276700 1 -- 172.21.15.39:0/1560917809 shutdown_connections 2023-07-21T19:02:13.703 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.691+0000 7f1357276700 1 --2- 172.21.15.39:0/1560917809 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13500ff3e0 0x7f135010b860 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:13.703 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.691+0000 7f1357276700 1 --2- 172.21.15.39:0/1560917809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1350104a30 0x7f1350104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:13.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.691+0000 7f1357276700 1 --2- 172.21.15.39:0/1560917809 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13500fea30 0x7f13500feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:13.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.691+0000 7f1357276700 1 -- 172.21.15.39:0/1560917809 >> 172.21.15.39:0/1560917809 conn(0x7f1350074760 msgr2=0x7f1350074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:02:13.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.691+0000 7f1357276700 1 -- 172.21.15.39:0/1560917809 shutdown_connections 2023-07-21T19:02:13.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.691+0000 7f1357276700 1 -- 172.21.15.39:0/1560917809 wait complete. 2023-07-21T19:02:13.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.692+0000 7f1357276700 1 Processor -- start 2023-07-21T19:02:13.705 INFO:tasks.workunit.client.0.smithi039.stdout: 10 248659 30.02 MB/sec execute 326 sec latency 81.603 ms 2023-07-21T19:02:13.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.692+0000 7f1357276700 1 -- start start 2023-07-21T19:02:13.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.692+0000 7f1357276700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13500fea30 0x7f135019c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:13.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.692+0000 7f1357276700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13500ff3e0 0x7f135019c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:13.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.692+0000 7f1357276700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1350104a30 0x7f13501a0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:13.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.692+0000 7f1357276700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13501a14e0 con 0x7f13500ff3e0 2023-07-21T19:02:13.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.692+0000 7f1357276700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13501a1650 con 0x7f1350104a30 2023-07-21T19:02:13.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.692+0000 7f1357276700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13501a1810 con 0x7f13500fea30 2023-07-21T19:02:13.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.692+0000 7f1355012700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13500fea30 0x7f135019c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:13.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.692+0000 7f1355012700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13500fea30 0x7f135019c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:52884/0 (socket says 172.21.15.39:52884) 2023-07-21T19:02:13.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.692+0000 7f1355012700 1 -- 172.21.15.39:0/145913789 learned_addr learned my addr 172.21.15.39:0/145913789 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:02:13.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.693+0000 7f1355813700 1 --2- 172.21.15.39:0/145913789 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1350104a30 0x7f13501a0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:13.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.693+0000 7f1355813700 1 -- 172.21.15.39:0/145913789 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13500fea30 msgr2=0x7f135019c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:02:13.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.693+0000 7f1355813700 1 --2- 172.21.15.39:0/145913789 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13500fea30 0x7f135019c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:13.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.693+0000 7f1355813700 1 -- 172.21.15.39:0/145913789 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13500ff3e0 msgr2=0x7f135019c830 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-21T19:02:13.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.693+0000 7f1355813700 1 --2- 172.21.15.39:0/145913789 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13500ff3e0 0x7f135019c830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:13.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.693+0000 7f1355813700 1 -- 172.21.15.39:0/145913789 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1348009c40 con 0x7f1350104a30 2023-07-21T19:02:13.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.693+0000 7f1355813700 1 --2- 172.21.15.39:0/145913789 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1350104a30 0x7f13501a0ea0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1348009f90 tx=0x7f1348009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:02:13.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.694+0000 7f13467fc700 1 -- 172.21.15.39:0/145913789 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f134800e040 con 0x7f1350104a30 2023-07-21T19:02:13.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.694+0000 7f13467fc700 1 -- 172.21.15.39:0/145913789 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f134803f480 con 0x7f1350104a30 2023-07-21T19:02:13.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.694+0000 7f13467fc700 1 -- 172.21.15.39:0/145913789 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f134803d6d0 con 0x7f1350104a30 2023-07-21T19:02:13.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.694+0000 7f1357276700 1 -- 172.21.15.39:0/145913789 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13501a1a90 con 0x7f1350104a30 2023-07-21T19:02:13.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.694+0000 7f1357276700 1 -- 172.21.15.39:0/145913789 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f135006a110 con 0x7f1350104a30 2023-07-21T19:02:13.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.699+0000 7f13633fcb80 1 Processor -- start 2023-07-21T19:02:13.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.699+0000 7f13633fcb80 1 -- start start 2023-07-21T19:02:13.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.699+0000 7f13633fcb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3d79616b0 0x55d3d7b0ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:13.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.699+0000 7f13633fcb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d3d7b0b2d0 0x55d3d7b05040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:13.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.699+0000 7f13633fcb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3d7b05580 0x55d3d7b059d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:13.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.699+0000 7f13633fcb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3d7a0ac50 con 0x55d3d7b05580 2023-07-21T19:02:13.710 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.699+0000 7f13633fcb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3d7a5d9e0 con 0x55d3d79616b0 2023-07-21T19:02:13.710 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.699+0000 7f13633fcb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3d7a56e70 con 0x55d3d7b0b2d0 2023-07-21T19:02:13.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.700+0000 7f1354811700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3d7b05580 0x55d3d7b059d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:13.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.700+0000 7f1355012700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d3d7b0b2d0 0x55d3d7b05040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:13.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.700+0000 7f1354811700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3d7b05580 0x55d3d7b059d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34338/0 (socket says 172.21.15.39:34338) 2023-07-21T19:02:13.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.700+0000 7f1354811700 1 -- 172.21.15.39:0/4013698537 learned_addr learned my addr 172.21.15.39:0/4013698537 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:02:13.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.700+0000 7f1354811700 1 --2- 172.21.15.39:0/4013698537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3d79616b0 0x55d3d7b0ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:13.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.700+0000 7f13467fc700 1 -- 172.21.15.39:0/145913789 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f134803f8f0 con 0x7f1350104a30 2023-07-21T19:02:13.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.701+0000 7f13467fc700 1 --2- 172.21.15.39:0/145913789 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f133c0a14d0 0x7f133c0a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:13.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.702+0000 7f13467fc700 1 -- 172.21.15.39:0/145913789 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f13480efb90 con 0x7f1350104a30 2023-07-21T19:02:13.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.702+0000 7f1355012700 1 -- 172.21.15.39:0/4013698537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3d79616b0 msgr2=0x55d3d7b0ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:02:13.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.702+0000 7f1355012700 1 --2- 172.21.15.39:0/4013698537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3d79616b0 0x55d3d7b0ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:13.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.702+0000 7f1355012700 1 -- 172.21.15.39:0/4013698537 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3d7b05580 msgr2=0x55d3d7b059d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:02:13.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.702+0000 7f1355012700 1 --2- 172.21.15.39:0/4013698537 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3d7b05580 0x55d3d7b059d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:13.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.702+0000 7f1355012700 1 -- 172.21.15.39:0/4013698537 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3d7884d80 con 0x55d3d7b0b2d0 2023-07-21T19:02:13.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.702+0000 7f1355813700 1 --2- 172.21.15.39:0/145913789 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f133c0a14d0 0x7f133c0a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:13.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.702+0000 7f1355012700 1 --2- 172.21.15.39:0/4013698537 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d3d7b0b2d0 0x55d3d7b05040 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f134c009d10 tx=0x7f134c011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af8b913729a9e027 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:02:13.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.702+0000 7f1355813700 1 --2- 172.21.15.39:0/145913789 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f133c0a14d0 0x7f133c0a3980 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f13480b9190 tx=0x7f134800b330 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:02:13.714 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.705+0000 7f133b7fe700 1 -- 172.21.15.39:0/4013698537 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f134c004220 con 0x55d3d7b0b2d0 2023-07-21T19:02:13.714 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.705+0000 7f133b7fe700 1 -- 172.21.15.39:0/4013698537 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f134c004cf0 con 0x55d3d7b0b2d0 2023-07-21T19:02:13.714 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.705+0000 7f133b7fe700 1 -- 172.21.15.39:0/4013698537 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f134c005530 con 0x55d3d7b0b2d0 2023-07-21T19:02:13.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.706+0000 7f13633fcb80 1 -- 172.21.15.39:0/4013698537 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d3d7b0b2d0 msgr2=0x55d3d7b05040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:02:13.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.706+0000 7f13633fcb80 1 --2- 172.21.15.39:0/4013698537 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d3d7b0b2d0 0x55d3d7b05040 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f134c009d10 tx=0x7f134c011720 comp rx=0 tx=0).stop 2023-07-21T19:02:13.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.706+0000 7f13633fcb80 1 -- 172.21.15.39:0/4013698537 shutdown_connections 2023-07-21T19:02:13.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.706+0000 7f13633fcb80 1 --2- 172.21.15.39:0/4013698537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3d79616b0 0x55d3d7b0ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:13.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.706+0000 7f13633fcb80 1 --2- 172.21.15.39:0/4013698537 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3d7b05580 0x55d3d7b059d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:13.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.706+0000 7f13633fcb80 1 --2- 172.21.15.39:0/4013698537 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d3d7b0b2d0 0x55d3d7b05040 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:13.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.706+0000 7f13633fcb80 1 -- 172.21.15.39:0/4013698537 >> 172.21.15.39:0/4013698537 conn(0x55d3d7afabe0 msgr2=0x55d3d7afff10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:02:13.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.706+0000 7f13633fcb80 1 -- 172.21.15.39:0/4013698537 shutdown_connections 2023-07-21T19:02:13.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.706+0000 7f13633fcb80 1 -- 172.21.15.39:0/4013698537 wait complete. 2023-07-21T19:02:13.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.706+0000 7f13633fcb80 1 -- start start 2023-07-21T19:02:13.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.706+0000 7f13633fcb80 1 Processor -- start 2023-07-21T19:02:13.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.707+0000 7f13633fcb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3d79616b0 0x55d3d7b11600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:13.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.707+0000 7f13633fcb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3d7b05580 0x55d3d7b0e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:13.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.707+0000 7f13633fcb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d3d7b0b2d0 0x55d3d7b0f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:13.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.707+0000 7f13633fcb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3d79f6a60 con 0x55d3d79616b0 2023-07-21T19:02:13.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.707+0000 7f13633fcb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3d79f6760 con 0x55d3d7b05580 2023-07-21T19:02:13.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.707+0000 7f13633fcb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3d78da8c0 con 0x55d3d7b0b2d0 2023-07-21T19:02:13.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.707+0000 7f13633fcb80 10 client.0 _ll_register_callbacks cb 0x55d3d79f4700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:02:13.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.707+0000 7f1354811700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d3d7b0b2d0 0x55d3d7b0f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:13.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.707+0000 7f1354811700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d3d7b0b2d0 0x55d3d7b0f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:52898/0 (socket says 172.21.15.39:52898) 2023-07-21T19:02:13.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.707+0000 7f1354811700 1 -- 172.21.15.39:0/2082089375 learned_addr learned my addr 172.21.15.39:0/2082089375 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:02:13.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.708+0000 7f1354811700 1 -- 172.21.15.39:0/2082089375 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3d7b05580 msgr2=0x55d3d7b0e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:02:13.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.708+0000 7f1354811700 1 --2- 172.21.15.39:0/2082089375 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3d7b05580 0x55d3d7b0e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:13.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.708+0000 7f1354811700 1 -- 172.21.15.39:0/2082089375 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3d79616b0 msgr2=0x55d3d7b11600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:02:13.720 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.708+0000 7f1354811700 1 --2- 172.21.15.39:0/2082089375 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3d79616b0 0x55d3d7b11600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:13.720 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.708+0000 7f1354811700 1 -- 172.21.15.39:0/2082089375 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f134c009940 con 0x55d3d7b0b2d0 2023-07-21T19:02:13.720 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.708+0000 7f1354811700 1 --2- 172.21.15.39:0/2082089375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d3d7b0b2d0 0x55d3d7b0f030 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f13400096b0 tx=0x7f13400096e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=181a760b820e5c33 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:02:13.720 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.708+0000 7f1338ff9700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2023-07-21T19:02:13.722 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.708+0000 7f1338ff9700 1 -- 172.21.15.39:0/2082089375 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1340010d20 con 0x55d3d7b0b2d0 2023-07-21T19:02:13.722 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.708+0000 7f1338ff9700 1 -- 172.21.15.39:0/2082089375 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f134001d430 con 0x55d3d7b0b2d0 2023-07-21T19:02:13.722 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.708+0000 7f13633fcb80 1 -- 172.21.15.39:0/2082089375 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d3d79b0490 con 0x55d3d7b0b2d0 2023-07-21T19:02:13.722 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.709+0000 7f1338ff9700 1 -- 172.21.15.39:0/2082089375 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=101) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1340010e80 con 0x55d3d7b0b2d0 2023-07-21T19:02:13.722 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.709+0000 7f13633fcb80 10 client.27148 fetch_fsmap learned FSMap version 101 2023-07-21T19:02:13.722 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.709+0000 7f13633fcb80 1 -- 172.21.15.39:0/2082089375 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=101,monmap=4+}) v3 -- 0x55d3d786dc00 con 0x55d3d7b0b2d0 2023-07-21T19:02:13.722 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.709+0000 7f1338ff9700 1 -- 172.21.15.39:0/2082089375 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f134002a020 con 0x55d3d7b0b2d0 2023-07-21T19:02:13.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.709+0000 7f1338ff9700 1 -- 172.21.15.39:0/2082089375 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 101) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f134000e3c0 con 0x55d3d7b0b2d0 2023-07-21T19:02:13.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.709+0000 7f13633fcb80 10 client.27148 fetch_fsmap finished waiting for FSMap version 101 2023-07-21T19:02:13.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.709+0000 7f13633fcb80 10 client.27148 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:02:13.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.709+0000 7f13633fcb80 20 client.27148 populate_metadata read hostname 'smithi039' 2023-07-21T19:02:13.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.709+0000 7f13633fcb80 1 --2- 172.21.15.39:0/2082089375 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d3d7b2fc20 0x55d3d7b32030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:13.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.709+0000 7f13633fcb80 4 client.27148 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:02:13.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.709+0000 7f13633fcb80 1 -- 172.21.15.39:0/2082089375 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d3d7a4fe50 con 0x55d3d7b2fc20 2023-07-21T19:02:13.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.709+0000 7f1355012700 1 --2- 172.21.15.39:0/2082089375 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d3d7b2fc20 0x55d3d7b32030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:13.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.710+0000 7f1338ff9700 10 client.27148 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T19:02:13.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.710+0000 7f1355012700 1 --2- 172.21.15.39:0/2082089375 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d3d7b2fc20 0x55d3d7b32030 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48b35284c710f8f8 server_cookie=7c417b766d1d1b74 in_seq=0 out_seq=0 2023-07-21T19:02:13.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.711+0000 7f1338ff9700 1 -- 172.21.15.39:0/2082089375 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d3d7a4fe50 con 0x55d3d7b2fc20 2023-07-21T19:02:13.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.711+0000 7f1338ff9700 10 client.27148 handle_command_reply: tid=0 2023-07-21T19:02:13.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.711+0000 7f13633fcb80 1 client.27148 shutdown 2023-07-21T19:02:13.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.711+0000 7f13633fcb80 1 -- 172.21.15.39:0/2082089375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d3d7b0b2d0 msgr2=0x55d3d7b0f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:02:13.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.711+0000 7f13633fcb80 1 --2- 172.21.15.39:0/2082089375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d3d7b0b2d0 0x55d3d7b0f030 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f13400096b0 tx=0x7f13400096e0 comp rx=0 tx=0).stop 2023-07-21T19:02:13.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.711+0000 7f13633fcb80 1 -- 172.21.15.39:0/2082089375 shutdown_connections 2023-07-21T19:02:13.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.711+0000 7f13633fcb80 1 --2- 172.21.15.39:0/2082089375 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d3d7b2fc20 0x55d3d7b32030 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:13.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.711+0000 7f13633fcb80 1 --2- 172.21.15.39:0/2082089375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d3d7b0b2d0 0x55d3d7b0f030 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:13.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.711+0000 7f13633fcb80 1 --2- 172.21.15.39:0/2082089375 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3d79616b0 0x55d3d7b11600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:13.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.711+0000 7f13633fcb80 1 --2- 172.21.15.39:0/2082089375 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3d7b05580 0x55d3d7b0e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:13.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.711+0000 7f13633fcb80 1 -- 172.21.15.39:0/2082089375 >> 172.21.15.39:0/2082089375 conn(0x55d3d7afabe0 msgr2=0x55d3d7aff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:02:13.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.711+0000 7f1338ff9700 0 client.27148 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T19:02:13.726 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.711+0000 7f13633fcb80 1 -- 172.21.15.39:0/2082089375 shutdown_connections 2023-07-21T19:02:13.726 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.711+0000 7f13633fcb80 1 -- 172.21.15.39:0/2082089375 wait complete. 2023-07-21T19:02:13.726 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.712+0000 7f13633fcb80 20 client.27148 trim_cache size 0 max 16384 2023-07-21T19:02:13.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.735+0000 7f13633fcb80 1 Processor -- start 2023-07-21T19:02:13.745 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.735+0000 7f13633fcb80 1 -- start start 2023-07-21T19:02:13.746 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.735+0000 7f13633fcb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3d79616b0 0x55d3d7b26c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:13.746 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.735+0000 7f13633fcb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3d7b05580 0x55d3d7b0e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:13.746 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.735+0000 7f13633fcb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d3d7b0b2d0 0x55d3d7b111a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:13.746 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.735+0000 7f13633fcb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3d7b0b6a0 con 0x55d3d79616b0 2023-07-21T19:02:13.746 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.735+0000 7f13633fcb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3d7af7bc0 con 0x55d3d7b05580 2023-07-21T19:02:13.746 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.735+0000 7f13633fcb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3d7b01970 con 0x55d3d7b0b2d0 2023-07-21T19:02:13.746 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.735+0000 7f1354811700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3d79616b0 0x55d3d7b26c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:13.747 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.735+0000 7f1354811700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3d79616b0 0x55d3d7b26c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34352/0 (socket says 172.21.15.39:34352) 2023-07-21T19:02:13.747 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.735+0000 7f1355012700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3d7b05580 0x55d3d7b0e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:13.747 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.735+0000 7f1354811700 1 -- 172.21.15.39:0/3798714737 learned_addr learned my addr 172.21.15.39:0/3798714737 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:02:13.747 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.736+0000 7f1354811700 1 --2- 172.21.15.39:0/3798714737 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d3d7b0b2d0 0x55d3d7b111a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:13.747 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.736+0000 7f1354811700 1 -- 172.21.15.39:0/3798714737 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d3d7b0b2d0 msgr2=0x55d3d7b111a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:02:13.747 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.736+0000 7f1354811700 1 --2- 172.21.15.39:0/3798714737 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d3d7b0b2d0 0x55d3d7b111a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:13.747 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.736+0000 7f1354811700 1 -- 172.21.15.39:0/3798714737 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3d7b05580 msgr2=0x55d3d7b0e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:02:13.747 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.736+0000 7f1354811700 1 --2- 172.21.15.39:0/3798714737 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3d7b05580 0x55d3d7b0e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:13.748 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.736+0000 7f1354811700 1 -- 172.21.15.39:0/3798714737 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3d7b36c60 con 0x55d3d79616b0 2023-07-21T19:02:13.748 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.736+0000 7f1354811700 1 --2- 172.21.15.39:0/3798714737 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3d79616b0 0x55d3d7b26c70 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f1340006010 tx=0x7f1340005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8980746c7ab78573 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:02:13.748 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.737+0000 7f133bfff700 1 -- 172.21.15.39:0/3798714737 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f134000ea40 con 0x55d3d79616b0 2023-07-21T19:02:13.748 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.737+0000 7f133bfff700 1 -- 172.21.15.39:0/3798714737 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f134002a070 con 0x55d3d79616b0 2023-07-21T19:02:13.748 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.737+0000 7f133bfff700 1 -- 172.21.15.39:0/3798714737 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1340004450 con 0x55d3d79616b0 2023-07-21T19:02:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.741+0000 7f13633fcb80 1 -- 172.21.15.39:0/3798714737 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3d79616b0 msgr2=0x55d3d7b26c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:02:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.741+0000 7f13633fcb80 1 --2- 172.21.15.39:0/3798714737 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3d79616b0 0x55d3d7b26c70 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f1340006010 tx=0x7f1340005fa0 comp rx=0 tx=0).stop 2023-07-21T19:02:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.741+0000 7f13633fcb80 1 -- 172.21.15.39:0/3798714737 shutdown_connections 2023-07-21T19:02:13.752 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.741+0000 7f13633fcb80 1 --2- 172.21.15.39:0/3798714737 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d3d7b0b2d0 0x55d3d7b111a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:13.752 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.741+0000 7f13633fcb80 1 --2- 172.21.15.39:0/3798714737 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3d79616b0 0x55d3d7b26c70 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:13.752 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.741+0000 7f13633fcb80 1 --2- 172.21.15.39:0/3798714737 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3d7b05580 0x55d3d7b0e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:13.752 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.741+0000 7f13633fcb80 1 -- 172.21.15.39:0/3798714737 >> 172.21.15.39:0/3798714737 conn(0x55d3d7afabe0 msgr2=0x55d3d7b08b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:02:13.752 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.742+0000 7f13633fcb80 1 -- 172.21.15.39:0/3798714737 shutdown_connections 2023-07-21T19:02:13.752 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.742+0000 7f13633fcb80 1 -- 172.21.15.39:0/3798714737 wait complete. 2023-07-21T19:02:13.753 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.742+0000 7f13633fcb80 1 -- start start 2023-07-21T19:02:13.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.742+0000 7f13633fcb80 1 Processor -- start 2023-07-21T19:02:13.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.743+0000 7f13633fcb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3d79616b0 0x55d3d7b079c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:13.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.743+0000 7f13633fcb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3d7b05580 0x55d3d7b07f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:13.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.743+0000 7f13633fcb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d3d7b0b2d0 0x55d3d7b06860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:13.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.743+0000 7f13633fcb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3d7b08610 con 0x55d3d79616b0 2023-07-21T19:02:13.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.743+0000 7f13633fcb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3d7b06da0 con 0x55d3d7b05580 2023-07-21T19:02:13.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.743+0000 7f13633fcb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3d7b06f60 con 0x55d3d7b0b2d0 2023-07-21T19:02:13.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.743+0000 7f13633fcb80 10 client.0 _ll_register_callbacks cb 0x55d3d79f4700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:02:13.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.743+0000 7f1354811700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3d79616b0 0x55d3d7b079c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:13.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.743+0000 7f1354811700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3d79616b0 0x55d3d7b079c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34358/0 (socket says 172.21.15.39:34358) 2023-07-21T19:02:13.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.743+0000 7f1354811700 1 -- 172.21.15.39:0/1622390877 learned_addr learned my addr 172.21.15.39:0/1622390877 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:02:13.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.743+0000 7f1355012700 1 --2- 172.21.15.39:0/1622390877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3d7b05580 0x55d3d7b07f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:13.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.743+0000 7f1354811700 1 --2- 172.21.15.39:0/1622390877 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d3d7b0b2d0 0x55d3d7b06860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:13.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.743+0000 7f1354811700 1 -- 172.21.15.39:0/1622390877 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d3d7b0b2d0 msgr2=0x55d3d7b06860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:02:13.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.743+0000 7f1354811700 1 --2- 172.21.15.39:0/1622390877 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d3d7b0b2d0 0x55d3d7b06860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:13.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.743+0000 7f1354811700 1 -- 172.21.15.39:0/1622390877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3d7b05580 msgr2=0x55d3d7b07f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:02:13.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.743+0000 7f1354811700 1 --2- 172.21.15.39:0/1622390877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3d7b05580 0x55d3d7b07f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:13.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.743+0000 7f1354811700 1 -- 172.21.15.39:0/1622390877 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d3d786dc00 con 0x55d3d79616b0 2023-07-21T19:02:13.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.744+0000 7f1354811700 1 --2- 172.21.15.39:0/1622390877 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3d79616b0 0x55d3d7b079c0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f134000e6d0 tx=0x7f13400092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc31512622ce0ab server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:02:13.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.744+0000 7f1338ff9700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T19:02:13.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.744+0000 7f1338ff9700 1 -- 172.21.15.39:0/1622390877 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13400042d0 con 0x55d3d79616b0 2023-07-21T19:02:13.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.744+0000 7f13633fcb80 1 -- 172.21.15.39:0/1622390877 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f134c004cf0 con 0x55d3d79616b0 2023-07-21T19:02:13.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.744+0000 7f1338ff9700 1 -- 172.21.15.39:0/1622390877 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f1340000e40 con 0x55d3d79616b0 2023-07-21T19:02:13.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.744+0000 7f1338ff9700 1 -- 172.21.15.39:0/1622390877 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=101) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1340004430 con 0x55d3d79616b0 2023-07-21T19:02:13.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.744+0000 7f13633fcb80 10 client.17649 fetch_fsmap learned FSMap version 101 2023-07-21T19:02:13.758 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.744+0000 7f13633fcb80 1 -- 172.21.15.39:0/1622390877 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=101,monmap=4+}) v3 -- 0x55d3d78ab1f0 con 0x55d3d79616b0 2023-07-21T19:02:13.758 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.745+0000 7f1338ff9700 1 -- 172.21.15.39:0/1622390877 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f134c004cf0 con 0x55d3d79616b0 2023-07-21T19:02:13.758 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.745+0000 7f1338ff9700 1 -- 172.21.15.39:0/1622390877 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 101) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f134003d070 con 0x55d3d79616b0 2023-07-21T19:02:13.758 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.745+0000 7f13633fcb80 10 client.17649 fetch_fsmap finished waiting for FSMap version 101 2023-07-21T19:02:13.758 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.745+0000 7f13633fcb80 10 client.17649 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:02:13.758 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.745+0000 7f13633fcb80 20 client.17649 populate_metadata read hostname 'smithi039' 2023-07-21T19:02:13.758 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.745+0000 7f13633fcb80 1 --2- 172.21.15.39:0/1622390877 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d3d7b84300 0x55d3d7b86710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:13.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.745+0000 7f13633fcb80 4 client.17649 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:02:13.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.745+0000 7f13633fcb80 1 -- 172.21.15.39:0/1622390877 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d3d7a11ff0 con 0x55d3d7b84300 2023-07-21T19:02:13.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.745+0000 7f1355012700 1 --2- 172.21.15.39:0/1622390877 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d3d7b84300 0x55d3d7b86710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:13.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.745+0000 7f1338ff9700 10 client.17649 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T19:02:13.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.745+0000 7f1355012700 1 --2- 172.21.15.39:0/1622390877 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d3d7b84300 0x55d3d7b86710 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=556956ee97f08aa6 server_cookie=d3d95f8ef8bf3621 in_seq=0 out_seq=0 2023-07-21T19:02:14.001 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.990+0000 7f1338ff9700 1 -- 172.21.15.39:0/1622390877 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d3d7a11ff0 con 0x55d3d7b84300 2023-07-21T19:02:14.001 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.990+0000 7f1338ff9700 10 client.17649 handle_command_reply: tid=0 2023-07-21T19:02:14.001 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.990+0000 7f13633fcb80 1 client.17649 shutdown 2023-07-21T19:02:14.001 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.991+0000 7f13633fcb80 1 -- 172.21.15.39:0/1622390877 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3d79616b0 msgr2=0x55d3d7b079c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:02:14.001 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.991+0000 7f13633fcb80 1 --2- 172.21.15.39:0/1622390877 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3d79616b0 0x55d3d7b079c0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f134000e6d0 tx=0x7f13400092d0 comp rx=0 tx=0).stop 2023-07-21T19:02:14.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.991+0000 7f13633fcb80 1 -- 172.21.15.39:0/1622390877 shutdown_connections 2023-07-21T19:02:14.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.991+0000 7f13633fcb80 1 --2- 172.21.15.39:0/1622390877 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d3d7b84300 0x55d3d7b86710 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:14.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.991+0000 7f13633fcb80 1 --2- 172.21.15.39:0/1622390877 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d3d7b0b2d0 0x55d3d7b06860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:14.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.991+0000 7f13633fcb80 1 --2- 172.21.15.39:0/1622390877 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d3d79616b0 0x55d3d7b079c0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:14.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.991+0000 7f13633fcb80 1 --2- 172.21.15.39:0/1622390877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3d7b05580 0x55d3d7b07f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:14.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.991+0000 7f13633fcb80 1 -- 172.21.15.39:0/1622390877 >> 172.21.15.39:0/1622390877 conn(0x55d3d7afabe0 msgr2=0x55d3d7b09ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:02:14.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.991+0000 7f1338ff9700 0 client.17649 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T19:02:14.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.991+0000 7f13633fcb80 1 -- 172.21.15.39:0/1622390877 shutdown_connections 2023-07-21T19:02:14.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.991+0000 7f13633fcb80 1 -- 172.21.15.39:0/1622390877 wait complete. 2023-07-21T19:02:14.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.991+0000 7f13633fcb80 20 client.17649 trim_cache size 0 max 16384 2023-07-21T19:02:14.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T19:02:14.004 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T19:02:14.004 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T19:02:14.004 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T19:02:14.008 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.996+0000 7f1357276700 1 -- 172.21.15.39:0/145913789 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f133c0a14d0 msgr2=0x7f133c0a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:02:14.008 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.996+0000 7f1357276700 1 --2- 172.21.15.39:0/145913789 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f133c0a14d0 0x7f133c0a3980 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f13480b9190 tx=0x7f134800b330 comp rx=0 tx=0).stop 2023-07-21T19:02:14.008 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.996+0000 7f1357276700 1 -- 172.21.15.39:0/145913789 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1350104a30 msgr2=0x7f13501a0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:02:14.009 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.996+0000 7f1357276700 1 --2- 172.21.15.39:0/145913789 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1350104a30 0x7f13501a0ea0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1348009f90 tx=0x7f1348009500 comp rx=0 tx=0).stop 2023-07-21T19:02:14.009 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.996+0000 7f1357276700 1 -- 172.21.15.39:0/145913789 shutdown_connections 2023-07-21T19:02:14.009 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.996+0000 7f1357276700 1 --2- 172.21.15.39:0/145913789 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f133c0a14d0 0x7f133c0a3980 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:14.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.996+0000 7f1357276700 1 --2- 172.21.15.39:0/145913789 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1350104a30 0x7f13501a0ea0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:14.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.996+0000 7f1357276700 1 --2- 172.21.15.39:0/145913789 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13500fea30 0x7f135019c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:14.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.996+0000 7f1357276700 1 --2- 172.21.15.39:0/145913789 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13500ff3e0 0x7f135019c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:14.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.996+0000 7f1357276700 1 -- 172.21.15.39:0/145913789 >> 172.21.15.39:0/145913789 conn(0x7f1350074760 msgr2=0x7f13501058a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:02:14.011 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.997+0000 7f1357276700 1 -- 172.21.15.39:0/145913789 shutdown_connections 2023-07-21T19:02:14.011 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:13.998+0000 7f1357276700 1 -- 172.21.15.39:0/145913789 wait complete. 2023-07-21T19:02:14.607 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:14 smithi193 ceph-mon[98204]: pgmap v651: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 8.1 MiB/s wr, 657 op/s 2023-07-21T19:02:14.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:14 smithi155 ceph-mon[97934]: pgmap v651: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 8.1 MiB/s wr, 657 op/s 2023-07-21T19:02:14.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:14 smithi039 ceph-mon[96489]: pgmap v651: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 8.1 MiB/s wr, 657 op/s 2023-07-21T19:02:14.702 INFO:tasks.workunit.client.0.smithi039.stdout: 10 249048 29.99 MB/sec execute 327 sec latency 220.599 ms 2023-07-21T19:02:15.702 INFO:tasks.workunit.client.0.smithi039.stdout: 10 249505 29.98 MB/sec execute 328 sec latency 126.503 ms 2023-07-21T19:02:16.607 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:16 smithi193 ceph-mon[98204]: pgmap v652: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 7.4 MiB/s wr, 602 op/s 2023-07-21T19:02:16.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:16 smithi155 ceph-mon[97934]: pgmap v652: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 7.4 MiB/s wr, 602 op/s 2023-07-21T19:02:16.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:16 smithi039 ceph-mon[96489]: pgmap v652: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 7.4 MiB/s wr, 602 op/s 2023-07-21T19:02:16.702 INFO:tasks.workunit.client.0.smithi039.stdout: 10 249918 29.93 MB/sec execute 329 sec latency 675.256 ms 2023-07-21T19:02:17.703 INFO:tasks.workunit.client.0.smithi039.stdout: 10 250581 29.93 MB/sec execute 330 sec latency 1675.293 ms 2023-07-21T19:02:18.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:18 smithi155 ceph-mon[97934]: pgmap v653: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 9.2 MiB/s wr, 565 op/s 2023-07-21T19:02:18.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:18 smithi039 ceph-mon[96489]: pgmap v653: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 9.2 MiB/s wr, 565 op/s 2023-07-21T19:02:18.703 INFO:tasks.workunit.client.0.smithi039.stdout: 10 251345 30.01 MB/sec execute 331 sec latency 2675.327 ms 2023-07-21T19:02:18.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:18 smithi193 ceph-mon[98204]: pgmap v653: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 9.2 MiB/s wr, 565 op/s 2023-07-21T19:02:19.702 INFO:tasks.workunit.client.0.smithi039.stdout: 10 251787 29.98 MB/sec execute 332 sec latency 3675.357 ms 2023-07-21T19:02:20.636 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:20 smithi155 ceph-mon[97934]: pgmap v654: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 9.2 MiB/s wr, 565 op/s 2023-07-21T19:02:20.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:20 smithi039 ceph-mon[96489]: pgmap v654: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 9.2 MiB/s wr, 565 op/s 2023-07-21T19:02:20.704 INFO:tasks.workunit.client.0.smithi039.stdout: 10 251823 29.90 MB/sec execute 333 sec latency 4677.047 ms 2023-07-21T19:02:20.859 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:20 smithi193 ceph-mon[98204]: pgmap v654: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 9.2 MiB/s wr, 565 op/s 2023-07-21T19:02:21.704 INFO:tasks.workunit.client.0.smithi039.stdout: 10 251906 29.81 MB/sec execute 334 sec latency 5677.091 ms 2023-07-21T19:02:22.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:22 smithi039 ceph-mon[96489]: pgmap v655: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 9.5 MiB/s wr, 553 op/s 2023-07-21T19:02:22.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:22 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:02:22.704 INFO:tasks.workunit.client.0.smithi039.stdout: 10 252151 29.76 MB/sec execute 335 sec latency 6677.129 ms 2023-07-21T19:02:22.868 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:22 smithi193 ceph-mon[98204]: pgmap v655: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 9.5 MiB/s wr, 553 op/s 2023-07-21T19:02:22.869 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:22 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:02:22.879 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:22 smithi155 ceph-mon[97934]: pgmap v655: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 9.5 MiB/s wr, 553 op/s 2023-07-21T19:02:22.879 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:22 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:02:23.705 INFO:tasks.workunit.client.0.smithi039.stdout: 10 252436 29.71 MB/sec execute 336 sec latency 7070.120 ms 2023-07-21T19:02:24.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:24 smithi039 ceph-mon[96489]: pgmap v656: 97 pgs: 97 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 20 MiB/s wr, 665 op/s 2023-07-21T19:02:24.704 INFO:tasks.workunit.client.0.smithi039.stdout: 10 253030 29.72 MB/sec execute 337 sec latency 89.547 ms 2023-07-21T19:02:24.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:24 smithi193 ceph-mon[98204]: pgmap v656: 97 pgs: 97 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 20 MiB/s wr, 665 op/s 2023-07-21T19:02:24.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:24 smithi155 ceph-mon[97934]: pgmap v656: 97 pgs: 97 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 20 MiB/s wr, 665 op/s 2023-07-21T19:02:25.704 INFO:tasks.workunit.client.0.smithi039.stdout: 10 253680 29.71 MB/sec execute 338 sec latency 69.718 ms 2023-07-21T19:02:26.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:26 smithi193 ceph-mon[98204]: pgmap v657: 97 pgs: 97 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 19 MiB/s wr, 628 op/s 2023-07-21T19:02:26.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:26 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T19:02:26.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:26 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T19:02:26.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:26 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T19:02:26.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:26 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T19:02:26.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:26 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:26.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:26 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:26.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:26 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:26.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:26 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:26.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:26 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:26.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:26 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:26.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:26 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:26.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:26 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:26.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:26 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:26.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:26 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:26.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:26 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:26.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:26 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:26.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:26 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:26.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:26 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:26.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:26 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:26.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:26 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:26.630 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:26 smithi155 ceph-mon[97934]: pgmap v657: 97 pgs: 97 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 19 MiB/s wr, 628 op/s 2023-07-21T19:02:26.631 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:26 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T19:02:26.631 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:26 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T19:02:26.631 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:26 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T19:02:26.631 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:26 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T19:02:26.631 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:26 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:26.631 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:26 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:26.632 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:26 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:26.632 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:26 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:26.632 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:26 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:26.632 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:26 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:26.632 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:26 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:26.632 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:26 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:26.632 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:26 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:26.633 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:26 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:26.633 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:26 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:26.633 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:26 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:26.633 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:26 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:26.633 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:26 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:26.633 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:26 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:26.633 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:26 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:26.704 INFO:tasks.workunit.client.0.smithi039.stdout: 10 254248 29.74 MB/sec execute 339 sec latency 80.402 ms 2023-07-21T19:02:26.812 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:26 smithi039 ceph-mon[96489]: pgmap v657: 97 pgs: 97 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 19 MiB/s wr, 628 op/s 2023-07-21T19:02:26.813 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:26 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T19:02:26.813 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:26 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T19:02:26.813 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:26 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T19:02:26.813 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:26 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T19:02:26.813 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:26 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:26.814 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:26 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:26.814 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:26 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:26.814 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:26 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:26.814 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:26 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:26.814 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:26 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:26.814 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:26 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:26.814 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:26 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:26.815 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:26 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:26.815 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:26 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:26.815 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:26 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:26.815 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:26 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:26.815 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:26 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:26.815 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:26 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:26.815 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:26 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:26.816 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:26 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:27.611 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:27 smithi193 ceph-mon[98204]: Adjusting osd_memory_target on smithi193 to 196.2M 2023-07-21T19:02:27.611 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:27 smithi193 ceph-mon[98204]: Unable to set osd_memory_target on smithi193 to 205773926: error parsing value: Value '205773926' is below minimum 939524096 2023-07-21T19:02:27.611 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:27 smithi193 ceph-mon[98204]: Adjusting osd_memory_target on smithi039 to 196.2M 2023-07-21T19:02:27.612 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:27 smithi193 ceph-mon[98204]: Unable to set osd_memory_target on smithi039 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-07-21T19:02:27.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:27 smithi155 ceph-mon[97934]: Adjusting osd_memory_target on smithi193 to 196.2M 2023-07-21T19:02:27.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:27 smithi155 ceph-mon[97934]: Unable to set osd_memory_target on smithi193 to 205773926: error parsing value: Value '205773926' is below minimum 939524096 2023-07-21T19:02:27.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:27 smithi155 ceph-mon[97934]: Adjusting osd_memory_target on smithi039 to 196.2M 2023-07-21T19:02:27.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:27 smithi155 ceph-mon[97934]: Unable to set osd_memory_target on smithi039 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-07-21T19:02:27.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:27 smithi039 ceph-mon[96489]: Adjusting osd_memory_target on smithi193 to 196.2M 2023-07-21T19:02:27.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:27 smithi039 ceph-mon[96489]: Unable to set osd_memory_target on smithi193 to 205773926: error parsing value: Value '205773926' is below minimum 939524096 2023-07-21T19:02:27.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:27 smithi039 ceph-mon[96489]: Adjusting osd_memory_target on smithi039 to 196.2M 2023-07-21T19:02:27.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:27 smithi039 ceph-mon[96489]: Unable to set osd_memory_target on smithi039 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-07-21T19:02:27.704 INFO:tasks.workunit.client.0.smithi039.stdout: 10 254897 29.77 MB/sec execute 340 sec latency 787.486 ms 2023-07-21T19:02:28.607 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:28 smithi193 ceph-mon[98204]: pgmap v658: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 23 MiB/s wr, 988 op/s 2023-07-21T19:02:28.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:28 smithi155 ceph-mon[97934]: pgmap v658: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 23 MiB/s wr, 988 op/s 2023-07-21T19:02:28.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:28 smithi039 ceph-mon[96489]: pgmap v658: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 23 MiB/s wr, 988 op/s 2023-07-21T19:02:28.704 INFO:tasks.workunit.client.0.smithi039.stdout: 10 255352 29.76 MB/sec execute 341 sec latency 1787.521 ms 2023-07-21T19:02:29.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:29 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T19:02:29.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:29 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T19:02:29.705 INFO:tasks.workunit.client.0.smithi039.stdout: 10 255817 29.74 MB/sec execute 342 sec latency 2787.559 ms 2023-07-21T19:02:29.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:29 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T19:02:29.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.915+0000 7f7245784700 1 -- 172.21.15.39:0/2658031715 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72400fed90 msgr2=0x7f7240102d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:02:29.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.915+0000 7f7245784700 1 --2- 172.21.15.39:0/2658031715 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72400fed90 0x7f7240102d50 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f72340099c0 tx=0x7f7234009cd0 comp rx=0 tx=0).stop 2023-07-21T19:02:29.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.915+0000 7f7245784700 1 -- 172.21.15.39:0/2658031715 shutdown_connections 2023-07-21T19:02:29.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.915+0000 7f7245784700 1 --2- 172.21.15.39:0/2658031715 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72400fe3f0 0x7f72400fe7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:29.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.915+0000 7f7245784700 1 --2- 172.21.15.39:0/2658031715 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7240103430 0x7f724010b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:29.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.915+0000 7f7245784700 1 --2- 172.21.15.39:0/2658031715 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72400fed90 0x7f7240102d50 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:29.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.915+0000 7f7245784700 1 -- 172.21.15.39:0/2658031715 >> 172.21.15.39:0/2658031715 conn(0x7f7240075f90 msgr2=0x7f7240076390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:02:29.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.915+0000 7f7245784700 1 -- 172.21.15.39:0/2658031715 shutdown_connections 2023-07-21T19:02:29.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.915+0000 7f7245784700 1 -- 172.21.15.39:0/2658031715 wait complete. 2023-07-21T19:02:29.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.916+0000 7f7245784700 1 Processor -- start 2023-07-21T19:02:29.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.916+0000 7f7245784700 1 -- start start 2023-07-21T19:02:29.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.917+0000 7f7245784700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72400fe3f0 0x7f72401988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:29.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.917+0000 7f7245784700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72400fed90 0x7f7240198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:29.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.917+0000 7f7245784700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7240103430 0x7f724019ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:29.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.917+0000 7f7245784700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f724019d0c0 con 0x7f7240103430 2023-07-21T19:02:29.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.917+0000 7f7245784700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f724019d230 con 0x7f72400fe3f0 2023-07-21T19:02:29.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.917+0000 7f7245784700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f724019d3f0 con 0x7f72400fed90 2023-07-21T19:02:29.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.917+0000 7f723effd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72400fe3f0 0x7f72401988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:29.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.917+0000 7f723f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7240103430 0x7f724019ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:29.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.917+0000 7f723e7fc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72400fed90 0x7f7240198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:29.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.917+0000 7f723f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7240103430 0x7f724019ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34372/0 (socket says 172.21.15.39:34372) 2023-07-21T19:02:29.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.917+0000 7f723e7fc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72400fed90 0x7f7240198e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:52922/0 (socket says 172.21.15.39:52922) 2023-07-21T19:02:29.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.917+0000 7f723effd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72400fe3f0 0x7f72401988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:46964/0 (socket says 172.21.15.39:46964) 2023-07-21T19:02:29.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.917+0000 7f723f7fe700 1 -- 172.21.15.39:0/2325965108 learned_addr learned my addr 172.21.15.39:0/2325965108 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:02:29.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.917+0000 7f723f7fe700 1 -- 172.21.15.39:0/2325965108 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72400fed90 msgr2=0x7f7240198e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:02:29.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.917+0000 7f723f7fe700 1 --2- 172.21.15.39:0/2325965108 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72400fed90 0x7f7240198e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:29.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.917+0000 7f723f7fe700 1 -- 172.21.15.39:0/2325965108 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72400fe3f0 msgr2=0x7f72401988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:02:29.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.917+0000 7f723f7fe700 1 --2- 172.21.15.39:0/2325965108 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72400fe3f0 0x7f72401988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:29.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.917+0000 7f723f7fe700 1 -- 172.21.15.39:0/2325965108 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7238009c40 con 0x7f7240103430 2023-07-21T19:02:29.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.917+0000 7f723f7fe700 1 --2- 172.21.15.39:0/2325965108 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7240103430 0x7f724019ca80 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f723800c370 tx=0x7f723800c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:02:29.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.918+0000 7f7227fff700 1 -- 172.21.15.39:0/2325965108 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7238017070 con 0x7f7240103430 2023-07-21T19:02:29.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.918+0000 7f7245784700 1 -- 172.21.15.39:0/2325965108 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72340095f0 con 0x7f7240103430 2023-07-21T19:02:29.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.918+0000 7f7227fff700 1 -- 172.21.15.39:0/2325965108 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f723800f040 con 0x7f7240103430 2023-07-21T19:02:29.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.918+0000 7f7227fff700 1 -- 172.21.15.39:0/2325965108 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723800e050 con 0x7f7240103430 2023-07-21T19:02:29.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.918+0000 7f7245784700 1 -- 172.21.15.39:0/2325965108 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72401a4d30 con 0x7f7240103430 2023-07-21T19:02:29.933 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.920+0000 7f7227fff700 1 -- 172.21.15.39:0/2325965108 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f723800cb60 con 0x7f7240103430 2023-07-21T19:02:29.933 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.921+0000 7f7227fff700 1 --2- 172.21.15.39:0/2325965108 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f72280a12c0 0x7f72280a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:29.933 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.921+0000 7f723effd700 1 --2- 172.21.15.39:0/2325965108 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f72280a12c0 0x7f72280a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:29.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.921+0000 7f7227fff700 1 -- 172.21.15.39:0/2325965108 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f72380cc6b0 con 0x7f7240103430 2023-07-21T19:02:29.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.921+0000 7f723effd700 1 --2- 172.21.15.39:0/2325965108 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f72280a12c0 0x7f72280a3770 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7230009c60 tx=0x7f7230009260 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:02:29.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.922+0000 7f725190ab80 1 Processor -- start 2023-07-21T19:02:29.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.922+0000 7f725190ab80 1 -- start start 2023-07-21T19:02:29.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.922+0000 7f725190ab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5615b516c6b0 0x5615b5315d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:29.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.922+0000 7f725190ab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5615b53162d0 0x5615b5310040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:29.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.922+0000 7f725190ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615b5310580 0x5615b53109d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:29.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.922+0000 7f725190ab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615b5215c50 con 0x5615b516c6b0 2023-07-21T19:02:29.935 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.922+0000 7f725190ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615b52689e0 con 0x5615b5310580 2023-07-21T19:02:29.935 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.922+0000 7f725190ab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615b5261e70 con 0x5615b53162d0 2023-07-21T19:02:29.935 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.923+0000 7f723effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5615b516c6b0 0x5615b5315d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:29.935 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.923+0000 7f723effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5615b516c6b0 0x5615b5315d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34376/0 (socket says 172.21.15.39:34376) 2023-07-21T19:02:29.935 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.923+0000 7f723effd700 1 -- 172.21.15.39:0/1672657036 learned_addr learned my addr 172.21.15.39:0/1672657036 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:02:29.935 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.923+0000 7f723e7fc700 1 --2- 172.21.15.39:0/1672657036 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5615b53162d0 0x5615b5310040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:29.935 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.923+0000 7f723effd700 1 -- 172.21.15.39:0/1672657036 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5615b53162d0 msgr2=0x5615b5310040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:02:29.936 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.923+0000 7f723effd700 1 --2- 172.21.15.39:0/1672657036 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5615b53162d0 0x5615b5310040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:29.936 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.923+0000 7f723effd700 1 -- 172.21.15.39:0/1672657036 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615b5310580 msgr2=0x5615b53109d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:02:29.936 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.923+0000 7f723effd700 1 --2- 172.21.15.39:0/1672657036 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615b5310580 0x5615b53109d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:29.936 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.923+0000 7f723effd700 1 -- 172.21.15.39:0/1672657036 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5615b508fd80 con 0x5615b516c6b0 2023-07-21T19:02:29.936 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.923+0000 7f723effd700 1 --2- 172.21.15.39:0/1672657036 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5615b516c6b0 0x5615b5315d90 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f723000e4e0 tx=0x7f7230014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a765126d1bad77b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:02:29.936 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.924+0000 7f72257fa700 1 -- 172.21.15.39:0/1672657036 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723000ee00 con 0x5615b516c6b0 2023-07-21T19:02:29.936 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.924+0000 7f72257fa700 1 -- 172.21.15.39:0/1672657036 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7230021070 con 0x5615b516c6b0 2023-07-21T19:02:29.936 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.924+0000 7f72257fa700 1 -- 172.21.15.39:0/1672657036 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723001d810 con 0x5615b516c6b0 2023-07-21T19:02:29.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.925+0000 7f725190ab80 1 -- 172.21.15.39:0/1672657036 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5615b516c6b0 msgr2=0x5615b5315d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:02:29.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.925+0000 7f725190ab80 1 --2- 172.21.15.39:0/1672657036 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5615b516c6b0 0x5615b5315d90 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f723000e4e0 tx=0x7f7230014000 comp rx=0 tx=0).stop 2023-07-21T19:02:29.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.925+0000 7f725190ab80 1 -- 172.21.15.39:0/1672657036 shutdown_connections 2023-07-21T19:02:29.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.925+0000 7f725190ab80 1 --2- 172.21.15.39:0/1672657036 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615b5310580 0x5615b53109d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:29.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.925+0000 7f725190ab80 1 --2- 172.21.15.39:0/1672657036 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5615b516c6b0 0x5615b5315d90 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:29.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.925+0000 7f725190ab80 1 --2- 172.21.15.39:0/1672657036 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5615b53162d0 0x5615b5310040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:29.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.925+0000 7f725190ab80 1 -- 172.21.15.39:0/1672657036 >> 172.21.15.39:0/1672657036 conn(0x5615b5305be0 msgr2=0x5615b530af10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:02:29.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.925+0000 7f725190ab80 1 -- 172.21.15.39:0/1672657036 shutdown_connections 2023-07-21T19:02:29.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.925+0000 7f725190ab80 1 -- 172.21.15.39:0/1672657036 wait complete. 2023-07-21T19:02:29.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.925+0000 7f725190ab80 1 -- start start 2023-07-21T19:02:29.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.926+0000 7f725190ab80 1 Processor -- start 2023-07-21T19:02:29.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.926+0000 7f725190ab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5615b516c6b0 0x5615b53199e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:29.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.926+0000 7f725190ab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5615b5310580 0x5615b5319f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:29.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.926+0000 7f725190ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615b53162d0 0x5615b531a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:29.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.926+0000 7f725190ab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615b5201a60 con 0x5615b516c6b0 2023-07-21T19:02:29.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.926+0000 7f725190ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615b5201760 con 0x5615b53162d0 2023-07-21T19:02:29.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.926+0000 7f725190ab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615b50e58c0 con 0x5615b5310580 2023-07-21T19:02:29.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.926+0000 7f725190ab80 10 client.0 _ll_register_callbacks cb 0x5615b51ff700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:02:29.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.926+0000 7f723effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5615b516c6b0 0x5615b53199e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:29.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.926+0000 7f723effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5615b516c6b0 0x5615b53199e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34382/0 (socket says 172.21.15.39:34382) 2023-07-21T19:02:29.942 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.926+0000 7f723effd700 1 -- 172.21.15.39:0/1308774402 learned_addr learned my addr 172.21.15.39:0/1308774402 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:02:29.942 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.926+0000 7f723f7fe700 1 --2- 172.21.15.39:0/1308774402 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615b53162d0 0x5615b531a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:29.942 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.926+0000 7f723e7fc700 1 --2- 172.21.15.39:0/1308774402 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5615b5310580 0x5615b5319f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:29.942 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.926+0000 7f723effd700 1 -- 172.21.15.39:0/1308774402 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5615b5310580 msgr2=0x5615b5319f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:02:29.942 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.926+0000 7f723effd700 1 --2- 172.21.15.39:0/1308774402 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5615b5310580 0x5615b5319f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:29.942 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.926+0000 7f723effd700 1 -- 172.21.15.39:0/1308774402 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615b53162d0 msgr2=0x5615b531a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:02:29.942 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.926+0000 7f723effd700 1 --2- 172.21.15.39:0/1308774402 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615b53162d0 0x5615b531a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:29.942 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.926+0000 7f723effd700 1 -- 172.21.15.39:0/1308774402 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7230017070 con 0x5615b516c6b0 2023-07-21T19:02:29.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.926+0000 7f723f7fe700 1 --2- 172.21.15.39:0/1308774402 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615b53162d0 0x5615b531a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:02:29.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.927+0000 7f723effd700 1 --2- 172.21.15.39:0/1308774402 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5615b516c6b0 0x5615b53199e0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f7230009ec0 tx=0x7f723000ede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fc431bb3429c559 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:02:29.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.927+0000 7f7222ff5700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T19:02:29.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.927+0000 7f7222ff5700 1 -- 172.21.15.39:0/1308774402 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7230005310 con 0x5615b516c6b0 2023-07-21T19:02:29.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.927+0000 7f7222ff5700 1 -- 172.21.15.39:0/1308774402 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f7230005470 con 0x5615b516c6b0 2023-07-21T19:02:29.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.927+0000 7f725190ab80 1 -- 172.21.15.39:0/1308774402 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5615b51bb490 con 0x5615b516c6b0 2023-07-21T19:02:29.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.927+0000 7f7222ff5700 1 -- 172.21.15.39:0/1308774402 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=102) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7230005630 con 0x5615b516c6b0 2023-07-21T19:02:29.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.927+0000 7f725190ab80 10 client.17661 fetch_fsmap learned FSMap version 102 2023-07-21T19:02:29.944 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.927+0000 7f725190ab80 1 -- 172.21.15.39:0/1308774402 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=102,monmap=4+}) v3 -- 0x5615b5078c00 con 0x5615b516c6b0 2023-07-21T19:02:29.944 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.927+0000 7f7222ff5700 1 -- 172.21.15.39:0/1308774402 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f723003b020 con 0x5615b516c6b0 2023-07-21T19:02:29.944 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.928+0000 7f7222ff5700 1 -- 172.21.15.39:0/1308774402 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 102) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f7230003680 con 0x5615b516c6b0 2023-07-21T19:02:29.944 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.929+0000 7f725190ab80 10 client.17661 fetch_fsmap finished waiting for FSMap version 102 2023-07-21T19:02:29.944 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.929+0000 7f725190ab80 10 client.17661 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:02:29.944 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.929+0000 7f725190ab80 20 client.17661 populate_metadata read hostname 'smithi039' 2023-07-21T19:02:29.944 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.929+0000 7f725190ab80 1 --2- 172.21.15.39:0/1308774402 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5615b5329a70 0x5615b532be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:29.945 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.929+0000 7f725190ab80 4 client.17661 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:02:29.945 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.929+0000 7f725190ab80 1 -- 172.21.15.39:0/1308774402 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5615b525ae50 con 0x5615b5329a70 2023-07-21T19:02:29.945 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.929+0000 7f723e7fc700 1 --2- 172.21.15.39:0/1308774402 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5615b5329a70 0x5615b532be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:29.945 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.929+0000 7f723e7fc700 1 --2- 172.21.15.39:0/1308774402 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5615b5329a70 0x5615b532be50 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f21f56fc0695e356 server_cookie=d2587af794839a4b in_seq=0 out_seq=0 2023-07-21T19:02:29.945 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.929+0000 7f7222ff5700 10 client.17661 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T19:02:29.946 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.930+0000 7f7222ff5700 1 -- 172.21.15.39:0/1308774402 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5615b525ae50 con 0x5615b5329a70 2023-07-21T19:02:29.946 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.930+0000 7f7222ff5700 10 client.17661 handle_command_reply: tid=0 2023-07-21T19:02:29.946 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.931+0000 7f725190ab80 1 client.17661 shutdown 2023-07-21T19:02:29.946 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.931+0000 7f725190ab80 1 -- 172.21.15.39:0/1308774402 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5615b516c6b0 msgr2=0x5615b53199e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:02:29.947 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.931+0000 7f725190ab80 1 --2- 172.21.15.39:0/1308774402 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5615b516c6b0 0x5615b53199e0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f7230009ec0 tx=0x7f723000ede0 comp rx=0 tx=0).stop 2023-07-21T19:02:29.947 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.931+0000 7f725190ab80 1 -- 172.21.15.39:0/1308774402 shutdown_connections 2023-07-21T19:02:29.947 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.931+0000 7f725190ab80 1 --2- 172.21.15.39:0/1308774402 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5615b5329a70 0x5615b532be50 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:29.947 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.931+0000 7f725190ab80 1 --2- 172.21.15.39:0/1308774402 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615b53162d0 0x5615b531a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:29.947 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.931+0000 7f725190ab80 1 --2- 172.21.15.39:0/1308774402 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5615b516c6b0 0x5615b53199e0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:29.947 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.931+0000 7f725190ab80 1 --2- 172.21.15.39:0/1308774402 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5615b5310580 0x5615b5319f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:29.947 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.931+0000 7f725190ab80 1 -- 172.21.15.39:0/1308774402 >> 172.21.15.39:0/1308774402 conn(0x5615b5305be0 msgr2=0x5615b530a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:02:29.947 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.931+0000 7f7222ff5700 0 client.17661 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T19:02:29.948 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.931+0000 7f725190ab80 1 -- 172.21.15.39:0/1308774402 shutdown_connections 2023-07-21T19:02:29.948 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.931+0000 7f725190ab80 1 -- 172.21.15.39:0/1308774402 wait complete. 2023-07-21T19:02:29.948 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.931+0000 7f725190ab80 20 client.17661 trim_cache size 0 max 16384 2023-07-21T19:02:29.958 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.949+0000 7f725190ab80 1 Processor -- start 2023-07-21T19:02:29.959 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.949+0000 7f725190ab80 1 -- start start 2023-07-21T19:02:29.960 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.949+0000 7f725190ab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5615b516c6b0 0x5615b5310040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:29.960 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.949+0000 7f725190ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615b5310580 0x5615b5320c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:29.960 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.949+0000 7f725190ab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5615b53162d0 0x5615b531a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:29.960 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.949+0000 7f725190ab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615b5317fe0 con 0x5615b516c6b0 2023-07-21T19:02:29.960 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.949+0000 7f725190ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615b5314770 con 0x5615b5310580 2023-07-21T19:02:29.961 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.949+0000 7f725190ab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615b5315fe0 con 0x5615b53162d0 2023-07-21T19:02:29.961 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.950+0000 7f723effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5615b516c6b0 0x5615b5310040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:29.961 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.950+0000 7f723effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5615b516c6b0 0x5615b5310040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34390/0 (socket says 172.21.15.39:34390) 2023-07-21T19:02:29.961 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.950+0000 7f723effd700 1 -- 172.21.15.39:0/2416665202 learned_addr learned my addr 172.21.15.39:0/2416665202 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:02:29.961 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.950+0000 7f723e7fc700 1 --2- 172.21.15.39:0/2416665202 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615b5310580 0x5615b5320c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:29.961 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.950+0000 7f723f7fe700 1 --2- 172.21.15.39:0/2416665202 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5615b53162d0 0x5615b531a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:29.961 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.950+0000 7f723e7fc700 1 -- 172.21.15.39:0/2416665202 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5615b53162d0 msgr2=0x5615b531a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:02:29.962 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.950+0000 7f723e7fc700 1 --2- 172.21.15.39:0/2416665202 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5615b53162d0 0x5615b531a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:29.962 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.950+0000 7f723e7fc700 1 -- 172.21.15.39:0/2416665202 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5615b516c6b0 msgr2=0x5615b5310040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:02:29.962 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.950+0000 7f723e7fc700 1 --2- 172.21.15.39:0/2416665202 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5615b516c6b0 0x5615b5310040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:29.962 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.950+0000 7f723e7fc700 1 -- 172.21.15.39:0/2416665202 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5615b50b61f0 con 0x5615b5310580 2023-07-21T19:02:29.962 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.950+0000 7f723f7fe700 1 --2- 172.21.15.39:0/2416665202 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5615b53162d0 0x5615b531a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:02:29.962 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.950+0000 7f723e7fc700 1 --2- 172.21.15.39:0/2416665202 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615b5310580 0x5615b5320c60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f723400b580 tx=0x7f7234005040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a28e949df34641f5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:02:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.950+0000 7f7225ffb700 1 -- 172.21.15.39:0/2416665202 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7234005690 con 0x5615b5310580 2023-07-21T19:02:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.951+0000 7f7225ffb700 1 -- 172.21.15.39:0/2416665202 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f723401e040 con 0x5615b5310580 2023-07-21T19:02:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.951+0000 7f7225ffb700 1 -- 172.21.15.39:0/2416665202 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7234003bc0 con 0x5615b5310580 2023-07-21T19:02:29.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.951+0000 7f725190ab80 1 -- 172.21.15.39:0/2416665202 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615b5310580 msgr2=0x5615b5320c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:02:29.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.951+0000 7f725190ab80 1 --2- 172.21.15.39:0/2416665202 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615b5310580 0x5615b5320c60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f723400b580 tx=0x7f7234005040 comp rx=0 tx=0).stop 2023-07-21T19:02:29.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.951+0000 7f725190ab80 1 -- 172.21.15.39:0/2416665202 shutdown_connections 2023-07-21T19:02:29.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.951+0000 7f725190ab80 1 --2- 172.21.15.39:0/2416665202 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5615b53162d0 0x5615b531a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:29.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.951+0000 7f725190ab80 1 --2- 172.21.15.39:0/2416665202 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5615b516c6b0 0x5615b5310040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:29.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.951+0000 7f725190ab80 1 --2- 172.21.15.39:0/2416665202 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615b5310580 0x5615b5320c60 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:29.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.951+0000 7f725190ab80 1 -- 172.21.15.39:0/2416665202 >> 172.21.15.39:0/2416665202 conn(0x5615b5305be0 msgr2=0x5615b53181b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:02:29.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.951+0000 7f725190ab80 1 -- 172.21.15.39:0/2416665202 shutdown_connections 2023-07-21T19:02:29.965 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.951+0000 7f725190ab80 1 -- 172.21.15.39:0/2416665202 wait complete. 2023-07-21T19:02:29.965 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.952+0000 7f725190ab80 1 -- start start 2023-07-21T19:02:29.965 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.952+0000 7f725190ab80 1 Processor -- start 2023-07-21T19:02:29.966 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.955+0000 7f725190ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615b516c6b0 0x5615b531e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:29.966 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.955+0000 7f725190ab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5615b5310580 0x5615b531e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:29.966 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.955+0000 7f725190ab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5615b53162d0 0x5615b531ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:29.967 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.955+0000 7f725190ab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615b5385270 con 0x5615b5310580 2023-07-21T19:02:29.967 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.955+0000 7f725190ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615b53853e0 con 0x5615b516c6b0 2023-07-21T19:02:29.967 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.955+0000 7f725190ab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615b53855a0 con 0x5615b53162d0 2023-07-21T19:02:29.967 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.955+0000 7f725190ab80 10 client.0 _ll_register_callbacks cb 0x7f7230005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:02:29.967 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.955+0000 7f723e7fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5615b5310580 0x5615b531e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:29.967 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.955+0000 7f723e7fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5615b5310580 0x5615b531e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34398/0 (socket says 172.21.15.39:34398) 2023-07-21T19:02:29.967 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.955+0000 7f723e7fc700 1 -- 172.21.15.39:0/2326850968 learned_addr learned my addr 172.21.15.39:0/2326850968 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:02:29.967 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.955+0000 7f723effd700 1 --2- 172.21.15.39:0/2326850968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615b516c6b0 0x5615b531e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:29.968 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.956+0000 7f723f7fe700 1 --2- 172.21.15.39:0/2326850968 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5615b53162d0 0x5615b531ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:29.968 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.956+0000 7f723e7fc700 1 -- 172.21.15.39:0/2326850968 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5615b53162d0 msgr2=0x5615b531ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:02:29.968 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.956+0000 7f723e7fc700 1 --2- 172.21.15.39:0/2326850968 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5615b53162d0 0x5615b531ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:29.968 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.956+0000 7f723e7fc700 1 -- 172.21.15.39:0/2326850968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615b516c6b0 msgr2=0x5615b531e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:02:29.968 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.956+0000 7f723e7fc700 1 --2- 172.21.15.39:0/2326850968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615b516c6b0 0x5615b531e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:29.968 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.956+0000 7f723e7fc700 1 -- 172.21.15.39:0/2326850968 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72401a4d30 con 0x5615b5310580 2023-07-21T19:02:29.968 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.956+0000 7f723e7fc700 1 --2- 172.21.15.39:0/2326850968 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5615b5310580 0x5615b531e820 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f723400a010 tx=0x7f7234005510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3568a301fbe4d9d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:02:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.956+0000 7f7222ff5700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T19:02:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.956+0000 7f7222ff5700 1 -- 172.21.15.39:0/2326850968 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7234028d30 con 0x5615b5310580 2023-07-21T19:02:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.956+0000 7f7222ff5700 1 -- 172.21.15.39:0/2326850968 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f7234028160 con 0x5615b5310580 2023-07-21T19:02:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.956+0000 7f725190ab80 1 -- 172.21.15.39:0/2326850968 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7230021070 con 0x5615b5310580 2023-07-21T19:02:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.956+0000 7f7222ff5700 1 -- 172.21.15.39:0/2326850968 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=102) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7234028e90 con 0x5615b5310580 2023-07-21T19:02:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.956+0000 7f725190ab80 10 client.17667 fetch_fsmap learned FSMap version 102 2023-07-21T19:02:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.957+0000 7f725190ab80 1 -- 172.21.15.39:0/2326850968 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=102,monmap=4+}) v3 -- 0x7f7240067070 con 0x5615b5310580 2023-07-21T19:02:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.957+0000 7f7222ff5700 1 -- 172.21.15.39:0/2326850968 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7230021070 con 0x5615b5310580 2023-07-21T19:02:29.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.957+0000 7f7222ff5700 1 -- 172.21.15.39:0/2326850968 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 102) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f723400b950 con 0x5615b5310580 2023-07-21T19:02:29.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.957+0000 7f725190ab80 10 client.17667 fetch_fsmap finished waiting for FSMap version 102 2023-07-21T19:02:29.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.957+0000 7f725190ab80 10 client.17667 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:02:29.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.957+0000 7f725190ab80 20 client.17667 populate_metadata read hostname 'smithi039' 2023-07-21T19:02:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.957+0000 7f725190ab80 1 --2- 172.21.15.39:0/2326850968 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5615b53868b0 0x5615b5388cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.957+0000 7f725190ab80 4 client.17667 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:02:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.957+0000 7f725190ab80 1 -- 172.21.15.39:0/2326850968 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5615b521cff0 con 0x5615b53868b0 2023-07-21T19:02:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.957+0000 7f723effd700 1 --2- 172.21.15.39:0/2326850968 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5615b53868b0 0x5615b5388cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.958+0000 7f7222ff5700 10 client.17667 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T19:02:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:29.958+0000 7f723effd700 1 --2- 172.21.15.39:0/2326850968 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5615b53868b0 0x5615b5388cc0 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=436119b5aa3a9aad server_cookie=391198d10049dd40 in_seq=0 out_seq=0 2023-07-21T19:02:30.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:30 smithi155 ceph-mon[97934]: pgmap v659: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 19 MiB/s wr, 725 op/s 2023-07-21T19:02:30.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:30 smithi039 ceph-mon[96489]: pgmap v659: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 19 MiB/s wr, 725 op/s 2023-07-21T19:02:30.704 INFO:tasks.workunit.client.0.smithi039.stdout: 10 256305 29.72 MB/sec execute 343 sec latency 3787.597 ms 2023-07-21T19:02:30.860 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:30 smithi193 ceph-mon[98204]: pgmap v659: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 19 MiB/s wr, 725 op/s 2023-07-21T19:02:31.705 INFO:tasks.workunit.client.0.smithi039.stdout: 10 256528 29.66 MB/sec execute 344 sec latency 4787.630 ms 2023-07-21T19:02:32.705 INFO:tasks.workunit.client.0.smithi039.stdout: 10 257008 29.64 MB/sec execute 345 sec latency 5787.668 ms 2023-07-21T19:02:32.731 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:32 smithi039 ceph-mon[96489]: pgmap v660: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 19 MiB/s wr, 745 op/s 2023-07-21T19:02:32.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:32 smithi193 ceph-mon[98204]: pgmap v660: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 19 MiB/s wr, 745 op/s 2023-07-21T19:02:32.882 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:32 smithi155 ceph-mon[97934]: pgmap v660: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 19 MiB/s wr, 745 op/s 2023-07-21T19:02:33.506 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:33.496+0000 7f7222ff5700 1 -- 172.21.15.39:0/2326850968 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5615b521cff0 con 0x5615b53868b0 2023-07-21T19:02:33.507 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:33.496+0000 7f7222ff5700 10 client.17667 handle_command_reply: tid=0 2023-07-21T19:02:33.507 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:33.496+0000 7f725190ab80 1 client.17667 shutdown 2023-07-21T19:02:33.507 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:33.496+0000 7f725190ab80 1 -- 172.21.15.39:0/2326850968 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5615b5310580 msgr2=0x5615b531e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:02:33.507 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:33.496+0000 7f725190ab80 1 --2- 172.21.15.39:0/2326850968 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5615b5310580 0x5615b531e820 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f723400a010 tx=0x7f7234005510 comp rx=0 tx=0).stop 2023-07-21T19:02:33.507 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:33.496+0000 7f725190ab80 1 -- 172.21.15.39:0/2326850968 shutdown_connections 2023-07-21T19:02:33.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:33.496+0000 7f725190ab80 1 --2- 172.21.15.39:0/2326850968 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5615b53868b0 0x5615b5388cc0 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:33.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:33.496+0000 7f725190ab80 1 --2- 172.21.15.39:0/2326850968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615b516c6b0 0x5615b531e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:33.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:33.496+0000 7f725190ab80 1 --2- 172.21.15.39:0/2326850968 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5615b53162d0 0x5615b531ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:33.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:33.496+0000 7f725190ab80 1 --2- 172.21.15.39:0/2326850968 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5615b5310580 0x5615b531e820 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:33.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:33.496+0000 7f725190ab80 1 -- 172.21.15.39:0/2326850968 >> 172.21.15.39:0/2326850968 conn(0x5615b5305be0 msgr2=0x5615b5315410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:02:33.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:33.496+0000 7f7222ff5700 0 client.17667 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T19:02:33.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:33.497+0000 7f725190ab80 1 -- 172.21.15.39:0/2326850968 shutdown_connections 2023-07-21T19:02:33.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:33.497+0000 7f725190ab80 1 -- 172.21.15.39:0/2326850968 wait complete. 2023-07-21T19:02:33.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:33.497+0000 7f725190ab80 20 client.17667 trim_cache size 0 max 16384 2023-07-21T19:02:33.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T19:02:33.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T19:02:33.510 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T19:02:33.510 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T19:02:33.510 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:33.497+0000 7f7245784700 1 -- 172.21.15.39:0/2325965108 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f72280a12c0 msgr2=0x7f72280a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:02:33.511 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:33.497+0000 7f7245784700 1 --2- 172.21.15.39:0/2325965108 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f72280a12c0 0x7f72280a3770 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7230009c60 tx=0x7f7230009260 comp rx=0 tx=0).stop 2023-07-21T19:02:33.511 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:33.497+0000 7f7245784700 1 -- 172.21.15.39:0/2325965108 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7240103430 msgr2=0x7f724019ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:02:33.511 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:33.497+0000 7f7245784700 1 --2- 172.21.15.39:0/2325965108 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7240103430 0x7f724019ca80 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f723800c370 tx=0x7f723800c680 comp rx=0 tx=0).stop 2023-07-21T19:02:33.511 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:33.497+0000 7f7245784700 1 -- 172.21.15.39:0/2325965108 shutdown_connections 2023-07-21T19:02:33.511 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:33.497+0000 7f7245784700 1 --2- 172.21.15.39:0/2325965108 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f72280a12c0 0x7f72280a3770 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:33.511 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:33.497+0000 7f7245784700 1 --2- 172.21.15.39:0/2325965108 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72400fe3f0 0x7f72401988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:33.511 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:33.497+0000 7f7245784700 1 --2- 172.21.15.39:0/2325965108 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7240103430 0x7f724019ca80 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:33.497+0000 7f7245784700 1 --2- 172.21.15.39:0/2325965108 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72400fed90 0x7f7240198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:33.497+0000 7f7245784700 1 -- 172.21.15.39:0/2325965108 >> 172.21.15.39:0/2325965108 conn(0x7f7240075f90 msgr2=0x7f7240101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:02:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:33.498+0000 7f7245784700 1 -- 172.21.15.39:0/2325965108 shutdown_connections 2023-07-21T19:02:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:33.498+0000 7f7245784700 1 -- 172.21.15.39:0/2325965108 wait complete. 2023-07-21T19:02:33.705 INFO:tasks.workunit.client.0.smithi039.stdout: 10 257521 29.66 MB/sec execute 346 sec latency 6609.817 ms 2023-07-21T19:02:34.449 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:34 smithi039 ceph-mon[96489]: pgmap v661: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 21 MiB/s wr, 927 op/s 2023-07-21T19:02:34.449 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:34 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T19:02:34.449 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:34 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T19:02:34.449 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:34 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T19:02:34.609 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:34 smithi193 ceph-mon[98204]: pgmap v661: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 21 MiB/s wr, 927 op/s 2023-07-21T19:02:34.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:34 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T19:02:34.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:34 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T19:02:34.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:34 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T19:02:34.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:34 smithi155 ceph-mon[97934]: pgmap v661: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 21 MiB/s wr, 927 op/s 2023-07-21T19:02:34.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:34 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T19:02:34.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:34 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T19:02:34.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:34 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T19:02:34.705 INFO:tasks.workunit.client.0.smithi039.stdout: 10 258023 29.66 MB/sec execute 347 sec latency 87.630 ms 2023-07-21T19:02:35.705 INFO:tasks.workunit.client.0.smithi039.stdout: 10 258520 29.65 MB/sec execute 348 sec latency 80.610 ms 2023-07-21T19:02:36.426 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:36 smithi039 ceph-mon[96489]: pgmap v662: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.7 MiB/s wr, 588 op/s 2023-07-21T19:02:36.426 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:36 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:36.427 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:36 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:36.427 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:36 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:36.427 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:36 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:36.427 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:36 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:36.427 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:36 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:36.427 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:36 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:36.427 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:36 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:36.427 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:36 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:02:36.428 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:36 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:02:36.428 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:36 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T19:02:36.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:36 smithi155 ceph-mon[97934]: pgmap v662: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.7 MiB/s wr, 588 op/s 2023-07-21T19:02:36.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:36 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:36.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:36 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:36.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:36 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:36.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:36 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:36.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:36 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:36.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:36 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:36.629 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:36 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:36.630 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:36 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:36.630 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:36 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:02:36.630 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:36 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:02:36.630 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:36 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T19:02:36.705 INFO:tasks.workunit.client.0.smithi039.stdout: 10 259001 29.63 MB/sec execute 349 sec latency 100.096 ms 2023-07-21T19:02:36.714 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:36 smithi193 ceph-mon[98204]: pgmap v662: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.7 MiB/s wr, 588 op/s 2023-07-21T19:02:36.714 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:36 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:36.715 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:36 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:36.715 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:36 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:36.715 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:36 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:36.715 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:36 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:36.715 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:36 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:36.715 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:36 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:36.715 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:36 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:36.716 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:36 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:02:36.716 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:36 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:02:36.716 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:36 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T19:02:37.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:37 smithi155 ceph-mon[97934]: Adjusting osd_memory_target on smithi155 to 196.2M 2023-07-21T19:02:37.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:37 smithi155 ceph-mon[97934]: Unable to set osd_memory_target on smithi155 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-07-21T19:02:37.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:37 smithi039 ceph-mon[96489]: Adjusting osd_memory_target on smithi155 to 196.2M 2023-07-21T19:02:37.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:37 smithi039 ceph-mon[96489]: Unable to set osd_memory_target on smithi155 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-07-21T19:02:37.705 INFO:tasks.workunit.client.0.smithi039.stdout: 10 259560 29.63 MB/sec execute 350 sec latency 75.180 ms 2023-07-21T19:02:37.738 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:37 smithi193 ceph-mon[98204]: Adjusting osd_memory_target on smithi155 to 196.2M 2023-07-21T19:02:37.738 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:37 smithi193 ceph-mon[98204]: Unable to set osd_memory_target on smithi155 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-07-21T19:02:38.607 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:38 smithi193 ceph-mon[98204]: pgmap v663: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 15 MiB/s wr, 871 op/s 2023-07-21T19:02:38.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:38 smithi155 ceph-mon[97934]: pgmap v663: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 15 MiB/s wr, 871 op/s 2023-07-21T19:02:38.678 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:38 smithi039 ceph-mon[96489]: pgmap v663: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 15 MiB/s wr, 871 op/s 2023-07-21T19:02:38.706 INFO:tasks.workunit.client.0.smithi039.stdout: 10 260015 29.61 MB/sec execute 351 sec latency 166.799 ms 2023-07-21T19:02:39.705 INFO:tasks.workunit.client.0.smithi039.stdout: 10 260731 29.62 MB/sec execute 352 sec latency 96.080 ms 2023-07-21T19:02:40.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:40 smithi193 ceph-mon[98204]: pgmap v664: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 10 MiB/s wr, 511 op/s 2023-07-21T19:02:40.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:40 smithi155 ceph-mon[97934]: pgmap v664: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 10 MiB/s wr, 511 op/s 2023-07-21T19:02:40.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:40 smithi039 ceph-mon[96489]: pgmap v664: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 10 MiB/s wr, 511 op/s 2023-07-21T19:02:40.706 INFO:tasks.workunit.client.0.smithi039.stdout: 10 261456 29.68 MB/sec execute 353 sec latency 115.569 ms 2023-07-21T19:02:41.706 INFO:tasks.workunit.client.0.smithi039.stdout: 10 261904 29.67 MB/sec execute 354 sec latency 96.443 ms 2023-07-21T19:02:42.607 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:42 smithi193 ceph-mon[98204]: pgmap v665: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 11 MiB/s wr, 551 op/s 2023-07-21T19:02:42.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:42 smithi155 ceph-mon[97934]: pgmap v665: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 11 MiB/s wr, 551 op/s 2023-07-21T19:02:42.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:42 smithi039 ceph-mon[96489]: pgmap v665: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 11 MiB/s wr, 551 op/s 2023-07-21T19:02:42.708 INFO:tasks.workunit.client.0.smithi039.stdout: 10 262365 29.64 MB/sec execute 355 sec latency 92.032 ms 2023-07-21T19:02:43.707 INFO:tasks.workunit.client.0.smithi039.stdout: 10 262782 29.61 MB/sec execute 356 sec latency 130.874 ms 2023-07-21T19:02:44.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:44 smithi155 ceph-mon[97934]: pgmap v666: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 14 MiB/s wr, 693 op/s 2023-07-21T19:02:44.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:44 smithi039 ceph-mon[96489]: pgmap v666: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 14 MiB/s wr, 693 op/s 2023-07-21T19:02:44.707 INFO:tasks.workunit.client.0.smithi039.stdout: 10 263358 29.61 MB/sec execute 357 sec latency 113.960 ms 2023-07-21T19:02:44.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:44 smithi193 ceph-mon[98204]: pgmap v666: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 14 MiB/s wr, 693 op/s 2023-07-21T19:02:45.707 INFO:tasks.workunit.client.0.smithi039.stdout: 10 264058 29.61 MB/sec execute 358 sec latency 88.227 ms 2023-07-21T19:02:46.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:46 smithi155 ceph-mon[97934]: pgmap v667: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 10 MiB/s wr, 485 op/s 2023-07-21T19:02:46.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:46 smithi039 ceph-mon[96489]: pgmap v667: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 10 MiB/s wr, 485 op/s 2023-07-21T19:02:46.707 INFO:tasks.workunit.client.0.smithi039.stdout: 10 264784 29.67 MB/sec execute 359 sec latency 65.155 ms 2023-07-21T19:02:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.824+0000 7feff5835700 1 -- 172.21.15.39:0/766271246 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feff00fea30 msgr2=0x7feff00feea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:02:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.824+0000 7feff5835700 1 --2- 172.21.15.39:0/766271246 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feff00fea30 0x7feff00feea0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fefe4009990 tx=0x7fefe4009ca0 comp rx=0 tx=0).stop 2023-07-21T19:02:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.824+0000 7feff5835700 1 -- 172.21.15.39:0/766271246 shutdown_connections 2023-07-21T19:02:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.824+0000 7feff5835700 1 --2- 172.21.15.39:0/766271246 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feff00ff3e0 0x7feff010b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.824+0000 7feff5835700 1 --2- 172.21.15.39:0/766271246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feff0104a40 0x7feff0104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.824+0000 7feff5835700 1 --2- 172.21.15.39:0/766271246 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feff00fea30 0x7feff00feea0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:46.836 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.824+0000 7feff5835700 1 -- 172.21.15.39:0/766271246 >> 172.21.15.39:0/766271246 conn(0x7feff00fa4f0 msgr2=0x7feff00fc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:02:46.836 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.824+0000 7feff5835700 1 -- 172.21.15.39:0/766271246 shutdown_connections 2023-07-21T19:02:46.836 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.824+0000 7feff5835700 1 -- 172.21.15.39:0/766271246 wait complete. 2023-07-21T19:02:46.836 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.824+0000 7feff5835700 1 Processor -- start 2023-07-21T19:02:46.838 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.824+0000 7feff5835700 1 -- start start 2023-07-21T19:02:46.838 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.825+0000 7feff5835700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feff00fea30 0x7feff0196bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:46.838 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.825+0000 7feff5835700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feff00ff3e0 0x7feff0197100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.825+0000 7feff5835700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feff0104a40 0x7feff01a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.825+0000 7feff5835700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff01a2030 con 0x7feff00fea30 2023-07-21T19:02:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.825+0000 7feff5835700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff01a21a0 con 0x7feff0104a40 2023-07-21T19:02:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.825+0000 7feff5835700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff01a2360 con 0x7feff00ff3e0 2023-07-21T19:02:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.825+0000 7fefeeffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feff00fea30 0x7feff0196bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.825+0000 7fefeeffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feff00fea30 0x7feff0196bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34410/0 (socket says 172.21.15.39:34410) 2023-07-21T19:02:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.825+0000 7fefeeffd700 1 -- 172.21.15.39:0/2243896190 learned_addr learned my addr 172.21.15.39:0/2243896190 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:02:46.840 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.825+0000 7fefef7fe700 1 --2- 172.21.15.39:0/2243896190 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feff0104a40 0x7feff01a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:46.840 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.825+0000 7fefee7fc700 1 --2- 172.21.15.39:0/2243896190 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feff00ff3e0 0x7feff0197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:46.840 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.825+0000 7fefeeffd700 1 -- 172.21.15.39:0/2243896190 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feff00ff3e0 msgr2=0x7feff0197100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:02:46.840 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.825+0000 7fefeeffd700 1 --2- 172.21.15.39:0/2243896190 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feff00ff3e0 0x7feff0197100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:46.840 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.825+0000 7fefeeffd700 1 -- 172.21.15.39:0/2243896190 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feff0104a40 msgr2=0x7feff01a1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:02:46.840 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.825+0000 7fefeeffd700 1 --2- 172.21.15.39:0/2243896190 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feff0104a40 0x7feff01a1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:46.840 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.825+0000 7fefeeffd700 1 -- 172.21.15.39:0/2243896190 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefe40095f0 con 0x7feff00fea30 2023-07-21T19:02:46.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.825+0000 7fefeeffd700 1 --2- 172.21.15.39:0/2243896190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feff00fea30 0x7feff0196bc0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fefe000e970 tx=0x7fefe000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:02:46.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.826+0000 7feff4833700 1 -- 172.21.15.39:0/2243896190 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe0009790 con 0x7feff00fea30 2023-07-21T19:02:46.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.826+0000 7feff4833700 1 -- 172.21.15.39:0/2243896190 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fefe000c7e0 con 0x7feff00fea30 2023-07-21T19:02:46.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.826+0000 7feff5835700 1 -- 172.21.15.39:0/2243896190 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feff01a2640 con 0x7feff00fea30 2023-07-21T19:02:46.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.826+0000 7feff4833700 1 -- 172.21.15.39:0/2243896190 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe0003c70 con 0x7feff00fea30 2023-07-21T19:02:46.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.826+0000 7feff5835700 1 -- 172.21.15.39:0/2243896190 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feff01a2b50 con 0x7feff00fea30 2023-07-21T19:02:46.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.828+0000 7feff4833700 1 -- 172.21.15.39:0/2243896190 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fefe000cc50 con 0x7feff00fea30 2023-07-21T19:02:46.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.828+0000 7feff4833700 1 --2- 172.21.15.39:0/2243896190 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fefd80a12c0 0x7fefd80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:46.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.829+0000 7fefee7fc700 1 --2- 172.21.15.39:0/2243896190 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fefd80a12c0 0x7fefd80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:46.846 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.829+0000 7feff4833700 1 -- 172.21.15.39:0/2243896190 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fefe0014070 con 0x7feff00fea30 2023-07-21T19:02:46.846 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.829+0000 7fefee7fc700 1 --2- 172.21.15.39:0/2243896190 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fefd80a12c0 0x7fefd80a3770 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7feff0069500 tx=0x7fefe4005fb0 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:02:46.846 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.829+0000 7ff0019bbb80 1 Processor -- start 2023-07-21T19:02:46.846 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.829+0000 7ff0019bbb80 1 -- start start 2023-07-21T19:02:46.846 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.829+0000 7ff0019bbb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bb173096b0 0x55bb174b2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:46.846 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.829+0000 7ff0019bbb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bb174b32d0 0x55bb174ad040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:46.846 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.830+0000 7ff0019bbb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bb174ad580 0x55bb174ad9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:46.847 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.830+0000 7ff0019bbb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb173b2c50 con 0x55bb173096b0 2023-07-21T19:02:46.847 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.830+0000 7ff0019bbb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb174059e0 con 0x55bb174ad580 2023-07-21T19:02:46.847 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.830+0000 7ff0019bbb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb173fee70 con 0x55bb174b32d0 2023-07-21T19:02:46.847 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.830+0000 7fefeeffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bb173096b0 0x55bb174b2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:46.847 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.830+0000 7fefeeffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bb173096b0 0x55bb174b2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34418/0 (socket says 172.21.15.39:34418) 2023-07-21T19:02:46.847 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.830+0000 7fefeeffd700 1 -- 172.21.15.39:0/85977924 learned_addr learned my addr 172.21.15.39:0/85977924 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:02:46.847 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.830+0000 7fefef7fe700 1 --2- 172.21.15.39:0/85977924 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bb174ad580 0x55bb174ad9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:46.847 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.830+0000 7fefee7fc700 1 --2- 172.21.15.39:0/85977924 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bb174b32d0 0x55bb174ad040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:46.848 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.830+0000 7fefeeffd700 1 -- 172.21.15.39:0/85977924 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bb174b32d0 msgr2=0x55bb174ad040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:02:46.848 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.830+0000 7fefeeffd700 1 --2- 172.21.15.39:0/85977924 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bb174b32d0 0x55bb174ad040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:46.848 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.830+0000 7fefeeffd700 1 -- 172.21.15.39:0/85977924 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bb174ad580 msgr2=0x55bb174ad9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:02:46.848 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.830+0000 7fefeeffd700 1 --2- 172.21.15.39:0/85977924 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bb174ad580 0x55bb174ad9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:46.848 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.830+0000 7fefeeffd700 1 -- 172.21.15.39:0/85977924 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb1722cd80 con 0x55bb173096b0 2023-07-21T19:02:46.849 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.830+0000 7fefee7fc700 1 --2- 172.21.15.39:0/85977924 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bb174b32d0 0x55bb174ad040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:02:46.849 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.830+0000 7fefef7fe700 1 --2- 172.21.15.39:0/85977924 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bb174ad580 0x55bb174ad9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:02:46.849 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.830+0000 7fefeeffd700 1 --2- 172.21.15.39:0/85977924 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bb173096b0 0x55bb174b2d90 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fefe00a1ec0 tx=0x7fefe00999d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e31cbe3adbaaa781 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:02:46.849 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.830+0000 7fefd5ffb700 1 -- 172.21.15.39:0/85977924 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe00a23f0 con 0x55bb173096b0 2023-07-21T19:02:46.849 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.830+0000 7fefd5ffb700 1 -- 172.21.15.39:0/85977924 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fefe00966b0 con 0x55bb173096b0 2023-07-21T19:02:46.849 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.830+0000 7fefd5ffb700 1 -- 172.21.15.39:0/85977924 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe0099490 con 0x55bb173096b0 2023-07-21T19:02:46.849 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.831+0000 7ff0019bbb80 1 -- 172.21.15.39:0/85977924 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bb173096b0 msgr2=0x55bb174b2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:02:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.831+0000 7ff0019bbb80 1 --2- 172.21.15.39:0/85977924 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bb173096b0 0x55bb174b2d90 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fefe00a1ec0 tx=0x7fefe00999d0 comp rx=0 tx=0).stop 2023-07-21T19:02:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.831+0000 7ff0019bbb80 1 -- 172.21.15.39:0/85977924 shutdown_connections 2023-07-21T19:02:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.831+0000 7ff0019bbb80 1 --2- 172.21.15.39:0/85977924 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bb174ad580 0x55bb174ad9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.831+0000 7ff0019bbb80 1 --2- 172.21.15.39:0/85977924 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bb173096b0 0x55bb174b2d90 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.831+0000 7ff0019bbb80 1 --2- 172.21.15.39:0/85977924 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bb174b32d0 0x55bb174ad040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.831+0000 7ff0019bbb80 1 -- 172.21.15.39:0/85977924 >> 172.21.15.39:0/85977924 conn(0x55bb174a2be0 msgr2=0x55bb174a7f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:02:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.831+0000 7ff0019bbb80 1 -- 172.21.15.39:0/85977924 shutdown_connections 2023-07-21T19:02:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.831+0000 7ff0019bbb80 1 -- 172.21.15.39:0/85977924 wait complete. 2023-07-21T19:02:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.831+0000 7ff0019bbb80 1 -- start start 2023-07-21T19:02:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.832+0000 7ff0019bbb80 1 Processor -- start 2023-07-21T19:02:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.832+0000 7ff0019bbb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bb173096b0 0x55bb174b9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.832+0000 7ff0019bbb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bb174ad580 0x55bb174b6850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.832+0000 7ff0019bbb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bb174b32d0 0x55bb174b6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.832+0000 7ff0019bbb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb1739ea60 con 0x55bb174b32d0 2023-07-21T19:02:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.832+0000 7ff0019bbb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb1739e760 con 0x55bb174ad580 2023-07-21T19:02:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.832+0000 7ff0019bbb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb172828c0 con 0x55bb173096b0 2023-07-21T19:02:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.832+0000 7ff0019bbb80 10 client.0 _ll_register_callbacks cb 0x55bb1739c700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:02:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.832+0000 7fefef7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bb174b32d0 0x55bb174b6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.832+0000 7fefef7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bb174b32d0 0x55bb174b6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34428/0 (socket says 172.21.15.39:34428) 2023-07-21T19:02:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.832+0000 7fefef7fe700 1 -- 172.21.15.39:0/927816344 learned_addr learned my addr 172.21.15.39:0/927816344 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:02:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.832+0000 7fefee7fc700 1 --2- 172.21.15.39:0/927816344 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bb174ad580 0x55bb174b6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.832+0000 7fefeeffd700 1 --2- 172.21.15.39:0/927816344 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bb173096b0 0x55bb174b9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.832+0000 7fefef7fe700 1 -- 172.21.15.39:0/927816344 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bb173096b0 msgr2=0x55bb174b9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:02:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.833+0000 7fefef7fe700 1 --2- 172.21.15.39:0/927816344 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bb173096b0 0x55bb174b9640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.833+0000 7fefef7fe700 1 -- 172.21.15.39:0/927816344 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bb174ad580 msgr2=0x55bb174b6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:02:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.833+0000 7fefef7fe700 1 --2- 172.21.15.39:0/927816344 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bb174ad580 0x55bb174b6850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.833+0000 7fefef7fe700 1 -- 172.21.15.39:0/927816344 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feff01a2b50 con 0x55bb174b32d0 2023-07-21T19:02:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.833+0000 7fefef7fe700 1 --2- 172.21.15.39:0/927816344 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bb174b32d0 0x55bb174b6f90 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fefe8007af0 tx=0x7fefe8007b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb4db08785cdbeb2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:02:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.833+0000 7fefceffd700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T19:02:46.854 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.833+0000 7fefceffd700 1 -- 172.21.15.39:0/927816344 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe800e040 con 0x55bb174b32d0 2023-07-21T19:02:46.854 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.833+0000 7fefceffd700 1 -- 172.21.15.39:0/927816344 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fefe8015860 con 0x55bb174b32d0 2023-07-21T19:02:46.854 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.833+0000 7ff0019bbb80 1 -- 172.21.15.39:0/927816344 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bb17358490 con 0x55bb174b32d0 2023-07-21T19:02:46.854 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.833+0000 7fefceffd700 1 -- 172.21.15.39:0/927816344 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) v2 ==== 24+0+0 (secure 0 0 0) 0x7fefe8015a60 con 0x55bb174b32d0 2023-07-21T19:02:46.854 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.833+0000 7ff0019bbb80 10 client.17679 fetch_fsmap learned FSMap version 103 2023-07-21T19:02:46.854 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.833+0000 7ff0019bbb80 1 -- 172.21.15.39:0/927816344 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=4+}) v3 -- 0x55bb17215c00 con 0x55bb174b32d0 2023-07-21T19:02:46.854 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.833+0000 7fefceffd700 1 -- 172.21.15.39:0/927816344 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bb17358490 con 0x55bb174b32d0 2023-07-21T19:02:46.854 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.833+0000 7fefceffd700 1 -- 172.21.15.39:0/927816344 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 103) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fefe801e3a0 con 0x55bb174b32d0 2023-07-21T19:02:46.855 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.833+0000 7ff0019bbb80 10 client.17679 fetch_fsmap finished waiting for FSMap version 103 2023-07-21T19:02:46.855 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.833+0000 7ff0019bbb80 10 client.17679 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:02:46.855 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.833+0000 7ff0019bbb80 20 client.17679 populate_metadata read hostname 'smithi039' 2023-07-21T19:02:46.855 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.833+0000 7ff0019bbb80 1 --2- 172.21.15.39:0/927816344 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55bb174c6c10 0x55bb174c9020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:46.855 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.834+0000 7ff0019bbb80 4 client.17679 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:02:46.855 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.834+0000 7ff0019bbb80 1 -- 172.21.15.39:0/927816344 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bb173f7e50 con 0x55bb174c6c10 2023-07-21T19:02:46.855 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.834+0000 7fefeeffd700 1 --2- 172.21.15.39:0/927816344 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55bb174c6c10 0x55bb174c9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:46.855 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.834+0000 7fefceffd700 10 client.17679 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T19:02:46.856 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.834+0000 7fefeeffd700 1 --2- 172.21.15.39:0/927816344 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55bb174c6c10 0x55bb174c9020 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62a2b6784669d41b server_cookie=9bd4b8d3e61454ab in_seq=0 out_seq=0 2023-07-21T19:02:46.856 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.835+0000 7fefceffd700 1 -- 172.21.15.39:0/927816344 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55bb173f7e50 con 0x55bb174c6c10 2023-07-21T19:02:46.856 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.835+0000 7fefceffd700 10 client.17679 handle_command_reply: tid=0 2023-07-21T19:02:46.856 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.835+0000 7ff0019bbb80 1 client.17679 shutdown 2023-07-21T19:02:46.857 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.835+0000 7ff0019bbb80 1 -- 172.21.15.39:0/927816344 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bb174b32d0 msgr2=0x55bb174b6f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:02:46.857 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.835+0000 7ff0019bbb80 1 --2- 172.21.15.39:0/927816344 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bb174b32d0 0x55bb174b6f90 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fefe8007af0 tx=0x7fefe8007b20 comp rx=0 tx=0).stop 2023-07-21T19:02:46.857 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.835+0000 7ff0019bbb80 1 -- 172.21.15.39:0/927816344 shutdown_connections 2023-07-21T19:02:46.857 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.835+0000 7ff0019bbb80 1 --2- 172.21.15.39:0/927816344 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55bb174c6c10 0x55bb174c9020 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:46.858 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.835+0000 7ff0019bbb80 1 --2- 172.21.15.39:0/927816344 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bb174b32d0 0x55bb174b6f90 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:46.858 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.835+0000 7ff0019bbb80 1 --2- 172.21.15.39:0/927816344 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bb173096b0 0x55bb174b9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:46.858 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.835+0000 7ff0019bbb80 1 --2- 172.21.15.39:0/927816344 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bb174ad580 0x55bb174b6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:46.858 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.835+0000 7ff0019bbb80 1 -- 172.21.15.39:0/927816344 >> 172.21.15.39:0/927816344 conn(0x55bb174a2be0 msgr2=0x55bb174a74e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:02:46.858 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.835+0000 7fefceffd700 0 client.17679 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T19:02:46.858 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.835+0000 7ff0019bbb80 1 -- 172.21.15.39:0/927816344 shutdown_connections 2023-07-21T19:02:46.859 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.835+0000 7ff0019bbb80 1 -- 172.21.15.39:0/927816344 wait complete. 2023-07-21T19:02:46.859 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.835+0000 7ff0019bbb80 20 client.17679 trim_cache size 0 max 16384 2023-07-21T19:02:46.859 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:46 smithi193 ceph-mon[98204]: pgmap v667: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 10 MiB/s wr, 485 op/s 2023-07-21T19:02:46.866 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.850+0000 7ff0019bbb80 1 Processor -- start 2023-07-21T19:02:46.866 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.850+0000 7ff0019bbb80 1 -- start start 2023-07-21T19:02:46.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.850+0000 7ff0019bbb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bb173096b0 0x55bb174b6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:46.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.850+0000 7ff0019bbb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bb174ad580 0x55bb174b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:46.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.850+0000 7ff0019bbb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bb174b32d0 0x55bb174a7c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:46.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.850+0000 7ff0019bbb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb174a8fb0 con 0x55bb173096b0 2023-07-21T19:02:46.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.850+0000 7ff0019bbb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb174b72f0 con 0x55bb174ad580 2023-07-21T19:02:46.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.850+0000 7ff0019bbb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb174b1640 con 0x55bb174b32d0 2023-07-21T19:02:46.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.850+0000 7fefeeffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bb173096b0 0x55bb174b6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:46.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.850+0000 7fefeeffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bb173096b0 0x55bb174b6030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34432/0 (socket says 172.21.15.39:34432) 2023-07-21T19:02:46.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.850+0000 7fefeeffd700 1 -- 172.21.15.39:0/2061579028 learned_addr learned my addr 172.21.15.39:0/2061579028 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:02:46.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.850+0000 7fefee7fc700 1 --2- 172.21.15.39:0/2061579028 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bb174ad580 0x55bb174b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:46.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.851+0000 7fefef7fe700 1 --2- 172.21.15.39:0/2061579028 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bb174b32d0 0x55bb174a7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:46.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.851+0000 7fefeeffd700 1 -- 172.21.15.39:0/2061579028 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bb174b32d0 msgr2=0x55bb174a7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:02:46.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.851+0000 7fefeeffd700 1 --2- 172.21.15.39:0/2061579028 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bb174b32d0 0x55bb174a7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:46.869 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.851+0000 7fefeeffd700 1 -- 172.21.15.39:0/2061579028 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bb174ad580 msgr2=0x55bb174b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:02:46.869 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.851+0000 7fefeeffd700 1 --2- 172.21.15.39:0/2061579028 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bb174ad580 0x55bb174b2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:46.869 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.851+0000 7fefeeffd700 1 -- 172.21.15.39:0/2061579028 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb172531f0 con 0x55bb173096b0 2023-07-21T19:02:46.869 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.851+0000 7fefeeffd700 1 --2- 172.21.15.39:0/2061579028 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bb173096b0 0x55bb174b6030 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fefe0099f30 tx=0x7fefe0003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb65ad6bb326393a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:02:46.869 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.851+0000 7fefd67fc700 1 -- 172.21.15.39:0/2061579028 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe00a23f0 con 0x55bb173096b0 2023-07-21T19:02:46.869 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.851+0000 7fefd67fc700 1 -- 172.21.15.39:0/2061579028 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fefe0098340 con 0x55bb173096b0 2023-07-21T19:02:46.870 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.851+0000 7fefd67fc700 1 -- 172.21.15.39:0/2061579028 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe00ca3d0 con 0x55bb173096b0 2023-07-21T19:02:46.870 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.852+0000 7ff0019bbb80 1 -- 172.21.15.39:0/2061579028 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bb173096b0 msgr2=0x55bb174b6030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:02:46.870 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.852+0000 7ff0019bbb80 1 --2- 172.21.15.39:0/2061579028 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bb173096b0 0x55bb174b6030 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fefe0099f30 tx=0x7fefe0003680 comp rx=0 tx=0).stop 2023-07-21T19:02:46.870 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.852+0000 7ff0019bbb80 1 -- 172.21.15.39:0/2061579028 shutdown_connections 2023-07-21T19:02:46.870 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.852+0000 7ff0019bbb80 1 --2- 172.21.15.39:0/2061579028 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bb174b32d0 0x55bb174a7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:46.870 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.852+0000 7ff0019bbb80 1 --2- 172.21.15.39:0/2061579028 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bb173096b0 0x55bb174b6030 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:46.870 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.852+0000 7ff0019bbb80 1 --2- 172.21.15.39:0/2061579028 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bb174ad580 0x55bb174b2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:46.870 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.852+0000 7ff0019bbb80 1 -- 172.21.15.39:0/2061579028 >> 172.21.15.39:0/2061579028 conn(0x55bb174a2be0 msgr2=0x55bb174b1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:02:46.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.852+0000 7ff0019bbb80 1 -- 172.21.15.39:0/2061579028 shutdown_connections 2023-07-21T19:02:46.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.852+0000 7ff0019bbb80 1 -- 172.21.15.39:0/2061579028 wait complete. 2023-07-21T19:02:46.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.852+0000 7ff0019bbb80 1 -- start start 2023-07-21T19:02:46.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.852+0000 7ff0019bbb80 1 Processor -- start 2023-07-21T19:02:46.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.852+0000 7ff0019bbb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bb173096b0 0x55bb174bbe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:46.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.852+0000 7ff0019bbb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bb174ad580 0x55bb174bc380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:46.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.852+0000 7ff0019bbb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bb174b32d0 0x55bb17513aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:46.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.852+0000 7ff0019bbb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb17513fe0 con 0x55bb174ad580 2023-07-21T19:02:46.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.852+0000 7ff0019bbb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb17514150 con 0x55bb173096b0 2023-07-21T19:02:46.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.852+0000 7ff0019bbb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb175142c0 con 0x55bb174b32d0 2023-07-21T19:02:46.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.853+0000 7ff0019bbb80 10 client.0 _ll_register_callbacks cb 0x7fefe80094d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:02:46.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.853+0000 7fefee7fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bb174ad580 0x55bb174bc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:46.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.853+0000 7fefee7fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bb174ad580 0x55bb174bc380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34440/0 (socket says 172.21.15.39:34440) 2023-07-21T19:02:46.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.853+0000 7fefee7fc700 1 -- 172.21.15.39:0/367762564 learned_addr learned my addr 172.21.15.39:0/367762564 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:02:46.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.853+0000 7fefef7fe700 1 --2- 172.21.15.39:0/367762564 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bb174b32d0 0x55bb17513aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:46.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.853+0000 7fefeeffd700 1 --2- 172.21.15.39:0/367762564 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bb173096b0 0x55bb174bbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:46.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.853+0000 7fefee7fc700 1 -- 172.21.15.39:0/367762564 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bb174b32d0 msgr2=0x55bb17513aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:02:46.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.853+0000 7fefee7fc700 1 --2- 172.21.15.39:0/367762564 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bb174b32d0 0x55bb17513aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:46.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.853+0000 7fefee7fc700 1 -- 172.21.15.39:0/367762564 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bb173096b0 msgr2=0x55bb174bbe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:02:46.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.853+0000 7fefee7fc700 1 --2- 172.21.15.39:0/367762564 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bb173096b0 0x55bb174bbe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:46.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.853+0000 7fefee7fc700 1 -- 172.21.15.39:0/367762564 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefe00b6040 con 0x55bb174ad580 2023-07-21T19:02:46.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.853+0000 7fefee7fc700 1 --2- 172.21.15.39:0/367762564 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bb174ad580 0x55bb174bc380 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fefe4009990 tx=0x7fefe4009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7783264a2882f3df server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:02:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.853+0000 7fefceffd700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T19:02:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.853+0000 7fefceffd700 1 -- 172.21.15.39:0/367762564 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe400f460 con 0x55bb174ad580 2023-07-21T19:02:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.853+0000 7fefceffd700 1 -- 172.21.15.39:0/367762564 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fefe400f5c0 con 0x55bb174ad580 2023-07-21T19:02:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.853+0000 7ff0019bbb80 1 -- 172.21.15.39:0/367762564 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fefe00966b0 con 0x55bb174ad580 2023-07-21T19:02:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.853+0000 7fefceffd700 1 -- 172.21.15.39:0/367762564 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) v2 ==== 24+0+0 (secure 0 0 0) 0x7fefe400f780 con 0x55bb174ad580 2023-07-21T19:02:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.853+0000 7ff0019bbb80 10 client.17685 fetch_fsmap learned FSMap version 103 2023-07-21T19:02:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.853+0000 7ff0019bbb80 1 -- 172.21.15.39:0/367762564 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=4+}) v3 -- 0x55bb1722b250 con 0x55bb174ad580 2023-07-21T19:02:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.854+0000 7fefceffd700 1 -- 172.21.15.39:0/367762564 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fefe4040020 con 0x55bb174ad580 2023-07-21T19:02:46.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.854+0000 7fefceffd700 1 -- 172.21.15.39:0/367762564 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 103) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fefe4031040 con 0x55bb174ad580 2023-07-21T19:02:46.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.854+0000 7ff0019bbb80 10 client.17685 fetch_fsmap finished waiting for FSMap version 103 2023-07-21T19:02:46.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.854+0000 7ff0019bbb80 10 client.17685 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:02:46.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.854+0000 7ff0019bbb80 20 client.17685 populate_metadata read hostname 'smithi039' 2023-07-21T19:02:46.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.854+0000 7ff0019bbb80 1 --2- 172.21.15.39:0/367762564 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55bb1751ba40 0x55bb1751de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:46.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.854+0000 7ff0019bbb80 4 client.17685 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:02:46.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.854+0000 7ff0019bbb80 1 -- 172.21.15.39:0/367762564 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55bb173b9ff0 con 0x55bb1751ba40 2023-07-21T19:02:46.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.854+0000 7fefeeffd700 1 --2- 172.21.15.39:0/367762564 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55bb1751ba40 0x55bb1751de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:46.876 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.854+0000 7fefceffd700 10 client.17685 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T19:02:46.876 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:46.854+0000 7fefeeffd700 1 --2- 172.21.15.39:0/367762564 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55bb1751ba40 0x55bb1751de50 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b42cbc29793a6680 server_cookie=646cf2f647d3cc5d in_seq=0 out_seq=0 2023-07-21T19:02:47.101 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:47.089+0000 7fefceffd700 1 -- 172.21.15.39:0/367762564 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55bb173b9ff0 con 0x55bb1751ba40 2023-07-21T19:02:47.102 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:47.089+0000 7fefceffd700 10 client.17685 handle_command_reply: tid=0 2023-07-21T19:02:47.102 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:47.089+0000 7ff0019bbb80 1 client.17685 shutdown 2023-07-21T19:02:47.102 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:47.089+0000 7ff0019bbb80 1 -- 172.21.15.39:0/367762564 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bb174ad580 msgr2=0x55bb174bc380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:02:47.103 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:47.089+0000 7ff0019bbb80 1 --2- 172.21.15.39:0/367762564 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bb174ad580 0x55bb174bc380 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fefe4009990 tx=0x7fefe4009ca0 comp rx=0 tx=0).stop 2023-07-21T19:02:47.103 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:47.089+0000 7ff0019bbb80 1 -- 172.21.15.39:0/367762564 shutdown_connections 2023-07-21T19:02:47.103 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:47.089+0000 7ff0019bbb80 1 --2- 172.21.15.39:0/367762564 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55bb1751ba40 0x55bb1751de50 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:47.104 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:47.089+0000 7ff0019bbb80 1 --2- 172.21.15.39:0/367762564 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bb173096b0 0x55bb174bbe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:47.104 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:47.089+0000 7ff0019bbb80 1 --2- 172.21.15.39:0/367762564 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bb174b32d0 0x55bb17513aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:47.104 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:47.089+0000 7fefceffd700 0 client.17685 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T19:02:47.105 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:47.089+0000 7ff0019bbb80 1 --2- 172.21.15.39:0/367762564 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bb174ad580 0x55bb174bc380 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:47.106 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:47.089+0000 7ff0019bbb80 1 -- 172.21.15.39:0/367762564 >> 172.21.15.39:0/367762564 conn(0x55bb174a2be0 msgr2=0x55bb174b22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:02:47.106 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:47.090+0000 7ff0019bbb80 1 -- 172.21.15.39:0/367762564 shutdown_connections 2023-07-21T19:02:47.106 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:47.090+0000 7ff0019bbb80 1 -- 172.21.15.39:0/367762564 wait complete. 2023-07-21T19:02:47.106 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:47.090+0000 7ff0019bbb80 20 client.17685 trim_cache size 0 max 16384 2023-07-21T19:02:47.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T19:02:47.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T19:02:47.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T19:02:47.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T19:02:47.109 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:47.090+0000 7feff5835700 1 -- 172.21.15.39:0/2243896190 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fefd80a12c0 msgr2=0x7fefd80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:02:47.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:47.090+0000 7feff5835700 1 --2- 172.21.15.39:0/2243896190 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fefd80a12c0 0x7fefd80a3770 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7feff0069500 tx=0x7fefe4005fb0 comp rx=0 tx=0).stop 2023-07-21T19:02:47.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:47.090+0000 7feff5835700 1 -- 172.21.15.39:0/2243896190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feff00fea30 msgr2=0x7feff0196bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:02:47.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:47.090+0000 7feff5835700 1 --2- 172.21.15.39:0/2243896190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feff00fea30 0x7feff0196bc0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fefe000e970 tx=0x7fefe000ec80 comp rx=0 tx=0).stop 2023-07-21T19:02:47.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:47.091+0000 7feff5835700 1 -- 172.21.15.39:0/2243896190 shutdown_connections 2023-07-21T19:02:47.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:47.091+0000 7feff5835700 1 --2- 172.21.15.39:0/2243896190 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fefd80a12c0 0x7fefd80a3770 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:47.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:47.091+0000 7feff5835700 1 --2- 172.21.15.39:0/2243896190 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feff0104a40 0x7feff01a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:47.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:47.091+0000 7feff5835700 1 --2- 172.21.15.39:0/2243896190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feff00fea30 0x7feff0196bc0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:47.112 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:47.091+0000 7feff5835700 1 --2- 172.21.15.39:0/2243896190 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feff00ff3e0 0x7feff0197100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:47.112 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:47.091+0000 7feff5835700 1 -- 172.21.15.39:0/2243896190 >> 172.21.15.39:0/2243896190 conn(0x7feff00fa4f0 msgr2=0x7feff01058b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:02:47.112 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:47.091+0000 7feff5835700 1 -- 172.21.15.39:0/2243896190 shutdown_connections 2023-07-21T19:02:47.112 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:02:47.091+0000 7feff5835700 1 -- 172.21.15.39:0/2243896190 wait complete. 2023-07-21T19:02:47.707 INFO:tasks.workunit.client.0.smithi039.stdout: 10 265282 29.69 MB/sec execute 360 sec latency 272.243 ms 2023-07-21T19:02:48.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:48 smithi155 ceph-mon[97934]: pgmap v668: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 14 MiB/s wr, 670 op/s 2023-07-21T19:02:48.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:48 smithi039 ceph-mon[96489]: pgmap v668: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 14 MiB/s wr, 670 op/s 2023-07-21T19:02:48.707 INFO:tasks.workunit.client.0.smithi039.stdout: 10 265666 29.65 MB/sec execute 361 sec latency 140.001 ms 2023-07-21T19:02:48.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:48 smithi193 ceph-mon[98204]: pgmap v668: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 14 MiB/s wr, 670 op/s 2023-07-21T19:02:49.707 INFO:tasks.workunit.client.0.smithi039.stdout: 10 266036 29.61 MB/sec execute 362 sec latency 144.737 ms 2023-07-21T19:02:50.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:50 smithi155 ceph-mon[97934]: pgmap v669: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.7 MiB/s wr, 386 op/s 2023-07-21T19:02:50.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:50 smithi039 ceph-mon[96489]: pgmap v669: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.7 MiB/s wr, 386 op/s 2023-07-21T19:02:50.707 INFO:tasks.workunit.client.0.smithi039.stdout: 10 266543 29.61 MB/sec execute 363 sec latency 83.375 ms 2023-07-21T19:02:50.859 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:50 smithi193 ceph-mon[98204]: pgmap v669: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.7 MiB/s wr, 386 op/s 2023-07-21T19:02:51.707 INFO:tasks.workunit.client.0.smithi039.stdout: 10 267110 29.58 MB/sec execute 364 sec latency 88.994 ms 2023-07-21T19:02:52.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:52 smithi155 ceph-mon[97934]: pgmap v670: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.7 MiB/s wr, 401 op/s 2023-07-21T19:02:52.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:52 smithi039 ceph-mon[96489]: pgmap v670: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.7 MiB/s wr, 401 op/s 2023-07-21T19:02:52.707 INFO:tasks.workunit.client.0.smithi039.stdout: 10 267837 29.61 MB/sec execute 365 sec latency 78.553 ms 2023-07-21T19:02:52.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:52 smithi193 ceph-mon[98204]: pgmap v670: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.7 MiB/s wr, 401 op/s 2023-07-21T19:02:53.707 INFO:tasks.workunit.client.0.smithi039.stdout: 10 268555 29.67 MB/sec execute 366 sec latency 74.812 ms 2023-07-21T19:02:54.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:54 smithi155 ceph-mon[97934]: pgmap v671: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 9.4 MiB/s wr, 588 op/s 2023-07-21T19:02:54.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:54 smithi039 ceph-mon[96489]: pgmap v671: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 9.4 MiB/s wr, 588 op/s 2023-07-21T19:02:54.707 INFO:tasks.workunit.client.0.smithi039.stdout: 10 269081 29.67 MB/sec execute 367 sec latency 69.824 ms 2023-07-21T19:02:54.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:54 smithi193 ceph-mon[98204]: pgmap v671: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 9.4 MiB/s wr, 588 op/s 2023-07-21T19:02:55.707 INFO:tasks.workunit.client.0.smithi039.stdout: 10 269492 29.64 MB/sec execute 368 sec latency 108.862 ms 2023-07-21T19:02:56.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:56 smithi039 ceph-mon[96489]: pgmap v672: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 6.4 MiB/s wr, 426 op/s 2023-07-21T19:02:56.707 INFO:tasks.workunit.client.0.smithi039.stdout: 10 269979 29.63 MB/sec execute 369 sec latency 86.168 ms 2023-07-21T19:02:56.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:56 smithi193 ceph-mon[98204]: pgmap v672: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 6.4 MiB/s wr, 426 op/s 2023-07-21T19:02:56.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:56 smithi155 ceph-mon[97934]: pgmap v672: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 6.4 MiB/s wr, 426 op/s 2023-07-21T19:02:57.707 INFO:tasks.workunit.client.0.smithi039.stdout: 10 270509 29.61 MB/sec execute 370 sec latency 139.175 ms 2023-07-21T19:02:58.678 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:02:58 smithi039 ceph-mon[96489]: pgmap v673: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 9.7 MiB/s wr, 611 op/s 2023-07-21T19:02:58.707 INFO:tasks.workunit.client.0.smithi039.stdout: 10 271060 29.60 MB/sec execute 371 sec latency 82.603 ms 2023-07-21T19:02:58.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:02:58 smithi193 ceph-mon[98204]: pgmap v673: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 9.7 MiB/s wr, 611 op/s 2023-07-21T19:02:58.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:02:58 smithi155 ceph-mon[97934]: pgmap v673: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 9.7 MiB/s wr, 611 op/s 2023-07-21T19:02:59.707 INFO:tasks.workunit.client.0.smithi039.stdout: 10 271703 29.63 MB/sec execute 372 sec latency 86.553 ms 2023-07-21T19:03:00.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:03:00 smithi039 ceph-mon[96489]: pgmap v674: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.4 MiB/s wr, 427 op/s 2023-07-21T19:03:00.707 INFO:tasks.workunit.client.0.smithi039.stdout: 10 271952 29.60 MB/sec execute 373 sec latency 389.842 ms 2023-07-21T19:03:00.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:03:00 smithi193 ceph-mon[98204]: pgmap v674: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.4 MiB/s wr, 427 op/s 2023-07-21T19:03:00.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:03:00 smithi155 ceph-mon[97934]: pgmap v674: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.4 MiB/s wr, 427 op/s 2023-07-21T19:03:01.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.002+0000 7f08379a8700 1 -- 172.21.15.39:0/3683137466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08300fea30 msgr2=0x7f08300feea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:03:01.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.002+0000 7f08379a8700 1 --2- 172.21.15.39:0/3683137466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08300fea30 0x7f08300feea0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f0820009990 tx=0x7f0820009ca0 comp rx=0 tx=0).stop 2023-07-21T19:03:01.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.002+0000 7f08379a8700 1 -- 172.21.15.39:0/3683137466 shutdown_connections 2023-07-21T19:03:01.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.002+0000 7f08379a8700 1 --2- 172.21.15.39:0/3683137466 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08300ff3e0 0x7f083010b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:01.013 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.002+0000 7f08379a8700 1 --2- 172.21.15.39:0/3683137466 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0830104a30 0x7f0830104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:01.013 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.002+0000 7f08379a8700 1 --2- 172.21.15.39:0/3683137466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08300fea30 0x7f08300feea0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:01.013 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.002+0000 7f08379a8700 1 -- 172.21.15.39:0/3683137466 >> 172.21.15.39:0/3683137466 conn(0x7f0830074760 msgr2=0x7f0830074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:03:01.013 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.002+0000 7f08379a8700 1 -- 172.21.15.39:0/3683137466 shutdown_connections 2023-07-21T19:03:01.013 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.002+0000 7f08379a8700 1 -- 172.21.15.39:0/3683137466 wait complete. 2023-07-21T19:03:01.015 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.003+0000 7f08379a8700 1 Processor -- start 2023-07-21T19:03:01.015 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.003+0000 7f08379a8700 1 -- start start 2023-07-21T19:03:01.015 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.003+0000 7f08379a8700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08300fea30 0x7f083019a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:01.015 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.003+0000 7f08379a8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08300ff3e0 0x7f083019af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:01.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.003+0000 7f08379a8700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0830104a30 0x7f0830194850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:01.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.003+0000 7f08379a8700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0830194ec0 con 0x7f08300ff3e0 2023-07-21T19:03:01.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.003+0000 7f08379a8700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0830195030 con 0x7f0830104a30 2023-07-21T19:03:01.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.003+0000 7f08379a8700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08301951f0 con 0x7f08300fea30 2023-07-21T19:03:01.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.003+0000 7f0834f43700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08300ff3e0 0x7f083019af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:01.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.003+0000 7f0834f43700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08300ff3e0 0x7f083019af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34454/0 (socket says 172.21.15.39:34454) 2023-07-21T19:03:01.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.003+0000 7f0834f43700 1 -- 172.21.15.39:0/3046661418 learned_addr learned my addr 172.21.15.39:0/3046661418 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:03:01.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.003+0000 7f0835f45700 1 --2- 172.21.15.39:0/3046661418 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0830104a30 0x7f0830194850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:01.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.003+0000 7f0835744700 1 --2- 172.21.15.39:0/3046661418 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08300fea30 0x7f083019a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:01.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.003+0000 7f0834f43700 1 -- 172.21.15.39:0/3046661418 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08300fea30 msgr2=0x7f083019a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:03:01.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.003+0000 7f0834f43700 1 --2- 172.21.15.39:0/3046661418 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08300fea30 0x7f083019a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:01.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.003+0000 7f0834f43700 1 -- 172.21.15.39:0/3046661418 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0830104a30 msgr2=0x7f0830194850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:03:01.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.003+0000 7f0834f43700 1 --2- 172.21.15.39:0/3046661418 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0830104a30 0x7f0830194850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:01.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.003+0000 7f0834f43700 1 -- 172.21.15.39:0/3046661418 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08200095f0 con 0x7f08300ff3e0 2023-07-21T19:03:01.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.003+0000 7f0834f43700 1 --2- 172.21.15.39:0/3046661418 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08300ff3e0 0x7f083019af30 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f0820000c00 tx=0x7f082001a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:03:01.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.004+0000 7f08267fc700 1 -- 172.21.15.39:0/3046661418 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f082001f070 con 0x7f08300ff3e0 2023-07-21T19:03:01.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.004+0000 7f08379a8700 1 -- 172.21.15.39:0/3046661418 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0830195470 con 0x7f08300ff3e0 2023-07-21T19:03:01.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.004+0000 7f08267fc700 1 -- 172.21.15.39:0/3046661418 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f082000f460 con 0x7f08300ff3e0 2023-07-21T19:03:01.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.004+0000 7f08267fc700 1 -- 172.21.15.39:0/3046661418 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0820025680 con 0x7f08300ff3e0 2023-07-21T19:03:01.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.004+0000 7f08379a8700 1 -- 172.21.15.39:0/3046661418 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0830067ae0 con 0x7f08300ff3e0 2023-07-21T19:03:01.024 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.006+0000 7f08267fc700 1 -- 172.21.15.39:0/3046661418 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f0820023070 con 0x7f08300ff3e0 2023-07-21T19:03:01.024 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.006+0000 7f08267fc700 1 --2- 172.21.15.39:0/3046661418 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f081c0a12c0 0x7f081c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.006+0000 7f08267fc700 1 -- 172.21.15.39:0/3046661418 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f08200cff50 con 0x7f08300ff3e0 2023-07-21T19:03:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.006+0000 7f0835744700 1 --2- 172.21.15.39:0/3046661418 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f081c0a12c0 0x7f081c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.007+0000 7f0835744700 1 --2- 172.21.15.39:0/3046661418 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f081c0a12c0 0x7f081c0a3770 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f082c005fd0 tx=0x7f082c005e50 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:03:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.007+0000 7f0843b2eb80 1 Processor -- start 2023-07-21T19:03:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.007+0000 7f0843b2eb80 1 -- start start 2023-07-21T19:03:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.008+0000 7f0843b2eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597ed8696b0 0x5597eda11640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.008+0000 7f0843b2eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5597eda11b80 0x5597eda15dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:01.026 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.008+0000 7f0843b2eb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5597eda11ff0 0x5597eda16300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:01.026 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.008+0000 7f0843b2eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597ed912c50 con 0x5597eda11b80 2023-07-21T19:03:01.026 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.008+0000 7f0843b2eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597ed9659e0 con 0x5597ed8696b0 2023-07-21T19:03:01.026 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.008+0000 7f0843b2eb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597ed95ee70 con 0x5597eda11ff0 2023-07-21T19:03:01.026 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.008+0000 7f0834f43700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5597eda11b80 0x5597eda15dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:01.026 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.008+0000 7f0834f43700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5597eda11b80 0x5597eda15dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34464/0 (socket says 172.21.15.39:34464) 2023-07-21T19:03:01.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.008+0000 7f0834f43700 1 -- 172.21.15.39:0/60357328 learned_addr learned my addr 172.21.15.39:0/60357328 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:03:01.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.008+0000 7f0835f45700 1 --2- 172.21.15.39:0/60357328 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5597eda11ff0 0x5597eda16300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:01.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.008+0000 7f0835744700 1 --2- 172.21.15.39:0/60357328 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597ed8696b0 0x5597eda11640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:01.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.008+0000 7f0834f43700 1 -- 172.21.15.39:0/60357328 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5597eda11ff0 msgr2=0x5597eda16300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:03:01.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.008+0000 7f0834f43700 1 --2- 172.21.15.39:0/60357328 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5597eda11ff0 0x5597eda16300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:01.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.008+0000 7f0834f43700 1 -- 172.21.15.39:0/60357328 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597ed8696b0 msgr2=0x5597eda11640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:03:01.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.008+0000 7f0834f43700 1 --2- 172.21.15.39:0/60357328 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597ed8696b0 0x5597eda11640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:01.028 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.008+0000 7f0834f43700 1 -- 172.21.15.39:0/60357328 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5597ed78cd80 con 0x5597eda11b80 2023-07-21T19:03:01.028 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.008+0000 7f0834f43700 1 --2- 172.21.15.39:0/60357328 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5597eda11b80 0x5597eda15dc0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f08200a4f40 tx=0x7f082009ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23d2c0a0337e364b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:03:01.028 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.008+0000 7f081b7fe700 1 -- 172.21.15.39:0/60357328 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08200a10e0 con 0x5597eda11b80 2023-07-21T19:03:01.028 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.009+0000 7f081b7fe700 1 -- 172.21.15.39:0/60357328 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08200a5470 con 0x5597eda11b80 2023-07-21T19:03:01.028 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.009+0000 7f081b7fe700 1 -- 172.21.15.39:0/60357328 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f082009d530 con 0x5597eda11b80 2023-07-21T19:03:01.028 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.009+0000 7f0843b2eb80 1 -- 172.21.15.39:0/60357328 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5597eda11b80 msgr2=0x5597eda15dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:03:01.028 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.009+0000 7f0843b2eb80 1 --2- 172.21.15.39:0/60357328 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5597eda11b80 0x5597eda15dc0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f08200a4f40 tx=0x7f082009ca90 comp rx=0 tx=0).stop 2023-07-21T19:03:01.028 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.009+0000 7f0843b2eb80 1 -- 172.21.15.39:0/60357328 shutdown_connections 2023-07-21T19:03:01.029 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.009+0000 7f0843b2eb80 1 --2- 172.21.15.39:0/60357328 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597ed8696b0 0x5597eda11640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:01.029 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.009+0000 7f0843b2eb80 1 --2- 172.21.15.39:0/60357328 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5597eda11ff0 0x5597eda16300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:01.029 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.009+0000 7f0843b2eb80 1 --2- 172.21.15.39:0/60357328 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5597eda11b80 0x5597eda15dc0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:01.029 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.009+0000 7f0843b2eb80 1 -- 172.21.15.39:0/60357328 >> 172.21.15.39:0/60357328 conn(0x5597eda02be0 msgr2=0x5597eda07f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:03:01.029 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.009+0000 7f0843b2eb80 1 -- 172.21.15.39:0/60357328 shutdown_connections 2023-07-21T19:03:01.029 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.009+0000 7f0843b2eb80 1 -- 172.21.15.39:0/60357328 wait complete. 2023-07-21T19:03:01.029 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.010+0000 7f0843b2eb80 1 -- start start 2023-07-21T19:03:01.030 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.010+0000 7f0843b2eb80 1 Processor -- start 2023-07-21T19:03:01.030 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.010+0000 7f0843b2eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597ed8696b0 0x5597eda20770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:01.030 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.010+0000 7f0843b2eb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5597eda11b80 0x5597eda20cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:01.030 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.011+0000 7f0835744700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597ed8696b0 0x5597eda20770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:01.030 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.011+0000 7f0843b2eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5597eda11ff0 0x5597eda1a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:01.030 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.011+0000 7f0835744700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597ed8696b0 0x5597eda20770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:47062/0 (socket says 172.21.15.39:47062) 2023-07-21T19:03:01.030 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.011+0000 7f0835744700 1 -- 172.21.15.39:0/3204571080 learned_addr learned my addr 172.21.15.39:0/3204571080 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:03:01.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.011+0000 7f0843b2eb80 1 -- 172.21.15.39:0/3204571080 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597ed8fea60 con 0x5597eda11ff0 2023-07-21T19:03:01.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.011+0000 7f0843b2eb80 1 -- 172.21.15.39:0/3204571080 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597ed8fe760 con 0x5597ed8696b0 2023-07-21T19:03:01.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.011+0000 7f0843b2eb80 1 -- 172.21.15.39:0/3204571080 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597ed7e28c0 con 0x5597eda11b80 2023-07-21T19:03:01.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.011+0000 7f0843b2eb80 10 client.0 _ll_register_callbacks cb 0x5597ed8fc700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:03:01.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.011+0000 7f0835f45700 1 --2- 172.21.15.39:0/3204571080 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5597eda11ff0 0x5597eda1a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:01.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.011+0000 7f0834f43700 1 --2- 172.21.15.39:0/3204571080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5597eda11b80 0x5597eda20cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:01.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.011+0000 7f0835744700 1 -- 172.21.15.39:0/3204571080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5597eda11b80 msgr2=0x5597eda20cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:03:01.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.011+0000 7f0835744700 1 --2- 172.21.15.39:0/3204571080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5597eda11b80 0x5597eda20cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:01.032 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.011+0000 7f0835744700 1 -- 172.21.15.39:0/3204571080 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5597eda11ff0 msgr2=0x5597eda1a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:03:01.032 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.011+0000 7f0835744700 1 --2- 172.21.15.39:0/3204571080 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5597eda11ff0 0x5597eda1a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:01.032 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.011+0000 7f0835744700 1 -- 172.21.15.39:0/3204571080 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0830067ae0 con 0x5597ed8696b0 2023-07-21T19:03:01.032 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.011+0000 7f0834f43700 1 --2- 172.21.15.39:0/3204571080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5597eda11b80 0x5597eda20cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:03:01.032 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.011+0000 7f0835744700 1 --2- 172.21.15.39:0/3204571080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597ed8696b0 0x5597eda20770 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f082c0109e0 tx=0x7f082c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12c84d67ed82ecfb server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:03:01.032 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.011+0000 7f0818ff9700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-21T19:03:01.032 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.011+0000 7f0818ff9700 1 -- 172.21.15.39:0/3204571080 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f082c004a30 con 0x5597ed8696b0 2023-07-21T19:03:01.033 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.011+0000 7f0818ff9700 1 -- 172.21.15.39:0/3204571080 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f082c004b90 con 0x5597ed8696b0 2023-07-21T19:03:01.033 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.011+0000 7f0843b2eb80 1 -- 172.21.15.39:0/3204571080 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5597ed8b8490 con 0x5597ed8696b0 2023-07-21T19:03:01.033 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.012+0000 7f0818ff9700 1 -- 172.21.15.39:0/3204571080 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) v2 ==== 24+0+0 (secure 0 0 0) 0x7f082c004d50 con 0x5597ed8696b0 2023-07-21T19:03:01.033 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.012+0000 7f0843b2eb80 10 client.27116 fetch_fsmap learned FSMap version 103 2023-07-21T19:03:01.033 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.012+0000 7f0843b2eb80 1 -- 172.21.15.39:0/3204571080 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=4+}) v3 -- 0x5597ed775c00 con 0x5597ed8696b0 2023-07-21T19:03:01.033 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.012+0000 7f0818ff9700 1 -- 172.21.15.39:0/3204571080 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f082c032080 con 0x5597ed8696b0 2023-07-21T19:03:01.034 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.012+0000 7f0818ff9700 1 -- 172.21.15.39:0/3204571080 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 103) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f082c02c030 con 0x5597ed8696b0 2023-07-21T19:03:01.034 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.012+0000 7f0843b2eb80 10 client.27116 fetch_fsmap finished waiting for FSMap version 103 2023-07-21T19:03:01.034 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.012+0000 7f0843b2eb80 10 client.27116 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:03:01.034 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.012+0000 7f0843b2eb80 20 client.27116 populate_metadata read hostname 'smithi039' 2023-07-21T19:03:01.035 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.012+0000 7f0843b2eb80 1 --2- 172.21.15.39:0/3204571080 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5597eda2acd0 0x5597eda2d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:01.035 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.012+0000 7f0843b2eb80 4 client.27116 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:03:01.035 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.012+0000 7f0843b2eb80 1 -- 172.21.15.39:0/3204571080 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5597ed957e50 con 0x5597eda2acd0 2023-07-21T19:03:01.035 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.012+0000 7f0834f43700 1 --2- 172.21.15.39:0/3204571080 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5597eda2acd0 0x5597eda2d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:01.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.013+0000 7f0834f43700 1 --2- 172.21.15.39:0/3204571080 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5597eda2acd0 0x5597eda2d0e0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6539ec9e66d4efbf server_cookie=ff1e3c37b791e803 in_seq=0 out_seq=0 2023-07-21T19:03:01.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.013+0000 7f0818ff9700 10 client.27116 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T19:03:01.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.013+0000 7f0818ff9700 1 -- 172.21.15.39:0/3204571080 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5597ed957e50 con 0x5597eda2acd0 2023-07-21T19:03:01.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.013+0000 7f0818ff9700 10 client.27116 handle_command_reply: tid=0 2023-07-21T19:03:01.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.013+0000 7f0843b2eb80 1 client.27116 shutdown 2023-07-21T19:03:01.037 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.013+0000 7f0843b2eb80 1 -- 172.21.15.39:0/3204571080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597ed8696b0 msgr2=0x5597eda20770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:03:01.037 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.013+0000 7f0843b2eb80 1 --2- 172.21.15.39:0/3204571080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597ed8696b0 0x5597eda20770 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f082c0109e0 tx=0x7f082c004750 comp rx=0 tx=0).stop 2023-07-21T19:03:01.037 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.014+0000 7f0843b2eb80 1 -- 172.21.15.39:0/3204571080 shutdown_connections 2023-07-21T19:03:01.037 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.014+0000 7f0843b2eb80 1 --2- 172.21.15.39:0/3204571080 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5597eda2acd0 0x5597eda2d0e0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:01.038 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.014+0000 7f0843b2eb80 1 --2- 172.21.15.39:0/3204571080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597ed8696b0 0x5597eda20770 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:01.038 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.014+0000 7f0818ff9700 0 client.27116 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T19:03:01.038 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.014+0000 7f0843b2eb80 1 --2- 172.21.15.39:0/3204571080 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5597eda11ff0 0x5597eda1a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:01.038 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.014+0000 7f0843b2eb80 1 --2- 172.21.15.39:0/3204571080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5597eda11b80 0x5597eda20cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:01.038 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.014+0000 7f0843b2eb80 1 -- 172.21.15.39:0/3204571080 >> 172.21.15.39:0/3204571080 conn(0x5597eda02be0 msgr2=0x5597eda074e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:03:01.039 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.014+0000 7f0843b2eb80 1 -- 172.21.15.39:0/3204571080 shutdown_connections 2023-07-21T19:03:01.039 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.014+0000 7f0843b2eb80 1 -- 172.21.15.39:0/3204571080 wait complete. 2023-07-21T19:03:01.039 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.014+0000 7f0843b2eb80 20 client.27116 trim_cache size 0 max 16384 2023-07-21T19:03:01.045 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.029+0000 7f0843b2eb80 1 Processor -- start 2023-07-21T19:03:01.045 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.029+0000 7f0843b2eb80 1 -- start start 2023-07-21T19:03:01.045 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.029+0000 7f0843b2eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597ed8696b0 0x5597eda13bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:01.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.029+0000 7f0843b2eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5597eda11b80 0x5597eda15cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:01.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.029+0000 7f0843b2eb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5597eda11ff0 0x5597eda16210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:01.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.029+0000 7f0843b2eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597ed9ffc00 con 0x5597eda11b80 2023-07-21T19:03:01.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.029+0000 7f0843b2eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597ed9d3060 con 0x5597ed8696b0 2023-07-21T19:03:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.029+0000 7f0843b2eb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597eda04d00 con 0x5597eda11ff0 2023-07-21T19:03:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.029+0000 7f0834f43700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5597eda11b80 0x5597eda15cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.029+0000 7f0834f43700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5597eda11b80 0x5597eda15cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34476/0 (socket says 172.21.15.39:34476) 2023-07-21T19:03:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.029+0000 7f0834f43700 1 -- 172.21.15.39:0/715158732 learned_addr learned my addr 172.21.15.39:0/715158732 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:03:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.029+0000 7f0835744700 1 --2- 172.21.15.39:0/715158732 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597ed8696b0 0x5597eda13bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.030+0000 7f0834f43700 1 -- 172.21.15.39:0/715158732 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5597eda11ff0 msgr2=0x5597eda16210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:03:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.030+0000 7f0835f45700 1 --2- 172.21.15.39:0/715158732 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5597eda11ff0 0x5597eda16210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:01.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.030+0000 7f0834f43700 1 --2- 172.21.15.39:0/715158732 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5597eda11ff0 0x5597eda16210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:01.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.030+0000 7f0834f43700 1 -- 172.21.15.39:0/715158732 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597ed8696b0 msgr2=0x5597eda13bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:03:01.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.030+0000 7f0834f43700 1 --2- 172.21.15.39:0/715158732 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597ed8696b0 0x5597eda13bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:01.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.030+0000 7f0834f43700 1 -- 172.21.15.39:0/715158732 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5597ed7b31f0 con 0x5597eda11b80 2023-07-21T19:03:01.049 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.030+0000 7f0834f43700 1 --2- 172.21.15.39:0/715158732 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5597eda11b80 0x5597eda15cd0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f0820009350 tx=0x7f082000fc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8aea1dc7f520dfd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:03:01.049 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.030+0000 7f081bfff700 1 -- 172.21.15.39:0/715158732 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f082009b270 con 0x5597eda11b80 2023-07-21T19:03:01.049 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.030+0000 7f081bfff700 1 -- 172.21.15.39:0/715158732 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f082000f8b0 con 0x5597eda11b80 2023-07-21T19:03:01.049 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.030+0000 7f081bfff700 1 -- 172.21.15.39:0/715158732 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f082009d570 con 0x5597eda11b80 2023-07-21T19:03:01.049 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.030+0000 7f0843b2eb80 1 -- 172.21.15.39:0/715158732 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5597eda11b80 msgr2=0x5597eda15cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:03:01.049 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.030+0000 7f0843b2eb80 1 --2- 172.21.15.39:0/715158732 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5597eda11b80 0x5597eda15cd0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f0820009350 tx=0x7f082000fc50 comp rx=0 tx=0).stop 2023-07-21T19:03:01.049 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.031+0000 7f0843b2eb80 1 -- 172.21.15.39:0/715158732 shutdown_connections 2023-07-21T19:03:01.050 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.031+0000 7f0843b2eb80 1 --2- 172.21.15.39:0/715158732 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597ed8696b0 0x5597eda13bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:01.050 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.031+0000 7f0843b2eb80 1 --2- 172.21.15.39:0/715158732 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5597eda11ff0 0x5597eda16210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:01.050 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.031+0000 7f0843b2eb80 1 --2- 172.21.15.39:0/715158732 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5597eda11b80 0x5597eda15cd0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:01.050 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.031+0000 7f0843b2eb80 1 -- 172.21.15.39:0/715158732 >> 172.21.15.39:0/715158732 conn(0x5597eda02be0 msgr2=0x5597eda155c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:03:01.050 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.031+0000 7f0843b2eb80 1 -- 172.21.15.39:0/715158732 shutdown_connections 2023-07-21T19:03:01.050 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.031+0000 7f0843b2eb80 1 -- 172.21.15.39:0/715158732 wait complete. 2023-07-21T19:03:01.050 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.031+0000 7f0843b2eb80 1 -- start start 2023-07-21T19:03:01.050 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.031+0000 7f0843b2eb80 1 Processor -- start 2023-07-21T19:03:01.051 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.032+0000 7f0843b2eb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5597ed8696b0 0x5597eda1e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:01.051 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.032+0000 7f0843b2eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597eda11b80 0x5597eda1ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:01.051 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.032+0000 7f0843b2eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5597eda11ff0 0x5597eda7dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:01.051 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.032+0000 7f0843b2eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597eda7e0f0 con 0x5597eda11ff0 2023-07-21T19:03:01.051 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.032+0000 7f0843b2eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597eda7e230 con 0x5597eda11b80 2023-07-21T19:03:01.051 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.032+0000 7f0843b2eb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597eda7e3f0 con 0x5597ed8696b0 2023-07-21T19:03:01.051 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.032+0000 7f0843b2eb80 10 client.0 _ll_register_callbacks cb 0x7f082c00ee50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:03:01.051 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.032+0000 7f0835f45700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5597eda11ff0 0x5597eda7dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:01.052 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.032+0000 7f0835f45700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5597eda11ff0 0x5597eda7dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34484/0 (socket says 172.21.15.39:34484) 2023-07-21T19:03:01.052 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.032+0000 7f0835f45700 1 -- 172.21.15.39:0/1245152879 learned_addr learned my addr 172.21.15.39:0/1245152879 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:03:01.053 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.032+0000 7f0834f43700 1 --2- 172.21.15.39:0/1245152879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597eda11b80 0x5597eda1ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:01.053 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.032+0000 7f0835744700 1 --2- 172.21.15.39:0/1245152879 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5597ed8696b0 0x5597eda1e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:01.053 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.032+0000 7f0834f43700 1 -- 172.21.15.39:0/1245152879 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5597ed8696b0 msgr2=0x5597eda1e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:03:01.053 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.032+0000 7f0834f43700 1 --2- 172.21.15.39:0/1245152879 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5597ed8696b0 0x5597eda1e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:01.053 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.032+0000 7f0834f43700 1 -- 172.21.15.39:0/1245152879 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5597eda11ff0 msgr2=0x5597eda7dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:03:01.054 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.032+0000 7f0834f43700 1 --2- 172.21.15.39:0/1245152879 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5597eda11ff0 0x5597eda7dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:01.054 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.032+0000 7f0834f43700 1 -- 172.21.15.39:0/1245152879 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5597ed78cd80 con 0x5597eda11b80 2023-07-21T19:03:01.054 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.032+0000 7f0835744700 1 --2- 172.21.15.39:0/1245152879 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5597ed8696b0 0x5597eda1e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:03:01.054 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.032+0000 7f0835f45700 1 --2- 172.21.15.39:0/1245152879 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5597eda11ff0 0x5597eda7dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:03:01.054 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.032+0000 7f0834f43700 1 --2- 172.21.15.39:0/1245152879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597eda11b80 0x5597eda1ee20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f08200098e0 tx=0x7f082009c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3c61952f63c244c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:03:01.054 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.032+0000 7f0818ff9700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-21T19:03:01.054 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.032+0000 7f0818ff9700 1 -- 172.21.15.39:0/1245152879 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f082009b270 con 0x5597eda11b80 2023-07-21T19:03:01.055 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.032+0000 7f0843b2eb80 1 -- 172.21.15.39:0/1245152879 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f08200a5470 con 0x5597eda11b80 2023-07-21T19:03:01.055 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.032+0000 7f0818ff9700 1 -- 172.21.15.39:0/1245152879 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f082009c5a0 con 0x5597eda11b80 2023-07-21T19:03:01.055 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.033+0000 7f0818ff9700 1 -- 172.21.15.39:0/1245152879 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) v2 ==== 24+0+0 (secure 0 0 0) 0x7f082009d310 con 0x5597eda11b80 2023-07-21T19:03:01.055 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.033+0000 7f0843b2eb80 10 client.27122 fetch_fsmap learned FSMap version 103 2023-07-21T19:03:01.055 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.033+0000 7f0843b2eb80 1 -- 172.21.15.39:0/1245152879 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=4+}) v3 -- 0x5597ed78b250 con 0x5597eda11b80 2023-07-21T19:03:01.055 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.033+0000 7f0818ff9700 1 -- 172.21.15.39:0/1245152879 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08200a5470 con 0x5597eda11b80 2023-07-21T19:03:01.055 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.033+0000 7f0818ff9700 1 -- 172.21.15.39:0/1245152879 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 103) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f08200cd920 con 0x5597eda11b80 2023-07-21T19:03:01.056 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.033+0000 7f0843b2eb80 10 client.27122 fetch_fsmap finished waiting for FSMap version 103 2023-07-21T19:03:01.056 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.033+0000 7f0843b2eb80 10 client.27122 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:03:01.056 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.033+0000 7f0843b2eb80 20 client.27122 populate_metadata read hostname 'smithi039' 2023-07-21T19:03:01.056 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.033+0000 7f0843b2eb80 1 --2- 172.21.15.39:0/1245152879 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5597eda7f650 0x5597eda81a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:01.056 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.033+0000 7f0843b2eb80 4 client.27122 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:03:01.056 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.033+0000 7f0843b2eb80 1 -- 172.21.15.39:0/1245152879 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5597ed919ff0 con 0x5597eda7f650 2023-07-21T19:03:01.056 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.033+0000 7f0835744700 1 --2- 172.21.15.39:0/1245152879 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5597eda7f650 0x5597eda81a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:01.056 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.034+0000 7f0818ff9700 10 client.27122 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T19:03:01.057 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.034+0000 7f0835744700 1 --2- 172.21.15.39:0/1245152879 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5597eda7f650 0x5597eda81a30 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53b0815c0338d509 server_cookie=b1d5a506bb370ca1 in_seq=0 out_seq=0 2023-07-21T19:03:01.235 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.225+0000 7f0818ff9700 1 -- 172.21.15.39:0/1245152879 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5597ed919ff0 con 0x5597eda7f650 2023-07-21T19:03:01.236 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.225+0000 7f0818ff9700 10 client.27122 handle_command_reply: tid=0 2023-07-21T19:03:01.236 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.225+0000 7f0843b2eb80 1 client.27122 shutdown 2023-07-21T19:03:01.236 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.225+0000 7f0843b2eb80 1 -- 172.21.15.39:0/1245152879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597eda11b80 msgr2=0x5597eda1ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:03:01.236 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.225+0000 7f0843b2eb80 1 --2- 172.21.15.39:0/1245152879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597eda11b80 0x5597eda1ee20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f08200098e0 tx=0x7f082009c830 comp rx=0 tx=0).stop 2023-07-21T19:03:01.236 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.226+0000 7f0843b2eb80 1 -- 172.21.15.39:0/1245152879 shutdown_connections 2023-07-21T19:03:01.236 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.226+0000 7f0843b2eb80 1 --2- 172.21.15.39:0/1245152879 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5597eda7f650 0x5597eda81a30 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:01.236 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.226+0000 7f0843b2eb80 1 --2- 172.21.15.39:0/1245152879 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5597eda11ff0 0x5597eda7dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:01.237 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.226+0000 7f0843b2eb80 1 --2- 172.21.15.39:0/1245152879 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5597ed8696b0 0x5597eda1e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:01.237 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.226+0000 7f0843b2eb80 1 --2- 172.21.15.39:0/1245152879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597eda11b80 0x5597eda1ee20 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:01.237 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.226+0000 7f0843b2eb80 1 -- 172.21.15.39:0/1245152879 >> 172.21.15.39:0/1245152879 conn(0x5597eda02be0 msgr2=0x5597eda15e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:03:01.237 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.226+0000 7f0818ff9700 0 client.27122 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T19:03:01.237 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.226+0000 7f0843b2eb80 1 -- 172.21.15.39:0/1245152879 shutdown_connections 2023-07-21T19:03:01.237 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.226+0000 7f0843b2eb80 1 -- 172.21.15.39:0/1245152879 wait complete. 2023-07-21T19:03:01.238 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.228+0000 7f0843b2eb80 20 client.27122 trim_cache size 0 max 16384 2023-07-21T19:03:01.238 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T19:03:01.238 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T19:03:01.239 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T19:03:01.239 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T19:03:01.239 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.229+0000 7f08379a8700 1 -- 172.21.15.39:0/3046661418 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f081c0a12c0 msgr2=0x7f081c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:03:01.240 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.229+0000 7f08379a8700 1 --2- 172.21.15.39:0/3046661418 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f081c0a12c0 0x7f081c0a3770 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f082c005fd0 tx=0x7f082c005e50 comp rx=0 tx=0).stop 2023-07-21T19:03:01.240 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.229+0000 7f08379a8700 1 -- 172.21.15.39:0/3046661418 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08300ff3e0 msgr2=0x7f083019af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:03:01.240 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.229+0000 7f08379a8700 1 --2- 172.21.15.39:0/3046661418 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08300ff3e0 0x7f083019af30 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f0820000c00 tx=0x7f082001a890 comp rx=0 tx=0).stop 2023-07-21T19:03:01.240 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.230+0000 7f08379a8700 1 -- 172.21.15.39:0/3046661418 shutdown_connections 2023-07-21T19:03:01.240 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.230+0000 7f08379a8700 1 --2- 172.21.15.39:0/3046661418 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f081c0a12c0 0x7f081c0a3770 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:01.240 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.230+0000 7f08379a8700 1 --2- 172.21.15.39:0/3046661418 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0830104a30 0x7f0830194850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:01.240 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.230+0000 7f08379a8700 1 --2- 172.21.15.39:0/3046661418 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08300fea30 0x7f083019a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:01.241 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.230+0000 7f08379a8700 1 --2- 172.21.15.39:0/3046661418 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08300ff3e0 0x7f083019af30 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:01.241 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.230+0000 7f08379a8700 1 -- 172.21.15.39:0/3046661418 >> 172.21.15.39:0/3046661418 conn(0x7f0830074760 msgr2=0x7f08301058a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:03:01.241 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.230+0000 7f08379a8700 1 -- 172.21.15.39:0/3046661418 shutdown_connections 2023-07-21T19:03:01.241 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:01.230+0000 7f08379a8700 1 -- 172.21.15.39:0/3046661418 wait complete. 2023-07-21T19:03:01.707 INFO:tasks.workunit.client.0.smithi039.stdout: 10 272055 29.54 MB/sec execute 374 sec latency 923.781 ms 2023-07-21T19:03:02.683 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:03:02 smithi039 ceph-mon[96489]: pgmap v675: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.4 MiB/s wr, 432 op/s 2023-07-21T19:03:02.707 INFO:tasks.workunit.client.0.smithi039.stdout: 10 272057 29.46 MB/sec execute 375 sec latency 1265.636 ms 2023-07-21T19:03:02.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:03:02 smithi193 ceph-mon[98204]: pgmap v675: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.4 MiB/s wr, 432 op/s 2023-07-21T19:03:02.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:03:02 smithi155 ceph-mon[97934]: pgmap v675: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.4 MiB/s wr, 432 op/s 2023-07-21T19:03:03.707 INFO:tasks.workunit.client.0.smithi039.stdout: 10 272089 29.39 MB/sec execute 376 sec latency 2162.229 ms 2023-07-21T19:03:04.707 INFO:tasks.workunit.client.0.smithi039.stdout: 10 272407 29.34 MB/sec execute 377 sec latency 208.533 ms 2023-07-21T19:03:04.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:03:04 smithi193 ceph-mon[98204]: pgmap v676: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 9.8 MiB/s wr, 569 op/s 2023-07-21T19:03:04.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:03:04 smithi155 ceph-mon[97934]: pgmap v676: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 9.8 MiB/s wr, 569 op/s 2023-07-21T19:03:04.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:03:04 smithi039 ceph-mon[96489]: pgmap v676: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 9.8 MiB/s wr, 569 op/s 2023-07-21T19:03:05.707 INFO:tasks.workunit.client.0.smithi039.stdout: 10 272621 29.29 MB/sec execute 378 sec latency 180.354 ms 2023-07-21T19:03:06.707 INFO:tasks.workunit.client.0.smithi039.stdout: 10 273138 29.29 MB/sec execute 379 sec latency 108.521 ms 2023-07-21T19:03:06.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:03:06 smithi193 ceph-mon[98204]: pgmap v677: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 6.7 MiB/s wr, 343 op/s 2023-07-21T19:03:06.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:03:06 smithi155 ceph-mon[97934]: pgmap v677: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 6.7 MiB/s wr, 343 op/s 2023-07-21T19:03:06.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:03:06 smithi039 ceph-mon[96489]: pgmap v677: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 6.7 MiB/s wr, 343 op/s 2023-07-21T19:03:07.707 INFO:tasks.workunit.client.0.smithi039.stdout: 10 273601 29.29 MB/sec execute 380 sec latency 109.744 ms 2023-07-21T19:03:08.707 INFO:tasks.workunit.client.0.smithi039.stdout: 10 274014 29.25 MB/sec execute 381 sec latency 150.287 ms 2023-07-21T19:03:08.859 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:03:08 smithi193 ceph-mon[98204]: pgmap v678: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 21 MiB/s wr, 879 op/s 2023-07-21T19:03:08.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:03:08 smithi155 ceph-mon[97934]: pgmap v678: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 21 MiB/s wr, 879 op/s 2023-07-21T19:03:08.927 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:03:08 smithi039 ceph-mon[96489]: pgmap v678: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 21 MiB/s wr, 879 op/s 2023-07-21T19:03:09.709 INFO:tasks.workunit.client.0.smithi039.stdout: 10 274239 29.21 MB/sec execute 382 sec latency 161.783 ms 2023-07-21T19:03:09.861 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:03:09 smithi193 ceph-mon[98204]: osdmap e117: 12 total, 12 up, 12 in 2023-07-21T19:03:09.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:03:09 smithi155 ceph-mon[97934]: osdmap e117: 12 total, 12 up, 12 in 2023-07-21T19:03:09.928 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:03:09 smithi039 ceph-mon[96489]: osdmap e117: 12 total, 12 up, 12 in 2023-07-21T19:03:10.708 INFO:tasks.workunit.client.0.smithi039.stdout: 10 274549 29.17 MB/sec execute 383 sec latency 100.148 ms 2023-07-21T19:03:10.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:03:10 smithi193 ceph-mon[98204]: pgmap v680: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 21 MiB/s wr, 833 op/s 2023-07-21T19:03:10.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:03:10 smithi155 ceph-mon[97934]: pgmap v680: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 21 MiB/s wr, 833 op/s 2023-07-21T19:03:10.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:03:10 smithi039 ceph-mon[96489]: pgmap v680: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 21 MiB/s wr, 833 op/s 2023-07-21T19:03:11.708 INFO:tasks.workunit.client.0.smithi039.stdout: 10 275172 29.19 MB/sec execute 384 sec latency 189.121 ms 2023-07-21T19:03:12.607 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:03:12 smithi193 ceph-mon[98204]: pgmap v681: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 22 MiB/s wr, 863 op/s 2023-07-21T19:03:12.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:03:12 smithi155 ceph-mon[97934]: pgmap v681: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 22 MiB/s wr, 863 op/s 2023-07-21T19:03:12.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:03:12 smithi039 ceph-mon[96489]: pgmap v681: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 22 MiB/s wr, 863 op/s 2023-07-21T19:03:12.708 INFO:tasks.workunit.client.0.smithi039.stdout: 10 275763 29.22 MB/sec execute 385 sec latency 64.598 ms 2023-07-21T19:03:13.708 INFO:tasks.workunit.client.0.smithi039.stdout: 10 276359 29.20 MB/sec execute 386 sec latency 70.422 ms 2023-07-21T19:03:14.404 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.394+0000 7f2668dca700 1 -- 172.21.15.39:0/252856251 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2664104a40 msgr2=0x7f2664104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:03:14.404 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.394+0000 7f2668dca700 1 --2- 172.21.15.39:0/252856251 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2664104a40 0x7f2664104e10 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f2654009990 tx=0x7f2654009ca0 comp rx=0 tx=0).stop 2023-07-21T19:03:14.404 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.394+0000 7f2668dca700 1 -- 172.21.15.39:0/252856251 shutdown_connections 2023-07-21T19:03:14.404 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.394+0000 7f2668dca700 1 --2- 172.21.15.39:0/252856251 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26640ff3e0 0x7f266410b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:14.405 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.394+0000 7f2668dca700 1 --2- 172.21.15.39:0/252856251 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2664104a40 0x7f2664104e10 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:14.405 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.394+0000 7f2668dca700 1 --2- 172.21.15.39:0/252856251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26640fea30 0x7f26640feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:14.405 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.394+0000 7f2668dca700 1 -- 172.21.15.39:0/252856251 >> 172.21.15.39:0/252856251 conn(0x7f26640fa4f0 msgr2=0x7f26640fc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:03:14.405 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.394+0000 7f2668dca700 1 -- 172.21.15.39:0/252856251 shutdown_connections 2023-07-21T19:03:14.405 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.394+0000 7f2668dca700 1 -- 172.21.15.39:0/252856251 wait complete. 2023-07-21T19:03:14.406 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.395+0000 7f2668dca700 1 Processor -- start 2023-07-21T19:03:14.407 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.395+0000 7f2668dca700 1 -- start start 2023-07-21T19:03:14.407 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.396+0000 7f2668dca700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26640fea30 0x7f2664100820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:14.407 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.396+0000 7f2668dca700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26640ff3e0 0x7f2664100d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:14.407 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.396+0000 7f2668dca700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2664104a40 0x7f26641a3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:14.408 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.396+0000 7f2668dca700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26641a4370 con 0x7f2664104a40 2023-07-21T19:03:14.408 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.396+0000 7f2668dca700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26641a44b0 con 0x7f26640ff3e0 2023-07-21T19:03:14.408 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.396+0000 7f2668dca700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26641a4620 con 0x7f26640fea30 2023-07-21T19:03:14.408 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.396+0000 7f266259c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26640fea30 0x7f2664100820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:14.408 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.396+0000 7f266259c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26640fea30 0x7f2664100820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:53046/0 (socket says 172.21.15.39:53046) 2023-07-21T19:03:14.408 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.396+0000 7f266259c700 1 -- 172.21.15.39:0/537467647 learned_addr learned my addr 172.21.15.39:0/537467647 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:03:14.409 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.396+0000 7f2662d9d700 1 --2- 172.21.15.39:0/537467647 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2664104a40 0x7f26641a3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:14.409 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.396+0000 7f266259c700 1 -- 172.21.15.39:0/537467647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26640ff3e0 msgr2=0x7f2664100d60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-21T19:03:14.409 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.396+0000 7f266259c700 1 --2- 172.21.15.39:0/537467647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26640ff3e0 0x7f2664100d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:14.409 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.396+0000 7f266259c700 1 -- 172.21.15.39:0/537467647 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2664104a40 msgr2=0x7f26641a3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:03:14.409 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.396+0000 7f266259c700 1 --2- 172.21.15.39:0/537467647 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2664104a40 0x7f26641a3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:14.409 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.396+0000 7f266259c700 1 -- 172.21.15.39:0/537467647 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f265c009c40 con 0x7f26640fea30 2023-07-21T19:03:14.410 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.397+0000 7f266259c700 1 --2- 172.21.15.39:0/537467647 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26640fea30 0x7f2664100820 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2654000c00 tx=0x7f2654005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:03:14.410 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.397+0000 7f26537fe700 1 -- 172.21.15.39:0/537467647 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f265401d070 con 0x7f26640fea30 2023-07-21T19:03:14.410 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.397+0000 7f2668dca700 1 -- 172.21.15.39:0/537467647 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26540095f0 con 0x7f26640fea30 2023-07-21T19:03:14.410 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.397+0000 7f2668dca700 1 -- 172.21.15.39:0/537467647 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26641a4b90 con 0x7f26640fea30 2023-07-21T19:03:14.410 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.397+0000 7f26537fe700 1 -- 172.21.15.39:0/537467647 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2654021410 con 0x7f26640fea30 2023-07-21T19:03:14.410 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.397+0000 7f26537fe700 1 -- 172.21.15.39:0/537467647 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2654017650 con 0x7f26640fea30 2023-07-21T19:03:14.412 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.400+0000 7f26537fe700 1 -- 172.21.15.39:0/537467647 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f265400fb40 con 0x7f26640fea30 2023-07-21T19:03:14.412 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.401+0000 7f26537fe700 1 --2- 172.21.15.39:0/537467647 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f264c0a1340 0x7f264c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.401+0000 7f26537fe700 1 -- 172.21.15.39:0/537467647 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f26540cdfc0 con 0x7f26640fea30 2023-07-21T19:03:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.403+0000 7f2661d9b700 1 --2- 172.21.15.39:0/537467647 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f264c0a1340 0x7f264c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.403+0000 7f2661d9b700 1 --2- 172.21.15.39:0/537467647 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f264c0a1340 0x7f264c0a37f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2664102040 tx=0x7f265800b3f0 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:03:14.431 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.421+0000 7f2674f50b80 1 Processor -- start 2023-07-21T19:03:14.431 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.422+0000 7f2674f50b80 1 -- start start 2023-07-21T19:03:14.432 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.422+0000 7f2674f50b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ad45fd6b0 0x558ad47a6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:14.432 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.422+0000 7f2674f50b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ad47a72d0 0x558ad47a1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:14.433 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.422+0000 7f2674f50b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558ad47a1580 0x558ad47a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:14.433 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.422+0000 7f2674f50b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ad46a6c50 con 0x558ad47a1580 2023-07-21T19:03:14.433 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.422+0000 7f2674f50b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ad46f99e0 con 0x558ad47a72d0 2023-07-21T19:03:14.433 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.422+0000 7f2674f50b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ad46f2e70 con 0x558ad45fd6b0 2023-07-21T19:03:14.433 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.422+0000 7f2662d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558ad47a1580 0x558ad47a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:14.433 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.422+0000 7f2662d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558ad47a1580 0x558ad47a19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34504/0 (socket says 172.21.15.39:34504) 2023-07-21T19:03:14.433 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.422+0000 7f2662d9d700 1 -- 172.21.15.39:0/2060201910 learned_addr learned my addr 172.21.15.39:0/2060201910 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:03:14.434 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.422+0000 7f266259c700 1 --2- 172.21.15.39:0/2060201910 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ad45fd6b0 0x558ad47a6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:14.434 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.423+0000 7f2662d9d700 1 -- 172.21.15.39:0/2060201910 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ad45fd6b0 msgr2=0x558ad47a6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:03:14.434 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.423+0000 7f2662d9d700 1 --2- 172.21.15.39:0/2060201910 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ad45fd6b0 0x558ad47a6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:14.434 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.423+0000 7f2662d9d700 1 -- 172.21.15.39:0/2060201910 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ad47a72d0 msgr2=0x558ad47a1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:03:14.434 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.423+0000 7f2662d9d700 1 --2- 172.21.15.39:0/2060201910 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ad47a72d0 0x558ad47a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:14.434 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.423+0000 7f2662d9d700 1 -- 172.21.15.39:0/2060201910 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ad4520d80 con 0x558ad47a1580 2023-07-21T19:03:14.434 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.423+0000 7f2662d9d700 1 --2- 172.21.15.39:0/2060201910 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558ad47a1580 0x558ad47a19d0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f265c00e730 tx=0x7f265c009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cae240c6ec8fef1f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:03:14.435 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.423+0000 7f2650ff9700 1 -- 172.21.15.39:0/2060201910 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f265c00e040 con 0x558ad47a1580 2023-07-21T19:03:14.435 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.423+0000 7f2650ff9700 1 -- 172.21.15.39:0/2060201910 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f265c00f040 con 0x558ad47a1580 2023-07-21T19:03:14.435 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.423+0000 7f2650ff9700 1 -- 172.21.15.39:0/2060201910 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f265c007760 con 0x558ad47a1580 2023-07-21T19:03:14.435 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.424+0000 7f2674f50b80 1 -- 172.21.15.39:0/2060201910 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558ad47a1580 msgr2=0x558ad47a19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:03:14.436 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.424+0000 7f2674f50b80 1 --2- 172.21.15.39:0/2060201910 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558ad47a1580 0x558ad47a19d0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f265c00e730 tx=0x7f265c009920 comp rx=0 tx=0).stop 2023-07-21T19:03:14.436 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.424+0000 7f2674f50b80 1 -- 172.21.15.39:0/2060201910 shutdown_connections 2023-07-21T19:03:14.436 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.424+0000 7f2674f50b80 1 --2- 172.21.15.39:0/2060201910 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558ad47a1580 0x558ad47a19d0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:14.436 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.424+0000 7f2674f50b80 1 --2- 172.21.15.39:0/2060201910 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ad45fd6b0 0x558ad47a6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:14.436 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.424+0000 7f2674f50b80 1 --2- 172.21.15.39:0/2060201910 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ad47a72d0 0x558ad47a1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:14.436 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.424+0000 7f2674f50b80 1 -- 172.21.15.39:0/2060201910 >> 172.21.15.39:0/2060201910 conn(0x558ad4796be0 msgr2=0x558ad479bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:03:14.437 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.428+0000 7f2674f50b80 1 -- 172.21.15.39:0/2060201910 shutdown_connections 2023-07-21T19:03:14.437 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.428+0000 7f2674f50b80 1 -- 172.21.15.39:0/2060201910 wait complete. 2023-07-21T19:03:14.438 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.428+0000 7f2674f50b80 1 -- start start 2023-07-21T19:03:14.438 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.428+0000 7f2674f50b80 1 Processor -- start 2023-07-21T19:03:14.439 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.429+0000 7f2674f50b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ad45fd6b0 0x558ad47aaa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:14.439 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.429+0000 7f2674f50b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558ad47a1580 0x558ad47aaf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:14.439 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.429+0000 7f2674f50b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ad47ab4b0 0x558ad47bf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:14.440 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.429+0000 7f2674f50b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ad4692a60 con 0x558ad47a1580 2023-07-21T19:03:14.440 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.429+0000 7f2674f50b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ad4692760 con 0x558ad45fd6b0 2023-07-21T19:03:14.440 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.429+0000 7f2674f50b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ad45768c0 con 0x558ad47ab4b0 2023-07-21T19:03:14.440 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.429+0000 7f2674f50b80 10 client.0 _ll_register_callbacks cb 0x558ad4690700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:03:14.440 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.429+0000 7f2661d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558ad47a1580 0x558ad47aaf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:14.440 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.429+0000 7f266259c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ad45fd6b0 0x558ad47aaa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:14.440 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.429+0000 7f2661d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558ad47a1580 0x558ad47aaf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34508/0 (socket says 172.21.15.39:34508) 2023-07-21T19:03:14.441 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.429+0000 7f2661d9b700 1 -- 172.21.15.39:0/3657455840 learned_addr learned my addr 172.21.15.39:0/3657455840 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:03:14.441 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.429+0000 7f2662d9d700 1 --2- 172.21.15.39:0/3657455840 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ad47ab4b0 0x558ad47bf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:14.441 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.429+0000 7f2661d9b700 1 -- 172.21.15.39:0/3657455840 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ad47ab4b0 msgr2=0x558ad47bf4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:03:14.441 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.429+0000 7f2661d9b700 1 --2- 172.21.15.39:0/3657455840 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ad47ab4b0 0x558ad47bf4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:14.441 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.429+0000 7f2661d9b700 1 -- 172.21.15.39:0/3657455840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ad45fd6b0 msgr2=0x558ad47aaa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:03:14.441 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.429+0000 7f2661d9b700 1 --2- 172.21.15.39:0/3657455840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ad45fd6b0 0x558ad47aaa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:14.441 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.430+0000 7f2661d9b700 1 -- 172.21.15.39:0/3657455840 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f265c00e410 con 0x558ad47a1580 2023-07-21T19:03:14.442 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.430+0000 7f2661d9b700 1 --2- 172.21.15.39:0/3657455840 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558ad47a1580 0x558ad47aaf70 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f265800dba0 tx=0x7f26580047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53026943ba919567 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:03:14.442 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.430+0000 7f2649ffb700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T19:03:14.443 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.430+0000 7f2649ffb700 1 -- 172.21.15.39:0/3657455840 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2658010a70 con 0x558ad47a1580 2023-07-21T19:03:14.443 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.430+0000 7f2649ffb700 1 -- 172.21.15.39:0/3657455840 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f2658010bd0 con 0x558ad47a1580 2023-07-21T19:03:14.443 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.430+0000 7f2674f50b80 1 -- 172.21.15.39:0/3657455840 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558ad464c490 con 0x558ad47a1580 2023-07-21T19:03:14.443 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.430+0000 7f2649ffb700 1 -- 172.21.15.39:0/3657455840 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) v2 ==== 24+0+0 (secure 0 0 0) 0x7f265800deb0 con 0x558ad47a1580 2023-07-21T19:03:14.443 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.430+0000 7f2674f50b80 10 client.17715 fetch_fsmap learned FSMap version 103 2023-07-21T19:03:14.444 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.430+0000 7f2674f50b80 1 -- 172.21.15.39:0/3657455840 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=4+}) v3 -- 0x558ad4509c00 con 0x558ad47a1580 2023-07-21T19:03:14.444 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.431+0000 7f2649ffb700 1 -- 172.21.15.39:0/3657455840 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558ad464c490 con 0x558ad47a1580 2023-07-21T19:03:14.444 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.431+0000 7f2649ffb700 1 -- 172.21.15.39:0/3657455840 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 103) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f265800f3c0 con 0x558ad47a1580 2023-07-21T19:03:14.444 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.431+0000 7f2674f50b80 10 client.17715 fetch_fsmap finished waiting for FSMap version 103 2023-07-21T19:03:14.444 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.431+0000 7f2674f50b80 10 client.17715 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:03:14.444 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.431+0000 7f2674f50b80 20 client.17715 populate_metadata read hostname 'smithi039' 2023-07-21T19:03:14.444 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.431+0000 7f2674f50b80 1 --2- 172.21.15.39:0/3657455840 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x558ad47c39a0 0x558ad47c5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:14.444 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.431+0000 7f2674f50b80 4 client.17715 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:03:14.445 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.431+0000 7f2674f50b80 1 -- 172.21.15.39:0/3657455840 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558ad46ebe50 con 0x558ad47c39a0 2023-07-21T19:03:14.445 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.431+0000 7f266259c700 1 --2- 172.21.15.39:0/3657455840 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x558ad47c39a0 0x558ad47c5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:14.445 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.432+0000 7f2649ffb700 10 client.17715 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T19:03:14.445 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.432+0000 7f266259c700 1 --2- 172.21.15.39:0/3657455840 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x558ad47c39a0 0x558ad47c5db0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82c34f0e3deb55b8 server_cookie=6a38b5afb95f086e in_seq=0 out_seq=0 2023-07-21T19:03:14.446 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.434+0000 7f2649ffb700 1 -- 172.21.15.39:0/3657455840 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558ad46ebe50 con 0x558ad47c39a0 2023-07-21T19:03:14.446 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.434+0000 7f2649ffb700 10 client.17715 handle_command_reply: tid=0 2023-07-21T19:03:14.446 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.434+0000 7f2674f50b80 1 client.17715 shutdown 2023-07-21T19:03:14.446 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.434+0000 7f2674f50b80 1 -- 172.21.15.39:0/3657455840 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558ad47a1580 msgr2=0x558ad47aaf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:03:14.447 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.434+0000 7f2674f50b80 1 --2- 172.21.15.39:0/3657455840 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558ad47a1580 0x558ad47aaf70 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f265800dba0 tx=0x7f26580047c0 comp rx=0 tx=0).stop 2023-07-21T19:03:14.447 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.434+0000 7f2674f50b80 1 -- 172.21.15.39:0/3657455840 shutdown_connections 2023-07-21T19:03:14.447 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.434+0000 7f2674f50b80 1 --2- 172.21.15.39:0/3657455840 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x558ad47c39a0 0x558ad47c5db0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:14.447 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.434+0000 7f2674f50b80 1 --2- 172.21.15.39:0/3657455840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ad45fd6b0 0x558ad47aaa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:14.447 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.434+0000 7f2674f50b80 1 --2- 172.21.15.39:0/3657455840 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ad47ab4b0 0x558ad47bf4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:14.447 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.434+0000 7f2674f50b80 1 --2- 172.21.15.39:0/3657455840 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558ad47a1580 0x558ad47aaf70 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:14.447 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.434+0000 7f2674f50b80 1 -- 172.21.15.39:0/3657455840 >> 172.21.15.39:0/3657455840 conn(0x558ad4796be0 msgr2=0x558ad479ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:03:14.447 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.435+0000 7f2649ffb700 0 client.17715 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T19:03:14.448 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.435+0000 7f2674f50b80 1 -- 172.21.15.39:0/3657455840 shutdown_connections 2023-07-21T19:03:14.448 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.435+0000 7f2674f50b80 1 -- 172.21.15.39:0/3657455840 wait complete. 2023-07-21T19:03:14.448 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.438+0000 7f2674f50b80 20 client.17715 trim_cache size 0 max 16384 2023-07-21T19:03:14.472 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.463+0000 7f2674f50b80 1 Processor -- start 2023-07-21T19:03:14.475 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.464+0000 7f2674f50b80 1 -- start start 2023-07-21T19:03:14.475 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.464+0000 7f2674f50b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558ad4796be0 0x558ad47a1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:14.476 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.464+0000 7f2674f50b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ad45fd6b0 0x558ad47bf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:14.476 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.464+0000 7f2674f50b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ad47a1580 0x558ad479bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:14.476 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.464+0000 7f2674f50b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ad47c0ff0 con 0x558ad4796be0 2023-07-21T19:03:14.476 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.464+0000 7f2674f50b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ad47a2000 con 0x558ad47a1580 2023-07-21T19:03:14.477 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.464+0000 7f2674f50b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ad4793bb0 con 0x558ad45fd6b0 2023-07-21T19:03:14.477 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.464+0000 7f2661d9b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ad45fd6b0 0x558ad47bf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:14.477 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.464+0000 7f2661d9b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ad45fd6b0 0x558ad47bf4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:53068/0 (socket says 172.21.15.39:53068) 2023-07-21T19:03:14.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.464+0000 7f2661d9b700 1 -- 172.21.15.39:0/4161552823 learned_addr learned my addr 172.21.15.39:0/4161552823 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:03:14.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.464+0000 7f2662d9d700 1 --2- 172.21.15.39:0/4161552823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ad47a1580 0x558ad479bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:14.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.465+0000 7f2661d9b700 1 -- 172.21.15.39:0/4161552823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ad47a1580 msgr2=0x558ad479bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:03:14.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.465+0000 7f2661d9b700 1 --2- 172.21.15.39:0/4161552823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ad47a1580 0x558ad479bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:14.479 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.465+0000 7f2661d9b700 1 -- 172.21.15.39:0/4161552823 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558ad4796be0 msgr2=0x558ad47a1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:03:14.479 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.465+0000 7f2661d9b700 1 --2- 172.21.15.39:0/4161552823 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558ad4796be0 0x558ad47a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:14.479 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.465+0000 7f2661d9b700 1 -- 172.21.15.39:0/4161552823 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ad47a1c20 con 0x558ad45fd6b0 2023-07-21T19:03:14.479 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.465+0000 7f2661d9b700 1 --2- 172.21.15.39:0/4161552823 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ad45fd6b0 0x558ad47bf4f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f265800f7a0 tx=0x7f2658005a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1958d3b548036e9e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:03:14.479 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.465+0000 7f26517fa700 1 -- 172.21.15.39:0/4161552823 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f265800fcd0 con 0x558ad45fd6b0 2023-07-21T19:03:14.479 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.465+0000 7f26517fa700 1 -- 172.21.15.39:0/4161552823 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f265800fe30 con 0x558ad45fd6b0 2023-07-21T19:03:14.479 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.465+0000 7f26517fa700 1 -- 172.21.15.39:0/4161552823 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f265801d860 con 0x558ad45fd6b0 2023-07-21T19:03:14.482 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.466+0000 7f2674f50b80 1 -- 172.21.15.39:0/4161552823 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ad45fd6b0 msgr2=0x558ad47bf4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:03:14.482 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.466+0000 7f2674f50b80 1 --2- 172.21.15.39:0/4161552823 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ad45fd6b0 0x558ad47bf4f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f265800f7a0 tx=0x7f2658005a00 comp rx=0 tx=0).stop 2023-07-21T19:03:14.483 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.466+0000 7f2674f50b80 1 -- 172.21.15.39:0/4161552823 shutdown_connections 2023-07-21T19:03:14.483 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.466+0000 7f2674f50b80 1 --2- 172.21.15.39:0/4161552823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ad47a1580 0x558ad479bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:14.483 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.466+0000 7f2674f50b80 1 --2- 172.21.15.39:0/4161552823 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558ad4796be0 0x558ad47a1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:14.483 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.466+0000 7f2674f50b80 1 --2- 172.21.15.39:0/4161552823 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ad45fd6b0 0x558ad47bf4f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:14.483 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.466+0000 7f2674f50b80 1 -- 172.21.15.39:0/4161552823 >> 172.21.15.39:0/4161552823 conn(0x558ad47aad50 msgr2=0x558ad47a5bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:03:14.483 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.466+0000 7f2674f50b80 1 -- 172.21.15.39:0/4161552823 shutdown_connections 2023-07-21T19:03:14.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.466+0000 7f2674f50b80 1 -- 172.21.15.39:0/4161552823 wait complete. 2023-07-21T19:03:14.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.467+0000 7f2674f50b80 1 -- start start 2023-07-21T19:03:14.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.467+0000 7f2674f50b80 1 Processor -- start 2023-07-21T19:03:14.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.467+0000 7f2674f50b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558ad45fd6b0 0x558ad48131c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:14.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.467+0000 7f2674f50b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ad4796be0 0x558ad480f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:14.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.467+0000 7f2674f50b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ad47a1580 0x558ad480fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:14.485 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.467+0000 7f2674f50b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ad4810460 con 0x558ad45fd6b0 2023-07-21T19:03:14.485 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.467+0000 7f2674f50b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ad48105d0 con 0x558ad47a1580 2023-07-21T19:03:14.485 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.467+0000 7f2674f50b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ad47b5c90 con 0x558ad4796be0 2023-07-21T19:03:14.486 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.467+0000 7f2674f50b80 10 client.0 _ll_register_callbacks cb 0x7f265800d7d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:03:14.486 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.468+0000 7f2662d9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ad47a1580 0x558ad480fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:14.486 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.468+0000 7f2662d9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ad47a1580 0x558ad480fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:47116/0 (socket says 172.21.15.39:47116) 2023-07-21T19:03:14.487 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.468+0000 7f2662d9d700 1 -- 172.21.15.39:0/832684894 learned_addr learned my addr 172.21.15.39:0/832684894 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:03:14.487 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.468+0000 7f2661d9b700 1 --2- 172.21.15.39:0/832684894 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ad4796be0 0x558ad480f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:14.487 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.468+0000 7f266259c700 1 --2- 172.21.15.39:0/832684894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558ad45fd6b0 0x558ad48131c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:14.487 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.468+0000 7f2662d9d700 1 -- 172.21.15.39:0/832684894 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ad4796be0 msgr2=0x558ad480f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:03:14.488 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.468+0000 7f2662d9d700 1 --2- 172.21.15.39:0/832684894 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ad4796be0 0x558ad480f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:14.488 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.468+0000 7f2662d9d700 1 -- 172.21.15.39:0/832684894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558ad45fd6b0 msgr2=0x558ad48131c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:03:14.488 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.468+0000 7f2662d9d700 1 --2- 172.21.15.39:0/832684894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558ad45fd6b0 0x558ad48131c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:14.488 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.468+0000 7f2662d9d700 1 -- 172.21.15.39:0/832684894 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558ad4509c00 con 0x558ad47a1580 2023-07-21T19:03:14.489 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.468+0000 7f266259c700 1 --2- 172.21.15.39:0/832684894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558ad45fd6b0 0x558ad48131c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:03:14.489 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.468+0000 7f2661d9b700 1 --2- 172.21.15.39:0/832684894 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ad4796be0 0x558ad480f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:03:14.489 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.468+0000 7f2662d9d700 1 --2- 172.21.15.39:0/832684894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ad47a1580 0x558ad480fd50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f265c00e730 tx=0x7f265c003f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=363f558917148f8b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:03:14.490 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.468+0000 7f2649ffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-21T19:03:14.490 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.468+0000 7f2649ffb700 1 -- 172.21.15.39:0/832684894 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f265c024070 con 0x558ad47a1580 2023-07-21T19:03:14.490 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.468+0000 7f2649ffb700 1 -- 172.21.15.39:0/832684894 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f265c004710 con 0x558ad47a1580 2023-07-21T19:03:14.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.468+0000 7f2674f50b80 1 -- 172.21.15.39:0/832684894 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f265c00f040 con 0x558ad47a1580 2023-07-21T19:03:14.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.469+0000 7f2649ffb700 1 -- 172.21.15.39:0/832684894 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) v2 ==== 24+0+0 (secure 0 0 0) 0x7f265c00e040 con 0x558ad47a1580 2023-07-21T19:03:14.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.469+0000 7f2674f50b80 10 client.27134 fetch_fsmap learned FSMap version 103 2023-07-21T19:03:14.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.469+0000 7f2674f50b80 1 -- 172.21.15.39:0/832684894 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=4+}) v3 -- 0x558ad45471f0 con 0x558ad47a1580 2023-07-21T19:03:14.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.469+0000 7f2649ffb700 1 -- 172.21.15.39:0/832684894 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f265c00f040 con 0x558ad47a1580 2023-07-21T19:03:14.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.469+0000 7f2649ffb700 1 -- 172.21.15.39:0/832684894 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 103) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f265c0159f0 con 0x558ad47a1580 2023-07-21T19:03:14.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.469+0000 7f2674f50b80 10 client.27134 fetch_fsmap finished waiting for FSMap version 103 2023-07-21T19:03:14.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.469+0000 7f2674f50b80 10 client.27134 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:03:14.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.469+0000 7f2674f50b80 20 client.27134 populate_metadata read hostname 'smithi039' 2023-07-21T19:03:14.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.469+0000 7f2674f50b80 1 --2- 172.21.15.39:0/832684894 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x558ad47ab4b0 0x558ad47b9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:14.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.469+0000 7f2674f50b80 4 client.27134 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:03:14.494 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.469+0000 7f2674f50b80 1 -- 172.21.15.39:0/832684894 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558ad46adff0 con 0x558ad47ab4b0 2023-07-21T19:03:14.494 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.469+0000 7f266259c700 1 --2- 172.21.15.39:0/832684894 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x558ad47ab4b0 0x558ad47b9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:14.494 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.470+0000 7f2649ffb700 10 client.27134 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T19:03:14.495 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.470+0000 7f266259c700 1 --2- 172.21.15.39:0/832684894 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x558ad47ab4b0 0x558ad47b9260 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88fe66ee4837e2dc server_cookie=d6a690a5482ded02 in_seq=0 out_seq=0 2023-07-21T19:03:14.611 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:03:14 smithi193 ceph-mon[98204]: pgmap v682: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 20 MiB/s wr, 834 op/s 2023-07-21T19:03:14.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:03:14 smithi155 ceph-mon[97934]: pgmap v682: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 20 MiB/s wr, 834 op/s 2023-07-21T19:03:14.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:03:14 smithi039 ceph-mon[96489]: pgmap v682: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 20 MiB/s wr, 834 op/s 2023-07-21T19:03:14.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.680+0000 7f2649ffb700 1 -- 172.21.15.39:0/832684894 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558ad46adff0 con 0x558ad47ab4b0 2023-07-21T19:03:14.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.680+0000 7f2649ffb700 10 client.27134 handle_command_reply: tid=0 2023-07-21T19:03:14.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.680+0000 7f2674f50b80 1 client.27134 shutdown 2023-07-21T19:03:14.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.680+0000 7f2674f50b80 1 -- 172.21.15.39:0/832684894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ad47a1580 msgr2=0x558ad480fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:03:14.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.680+0000 7f2674f50b80 1 --2- 172.21.15.39:0/832684894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ad47a1580 0x558ad480fd50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f265c00e730 tx=0x7f265c003f90 comp rx=0 tx=0).stop 2023-07-21T19:03:14.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.680+0000 7f2674f50b80 1 -- 172.21.15.39:0/832684894 shutdown_connections 2023-07-21T19:03:14.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.680+0000 7f2674f50b80 1 --2- 172.21.15.39:0/832684894 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x558ad47ab4b0 0x558ad47b9260 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:14.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.680+0000 7f2674f50b80 1 --2- 172.21.15.39:0/832684894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ad47a1580 0x558ad480fd50 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:14.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.680+0000 7f2674f50b80 1 --2- 172.21.15.39:0/832684894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558ad45fd6b0 0x558ad48131c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:14.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.680+0000 7f2674f50b80 1 --2- 172.21.15.39:0/832684894 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ad4796be0 0x558ad480f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:14.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.680+0000 7f2674f50b80 1 -- 172.21.15.39:0/832684894 >> 172.21.15.39:0/832684894 conn(0x558ad47aad50 msgr2=0x558ad47a6440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:03:14.693 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.680+0000 7f2649ffb700 0 client.27134 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T19:03:14.693 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.680+0000 7f2674f50b80 1 -- 172.21.15.39:0/832684894 shutdown_connections 2023-07-21T19:03:14.693 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.680+0000 7f2674f50b80 1 -- 172.21.15.39:0/832684894 wait complete. 2023-07-21T19:03:14.693 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.680+0000 7f2674f50b80 20 client.27134 trim_cache size 0 max 16384 2023-07-21T19:03:14.693 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T19:03:14.694 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T19:03:14.694 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T19:03:14.694 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T19:03:14.696 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.681+0000 7f2668dca700 1 -- 172.21.15.39:0/537467647 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f264c0a1340 msgr2=0x7f264c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:03:14.696 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.681+0000 7f2668dca700 1 --2- 172.21.15.39:0/537467647 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f264c0a1340 0x7f264c0a37f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2664102040 tx=0x7f265800b3f0 comp rx=0 tx=0).stop 2023-07-21T19:03:14.696 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.681+0000 7f2668dca700 1 -- 172.21.15.39:0/537467647 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26640fea30 msgr2=0x7f2664100820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:03:14.696 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.681+0000 7f2668dca700 1 --2- 172.21.15.39:0/537467647 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26640fea30 0x7f2664100820 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2654000c00 tx=0x7f2654005ef0 comp rx=0 tx=0).stop 2023-07-21T19:03:14.696 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.681+0000 7f2668dca700 1 -- 172.21.15.39:0/537467647 shutdown_connections 2023-07-21T19:03:14.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.681+0000 7f2668dca700 1 --2- 172.21.15.39:0/537467647 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2664104a40 0x7f26641a3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:14.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.681+0000 7f2668dca700 1 --2- 172.21.15.39:0/537467647 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26640fea30 0x7f2664100820 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:14.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.681+0000 7f2668dca700 1 --2- 172.21.15.39:0/537467647 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f264c0a1340 0x7f264c0a37f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:14.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.681+0000 7f2668dca700 1 --2- 172.21.15.39:0/537467647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26640ff3e0 0x7f2664100d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:14.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.681+0000 7f2668dca700 1 -- 172.21.15.39:0/537467647 >> 172.21.15.39:0/537467647 conn(0x7f26640fa4f0 msgr2=0x7f2664068440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:03:14.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.683+0000 7f2668dca700 1 -- 172.21.15.39:0/537467647 shutdown_connections 2023-07-21T19:03:14.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:14.685+0000 7f2668dca700 1 -- 172.21.15.39:0/537467647 wait complete. 2023-07-21T19:03:14.708 INFO:tasks.workunit.client.0.smithi039.stdout: 10 276783 29.20 MB/sec execute 387 sec latency 166.443 ms 2023-07-21T19:03:15.708 INFO:tasks.workunit.client.0.smithi039.stdout: 10 277287 29.19 MB/sec execute 388 sec latency 137.283 ms 2023-07-21T19:03:16.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:03:16 smithi155 ceph-mon[97934]: pgmap v683: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 20 MiB/s wr, 834 op/s 2023-07-21T19:03:16.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:03:16 smithi039 ceph-mon[96489]: pgmap v683: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 20 MiB/s wr, 834 op/s 2023-07-21T19:03:16.708 INFO:tasks.workunit.client.0.smithi039.stdout: 10 277733 29.17 MB/sec execute 389 sec latency 92.127 ms 2023-07-21T19:03:16.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:03:16 smithi193 ceph-mon[98204]: pgmap v683: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 20 MiB/s wr, 834 op/s 2023-07-21T19:03:17.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:03:17 smithi155 ceph-mon[97934]: osdmap e118: 12 total, 12 up, 12 in 2023-07-21T19:03:17.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:03:17 smithi039 ceph-mon[96489]: osdmap e118: 12 total, 12 up, 12 in 2023-07-21T19:03:17.708 INFO:tasks.workunit.client.0.smithi039.stdout: 10 278298 29.19 MB/sec execute 390 sec latency 85.504 ms 2023-07-21T19:03:17.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:03:17 smithi193 ceph-mon[98204]: osdmap e118: 12 total, 12 up, 12 in 2023-07-21T19:03:18.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:03:18 smithi039 ceph-mon[96489]: pgmap v685: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.7 MiB/s wr, 562 op/s 2023-07-21T19:03:18.708 INFO:tasks.workunit.client.0.smithi039.stdout: 10 278857 29.17 MB/sec execute 391 sec latency 91.448 ms 2023-07-21T19:03:18.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:03:18 smithi193 ceph-mon[98204]: pgmap v685: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.7 MiB/s wr, 562 op/s 2023-07-21T19:03:18.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:03:18 smithi155 ceph-mon[97934]: pgmap v685: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.7 MiB/s wr, 562 op/s 2023-07-21T19:03:19.708 INFO:tasks.workunit.client.0.smithi039.stdout: 10 279448 29.18 MB/sec execute 392 sec latency 111.361 ms 2023-07-21T19:03:20.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:03:20 smithi039 ceph-mon[96489]: pgmap v686: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.2 MiB/s wr, 462 op/s 2023-07-21T19:03:20.708 INFO:tasks.workunit.client.0.smithi039.stdout: 10 280011 29.19 MB/sec execute 393 sec latency 90.340 ms 2023-07-21T19:03:20.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:03:20 smithi193 ceph-mon[98204]: pgmap v686: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.2 MiB/s wr, 462 op/s 2023-07-21T19:03:20.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:03:20 smithi155 ceph-mon[97934]: pgmap v686: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.2 MiB/s wr, 462 op/s 2023-07-21T19:03:21.708 INFO:tasks.workunit.client.0.smithi039.stdout: 10 280578 29.18 MB/sec execute 394 sec latency 117.649 ms 2023-07-21T19:03:22.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:03:22 smithi039 ceph-mon[96489]: pgmap v687: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 6.9 MiB/s wr, 450 op/s 2023-07-21T19:03:22.708 INFO:tasks.workunit.client.0.smithi039.stdout: 10 281010 29.17 MB/sec execute 395 sec latency 246.719 ms 2023-07-21T19:03:22.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:03:22 smithi193 ceph-mon[98204]: pgmap v687: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 6.9 MiB/s wr, 450 op/s 2023-07-21T19:03:22.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:03:22 smithi155 ceph-mon[97934]: pgmap v687: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 6.9 MiB/s wr, 450 op/s 2023-07-21T19:03:23.708 INFO:tasks.workunit.client.0.smithi039.stdout: 10 281610 29.18 MB/sec execute 396 sec latency 85.347 ms 2023-07-21T19:03:24.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:03:24 smithi039 ceph-mon[96489]: pgmap v688: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.0 MiB/s wr, 487 op/s 2023-07-21T19:03:24.708 INFO:tasks.workunit.client.0.smithi039.stdout: 10 282223 29.20 MB/sec execute 397 sec latency 92.583 ms 2023-07-21T19:03:24.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:03:24 smithi193 ceph-mon[98204]: pgmap v688: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.0 MiB/s wr, 487 op/s 2023-07-21T19:03:24.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:03:24 smithi155 ceph-mon[97934]: pgmap v688: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.0 MiB/s wr, 487 op/s 2023-07-21T19:03:25.708 INFO:tasks.workunit.client.0.smithi039.stdout: 10 282780 29.20 MB/sec execute 398 sec latency 99.031 ms 2023-07-21T19:03:26.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:03:26 smithi039 ceph-mon[96489]: pgmap v689: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.0 MiB/s wr, 487 op/s 2023-07-21T19:03:26.708 INFO:tasks.workunit.client.0.smithi039.stdout: 10 283272 29.19 MB/sec execute 399 sec latency 112.248 ms 2023-07-21T19:03:26.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:03:26 smithi193 ceph-mon[98204]: pgmap v689: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.0 MiB/s wr, 487 op/s 2023-07-21T19:03:26.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:03:26 smithi155 ceph-mon[97934]: pgmap v689: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.0 MiB/s wr, 487 op/s 2023-07-21T19:03:27.708 INFO:tasks.workunit.client.0.smithi039.stdout: 10 283818 29.18 MB/sec execute 400 sec latency 125.222 ms 2023-07-21T19:03:27.949 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.939+0000 7f20ba9bc700 1 -- 172.21.15.39:0/416556237 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20b4112930 msgr2=0x7f20b4114d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:03:27.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.940+0000 7f20ba9bc700 1 --2- 172.21.15.39:0/416556237 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20b4112930 0x7f20b4114d10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f20ac00b0a0 tx=0x7f20ac00b3b0 comp rx=0 tx=0).stop 2023-07-21T19:03:27.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.940+0000 7f20ba9bc700 1 -- 172.21.15.39:0/416556237 shutdown_connections 2023-07-21T19:03:27.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.940+0000 7f20ba9bc700 1 --2- 172.21.15.39:0/416556237 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20b4112930 0x7f20b4114d10 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:27.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.940+0000 7f20ba9bc700 1 --2- 172.21.15.39:0/416556237 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20b4108210 0x7f20b406c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:27.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.940+0000 7f20ba9bc700 1 --2- 172.21.15.39:0/416556237 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f20b406cd70 0x7f20b406d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:27.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.940+0000 7f20ba9bc700 1 -- 172.21.15.39:0/416556237 >> 172.21.15.39:0/416556237 conn(0x7f20b406bd70 msgr2=0x7f20b4070f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:03:27.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.942+0000 7f20ba9bc700 1 -- 172.21.15.39:0/416556237 shutdown_connections 2023-07-21T19:03:27.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.942+0000 7f20ba9bc700 1 -- 172.21.15.39:0/416556237 wait complete. 2023-07-21T19:03:27.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.942+0000 7f20ba9bc700 1 Processor -- start 2023-07-21T19:03:27.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.943+0000 7f20ba9bc700 1 -- start start 2023-07-21T19:03:27.955 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.943+0000 7f20ba9bc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20b406cd70 0x7f20b411c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:27.955 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.943+0000 7f20ba9bc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20b4112930 0x7f20b411cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:27.955 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.943+0000 7f20ba9bc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f20b41225d0 0x7f20b4122a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:27.956 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.943+0000 7f20ba9bc700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b4123180 con 0x7f20b41225d0 2023-07-21T19:03:27.956 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.943+0000 7f20ba9bc700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b41232f0 con 0x7f20b406cd70 2023-07-21T19:03:27.956 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.943+0000 7f20ba9bc700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b4117760 con 0x7f20b4112930 2023-07-21T19:03:27.956 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.943+0000 7f20b37fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20b4112930 0x7f20b411cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:27.956 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.943+0000 7f20b3fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20b406cd70 0x7f20b411c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:27.956 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.943+0000 7f20b37fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20b4112930 0x7f20b411cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:53086/0 (socket says 172.21.15.39:53086) 2023-07-21T19:03:27.956 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.943+0000 7f20b37fe700 1 -- 172.21.15.39:0/4016412301 learned_addr learned my addr 172.21.15.39:0/4016412301 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:03:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.944+0000 7f20b8f59700 1 --2- 172.21.15.39:0/4016412301 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f20b41225d0 0x7f20b4122a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.944+0000 7f20b3fff700 1 -- 172.21.15.39:0/4016412301 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20b4112930 msgr2=0x7f20b411cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:03:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.944+0000 7f20b3fff700 1 --2- 172.21.15.39:0/4016412301 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20b4112930 0x7f20b411cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.944+0000 7f20b3fff700 1 -- 172.21.15.39:0/4016412301 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f20b41225d0 msgr2=0x7f20b4122a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:03:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.944+0000 7f20b3fff700 1 --2- 172.21.15.39:0/4016412301 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f20b41225d0 0x7f20b4122a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.944+0000 7f20b3fff700 1 -- 172.21.15.39:0/4016412301 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20ac009d00 con 0x7f20b406cd70 2023-07-21T19:03:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.944+0000 7f20b8f59700 1 --2- 172.21.15.39:0/4016412301 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f20b41225d0 0x7f20b4122a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:03:27.958 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.944+0000 7f20b3fff700 1 --2- 172.21.15.39:0/4016412301 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20b406cd70 0x7f20b411c850 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f20b406dbe0 tx=0x7f20a800b6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:03:27.958 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.944+0000 7f20b17fa700 1 -- 172.21.15.39:0/4016412301 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a8011830 con 0x7f20b406cd70 2023-07-21T19:03:27.958 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.944+0000 7f20ba9bc700 1 -- 172.21.15.39:0/4016412301 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20b4117a40 con 0x7f20b406cd70 2023-07-21T19:03:27.958 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.944+0000 7f20b17fa700 1 -- 172.21.15.39:0/4016412301 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f20a800be60 con 0x7f20b406cd70 2023-07-21T19:03:27.958 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.944+0000 7f20b17fa700 1 -- 172.21.15.39:0/4016412301 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a8010d20 con 0x7f20b406cd70 2023-07-21T19:03:27.959 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.944+0000 7f20ba9bc700 1 -- 172.21.15.39:0/4016412301 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20b4117f00 con 0x7f20b406cd70 2023-07-21T19:03:27.960 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.947+0000 7f20b17fa700 1 -- 172.21.15.39:0/4016412301 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f20a8010380 con 0x7f20b406cd70 2023-07-21T19:03:27.961 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.948+0000 7f20b17fa700 1 --2- 172.21.15.39:0/4016412301 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f209c0a12c0 0x7f209c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:27.961 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.948+0000 7f20b17fa700 1 -- 172.21.15.39:0/4016412301 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f20a80cc5d0 con 0x7f20b406cd70 2023-07-21T19:03:27.961 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.949+0000 7f20c6b42b80 1 Processor -- start 2023-07-21T19:03:27.962 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.949+0000 7f20c6b42b80 1 -- start start 2023-07-21T19:03:27.962 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.950+0000 7f20c6b42b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f815cc16b0 0x55f815e6ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:27.962 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.950+0000 7f20c6b42b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f815e6b2d0 0x55f815e65040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:27.962 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.950+0000 7f20c6b42b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f815e65580 0x55f815e659d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:27.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.950+0000 7f20c6b42b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f815d6ac50 con 0x55f815e65580 2023-07-21T19:03:27.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.950+0000 7f20c6b42b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f815dbd9e0 con 0x55f815e6b2d0 2023-07-21T19:03:27.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.950+0000 7f20c6b42b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f815db6e70 con 0x55f815cc16b0 2023-07-21T19:03:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.950+0000 7f20b8f59700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f815e65580 0x55f815e659d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.950+0000 7f20b8f59700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f815e65580 0x55f815e659d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34540/0 (socket says 172.21.15.39:34540) 2023-07-21T19:03:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.950+0000 7f20b8f59700 1 -- 172.21.15.39:0/2008549308 learned_addr learned my addr 172.21.15.39:0/2008549308 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:03:27.965 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.951+0000 7f20b8f59700 1 -- 172.21.15.39:0/2008549308 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f815cc16b0 msgr2=0x55f815e6ad90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T19:03:27.965 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.951+0000 7f20b8f59700 1 --2- 172.21.15.39:0/2008549308 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f815cc16b0 0x55f815e6ad90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:27.965 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.951+0000 7f20b8f59700 1 -- 172.21.15.39:0/2008549308 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f815e6b2d0 msgr2=0x55f815e65040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T19:03:27.965 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.951+0000 7f20b8f59700 1 --2- 172.21.15.39:0/2008549308 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f815e6b2d0 0x55f815e65040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:27.966 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.951+0000 7f20b8f59700 1 -- 172.21.15.39:0/2008549308 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f815be4d80 con 0x55f815e65580 2023-07-21T19:03:27.968 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.952+0000 7f20b37fe700 1 --2- 172.21.15.39:0/4016412301 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f209c0a12c0 0x7f209c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:27.968 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.952+0000 7f20b37fe700 1 --2- 172.21.15.39:0/4016412301 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f209c0a12c0 0x7f209c0a3770 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x55f815e61b40 tx=0x7f20a4009210 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:03:27.968 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.954+0000 7f20b8f59700 1 --2- 172.21.15.39:0/2008549308 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f815e65580 0x55f815e659d0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f20ac00b830 tx=0x7f20ac008f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7ab4ac73eca7dc2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:03:27.969 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.954+0000 7f209a7fc700 1 -- 172.21.15.39:0/2008549308 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ac00e050 con 0x55f815e65580 2023-07-21T19:03:27.969 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.954+0000 7f209a7fc700 1 -- 172.21.15.39:0/2008549308 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f20ac009450 con 0x55f815e65580 2023-07-21T19:03:27.969 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.955+0000 7f209a7fc700 1 -- 172.21.15.39:0/2008549308 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ac005330 con 0x55f815e65580 2023-07-21T19:03:27.969 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.956+0000 7f20c6b42b80 1 -- 172.21.15.39:0/2008549308 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f815e65580 msgr2=0x55f815e659d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:03:27.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.956+0000 7f20c6b42b80 1 --2- 172.21.15.39:0/2008549308 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f815e65580 0x55f815e659d0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f20ac00b830 tx=0x7f20ac008f90 comp rx=0 tx=0).stop 2023-07-21T19:03:27.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.957+0000 7f20c6b42b80 1 -- 172.21.15.39:0/2008549308 shutdown_connections 2023-07-21T19:03:27.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.957+0000 7f20c6b42b80 1 --2- 172.21.15.39:0/2008549308 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f815e65580 0x55f815e659d0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:27.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.957+0000 7f20c6b42b80 1 --2- 172.21.15.39:0/2008549308 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f815cc16b0 0x55f815e6ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:27.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.957+0000 7f20c6b42b80 1 --2- 172.21.15.39:0/2008549308 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f815e6b2d0 0x55f815e65040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:27.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.957+0000 7f20c6b42b80 1 -- 172.21.15.39:0/2008549308 >> 172.21.15.39:0/2008549308 conn(0x55f815e5abe0 msgr2=0x55f815e5ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:03:27.971 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.957+0000 7f20c6b42b80 1 -- 172.21.15.39:0/2008549308 shutdown_connections 2023-07-21T19:03:27.971 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.957+0000 7f20c6b42b80 1 -- 172.21.15.39:0/2008549308 wait complete. 2023-07-21T19:03:27.971 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.957+0000 7f20c6b42b80 1 -- start start 2023-07-21T19:03:27.971 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.958+0000 7f20c6b42b80 1 Processor -- start 2023-07-21T19:03:27.971 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.958+0000 7f20c6b42b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f815cc16b0 0x55f815e6e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:27.971 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.958+0000 7f20c6b42b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f815e65580 0x55f815e6ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:27.971 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.958+0000 7f20c6b42b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f815e6b2d0 0x55f815e6f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:27.971 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.958+0000 7f20c6b42b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f815d56a60 con 0x55f815cc16b0 2023-07-21T19:03:27.972 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.958+0000 7f20c6b42b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f815d56760 con 0x55f815e65580 2023-07-21T19:03:27.972 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.958+0000 7f20c6b42b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f815c3a8c0 con 0x55f815e6b2d0 2023-07-21T19:03:27.972 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.958+0000 7f20c6b42b80 10 client.0 _ll_register_callbacks cb 0x55f815d54700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:03:27.973 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.959+0000 7f20b8f59700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f815e6b2d0 0x55f815e6f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:27.973 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.959+0000 7f20b8f59700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f815e6b2d0 0x55f815e6f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:53100/0 (socket says 172.21.15.39:53100) 2023-07-21T19:03:27.973 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.959+0000 7f20b8f59700 1 -- 172.21.15.39:0/1608755303 learned_addr learned my addr 172.21.15.39:0/1608755303 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:03:27.973 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.959+0000 7f20b37fe700 1 --2- 172.21.15.39:0/1608755303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f815e65580 0x55f815e6ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:27.973 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.959+0000 7f20b8f59700 1 -- 172.21.15.39:0/1608755303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f815e65580 msgr2=0x55f815e6ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:03:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.959+0000 7f20b8f59700 1 --2- 172.21.15.39:0/1608755303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f815e65580 0x55f815e6ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.959+0000 7f20b8f59700 1 -- 172.21.15.39:0/1608755303 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f815cc16b0 msgr2=0x55f815e6e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:03:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.959+0000 7f20b8f59700 1 --2- 172.21.15.39:0/1608755303 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f815cc16b0 0x55f815e6e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.959+0000 7f20b8f59700 1 -- 172.21.15.39:0/1608755303 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20b41159b0 con 0x55f815e6b2d0 2023-07-21T19:03:27.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.959+0000 7f20b8f59700 1 --2- 172.21.15.39:0/1608755303 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f815e6b2d0 0x55f815e6f370 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f20ac007a90 tx=0x7f20ac004c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a3b465eea275e35 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:03:27.978 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.959+0000 7f20937fe700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2023-07-21T19:03:27.978 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.959+0000 7f20937fe700 1 -- 172.21.15.39:0/1608755303 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ac007af0 con 0x55f815e6b2d0 2023-07-21T19:03:27.978 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.959+0000 7f20937fe700 1 -- 172.21.15.39:0/1608755303 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f20ac005250 con 0x55f815e6b2d0 2023-07-21T19:03:27.978 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.960+0000 7f20c6b42b80 1 -- 172.21.15.39:0/1608755303 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f815d10490 con 0x55f815e6b2d0 2023-07-21T19:03:27.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.960+0000 7f20937fe700 1 -- 172.21.15.39:0/1608755303 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20ac005410 con 0x55f815e6b2d0 2023-07-21T19:03:27.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.961+0000 7f20c6b42b80 10 client.27226 fetch_fsmap learned FSMap version 103 2023-07-21T19:03:27.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.961+0000 7f20c6b42b80 1 -- 172.21.15.39:0/1608755303 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=4+}) v3 -- 0x55f815bcdc00 con 0x55f815e6b2d0 2023-07-21T19:03:27.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.964+0000 7f20937fe700 1 -- 172.21.15.39:0/1608755303 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f815d10490 con 0x55f815e6b2d0 2023-07-21T19:03:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.964+0000 7f20937fe700 1 -- 172.21.15.39:0/1608755303 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 103) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f20ac00b0d0 con 0x55f815e6b2d0 2023-07-21T19:03:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.965+0000 7f20c6b42b80 10 client.27226 fetch_fsmap finished waiting for FSMap version 103 2023-07-21T19:03:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.965+0000 7f20c6b42b80 10 client.27226 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:03:27.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.966+0000 7f20c6b42b80 20 client.27226 populate_metadata read hostname 'smithi039' 2023-07-21T19:03:27.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.966+0000 7f20c6b42b80 1 --2- 172.21.15.39:0/1608755303 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55f815e7e710 0x55f815e80b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:27.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.966+0000 7f20c6b42b80 4 client.27226 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:03:27.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.966+0000 7f20c6b42b80 1 -- 172.21.15.39:0/1608755303 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f815dafe50 con 0x55f815e7e710 2023-07-21T19:03:27.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.968+0000 7f20b3fff700 1 --2- 172.21.15.39:0/1608755303 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55f815e7e710 0x55f815e80b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:27.983 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.968+0000 7f20937fe700 10 client.27226 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T19:03:27.983 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.968+0000 7f20b3fff700 1 --2- 172.21.15.39:0/1608755303 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55f815e7e710 0x55f815e80b20 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8c4cc47fd6fa5b6 server_cookie=18759f05426022d5 in_seq=0 out_seq=0 2023-07-21T19:03:27.983 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.971+0000 7f20937fe700 1 -- 172.21.15.39:0/1608755303 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f815dafe50 con 0x55f815e7e710 2023-07-21T19:03:27.983 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.971+0000 7f20937fe700 10 client.27226 handle_command_reply: tid=0 2023-07-21T19:03:27.987 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.978+0000 7f20c6b42b80 1 client.27226 shutdown 2023-07-21T19:03:27.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.978+0000 7f20c6b42b80 1 -- 172.21.15.39:0/1608755303 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f815e6b2d0 msgr2=0x55f815e6f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:03:27.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.978+0000 7f20c6b42b80 1 --2- 172.21.15.39:0/1608755303 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f815e6b2d0 0x55f815e6f370 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f20ac007a90 tx=0x7f20ac004c80 comp rx=0 tx=0).stop 2023-07-21T19:03:27.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.978+0000 7f20c6b42b80 1 -- 172.21.15.39:0/1608755303 shutdown_connections 2023-07-21T19:03:27.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.978+0000 7f20c6b42b80 1 --2- 172.21.15.39:0/1608755303 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55f815e7e710 0x55f815e80b20 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:27.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.978+0000 7f20c6b42b80 1 --2- 172.21.15.39:0/1608755303 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f815e6b2d0 0x55f815e6f370 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:27.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.978+0000 7f20c6b42b80 1 --2- 172.21.15.39:0/1608755303 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f815cc16b0 0x55f815e6e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:27.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.978+0000 7f20c6b42b80 1 --2- 172.21.15.39:0/1608755303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f815e65580 0x55f815e6ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:27.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.978+0000 7f20c6b42b80 1 -- 172.21.15.39:0/1608755303 >> 172.21.15.39:0/1608755303 conn(0x55f815e5abe0 msgr2=0x55f815e5ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:03:27.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.978+0000 7f20937fe700 0 client.27226 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T19:03:27.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.978+0000 7f20c6b42b80 1 -- 172.21.15.39:0/1608755303 shutdown_connections 2023-07-21T19:03:27.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.979+0000 7f20c6b42b80 1 -- 172.21.15.39:0/1608755303 wait complete. 2023-07-21T19:03:28.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:27.993+0000 7f20c6b42b80 20 client.27226 trim_cache size 0 max 16384 2023-07-21T19:03:28.030 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.020+0000 7f20c6b42b80 1 Processor -- start 2023-07-21T19:03:28.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.020+0000 7f20c6b42b80 1 -- start start 2023-07-21T19:03:28.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.021+0000 7f20c6b42b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f815cc16b0 0x55f815e65040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:28.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.021+0000 7f20c6b42b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f815e65580 0x55f815e75bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:28.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.021+0000 7f20c6b42b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f815e6b2d0 0x55f815e6a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:28.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.021+0000 7f20c6b42b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f815e61b70 con 0x55f815e65580 2023-07-21T19:03:28.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.021+0000 7f20c6b42b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f815e7b590 con 0x55f815e6b2d0 2023-07-21T19:03:28.032 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.021+0000 7f20c6b42b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f815e71980 con 0x55f815cc16b0 2023-07-21T19:03:28.032 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.021+0000 7f20b37fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f815e65580 0x55f815e75bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:28.032 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.021+0000 7f20b37fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f815e65580 0x55f815e75bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34554/0 (socket says 172.21.15.39:34554) 2023-07-21T19:03:28.032 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.021+0000 7f20b37fe700 1 -- 172.21.15.39:0/3269868928 learned_addr learned my addr 172.21.15.39:0/3269868928 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:03:28.032 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.021+0000 7f20b8f59700 1 --2- 172.21.15.39:0/3269868928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f815e6b2d0 0x55f815e6a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:28.033 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.021+0000 7f20b37fe700 1 -- 172.21.15.39:0/3269868928 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f815cc16b0 msgr2=0x55f815e65040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:03:28.033 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.021+0000 7f20b37fe700 1 --2- 172.21.15.39:0/3269868928 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f815cc16b0 0x55f815e65040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:28.033 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.021+0000 7f20b37fe700 1 -- 172.21.15.39:0/3269868928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f815e6b2d0 msgr2=0x55f815e6a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:03:28.033 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.021+0000 7f20b37fe700 1 --2- 172.21.15.39:0/3269868928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f815e6b2d0 0x55f815e6a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:28.034 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.021+0000 7f20b37fe700 1 -- 172.21.15.39:0/3269868928 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f815e65c20 con 0x55f815e65580 2023-07-21T19:03:28.034 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.022+0000 7f20b37fe700 1 --2- 172.21.15.39:0/3269868928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f815e65580 0x55f815e75bb0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f20a400e950 tx=0x7f20a4011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cde090e16d10a95a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:03:28.034 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.022+0000 7f209affd700 1 -- 172.21.15.39:0/3269868928 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a400eed0 con 0x55f815e65580 2023-07-21T19:03:28.034 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.022+0000 7f209affd700 1 -- 172.21.15.39:0/3269868928 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f20a4015940 con 0x55f815e65580 2023-07-21T19:03:28.034 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.022+0000 7f209affd700 1 -- 172.21.15.39:0/3269868928 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a40139c0 con 0x55f815e65580 2023-07-21T19:03:28.035 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.025+0000 7f20c6b42b80 1 -- 172.21.15.39:0/3269868928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f815e65580 msgr2=0x55f815e75bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:03:28.035 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.025+0000 7f20c6b42b80 1 --2- 172.21.15.39:0/3269868928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f815e65580 0x55f815e75bb0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f20a400e950 tx=0x7f20a4011000 comp rx=0 tx=0).stop 2023-07-21T19:03:28.035 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.025+0000 7f20c6b42b80 1 -- 172.21.15.39:0/3269868928 shutdown_connections 2023-07-21T19:03:28.035 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.025+0000 7f20c6b42b80 1 --2- 172.21.15.39:0/3269868928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f815e6b2d0 0x55f815e6a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:28.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.025+0000 7f20c6b42b80 1 --2- 172.21.15.39:0/3269868928 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f815cc16b0 0x55f815e65040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:28.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.025+0000 7f20c6b42b80 1 --2- 172.21.15.39:0/3269868928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f815e65580 0x55f815e75bb0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:28.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.025+0000 7f20c6b42b80 1 -- 172.21.15.39:0/3269868928 >> 172.21.15.39:0/3269868928 conn(0x55f815e5abe0 msgr2=0x55f815e69ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:03:28.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.025+0000 7f20c6b42b80 1 -- 172.21.15.39:0/3269868928 shutdown_connections 2023-07-21T19:03:28.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.025+0000 7f20c6b42b80 1 -- 172.21.15.39:0/3269868928 wait complete. 2023-07-21T19:03:28.037 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.026+0000 7f20c6b42b80 1 -- start start 2023-07-21T19:03:28.038 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.026+0000 7f20c6b42b80 1 Processor -- start 2023-07-21T19:03:28.038 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.027+0000 7f20c6b42b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f815cc16b0 0x55f815e73b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:28.038 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.027+0000 7f20c6b42b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f815e6b2d0 0x55f815e74060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:28.038 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.027+0000 7f20c6b42b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f815e745f0 0x55f815ed15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:28.038 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.027+0000 7f20c6b42b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f815ed1bf0 con 0x55f815e745f0 2023-07-21T19:03:28.038 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.027+0000 7f20c6b42b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f815ed1d30 con 0x55f815cc16b0 2023-07-21T19:03:28.039 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.027+0000 7f20c6b42b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f815ed1ef0 con 0x55f815e6b2d0 2023-07-21T19:03:28.039 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.027+0000 7f20c6b42b80 10 client.0 _ll_register_callbacks cb 0x7f20ac00b420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:03:28.039 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.027+0000 7f20b37fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f815e6b2d0 0x55f815e74060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:28.039 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.027+0000 7f20b37fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f815e6b2d0 0x55f815e74060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:53112/0 (socket says 172.21.15.39:53112) 2023-07-21T19:03:28.039 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.027+0000 7f20b37fe700 1 -- 172.21.15.39:0/3934459184 learned_addr learned my addr 172.21.15.39:0/3934459184 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:03:28.039 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.027+0000 7f20b3fff700 1 --2- 172.21.15.39:0/3934459184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f815cc16b0 0x55f815e73b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:28.039 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.028+0000 7f20b37fe700 1 -- 172.21.15.39:0/3934459184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f815cc16b0 msgr2=0x55f815e73b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:03:28.039 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.028+0000 7f20b37fe700 1 --2- 172.21.15.39:0/3934459184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f815cc16b0 0x55f815e73b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:28.040 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.028+0000 7f20b37fe700 1 -- 172.21.15.39:0/3934459184 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f815e745f0 msgr2=0x55f815ed15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:03:28.040 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.028+0000 7f20b37fe700 1 --2- 172.21.15.39:0/3934459184 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f815e745f0 0x55f815ed15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:28.040 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.028+0000 7f20b37fe700 1 -- 172.21.15.39:0/3934459184 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20a401a040 con 0x55f815e6b2d0 2023-07-21T19:03:28.040 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.028+0000 7f20b37fe700 1 --2- 172.21.15.39:0/3934459184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f815e6b2d0 0x55f815e74060 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f20a400ec60 tx=0x7f20a4015710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2dba464389cf57d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:03:28.040 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.028+0000 7f20937fe700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2023-07-21T19:03:28.041 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.030+0000 7f20937fe700 1 -- 172.21.15.39:0/3934459184 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a4013970 con 0x55f815e6b2d0 2023-07-21T19:03:28.041 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.030+0000 7f20937fe700 1 -- 172.21.15.39:0/3934459184 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f20a4013ad0 con 0x55f815e6b2d0 2023-07-21T19:03:28.041 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.030+0000 7f20c6b42b80 1 -- 172.21.15.39:0/3934459184 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f20ac009450 con 0x55f815e6b2d0 2023-07-21T19:03:28.045 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.035+0000 7f20937fe700 1 -- 172.21.15.39:0/3934459184 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20a4013c90 con 0x55f815e6b2d0 2023-07-21T19:03:28.045 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.036+0000 7f20c6b42b80 10 client.27229 fetch_fsmap learned FSMap version 103 2023-07-21T19:03:28.045 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.036+0000 7f20c6b42b80 1 -- 172.21.15.39:0/3934459184 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=4+}) v3 -- 0x55f815c0b1f0 con 0x55f815e6b2d0 2023-07-21T19:03:28.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.036+0000 7f20937fe700 1 -- 172.21.15.39:0/3934459184 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f20ac009450 con 0x55f815e6b2d0 2023-07-21T19:03:28.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.036+0000 7f20937fe700 1 -- 172.21.15.39:0/3934459184 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 103) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f20a4006510 con 0x55f815e6b2d0 2023-07-21T19:03:28.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.036+0000 7f20c6b42b80 10 client.27229 fetch_fsmap finished waiting for FSMap version 103 2023-07-21T19:03:28.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.036+0000 7f20c6b42b80 10 client.27229 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:03:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.036+0000 7f20c6b42b80 20 client.27229 populate_metadata read hostname 'smithi039' 2023-07-21T19:03:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.036+0000 7f20c6b42b80 1 --2- 172.21.15.39:0/3934459184 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55f815ed3540 0x55f815ed5950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.036+0000 7f20c6b42b80 4 client.27229 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:03:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.036+0000 7f20c6b42b80 1 -- 172.21.15.39:0/3934459184 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f815d71ff0 con 0x55f815ed3540 2023-07-21T19:03:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.037+0000 7f20b3fff700 1 --2- 172.21.15.39:0/3934459184 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55f815ed3540 0x55f815ed5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.037+0000 7f20937fe700 10 client.27229 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T19:03:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.037+0000 7f20b3fff700 1 --2- 172.21.15.39:0/3934459184 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55f815ed3540 0x55f815ed5950 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6c9a9e5283ea797 server_cookie=9596079a8344a724 in_seq=0 out_seq=0 2023-07-21T19:03:28.282 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.272+0000 7f20937fe700 1 -- 172.21.15.39:0/3934459184 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f815d71ff0 con 0x55f815ed3540 2023-07-21T19:03:28.282 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.272+0000 7f20937fe700 10 client.27229 handle_command_reply: tid=0 2023-07-21T19:03:28.283 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.272+0000 7f20c6b42b80 1 client.27229 shutdown 2023-07-21T19:03:28.283 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.272+0000 7f20c6b42b80 1 -- 172.21.15.39:0/3934459184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f815e6b2d0 msgr2=0x55f815e74060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:03:28.283 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.272+0000 7f20c6b42b80 1 --2- 172.21.15.39:0/3934459184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f815e6b2d0 0x55f815e74060 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f20a400ec60 tx=0x7f20a4015710 comp rx=0 tx=0).stop 2023-07-21T19:03:28.283 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.272+0000 7f20c6b42b80 1 -- 172.21.15.39:0/3934459184 shutdown_connections 2023-07-21T19:03:28.283 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.272+0000 7f20c6b42b80 1 --2- 172.21.15.39:0/3934459184 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55f815ed3540 0x55f815ed5950 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:28.283 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.272+0000 7f20c6b42b80 1 --2- 172.21.15.39:0/3934459184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f815cc16b0 0x55f815e73b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:28.283 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.272+0000 7f20937fe700 0 client.27229 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T19:03:28.284 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.272+0000 7f20c6b42b80 1 --2- 172.21.15.39:0/3934459184 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f815e745f0 0x55f815ed15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:28.284 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.272+0000 7f20c6b42b80 1 --2- 172.21.15.39:0/3934459184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f815e6b2d0 0x55f815e74060 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:28.284 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.272+0000 7f20c6b42b80 1 -- 172.21.15.39:0/3934459184 >> 172.21.15.39:0/3934459184 conn(0x55f815e5abe0 msgr2=0x55f815e6ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:03:28.284 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.273+0000 7f20c6b42b80 1 -- 172.21.15.39:0/3934459184 shutdown_connections 2023-07-21T19:03:28.284 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.273+0000 7f20c6b42b80 1 -- 172.21.15.39:0/3934459184 wait complete. 2023-07-21T19:03:28.284 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.273+0000 7f20c6b42b80 20 client.27229 trim_cache size 0 max 16384 2023-07-21T19:03:28.285 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T19:03:28.285 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T19:03:28.285 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T19:03:28.285 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T19:03:28.286 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.273+0000 7f20ba9bc700 1 -- 172.21.15.39:0/4016412301 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f209c0a12c0 msgr2=0x7f209c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:03:28.286 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.273+0000 7f20ba9bc700 1 --2- 172.21.15.39:0/4016412301 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f209c0a12c0 0x7f209c0a3770 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x55f815e61b40 tx=0x7f20a4009210 comp rx=0 tx=0).stop 2023-07-21T19:03:28.286 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.273+0000 7f20ba9bc700 1 -- 172.21.15.39:0/4016412301 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20b406cd70 msgr2=0x7f20b411c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:03:28.286 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.273+0000 7f20ba9bc700 1 --2- 172.21.15.39:0/4016412301 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20b406cd70 0x7f20b411c850 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f20b406dbe0 tx=0x7f20a800b6e0 comp rx=0 tx=0).stop 2023-07-21T19:03:28.287 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.273+0000 7f20ba9bc700 1 -- 172.21.15.39:0/4016412301 shutdown_connections 2023-07-21T19:03:28.287 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.273+0000 7f20ba9bc700 1 --2- 172.21.15.39:0/4016412301 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f209c0a12c0 0x7f209c0a3770 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:28.287 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.273+0000 7f20ba9bc700 1 --2- 172.21.15.39:0/4016412301 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20b406cd70 0x7f20b411c850 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:28.287 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.273+0000 7f20ba9bc700 1 --2- 172.21.15.39:0/4016412301 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f20b41225d0 0x7f20b4122a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:28.287 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.273+0000 7f20ba9bc700 1 --2- 172.21.15.39:0/4016412301 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20b4112930 0x7f20b411cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:28.287 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.273+0000 7f20ba9bc700 1 -- 172.21.15.39:0/4016412301 >> 172.21.15.39:0/4016412301 conn(0x7f20b406bd70 msgr2=0x7f20b410a830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:03:28.287 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.275+0000 7f20ba9bc700 1 -- 172.21.15.39:0/4016412301 shutdown_connections 2023-07-21T19:03:28.288 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:28.275+0000 7f20ba9bc700 1 -- 172.21.15.39:0/4016412301 wait complete. 2023-07-21T19:03:28.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:03:28 smithi039 ceph-mon[96489]: pgmap v690: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 8.8 MiB/s wr, 441 op/s 2023-07-21T19:03:28.708 INFO:tasks.workunit.client.0.smithi039.stdout: 10 284197 29.16 MB/sec execute 401 sec latency 222.884 ms 2023-07-21T19:03:28.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:03:28 smithi193 ceph-mon[98204]: pgmap v690: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 8.8 MiB/s wr, 441 op/s 2023-07-21T19:03:28.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:03:28 smithi155 ceph-mon[97934]: pgmap v690: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 8.8 MiB/s wr, 441 op/s 2023-07-21T19:03:29.708 INFO:tasks.workunit.client.0.smithi039.stdout: 10 284825 29.19 MB/sec execute 402 sec latency 90.042 ms 2023-07-21T19:03:30.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:03:30 smithi039 ceph-mon[96489]: pgmap v691: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.6 MiB/s wr, 379 op/s 2023-07-21T19:03:30.708 INFO:tasks.workunit.client.0.smithi039.stdout: 10 285375 29.17 MB/sec execute 403 sec latency 93.107 ms 2023-07-21T19:03:30.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:03:30 smithi193 ceph-mon[98204]: pgmap v691: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.6 MiB/s wr, 379 op/s 2023-07-21T19:03:30.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:03:30 smithi155 ceph-mon[97934]: pgmap v691: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.6 MiB/s wr, 379 op/s 2023-07-21T19:03:31.708 INFO:tasks.workunit.client.0.smithi039.stdout: 10 285843 29.17 MB/sec execute 404 sec latency 86.361 ms 2023-07-21T19:03:32.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:03:32 smithi039 ceph-mon[96489]: pgmap v692: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.9 MiB/s wr, 404 op/s 2023-07-21T19:03:32.709 INFO:tasks.workunit.client.0.smithi039.stdout: 10 286429 29.18 MB/sec execute 405 sec latency 74.232 ms 2023-07-21T19:03:32.860 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:03:32 smithi193 ceph-mon[98204]: pgmap v692: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.9 MiB/s wr, 404 op/s 2023-07-21T19:03:32.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:03:32 smithi155 ceph-mon[97934]: pgmap v692: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.9 MiB/s wr, 404 op/s 2023-07-21T19:03:33.708 INFO:tasks.workunit.client.0.smithi039.stdout: 10 286985 29.18 MB/sec execute 406 sec latency 101.000 ms 2023-07-21T19:03:34.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:03:34 smithi039 ceph-mon[96489]: pgmap v693: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 588 op/s 2023-07-21T19:03:34.713 INFO:tasks.workunit.client.0.smithi039.stdout: 10 287407 29.17 MB/sec execute 407 sec latency 235.129 ms 2023-07-21T19:03:34.859 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:03:34 smithi193 ceph-mon[98204]: pgmap v693: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 588 op/s 2023-07-21T19:03:34.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:03:34 smithi155 ceph-mon[97934]: pgmap v693: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 588 op/s 2023-07-21T19:03:35.710 INFO:tasks.workunit.client.0.smithi039.stdout: 10 287430 29.10 MB/sec execute 408 sec latency 1220.853 ms 2023-07-21T19:03:36.710 INFO:tasks.workunit.client.0.smithi039.stdout: 10 287624 29.05 MB/sec execute 409 sec latency 1618.311 ms 2023-07-21T19:03:36.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:03:36 smithi193 ceph-mon[98204]: pgmap v694: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.8 MiB/s wr, 429 op/s 2023-07-21T19:03:36.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:03:36 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:03:36.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:03:36 smithi155 ceph-mon[97934]: pgmap v694: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.8 MiB/s wr, 429 op/s 2023-07-21T19:03:36.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:03:36 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:03:36.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:03:36 smithi039 ceph-mon[96489]: pgmap v694: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.8 MiB/s wr, 429 op/s 2023-07-21T19:03:36.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:03:36 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:03:37.710 INFO:tasks.workunit.client.0.smithi039.stdout: 10 287898 29.02 MB/sec execute 410 sec latency 189.123 ms 2023-07-21T19:03:38.710 INFO:tasks.workunit.client.0.smithi039.stdout: 10 288256 29.00 MB/sec execute 411 sec latency 172.836 ms 2023-07-21T19:03:38.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:03:38 smithi193 ceph-mon[98204]: pgmap v695: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 22 MiB/s wr, 759 op/s 2023-07-21T19:03:38.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:03:38 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:03:38.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:03:38 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:03:38.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:03:38 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T19:03:38.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:03:38 smithi155 ceph-mon[97934]: pgmap v695: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 22 MiB/s wr, 759 op/s 2023-07-21T19:03:38.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:03:38 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:03:38.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:03:38 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:03:38.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:03:38 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T19:03:38.926 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:03:38 smithi039 ceph-mon[96489]: pgmap v695: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 22 MiB/s wr, 759 op/s 2023-07-21T19:03:38.927 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:03:38 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:03:38.927 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:03:38 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:03:38.927 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:03:38 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T19:03:39.710 INFO:tasks.workunit.client.0.smithi039.stdout: 10 288733 28.99 MB/sec execute 412 sec latency 177.418 ms 2023-07-21T19:03:40.710 INFO:tasks.workunit.client.0.smithi039.stdout: 10 289246 28.98 MB/sec execute 413 sec latency 147.111 ms 2023-07-21T19:03:40.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:03:40 smithi193 ceph-mon[98204]: pgmap v696: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 18 MiB/s wr, 559 op/s 2023-07-21T19:03:40.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:03:40 smithi155 ceph-mon[97934]: pgmap v696: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 18 MiB/s wr, 559 op/s 2023-07-21T19:03:40.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:03:40 smithi039 ceph-mon[96489]: pgmap v696: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 18 MiB/s wr, 559 op/s 2023-07-21T19:03:41.451 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.440+0000 7f2d21449700 1 -- 172.21.15.39:0/2942619375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2d1c101fa0 msgr2=0x7f2d1c102370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:03:41.451 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.440+0000 7f2d21449700 1 --2- 172.21.15.39:0/2942619375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2d1c101fa0 0x7f2d1c102370 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2d100099c0 tx=0x7f2d10009cd0 comp rx=0 tx=0).stop 2023-07-21T19:03:41.452 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.440+0000 7f2d21449700 1 -- 172.21.15.39:0/2942619375 shutdown_connections 2023-07-21T19:03:41.452 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.440+0000 7f2d21449700 1 --2- 172.21.15.39:0/2942619375 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d1c106fe0 0x7f2d1c10ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:41.452 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.440+0000 7f2d21449700 1 --2- 172.21.15.39:0/2942619375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2d1c101fa0 0x7f2d1c102370 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:41.453 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.440+0000 7f2d21449700 1 --2- 172.21.15.39:0/2942619375 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d1c102940 0x7f2d1c106900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:41.453 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.440+0000 7f2d21449700 1 -- 172.21.15.39:0/2942619375 >> 172.21.15.39:0/2942619375 conn(0x7f2d1c0fd830 msgr2=0x7f2d1c0ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:03:41.456 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.442+0000 7f2d21449700 1 -- 172.21.15.39:0/2942619375 shutdown_connections 2023-07-21T19:03:41.456 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.442+0000 7f2d21449700 1 -- 172.21.15.39:0/2942619375 wait complete. 2023-07-21T19:03:41.456 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.443+0000 7f2d21449700 1 Processor -- start 2023-07-21T19:03:41.457 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.443+0000 7f2d21449700 1 -- start start 2023-07-21T19:03:41.457 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.443+0000 7f2d21449700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d1c101fa0 0x7f2d1c197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:41.457 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.443+0000 7f2d21449700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2d1c102940 0x7f2d1c198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:41.457 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.444+0000 7f2d21449700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d1c106fe0 0x7f2d1c19c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:41.458 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.444+0000 7f2d21449700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1c19d000 con 0x7f2d1c101fa0 2023-07-21T19:03:41.458 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.444+0000 7f2d21449700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1c19d170 con 0x7f2d1c106fe0 2023-07-21T19:03:41.458 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.444+0000 7f2d21449700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1c19d330 con 0x7f2d1c102940 2023-07-21T19:03:41.460 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.444+0000 7f2d1affd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d1c101fa0 0x7f2d1c197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:41.460 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.444+0000 7f2d1affd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d1c101fa0 0x7f2d1c197e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34572/0 (socket says 172.21.15.39:34572) 2023-07-21T19:03:41.460 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.444+0000 7f2d1affd700 1 -- 172.21.15.39:0/1406569686 learned_addr learned my addr 172.21.15.39:0/1406569686 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:03:41.461 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.444+0000 7f2d1b7fe700 1 --2- 172.21.15.39:0/1406569686 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d1c106fe0 0x7f2d1c19c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:41.461 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.444+0000 7f2d1a7fc700 1 --2- 172.21.15.39:0/1406569686 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2d1c102940 0x7f2d1c198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:41.461 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.444+0000 7f2d1affd700 1 -- 172.21.15.39:0/1406569686 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2d1c102940 msgr2=0x7f2d1c198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:03:41.461 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.444+0000 7f2d1affd700 1 --2- 172.21.15.39:0/1406569686 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2d1c102940 0x7f2d1c198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:41.461 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.444+0000 7f2d1affd700 1 -- 172.21.15.39:0/1406569686 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d1c106fe0 msgr2=0x7f2d1c19c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:03:41.461 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.444+0000 7f2d1affd700 1 --2- 172.21.15.39:0/1406569686 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d1c106fe0 0x7f2d1c19c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:41.462 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.444+0000 7f2d1affd700 1 -- 172.21.15.39:0/1406569686 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d100095f0 con 0x7f2d1c101fa0 2023-07-21T19:03:41.462 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.444+0000 7f2d1a7fc700 1 --2- 172.21.15.39:0/1406569686 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2d1c102940 0x7f2d1c198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:03:41.462 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.444+0000 7f2d1affd700 1 --2- 172.21.15.39:0/1406569686 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d1c101fa0 0x7f2d1c197e10 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f2d10005b40 tx=0x7f2d10005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:03:41.462 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.444+0000 7f2d03fff700 1 -- 172.21.15.39:0/1406569686 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1001d070 con 0x7f2d1c101fa0 2023-07-21T19:03:41.463 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.445+0000 7f2d21449700 1 -- 172.21.15.39:0/1406569686 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d1c19d5b0 con 0x7f2d1c101fa0 2023-07-21T19:03:41.463 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.445+0000 7f2d21449700 1 -- 172.21.15.39:0/1406569686 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d1c074b30 con 0x7f2d1c101fa0 2023-07-21T19:03:41.463 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.445+0000 7f2d03fff700 1 -- 172.21.15.39:0/1406569686 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d100040a0 con 0x7f2d1c101fa0 2023-07-21T19:03:41.464 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.445+0000 7f2d03fff700 1 -- 172.21.15.39:0/1406569686 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d10022dd0 con 0x7f2d1c101fa0 2023-07-21T19:03:41.464 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.448+0000 7f2d03fff700 1 -- 172.21.15.39:0/1406569686 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f2d1000f460 con 0x7f2d1c101fa0 2023-07-21T19:03:41.464 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.448+0000 7f2d03fff700 1 --2- 172.21.15.39:0/1406569686 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f2d040b2350 0x7f2d040b4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:41.464 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.449+0000 7f2d03fff700 1 -- 172.21.15.39:0/1406569686 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f2d100cee60 con 0x7f2d1c101fa0 2023-07-21T19:03:41.465 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.449+0000 7f2d1a7fc700 1 --2- 172.21.15.39:0/1406569686 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f2d040b2350 0x7f2d040b4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:41.710 INFO:tasks.workunit.client.0.smithi039.stdout: 10 289803 28.98 MB/sec execute 414 sec latency 103.506 ms 2023-07-21T19:03:41.714 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.703+0000 7f2d1a7fc700 1 --2- 172.21.15.39:0/1406569686 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f2d040b2350 0x7f2d040b4800 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c199630 tx=0x7f2d0c008040 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:03:41.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.704+0000 7f2d2d5cfb80 1 Processor -- start 2023-07-21T19:03:41.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.705+0000 7f2d2d5cfb80 1 -- start start 2023-07-21T19:03:41.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.705+0000 7f2d2d5cfb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55963b8266b0 0x55963b9cfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:41.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.705+0000 7f2d2d5cfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55963b9d02d0 0x55963b9ca040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:41.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.705+0000 7f2d2d5cfb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55963b9ca580 0x55963b9ca9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:41.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.705+0000 7f2d2d5cfb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55963b8cfc50 con 0x55963b8266b0 2023-07-21T19:03:41.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.705+0000 7f2d2d5cfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55963b9229e0 con 0x55963b9d02d0 2023-07-21T19:03:41.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.705+0000 7f2d2d5cfb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55963b91be70 con 0x55963b9ca580 2023-07-21T19:03:41.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.705+0000 7f2d1affd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55963b8266b0 0x55963b9cfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:41.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.705+0000 7f2d1a7fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55963b9d02d0 0x55963b9ca040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:41.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.705+0000 7f2d1affd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55963b8266b0 0x55963b9cfd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34580/0 (socket says 172.21.15.39:34580) 2023-07-21T19:03:41.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.705+0000 7f2d1affd700 1 -- 172.21.15.39:0/2777705783 learned_addr learned my addr 172.21.15.39:0/2777705783 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:03:41.720 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.705+0000 7f2d1a7fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55963b9d02d0 0x55963b9ca040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:47176/0 (socket says 172.21.15.39:47176) 2023-07-21T19:03:41.720 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.705+0000 7f2d1b7fe700 1 --2- 172.21.15.39:0/2777705783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55963b9ca580 0x55963b9ca9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:41.720 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.705+0000 7f2d1affd700 1 -- 172.21.15.39:0/2777705783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55963b9ca580 msgr2=0x55963b9ca9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:03:41.720 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.705+0000 7f2d1affd700 1 --2- 172.21.15.39:0/2777705783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55963b9ca580 0x55963b9ca9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:41.720 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.705+0000 7f2d1affd700 1 -- 172.21.15.39:0/2777705783 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55963b9d02d0 msgr2=0x55963b9ca040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:03:41.720 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.705+0000 7f2d1affd700 1 --2- 172.21.15.39:0/2777705783 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55963b9d02d0 0x55963b9ca040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:41.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.705+0000 7f2d1affd700 1 -- 172.21.15.39:0/2777705783 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55963b749d80 con 0x55963b8266b0 2023-07-21T19:03:41.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.706+0000 7f2d1affd700 1 --2- 172.21.15.39:0/2777705783 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55963b8266b0 0x55963b9cfd90 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f2d10000c00 tx=0x7f2d10005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7a2772737f27179 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:03:41.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.706+0000 7f2d017fa700 1 -- 172.21.15.39:0/2777705783 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1009fff0 con 0x55963b8266b0 2023-07-21T19:03:41.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.706+0000 7f2d017fa700 1 -- 172.21.15.39:0/2777705783 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d100a4380 con 0x55963b8266b0 2023-07-21T19:03:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.706+0000 7f2d017fa700 1 -- 172.21.15.39:0/2777705783 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d10096e30 con 0x55963b8266b0 2023-07-21T19:03:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.707+0000 7f2d2d5cfb80 1 -- 172.21.15.39:0/2777705783 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55963b8266b0 msgr2=0x55963b9cfd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:03:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.707+0000 7f2d2d5cfb80 1 --2- 172.21.15.39:0/2777705783 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55963b8266b0 0x55963b9cfd90 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f2d10000c00 tx=0x7f2d10005fd0 comp rx=0 tx=0).stop 2023-07-21T19:03:41.726 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.708+0000 7f2d2d5cfb80 1 -- 172.21.15.39:0/2777705783 shutdown_connections 2023-07-21T19:03:41.726 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.708+0000 7f2d2d5cfb80 1 --2- 172.21.15.39:0/2777705783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55963b9ca580 0x55963b9ca9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:41.726 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.708+0000 7f2d2d5cfb80 1 --2- 172.21.15.39:0/2777705783 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55963b8266b0 0x55963b9cfd90 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:41.727 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.708+0000 7f2d2d5cfb80 1 --2- 172.21.15.39:0/2777705783 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55963b9d02d0 0x55963b9ca040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:41.727 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.708+0000 7f2d2d5cfb80 1 -- 172.21.15.39:0/2777705783 >> 172.21.15.39:0/2777705783 conn(0x55963b9bfbe0 msgr2=0x55963b9c4f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:03:41.727 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.708+0000 7f2d2d5cfb80 1 -- 172.21.15.39:0/2777705783 shutdown_connections 2023-07-21T19:03:41.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.708+0000 7f2d2d5cfb80 1 -- 172.21.15.39:0/2777705783 wait complete. 2023-07-21T19:03:41.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.708+0000 7f2d2d5cfb80 1 -- start start 2023-07-21T19:03:41.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.708+0000 7f2d2d5cfb80 1 Processor -- start 2023-07-21T19:03:41.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.708+0000 7f2d2d5cfb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55963b8266b0 0x55963b9d92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:41.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.709+0000 7f2d2d5cfb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55963b9ca580 0x55963b9d9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:41.730 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.709+0000 7f2d2d5cfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55963b9d02d0 0x55963b9d3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:41.730 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.709+0000 7f2d2d5cfb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55963b8bba60 con 0x55963b9ca580 2023-07-21T19:03:41.731 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.709+0000 7f2d2d5cfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55963b8bb760 con 0x55963b9d02d0 2023-07-21T19:03:41.731 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.709+0000 7f2d2d5cfb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55963b79f8c0 con 0x55963b8266b0 2023-07-21T19:03:41.731 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.709+0000 7f2d2d5cfb80 10 client.0 _ll_register_callbacks cb 0x55963b8b9700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:03:41.732 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.709+0000 7f2d1affd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55963b8266b0 0x55963b9d92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:41.732 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.709+0000 7f2d1affd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55963b8266b0 0x55963b9d92c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:53136/0 (socket says 172.21.15.39:53136) 2023-07-21T19:03:41.732 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.709+0000 7f2d1affd700 1 -- 172.21.15.39:0/1273651339 learned_addr learned my addr 172.21.15.39:0/1273651339 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:03:41.732 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.709+0000 7f2d1a7fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55963b9ca580 0x55963b9d9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:41.733 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.709+0000 7f2d1b7fe700 1 --2- 172.21.15.39:0/1273651339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55963b9d02d0 0x55963b9d3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:41.733 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.709+0000 7f2d1affd700 1 -- 172.21.15.39:0/1273651339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55963b9d02d0 msgr2=0x55963b9d3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:03:41.733 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.709+0000 7f2d1affd700 1 --2- 172.21.15.39:0/1273651339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55963b9d02d0 0x55963b9d3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:41.733 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.709+0000 7f2d1affd700 1 -- 172.21.15.39:0/1273651339 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55963b9ca580 msgr2=0x55963b9d9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:03:41.733 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.709+0000 7f2d1affd700 1 --2- 172.21.15.39:0/1273651339 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55963b9ca580 0x55963b9d9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:41.734 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.709+0000 7f2d1affd700 1 -- 172.21.15.39:0/1273651339 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d1c074b30 con 0x55963b8266b0 2023-07-21T19:03:41.734 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.709+0000 7f2d1affd700 1 --2- 172.21.15.39:0/1273651339 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55963b8266b0 0x55963b9d92c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2d1009bf50 tx=0x7f2d1009b760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85a28486cacd76ff server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:03:41.734 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.709+0000 7f2cfeff5700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2023-07-21T19:03:41.734 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.709+0000 7f2cfeff5700 1 -- 172.21.15.39:0/1273651339 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1009fff0 con 0x55963b8266b0 2023-07-21T19:03:41.734 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.709+0000 7f2cfeff5700 1 -- 172.21.15.39:0/1273651339 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f2d10096de0 con 0x55963b8266b0 2023-07-21T19:03:41.734 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.710+0000 7f2d2d5cfb80 1 -- 172.21.15.39:0/1273651339 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55963b875490 con 0x55963b8266b0 2023-07-21T19:03:41.734 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.710+0000 7f2cfeff5700 1 -- 172.21.15.39:0/1273651339 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d100ba300 con 0x55963b8266b0 2023-07-21T19:03:41.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.710+0000 7f2d2d5cfb80 10 client.27238 fetch_fsmap learned FSMap version 103 2023-07-21T19:03:41.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.710+0000 7f2d2d5cfb80 1 -- 172.21.15.39:0/1273651339 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=4+}) v3 -- 0x55963b732c00 con 0x55963b8266b0 2023-07-21T19:03:41.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.710+0000 7f2cfeff5700 1 -- 172.21.15.39:0/1273651339 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55963b875490 con 0x55963b8266b0 2023-07-21T19:03:41.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.710+0000 7f2cfeff5700 1 -- 172.21.15.39:0/1273651339 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 103) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f2d100ba020 con 0x55963b8266b0 2023-07-21T19:03:41.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.710+0000 7f2d2d5cfb80 10 client.27238 fetch_fsmap finished waiting for FSMap version 103 2023-07-21T19:03:41.736 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.710+0000 7f2d2d5cfb80 10 client.27238 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:03:41.736 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.710+0000 7f2d2d5cfb80 20 client.27238 populate_metadata read hostname 'smithi039' 2023-07-21T19:03:41.736 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.710+0000 7f2d2d5cfb80 1 --2- 172.21.15.39:0/1273651339 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55963b9e32d0 0x55963b9e56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:41.736 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.710+0000 7f2d2d5cfb80 4 client.27238 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:03:41.736 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.710+0000 7f2d2d5cfb80 1 -- 172.21.15.39:0/1273651339 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55963b914e50 con 0x55963b9e32d0 2023-07-21T19:03:41.736 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.710+0000 7f2d1a7fc700 1 --2- 172.21.15.39:0/1273651339 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55963b9e32d0 0x55963b9e56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:41.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.711+0000 7f2cfeff5700 10 client.27238 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T19:03:41.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.711+0000 7f2d1a7fc700 1 --2- 172.21.15.39:0/1273651339 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55963b9e32d0 0x55963b9e56b0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e37f95728c5e8617 server_cookie=bc43c91d9b854afa in_seq=0 out_seq=0 2023-07-21T19:03:41.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.714+0000 7f2cfeff5700 1 -- 172.21.15.39:0/1273651339 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55963b914e50 con 0x55963b9e32d0 2023-07-21T19:03:41.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.714+0000 7f2cfeff5700 10 client.27238 handle_command_reply: tid=0 2023-07-21T19:03:41.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.714+0000 7f2d2d5cfb80 1 client.27238 shutdown 2023-07-21T19:03:41.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.714+0000 7f2d2d5cfb80 1 -- 172.21.15.39:0/1273651339 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55963b8266b0 msgr2=0x55963b9d92c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:03:41.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.714+0000 7f2d2d5cfb80 1 --2- 172.21.15.39:0/1273651339 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55963b8266b0 0x55963b9d92c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2d1009bf50 tx=0x7f2d1009b760 comp rx=0 tx=0).stop 2023-07-21T19:03:41.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.714+0000 7f2d2d5cfb80 1 -- 172.21.15.39:0/1273651339 shutdown_connections 2023-07-21T19:03:41.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.714+0000 7f2d2d5cfb80 1 --2- 172.21.15.39:0/1273651339 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55963b9e32d0 0x55963b9e56b0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:41.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.714+0000 7f2d2d5cfb80 1 --2- 172.21.15.39:0/1273651339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55963b9d02d0 0x55963b9d3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:41.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.714+0000 7f2cfeff5700 0 client.27238 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T19:03:41.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.714+0000 7f2d2d5cfb80 1 --2- 172.21.15.39:0/1273651339 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55963b8266b0 0x55963b9d92c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:41.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.714+0000 7f2d2d5cfb80 1 --2- 172.21.15.39:0/1273651339 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55963b9ca580 0x55963b9d9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:41.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.714+0000 7f2d2d5cfb80 1 -- 172.21.15.39:0/1273651339 >> 172.21.15.39:0/1273651339 conn(0x55963b9bfbe0 msgr2=0x55963b9c44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:03:41.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.714+0000 7f2d2d5cfb80 1 -- 172.21.15.39:0/1273651339 shutdown_connections 2023-07-21T19:03:41.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.714+0000 7f2d2d5cfb80 1 -- 172.21.15.39:0/1273651339 wait complete. 2023-07-21T19:03:41.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.714+0000 7f2d2d5cfb80 20 client.27238 trim_cache size 0 max 16384 2023-07-21T19:03:41.747 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.736+0000 7f2d2d5cfb80 1 Processor -- start 2023-07-21T19:03:41.747 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.736+0000 7f2d2d5cfb80 1 -- start start 2023-07-21T19:03:41.747 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.736+0000 7f2d2d5cfb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55963b8266b0 0x55963b9ca040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:41.747 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.736+0000 7f2d2d5cfb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55963b9ca580 0x55963b9d1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:41.747 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.736+0000 7f2d2d5cfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55963b9d02d0 0x55963b9ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:41.748 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.736+0000 7f2d2d5cfb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55963b9c1d00 con 0x55963b8266b0 2023-07-21T19:03:41.748 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.736+0000 7f2d2d5cfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55963b9d49b0 con 0x55963b9d02d0 2023-07-21T19:03:41.748 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.736+0000 7f2d2d5cfb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55963b9cafd0 con 0x55963b9ca580 2023-07-21T19:03:41.750 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.736+0000 7f2d1affd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55963b8266b0 0x55963b9ca040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:41.751 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.737+0000 7f2d1affd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55963b8266b0 0x55963b9ca040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34592/0 (socket says 172.21.15.39:34592) 2023-07-21T19:03:41.751 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.737+0000 7f2d1affd700 1 -- 172.21.15.39:0/1698135546 learned_addr learned my addr 172.21.15.39:0/1698135546 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:03:41.751 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.737+0000 7f2d1b7fe700 1 --2- 172.21.15.39:0/1698135546 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55963b9d02d0 0x55963b9ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:41.751 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.737+0000 7f2d1a7fc700 1 --2- 172.21.15.39:0/1698135546 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55963b9ca580 0x55963b9d1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:41.751 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.737+0000 7f2d1affd700 1 -- 172.21.15.39:0/1698135546 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55963b9ca580 msgr2=0x55963b9d1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:03:41.751 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.737+0000 7f2d1affd700 1 --2- 172.21.15.39:0/1698135546 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55963b9ca580 0x55963b9d1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:41.752 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.737+0000 7f2d1affd700 1 -- 172.21.15.39:0/1698135546 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55963b9d02d0 msgr2=0x55963b9ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:03:41.752 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.737+0000 7f2d1affd700 1 --2- 172.21.15.39:0/1698135546 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55963b9d02d0 0x55963b9ceff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:41.752 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.737+0000 7f2d1affd700 1 -- 172.21.15.39:0/1698135546 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55963b9ea2e0 con 0x55963b8266b0 2023-07-21T19:03:41.752 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.737+0000 7f2d1b7fe700 1 --2- 172.21.15.39:0/1698135546 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55963b9d02d0 0x55963b9ceff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:03:41.752 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.737+0000 7f2d1a7fc700 1 --2- 172.21.15.39:0/1698135546 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55963b9ca580 0x55963b9d1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:03:41.752 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.737+0000 7f2d1affd700 1 --2- 172.21.15.39:0/1698135546 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55963b8266b0 0x55963b9ca040 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f2d10012040 tx=0x7f2d10005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c53c4d38df084f0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:03:41.752 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.737+0000 7f2d01ffb700 1 -- 172.21.15.39:0/1698135546 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1009fff0 con 0x55963b8266b0 2023-07-21T19:03:41.753 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.738+0000 7f2d01ffb700 1 -- 172.21.15.39:0/1698135546 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d100980c0 con 0x55963b8266b0 2023-07-21T19:03:41.753 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.738+0000 7f2d01ffb700 1 -- 172.21.15.39:0/1698135546 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d10022470 con 0x55963b8266b0 2023-07-21T19:03:41.753 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.738+0000 7f2d2d5cfb80 1 -- 172.21.15.39:0/1698135546 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55963b8266b0 msgr2=0x55963b9ca040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:03:41.753 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.738+0000 7f2d2d5cfb80 1 --2- 172.21.15.39:0/1698135546 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55963b8266b0 0x55963b9ca040 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f2d10012040 tx=0x7f2d10005fd0 comp rx=0 tx=0).stop 2023-07-21T19:03:41.753 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.738+0000 7f2d2d5cfb80 1 -- 172.21.15.39:0/1698135546 shutdown_connections 2023-07-21T19:03:41.753 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.738+0000 7f2d2d5cfb80 1 --2- 172.21.15.39:0/1698135546 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55963b9d02d0 0x55963b9ceff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:41.753 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.738+0000 7f2d2d5cfb80 1 --2- 172.21.15.39:0/1698135546 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55963b8266b0 0x55963b9ca040 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:41.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.738+0000 7f2d2d5cfb80 1 --2- 172.21.15.39:0/1698135546 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55963b9ca580 0x55963b9d1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:41.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.738+0000 7f2d2d5cfb80 1 -- 172.21.15.39:0/1698135546 >> 172.21.15.39:0/1698135546 conn(0x55963b9bfbe0 msgr2=0x55963b9ce8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:03:41.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.738+0000 7f2d2d5cfb80 1 -- 172.21.15.39:0/1698135546 shutdown_connections 2023-07-21T19:03:41.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.738+0000 7f2d2d5cfb80 1 -- 172.21.15.39:0/1698135546 wait complete. 2023-07-21T19:03:41.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.739+0000 7f2d2d5cfb80 1 -- start start 2023-07-21T19:03:41.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.739+0000 7f2d2d5cfb80 1 Processor -- start 2023-07-21T19:03:41.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.739+0000 7f2d2d5cfb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55963b8266b0 0x55963b9d6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:41.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.739+0000 7f2d2d5cfb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55963b9ca580 0x55963b9d7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:41.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.739+0000 7f2d2d5cfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55963b9d02d0 0x55963ba36160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:41.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.739+0000 7f2d2d5cfb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55963b9d7ba0 con 0x55963b9ca580 2023-07-21T19:03:41.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.739+0000 7f2d2d5cfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55963b9d7d10 con 0x55963b9d02d0 2023-07-21T19:03:41.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.739+0000 7f2d2d5cfb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55963ba366a0 con 0x55963b8266b0 2023-07-21T19:03:41.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.739+0000 7f2d2d5cfb80 10 client.0 _ll_register_callbacks cb 0x55963b8b9700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:03:41.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.739+0000 7f2d1a7fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55963b9ca580 0x55963b9d7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:41.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.739+0000 7f2d1affd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55963b8266b0 0x55963b9d6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:41.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.739+0000 7f2d1a7fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55963b9ca580 0x55963b9d7410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34600/0 (socket says 172.21.15.39:34600) 2023-07-21T19:03:41.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.740+0000 7f2d1a7fc700 1 -- 172.21.15.39:0/1691560324 learned_addr learned my addr 172.21.15.39:0/1691560324 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:03:41.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.739+0000 7f2d1affd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55963b8266b0 0x55963b9d6ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:53150/0 (socket says 172.21.15.39:53150) 2023-07-21T19:03:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.740+0000 7f2d1b7fe700 1 --2- 172.21.15.39:0/1691560324 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55963b9d02d0 0x55963ba36160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.740+0000 7f2d1a7fc700 1 -- 172.21.15.39:0/1691560324 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55963b8266b0 msgr2=0x55963b9d6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:03:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.740+0000 7f2d1a7fc700 1 --2- 172.21.15.39:0/1691560324 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55963b8266b0 0x55963b9d6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.740+0000 7f2d1a7fc700 1 -- 172.21.15.39:0/1691560324 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55963b9d02d0 msgr2=0x55963ba36160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:03:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.740+0000 7f2d1a7fc700 1 --2- 172.21.15.39:0/1691560324 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55963b9d02d0 0x55963ba36160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.740+0000 7f2d1a7fc700 1 -- 172.21.15.39:0/1691560324 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d1400f040 con 0x55963b9ca580 2023-07-21T19:03:41.758 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.740+0000 7f2d1a7fc700 1 --2- 172.21.15.39:0/1691560324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55963b9ca580 0x55963b9d7410 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c069ad0 tx=0x7f2d0c00f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3950d3f2ee0dd33 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:03:41.758 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.740+0000 7f2cfeff5700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T19:03:41.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.740+0000 7f2cfeff5700 1 -- 172.21.15.39:0/1691560324 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c027040 con 0x55963b9ca580 2023-07-21T19:03:41.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.740+0000 7f2cfeff5700 1 -- 172.21.15.39:0/1691560324 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f2d0c027bd0 con 0x55963b9ca580 2023-07-21T19:03:41.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.740+0000 7f2d2d5cfb80 1 -- 172.21.15.39:0/1691560324 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2d100a4380 con 0x55963b9ca580 2023-07-21T19:03:41.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.740+0000 7f2cfeff5700 1 -- 172.21.15.39:0/1691560324 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d0c007eb0 con 0x55963b9ca580 2023-07-21T19:03:41.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.740+0000 7f2d2d5cfb80 10 client.17739 fetch_fsmap learned FSMap version 103 2023-07-21T19:03:41.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.740+0000 7f2d2d5cfb80 1 -- 172.21.15.39:0/1691560324 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=4+}) v3 -- 0x55963b732c00 con 0x55963b9ca580 2023-07-21T19:03:41.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.741+0000 7f2cfeff5700 1 -- 172.21.15.39:0/1691560324 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d100a4380 con 0x55963b9ca580 2023-07-21T19:03:41.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.741+0000 7f2cfeff5700 1 -- 172.21.15.39:0/1691560324 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 103) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f2d0c039990 con 0x55963b9ca580 2023-07-21T19:03:41.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.741+0000 7f2d2d5cfb80 10 client.17739 fetch_fsmap finished waiting for FSMap version 103 2023-07-21T19:03:41.761 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.741+0000 7f2d2d5cfb80 10 client.17739 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:03:41.761 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.741+0000 7f2d2d5cfb80 20 client.17739 populate_metadata read hostname 'smithi039' 2023-07-21T19:03:41.761 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.741+0000 7f2d2d5cfb80 1 --2- 172.21.15.39:0/1691560324 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55963ba37ea0 0x55963ba3a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:41.761 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.741+0000 7f2d2d5cfb80 4 client.17739 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:03:41.761 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.741+0000 7f2d2d5cfb80 1 -- 172.21.15.39:0/1691560324 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55963b8d6ff0 con 0x55963ba37ea0 2023-07-21T19:03:41.761 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.741+0000 7f2d1affd700 1 --2- 172.21.15.39:0/1691560324 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55963ba37ea0 0x55963ba3a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:41.761 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.742+0000 7f2d1affd700 1 --2- 172.21.15.39:0/1691560324 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55963ba37ea0 0x55963ba3a2b0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4247da94eda07702 server_cookie=f62753c8c741e1bb in_seq=0 out_seq=0 2023-07-21T19:03:41.762 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.742+0000 7f2cfeff5700 10 client.17739 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T19:03:41.949 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.938+0000 7f2cfeff5700 1 -- 172.21.15.39:0/1691560324 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55963b8d6ff0 con 0x55963ba37ea0 2023-07-21T19:03:41.949 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.938+0000 7f2cfeff5700 10 client.17739 handle_command_reply: tid=0 2023-07-21T19:03:41.949 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.938+0000 7f2d2d5cfb80 1 client.17739 shutdown 2023-07-21T19:03:41.949 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.938+0000 7f2d2d5cfb80 1 -- 172.21.15.39:0/1691560324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55963b9ca580 msgr2=0x55963b9d7410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:03:41.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.938+0000 7f2d2d5cfb80 1 --2- 172.21.15.39:0/1691560324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55963b9ca580 0x55963b9d7410 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c069ad0 tx=0x7f2d0c00f000 comp rx=0 tx=0).stop 2023-07-21T19:03:41.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.938+0000 7f2d2d5cfb80 1 -- 172.21.15.39:0/1691560324 shutdown_connections 2023-07-21T19:03:41.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.938+0000 7f2d2d5cfb80 1 --2- 172.21.15.39:0/1691560324 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55963ba37ea0 0x55963ba3a2b0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:41.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.938+0000 7f2d2d5cfb80 1 --2- 172.21.15.39:0/1691560324 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55963b9d02d0 0x55963ba36160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:41.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.938+0000 7f2d2d5cfb80 1 --2- 172.21.15.39:0/1691560324 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55963b8266b0 0x55963b9d6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:41.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.938+0000 7f2d2d5cfb80 1 --2- 172.21.15.39:0/1691560324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55963b9ca580 0x55963b9d7410 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:41.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.938+0000 7f2d2d5cfb80 1 -- 172.21.15.39:0/1691560324 >> 172.21.15.39:0/1691560324 conn(0x55963b9bfbe0 msgr2=0x55963b9cf170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:03:41.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.938+0000 7f2cfeff5700 0 client.17739 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T19:03:41.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.938+0000 7f2d2d5cfb80 1 -- 172.21.15.39:0/1691560324 shutdown_connections 2023-07-21T19:03:41.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.938+0000 7f2d2d5cfb80 1 -- 172.21.15.39:0/1691560324 wait complete. 2023-07-21T19:03:41.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.938+0000 7f2d2d5cfb80 20 client.17739 trim_cache size 0 max 16384 2023-07-21T19:03:41.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T19:03:41.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T19:03:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T19:03:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T19:03:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.939+0000 7f2d21449700 1 -- 172.21.15.39:0/1406569686 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f2d040b2350 msgr2=0x7f2d040b4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:03:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.939+0000 7f2d21449700 1 --2- 172.21.15.39:0/1406569686 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f2d040b2350 0x7f2d040b4800 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c199630 tx=0x7f2d0c008040 comp rx=0 tx=0).stop 2023-07-21T19:03:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.939+0000 7f2d21449700 1 -- 172.21.15.39:0/1406569686 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d1c101fa0 msgr2=0x7f2d1c197e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:03:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.939+0000 7f2d21449700 1 --2- 172.21.15.39:0/1406569686 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d1c101fa0 0x7f2d1c197e10 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f2d10005b40 tx=0x7f2d10005740 comp rx=0 tx=0).stop 2023-07-21T19:03:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.939+0000 7f2d21449700 1 -- 172.21.15.39:0/1406569686 shutdown_connections 2023-07-21T19:03:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.939+0000 7f2d21449700 1 --2- 172.21.15.39:0/1406569686 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f2d040b2350 0x7f2d040b4800 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.939+0000 7f2d21449700 1 --2- 172.21.15.39:0/1406569686 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d1c106fe0 0x7f2d1c19c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.939+0000 7f2d21449700 1 --2- 172.21.15.39:0/1406569686 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d1c101fa0 0x7f2d1c197e10 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.939+0000 7f2d21449700 1 --2- 172.21.15.39:0/1406569686 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2d1c102940 0x7f2d1c198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.939+0000 7f2d21449700 1 -- 172.21.15.39:0/1406569686 >> 172.21.15.39:0/1406569686 conn(0x7f2d1c0fd830 msgr2=0x7f2d1c0ff2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:03:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.939+0000 7f2d21449700 1 -- 172.21.15.39:0/1406569686 shutdown_connections 2023-07-21T19:03:41.955 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:41.939+0000 7f2d21449700 1 -- 172.21.15.39:0/1406569686 wait complete. 2023-07-21T19:03:42.710 INFO:tasks.workunit.client.0.smithi039.stdout: 10 290338 28.97 MB/sec execute 415 sec latency 145.240 ms 2023-07-21T19:03:42.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:03:42 smithi193 ceph-mon[98204]: pgmap v697: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 18 MiB/s wr, 587 op/s 2023-07-21T19:03:42.879 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:03:42 smithi155 ceph-mon[97934]: pgmap v697: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 18 MiB/s wr, 587 op/s 2023-07-21T19:03:42.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:03:42 smithi039 ceph-mon[96489]: pgmap v697: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 18 MiB/s wr, 587 op/s 2023-07-21T19:03:43.710 INFO:tasks.workunit.client.0.smithi039.stdout: 10 290901 28.98 MB/sec execute 416 sec latency 83.181 ms 2023-07-21T19:03:44.710 INFO:tasks.workunit.client.0.smithi039.stdout: 10 291362 28.98 MB/sec execute 417 sec latency 125.447 ms 2023-07-21T19:03:44.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:03:44 smithi193 ceph-mon[98204]: pgmap v698: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 21 MiB/s wr, 854 op/s 2023-07-21T19:03:44.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:03:44 smithi155 ceph-mon[97934]: pgmap v698: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 21 MiB/s wr, 854 op/s 2023-07-21T19:03:44.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:03:44 smithi039 ceph-mon[96489]: pgmap v698: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 21 MiB/s wr, 854 op/s 2023-07-21T19:03:45.710 INFO:tasks.workunit.client.0.smithi039.stdout: 10 291941 28.98 MB/sec execute 418 sec latency 78.289 ms 2023-07-21T19:03:46.710 INFO:tasks.workunit.client.0.smithi039.stdout: 10 292456 28.98 MB/sec execute 419 sec latency 105.518 ms 2023-07-21T19:03:46.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:03:46 smithi193 ceph-mon[98204]: pgmap v699: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 17 MiB/s wr, 649 op/s 2023-07-21T19:03:46.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:03:46 smithi155 ceph-mon[97934]: pgmap v699: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 17 MiB/s wr, 649 op/s 2023-07-21T19:03:46.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:03:46 smithi039 ceph-mon[96489]: pgmap v699: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 17 MiB/s wr, 649 op/s 2023-07-21T19:03:47.710 INFO:tasks.workunit.client.0.smithi039.stdout: 10 292922 28.97 MB/sec execute 420 sec latency 93.888 ms 2023-07-21T19:03:48.710 INFO:tasks.workunit.client.0.smithi039.stdout: 10 293487 28.95 MB/sec execute 421 sec latency 122.845 ms 2023-07-21T19:03:48.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:03:48 smithi193 ceph-mon[98204]: pgmap v700: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 21 MiB/s wr, 862 op/s 2023-07-21T19:03:48.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:03:48 smithi155 ceph-mon[97934]: pgmap v700: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 21 MiB/s wr, 862 op/s 2023-07-21T19:03:48.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:03:48 smithi039 ceph-mon[96489]: pgmap v700: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 21 MiB/s wr, 862 op/s 2023-07-21T19:03:49.711 INFO:tasks.workunit.client.0.smithi039.stdout: 10 293982 28.96 MB/sec execute 422 sec latency 160.089 ms 2023-07-21T19:03:50.711 INFO:tasks.workunit.client.0.smithi039.stdout: 10 294555 28.97 MB/sec execute 423 sec latency 76.430 ms 2023-07-21T19:03:50.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:03:50 smithi193 ceph-mon[98204]: pgmap v701: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.6 MiB/s wr, 532 op/s 2023-07-21T19:03:50.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:03:50 smithi155 ceph-mon[97934]: pgmap v701: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.6 MiB/s wr, 532 op/s 2023-07-21T19:03:50.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:03:50 smithi039 ceph-mon[96489]: pgmap v701: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.6 MiB/s wr, 532 op/s 2023-07-21T19:03:51.711 INFO:tasks.workunit.client.0.smithi039.stdout: 10 295145 28.97 MB/sec execute 424 sec latency 62.899 ms 2023-07-21T19:03:52.711 INFO:tasks.workunit.client.0.smithi039.stdout: 10 295721 29.00 MB/sec execute 425 sec latency 66.703 ms 2023-07-21T19:03:52.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:03:52 smithi193 ceph-mon[98204]: pgmap v702: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.1 MiB/s wr, 562 op/s 2023-07-21T19:03:52.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:03:52 smithi155 ceph-mon[97934]: pgmap v702: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.1 MiB/s wr, 562 op/s 2023-07-21T19:03:52.928 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:03:52 smithi039 ceph-mon[96489]: pgmap v702: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.1 MiB/s wr, 562 op/s 2023-07-21T19:03:53.711 INFO:tasks.workunit.client.0.smithi039.stdout: 10 296148 28.97 MB/sec execute 426 sec latency 89.282 ms 2023-07-21T19:03:54.715 INFO:tasks.workunit.client.0.smithi039.stdout: 10 296493 28.93 MB/sec execute 427 sec latency 190.982 ms 2023-07-21T19:03:54.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:03:54 smithi193 ceph-mon[98204]: pgmap v703: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.9 MiB/s wr, 690 op/s 2023-07-21T19:03:54.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:03:54 smithi155 ceph-mon[97934]: pgmap v703: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.9 MiB/s wr, 690 op/s 2023-07-21T19:03:54.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:03:54 smithi039 ceph-mon[96489]: pgmap v703: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.9 MiB/s wr, 690 op/s 2023-07-21T19:03:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.145+0000 7f7b485d0700 1 -- 172.21.15.39:0/597420035 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b400febc0 msgr2=0x7f7b40102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:03:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.145+0000 7f7b485d0700 1 --2- 172.21.15.39:0/597420035 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b400febc0 0x7f7b40102b80 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f7b3001a9d0 tx=0x7f7b3001ace0 comp rx=0 tx=0).stop 2023-07-21T19:03:55.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.149+0000 7f7b485d0700 1 -- 172.21.15.39:0/597420035 shutdown_connections 2023-07-21T19:03:55.160 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.149+0000 7f7b485d0700 1 --2- 172.21.15.39:0/597420035 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b400fe220 0x7f7b400fe5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:55.160 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.149+0000 7f7b485d0700 1 --2- 172.21.15.39:0/597420035 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b40103260 0x7f7b4010ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:55.160 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.149+0000 7f7b485d0700 1 --2- 172.21.15.39:0/597420035 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b400febc0 0x7f7b40102b80 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:55.160 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.149+0000 7f7b485d0700 1 -- 172.21.15.39:0/597420035 >> 172.21.15.39:0/597420035 conn(0x7f7b40073d50 msgr2=0x7f7b40074150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:03:55.160 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.149+0000 7f7b485d0700 1 -- 172.21.15.39:0/597420035 shutdown_connections 2023-07-21T19:03:55.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.149+0000 7f7b485d0700 1 -- 172.21.15.39:0/597420035 wait complete. 2023-07-21T19:03:55.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.150+0000 7f7b485d0700 1 Processor -- start 2023-07-21T19:03:55.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.150+0000 7f7b485d0700 1 -- start start 2023-07-21T19:03:55.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.150+0000 7f7b485d0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b400fe220 0x7f7b40105fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:55.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.150+0000 7f7b485d0700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b400febc0 0x7f7b40106500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:55.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.150+0000 7f7b485d0700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b40103260 0x7f7b40106a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:55.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.150+0000 7f7b485d0700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b401978a0 con 0x7f7b400febc0 2023-07-21T19:03:55.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.150+0000 7f7b485d0700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b40197a10 con 0x7f7b400fe220 2023-07-21T19:03:55.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.150+0000 7f7b485d0700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b40197bd0 con 0x7f7b40103260 2023-07-21T19:03:55.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.150+0000 7f7b4636c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b400fe220 0x7f7b40105fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:55.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.150+0000 7f7b46b6d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b40103260 0x7f7b40106a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:55.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.150+0000 7f7b46b6d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b40103260 0x7f7b40106a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:53166/0 (socket says 172.21.15.39:53166) 2023-07-21T19:03:55.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.150+0000 7f7b4636c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b400fe220 0x7f7b40105fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:47208/0 (socket says 172.21.15.39:47208) 2023-07-21T19:03:55.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.150+0000 7f7b46b6d700 1 -- 172.21.15.39:0/3341692811 learned_addr learned my addr 172.21.15.39:0/3341692811 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:03:55.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.151+0000 7f7b46b6d700 1 -- 172.21.15.39:0/3341692811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b400fe220 msgr2=0x7f7b40105fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:03:55.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.151+0000 7f7b46b6d700 1 --2- 172.21.15.39:0/3341692811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b400fe220 0x7f7b40105fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:55.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.151+0000 7f7b46b6d700 1 -- 172.21.15.39:0/3341692811 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b400febc0 msgr2=0x7f7b40106500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-21T19:03:55.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.151+0000 7f7b46b6d700 1 --2- 172.21.15.39:0/3341692811 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b400febc0 0x7f7b40106500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:55.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.151+0000 7f7b46b6d700 1 -- 172.21.15.39:0/3341692811 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b3001a600 con 0x7f7b40103260 2023-07-21T19:03:55.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.151+0000 7f7b46b6d700 1 --2- 172.21.15.39:0/3341692811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b40103260 0x7f7b40106a80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c00e940 tx=0x7f7b3c00ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:03:55.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.151+0000 7f7b377fe700 1 -- 172.21.15.39:0/3341692811 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c00cbd0 con 0x7f7b40103260 2023-07-21T19:03:55.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.151+0000 7f7b377fe700 1 -- 172.21.15.39:0/3341692811 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b3c00ee90 con 0x7f7b40103260 2023-07-21T19:03:55.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.151+0000 7f7b377fe700 1 -- 172.21.15.39:0/3341692811 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c005590 con 0x7f7b40103260 2023-07-21T19:03:55.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.152+0000 7f7b485d0700 1 -- 172.21.15.39:0/3341692811 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b40197eb0 con 0x7f7b40103260 2023-07-21T19:03:55.166 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.152+0000 7f7b485d0700 1 -- 172.21.15.39:0/3341692811 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b40198370 con 0x7f7b40103260 2023-07-21T19:03:55.166 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.155+0000 7f7b377fe700 1 -- 172.21.15.39:0/3341692811 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f7b3c004750 con 0x7f7b40103260 2023-07-21T19:03:55.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.156+0000 7f7b377fe700 1 --2- 172.21.15.39:0/3341692811 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f7b2c0a12a0 0x7f7b2c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:55.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.157+0000 7f7b377fe700 1 -- 172.21.15.39:0/3341692811 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f7b3c014070 con 0x7f7b40103260 2023-07-21T19:03:55.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.157+0000 7f7b4636c700 1 --2- 172.21.15.39:0/3341692811 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f7b2c0a12a0 0x7f7b2c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:55.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.157+0000 7f7b4636c700 1 --2- 172.21.15.39:0/3341692811 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f7b2c0a12a0 0x7f7b2c0a3750 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7b38009550 tx=0x7f7b380091b0 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:03:55.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.158+0000 7f7b54756b80 1 Processor -- start 2023-07-21T19:03:55.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.158+0000 7f7b54756b80 1 -- start start 2023-07-21T19:03:55.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.158+0000 7f7b54756b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561123bc96b0 0x561123d72d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:55.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.158+0000 7f7b54756b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561123d732d0 0x561123d6d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:55.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.158+0000 7f7b54756b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561123d6d580 0x561123d6d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:55.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.158+0000 7f7b54756b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x561123c72c50 con 0x561123bc96b0 2023-07-21T19:03:55.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.158+0000 7f7b54756b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561123cc59e0 con 0x561123d732d0 2023-07-21T19:03:55.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.158+0000 7f7b54756b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x561123cbee70 con 0x561123d6d580 2023-07-21T19:03:55.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.159+0000 7f7b4636c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561123bc96b0 0x561123d72d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:55.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.159+0000 7f7b4636c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561123bc96b0 0x561123d72d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34618/0 (socket says 172.21.15.39:34618) 2023-07-21T19:03:55.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.159+0000 7f7b4636c700 1 -- 172.21.15.39:0/933957115 learned_addr learned my addr 172.21.15.39:0/933957115 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:03:55.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.159+0000 7f7b46b6d700 1 --2- 172.21.15.39:0/933957115 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561123d6d580 0x561123d6d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:55.172 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.159+0000 7f7b45b6b700 1 --2- 172.21.15.39:0/933957115 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561123d732d0 0x561123d6d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:55.172 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.159+0000 7f7b46b6d700 1 -- 172.21.15.39:0/933957115 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561123d732d0 msgr2=0x561123d6d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:03:55.172 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.159+0000 7f7b46b6d700 1 --2- 172.21.15.39:0/933957115 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561123d732d0 0x561123d6d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:55.172 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.159+0000 7f7b46b6d700 1 -- 172.21.15.39:0/933957115 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561123bc96b0 msgr2=0x561123d72d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:03:55.172 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.159+0000 7f7b46b6d700 1 --2- 172.21.15.39:0/933957115 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561123bc96b0 0x561123d72d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:55.172 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.159+0000 7f7b46b6d700 1 -- 172.21.15.39:0/933957115 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561123aecd80 con 0x561123d6d580 2023-07-21T19:03:55.172 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.159+0000 7f7b4636c700 1 --2- 172.21.15.39:0/933957115 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561123bc96b0 0x561123d72d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:03:55.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.159+0000 7f7b46b6d700 1 --2- 172.21.15.39:0/933957115 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561123d6d580 0x561123d6d9d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c0ca630 tx=0x7f7b3c00cd30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22203f145d9a1e15 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:03:55.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.159+0000 7f7b34ff9700 1 -- 172.21.15.39:0/933957115 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c09dfe0 con 0x561123d6d580 2023-07-21T19:03:55.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.159+0000 7f7b34ff9700 1 -- 172.21.15.39:0/933957115 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b3c0a2400 con 0x561123d6d580 2023-07-21T19:03:55.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.159+0000 7f7b34ff9700 1 -- 172.21.15.39:0/933957115 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c0994a0 con 0x561123d6d580 2023-07-21T19:03:55.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.160+0000 7f7b54756b80 1 -- 172.21.15.39:0/933957115 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561123d6d580 msgr2=0x561123d6d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:03:55.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.160+0000 7f7b54756b80 1 --2- 172.21.15.39:0/933957115 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561123d6d580 0x561123d6d9d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c0ca630 tx=0x7f7b3c00cd30 comp rx=0 tx=0).stop 2023-07-21T19:03:55.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.160+0000 7f7b54756b80 1 -- 172.21.15.39:0/933957115 shutdown_connections 2023-07-21T19:03:55.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.160+0000 7f7b54756b80 1 --2- 172.21.15.39:0/933957115 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561123d6d580 0x561123d6d9d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:55.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.160+0000 7f7b54756b80 1 --2- 172.21.15.39:0/933957115 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561123bc96b0 0x561123d72d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:55.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.160+0000 7f7b54756b80 1 --2- 172.21.15.39:0/933957115 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561123d732d0 0x561123d6d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:55.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.160+0000 7f7b54756b80 1 -- 172.21.15.39:0/933957115 >> 172.21.15.39:0/933957115 conn(0x561123d62be0 msgr2=0x561123d67f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:03:55.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.160+0000 7f7b54756b80 1 -- 172.21.15.39:0/933957115 shutdown_connections 2023-07-21T19:03:55.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.160+0000 7f7b54756b80 1 -- 172.21.15.39:0/933957115 wait complete. 2023-07-21T19:03:55.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.161+0000 7f7b54756b80 1 -- start start 2023-07-21T19:03:55.177 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.161+0000 7f7b54756b80 1 Processor -- start 2023-07-21T19:03:55.177 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.161+0000 7f7b54756b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561123bc96b0 0x561123d768e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:55.177 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.161+0000 7f7b54756b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561123d6d580 0x561123d76e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:55.177 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.161+0000 7f7b54756b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561123d732d0 0x561123d77360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:55.177 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.161+0000 7f7b54756b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x561123c5ea60 con 0x561123d6d580 2023-07-21T19:03:55.177 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.161+0000 7f7b54756b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561123c5e760 con 0x561123bc96b0 2023-07-21T19:03:55.177 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.161+0000 7f7b54756b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x561123b428c0 con 0x561123d732d0 2023-07-21T19:03:55.177 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.161+0000 7f7b54756b80 10 client.0 _ll_register_callbacks cb 0x561123c5c700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:03:55.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.162+0000 7f7b4636c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561123bc96b0 0x561123d768e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:55.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.162+0000 7f7b4636c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561123bc96b0 0x561123d768e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:47220/0 (socket says 172.21.15.39:47220) 2023-07-21T19:03:55.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.162+0000 7f7b4636c700 1 -- 172.21.15.39:0/563256557 learned_addr learned my addr 172.21.15.39:0/563256557 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:03:55.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.162+0000 7f7b46b6d700 1 --2- 172.21.15.39:0/563256557 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561123d732d0 0x561123d77360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:55.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.162+0000 7f7b45b6b700 1 --2- 172.21.15.39:0/563256557 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561123d6d580 0x561123d76e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:55.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.162+0000 7f7b4636c700 1 -- 172.21.15.39:0/563256557 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561123d732d0 msgr2=0x561123d77360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:03:55.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.162+0000 7f7b4636c700 1 --2- 172.21.15.39:0/563256557 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561123d732d0 0x561123d77360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:55.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.162+0000 7f7b4636c700 1 -- 172.21.15.39:0/563256557 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561123d6d580 msgr2=0x561123d76e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:03:55.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.162+0000 7f7b4636c700 1 --2- 172.21.15.39:0/563256557 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561123d6d580 0x561123d76e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:55.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.162+0000 7f7b4636c700 1 -- 172.21.15.39:0/563256557 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b380043c0 con 0x561123bc96b0 2023-07-21T19:03:55.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.162+0000 7f7b46b6d700 1 --2- 172.21.15.39:0/563256557 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561123d732d0 0x561123d77360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:03:55.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.163+0000 7f7b4636c700 1 --2- 172.21.15.39:0/563256557 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561123bc96b0 0x561123d768e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7b38007a80 tx=0x7f7b38004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5319aed7a9e62ca1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:03:55.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.163+0000 7f7b29ffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-21T19:03:55.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.163+0000 7f7b29ffb700 1 -- 172.21.15.39:0/563256557 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b38015ce0 con 0x561123bc96b0 2023-07-21T19:03:55.180 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.163+0000 7f7b29ffb700 1 -- 172.21.15.39:0/563256557 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f7b38015e40 con 0x561123bc96b0 2023-07-21T19:03:55.180 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.163+0000 7f7b54756b80 1 -- 172.21.15.39:0/563256557 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561123c18490 con 0x561123bc96b0 2023-07-21T19:03:55.180 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.164+0000 7f7b29ffb700 1 -- 172.21.15.39:0/563256557 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b3802b440 con 0x561123bc96b0 2023-07-21T19:03:55.180 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.164+0000 7f7b54756b80 10 client.27176 fetch_fsmap learned FSMap version 103 2023-07-21T19:03:55.180 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.164+0000 7f7b54756b80 1 -- 172.21.15.39:0/563256557 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=4+}) v3 -- 0x7f7b40198370 con 0x561123bc96b0 2023-07-21T19:03:55.180 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.164+0000 7f7b29ffb700 1 -- 172.21.15.39:0/563256557 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561123c18490 con 0x561123bc96b0 2023-07-21T19:03:55.180 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.164+0000 7f7b29ffb700 1 -- 172.21.15.39:0/563256557 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 103) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f7b38022630 con 0x561123bc96b0 2023-07-21T19:03:55.180 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.164+0000 7f7b54756b80 10 client.27176 fetch_fsmap finished waiting for FSMap version 103 2023-07-21T19:03:55.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.164+0000 7f7b54756b80 10 client.27176 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:03:55.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.164+0000 7f7b54756b80 20 client.27176 populate_metadata read hostname 'smithi039' 2023-07-21T19:03:55.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.164+0000 7f7b54756b80 1 --2- 172.21.15.39:0/563256557 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x561123d86c90 0x561123d890a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:55.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.164+0000 7f7b54756b80 4 client.27176 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:03:55.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.164+0000 7f7b54756b80 1 -- 172.21.15.39:0/563256557 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561123cb7e50 con 0x561123d86c90 2023-07-21T19:03:55.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.166+0000 7f7b45b6b700 1 --2- 172.21.15.39:0/563256557 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x561123d86c90 0x561123d890a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:55.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.166+0000 7f7b29ffb700 10 client.27176 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T19:03:55.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.166+0000 7f7b45b6b700 1 --2- 172.21.15.39:0/563256557 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x561123d86c90 0x561123d890a0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc3d2626d4365e43 server_cookie=47484963987470fa in_seq=0 out_seq=0 2023-07-21T19:03:55.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.169+0000 7f7b29ffb700 1 -- 172.21.15.39:0/563256557 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561123cb7e50 con 0x561123d86c90 2023-07-21T19:03:55.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.169+0000 7f7b29ffb700 10 client.27176 handle_command_reply: tid=0 2023-07-21T19:03:55.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.169+0000 7f7b54756b80 1 client.27176 shutdown 2023-07-21T19:03:55.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.170+0000 7f7b54756b80 1 -- 172.21.15.39:0/563256557 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561123bc96b0 msgr2=0x561123d768e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:03:55.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.170+0000 7f7b54756b80 1 --2- 172.21.15.39:0/563256557 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561123bc96b0 0x561123d768e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7b38007a80 tx=0x7f7b38004b40 comp rx=0 tx=0).stop 2023-07-21T19:03:55.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.170+0000 7f7b54756b80 1 -- 172.21.15.39:0/563256557 shutdown_connections 2023-07-21T19:03:55.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.170+0000 7f7b54756b80 1 --2- 172.21.15.39:0/563256557 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x561123d86c90 0x561123d890a0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:55.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.170+0000 7f7b54756b80 1 --2- 172.21.15.39:0/563256557 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561123bc96b0 0x561123d768e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:55.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.170+0000 7f7b54756b80 1 --2- 172.21.15.39:0/563256557 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561123d732d0 0x561123d77360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:55.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.170+0000 7f7b29ffb700 0 client.27176 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T19:03:55.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.170+0000 7f7b54756b80 1 --2- 172.21.15.39:0/563256557 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561123d6d580 0x561123d76e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:55.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.170+0000 7f7b54756b80 1 -- 172.21.15.39:0/563256557 >> 172.21.15.39:0/563256557 conn(0x561123d62be0 msgr2=0x561123d674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:03:55.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.171+0000 7f7b54756b80 1 -- 172.21.15.39:0/563256557 shutdown_connections 2023-07-21T19:03:55.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.171+0000 7f7b54756b80 1 -- 172.21.15.39:0/563256557 wait complete. 2023-07-21T19:03:55.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.171+0000 7f7b54756b80 20 client.27176 trim_cache size 0 max 16384 2023-07-21T19:03:55.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.194+0000 7f7b54756b80 1 Processor -- start 2023-07-21T19:03:55.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.194+0000 7f7b54756b80 1 -- start start 2023-07-21T19:03:55.206 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.194+0000 7f7b54756b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561123bc96b0 0x561123d6d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:55.206 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.194+0000 7f7b54756b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561123d6d580 0x561123d722f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:55.206 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.194+0000 7f7b54756b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561123d732d0 0x561123d7dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:55.206 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.194+0000 7f7b54756b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x561123d697d0 con 0x561123d732d0 2023-07-21T19:03:55.206 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.194+0000 7f7b54756b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561123d64620 con 0x561123d6d580 2023-07-21T19:03:55.206 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.194+0000 7f7b54756b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x561123d74e90 con 0x561123bc96b0 2023-07-21T19:03:55.207 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.195+0000 7f7b4636c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561123bc96b0 0x561123d6d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:55.207 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.195+0000 7f7b46b6d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561123d732d0 0x561123d7dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:55.207 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.195+0000 7f7b4636c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561123bc96b0 0x561123d6d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:53184/0 (socket says 172.21.15.39:53184) 2023-07-21T19:03:55.207 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.195+0000 7f7b4636c700 1 -- 172.21.15.39:0/3547590573 learned_addr learned my addr 172.21.15.39:0/3547590573 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:03:55.207 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.195+0000 7f7b45b6b700 1 --2- 172.21.15.39:0/3547590573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561123d6d580 0x561123d722f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:55.207 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.195+0000 7f7b46b6d700 1 -- 172.21.15.39:0/3547590573 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561123bc96b0 msgr2=0x561123d6d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:03:55.208 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.195+0000 7f7b46b6d700 1 --2- 172.21.15.39:0/3547590573 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561123bc96b0 0x561123d6d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:55.208 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.195+0000 7f7b46b6d700 1 -- 172.21.15.39:0/3547590573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561123d6d580 msgr2=0x561123d722f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:03:55.208 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.195+0000 7f7b46b6d700 1 --2- 172.21.15.39:0/3547590573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561123d6d580 0x561123d722f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:55.208 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.195+0000 7f7b46b6d700 1 -- 172.21.15.39:0/3547590573 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561123ad5c00 con 0x561123d732d0 2023-07-21T19:03:55.209 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.195+0000 7f7b46b6d700 1 --2- 172.21.15.39:0/3547590573 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561123d732d0 0x561123d7dc60 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x561123d73810 tx=0x7f7b3c009870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d12a0e8c7b9d8127 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:03:55.209 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.196+0000 7f7b357fa700 1 -- 172.21.15.39:0/3547590573 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c0b8070 con 0x561123d732d0 2023-07-21T19:03:55.209 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.196+0000 7f7b357fa700 1 -- 172.21.15.39:0/3547590573 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b3c09dfe0 con 0x561123d732d0 2023-07-21T19:03:55.209 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.196+0000 7f7b357fa700 1 -- 172.21.15.39:0/3547590573 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c0996e0 con 0x561123d732d0 2023-07-21T19:03:55.209 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.197+0000 7f7b54756b80 1 -- 172.21.15.39:0/3547590573 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561123d732d0 msgr2=0x561123d7dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:03:55.210 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.197+0000 7f7b54756b80 1 --2- 172.21.15.39:0/3547590573 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561123d732d0 0x561123d7dc60 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x561123d73810 tx=0x7f7b3c009870 comp rx=0 tx=0).stop 2023-07-21T19:03:55.210 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.199+0000 7f7b54756b80 1 -- 172.21.15.39:0/3547590573 shutdown_connections 2023-07-21T19:03:55.210 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.199+0000 7f7b54756b80 1 --2- 172.21.15.39:0/3547590573 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561123d732d0 0x561123d7dc60 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:55.210 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.199+0000 7f7b54756b80 1 --2- 172.21.15.39:0/3547590573 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561123bc96b0 0x561123d6d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:55.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.199+0000 7f7b54756b80 1 --2- 172.21.15.39:0/3547590573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561123d6d580 0x561123d722f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:55.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.199+0000 7f7b54756b80 1 -- 172.21.15.39:0/3547590573 >> 172.21.15.39:0/3547590573 conn(0x561123d62be0 msgr2=0x561123d750b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:03:55.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.200+0000 7f7b54756b80 1 -- 172.21.15.39:0/3547590573 shutdown_connections 2023-07-21T19:03:55.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.200+0000 7f7b54756b80 1 -- 172.21.15.39:0/3547590573 wait complete. 2023-07-21T19:03:55.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.200+0000 7f7b54756b80 1 -- start start 2023-07-21T19:03:55.214 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.201+0000 7f7b54756b80 1 Processor -- start 2023-07-21T19:03:55.214 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.201+0000 7f7b54756b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561123bc96b0 0x561123d79c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:55.214 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.201+0000 7f7b54756b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561123d6d580 0x561123d7a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:55.214 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.202+0000 7f7b54756b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561123d732d0 0x561123dd9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:55.214 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.202+0000 7f7b54756b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x561123d7aa90 con 0x561123d732d0 2023-07-21T19:03:55.214 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.202+0000 7f7b54756b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561123dda080 con 0x561123bc96b0 2023-07-21T19:03:55.214 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.202+0000 7f7b54756b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x561123dda1c0 con 0x561123d6d580 2023-07-21T19:03:55.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.202+0000 7f7b54756b80 10 client.0 _ll_register_callbacks cb 0x7f7b38004950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:03:55.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.202+0000 7f7b46b6d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561123d732d0 0x561123dd9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:55.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.202+0000 7f7b45b6b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561123d6d580 0x561123d7a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:55.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.202+0000 7f7b4636c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561123bc96b0 0x561123d79c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:55.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.202+0000 7f7b46b6d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561123d732d0 0x561123dd9b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34642/0 (socket says 172.21.15.39:34642) 2023-07-21T19:03:55.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.202+0000 7f7b46b6d700 1 -- 172.21.15.39:0/2897498836 learned_addr learned my addr 172.21.15.39:0/2897498836 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:03:55.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.202+0000 7f7b46b6d700 1 -- 172.21.15.39:0/2897498836 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561123d6d580 msgr2=0x561123d7a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:03:55.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.202+0000 7f7b46b6d700 1 --2- 172.21.15.39:0/2897498836 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561123d6d580 0x561123d7a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:55.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.202+0000 7f7b46b6d700 1 -- 172.21.15.39:0/2897498836 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561123bc96b0 msgr2=0x561123d79c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:03:55.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.202+0000 7f7b46b6d700 1 --2- 172.21.15.39:0/2897498836 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561123bc96b0 0x561123d79c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:55.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.202+0000 7f7b46b6d700 1 -- 172.21.15.39:0/2897498836 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561123aecd80 con 0x561123d732d0 2023-07-21T19:03:55.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.202+0000 7f7b46b6d700 1 --2- 172.21.15.39:0/2897498836 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561123d732d0 0x561123dd9b40 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c0a1ed0 tx=0x7f7b3c0056f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e878af4ff49c5db8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:03:55.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.202+0000 7f7b29ffb700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T19:03:55.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.202+0000 7f7b29ffb700 1 -- 172.21.15.39:0/2897498836 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c0b8040 con 0x561123d732d0 2023-07-21T19:03:55.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.202+0000 7f7b29ffb700 1 -- 172.21.15.39:0/2897498836 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f7b3c09a410 con 0x561123d732d0 2023-07-21T19:03:55.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.202+0000 7f7b54756b80 1 -- 172.21.15.39:0/2897498836 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7b3c0a2400 con 0x561123d732d0 2023-07-21T19:03:55.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.203+0000 7f7b29ffb700 1 -- 172.21.15.39:0/2897498836 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b3c0994a0 con 0x561123d732d0 2023-07-21T19:03:55.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.203+0000 7f7b54756b80 10 client.17754 fetch_fsmap learned FSMap version 103 2023-07-21T19:03:55.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.203+0000 7f7b54756b80 1 -- 172.21.15.39:0/2897498836 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=4+}) v3 -- 0x561123b131f0 con 0x561123d732d0 2023-07-21T19:03:55.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.203+0000 7f7b29ffb700 1 -- 172.21.15.39:0/2897498836 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b3c0a2400 con 0x561123d732d0 2023-07-21T19:03:55.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.203+0000 7f7b29ffb700 1 -- 172.21.15.39:0/2897498836 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 103) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f7b3c0ca260 con 0x561123d732d0 2023-07-21T19:03:55.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.203+0000 7f7b54756b80 10 client.17754 fetch_fsmap finished waiting for FSMap version 103 2023-07-21T19:03:55.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.203+0000 7f7b54756b80 10 client.17754 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:03:55.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.203+0000 7f7b54756b80 20 client.17754 populate_metadata read hostname 'smithi039' 2023-07-21T19:03:55.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.203+0000 7f7b54756b80 1 --2- 172.21.15.39:0/2897498836 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x561123ddb320 0x561123ddd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:55.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.203+0000 7f7b54756b80 4 client.17754 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:03:55.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.203+0000 7f7b54756b80 1 -- 172.21.15.39:0/2897498836 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561123c79ff0 con 0x561123ddb320 2023-07-21T19:03:55.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.204+0000 7f7b4636c700 1 --2- 172.21.15.39:0/2897498836 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x561123ddb320 0x561123ddd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:55.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.204+0000 7f7b29ffb700 10 client.17754 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T19:03:55.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.204+0000 7f7b4636c700 1 --2- 172.21.15.39:0/2897498836 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x561123ddb320 0x561123ddd700 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f20758b9b8a06a5 server_cookie=87946ae2a39b2396 in_seq=0 out_seq=0 2023-07-21T19:03:55.449 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.438+0000 7f7b29ffb700 1 -- 172.21.15.39:0/2897498836 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561123c79ff0 con 0x561123ddb320 2023-07-21T19:03:55.449 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.438+0000 7f7b29ffb700 10 client.17754 handle_command_reply: tid=0 2023-07-21T19:03:55.450 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.438+0000 7f7b54756b80 1 client.17754 shutdown 2023-07-21T19:03:55.450 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.439+0000 7f7b54756b80 1 -- 172.21.15.39:0/2897498836 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561123d732d0 msgr2=0x561123dd9b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:03:55.450 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.439+0000 7f7b54756b80 1 --2- 172.21.15.39:0/2897498836 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561123d732d0 0x561123dd9b40 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c0a1ed0 tx=0x7f7b3c0056f0 comp rx=0 tx=0).stop 2023-07-21T19:03:55.454 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.442+0000 7f7b54756b80 1 -- 172.21.15.39:0/2897498836 shutdown_connections 2023-07-21T19:03:55.455 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.442+0000 7f7b54756b80 1 --2- 172.21.15.39:0/2897498836 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x561123ddb320 0x561123ddd700 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:55.455 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.442+0000 7f7b54756b80 1 --2- 172.21.15.39:0/2897498836 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561123bc96b0 0x561123d79c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:55.455 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.442+0000 7f7b54756b80 1 --2- 172.21.15.39:0/2897498836 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561123d732d0 0x561123dd9b40 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:55.455 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.442+0000 7f7b54756b80 1 --2- 172.21.15.39:0/2897498836 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561123d6d580 0x561123d7a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:55.455 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.442+0000 7f7b54756b80 1 -- 172.21.15.39:0/2897498836 >> 172.21.15.39:0/2897498836 conn(0x561123d62be0 msgr2=0x561123d74190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:03:55.455 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.444+0000 7f7b29ffb700 0 client.17754 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T19:03:55.457 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.447+0000 7f7b54756b80 1 -- 172.21.15.39:0/2897498836 shutdown_connections 2023-07-21T19:03:55.458 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.447+0000 7f7b54756b80 1 -- 172.21.15.39:0/2897498836 wait complete. 2023-07-21T19:03:55.460 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.450+0000 7f7b54756b80 20 client.17754 trim_cache size 0 max 16384 2023-07-21T19:03:55.461 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T19:03:55.461 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T19:03:55.461 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T19:03:55.461 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T19:03:55.462 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.451+0000 7f7b485d0700 1 -- 172.21.15.39:0/3341692811 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f7b2c0a12a0 msgr2=0x7f7b2c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:03:55.462 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.451+0000 7f7b485d0700 1 --2- 172.21.15.39:0/3341692811 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f7b2c0a12a0 0x7f7b2c0a3750 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7b38009550 tx=0x7f7b380091b0 comp rx=0 tx=0).stop 2023-07-21T19:03:55.463 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.451+0000 7f7b485d0700 1 -- 172.21.15.39:0/3341692811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b40103260 msgr2=0x7f7b40106a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:03:55.463 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.451+0000 7f7b485d0700 1 --2- 172.21.15.39:0/3341692811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b40103260 0x7f7b40106a80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c00e940 tx=0x7f7b3c00ec50 comp rx=0 tx=0).stop 2023-07-21T19:03:55.463 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.451+0000 7f7b485d0700 1 -- 172.21.15.39:0/3341692811 shutdown_connections 2023-07-21T19:03:55.463 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.451+0000 7f7b485d0700 1 --2- 172.21.15.39:0/3341692811 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f7b2c0a12a0 0x7f7b2c0a3750 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:55.463 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.451+0000 7f7b485d0700 1 --2- 172.21.15.39:0/3341692811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b400fe220 0x7f7b40105fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:55.463 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.451+0000 7f7b485d0700 1 --2- 172.21.15.39:0/3341692811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b40103260 0x7f7b40106a80 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:55.463 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.451+0000 7f7b485d0700 1 --2- 172.21.15.39:0/3341692811 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b400febc0 0x7f7b40106500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:55.464 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.451+0000 7f7b485d0700 1 -- 172.21.15.39:0/3341692811 >> 172.21.15.39:0/3341692811 conn(0x7f7b40073d50 msgr2=0x7f7b40101bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:03:55.464 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.451+0000 7f7b485d0700 1 -- 172.21.15.39:0/3341692811 shutdown_connections 2023-07-21T19:03:55.464 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:03:55.451+0000 7f7b485d0700 1 -- 172.21.15.39:0/3341692811 wait complete. 2023-07-21T19:03:55.715 INFO:tasks.workunit.client.0.smithi039.stdout: 10 297001 28.93 MB/sec execute 428 sec latency 176.531 ms 2023-07-21T19:03:56.715 INFO:tasks.workunit.client.0.smithi039.stdout: 10 297342 28.92 MB/sec execute 429 sec latency 405.776 ms 2023-07-21T19:03:56.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:03:56 smithi193 ceph-mon[98204]: pgmap v704: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 7.1 MiB/s wr, 397 op/s 2023-07-21T19:03:56.880 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:03:56 smithi155 ceph-mon[97934]: pgmap v704: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 7.1 MiB/s wr, 397 op/s 2023-07-21T19:03:56.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:03:56 smithi039 ceph-mon[96489]: pgmap v704: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 7.1 MiB/s wr, 397 op/s 2023-07-21T19:03:57.715 INFO:tasks.workunit.client.0.smithi039.stdout: 10 297455 28.87 MB/sec execute 430 sec latency 1060.272 ms 2023-07-21T19:03:58.715 INFO:tasks.workunit.client.0.smithi039.stdout: 10 297730 28.83 MB/sec execute 431 sec latency 312.185 ms 2023-07-21T19:03:58.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:03:58 smithi193 ceph-mon[98204]: pgmap v705: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 10 MiB/s wr, 617 op/s 2023-07-21T19:03:58.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:03:58 smithi155 ceph-mon[97934]: pgmap v705: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 10 MiB/s wr, 617 op/s 2023-07-21T19:03:58.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:03:58 smithi039 ceph-mon[96489]: pgmap v705: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 10 MiB/s wr, 617 op/s 2023-07-21T19:03:59.715 INFO:tasks.workunit.client.0.smithi039.stdout: 10 298248 28.82 MB/sec execute 432 sec latency 88.342 ms 2023-07-21T19:04:00.715 INFO:tasks.workunit.client.0.smithi039.stdout: 10 298647 28.80 MB/sec execute 433 sec latency 274.953 ms 2023-07-21T19:04:00.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:04:00 smithi193 ceph-mon[98204]: pgmap v706: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 6.6 MiB/s wr, 404 op/s 2023-07-21T19:04:00.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:04:00 smithi155 ceph-mon[97934]: pgmap v706: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 6.6 MiB/s wr, 404 op/s 2023-07-21T19:04:00.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:04:00 smithi039 ceph-mon[96489]: pgmap v706: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 6.6 MiB/s wr, 404 op/s 2023-07-21T19:04:01.715 INFO:tasks.workunit.client.0.smithi039.stdout: 10 298647 28.74 MB/sec execute 434 sec latency 1275.050 ms 2023-07-21T19:04:02.715 INFO:tasks.workunit.client.0.smithi039.stdout: 10 298647 28.67 MB/sec execute 435 sec latency 2275.178 ms 2023-07-21T19:04:02.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:04:02 smithi193 ceph-mon[98204]: pgmap v707: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.8 MiB/s wr, 462 op/s 2023-07-21T19:04:02.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:04:02 smithi155 ceph-mon[97934]: pgmap v707: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.8 MiB/s wr, 462 op/s 2023-07-21T19:04:02.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:04:02 smithi039 ceph-mon[96489]: pgmap v707: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.8 MiB/s wr, 462 op/s 2023-07-21T19:04:03.715 INFO:tasks.workunit.client.0.smithi039.stdout: 10 298841 28.63 MB/sec execute 436 sec latency 2564.838 ms 2023-07-21T19:04:03.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:04:03 smithi193 ceph-mon[98204]: pgmap v708: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 16 MiB/s wr, 737 op/s 2023-07-21T19:04:03.878 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:04:03 smithi155 ceph-mon[97934]: pgmap v708: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 16 MiB/s wr, 737 op/s 2023-07-21T19:04:03.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:04:03 smithi039 ceph-mon[96489]: pgmap v708: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 16 MiB/s wr, 737 op/s 2023-07-21T19:04:04.715 INFO:tasks.workunit.client.0.smithi039.stdout: 10 299182 28.60 MB/sec execute 437 sec latency 154.943 ms 2023-07-21T19:04:05.716 INFO:tasks.workunit.client.0.smithi039.stdout: 10 299448 28.57 MB/sec execute 438 sec latency 158.180 ms 2023-07-21T19:04:06.107 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:04:05 smithi193 ceph-mon[98204]: pgmap v709: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 13 MiB/s wr, 582 op/s 2023-07-21T19:04:06.126 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:04:05 smithi155 ceph-mon[97934]: pgmap v709: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 13 MiB/s wr, 582 op/s 2023-07-21T19:04:06.175 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:04:05 smithi039 ceph-mon[96489]: pgmap v709: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 13 MiB/s wr, 582 op/s 2023-07-21T19:04:06.716 INFO:tasks.workunit.client.0.smithi039.stdout: 10 299711 28.55 MB/sec execute 439 sec latency 171.319 ms 2023-07-21T19:04:07.715 INFO:tasks.workunit.client.0.smithi039.stdout: 10 300149 28.54 MB/sec execute 440 sec latency 157.366 ms 2023-07-21T19:04:08.616 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.606+0000 7f0807603700 1 -- 172.21.15.39:0/654381772 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08000fea10 msgr2=0x7f08000fee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:08.616 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.606+0000 7f0807603700 1 --2- 172.21.15.39:0/654381772 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08000fea10 0x7f08000fee80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f07f00098f0 tx=0x7f07f0009c00 comp rx=0 tx=0).stop 2023-07-21T19:04:08.616 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.606+0000 7f0807603700 1 -- 172.21.15.39:0/654381772 shutdown_connections 2023-07-21T19:04:08.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.606+0000 7f0807603700 1 --2- 172.21.15.39:0/654381772 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0800104a20 0x7f0800104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:08.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.606+0000 7f0807603700 1 --2- 172.21.15.39:0/654381772 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08000ff3c0 0x7f080010b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:08.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.606+0000 7f0807603700 1 --2- 172.21.15.39:0/654381772 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08000fea10 0x7f08000fee80 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:08.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.606+0000 7f0807603700 1 -- 172.21.15.39:0/654381772 >> 172.21.15.39:0/654381772 conn(0x7f08000fa4f0 msgr2=0x7f08000fc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:04:08.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.606+0000 7f0807603700 1 -- 172.21.15.39:0/654381772 shutdown_connections 2023-07-21T19:04:08.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.606+0000 7f0807603700 1 -- 172.21.15.39:0/654381772 wait complete. 2023-07-21T19:04:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.607+0000 7f0807603700 1 Processor -- start 2023-07-21T19:04:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.607+0000 7f0807603700 1 -- start start 2023-07-21T19:04:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.607+0000 7f0807603700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08000fea10 0x7f080010c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.607+0000 7f0807603700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08000ff3c0 0x7f080010c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.607+0000 7f0807603700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0800104a20 0x7f08001a3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:08.620 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.607+0000 7f0807603700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08001a4480 con 0x7f0800104a20 2023-07-21T19:04:08.620 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.607+0000 7f0807603700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08001a45c0 con 0x7f08000fea10 2023-07-21T19:04:08.620 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.607+0000 7f0807603700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08001a4730 con 0x7f08000ff3c0 2023-07-21T19:04:08.620 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.607+0000 7f0805ba0700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0800104a20 0x7f08001a3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:08.620 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.607+0000 7f080539f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08000fea10 0x7f080010c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:08.620 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.607+0000 7f0805ba0700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0800104a20 0x7f08001a3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34656/0 (socket says 172.21.15.39:34656) 2023-07-21T19:04:08.620 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.607+0000 7f080539f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08000fea10 0x7f080010c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:47248/0 (socket says 172.21.15.39:47248) 2023-07-21T19:04:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.607+0000 7f0805ba0700 1 -- 172.21.15.39:0/1964972130 learned_addr learned my addr 172.21.15.39:0/1964972130 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:04:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.607+0000 7f0804b9e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08000ff3c0 0x7f080010c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.608+0000 7f0805ba0700 1 -- 172.21.15.39:0/1964972130 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08000ff3c0 msgr2=0x7f080010c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.608+0000 7f0805ba0700 1 --2- 172.21.15.39:0/1964972130 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08000ff3c0 0x7f080010c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.608+0000 7f0805ba0700 1 -- 172.21.15.39:0/1964972130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08000fea10 msgr2=0x7f080010c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.608+0000 7f0805ba0700 1 --2- 172.21.15.39:0/1964972130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08000fea10 0x7f080010c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.608+0000 7f0805ba0700 1 -- 172.21.15.39:0/1964972130 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07f8009d00 con 0x7f0800104a20 2023-07-21T19:04:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.608+0000 7f0804b9e700 1 --2- 172.21.15.39:0/1964972130 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08000ff3c0 0x7f080010c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:04:08.622 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.608+0000 7f080539f700 1 --2- 172.21.15.39:0/1964972130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08000fea10 0x7f080010c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:04:08.622 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.608+0000 7f0805ba0700 1 --2- 172.21.15.39:0/1964972130 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0800104a20 0x7f08001a3f40 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f07f800e3c0 tx=0x7f07f800e6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:04:08.622 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.608+0000 7f07f67fc700 1 -- 172.21.15.39:0/1964972130 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f80090d0 con 0x7f0800104a20 2023-07-21T19:04:08.622 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.608+0000 7f0807603700 1 -- 172.21.15.39:0/1964972130 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07f0009520 con 0x7f0800104a20 2023-07-21T19:04:08.622 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.608+0000 7f07f67fc700 1 -- 172.21.15.39:0/1964972130 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07f800f040 con 0x7f0800104a20 2023-07-21T19:04:08.622 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.608+0000 7f07f67fc700 1 -- 172.21.15.39:0/1964972130 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f8007c20 con 0x7f0800104a20 2023-07-21T19:04:08.622 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.608+0000 7f0807603700 1 -- 172.21.15.39:0/1964972130 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08001a4d60 con 0x7f0800104a20 2023-07-21T19:04:08.624 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.611+0000 7f07f67fc700 1 -- 172.21.15.39:0/1964972130 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f07f8003a00 con 0x7f0800104a20 2023-07-21T19:04:08.624 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.612+0000 7f07f67fc700 1 --2- 172.21.15.39:0/1964972130 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f07ec0a12c0 0x7f07ec0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:08.625 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.612+0000 7f080539f700 1 --2- 172.21.15.39:0/1964972130 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f07ec0a12c0 0x7f07ec0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:08.625 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.612+0000 7f07f67fc700 1 -- 172.21.15.39:0/1964972130 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f07f80ccb10 con 0x7f0800104a20 2023-07-21T19:04:08.625 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.613+0000 7f080539f700 1 --2- 172.21.15.39:0/1964972130 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f07ec0a12c0 0x7f07ec0a3770 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f08000ffb50 tx=0x7f07fc009260 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:04:08.625 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.613+0000 7f0813789b80 1 Processor -- start 2023-07-21T19:04:08.625 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.613+0000 7f0813789b80 1 -- start start 2023-07-21T19:04:08.625 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.614+0000 7f0813789b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aebf1186b0 0x55aebf2c1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:08.625 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.614+0000 7f0813789b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aebf2c22d0 0x55aebf2bc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:08.626 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.614+0000 7f0813789b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aebf2bc580 0x55aebf2bc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:08.626 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.614+0000 7f0813789b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aebf1c1c50 con 0x55aebf2c22d0 2023-07-21T19:04:08.626 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.614+0000 7f0813789b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aebf2149e0 con 0x55aebf1186b0 2023-07-21T19:04:08.626 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.614+0000 7f0813789b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aebf20de70 con 0x55aebf2bc580 2023-07-21T19:04:08.626 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.614+0000 7f0805ba0700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aebf2bc580 0x55aebf2bc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:08.626 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.614+0000 7f080539f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aebf1186b0 0x55aebf2c1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:08.626 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.614+0000 7f080539f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aebf1186b0 0x55aebf2c1d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:47256/0 (socket says 172.21.15.39:47256) 2023-07-21T19:04:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.614+0000 7f080539f700 1 -- 172.21.15.39:0/2212128623 learned_addr learned my addr 172.21.15.39:0/2212128623 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:04:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.614+0000 7f0805ba0700 1 -- 172.21.15.39:0/2212128623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aebf1186b0 msgr2=0x55aebf2c1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:04:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.614+0000 7f0805ba0700 1 --2- 172.21.15.39:0/2212128623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aebf1186b0 0x55aebf2c1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.614+0000 7f0805ba0700 1 -- 172.21.15.39:0/2212128623 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aebf2c22d0 msgr2=0x55aebf2bc040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T19:04:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.614+0000 7f0805ba0700 1 --2- 172.21.15.39:0/2212128623 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aebf2c22d0 0x55aebf2bc040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.614+0000 7f0805ba0700 1 -- 172.21.15.39:0/2212128623 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aebf03bd80 con 0x55aebf2bc580 2023-07-21T19:04:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.615+0000 7f0805ba0700 1 --2- 172.21.15.39:0/2212128623 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aebf2bc580 0x55aebf2bc9d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f07f80ca260 tx=0x7f07f80997b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ee76e4f7c5944e4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:04:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.615+0000 7f07eb7fe700 1 -- 172.21.15.39:0/2212128623 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f8099c70 con 0x55aebf2bc580 2023-07-21T19:04:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.615+0000 7f07eb7fe700 1 -- 172.21.15.39:0/2212128623 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07f809dca0 con 0x55aebf2bc580 2023-07-21T19:04:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.615+0000 7f07eb7fe700 1 -- 172.21.15.39:0/2212128623 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f8096380 con 0x55aebf2bc580 2023-07-21T19:04:08.628 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:04:08 smithi155 ceph-mon[97934]: pgmap v710: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 15 MiB/s wr, 814 op/s 2023-07-21T19:04:08.631 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.616+0000 7f0813789b80 1 -- 172.21.15.39:0/2212128623 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aebf2bc580 msgr2=0x55aebf2bc9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:08.631 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.616+0000 7f0813789b80 1 --2- 172.21.15.39:0/2212128623 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aebf2bc580 0x55aebf2bc9d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f07f80ca260 tx=0x7f07f80997b0 comp rx=0 tx=0).stop 2023-07-21T19:04:08.631 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.616+0000 7f0813789b80 1 -- 172.21.15.39:0/2212128623 shutdown_connections 2023-07-21T19:04:08.632 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.616+0000 7f0813789b80 1 --2- 172.21.15.39:0/2212128623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aebf1186b0 0x55aebf2c1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:08.632 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.616+0000 7f0813789b80 1 --2- 172.21.15.39:0/2212128623 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aebf2bc580 0x55aebf2bc9d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:08.632 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.616+0000 7f0813789b80 1 --2- 172.21.15.39:0/2212128623 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aebf2c22d0 0x55aebf2bc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:08.632 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.616+0000 7f0813789b80 1 -- 172.21.15.39:0/2212128623 >> 172.21.15.39:0/2212128623 conn(0x55aebf2b1be0 msgr2=0x55aebf2b6f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:04:08.632 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.616+0000 7f0813789b80 1 -- 172.21.15.39:0/2212128623 shutdown_connections 2023-07-21T19:04:08.632 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.616+0000 7f0813789b80 1 -- 172.21.15.39:0/2212128623 wait complete. 2023-07-21T19:04:08.632 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.617+0000 7f0813789b80 1 -- start start 2023-07-21T19:04:08.633 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.617+0000 7f0813789b80 1 Processor -- start 2023-07-21T19:04:08.633 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.617+0000 7f0813789b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aebf1186b0 0x55aebf2c85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:08.633 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.617+0000 7f0813789b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aebf2bc580 0x55aebf2c58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:08.633 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.617+0000 7f0813789b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aebf2c22d0 0x55aebf2c5f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:08.633 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.617+0000 7f0813789b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aebf1ada60 con 0x55aebf2bc580 2023-07-21T19:04:08.633 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.617+0000 7f0813789b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aebf1ad760 con 0x55aebf2c22d0 2023-07-21T19:04:08.633 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.617+0000 7f0813789b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aebf0918c0 con 0x55aebf1186b0 2023-07-21T19:04:08.633 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.617+0000 7f0813789b80 10 client.0 _ll_register_callbacks cb 0x55aebf1ab700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:04:08.634 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.617+0000 7f0804b9e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aebf2bc580 0x55aebf2c58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:08.634 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.617+0000 7f080539f700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aebf1186b0 0x55aebf2c85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:08.634 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.617+0000 7f0804b9e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aebf2bc580 0x55aebf2c58b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34666/0 (socket says 172.21.15.39:34666) 2023-07-21T19:04:08.634 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.617+0000 7f0804b9e700 1 -- 172.21.15.39:0/730779417 learned_addr learned my addr 172.21.15.39:0/730779417 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:04:08.634 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.617+0000 7f0805ba0700 1 --2- 172.21.15.39:0/730779417 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aebf2c22d0 0x55aebf2c5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:08.634 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.618+0000 7f080539f700 1 -- 172.21.15.39:0/730779417 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aebf2c22d0 msgr2=0x55aebf2c5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:04:08.634 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.618+0000 7f080539f700 1 --2- 172.21.15.39:0/730779417 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aebf2c22d0 0x55aebf2c5f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:08.635 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.618+0000 7f080539f700 1 -- 172.21.15.39:0/730779417 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aebf2bc580 msgr2=0x55aebf2c58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:04:08.635 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.618+0000 7f080539f700 1 --2- 172.21.15.39:0/730779417 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aebf2bc580 0x55aebf2c58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:08.635 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.618+0000 7f080539f700 1 -- 172.21.15.39:0/730779417 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08001a4d60 con 0x55aebf1186b0 2023-07-21T19:04:08.635 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.618+0000 7f0804b9e700 1 --2- 172.21.15.39:0/730779417 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aebf2bc580 0x55aebf2c58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:04:08.635 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.618+0000 7f080539f700 1 --2- 172.21.15.39:0/730779417 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aebf1186b0 0x55aebf2c85c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f07fc009fd0 tx=0x7f07fc0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2488cb108faf869 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:04:08.635 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.618+0000 7f07e8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2023-07-21T19:04:08.635 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.618+0000 7f07e8ff9700 1 -- 172.21.15.39:0/730779417 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07fc01fca0 con 0x55aebf1186b0 2023-07-21T19:04:08.635 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.618+0000 7f07e8ff9700 1 -- 172.21.15.39:0/730779417 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f07fc01fe00 con 0x55aebf1186b0 2023-07-21T19:04:08.636 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.618+0000 7f0813789b80 1 -- 172.21.15.39:0/730779417 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aebf167490 con 0x55aebf1186b0 2023-07-21T19:04:08.636 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.618+0000 7f07e8ff9700 1 -- 172.21.15.39:0/730779417 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07fc00ee90 con 0x55aebf1186b0 2023-07-21T19:04:08.636 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.618+0000 7f0813789b80 10 client.27271 fetch_fsmap learned FSMap version 103 2023-07-21T19:04:08.636 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.618+0000 7f0813789b80 1 -- 172.21.15.39:0/730779417 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=4+}) v3 -- 0x55aebf024c00 con 0x55aebf1186b0 2023-07-21T19:04:08.636 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.619+0000 7f07e8ff9700 1 -- 172.21.15.39:0/730779417 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07fc034080 con 0x55aebf1186b0 2023-07-21T19:04:08.636 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.619+0000 7f07e8ff9700 1 -- 172.21.15.39:0/730779417 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 103) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f07fc02e030 con 0x55aebf1186b0 2023-07-21T19:04:08.636 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.619+0000 7f0813789b80 10 client.27271 fetch_fsmap finished waiting for FSMap version 103 2023-07-21T19:04:08.637 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.619+0000 7f0813789b80 10 client.27271 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:04:08.637 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.619+0000 7f0813789b80 20 client.27271 populate_metadata read hostname 'smithi039' 2023-07-21T19:04:08.637 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.619+0000 7f0813789b80 1 --2- 172.21.15.39:0/730779417 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55aebf2e6c40 0x55aebf2e9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:08.637 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.619+0000 7f0813789b80 4 client.27271 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:04:08.637 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.619+0000 7f0813789b80 1 -- 172.21.15.39:0/730779417 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aebf206e50 con 0x55aebf2e6c40 2023-07-21T19:04:08.637 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.619+0000 7f0804b9e700 1 --2- 172.21.15.39:0/730779417 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55aebf2e6c40 0x55aebf2e9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:08.637 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.620+0000 7f07e8ff9700 10 client.27271 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T19:04:08.638 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.620+0000 7f0804b9e700 1 --2- 172.21.15.39:0/730779417 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55aebf2e6c40 0x55aebf2e9050 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a71ad22707668e7 server_cookie=757d0265ee5ea43e in_seq=0 out_seq=0 2023-07-21T19:04:08.638 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.621+0000 7f07e8ff9700 1 -- 172.21.15.39:0/730779417 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55aebf206e50 con 0x55aebf2e6c40 2023-07-21T19:04:08.638 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.621+0000 7f07e8ff9700 10 client.27271 handle_command_reply: tid=0 2023-07-21T19:04:08.638 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.621+0000 7f0813789b80 1 client.27271 shutdown 2023-07-21T19:04:08.638 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.621+0000 7f0813789b80 1 -- 172.21.15.39:0/730779417 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aebf1186b0 msgr2=0x55aebf2c85c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:08.638 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.621+0000 7f0813789b80 1 --2- 172.21.15.39:0/730779417 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aebf1186b0 0x55aebf2c85c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f07fc009fd0 tx=0x7f07fc0047c0 comp rx=0 tx=0).stop 2023-07-21T19:04:08.638 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.621+0000 7f0813789b80 1 -- 172.21.15.39:0/730779417 shutdown_connections 2023-07-21T19:04:08.638 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.621+0000 7f0813789b80 1 --2- 172.21.15.39:0/730779417 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55aebf2e6c40 0x55aebf2e9050 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:08.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.621+0000 7f0813789b80 1 --2- 172.21.15.39:0/730779417 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aebf2c22d0 0x55aebf2c5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:08.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.621+0000 7f0813789b80 1 --2- 172.21.15.39:0/730779417 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aebf1186b0 0x55aebf2c85c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:08.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.621+0000 7f0813789b80 1 --2- 172.21.15.39:0/730779417 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aebf2bc580 0x55aebf2c58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:08.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.621+0000 7f0813789b80 1 -- 172.21.15.39:0/730779417 >> 172.21.15.39:0/730779417 conn(0x55aebf2b1be0 msgr2=0x55aebf2b5c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:04:08.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.622+0000 7f07e8ff9700 0 client.27271 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T19:04:08.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.622+0000 7f0813789b80 1 -- 172.21.15.39:0/730779417 shutdown_connections 2023-07-21T19:04:08.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.622+0000 7f0813789b80 1 -- 172.21.15.39:0/730779417 wait complete. 2023-07-21T19:04:08.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.622+0000 7f0813789b80 20 client.27271 trim_cache size 0 max 16384 2023-07-21T19:04:08.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.641+0000 7f0813789b80 1 Processor -- start 2023-07-21T19:04:08.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.641+0000 7f0813789b80 1 -- start start 2023-07-21T19:04:08.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.642+0000 7f0813789b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aebf1186b0 0x55aebf2b6c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:08.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.642+0000 7f0813789b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aebf2bc580 0x55aebf2c8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:08.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.642+0000 7f0813789b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aebf2c22d0 0x55aebf2c0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:08.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.642+0000 7f0813789b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aebf2c62c0 con 0x55aebf2bc580 2023-07-21T19:04:08.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.642+0000 7f0813789b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aebf2c3e10 con 0x55aebf2c22d0 2023-07-21T19:04:08.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.642+0000 7f0813789b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aebf2c0640 con 0x55aebf1186b0 2023-07-21T19:04:08.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.642+0000 7f080539f700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aebf1186b0 0x55aebf2b6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:08.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.642+0000 7f080539f700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aebf1186b0 0x55aebf2b6c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:53224/0 (socket says 172.21.15.39:53224) 2023-07-21T19:04:08.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.642+0000 7f080539f700 1 -- 172.21.15.39:0/600716301 learned_addr learned my addr 172.21.15.39:0/600716301 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:04:08.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.642+0000 7f0805ba0700 1 --2- 172.21.15.39:0/600716301 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aebf2c22d0 0x55aebf2c0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:08.655 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.642+0000 7f080539f700 1 -- 172.21.15.39:0/600716301 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aebf2c22d0 msgr2=0x55aebf2c0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:04:08.655 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.642+0000 7f080539f700 1 --2- 172.21.15.39:0/600716301 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aebf2c22d0 0x55aebf2c0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:08.655 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.642+0000 7f080539f700 1 -- 172.21.15.39:0/600716301 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aebf2bc580 msgr2=0x55aebf2c8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:04:08.655 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.642+0000 7f080539f700 1 --2- 172.21.15.39:0/600716301 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aebf2bc580 0x55aebf2c8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:08.655 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.642+0000 7f080539f700 1 -- 172.21.15.39:0/600716301 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aebf0621f0 con 0x55aebf1186b0 2023-07-21T19:04:08.656 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.643+0000 7f080539f700 1 --2- 172.21.15.39:0/600716301 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aebf1186b0 0x55aebf2b6c60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f07fc000c00 tx=0x7f07fc009370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4545481e7a20bce server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:04:08.656 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.643+0000 7f07ebfff700 1 -- 172.21.15.39:0/600716301 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07fc00ee90 con 0x55aebf1186b0 2023-07-21T19:04:08.656 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.643+0000 7f07ebfff700 1 -- 172.21.15.39:0/600716301 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55aebf167490 con 0x55aebf1186b0 2023-07-21T19:04:08.656 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.643+0000 7f07ebfff700 1 -- 172.21.15.39:0/600716301 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07fc004330 con 0x55aebf1186b0 2023-07-21T19:04:08.656 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.644+0000 7f0813789b80 1 -- 172.21.15.39:0/600716301 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aebf1186b0 msgr2=0x55aebf2b6c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:08.656 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.644+0000 7f0813789b80 1 --2- 172.21.15.39:0/600716301 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aebf1186b0 0x55aebf2b6c60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f07fc000c00 tx=0x7f07fc009370 comp rx=0 tx=0).stop 2023-07-21T19:04:08.656 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.644+0000 7f0813789b80 1 -- 172.21.15.39:0/600716301 shutdown_connections 2023-07-21T19:04:08.657 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.644+0000 7f0813789b80 1 --2- 172.21.15.39:0/600716301 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aebf2c22d0 0x55aebf2c0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:08.657 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.644+0000 7f0813789b80 1 --2- 172.21.15.39:0/600716301 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aebf1186b0 0x55aebf2b6c60 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:08.657 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.644+0000 7f0813789b80 1 --2- 172.21.15.39:0/600716301 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aebf2bc580 0x55aebf2c8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:08.657 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.644+0000 7f0813789b80 1 -- 172.21.15.39:0/600716301 >> 172.21.15.39:0/600716301 conn(0x55aebf2b1be0 msgr2=0x55aebf2bfad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:04:08.657 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.644+0000 7f0813789b80 1 -- 172.21.15.39:0/600716301 shutdown_connections 2023-07-21T19:04:08.657 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.644+0000 7f0813789b80 1 -- 172.21.15.39:0/600716301 wait complete. 2023-07-21T19:04:08.657 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.644+0000 7f0813789b80 1 -- start start 2023-07-21T19:04:08.657 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.644+0000 7f0813789b80 1 Processor -- start 2023-07-21T19:04:08.658 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.645+0000 7f0813789b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aebf1186b0 0x55aebf2ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:08.658 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.645+0000 7f0813789b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aebf2bc580 0x55aebf2c9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:08.658 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.645+0000 7f0813789b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aebf2c22d0 0x55aebf2ca540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:08.658 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.645+0000 7f0813789b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aebf2cac80 con 0x55aebf1186b0 2023-07-21T19:04:08.658 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.645+0000 7f0813789b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aebf2cadf0 con 0x55aebf2bc580 2023-07-21T19:04:08.658 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.645+0000 7f0813789b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aebf342310 con 0x55aebf2c22d0 2023-07-21T19:04:08.659 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.645+0000 7f0813789b80 10 client.0 _ll_register_callbacks cb 0x7f07fc00e9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:04:08.659 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.645+0000 7f0805ba0700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aebf2c22d0 0x55aebf2ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:08.659 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.645+0000 7f0805ba0700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aebf2c22d0 0x55aebf2ca540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:53232/0 (socket says 172.21.15.39:53232) 2023-07-21T19:04:08.659 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.645+0000 7f0805ba0700 1 -- 172.21.15.39:0/734534702 learned_addr learned my addr 172.21.15.39:0/734534702 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:04:08.659 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.645+0000 7f0804b9e700 1 --2- 172.21.15.39:0/734534702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aebf2bc580 0x55aebf2c9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:08.661 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.646+0000 7f0805ba0700 1 -- 172.21.15.39:0/734534702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aebf2bc580 msgr2=0x55aebf2c9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:04:08.661 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.646+0000 7f0805ba0700 1 --2- 172.21.15.39:0/734534702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aebf2bc580 0x55aebf2c9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:08.661 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.646+0000 7f0805ba0700 1 -- 172.21.15.39:0/734534702 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aebf1186b0 msgr2=0x55aebf2ccc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:04:08.661 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.646+0000 7f0805ba0700 1 --2- 172.21.15.39:0/734534702 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aebf1186b0 0x55aebf2ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:08.661 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.646+0000 7f0805ba0700 1 -- 172.21.15.39:0/734534702 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aebf024c00 con 0x55aebf2c22d0 2023-07-21T19:04:08.661 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.646+0000 7f0805ba0700 1 --2- 172.21.15.39:0/734534702 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aebf2c22d0 0x55aebf2ca540 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f07f80ca230 tx=0x7f07f80998e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77fb334db9df81dd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:04:08.662 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.646+0000 7f07e8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2023-07-21T19:04:08.662 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.646+0000 7f07e8ff9700 1 -- 172.21.15.39:0/734534702 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f80b2230 con 0x55aebf2c22d0 2023-07-21T19:04:08.662 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.646+0000 7f07e8ff9700 1 -- 172.21.15.39:0/734534702 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f07f80b2390 con 0x55aebf2c22d0 2023-07-21T19:04:08.662 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.647+0000 7f0813789b80 1 -- 172.21.15.39:0/734534702 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f07f809dca0 con 0x55aebf2c22d0 2023-07-21T19:04:08.662 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.647+0000 7f07e8ff9700 1 -- 172.21.15.39:0/734534702 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07f8096380 con 0x55aebf2c22d0 2023-07-21T19:04:08.662 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.647+0000 7f0813789b80 10 client.27277 fetch_fsmap learned FSMap version 103 2023-07-21T19:04:08.662 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.647+0000 7f0813789b80 1 -- 172.21.15.39:0/734534702 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=4+}) v3 -- 0x55aebf03a250 con 0x55aebf2c22d0 2023-07-21T19:04:08.663 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.647+0000 7f07e8ff9700 1 -- 172.21.15.39:0/734534702 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07f809dca0 con 0x55aebf2c22d0 2023-07-21T19:04:08.663 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.647+0000 7f07e8ff9700 1 -- 172.21.15.39:0/734534702 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 103) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f07f80991f0 con 0x55aebf2c22d0 2023-07-21T19:04:08.663 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.647+0000 7f0813789b80 10 client.27277 fetch_fsmap finished waiting for FSMap version 103 2023-07-21T19:04:08.663 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.647+0000 7f0813789b80 10 client.27277 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:04:08.663 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.647+0000 7f0813789b80 20 client.27277 populate_metadata read hostname 'smithi039' 2023-07-21T19:04:08.663 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.647+0000 7f0813789b80 1 --2- 172.21.15.39:0/734534702 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55aebf343780 0x55aebf345b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:08.663 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.647+0000 7f0813789b80 4 client.27277 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:04:08.664 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.647+0000 7f0813789b80 1 -- 172.21.15.39:0/734534702 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55aebf1c8ff0 con 0x55aebf343780 2023-07-21T19:04:08.664 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.648+0000 7f080539f700 1 --2- 172.21.15.39:0/734534702 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55aebf343780 0x55aebf345b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:08.664 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.648+0000 7f07e8ff9700 10 client.27277 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T19:04:08.664 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.648+0000 7f080539f700 1 --2- 172.21.15.39:0/734534702 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55aebf343780 0x55aebf345b60 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44c47d156ccf3434 server_cookie=160f19c39a8cd54a in_seq=0 out_seq=0 2023-07-21T19:04:08.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:04:08 smithi039 ceph-mon[96489]: pgmap v710: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 15 MiB/s wr, 814 op/s 2023-07-21T19:04:08.715 INFO:tasks.workunit.client.0.smithi039.stdout: 10 300683 28.55 MB/sec execute 441 sec latency 67.277 ms 2023-07-21T19:04:08.860 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:04:08 smithi193 ceph-mon[98204]: pgmap v710: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 15 MiB/s wr, 814 op/s 2023-07-21T19:04:08.953 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.943+0000 7f07e8ff9700 1 -- 172.21.15.39:0/734534702 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55aebf1c8ff0 con 0x55aebf343780 2023-07-21T19:04:08.953 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.943+0000 7f07e8ff9700 10 client.27277 handle_command_reply: tid=0 2023-07-21T19:04:08.953 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.943+0000 7f0813789b80 1 client.27277 shutdown 2023-07-21T19:04:08.954 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.944+0000 7f0813789b80 1 -- 172.21.15.39:0/734534702 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aebf2c22d0 msgr2=0x55aebf2ca540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:08.954 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.944+0000 7f0813789b80 1 --2- 172.21.15.39:0/734534702 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aebf2c22d0 0x55aebf2ca540 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f07f80ca230 tx=0x7f07f80998e0 comp rx=0 tx=0).stop 2023-07-21T19:04:08.955 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.945+0000 7f0813789b80 1 -- 172.21.15.39:0/734534702 shutdown_connections 2023-07-21T19:04:08.955 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.945+0000 7f0813789b80 1 --2- 172.21.15.39:0/734534702 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55aebf343780 0x55aebf345b60 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:08.955 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.945+0000 7f0813789b80 1 --2- 172.21.15.39:0/734534702 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aebf2c22d0 0x55aebf2ca540 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:08.955 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.945+0000 7f0813789b80 1 --2- 172.21.15.39:0/734534702 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aebf1186b0 0x55aebf2ccc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:08.955 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.945+0000 7f07e8ff9700 0 client.27277 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T19:04:08.955 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.945+0000 7f0813789b80 1 --2- 172.21.15.39:0/734534702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aebf2bc580 0x55aebf2c9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:08.956 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.945+0000 7f0813789b80 1 -- 172.21.15.39:0/734534702 >> 172.21.15.39:0/734534702 conn(0x55aebf2b1be0 msgr2=0x55aebf2c0a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:04:08.956 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.945+0000 7f0813789b80 1 -- 172.21.15.39:0/734534702 shutdown_connections 2023-07-21T19:04:08.956 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.945+0000 7f0813789b80 1 -- 172.21.15.39:0/734534702 wait complete. 2023-07-21T19:04:08.956 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.945+0000 7f0813789b80 20 client.27277 trim_cache size 0 max 16384 2023-07-21T19:04:08.956 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T19:04:08.957 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T19:04:08.957 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T19:04:08.957 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T19:04:08.957 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.945+0000 7f0807603700 1 -- 172.21.15.39:0/1964972130 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f07ec0a12c0 msgr2=0x7f07ec0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:08.958 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.945+0000 7f0807603700 1 --2- 172.21.15.39:0/1964972130 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f07ec0a12c0 0x7f07ec0a3770 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f08000ffb50 tx=0x7f07fc009260 comp rx=0 tx=0).stop 2023-07-21T19:04:08.958 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.945+0000 7f0807603700 1 -- 172.21.15.39:0/1964972130 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0800104a20 msgr2=0x7f08001a3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:08.958 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.946+0000 7f0807603700 1 --2- 172.21.15.39:0/1964972130 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0800104a20 0x7f08001a3f40 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f07f800e3c0 tx=0x7f07f800e6d0 comp rx=0 tx=0).stop 2023-07-21T19:04:08.958 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.946+0000 7f0807603700 1 -- 172.21.15.39:0/1964972130 shutdown_connections 2023-07-21T19:04:08.958 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.946+0000 7f0807603700 1 --2- 172.21.15.39:0/1964972130 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f07ec0a12c0 0x7f07ec0a3770 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:08.958 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.946+0000 7f0807603700 1 --2- 172.21.15.39:0/1964972130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08000fea10 0x7f080010c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:08.958 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.946+0000 7f0807603700 1 --2- 172.21.15.39:0/1964972130 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0800104a20 0x7f08001a3f40 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:08.959 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.946+0000 7f0807603700 1 --2- 172.21.15.39:0/1964972130 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08000ff3c0 0x7f080010c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:08.959 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.946+0000 7f0807603700 1 -- 172.21.15.39:0/1964972130 >> 172.21.15.39:0/1964972130 conn(0x7f08000fa4f0 msgr2=0x7f08001025d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:04:08.959 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.946+0000 7f0807603700 1 -- 172.21.15.39:0/1964972130 shutdown_connections 2023-07-21T19:04:08.959 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:08.946+0000 7f0807603700 1 -- 172.21.15.39:0/1964972130 wait complete. 2023-07-21T19:04:09.717 INFO:tasks.workunit.client.0.smithi039.stdout: 10 300941 28.51 MB/sec execute 442 sec latency 231.298 ms 2023-07-21T19:04:10.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:04:10 smithi039 ceph-mon[96489]: pgmap v711: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 12 MiB/s wr, 594 op/s 2023-07-21T19:04:10.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:04:10 smithi039 ceph-mon[96489]: osdmap e119: 12 total, 12 up, 12 in 2023-07-21T19:04:10.717 INFO:tasks.workunit.client.0.smithi039.stdout: 10 301089 28.45 MB/sec execute 443 sec latency 331.928 ms 2023-07-21T19:04:10.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:04:10 smithi193 ceph-mon[98204]: pgmap v711: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 12 MiB/s wr, 594 op/s 2023-07-21T19:04:10.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:04:10 smithi193 ceph-mon[98204]: osdmap e119: 12 total, 12 up, 12 in 2023-07-21T19:04:10.883 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:04:10 smithi155 ceph-mon[97934]: pgmap v711: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 12 MiB/s wr, 594 op/s 2023-07-21T19:04:10.883 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:04:10 smithi155 ceph-mon[97934]: osdmap e119: 12 total, 12 up, 12 in 2023-07-21T19:04:11.717 INFO:tasks.workunit.client.0.smithi039.stdout: 10 301362 28.42 MB/sec execute 444 sec latency 398.474 ms 2023-07-21T19:04:12.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:04:12 smithi039 ceph-mon[96489]: pgmap v713: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 13 MiB/s wr, 658 op/s 2023-07-21T19:04:12.718 INFO:tasks.workunit.client.0.smithi039.stdout: 10 301961 28.44 MB/sec execute 445 sec latency 64.415 ms 2023-07-21T19:04:12.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:04:12 smithi193 ceph-mon[98204]: pgmap v713: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 13 MiB/s wr, 658 op/s 2023-07-21T19:04:12.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:04:12 smithi155 ceph-mon[97934]: pgmap v713: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 13 MiB/s wr, 658 op/s 2023-07-21T19:04:13.717 INFO:tasks.workunit.client.0.smithi039.stdout: 10 302615 28.44 MB/sec execute 446 sec latency 72.016 ms 2023-07-21T19:04:14.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:04:14 smithi039 ceph-mon[96489]: pgmap v714: 97 pgs: 32 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 48 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.5 MiB/s wr, 531 op/s 2023-07-21T19:04:14.717 INFO:tasks.workunit.client.0.smithi039.stdout: 10 303254 28.47 MB/sec execute 447 sec latency 70.704 ms 2023-07-21T19:04:14.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:04:14 smithi193 ceph-mon[98204]: pgmap v714: 97 pgs: 32 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 48 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.5 MiB/s wr, 531 op/s 2023-07-21T19:04:14.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:04:14 smithi155 ceph-mon[97934]: pgmap v714: 97 pgs: 32 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 48 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.5 MiB/s wr, 531 op/s 2023-07-21T19:04:15.717 INFO:tasks.workunit.client.0.smithi039.stdout: 10 303796 28.47 MB/sec execute 448 sec latency 79.525 ms 2023-07-21T19:04:16.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:04:16 smithi039 ceph-mon[96489]: pgmap v715: 97 pgs: 32 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 48 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.5 MiB/s wr, 531 op/s 2023-07-21T19:04:16.718 INFO:tasks.workunit.client.0.smithi039.stdout: 10 304272 28.48 MB/sec execute 449 sec latency 74.511 ms 2023-07-21T19:04:16.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:04:16 smithi193 ceph-mon[98204]: pgmap v715: 97 pgs: 32 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 48 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.5 MiB/s wr, 531 op/s 2023-07-21T19:04:16.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:04:16 smithi155 ceph-mon[97934]: pgmap v715: 97 pgs: 32 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 48 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.5 MiB/s wr, 531 op/s 2023-07-21T19:04:17.718 INFO:tasks.workunit.client.0.smithi039.stdout: 10 304679 28.45 MB/sec execute 450 sec latency 93.268 ms 2023-07-21T19:04:18.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:04:18 smithi039 ceph-mon[96489]: pgmap v716: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 7.7 MiB/s wr, 475 op/s 2023-07-21T19:04:18.718 INFO:tasks.workunit.client.0.smithi039.stdout: 10 305081 28.42 MB/sec execute 451 sec latency 114.861 ms 2023-07-21T19:04:18.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:04:18 smithi193 ceph-mon[98204]: pgmap v716: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 7.7 MiB/s wr, 475 op/s 2023-07-21T19:04:18.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:04:18 smithi155 ceph-mon[97934]: pgmap v716: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 7.7 MiB/s wr, 475 op/s 2023-07-21T19:04:19.718 INFO:tasks.workunit.client.0.smithi039.stdout: 10 305601 28.43 MB/sec execute 452 sec latency 142.838 ms 2023-07-21T19:04:20.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:04:20 smithi039 ceph-mon[96489]: pgmap v717: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 7.7 MiB/s wr, 475 op/s 2023-07-21T19:04:20.718 INFO:tasks.workunit.client.0.smithi039.stdout: 10 306091 28.41 MB/sec execute 453 sec latency 166.025 ms 2023-07-21T19:04:20.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:04:20 smithi193 ceph-mon[98204]: pgmap v717: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 7.7 MiB/s wr, 475 op/s 2023-07-21T19:04:20.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:04:20 smithi155 ceph-mon[97934]: pgmap v717: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 7.7 MiB/s wr, 475 op/s 2023-07-21T19:04:21.718 INFO:tasks.workunit.client.0.smithi039.stdout: 10 306544 28.41 MB/sec execute 454 sec latency 354.096 ms 2023-07-21T19:04:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.098+0000 7f4f1c1ab700 1 -- 172.21.15.39:0/2756079386 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f140fe970 msgr2=0x7f4f140fed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.098+0000 7f4f1c1ab700 1 --2- 172.21.15.39:0/2756079386 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f140fe970 0x7f4f140fed40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c00b0a0 tx=0x7f4f0c00b3b0 comp rx=0 tx=0).stop 2023-07-21T19:04:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.098+0000 7f4f1c1ab700 1 -- 172.21.15.39:0/2756079386 shutdown_connections 2023-07-21T19:04:22.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.098+0000 7f4f1c1ab700 1 --2- 172.21.15.39:0/2756079386 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f140fe970 0x7f4f140fed40 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:22.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.098+0000 7f4f1c1ab700 1 --2- 172.21.15.39:0/2756079386 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f1410c520 0x7f4f1410e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:22.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.098+0000 7f4f1c1ab700 1 --2- 172.21.15.39:0/2756079386 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f140ff280 0x7f4f1410bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:22.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.098+0000 7f4f1c1ab700 1 -- 172.21.15.39:0/2756079386 >> 172.21.15.39:0/2756079386 conn(0x7f4f14075bd0 msgr2=0x7f4f14075fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:04:22.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.098+0000 7f4f1c1ab700 1 -- 172.21.15.39:0/2756079386 shutdown_connections 2023-07-21T19:04:22.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.098+0000 7f4f1c1ab700 1 -- 172.21.15.39:0/2756079386 wait complete. 2023-07-21T19:04:22.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.099+0000 7f4f1c1ab700 1 Processor -- start 2023-07-21T19:04:22.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.099+0000 7f4f1c1ab700 1 -- start start 2023-07-21T19:04:22.112 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.099+0000 7f4f1c1ab700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f140fe970 0x7f4f14197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:22.112 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.099+0000 7f4f1c1ab700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f140ff280 0x7f4f14198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:22.112 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.099+0000 7f4f1c1ab700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f1410c520 0x7f4f1419cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:22.112 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.099+0000 7f4f1c1ab700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1419d140 con 0x7f4f1410c520 2023-07-21T19:04:22.112 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.099+0000 7f4f1c1ab700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1419d2b0 con 0x7f4f140ff280 2023-07-21T19:04:22.112 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.099+0000 7f4f1c1ab700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1419d470 con 0x7f4f140fe970 2023-07-21T19:04:22.112 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.099+0000 7f4f19f47700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f140fe970 0x7f4f14197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:22.113 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.099+0000 7f4f19746700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f140ff280 0x7f4f14198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:22.113 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.099+0000 7f4f19f47700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f140fe970 0x7f4f14197f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:53244/0 (socket says 172.21.15.39:53244) 2023-07-21T19:04:22.113 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.099+0000 7f4f19f47700 1 -- 172.21.15.39:0/1078709178 learned_addr learned my addr 172.21.15.39:0/1078709178 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:04:22.113 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.099+0000 7f4f1a748700 1 --2- 172.21.15.39:0/1078709178 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f1410c520 0x7f4f1419cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:22.113 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.099+0000 7f4f19f47700 1 -- 172.21.15.39:0/1078709178 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f140ff280 msgr2=0x7f4f14198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:22.113 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.099+0000 7f4f19f47700 1 --2- 172.21.15.39:0/1078709178 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f140ff280 0x7f4f14198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:22.113 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.099+0000 7f4f19f47700 1 -- 172.21.15.39:0/1078709178 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f1410c520 msgr2=0x7f4f1419cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:22.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.099+0000 7f4f19f47700 1 --2- 172.21.15.39:0/1078709178 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f1410c520 0x7f4f1419cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:22.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.099+0000 7f4f19f47700 1 -- 172.21.15.39:0/1078709178 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f10009520 con 0x7f4f140fe970 2023-07-21T19:04:22.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.100+0000 7f4f19f47700 1 --2- 172.21.15.39:0/1078709178 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f140fe970 0x7f4f14197f50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c0062a0 tx=0x7f4f0c004d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:04:22.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.100+0000 7f4f0affd700 1 -- 172.21.15.39:0/1078709178 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c00e040 con 0x7f4f140fe970 2023-07-21T19:04:22.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.100+0000 7f4f0affd700 1 -- 172.21.15.39:0/1078709178 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f0c009450 con 0x7f4f140fe970 2023-07-21T19:04:22.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.100+0000 7f4f1c1ab700 1 -- 172.21.15.39:0/1078709178 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f0c009d00 con 0x7f4f140fe970 2023-07-21T19:04:22.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.100+0000 7f4f0affd700 1 -- 172.21.15.39:0/1078709178 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c0045d0 con 0x7f4f140fe970 2023-07-21T19:04:22.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.100+0000 7f4f1c1ab700 1 -- 172.21.15.39:0/1078709178 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f141a4db0 con 0x7f4f140fe970 2023-07-21T19:04:22.116 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.103+0000 7f4f0affd700 1 -- 172.21.15.39:0/1078709178 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f4f0c004730 con 0x7f4f140fe970 2023-07-21T19:04:22.117 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.104+0000 7f4f0affd700 1 --2- 172.21.15.39:0/1078709178 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f4f000a1340 0x7f4f000a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:22.117 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.104+0000 7f4f19746700 1 --2- 172.21.15.39:0/1078709178 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f4f000a1340 0x7f4f000a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:22.117 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.104+0000 7f4f0affd700 1 -- 172.21.15.39:0/1078709178 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f4f0c0cde20 con 0x7f4f140fe970 2023-07-21T19:04:22.117 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.104+0000 7f4f28331b80 1 Processor -- start 2023-07-21T19:04:22.117 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.104+0000 7f4f28331b80 1 -- start start 2023-07-21T19:04:22.117 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.104+0000 7f4f28331b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55884eb3c6b0 0x55884ece8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:22.118 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.104+0000 7f4f28331b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55884ece8f80 0x55884eced1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:22.118 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.104+0000 7f4f28331b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55884ece93f0 0x55884eced6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:22.118 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.104+0000 7f4f28331b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55884ebe5c50 con 0x55884ece93f0 2023-07-21T19:04:22.118 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.104+0000 7f4f28331b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55884ec389e0 con 0x55884ece8f80 2023-07-21T19:04:22.118 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.105+0000 7f4f28331b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55884ec31e70 con 0x55884eb3c6b0 2023-07-21T19:04:22.118 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.105+0000 7f4f19f47700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55884eb3c6b0 0x55884ece8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:22.118 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.105+0000 7f4f1a748700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55884ece93f0 0x55884eced6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:22.118 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.105+0000 7f4f19f47700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55884eb3c6b0 0x55884ece8a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:53252/0 (socket says 172.21.15.39:53252) 2023-07-21T19:04:22.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.105+0000 7f4f19f47700 1 -- 172.21.15.39:0/599377540 learned_addr learned my addr 172.21.15.39:0/599377540 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:04:22.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.105+0000 7f4f19746700 1 --2- 172.21.15.39:0/599377540 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55884ece8f80 0x55884eced1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:22.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.105+0000 7f4f19746700 1 --2- 172.21.15.39:0/1078709178 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f4f000a1340 0x7f4f000a37f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4f14199770 tx=0x7f4f10009260 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:04:22.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.105+0000 7f4f19f47700 1 -- 172.21.15.39:0/599377540 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55884ece8f80 msgr2=0x55884eced1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:04:22.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.105+0000 7f4f19f47700 1 --2- 172.21.15.39:0/599377540 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55884ece8f80 0x55884eced1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:22.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.105+0000 7f4f19f47700 1 -- 172.21.15.39:0/599377540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55884ece93f0 msgr2=0x55884eced6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:04:22.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.105+0000 7f4f19f47700 1 --2- 172.21.15.39:0/599377540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55884ece93f0 0x55884eced6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.105+0000 7f4f19f47700 1 -- 172.21.15.39:0/599377540 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55884ea5fd80 con 0x55884eb3c6b0 2023-07-21T19:04:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.105+0000 7f4f19f47700 1 --2- 172.21.15.39:0/599377540 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55884eb3c6b0 0x55884ece8a40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c0979f0 tx=0x7f4f0c0049e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b433e02922b68ce2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:04:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.105+0000 7f4efffff700 1 -- 172.21.15.39:0/599377540 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c0a3340 con 0x55884eb3c6b0 2023-07-21T19:04:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.106+0000 7f4efffff700 1 -- 172.21.15.39:0/599377540 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f0c097690 con 0x55884eb3c6b0 2023-07-21T19:04:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.106+0000 7f4efffff700 1 -- 172.21.15.39:0/599377540 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c09a470 con 0x55884eb3c6b0 2023-07-21T19:04:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.106+0000 7f4f28331b80 1 -- 172.21.15.39:0/599377540 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55884eb3c6b0 msgr2=0x55884ece8a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.106+0000 7f4f28331b80 1 --2- 172.21.15.39:0/599377540 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55884eb3c6b0 0x55884ece8a40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c0979f0 tx=0x7f4f0c0049e0 comp rx=0 tx=0).stop 2023-07-21T19:04:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.107+0000 7f4f28331b80 1 -- 172.21.15.39:0/599377540 shutdown_connections 2023-07-21T19:04:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.107+0000 7f4f28331b80 1 --2- 172.21.15.39:0/599377540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55884ece93f0 0x55884eced6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.107+0000 7f4f28331b80 1 --2- 172.21.15.39:0/599377540 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55884eb3c6b0 0x55884ece8a40 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.107+0000 7f4f28331b80 1 --2- 172.21.15.39:0/599377540 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55884ece8f80 0x55884eced1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.107+0000 7f4f28331b80 1 -- 172.21.15.39:0/599377540 >> 172.21.15.39:0/599377540 conn(0x55884ecd5be0 msgr2=0x55884ecdaf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:04:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.107+0000 7f4f28331b80 1 -- 172.21.15.39:0/599377540 shutdown_connections 2023-07-21T19:04:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.107+0000 7f4f28331b80 1 -- 172.21.15.39:0/599377540 wait complete. 2023-07-21T19:04:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.107+0000 7f4f28331b80 1 -- start start 2023-07-21T19:04:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.107+0000 7f4f28331b80 1 Processor -- start 2023-07-21T19:04:22.123 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.107+0000 7f4f28331b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55884eb3c6b0 0x55884ecf7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:22.123 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.107+0000 7f4f28331b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55884ece8f80 0x55884ecf8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:22.123 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.107+0000 7f4f28331b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55884ece93f0 0x55884ecf1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:22.123 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.107+0000 7f4f28331b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55884ebd1a60 con 0x55884ece93f0 2023-07-21T19:04:22.123 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.108+0000 7f4f28331b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55884ebd1760 con 0x55884ece8f80 2023-07-21T19:04:22.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.108+0000 7f4f28331b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55884eab58c0 con 0x55884eb3c6b0 2023-07-21T19:04:22.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.108+0000 7f4f28331b80 10 client.0 _ll_register_callbacks cb 0x55884ebcf700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:04:22.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.108+0000 7f4f1a748700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55884ece93f0 0x55884ecf1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:22.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.108+0000 7f4f19746700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55884ece8f80 0x55884ecf8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:22.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.108+0000 7f4f19746700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55884ece8f80 0x55884ecf8210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:47304/0 (socket says 172.21.15.39:47304) 2023-07-21T19:04:22.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.108+0000 7f4f1a748700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55884ece93f0 0x55884ecf1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34710/0 (socket says 172.21.15.39:34710) 2023-07-21T19:04:22.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.108+0000 7f4f19746700 1 -- 172.21.15.39:0/3687134454 learned_addr learned my addr 172.21.15.39:0/3687134454 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:04:22.125 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.108+0000 7f4f19746700 1 -- 172.21.15.39:0/3687134454 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55884eb3c6b0 msgr2=0x55884ecf7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:04:22.125 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.108+0000 7f4f19746700 1 --2- 172.21.15.39:0/3687134454 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55884eb3c6b0 0x55884ecf7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:22.125 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.108+0000 7f4f19746700 1 -- 172.21.15.39:0/3687134454 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55884ece93f0 msgr2=0x55884ecf1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:04:22.125 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.108+0000 7f4f19746700 1 --2- 172.21.15.39:0/3687134454 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55884ece93f0 0x55884ecf1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:22.125 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.108+0000 7f4f19746700 1 -- 172.21.15.39:0/3687134454 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f141a4db0 con 0x55884ece8f80 2023-07-21T19:04:22.125 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.108+0000 7f4f19746700 1 --2- 172.21.15.39:0/3687134454 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55884ece8f80 0x55884ecf8210 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4f1000ed20 tx=0x7f4f100047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4dec11f25396644 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:04:22.125 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.108+0000 7f4efd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-21T19:04:22.125 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.109+0000 7f4efd7fa700 1 -- 172.21.15.39:0/3687134454 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f10022070 con 0x55884ece8f80 2023-07-21T19:04:22.126 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.109+0000 7f4efd7fa700 1 -- 172.21.15.39:0/3687134454 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f4f1001dbb0 con 0x55884ece8f80 2023-07-21T19:04:22.126 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.109+0000 7f4f28331b80 1 -- 172.21.15.39:0/3687134454 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55884eb8b490 con 0x55884ece8f80 2023-07-21T19:04:22.126 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.109+0000 7f4efd7fa700 1 -- 172.21.15.39:0/3687134454 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f1001dd70 con 0x55884ece8f80 2023-07-21T19:04:22.126 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.109+0000 7f4f28331b80 10 client.27212 fetch_fsmap learned FSMap version 103 2023-07-21T19:04:22.126 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.109+0000 7f4f28331b80 1 -- 172.21.15.39:0/3687134454 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=4+}) v3 -- 0x55884ea48c00 con 0x55884ece8f80 2023-07-21T19:04:22.126 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.114+0000 7f4efd7fa700 1 -- 172.21.15.39:0/3687134454 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f10033020 con 0x55884ece8f80 2023-07-21T19:04:22.127 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.114+0000 7f4efd7fa700 1 -- 172.21.15.39:0/3687134454 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 103) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f4f10003bf0 con 0x55884ece8f80 2023-07-21T19:04:22.128 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.114+0000 7f4f28331b80 10 client.27212 fetch_fsmap finished waiting for FSMap version 103 2023-07-21T19:04:22.128 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.114+0000 7f4f28331b80 10 client.27212 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:04:22.128 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.114+0000 7f4f28331b80 20 client.27212 populate_metadata read hostname 'smithi039' 2023-07-21T19:04:22.128 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.114+0000 7f4f28331b80 1 --2- 172.21.15.39:0/3687134454 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55884ed02240 0x55884ed04650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:22.128 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.114+0000 7f4f28331b80 4 client.27212 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:04:22.128 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.114+0000 7f4f28331b80 1 -- 172.21.15.39:0/3687134454 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55884ec2ae50 con 0x55884ed02240 2023-07-21T19:04:22.129 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.114+0000 7f4f19f47700 1 --2- 172.21.15.39:0/3687134454 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55884ed02240 0x55884ed04650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:22.129 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.115+0000 7f4efd7fa700 10 client.27212 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T19:04:22.129 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.115+0000 7f4f19f47700 1 --2- 172.21.15.39:0/3687134454 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55884ed02240 0x55884ed04650 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2ebbfc2fb498fcc server_cookie=d6e61fc360c9a5ef in_seq=0 out_seq=0 2023-07-21T19:04:22.129 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.117+0000 7f4efd7fa700 1 -- 172.21.15.39:0/3687134454 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55884ec2ae50 con 0x55884ed02240 2023-07-21T19:04:22.129 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.117+0000 7f4efd7fa700 10 client.27212 handle_command_reply: tid=0 2023-07-21T19:04:22.129 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.117+0000 7f4f28331b80 1 client.27212 shutdown 2023-07-21T19:04:22.129 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.117+0000 7f4f28331b80 1 -- 172.21.15.39:0/3687134454 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55884ece8f80 msgr2=0x55884ecf8210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:22.129 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.117+0000 7f4f28331b80 1 --2- 172.21.15.39:0/3687134454 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55884ece8f80 0x55884ecf8210 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4f1000ed20 tx=0x7f4f100047c0 comp rx=0 tx=0).stop 2023-07-21T19:04:22.130 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.117+0000 7f4f28331b80 1 -- 172.21.15.39:0/3687134454 shutdown_connections 2023-07-21T19:04:22.130 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.117+0000 7f4f28331b80 1 --2- 172.21.15.39:0/3687134454 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55884ed02240 0x55884ed04650 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:22.130 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.117+0000 7f4f28331b80 1 --2- 172.21.15.39:0/3687134454 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55884ece93f0 0x55884ecf1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:22.130 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.117+0000 7f4efd7fa700 0 client.27212 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T19:04:22.130 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.117+0000 7f4f28331b80 1 --2- 172.21.15.39:0/3687134454 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55884eb3c6b0 0x55884ecf7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:22.130 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.117+0000 7f4f28331b80 1 --2- 172.21.15.39:0/3687134454 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55884ece8f80 0x55884ecf8210 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:22.130 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.117+0000 7f4f28331b80 1 -- 172.21.15.39:0/3687134454 >> 172.21.15.39:0/3687134454 conn(0x55884ecd5be0 msgr2=0x55884ecda4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:04:22.131 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.117+0000 7f4f28331b80 1 -- 172.21.15.39:0/3687134454 shutdown_connections 2023-07-21T19:04:22.131 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.117+0000 7f4f28331b80 1 -- 172.21.15.39:0/3687134454 wait complete. 2023-07-21T19:04:22.131 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.117+0000 7f4f28331b80 20 client.27212 trim_cache size 0 max 16384 2023-07-21T19:04:22.147 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.138+0000 7f4f28331b80 1 Processor -- start 2023-07-21T19:04:22.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.138+0000 7f4f28331b80 1 -- start start 2023-07-21T19:04:22.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.138+0000 7f4f28331b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55884eb3c6b0 0x55884ecdf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:22.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.138+0000 7f4f28331b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55884ece8f80 0x55884eceafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:22.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.138+0000 7f4f28331b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55884ece93f0 0x55884ecdac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:22.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.138+0000 7f4f28331b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55884ece99f0 con 0x55884eb3c6b0 2023-07-21T19:04:22.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.138+0000 7f4f28331b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55884ece97c0 con 0x55884ece8f80 2023-07-21T19:04:22.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.138+0000 7f4f28331b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55884ecdba00 con 0x55884ece93f0 2023-07-21T19:04:22.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.139+0000 7f4f19f47700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55884eb3c6b0 0x55884ecdf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:22.150 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.139+0000 7f4f19f47700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55884eb3c6b0 0x55884ecdf740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34714/0 (socket says 172.21.15.39:34714) 2023-07-21T19:04:22.150 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.139+0000 7f4f19f47700 1 -- 172.21.15.39:0/2198623944 learned_addr learned my addr 172.21.15.39:0/2198623944 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:04:22.150 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.139+0000 7f4f19746700 1 --2- 172.21.15.39:0/2198623944 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55884ece8f80 0x55884eceafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:22.150 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.139+0000 7f4f19f47700 1 -- 172.21.15.39:0/2198623944 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55884ece93f0 msgr2=0x55884ecdac60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T19:04:22.150 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.139+0000 7f4f19f47700 1 --2- 172.21.15.39:0/2198623944 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55884ece93f0 0x55884ecdac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:22.150 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.139+0000 7f4f19f47700 1 -- 172.21.15.39:0/2198623944 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55884ece8f80 msgr2=0x55884eceafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:04:22.150 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.139+0000 7f4f19f47700 1 --2- 172.21.15.39:0/2198623944 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55884ece8f80 0x55884eceafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:22.151 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.139+0000 7f4f19f47700 1 -- 172.21.15.39:0/2198623944 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55884ea861f0 con 0x55884eb3c6b0 2023-07-21T19:04:22.151 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.139+0000 7f4f19746700 1 --2- 172.21.15.39:0/2198623944 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55884ece8f80 0x55884eceafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:04:22.151 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.139+0000 7f4f19f47700 1 --2- 172.21.15.39:0/2198623944 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55884eb3c6b0 0x55884ecdf740 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c09af10 tx=0x7f4f0c00b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a38c00e9eaf8de3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:04:22.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.140+0000 7f4f08ff9700 1 -- 172.21.15.39:0/2198623944 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c0a3340 con 0x55884eb3c6b0 2023-07-21T19:04:22.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.140+0000 7f4f08ff9700 1 -- 172.21.15.39:0/2198623944 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f0c097800 con 0x55884eb3c6b0 2023-07-21T19:04:22.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.140+0000 7f4f08ff9700 1 -- 172.21.15.39:0/2198623944 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c09b1e0 con 0x55884eb3c6b0 2023-07-21T19:04:22.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.141+0000 7f4f28331b80 1 -- 172.21.15.39:0/2198623944 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55884eb3c6b0 msgr2=0x55884ecdf740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:22.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.141+0000 7f4f28331b80 1 --2- 172.21.15.39:0/2198623944 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55884eb3c6b0 0x55884ecdf740 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c09af10 tx=0x7f4f0c00b3b0 comp rx=0 tx=0).stop 2023-07-21T19:04:22.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.141+0000 7f4f28331b80 1 -- 172.21.15.39:0/2198623944 shutdown_connections 2023-07-21T19:04:22.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.141+0000 7f4f28331b80 1 --2- 172.21.15.39:0/2198623944 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55884ece93f0 0x55884ecdac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:22.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.141+0000 7f4f28331b80 1 --2- 172.21.15.39:0/2198623944 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55884eb3c6b0 0x55884ecdf740 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:22.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.141+0000 7f4f28331b80 1 --2- 172.21.15.39:0/2198623944 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55884ece8f80 0x55884eceafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:22.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.141+0000 7f4f28331b80 1 -- 172.21.15.39:0/2198623944 >> 172.21.15.39:0/2198623944 conn(0x55884ecd5be0 msgr2=0x55884ecdf030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:04:22.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.143+0000 7f4f28331b80 1 -- 172.21.15.39:0/2198623944 shutdown_connections 2023-07-21T19:04:22.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.143+0000 7f4f28331b80 1 -- 172.21.15.39:0/2198623944 wait complete. 2023-07-21T19:04:22.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.144+0000 7f4f28331b80 1 -- start start 2023-07-21T19:04:22.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.144+0000 7f4f28331b80 1 Processor -- start 2023-07-21T19:04:22.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.144+0000 7f4f28331b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55884eb3c6b0 0x55884ed4a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:22.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.144+0000 7f4f28331b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55884ece8f80 0x55884ed4a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:22.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.144+0000 7f4f28331b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55884ece93f0 0x55884ed4f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:22.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.144+0000 7f4f28331b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55884ed4f6a0 con 0x55884ece93f0 2023-07-21T19:04:22.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.144+0000 7f4f28331b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55884ed4f810 con 0x55884ece8f80 2023-07-21T19:04:22.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.144+0000 7f4f28331b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55884ed4f9d0 con 0x55884eb3c6b0 2023-07-21T19:04:22.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.144+0000 7f4f28331b80 10 client.0 _ll_register_callbacks cb 0x7f4f1000e8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:04:22.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.144+0000 7f4f19f47700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55884eb3c6b0 0x55884ed4a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:22.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.144+0000 7f4f19746700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55884ece8f80 0x55884ed4a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:22.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.144+0000 7f4f19f47700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55884eb3c6b0 0x55884ed4a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:53272/0 (socket says 172.21.15.39:53272) 2023-07-21T19:04:22.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.144+0000 7f4f19f47700 1 -- 172.21.15.39:0/2797306707 learned_addr learned my addr 172.21.15.39:0/2797306707 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:04:22.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.145+0000 7f4f1a748700 1 --2- 172.21.15.39:0/2797306707 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55884ece93f0 0x55884ed4f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:22.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.145+0000 7f4f19746700 1 -- 172.21.15.39:0/2797306707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55884eb3c6b0 msgr2=0x55884ed4a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:04:22.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.145+0000 7f4f19746700 1 --2- 172.21.15.39:0/2797306707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55884eb3c6b0 0x55884ed4a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:22.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.145+0000 7f4f19746700 1 -- 172.21.15.39:0/2797306707 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55884ece93f0 msgr2=0x55884ed4f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:04:22.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.145+0000 7f4f19746700 1 --2- 172.21.15.39:0/2797306707 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55884ece93f0 0x55884ed4f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:22.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.145+0000 7f4f19746700 1 -- 172.21.15.39:0/2797306707 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55884ea48c00 con 0x55884ece8f80 2023-07-21T19:04:22.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.145+0000 7f4f19746700 1 --2- 172.21.15.39:0/2797306707 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55884ece8f80 0x55884ed4a7a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4f10000c00 tx=0x7f4f1002fb90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91f9deab8f2e815a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:04:22.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.145+0000 7f4efd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-21T19:04:22.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.145+0000 7f4efd7fa700 1 -- 172.21.15.39:0/2797306707 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f10022040 con 0x55884ece8f80 2023-07-21T19:04:22.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.145+0000 7f4f28331b80 1 -- 172.21.15.39:0/2797306707 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4f0c097690 con 0x55884ece8f80 2023-07-21T19:04:22.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.145+0000 7f4efd7fa700 1 -- 172.21.15.39:0/2797306707 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f4f10034080 con 0x55884ece8f80 2023-07-21T19:04:22.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.146+0000 7f4efd7fa700 1 -- 172.21.15.39:0/2797306707 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f1002f650 con 0x55884ece8f80 2023-07-21T19:04:22.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.146+0000 7f4f28331b80 10 client.27218 fetch_fsmap learned FSMap version 103 2023-07-21T19:04:22.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.146+0000 7f4f28331b80 1 -- 172.21.15.39:0/2797306707 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=4+}) v3 -- 0x55884ea5fd80 con 0x55884ece8f80 2023-07-21T19:04:22.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.146+0000 7f4efd7fa700 1 -- 172.21.15.39:0/2797306707 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f0c097690 con 0x55884ece8f80 2023-07-21T19:04:22.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.146+0000 7f4efd7fa700 1 -- 172.21.15.39:0/2797306707 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 103) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f4f10003950 con 0x55884ece8f80 2023-07-21T19:04:22.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.146+0000 7f4f28331b80 10 client.27218 fetch_fsmap finished waiting for FSMap version 103 2023-07-21T19:04:22.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.146+0000 7f4f28331b80 10 client.27218 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:04:22.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.146+0000 7f4f28331b80 20 client.27218 populate_metadata read hostname 'smithi039' 2023-07-21T19:04:22.160 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.146+0000 7f4f28331b80 1 --2- 172.21.15.39:0/2797306707 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55884ece0f40 0x55884ece3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:22.160 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.146+0000 7f4f28331b80 4 client.27218 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:04:22.160 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.146+0000 7f4f28331b80 1 -- 172.21.15.39:0/2797306707 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55884ebecff0 con 0x55884ece0f40 2023-07-21T19:04:22.160 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.147+0000 7f4f19f47700 1 --2- 172.21.15.39:0/2797306707 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55884ece0f40 0x55884ece3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:22.160 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.148+0000 7f4f19f47700 1 --2- 172.21.15.39:0/2797306707 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55884ece0f40 0x55884ece3350 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47fe1097d1efa2bc server_cookie=d1201df862a23a55 in_seq=0 out_seq=0 2023-07-21T19:04:22.160 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.148+0000 7f4efd7fa700 10 client.27218 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T19:04:22.421 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.411+0000 7f4efd7fa700 1 -- 172.21.15.39:0/2797306707 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55884ebecff0 con 0x55884ece0f40 2023-07-21T19:04:22.421 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.411+0000 7f4efd7fa700 10 client.27218 handle_command_reply: tid=0 2023-07-21T19:04:22.421 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.411+0000 7f4f28331b80 1 client.27218 shutdown 2023-07-21T19:04:22.421 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.411+0000 7f4f28331b80 1 -- 172.21.15.39:0/2797306707 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55884ece8f80 msgr2=0x55884ed4a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:22.421 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.411+0000 7f4f28331b80 1 --2- 172.21.15.39:0/2797306707 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55884ece8f80 0x55884ed4a7a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4f10000c00 tx=0x7f4f1002fb90 comp rx=0 tx=0).stop 2023-07-21T19:04:22.422 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.411+0000 7f4f28331b80 1 -- 172.21.15.39:0/2797306707 shutdown_connections 2023-07-21T19:04:22.422 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.411+0000 7f4f28331b80 1 --2- 172.21.15.39:0/2797306707 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55884ece0f40 0x55884ece3350 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:22.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.411+0000 7f4f28331b80 1 --2- 172.21.15.39:0/2797306707 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55884ece93f0 0x55884ed4f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:22.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.411+0000 7f4f28331b80 1 --2- 172.21.15.39:0/2797306707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55884eb3c6b0 0x55884ed4a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:22.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.411+0000 7f4f28331b80 1 --2- 172.21.15.39:0/2797306707 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55884ece8f80 0x55884ed4a7a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:22.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.411+0000 7f4f28331b80 1 -- 172.21.15.39:0/2797306707 >> 172.21.15.39:0/2797306707 conn(0x55884ecd5be0 msgr2=0x55884ecefaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:04:22.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.412+0000 7f4efd7fa700 0 client.27218 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T19:04:22.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.412+0000 7f4f28331b80 1 -- 172.21.15.39:0/2797306707 shutdown_connections 2023-07-21T19:04:22.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.412+0000 7f4f28331b80 1 -- 172.21.15.39:0/2797306707 wait complete. 2023-07-21T19:04:22.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.412+0000 7f4f28331b80 20 client.27218 trim_cache size 0 max 16384 2023-07-21T19:04:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T19:04:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T19:04:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T19:04:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T19:04:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.412+0000 7f4f1c1ab700 1 -- 172.21.15.39:0/1078709178 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f4f000a1340 msgr2=0x7f4f000a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.412+0000 7f4f1c1ab700 1 --2- 172.21.15.39:0/1078709178 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f4f000a1340 0x7f4f000a37f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4f14199770 tx=0x7f4f10009260 comp rx=0 tx=0).stop 2023-07-21T19:04:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.412+0000 7f4f1c1ab700 1 -- 172.21.15.39:0/1078709178 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f140fe970 msgr2=0x7f4f14197f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.412+0000 7f4f1c1ab700 1 --2- 172.21.15.39:0/1078709178 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f140fe970 0x7f4f14197f50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c0062a0 tx=0x7f4f0c004d00 comp rx=0 tx=0).stop 2023-07-21T19:04:22.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.412+0000 7f4f1c1ab700 1 -- 172.21.15.39:0/1078709178 shutdown_connections 2023-07-21T19:04:22.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.412+0000 7f4f1c1ab700 1 --2- 172.21.15.39:0/1078709178 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f1410c520 0x7f4f1419cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:22.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.412+0000 7f4f1c1ab700 1 --2- 172.21.15.39:0/1078709178 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f140fe970 0x7f4f14197f50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:22.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.412+0000 7f4f1c1ab700 1 --2- 172.21.15.39:0/1078709178 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f4f000a1340 0x7f4f000a37f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:22.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.412+0000 7f4f1c1ab700 1 --2- 172.21.15.39:0/1078709178 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f140ff280 0x7f4f14198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:22.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.413+0000 7f4f1c1ab700 1 -- 172.21.15.39:0/1078709178 >> 172.21.15.39:0/1078709178 conn(0x7f4f14075bd0 msgr2=0x7f4f140fc350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:04:22.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.413+0000 7f4f1c1ab700 1 -- 172.21.15.39:0/1078709178 shutdown_connections 2023-07-21T19:04:22.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:22.413+0000 7f4f1c1ab700 1 -- 172.21.15.39:0/1078709178 wait complete. 2023-07-21T19:04:22.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:04:22 smithi155 ceph-mon[97934]: pgmap v718: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 7.0 MiB/s wr, 434 op/s 2023-07-21T19:04:22.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:04:22 smithi155 ceph-mon[97934]: osdmap e120: 12 total, 12 up, 12 in 2023-07-21T19:04:22.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:04:22 smithi039 ceph-mon[96489]: pgmap v718: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 7.0 MiB/s wr, 434 op/s 2023-07-21T19:04:22.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:04:22 smithi039 ceph-mon[96489]: osdmap e120: 12 total, 12 up, 12 in 2023-07-21T19:04:22.718 INFO:tasks.workunit.client.0.smithi039.stdout: 10 306981 28.41 MB/sec execute 455 sec latency 265.212 ms 2023-07-21T19:04:22.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:04:22 smithi193 ceph-mon[98204]: pgmap v718: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 7.0 MiB/s wr, 434 op/s 2023-07-21T19:04:22.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:04:22 smithi193 ceph-mon[98204]: osdmap e120: 12 total, 12 up, 12 in 2023-07-21T19:04:23.718 INFO:tasks.workunit.client.0.smithi039.stdout: 10 307446 28.40 MB/sec execute 456 sec latency 196.289 ms 2023-07-21T19:04:24.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:04:24 smithi155 ceph-mon[97934]: pgmap v720: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 8.6 MiB/s wr, 431 op/s 2023-07-21T19:04:24.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:04:24 smithi039 ceph-mon[96489]: pgmap v720: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 8.6 MiB/s wr, 431 op/s 2023-07-21T19:04:24.718 INFO:tasks.workunit.client.0.smithi039.stdout: 10 307958 28.41 MB/sec execute 457 sec latency 72.791 ms 2023-07-21T19:04:24.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:04:24 smithi193 ceph-mon[98204]: pgmap v720: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 8.6 MiB/s wr, 431 op/s 2023-07-21T19:04:25.718 INFO:tasks.workunit.client.0.smithi039.stdout: 10 308391 28.38 MB/sec execute 458 sec latency 75.777 ms 2023-07-21T19:04:26.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:04:26 smithi039 ceph-mon[96489]: pgmap v721: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 8.6 MiB/s wr, 431 op/s 2023-07-21T19:04:26.718 INFO:tasks.workunit.client.0.smithi039.stdout: 10 308992 28.40 MB/sec execute 459 sec latency 74.111 ms 2023-07-21T19:04:26.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:04:26 smithi193 ceph-mon[98204]: pgmap v721: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 8.6 MiB/s wr, 431 op/s 2023-07-21T19:04:26.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:04:26 smithi155 ceph-mon[97934]: pgmap v721: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 8.6 MiB/s wr, 431 op/s 2023-07-21T19:04:27.718 INFO:tasks.workunit.client.0.smithi039.stdout: 10 309563 28.41 MB/sec execute 460 sec latency 83.497 ms 2023-07-21T19:04:28.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:04:28 smithi039 ceph-mon[96489]: pgmap v722: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 9.2 MiB/s wr, 481 op/s 2023-07-21T19:04:28.718 INFO:tasks.workunit.client.0.smithi039.stdout: 10 310089 28.39 MB/sec execute 461 sec latency 116.270 ms 2023-07-21T19:04:28.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:04:28 smithi193 ceph-mon[98204]: pgmap v722: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 9.2 MiB/s wr, 481 op/s 2023-07-21T19:04:28.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:04:28 smithi155 ceph-mon[97934]: pgmap v722: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 9.2 MiB/s wr, 481 op/s 2023-07-21T19:04:29.718 INFO:tasks.workunit.client.0.smithi039.stdout: 10 310674 28.40 MB/sec execute 462 sec latency 95.914 ms 2023-07-21T19:04:30.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:04:30 smithi039 ceph-mon[96489]: pgmap v723: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 9.2 MiB/s wr, 481 op/s 2023-07-21T19:04:30.718 INFO:tasks.workunit.client.0.smithi039.stdout: 10 311259 28.43 MB/sec execute 463 sec latency 67.292 ms 2023-07-21T19:04:30.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:04:30 smithi193 ceph-mon[98204]: pgmap v723: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 9.2 MiB/s wr, 481 op/s 2023-07-21T19:04:30.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:04:30 smithi155 ceph-mon[97934]: pgmap v723: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 9.2 MiB/s wr, 481 op/s 2023-07-21T19:04:31.718 INFO:tasks.workunit.client.0.smithi039.stdout: 10 311733 28.40 MB/sec execute 464 sec latency 79.650 ms 2023-07-21T19:04:32.721 INFO:tasks.workunit.client.0.smithi039.stdout: 10 312050 28.38 MB/sec execute 465 sec latency 371.063 ms 2023-07-21T19:04:32.867 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:04:32 smithi193 ceph-mon[98204]: pgmap v724: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 9.4 MiB/s wr, 491 op/s 2023-07-21T19:04:32.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:04:32 smithi155 ceph-mon[97934]: pgmap v724: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 9.4 MiB/s wr, 491 op/s 2023-07-21T19:04:32.932 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:04:32 smithi039 ceph-mon[96489]: pgmap v724: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 9.4 MiB/s wr, 491 op/s 2023-07-21T19:04:33.720 INFO:tasks.workunit.client.0.smithi039.stdout: 10 312077 28.32 MB/sec execute 466 sec latency 1371.106 ms 2023-07-21T19:04:34.720 INFO:tasks.workunit.client.0.smithi039.stdout: 10 312292 28.28 MB/sec execute 467 sec latency 1405.791 ms 2023-07-21T19:04:34.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:04:34 smithi193 ceph-mon[98204]: pgmap v725: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 8.2 MiB/s wr, 442 op/s 2023-07-21T19:04:34.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:04:34 smithi155 ceph-mon[97934]: pgmap v725: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 8.2 MiB/s wr, 442 op/s 2023-07-21T19:04:34.927 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:04:34 smithi039 ceph-mon[96489]: pgmap v725: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 8.2 MiB/s wr, 442 op/s 2023-07-21T19:04:35.648 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.637+0000 7fb49ef19700 1 -- 172.21.15.39:0/3043688263 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb49810a130 msgr2=0x7fb49810c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:35.648 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.637+0000 7fb49ef19700 1 --2- 172.21.15.39:0/3043688263 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb49810a130 0x7fb49810c570 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fb490014a30 tx=0x7fb490014d40 comp rx=0 tx=0).stop 2023-07-21T19:04:35.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.640+0000 7fb49ef19700 1 -- 172.21.15.39:0/3043688263 shutdown_connections 2023-07-21T19:04:35.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.640+0000 7fb49ef19700 1 --2- 172.21.15.39:0/3043688263 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb498100c10 0x7fb498100fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:35.654 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.640+0000 7fb49ef19700 1 --2- 172.21.15.39:0/3043688263 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb49810a130 0x7fb49810c570 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:35.654 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.640+0000 7fb49ef19700 1 --2- 172.21.15.39:0/3043688263 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb498101520 0x7fb498109bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:35.654 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.640+0000 7fb49ef19700 1 -- 172.21.15.39:0/3043688263 >> 172.21.15.39:0/3043688263 conn(0x7fb4980769b0 msgr2=0x7fb498076db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:04:35.656 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.642+0000 7fb49ef19700 1 -- 172.21.15.39:0/3043688263 shutdown_connections 2023-07-21T19:04:35.656 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.642+0000 7fb49ef19700 1 -- 172.21.15.39:0/3043688263 wait complete. 2023-07-21T19:04:35.656 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.643+0000 7fb49ef19700 1 Processor -- start 2023-07-21T19:04:35.657 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.643+0000 7fb49ef19700 1 -- start start 2023-07-21T19:04:35.657 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.644+0000 7fb49ef19700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb498100c10 0x7fb4981aaa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:35.657 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.644+0000 7fb49ef19700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb498101520 0x7fb4981aaf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:35.657 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.644+0000 7fb49ef19700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb49810a130 0x7fb4981a48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:35.657 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.644+0000 7fb49ef19700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4981a4f20 con 0x7fb49810a130 2023-07-21T19:04:35.657 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.644+0000 7fb49ef19700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4981a5090 con 0x7fb498100c10 2023-07-21T19:04:35.657 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.644+0000 7fb49ef19700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4981a5250 con 0x7fb498101520 2023-07-21T19:04:35.658 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.644+0000 7fb497fff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb498101520 0x7fb4981aaf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:35.658 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.644+0000 7fb497fff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb498101520 0x7fb4981aaf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:53290/0 (socket says 172.21.15.39:53290) 2023-07-21T19:04:35.658 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.644+0000 7fb497fff700 1 -- 172.21.15.39:0/2913747643 learned_addr learned my addr 172.21.15.39:0/2913747643 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:04:35.658 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.644+0000 7fb497fff700 1 -- 172.21.15.39:0/2913747643 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb498100c10 msgr2=0x7fb4981aaa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:35.658 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.644+0000 7fb497fff700 1 --2- 172.21.15.39:0/2913747643 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb498100c10 0x7fb4981aaa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:35.659 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.644+0000 7fb497fff700 1 -- 172.21.15.39:0/2913747643 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb49810a130 msgr2=0x7fb4981a48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:35.659 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.644+0000 7fb497fff700 1 --2- 172.21.15.39:0/2913747643 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb49810a130 0x7fb4981a48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:35.659 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.644+0000 7fb497fff700 1 -- 172.21.15.39:0/2913747643 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb490014660 con 0x7fb498101520 2023-07-21T19:04:35.659 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.645+0000 7fb497fff700 1 --2- 172.21.15.39:0/2913747643 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb498101520 0x7fb4981aaf90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb48400d950 tx=0x7fb48400dd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:04:35.659 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.645+0000 7fb495ffb700 1 -- 172.21.15.39:0/2913747643 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb484009b40 con 0x7fb498101520 2023-07-21T19:04:35.659 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.645+0000 7fb495ffb700 1 -- 172.21.15.39:0/2913747643 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb484009ca0 con 0x7fb498101520 2023-07-21T19:04:35.659 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.645+0000 7fb49ef19700 1 -- 172.21.15.39:0/2913747643 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4981a5530 con 0x7fb498101520 2023-07-21T19:04:35.660 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.645+0000 7fb49ef19700 1 -- 172.21.15.39:0/2913747643 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4981b1a40 con 0x7fb498101520 2023-07-21T19:04:35.661 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.650+0000 7fb4ab09fb80 1 Processor -- start 2023-07-21T19:04:35.661 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.650+0000 7fb495ffb700 1 -- 172.21.15.39:0/2913747643 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48400f620 con 0x7fb498101520 2023-07-21T19:04:35.661 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.650+0000 7fb4ab09fb80 1 -- start start 2023-07-21T19:04:35.665 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.652+0000 7fb4ab09fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56106dd8d6b0 0x56106df36d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:35.665 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.652+0000 7fb4ab09fb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106df372d0 0x56106df31040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:35.665 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.652+0000 7fb4ab09fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56106df31580 0x56106df319d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:35.665 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.652+0000 7fb4ab09fb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x56106de36c50 con 0x56106df372d0 2023-07-21T19:04:35.665 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.652+0000 7fb4ab09fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56106de899e0 con 0x56106dd8d6b0 2023-07-21T19:04:35.666 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.652+0000 7fb4ab09fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x56106de82e70 con 0x56106df31580 2023-07-21T19:04:35.666 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.652+0000 7fb49d4b6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56106dd8d6b0 0x56106df36d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:35.666 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.652+0000 7fb49d4b6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56106dd8d6b0 0x56106df36d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:47336/0 (socket says 172.21.15.39:47336) 2023-07-21T19:04:35.666 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.652+0000 7fb49d4b6700 1 -- 172.21.15.39:0/3954401 learned_addr learned my addr 172.21.15.39:0/3954401 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:04:35.666 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.652+0000 7fb49d4b6700 1 -- 172.21.15.39:0/3954401 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56106df31580 msgr2=0x56106df319d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T19:04:35.666 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.652+0000 7fb49d4b6700 1 --2- 172.21.15.39:0/3954401 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56106df31580 0x56106df319d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:35.667 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.652+0000 7fb49d4b6700 1 -- 172.21.15.39:0/3954401 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106df372d0 msgr2=0x56106df31040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T19:04:35.667 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.652+0000 7fb49d4b6700 1 --2- 172.21.15.39:0/3954401 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106df372d0 0x56106df31040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:35.667 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.652+0000 7fb49d4b6700 1 -- 172.21.15.39:0/3954401 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56106dcb0d80 con 0x56106dd8d6b0 2023-07-21T19:04:35.667 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.652+0000 7fb49d4b6700 1 --2- 172.21.15.39:0/3954401 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56106dd8d6b0 0x56106df36d90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb49001f040 tx=0x7fb4900142e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56bbd4b8b027880b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:04:35.668 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.655+0000 7fb482ffd700 1 -- 172.21.15.39:0/3954401 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb49001dda0 con 0x56106dd8d6b0 2023-07-21T19:04:35.669 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.655+0000 7fb482ffd700 1 -- 172.21.15.39:0/3954401 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb490013af0 con 0x56106dd8d6b0 2023-07-21T19:04:35.669 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.655+0000 7fb482ffd700 1 -- 172.21.15.39:0/3954401 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb49001c880 con 0x56106dd8d6b0 2023-07-21T19:04:35.669 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.656+0000 7fb4ab09fb80 1 -- 172.21.15.39:0/3954401 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56106dd8d6b0 msgr2=0x56106df36d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:35.670 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.656+0000 7fb4ab09fb80 1 --2- 172.21.15.39:0/3954401 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56106dd8d6b0 0x56106df36d90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb49001f040 tx=0x7fb4900142e0 comp rx=0 tx=0).stop 2023-07-21T19:04:35.670 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.658+0000 7fb495ffb700 1 -- 172.21.15.39:0/2913747643 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fb48400f780 con 0x7fb498101520 2023-07-21T19:04:35.670 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.658+0000 7fb495ffb700 1 --2- 172.21.15.39:0/2913747643 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fb4880a1340 0x7fb4880a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:35.671 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.659+0000 7fb495ffb700 1 -- 172.21.15.39:0/2913747643 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fb4840cd0e0 con 0x7fb498101520 2023-07-21T19:04:35.671 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.659+0000 7fb49d4b6700 1 --2- 172.21.15.39:0/2913747643 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fb4880a1340 0x7fb4880a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:35.671 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.659+0000 7fb49d4b6700 1 --2- 172.21.15.39:0/2913747643 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fb4880a1340 0x7fb4880a37f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb49001caf0 tx=0x7fb49001c9e0 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:04:35.672 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.661+0000 7fb4ab09fb80 1 -- 172.21.15.39:0/3954401 shutdown_connections 2023-07-21T19:04:35.672 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.661+0000 7fb4ab09fb80 1 --2- 172.21.15.39:0/3954401 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56106dd8d6b0 0x56106df36d90 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:35.673 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.661+0000 7fb4ab09fb80 1 --2- 172.21.15.39:0/3954401 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56106df31580 0x56106df319d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:35.673 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.661+0000 7fb4ab09fb80 1 --2- 172.21.15.39:0/3954401 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106df372d0 0x56106df31040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:35.673 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.661+0000 7fb4ab09fb80 1 -- 172.21.15.39:0/3954401 >> 172.21.15.39:0/3954401 conn(0x56106df26be0 msgr2=0x56106df2bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:04:35.675 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.664+0000 7fb4ab09fb80 1 -- 172.21.15.39:0/3954401 shutdown_connections 2023-07-21T19:04:35.678 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.668+0000 7fb4ab09fb80 1 -- 172.21.15.39:0/3954401 wait complete. 2023-07-21T19:04:35.679 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.668+0000 7fb4ab09fb80 1 -- start start 2023-07-21T19:04:35.679 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.668+0000 7fb4ab09fb80 1 Processor -- start 2023-07-21T19:04:35.681 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.669+0000 7fb4ab09fb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106dd8d6b0 0x56106df3a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:35.681 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.669+0000 7fb4ab09fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56106df31580 0x56106df3ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:35.682 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.669+0000 7fb4ab09fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56106df372d0 0x56106df3b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:35.682 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.669+0000 7fb4ab09fb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x56106de22a60 con 0x56106dd8d6b0 2023-07-21T19:04:35.682 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.669+0000 7fb4ab09fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56106de22760 con 0x56106df31580 2023-07-21T19:04:35.682 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.669+0000 7fb4ab09fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x56106dd068c0 con 0x56106df372d0 2023-07-21T19:04:35.682 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.669+0000 7fb4ab09fb80 10 client.0 _ll_register_callbacks cb 0x56106de20700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:04:35.682 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.669+0000 7fb49ccb5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106dd8d6b0 0x56106df3a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:35.682 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.669+0000 7fb49ccb5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106dd8d6b0 0x56106df3a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34744/0 (socket says 172.21.15.39:34744) 2023-07-21T19:04:35.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.669+0000 7fb49ccb5700 1 -- 172.21.15.39:0/1328914795 learned_addr learned my addr 172.21.15.39:0/1328914795 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:04:35.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.669+0000 7fb49ccb5700 1 -- 172.21.15.39:0/1328914795 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56106df372d0 msgr2=0x56106df3b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:04:35.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.669+0000 7fb49ccb5700 1 --2- 172.21.15.39:0/1328914795 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56106df372d0 0x56106df3b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:35.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.669+0000 7fb49ccb5700 1 -- 172.21.15.39:0/1328914795 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56106df31580 msgr2=0x56106df3ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:04:35.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.669+0000 7fb49ccb5700 1 --2- 172.21.15.39:0/1328914795 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56106df31580 0x56106df3ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:35.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.669+0000 7fb49ccb5700 1 -- 172.21.15.39:0/1328914795 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4980fee60 con 0x56106dd8d6b0 2023-07-21T19:04:35.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.670+0000 7fb49ccb5700 1 --2- 172.21.15.39:0/1328914795 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106dd8d6b0 0x56106df3a940 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fb48c009f00 tx=0x7fb48c00b3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f35d0d11f699c8f8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:04:35.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.670+0000 7fb47bfff700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T19:04:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.670+0000 7fb47bfff700 1 -- 172.21.15.39:0/1328914795 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48c011420 con 0x56106dd8d6b0 2023-07-21T19:04:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.670+0000 7fb47bfff700 1 -- 172.21.15.39:0/1328914795 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fb48c011580 con 0x56106dd8d6b0 2023-07-21T19:04:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.670+0000 7fb4ab09fb80 1 -- 172.21.15.39:0/1328914795 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56106dddc490 con 0x56106dd8d6b0 2023-07-21T19:04:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.670+0000 7fb47bfff700 1 -- 172.21.15.39:0/1328914795 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb48c011740 con 0x56106dd8d6b0 2023-07-21T19:04:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.670+0000 7fb4ab09fb80 10 client.17784 fetch_fsmap learned FSMap version 103 2023-07-21T19:04:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.670+0000 7fb4ab09fb80 1 -- 172.21.15.39:0/1328914795 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=4+}) v3 -- 0x56106dc99c00 con 0x56106dd8d6b0 2023-07-21T19:04:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.670+0000 7fb47bfff700 1 -- 172.21.15.39:0/1328914795 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56106dddc490 con 0x56106dd8d6b0 2023-07-21T19:04:35.685 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.670+0000 7fb47bfff700 1 -- 172.21.15.39:0/1328914795 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 103) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fb48c019ad0 con 0x56106dd8d6b0 2023-07-21T19:04:35.685 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.671+0000 7fb4ab09fb80 10 client.17784 fetch_fsmap finished waiting for FSMap version 103 2023-07-21T19:04:35.685 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.671+0000 7fb4ab09fb80 10 client.17784 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:04:35.685 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.671+0000 7fb4ab09fb80 20 client.17784 populate_metadata read hostname 'smithi039' 2023-07-21T19:04:35.685 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.671+0000 7fb4ab09fb80 1 --2- 172.21.15.39:0/1328914795 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x56106df4adf0 0x56106df4d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:35.685 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.671+0000 7fb4ab09fb80 4 client.17784 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:04:35.685 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.671+0000 7fb4ab09fb80 1 -- 172.21.15.39:0/1328914795 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56106de7be50 con 0x56106df4adf0 2023-07-21T19:04:35.685 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.671+0000 7fb497fff700 1 --2- 172.21.15.39:0/1328914795 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x56106df4adf0 0x56106df4d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:35.686 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.671+0000 7fb47bfff700 10 client.17784 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T19:04:35.686 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.671+0000 7fb497fff700 1 --2- 172.21.15.39:0/1328914795 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x56106df4adf0 0x56106df4d200 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26d87a860146a7e1 server_cookie=f69a6aa9cd3ba23b in_seq=0 out_seq=0 2023-07-21T19:04:35.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.678+0000 7fb47bfff700 1 -- 172.21.15.39:0/1328914795 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56106de7be50 con 0x56106df4adf0 2023-07-21T19:04:35.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.678+0000 7fb47bfff700 10 client.17784 handle_command_reply: tid=0 2023-07-21T19:04:35.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.678+0000 7fb4ab09fb80 1 client.17784 shutdown 2023-07-21T19:04:35.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.679+0000 7fb4ab09fb80 1 -- 172.21.15.39:0/1328914795 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106dd8d6b0 msgr2=0x56106df3a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:35.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.679+0000 7fb4ab09fb80 1 --2- 172.21.15.39:0/1328914795 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106dd8d6b0 0x56106df3a940 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fb48c009f00 tx=0x7fb48c00b3c0 comp rx=0 tx=0).stop 2023-07-21T19:04:35.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.679+0000 7fb4ab09fb80 1 -- 172.21.15.39:0/1328914795 shutdown_connections 2023-07-21T19:04:35.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.679+0000 7fb4ab09fb80 1 --2- 172.21.15.39:0/1328914795 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x56106df4adf0 0x56106df4d200 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:35.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.679+0000 7fb4ab09fb80 1 --2- 172.21.15.39:0/1328914795 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56106df372d0 0x56106df3b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:35.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.679+0000 7fb4ab09fb80 1 --2- 172.21.15.39:0/1328914795 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106dd8d6b0 0x56106df3a940 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:35.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.679+0000 7fb4ab09fb80 1 --2- 172.21.15.39:0/1328914795 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56106df31580 0x56106df3ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:35.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.679+0000 7fb4ab09fb80 1 -- 172.21.15.39:0/1328914795 >> 172.21.15.39:0/1328914795 conn(0x56106df26be0 msgr2=0x56106df2b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:04:35.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.679+0000 7fb47bfff700 0 client.17784 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T19:04:35.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.679+0000 7fb4ab09fb80 1 -- 172.21.15.39:0/1328914795 shutdown_connections 2023-07-21T19:04:35.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.679+0000 7fb4ab09fb80 1 -- 172.21.15.39:0/1328914795 wait complete. 2023-07-21T19:04:35.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.679+0000 7fb4ab09fb80 20 client.17784 trim_cache size 0 max 16384 2023-07-21T19:04:35.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.705+0000 7fb4ab09fb80 1 Processor -- start 2023-07-21T19:04:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.705+0000 7fb4ab09fb80 1 -- start start 2023-07-21T19:04:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.705+0000 7fb4ab09fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56106dd8d6b0 0x56106df31040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.705+0000 7fb4ab09fb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106df31580 0x56106df41c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.705+0000 7fb4ab09fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56106df372d0 0x56106df361a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.705+0000 7fb4ab09fb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x56106df28d40 con 0x56106df31580 2023-07-21T19:04:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.705+0000 7fb4ab09fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56106df37110 con 0x56106df372d0 2023-07-21T19:04:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.705+0000 7fb4ab09fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x56106df369d0 con 0x56106dd8d6b0 2023-07-21T19:04:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.706+0000 7fb49d4b6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56106df372d0 0x56106df361a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.706+0000 7fb497fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106df31580 0x56106df41c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.706+0000 7fb49d4b6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56106df372d0 0x56106df361a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:47350/0 (socket says 172.21.15.39:47350) 2023-07-21T19:04:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.706+0000 7fb497fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106df31580 0x56106df41c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34752/0 (socket says 172.21.15.39:34752) 2023-07-21T19:04:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.706+0000 7fb49d4b6700 1 -- 172.21.15.39:0/1243190235 learned_addr learned my addr 172.21.15.39:0/1243190235 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:04:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.706+0000 7fb497fff700 1 -- 172.21.15.39:0/1243190235 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56106dd8d6b0 msgr2=0x56106df31040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T19:04:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.706+0000 7fb497fff700 1 --2- 172.21.15.39:0/1243190235 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56106dd8d6b0 0x56106df31040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.706+0000 7fb497fff700 1 -- 172.21.15.39:0/1243190235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56106df372d0 msgr2=0x56106df361a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:04:35.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.706+0000 7fb497fff700 1 --2- 172.21.15.39:0/1243190235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56106df372d0 0x56106df361a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:35.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.706+0000 7fb497fff700 1 -- 172.21.15.39:0/1243190235 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56106dcd71f0 con 0x56106df31580 2023-07-21T19:04:35.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.708+0000 7fb497fff700 1 --2- 172.21.15.39:0/1243190235 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106df31580 0x56106df41c70 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fb4840ca7d0 tx=0x7fb4840965c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f63a4590fe751d9e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:04:35.720 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.710+0000 7fb4837fe700 1 -- 172.21.15.39:0/1243190235 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4840984f0 con 0x56106df31580 2023-07-21T19:04:35.720 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.710+0000 7fb4837fe700 1 -- 172.21.15.39:0/1243190235 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4840caa50 con 0x56106df31580 2023-07-21T19:04:35.720 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.710+0000 7fb4837fe700 1 -- 172.21.15.39:0/1243190235 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4840998a0 con 0x56106df31580 2023-07-21T19:04:35.721 INFO:tasks.workunit.client.0.smithi039.stdout: 10 312548 28.26 MB/sec execute 468 sec latency 160.784 ms 2023-07-21T19:04:35.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.710+0000 7fb4ab09fb80 1 -- 172.21.15.39:0/1243190235 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106df31580 msgr2=0x56106df41c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:35.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.710+0000 7fb4ab09fb80 1 --2- 172.21.15.39:0/1243190235 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106df31580 0x56106df41c70 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fb4840ca7d0 tx=0x7fb4840965c0 comp rx=0 tx=0).stop 2023-07-21T19:04:35.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.711+0000 7fb4ab09fb80 1 -- 172.21.15.39:0/1243190235 shutdown_connections 2023-07-21T19:04:35.722 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.711+0000 7fb4ab09fb80 1 --2- 172.21.15.39:0/1243190235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56106df372d0 0x56106df361a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:35.722 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.711+0000 7fb4ab09fb80 1 --2- 172.21.15.39:0/1243190235 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56106dd8d6b0 0x56106df31040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:35.722 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.711+0000 7fb4ab09fb80 1 --2- 172.21.15.39:0/1243190235 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106df31580 0x56106df41c70 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:35.722 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.711+0000 7fb4ab09fb80 1 -- 172.21.15.39:0/1243190235 >> 172.21.15.39:0/1243190235 conn(0x56106df26be0 msgr2=0x56106df35a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:04:35.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.712+0000 7fb4ab09fb80 1 -- 172.21.15.39:0/1243190235 shutdown_connections 2023-07-21T19:04:35.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.712+0000 7fb4ab09fb80 1 -- 172.21.15.39:0/1243190235 wait complete. 2023-07-21T19:04:35.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.712+0000 7fb4ab09fb80 1 -- start start 2023-07-21T19:04:35.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.712+0000 7fb4ab09fb80 1 Processor -- start 2023-07-21T19:04:35.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.713+0000 7fb4ab09fb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106dd8d6b0 0x56106df97020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:35.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.713+0000 7fb4ab09fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56106df31580 0x56106df97560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:35.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.713+0000 7fb4ab09fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56106df372d0 0x56106df97d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:35.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.713+0000 7fb4ab09fb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x56106df983a0 con 0x56106dd8d6b0 2023-07-21T19:04:35.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.713+0000 7fb4ab09fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56106df98510 con 0x56106df372d0 2023-07-21T19:04:35.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.713+0000 7fb4ab09fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x56106df986d0 con 0x56106df31580 2023-07-21T19:04:35.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.713+0000 7fb4ab09fb80 10 client.0 _ll_register_callbacks cb 0x7fb48c009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:04:35.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.713+0000 7fb49ccb5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106dd8d6b0 0x56106df97020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:35.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.713+0000 7fb49ccb5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106dd8d6b0 0x56106df97020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34756/0 (socket says 172.21.15.39:34756) 2023-07-21T19:04:35.726 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.713+0000 7fb49ccb5700 1 -- 172.21.15.39:0/1193321082 learned_addr learned my addr 172.21.15.39:0/1193321082 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:04:35.726 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.713+0000 7fb497fff700 1 --2- 172.21.15.39:0/1193321082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56106df31580 0x56106df97560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:35.726 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.713+0000 7fb49d4b6700 1 --2- 172.21.15.39:0/1193321082 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56106df372d0 0x56106df97d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:35.726 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.713+0000 7fb49ccb5700 1 -- 172.21.15.39:0/1193321082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56106df31580 msgr2=0x56106df97560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:04:35.726 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.713+0000 7fb49ccb5700 1 --2- 172.21.15.39:0/1193321082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56106df31580 0x56106df97560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:35.726 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.713+0000 7fb49ccb5700 1 -- 172.21.15.39:0/1193321082 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56106df372d0 msgr2=0x56106df97d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:04:35.726 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.713+0000 7fb49ccb5700 1 --2- 172.21.15.39:0/1193321082 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56106df372d0 0x56106df97d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:35.727 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.713+0000 7fb49ccb5700 1 -- 172.21.15.39:0/1193321082 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4981b1a40 con 0x56106dd8d6b0 2023-07-21T19:04:35.727 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.713+0000 7fb497fff700 1 --2- 172.21.15.39:0/1193321082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56106df31580 0x56106df97560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:04:35.727 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.713+0000 7fb49ccb5700 1 --2- 172.21.15.39:0/1193321082 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106dd8d6b0 0x56106df97020 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fb48c009830 tx=0x7fb48c006d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c044d138cb10997 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:04:35.727 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.713+0000 7fb47bfff700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T19:04:35.727 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.714+0000 7fb47bfff700 1 -- 172.21.15.39:0/1193321082 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48c010970 con 0x56106dd8d6b0 2023-07-21T19:04:35.727 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.714+0000 7fb4ab09fb80 1 -- 172.21.15.39:0/1193321082 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb490013af0 con 0x56106dd8d6b0 2023-07-21T19:04:35.727 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.714+0000 7fb47bfff700 1 -- 172.21.15.39:0/1193321082 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fb48c010ad0 con 0x56106dd8d6b0 2023-07-21T19:04:35.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.714+0000 7fb47bfff700 1 -- 172.21.15.39:0/1193321082 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb48c010c90 con 0x56106dd8d6b0 2023-07-21T19:04:35.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.714+0000 7fb4ab09fb80 10 client.17790 fetch_fsmap learned FSMap version 103 2023-07-21T19:04:35.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.714+0000 7fb4ab09fb80 1 -- 172.21.15.39:0/1193321082 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=4+}) v3 -- 0x56106dcaf250 con 0x56106dd8d6b0 2023-07-21T19:04:35.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.714+0000 7fb47bfff700 1 -- 172.21.15.39:0/1193321082 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb490013af0 con 0x56106dd8d6b0 2023-07-21T19:04:35.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.715+0000 7fb47bfff700 1 -- 172.21.15.39:0/1193321082 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 103) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fb48c011420 con 0x56106dd8d6b0 2023-07-21T19:04:35.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.715+0000 7fb4ab09fb80 10 client.17790 fetch_fsmap finished waiting for FSMap version 103 2023-07-21T19:04:35.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.715+0000 7fb4ab09fb80 10 client.17790 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:04:35.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.715+0000 7fb4ab09fb80 20 client.17790 populate_metadata read hostname 'smithi039' 2023-07-21T19:04:35.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.715+0000 7fb4ab09fb80 1 --2- 172.21.15.39:0/1193321082 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x56106df9ff50 0x56106dfa2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:35.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.715+0000 7fb4ab09fb80 4 client.17790 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:04:35.730 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.715+0000 7fb4ab09fb80 1 -- 172.21.15.39:0/1193321082 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56106de3dff0 con 0x56106df9ff50 2023-07-21T19:04:35.730 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.715+0000 7fb497fff700 1 --2- 172.21.15.39:0/1193321082 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x56106df9ff50 0x56106dfa2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:35.730 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.716+0000 7fb47bfff700 10 client.17790 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T19:04:35.730 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.716+0000 7fb497fff700 1 --2- 172.21.15.39:0/1193321082 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x56106df9ff50 0x56106dfa2360 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=805304319326d9fb server_cookie=d65c8687cf99e7af in_seq=0 out_seq=0 2023-07-21T19:04:35.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.911+0000 7fb47bfff700 1 -- 172.21.15.39:0/1193321082 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56106de3dff0 con 0x56106df9ff50 2023-07-21T19:04:35.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.911+0000 7fb47bfff700 10 client.17790 handle_command_reply: tid=0 2023-07-21T19:04:35.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.912+0000 7fb4ab09fb80 1 client.17790 shutdown 2023-07-21T19:04:35.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.912+0000 7fb4ab09fb80 1 -- 172.21.15.39:0/1193321082 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106dd8d6b0 msgr2=0x56106df97020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:35.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.912+0000 7fb4ab09fb80 1 --2- 172.21.15.39:0/1193321082 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106dd8d6b0 0x56106df97020 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fb48c009830 tx=0x7fb48c006d40 comp rx=0 tx=0).stop 2023-07-21T19:04:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.916+0000 7fb4ab09fb80 1 -- 172.21.15.39:0/1193321082 shutdown_connections 2023-07-21T19:04:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.916+0000 7fb4ab09fb80 1 --2- 172.21.15.39:0/1193321082 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x56106df9ff50 0x56106dfa2360 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.916+0000 7fb4ab09fb80 1 --2- 172.21.15.39:0/1193321082 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56106df372d0 0x56106df97d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:35.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.916+0000 7fb4ab09fb80 1 --2- 172.21.15.39:0/1193321082 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106dd8d6b0 0x56106df97020 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:35.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.916+0000 7fb4ab09fb80 1 --2- 172.21.15.39:0/1193321082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56106df31580 0x56106df97560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:35.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.916+0000 7fb4ab09fb80 1 -- 172.21.15.39:0/1193321082 >> 172.21.15.39:0/1193321082 conn(0x56106df26be0 msgr2=0x56106df378a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:04:35.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.916+0000 7fb47bfff700 0 client.17790 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T19:04:35.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.917+0000 7fb4ab09fb80 1 -- 172.21.15.39:0/1193321082 shutdown_connections 2023-07-21T19:04:35.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.917+0000 7fb4ab09fb80 1 -- 172.21.15.39:0/1193321082 wait complete. 2023-07-21T19:04:35.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.917+0000 7fb4ab09fb80 20 client.17790 trim_cache size 0 max 16384 2023-07-21T19:04:35.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T19:04:35.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T19:04:35.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T19:04:35.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T19:04:35.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.919+0000 7fb49ef19700 1 -- 172.21.15.39:0/2913747643 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fb4880a1340 msgr2=0x7fb4880a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:35.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.919+0000 7fb49ef19700 1 --2- 172.21.15.39:0/2913747643 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fb4880a1340 0x7fb4880a37f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb49001caf0 tx=0x7fb49001c9e0 comp rx=0 tx=0).stop 2023-07-21T19:04:35.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.919+0000 7fb49ef19700 1 -- 172.21.15.39:0/2913747643 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb498101520 msgr2=0x7fb4981aaf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:35.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.919+0000 7fb49ef19700 1 --2- 172.21.15.39:0/2913747643 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb498101520 0x7fb4981aaf90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb48400d950 tx=0x7fb48400dd10 comp rx=0 tx=0).stop 2023-07-21T19:04:35.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.919+0000 7fb49ef19700 1 -- 172.21.15.39:0/2913747643 shutdown_connections 2023-07-21T19:04:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.919+0000 7fb49ef19700 1 --2- 172.21.15.39:0/2913747643 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fb4880a1340 0x7fb4880a37f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.919+0000 7fb49ef19700 1 --2- 172.21.15.39:0/2913747643 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb498100c10 0x7fb4981aaa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.919+0000 7fb49ef19700 1 --2- 172.21.15.39:0/2913747643 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb49810a130 0x7fb4981a48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.919+0000 7fb49ef19700 1 --2- 172.21.15.39:0/2913747643 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb498101520 0x7fb4981aaf90 secure :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb48400d950 tx=0x7fb48400dd10 comp rx=0 tx=0).stop 2023-07-21T19:04:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.919+0000 7fb49ef19700 1 -- 172.21.15.39:0/2913747643 >> 172.21.15.39:0/2913747643 conn(0x7fb4980769b0 msgr2=0x7fb498101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:04:35.935 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.925+0000 7fb49ef19700 1 -- 172.21.15.39:0/2913747643 shutdown_connections 2023-07-21T19:04:35.935 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:35.925+0000 7fb49ef19700 1 -- 172.21.15.39:0/2913747643 wait complete. 2023-07-21T19:04:36.721 INFO:tasks.workunit.client.0.smithi039.stdout: 10 313005 28.26 MB/sec execute 469 sec latency 165.212 ms 2023-07-21T19:04:36.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:04:36 smithi193 ceph-mon[98204]: pgmap v726: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.8 MiB/s wr, 418 op/s 2023-07-21T19:04:36.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:04:36 smithi155 ceph-mon[97934]: pgmap v726: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.8 MiB/s wr, 418 op/s 2023-07-21T19:04:36.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:04:36 smithi039 ceph-mon[96489]: pgmap v726: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.8 MiB/s wr, 418 op/s 2023-07-21T19:04:37.720 INFO:tasks.workunit.client.0.smithi039.stdout: 10 313585 28.26 MB/sec execute 470 sec latency 91.526 ms 2023-07-21T19:04:38.720 INFO:tasks.workunit.client.0.smithi039.stdout: 10 314164 28.25 MB/sec execute 471 sec latency 115.157 ms 2023-07-21T19:04:38.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:04:38 smithi193 ceph-mon[98204]: pgmap v727: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 21 MiB/s wr, 887 op/s 2023-07-21T19:04:38.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:04:38 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:04:38.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:04:38 smithi155 ceph-mon[97934]: pgmap v727: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 21 MiB/s wr, 887 op/s 2023-07-21T19:04:38.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:04:38 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:04:38.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:04:38 smithi039 ceph-mon[96489]: pgmap v727: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 21 MiB/s wr, 887 op/s 2023-07-21T19:04:38.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:04:38 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:04:39.720 INFO:tasks.workunit.client.0.smithi039.stdout: 10 314749 28.28 MB/sec execute 472 sec latency 129.495 ms 2023-07-21T19:04:39.860 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:04:39 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:04:39.861 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:04:39 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:04:39.861 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:04:39 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T19:04:39.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:04:39 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:04:39.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:04:39 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:04:39.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:04:39 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T19:04:39.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:04:39 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:04:39.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:04:39 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:04:39.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:04:39 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T19:04:40.721 INFO:tasks.workunit.client.0.smithi039.stdout: 10 315197 28.27 MB/sec execute 473 sec latency 82.188 ms 2023-07-21T19:04:40.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:04:40 smithi193 ceph-mon[98204]: pgmap v728: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 17 MiB/s wr, 660 op/s 2023-07-21T19:04:40.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:04:40 smithi155 ceph-mon[97934]: pgmap v728: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 17 MiB/s wr, 660 op/s 2023-07-21T19:04:40.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:04:40 smithi039 ceph-mon[96489]: pgmap v728: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 17 MiB/s wr, 660 op/s 2023-07-21T19:04:41.721 INFO:tasks.workunit.client.0.smithi039.stdout: 10 315773 28.27 MB/sec execute 474 sec latency 107.022 ms 2023-07-21T19:04:42.721 INFO:tasks.workunit.client.0.smithi039.stdout: 10 316361 28.29 MB/sec execute 475 sec latency 70.876 ms 2023-07-21T19:04:42.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:04:42 smithi193 ceph-mon[98204]: pgmap v729: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 17 MiB/s wr, 681 op/s 2023-07-21T19:04:42.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:04:42 smithi155 ceph-mon[97934]: pgmap v729: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 17 MiB/s wr, 681 op/s 2023-07-21T19:04:42.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:04:42 smithi039 ceph-mon[96489]: pgmap v729: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 17 MiB/s wr, 681 op/s 2023-07-21T19:04:43.721 INFO:tasks.workunit.client.0.smithi039.stdout: 10 316778 28.25 MB/sec execute 476 sec latency 145.479 ms 2023-07-21T19:04:44.721 INFO:tasks.workunit.client.0.smithi039.stdout: 10 317263 28.26 MB/sec execute 477 sec latency 143.643 ms 2023-07-21T19:04:44.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:04:44 smithi193 ceph-mon[98204]: pgmap v730: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 20 MiB/s wr, 836 op/s 2023-07-21T19:04:44.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:04:44 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T19:04:44.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:04:44 smithi155 ceph-mon[97934]: pgmap v730: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 20 MiB/s wr, 836 op/s 2023-07-21T19:04:44.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:04:44 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T19:04:44.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:04:44 smithi039 ceph-mon[96489]: pgmap v730: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 20 MiB/s wr, 836 op/s 2023-07-21T19:04:44.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:04:44 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T19:04:45.721 INFO:tasks.workunit.client.0.smithi039.stdout: 10 317836 28.26 MB/sec execute 478 sec latency 225.205 ms 2023-07-21T19:04:46.721 INFO:tasks.workunit.client.0.smithi039.stdout: 10 318400 28.27 MB/sec execute 479 sec latency 156.493 ms 2023-07-21T19:04:46.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:04:46 smithi193 ceph-mon[98204]: pgmap v731: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 17 MiB/s wr, 666 op/s 2023-07-21T19:04:46.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:04:46 smithi155 ceph-mon[97934]: pgmap v731: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 17 MiB/s wr, 666 op/s 2023-07-21T19:04:46.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:04:46 smithi039 ceph-mon[96489]: pgmap v731: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 17 MiB/s wr, 666 op/s 2023-07-21T19:04:47.721 INFO:tasks.workunit.client.0.smithi039.stdout: 10 318984 28.27 MB/sec execute 480 sec latency 97.024 ms 2023-07-21T19:04:48.721 INFO:tasks.workunit.client.0.smithi039.stdout: 10 319524 28.28 MB/sec execute 481 sec latency 107.599 ms 2023-07-21T19:04:48.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:04:48 smithi193 ceph-mon[98204]: pgmap v732: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 21 MiB/s wr, 852 op/s 2023-07-21T19:04:48.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:04:48 smithi155 ceph-mon[97934]: pgmap v732: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 21 MiB/s wr, 852 op/s 2023-07-21T19:04:48.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:04:48 smithi039 ceph-mon[96489]: pgmap v732: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 21 MiB/s wr, 852 op/s 2023-07-21T19:04:49.122 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.111+0000 7ff0b074c700 1 -- 172.21.15.39:0/1958800682 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0a810a550 msgr2=0x7ff0a810a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:49.122 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.111+0000 7ff0b074c700 1 --2- 172.21.15.39:0/1958800682 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0a810a550 0x7ff0a810a9c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff0980098f0 tx=0x7ff098009c00 comp rx=0 tx=0).stop 2023-07-21T19:04:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.111+0000 7ff0b074c700 1 -- 172.21.15.39:0/1958800682 shutdown_connections 2023-07-21T19:04:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.111+0000 7ff0b074c700 1 --2- 172.21.15.39:0/1958800682 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff0a810af00 0x7ff0a8117330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.111+0000 7ff0b074c700 1 --2- 172.21.15.39:0/1958800682 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0a8110570 0x7ff0a8110940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.111+0000 7ff0b074c700 1 --2- 172.21.15.39:0/1958800682 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0a810a550 0x7ff0a810a9c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.111+0000 7ff0b074c700 1 -- 172.21.15.39:0/1958800682 >> 172.21.15.39:0/1958800682 conn(0x7ff0a806a900 msgr2=0x7ff0a806ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:04:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.111+0000 7ff0b074c700 1 -- 172.21.15.39:0/1958800682 shutdown_connections 2023-07-21T19:04:49.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.111+0000 7ff0b074c700 1 -- 172.21.15.39:0/1958800682 wait complete. 2023-07-21T19:04:49.126 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.112+0000 7ff0b074c700 1 Processor -- start 2023-07-21T19:04:49.126 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.112+0000 7ff0b074c700 1 -- start start 2023-07-21T19:04:49.127 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.113+0000 7ff0b074c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0a810a550 0x7ff0a8117010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:49.127 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.113+0000 7ff0b074c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff0a810af00 0x7ff0a8113e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:49.127 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.113+0000 7ff0b074c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0a8110570 0x7ff0a81143a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:49.128 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.113+0000 7ff0b074c700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a8114a30 con 0x7ff0a810a550 2023-07-21T19:04:49.128 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.113+0000 7ff0b074c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a8114ba0 con 0x7ff0a8110570 2023-07-21T19:04:49.128 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.113+0000 7ff0b074c700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a81a6bb0 con 0x7ff0a810af00 2023-07-21T19:04:49.128 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.113+0000 7ff0ae4e8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0a810a550 0x7ff0a8117010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:49.129 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.113+0000 7ff0ae4e8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0a810a550 0x7ff0a8117010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34768/0 (socket says 172.21.15.39:34768) 2023-07-21T19:04:49.129 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.113+0000 7ff0ae4e8700 1 -- 172.21.15.39:0/2516088170 learned_addr learned my addr 172.21.15.39:0/2516088170 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:04:49.129 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.113+0000 7ff0aece9700 1 --2- 172.21.15.39:0/2516088170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0a8110570 0x7ff0a81143a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.113+0000 7ff0adce7700 1 --2- 172.21.15.39:0/2516088170 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff0a810af00 0x7ff0a8113e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.113+0000 7ff0ae4e8700 1 -- 172.21.15.39:0/2516088170 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff0a810af00 msgr2=0x7ff0a8113e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.113+0000 7ff0ae4e8700 1 --2- 172.21.15.39:0/2516088170 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff0a810af00 0x7ff0a8113e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.113+0000 7ff0ae4e8700 1 -- 172.21.15.39:0/2516088170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0a8110570 msgr2=0x7ff0a81143a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:49.131 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.113+0000 7ff0ae4e8700 1 --2- 172.21.15.39:0/2516088170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0a8110570 0x7ff0a81143a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:49.131 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.113+0000 7ff0ae4e8700 1 -- 172.21.15.39:0/2516088170 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff098009520 con 0x7ff0a810a550 2023-07-21T19:04:49.131 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.113+0000 7ff0ae4e8700 1 --2- 172.21.15.39:0/2516088170 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0a810a550 0x7ff0a8117010 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7ff0a400e9c0 tx=0x7ff0a400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:04:49.131 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.114+0000 7ff09f7fe700 1 -- 172.21.15.39:0/2516088170 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a4004a30 con 0x7ff0a810a550 2023-07-21T19:04:49.131 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.114+0000 7ff09f7fe700 1 -- 172.21.15.39:0/2516088170 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0a400cc80 con 0x7ff0a810a550 2023-07-21T19:04:49.131 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.114+0000 7ff09f7fe700 1 -- 172.21.15.39:0/2516088170 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a40054e0 con 0x7ff0a810a550 2023-07-21T19:04:49.132 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.114+0000 7ff0b074c700 1 -- 172.21.15.39:0/2516088170 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0a81a6e90 con 0x7ff0a810a550 2023-07-21T19:04:49.132 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.114+0000 7ff0b074c700 1 -- 172.21.15.39:0/2516088170 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0a81a7370 con 0x7ff0a810a550 2023-07-21T19:04:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.117+0000 7ff09f7fe700 1 -- 172.21.15.39:0/2516088170 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7ff0a4004b90 con 0x7ff0a810a550 2023-07-21T19:04:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.118+0000 7ff09f7fe700 1 --2- 172.21.15.39:0/2516088170 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7ff0940a1340 0x7ff0940a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.118+0000 7ff09f7fe700 1 -- 172.21.15.39:0/2516088170 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 8527+0+0 (secure 0 0 0) 0x7ff0a4014070 con 0x7ff0a810a550 2023-07-21T19:04:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.119+0000 7ff0bc8d2b80 1 Processor -- start 2023-07-21T19:04:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.121+0000 7ff0bc8d2b80 1 -- start start 2023-07-21T19:04:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.121+0000 7ff0bc8d2b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588f36f76b0 0x5588f38a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.121+0000 7ff0bc8d2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5588f38a12d0 0x5588f389b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.121+0000 7ff0bc8d2b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588f389b580 0x5588f389b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.121+0000 7ff0bc8d2b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588f37a0c50 con 0x5588f36f76b0 2023-07-21T19:04:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.121+0000 7ff0bc8d2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588f37f39e0 con 0x5588f38a12d0 2023-07-21T19:04:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.121+0000 7ff0bc8d2b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588f37ece70 con 0x5588f389b580 2023-07-21T19:04:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.121+0000 7ff0ae4e8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588f36f76b0 0x5588f38a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.121+0000 7ff0adce7700 1 --2- 172.21.15.39:0/2516088170 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7ff0940a1340 0x7ff0940a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.122+0000 7ff0ae4e8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588f36f76b0 0x5588f38a0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34776/0 (socket says 172.21.15.39:34776) 2023-07-21T19:04:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.122+0000 7ff0aece9700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588f389b580 0x5588f389b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.122+0000 7ff0ae4e8700 1 -- 172.21.15.39:0/4119472205 learned_addr learned my addr 172.21.15.39:0/4119472205 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:04:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.122+0000 7ff0ae4e8700 1 -- 172.21.15.39:0/4119472205 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588f389b580 msgr2=0x5588f389b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:04:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.122+0000 7ff0ae4e8700 1 --2- 172.21.15.39:0/4119472205 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588f389b580 0x5588f389b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.122+0000 7ff0ae4e8700 1 -- 172.21.15.39:0/4119472205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5588f38a12d0 msgr2=0x5588f389b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:04:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.122+0000 7ff0ae4e8700 1 --2- 172.21.15.39:0/4119472205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5588f38a12d0 0x5588f389b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.122+0000 7ff0ae4e8700 1 -- 172.21.15.39:0/4119472205 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588f361ad80 con 0x5588f36f76b0 2023-07-21T19:04:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.122+0000 7ff0adce7700 1 --2- 172.21.15.39:0/2516088170 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7ff0940a1340 0x7ff0940a37f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff098009fd0 tx=0x7ff098005b60 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:04:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.122+0000 7ff0ae4e8700 1 --2- 172.21.15.39:0/4119472205 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588f36f76b0 0x5588f38a0d90 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4099d30 tx=0x7ff0a40997d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9436ee1473b7b300 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:04:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.123+0000 7ff09cff9700 1 -- 172.21.15.39:0/4119472205 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a409ddd0 con 0x5588f36f76b0 2023-07-21T19:04:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.123+0000 7ff09cff9700 1 -- 172.21.15.39:0/4119472205 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0a40a21f0 con 0x5588f36f76b0 2023-07-21T19:04:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.123+0000 7ff09cff9700 1 -- 172.21.15.39:0/4119472205 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a409a260 con 0x5588f36f76b0 2023-07-21T19:04:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.124+0000 7ff0bc8d2b80 1 -- 172.21.15.39:0/4119472205 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588f36f76b0 msgr2=0x5588f38a0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.124+0000 7ff0bc8d2b80 1 --2- 172.21.15.39:0/4119472205 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588f36f76b0 0x5588f38a0d90 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4099d30 tx=0x7ff0a40997d0 comp rx=0 tx=0).stop 2023-07-21T19:04:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.124+0000 7ff0bc8d2b80 1 -- 172.21.15.39:0/4119472205 shutdown_connections 2023-07-21T19:04:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.124+0000 7ff0bc8d2b80 1 --2- 172.21.15.39:0/4119472205 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588f389b580 0x5588f389b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.124+0000 7ff0bc8d2b80 1 --2- 172.21.15.39:0/4119472205 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588f36f76b0 0x5588f38a0d90 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.124+0000 7ff0bc8d2b80 1 --2- 172.21.15.39:0/4119472205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5588f38a12d0 0x5588f389b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.124+0000 7ff0bc8d2b80 1 -- 172.21.15.39:0/4119472205 >> 172.21.15.39:0/4119472205 conn(0x5588f3890be0 msgr2=0x5588f3895f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:04:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.124+0000 7ff0bc8d2b80 1 -- 172.21.15.39:0/4119472205 shutdown_connections 2023-07-21T19:04:49.140 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.124+0000 7ff0bc8d2b80 1 -- 172.21.15.39:0/4119472205 wait complete. 2023-07-21T19:04:49.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.124+0000 7ff0bc8d2b80 1 -- start start 2023-07-21T19:04:49.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.124+0000 7ff0bc8d2b80 1 Processor -- start 2023-07-21T19:04:49.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.125+0000 7ff0bc8d2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5588f36f76b0 0x5588f38a4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:49.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.125+0000 7ff0bc8d2b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588f389b580 0x5588f38a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:49.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.125+0000 7ff0bc8d2b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588f38a12d0 0x5588f38a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:49.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.125+0000 7ff0bc8d2b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588f378ca60 con 0x5588f389b580 2023-07-21T19:04:49.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.125+0000 7ff0bc8d2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588f378c760 con 0x5588f36f76b0 2023-07-21T19:04:49.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.125+0000 7ff0bc8d2b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588f36708c0 con 0x5588f38a12d0 2023-07-21T19:04:49.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.125+0000 7ff0bc8d2b80 10 client.0 _ll_register_callbacks cb 0x5588f378a700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:04:49.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.125+0000 7ff0aece9700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588f38a12d0 0x5588f38a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:49.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.125+0000 7ff0adce7700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588f389b580 0x5588f38a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:49.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.125+0000 7ff0ae4e8700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5588f36f76b0 0x5588f38a4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:49.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.125+0000 7ff0aece9700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588f38a12d0 0x5588f38a53f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:53338/0 (socket says 172.21.15.39:53338) 2023-07-21T19:04:49.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.125+0000 7ff0aece9700 1 -- 172.21.15.39:0/4277635674 learned_addr learned my addr 172.21.15.39:0/4277635674 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:04:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.125+0000 7ff0adce7700 1 -- 172.21.15.39:0/4277635674 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588f38a12d0 msgr2=0x5588f38a53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:04:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.125+0000 7ff0adce7700 1 --2- 172.21.15.39:0/4277635674 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588f38a12d0 0x5588f38a53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.125+0000 7ff0adce7700 1 -- 172.21.15.39:0/4277635674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5588f36f76b0 msgr2=0x5588f38a4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:04:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.125+0000 7ff0adce7700 1 --2- 172.21.15.39:0/4277635674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5588f36f76b0 0x5588f38a4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.126+0000 7ff0adce7700 1 -- 172.21.15.39:0/4277635674 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0a81a7370 con 0x5588f389b580 2023-07-21T19:04:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.126+0000 7ff0aece9700 1 --2- 172.21.15.39:0/4277635674 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588f38a12d0 0x5588f38a53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:04:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.126+0000 7ff0ae4e8700 1 --2- 172.21.15.39:0/4277635674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5588f36f76b0 0x5588f38a4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:04:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.126+0000 7ff0adce7700 1 --2- 172.21.15.39:0/4277635674 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588f389b580 0x5588f38a4eb0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7ff0980098c0 tx=0x7ff098009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36ec4315c6a061ff server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:04:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.126+0000 7ff091ffb700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T19:04:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.127+0000 7ff091ffb700 1 -- 172.21.15.39:0/4277635674 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09800bab0 con 0x5588f389b580 2023-07-21T19:04:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.127+0000 7ff091ffb700 1 -- 172.21.15.39:0/4277635674 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 8527+0+0 (secure 0 0 0) 0x7ff098003680 con 0x5588f389b580 2023-07-21T19:04:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.127+0000 7ff0bc8d2b80 1 -- 172.21.15.39:0/4277635674 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5588f3746490 con 0x5588f389b580 2023-07-21T19:04:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.127+0000 7ff091ffb700 1 -- 172.21.15.39:0/4277635674 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=104) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff09800bc10 con 0x5588f389b580 2023-07-21T19:04:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.129+0000 7ff0bc8d2b80 10 client.17802 fetch_fsmap learned FSMap version 104 2023-07-21T19:04:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.129+0000 7ff0bc8d2b80 1 -- 172.21.15.39:0/4277635674 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=104,monmap=4+}) v3 -- 0x5588f3603c00 con 0x5588f389b580 2023-07-21T19:04:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.131+0000 7ff091ffb700 1 -- 172.21.15.39:0/4277635674 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5588f3746490 con 0x5588f389b580 2023-07-21T19:04:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.132+0000 7ff091ffb700 1 -- 172.21.15.39:0/4277635674 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 104) v1 ==== 7466+0+0 (secure 0 0 0) 0x7ff098039880 con 0x5588f389b580 2023-07-21T19:04:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.132+0000 7ff0bc8d2b80 10 client.17802 fetch_fsmap finished waiting for FSMap version 104 2023-07-21T19:04:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.132+0000 7ff0bc8d2b80 10 client.17802 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:04:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.132+0000 7ff0bc8d2b80 20 client.17802 populate_metadata read hostname 'smithi039' 2023-07-21T19:04:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.132+0000 7ff0bc8d2b80 1 --2- 172.21.15.39:0/4277635674 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5588f38b4b50 0x5588f38b6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.132+0000 7ff0bc8d2b80 4 client.17802 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:04:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.132+0000 7ff0bc8d2b80 1 -- 172.21.15.39:0/4277635674 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5588f37e5e50 con 0x5588f38b4b50 2023-07-21T19:04:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.133+0000 7ff0ae4e8700 1 --2- 172.21.15.39:0/4277635674 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5588f38b4b50 0x5588f38b6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.135+0000 7ff091ffb700 10 client.17802 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T19:04:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.135+0000 7ff0ae4e8700 1 --2- 172.21.15.39:0/4277635674 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5588f38b4b50 0x5588f38b6f60 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26ae66bc6cfcd2f7 server_cookie=cf98c829b09a7ded in_seq=0 out_seq=0 2023-07-21T19:04:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.140+0000 7ff091ffb700 1 -- 172.21.15.39:0/4277635674 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5588f37e5e50 con 0x5588f38b4b50 2023-07-21T19:04:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.140+0000 7ff091ffb700 10 client.17802 handle_command_reply: tid=0 2023-07-21T19:04:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.140+0000 7ff0bc8d2b80 1 client.17802 shutdown 2023-07-21T19:04:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.141+0000 7ff0bc8d2b80 1 -- 172.21.15.39:0/4277635674 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588f389b580 msgr2=0x5588f38a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.141+0000 7ff0bc8d2b80 1 --2- 172.21.15.39:0/4277635674 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588f389b580 0x5588f38a4eb0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7ff0980098c0 tx=0x7ff098009c00 comp rx=0 tx=0).stop 2023-07-21T19:04:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.141+0000 7ff0bc8d2b80 1 -- 172.21.15.39:0/4277635674 shutdown_connections 2023-07-21T19:04:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.141+0000 7ff0bc8d2b80 1 --2- 172.21.15.39:0/4277635674 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5588f38b4b50 0x5588f38b6f60 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.141+0000 7ff0bc8d2b80 1 --2- 172.21.15.39:0/4277635674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5588f36f76b0 0x5588f38a4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.141+0000 7ff091ffb700 0 client.17802 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T19:04:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.141+0000 7ff0bc8d2b80 1 --2- 172.21.15.39:0/4277635674 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588f38a12d0 0x5588f38a53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.141+0000 7ff0bc8d2b80 1 --2- 172.21.15.39:0/4277635674 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588f389b580 0x5588f38a4eb0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.141+0000 7ff0bc8d2b80 1 -- 172.21.15.39:0/4277635674 >> 172.21.15.39:0/4277635674 conn(0x5588f3890be0 msgr2=0x5588f38954e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:04:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.141+0000 7ff0bc8d2b80 1 -- 172.21.15.39:0/4277635674 shutdown_connections 2023-07-21T19:04:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.141+0000 7ff0bc8d2b80 1 -- 172.21.15.39:0/4277635674 wait complete. 2023-07-21T19:04:49.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.141+0000 7ff0bc8d2b80 20 client.17802 trim_cache size 0 max 16384 2023-07-21T19:04:49.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.167+0000 7ff0bc8d2b80 1 Processor -- start 2023-07-21T19:04:49.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.167+0000 7ff0bc8d2b80 1 -- start start 2023-07-21T19:04:49.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.167+0000 7ff0bc8d2b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588f3890be0 0x5588f389b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:49.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.167+0000 7ff0bc8d2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5588f36f76b0 0x5588f38abc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:49.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.167+0000 7ff0bc8d2b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588f389b580 0x5588f38a0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:49.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.167+0000 7ff0bc8d2b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588f38925f0 con 0x5588f3890be0 2023-07-21T19:04:49.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.167+0000 7ff0bc8d2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588f389f6e0 con 0x5588f36f76b0 2023-07-21T19:04:49.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.167+0000 7ff0bc8d2b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588f38a2910 con 0x5588f389b580 2023-07-21T19:04:49.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.168+0000 7ff0ae4e8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588f3890be0 0x5588f389b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:49.180 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.168+0000 7ff0ae4e8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588f3890be0 0x5588f389b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34790/0 (socket says 172.21.15.39:34790) 2023-07-21T19:04:49.180 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.168+0000 7ff0ae4e8700 1 -- 172.21.15.39:0/1206141819 learned_addr learned my addr 172.21.15.39:0/1206141819 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:04:49.180 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.168+0000 7ff0adce7700 1 --2- 172.21.15.39:0/1206141819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5588f36f76b0 0x5588f38abc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:49.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.168+0000 7ff0ae4e8700 1 -- 172.21.15.39:0/1206141819 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588f389b580 msgr2=0x5588f38a0200 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T19:04:49.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.168+0000 7ff0ae4e8700 1 --2- 172.21.15.39:0/1206141819 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588f389b580 0x5588f38a0200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:49.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.168+0000 7ff0ae4e8700 1 -- 172.21.15.39:0/1206141819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5588f36f76b0 msgr2=0x5588f38abc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:04:49.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.168+0000 7ff0ae4e8700 1 --2- 172.21.15.39:0/1206141819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5588f36f76b0 0x5588f38abc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:49.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.168+0000 7ff0ae4e8700 1 -- 172.21.15.39:0/1206141819 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588f389bc20 con 0x5588f3890be0 2023-07-21T19:04:49.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.168+0000 7ff0adce7700 1 --2- 172.21.15.39:0/1206141819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5588f36f76b0 0x5588f38abc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:04:49.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.168+0000 7ff0ae4e8700 1 --2- 172.21.15.39:0/1206141819 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588f3890be0 0x5588f389b040 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4099ce0 tx=0x7ff0a40cc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6a6dbca8eb6e02b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:04:49.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.169+0000 7ff09d7fa700 1 -- 172.21.15.39:0/1206141819 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a4098240 con 0x5588f3890be0 2023-07-21T19:04:49.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.169+0000 7ff09d7fa700 1 -- 172.21.15.39:0/1206141819 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0a4005640 con 0x5588f3890be0 2023-07-21T19:04:49.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.169+0000 7ff09d7fa700 1 -- 172.21.15.39:0/1206141819 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a409a260 con 0x5588f3890be0 2023-07-21T19:04:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.170+0000 7ff0bc8d2b80 1 -- 172.21.15.39:0/1206141819 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588f3890be0 msgr2=0x5588f389b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.170+0000 7ff0bc8d2b80 1 --2- 172.21.15.39:0/1206141819 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588f3890be0 0x5588f389b040 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4099ce0 tx=0x7ff0a40cc040 comp rx=0 tx=0).stop 2023-07-21T19:04:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.170+0000 7ff0bc8d2b80 1 -- 172.21.15.39:0/1206141819 shutdown_connections 2023-07-21T19:04:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.170+0000 7ff0bc8d2b80 1 --2- 172.21.15.39:0/1206141819 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588f389b580 0x5588f38a0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:49.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.170+0000 7ff0bc8d2b80 1 --2- 172.21.15.39:0/1206141819 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588f3890be0 0x5588f389b040 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:49.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.170+0000 7ff0bc8d2b80 1 --2- 172.21.15.39:0/1206141819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5588f36f76b0 0x5588f38abc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:49.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.170+0000 7ff0bc8d2b80 1 -- 172.21.15.39:0/1206141819 >> 172.21.15.39:0/1206141819 conn(0x5588f38b1290 msgr2=0x5588f389faf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:04:49.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.170+0000 7ff0bc8d2b80 1 -- 172.21.15.39:0/1206141819 shutdown_connections 2023-07-21T19:04:49.188 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.170+0000 7ff0bc8d2b80 1 -- 172.21.15.39:0/1206141819 wait complete. 2023-07-21T19:04:49.188 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.170+0000 7ff0bc8d2b80 1 -- start start 2023-07-21T19:04:49.188 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.170+0000 7ff0bc8d2b80 1 Processor -- start 2023-07-21T19:04:49.188 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.171+0000 7ff0bc8d2b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588f36f76b0 0x5588f3901c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:49.188 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.171+0000 7ff0bc8d2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5588f3890be0 0x5588f3902190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.171+0000 7ff0bc8d2b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588f389b580 0x5588f39009e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.171+0000 7ff0bc8d2b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588f3900f90 con 0x5588f36f76b0 2023-07-21T19:04:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.171+0000 7ff0bc8d2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588f3901100 con 0x5588f3890be0 2023-07-21T19:04:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.171+0000 7ff0bc8d2b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588f39012c0 con 0x5588f389b580 2023-07-21T19:04:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.171+0000 7ff0bc8d2b80 10 client.0 _ll_register_callbacks cb 0x7ff0980053d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:04:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.171+0000 7ff0adce7700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5588f3890be0 0x5588f3902190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.171+0000 7ff0adce7700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5588f3890be0 0x5588f3902190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:47392/0 (socket says 172.21.15.39:47392) 2023-07-21T19:04:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.171+0000 7ff0adce7700 1 -- 172.21.15.39:0/4252816146 learned_addr learned my addr 172.21.15.39:0/4252816146 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:04:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.171+0000 7ff0ae4e8700 1 --2- 172.21.15.39:0/4252816146 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588f36f76b0 0x5588f3901c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.171+0000 7ff0aece9700 1 --2- 172.21.15.39:0/4252816146 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588f389b580 0x5588f39009e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.172+0000 7ff0ae4e8700 1 -- 172.21.15.39:0/4252816146 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588f389b580 msgr2=0x5588f39009e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:04:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.172+0000 7ff0ae4e8700 1 --2- 172.21.15.39:0/4252816146 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588f389b580 0x5588f39009e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.172+0000 7ff0ae4e8700 1 -- 172.21.15.39:0/4252816146 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5588f3890be0 msgr2=0x5588f3902190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:04:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.172+0000 7ff0ae4e8700 1 --2- 172.21.15.39:0/4252816146 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5588f3890be0 0x5588f3902190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.172+0000 7ff0ae4e8700 1 -- 172.21.15.39:0/4252816146 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5588f3603c00 con 0x5588f36f76b0 2023-07-21T19:04:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.172+0000 7ff0ae4e8700 1 --2- 172.21.15.39:0/4252816146 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588f36f76b0 0x5588f3901c50 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4096810 tx=0x7ff0a4099f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaddff0cfdb323cf server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:04:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.172+0000 7ff091ffb700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T19:04:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.172+0000 7ff091ffb700 1 -- 172.21.15.39:0/4252816146 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a409a000 con 0x5588f36f76b0 2023-07-21T19:04:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.172+0000 7ff0bc8d2b80 1 -- 172.21.15.39:0/4252816146 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff0a40a21f0 con 0x5588f36f76b0 2023-07-21T19:04:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.172+0000 7ff091ffb700 1 -- 172.21.15.39:0/4252816146 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 8527+0+0 (secure 0 0 0) 0x7ff0a40ca030 con 0x5588f36f76b0 2023-07-21T19:04:49.192 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.174+0000 7ff091ffb700 1 -- 172.21.15.39:0/4252816146 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=104) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff0a409a160 con 0x5588f36f76b0 2023-07-21T19:04:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.174+0000 7ff0bc8d2b80 10 client.17808 fetch_fsmap learned FSMap version 104 2023-07-21T19:04:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.174+0000 7ff0bc8d2b80 1 -- 172.21.15.39:0/4252816146 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=104,monmap=4+}) v3 -- 0x7ff0a40b6040 con 0x5588f36f76b0 2023-07-21T19:04:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.177+0000 7ff091ffb700 1 -- 172.21.15.39:0/4252816146 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0a40bf070 con 0x5588f36f76b0 2023-07-21T19:04:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.177+0000 7ff091ffb700 1 -- 172.21.15.39:0/4252816146 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 104) v1 ==== 7466+0+0 (secure 0 0 0) 0x7ff0a4016c00 con 0x5588f36f76b0 2023-07-21T19:04:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.177+0000 7ff0bc8d2b80 10 client.17808 fetch_fsmap finished waiting for FSMap version 104 2023-07-21T19:04:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.177+0000 7ff0bc8d2b80 10 client.17808 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:04:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.177+0000 7ff0bc8d2b80 20 client.17808 populate_metadata read hostname 'smithi039' 2023-07-21T19:04:49.194 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.177+0000 7ff0bc8d2b80 1 --2- 172.21.15.39:0/4252816146 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5588f38a12d0 0x5588f390b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:49.194 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.177+0000 7ff0bc8d2b80 4 client.17808 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:04:49.194 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.177+0000 7ff0bc8d2b80 1 -- 172.21.15.39:0/4252816146 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5588f37a7ff0 con 0x5588f38a12d0 2023-07-21T19:04:49.194 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.178+0000 7ff0adce7700 1 --2- 172.21.15.39:0/4252816146 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5588f38a12d0 0x5588f390b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:49.194 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.180+0000 7ff0adce7700 1 --2- 172.21.15.39:0/4252816146 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5588f38a12d0 0x5588f390b700 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c469f94901527cb server_cookie=851ae84dbea922fa in_seq=0 out_seq=0 2023-07-21T19:04:49.195 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.185+0000 7ff091ffb700 10 client.17808 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T19:04:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.467+0000 7ff091ffb700 1 -- 172.21.15.39:0/4252816146 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5588f37a7ff0 con 0x5588f38a12d0 2023-07-21T19:04:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.467+0000 7ff091ffb700 10 client.17808 handle_command_reply: tid=0 2023-07-21T19:04:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.467+0000 7ff0bc8d2b80 1 client.17808 shutdown 2023-07-21T19:04:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.468+0000 7ff0bc8d2b80 1 -- 172.21.15.39:0/4252816146 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588f36f76b0 msgr2=0x5588f3901c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.468+0000 7ff0bc8d2b80 1 --2- 172.21.15.39:0/4252816146 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588f36f76b0 0x5588f3901c50 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4096810 tx=0x7ff0a4099f20 comp rx=0 tx=0).stop 2023-07-21T19:04:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.468+0000 7ff0bc8d2b80 1 -- 172.21.15.39:0/4252816146 shutdown_connections 2023-07-21T19:04:49.480 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.468+0000 7ff0bc8d2b80 1 --2- 172.21.15.39:0/4252816146 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5588f38a12d0 0x5588f390b700 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:49.480 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.468+0000 7ff0bc8d2b80 1 --2- 172.21.15.39:0/4252816146 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588f389b580 0x5588f39009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:49.480 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.468+0000 7ff0bc8d2b80 1 --2- 172.21.15.39:0/4252816146 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588f36f76b0 0x5588f3901c50 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:49.480 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.468+0000 7ff0bc8d2b80 1 --2- 172.21.15.39:0/4252816146 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5588f3890be0 0x5588f3902190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:49.481 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.468+0000 7ff0bc8d2b80 1 -- 172.21.15.39:0/4252816146 >> 172.21.15.39:0/4252816146 conn(0x5588f38b1290 msgr2=0x5588f38a2190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:04:49.481 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.468+0000 7ff091ffb700 0 client.17808 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T19:04:49.481 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.468+0000 7ff0bc8d2b80 1 -- 172.21.15.39:0/4252816146 shutdown_connections 2023-07-21T19:04:49.481 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.468+0000 7ff0bc8d2b80 1 -- 172.21.15.39:0/4252816146 wait complete. 2023-07-21T19:04:49.482 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.468+0000 7ff0bc8d2b80 20 client.17808 trim_cache size 0 max 16384 2023-07-21T19:04:49.482 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T19:04:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T19:04:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T19:04:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T19:04:49.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.469+0000 7ff0b074c700 1 -- 172.21.15.39:0/2516088170 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7ff0940a1340 msgr2=0x7ff0940a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:49.485 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.469+0000 7ff0b074c700 1 --2- 172.21.15.39:0/2516088170 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7ff0940a1340 0x7ff0940a37f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff098009fd0 tx=0x7ff098005b60 comp rx=0 tx=0).stop 2023-07-21T19:04:49.485 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.469+0000 7ff0b074c700 1 -- 172.21.15.39:0/2516088170 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0a810a550 msgr2=0x7ff0a8117010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:49.485 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.469+0000 7ff0b074c700 1 --2- 172.21.15.39:0/2516088170 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0a810a550 0x7ff0a8117010 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7ff0a400e9c0 tx=0x7ff0a400ecd0 comp rx=0 tx=0).stop 2023-07-21T19:04:49.485 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.469+0000 7ff0b074c700 1 -- 172.21.15.39:0/2516088170 shutdown_connections 2023-07-21T19:04:49.485 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.469+0000 7ff0b074c700 1 --2- 172.21.15.39:0/2516088170 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7ff0940a1340 0x7ff0940a37f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:49.486 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.469+0000 7ff0b074c700 1 --2- 172.21.15.39:0/2516088170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0a8110570 0x7ff0a81143a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:49.486 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.469+0000 7ff0b074c700 1 --2- 172.21.15.39:0/2516088170 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0a810a550 0x7ff0a8117010 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:49.486 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.469+0000 7ff0b074c700 1 --2- 172.21.15.39:0/2516088170 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff0a810af00 0x7ff0a8113e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:49.486 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.469+0000 7ff0b074c700 1 -- 172.21.15.39:0/2516088170 >> 172.21.15.39:0/2516088170 conn(0x7ff0a806a900 msgr2=0x7ff0a8109030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:04:49.486 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.469+0000 7ff0b074c700 1 -- 172.21.15.39:0/2516088170 shutdown_connections 2023-07-21T19:04:49.486 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:04:49.469+0000 7ff0b074c700 1 -- 172.21.15.39:0/2516088170 wait complete. 2023-07-21T19:04:49.721 INFO:tasks.workunit.client.0.smithi039.stdout: 10 319842 28.27 MB/sec execute 482 sec latency 253.577 ms 2023-07-21T19:04:49.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:04:49 smithi193 ceph-mon[98204]: pgmap v733: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.9 MiB/s wr, 383 op/s 2023-07-21T19:04:49.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:04:49 smithi155 ceph-mon[97934]: pgmap v733: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.9 MiB/s wr, 383 op/s 2023-07-21T19:04:49.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:04:49 smithi039 ceph-mon[96489]: pgmap v733: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.9 MiB/s wr, 383 op/s 2023-07-21T19:04:50.721 INFO:tasks.workunit.client.0.smithi039.stdout: 10 320220 28.23 MB/sec execute 483 sec latency 137.430 ms 2023-07-21T19:04:51.721 INFO:tasks.workunit.client.0.smithi039.stdout: 10 320727 28.23 MB/sec execute 484 sec latency 68.451 ms 2023-07-21T19:04:52.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:04:52 smithi039 ceph-mon[96489]: pgmap v734: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 8.2 MiB/s wr, 418 op/s 2023-07-21T19:04:52.721 INFO:tasks.workunit.client.0.smithi039.stdout: 10 321357 28.24 MB/sec execute 485 sec latency 84.028 ms 2023-07-21T19:04:52.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:04:52 smithi193 ceph-mon[98204]: pgmap v734: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 8.2 MiB/s wr, 418 op/s 2023-07-21T19:04:52.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:04:52 smithi155 ceph-mon[97934]: pgmap v734: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 8.2 MiB/s wr, 418 op/s 2023-07-21T19:04:53.721 INFO:tasks.workunit.client.0.smithi039.stdout: 10 322070 28.26 MB/sec execute 486 sec latency 69.398 ms 2023-07-21T19:04:54.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:04:54 smithi039 ceph-mon[96489]: pgmap v735: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 12 MiB/s wr, 605 op/s 2023-07-21T19:04:54.721 INFO:tasks.workunit.client.0.smithi039.stdout: 10 322707 28.28 MB/sec execute 487 sec latency 82.079 ms 2023-07-21T19:04:54.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:04:54 smithi193 ceph-mon[98204]: pgmap v735: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 12 MiB/s wr, 605 op/s 2023-07-21T19:04:54.879 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:04:54 smithi155 ceph-mon[97934]: pgmap v735: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 12 MiB/s wr, 605 op/s 2023-07-21T19:04:55.721 INFO:tasks.workunit.client.0.smithi039.stdout: 10 323153 28.28 MB/sec execute 488 sec latency 153.138 ms 2023-07-21T19:04:56.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:04:56 smithi039 ceph-mon[96489]: pgmap v736: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 8.3 MiB/s wr, 429 op/s 2023-07-21T19:04:56.721 INFO:tasks.workunit.client.0.smithi039.stdout: 10 323526 28.26 MB/sec execute 489 sec latency 117.389 ms 2023-07-21T19:04:56.860 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:04:56 smithi193 ceph-mon[98204]: pgmap v736: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 8.3 MiB/s wr, 429 op/s 2023-07-21T19:04:56.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:04:56 smithi155 ceph-mon[97934]: pgmap v736: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 8.3 MiB/s wr, 429 op/s 2023-07-21T19:04:57.721 INFO:tasks.workunit.client.0.smithi039.stdout: 10 323891 28.24 MB/sec execute 490 sec latency 96.119 ms 2023-07-21T19:04:58.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:04:58 smithi039 ceph-mon[96489]: pgmap v737: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 618 op/s 2023-07-21T19:04:58.723 INFO:tasks.workunit.client.0.smithi039.stdout: 10 324431 28.23 MB/sec execute 491 sec latency 90.941 ms 2023-07-21T19:04:58.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:04:58 smithi193 ceph-mon[98204]: pgmap v737: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 618 op/s 2023-07-21T19:04:58.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:04:58 smithi155 ceph-mon[97934]: pgmap v737: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 618 op/s 2023-07-21T19:04:59.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:04:59 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T19:04:59.721 INFO:tasks.workunit.client.0.smithi039.stdout: 10 325044 28.24 MB/sec execute 492 sec latency 64.069 ms 2023-07-21T19:04:59.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:04:59 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T19:04:59.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:04:59 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T19:05:00.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:05:00 smithi039 ceph-mon[96489]: pgmap v738: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.9 MiB/s wr, 432 op/s 2023-07-21T19:05:00.722 INFO:tasks.workunit.client.0.smithi039.stdout: 10 325408 28.23 MB/sec execute 493 sec latency 567.703 ms 2023-07-21T19:05:00.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:05:00 smithi193 ceph-mon[98204]: pgmap v738: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.9 MiB/s wr, 432 op/s 2023-07-21T19:05:00.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:05:00 smithi155 ceph-mon[97934]: pgmap v738: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.9 MiB/s wr, 432 op/s 2023-07-21T19:05:01.721 INFO:tasks.workunit.client.0.smithi039.stdout: 10 325408 28.17 MB/sec execute 494 sec latency 1567.795 ms 2023-07-21T19:05:02.603 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.593+0000 7fcb64a83700 1 -- 172.21.15.39:0/692036235 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb60104a30 msgr2=0x7fcb60104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:02.603 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.593+0000 7fcb64a83700 1 --2- 172.21.15.39:0/692036235 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb60104a30 0x7fcb60104e00 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fcb48009990 tx=0x7fcb48009ca0 comp rx=0 tx=0).stop 2023-07-21T19:05:02.604 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.593+0000 7fcb64a83700 1 -- 172.21.15.39:0/692036235 shutdown_connections 2023-07-21T19:05:02.604 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.593+0000 7fcb64a83700 1 --2- 172.21.15.39:0/692036235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb600ff3e0 0x7fcb6010b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:02.604 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.593+0000 7fcb64a83700 1 --2- 172.21.15.39:0/692036235 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb60104a30 0x7fcb60104e00 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:02.604 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.593+0000 7fcb64a83700 1 --2- 172.21.15.39:0/692036235 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb600fea30 0x7fcb600feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:02.604 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.593+0000 7fcb64a83700 1 -- 172.21.15.39:0/692036235 >> 172.21.15.39:0/692036235 conn(0x7fcb60074760 msgr2=0x7fcb60074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:02.604 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.593+0000 7fcb64a83700 1 -- 172.21.15.39:0/692036235 shutdown_connections 2023-07-21T19:05:02.604 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.593+0000 7fcb64a83700 1 -- 172.21.15.39:0/692036235 wait complete. 2023-07-21T19:05:02.605 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.594+0000 7fcb64a83700 1 Processor -- start 2023-07-21T19:05:02.605 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.594+0000 7fcb64a83700 1 -- start start 2023-07-21T19:05:02.606 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.594+0000 7fcb64a83700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb600fea30 0x7fcb6019aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:02.606 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.594+0000 7fcb64a83700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb600ff3e0 0x7fcb6019afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:02.607 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.594+0000 7fcb64a83700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb60104a30 0x7fcb601948e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:02.607 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.594+0000 7fcb64a83700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb60194f50 con 0x7fcb60104a30 2023-07-21T19:05:02.607 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.594+0000 7fcb64a83700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb601950c0 con 0x7fcb600ff3e0 2023-07-21T19:05:02.607 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.594+0000 7fcb64a83700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb60195280 con 0x7fcb600fea30 2023-07-21T19:05:02.607 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.594+0000 7fcb5e59c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb600fea30 0x7fcb6019aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:02.607 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.594+0000 7fcb5ed9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb60104a30 0x7fcb601948e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:02.607 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.594+0000 7fcb5e59c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb600fea30 0x7fcb6019aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:53362/0 (socket says 172.21.15.39:53362) 2023-07-21T19:05:02.608 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.594+0000 7fcb5ed9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb60104a30 0x7fcb601948e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34814/0 (socket says 172.21.15.39:34814) 2023-07-21T19:05:02.608 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.594+0000 7fcb5e59c700 1 -- 172.21.15.39:0/1207331221 learned_addr learned my addr 172.21.15.39:0/1207331221 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:05:02.608 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.594+0000 7fcb5dd9b700 1 --2- 172.21.15.39:0/1207331221 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb600ff3e0 0x7fcb6019afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:02.608 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.594+0000 7fcb5ed9d700 1 -- 172.21.15.39:0/1207331221 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb600fea30 msgr2=0x7fcb6019aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:02.608 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.594+0000 7fcb5ed9d700 1 --2- 172.21.15.39:0/1207331221 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb600fea30 0x7fcb6019aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:02.608 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.594+0000 7fcb5ed9d700 1 -- 172.21.15.39:0/1207331221 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb600ff3e0 msgr2=0x7fcb6019afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:02.608 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.594+0000 7fcb5ed9d700 1 --2- 172.21.15.39:0/1207331221 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb600ff3e0 0x7fcb6019afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:02.608 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.594+0000 7fcb5ed9d700 1 -- 172.21.15.39:0/1207331221 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb480095f0 con 0x7fcb60104a30 2023-07-21T19:05:02.609 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.595+0000 7fcb5ed9d700 1 --2- 172.21.15.39:0/1207331221 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb60104a30 0x7fcb601948e0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fcb58007f10 tx=0x7fcb5800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:02.609 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.595+0000 7fcb577fe700 1 -- 172.21.15.39:0/1207331221 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5800ddd0 con 0x7fcb60104a30 2023-07-21T19:05:02.609 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.595+0000 7fcb577fe700 1 -- 172.21.15.39:0/1207331221 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb5800f040 con 0x7fcb60104a30 2023-07-21T19:05:02.609 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.595+0000 7fcb64a83700 1 -- 172.21.15.39:0/1207331221 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb60195560 con 0x7fcb60104a30 2023-07-21T19:05:02.609 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.595+0000 7fcb577fe700 1 -- 172.21.15.39:0/1207331221 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb58005a90 con 0x7fcb60104a30 2023-07-21T19:05:02.609 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.595+0000 7fcb64a83700 1 -- 172.21.15.39:0/1207331221 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb601a1a70 con 0x7fcb60104a30 2023-07-21T19:05:02.614 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.597+0000 7fcb577fe700 1 -- 172.21.15.39:0/1207331221 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fcb58004ad0 con 0x7fcb60104a30 2023-07-21T19:05:02.614 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.598+0000 7fcb577fe700 1 --2- 172.21.15.39:0/1207331221 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fcb4c0a12c0 0x7fcb4c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:02.614 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.598+0000 7fcb70c09b80 1 Processor -- start 2023-07-21T19:05:02.614 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.598+0000 7fcb70c09b80 1 -- start start 2023-07-21T19:05:02.614 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.598+0000 7fcb5e59c700 1 --2- 172.21.15.39:0/1207331221 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fcb4c0a12c0 0x7fcb4c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:02.615 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.598+0000 7fcb577fe700 1 -- 172.21.15.39:0/1207331221 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fcb580cc670 con 0x7fcb60104a30 2023-07-21T19:05:02.615 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.598+0000 7fcb70c09b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a90723b6b0 0x55a9073df040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:02.615 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.598+0000 7fcb70c09b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9073e0de0 0x55a9073df580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:02.615 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.598+0000 7fcb70c09b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9073dfac0 0x55a9073ef6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:02.615 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.598+0000 7fcb70c09b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9072e4c50 con 0x55a90723b6b0 2023-07-21T19:05:02.615 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.598+0000 7fcb70c09b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9073379e0 con 0x55a9073dfac0 2023-07-21T19:05:02.615 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.598+0000 7fcb70c09b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a907330e70 con 0x55a9073e0de0 2023-07-21T19:05:02.616 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.599+0000 7fcb5dd9b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9073e0de0 0x55a9073df580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:02.616 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.599+0000 7fcb5e59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a90723b6b0 0x55a9073df040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:02.616 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.599+0000 7fcb5dd9b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9073e0de0 0x55a9073df580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:53372/0 (socket says 172.21.15.39:53372) 2023-07-21T19:05:02.616 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.599+0000 7fcb5dd9b700 1 -- 172.21.15.39:0/3622463135 learned_addr learned my addr 172.21.15.39:0/3622463135 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:05:02.616 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.599+0000 7fcb5e59c700 1 --2- 172.21.15.39:0/1207331221 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fcb4c0a12c0 0x7fcb4c0a3770 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcb48006010 tx=0x7fcb48003780 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:02.616 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.599+0000 7fcb5ed9d700 1 --2- 172.21.15.39:0/3622463135 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9073dfac0 0x55a9073ef6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:02.616 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.599+0000 7fcb5e59c700 1 -- 172.21.15.39:0/3622463135 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9073e0de0 msgr2=0x55a9073df580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:05:02.616 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.599+0000 7fcb5e59c700 1 --2- 172.21.15.39:0/3622463135 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9073e0de0 0x55a9073df580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:02.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.599+0000 7fcb5e59c700 1 -- 172.21.15.39:0/3622463135 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9073dfac0 msgr2=0x55a9073ef6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:05:02.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.599+0000 7fcb5e59c700 1 --2- 172.21.15.39:0/3622463135 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9073dfac0 0x55a9073ef6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:02.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.599+0000 7fcb5e59c700 1 -- 172.21.15.39:0/3622463135 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a90715ed80 con 0x55a90723b6b0 2023-07-21T19:05:02.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.599+0000 7fcb5ed9d700 1 --2- 172.21.15.39:0/3622463135 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9073dfac0 0x55a9073ef6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:05:02.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.599+0000 7fcb5dd9b700 1 --2- 172.21.15.39:0/3622463135 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9073e0de0 0x55a9073df580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:05:02.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.599+0000 7fcb5e59c700 1 --2- 172.21.15.39:0/3622463135 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a90723b6b0 0x55a9073df040 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fcb480098e0 tx=0x7fcb48024000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f2e3a536b365abe server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:02.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.599+0000 7fcb54ff9700 1 -- 172.21.15.39:0/3622463135 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4802b040 con 0x55a90723b6b0 2023-07-21T19:05:02.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.599+0000 7fcb54ff9700 1 -- 172.21.15.39:0/3622463135 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb4801e040 con 0x55a90723b6b0 2023-07-21T19:05:02.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.599+0000 7fcb54ff9700 1 -- 172.21.15.39:0/3622463135 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb48026d60 con 0x55a90723b6b0 2023-07-21T19:05:02.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.600+0000 7fcb70c09b80 1 -- 172.21.15.39:0/3622463135 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a90723b6b0 msgr2=0x55a9073df040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:02.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.600+0000 7fcb70c09b80 1 --2- 172.21.15.39:0/3622463135 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a90723b6b0 0x55a9073df040 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fcb480098e0 tx=0x7fcb48024000 comp rx=0 tx=0).stop 2023-07-21T19:05:02.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.600+0000 7fcb70c09b80 1 -- 172.21.15.39:0/3622463135 shutdown_connections 2023-07-21T19:05:02.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.600+0000 7fcb70c09b80 1 --2- 172.21.15.39:0/3622463135 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9073dfac0 0x55a9073ef6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:02.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.600+0000 7fcb70c09b80 1 --2- 172.21.15.39:0/3622463135 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a90723b6b0 0x55a9073df040 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:02.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.600+0000 7fcb70c09b80 1 --2- 172.21.15.39:0/3622463135 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9073e0de0 0x55a9073df580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:02.619 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.600+0000 7fcb70c09b80 1 -- 172.21.15.39:0/3622463135 >> 172.21.15.39:0/3622463135 conn(0x55a9073d4be0 msgr2=0x55a9073d9f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:02.619 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.600+0000 7fcb70c09b80 1 -- 172.21.15.39:0/3622463135 shutdown_connections 2023-07-21T19:05:02.619 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.600+0000 7fcb70c09b80 1 -- 172.21.15.39:0/3622463135 wait complete. 2023-07-21T19:05:02.619 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.600+0000 7fcb70c09b80 1 -- start start 2023-07-21T19:05:02.619 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.601+0000 7fcb70c09b80 1 Processor -- start 2023-07-21T19:05:02.619 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.601+0000 7fcb70c09b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a90723b6b0 0x55a9073ec990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:02.619 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.601+0000 7fcb70c09b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9073dfac0 0x55a9073eced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:02.620 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.601+0000 7fcb70c09b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a9073e0de0 0x55a9073ed410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:02.620 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.601+0000 7fcb70c09b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9072d0a60 con 0x55a9073e0de0 2023-07-21T19:05:02.620 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.601+0000 7fcb70c09b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9072d0760 con 0x55a90723b6b0 2023-07-21T19:05:02.620 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.601+0000 7fcb70c09b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9071b48c0 con 0x55a9073dfac0 2023-07-21T19:05:02.620 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.601+0000 7fcb70c09b80 10 client.0 _ll_register_callbacks cb 0x55a9072ce700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:05:02.620 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.601+0000 7fcb5ed9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a9073e0de0 0x55a9073ed410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:02.620 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.601+0000 7fcb5ed9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a9073e0de0 0x55a9073ed410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34828/0 (socket says 172.21.15.39:34828) 2023-07-21T19:05:02.620 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.601+0000 7fcb5ed9d700 1 -- 172.21.15.39:0/4138684854 learned_addr learned my addr 172.21.15.39:0/4138684854 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:05:02.621 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.601+0000 7fcb5e59c700 1 --2- 172.21.15.39:0/4138684854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a90723b6b0 0x55a9073ec990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:02.621 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.601+0000 7fcb5dd9b700 1 --2- 172.21.15.39:0/4138684854 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9073dfac0 0x55a9073eced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:02.621 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.601+0000 7fcb5ed9d700 1 -- 172.21.15.39:0/4138684854 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9073dfac0 msgr2=0x55a9073eced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:05:02.621 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.601+0000 7fcb5ed9d700 1 --2- 172.21.15.39:0/4138684854 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9073dfac0 0x55a9073eced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:02.621 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.601+0000 7fcb5ed9d700 1 -- 172.21.15.39:0/4138684854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a90723b6b0 msgr2=0x55a9073ec990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:05:02.621 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.601+0000 7fcb5ed9d700 1 --2- 172.21.15.39:0/4138684854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a90723b6b0 0x55a9073ec990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:02.621 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.601+0000 7fcb5ed9d700 1 -- 172.21.15.39:0/4138684854 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb5000b940 con 0x55a9073e0de0 2023-07-21T19:05:02.622 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.602+0000 7fcb5ed9d700 1 --2- 172.21.15.39:0/4138684854 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a9073e0de0 0x55a9073ed410 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fcb5800d7b0 tx=0x7fcb580990c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=128a731934eb64e5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:02.622 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.602+0000 7fcb45ffb700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T19:05:02.622 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.602+0000 7fcb45ffb700 1 -- 172.21.15.39:0/4138684854 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb58005200 con 0x55a9073e0de0 2023-07-21T19:05:02.622 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.602+0000 7fcb45ffb700 1 -- 172.21.15.39:0/4138684854 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fcb580996e0 con 0x55a9073e0de0 2023-07-21T19:05:02.622 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.602+0000 7fcb70c09b80 1 -- 172.21.15.39:0/4138684854 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a90728a490 con 0x55a9073e0de0 2023-07-21T19:05:02.622 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.602+0000 7fcb45ffb700 1 -- 172.21.15.39:0/4138684854 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb5809d710 con 0x55a9073e0de0 2023-07-21T19:05:02.622 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.602+0000 7fcb70c09b80 10 client.17820 fetch_fsmap learned FSMap version 105 2023-07-21T19:05:02.622 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.602+0000 7fcb70c09b80 1 -- 172.21.15.39:0/4138684854 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=4+}) v3 -- 0x7fcb6010c160 con 0x55a9073e0de0 2023-07-21T19:05:02.623 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.602+0000 7fcb45ffb700 1 -- 172.21.15.39:0/4138684854 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb580c7080 con 0x55a9073e0de0 2023-07-21T19:05:02.623 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.602+0000 7fcb45ffb700 1 -- 172.21.15.39:0/4138684854 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 105) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fcb580b6030 con 0x55a9073e0de0 2023-07-21T19:05:02.623 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.602+0000 7fcb70c09b80 10 client.17820 fetch_fsmap finished waiting for FSMap version 105 2023-07-21T19:05:02.623 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.602+0000 7fcb70c09b80 10 client.17820 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:05:02.623 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.602+0000 7fcb70c09b80 20 client.17820 populate_metadata read hostname 'smithi039' 2023-07-21T19:05:02.623 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.602+0000 7fcb70c09b80 1 --2- 172.21.15.39:0/4138684854 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55a9073fd010 0x55a9073ff420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:02.623 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.602+0000 7fcb70c09b80 4 client.17820 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:05:02.624 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.602+0000 7fcb70c09b80 1 -- 172.21.15.39:0/4138684854 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a907329e50 con 0x55a9073fd010 2023-07-21T19:05:02.624 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.603+0000 7fcb5e59c700 1 --2- 172.21.15.39:0/4138684854 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55a9073fd010 0x55a9073ff420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:02.624 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.603+0000 7fcb45ffb700 10 client.17820 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T19:05:02.624 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.603+0000 7fcb5e59c700 1 --2- 172.21.15.39:0/4138684854 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55a9073fd010 0x55a9073ff420 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a39a05e99b8ee700 server_cookie=d626dea74b2f7f13 in_seq=0 out_seq=0 2023-07-21T19:05:02.624 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.604+0000 7fcb45ffb700 1 -- 172.21.15.39:0/4138684854 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a907329e50 con 0x55a9073fd010 2023-07-21T19:05:02.624 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.604+0000 7fcb45ffb700 10 client.17820 handle_command_reply: tid=0 2023-07-21T19:05:02.624 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.604+0000 7fcb70c09b80 1 client.17820 shutdown 2023-07-21T19:05:02.624 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.604+0000 7fcb70c09b80 1 -- 172.21.15.39:0/4138684854 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a9073e0de0 msgr2=0x55a9073ed410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:02.625 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.604+0000 7fcb70c09b80 1 --2- 172.21.15.39:0/4138684854 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a9073e0de0 0x55a9073ed410 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fcb5800d7b0 tx=0x7fcb580990c0 comp rx=0 tx=0).stop 2023-07-21T19:05:02.625 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.604+0000 7fcb70c09b80 1 -- 172.21.15.39:0/4138684854 shutdown_connections 2023-07-21T19:05:02.625 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.604+0000 7fcb70c09b80 1 --2- 172.21.15.39:0/4138684854 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55a9073fd010 0x55a9073ff420 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:02.625 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.604+0000 7fcb70c09b80 1 --2- 172.21.15.39:0/4138684854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a90723b6b0 0x55a9073ec990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:02.625 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.604+0000 7fcb45ffb700 0 client.17820 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T19:05:02.625 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.604+0000 7fcb70c09b80 1 --2- 172.21.15.39:0/4138684854 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a9073e0de0 0x55a9073ed410 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:02.625 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.604+0000 7fcb70c09b80 1 --2- 172.21.15.39:0/4138684854 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9073dfac0 0x55a9073eced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:02.626 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.604+0000 7fcb70c09b80 1 -- 172.21.15.39:0/4138684854 >> 172.21.15.39:0/4138684854 conn(0x55a9073d4be0 msgr2=0x55a9073d94e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:02.626 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.604+0000 7fcb70c09b80 1 -- 172.21.15.39:0/4138684854 shutdown_connections 2023-07-21T19:05:02.626 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.604+0000 7fcb70c09b80 1 -- 172.21.15.39:0/4138684854 wait complete. 2023-07-21T19:05:02.626 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.604+0000 7fcb70c09b80 20 client.17820 trim_cache size 0 max 16384 2023-07-21T19:05:02.630 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.619+0000 7fcb70c09b80 1 Processor -- start 2023-07-21T19:05:02.630 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.619+0000 7fcb70c09b80 1 -- start start 2023-07-21T19:05:02.631 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.619+0000 7fcb70c09b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a9073d4be0 0x55a9073d9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:02.631 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.619+0000 7fcb70c09b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a90723b6b0 0x55a9073df040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:02.631 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.619+0000 7fcb70c09b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9073dfac0 0x55a9073df580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:02.631 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.619+0000 7fcb70c09b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9073ef9d0 con 0x55a9073d4be0 2023-07-21T19:05:02.631 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.619+0000 7fcb70c09b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9073efda0 con 0x55a9073dfac0 2023-07-21T19:05:02.631 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.619+0000 7fcb70c09b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9073dab50 con 0x55a90723b6b0 2023-07-21T19:05:02.631 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.619+0000 7fcb5e59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a9073d4be0 0x55a9073d9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:02.632 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.619+0000 7fcb5e59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a9073d4be0 0x55a9073d9c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34832/0 (socket says 172.21.15.39:34832) 2023-07-21T19:05:02.632 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.619+0000 7fcb5e59c700 1 -- 172.21.15.39:0/555587322 learned_addr learned my addr 172.21.15.39:0/555587322 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:05:02.632 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.619+0000 7fcb5dd9b700 1 --2- 172.21.15.39:0/555587322 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a90723b6b0 0x55a9073df040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:02.632 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.619+0000 7fcb5ed9d700 1 --2- 172.21.15.39:0/555587322 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9073dfac0 0x55a9073df580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:02.632 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.619+0000 7fcb5e59c700 1 -- 172.21.15.39:0/555587322 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a90723b6b0 msgr2=0x55a9073df040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:05:02.632 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.619+0000 7fcb5e59c700 1 --2- 172.21.15.39:0/555587322 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a90723b6b0 0x55a9073df040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.619+0000 7fcb5e59c700 1 -- 172.21.15.39:0/555587322 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9073dfac0 msgr2=0x55a9073df580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:05:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.619+0000 7fcb5e59c700 1 --2- 172.21.15.39:0/555587322 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9073dfac0 0x55a9073df580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.619+0000 7fcb5e59c700 1 -- 172.21.15.39:0/555587322 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a907147c00 con 0x55a9073d4be0 2023-07-21T19:05:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.619+0000 7fcb5e59c700 1 --2- 172.21.15.39:0/555587322 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a9073d4be0 0x55a9073d9c60 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fcb48026930 tx=0x7fcb480268c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c28af56f57c61720 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:02.634 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.620+0000 7fcb557fa700 1 -- 172.21.15.39:0/555587322 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4803f070 con 0x55a9073d4be0 2023-07-21T19:05:02.634 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.620+0000 7fcb557fa700 1 -- 172.21.15.39:0/555587322 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb4800b540 con 0x55a9073d4be0 2023-07-21T19:05:02.634 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.620+0000 7fcb557fa700 1 -- 172.21.15.39:0/555587322 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4802b040 con 0x55a9073d4be0 2023-07-21T19:05:02.634 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.620+0000 7fcb70c09b80 1 -- 172.21.15.39:0/555587322 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a9073d4be0 msgr2=0x55a9073d9c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:02.634 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.620+0000 7fcb70c09b80 1 --2- 172.21.15.39:0/555587322 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a9073d4be0 0x55a9073d9c60 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fcb48026930 tx=0x7fcb480268c0 comp rx=0 tx=0).stop 2023-07-21T19:05:02.634 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.620+0000 7fcb70c09b80 1 -- 172.21.15.39:0/555587322 shutdown_connections 2023-07-21T19:05:02.634 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.620+0000 7fcb70c09b80 1 --2- 172.21.15.39:0/555587322 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9073dfac0 0x55a9073df580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.620+0000 7fcb70c09b80 1 --2- 172.21.15.39:0/555587322 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a9073d4be0 0x55a9073d9c60 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.620+0000 7fcb70c09b80 1 --2- 172.21.15.39:0/555587322 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a90723b6b0 0x55a9073df040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.620+0000 7fcb70c09b80 1 -- 172.21.15.39:0/555587322 >> 172.21.15.39:0/555587322 conn(0x55a9073f95f0 msgr2=0x55a9073ed1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.620+0000 7fcb70c09b80 1 -- 172.21.15.39:0/555587322 shutdown_connections 2023-07-21T19:05:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.620+0000 7fcb70c09b80 1 -- 172.21.15.39:0/555587322 wait complete. 2023-07-21T19:05:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.621+0000 7fcb70c09b80 1 -- start start 2023-07-21T19:05:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.621+0000 7fcb70c09b80 1 Processor -- start 2023-07-21T19:05:02.638 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.621+0000 7fcb70c09b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a90723b6b0 0x55a9073f3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:02.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.621+0000 7fcb70c09b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a9073d4be0 0x55a9073f4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:02.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.621+0000 7fcb70c09b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9073dfac0 0x55a9073f21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:02.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.621+0000 7fcb70c09b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9073f28c0 con 0x55a9073d4be0 2023-07-21T19:05:02.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.621+0000 7fcb70c09b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9073f2a30 con 0x55a9073dfac0 2023-07-21T19:05:02.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.621+0000 7fcb70c09b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9073f2bf0 con 0x55a90723b6b0 2023-07-21T19:05:02.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.621+0000 7fcb70c09b80 10 client.0 _ll_register_callbacks cb 0x7fcb580994b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:05:02.640 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.621+0000 7fcb5dd9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a9073d4be0 0x55a9073f4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:02.640 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.621+0000 7fcb5dd9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a9073d4be0 0x55a9073f4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34840/0 (socket says 172.21.15.39:34840) 2023-07-21T19:05:02.640 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.621+0000 7fcb5dd9b700 1 -- 172.21.15.39:0/2470461058 learned_addr learned my addr 172.21.15.39:0/2470461058 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:05:02.640 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.621+0000 7fcb5e59c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a90723b6b0 0x55a9073f3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:02.640 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.621+0000 7fcb5ed9d700 1 --2- 172.21.15.39:0/2470461058 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9073dfac0 0x55a9073f21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:02.640 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.621+0000 7fcb5dd9b700 1 -- 172.21.15.39:0/2470461058 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a90723b6b0 msgr2=0x55a9073f3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:05:02.640 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.621+0000 7fcb5dd9b700 1 --2- 172.21.15.39:0/2470461058 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a90723b6b0 0x55a9073f3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:02.641 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.621+0000 7fcb5dd9b700 1 -- 172.21.15.39:0/2470461058 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9073dfac0 msgr2=0x55a9073f21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:05:02.641 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.621+0000 7fcb5dd9b700 1 --2- 172.21.15.39:0/2470461058 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9073dfac0 0x55a9073f21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:02.641 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.621+0000 7fcb5dd9b700 1 -- 172.21.15.39:0/2470461058 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a90715ed80 con 0x55a9073d4be0 2023-07-21T19:05:02.641 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.622+0000 7fcb5dd9b700 1 --2- 172.21.15.39:0/2470461058 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a9073d4be0 0x55a9073f4210 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fcb500116c0 tx=0x7fcb5000bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3481410c51a6ec45 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:02.641 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.622+0000 7fcb45ffb700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T19:05:02.641 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.622+0000 7fcb45ffb700 1 -- 172.21.15.39:0/2470461058 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb50004440 con 0x55a9073d4be0 2023-07-21T19:05:02.641 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.622+0000 7fcb45ffb700 1 -- 172.21.15.39:0/2470461058 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fcb50003680 con 0x55a9073d4be0 2023-07-21T19:05:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.622+0000 7fcb70c09b80 1 -- 172.21.15.39:0/2470461058 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a9073f99c0 con 0x55a9073d4be0 2023-07-21T19:05:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.622+0000 7fcb45ffb700 1 -- 172.21.15.39:0/2470461058 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb50003840 con 0x55a9073d4be0 2023-07-21T19:05:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.622+0000 7fcb70c09b80 10 client.17826 fetch_fsmap learned FSMap version 105 2023-07-21T19:05:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.622+0000 7fcb70c09b80 1 -- 172.21.15.39:0/2470461058 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=4+}) v3 -- 0x55a9071851f0 con 0x55a9073d4be0 2023-07-21T19:05:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.622+0000 7fcb45ffb700 1 -- 172.21.15.39:0/2470461058 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb50029020 con 0x55a9073d4be0 2023-07-21T19:05:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.622+0000 7fcb45ffb700 1 -- 172.21.15.39:0/2470461058 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 105) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fcb50010a50 con 0x55a9073d4be0 2023-07-21T19:05:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.622+0000 7fcb70c09b80 10 client.17826 fetch_fsmap finished waiting for FSMap version 105 2023-07-21T19:05:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.622+0000 7fcb70c09b80 10 client.17826 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:05:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.622+0000 7fcb70c09b80 20 client.17826 populate_metadata read hostname 'smithi039' 2023-07-21T19:05:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.622+0000 7fcb70c09b80 1 --2- 172.21.15.39:0/2470461058 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55a9073e0de0 0x55a9074542c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.622+0000 7fcb70c09b80 4 client.17826 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:05:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.622+0000 7fcb70c09b80 1 -- 172.21.15.39:0/2470461058 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7fcb4801e040 con 0x55a9073e0de0 2023-07-21T19:05:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.623+0000 7fcb5e59c700 1 --2- 172.21.15.39:0/2470461058 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55a9073e0de0 0x55a9074542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.623+0000 7fcb45ffb700 10 client.17826 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T19:05:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.623+0000 7fcb5e59c700 1 --2- 172.21.15.39:0/2470461058 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55a9073e0de0 0x55a9074542c0 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ce55b56580b6273 server_cookie=e3ae99f5a944dd25 in_seq=0 out_seq=0 2023-07-21T19:05:02.685 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:05:02 smithi039 ceph-mon[96489]: pgmap v739: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 8.0 MiB/s wr, 452 op/s 2023-07-21T19:05:02.723 INFO:tasks.workunit.client.0.smithi039.stdout: 10 325408 28.11 MB/sec execute 495 sec latency 2569.506 ms 2023-07-21T19:05:02.834 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.824+0000 7fcb45ffb700 1 -- 172.21.15.39:0/2470461058 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7fcb4801e040 con 0x55a9073e0de0 2023-07-21T19:05:02.835 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.824+0000 7fcb45ffb700 10 client.17826 handle_command_reply: tid=0 2023-07-21T19:05:02.835 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.825+0000 7fcb70c09b80 1 client.17826 shutdown 2023-07-21T19:05:02.838 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.828+0000 7fcb70c09b80 1 -- 172.21.15.39:0/2470461058 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a9073d4be0 msgr2=0x55a9073f4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:02.838 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.828+0000 7fcb70c09b80 1 --2- 172.21.15.39:0/2470461058 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a9073d4be0 0x55a9073f4210 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fcb500116c0 tx=0x7fcb5000bf90 comp rx=0 tx=0).stop 2023-07-21T19:05:02.844 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T19:05:02.844 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T19:05:02.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T19:05:02.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T19:05:02.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.830+0000 7fcb70c09b80 1 -- 172.21.15.39:0/2470461058 shutdown_connections 2023-07-21T19:05:02.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.830+0000 7fcb70c09b80 1 --2- 172.21.15.39:0/2470461058 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55a9073e0de0 0x55a9074542c0 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:02.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.831+0000 7fcb70c09b80 1 --2- 172.21.15.39:0/2470461058 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9073dfac0 0x55a9073f21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:02.846 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.831+0000 7fcb70c09b80 1 --2- 172.21.15.39:0/2470461058 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a90723b6b0 0x55a9073f3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:02.846 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.831+0000 7fcb70c09b80 1 --2- 172.21.15.39:0/2470461058 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a9073d4be0 0x55a9073f4210 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:02.846 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.831+0000 7fcb70c09b80 1 -- 172.21.15.39:0/2470461058 >> 172.21.15.39:0/2470461058 conn(0x55a9073f95f0 msgr2=0x55a9073eaef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:02.846 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.831+0000 7fcb45ffb700 0 client.17826 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T19:05:02.846 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.832+0000 7fcb70c09b80 1 -- 172.21.15.39:0/2470461058 shutdown_connections 2023-07-21T19:05:02.846 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.832+0000 7fcb70c09b80 1 -- 172.21.15.39:0/2470461058 wait complete. 2023-07-21T19:05:02.846 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.832+0000 7fcb70c09b80 20 client.17826 trim_cache size 0 max 16384 2023-07-21T19:05:02.847 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.833+0000 7fcb64a83700 1 -- 172.21.15.39:0/1207331221 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fcb4c0a12c0 msgr2=0x7fcb4c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:02.847 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.833+0000 7fcb64a83700 1 --2- 172.21.15.39:0/1207331221 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fcb4c0a12c0 0x7fcb4c0a3770 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcb48006010 tx=0x7fcb48003780 comp rx=0 tx=0).stop 2023-07-21T19:05:02.847 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.833+0000 7fcb64a83700 1 -- 172.21.15.39:0/1207331221 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb60104a30 msgr2=0x7fcb601948e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:02.847 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.833+0000 7fcb64a83700 1 --2- 172.21.15.39:0/1207331221 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb60104a30 0x7fcb601948e0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fcb58007f10 tx=0x7fcb5800d3b0 comp rx=0 tx=0).stop 2023-07-21T19:05:02.847 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.834+0000 7fcb64a83700 1 -- 172.21.15.39:0/1207331221 shutdown_connections 2023-07-21T19:05:02.847 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.834+0000 7fcb64a83700 1 --2- 172.21.15.39:0/1207331221 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb60104a30 0x7fcb601948e0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:02.847 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.834+0000 7fcb64a83700 1 --2- 172.21.15.39:0/1207331221 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb600fea30 0x7fcb6019aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:02.847 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.834+0000 7fcb64a83700 1 --2- 172.21.15.39:0/1207331221 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fcb4c0a12c0 0x7fcb4c0a3770 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:02.848 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.834+0000 7fcb64a83700 1 --2- 172.21.15.39:0/1207331221 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb600ff3e0 0x7fcb6019afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:02.848 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.834+0000 7fcb64a83700 1 -- 172.21.15.39:0/1207331221 >> 172.21.15.39:0/1207331221 conn(0x7fcb60074760 msgr2=0x7fcb601026c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:02.848 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.834+0000 7fcb64a83700 1 -- 172.21.15.39:0/1207331221 shutdown_connections 2023-07-21T19:05:02.848 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:02.834+0000 7fcb64a83700 1 -- 172.21.15.39:0/1207331221 wait complete. 2023-07-21T19:05:02.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:05:02 smithi193 ceph-mon[98204]: pgmap v739: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 8.0 MiB/s wr, 452 op/s 2023-07-21T19:05:02.878 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:05:02 smithi155 ceph-mon[97934]: pgmap v739: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 8.0 MiB/s wr, 452 op/s 2023-07-21T19:05:03.723 INFO:tasks.workunit.client.0.smithi039.stdout: 10 325580 28.07 MB/sec execute 496 sec latency 2660.190 ms 2023-07-21T19:05:04.723 INFO:tasks.workunit.client.0.smithi039.stdout: 10 325689 28.03 MB/sec execute 497 sec latency 1158.898 ms 2023-07-21T19:05:04.860 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:05:04 smithi193 ceph-mon[98204]: pgmap v740: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 11 MiB/s wr, 559 op/s 2023-07-21T19:05:04.879 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:05:04 smithi155 ceph-mon[97934]: pgmap v740: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 11 MiB/s wr, 559 op/s 2023-07-21T19:05:04.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:05:04 smithi039 ceph-mon[96489]: pgmap v740: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 11 MiB/s wr, 559 op/s 2023-07-21T19:05:05.724 INFO:tasks.workunit.client.0.smithi039.stdout: 10 325993 28.01 MB/sec execute 498 sec latency 220.103 ms 2023-07-21T19:05:06.724 INFO:tasks.workunit.client.0.smithi039.stdout: 10 326374 28.01 MB/sec execute 499 sec latency 113.924 ms 2023-07-21T19:05:06.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:05:06 smithi193 ceph-mon[98204]: pgmap v741: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 6.8 MiB/s wr, 350 op/s 2023-07-21T19:05:06.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:05:06 smithi155 ceph-mon[97934]: pgmap v741: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 6.8 MiB/s wr, 350 op/s 2023-07-21T19:05:06.927 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:05:06 smithi039 ceph-mon[96489]: pgmap v741: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 6.8 MiB/s wr, 350 op/s 2023-07-21T19:05:07.724 INFO:tasks.workunit.client.0.smithi039.stdout: 10 326794 27.99 MB/sec execute 500 sec latency 101.837 ms 2023-07-21T19:05:08.723 INFO:tasks.workunit.client.0.smithi039.stdout: 10 327250 27.99 MB/sec execute 501 sec latency 112.715 ms 2023-07-21T19:05:08.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:05:08 smithi193 ceph-mon[98204]: pgmap v742: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 20 MiB/s wr, 756 op/s 2023-07-21T19:05:08.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:05:08 smithi155 ceph-mon[97934]: pgmap v742: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 20 MiB/s wr, 756 op/s 2023-07-21T19:05:08.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:05:08 smithi039 ceph-mon[96489]: pgmap v742: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 20 MiB/s wr, 756 op/s 2023-07-21T19:05:09.724 INFO:tasks.workunit.client.0.smithi039.stdout: 10 327645 27.96 MB/sec execute 502 sec latency 131.906 ms 2023-07-21T19:05:10.724 INFO:tasks.workunit.client.0.smithi039.stdout: 10 328101 27.96 MB/sec execute 503 sec latency 160.183 ms 2023-07-21T19:05:10.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:05:10 smithi193 ceph-mon[98204]: pgmap v743: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 16 MiB/s wr, 567 op/s 2023-07-21T19:05:10.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:05:10 smithi155 ceph-mon[97934]: pgmap v743: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 16 MiB/s wr, 567 op/s 2023-07-21T19:05:10.927 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:05:10 smithi039 ceph-mon[96489]: pgmap v743: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 16 MiB/s wr, 567 op/s 2023-07-21T19:05:11.724 INFO:tasks.workunit.client.0.smithi039.stdout: 10 328632 27.96 MB/sec execute 504 sec latency 145.923 ms 2023-07-21T19:05:12.724 INFO:tasks.workunit.client.0.smithi039.stdout: 10 329142 27.97 MB/sec execute 505 sec latency 109.793 ms 2023-07-21T19:05:12.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:05:12 smithi193 ceph-mon[98204]: pgmap v744: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 16 MiB/s wr, 610 op/s 2023-07-21T19:05:12.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:05:12 smithi155 ceph-mon[97934]: pgmap v744: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 16 MiB/s wr, 610 op/s 2023-07-21T19:05:12.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:05:12 smithi039 ceph-mon[96489]: pgmap v744: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 16 MiB/s wr, 610 op/s 2023-07-21T19:05:13.724 INFO:tasks.workunit.client.0.smithi039.stdout: 10 329679 27.94 MB/sec execute 506 sec latency 129.203 ms 2023-07-21T19:05:14.724 INFO:tasks.workunit.client.0.smithi039.stdout: 10 330191 27.96 MB/sec execute 507 sec latency 89.984 ms 2023-07-21T19:05:14.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:05:14 smithi193 ceph-mon[98204]: pgmap v745: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 19 MiB/s wr, 743 op/s 2023-07-21T19:05:14.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:05:14 smithi155 ceph-mon[97934]: pgmap v745: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 19 MiB/s wr, 743 op/s 2023-07-21T19:05:14.928 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:05:14 smithi039 ceph-mon[96489]: pgmap v745: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 19 MiB/s wr, 743 op/s 2023-07-21T19:05:15.724 INFO:tasks.workunit.client.0.smithi039.stdout: 10 330745 27.98 MB/sec execute 508 sec latency 110.631 ms 2023-07-21T19:05:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.013+0000 7f4e61442700 1 -- 172.21.15.39:0/1158667489 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e5c104a30 msgr2=0x7f4e5c104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.013+0000 7f4e61442700 1 --2- 172.21.15.39:0/1158667489 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e5c104a30 0x7f4e5c104e00 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f4e50009990 tx=0x7f4e50009ca0 comp rx=0 tx=0).stop 2023-07-21T19:05:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.013+0000 7f4e61442700 1 -- 172.21.15.39:0/1158667489 shutdown_connections 2023-07-21T19:05:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.013+0000 7f4e61442700 1 --2- 172.21.15.39:0/1158667489 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e5c0ff3e0 0x7f4e5c067900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.013+0000 7f4e61442700 1 --2- 172.21.15.39:0/1158667489 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e5c104a30 0x7f4e5c104e00 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.013+0000 7f4e61442700 1 --2- 172.21.15.39:0/1158667489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e5c0fea30 0x7f4e5c0feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.013+0000 7f4e61442700 1 -- 172.21.15.39:0/1158667489 >> 172.21.15.39:0/1158667489 conn(0x7f4e5c074760 msgr2=0x7f4e5c074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:16.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.017+0000 7f4e61442700 1 -- 172.21.15.39:0/1158667489 shutdown_connections 2023-07-21T19:05:16.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.018+0000 7f4e61442700 1 -- 172.21.15.39:0/1158667489 wait complete. 2023-07-21T19:05:16.028 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.019+0000 7f4e61442700 1 Processor -- start 2023-07-21T19:05:16.028 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.019+0000 7f4e61442700 1 -- start start 2023-07-21T19:05:16.030 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.019+0000 7f4e61442700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e5c0fea30 0x7f4e5c197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:16.030 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.019+0000 7f4e61442700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e5c0ff3e0 0x7f4e5c198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:16.030 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.019+0000 7f4e61442700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e5c104a30 0x7f4e5c19c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:16.030 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.019+0000 7f4e61442700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5c19cff0 con 0x7f4e5c104a30 2023-07-21T19:05:16.030 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.019+0000 7f4e61442700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5c19d160 con 0x7f4e5c0fea30 2023-07-21T19:05:16.030 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.019+0000 7f4e61442700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5c19d320 con 0x7f4e5c0ff3e0 2023-07-21T19:05:16.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.019+0000 7f4e5a7fc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e5c0ff3e0 0x7f4e5c198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:16.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.019+0000 7f4e5a7fc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e5c0ff3e0 0x7f4e5c198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:53406/0 (socket says 172.21.15.39:53406) 2023-07-21T19:05:16.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.019+0000 7f4e5a7fc700 1 -- 172.21.15.39:0/1737578818 learned_addr learned my addr 172.21.15.39:0/1737578818 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:05:16.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.019+0000 7f4e5affd700 1 --2- 172.21.15.39:0/1737578818 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e5c0fea30 0x7f4e5c197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:16.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.019+0000 7f4e5b7fe700 1 --2- 172.21.15.39:0/1737578818 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e5c104a30 0x7f4e5c19c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:16.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.020+0000 7f4e5a7fc700 1 -- 172.21.15.39:0/1737578818 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e5c0fea30 msgr2=0x7f4e5c197e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:16.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.020+0000 7f4e5a7fc700 1 --2- 172.21.15.39:0/1737578818 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e5c0fea30 0x7f4e5c197e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.020+0000 7f4e5a7fc700 1 -- 172.21.15.39:0/1737578818 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e5c104a30 msgr2=0x7f4e5c19c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.020+0000 7f4e5a7fc700 1 --2- 172.21.15.39:0/1737578818 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e5c104a30 0x7f4e5c19c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.020+0000 7f4e5a7fc700 1 -- 172.21.15.39:0/1737578818 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e500095f0 con 0x7f4e5c0ff3e0 2023-07-21T19:05:16.034 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.020+0000 7f4e5a7fc700 1 --2- 172.21.15.39:0/1737578818 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e5c0ff3e0 0x7f4e5c198340 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c00d980 tx=0x7f4e4c00dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:16.034 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.020+0000 7f4e43fff700 1 -- 172.21.15.39:0/1737578818 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c004220 con 0x7f4e5c0ff3e0 2023-07-21T19:05:16.034 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.020+0000 7f4e43fff700 1 -- 172.21.15.39:0/1737578818 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e4c007640 con 0x7f4e5c0ff3e0 2023-07-21T19:05:16.035 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.020+0000 7f4e61442700 1 -- 172.21.15.39:0/1737578818 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e5c19d600 con 0x7f4e5c0ff3e0 2023-07-21T19:05:16.035 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.020+0000 7f4e43fff700 1 -- 172.21.15.39:0/1737578818 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c005690 con 0x7f4e5c0ff3e0 2023-07-21T19:05:16.035 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.020+0000 7f4e61442700 1 -- 172.21.15.39:0/1737578818 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e5c1a4e40 con 0x7f4e5c0ff3e0 2023-07-21T19:05:16.038 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.023+0000 7f4e43fff700 1 -- 172.21.15.39:0/1737578818 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f4e4c004830 con 0x7f4e5c0ff3e0 2023-07-21T19:05:16.038 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.023+0000 7f4e43fff700 1 --2- 172.21.15.39:0/1737578818 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f4e440a1340 0x7f4e440a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:16.038 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.023+0000 7f4e43fff700 1 -- 172.21.15.39:0/1737578818 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f4e4c0cca00 con 0x7f4e5c0ff3e0 2023-07-21T19:05:16.038 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.024+0000 7f4e5affd700 1 --2- 172.21.15.39:0/1737578818 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f4e440a1340 0x7f4e440a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:16.039 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.024+0000 7f4e5affd700 1 --2- 172.21.15.39:0/1737578818 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f4e440a1340 0x7f4e440a37f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4e5000b2e0 tx=0x7f4e50005fd0 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:16.039 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.024+0000 7f4e6d5c8b80 1 Processor -- start 2023-07-21T19:05:16.039 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.025+0000 7f4e6d5c8b80 1 -- start start 2023-07-21T19:05:16.039 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.025+0000 7f4e6d5c8b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55dd598936b0 0x55dd59a3cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:16.039 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.025+0000 7f4e6d5c8b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd59a3d2d0 0x55dd59a37040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:16.039 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.025+0000 7f4e6d5c8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd59a37580 0x55dd59a379d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:16.039 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.025+0000 7f4e6d5c8b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd5993cc50 con 0x55dd598936b0 2023-07-21T19:05:16.040 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.025+0000 7f4e6d5c8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd5998f9e0 con 0x55dd59a37580 2023-07-21T19:05:16.040 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.025+0000 7f4e6d5c8b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd59988e70 con 0x55dd59a3d2d0 2023-07-21T19:05:16.040 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.025+0000 7f4e5b7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd59a37580 0x55dd59a379d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:16.040 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.025+0000 7f4e5a7fc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd59a3d2d0 0x55dd59a37040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:16.040 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.025+0000 7f4e5b7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd59a37580 0x55dd59a379d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:47458/0 (socket says 172.21.15.39:47458) 2023-07-21T19:05:16.040 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.025+0000 7f4e5a7fc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd59a3d2d0 0x55dd59a37040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:53416/0 (socket says 172.21.15.39:53416) 2023-07-21T19:05:16.040 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.025+0000 7f4e5b7fe700 1 -- 172.21.15.39:0/2745483288 learned_addr learned my addr 172.21.15.39:0/2745483288 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:05:16.041 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.025+0000 7f4e5a7fc700 1 -- 172.21.15.39:0/2745483288 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd59a37580 msgr2=0x55dd59a379d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:05:16.041 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.025+0000 7f4e5a7fc700 1 --2- 172.21.15.39:0/2745483288 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd59a37580 0x55dd59a379d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:16.041 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.025+0000 7f4e5a7fc700 1 -- 172.21.15.39:0/2745483288 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55dd598936b0 msgr2=0x55dd59a3cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:05:16.041 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.025+0000 7f4e5a7fc700 1 --2- 172.21.15.39:0/2745483288 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55dd598936b0 0x55dd59a3cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:16.041 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.025+0000 7f4e5a7fc700 1 -- 172.21.15.39:0/2745483288 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd597b6d80 con 0x55dd59a3d2d0 2023-07-21T19:05:16.041 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.025+0000 7f4e5b7fe700 1 --2- 172.21.15.39:0/2745483288 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd59a37580 0x55dd59a379d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:05:16.041 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.026+0000 7f4e5a7fc700 1 --2- 172.21.15.39:0/2745483288 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd59a3d2d0 0x55dd59a37040 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c0ca150 tx=0x7f4e4c099ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b161392b865e8c36 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:16.041 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.026+0000 7f4e417fa700 1 -- 172.21.15.39:0/2745483288 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c0a1f20 con 0x55dd59a3d2d0 2023-07-21T19:05:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.026+0000 7f4e417fa700 1 -- 172.21.15.39:0/2745483288 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e4c004380 con 0x55dd59a3d2d0 2023-07-21T19:05:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.026+0000 7f4e417fa700 1 -- 172.21.15.39:0/2745483288 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c0a1a90 con 0x55dd59a3d2d0 2023-07-21T19:05:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.027+0000 7f4e6d5c8b80 1 -- 172.21.15.39:0/2745483288 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd59a3d2d0 msgr2=0x55dd59a37040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.027+0000 7f4e6d5c8b80 1 --2- 172.21.15.39:0/2745483288 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd59a3d2d0 0x55dd59a37040 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c0ca150 tx=0x7f4e4c099ad0 comp rx=0 tx=0).stop 2023-07-21T19:05:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.027+0000 7f4e6d5c8b80 1 -- 172.21.15.39:0/2745483288 shutdown_connections 2023-07-21T19:05:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.027+0000 7f4e6d5c8b80 1 --2- 172.21.15.39:0/2745483288 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd59a37580 0x55dd59a379d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.027+0000 7f4e6d5c8b80 1 --2- 172.21.15.39:0/2745483288 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55dd598936b0 0x55dd59a3cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.027+0000 7f4e6d5c8b80 1 --2- 172.21.15.39:0/2745483288 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd59a3d2d0 0x55dd59a37040 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.027+0000 7f4e6d5c8b80 1 -- 172.21.15.39:0/2745483288 >> 172.21.15.39:0/2745483288 conn(0x55dd59a2cbe0 msgr2=0x55dd59a31f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.027+0000 7f4e6d5c8b80 1 -- 172.21.15.39:0/2745483288 shutdown_connections 2023-07-21T19:05:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.027+0000 7f4e6d5c8b80 1 -- 172.21.15.39:0/2745483288 wait complete. 2023-07-21T19:05:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.028+0000 7f4e6d5c8b80 1 -- start start 2023-07-21T19:05:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.028+0000 7f4e6d5c8b80 1 Processor -- start 2023-07-21T19:05:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.028+0000 7f4e6d5c8b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd598936b0 0x55dd59a409b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.028+0000 7f4e6d5c8b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55dd59a37580 0x55dd59a40ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.028+0000 7f4e6d5c8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd59a3d2d0 0x55dd59a41430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.028+0000 7f4e6d5c8b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd59928a60 con 0x55dd59a37580 2023-07-21T19:05:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.028+0000 7f4e6d5c8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd59928760 con 0x55dd59a3d2d0 2023-07-21T19:05:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.028+0000 7f4e6d5c8b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd5980c8c0 con 0x55dd598936b0 2023-07-21T19:05:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.028+0000 7f4e6d5c8b80 10 client.0 _ll_register_callbacks cb 0x55dd59926700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:05:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.028+0000 7f4e5affd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd598936b0 0x55dd59a409b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.028+0000 7f4e5affd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd598936b0 0x55dd59a409b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:53418/0 (socket says 172.21.15.39:53418) 2023-07-21T19:05:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.028+0000 7f4e5affd700 1 -- 172.21.15.39:0/688556383 learned_addr learned my addr 172.21.15.39:0/688556383 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:05:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.029+0000 7f4e5affd700 1 -- 172.21.15.39:0/688556383 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd59a3d2d0 msgr2=0x55dd59a41430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:05:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.029+0000 7f4e5affd700 1 --2- 172.21.15.39:0/688556383 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd59a3d2d0 0x55dd59a41430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.029+0000 7f4e5affd700 1 -- 172.21.15.39:0/688556383 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55dd59a37580 msgr2=0x55dd59a40ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:05:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.029+0000 7f4e5affd700 1 --2- 172.21.15.39:0/688556383 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55dd59a37580 0x55dd59a40ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.029+0000 7f4e5affd700 1 -- 172.21.15.39:0/688556383 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e54011330 con 0x55dd598936b0 2023-07-21T19:05:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.032+0000 7f4e5affd700 1 --2- 172.21.15.39:0/688556383 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd598936b0 0x55dd59a409b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c0ffba0 tx=0x7f4e50009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2464c5821635d6c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.032+0000 7f4e3eff5700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2023-07-21T19:05:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.032+0000 7f4e3eff5700 1 -- 172.21.15.39:0/688556383 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5002b070 con 0x55dd598936b0 2023-07-21T19:05:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.032+0000 7f4e3eff5700 1 -- 172.21.15.39:0/688556383 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f4e50004bf0 con 0x55dd598936b0 2023-07-21T19:05:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.032+0000 7f4e6d5c8b80 1 -- 172.21.15.39:0/688556383 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dd598e2490 con 0x55dd598936b0 2023-07-21T19:05:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.032+0000 7f4e3eff5700 1 -- 172.21.15.39:0/688556383 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e50004db0 con 0x55dd598936b0 2023-07-21T19:05:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.032+0000 7f4e6d5c8b80 10 client.27343 fetch_fsmap learned FSMap version 105 2023-07-21T19:05:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.032+0000 7f4e6d5c8b80 1 -- 172.21.15.39:0/688556383 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=4+}) v3 -- 0x7f4e5c1a4e40 con 0x55dd598936b0 2023-07-21T19:05:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.033+0000 7f4e3eff5700 1 -- 172.21.15.39:0/688556383 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e5003c020 con 0x55dd598936b0 2023-07-21T19:05:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.033+0000 7f4e3eff5700 1 -- 172.21.15.39:0/688556383 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 105) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4e5000f580 con 0x55dd598936b0 2023-07-21T19:05:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.033+0000 7f4e6d5c8b80 10 client.27343 fetch_fsmap finished waiting for FSMap version 105 2023-07-21T19:05:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.033+0000 7f4e6d5c8b80 10 client.27343 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:05:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.033+0000 7f4e6d5c8b80 20 client.27343 populate_metadata read hostname 'smithi039' 2023-07-21T19:05:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.033+0000 7f4e6d5c8b80 1 --2- 172.21.15.39:0/688556383 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55dd59a59440 0x55dd59a5b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.033+0000 7f4e6d5c8b80 4 client.27343 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:05:16.049 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.033+0000 7f4e6d5c8b80 1 -- 172.21.15.39:0/688556383 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dd59981e50 con 0x55dd59a59440 2023-07-21T19:05:16.049 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.034+0000 7f4e5a7fc700 1 --2- 172.21.15.39:0/688556383 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55dd59a59440 0x55dd59a5b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:16.049 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.034+0000 7f4e5a7fc700 1 --2- 172.21.15.39:0/688556383 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55dd59a59440 0x55dd59a5b870 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4d156aa43dec164 server_cookie=ecebd44e29d30db4 in_seq=0 out_seq=0 2023-07-21T19:05:16.049 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.034+0000 7f4e3eff5700 10 client.27343 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T19:05:16.049 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.036+0000 7f4e3eff5700 1 -- 172.21.15.39:0/688556383 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55dd59981e50 con 0x55dd59a59440 2023-07-21T19:05:16.049 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.036+0000 7f4e3eff5700 10 client.27343 handle_command_reply: tid=0 2023-07-21T19:05:16.049 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.037+0000 7f4e6d5c8b80 1 client.27343 shutdown 2023-07-21T19:05:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.037+0000 7f4e6d5c8b80 1 -- 172.21.15.39:0/688556383 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd598936b0 msgr2=0x55dd59a409b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:16.051 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.037+0000 7f4e6d5c8b80 1 --2- 172.21.15.39:0/688556383 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd598936b0 0x55dd59a409b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c0ffba0 tx=0x7f4e50009310 comp rx=0 tx=0).stop 2023-07-21T19:05:16.051 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.037+0000 7f4e6d5c8b80 1 -- 172.21.15.39:0/688556383 shutdown_connections 2023-07-21T19:05:16.051 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.037+0000 7f4e6d5c8b80 1 --2- 172.21.15.39:0/688556383 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55dd59a59440 0x55dd59a5b870 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:16.051 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.037+0000 7f4e6d5c8b80 1 --2- 172.21.15.39:0/688556383 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd59a3d2d0 0x55dd59a41430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:16.051 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.037+0000 7f4e6d5c8b80 1 --2- 172.21.15.39:0/688556383 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd598936b0 0x55dd59a409b0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:16.051 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.037+0000 7f4e3eff5700 0 client.27343 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T19:05:16.051 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.037+0000 7f4e6d5c8b80 1 --2- 172.21.15.39:0/688556383 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55dd59a37580 0x55dd59a40ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:16.052 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.037+0000 7f4e6d5c8b80 1 -- 172.21.15.39:0/688556383 >> 172.21.15.39:0/688556383 conn(0x55dd59a2cbe0 msgr2=0x55dd59a314e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:16.052 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.037+0000 7f4e6d5c8b80 1 -- 172.21.15.39:0/688556383 shutdown_connections 2023-07-21T19:05:16.052 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.038+0000 7f4e6d5c8b80 1 -- 172.21.15.39:0/688556383 wait complete. 2023-07-21T19:05:16.052 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.038+0000 7f4e6d5c8b80 20 client.27343 trim_cache size 0 max 16384 2023-07-21T19:05:16.074 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.063+0000 7f4e6d5c8b80 1 Processor -- start 2023-07-21T19:05:16.074 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.063+0000 7f4e6d5c8b80 1 -- start start 2023-07-21T19:05:16.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.064+0000 7f4e6d5c8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd598936b0 0x55dd59a43150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:16.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.064+0000 7f4e6d5c8b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55dd59a37580 0x55dd59a37040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:16.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.064+0000 7f4e6d5c8b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd59a3d2d0 0x55dd59a47c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:16.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.064+0000 7f4e6d5c8b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd59a29be0 con 0x55dd59a37580 2023-07-21T19:05:16.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.064+0000 7f4e6d5c8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd59a2e5f0 con 0x55dd598936b0 2023-07-21T19:05:16.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.064+0000 7f4e6d5c8b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd59a37fd0 con 0x55dd59a3d2d0 2023-07-21T19:05:16.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.064+0000 7f4e5a7fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55dd59a37580 0x55dd59a37040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:16.076 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.064+0000 7f4e5a7fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55dd59a37580 0x55dd59a37040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34876/0 (socket says 172.21.15.39:34876) 2023-07-21T19:05:16.076 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.064+0000 7f4e5a7fc700 1 -- 172.21.15.39:0/2390119042 learned_addr learned my addr 172.21.15.39:0/2390119042 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:05:16.076 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.064+0000 7f4e5affd700 1 --2- 172.21.15.39:0/2390119042 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd598936b0 0x55dd59a43150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:16.076 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.064+0000 7f4e5b7fe700 1 --2- 172.21.15.39:0/2390119042 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd59a3d2d0 0x55dd59a47c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:16.076 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.064+0000 7f4e5a7fc700 1 -- 172.21.15.39:0/2390119042 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd59a3d2d0 msgr2=0x55dd59a47c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:05:16.076 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.064+0000 7f4e5a7fc700 1 --2- 172.21.15.39:0/2390119042 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd59a3d2d0 0x55dd59a47c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:16.076 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.064+0000 7f4e5a7fc700 1 -- 172.21.15.39:0/2390119042 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd598936b0 msgr2=0x55dd59a43150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:05:16.077 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.064+0000 7f4e5a7fc700 1 --2- 172.21.15.39:0/2390119042 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd598936b0 0x55dd59a43150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:16.077 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.064+0000 7f4e5a7fc700 1 -- 172.21.15.39:0/2390119042 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd59a604e0 con 0x55dd59a37580 2023-07-21T19:05:16.077 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.064+0000 7f4e5a7fc700 1 --2- 172.21.15.39:0/2390119042 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55dd59a37580 0x55dd59a37040 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c00d950 tx=0x7f4e4c099610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7b678ec7aa3ca17 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:16.077 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.065+0000 7f4e41ffb700 1 -- 172.21.15.39:0/2390119042 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c0b6070 con 0x55dd59a37580 2023-07-21T19:05:16.077 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.065+0000 7f4e41ffb700 1 -- 172.21.15.39:0/2390119042 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e4c0b2230 con 0x55dd59a37580 2023-07-21T19:05:16.077 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.065+0000 7f4e41ffb700 1 -- 172.21.15.39:0/2390119042 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c099220 con 0x55dd59a37580 2023-07-21T19:05:16.080 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.065+0000 7f4e6d5c8b80 1 -- 172.21.15.39:0/2390119042 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55dd59a37580 msgr2=0x55dd59a37040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:16.080 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.065+0000 7f4e6d5c8b80 1 --2- 172.21.15.39:0/2390119042 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55dd59a37580 0x55dd59a37040 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c00d950 tx=0x7f4e4c099610 comp rx=0 tx=0).stop 2023-07-21T19:05:16.080 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.065+0000 7f4e6d5c8b80 1 -- 172.21.15.39:0/2390119042 shutdown_connections 2023-07-21T19:05:16.081 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.065+0000 7f4e6d5c8b80 1 --2- 172.21.15.39:0/2390119042 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd598936b0 0x55dd59a43150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:16.081 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.065+0000 7f4e6d5c8b80 1 --2- 172.21.15.39:0/2390119042 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd59a3d2d0 0x55dd59a47c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:16.081 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.065+0000 7f4e6d5c8b80 1 --2- 172.21.15.39:0/2390119042 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55dd59a37580 0x55dd59a37040 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:16.081 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.065+0000 7f4e6d5c8b80 1 -- 172.21.15.39:0/2390119042 >> 172.21.15.39:0/2390119042 conn(0x55dd59a2cbe0 msgr2=0x55dd59a3acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:16.081 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.066+0000 7f4e6d5c8b80 1 -- 172.21.15.39:0/2390119042 shutdown_connections 2023-07-21T19:05:16.081 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.066+0000 7f4e6d5c8b80 1 -- 172.21.15.39:0/2390119042 wait complete. 2023-07-21T19:05:16.081 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.066+0000 7f4e6d5c8b80 1 -- start start 2023-07-21T19:05:16.082 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.066+0000 7f4e6d5c8b80 1 Processor -- start 2023-07-21T19:05:16.082 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.066+0000 7f4e6d5c8b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55dd598936b0 0x55dd59a43c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:16.082 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.066+0000 7f4e6d5c8b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd59a37580 0x55dd59a44180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:16.082 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.066+0000 7f4e6d5c8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd59a3d2d0 0x55dd59a446c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:16.082 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.066+0000 7f4e6d5c8b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd59aac350 con 0x55dd598936b0 2023-07-21T19:05:16.082 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.066+0000 7f4e6d5c8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd59aac490 con 0x55dd59a3d2d0 2023-07-21T19:05:16.082 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.066+0000 7f4e6d5c8b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd59aac600 con 0x55dd59a37580 2023-07-21T19:05:16.082 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.066+0000 7f4e6d5c8b80 10 client.0 _ll_register_callbacks cb 0x7f4e50005350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:05:16.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.066+0000 7f4e5affd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55dd598936b0 0x55dd59a43c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:16.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.066+0000 7f4e5affd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55dd598936b0 0x55dd59a43c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34880/0 (socket says 172.21.15.39:34880) 2023-07-21T19:05:16.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.066+0000 7f4e5affd700 1 -- 172.21.15.39:0/1128839600 learned_addr learned my addr 172.21.15.39:0/1128839600 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:05:16.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.067+0000 7f4e5a7fc700 1 --2- 172.21.15.39:0/1128839600 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd59a37580 0x55dd59a44180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:16.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.067+0000 7f4e5b7fe700 1 --2- 172.21.15.39:0/1128839600 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd59a3d2d0 0x55dd59a446c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:16.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.067+0000 7f4e5affd700 1 -- 172.21.15.39:0/1128839600 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd59a37580 msgr2=0x55dd59a44180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:05:16.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.067+0000 7f4e5affd700 1 --2- 172.21.15.39:0/1128839600 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd59a37580 0x55dd59a44180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:16.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.067+0000 7f4e5affd700 1 -- 172.21.15.39:0/1128839600 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd59a3d2d0 msgr2=0x55dd59a446c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:05:16.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.067+0000 7f4e5affd700 1 --2- 172.21.15.39:0/1128839600 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd59a3d2d0 0x55dd59a446c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:16.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.067+0000 7f4e5affd700 1 -- 172.21.15.39:0/1128839600 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dd597b6d80 con 0x55dd598936b0 2023-07-21T19:05:16.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.067+0000 7f4e5affd700 1 --2- 172.21.15.39:0/1128839600 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55dd598936b0 0x55dd59a43c40 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f4e500095c0 tx=0x7f4e5000f7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34f7483f59741a7c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:16.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.067+0000 7f4e3eff5700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T19:05:16.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.067+0000 7f4e3eff5700 1 -- 172.21.15.39:0/1128839600 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5002b070 con 0x55dd598936b0 2023-07-21T19:05:16.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.067+0000 7f4e3eff5700 1 -- 172.21.15.39:0/1128839600 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f4e5003d080 con 0x55dd598936b0 2023-07-21T19:05:16.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.067+0000 7f4e6d5c8b80 1 -- 172.21.15.39:0/1128839600 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4e4c004380 con 0x55dd598936b0 2023-07-21T19:05:16.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.067+0000 7f4e3eff5700 1 -- 172.21.15.39:0/1128839600 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e50038660 con 0x55dd598936b0 2023-07-21T19:05:16.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.067+0000 7f4e6d5c8b80 10 client.17838 fetch_fsmap learned FSMap version 105 2023-07-21T19:05:16.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.067+0000 7f4e6d5c8b80 1 -- 172.21.15.39:0/1128839600 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=4+}) v3 -- 0x55dd5979fc00 con 0x55dd598936b0 2023-07-21T19:05:16.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.068+0000 7f4e3eff5700 1 -- 172.21.15.39:0/1128839600 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e4c004380 con 0x55dd598936b0 2023-07-21T19:05:16.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.068+0000 7f4e3eff5700 1 -- 172.21.15.39:0/1128839600 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 105) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4e500248f0 con 0x55dd598936b0 2023-07-21T19:05:16.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.068+0000 7f4e6d5c8b80 10 client.17838 fetch_fsmap finished waiting for FSMap version 105 2023-07-21T19:05:16.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.068+0000 7f4e6d5c8b80 10 client.17838 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:05:16.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.068+0000 7f4e6d5c8b80 20 client.17838 populate_metadata read hostname 'smithi039' 2023-07-21T19:05:16.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.068+0000 7f4e6d5c8b80 1 --2- 172.21.15.39:0/1128839600 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55dd59aad860 0x55dd59aafc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:16.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.068+0000 7f4e6d5c8b80 4 client.17838 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:05:16.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.068+0000 7f4e6d5c8b80 1 -- 172.21.15.39:0/1128839600 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55dd59943ff0 con 0x55dd59aad860 2023-07-21T19:05:16.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.068+0000 7f4e5a7fc700 1 --2- 172.21.15.39:0/1128839600 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55dd59aad860 0x55dd59aafc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:16.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.068+0000 7f4e5a7fc700 1 --2- 172.21.15.39:0/1128839600 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55dd59aad860 0x55dd59aafc40 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed75bf7165043105 server_cookie=51446d1d019ebb57 in_seq=0 out_seq=0 2023-07-21T19:05:16.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.068+0000 7f4e3eff5700 10 client.17838 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T19:05:16.319 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.309+0000 7f4e3eff5700 1 -- 172.21.15.39:0/1128839600 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55dd59943ff0 con 0x55dd59aad860 2023-07-21T19:05:16.319 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.309+0000 7f4e3eff5700 10 client.17838 handle_command_reply: tid=0 2023-07-21T19:05:16.319 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.309+0000 7f4e6d5c8b80 1 client.17838 shutdown 2023-07-21T19:05:16.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.309+0000 7f4e6d5c8b80 1 -- 172.21.15.39:0/1128839600 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55dd598936b0 msgr2=0x55dd59a43c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:16.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.309+0000 7f4e6d5c8b80 1 --2- 172.21.15.39:0/1128839600 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55dd598936b0 0x55dd59a43c40 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f4e500095c0 tx=0x7f4e5000f7d0 comp rx=0 tx=0).stop 2023-07-21T19:05:16.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.309+0000 7f4e6d5c8b80 1 -- 172.21.15.39:0/1128839600 shutdown_connections 2023-07-21T19:05:16.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.309+0000 7f4e6d5c8b80 1 --2- 172.21.15.39:0/1128839600 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55dd59aad860 0x55dd59aafc40 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:16.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.309+0000 7f4e6d5c8b80 1 --2- 172.21.15.39:0/1128839600 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd59a3d2d0 0x55dd59a446c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:16.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.309+0000 7f4e3eff5700 0 client.17838 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T19:05:16.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.309+0000 7f4e6d5c8b80 1 --2- 172.21.15.39:0/1128839600 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55dd598936b0 0x55dd59a43c40 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:16.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.309+0000 7f4e6d5c8b80 1 --2- 172.21.15.39:0/1128839600 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd59a37580 0x55dd59a44180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:16.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.309+0000 7f4e6d5c8b80 1 -- 172.21.15.39:0/1128839600 >> 172.21.15.39:0/1128839600 conn(0x55dd59a2cbe0 msgr2=0x55dd59a40f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:16.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.309+0000 7f4e6d5c8b80 1 -- 172.21.15.39:0/1128839600 shutdown_connections 2023-07-21T19:05:16.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.309+0000 7f4e6d5c8b80 1 -- 172.21.15.39:0/1128839600 wait complete. 2023-07-21T19:05:16.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.309+0000 7f4e6d5c8b80 20 client.17838 trim_cache size 0 max 16384 2023-07-21T19:05:16.322 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T19:05:16.322 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T19:05:16.322 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T19:05:16.322 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T19:05:16.323 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.310+0000 7f4e61442700 1 -- 172.21.15.39:0/1737578818 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f4e440a1340 msgr2=0x7f4e440a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:16.323 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.310+0000 7f4e61442700 1 --2- 172.21.15.39:0/1737578818 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f4e440a1340 0x7f4e440a37f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4e5000b2e0 tx=0x7f4e50005fd0 comp rx=0 tx=0).stop 2023-07-21T19:05:16.323 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.310+0000 7f4e61442700 1 -- 172.21.15.39:0/1737578818 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e5c0ff3e0 msgr2=0x7f4e5c198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:16.323 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.310+0000 7f4e61442700 1 --2- 172.21.15.39:0/1737578818 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e5c0ff3e0 0x7f4e5c198340 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c00d980 tx=0x7f4e4c00dc90 comp rx=0 tx=0).stop 2023-07-21T19:05:16.323 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.310+0000 7f4e61442700 1 -- 172.21.15.39:0/1737578818 shutdown_connections 2023-07-21T19:05:16.324 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.310+0000 7f4e61442700 1 --2- 172.21.15.39:0/1737578818 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f4e440a1340 0x7f4e440a37f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:16.324 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.310+0000 7f4e61442700 1 --2- 172.21.15.39:0/1737578818 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e5c0fea30 0x7f4e5c197e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:16.324 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.310+0000 7f4e61442700 1 --2- 172.21.15.39:0/1737578818 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e5c104a30 0x7f4e5c19c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:16.324 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.310+0000 7f4e61442700 1 --2- 172.21.15.39:0/1737578818 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e5c0ff3e0 0x7f4e5c198340 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:16.324 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.310+0000 7f4e61442700 1 -- 172.21.15.39:0/1737578818 >> 172.21.15.39:0/1737578818 conn(0x7f4e5c074760 msgr2=0x7f4e5c102570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:16.324 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.310+0000 7f4e61442700 1 -- 172.21.15.39:0/1737578818 shutdown_connections 2023-07-21T19:05:16.324 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:16.310+0000 7f4e61442700 1 -- 172.21.15.39:0/1737578818 wait complete. 2023-07-21T19:05:16.724 INFO:tasks.workunit.client.0.smithi039.stdout: 10 331149 27.96 MB/sec execute 509 sec latency 227.914 ms 2023-07-21T19:05:16.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:05:16 smithi193 ceph-mon[98204]: pgmap v746: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 16 MiB/s wr, 602 op/s 2023-07-21T19:05:16.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:05:16 smithi155 ceph-mon[97934]: pgmap v746: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 16 MiB/s wr, 602 op/s 2023-07-21T19:05:16.926 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:05:16 smithi039 ceph-mon[96489]: pgmap v746: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 16 MiB/s wr, 602 op/s 2023-07-21T19:05:17.724 INFO:tasks.workunit.client.0.smithi039.stdout: 10 331547 27.95 MB/sec execute 510 sec latency 147.497 ms 2023-07-21T19:05:18.724 INFO:tasks.workunit.client.0.smithi039.stdout: 10 332077 27.95 MB/sec execute 511 sec latency 190.603 ms 2023-07-21T19:05:18.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:05:18 smithi193 ceph-mon[98204]: pgmap v747: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 20 MiB/s wr, 779 op/s 2023-07-21T19:05:18.878 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:05:18 smithi155 ceph-mon[97934]: pgmap v747: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 20 MiB/s wr, 779 op/s 2023-07-21T19:05:18.926 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:05:18 smithi039 ceph-mon[96489]: pgmap v747: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 20 MiB/s wr, 779 op/s 2023-07-21T19:05:19.724 INFO:tasks.workunit.client.0.smithi039.stdout: 10 332596 27.95 MB/sec execute 512 sec latency 71.778 ms 2023-07-21T19:05:20.730 INFO:tasks.workunit.client.0.smithi039.stdout: 10 333261 27.94 MB/sec execute 513 sec latency 101.815 ms 2023-07-21T19:05:20.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:05:20 smithi193 ceph-mon[98204]: pgmap v748: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.9 MiB/s wr, 373 op/s 2023-07-21T19:05:20.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:05:20 smithi155 ceph-mon[97934]: pgmap v748: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.9 MiB/s wr, 373 op/s 2023-07-21T19:05:20.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:05:20 smithi039 ceph-mon[96489]: pgmap v748: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.9 MiB/s wr, 373 op/s 2023-07-21T19:05:21.724 INFO:tasks.workunit.client.0.smithi039.stdout: 10 333938 27.97 MB/sec execute 514 sec latency 68.548 ms 2023-07-21T19:05:22.725 INFO:tasks.workunit.client.0.smithi039.stdout: 10 334559 27.99 MB/sec execute 515 sec latency 72.314 ms 2023-07-21T19:05:22.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:05:22 smithi193 ceph-mon[98204]: pgmap v749: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.0 MiB/s wr, 389 op/s 2023-07-21T19:05:22.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:05:22 smithi155 ceph-mon[97934]: pgmap v749: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.0 MiB/s wr, 389 op/s 2023-07-21T19:05:22.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:05:22 smithi039 ceph-mon[96489]: pgmap v749: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.0 MiB/s wr, 389 op/s 2023-07-21T19:05:23.725 INFO:tasks.workunit.client.0.smithi039.stdout: 10 334987 27.99 MB/sec execute 516 sec latency 121.230 ms 2023-07-21T19:05:24.724 INFO:tasks.workunit.client.0.smithi039.stdout: 10 335400 27.98 MB/sec execute 517 sec latency 85.178 ms 2023-07-21T19:05:24.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:05:24 smithi193 ceph-mon[98204]: pgmap v750: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 10 MiB/s wr, 554 op/s 2023-07-21T19:05:24.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:05:24 smithi155 ceph-mon[97934]: pgmap v750: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 10 MiB/s wr, 554 op/s 2023-07-21T19:05:24.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:05:24 smithi039 ceph-mon[96489]: pgmap v750: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 10 MiB/s wr, 554 op/s 2023-07-21T19:05:25.724 INFO:tasks.workunit.client.0.smithi039.stdout: 10 335769 27.95 MB/sec execute 518 sec latency 105.257 ms 2023-07-21T19:05:26.724 INFO:tasks.workunit.client.0.smithi039.stdout: 10 336255 27.94 MB/sec execute 519 sec latency 130.972 ms 2023-07-21T19:05:26.861 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:05:26 smithi193 ceph-mon[98204]: pgmap v751: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.4 MiB/s wr, 401 op/s 2023-07-21T19:05:26.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:05:26 smithi155 ceph-mon[97934]: pgmap v751: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.4 MiB/s wr, 401 op/s 2023-07-21T19:05:26.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:05:26 smithi039 ceph-mon[96489]: pgmap v751: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.4 MiB/s wr, 401 op/s 2023-07-21T19:05:27.732 INFO:tasks.workunit.client.0.smithi039.stdout: 10 336829 27.94 MB/sec execute 520 sec latency 78.605 ms 2023-07-21T19:05:28.724 INFO:tasks.workunit.client.0.smithi039.stdout: 10 337562 27.96 MB/sec execute 521 sec latency 85.473 ms 2023-07-21T19:05:28.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:05:28 smithi193 ceph-mon[98204]: pgmap v752: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 577 op/s 2023-07-21T19:05:28.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:05:28 smithi155 ceph-mon[97934]: pgmap v752: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 577 op/s 2023-07-21T19:05:28.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:05:28 smithi039 ceph-mon[96489]: pgmap v752: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 577 op/s 2023-07-21T19:05:29.499 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.487+0000 7f7accf66700 1 -- 172.21.15.39:0/2386742936 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ac8104a20 msgr2=0x7f7ac8104df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:29.499 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.487+0000 7f7accf66700 1 --2- 172.21.15.39:0/2386742936 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ac8104a20 0x7f7ac8104df0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7ab800b9b0 tx=0x7f7ab8009260 comp rx=0 tx=0).stop 2023-07-21T19:05:29.499 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.487+0000 7f7accf66700 1 -- 172.21.15.39:0/2386742936 shutdown_connections 2023-07-21T19:05:29.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.487+0000 7f7accf66700 1 --2- 172.21.15.39:0/2386742936 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ac80ff3c0 0x7f7ac810b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:29.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.487+0000 7f7accf66700 1 --2- 172.21.15.39:0/2386742936 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ac8104a20 0x7f7ac8104df0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:29.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.487+0000 7f7accf66700 1 --2- 172.21.15.39:0/2386742936 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ac80fea10 0x7f7ac80fee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:29.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.487+0000 7f7accf66700 1 -- 172.21.15.39:0/2386742936 >> 172.21.15.39:0/2386742936 conn(0x7f7ac80fa4d0 msgr2=0x7f7ac80fc8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:29.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.488+0000 7f7accf66700 1 -- 172.21.15.39:0/2386742936 shutdown_connections 2023-07-21T19:05:29.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.491+0000 7f7accf66700 1 -- 172.21.15.39:0/2386742936 wait complete. 2023-07-21T19:05:29.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.492+0000 7f7accf66700 1 Processor -- start 2023-07-21T19:05:29.503 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.492+0000 7f7accf66700 1 -- start start 2023-07-21T19:05:29.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.492+0000 7f7accf66700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ac80fea10 0x7f7ac8111da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:29.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.492+0000 7f7accf66700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ac80ff3c0 0x7f7ac8112300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:29.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.492+0000 7f7accf66700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ac8104a20 0x7f7ac810cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:29.505 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.492+0000 7f7accf66700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac810d420 con 0x7f7ac80fea10 2023-07-21T19:05:29.505 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.492+0000 7f7accf66700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac810d590 con 0x7f7ac80ff3c0 2023-07-21T19:05:29.505 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.492+0000 7f7accf66700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac810d750 con 0x7f7ac8104a20 2023-07-21T19:05:29.505 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.493+0000 7f7ac659c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ac80fea10 0x7f7ac8111da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:29.505 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.493+0000 7f7ac659c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ac80fea10 0x7f7ac8111da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34894/0 (socket says 172.21.15.39:34894) 2023-07-21T19:05:29.505 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.493+0000 7f7ac659c700 1 -- 172.21.15.39:0/4002399095 learned_addr learned my addr 172.21.15.39:0/4002399095 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:05:29.505 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.493+0000 7f7ac5d9b700 1 --2- 172.21.15.39:0/4002399095 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ac80ff3c0 0x7f7ac8112300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:29.506 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.493+0000 7f7ac6d9d700 1 --2- 172.21.15.39:0/4002399095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ac8104a20 0x7f7ac810cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:29.506 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.493+0000 7f7ac659c700 1 -- 172.21.15.39:0/4002399095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ac8104a20 msgr2=0x7f7ac810cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:29.506 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.493+0000 7f7ac659c700 1 --2- 172.21.15.39:0/4002399095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ac8104a20 0x7f7ac810cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:29.506 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.493+0000 7f7ac659c700 1 -- 172.21.15.39:0/4002399095 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ac80ff3c0 msgr2=0x7f7ac8112300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:29.506 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.493+0000 7f7ac659c700 1 --2- 172.21.15.39:0/4002399095 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ac80ff3c0 0x7f7ac8112300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:29.506 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.493+0000 7f7ac659c700 1 -- 172.21.15.39:0/4002399095 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ac000f0b0 con 0x7f7ac80fea10 2023-07-21T19:05:29.506 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.493+0000 7f7ac659c700 1 --2- 172.21.15.39:0/4002399095 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ac80fea10 0x7f7ac8111da0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8000c00 tx=0x7f7ab8003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:29.506 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.493+0000 7f7ab77fe700 1 -- 172.21.15.39:0/4002399095 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab8010040 con 0x7f7ac80fea10 2023-07-21T19:05:29.507 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.493+0000 7f7accf66700 1 -- 172.21.15.39:0/4002399095 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ab800b610 con 0x7f7ac80fea10 2023-07-21T19:05:29.507 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.494+0000 7f7ab77fe700 1 -- 172.21.15.39:0/4002399095 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ab80042a0 con 0x7f7ac80fea10 2023-07-21T19:05:29.507 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.494+0000 7f7ab77fe700 1 -- 172.21.15.39:0/4002399095 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab8017600 con 0x7f7ac80fea10 2023-07-21T19:05:29.507 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.494+0000 7f7accf66700 1 -- 172.21.15.39:0/4002399095 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ac8119d10 con 0x7f7ac80fea10 2023-07-21T19:05:29.507 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.497+0000 7f7ab77fe700 1 -- 172.21.15.39:0/4002399095 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f7ab8015580 con 0x7f7ac80fea10 2023-07-21T19:05:29.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.498+0000 7f7ab77fe700 1 --2- 172.21.15.39:0/4002399095 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f7ab00a12c0 0x7f7ab00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:29.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.498+0000 7f7ab77fe700 1 -- 172.21.15.39:0/4002399095 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f7ab80ce0d0 con 0x7f7ac80fea10 2023-07-21T19:05:29.510 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.499+0000 7f7ac5d9b700 1 --2- 172.21.15.39:0/4002399095 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f7ab00a12c0 0x7f7ab00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:29.513 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.503+0000 7f7ac5d9b700 1 --2- 172.21.15.39:0/4002399095 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f7ab00a12c0 0x7f7ab00a3770 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7ac810e630 tx=0x7f7abc009190 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:29.513 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.503+0000 7f7ad90ecb80 1 Processor -- start 2023-07-21T19:05:29.513 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.503+0000 7f7ad90ecb80 1 -- start start 2023-07-21T19:05:29.514 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.503+0000 7f7ad90ecb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cb53d016b0 0x55cb53eaad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:29.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.503+0000 7f7ad90ecb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cb53eab2d0 0x55cb53ea5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:29.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.503+0000 7f7ad90ecb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb53ea5580 0x55cb53ea59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:29.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.503+0000 7f7ad90ecb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb53daac50 con 0x55cb53ea5580 2023-07-21T19:05:29.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.503+0000 7f7ad90ecb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb53dfd9e0 con 0x55cb53d016b0 2023-07-21T19:05:29.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.503+0000 7f7ad90ecb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb53df6e70 con 0x55cb53eab2d0 2023-07-21T19:05:29.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.503+0000 7f7ac659c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cb53d016b0 0x55cb53eaad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:29.516 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.503+0000 7f7ac6d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb53ea5580 0x55cb53ea59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:29.516 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.503+0000 7f7ac659c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cb53d016b0 0x55cb53eaad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:47498/0 (socket says 172.21.15.39:47498) 2023-07-21T19:05:29.516 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.503+0000 7f7ac659c700 1 -- 172.21.15.39:0/2478917226 learned_addr learned my addr 172.21.15.39:0/2478917226 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:05:29.516 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.503+0000 7f7ac6d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb53ea5580 0x55cb53ea59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34904/0 (socket says 172.21.15.39:34904) 2023-07-21T19:05:29.516 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.504+0000 7f7ac659c700 1 -- 172.21.15.39:0/2478917226 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cb53eab2d0 msgr2=0x55cb53ea5040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T19:05:29.516 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.504+0000 7f7ac659c700 1 --2- 172.21.15.39:0/2478917226 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cb53eab2d0 0x55cb53ea5040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:29.516 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.504+0000 7f7ac659c700 1 -- 172.21.15.39:0/2478917226 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb53ea5580 msgr2=0x55cb53ea59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:05:29.517 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.504+0000 7f7ac659c700 1 --2- 172.21.15.39:0/2478917226 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb53ea5580 0x55cb53ea59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:29.517 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.504+0000 7f7ac659c700 1 -- 172.21.15.39:0/2478917226 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb53c24d80 con 0x55cb53d016b0 2023-07-21T19:05:29.517 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.504+0000 7f7ac659c700 1 --2- 172.21.15.39:0/2478917226 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cb53d016b0 0x55cb53eaad90 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7ab801efd0 tx=0x7f7ab8005a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84b14d9f15fb209d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:29.518 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.505+0000 7f7ab4ff9700 1 -- 172.21.15.39:0/2478917226 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab809b3e0 con 0x55cb53d016b0 2023-07-21T19:05:29.519 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.505+0000 7f7ab4ff9700 1 -- 172.21.15.39:0/2478917226 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ab809f410 con 0x55cb53d016b0 2023-07-21T19:05:29.519 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.505+0000 7f7ab4ff9700 1 -- 172.21.15.39:0/2478917226 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab8084be0 con 0x55cb53d016b0 2023-07-21T19:05:29.519 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.506+0000 7f7ad90ecb80 1 -- 172.21.15.39:0/2478917226 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cb53d016b0 msgr2=0x55cb53eaad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:29.519 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.506+0000 7f7ad90ecb80 1 --2- 172.21.15.39:0/2478917226 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cb53d016b0 0x55cb53eaad90 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7ab801efd0 tx=0x7f7ab8005a00 comp rx=0 tx=0).stop 2023-07-21T19:05:29.519 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.506+0000 7f7ad90ecb80 1 -- 172.21.15.39:0/2478917226 shutdown_connections 2023-07-21T19:05:29.519 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.506+0000 7f7ad90ecb80 1 --2- 172.21.15.39:0/2478917226 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cb53d016b0 0x55cb53eaad90 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:29.519 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.506+0000 7f7ad90ecb80 1 --2- 172.21.15.39:0/2478917226 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb53ea5580 0x55cb53ea59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:29.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.506+0000 7f7ad90ecb80 1 --2- 172.21.15.39:0/2478917226 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cb53eab2d0 0x55cb53ea5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:29.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.506+0000 7f7ad90ecb80 1 -- 172.21.15.39:0/2478917226 >> 172.21.15.39:0/2478917226 conn(0x55cb53e9abe0 msgr2=0x55cb53e9ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:29.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.507+0000 7f7ad90ecb80 1 -- 172.21.15.39:0/2478917226 shutdown_connections 2023-07-21T19:05:29.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.507+0000 7f7ad90ecb80 1 -- 172.21.15.39:0/2478917226 wait complete. 2023-07-21T19:05:29.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.507+0000 7f7ad90ecb80 1 -- start start 2023-07-21T19:05:29.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.507+0000 7f7ad90ecb80 1 Processor -- start 2023-07-21T19:05:29.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.507+0000 7f7ad90ecb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb53d016b0 0x55cb53eb44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:29.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.508+0000 7f7ad90ecb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cb53ea5580 0x55cb53eb4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:29.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.508+0000 7f7ad90ecb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cb53eae740 0x55cb53eaebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:29.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.508+0000 7f7ad90ecb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb53d96a60 con 0x55cb53d016b0 2023-07-21T19:05:29.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.508+0000 7f7ad90ecb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb53d96760 con 0x55cb53eae740 2023-07-21T19:05:29.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.508+0000 7f7ad90ecb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb53c7a8c0 con 0x55cb53ea5580 2023-07-21T19:05:29.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.508+0000 7f7ad90ecb80 10 client.0 _ll_register_callbacks cb 0x55cb53d94700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:05:29.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.508+0000 7f7ac659c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb53d016b0 0x55cb53eb44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:29.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.508+0000 7f7ac5d9b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cb53ea5580 0x55cb53eb4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:29.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.508+0000 7f7ac6d9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cb53eae740 0x55cb53eaebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:29.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.508+0000 7f7ac659c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb53d016b0 0x55cb53eb44f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34906/0 (socket says 172.21.15.39:34906) 2023-07-21T19:05:29.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.508+0000 7f7ac6d9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cb53eae740 0x55cb53eaebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:47506/0 (socket says 172.21.15.39:47506) 2023-07-21T19:05:29.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.508+0000 7f7ac659c700 1 -- 172.21.15.39:0/1482826244 learned_addr learned my addr 172.21.15.39:0/1482826244 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:05:29.524 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.508+0000 7f7ac6d9d700 1 -- 172.21.15.39:0/1482826244 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cb53ea5580 msgr2=0x55cb53eb4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:05:29.524 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.508+0000 7f7ac6d9d700 1 --2- 172.21.15.39:0/1482826244 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cb53ea5580 0x55cb53eb4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:29.525 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.508+0000 7f7ac6d9d700 1 -- 172.21.15.39:0/1482826244 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb53d016b0 msgr2=0x55cb53eb44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:05:29.525 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.508+0000 7f7ac6d9d700 1 --2- 172.21.15.39:0/1482826244 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb53d016b0 0x55cb53eb44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:29.525 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.508+0000 7f7ac6d9d700 1 -- 172.21.15.39:0/1482826244 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ac8119d10 con 0x55cb53eae740 2023-07-21T19:05:29.525 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.508+0000 7f7ac659c700 1 --2- 172.21.15.39:0/1482826244 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb53d016b0 0x55cb53eb44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:05:29.525 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.509+0000 7f7ac6d9d700 1 --2- 172.21.15.39:0/1482826244 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cb53eae740 0x55cb53eaebb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7ac000dfd0 tx=0x7f7ac000d320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1efebc27502574 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:29.525 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.509+0000 7f7aadffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-21T19:05:29.526 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.509+0000 7f7aadffb700 1 -- 172.21.15.39:0/1482826244 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac0007a20 con 0x55cb53eae740 2023-07-21T19:05:29.526 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.509+0000 7f7aadffb700 1 -- 172.21.15.39:0/1482826244 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f7ac0007b80 con 0x55cb53eae740 2023-07-21T19:05:29.526 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.509+0000 7f7ad90ecb80 1 -- 172.21.15.39:0/1482826244 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cb53d50490 con 0x55cb53eae740 2023-07-21T19:05:29.526 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.510+0000 7f7aadffb700 1 -- 172.21.15.39:0/1482826244 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ac000d800 con 0x55cb53eae740 2023-07-21T19:05:29.526 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.510+0000 7f7ad90ecb80 10 client.27281 fetch_fsmap learned FSMap version 105 2023-07-21T19:05:29.526 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.510+0000 7f7ad90ecb80 1 -- 172.21.15.39:0/1482826244 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=4+}) v3 -- 0x55cb53c0dc00 con 0x55cb53eae740 2023-07-21T19:05:29.526 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.510+0000 7f7aadffb700 1 -- 172.21.15.39:0/1482826244 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ac0026020 con 0x55cb53eae740 2023-07-21T19:05:29.526 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.510+0000 7f7aadffb700 1 -- 172.21.15.39:0/1482826244 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 105) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7ac00213b0 con 0x55cb53eae740 2023-07-21T19:05:29.527 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.510+0000 7f7ad90ecb80 10 client.27281 fetch_fsmap finished waiting for FSMap version 105 2023-07-21T19:05:29.527 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.510+0000 7f7ad90ecb80 10 client.27281 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:05:29.527 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.510+0000 7f7ad90ecb80 20 client.27281 populate_metadata read hostname 'smithi039' 2023-07-21T19:05:29.527 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.510+0000 7f7ad90ecb80 1 --2- 172.21.15.39:0/1482826244 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55cb53ebee50 0x55cb53ec1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:29.527 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.510+0000 7f7ad90ecb80 4 client.27281 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:05:29.527 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.510+0000 7f7ad90ecb80 1 -- 172.21.15.39:0/1482826244 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cb53defe50 con 0x55cb53ebee50 2023-07-21T19:05:29.527 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.510+0000 7f7ac659c700 1 --2- 172.21.15.39:0/1482826244 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55cb53ebee50 0x55cb53ec1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:29.528 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.511+0000 7f7aadffb700 10 client.27281 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T19:05:29.528 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.511+0000 7f7ac659c700 1 --2- 172.21.15.39:0/1482826244 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55cb53ebee50 0x55cb53ec1260 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c716cd6f37ba8bca server_cookie=b8fb7d6529dffac7 in_seq=0 out_seq=0 2023-07-21T19:05:29.528 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.512+0000 7f7aadffb700 1 -- 172.21.15.39:0/1482826244 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55cb53defe50 con 0x55cb53ebee50 2023-07-21T19:05:29.528 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.512+0000 7f7aadffb700 10 client.27281 handle_command_reply: tid=0 2023-07-21T19:05:29.528 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.512+0000 7f7ad90ecb80 1 client.27281 shutdown 2023-07-21T19:05:29.528 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.512+0000 7f7ad90ecb80 1 -- 172.21.15.39:0/1482826244 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cb53eae740 msgr2=0x55cb53eaebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:29.528 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.512+0000 7f7ad90ecb80 1 --2- 172.21.15.39:0/1482826244 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cb53eae740 0x55cb53eaebb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7ac000dfd0 tx=0x7f7ac000d320 comp rx=0 tx=0).stop 2023-07-21T19:05:29.529 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.512+0000 7f7ad90ecb80 1 -- 172.21.15.39:0/1482826244 shutdown_connections 2023-07-21T19:05:29.529 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.512+0000 7f7ad90ecb80 1 --2- 172.21.15.39:0/1482826244 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55cb53ebee50 0x55cb53ec1260 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:29.529 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.512+0000 7f7ad90ecb80 1 --2- 172.21.15.39:0/1482826244 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cb53eae740 0x55cb53eaebb0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:29.529 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.512+0000 7f7ad90ecb80 1 --2- 172.21.15.39:0/1482826244 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb53d016b0 0x55cb53eb44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:29.529 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.512+0000 7f7ad90ecb80 1 --2- 172.21.15.39:0/1482826244 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cb53ea5580 0x55cb53eb4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:29.529 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.512+0000 7f7ad90ecb80 1 -- 172.21.15.39:0/1482826244 >> 172.21.15.39:0/1482826244 conn(0x55cb53e9abe0 msgr2=0x55cb53e9f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:29.529 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.513+0000 7f7aadffb700 0 client.27281 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T19:05:29.529 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.513+0000 7f7ad90ecb80 1 -- 172.21.15.39:0/1482826244 shutdown_connections 2023-07-21T19:05:29.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.513+0000 7f7ad90ecb80 1 -- 172.21.15.39:0/1482826244 wait complete. 2023-07-21T19:05:29.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.513+0000 7f7ad90ecb80 20 client.27281 trim_cache size 0 max 16384 2023-07-21T19:05:29.545 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.534+0000 7f7ad90ecb80 1 Processor -- start 2023-07-21T19:05:29.545 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.534+0000 7f7ad90ecb80 1 -- start start 2023-07-21T19:05:29.545 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.535+0000 7f7ad90ecb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cb53d016b0 0x55cb53ea5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:29.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.535+0000 7f7ad90ecb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb53ea5580 0x55cb53eaa1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:29.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.535+0000 7f7ad90ecb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cb53eae740 0x55cb53e9fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:29.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.535+0000 7f7ad90ecb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb53eaa9e0 con 0x55cb53ea5580 2023-07-21T19:05:29.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.535+0000 7f7ad90ecb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb53ea0b40 con 0x55cb53eae740 2023-07-21T19:05:29.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.535+0000 7f7ad90ecb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb53e9c5f0 con 0x55cb53d016b0 2023-07-21T19:05:29.547 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.535+0000 7f7ac659c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cb53d016b0 0x55cb53ea5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:29.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.535+0000 7f7ac5d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb53ea5580 0x55cb53eaa1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:29.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.535+0000 7f7ac5d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb53ea5580 0x55cb53eaa1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34918/0 (socket says 172.21.15.39:34918) 2023-07-21T19:05:29.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.535+0000 7f7ac659c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cb53d016b0 0x55cb53ea5040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:53466/0 (socket says 172.21.15.39:53466) 2023-07-21T19:05:29.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.535+0000 7f7ac5d9b700 1 -- 172.21.15.39:0/3414617094 learned_addr learned my addr 172.21.15.39:0/3414617094 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:05:29.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.535+0000 7f7ac6d9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cb53eae740 0x55cb53e9fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:29.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.535+0000 7f7ac5d9b700 1 -- 172.21.15.39:0/3414617094 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cb53d016b0 msgr2=0x55cb53ea5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:05:29.549 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.535+0000 7f7ac5d9b700 1 --2- 172.21.15.39:0/3414617094 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cb53d016b0 0x55cb53ea5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:29.549 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.535+0000 7f7ac5d9b700 1 -- 172.21.15.39:0/3414617094 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cb53eae740 msgr2=0x55cb53e9fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:05:29.549 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.535+0000 7f7ac5d9b700 1 --2- 172.21.15.39:0/3414617094 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cb53eae740 0x55cb53e9fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:29.549 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.535+0000 7f7ac5d9b700 1 -- 172.21.15.39:0/3414617094 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb53eae2b0 con 0x55cb53ea5580 2023-07-21T19:05:29.549 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.536+0000 7f7ac5d9b700 1 --2- 172.21.15.39:0/3414617094 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb53ea5580 0x55cb53eaa1c0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f7ac80fde30 tx=0x7f7abc011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cbcc2c1c63bac56 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:29.549 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.536+0000 7f7ab57fa700 1 -- 172.21.15.39:0/3414617094 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7abc00ec50 con 0x55cb53ea5580 2023-07-21T19:05:29.549 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.536+0000 7f7ab57fa700 1 -- 172.21.15.39:0/3414617094 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7abc00edb0 con 0x55cb53ea5580 2023-07-21T19:05:29.549 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.536+0000 7f7ab57fa700 1 -- 172.21.15.39:0/3414617094 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7abc013870 con 0x55cb53ea5580 2023-07-21T19:05:29.550 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.537+0000 7f7ad90ecb80 1 -- 172.21.15.39:0/3414617094 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb53ea5580 msgr2=0x55cb53eaa1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:29.550 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.537+0000 7f7ad90ecb80 1 --2- 172.21.15.39:0/3414617094 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb53ea5580 0x55cb53eaa1c0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f7ac80fde30 tx=0x7f7abc011000 comp rx=0 tx=0).stop 2023-07-21T19:05:29.550 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.537+0000 7f7ad90ecb80 1 -- 172.21.15.39:0/3414617094 shutdown_connections 2023-07-21T19:05:29.550 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.537+0000 7f7ad90ecb80 1 --2- 172.21.15.39:0/3414617094 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cb53eae740 0x55cb53e9fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:29.550 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.537+0000 7f7ad90ecb80 1 --2- 172.21.15.39:0/3414617094 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cb53d016b0 0x55cb53ea5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:29.550 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.537+0000 7f7ad90ecb80 1 --2- 172.21.15.39:0/3414617094 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb53ea5580 0x55cb53eaa1c0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:29.551 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.537+0000 7f7ad90ecb80 1 -- 172.21.15.39:0/3414617094 >> 172.21.15.39:0/3414617094 conn(0x55cb53e9abe0 msgr2=0x55cb53ea9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:29.551 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.537+0000 7f7ad90ecb80 1 -- 172.21.15.39:0/3414617094 shutdown_connections 2023-07-21T19:05:29.551 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.537+0000 7f7ad90ecb80 1 -- 172.21.15.39:0/3414617094 wait complete. 2023-07-21T19:05:29.551 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.537+0000 7f7ad90ecb80 1 -- start start 2023-07-21T19:05:29.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.537+0000 7f7ad90ecb80 1 Processor -- start 2023-07-21T19:05:29.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.538+0000 7f7ad90ecb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb53d016b0 0x55cb53f06c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:29.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.538+0000 7f7ad90ecb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cb53ea5580 0x55cb53f071a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:29.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.538+0000 7f7ad90ecb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cb53eae740 0x55cb53f076e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:29.554 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.538+0000 7f7ad90ecb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb53f11e20 con 0x55cb53d016b0 2023-07-21T19:05:29.554 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.538+0000 7f7ad90ecb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb53f11f90 con 0x55cb53ea5580 2023-07-21T19:05:29.554 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.538+0000 7f7ad90ecb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb53f12150 con 0x55cb53eae740 2023-07-21T19:05:29.554 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.538+0000 7f7ad90ecb80 10 client.0 _ll_register_callbacks cb 0x55cb53d94700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:05:29.554 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.538+0000 7f7ac659c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb53d016b0 0x55cb53f06c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:29.554 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.538+0000 7f7ac659c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb53d016b0 0x55cb53f06c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34920/0 (socket says 172.21.15.39:34920) 2023-07-21T19:05:29.554 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.538+0000 7f7ac659c700 1 -- 172.21.15.39:0/419405289 learned_addr learned my addr 172.21.15.39:0/419405289 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:05:29.555 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.538+0000 7f7ac6d9d700 1 --2- 172.21.15.39:0/419405289 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cb53eae740 0x55cb53f076e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:29.555 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.538+0000 7f7ac5d9b700 1 --2- 172.21.15.39:0/419405289 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cb53ea5580 0x55cb53f071a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:29.555 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.538+0000 7f7ac659c700 1 -- 172.21.15.39:0/419405289 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cb53eae740 msgr2=0x55cb53f076e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:05:29.555 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.538+0000 7f7ac659c700 1 --2- 172.21.15.39:0/419405289 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cb53eae740 0x55cb53f076e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:29.555 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.538+0000 7f7ac659c700 1 -- 172.21.15.39:0/419405289 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cb53ea5580 msgr2=0x55cb53f071a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:05:29.555 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.538+0000 7f7ac659c700 1 --2- 172.21.15.39:0/419405289 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cb53ea5580 0x55cb53f071a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:29.555 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.538+0000 7f7ac659c700 1 -- 172.21.15.39:0/419405289 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7abc01a040 con 0x55cb53d016b0 2023-07-21T19:05:29.555 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.539+0000 7f7ac659c700 1 --2- 172.21.15.39:0/419405289 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb53d016b0 0x55cb53f06c60 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8006010 tx=0x7f7ab8005ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27d8bcc86d41d7a1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:29.556 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.539+0000 7f7aadffb700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T19:05:29.556 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.539+0000 7f7aadffb700 1 -- 172.21.15.39:0/419405289 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab8097af0 con 0x55cb53d016b0 2023-07-21T19:05:29.556 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.539+0000 7f7aadffb700 1 -- 172.21.15.39:0/419405289 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f7ab8097c50 con 0x55cb53d016b0 2023-07-21T19:05:29.556 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.539+0000 7f7ad90ecb80 1 -- 172.21.15.39:0/419405289 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7ab809f410 con 0x55cb53d016b0 2023-07-21T19:05:29.556 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.539+0000 7f7aadffb700 1 -- 172.21.15.39:0/419405289 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ab80a10d0 con 0x55cb53d016b0 2023-07-21T19:05:29.556 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.539+0000 7f7ad90ecb80 10 client.17856 fetch_fsmap learned FSMap version 105 2023-07-21T19:05:29.556 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.539+0000 7f7ad90ecb80 1 -- 172.21.15.39:0/419405289 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=4+}) v3 -- 0x55cb53c4b1f0 con 0x55cb53d016b0 2023-07-21T19:05:29.557 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.539+0000 7f7aadffb700 1 -- 172.21.15.39:0/419405289 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ab809f410 con 0x55cb53d016b0 2023-07-21T19:05:29.557 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.539+0000 7f7aadffb700 1 -- 172.21.15.39:0/419405289 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 105) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7ab809a960 con 0x55cb53d016b0 2023-07-21T19:05:29.557 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.539+0000 7f7ad90ecb80 10 client.17856 fetch_fsmap finished waiting for FSMap version 105 2023-07-21T19:05:29.557 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.539+0000 7f7ad90ecb80 10 client.17856 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:05:29.557 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.539+0000 7f7ad90ecb80 20 client.17856 populate_metadata read hostname 'smithi039' 2023-07-21T19:05:29.557 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.539+0000 7f7ad90ecb80 1 --2- 172.21.15.39:0/419405289 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55cb53f13480 0x55cb53f15860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:29.557 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.539+0000 7f7ad90ecb80 4 client.17856 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:05:29.557 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.539+0000 7f7ad90ecb80 1 -- 172.21.15.39:0/419405289 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55cb53db1ff0 con 0x55cb53f13480 2023-07-21T19:05:29.558 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.540+0000 7f7ac5d9b700 1 --2- 172.21.15.39:0/419405289 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55cb53f13480 0x55cb53f15860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:29.558 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.540+0000 7f7ac5d9b700 1 --2- 172.21.15.39:0/419405289 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55cb53f13480 0x55cb53f15860 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8de5d3ff9fed6373 server_cookie=87a0bfd1f6a23f5e in_seq=0 out_seq=0 2023-07-21T19:05:29.558 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.540+0000 7f7aadffb700 10 client.17856 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T19:05:29.727 INFO:tasks.workunit.client.0.smithi039.stdout: 10 337960 27.96 MB/sec execute 522 sec latency 279.552 ms 2023-07-21T19:05:29.852 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.839+0000 7f7aadffb700 1 -- 172.21.15.39:0/419405289 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55cb53db1ff0 con 0x55cb53f13480 2023-07-21T19:05:29.853 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.839+0000 7f7aadffb700 10 client.17856 handle_command_reply: tid=0 2023-07-21T19:05:29.853 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.839+0000 7f7ad90ecb80 1 client.17856 shutdown 2023-07-21T19:05:29.853 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.839+0000 7f7ad90ecb80 1 -- 172.21.15.39:0/419405289 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb53d016b0 msgr2=0x55cb53f06c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:29.853 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.839+0000 7f7ad90ecb80 1 --2- 172.21.15.39:0/419405289 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb53d016b0 0x55cb53f06c60 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8006010 tx=0x7f7ab8005ad0 comp rx=0 tx=0).stop 2023-07-21T19:05:29.854 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.839+0000 7f7ad90ecb80 1 -- 172.21.15.39:0/419405289 shutdown_connections 2023-07-21T19:05:29.854 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.839+0000 7f7ad90ecb80 1 --2- 172.21.15.39:0/419405289 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55cb53f13480 0x55cb53f15860 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:29.854 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.839+0000 7f7ad90ecb80 1 --2- 172.21.15.39:0/419405289 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cb53eae740 0x55cb53f076e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:29.854 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.839+0000 7f7ad90ecb80 1 --2- 172.21.15.39:0/419405289 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb53d016b0 0x55cb53f06c60 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:29.854 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.839+0000 7f7ad90ecb80 1 --2- 172.21.15.39:0/419405289 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cb53ea5580 0x55cb53f071a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:29.855 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.839+0000 7f7ad90ecb80 1 -- 172.21.15.39:0/419405289 >> 172.21.15.39:0/419405289 conn(0x55cb53e9abe0 msgr2=0x55cb53eacca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:29.855 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.839+0000 7f7aadffb700 0 client.17856 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T19:05:29.855 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.839+0000 7f7ad90ecb80 1 -- 172.21.15.39:0/419405289 shutdown_connections 2023-07-21T19:05:29.855 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.839+0000 7f7ad90ecb80 1 -- 172.21.15.39:0/419405289 wait complete. 2023-07-21T19:05:29.855 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.839+0000 7f7ad90ecb80 20 client.17856 trim_cache size 0 max 16384 2023-07-21T19:05:29.856 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T19:05:29.856 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T19:05:29.857 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T19:05:29.857 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T19:05:29.858 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.840+0000 7f7accf66700 1 -- 172.21.15.39:0/4002399095 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f7ab00a12c0 msgr2=0x7f7ab00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:29.858 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.840+0000 7f7accf66700 1 --2- 172.21.15.39:0/4002399095 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f7ab00a12c0 0x7f7ab00a3770 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7ac810e630 tx=0x7f7abc009190 comp rx=0 tx=0).stop 2023-07-21T19:05:29.859 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.840+0000 7f7accf66700 1 -- 172.21.15.39:0/4002399095 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ac80fea10 msgr2=0x7f7ac8111da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:29.859 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.840+0000 7f7accf66700 1 --2- 172.21.15.39:0/4002399095 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ac80fea10 0x7f7ac8111da0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8000c00 tx=0x7f7ab8003c20 comp rx=0 tx=0).stop 2023-07-21T19:05:29.859 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.840+0000 7f7accf66700 1 -- 172.21.15.39:0/4002399095 shutdown_connections 2023-07-21T19:05:29.859 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.840+0000 7f7accf66700 1 --2- 172.21.15.39:0/4002399095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ac8104a20 0x7f7ac810cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:29.859 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.840+0000 7f7accf66700 1 --2- 172.21.15.39:0/4002399095 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ac80fea10 0x7f7ac8111da0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:29.859 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.840+0000 7f7accf66700 1 --2- 172.21.15.39:0/4002399095 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f7ab00a12c0 0x7f7ab00a3770 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:29.859 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.840+0000 7f7accf66700 1 --2- 172.21.15.39:0/4002399095 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ac80ff3c0 0x7f7ac8112300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:29.860 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.840+0000 7f7accf66700 1 -- 172.21.15.39:0/4002399095 >> 172.21.15.39:0/4002399095 conn(0x7f7ac80fa4d0 msgr2=0x7f7ac8102640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:29.860 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.840+0000 7f7accf66700 1 -- 172.21.15.39:0/4002399095 shutdown_connections 2023-07-21T19:05:29.860 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:29.840+0000 7f7accf66700 1 -- 172.21.15.39:0/4002399095 wait complete. 2023-07-21T19:05:30.726 INFO:tasks.workunit.client.0.smithi039.stdout: 10 338484 27.98 MB/sec execute 523 sec latency 244.683 ms 2023-07-21T19:05:30.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:05:30 smithi193 ceph-mon[98204]: pgmap v753: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.2 MiB/s wr, 400 op/s 2023-07-21T19:05:30.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:05:30 smithi155 ceph-mon[97934]: pgmap v753: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.2 MiB/s wr, 400 op/s 2023-07-21T19:05:30.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:05:30 smithi039 ceph-mon[96489]: pgmap v753: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.2 MiB/s wr, 400 op/s 2023-07-21T19:05:31.725 INFO:tasks.workunit.client.0.smithi039.stdout: 10 338896 27.97 MB/sec execute 524 sec latency 76.737 ms 2023-07-21T19:05:32.725 INFO:tasks.workunit.client.0.smithi039.stdout: 10 339294 27.94 MB/sec execute 525 sec latency 88.176 ms 2023-07-21T19:05:32.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:05:32 smithi193 ceph-mon[98204]: pgmap v754: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.8 MiB/s wr, 430 op/s 2023-07-21T19:05:32.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:05:32 smithi155 ceph-mon[97934]: pgmap v754: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.8 MiB/s wr, 430 op/s 2023-07-21T19:05:32.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:05:32 smithi039 ceph-mon[96489]: pgmap v754: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.8 MiB/s wr, 430 op/s 2023-07-21T19:05:33.726 INFO:tasks.workunit.client.0.smithi039.stdout: 10 339761 27.93 MB/sec execute 526 sec latency 195.452 ms 2023-07-21T19:05:34.727 INFO:tasks.workunit.client.0.smithi039.stdout: 10 339812 27.88 MB/sec execute 527 sec latency 947.099 ms 2023-07-21T19:05:34.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:05:34 smithi193 ceph-mon[98204]: pgmap v755: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 633 op/s 2023-07-21T19:05:34.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:05:34 smithi155 ceph-mon[97934]: pgmap v755: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 633 op/s 2023-07-21T19:05:34.933 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:05:34 smithi039 ceph-mon[96489]: pgmap v755: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 633 op/s 2023-07-21T19:05:35.726 INFO:tasks.workunit.client.0.smithi039.stdout: 10 339922 27.84 MB/sec execute 528 sec latency 1315.388 ms 2023-07-21T19:05:35.860 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:05:35 smithi193 ceph-mon[98204]: pgmap v756: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.9 MiB/s wr, 425 op/s 2023-07-21T19:05:35.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:05:35 smithi155 ceph-mon[97934]: pgmap v756: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.9 MiB/s wr, 425 op/s 2023-07-21T19:05:35.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:05:35 smithi039 ceph-mon[96489]: pgmap v756: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.9 MiB/s wr, 425 op/s 2023-07-21T19:05:36.726 INFO:tasks.workunit.client.0.smithi039.stdout: 10 340179 27.81 MB/sec execute 529 sec latency 293.544 ms 2023-07-21T19:05:37.726 INFO:tasks.workunit.client.0.smithi039.stdout: 10 340627 27.79 MB/sec execute 530 sec latency 214.994 ms 2023-07-21T19:05:38.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:05:38 smithi039 ceph-mon[96489]: pgmap v757: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 22 MiB/s wr, 771 op/s 2023-07-21T19:05:38.726 INFO:tasks.workunit.client.0.smithi039.stdout: 10 341274 27.83 MB/sec execute 531 sec latency 125.243 ms 2023-07-21T19:05:38.862 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:05:38 smithi193 ceph-mon[98204]: pgmap v757: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 22 MiB/s wr, 771 op/s 2023-07-21T19:05:38.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:05:38 smithi155 ceph-mon[97934]: pgmap v757: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 22 MiB/s wr, 771 op/s 2023-07-21T19:05:39.726 INFO:tasks.workunit.client.0.smithi039.stdout: 10 341851 27.84 MB/sec execute 532 sec latency 88.017 ms 2023-07-21T19:05:40.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:05:40 smithi039 ceph-mon[96489]: pgmap v758: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 19 MiB/s wr, 594 op/s 2023-07-21T19:05:40.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:05:40 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:05:40.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:05:40 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T19:05:40.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:05:40 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T19:05:40.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:05:40 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T19:05:40.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:05:40 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T19:05:40.726 INFO:tasks.workunit.client.0.smithi039.stdout: 10 342227 27.83 MB/sec execute 533 sec latency 96.587 ms 2023-07-21T19:05:40.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:05:40 smithi193 ceph-mon[98204]: pgmap v758: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 19 MiB/s wr, 594 op/s 2023-07-21T19:05:40.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:05:40 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:05:40.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:05:40 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T19:05:40.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:05:40 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T19:05:40.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:05:40 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T19:05:40.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:05:40 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T19:05:40.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:05:40 smithi155 ceph-mon[97934]: pgmap v758: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 19 MiB/s wr, 594 op/s 2023-07-21T19:05:40.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:05:40 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:05:40.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:05:40 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T19:05:40.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:05:40 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T19:05:40.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:05:40 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T19:05:40.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:05:40 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T19:05:41.726 INFO:tasks.workunit.client.0.smithi039.stdout: 10 342584 27.81 MB/sec execute 534 sec latency 126.188 ms 2023-07-21T19:05:41.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:05:41 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T19:05:41.859 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:05:41 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T19:05:41.859 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:05:41 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:05:41.859 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:05:41 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:05:41.859 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:05:41 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T19:05:41.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:05:41 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T19:05:41.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:05:41 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T19:05:41.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:05:41 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:05:41.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:05:41 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:05:41.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:05:41 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T19:05:41.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:05:41 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T19:05:41.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:05:41 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T19:05:41.926 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:05:41 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:05:41.926 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:05:41 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:05:41.926 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:05:41 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T19:05:42.726 INFO:tasks.workunit.client.0.smithi039.stdout: 10 343009 27.79 MB/sec execute 535 sec latency 110.433 ms 2023-07-21T19:05:42.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:05:42 smithi193 ceph-mon[98204]: pgmap v759: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 20 MiB/s wr, 638 op/s 2023-07-21T19:05:42.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:05:42 smithi155 ceph-mon[97934]: pgmap v759: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 20 MiB/s wr, 638 op/s 2023-07-21T19:05:42.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:05:42 smithi039 ceph-mon[96489]: pgmap v759: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 20 MiB/s wr, 638 op/s 2023-07-21T19:05:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.986+0000 7f2ad37c5700 1 -- 172.21.15.39:0/2438343856 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2acc104a30 msgr2=0x7f2acc104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.986+0000 7f2ad37c5700 1 --2- 172.21.15.39:0/2438343856 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2acc104a30 0x7f2acc104e00 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f2ac8009990 tx=0x7f2ac8009ca0 comp rx=0 tx=0).stop 2023-07-21T19:05:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.986+0000 7f2ad37c5700 1 -- 172.21.15.39:0/2438343856 shutdown_connections 2023-07-21T19:05:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.986+0000 7f2ad37c5700 1 --2- 172.21.15.39:0/2438343856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2acc0ff3e0 0x7f2acc10b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.986+0000 7f2ad37c5700 1 --2- 172.21.15.39:0/2438343856 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2acc104a30 0x7f2acc104e00 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.986+0000 7f2ad37c5700 1 --2- 172.21.15.39:0/2438343856 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2acc0fea30 0x7f2acc0feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.986+0000 7f2ad37c5700 1 -- 172.21.15.39:0/2438343856 >> 172.21.15.39:0/2438343856 conn(0x7f2acc074760 msgr2=0x7f2acc074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.987+0000 7f2ad37c5700 1 -- 172.21.15.39:0/2438343856 shutdown_connections 2023-07-21T19:05:42.999 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.987+0000 7f2ad37c5700 1 -- 172.21.15.39:0/2438343856 wait complete. 2023-07-21T19:05:43.000 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.987+0000 7f2ad37c5700 1 Processor -- start 2023-07-21T19:05:43.001 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.987+0000 7f2ad37c5700 1 -- start start 2023-07-21T19:05:43.001 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.988+0000 7f2ad37c5700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2acc0fea30 0x7f2acc19ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:43.001 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.988+0000 7f2ad37c5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2acc0ff3e0 0x7f2acc19b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:43.001 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.988+0000 7f2ad37c5700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2acc104a30 0x7f2acc1949a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:43.001 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.988+0000 7f2ad37c5700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2acc195010 con 0x7f2acc0ff3e0 2023-07-21T19:05:43.001 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.988+0000 7f2ad37c5700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2acc195180 con 0x7f2acc0fea30 2023-07-21T19:05:43.001 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.988+0000 7f2ad37c5700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2acc195340 con 0x7f2acc104a30 2023-07-21T19:05:43.001 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.988+0000 7f2ad0d60700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2acc0ff3e0 0x7f2acc19b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:43.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.988+0000 7f2ad1561700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2acc0fea30 0x7f2acc19ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:43.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.988+0000 7f2ad0d60700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2acc0ff3e0 0x7f2acc19b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34934/0 (socket says 172.21.15.39:34934) 2023-07-21T19:05:43.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.988+0000 7f2ad0d60700 1 -- 172.21.15.39:0/1815642776 learned_addr learned my addr 172.21.15.39:0/1815642776 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:05:43.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.988+0000 7f2ad1d62700 1 --2- 172.21.15.39:0/1815642776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2acc104a30 0x7f2acc1949a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:43.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.988+0000 7f2ad0d60700 1 -- 172.21.15.39:0/1815642776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2acc104a30 msgr2=0x7f2acc1949a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:43.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.988+0000 7f2ad0d60700 1 --2- 172.21.15.39:0/1815642776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2acc104a30 0x7f2acc1949a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:43.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.988+0000 7f2ad0d60700 1 -- 172.21.15.39:0/1815642776 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2acc0fea30 msgr2=0x7f2acc19ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.988+0000 7f2ad0d60700 1 --2- 172.21.15.39:0/1815642776 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2acc0fea30 0x7f2acc19ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.988+0000 7f2ad0d60700 1 -- 172.21.15.39:0/1815642776 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ac4009c40 con 0x7f2acc0ff3e0 2023-07-21T19:05:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.988+0000 7f2ad1d62700 1 --2- 172.21.15.39:0/1815642776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2acc104a30 0x7f2acc1949a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:05:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.988+0000 7f2ad0d60700 1 --2- 172.21.15.39:0/1815642776 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2acc0ff3e0 0x7f2acc19b080 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f2abc00d9e0 tx=0x7f2abc00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.989+0000 7f2ac27fc700 1 -- 172.21.15.39:0/1815642776 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2abc00ec50 con 0x7f2acc0ff3e0 2023-07-21T19:05:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.989+0000 7f2ad37c5700 1 -- 172.21.15.39:0/1815642776 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ac80095f0 con 0x7f2acc0ff3e0 2023-07-21T19:05:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.989+0000 7f2ac27fc700 1 -- 172.21.15.39:0/1815642776 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2abc00edb0 con 0x7f2acc0ff3e0 2023-07-21T19:05:43.004 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.989+0000 7f2ac27fc700 1 -- 172.21.15.39:0/1815642776 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2abc014760 con 0x7f2acc0ff3e0 2023-07-21T19:05:43.004 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.989+0000 7f2ad37c5700 1 -- 172.21.15.39:0/1815642776 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2acc067b60 con 0x7f2acc0ff3e0 2023-07-21T19:05:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.992+0000 7f2ac27fc700 1 -- 172.21.15.39:0/1815642776 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f2abc016440 con 0x7f2acc0ff3e0 2023-07-21T19:05:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.992+0000 7f2ac27fc700 1 --2- 172.21.15.39:0/1815642776 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f2ab80a1340 0x7f2ab80a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.993+0000 7f2ac27fc700 1 -- 172.21.15.39:0/1815642776 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f2abc0ce560 con 0x7f2acc0ff3e0 2023-07-21T19:05:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.993+0000 7f2ad1561700 1 --2- 172.21.15.39:0/1815642776 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f2ab80a1340 0x7f2ab80a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.993+0000 7f2ad1561700 1 --2- 172.21.15.39:0/1815642776 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f2ab80a1340 0x7f2ab80a37f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2ac800b6f0 tx=0x7f2ac8005fd0 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.994+0000 7f2adf94bb80 1 Processor -- start 2023-07-21T19:05:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.994+0000 7f2adf94bb80 1 -- start start 2023-07-21T19:05:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.994+0000 7f2adf94bb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c9bf656b0 0x561c9c10ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:43.009 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.994+0000 7f2adf94bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561c9c10f2d0 0x561c9c109040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:43.009 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.994+0000 7f2adf94bb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c9c109580 0x561c9c1099d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:43.009 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.994+0000 7f2adf94bb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c9c00ec50 con 0x561c9c109580 2023-07-21T19:05:43.009 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.994+0000 7f2adf94bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c9c0619e0 con 0x561c9c10f2d0 2023-07-21T19:05:43.009 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.994+0000 7f2adf94bb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c9c05ae70 con 0x561c9bf656b0 2023-07-21T19:05:43.009 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.994+0000 7f2ad1d62700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c9c109580 0x561c9c1099d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:43.009 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.994+0000 7f2ad1d62700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c9c109580 0x561c9c1099d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34942/0 (socket says 172.21.15.39:34942) 2023-07-21T19:05:43.009 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.994+0000 7f2ad1561700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c9bf656b0 0x561c9c10ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:43.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.994+0000 7f2ad1d62700 1 -- 172.21.15.39:0/1282243134 learned_addr learned my addr 172.21.15.39:0/1282243134 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:05:43.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.994+0000 7f2ad0d60700 1 --2- 172.21.15.39:0/1282243134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561c9c10f2d0 0x561c9c109040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:43.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.995+0000 7f2ad1d62700 1 -- 172.21.15.39:0/1282243134 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c9bf656b0 msgr2=0x561c9c10ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:05:43.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.995+0000 7f2ad1d62700 1 --2- 172.21.15.39:0/1282243134 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c9bf656b0 0x561c9c10ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:43.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.995+0000 7f2ad1d62700 1 -- 172.21.15.39:0/1282243134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561c9c10f2d0 msgr2=0x561c9c109040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:05:43.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.995+0000 7f2ad1d62700 1 --2- 172.21.15.39:0/1282243134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561c9c10f2d0 0x561c9c109040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:43.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.995+0000 7f2ad1d62700 1 -- 172.21.15.39:0/1282243134 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c9be88d80 con 0x561c9c109580 2023-07-21T19:05:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.995+0000 7f2ad1561700 1 --2- 172.21.15.39:0/1282243134 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c9bf656b0 0x561c9c10ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:05:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.995+0000 7f2ad0d60700 1 --2- 172.21.15.39:0/1282243134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561c9c10f2d0 0x561c9c109040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:05:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.995+0000 7f2ad1d62700 1 --2- 172.21.15.39:0/1282243134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c9c109580 0x561c9c1099d0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f2ac4007550 tx=0x7f2ac4009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90be79334f16c367 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.995+0000 7f2ab77fe700 1 -- 172.21.15.39:0/1282243134 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac400e070 con 0x561c9c109580 2023-07-21T19:05:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.995+0000 7f2ab77fe700 1 -- 172.21.15.39:0/1282243134 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ac400f040 con 0x561c9c109580 2023-07-21T19:05:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.995+0000 7f2ab77fe700 1 -- 172.21.15.39:0/1282243134 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac4014860 con 0x561c9c109580 2023-07-21T19:05:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.996+0000 7f2adf94bb80 1 -- 172.21.15.39:0/1282243134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c9c109580 msgr2=0x561c9c1099d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.996+0000 7f2adf94bb80 1 --2- 172.21.15.39:0/1282243134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c9c109580 0x561c9c1099d0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f2ac4007550 tx=0x7f2ac4009920 comp rx=0 tx=0).stop 2023-07-21T19:05:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.996+0000 7f2adf94bb80 1 -- 172.21.15.39:0/1282243134 shutdown_connections 2023-07-21T19:05:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.996+0000 7f2adf94bb80 1 --2- 172.21.15.39:0/1282243134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c9c109580 0x561c9c1099d0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.996+0000 7f2adf94bb80 1 --2- 172.21.15.39:0/1282243134 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c9bf656b0 0x561c9c10ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.996+0000 7f2adf94bb80 1 --2- 172.21.15.39:0/1282243134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561c9c10f2d0 0x561c9c109040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.996+0000 7f2adf94bb80 1 -- 172.21.15.39:0/1282243134 >> 172.21.15.39:0/1282243134 conn(0x561c9c0febe0 msgr2=0x561c9c103f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.996+0000 7f2adf94bb80 1 -- 172.21.15.39:0/1282243134 shutdown_connections 2023-07-21T19:05:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.996+0000 7f2adf94bb80 1 -- 172.21.15.39:0/1282243134 wait complete. 2023-07-21T19:05:43.013 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.997+0000 7f2adf94bb80 1 -- start start 2023-07-21T19:05:43.013 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.997+0000 7f2adf94bb80 1 Processor -- start 2023-07-21T19:05:43.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.997+0000 7f2adf94bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561c9bf656b0 0x561c9c115620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:43.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.997+0000 7f2adf94bb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c9c109580 0x561c9c112910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:43.015 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.997+0000 7f2adf94bb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c9c10f2d0 0x561c9c113000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:43.015 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.997+0000 7f2adf94bb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c9bffaa60 con 0x561c9c109580 2023-07-21T19:05:43.015 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.997+0000 7f2adf94bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c9bffa760 con 0x561c9bf656b0 2023-07-21T19:05:43.015 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.997+0000 7f2adf94bb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c9bede8c0 con 0x561c9c10f2d0 2023-07-21T19:05:43.015 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.997+0000 7f2adf94bb80 10 client.0 _ll_register_callbacks cb 0x561c9bff8700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:05:43.015 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.997+0000 7f2ad1561700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561c9bf656b0 0x561c9c115620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:43.015 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.997+0000 7f2ad1561700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561c9bf656b0 0x561c9c115620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:47542/0 (socket says 172.21.15.39:47542) 2023-07-21T19:05:43.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.997+0000 7f2ad1561700 1 -- 172.21.15.39:0/1134012509 learned_addr learned my addr 172.21.15.39:0/1134012509 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:05:43.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.998+0000 7f2ad0d60700 1 --2- 172.21.15.39:0/1134012509 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c9c109580 0x561c9c112910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:43.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.998+0000 7f2ad0d60700 1 -- 172.21.15.39:0/1134012509 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c9c10f2d0 msgr2=0x561c9c113000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:05:43.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.998+0000 7f2ad0d60700 1 --2- 172.21.15.39:0/1134012509 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c9c10f2d0 0x561c9c113000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:43.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.998+0000 7f2ad0d60700 1 -- 172.21.15.39:0/1134012509 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561c9bf656b0 msgr2=0x561c9c115620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:05:43.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.998+0000 7f2ad0d60700 1 --2- 172.21.15.39:0/1134012509 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561c9bf656b0 0x561c9c115620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:43.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.998+0000 7f2ad0d60700 1 -- 172.21.15.39:0/1134012509 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2acc10c160 con 0x561c9c109580 2023-07-21T19:05:43.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.998+0000 7f2ad0d60700 1 --2- 172.21.15.39:0/1134012509 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c9c109580 0x561c9c112910 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f2abc0cbdc0 tx=0x7f2abc09b1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=456624645426feb0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:43.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.998+0000 7f2ab4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T19:05:43.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.999+0000 7f2ab4ff9700 1 -- 172.21.15.39:0/1134012509 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2abc0a3b90 con 0x561c9c109580 2023-07-21T19:05:43.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.999+0000 7f2ab4ff9700 1 -- 172.21.15.39:0/1134012509 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f2abc09b9a0 con 0x561c9c109580 2023-07-21T19:05:43.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.999+0000 7f2adf94bb80 1 -- 172.21.15.39:0/1134012509 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561c9bfb4490 con 0x561c9c109580 2023-07-21T19:05:43.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.999+0000 7f2ab4ff9700 1 -- 172.21.15.39:0/1134012509 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2abc09bb60 con 0x561c9c109580 2023-07-21T19:05:43.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.999+0000 7f2adf94bb80 10 client.17868 fetch_fsmap learned FSMap version 105 2023-07-21T19:05:43.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:42.999+0000 7f2adf94bb80 1 -- 172.21.15.39:0/1134012509 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=4+}) v3 -- 0x7f2ac4009280 con 0x561c9c109580 2023-07-21T19:05:43.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.000+0000 7f2ab4ff9700 1 -- 172.21.15.39:0/1134012509 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561c9bfb4490 con 0x561c9c109580 2023-07-21T19:05:43.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.000+0000 7f2ab4ff9700 1 -- 172.21.15.39:0/1134012509 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 105) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2abc0a3790 con 0x561c9c109580 2023-07-21T19:05:43.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.000+0000 7f2adf94bb80 10 client.17868 fetch_fsmap finished waiting for FSMap version 105 2023-07-21T19:05:43.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.000+0000 7f2adf94bb80 10 client.17868 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:05:43.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.000+0000 7f2adf94bb80 20 client.17868 populate_metadata read hostname 'smithi039' 2023-07-21T19:05:43.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.000+0000 7f2adf94bb80 1 --2- 172.21.15.39:0/1134012509 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x561c9c137ed0 0x561c9c13a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:43.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.000+0000 7f2adf94bb80 4 client.17868 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:05:43.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.000+0000 7f2adf94bb80 1 -- 172.21.15.39:0/1134012509 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561c9c053e50 con 0x561c9c137ed0 2023-07-21T19:05:43.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.000+0000 7f2ad1561700 1 --2- 172.21.15.39:0/1134012509 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x561c9c137ed0 0x561c9c13a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:43.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.001+0000 7f2ab4ff9700 10 client.17868 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T19:05:43.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.001+0000 7f2ad1561700 1 --2- 172.21.15.39:0/1134012509 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x561c9c137ed0 0x561c9c13a2e0 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edb8e9e35726cc3e server_cookie=903fbccdd00c5d0e in_seq=0 out_seq=0 2023-07-21T19:05:43.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.004+0000 7f2ab4ff9700 1 -- 172.21.15.39:0/1134012509 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561c9c053e50 con 0x561c9c137ed0 2023-07-21T19:05:43.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.004+0000 7f2ab4ff9700 10 client.17868 handle_command_reply: tid=0 2023-07-21T19:05:43.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.004+0000 7f2adf94bb80 1 client.17868 shutdown 2023-07-21T19:05:43.021 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.005+0000 7f2adf94bb80 1 -- 172.21.15.39:0/1134012509 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c9c109580 msgr2=0x561c9c112910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:43.021 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.005+0000 7f2adf94bb80 1 --2- 172.21.15.39:0/1134012509 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c9c109580 0x561c9c112910 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f2abc0cbdc0 tx=0x7f2abc09b1d0 comp rx=0 tx=0).stop 2023-07-21T19:05:43.021 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.005+0000 7f2adf94bb80 1 -- 172.21.15.39:0/1134012509 shutdown_connections 2023-07-21T19:05:43.021 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.005+0000 7f2adf94bb80 1 --2- 172.21.15.39:0/1134012509 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x561c9c137ed0 0x561c9c13a2e0 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:43.021 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.005+0000 7f2adf94bb80 1 --2- 172.21.15.39:0/1134012509 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561c9bf656b0 0x561c9c115620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:43.021 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.005+0000 7f2adf94bb80 1 --2- 172.21.15.39:0/1134012509 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c9c10f2d0 0x561c9c113000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:43.021 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.005+0000 7f2ab4ff9700 0 client.17868 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T19:05:43.021 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.005+0000 7f2adf94bb80 1 --2- 172.21.15.39:0/1134012509 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c9c109580 0x561c9c112910 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:43.022 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.005+0000 7f2adf94bb80 1 -- 172.21.15.39:0/1134012509 >> 172.21.15.39:0/1134012509 conn(0x561c9c0febe0 msgr2=0x561c9c102c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:43.022 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.005+0000 7f2adf94bb80 1 -- 172.21.15.39:0/1134012509 shutdown_connections 2023-07-21T19:05:43.022 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.005+0000 7f2adf94bb80 1 -- 172.21.15.39:0/1134012509 wait complete. 2023-07-21T19:05:43.022 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.005+0000 7f2adf94bb80 20 client.17868 trim_cache size 0 max 16384 2023-07-21T19:05:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.026+0000 7f2adf94bb80 1 Processor -- start 2023-07-21T19:05:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.026+0000 7f2adf94bb80 1 -- start start 2023-07-21T19:05:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.026+0000 7f2adf94bb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c9bf656b0 0x561c9c112110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.026+0000 7f2adf94bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561c9c109580 0x561c9c103c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.026+0000 7f2adf94bb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c9c10f2d0 0x561c9c1151c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.026+0000 7f2adf94bb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c9c100d00 con 0x561c9c10f2d0 2023-07-21T19:05:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.026+0000 7f2adf94bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c9c113e80 con 0x561c9c109580 2023-07-21T19:05:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.026+0000 7f2adf94bb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c9c104fd0 con 0x561c9bf656b0 2023-07-21T19:05:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.026+0000 7f2ad1d62700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c9c10f2d0 0x561c9c1151c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.026+0000 7f2ad1d62700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c9c10f2d0 0x561c9c1151c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34958/0 (socket says 172.21.15.39:34958) 2023-07-21T19:05:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.026+0000 7f2ad1d62700 1 -- 172.21.15.39:0/3704264078 learned_addr learned my addr 172.21.15.39:0/3704264078 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:05:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.026+0000 7f2ad0d60700 1 --2- 172.21.15.39:0/3704264078 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561c9c109580 0x561c9c103c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.027+0000 7f2ad1561700 1 --2- 172.21.15.39:0/3704264078 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c9bf656b0 0x561c9c112110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.027+0000 7f2ad1d62700 1 -- 172.21.15.39:0/3704264078 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c9bf656b0 msgr2=0x561c9c112110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:05:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.027+0000 7f2ad1d62700 1 --2- 172.21.15.39:0/3704264078 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c9bf656b0 0x561c9c112110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.027+0000 7f2ad1d62700 1 -- 172.21.15.39:0/3704264078 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561c9c109580 msgr2=0x561c9c103c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:05:43.040 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.027+0000 7f2ad1d62700 1 --2- 172.21.15.39:0/3704264078 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561c9c109580 0x561c9c103c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:43.040 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.027+0000 7f2ad1d62700 1 -- 172.21.15.39:0/3704264078 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c9be71c00 con 0x561c9c10f2d0 2023-07-21T19:05:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.027+0000 7f2ad1d62700 1 --2- 172.21.15.39:0/3704264078 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c9c10f2d0 0x561c9c1151c0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f2ac4009b70 tx=0x7f2ac4009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=302f3d0619e2edd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:43.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.027+0000 7f2ab7fff700 1 -- 172.21.15.39:0/3704264078 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac400e030 con 0x561c9c10f2d0 2023-07-21T19:05:43.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.027+0000 7f2ab7fff700 1 -- 172.21.15.39:0/3704264078 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ac4008f00 con 0x561c9c10f2d0 2023-07-21T19:05:43.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.027+0000 7f2ab7fff700 1 -- 172.21.15.39:0/3704264078 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac4015c10 con 0x561c9c10f2d0 2023-07-21T19:05:43.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.028+0000 7f2adf94bb80 1 -- 172.21.15.39:0/3704264078 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c9c10f2d0 msgr2=0x561c9c1151c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:43.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.028+0000 7f2adf94bb80 1 --2- 172.21.15.39:0/3704264078 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c9c10f2d0 0x561c9c1151c0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f2ac4009b70 tx=0x7f2ac4009ba0 comp rx=0 tx=0).stop 2023-07-21T19:05:43.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.028+0000 7f2adf94bb80 1 -- 172.21.15.39:0/3704264078 shutdown_connections 2023-07-21T19:05:43.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.028+0000 7f2adf94bb80 1 --2- 172.21.15.39:0/3704264078 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c9c10f2d0 0x561c9c1151c0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:43.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.028+0000 7f2adf94bb80 1 --2- 172.21.15.39:0/3704264078 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c9bf656b0 0x561c9c112110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:43.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.028+0000 7f2adf94bb80 1 --2- 172.21.15.39:0/3704264078 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561c9c109580 0x561c9c103c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:43.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.028+0000 7f2adf94bb80 1 -- 172.21.15.39:0/3704264078 >> 172.21.15.39:0/3704264078 conn(0x561c9c0febe0 msgr2=0x561c9c10cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:43.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.028+0000 7f2adf94bb80 1 -- 172.21.15.39:0/3704264078 shutdown_connections 2023-07-21T19:05:43.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.028+0000 7f2adf94bb80 1 -- 172.21.15.39:0/3704264078 wait complete. 2023-07-21T19:05:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.028+0000 7f2adf94bb80 1 -- start start 2023-07-21T19:05:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.029+0000 7f2adf94bb80 1 Processor -- start 2023-07-21T19:05:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.029+0000 7f2adf94bb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c9bf656b0 0x561c9c11dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.029+0000 7f2adf94bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561c9c109580 0x561c9c11dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.029+0000 7f2adf94bb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c9c10f2d0 0x561c9c11c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.029+0000 7f2adf94bb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c9c11c630 con 0x561c9bf656b0 2023-07-21T19:05:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.029+0000 7f2adf94bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c9c11c7a0 con 0x561c9c109580 2023-07-21T19:05:43.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.029+0000 7f2adf94bb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c9c11c960 con 0x561c9c10f2d0 2023-07-21T19:05:43.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.029+0000 7f2adf94bb80 10 client.0 _ll_register_callbacks cb 0x7f2abc09b280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:05:43.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.029+0000 7f2ad1561700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c9bf656b0 0x561c9c11dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:43.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.029+0000 7f2ad1561700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c9bf656b0 0x561c9c11dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34960/0 (socket says 172.21.15.39:34960) 2023-07-21T19:05:43.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.029+0000 7f2ad1561700 1 -- 172.21.15.39:0/3905409170 learned_addr learned my addr 172.21.15.39:0/3905409170 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:05:43.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.029+0000 7f2ad0d60700 1 --2- 172.21.15.39:0/3905409170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561c9c109580 0x561c9c11dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:43.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.029+0000 7f2ad1d62700 1 --2- 172.21.15.39:0/3905409170 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c9c10f2d0 0x561c9c11c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.029+0000 7f2ad1561700 1 -- 172.21.15.39:0/3905409170 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c9c10f2d0 msgr2=0x561c9c11c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:05:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.029+0000 7f2ad1561700 1 --2- 172.21.15.39:0/3905409170 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c9c10f2d0 0x561c9c11c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.029+0000 7f2ad1561700 1 -- 172.21.15.39:0/3905409170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561c9c109580 msgr2=0x561c9c11dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:05:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.029+0000 7f2ad1561700 1 --2- 172.21.15.39:0/3905409170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561c9c109580 0x561c9c11dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.029+0000 7f2ad1561700 1 -- 172.21.15.39:0/3905409170 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561c9be88d80 con 0x561c9bf656b0 2023-07-21T19:05:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.029+0000 7f2ad0d60700 1 --2- 172.21.15.39:0/3905409170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561c9c109580 0x561c9c11dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:05:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.030+0000 7f2ad1561700 1 --2- 172.21.15.39:0/3905409170 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c9bf656b0 0x561c9c11dab0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f2ac800c010 tx=0x7f2ac8009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19fde0f66d818711 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:43.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.030+0000 7f2ab4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T19:05:43.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.030+0000 7f2ab4ff9700 1 -- 172.21.15.39:0/3905409170 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac802b040 con 0x561c9bf656b0 2023-07-21T19:05:43.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.030+0000 7f2adf94bb80 1 -- 172.21.15.39:0/3905409170 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2ac400f040 con 0x561c9bf656b0 2023-07-21T19:05:43.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.030+0000 7f2ab4ff9700 1 -- 172.21.15.39:0/3905409170 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f2ac8034c50 con 0x561c9bf656b0 2023-07-21T19:05:43.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.030+0000 7f2ab4ff9700 1 -- 172.21.15.39:0/3905409170 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ac8034e10 con 0x561c9bf656b0 2023-07-21T19:05:43.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.030+0000 7f2adf94bb80 10 client.17874 fetch_fsmap learned FSMap version 105 2023-07-21T19:05:43.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.030+0000 7f2adf94bb80 1 -- 172.21.15.39:0/3905409170 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=4+}) v3 -- 0x561c9beaf1f0 con 0x561c9bf656b0 2023-07-21T19:05:43.049 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.030+0000 7f2ab4ff9700 1 -- 172.21.15.39:0/3905409170 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ac400f040 con 0x561c9bf656b0 2023-07-21T19:05:43.049 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.030+0000 7f2ab4ff9700 1 -- 172.21.15.39:0/3905409170 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 105) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2ac800f460 con 0x561c9bf656b0 2023-07-21T19:05:43.049 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.031+0000 7f2adf94bb80 10 client.17874 fetch_fsmap finished waiting for FSMap version 105 2023-07-21T19:05:43.049 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.031+0000 7f2adf94bb80 10 client.17874 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:05:43.049 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.031+0000 7f2adf94bb80 20 client.17874 populate_metadata read hostname 'smithi039' 2023-07-21T19:05:43.049 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.031+0000 7f2adf94bb80 1 --2- 172.21.15.39:0/3905409170 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x561c9c18ceb0 0x561c9c18f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:43.050 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.031+0000 7f2adf94bb80 4 client.17874 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:05:43.050 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.031+0000 7f2adf94bb80 1 -- 172.21.15.39:0/3905409170 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561c9c015ff0 con 0x561c9c18ceb0 2023-07-21T19:05:43.050 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.031+0000 7f2ad0d60700 1 --2- 172.21.15.39:0/3905409170 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x561c9c18ceb0 0x561c9c18f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:43.050 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.031+0000 7f2ad0d60700 1 --2- 172.21.15.39:0/3905409170 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x561c9c18ceb0 0x561c9c18f2c0 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74fa81cd80d76e23 server_cookie=a74b0861c0fd0979 in_seq=0 out_seq=0 2023-07-21T19:05:43.050 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.031+0000 7f2ab4ff9700 10 client.17874 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T19:05:43.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.323+0000 7f2ab4ff9700 1 -- 172.21.15.39:0/3905409170 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561c9c015ff0 con 0x561c9c18ceb0 2023-07-21T19:05:43.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.323+0000 7f2ab4ff9700 10 client.17874 handle_command_reply: tid=0 2023-07-21T19:05:43.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.323+0000 7f2adf94bb80 1 client.17874 shutdown 2023-07-21T19:05:43.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.323+0000 7f2adf94bb80 1 -- 172.21.15.39:0/3905409170 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c9bf656b0 msgr2=0x561c9c11dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:43.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.323+0000 7f2adf94bb80 1 --2- 172.21.15.39:0/3905409170 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c9bf656b0 0x561c9c11dab0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f2ac800c010 tx=0x7f2ac8009ca0 comp rx=0 tx=0).stop 2023-07-21T19:05:43.342 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.324+0000 7f2adf94bb80 1 -- 172.21.15.39:0/3905409170 shutdown_connections 2023-07-21T19:05:43.342 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.324+0000 7f2adf94bb80 1 --2- 172.21.15.39:0/3905409170 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x561c9c18ceb0 0x561c9c18f2c0 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:43.342 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.324+0000 7f2adf94bb80 1 --2- 172.21.15.39:0/3905409170 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c9c10f2d0 0x561c9c11c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:43.342 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.324+0000 7f2adf94bb80 1 --2- 172.21.15.39:0/3905409170 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c9bf656b0 0x561c9c11dab0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:43.342 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.324+0000 7f2adf94bb80 1 --2- 172.21.15.39:0/3905409170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561c9c109580 0x561c9c11dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:43.342 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.324+0000 7f2adf94bb80 1 -- 172.21.15.39:0/3905409170 >> 172.21.15.39:0/3905409170 conn(0x561c9c0febe0 msgr2=0x561c9c10db30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:43.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.324+0000 7f2ab4ff9700 0 client.17874 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T19:05:43.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.324+0000 7f2adf94bb80 1 -- 172.21.15.39:0/3905409170 shutdown_connections 2023-07-21T19:05:43.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.324+0000 7f2adf94bb80 1 -- 172.21.15.39:0/3905409170 wait complete. 2023-07-21T19:05:43.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.324+0000 7f2adf94bb80 20 client.17874 trim_cache size 0 max 16384 2023-07-21T19:05:43.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T19:05:43.344 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T19:05:43.344 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T19:05:43.344 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T19:05:43.345 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.328+0000 7f2ad37c5700 1 -- 172.21.15.39:0/1815642776 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f2ab80a1340 msgr2=0x7f2ab80a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:43.345 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.328+0000 7f2ad37c5700 1 --2- 172.21.15.39:0/1815642776 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f2ab80a1340 0x7f2ab80a37f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2ac800b6f0 tx=0x7f2ac8005fd0 comp rx=0 tx=0).stop 2023-07-21T19:05:43.345 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.328+0000 7f2ad37c5700 1 -- 172.21.15.39:0/1815642776 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2acc0ff3e0 msgr2=0x7f2acc19b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:43.345 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.328+0000 7f2ad37c5700 1 --2- 172.21.15.39:0/1815642776 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2acc0ff3e0 0x7f2acc19b080 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f2abc00d9e0 tx=0x7f2abc00dcf0 comp rx=0 tx=0).stop 2023-07-21T19:05:43.345 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.329+0000 7f2ad37c5700 1 -- 172.21.15.39:0/1815642776 shutdown_connections 2023-07-21T19:05:43.345 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.329+0000 7f2ad37c5700 1 --2- 172.21.15.39:0/1815642776 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f2ab80a1340 0x7f2ab80a37f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:43.345 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.329+0000 7f2ad37c5700 1 --2- 172.21.15.39:0/1815642776 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2acc0fea30 0x7f2acc19ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:43.346 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.329+0000 7f2ad37c5700 1 --2- 172.21.15.39:0/1815642776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2acc104a30 0x7f2acc1949a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:43.346 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.329+0000 7f2ad37c5700 1 --2- 172.21.15.39:0/1815642776 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2acc0ff3e0 0x7f2acc19b080 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:43.346 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.329+0000 7f2ad37c5700 1 -- 172.21.15.39:0/1815642776 >> 172.21.15.39:0/1815642776 conn(0x7f2acc074760 msgr2=0x7f2acc0fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:43.346 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.329+0000 7f2ad37c5700 1 -- 172.21.15.39:0/1815642776 shutdown_connections 2023-07-21T19:05:43.346 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:43.329+0000 7f2ad37c5700 1 -- 172.21.15.39:0/1815642776 wait complete. 2023-07-21T19:05:43.726 INFO:tasks.workunit.client.0.smithi039.stdout: 10 343476 27.77 MB/sec execute 536 sec latency 195.350 ms 2023-07-21T19:05:44.727 INFO:tasks.workunit.client.0.smithi039.stdout: 10 344125 27.80 MB/sec execute 537 sec latency 100.378 ms 2023-07-21T19:05:44.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:05:44 smithi193 ceph-mon[98204]: pgmap v760: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 21 MiB/s wr, 836 op/s 2023-07-21T19:05:44.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:05:44 smithi155 ceph-mon[97934]: pgmap v760: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 21 MiB/s wr, 836 op/s 2023-07-21T19:05:44.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:05:44 smithi039 ceph-mon[96489]: pgmap v760: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 21 MiB/s wr, 836 op/s 2023-07-21T19:05:45.726 INFO:tasks.workunit.client.0.smithi039.stdout: 10 344713 27.80 MB/sec execute 538 sec latency 142.953 ms 2023-07-21T19:05:46.726 INFO:tasks.workunit.client.0.smithi039.stdout: 10 345301 27.81 MB/sec execute 539 sec latency 123.892 ms 2023-07-21T19:05:46.862 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:05:46 smithi193 ceph-mon[98204]: pgmap v761: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 17 MiB/s wr, 618 op/s 2023-07-21T19:05:46.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:05:46 smithi155 ceph-mon[97934]: pgmap v761: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 17 MiB/s wr, 618 op/s 2023-07-21T19:05:46.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:05:46 smithi039 ceph-mon[96489]: pgmap v761: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 17 MiB/s wr, 618 op/s 2023-07-21T19:05:47.726 INFO:tasks.workunit.client.0.smithi039.stdout: 10 345803 27.82 MB/sec execute 540 sec latency 77.860 ms 2023-07-21T19:05:48.727 INFO:tasks.workunit.client.0.smithi039.stdout: 10 346222 27.80 MB/sec execute 541 sec latency 135.515 ms 2023-07-21T19:05:48.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:05:48 smithi193 ceph-mon[98204]: pgmap v762: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 21 MiB/s wr, 850 op/s 2023-07-21T19:05:48.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:05:48 smithi155 ceph-mon[97934]: pgmap v762: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 21 MiB/s wr, 850 op/s 2023-07-21T19:05:48.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:05:48 smithi039 ceph-mon[96489]: pgmap v762: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 21 MiB/s wr, 850 op/s 2023-07-21T19:05:49.727 INFO:tasks.workunit.client.0.smithi039.stdout: 10 346413 27.76 MB/sec execute 542 sec latency 641.185 ms 2023-07-21T19:05:49.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:05:49 smithi039 ceph-mon[96489]: pgmap v763: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.5 MiB/s wr, 504 op/s 2023-07-21T19:05:50.107 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:05:49 smithi193 ceph-mon[98204]: pgmap v763: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.5 MiB/s wr, 504 op/s 2023-07-21T19:05:50.128 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:05:49 smithi155 ceph-mon[97934]: pgmap v763: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.5 MiB/s wr, 504 op/s 2023-07-21T19:05:50.730 INFO:tasks.workunit.client.0.smithi039.stdout: 10 346643 27.74 MB/sec execute 543 sec latency 754.997 ms 2023-07-21T19:05:51.730 INFO:tasks.workunit.client.0.smithi039.stdout: 10 347053 27.72 MB/sec execute 544 sec latency 153.815 ms 2023-07-21T19:05:52.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:05:52 smithi039 ceph-mon[96489]: pgmap v764: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.9 MiB/s wr, 514 op/s 2023-07-21T19:05:52.730 INFO:tasks.workunit.client.0.smithi039.stdout: 10 347692 27.75 MB/sec execute 545 sec latency 68.427 ms 2023-07-21T19:05:52.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:05:52 smithi193 ceph-mon[98204]: pgmap v764: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.9 MiB/s wr, 514 op/s 2023-07-21T19:05:52.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:05:52 smithi155 ceph-mon[97934]: pgmap v764: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.9 MiB/s wr, 514 op/s 2023-07-21T19:05:53.730 INFO:tasks.workunit.client.0.smithi039.stdout: 10 348259 27.75 MB/sec execute 546 sec latency 69.973 ms 2023-07-21T19:05:54.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:05:54 smithi039 ceph-mon[96489]: pgmap v765: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 14 MiB/s wr, 744 op/s 2023-07-21T19:05:54.730 INFO:tasks.workunit.client.0.smithi039.stdout: 10 348782 27.74 MB/sec execute 547 sec latency 98.516 ms 2023-07-21T19:05:54.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:05:54 smithi193 ceph-mon[98204]: pgmap v765: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 14 MiB/s wr, 744 op/s 2023-07-21T19:05:54.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:05:54 smithi155 ceph-mon[97934]: pgmap v765: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 14 MiB/s wr, 744 op/s 2023-07-21T19:05:55.730 INFO:tasks.workunit.client.0.smithi039.stdout: 10 349302 27.75 MB/sec execute 548 sec latency 171.830 ms 2023-07-21T19:05:56.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.520+0000 7f59c90e7700 1 -- 172.21.15.39:0/1855843319 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59c40febc0 msgr2=0x7f59c4102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:56.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.520+0000 7f59c90e7700 1 --2- 172.21.15.39:0/1855843319 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59c40febc0 0x7f59c4102b80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f59b4008570 tx=0x7f59b400acb0 comp rx=0 tx=0).stop 2023-07-21T19:05:56.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.520+0000 7f59c90e7700 1 -- 172.21.15.39:0/1855843319 shutdown_connections 2023-07-21T19:05:56.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.520+0000 7f59c90e7700 1 --2- 172.21.15.39:0/1855843319 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f59c4103260 0x7f59c410ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:56.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.520+0000 7f59c90e7700 1 --2- 172.21.15.39:0/1855843319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59c40fe220 0x7f59c40fe5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:56.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.520+0000 7f59c90e7700 1 --2- 172.21.15.39:0/1855843319 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59c40febc0 0x7f59c4102b80 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:56.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.520+0000 7f59c90e7700 1 -- 172.21.15.39:0/1855843319 >> 172.21.15.39:0/1855843319 conn(0x7f59c4073d50 msgr2=0x7f59c4074150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:56.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.521+0000 7f59c90e7700 1 -- 172.21.15.39:0/1855843319 shutdown_connections 2023-07-21T19:05:56.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.521+0000 7f59c90e7700 1 -- 172.21.15.39:0/1855843319 wait complete. 2023-07-21T19:05:56.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.522+0000 7f59c90e7700 1 Processor -- start 2023-07-21T19:05:56.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.522+0000 7f59c90e7700 1 -- start start 2023-07-21T19:05:56.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.522+0000 7f59c90e7700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59c40febc0 0x7f59c419ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:56.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.522+0000 7f59c90e7700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f59c4103260 0x7f59c419b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:56.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.522+0000 7f59c90e7700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59c4193dc0 0x7f59c4194260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:56.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.522+0000 7f59c90e7700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59c41949f0 con 0x7f59c40febc0 2023-07-21T19:05:56.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.522+0000 7f59c90e7700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59c41989f0 con 0x7f59c4193dc0 2023-07-21T19:05:56.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.522+0000 7f59c90e7700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59c4198bb0 con 0x7f59c4103260 2023-07-21T19:05:56.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.522+0000 7f59c2d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59c40febc0 0x7f59c419ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:56.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.522+0000 7f59c2d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59c40febc0 0x7f59c419ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34974/0 (socket says 172.21.15.39:34974) 2023-07-21T19:05:56.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.522+0000 7f59c2d9d700 1 -- 172.21.15.39:0/1042139858 learned_addr learned my addr 172.21.15.39:0/1042139858 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:05:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.523+0000 7f59c2d9d700 1 -- 172.21.15.39:0/1042139858 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f59c4103260 msgr2=0x7f59c419b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-21T19:05:56.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.523+0000 7f59c2d9d700 1 --2- 172.21.15.39:0/1042139858 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f59c4103260 0x7f59c419b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:56.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.523+0000 7f59c2d9d700 1 -- 172.21.15.39:0/1042139858 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59c4193dc0 msgr2=0x7f59c4194260 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-21T19:05:56.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.523+0000 7f59c2d9d700 1 --2- 172.21.15.39:0/1042139858 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59c4193dc0 0x7f59c4194260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:56.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.523+0000 7f59c2d9d700 1 -- 172.21.15.39:0/1042139858 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59b40081d0 con 0x7f59c40febc0 2023-07-21T19:05:56.538 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.524+0000 7f59c2d9d700 1 --2- 172.21.15.39:0/1042139858 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59c40febc0 0x7f59c419ae60 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f59b800b480 tx=0x7f59b800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:56.538 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.524+0000 7f59abfff700 1 -- 172.21.15.39:0/1042139858 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b80118f0 con 0x7f59c40febc0 2023-07-21T19:05:56.538 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.524+0000 7f59abfff700 1 -- 172.21.15.39:0/1042139858 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59b8010380 con 0x7f59c40febc0 2023-07-21T19:05:56.538 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.524+0000 7f59abfff700 1 -- 172.21.15.39:0/1042139858 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b800f540 con 0x7f59c40febc0 2023-07-21T19:05:56.539 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.524+0000 7f59c90e7700 1 -- 172.21.15.39:0/1042139858 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59c4198e90 con 0x7f59c40febc0 2023-07-21T19:05:56.539 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.524+0000 7f59c90e7700 1 -- 172.21.15.39:0/1042139858 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59c4199350 con 0x7f59c40febc0 2023-07-21T19:05:56.540 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.528+0000 7f59abfff700 1 -- 172.21.15.39:0/1042139858 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f59b80104f0 con 0x7f59c40febc0 2023-07-21T19:05:56.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.529+0000 7f59d526db80 1 Processor -- start 2023-07-21T19:05:56.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.529+0000 7f59abfff700 1 --2- 172.21.15.39:0/1042139858 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f59ac0a1340 0x7f59ac0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:56.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.529+0000 7f59abfff700 1 -- 172.21.15.39:0/1042139858 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f59b80cc7b0 con 0x7f59c40febc0 2023-07-21T19:05:56.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.530+0000 7f59d526db80 1 -- start start 2023-07-21T19:05:56.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.530+0000 7f59d526db80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558c5190e6b0 0x558c51ab7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:56.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.530+0000 7f59d526db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c51ab82d0 0x558c51ab2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:56.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.530+0000 7f59d526db80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c51ab2580 0x558c51ab29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:56.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.530+0000 7f59d526db80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c519b7c50 con 0x558c51ab2580 2023-07-21T19:05:56.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.530+0000 7f59d526db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c51a0a9e0 con 0x558c51ab82d0 2023-07-21T19:05:56.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.530+0000 7f59d526db80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c51a03e70 con 0x558c5190e6b0 2023-07-21T19:05:56.543 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.532+0000 7f59c2d9d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558c5190e6b0 0x558c51ab7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:56.543 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.532+0000 7f59c2d9d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558c5190e6b0 0x558c51ab7d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:53534/0 (socket says 172.21.15.39:53534) 2023-07-21T19:05:56.544 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.532+0000 7f59c2d9d700 1 -- 172.21.15.39:0/3975800002 learned_addr learned my addr 172.21.15.39:0/3975800002 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:05:56.544 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.532+0000 7f59c2d9d700 1 -- 172.21.15.39:0/3975800002 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c51ab82d0 msgr2=0x558c51ab2040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T19:05:56.544 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.532+0000 7f59c2d9d700 1 --2- 172.21.15.39:0/3975800002 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c51ab82d0 0x558c51ab2040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:56.544 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.532+0000 7f59c2d9d700 1 -- 172.21.15.39:0/3975800002 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c51ab2580 msgr2=0x558c51ab29d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T19:05:56.544 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.532+0000 7f59c2d9d700 1 --2- 172.21.15.39:0/3975800002 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c51ab2580 0x558c51ab29d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:56.544 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.532+0000 7f59c2d9d700 1 -- 172.21.15.39:0/3975800002 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c51831d80 con 0x558c5190e6b0 2023-07-21T19:05:56.545 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.532+0000 7f59c2d9d700 1 --2- 172.21.15.39:0/3975800002 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558c5190e6b0 0x558c51ab7d90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f59b80963a0 tx=0x7f59b8099360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae196c34d5bb2618 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:56.545 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.533+0000 7f59c359e700 1 --2- 172.21.15.39:0/1042139858 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f59ac0a1340 0x7f59ac0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:56.545 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.533+0000 7f59a97fa700 1 -- 172.21.15.39:0/3975800002 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b8084050 con 0x558c5190e6b0 2023-07-21T19:05:56.545 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.535+0000 7f59c359e700 1 --2- 172.21.15.39:0/1042139858 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f59ac0a1340 0x7f59ac0a37f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f59bc0095a0 tx=0x7f59bc00d040 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:56.545 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.536+0000 7f59a97fa700 1 -- 172.21.15.39:0/3975800002 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59b80a1d80 con 0x558c5190e6b0 2023-07-21T19:05:56.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.536+0000 7f59a97fa700 1 -- 172.21.15.39:0/3975800002 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b8099040 con 0x558c5190e6b0 2023-07-21T19:05:56.547 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.537+0000 7f59d526db80 1 -- 172.21.15.39:0/3975800002 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558c5190e6b0 msgr2=0x558c51ab7d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:56.547 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.537+0000 7f59d526db80 1 --2- 172.21.15.39:0/3975800002 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558c5190e6b0 0x558c51ab7d90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f59b80963a0 tx=0x7f59b8099360 comp rx=0 tx=0).stop 2023-07-21T19:05:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.539+0000 7f59d526db80 1 -- 172.21.15.39:0/3975800002 shutdown_connections 2023-07-21T19:05:56.550 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.539+0000 7f59d526db80 1 --2- 172.21.15.39:0/3975800002 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c51ab2580 0x558c51ab29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:56.550 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.540+0000 7f59d526db80 1 --2- 172.21.15.39:0/3975800002 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558c5190e6b0 0x558c51ab7d90 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:56.550 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.540+0000 7f59d526db80 1 --2- 172.21.15.39:0/3975800002 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c51ab82d0 0x558c51ab2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:56.550 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.540+0000 7f59d526db80 1 -- 172.21.15.39:0/3975800002 >> 172.21.15.39:0/3975800002 conn(0x558c51aa7be0 msgr2=0x558c51aacf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:56.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.541+0000 7f59d526db80 1 -- 172.21.15.39:0/3975800002 shutdown_connections 2023-07-21T19:05:56.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.541+0000 7f59d526db80 1 -- 172.21.15.39:0/3975800002 wait complete. 2023-07-21T19:05:56.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.541+0000 7f59d526db80 1 -- start start 2023-07-21T19:05:56.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.541+0000 7f59d526db80 1 Processor -- start 2023-07-21T19:05:56.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.542+0000 7f59d526db80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c5190e6b0 0x558c51abb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:56.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.542+0000 7f59d526db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c51ab2580 0x558c51abbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:56.554 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.542+0000 7f59d526db80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558c51ab82d0 0x558c51abc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:56.554 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.542+0000 7f59d526db80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c519a3a60 con 0x558c5190e6b0 2023-07-21T19:05:56.554 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.542+0000 7f59d526db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c519a3760 con 0x558c51ab2580 2023-07-21T19:05:56.554 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.542+0000 7f59d526db80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c518878c0 con 0x558c51ab82d0 2023-07-21T19:05:56.554 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.542+0000 7f59d526db80 10 client.0 _ll_register_callbacks cb 0x558c519a1700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:05:56.555 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.542+0000 7f59c2d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c5190e6b0 0x558c51abb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:56.555 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.542+0000 7f59c2d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c5190e6b0 0x558c51abb970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34984/0 (socket says 172.21.15.39:34984) 2023-07-21T19:05:56.555 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.542+0000 7f59c2d9d700 1 -- 172.21.15.39:0/2491390098 learned_addr learned my addr 172.21.15.39:0/2491390098 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:05:56.556 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.542+0000 7f59c359e700 1 --2- 172.21.15.39:0/2491390098 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558c51ab82d0 0x558c51abc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:56.556 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.542+0000 7f59c259c700 1 --2- 172.21.15.39:0/2491390098 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c51ab2580 0x558c51abbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:56.557 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.542+0000 7f59c2d9d700 1 -- 172.21.15.39:0/2491390098 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558c51ab82d0 msgr2=0x558c51abc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:05:56.557 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.542+0000 7f59c2d9d700 1 --2- 172.21.15.39:0/2491390098 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558c51ab82d0 0x558c51abc3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:56.557 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.542+0000 7f59c2d9d700 1 -- 172.21.15.39:0/2491390098 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c51ab2580 msgr2=0x558c51abbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:05:56.557 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.542+0000 7f59c2d9d700 1 --2- 172.21.15.39:0/2491390098 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c51ab2580 0x558c51abbeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:56.557 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.542+0000 7f59c2d9d700 1 -- 172.21.15.39:0/2491390098 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59c4199350 con 0x558c5190e6b0 2023-07-21T19:05:56.557 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.542+0000 7f59c359e700 1 --2- 172.21.15.39:0/2491390098 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558c51ab82d0 0x558c51abc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:05:56.558 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.543+0000 7f59c2d9d700 1 --2- 172.21.15.39:0/2491390098 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c5190e6b0 0x558c51abb970 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f59b8099870 tx=0x7f59b80107d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5579f8f69fa97907 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:56.558 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.543+0000 7f59a6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T19:05:56.558 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.543+0000 7f59a6ff5700 1 -- 172.21.15.39:0/2491390098 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b8099d90 con 0x558c5190e6b0 2023-07-21T19:05:56.558 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.543+0000 7f59a6ff5700 1 -- 172.21.15.39:0/2491390098 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f59b80a18f0 con 0x558c5190e6b0 2023-07-21T19:05:56.558 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.543+0000 7f59d526db80 1 -- 172.21.15.39:0/2491390098 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558c5195d490 con 0x558c5190e6b0 2023-07-21T19:05:56.558 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.543+0000 7f59a6ff5700 1 -- 172.21.15.39:0/2491390098 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59b8084020 con 0x558c5190e6b0 2023-07-21T19:05:56.558 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.543+0000 7f59d526db80 10 client.17880 fetch_fsmap learned FSMap version 105 2023-07-21T19:05:56.559 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.543+0000 7f59d526db80 1 -- 172.21.15.39:0/2491390098 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=4+}) v3 -- 0x558c5181ac00 con 0x558c5190e6b0 2023-07-21T19:05:56.559 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.545+0000 7f59a6ff5700 1 -- 172.21.15.39:0/2491390098 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558c5195d490 con 0x558c5190e6b0 2023-07-21T19:05:56.559 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.545+0000 7f59a6ff5700 1 -- 172.21.15.39:0/2491390098 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 105) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f59b8098890 con 0x558c5190e6b0 2023-07-21T19:05:56.559 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.545+0000 7f59d526db80 10 client.17880 fetch_fsmap finished waiting for FSMap version 105 2023-07-21T19:05:56.559 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.545+0000 7f59d526db80 10 client.17880 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:05:56.559 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.545+0000 7f59d526db80 20 client.17880 populate_metadata read hostname 'smithi039' 2023-07-21T19:05:56.559 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.545+0000 7f59d526db80 1 --2- 172.21.15.39:0/2491390098 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x558c51acb820 0x558c51acdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:56.559 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.545+0000 7f59d526db80 4 client.17880 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:05:56.560 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.545+0000 7f59d526db80 1 -- 172.21.15.39:0/2491390098 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558c519fce50 con 0x558c51acb820 2023-07-21T19:05:56.560 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.547+0000 7f59c259c700 1 --2- 172.21.15.39:0/2491390098 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x558c51acb820 0x558c51acdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:56.560 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.548+0000 7f59a6ff5700 10 client.17880 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T19:05:56.560 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.548+0000 7f59c259c700 1 --2- 172.21.15.39:0/2491390098 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x558c51acb820 0x558c51acdc30 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d72348f53eafc9d server_cookie=c735257ec78feee4 in_seq=0 out_seq=0 2023-07-21T19:05:56.560 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.549+0000 7f59a6ff5700 1 -- 172.21.15.39:0/2491390098 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558c519fce50 con 0x558c51acb820 2023-07-21T19:05:56.561 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.549+0000 7f59a6ff5700 10 client.17880 handle_command_reply: tid=0 2023-07-21T19:05:56.561 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.549+0000 7f59d526db80 1 client.17880 shutdown 2023-07-21T19:05:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.553+0000 7f59d526db80 1 -- 172.21.15.39:0/2491390098 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c5190e6b0 msgr2=0x558c51abb970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.553+0000 7f59d526db80 1 --2- 172.21.15.39:0/2491390098 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c5190e6b0 0x558c51abb970 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f59b8099870 tx=0x7f59b80107d0 comp rx=0 tx=0).stop 2023-07-21T19:05:56.565 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.555+0000 7f59d526db80 1 -- 172.21.15.39:0/2491390098 shutdown_connections 2023-07-21T19:05:56.565 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.555+0000 7f59d526db80 1 --2- 172.21.15.39:0/2491390098 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x558c51acb820 0x558c51acdc30 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:56.565 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.555+0000 7f59d526db80 1 --2- 172.21.15.39:0/2491390098 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558c51ab82d0 0x558c51abc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:56.566 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.555+0000 7f59d526db80 1 --2- 172.21.15.39:0/2491390098 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c5190e6b0 0x558c51abb970 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:56.566 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.555+0000 7f59d526db80 1 --2- 172.21.15.39:0/2491390098 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c51ab2580 0x558c51abbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:56.566 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.555+0000 7f59d526db80 1 -- 172.21.15.39:0/2491390098 >> 172.21.15.39:0/2491390098 conn(0x558c51aa7be0 msgr2=0x558c51aabc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:56.566 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.556+0000 7f59a6ff5700 0 client.17880 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T19:05:56.566 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.556+0000 7f59d526db80 1 -- 172.21.15.39:0/2491390098 shutdown_connections 2023-07-21T19:05:56.566 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.556+0000 7f59d526db80 1 -- 172.21.15.39:0/2491390098 wait complete. 2023-07-21T19:05:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.556+0000 7f59d526db80 20 client.17880 trim_cache size 0 max 16384 2023-07-21T19:05:56.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.580+0000 7f59d526db80 1 Processor -- start 2023-07-21T19:05:56.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.581+0000 7f59d526db80 1 -- start start 2023-07-21T19:05:56.594 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.581+0000 7f59d526db80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558c51ac7770 0x558c51ab2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:56.594 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.581+0000 7f59d526db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c51aa7be0 0x558c51abc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:56.594 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.581+0000 7f59d526db80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c5190e6b0 0x558c51aacc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:56.594 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.581+0000 7f59d526db80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c519fce50 con 0x558c5190e6b0 2023-07-21T19:05:56.594 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.581+0000 7f59d526db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c51aaeb70 con 0x558c51aa7be0 2023-07-21T19:05:56.594 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.581+0000 7f59d526db80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c51aadd50 con 0x558c51ac7770 2023-07-21T19:05:56.595 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.581+0000 7f59c2d9d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558c51ac7770 0x558c51ab2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:56.595 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.581+0000 7f59c259c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c51aa7be0 0x558c51abc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:56.595 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.581+0000 7f59c2d9d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558c51ac7770 0x558c51ab2040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:53544/0 (socket says 172.21.15.39:53544) 2023-07-21T19:05:56.595 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.581+0000 7f59c2d9d700 1 -- 172.21.15.39:0/2484106407 learned_addr learned my addr 172.21.15.39:0/2484106407 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:05:56.595 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.581+0000 7f59c359e700 1 --2- 172.21.15.39:0/2484106407 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c5190e6b0 0x558c51aacc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:56.595 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.581+0000 7f59c2d9d700 1 -- 172.21.15.39:0/2484106407 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c51aa7be0 msgr2=0x558c51abc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:05:56.596 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.581+0000 7f59c2d9d700 1 --2- 172.21.15.39:0/2484106407 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c51aa7be0 0x558c51abc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:56.596 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.581+0000 7f59c2d9d700 1 -- 172.21.15.39:0/2484106407 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c5190e6b0 msgr2=0x558c51aacc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:05:56.596 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.581+0000 7f59c2d9d700 1 --2- 172.21.15.39:0/2484106407 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c5190e6b0 0x558c51aacc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:56.596 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.581+0000 7f59c2d9d700 1 -- 172.21.15.39:0/2484106407 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c518581f0 con 0x558c51ac7770 2023-07-21T19:05:56.596 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.582+0000 7f59c2d9d700 1 --2- 172.21.15.39:0/2484106407 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558c51ac7770 0x558c51ab2040 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f59b8099630 tx=0x7f59b8097ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbd861640666275f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:56.596 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.582+0000 7f59a9ffb700 1 -- 172.21.15.39:0/2484106407 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b8084040 con 0x558c51ac7770 2023-07-21T19:05:56.597 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.582+0000 7f59a9ffb700 1 -- 172.21.15.39:0/2484106407 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59b80990b0 con 0x558c51ac7770 2023-07-21T19:05:56.597 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.582+0000 7f59a9ffb700 1 -- 172.21.15.39:0/2484106407 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b80a18f0 con 0x558c51ac7770 2023-07-21T19:05:56.597 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.588+0000 7f59d526db80 1 -- 172.21.15.39:0/2484106407 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558c51ac7770 msgr2=0x558c51ab2040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:56.598 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.588+0000 7f59d526db80 1 --2- 172.21.15.39:0/2484106407 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558c51ac7770 0x558c51ab2040 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f59b8099630 tx=0x7f59b8097ee0 comp rx=0 tx=0).stop 2023-07-21T19:05:56.599 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.589+0000 7f59d526db80 1 -- 172.21.15.39:0/2484106407 shutdown_connections 2023-07-21T19:05:56.599 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.589+0000 7f59d526db80 1 --2- 172.21.15.39:0/2484106407 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c5190e6b0 0x558c51aacc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:56.599 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.589+0000 7f59d526db80 1 --2- 172.21.15.39:0/2484106407 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558c51ac7770 0x558c51ab2040 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:56.600 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.589+0000 7f59d526db80 1 --2- 172.21.15.39:0/2484106407 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c51aa7be0 0x558c51abc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:56.600 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.589+0000 7f59d526db80 1 -- 172.21.15.39:0/2484106407 >> 172.21.15.39:0/2484106407 conn(0x558c51ab5f10 msgr2=0x558c51ab71c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:56.600 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.589+0000 7f59d526db80 1 -- 172.21.15.39:0/2484106407 shutdown_connections 2023-07-21T19:05:56.600 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.589+0000 7f59d526db80 1 -- 172.21.15.39:0/2484106407 wait complete. 2023-07-21T19:05:56.600 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.590+0000 7f59d526db80 1 -- start start 2023-07-21T19:05:56.600 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.590+0000 7f59d526db80 1 Processor -- start 2023-07-21T19:05:56.602 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.590+0000 7f59d526db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c5190e6b0 0x558c51ac29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:56.603 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.590+0000 7f59d526db80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c51aa7be0 0x558c51ac0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:56.603 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.590+0000 7f59d526db80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558c51ac7770 0x558c51ac1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:56.603 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.590+0000 7f59d526db80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c51ac3280 con 0x558c51aa7be0 2023-07-21T19:05:56.603 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.590+0000 7f59d526db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c51ac16d0 con 0x558c5190e6b0 2023-07-21T19:05:56.603 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.590+0000 7f59d526db80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c51ac1890 con 0x558c51ac7770 2023-07-21T19:05:56.603 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.590+0000 7f59d526db80 10 client.0 _ll_register_callbacks cb 0x7f59b809da20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:05:56.604 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.591+0000 7f59c259c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c51aa7be0 0x558c51ac0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:56.604 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.591+0000 7f59c259c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c51aa7be0 0x558c51ac0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35002/0 (socket says 172.21.15.39:35002) 2023-07-21T19:05:56.604 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.591+0000 7f59c259c700 1 -- 172.21.15.39:0/3209090932 learned_addr learned my addr 172.21.15.39:0/3209090932 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:05:56.604 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.591+0000 7f59c2d9d700 1 --2- 172.21.15.39:0/3209090932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c5190e6b0 0x558c51ac29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:56.604 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.591+0000 7f59c259c700 1 -- 172.21.15.39:0/3209090932 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558c51ac7770 msgr2=0x558c51ac1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:05:56.604 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.591+0000 7f59c259c700 1 --2- 172.21.15.39:0/3209090932 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558c51ac7770 0x558c51ac1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:56.605 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.591+0000 7f59c259c700 1 -- 172.21.15.39:0/3209090932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c5190e6b0 msgr2=0x558c51ac29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:05:56.605 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.591+0000 7f59c259c700 1 --2- 172.21.15.39:0/3209090932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c5190e6b0 0x558c51ac29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:56.605 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.591+0000 7f59c259c700 1 -- 172.21.15.39:0/3209090932 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558c5181ac00 con 0x558c51aa7be0 2023-07-21T19:05:56.605 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.591+0000 7f59c259c700 1 --2- 172.21.15.39:0/3209090932 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c51aa7be0 0x558c51ac0c50 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f59b40084c0 tx=0x7f59b400a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8e72fa420376bd9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:56.605 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.591+0000 7f59a6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T19:05:56.605 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.591+0000 7f59a6ff5700 1 -- 172.21.15.39:0/3209090932 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b400a9c0 con 0x558c51aa7be0 2023-07-21T19:05:56.606 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.592+0000 7f59d526db80 1 -- 172.21.15.39:0/3209090932 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f59b80a1d80 con 0x558c51aa7be0 2023-07-21T19:05:56.606 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.592+0000 7f59a6ff5700 1 -- 172.21.15.39:0/3209090932 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f59b4005040 con 0x558c51aa7be0 2023-07-21T19:05:56.606 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.592+0000 7f59a6ff5700 1 -- 172.21.15.39:0/3209090932 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59b4005200 con 0x558c51aa7be0 2023-07-21T19:05:56.606 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.592+0000 7f59d526db80 10 client.17886 fetch_fsmap learned FSMap version 105 2023-07-21T19:05:56.606 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.592+0000 7f59d526db80 1 -- 172.21.15.39:0/3209090932 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=4+}) v3 -- 0x558c51830250 con 0x558c51aa7be0 2023-07-21T19:05:56.606 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.592+0000 7f59a6ff5700 1 -- 172.21.15.39:0/3209090932 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59b4034020 con 0x558c51aa7be0 2023-07-21T19:05:56.606 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.592+0000 7f59a6ff5700 1 -- 172.21.15.39:0/3209090932 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 105) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f59b4027460 con 0x558c51aa7be0 2023-07-21T19:05:56.606 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.592+0000 7f59d526db80 10 client.17886 fetch_fsmap finished waiting for FSMap version 105 2023-07-21T19:05:56.607 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.592+0000 7f59d526db80 10 client.17886 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:05:56.607 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.592+0000 7f59d526db80 20 client.17886 populate_metadata read hostname 'smithi039' 2023-07-21T19:05:56.607 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.592+0000 7f59d526db80 1 --2- 172.21.15.39:0/3209090932 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x558c51ab2580 0x558c51b22a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:56.607 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.592+0000 7f59d526db80 4 client.17886 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:05:56.607 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.592+0000 7f59d526db80 1 -- 172.21.15.39:0/3209090932 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558c519beff0 con 0x558c51ab2580 2023-07-21T19:05:56.607 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.593+0000 7f59c2d9d700 1 --2- 172.21.15.39:0/3209090932 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x558c51ab2580 0x558c51b22a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:56.608 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.593+0000 7f59a6ff5700 10 client.17886 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T19:05:56.608 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.593+0000 7f59c2d9d700 1 --2- 172.21.15.39:0/3209090932 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x558c51ab2580 0x558c51b22a20 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9dc355aa8f86d5 server_cookie=8c75d978e85def3 in_seq=0 out_seq=0 2023-07-21T19:05:56.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:05:56 smithi039 ceph-mon[96489]: pgmap v766: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 12 MiB/s wr, 516 op/s 2023-07-21T19:05:56.732 INFO:tasks.workunit.client.0.smithi039.stdout: 10 349647 27.73 MB/sec execute 549 sec latency 141.588 ms 2023-07-21T19:05:56.802 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.792+0000 7f59a6ff5700 1 -- 172.21.15.39:0/3209090932 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558c519beff0 con 0x558c51ab2580 2023-07-21T19:05:56.803 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.792+0000 7f59a6ff5700 10 client.17886 handle_command_reply: tid=0 2023-07-21T19:05:56.803 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.792+0000 7f59d526db80 1 client.17886 shutdown 2023-07-21T19:05:56.803 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.792+0000 7f59d526db80 1 -- 172.21.15.39:0/3209090932 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c51aa7be0 msgr2=0x558c51ac0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:56.803 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.792+0000 7f59d526db80 1 --2- 172.21.15.39:0/3209090932 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c51aa7be0 0x558c51ac0c50 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f59b40084c0 tx=0x7f59b400a480 comp rx=0 tx=0).stop 2023-07-21T19:05:56.803 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.792+0000 7f59d526db80 1 -- 172.21.15.39:0/3209090932 shutdown_connections 2023-07-21T19:05:56.803 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.792+0000 7f59d526db80 1 --2- 172.21.15.39:0/3209090932 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x558c51ab2580 0x558c51b22a20 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:56.803 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.792+0000 7f59d526db80 1 --2- 172.21.15.39:0/3209090932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c5190e6b0 0x558c51ac29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:56.804 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.792+0000 7f59d526db80 1 --2- 172.21.15.39:0/3209090932 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558c51ac7770 0x558c51ac1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:56.804 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.792+0000 7f59d526db80 1 --2- 172.21.15.39:0/3209090932 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c51aa7be0 0x558c51ac0c50 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:56.804 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.792+0000 7f59d526db80 1 -- 172.21.15.39:0/3209090932 >> 172.21.15.39:0/3209090932 conn(0x558c51ab5f10 msgr2=0x558c51aba760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:56.804 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.793+0000 7f59a6ff5700 0 client.17886 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T19:05:56.804 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.793+0000 7f59d526db80 1 -- 172.21.15.39:0/3209090932 shutdown_connections 2023-07-21T19:05:56.804 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.793+0000 7f59d526db80 1 -- 172.21.15.39:0/3209090932 wait complete. 2023-07-21T19:05:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.793+0000 7f59d526db80 20 client.17886 trim_cache size 0 max 16384 2023-07-21T19:05:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T19:05:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T19:05:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T19:05:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T19:05:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.793+0000 7f59c90e7700 1 -- 172.21.15.39:0/1042139858 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f59ac0a1340 msgr2=0x7f59ac0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.793+0000 7f59c90e7700 1 --2- 172.21.15.39:0/1042139858 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f59ac0a1340 0x7f59ac0a37f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f59bc0095a0 tx=0x7f59bc00d040 comp rx=0 tx=0).stop 2023-07-21T19:05:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.793+0000 7f59c90e7700 1 -- 172.21.15.39:0/1042139858 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59c40febc0 msgr2=0x7f59c419ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.793+0000 7f59c90e7700 1 --2- 172.21.15.39:0/1042139858 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59c40febc0 0x7f59c419ae60 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f59b800b480 tx=0x7f59b800b840 comp rx=0 tx=0).stop 2023-07-21T19:05:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.793+0000 7f59c90e7700 1 -- 172.21.15.39:0/1042139858 shutdown_connections 2023-07-21T19:05:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.793+0000 7f59c90e7700 1 --2- 172.21.15.39:0/1042139858 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f59ac0a1340 0x7f59ac0a37f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.793+0000 7f59c90e7700 1 --2- 172.21.15.39:0/1042139858 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59c4193dc0 0x7f59c4194260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.793+0000 7f59c90e7700 1 --2- 172.21.15.39:0/1042139858 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59c40febc0 0x7f59c419ae60 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.793+0000 7f59c90e7700 1 --2- 172.21.15.39:0/1042139858 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f59c4103260 0x7f59c419b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.793+0000 7f59c90e7700 1 -- 172.21.15.39:0/1042139858 >> 172.21.15.39:0/1042139858 conn(0x7f59c4073d50 msgr2=0x7f59c40fba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:56.808 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.793+0000 7f59c90e7700 1 -- 172.21.15.39:0/1042139858 shutdown_connections 2023-07-21T19:05:56.808 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:05:56.793+0000 7f59c90e7700 1 -- 172.21.15.39:0/1042139858 wait complete. 2023-07-21T19:05:56.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:05:56 smithi193 ceph-mon[98204]: pgmap v766: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 12 MiB/s wr, 516 op/s 2023-07-21T19:05:56.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:05:56 smithi155 ceph-mon[97934]: pgmap v766: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 12 MiB/s wr, 516 op/s 2023-07-21T19:05:57.732 INFO:tasks.workunit.client.0.smithi039.stdout: 10 350205 27.74 MB/sec execute 550 sec latency 185.576 ms 2023-07-21T19:05:58.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:05:58 smithi039 ceph-mon[96489]: pgmap v767: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 16 MiB/s wr, 725 op/s 2023-07-21T19:05:58.732 INFO:tasks.workunit.client.0.smithi039.stdout: 10 350801 27.75 MB/sec execute 551 sec latency 74.341 ms 2023-07-21T19:05:58.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:05:58 smithi193 ceph-mon[98204]: pgmap v767: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 16 MiB/s wr, 725 op/s 2023-07-21T19:05:58.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:05:58 smithi155 ceph-mon[97934]: pgmap v767: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 16 MiB/s wr, 725 op/s 2023-07-21T19:05:59.732 INFO:tasks.workunit.client.0.smithi039.stdout: 10 351314 27.75 MB/sec execute 552 sec latency 79.292 ms 2023-07-21T19:06:00.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:06:00 smithi039 ceph-mon[96489]: pgmap v768: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 12 MiB/s wr, 492 op/s 2023-07-21T19:06:00.732 INFO:tasks.workunit.client.0.smithi039.stdout: 10 351845 27.74 MB/sec execute 553 sec latency 82.269 ms 2023-07-21T19:06:00.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:06:00 smithi193 ceph-mon[98204]: pgmap v768: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 12 MiB/s wr, 492 op/s 2023-07-21T19:06:00.878 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:06:00 smithi155 ceph-mon[97934]: pgmap v768: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 12 MiB/s wr, 492 op/s 2023-07-21T19:06:01.732 INFO:tasks.workunit.client.0.smithi039.stdout: 10 352036 27.71 MB/sec execute 554 sec latency 710.721 ms 2023-07-21T19:06:02.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:06:02 smithi039 ceph-mon[96489]: pgmap v769: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 13 MiB/s wr, 517 op/s 2023-07-21T19:06:02.732 INFO:tasks.workunit.client.0.smithi039.stdout: 10 352036 27.66 MB/sec execute 555 sec latency 1710.780 ms 2023-07-21T19:06:02.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:06:02 smithi193 ceph-mon[98204]: pgmap v769: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 13 MiB/s wr, 517 op/s 2023-07-21T19:06:02.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:06:02 smithi155 ceph-mon[97934]: pgmap v769: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 13 MiB/s wr, 517 op/s 2023-07-21T19:06:03.732 INFO:tasks.workunit.client.0.smithi039.stdout: 10 352153 27.62 MB/sec execute 556 sec latency 2034.386 ms 2023-07-21T19:06:04.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:06:04 smithi039 ceph-mon[96489]: pgmap v770: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 16 MiB/s wr, 704 op/s 2023-07-21T19:06:04.732 INFO:tasks.workunit.client.0.smithi039.stdout: 10 352153 27.57 MB/sec execute 557 sec latency 1488.004 ms 2023-07-21T19:06:04.860 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:06:04 smithi193 ceph-mon[98204]: pgmap v770: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 16 MiB/s wr, 704 op/s 2023-07-21T19:06:04.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:06:04 smithi155 ceph-mon[97934]: pgmap v770: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 16 MiB/s wr, 704 op/s 2023-07-21T19:06:05.733 INFO:tasks.workunit.client.0.smithi039.stdout: 10 352153 27.52 MB/sec execute 558 sec latency 2488.120 ms 2023-07-21T19:06:06.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:06:06 smithi039 ceph-mon[96489]: pgmap v771: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 7.5 MiB/s wr, 430 op/s 2023-07-21T19:06:06.733 INFO:tasks.workunit.client.0.smithi039.stdout: 10 352406 27.50 MB/sec execute 559 sec latency 2959.475 ms 2023-07-21T19:06:06.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:06:06 smithi193 ceph-mon[98204]: pgmap v771: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 7.5 MiB/s wr, 430 op/s 2023-07-21T19:06:06.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:06:06 smithi155 ceph-mon[97934]: pgmap v771: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 7.5 MiB/s wr, 430 op/s 2023-07-21T19:06:07.734 INFO:tasks.workunit.client.0.smithi039.stdout: 10 352770 27.47 MB/sec execute 560 sec latency 159.401 ms 2023-07-21T19:06:08.734 INFO:tasks.workunit.client.0.smithi039.stdout: 10 352970 27.44 MB/sec execute 561 sec latency 175.938 ms 2023-07-21T19:06:08.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:06:08 smithi193 ceph-mon[98204]: pgmap v772: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 7.9 MiB/s wr, 486 op/s 2023-07-21T19:06:08.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:06:08 smithi155 ceph-mon[97934]: pgmap v772: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 7.9 MiB/s wr, 486 op/s 2023-07-21T19:06:08.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:06:08 smithi039 ceph-mon[96489]: pgmap v772: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 7.9 MiB/s wr, 486 op/s 2023-07-21T19:06:09.734 INFO:tasks.workunit.client.0.smithi039.stdout: 10 353160 27.42 MB/sec execute 562 sec latency 1028.370 ms 2023-07-21T19:06:09.993 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:09.983+0000 7fa684a84700 1 -- 172.21.15.39:0/592066686 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa68010e2a0 msgr2=0x7fa68010e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:09.993 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:09.983+0000 7fa684a84700 1 --2- 172.21.15.39:0/592066686 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa68010e2a0 0x7fa68010e670 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fa6740098f0 tx=0x7fa674009c00 comp rx=0 tx=0).stop 2023-07-21T19:06:09.994 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:09.985+0000 7fa684a84700 1 -- 172.21.15.39:0/592066686 shutdown_connections 2023-07-21T19:06:09.994 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:09.985+0000 7fa684a84700 1 --2- 172.21.15.39:0/592066686 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa68006d3a0 0x7fa68011b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:09.994 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:09.985+0000 7fa684a84700 1 --2- 172.21.15.39:0/592066686 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa68010e2a0 0x7fa68010e670 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:09.995 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:09.985+0000 7fa684a84700 1 --2- 172.21.15.39:0/592066686 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa68006c910 0x7fa68006cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:09.995 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:09.985+0000 7fa684a84700 1 -- 172.21.15.39:0/592066686 >> 172.21.15.39:0/592066686 conn(0x7fa68006bd70 msgr2=0x7fa680070f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:10.008 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:09.990+0000 7fa684a84700 1 -- 172.21.15.39:0/592066686 shutdown_connections 2023-07-21T19:06:10.009 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:09.990+0000 7fa684a84700 1 -- 172.21.15.39:0/592066686 wait complete. 2023-07-21T19:06:10.009 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:09.991+0000 7fa684a84700 1 Processor -- start 2023-07-21T19:06:10.009 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:09.991+0000 7fa684a84700 1 -- start start 2023-07-21T19:06:10.009 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:09.991+0000 7fa684a84700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa68006c910 0x7fa6801aab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:10.009 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:09.991+0000 7fa684a84700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa68006d3a0 0x7fa6801ab0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:10.009 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:09.991+0000 7fa684a84700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa68010e2a0 0x7fa6801a4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:10.009 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:09.991+0000 7fa684a84700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6801a5110 con 0x7fa68006d3a0 2023-07-21T19:06:10.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:09.991+0000 7fa684a84700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6801a5280 con 0x7fa68010e2a0 2023-07-21T19:06:10.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:09.991+0000 7fa684a84700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6801a5440 con 0x7fa68006c910 2023-07-21T19:06:10.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:09.992+0000 7fa67e59c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa68006c910 0x7fa6801aab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:10.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:09.992+0000 7fa67e59c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa68006c910 0x7fa6801aab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:53564/0 (socket says 172.21.15.39:53564) 2023-07-21T19:06:10.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:09.992+0000 7fa67e59c700 1 -- 172.21.15.39:0/602090520 learned_addr learned my addr 172.21.15.39:0/602090520 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:06:10.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:09.992+0000 7fa67dd9b700 1 --2- 172.21.15.39:0/602090520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa68006d3a0 0x7fa6801ab0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:10.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:09.992+0000 7fa67dd9b700 1 -- 172.21.15.39:0/602090520 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa68006c910 msgr2=0x7fa6801aab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:10.011 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:09.992+0000 7fa67dd9b700 1 --2- 172.21.15.39:0/602090520 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa68006c910 0x7fa6801aab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:10.011 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:09.992+0000 7fa67dd9b700 1 -- 172.21.15.39:0/602090520 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa68010e2a0 msgr2=0x7fa6801a4940 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-21T19:06:10.011 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:09.992+0000 7fa67dd9b700 1 --2- 172.21.15.39:0/602090520 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa68010e2a0 0x7fa6801a4940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:10.011 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:09.992+0000 7fa67dd9b700 1 -- 172.21.15.39:0/602090520 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa674009520 con 0x7fa68006d3a0 2023-07-21T19:06:10.011 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:09.992+0000 7fa67dd9b700 1 --2- 172.21.15.39:0/602090520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa68006d3a0 0x7fa6801ab0b0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fa678007fd0 tx=0x7fa67800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:10.011 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:09.993+0000 7fa66f7fe700 1 -- 172.21.15.39:0/602090520 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa67800dd50 con 0x7fa68006d3a0 2023-07-21T19:06:10.011 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:09.993+0000 7fa66f7fe700 1 -- 172.21.15.39:0/602090520 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa67800f040 con 0x7fa68006d3a0 2023-07-21T19:06:10.011 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:09.993+0000 7fa66f7fe700 1 -- 172.21.15.39:0/602090520 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa678014600 con 0x7fa68006d3a0 2023-07-21T19:06:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:09.993+0000 7fa684a84700 1 -- 172.21.15.39:0/602090520 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6801a5720 con 0x7fa68006d3a0 2023-07-21T19:06:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:09.993+0000 7fa684a84700 1 -- 172.21.15.39:0/602090520 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa680078950 con 0x7fa68006d3a0 2023-07-21T19:06:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:09.998+0000 7fa690c0ab80 1 Processor -- start 2023-07-21T19:06:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:09.999+0000 7fa690c0ab80 1 -- start start 2023-07-21T19:06:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:09.999+0000 7fa690c0ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4ef5e26b0 0x55d4ef78bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:09.999+0000 7fa690c0ab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d4ef78c2d0 0x55d4ef786040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:09.999+0000 7fa690c0ab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4ef786580 0x55d4ef7869d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:09.999+0000 7fa690c0ab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4ef68bc50 con 0x55d4ef786580 2023-07-21T19:06:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:09.999+0000 7fa690c0ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4ef6de9e0 con 0x55d4ef5e26b0 2023-07-21T19:06:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:09.999+0000 7fa690c0ab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4ef6d7e70 con 0x55d4ef78c2d0 2023-07-21T19:06:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:09.999+0000 7fa67dd9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4ef786580 0x55d4ef7869d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:10.015 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:09.999+0000 7fa67dd9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4ef786580 0x55d4ef7869d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35020/0 (socket says 172.21.15.39:35020) 2023-07-21T19:06:10.015 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:09.999+0000 7fa67dd9b700 1 -- 172.21.15.39:0/2344656457 learned_addr learned my addr 172.21.15.39:0/2344656457 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:06:10.015 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.000+0000 7fa67dd9b700 1 -- 172.21.15.39:0/2344656457 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d4ef78c2d0 msgr2=0x55d4ef786040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T19:06:10.015 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.000+0000 7fa67dd9b700 1 --2- 172.21.15.39:0/2344656457 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d4ef78c2d0 0x55d4ef786040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:10.015 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.000+0000 7fa67dd9b700 1 -- 172.21.15.39:0/2344656457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4ef5e26b0 msgr2=0x55d4ef78bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:06:10.015 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.000+0000 7fa67dd9b700 1 --2- 172.21.15.39:0/2344656457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4ef5e26b0 0x55d4ef78bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.000+0000 7fa67dd9b700 1 -- 172.21.15.39:0/2344656457 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d4ef505d80 con 0x55d4ef786580 2023-07-21T19:06:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.000+0000 7fa67dd9b700 1 --2- 172.21.15.39:0/2344656457 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4ef786580 0x55d4ef7869d0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fa678015c60 tx=0x7fa678004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28160b5b5e2f85b8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.005+0000 7fa66f7fe700 1 -- 172.21.15.39:0/602090520 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fa678015610 con 0x7fa68006d3a0 2023-07-21T19:06:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.005+0000 7fa66cff9700 1 -- 172.21.15.39:0/2344656457 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6780cf950 con 0x55d4ef786580 2023-07-21T19:06:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.005+0000 7fa66cff9700 1 -- 172.21.15.39:0/2344656457 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa6780d0160 con 0x55d4ef786580 2023-07-21T19:06:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.005+0000 7fa66cff9700 1 -- 172.21.15.39:0/2344656457 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6780d1370 con 0x55d4ef786580 2023-07-21T19:06:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.005+0000 7fa66f7fe700 1 --2- 172.21.15.39:0/602090520 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fa6680a1450 0x7fa6680a3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.006+0000 7fa66f7fe700 1 -- 172.21.15.39:0/602090520 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fa6780cf5c0 con 0x7fa68006d3a0 2023-07-21T19:06:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.006+0000 7fa67ed9d700 1 --2- 172.21.15.39:0/602090520 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fa6680a1450 0x7fa6680a3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:10.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.007+0000 7fa690c0ab80 1 -- 172.21.15.39:0/2344656457 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4ef786580 msgr2=0x55d4ef7869d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:10.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.007+0000 7fa690c0ab80 1 --2- 172.21.15.39:0/2344656457 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4ef786580 0x55d4ef7869d0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fa678015c60 tx=0x7fa678004b50 comp rx=0 tx=0).stop 2023-07-21T19:06:10.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.007+0000 7fa690c0ab80 1 -- 172.21.15.39:0/2344656457 shutdown_connections 2023-07-21T19:06:10.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.007+0000 7fa690c0ab80 1 --2- 172.21.15.39:0/2344656457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4ef5e26b0 0x55d4ef78bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:10.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.007+0000 7fa690c0ab80 1 --2- 172.21.15.39:0/2344656457 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4ef786580 0x55d4ef7869d0 secure :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fa678015c60 tx=0x7fa678004b50 comp rx=0 tx=0).stop 2023-07-21T19:06:10.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.007+0000 7fa690c0ab80 1 --2- 172.21.15.39:0/2344656457 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d4ef78c2d0 0x55d4ef786040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:10.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.007+0000 7fa690c0ab80 1 -- 172.21.15.39:0/2344656457 >> 172.21.15.39:0/2344656457 conn(0x55d4ef77bbe0 msgr2=0x55d4ef780f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:10.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.007+0000 7fa67ed9d700 1 --2- 172.21.15.39:0/602090520 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fa6680a1450 0x7fa6680a3900 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa6801ab800 tx=0x7fa670007bb0 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.023+0000 7fa690c0ab80 1 -- 172.21.15.39:0/2344656457 shutdown_connections 2023-07-21T19:06:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.026+0000 7fa690c0ab80 1 -- 172.21.15.39:0/2344656457 wait complete. 2023-07-21T19:06:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.026+0000 7fa690c0ab80 1 -- start start 2023-07-21T19:06:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.026+0000 7fa690c0ab80 1 Processor -- start 2023-07-21T19:06:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.027+0000 7fa690c0ab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d4ef5e26b0 0x55d4ef78fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.027+0000 7fa690c0ab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4ef78c2d0 0x55d4ef790160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.027+0000 7fa690c0ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4ef7906a0 0x55d4ef79bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.027+0000 7fa690c0ab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4ef677a60 con 0x55d4ef78c2d0 2023-07-21T19:06:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.027+0000 7fa690c0ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4ef677760 con 0x55d4ef7906a0 2023-07-21T19:06:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.027+0000 7fa690c0ab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4ef55b8c0 con 0x55d4ef5e26b0 2023-07-21T19:06:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.027+0000 7fa690c0ab80 10 client.0 _ll_register_callbacks cb 0x55d4ef675700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:06:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.027+0000 7fa67ed9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4ef7906a0 0x55d4ef79bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.027+0000 7fa67ed9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4ef7906a0 0x55d4ef79bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:47622/0 (socket says 172.21.15.39:47622) 2023-07-21T19:06:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.027+0000 7fa67ed9d700 1 -- 172.21.15.39:0/631511033 learned_addr learned my addr 172.21.15.39:0/631511033 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:06:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.028+0000 7fa67ed9d700 1 -- 172.21.15.39:0/631511033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d4ef5e26b0 msgr2=0x55d4ef78fc20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T19:06:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.028+0000 7fa67ed9d700 1 --2- 172.21.15.39:0/631511033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d4ef5e26b0 0x55d4ef78fc20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.028+0000 7fa67ed9d700 1 -- 172.21.15.39:0/631511033 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4ef78c2d0 msgr2=0x55d4ef790160 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T19:06:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.028+0000 7fa67ed9d700 1 --2- 172.21.15.39:0/631511033 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4ef78c2d0 0x55d4ef790160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.028+0000 7fa67ed9d700 1 -- 172.21.15.39:0/631511033 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa680078950 con 0x55d4ef7906a0 2023-07-21T19:06:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.028+0000 7fa67ed9d700 1 --2- 172.21.15.39:0/631511033 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4ef7906a0 0x55d4ef79bc50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa670016f70 tx=0x7fa670000f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49e5a5cc603ae503 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.028+0000 7fa665ffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-21T19:06:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.028+0000 7fa665ffb700 1 -- 172.21.15.39:0/631511033 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa67001b070 con 0x55d4ef7906a0 2023-07-21T19:06:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.028+0000 7fa690c0ab80 1 -- 172.21.15.39:0/631511033 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d4ef631490 con 0x55d4ef7906a0 2023-07-21T19:06:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.028+0000 7fa665ffb700 1 -- 172.21.15.39:0/631511033 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fa670013e10 con 0x55d4ef7906a0 2023-07-21T19:06:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.028+0000 7fa665ffb700 1 -- 172.21.15.39:0/631511033 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa6700160d0 con 0x55d4ef7906a0 2023-07-21T19:06:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.028+0000 7fa690c0ab80 10 client.27314 fetch_fsmap learned FSMap version 105 2023-07-21T19:06:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.028+0000 7fa690c0ab80 1 -- 172.21.15.39:0/631511033 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=4+}) v3 -- 0x55d4ef4eec00 con 0x55d4ef7906a0 2023-07-21T19:06:10.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.029+0000 7fa665ffb700 1 -- 172.21.15.39:0/631511033 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa67002b020 con 0x55d4ef7906a0 2023-07-21T19:06:10.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.029+0000 7fa665ffb700 1 -- 172.21.15.39:0/631511033 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 105) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa670014420 con 0x55d4ef7906a0 2023-07-21T19:06:10.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.029+0000 7fa690c0ab80 10 client.27314 fetch_fsmap finished waiting for FSMap version 105 2023-07-21T19:06:10.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.029+0000 7fa690c0ab80 10 client.27314 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:06:10.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.029+0000 7fa690c0ab80 20 client.27314 populate_metadata read hostname 'smithi039' 2023-07-21T19:06:10.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.029+0000 7fa690c0ab80 1 --2- 172.21.15.39:0/631511033 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d4ef7a02e0 0x55d4ef7a26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:10.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.029+0000 7fa690c0ab80 4 client.27314 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:06:10.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.029+0000 7fa690c0ab80 1 -- 172.21.15.39:0/631511033 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d4ef6d0e50 con 0x55d4ef7a02e0 2023-07-21T19:06:10.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.029+0000 7fa67e59c700 1 --2- 172.21.15.39:0/631511033 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d4ef7a02e0 0x55d4ef7a26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.045+0000 7fa665ffb700 10 client.27314 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T19:06:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.045+0000 7fa67e59c700 1 --2- 172.21.15.39:0/631511033 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d4ef7a02e0 0x55d4ef7a26f0 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=492a4330c69a8f2a server_cookie=d22a94e60a84248d in_seq=0 out_seq=0 2023-07-21T19:06:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.046+0000 7fa665ffb700 1 -- 172.21.15.39:0/631511033 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d4ef6d0e50 con 0x55d4ef7a02e0 2023-07-21T19:06:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.046+0000 7fa665ffb700 10 client.27314 handle_command_reply: tid=0 2023-07-21T19:06:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.046+0000 7fa690c0ab80 1 client.27314 shutdown 2023-07-21T19:06:10.061 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.051+0000 7fa690c0ab80 1 -- 172.21.15.39:0/631511033 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4ef7906a0 msgr2=0x55d4ef79bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:10.061 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.051+0000 7fa690c0ab80 1 --2- 172.21.15.39:0/631511033 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4ef7906a0 0x55d4ef79bc50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa670016f70 tx=0x7fa670000f70 comp rx=0 tx=0).stop 2023-07-21T19:06:10.061 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.051+0000 7fa690c0ab80 1 -- 172.21.15.39:0/631511033 shutdown_connections 2023-07-21T19:06:10.061 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.051+0000 7fa690c0ab80 1 --2- 172.21.15.39:0/631511033 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d4ef7a02e0 0x55d4ef7a26f0 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.051+0000 7fa690c0ab80 1 --2- 172.21.15.39:0/631511033 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4ef7906a0 0x55d4ef79bc50 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.051+0000 7fa690c0ab80 1 --2- 172.21.15.39:0/631511033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d4ef5e26b0 0x55d4ef78fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.051+0000 7fa690c0ab80 1 --2- 172.21.15.39:0/631511033 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4ef78c2d0 0x55d4ef790160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.051+0000 7fa690c0ab80 1 -- 172.21.15.39:0/631511033 >> 172.21.15.39:0/631511033 conn(0x55d4ef77bbe0 msgr2=0x55d4ef7804e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.051+0000 7fa665ffb700 0 client.27314 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T19:06:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.051+0000 7fa690c0ab80 1 -- 172.21.15.39:0/631511033 shutdown_connections 2023-07-21T19:06:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.051+0000 7fa690c0ab80 1 -- 172.21.15.39:0/631511033 wait complete. 2023-07-21T19:06:10.066 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.053+0000 7fa690c0ab80 20 client.27314 trim_cache size 0 max 16384 2023-07-21T19:06:10.091 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.081+0000 7fa690c0ab80 1 Processor -- start 2023-07-21T19:06:10.093 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.082+0000 7fa690c0ab80 1 -- start start 2023-07-21T19:06:10.093 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.082+0000 7fa690c0ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4ef5e26b0 0x55d4ef786040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:10.094 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.082+0000 7fa690c0ab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d4ef78c2d0 0x55d4ef796af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:10.094 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.082+0000 7fa690c0ab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4ef7906a0 0x55d4ef780c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:10.094 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.082+0000 7fa690c0ab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4ef74c060 con 0x55d4ef7906a0 2023-07-21T19:06:10.095 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.082+0000 7fa690c0ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4ef790bd0 con 0x55d4ef5e26b0 2023-07-21T19:06:10.095 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.082+0000 7fa690c0ab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4ef781f00 con 0x55d4ef78c2d0 2023-07-21T19:06:10.095 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.083+0000 7fa67e59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4ef5e26b0 0x55d4ef786040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:10.095 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.083+0000 7fa67e59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4ef5e26b0 0x55d4ef786040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:47630/0 (socket says 172.21.15.39:47630) 2023-07-21T19:06:10.095 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.083+0000 7fa67e59c700 1 -- 172.21.15.39:0/1630445529 learned_addr learned my addr 172.21.15.39:0/1630445529 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:06:10.096 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.083+0000 7fa67ed9d700 1 --2- 172.21.15.39:0/1630445529 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4ef7906a0 0x55d4ef780c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:10.096 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.083+0000 7fa67e59c700 1 -- 172.21.15.39:0/1630445529 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d4ef78c2d0 msgr2=0x55d4ef796af0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T19:06:10.096 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.083+0000 7fa67e59c700 1 --2- 172.21.15.39:0/1630445529 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d4ef78c2d0 0x55d4ef796af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:10.096 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.083+0000 7fa67e59c700 1 -- 172.21.15.39:0/1630445529 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4ef7906a0 msgr2=0x55d4ef780c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:06:10.097 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.083+0000 7fa67e59c700 1 --2- 172.21.15.39:0/1630445529 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4ef7906a0 0x55d4ef780c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:10.097 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.083+0000 7fa67e59c700 1 -- 172.21.15.39:0/1630445529 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d4ef52c1f0 con 0x55d4ef5e26b0 2023-07-21T19:06:10.097 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.084+0000 7fa67e59c700 1 --2- 172.21.15.39:0/1630445529 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4ef5e26b0 0x55d4ef786040 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa674005dc0 tx=0x7fa674005df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f71aa3020f2135e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:10.098 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.086+0000 7fa66d7fa700 1 -- 172.21.15.39:0/1630445529 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa67401c030 con 0x55d4ef5e26b0 2023-07-21T19:06:10.098 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.086+0000 7fa66d7fa700 1 -- 172.21.15.39:0/1630445529 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa67401d040 con 0x55d4ef5e26b0 2023-07-21T19:06:10.098 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.086+0000 7fa66d7fa700 1 -- 172.21.15.39:0/1630445529 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6740254e0 con 0x55d4ef5e26b0 2023-07-21T19:06:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.088+0000 7fa690c0ab80 1 -- 172.21.15.39:0/1630445529 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4ef5e26b0 msgr2=0x55d4ef786040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.088+0000 7fa690c0ab80 1 --2- 172.21.15.39:0/1630445529 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4ef5e26b0 0x55d4ef786040 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa674005dc0 tx=0x7fa674005df0 comp rx=0 tx=0).stop 2023-07-21T19:06:10.106 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.097+0000 7fa690c0ab80 1 -- 172.21.15.39:0/1630445529 shutdown_connections 2023-07-21T19:06:10.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.097+0000 7fa690c0ab80 1 --2- 172.21.15.39:0/1630445529 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4ef5e26b0 0x55d4ef786040 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:10.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.097+0000 7fa690c0ab80 1 --2- 172.21.15.39:0/1630445529 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4ef7906a0 0x55d4ef780c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:10.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.097+0000 7fa690c0ab80 1 --2- 172.21.15.39:0/1630445529 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d4ef78c2d0 0x55d4ef796af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:10.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.097+0000 7fa690c0ab80 1 -- 172.21.15.39:0/1630445529 >> 172.21.15.39:0/1630445529 conn(0x55d4ef77bbe0 msgr2=0x55d4ef78fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:10.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.097+0000 7fa690c0ab80 1 -- 172.21.15.39:0/1630445529 shutdown_connections 2023-07-21T19:06:10.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.097+0000 7fa690c0ab80 1 -- 172.21.15.39:0/1630445529 wait complete. 2023-07-21T19:06:10.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.097+0000 7fa690c0ab80 1 -- start start 2023-07-21T19:06:10.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.097+0000 7fa690c0ab80 1 Processor -- start 2023-07-21T19:06:10.112 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.098+0000 7fa690c0ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4ef5e26b0 0x55d4ef780aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:10.112 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.098+0000 7fa690c0ab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4ef78c2d0 0x55d4ef780fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:10.112 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.098+0000 7fa690c0ab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d4ef7906a0 0x55d4ef787860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:10.112 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.098+0000 7fa690c0ab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4ef787da0 con 0x55d4ef78c2d0 2023-07-21T19:06:10.112 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.098+0000 7fa690c0ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4ef787f10 con 0x55d4ef5e26b0 2023-07-21T19:06:10.112 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.098+0000 7fa690c0ab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4ef7880d0 con 0x55d4ef7906a0 2023-07-21T19:06:10.113 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.098+0000 7fa690c0ab80 10 client.0 _ll_register_callbacks cb 0x7fa670016940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:06:10.113 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.098+0000 7fa67e59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4ef5e26b0 0x55d4ef780aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:10.113 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.098+0000 7fa67e59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4ef5e26b0 0x55d4ef780aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:47636/0 (socket says 172.21.15.39:47636) 2023-07-21T19:06:10.113 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.098+0000 7fa67e59c700 1 -- 172.21.15.39:0/4235394254 learned_addr learned my addr 172.21.15.39:0/4235394254 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:06:10.113 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.098+0000 7fa67ed9d700 1 --2- 172.21.15.39:0/4235394254 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d4ef7906a0 0x55d4ef787860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:10.113 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.099+0000 7fa67e59c700 1 -- 172.21.15.39:0/4235394254 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d4ef7906a0 msgr2=0x55d4ef787860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:06:10.113 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.099+0000 7fa67e59c700 1 --2- 172.21.15.39:0/4235394254 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d4ef7906a0 0x55d4ef787860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:10.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.099+0000 7fa67e59c700 1 -- 172.21.15.39:0/4235394254 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4ef78c2d0 msgr2=0x55d4ef780fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:06:10.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.099+0000 7fa67e59c700 1 --2- 172.21.15.39:0/4235394254 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4ef78c2d0 0x55d4ef780fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:10.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.099+0000 7fa67e59c700 1 -- 172.21.15.39:0/4235394254 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa68011bfb0 con 0x55d4ef5e26b0 2023-07-21T19:06:10.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.099+0000 7fa665ffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-21T19:06:10.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.099+0000 7fa67e59c700 1 --2- 172.21.15.39:0/4235394254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4ef5e26b0 0x55d4ef780aa0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa674009230 tx=0x7fa6740054a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a3ef3d4ecfa2fcc server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:10.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.100+0000 7fa665ffb700 1 -- 172.21.15.39:0/4235394254 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa67401c040 con 0x55d4ef5e26b0 2023-07-21T19:06:10.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.100+0000 7fa665ffb700 1 -- 172.21.15.39:0/4235394254 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fa674005290 con 0x55d4ef5e26b0 2023-07-21T19:06:10.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.100+0000 7fa690c0ab80 1 -- 172.21.15.39:0/4235394254 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa6780d0160 con 0x55d4ef5e26b0 2023-07-21T19:06:10.115 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.100+0000 7fa665ffb700 1 -- 172.21.15.39:0/4235394254 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa6740250e0 con 0x55d4ef5e26b0 2023-07-21T19:06:10.115 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.100+0000 7fa690c0ab80 10 client.27320 fetch_fsmap learned FSMap version 105 2023-07-21T19:06:10.115 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.100+0000 7fa690c0ab80 1 -- 172.21.15.39:0/4235394254 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=4+}) v3 -- 0x55d4ef504250 con 0x55d4ef5e26b0 2023-07-21T19:06:10.115 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.100+0000 7fa665ffb700 1 -- 172.21.15.39:0/4235394254 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa6780d0160 con 0x55d4ef5e26b0 2023-07-21T19:06:10.115 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.100+0000 7fa665ffb700 1 -- 172.21.15.39:0/4235394254 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 105) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa674015020 con 0x55d4ef5e26b0 2023-07-21T19:06:10.115 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.100+0000 7fa690c0ab80 10 client.27320 fetch_fsmap finished waiting for FSMap version 105 2023-07-21T19:06:10.116 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.100+0000 7fa690c0ab80 10 client.27320 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:06:10.116 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.100+0000 7fa690c0ab80 20 client.27320 populate_metadata read hostname 'smithi039' 2023-07-21T19:06:10.116 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.101+0000 7fa690c0ab80 1 --2- 172.21.15.39:0/4235394254 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d4ef7f4c20 0x55d4ef7f7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:10.116 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.101+0000 7fa690c0ab80 4 client.27320 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:06:10.116 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.101+0000 7fa690c0ab80 1 -- 172.21.15.39:0/4235394254 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d4ef692ff0 con 0x55d4ef7f4c20 2023-07-21T19:06:10.121 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.112+0000 7fa67dd9b700 1 --2- 172.21.15.39:0/4235394254 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d4ef7f4c20 0x55d4ef7f7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:10.129 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.119+0000 7fa67dd9b700 1 --2- 172.21.15.39:0/4235394254 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d4ef7f4c20 0x55d4ef7f7030 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f47d0a72a891d31 server_cookie=34f0a39171437629 in_seq=0 out_seq=0 2023-07-21T19:06:10.130 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.119+0000 7fa665ffb700 10 client.27320 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T19:06:10.450 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.440+0000 7fa665ffb700 1 -- 172.21.15.39:0/4235394254 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d4ef692ff0 con 0x55d4ef7f4c20 2023-07-21T19:06:10.450 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.440+0000 7fa665ffb700 10 client.27320 handle_command_reply: tid=0 2023-07-21T19:06:10.450 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.440+0000 7fa690c0ab80 1 client.27320 shutdown 2023-07-21T19:06:10.451 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.440+0000 7fa690c0ab80 1 -- 172.21.15.39:0/4235394254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4ef5e26b0 msgr2=0x55d4ef780aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:10.451 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.440+0000 7fa690c0ab80 1 --2- 172.21.15.39:0/4235394254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4ef5e26b0 0x55d4ef780aa0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa674009230 tx=0x7fa6740054a0 comp rx=0 tx=0).stop 2023-07-21T19:06:10.451 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.440+0000 7fa690c0ab80 1 -- 172.21.15.39:0/4235394254 shutdown_connections 2023-07-21T19:06:10.451 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.440+0000 7fa690c0ab80 1 --2- 172.21.15.39:0/4235394254 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55d4ef7f4c20 0x55d4ef7f7030 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:10.451 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.440+0000 7fa690c0ab80 1 --2- 172.21.15.39:0/4235394254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4ef5e26b0 0x55d4ef780aa0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:10.451 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.440+0000 7fa690c0ab80 1 --2- 172.21.15.39:0/4235394254 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d4ef7906a0 0x55d4ef787860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:10.451 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.440+0000 7fa690c0ab80 1 --2- 172.21.15.39:0/4235394254 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4ef78c2d0 0x55d4ef780fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:10.452 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.440+0000 7fa690c0ab80 1 -- 172.21.15.39:0/4235394254 >> 172.21.15.39:0/4235394254 conn(0x55d4ef77bbe0 msgr2=0x55d4ef789d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:10.452 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.440+0000 7fa665ffb700 0 client.27320 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T19:06:10.452 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.440+0000 7fa690c0ab80 1 -- 172.21.15.39:0/4235394254 shutdown_connections 2023-07-21T19:06:10.452 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.441+0000 7fa690c0ab80 1 -- 172.21.15.39:0/4235394254 wait complete. 2023-07-21T19:06:10.454 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T19:06:10.455 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T19:06:10.455 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T19:06:10.455 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T19:06:10.455 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.445+0000 7fa690c0ab80 20 client.27320 trim_cache size 0 max 16384 2023-07-21T19:06:10.458 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.448+0000 7fa684a84700 1 -- 172.21.15.39:0/602090520 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fa6680a1450 msgr2=0x7fa6680a3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:10.458 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.448+0000 7fa684a84700 1 --2- 172.21.15.39:0/602090520 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fa6680a1450 0x7fa6680a3900 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa6801ab800 tx=0x7fa670007bb0 comp rx=0 tx=0).stop 2023-07-21T19:06:10.458 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.448+0000 7fa684a84700 1 -- 172.21.15.39:0/602090520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa68006d3a0 msgr2=0x7fa6801ab0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:10.458 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.448+0000 7fa684a84700 1 --2- 172.21.15.39:0/602090520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa68006d3a0 0x7fa6801ab0b0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fa678007fd0 tx=0x7fa67800d3b0 comp rx=0 tx=0).stop 2023-07-21T19:06:10.458 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.448+0000 7fa684a84700 1 -- 172.21.15.39:0/602090520 shutdown_connections 2023-07-21T19:06:10.458 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.448+0000 7fa684a84700 1 --2- 172.21.15.39:0/602090520 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fa6680a1450 0x7fa6680a3900 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:10.459 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.448+0000 7fa684a84700 1 --2- 172.21.15.39:0/602090520 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa68010e2a0 0x7fa6801a4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:10.459 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.448+0000 7fa684a84700 1 --2- 172.21.15.39:0/602090520 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa68006c910 0x7fa6801aab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:10.459 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.448+0000 7fa684a84700 1 --2- 172.21.15.39:0/602090520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa68006d3a0 0x7fa6801ab0b0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:10.459 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.448+0000 7fa684a84700 1 -- 172.21.15.39:0/602090520 >> 172.21.15.39:0/602090520 conn(0x7fa68006bd70 msgr2=0x7fa680070770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:10.459 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.448+0000 7fa684a84700 1 -- 172.21.15.39:0/602090520 shutdown_connections 2023-07-21T19:06:10.459 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:10.448+0000 7fa684a84700 1 -- 172.21.15.39:0/602090520 wait complete. 2023-07-21T19:06:10.734 INFO:tasks.workunit.client.0.smithi039.stdout: 10 353253 27.38 MB/sec execute 563 sec latency 1634.959 ms 2023-07-21T19:06:10.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:06:10 smithi193 ceph-mon[98204]: pgmap v773: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.0 MiB/s wr, 277 op/s 2023-07-21T19:06:10.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:06:10 smithi193 ceph-mon[98204]: osdmap e121: 12 total, 12 up, 12 in 2023-07-21T19:06:10.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:06:10 smithi155 ceph-mon[97934]: pgmap v773: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.0 MiB/s wr, 277 op/s 2023-07-21T19:06:10.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:06:10 smithi155 ceph-mon[97934]: osdmap e121: 12 total, 12 up, 12 in 2023-07-21T19:06:10.926 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:06:10 smithi039 ceph-mon[96489]: pgmap v773: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.0 MiB/s wr, 277 op/s 2023-07-21T19:06:10.926 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:06:10 smithi039 ceph-mon[96489]: osdmap e121: 12 total, 12 up, 12 in 2023-07-21T19:06:11.734 INFO:tasks.workunit.client.0.smithi039.stdout: 10 353522 27.36 MB/sec execute 564 sec latency 605.539 ms 2023-07-21T19:06:12.735 INFO:tasks.workunit.client.0.smithi039.stdout: 10 354011 27.37 MB/sec execute 565 sec latency 141.675 ms 2023-07-21T19:06:12.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:06:12 smithi193 ceph-mon[98204]: pgmap v775: 97 pgs: 3 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 92 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.8 MiB/s wr, 337 op/s 2023-07-21T19:06:12.880 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:06:12 smithi155 ceph-mon[97934]: pgmap v775: 97 pgs: 3 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 92 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.8 MiB/s wr, 337 op/s 2023-07-21T19:06:12.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:06:12 smithi039 ceph-mon[96489]: pgmap v775: 97 pgs: 3 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 92 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.8 MiB/s wr, 337 op/s 2023-07-21T19:06:13.739 INFO:tasks.workunit.client.0.smithi039.stdout: 10 354381 27.36 MB/sec execute 566 sec latency 187.413 ms 2023-07-21T19:06:14.735 INFO:tasks.workunit.client.0.smithi039.stdout: 10 354808 27.35 MB/sec execute 567 sec latency 93.299 ms 2023-07-21T19:06:14.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:06:14 smithi193 ceph-mon[98204]: pgmap v776: 97 pgs: 29 active+clean+snaptrim_wait, 14 active+clean+snaptrim, 54 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.2 MiB/s wr, 411 op/s 2023-07-21T19:06:14.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:06:14 smithi155 ceph-mon[97934]: pgmap v776: 97 pgs: 29 active+clean+snaptrim_wait, 14 active+clean+snaptrim, 54 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.2 MiB/s wr, 411 op/s 2023-07-21T19:06:14.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:06:14 smithi039 ceph-mon[96489]: pgmap v776: 97 pgs: 29 active+clean+snaptrim_wait, 14 active+clean+snaptrim, 54 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.2 MiB/s wr, 411 op/s 2023-07-21T19:06:15.735 INFO:tasks.workunit.client.0.smithi039.stdout: 10 355340 27.33 MB/sec execute 568 sec latency 78.971 ms 2023-07-21T19:06:16.735 INFO:tasks.workunit.client.0.smithi039.stdout: 10 355931 27.36 MB/sec execute 569 sec latency 98.021 ms 2023-07-21T19:06:16.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:06:16 smithi193 ceph-mon[98204]: pgmap v777: 97 pgs: 29 active+clean+snaptrim_wait, 14 active+clean+snaptrim, 54 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.2 MiB/s wr, 411 op/s 2023-07-21T19:06:16.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:06:16 smithi155 ceph-mon[97934]: pgmap v777: 97 pgs: 29 active+clean+snaptrim_wait, 14 active+clean+snaptrim, 54 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.2 MiB/s wr, 411 op/s 2023-07-21T19:06:16.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:06:16 smithi039 ceph-mon[96489]: pgmap v777: 97 pgs: 29 active+clean+snaptrim_wait, 14 active+clean+snaptrim, 54 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.2 MiB/s wr, 411 op/s 2023-07-21T19:06:17.735 INFO:tasks.workunit.client.0.smithi039.stdout: 10 356532 27.35 MB/sec execute 570 sec latency 95.998 ms 2023-07-21T19:06:18.735 INFO:tasks.workunit.client.0.smithi039.stdout: 10 356993 27.36 MB/sec execute 571 sec latency 99.761 ms 2023-07-21T19:06:18.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:06:18 smithi193 ceph-mon[98204]: pgmap v778: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 7.8 MiB/s wr, 579 op/s 2023-07-21T19:06:18.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:06:18 smithi155 ceph-mon[97934]: pgmap v778: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 7.8 MiB/s wr, 579 op/s 2023-07-21T19:06:18.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:06:18 smithi039 ceph-mon[96489]: pgmap v778: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 7.8 MiB/s wr, 579 op/s 2023-07-21T19:06:19.735 INFO:tasks.workunit.client.0.smithi039.stdout: 10 357599 27.38 MB/sec execute 572 sec latency 90.743 ms 2023-07-21T19:06:20.735 INFO:tasks.workunit.client.0.smithi039.stdout: 10 357994 27.37 MB/sec execute 573 sec latency 130.251 ms 2023-07-21T19:06:20.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:06:20 smithi193 ceph-mon[98204]: pgmap v779: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 7.8 MiB/s wr, 579 op/s 2023-07-21T19:06:20.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:06:20 smithi155 ceph-mon[97934]: pgmap v779: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 7.8 MiB/s wr, 579 op/s 2023-07-21T19:06:20.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:06:20 smithi039 ceph-mon[96489]: pgmap v779: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 7.8 MiB/s wr, 579 op/s 2023-07-21T19:06:21.735 INFO:tasks.workunit.client.0.smithi039.stdout: 10 358364 27.35 MB/sec execute 574 sec latency 172.793 ms 2023-07-21T19:06:22.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:06:22 smithi039 ceph-mon[96489]: pgmap v780: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 7.3 MiB/s wr, 531 op/s 2023-07-21T19:06:22.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:06:22 smithi039 ceph-mon[96489]: osdmap e122: 12 total, 12 up, 12 in 2023-07-21T19:06:22.735 INFO:tasks.workunit.client.0.smithi039.stdout: 10 358783 27.32 MB/sec execute 575 sec latency 84.502 ms 2023-07-21T19:06:22.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:06:22 smithi193 ceph-mon[98204]: pgmap v780: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 7.3 MiB/s wr, 531 op/s 2023-07-21T19:06:22.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:06:22 smithi193 ceph-mon[98204]: osdmap e122: 12 total, 12 up, 12 in 2023-07-21T19:06:22.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:06:22 smithi155 ceph-mon[97934]: pgmap v780: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 7.3 MiB/s wr, 531 op/s 2023-07-21T19:06:22.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:06:22 smithi155 ceph-mon[97934]: osdmap e122: 12 total, 12 up, 12 in 2023-07-21T19:06:23.696 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.685+0000 7f5499210700 1 -- 172.21.15.39:0/3327776331 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f54940ff470 msgr2=0x7f549410b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:23.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.685+0000 7f5499210700 1 --2- 172.21.15.39:0/3327776331 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f54940ff470 0x7f549410b900 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f548c009f90 tx=0x7f548c00b330 comp rx=0 tx=0).stop 2023-07-21T19:06:23.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.686+0000 7f5499210700 1 -- 172.21.15.39:0/3327776331 shutdown_connections 2023-07-21T19:06:23.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.686+0000 7f5499210700 1 --2- 172.21.15.39:0/3327776331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5494104ad0 0x7f5494104ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:23.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.686+0000 7f5499210700 1 --2- 172.21.15.39:0/3327776331 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f54940ff470 0x7f549410b900 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:23.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.686+0000 7f5499210700 1 --2- 172.21.15.39:0/3327776331 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54940feac0 0x7f54940fef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:23.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.686+0000 7f5499210700 1 -- 172.21.15.39:0/3327776331 >> 172.21.15.39:0/3327776331 conn(0x7f5494075c10 msgr2=0x7f5494076010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:23.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.686+0000 7f5499210700 1 -- 172.21.15.39:0/3327776331 shutdown_connections 2023-07-21T19:06:23.698 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.686+0000 7f5499210700 1 -- 172.21.15.39:0/3327776331 wait complete. 2023-07-21T19:06:23.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.686+0000 7f5499210700 1 Processor -- start 2023-07-21T19:06:23.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.686+0000 7f5499210700 1 -- start start 2023-07-21T19:06:23.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.687+0000 7f5499210700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54940feac0 0x7f549410c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:23.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.687+0000 7f5499210700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f54940ff470 0x7f549410c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:23.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.687+0000 7f5499210700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5494104ad0 0x7f54941a3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:23.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.687+0000 7f5499210700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54941a43a0 con 0x7f54940feac0 2023-07-21T19:06:23.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.687+0000 7f5499210700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54941a44e0 con 0x7f54940ff470 2023-07-21T19:06:23.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.687+0000 7f5499210700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54941a4650 con 0x7f5494104ad0 2023-07-21T19:06:23.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.687+0000 7f5492d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54940feac0 0x7f549410c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:23.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.687+0000 7f5492d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54940feac0 0x7f549410c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35052/0 (socket says 172.21.15.39:35052) 2023-07-21T19:06:23.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.687+0000 7f5492d9d700 1 -- 172.21.15.39:0/3438048020 learned_addr learned my addr 172.21.15.39:0/3438048020 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:06:23.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.687+0000 7f549259c700 1 --2- 172.21.15.39:0/3438048020 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f54940ff470 0x7f549410c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:23.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.687+0000 7f549359e700 1 --2- 172.21.15.39:0/3438048020 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5494104ad0 0x7f54941a3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:23.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.687+0000 7f5492d9d700 1 -- 172.21.15.39:0/3438048020 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5494104ad0 msgr2=0x7f54941a3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:23.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.687+0000 7f5492d9d700 1 --2- 172.21.15.39:0/3438048020 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5494104ad0 0x7f54941a3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:23.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.687+0000 7f5492d9d700 1 -- 172.21.15.39:0/3438048020 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f54940ff470 msgr2=0x7f549410c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:23.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.687+0000 7f5492d9d700 1 --2- 172.21.15.39:0/3438048020 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f54940ff470 0x7f549410c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:23.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.687+0000 7f5492d9d700 1 -- 172.21.15.39:0/3438048020 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f548c009c40 con 0x7f54940feac0 2023-07-21T19:06:23.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.688+0000 7f5492d9d700 1 --2- 172.21.15.39:0/3438048020 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54940feac0 0x7f549410c370 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f548400e950 tx=0x7f548400ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:23.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.688+0000 7f547bfff700 1 -- 172.21.15.39:0/3438048020 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5484009770 con 0x7f54940feac0 2023-07-21T19:06:23.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.688+0000 7f547bfff700 1 -- 172.21.15.39:0/3438048020 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f548400c7c0 con 0x7f54940feac0 2023-07-21T19:06:23.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.688+0000 7f547bfff700 1 -- 172.21.15.39:0/3438048020 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5484003c30 con 0x7f54940feac0 2023-07-21T19:06:23.703 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.688+0000 7f5499210700 1 -- 172.21.15.39:0/3438048020 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54941a4930 con 0x7f54940feac0 2023-07-21T19:06:23.703 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.688+0000 7f5499210700 1 -- 172.21.15.39:0/3438048020 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54941a4df0 con 0x7f54940feac0 2023-07-21T19:06:23.703 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.691+0000 7f547bfff700 1 -- 172.21.15.39:0/3438048020 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f548400c930 con 0x7f54940feac0 2023-07-21T19:06:23.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.691+0000 7f547bfff700 1 --2- 172.21.15.39:0/3438048020 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f547c0a1410 0x7f547c0a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:23.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.692+0000 7f547bfff700 1 -- 172.21.15.39:0/3438048020 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f5484014070 con 0x7f54940feac0 2023-07-21T19:06:23.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.693+0000 7f54a5396b80 1 Processor -- start 2023-07-21T19:06:23.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.696+0000 7f54a5396b80 1 -- start start 2023-07-21T19:06:23.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.697+0000 7f54a5396b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611b53d76b0 0x5611b5580d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:23.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.697+0000 7f54a5396b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611b55812d0 0x5611b557b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:23.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.697+0000 7f54a5396b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5611b557b580 0x5611b557b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:23.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.697+0000 7f54a5396b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611b5480c50 con 0x5611b557b580 2023-07-21T19:06:23.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.697+0000 7f54a5396b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611b54d39e0 con 0x5611b53d76b0 2023-07-21T19:06:23.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.697+0000 7f54a5396b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611b54cce70 con 0x5611b55812d0 2023-07-21T19:06:23.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.697+0000 7f5492d9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611b53d76b0 0x5611b5580d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:23.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.697+0000 7f5492d9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611b53d76b0 0x5611b5580d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:47656/0 (socket says 172.21.15.39:47656) 2023-07-21T19:06:23.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.697+0000 7f5492d9d700 1 -- 172.21.15.39:0/2665233849 learned_addr learned my addr 172.21.15.39:0/2665233849 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:06:23.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.697+0000 7f549259c700 1 --2- 172.21.15.39:0/2665233849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611b55812d0 0x5611b557b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:23.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.697+0000 7f549359e700 1 --2- 172.21.15.39:0/2665233849 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5611b557b580 0x5611b557b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:23.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.697+0000 7f549259c700 1 --2- 172.21.15.39:0/3438048020 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f547c0a1410 0x7f547c0a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:23.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.697+0000 7f5492d9d700 1 -- 172.21.15.39:0/2665233849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611b55812d0 msgr2=0x5611b557b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:06:23.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.697+0000 7f5492d9d700 1 --2- 172.21.15.39:0/2665233849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611b55812d0 0x5611b557b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:23.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.697+0000 7f5492d9d700 1 -- 172.21.15.39:0/2665233849 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5611b557b580 msgr2=0x5611b557b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:06:23.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.697+0000 7f5492d9d700 1 --2- 172.21.15.39:0/2665233849 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5611b557b580 0x5611b557b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:23.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.697+0000 7f5492d9d700 1 -- 172.21.15.39:0/2665233849 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5611b52fad80 con 0x5611b53d76b0 2023-07-21T19:06:23.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.698+0000 7f5492d9d700 1 --2- 172.21.15.39:0/2665233849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611b53d76b0 0x5611b5580d90 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5484096490 tx=0x7f5484099f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bfc881990a56659 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:23.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.698+0000 7f549259c700 1 --2- 172.21.15.39:0/3438048020 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f547c0a1410 0x7f547c0a38c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x5611b5577b40 tx=0x7f5488009310 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:23.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.698+0000 7f54797fa700 1 -- 172.21.15.39:0/2665233849 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5484098330 con 0x5611b53d76b0 2023-07-21T19:06:23.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.699+0000 7f54797fa700 1 -- 172.21.15.39:0/2665233849 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f548400cc10 con 0x5611b53d76b0 2023-07-21T19:06:23.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.699+0000 7f54797fa700 1 -- 172.21.15.39:0/2665233849 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54840994a0 con 0x5611b53d76b0 2023-07-21T19:06:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.699+0000 7f54a5396b80 1 -- 172.21.15.39:0/2665233849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611b53d76b0 msgr2=0x5611b5580d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.699+0000 7f54a5396b80 1 --2- 172.21.15.39:0/2665233849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611b53d76b0 0x5611b5580d90 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5484096490 tx=0x7f5484099f90 comp rx=0 tx=0).stop 2023-07-21T19:06:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.700+0000 7f54a5396b80 1 -- 172.21.15.39:0/2665233849 shutdown_connections 2023-07-21T19:06:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.700+0000 7f54a5396b80 1 --2- 172.21.15.39:0/2665233849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611b53d76b0 0x5611b5580d90 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.700+0000 7f54a5396b80 1 --2- 172.21.15.39:0/2665233849 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5611b557b580 0x5611b557b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.700+0000 7f54a5396b80 1 --2- 172.21.15.39:0/2665233849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611b55812d0 0x5611b557b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.700+0000 7f54a5396b80 1 -- 172.21.15.39:0/2665233849 >> 172.21.15.39:0/2665233849 conn(0x5611b5570be0 msgr2=0x5611b5575f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.700+0000 7f54a5396b80 1 -- 172.21.15.39:0/2665233849 shutdown_connections 2023-07-21T19:06:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.700+0000 7f54a5396b80 1 -- 172.21.15.39:0/2665233849 wait complete. 2023-07-21T19:06:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.700+0000 7f54a5396b80 1 -- start start 2023-07-21T19:06:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.700+0000 7f54a5396b80 1 Processor -- start 2023-07-21T19:06:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.700+0000 7f54a5396b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5611b53d76b0 0x5611b55849b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.701+0000 7f54a5396b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611b557b580 0x5611b5584ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.701+0000 7f54a5396b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611b55812d0 0x5611b5585430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.701+0000 7f54a5396b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611b546ca60 con 0x5611b53d76b0 2023-07-21T19:06:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.701+0000 7f54a5396b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611b546c760 con 0x5611b557b580 2023-07-21T19:06:23.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.701+0000 7f54a5396b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611b53508c0 con 0x5611b55812d0 2023-07-21T19:06:23.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.701+0000 7f54a5396b80 10 client.0 _ll_register_callbacks cb 0x5611b546a700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:06:23.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.701+0000 7f5492d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5611b53d76b0 0x5611b55849b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:23.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.701+0000 7f5492d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5611b53d76b0 0x5611b55849b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35066/0 (socket says 172.21.15.39:35066) 2023-07-21T19:06:23.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.701+0000 7f5492d9d700 1 -- 172.21.15.39:0/1953171356 learned_addr learned my addr 172.21.15.39:0/1953171356 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:06:23.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.701+0000 7f549359e700 1 --2- 172.21.15.39:0/1953171356 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611b55812d0 0x5611b5585430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:23.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.701+0000 7f5492d9d700 1 -- 172.21.15.39:0/1953171356 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611b55812d0 msgr2=0x5611b5585430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:06:23.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.701+0000 7f5492d9d700 1 --2- 172.21.15.39:0/1953171356 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611b55812d0 0x5611b5585430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:23.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.701+0000 7f5492d9d700 1 -- 172.21.15.39:0/1953171356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611b557b580 msgr2=0x5611b5584ef0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T19:06:23.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.701+0000 7f5492d9d700 1 --2- 172.21.15.39:0/1953171356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611b557b580 0x5611b5584ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:23.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.701+0000 7f5492d9d700 1 -- 172.21.15.39:0/1953171356 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54941a4df0 con 0x5611b53d76b0 2023-07-21T19:06:23.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.701+0000 7f5492d9d700 1 --2- 172.21.15.39:0/1953171356 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5611b53d76b0 0x5611b55849b0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f5484096a00 tx=0x7f5484098540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5af5c7398fff6705 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:23.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.701+0000 7f5476ff5700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T19:06:23.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.702+0000 7f5476ff5700 1 -- 172.21.15.39:0/1953171356 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5484099480 con 0x5611b53d76b0 2023-07-21T19:06:23.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.702+0000 7f5476ff5700 1 -- 172.21.15.39:0/1953171356 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f54840995e0 con 0x5611b53d76b0 2023-07-21T19:06:23.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.702+0000 7f54a5396b80 1 -- 172.21.15.39:0/1953171356 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5611b5426490 con 0x5611b53d76b0 2023-07-21T19:06:23.720 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.703+0000 7f5476ff5700 1 -- 172.21.15.39:0/1953171356 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54840bbea0 con 0x5611b53d76b0 2023-07-21T19:06:23.720 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.703+0000 7f54a5396b80 10 client.17910 fetch_fsmap learned FSMap version 105 2023-07-21T19:06:23.720 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.703+0000 7f54a5396b80 1 -- 172.21.15.39:0/1953171356 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=4+}) v3 -- 0x5611b52e3c00 con 0x5611b53d76b0 2023-07-21T19:06:23.720 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.703+0000 7f5476ff5700 1 -- 172.21.15.39:0/1953171356 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54840c0020 con 0x5611b53d76b0 2023-07-21T19:06:23.720 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.703+0000 7f5476ff5700 1 -- 172.21.15.39:0/1953171356 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 105) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5484094ba0 con 0x5611b53d76b0 2023-07-21T19:06:23.720 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.703+0000 7f54a5396b80 10 client.17910 fetch_fsmap finished waiting for FSMap version 105 2023-07-21T19:06:23.720 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.703+0000 7f54a5396b80 10 client.17910 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:06:23.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.703+0000 7f54a5396b80 20 client.17910 populate_metadata read hostname 'smithi039' 2023-07-21T19:06:23.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.703+0000 7f54a5396b80 1 --2- 172.21.15.39:0/1953171356 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5611b5594750 0x5611b5596b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:23.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.703+0000 7f54a5396b80 4 client.17910 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:06:23.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.703+0000 7f54a5396b80 1 -- 172.21.15.39:0/1953171356 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5611b54c5e50 con 0x5611b5594750 2023-07-21T19:06:23.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.703+0000 7f549259c700 1 --2- 172.21.15.39:0/1953171356 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5611b5594750 0x5611b5596b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:23.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.704+0000 7f5476ff5700 10 client.17910 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T19:06:23.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.704+0000 7f549259c700 1 --2- 172.21.15.39:0/1953171356 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5611b5594750 0x5611b5596b30 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8d8c07cd92b6c86 server_cookie=139b0762ff982161 in_seq=0 out_seq=0 2023-07-21T19:06:23.722 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.707+0000 7f5476ff5700 1 -- 172.21.15.39:0/1953171356 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5611b54c5e50 con 0x5611b5594750 2023-07-21T19:06:23.722 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.707+0000 7f5476ff5700 10 client.17910 handle_command_reply: tid=0 2023-07-21T19:06:23.722 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.707+0000 7f54a5396b80 1 client.17910 shutdown 2023-07-21T19:06:23.722 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.707+0000 7f54a5396b80 1 -- 172.21.15.39:0/1953171356 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5611b53d76b0 msgr2=0x5611b55849b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:23.722 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.707+0000 7f54a5396b80 1 --2- 172.21.15.39:0/1953171356 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5611b53d76b0 0x5611b55849b0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f5484096a00 tx=0x7f5484098540 comp rx=0 tx=0).stop 2023-07-21T19:06:23.722 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.707+0000 7f54a5396b80 1 -- 172.21.15.39:0/1953171356 shutdown_connections 2023-07-21T19:06:23.722 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.707+0000 7f54a5396b80 1 --2- 172.21.15.39:0/1953171356 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5611b5594750 0x5611b5596b30 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:23.722 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.707+0000 7f54a5396b80 1 --2- 172.21.15.39:0/1953171356 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611b55812d0 0x5611b5585430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:23.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.707+0000 7f54a5396b80 1 --2- 172.21.15.39:0/1953171356 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5611b53d76b0 0x5611b55849b0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:23.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.707+0000 7f54a5396b80 1 --2- 172.21.15.39:0/1953171356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611b557b580 0x5611b5584ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:23.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.707+0000 7f54a5396b80 1 -- 172.21.15.39:0/1953171356 >> 172.21.15.39:0/1953171356 conn(0x5611b5570be0 msgr2=0x5611b55754e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:23.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.707+0000 7f5476ff5700 0 client.17910 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T19:06:23.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.707+0000 7f54a5396b80 1 -- 172.21.15.39:0/1953171356 shutdown_connections 2023-07-21T19:06:23.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.707+0000 7f54a5396b80 1 -- 172.21.15.39:0/1953171356 wait complete. 2023-07-21T19:06:23.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.707+0000 7f54a5396b80 20 client.17910 trim_cache size 0 max 16384 2023-07-21T19:06:23.735 INFO:tasks.workunit.client.0.smithi039.stdout: 10 359298 27.34 MB/sec execute 576 sec latency 93.444 ms 2023-07-21T19:06:23.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.728+0000 7f54a5396b80 1 Processor -- start 2023-07-21T19:06:23.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.728+0000 7f54a5396b80 1 -- start start 2023-07-21T19:06:23.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.729+0000 7f54a5396b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5611b53d76b0 0x5611b5580400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:23.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.729+0000 7f54a5396b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611b557b580 0x5611b5583c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:23.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.729+0000 7f54a5396b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611b55812d0 0x5611b5585260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:23.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.729+0000 7f54a5396b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611b5580a50 con 0x5611b53d76b0 2023-07-21T19:06:23.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.729+0000 7f54a5396b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611b5572d30 con 0x5611b557b580 2023-07-21T19:06:23.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.729+0000 7f54a5396b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611b556dc40 con 0x5611b55812d0 2023-07-21T19:06:23.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.729+0000 7f549259c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611b557b580 0x5611b5583c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:23.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.729+0000 7f549259c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611b557b580 0x5611b5583c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:47670/0 (socket says 172.21.15.39:47670) 2023-07-21T19:06:23.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.729+0000 7f549259c700 1 -- 172.21.15.39:0/3046012527 learned_addr learned my addr 172.21.15.39:0/3046012527 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:06:23.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.729+0000 7f549359e700 1 --2- 172.21.15.39:0/3046012527 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611b55812d0 0x5611b5585260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:23.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.729+0000 7f549259c700 1 -- 172.21.15.39:0/3046012527 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611b55812d0 msgr2=0x5611b5585260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:06:23.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.729+0000 7f549259c700 1 --2- 172.21.15.39:0/3046012527 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611b55812d0 0x5611b5585260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:23.745 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.729+0000 7f549259c700 1 -- 172.21.15.39:0/3046012527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5611b53d76b0 msgr2=0x5611b5580400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:06:23.745 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.729+0000 7f549259c700 1 --2- 172.21.15.39:0/3046012527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5611b53d76b0 0x5611b5580400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:23.745 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.729+0000 7f549259c700 1 -- 172.21.15.39:0/3046012527 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5611b53211f0 con 0x5611b557b580 2023-07-21T19:06:23.745 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.730+0000 7f549259c700 1 --2- 172.21.15.39:0/3046012527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611b557b580 0x5611b5583c40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f54880116a0 tx=0x7f5488013070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cd1448c2a937e64 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:23.746 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.730+0000 7f5479ffb700 1 -- 172.21.15.39:0/3046012527 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5488024040 con 0x5611b557b580 2023-07-21T19:06:23.746 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.730+0000 7f5479ffb700 1 -- 172.21.15.39:0/3046012527 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5488004cf0 con 0x5611b557b580 2023-07-21T19:06:23.746 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.730+0000 7f5479ffb700 1 -- 172.21.15.39:0/3046012527 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54880053e0 con 0x5611b557b580 2023-07-21T19:06:23.750 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.731+0000 7f54a5396b80 1 -- 172.21.15.39:0/3046012527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611b557b580 msgr2=0x5611b5583c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:23.750 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.731+0000 7f54a5396b80 1 --2- 172.21.15.39:0/3046012527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611b557b580 0x5611b5583c40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f54880116a0 tx=0x7f5488013070 comp rx=0 tx=0).stop 2023-07-21T19:06:23.750 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.731+0000 7f54a5396b80 1 -- 172.21.15.39:0/3046012527 shutdown_connections 2023-07-21T19:06:23.751 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.731+0000 7f54a5396b80 1 --2- 172.21.15.39:0/3046012527 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611b55812d0 0x5611b5585260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:23.751 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.731+0000 7f54a5396b80 1 --2- 172.21.15.39:0/3046012527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5611b53d76b0 0x5611b5580400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:23.751 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.731+0000 7f54a5396b80 1 --2- 172.21.15.39:0/3046012527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611b557b580 0x5611b5583c40 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:23.751 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.731+0000 7f54a5396b80 1 -- 172.21.15.39:0/3046012527 >> 172.21.15.39:0/3046012527 conn(0x5611b5570be0 msgr2=0x5611b5581900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:23.752 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.732+0000 7f54a5396b80 1 -- 172.21.15.39:0/3046012527 shutdown_connections 2023-07-21T19:06:23.752 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.732+0000 7f54a5396b80 1 -- 172.21.15.39:0/3046012527 wait complete. 2023-07-21T19:06:23.752 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.732+0000 7f54a5396b80 1 -- start start 2023-07-21T19:06:23.752 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.732+0000 7f54a5396b80 1 Processor -- start 2023-07-21T19:06:23.753 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.733+0000 7f54a5396b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611b53d76b0 0x5611b558bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:23.753 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.733+0000 7f54a5396b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5611b557b580 0x5611b5589e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:23.753 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.733+0000 7f54a5396b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611b55812d0 0x5611b558a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:23.753 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.733+0000 7f54a5396b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611b558a8b0 con 0x5611b557b580 2023-07-21T19:06:23.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.733+0000 7f54a5396b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611b558aa20 con 0x5611b55812d0 2023-07-21T19:06:23.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.733+0000 7f54a5396b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611b558abe0 con 0x5611b53d76b0 2023-07-21T19:06:23.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.733+0000 7f54a5396b80 10 client.0 _ll_register_callbacks cb 0x7f54880091e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:06:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.733+0000 7f549259c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5611b557b580 0x5611b5589e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.733+0000 7f5492d9d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611b53d76b0 0x5611b558bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.733+0000 7f549359e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611b55812d0 0x5611b558a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.733+0000 7f549259c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5611b557b580 0x5611b5589e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35080/0 (socket says 172.21.15.39:35080) 2023-07-21T19:06:23.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.733+0000 7f5492d9d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611b53d76b0 0x5611b558bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:53630/0 (socket says 172.21.15.39:53630) 2023-07-21T19:06:23.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.733+0000 7f549259c700 1 -- 172.21.15.39:0/3403128286 learned_addr learned my addr 172.21.15.39:0/3403128286 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:06:23.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.733+0000 7f549359e700 1 -- 172.21.15.39:0/3403128286 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611b53d76b0 msgr2=0x5611b558bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:06:23.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.733+0000 7f549359e700 1 --2- 172.21.15.39:0/3403128286 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611b53d76b0 0x5611b558bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:23.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.733+0000 7f549359e700 1 -- 172.21.15.39:0/3403128286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5611b557b580 msgr2=0x5611b5589e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:06:23.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.733+0000 7f549359e700 1 --2- 172.21.15.39:0/3403128286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5611b557b580 0x5611b5589e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:23.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.733+0000 7f549359e700 1 -- 172.21.15.39:0/3403128286 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5488019040 con 0x5611b55812d0 2023-07-21T19:06:23.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.733+0000 7f5492d9d700 1 --2- 172.21.15.39:0/3403128286 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611b53d76b0 0x5611b558bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:06:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.733+0000 7f549259c700 1 --2- 172.21.15.39:0/3403128286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5611b557b580 0x5611b5589e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:06:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.733+0000 7f549359e700 1 --2- 172.21.15.39:0/3403128286 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611b55812d0 0x5611b558a370 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x5611b5570aa0 tx=0x7f548c0076e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f67bcd7c99f3ea74 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.733+0000 7f5476ff5700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-21T19:06:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.734+0000 7f5476ff5700 1 -- 172.21.15.39:0/3403128286 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548c00e040 con 0x5611b55812d0 2023-07-21T19:06:23.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.734+0000 7f5476ff5700 1 -- 172.21.15.39:0/3403128286 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f548c003e10 con 0x5611b55812d0 2023-07-21T19:06:23.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.734+0000 7f54a5396b80 1 -- 172.21.15.39:0/3403128286 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f548400cc10 con 0x5611b55812d0 2023-07-21T19:06:23.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.734+0000 7f5476ff5700 1 -- 172.21.15.39:0/3403128286 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7f548c00b9f0 con 0x5611b55812d0 2023-07-21T19:06:23.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.734+0000 7f54a5396b80 10 client.27335 fetch_fsmap learned FSMap version 105 2023-07-21T19:06:23.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.734+0000 7f54a5396b80 1 -- 172.21.15.39:0/3403128286 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=4+}) v3 -- 0x5611b52f9250 con 0x5611b55812d0 2023-07-21T19:06:23.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.735+0000 7f5476ff5700 1 -- 172.21.15.39:0/3403128286 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f548400cc10 con 0x5611b55812d0 2023-07-21T19:06:23.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.735+0000 7f5476ff5700 1 -- 172.21.15.39:0/3403128286 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 105) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f548c00b050 con 0x5611b55812d0 2023-07-21T19:06:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.735+0000 7f54a5396b80 10 client.27335 fetch_fsmap finished waiting for FSMap version 105 2023-07-21T19:06:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.735+0000 7f54a5396b80 10 client.27335 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:06:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.735+0000 7f54a5396b80 20 client.27335 populate_metadata read hostname 'smithi039' 2023-07-21T19:06:23.762 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.735+0000 7f54a5396b80 1 --2- 172.21.15.39:0/3403128286 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5611b55e9b00 0x5611b55ebf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:23.762 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.735+0000 7f54a5396b80 4 client.27335 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:06:23.762 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.735+0000 7f54a5396b80 1 -- 172.21.15.39:0/3403128286 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5611b5487ff0 con 0x5611b55e9b00 2023-07-21T19:06:23.762 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.735+0000 7f5492d9d700 1 --2- 172.21.15.39:0/3403128286 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5611b55e9b00 0x5611b55ebf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:23.762 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.736+0000 7f5476ff5700 10 client.27335 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T19:06:23.763 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:23.736+0000 7f5492d9d700 1 --2- 172.21.15.39:0/3403128286 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5611b55e9b00 0x5611b55ebf10 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cfbf40e4eb85713 server_cookie=297f5a672a1384a3 in_seq=0 out_seq=0 2023-07-21T19:06:24.035 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:24.024+0000 7f5476ff5700 1 -- 172.21.15.39:0/3403128286 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5611b5487ff0 con 0x5611b55e9b00 2023-07-21T19:06:24.035 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:24.024+0000 7f5476ff5700 10 client.27335 handle_command_reply: tid=0 2023-07-21T19:06:24.035 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:24.024+0000 7f54a5396b80 1 client.27335 shutdown 2023-07-21T19:06:24.038 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:24.028+0000 7f54a5396b80 1 -- 172.21.15.39:0/3403128286 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611b55812d0 msgr2=0x5611b558a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:24.039 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:24.028+0000 7f54a5396b80 1 --2- 172.21.15.39:0/3403128286 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611b55812d0 0x5611b558a370 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x5611b5570aa0 tx=0x7f548c0076e0 comp rx=0 tx=0).stop 2023-07-21T19:06:24.039 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:24.028+0000 7f54a5396b80 1 -- 172.21.15.39:0/3403128286 shutdown_connections 2023-07-21T19:06:24.039 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:24.028+0000 7f54a5396b80 1 --2- 172.21.15.39:0/3403128286 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x5611b55e9b00 0x5611b55ebf10 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:24.039 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:24.028+0000 7f54a5396b80 1 --2- 172.21.15.39:0/3403128286 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611b55812d0 0x5611b558a370 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:24.039 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:24.028+0000 7f54a5396b80 1 --2- 172.21.15.39:0/3403128286 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611b53d76b0 0x5611b558bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:24.039 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:24.028+0000 7f54a5396b80 1 --2- 172.21.15.39:0/3403128286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5611b557b580 0x5611b5589e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:24.040 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:24.028+0000 7f54a5396b80 1 -- 172.21.15.39:0/3403128286 >> 172.21.15.39:0/3403128286 conn(0x5611b5570be0 msgr2=0x5611b5582190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:24.040 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:24.028+0000 7f5476ff5700 0 client.27335 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T19:06:24.040 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:24.028+0000 7f54a5396b80 1 -- 172.21.15.39:0/3403128286 shutdown_connections 2023-07-21T19:06:24.040 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:24.028+0000 7f54a5396b80 1 -- 172.21.15.39:0/3403128286 wait complete. 2023-07-21T19:06:24.040 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:24.028+0000 7f54a5396b80 20 client.27335 trim_cache size 0 max 16384 2023-07-21T19:06:24.041 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T19:06:24.041 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T19:06:24.041 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T19:06:24.041 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T19:06:24.042 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:24.031+0000 7f5499210700 1 -- 172.21.15.39:0/3438048020 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f547c0a1410 msgr2=0x7f547c0a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:24.042 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:24.031+0000 7f5499210700 1 --2- 172.21.15.39:0/3438048020 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f547c0a1410 0x7f547c0a38c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x5611b5577b40 tx=0x7f5488009310 comp rx=0 tx=0).stop 2023-07-21T19:06:24.042 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:24.031+0000 7f5499210700 1 -- 172.21.15.39:0/3438048020 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54940feac0 msgr2=0x7f549410c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:24.042 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:24.031+0000 7f5499210700 1 --2- 172.21.15.39:0/3438048020 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54940feac0 0x7f549410c370 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f548400e950 tx=0x7f548400ec60 comp rx=0 tx=0).stop 2023-07-21T19:06:24.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:24.032+0000 7f5499210700 1 -- 172.21.15.39:0/3438048020 shutdown_connections 2023-07-21T19:06:24.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:24.032+0000 7f5499210700 1 --2- 172.21.15.39:0/3438048020 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5494104ad0 0x7f54941a3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:24.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:24.032+0000 7f5499210700 1 --2- 172.21.15.39:0/3438048020 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54940feac0 0x7f549410c370 secure :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f548400e950 tx=0x7f548400ec60 comp rx=0 tx=0).stop 2023-07-21T19:06:24.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:24.032+0000 7f5499210700 1 --2- 172.21.15.39:0/3438048020 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f547c0a1410 0x7f547c0a38c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:24.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:24.032+0000 7f5499210700 1 --2- 172.21.15.39:0/3438048020 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f54940ff470 0x7f549410c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:24.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:24.032+0000 7f5499210700 1 -- 172.21.15.39:0/3438048020 >> 172.21.15.39:0/3438048020 conn(0x7f5494075c10 msgr2=0x7f54940fce30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:24.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:24.033+0000 7f5499210700 1 -- 172.21.15.39:0/3438048020 shutdown_connections 2023-07-21T19:06:24.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:24.033+0000 7f5499210700 1 -- 172.21.15.39:0/3438048020 wait complete. 2023-07-21T19:06:24.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:06:24 smithi039 ceph-mon[96489]: pgmap v782: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 10 MiB/s wr, 448 op/s 2023-07-21T19:06:24.737 INFO:tasks.workunit.client.0.smithi039.stdout: 10 359837 27.32 MB/sec execute 577 sec latency 229.739 ms 2023-07-21T19:06:24.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:06:24 smithi193 ceph-mon[98204]: pgmap v782: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 10 MiB/s wr, 448 op/s 2023-07-21T19:06:24.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:06:24 smithi155 ceph-mon[97934]: pgmap v782: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 10 MiB/s wr, 448 op/s 2023-07-21T19:06:25.737 INFO:tasks.workunit.client.0.smithi039.stdout: 10 360027 27.29 MB/sec execute 578 sec latency 747.621 ms 2023-07-21T19:06:26.737 INFO:tasks.workunit.client.0.smithi039.stdout: 10 360111 27.25 MB/sec execute 579 sec latency 1039.895 ms 2023-07-21T19:06:26.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:06:26 smithi193 ceph-mon[98204]: pgmap v783: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 10 MiB/s wr, 448 op/s 2023-07-21T19:06:26.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:06:26 smithi155 ceph-mon[97934]: pgmap v783: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 10 MiB/s wr, 448 op/s 2023-07-21T19:06:26.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:06:26 smithi039 ceph-mon[96489]: pgmap v783: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 10 MiB/s wr, 448 op/s 2023-07-21T19:06:27.738 INFO:tasks.workunit.client.0.smithi039.stdout: 10 360380 27.24 MB/sec execute 580 sec latency 153.123 ms 2023-07-21T19:06:28.737 INFO:tasks.workunit.client.0.smithi039.stdout: 10 360864 27.25 MB/sec execute 581 sec latency 130.227 ms 2023-07-21T19:06:28.859 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:06:28 smithi193 ceph-mon[98204]: pgmap v784: 97 pgs: 97 active+clean; 2.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 19 MiB/s wr, 620 op/s 2023-07-21T19:06:28.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:06:28 smithi155 ceph-mon[97934]: pgmap v784: 97 pgs: 97 active+clean; 2.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 19 MiB/s wr, 620 op/s 2023-07-21T19:06:28.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:06:28 smithi039 ceph-mon[96489]: pgmap v784: 97 pgs: 97 active+clean; 2.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 19 MiB/s wr, 620 op/s 2023-07-21T19:06:29.737 INFO:tasks.workunit.client.0.smithi039.stdout: 10 361287 27.25 MB/sec execute 582 sec latency 86.696 ms 2023-07-21T19:06:30.737 INFO:tasks.workunit.client.0.smithi039.stdout: 10 361680 27.24 MB/sec execute 583 sec latency 72.252 ms 2023-07-21T19:06:30.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:06:30 smithi193 ceph-mon[98204]: pgmap v785: 97 pgs: 97 active+clean; 2.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 19 MiB/s wr, 620 op/s 2023-07-21T19:06:30.878 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:06:30 smithi155 ceph-mon[97934]: pgmap v785: 97 pgs: 97 active+clean; 2.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 19 MiB/s wr, 620 op/s 2023-07-21T19:06:30.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:06:30 smithi039 ceph-mon[96489]: pgmap v785: 97 pgs: 97 active+clean; 2.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 19 MiB/s wr, 620 op/s 2023-07-21T19:06:31.737 INFO:tasks.workunit.client.0.smithi039.stdout: 10 362046 27.22 MB/sec execute 584 sec latency 114.336 ms 2023-07-21T19:06:32.737 INFO:tasks.workunit.client.0.smithi039.stdout: 10 362487 27.20 MB/sec execute 585 sec latency 135.803 ms 2023-07-21T19:06:32.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:06:32 smithi193 ceph-mon[98204]: pgmap v786: 97 pgs: 97 active+clean; 2.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 19 MiB/s wr, 631 op/s 2023-07-21T19:06:32.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:06:32 smithi155 ceph-mon[97934]: pgmap v786: 97 pgs: 97 active+clean; 2.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 19 MiB/s wr, 631 op/s 2023-07-21T19:06:32.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:06:32 smithi039 ceph-mon[96489]: pgmap v786: 97 pgs: 97 active+clean; 2.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 19 MiB/s wr, 631 op/s 2023-07-21T19:06:33.738 INFO:tasks.workunit.client.0.smithi039.stdout: 10 363049 27.20 MB/sec execute 586 sec latency 96.196 ms 2023-07-21T19:06:34.738 INFO:tasks.workunit.client.0.smithi039.stdout: 10 363667 27.22 MB/sec execute 587 sec latency 99.462 ms 2023-07-21T19:06:34.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:06:34 smithi193 ceph-mon[98204]: pgmap v787: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 17 MiB/s wr, 674 op/s 2023-07-21T19:06:34.878 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:06:34 smithi155 ceph-mon[97934]: pgmap v787: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 17 MiB/s wr, 674 op/s 2023-07-21T19:06:34.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:06:34 smithi039 ceph-mon[96489]: pgmap v787: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 17 MiB/s wr, 674 op/s 2023-07-21T19:06:35.738 INFO:tasks.workunit.client.0.smithi039.stdout: 10 364192 27.23 MB/sec execute 588 sec latency 139.206 ms 2023-07-21T19:06:36.738 INFO:tasks.workunit.client.0.smithi039.stdout: 10 364690 27.23 MB/sec execute 589 sec latency 119.473 ms 2023-07-21T19:06:36.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:06:36 smithi193 ceph-mon[98204]: pgmap v788: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 16 MiB/s wr, 637 op/s 2023-07-21T19:06:36.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:06:36 smithi155 ceph-mon[97934]: pgmap v788: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 16 MiB/s wr, 637 op/s 2023-07-21T19:06:36.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:06:36 smithi039 ceph-mon[96489]: pgmap v788: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 16 MiB/s wr, 637 op/s 2023-07-21T19:06:37.196 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.185+0000 7f2d0c9db700 1 -- 172.21.15.39:0/2296616654 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d080ff100 msgr2=0x7f2d080ff570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:37.197 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.185+0000 7f2d0c9db700 1 --2- 172.21.15.39:0/2296616654 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d080ff100 0x7f2d080ff570 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc0099c0 tx=0x7f2cfc009cd0 comp rx=0 tx=0).stop 2023-07-21T19:06:37.197 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.186+0000 7f2d0c9db700 1 -- 172.21.15.39:0/2296616654 shutdown_connections 2023-07-21T19:06:37.197 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.186+0000 7f2d0c9db700 1 --2- 172.21.15.39:0/2296616654 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d081039b0 0x7f2d08103d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:37.197 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.186+0000 7f2d0c9db700 1 --2- 172.21.15.39:0/2296616654 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2d080ffab0 0x7f2d0810b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:37.197 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.186+0000 7f2d0c9db700 1 --2- 172.21.15.39:0/2296616654 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d080ff100 0x7f2d080ff570 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:37.197 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.186+0000 7f2d0c9db700 1 -- 172.21.15.39:0/2296616654 >> 172.21.15.39:0/2296616654 conn(0x7f2d08075f90 msgr2=0x7f2d08076390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:37.198 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.186+0000 7f2d0c9db700 1 -- 172.21.15.39:0/2296616654 shutdown_connections 2023-07-21T19:06:37.198 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.186+0000 7f2d0c9db700 1 -- 172.21.15.39:0/2296616654 wait complete. 2023-07-21T19:06:37.199 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.186+0000 7f2d0c9db700 1 Processor -- start 2023-07-21T19:06:37.199 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.186+0000 7f2d0c9db700 1 -- start start 2023-07-21T19:06:37.199 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.187+0000 7f2d0c9db700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d080ff100 0x7f2d08194ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:37.200 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.187+0000 7f2d0c9db700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d080ffab0 0x7f2d08195010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:37.202 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.187+0000 7f2d0c9db700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2d081039b0 0x7f2d08199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:37.202 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.187+0000 7f2d0c9db700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d08199f40 con 0x7f2d080ffab0 2023-07-21T19:06:37.202 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.187+0000 7f2d0c9db700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0819a0b0 con 0x7f2d080ff100 2023-07-21T19:06:37.202 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.187+0000 7f2d0c9db700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0819a270 con 0x7f2d081039b0 2023-07-21T19:06:37.202 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.187+0000 7f2d0659c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d080ff100 0x7f2d08194ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:37.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.187+0000 7f2d05d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d080ffab0 0x7f2d08195010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:37.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.187+0000 7f2d05d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d080ffab0 0x7f2d08195010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35094/0 (socket says 172.21.15.39:35094) 2023-07-21T19:06:37.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.187+0000 7f2d0659c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d080ff100 0x7f2d08194ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:47688/0 (socket says 172.21.15.39:47688) 2023-07-21T19:06:37.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.187+0000 7f2d05d9b700 1 -- 172.21.15.39:0/3156949821 learned_addr learned my addr 172.21.15.39:0/3156949821 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:06:37.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.187+0000 7f2d06d9d700 1 --2- 172.21.15.39:0/3156949821 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2d081039b0 0x7f2d08199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:37.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.187+0000 7f2d0659c700 1 -- 172.21.15.39:0/3156949821 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2d081039b0 msgr2=0x7f2d08199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:37.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.187+0000 7f2d0659c700 1 --2- 172.21.15.39:0/3156949821 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2d081039b0 0x7f2d08199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.187+0000 7f2d0659c700 1 -- 172.21.15.39:0/3156949821 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d080ffab0 msgr2=0x7f2d08195010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.187+0000 7f2d0659c700 1 --2- 172.21.15.39:0/3156949821 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d080ffab0 0x7f2d08195010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.187+0000 7f2d0659c700 1 -- 172.21.15.39:0/3156949821 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cfc0095f0 con 0x7f2d080ff100 2023-07-21T19:06:37.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.188+0000 7f2d0659c700 1 --2- 172.21.15.39:0/3156949821 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d080ff100 0x7f2d08194ad0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2cf800d750 tx=0x7f2cf800da60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:37.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.188+0000 7f2cf77fe700 1 -- 172.21.15.39:0/3156949821 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf8004020 con 0x7f2d080ff100 2023-07-21T19:06:37.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.188+0000 7f2cf77fe700 1 -- 172.21.15.39:0/3156949821 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2cf8004830 con 0x7f2d080ff100 2023-07-21T19:06:37.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.188+0000 7f2cf77fe700 1 -- 172.21.15.39:0/3156949821 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf80054c0 con 0x7f2d080ff100 2023-07-21T19:06:37.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.188+0000 7f2d0c9db700 1 -- 172.21.15.39:0/3156949821 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d0819a550 con 0x7f2d080ff100 2023-07-21T19:06:37.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.188+0000 7f2d0c9db700 1 -- 172.21.15.39:0/3156949821 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d0819aa60 con 0x7f2d080ff100 2023-07-21T19:06:37.207 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.192+0000 7f2cf77fe700 1 -- 172.21.15.39:0/3156949821 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f2cf80049a0 con 0x7f2d080ff100 2023-07-21T19:06:37.207 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.193+0000 7f2cf77fe700 1 --2- 172.21.15.39:0/3156949821 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f2cf00a13c0 0x7f2cf00a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:37.208 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.193+0000 7f2cf77fe700 1 -- 172.21.15.39:0/3156949821 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f2cf80cc580 con 0x7f2d080ff100 2023-07-21T19:06:37.208 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.193+0000 7f2d05d9b700 1 --2- 172.21.15.39:0/3156949821 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f2cf00a13c0 0x7f2cf00a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:37.208 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.193+0000 7f2d18b61b80 1 Processor -- start 2023-07-21T19:06:37.208 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.193+0000 7f2d05d9b700 1 --2- 172.21.15.39:0/3156949821 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f2cf00a13c0 0x7f2cf00a3870 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2d080feb10 tx=0x7f2cfc005fb0 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:37.208 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.193+0000 7f2d18b61b80 1 -- start start 2023-07-21T19:06:37.209 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.194+0000 7f2d18b61b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563aea21c6b0 0x563aea3c5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:37.209 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.194+0000 7f2d18b61b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563aea3c62d0 0x563aea3c0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:37.209 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.194+0000 7f2d18b61b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563aea3c0580 0x563aea3c09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:37.209 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.194+0000 7f2d18b61b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aea2c5c50 con 0x563aea3c0580 2023-07-21T19:06:37.209 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.194+0000 7f2d18b61b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aea3189e0 con 0x563aea21c6b0 2023-07-21T19:06:37.209 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.194+0000 7f2d18b61b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aea311e70 con 0x563aea3c62d0 2023-07-21T19:06:37.209 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.194+0000 7f2d06d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563aea3c0580 0x563aea3c09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:37.209 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.194+0000 7f2d05d9b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563aea3c62d0 0x563aea3c0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:37.210 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.194+0000 7f2d06d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563aea3c0580 0x563aea3c09d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35102/0 (socket says 172.21.15.39:35102) 2023-07-21T19:06:37.210 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.194+0000 7f2d06d9d700 1 -- 172.21.15.39:0/1858167896 learned_addr learned my addr 172.21.15.39:0/1858167896 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:06:37.210 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.194+0000 7f2d0659c700 1 --2- 172.21.15.39:0/1858167896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563aea21c6b0 0x563aea3c5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:37.210 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.194+0000 7f2d06d9d700 1 -- 172.21.15.39:0/1858167896 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563aea3c62d0 msgr2=0x563aea3c0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:06:37.210 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.194+0000 7f2d06d9d700 1 --2- 172.21.15.39:0/1858167896 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563aea3c62d0 0x563aea3c0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:37.210 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.194+0000 7f2d06d9d700 1 -- 172.21.15.39:0/1858167896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563aea21c6b0 msgr2=0x563aea3c5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:06:37.210 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.194+0000 7f2d06d9d700 1 --2- 172.21.15.39:0/1858167896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563aea21c6b0 0x563aea3c5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:37.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.194+0000 7f2d06d9d700 1 -- 172.21.15.39:0/1858167896 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563aea13fd80 con 0x563aea3c0580 2023-07-21T19:06:37.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.194+0000 7f2d05d9b700 1 --2- 172.21.15.39:0/1858167896 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563aea3c62d0 0x563aea3c0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:06:37.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.194+0000 7f2d0659c700 1 --2- 172.21.15.39:0/1858167896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563aea21c6b0 0x563aea3c5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:06:37.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.195+0000 7f2d06d9d700 1 --2- 172.21.15.39:0/1858167896 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563aea3c0580 0x563aea3c09d0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f2d00011600 tx=0x7f2d00009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=307e6224b0c1c117 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:37.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.195+0000 7f2cf4ff9700 1 -- 172.21.15.39:0/1858167896 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0000e040 con 0x563aea3c0580 2023-07-21T19:06:37.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.195+0000 7f2cf4ff9700 1 -- 172.21.15.39:0/1858167896 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d0000f040 con 0x563aea3c0580 2023-07-21T19:06:37.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.195+0000 7f2cf4ff9700 1 -- 172.21.15.39:0/1858167896 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d00004780 con 0x563aea3c0580 2023-07-21T19:06:37.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.196+0000 7f2d18b61b80 1 -- 172.21.15.39:0/1858167896 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563aea3c0580 msgr2=0x563aea3c09d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:37.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.196+0000 7f2d18b61b80 1 --2- 172.21.15.39:0/1858167896 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563aea3c0580 0x563aea3c09d0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f2d00011600 tx=0x7f2d00009920 comp rx=0 tx=0).stop 2023-07-21T19:06:37.214 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.198+0000 7f2d18b61b80 1 -- 172.21.15.39:0/1858167896 shutdown_connections 2023-07-21T19:06:37.214 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.198+0000 7f2d18b61b80 1 --2- 172.21.15.39:0/1858167896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563aea21c6b0 0x563aea3c5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:37.214 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.198+0000 7f2d18b61b80 1 --2- 172.21.15.39:0/1858167896 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563aea3c0580 0x563aea3c09d0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:37.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.198+0000 7f2d18b61b80 1 --2- 172.21.15.39:0/1858167896 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563aea3c62d0 0x563aea3c0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:37.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.198+0000 7f2d18b61b80 1 -- 172.21.15.39:0/1858167896 >> 172.21.15.39:0/1858167896 conn(0x563aea3b5be0 msgr2=0x563aea3baf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:37.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.198+0000 7f2d18b61b80 1 -- 172.21.15.39:0/1858167896 shutdown_connections 2023-07-21T19:06:37.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.198+0000 7f2d18b61b80 1 -- 172.21.15.39:0/1858167896 wait complete. 2023-07-21T19:06:37.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.198+0000 7f2d18b61b80 1 -- start start 2023-07-21T19:06:37.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.198+0000 7f2d18b61b80 1 Processor -- start 2023-07-21T19:06:37.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.199+0000 7f2d18b61b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563aea21c6b0 0x563aea3cc610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:37.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.199+0000 7f2d18b61b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563aea3c0580 0x563aea3c9820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:37.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.199+0000 7f2d18b61b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563aea3c62d0 0x563aea3c9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:37.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.199+0000 7f2d18b61b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aea2b1a60 con 0x563aea3c0580 2023-07-21T19:06:37.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.199+0000 7f2d18b61b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aea2b1760 con 0x563aea3c62d0 2023-07-21T19:06:37.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.199+0000 7f2d18b61b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aea1958c0 con 0x563aea21c6b0 2023-07-21T19:06:37.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.199+0000 7f2d18b61b80 10 client.0 _ll_register_callbacks cb 0x563aea2af700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:06:37.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.199+0000 7f2d05d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563aea3c0580 0x563aea3c9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:37.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.199+0000 7f2d05d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563aea3c0580 0x563aea3c9820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35104/0 (socket says 172.21.15.39:35104) 2023-07-21T19:06:37.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.199+0000 7f2d05d9b700 1 -- 172.21.15.39:0/2659307070 learned_addr learned my addr 172.21.15.39:0/2659307070 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:06:37.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.199+0000 7f2d06d9d700 1 --2- 172.21.15.39:0/2659307070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563aea3c62d0 0x563aea3c9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:37.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.199+0000 7f2d0659c700 1 --2- 172.21.15.39:0/2659307070 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563aea21c6b0 0x563aea3cc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:37.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.199+0000 7f2d06d9d700 1 -- 172.21.15.39:0/2659307070 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563aea21c6b0 msgr2=0x563aea3cc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:06:37.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.199+0000 7f2d06d9d700 1 --2- 172.21.15.39:0/2659307070 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563aea21c6b0 0x563aea3cc610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:37.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.199+0000 7f2d06d9d700 1 -- 172.21.15.39:0/2659307070 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563aea3c0580 msgr2=0x563aea3c9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:06:37.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.199+0000 7f2d06d9d700 1 --2- 172.21.15.39:0/2659307070 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563aea3c0580 0x563aea3c9820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:37.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.199+0000 7f2d06d9d700 1 -- 172.21.15.39:0/2659307070 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d0810c0d0 con 0x563aea3c62d0 2023-07-21T19:06:37.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.199+0000 7f2d05d9b700 1 --2- 172.21.15.39:0/2659307070 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563aea3c0580 0x563aea3c9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:06:37.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.200+0000 7f2d0659c700 1 --2- 172.21.15.39:0/2659307070 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563aea21c6b0 0x563aea3cc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:06:37.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.200+0000 7f2d06d9d700 1 --2- 172.21.15.39:0/2659307070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563aea3c62d0 0x563aea3c9f60 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2d00004280 tx=0x7f2d00004510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73cfda9056eede4f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:37.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.200+0000 7f2cedffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-21T19:06:37.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.200+0000 7f2cedffb700 1 -- 172.21.15.39:0/2659307070 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0000e030 con 0x563aea3c62d0 2023-07-21T19:06:37.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.200+0000 7f2cedffb700 1 -- 172.21.15.39:0/2659307070 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f2d000092e0 con 0x563aea3c62d0 2023-07-21T19:06:37.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.200+0000 7f2d18b61b80 1 -- 172.21.15.39:0/2659307070 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563aea26b490 con 0x563aea3c62d0 2023-07-21T19:06:37.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.200+0000 7f2cedffb700 1 -- 172.21.15.39:0/2659307070 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d00004910 con 0x563aea3c62d0 2023-07-21T19:06:37.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.200+0000 7f2d18b61b80 10 client.27347 fetch_fsmap learned FSMap version 105 2023-07-21T19:06:37.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.200+0000 7f2d18b61b80 1 -- 172.21.15.39:0/2659307070 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=4+}) v3 -- 0x7f2d000112b0 con 0x563aea3c62d0 2023-07-21T19:06:37.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.201+0000 7f2cedffb700 1 -- 172.21.15.39:0/2659307070 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563aea26b490 con 0x563aea3c62d0 2023-07-21T19:06:37.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.201+0000 7f2cedffb700 1 -- 172.21.15.39:0/2659307070 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 105) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2d00024030 con 0x563aea3c62d0 2023-07-21T19:06:37.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.201+0000 7f2d18b61b80 10 client.27347 fetch_fsmap finished waiting for FSMap version 105 2023-07-21T19:06:37.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.201+0000 7f2d18b61b80 10 client.27347 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:06:37.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.201+0000 7f2d18b61b80 20 client.27347 populate_metadata read hostname 'smithi039' 2023-07-21T19:06:37.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.201+0000 7f2d18b61b80 1 --2- 172.21.15.39:0/2659307070 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x563aea3eab10 0x563aea3ecf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:37.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.201+0000 7f2d18b61b80 4 client.27347 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:06:37.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.201+0000 7f2d18b61b80 1 -- 172.21.15.39:0/2659307070 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563aea30ae50 con 0x563aea3eab10 2023-07-21T19:06:37.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.201+0000 7f2d0659c700 1 --2- 172.21.15.39:0/2659307070 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x563aea3eab10 0x563aea3ecf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:37.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.202+0000 7f2cedffb700 10 client.27347 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T19:06:37.221 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.202+0000 7f2d0659c700 1 --2- 172.21.15.39:0/2659307070 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x563aea3eab10 0x563aea3ecf20 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9665f89ea7b5a0c7 server_cookie=3d64921135d1efd5 in_seq=0 out_seq=0 2023-07-21T19:06:37.221 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.203+0000 7f2cedffb700 1 -- 172.21.15.39:0/2659307070 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563aea30ae50 con 0x563aea3eab10 2023-07-21T19:06:37.221 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.203+0000 7f2cedffb700 10 client.27347 handle_command_reply: tid=0 2023-07-21T19:06:37.221 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.203+0000 7f2d18b61b80 1 client.27347 shutdown 2023-07-21T19:06:37.221 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.203+0000 7f2d18b61b80 1 -- 172.21.15.39:0/2659307070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563aea3c62d0 msgr2=0x563aea3c9f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:37.221 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.203+0000 7f2d18b61b80 1 --2- 172.21.15.39:0/2659307070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563aea3c62d0 0x563aea3c9f60 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2d00004280 tx=0x7f2d00004510 comp rx=0 tx=0).stop 2023-07-21T19:06:37.221 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.203+0000 7f2d18b61b80 1 -- 172.21.15.39:0/2659307070 shutdown_connections 2023-07-21T19:06:37.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.203+0000 7f2d18b61b80 1 --2- 172.21.15.39:0/2659307070 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x563aea3eab10 0x563aea3ecf20 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:37.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.203+0000 7f2d18b61b80 1 --2- 172.21.15.39:0/2659307070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563aea3c62d0 0x563aea3c9f60 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:37.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.203+0000 7f2d18b61b80 1 --2- 172.21.15.39:0/2659307070 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563aea21c6b0 0x563aea3cc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:37.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.203+0000 7f2cedffb700 0 client.27347 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T19:06:37.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.203+0000 7f2d18b61b80 1 --2- 172.21.15.39:0/2659307070 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563aea3c0580 0x563aea3c9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:37.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.203+0000 7f2d18b61b80 1 -- 172.21.15.39:0/2659307070 >> 172.21.15.39:0/2659307070 conn(0x563aea3b5be0 msgr2=0x563aea3b9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:37.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.203+0000 7f2d18b61b80 1 -- 172.21.15.39:0/2659307070 shutdown_connections 2023-07-21T19:06:37.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.203+0000 7f2d18b61b80 1 -- 172.21.15.39:0/2659307070 wait complete. 2023-07-21T19:06:37.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.203+0000 7f2d18b61b80 20 client.27347 trim_cache size 0 max 16384 2023-07-21T19:06:37.239 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.228+0000 7f2d18b61b80 1 Processor -- start 2023-07-21T19:06:37.240 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.229+0000 7f2d18b61b80 1 -- start start 2023-07-21T19:06:37.240 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.229+0000 7f2d18b61b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563aea21c6b0 0x563aea3c0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:37.240 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.229+0000 7f2d18b61b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563aea3c0580 0x563aea3c97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:37.240 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.229+0000 7f2d18b61b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563aea3c62d0 0x563aea3e1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:37.242 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.229+0000 7f2d18b61b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aea3bc030 con 0x563aea21c6b0 2023-07-21T19:06:37.242 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.229+0000 7f2d18b61b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aea3bcb50 con 0x563aea3c0580 2023-07-21T19:06:37.242 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.229+0000 7f2d18b61b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aea3ca570 con 0x563aea3c62d0 2023-07-21T19:06:37.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.230+0000 7f2d0659c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563aea21c6b0 0x563aea3c0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:37.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.230+0000 7f2d0659c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563aea21c6b0 0x563aea3c0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35112/0 (socket says 172.21.15.39:35112) 2023-07-21T19:06:37.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.230+0000 7f2d0659c700 1 -- 172.21.15.39:0/2429279121 learned_addr learned my addr 172.21.15.39:0/2429279121 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:06:37.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.231+0000 7f2d06d9d700 1 --2- 172.21.15.39:0/2429279121 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563aea3c62d0 0x563aea3e1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:37.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.231+0000 7f2d05d9b700 1 --2- 172.21.15.39:0/2429279121 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563aea3c0580 0x563aea3c97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:37.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.231+0000 7f2d0659c700 1 -- 172.21.15.39:0/2429279121 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563aea3c62d0 msgr2=0x563aea3e1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:06:37.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.231+0000 7f2d0659c700 1 --2- 172.21.15.39:0/2429279121 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563aea3c62d0 0x563aea3e1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:37.245 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.231+0000 7f2d0659c700 1 -- 172.21.15.39:0/2429279121 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563aea3c0580 msgr2=0x563aea3c97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:06:37.245 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.231+0000 7f2d0659c700 1 --2- 172.21.15.39:0/2429279121 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563aea3c0580 0x563aea3c97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:37.245 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.231+0000 7f2d0659c700 1 -- 172.21.15.39:0/2429279121 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563aea3f1b50 con 0x563aea21c6b0 2023-07-21T19:06:37.245 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.231+0000 7f2d05d9b700 1 --2- 172.21.15.39:0/2429279121 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563aea3c0580 0x563aea3c97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:06:37.245 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.231+0000 7f2d0659c700 1 --2- 172.21.15.39:0/2429279121 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563aea21c6b0 0x563aea3c0040 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8099580 tx=0x7f2cf8095a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5021fc2d8ddf8672 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:37.245 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.231+0000 7f2cf57fa700 1 -- 172.21.15.39:0/2429279121 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf80a1a40 con 0x563aea21c6b0 2023-07-21T19:06:37.245 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.231+0000 7f2cf57fa700 1 -- 172.21.15.39:0/2429279121 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2cf8097730 con 0x563aea21c6b0 2023-07-21T19:06:37.246 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.231+0000 7f2cf57fa700 1 -- 172.21.15.39:0/2429279121 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf80c9ef0 con 0x563aea21c6b0 2023-07-21T19:06:37.246 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.232+0000 7f2d18b61b80 1 -- 172.21.15.39:0/2429279121 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563aea21c6b0 msgr2=0x563aea3c0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:37.246 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.232+0000 7f2d18b61b80 1 --2- 172.21.15.39:0/2429279121 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563aea21c6b0 0x563aea3c0040 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8099580 tx=0x7f2cf8095a60 comp rx=0 tx=0).stop 2023-07-21T19:06:37.246 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.232+0000 7f2d18b61b80 1 -- 172.21.15.39:0/2429279121 shutdown_connections 2023-07-21T19:06:37.246 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.232+0000 7f2d18b61b80 1 --2- 172.21.15.39:0/2429279121 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563aea3c62d0 0x563aea3e1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:37.246 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.232+0000 7f2d18b61b80 1 --2- 172.21.15.39:0/2429279121 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563aea21c6b0 0x563aea3c0040 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:37.246 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.232+0000 7f2d18b61b80 1 --2- 172.21.15.39:0/2429279121 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563aea3c0580 0x563aea3c97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:37.246 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.232+0000 7f2d18b61b80 1 -- 172.21.15.39:0/2429279121 >> 172.21.15.39:0/2429279121 conn(0x563aea3b5be0 msgr2=0x563aea3c90c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:37.247 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.233+0000 7f2d18b61b80 1 -- 172.21.15.39:0/2429279121 shutdown_connections 2023-07-21T19:06:37.247 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.233+0000 7f2d18b61b80 1 -- 172.21.15.39:0/2429279121 wait complete. 2023-07-21T19:06:37.247 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.233+0000 7f2d18b61b80 1 -- start start 2023-07-21T19:06:37.247 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.233+0000 7f2d18b61b80 1 Processor -- start 2023-07-21T19:06:37.249 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.234+0000 7f2d18b61b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563aea21c6b0 0x563aea3c28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:37.250 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.234+0000 7f2d18b61b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563aea3c0580 0x563aea3c2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:37.250 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.234+0000 7f2d18b61b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563aea3c62d0 0x563aea3c1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:37.250 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.234+0000 7f2d18b61b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aea3c3450 con 0x563aea21c6b0 2023-07-21T19:06:37.251 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.234+0000 7f2d18b61b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aea3c35c0 con 0x563aea3c0580 2023-07-21T19:06:37.251 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.234+0000 7f2d18b61b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aea3c1d00 con 0x563aea3c62d0 2023-07-21T19:06:37.251 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.234+0000 7f2d18b61b80 10 client.0 _ll_register_callbacks cb 0x7f2cfc003830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:06:37.251 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.234+0000 7f2d0659c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563aea21c6b0 0x563aea3c28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:37.251 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.234+0000 7f2d0659c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563aea21c6b0 0x563aea3c28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35118/0 (socket says 172.21.15.39:35118) 2023-07-21T19:06:37.251 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.234+0000 7f2d0659c700 1 -- 172.21.15.39:0/1793550335 learned_addr learned my addr 172.21.15.39:0/1793550335 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:06:37.252 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.234+0000 7f2d06d9d700 1 --2- 172.21.15.39:0/1793550335 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563aea3c62d0 0x563aea3c1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:37.252 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.234+0000 7f2d05d9b700 1 --2- 172.21.15.39:0/1793550335 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563aea3c0580 0x563aea3c2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:37.252 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.234+0000 7f2d0659c700 1 -- 172.21.15.39:0/1793550335 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563aea3c62d0 msgr2=0x563aea3c1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:06:37.252 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.234+0000 7f2d0659c700 1 --2- 172.21.15.39:0/1793550335 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563aea3c62d0 0x563aea3c1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:37.252 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.234+0000 7f2d0659c700 1 -- 172.21.15.39:0/1793550335 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563aea3c0580 msgr2=0x563aea3c2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:06:37.252 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.234+0000 7f2d0659c700 1 --2- 172.21.15.39:0/1793550335 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563aea3c0580 0x563aea3c2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:37.253 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.234+0000 7f2d0659c700 1 -- 172.21.15.39:0/1793550335 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d0819aa60 con 0x563aea21c6b0 2023-07-21T19:06:37.253 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.235+0000 7f2d0659c700 1 --2- 172.21.15.39:0/1793550335 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563aea21c6b0 0x563aea3c28f0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8095af0 tx=0x7f2cf80a1b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35799b1e2f882c78 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:37.253 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.235+0000 7f2cedffb700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2023-07-21T19:06:37.253 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.235+0000 7f2cedffb700 1 -- 172.21.15.39:0/1793550335 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf8004c80 con 0x563aea21c6b0 2023-07-21T19:06:37.253 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.235+0000 7f2cedffb700 1 -- 172.21.15.39:0/1793550335 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f2cf80c9ef0 con 0x563aea21c6b0 2023-07-21T19:06:37.253 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.238+0000 7f2d18b61b80 1 -- 172.21.15.39:0/1793550335 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2d0000f040 con 0x563aea21c6b0 2023-07-21T19:06:37.254 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.238+0000 7f2cedffb700 1 -- 172.21.15.39:0/1793550335 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2cf8004de0 con 0x563aea21c6b0 2023-07-21T19:06:37.254 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.238+0000 7f2d18b61b80 10 client.17931 fetch_fsmap learned FSMap version 105 2023-07-21T19:06:37.254 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.238+0000 7f2d18b61b80 1 -- 172.21.15.39:0/1793550335 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=4+}) v3 -- 0x563aea128c00 con 0x563aea21c6b0 2023-07-21T19:06:37.254 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.238+0000 7f2cedffb700 1 -- 172.21.15.39:0/1793550335 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d0000f040 con 0x563aea21c6b0 2023-07-21T19:06:37.254 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.238+0000 7f2cedffb700 1 -- 172.21.15.39:0/1793550335 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 105) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2cf80a15b0 con 0x563aea21c6b0 2023-07-21T19:06:37.254 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.238+0000 7f2d18b61b80 10 client.17931 fetch_fsmap finished waiting for FSMap version 105 2023-07-21T19:06:37.255 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.238+0000 7f2d18b61b80 10 client.17931 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:06:37.255 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.238+0000 7f2d18b61b80 20 client.17931 populate_metadata read hostname 'smithi039' 2023-07-21T19:06:37.255 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.238+0000 7f2d18b61b80 1 --2- 172.21.15.39:0/1793550335 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x563aea43f310 0x563aea441720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:37.255 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.238+0000 7f2d18b61b80 4 client.17931 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:06:37.256 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.238+0000 7f2d18b61b80 1 -- 172.21.15.39:0/1793550335 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563aea2ccff0 con 0x563aea43f310 2023-07-21T19:06:37.256 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.238+0000 7f2d05d9b700 1 --2- 172.21.15.39:0/1793550335 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x563aea43f310 0x563aea441720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:37.257 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.239+0000 7f2d05d9b700 1 --2- 172.21.15.39:0/1793550335 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x563aea43f310 0x563aea441720 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ce9f26fb2b0dab1 server_cookie=a60145e7a32662e3 in_seq=0 out_seq=0 2023-07-21T19:06:37.257 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.239+0000 7f2cedffb700 10 client.17931 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T19:06:37.527 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.516+0000 7f2cedffb700 1 -- 172.21.15.39:0/1793550335 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563aea2ccff0 con 0x563aea43f310 2023-07-21T19:06:37.527 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.516+0000 7f2cedffb700 10 client.17931 handle_command_reply: tid=0 2023-07-21T19:06:37.527 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.516+0000 7f2d18b61b80 1 client.17931 shutdown 2023-07-21T19:06:37.528 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.516+0000 7f2d18b61b80 1 -- 172.21.15.39:0/1793550335 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563aea21c6b0 msgr2=0x563aea3c28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:37.528 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.516+0000 7f2d18b61b80 1 --2- 172.21.15.39:0/1793550335 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563aea21c6b0 0x563aea3c28f0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8095af0 tx=0x7f2cf80a1b80 comp rx=0 tx=0).stop 2023-07-21T19:06:37.528 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.516+0000 7f2d18b61b80 1 -- 172.21.15.39:0/1793550335 shutdown_connections 2023-07-21T19:06:37.528 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.516+0000 7f2d18b61b80 1 --2- 172.21.15.39:0/1793550335 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x563aea43f310 0x563aea441720 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:37.528 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.516+0000 7f2d18b61b80 1 --2- 172.21.15.39:0/1793550335 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563aea3c62d0 0x563aea3c1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:37.528 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.516+0000 7f2d18b61b80 1 --2- 172.21.15.39:0/1793550335 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563aea21c6b0 0x563aea3c28f0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:37.528 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.516+0000 7f2d18b61b80 1 --2- 172.21.15.39:0/1793550335 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563aea3c0580 0x563aea3c2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:37.528 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.516+0000 7f2d18b61b80 1 -- 172.21.15.39:0/1793550335 >> 172.21.15.39:0/1793550335 conn(0x563aea3b5be0 msgr2=0x563aea3c49f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:37.529 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.516+0000 7f2cedffb700 0 client.17931 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T19:06:37.529 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.516+0000 7f2d18b61b80 1 -- 172.21.15.39:0/1793550335 shutdown_connections 2023-07-21T19:06:37.529 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.516+0000 7f2d18b61b80 1 -- 172.21.15.39:0/1793550335 wait complete. 2023-07-21T19:06:37.529 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.517+0000 7f2d18b61b80 20 client.17931 trim_cache size 0 max 16384 2023-07-21T19:06:37.529 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T19:06:37.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T19:06:37.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T19:06:37.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T19:06:37.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.517+0000 7f2d0c9db700 1 -- 172.21.15.39:0/3156949821 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f2cf00a13c0 msgr2=0x7f2cf00a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:37.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.517+0000 7f2d0c9db700 1 --2- 172.21.15.39:0/3156949821 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f2cf00a13c0 0x7f2cf00a3870 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2d080feb10 tx=0x7f2cfc005fb0 comp rx=0 tx=0).stop 2023-07-21T19:06:37.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.517+0000 7f2d0c9db700 1 -- 172.21.15.39:0/3156949821 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d080ff100 msgr2=0x7f2d08194ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:37.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.517+0000 7f2d0c9db700 1 --2- 172.21.15.39:0/3156949821 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d080ff100 0x7f2d08194ad0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2cf800d750 tx=0x7f2cf800da60 comp rx=0 tx=0).stop 2023-07-21T19:06:37.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.517+0000 7f2d0c9db700 1 -- 172.21.15.39:0/3156949821 shutdown_connections 2023-07-21T19:06:37.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.517+0000 7f2d0c9db700 1 --2- 172.21.15.39:0/3156949821 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f2cf00a13c0 0x7f2cf00a3870 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:37.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.517+0000 7f2d0c9db700 1 --2- 172.21.15.39:0/3156949821 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d080ff100 0x7f2d08194ad0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:37.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.517+0000 7f2d0c9db700 1 --2- 172.21.15.39:0/3156949821 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2d081039b0 0x7f2d08199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:37.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.517+0000 7f2d0c9db700 1 --2- 172.21.15.39:0/3156949821 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d080ffab0 0x7f2d08195010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:37.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.517+0000 7f2d0c9db700 1 -- 172.21.15.39:0/3156949821 >> 172.21.15.39:0/3156949821 conn(0x7f2d08075f90 msgr2=0x7f2d080fbaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:37.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.517+0000 7f2d0c9db700 1 -- 172.21.15.39:0/3156949821 shutdown_connections 2023-07-21T19:06:37.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:37.517+0000 7f2d0c9db700 1 -- 172.21.15.39:0/3156949821 wait complete. 2023-07-21T19:06:37.738 INFO:tasks.workunit.client.0.smithi039.stdout: 10 365052 27.22 MB/sec execute 590 sec latency 162.264 ms 2023-07-21T19:06:38.738 INFO:tasks.workunit.client.0.smithi039.stdout: 10 365515 27.22 MB/sec execute 591 sec latency 110.988 ms 2023-07-21T19:06:38.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:06:38 smithi193 ceph-mon[98204]: pgmap v789: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 20 MiB/s wr, 828 op/s 2023-07-21T19:06:38.880 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:06:38 smithi155 ceph-mon[97934]: pgmap v789: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 20 MiB/s wr, 828 op/s 2023-07-21T19:06:38.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:06:38 smithi039 ceph-mon[96489]: pgmap v789: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 20 MiB/s wr, 828 op/s 2023-07-21T19:06:39.738 INFO:tasks.workunit.client.0.smithi039.stdout: 10 365972 27.20 MB/sec execute 592 sec latency 106.272 ms 2023-07-21T19:06:40.738 INFO:tasks.workunit.client.0.smithi039.stdout: 10 366304 27.18 MB/sec execute 593 sec latency 218.447 ms 2023-07-21T19:06:40.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:06:40 smithi193 ceph-mon[98204]: pgmap v790: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.8 MiB/s wr, 488 op/s 2023-07-21T19:06:40.878 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:06:40 smithi155 ceph-mon[97934]: pgmap v790: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.8 MiB/s wr, 488 op/s 2023-07-21T19:06:40.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:06:40 smithi039 ceph-mon[96489]: pgmap v790: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.8 MiB/s wr, 488 op/s 2023-07-21T19:06:41.738 INFO:tasks.workunit.client.0.smithi039.stdout: 10 366844 27.18 MB/sec execute 594 sec latency 121.081 ms 2023-07-21T19:06:41.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:06:41 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:06:41.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:06:41 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:06:41.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:06:41 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:06:42.738 INFO:tasks.workunit.client.0.smithi039.stdout: 10 367404 27.21 MB/sec execute 595 sec latency 91.902 ms 2023-07-21T19:06:42.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:06:42 smithi193 ceph-mon[98204]: pgmap v791: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.9 MiB/s wr, 498 op/s 2023-07-21T19:06:42.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:06:42 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:06:42.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:06:42 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:06:42.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:06:42 smithi193 ceph-mon[98204]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T19:06:42.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:06:42 smithi155 ceph-mon[97934]: pgmap v791: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.9 MiB/s wr, 498 op/s 2023-07-21T19:06:42.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:06:42 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:06:42.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:06:42 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:06:42.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:06:42 smithi155 ceph-mon[97934]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T19:06:42.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:06:42 smithi039 ceph-mon[96489]: pgmap v791: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.9 MiB/s wr, 498 op/s 2023-07-21T19:06:42.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:06:42 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:06:42.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:06:42 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:06:42.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:06:42 smithi039 ceph-mon[96489]: from='mgr.15471 ' entity='mgr.x' 2023-07-21T19:06:43.738 INFO:tasks.workunit.client.0.smithi039.stdout: 10 367921 27.20 MB/sec execute 596 sec latency 98.641 ms 2023-07-21T19:06:44.738 INFO:tasks.workunit.client.0.smithi039.stdout: 10 368365 27.19 MB/sec execute 597 sec latency 78.969 ms 2023-07-21T19:06:44.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:06:44 smithi193 ceph-mon[98204]: pgmap v792: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 687 op/s 2023-07-21T19:06:44.878 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:06:44 smithi155 ceph-mon[97934]: pgmap v792: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 687 op/s 2023-07-21T19:06:44.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:06:44 smithi039 ceph-mon[96489]: pgmap v792: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 687 op/s 2023-07-21T19:06:45.739 INFO:tasks.workunit.client.0.smithi039.stdout: 10 368767 27.18 MB/sec execute 598 sec latency 129.802 ms 2023-07-21T19:06:46.739 INFO:tasks.workunit.client.0.smithi039.stdout: 10 369274 27.19 MB/sec execute 599 sec latency 88.581 ms 2023-07-21T19:06:46.860 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:06:46 smithi193 ceph-mon[98204]: pgmap v793: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 8.5 MiB/s wr, 416 op/s 2023-07-21T19:06:46.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:06:46 smithi155 ceph-mon[97934]: pgmap v793: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 8.5 MiB/s wr, 416 op/s 2023-07-21T19:06:46.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:06:46 smithi039 ceph-mon[96489]: pgmap v793: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 8.5 MiB/s wr, 416 op/s 2023-07-21T19:06:47.740 INFO:tasks.workunit.client.0.smithi039.stdout: 10 cleanup 600 sec 2023-07-21T19:06:48.699 INFO:tasks.workunit.client.0.smithi039.stdout: 0 cleanup 601 sec 2023-07-21T19:06:48.699 INFO:tasks.workunit.client.0.smithi039.stdout: 2023-07-21T19:06:48.699 INFO:tasks.workunit.client.0.smithi039.stdout: Operation Count AvgLat MaxLat 2023-07-21T19:06:48.700 INFO:tasks.workunit.client.0.smithi039.stdout: ---------------------------------------- 2023-07-21T19:06:48.700 INFO:tasks.workunit.client.0.smithi039.stdout: NTCreateX 522753 4.820 9875.596 2023-07-21T19:06:48.700 INFO:tasks.workunit.client.0.smithi039.stdout: Close 383917 0.003 0.220 2023-07-21T19:06:48.700 INFO:tasks.workunit.client.0.smithi039.stdout: Rename 22168 21.731 2634.969 2023-07-21T19:06:48.700 INFO:tasks.workunit.client.0.smithi039.stdout: Unlink 105652 5.623 7070.114 2023-07-21T19:06:48.700 INFO:tasks.workunit.client.0.smithi039.stdout: Qpathinfo 474283 3.584 10617.173 2023-07-21T19:06:48.700 INFO:tasks.workunit.client.0.smithi039.stdout: Qfileinfo 82699 0.003 12.564 2023-07-21T19:06:48.701 INFO:tasks.workunit.client.0.smithi039.stdout: Qfsinfo 86956 0.290 25.638 2023-07-21T19:06:48.701 INFO:tasks.workunit.client.0.smithi039.stdout: Sfileinfo 42591 0.172 158.905 2023-07-21T19:06:48.701 INFO:tasks.workunit.client.0.smithi039.stdout: Find 183269 0.454 4518.714 2023-07-21T19:06:48.701 INFO:tasks.workunit.client.0.smithi039.stdout: WriteX 258390 0.553 1086.737 2023-07-21T19:06:48.701 INFO:tasks.workunit.client.0.smithi039.stdout: ReadX 820770 0.027 454.428 2023-07-21T19:06:48.701 INFO:tasks.workunit.client.0.smithi039.stdout: LockX 1706 2.774 107.423 2023-07-21T19:06:48.701 INFO:tasks.workunit.client.0.smithi039.stdout: UnlockX 1706 2.593 340.791 2023-07-21T19:06:48.701 INFO:tasks.workunit.client.0.smithi039.stdout: Flush 36633 11.100 1306.528 2023-07-21T19:06:48.702 INFO:tasks.workunit.client.0.smithi039.stdout: 2023-07-21T19:06:48.702 INFO:tasks.workunit.client.0.smithi039.stdout:Throughput 27.1895 MB/sec 10 clients 10 procs max_latency=10617.178 ms 2023-07-21T19:06:48.710 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-07-21T19:06:48.711 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-07-21T19:06:48.811 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:06:48 smithi039 ceph-mon[96489]: pgmap v794: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 582 op/s 2023-07-21T19:06:48.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:06:48 smithi193 ceph-mon[98204]: pgmap v794: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 582 op/s 2023-07-21T19:06:48.880 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:06:48 smithi155 ceph-mon[97934]: pgmap v794: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 582 op/s 2023-07-21T19:06:50.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.691+0000 7f145b783700 1 -- 172.21.15.39:0/1649210519 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1454104a50 msgr2=0x7f1454104e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:50.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.691+0000 7f145b783700 1 --2- 172.21.15.39:0/1649210519 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1454104a50 0x7f1454104e20 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f1450009990 tx=0x7f1450009ca0 comp rx=0 tx=0).stop 2023-07-21T19:06:50.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.691+0000 7f145b783700 1 -- 172.21.15.39:0/1649210519 shutdown_connections 2023-07-21T19:06:50.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.691+0000 7f145b783700 1 --2- 172.21.15.39:0/1649210519 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14540ff3f0 0x7f145410b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:50.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.691+0000 7f145b783700 1 --2- 172.21.15.39:0/1649210519 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1454104a50 0x7f1454104e20 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:50.703 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.691+0000 7f145b783700 1 --2- 172.21.15.39:0/1649210519 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14540fea40 0x7f14540feeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:50.703 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.691+0000 7f145b783700 1 -- 172.21.15.39:0/1649210519 >> 172.21.15.39:0/1649210519 conn(0x7f14540fa540 msgr2=0x7f14540fc950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:50.703 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.691+0000 7f145b783700 1 -- 172.21.15.39:0/1649210519 shutdown_connections 2023-07-21T19:06:50.703 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.691+0000 7f145b783700 1 -- 172.21.15.39:0/1649210519 wait complete. 2023-07-21T19:06:50.703 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.692+0000 7f145b783700 1 Processor -- start 2023-07-21T19:06:50.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.692+0000 7f145b783700 1 -- start start 2023-07-21T19:06:50.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.692+0000 7f145b783700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14540fea40 0x7f1454199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:50.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.692+0000 7f145b783700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14540ff3f0 0x7f1454199eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:50.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.692+0000 7f145b783700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1454104a50 0x7f14541937d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:50.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.692+0000 7f145b783700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1454193e40 con 0x7f1454104a50 2023-07-21T19:06:50.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.692+0000 7f145b783700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1454193fb0 con 0x7f14540ff3f0 2023-07-21T19:06:50.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.692+0000 7f145b783700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1454194170 con 0x7f14540fea40 2023-07-21T19:06:50.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.693+0000 7f1458d1e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14540ff3f0 0x7f1454199eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:50.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.693+0000 7f1458d1e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14540ff3f0 0x7f1454199eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:47728/0 (socket says 172.21.15.39:47728) 2023-07-21T19:06:50.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.693+0000 7f1458d1e700 1 -- 172.21.15.39:0/4127713537 learned_addr learned my addr 172.21.15.39:0/4127713537 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:06:50.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.693+0000 7f1459d20700 1 --2- 172.21.15.39:0/4127713537 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1454104a50 0x7f14541937d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:50.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.693+0000 7f145951f700 1 --2- 172.21.15.39:0/4127713537 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14540fea40 0x7f1454199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.693+0000 7f1458d1e700 1 -- 172.21.15.39:0/4127713537 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14540fea40 msgr2=0x7f1454199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.693+0000 7f1458d1e700 1 --2- 172.21.15.39:0/4127713537 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14540fea40 0x7f1454199970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.693+0000 7f1458d1e700 1 -- 172.21.15.39:0/4127713537 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1454104a50 msgr2=0x7f14541937d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.693+0000 7f1458d1e700 1 --2- 172.21.15.39:0/4127713537 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1454104a50 0x7f14541937d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.693+0000 7f1458d1e700 1 -- 172.21.15.39:0/4127713537 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14500095f0 con 0x7f14540ff3f0 2023-07-21T19:06:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.693+0000 7f145951f700 1 --2- 172.21.15.39:0/4127713537 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14540fea40 0x7f1454199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:06:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.693+0000 7f1458d1e700 1 --2- 172.21.15.39:0/4127713537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14540ff3f0 0x7f1454199eb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f144400e9a0 tx=0x7f144400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.694+0000 7f144a7fc700 1 -- 172.21.15.39:0/4127713537 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1444009790 con 0x7f14540ff3f0 2023-07-21T19:06:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.694+0000 7f144a7fc700 1 -- 172.21.15.39:0/4127713537 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f144400cb50 con 0x7f14540ff3f0 2023-07-21T19:06:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.694+0000 7f144a7fc700 1 -- 172.21.15.39:0/4127713537 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1444018680 con 0x7f14540ff3f0 2023-07-21T19:06:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.694+0000 7f145b783700 1 -- 172.21.15.39:0/4127713537 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1454194450 con 0x7f14540ff3f0 2023-07-21T19:06:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.694+0000 7f145b783700 1 -- 172.21.15.39:0/4127713537 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14541a0960 con 0x7f14540ff3f0 2023-07-21T19:06:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.697+0000 7f144a7fc700 1 -- 172.21.15.39:0/4127713537 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f144400ccc0 con 0x7f14540ff3f0 2023-07-21T19:06:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.698+0000 7f144a7fc700 1 --2- 172.21.15.39:0/4127713537 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f14400a13c0 0x7f14400a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.698+0000 7f144a7fc700 1 -- 172.21.15.39:0/4127713537 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f1444014070 con 0x7f14540ff3f0 2023-07-21T19:06:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.698+0000 7f145951f700 1 --2- 172.21.15.39:0/4127713537 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f14400a13c0 0x7f14400a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.699+0000 7f145951f700 1 --2- 172.21.15.39:0/4127713537 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f14400a13c0 0x7f14400a3870 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1450000c00 tx=0x7f1450005e80 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.699+0000 7f1467909b80 1 Processor -- start 2023-07-21T19:06:50.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.699+0000 7f1467909b80 1 -- start start 2023-07-21T19:06:50.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.699+0000 7f1467909b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba4a3316b0 0x55ba4a4dad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:50.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.699+0000 7f1467909b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba4a4db2d0 0x55ba4a4d5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:50.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.699+0000 7f1467909b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ba4a4d5580 0x55ba4a4d59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:50.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.699+0000 7f1467909b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba4a3dac50 con 0x55ba4a4d5580 2023-07-21T19:06:50.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.699+0000 7f1467909b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba4a42d9e0 con 0x55ba4a3316b0 2023-07-21T19:06:50.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.699+0000 7f1467909b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba4a426e70 con 0x55ba4a4db2d0 2023-07-21T19:06:50.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.699+0000 7f1459d20700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ba4a4d5580 0x55ba4a4d59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:50.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.699+0000 7f145951f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba4a3316b0 0x55ba4a4dad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:50.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.699+0000 7f1459d20700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ba4a4d5580 0x55ba4a4d59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35142/0 (socket says 172.21.15.39:35142) 2023-07-21T19:06:50.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.699+0000 7f1459d20700 1 -- 172.21.15.39:0/2348623489 learned_addr learned my addr 172.21.15.39:0/2348623489 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:06:50.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.699+0000 7f145951f700 1 --2- 172.21.15.39:0/2348623489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba4a3316b0 0x55ba4a4dad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:47734/0 (socket says 172.21.15.39:47734) 2023-07-21T19:06:50.714 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.699+0000 7f1459d20700 1 -- 172.21.15.39:0/2348623489 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba4a4db2d0 msgr2=0x55ba4a4d5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:06:50.714 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.699+0000 7f1459d20700 1 --2- 172.21.15.39:0/2348623489 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba4a4db2d0 0x55ba4a4d5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:50.714 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.700+0000 7f1459d20700 1 -- 172.21.15.39:0/2348623489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba4a3316b0 msgr2=0x55ba4a4dad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:06:50.714 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.700+0000 7f1459d20700 1 --2- 172.21.15.39:0/2348623489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba4a3316b0 0x55ba4a4dad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:50.714 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.700+0000 7f1459d20700 1 -- 172.21.15.39:0/2348623489 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba4a254d80 con 0x55ba4a4d5580 2023-07-21T19:06:50.714 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.700+0000 7f145951f700 1 --2- 172.21.15.39:0/2348623489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba4a3316b0 0x55ba4a4dad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:06:50.714 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.700+0000 7f1459d20700 1 --2- 172.21.15.39:0/2348623489 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ba4a4d5580 0x55ba4a4d59d0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f144c0108a0 tx=0x7f144c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad444428049eea7b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:50.714 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.700+0000 7f143f7fe700 1 -- 172.21.15.39:0/2348623489 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144c01f070 con 0x55ba4a4d5580 2023-07-21T19:06:50.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.700+0000 7f143f7fe700 1 -- 172.21.15.39:0/2348623489 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f144c016040 con 0x55ba4a4d5580 2023-07-21T19:06:50.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.700+0000 7f143f7fe700 1 -- 172.21.15.39:0/2348623489 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144c007e20 con 0x55ba4a4d5580 2023-07-21T19:06:50.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.701+0000 7f1467909b80 1 -- 172.21.15.39:0/2348623489 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ba4a4d5580 msgr2=0x55ba4a4d59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:50.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.701+0000 7f1467909b80 1 --2- 172.21.15.39:0/2348623489 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ba4a4d5580 0x55ba4a4d59d0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f144c0108a0 tx=0x7f144c00b040 comp rx=0 tx=0).stop 2023-07-21T19:06:50.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.701+0000 7f1467909b80 1 -- 172.21.15.39:0/2348623489 shutdown_connections 2023-07-21T19:06:50.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.701+0000 7f1467909b80 1 --2- 172.21.15.39:0/2348623489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba4a3316b0 0x55ba4a4dad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:50.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.701+0000 7f1467909b80 1 --2- 172.21.15.39:0/2348623489 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ba4a4d5580 0x55ba4a4d59d0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:50.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.701+0000 7f1467909b80 1 --2- 172.21.15.39:0/2348623489 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba4a4db2d0 0x55ba4a4d5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:50.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.701+0000 7f1467909b80 1 -- 172.21.15.39:0/2348623489 >> 172.21.15.39:0/2348623489 conn(0x55ba4a4cabe0 msgr2=0x55ba4a4cff10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:50.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.701+0000 7f1467909b80 1 -- 172.21.15.39:0/2348623489 shutdown_connections 2023-07-21T19:06:50.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.701+0000 7f1467909b80 1 -- 172.21.15.39:0/2348623489 wait complete. 2023-07-21T19:06:50.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.701+0000 7f1467909b80 1 -- start start 2023-07-21T19:06:50.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.702+0000 7f1467909b80 1 Processor -- start 2023-07-21T19:06:50.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.702+0000 7f1467909b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba4a3316b0 0x55ba4a4e1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:50.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.702+0000 7f1467909b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ba4a4d5580 0x55ba4a4de960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:50.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.702+0000 7f1467909b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba4a4db2d0 0x55ba4a4df0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:50.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.702+0000 7f1467909b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba4a3c6a60 con 0x55ba4a4d5580 2023-07-21T19:06:50.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.702+0000 7f1467909b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba4a3c6760 con 0x55ba4a4db2d0 2023-07-21T19:06:50.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.702+0000 7f1467909b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba4a2aa8c0 con 0x55ba4a3316b0 2023-07-21T19:06:50.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.702+0000 7f1467909b80 10 client.0 _ll_register_callbacks cb 0x55ba4a3c4700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:06:50.720 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.702+0000 7f145951f700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba4a3316b0 0x55ba4a4e1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:50.720 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.702+0000 7f145951f700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba4a3316b0 0x55ba4a4e1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:53696/0 (socket says 172.21.15.39:53696) 2023-07-21T19:06:50.720 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.702+0000 7f145951f700 1 -- 172.21.15.39:0/3884818391 learned_addr learned my addr 172.21.15.39:0/3884818391 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:06:50.720 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.702+0000 7f1459d20700 1 --2- 172.21.15.39:0/3884818391 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba4a4db2d0 0x55ba4a4df0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:50.720 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.702+0000 7f145951f700 1 -- 172.21.15.39:0/3884818391 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba4a4db2d0 msgr2=0x55ba4a4df0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:06:50.720 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.702+0000 7f145951f700 1 --2- 172.21.15.39:0/3884818391 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba4a4db2d0 0x55ba4a4df0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:50.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.702+0000 7f145951f700 1 -- 172.21.15.39:0/3884818391 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ba4a4d5580 msgr2=0x55ba4a4de960 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T19:06:50.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.702+0000 7f145951f700 1 --2- 172.21.15.39:0/3884818391 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ba4a4d5580 0x55ba4a4de960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:50.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.703+0000 7f145951f700 1 -- 172.21.15.39:0/3884818391 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f145410c180 con 0x55ba4a3316b0 2023-07-21T19:06:50.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.703+0000 7f145951f700 1 --2- 172.21.15.39:0/3884818391 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba4a3316b0 0x55ba4a4e1640 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f14540ffb80 tx=0x7f1450009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=959b21f321398e3e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:50.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.703+0000 7f143cff9700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2023-07-21T19:06:50.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.703+0000 7f143cff9700 1 -- 172.21.15.39:0/3884818391 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1450026050 con 0x55ba4a3316b0 2023-07-21T19:06:50.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.703+0000 7f1467909b80 1 -- 172.21.15.39:0/3884818391 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ba4a380490 con 0x55ba4a3316b0 2023-07-21T19:06:50.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.703+0000 7f143cff9700 1 -- 172.21.15.39:0/3884818391 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f14500056c0 con 0x55ba4a3316b0 2023-07-21T19:06:50.722 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.703+0000 7f143cff9700 1 -- 172.21.15.39:0/3884818391 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1450021a70 con 0x55ba4a3316b0 2023-07-21T19:06:50.722 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.703+0000 7f1467909b80 10 client.27439 fetch_fsmap learned FSMap version 105 2023-07-21T19:06:50.722 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.703+0000 7f1467909b80 1 -- 172.21.15.39:0/3884818391 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=4+}) v3 -- 0x7f144c010550 con 0x55ba4a3316b0 2023-07-21T19:06:50.722 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.703+0000 7f143cff9700 1 -- 172.21.15.39:0/3884818391 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ba4a380490 con 0x55ba4a3316b0 2023-07-21T19:06:50.722 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.703+0000 7f143cff9700 1 -- 172.21.15.39:0/3884818391 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 105) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1450015450 con 0x55ba4a3316b0 2023-07-21T19:06:50.722 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.703+0000 7f1467909b80 10 client.27439 fetch_fsmap finished waiting for FSMap version 105 2023-07-21T19:06:50.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.703+0000 7f1467909b80 10 client.27439 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:06:50.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.703+0000 7f1467909b80 20 client.27439 populate_metadata read hostname 'smithi039' 2023-07-21T19:06:50.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.704+0000 7f1467909b80 1 --2- 172.21.15.39:0/3884818391 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55ba4a4eec50 0x55ba4a4f1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:50.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.704+0000 7f1467909b80 4 client.27439 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:06:50.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.704+0000 7f1467909b80 1 -- 172.21.15.39:0/3884818391 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ba4a41fe50 con 0x55ba4a4eec50 2023-07-21T19:06:50.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.706+0000 7f1458d1e700 1 --2- 172.21.15.39:0/3884818391 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55ba4a4eec50 0x55ba4a4f1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:50.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.706+0000 7f143cff9700 10 client.27439 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T19:06:50.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.706+0000 7f1458d1e700 1 --2- 172.21.15.39:0/3884818391 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55ba4a4eec50 0x55ba4a4f1060 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd2a381788ebb0a4 server_cookie=5ad6290b58ea2399 in_seq=0 out_seq=0 2023-07-21T19:06:50.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.707+0000 7f143cff9700 1 -- 172.21.15.39:0/3884818391 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ba4a41fe50 con 0x55ba4a4eec50 2023-07-21T19:06:50.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.707+0000 7f143cff9700 10 client.27439 handle_command_reply: tid=0 2023-07-21T19:06:50.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.707+0000 7f1467909b80 1 client.27439 shutdown 2023-07-21T19:06:50.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.707+0000 7f1467909b80 1 -- 172.21.15.39:0/3884818391 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba4a3316b0 msgr2=0x55ba4a4e1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:50.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.707+0000 7f1467909b80 1 --2- 172.21.15.39:0/3884818391 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba4a3316b0 0x55ba4a4e1640 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f14540ffb80 tx=0x7f1450009ca0 comp rx=0 tx=0).stop 2023-07-21T19:06:50.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.707+0000 7f1467909b80 1 -- 172.21.15.39:0/3884818391 shutdown_connections 2023-07-21T19:06:50.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.707+0000 7f1467909b80 1 --2- 172.21.15.39:0/3884818391 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55ba4a4eec50 0x55ba4a4f1060 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:50.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.707+0000 7f1467909b80 1 --2- 172.21.15.39:0/3884818391 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba4a4db2d0 0x55ba4a4df0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:50.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.707+0000 7f1467909b80 1 --2- 172.21.15.39:0/3884818391 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba4a3316b0 0x55ba4a4e1640 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:50.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.707+0000 7f1467909b80 1 --2- 172.21.15.39:0/3884818391 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ba4a4d5580 0x55ba4a4de960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:50.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.707+0000 7f143cff9700 0 client.27439 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T19:06:50.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.707+0000 7f1467909b80 1 -- 172.21.15.39:0/3884818391 >> 172.21.15.39:0/3884818391 conn(0x55ba4a4cabe0 msgr2=0x55ba4a4cec50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:50.726 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.707+0000 7f1467909b80 1 -- 172.21.15.39:0/3884818391 shutdown_connections 2023-07-21T19:06:50.726 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.707+0000 7f1467909b80 1 -- 172.21.15.39:0/3884818391 wait complete. 2023-07-21T19:06:50.726 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.707+0000 7f1467909b80 20 client.27439 trim_cache size 0 max 16384 2023-07-21T19:06:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.727+0000 7f1467909b80 1 Processor -- start 2023-07-21T19:06:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.727+0000 7f1467909b80 1 -- start start 2023-07-21T19:06:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.727+0000 7f1467909b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba4a3316b0 0x55ba4a4d5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.727+0000 7f1467909b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba4a4d5580 0x55ba4a4e5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.727+0000 7f1467909b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ba4a4db2d0 0x55ba4a4cfc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.727+0000 7f1467909b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba4a4c7bb0 con 0x55ba4a4db2d0 2023-07-21T19:06:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.727+0000 7f1467909b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba4a4d96f0 con 0x55ba4a3316b0 2023-07-21T19:06:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.727+0000 7f1467909b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba4a4df680 con 0x55ba4a4d5580 2023-07-21T19:06:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.727+0000 7f145951f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba4a3316b0 0x55ba4a4d5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.727+0000 7f1459d20700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ba4a4db2d0 0x55ba4a4cfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.727+0000 7f1459d20700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ba4a4db2d0 0x55ba4a4cfc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35154/0 (socket says 172.21.15.39:35154) 2023-07-21T19:06:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.727+0000 7f145951f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba4a3316b0 0x55ba4a4d5040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:47746/0 (socket says 172.21.15.39:47746) 2023-07-21T19:06:50.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.727+0000 7f1459d20700 1 -- 172.21.15.39:0/2783958089 learned_addr learned my addr 172.21.15.39:0/2783958089 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:06:50.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.727+0000 7f1458d1e700 1 --2- 172.21.15.39:0/2783958089 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba4a4d5580 0x55ba4a4e5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:50.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.728+0000 7f1459d20700 1 -- 172.21.15.39:0/2783958089 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba4a4d5580 msgr2=0x55ba4a4e5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:06:50.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.728+0000 7f1459d20700 1 --2- 172.21.15.39:0/2783958089 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba4a4d5580 0x55ba4a4e5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:50.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.728+0000 7f1459d20700 1 -- 172.21.15.39:0/2783958089 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba4a3316b0 msgr2=0x55ba4a4d5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:06:50.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.728+0000 7f1459d20700 1 --2- 172.21.15.39:0/2783958089 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba4a3316b0 0x55ba4a4d5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:50.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.728+0000 7f1459d20700 1 -- 172.21.15.39:0/2783958089 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba4a4d5c20 con 0x55ba4a4db2d0 2023-07-21T19:06:50.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.728+0000 7f145951f700 1 --2- 172.21.15.39:0/2783958089 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba4a3316b0 0x55ba4a4d5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:06:50.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.728+0000 7f1459d20700 1 --2- 172.21.15.39:0/2783958089 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ba4a4db2d0 0x55ba4a4cfc60 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f144c009960 tx=0x7f144c009690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14d8f8e50e15783b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:50.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.728+0000 7f143ffff700 1 -- 172.21.15.39:0/2783958089 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144c01f070 con 0x55ba4a4db2d0 2023-07-21T19:06:50.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.728+0000 7f143ffff700 1 -- 172.21.15.39:0/2783958089 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f144c020040 con 0x55ba4a4db2d0 2023-07-21T19:06:50.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.728+0000 7f143ffff700 1 -- 172.21.15.39:0/2783958089 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144c025460 con 0x55ba4a4db2d0 2023-07-21T19:06:50.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.729+0000 7f1467909b80 1 -- 172.21.15.39:0/2783958089 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ba4a4db2d0 msgr2=0x55ba4a4cfc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:50.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.729+0000 7f1467909b80 1 --2- 172.21.15.39:0/2783958089 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ba4a4db2d0 0x55ba4a4cfc60 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f144c009960 tx=0x7f144c009690 comp rx=0 tx=0).stop 2023-07-21T19:06:50.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.729+0000 7f1467909b80 1 -- 172.21.15.39:0/2783958089 shutdown_connections 2023-07-21T19:06:50.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.729+0000 7f1467909b80 1 --2- 172.21.15.39:0/2783958089 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba4a3316b0 0x55ba4a4d5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:50.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.729+0000 7f1467909b80 1 --2- 172.21.15.39:0/2783958089 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ba4a4db2d0 0x55ba4a4cfc60 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:50.745 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.729+0000 7f1467909b80 1 --2- 172.21.15.39:0/2783958089 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba4a4d5580 0x55ba4a4e5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:50.745 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.729+0000 7f1467909b80 1 -- 172.21.15.39:0/2783958089 >> 172.21.15.39:0/2783958089 conn(0x55ba4a4cabe0 msgr2=0x55ba4a4d8c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:50.745 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.729+0000 7f1467909b80 1 -- 172.21.15.39:0/2783958089 shutdown_connections 2023-07-21T19:06:50.745 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.729+0000 7f1467909b80 1 -- 172.21.15.39:0/2783958089 wait complete. 2023-07-21T19:06:50.745 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.729+0000 7f1467909b80 1 -- start start 2023-07-21T19:06:50.745 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.729+0000 7f1467909b80 1 Processor -- start 2023-07-21T19:06:50.745 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.730+0000 7f1467909b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ba4a3316b0 0x55ba4a4e3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:50.745 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.730+0000 7f1467909b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba4a4d5580 0x55ba4a4e4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:50.746 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.730+0000 7f1467909b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba4a4db2d0 0x55ba4a541b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:50.746 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.730+0000 7f1467909b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba4a4e4a90 con 0x55ba4a3316b0 2023-07-21T19:06:50.746 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.730+0000 7f1467909b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba4a542070 con 0x55ba4a4d5580 2023-07-21T19:06:50.746 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.730+0000 7f1467909b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba4a5421e0 con 0x55ba4a4db2d0 2023-07-21T19:06:50.746 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.730+0000 7f1467909b80 10 client.0 _ll_register_callbacks cb 0x7f145000ed50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:06:50.746 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.730+0000 7f145951f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ba4a3316b0 0x55ba4a4e3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:50.746 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.730+0000 7f1458d1e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba4a4d5580 0x55ba4a4e4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:50.747 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.730+0000 7f1459d20700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba4a4db2d0 0x55ba4a541b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:50.747 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.730+0000 7f1458d1e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba4a4d5580 0x55ba4a4e4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.39:47754/0 (socket says 172.21.15.39:47754) 2023-07-21T19:06:50.747 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.730+0000 7f145951f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ba4a3316b0 0x55ba4a4e3e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35156/0 (socket says 172.21.15.39:35156) 2023-07-21T19:06:50.747 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.730+0000 7f1458d1e700 1 -- 172.21.15.39:0/1479724908 learned_addr learned my addr 172.21.15.39:0/1479724908 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:06:50.747 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.730+0000 7f1458d1e700 1 -- 172.21.15.39:0/1479724908 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba4a4db2d0 msgr2=0x55ba4a541b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:06:50.747 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.730+0000 7f1458d1e700 1 --2- 172.21.15.39:0/1479724908 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba4a4db2d0 0x55ba4a541b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:50.748 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.730+0000 7f1458d1e700 1 -- 172.21.15.39:0/1479724908 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ba4a3316b0 msgr2=0x55ba4a4e3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:06:50.748 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.730+0000 7f1458d1e700 1 --2- 172.21.15.39:0/1479724908 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ba4a3316b0 0x55ba4a4e3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:50.748 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.730+0000 7f1458d1e700 1 -- 172.21.15.39:0/1479724908 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f144c010550 con 0x55ba4a4d5580 2023-07-21T19:06:50.748 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.730+0000 7f145951f700 1 --2- 172.21.15.39:0/1479724908 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ba4a3316b0 0x55ba4a4e3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:06:50.748 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.730+0000 7f1458d1e700 1 --2- 172.21.15.39:0/1479724908 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba4a4d5580 0x55ba4a4e4380 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f144400c5b0 tx=0x7f144400efa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea3f879238c0ea5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:50.748 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.730+0000 7f143cff9700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-21T19:06:50.748 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.731+0000 7f143cff9700 1 -- 172.21.15.39:0/1479724908 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144409a400 con 0x55ba4a4d5580 2023-07-21T19:06:50.749 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.731+0000 7f143cff9700 1 -- 172.21.15.39:0/1479724908 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f14440a23f0 con 0x55ba4a4d5580 2023-07-21T19:06:50.749 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.731+0000 7f1467909b80 1 -- 172.21.15.39:0/1479724908 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f144c016040 con 0x55ba4a4d5580 2023-07-21T19:06:50.749 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.731+0000 7f143cff9700 1 -- 172.21.15.39:0/1479724908 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1444099490 con 0x55ba4a4d5580 2023-07-21T19:06:50.749 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.731+0000 7f1467909b80 10 client.27371 fetch_fsmap learned FSMap version 105 2023-07-21T19:06:50.749 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.731+0000 7f1467909b80 1 -- 172.21.15.39:0/1479724908 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=4+}) v3 -- 0x55ba4a254d80 con 0x55ba4a4d5580 2023-07-21T19:06:50.749 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.731+0000 7f143cff9700 1 -- 172.21.15.39:0/1479724908 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f144c016040 con 0x55ba4a4d5580 2023-07-21T19:06:50.749 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.731+0000 7f143cff9700 1 -- 172.21.15.39:0/1479724908 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 105) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1444094bb0 con 0x55ba4a4d5580 2023-07-21T19:06:50.749 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.731+0000 7f1467909b80 10 client.27371 fetch_fsmap finished waiting for FSMap version 105 2023-07-21T19:06:50.750 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.731+0000 7f1467909b80 10 client.27371 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:06:50.750 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.731+0000 7f1467909b80 20 client.27371 populate_metadata read hostname 'smithi039' 2023-07-21T19:06:50.750 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.731+0000 7f1467909b80 1 --2- 172.21.15.39:0/1479724908 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55ba4a543860 0x55ba4a545c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:50.750 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.731+0000 7f1467909b80 4 client.27371 mds_command: new command op to 24536 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:06:50.750 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.731+0000 7f1467909b80 1 -- 172.21.15.39:0/1479724908 --> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ba4a3e1ff0 con 0x55ba4a543860 2023-07-21T19:06:50.750 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.731+0000 7f145951f700 1 --2- 172.21.15.39:0/1479724908 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55ba4a543860 0x55ba4a545c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:50.750 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.732+0000 7f145951f700 1 --2- 172.21.15.39:0/1479724908 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55ba4a543860 0x55ba4a545c70 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4355525e1fd134d6 server_cookie=da140d59cca45a33 in_seq=0 out_seq=0 2023-07-21T19:06:50.751 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.732+0000 7f143cff9700 10 client.27371 ms_handle_connect on v2:172.21.15.155:6838/431690342 2023-07-21T19:06:50.861 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:06:50 smithi193 ceph-mon[98204]: pgmap v795: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 8.0 MiB/s wr, 391 op/s 2023-07-21T19:06:50.886 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:06:50 smithi155 ceph-mon[97934]: pgmap v795: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 8.0 MiB/s wr, 391 op/s 2023-07-21T19:06:50.929 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:06:50 smithi039 ceph-mon[96489]: pgmap v795: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 8.0 MiB/s wr, 391 op/s 2023-07-21T19:06:50.996 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.986+0000 7f143cff9700 1 -- 172.21.15.39:0/1479724908 <== mds.0 v2:172.21.15.155:6838/431690342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ba4a3e1ff0 con 0x55ba4a543860 2023-07-21T19:06:50.996 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.986+0000 7f143cff9700 10 client.27371 handle_command_reply: tid=0 2023-07-21T19:06:50.997 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.986+0000 7f1467909b80 1 client.27371 shutdown 2023-07-21T19:06:50.997 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.986+0000 7f1467909b80 1 -- 172.21.15.39:0/1479724908 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba4a4d5580 msgr2=0x55ba4a4e4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:50.997 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.986+0000 7f1467909b80 1 --2- 172.21.15.39:0/1479724908 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba4a4d5580 0x55ba4a4e4380 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f144400c5b0 tx=0x7f144400efa0 comp rx=0 tx=0).stop 2023-07-21T19:06:51.001 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2023-07-21T19:06:51.001 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2023-07-21T19:06:51.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2023-07-21T19:06:51.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2023-07-21T19:06:51.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.986+0000 7f1467909b80 1 -- 172.21.15.39:0/1479724908 shutdown_connections 2023-07-21T19:06:51.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.986+0000 7f1467909b80 1 --2- 172.21.15.39:0/1479724908 >> [v2:172.21.15.155:6838/431690342,v1:172.21.15.155:6839/431690342] conn(0x55ba4a543860 0x55ba4a545c70 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:51.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.986+0000 7f1467909b80 1 --2- 172.21.15.39:0/1479724908 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba4a4db2d0 0x55ba4a541b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:51.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.987+0000 7f1467909b80 1 --2- 172.21.15.39:0/1479724908 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ba4a3316b0 0x55ba4a4e3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:51.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.987+0000 7f1467909b80 1 --2- 172.21.15.39:0/1479724908 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba4a4d5580 0x55ba4a4e4380 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:51.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.987+0000 7f1467909b80 1 -- 172.21.15.39:0/1479724908 >> 172.21.15.39:0/1479724908 conn(0x55ba4a4cabe0 msgr2=0x55ba4a4de200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:51.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.987+0000 7f143cff9700 0 client.27371 ms_handle_reset on v2:172.21.15.155:6838/431690342 2023-07-21T19:06:51.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.990+0000 7f1467909b80 1 -- 172.21.15.39:0/1479724908 shutdown_connections 2023-07-21T19:06:51.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.990+0000 7f1467909b80 1 -- 172.21.15.39:0/1479724908 wait complete. 2023-07-21T19:06:51.004 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.991+0000 7f1467909b80 20 client.27371 trim_cache size 0 max 16384 2023-07-21T19:06:51.004 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.991+0000 7f145b783700 1 -- 172.21.15.39:0/4127713537 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f14400a13c0 msgr2=0x7f14400a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:51.004 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.991+0000 7f145b783700 1 --2- 172.21.15.39:0/4127713537 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f14400a13c0 0x7f14400a3870 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1450000c00 tx=0x7f1450005e80 comp rx=0 tx=0).stop 2023-07-21T19:06:51.004 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.991+0000 7f145b783700 1 -- 172.21.15.39:0/4127713537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14540ff3f0 msgr2=0x7f1454199eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:51.004 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.991+0000 7f145b783700 1 --2- 172.21.15.39:0/4127713537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14540ff3f0 0x7f1454199eb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f144400e9a0 tx=0x7f144400ecb0 comp rx=0 tx=0).stop 2023-07-21T19:06:51.004 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.993+0000 7f145b783700 1 -- 172.21.15.39:0/4127713537 shutdown_connections 2023-07-21T19:06:51.005 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.993+0000 7f145b783700 1 --2- 172.21.15.39:0/4127713537 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1454104a50 0x7f14541937d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:51.005 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.993+0000 7f145b783700 1 --2- 172.21.15.39:0/4127713537 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14540fea40 0x7f1454199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:51.005 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.993+0000 7f145b783700 1 --2- 172.21.15.39:0/4127713537 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f14400a13c0 0x7f14400a3870 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:51.005 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.993+0000 7f145b783700 1 --2- 172.21.15.39:0/4127713537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14540ff3f0 0x7f1454199eb0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:51.005 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.993+0000 7f145b783700 1 -- 172.21.15.39:0/4127713537 >> 172.21.15.39:0/4127713537 conn(0x7f14540fa540 msgr2=0x7f1454102600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:51.005 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.995+0000 7f145b783700 1 -- 172.21.15.39:0/4127713537 shutdown_connections 2023-07-21T19:06:51.006 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2023-07-21T19:06:50.995+0000 7f145b783700 1 -- 172.21.15.39:0/4127713537 wait complete. 2023-07-21T19:06:51.483 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2023-07-21T19:06:51.483 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2023-07-21T19:06:51.865 DEBUG:teuthology.parallel:result is None 2023-07-21T19:06:51.866 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2023-07-21T19:06:51.932 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2023-07-21T19:06:51.932 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2023-07-21T19:06:51.952 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi039.front.sepia.ceph.com 2023-07-21T19:06:51.962 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2023-07-21T19:06:51.974 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2023-07-21T19:06:51.975 INFO:teuthology.task.exec:Executing custom commands... 2023-07-21T19:06:51.975 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi039.front.sepia.ceph.com 2023-07-21T19:06:51.975 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2023-07-21T19:06:52.053 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2023-07-21T19:06:52.092 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2023-07-21T19:06:52.269 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.259+0000 7f8ca6319700 1 -- 172.21.15.39:0/74866872 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ca00ffab0 msgr2=0x7f8ca010b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:52.269 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.259+0000 7f8ca6319700 1 --2- 172.21.15.39:0/74866872 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ca00ffab0 0x7f8ca010b840 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f8c9800b0d0 tx=0x7f8c9800b3e0 comp rx=0 tx=0).stop 2023-07-21T19:06:52.269 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.259+0000 7f8ca6319700 1 -- 172.21.15.39:0/74866872 shutdown_connections 2023-07-21T19:06:52.270 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.259+0000 7f8ca6319700 1 --2- 172.21.15.39:0/74866872 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ca00ffab0 0x7f8ca010b840 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:52.270 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.259+0000 7f8ca6319700 1 --2- 172.21.15.39:0/74866872 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ca01039b0 0x7f8ca0103d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:52.270 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.259+0000 7f8ca6319700 1 --2- 172.21.15.39:0/74866872 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ca00ff100 0x7f8ca00ff570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:52.270 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.259+0000 7f8ca6319700 1 -- 172.21.15.39:0/74866872 >> 172.21.15.39:0/74866872 conn(0x7f8ca0075f90 msgr2=0x7f8ca0076390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:52.270 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.259+0000 7f8ca6319700 1 -- 172.21.15.39:0/74866872 shutdown_connections 2023-07-21T19:06:52.270 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.259+0000 7f8ca6319700 1 -- 172.21.15.39:0/74866872 wait complete. 2023-07-21T19:06:52.272 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.260+0000 7f8ca6319700 1 Processor -- start 2023-07-21T19:06:52.272 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.260+0000 7f8ca6319700 1 -- start start 2023-07-21T19:06:52.272 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.260+0000 7f8ca6319700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ca00ff100 0x7f8ca0196cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:52.272 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.260+0000 7f8ca6319700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ca00ffab0 0x7f8ca01971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:52.272 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.260+0000 7f8ca6319700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ca01039b0 0x7f8ca01a1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:52.273 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.260+0000 7f8ca6319700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca01a2120 con 0x7f8ca01039b0 2023-07-21T19:06:52.273 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.260+0000 7f8ca6319700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca01a2290 con 0x7f8ca00ff100 2023-07-21T19:06:52.273 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.260+0000 7f8ca6319700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca01a2450 con 0x7f8ca00ffab0 2023-07-21T19:06:52.273 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.260+0000 7f8ca48b6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ca01039b0 0x7f8ca01a1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:52.273 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.260+0000 7f8ca48b6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ca01039b0 0x7f8ca01a1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35174/0 (socket says 172.21.15.39:35174) 2023-07-21T19:06:52.273 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.260+0000 7f8ca48b6700 1 -- 172.21.15.39:0/3018636973 learned_addr learned my addr 172.21.15.39:0/3018636973 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:06:52.273 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.260+0000 7f8c97fff700 1 --2- 172.21.15.39:0/3018636973 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ca00ffab0 0x7f8ca01971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:52.273 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.260+0000 7f8c9ffff700 1 --2- 172.21.15.39:0/3018636973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ca00ff100 0x7f8ca0196cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:52.274 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.261+0000 7f8ca48b6700 1 -- 172.21.15.39:0/3018636973 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ca00ffab0 msgr2=0x7f8ca01971f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:52.274 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.261+0000 7f8ca48b6700 1 --2- 172.21.15.39:0/3018636973 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ca00ffab0 0x7f8ca01971f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:52.274 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.261+0000 7f8ca48b6700 1 -- 172.21.15.39:0/3018636973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ca00ff100 msgr2=0x7f8ca0196cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:52.274 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.261+0000 7f8ca48b6700 1 --2- 172.21.15.39:0/3018636973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ca00ff100 0x7f8ca0196cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:52.274 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.261+0000 7f8ca48b6700 1 -- 172.21.15.39:0/3018636973 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c98009d00 con 0x7f8ca01039b0 2023-07-21T19:06:52.274 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.261+0000 7f8ca48b6700 1 --2- 172.21.15.39:0/3018636973 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ca01039b0 0x7f8ca01a1b50 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f8c98007860 tx=0x7f8c98007890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:52.274 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.261+0000 7f8c9dffb700 1 -- 172.21.15.39:0/3018636973 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9800e040 con 0x7f8ca01039b0 2023-07-21T19:06:52.275 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.261+0000 7f8c9dffb700 1 -- 172.21.15.39:0/3018636973 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c9800bcb0 con 0x7f8ca01039b0 2023-07-21T19:06:52.275 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.261+0000 7f8ca6319700 1 -- 172.21.15.39:0/3018636973 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ca01a26d0 con 0x7f8ca01039b0 2023-07-21T19:06:52.275 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.261+0000 7f8c9dffb700 1 -- 172.21.15.39:0/3018636973 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c98005930 con 0x7f8ca01039b0 2023-07-21T19:06:52.275 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.261+0000 7f8ca6319700 1 -- 172.21.15.39:0/3018636973 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ca01a2b50 con 0x7f8ca01039b0 2023-07-21T19:06:52.275 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.262+0000 7f8ca6319700 1 -- 172.21.15.39:0/3018636973 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ca0076aa0 con 0x7f8ca01039b0 2023-07-21T19:06:52.276 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.264+0000 7f8c9dffb700 1 -- 172.21.15.39:0/3018636973 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f8c98004a50 con 0x7f8ca01039b0 2023-07-21T19:06:52.276 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.264+0000 7f8c9dffb700 1 --2- 172.21.15.39:0/3018636973 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f8c8c0a12c0 0x7f8c8c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:52.276 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.264+0000 7f8c9ffff700 1 --2- 172.21.15.39:0/3018636973 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f8c8c0a12c0 0x7f8c8c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:52.276 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.264+0000 7f8c9dffb700 1 -- 172.21.15.39:0/3018636973 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f8c980ce210 con 0x7f8ca01039b0 2023-07-21T19:06:52.276 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.265+0000 7f8c9ffff700 1 --2- 172.21.15.39:0/3018636973 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f8c8c0a12c0 0x7f8c8c0a3770 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8c90009c60 tx=0x7f8c90009260 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:52.277 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.268+0000 7f8c9dffb700 1 -- 172.21.15.39:0/3018636973 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f8c980976b0 con 0x7f8ca01039b0 2023-07-21T19:06:52.518 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.508+0000 7f8ca6319700 1 -- 172.21.15.39:0/3018636973 --> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f8ca00fc1b0 con 0x7f8c8c0a12c0 2023-07-21T19:06:52.533 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.523+0000 7f8c9dffb700 1 -- 172.21.15.39:0/3018636973 <== mgr.15471 v2:172.21.15.39:6800/13616514 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f8ca00fc1b0 con 0x7f8c8c0a12c0 2023-07-21T19:06:52.533 INFO:teuthology.orchestra.run.smithi039.stdout:{"fs": "cephfs", "subvol": null, "path": "/", "rel_path": "/", "schedule": "1M", "retention": {"h": 3, "M": 6}, "start": "2023-07-21T00:00:00", "created": "2023-07-21T18:41:45", "first": "2023-07-21T18:42:00", "last": "2023-07-21T19:06:01", "last_pruned": "2023-07-21T19:06:03", "created_count": 25, "pruned_count": 18, "active": true} 2023-07-21T19:06:52.535 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.525+0000 7f8ca6319700 1 -- 172.21.15.39:0/3018636973 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f8c8c0a12c0 msgr2=0x7f8c8c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:52.535 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.525+0000 7f8ca6319700 1 --2- 172.21.15.39:0/3018636973 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f8c8c0a12c0 0x7f8c8c0a3770 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8c90009c60 tx=0x7f8c90009260 comp rx=0 tx=0).stop 2023-07-21T19:06:52.535 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.526+0000 7f8ca6319700 1 -- 172.21.15.39:0/3018636973 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ca01039b0 msgr2=0x7f8ca01a1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:52.536 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.526+0000 7f8ca6319700 1 --2- 172.21.15.39:0/3018636973 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ca01039b0 0x7f8ca01a1b50 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f8c98007860 tx=0x7f8c98007890 comp rx=0 tx=0).stop 2023-07-21T19:06:52.536 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.526+0000 7f8ca6319700 1 -- 172.21.15.39:0/3018636973 shutdown_connections 2023-07-21T19:06:52.536 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.526+0000 7f8ca6319700 1 --2- 172.21.15.39:0/3018636973 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f8c8c0a12c0 0x7f8c8c0a3770 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:52.536 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.526+0000 7f8ca6319700 1 --2- 172.21.15.39:0/3018636973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ca00ff100 0x7f8ca0196cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:52.536 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.526+0000 7f8ca6319700 1 --2- 172.21.15.39:0/3018636973 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ca01039b0 0x7f8ca01a1b50 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:52.536 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.526+0000 7f8ca6319700 1 --2- 172.21.15.39:0/3018636973 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ca00ffab0 0x7f8ca01971f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:52.536 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.526+0000 7f8ca6319700 1 -- 172.21.15.39:0/3018636973 >> 172.21.15.39:0/3018636973 conn(0x7f8ca0075f90 msgr2=0x7f8ca00fbaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:52.536 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.526+0000 7f8ca6319700 1 -- 172.21.15.39:0/3018636973 shutdown_connections 2023-07-21T19:06:52.537 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.526+0000 7f8ca6319700 1 -- 172.21.15.39:0/3018636973 wait complete. 2023-07-21T19:06:52.548 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2023-07-21T19:06:52.681 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.669+0000 7f4c03dac700 1 -- 172.21.15.39:0/3923824093 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bfc0fe3f0 msgr2=0x7f4bfc0fe7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:52.682 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.669+0000 7f4c03dac700 1 --2- 172.21.15.39:0/3923824093 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bfc0fe3f0 0x7f4bfc0fe7c0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f4bf0009990 tx=0x7f4bf0009ca0 comp rx=0 tx=0).stop 2023-07-21T19:06:52.682 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.669+0000 7f4c03dac700 1 -- 172.21.15.39:0/3923824093 shutdown_connections 2023-07-21T19:06:52.683 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.669+0000 7f4c03dac700 1 --2- 172.21.15.39:0/3923824093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4bfc103430 0x7f4bfc10b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:52.683 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.669+0000 7f4c03dac700 1 --2- 172.21.15.39:0/3923824093 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bfc0fe3f0 0x7f4bfc0fe7c0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:52.683 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.669+0000 7f4c03dac700 1 --2- 172.21.15.39:0/3923824093 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4bfc0fed90 0x7f4bfc102d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:52.683 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.669+0000 7f4c03dac700 1 -- 172.21.15.39:0/3923824093 >> 172.21.15.39:0/3923824093 conn(0x7f4bfc075f90 msgr2=0x7f4bfc076390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:52.684 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.669+0000 7f4c03dac700 1 -- 172.21.15.39:0/3923824093 shutdown_connections 2023-07-21T19:06:52.684 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.669+0000 7f4c03dac700 1 -- 172.21.15.39:0/3923824093 wait complete. 2023-07-21T19:06:52.684 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.670+0000 7f4c03dac700 1 Processor -- start 2023-07-21T19:06:52.684 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.670+0000 7f4c03dac700 1 -- start start 2023-07-21T19:06:52.684 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.670+0000 7f4c03dac700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4bfc0fe3f0 0x7f4bfc196c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:52.684 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.670+0000 7f4c03dac700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4bfc0fed90 0x7f4bfc1971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:52.685 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.670+0000 7f4c03dac700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bfc103430 0x7f4bfc1a1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:52.685 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.670+0000 7f4c03dac700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bfc1a2100 con 0x7f4bfc103430 2023-07-21T19:06:52.685 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.670+0000 7f4c03dac700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bfc1a2270 con 0x7f4bfc0fe3f0 2023-07-21T19:06:52.685 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.670+0000 7f4c03dac700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bfc1a2430 con 0x7f4bfc0fed90 2023-07-21T19:06:52.685 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.670+0000 7f4c02349700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bfc103430 0x7f4bfc1a1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:52.685 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.670+0000 7f4c01347700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4bfc0fed90 0x7f4bfc1971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:52.685 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.670+0000 7f4c02349700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bfc103430 0x7f4bfc1a1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35188/0 (socket says 172.21.15.39:35188) 2023-07-21T19:06:52.685 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.670+0000 7f4c02349700 1 -- 172.21.15.39:0/2014798943 learned_addr learned my addr 172.21.15.39:0/2014798943 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:06:52.686 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.670+0000 7f4c01347700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4bfc0fed90 0x7f4bfc1971d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.39:53738/0 (socket says 172.21.15.39:53738) 2023-07-21T19:06:52.686 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.670+0000 7f4c01b48700 1 --2- 172.21.15.39:0/2014798943 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4bfc0fe3f0 0x7f4bfc196c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:52.686 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.670+0000 7f4c02349700 1 -- 172.21.15.39:0/2014798943 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4bfc0fed90 msgr2=0x7f4bfc1971d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:52.686 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.670+0000 7f4c02349700 1 --2- 172.21.15.39:0/2014798943 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4bfc0fed90 0x7f4bfc1971d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:52.686 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.670+0000 7f4c02349700 1 -- 172.21.15.39:0/2014798943 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4bfc0fe3f0 msgr2=0x7f4bfc196c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:52.686 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.670+0000 7f4c02349700 1 --2- 172.21.15.39:0/2014798943 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4bfc0fe3f0 0x7f4bfc196c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:52.686 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.671+0000 7f4c02349700 1 -- 172.21.15.39:0/2014798943 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf00095f0 con 0x7f4bfc103430 2023-07-21T19:06:52.687 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.671+0000 7f4c02349700 1 --2- 172.21.15.39:0/2014798943 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bfc103430 0x7f4bfc1a1b30 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4007ee0 tx=0x7f4bf400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:52.687 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.671+0000 7f4beeffd700 1 -- 172.21.15.39:0/2014798943 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf4017070 con 0x7f4bfc103430 2023-07-21T19:06:52.687 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.671+0000 7f4beeffd700 1 -- 172.21.15.39:0/2014798943 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4bf400f040 con 0x7f4bfc103430 2023-07-21T19:06:52.687 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.671+0000 7f4beeffd700 1 -- 172.21.15.39:0/2014798943 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf4005580 con 0x7f4bfc103430 2023-07-21T19:06:52.687 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.671+0000 7f4c03dac700 1 -- 172.21.15.39:0/2014798943 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bfc1a2710 con 0x7f4bfc103430 2023-07-21T19:06:52.687 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.671+0000 7f4c03dac700 1 -- 172.21.15.39:0/2014798943 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bfc1a2bd0 con 0x7f4bfc103430 2023-07-21T19:06:52.687 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.672+0000 7f4c03dac700 1 -- 172.21.15.39:0/2014798943 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bfc076aa0 con 0x7f4bfc103430 2023-07-21T19:06:52.688 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.673+0000 7f4beeffd700 1 -- 172.21.15.39:0/2014798943 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f4bf400db20 con 0x7f4bfc103430 2023-07-21T19:06:52.688 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.674+0000 7f4beeffd700 1 --2- 172.21.15.39:0/2014798943 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f4be80a1340 0x7f4be80a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:52.689 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.674+0000 7f4beeffd700 1 -- 172.21.15.39:0/2014798943 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f4bf40cc9c0 con 0x7f4bfc103430 2023-07-21T19:06:52.689 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.674+0000 7f4c01b48700 1 --2- 172.21.15.39:0/2014798943 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f4be80a1340 0x7f4be80a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:52.689 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.674+0000 7f4c01b48700 1 --2- 172.21.15.39:0/2014798943 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f4be80a1340 0x7f4be80a37f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4bf000b5c0 tx=0x7f4bf0005fb0 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:52.689 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.677+0000 7f4beeffd700 1 -- 172.21.15.39:0/2014798943 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f4bf4095dd0 con 0x7f4bfc103430 2023-07-21T19:06:52.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:06:52 smithi193 ceph-mon[98204]: pgmap v796: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.4 MiB/s wr, 399 op/s 2023-07-21T19:06:52.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:06:52 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T19:06:52.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:06:52 smithi155 ceph-mon[97934]: pgmap v796: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.4 MiB/s wr, 399 op/s 2023-07-21T19:06:52.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:06:52 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T19:06:52.913 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.903+0000 7f4c03dac700 1 -- 172.21.15.39:0/2014798943 --> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f4bfc0fc300 con 0x7f4be80a1340 2023-07-21T19:06:52.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:06:52 smithi039 ceph-mon[96489]: pgmap v796: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.4 MiB/s wr, 399 op/s 2023-07-21T19:06:52.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:06:52 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T19:06:52.927 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.917+0000 7f4beeffd700 1 -- 172.21.15.39:0/2014798943 <== mgr.15471 v2:172.21.15.39:6800/13616514 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f4bfc0fc300 con 0x7f4be80a1340 2023-07-21T19:06:52.927 INFO:teuthology.orchestra.run.smithi039.stdout:/ 1M 6M3h 2023-07-21T19:06:52.929 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.919+0000 7f4c03dac700 1 -- 172.21.15.39:0/2014798943 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f4be80a1340 msgr2=0x7f4be80a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:52.929 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.919+0000 7f4c03dac700 1 --2- 172.21.15.39:0/2014798943 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f4be80a1340 0x7f4be80a37f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4bf000b5c0 tx=0x7f4bf0005fb0 comp rx=0 tx=0).stop 2023-07-21T19:06:52.930 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.919+0000 7f4c03dac700 1 -- 172.21.15.39:0/2014798943 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bfc103430 msgr2=0x7f4bfc1a1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:52.930 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.919+0000 7f4c03dac700 1 --2- 172.21.15.39:0/2014798943 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bfc103430 0x7f4bfc1a1b30 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4007ee0 tx=0x7f4bf400d3b0 comp rx=0 tx=0).stop 2023-07-21T19:06:52.930 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.919+0000 7f4c03dac700 1 -- 172.21.15.39:0/2014798943 shutdown_connections 2023-07-21T19:06:52.931 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.919+0000 7f4c03dac700 1 --2- 172.21.15.39:0/2014798943 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f4be80a1340 0x7f4be80a37f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:52.931 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.919+0000 7f4c03dac700 1 --2- 172.21.15.39:0/2014798943 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4bfc0fe3f0 0x7f4bfc196c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:52.931 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.919+0000 7f4c03dac700 1 --2- 172.21.15.39:0/2014798943 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bfc103430 0x7f4bfc1a1b30 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:52.931 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.919+0000 7f4c03dac700 1 --2- 172.21.15.39:0/2014798943 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4bfc0fed90 0x7f4bfc1971d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:52.932 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.919+0000 7f4c03dac700 1 -- 172.21.15.39:0/2014798943 >> 172.21.15.39:0/2014798943 conn(0x7f4bfc075f90 msgr2=0x7f4bfc0fbbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:52.932 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.919+0000 7f4c03dac700 1 -- 172.21.15.39:0/2014798943 shutdown_connections 2023-07-21T19:06:52.932 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:52.919+0000 7f4c03dac700 1 -- 172.21.15.39:0/2014798943 wait complete. 2023-07-21T19:06:52.941 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2023-07-21T19:06:52.958 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2023-07-21T19:06:52.959 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_79491cb69dd37a1eafb93b3471998174bdfa2fd9/teuthology/run_tasks.py", line 192, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_81cf785aa7781c4e08381c540f890cb9bc987ef9/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_81cf785aa7781c4e08381c540f890cb9bc987ef9/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2023-07-21T19:06:52.959 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2023-07-21T19:06:52.979 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T19:06:53.116 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.106+0000 7f462f297700 1 -- 172.21.15.39:0/3794374177 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4628104a00 msgr2=0x7f4628104dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:53.116 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.106+0000 7f462f297700 1 --2- 172.21.15.39:0/3794374177 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4628104a00 0x7f4628104dd0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f4618009990 tx=0x7f4618009ca0 comp rx=0 tx=0).stop 2023-07-21T19:06:53.117 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.106+0000 7f462f297700 1 -- 172.21.15.39:0/3794374177 shutdown_connections 2023-07-21T19:06:53.117 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.106+0000 7f462f297700 1 --2- 172.21.15.39:0/3794374177 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f46280ff3a0 0x7f462810b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:53.117 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.106+0000 7f462f297700 1 --2- 172.21.15.39:0/3794374177 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4628104a00 0x7f4628104dd0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:53.117 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.106+0000 7f462f297700 1 --2- 172.21.15.39:0/3794374177 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46280fe9f0 0x7f46280fee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:53.117 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.106+0000 7f462f297700 1 -- 172.21.15.39:0/3794374177 >> 172.21.15.39:0/3794374177 conn(0x7f46280fa4f0 msgr2=0x7f46280fc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:53.117 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.106+0000 7f462f297700 1 -- 172.21.15.39:0/3794374177 shutdown_connections 2023-07-21T19:06:53.118 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.106+0000 7f462f297700 1 -- 172.21.15.39:0/3794374177 wait complete. 2023-07-21T19:06:53.120 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.107+0000 7f462f297700 1 Processor -- start 2023-07-21T19:06:53.121 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.107+0000 7f462f297700 1 -- start start 2023-07-21T19:06:53.121 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.107+0000 7f462f297700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46280fe9f0 0x7f462819b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:53.121 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.107+0000 7f462f297700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46280ff3a0 0x7f462819be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:53.121 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.107+0000 7f462f297700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4628104a00 0x7f46281959c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:53.122 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.107+0000 7f462f297700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4628196030 con 0x7f46280fe9f0 2023-07-21T19:06:53.122 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.107+0000 7f462f297700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46281961a0 con 0x7f46280ff3a0 2023-07-21T19:06:53.122 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.107+0000 7f462f297700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4628196360 con 0x7f4628104a00 2023-07-21T19:06:53.122 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.107+0000 7f462d033700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46280fe9f0 0x7f462819b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:53.123 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.107+0000 7f462d033700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46280fe9f0 0x7f462819b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35198/0 (socket says 172.21.15.39:35198) 2023-07-21T19:06:53.123 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.107+0000 7f462d033700 1 -- 172.21.15.39:0/2687278964 learned_addr learned my addr 172.21.15.39:0/2687278964 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:06:53.123 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.107+0000 7f462c832700 1 --2- 172.21.15.39:0/2687278964 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46280ff3a0 0x7f462819be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:53.124 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.107+0000 7f462d834700 1 --2- 172.21.15.39:0/2687278964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4628104a00 0x7f46281959c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:53.124 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.108+0000 7f462d033700 1 -- 172.21.15.39:0/2687278964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4628104a00 msgr2=0x7f46281959c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:53.124 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.108+0000 7f462d033700 1 --2- 172.21.15.39:0/2687278964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4628104a00 0x7f46281959c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:53.125 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.108+0000 7f462d033700 1 -- 172.21.15.39:0/2687278964 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46280ff3a0 msgr2=0x7f462819be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:53.125 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.108+0000 7f462d033700 1 --2- 172.21.15.39:0/2687278964 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46280ff3a0 0x7f462819be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:53.125 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.108+0000 7f462d033700 1 -- 172.21.15.39:0/2687278964 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46180095f0 con 0x7f46280fe9f0 2023-07-21T19:06:53.125 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.108+0000 7f462d834700 1 --2- 172.21.15.39:0/2687278964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4628104a00 0x7f46281959c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:06:53.126 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.108+0000 7f462d033700 1 --2- 172.21.15.39:0/2687278964 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46280fe9f0 0x7f462819b950 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f4618000c00 tx=0x7f4618009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:53.126 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.108+0000 7f461e7fc700 1 -- 172.21.15.39:0/2687278964 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461801d070 con 0x7f46280fe9f0 2023-07-21T19:06:53.126 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.108+0000 7f461e7fc700 1 -- 172.21.15.39:0/2687278964 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4618022470 con 0x7f46280fe9f0 2023-07-21T19:06:53.127 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.108+0000 7f461e7fc700 1 -- 172.21.15.39:0/2687278964 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4618017650 con 0x7f46280fe9f0 2023-07-21T19:06:53.127 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.108+0000 7f462f297700 1 -- 172.21.15.39:0/2687278964 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46281965e0 con 0x7f46280fe9f0 2023-07-21T19:06:53.127 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.108+0000 7f462f297700 1 -- 172.21.15.39:0/2687278964 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46281a2940 con 0x7f46280fe9f0 2023-07-21T19:06:53.128 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.109+0000 7f462f297700 1 -- 172.21.15.39:0/2687278964 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4628066a70 con 0x7f46280fe9f0 2023-07-21T19:06:53.128 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.111+0000 7f461e7fc700 1 -- 172.21.15.39:0/2687278964 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f4618022870 con 0x7f46280fe9f0 2023-07-21T19:06:53.129 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.111+0000 7f461e7fc700 1 --2- 172.21.15.39:0/2687278964 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f46140a1300 0x7f46140a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:53.129 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.112+0000 7f462c832700 1 --2- 172.21.15.39:0/2687278964 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f46140a1300 0x7f46140a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:53.129 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.112+0000 7f461e7fc700 1 -- 172.21.15.39:0/2687278964 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f46180ce410 con 0x7f46280fe9f0 2023-07-21T19:06:53.130 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.112+0000 7f462c832700 1 --2- 172.21.15.39:0/2687278964 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f46140a1300 0x7f46140a37b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f46280fdbf0 tx=0x7f4624009260 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:53.130 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.115+0000 7f461e7fc700 1 -- 172.21.15.39:0/2687278964 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f46180978b0 con 0x7f46280fe9f0 2023-07-21T19:06:53.326 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.316+0000 7f462f297700 1 -- 172.21.15.39:0/2687278964 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4628196ed0 con 0x7f46280fe9f0 2023-07-21T19:06:53.327 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.318+0000 7f461e7fc700 1 -- 172.21.15.39:0/2687278964 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) v1 ==== 74+0+22980 (secure 0 0 0) 0x7f461809aed0 con 0x7f46280fe9f0 2023-07-21T19:06:53.328 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T19:06:53.329 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":122,"fsid":"eaf8e25a-27f4-11ee-9b34-001a4aab830c","created":"2023-07-21T18:33:06.504979+0000","modified":"2023-07-21T19:06:21.389488+0000","last_up_change":"2023-07-21T18:39:23.531009+0000","last_in_change":"2023-07-21T18:39:05.993024+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T18:36:13.572583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T18:40:02.726735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T18:40:03.990112+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":44,"snap_epoch":121,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03502c07-2d0f-4723-8d0b-d34c9f5b91bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6803","nonce":860193857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6805","nonce":860193857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6809","nonce":860193857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6807","nonce":860193857}]},"public_addr":"172.21.15.39:6803/860193857","cluster_addr":"172.21.15.39:6805/860193857","heartbeat_back_addr":"172.21.15.39:6809/860193857","heartbeat_front_addr":"172.21.15.39:6807/860193857","state":["exists","up"]},{"osd":1,"uuid":"0831a0c0-a1fa-4d30-9a28-513223aa69eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6801","nonce":3246721764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6803","nonce":3246721764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6807","nonce":3246721764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6805","nonce":3246721764}]},"public_addr":"172.21.15.155:6801/3246721764","cluster_addr":"172.21.15.155:6803/3246721764","heartbeat_back_addr":"172.21.15.155:6807/3246721764","heartbeat_front_addr":"172.21.15.155:6805/3246721764","state":["exists","up"]},{"osd":2,"uuid":"22df0003-bde9-48e4-b47a-cb8eafbab0db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6801","nonce":2563594449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6803","nonce":2563594449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6807","nonce":2563594449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6805","nonce":2563594449}]},"public_addr":"172.21.15.193:6801/2563594449","cluster_addr":"172.21.15.193:6803/2563594449","heartbeat_back_addr":"172.21.15.193:6807/2563594449","heartbeat_front_addr":"172.21.15.193:6805/2563594449","state":["exists","up"]},{"osd":3,"uuid":"54562558-a2e8-45dd-bc95-561569b9ae6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6811","nonce":863708386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6813","nonce":863708386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6817","nonce":863708386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6815","nonce":863708386}]},"public_addr":"172.21.15.39:6811/863708386","cluster_addr":"172.21.15.39:6813/863708386","heartbeat_back_addr":"172.21.15.39:6817/863708386","heartbeat_front_addr":"172.21.15.39:6815/863708386","state":["exists","up"]},{"osd":4,"uuid":"491118e7-ae1e-495c-9eac-ef1cdcae9600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6809","nonce":3361780673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6811","nonce":3361780673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6815","nonce":3361780673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6813","nonce":3361780673}]},"public_addr":"172.21.15.155:6809/3361780673","cluster_addr":"172.21.15.155:6811/3361780673","heartbeat_back_addr":"172.21.15.155:6815/3361780673","heartbeat_front_addr":"172.21.15.155:6813/3361780673","state":["exists","up"]},{"osd":5,"uuid":"d267bc2c-42b3-4d96-b2d1-b186dc7e6b18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6809","nonce":2376867721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6811","nonce":2376867721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6815","nonce":2376867721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6813","nonce":2376867721}]},"public_addr":"172.21.15.193:6809/2376867721","cluster_addr":"172.21.15.193:6811/2376867721","heartbeat_back_addr":"172.21.15.193:6815/2376867721","heartbeat_front_addr":"172.21.15.193:6813/2376867721","state":["exists","up"]},{"osd":6,"uuid":"beec8088-970a-473e-bd67-6eee7cbdef24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6819","nonce":2417770529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6821","nonce":2417770529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6825","nonce":2417770529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6823","nonce":2417770529}]},"public_addr":"172.21.15.39:6819/2417770529","cluster_addr":"172.21.15.39:6821/2417770529","heartbeat_back_addr":"172.21.15.39:6825/2417770529","heartbeat_front_addr":"172.21.15.39:6823/2417770529","state":["exists","up"]},{"osd":7,"uuid":"5a88ce30-1835-4254-90cc-804ed5cf1066","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6817","nonce":2542002111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6819","nonce":2542002111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6823","nonce":2542002111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6821","nonce":2542002111}]},"public_addr":"172.21.15.155:6817/2542002111","cluster_addr":"172.21.15.155:6819/2542002111","heartbeat_back_addr":"172.21.15.155:6823/2542002111","heartbeat_front_addr":"172.21.15.155:6821/2542002111","state":["exists","up"]},{"osd":8,"uuid":"d964a183-3021-4ace-9a0c-aad00f2570e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6817","nonce":2985835085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6819","nonce":2985835085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6823","nonce":2985835085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6821","nonce":2985835085}]},"public_addr":"172.21.15.193:6817/2985835085","cluster_addr":"172.21.15.193:6819/2985835085","heartbeat_back_addr":"172.21.15.193:6823/2985835085","heartbeat_front_addr":"172.21.15.193:6821/2985835085","state":["exists","up"]},{"osd":9,"uuid":"9725ea5c-49a3-4c99-bac4-e95069100b1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6827","nonce":1946493651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6829","nonce":1946493651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6833","nonce":1946493651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6831","nonce":1946493651}]},"public_addr":"172.21.15.39:6827/1946493651","cluster_addr":"172.21.15.39:6829/1946493651","heartbeat_back_addr":"172.21.15.39:6833/1946493651","heartbeat_front_addr":"172.21.15.39:6831/1946493651","state":["exists","up"]},{"osd":10,"uuid":"1fd60ccb-92db-45f0-a580-2003db0976e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6825","nonce":1234615022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6827","nonce":1234615022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6831","nonce":1234615022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6829","nonce":1234615022}]},"public_addr":"172.21.15.155:6825/1234615022","cluster_addr":"172.21.15.155:6827/1234615022","heartbeat_back_addr":"172.21.15.155:6831/1234615022","heartbeat_front_addr":"172.21.15.155:6829/1234615022","state":["exists","up"]},{"osd":11,"uuid":"efb31ea1-4775-4e5f-8740-728d6f4aeea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6825","nonce":215104663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6827","nonce":215104663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6831","nonce":215104663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6829","nonce":215104663}]},"public_addr":"172.21.15.193:6825/215104663","cluster_addr":"172.21.15.193:6827/215104663","heartbeat_back_addr":"172.21.15.193:6831/215104663","heartbeat_front_addr":"172.21.15.193:6829/215104663","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:35:30.211698+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:35:50.883846+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:36:11.124168+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:36:32.711905+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:36:54.442007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:37:14.843526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:37:36.587354+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:37:57.882374+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:38:18.729407+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:38:40.139231+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:39:01.290862+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:39:21.767537+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:0/1762298871":"2023-07-22T18:41:44.230582+0000","172.21.15.39:0/1127767082":"2023-07-22T18:34:07.039823+0000","172.21.15.39:6800/2116291712":"2023-07-22T18:41:44.230582+0000","172.21.15.39:0/3819152602":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/4249110368":"2023-07-22T18:41:44.230582+0000","172.21.15.39:6801/2116291712":"2023-07-22T18:41:44.230582+0000","172.21.15.39:0/1275748831":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/545627125":"2023-07-22T18:41:44.230582+0000","172.21.15.39:6800/889516631":"2023-07-22T18:33:40.852706+0000","172.21.15.39:6800/2893866425":"2023-07-22T18:34:07.039823+0000","172.21.15.39:0/3636510379":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/19717002":"2023-07-22T18:34:07.039823+0000","172.21.15.39:6801/2893866425":"2023-07-22T18:34:07.039823+0000","172.21.15.39:0/2902600414":"2023-07-22T18:34:07.039823+0000","172.21.15.39:0/1870486291":"2023-07-22T18:41:44.230582+0000","172.21.15.39:6801/889516631":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/3797141824":"2023-07-22T18:41:44.230582+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":32,"length":1},{"begin":44,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T19:06:53.330 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.320+0000 7f462f297700 1 -- 172.21.15.39:0/2687278964 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f46140a1300 msgr2=0x7f46140a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:53.330 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.320+0000 7f462f297700 1 --2- 172.21.15.39:0/2687278964 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f46140a1300 0x7f46140a37b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f46280fdbf0 tx=0x7f4624009260 comp rx=0 tx=0).stop 2023-07-21T19:06:53.331 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.320+0000 7f462f297700 1 -- 172.21.15.39:0/2687278964 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46280fe9f0 msgr2=0x7f462819b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:53.331 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.320+0000 7f462f297700 1 --2- 172.21.15.39:0/2687278964 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46280fe9f0 0x7f462819b950 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f4618000c00 tx=0x7f4618009580 comp rx=0 tx=0).stop 2023-07-21T19:06:53.331 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.320+0000 7f462f297700 1 -- 172.21.15.39:0/2687278964 shutdown_connections 2023-07-21T19:06:53.331 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.320+0000 7f462f297700 1 --2- 172.21.15.39:0/2687278964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4628104a00 0x7f46281959c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:53.331 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.320+0000 7f462f297700 1 --2- 172.21.15.39:0/2687278964 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46280fe9f0 0x7f462819b950 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:53.332 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.320+0000 7f462f297700 1 --2- 172.21.15.39:0/2687278964 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f46140a1300 0x7f46140a37b0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:53.332 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.320+0000 7f462f297700 1 --2- 172.21.15.39:0/2687278964 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46280ff3a0 0x7f462819be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:53.332 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.320+0000 7f462f297700 1 -- 172.21.15.39:0/2687278964 >> 172.21.15.39:0/2687278964 conn(0x7f46280fa4f0 msgr2=0x7f46280fba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:53.332 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.320+0000 7f462f297700 1 -- 172.21.15.39:0/2687278964 shutdown_connections 2023-07-21T19:06:53.333 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.320+0000 7f462f297700 1 -- 172.21.15.39:0/2687278964 wait complete. 2023-07-21T19:06:53.343 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-21T18:36:13.572583+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-21T18:40:02.726735+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-21T18:40:03.990112+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 44, 'snap_epoch': 121, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-21T19:06:53.344 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-21T19:06:53.476 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.464+0000 7fa44f4ae700 1 -- 172.21.15.39:0/3601772737 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa44810c3d0 msgr2=0x7fa44810e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:53.476 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.464+0000 7fa44f4ae700 1 --2- 172.21.15.39:0/3601772737 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa44810c3d0 0x7fa44810e7b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa440009f90 tx=0x7fa44000b330 comp rx=0 tx=0).stop 2023-07-21T19:06:53.476 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.464+0000 7fa44f4ae700 1 -- 172.21.15.39:0/3601772737 shutdown_connections 2023-07-21T19:06:53.476 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.464+0000 7fa44f4ae700 1 --2- 172.21.15.39:0/3601772737 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa44810c3d0 0x7fa44810e7b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:53.477 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.464+0000 7fa44f4ae700 1 --2- 172.21.15.39:0/3601772737 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4480fe820 0x7fa4480febf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:53.477 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.464+0000 7fa44f4ae700 1 --2- 172.21.15.39:0/3601772737 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4480ff1c0 0x7fa44810be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:53.477 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.464+0000 7fa44f4ae700 1 -- 172.21.15.39:0/3601772737 >> 172.21.15.39:0/3601772737 conn(0x7fa4480fa4b0 msgr2=0x7fa4480fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:53.477 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.465+0000 7fa44f4ae700 1 -- 172.21.15.39:0/3601772737 shutdown_connections 2023-07-21T19:06:53.477 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.465+0000 7fa44f4ae700 1 -- 172.21.15.39:0/3601772737 wait complete. 2023-07-21T19:06:53.477 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.465+0000 7fa44f4ae700 1 Processor -- start 2023-07-21T19:06:53.477 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.466+0000 7fa44f4ae700 1 -- start start 2023-07-21T19:06:53.478 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.466+0000 7fa44f4ae700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4480fe820 0x7fa448113b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:53.478 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.466+0000 7fa44f4ae700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa44810c3d0 0x7fa4481140e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:53.478 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.466+0000 7fa44f4ae700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa44810f5e0 0x7fa44810fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:53.478 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.466+0000 7fa44f4ae700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa448110190 con 0x7fa44810f5e0 2023-07-21T19:06:53.478 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.466+0000 7fa44f4ae700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa448110300 con 0x7fa44810c3d0 2023-07-21T19:06:53.478 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.466+0000 7fa44f4ae700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa44811b910 con 0x7fa4480fe820 2023-07-21T19:06:53.478 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.466+0000 7fa44da4b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa44810f5e0 0x7fa44810fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:53.478 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.466+0000 7fa44da4b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa44810f5e0 0x7fa44810fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35212/0 (socket says 172.21.15.39:35212) 2023-07-21T19:06:53.479 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.466+0000 7fa44da4b700 1 -- 172.21.15.39:0/2185103094 learned_addr learned my addr 172.21.15.39:0/2185103094 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:06:53.479 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.466+0000 7fa44da4b700 1 -- 172.21.15.39:0/2185103094 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4480fe820 msgr2=0x7fa448113b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:53.479 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.466+0000 7fa44da4b700 1 --2- 172.21.15.39:0/2185103094 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4480fe820 0x7fa448113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:53.479 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.466+0000 7fa44da4b700 1 -- 172.21.15.39:0/2185103094 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa44810c3d0 msgr2=0x7fa4481140e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:53.479 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.466+0000 7fa44da4b700 1 --2- 172.21.15.39:0/2185103094 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa44810c3d0 0x7fa4481140e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:53.479 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.466+0000 7fa44da4b700 1 -- 172.21.15.39:0/2185103094 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa440009c40 con 0x7fa44810f5e0 2023-07-21T19:06:53.480 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.467+0000 7fa44da4b700 1 --2- 172.21.15.39:0/2185103094 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa44810f5e0 0x7fa44810fa80 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fa44000b6c0 tx=0x7fa44000bbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:53.481 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.467+0000 7fa43a7fc700 1 -- 172.21.15.39:0/2185103094 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa440009ae0 con 0x7fa44810f5e0 2023-07-21T19:06:53.481 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.467+0000 7fa44f4ae700 1 -- 172.21.15.39:0/2185103094 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa44811bb90 con 0x7fa44810f5e0 2023-07-21T19:06:53.481 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.467+0000 7fa43a7fc700 1 -- 172.21.15.39:0/2185103094 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa440003e10 con 0x7fa44810f5e0 2023-07-21T19:06:53.481 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.467+0000 7fa44f4ae700 1 -- 172.21.15.39:0/2185103094 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa44811c0f0 con 0x7fa44810f5e0 2023-07-21T19:06:53.481 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.467+0000 7fa43a7fc700 1 -- 172.21.15.39:0/2185103094 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4400054e0 con 0x7fa44810f5e0 2023-07-21T19:06:53.481 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.468+0000 7fa44f4ae700 1 -- 172.21.15.39:0/2185103094 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa42c005320 con 0x7fa44810f5e0 2023-07-21T19:06:53.481 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.469+0000 7fa43a7fc700 1 -- 172.21.15.39:0/2185103094 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fa440014350 con 0x7fa44810f5e0 2023-07-21T19:06:53.482 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.470+0000 7fa43a7fc700 1 --2- 172.21.15.39:0/2185103094 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fa4340a1340 0x7fa4340a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:53.482 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.470+0000 7fa44d24a700 1 --2- 172.21.15.39:0/2185103094 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fa4340a1340 0x7fa4340a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:53.482 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.470+0000 7fa43a7fc700 1 -- 172.21.15.39:0/2185103094 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fa4400ce8d0 con 0x7fa44810f5e0 2023-07-21T19:06:53.482 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.471+0000 7fa44d24a700 1 --2- 172.21.15.39:0/2185103094 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fa4340a1340 0x7fa4340a37f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa444005950 tx=0x7fa4440058e0 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:53.486 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.473+0000 7fa43a7fc700 1 -- 172.21.15.39:0/2185103094 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fa44005e790 con 0x7fa44810f5e0 2023-07-21T19:06:53.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:06:53 smithi039 ceph-mon[96489]: from='client.17952 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:06:53.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:06:53 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T19:06:53.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:06:53 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/2687278964' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:06:53.713 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.703+0000 7fa44f4ae700 1 -- 172.21.15.39:0/2185103094 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa42c005f70 con 0x7fa44810f5e0 2023-07-21T19:06:53.713 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.704+0000 7fa43a7fc700 1 -- 172.21.15.39:0/2185103094 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v122) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa440097e20 con 0x7fa44810f5e0 2023-07-21T19:06:53.714 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2023-07-21T19:06:53.716 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.706+0000 7fa44f4ae700 1 -- 172.21.15.39:0/2185103094 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fa4340a1340 msgr2=0x7fa4340a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:53.716 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.706+0000 7fa44f4ae700 1 --2- 172.21.15.39:0/2185103094 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fa4340a1340 0x7fa4340a37f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa444005950 tx=0x7fa4440058e0 comp rx=0 tx=0).stop 2023-07-21T19:06:53.716 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.706+0000 7fa44f4ae700 1 -- 172.21.15.39:0/2185103094 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa44810f5e0 msgr2=0x7fa44810fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:53.716 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.706+0000 7fa44f4ae700 1 --2- 172.21.15.39:0/2185103094 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa44810f5e0 0x7fa44810fa80 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fa44000b6c0 tx=0x7fa44000bbb0 comp rx=0 tx=0).stop 2023-07-21T19:06:53.717 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.706+0000 7fa44f4ae700 1 -- 172.21.15.39:0/2185103094 shutdown_connections 2023-07-21T19:06:53.717 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.706+0000 7fa44f4ae700 1 --2- 172.21.15.39:0/2185103094 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa44810f5e0 0x7fa44810fa80 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:53.717 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.706+0000 7fa44f4ae700 1 --2- 172.21.15.39:0/2185103094 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4480fe820 0x7fa448113b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:53.717 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.706+0000 7fa44f4ae700 1 --2- 172.21.15.39:0/2185103094 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fa4340a1340 0x7fa4340a37f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:53.718 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.706+0000 7fa44f4ae700 1 --2- 172.21.15.39:0/2185103094 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa44810c3d0 0x7fa4481140e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:53.718 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.706+0000 7fa44f4ae700 1 -- 172.21.15.39:0/2185103094 >> 172.21.15.39:0/2185103094 conn(0x7fa4480fa4b0 msgr2=0x7fa44810d750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:53.718 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.706+0000 7fa44f4ae700 1 -- 172.21.15.39:0/2185103094 shutdown_connections 2023-07-21T19:06:53.718 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.706+0000 7fa44f4ae700 1 -- 172.21.15.39:0/2185103094 wait complete. 2023-07-21T19:06:53.728 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-21T19:06:53.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:06:53 smithi193 ceph-mon[98204]: from='client.17952 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:06:53.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:06:53 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T19:06:53.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:06:53 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/2687278964' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:06:53.860 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.848+0000 7f765e1ca700 1 -- 172.21.15.39:0/3996142812 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76580fe9d0 msgr2=0x7f76580fee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:53.860 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.848+0000 7f765e1ca700 1 --2- 172.21.15.39:0/3996142812 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76580fe9d0 0x7f76580fee40 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f7640009990 tx=0x7f7640009ca0 comp rx=0 tx=0).stop 2023-07-21T19:06:53.860 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.848+0000 7f765e1ca700 1 -- 172.21.15.39:0/3996142812 shutdown_connections 2023-07-21T19:06:53.860 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.848+0000 7f765e1ca700 1 --2- 172.21.15.39:0/3996142812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76581049e0 0x7f7658104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:53.860 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.848+0000 7f765e1ca700 1 --2- 172.21.15.39:0/3996142812 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f76580ff380 0x7f765810b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:53.861 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.848+0000 7f765e1ca700 1 --2- 172.21.15.39:0/3996142812 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76580fe9d0 0x7f76580fee40 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:53.861 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.848+0000 7f765e1ca700 1 -- 172.21.15.39:0/3996142812 >> 172.21.15.39:0/3996142812 conn(0x7f76580fa4b0 msgr2=0x7f76580fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:53.861 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.848+0000 7f765e1ca700 1 -- 172.21.15.39:0/3996142812 shutdown_connections 2023-07-21T19:06:53.861 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.848+0000 7f765e1ca700 1 -- 172.21.15.39:0/3996142812 wait complete. 2023-07-21T19:06:53.861 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.849+0000 7f765e1ca700 1 Processor -- start 2023-07-21T19:06:53.861 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.849+0000 7f765e1ca700 1 -- start start 2023-07-21T19:06:53.862 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.849+0000 7f765e1ca700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f76580fe9d0 0x7f76581969d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:53.862 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.849+0000 7f765e1ca700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76580ff380 0x7f7658196f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:53.862 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.849+0000 7f765e1ca700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76581049e0 0x7f76581a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:53.862 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.849+0000 7f765e1ca700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7658197850 con 0x7f76581049e0 2023-07-21T19:06:53.862 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.849+0000 7f765e1ca700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76581a1f90 con 0x7f76580ff380 2023-07-21T19:06:53.862 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.849+0000 7f765e1ca700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76581a2150 con 0x7f76580fe9d0 2023-07-21T19:06:53.862 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.849+0000 7f7657fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76581049e0 0x7f76581a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:53.863 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.849+0000 7f7657fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76581049e0 0x7f76581a1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35230/0 (socket says 172.21.15.39:35230) 2023-07-21T19:06:53.863 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.849+0000 7f7657fff700 1 -- 172.21.15.39:0/2000085165 learned_addr learned my addr 172.21.15.39:0/2000085165 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:06:53.863 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.849+0000 7f7656ffd700 1 --2- 172.21.15.39:0/2000085165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76580ff380 0x7f7658196f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:53.863 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.849+0000 7f76577fe700 1 --2- 172.21.15.39:0/2000085165 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f76580fe9d0 0x7f76581969d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:53.863 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.850+0000 7f7657fff700 1 -- 172.21.15.39:0/2000085165 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f76580fe9d0 msgr2=0x7f76581969d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:53.863 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.850+0000 7f7657fff700 1 --2- 172.21.15.39:0/2000085165 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f76580fe9d0 0x7f76581969d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:53.863 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.850+0000 7f7657fff700 1 -- 172.21.15.39:0/2000085165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76580ff380 msgr2=0x7f7658196f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:53.864 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.850+0000 7f7657fff700 1 --2- 172.21.15.39:0/2000085165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76580ff380 0x7f7658196f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:53.864 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.850+0000 7f7657fff700 1 -- 172.21.15.39:0/2000085165 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76400095f0 con 0x7f76581049e0 2023-07-21T19:06:53.864 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.850+0000 7f7657fff700 1 --2- 172.21.15.39:0/2000085165 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76581049e0 0x7f76581a1a50 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f7650007ee0 tx=0x7f765000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:53.864 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.850+0000 7f7654ff9700 1 -- 172.21.15.39:0/2000085165 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f765000dd20 con 0x7f76581049e0 2023-07-21T19:06:53.864 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.850+0000 7f7654ff9700 1 -- 172.21.15.39:0/2000085165 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f765000f040 con 0x7f76581049e0 2023-07-21T19:06:53.864 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.850+0000 7f765e1ca700 1 -- 172.21.15.39:0/2000085165 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76581a2430 con 0x7f76581049e0 2023-07-21T19:06:53.864 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.850+0000 7f7654ff9700 1 -- 172.21.15.39:0/2000085165 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76500048a0 con 0x7f76581049e0 2023-07-21T19:06:53.865 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.850+0000 7f765e1ca700 1 -- 172.21.15.39:0/2000085165 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76581a2910 con 0x7f76581049e0 2023-07-21T19:06:53.866 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.851+0000 7f765e1ca700 1 -- 172.21.15.39:0/2000085165 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7658067290 con 0x7f76581049e0 2023-07-21T19:06:53.866 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.852+0000 7f7654ff9700 1 -- 172.21.15.39:0/2000085165 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f7650003a00 con 0x7f76581049e0 2023-07-21T19:06:53.866 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.853+0000 7f7654ff9700 1 --2- 172.21.15.39:0/2000085165 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f76440a12c0 0x7f76440a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:53.866 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.853+0000 7f76577fe700 1 --2- 172.21.15.39:0/2000085165 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f76440a12c0 0x7f76440a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:53.866 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.853+0000 7f7654ff9700 1 -- 172.21.15.39:0/2000085165 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f76500ccd60 con 0x7f76581049e0 2023-07-21T19:06:53.866 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.853+0000 7f76577fe700 1 --2- 172.21.15.39:0/2000085165 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f76440a12c0 0x7f76440a3770 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f76580ffb10 tx=0x7f764800b410 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:53.867 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:53.856+0000 7f7654ff9700 1 -- 172.21.15.39:0/2000085165 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f7650096170 con 0x7f76581049e0 2023-07-21T19:06:53.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:06:53 smithi155 ceph-mon[97934]: from='client.17952 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:06:53.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:06:53 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T19:06:53.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:06:53 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/2687278964' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:06:54.063 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.054+0000 7f765e1ca700 1 -- 172.21.15.39:0/2000085165 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f76581a2b30 con 0x7f76581049e0 2023-07-21T19:06:54.064 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.054+0000 7f7654ff9700 1 -- 172.21.15.39:0/2000085165 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v122) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7650099790 con 0x7f76581049e0 2023-07-21T19:06:54.064 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2023-07-21T19:06:54.066 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.056+0000 7f765e1ca700 1 -- 172.21.15.39:0/2000085165 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f76440a12c0 msgr2=0x7f76440a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:54.066 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.056+0000 7f765e1ca700 1 --2- 172.21.15.39:0/2000085165 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f76440a12c0 0x7f76440a3770 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f76580ffb10 tx=0x7f764800b410 comp rx=0 tx=0).stop 2023-07-21T19:06:54.066 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.056+0000 7f765e1ca700 1 -- 172.21.15.39:0/2000085165 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76581049e0 msgr2=0x7f76581a1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:54.066 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.056+0000 7f765e1ca700 1 --2- 172.21.15.39:0/2000085165 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76581049e0 0x7f76581a1a50 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f7650007ee0 tx=0x7f765000d3b0 comp rx=0 tx=0).stop 2023-07-21T19:06:54.066 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.056+0000 7f765e1ca700 1 -- 172.21.15.39:0/2000085165 shutdown_connections 2023-07-21T19:06:54.066 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.056+0000 7f765e1ca700 1 --2- 172.21.15.39:0/2000085165 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76581049e0 0x7f76581a1a50 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:54.067 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.056+0000 7f765e1ca700 1 --2- 172.21.15.39:0/2000085165 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f76580fe9d0 0x7f76581969d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:54.067 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.056+0000 7f765e1ca700 1 --2- 172.21.15.39:0/2000085165 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f76440a12c0 0x7f76440a3770 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:54.067 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.056+0000 7f765e1ca700 1 --2- 172.21.15.39:0/2000085165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76580ff380 0x7f7658196f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:54.067 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.056+0000 7f765e1ca700 1 -- 172.21.15.39:0/2000085165 >> 172.21.15.39:0/2000085165 conn(0x7f76580fa4b0 msgr2=0x7f76580fbad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:54.067 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.056+0000 7f765e1ca700 1 -- 172.21.15.39:0/2000085165 shutdown_connections 2023-07-21T19:06:54.067 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.056+0000 7f765e1ca700 1 -- 172.21.15.39:0/2000085165 wait complete. 2023-07-21T19:06:54.079 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-21T19:06:54.208 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.198+0000 7fb05bfff700 1 -- 172.21.15.39:0/3629466746 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb05c0fe8d0 msgr2=0x7fb05c0fed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:54.209 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.198+0000 7fb05bfff700 1 --2- 172.21.15.39:0/3629466746 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb05c0fe8d0 0x7fb05c0fed40 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fb04c009990 tx=0x7fb04c009ca0 comp rx=0 tx=0).stop 2023-07-21T19:06:54.209 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.198+0000 7fb05bfff700 1 -- 172.21.15.39:0/3629466746 shutdown_connections 2023-07-21T19:06:54.209 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.198+0000 7fb05bfff700 1 --2- 172.21.15.39:0/3629466746 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb05c0ff280 0x7fb05c10b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:54.209 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.198+0000 7fb05bfff700 1 --2- 172.21.15.39:0/3629466746 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb05c1048e0 0x7fb05c104cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:54.209 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.198+0000 7fb05bfff700 1 --2- 172.21.15.39:0/3629466746 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb05c0fe8d0 0x7fb05c0fed40 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:54.209 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.198+0000 7fb05bfff700 1 -- 172.21.15.39:0/3629466746 >> 172.21.15.39:0/3629466746 conn(0x7fb05c0745d0 msgr2=0x7fb05c0749d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:54.209 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.199+0000 7fb05bfff700 1 -- 172.21.15.39:0/3629466746 shutdown_connections 2023-07-21T19:06:54.210 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.199+0000 7fb05bfff700 1 -- 172.21.15.39:0/3629466746 wait complete. 2023-07-21T19:06:54.210 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.199+0000 7fb05bfff700 1 Processor -- start 2023-07-21T19:06:54.211 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.199+0000 7fb05bfff700 1 -- start start 2023-07-21T19:06:54.212 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.200+0000 7fb05bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb05c0fe8d0 0x7fb05c1967e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:54.212 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.200+0000 7fb05bfff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb05c0ff280 0x7fb05c196d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:54.212 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.200+0000 7fb05bfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb05c1048e0 0x7fb05c1a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:54.212 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.200+0000 7fb05bfff700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb05c197630 con 0x7fb05c0fe8d0 2023-07-21T19:06:54.212 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.200+0000 7fb05bfff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb05c1a1da0 con 0x7fb05c1048e0 2023-07-21T19:06:54.212 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.200+0000 7fb05bfff700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb05c1a1f10 con 0x7fb05c0ff280 2023-07-21T19:06:54.213 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.200+0000 7fb05affd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb05c0fe8d0 0x7fb05c1967e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:54.213 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.200+0000 7fb05affd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb05c0fe8d0 0x7fb05c1967e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35238/0 (socket says 172.21.15.39:35238) 2023-07-21T19:06:54.213 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.200+0000 7fb05affd700 1 -- 172.21.15.39:0/2067267084 learned_addr learned my addr 172.21.15.39:0/2067267084 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:06:54.213 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.200+0000 7fb05a7fc700 1 --2- 172.21.15.39:0/2067267084 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb05c0ff280 0x7fb05c196d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:54.213 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.200+0000 7fb05affd700 1 -- 172.21.15.39:0/2067267084 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb05c0ff280 msgr2=0x7fb05c196d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:54.213 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.200+0000 7fb05affd700 1 --2- 172.21.15.39:0/2067267084 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb05c0ff280 0x7fb05c196d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:54.213 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.200+0000 7fb05affd700 1 -- 172.21.15.39:0/2067267084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb05c1048e0 msgr2=0x7fb05c1a1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:54.214 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.200+0000 7fb05b7fe700 1 --2- 172.21.15.39:0/2067267084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb05c1048e0 0x7fb05c1a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:54.214 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.200+0000 7fb05affd700 1 --2- 172.21.15.39:0/2067267084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb05c1048e0 0x7fb05c1a1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:54.214 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.200+0000 7fb05affd700 1 -- 172.21.15.39:0/2067267084 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb04c0095f0 con 0x7fb05c0fe8d0 2023-07-21T19:06:54.214 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.200+0000 7fb05affd700 1 --2- 172.21.15.39:0/2067267084 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb05c0fe8d0 0x7fb05c1967e0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fb05000e970 tx=0x7fb05000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:54.214 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.200+0000 7fb043fff700 1 -- 172.21.15.39:0/2067267084 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb050009790 con 0x7fb05c0fe8d0 2023-07-21T19:06:54.214 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.200+0000 7fb043fff700 1 -- 172.21.15.39:0/2067267084 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb05000eec0 con 0x7fb05c0fe8d0 2023-07-21T19:06:54.214 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.200+0000 7fb05bfff700 1 -- 172.21.15.39:0/2067267084 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb05c1a21f0 con 0x7fb05c0fe8d0 2023-07-21T19:06:54.214 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.200+0000 7fb043fff700 1 -- 172.21.15.39:0/2067267084 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb050003cf0 con 0x7fb05c0fe8d0 2023-07-21T19:06:54.215 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.200+0000 7fb05bfff700 1 -- 172.21.15.39:0/2067267084 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb05c1a2700 con 0x7fb05c0fe8d0 2023-07-21T19:06:54.215 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.201+0000 7fb05bfff700 1 -- 172.21.15.39:0/2067267084 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb05c067220 con 0x7fb05c0fe8d0 2023-07-21T19:06:54.215 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.203+0000 7fb043fff700 1 -- 172.21.15.39:0/2067267084 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fb05000ccd0 con 0x7fb05c0fe8d0 2023-07-21T19:06:54.216 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.204+0000 7fb043fff700 1 --2- 172.21.15.39:0/2067267084 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fb0440a1360 0x7fb0440a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:54.216 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.204+0000 7fb05a7fc700 1 --2- 172.21.15.39:0/2067267084 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fb0440a1360 0x7fb0440a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:54.216 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.204+0000 7fb043fff700 1 -- 172.21.15.39:0/2067267084 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fb050014070 con 0x7fb05c0fe8d0 2023-07-21T19:06:54.216 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.204+0000 7fb05a7fc700 1 --2- 172.21.15.39:0/2067267084 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fb0440a1360 0x7fb0440a3810 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb05c067f20 tx=0x7fb04c005fd0 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:54.217 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.207+0000 7fb043fff700 1 -- 172.21.15.39:0/2067267084 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fb050096470 con 0x7fb05c0fe8d0 2023-07-21T19:06:54.414 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.404+0000 7fb05bfff700 1 -- 172.21.15.39:0/2067267084 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb05c1a29b0 con 0x7fb05c0fe8d0 2023-07-21T19:06:54.414 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.405+0000 7fb043fff700 1 -- 172.21.15.39:0/2067267084 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v122) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb050099a90 con 0x7fb05c0fe8d0 2023-07-21T19:06:54.415 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 64 2023-07-21T19:06:54.416 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.407+0000 7fb05bfff700 1 -- 172.21.15.39:0/2067267084 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fb0440a1360 msgr2=0x7fb0440a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:54.416 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.407+0000 7fb05bfff700 1 --2- 172.21.15.39:0/2067267084 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fb0440a1360 0x7fb0440a3810 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb05c067f20 tx=0x7fb04c005fd0 comp rx=0 tx=0).stop 2023-07-21T19:06:54.417 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.407+0000 7fb05bfff700 1 -- 172.21.15.39:0/2067267084 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb05c0fe8d0 msgr2=0x7fb05c1967e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:54.417 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.407+0000 7fb05bfff700 1 --2- 172.21.15.39:0/2067267084 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb05c0fe8d0 0x7fb05c1967e0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fb05000e970 tx=0x7fb05000ec80 comp rx=0 tx=0).stop 2023-07-21T19:06:54.417 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.407+0000 7fb05bfff700 1 -- 172.21.15.39:0/2067267084 shutdown_connections 2023-07-21T19:06:54.417 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.407+0000 7fb05bfff700 1 --2- 172.21.15.39:0/2067267084 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7fb0440a1360 0x7fb0440a3810 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:54.417 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.407+0000 7fb05bfff700 1 --2- 172.21.15.39:0/2067267084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb05c1048e0 0x7fb05c1a1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:54.417 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.407+0000 7fb05bfff700 1 --2- 172.21.15.39:0/2067267084 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb05c0fe8d0 0x7fb05c1967e0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:54.417 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.407+0000 7fb05bfff700 1 --2- 172.21.15.39:0/2067267084 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb05c0ff280 0x7fb05c196d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:54.418 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.407+0000 7fb05bfff700 1 -- 172.21.15.39:0/2067267084 >> 172.21.15.39:0/2067267084 conn(0x7fb05c0745d0 msgr2=0x7fb05c0fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:54.418 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.407+0000 7fb05bfff700 1 -- 172.21.15.39:0/2067267084 shutdown_connections 2023-07-21T19:06:54.418 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.407+0000 7fb05bfff700 1 -- 172.21.15.39:0/2067267084 wait complete. 2023-07-21T19:06:54.430 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2023-07-21T19:06:54.558 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.547+0000 7f1584561700 1 -- 172.21.15.39:0/396507070 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f157c1049e0 msgr2=0x7f157c104db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:54.558 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.547+0000 7f1584561700 1 --2- 172.21.15.39:0/396507070 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f157c1049e0 0x7f157c104db0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f156c009990 tx=0x7f156c009ca0 comp rx=0 tx=0).stop 2023-07-21T19:06:54.558 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.547+0000 7f1584561700 1 -- 172.21.15.39:0/396507070 shutdown_connections 2023-07-21T19:06:54.559 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.547+0000 7f1584561700 1 --2- 172.21.15.39:0/396507070 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f157c0ff380 0x7f157c10b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:54.559 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.547+0000 7f1584561700 1 --2- 172.21.15.39:0/396507070 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f157c1049e0 0x7f157c104db0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:54.559 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.547+0000 7f1584561700 1 --2- 172.21.15.39:0/396507070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f157c0fe9d0 0x7f157c0fee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:54.559 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.547+0000 7f1584561700 1 -- 172.21.15.39:0/396507070 >> 172.21.15.39:0/396507070 conn(0x7f157c0fa490 msgr2=0x7f157c0fc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:54.559 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.547+0000 7f1584561700 1 -- 172.21.15.39:0/396507070 shutdown_connections 2023-07-21T19:06:54.559 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.547+0000 7f1584561700 1 -- 172.21.15.39:0/396507070 wait complete. 2023-07-21T19:06:54.559 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.547+0000 7f1584561700 1 Processor -- start 2023-07-21T19:06:54.560 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.547+0000 7f1584561700 1 -- start start 2023-07-21T19:06:54.560 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.547+0000 7f1584561700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f157c0fe9d0 0x7f157c1005e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:54.560 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.548+0000 7f1584561700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f157c0ff380 0x7f157c100b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:54.560 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.548+0000 7f1584561700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f157c1049e0 0x7f157c1a3d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:54.560 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.548+0000 7f1584561700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f157c1013c0 con 0x7f157c0ff380 2023-07-21T19:06:54.560 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.548+0000 7f1584561700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f157c1a4250 con 0x7f157c0fe9d0 2023-07-21T19:06:54.560 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.548+0000 7f1584561700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f157c1a4390 con 0x7f157c1049e0 2023-07-21T19:06:54.561 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.548+0000 7f1581afc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f157c0ff380 0x7f157c100b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:54.561 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.548+0000 7f1581afc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f157c0ff380 0x7f157c100b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35254/0 (socket says 172.21.15.39:35254) 2023-07-21T19:06:54.561 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.548+0000 7f1581afc700 1 -- 172.21.15.39:0/4013105668 learned_addr learned my addr 172.21.15.39:0/4013105668 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:06:54.561 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.548+0000 7f1582afe700 1 --2- 172.21.15.39:0/4013105668 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f157c1049e0 0x7f157c1a3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:54.561 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.548+0000 7f15822fd700 1 --2- 172.21.15.39:0/4013105668 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f157c0fe9d0 0x7f157c1005e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:54.562 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.548+0000 7f1581afc700 1 -- 172.21.15.39:0/4013105668 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f157c1049e0 msgr2=0x7f157c1a3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:54.562 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.548+0000 7f1581afc700 1 --2- 172.21.15.39:0/4013105668 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f157c1049e0 0x7f157c1a3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:54.562 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.548+0000 7f1581afc700 1 -- 172.21.15.39:0/4013105668 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f157c0fe9d0 msgr2=0x7f157c1005e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:54.562 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.548+0000 7f1581afc700 1 --2- 172.21.15.39:0/4013105668 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f157c0fe9d0 0x7f157c1005e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:54.562 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.548+0000 7f1581afc700 1 -- 172.21.15.39:0/4013105668 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f156c0095f0 con 0x7f157c0ff380 2023-07-21T19:06:54.562 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.548+0000 7f15822fd700 1 --2- 172.21.15.39:0/4013105668 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f157c0fe9d0 0x7f157c1005e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:06:54.563 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.548+0000 7f1581afc700 1 --2- 172.21.15.39:0/4013105668 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f157c0ff380 0x7f157c100b20 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f157800c9a0 tx=0x7f157800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:54.563 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.548+0000 7f15737fe700 1 -- 172.21.15.39:0/4013105668 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1578007a60 con 0x7f157c0ff380 2023-07-21T19:06:54.563 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.548+0000 7f15737fe700 1 -- 172.21.15.39:0/4013105668 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1578004cf0 con 0x7f157c0ff380 2023-07-21T19:06:54.564 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.548+0000 7f1584561700 1 -- 172.21.15.39:0/4013105668 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f157c1a4640 con 0x7f157c0ff380 2023-07-21T19:06:54.564 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.548+0000 7f15737fe700 1 -- 172.21.15.39:0/4013105668 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1578005560 con 0x7f157c0ff380 2023-07-21T19:06:54.564 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.548+0000 7f1584561700 1 -- 172.21.15.39:0/4013105668 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f157c1a4b50 con 0x7f157c0ff380 2023-07-21T19:06:54.565 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.549+0000 7f1584561700 1 -- 172.21.15.39:0/4013105668 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f157c067300 con 0x7f157c0ff380 2023-07-21T19:06:54.566 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.550+0000 7f15737fe700 1 -- 172.21.15.39:0/4013105668 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f1578004750 con 0x7f157c0ff380 2023-07-21T19:06:54.566 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.551+0000 7f15737fe700 1 --2- 172.21.15.39:0/4013105668 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f15680a1380 0x7f15680a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:54.566 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.551+0000 7f15737fe700 1 -- 172.21.15.39:0/4013105668 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f1578013070 con 0x7f157c0ff380 2023-07-21T19:06:54.566 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.551+0000 7f15822fd700 1 --2- 172.21.15.39:0/4013105668 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f15680a1380 0x7f15680a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:54.567 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.552+0000 7f15822fd700 1 --2- 172.21.15.39:0/4013105668 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f15680a1380 0x7f15680a3830 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f156c00b5c0 tx=0x7f156c005fb0 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:54.567 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.555+0000 7f15737fe700 1 -- 172.21.15.39:0/4013105668 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f1578095ef0 con 0x7f157c0ff380 2023-07-21T19:06:54.833 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.822+0000 7f1584561700 1 -- 172.21.15.39:0/4013105668 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f157c101b10 con 0x7f157c0ff380 2023-07-21T19:06:54.840 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.831+0000 7f15737fe700 1 -- 172.21.15.39:0/4013105668 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v24) v1 ==== 81+0+733769 (secure 0 0 0) 0x7f1578004e60 con 0x7f157c0ff380 2023-07-21T19:06:54.853 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T19:06:54.854 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2023-07-21T19:06:54.854 INFO:teuthology.orchestra.run.smithi039.stdout: "epoch": 24, 2023-07-21T19:06:54.854 INFO:teuthology.orchestra.run.smithi039.stdout: "active_gid": 15471, 2023-07-21T19:06:54.854 INFO:teuthology.orchestra.run.smithi039.stdout: "active_name": "x", 2023-07-21T19:06:54.855 INFO:teuthology.orchestra.run.smithi039.stdout: "active_addrs": { 2023-07-21T19:06:54.855 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2023-07-21T19:06:54.855 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:54.855 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2023-07-21T19:06:54.856 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6800", 2023-07-21T19:06:54.856 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 13616514 2023-07-21T19:06:54.856 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:54.856 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:54.856 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2023-07-21T19:06:54.856 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6801", 2023-07-21T19:06:54.857 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 13616514 2023-07-21T19:06:54.857 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:54.857 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2023-07-21T19:06:54.857 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:54.857 INFO:teuthology.orchestra.run.smithi039.stdout: "active_addr": "172.21.15.39:6801/13616514", 2023-07-21T19:06:54.858 INFO:teuthology.orchestra.run.smithi039.stdout: "active_change": "2023-07-21T18:41:44.230964+0000", 2023-07-21T19:06:54.858 INFO:teuthology.orchestra.run.smithi039.stdout: "active_mgr_features": 4540138322906710015, 2023-07-21T19:06:54.858 INFO:teuthology.orchestra.run.smithi039.stdout: "available": true, 2023-07-21T19:06:54.858 INFO:teuthology.orchestra.run.smithi039.stdout: "standbys": [ 2023-07-21T19:06:54.858 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:54.859 INFO:teuthology.orchestra.run.smithi039.stdout: "gid": 15462, 2023-07-21T19:06:54.859 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "y", 2023-07-21T19:06:54.859 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr_features": 4540138322906710015, 2023-07-21T19:06:54.860 INFO:teuthology.orchestra.run.smithi039.stdout: "available_modules": [ 2023-07-21T19:06:54.860 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:54.861 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "alerts", 2023-07-21T19:06:54.862 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:54.862 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:54.863 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:54.863 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2023-07-21T19:06:54.863 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2023-07-21T19:06:54.864 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2023-07-21T19:06:54.864 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:54.864 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:54.864 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2023-07-21T19:06:54.865 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:54.865 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:54.866 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:54.866 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How frequently to reexamine health status", 2023-07-21T19:06:54.866 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:54.866 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:54.867 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:54.867 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:54.867 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:54.867 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:54.867 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:54.867 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:54.868 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:54.868 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:54.868 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:54.868 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:54.868 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:54.869 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:54.869 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:54.869 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:54.869 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:54.869 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:54.870 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:54.870 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:54.870 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:54.871 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:54.871 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:54.871 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:54.871 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:54.872 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:54.872 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:54.872 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:54.872 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:54.873 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:54.873 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:54.873 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:54.874 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:54.874 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:54.874 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:54.874 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:54.875 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:54.875 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:54.875 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:54.875 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:54.876 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:54.876 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:54.876 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:54.876 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:54.876 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:54.877 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:54.877 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:54.877 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:54.878 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:54.878 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:54.878 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:54.878 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:54.878 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:54.879 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:54.879 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:54.879 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:54.879 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:54.879 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:54.879 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:54.880 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:54.880 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:54.880 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:54.880 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:54.880 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:54.880 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:54.880 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:54.881 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:54.881 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:54.881 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:54.881 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:54.881 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:54.881 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:54.882 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:54.882 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:54.882 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_destination": { 2023-07-21T19:06:54.882 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_destination", 2023-07-21T19:06:54.882 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:54.882 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:54.883 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:54.883 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:54.883 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:54.883 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:54.884 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:54.884 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Email address to send alerts to", 2023-07-21T19:06:54.884 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:54.884 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:54.885 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:54.885 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:54.885 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_from_name": { 2023-07-21T19:06:54.885 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_from_name", 2023-07-21T19:06:54.885 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:54.885 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:54.885 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:54.886 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "Ceph", 2023-07-21T19:06:54.886 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:54.886 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:54.886 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:54.886 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Email From: name", 2023-07-21T19:06:54.886 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:54.886 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:54.887 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:54.887 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:54.887 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_host": { 2023-07-21T19:06:54.887 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_host", 2023-07-21T19:06:54.887 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:54.887 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:54.887 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:54.888 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:54.888 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:54.888 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:54.888 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:54.888 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SMTP server", 2023-07-21T19:06:54.888 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:54.888 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:54.889 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:54.889 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:54.889 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_password": { 2023-07-21T19:06:54.889 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_password", 2023-07-21T19:06:54.889 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:54.889 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:54.889 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:54.889 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:54.890 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:54.890 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:54.890 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:54.890 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Password to authenticate with", 2023-07-21T19:06:54.890 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:54.890 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:54.890 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:54.890 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:54.891 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_port": { 2023-07-21T19:06:54.891 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_port", 2023-07-21T19:06:54.891 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:54.891 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:54.891 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:54.891 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "465", 2023-07-21T19:06:54.891 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:54.892 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:54.892 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:54.892 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SMTP port", 2023-07-21T19:06:54.892 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:54.892 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:54.892 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:54.893 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:54.893 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_sender": { 2023-07-21T19:06:54.893 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_sender", 2023-07-21T19:06:54.893 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:54.893 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:54.894 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:54.894 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:54.894 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:54.894 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:54.894 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:54.894 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SMTP envelope sender", 2023-07-21T19:06:54.895 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:54.895 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:54.895 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:54.895 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:54.896 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_ssl": { 2023-07-21T19:06:54.896 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_ssl", 2023-07-21T19:06:54.896 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:54.897 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:54.897 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:54.897 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:54.897 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:54.898 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:54.898 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:54.898 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use SSL to connect to SMTP server", 2023-07-21T19:06:54.899 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:54.899 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:54.899 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:54.899 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:54.900 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_user": { 2023-07-21T19:06:54.900 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_user", 2023-07-21T19:06:54.900 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:54.901 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:54.901 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:54.901 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:54.901 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:54.902 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:54.902 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:54.902 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "User to authenticate as", 2023-07-21T19:06:54.902 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:54.903 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:54.903 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:54.903 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:54.903 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:54.903 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:54.904 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:54.904 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "balancer", 2023-07-21T19:06:54.904 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:54.904 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:54.905 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:54.905 INFO:teuthology.orchestra.run.smithi039.stdout: "active": { 2023-07-21T19:06:54.905 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "active", 2023-07-21T19:06:54.905 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:54.906 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:54.906 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:54.906 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:54.906 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:54.906 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:54.907 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:54.907 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "automatically balance PGs across cluster", 2023-07-21T19:06:54.907 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:54.907 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:54.908 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:54.908 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:54.908 INFO:teuthology.orchestra.run.smithi039.stdout: "begin_time": { 2023-07-21T19:06:54.909 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "begin_time", 2023-07-21T19:06:54.909 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:54.909 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:54.909 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:54.910 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0000", 2023-07-21T19:06:54.910 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:54.910 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:54.910 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:54.911 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "beginning time of day to automatically balance", 2023-07-21T19:06:54.911 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-21T19:06:54.911 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:54.911 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:54.912 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:54.912 INFO:teuthology.orchestra.run.smithi039.stdout: "begin_weekday": { 2023-07-21T19:06:54.912 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "begin_weekday", 2023-07-21T19:06:54.912 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2023-07-21T19:06:54.913 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:54.913 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:54.913 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2023-07-21T19:06:54.913 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "0", 2023-07-21T19:06:54.914 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "6", 2023-07-21T19:06:54.914 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:54.914 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-07-21T19:06:54.914 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-21T19:06:54.915 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:54.915 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:54.915 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:54.915 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_compat_max_iterations": { 2023-07-21T19:06:54.916 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crush_compat_max_iterations", 2023-07-21T19:06:54.916 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2023-07-21T19:06:54.916 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:54.917 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:54.917 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "25", 2023-07-21T19:06:54.917 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2023-07-21T19:06:54.917 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "250", 2023-07-21T19:06:54.918 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:54.918 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-07-21T19:06:54.918 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:54.919 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:54.919 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:54.919 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:54.919 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_compat_metrics": { 2023-07-21T19:06:54.920 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crush_compat_metrics", 2023-07-21T19:06:54.920 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:54.920 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:54.921 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:54.921 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "pgs,objects,bytes", 2023-07-21T19:06:54.921 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:54.921 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:54.922 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:54.922 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-07-21T19:06:54.922 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-07-21T19:06:54.922 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:54.922 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:54.922 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:54.922 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_compat_step": { 2023-07-21T19:06:54.922 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crush_compat_step", 2023-07-21T19:06:54.923 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2023-07-21T19:06:54.923 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:54.923 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:54.923 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0.5", 2023-07-21T19:06:54.923 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "0.001", 2023-07-21T19:06:54.923 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "0.999", 2023-07-21T19:06:54.923 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:54.923 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "aggressiveness of optimization", 2023-07-21T19:06:54.924 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-07-21T19:06:54.924 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:54.924 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:54.924 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:54.925 INFO:teuthology.orchestra.run.smithi039.stdout: "end_time": { 2023-07-21T19:06:54.925 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "end_time", 2023-07-21T19:06:54.925 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:54.926 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:54.926 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:54.926 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "2359", 2023-07-21T19:06:54.926 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:54.927 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:54.927 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:54.927 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "ending time of day to automatically balance", 2023-07-21T19:06:54.927 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-21T19:06:54.928 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:54.928 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:54.928 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:54.928 INFO:teuthology.orchestra.run.smithi039.stdout: "end_weekday": { 2023-07-21T19:06:54.929 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "end_weekday", 2023-07-21T19:06:54.929 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2023-07-21T19:06:54.929 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:54.929 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:54.930 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2023-07-21T19:06:54.930 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "0", 2023-07-21T19:06:54.930 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "6", 2023-07-21T19:06:54.930 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:54.930 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-07-21T19:06:54.931 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-21T19:06:54.931 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:54.931 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:54.931 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:54.931 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:54.932 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:54.932 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:54.932 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:54.932 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:54.933 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:54.933 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:54.933 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:54.934 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:54.934 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:54.934 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:54.934 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:54.935 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:54.935 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:54.935 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:54.935 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:54.936 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:54.936 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:54.936 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:54.937 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:54.937 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:54.937 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:54.937 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:54.938 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:54.938 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:54.938 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:54.938 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:54.939 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:54.939 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:54.939 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:54.940 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:54.940 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:54.940 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:54.940 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:54.941 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:54.941 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:54.941 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:54.941 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:54.942 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:54.942 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:54.942 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:54.942 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:54.942 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:54.942 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:54.943 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:54.943 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:54.943 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:54.943 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:54.943 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:54.943 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:54.943 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:54.944 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:54.944 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:54.944 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:54.944 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:54.944 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:54.944 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:54.944 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:54.944 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:54.945 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:54.945 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:54.945 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:54.945 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:54.945 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:54.945 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:54.945 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:54.945 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:54.946 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:54.946 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:54.946 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:54.946 INFO:teuthology.orchestra.run.smithi039.stdout: "min_score": { 2023-07-21T19:06:54.946 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "min_score", 2023-07-21T19:06:54.946 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2023-07-21T19:06:54.946 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:54.946 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:54.947 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2023-07-21T19:06:54.947 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:54.947 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:54.947 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:54.947 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-07-21T19:06:54.947 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:54.947 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:54.947 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:54.948 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:54.948 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": { 2023-07-21T19:06:54.948 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mode", 2023-07-21T19:06:54.948 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:54.948 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:54.948 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:54.948 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "upmap", 2023-07-21T19:06:54.949 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:54.949 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:54.949 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:54.949 INFO:teuthology.orchestra.run.smithi039.stdout: "crush-compat", 2023-07-21T19:06:54.949 INFO:teuthology.orchestra.run.smithi039.stdout: "none", 2023-07-21T19:06:54.949 INFO:teuthology.orchestra.run.smithi039.stdout: "upmap" 2023-07-21T19:06:54.949 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:54.949 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Balancer mode", 2023-07-21T19:06:54.950 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:54.950 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:54.950 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:54.950 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:54.950 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_ids": { 2023-07-21T19:06:54.950 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pool_ids", 2023-07-21T19:06:54.950 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:54.950 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:54.951 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:54.951 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:54.951 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:54.951 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:54.951 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:54.951 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-07-21T19:06:54.951 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:54.951 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:54.952 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:54.952 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:54.952 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2023-07-21T19:06:54.952 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2023-07-21T19:06:54.952 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2023-07-21T19:06:54.952 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:54.952 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:54.952 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2023-07-21T19:06:54.953 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:54.953 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:54.953 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:54.953 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-07-21T19:06:54.953 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:54.953 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:54.953 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:54.953 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:54.954 INFO:teuthology.orchestra.run.smithi039.stdout: "upmap_max_deviation": { 2023-07-21T19:06:54.954 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "upmap_max_deviation", 2023-07-21T19:06:54.954 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:54.954 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:54.954 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:54.954 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2023-07-21T19:06:54.954 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2023-07-21T19:06:54.955 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:54.955 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:54.955 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "deviation below which no optimization is attempted", 2023-07-21T19:06:54.955 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-07-21T19:06:54.955 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:54.955 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:54.955 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:54.955 INFO:teuthology.orchestra.run.smithi039.stdout: "upmap_max_optimizations": { 2023-07-21T19:06:54.956 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "upmap_max_optimizations", 2023-07-21T19:06:54.956 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2023-07-21T19:06:54.956 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:54.956 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:54.956 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2023-07-21T19:06:54.956 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:54.956 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:54.956 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:54.957 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-07-21T19:06:54.957 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:54.957 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:54.957 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:54.957 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:54.957 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:54.957 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:54.957 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:54.958 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cephadm", 2023-07-21T19:06:54.958 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:54.958 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:54.958 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:54.958 INFO:teuthology.orchestra.run.smithi039.stdout: "agent_down_multiplier": { 2023-07-21T19:06:54.958 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "agent_down_multiplier", 2023-07-21T19:06:54.958 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2023-07-21T19:06:54.958 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:54.959 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:54.959 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "3.0", 2023-07-21T19:06:54.959 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:54.959 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:54.959 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:54.959 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-07-21T19:06:54.959 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:54.959 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:54.960 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:54.960 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:54.960 INFO:teuthology.orchestra.run.smithi039.stdout: "agent_refresh_rate": { 2023-07-21T19:06:54.960 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "agent_refresh_rate", 2023-07-21T19:06:54.961 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2023-07-21T19:06:54.961 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:54.961 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:54.961 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "20", 2023-07-21T19:06:54.961 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:54.961 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:54.961 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:54.961 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-07-21T19:06:54.962 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:54.962 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:54.962 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:54.962 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:54.962 INFO:teuthology.orchestra.run.smithi039.stdout: "agent_starting_port": { 2023-07-21T19:06:54.962 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "agent_starting_port", 2023-07-21T19:06:54.962 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:54.962 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:54.963 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:54.963 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "4721", 2023-07-21T19:06:55.008 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.008 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.009 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.009 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-07-21T19:06:55.009 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.009 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.009 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.009 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.010 INFO:teuthology.orchestra.run.smithi039.stdout: "alertmanager_web_password": { 2023-07-21T19:06:55.010 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "alertmanager_web_password", 2023-07-21T19:06:55.010 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.010 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.010 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.010 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "admin", 2023-07-21T19:06:55.010 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.011 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.011 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.011 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Alertmanager web password", 2023-07-21T19:06:55.011 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.011 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.011 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.011 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.011 INFO:teuthology.orchestra.run.smithi039.stdout: "alertmanager_web_user": { 2023-07-21T19:06:55.012 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "alertmanager_web_user", 2023-07-21T19:06:55.012 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.012 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.012 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.012 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "admin", 2023-07-21T19:06:55.012 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.012 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.013 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.013 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Alertmanager web user", 2023-07-21T19:06:55.013 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.013 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.013 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.013 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.013 INFO:teuthology.orchestra.run.smithi039.stdout: "allow_ptrace": { 2023-07-21T19:06:55.013 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "allow_ptrace", 2023-07-21T19:06:55.014 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.014 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.014 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.014 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.014 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.014 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.014 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.014 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-07-21T19:06:55.015 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-07-21T19:06:55.015 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.015 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.015 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.015 INFO:teuthology.orchestra.run.smithi039.stdout: "autotune_interval": { 2023-07-21T19:06:55.015 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "autotune_interval", 2023-07-21T19:06:55.015 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2023-07-21T19:06:55.016 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.016 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.016 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2023-07-21T19:06:55.016 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.016 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.016 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.016 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to autotune daemon memory", 2023-07-21T19:06:55.016 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.017 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.017 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.017 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.017 INFO:teuthology.orchestra.run.smithi039.stdout: "autotune_memory_target_ratio": { 2023-07-21T19:06:55.017 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "autotune_memory_target_ratio", 2023-07-21T19:06:55.017 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2023-07-21T19:06:55.017 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.017 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.018 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0.7", 2023-07-21T19:06:55.018 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.018 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.018 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.018 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-07-21T19:06:55.018 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.018 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.018 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.019 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.019 INFO:teuthology.orchestra.run.smithi039.stdout: "cgroups_split": { 2023-07-21T19:06:55.019 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cgroups_split", 2023-07-21T19:06:55.019 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.019 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.019 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.019 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:55.019 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.020 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.020 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.020 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-07-21T19:06:55.020 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.020 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.020 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.020 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.021 INFO:teuthology.orchestra.run.smithi039.stdout: "config_checks_enabled": { 2023-07-21T19:06:55.021 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "config_checks_enabled", 2023-07-21T19:06:55.021 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.021 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.021 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.021 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.021 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.021 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.022 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.022 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-07-21T19:06:55.022 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.022 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.022 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.022 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.022 INFO:teuthology.orchestra.run.smithi039.stdout: "config_dashboard": { 2023-07-21T19:06:55.022 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "config_dashboard", 2023-07-21T19:06:55.023 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.023 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.023 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.023 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:55.023 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.023 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.023 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.023 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-07-21T19:06:55.024 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.024 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.024 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.024 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.024 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_alertmanager": { 2023-07-21T19:06:55.024 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_alertmanager", 2023-07-21T19:06:55.024 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.024 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.025 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.025 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-07-21T19:06:55.025 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.025 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.025 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.025 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2023-07-21T19:06:55.025 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.025 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.026 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.026 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.026 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_base": { 2023-07-21T19:06:55.026 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_base", 2023-07-21T19:06:55.026 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.026 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.026 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.026 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/ceph", 2023-07-21T19:06:55.027 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.027 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.027 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.027 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Container image name, without the tag", 2023-07-21T19:06:55.027 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.027 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.027 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.027 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.028 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_elasticsearch": { 2023-07-21T19:06:55.028 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_elasticsearch", 2023-07-21T19:06:55.028 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.028 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.028 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.028 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-07-21T19:06:55.028 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.028 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.029 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.029 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "elasticsearch container image", 2023-07-21T19:06:55.029 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.029 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.029 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.029 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.029 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_grafana": { 2023-07-21T19:06:55.029 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_grafana", 2023-07-21T19:06:55.030 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.030 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.030 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.030 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-07-21T19:06:55.030 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.030 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.030 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.031 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2023-07-21T19:06:55.031 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.031 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.031 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.031 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.031 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_haproxy": { 2023-07-21T19:06:55.031 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_haproxy", 2023-07-21T19:06:55.032 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.032 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.032 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.032 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-07-21T19:06:55.032 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.032 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.032 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.033 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "HAproxy container image", 2023-07-21T19:06:55.033 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.033 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.033 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.033 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.033 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_jaeger_agent": { 2023-07-21T19:06:55.033 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_jaeger_agent", 2023-07-21T19:06:55.033 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.034 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.034 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.034 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-07-21T19:06:55.034 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.034 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.034 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.034 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Jaeger agent container image", 2023-07-21T19:06:55.034 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.035 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.035 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.035 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.035 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_jaeger_collector": { 2023-07-21T19:06:55.035 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_jaeger_collector", 2023-07-21T19:06:55.035 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.035 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.035 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.036 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-07-21T19:06:55.036 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.036 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.036 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.036 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Jaeger collector container image", 2023-07-21T19:06:55.036 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.036 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.037 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.037 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.037 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_jaeger_query": { 2023-07-21T19:06:55.037 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_jaeger_query", 2023-07-21T19:06:55.037 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.037 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.037 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.037 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-07-21T19:06:55.038 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.038 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.038 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.038 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Jaeger query container image", 2023-07-21T19:06:55.038 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.038 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.038 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.038 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.039 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_keepalived": { 2023-07-21T19:06:55.039 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_keepalived", 2023-07-21T19:06:55.039 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.039 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.039 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.039 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-07-21T19:06:55.039 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.039 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.040 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.040 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Keepalived container image", 2023-07-21T19:06:55.040 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.040 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.040 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.040 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.040 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_loki": { 2023-07-21T19:06:55.040 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_loki", 2023-07-21T19:06:55.041 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.041 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.041 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.041 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-07-21T19:06:55.041 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.041 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.041 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.042 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Loki container image", 2023-07-21T19:06:55.042 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.042 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.042 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.042 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.042 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_node_exporter": { 2023-07-21T19:06:55.042 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_node_exporter", 2023-07-21T19:06:55.042 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.043 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.043 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.043 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-07-21T19:06:55.043 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.043 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.043 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.043 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2023-07-21T19:06:55.043 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.044 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.044 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.044 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.044 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_prometheus": { 2023-07-21T19:06:55.044 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_prometheus", 2023-07-21T19:06:55.044 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.044 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.045 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.045 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-07-21T19:06:55.045 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.045 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.045 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.045 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2023-07-21T19:06:55.045 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.045 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.046 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.046 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.046 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_promtail": { 2023-07-21T19:06:55.046 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_promtail", 2023-07-21T19:06:55.046 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.046 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.046 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.046 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-07-21T19:06:55.047 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.047 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.047 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.047 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Promtail container image", 2023-07-21T19:06:55.047 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.047 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.047 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.047 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.048 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_snmp_gateway": { 2023-07-21T19:06:55.048 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_snmp_gateway", 2023-07-21T19:06:55.048 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.048 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.048 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.048 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-07-21T19:06:55.048 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.048 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.049 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.049 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SNMP Gateway container image", 2023-07-21T19:06:55.049 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.049 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.049 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.049 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.049 INFO:teuthology.orchestra.run.smithi039.stdout: "container_init": { 2023-07-21T19:06:55.049 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_init", 2023-07-21T19:06:55.050 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.050 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.050 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.050 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:55.050 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.050 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.050 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.051 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Run podman/docker with `--init`", 2023-07-21T19:06:55.051 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.051 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.051 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.051 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.051 INFO:teuthology.orchestra.run.smithi039.stdout: "daemon_cache_timeout": { 2023-07-21T19:06:55.051 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "daemon_cache_timeout", 2023-07-21T19:06:55.051 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2023-07-21T19:06:55.052 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.052 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.052 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2023-07-21T19:06:55.052 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.052 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.052 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.052 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-07-21T19:06:55.052 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.053 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.053 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.053 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.053 INFO:teuthology.orchestra.run.smithi039.stdout: "default_cephadm_command_timeout": { 2023-07-21T19:06:55.053 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "default_cephadm_command_timeout", 2023-07-21T19:06:55.053 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2023-07-21T19:06:55.053 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.054 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.054 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "900", 2023-07-21T19:06:55.054 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.054 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.054 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.054 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-07-21T19:06:55.054 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.054 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.055 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.055 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.055 INFO:teuthology.orchestra.run.smithi039.stdout: "default_registry": { 2023-07-21T19:06:55.055 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "default_registry", 2023-07-21T19:06:55.055 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.055 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.055 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.056 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io", 2023-07-21T19:06:55.056 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.056 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.056 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.056 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-07-21T19:06:55.056 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.056 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.056 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.057 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.057 INFO:teuthology.orchestra.run.smithi039.stdout: "device_cache_timeout": { 2023-07-21T19:06:55.057 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "device_cache_timeout", 2023-07-21T19:06:55.057 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2023-07-21T19:06:55.057 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.057 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.057 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "1800", 2023-07-21T19:06:55.057 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.058 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.058 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.058 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "seconds to cache device inventory", 2023-07-21T19:06:55.058 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.058 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.058 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.058 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.058 INFO:teuthology.orchestra.run.smithi039.stdout: "device_enhanced_scan": { 2023-07-21T19:06:55.059 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "device_enhanced_scan", 2023-07-21T19:06:55.059 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.059 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.059 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.059 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.059 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.059 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.059 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.060 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use libstoragemgmt during device scans", 2023-07-21T19:06:55.060 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.060 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.060 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.060 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.060 INFO:teuthology.orchestra.run.smithi039.stdout: "facts_cache_timeout": { 2023-07-21T19:06:55.060 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "facts_cache_timeout", 2023-07-21T19:06:55.061 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2023-07-21T19:06:55.061 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.061 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.061 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2023-07-21T19:06:55.061 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.061 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.061 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.061 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "seconds to cache host facts data", 2023-07-21T19:06:55.062 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.062 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.062 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.062 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.062 INFO:teuthology.orchestra.run.smithi039.stdout: "host_check_interval": { 2023-07-21T19:06:55.062 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "host_check_interval", 2023-07-21T19:06:55.062 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2023-07-21T19:06:55.062 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.063 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.063 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2023-07-21T19:06:55.063 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.063 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.063 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.063 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to perform a host check", 2023-07-21T19:06:55.063 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.063 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.064 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.064 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.064 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:55.064 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:55.064 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.064 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.064 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.064 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.065 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.065 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.065 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.065 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.065 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.065 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.065 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.065 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.066 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.066 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.066 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.066 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.066 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.066 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.066 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.067 INFO:teuthology.orchestra.run.smithi039.stdout: "log_refresh_metadata": { 2023-07-21T19:06:55.067 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_refresh_metadata", 2023-07-21T19:06:55.067 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.067 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.067 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.067 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.067 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.067 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.068 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.068 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-07-21T19:06:55.068 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.068 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.068 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.068 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.068 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:55.068 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:55.069 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.069 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.069 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.069 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:55.069 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.069 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.069 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.069 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-07-21T19:06:55.070 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.070 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.070 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.070 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.070 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:55.070 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:55.070 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.070 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.071 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.071 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:55.071 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.071 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.071 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.071 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.071 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.071 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.072 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.072 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.072 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.072 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.072 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.072 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.072 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.072 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.073 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.073 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:55.073 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:55.073 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.073 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.073 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.073 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.073 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.074 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.074 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.074 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.074 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.074 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.074 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.074 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.074 INFO:teuthology.orchestra.run.smithi039.stdout: "manage_etc_ceph_ceph_conf": { 2023-07-21T19:06:55.075 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-07-21T19:06:55.075 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.075 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.075 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.075 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.075 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.075 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.076 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.076 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-07-21T19:06:55.076 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.076 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.076 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.076 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.076 INFO:teuthology.orchestra.run.smithi039.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-07-21T19:06:55.077 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-07-21T19:06:55.077 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.077 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.077 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.077 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "*", 2023-07-21T19:06:55.077 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.077 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.077 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.078 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-07-21T19:06:55.078 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.078 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.078 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.078 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.078 INFO:teuthology.orchestra.run.smithi039.stdout: "max_count_per_host": { 2023-07-21T19:06:55.078 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_count_per_host", 2023-07-21T19:06:55.078 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:55.079 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.079 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.079 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2023-07-21T19:06:55.079 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.079 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.079 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.079 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "max number of daemons per service per host", 2023-07-21T19:06:55.079 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.080 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.080 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.080 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.080 INFO:teuthology.orchestra.run.smithi039.stdout: "max_osd_draining_count": { 2023-07-21T19:06:55.080 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_osd_draining_count", 2023-07-21T19:06:55.080 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:55.080 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.081 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.081 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2023-07-21T19:06:55.081 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.081 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.081 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.081 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-07-21T19:06:55.081 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.081 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.082 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.082 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.082 INFO:teuthology.orchestra.run.smithi039.stdout: "migration_current": { 2023-07-21T19:06:55.082 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "migration_current", 2023-07-21T19:06:55.082 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:55.082 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.082 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.082 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.083 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.083 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.083 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.083 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "internal - do not modify", 2023-07-21T19:06:55.083 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.083 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.083 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.083 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.084 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": { 2023-07-21T19:06:55.084 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mode", 2023-07-21T19:06:55.084 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.084 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.084 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.084 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "root", 2023-07-21T19:06:55.084 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.085 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.085 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.085 INFO:teuthology.orchestra.run.smithi039.stdout: "cephadm-package", 2023-07-21T19:06:55.085 INFO:teuthology.orchestra.run.smithi039.stdout: "root" 2023-07-21T19:06:55.085 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.085 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "mode for remote execution of cephadm", 2023-07-21T19:06:55.085 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.085 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.086 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.086 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.086 INFO:teuthology.orchestra.run.smithi039.stdout: "prometheus_alerts_path": { 2023-07-21T19:06:55.086 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "prometheus_alerts_path", 2023-07-21T19:06:55.086 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.086 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.086 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.086 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-07-21T19:06:55.087 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.087 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.087 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.087 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-07-21T19:06:55.087 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.087 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.087 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.087 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.088 INFO:teuthology.orchestra.run.smithi039.stdout: "prometheus_web_password": { 2023-07-21T19:06:55.088 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "prometheus_web_password", 2023-07-21T19:06:55.088 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.088 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.088 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.088 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "admin", 2023-07-21T19:06:55.088 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.088 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.089 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.089 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus web password", 2023-07-21T19:06:55.089 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.089 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.089 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.089 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.089 INFO:teuthology.orchestra.run.smithi039.stdout: "prometheus_web_user": { 2023-07-21T19:06:55.089 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "prometheus_web_user", 2023-07-21T19:06:55.090 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.090 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.090 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.090 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "admin", 2023-07-21T19:06:55.090 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.090 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.090 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.090 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus web user", 2023-07-21T19:06:55.091 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.091 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.091 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.091 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.091 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_insecure": { 2023-07-21T19:06:55.091 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_insecure", 2023-07-21T19:06:55.091 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.091 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.092 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.092 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.092 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.092 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.092 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.092 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-07-21T19:06:55.092 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.093 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.093 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.093 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.093 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_password": { 2023-07-21T19:06:55.093 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_password", 2023-07-21T19:06:55.093 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.093 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.093 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.094 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.094 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.094 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.094 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.094 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-07-21T19:06:55.094 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.094 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.094 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.095 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.095 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_url": { 2023-07-21T19:06:55.095 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_url", 2023-07-21T19:06:55.095 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.095 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.095 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.095 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.095 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.096 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.096 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.096 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-07-21T19:06:55.096 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.096 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.096 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.096 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.097 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_username": { 2023-07-21T19:06:55.097 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_username", 2023-07-21T19:06:55.097 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.097 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.097 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.097 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.097 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.097 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.098 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.098 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-07-21T19:06:55.098 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.098 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.098 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.098 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.098 INFO:teuthology.orchestra.run.smithi039.stdout: "secure_monitoring_stack": { 2023-07-21T19:06:55.099 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "secure_monitoring_stack", 2023-07-21T19:06:55.099 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.099 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.099 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.099 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.099 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.099 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.099 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.100 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-07-21T19:06:55.100 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.100 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.100 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.100 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.100 INFO:teuthology.orchestra.run.smithi039.stdout: "service_discovery_port": { 2023-07-21T19:06:55.100 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "service_discovery_port", 2023-07-21T19:06:55.101 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:55.101 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.101 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.101 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8765", 2023-07-21T19:06:55.101 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.101 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.101 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.101 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "cephadm service discovery port", 2023-07-21T19:06:55.102 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.102 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.102 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.102 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.102 INFO:teuthology.orchestra.run.smithi039.stdout: "ssh_config_file": { 2023-07-21T19:06:55.102 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssh_config_file", 2023-07-21T19:06:55.102 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.102 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.103 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.103 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.103 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.103 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.103 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.103 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-07-21T19:06:55.103 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.103 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.104 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.104 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.104 INFO:teuthology.orchestra.run.smithi039.stdout: "use_agent": { 2023-07-21T19:06:55.104 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "use_agent", 2023-07-21T19:06:55.104 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.104 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.104 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.104 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.105 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.105 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.105 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.105 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-07-21T19:06:55.105 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.105 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.105 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.105 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.106 INFO:teuthology.orchestra.run.smithi039.stdout: "use_repo_digest": { 2023-07-21T19:06:55.106 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "use_repo_digest", 2023-07-21T19:06:55.106 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.106 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.106 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.106 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:55.106 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.107 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.107 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.107 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-07-21T19:06:55.107 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.107 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.107 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.107 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.107 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_on_failed_host_check": { 2023-07-21T19:06:55.108 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_on_failed_host_check", 2023-07-21T19:06:55.108 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.108 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.108 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.108 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:55.108 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.108 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.108 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.109 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise a health warning if the host check fails", 2023-07-21T19:06:55.109 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.109 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.109 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.109 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.109 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_on_stray_daemons": { 2023-07-21T19:06:55.109 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_on_stray_daemons", 2023-07-21T19:06:55.109 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.110 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.110 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.110 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:55.110 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.110 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.110 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.110 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-07-21T19:06:55.110 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.111 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.111 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.111 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.111 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_on_stray_hosts": { 2023-07-21T19:06:55.111 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_on_stray_hosts", 2023-07-21T19:06:55.111 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.111 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.111 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.112 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:55.112 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.112 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.112 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.112 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-07-21T19:06:55.112 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.112 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.113 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.113 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.113 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.113 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.113 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:55.113 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crash", 2023-07-21T19:06:55.113 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:55.113 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:55.114 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:55.114 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:55.114 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:55.114 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.114 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.114 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.114 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.114 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.115 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.115 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.115 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.115 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.115 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.115 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.115 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.115 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.116 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.116 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.116 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.116 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.116 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.116 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.116 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:55.116 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:55.117 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.117 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.117 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.117 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.117 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.117 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.117 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.117 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.118 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.118 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.118 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.118 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.118 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:55.118 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:55.118 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.118 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.119 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.119 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:55.119 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.119 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.119 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.119 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.119 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.119 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.120 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.120 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.120 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.120 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.120 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.120 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.120 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.121 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.121 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.121 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:55.121 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:55.121 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.121 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.121 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.122 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.122 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.122 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.122 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.122 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.122 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.122 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.122 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.123 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.123 INFO:teuthology.orchestra.run.smithi039.stdout: "retain_interval": { 2023-07-21T19:06:55.123 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "retain_interval", 2023-07-21T19:06:55.123 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2023-07-21T19:06:55.123 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.123 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.123 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "31536000", 2023-07-21T19:06:55.123 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.124 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.124 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.124 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how long to retain crashes before pruning them", 2023-07-21T19:06:55.124 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.124 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.124 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.124 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.125 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_recent_interval": { 2023-07-21T19:06:55.125 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_recent_interval", 2023-07-21T19:06:55.125 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2023-07-21T19:06:55.125 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.125 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.125 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "1209600", 2023-07-21T19:06:55.125 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.125 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.126 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.126 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "time interval in which to warn about recent crashes", 2023-07-21T19:06:55.126 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.126 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.126 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.126 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.126 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.126 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.127 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:55.127 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "dashboard", 2023-07-21T19:06:55.127 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:55.127 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:55.127 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:55.127 INFO:teuthology.orchestra.run.smithi039.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-07-21T19:06:55.127 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-07-21T19:06:55.127 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:55.128 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.128 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.128 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2023-07-21T19:06:55.128 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.128 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.128 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.128 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.128 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.129 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.129 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.129 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.129 INFO:teuthology.orchestra.run.smithi039.stdout: "ALERTMANAGER_API_HOST": { 2023-07-21T19:06:55.129 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ALERTMANAGER_API_HOST", 2023-07-21T19:06:55.129 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.129 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.130 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.130 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.130 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.130 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.130 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.130 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.130 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.130 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.131 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.131 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.131 INFO:teuthology.orchestra.run.smithi039.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-07-21T19:06:55.131 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-07-21T19:06:55.131 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.131 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.131 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.131 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:55.132 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.132 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.132 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.132 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.132 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.132 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.132 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.132 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.133 INFO:teuthology.orchestra.run.smithi039.stdout: "AUDIT_API_ENABLED": { 2023-07-21T19:06:55.133 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "AUDIT_API_ENABLED", 2023-07-21T19:06:55.133 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.133 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.133 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.133 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.133 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.133 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.134 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.134 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.134 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.134 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.134 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.134 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.134 INFO:teuthology.orchestra.run.smithi039.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-07-21T19:06:55.134 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-07-21T19:06:55.135 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.135 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.135 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.135 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:55.135 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.135 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.135 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.135 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.136 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.136 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.136 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.136 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.136 INFO:teuthology.orchestra.run.smithi039.stdout: "ENABLE_BROWSABLE_API": { 2023-07-21T19:06:55.136 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ENABLE_BROWSABLE_API", 2023-07-21T19:06:55.136 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.137 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.137 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.137 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:55.137 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.137 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.137 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.137 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.137 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.138 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.138 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.138 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.138 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-07-21T19:06:55.138 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-07-21T19:06:55.138 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.138 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.138 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.139 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:55.139 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.139 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.139 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.139 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.139 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.139 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.139 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.140 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.140 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-07-21T19:06:55.140 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-07-21T19:06:55.140 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.140 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.140 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.140 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:55.140 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.141 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.141 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.141 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.141 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.141 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.141 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.141 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.141 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-07-21T19:06:55.142 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-07-21T19:06:55.142 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.142 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.142 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.142 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:55.142 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.142 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.143 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.143 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.143 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.143 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.143 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.143 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.143 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-07-21T19:06:55.143 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-07-21T19:06:55.144 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.144 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.144 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.144 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:55.144 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.144 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.144 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.144 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.145 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.145 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.145 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.145 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.145 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_NFS": { 2023-07-21T19:06:55.145 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-07-21T19:06:55.145 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.146 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.146 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.146 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:55.146 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.146 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.146 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.146 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.146 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.147 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.147 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.147 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.147 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_RBD": { 2023-07-21T19:06:55.147 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-07-21T19:06:55.147 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.147 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.147 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.148 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:55.148 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.148 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.148 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.148 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.148 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.148 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.149 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.149 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.149 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_RGW": { 2023-07-21T19:06:55.149 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-07-21T19:06:55.149 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.149 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.149 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.149 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:55.150 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.150 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.150 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.150 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.150 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.150 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.150 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.150 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.151 INFO:teuthology.orchestra.run.smithi039.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-07-21T19:06:55.151 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-07-21T19:06:55.151 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.152 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.152 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.152 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.152 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.152 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.152 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.152 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.153 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.153 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.153 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.153 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.153 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_PASSWORD": { 2023-07-21T19:06:55.153 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_PASSWORD", 2023-07-21T19:06:55.153 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.153 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.154 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.154 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "admin", 2023-07-21T19:06:55.154 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.154 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.154 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.154 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.154 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.154 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.155 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.155 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.155 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-07-21T19:06:55.155 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-07-21T19:06:55.155 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.155 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.155 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.156 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:55.156 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.156 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.156 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.156 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.156 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.156 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.156 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.157 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.157 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_URL": { 2023-07-21T19:06:55.157 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_URL", 2023-07-21T19:06:55.157 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.157 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.157 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.157 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.157 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.158 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.158 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.158 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.158 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.158 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.158 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.158 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.158 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_USERNAME": { 2023-07-21T19:06:55.159 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_USERNAME", 2023-07-21T19:06:55.159 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.159 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.159 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.159 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "admin", 2023-07-21T19:06:55.159 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.159 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.159 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.160 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.160 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.160 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.160 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.160 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.160 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-07-21T19:06:55.160 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-07-21T19:06:55.160 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.161 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.161 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.161 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.161 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.161 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.161 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.161 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.161 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.162 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.162 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.162 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.162 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-07-21T19:06:55.162 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-07-21T19:06:55.162 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.162 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.162 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.163 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.163 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.163 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.163 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.163 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.163 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.163 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.164 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.164 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.164 INFO:teuthology.orchestra.run.smithi039.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-07-21T19:06:55.164 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-07-21T19:06:55.164 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.164 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.164 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.164 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:55.165 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.165 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.165 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.165 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.165 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.165 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.165 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.165 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.166 INFO:teuthology.orchestra.run.smithi039.stdout: "ISSUE_TRACKER_API_KEY": { 2023-07-21T19:06:55.166 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-07-21T19:06:55.166 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.166 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.166 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.166 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.166 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.166 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.167 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.167 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.167 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.167 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.167 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.167 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.167 INFO:teuthology.orchestra.run.smithi039.stdout: "PROMETHEUS_API_HOST": { 2023-07-21T19:06:55.167 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PROMETHEUS_API_HOST", 2023-07-21T19:06:55.168 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.168 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.168 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.168 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.168 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.168 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.168 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.168 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.169 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.169 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.169 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.169 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.169 INFO:teuthology.orchestra.run.smithi039.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-07-21T19:06:55.169 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-07-21T19:06:55.169 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.170 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.170 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.170 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:55.170 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.170 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.170 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.170 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.171 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.171 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.171 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.171 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.171 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-07-21T19:06:55.171 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-07-21T19:06:55.171 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.171 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.172 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.172 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.172 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.172 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.172 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.172 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.172 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.172 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.173 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.173 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.173 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-07-21T19:06:55.173 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-07-21T19:06:55.173 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.173 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.173 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.173 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.174 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.174 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.174 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.174 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.174 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.174 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.174 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.174 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.175 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-07-21T19:06:55.175 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-07-21T19:06:55.175 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.175 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.175 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.175 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:55.175 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.176 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.176 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.176 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.176 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.176 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.176 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.176 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.176 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-07-21T19:06:55.177 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-07-21T19:06:55.177 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.177 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.177 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.177 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:55.177 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.177 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.177 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.178 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.178 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.178 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.178 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.178 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.178 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-07-21T19:06:55.178 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-07-21T19:06:55.178 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.179 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.179 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.179 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.179 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.179 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.179 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.179 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.179 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.180 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.180 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.180 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.180 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-07-21T19:06:55.180 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-07-21T19:06:55.180 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.180 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.180 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.181 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.181 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.181 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.181 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.181 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.181 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.181 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.181 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.182 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.182 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-07-21T19:06:55.182 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-07-21T19:06:55.182 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.182 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.182 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.182 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.182 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.183 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.183 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.183 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.183 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.183 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.183 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.183 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.183 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_ENABLED": { 2023-07-21T19:06:55.184 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_ENABLED", 2023-07-21T19:06:55.184 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.184 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.184 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.184 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:55.184 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.184 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.185 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.185 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.185 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.185 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.185 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.185 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.185 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-07-21T19:06:55.185 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-07-21T19:06:55.186 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.186 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.186 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.186 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-07-21T19:06:55.186 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.186 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.186 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.186 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.187 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.187 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.187 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.187 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.187 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-07-21T19:06:55.187 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-07-21T19:06:55.187 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:55.187 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.188 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.188 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2023-07-21T19:06:55.188 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.188 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.188 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.188 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.188 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.188 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.189 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.189 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.189 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-07-21T19:06:55.189 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-07-21T19:06:55.189 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:55.189 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.189 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.190 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8", 2023-07-21T19:06:55.190 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.190 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.190 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.190 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.190 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.190 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.190 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.191 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.191 INFO:teuthology.orchestra.run.smithi039.stdout: "REST_REQUESTS_TIMEOUT": { 2023-07-21T19:06:55.191 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-07-21T19:06:55.191 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:55.191 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.191 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.191 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "45", 2023-07-21T19:06:55.191 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.192 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.192 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.192 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.192 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.192 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.192 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.192 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.193 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_ACCESS_KEY": { 2023-07-21T19:06:55.193 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_ACCESS_KEY", 2023-07-21T19:06:55.193 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.193 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.193 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.193 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.193 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.193 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.194 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.194 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.194 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.194 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.194 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.194 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.194 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-07-21T19:06:55.194 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-07-21T19:06:55.195 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.195 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.195 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.195 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "admin", 2023-07-21T19:06:55.195 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.195 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.195 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.195 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.196 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.196 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.196 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.196 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.196 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_SECRET_KEY": { 2023-07-21T19:06:55.196 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_SECRET_KEY", 2023-07-21T19:06:55.196 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.197 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.197 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.197 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.197 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.197 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.197 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.197 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.197 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.198 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.198 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.198 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.198 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_SSL_VERIFY": { 2023-07-21T19:06:55.198 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_SSL_VERIFY", 2023-07-21T19:06:55.198 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.198 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.198 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.199 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:55.199 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.199 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.199 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.199 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.199 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.199 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.199 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.200 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.200 INFO:teuthology.orchestra.run.smithi039.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-07-21T19:06:55.200 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-07-21T19:06:55.200 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:55.200 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.200 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.200 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2023-07-21T19:06:55.200 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.201 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.201 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.201 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.201 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.201 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.201 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.201 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.201 INFO:teuthology.orchestra.run.smithi039.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-07-21T19:06:55.202 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-07-21T19:06:55.202 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:55.202 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.202 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.202 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2023-07-21T19:06:55.202 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.202 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.203 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.203 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.203 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.203 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.203 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.203 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.203 INFO:teuthology.orchestra.run.smithi039.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-07-21T19:06:55.203 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-07-21T19:06:55.204 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:55.204 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.204 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.204 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2023-07-21T19:06:55.204 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.204 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.204 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.204 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.205 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.205 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.205 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.205 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.205 INFO:teuthology.orchestra.run.smithi039.stdout: "cross_origin_url": { 2023-07-21T19:06:55.205 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cross_origin_url", 2023-07-21T19:06:55.205 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.205 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.206 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.206 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.206 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.206 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.206 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.206 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.206 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.206 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.207 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.207 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.207 INFO:teuthology.orchestra.run.smithi039.stdout: "crt_file": { 2023-07-21T19:06:55.207 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crt_file", 2023-07-21T19:06:55.207 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.207 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.207 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.208 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.208 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.208 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.208 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.208 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.208 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.208 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.208 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.209 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.209 INFO:teuthology.orchestra.run.smithi039.stdout: "debug": { 2023-07-21T19:06:55.209 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "debug", 2023-07-21T19:06:55.209 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.209 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.209 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.209 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.209 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.210 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.210 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.210 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Enable/disable debug options", 2023-07-21T19:06:55.210 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.210 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.210 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.210 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.210 INFO:teuthology.orchestra.run.smithi039.stdout: "jwt_token_ttl": { 2023-07-21T19:06:55.211 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "jwt_token_ttl", 2023-07-21T19:06:55.211 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:55.211 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.211 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.211 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "28800", 2023-07-21T19:06:55.211 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.211 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.211 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.212 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.212 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.212 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.212 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.212 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.212 INFO:teuthology.orchestra.run.smithi039.stdout: "key_file": { 2023-07-21T19:06:55.212 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "key_file", 2023-07-21T19:06:55.213 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.213 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.213 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.213 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.213 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.213 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.213 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.213 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.214 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.214 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.214 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.214 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.214 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:55.214 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:55.214 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.214 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.215 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.215 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.215 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.215 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.215 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.215 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.215 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.215 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.216 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.216 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.216 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.216 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.216 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.216 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.216 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.217 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.217 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.217 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:55.217 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:55.217 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.217 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.217 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.218 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.218 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.218 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.218 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.218 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.218 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.218 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.218 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.219 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.219 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:55.219 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:55.219 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.219 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.219 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.219 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:55.219 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.220 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.220 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.220 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.220 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.220 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.220 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.220 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.220 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.221 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.221 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.221 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.221 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.221 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.221 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.221 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:55.221 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:55.222 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.222 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.222 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.222 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.222 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.222 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.222 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.222 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.223 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.223 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.223 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.223 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.223 INFO:teuthology.orchestra.run.smithi039.stdout: "motd": { 2023-07-21T19:06:55.223 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "motd", 2023-07-21T19:06:55.223 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.223 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.224 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.224 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.224 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.224 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.224 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.224 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "The message of the day", 2023-07-21T19:06:55.224 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.225 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.225 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.225 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.225 INFO:teuthology.orchestra.run.smithi039.stdout: "redirect_resolve_ip_addr": { 2023-07-21T19:06:55.225 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "redirect_resolve_ip_addr", 2023-07-21T19:06:55.225 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.225 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.225 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.226 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.226 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.226 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.226 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.226 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.226 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.226 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.226 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.227 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.227 INFO:teuthology.orchestra.run.smithi039.stdout: "server_addr": { 2023-07-21T19:06:55.227 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_addr", 2023-07-21T19:06:55.227 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.227 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.227 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.227 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "::", 2023-07-21T19:06:55.227 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.228 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.228 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.228 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.228 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.228 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.228 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.228 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.228 INFO:teuthology.orchestra.run.smithi039.stdout: "server_port": { 2023-07-21T19:06:55.229 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_port", 2023-07-21T19:06:55.229 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:55.229 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.229 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.229 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8080", 2023-07-21T19:06:55.229 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.229 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.229 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.230 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.230 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.230 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.230 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.230 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.230 INFO:teuthology.orchestra.run.smithi039.stdout: "ssl": { 2023-07-21T19:06:55.230 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssl", 2023-07-21T19:06:55.230 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.231 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.231 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.231 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:55.231 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.231 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.231 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.231 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.231 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.232 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.232 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.232 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.232 INFO:teuthology.orchestra.run.smithi039.stdout: "ssl_server_port": { 2023-07-21T19:06:55.232 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssl_server_port", 2023-07-21T19:06:55.232 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:55.232 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.232 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.233 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8443", 2023-07-21T19:06:55.233 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.233 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.233 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.233 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.233 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.233 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.233 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.234 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.234 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_behaviour": { 2023-07-21T19:06:55.234 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_behaviour", 2023-07-21T19:06:55.234 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.234 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.234 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.234 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "redirect", 2023-07-21T19:06:55.234 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.235 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.235 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.235 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.235 INFO:teuthology.orchestra.run.smithi039.stdout: "redirect" 2023-07-21T19:06:55.235 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.235 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.235 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.235 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.236 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.236 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.236 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_error_status_code": { 2023-07-21T19:06:55.236 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_error_status_code", 2023-07-21T19:06:55.236 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:55.236 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.236 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.237 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "500", 2023-07-21T19:06:55.237 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "400", 2023-07-21T19:06:55.237 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "599", 2023-07-21T19:06:55.237 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.237 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.237 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.237 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.237 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.238 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.238 INFO:teuthology.orchestra.run.smithi039.stdout: "url_prefix": { 2023-07-21T19:06:55.238 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "url_prefix", 2023-07-21T19:06:55.238 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.238 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.238 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.238 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.238 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.239 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.239 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.239 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.239 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.239 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.239 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.239 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.239 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.240 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.240 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:55.240 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "devicehealth", 2023-07-21T19:06:55.240 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:55.240 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:55.240 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:55.240 INFO:teuthology.orchestra.run.smithi039.stdout: "enable_monitoring": { 2023-07-21T19:06:55.240 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enable_monitoring", 2023-07-21T19:06:55.241 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.241 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.241 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.241 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:55.241 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.241 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.241 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.242 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "monitor device health metrics", 2023-07-21T19:06:55.242 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.242 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.242 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.242 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.242 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:55.242 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:55.242 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.243 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.243 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.243 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.243 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.243 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.243 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.243 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.243 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.244 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.244 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.244 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.244 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.244 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.244 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.244 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.245 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.245 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.245 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.245 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:55.245 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:55.246 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.246 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.246 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.246 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.246 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.246 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.246 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.246 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.247 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.247 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.247 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.247 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.247 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:55.247 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:55.247 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.247 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.248 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.248 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:55.248 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.248 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.248 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.248 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.248 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.248 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.249 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.249 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.249 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.249 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.249 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.249 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.249 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.249 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.250 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.250 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:55.250 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:55.250 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.250 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.250 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.250 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.251 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.251 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.251 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.251 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.251 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.251 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.251 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.251 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.252 INFO:teuthology.orchestra.run.smithi039.stdout: "mark_out_threshold": { 2023-07-21T19:06:55.252 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mark_out_threshold", 2023-07-21T19:06:55.252 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2023-07-21T19:06:55.252 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.252 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.252 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "2419200", 2023-07-21T19:06:55.252 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.252 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.253 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.253 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-07-21T19:06:55.253 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.253 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.253 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.253 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.253 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_name": { 2023-07-21T19:06:55.253 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pool_name", 2023-07-21T19:06:55.254 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.254 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.254 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.254 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "device_health_metrics", 2023-07-21T19:06:55.254 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.254 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.254 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.254 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "name of pool in which to store device health metrics", 2023-07-21T19:06:55.255 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.255 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.255 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.255 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.255 INFO:teuthology.orchestra.run.smithi039.stdout: "retention_period": { 2023-07-21T19:06:55.255 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "retention_period", 2023-07-21T19:06:55.255 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2023-07-21T19:06:55.255 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.256 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.256 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "15552000", 2023-07-21T19:06:55.256 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.256 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.256 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.256 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how long to retain device health metrics", 2023-07-21T19:06:55.256 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.256 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.257 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.257 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.257 INFO:teuthology.orchestra.run.smithi039.stdout: "scrape_frequency": { 2023-07-21T19:06:55.257 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "scrape_frequency", 2023-07-21T19:06:55.257 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2023-07-21T19:06:55.257 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.257 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.257 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "86400", 2023-07-21T19:06:55.258 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.258 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.258 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.258 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to scrape device health metrics", 2023-07-21T19:06:55.258 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.258 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.258 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.258 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.259 INFO:teuthology.orchestra.run.smithi039.stdout: "self_heal": { 2023-07-21T19:06:55.259 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "self_heal", 2023-07-21T19:06:55.259 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.259 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.259 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.259 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:55.259 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.259 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.260 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.260 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-07-21T19:06:55.260 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.260 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.260 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.260 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.260 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2023-07-21T19:06:55.261 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2023-07-21T19:06:55.261 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2023-07-21T19:06:55.261 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.261 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.261 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2023-07-21T19:06:55.261 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.261 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.261 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.262 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to wake up and check device health", 2023-07-21T19:06:55.262 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.262 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.262 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.262 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.262 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_threshold": { 2023-07-21T19:06:55.262 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_threshold", 2023-07-21T19:06:55.262 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2023-07-21T19:06:55.263 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.263 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.263 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "7257600", 2023-07-21T19:06:55.263 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.263 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.263 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.263 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-07-21T19:06:55.263 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.264 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.264 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.264 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.264 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.264 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.264 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:55.264 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "diskprediction_local", 2023-07-21T19:06:55.264 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:55.265 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:55.265 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:55.265 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:55.265 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:55.265 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.265 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.265 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.266 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.266 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.266 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.266 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.266 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.266 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.266 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.266 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.267 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.267 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.267 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.267 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.267 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.267 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.267 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.268 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.268 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:55.268 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:55.268 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.268 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.268 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.268 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.268 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.269 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.269 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.269 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.269 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.269 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.269 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.269 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.269 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:55.270 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:55.270 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.270 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.270 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.270 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:55.270 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.270 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.270 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.271 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.271 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.271 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.271 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.271 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.271 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.271 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.272 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.272 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.272 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.272 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.272 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.272 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:55.272 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:55.272 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.273 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.273 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.273 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.273 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.274 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.274 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.274 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.274 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.274 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.274 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.274 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.274 INFO:teuthology.orchestra.run.smithi039.stdout: "predict_interval": { 2023-07-21T19:06:55.275 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "predict_interval", 2023-07-21T19:06:55.275 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.275 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.275 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.275 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "86400", 2023-07-21T19:06:55.275 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.275 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.276 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.276 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.276 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.276 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.276 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.276 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.276 INFO:teuthology.orchestra.run.smithi039.stdout: "predictor_model": { 2023-07-21T19:06:55.276 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "predictor_model", 2023-07-21T19:06:55.277 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.277 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.277 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.277 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "prophetstor", 2023-07-21T19:06:55.277 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.277 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.277 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.277 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.278 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.278 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.278 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.278 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.278 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2023-07-21T19:06:55.278 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2023-07-21T19:06:55.278 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.279 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.279 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.279 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2023-07-21T19:06:55.279 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.279 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.279 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.279 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.279 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.280 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.280 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.280 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.280 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.280 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.280 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:55.280 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "influx", 2023-07-21T19:06:55.280 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": false, 2023-07-21T19:06:55.281 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "influxdb python module not found", 2023-07-21T19:06:55.281 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:55.281 INFO:teuthology.orchestra.run.smithi039.stdout: "batch_size": { 2023-07-21T19:06:55.281 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "batch_size", 2023-07-21T19:06:55.281 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:55.281 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.281 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.281 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5000", 2023-07-21T19:06:55.282 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.282 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.282 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.282 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-07-21T19:06:55.282 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.282 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.282 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.282 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.283 INFO:teuthology.orchestra.run.smithi039.stdout: "database": { 2023-07-21T19:06:55.283 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "database", 2023-07-21T19:06:55.283 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.283 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.283 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.283 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "ceph", 2023-07-21T19:06:55.283 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.284 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.284 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.284 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-07-21T19:06:55.284 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.284 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.284 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.284 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.284 INFO:teuthology.orchestra.run.smithi039.stdout: "hostname": { 2023-07-21T19:06:55.285 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "hostname", 2023-07-21T19:06:55.285 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.285 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.285 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.285 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.285 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.285 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.285 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.286 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "InfluxDB server hostname", 2023-07-21T19:06:55.286 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.286 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.286 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.286 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.286 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2023-07-21T19:06:55.286 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2023-07-21T19:06:55.287 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2023-07-21T19:06:55.287 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.287 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.287 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "30", 2023-07-21T19:06:55.287 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "5", 2023-07-21T19:06:55.287 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.287 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.287 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-07-21T19:06:55.288 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.288 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.288 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.288 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.288 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:55.288 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:55.288 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.288 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.289 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.289 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.289 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.289 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.289 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.289 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.289 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.290 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.290 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.290 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.290 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.290 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.290 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.290 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.291 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.291 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.291 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.291 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:55.291 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:55.291 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.291 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.291 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.292 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.292 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.292 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.292 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.292 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.292 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.292 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.292 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.293 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.293 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:55.293 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:55.293 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.293 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.293 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.293 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:55.293 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.294 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.294 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.294 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.294 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.294 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.294 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.294 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.294 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.295 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.295 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.295 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.295 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.295 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.295 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.295 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:55.295 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:55.296 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.296 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.296 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.296 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.296 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.296 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.296 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.297 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.297 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.297 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.297 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.297 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.297 INFO:teuthology.orchestra.run.smithi039.stdout: "password": { 2023-07-21T19:06:55.297 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "password", 2023-07-21T19:06:55.297 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.298 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.298 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.298 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.298 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.298 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.298 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.298 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "password of InfluxDB server user", 2023-07-21T19:06:55.298 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.299 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.299 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.299 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.299 INFO:teuthology.orchestra.run.smithi039.stdout: "port": { 2023-07-21T19:06:55.299 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "port", 2023-07-21T19:06:55.299 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:55.299 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.299 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.300 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8086", 2023-07-21T19:06:55.300 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.300 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.300 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.300 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "InfluxDB server port", 2023-07-21T19:06:55.300 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.300 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.300 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.301 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.301 INFO:teuthology.orchestra.run.smithi039.stdout: "ssl": { 2023-07-21T19:06:55.301 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssl", 2023-07-21T19:06:55.301 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.301 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.301 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.301 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "false", 2023-07-21T19:06:55.301 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.302 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.302 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.302 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-07-21T19:06:55.302 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.302 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.302 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.302 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.302 INFO:teuthology.orchestra.run.smithi039.stdout: "threads": { 2023-07-21T19:06:55.303 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "threads", 2023-07-21T19:06:55.303 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:55.303 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.303 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.303 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2023-07-21T19:06:55.303 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2023-07-21T19:06:55.303 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "32", 2023-07-21T19:06:55.303 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.304 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-07-21T19:06:55.304 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.304 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.304 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.304 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.304 INFO:teuthology.orchestra.run.smithi039.stdout: "username": { 2023-07-21T19:06:55.304 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "username", 2023-07-21T19:06:55.304 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.305 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.305 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.305 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.305 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.305 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.305 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.305 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "username of InfluxDB server user", 2023-07-21T19:06:55.305 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.306 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.306 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.306 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.306 INFO:teuthology.orchestra.run.smithi039.stdout: "verify_ssl": { 2023-07-21T19:06:55.306 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "verify_ssl", 2023-07-21T19:06:55.306 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.306 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.306 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.307 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "true", 2023-07-21T19:06:55.307 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.307 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.307 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.307 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-07-21T19:06:55.307 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.307 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.307 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.308 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.308 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.308 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.308 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:55.308 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "insights", 2023-07-21T19:06:55.308 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:55.308 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:55.309 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:55.309 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:55.309 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:55.309 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.309 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.309 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.309 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.309 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.310 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.310 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.310 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.310 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.310 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.310 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.310 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.310 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.311 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.311 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.311 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.311 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.311 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.311 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.311 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:55.311 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:55.312 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.312 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.312 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.312 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.312 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.312 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.312 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.312 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.313 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.313 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.313 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.313 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.313 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:55.313 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:55.313 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.314 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.314 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.314 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:55.314 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.314 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.314 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.314 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.315 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.315 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.315 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.315 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.315 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.315 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.315 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.315 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.316 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.316 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.316 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.316 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:55.316 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:55.316 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.316 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.316 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.317 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.317 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.317 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.317 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.317 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.317 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.317 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.317 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.318 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.318 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.318 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.318 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:55.318 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "iostat", 2023-07-21T19:06:55.318 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:55.318 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:55.318 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:55.319 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:55.319 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:55.319 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.319 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.319 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.319 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.319 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.319 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.320 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.320 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.320 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.320 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.320 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.320 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.320 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.320 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.321 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.321 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.321 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.321 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.321 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.321 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:55.321 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:55.321 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.322 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.322 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.322 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.322 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.322 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.322 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.322 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.322 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.323 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.323 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.323 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.323 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:55.323 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:55.323 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.323 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.323 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.324 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:55.324 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.324 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.324 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.324 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.324 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.324 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.325 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.325 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.325 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.325 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.325 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.325 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.325 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.325 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.326 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.326 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:55.326 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:55.326 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.326 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.326 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.326 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.327 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.327 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.327 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.327 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.327 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.327 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.327 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.327 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.328 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.328 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.328 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:55.328 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "k8sevents", 2023-07-21T19:06:55.328 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:55.328 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:55.328 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:55.328 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_event_retention_days": { 2023-07-21T19:06:55.329 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ceph_event_retention_days", 2023-07-21T19:06:55.329 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:55.329 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.329 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.329 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "7", 2023-07-21T19:06:55.329 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.329 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.329 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.330 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Days to hold ceph event information within local cache", 2023-07-21T19:06:55.330 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.330 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.330 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.330 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.330 INFO:teuthology.orchestra.run.smithi039.stdout: "config_check_secs": { 2023-07-21T19:06:55.330 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "config_check_secs", 2023-07-21T19:06:55.330 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:55.331 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.331 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.331 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2023-07-21T19:06:55.331 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "10", 2023-07-21T19:06:55.331 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.331 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.331 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-07-21T19:06:55.331 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.332 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.332 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.332 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.332 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:55.332 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:55.332 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.332 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.332 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.333 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.333 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.333 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.333 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.333 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.333 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.333 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.333 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.334 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.334 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.334 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.334 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.334 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.334 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.334 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.334 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.335 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:55.335 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:55.335 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.335 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.335 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.335 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.335 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.335 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.336 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.336 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.336 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.336 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.336 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.336 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.336 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:55.337 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:55.337 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.337 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.337 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.337 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:55.337 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.337 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.337 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.338 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.338 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.338 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.338 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.338 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.338 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.338 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.339 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.339 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.339 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.339 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.339 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.339 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:55.339 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:55.339 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.340 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.340 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.340 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.340 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.340 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.340 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.340 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.341 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.341 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.341 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.341 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.341 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.341 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.341 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:55.341 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "localpool", 2023-07-21T19:06:55.342 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:55.342 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:55.342 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:55.342 INFO:teuthology.orchestra.run.smithi039.stdout: "failure_domain": { 2023-07-21T19:06:55.342 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "failure_domain", 2023-07-21T19:06:55.342 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.342 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.342 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.343 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "host", 2023-07-21T19:06:55.343 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.343 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.343 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.343 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "failure domain for any created local pool", 2023-07-21T19:06:55.343 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-07-21T19:06:55.343 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.343 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.344 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.344 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:55.344 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:55.344 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.344 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.344 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.344 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.344 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.345 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.345 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.345 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.345 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.345 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.345 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.345 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.345 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.346 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.346 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.346 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.346 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.346 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.346 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.346 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:55.347 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:55.347 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.347 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.347 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.347 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.347 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.347 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.347 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.348 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.348 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.348 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.348 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.348 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.348 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:55.348 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:55.348 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.349 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.349 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.349 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:55.349 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.349 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.349 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.349 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.349 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.350 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.350 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.350 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.350 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.350 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.350 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.350 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.350 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.351 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.351 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.351 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:55.351 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:55.351 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.351 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.351 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.351 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.352 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.352 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.352 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.352 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.352 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.352 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.352 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.352 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.353 INFO:teuthology.orchestra.run.smithi039.stdout: "min_size": { 2023-07-21T19:06:55.353 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "min_size", 2023-07-21T19:06:55.353 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:55.353 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.353 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.353 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.353 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.353 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.354 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.354 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "default min_size for any created local pool", 2023-07-21T19:06:55.354 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-07-21T19:06:55.354 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.354 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.354 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.354 INFO:teuthology.orchestra.run.smithi039.stdout: "num_rep": { 2023-07-21T19:06:55.354 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "num_rep", 2023-07-21T19:06:55.355 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:55.355 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.355 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.355 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "3", 2023-07-21T19:06:55.355 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.355 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.355 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.355 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "default replica count for any created local pool", 2023-07-21T19:06:55.356 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.356 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.356 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.356 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.356 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num": { 2023-07-21T19:06:55.356 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pg_num", 2023-07-21T19:06:55.356 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:55.357 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.357 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.357 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "128", 2023-07-21T19:06:55.357 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.357 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.357 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.357 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "default pg_num for any created local pool", 2023-07-21T19:06:55.357 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.358 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.358 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.358 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.358 INFO:teuthology.orchestra.run.smithi039.stdout: "prefix": { 2023-07-21T19:06:55.358 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "prefix", 2023-07-21T19:06:55.358 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.358 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.358 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.359 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.359 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.359 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.359 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.359 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "name prefix for any created local pool", 2023-07-21T19:06:55.359 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.359 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.359 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.360 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.360 INFO:teuthology.orchestra.run.smithi039.stdout: "subtree": { 2023-07-21T19:06:55.360 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "subtree", 2023-07-21T19:06:55.360 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.360 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.360 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.360 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "rack", 2023-07-21T19:06:55.360 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.361 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.361 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.361 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "CRUSH level for which to create a local pool", 2023-07-21T19:06:55.361 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-07-21T19:06:55.361 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.361 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.361 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.361 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.362 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.362 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:55.362 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mds_autoscaler", 2023-07-21T19:06:55.362 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:55.362 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:55.362 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:55.362 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:55.363 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:55.363 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.363 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.363 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.363 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.363 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.363 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.363 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.364 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.364 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.364 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.364 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.364 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.364 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.364 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.365 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.365 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.365 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.365 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.365 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.365 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:55.365 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:55.365 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.366 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.366 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.366 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.366 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.366 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.366 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.366 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.366 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.367 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.367 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.367 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.367 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:55.367 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:55.367 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.367 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.367 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.368 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:55.368 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.368 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.368 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.368 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.368 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.368 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.369 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.369 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.369 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.369 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.369 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.369 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.369 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.369 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.370 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.370 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:55.370 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:55.370 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.370 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.370 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.370 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.370 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.371 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.371 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.371 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.371 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.371 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.371 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.371 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.371 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.372 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.372 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:55.372 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mirroring", 2023-07-21T19:06:55.372 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:55.372 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:55.372 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:55.372 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:55.372 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:55.373 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.373 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.373 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.373 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.373 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.373 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.373 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.373 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.374 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.374 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.374 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.374 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.374 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.374 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.374 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.374 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.375 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.375 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.375 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.375 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:55.375 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:55.375 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.375 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.375 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.376 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.376 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.376 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.376 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.376 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.376 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.376 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.376 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.377 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.377 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:55.377 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:55.377 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.377 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.377 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.377 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:55.377 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.378 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.378 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.378 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.378 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.378 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.378 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.378 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.378 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.379 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.379 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.379 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.379 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.379 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.379 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.379 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:55.379 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:55.380 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.380 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.380 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.380 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.380 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.380 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.380 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.380 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.381 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.381 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.381 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.381 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.381 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.381 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.381 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:55.381 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "nfs", 2023-07-21T19:06:55.382 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:55.382 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:55.382 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:55.382 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:55.382 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:55.382 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.382 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.382 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.383 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.383 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.383 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.383 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.383 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.383 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.383 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.383 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.384 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.384 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.384 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.384 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.384 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.384 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.384 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.384 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.385 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:55.385 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:55.385 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.385 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.385 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.385 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.385 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.386 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.386 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.386 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.386 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.386 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.386 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.386 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.386 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:55.386 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:55.387 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.387 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.387 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.387 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:55.387 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.387 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.387 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.388 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.388 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.388 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.388 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.388 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.388 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.388 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.389 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.389 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.389 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.389 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.389 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.389 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:55.389 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:55.389 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.390 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.390 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.390 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.390 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.390 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.390 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.390 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.390 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.391 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.391 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.391 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.391 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.391 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.391 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:55.391 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "orchestrator", 2023-07-21T19:06:55.391 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:55.392 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:55.392 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:55.392 INFO:teuthology.orchestra.run.smithi039.stdout: "fail_fs": { 2023-07-21T19:06:55.392 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "fail_fs", 2023-07-21T19:06:55.392 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.392 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.392 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.393 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.393 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.393 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.393 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.393 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-07-21T19:06:55.393 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.393 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.393 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.394 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.394 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:55.394 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:55.394 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.394 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.394 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.394 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.394 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.395 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.395 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.395 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.395 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.395 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.395 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.395 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.395 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.396 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.396 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.396 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.396 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.396 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.396 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.396 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:55.397 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:55.397 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.397 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.397 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.397 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.397 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.397 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.397 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.398 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.398 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.398 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.398 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.398 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.398 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:55.398 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:55.399 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.399 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.399 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.399 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:55.399 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.399 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.399 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.399 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.400 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.400 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.400 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.400 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.400 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.400 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.400 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.400 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.401 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.401 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.401 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.401 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:55.401 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:55.401 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.401 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.401 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.402 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.402 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.402 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.402 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.402 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.402 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.402 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.402 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.403 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.403 INFO:teuthology.orchestra.run.smithi039.stdout: "orchestrator": { 2023-07-21T19:06:55.403 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "orchestrator", 2023-07-21T19:06:55.403 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.403 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.403 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.403 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.404 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.404 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.404 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.404 INFO:teuthology.orchestra.run.smithi039.stdout: "cephadm", 2023-07-21T19:06:55.404 INFO:teuthology.orchestra.run.smithi039.stdout: "rook", 2023-07-21T19:06:55.404 INFO:teuthology.orchestra.run.smithi039.stdout: "test_orchestrator" 2023-07-21T19:06:55.404 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.404 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Orchestrator backend", 2023-07-21T19:06:55.405 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.405 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.405 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.405 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.405 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.405 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.405 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:55.405 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "osd_perf_query", 2023-07-21T19:06:55.406 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:55.406 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:55.406 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:55.406 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:55.406 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:55.406 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.406 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.406 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.407 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.407 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.407 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.407 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.407 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.407 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.407 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.407 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.408 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.408 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.408 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.408 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.408 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.408 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.408 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.409 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.409 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:55.409 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:55.409 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.409 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.409 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.409 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.409 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.410 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.410 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.410 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.410 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.410 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.410 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.410 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.410 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:55.411 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:55.411 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.411 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.411 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.411 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:55.411 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.411 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.411 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.412 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.412 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.412 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.412 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.412 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.412 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.412 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.412 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.413 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.413 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.413 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.413 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.413 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:55.413 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:55.413 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.414 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.414 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.414 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.414 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.414 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.414 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.414 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.414 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.415 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.415 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.415 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.415 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.415 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.415 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:55.415 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "osd_support", 2023-07-21T19:06:55.415 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:55.416 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:55.416 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:55.416 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:55.416 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:55.416 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.416 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.416 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.417 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.417 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.417 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.417 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.417 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.417 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.417 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.417 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.418 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.418 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.418 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.418 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.418 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.418 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.418 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.418 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.419 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:55.419 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:55.419 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.419 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.419 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.419 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.419 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.419 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.420 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.420 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.420 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.420 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.420 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.420 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.420 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:55.420 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:55.421 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.421 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.421 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.421 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:55.421 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.421 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.421 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.421 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.422 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.422 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.422 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.422 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.422 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.422 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.422 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.422 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.423 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.423 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.423 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.423 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:55.423 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:55.423 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.423 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.423 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.424 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.424 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.424 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.424 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.424 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.424 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.424 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.424 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.425 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.425 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.425 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.425 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:55.425 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pg_autoscaler", 2023-07-21T19:06:55.425 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:55.425 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:55.425 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:55.426 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:55.426 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:55.426 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.426 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.426 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.426 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.426 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.426 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.427 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.427 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.427 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.427 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.427 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.427 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.427 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.427 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.428 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.428 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.428 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.428 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.428 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.428 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:55.428 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:55.428 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.429 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.429 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.429 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.429 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.429 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.429 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.429 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.429 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.430 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.430 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.430 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.430 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:55.430 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:55.430 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.430 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.430 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.431 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:55.431 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.431 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.431 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.431 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.431 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.431 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.431 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.432 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.432 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.432 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.432 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.432 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.432 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.432 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.432 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.433 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:55.433 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:55.433 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.433 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.433 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.433 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.433 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.433 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.434 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.434 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.434 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.434 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.434 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.434 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.434 INFO:teuthology.orchestra.run.smithi039.stdout: "noautoscale": { 2023-07-21T19:06:55.434 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "noautoscale", 2023-07-21T19:06:55.435 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.435 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.435 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.435 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.435 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.435 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.435 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.435 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "global autoscale flag", 2023-07-21T19:06:55.436 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-07-21T19:06:55.436 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.436 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.436 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.436 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2023-07-21T19:06:55.436 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2023-07-21T19:06:55.436 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2023-07-21T19:06:55.437 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.437 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.437 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2023-07-21T19:06:55.437 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.437 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.437 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.437 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.438 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.438 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.438 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.438 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.438 INFO:teuthology.orchestra.run.smithi039.stdout: "threshold": { 2023-07-21T19:06:55.438 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "threshold", 2023-07-21T19:06:55.438 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2023-07-21T19:06:55.438 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.439 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.439 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "3.0", 2023-07-21T19:06:55.439 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1.0", 2023-07-21T19:06:55.439 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.439 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.439 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "scaling threshold", 2023-07-21T19:06:55.439 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-07-21T19:06:55.439 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.440 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.440 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.440 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.440 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.440 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:55.440 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "progress", 2023-07-21T19:06:55.440 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:55.441 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:55.441 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:55.441 INFO:teuthology.orchestra.run.smithi039.stdout: "allow_pg_recovery_event": { 2023-07-21T19:06:55.441 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "allow_pg_recovery_event", 2023-07-21T19:06:55.441 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.441 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.441 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.441 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.442 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.442 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.442 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.442 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "allow the module to show pg recovery progress", 2023-07-21T19:06:55.442 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.442 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.442 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.442 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.443 INFO:teuthology.orchestra.run.smithi039.stdout: "enabled": { 2023-07-21T19:06:55.443 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enabled", 2023-07-21T19:06:55.443 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.443 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.443 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.443 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:55.443 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.443 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.444 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.444 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.444 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.444 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.444 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.444 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.444 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:55.444 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:55.445 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.445 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.445 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.445 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.445 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.445 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.445 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.445 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.446 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.446 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.446 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.446 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.446 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.446 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.446 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.446 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.447 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.447 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.447 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.447 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:55.447 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:55.447 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.447 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.447 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.448 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.448 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.448 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.448 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.448 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.448 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.448 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.448 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.449 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.449 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:55.449 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:55.449 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.449 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.449 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.449 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:55.449 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.450 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.450 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.450 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.450 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.450 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.450 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.450 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.450 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.451 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.451 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.451 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.451 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.451 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.451 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.451 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:55.451 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:55.452 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.452 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.452 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.452 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.452 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.452 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.452 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.452 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.453 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.453 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.453 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.453 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.453 INFO:teuthology.orchestra.run.smithi039.stdout: "max_completed_events": { 2023-07-21T19:06:55.453 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_completed_events", 2023-07-21T19:06:55.453 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:55.453 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.454 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.454 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "50", 2023-07-21T19:06:55.454 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.454 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.454 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.454 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "number of past completed events to remember", 2023-07-21T19:06:55.454 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.454 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.455 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.455 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.455 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2023-07-21T19:06:55.455 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2023-07-21T19:06:55.455 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2023-07-21T19:06:55.455 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.455 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.455 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2023-07-21T19:06:55.456 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.456 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.456 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.456 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how long the module is going to sleep", 2023-07-21T19:06:55.456 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.456 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.456 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.457 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.457 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.457 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.457 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:55.457 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "prometheus", 2023-07-21T19:06:55.457 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:55.457 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:55.457 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:55.458 INFO:teuthology.orchestra.run.smithi039.stdout: "cache": { 2023-07-21T19:06:55.458 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cache", 2023-07-21T19:06:55.458 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.458 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.458 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.458 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:55.458 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.458 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.459 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.459 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.459 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.459 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.459 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.459 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.459 INFO:teuthology.orchestra.run.smithi039.stdout: "exclude_perf_counters": { 2023-07-21T19:06:55.459 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "exclude_perf_counters", 2023-07-21T19:06:55.460 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.460 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.460 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.460 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:55.460 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.460 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.460 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.460 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-07-21T19:06:55.461 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-07-21T19:06:55.461 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.461 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.461 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.461 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:55.461 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:55.461 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.462 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.462 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.462 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.462 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.462 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.462 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.462 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.462 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.463 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.463 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.463 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.463 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.463 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.463 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.463 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.464 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.464 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.464 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.464 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:55.464 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:55.464 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.464 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.464 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.465 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.465 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.465 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.465 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.465 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.465 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.465 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.465 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.466 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.466 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:55.466 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:55.466 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.466 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.466 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.466 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:55.466 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.467 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.467 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.467 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.467 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.467 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.467 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.467 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.467 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.468 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.468 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.468 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.468 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.468 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.468 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.468 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:55.468 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:55.469 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.469 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.469 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.469 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.469 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.469 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.469 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.469 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.470 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.470 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.470 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.470 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.470 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_stats_pools": { 2023-07-21T19:06:55.470 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rbd_stats_pools", 2023-07-21T19:06:55.470 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.470 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.471 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.471 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.471 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.471 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.471 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.471 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.471 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.471 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.472 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.472 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.472 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_stats_pools_refresh_interval": { 2023-07-21T19:06:55.472 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-07-21T19:06:55.472 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:55.472 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.472 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.473 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "300", 2023-07-21T19:06:55.473 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.473 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.473 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.473 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.473 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.473 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.473 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.474 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.474 INFO:teuthology.orchestra.run.smithi039.stdout: "scrape_interval": { 2023-07-21T19:06:55.474 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "scrape_interval", 2023-07-21T19:06:55.474 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2023-07-21T19:06:55.474 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.474 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.474 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "15.0", 2023-07-21T19:06:55.474 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.475 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.475 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.475 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.475 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.475 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.475 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.475 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.475 INFO:teuthology.orchestra.run.smithi039.stdout: "server_addr": { 2023-07-21T19:06:55.476 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_addr", 2023-07-21T19:06:55.476 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.476 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.476 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.476 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "::", 2023-07-21T19:06:55.476 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.476 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.476 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.477 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-07-21T19:06:55.477 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.477 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.477 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.477 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.477 INFO:teuthology.orchestra.run.smithi039.stdout: "server_port": { 2023-07-21T19:06:55.477 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_port", 2023-07-21T19:06:55.477 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:55.478 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.478 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.478 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "9283", 2023-07-21T19:06:55.480 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.480 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.480 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.480 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-07-21T19:06:55.481 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.481 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.481 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.481 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.481 INFO:teuthology.orchestra.run.smithi039.stdout: "stale_cache_strategy": { 2023-07-21T19:06:55.481 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "stale_cache_strategy", 2023-07-21T19:06:55.481 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.482 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.482 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.482 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "log", 2023-07-21T19:06:55.482 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.482 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.482 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.482 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.482 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.483 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.483 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.483 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.483 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_behaviour": { 2023-07-21T19:06:55.483 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_behaviour", 2023-07-21T19:06:55.483 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.483 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.483 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.484 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "default", 2023-07-21T19:06:55.484 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.484 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.484 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.484 INFO:teuthology.orchestra.run.smithi039.stdout: "default", 2023-07-21T19:06:55.484 INFO:teuthology.orchestra.run.smithi039.stdout: "error" 2023-07-21T19:06:55.484 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.484 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.485 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.485 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.485 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.485 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.485 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_error_status_code": { 2023-07-21T19:06:55.485 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_error_status_code", 2023-07-21T19:06:55.485 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:55.486 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.486 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.486 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "500", 2023-07-21T19:06:55.486 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "400", 2023-07-21T19:06:55.486 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "599", 2023-07-21T19:06:55.486 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.486 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.486 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.487 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.487 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.487 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.487 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.487 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.487 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:55.487 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rbd_support", 2023-07-21T19:06:55.488 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:55.488 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:55.488 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:55.488 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:55.488 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:55.488 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.488 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.488 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.489 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.489 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.489 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.489 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.489 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.489 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.489 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.490 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.490 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.490 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.490 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.490 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.490 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.490 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.490 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.491 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.491 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:55.491 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:55.491 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.491 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.491 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.491 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.491 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.492 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.492 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.492 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.492 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.492 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.492 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.492 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.492 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:55.493 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:55.493 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.493 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.493 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.493 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:55.493 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.493 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.493 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.494 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.494 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.494 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.494 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.494 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.494 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.494 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.495 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.495 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.495 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.495 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.495 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.495 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:55.495 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:55.495 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.496 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.496 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.496 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.496 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.496 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.496 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.496 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.496 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.497 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.497 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.497 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.497 INFO:teuthology.orchestra.run.smithi039.stdout: "max_concurrent_snap_create": { 2023-07-21T19:06:55.497 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_concurrent_snap_create", 2023-07-21T19:06:55.497 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:55.497 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.497 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.498 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2023-07-21T19:06:55.498 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.498 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.498 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.498 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.498 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.498 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.498 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.499 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.499 INFO:teuthology.orchestra.run.smithi039.stdout: "mirror_snapshot_schedule": { 2023-07-21T19:06:55.499 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mirror_snapshot_schedule", 2023-07-21T19:06:55.499 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.499 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.499 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.499 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.499 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.500 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.500 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.500 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.500 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.500 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.500 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.500 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.501 INFO:teuthology.orchestra.run.smithi039.stdout: "trash_purge_schedule": { 2023-07-21T19:06:55.501 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "trash_purge_schedule", 2023-07-21T19:06:55.501 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.501 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.501 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.501 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.501 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.501 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.502 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.502 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.502 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.502 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.502 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.502 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.502 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.502 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.503 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:55.503 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "restful", 2023-07-21T19:06:55.503 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:55.503 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:55.503 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:55.503 INFO:teuthology.orchestra.run.smithi039.stdout: "enable_auth": { 2023-07-21T19:06:55.503 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enable_auth", 2023-07-21T19:06:55.503 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.504 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.504 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.504 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:55.504 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.504 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.504 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.504 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.504 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.505 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.505 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.505 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.505 INFO:teuthology.orchestra.run.smithi039.stdout: "key_file": { 2023-07-21T19:06:55.505 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "key_file", 2023-07-21T19:06:55.505 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.505 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.505 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.506 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.506 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.506 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.506 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.506 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.506 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.506 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.506 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.507 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.507 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:55.507 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:55.507 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.507 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.507 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.507 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.507 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.508 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.508 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.508 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.508 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.508 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.508 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.508 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.508 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.509 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.509 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.509 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.509 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.509 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.509 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.509 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:55.509 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:55.510 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.510 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.510 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.510 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.510 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.510 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.510 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.510 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.511 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.511 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.511 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.511 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.511 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:55.511 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:55.511 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.511 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.512 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.512 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:55.512 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.512 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.512 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.513 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.513 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.513 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.513 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.513 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.513 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.513 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.513 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.514 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.514 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.514 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.514 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.514 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:55.514 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:55.514 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.515 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.515 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.515 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.515 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.515 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.515 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.515 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.515 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.516 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.516 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.516 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.516 INFO:teuthology.orchestra.run.smithi039.stdout: "server_addr": { 2023-07-21T19:06:55.516 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_addr", 2023-07-21T19:06:55.516 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.516 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.516 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.517 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.517 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.517 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.517 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.517 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.517 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.517 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.517 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.518 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.518 INFO:teuthology.orchestra.run.smithi039.stdout: "server_port": { 2023-07-21T19:06:55.518 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_port", 2023-07-21T19:06:55.518 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.518 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.518 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.518 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.518 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.519 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.519 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.519 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.519 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.519 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.519 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.519 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.519 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.520 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.520 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:55.520 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rgw", 2023-07-21T19:06:55.520 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:55.520 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:55.520 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:55.520 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:55.520 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:55.521 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.521 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.521 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.521 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.521 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.521 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.521 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.521 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.522 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.522 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.522 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.522 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.522 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.522 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.522 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.522 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.523 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.523 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.523 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.523 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:55.523 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:55.523 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.523 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.524 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.524 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.524 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.524 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.524 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.524 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.524 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.524 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.525 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.525 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.525 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:55.525 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:55.525 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.525 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.525 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.525 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:55.526 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.526 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.526 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.526 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.526 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.526 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.526 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.526 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.527 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.527 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.527 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.527 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.527 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.527 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.527 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.527 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:55.528 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:55.528 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.528 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.528 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.528 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.528 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.528 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.529 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.529 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.529 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.529 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.529 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.529 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.529 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.529 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.530 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:55.530 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rook", 2023-07-21T19:06:55.530 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:55.530 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:55.530 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:55.530 INFO:teuthology.orchestra.run.smithi039.stdout: "drive_group_interval": { 2023-07-21T19:06:55.530 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "drive_group_interval", 2023-07-21T19:06:55.530 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2023-07-21T19:06:55.531 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.531 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.531 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "300.0", 2023-07-21T19:06:55.531 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.531 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.531 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.531 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-07-21T19:06:55.531 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.532 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.532 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.532 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.532 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:55.532 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:55.532 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.532 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.532 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.533 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.533 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.533 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.533 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.533 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.533 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.533 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.534 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.534 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.534 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.534 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.534 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.534 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.534 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.534 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.535 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.535 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:55.535 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:55.535 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.535 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.535 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.535 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.535 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.536 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.536 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.536 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.536 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.536 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.536 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.536 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.536 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:55.537 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:55.537 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.537 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.537 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.537 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:55.537 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.537 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.538 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.538 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.538 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.538 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.538 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.538 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.538 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.538 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.539 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.539 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.539 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.539 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.539 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.539 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:55.539 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:55.539 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.540 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.540 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.540 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.540 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.540 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.540 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.540 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.541 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.541 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.541 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.541 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.541 INFO:teuthology.orchestra.run.smithi039.stdout: "storage_class": { 2023-07-21T19:06:55.541 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "storage_class", 2023-07-21T19:06:55.541 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.541 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.542 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.542 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "local", 2023-07-21T19:06:55.542 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.542 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.542 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.542 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-07-21T19:06:55.542 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.542 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.543 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.543 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.543 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.543 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.543 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:55.543 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "selftest", 2023-07-21T19:06:55.543 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:55.543 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:55.544 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:55.544 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:55.544 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:55.544 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.544 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.544 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.544 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.544 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.545 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.545 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.545 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.545 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.545 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.546 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.546 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.546 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.546 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.546 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.546 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.546 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.547 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.547 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.547 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:55.547 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:55.547 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.547 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.547 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.547 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.548 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.548 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.548 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.548 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.548 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.548 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.548 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.548 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.549 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:55.549 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:55.549 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.549 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.549 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.549 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:55.549 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.549 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.550 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.550 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.550 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.550 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.550 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.550 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.550 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.550 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.551 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.551 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.551 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.551 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.551 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.551 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:55.551 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:55.551 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.552 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.552 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.552 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.552 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.552 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.552 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.552 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.553 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.553 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.553 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.553 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.553 INFO:teuthology.orchestra.run.smithi039.stdout: "roption1": { 2023-07-21T19:06:55.553 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "roption1", 2023-07-21T19:06:55.553 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.553 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.554 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.554 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.554 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.554 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.554 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.554 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.554 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.554 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.555 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.555 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.555 INFO:teuthology.orchestra.run.smithi039.stdout: "roption2": { 2023-07-21T19:06:55.555 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "roption2", 2023-07-21T19:06:55.555 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.555 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.555 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.555 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "xyz", 2023-07-21T19:06:55.556 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.556 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.556 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.556 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.556 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.556 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.556 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.556 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.557 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption1": { 2023-07-21T19:06:55.557 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption1", 2023-07-21T19:06:55.557 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.557 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.557 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.557 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.557 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.557 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.558 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.558 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.558 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.558 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.558 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.558 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.558 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption2": { 2023-07-21T19:06:55.558 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption2", 2023-07-21T19:06:55.559 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:55.559 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.559 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.559 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.559 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.560 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.560 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.560 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.560 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.560 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.560 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.560 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.561 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption3": { 2023-07-21T19:06:55.561 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption3", 2023-07-21T19:06:55.561 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2023-07-21T19:06:55.561 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.561 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.561 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.561 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.561 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.562 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.562 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.562 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.562 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.562 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.562 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.562 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption4": { 2023-07-21T19:06:55.562 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption4", 2023-07-21T19:06:55.563 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.563 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.563 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.563 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.563 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.563 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.563 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.564 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.564 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.564 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.564 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.564 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.564 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption5": { 2023-07-21T19:06:55.564 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption5", 2023-07-21T19:06:55.565 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.565 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.565 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.565 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.565 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.565 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.565 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.565 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.566 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.566 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.566 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.566 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.566 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption6": { 2023-07-21T19:06:55.566 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption6", 2023-07-21T19:06:55.566 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.566 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.567 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.567 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:55.567 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.567 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.567 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.567 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.567 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.568 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.568 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.568 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.568 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption7": { 2023-07-21T19:06:55.568 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption7", 2023-07-21T19:06:55.568 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:55.568 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.568 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.569 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.569 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2023-07-21T19:06:55.569 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "42", 2023-07-21T19:06:55.569 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.569 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.569 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.569 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.569 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.570 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.570 INFO:teuthology.orchestra.run.smithi039.stdout: "testkey": { 2023-07-21T19:06:55.570 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "testkey", 2023-07-21T19:06:55.570 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.570 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.570 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.570 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.571 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.571 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.571 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.571 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.571 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.571 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.571 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.571 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.572 INFO:teuthology.orchestra.run.smithi039.stdout: "testlkey": { 2023-07-21T19:06:55.572 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "testlkey", 2023-07-21T19:06:55.572 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.572 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.572 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.572 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.572 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.572 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.573 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.573 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.573 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.573 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.573 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.573 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.573 INFO:teuthology.orchestra.run.smithi039.stdout: "testnewline": { 2023-07-21T19:06:55.573 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "testnewline", 2023-07-21T19:06:55.574 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.574 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.574 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.574 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.574 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.574 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.574 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.574 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.575 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.575 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.575 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.575 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.575 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.575 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.575 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:55.575 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "snap_schedule", 2023-07-21T19:06:55.576 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:55.576 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:55.576 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:55.576 INFO:teuthology.orchestra.run.smithi039.stdout: "allow_m_granularity": { 2023-07-21T19:06:55.576 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "allow_m_granularity", 2023-07-21T19:06:55.576 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.576 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.577 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.577 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.577 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.577 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.577 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.577 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "allow minute scheduled snapshots", 2023-07-21T19:06:55.577 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.577 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.578 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.578 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.578 INFO:teuthology.orchestra.run.smithi039.stdout: "dump_on_update": { 2023-07-21T19:06:55.578 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "dump_on_update", 2023-07-21T19:06:55.578 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.578 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.578 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.578 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.579 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.579 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.579 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.579 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "dump database to debug log on update", 2023-07-21T19:06:55.579 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.579 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.579 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.579 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.580 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:55.580 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:55.580 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.580 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.580 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.580 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.580 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.580 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.581 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.581 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.581 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.581 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.581 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.581 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.581 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.581 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.582 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.582 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.582 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.582 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.582 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.582 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:55.582 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:55.582 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.583 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.583 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.583 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.583 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.583 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.583 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.583 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.583 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.584 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.584 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.584 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.584 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:55.584 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:55.584 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.584 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.585 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.585 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:55.585 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.585 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.585 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.585 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.585 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.585 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.586 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.586 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.586 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.586 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.586 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.586 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.586 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.586 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.587 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.587 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:55.587 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:55.587 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.587 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.587 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.587 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.587 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.588 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.588 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.588 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.588 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.588 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.588 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.589 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.589 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.589 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.589 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:55.589 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "stats", 2023-07-21T19:06:55.589 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:55.589 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:55.589 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:55.590 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:55.590 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:55.590 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.590 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.590 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.590 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.590 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.590 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.591 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.591 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.591 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.591 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.591 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.591 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.591 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.591 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.592 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.592 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.592 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.592 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.592 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.592 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:55.592 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:55.593 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.593 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.593 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.593 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.593 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.593 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.593 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.593 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.594 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.594 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.594 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.594 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.594 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:55.594 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:55.594 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.594 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.595 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.595 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:55.595 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.595 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.595 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.595 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.595 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.595 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.596 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.596 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.596 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.596 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.596 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.596 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.596 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.597 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.597 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.597 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:55.597 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:55.597 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.597 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.597 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.597 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.598 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.598 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.598 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.598 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.598 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.598 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.598 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.598 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.599 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.599 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.599 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:55.599 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "status", 2023-07-21T19:06:55.599 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:55.599 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:55.599 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:55.599 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:55.600 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:55.600 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.600 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.600 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.600 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.600 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.600 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.600 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.601 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.601 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.601 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.601 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.601 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.601 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.601 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.601 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.602 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.602 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.602 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.602 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.602 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:55.602 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:55.602 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.602 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.603 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.603 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.603 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.603 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.603 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.603 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.603 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.604 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.604 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.604 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.604 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:55.604 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:55.604 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.604 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.604 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.605 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:55.605 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.605 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.605 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.605 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.605 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.605 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.605 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.606 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.606 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.606 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.606 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.606 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.606 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.606 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.606 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.607 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:55.607 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:55.607 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.607 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.607 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.607 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.607 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.607 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.608 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.608 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.608 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.608 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.608 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.608 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.608 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.608 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.609 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:55.609 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "telegraf", 2023-07-21T19:06:55.609 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:55.609 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:55.609 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:55.609 INFO:teuthology.orchestra.run.smithi039.stdout: "address": { 2023-07-21T19:06:55.609 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "address", 2023-07-21T19:06:55.609 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.610 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.610 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.610 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-07-21T19:06:55.610 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.610 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.610 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.610 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.610 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.611 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.611 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.611 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.611 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2023-07-21T19:06:55.611 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2023-07-21T19:06:55.611 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2023-07-21T19:06:55.611 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.612 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.612 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "15", 2023-07-21T19:06:55.612 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.612 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.612 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.612 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.612 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.612 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.613 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.613 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.613 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:55.613 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:55.613 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.613 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.613 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.614 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.614 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.614 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.614 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.614 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.614 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.614 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.615 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.615 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.615 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.615 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.615 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.615 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.615 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.615 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.616 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.616 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:55.616 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:55.616 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.616 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.616 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.616 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.616 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.617 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.617 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.617 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.617 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.617 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.617 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.617 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.618 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:55.618 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:55.618 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.618 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.618 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.618 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:55.618 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.618 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.619 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.619 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.619 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.619 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.619 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.619 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.619 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.619 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.620 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.620 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.620 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.620 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.620 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.620 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:55.620 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:55.620 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.621 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.621 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.621 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.621 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.621 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.621 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.621 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.621 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.622 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.622 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.622 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.622 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.622 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.622 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:55.622 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "telemetry", 2023-07-21T19:06:55.622 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:55.623 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:55.623 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:55.623 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_basic": { 2023-07-21T19:06:55.623 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_basic", 2023-07-21T19:06:55.623 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.623 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.623 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.623 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:55.624 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.624 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.624 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.624 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share basic cluster information (size, version)", 2023-07-21T19:06:55.624 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.624 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.624 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.624 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.625 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_crash": { 2023-07-21T19:06:55.625 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_crash", 2023-07-21T19:06:55.625 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.625 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.625 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.625 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:55.625 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.625 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.626 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.626 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-07-21T19:06:55.626 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.626 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.626 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.626 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.626 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_device": { 2023-07-21T19:06:55.627 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_device", 2023-07-21T19:06:55.627 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.627 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.627 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.627 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:55.627 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.627 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.627 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.628 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-07-21T19:06:55.628 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.628 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.628 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.628 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.628 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_ident": { 2023-07-21T19:06:55.628 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_ident", 2023-07-21T19:06:55.628 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.629 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.629 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.629 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.629 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.629 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.629 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.629 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-07-21T19:06:55.629 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.630 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.630 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.630 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.630 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_perf": { 2023-07-21T19:06:55.630 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_perf", 2023-07-21T19:06:55.630 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.630 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.630 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.631 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.631 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.631 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.631 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.631 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share various performance metrics of a cluster", 2023-07-21T19:06:55.631 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.631 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.631 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.632 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.632 INFO:teuthology.orchestra.run.smithi039.stdout: "contact": { 2023-07-21T19:06:55.632 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "contact", 2023-07-21T19:06:55.632 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.632 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.632 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.632 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.633 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.633 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.633 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.633 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.633 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.633 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.633 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.633 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.634 INFO:teuthology.orchestra.run.smithi039.stdout: "description": { 2023-07-21T19:06:55.634 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "description", 2023-07-21T19:06:55.634 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.634 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.634 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.634 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.634 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.634 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.635 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.635 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.635 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.635 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.635 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.635 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.635 INFO:teuthology.orchestra.run.smithi039.stdout: "device_url": { 2023-07-21T19:06:55.635 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "device_url", 2023-07-21T19:06:55.636 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.636 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.636 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.636 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-07-21T19:06:55.636 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.636 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.636 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.636 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.637 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.637 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.637 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.637 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.637 INFO:teuthology.orchestra.run.smithi039.stdout: "enabled": { 2023-07-21T19:06:55.637 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enabled", 2023-07-21T19:06:55.637 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.638 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.638 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.638 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.638 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.638 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.638 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.638 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.638 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.639 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.639 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.640 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.640 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2023-07-21T19:06:55.640 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2023-07-21T19:06:55.640 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:55.640 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.640 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.640 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "24", 2023-07-21T19:06:55.640 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "8", 2023-07-21T19:06:55.641 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.641 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.641 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.641 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.641 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.641 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.641 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.641 INFO:teuthology.orchestra.run.smithi039.stdout: "last_opt_revision": { 2023-07-21T19:06:55.642 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "last_opt_revision", 2023-07-21T19:06:55.642 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:55.642 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.642 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.642 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "1", 2023-07-21T19:06:55.642 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.642 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.643 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.643 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.643 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.643 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.643 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.643 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.643 INFO:teuthology.orchestra.run.smithi039.stdout: "leaderboard": { 2023-07-21T19:06:55.643 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "leaderboard", 2023-07-21T19:06:55.644 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.644 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.644 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.644 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.644 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.644 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.644 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.644 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.645 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.645 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.645 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.645 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.645 INFO:teuthology.orchestra.run.smithi039.stdout: "leaderboard_description": { 2023-07-21T19:06:55.645 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "leaderboard_description", 2023-07-21T19:06:55.645 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.645 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.646 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.646 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.646 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.646 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.646 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.646 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.646 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.646 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.647 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.647 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.647 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:55.647 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:55.647 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.647 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.647 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.647 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.648 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.648 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.648 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.648 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.648 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.648 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.648 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.649 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.649 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.649 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.649 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.649 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.649 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.649 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.649 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.650 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:55.650 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:55.650 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.650 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.650 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.650 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.650 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.650 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.651 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.651 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.651 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.651 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.651 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.651 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.651 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:55.651 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:55.652 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.652 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.652 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.652 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:55.652 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.652 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.652 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.652 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.653 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.653 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.653 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.653 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.653 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.653 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.654 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.654 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.654 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.654 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.654 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.654 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:55.654 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:55.655 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.655 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.655 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.655 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.655 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.655 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.655 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.656 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.656 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.656 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.656 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.656 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.656 INFO:teuthology.orchestra.run.smithi039.stdout: "organization": { 2023-07-21T19:06:55.656 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "organization", 2023-07-21T19:06:55.657 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.657 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.657 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.657 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.657 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.657 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.657 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.657 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.658 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.658 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.658 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.658 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.658 INFO:teuthology.orchestra.run.smithi039.stdout: "proxy": { 2023-07-21T19:06:55.658 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "proxy", 2023-07-21T19:06:55.659 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.659 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.659 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.659 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.659 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.659 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.659 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.660 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.660 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.660 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.660 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.660 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.660 INFO:teuthology.orchestra.run.smithi039.stdout: "url": { 2023-07-21T19:06:55.660 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "url", 2023-07-21T19:06:55.661 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.661 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.661 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.661 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-07-21T19:06:55.661 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.661 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.661 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.662 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.662 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.662 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.662 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.662 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.662 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.662 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.662 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:55.663 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "test_orchestrator", 2023-07-21T19:06:55.663 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:55.663 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:55.663 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:55.663 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:55.663 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:55.663 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.663 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.664 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.664 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.664 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.664 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.664 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.664 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.665 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.665 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.665 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.665 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.665 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.665 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.665 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.666 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.666 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.666 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.666 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.666 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:55.666 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:55.666 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.666 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.667 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.667 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.667 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.667 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.667 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.667 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.667 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.667 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.668 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.668 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.668 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:55.668 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:55.668 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.668 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.668 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.669 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:55.669 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.669 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.669 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.669 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.669 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.669 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.669 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.670 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.670 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.670 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.670 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.670 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.670 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.670 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.671 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.671 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:55.671 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:55.671 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.671 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.671 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.671 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.671 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.672 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.672 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.672 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.672 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.672 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.672 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.672 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.672 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.673 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.673 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:55.673 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "volumes", 2023-07-21T19:06:55.673 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:55.673 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:55.673 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:55.673 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:55.674 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:55.674 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.674 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.674 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.674 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.674 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.674 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.674 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.675 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.675 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.675 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.675 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.675 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.675 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.675 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.675 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.676 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.676 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.676 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.676 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.676 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:55.676 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:55.676 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.676 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.677 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.677 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.677 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.677 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.677 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.677 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.677 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.678 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.678 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.678 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.678 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:55.678 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:55.678 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.678 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.678 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.679 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:55.679 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.679 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.679 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.679 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.679 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.679 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.679 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.680 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.680 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.680 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.680 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.680 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.680 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.680 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.681 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.681 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:55.681 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:55.681 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.681 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.681 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.681 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.681 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.682 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.682 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.682 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.682 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.682 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.682 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.682 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.682 INFO:teuthology.orchestra.run.smithi039.stdout: "max_concurrent_clones": { 2023-07-21T19:06:55.683 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_concurrent_clones", 2023-07-21T19:06:55.683 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:55.683 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.683 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.683 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "4", 2023-07-21T19:06:55.683 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.683 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.683 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.684 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Number of asynchronous cloner threads", 2023-07-21T19:06:55.684 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.684 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.684 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.684 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.684 INFO:teuthology.orchestra.run.smithi039.stdout: "snapshot_clone_delay": { 2023-07-21T19:06:55.684 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "snapshot_clone_delay", 2023-07-21T19:06:55.685 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:55.685 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.685 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.685 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2023-07-21T19:06:55.685 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.685 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.685 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.685 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-07-21T19:06:55.686 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.686 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.686 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.686 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.686 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.686 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.686 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:55.686 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix", 2023-07-21T19:06:55.687 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:55.687 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:55.687 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:55.687 INFO:teuthology.orchestra.run.smithi039.stdout: "discovery_interval": { 2023-07-21T19:06:55.687 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "discovery_interval", 2023-07-21T19:06:55.687 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2023-07-21T19:06:55.687 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.687 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.688 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "100", 2023-07-21T19:06:55.688 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.688 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.688 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.688 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.688 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.688 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.689 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.689 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.689 INFO:teuthology.orchestra.run.smithi039.stdout: "identifier": { 2023-07-21T19:06:55.689 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "identifier", 2023-07-21T19:06:55.689 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.689 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.689 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.690 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.690 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.690 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.690 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.690 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.690 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.690 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.690 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.691 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.691 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2023-07-21T19:06:55.691 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2023-07-21T19:06:55.691 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2023-07-21T19:06:55.691 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.691 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.691 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2023-07-21T19:06:55.691 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.692 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.692 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.692 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.692 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.692 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.692 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.692 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.693 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:55.693 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:55.693 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.693 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.693 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.693 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.693 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.693 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.694 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.694 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.694 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.694 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.694 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.694 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.694 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.695 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.695 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.695 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.695 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.695 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.695 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.695 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:55.695 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:55.696 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.696 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.696 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.696 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.696 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.696 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.696 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.696 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.697 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.697 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.697 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.697 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.697 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:55.697 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:55.697 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.698 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.698 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.698 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:55.698 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.698 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.698 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.698 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.698 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.699 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.699 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.699 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.699 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.699 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.699 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.699 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.699 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.700 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.700 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.700 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:55.700 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:55.700 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.700 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.700 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.700 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.701 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.701 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.701 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.701 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.701 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.701 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.701 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.701 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.702 INFO:teuthology.orchestra.run.smithi039.stdout: "zabbix_host": { 2023-07-21T19:06:55.702 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix_host", 2023-07-21T19:06:55.702 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.702 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.702 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.702 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.702 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.703 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.703 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.703 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.703 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.703 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.703 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.703 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.703 INFO:teuthology.orchestra.run.smithi039.stdout: "zabbix_port": { 2023-07-21T19:06:55.704 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix_port", 2023-07-21T19:06:55.704 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:55.704 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.704 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.704 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10051", 2023-07-21T19:06:55.704 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.704 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.704 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.705 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.705 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.705 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.705 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.705 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.705 INFO:teuthology.orchestra.run.smithi039.stdout: "zabbix_sender": { 2023-07-21T19:06:55.705 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix_sender", 2023-07-21T19:06:55.705 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.706 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.706 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.706 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-07-21T19:06:55.706 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.706 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.706 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.706 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.707 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.707 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.707 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.707 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.707 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.707 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.707 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2023-07-21T19:06:55.707 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.708 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:55.708 INFO:teuthology.orchestra.run.smithi039.stdout: "gid": 25093, 2023-07-21T19:06:55.708 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "z", 2023-07-21T19:06:55.708 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr_features": 4540138322906710015, 2023-07-21T19:06:55.708 INFO:teuthology.orchestra.run.smithi039.stdout: "available_modules": [ 2023-07-21T19:06:55.708 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:55.708 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "alerts", 2023-07-21T19:06:55.709 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:55.709 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:55.709 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:55.709 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2023-07-21T19:06:55.709 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2023-07-21T19:06:55.709 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2023-07-21T19:06:55.709 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.709 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.710 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2023-07-21T19:06:55.710 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.710 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.710 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.710 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How frequently to reexamine health status", 2023-07-21T19:06:55.710 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.710 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.710 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.711 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.711 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:55.711 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:55.711 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.711 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.711 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.711 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.711 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.712 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.712 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.712 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.712 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.712 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.712 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.712 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.713 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.713 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.713 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.713 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.713 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.713 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.713 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.713 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:55.714 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:55.714 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.714 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.714 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.714 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.714 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.714 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.714 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.715 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.715 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.715 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.715 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.715 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.715 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:55.716 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:55.716 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.716 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.716 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.716 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:55.716 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.716 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.716 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.717 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.717 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.717 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.717 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.717 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.717 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.717 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.718 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.718 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.718 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.718 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.718 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.718 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:55.718 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:55.718 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.719 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.719 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.719 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.719 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.719 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.719 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.719 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.719 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.720 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.720 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.720 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.720 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_destination": { 2023-07-21T19:06:55.720 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_destination", 2023-07-21T19:06:55.720 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.720 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.721 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.721 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.721 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.721 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.721 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.721 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Email address to send alerts to", 2023-07-21T19:06:55.721 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.721 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.722 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.722 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.722 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_from_name": { 2023-07-21T19:06:55.722 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_from_name", 2023-07-21T19:06:55.722 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.722 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.722 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.723 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "Ceph", 2023-07-21T19:06:55.723 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.723 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.723 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.723 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Email From: name", 2023-07-21T19:06:55.723 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.723 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.723 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.724 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.724 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_host": { 2023-07-21T19:06:55.724 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_host", 2023-07-21T19:06:55.724 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.724 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.724 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.724 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.724 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.725 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.725 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.725 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SMTP server", 2023-07-21T19:06:55.725 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.725 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.725 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.725 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.726 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_password": { 2023-07-21T19:06:55.726 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_password", 2023-07-21T19:06:55.726 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.726 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.726 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.726 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.726 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.726 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.727 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.727 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Password to authenticate with", 2023-07-21T19:06:55.727 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.727 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.727 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.727 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.727 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_port": { 2023-07-21T19:06:55.728 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_port", 2023-07-21T19:06:55.728 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:55.728 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.728 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.728 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "465", 2023-07-21T19:06:55.728 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.728 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.728 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.729 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SMTP port", 2023-07-21T19:06:55.729 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.729 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.729 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.729 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.729 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_sender": { 2023-07-21T19:06:55.729 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_sender", 2023-07-21T19:06:55.730 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.730 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.730 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.730 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.730 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.730 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.730 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.730 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SMTP envelope sender", 2023-07-21T19:06:55.731 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.731 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.731 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.731 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.731 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_ssl": { 2023-07-21T19:06:55.731 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_ssl", 2023-07-21T19:06:55.731 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.731 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.732 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.732 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:55.732 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.732 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.732 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.732 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use SSL to connect to SMTP server", 2023-07-21T19:06:55.732 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.733 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.733 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.733 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.733 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_user": { 2023-07-21T19:06:55.733 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_user", 2023-07-21T19:06:55.733 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.733 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.733 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.734 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.734 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.734 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.734 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.734 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "User to authenticate as", 2023-07-21T19:06:55.734 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.734 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.734 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.735 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.735 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.735 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.735 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:55.735 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "balancer", 2023-07-21T19:06:55.735 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:55.735 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:55.736 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:55.736 INFO:teuthology.orchestra.run.smithi039.stdout: "active": { 2023-07-21T19:06:55.736 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "active", 2023-07-21T19:06:55.736 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.736 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.736 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.736 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:55.736 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.737 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.737 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.737 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "automatically balance PGs across cluster", 2023-07-21T19:06:55.737 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.737 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.737 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.737 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.737 INFO:teuthology.orchestra.run.smithi039.stdout: "begin_time": { 2023-07-21T19:06:55.738 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "begin_time", 2023-07-21T19:06:55.738 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.738 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.738 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.738 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0000", 2023-07-21T19:06:55.738 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.738 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.738 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.739 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "beginning time of day to automatically balance", 2023-07-21T19:06:55.739 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-21T19:06:55.739 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.739 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.739 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.739 INFO:teuthology.orchestra.run.smithi039.stdout: "begin_weekday": { 2023-07-21T19:06:55.739 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "begin_weekday", 2023-07-21T19:06:55.740 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2023-07-21T19:06:55.740 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.740 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.740 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2023-07-21T19:06:55.740 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "0", 2023-07-21T19:06:55.740 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "6", 2023-07-21T19:06:55.740 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.741 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-07-21T19:06:55.741 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-21T19:06:55.741 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.741 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.741 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.741 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_compat_max_iterations": { 2023-07-21T19:06:55.741 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crush_compat_max_iterations", 2023-07-21T19:06:55.742 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2023-07-21T19:06:55.742 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.742 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.742 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "25", 2023-07-21T19:06:55.742 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2023-07-21T19:06:55.742 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "250", 2023-07-21T19:06:55.742 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.742 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-07-21T19:06:55.743 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.743 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.743 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.743 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.743 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_compat_metrics": { 2023-07-21T19:06:55.743 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crush_compat_metrics", 2023-07-21T19:06:55.743 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.743 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.744 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.744 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "pgs,objects,bytes", 2023-07-21T19:06:55.744 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.744 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.744 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.744 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-07-21T19:06:55.744 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-07-21T19:06:55.745 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.745 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.745 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.745 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_compat_step": { 2023-07-21T19:06:55.745 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crush_compat_step", 2023-07-21T19:06:55.745 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2023-07-21T19:06:55.745 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.745 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.746 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0.5", 2023-07-21T19:06:55.746 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "0.001", 2023-07-21T19:06:55.746 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "0.999", 2023-07-21T19:06:55.746 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.746 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "aggressiveness of optimization", 2023-07-21T19:06:55.746 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-07-21T19:06:55.746 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.746 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.747 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.747 INFO:teuthology.orchestra.run.smithi039.stdout: "end_time": { 2023-07-21T19:06:55.747 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "end_time", 2023-07-21T19:06:55.747 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.747 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.747 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.747 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "2359", 2023-07-21T19:06:55.747 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.748 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.748 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.748 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "ending time of day to automatically balance", 2023-07-21T19:06:55.748 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-21T19:06:55.748 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.748 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.748 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.749 INFO:teuthology.orchestra.run.smithi039.stdout: "end_weekday": { 2023-07-21T19:06:55.749 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "end_weekday", 2023-07-21T19:06:55.749 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2023-07-21T19:06:55.749 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.749 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.749 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2023-07-21T19:06:55.749 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "0", 2023-07-21T19:06:55.749 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "6", 2023-07-21T19:06:55.750 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.750 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-07-21T19:06:55.750 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-21T19:06:55.750 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.750 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.750 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.750 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:55.751 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:55.751 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.751 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.751 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.751 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.751 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.751 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.751 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.752 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.752 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.752 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.752 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.752 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.752 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.752 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.752 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.753 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.753 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.753 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.753 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.753 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:55.753 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:55.753 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.753 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.754 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.754 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.754 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.754 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.754 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.754 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.754 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.755 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.755 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.755 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.755 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:55.755 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:55.755 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.755 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.755 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.756 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:55.756 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.756 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.756 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.756 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.756 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.756 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.756 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.757 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.757 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.757 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.757 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.757 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.757 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.757 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.757 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.758 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:55.758 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:55.758 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.758 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.758 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.758 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.758 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.759 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.759 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.759 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.759 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.759 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.759 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.759 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.759 INFO:teuthology.orchestra.run.smithi039.stdout: "min_score": { 2023-07-21T19:06:55.760 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "min_score", 2023-07-21T19:06:55.760 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2023-07-21T19:06:55.760 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.760 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.760 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2023-07-21T19:06:55.760 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.760 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.761 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.761 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-07-21T19:06:55.761 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.761 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.761 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.761 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.761 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": { 2023-07-21T19:06:55.761 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mode", 2023-07-21T19:06:55.762 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.762 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.762 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.762 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "upmap", 2023-07-21T19:06:55.762 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.762 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.762 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.762 INFO:teuthology.orchestra.run.smithi039.stdout: "crush-compat", 2023-07-21T19:06:55.763 INFO:teuthology.orchestra.run.smithi039.stdout: "none", 2023-07-21T19:06:55.763 INFO:teuthology.orchestra.run.smithi039.stdout: "upmap" 2023-07-21T19:06:55.763 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.763 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Balancer mode", 2023-07-21T19:06:55.763 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.763 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.763 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.764 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.764 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_ids": { 2023-07-21T19:06:55.764 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pool_ids", 2023-07-21T19:06:55.764 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.764 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.764 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.764 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.765 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.765 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.765 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.765 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-07-21T19:06:55.765 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.765 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.765 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.765 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.766 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2023-07-21T19:06:55.766 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2023-07-21T19:06:55.766 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2023-07-21T19:06:55.766 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.766 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.766 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2023-07-21T19:06:55.766 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.766 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.767 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.767 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-07-21T19:06:55.767 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.767 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.767 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.767 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.767 INFO:teuthology.orchestra.run.smithi039.stdout: "upmap_max_deviation": { 2023-07-21T19:06:55.768 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "upmap_max_deviation", 2023-07-21T19:06:55.768 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:55.768 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.768 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.768 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2023-07-21T19:06:55.768 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2023-07-21T19:06:55.768 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.768 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.769 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "deviation below which no optimization is attempted", 2023-07-21T19:06:55.769 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-07-21T19:06:55.769 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.769 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.769 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.769 INFO:teuthology.orchestra.run.smithi039.stdout: "upmap_max_optimizations": { 2023-07-21T19:06:55.769 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "upmap_max_optimizations", 2023-07-21T19:06:55.769 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2023-07-21T19:06:55.770 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.770 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.770 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2023-07-21T19:06:55.770 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.770 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.770 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.770 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-07-21T19:06:55.771 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.771 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.771 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.771 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.771 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.771 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.771 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:55.771 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cephadm", 2023-07-21T19:06:55.772 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:55.772 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:55.772 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:55.772 INFO:teuthology.orchestra.run.smithi039.stdout: "agent_down_multiplier": { 2023-07-21T19:06:55.772 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "agent_down_multiplier", 2023-07-21T19:06:55.772 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2023-07-21T19:06:55.772 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.772 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.773 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "3.0", 2023-07-21T19:06:55.773 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.773 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.773 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.773 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-07-21T19:06:55.773 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.773 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.774 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.774 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.774 INFO:teuthology.orchestra.run.smithi039.stdout: "agent_refresh_rate": { 2023-07-21T19:06:55.774 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "agent_refresh_rate", 2023-07-21T19:06:55.774 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2023-07-21T19:06:55.774 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.774 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.774 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "20", 2023-07-21T19:06:55.775 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.775 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.775 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.775 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-07-21T19:06:55.775 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.775 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.775 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.775 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.776 INFO:teuthology.orchestra.run.smithi039.stdout: "agent_starting_port": { 2023-07-21T19:06:55.776 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "agent_starting_port", 2023-07-21T19:06:55.776 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:55.776 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.776 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.776 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "4721", 2023-07-21T19:06:55.776 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.777 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.777 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.777 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-07-21T19:06:55.777 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.777 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.777 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.777 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.777 INFO:teuthology.orchestra.run.smithi039.stdout: "alertmanager_web_password": { 2023-07-21T19:06:55.778 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "alertmanager_web_password", 2023-07-21T19:06:55.778 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.778 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.778 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.778 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "admin", 2023-07-21T19:06:55.778 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.778 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.778 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.779 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Alertmanager web password", 2023-07-21T19:06:55.779 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.779 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.779 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.779 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.779 INFO:teuthology.orchestra.run.smithi039.stdout: "alertmanager_web_user": { 2023-07-21T19:06:55.779 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "alertmanager_web_user", 2023-07-21T19:06:55.779 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.780 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.780 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.780 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "admin", 2023-07-21T19:06:55.780 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.780 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.780 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.780 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Alertmanager web user", 2023-07-21T19:06:55.781 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.781 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.781 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.781 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.781 INFO:teuthology.orchestra.run.smithi039.stdout: "allow_ptrace": { 2023-07-21T19:06:55.781 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "allow_ptrace", 2023-07-21T19:06:55.781 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.781 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.782 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.782 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.782 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.782 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.782 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.782 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-07-21T19:06:55.782 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-07-21T19:06:55.782 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.783 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.783 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.783 INFO:teuthology.orchestra.run.smithi039.stdout: "autotune_interval": { 2023-07-21T19:06:55.783 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "autotune_interval", 2023-07-21T19:06:55.783 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2023-07-21T19:06:55.783 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.783 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.784 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2023-07-21T19:06:55.784 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.784 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.784 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.784 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to autotune daemon memory", 2023-07-21T19:06:55.784 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.784 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.784 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.785 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.785 INFO:teuthology.orchestra.run.smithi039.stdout: "autotune_memory_target_ratio": { 2023-07-21T19:06:55.785 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "autotune_memory_target_ratio", 2023-07-21T19:06:55.785 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2023-07-21T19:06:55.785 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.785 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.785 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0.7", 2023-07-21T19:06:55.786 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.786 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.786 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.786 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-07-21T19:06:55.786 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.786 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.786 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.787 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.787 INFO:teuthology.orchestra.run.smithi039.stdout: "cgroups_split": { 2023-07-21T19:06:55.787 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cgroups_split", 2023-07-21T19:06:55.787 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.787 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.787 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.787 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:55.787 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.788 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.788 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.788 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-07-21T19:06:55.788 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.788 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.788 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.788 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.788 INFO:teuthology.orchestra.run.smithi039.stdout: "config_checks_enabled": { 2023-07-21T19:06:55.789 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "config_checks_enabled", 2023-07-21T19:06:55.789 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.789 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.789 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.789 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.789 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.789 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.790 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.790 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-07-21T19:06:55.790 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.790 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.790 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.790 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.790 INFO:teuthology.orchestra.run.smithi039.stdout: "config_dashboard": { 2023-07-21T19:06:55.790 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "config_dashboard", 2023-07-21T19:06:55.791 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.791 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.791 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.791 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:55.791 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.791 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.791 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.791 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-07-21T19:06:55.792 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.792 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.792 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.792 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.792 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_alertmanager": { 2023-07-21T19:06:55.792 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_alertmanager", 2023-07-21T19:06:55.792 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.793 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.793 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.793 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-07-21T19:06:55.793 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.793 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.793 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.793 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2023-07-21T19:06:55.793 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.794 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.794 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.794 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.794 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_base": { 2023-07-21T19:06:55.794 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_base", 2023-07-21T19:06:55.794 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.794 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.794 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.795 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/ceph", 2023-07-21T19:06:55.795 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.795 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.795 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.795 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Container image name, without the tag", 2023-07-21T19:06:55.795 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.795 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.795 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.796 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.796 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_elasticsearch": { 2023-07-21T19:06:55.796 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_elasticsearch", 2023-07-21T19:06:55.796 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.796 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.796 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.796 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-07-21T19:06:55.797 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.797 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.797 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.797 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "elasticsearch container image", 2023-07-21T19:06:55.797 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.797 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.797 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.797 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.798 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_grafana": { 2023-07-21T19:06:55.798 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_grafana", 2023-07-21T19:06:55.798 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.798 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.798 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.798 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-07-21T19:06:55.798 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.798 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.799 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.799 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2023-07-21T19:06:55.799 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.799 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.799 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.799 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.799 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_haproxy": { 2023-07-21T19:06:55.799 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_haproxy", 2023-07-21T19:06:55.800 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.800 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.800 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.800 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-07-21T19:06:55.800 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.800 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.800 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.801 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "HAproxy container image", 2023-07-21T19:06:55.801 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.801 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.801 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.801 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.801 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_jaeger_agent": { 2023-07-21T19:06:55.801 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_jaeger_agent", 2023-07-21T19:06:55.801 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.802 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.802 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.802 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-07-21T19:06:55.802 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.802 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.802 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.802 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Jaeger agent container image", 2023-07-21T19:06:55.802 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.803 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.803 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.803 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.803 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_jaeger_collector": { 2023-07-21T19:06:55.803 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_jaeger_collector", 2023-07-21T19:06:55.803 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.803 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.803 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.804 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-07-21T19:06:55.804 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.804 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.804 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.804 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Jaeger collector container image", 2023-07-21T19:06:55.804 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.804 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.805 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.805 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.805 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_jaeger_query": { 2023-07-21T19:06:55.805 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_jaeger_query", 2023-07-21T19:06:55.805 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.805 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.805 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.805 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-07-21T19:06:55.806 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.806 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.806 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.806 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Jaeger query container image", 2023-07-21T19:06:55.806 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.806 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.806 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.806 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.807 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_keepalived": { 2023-07-21T19:06:55.807 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_keepalived", 2023-07-21T19:06:55.807 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.807 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.807 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.807 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-07-21T19:06:55.808 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.808 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.808 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.808 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Keepalived container image", 2023-07-21T19:06:55.808 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.808 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.808 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.808 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.809 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_loki": { 2023-07-21T19:06:55.809 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_loki", 2023-07-21T19:06:55.809 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.809 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.809 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.809 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-07-21T19:06:55.809 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.810 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.810 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.810 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Loki container image", 2023-07-21T19:06:55.810 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.810 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.810 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.810 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.810 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_node_exporter": { 2023-07-21T19:06:55.811 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_node_exporter", 2023-07-21T19:06:55.811 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.811 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.811 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.811 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-07-21T19:06:55.811 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.811 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.811 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.812 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2023-07-21T19:06:55.812 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.812 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.812 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.812 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.812 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_prometheus": { 2023-07-21T19:06:55.812 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_prometheus", 2023-07-21T19:06:55.813 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.813 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.813 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.813 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-07-21T19:06:55.813 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.813 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.813 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.813 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2023-07-21T19:06:55.814 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.814 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.814 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.814 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.814 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_promtail": { 2023-07-21T19:06:55.814 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_promtail", 2023-07-21T19:06:55.814 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.814 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.815 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.815 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-07-21T19:06:55.815 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.815 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.815 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.815 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Promtail container image", 2023-07-21T19:06:55.815 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.815 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.816 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.816 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.816 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_snmp_gateway": { 2023-07-21T19:06:55.816 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_snmp_gateway", 2023-07-21T19:06:55.816 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.816 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.816 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.817 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-07-21T19:06:55.817 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.817 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.817 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.817 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SNMP Gateway container image", 2023-07-21T19:06:55.817 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.817 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.817 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.818 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.818 INFO:teuthology.orchestra.run.smithi039.stdout: "container_init": { 2023-07-21T19:06:55.818 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_init", 2023-07-21T19:06:55.818 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.818 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.818 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.818 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:55.818 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.819 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.819 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.819 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Run podman/docker with `--init`", 2023-07-21T19:06:55.819 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.819 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.819 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.819 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.820 INFO:teuthology.orchestra.run.smithi039.stdout: "daemon_cache_timeout": { 2023-07-21T19:06:55.820 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "daemon_cache_timeout", 2023-07-21T19:06:55.820 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2023-07-21T19:06:55.820 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.820 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.820 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2023-07-21T19:06:55.820 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.820 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.821 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.821 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-07-21T19:06:55.821 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.821 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.821 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.821 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.821 INFO:teuthology.orchestra.run.smithi039.stdout: "default_cephadm_command_timeout": { 2023-07-21T19:06:55.821 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "default_cephadm_command_timeout", 2023-07-21T19:06:55.822 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2023-07-21T19:06:55.822 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.822 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.822 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "900", 2023-07-21T19:06:55.822 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.822 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.822 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.823 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-07-21T19:06:55.823 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.823 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.823 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.823 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.823 INFO:teuthology.orchestra.run.smithi039.stdout: "default_registry": { 2023-07-21T19:06:55.823 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "default_registry", 2023-07-21T19:06:55.823 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.824 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.824 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.824 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io", 2023-07-21T19:06:55.824 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.824 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.824 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.824 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-07-21T19:06:55.824 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.825 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.825 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.825 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.825 INFO:teuthology.orchestra.run.smithi039.stdout: "device_cache_timeout": { 2023-07-21T19:06:55.825 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "device_cache_timeout", 2023-07-21T19:06:55.825 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2023-07-21T19:06:55.825 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.826 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.826 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "1800", 2023-07-21T19:06:55.826 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.826 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.826 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.826 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "seconds to cache device inventory", 2023-07-21T19:06:55.826 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.826 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.827 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.827 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.827 INFO:teuthology.orchestra.run.smithi039.stdout: "device_enhanced_scan": { 2023-07-21T19:06:55.827 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "device_enhanced_scan", 2023-07-21T19:06:55.827 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.827 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.827 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.827 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.828 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.828 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.828 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.828 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use libstoragemgmt during device scans", 2023-07-21T19:06:55.828 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.828 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.828 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.829 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.829 INFO:teuthology.orchestra.run.smithi039.stdout: "facts_cache_timeout": { 2023-07-21T19:06:55.829 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "facts_cache_timeout", 2023-07-21T19:06:55.829 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2023-07-21T19:06:55.829 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.829 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.829 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2023-07-21T19:06:55.830 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.830 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.830 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.830 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "seconds to cache host facts data", 2023-07-21T19:06:55.830 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.830 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.830 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.830 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.831 INFO:teuthology.orchestra.run.smithi039.stdout: "host_check_interval": { 2023-07-21T19:06:55.831 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "host_check_interval", 2023-07-21T19:06:55.831 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2023-07-21T19:06:55.831 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.831 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.831 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2023-07-21T19:06:55.831 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.831 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.832 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.832 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to perform a host check", 2023-07-21T19:06:55.832 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.832 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.832 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.832 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.832 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:55.833 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:55.833 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.833 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.833 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.833 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.833 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.833 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.833 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.834 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.834 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.834 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.834 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.834 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.834 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.834 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.834 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.835 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.835 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.835 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.835 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.835 INFO:teuthology.orchestra.run.smithi039.stdout: "log_refresh_metadata": { 2023-07-21T19:06:55.835 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_refresh_metadata", 2023-07-21T19:06:55.835 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.836 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.836 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.836 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.836 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.836 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.836 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.836 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-07-21T19:06:55.836 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.837 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.837 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.837 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.837 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:55.837 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:55.837 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.837 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.837 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.838 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:55.838 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.838 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.838 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.838 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-07-21T19:06:55.838 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.838 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.838 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.839 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.839 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:55.839 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:55.839 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.839 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.839 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.839 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:55.840 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.840 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.840 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.840 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.840 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.840 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.840 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.840 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.841 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.841 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.841 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.841 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.841 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.841 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.841 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.841 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:55.842 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:55.842 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.842 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.842 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.842 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.842 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.842 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.842 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.843 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.843 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.843 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.843 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.843 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.843 INFO:teuthology.orchestra.run.smithi039.stdout: "manage_etc_ceph_ceph_conf": { 2023-07-21T19:06:55.843 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-07-21T19:06:55.843 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.844 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.844 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.844 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.844 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.844 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.844 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.844 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-07-21T19:06:55.845 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.845 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.845 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.845 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.845 INFO:teuthology.orchestra.run.smithi039.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-07-21T19:06:55.845 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-07-21T19:06:55.845 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.845 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.846 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.846 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "*", 2023-07-21T19:06:55.846 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.846 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.846 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.846 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-07-21T19:06:55.846 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.847 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.847 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.847 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.847 INFO:teuthology.orchestra.run.smithi039.stdout: "max_count_per_host": { 2023-07-21T19:06:55.847 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_count_per_host", 2023-07-21T19:06:55.847 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:55.847 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.847 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.848 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2023-07-21T19:06:55.848 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.848 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.848 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.848 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "max number of daemons per service per host", 2023-07-21T19:06:55.848 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.848 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.848 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.849 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.849 INFO:teuthology.orchestra.run.smithi039.stdout: "max_osd_draining_count": { 2023-07-21T19:06:55.849 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_osd_draining_count", 2023-07-21T19:06:55.849 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:55.849 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.849 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.849 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2023-07-21T19:06:55.850 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.850 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.850 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.850 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-07-21T19:06:55.850 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.850 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.850 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.850 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.851 INFO:teuthology.orchestra.run.smithi039.stdout: "migration_current": { 2023-07-21T19:06:55.851 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "migration_current", 2023-07-21T19:06:55.851 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:55.851 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.851 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.851 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.851 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.851 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.852 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.852 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "internal - do not modify", 2023-07-21T19:06:55.852 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.852 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.852 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.852 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.852 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": { 2023-07-21T19:06:55.853 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mode", 2023-07-21T19:06:55.853 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.853 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.853 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.853 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "root", 2023-07-21T19:06:55.853 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.853 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.854 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.854 INFO:teuthology.orchestra.run.smithi039.stdout: "cephadm-package", 2023-07-21T19:06:55.854 INFO:teuthology.orchestra.run.smithi039.stdout: "root" 2023-07-21T19:06:55.854 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.854 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "mode for remote execution of cephadm", 2023-07-21T19:06:55.854 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.854 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.854 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.855 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.855 INFO:teuthology.orchestra.run.smithi039.stdout: "prometheus_alerts_path": { 2023-07-21T19:06:55.855 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "prometheus_alerts_path", 2023-07-21T19:06:55.855 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.855 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.855 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.855 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-07-21T19:06:55.855 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.856 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.856 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.856 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-07-21T19:06:55.856 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.856 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.856 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.856 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.857 INFO:teuthology.orchestra.run.smithi039.stdout: "prometheus_web_password": { 2023-07-21T19:06:55.857 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "prometheus_web_password", 2023-07-21T19:06:55.857 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.857 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.857 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.857 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "admin", 2023-07-21T19:06:55.857 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.857 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.858 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.858 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus web password", 2023-07-21T19:06:55.858 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.858 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.858 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.858 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.858 INFO:teuthology.orchestra.run.smithi039.stdout: "prometheus_web_user": { 2023-07-21T19:06:55.858 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "prometheus_web_user", 2023-07-21T19:06:55.859 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.859 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.859 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.859 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "admin", 2023-07-21T19:06:55.859 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.859 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.859 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.859 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus web user", 2023-07-21T19:06:55.860 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.860 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.860 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.860 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.860 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_insecure": { 2023-07-21T19:06:55.860 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_insecure", 2023-07-21T19:06:55.860 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.861 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.861 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.861 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.861 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.861 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.861 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.861 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-07-21T19:06:55.861 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.862 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.862 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.862 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.862 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_password": { 2023-07-21T19:06:55.862 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_password", 2023-07-21T19:06:55.862 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.862 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.862 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.863 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.863 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.863 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.863 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.863 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-07-21T19:06:55.863 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.863 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.863 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.864 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.864 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_url": { 2023-07-21T19:06:55.864 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_url", 2023-07-21T19:06:55.864 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.864 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.864 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.864 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.865 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.865 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.865 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.865 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-07-21T19:06:55.865 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.865 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.865 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.865 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.866 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_username": { 2023-07-21T19:06:55.866 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_username", 2023-07-21T19:06:55.866 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.866 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.866 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.866 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.866 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.866 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.867 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.867 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-07-21T19:06:55.867 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.867 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.867 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.867 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.867 INFO:teuthology.orchestra.run.smithi039.stdout: "secure_monitoring_stack": { 2023-07-21T19:06:55.868 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "secure_monitoring_stack", 2023-07-21T19:06:55.868 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.868 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.868 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.868 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.868 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.868 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.868 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.869 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-07-21T19:06:55.869 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.869 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.869 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.869 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.870 INFO:teuthology.orchestra.run.smithi039.stdout: "service_discovery_port": { 2023-07-21T19:06:55.871 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "service_discovery_port", 2023-07-21T19:06:55.871 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:55.871 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.871 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.871 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8765", 2023-07-21T19:06:55.871 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.872 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.872 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.872 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "cephadm service discovery port", 2023-07-21T19:06:55.872 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.872 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.872 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.872 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.873 INFO:teuthology.orchestra.run.smithi039.stdout: "ssh_config_file": { 2023-07-21T19:06:55.873 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssh_config_file", 2023-07-21T19:06:55.873 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.873 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.873 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.873 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.873 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.873 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.874 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.874 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-07-21T19:06:55.874 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.874 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.874 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.874 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.874 INFO:teuthology.orchestra.run.smithi039.stdout: "use_agent": { 2023-07-21T19:06:55.874 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "use_agent", 2023-07-21T19:06:55.875 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.875 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.875 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.875 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.875 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.875 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.875 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.875 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-07-21T19:06:55.876 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.876 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.876 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.876 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.876 INFO:teuthology.orchestra.run.smithi039.stdout: "use_repo_digest": { 2023-07-21T19:06:55.877 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "use_repo_digest", 2023-07-21T19:06:55.877 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.877 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.877 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.877 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:55.877 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.877 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.878 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.878 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-07-21T19:06:55.878 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.878 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.878 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.878 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.878 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_on_failed_host_check": { 2023-07-21T19:06:55.878 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_on_failed_host_check", 2023-07-21T19:06:55.879 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.879 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.879 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.879 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:55.879 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.879 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.879 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.879 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise a health warning if the host check fails", 2023-07-21T19:06:55.880 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.880 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.880 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.880 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.880 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_on_stray_daemons": { 2023-07-21T19:06:55.880 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_on_stray_daemons", 2023-07-21T19:06:55.880 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.881 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.881 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.881 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:55.881 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.881 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.881 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.881 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-07-21T19:06:55.881 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.882 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.882 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.882 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.882 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_on_stray_hosts": { 2023-07-21T19:06:55.882 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_on_stray_hosts", 2023-07-21T19:06:55.882 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.882 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.882 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.883 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:55.883 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.883 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.883 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.883 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-07-21T19:06:55.883 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.883 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.883 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.884 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.884 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.884 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.884 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:55.884 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crash", 2023-07-21T19:06:55.884 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:55.884 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:55.885 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:55.885 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:55.885 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:55.885 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.885 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.885 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.885 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.885 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.886 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.886 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.886 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.886 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.886 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.886 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.886 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.886 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.887 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.887 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.887 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.887 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.887 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.887 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.887 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:55.888 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:55.888 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.888 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.888 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.888 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.888 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.888 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.888 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.889 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.889 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.889 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.889 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.889 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.889 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:55.889 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:55.889 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.890 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.890 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.890 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:55.890 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.890 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.890 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.890 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.890 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.891 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.891 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.891 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.891 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.891 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.891 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.891 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.891 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.892 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.892 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.892 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:55.892 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:55.892 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.892 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.892 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.893 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.893 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.893 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.893 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.893 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.893 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.893 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.893 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.894 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.894 INFO:teuthology.orchestra.run.smithi039.stdout: "retain_interval": { 2023-07-21T19:06:55.894 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "retain_interval", 2023-07-21T19:06:55.894 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2023-07-21T19:06:55.894 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.894 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.894 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "31536000", 2023-07-21T19:06:55.894 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.895 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.895 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.895 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how long to retain crashes before pruning them", 2023-07-21T19:06:55.895 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.895 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.895 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.895 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.895 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_recent_interval": { 2023-07-21T19:06:55.896 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_recent_interval", 2023-07-21T19:06:55.896 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2023-07-21T19:06:55.896 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.896 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.896 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "1209600", 2023-07-21T19:06:55.896 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.896 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.897 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.897 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "time interval in which to warn about recent crashes", 2023-07-21T19:06:55.897 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.897 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.897 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.897 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.897 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:55.897 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.898 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:55.898 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "dashboard", 2023-07-21T19:06:55.898 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:55.898 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:55.898 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:55.898 INFO:teuthology.orchestra.run.smithi039.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-07-21T19:06:55.898 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-07-21T19:06:55.898 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:55.899 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.899 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.899 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2023-07-21T19:06:55.899 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.899 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.899 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.899 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.899 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.900 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.900 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.900 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.900 INFO:teuthology.orchestra.run.smithi039.stdout: "ALERTMANAGER_API_HOST": { 2023-07-21T19:06:55.900 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ALERTMANAGER_API_HOST", 2023-07-21T19:06:55.900 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.900 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.901 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.901 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.901 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.901 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.901 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.901 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.901 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.902 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.902 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.902 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.902 INFO:teuthology.orchestra.run.smithi039.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-07-21T19:06:55.902 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-07-21T19:06:55.902 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.902 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.902 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.903 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:55.903 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.903 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.903 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.903 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.903 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.903 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.903 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.904 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.904 INFO:teuthology.orchestra.run.smithi039.stdout: "AUDIT_API_ENABLED": { 2023-07-21T19:06:55.904 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "AUDIT_API_ENABLED", 2023-07-21T19:06:55.904 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.904 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.904 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.904 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.905 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.905 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.905 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.905 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.905 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.905 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.905 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.905 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.906 INFO:teuthology.orchestra.run.smithi039.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-07-21T19:06:55.906 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-07-21T19:06:55.906 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.906 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.906 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.906 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:55.906 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.906 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.907 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.907 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.907 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.907 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.907 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.907 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.907 INFO:teuthology.orchestra.run.smithi039.stdout: "ENABLE_BROWSABLE_API": { 2023-07-21T19:06:55.907 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ENABLE_BROWSABLE_API", 2023-07-21T19:06:55.908 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.908 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.908 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.908 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:55.908 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.908 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.908 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.908 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.909 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.909 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.909 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.909 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.909 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-07-21T19:06:55.909 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-07-21T19:06:55.909 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.910 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.910 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.910 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:55.910 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.910 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.910 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.910 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.910 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.911 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.911 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.911 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.911 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-07-21T19:06:55.911 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-07-21T19:06:55.911 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.911 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.911 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.912 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:55.912 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.912 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.912 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.912 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.912 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.912 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.912 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.913 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.913 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-07-21T19:06:55.913 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-07-21T19:06:55.913 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.913 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.913 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.913 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:55.913 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.914 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.914 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.914 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.914 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.914 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.914 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.914 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.914 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-07-21T19:06:55.915 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-07-21T19:06:55.915 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.915 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.915 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.915 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:55.915 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.915 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.916 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.916 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.916 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.916 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.916 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.916 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.916 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_NFS": { 2023-07-21T19:06:55.916 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-07-21T19:06:55.917 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.917 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.917 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.917 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:55.917 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.917 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.917 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.917 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.918 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.918 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.918 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.918 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.918 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_RBD": { 2023-07-21T19:06:55.918 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-07-21T19:06:55.918 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.918 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.919 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.919 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:55.919 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.919 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.919 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.919 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.919 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.919 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.920 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.920 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.920 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_RGW": { 2023-07-21T19:06:55.920 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-07-21T19:06:55.920 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.920 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.920 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.920 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:55.921 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.921 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.921 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.921 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.921 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.921 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.921 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.921 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.922 INFO:teuthology.orchestra.run.smithi039.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-07-21T19:06:55.922 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-07-21T19:06:55.922 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.922 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.922 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.922 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.922 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.923 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.923 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.923 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.923 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.923 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.923 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.923 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.923 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_PASSWORD": { 2023-07-21T19:06:55.924 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_PASSWORD", 2023-07-21T19:06:55.924 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.924 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.924 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.924 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "admin", 2023-07-21T19:06:55.924 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.924 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.925 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.925 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.925 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.925 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.925 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.925 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.925 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-07-21T19:06:55.925 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-07-21T19:06:55.926 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.926 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.926 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.926 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:55.926 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.926 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.926 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.926 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.927 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.927 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.927 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.927 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.927 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_URL": { 2023-07-21T19:06:55.927 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_URL", 2023-07-21T19:06:55.928 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.928 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.928 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.928 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.928 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.928 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.928 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.929 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.929 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.929 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.929 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.929 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.929 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_USERNAME": { 2023-07-21T19:06:55.929 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_USERNAME", 2023-07-21T19:06:55.929 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.930 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.930 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.930 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "admin", 2023-07-21T19:06:55.930 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.930 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.930 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.930 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.930 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.931 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.931 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.931 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.931 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-07-21T19:06:55.931 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-07-21T19:06:55.931 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.931 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.931 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.932 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.932 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.932 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.932 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.932 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.932 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.932 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.932 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.933 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.933 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-07-21T19:06:55.933 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-07-21T19:06:55.933 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.933 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.933 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.933 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.933 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.934 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.934 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.934 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.934 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.934 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.934 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.934 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.934 INFO:teuthology.orchestra.run.smithi039.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-07-21T19:06:55.935 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-07-21T19:06:55.935 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.935 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.935 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.935 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:55.935 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.935 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.936 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.936 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.936 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.936 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.936 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.936 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.936 INFO:teuthology.orchestra.run.smithi039.stdout: "ISSUE_TRACKER_API_KEY": { 2023-07-21T19:06:55.936 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-07-21T19:06:55.937 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.937 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.937 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.937 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.937 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.937 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.937 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.937 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.938 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.938 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.938 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.938 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.938 INFO:teuthology.orchestra.run.smithi039.stdout: "PROMETHEUS_API_HOST": { 2023-07-21T19:06:55.938 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PROMETHEUS_API_HOST", 2023-07-21T19:06:55.938 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.938 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.939 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.939 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.939 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.939 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.939 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.939 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.939 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.939 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.940 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.940 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.942 INFO:teuthology.orchestra.run.smithi039.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-07-21T19:06:55.942 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-07-21T19:06:55.942 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.942 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.942 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.943 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:55.943 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.943 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.943 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.943 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.943 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.943 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.943 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.944 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.944 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-07-21T19:06:55.944 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-07-21T19:06:55.944 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.944 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.944 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.944 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.945 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.945 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.945 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.945 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.945 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.946 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.946 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.946 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.947 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-07-21T19:06:55.947 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-07-21T19:06:55.947 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.947 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.948 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.948 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.948 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.948 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.948 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.948 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.948 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.949 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.949 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.949 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.949 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-07-21T19:06:55.949 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-07-21T19:06:55.950 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.950 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.950 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.950 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:55.950 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.951 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.951 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.951 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.951 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.951 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.951 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.951 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.951 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-07-21T19:06:55.952 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-07-21T19:06:55.952 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.952 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.952 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.952 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:55.952 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.953 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.953 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.953 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.953 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.954 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.954 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.954 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.954 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-07-21T19:06:55.954 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-07-21T19:06:55.954 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.954 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.954 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.955 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.955 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.955 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.955 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.955 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.955 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.955 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.955 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.956 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.956 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-07-21T19:06:55.956 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-07-21T19:06:55.956 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.956 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.957 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.957 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.957 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.957 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.958 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.958 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.958 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.958 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.958 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.958 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.958 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-07-21T19:06:55.959 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-07-21T19:06:55.959 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.959 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.959 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.959 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.959 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.959 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.960 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.960 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.960 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.960 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.960 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.960 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.960 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_ENABLED": { 2023-07-21T19:06:55.961 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_ENABLED", 2023-07-21T19:06:55.961 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.961 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.961 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.961 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:55.961 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.961 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.961 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.962 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.962 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.962 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.962 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.962 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.962 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-07-21T19:06:55.962 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-07-21T19:06:55.962 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.963 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.963 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.963 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-07-21T19:06:55.963 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.963 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.963 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.963 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.964 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.964 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.964 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.964 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.964 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-07-21T19:06:55.964 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-07-21T19:06:55.965 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:55.965 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.965 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.965 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2023-07-21T19:06:55.965 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.965 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.966 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.966 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.966 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.966 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.966 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.966 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.966 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-07-21T19:06:55.967 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-07-21T19:06:55.967 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:55.967 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.967 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.968 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8", 2023-07-21T19:06:55.968 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.968 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.968 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.969 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.969 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.969 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.969 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.969 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.969 INFO:teuthology.orchestra.run.smithi039.stdout: "REST_REQUESTS_TIMEOUT": { 2023-07-21T19:06:55.969 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-07-21T19:06:55.969 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:55.970 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.970 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.970 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "45", 2023-07-21T19:06:55.970 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.970 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.970 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.971 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.971 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.971 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.971 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.971 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.972 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_ACCESS_KEY": { 2023-07-21T19:06:55.972 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_ACCESS_KEY", 2023-07-21T19:06:55.972 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.972 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.972 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.972 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.973 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.973 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.973 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.973 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.973 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.973 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.973 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.973 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.974 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-07-21T19:06:55.974 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-07-21T19:06:55.974 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.974 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.974 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.974 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "admin", 2023-07-21T19:06:55.974 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.975 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.975 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.975 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.975 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.975 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.975 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.975 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.976 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_SECRET_KEY": { 2023-07-21T19:06:55.976 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_SECRET_KEY", 2023-07-21T19:06:55.976 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.976 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.976 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.976 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.976 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.976 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.977 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.977 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.977 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.977 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.977 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.978 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.978 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_SSL_VERIFY": { 2023-07-21T19:06:55.978 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_SSL_VERIFY", 2023-07-21T19:06:55.978 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.979 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.979 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.979 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:55.979 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.979 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.979 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.979 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.979 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.980 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.980 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.980 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.980 INFO:teuthology.orchestra.run.smithi039.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-07-21T19:06:55.980 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-07-21T19:06:55.980 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:55.980 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.981 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.981 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2023-07-21T19:06:55.981 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.981 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.981 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.982 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.982 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.982 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.982 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.982 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.983 INFO:teuthology.orchestra.run.smithi039.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-07-21T19:06:55.983 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-07-21T19:06:55.983 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:55.983 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.983 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.983 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2023-07-21T19:06:55.983 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.983 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.984 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.984 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.984 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.984 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.984 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.984 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.984 INFO:teuthology.orchestra.run.smithi039.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-07-21T19:06:55.985 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-07-21T19:06:55.985 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:55.985 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.985 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.985 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2023-07-21T19:06:55.985 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.985 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.985 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.986 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.986 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.986 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.986 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.986 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.986 INFO:teuthology.orchestra.run.smithi039.stdout: "cross_origin_url": { 2023-07-21T19:06:55.987 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cross_origin_url", 2023-07-21T19:06:55.987 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.987 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.987 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.987 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.987 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.988 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.989 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.989 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.989 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.989 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.989 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.989 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.989 INFO:teuthology.orchestra.run.smithi039.stdout: "crt_file": { 2023-07-21T19:06:55.989 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crt_file", 2023-07-21T19:06:55.990 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.990 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.990 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.990 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.990 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.990 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.990 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.991 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.991 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.991 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.991 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.991 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.991 INFO:teuthology.orchestra.run.smithi039.stdout: "debug": { 2023-07-21T19:06:55.991 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "debug", 2023-07-21T19:06:55.991 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:55.992 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.992 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.992 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:55.992 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.992 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.992 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.992 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Enable/disable debug options", 2023-07-21T19:06:55.992 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.993 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.993 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.993 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.993 INFO:teuthology.orchestra.run.smithi039.stdout: "jwt_token_ttl": { 2023-07-21T19:06:55.993 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "jwt_token_ttl", 2023-07-21T19:06:55.993 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:55.993 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.994 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.994 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "28800", 2023-07-21T19:06:55.994 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.994 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.994 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.994 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.995 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.995 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.995 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.995 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.995 INFO:teuthology.orchestra.run.smithi039.stdout: "key_file": { 2023-07-21T19:06:55.995 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "key_file", 2023-07-21T19:06:55.995 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.996 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.996 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:55.996 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.996 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.996 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.996 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:55.996 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:55.997 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:55.997 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:55.997 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:55.997 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:55.997 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:55.997 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:55.998 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:55.998 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:55.998 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:55.998 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:55.998 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:55.998 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:55.998 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:55.999 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:55.999 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:55.999 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:55.999 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:55.999 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:55.999 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:55.999 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:55.999 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.000 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.000 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.000 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.000 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.000 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:56.000 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:56.000 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.001 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.001 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.001 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.001 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.001 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.001 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.001 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.001 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.002 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.002 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.002 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.002 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:56.002 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:56.002 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.002 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.003 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.003 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:56.003 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.003 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.003 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.004 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.004 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.004 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.004 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.004 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.004 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.004 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.005 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.005 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.005 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.005 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.005 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.005 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:56.005 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:56.005 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.006 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.006 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.006 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.006 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.006 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.006 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.006 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.006 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.007 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.007 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.007 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.007 INFO:teuthology.orchestra.run.smithi039.stdout: "motd": { 2023-07-21T19:06:56.007 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "motd", 2023-07-21T19:06:56.007 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.007 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.007 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.008 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.008 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.008 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.008 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.008 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "The message of the day", 2023-07-21T19:06:56.008 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.008 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.009 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.009 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.009 INFO:teuthology.orchestra.run.smithi039.stdout: "redirect_resolve_ip_addr": { 2023-07-21T19:06:56.009 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "redirect_resolve_ip_addr", 2023-07-21T19:06:56.009 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.009 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.009 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.010 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.010 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.010 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.010 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.010 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.010 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.011 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.011 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.011 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.011 INFO:teuthology.orchestra.run.smithi039.stdout: "server_addr": { 2023-07-21T19:06:56.011 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_addr", 2023-07-21T19:06:56.012 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.012 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.012 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.012 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "::", 2023-07-21T19:06:56.012 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.012 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.012 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.012 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.013 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.013 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.013 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.013 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.013 INFO:teuthology.orchestra.run.smithi039.stdout: "server_port": { 2023-07-21T19:06:56.013 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_port", 2023-07-21T19:06:56.013 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:56.014 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.014 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.014 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8080", 2023-07-21T19:06:56.014 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.014 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.014 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.014 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.014 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.015 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.015 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.015 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.015 INFO:teuthology.orchestra.run.smithi039.stdout: "ssl": { 2023-07-21T19:06:56.015 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssl", 2023-07-21T19:06:56.015 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.015 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.016 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.016 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:56.016 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.016 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.016 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.016 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.016 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.016 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.017 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.017 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.017 INFO:teuthology.orchestra.run.smithi039.stdout: "ssl_server_port": { 2023-07-21T19:06:56.018 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssl_server_port", 2023-07-21T19:06:56.018 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:56.018 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.018 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.018 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8443", 2023-07-21T19:06:56.018 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.019 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.019 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.019 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.019 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.019 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.019 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.020 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.020 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_behaviour": { 2023-07-21T19:06:56.020 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_behaviour", 2023-07-21T19:06:56.020 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.020 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.020 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.021 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "redirect", 2023-07-21T19:06:56.021 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.021 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.021 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.021 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.021 INFO:teuthology.orchestra.run.smithi039.stdout: "redirect" 2023-07-21T19:06:56.021 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.021 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.022 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.022 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.022 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.022 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.022 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_error_status_code": { 2023-07-21T19:06:56.022 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_error_status_code", 2023-07-21T19:06:56.022 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:56.022 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.023 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.023 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "500", 2023-07-21T19:06:56.023 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "400", 2023-07-21T19:06:56.023 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "599", 2023-07-21T19:06:56.023 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.023 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.023 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.023 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.024 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.024 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.024 INFO:teuthology.orchestra.run.smithi039.stdout: "url_prefix": { 2023-07-21T19:06:56.024 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "url_prefix", 2023-07-21T19:06:56.024 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.024 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.024 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.024 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.025 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.025 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.025 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.025 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.025 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.025 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.025 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.026 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.026 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.026 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.026 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:56.026 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "devicehealth", 2023-07-21T19:06:56.026 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:56.026 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:56.026 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:56.027 INFO:teuthology.orchestra.run.smithi039.stdout: "enable_monitoring": { 2023-07-21T19:06:56.027 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enable_monitoring", 2023-07-21T19:06:56.027 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.027 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.027 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.027 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:56.027 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.028 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.028 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.028 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "monitor device health metrics", 2023-07-21T19:06:56.028 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.028 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.028 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.028 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.028 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:56.029 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:56.029 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.029 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.029 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.029 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.029 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.029 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.029 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.030 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.030 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.030 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.030 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.030 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.031 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.031 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.031 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.031 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.031 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.031 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.031 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.031 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:56.032 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:56.032 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.032 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.032 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.032 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.032 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.032 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.032 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.033 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.033 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.033 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.033 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.033 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.033 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:56.034 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:56.034 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.034 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.034 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.034 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:56.034 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.034 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.034 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.035 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.035 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.035 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.035 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.035 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.035 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.035 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.035 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.036 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.036 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.036 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.036 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.037 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:56.037 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:56.037 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.038 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.038 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.038 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.038 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.038 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.038 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.039 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.039 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.039 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.039 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.040 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.040 INFO:teuthology.orchestra.run.smithi039.stdout: "mark_out_threshold": { 2023-07-21T19:06:56.040 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mark_out_threshold", 2023-07-21T19:06:56.040 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2023-07-21T19:06:56.041 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.041 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.041 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "2419200", 2023-07-21T19:06:56.041 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.041 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.041 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.041 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-07-21T19:06:56.042 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.042 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.042 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.042 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.042 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_name": { 2023-07-21T19:06:56.042 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pool_name", 2023-07-21T19:06:56.043 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.043 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.043 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.043 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "device_health_metrics", 2023-07-21T19:06:56.043 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.043 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.043 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.043 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "name of pool in which to store device health metrics", 2023-07-21T19:06:56.044 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.044 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.044 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.044 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.044 INFO:teuthology.orchestra.run.smithi039.stdout: "retention_period": { 2023-07-21T19:06:56.044 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "retention_period", 2023-07-21T19:06:56.044 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2023-07-21T19:06:56.045 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.045 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.045 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "15552000", 2023-07-21T19:06:56.045 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.045 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.045 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.045 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how long to retain device health metrics", 2023-07-21T19:06:56.046 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.046 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.046 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.046 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.046 INFO:teuthology.orchestra.run.smithi039.stdout: "scrape_frequency": { 2023-07-21T19:06:56.046 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "scrape_frequency", 2023-07-21T19:06:56.046 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2023-07-21T19:06:56.046 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.047 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.047 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "86400", 2023-07-21T19:06:56.047 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.047 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.047 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.047 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to scrape device health metrics", 2023-07-21T19:06:56.047 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.047 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.048 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.048 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.048 INFO:teuthology.orchestra.run.smithi039.stdout: "self_heal": { 2023-07-21T19:06:56.048 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "self_heal", 2023-07-21T19:06:56.048 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.048 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.048 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.049 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:56.049 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.049 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.049 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.049 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-07-21T19:06:56.049 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.049 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.049 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.050 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.050 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2023-07-21T19:06:56.050 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2023-07-21T19:06:56.050 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2023-07-21T19:06:56.050 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.050 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.050 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2023-07-21T19:06:56.051 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.051 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.051 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.051 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to wake up and check device health", 2023-07-21T19:06:56.051 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.051 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.052 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.052 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.052 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_threshold": { 2023-07-21T19:06:56.052 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_threshold", 2023-07-21T19:06:56.053 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2023-07-21T19:06:56.053 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.053 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.053 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "7257600", 2023-07-21T19:06:56.053 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.054 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.054 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.054 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-07-21T19:06:56.054 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.054 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.054 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.054 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.055 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.055 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.055 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:56.055 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "diskprediction_local", 2023-07-21T19:06:56.055 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:56.055 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:56.055 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:56.055 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:56.056 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:56.056 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.056 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.056 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.056 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.056 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.056 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.056 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.057 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.057 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.057 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.057 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.057 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.057 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.057 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.057 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.058 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.058 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.058 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.058 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.058 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:56.058 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:56.058 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.058 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.059 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.059 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.059 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.059 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.059 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.059 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.059 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.059 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.060 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.060 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.060 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:56.060 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:56.060 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.060 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.060 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.061 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:56.061 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.061 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.061 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.061 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.061 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.061 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.061 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.062 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.062 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.062 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.062 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.062 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.062 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.062 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.062 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.063 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:56.063 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:56.063 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.063 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.063 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.063 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.063 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.063 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.064 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.064 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.064 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.064 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.064 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.064 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.065 INFO:teuthology.orchestra.run.smithi039.stdout: "predict_interval": { 2023-07-21T19:06:56.065 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "predict_interval", 2023-07-21T19:06:56.065 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.065 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.065 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.066 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "86400", 2023-07-21T19:06:56.066 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.066 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.066 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.066 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.066 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.067 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.067 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.067 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.067 INFO:teuthology.orchestra.run.smithi039.stdout: "predictor_model": { 2023-07-21T19:06:56.067 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "predictor_model", 2023-07-21T19:06:56.067 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.067 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.068 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.068 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "prophetstor", 2023-07-21T19:06:56.068 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.068 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.068 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.068 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.069 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.069 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.069 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.069 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.069 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2023-07-21T19:06:56.069 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2023-07-21T19:06:56.070 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.070 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.070 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.070 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2023-07-21T19:06:56.070 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.070 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.070 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.070 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.071 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.071 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.071 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.071 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.071 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.071 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.071 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:56.071 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "influx", 2023-07-21T19:06:56.072 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": false, 2023-07-21T19:06:56.072 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "influxdb python module not found", 2023-07-21T19:06:56.072 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:56.072 INFO:teuthology.orchestra.run.smithi039.stdout: "batch_size": { 2023-07-21T19:06:56.072 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "batch_size", 2023-07-21T19:06:56.073 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:56.073 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.073 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.073 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5000", 2023-07-21T19:06:56.073 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.073 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.074 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.074 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-07-21T19:06:56.074 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.074 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.075 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.075 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.075 INFO:teuthology.orchestra.run.smithi039.stdout: "database": { 2023-07-21T19:06:56.075 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "database", 2023-07-21T19:06:56.075 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.075 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.075 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.075 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "ceph", 2023-07-21T19:06:56.076 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.076 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.076 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.076 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-07-21T19:06:56.076 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.076 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.076 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.077 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.077 INFO:teuthology.orchestra.run.smithi039.stdout: "hostname": { 2023-07-21T19:06:56.077 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "hostname", 2023-07-21T19:06:56.077 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.077 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.077 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.077 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.077 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.078 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.078 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.078 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "InfluxDB server hostname", 2023-07-21T19:06:56.078 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.078 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.078 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.078 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.078 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2023-07-21T19:06:56.079 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2023-07-21T19:06:56.079 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2023-07-21T19:06:56.079 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.079 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.079 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "30", 2023-07-21T19:06:56.079 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "5", 2023-07-21T19:06:56.079 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.079 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.080 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-07-21T19:06:56.080 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.080 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.080 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.080 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.080 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:56.080 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:56.081 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.081 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.081 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.081 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.081 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.081 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.081 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.081 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.082 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.082 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.082 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.082 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.082 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.082 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.082 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.082 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.083 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.083 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.083 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.083 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:56.083 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:56.083 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.083 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.083 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.084 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.084 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.084 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.084 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.084 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.084 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.084 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.084 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.085 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.085 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:56.085 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:56.085 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.085 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.085 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.085 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:56.085 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.086 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.086 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.086 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.086 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.086 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.086 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.086 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.087 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.087 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.087 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.087 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.087 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.087 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.087 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.087 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:56.088 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:56.088 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.088 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.088 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.088 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.088 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.088 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.089 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.089 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.089 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.089 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.089 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.089 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.089 INFO:teuthology.orchestra.run.smithi039.stdout: "password": { 2023-07-21T19:06:56.089 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "password", 2023-07-21T19:06:56.090 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.090 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.090 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.090 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.090 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.090 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.090 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.091 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "password of InfluxDB server user", 2023-07-21T19:06:56.091 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.091 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.091 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.091 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.091 INFO:teuthology.orchestra.run.smithi039.stdout: "port": { 2023-07-21T19:06:56.091 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "port", 2023-07-21T19:06:56.091 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:56.092 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.092 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.092 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8086", 2023-07-21T19:06:56.092 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.092 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.092 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.092 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "InfluxDB server port", 2023-07-21T19:06:56.092 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.093 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.093 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.093 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.093 INFO:teuthology.orchestra.run.smithi039.stdout: "ssl": { 2023-07-21T19:06:56.093 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssl", 2023-07-21T19:06:56.093 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.093 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.094 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.094 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "false", 2023-07-21T19:06:56.094 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.094 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.094 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.094 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-07-21T19:06:56.094 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.095 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.095 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.095 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.095 INFO:teuthology.orchestra.run.smithi039.stdout: "threads": { 2023-07-21T19:06:56.095 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "threads", 2023-07-21T19:06:56.095 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:56.095 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.095 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.096 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2023-07-21T19:06:56.096 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2023-07-21T19:06:56.096 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "32", 2023-07-21T19:06:56.096 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.096 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-07-21T19:06:56.096 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.096 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.096 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.097 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.097 INFO:teuthology.orchestra.run.smithi039.stdout: "username": { 2023-07-21T19:06:56.097 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "username", 2023-07-21T19:06:56.097 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.097 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.097 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.097 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.098 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.098 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.098 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.098 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "username of InfluxDB server user", 2023-07-21T19:06:56.098 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.098 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.098 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.098 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.099 INFO:teuthology.orchestra.run.smithi039.stdout: "verify_ssl": { 2023-07-21T19:06:56.099 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "verify_ssl", 2023-07-21T19:06:56.099 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.099 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.099 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.099 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "true", 2023-07-21T19:06:56.099 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.099 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.100 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.100 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-07-21T19:06:56.100 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.100 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.100 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.100 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.100 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.100 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.101 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:56.101 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "insights", 2023-07-21T19:06:56.101 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:56.101 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:56.101 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:56.101 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:56.101 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:56.101 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.102 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.102 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.102 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.102 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.102 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.102 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.102 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.102 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.103 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.103 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.103 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.103 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.103 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.104 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.104 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.104 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.104 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.104 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.104 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:56.104 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:56.105 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.105 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.105 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.105 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.105 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.105 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.105 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.105 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.106 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.106 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.106 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.106 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.106 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:56.106 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:56.106 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.106 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.107 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.107 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:56.107 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.107 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.107 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.107 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.107 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.107 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.108 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.108 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.108 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.108 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.108 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.108 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.108 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.109 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.109 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.109 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:56.109 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:56.109 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.109 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.109 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.109 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.110 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.110 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.110 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.110 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.110 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.110 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.111 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.111 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.111 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.111 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.111 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:56.111 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "iostat", 2023-07-21T19:06:56.111 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:56.111 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:56.112 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:56.112 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:56.112 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:56.112 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.112 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.112 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.113 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.113 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.113 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.113 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.113 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.113 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.113 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.113 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.114 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.114 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.114 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.114 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.114 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.114 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.114 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.114 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.115 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:56.115 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:56.115 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.115 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.115 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.115 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.115 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.115 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.116 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.116 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.116 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.116 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.116 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.116 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.116 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:56.117 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:56.117 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.117 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.117 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.117 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:56.117 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.117 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.118 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.118 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.118 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.118 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.118 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.118 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.118 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.118 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.119 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.119 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.119 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.119 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.119 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.119 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:56.119 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:56.119 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.120 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.120 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.120 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.120 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.120 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.120 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.120 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.120 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.121 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.121 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.121 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.121 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.121 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.121 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:56.121 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "k8sevents", 2023-07-21T19:06:56.122 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:56.122 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:56.122 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:56.122 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_event_retention_days": { 2023-07-21T19:06:56.122 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ceph_event_retention_days", 2023-07-21T19:06:56.122 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:56.122 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.123 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.123 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "7", 2023-07-21T19:06:56.123 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.123 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.123 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.123 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Days to hold ceph event information within local cache", 2023-07-21T19:06:56.123 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.123 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.124 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.124 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.124 INFO:teuthology.orchestra.run.smithi039.stdout: "config_check_secs": { 2023-07-21T19:06:56.124 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "config_check_secs", 2023-07-21T19:06:56.124 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:56.124 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.124 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.125 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2023-07-21T19:06:56.125 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "10", 2023-07-21T19:06:56.125 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.125 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.125 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-07-21T19:06:56.125 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.126 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.126 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.126 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.126 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:56.126 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:56.126 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.126 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.127 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.127 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.127 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.127 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.127 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.127 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.127 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.127 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.128 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.128 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.128 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.128 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.128 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.128 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.128 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.128 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.129 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.129 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:56.129 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:56.129 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.129 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.129 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.129 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.129 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.130 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.130 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.130 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.130 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.130 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.130 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.130 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.130 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:56.131 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:56.131 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.131 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.131 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.131 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:56.131 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.131 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.132 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.132 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.132 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.132 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.132 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.132 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.132 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.132 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.133 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.133 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.133 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.133 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.133 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.133 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:56.133 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:56.134 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.134 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.134 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.134 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.134 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.134 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.134 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.134 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.135 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.135 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.135 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.135 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.135 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.135 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.135 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:56.136 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "localpool", 2023-07-21T19:06:56.136 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:56.136 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:56.136 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:56.136 INFO:teuthology.orchestra.run.smithi039.stdout: "failure_domain": { 2023-07-21T19:06:56.136 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "failure_domain", 2023-07-21T19:06:56.136 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.136 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.137 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.137 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "host", 2023-07-21T19:06:56.137 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.137 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.137 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.138 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "failure domain for any created local pool", 2023-07-21T19:06:56.138 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-07-21T19:06:56.138 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.138 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.138 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.138 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:56.138 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:56.138 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.139 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.139 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.139 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.139 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.139 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.139 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.139 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.139 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.140 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.140 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.140 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.140 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.140 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.140 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.140 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.141 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.141 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.141 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.141 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:56.141 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:56.141 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.141 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.142 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.142 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.142 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.142 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.142 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.142 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.143 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.143 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.143 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.143 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.143 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:56.143 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:56.143 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.143 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.144 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.144 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:56.144 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.144 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.144 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.144 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.144 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.144 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.145 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.145 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.145 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.145 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.145 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.145 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.145 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.145 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.146 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.146 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:56.146 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:56.146 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.146 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.146 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.146 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.147 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.147 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.147 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.147 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.147 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.147 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.147 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.147 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.148 INFO:teuthology.orchestra.run.smithi039.stdout: "min_size": { 2023-07-21T19:06:56.148 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "min_size", 2023-07-21T19:06:56.148 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:56.148 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.148 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.148 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.148 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.149 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.149 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.149 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "default min_size for any created local pool", 2023-07-21T19:06:56.149 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-07-21T19:06:56.149 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.149 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.149 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.149 INFO:teuthology.orchestra.run.smithi039.stdout: "num_rep": { 2023-07-21T19:06:56.150 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "num_rep", 2023-07-21T19:06:56.150 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:56.150 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.150 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.150 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "3", 2023-07-21T19:06:56.150 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.150 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.151 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.151 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "default replica count for any created local pool", 2023-07-21T19:06:56.151 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.151 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.151 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.151 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.151 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num": { 2023-07-21T19:06:56.152 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pg_num", 2023-07-21T19:06:56.152 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:56.152 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.152 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.152 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "128", 2023-07-21T19:06:56.153 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.153 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.153 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.153 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "default pg_num for any created local pool", 2023-07-21T19:06:56.153 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.153 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.153 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.153 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.154 INFO:teuthology.orchestra.run.smithi039.stdout: "prefix": { 2023-07-21T19:06:56.154 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "prefix", 2023-07-21T19:06:56.154 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.154 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.154 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.154 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.154 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.155 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.155 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.155 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "name prefix for any created local pool", 2023-07-21T19:06:56.155 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.155 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.155 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.155 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.155 INFO:teuthology.orchestra.run.smithi039.stdout: "subtree": { 2023-07-21T19:06:56.156 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "subtree", 2023-07-21T19:06:56.156 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.156 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.156 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.156 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "rack", 2023-07-21T19:06:56.156 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.156 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.157 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.157 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "CRUSH level for which to create a local pool", 2023-07-21T19:06:56.157 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-07-21T19:06:56.157 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.157 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.157 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.157 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.158 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.158 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:56.158 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mds_autoscaler", 2023-07-21T19:06:56.158 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:56.158 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:56.159 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:56.159 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:56.159 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:56.159 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.159 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.159 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.159 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.159 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.160 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.160 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.160 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.160 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.160 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.160 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.160 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.161 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.161 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.161 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.161 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.161 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.161 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.161 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.162 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:56.162 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:56.162 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.162 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.162 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.162 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.162 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.162 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.163 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.163 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.163 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.163 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.163 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.163 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.163 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:56.164 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:56.164 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.164 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.164 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.164 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:56.164 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.164 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.165 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.165 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.165 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.165 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.165 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.165 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.165 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.166 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.166 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.166 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.166 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.166 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.166 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.166 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:56.166 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:56.167 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.167 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.167 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.167 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.167 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.167 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.167 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.167 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.168 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.168 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.168 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.169 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.169 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.169 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.169 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:56.169 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mirroring", 2023-07-21T19:06:56.169 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:56.169 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:56.170 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:56.170 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:56.170 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:56.170 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.170 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.170 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.170 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.170 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.171 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.171 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.171 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.171 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.171 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.171 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.171 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.172 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.172 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.172 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.172 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.172 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.172 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.172 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.173 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:56.173 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:56.173 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.173 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.173 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.173 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.173 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.173 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.174 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.174 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.174 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.174 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.174 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.174 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.174 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:56.175 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:56.175 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.175 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.175 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.175 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:56.175 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.175 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.175 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.176 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.176 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.176 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.176 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.176 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.176 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.176 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.177 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.177 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.177 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.177 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.177 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.177 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:56.177 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:56.177 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.178 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.178 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.178 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.178 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.178 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.178 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.178 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.178 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.179 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.179 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.179 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.179 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.179 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.179 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:56.179 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "nfs", 2023-07-21T19:06:56.179 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:56.180 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:56.180 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:56.180 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:56.180 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:56.180 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.180 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.180 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.180 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.181 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.181 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.181 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.181 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.181 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.181 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.181 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.181 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.182 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.182 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.182 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.182 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.182 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.182 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.182 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.182 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:56.183 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:56.183 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.183 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.183 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.183 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.183 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.183 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.183 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.184 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.184 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.184 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.184 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.184 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.184 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:56.184 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:56.185 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.185 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.185 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.185 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:56.185 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.185 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.185 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.185 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.186 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.186 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.186 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.186 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.186 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.186 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.186 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.186 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.187 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.187 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.187 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.187 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:56.187 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:56.187 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.187 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.188 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.188 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.188 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.188 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.188 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.189 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.189 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.189 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.189 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.189 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.190 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.190 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.190 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:56.190 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "orchestrator", 2023-07-21T19:06:56.190 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:56.190 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:56.190 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:56.190 INFO:teuthology.orchestra.run.smithi039.stdout: "fail_fs": { 2023-07-21T19:06:56.191 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "fail_fs", 2023-07-21T19:06:56.191 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.191 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.191 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.191 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.191 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.191 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.192 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.192 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-07-21T19:06:56.192 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.192 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.192 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.192 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.192 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:56.193 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:56.193 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.193 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.193 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.193 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.193 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.194 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.194 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.194 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.194 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.194 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.194 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.194 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.194 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.195 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.195 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.195 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.195 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.195 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.195 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.195 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:56.196 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:56.196 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.196 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.196 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.196 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.196 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.196 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.197 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.197 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.197 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.197 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.197 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.197 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.197 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:56.198 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:56.198 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.198 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.198 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.198 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:56.199 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.199 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.199 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.199 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.199 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.199 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.200 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.200 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.200 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.200 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.200 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.200 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.200 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.201 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.201 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.201 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:56.201 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:56.201 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.201 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.201 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.201 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.202 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.202 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.202 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.202 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.202 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.202 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.202 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.202 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.203 INFO:teuthology.orchestra.run.smithi039.stdout: "orchestrator": { 2023-07-21T19:06:56.203 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "orchestrator", 2023-07-21T19:06:56.203 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.203 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.203 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.203 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.203 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.204 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.204 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.204 INFO:teuthology.orchestra.run.smithi039.stdout: "cephadm", 2023-07-21T19:06:56.204 INFO:teuthology.orchestra.run.smithi039.stdout: "rook", 2023-07-21T19:06:56.204 INFO:teuthology.orchestra.run.smithi039.stdout: "test_orchestrator" 2023-07-21T19:06:56.204 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.204 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Orchestrator backend", 2023-07-21T19:06:56.205 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.205 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.205 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.205 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.205 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.205 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.205 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:56.205 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "osd_perf_query", 2023-07-21T19:06:56.206 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:56.206 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:56.206 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:56.206 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:56.206 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:56.206 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.206 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.206 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.207 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.207 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.207 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.207 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.207 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.207 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.207 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.208 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.208 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.208 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.208 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.208 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.208 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.208 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.208 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.209 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.209 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:56.209 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:56.209 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.209 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.209 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.209 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.209 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.210 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.210 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.210 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.210 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.210 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.210 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.210 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.210 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:56.211 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:56.211 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.211 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.211 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.211 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:56.211 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.211 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.211 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.212 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.212 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.212 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.212 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.212 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.212 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.212 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.212 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.213 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.213 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.213 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.213 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.213 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:56.213 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:56.213 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.214 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.214 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.214 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.214 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.214 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.214 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.214 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.214 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.215 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.215 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.215 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.215 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.215 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.215 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:56.215 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "osd_support", 2023-07-21T19:06:56.215 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:56.216 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:56.216 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:56.216 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:56.216 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:56.216 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.216 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.216 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.216 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.217 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.217 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.217 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.217 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.217 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.217 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.217 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.217 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.218 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.218 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.218 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.218 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.218 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.218 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.218 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.219 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:56.219 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:56.219 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.219 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.219 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.219 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.219 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.219 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.220 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.220 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.220 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.220 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.220 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.220 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.220 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:56.221 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:56.221 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.221 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.221 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.221 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:56.221 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.221 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.221 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.222 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.222 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.222 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.222 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.222 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.222 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.222 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.223 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.223 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.223 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.223 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.223 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.223 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:56.224 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:56.224 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.224 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.224 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.224 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.224 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.224 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.225 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.225 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.225 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.225 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.225 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.225 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.226 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.226 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.226 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:56.226 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pg_autoscaler", 2023-07-21T19:06:56.226 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:56.226 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:56.226 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:56.227 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:56.227 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:56.227 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.227 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.227 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.227 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.227 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.228 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.228 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.228 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.228 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.228 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.228 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.228 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.229 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.229 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.229 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.229 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.229 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.229 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.229 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.230 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:56.230 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:56.230 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.230 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.230 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.230 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.230 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.230 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.231 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.231 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.231 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.231 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.231 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.231 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.231 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:56.232 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:56.232 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.232 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.232 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.232 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:56.232 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.232 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.232 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.233 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.233 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.233 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.233 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.233 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.233 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.233 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.234 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.234 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.234 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.234 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.234 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.234 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:56.234 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:56.235 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.235 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.235 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.235 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.235 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.235 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.235 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.236 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.236 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.236 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.236 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.236 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.236 INFO:teuthology.orchestra.run.smithi039.stdout: "noautoscale": { 2023-07-21T19:06:56.236 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "noautoscale", 2023-07-21T19:06:56.236 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.237 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.237 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.237 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.237 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.237 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.237 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.237 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "global autoscale flag", 2023-07-21T19:06:56.238 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-07-21T19:06:56.238 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.238 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.238 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.238 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2023-07-21T19:06:56.238 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2023-07-21T19:06:56.238 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2023-07-21T19:06:56.238 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.239 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.239 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2023-07-21T19:06:56.239 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.239 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.239 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.239 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.239 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.240 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.240 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.240 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.240 INFO:teuthology.orchestra.run.smithi039.stdout: "threshold": { 2023-07-21T19:06:56.240 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "threshold", 2023-07-21T19:06:56.240 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2023-07-21T19:06:56.240 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.240 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.241 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "3.0", 2023-07-21T19:06:56.241 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1.0", 2023-07-21T19:06:56.241 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.241 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.241 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "scaling threshold", 2023-07-21T19:06:56.241 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-07-21T19:06:56.241 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.241 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.242 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.242 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.242 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.242 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:56.242 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "progress", 2023-07-21T19:06:56.242 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:56.242 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:56.243 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:56.243 INFO:teuthology.orchestra.run.smithi039.stdout: "allow_pg_recovery_event": { 2023-07-21T19:06:56.243 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "allow_pg_recovery_event", 2023-07-21T19:06:56.243 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.243 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.243 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.243 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.243 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.244 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.244 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.244 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "allow the module to show pg recovery progress", 2023-07-21T19:06:56.244 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.244 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.244 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.244 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.244 INFO:teuthology.orchestra.run.smithi039.stdout: "enabled": { 2023-07-21T19:06:56.245 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enabled", 2023-07-21T19:06:56.245 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.245 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.245 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.245 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:56.245 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.245 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.245 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.246 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.246 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.246 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.246 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.246 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.246 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:56.246 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:56.246 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.247 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.247 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.247 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.247 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.247 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.247 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.247 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.247 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.248 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.248 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.248 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.248 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.248 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.248 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.249 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.249 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.249 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.249 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.249 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:56.249 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:56.249 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.249 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.250 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.250 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.250 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.250 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.250 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.250 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.250 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.250 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.251 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.251 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.251 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:56.251 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:56.251 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.251 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.251 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.251 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:56.252 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.252 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.252 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.252 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.252 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.252 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.252 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.253 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.253 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.253 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.253 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.254 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.254 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.254 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.254 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.254 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:56.254 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:56.254 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.254 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.255 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.255 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.255 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.255 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.255 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.255 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.255 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.256 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.256 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.256 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.256 INFO:teuthology.orchestra.run.smithi039.stdout: "max_completed_events": { 2023-07-21T19:06:56.256 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_completed_events", 2023-07-21T19:06:56.256 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:56.256 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.257 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.257 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "50", 2023-07-21T19:06:56.257 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.257 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.257 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.257 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "number of past completed events to remember", 2023-07-21T19:06:56.257 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.257 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.258 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.258 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.258 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2023-07-21T19:06:56.258 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2023-07-21T19:06:56.258 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2023-07-21T19:06:56.258 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.258 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.258 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2023-07-21T19:06:56.259 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.259 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.259 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.259 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how long the module is going to sleep", 2023-07-21T19:06:56.259 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.259 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.259 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.260 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.260 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.260 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.260 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:56.260 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "prometheus", 2023-07-21T19:06:56.260 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:56.260 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:56.261 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:56.261 INFO:teuthology.orchestra.run.smithi039.stdout: "cache": { 2023-07-21T19:06:56.261 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cache", 2023-07-21T19:06:56.261 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.261 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.261 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.261 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:56.261 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.262 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.262 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.262 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.262 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.262 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.262 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.262 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.262 INFO:teuthology.orchestra.run.smithi039.stdout: "exclude_perf_counters": { 2023-07-21T19:06:56.263 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "exclude_perf_counters", 2023-07-21T19:06:56.263 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.263 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.263 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.263 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:56.263 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.263 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.264 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.264 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-07-21T19:06:56.264 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-07-21T19:06:56.264 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.264 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.264 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.264 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:56.265 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:56.265 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.265 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.265 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.265 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.265 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.265 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.265 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.266 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.266 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.266 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.266 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.266 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.266 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.266 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.266 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.267 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.267 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.267 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.267 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.267 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:56.267 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:56.267 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.267 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.268 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.268 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.268 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.268 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.268 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.268 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.268 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.269 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.269 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.269 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.269 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:56.269 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:56.269 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.269 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.269 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.270 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:56.270 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.270 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.270 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.270 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.270 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.270 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.270 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.271 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.271 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.271 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.271 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.271 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.271 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.271 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.271 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.272 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:56.272 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:56.272 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.272 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.272 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.272 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.272 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.272 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.273 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.273 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.273 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.273 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.273 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.273 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.274 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_stats_pools": { 2023-07-21T19:06:56.274 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rbd_stats_pools", 2023-07-21T19:06:56.274 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.274 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.274 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.274 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.274 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.274 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.275 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.275 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.275 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.275 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.275 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.275 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.275 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_stats_pools_refresh_interval": { 2023-07-21T19:06:56.276 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-07-21T19:06:56.276 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:56.276 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.276 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.276 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "300", 2023-07-21T19:06:56.276 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.276 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.277 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.277 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.277 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.277 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.277 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.277 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.277 INFO:teuthology.orchestra.run.smithi039.stdout: "scrape_interval": { 2023-07-21T19:06:56.277 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "scrape_interval", 2023-07-21T19:06:56.278 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2023-07-21T19:06:56.278 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.278 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.278 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "15.0", 2023-07-21T19:06:56.278 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.278 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.278 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.278 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.279 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.279 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.279 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.279 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.279 INFO:teuthology.orchestra.run.smithi039.stdout: "server_addr": { 2023-07-21T19:06:56.279 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_addr", 2023-07-21T19:06:56.279 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.280 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.280 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.280 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "::", 2023-07-21T19:06:56.280 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.280 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.280 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.280 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-07-21T19:06:56.281 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.281 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.281 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.281 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.281 INFO:teuthology.orchestra.run.smithi039.stdout: "server_port": { 2023-07-21T19:06:56.281 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_port", 2023-07-21T19:06:56.281 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:56.281 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.282 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.282 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "9283", 2023-07-21T19:06:56.282 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.282 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.282 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.282 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-07-21T19:06:56.282 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.283 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.283 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.283 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.283 INFO:teuthology.orchestra.run.smithi039.stdout: "stale_cache_strategy": { 2023-07-21T19:06:56.283 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "stale_cache_strategy", 2023-07-21T19:06:56.283 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.283 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.284 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.284 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "log", 2023-07-21T19:06:56.284 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.284 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.284 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.284 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.285 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.285 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.285 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.285 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.285 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_behaviour": { 2023-07-21T19:06:56.285 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_behaviour", 2023-07-21T19:06:56.285 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.285 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.286 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.286 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "default", 2023-07-21T19:06:56.286 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.286 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.286 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.286 INFO:teuthology.orchestra.run.smithi039.stdout: "default", 2023-07-21T19:06:56.286 INFO:teuthology.orchestra.run.smithi039.stdout: "error" 2023-07-21T19:06:56.286 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.287 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.287 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.287 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.287 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.287 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.287 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_error_status_code": { 2023-07-21T19:06:56.287 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_error_status_code", 2023-07-21T19:06:56.287 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:56.288 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.288 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.288 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "500", 2023-07-21T19:06:56.288 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "400", 2023-07-21T19:06:56.288 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "599", 2023-07-21T19:06:56.289 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.289 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.289 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.289 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.289 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.289 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.289 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.289 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.290 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:56.290 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rbd_support", 2023-07-21T19:06:56.290 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:56.290 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:56.290 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:56.290 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:56.290 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:56.290 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.291 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.291 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.291 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.291 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.291 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.291 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.291 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.291 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.292 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.292 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.292 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.292 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.292 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.292 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.292 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.293 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.293 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.293 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.293 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:56.293 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:56.293 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.293 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.294 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.294 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.294 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.294 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.294 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.294 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.294 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.295 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.295 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.295 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.295 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:56.295 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:56.295 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.295 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.296 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.296 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:56.296 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.296 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.296 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.296 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.296 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.296 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.297 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.297 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.297 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.297 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.297 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.297 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.297 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.298 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.298 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.298 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:56.298 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:56.298 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.298 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.299 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.299 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.299 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.299 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.299 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.299 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.299 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.300 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.300 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.300 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.300 INFO:teuthology.orchestra.run.smithi039.stdout: "max_concurrent_snap_create": { 2023-07-21T19:06:56.300 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_concurrent_snap_create", 2023-07-21T19:06:56.300 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:56.300 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.300 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.301 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2023-07-21T19:06:56.301 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.301 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.301 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.301 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.301 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.301 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.301 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.302 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.302 INFO:teuthology.orchestra.run.smithi039.stdout: "mirror_snapshot_schedule": { 2023-07-21T19:06:56.302 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mirror_snapshot_schedule", 2023-07-21T19:06:56.302 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.302 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.302 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.302 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.303 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.303 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.303 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.303 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.303 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.303 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.303 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.304 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.304 INFO:teuthology.orchestra.run.smithi039.stdout: "trash_purge_schedule": { 2023-07-21T19:06:56.304 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "trash_purge_schedule", 2023-07-21T19:06:56.304 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.304 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.304 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.304 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.304 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.305 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.305 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.305 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.305 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.305 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.305 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.305 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.305 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.306 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.306 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:56.306 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "restful", 2023-07-21T19:06:56.306 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:56.306 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:56.306 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:56.306 INFO:teuthology.orchestra.run.smithi039.stdout: "enable_auth": { 2023-07-21T19:06:56.306 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enable_auth", 2023-07-21T19:06:56.307 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.307 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.307 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.307 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:56.307 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.307 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.307 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.308 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.308 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.308 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.308 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.308 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.308 INFO:teuthology.orchestra.run.smithi039.stdout: "key_file": { 2023-07-21T19:06:56.308 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "key_file", 2023-07-21T19:06:56.308 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.309 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.309 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.309 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.309 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.309 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.309 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.310 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.310 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.310 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.310 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.310 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.310 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:56.310 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:56.310 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.311 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.311 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.311 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.311 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.311 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.311 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.311 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.311 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.312 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.312 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.312 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.312 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.312 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.312 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.312 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.312 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.313 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.313 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.313 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:56.313 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:56.313 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.313 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.314 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.314 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.314 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.314 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.314 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.314 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.315 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.315 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.315 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.315 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.315 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:56.315 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:56.315 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.315 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.316 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.316 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:56.316 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.316 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.316 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.316 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.316 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.316 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.317 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.317 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.317 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.317 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.317 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.317 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.317 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.318 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.318 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.318 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:56.318 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:56.318 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.318 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.318 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.318 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.319 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.319 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.319 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.319 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.319 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.319 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.319 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.319 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.320 INFO:teuthology.orchestra.run.smithi039.stdout: "server_addr": { 2023-07-21T19:06:56.320 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_addr", 2023-07-21T19:06:56.320 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.320 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.320 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.320 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.320 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.320 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.321 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.321 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.321 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.321 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.321 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.321 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.321 INFO:teuthology.orchestra.run.smithi039.stdout: "server_port": { 2023-07-21T19:06:56.322 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_port", 2023-07-21T19:06:56.322 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.322 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.322 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.322 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.322 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.322 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.323 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.323 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.323 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.323 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.323 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.323 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.323 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.324 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.324 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:56.324 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rgw", 2023-07-21T19:06:56.324 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:56.324 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:56.324 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:56.324 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:56.324 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:56.325 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.325 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.325 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.325 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.325 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.325 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.325 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.326 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.326 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.326 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.326 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.326 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.326 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.326 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.326 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.327 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.327 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.327 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.327 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.327 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:56.327 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:56.328 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.328 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.328 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.328 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.328 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.328 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.328 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.328 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.329 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.329 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.329 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.329 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.329 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:56.329 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:56.329 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.329 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.330 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.330 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:56.330 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.330 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.330 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.330 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.330 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.331 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.331 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.331 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.331 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.331 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.331 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.331 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.331 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.332 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.332 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.332 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:56.332 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:56.332 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.332 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.332 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.332 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.333 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.333 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.333 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.333 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.333 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.333 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.334 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.334 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.334 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.334 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.334 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:56.334 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rook", 2023-07-21T19:06:56.334 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:56.334 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:56.335 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:56.335 INFO:teuthology.orchestra.run.smithi039.stdout: "drive_group_interval": { 2023-07-21T19:06:56.335 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "drive_group_interval", 2023-07-21T19:06:56.335 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2023-07-21T19:06:56.335 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.335 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.335 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "300.0", 2023-07-21T19:06:56.336 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.336 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.336 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.336 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-07-21T19:06:56.336 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.336 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.336 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.337 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.337 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:56.337 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:56.337 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.337 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.337 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.337 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.338 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.338 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.338 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.338 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.338 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.338 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.338 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.339 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.339 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.339 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.339 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.339 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.339 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.339 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.339 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.340 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:56.340 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:56.340 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.340 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.340 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.340 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.340 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.341 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.341 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.341 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.341 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.341 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.341 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.341 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.341 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:56.342 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:56.342 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.342 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.342 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.342 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:56.342 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.342 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.342 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.343 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.343 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.343 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.343 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.343 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.343 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.344 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.344 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.344 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.344 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.344 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.344 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.345 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:56.345 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:56.345 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.345 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.345 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.345 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.345 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.345 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.346 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.346 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.346 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.346 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.346 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.346 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.346 INFO:teuthology.orchestra.run.smithi039.stdout: "storage_class": { 2023-07-21T19:06:56.347 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "storage_class", 2023-07-21T19:06:56.347 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.347 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.347 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.347 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "local", 2023-07-21T19:06:56.347 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.347 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.347 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.348 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-07-21T19:06:56.348 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.348 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.348 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.348 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.348 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.348 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.349 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:56.349 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "selftest", 2023-07-21T19:06:56.349 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:56.349 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:56.349 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:56.349 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:56.350 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:56.350 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.350 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.350 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.350 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.350 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.350 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.350 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.351 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.351 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.351 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.351 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.351 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.351 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.351 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.351 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.352 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.352 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.352 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.352 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.352 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:56.352 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:56.353 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.353 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.353 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.353 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.353 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.353 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.353 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.353 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.354 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.354 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.354 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.354 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.354 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:56.354 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:56.355 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.355 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.355 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.355 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:56.355 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.355 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.355 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.356 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.356 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.356 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.356 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.356 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.356 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.356 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.356 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.357 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.357 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.357 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.357 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.357 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:56.357 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:56.358 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.358 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.358 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.358 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.358 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.358 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.359 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.359 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.359 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.359 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.359 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.359 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.359 INFO:teuthology.orchestra.run.smithi039.stdout: "roption1": { 2023-07-21T19:06:56.359 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "roption1", 2023-07-21T19:06:56.360 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.360 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.360 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.360 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.360 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.360 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.360 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.361 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.361 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.361 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.361 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.361 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.361 INFO:teuthology.orchestra.run.smithi039.stdout: "roption2": { 2023-07-21T19:06:56.361 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "roption2", 2023-07-21T19:06:56.361 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.362 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.362 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.362 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "xyz", 2023-07-21T19:06:56.362 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.362 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.362 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.362 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.362 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.363 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.363 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.363 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.363 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption1": { 2023-07-21T19:06:56.363 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption1", 2023-07-21T19:06:56.363 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.363 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.363 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.364 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.364 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.364 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.364 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.364 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.364 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.364 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.365 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.365 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.365 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption2": { 2023-07-21T19:06:56.365 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption2", 2023-07-21T19:06:56.365 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:56.365 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.366 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.366 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.366 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.366 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.366 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.366 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.366 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.367 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.367 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.367 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.367 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption3": { 2023-07-21T19:06:56.367 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption3", 2023-07-21T19:06:56.367 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2023-07-21T19:06:56.367 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.367 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.368 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.368 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.368 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.368 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.368 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.368 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.368 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.368 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.369 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.369 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption4": { 2023-07-21T19:06:56.369 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption4", 2023-07-21T19:06:56.369 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.369 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.369 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.369 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.369 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.370 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.370 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.370 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.370 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.370 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.370 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.370 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.370 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption5": { 2023-07-21T19:06:56.371 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption5", 2023-07-21T19:06:56.371 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.371 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.371 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.371 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.371 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.371 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.371 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.372 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.372 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.372 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.372 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.372 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.372 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption6": { 2023-07-21T19:06:56.372 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption6", 2023-07-21T19:06:56.373 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.373 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.373 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.373 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:56.373 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.373 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.373 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.374 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.374 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.374 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.374 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.374 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.374 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption7": { 2023-07-21T19:06:56.374 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption7", 2023-07-21T19:06:56.374 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:56.375 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.375 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.375 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.375 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2023-07-21T19:06:56.375 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "42", 2023-07-21T19:06:56.375 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.375 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.376 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.376 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.376 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.376 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.376 INFO:teuthology.orchestra.run.smithi039.stdout: "testkey": { 2023-07-21T19:06:56.376 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "testkey", 2023-07-21T19:06:56.376 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.377 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.377 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.377 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.377 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.377 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.377 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.377 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.377 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.378 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.378 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.378 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.378 INFO:teuthology.orchestra.run.smithi039.stdout: "testlkey": { 2023-07-21T19:06:56.378 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "testlkey", 2023-07-21T19:06:56.378 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.378 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.379 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.379 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.379 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.379 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.379 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.379 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.379 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.379 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.380 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.380 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.380 INFO:teuthology.orchestra.run.smithi039.stdout: "testnewline": { 2023-07-21T19:06:56.380 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "testnewline", 2023-07-21T19:06:56.380 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.380 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.380 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.381 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.381 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.381 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.381 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.381 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.381 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.382 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.382 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.382 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.382 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.382 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.382 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:56.382 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "snap_schedule", 2023-07-21T19:06:56.382 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:56.383 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:56.383 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:56.383 INFO:teuthology.orchestra.run.smithi039.stdout: "allow_m_granularity": { 2023-07-21T19:06:56.383 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "allow_m_granularity", 2023-07-21T19:06:56.383 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.383 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.383 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.384 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.384 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.384 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.384 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.384 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "allow minute scheduled snapshots", 2023-07-21T19:06:56.384 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.384 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.384 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.385 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.385 INFO:teuthology.orchestra.run.smithi039.stdout: "dump_on_update": { 2023-07-21T19:06:56.385 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "dump_on_update", 2023-07-21T19:06:56.385 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.385 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.385 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.385 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.385 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.386 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.386 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.386 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "dump database to debug log on update", 2023-07-21T19:06:56.386 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.386 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.386 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.387 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.387 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:56.387 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:56.387 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.387 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.387 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.387 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.388 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.388 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.388 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.388 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.388 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.388 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.388 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.388 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.389 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.389 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.389 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.389 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.389 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.389 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.389 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.390 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:56.390 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:56.390 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.390 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.390 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.390 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.390 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.391 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.391 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.391 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.391 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.391 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.391 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.391 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.391 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:56.392 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:56.392 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.392 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.392 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.392 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:56.392 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.392 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.392 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.393 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.393 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.393 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.393 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.393 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.393 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.393 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.393 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.394 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.394 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.394 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.394 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.394 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:56.394 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:56.394 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.395 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.395 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.395 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.395 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.395 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.395 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.395 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.395 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.396 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.396 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.396 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.396 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.396 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.396 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:56.396 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "stats", 2023-07-21T19:06:56.397 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:56.397 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:56.397 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:56.397 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:56.397 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:56.397 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.397 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.397 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.398 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.398 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.398 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.398 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.398 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.398 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.398 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.398 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.399 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.399 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.399 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.399 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.399 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.399 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.399 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.399 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.400 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:56.400 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:56.400 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.400 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.400 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.400 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.400 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.401 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.401 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.401 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.401 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.401 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.401 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.401 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.402 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:56.402 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:56.402 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.402 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.402 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.402 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:56.402 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.402 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.403 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.403 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.403 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.403 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.403 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.403 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.403 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.404 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.404 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.404 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.404 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.404 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.404 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.404 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:56.405 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:56.405 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.405 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.405 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.405 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.405 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.405 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.406 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.406 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.406 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.406 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.406 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.406 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.406 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.406 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.407 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:56.407 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "status", 2023-07-21T19:06:56.407 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:56.407 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:56.407 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:56.407 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:56.407 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:56.408 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.408 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.408 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.408 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.408 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.408 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.408 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.408 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.409 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.409 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.409 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.409 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.409 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.409 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.409 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.410 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.410 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.410 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.410 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.410 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:56.410 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:56.410 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.410 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.411 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.411 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.411 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.411 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.411 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.411 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.411 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.411 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.412 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.412 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.412 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:56.412 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:56.412 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.412 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.412 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.413 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:56.413 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.413 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.413 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.413 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.413 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.413 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.414 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.414 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.414 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.414 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.414 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.414 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.414 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.414 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.415 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.415 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:56.415 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:56.415 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.415 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.415 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.415 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.416 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.416 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.416 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.416 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.416 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.416 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.416 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.416 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.417 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.417 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.417 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:56.417 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "telegraf", 2023-07-21T19:06:56.417 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:56.417 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:56.417 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:56.417 INFO:teuthology.orchestra.run.smithi039.stdout: "address": { 2023-07-21T19:06:56.418 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "address", 2023-07-21T19:06:56.418 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.418 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.418 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.418 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-07-21T19:06:56.418 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.418 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.418 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.419 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.419 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.419 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.419 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.419 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.419 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2023-07-21T19:06:56.419 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2023-07-21T19:06:56.419 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2023-07-21T19:06:56.420 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.420 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.420 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "15", 2023-07-21T19:06:56.420 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.420 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.420 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.420 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.420 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.421 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.421 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.421 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.421 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:56.421 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:56.421 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.421 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.422 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.422 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.422 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.422 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.422 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.422 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.422 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.423 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.423 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.423 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.423 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.423 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.423 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.423 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.424 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.424 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.424 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.424 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:56.424 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:56.424 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.424 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.424 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.425 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.425 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.425 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.425 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.425 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.425 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.425 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.425 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.426 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.426 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:56.426 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:56.426 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.426 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.426 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.426 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:56.427 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.427 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.427 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.427 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.427 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.427 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.427 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.427 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.428 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.428 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.428 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.428 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.428 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.428 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.429 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.429 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:56.429 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:56.429 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.429 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.429 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.429 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.430 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.430 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.430 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.430 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.430 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.430 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.430 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.430 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.431 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.431 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.431 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:56.431 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "telemetry", 2023-07-21T19:06:56.431 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:56.431 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:56.432 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:56.432 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_basic": { 2023-07-21T19:06:56.432 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_basic", 2023-07-21T19:06:56.432 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.432 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.432 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.433 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:56.433 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.433 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.433 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.433 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share basic cluster information (size, version)", 2023-07-21T19:06:56.433 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.433 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.434 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.434 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.434 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_crash": { 2023-07-21T19:06:56.434 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_crash", 2023-07-21T19:06:56.434 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.434 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.434 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.434 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:56.435 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.435 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.435 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.435 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-07-21T19:06:56.435 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.435 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.435 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.436 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.436 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_device": { 2023-07-21T19:06:56.436 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_device", 2023-07-21T19:06:56.436 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.436 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.436 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.437 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:56.437 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.437 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.437 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.437 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-07-21T19:06:56.437 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.437 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.437 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.438 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.438 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_ident": { 2023-07-21T19:06:56.438 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_ident", 2023-07-21T19:06:56.438 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.438 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.438 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.438 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.439 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.439 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.439 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.439 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-07-21T19:06:56.439 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.439 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.439 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.439 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.440 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_perf": { 2023-07-21T19:06:56.440 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_perf", 2023-07-21T19:06:56.440 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.440 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.440 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.440 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.440 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.440 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.441 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.441 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share various performance metrics of a cluster", 2023-07-21T19:06:56.441 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.441 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.441 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.441 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.441 INFO:teuthology.orchestra.run.smithi039.stdout: "contact": { 2023-07-21T19:06:56.442 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "contact", 2023-07-21T19:06:56.442 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.442 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.442 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.442 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.442 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.442 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.443 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.443 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.443 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.443 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.443 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.443 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.443 INFO:teuthology.orchestra.run.smithi039.stdout: "description": { 2023-07-21T19:06:56.444 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "description", 2023-07-21T19:06:56.444 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.444 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.444 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.444 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.444 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.444 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.445 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.445 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.445 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.445 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.445 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.445 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.445 INFO:teuthology.orchestra.run.smithi039.stdout: "device_url": { 2023-07-21T19:06:56.445 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "device_url", 2023-07-21T19:06:56.446 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.446 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.446 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.446 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-07-21T19:06:56.446 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.446 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.446 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.447 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.447 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.447 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.447 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.447 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.447 INFO:teuthology.orchestra.run.smithi039.stdout: "enabled": { 2023-07-21T19:06:56.447 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enabled", 2023-07-21T19:06:56.447 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.448 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.448 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.448 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.448 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.448 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.448 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.448 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.449 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.449 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.449 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.449 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.449 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2023-07-21T19:06:56.449 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2023-07-21T19:06:56.450 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:56.450 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.450 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.450 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "24", 2023-07-21T19:06:56.450 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "8", 2023-07-21T19:06:56.450 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.450 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.450 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.451 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.451 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.451 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.451 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.451 INFO:teuthology.orchestra.run.smithi039.stdout: "last_opt_revision": { 2023-07-21T19:06:56.451 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "last_opt_revision", 2023-07-21T19:06:56.451 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:56.451 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.452 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.452 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "1", 2023-07-21T19:06:56.452 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.452 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.452 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.452 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.452 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.453 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.453 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.453 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.453 INFO:teuthology.orchestra.run.smithi039.stdout: "leaderboard": { 2023-07-21T19:06:56.453 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "leaderboard", 2023-07-21T19:06:56.453 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.453 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.453 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.454 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.454 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.454 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.454 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.454 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.454 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.454 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.454 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.455 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.455 INFO:teuthology.orchestra.run.smithi039.stdout: "leaderboard_description": { 2023-07-21T19:06:56.455 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "leaderboard_description", 2023-07-21T19:06:56.455 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.455 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.455 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.455 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.456 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.456 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.456 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.456 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.456 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.456 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.456 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.457 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.457 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:56.457 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:56.457 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.457 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.457 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.457 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.458 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.458 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.458 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.458 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.458 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.458 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.458 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.458 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.459 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.459 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.459 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.459 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.459 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.459 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.459 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.459 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:56.460 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:56.460 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.460 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.460 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.460 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.460 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.461 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.461 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.461 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.461 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.461 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.461 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.461 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.462 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:56.462 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:56.462 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.462 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.462 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.462 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:56.462 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.462 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.463 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.463 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.463 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.463 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.463 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.463 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.464 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.464 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.464 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.464 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.464 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.464 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.464 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.465 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:56.465 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:56.465 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.465 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.465 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.465 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.465 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.466 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.466 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.466 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.466 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.466 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.466 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.466 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.466 INFO:teuthology.orchestra.run.smithi039.stdout: "organization": { 2023-07-21T19:06:56.467 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "organization", 2023-07-21T19:06:56.467 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.467 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.467 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.467 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.467 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.467 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.468 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.468 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.468 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.468 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.468 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.468 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.468 INFO:teuthology.orchestra.run.smithi039.stdout: "proxy": { 2023-07-21T19:06:56.469 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "proxy", 2023-07-21T19:06:56.469 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.469 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.469 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.469 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.469 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.469 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.469 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.470 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.470 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.470 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.470 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.470 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.470 INFO:teuthology.orchestra.run.smithi039.stdout: "url": { 2023-07-21T19:06:56.470 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "url", 2023-07-21T19:06:56.470 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.471 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.471 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.471 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-07-21T19:06:56.471 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.471 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.471 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.471 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.472 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.472 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.472 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.472 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.472 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.472 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.472 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:56.472 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "test_orchestrator", 2023-07-21T19:06:56.473 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:56.473 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:56.473 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:56.473 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:56.473 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:56.473 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.473 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.473 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.474 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.474 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.474 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.474 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.474 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.474 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.474 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.475 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.475 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.475 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.475 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.475 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.475 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.475 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.475 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.476 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.476 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:56.476 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:56.476 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.476 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.476 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.476 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.477 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.477 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.477 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.477 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.477 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.477 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.477 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.477 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.478 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:56.478 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:56.478 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.478 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.478 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.478 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:56.478 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.478 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.479 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.479 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.479 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.479 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.479 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.479 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.479 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.479 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.480 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.480 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.480 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.480 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.480 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.480 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:56.481 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:56.481 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.481 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.481 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.481 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.481 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.481 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.481 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.482 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.482 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.482 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.482 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.482 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.482 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.482 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.482 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:56.483 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "volumes", 2023-07-21T19:06:56.483 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:56.483 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:56.483 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:56.483 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:56.483 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:56.483 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.484 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.484 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.484 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.484 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.484 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.484 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.484 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.484 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.485 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.485 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.485 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.485 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.485 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.486 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.486 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.486 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.486 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.486 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.486 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:56.486 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:56.486 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.487 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.487 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.487 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.487 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.487 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.487 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.487 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.488 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.488 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.488 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.488 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.488 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:56.488 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:56.489 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.489 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.489 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.489 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:56.489 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.489 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.489 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.489 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.490 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.490 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.490 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.490 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.490 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.490 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.491 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.491 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.491 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.491 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.491 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.491 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:56.491 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:56.491 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.492 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.492 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.492 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.492 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.492 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.492 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.492 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.492 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.493 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.493 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.493 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.493 INFO:teuthology.orchestra.run.smithi039.stdout: "max_concurrent_clones": { 2023-07-21T19:06:56.493 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_concurrent_clones", 2023-07-21T19:06:56.494 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:56.494 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.494 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.494 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "4", 2023-07-21T19:06:56.494 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.494 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.494 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.494 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Number of asynchronous cloner threads", 2023-07-21T19:06:56.495 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.495 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.495 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.495 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.495 INFO:teuthology.orchestra.run.smithi039.stdout: "snapshot_clone_delay": { 2023-07-21T19:06:56.495 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "snapshot_clone_delay", 2023-07-21T19:06:56.495 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:56.495 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.496 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.496 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2023-07-21T19:06:56.496 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.496 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.496 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.496 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-07-21T19:06:56.496 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.497 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.497 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.497 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.497 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.497 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.497 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:56.497 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix", 2023-07-21T19:06:56.497 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:56.498 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:56.498 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:56.498 INFO:teuthology.orchestra.run.smithi039.stdout: "discovery_interval": { 2023-07-21T19:06:56.498 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "discovery_interval", 2023-07-21T19:06:56.498 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2023-07-21T19:06:56.498 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.498 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.499 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "100", 2023-07-21T19:06:56.499 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.499 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.499 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.499 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.499 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.499 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.499 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.500 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.500 INFO:teuthology.orchestra.run.smithi039.stdout: "identifier": { 2023-07-21T19:06:56.500 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "identifier", 2023-07-21T19:06:56.500 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.500 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.500 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.500 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.500 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.501 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.501 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.501 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.501 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.501 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.501 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.501 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.501 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2023-07-21T19:06:56.502 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2023-07-21T19:06:56.502 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2023-07-21T19:06:56.502 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.502 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.502 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2023-07-21T19:06:56.502 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.502 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.502 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.503 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.503 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.503 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.503 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.503 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.503 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:56.503 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:56.503 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.504 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.504 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.504 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.504 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.504 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.504 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.504 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.505 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.505 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.505 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.505 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.505 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.505 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.505 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.505 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.506 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.506 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.506 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.506 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:56.506 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:56.506 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.506 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.506 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.507 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.507 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.507 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.507 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.507 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.507 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.507 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.507 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.508 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.508 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:56.508 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:56.508 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.508 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.508 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.508 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:56.509 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.509 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.509 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.509 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.509 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.509 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.509 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.510 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.510 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.510 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.510 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.510 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.510 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.510 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.510 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.511 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:56.511 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:56.511 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.511 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.511 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.511 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.511 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.511 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.512 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.512 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.512 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.512 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.512 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.512 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.512 INFO:teuthology.orchestra.run.smithi039.stdout: "zabbix_host": { 2023-07-21T19:06:56.513 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix_host", 2023-07-21T19:06:56.513 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.513 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.513 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.513 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.513 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.513 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.514 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.514 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.514 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.514 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.514 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.514 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.514 INFO:teuthology.orchestra.run.smithi039.stdout: "zabbix_port": { 2023-07-21T19:06:56.514 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix_port", 2023-07-21T19:06:56.515 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:56.515 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.515 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.515 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10051", 2023-07-21T19:06:56.515 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.515 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.515 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.516 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.516 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.516 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.516 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.516 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.516 INFO:teuthology.orchestra.run.smithi039.stdout: "zabbix_sender": { 2023-07-21T19:06:56.516 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix_sender", 2023-07-21T19:06:56.516 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.517 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.517 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.517 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-07-21T19:06:56.517 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.517 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.517 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.518 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.518 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.518 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.518 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.518 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.518 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.518 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.518 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2023-07-21T19:06:56.519 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.519 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.519 INFO:teuthology.orchestra.run.smithi039.stdout: "modules": [ 2023-07-21T19:06:56.519 INFO:teuthology.orchestra.run.smithi039.stdout: "cephadm", 2023-07-21T19:06:56.519 INFO:teuthology.orchestra.run.smithi039.stdout: "dashboard", 2023-07-21T19:06:56.519 INFO:teuthology.orchestra.run.smithi039.stdout: "iostat", 2023-07-21T19:06:56.519 INFO:teuthology.orchestra.run.smithi039.stdout: "nfs", 2023-07-21T19:06:56.519 INFO:teuthology.orchestra.run.smithi039.stdout: "restful", 2023-07-21T19:06:56.520 INFO:teuthology.orchestra.run.smithi039.stdout: "snap_schedule" 2023-07-21T19:06:56.520 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.520 INFO:teuthology.orchestra.run.smithi039.stdout: "available_modules": [ 2023-07-21T19:06:56.520 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:56.520 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "alerts", 2023-07-21T19:06:56.520 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:56.520 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:56.521 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:56.521 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2023-07-21T19:06:56.521 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2023-07-21T19:06:56.521 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2023-07-21T19:06:56.521 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.521 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.521 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2023-07-21T19:06:56.522 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.522 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.522 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.522 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How frequently to reexamine health status", 2023-07-21T19:06:56.522 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.522 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.523 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.523 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.523 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:56.523 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:56.523 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.523 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.524 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.524 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.524 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.524 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.524 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.524 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.524 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.525 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.525 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.525 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.525 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.525 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.525 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.526 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.526 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.526 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.526 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.526 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:56.526 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:56.526 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.527 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.527 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.527 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.527 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.527 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.527 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.527 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.527 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.528 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.528 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.528 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.528 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:56.528 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:56.528 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.528 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.529 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.529 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:56.529 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.529 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.529 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.529 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.529 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.530 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.530 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.530 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.530 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.530 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.530 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.530 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.530 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.531 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.531 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.531 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:56.531 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:56.531 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.531 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.531 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.531 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.532 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.532 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.532 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.532 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.532 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.532 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.532 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.533 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.533 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_destination": { 2023-07-21T19:06:56.533 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_destination", 2023-07-21T19:06:56.533 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.533 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.533 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.533 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.533 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.534 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.534 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.534 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Email address to send alerts to", 2023-07-21T19:06:56.534 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.534 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.534 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.534 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.535 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_from_name": { 2023-07-21T19:06:56.535 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_from_name", 2023-07-21T19:06:56.535 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.535 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.535 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.535 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "Ceph", 2023-07-21T19:06:56.535 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.535 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.536 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.536 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Email From: name", 2023-07-21T19:06:56.536 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.536 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.536 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.536 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.536 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_host": { 2023-07-21T19:06:56.537 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_host", 2023-07-21T19:06:56.537 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.537 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.537 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.537 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.537 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.537 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.538 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.538 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SMTP server", 2023-07-21T19:06:56.538 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.538 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.538 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.538 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.538 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_password": { 2023-07-21T19:06:56.538 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_password", 2023-07-21T19:06:56.539 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.539 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.539 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.539 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.539 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.539 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.539 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.540 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Password to authenticate with", 2023-07-21T19:06:56.540 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.540 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.540 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.540 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.540 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_port": { 2023-07-21T19:06:56.540 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_port", 2023-07-21T19:06:56.541 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:56.541 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.541 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.541 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "465", 2023-07-21T19:06:56.541 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.541 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.542 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.542 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SMTP port", 2023-07-21T19:06:56.542 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.542 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.542 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.542 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.543 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_sender": { 2023-07-21T19:06:56.543 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_sender", 2023-07-21T19:06:56.543 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.543 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.543 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.543 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.543 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.544 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.544 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.544 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SMTP envelope sender", 2023-07-21T19:06:56.544 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.544 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.544 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.544 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.545 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_ssl": { 2023-07-21T19:06:56.545 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_ssl", 2023-07-21T19:06:56.545 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.545 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.545 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.545 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:56.545 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.546 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.546 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.546 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use SSL to connect to SMTP server", 2023-07-21T19:06:56.546 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.546 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.546 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.546 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.547 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_user": { 2023-07-21T19:06:56.547 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_user", 2023-07-21T19:06:56.547 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.547 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.547 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.547 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.547 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.548 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.548 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.548 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "User to authenticate as", 2023-07-21T19:06:56.548 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.548 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.549 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.549 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.549 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.549 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.549 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:56.549 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "balancer", 2023-07-21T19:06:56.550 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:56.550 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:56.550 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:56.550 INFO:teuthology.orchestra.run.smithi039.stdout: "active": { 2023-07-21T19:06:56.550 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "active", 2023-07-21T19:06:56.550 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.550 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.551 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.551 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:56.551 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.551 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.551 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.551 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "automatically balance PGs across cluster", 2023-07-21T19:06:56.551 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.551 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.552 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.552 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.552 INFO:teuthology.orchestra.run.smithi039.stdout: "begin_time": { 2023-07-21T19:06:56.552 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "begin_time", 2023-07-21T19:06:56.552 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.552 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.552 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.553 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0000", 2023-07-21T19:06:56.553 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.553 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.553 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.553 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "beginning time of day to automatically balance", 2023-07-21T19:06:56.553 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-21T19:06:56.553 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.554 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.554 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.554 INFO:teuthology.orchestra.run.smithi039.stdout: "begin_weekday": { 2023-07-21T19:06:56.554 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "begin_weekday", 2023-07-21T19:06:56.554 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2023-07-21T19:06:56.554 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.554 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.555 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2023-07-21T19:06:56.555 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "0", 2023-07-21T19:06:56.555 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "6", 2023-07-21T19:06:56.555 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.555 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-07-21T19:06:56.555 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-21T19:06:56.555 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.555 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.556 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.556 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_compat_max_iterations": { 2023-07-21T19:06:56.556 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crush_compat_max_iterations", 2023-07-21T19:06:56.556 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2023-07-21T19:06:56.556 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.556 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.556 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "25", 2023-07-21T19:06:56.556 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2023-07-21T19:06:56.557 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "250", 2023-07-21T19:06:56.557 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.557 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-07-21T19:06:56.557 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.557 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.557 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.557 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.558 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_compat_metrics": { 2023-07-21T19:06:56.558 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crush_compat_metrics", 2023-07-21T19:06:56.558 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.558 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.558 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.558 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "pgs,objects,bytes", 2023-07-21T19:06:56.558 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.558 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.559 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.559 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-07-21T19:06:56.559 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-07-21T19:06:56.559 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.559 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.559 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.559 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_compat_step": { 2023-07-21T19:06:56.560 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crush_compat_step", 2023-07-21T19:06:56.560 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2023-07-21T19:06:56.560 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.560 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.560 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0.5", 2023-07-21T19:06:56.560 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "0.001", 2023-07-21T19:06:56.560 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "0.999", 2023-07-21T19:06:56.560 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.561 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "aggressiveness of optimization", 2023-07-21T19:06:56.561 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-07-21T19:06:56.561 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.561 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.561 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.561 INFO:teuthology.orchestra.run.smithi039.stdout: "end_time": { 2023-07-21T19:06:56.561 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "end_time", 2023-07-21T19:06:56.561 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.562 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.562 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.562 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "2359", 2023-07-21T19:06:56.562 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.562 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.562 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.562 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "ending time of day to automatically balance", 2023-07-21T19:06:56.562 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-21T19:06:56.563 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.563 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.563 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.563 INFO:teuthology.orchestra.run.smithi039.stdout: "end_weekday": { 2023-07-21T19:06:56.563 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "end_weekday", 2023-07-21T19:06:56.563 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2023-07-21T19:06:56.563 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.564 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.564 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2023-07-21T19:06:56.564 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "0", 2023-07-21T19:06:56.564 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "6", 2023-07-21T19:06:56.564 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.564 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-07-21T19:06:56.564 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-21T19:06:56.564 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.565 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.565 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.565 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:56.565 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:56.565 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.566 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.566 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.566 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.566 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.566 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.566 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.566 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.567 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.567 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.567 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.567 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.567 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.567 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.567 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.567 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.568 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.568 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.568 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.568 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:56.568 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:56.568 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.568 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.569 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.569 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.569 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.569 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.569 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.569 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.569 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.569 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.570 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.570 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.570 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:56.570 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:56.570 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.570 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.570 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.571 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:56.571 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.571 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.571 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.571 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.571 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.571 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.571 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.572 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.572 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.572 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.572 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.572 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.573 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.573 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.573 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.573 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:56.573 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:56.573 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.573 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.573 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.574 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.574 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.574 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.574 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.574 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.574 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.574 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.574 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.575 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.575 INFO:teuthology.orchestra.run.smithi039.stdout: "min_score": { 2023-07-21T19:06:56.575 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "min_score", 2023-07-21T19:06:56.575 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2023-07-21T19:06:56.575 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.575 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.575 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2023-07-21T19:06:56.575 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.576 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.576 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.576 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-07-21T19:06:56.576 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.576 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.576 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.576 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.576 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": { 2023-07-21T19:06:56.577 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mode", 2023-07-21T19:06:56.577 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.577 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.577 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.577 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "upmap", 2023-07-21T19:06:56.577 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.577 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.577 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.578 INFO:teuthology.orchestra.run.smithi039.stdout: "crush-compat", 2023-07-21T19:06:56.578 INFO:teuthology.orchestra.run.smithi039.stdout: "none", 2023-07-21T19:06:56.578 INFO:teuthology.orchestra.run.smithi039.stdout: "upmap" 2023-07-21T19:06:56.578 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.578 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Balancer mode", 2023-07-21T19:06:56.578 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.578 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.579 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.579 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.579 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_ids": { 2023-07-21T19:06:56.579 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pool_ids", 2023-07-21T19:06:56.579 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.579 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.579 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.579 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.580 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.580 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.580 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.580 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-07-21T19:06:56.580 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.580 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.580 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.580 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.581 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2023-07-21T19:06:56.581 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2023-07-21T19:06:56.581 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2023-07-21T19:06:56.581 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.581 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.581 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2023-07-21T19:06:56.581 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.581 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.582 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.582 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-07-21T19:06:56.582 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.582 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.582 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.582 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.582 INFO:teuthology.orchestra.run.smithi039.stdout: "upmap_max_deviation": { 2023-07-21T19:06:56.583 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "upmap_max_deviation", 2023-07-21T19:06:56.583 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:56.583 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.583 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.583 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2023-07-21T19:06:56.584 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2023-07-21T19:06:56.584 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.584 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.584 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "deviation below which no optimization is attempted", 2023-07-21T19:06:56.584 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-07-21T19:06:56.584 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.584 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.585 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.585 INFO:teuthology.orchestra.run.smithi039.stdout: "upmap_max_optimizations": { 2023-07-21T19:06:56.585 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "upmap_max_optimizations", 2023-07-21T19:06:56.585 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2023-07-21T19:06:56.585 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.585 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.585 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2023-07-21T19:06:56.586 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.586 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.586 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.586 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-07-21T19:06:56.586 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.586 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.586 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.587 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.587 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.587 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.587 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:56.587 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cephadm", 2023-07-21T19:06:56.587 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:56.587 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:56.587 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:56.588 INFO:teuthology.orchestra.run.smithi039.stdout: "agent_down_multiplier": { 2023-07-21T19:06:56.588 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "agent_down_multiplier", 2023-07-21T19:06:56.588 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2023-07-21T19:06:56.588 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.588 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.588 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "3.0", 2023-07-21T19:06:56.588 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.589 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.589 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.589 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-07-21T19:06:56.589 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.589 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.589 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.589 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.590 INFO:teuthology.orchestra.run.smithi039.stdout: "agent_refresh_rate": { 2023-07-21T19:06:56.590 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "agent_refresh_rate", 2023-07-21T19:06:56.590 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2023-07-21T19:06:56.590 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.590 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.590 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "20", 2023-07-21T19:06:56.590 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.591 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.591 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.591 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-07-21T19:06:56.591 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.591 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.591 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.591 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.591 INFO:teuthology.orchestra.run.smithi039.stdout: "agent_starting_port": { 2023-07-21T19:06:56.592 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "agent_starting_port", 2023-07-21T19:06:56.592 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:56.592 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.592 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.592 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "4721", 2023-07-21T19:06:56.593 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.593 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.593 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.593 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-07-21T19:06:56.593 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.593 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.593 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.593 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.594 INFO:teuthology.orchestra.run.smithi039.stdout: "alertmanager_web_password": { 2023-07-21T19:06:56.594 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "alertmanager_web_password", 2023-07-21T19:06:56.594 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.594 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.594 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.594 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "admin", 2023-07-21T19:06:56.594 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.595 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.595 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.595 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Alertmanager web password", 2023-07-21T19:06:56.595 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.595 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.595 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.595 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.596 INFO:teuthology.orchestra.run.smithi039.stdout: "alertmanager_web_user": { 2023-07-21T19:06:56.596 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "alertmanager_web_user", 2023-07-21T19:06:56.596 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.596 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.596 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.596 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "admin", 2023-07-21T19:06:56.596 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.597 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.597 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.597 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Alertmanager web user", 2023-07-21T19:06:56.597 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.597 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.597 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.597 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.597 INFO:teuthology.orchestra.run.smithi039.stdout: "allow_ptrace": { 2023-07-21T19:06:56.598 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "allow_ptrace", 2023-07-21T19:06:56.598 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.598 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.598 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.598 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.598 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.599 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.599 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.599 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-07-21T19:06:56.599 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-07-21T19:06:56.599 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.599 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.599 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.600 INFO:teuthology.orchestra.run.smithi039.stdout: "autotune_interval": { 2023-07-21T19:06:56.600 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "autotune_interval", 2023-07-21T19:06:56.600 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2023-07-21T19:06:56.600 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.600 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.600 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2023-07-21T19:06:56.600 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.601 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.601 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.601 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to autotune daemon memory", 2023-07-21T19:06:56.601 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.601 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.601 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.601 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.601 INFO:teuthology.orchestra.run.smithi039.stdout: "autotune_memory_target_ratio": { 2023-07-21T19:06:56.602 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "autotune_memory_target_ratio", 2023-07-21T19:06:56.602 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2023-07-21T19:06:56.602 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.602 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.602 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0.7", 2023-07-21T19:06:56.602 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.602 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.602 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.603 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-07-21T19:06:56.603 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.603 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.603 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.603 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.603 INFO:teuthology.orchestra.run.smithi039.stdout: "cgroups_split": { 2023-07-21T19:06:56.603 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cgroups_split", 2023-07-21T19:06:56.604 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.604 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.604 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.604 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:56.604 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.604 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.604 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.604 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-07-21T19:06:56.605 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.605 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.605 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.605 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.605 INFO:teuthology.orchestra.run.smithi039.stdout: "config_checks_enabled": { 2023-07-21T19:06:56.605 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "config_checks_enabled", 2023-07-21T19:06:56.605 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.605 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.606 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.606 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.606 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.606 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.606 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.606 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-07-21T19:06:56.606 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.606 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.607 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.607 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.607 INFO:teuthology.orchestra.run.smithi039.stdout: "config_dashboard": { 2023-07-21T19:06:56.607 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "config_dashboard", 2023-07-21T19:06:56.607 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.607 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.607 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.608 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:56.608 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.608 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.608 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.608 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-07-21T19:06:56.608 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.608 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.608 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.609 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.609 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_alertmanager": { 2023-07-21T19:06:56.609 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_alertmanager", 2023-07-21T19:06:56.609 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.609 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.609 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.609 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-07-21T19:06:56.610 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.610 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.610 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.610 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2023-07-21T19:06:56.610 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.610 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.610 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.610 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.611 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_base": { 2023-07-21T19:06:56.611 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_base", 2023-07-21T19:06:56.611 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.611 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.611 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.611 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/ceph", 2023-07-21T19:06:56.611 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.611 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.612 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.612 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Container image name, without the tag", 2023-07-21T19:06:56.612 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.612 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.613 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.613 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.613 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_elasticsearch": { 2023-07-21T19:06:56.613 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_elasticsearch", 2023-07-21T19:06:56.613 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.613 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.613 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.614 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-07-21T19:06:56.614 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.614 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.614 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.614 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "elasticsearch container image", 2023-07-21T19:06:56.614 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.614 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.615 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.615 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.615 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_grafana": { 2023-07-21T19:06:56.615 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_grafana", 2023-07-21T19:06:56.615 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.615 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.615 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.615 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-07-21T19:06:56.616 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.616 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.616 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.616 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2023-07-21T19:06:56.616 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.616 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.617 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.617 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.617 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_haproxy": { 2023-07-21T19:06:56.617 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_haproxy", 2023-07-21T19:06:56.617 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.617 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.617 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.617 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-07-21T19:06:56.618 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.618 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.618 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.618 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "HAproxy container image", 2023-07-21T19:06:56.618 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.618 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.618 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.618 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.619 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_jaeger_agent": { 2023-07-21T19:06:56.619 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_jaeger_agent", 2023-07-21T19:06:56.619 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.619 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.619 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.619 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-07-21T19:06:56.619 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.620 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.620 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.620 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Jaeger agent container image", 2023-07-21T19:06:56.620 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.620 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.620 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.620 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.621 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_jaeger_collector": { 2023-07-21T19:06:56.621 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_jaeger_collector", 2023-07-21T19:06:56.621 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.621 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.621 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.621 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-07-21T19:06:56.621 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.621 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.622 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.622 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Jaeger collector container image", 2023-07-21T19:06:56.622 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.622 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.622 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.622 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.622 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_jaeger_query": { 2023-07-21T19:06:56.622 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_jaeger_query", 2023-07-21T19:06:56.623 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.623 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.623 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.623 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-07-21T19:06:56.623 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.623 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.623 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.623 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Jaeger query container image", 2023-07-21T19:06:56.624 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.624 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.624 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.624 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.624 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_keepalived": { 2023-07-21T19:06:56.624 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_keepalived", 2023-07-21T19:06:56.624 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.624 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.625 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.625 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-07-21T19:06:56.625 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.625 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.625 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.625 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Keepalived container image", 2023-07-21T19:06:56.625 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.626 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.626 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.626 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.626 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_loki": { 2023-07-21T19:06:56.626 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_loki", 2023-07-21T19:06:56.626 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.626 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.626 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.627 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-07-21T19:06:56.627 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.627 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.627 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.627 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Loki container image", 2023-07-21T19:06:56.627 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.627 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.628 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.628 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.628 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_node_exporter": { 2023-07-21T19:06:56.628 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_node_exporter", 2023-07-21T19:06:56.628 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.628 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.628 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.628 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-07-21T19:06:56.629 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.629 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.629 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.629 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2023-07-21T19:06:56.629 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.629 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.629 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.629 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.630 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_prometheus": { 2023-07-21T19:06:56.630 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_prometheus", 2023-07-21T19:06:56.630 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.630 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.630 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.630 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-07-21T19:06:56.630 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.631 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.631 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.631 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2023-07-21T19:06:56.631 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.631 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.631 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.631 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.632 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_promtail": { 2023-07-21T19:06:56.632 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_promtail", 2023-07-21T19:06:56.632 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.632 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.632 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.632 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-07-21T19:06:56.632 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.633 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.633 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.633 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Promtail container image", 2023-07-21T19:06:56.633 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.633 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.633 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.633 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.634 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_snmp_gateway": { 2023-07-21T19:06:56.634 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_snmp_gateway", 2023-07-21T19:06:56.634 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.634 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.634 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.634 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-07-21T19:06:56.634 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.635 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.635 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.635 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SNMP Gateway container image", 2023-07-21T19:06:56.635 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.635 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.635 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.635 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.636 INFO:teuthology.orchestra.run.smithi039.stdout: "container_init": { 2023-07-21T19:06:56.636 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_init", 2023-07-21T19:06:56.636 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.636 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.636 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.636 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:56.636 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.637 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.637 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.637 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Run podman/docker with `--init`", 2023-07-21T19:06:56.637 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.637 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.637 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.637 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.638 INFO:teuthology.orchestra.run.smithi039.stdout: "daemon_cache_timeout": { 2023-07-21T19:06:56.638 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "daemon_cache_timeout", 2023-07-21T19:06:56.638 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2023-07-21T19:06:56.638 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.638 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.638 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2023-07-21T19:06:56.638 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.639 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.639 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.639 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-07-21T19:06:56.639 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.639 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.639 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.639 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.640 INFO:teuthology.orchestra.run.smithi039.stdout: "default_cephadm_command_timeout": { 2023-07-21T19:06:56.640 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "default_cephadm_command_timeout", 2023-07-21T19:06:56.640 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2023-07-21T19:06:56.640 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.640 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.640 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "900", 2023-07-21T19:06:56.640 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.640 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.641 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.641 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-07-21T19:06:56.641 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.641 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.641 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.641 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.641 INFO:teuthology.orchestra.run.smithi039.stdout: "default_registry": { 2023-07-21T19:06:56.642 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "default_registry", 2023-07-21T19:06:56.642 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.642 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.642 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.642 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io", 2023-07-21T19:06:56.642 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.642 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.642 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.643 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-07-21T19:06:56.643 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.643 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.643 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.643 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.643 INFO:teuthology.orchestra.run.smithi039.stdout: "device_cache_timeout": { 2023-07-21T19:06:56.644 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "device_cache_timeout", 2023-07-21T19:06:56.644 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2023-07-21T19:06:56.644 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.644 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.644 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "1800", 2023-07-21T19:06:56.644 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.644 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.645 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.645 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "seconds to cache device inventory", 2023-07-21T19:06:56.645 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.645 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.645 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.645 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.645 INFO:teuthology.orchestra.run.smithi039.stdout: "device_enhanced_scan": { 2023-07-21T19:06:56.645 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "device_enhanced_scan", 2023-07-21T19:06:56.646 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.646 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.646 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.646 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.646 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.646 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.646 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.646 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use libstoragemgmt during device scans", 2023-07-21T19:06:56.647 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.647 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.647 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.647 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.647 INFO:teuthology.orchestra.run.smithi039.stdout: "facts_cache_timeout": { 2023-07-21T19:06:56.647 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "facts_cache_timeout", 2023-07-21T19:06:56.647 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2023-07-21T19:06:56.647 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.648 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.648 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2023-07-21T19:06:56.648 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.648 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.648 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.648 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "seconds to cache host facts data", 2023-07-21T19:06:56.648 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.649 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.649 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.649 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.649 INFO:teuthology.orchestra.run.smithi039.stdout: "host_check_interval": { 2023-07-21T19:06:56.649 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "host_check_interval", 2023-07-21T19:06:56.649 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2023-07-21T19:06:56.649 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.649 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.650 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2023-07-21T19:06:56.650 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.650 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.650 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.650 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to perform a host check", 2023-07-21T19:06:56.650 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.650 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.650 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.651 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.651 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:56.651 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:56.651 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.651 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.651 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.651 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.651 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.652 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.652 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.652 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.652 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.652 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.652 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.652 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.653 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.653 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.653 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.653 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.653 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.653 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.653 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.653 INFO:teuthology.orchestra.run.smithi039.stdout: "log_refresh_metadata": { 2023-07-21T19:06:56.654 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_refresh_metadata", 2023-07-21T19:06:56.654 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.654 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.654 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.654 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.654 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.654 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.655 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.655 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-07-21T19:06:56.655 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.655 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.655 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.655 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.655 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:56.656 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:56.656 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.656 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.656 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.656 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:56.656 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.656 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.657 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.657 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-07-21T19:06:56.657 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.657 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.657 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.657 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.657 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:56.657 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:56.658 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.658 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.658 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.658 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:56.658 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.658 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.658 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.659 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.659 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.659 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.659 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.659 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.659 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.659 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.660 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.660 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.660 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.660 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.660 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.660 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:56.660 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:56.660 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.661 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.661 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.661 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.661 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.661 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.661 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.661 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.661 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.662 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.662 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.662 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.662 INFO:teuthology.orchestra.run.smithi039.stdout: "manage_etc_ceph_ceph_conf": { 2023-07-21T19:06:56.662 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-07-21T19:06:56.662 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.662 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.662 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.663 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.663 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.663 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.663 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.663 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-07-21T19:06:56.663 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.664 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.664 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.664 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.664 INFO:teuthology.orchestra.run.smithi039.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-07-21T19:06:56.664 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-07-21T19:06:56.664 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.664 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.665 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.665 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "*", 2023-07-21T19:06:56.665 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.665 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.665 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.665 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-07-21T19:06:56.665 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.666 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.666 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.666 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.666 INFO:teuthology.orchestra.run.smithi039.stdout: "max_count_per_host": { 2023-07-21T19:06:56.666 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_count_per_host", 2023-07-21T19:06:56.666 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:56.666 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.666 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.667 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2023-07-21T19:06:56.667 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.667 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.667 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.667 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "max number of daemons per service per host", 2023-07-21T19:06:56.667 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.667 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.667 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.668 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.668 INFO:teuthology.orchestra.run.smithi039.stdout: "max_osd_draining_count": { 2023-07-21T19:06:56.668 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_osd_draining_count", 2023-07-21T19:06:56.668 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:56.668 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.668 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.668 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2023-07-21T19:06:56.668 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.669 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.669 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.669 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-07-21T19:06:56.669 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.669 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.669 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.669 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.670 INFO:teuthology.orchestra.run.smithi039.stdout: "migration_current": { 2023-07-21T19:06:56.670 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "migration_current", 2023-07-21T19:06:56.670 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:56.670 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.670 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.670 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.670 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.671 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.671 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.671 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "internal - do not modify", 2023-07-21T19:06:56.671 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.671 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.671 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.672 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.672 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": { 2023-07-21T19:06:56.672 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mode", 2023-07-21T19:06:56.672 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.672 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.672 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.672 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "root", 2023-07-21T19:06:56.672 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.673 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.673 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.673 INFO:teuthology.orchestra.run.smithi039.stdout: "cephadm-package", 2023-07-21T19:06:56.673 INFO:teuthology.orchestra.run.smithi039.stdout: "root" 2023-07-21T19:06:56.673 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.673 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "mode for remote execution of cephadm", 2023-07-21T19:06:56.673 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.674 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.674 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.674 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.674 INFO:teuthology.orchestra.run.smithi039.stdout: "prometheus_alerts_path": { 2023-07-21T19:06:56.674 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "prometheus_alerts_path", 2023-07-21T19:06:56.674 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.674 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.675 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.675 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-07-21T19:06:56.675 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.675 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.675 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.675 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-07-21T19:06:56.675 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.676 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.676 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.676 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.676 INFO:teuthology.orchestra.run.smithi039.stdout: "prometheus_web_password": { 2023-07-21T19:06:56.676 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "prometheus_web_password", 2023-07-21T19:06:56.676 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.676 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.677 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.677 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "admin", 2023-07-21T19:06:56.677 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.677 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.677 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.677 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus web password", 2023-07-21T19:06:56.677 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.678 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.678 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.678 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.678 INFO:teuthology.orchestra.run.smithi039.stdout: "prometheus_web_user": { 2023-07-21T19:06:56.678 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "prometheus_web_user", 2023-07-21T19:06:56.678 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.679 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.679 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.679 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "admin", 2023-07-21T19:06:56.679 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.679 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.679 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.679 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus web user", 2023-07-21T19:06:56.679 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.680 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.680 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.680 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.680 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_insecure": { 2023-07-21T19:06:56.680 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_insecure", 2023-07-21T19:06:56.681 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.681 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.681 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.681 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.681 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.681 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.681 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.682 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-07-21T19:06:56.682 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.682 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.682 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.682 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.682 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_password": { 2023-07-21T19:06:56.682 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_password", 2023-07-21T19:06:56.683 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.683 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.683 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.683 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.683 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.683 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.683 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.683 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-07-21T19:06:56.684 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.684 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.684 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.684 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.684 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_url": { 2023-07-21T19:06:56.684 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_url", 2023-07-21T19:06:56.685 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.685 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.685 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.685 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.685 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.685 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.686 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.686 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-07-21T19:06:56.686 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.686 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.686 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.686 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.686 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_username": { 2023-07-21T19:06:56.686 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_username", 2023-07-21T19:06:56.687 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.687 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.687 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.687 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.687 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.687 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.687 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.688 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-07-21T19:06:56.688 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.688 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.688 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.688 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.688 INFO:teuthology.orchestra.run.smithi039.stdout: "secure_monitoring_stack": { 2023-07-21T19:06:56.688 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "secure_monitoring_stack", 2023-07-21T19:06:56.689 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.689 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.689 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.689 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.689 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.689 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.689 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.689 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-07-21T19:06:56.690 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.690 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.690 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.690 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.690 INFO:teuthology.orchestra.run.smithi039.stdout: "service_discovery_port": { 2023-07-21T19:06:56.690 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "service_discovery_port", 2023-07-21T19:06:56.691 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:56.691 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.691 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.691 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8765", 2023-07-21T19:06:56.691 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.691 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.691 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.691 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "cephadm service discovery port", 2023-07-21T19:06:56.692 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.692 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.692 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.692 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.692 INFO:teuthology.orchestra.run.smithi039.stdout: "ssh_config_file": { 2023-07-21T19:06:56.692 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssh_config_file", 2023-07-21T19:06:56.693 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.693 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.693 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.693 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.693 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.693 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.693 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.694 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-07-21T19:06:56.694 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.694 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.694 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.694 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.694 INFO:teuthology.orchestra.run.smithi039.stdout: "use_agent": { 2023-07-21T19:06:56.694 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "use_agent", 2023-07-21T19:06:56.695 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.695 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.695 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.695 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.695 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.695 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.695 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.696 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-07-21T19:06:56.696 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.696 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.696 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.696 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.696 INFO:teuthology.orchestra.run.smithi039.stdout: "use_repo_digest": { 2023-07-21T19:06:56.696 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "use_repo_digest", 2023-07-21T19:06:56.697 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.697 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.697 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.697 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:56.697 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.697 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.697 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.697 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-07-21T19:06:56.698 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.698 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.698 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.698 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.698 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_on_failed_host_check": { 2023-07-21T19:06:56.698 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_on_failed_host_check", 2023-07-21T19:06:56.698 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.698 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.699 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.699 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:56.699 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.699 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.699 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.699 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise a health warning if the host check fails", 2023-07-21T19:06:56.699 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.699 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.700 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.700 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.700 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_on_stray_daemons": { 2023-07-21T19:06:56.700 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_on_stray_daemons", 2023-07-21T19:06:56.700 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.700 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.700 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.701 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:56.701 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.701 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.701 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.701 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-07-21T19:06:56.701 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.701 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.702 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.702 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.702 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_on_stray_hosts": { 2023-07-21T19:06:56.702 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_on_stray_hosts", 2023-07-21T19:06:56.702 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.702 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.702 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.702 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:56.703 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.703 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.703 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.703 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-07-21T19:06:56.703 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.703 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.703 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.704 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.704 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.704 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.704 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:56.704 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crash", 2023-07-21T19:06:56.704 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:56.704 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:56.704 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:56.705 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:56.705 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:56.705 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.705 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.705 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.705 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.705 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.706 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.706 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.706 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.706 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.706 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.706 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.706 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.706 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.707 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.707 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.707 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.707 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.707 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.707 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.707 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:56.707 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:56.708 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.708 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.708 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.708 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.708 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.708 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.708 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.709 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.709 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.709 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.709 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.709 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.709 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:56.709 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:56.709 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.710 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.710 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.710 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:56.710 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.710 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.710 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.710 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.711 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.711 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.711 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.711 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.711 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.711 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.711 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.711 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.712 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.712 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.712 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.712 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:56.712 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:56.712 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.712 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.713 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.713 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.713 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.713 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.713 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.713 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.713 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.713 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.714 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.714 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.714 INFO:teuthology.orchestra.run.smithi039.stdout: "retain_interval": { 2023-07-21T19:06:56.714 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "retain_interval", 2023-07-21T19:06:56.714 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2023-07-21T19:06:56.714 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.714 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.715 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "31536000", 2023-07-21T19:06:56.715 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.715 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.715 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.715 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how long to retain crashes before pruning them", 2023-07-21T19:06:56.715 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.715 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.715 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.716 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.716 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_recent_interval": { 2023-07-21T19:06:56.716 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_recent_interval", 2023-07-21T19:06:56.716 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2023-07-21T19:06:56.716 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.716 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.716 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "1209600", 2023-07-21T19:06:56.717 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.717 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.717 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.717 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "time interval in which to warn about recent crashes", 2023-07-21T19:06:56.717 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.717 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.717 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.718 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.718 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.718 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.718 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:56.718 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "dashboard", 2023-07-21T19:06:56.718 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:56.719 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:56.719 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:56.719 INFO:teuthology.orchestra.run.smithi039.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-07-21T19:06:56.719 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-07-21T19:06:56.719 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:56.719 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.719 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.720 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2023-07-21T19:06:56.720 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.720 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.720 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.720 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.720 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.720 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.721 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.721 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.721 INFO:teuthology.orchestra.run.smithi039.stdout: "ALERTMANAGER_API_HOST": { 2023-07-21T19:06:56.721 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ALERTMANAGER_API_HOST", 2023-07-21T19:06:56.721 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.721 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.722 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.722 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.722 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.722 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.722 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.723 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.723 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.723 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.723 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.723 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.723 INFO:teuthology.orchestra.run.smithi039.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-07-21T19:06:56.723 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-07-21T19:06:56.724 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.724 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.724 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.724 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:56.724 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.724 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.725 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.725 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.725 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.725 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.725 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.725 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.725 INFO:teuthology.orchestra.run.smithi039.stdout: "AUDIT_API_ENABLED": { 2023-07-21T19:06:56.726 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "AUDIT_API_ENABLED", 2023-07-21T19:06:56.726 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.726 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.726 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.726 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.727 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.727 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.727 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.727 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.727 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.727 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.728 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.728 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.728 INFO:teuthology.orchestra.run.smithi039.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-07-21T19:06:56.728 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-07-21T19:06:56.728 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.728 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.728 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.728 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:56.729 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.729 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.729 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.729 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.729 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.729 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.729 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.730 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.730 INFO:teuthology.orchestra.run.smithi039.stdout: "ENABLE_BROWSABLE_API": { 2023-07-21T19:06:56.730 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ENABLE_BROWSABLE_API", 2023-07-21T19:06:56.730 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.730 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.730 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.730 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:56.730 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.731 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.731 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.731 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.731 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.731 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.731 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.731 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.732 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-07-21T19:06:56.732 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-07-21T19:06:56.732 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.732 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.732 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.732 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:56.732 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.732 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.733 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.733 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.733 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.733 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.733 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.733 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.733 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-07-21T19:06:56.734 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-07-21T19:06:56.734 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.734 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.734 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.734 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:56.734 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.734 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.735 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.735 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.735 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.735 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.735 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.735 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.735 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-07-21T19:06:56.736 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-07-21T19:06:56.736 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.736 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.736 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.736 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:56.736 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.736 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.736 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.737 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.737 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.737 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.737 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.737 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.737 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-07-21T19:06:56.737 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-07-21T19:06:56.737 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.738 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.738 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.738 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:56.738 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.738 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.738 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.738 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.739 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.739 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.739 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.739 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.739 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_NFS": { 2023-07-21T19:06:56.739 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-07-21T19:06:56.739 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.739 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.740 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.740 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:56.740 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.740 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.740 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.740 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.740 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.740 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.741 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.741 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.741 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_RBD": { 2023-07-21T19:06:56.741 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-07-21T19:06:56.741 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.741 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.741 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.741 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:56.742 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.742 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.742 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.742 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.742 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.742 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.742 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.742 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.743 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_RGW": { 2023-07-21T19:06:56.743 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-07-21T19:06:56.743 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.743 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.743 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.743 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:56.743 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.743 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.744 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.744 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.744 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.744 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.744 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.744 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.744 INFO:teuthology.orchestra.run.smithi039.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-07-21T19:06:56.745 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-07-21T19:06:56.745 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.745 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.745 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.745 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.745 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.745 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.746 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.746 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.746 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.746 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.746 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.746 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.747 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_PASSWORD": { 2023-07-21T19:06:56.747 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_PASSWORD", 2023-07-21T19:06:56.747 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.747 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.747 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.747 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "admin", 2023-07-21T19:06:56.747 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.747 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.748 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.748 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.748 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.748 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.748 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.748 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.748 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-07-21T19:06:56.749 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-07-21T19:06:56.749 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.749 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.749 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.749 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:56.749 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.749 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.749 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.750 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.750 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.750 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.750 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.750 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.750 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_URL": { 2023-07-21T19:06:56.750 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_URL", 2023-07-21T19:06:56.750 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.751 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.751 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.751 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.751 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.751 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.751 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.751 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.751 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.752 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.752 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.752 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.752 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_USERNAME": { 2023-07-21T19:06:56.752 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_USERNAME", 2023-07-21T19:06:56.752 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.752 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.753 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.753 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "admin", 2023-07-21T19:06:56.753 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.753 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.753 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.753 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.753 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.753 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.754 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.754 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.754 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-07-21T19:06:56.754 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-07-21T19:06:56.754 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.754 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.755 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.755 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.755 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.755 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.755 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.755 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.755 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.755 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.756 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.756 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.756 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-07-21T19:06:56.756 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-07-21T19:06:56.756 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.756 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.756 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.757 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.757 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.757 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.757 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.757 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.757 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.757 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.757 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.758 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.758 INFO:teuthology.orchestra.run.smithi039.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-07-21T19:06:56.758 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-07-21T19:06:56.758 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.758 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.758 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.759 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:56.759 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.759 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.759 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.759 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.759 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.759 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.759 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.760 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.760 INFO:teuthology.orchestra.run.smithi039.stdout: "ISSUE_TRACKER_API_KEY": { 2023-07-21T19:06:56.760 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-07-21T19:06:56.760 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.760 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.760 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.760 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.761 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.761 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.761 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.761 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.761 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.761 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.761 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.761 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.762 INFO:teuthology.orchestra.run.smithi039.stdout: "PROMETHEUS_API_HOST": { 2023-07-21T19:06:56.762 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PROMETHEUS_API_HOST", 2023-07-21T19:06:56.762 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.762 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.762 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.762 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.763 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.763 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.763 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.763 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.763 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.763 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.763 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.764 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.764 INFO:teuthology.orchestra.run.smithi039.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-07-21T19:06:56.764 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-07-21T19:06:56.764 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.764 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.764 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.764 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:56.765 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.765 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.765 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.765 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.765 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.766 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.766 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.766 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.766 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-07-21T19:06:56.766 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-07-21T19:06:56.766 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.766 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.766 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.767 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.767 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.767 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.767 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.767 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.767 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.767 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.768 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.768 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.768 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-07-21T19:06:56.768 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-07-21T19:06:56.768 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.768 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.768 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.769 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.769 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.769 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.769 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.769 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.769 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.769 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.769 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.770 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.770 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-07-21T19:06:56.770 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-07-21T19:06:56.770 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.770 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.770 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.770 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:56.771 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.771 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.771 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.771 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.771 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.771 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.771 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.771 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.772 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-07-21T19:06:56.772 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-07-21T19:06:56.772 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.772 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.772 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.772 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:56.772 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.773 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.773 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.773 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.773 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.773 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.773 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.773 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.773 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-07-21T19:06:56.774 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-07-21T19:06:56.774 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.774 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.774 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.774 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.774 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.774 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.774 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.775 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.775 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.775 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.775 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.775 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.775 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-07-21T19:06:56.775 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-07-21T19:06:56.776 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.776 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.776 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.776 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.776 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.776 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.776 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.777 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.777 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.777 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.777 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.777 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.777 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-07-21T19:06:56.777 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-07-21T19:06:56.777 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.778 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.778 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.778 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.778 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.778 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.778 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.778 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.778 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.779 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.779 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.779 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.779 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_ENABLED": { 2023-07-21T19:06:56.779 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_ENABLED", 2023-07-21T19:06:56.779 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.779 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.779 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.780 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:56.780 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.780 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.780 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.780 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.780 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.780 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.781 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.781 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.781 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-07-21T19:06:56.781 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-07-21T19:06:56.781 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.781 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.781 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.781 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-07-21T19:06:56.782 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.782 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.782 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.782 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.782 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.782 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.782 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.782 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.783 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-07-21T19:06:56.783 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-07-21T19:06:56.783 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:56.783 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.783 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.783 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2023-07-21T19:06:56.783 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.784 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.784 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.784 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.784 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.784 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.784 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.784 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.784 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-07-21T19:06:56.785 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-07-21T19:06:56.785 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:56.785 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.785 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.785 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8", 2023-07-21T19:06:56.785 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.785 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.785 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.786 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.786 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.786 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.786 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.786 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.786 INFO:teuthology.orchestra.run.smithi039.stdout: "REST_REQUESTS_TIMEOUT": { 2023-07-21T19:06:56.786 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-07-21T19:06:56.786 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:56.787 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.787 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.787 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "45", 2023-07-21T19:06:56.787 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.787 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.787 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.787 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.787 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.788 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.788 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.788 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.788 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_ACCESS_KEY": { 2023-07-21T19:06:56.788 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_ACCESS_KEY", 2023-07-21T19:06:56.788 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.788 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.789 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.789 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.789 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.789 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.789 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.789 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.789 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.789 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.790 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.790 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.790 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-07-21T19:06:56.790 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-07-21T19:06:56.790 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.790 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.790 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.790 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "admin", 2023-07-21T19:06:56.791 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.791 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.791 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.791 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.791 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.791 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.791 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.791 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.792 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_SECRET_KEY": { 2023-07-21T19:06:56.792 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_SECRET_KEY", 2023-07-21T19:06:56.792 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.792 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.792 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.792 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.792 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.793 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.793 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.793 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.793 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.793 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.793 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.793 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.794 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_SSL_VERIFY": { 2023-07-21T19:06:56.794 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_SSL_VERIFY", 2023-07-21T19:06:56.794 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.794 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.794 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.794 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:56.794 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.795 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.795 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.795 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.795 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.795 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.795 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.795 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.796 INFO:teuthology.orchestra.run.smithi039.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-07-21T19:06:56.796 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-07-21T19:06:56.796 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:56.796 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.796 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.796 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2023-07-21T19:06:56.797 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.797 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.797 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.797 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.797 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.797 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.797 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.798 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.798 INFO:teuthology.orchestra.run.smithi039.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-07-21T19:06:56.798 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-07-21T19:06:56.798 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:56.798 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.798 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.799 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2023-07-21T19:06:56.799 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.799 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.799 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.799 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.799 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.799 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.799 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.800 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.800 INFO:teuthology.orchestra.run.smithi039.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-07-21T19:06:56.800 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-07-21T19:06:56.800 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:56.800 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.800 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.800 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2023-07-21T19:06:56.801 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.801 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.801 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.801 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.801 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.801 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.801 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.801 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.802 INFO:teuthology.orchestra.run.smithi039.stdout: "cross_origin_url": { 2023-07-21T19:06:56.802 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cross_origin_url", 2023-07-21T19:06:56.802 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.802 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.802 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.802 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.802 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.802 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.803 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.803 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.803 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.803 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.803 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.803 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.803 INFO:teuthology.orchestra.run.smithi039.stdout: "crt_file": { 2023-07-21T19:06:56.803 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crt_file", 2023-07-21T19:06:56.804 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.804 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.804 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.804 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.804 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.804 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.804 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.804 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.805 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.805 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.805 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.805 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.805 INFO:teuthology.orchestra.run.smithi039.stdout: "debug": { 2023-07-21T19:06:56.805 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "debug", 2023-07-21T19:06:56.805 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.805 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.806 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.806 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.806 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.806 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.806 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.806 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Enable/disable debug options", 2023-07-21T19:06:56.806 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.807 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.807 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.807 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.807 INFO:teuthology.orchestra.run.smithi039.stdout: "jwt_token_ttl": { 2023-07-21T19:06:56.807 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "jwt_token_ttl", 2023-07-21T19:06:56.807 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:56.807 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.807 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.808 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "28800", 2023-07-21T19:06:56.808 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.808 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.808 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.808 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.808 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.808 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.809 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.809 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.809 INFO:teuthology.orchestra.run.smithi039.stdout: "key_file": { 2023-07-21T19:06:56.809 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "key_file", 2023-07-21T19:06:56.809 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.810 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.810 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.810 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.810 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.810 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.810 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.810 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.810 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.811 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.811 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.811 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.811 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:56.811 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:56.811 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.812 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.812 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.812 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.812 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.812 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.812 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.812 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.813 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.813 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.813 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.813 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.813 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.813 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.813 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.814 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.814 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.814 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.814 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.814 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:56.814 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:56.814 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.815 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.815 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.815 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.815 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.815 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.815 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.815 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.815 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.816 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.816 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.816 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.816 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:56.816 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:56.816 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.816 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.817 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.817 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:56.817 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.817 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.817 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.817 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.817 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.817 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.818 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.818 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.818 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.818 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.818 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.818 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.819 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.819 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.819 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.819 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:56.819 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:56.819 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.819 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.819 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.820 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.820 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.820 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.820 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.820 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.820 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.820 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.821 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.821 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.821 INFO:teuthology.orchestra.run.smithi039.stdout: "motd": { 2023-07-21T19:06:56.821 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "motd", 2023-07-21T19:06:56.821 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.821 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.821 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.821 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.822 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.822 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.822 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.822 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "The message of the day", 2023-07-21T19:06:56.822 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.822 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.822 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.823 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.823 INFO:teuthology.orchestra.run.smithi039.stdout: "redirect_resolve_ip_addr": { 2023-07-21T19:06:56.823 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "redirect_resolve_ip_addr", 2023-07-21T19:06:56.823 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.823 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.823 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.823 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.823 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.824 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.824 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.824 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.824 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.824 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.824 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.824 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.825 INFO:teuthology.orchestra.run.smithi039.stdout: "server_addr": { 2023-07-21T19:06:56.825 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_addr", 2023-07-21T19:06:56.825 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.825 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.825 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.825 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "::", 2023-07-21T19:06:56.825 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.825 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.826 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.826 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.826 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.826 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.826 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.826 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.826 INFO:teuthology.orchestra.run.smithi039.stdout: "server_port": { 2023-07-21T19:06:56.826 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_port", 2023-07-21T19:06:56.827 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:56.827 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.827 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.827 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8080", 2023-07-21T19:06:56.827 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.827 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.827 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.828 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.828 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.828 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.828 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.828 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.828 INFO:teuthology.orchestra.run.smithi039.stdout: "ssl": { 2023-07-21T19:06:56.829 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssl", 2023-07-21T19:06:56.829 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.829 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.829 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.829 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:56.829 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.829 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.829 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.830 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.830 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.830 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.830 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.830 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.830 INFO:teuthology.orchestra.run.smithi039.stdout: "ssl_server_port": { 2023-07-21T19:06:56.830 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssl_server_port", 2023-07-21T19:06:56.830 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:56.831 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.831 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.831 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8443", 2023-07-21T19:06:56.831 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.831 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.831 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.831 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.832 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.832 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.832 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.832 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.832 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_behaviour": { 2023-07-21T19:06:56.832 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_behaviour", 2023-07-21T19:06:56.832 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.832 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.833 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.833 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "redirect", 2023-07-21T19:06:56.833 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.833 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.833 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.833 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.833 INFO:teuthology.orchestra.run.smithi039.stdout: "redirect" 2023-07-21T19:06:56.833 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.834 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.834 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.834 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.834 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.834 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.834 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_error_status_code": { 2023-07-21T19:06:56.834 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_error_status_code", 2023-07-21T19:06:56.834 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:56.835 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.835 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.835 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "500", 2023-07-21T19:06:56.835 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "400", 2023-07-21T19:06:56.835 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "599", 2023-07-21T19:06:56.835 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.835 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.835 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.836 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.836 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.836 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.836 INFO:teuthology.orchestra.run.smithi039.stdout: "url_prefix": { 2023-07-21T19:06:56.836 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "url_prefix", 2023-07-21T19:06:56.836 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.836 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.837 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.837 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.837 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.837 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.837 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.837 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.837 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.837 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.838 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.838 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.838 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.838 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.838 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:56.838 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "devicehealth", 2023-07-21T19:06:56.838 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:56.838 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:56.839 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:56.839 INFO:teuthology.orchestra.run.smithi039.stdout: "enable_monitoring": { 2023-07-21T19:06:56.839 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enable_monitoring", 2023-07-21T19:06:56.839 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.839 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.839 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.839 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:56.840 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.840 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.840 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.840 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "monitor device health metrics", 2023-07-21T19:06:56.840 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.840 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.840 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.840 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.841 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:56.841 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:56.841 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.841 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.841 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.841 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.841 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.842 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.842 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.842 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.842 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.842 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.842 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.842 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.842 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.843 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.843 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.843 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.843 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.843 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.843 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.843 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:56.843 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:56.844 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.844 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.844 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.844 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.844 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.844 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.844 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.845 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.845 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.845 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.845 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.845 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.845 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:56.845 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:56.845 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.846 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.846 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.846 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:56.846 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.846 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.846 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.846 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.846 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.847 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.847 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.847 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.847 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.847 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.847 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.847 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.848 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.848 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.848 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.848 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:56.848 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:56.848 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.848 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.848 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.849 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.849 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.849 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.849 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.849 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.849 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.849 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.850 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.850 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.850 INFO:teuthology.orchestra.run.smithi039.stdout: "mark_out_threshold": { 2023-07-21T19:06:56.850 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mark_out_threshold", 2023-07-21T19:06:56.850 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2023-07-21T19:06:56.850 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.850 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.850 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "2419200", 2023-07-21T19:06:56.851 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.851 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.851 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.851 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-07-21T19:06:56.851 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.851 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.851 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.851 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.852 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_name": { 2023-07-21T19:06:56.852 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pool_name", 2023-07-21T19:06:56.852 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.852 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.852 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.852 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "device_health_metrics", 2023-07-21T19:06:56.852 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.853 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.853 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.853 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "name of pool in which to store device health metrics", 2023-07-21T19:06:56.853 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.853 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.853 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.853 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.853 INFO:teuthology.orchestra.run.smithi039.stdout: "retention_period": { 2023-07-21T19:06:56.854 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "retention_period", 2023-07-21T19:06:56.854 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2023-07-21T19:06:56.854 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.854 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.854 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "15552000", 2023-07-21T19:06:56.854 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.854 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.855 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.855 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how long to retain device health metrics", 2023-07-21T19:06:56.855 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.855 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.855 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.855 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.855 INFO:teuthology.orchestra.run.smithi039.stdout: "scrape_frequency": { 2023-07-21T19:06:56.855 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "scrape_frequency", 2023-07-21T19:06:56.856 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2023-07-21T19:06:56.856 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.856 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.856 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "86400", 2023-07-21T19:06:56.856 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.856 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.856 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.857 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to scrape device health metrics", 2023-07-21T19:06:56.857 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.857 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.857 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.857 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.857 INFO:teuthology.orchestra.run.smithi039.stdout: "self_heal": { 2023-07-21T19:06:56.857 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "self_heal", 2023-07-21T19:06:56.857 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.858 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.858 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.858 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:56.858 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.858 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.858 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.858 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-07-21T19:06:56.858 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.859 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.859 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.859 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.859 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2023-07-21T19:06:56.859 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2023-07-21T19:06:56.859 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2023-07-21T19:06:56.859 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.859 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.860 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2023-07-21T19:06:56.860 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.860 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.860 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.860 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to wake up and check device health", 2023-07-21T19:06:56.860 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.860 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.861 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.861 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.861 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_threshold": { 2023-07-21T19:06:56.861 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_threshold", 2023-07-21T19:06:56.861 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2023-07-21T19:06:56.861 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.861 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.861 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "7257600", 2023-07-21T19:06:56.862 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.862 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.862 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.862 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-07-21T19:06:56.862 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.862 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.862 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.862 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.863 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.863 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.863 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:56.863 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "diskprediction_local", 2023-07-21T19:06:56.863 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:56.864 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:56.864 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:56.864 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:56.864 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:56.864 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.864 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.865 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.865 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.865 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.865 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.865 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.865 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.865 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.866 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.866 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.866 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.866 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.866 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.866 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.866 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.867 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.867 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.867 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.867 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:56.867 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:56.867 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.867 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.867 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.868 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.868 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.868 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.868 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.868 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.868 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.868 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.869 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.869 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.869 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:56.869 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:56.869 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.869 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.869 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.870 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:56.870 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.870 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.870 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.870 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.870 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.870 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.870 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.871 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.871 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.871 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.871 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.871 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.871 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.871 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.871 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.872 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:56.872 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:56.872 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.872 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.872 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.872 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.872 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.873 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.873 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.873 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.873 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.873 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.873 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.873 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.874 INFO:teuthology.orchestra.run.smithi039.stdout: "predict_interval": { 2023-07-21T19:06:56.874 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "predict_interval", 2023-07-21T19:06:56.874 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.874 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.874 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.874 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "86400", 2023-07-21T19:06:56.874 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.874 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.875 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.875 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.875 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.875 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.875 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.875 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.875 INFO:teuthology.orchestra.run.smithi039.stdout: "predictor_model": { 2023-07-21T19:06:56.875 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "predictor_model", 2023-07-21T19:06:56.876 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.876 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.876 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.876 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "prophetstor", 2023-07-21T19:06:56.876 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.876 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.876 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.877 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.877 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.877 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.877 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.877 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.877 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2023-07-21T19:06:56.877 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2023-07-21T19:06:56.877 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.878 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.878 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.878 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2023-07-21T19:06:56.878 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.878 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.878 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.879 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.879 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.879 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.879 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.879 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.879 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.879 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.879 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:56.880 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "influx", 2023-07-21T19:06:56.880 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": false, 2023-07-21T19:06:56.880 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "influxdb python module not found", 2023-07-21T19:06:56.880 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:56.880 INFO:teuthology.orchestra.run.smithi039.stdout: "batch_size": { 2023-07-21T19:06:56.880 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "batch_size", 2023-07-21T19:06:56.881 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:56.881 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.881 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.881 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5000", 2023-07-21T19:06:56.881 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.881 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.882 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.882 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-07-21T19:06:56.882 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.882 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.882 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.882 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.883 INFO:teuthology.orchestra.run.smithi039.stdout: "database": { 2023-07-21T19:06:56.883 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "database", 2023-07-21T19:06:56.883 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.883 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.883 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.883 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "ceph", 2023-07-21T19:06:56.883 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.884 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.884 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.884 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-07-21T19:06:56.884 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.884 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.884 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.885 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.885 INFO:teuthology.orchestra.run.smithi039.stdout: "hostname": { 2023-07-21T19:06:56.885 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "hostname", 2023-07-21T19:06:56.885 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.885 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.885 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.886 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.886 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.886 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.886 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.886 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "InfluxDB server hostname", 2023-07-21T19:06:56.886 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.886 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.887 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.887 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.887 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2023-07-21T19:06:56.887 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2023-07-21T19:06:56.887 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2023-07-21T19:06:56.887 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.888 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.888 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "30", 2023-07-21T19:06:56.888 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "5", 2023-07-21T19:06:56.888 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.888 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.888 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-07-21T19:06:56.888 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.889 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.889 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.889 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.889 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:56.889 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:56.889 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.889 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.889 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.890 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.890 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.890 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.890 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.890 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.890 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.890 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.890 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.891 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.891 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.891 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.891 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.891 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.891 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.891 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.891 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.892 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:56.892 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:56.892 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.892 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.892 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.892 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.892 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.893 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.893 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.893 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.893 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.893 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.893 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.893 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.893 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:56.894 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:56.894 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.894 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.894 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.894 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:56.894 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.894 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.894 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.895 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.895 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.895 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.895 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.895 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.895 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.895 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.896 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.896 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.896 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.896 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.896 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.896 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:56.896 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:56.897 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.897 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.897 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.897 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.897 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.897 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.897 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.898 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.898 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.898 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.898 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.898 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.898 INFO:teuthology.orchestra.run.smithi039.stdout: "password": { 2023-07-21T19:06:56.898 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "password", 2023-07-21T19:06:56.898 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.899 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.899 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.899 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.899 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.899 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.899 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.899 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "password of InfluxDB server user", 2023-07-21T19:06:56.899 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.900 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.900 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.900 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.900 INFO:teuthology.orchestra.run.smithi039.stdout: "port": { 2023-07-21T19:06:56.900 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "port", 2023-07-21T19:06:56.900 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:56.900 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.901 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.901 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8086", 2023-07-21T19:06:56.901 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.901 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.901 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.902 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "InfluxDB server port", 2023-07-21T19:06:56.902 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.902 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.902 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.902 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.902 INFO:teuthology.orchestra.run.smithi039.stdout: "ssl": { 2023-07-21T19:06:56.902 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssl", 2023-07-21T19:06:56.902 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.903 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.903 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.903 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "false", 2023-07-21T19:06:56.903 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.903 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.903 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.903 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-07-21T19:06:56.904 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.904 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.904 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.904 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.904 INFO:teuthology.orchestra.run.smithi039.stdout: "threads": { 2023-07-21T19:06:56.904 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "threads", 2023-07-21T19:06:56.904 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:56.905 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.905 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.905 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2023-07-21T19:06:56.905 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2023-07-21T19:06:56.905 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "32", 2023-07-21T19:06:56.905 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.905 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-07-21T19:06:56.906 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.906 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.906 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.906 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.906 INFO:teuthology.orchestra.run.smithi039.stdout: "username": { 2023-07-21T19:06:56.906 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "username", 2023-07-21T19:06:56.906 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.907 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.907 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.907 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.907 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.907 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.907 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.907 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "username of InfluxDB server user", 2023-07-21T19:06:56.908 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.908 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.908 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.908 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.908 INFO:teuthology.orchestra.run.smithi039.stdout: "verify_ssl": { 2023-07-21T19:06:56.908 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "verify_ssl", 2023-07-21T19:06:56.908 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.909 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.909 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.909 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "true", 2023-07-21T19:06:56.909 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.909 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.909 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.909 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-07-21T19:06:56.910 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.910 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.910 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.910 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.910 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.910 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.910 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:56.911 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "insights", 2023-07-21T19:06:56.911 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:56.911 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:56.911 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:56.911 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:56.911 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:56.911 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.912 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.912 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.912 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.912 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.912 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.912 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.912 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.912 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.913 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.913 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.913 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.913 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.913 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.913 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.913 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.914 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.914 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.914 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.914 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:56.914 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:56.914 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.914 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.914 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.915 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.915 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.915 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.915 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.915 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.915 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.915 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.915 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.916 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.916 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:56.916 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:56.916 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.916 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.916 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.916 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:56.917 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.917 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.917 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.917 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.917 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.917 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.917 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.917 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.918 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.918 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.918 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.918 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.918 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.918 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.918 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.919 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:56.919 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:56.919 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.919 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.919 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.919 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.919 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.919 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.920 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.920 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.920 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.920 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.920 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.920 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.920 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.920 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.921 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:56.921 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "iostat", 2023-07-21T19:06:56.921 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:56.921 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:56.921 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:56.921 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:56.921 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:56.922 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.922 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.922 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.922 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.922 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.922 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.922 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.922 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.923 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.923 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.923 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.923 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.923 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.923 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.923 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.923 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.924 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.924 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.924 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.924 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:56.924 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:56.924 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.924 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.924 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.925 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.925 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.925 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.925 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.925 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.925 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.925 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.926 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.926 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.926 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:56.926 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:56.926 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.926 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.926 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.926 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:56.927 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.927 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.927 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.927 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.927 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.927 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.927 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.928 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.928 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.928 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.928 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.928 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.928 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.928 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.928 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.929 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:56.929 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:56.929 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.929 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.929 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.929 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.929 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.930 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.930 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.930 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.930 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.930 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.930 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.930 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.930 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.931 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.931 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:56.931 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "k8sevents", 2023-07-21T19:06:56.931 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:56.931 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:56.931 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:56.931 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_event_retention_days": { 2023-07-21T19:06:56.931 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ceph_event_retention_days", 2023-07-21T19:06:56.932 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:56.932 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.932 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.932 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "7", 2023-07-21T19:06:56.932 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.932 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.932 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.933 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Days to hold ceph event information within local cache", 2023-07-21T19:06:56.933 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.933 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.933 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.933 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.933 INFO:teuthology.orchestra.run.smithi039.stdout: "config_check_secs": { 2023-07-21T19:06:56.933 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "config_check_secs", 2023-07-21T19:06:56.934 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:56.934 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.934 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.934 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2023-07-21T19:06:56.934 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "10", 2023-07-21T19:06:56.934 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.934 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.934 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-07-21T19:06:56.935 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.935 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.935 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.935 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.935 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:56.935 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:56.935 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.935 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.936 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.936 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.936 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.936 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.936 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.936 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.936 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.936 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.937 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.937 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.937 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.937 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.937 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.937 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.937 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.937 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.938 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.938 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:56.938 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:56.938 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.938 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.938 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.938 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.938 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.939 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.939 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.939 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.939 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.939 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.939 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.939 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.940 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:56.940 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:56.940 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.940 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.940 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.940 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:56.940 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.941 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.941 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.941 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.941 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.941 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.941 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.941 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.942 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.942 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.942 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.942 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.942 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.942 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.942 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.942 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:56.943 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:56.943 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.943 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.943 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.943 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.943 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.943 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.943 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.944 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.944 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.944 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.944 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.944 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.944 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.944 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.945 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:56.945 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "localpool", 2023-07-21T19:06:56.945 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:56.945 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:56.945 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:56.945 INFO:teuthology.orchestra.run.smithi039.stdout: "failure_domain": { 2023-07-21T19:06:56.945 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "failure_domain", 2023-07-21T19:06:56.946 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.946 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.946 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.946 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "host", 2023-07-21T19:06:56.946 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.946 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.946 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.947 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "failure domain for any created local pool", 2023-07-21T19:06:56.947 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-07-21T19:06:56.947 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.947 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.947 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.947 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:56.947 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:56.948 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.948 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.948 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.948 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.948 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.948 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.948 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.948 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.949 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.949 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.949 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.949 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.949 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.949 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.949 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.950 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.950 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.950 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.950 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.950 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:56.950 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:56.950 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.950 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.951 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.951 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.951 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.951 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.951 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.951 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.951 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.952 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.952 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.952 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.952 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:56.952 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:56.952 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.953 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.953 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.953 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:56.953 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.953 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.953 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.953 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.954 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.954 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.954 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.954 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.954 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.954 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.954 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.954 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.955 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.955 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.955 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.955 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:56.955 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:56.955 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.955 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.956 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.956 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.956 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.956 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.956 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.956 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.956 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.956 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.957 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.957 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.957 INFO:teuthology.orchestra.run.smithi039.stdout: "min_size": { 2023-07-21T19:06:56.957 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "min_size", 2023-07-21T19:06:56.957 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:56.957 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.957 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.958 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.958 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.958 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.958 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.958 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "default min_size for any created local pool", 2023-07-21T19:06:56.958 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-07-21T19:06:56.958 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.958 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.959 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.959 INFO:teuthology.orchestra.run.smithi039.stdout: "num_rep": { 2023-07-21T19:06:56.959 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "num_rep", 2023-07-21T19:06:56.959 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:56.959 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.959 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.959 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "3", 2023-07-21T19:06:56.959 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.960 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.960 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.960 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "default replica count for any created local pool", 2023-07-21T19:06:56.960 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.960 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.960 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.960 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.960 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num": { 2023-07-21T19:06:56.961 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pg_num", 2023-07-21T19:06:56.961 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:56.961 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.961 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.961 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "128", 2023-07-21T19:06:56.961 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.961 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.962 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.962 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "default pg_num for any created local pool", 2023-07-21T19:06:56.962 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.962 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.962 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.962 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.962 INFO:teuthology.orchestra.run.smithi039.stdout: "prefix": { 2023-07-21T19:06:56.962 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "prefix", 2023-07-21T19:06:56.963 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.963 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.963 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.963 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.963 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.963 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.963 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.964 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "name prefix for any created local pool", 2023-07-21T19:06:56.964 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.964 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.964 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.964 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.964 INFO:teuthology.orchestra.run.smithi039.stdout: "subtree": { 2023-07-21T19:06:56.965 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "subtree", 2023-07-21T19:06:56.965 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.965 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.965 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.965 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "rack", 2023-07-21T19:06:56.965 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.965 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.965 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.966 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "CRUSH level for which to create a local pool", 2023-07-21T19:06:56.966 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-07-21T19:06:56.966 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.966 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.966 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.966 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.966 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.967 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:56.967 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mds_autoscaler", 2023-07-21T19:06:56.967 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:56.967 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:56.967 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:56.967 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:56.967 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:56.967 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.968 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.968 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.968 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.968 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.968 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.968 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.968 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.968 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.969 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.969 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.969 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.969 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.969 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.969 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.969 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.969 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.970 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.970 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.970 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:56.970 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:56.970 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.970 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.970 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.970 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.971 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.971 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.971 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.971 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.971 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.971 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.971 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.972 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.972 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:56.972 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:56.972 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.972 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.972 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.972 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:56.972 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.973 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.973 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.973 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.973 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.973 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.973 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.973 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.973 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.974 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.974 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.974 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.974 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.974 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.974 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.974 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:56.975 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:56.975 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.975 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.975 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.975 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.975 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.975 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.975 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.976 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.976 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.976 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.976 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.976 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.976 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.977 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.977 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:56.977 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mirroring", 2023-07-21T19:06:56.977 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:56.977 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:56.977 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:56.977 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:56.977 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:56.978 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.978 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.978 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.978 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.978 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.978 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.978 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.979 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.979 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.979 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.979 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.979 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.979 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.979 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.979 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.980 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.980 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.980 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.980 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.980 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:56.980 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:56.980 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.980 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.981 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.981 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.981 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.981 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.981 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.981 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.981 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.982 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.982 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.982 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.982 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:56.982 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:56.982 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.982 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.982 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.983 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:56.983 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.983 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.983 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.983 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.983 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.983 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.983 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.984 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.984 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.984 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.984 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.984 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.984 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.984 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.984 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.985 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:56.985 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:56.985 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.985 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.985 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.985 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.985 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.986 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.986 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.986 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.986 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.986 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.986 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.986 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.986 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.987 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.987 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:56.987 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "nfs", 2023-07-21T19:06:56.987 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:56.987 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:56.987 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:56.987 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:56.987 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:56.988 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.988 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.988 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.988 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:56.988 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.988 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.988 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.988 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.989 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.989 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.989 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.989 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.989 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.989 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.989 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.989 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.990 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.990 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.990 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.990 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:56.990 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:56.990 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.990 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.991 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.991 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.991 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.991 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.991 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.991 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.991 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.991 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.992 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.992 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.992 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:56.992 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:56.992 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:56.992 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.992 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.993 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:56.993 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.993 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.993 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:56.993 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:56.993 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:56.993 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:56.993 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:56.994 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:56.994 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:56.994 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:56.994 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.994 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.994 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.994 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.994 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.995 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:56.995 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:56.995 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.995 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.995 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:56.995 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.995 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.995 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.996 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.996 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:56.996 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.996 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.996 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.996 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.996 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:56.997 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.997 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:56.997 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "orchestrator", 2023-07-21T19:06:56.997 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:56.997 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:56.997 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:56.997 INFO:teuthology.orchestra.run.smithi039.stdout: "fail_fs": { 2023-07-21T19:06:56.997 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "fail_fs", 2023-07-21T19:06:56.998 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:56.998 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:56.998 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:56.998 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:56.998 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:56.998 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:56.998 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:56.998 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-07-21T19:06:56.999 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:56.999 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:56.999 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:56.999 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:56.999 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:56.999 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:56.999 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:57.000 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.000 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.000 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:57.000 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.000 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.000 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:57.000 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:57.000 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:57.001 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:57.001 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:57.001 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:57.001 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:57.001 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:57.001 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.001 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.001 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.002 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.002 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.002 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:57.002 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:57.002 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:57.002 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.002 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.002 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:57.003 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.003 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.003 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.003 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.003 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.003 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.003 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.003 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.004 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:57.004 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:57.004 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:57.004 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.004 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.004 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:57.004 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.005 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.005 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:57.005 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:57.005 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:57.005 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:57.005 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:57.005 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:57.005 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:57.006 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:57.006 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.006 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.006 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.006 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.006 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.006 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:57.006 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:57.007 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:57.007 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.007 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.007 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:57.007 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.007 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.007 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.008 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.008 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.008 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.008 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.008 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.008 INFO:teuthology.orchestra.run.smithi039.stdout: "orchestrator": { 2023-07-21T19:06:57.008 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "orchestrator", 2023-07-21T19:06:57.009 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:57.009 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.009 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.009 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:57.009 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.009 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.009 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:57.009 INFO:teuthology.orchestra.run.smithi039.stdout: "cephadm", 2023-07-21T19:06:57.010 INFO:teuthology.orchestra.run.smithi039.stdout: "rook", 2023-07-21T19:06:57.010 INFO:teuthology.orchestra.run.smithi039.stdout: "test_orchestrator" 2023-07-21T19:06:57.010 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:57.010 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Orchestrator backend", 2023-07-21T19:06:57.010 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.010 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.010 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.010 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:57.011 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:57.011 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.011 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:57.011 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "osd_perf_query", 2023-07-21T19:06:57.011 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:57.011 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:57.011 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:57.012 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:57.012 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:57.012 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:57.012 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.012 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.012 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:57.012 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.013 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.013 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:57.013 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:57.013 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:57.013 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:57.013 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:57.013 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:57.013 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:57.014 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:57.014 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.014 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.014 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.014 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.014 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.014 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:57.015 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:57.015 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:57.015 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.015 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.015 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:57.015 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.015 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.015 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.016 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.016 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.016 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.016 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.016 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.017 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:57.017 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:57.017 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:57.017 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.018 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.018 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:57.018 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.018 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.018 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:57.018 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:57.019 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:57.019 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:57.019 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:57.019 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:57.019 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:57.019 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:57.019 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.019 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.020 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.020 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.020 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.020 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:57.020 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:57.020 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:57.021 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.021 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.021 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:57.021 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.021 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.021 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.021 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.022 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.022 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.022 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.022 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:57.022 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:57.022 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.022 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:57.022 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "osd_support", 2023-07-21T19:06:57.023 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:57.023 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:57.023 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:57.023 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:57.023 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:57.023 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:57.023 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.023 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.024 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:57.024 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.024 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.024 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:57.024 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:57.024 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:57.024 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:57.025 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:57.025 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:57.025 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:57.025 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:57.025 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.025 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.025 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.025 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.026 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.026 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:57.026 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:57.026 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:57.026 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.026 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.026 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:57.027 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.027 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.027 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.027 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.027 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.027 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.028 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.028 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.028 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:57.028 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:57.028 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:57.028 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.028 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.029 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:57.029 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.029 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.029 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:57.029 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:57.029 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:57.029 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:57.030 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:57.030 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:57.030 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:57.030 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:57.030 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.030 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.030 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.030 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.031 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.031 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:57.031 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:57.031 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:57.031 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.031 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.031 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:57.032 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.032 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.032 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.032 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.032 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.032 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.032 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.032 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:57.033 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:57.033 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.033 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:57.033 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pg_autoscaler", 2023-07-21T19:06:57.033 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:57.033 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:57.033 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:57.034 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:57.034 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:57.034 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:57.034 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.034 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.034 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:57.034 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.035 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.035 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:57.035 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:57.035 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:57.035 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:57.035 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:57.035 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:57.036 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:57.036 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:57.036 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.036 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.036 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.036 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.037 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.037 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:57.037 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:57.037 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:57.037 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.037 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.037 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:57.038 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.038 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.038 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.038 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.038 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.038 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.038 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.039 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.039 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:57.039 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:57.039 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:57.039 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.040 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.040 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:57.040 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.040 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.040 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:57.041 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:57.041 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:57.041 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:57.041 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:57.041 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:57.041 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:57.041 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:57.042 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.042 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.042 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.042 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.042 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.042 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:57.042 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:57.043 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:57.043 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.043 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.043 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:57.043 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.043 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.043 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.043 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.044 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.044 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.044 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.044 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.044 INFO:teuthology.orchestra.run.smithi039.stdout: "noautoscale": { 2023-07-21T19:06:57.044 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "noautoscale", 2023-07-21T19:06:57.044 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:57.045 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.045 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:57.045 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:57.045 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.045 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.045 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.045 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "global autoscale flag", 2023-07-21T19:06:57.045 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-07-21T19:06:57.046 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.046 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.046 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.046 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2023-07-21T19:06:57.046 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2023-07-21T19:06:57.046 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2023-07-21T19:06:57.046 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.047 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:57.047 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2023-07-21T19:06:57.047 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.047 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.047 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.047 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.047 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.047 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.048 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.048 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.048 INFO:teuthology.orchestra.run.smithi039.stdout: "threshold": { 2023-07-21T19:06:57.048 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "threshold", 2023-07-21T19:06:57.048 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2023-07-21T19:06:57.048 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.048 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:57.049 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "3.0", 2023-07-21T19:06:57.049 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1.0", 2023-07-21T19:06:57.049 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.049 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.049 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "scaling threshold", 2023-07-21T19:06:57.049 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-07-21T19:06:57.049 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.049 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.050 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:57.050 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:57.050 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.050 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:57.050 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "progress", 2023-07-21T19:06:57.050 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:57.050 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:57.051 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:57.051 INFO:teuthology.orchestra.run.smithi039.stdout: "allow_pg_recovery_event": { 2023-07-21T19:06:57.051 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "allow_pg_recovery_event", 2023-07-21T19:06:57.051 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:57.051 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.051 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.051 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:57.051 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.052 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.052 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.052 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "allow the module to show pg recovery progress", 2023-07-21T19:06:57.052 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.052 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.052 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.052 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.052 INFO:teuthology.orchestra.run.smithi039.stdout: "enabled": { 2023-07-21T19:06:57.053 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enabled", 2023-07-21T19:06:57.053 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:57.053 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.053 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:57.053 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:57.053 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.053 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.053 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.054 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.054 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.054 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.054 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.054 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.054 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:57.054 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:57.055 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:57.055 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.055 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.055 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:57.055 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.055 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.055 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:57.055 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:57.056 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:57.056 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:57.056 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:57.056 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:57.056 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:57.056 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:57.056 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.056 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.057 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.057 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.057 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.057 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:57.057 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:57.057 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:57.057 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.058 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.058 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:57.058 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.058 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.058 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.058 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.058 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.058 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.059 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.059 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.059 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:57.059 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:57.059 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:57.059 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.059 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.060 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:57.060 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.060 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.060 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:57.060 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:57.060 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:57.060 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:57.060 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:57.061 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:57.061 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:57.061 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:57.061 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.061 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.061 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.061 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.062 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.062 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:57.062 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:57.062 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:57.062 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.062 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.062 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:57.063 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.063 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.063 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.063 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.063 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.063 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.063 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.064 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.064 INFO:teuthology.orchestra.run.smithi039.stdout: "max_completed_events": { 2023-07-21T19:06:57.064 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_completed_events", 2023-07-21T19:06:57.064 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:57.064 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.064 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.064 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "50", 2023-07-21T19:06:57.065 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.065 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.065 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.065 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "number of past completed events to remember", 2023-07-21T19:06:57.065 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.065 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.065 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.066 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.066 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2023-07-21T19:06:57.066 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2023-07-21T19:06:57.066 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2023-07-21T19:06:57.066 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.066 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.066 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2023-07-21T19:06:57.066 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.067 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.067 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.067 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how long the module is going to sleep", 2023-07-21T19:06:57.067 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.067 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.067 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.067 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:57.068 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:57.068 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.068 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:57.068 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "prometheus", 2023-07-21T19:06:57.068 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:57.068 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:57.068 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:57.068 INFO:teuthology.orchestra.run.smithi039.stdout: "cache": { 2023-07-21T19:06:57.069 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cache", 2023-07-21T19:06:57.069 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:57.069 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.069 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:57.069 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:57.069 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.069 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.070 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.070 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.070 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.070 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.070 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.070 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.070 INFO:teuthology.orchestra.run.smithi039.stdout: "exclude_perf_counters": { 2023-07-21T19:06:57.070 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "exclude_perf_counters", 2023-07-21T19:06:57.071 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:57.071 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.071 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.071 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:57.071 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.071 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.071 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.071 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-07-21T19:06:57.072 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-07-21T19:06:57.072 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.072 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.072 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.072 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:57.072 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:57.072 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:57.073 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.073 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.073 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:57.073 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.073 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.073 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:57.073 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:57.073 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:57.074 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:57.074 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:57.074 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:57.074 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:57.074 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:57.074 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.074 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.075 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.075 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.075 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.075 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:57.075 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:57.075 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:57.075 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.076 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.076 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:57.076 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.076 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.076 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.076 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.076 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.076 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.077 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.077 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.077 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:57.077 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:57.077 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:57.077 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.077 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.077 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:57.078 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.078 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.078 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:57.078 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:57.078 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:57.078 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:57.078 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:57.078 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:57.079 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:57.079 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:57.079 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.079 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.079 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.079 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.079 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.080 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:57.080 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:57.080 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:57.080 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.080 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.080 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:57.080 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.081 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.081 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.081 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.081 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.081 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.081 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.081 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.081 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_stats_pools": { 2023-07-21T19:06:57.082 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rbd_stats_pools", 2023-07-21T19:06:57.082 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:57.082 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.082 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:57.082 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:57.082 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.082 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.082 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.083 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.083 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.083 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.083 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.083 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.083 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_stats_pools_refresh_interval": { 2023-07-21T19:06:57.083 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-07-21T19:06:57.084 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:57.084 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.084 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:57.084 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "300", 2023-07-21T19:06:57.084 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.084 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.084 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.084 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.085 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.085 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.085 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.085 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.085 INFO:teuthology.orchestra.run.smithi039.stdout: "scrape_interval": { 2023-07-21T19:06:57.085 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "scrape_interval", 2023-07-21T19:06:57.085 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2023-07-21T19:06:57.086 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.086 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:57.086 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "15.0", 2023-07-21T19:06:57.086 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.086 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.086 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.086 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.086 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.087 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.087 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.087 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.087 INFO:teuthology.orchestra.run.smithi039.stdout: "server_addr": { 2023-07-21T19:06:57.087 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_addr", 2023-07-21T19:06:57.087 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:57.087 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.088 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:57.088 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "::", 2023-07-21T19:06:57.088 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.088 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.088 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.088 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-07-21T19:06:57.088 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.089 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.089 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.089 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.089 INFO:teuthology.orchestra.run.smithi039.stdout: "server_port": { 2023-07-21T19:06:57.089 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_port", 2023-07-21T19:06:57.089 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:57.089 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.089 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.090 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "9283", 2023-07-21T19:06:57.090 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.090 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.090 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.090 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-07-21T19:06:57.090 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.090 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.090 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.091 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.091 INFO:teuthology.orchestra.run.smithi039.stdout: "stale_cache_strategy": { 2023-07-21T19:06:57.091 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "stale_cache_strategy", 2023-07-21T19:06:57.091 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:57.091 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.091 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:57.091 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "log", 2023-07-21T19:06:57.092 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.092 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.092 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.092 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.092 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.092 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.092 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.093 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.093 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_behaviour": { 2023-07-21T19:06:57.093 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_behaviour", 2023-07-21T19:06:57.093 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:57.093 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.093 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.093 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "default", 2023-07-21T19:06:57.093 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.094 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.094 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:57.094 INFO:teuthology.orchestra.run.smithi039.stdout: "default", 2023-07-21T19:06:57.094 INFO:teuthology.orchestra.run.smithi039.stdout: "error" 2023-07-21T19:06:57.094 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:57.094 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.094 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.095 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.095 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.095 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.095 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_error_status_code": { 2023-07-21T19:06:57.095 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_error_status_code", 2023-07-21T19:06:57.095 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:57.095 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.095 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.096 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "500", 2023-07-21T19:06:57.096 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "400", 2023-07-21T19:06:57.096 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "599", 2023-07-21T19:06:57.096 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.096 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.096 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.096 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.096 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.097 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:57.097 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:57.097 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.097 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:57.097 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rbd_support", 2023-07-21T19:06:57.097 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:57.097 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:57.098 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:57.098 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:57.098 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:57.098 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:57.098 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.098 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.098 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:57.098 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.099 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.099 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:57.099 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:57.099 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:57.099 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:57.099 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:57.099 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:57.099 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:57.100 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:57.100 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.100 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.100 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.100 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.100 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.100 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:57.101 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:57.101 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:57.101 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.101 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.101 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:57.101 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.102 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.102 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.102 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.102 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.102 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.102 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.102 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.103 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:57.103 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:57.103 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:57.103 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.103 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.103 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:57.104 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.104 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.104 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:57.104 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:57.104 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:57.104 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:57.105 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:57.105 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:57.105 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:57.105 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:57.105 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.105 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.105 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.106 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.106 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.106 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:57.106 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:57.106 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:57.106 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.107 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.107 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:57.107 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.107 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.107 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.107 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.107 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.108 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.108 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.108 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.108 INFO:teuthology.orchestra.run.smithi039.stdout: "max_concurrent_snap_create": { 2023-07-21T19:06:57.108 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_concurrent_snap_create", 2023-07-21T19:06:57.108 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:57.109 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.109 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:57.109 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2023-07-21T19:06:57.109 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.109 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.109 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.110 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.110 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.110 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.110 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.110 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.111 INFO:teuthology.orchestra.run.smithi039.stdout: "mirror_snapshot_schedule": { 2023-07-21T19:06:57.111 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mirror_snapshot_schedule", 2023-07-21T19:06:57.111 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:57.111 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.111 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:57.111 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:57.111 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.112 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.112 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.112 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.112 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.112 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.112 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.112 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.113 INFO:teuthology.orchestra.run.smithi039.stdout: "trash_purge_schedule": { 2023-07-21T19:06:57.113 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "trash_purge_schedule", 2023-07-21T19:06:57.113 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:57.113 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.113 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:57.113 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:57.113 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.113 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.114 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.114 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.114 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.114 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.114 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.114 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:57.114 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:57.114 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.115 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:57.115 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "restful", 2023-07-21T19:06:57.115 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:57.115 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:57.115 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:57.115 INFO:teuthology.orchestra.run.smithi039.stdout: "enable_auth": { 2023-07-21T19:06:57.115 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enable_auth", 2023-07-21T19:06:57.115 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:57.116 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.116 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:57.116 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:57.116 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.116 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.116 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.116 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.117 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.117 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.117 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.117 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.117 INFO:teuthology.orchestra.run.smithi039.stdout: "key_file": { 2023-07-21T19:06:57.117 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "key_file", 2023-07-21T19:06:57.117 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:57.117 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.118 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:57.118 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:57.118 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.118 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.118 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.118 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.118 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.119 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.119 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.119 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.119 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:57.119 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:57.119 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:57.119 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.119 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.120 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:57.120 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.120 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.120 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:57.120 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:57.120 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:57.120 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:57.120 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:57.121 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:57.121 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:57.121 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:57.121 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.121 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.121 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.121 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.122 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.122 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:57.122 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:57.122 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:57.122 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.122 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.122 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:57.122 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.123 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.123 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.123 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.123 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.123 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.123 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.123 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.123 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:57.124 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:57.124 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:57.124 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.124 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.124 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:57.124 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.124 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.125 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:57.125 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:57.125 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:57.125 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:57.125 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:57.125 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:57.126 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:57.126 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:57.126 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.126 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.126 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.126 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.126 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.126 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:57.127 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:57.127 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:57.127 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.127 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.127 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:57.127 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.127 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.128 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.128 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.128 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.128 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.128 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.128 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.128 INFO:teuthology.orchestra.run.smithi039.stdout: "server_addr": { 2023-07-21T19:06:57.128 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_addr", 2023-07-21T19:06:57.129 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:57.129 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.129 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:57.129 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:57.129 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.129 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.129 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.129 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.130 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.130 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.130 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.130 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.130 INFO:teuthology.orchestra.run.smithi039.stdout: "server_port": { 2023-07-21T19:06:57.130 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_port", 2023-07-21T19:06:57.130 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:57.131 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.131 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:57.131 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:57.131 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.131 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.131 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.131 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.131 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.132 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.132 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.132 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:57.132 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:57.132 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.132 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:57.132 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rgw", 2023-07-21T19:06:57.133 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:57.133 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:57.133 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:57.133 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:57.133 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:57.133 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:57.133 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.133 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.134 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:57.134 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.134 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.134 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:57.134 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:57.134 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:57.134 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:57.134 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:57.135 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:57.135 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:57.135 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:57.135 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.135 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.135 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.135 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.135 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.136 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:57.136 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:57.136 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:57.136 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.136 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.136 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:57.137 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.137 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.137 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.137 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.137 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.137 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.137 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.137 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.138 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:57.138 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:57.138 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:57.138 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.138 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.138 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:57.138 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.139 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.139 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:57.139 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:57.139 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:57.139 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:57.139 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:57.139 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:57.139 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:57.140 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:57.140 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.140 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.140 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.140 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.140 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.140 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:57.140 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:57.141 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:57.141 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.141 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.141 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:57.141 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.141 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.141 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.142 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.142 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.142 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.142 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.142 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:57.142 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:57.142 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.142 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:57.143 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rook", 2023-07-21T19:06:57.143 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:57.143 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:57.143 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:57.143 INFO:teuthology.orchestra.run.smithi039.stdout: "drive_group_interval": { 2023-07-21T19:06:57.143 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "drive_group_interval", 2023-07-21T19:06:57.143 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2023-07-21T19:06:57.143 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.144 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:57.144 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "300.0", 2023-07-21T19:06:57.144 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.144 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.144 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.144 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-07-21T19:06:57.144 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.144 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.145 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.145 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.145 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:57.145 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:57.145 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:57.145 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.145 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.145 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:57.146 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.146 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.146 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:57.146 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:57.146 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:57.146 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:57.146 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:57.147 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:57.147 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:57.147 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:57.147 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.147 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.147 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.147 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.148 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.148 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:57.148 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:57.148 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:57.148 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.148 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.148 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:57.148 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.149 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.149 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.149 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.149 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.149 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.149 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.149 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.150 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:57.150 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:57.150 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:57.150 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.150 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.150 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:57.150 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.150 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.151 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:57.151 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:57.151 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:57.151 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:57.151 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:57.151 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:57.151 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:57.151 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:57.152 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.152 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.152 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.152 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.152 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.152 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:57.152 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:57.153 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:57.153 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.153 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.153 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:57.153 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.153 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.153 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.153 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.154 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.154 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.154 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.154 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.154 INFO:teuthology.orchestra.run.smithi039.stdout: "storage_class": { 2023-07-21T19:06:57.154 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "storage_class", 2023-07-21T19:06:57.154 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:57.155 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.155 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:57.155 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "local", 2023-07-21T19:06:57.155 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.155 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.155 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.155 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-07-21T19:06:57.155 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.156 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.156 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.156 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:57.156 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:57.156 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.156 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:57.156 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "selftest", 2023-07-21T19:06:57.156 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:57.157 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:57.157 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:57.157 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:57.157 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:57.157 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:57.157 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.157 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.158 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:57.158 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.158 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.158 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:57.158 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:57.158 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:57.158 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:57.158 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:57.159 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:57.159 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:57.159 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:57.159 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.159 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.159 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.159 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.159 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.160 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:57.160 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:57.160 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:57.160 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.160 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.160 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:57.160 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.161 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.161 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.161 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.161 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.161 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.161 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.161 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.161 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:57.162 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:57.162 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:57.162 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.162 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.162 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:57.162 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.162 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.162 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:57.163 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:57.163 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:57.163 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:57.163 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:57.163 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:57.163 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:57.163 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:57.163 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.164 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.164 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.164 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.164 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.164 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:57.164 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:57.164 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:57.165 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.165 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.165 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:57.165 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.165 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.165 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.165 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.166 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.166 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.166 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.166 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.166 INFO:teuthology.orchestra.run.smithi039.stdout: "roption1": { 2023-07-21T19:06:57.166 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "roption1", 2023-07-21T19:06:57.166 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:57.166 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.167 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:57.167 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:57.167 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.167 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.167 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.167 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.167 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.167 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.168 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.168 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.168 INFO:teuthology.orchestra.run.smithi039.stdout: "roption2": { 2023-07-21T19:06:57.168 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "roption2", 2023-07-21T19:06:57.168 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:57.168 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.168 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:57.169 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "xyz", 2023-07-21T19:06:57.169 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.169 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.169 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.169 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.169 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.169 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.169 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.170 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.170 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption1": { 2023-07-21T19:06:57.170 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption1", 2023-07-21T19:06:57.170 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:57.170 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.170 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:57.170 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:57.171 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.171 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.171 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.171 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.171 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.171 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.171 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.171 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.172 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption2": { 2023-07-21T19:06:57.172 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption2", 2023-07-21T19:06:57.172 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:57.172 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.172 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:57.172 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:57.172 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.172 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.173 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.173 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.173 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.173 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.173 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.173 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.173 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption3": { 2023-07-21T19:06:57.174 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption3", 2023-07-21T19:06:57.174 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2023-07-21T19:06:57.174 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.174 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:57.174 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:57.174 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.174 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.174 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.175 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.175 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.175 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.175 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.175 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.175 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption4": { 2023-07-21T19:06:57.175 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption4", 2023-07-21T19:06:57.175 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:57.176 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.176 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:57.176 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:57.176 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.176 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.176 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.176 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.176 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.177 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.177 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.177 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.177 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption5": { 2023-07-21T19:06:57.177 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption5", 2023-07-21T19:06:57.177 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:57.177 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.177 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:57.178 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:57.178 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.178 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.178 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.178 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.178 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.178 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.178 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.179 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.179 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption6": { 2023-07-21T19:06:57.179 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption6", 2023-07-21T19:06:57.179 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:57.179 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.179 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:57.179 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:57.180 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.180 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.180 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.180 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.180 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.180 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.180 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.181 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.181 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption7": { 2023-07-21T19:06:57.181 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption7", 2023-07-21T19:06:57.181 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:57.181 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.181 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:57.181 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:57.181 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2023-07-21T19:06:57.182 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "42", 2023-07-21T19:06:57.182 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.182 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.182 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.182 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.182 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.182 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.182 INFO:teuthology.orchestra.run.smithi039.stdout: "testkey": { 2023-07-21T19:06:57.183 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "testkey", 2023-07-21T19:06:57.183 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:57.183 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.183 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:57.183 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:57.183 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.183 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.184 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.184 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.184 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.184 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.184 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.184 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.184 INFO:teuthology.orchestra.run.smithi039.stdout: "testlkey": { 2023-07-21T19:06:57.185 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "testlkey", 2023-07-21T19:06:57.185 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:57.185 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.185 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:57.185 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:57.185 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.185 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.186 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.186 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.186 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.186 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.186 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.186 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.186 INFO:teuthology.orchestra.run.smithi039.stdout: "testnewline": { 2023-07-21T19:06:57.187 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "testnewline", 2023-07-21T19:06:57.187 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:57.187 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.187 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:57.187 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:57.187 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.187 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.187 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.188 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.188 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.188 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.188 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.188 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:57.188 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:57.188 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.189 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:57.189 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "snap_schedule", 2023-07-21T19:06:57.189 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:57.189 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:57.189 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:57.189 INFO:teuthology.orchestra.run.smithi039.stdout: "allow_m_granularity": { 2023-07-21T19:06:57.189 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "allow_m_granularity", 2023-07-21T19:06:57.189 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:57.190 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.190 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.190 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:57.190 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.190 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.190 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.190 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "allow minute scheduled snapshots", 2023-07-21T19:06:57.191 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.191 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.191 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.191 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.191 INFO:teuthology.orchestra.run.smithi039.stdout: "dump_on_update": { 2023-07-21T19:06:57.191 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "dump_on_update", 2023-07-21T19:06:57.191 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:57.191 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.192 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.192 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:57.192 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.192 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.192 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.192 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "dump database to debug log on update", 2023-07-21T19:06:57.192 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.192 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.193 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.193 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.193 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:57.193 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:57.193 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:57.193 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.193 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.194 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:57.194 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.194 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.194 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:57.194 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:57.194 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:57.194 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:57.194 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:57.195 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:57.195 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:57.195 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:57.195 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.195 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.195 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.195 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.195 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.196 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:57.196 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:57.196 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:57.196 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.196 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.196 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:57.196 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.197 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.197 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.197 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.197 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.197 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.197 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.197 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.197 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:57.198 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:57.198 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:57.198 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.198 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.198 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:57.198 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.198 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.198 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:57.199 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:57.199 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:57.199 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:57.199 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:57.199 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:57.199 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:57.199 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:57.199 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.200 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.200 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.200 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.200 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.200 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:57.200 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:57.200 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:57.200 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.201 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.201 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:57.201 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.201 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.201 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.201 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.201 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.201 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.202 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.202 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:57.202 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:57.202 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.202 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:57.202 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "stats", 2023-07-21T19:06:57.202 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:57.202 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:57.203 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:57.203 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:57.203 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:57.203 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:57.203 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.203 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.203 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:57.204 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.204 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.204 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:57.204 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:57.204 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:57.204 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:57.204 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:57.204 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:57.205 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:57.205 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:57.205 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.205 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.205 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.205 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.205 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.205 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:57.206 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:57.206 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:57.206 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.206 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.206 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:57.206 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.206 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.206 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.207 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.207 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.207 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.207 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.207 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.207 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:57.207 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:57.207 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:57.208 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.208 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.208 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:57.208 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.208 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.208 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:57.208 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:57.209 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:57.209 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:57.209 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:57.209 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:57.209 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:57.209 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:57.209 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.209 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.210 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.210 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.210 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.210 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:57.210 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:57.210 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:57.210 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.210 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.211 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:57.211 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.211 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.211 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.211 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.211 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.211 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.211 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.212 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:57.212 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:57.212 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.212 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:57.212 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "status", 2023-07-21T19:06:57.212 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:57.212 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:57.213 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:57.213 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:57.213 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:57.213 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:57.213 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.213 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.213 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:57.213 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.214 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.214 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:57.214 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:57.214 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:57.214 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:57.214 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:57.214 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:57.214 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:57.215 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:57.215 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.215 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.215 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.215 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.215 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.215 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:57.215 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:57.216 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:57.216 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.216 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.216 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:57.216 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.216 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.216 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.217 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.217 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.217 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.217 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.217 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.217 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:57.217 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:57.217 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:57.218 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.218 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.218 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:57.218 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.218 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.218 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:57.218 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:57.218 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:57.219 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:57.219 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:57.219 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:57.219 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:57.219 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:57.219 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.219 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.219 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.220 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.220 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.220 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:57.220 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:57.220 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:57.220 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.220 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.221 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:57.221 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.221 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.221 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.221 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.221 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.221 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.221 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.222 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:57.222 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:57.222 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.222 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:57.222 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "telegraf", 2023-07-21T19:06:57.222 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:57.222 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:57.222 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:57.223 INFO:teuthology.orchestra.run.smithi039.stdout: "address": { 2023-07-21T19:06:57.223 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "address", 2023-07-21T19:06:57.223 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:57.223 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.223 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:57.223 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-07-21T19:06:57.223 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.223 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.224 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.224 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.224 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.224 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.224 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.224 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.224 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2023-07-21T19:06:57.225 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2023-07-21T19:06:57.225 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2023-07-21T19:06:57.225 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.225 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:57.225 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "15", 2023-07-21T19:06:57.225 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.225 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.225 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.226 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.226 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.226 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.226 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.226 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.226 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:57.226 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:57.226 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:57.227 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.227 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.227 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:57.227 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.227 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.227 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:57.227 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:57.228 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:57.228 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:57.228 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:57.228 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:57.228 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:57.228 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:57.228 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.228 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.229 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.229 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.229 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.229 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:57.229 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:57.229 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:57.229 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.229 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.230 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:57.230 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.230 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.230 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.230 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.230 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.230 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.230 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.231 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.231 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:57.231 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:57.231 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:57.231 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.231 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.231 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:57.231 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.232 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.232 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:57.232 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:57.232 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:57.232 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:57.232 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:57.232 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:57.233 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:57.233 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:57.233 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.233 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.233 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.233 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.233 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.233 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:57.234 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:57.234 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:57.234 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.234 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.234 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:57.234 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.234 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.234 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.235 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.235 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.235 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.235 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.235 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:57.235 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:57.235 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.235 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:57.236 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "telemetry", 2023-07-21T19:06:57.236 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:57.236 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:57.236 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:57.236 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_basic": { 2023-07-21T19:06:57.236 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_basic", 2023-07-21T19:06:57.236 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:57.237 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.237 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:57.237 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:57.237 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.237 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.237 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.237 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share basic cluster information (size, version)", 2023-07-21T19:06:57.237 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.238 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.238 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.238 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.238 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_crash": { 2023-07-21T19:06:57.238 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_crash", 2023-07-21T19:06:57.238 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:57.238 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.238 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:57.239 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:57.239 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.239 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.239 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.239 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-07-21T19:06:57.239 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.239 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.240 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.240 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.240 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_device": { 2023-07-21T19:06:57.240 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_device", 2023-07-21T19:06:57.240 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:57.240 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.240 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:57.240 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2023-07-21T19:06:57.241 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.241 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.241 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.241 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-07-21T19:06:57.241 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.241 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.241 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.241 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.242 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_ident": { 2023-07-21T19:06:57.242 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_ident", 2023-07-21T19:06:57.242 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:57.242 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.242 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:57.242 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:57.242 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.242 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.243 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.243 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-07-21T19:06:57.243 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.243 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.243 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.243 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.243 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_perf": { 2023-07-21T19:06:57.243 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_perf", 2023-07-21T19:06:57.244 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:57.244 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.244 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:57.244 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:57.244 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.244 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.244 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.245 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share various performance metrics of a cluster", 2023-07-21T19:06:57.245 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.245 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.245 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.245 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.245 INFO:teuthology.orchestra.run.smithi039.stdout: "contact": { 2023-07-21T19:06:57.245 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "contact", 2023-07-21T19:06:57.245 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:57.246 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.246 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:57.246 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:57.246 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.246 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.246 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.246 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.246 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.247 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.247 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.247 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.247 INFO:teuthology.orchestra.run.smithi039.stdout: "description": { 2023-07-21T19:06:57.247 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "description", 2023-07-21T19:06:57.247 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:57.247 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.247 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:57.248 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:57.248 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.248 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.248 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.248 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.248 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.248 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.249 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.249 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.249 INFO:teuthology.orchestra.run.smithi039.stdout: "device_url": { 2023-07-21T19:06:57.249 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "device_url", 2023-07-21T19:06:57.249 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:57.249 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.249 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:57.249 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-07-21T19:06:57.250 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.250 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.250 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.250 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.250 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.250 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.250 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.251 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.251 INFO:teuthology.orchestra.run.smithi039.stdout: "enabled": { 2023-07-21T19:06:57.251 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enabled", 2023-07-21T19:06:57.251 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:57.251 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.251 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:57.251 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:57.251 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.252 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.252 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.252 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.252 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.252 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.252 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.252 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.252 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2023-07-21T19:06:57.253 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2023-07-21T19:06:57.253 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:57.253 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.253 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:57.253 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "24", 2023-07-21T19:06:57.253 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "8", 2023-07-21T19:06:57.253 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.253 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.254 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.254 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.254 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.254 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.254 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.254 INFO:teuthology.orchestra.run.smithi039.stdout: "last_opt_revision": { 2023-07-21T19:06:57.254 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "last_opt_revision", 2023-07-21T19:06:57.254 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:57.255 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.255 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:57.255 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "1", 2023-07-21T19:06:57.255 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.255 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.255 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.255 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.256 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.256 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.256 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.256 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.256 INFO:teuthology.orchestra.run.smithi039.stdout: "leaderboard": { 2023-07-21T19:06:57.256 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "leaderboard", 2023-07-21T19:06:57.256 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:57.256 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.257 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:57.257 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:57.257 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.257 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.257 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.257 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.257 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.258 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.258 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.258 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.258 INFO:teuthology.orchestra.run.smithi039.stdout: "leaderboard_description": { 2023-07-21T19:06:57.258 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "leaderboard_description", 2023-07-21T19:06:57.258 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:57.258 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.258 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:57.259 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:57.259 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.259 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.259 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.259 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.259 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.259 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.260 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.260 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.260 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:57.260 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:57.260 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:57.260 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.260 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.260 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:57.261 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.261 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.261 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:57.261 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:57.261 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:57.261 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:57.262 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:57.262 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:57.262 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:57.262 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:57.262 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.262 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.262 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.263 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.263 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.263 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:57.263 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:57.263 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:57.263 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.263 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.264 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:57.264 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.264 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.264 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.264 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.264 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.264 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.265 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.265 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.265 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:57.265 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:57.265 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:57.265 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.265 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.265 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:57.266 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.266 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.266 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:57.266 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:57.266 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:57.266 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:57.266 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:57.266 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:57.267 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:57.267 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:57.267 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.267 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.267 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.267 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.267 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.268 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:57.268 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:57.268 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:57.268 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.268 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.268 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:57.268 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.269 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.269 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.269 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.269 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.269 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.269 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.269 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.269 INFO:teuthology.orchestra.run.smithi039.stdout: "organization": { 2023-07-21T19:06:57.270 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "organization", 2023-07-21T19:06:57.270 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:57.270 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.270 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:57.270 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:57.270 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.270 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.270 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.271 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.271 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.271 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.271 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.271 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.271 INFO:teuthology.orchestra.run.smithi039.stdout: "proxy": { 2023-07-21T19:06:57.271 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "proxy", 2023-07-21T19:06:57.271 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:57.272 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.272 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:57.272 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:57.272 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.272 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.272 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.272 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.273 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.273 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.273 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.273 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.273 INFO:teuthology.orchestra.run.smithi039.stdout: "url": { 2023-07-21T19:06:57.273 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "url", 2023-07-21T19:06:57.273 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:57.273 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.274 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:57.274 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-07-21T19:06:57.274 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.274 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.274 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.274 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.274 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.275 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.275 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.275 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:57.275 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:57.275 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.275 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:57.275 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "test_orchestrator", 2023-07-21T19:06:57.276 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:57.276 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:57.276 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:57.276 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:57.276 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:57.276 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:57.276 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.276 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.277 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:57.277 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.277 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.277 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:57.277 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:57.277 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:57.277 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:57.278 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:57.278 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:57.278 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:57.278 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:57.278 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.278 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.278 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.278 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.279 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.279 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:57.279 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:57.279 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:57.279 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.279 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.279 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:57.279 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.280 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.280 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.280 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.280 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.280 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.280 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.280 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.281 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:57.281 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:57.281 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:57.281 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.281 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.281 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:57.281 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.281 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.282 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:57.282 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:57.282 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:57.282 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:57.282 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:57.282 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:57.282 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:57.283 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:57.283 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.283 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.283 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.283 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.283 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.283 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:57.283 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:57.284 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:57.284 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.284 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.284 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:57.284 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.284 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.284 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.285 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.285 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.285 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.285 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.285 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:57.285 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:57.285 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.286 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:57.286 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "volumes", 2023-07-21T19:06:57.286 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:57.286 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:57.286 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:57.286 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:57.286 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:57.286 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:57.287 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.287 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.287 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:57.287 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.287 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.287 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:57.287 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:57.288 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:57.288 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:57.288 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:57.288 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:57.288 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:57.288 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:57.288 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.288 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.289 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.289 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.289 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.289 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:57.289 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:57.289 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:57.289 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.289 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.290 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:57.290 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.290 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.290 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.290 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.290 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.290 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.291 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.291 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.291 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:57.291 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:57.291 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:57.291 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.291 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.291 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:57.292 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.292 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.292 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:57.292 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:57.292 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:57.292 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:57.292 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:57.292 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:57.293 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:57.293 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:57.293 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.293 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.293 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.293 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.293 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.293 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:57.294 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:57.294 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:57.294 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.294 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.294 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:57.294 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.294 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.294 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.295 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.295 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.295 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.295 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.295 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.295 INFO:teuthology.orchestra.run.smithi039.stdout: "max_concurrent_clones": { 2023-07-21T19:06:57.295 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_concurrent_clones", 2023-07-21T19:06:57.296 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:57.296 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.296 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:57.296 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "4", 2023-07-21T19:06:57.296 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.296 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.296 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.296 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Number of asynchronous cloner threads", 2023-07-21T19:06:57.297 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.297 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.297 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.297 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.297 INFO:teuthology.orchestra.run.smithi039.stdout: "snapshot_clone_delay": { 2023-07-21T19:06:57.297 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "snapshot_clone_delay", 2023-07-21T19:06:57.297 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:57.298 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.298 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:57.298 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2023-07-21T19:06:57.298 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.298 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.298 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.298 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-07-21T19:06:57.298 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.299 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.299 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.299 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:57.299 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:57.299 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.299 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:57.299 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix", 2023-07-21T19:06:57.299 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2023-07-21T19:06:57.300 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2023-07-21T19:06:57.300 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2023-07-21T19:06:57.300 INFO:teuthology.orchestra.run.smithi039.stdout: "discovery_interval": { 2023-07-21T19:06:57.300 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "discovery_interval", 2023-07-21T19:06:57.300 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2023-07-21T19:06:57.300 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.300 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:57.301 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "100", 2023-07-21T19:06:57.301 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.301 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.301 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.301 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.301 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.301 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.301 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.302 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.302 INFO:teuthology.orchestra.run.smithi039.stdout: "identifier": { 2023-07-21T19:06:57.302 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "identifier", 2023-07-21T19:06:57.302 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:57.302 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.302 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:57.302 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:57.302 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.303 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.303 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.303 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.303 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.303 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.303 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.303 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.303 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2023-07-21T19:06:57.304 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2023-07-21T19:06:57.304 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2023-07-21T19:06:57.304 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.304 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:57.304 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2023-07-21T19:06:57.304 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.305 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.305 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.305 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.305 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.305 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.305 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.305 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.305 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2023-07-21T19:06:57.306 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2023-07-21T19:06:57.306 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:57.306 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.306 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.306 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:57.306 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.306 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.306 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:57.307 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:57.307 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:57.307 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:57.307 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:57.307 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:57.307 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:57.307 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:57.308 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.308 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.308 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.308 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.308 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.308 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2023-07-21T19:06:57.308 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2023-07-21T19:06:57.309 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:57.309 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.309 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.309 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:57.309 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.309 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.309 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.309 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.310 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.310 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.310 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.310 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.310 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2023-07-21T19:06:57.310 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2023-07-21T19:06:57.310 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:57.310 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.311 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.311 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2023-07-21T19:06:57.311 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.311 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.311 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2023-07-21T19:06:57.311 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2023-07-21T19:06:57.311 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2023-07-21T19:06:57.312 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2023-07-21T19:06:57.312 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2023-07-21T19:06:57.312 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2023-07-21T19:06:57.312 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2023-07-21T19:06:57.312 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:57.312 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.312 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.312 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.313 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.313 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.313 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2023-07-21T19:06:57.313 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2023-07-21T19:06:57.313 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2023-07-21T19:06:57.313 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.313 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2023-07-21T19:06:57.314 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2023-07-21T19:06:57.314 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.314 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.314 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.314 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.314 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.314 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.314 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.315 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.315 INFO:teuthology.orchestra.run.smithi039.stdout: "zabbix_host": { 2023-07-21T19:06:57.315 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix_host", 2023-07-21T19:06:57.315 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:57.315 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.315 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:57.315 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2023-07-21T19:06:57.315 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.316 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.316 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.316 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.316 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.316 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.316 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.316 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.317 INFO:teuthology.orchestra.run.smithi039.stdout: "zabbix_port": { 2023-07-21T19:06:57.317 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix_port", 2023-07-21T19:06:57.317 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2023-07-21T19:06:57.317 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.317 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:57.317 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10051", 2023-07-21T19:06:57.317 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.317 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.318 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.318 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.318 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.318 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.318 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.318 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.318 INFO:teuthology.orchestra.run.smithi039.stdout: "zabbix_sender": { 2023-07-21T19:06:57.319 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix_sender", 2023-07-21T19:06:57.319 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2023-07-21T19:06:57.319 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2023-07-21T19:06:57.319 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2023-07-21T19:06:57.319 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-07-21T19:06:57.319 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2023-07-21T19:06:57.319 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2023-07-21T19:06:57.319 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2023-07-21T19:06:57.320 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2023-07-21T19:06:57.320 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2023-07-21T19:06:57.320 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2023-07-21T19:06:57.320 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2023-07-21T19:06:57.320 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:57.320 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:57.320 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:57.320 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:57.321 INFO:teuthology.orchestra.run.smithi039.stdout: "services": { 2023-07-21T19:06:57.321 INFO:teuthology.orchestra.run.smithi039.stdout: "dashboard": "https://172.21.15.39:8443/" 2023-07-21T19:06:57.321 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.321 INFO:teuthology.orchestra.run.smithi039.stdout: "always_on_modules": { 2023-07-21T19:06:57.321 INFO:teuthology.orchestra.run.smithi039.stdout: "octopus": [ 2023-07-21T19:06:57.321 INFO:teuthology.orchestra.run.smithi039.stdout: "balancer", 2023-07-21T19:06:57.321 INFO:teuthology.orchestra.run.smithi039.stdout: "crash", 2023-07-21T19:06:57.322 INFO:teuthology.orchestra.run.smithi039.stdout: "devicehealth", 2023-07-21T19:06:57.322 INFO:teuthology.orchestra.run.smithi039.stdout: "orchestrator", 2023-07-21T19:06:57.322 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_autoscaler", 2023-07-21T19:06:57.322 INFO:teuthology.orchestra.run.smithi039.stdout: "progress", 2023-07-21T19:06:57.322 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_support", 2023-07-21T19:06:57.322 INFO:teuthology.orchestra.run.smithi039.stdout: "status", 2023-07-21T19:06:57.322 INFO:teuthology.orchestra.run.smithi039.stdout: "telemetry", 2023-07-21T19:06:57.322 INFO:teuthology.orchestra.run.smithi039.stdout: "volumes" 2023-07-21T19:06:57.323 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:57.323 INFO:teuthology.orchestra.run.smithi039.stdout: "pacific": [ 2023-07-21T19:06:57.323 INFO:teuthology.orchestra.run.smithi039.stdout: "balancer", 2023-07-21T19:06:57.323 INFO:teuthology.orchestra.run.smithi039.stdout: "crash", 2023-07-21T19:06:57.323 INFO:teuthology.orchestra.run.smithi039.stdout: "devicehealth", 2023-07-21T19:06:57.323 INFO:teuthology.orchestra.run.smithi039.stdout: "orchestrator", 2023-07-21T19:06:57.324 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_autoscaler", 2023-07-21T19:06:57.324 INFO:teuthology.orchestra.run.smithi039.stdout: "progress", 2023-07-21T19:06:57.324 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_support", 2023-07-21T19:06:57.324 INFO:teuthology.orchestra.run.smithi039.stdout: "status", 2023-07-21T19:06:57.324 INFO:teuthology.orchestra.run.smithi039.stdout: "telemetry", 2023-07-21T19:06:57.324 INFO:teuthology.orchestra.run.smithi039.stdout: "volumes" 2023-07-21T19:06:57.324 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:57.325 INFO:teuthology.orchestra.run.smithi039.stdout: "quincy": [ 2023-07-21T19:06:57.325 INFO:teuthology.orchestra.run.smithi039.stdout: "balancer", 2023-07-21T19:06:57.325 INFO:teuthology.orchestra.run.smithi039.stdout: "crash", 2023-07-21T19:06:57.325 INFO:teuthology.orchestra.run.smithi039.stdout: "devicehealth", 2023-07-21T19:06:57.325 INFO:teuthology.orchestra.run.smithi039.stdout: "orchestrator", 2023-07-21T19:06:57.325 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_autoscaler", 2023-07-21T19:06:57.325 INFO:teuthology.orchestra.run.smithi039.stdout: "progress", 2023-07-21T19:06:57.325 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_support", 2023-07-21T19:06:57.326 INFO:teuthology.orchestra.run.smithi039.stdout: "status", 2023-07-21T19:06:57.326 INFO:teuthology.orchestra.run.smithi039.stdout: "telemetry", 2023-07-21T19:06:57.326 INFO:teuthology.orchestra.run.smithi039.stdout: "volumes" 2023-07-21T19:06:57.326 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2023-07-21T19:06:57.326 INFO:teuthology.orchestra.run.smithi039.stdout: "reef": [ 2023-07-21T19:06:57.326 INFO:teuthology.orchestra.run.smithi039.stdout: "balancer", 2023-07-21T19:06:57.326 INFO:teuthology.orchestra.run.smithi039.stdout: "crash", 2023-07-21T19:06:57.326 INFO:teuthology.orchestra.run.smithi039.stdout: "devicehealth", 2023-07-21T19:06:57.327 INFO:teuthology.orchestra.run.smithi039.stdout: "orchestrator", 2023-07-21T19:06:57.327 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_autoscaler", 2023-07-21T19:06:57.327 INFO:teuthology.orchestra.run.smithi039.stdout: "progress", 2023-07-21T19:06:57.327 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_support", 2023-07-21T19:06:57.327 INFO:teuthology.orchestra.run.smithi039.stdout: "status", 2023-07-21T19:06:57.327 INFO:teuthology.orchestra.run.smithi039.stdout: "telemetry", 2023-07-21T19:06:57.327 INFO:teuthology.orchestra.run.smithi039.stdout: "volumes" 2023-07-21T19:06:57.327 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2023-07-21T19:06:57.328 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.328 INFO:teuthology.orchestra.run.smithi039.stdout: "last_failure_osd_epoch": 82, 2023-07-21T19:06:57.328 INFO:teuthology.orchestra.run.smithi039.stdout: "active_clients": [ 2023-07-21T19:06:57.328 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:57.328 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "devicehealth", 2023-07-21T19:06:57.328 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2023-07-21T19:06:57.328 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:57.329 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2023-07-21T19:06:57.329 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:0", 2023-07-21T19:06:57.329 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 2053710613 2023-07-21T19:06:57.329 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:57.329 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2023-07-21T19:06:57.329 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.329 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:57.329 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "libcephsqlite", 2023-07-21T19:06:57.330 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2023-07-21T19:06:57.330 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:57.330 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2023-07-21T19:06:57.330 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:0", 2023-07-21T19:06:57.330 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3514299412 2023-07-21T19:06:57.330 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:57.330 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2023-07-21T19:06:57.330 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.331 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:57.331 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rbd_support", 2023-07-21T19:06:57.331 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2023-07-21T19:06:57.331 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:57.331 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2023-07-21T19:06:57.331 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:0", 2023-07-21T19:06:57.331 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 604056272 2023-07-21T19:06:57.332 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:57.332 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2023-07-21T19:06:57.332 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.332 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:57.332 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "snap_schedule", 2023-07-21T19:06:57.332 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2023-07-21T19:06:57.332 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:57.332 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2023-07-21T19:06:57.333 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:0", 2023-07-21T19:06:57.333 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3424249523 2023-07-21T19:06:57.333 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:57.333 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2023-07-21T19:06:57.333 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.333 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:57.333 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "snap_schedule", 2023-07-21T19:06:57.333 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2023-07-21T19:06:57.334 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:57.334 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2023-07-21T19:06:57.334 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:0", 2023-07-21T19:06:57.334 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3641799145 2023-07-21T19:06:57.334 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:57.334 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2023-07-21T19:06:57.334 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:57.335 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:57.335 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "volumes", 2023-07-21T19:06:57.335 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2023-07-21T19:06:57.335 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-07-21T19:06:57.335 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2023-07-21T19:06:57.335 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:0", 2023-07-21T19:06:57.335 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3294277186 2023-07-21T19:06:57.335 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:57.336 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2023-07-21T19:06:57.336 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:57.336 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2023-07-21T19:06:57.336 INFO:teuthology.orchestra.run.smithi039.stdout:} 2023-07-21T19:06:57.337 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.836+0000 7f1584561700 1 -- 172.21.15.39:0/4013105668 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f15680a1380 msgr2=0x7f15680a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:57.338 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.836+0000 7f1584561700 1 --2- 172.21.15.39:0/4013105668 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f15680a1380 0x7f15680a3830 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f156c00b5c0 tx=0x7f156c005fb0 comp rx=0 tx=0).stop 2023-07-21T19:06:57.338 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.836+0000 7f1584561700 1 -- 172.21.15.39:0/4013105668 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f157c0ff380 msgr2=0x7f157c100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:57.338 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.836+0000 7f1584561700 1 --2- 172.21.15.39:0/4013105668 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f157c0ff380 0x7f157c100b20 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f157800c9a0 tx=0x7f157800ccb0 comp rx=0 tx=0).stop 2023-07-21T19:06:57.338 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.836+0000 7f1584561700 1 -- 172.21.15.39:0/4013105668 shutdown_connections 2023-07-21T19:06:57.338 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.836+0000 7f1584561700 1 --2- 172.21.15.39:0/4013105668 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f15680a1380 0x7f15680a3830 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:57.338 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.836+0000 7f1584561700 1 --2- 172.21.15.39:0/4013105668 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f157c0fe9d0 0x7f157c1005e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:57.339 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.836+0000 7f1584561700 1 --2- 172.21.15.39:0/4013105668 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f157c1049e0 0x7f157c1a3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:57.339 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.836+0000 7f1584561700 1 --2- 172.21.15.39:0/4013105668 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f157c0ff380 0x7f157c100b20 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:57.339 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.836+0000 7f1584561700 1 -- 172.21.15.39:0/4013105668 >> 172.21.15.39:0/4013105668 conn(0x7f157c0fa490 msgr2=0x7f157c0fbeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:57.339 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.836+0000 7f1584561700 1 -- 172.21.15.39:0/4013105668 shutdown_connections 2023-07-21T19:06:57.339 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:06:54.836+0000 7f1584561700 1 -- 172.21.15.39:0/4013105668 wait complete. 2023-07-21T19:06:57.340 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:06:54 smithi039 ceph-mon[96489]: pgmap v797: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 545 op/s 2023-07-21T19:06:57.340 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:06:54 smithi039 ceph-mon[96489]: from='client.17958 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:06:57.340 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:06:54 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T19:06:57.340 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:06:54 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/2185103094' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T19:06:57.340 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:06:54 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/2000085165' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T19:06:57.341 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:06:54 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/2067267084' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T19:06:57.341 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:06:54 smithi155 ceph-mon[97934]: pgmap v797: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 545 op/s 2023-07-21T19:06:57.341 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:06:54 smithi155 ceph-mon[97934]: from='client.17958 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:06:57.341 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:06:54 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T19:06:57.342 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:06:54 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/2185103094' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T19:06:57.342 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:06:54 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/2000085165' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T19:06:57.342 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:06:54 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/2067267084' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T19:06:57.342 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:06:55 smithi155 ceph-mon[97934]: pgmap v798: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.6 MiB/s wr, 330 op/s 2023-07-21T19:06:57.342 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:06:55 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/4013105668' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-07-21T19:06:57.342 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:06:56 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T19:06:57.342 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:06:54 smithi193 ceph-mon[98204]: pgmap v797: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 545 op/s 2023-07-21T19:06:57.343 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:06:54 smithi193 ceph-mon[98204]: from='client.17958 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:06:57.343 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:06:54 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T19:06:57.343 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:06:54 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/2185103094' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T19:06:57.343 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:06:54 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/2000085165' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T19:06:57.343 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:06:54 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/2067267084' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T19:06:57.343 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:06:55 smithi193 ceph-mon[98204]: pgmap v798: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.6 MiB/s wr, 330 op/s 2023-07-21T19:06:57.344 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:06:55 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/4013105668' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-07-21T19:06:57.344 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:06:56 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T19:06:57.352 DEBUG:tasks.check_counter:Getting stats from a 2023-07-21T19:06:57.352 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph daemon mds.a perf dump 2023-07-21T19:06:57.353 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:06:55 smithi039 ceph-mon[96489]: pgmap v798: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.6 MiB/s wr, 330 op/s 2023-07-21T19:06:57.353 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:06:55 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/4013105668' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-07-21T19:06:57.353 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:06:56 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T19:06:57.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:06:57 smithi039 ceph-mon[96489]: pgmap v799: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.5 MiB/s wr, 377 op/s 2023-07-21T19:06:57.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:06:57 smithi193 ceph-mon[98204]: pgmap v799: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.5 MiB/s wr, 377 op/s 2023-07-21T19:06:57.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:06:57 smithi155 ceph-mon[97934]: pgmap v799: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.5 MiB/s wr, 377 op/s 2023-07-21T19:06:57.964 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T19:06:59.635 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2023-07-21T19:06:59.636 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-0": { 2023-07-21T19:06:59.636 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 517448, 2023-07-21T19:06:59.636 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 400795, 2023-07-21T19:06:59.636 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 154640780, 2023-07-21T19:06:59.636 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 392332739, 2023-07-21T19:06:59.637 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 17, 2023-07-21T19:06:59.637 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 5, 2023-07-21T19:06:59.637 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 27.975250268, 2023-07-21T19:06:59.637 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 10.829136661, 2023-07-21T19:06:59.637 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 20.623943917, 2023-07-21T19:06:59.637 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 1.222952714, 2023-07-21T19:06:59.637 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:06:59.638 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 400793, 2023-07-21T19:06:59.638 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 27.855860444, 2023-07-21T19:06:59.638 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000069501 2023-07-21T19:06:59.638 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:59.638 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:06:59.638 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 844840, 2023-07-21T19:06:59.639 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.144337904, 2023-07-21T19:06:59.639 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000170 2023-07-21T19:06:59.639 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:59.639 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-21T19:06:59.639 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 0 2023-07-21T19:06:59.639 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:59.639 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-1": { 2023-07-21T19:06:59.639 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 495761, 2023-07-21T19:06:59.640 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 630787, 2023-07-21T19:06:59.640 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 159200601, 2023-07-21T19:06:59.640 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 640998111, 2023-07-21T19:06:59.640 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 13, 2023-07-21T19:06:59.640 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 7, 2023-07-21T19:06:59.641 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 32.905675079, 2023-07-21T19:06:59.641 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 17.976611414, 2023-07-21T19:06:59.641 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 20.501896274, 2023-07-21T19:06:59.641 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.945089144, 2023-07-21T19:06:59.642 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:06:59.642 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 630787, 2023-07-21T19:06:59.642 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 72.162445830, 2023-07-21T19:06:59.642 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000114400 2023-07-21T19:06:59.642 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:59.643 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:06:59.643 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 756877, 2023-07-21T19:06:59.643 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.200246091, 2023-07-21T19:06:59.643 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000264 2023-07-21T19:06:59.643 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:59.644 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 1924288, 2023-07-21T19:06:59.644 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 275504 2023-07-21T19:06:59.644 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:59.644 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-2": { 2023-07-21T19:06:59.644 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 214119, 2023-07-21T19:06:59.644 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 182639, 2023-07-21T19:06:59.644 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 70514382, 2023-07-21T19:06:59.645 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 465297235, 2023-07-21T19:06:59.645 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 14, 2023-07-21T19:06:59.645 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 6, 2023-07-21T19:06:59.645 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 12.924302731, 2023-07-21T19:06:59.645 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 5.196329223, 2023-07-21T19:06:59.645 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 7.953347421, 2023-07-21T19:06:59.645 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 1.267044849, 2023-07-21T19:06:59.646 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:06:59.646 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 182636, 2023-07-21T19:06:59.646 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 17.703095118, 2023-07-21T19:06:59.646 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000096931 2023-07-21T19:06:59.646 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:59.646 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:06:59.646 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 329317, 2023-07-21T19:06:59.647 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.060278357, 2023-07-21T19:06:59.647 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000183 2023-07-21T19:06:59.647 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:59.647 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 2432, 2023-07-21T19:06:59.647 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 726880 2023-07-21T19:06:59.647 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:59.647 INFO:teuthology.orchestra.run.smithi039.stdout: "cct": { 2023-07-21T19:06:59.648 INFO:teuthology.orchestra.run.smithi039.stdout: "total_workers": 1, 2023-07-21T19:06:59.648 INFO:teuthology.orchestra.run.smithi039.stdout: "unhealthy_workers": 0 2023-07-21T19:06:59.648 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:59.648 INFO:teuthology.orchestra.run.smithi039.stdout: "finisher-MDSRank": { 2023-07-21T19:06:59.648 INFO:teuthology.orchestra.run.smithi039.stdout: "queue_len": 0, 2023-07-21T19:06:59.648 INFO:teuthology.orchestra.run.smithi039.stdout: "complete_latency": { 2023-07-21T19:06:59.648 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 79389, 2023-07-21T19:06:59.649 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 126.293683035, 2023-07-21T19:06:59.649 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.001590820 2023-07-21T19:06:59.649 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:59.649 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:59.649 INFO:teuthology.orchestra.run.smithi039.stdout: "finisher-PurgeQueue": { 2023-07-21T19:06:59.649 INFO:teuthology.orchestra.run.smithi039.stdout: "queue_len": 0, 2023-07-21T19:06:59.649 INFO:teuthology.orchestra.run.smithi039.stdout: "complete_latency": { 2023-07-21T19:06:59.650 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 53426, 2023-07-21T19:06:59.650 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 11.078953589, 2023-07-21T19:06:59.650 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000207370 2023-07-21T19:06:59.650 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:59.650 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:59.650 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": { 2023-07-21T19:06:59.650 INFO:teuthology.orchestra.run.smithi039.stdout: "request": 209081, 2023-07-21T19:06:59.651 INFO:teuthology.orchestra.run.smithi039.stdout: "reply": 207108, 2023-07-21T19:06:59.651 INFO:teuthology.orchestra.run.smithi039.stdout: "reply_latency": { 2023-07-21T19:06:59.652 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 207108, 2023-07-21T19:06:59.652 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 930.549306513, 2023-07-21T19:06:59.652 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.004493063 2023-07-21T19:06:59.652 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:59.652 INFO:teuthology.orchestra.run.smithi039.stdout: "slow_reply": 0, 2023-07-21T19:06:59.652 INFO:teuthology.orchestra.run.smithi039.stdout: "forward": 1972, 2023-07-21T19:06:59.653 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_fetch_complete": 15, 2023-07-21T19:06:59.653 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_fetch_keys": 0, 2023-07-21T19:06:59.653 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_commit": 72, 2023-07-21T19:06:59.653 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_split": 3, 2023-07-21T19:06:59.654 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_merge": 0, 2023-07-21T19:06:59.654 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes": 2020, 2023-07-21T19:06:59.654 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_top": 525, 2023-07-21T19:06:59.654 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_bottom": 1074, 2023-07-21T19:06:59.654 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_pin_tail": 421, 2023-07-21T19:06:59.654 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_pinned": 1082, 2023-07-21T19:06:59.654 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_expired": 995, 2023-07-21T19:06:59.655 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_with_caps": 329, 2023-07-21T19:06:59.655 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": 329, 2023-07-21T19:06:59.655 INFO:teuthology.orchestra.run.smithi039.stdout: "subtrees": 6, 2023-07-21T19:06:59.655 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse": 342778, 2023-07-21T19:06:59.655 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_hit": 193012, 2023-07-21T19:06:59.656 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_forward": 1959, 2023-07-21T19:06:59.656 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_discover": 293, 2023-07-21T19:06:59.656 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_dir_fetch": 2, 2023-07-21T19:06:59.656 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_remote_ino": 0, 2023-07-21T19:06:59.656 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_lock": 4426, 2023-07-21T19:06:59.656 INFO:teuthology.orchestra.run.smithi039.stdout: "load_cent": 9666, 2023-07-21T19:06:59.656 INFO:teuthology.orchestra.run.smithi039.stdout: "q": 0, 2023-07-21T19:06:59.657 INFO:teuthology.orchestra.run.smithi039.stdout: "exported": 17, 2023-07-21T19:06:59.657 INFO:teuthology.orchestra.run.smithi039.stdout: "exported_inodes": 539, 2023-07-21T19:06:59.657 INFO:teuthology.orchestra.run.smithi039.stdout: "imported": 20, 2023-07-21T19:06:59.657 INFO:teuthology.orchestra.run.smithi039.stdout: "imported_inodes": 872, 2023-07-21T19:06:59.657 INFO:teuthology.orchestra.run.smithi039.stdout: "openino_dir_fetch": 0, 2023-07-21T19:06:59.657 INFO:teuthology.orchestra.run.smithi039.stdout: "openino_backtrace_fetch": 1, 2023-07-21T19:06:59.657 INFO:teuthology.orchestra.run.smithi039.stdout: "openino_peer_discover": 0, 2023-07-21T19:06:59.658 INFO:teuthology.orchestra.run.smithi039.stdout: "root_rfiles": 414, 2023-07-21T19:06:59.658 INFO:teuthology.orchestra.run.smithi039.stdout: "root_rbytes": 121586535, 2023-07-21T19:06:59.658 INFO:teuthology.orchestra.run.smithi039.stdout: "root_rsnaps": 7, 2023-07-21T19:06:59.658 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_backtrace_fetch": 1, 2023-07-21T19:06:59.658 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_set_tag": 0, 2023-07-21T19:06:59.658 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_backtrace_repaired": 0, 2023-07-21T19:06:59.658 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_inotable_repaired": 0, 2023-07-21T19:06:59.659 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_dir_inodes": 0, 2023-07-21T19:06:59.659 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_dir_base_inodes": 0, 2023-07-21T19:06:59.659 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_dirfrag_rstats": 0, 2023-07-21T19:06:59.659 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_file_inodes": 1, 2023-07-21T19:06:59.659 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_inode_file_caps": 151, 2023-07-21T19:06:59.659 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_revoke": 15410, 2023-07-21T19:06:59.659 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_grant": 117756, 2023-07-21T19:06:59.660 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_trunc": 0, 2023-07-21T19:06:59.660 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_flushsnap_ack": 1156, 2023-07-21T19:06:59.660 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-21T19:06:59.660 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_caps": 49463, 2023-07-21T19:06:59.660 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_caps_dirty": 22743, 2023-07-21T19:06:59.660 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_cap_release": 4937, 2023-07-21T19:06:59.660 INFO:teuthology.orchestra.run.smithi039.stdout: "process_request_cap_release": 194335 2023-07-21T19:06:59.661 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:59.661 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_cache": { 2023-07-21T19:06:59.661 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_update": 2801, 2023-07-21T19:06:59.661 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_update_receipt": 4741, 2023-07-21T19:06:59.661 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_try_discover": 2431, 2023-07-21T19:06:59.661 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_send_discover": 4826, 2023-07-21T19:06:59.661 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_handle_discover": 3277, 2023-07-21T19:06:59.662 INFO:teuthology.orchestra.run.smithi039.stdout: "num_strays": 345, 2023-07-21T19:06:59.662 INFO:teuthology.orchestra.run.smithi039.stdout: "num_strays_delayed": 0, 2023-07-21T19:06:59.662 INFO:teuthology.orchestra.run.smithi039.stdout: "num_strays_enqueuing": 0, 2023-07-21T19:06:59.662 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_created": 29361, 2023-07-21T19:06:59.662 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_enqueued": 29016, 2023-07-21T19:06:59.662 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_reintegrated": 0, 2023-07-21T19:06:59.662 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_migrated": 0, 2023-07-21T19:06:59.663 INFO:teuthology.orchestra.run.smithi039.stdout: "num_recovering_processing": 0, 2023-07-21T19:06:59.663 INFO:teuthology.orchestra.run.smithi039.stdout: "num_recovering_enqueued": 0, 2023-07-21T19:06:59.663 INFO:teuthology.orchestra.run.smithi039.stdout: "num_recovering_prioritized": 0, 2023-07-21T19:06:59.663 INFO:teuthology.orchestra.run.smithi039.stdout: "recovery_started": 0, 2023-07-21T19:06:59.663 INFO:teuthology.orchestra.run.smithi039.stdout: "recovery_completed": 0, 2023-07-21T19:06:59.663 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_enqueue_scrub": 0, 2023-07-21T19:06:59.663 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_exportdir": 18, 2023-07-21T19:06:59.664 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_flush": 0, 2023-07-21T19:06:59.664 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_fragmentdir": 3, 2023-07-21T19:06:59.664 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_fragstats": 0, 2023-07-21T19:06:59.664 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_inodestats": 0 2023-07-21T19:06:59.664 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:59.664 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_log": { 2023-07-21T19:06:59.664 INFO:teuthology.orchestra.run.smithi039.stdout: "evadd": 159806, 2023-07-21T19:06:59.664 INFO:teuthology.orchestra.run.smithi039.stdout: "evex": 96620, 2023-07-21T19:06:59.665 INFO:teuthology.orchestra.run.smithi039.stdout: "evtrm": 96620, 2023-07-21T19:06:59.665 INFO:teuthology.orchestra.run.smithi039.stdout: "ev": 63186, 2023-07-21T19:06:59.665 INFO:teuthology.orchestra.run.smithi039.stdout: "evexg": 0, 2023-07-21T19:06:59.665 INFO:teuthology.orchestra.run.smithi039.stdout: "evexd": 501, 2023-07-21T19:06:59.665 INFO:teuthology.orchestra.run.smithi039.stdout: "segadd": 260, 2023-07-21T19:06:59.665 INFO:teuthology.orchestra.run.smithi039.stdout: "segex": 132, 2023-07-21T19:06:59.665 INFO:teuthology.orchestra.run.smithi039.stdout: "segtrm": 132, 2023-07-21T19:06:59.666 INFO:teuthology.orchestra.run.smithi039.stdout: "seg": 129, 2023-07-21T19:06:59.666 INFO:teuthology.orchestra.run.smithi039.stdout: "segexg": 0, 2023-07-21T19:06:59.666 INFO:teuthology.orchestra.run.smithi039.stdout: "segexd": 1, 2023-07-21T19:06:59.666 INFO:teuthology.orchestra.run.smithi039.stdout: "expos": 436217215, 2023-07-21T19:06:59.666 INFO:teuthology.orchestra.run.smithi039.stdout: "wrpos": 962062957, 2023-07-21T19:06:59.666 INFO:teuthology.orchestra.run.smithi039.stdout: "rdpos": 0, 2023-07-21T19:06:59.666 INFO:teuthology.orchestra.run.smithi039.stdout: "jlat": { 2023-07-21T19:06:59.667 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 43051, 2023-07-21T19:06:59.667 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 380.093462333, 2023-07-21T19:06:59.667 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.008828911 2023-07-21T19:06:59.667 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:59.667 INFO:teuthology.orchestra.run.smithi039.stdout: "replayed": 0 2023-07-21T19:06:59.667 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:59.667 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_mem": { 2023-07-21T19:06:59.668 INFO:teuthology.orchestra.run.smithi039.stdout: "ino": 1651, 2023-07-21T19:06:59.668 INFO:teuthology.orchestra.run.smithi039.stdout: "ino+": 31381, 2023-07-21T19:06:59.668 INFO:teuthology.orchestra.run.smithi039.stdout: "ino-": 29730, 2023-07-21T19:06:59.668 INFO:teuthology.orchestra.run.smithi039.stdout: "dir": 93, 2023-07-21T19:06:59.668 INFO:teuthology.orchestra.run.smithi039.stdout: "dir+": 2525, 2023-07-21T19:06:59.668 INFO:teuthology.orchestra.run.smithi039.stdout: "dir-": 2432, 2023-07-21T19:06:59.668 INFO:teuthology.orchestra.run.smithi039.stdout: "dn": 2112, 2023-07-21T19:06:59.669 INFO:teuthology.orchestra.run.smithi039.stdout: "dn+": 50406, 2023-07-21T19:06:59.669 INFO:teuthology.orchestra.run.smithi039.stdout: "dn-": 48294, 2023-07-21T19:06:59.669 INFO:teuthology.orchestra.run.smithi039.stdout: "cap": 327, 2023-07-21T19:06:59.669 INFO:teuthology.orchestra.run.smithi039.stdout: "cap+": 31163, 2023-07-21T19:06:59.669 INFO:teuthology.orchestra.run.smithi039.stdout: "cap-": 30836, 2023-07-21T19:06:59.669 INFO:teuthology.orchestra.run.smithi039.stdout: "rss": 77524, 2023-07-21T19:06:59.669 INFO:teuthology.orchestra.run.smithi039.stdout: "heap": 198964 2023-07-21T19:06:59.670 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:59.670 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_server": { 2023-07-21T19:06:59.670 INFO:teuthology.orchestra.run.smithi039.stdout: "dispatch_client_request": 300410, 2023-07-21T19:06:59.670 INFO:teuthology.orchestra.run.smithi039.stdout: "dispatch_server_request": 27369, 2023-07-21T19:06:59.670 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_request": 209081, 2023-07-21T19:06:59.670 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_session": 2836, 2023-07-21T19:06:59.670 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_peer_request": 66696, 2023-07-21T19:06:59.671 INFO:teuthology.orchestra.run.smithi039.stdout: "req_create_latency": { 2023-07-21T19:06:59.671 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 29140, 2023-07-21T19:06:59.671 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 233.136521742, 2023-07-21T19:06:59.671 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.008000566 2023-07-21T19:06:59.671 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:59.671 INFO:teuthology.orchestra.run.smithi039.stdout: "req_getattr_latency": { 2023-07-21T19:06:59.671 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 120, 2023-07-21T19:06:59.672 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.240458023, 2023-07-21T19:06:59.672 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.002003816 2023-07-21T19:06:59.672 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:59.672 INFO:teuthology.orchestra.run.smithi039.stdout: "req_getfilelock_latency": { 2023-07-21T19:06:59.672 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:06:59.672 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:06:59.672 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:06:59.673 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:59.673 INFO:teuthology.orchestra.run.smithi039.stdout: "req_link_latency": { 2023-07-21T19:06:59.673 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:06:59.673 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:06:59.673 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:06:59.673 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:59.673 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookup_latency": { 2023-07-21T19:06:59.674 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 44009, 2023-07-21T19:06:59.674 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 115.193402205, 2023-07-21T19:06:59.674 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.002617496 2023-07-21T19:06:59.674 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:59.674 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookuphash_latency": { 2023-07-21T19:06:59.674 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:06:59.674 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:06:59.674 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:06:59.675 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:59.675 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupino_latency": { 2023-07-21T19:06:59.675 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 1, 2023-07-21T19:06:59.675 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000228541, 2023-07-21T19:06:59.675 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000228541 2023-07-21T19:06:59.676 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:59.676 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupname_latency": { 2023-07-21T19:06:59.676 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:06:59.676 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:06:59.676 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:06:59.676 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:59.676 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupparent_latency": { 2023-07-21T19:06:59.677 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:06:59.677 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:06:59.677 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:06:59.677 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:59.677 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupsnap_latency": { 2023-07-21T19:06:59.677 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:06:59.678 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:06:59.678 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:06:59.678 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:59.678 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lssnap_latency": { 2023-07-21T19:06:59.678 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:06:59.678 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:06:59.679 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:06:59.679 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:59.679 INFO:teuthology.orchestra.run.smithi039.stdout: "req_mkdir_latency": { 2023-07-21T19:06:59.679 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:06:59.679 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:06:59.679 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:06:59.679 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:59.680 INFO:teuthology.orchestra.run.smithi039.stdout: "req_mknod_latency": { 2023-07-21T19:06:59.680 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:06:59.680 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:06:59.680 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:06:59.680 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:59.680 INFO:teuthology.orchestra.run.smithi039.stdout: "req_mksnap_latency": { 2023-07-21T19:06:59.680 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:06:59.681 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:06:59.681 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:06:59.681 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:59.681 INFO:teuthology.orchestra.run.smithi039.stdout: "req_open_latency": { 2023-07-21T19:06:59.681 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 2, 2023-07-21T19:06:59.681 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.001524390, 2023-07-21T19:06:59.681 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000762195 2023-07-21T19:06:59.682 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:59.682 INFO:teuthology.orchestra.run.smithi039.stdout: "req_readdir_latency": { 2023-07-21T19:06:59.682 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 99476, 2023-07-21T19:06:59.682 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 235.890454621, 2023-07-21T19:06:59.682 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.002371330 2023-07-21T19:06:59.682 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:59.682 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rename_latency": { 2023-07-21T19:06:59.682 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 4536, 2023-07-21T19:06:59.683 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 195.526251274, 2023-07-21T19:06:59.683 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.043105434 2023-07-21T19:06:59.683 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:59.683 INFO:teuthology.orchestra.run.smithi039.stdout: "req_renamesnap_latency": { 2023-07-21T19:06:59.683 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:06:59.683 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:06:59.683 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:06:59.684 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:59.684 INFO:teuthology.orchestra.run.smithi039.stdout: "req_snapdiff_latency": { 2023-07-21T19:06:59.684 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:06:59.684 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:06:59.684 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:06:59.684 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:59.684 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rmdir_latency": { 2023-07-21T19:06:59.685 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 20, 2023-07-21T19:06:59.685 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.470641348, 2023-07-21T19:06:59.685 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.023532067 2023-07-21T19:06:59.685 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:59.685 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rmsnap_latency": { 2023-07-21T19:06:59.685 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:06:59.686 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:06:59.686 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:06:59.686 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:59.686 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rmxattr_latency": { 2023-07-21T19:06:59.686 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:06:59.686 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:06:59.686 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:06:59.687 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:59.687 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setattr_latency": { 2023-07-21T19:06:59.687 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 31, 2023-07-21T19:06:59.687 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.725168320, 2023-07-21T19:06:59.687 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.023392526 2023-07-21T19:06:59.687 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:59.687 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setdirlayout_latency": { 2023-07-21T19:06:59.688 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:06:59.688 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:06:59.688 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:06:59.688 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:59.688 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setfilelock_latency": { 2023-07-21T19:06:59.688 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 432, 2023-07-21T19:06:59.688 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.705842976, 2023-07-21T19:06:59.689 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.001633895 2023-07-21T19:06:59.689 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:59.689 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setlayout_latency": { 2023-07-21T19:06:59.689 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:06:59.689 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:06:59.689 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:06:59.689 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:59.690 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setxattr_latency": { 2023-07-21T19:06:59.690 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:06:59.690 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:06:59.690 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:06:59.690 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:59.690 INFO:teuthology.orchestra.run.smithi039.stdout: "req_symlink_latency": { 2023-07-21T19:06:59.690 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:06:59.691 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:06:59.691 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:06:59.691 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:59.691 INFO:teuthology.orchestra.run.smithi039.stdout: "req_unlink_latency": { 2023-07-21T19:06:59.691 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 29341, 2023-07-21T19:06:59.691 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 148.658813073, 2023-07-21T19:06:59.691 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.005066589 2023-07-21T19:06:59.691 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:59.692 INFO:teuthology.orchestra.run.smithi039.stdout: "cap_revoke_eviction": 0, 2023-07-21T19:06:59.692 INFO:teuthology.orchestra.run.smithi039.stdout: "cap_acquisition_throttle": 0, 2023-07-21T19:06:59.692 INFO:teuthology.orchestra.run.smithi039.stdout: "req_getvxattr_latency": { 2023-07-21T19:06:59.692 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:06:59.692 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:06:59.692 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:06:59.693 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:59.693 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:59.693 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_sessions": { 2023-07-21T19:06:59.693 INFO:teuthology.orchestra.run.smithi039.stdout: "session_count": 1, 2023-07-21T19:06:59.693 INFO:teuthology.orchestra.run.smithi039.stdout: "session_add": 2, 2023-07-21T19:06:59.693 INFO:teuthology.orchestra.run.smithi039.stdout: "session_remove": 1, 2023-07-21T19:06:59.693 INFO:teuthology.orchestra.run.smithi039.stdout: "sessions_open": 1, 2023-07-21T19:06:59.694 INFO:teuthology.orchestra.run.smithi039.stdout: "sessions_stale": 0, 2023-07-21T19:06:59.694 INFO:teuthology.orchestra.run.smithi039.stdout: "total_load": 21145, 2023-07-21T19:06:59.694 INFO:teuthology.orchestra.run.smithi039.stdout: "average_load": 21145, 2023-07-21T19:06:59.694 INFO:teuthology.orchestra.run.smithi039.stdout: "avg_session_uptime": 1527 2023-07-21T19:06:59.694 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:59.695 INFO:teuthology.orchestra.run.smithi039.stdout: "mempool": { 2023-07-21T19:06:59.695 INFO:teuthology.orchestra.run.smithi039.stdout: "bloom_filter_bytes": 0, 2023-07-21T19:06:59.695 INFO:teuthology.orchestra.run.smithi039.stdout: "bloom_filter_items": 0, 2023-07-21T19:06:59.695 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_alloc_bytes": 0, 2023-07-21T19:06:59.695 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_alloc_items": 0, 2023-07-21T19:06:59.695 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_data_bytes": 0, 2023-07-21T19:06:59.695 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_data_items": 0, 2023-07-21T19:06:59.696 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-21T19:06:59.696 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_onode_items": 0, 2023-07-21T19:06:59.696 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-21T19:06:59.696 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_meta_items": 0, 2023-07-21T19:06:59.696 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_other_bytes": 0, 2023-07-21T19:06:59.696 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_other_items": 0, 2023-07-21T19:06:59.696 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-21T19:06:59.697 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_buffer_items": 0, 2023-07-21T19:06:59.697 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_extent_bytes": 0, 2023-07-21T19:06:59.697 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_extent_items": 0, 2023-07-21T19:06:59.697 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_blob_bytes": 0, 2023-07-21T19:06:59.697 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_blob_items": 0, 2023-07-21T19:06:59.697 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-21T19:06:59.697 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_shared_blob_items": 0, 2023-07-21T19:06:59.698 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-21T19:06:59.698 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_inline_bl_items": 0, 2023-07-21T19:06:59.698 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_bytes": 0, 2023-07-21T19:06:59.698 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_items": 0, 2023-07-21T19:06:59.698 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_txc_bytes": 0, 2023-07-21T19:06:59.698 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_txc_items": 0, 2023-07-21T19:06:59.698 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-21T19:06:59.699 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_deferred_items": 0, 2023-07-21T19:06:59.699 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_bytes": 0, 2023-07-21T19:06:59.699 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_items": 0, 2023-07-21T19:06:59.699 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_bytes": 0, 2023-07-21T19:06:59.699 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_items": 0, 2023-07-21T19:06:59.699 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_reader_bytes": 0, 2023-07-21T19:06:59.699 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_reader_items": 0, 2023-07-21T19:06:59.700 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_writer_bytes": 0, 2023-07-21T19:06:59.700 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_writer_items": 0, 2023-07-21T19:06:59.700 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_anon_bytes": 868724, 2023-07-21T19:06:59.700 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_anon_items": 1353, 2023-07-21T19:06:59.700 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_meta_bytes": 0, 2023-07-21T19:06:59.700 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_meta_items": 0, 2023-07-21T19:06:59.700 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_bytes": 0, 2023-07-21T19:06:59.701 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_items": 0, 2023-07-21T19:06:59.701 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mapbl_bytes": 0, 2023-07-21T19:06:59.701 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mapbl_items": 0, 2023-07-21T19:06:59.701 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pglog_bytes": 0, 2023-07-21T19:06:59.701 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pglog_items": 0, 2023-07-21T19:06:59.701 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_bytes": 8160, 2023-07-21T19:06:59.701 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_items": 186, 2023-07-21T19:06:59.702 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_mapping_bytes": 0, 2023-07-21T19:06:59.702 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_mapping_items": 0, 2023-07-21T19:06:59.702 INFO:teuthology.orchestra.run.smithi039.stdout: "pgmap_bytes": 0, 2023-07-21T19:06:59.702 INFO:teuthology.orchestra.run.smithi039.stdout: "pgmap_items": 0, 2023-07-21T19:06:59.702 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_co_bytes": 6088569, 2023-07-21T19:06:59.702 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_co_items": 91000, 2023-07-21T19:06:59.702 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_1_bytes": 0, 2023-07-21T19:06:59.703 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_1_items": 0, 2023-07-21T19:06:59.703 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_2_bytes": 0, 2023-07-21T19:06:59.703 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_2_items": 0 2023-07-21T19:06:59.703 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:59.703 INFO:teuthology.orchestra.run.smithi039.stdout: "objecter": { 2023-07-21T19:06:59.704 INFO:teuthology.orchestra.run.smithi039.stdout: "op_active": 0, 2023-07-21T19:06:59.704 INFO:teuthology.orchestra.run.smithi039.stdout: "op_laggy": 0, 2023-07-21T19:06:59.704 INFO:teuthology.orchestra.run.smithi039.stdout: "op_send": 75973, 2023-07-21T19:06:59.704 INFO:teuthology.orchestra.run.smithi039.stdout: "op_send_bytes": 963435042, 2023-07-21T19:06:59.704 INFO:teuthology.orchestra.run.smithi039.stdout: "op_resend": 33, 2023-07-21T19:06:59.704 INFO:teuthology.orchestra.run.smithi039.stdout: "op_reply": 75973, 2023-07-21T19:06:59.704 INFO:teuthology.orchestra.run.smithi039.stdout: "op_latency": { 2023-07-21T19:06:59.705 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 75973, 2023-07-21T19:06:59.705 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 694.549019155, 2023-07-21T19:06:59.705 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.009142050 2023-07-21T19:06:59.705 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:59.705 INFO:teuthology.orchestra.run.smithi039.stdout: "op_inflight": 0, 2023-07-21T19:06:59.705 INFO:teuthology.orchestra.run.smithi039.stdout: "oplen_avg": { 2023-07-21T19:06:59.705 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 75973, 2023-07-21T19:06:59.706 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 76757 2023-07-21T19:06:59.706 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:59.706 INFO:teuthology.orchestra.run.smithi039.stdout: "op": 75973, 2023-07-21T19:06:59.706 INFO:teuthology.orchestra.run.smithi039.stdout: "op_r": 1248, 2023-07-21T19:06:59.706 INFO:teuthology.orchestra.run.smithi039.stdout: "op_w": 74725, 2023-07-21T19:06:59.706 INFO:teuthology.orchestra.run.smithi039.stdout: "op_rmw": 0, 2023-07-21T19:06:59.706 INFO:teuthology.orchestra.run.smithi039.stdout: "op_pg": 0, 2023-07-21T19:06:59.707 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_stat": 37, 2023-07-21T19:06:59.707 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_create": 32, 2023-07-21T19:06:59.707 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_read": 1201, 2023-07-21T19:06:59.707 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_write": 44252, 2023-07-21T19:06:59.707 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_writefull": 716, 2023-07-21T19:06:59.707 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_writesame": 0, 2023-07-21T19:06:59.707 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_append": 0, 2023-07-21T19:06:59.708 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_zero": 0, 2023-07-21T19:06:59.708 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_truncate": 3, 2023-07-21T19:06:59.708 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_delete": 29359, 2023-07-21T19:06:59.708 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_mapext": 0, 2023-07-21T19:06:59.708 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_sparse_read": 0, 2023-07-21T19:06:59.708 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_clonerange": 0, 2023-07-21T19:06:59.708 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_getxattr": 8, 2023-07-21T19:06:59.709 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_setxattr": 64, 2023-07-21T19:06:59.709 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_cmpxattr": 0, 2023-07-21T19:06:59.709 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_rmxattr": 0, 2023-07-21T19:06:59.709 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_resetxattrs": 0, 2023-07-21T19:06:59.709 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_call": 0, 2023-07-21T19:06:59.709 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_watch": 0, 2023-07-21T19:06:59.709 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_notify": 0, 2023-07-21T19:06:59.710 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_src_cmpxattr": 0, 2023-07-21T19:06:59.710 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_pgls": 0, 2023-07-21T19:06:59.710 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_pgls_filter": 0, 2023-07-21T19:06:59.710 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_other": 0, 2023-07-21T19:06:59.710 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_active": 0, 2023-07-21T19:06:59.710 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_send": 0, 2023-07-21T19:06:59.710 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_resend": 0, 2023-07-21T19:06:59.710 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_ping": 0, 2023-07-21T19:06:59.711 INFO:teuthology.orchestra.run.smithi039.stdout: "poolop_active": 0, 2023-07-21T19:06:59.711 INFO:teuthology.orchestra.run.smithi039.stdout: "poolop_send": 0, 2023-07-21T19:06:59.711 INFO:teuthology.orchestra.run.smithi039.stdout: "poolop_resend": 0, 2023-07-21T19:06:59.711 INFO:teuthology.orchestra.run.smithi039.stdout: "poolstat_active": 0, 2023-07-21T19:06:59.711 INFO:teuthology.orchestra.run.smithi039.stdout: "poolstat_send": 0, 2023-07-21T19:06:59.711 INFO:teuthology.orchestra.run.smithi039.stdout: "poolstat_resend": 0, 2023-07-21T19:06:59.711 INFO:teuthology.orchestra.run.smithi039.stdout: "statfs_active": 0, 2023-07-21T19:06:59.712 INFO:teuthology.orchestra.run.smithi039.stdout: "statfs_send": 0, 2023-07-21T19:06:59.712 INFO:teuthology.orchestra.run.smithi039.stdout: "statfs_resend": 0, 2023-07-21T19:06:59.712 INFO:teuthology.orchestra.run.smithi039.stdout: "command_active": 0, 2023-07-21T19:06:59.712 INFO:teuthology.orchestra.run.smithi039.stdout: "command_send": 0, 2023-07-21T19:06:59.712 INFO:teuthology.orchestra.run.smithi039.stdout: "command_resend": 0, 2023-07-21T19:06:59.712 INFO:teuthology.orchestra.run.smithi039.stdout: "map_epoch": 122, 2023-07-21T19:06:59.712 INFO:teuthology.orchestra.run.smithi039.stdout: "map_full": 0, 2023-07-21T19:06:59.713 INFO:teuthology.orchestra.run.smithi039.stdout: "map_inc": 43, 2023-07-21T19:06:59.713 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_sessions": 12, 2023-07-21T19:06:59.713 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_session_open": 12, 2023-07-21T19:06:59.713 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_session_close": 0, 2023-07-21T19:06:59.713 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_laggy": 0, 2023-07-21T19:06:59.713 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_wr": 709, 2023-07-21T19:06:59.713 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_rd": 61, 2023-07-21T19:06:59.714 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_del": 315 2023-07-21T19:06:59.714 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:59.714 INFO:teuthology.orchestra.run.smithi039.stdout: "oft": { 2023-07-21T19:06:59.714 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_objs": 1, 2023-07-21T19:06:59.714 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_kv_pairs": 0, 2023-07-21T19:06:59.714 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_updates": 0, 2023-07-21T19:06:59.714 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_removes": 0 2023-07-21T19:06:59.715 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:59.715 INFO:teuthology.orchestra.run.smithi039.stdout: "osd-slow-ops": { 2023-07-21T19:06:59.715 INFO:teuthology.orchestra.run.smithi039.stdout: "slow_ops_count": 236073 2023-07-21T19:06:59.715 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:59.715 INFO:teuthology.orchestra.run.smithi039.stdout: "purge_queue": { 2023-07-21T19:06:59.715 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing_ops": 0, 2023-07-21T19:06:59.715 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing_ops_high_water": 6, 2023-07-21T19:06:59.716 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing": 0, 2023-07-21T19:06:59.716 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing_high_water": 6, 2023-07-21T19:06:59.716 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executed": 29016, 2023-07-21T19:06:59.716 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_item_in_journal": 0 2023-07-21T19:06:59.716 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:59.716 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-21T19:06:59.716 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2023-07-21T19:06:59.717 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 104857600, 2023-07-21T19:06:59.717 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2023-07-21T19:06:59.717 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 1227324, 2023-07-21T19:06:59.717 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 278368093, 2023-07-21T19:06:59.717 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2023-07-21T19:06:59.717 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 1227324, 2023-07-21T19:06:59.717 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 0, 2023-07-21T19:06:59.718 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 0, 2023-07-21T19:06:59.718 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 1227324, 2023-07-21T19:06:59.718 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 278368093, 2023-07-21T19:06:59.718 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2023-07-21T19:06:59.718 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:06:59.718 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:06:59.718 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:06:59.719 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:59.719 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:59.719 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-objecter_bytes": { 2023-07-21T19:06:59.719 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2023-07-21T19:06:59.719 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 104857600, 2023-07-21T19:06:59.719 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2023-07-21T19:06:59.719 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 0, 2023-07-21T19:06:59.720 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 0, 2023-07-21T19:06:59.720 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2023-07-21T19:06:59.720 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 0, 2023-07-21T19:06:59.720 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 75973, 2023-07-21T19:06:59.720 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 966373644, 2023-07-21T19:06:59.720 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 75973, 2023-07-21T19:06:59.720 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 966373644, 2023-07-21T19:06:59.721 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2023-07-21T19:06:59.721 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:06:59.721 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:06:59.721 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:06:59.721 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:59.721 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:59.721 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-objecter_ops": { 2023-07-21T19:06:59.722 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2023-07-21T19:06:59.722 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 1024, 2023-07-21T19:06:59.722 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2023-07-21T19:06:59.722 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 0, 2023-07-21T19:06:59.722 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 0, 2023-07-21T19:06:59.722 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2023-07-21T19:06:59.722 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 0, 2023-07-21T19:06:59.723 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 75973, 2023-07-21T19:06:59.723 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 75973, 2023-07-21T19:06:59.723 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 75973, 2023-07-21T19:06:59.723 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 75973, 2023-07-21T19:06:59.723 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2023-07-21T19:06:59.723 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:06:59.723 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:06:59.724 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:06:59.724 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:59.724 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:59.724 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-write_buf_throttle": { 2023-07-21T19:06:59.724 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2023-07-21T19:06:59.724 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 3758096384, 2023-07-21T19:06:59.725 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2023-07-21T19:06:59.725 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 29016, 2023-07-21T19:06:59.725 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 2939752, 2023-07-21T19:06:59.725 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2023-07-21T19:06:59.725 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 29016, 2023-07-21T19:06:59.725 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 0, 2023-07-21T19:06:59.725 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 0, 2023-07-21T19:06:59.726 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 1201, 2023-07-21T19:06:59.726 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 2939752, 2023-07-21T19:06:59.726 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2023-07-21T19:06:59.726 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:06:59.726 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:06:59.727 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:06:59.727 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:59.727 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:06:59.727 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-write_buf_throttle-0x562a0e6560a0": { 2023-07-21T19:06:59.727 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 1198, 2023-07-21T19:06:59.727 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 3758096384, 2023-07-21T19:06:59.727 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2023-07-21T19:06:59.728 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 159806, 2023-07-21T19:06:59.728 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 957868653, 2023-07-21T19:06:59.728 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2023-07-21T19:06:59.728 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 159806, 2023-07-21T19:06:59.728 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 0, 2023-07-21T19:06:59.728 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 0, 2023-07-21T19:06:59.729 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 43051, 2023-07-21T19:06:59.729 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 957867455, 2023-07-21T19:06:59.729 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2023-07-21T19:06:59.729 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:06:59.729 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:06:59.729 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:06:59.730 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:59.730 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:06:59.730 INFO:teuthology.orchestra.run.smithi039.stdout:} 2023-07-21T19:06:59.802 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:06:59 smithi039 ceph-mon[96489]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T19:06:59.802 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:06:59 smithi039 ceph-mon[96489]: pgmap v800: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.0 MiB/s wr, 211 op/s 2023-07-21T19:06:59.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:06:59 smithi193 ceph-mon[98204]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T19:06:59.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:06:59 smithi193 ceph-mon[98204]: pgmap v800: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.0 MiB/s wr, 211 op/s 2023-07-21T19:06:59.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:06:59 smithi155 ceph-mon[97934]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T19:06:59.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:06:59 smithi155 ceph-mon[97934]: pgmap v800: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.0 MiB/s wr, 211 op/s 2023-07-21T19:07:00.134 INFO:tasks.check_counter:Daemon mds.a mds.exported=17 2023-07-21T19:07:00.134 INFO:tasks.check_counter:Daemon mds.a mds.imported=20 2023-07-21T19:07:00.134 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=2801 2023-07-21T19:07:00.134 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=4741 2023-07-21T19:07:00.135 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=7 2023-07-21T19:07:00.135 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2023-07-21T19:07:00.135 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2023-07-21T19:07:00.135 DEBUG:tasks.check_counter:Getting stats from d 2023-07-21T19:07:00.135 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph daemon mds.d perf dump 2023-07-21T19:07:00.717 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T19:07:02.775 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:02 smithi039 ceph-mon[96489]: pgmap v801: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.0 MiB/s wr, 223 op/s 2023-07-21T19:07:02.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:02 smithi193 ceph-mon[98204]: pgmap v801: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.0 MiB/s wr, 223 op/s 2023-07-21T19:07:02.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:02 smithi155 ceph-mon[97934]: pgmap v801: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.0 MiB/s wr, 223 op/s 2023-07-21T19:07:03.851 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2023-07-21T19:07:03.851 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-0": { 2023-07-21T19:07:03.851 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 0, 2023-07-21T19:07:03.851 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 2, 2023-07-21T19:07:03.852 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 0, 2023-07-21T19:07:03.852 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 0, 2023-07-21T19:07:03.852 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 4, 2023-07-21T19:07:03.852 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 0, 2023-07-21T19:07:03.852 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.001499410, 2023-07-21T19:07:03.852 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.000000000, 2023-07-21T19:07:03.852 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.000294493, 2023-07-21T19:07:03.853 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-21T19:07:03.853 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:07:03.853 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:07:03.853 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:07:03.853 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:07:03.853 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:03.853 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:07:03.854 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:07:03.854 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:07:03.854 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:07:03.854 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:03.854 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-21T19:07:03.854 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 0 2023-07-21T19:07:03.855 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:03.855 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-1": { 2023-07-21T19:07:03.855 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 445, 2023-07-21T19:07:03.855 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 436, 2023-07-21T19:07:03.855 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 1350448, 2023-07-21T19:07:03.855 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 249952, 2023-07-21T19:07:03.855 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 2, 2023-07-21T19:07:03.856 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 1, 2023-07-21T19:07:03.856 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.064936317, 2023-07-21T19:07:03.856 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.023870685, 2023-07-21T19:07:03.856 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.034802435, 2023-07-21T19:07:03.856 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.010494804, 2023-07-21T19:07:03.857 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:07:03.857 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 435, 2023-07-21T19:07:03.857 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.025347089, 2023-07-21T19:07:03.857 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000058269 2023-07-21T19:07:03.857 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:03.857 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:07:03.857 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:07:03.857 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:07:03.858 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:07:03.858 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:03.858 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 1350448, 2023-07-21T19:07:03.858 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 249952 2023-07-21T19:07:03.858 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:03.858 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-2": { 2023-07-21T19:07:03.858 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 7, 2023-07-21T19:07:03.858 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 336, 2023-07-21T19:07:03.859 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 3472, 2023-07-21T19:07:03.859 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 286608, 2023-07-21T19:07:03.859 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 6, 2023-07-21T19:07:03.859 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 3, 2023-07-21T19:07:03.859 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.022351096, 2023-07-21T19:07:03.859 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.016141505, 2023-07-21T19:07:03.859 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.002547346, 2023-07-21T19:07:03.860 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-21T19:07:03.860 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:07:03.860 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 334, 2023-07-21T19:07:03.860 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.032724187, 2023-07-21T19:07:03.860 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000097976 2023-07-21T19:07:03.860 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:03.860 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:07:03.860 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:07:03.861 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:07:03.861 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:07:03.861 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:03.861 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 3472, 2023-07-21T19:07:03.861 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 286608 2023-07-21T19:07:03.861 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:03.861 INFO:teuthology.orchestra.run.smithi039.stdout: "cct": { 2023-07-21T19:07:03.861 INFO:teuthology.orchestra.run.smithi039.stdout: "total_workers": 0, 2023-07-21T19:07:03.862 INFO:teuthology.orchestra.run.smithi039.stdout: "unhealthy_workers": 0 2023-07-21T19:07:03.862 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:03.862 INFO:teuthology.orchestra.run.smithi039.stdout: "mempool": { 2023-07-21T19:07:03.862 INFO:teuthology.orchestra.run.smithi039.stdout: "bloom_filter_bytes": 0, 2023-07-21T19:07:03.862 INFO:teuthology.orchestra.run.smithi039.stdout: "bloom_filter_items": 0, 2023-07-21T19:07:03.862 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_alloc_bytes": 0, 2023-07-21T19:07:03.862 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_alloc_items": 0, 2023-07-21T19:07:03.862 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_data_bytes": 0, 2023-07-21T19:07:03.863 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_data_items": 0, 2023-07-21T19:07:03.863 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-21T19:07:03.863 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_onode_items": 0, 2023-07-21T19:07:03.863 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-21T19:07:03.863 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_meta_items": 0, 2023-07-21T19:07:03.863 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_other_bytes": 0, 2023-07-21T19:07:03.863 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_other_items": 0, 2023-07-21T19:07:03.864 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-21T19:07:03.864 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_buffer_items": 0, 2023-07-21T19:07:03.864 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_extent_bytes": 0, 2023-07-21T19:07:03.864 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_extent_items": 0, 2023-07-21T19:07:03.864 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_blob_bytes": 0, 2023-07-21T19:07:03.864 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_blob_items": 0, 2023-07-21T19:07:03.864 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-21T19:07:03.864 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_shared_blob_items": 0, 2023-07-21T19:07:03.865 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-21T19:07:03.865 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_inline_bl_items": 0, 2023-07-21T19:07:03.865 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_bytes": 0, 2023-07-21T19:07:03.865 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_items": 0, 2023-07-21T19:07:03.865 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_txc_bytes": 0, 2023-07-21T19:07:03.865 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_txc_items": 0, 2023-07-21T19:07:03.865 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-21T19:07:03.866 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_deferred_items": 0, 2023-07-21T19:07:03.866 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_bytes": 0, 2023-07-21T19:07:03.866 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_items": 0, 2023-07-21T19:07:03.866 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_bytes": 0, 2023-07-21T19:07:03.866 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_items": 0, 2023-07-21T19:07:03.866 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_reader_bytes": 0, 2023-07-21T19:07:03.866 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_reader_items": 0, 2023-07-21T19:07:03.866 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_writer_bytes": 0, 2023-07-21T19:07:03.867 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_writer_items": 0, 2023-07-21T19:07:03.867 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_anon_bytes": 148714, 2023-07-21T19:07:03.867 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_anon_items": 30, 2023-07-21T19:07:03.867 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_meta_bytes": 0, 2023-07-21T19:07:03.867 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_meta_items": 0, 2023-07-21T19:07:03.867 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_bytes": 0, 2023-07-21T19:07:03.867 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_items": 0, 2023-07-21T19:07:03.867 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mapbl_bytes": 0, 2023-07-21T19:07:03.868 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mapbl_items": 0, 2023-07-21T19:07:03.868 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pglog_bytes": 0, 2023-07-21T19:07:03.868 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pglog_items": 0, 2023-07-21T19:07:03.868 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_bytes": 0, 2023-07-21T19:07:03.868 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_items": 0, 2023-07-21T19:07:03.868 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_mapping_bytes": 0, 2023-07-21T19:07:03.868 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_mapping_items": 0, 2023-07-21T19:07:03.868 INFO:teuthology.orchestra.run.smithi039.stdout: "pgmap_bytes": 0, 2023-07-21T19:07:03.869 INFO:teuthology.orchestra.run.smithi039.stdout: "pgmap_items": 0, 2023-07-21T19:07:03.869 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_co_bytes": 928, 2023-07-21T19:07:03.869 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_co_items": 2, 2023-07-21T19:07:03.869 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_1_bytes": 0, 2023-07-21T19:07:03.869 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_1_items": 0, 2023-07-21T19:07:03.869 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_2_bytes": 0, 2023-07-21T19:07:03.869 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_2_items": 0 2023-07-21T19:07:03.869 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:03.870 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-21T19:07:03.870 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2023-07-21T19:07:03.870 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 104857600, 2023-07-21T19:07:03.870 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2023-07-21T19:07:03.870 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 448, 2023-07-21T19:07:03.870 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 1287558, 2023-07-21T19:07:03.870 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2023-07-21T19:07:03.870 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 448, 2023-07-21T19:07:03.871 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 0, 2023-07-21T19:07:03.871 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 0, 2023-07-21T19:07:03.871 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 448, 2023-07-21T19:07:03.871 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 1287558, 2023-07-21T19:07:03.871 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2023-07-21T19:07:03.871 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:07:03.871 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:07:03.871 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:07:03.872 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:07:03.872 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:07:03.872 INFO:teuthology.orchestra.run.smithi039.stdout:} 2023-07-21T19:07:04.288 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2023-07-21T19:07:04.289 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2023-07-21T19:07:04.290 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2023-07-21T19:07:04.290 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2023-07-21T19:07:04.290 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2023-07-21T19:07:04.290 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2023-07-21T19:07:04.290 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2023-07-21T19:07:04.291 DEBUG:tasks.check_counter:Getting stats from g 2023-07-21T19:07:04.291 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph daemon mds.g perf dump 2023-07-21T19:07:04.680 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:04 smithi039 ceph-mon[96489]: pgmap v802: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 8.7 MiB/s wr, 308 op/s 2023-07-21T19:07:04.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:04 smithi193 ceph-mon[98204]: pgmap v802: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 8.7 MiB/s wr, 308 op/s 2023-07-21T19:07:04.879 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:04 smithi155 ceph-mon[97934]: pgmap v802: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 8.7 MiB/s wr, 308 op/s 2023-07-21T19:07:04.919 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T19:07:05.559 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:05 smithi039 ceph-mon[96489]: osdmap e123: 12 total, 12 up, 12 in 2023-07-21T19:07:05.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:05 smithi193 ceph-mon[98204]: osdmap e123: 12 total, 12 up, 12 in 2023-07-21T19:07:05.878 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:05 smithi155 ceph-mon[97934]: osdmap e123: 12 total, 12 up, 12 in 2023-07-21T19:07:06.501 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2023-07-21T19:07:06.501 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-0": { 2023-07-21T19:07:06.502 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 2795, 2023-07-21T19:07:06.502 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 2797, 2023-07-21T19:07:06.502 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 277666622, 2023-07-21T19:07:06.503 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 869246, 2023-07-21T19:07:06.503 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 9, 2023-07-21T19:07:06.503 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 4, 2023-07-21T19:07:06.504 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.402916199, 2023-07-21T19:07:06.504 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.086024781, 2023-07-21T19:07:06.504 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.207993858, 2023-07-21T19:07:06.504 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.085364198, 2023-07-21T19:07:06.506 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:07:06.507 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 2795, 2023-07-21T19:07:06.507 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.211980893, 2023-07-21T19:07:06.507 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000075842 2023-07-21T19:07:06.507 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:06.507 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:07:06.507 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:07:06.507 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:07:06.508 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:07:06.508 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:06.508 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-21T19:07:06.508 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 0 2023-07-21T19:07:06.508 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:06.508 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-1": { 2023-07-21T19:07:06.509 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 2805, 2023-07-21T19:07:06.509 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 2710, 2023-07-21T19:07:06.509 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 304496471, 2023-07-21T19:07:06.509 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 949824, 2023-07-21T19:07:06.509 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 5, 2023-07-21T19:07:06.510 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 4, 2023-07-21T19:07:06.510 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.439666115, 2023-07-21T19:07:06.510 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.065176841, 2023-07-21T19:07:06.510 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.296724555, 2023-07-21T19:07:06.510 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.070187113, 2023-07-21T19:07:06.510 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:07:06.510 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 2710, 2023-07-21T19:07:06.511 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.111386016, 2023-07-21T19:07:06.511 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000041101 2023-07-21T19:07:06.511 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:06.511 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:07:06.511 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:07:06.511 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:07:06.511 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:07:06.511 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:06.512 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 1909760, 2023-07-21T19:07:06.512 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 257792 2023-07-21T19:07:06.512 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:06.512 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-2": { 2023-07-21T19:07:06.512 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 1505, 2023-07-21T19:07:06.512 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 1834, 2023-07-21T19:07:06.512 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 382083306, 2023-07-21T19:07:06.513 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 1279085, 2023-07-21T19:07:06.513 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 9, 2023-07-21T19:07:06.513 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 6, 2023-07-21T19:07:06.513 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.373546126, 2023-07-21T19:07:06.513 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.046853833, 2023-07-21T19:07:06.513 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.266492908, 2023-07-21T19:07:06.513 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.044655629, 2023-07-21T19:07:06.514 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:07:06.514 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 1831, 2023-07-21T19:07:06.514 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.140718924, 2023-07-21T19:07:06.514 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000076853 2023-07-21T19:07:06.514 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:06.514 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:07:06.514 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:07:06.515 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:07:06.515 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:07:06.515 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:06.515 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 480, 2023-07-21T19:07:06.515 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 811952 2023-07-21T19:07:06.515 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:06.515 INFO:teuthology.orchestra.run.smithi039.stdout: "cct": { 2023-07-21T19:07:06.516 INFO:teuthology.orchestra.run.smithi039.stdout: "total_workers": 1, 2023-07-21T19:07:06.516 INFO:teuthology.orchestra.run.smithi039.stdout: "unhealthy_workers": 0 2023-07-21T19:07:06.516 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:06.516 INFO:teuthology.orchestra.run.smithi039.stdout: "finisher-MDSRank": { 2023-07-21T19:07:06.516 INFO:teuthology.orchestra.run.smithi039.stdout: "queue_len": 0, 2023-07-21T19:07:06.517 INFO:teuthology.orchestra.run.smithi039.stdout: "complete_latency": { 2023-07-21T19:07:06.517 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 10807, 2023-07-21T19:07:06.517 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.499058108, 2023-07-21T19:07:06.517 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000046179 2023-07-21T19:07:06.517 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:07:06.517 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:06.518 INFO:teuthology.orchestra.run.smithi039.stdout: "finisher-PurgeQueue": { 2023-07-21T19:07:06.518 INFO:teuthology.orchestra.run.smithi039.stdout: "queue_len": 0, 2023-07-21T19:07:06.518 INFO:teuthology.orchestra.run.smithi039.stdout: "complete_latency": { 2023-07-21T19:07:06.518 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:07:06.518 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:07:06.518 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:07:06.518 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:07:06.519 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:06.519 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": { 2023-07-21T19:07:06.519 INFO:teuthology.orchestra.run.smithi039.stdout: "request": 0, 2023-07-21T19:07:06.519 INFO:teuthology.orchestra.run.smithi039.stdout: "reply": 0, 2023-07-21T19:07:06.519 INFO:teuthology.orchestra.run.smithi039.stdout: "reply_latency": { 2023-07-21T19:07:06.519 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:07:06.519 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:07:06.520 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:07:06.520 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:06.520 INFO:teuthology.orchestra.run.smithi039.stdout: "slow_reply": 0, 2023-07-21T19:07:06.520 INFO:teuthology.orchestra.run.smithi039.stdout: "forward": 0, 2023-07-21T19:07:06.520 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_fetch_complete": 0, 2023-07-21T19:07:06.520 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_fetch_keys": 0, 2023-07-21T19:07:06.520 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_commit": 0, 2023-07-21T19:07:06.521 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_split": 0, 2023-07-21T19:07:06.521 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_merge": 0, 2023-07-21T19:07:06.521 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes": 21643, 2023-07-21T19:07:06.521 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_top": 6179, 2023-07-21T19:07:06.521 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_bottom": 15446, 2023-07-21T19:07:06.521 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_pin_tail": 18, 2023-07-21T19:07:06.521 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_pinned": 12822, 2023-07-21T19:07:06.522 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_expired": 9250, 2023-07-21T19:07:06.522 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_with_caps": 0, 2023-07-21T19:07:06.522 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": 0, 2023-07-21T19:07:06.522 INFO:teuthology.orchestra.run.smithi039.stdout: "subtrees": 1, 2023-07-21T19:07:06.522 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse": 0, 2023-07-21T19:07:06.522 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_hit": 0, 2023-07-21T19:07:06.522 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_forward": 0, 2023-07-21T19:07:06.522 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_discover": 0, 2023-07-21T19:07:06.523 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_dir_fetch": 0, 2023-07-21T19:07:06.523 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_remote_ino": 0, 2023-07-21T19:07:06.523 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_lock": 0, 2023-07-21T19:07:06.523 INFO:teuthology.orchestra.run.smithi039.stdout: "load_cent": 0, 2023-07-21T19:07:06.523 INFO:teuthology.orchestra.run.smithi039.stdout: "q": 0, 2023-07-21T19:07:06.523 INFO:teuthology.orchestra.run.smithi039.stdout: "exported": 0, 2023-07-21T19:07:06.523 INFO:teuthology.orchestra.run.smithi039.stdout: "exported_inodes": 0, 2023-07-21T19:07:06.524 INFO:teuthology.orchestra.run.smithi039.stdout: "imported": 0, 2023-07-21T19:07:06.524 INFO:teuthology.orchestra.run.smithi039.stdout: "imported_inodes": 0, 2023-07-21T19:07:06.524 INFO:teuthology.orchestra.run.smithi039.stdout: "openino_dir_fetch": 0, 2023-07-21T19:07:06.524 INFO:teuthology.orchestra.run.smithi039.stdout: "openino_backtrace_fetch": 0, 2023-07-21T19:07:06.524 INFO:teuthology.orchestra.run.smithi039.stdout: "openino_peer_discover": 0, 2023-07-21T19:07:06.524 INFO:teuthology.orchestra.run.smithi039.stdout: "root_rfiles": 0, 2023-07-21T19:07:06.524 INFO:teuthology.orchestra.run.smithi039.stdout: "root_rbytes": 0, 2023-07-21T19:07:06.525 INFO:teuthology.orchestra.run.smithi039.stdout: "root_rsnaps": 0, 2023-07-21T19:07:06.525 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_backtrace_fetch": 0, 2023-07-21T19:07:06.525 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_set_tag": 0, 2023-07-21T19:07:06.525 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_backtrace_repaired": 0, 2023-07-21T19:07:06.525 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_inotable_repaired": 0, 2023-07-21T19:07:06.525 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_dir_inodes": 0, 2023-07-21T19:07:06.525 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_dir_base_inodes": 0, 2023-07-21T19:07:06.525 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_dirfrag_rstats": 0, 2023-07-21T19:07:06.526 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_file_inodes": 0, 2023-07-21T19:07:06.526 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_inode_file_caps": 0, 2023-07-21T19:07:06.526 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_revoke": 0, 2023-07-21T19:07:06.526 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_grant": 0, 2023-07-21T19:07:06.526 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_trunc": 0, 2023-07-21T19:07:06.526 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-21T19:07:06.526 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-21T19:07:06.527 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_caps": 0, 2023-07-21T19:07:06.527 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_caps_dirty": 0, 2023-07-21T19:07:06.527 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_cap_release": 0, 2023-07-21T19:07:06.527 INFO:teuthology.orchestra.run.smithi039.stdout: "process_request_cap_release": 0 2023-07-21T19:07:06.528 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:06.528 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_cache": { 2023-07-21T19:07:06.528 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_update": 0, 2023-07-21T19:07:06.528 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_update_receipt": 0, 2023-07-21T19:07:06.528 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_try_discover": 0, 2023-07-21T19:07:06.528 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_send_discover": 0, 2023-07-21T19:07:06.528 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_handle_discover": 0, 2023-07-21T19:07:06.529 INFO:teuthology.orchestra.run.smithi039.stdout: "num_strays": 0, 2023-07-21T19:07:06.529 INFO:teuthology.orchestra.run.smithi039.stdout: "num_strays_delayed": 0, 2023-07-21T19:07:06.529 INFO:teuthology.orchestra.run.smithi039.stdout: "num_strays_enqueuing": 0, 2023-07-21T19:07:06.529 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_created": 0, 2023-07-21T19:07:06.529 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_enqueued": 0, 2023-07-21T19:07:06.529 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_reintegrated": 0, 2023-07-21T19:07:06.530 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_migrated": 0, 2023-07-21T19:07:06.530 INFO:teuthology.orchestra.run.smithi039.stdout: "num_recovering_processing": 0, 2023-07-21T19:07:06.530 INFO:teuthology.orchestra.run.smithi039.stdout: "num_recovering_enqueued": 0, 2023-07-21T19:07:06.530 INFO:teuthology.orchestra.run.smithi039.stdout: "num_recovering_prioritized": 0, 2023-07-21T19:07:06.530 INFO:teuthology.orchestra.run.smithi039.stdout: "recovery_started": 0, 2023-07-21T19:07:06.530 INFO:teuthology.orchestra.run.smithi039.stdout: "recovery_completed": 0, 2023-07-21T19:07:06.530 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_enqueue_scrub": 0, 2023-07-21T19:07:06.531 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_exportdir": 0, 2023-07-21T19:07:06.531 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_flush": 0, 2023-07-21T19:07:06.531 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_fragmentdir": 0, 2023-07-21T19:07:06.531 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_fragstats": 0, 2023-07-21T19:07:06.531 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_inodestats": 0 2023-07-21T19:07:06.532 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:06.532 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_log": { 2023-07-21T19:07:06.532 INFO:teuthology.orchestra.run.smithi039.stdout: "evadd": 0, 2023-07-21T19:07:06.532 INFO:teuthology.orchestra.run.smithi039.stdout: "evex": 0, 2023-07-21T19:07:06.532 INFO:teuthology.orchestra.run.smithi039.stdout: "evtrm": 0, 2023-07-21T19:07:06.532 INFO:teuthology.orchestra.run.smithi039.stdout: "ev": 0, 2023-07-21T19:07:06.532 INFO:teuthology.orchestra.run.smithi039.stdout: "evexg": 0, 2023-07-21T19:07:06.533 INFO:teuthology.orchestra.run.smithi039.stdout: "evexd": 0, 2023-07-21T19:07:06.533 INFO:teuthology.orchestra.run.smithi039.stdout: "segadd": 0, 2023-07-21T19:07:06.533 INFO:teuthology.orchestra.run.smithi039.stdout: "segex": 0, 2023-07-21T19:07:06.533 INFO:teuthology.orchestra.run.smithi039.stdout: "segtrm": 0, 2023-07-21T19:07:06.533 INFO:teuthology.orchestra.run.smithi039.stdout: "seg": 129, 2023-07-21T19:07:06.533 INFO:teuthology.orchestra.run.smithi039.stdout: "segexg": 0, 2023-07-21T19:07:06.533 INFO:teuthology.orchestra.run.smithi039.stdout: "segexd": 0, 2023-07-21T19:07:06.533 INFO:teuthology.orchestra.run.smithi039.stdout: "expos": 436217215, 2023-07-21T19:07:06.534 INFO:teuthology.orchestra.run.smithi039.stdout: "wrpos": 0, 2023-07-21T19:07:06.534 INFO:teuthology.orchestra.run.smithi039.stdout: "rdpos": 964669125, 2023-07-21T19:07:06.534 INFO:teuthology.orchestra.run.smithi039.stdout: "jlat": { 2023-07-21T19:07:06.534 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:07:06.534 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:07:06.534 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:07:06.534 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:06.535 INFO:teuthology.orchestra.run.smithi039.stdout: "replayed": 160013 2023-07-21T19:07:06.535 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:06.535 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_mem": { 2023-07-21T19:07:06.535 INFO:teuthology.orchestra.run.smithi039.stdout: "ino": 1520, 2023-07-21T19:07:06.535 INFO:teuthology.orchestra.run.smithi039.stdout: "ino+": 30994, 2023-07-21T19:07:06.535 INFO:teuthology.orchestra.run.smithi039.stdout: "ino-": 29474, 2023-07-21T19:07:06.535 INFO:teuthology.orchestra.run.smithi039.stdout: "dir": 58, 2023-07-21T19:07:06.536 INFO:teuthology.orchestra.run.smithi039.stdout: "dir+": 100, 2023-07-21T19:07:06.536 INFO:teuthology.orchestra.run.smithi039.stdout: "dir-": 42, 2023-07-21T19:07:06.536 INFO:teuthology.orchestra.run.smithi039.stdout: "dn": 21643, 2023-07-21T19:07:06.536 INFO:teuthology.orchestra.run.smithi039.stdout: "dn+": 31455, 2023-07-21T19:07:06.536 INFO:teuthology.orchestra.run.smithi039.stdout: "dn-": 9812, 2023-07-21T19:07:06.536 INFO:teuthology.orchestra.run.smithi039.stdout: "cap": 0, 2023-07-21T19:07:06.536 INFO:teuthology.orchestra.run.smithi039.stdout: "cap+": 0, 2023-07-21T19:07:06.537 INFO:teuthology.orchestra.run.smithi039.stdout: "cap-": 0, 2023-07-21T19:07:06.537 INFO:teuthology.orchestra.run.smithi039.stdout: "rss": 77768, 2023-07-21T19:07:06.537 INFO:teuthology.orchestra.run.smithi039.stdout: "heap": 207156 2023-07-21T19:07:06.537 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:06.538 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_server": { 2023-07-21T19:07:06.538 INFO:teuthology.orchestra.run.smithi039.stdout: "dispatch_client_request": 0, 2023-07-21T19:07:06.538 INFO:teuthology.orchestra.run.smithi039.stdout: "dispatch_server_request": 0, 2023-07-21T19:07:06.538 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_request": 0, 2023-07-21T19:07:06.538 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_session": 0, 2023-07-21T19:07:06.538 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_peer_request": 0, 2023-07-21T19:07:06.538 INFO:teuthology.orchestra.run.smithi039.stdout: "req_create_latency": { 2023-07-21T19:07:06.539 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:07:06.539 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:07:06.539 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:07:06.539 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:06.539 INFO:teuthology.orchestra.run.smithi039.stdout: "req_getattr_latency": { 2023-07-21T19:07:06.539 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:07:06.539 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:07:06.540 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:07:06.540 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:06.540 INFO:teuthology.orchestra.run.smithi039.stdout: "req_getfilelock_latency": { 2023-07-21T19:07:06.540 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:07:06.540 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:07:06.540 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:07:06.540 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:06.541 INFO:teuthology.orchestra.run.smithi039.stdout: "req_link_latency": { 2023-07-21T19:07:06.541 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:07:06.541 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:07:06.541 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:07:06.541 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:06.541 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookup_latency": { 2023-07-21T19:07:06.541 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:07:06.541 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:07:06.542 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:07:06.542 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:06.542 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookuphash_latency": { 2023-07-21T19:07:06.542 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:07:06.542 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:07:06.542 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:07:06.542 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:06.543 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupino_latency": { 2023-07-21T19:07:06.543 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:07:06.543 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:07:06.543 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:07:06.543 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:06.543 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupname_latency": { 2023-07-21T19:07:06.543 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:07:06.544 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:07:06.544 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:07:06.544 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:06.544 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupparent_latency": { 2023-07-21T19:07:06.544 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:07:06.544 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:07:06.544 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:07:06.545 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:06.545 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupsnap_latency": { 2023-07-21T19:07:06.545 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:07:06.545 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:07:06.545 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:07:06.545 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:06.545 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lssnap_latency": { 2023-07-21T19:07:06.546 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:07:06.546 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:07:06.546 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:07:06.546 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:06.546 INFO:teuthology.orchestra.run.smithi039.stdout: "req_mkdir_latency": { 2023-07-21T19:07:06.546 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:07:06.546 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:07:06.547 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:07:06.547 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:06.547 INFO:teuthology.orchestra.run.smithi039.stdout: "req_mknod_latency": { 2023-07-21T19:07:06.547 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:07:06.547 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:07:06.547 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:07:06.548 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:06.548 INFO:teuthology.orchestra.run.smithi039.stdout: "req_mksnap_latency": { 2023-07-21T19:07:06.548 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:07:06.548 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:07:06.548 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:07:06.549 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:06.549 INFO:teuthology.orchestra.run.smithi039.stdout: "req_open_latency": { 2023-07-21T19:07:06.549 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:07:06.549 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:07:06.549 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:07:06.549 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:06.549 INFO:teuthology.orchestra.run.smithi039.stdout: "req_readdir_latency": { 2023-07-21T19:07:06.549 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:07:06.550 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:07:06.550 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:07:06.550 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:06.550 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rename_latency": { 2023-07-21T19:07:06.550 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:07:06.550 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:07:06.550 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:07:06.551 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:06.551 INFO:teuthology.orchestra.run.smithi039.stdout: "req_renamesnap_latency": { 2023-07-21T19:07:06.551 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:07:06.551 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:07:06.551 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:07:06.551 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:06.551 INFO:teuthology.orchestra.run.smithi039.stdout: "req_snapdiff_latency": { 2023-07-21T19:07:06.552 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:07:06.552 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:07:06.552 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:07:06.552 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:06.552 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rmdir_latency": { 2023-07-21T19:07:06.552 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:07:06.552 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:07:06.553 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:07:06.553 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:06.553 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rmsnap_latency": { 2023-07-21T19:07:06.553 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:07:06.553 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:07:06.554 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:07:06.554 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:06.554 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rmxattr_latency": { 2023-07-21T19:07:06.554 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:07:06.554 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:07:06.554 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:07:06.554 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:06.555 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setattr_latency": { 2023-07-21T19:07:06.555 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:07:06.555 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:07:06.555 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:07:06.555 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:06.555 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setdirlayout_latency": { 2023-07-21T19:07:06.555 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:07:06.556 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:07:06.556 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:07:06.556 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:06.556 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setfilelock_latency": { 2023-07-21T19:07:06.556 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:07:06.556 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:07:06.556 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:07:06.557 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:06.557 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setlayout_latency": { 2023-07-21T19:07:06.557 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:07:06.557 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:07:06.557 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:07:06.557 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:06.557 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setxattr_latency": { 2023-07-21T19:07:06.558 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:07:06.558 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:07:06.558 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:07:06.558 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:06.559 INFO:teuthology.orchestra.run.smithi039.stdout: "req_symlink_latency": { 2023-07-21T19:07:06.559 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:07:06.559 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:07:06.559 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:07:06.559 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:06.559 INFO:teuthology.orchestra.run.smithi039.stdout: "req_unlink_latency": { 2023-07-21T19:07:06.559 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:07:06.560 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:07:06.560 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:07:06.560 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:06.560 INFO:teuthology.orchestra.run.smithi039.stdout: "cap_revoke_eviction": 0, 2023-07-21T19:07:06.560 INFO:teuthology.orchestra.run.smithi039.stdout: "cap_acquisition_throttle": 0, 2023-07-21T19:07:06.560 INFO:teuthology.orchestra.run.smithi039.stdout: "req_getvxattr_latency": { 2023-07-21T19:07:06.560 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:07:06.560 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:07:06.561 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:07:06.561 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:07:06.561 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:06.561 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_sessions": { 2023-07-21T19:07:06.561 INFO:teuthology.orchestra.run.smithi039.stdout: "session_count": 1, 2023-07-21T19:07:06.561 INFO:teuthology.orchestra.run.smithi039.stdout: "session_add": 2, 2023-07-21T19:07:06.561 INFO:teuthology.orchestra.run.smithi039.stdout: "session_remove": 1, 2023-07-21T19:07:06.562 INFO:teuthology.orchestra.run.smithi039.stdout: "sessions_open": 2, 2023-07-21T19:07:06.562 INFO:teuthology.orchestra.run.smithi039.stdout: "sessions_stale": 0, 2023-07-21T19:07:06.562 INFO:teuthology.orchestra.run.smithi039.stdout: "total_load": 0, 2023-07-21T19:07:06.562 INFO:teuthology.orchestra.run.smithi039.stdout: "average_load": 0, 2023-07-21T19:07:06.562 INFO:teuthology.orchestra.run.smithi039.stdout: "avg_session_uptime": 4373 2023-07-21T19:07:06.562 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:06.562 INFO:teuthology.orchestra.run.smithi039.stdout: "mempool": { 2023-07-21T19:07:06.563 INFO:teuthology.orchestra.run.smithi039.stdout: "bloom_filter_bytes": 0, 2023-07-21T19:07:06.563 INFO:teuthology.orchestra.run.smithi039.stdout: "bloom_filter_items": 0, 2023-07-21T19:07:06.563 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_alloc_bytes": 0, 2023-07-21T19:07:06.563 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_alloc_items": 0, 2023-07-21T19:07:06.563 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_data_bytes": 0, 2023-07-21T19:07:06.563 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_data_items": 0, 2023-07-21T19:07:06.563 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-21T19:07:06.563 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_onode_items": 0, 2023-07-21T19:07:06.564 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-21T19:07:06.564 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_meta_items": 0, 2023-07-21T19:07:06.564 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_other_bytes": 0, 2023-07-21T19:07:06.564 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_other_items": 0, 2023-07-21T19:07:06.564 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-21T19:07:06.564 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_buffer_items": 0, 2023-07-21T19:07:06.564 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_extent_bytes": 0, 2023-07-21T19:07:06.565 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_extent_items": 0, 2023-07-21T19:07:06.565 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_blob_bytes": 0, 2023-07-21T19:07:06.565 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_blob_items": 0, 2023-07-21T19:07:06.565 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-21T19:07:06.565 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_shared_blob_items": 0, 2023-07-21T19:07:06.565 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-21T19:07:06.565 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_inline_bl_items": 0, 2023-07-21T19:07:06.566 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_bytes": 0, 2023-07-21T19:07:06.566 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_items": 0, 2023-07-21T19:07:06.566 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_txc_bytes": 0, 2023-07-21T19:07:06.566 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_txc_items": 0, 2023-07-21T19:07:06.566 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-21T19:07:06.566 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_deferred_items": 0, 2023-07-21T19:07:06.566 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_bytes": 0, 2023-07-21T19:07:06.567 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_items": 0, 2023-07-21T19:07:06.567 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_bytes": 0, 2023-07-21T19:07:06.567 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_items": 0, 2023-07-21T19:07:06.567 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_reader_bytes": 0, 2023-07-21T19:07:06.567 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_reader_items": 0, 2023-07-21T19:07:06.567 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_writer_bytes": 0, 2023-07-21T19:07:06.567 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_writer_items": 0, 2023-07-21T19:07:06.567 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_anon_bytes": 293515, 2023-07-21T19:07:06.568 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_anon_items": 1882, 2023-07-21T19:07:06.568 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_meta_bytes": 0, 2023-07-21T19:07:06.568 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_meta_items": 0, 2023-07-21T19:07:06.569 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_bytes": 0, 2023-07-21T19:07:06.569 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_items": 0, 2023-07-21T19:07:06.569 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mapbl_bytes": 0, 2023-07-21T19:07:06.569 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mapbl_items": 0, 2023-07-21T19:07:06.569 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pglog_bytes": 0, 2023-07-21T19:07:06.569 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pglog_items": 0, 2023-07-21T19:07:06.569 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_bytes": 8160, 2023-07-21T19:07:06.570 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_items": 186, 2023-07-21T19:07:06.570 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_mapping_bytes": 0, 2023-07-21T19:07:06.570 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_mapping_items": 0, 2023-07-21T19:07:06.570 INFO:teuthology.orchestra.run.smithi039.stdout: "pgmap_bytes": 0, 2023-07-21T19:07:06.570 INFO:teuthology.orchestra.run.smithi039.stdout: "pgmap_items": 0, 2023-07-21T19:07:06.570 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_co_bytes": 19219454, 2023-07-21T19:07:06.570 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_co_items": 166199, 2023-07-21T19:07:06.570 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_1_bytes": 0, 2023-07-21T19:07:06.571 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_1_items": 0, 2023-07-21T19:07:06.571 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_2_bytes": 0, 2023-07-21T19:07:06.571 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_2_items": 0 2023-07-21T19:07:06.571 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:06.571 INFO:teuthology.orchestra.run.smithi039.stdout: "objecter": { 2023-07-21T19:07:06.571 INFO:teuthology.orchestra.run.smithi039.stdout: "op_active": 0, 2023-07-21T19:07:06.571 INFO:teuthology.orchestra.run.smithi039.stdout: "op_laggy": 0, 2023-07-21T19:07:06.572 INFO:teuthology.orchestra.run.smithi039.stdout: "op_send": 6522, 2023-07-21T19:07:06.572 INFO:teuthology.orchestra.run.smithi039.stdout: "op_send_bytes": 21, 2023-07-21T19:07:06.572 INFO:teuthology.orchestra.run.smithi039.stdout: "op_resend": 3, 2023-07-21T19:07:06.572 INFO:teuthology.orchestra.run.smithi039.stdout: "op_reply": 6522, 2023-07-21T19:07:06.572 INFO:teuthology.orchestra.run.smithi039.stdout: "op_latency": { 2023-07-21T19:07:06.572 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 6522, 2023-07-21T19:07:06.572 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 19.566635749, 2023-07-21T19:07:06.573 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.003000097 2023-07-21T19:07:06.573 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:06.573 INFO:teuthology.orchestra.run.smithi039.stdout: "op_inflight": 0, 2023-07-21T19:07:06.573 INFO:teuthology.orchestra.run.smithi039.stdout: "oplen_avg": { 2023-07-21T19:07:06.573 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 6522, 2023-07-21T19:07:06.573 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 6523 2023-07-21T19:07:06.573 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:06.574 INFO:teuthology.orchestra.run.smithi039.stdout: "op": 6522, 2023-07-21T19:07:06.574 INFO:teuthology.orchestra.run.smithi039.stdout: "op_r": 6522, 2023-07-21T19:07:06.574 INFO:teuthology.orchestra.run.smithi039.stdout: "op_w": 0, 2023-07-21T19:07:06.574 INFO:teuthology.orchestra.run.smithi039.stdout: "op_rmw": 0, 2023-07-21T19:07:06.574 INFO:teuthology.orchestra.run.smithi039.stdout: "op_pg": 0, 2023-07-21T19:07:06.574 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_stat": 3531, 2023-07-21T19:07:06.574 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_create": 0, 2023-07-21T19:07:06.574 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_read": 2989, 2023-07-21T19:07:06.575 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_write": 0, 2023-07-21T19:07:06.575 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_writefull": 0, 2023-07-21T19:07:06.575 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_writesame": 0, 2023-07-21T19:07:06.575 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_append": 0, 2023-07-21T19:07:06.575 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_zero": 0, 2023-07-21T19:07:06.575 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_truncate": 0, 2023-07-21T19:07:06.575 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_delete": 0, 2023-07-21T19:07:06.576 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_mapext": 0, 2023-07-21T19:07:06.576 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_sparse_read": 0, 2023-07-21T19:07:06.576 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_clonerange": 0, 2023-07-21T19:07:06.576 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_getxattr": 1, 2023-07-21T19:07:06.576 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_setxattr": 0, 2023-07-21T19:07:06.576 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_cmpxattr": 0, 2023-07-21T19:07:06.576 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_rmxattr": 0, 2023-07-21T19:07:06.577 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_resetxattrs": 0, 2023-07-21T19:07:06.577 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_call": 0, 2023-07-21T19:07:06.577 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_watch": 0, 2023-07-21T19:07:06.577 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_notify": 0, 2023-07-21T19:07:06.577 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_src_cmpxattr": 0, 2023-07-21T19:07:06.577 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_pgls": 0, 2023-07-21T19:07:06.577 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_pgls_filter": 0, 2023-07-21T19:07:06.578 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_other": 0, 2023-07-21T19:07:06.578 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_active": 0, 2023-07-21T19:07:06.578 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_send": 0, 2023-07-21T19:07:06.578 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_resend": 0, 2023-07-21T19:07:06.578 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_ping": 0, 2023-07-21T19:07:06.578 INFO:teuthology.orchestra.run.smithi039.stdout: "poolop_active": 0, 2023-07-21T19:07:06.578 INFO:teuthology.orchestra.run.smithi039.stdout: "poolop_send": 0, 2023-07-21T19:07:06.579 INFO:teuthology.orchestra.run.smithi039.stdout: "poolop_resend": 0, 2023-07-21T19:07:06.579 INFO:teuthology.orchestra.run.smithi039.stdout: "poolstat_active": 0, 2023-07-21T19:07:06.579 INFO:teuthology.orchestra.run.smithi039.stdout: "poolstat_send": 0, 2023-07-21T19:07:06.579 INFO:teuthology.orchestra.run.smithi039.stdout: "poolstat_resend": 0, 2023-07-21T19:07:06.579 INFO:teuthology.orchestra.run.smithi039.stdout: "statfs_active": 0, 2023-07-21T19:07:06.580 INFO:teuthology.orchestra.run.smithi039.stdout: "statfs_send": 0, 2023-07-21T19:07:06.580 INFO:teuthology.orchestra.run.smithi039.stdout: "statfs_resend": 0, 2023-07-21T19:07:06.580 INFO:teuthology.orchestra.run.smithi039.stdout: "command_active": 0, 2023-07-21T19:07:06.580 INFO:teuthology.orchestra.run.smithi039.stdout: "command_send": 0, 2023-07-21T19:07:06.580 INFO:teuthology.orchestra.run.smithi039.stdout: "command_resend": 0, 2023-07-21T19:07:06.580 INFO:teuthology.orchestra.run.smithi039.stdout: "map_epoch": 123, 2023-07-21T19:07:06.580 INFO:teuthology.orchestra.run.smithi039.stdout: "map_full": 0, 2023-07-21T19:07:06.581 INFO:teuthology.orchestra.run.smithi039.stdout: "map_inc": 44, 2023-07-21T19:07:06.581 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_sessions": 10, 2023-07-21T19:07:06.581 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_session_open": 11, 2023-07-21T19:07:06.581 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_session_close": 1, 2023-07-21T19:07:06.581 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_laggy": 0, 2023-07-21T19:07:06.581 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_wr": 0, 2023-07-21T19:07:06.581 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_rd": 2, 2023-07-21T19:07:06.582 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_del": 0 2023-07-21T19:07:06.582 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:06.582 INFO:teuthology.orchestra.run.smithi039.stdout: "oft": { 2023-07-21T19:07:06.582 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_objs": 0, 2023-07-21T19:07:06.582 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_kv_pairs": 0, 2023-07-21T19:07:06.582 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_updates": 0, 2023-07-21T19:07:06.582 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_removes": 0 2023-07-21T19:07:06.583 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:06.583 INFO:teuthology.orchestra.run.smithi039.stdout: "osd-slow-ops": { 2023-07-21T19:07:06.583 INFO:teuthology.orchestra.run.smithi039.stdout: "slow_ops_count": 0 2023-07-21T19:07:06.583 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:06.583 INFO:teuthology.orchestra.run.smithi039.stdout: "purge_queue": { 2023-07-21T19:07:06.583 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing_ops": 0, 2023-07-21T19:07:06.583 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing_ops_high_water": 0, 2023-07-21T19:07:06.584 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing": 0, 2023-07-21T19:07:06.584 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing_high_water": 0, 2023-07-21T19:07:06.584 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executed": 0, 2023-07-21T19:07:06.584 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_item_in_journal": 0 2023-07-21T19:07:06.584 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:06.584 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-21T19:07:06.584 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2023-07-21T19:07:06.585 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 104857600, 2023-07-21T19:07:06.585 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2023-07-21T19:07:06.585 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 7101, 2023-07-21T19:07:06.585 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 963575326, 2023-07-21T19:07:06.585 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2023-07-21T19:07:06.585 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 7101, 2023-07-21T19:07:06.585 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 0, 2023-07-21T19:07:06.585 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 0, 2023-07-21T19:07:06.586 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 7101, 2023-07-21T19:07:06.586 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 963575326, 2023-07-21T19:07:06.586 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2023-07-21T19:07:06.586 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:07:06.586 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:07:06.586 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:07:06.586 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:07:06.587 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:06.587 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-objecter_bytes": { 2023-07-21T19:07:06.587 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2023-07-21T19:07:06.587 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 104857600, 2023-07-21T19:07:06.587 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2023-07-21T19:07:06.587 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 0, 2023-07-21T19:07:06.587 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 0, 2023-07-21T19:07:06.588 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2023-07-21T19:07:06.588 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 0, 2023-07-21T19:07:06.588 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 6522, 2023-07-21T19:07:06.588 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 960547503, 2023-07-21T19:07:06.588 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 6522, 2023-07-21T19:07:06.588 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 960547503, 2023-07-21T19:07:06.588 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2023-07-21T19:07:06.589 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:07:06.589 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:07:06.589 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:07:06.589 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:07:06.589 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:06.589 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-objecter_ops": { 2023-07-21T19:07:06.589 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2023-07-21T19:07:06.590 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 1024, 2023-07-21T19:07:06.590 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2023-07-21T19:07:06.590 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 0, 2023-07-21T19:07:06.590 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 0, 2023-07-21T19:07:06.590 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2023-07-21T19:07:06.590 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 0, 2023-07-21T19:07:06.590 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 6522, 2023-07-21T19:07:06.591 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 6522, 2023-07-21T19:07:06.591 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 6522, 2023-07-21T19:07:06.591 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 6522, 2023-07-21T19:07:06.591 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2023-07-21T19:07:06.591 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:07:06.591 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:07:06.591 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:07:06.591 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:07:06.592 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:06.592 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-write_buf_throttle": { 2023-07-21T19:07:06.592 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2023-07-21T19:07:06.592 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 3758096384, 2023-07-21T19:07:06.592 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2023-07-21T19:07:06.592 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 0, 2023-07-21T19:07:06.592 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 0, 2023-07-21T19:07:06.593 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2023-07-21T19:07:06.593 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 0, 2023-07-21T19:07:06.593 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 0, 2023-07-21T19:07:06.593 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 0, 2023-07-21T19:07:06.593 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 0, 2023-07-21T19:07:06.593 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 0, 2023-07-21T19:07:06.593 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2023-07-21T19:07:06.594 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:07:06.594 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:07:06.594 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:07:06.594 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:07:06.594 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:06.594 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-write_buf_throttle-0x55fdeb7ea460": { 2023-07-21T19:07:06.594 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2023-07-21T19:07:06.595 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 3758096384, 2023-07-21T19:07:06.595 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2023-07-21T19:07:06.595 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 0, 2023-07-21T19:07:06.595 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 0, 2023-07-21T19:07:06.595 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2023-07-21T19:07:06.595 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 0, 2023-07-21T19:07:06.595 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 0, 2023-07-21T19:07:06.595 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 0, 2023-07-21T19:07:06.596 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 0, 2023-07-21T19:07:06.596 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 0, 2023-07-21T19:07:06.596 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2023-07-21T19:07:06.596 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:07:06.596 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:07:06.596 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:07:06.596 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:07:06.597 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:07:06.597 INFO:teuthology.orchestra.run.smithi039.stdout:} 2023-07-21T19:07:06.822 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:06 smithi039 ceph-mon[96489]: pgmap v804: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 7.2 MiB/s wr, 183 op/s 2023-07-21T19:07:06.824 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2023-07-21T19:07:06.824 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2023-07-21T19:07:06.824 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2023-07-21T19:07:06.825 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2023-07-21T19:07:06.825 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2023-07-21T19:07:06.825 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2023-07-21T19:07:06.825 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2023-07-21T19:07:06.825 DEBUG:tasks.check_counter:Getting stats from j 2023-07-21T19:07:06.826 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph daemon mds.j perf dump 2023-07-21T19:07:06.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:06 smithi193 ceph-mon[98204]: pgmap v804: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 7.2 MiB/s wr, 183 op/s 2023-07-21T19:07:06.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:06 smithi155 ceph-mon[97934]: pgmap v804: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 7.2 MiB/s wr, 183 op/s 2023-07-21T19:07:07.403 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T19:07:08.638 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2023-07-21T19:07:08.638 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-0": { 2023-07-21T19:07:08.639 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 4, 2023-07-21T19:07:08.639 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 4, 2023-07-21T19:07:08.639 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 2992, 2023-07-21T19:07:08.639 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 448, 2023-07-21T19:07:08.639 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 4, 2023-07-21T19:07:08.639 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 0, 2023-07-21T19:07:08.639 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.002000781, 2023-07-21T19:07:08.640 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.000058800, 2023-07-21T19:07:08.640 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.000891486, 2023-07-21T19:07:08.640 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-21T19:07:08.640 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:07:08.640 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 3, 2023-07-21T19:07:08.640 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000966153, 2023-07-21T19:07:08.640 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000322051 2023-07-21T19:07:08.641 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:08.641 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:07:08.641 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:07:08.641 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:07:08.641 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:07:08.641 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:08.641 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-07-21T19:07:08.641 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 448 2023-07-21T19:07:08.642 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:08.642 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-1": { 2023-07-21T19:07:08.642 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 3, 2023-07-21T19:07:08.642 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 335, 2023-07-21T19:07:08.642 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 480, 2023-07-21T19:07:08.642 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 286336, 2023-07-21T19:07:08.642 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 6, 2023-07-21T19:07:08.642 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 3, 2023-07-21T19:07:08.643 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.023235653, 2023-07-21T19:07:08.643 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.016745508, 2023-07-21T19:07:08.643 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.002703925, 2023-07-21T19:07:08.643 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-21T19:07:08.643 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:07:08.643 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 332, 2023-07-21T19:07:08.643 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.035719761, 2023-07-21T19:07:08.643 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000107589 2023-07-21T19:07:08.644 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:08.644 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:07:08.644 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:07:08.644 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:07:08.644 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:07:08.644 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:08.644 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 480, 2023-07-21T19:07:08.645 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 286336 2023-07-21T19:07:08.645 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:08.645 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-2": { 2023-07-21T19:07:08.645 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 447, 2023-07-21T19:07:08.645 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 438, 2023-07-21T19:07:08.645 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 1350992, 2023-07-21T19:07:08.645 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 251136, 2023-07-21T19:07:08.645 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 2, 2023-07-21T19:07:08.646 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 1, 2023-07-21T19:07:08.646 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.065144947, 2023-07-21T19:07:08.646 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.022752788, 2023-07-21T19:07:08.646 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.036580795, 2023-07-21T19:07:08.646 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.010518682, 2023-07-21T19:07:08.646 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:07:08.646 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 437, 2023-07-21T19:07:08.646 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.033190003, 2023-07-21T19:07:08.647 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000075949 2023-07-21T19:07:08.647 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:08.647 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:07:08.647 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:07:08.647 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:07:08.647 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:07:08.647 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:08.647 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 1350992, 2023-07-21T19:07:08.648 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 251136 2023-07-21T19:07:08.648 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:08.648 INFO:teuthology.orchestra.run.smithi039.stdout: "cct": { 2023-07-21T19:07:08.648 INFO:teuthology.orchestra.run.smithi039.stdout: "total_workers": 0, 2023-07-21T19:07:08.648 INFO:teuthology.orchestra.run.smithi039.stdout: "unhealthy_workers": 0 2023-07-21T19:07:08.648 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:08.648 INFO:teuthology.orchestra.run.smithi039.stdout: "mempool": { 2023-07-21T19:07:08.648 INFO:teuthology.orchestra.run.smithi039.stdout: "bloom_filter_bytes": 0, 2023-07-21T19:07:08.649 INFO:teuthology.orchestra.run.smithi039.stdout: "bloom_filter_items": 0, 2023-07-21T19:07:08.649 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_alloc_bytes": 0, 2023-07-21T19:07:08.649 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_alloc_items": 0, 2023-07-21T19:07:08.649 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_data_bytes": 0, 2023-07-21T19:07:08.649 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_data_items": 0, 2023-07-21T19:07:08.649 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-21T19:07:08.649 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_onode_items": 0, 2023-07-21T19:07:08.650 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-21T19:07:08.650 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_meta_items": 0, 2023-07-21T19:07:08.650 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_other_bytes": 0, 2023-07-21T19:07:08.650 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_other_items": 0, 2023-07-21T19:07:08.650 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-21T19:07:08.650 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_buffer_items": 0, 2023-07-21T19:07:08.650 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_extent_bytes": 0, 2023-07-21T19:07:08.650 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_extent_items": 0, 2023-07-21T19:07:08.651 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_blob_bytes": 0, 2023-07-21T19:07:08.651 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_blob_items": 0, 2023-07-21T19:07:08.651 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-21T19:07:08.651 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_shared_blob_items": 0, 2023-07-21T19:07:08.651 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-21T19:07:08.651 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_inline_bl_items": 0, 2023-07-21T19:07:08.651 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_bytes": 0, 2023-07-21T19:07:08.651 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_items": 0, 2023-07-21T19:07:08.652 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_txc_bytes": 0, 2023-07-21T19:07:08.652 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_txc_items": 0, 2023-07-21T19:07:08.652 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-21T19:07:08.652 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_deferred_items": 0, 2023-07-21T19:07:08.652 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_bytes": 0, 2023-07-21T19:07:08.652 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_items": 0, 2023-07-21T19:07:08.652 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_bytes": 0, 2023-07-21T19:07:08.652 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_items": 0, 2023-07-21T19:07:08.653 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_reader_bytes": 0, 2023-07-21T19:07:08.653 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_reader_items": 0, 2023-07-21T19:07:08.653 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_writer_bytes": 0, 2023-07-21T19:07:08.653 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_writer_items": 0, 2023-07-21T19:07:08.653 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_anon_bytes": 149745, 2023-07-21T19:07:08.653 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_anon_items": 31, 2023-07-21T19:07:08.653 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_meta_bytes": 0, 2023-07-21T19:07:08.653 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_meta_items": 0, 2023-07-21T19:07:08.654 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_bytes": 0, 2023-07-21T19:07:08.654 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_items": 0, 2023-07-21T19:07:08.654 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mapbl_bytes": 0, 2023-07-21T19:07:08.654 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mapbl_items": 0, 2023-07-21T19:07:08.654 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pglog_bytes": 0, 2023-07-21T19:07:08.654 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pglog_items": 0, 2023-07-21T19:07:08.654 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_bytes": 0, 2023-07-21T19:07:08.654 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_items": 0, 2023-07-21T19:07:08.655 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_mapping_bytes": 0, 2023-07-21T19:07:08.655 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_mapping_items": 0, 2023-07-21T19:07:08.655 INFO:teuthology.orchestra.run.smithi039.stdout: "pgmap_bytes": 0, 2023-07-21T19:07:08.655 INFO:teuthology.orchestra.run.smithi039.stdout: "pgmap_items": 0, 2023-07-21T19:07:08.655 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_co_bytes": 928, 2023-07-21T19:07:08.655 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_co_items": 2, 2023-07-21T19:07:08.655 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_1_bytes": 0, 2023-07-21T19:07:08.655 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_1_items": 0, 2023-07-21T19:07:08.656 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_2_bytes": 0, 2023-07-21T19:07:08.656 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_2_items": 0 2023-07-21T19:07:08.656 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2023-07-21T19:07:08.656 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-21T19:07:08.656 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2023-07-21T19:07:08.656 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 104857600, 2023-07-21T19:07:08.656 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2023-07-21T19:07:08.656 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 450, 2023-07-21T19:07:08.657 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 1287818, 2023-07-21T19:07:08.657 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2023-07-21T19:07:08.657 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 450, 2023-07-21T19:07:08.657 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 0, 2023-07-21T19:07:08.657 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 0, 2023-07-21T19:07:08.657 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 450, 2023-07-21T19:07:08.657 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 1287818, 2023-07-21T19:07:08.658 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2023-07-21T19:07:08.658 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2023-07-21T19:07:08.658 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2023-07-21T19:07:08.658 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2023-07-21T19:07:08.658 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:07:08.658 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-07-21T19:07:08.658 INFO:teuthology.orchestra.run.smithi039.stdout:} 2023-07-21T19:07:08.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:08 smithi039 ceph-mon[96489]: pgmap v805: 97 pgs: 3 active+clean+snaptrim, 94 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 10 MiB/s wr, 338 op/s 2023-07-21T19:07:08.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:08 smithi193 ceph-mon[98204]: pgmap v805: 97 pgs: 3 active+clean+snaptrim, 94 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 10 MiB/s wr, 338 op/s 2023-07-21T19:07:08.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:08 smithi155 ceph-mon[97934]: pgmap v805: 97 pgs: 3 active+clean+snaptrim, 94 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 10 MiB/s wr, 338 op/s 2023-07-21T19:07:09.062 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2023-07-21T19:07:09.062 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2023-07-21T19:07:09.063 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2023-07-21T19:07:09.063 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2023-07-21T19:07:09.063 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2023-07-21T19:07:09.064 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2023-07-21T19:07:09.064 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2023-07-21T19:07:09.064 DEBUG:tasks.check_counter:Getting stats from b 2023-07-21T19:07:09.065 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph daemon mds.b perf dump 2023-07-21T19:07:09.724 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.b/config 2023-07-21T19:07:10.834 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:10 smithi155 ceph-mon[97934]: pgmap v806: 97 pgs: 3 active+clean+snaptrim, 94 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 10 MiB/s wr, 338 op/s 2023-07-21T19:07:10.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:10 smithi193 ceph-mon[98204]: pgmap v806: 97 pgs: 3 active+clean+snaptrim, 94 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 10 MiB/s wr, 338 op/s 2023-07-21T19:07:10.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:10 smithi039 ceph-mon[96489]: pgmap v806: 97 pgs: 3 active+clean+snaptrim, 94 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 10 MiB/s wr, 338 op/s 2023-07-21T19:07:11.874 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2023-07-21T19:07:11.875 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-0": { 2023-07-21T19:07:11.875 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 643989, 2023-07-21T19:07:11.875 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 801018, 2023-07-21T19:07:11.875 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 182642948, 2023-07-21T19:07:11.876 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 614063032, 2023-07-21T19:07:11.876 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 11, 2023-07-21T19:07:11.876 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 4, 2023-07-21T19:07:11.876 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 42.356819687, 2023-07-21T19:07:11.877 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 19.582503725, 2023-07-21T19:07:11.877 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 25.239419096, 2023-07-21T19:07:11.877 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 2.105084063, 2023-07-21T19:07:11.877 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:07:11.877 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 801023, 2023-07-21T19:07:11.877 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 88.966481845, 2023-07-21T19:07:11.878 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000111066 2023-07-21T19:07:11.878 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:11.878 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:07:11.878 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 1298972, 2023-07-21T19:07:11.878 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.224687804, 2023-07-21T19:07:11.878 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000172 2023-07-21T19:07:11.878 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:11.879 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 3152, 2023-07-21T19:07:11.879 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 624048 2023-07-21T19:07:11.879 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:11.879 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-1": { 2023-07-21T19:07:11.880 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 589249, 2023-07-21T19:07:11.880 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 780347, 2023-07-21T19:07:11.880 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 169297330, 2023-07-21T19:07:11.880 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 770642588, 2023-07-21T19:07:11.881 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 20, 2023-07-21T19:07:11.881 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 6, 2023-07-21T19:07:11.881 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 28.390799250, 2023-07-21T19:07:11.882 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 13.463449047, 2023-07-21T19:07:11.882 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 18.962603662, 2023-07-21T19:07:11.882 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 2.077879041, 2023-07-21T19:07:11.883 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:07:11.883 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 780344, 2023-07-21T19:07:11.883 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 155.039668650, 2023-07-21T19:07:11.883 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000198681 2023-07-21T19:07:11.884 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:11.884 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:07:11.884 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 616200, 2023-07-21T19:07:11.885 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.160198879, 2023-07-21T19:07:11.885 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000259 2023-07-21T19:07:11.885 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:11.885 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 904480, 2023-07-21T19:07:11.885 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 423456 2023-07-21T19:07:11.885 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:11.886 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-2": { 2023-07-21T19:07:11.886 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 544252, 2023-07-21T19:07:11.886 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 678473, 2023-07-21T19:07:11.886 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 160398034, 2023-07-21T19:07:11.886 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 287383893, 2023-07-21T19:07:11.886 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 262, 2023-07-21T19:07:11.886 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 10, 2023-07-21T19:07:11.887 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 29.649554340, 2023-07-21T19:07:11.887 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 11.384116048, 2023-07-21T19:07:11.887 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 20.917826228, 2023-07-21T19:07:11.887 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.869851550, 2023-07-21T19:07:11.887 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:07:11.887 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 678466, 2023-07-21T19:07:11.887 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 89.901217495, 2023-07-21T19:07:11.887 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000132506 2023-07-21T19:07:11.888 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:11.888 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:07:11.888 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 1165415, 2023-07-21T19:07:11.888 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.205208503, 2023-07-21T19:07:11.888 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000176 2023-07-21T19:07:11.888 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:11.888 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 1990848, 2023-07-21T19:07:11.888 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 278272 2023-07-21T19:07:11.889 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:11.889 INFO:teuthology.orchestra.run.smithi155.stdout: "cct": { 2023-07-21T19:07:11.889 INFO:teuthology.orchestra.run.smithi155.stdout: "total_workers": 1, 2023-07-21T19:07:11.889 INFO:teuthology.orchestra.run.smithi155.stdout: "unhealthy_workers": 0 2023-07-21T19:07:11.889 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:11.889 INFO:teuthology.orchestra.run.smithi155.stdout: "finisher-MDSRank": { 2023-07-21T19:07:11.889 INFO:teuthology.orchestra.run.smithi155.stdout: "queue_len": 0, 2023-07-21T19:07:11.889 INFO:teuthology.orchestra.run.smithi155.stdout: "complete_latency": { 2023-07-21T19:07:11.890 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 142603, 2023-07-21T19:07:11.890 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 132.998047611, 2023-07-21T19:07:11.890 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000932645 2023-07-21T19:07:11.890 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-07-21T19:07:11.890 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:11.890 INFO:teuthology.orchestra.run.smithi155.stdout: "finisher-PurgeQueue": { 2023-07-21T19:07:11.890 INFO:teuthology.orchestra.run.smithi155.stdout: "queue_len": 0, 2023-07-21T19:07:11.891 INFO:teuthology.orchestra.run.smithi155.stdout: "complete_latency": { 2023-07-21T19:07:11.891 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 46551, 2023-07-21T19:07:11.891 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 9.942899259, 2023-07-21T19:07:11.891 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000213591 2023-07-21T19:07:11.891 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-07-21T19:07:11.891 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:11.891 INFO:teuthology.orchestra.run.smithi155.stdout: "mds": { 2023-07-21T19:07:11.892 INFO:teuthology.orchestra.run.smithi155.stdout: "request": 362440, 2023-07-21T19:07:11.892 INFO:teuthology.orchestra.run.smithi155.stdout: "reply": 311098, 2023-07-21T19:07:11.892 INFO:teuthology.orchestra.run.smithi155.stdout: "reply_latency": { 2023-07-21T19:07:11.892 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 311098, 2023-07-21T19:07:11.892 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 2357.071338497, 2023-07-21T19:07:11.892 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.007576620 2023-07-21T19:07:11.892 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:11.893 INFO:teuthology.orchestra.run.smithi155.stdout: "slow_reply": 0, 2023-07-21T19:07:11.893 INFO:teuthology.orchestra.run.smithi155.stdout: "forward": 51342, 2023-07-21T19:07:11.893 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_fetch_complete": 39, 2023-07-21T19:07:11.893 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_fetch_keys": 1, 2023-07-21T19:07:11.893 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_commit": 4041, 2023-07-21T19:07:11.893 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_split": 10, 2023-07-21T19:07:11.893 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_merge": 6, 2023-07-21T19:07:11.894 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes": 1235, 2023-07-21T19:07:11.894 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_top": 450, 2023-07-21T19:07:11.894 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_bottom": 390, 2023-07-21T19:07:11.894 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_pin_tail": 395, 2023-07-21T19:07:11.894 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_pinned": 588, 2023-07-21T19:07:11.894 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_expired": 4873, 2023-07-21T19:07:11.894 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_with_caps": 2, 2023-07-21T19:07:11.894 INFO:teuthology.orchestra.run.smithi155.stdout: "caps": 2, 2023-07-21T19:07:11.895 INFO:teuthology.orchestra.run.smithi155.stdout: "subtrees": 6, 2023-07-21T19:07:11.895 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse": 582140, 2023-07-21T19:07:11.895 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_hit": 394074, 2023-07-21T19:07:11.895 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_forward": 51269, 2023-07-21T19:07:11.895 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_discover": 346, 2023-07-21T19:07:11.895 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_dir_fetch": 3, 2023-07-21T19:07:11.895 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_remote_ino": 0, 2023-07-21T19:07:11.895 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_lock": 3329, 2023-07-21T19:07:11.896 INFO:teuthology.orchestra.run.smithi155.stdout: "load_cent": 27645, 2023-07-21T19:07:11.896 INFO:teuthology.orchestra.run.smithi155.stdout: "q": 14, 2023-07-21T19:07:11.896 INFO:teuthology.orchestra.run.smithi155.stdout: "exported": 55, 2023-07-21T19:07:11.896 INFO:teuthology.orchestra.run.smithi155.stdout: "exported_inodes": 3159, 2023-07-21T19:07:11.896 INFO:teuthology.orchestra.run.smithi155.stdout: "imported": 37, 2023-07-21T19:07:11.896 INFO:teuthology.orchestra.run.smithi155.stdout: "imported_inodes": 1385, 2023-07-21T19:07:11.896 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_dir_fetch": 0, 2023-07-21T19:07:11.896 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_backtrace_fetch": 90, 2023-07-21T19:07:11.897 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_peer_discover": 0, 2023-07-21T19:07:11.897 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rfiles": 131, 2023-07-21T19:07:11.897 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rbytes": 42134725, 2023-07-21T19:07:11.897 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rsnaps": 7, 2023-07-21T19:07:11.897 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_backtrace_fetch": 90, 2023-07-21T19:07:11.897 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_set_tag": 0, 2023-07-21T19:07:11.897 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_backtrace_repaired": 0, 2023-07-21T19:07:11.898 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_inotable_repaired": 0, 2023-07-21T19:07:11.898 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dir_inodes": 44, 2023-07-21T19:07:11.898 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dir_base_inodes": 4, 2023-07-21T19:07:11.898 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dirfrag_rstats": 40, 2023-07-21T19:07:11.898 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_file_inodes": 46, 2023-07-21T19:07:11.898 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_inode_file_caps": 64, 2023-07-21T19:07:11.898 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_revoke": 108217, 2023-07-21T19:07:11.898 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_grant": 157358, 2023-07-21T19:07:11.899 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_trunc": 0, 2023-07-21T19:07:11.899 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_flushsnap_ack": 1144, 2023-07-21T19:07:11.899 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-21T19:07:11.899 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_caps": 147983, 2023-07-21T19:07:11.899 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_caps_dirty": 26147, 2023-07-21T19:07:11.899 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_cap_release": 4913, 2023-07-21T19:07:11.900 INFO:teuthology.orchestra.run.smithi155.stdout: "process_request_cap_release": 183454 2023-07-21T19:07:11.900 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:11.900 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_cache": { 2023-07-21T19:07:11.900 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_update": 4676, 2023-07-21T19:07:11.900 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_update_receipt": 493, 2023-07-21T19:07:11.900 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_try_discover": 287, 2023-07-21T19:07:11.901 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_send_discover": 671, 2023-07-21T19:07:11.901 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_handle_discover": 8287, 2023-07-21T19:07:11.901 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays": 23, 2023-07-21T19:07:11.901 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays_delayed": 0, 2023-07-21T19:07:11.901 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays_enqueuing": 0, 2023-07-21T19:07:11.901 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_created": 26415, 2023-07-21T19:07:11.901 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_enqueued": 26392, 2023-07-21T19:07:11.902 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_reintegrated": 0, 2023-07-21T19:07:11.902 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_migrated": 0, 2023-07-21T19:07:11.902 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_processing": 0, 2023-07-21T19:07:11.902 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_enqueued": 0, 2023-07-21T19:07:11.902 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_prioritized": 0, 2023-07-21T19:07:11.902 INFO:teuthology.orchestra.run.smithi155.stdout: "recovery_started": 0, 2023-07-21T19:07:11.902 INFO:teuthology.orchestra.run.smithi155.stdout: "recovery_completed": 0, 2023-07-21T19:07:11.902 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_enqueue_scrub": 4, 2023-07-21T19:07:11.903 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_exportdir": 62, 2023-07-21T19:07:11.903 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_flush": 0, 2023-07-21T19:07:11.903 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_fragmentdir": 16, 2023-07-21T19:07:11.903 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_fragstats": 0, 2023-07-21T19:07:11.903 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_inodestats": 0 2023-07-21T19:07:11.903 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:11.903 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_log": { 2023-07-21T19:07:11.904 INFO:teuthology.orchestra.run.smithi155.stdout: "evadd": 188530, 2023-07-21T19:07:11.904 INFO:teuthology.orchestra.run.smithi155.stdout: "evex": 187693, 2023-07-21T19:07:11.904 INFO:teuthology.orchestra.run.smithi155.stdout: "evtrm": 187693, 2023-07-21T19:07:11.904 INFO:teuthology.orchestra.run.smithi155.stdout: "ev": 837, 2023-07-21T19:07:11.904 INFO:teuthology.orchestra.run.smithi155.stdout: "evexg": 0, 2023-07-21T19:07:11.904 INFO:teuthology.orchestra.run.smithi155.stdout: "evexd": 1992, 2023-07-21T19:07:11.904 INFO:teuthology.orchestra.run.smithi155.stdout: "segadd": 365, 2023-07-21T19:07:11.904 INFO:teuthology.orchestra.run.smithi155.stdout: "segex": 363, 2023-07-21T19:07:11.905 INFO:teuthology.orchestra.run.smithi155.stdout: "segtrm": 363, 2023-07-21T19:07:11.905 INFO:teuthology.orchestra.run.smithi155.stdout: "seg": 2, 2023-07-21T19:07:11.905 INFO:teuthology.orchestra.run.smithi155.stdout: "segexg": 0, 2023-07-21T19:07:11.905 INFO:teuthology.orchestra.run.smithi155.stdout: "segexd": 3, 2023-07-21T19:07:11.906 INFO:teuthology.orchestra.run.smithi155.stdout: "expos": 703277767, 2023-07-21T19:07:11.906 INFO:teuthology.orchestra.run.smithi155.stdout: "wrpos": 705147570, 2023-07-21T19:07:11.906 INFO:teuthology.orchestra.run.smithi155.stdout: "rdpos": 0, 2023-07-21T19:07:11.906 INFO:teuthology.orchestra.run.smithi155.stdout: "jlat": { 2023-07-21T19:07:11.906 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 74911, 2023-07-21T19:07:11.906 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 526.586886520, 2023-07-21T19:07:11.906 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.007029500 2023-07-21T19:07:11.906 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:11.907 INFO:teuthology.orchestra.run.smithi155.stdout: "replayed": 0 2023-07-21T19:07:11.907 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:11.907 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_mem": { 2023-07-21T19:07:11.907 INFO:teuthology.orchestra.run.smithi155.stdout: "ino": 641, 2023-07-21T19:07:11.907 INFO:teuthology.orchestra.run.smithi155.stdout: "ino+": 30295, 2023-07-21T19:07:11.907 INFO:teuthology.orchestra.run.smithi155.stdout: "ino-": 29654, 2023-07-21T19:07:11.907 INFO:teuthology.orchestra.run.smithi155.stdout: "dir": 95, 2023-07-21T19:07:11.908 INFO:teuthology.orchestra.run.smithi155.stdout: "dir+": 649, 2023-07-21T19:07:11.908 INFO:teuthology.orchestra.run.smithi155.stdout: "dir-": 554, 2023-07-21T19:07:11.908 INFO:teuthology.orchestra.run.smithi155.stdout: "dn": 1334, 2023-07-21T19:07:11.908 INFO:teuthology.orchestra.run.smithi155.stdout: "dn+": 58730, 2023-07-21T19:07:11.908 INFO:teuthology.orchestra.run.smithi155.stdout: "dn-": 57396, 2023-07-21T19:07:11.908 INFO:teuthology.orchestra.run.smithi155.stdout: "cap": 2, 2023-07-21T19:07:11.908 INFO:teuthology.orchestra.run.smithi155.stdout: "cap+": 31304, 2023-07-21T19:07:11.908 INFO:teuthology.orchestra.run.smithi155.stdout: "cap-": 31302, 2023-07-21T19:07:11.909 INFO:teuthology.orchestra.run.smithi155.stdout: "rss": 89128, 2023-07-21T19:07:11.909 INFO:teuthology.orchestra.run.smithi155.stdout: "heap": 207156 2023-07-21T19:07:11.909 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:11.909 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_server": { 2023-07-21T19:07:11.909 INFO:teuthology.orchestra.run.smithi155.stdout: "dispatch_client_request": 634045, 2023-07-21T19:07:11.909 INFO:teuthology.orchestra.run.smithi155.stdout: "dispatch_server_request": 30283, 2023-07-21T19:07:11.909 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_request": 362440, 2023-07-21T19:07:11.909 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_session": 3986, 2023-07-21T19:07:11.910 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_peer_request": 127646, 2023-07-21T19:07:11.910 INFO:teuthology.orchestra.run.smithi155.stdout: "req_create_latency": { 2023-07-21T19:07:11.910 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 26894, 2023-07-21T19:07:11.910 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 220.089575286, 2023-07-21T19:07:11.910 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.008183593 2023-07-21T19:07:11.910 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:11.910 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getattr_latency": { 2023-07-21T19:07:11.911 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 97574, 2023-07-21T19:07:11.911 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 1254.667308472, 2023-07-21T19:07:11.911 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.012858623 2023-07-21T19:07:11.911 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:11.911 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getfilelock_latency": { 2023-07-21T19:07:11.911 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:11.911 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:11.911 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:11.912 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:11.912 INFO:teuthology.orchestra.run.smithi155.stdout: "req_link_latency": { 2023-07-21T19:07:11.912 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:11.912 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:11.912 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:11.912 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:11.912 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookup_latency": { 2023-07-21T19:07:11.912 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 50174, 2023-07-21T19:07:11.913 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 244.712790107, 2023-07-21T19:07:11.913 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.004877282 2023-07-21T19:07:11.913 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:11.913 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookuphash_latency": { 2023-07-21T19:07:11.913 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:11.913 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:11.913 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:11.914 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:11.914 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupino_latency": { 2023-07-21T19:07:11.914 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:11.914 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:11.914 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:11.914 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:11.914 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupname_latency": { 2023-07-21T19:07:11.915 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:11.915 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:11.915 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:11.915 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:11.915 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupparent_latency": { 2023-07-21T19:07:11.915 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:11.915 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:11.915 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:11.916 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:11.916 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupsnap_latency": { 2023-07-21T19:07:11.916 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:11.916 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:11.916 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:11.916 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:11.916 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lssnap_latency": { 2023-07-21T19:07:11.916 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 26, 2023-07-21T19:07:11.917 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.010661034, 2023-07-21T19:07:11.917 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000410039 2023-07-21T19:07:11.917 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:11.917 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mkdir_latency": { 2023-07-21T19:07:11.917 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 119, 2023-07-21T19:07:11.917 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 1.465673681, 2023-07-21T19:07:11.917 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.012316585 2023-07-21T19:07:11.917 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:11.918 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mknod_latency": { 2023-07-21T19:07:11.918 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:11.918 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:11.918 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:11.918 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:11.918 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mksnap_latency": { 2023-07-21T19:07:11.918 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 26, 2023-07-21T19:07:11.919 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 18.271998179, 2023-07-21T19:07:11.919 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.702769160 2023-07-21T19:07:11.919 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:11.919 INFO:teuthology.orchestra.run.smithi155.stdout: "req_open_latency": { 2023-07-21T19:07:11.919 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 9, 2023-07-21T19:07:11.919 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.015259296, 2023-07-21T19:07:11.920 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.001695477 2023-07-21T19:07:11.920 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:11.920 INFO:teuthology.orchestra.run.smithi155.stdout: "req_readdir_latency": { 2023-07-21T19:07:11.920 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 104346, 2023-07-21T19:07:11.920 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 367.032465082, 2023-07-21T19:07:11.920 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.003517456 2023-07-21T19:07:11.920 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:11.920 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rename_latency": { 2023-07-21T19:07:11.921 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 4597, 2023-07-21T19:07:11.921 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 48.448475349, 2023-07-21T19:07:11.921 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.010539150 2023-07-21T19:07:11.921 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:11.921 INFO:teuthology.orchestra.run.smithi155.stdout: "req_renamesnap_latency": { 2023-07-21T19:07:11.921 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:11.921 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:11.922 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:11.922 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:11.922 INFO:teuthology.orchestra.run.smithi155.stdout: "req_snapdiff_latency": { 2023-07-21T19:07:11.922 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:11.923 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:11.923 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:11.923 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:11.923 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmdir_latency": { 2023-07-21T19:07:11.923 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 34, 2023-07-21T19:07:11.924 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.663323889, 2023-07-21T19:07:11.924 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.019509526 2023-07-21T19:07:11.924 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:11.924 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmsnap_latency": { 2023-07-21T19:07:11.924 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 19, 2023-07-21T19:07:11.924 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 14.710754472, 2023-07-21T19:07:11.924 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.774250235 2023-07-21T19:07:11.925 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:11.925 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmxattr_latency": { 2023-07-21T19:07:11.925 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:11.925 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:11.925 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:11.925 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:11.925 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setattr_latency": { 2023-07-21T19:07:11.925 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 82, 2023-07-21T19:07:11.926 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 1.134586434, 2023-07-21T19:07:11.926 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.013836419 2023-07-21T19:07:11.926 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:11.926 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setdirlayout_latency": { 2023-07-21T19:07:11.926 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:11.926 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:11.926 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:11.927 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:11.927 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setfilelock_latency": { 2023-07-21T19:07:11.927 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 800, 2023-07-21T19:07:11.927 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 2.620859382, 2023-07-21T19:07:11.927 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.003276074 2023-07-21T19:07:11.927 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:11.927 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setlayout_latency": { 2023-07-21T19:07:11.927 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:11.928 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:11.928 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:11.928 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:11.928 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setxattr_latency": { 2023-07-21T19:07:11.928 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 6, 2023-07-21T19:07:11.928 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.018232326, 2023-07-21T19:07:11.928 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.003038721 2023-07-21T19:07:11.928 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:11.929 INFO:teuthology.orchestra.run.smithi155.stdout: "req_symlink_latency": { 2023-07-21T19:07:11.929 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:11.929 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:11.929 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:11.929 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:11.929 INFO:teuthology.orchestra.run.smithi155.stdout: "req_unlink_latency": { 2023-07-21T19:07:11.929 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 26392, 2023-07-21T19:07:11.929 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 183.209375508, 2023-07-21T19:07:11.930 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.006941852 2023-07-21T19:07:11.930 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:11.930 INFO:teuthology.orchestra.run.smithi155.stdout: "cap_revoke_eviction": 0, 2023-07-21T19:07:11.930 INFO:teuthology.orchestra.run.smithi155.stdout: "cap_acquisition_throttle": 0, 2023-07-21T19:07:11.930 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getvxattr_latency": { 2023-07-21T19:07:11.930 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:11.930 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:11.931 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:11.931 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-07-21T19:07:11.931 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:11.931 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_sessions": { 2023-07-21T19:07:11.931 INFO:teuthology.orchestra.run.smithi155.stdout: "session_count": 2, 2023-07-21T19:07:11.931 INFO:teuthology.orchestra.run.smithi155.stdout: "session_add": 5, 2023-07-21T19:07:11.931 INFO:teuthology.orchestra.run.smithi155.stdout: "session_remove": 3, 2023-07-21T19:07:11.931 INFO:teuthology.orchestra.run.smithi155.stdout: "sessions_open": 2, 2023-07-21T19:07:11.932 INFO:teuthology.orchestra.run.smithi155.stdout: "sessions_stale": 0, 2023-07-21T19:07:11.932 INFO:teuthology.orchestra.run.smithi155.stdout: "total_load": 22754, 2023-07-21T19:07:11.932 INFO:teuthology.orchestra.run.smithi155.stdout: "average_load": 11377, 2023-07-21T19:07:11.932 INFO:teuthology.orchestra.run.smithi155.stdout: "avg_session_uptime": 1523 2023-07-21T19:07:11.932 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:11.932 INFO:teuthology.orchestra.run.smithi155.stdout: "mempool": { 2023-07-21T19:07:11.932 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_bytes": 360, 2023-07-21T19:07:11.933 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_items": 360, 2023-07-21T19:07:11.933 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_bytes": 0, 2023-07-21T19:07:11.933 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_items": 0, 2023-07-21T19:07:11.933 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_bytes": 0, 2023-07-21T19:07:11.933 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_items": 0, 2023-07-21T19:07:11.933 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-21T19:07:11.933 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_items": 0, 2023-07-21T19:07:11.933 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-21T19:07:11.934 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_items": 0, 2023-07-21T19:07:11.934 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_bytes": 0, 2023-07-21T19:07:11.934 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_items": 0, 2023-07-21T19:07:11.934 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-21T19:07:11.934 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_items": 0, 2023-07-21T19:07:11.934 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_bytes": 0, 2023-07-21T19:07:11.934 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_items": 0, 2023-07-21T19:07:11.934 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_bytes": 0, 2023-07-21T19:07:11.935 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_items": 0, 2023-07-21T19:07:11.935 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-21T19:07:11.935 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_items": 0, 2023-07-21T19:07:11.935 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-21T19:07:11.935 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_items": 0, 2023-07-21T19:07:11.935 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_bytes": 0, 2023-07-21T19:07:11.935 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_items": 0, 2023-07-21T19:07:11.936 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_bytes": 0, 2023-07-21T19:07:11.936 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_items": 0, 2023-07-21T19:07:11.936 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-21T19:07:11.936 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_items": 0, 2023-07-21T19:07:11.936 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_bytes": 0, 2023-07-21T19:07:11.936 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_items": 0, 2023-07-21T19:07:11.936 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_bytes": 0, 2023-07-21T19:07:11.936 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_items": 0, 2023-07-21T19:07:11.937 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_bytes": 0, 2023-07-21T19:07:11.937 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_items": 0, 2023-07-21T19:07:11.937 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_bytes": 0, 2023-07-21T19:07:11.937 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_items": 0, 2023-07-21T19:07:11.937 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_bytes": 1562612, 2023-07-21T19:07:11.937 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_items": 899, 2023-07-21T19:07:11.937 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_bytes": 0, 2023-07-21T19:07:11.937 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_items": 0, 2023-07-21T19:07:11.938 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_bytes": 0, 2023-07-21T19:07:11.938 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_items": 0, 2023-07-21T19:07:11.938 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_bytes": 0, 2023-07-21T19:07:11.938 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_items": 0, 2023-07-21T19:07:11.938 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_bytes": 0, 2023-07-21T19:07:11.938 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_items": 0, 2023-07-21T19:07:11.938 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_bytes": 8280, 2023-07-21T19:07:11.939 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_items": 189, 2023-07-21T19:07:11.939 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_bytes": 0, 2023-07-21T19:07:11.939 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_items": 0, 2023-07-21T19:07:11.939 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_bytes": 0, 2023-07-21T19:07:11.939 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_items": 0, 2023-07-21T19:07:11.939 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_bytes": 3339971, 2023-07-21T19:07:11.939 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_items": 46510, 2023-07-21T19:07:11.939 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_bytes": 0, 2023-07-21T19:07:11.940 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_items": 0, 2023-07-21T19:07:11.940 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_bytes": 0, 2023-07-21T19:07:11.940 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_items": 0 2023-07-21T19:07:11.940 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:11.940 INFO:teuthology.orchestra.run.smithi155.stdout: "objecter": { 2023-07-21T19:07:11.940 INFO:teuthology.orchestra.run.smithi155.stdout: "op_active": 0, 2023-07-21T19:07:11.940 INFO:teuthology.orchestra.run.smithi155.stdout: "op_laggy": 0, 2023-07-21T19:07:11.941 INFO:teuthology.orchestra.run.smithi155.stdout: "op_send": 124505, 2023-07-21T19:07:11.941 INFO:teuthology.orchestra.run.smithi155.stdout: "op_send_bytes": 734513154, 2023-07-21T19:07:11.941 INFO:teuthology.orchestra.run.smithi155.stdout: "op_resend": 31, 2023-07-21T19:07:11.941 INFO:teuthology.orchestra.run.smithi155.stdout: "op_reply": 124505, 2023-07-21T19:07:11.941 INFO:teuthology.orchestra.run.smithi155.stdout: "op_latency": { 2023-07-21T19:07:11.941 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 124505, 2023-07-21T19:07:11.941 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 2252.173017476, 2023-07-21T19:07:11.941 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.018089016 2023-07-21T19:07:11.942 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:11.942 INFO:teuthology.orchestra.run.smithi155.stdout: "op_inflight": 0, 2023-07-21T19:07:11.942 INFO:teuthology.orchestra.run.smithi155.stdout: "oplen_avg": { 2023-07-21T19:07:11.942 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 124505, 2023-07-21T19:07:11.942 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 163561 2023-07-21T19:07:11.942 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:11.942 INFO:teuthology.orchestra.run.smithi155.stdout: "op": 124505, 2023-07-21T19:07:11.942 INFO:teuthology.orchestra.run.smithi155.stdout: "op_r": 1225, 2023-07-21T19:07:11.943 INFO:teuthology.orchestra.run.smithi155.stdout: "op_w": 123280, 2023-07-21T19:07:11.943 INFO:teuthology.orchestra.run.smithi155.stdout: "op_rmw": 0, 2023-07-21T19:07:11.943 INFO:teuthology.orchestra.run.smithi155.stdout: "op_pg": 0, 2023-07-21T19:07:11.943 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_stat": 3824, 2023-07-21T19:07:11.943 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_create": 14453, 2023-07-21T19:07:11.943 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_read": 1020, 2023-07-21T19:07:11.943 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_write": 75927, 2023-07-21T19:07:11.944 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_writefull": 1034, 2023-07-21T19:07:11.944 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_writesame": 0, 2023-07-21T19:07:11.944 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_append": 0, 2023-07-21T19:07:11.944 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_zero": 0, 2023-07-21T19:07:11.944 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_truncate": 10, 2023-07-21T19:07:11.944 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_delete": 27278, 2023-07-21T19:07:11.945 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_mapext": 0, 2023-07-21T19:07:11.945 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_sparse_read": 0, 2023-07-21T19:07:11.945 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_clonerange": 0, 2023-07-21T19:07:11.945 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_getxattr": 115, 2023-07-21T19:07:11.945 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_setxattr": 28906, 2023-07-21T19:07:11.945 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_cmpxattr": 0, 2023-07-21T19:07:11.945 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_rmxattr": 0, 2023-07-21T19:07:11.945 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_resetxattrs": 0, 2023-07-21T19:07:11.946 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_call": 0, 2023-07-21T19:07:11.946 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_watch": 0, 2023-07-21T19:07:11.946 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_notify": 0, 2023-07-21T19:07:11.946 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_src_cmpxattr": 0, 2023-07-21T19:07:11.946 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_pgls": 0, 2023-07-21T19:07:11.946 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_pgls_filter": 0, 2023-07-21T19:07:11.946 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_other": 0, 2023-07-21T19:07:11.946 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_active": 0, 2023-07-21T19:07:11.947 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_send": 0, 2023-07-21T19:07:11.947 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_resend": 0, 2023-07-21T19:07:11.947 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_ping": 0, 2023-07-21T19:07:11.947 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_active": 0, 2023-07-21T19:07:11.947 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_send": 0, 2023-07-21T19:07:11.947 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_resend": 0, 2023-07-21T19:07:11.947 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_active": 0, 2023-07-21T19:07:11.947 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_send": 0, 2023-07-21T19:07:11.948 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_resend": 0, 2023-07-21T19:07:11.948 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_active": 0, 2023-07-21T19:07:11.948 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_send": 0, 2023-07-21T19:07:11.948 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_resend": 0, 2023-07-21T19:07:11.948 INFO:teuthology.orchestra.run.smithi155.stdout: "command_active": 0, 2023-07-21T19:07:11.948 INFO:teuthology.orchestra.run.smithi155.stdout: "command_send": 0, 2023-07-21T19:07:11.948 INFO:teuthology.orchestra.run.smithi155.stdout: "command_resend": 0, 2023-07-21T19:07:11.948 INFO:teuthology.orchestra.run.smithi155.stdout: "map_epoch": 123, 2023-07-21T19:07:11.949 INFO:teuthology.orchestra.run.smithi155.stdout: "map_full": 0, 2023-07-21T19:07:11.949 INFO:teuthology.orchestra.run.smithi155.stdout: "map_inc": 45, 2023-07-21T19:07:11.949 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_sessions": 12, 2023-07-21T19:07:11.949 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_session_open": 12, 2023-07-21T19:07:11.949 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_session_close": 0, 2023-07-21T19:07:11.949 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_laggy": 0, 2023-07-21T19:07:11.949 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_wr": 8450, 2023-07-21T19:07:11.950 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_rd": 151, 2023-07-21T19:07:11.950 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_del": 2393 2023-07-21T19:07:11.950 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:11.950 INFO:teuthology.orchestra.run.smithi155.stdout: "oft": { 2023-07-21T19:07:11.950 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_objs": 1, 2023-07-21T19:07:11.950 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_kv_pairs": 0, 2023-07-21T19:07:11.950 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_updates": 0, 2023-07-21T19:07:11.950 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_removes": 0 2023-07-21T19:07:11.951 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:11.951 INFO:teuthology.orchestra.run.smithi155.stdout: "osd-slow-ops": { 2023-07-21T19:07:11.951 INFO:teuthology.orchestra.run.smithi155.stdout: "slow_ops_count": 392652 2023-07-21T19:07:11.951 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:11.951 INFO:teuthology.orchestra.run.smithi155.stdout: "purge_queue": { 2023-07-21T19:07:11.951 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_ops": 0, 2023-07-21T19:07:11.951 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_ops_high_water": 6, 2023-07-21T19:07:11.951 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing": 0, 2023-07-21T19:07:11.952 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_high_water": 6, 2023-07-21T19:07:11.952 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executed": 26392, 2023-07-21T19:07:11.952 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_item_in_journal": 0 2023-07-21T19:07:11.952 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:11.952 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-21T19:07:11.952 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2023-07-21T19:07:11.953 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 104857600, 2023-07-21T19:07:11.953 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2023-07-21T19:07:11.953 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 1777486, 2023-07-21T19:07:11.953 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 360822304, 2023-07-21T19:07:11.953 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2023-07-21T19:07:11.953 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 1777486, 2023-07-21T19:07:11.953 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2023-07-21T19:07:11.954 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2023-07-21T19:07:11.954 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 1777486, 2023-07-21T19:07:11.954 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 360822304, 2023-07-21T19:07:11.954 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2023-07-21T19:07:11.954 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:11.954 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:11.954 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:11.954 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-07-21T19:07:11.955 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:11.955 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-objecter_bytes": { 2023-07-21T19:07:11.955 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2023-07-21T19:07:11.955 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 104857600, 2023-07-21T19:07:11.955 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2023-07-21T19:07:11.955 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2023-07-21T19:07:11.955 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2023-07-21T19:07:11.955 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2023-07-21T19:07:11.956 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2023-07-21T19:07:11.956 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 124505, 2023-07-21T19:07:11.956 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 737184556, 2023-07-21T19:07:11.956 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 124505, 2023-07-21T19:07:11.956 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 737184556, 2023-07-21T19:07:11.956 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2023-07-21T19:07:11.956 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:11.956 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:11.957 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:11.957 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-07-21T19:07:11.957 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:11.957 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-objecter_ops": { 2023-07-21T19:07:11.957 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2023-07-21T19:07:11.957 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 1024, 2023-07-21T19:07:11.957 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2023-07-21T19:07:11.958 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2023-07-21T19:07:11.958 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2023-07-21T19:07:11.958 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2023-07-21T19:07:11.958 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2023-07-21T19:07:11.958 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 124505, 2023-07-21T19:07:11.958 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 124505, 2023-07-21T19:07:11.958 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 124505, 2023-07-21T19:07:11.958 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 124505, 2023-07-21T19:07:11.959 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2023-07-21T19:07:11.959 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:11.959 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:11.959 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:11.959 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-07-21T19:07:11.959 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:11.959 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-write_buf_throttle": { 2023-07-21T19:07:11.959 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2023-07-21T19:07:11.960 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 3758096384, 2023-07-21T19:07:11.960 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2023-07-21T19:07:11.960 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 26392, 2023-07-21T19:07:11.960 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 2674152, 2023-07-21T19:07:11.960 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2023-07-21T19:07:11.960 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 26392, 2023-07-21T19:07:11.960 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2023-07-21T19:07:11.961 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2023-07-21T19:07:11.961 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 1016, 2023-07-21T19:07:11.961 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 2674152, 2023-07-21T19:07:11.961 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2023-07-21T19:07:11.961 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:11.961 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:11.961 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:11.961 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-07-21T19:07:11.962 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:11.962 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-write_buf_throttle-0x55ae27a320a0": { 2023-07-21T19:07:11.962 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2023-07-21T19:07:11.962 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 3758096384, 2023-07-21T19:07:11.962 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2023-07-21T19:07:11.962 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 188530, 2023-07-21T19:07:11.962 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 700953266, 2023-07-21T19:07:11.962 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2023-07-21T19:07:11.963 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 188530, 2023-07-21T19:07:11.963 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2023-07-21T19:07:11.963 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2023-07-21T19:07:11.963 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 74911, 2023-07-21T19:07:11.963 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 700953266, 2023-07-21T19:07:11.964 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2023-07-21T19:07:11.964 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:11.964 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:11.964 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:11.964 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-07-21T19:07:11.965 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-07-21T19:07:11.965 INFO:teuthology.orchestra.run.smithi155.stdout:} 2023-07-21T19:07:12.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:12 smithi155 ceph-mon[97934]: pgmap v807: 97 pgs: 2 active+clean+snaptrim, 95 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 10 MiB/s wr, 325 op/s 2023-07-21T19:07:12.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:12 smithi193 ceph-mon[98204]: pgmap v807: 97 pgs: 2 active+clean+snaptrim, 95 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 10 MiB/s wr, 325 op/s 2023-07-21T19:07:12.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:12 smithi039 ceph-mon[96489]: pgmap v807: 97 pgs: 2 active+clean+snaptrim, 95 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 10 MiB/s wr, 325 op/s 2023-07-21T19:07:13.902 INFO:tasks.check_counter:Daemon mds.b mds.exported=55 2023-07-21T19:07:13.902 INFO:tasks.check_counter:Daemon mds.b mds.imported=37 2023-07-21T19:07:13.903 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=4676 2023-07-21T19:07:13.903 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=493 2023-07-21T19:07:13.903 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=7 2023-07-21T19:07:13.903 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=26 2023-07-21T19:07:13.904 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=19 2023-07-21T19:07:13.904 DEBUG:tasks.check_counter:Getting stats from e 2023-07-21T19:07:13.905 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph daemon mds.e perf dump 2023-07-21T19:07:14.499 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.b/config 2023-07-21T19:07:14.751 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:14 smithi155 ceph-mon[97934]: pgmap v808: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.6 MiB/s wr, 236 op/s 2023-07-21T19:07:14.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:14 smithi193 ceph-mon[98204]: pgmap v808: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.6 MiB/s wr, 236 op/s 2023-07-21T19:07:14.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:14 smithi039 ceph-mon[96489]: pgmap v808: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.6 MiB/s wr, 236 op/s 2023-07-21T19:07:15.886 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2023-07-21T19:07:15.886 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-0": { 2023-07-21T19:07:15.887 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 413083, 2023-07-21T19:07:15.887 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 610551, 2023-07-21T19:07:15.887 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 121810005, 2023-07-21T19:07:15.887 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 722657993, 2023-07-21T19:07:15.887 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 25, 2023-07-21T19:07:15.887 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 4, 2023-07-21T19:07:15.887 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 21.894824358, 2023-07-21T19:07:15.887 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 10.563747362, 2023-07-21T19:07:15.888 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 15.724731557, 2023-07-21T19:07:15.888 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 1.654792208, 2023-07-21T19:07:15.888 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:07:15.888 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 610546, 2023-07-21T19:07:15.888 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 113.284268000, 2023-07-21T19:07:15.888 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000185545 2023-07-21T19:07:15.889 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:15.889 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:07:15.889 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 471802, 2023-07-21T19:07:15.889 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.131871303, 2023-07-21T19:07:15.889 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000279 2023-07-21T19:07:15.889 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:15.889 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 912, 2023-07-21T19:07:15.889 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 1168 2023-07-21T19:07:15.890 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:15.890 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-1": { 2023-07-21T19:07:15.890 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 229604, 2023-07-21T19:07:15.890 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 221708, 2023-07-21T19:07:15.890 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 76826440, 2023-07-21T19:07:15.890 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 481461522, 2023-07-21T19:07:15.890 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 13, 2023-07-21T19:07:15.890 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 6, 2023-07-21T19:07:15.891 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 13.184610940, 2023-07-21T19:07:15.891 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 5.100941215, 2023-07-21T19:07:15.891 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 8.168980587, 2023-07-21T19:07:15.891 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 1.344170754, 2023-07-21T19:07:15.891 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:07:15.891 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 221707, 2023-07-21T19:07:15.891 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 44.307253822, 2023-07-21T19:07:15.892 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000199845 2023-07-21T19:07:15.892 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:15.892 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:07:15.892 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 378755, 2023-07-21T19:07:15.892 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.072826590, 2023-07-21T19:07:15.892 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000192 2023-07-21T19:07:15.892 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:15.892 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 6464, 2023-07-21T19:07:15.893 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 746688 2023-07-21T19:07:15.893 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:15.893 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-2": { 2023-07-21T19:07:15.893 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 725964, 2023-07-21T19:07:15.893 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 679928, 2023-07-21T19:07:15.893 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 207269231, 2023-07-21T19:07:15.893 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 467682112, 2023-07-21T19:07:15.894 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 8, 2023-07-21T19:07:15.894 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 6, 2023-07-21T19:07:15.894 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 40.794642351, 2023-07-21T19:07:15.894 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 18.020444876, 2023-07-21T19:07:15.894 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 27.920519947, 2023-07-21T19:07:15.894 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 1.300275174, 2023-07-21T19:07:15.894 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:07:15.895 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 679929, 2023-07-21T19:07:15.895 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 84.231355602, 2023-07-21T19:07:15.895 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000123882 2023-07-21T19:07:15.895 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:15.895 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:07:15.895 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 1314908, 2023-07-21T19:07:15.895 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.219890659, 2023-07-21T19:07:15.895 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000167 2023-07-21T19:07:15.896 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:15.896 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 1931760, 2023-07-21T19:07:15.896 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 269088 2023-07-21T19:07:15.896 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:15.896 INFO:teuthology.orchestra.run.smithi155.stdout: "cct": { 2023-07-21T19:07:15.896 INFO:teuthology.orchestra.run.smithi155.stdout: "total_workers": 1, 2023-07-21T19:07:15.896 INFO:teuthology.orchestra.run.smithi155.stdout: "unhealthy_workers": 0 2023-07-21T19:07:15.896 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:15.897 INFO:teuthology.orchestra.run.smithi155.stdout: "finisher-MDSRank": { 2023-07-21T19:07:15.897 INFO:teuthology.orchestra.run.smithi155.stdout: "queue_len": 0, 2023-07-21T19:07:15.897 INFO:teuthology.orchestra.run.smithi155.stdout: "complete_latency": { 2023-07-21T19:07:15.897 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 91746, 2023-07-21T19:07:15.897 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 117.875268603, 2023-07-21T19:07:15.897 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.001284800 2023-07-21T19:07:15.897 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-07-21T19:07:15.897 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:15.898 INFO:teuthology.orchestra.run.smithi155.stdout: "finisher-PurgeQueue": { 2023-07-21T19:07:15.898 INFO:teuthology.orchestra.run.smithi155.stdout: "queue_len": 0, 2023-07-21T19:07:15.898 INFO:teuthology.orchestra.run.smithi155.stdout: "complete_latency": { 2023-07-21T19:07:15.898 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 56425, 2023-07-21T19:07:15.898 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 11.063871480, 2023-07-21T19:07:15.898 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000196081 2023-07-21T19:07:15.898 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-07-21T19:07:15.899 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:15.899 INFO:teuthology.orchestra.run.smithi155.stdout: "mds": { 2023-07-21T19:07:15.899 INFO:teuthology.orchestra.run.smithi155.stdout: "request": 265379, 2023-07-21T19:07:15.899 INFO:teuthology.orchestra.run.smithi155.stdout: "reply": 222825, 2023-07-21T19:07:15.899 INFO:teuthology.orchestra.run.smithi155.stdout: "reply_latency": { 2023-07-21T19:07:15.899 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 222825, 2023-07-21T19:07:15.899 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 958.696785017, 2023-07-21T19:07:15.899 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.004302465 2023-07-21T19:07:15.900 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:15.900 INFO:teuthology.orchestra.run.smithi155.stdout: "slow_reply": 0, 2023-07-21T19:07:15.900 INFO:teuthology.orchestra.run.smithi155.stdout: "forward": 42548, 2023-07-21T19:07:15.900 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_fetch_complete": 9, 2023-07-21T19:07:15.900 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_fetch_keys": 0, 2023-07-21T19:07:15.900 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_commit": 106, 2023-07-21T19:07:15.900 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_split": 7, 2023-07-21T19:07:15.901 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_merge": 6, 2023-07-21T19:07:15.901 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes": 2162, 2023-07-21T19:07:15.901 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_top": 976, 2023-07-21T19:07:15.901 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_bottom": 1129, 2023-07-21T19:07:15.901 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_pin_tail": 57, 2023-07-21T19:07:15.901 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_pinned": 752, 2023-07-21T19:07:15.901 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_expired": 1332, 2023-07-21T19:07:15.901 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_with_caps": 1, 2023-07-21T19:07:15.902 INFO:teuthology.orchestra.run.smithi155.stdout: "caps": 1, 2023-07-21T19:07:15.902 INFO:teuthology.orchestra.run.smithi155.stdout: "subtrees": 6, 2023-07-21T19:07:15.902 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse": 432445, 2023-07-21T19:07:15.902 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_hit": 224341, 2023-07-21T19:07:15.902 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_forward": 42530, 2023-07-21T19:07:15.902 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_discover": 373, 2023-07-21T19:07:15.902 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_dir_fetch": 3, 2023-07-21T19:07:15.902 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_remote_ino": 0, 2023-07-21T19:07:15.903 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_lock": 6877, 2023-07-21T19:07:15.903 INFO:teuthology.orchestra.run.smithi155.stdout: "load_cent": 0, 2023-07-21T19:07:15.903 INFO:teuthology.orchestra.run.smithi155.stdout: "q": 0, 2023-07-21T19:07:15.903 INFO:teuthology.orchestra.run.smithi155.stdout: "exported": 18, 2023-07-21T19:07:15.903 INFO:teuthology.orchestra.run.smithi155.stdout: "exported_inodes": 637, 2023-07-21T19:07:15.903 INFO:teuthology.orchestra.run.smithi155.stdout: "imported": 19, 2023-07-21T19:07:15.903 INFO:teuthology.orchestra.run.smithi155.stdout: "imported_inodes": 870, 2023-07-21T19:07:15.904 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_dir_fetch": 0, 2023-07-21T19:07:15.904 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_backtrace_fetch": 0, 2023-07-21T19:07:15.904 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_peer_discover": 0, 2023-07-21T19:07:15.904 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rfiles": 131, 2023-07-21T19:07:15.904 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rbytes": 42134725, 2023-07-21T19:07:15.904 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rsnaps": 7, 2023-07-21T19:07:15.904 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_backtrace_fetch": 0, 2023-07-21T19:07:15.904 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_set_tag": 0, 2023-07-21T19:07:15.905 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_backtrace_repaired": 0, 2023-07-21T19:07:15.905 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_inotable_repaired": 0, 2023-07-21T19:07:15.905 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dir_inodes": 0, 2023-07-21T19:07:15.905 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dir_base_inodes": 0, 2023-07-21T19:07:15.905 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dirfrag_rstats": 0, 2023-07-21T19:07:15.905 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_file_inodes": 0, 2023-07-21T19:07:15.905 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_inode_file_caps": 29, 2023-07-21T19:07:15.906 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_revoke": 49473, 2023-07-21T19:07:15.906 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_grant": 149276, 2023-07-21T19:07:15.906 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_trunc": 0, 2023-07-21T19:07:15.906 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_flushsnap_ack": 1633, 2023-07-21T19:07:15.906 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-21T19:07:15.906 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_caps": 95671, 2023-07-21T19:07:15.906 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_caps_dirty": 31303, 2023-07-21T19:07:15.906 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_cap_release": 4814, 2023-07-21T19:07:15.907 INFO:teuthology.orchestra.run.smithi155.stdout: "process_request_cap_release": 183968 2023-07-21T19:07:15.907 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:15.907 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_cache": { 2023-07-21T19:07:15.907 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_update": 966, 2023-07-21T19:07:15.907 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_update_receipt": 5563, 2023-07-21T19:07:15.907 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_try_discover": 2851, 2023-07-21T19:07:15.907 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_send_discover": 5723, 2023-07-21T19:07:15.908 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_handle_discover": 2272, 2023-07-21T19:07:15.908 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays": 20, 2023-07-21T19:07:15.908 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays_delayed": 0, 2023-07-21T19:07:15.908 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays_enqueuing": 0, 2023-07-21T19:07:15.908 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_created": 32258, 2023-07-21T19:07:15.908 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_enqueued": 32238, 2023-07-21T19:07:15.908 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_reintegrated": 0, 2023-07-21T19:07:15.908 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_migrated": 0, 2023-07-21T19:07:15.909 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_processing": 0, 2023-07-21T19:07:15.909 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_enqueued": 0, 2023-07-21T19:07:15.909 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_prioritized": 0, 2023-07-21T19:07:15.909 INFO:teuthology.orchestra.run.smithi155.stdout: "recovery_started": 0, 2023-07-21T19:07:15.909 INFO:teuthology.orchestra.run.smithi155.stdout: "recovery_completed": 0, 2023-07-21T19:07:15.909 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_enqueue_scrub": 0, 2023-07-21T19:07:15.909 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_exportdir": 22, 2023-07-21T19:07:15.909 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_flush": 0, 2023-07-21T19:07:15.910 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_fragmentdir": 13, 2023-07-21T19:07:15.910 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_fragstats": 0, 2023-07-21T19:07:15.910 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_inodestats": 0 2023-07-21T19:07:15.910 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:15.910 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_log": { 2023-07-21T19:07:15.910 INFO:teuthology.orchestra.run.smithi155.stdout: "evadd": 167601, 2023-07-21T19:07:15.910 INFO:teuthology.orchestra.run.smithi155.stdout: "evex": 109355, 2023-07-21T19:07:15.911 INFO:teuthology.orchestra.run.smithi155.stdout: "evtrm": 109355, 2023-07-21T19:07:15.911 INFO:teuthology.orchestra.run.smithi155.stdout: "ev": 58246, 2023-07-21T19:07:15.911 INFO:teuthology.orchestra.run.smithi155.stdout: "evexg": 0, 2023-07-21T19:07:15.911 INFO:teuthology.orchestra.run.smithi155.stdout: "evexd": 430, 2023-07-21T19:07:15.911 INFO:teuthology.orchestra.run.smithi155.stdout: "segadd": 284, 2023-07-21T19:07:15.911 INFO:teuthology.orchestra.run.smithi155.stdout: "segex": 156, 2023-07-21T19:07:15.912 INFO:teuthology.orchestra.run.smithi155.stdout: "segtrm": 156, 2023-07-21T19:07:15.912 INFO:teuthology.orchestra.run.smithi155.stdout: "seg": 129, 2023-07-21T19:07:15.912 INFO:teuthology.orchestra.run.smithi155.stdout: "segexg": 0, 2023-07-21T19:07:15.912 INFO:teuthology.orchestra.run.smithi155.stdout: "segexd": 1, 2023-07-21T19:07:15.912 INFO:teuthology.orchestra.run.smithi155.stdout: "expos": 473973044, 2023-07-21T19:07:15.912 INFO:teuthology.orchestra.run.smithi155.stdout: "wrpos": 1005320000, 2023-07-21T19:07:15.912 INFO:teuthology.orchestra.run.smithi155.stdout: "rdpos": 0, 2023-07-21T19:07:15.913 INFO:teuthology.orchestra.run.smithi155.stdout: "jlat": { 2023-07-21T19:07:15.913 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 50234, 2023-07-21T19:07:15.913 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 435.372655619, 2023-07-21T19:07:15.913 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.008666892 2023-07-21T19:07:15.913 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:15.913 INFO:teuthology.orchestra.run.smithi155.stdout: "replayed": 0 2023-07-21T19:07:15.913 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:15.913 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_mem": { 2023-07-21T19:07:15.914 INFO:teuthology.orchestra.run.smithi155.stdout: "ino": 1293, 2023-07-21T19:07:15.914 INFO:teuthology.orchestra.run.smithi155.stdout: "ino+": 34424, 2023-07-21T19:07:15.914 INFO:teuthology.orchestra.run.smithi155.stdout: "ino-": 33131, 2023-07-21T19:07:15.914 INFO:teuthology.orchestra.run.smithi155.stdout: "dir": 91, 2023-07-21T19:07:15.914 INFO:teuthology.orchestra.run.smithi155.stdout: "dir+": 3018, 2023-07-21T19:07:15.914 INFO:teuthology.orchestra.run.smithi155.stdout: "dir-": 2927, 2023-07-21T19:07:15.914 INFO:teuthology.orchestra.run.smithi155.stdout: "dn": 2197, 2023-07-21T19:07:15.915 INFO:teuthology.orchestra.run.smithi155.stdout: "dn+": 61092, 2023-07-21T19:07:15.915 INFO:teuthology.orchestra.run.smithi155.stdout: "dn-": 58895, 2023-07-21T19:07:15.915 INFO:teuthology.orchestra.run.smithi155.stdout: "cap": 1, 2023-07-21T19:07:15.915 INFO:teuthology.orchestra.run.smithi155.stdout: "cap+": 34580, 2023-07-21T19:07:15.915 INFO:teuthology.orchestra.run.smithi155.stdout: "cap-": 34579, 2023-07-21T19:07:15.915 INFO:teuthology.orchestra.run.smithi155.stdout: "rss": 83104, 2023-07-21T19:07:15.915 INFO:teuthology.orchestra.run.smithi155.stdout: "heap": 198964 2023-07-21T19:07:15.915 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:15.916 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_server": { 2023-07-21T19:07:15.916 INFO:teuthology.orchestra.run.smithi155.stdout: "dispatch_client_request": 396230, 2023-07-21T19:07:15.916 INFO:teuthology.orchestra.run.smithi155.stdout: "dispatch_server_request": 17405, 2023-07-21T19:07:15.916 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_request": 265379, 2023-07-21T19:07:15.916 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_session": 5438, 2023-07-21T19:07:15.916 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_peer_request": 48033, 2023-07-21T19:07:15.916 INFO:teuthology.orchestra.run.smithi155.stdout: "req_create_latency": { 2023-07-21T19:07:15.916 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 32157, 2023-07-21T19:07:15.917 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 194.940647927, 2023-07-21T19:07:15.917 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.006062152 2023-07-21T19:07:15.917 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:15.917 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getattr_latency": { 2023-07-21T19:07:15.917 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 4623, 2023-07-21T19:07:15.917 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 65.992559906, 2023-07-21T19:07:15.917 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.014274834 2023-07-21T19:07:15.917 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:15.918 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getfilelock_latency": { 2023-07-21T19:07:15.918 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:15.918 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:15.918 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:15.918 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:15.918 INFO:teuthology.orchestra.run.smithi155.stdout: "req_link_latency": { 2023-07-21T19:07:15.918 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:15.919 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:15.919 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:15.919 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:15.919 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookup_latency": { 2023-07-21T19:07:15.919 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 63589, 2023-07-21T19:07:15.919 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 148.170330380, 2023-07-21T19:07:15.919 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.002330125 2023-07-21T19:07:15.919 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:15.920 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookuphash_latency": { 2023-07-21T19:07:15.920 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:15.920 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:15.920 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:15.920 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:15.920 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupino_latency": { 2023-07-21T19:07:15.920 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 1, 2023-07-21T19:07:15.920 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000157966, 2023-07-21T19:07:15.921 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000157966 2023-07-21T19:07:15.921 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:15.921 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupname_latency": { 2023-07-21T19:07:15.921 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:15.921 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:15.921 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:15.922 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:15.922 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupparent_latency": { 2023-07-21T19:07:15.922 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:15.922 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:15.922 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:15.922 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:15.922 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupsnap_latency": { 2023-07-21T19:07:15.922 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:15.923 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:15.923 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:15.923 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:15.923 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lssnap_latency": { 2023-07-21T19:07:15.923 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:15.923 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:15.923 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:15.924 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:15.924 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mkdir_latency": { 2023-07-21T19:07:15.924 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:15.924 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:15.924 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:15.924 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:15.924 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mknod_latency": { 2023-07-21T19:07:15.924 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:15.925 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:15.925 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:15.925 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:15.925 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mksnap_latency": { 2023-07-21T19:07:15.925 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:15.925 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:15.925 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:15.926 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:15.926 INFO:teuthology.orchestra.run.smithi155.stdout: "req_open_latency": { 2023-07-21T19:07:15.926 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:15.926 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:15.926 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:15.926 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:15.926 INFO:teuthology.orchestra.run.smithi155.stdout: "req_readdir_latency": { 2023-07-21T19:07:15.926 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 81004, 2023-07-21T19:07:15.927 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 333.080208228, 2023-07-21T19:07:15.927 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.004111898 2023-07-21T19:07:15.927 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:15.927 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rename_latency": { 2023-07-21T19:07:15.927 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 8107, 2023-07-21T19:07:15.927 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 58.415811749, 2023-07-21T19:07:15.927 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.007205601 2023-07-21T19:07:15.927 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:15.928 INFO:teuthology.orchestra.run.smithi155.stdout: "req_renamesnap_latency": { 2023-07-21T19:07:15.928 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:15.928 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:15.928 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:15.928 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:15.928 INFO:teuthology.orchestra.run.smithi155.stdout: "req_snapdiff_latency": { 2023-07-21T19:07:15.928 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:15.929 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:15.929 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:15.929 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:15.929 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmdir_latency": { 2023-07-21T19:07:15.929 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 20, 2023-07-21T19:07:15.929 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.161531274, 2023-07-21T19:07:15.929 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.008076563 2023-07-21T19:07:15.929 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:15.930 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmsnap_latency": { 2023-07-21T19:07:15.930 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:15.930 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:15.930 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:15.930 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:15.931 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmxattr_latency": { 2023-07-21T19:07:15.931 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:15.931 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:15.931 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:15.931 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:15.931 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setattr_latency": { 2023-07-21T19:07:15.931 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 206, 2023-07-21T19:07:15.931 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 1.724979044, 2023-07-21T19:07:15.932 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.008373684 2023-07-21T19:07:15.932 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:15.932 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setdirlayout_latency": { 2023-07-21T19:07:15.932 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:15.932 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:15.932 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:15.932 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:15.933 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setfilelock_latency": { 2023-07-21T19:07:15.933 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 880, 2023-07-21T19:07:15.933 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 1.554985612, 2023-07-21T19:07:15.933 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.001767029 2023-07-21T19:07:15.933 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:15.933 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setlayout_latency": { 2023-07-21T19:07:15.933 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:15.933 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:15.934 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:15.934 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:15.934 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setxattr_latency": { 2023-07-21T19:07:15.934 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:15.934 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:15.934 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:15.934 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:15.935 INFO:teuthology.orchestra.run.smithi155.stdout: "req_symlink_latency": { 2023-07-21T19:07:15.935 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:15.935 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:15.935 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:15.935 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:15.935 INFO:teuthology.orchestra.run.smithi155.stdout: "req_unlink_latency": { 2023-07-21T19:07:15.935 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 32238, 2023-07-21T19:07:15.935 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 154.655572931, 2023-07-21T19:07:15.936 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.004797306 2023-07-21T19:07:15.936 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:15.936 INFO:teuthology.orchestra.run.smithi155.stdout: "cap_revoke_eviction": 0, 2023-07-21T19:07:15.936 INFO:teuthology.orchestra.run.smithi155.stdout: "cap_acquisition_throttle": 0, 2023-07-21T19:07:15.936 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getvxattr_latency": { 2023-07-21T19:07:15.937 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:15.937 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:15.937 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:15.938 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-07-21T19:07:15.938 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:15.938 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_sessions": { 2023-07-21T19:07:15.938 INFO:teuthology.orchestra.run.smithi155.stdout: "session_count": 2, 2023-07-21T19:07:15.939 INFO:teuthology.orchestra.run.smithi155.stdout: "session_add": 3, 2023-07-21T19:07:15.939 INFO:teuthology.orchestra.run.smithi155.stdout: "session_remove": 1, 2023-07-21T19:07:15.939 INFO:teuthology.orchestra.run.smithi155.stdout: "sessions_open": 2, 2023-07-21T19:07:15.940 INFO:teuthology.orchestra.run.smithi155.stdout: "sessions_stale": 0, 2023-07-21T19:07:15.940 INFO:teuthology.orchestra.run.smithi155.stdout: "total_load": 20374, 2023-07-21T19:07:15.940 INFO:teuthology.orchestra.run.smithi155.stdout: "average_load": 10187, 2023-07-21T19:07:15.941 INFO:teuthology.orchestra.run.smithi155.stdout: "avg_session_uptime": 1519 2023-07-21T19:07:15.941 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:15.941 INFO:teuthology.orchestra.run.smithi155.stdout: "mempool": { 2023-07-21T19:07:15.942 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_bytes": 0, 2023-07-21T19:07:15.942 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_items": 0, 2023-07-21T19:07:15.942 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_bytes": 0, 2023-07-21T19:07:15.943 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_items": 0, 2023-07-21T19:07:15.943 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_bytes": 0, 2023-07-21T19:07:15.943 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_items": 0, 2023-07-21T19:07:15.944 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-21T19:07:15.944 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_items": 0, 2023-07-21T19:07:15.944 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-21T19:07:15.944 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_items": 0, 2023-07-21T19:07:15.944 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_bytes": 0, 2023-07-21T19:07:15.945 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_items": 0, 2023-07-21T19:07:15.945 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-21T19:07:15.945 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_items": 0, 2023-07-21T19:07:15.946 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_bytes": 0, 2023-07-21T19:07:15.946 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_items": 0, 2023-07-21T19:07:15.946 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_bytes": 0, 2023-07-21T19:07:15.947 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_items": 0, 2023-07-21T19:07:15.947 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-21T19:07:15.947 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_items": 0, 2023-07-21T19:07:15.947 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-21T19:07:15.947 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_items": 0, 2023-07-21T19:07:15.948 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_bytes": 0, 2023-07-21T19:07:15.948 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_items": 0, 2023-07-21T19:07:15.948 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_bytes": 0, 2023-07-21T19:07:15.948 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_items": 0, 2023-07-21T19:07:15.948 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-21T19:07:15.948 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_items": 0, 2023-07-21T19:07:15.949 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_bytes": 0, 2023-07-21T19:07:15.949 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_items": 0, 2023-07-21T19:07:15.949 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_bytes": 0, 2023-07-21T19:07:15.949 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_items": 0, 2023-07-21T19:07:15.949 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_bytes": 0, 2023-07-21T19:07:15.949 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_items": 0, 2023-07-21T19:07:15.949 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_bytes": 0, 2023-07-21T19:07:15.949 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_items": 0, 2023-07-21T19:07:15.950 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_bytes": 502592, 2023-07-21T19:07:15.950 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_items": 780, 2023-07-21T19:07:15.950 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_bytes": 88, 2023-07-21T19:07:15.950 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_items": 1, 2023-07-21T19:07:15.950 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_bytes": 0, 2023-07-21T19:07:15.950 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_items": 0, 2023-07-21T19:07:15.950 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_bytes": 0, 2023-07-21T19:07:15.951 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_items": 0, 2023-07-21T19:07:15.951 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_bytes": 0, 2023-07-21T19:07:15.951 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_items": 0, 2023-07-21T19:07:15.951 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_bytes": 8280, 2023-07-21T19:07:15.951 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_items": 189, 2023-07-21T19:07:15.952 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_bytes": 0, 2023-07-21T19:07:15.952 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_items": 0, 2023-07-21T19:07:15.952 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_bytes": 0, 2023-07-21T19:07:15.952 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_items": 0, 2023-07-21T19:07:15.952 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_bytes": 5163032, 2023-07-21T19:07:15.952 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_items": 42811, 2023-07-21T19:07:15.953 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_bytes": 0, 2023-07-21T19:07:15.953 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_items": 0, 2023-07-21T19:07:15.953 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_bytes": 0, 2023-07-21T19:07:15.953 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_items": 0 2023-07-21T19:07:15.953 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:15.953 INFO:teuthology.orchestra.run.smithi155.stdout: "objecter": { 2023-07-21T19:07:15.953 INFO:teuthology.orchestra.run.smithi155.stdout: "op_active": 0, 2023-07-21T19:07:15.954 INFO:teuthology.orchestra.run.smithi155.stdout: "op_laggy": 0, 2023-07-21T19:07:15.954 INFO:teuthology.orchestra.run.smithi155.stdout: "op_send": 86422, 2023-07-21T19:07:15.954 INFO:teuthology.orchestra.run.smithi155.stdout: "op_send_bytes": 1007643667, 2023-07-21T19:07:15.954 INFO:teuthology.orchestra.run.smithi155.stdout: "op_resend": 33, 2023-07-21T19:07:15.954 INFO:teuthology.orchestra.run.smithi155.stdout: "op_reply": 86422, 2023-07-21T19:07:15.954 INFO:teuthology.orchestra.run.smithi155.stdout: "op_latency": { 2023-07-21T19:07:15.954 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 86422, 2023-07-21T19:07:15.955 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 783.622120081, 2023-07-21T19:07:15.955 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.009067391 2023-07-21T19:07:15.955 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:15.955 INFO:teuthology.orchestra.run.smithi155.stdout: "op_inflight": 0, 2023-07-21T19:07:15.955 INFO:teuthology.orchestra.run.smithi155.stdout: "oplen_avg": { 2023-07-21T19:07:15.955 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 86422, 2023-07-21T19:07:15.955 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 87195 2023-07-21T19:07:15.955 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:15.956 INFO:teuthology.orchestra.run.smithi155.stdout: "op": 86422, 2023-07-21T19:07:15.956 INFO:teuthology.orchestra.run.smithi155.stdout: "op_r": 1205, 2023-07-21T19:07:15.956 INFO:teuthology.orchestra.run.smithi155.stdout: "op_w": 85217, 2023-07-21T19:07:15.956 INFO:teuthology.orchestra.run.smithi155.stdout: "op_rmw": 0, 2023-07-21T19:07:15.956 INFO:teuthology.orchestra.run.smithi155.stdout: "op_pg": 0, 2023-07-21T19:07:15.956 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_stat": 33, 2023-07-21T19:07:15.957 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_create": 21, 2023-07-21T19:07:15.957 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_read": 1189, 2023-07-21T19:07:15.957 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_write": 51423, 2023-07-21T19:07:15.957 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_writefull": 731, 2023-07-21T19:07:15.957 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_writesame": 0, 2023-07-21T19:07:15.958 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_append": 0, 2023-07-21T19:07:15.958 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_zero": 0, 2023-07-21T19:07:15.958 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_truncate": 7, 2023-07-21T19:07:15.958 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_delete": 32642, 2023-07-21T19:07:15.958 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_mapext": 0, 2023-07-21T19:07:15.959 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_sparse_read": 0, 2023-07-21T19:07:15.959 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_clonerange": 0, 2023-07-21T19:07:15.959 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_getxattr": 7, 2023-07-21T19:07:15.959 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_setxattr": 42, 2023-07-21T19:07:15.959 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_cmpxattr": 0, 2023-07-21T19:07:15.959 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_rmxattr": 0, 2023-07-21T19:07:15.959 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_resetxattrs": 0, 2023-07-21T19:07:15.960 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_call": 0, 2023-07-21T19:07:15.960 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_watch": 0, 2023-07-21T19:07:15.960 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_notify": 0, 2023-07-21T19:07:15.961 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_src_cmpxattr": 0, 2023-07-21T19:07:15.961 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_pgls": 0, 2023-07-21T19:07:15.961 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_pgls_filter": 0, 2023-07-21T19:07:15.961 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_other": 0, 2023-07-21T19:07:15.962 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_active": 0, 2023-07-21T19:07:15.962 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_send": 0, 2023-07-21T19:07:15.962 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_resend": 0, 2023-07-21T19:07:15.962 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_ping": 0, 2023-07-21T19:07:15.962 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_active": 0, 2023-07-21T19:07:15.962 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_send": 0, 2023-07-21T19:07:15.963 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_resend": 0, 2023-07-21T19:07:15.963 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_active": 0, 2023-07-21T19:07:15.963 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_send": 0, 2023-07-21T19:07:15.963 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_resend": 0, 2023-07-21T19:07:15.963 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_active": 0, 2023-07-21T19:07:15.963 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_send": 0, 2023-07-21T19:07:15.964 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_resend": 0, 2023-07-21T19:07:15.964 INFO:teuthology.orchestra.run.smithi155.stdout: "command_active": 0, 2023-07-21T19:07:15.964 INFO:teuthology.orchestra.run.smithi155.stdout: "command_send": 0, 2023-07-21T19:07:15.964 INFO:teuthology.orchestra.run.smithi155.stdout: "command_resend": 0, 2023-07-21T19:07:15.964 INFO:teuthology.orchestra.run.smithi155.stdout: "map_epoch": 123, 2023-07-21T19:07:15.964 INFO:teuthology.orchestra.run.smithi155.stdout: "map_full": 0, 2023-07-21T19:07:15.965 INFO:teuthology.orchestra.run.smithi155.stdout: "map_inc": 44, 2023-07-21T19:07:15.965 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_sessions": 12, 2023-07-21T19:07:15.965 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_session_open": 12, 2023-07-21T19:07:15.965 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_session_close": 0, 2023-07-21T19:07:15.965 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_laggy": 0, 2023-07-21T19:07:15.965 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_wr": 760, 2023-07-21T19:07:15.965 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_rd": 25, 2023-07-21T19:07:15.966 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_del": 315 2023-07-21T19:07:15.966 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:15.966 INFO:teuthology.orchestra.run.smithi155.stdout: "oft": { 2023-07-21T19:07:15.966 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_objs": 1, 2023-07-21T19:07:15.966 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_kv_pairs": 0, 2023-07-21T19:07:15.967 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_updates": 0, 2023-07-21T19:07:15.967 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_removes": 0 2023-07-21T19:07:15.967 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:15.967 INFO:teuthology.orchestra.run.smithi155.stdout: "osd-slow-ops": { 2023-07-21T19:07:15.968 INFO:teuthology.orchestra.run.smithi155.stdout: "slow_ops_count": 281466 2023-07-21T19:07:15.968 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:15.968 INFO:teuthology.orchestra.run.smithi155.stdout: "purge_queue": { 2023-07-21T19:07:15.969 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_ops": 0, 2023-07-21T19:07:15.969 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_ops_high_water": 6, 2023-07-21T19:07:15.969 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing": 0, 2023-07-21T19:07:15.969 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_high_water": 6, 2023-07-21T19:07:15.970 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executed": 32238, 2023-07-21T19:07:15.970 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_item_in_journal": 0 2023-07-21T19:07:15.970 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:15.971 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-21T19:07:15.971 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2023-07-21T19:07:15.971 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 104857600, 2023-07-21T19:07:15.971 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2023-07-21T19:07:15.972 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 1368647, 2023-07-21T19:07:15.972 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 288763726, 2023-07-21T19:07:15.972 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2023-07-21T19:07:15.972 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 1368647, 2023-07-21T19:07:15.973 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2023-07-21T19:07:15.973 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2023-07-21T19:07:15.973 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 1368647, 2023-07-21T19:07:15.973 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 288763726, 2023-07-21T19:07:15.973 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2023-07-21T19:07:15.973 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:15.973 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:15.974 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:15.974 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-07-21T19:07:15.974 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:15.974 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-objecter_bytes": { 2023-07-21T19:07:15.974 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2023-07-21T19:07:15.975 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 104857600, 2023-07-21T19:07:15.975 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2023-07-21T19:07:15.975 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2023-07-21T19:07:15.975 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2023-07-21T19:07:15.975 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2023-07-21T19:07:15.976 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2023-07-21T19:07:15.976 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 86422, 2023-07-21T19:07:15.976 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 1010909904, 2023-07-21T19:07:15.976 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 86422, 2023-07-21T19:07:15.976 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 1010909904, 2023-07-21T19:07:15.977 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2023-07-21T19:07:15.977 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:15.977 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:15.977 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:15.977 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-07-21T19:07:15.978 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:15.978 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-objecter_ops": { 2023-07-21T19:07:15.978 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2023-07-21T19:07:15.978 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 1024, 2023-07-21T19:07:15.978 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2023-07-21T19:07:15.978 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2023-07-21T19:07:15.979 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2023-07-21T19:07:15.979 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2023-07-21T19:07:15.979 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2023-07-21T19:07:15.979 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 86422, 2023-07-21T19:07:15.979 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 86422, 2023-07-21T19:07:15.980 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 86422, 2023-07-21T19:07:15.980 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 86422, 2023-07-21T19:07:15.980 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2023-07-21T19:07:15.980 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:15.980 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:15.981 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:15.981 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-07-21T19:07:15.981 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:15.981 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-write_buf_throttle": { 2023-07-21T19:07:15.981 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2023-07-21T19:07:15.982 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 3758096384, 2023-07-21T19:07:15.982 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2023-07-21T19:07:15.982 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 32238, 2023-07-21T19:07:15.982 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 3266598, 2023-07-21T19:07:15.983 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2023-07-21T19:07:15.983 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 32238, 2023-07-21T19:07:15.983 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2023-07-21T19:07:15.983 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2023-07-21T19:07:15.983 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 1189, 2023-07-21T19:07:15.984 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 3266598, 2023-07-21T19:07:15.984 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2023-07-21T19:07:15.984 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:15.984 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:15.984 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:15.985 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-07-21T19:07:15.985 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:15.985 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-write_buf_throttle-0x5565a8c1a0a0": { 2023-07-21T19:07:15.985 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2023-07-21T19:07:15.985 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 3758096384, 2023-07-21T19:07:15.985 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2023-07-21T19:07:15.986 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 167601, 2023-07-21T19:07:15.986 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 1001125696, 2023-07-21T19:07:15.986 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2023-07-21T19:07:15.986 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 167601, 2023-07-21T19:07:15.986 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2023-07-21T19:07:15.986 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2023-07-21T19:07:15.986 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 50234, 2023-07-21T19:07:15.986 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 1001125696, 2023-07-21T19:07:15.987 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2023-07-21T19:07:15.987 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:15.987 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:15.987 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:15.987 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-07-21T19:07:15.987 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-07-21T19:07:15.987 INFO:teuthology.orchestra.run.smithi155.stdout:} 2023-07-21T19:07:16.348 INFO:tasks.check_counter:Daemon mds.e mds.exported=18 2023-07-21T19:07:16.349 INFO:tasks.check_counter:Daemon mds.e mds.imported=19 2023-07-21T19:07:16.349 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=966 2023-07-21T19:07:16.349 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=5563 2023-07-21T19:07:16.349 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=7 2023-07-21T19:07:16.350 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2023-07-21T19:07:16.350 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2023-07-21T19:07:16.350 DEBUG:tasks.check_counter:Getting stats from h 2023-07-21T19:07:16.350 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph daemon mds.h perf dump 2023-07-21T19:07:16.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:16 smithi155 ceph-mon[97934]: pgmap v809: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.4 MiB/s wr, 229 op/s 2023-07-21T19:07:16.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:16 smithi155 ceph-mon[97934]: osdmap e124: 12 total, 12 up, 12 in 2023-07-21T19:07:16.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:16 smithi193 ceph-mon[98204]: pgmap v809: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.4 MiB/s wr, 229 op/s 2023-07-21T19:07:16.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:16 smithi193 ceph-mon[98204]: osdmap e124: 12 total, 12 up, 12 in 2023-07-21T19:07:16.915 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.b/config 2023-07-21T19:07:16.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:16 smithi039 ceph-mon[96489]: pgmap v809: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.4 MiB/s wr, 229 op/s 2023-07-21T19:07:16.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:16 smithi039 ceph-mon[96489]: osdmap e124: 12 total, 12 up, 12 in 2023-07-21T19:07:18.232 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2023-07-21T19:07:18.232 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-0": { 2023-07-21T19:07:18.232 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 583922, 2023-07-21T19:07:18.232 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 515343, 2023-07-21T19:07:18.233 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 164002477, 2023-07-21T19:07:18.233 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 639280330, 2023-07-21T19:07:18.233 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 15, 2023-07-21T19:07:18.233 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 8, 2023-07-21T19:07:18.233 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 33.589031215, 2023-07-21T19:07:18.233 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 14.462247918, 2023-07-21T19:07:18.233 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 21.876978386, 2023-07-21T19:07:18.233 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 2.039112013, 2023-07-21T19:07:18.234 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:07:18.234 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 515342, 2023-07-21T19:07:18.234 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 59.328063517, 2023-07-21T19:07:18.234 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000115123 2023-07-21T19:07:18.234 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:18.234 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:07:18.234 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 996273, 2023-07-21T19:07:18.235 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.161850798, 2023-07-21T19:07:18.235 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000162 2023-07-21T19:07:18.235 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:18.235 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-07-21T19:07:18.235 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 448 2023-07-21T19:07:18.235 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:18.235 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-1": { 2023-07-21T19:07:18.235 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 236281, 2023-07-21T19:07:18.236 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 231255, 2023-07-21T19:07:18.236 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 80248234, 2023-07-21T19:07:18.236 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 414144150, 2023-07-21T19:07:18.236 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 14, 2023-07-21T19:07:18.236 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 5, 2023-07-21T19:07:18.236 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 13.495082890, 2023-07-21T19:07:18.236 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 5.157774136, 2023-07-21T19:07:18.237 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 8.229227425, 2023-07-21T19:07:18.237 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 1.512267428, 2023-07-21T19:07:18.237 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:07:18.237 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 231254, 2023-07-21T19:07:18.237 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 42.424830729, 2023-07-21T19:07:18.237 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000183455 2023-07-21T19:07:18.237 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:18.237 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:07:18.238 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 390201, 2023-07-21T19:07:18.238 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.069066057, 2023-07-21T19:07:18.238 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000177 2023-07-21T19:07:18.238 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:18.238 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 1944688, 2023-07-21T19:07:18.238 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 311296 2023-07-21T19:07:18.238 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:18.238 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-2": { 2023-07-21T19:07:18.239 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 615977, 2023-07-21T19:07:18.239 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 844563, 2023-07-21T19:07:18.239 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 186944985, 2023-07-21T19:07:18.239 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 867263180, 2023-07-21T19:07:18.239 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 16, 2023-07-21T19:07:18.239 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 6, 2023-07-21T19:07:18.239 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 33.454625530, 2023-07-21T19:07:18.239 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 16.224980698, 2023-07-21T19:07:18.240 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 22.154436349, 2023-07-21T19:07:18.240 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 1.864244573, 2023-07-21T19:07:18.240 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:07:18.240 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 844560, 2023-07-21T19:07:18.240 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 148.692001995, 2023-07-21T19:07:18.240 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000176058 2023-07-21T19:07:18.240 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:18.241 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:07:18.241 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 866905, 2023-07-21T19:07:18.241 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.196892259, 2023-07-21T19:07:18.241 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000227 2023-07-21T19:07:18.241 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:18.242 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 320, 2023-07-21T19:07:18.242 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 732240 2023-07-21T19:07:18.242 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:18.242 INFO:teuthology.orchestra.run.smithi155.stdout: "cct": { 2023-07-21T19:07:18.242 INFO:teuthology.orchestra.run.smithi155.stdout: "total_workers": 1, 2023-07-21T19:07:18.242 INFO:teuthology.orchestra.run.smithi155.stdout: "unhealthy_workers": 0 2023-07-21T19:07:18.243 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:18.243 INFO:teuthology.orchestra.run.smithi155.stdout: "finisher-MDSRank": { 2023-07-21T19:07:18.243 INFO:teuthology.orchestra.run.smithi155.stdout: "queue_len": 0, 2023-07-21T19:07:18.243 INFO:teuthology.orchestra.run.smithi155.stdout: "complete_latency": { 2023-07-21T19:07:18.243 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 112333, 2023-07-21T19:07:18.243 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 141.702362388, 2023-07-21T19:07:18.243 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.001261449 2023-07-21T19:07:18.243 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-07-21T19:07:18.244 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:18.244 INFO:teuthology.orchestra.run.smithi155.stdout: "finisher-PurgeQueue": { 2023-07-21T19:07:18.244 INFO:teuthology.orchestra.run.smithi155.stdout: "queue_len": 0, 2023-07-21T19:07:18.244 INFO:teuthology.orchestra.run.smithi155.stdout: "complete_latency": { 2023-07-21T19:07:18.244 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 64556, 2023-07-21T19:07:18.244 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 13.242372011, 2023-07-21T19:07:18.244 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000205129 2023-07-21T19:07:18.245 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-07-21T19:07:18.245 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:18.245 INFO:teuthology.orchestra.run.smithi155.stdout: "mds": { 2023-07-21T19:07:18.245 INFO:teuthology.orchestra.run.smithi155.stdout: "request": 272310, 2023-07-21T19:07:18.245 INFO:teuthology.orchestra.run.smithi155.stdout: "reply": 230046, 2023-07-21T19:07:18.245 INFO:teuthology.orchestra.run.smithi155.stdout: "reply_latency": { 2023-07-21T19:07:18.245 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 230046, 2023-07-21T19:07:18.245 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 1103.359581745, 2023-07-21T19:07:18.246 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.004796256 2023-07-21T19:07:18.246 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:18.246 INFO:teuthology.orchestra.run.smithi155.stdout: "slow_reply": 0, 2023-07-21T19:07:18.246 INFO:teuthology.orchestra.run.smithi155.stdout: "forward": 42262, 2023-07-21T19:07:18.246 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_fetch_complete": 11, 2023-07-21T19:07:18.246 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_fetch_keys": 0, 2023-07-21T19:07:18.246 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_commit": 107, 2023-07-21T19:07:18.246 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_split": 5, 2023-07-21T19:07:18.247 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_merge": 4, 2023-07-21T19:07:18.247 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes": 2756, 2023-07-21T19:07:18.247 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_top": 1434, 2023-07-21T19:07:18.247 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_bottom": 1253, 2023-07-21T19:07:18.247 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_pin_tail": 69, 2023-07-21T19:07:18.247 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_pinned": 703, 2023-07-21T19:07:18.247 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_expired": 1242, 2023-07-21T19:07:18.247 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_with_caps": 1, 2023-07-21T19:07:18.248 INFO:teuthology.orchestra.run.smithi155.stdout: "caps": 1, 2023-07-21T19:07:18.248 INFO:teuthology.orchestra.run.smithi155.stdout: "subtrees": 6, 2023-07-21T19:07:18.248 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse": 457950, 2023-07-21T19:07:18.248 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_hit": 242047, 2023-07-21T19:07:18.248 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_forward": 42232, 2023-07-21T19:07:18.248 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_discover": 274, 2023-07-21T19:07:18.248 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_dir_fetch": 5, 2023-07-21T19:07:18.248 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_remote_ino": 0, 2023-07-21T19:07:18.249 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_lock": 7554, 2023-07-21T19:07:18.249 INFO:teuthology.orchestra.run.smithi155.stdout: "load_cent": 0, 2023-07-21T19:07:18.249 INFO:teuthology.orchestra.run.smithi155.stdout: "q": 0, 2023-07-21T19:07:18.249 INFO:teuthology.orchestra.run.smithi155.stdout: "exported": 14, 2023-07-21T19:07:18.249 INFO:teuthology.orchestra.run.smithi155.stdout: "exported_inodes": 707, 2023-07-21T19:07:18.249 INFO:teuthology.orchestra.run.smithi155.stdout: "imported": 28, 2023-07-21T19:07:18.249 INFO:teuthology.orchestra.run.smithi155.stdout: "imported_inodes": 1535, 2023-07-21T19:07:18.249 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_dir_fetch": 0, 2023-07-21T19:07:18.250 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_backtrace_fetch": 14, 2023-07-21T19:07:18.250 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_peer_discover": 0, 2023-07-21T19:07:18.250 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rfiles": 131, 2023-07-21T19:07:18.250 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rbytes": 42134725, 2023-07-21T19:07:18.250 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rsnaps": 7, 2023-07-21T19:07:18.250 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_backtrace_fetch": 14, 2023-07-21T19:07:18.250 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_set_tag": 0, 2023-07-21T19:07:18.251 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_backtrace_repaired": 0, 2023-07-21T19:07:18.251 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_inotable_repaired": 0, 2023-07-21T19:07:18.251 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dir_inodes": 0, 2023-07-21T19:07:18.251 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dir_base_inodes": 0, 2023-07-21T19:07:18.251 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dirfrag_rstats": 0, 2023-07-21T19:07:18.251 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_file_inodes": 14, 2023-07-21T19:07:18.251 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_inode_file_caps": 49, 2023-07-21T19:07:18.251 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_revoke": 51008, 2023-07-21T19:07:18.252 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_grant": 158065, 2023-07-21T19:07:18.252 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_trunc": 0, 2023-07-21T19:07:18.252 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_flushsnap_ack": 1791, 2023-07-21T19:07:18.252 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-21T19:07:18.252 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_caps": 101668, 2023-07-21T19:07:18.252 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_caps_dirty": 34038, 2023-07-21T19:07:18.252 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_cap_release": 5549, 2023-07-21T19:07:18.252 INFO:teuthology.orchestra.run.smithi155.stdout: "process_request_cap_release": 183509 2023-07-21T19:07:18.253 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:18.253 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_cache": { 2023-07-21T19:07:18.253 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_update": 1007, 2023-07-21T19:07:18.253 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_update_receipt": 3849, 2023-07-21T19:07:18.253 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_try_discover": 2017, 2023-07-21T19:07:18.253 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_send_discover": 3782, 2023-07-21T19:07:18.253 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_handle_discover": 5853, 2023-07-21T19:07:18.253 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays": 30, 2023-07-21T19:07:18.254 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays_delayed": 0, 2023-07-21T19:07:18.254 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays_enqueuing": 0, 2023-07-21T19:07:18.254 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_created": 36690, 2023-07-21T19:07:18.254 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_enqueued": 36660, 2023-07-21T19:07:18.254 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_reintegrated": 0, 2023-07-21T19:07:18.254 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_migrated": 0, 2023-07-21T19:07:18.254 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_processing": 0, 2023-07-21T19:07:18.254 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_enqueued": 0, 2023-07-21T19:07:18.255 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_prioritized": 0, 2023-07-21T19:07:18.255 INFO:teuthology.orchestra.run.smithi155.stdout: "recovery_started": 0, 2023-07-21T19:07:18.255 INFO:teuthology.orchestra.run.smithi155.stdout: "recovery_completed": 0, 2023-07-21T19:07:18.255 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_enqueue_scrub": 0, 2023-07-21T19:07:18.255 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_exportdir": 18, 2023-07-21T19:07:18.255 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_flush": 0, 2023-07-21T19:07:18.255 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_fragmentdir": 9, 2023-07-21T19:07:18.255 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_fragstats": 0, 2023-07-21T19:07:18.256 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_inodestats": 0 2023-07-21T19:07:18.256 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:18.256 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_log": { 2023-07-21T19:07:18.256 INFO:teuthology.orchestra.run.smithi155.stdout: "evadd": 186843, 2023-07-21T19:07:18.256 INFO:teuthology.orchestra.run.smithi155.stdout: "evex": 142064, 2023-07-21T19:07:18.256 INFO:teuthology.orchestra.run.smithi155.stdout: "evtrm": 142064, 2023-07-21T19:07:18.256 INFO:teuthology.orchestra.run.smithi155.stdout: "ev": 44779, 2023-07-21T19:07:18.257 INFO:teuthology.orchestra.run.smithi155.stdout: "evexg": 0, 2023-07-21T19:07:18.257 INFO:teuthology.orchestra.run.smithi155.stdout: "evexd": 527, 2023-07-21T19:07:18.257 INFO:teuthology.orchestra.run.smithi155.stdout: "segadd": 344, 2023-07-21T19:07:18.257 INFO:teuthology.orchestra.run.smithi155.stdout: "segex": 216, 2023-07-21T19:07:18.257 INFO:teuthology.orchestra.run.smithi155.stdout: "segtrm": 216, 2023-07-21T19:07:18.257 INFO:teuthology.orchestra.run.smithi155.stdout: "seg": 129, 2023-07-21T19:07:18.257 INFO:teuthology.orchestra.run.smithi155.stdout: "segexg": 0, 2023-07-21T19:07:18.257 INFO:teuthology.orchestra.run.smithi155.stdout: "segexd": 1, 2023-07-21T19:07:18.258 INFO:teuthology.orchestra.run.smithi155.stdout: "expos": 738218953, 2023-07-21T19:07:18.258 INFO:teuthology.orchestra.run.smithi155.stdout: "wrpos": 1273660668, 2023-07-21T19:07:18.258 INFO:teuthology.orchestra.run.smithi155.stdout: "rdpos": 0, 2023-07-21T19:07:18.258 INFO:teuthology.orchestra.run.smithi155.stdout: "jlat": { 2023-07-21T19:07:18.258 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 62199, 2023-07-21T19:07:18.258 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 506.784591492, 2023-07-21T19:07:18.258 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.008147793 2023-07-21T19:07:18.258 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:18.259 INFO:teuthology.orchestra.run.smithi155.stdout: "replayed": 0 2023-07-21T19:07:18.259 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:18.259 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_mem": { 2023-07-21T19:07:18.259 INFO:teuthology.orchestra.run.smithi155.stdout: "ino": 1717, 2023-07-21T19:07:18.259 INFO:teuthology.orchestra.run.smithi155.stdout: "ino+": 39132, 2023-07-21T19:07:18.259 INFO:teuthology.orchestra.run.smithi155.stdout: "ino-": 37415, 2023-07-21T19:07:18.259 INFO:teuthology.orchestra.run.smithi155.stdout: "dir": 90, 2023-07-21T19:07:18.259 INFO:teuthology.orchestra.run.smithi155.stdout: "dir+": 2123, 2023-07-21T19:07:18.260 INFO:teuthology.orchestra.run.smithi155.stdout: "dir-": 2033, 2023-07-21T19:07:18.260 INFO:teuthology.orchestra.run.smithi155.stdout: "dn": 2777, 2023-07-21T19:07:18.260 INFO:teuthology.orchestra.run.smithi155.stdout: "dn+": 63097, 2023-07-21T19:07:18.260 INFO:teuthology.orchestra.run.smithi155.stdout: "dn-": 60320, 2023-07-21T19:07:18.260 INFO:teuthology.orchestra.run.smithi155.stdout: "cap": 1, 2023-07-21T19:07:18.260 INFO:teuthology.orchestra.run.smithi155.stdout: "cap+": 38981, 2023-07-21T19:07:18.260 INFO:teuthology.orchestra.run.smithi155.stdout: "cap-": 38980, 2023-07-21T19:07:18.261 INFO:teuthology.orchestra.run.smithi155.stdout: "rss": 87148, 2023-07-21T19:07:18.261 INFO:teuthology.orchestra.run.smithi155.stdout: "heap": 198964 2023-07-21T19:07:18.261 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:18.261 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_server": { 2023-07-21T19:07:18.261 INFO:teuthology.orchestra.run.smithi155.stdout: "dispatch_client_request": 399246, 2023-07-21T19:07:18.261 INFO:teuthology.orchestra.run.smithi155.stdout: "dispatch_server_request": 31016, 2023-07-21T19:07:18.261 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_request": 272310, 2023-07-21T19:07:18.261 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_session": 6488, 2023-07-21T19:07:18.262 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_peer_request": 80381, 2023-07-21T19:07:18.262 INFO:teuthology.orchestra.run.smithi155.stdout: "req_create_latency": { 2023-07-21T19:07:18.262 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 36495, 2023-07-21T19:07:18.262 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 230.750176501, 2023-07-21T19:07:18.262 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.006322788 2023-07-21T19:07:18.262 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:18.262 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getattr_latency": { 2023-07-21T19:07:18.263 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 11605, 2023-07-21T19:07:18.264 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 140.046987398, 2023-07-21T19:07:18.264 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.012067814 2023-07-21T19:07:18.264 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:18.264 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getfilelock_latency": { 2023-07-21T19:07:18.264 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:18.264 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:18.264 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:18.265 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:18.265 INFO:teuthology.orchestra.run.smithi155.stdout: "req_link_latency": { 2023-07-21T19:07:18.265 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:18.265 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:18.265 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:18.265 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:18.265 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookup_latency": { 2023-07-21T19:07:18.266 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 67863, 2023-07-21T19:07:18.266 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 200.998089108, 2023-07-21T19:07:18.266 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.002961821 2023-07-21T19:07:18.266 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:18.266 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookuphash_latency": { 2023-07-21T19:07:18.267 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:18.267 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:18.267 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:18.267 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:18.267 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupino_latency": { 2023-07-21T19:07:18.267 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:18.267 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:18.267 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:18.268 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:18.268 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupname_latency": { 2023-07-21T19:07:18.268 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:18.268 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:18.268 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:18.268 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:18.268 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupparent_latency": { 2023-07-21T19:07:18.269 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:18.269 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:18.269 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:18.269 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:18.269 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupsnap_latency": { 2023-07-21T19:07:18.269 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:18.269 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:18.269 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:18.270 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:18.270 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lssnap_latency": { 2023-07-21T19:07:18.270 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:18.270 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:18.270 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:18.270 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:18.270 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mkdir_latency": { 2023-07-21T19:07:18.271 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:18.271 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:18.271 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:18.271 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:18.271 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mknod_latency": { 2023-07-21T19:07:18.271 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:18.272 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:18.272 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:18.272 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:18.272 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mksnap_latency": { 2023-07-21T19:07:18.272 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:18.272 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:18.272 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:18.272 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:18.273 INFO:teuthology.orchestra.run.smithi155.stdout: "req_open_latency": { 2023-07-21T19:07:18.273 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:18.273 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:18.273 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:18.274 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:18.274 INFO:teuthology.orchestra.run.smithi155.stdout: "req_readdir_latency": { 2023-07-21T19:07:18.274 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 65550, 2023-07-21T19:07:18.274 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 320.892229351, 2023-07-21T19:07:18.274 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.004895381 2023-07-21T19:07:18.274 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:18.275 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rename_latency": { 2023-07-21T19:07:18.275 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 9762, 2023-07-21T19:07:18.275 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 62.784480953, 2023-07-21T19:07:18.275 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.006431518 2023-07-21T19:07:18.275 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:18.275 INFO:teuthology.orchestra.run.smithi155.stdout: "req_renamesnap_latency": { 2023-07-21T19:07:18.275 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:18.276 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:18.276 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:18.276 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:18.276 INFO:teuthology.orchestra.run.smithi155.stdout: "req_snapdiff_latency": { 2023-07-21T19:07:18.276 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:18.276 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:18.276 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:18.277 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:18.277 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmdir_latency": { 2023-07-21T19:07:18.277 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 30, 2023-07-21T19:07:18.277 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.259716977, 2023-07-21T19:07:18.277 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.008657232 2023-07-21T19:07:18.277 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:18.277 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmsnap_latency": { 2023-07-21T19:07:18.277 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:18.278 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:18.278 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:18.278 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:18.278 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmxattr_latency": { 2023-07-21T19:07:18.278 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:18.278 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:18.278 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:18.279 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:18.279 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setattr_latency": { 2023-07-21T19:07:18.279 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 93, 2023-07-21T19:07:18.279 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 1.582750230, 2023-07-21T19:07:18.279 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.017018819 2023-07-21T19:07:18.279 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:18.279 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setdirlayout_latency": { 2023-07-21T19:07:18.279 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:18.280 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:18.280 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:18.280 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:18.280 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setfilelock_latency": { 2023-07-21T19:07:18.280 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 1988, 2023-07-21T19:07:18.280 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 3.408311669, 2023-07-21T19:07:18.280 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.001714442 2023-07-21T19:07:18.280 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:18.281 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setlayout_latency": { 2023-07-21T19:07:18.281 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:18.281 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:18.281 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:18.281 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:18.281 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setxattr_latency": { 2023-07-21T19:07:18.281 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:18.282 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:18.282 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:18.282 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:18.282 INFO:teuthology.orchestra.run.smithi155.stdout: "req_symlink_latency": { 2023-07-21T19:07:18.282 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:18.282 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:18.282 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:18.282 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:18.283 INFO:teuthology.orchestra.run.smithi155.stdout: "req_unlink_latency": { 2023-07-21T19:07:18.283 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 36660, 2023-07-21T19:07:18.283 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 142.636839558, 2023-07-21T19:07:18.283 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.003890803 2023-07-21T19:07:18.283 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:18.283 INFO:teuthology.orchestra.run.smithi155.stdout: "cap_revoke_eviction": 0, 2023-07-21T19:07:18.283 INFO:teuthology.orchestra.run.smithi155.stdout: "cap_acquisition_throttle": 0, 2023-07-21T19:07:18.283 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getvxattr_latency": { 2023-07-21T19:07:18.284 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:18.284 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:18.284 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:18.284 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-07-21T19:07:18.284 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:18.285 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_sessions": { 2023-07-21T19:07:18.285 INFO:teuthology.orchestra.run.smithi155.stdout: "session_count": 2, 2023-07-21T19:07:18.285 INFO:teuthology.orchestra.run.smithi155.stdout: "session_add": 2, 2023-07-21T19:07:18.285 INFO:teuthology.orchestra.run.smithi155.stdout: "session_remove": 0, 2023-07-21T19:07:18.285 INFO:teuthology.orchestra.run.smithi155.stdout: "sessions_open": 2, 2023-07-21T19:07:18.285 INFO:teuthology.orchestra.run.smithi155.stdout: "sessions_stale": 0, 2023-07-21T19:07:18.285 INFO:teuthology.orchestra.run.smithi155.stdout: "total_load": 19558, 2023-07-21T19:07:18.286 INFO:teuthology.orchestra.run.smithi155.stdout: "average_load": 9779, 2023-07-21T19:07:18.286 INFO:teuthology.orchestra.run.smithi155.stdout: "avg_session_uptime": 1490 2023-07-21T19:07:18.286 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:18.286 INFO:teuthology.orchestra.run.smithi155.stdout: "mempool": { 2023-07-21T19:07:18.286 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_bytes": 0, 2023-07-21T19:07:18.286 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_items": 0, 2023-07-21T19:07:18.286 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_bytes": 0, 2023-07-21T19:07:18.286 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_items": 0, 2023-07-21T19:07:18.287 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_bytes": 0, 2023-07-21T19:07:18.287 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_items": 0, 2023-07-21T19:07:18.287 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-21T19:07:18.287 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_items": 0, 2023-07-21T19:07:18.287 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-21T19:07:18.287 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_items": 0, 2023-07-21T19:07:18.287 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_bytes": 0, 2023-07-21T19:07:18.287 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_items": 0, 2023-07-21T19:07:18.288 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-21T19:07:18.288 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_items": 0, 2023-07-21T19:07:18.288 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_bytes": 0, 2023-07-21T19:07:18.288 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_items": 0, 2023-07-21T19:07:18.288 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_bytes": 0, 2023-07-21T19:07:18.288 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_items": 0, 2023-07-21T19:07:18.288 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-21T19:07:18.289 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_items": 0, 2023-07-21T19:07:18.289 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-21T19:07:18.289 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_items": 0, 2023-07-21T19:07:18.289 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_bytes": 0, 2023-07-21T19:07:18.289 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_items": 0, 2023-07-21T19:07:18.289 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_bytes": 0, 2023-07-21T19:07:18.289 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_items": 0, 2023-07-21T19:07:18.289 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-21T19:07:18.290 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_items": 0, 2023-07-21T19:07:18.290 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_bytes": 0, 2023-07-21T19:07:18.290 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_items": 0, 2023-07-21T19:07:18.290 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_bytes": 0, 2023-07-21T19:07:18.290 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_items": 0, 2023-07-21T19:07:18.290 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_bytes": 0, 2023-07-21T19:07:18.290 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_items": 0, 2023-07-21T19:07:18.290 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_bytes": 0, 2023-07-21T19:07:18.291 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_items": 0, 2023-07-21T19:07:18.291 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_bytes": 607281, 2023-07-21T19:07:18.291 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_items": 1249, 2023-07-21T19:07:18.291 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_bytes": 176, 2023-07-21T19:07:18.291 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_items": 2, 2023-07-21T19:07:18.291 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_bytes": 0, 2023-07-21T19:07:18.291 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_items": 0, 2023-07-21T19:07:18.291 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_bytes": 0, 2023-07-21T19:07:18.292 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_items": 0, 2023-07-21T19:07:18.292 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_bytes": 0, 2023-07-21T19:07:18.292 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_items": 0, 2023-07-21T19:07:18.292 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_bytes": 8280, 2023-07-21T19:07:18.292 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_items": 189, 2023-07-21T19:07:18.292 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_bytes": 0, 2023-07-21T19:07:18.292 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_items": 0, 2023-07-21T19:07:18.293 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_bytes": 0, 2023-07-21T19:07:18.293 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_items": 0, 2023-07-21T19:07:18.293 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_bytes": 6483877, 2023-07-21T19:07:18.293 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_items": 57131, 2023-07-21T19:07:18.293 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_bytes": 0, 2023-07-21T19:07:18.293 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_items": 0, 2023-07-21T19:07:18.293 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_bytes": 0, 2023-07-21T19:07:18.293 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_items": 0 2023-07-21T19:07:18.294 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:18.294 INFO:teuthology.orchestra.run.smithi155.stdout: "objecter": { 2023-07-21T19:07:18.294 INFO:teuthology.orchestra.run.smithi155.stdout: "op_active": 0, 2023-07-21T19:07:18.294 INFO:teuthology.orchestra.run.smithi155.stdout: "op_laggy": 0, 2023-07-21T19:07:18.294 INFO:teuthology.orchestra.run.smithi155.stdout: "op_send": 103183, 2023-07-21T19:07:18.294 INFO:teuthology.orchestra.run.smithi155.stdout: "op_send_bytes": 1276861362, 2023-07-21T19:07:18.294 INFO:teuthology.orchestra.run.smithi155.stdout: "op_resend": 27, 2023-07-21T19:07:18.294 INFO:teuthology.orchestra.run.smithi155.stdout: "op_reply": 103183, 2023-07-21T19:07:18.295 INFO:teuthology.orchestra.run.smithi155.stdout: "op_latency": { 2023-07-21T19:07:18.295 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 103183, 2023-07-21T19:07:18.295 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 856.025725152, 2023-07-21T19:07:18.295 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.008296189 2023-07-21T19:07:18.295 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:18.295 INFO:teuthology.orchestra.run.smithi155.stdout: "op_inflight": 0, 2023-07-21T19:07:18.295 INFO:teuthology.orchestra.run.smithi155.stdout: "oplen_avg": { 2023-07-21T19:07:18.295 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 103183, 2023-07-21T19:07:18.296 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 104017 2023-07-21T19:07:18.296 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:18.296 INFO:teuthology.orchestra.run.smithi155.stdout: "op": 103183, 2023-07-21T19:07:18.296 INFO:teuthology.orchestra.run.smithi155.stdout: "op_r": 1317, 2023-07-21T19:07:18.296 INFO:teuthology.orchestra.run.smithi155.stdout: "op_w": 101866, 2023-07-21T19:07:18.296 INFO:teuthology.orchestra.run.smithi155.stdout: "op_rmw": 0, 2023-07-21T19:07:18.296 INFO:teuthology.orchestra.run.smithi155.stdout: "op_pg": 0, 2023-07-21T19:07:18.297 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_stat": 52, 2023-07-21T19:07:18.297 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_create": 21, 2023-07-21T19:07:18.297 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_read": 1290, 2023-07-21T19:07:18.297 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_write": 63489, 2023-07-21T19:07:18.297 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_writefull": 768, 2023-07-21T19:07:18.297 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_writesame": 0, 2023-07-21T19:07:18.297 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_append": 0, 2023-07-21T19:07:18.297 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_zero": 0, 2023-07-21T19:07:18.298 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_truncate": 5, 2023-07-21T19:07:18.298 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_delete": 37181, 2023-07-21T19:07:18.298 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_mapext": 0, 2023-07-21T19:07:18.298 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_sparse_read": 0, 2023-07-21T19:07:18.298 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_clonerange": 0, 2023-07-21T19:07:18.298 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_getxattr": 25, 2023-07-21T19:07:18.298 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_setxattr": 42, 2023-07-21T19:07:18.298 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_cmpxattr": 0, 2023-07-21T19:07:18.299 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_rmxattr": 0, 2023-07-21T19:07:18.299 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_resetxattrs": 0, 2023-07-21T19:07:18.299 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_call": 0, 2023-07-21T19:07:18.299 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_watch": 0, 2023-07-21T19:07:18.299 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_notify": 0, 2023-07-21T19:07:18.299 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_src_cmpxattr": 0, 2023-07-21T19:07:18.299 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_pgls": 0, 2023-07-21T19:07:18.299 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_pgls_filter": 0, 2023-07-21T19:07:18.300 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_other": 0, 2023-07-21T19:07:18.300 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_active": 0, 2023-07-21T19:07:18.300 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_send": 0, 2023-07-21T19:07:18.300 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_resend": 0, 2023-07-21T19:07:18.300 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_ping": 0, 2023-07-21T19:07:18.300 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_active": 0, 2023-07-21T19:07:18.300 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_send": 0, 2023-07-21T19:07:18.301 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_resend": 0, 2023-07-21T19:07:18.301 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_active": 0, 2023-07-21T19:07:18.301 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_send": 0, 2023-07-21T19:07:18.301 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_resend": 0, 2023-07-21T19:07:18.301 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_active": 0, 2023-07-21T19:07:18.301 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_send": 0, 2023-07-21T19:07:18.301 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_resend": 0, 2023-07-21T19:07:18.301 INFO:teuthology.orchestra.run.smithi155.stdout: "command_active": 0, 2023-07-21T19:07:18.302 INFO:teuthology.orchestra.run.smithi155.stdout: "command_send": 0, 2023-07-21T19:07:18.302 INFO:teuthology.orchestra.run.smithi155.stdout: "command_resend": 0, 2023-07-21T19:07:18.302 INFO:teuthology.orchestra.run.smithi155.stdout: "map_epoch": 124, 2023-07-21T19:07:18.302 INFO:teuthology.orchestra.run.smithi155.stdout: "map_full": 0, 2023-07-21T19:07:18.302 INFO:teuthology.orchestra.run.smithi155.stdout: "map_inc": 45, 2023-07-21T19:07:18.302 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_sessions": 12, 2023-07-21T19:07:18.302 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_session_open": 12, 2023-07-21T19:07:18.302 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_session_close": 0, 2023-07-21T19:07:18.303 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_laggy": 0, 2023-07-21T19:07:18.303 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_wr": 779, 2023-07-21T19:07:18.303 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_rd": 24, 2023-07-21T19:07:18.303 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_del": 341 2023-07-21T19:07:18.303 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:18.303 INFO:teuthology.orchestra.run.smithi155.stdout: "oft": { 2023-07-21T19:07:18.303 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_objs": 1, 2023-07-21T19:07:18.304 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_kv_pairs": 0, 2023-07-21T19:07:18.304 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_updates": 0, 2023-07-21T19:07:18.304 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_removes": 0 2023-07-21T19:07:18.304 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:18.304 INFO:teuthology.orchestra.run.smithi155.stdout: "osd-slow-ops": { 2023-07-21T19:07:18.304 INFO:teuthology.orchestra.run.smithi155.stdout: "slow_ops_count": 303322 2023-07-21T19:07:18.304 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:18.305 INFO:teuthology.orchestra.run.smithi155.stdout: "purge_queue": { 2023-07-21T19:07:18.305 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_ops": 0, 2023-07-21T19:07:18.305 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_ops_high_water": 6, 2023-07-21T19:07:18.305 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing": 0, 2023-07-21T19:07:18.305 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_high_water": 6, 2023-07-21T19:07:18.305 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executed": 36660, 2023-07-21T19:07:18.305 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_item_in_journal": 0 2023-07-21T19:07:18.305 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:18.306 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-21T19:07:18.306 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2023-07-21T19:07:18.306 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 104857600, 2023-07-21T19:07:18.306 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2023-07-21T19:07:18.306 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 1436176, 2023-07-21T19:07:18.306 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 308280448, 2023-07-21T19:07:18.306 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2023-07-21T19:07:18.307 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 1436176, 2023-07-21T19:07:18.307 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2023-07-21T19:07:18.307 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2023-07-21T19:07:18.307 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 1436176, 2023-07-21T19:07:18.307 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 308280448, 2023-07-21T19:07:18.307 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2023-07-21T19:07:18.307 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:18.308 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:18.308 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:18.308 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-07-21T19:07:18.308 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:18.308 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-objecter_bytes": { 2023-07-21T19:07:18.308 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2023-07-21T19:07:18.308 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 104857600, 2023-07-21T19:07:18.309 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2023-07-21T19:07:18.309 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2023-07-21T19:07:18.309 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2023-07-21T19:07:18.309 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2023-07-21T19:07:18.309 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2023-07-21T19:07:18.309 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 103183, 2023-07-21T19:07:18.309 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 1280574399, 2023-07-21T19:07:18.310 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 103183, 2023-07-21T19:07:18.310 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 1280574399, 2023-07-21T19:07:18.310 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2023-07-21T19:07:18.310 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:18.310 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:18.310 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:18.310 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-07-21T19:07:18.310 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:18.311 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-objecter_ops": { 2023-07-21T19:07:18.311 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2023-07-21T19:07:18.311 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 1024, 2023-07-21T19:07:18.311 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2023-07-21T19:07:18.311 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2023-07-21T19:07:18.311 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2023-07-21T19:07:18.311 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2023-07-21T19:07:18.311 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2023-07-21T19:07:18.312 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 103183, 2023-07-21T19:07:18.312 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 103183, 2023-07-21T19:07:18.312 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 103183, 2023-07-21T19:07:18.312 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 103183, 2023-07-21T19:07:18.312 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2023-07-21T19:07:18.312 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:18.312 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:18.312 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:18.313 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-07-21T19:07:18.313 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:18.313 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-write_buf_throttle": { 2023-07-21T19:07:18.313 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2023-07-21T19:07:18.313 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 3758096384, 2023-07-21T19:07:18.314 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2023-07-21T19:07:18.314 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 36660, 2023-07-21T19:07:18.314 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 3713276, 2023-07-21T19:07:18.314 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2023-07-21T19:07:18.314 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 36660, 2023-07-21T19:07:18.314 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2023-07-21T19:07:18.314 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2023-07-21T19:07:18.315 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 1290, 2023-07-21T19:07:18.315 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 3713276, 2023-07-21T19:07:18.315 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2023-07-21T19:07:18.315 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:18.315 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:18.315 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:18.315 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-07-21T19:07:18.315 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:18.316 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-write_buf_throttle-0x562377b400a0": { 2023-07-21T19:07:18.316 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2023-07-21T19:07:18.316 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 3758096384, 2023-07-21T19:07:18.316 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2023-07-21T19:07:18.316 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 186843, 2023-07-21T19:07:18.316 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 1269466364, 2023-07-21T19:07:18.316 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2023-07-21T19:07:18.317 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 186843, 2023-07-21T19:07:18.317 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2023-07-21T19:07:18.317 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2023-07-21T19:07:18.317 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 62199, 2023-07-21T19:07:18.317 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 1269466364, 2023-07-21T19:07:18.317 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2023-07-21T19:07:18.317 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:18.318 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:18.318 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:18.318 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-07-21T19:07:18.318 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-07-21T19:07:18.318 INFO:teuthology.orchestra.run.smithi155.stdout:} 2023-07-21T19:07:18.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:18 smithi155 ceph-mon[97934]: pgmap v811: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 236 KiB/s wr, 32 op/s 2023-07-21T19:07:18.662 INFO:tasks.check_counter:Daemon mds.h mds.exported=14 2023-07-21T19:07:18.663 INFO:tasks.check_counter:Daemon mds.h mds.imported=28 2023-07-21T19:07:18.663 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=1007 2023-07-21T19:07:18.663 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=3849 2023-07-21T19:07:18.663 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=7 2023-07-21T19:07:18.663 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2023-07-21T19:07:18.663 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2023-07-21T19:07:18.664 DEBUG:tasks.check_counter:Getting stats from k 2023-07-21T19:07:18.664 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph daemon mds.k perf dump 2023-07-21T19:07:18.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:18 smithi193 ceph-mon[98204]: pgmap v811: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 236 KiB/s wr, 32 op/s 2023-07-21T19:07:18.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:18 smithi039 ceph-mon[96489]: pgmap v811: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 236 KiB/s wr, 32 op/s 2023-07-21T19:07:19.220 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.b/config 2023-07-21T19:07:20.433 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2023-07-21T19:07:20.433 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-0": { 2023-07-21T19:07:20.434 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 1582, 2023-07-21T19:07:20.434 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 1582, 2023-07-21T19:07:20.434 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 297673355, 2023-07-21T19:07:20.434 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 491270, 2023-07-21T19:07:20.434 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 8, 2023-07-21T19:07:20.434 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 4, 2023-07-21T19:07:20.435 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.304759257, 2023-07-21T19:07:20.435 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.034275654, 2023-07-21T19:07:20.435 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.212726923, 2023-07-21T19:07:20.435 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.045464789, 2023-07-21T19:07:20.435 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:07:20.435 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 1581, 2023-07-21T19:07:20.436 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.171384544, 2023-07-21T19:07:20.436 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000108402 2023-07-21T19:07:20.436 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:20.436 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:07:20.437 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:20.437 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:20.437 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:20.437 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:20.438 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-07-21T19:07:20.438 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 448 2023-07-21T19:07:20.438 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:20.439 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-1": { 2023-07-21T19:07:20.439 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 1994, 2023-07-21T19:07:20.439 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 2329, 2023-07-21T19:07:20.439 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 361222777, 2023-07-21T19:07:20.440 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 1352237, 2023-07-21T19:07:20.440 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 8, 2023-07-21T19:07:20.440 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 6, 2023-07-21T19:07:20.440 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.419287351, 2023-07-21T19:07:20.441 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.046749056, 2023-07-21T19:07:20.441 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.298229492, 2023-07-21T19:07:20.441 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.057072271, 2023-07-21T19:07:20.442 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:07:20.442 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 2326, 2023-07-21T19:07:20.442 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.191311145, 2023-07-21T19:07:20.442 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000082248 2023-07-21T19:07:20.443 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:20.443 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:07:20.443 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:20.444 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:20.444 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:20.444 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:20.444 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 320, 2023-07-21T19:07:20.445 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 732720 2023-07-21T19:07:20.445 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:20.445 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-2": { 2023-07-21T19:07:20.445 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 3373, 2023-07-21T19:07:20.445 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 3277, 2023-07-21T19:07:20.446 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 254430164, 2023-07-21T19:07:20.446 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 1128110, 2023-07-21T19:07:20.446 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 5, 2023-07-21T19:07:20.446 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 4, 2023-07-21T19:07:20.447 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.439926236, 2023-07-21T19:07:20.447 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.086214673, 2023-07-21T19:07:20.447 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.240632256, 2023-07-21T19:07:20.448 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.100780915, 2023-07-21T19:07:20.448 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:07:20.448 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 3276, 2023-07-21T19:07:20.448 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.334629146, 2023-07-21T19:07:20.448 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000102145 2023-07-21T19:07:20.449 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:20.449 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:07:20.449 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:20.449 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:20.449 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:20.449 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:20.450 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 1925312, 2023-07-21T19:07:20.450 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 261664 2023-07-21T19:07:20.450 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:20.450 INFO:teuthology.orchestra.run.smithi155.stdout: "cct": { 2023-07-21T19:07:20.450 INFO:teuthology.orchestra.run.smithi155.stdout: "total_workers": 1, 2023-07-21T19:07:20.451 INFO:teuthology.orchestra.run.smithi155.stdout: "unhealthy_workers": 0 2023-07-21T19:07:20.451 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:20.451 INFO:teuthology.orchestra.run.smithi155.stdout: "finisher-MDSRank": { 2023-07-21T19:07:20.451 INFO:teuthology.orchestra.run.smithi155.stdout: "queue_len": 0, 2023-07-21T19:07:20.451 INFO:teuthology.orchestra.run.smithi155.stdout: "complete_latency": { 2023-07-21T19:07:20.452 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 10549, 2023-07-21T19:07:20.452 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.473160871, 2023-07-21T19:07:20.452 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000044853 2023-07-21T19:07:20.452 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-07-21T19:07:20.452 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:20.452 INFO:teuthology.orchestra.run.smithi155.stdout: "finisher-PurgeQueue": { 2023-07-21T19:07:20.453 INFO:teuthology.orchestra.run.smithi155.stdout: "queue_len": 0, 2023-07-21T19:07:20.453 INFO:teuthology.orchestra.run.smithi155.stdout: "complete_latency": { 2023-07-21T19:07:20.453 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:20.453 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:20.453 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:20.453 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-07-21T19:07:20.453 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:20.454 INFO:teuthology.orchestra.run.smithi155.stdout: "mds": { 2023-07-21T19:07:20.454 INFO:teuthology.orchestra.run.smithi155.stdout: "request": 0, 2023-07-21T19:07:20.454 INFO:teuthology.orchestra.run.smithi155.stdout: "reply": 0, 2023-07-21T19:07:20.454 INFO:teuthology.orchestra.run.smithi155.stdout: "reply_latency": { 2023-07-21T19:07:20.454 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:20.454 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:20.454 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:20.455 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:20.455 INFO:teuthology.orchestra.run.smithi155.stdout: "slow_reply": 0, 2023-07-21T19:07:20.455 INFO:teuthology.orchestra.run.smithi155.stdout: "forward": 0, 2023-07-21T19:07:20.456 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_fetch_complete": 0, 2023-07-21T19:07:20.456 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_fetch_keys": 0, 2023-07-21T19:07:20.456 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_commit": 0, 2023-07-21T19:07:20.456 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_split": 0, 2023-07-21T19:07:20.457 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_merge": 0, 2023-07-21T19:07:20.457 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes": 18222, 2023-07-21T19:07:20.457 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_top": 4245, 2023-07-21T19:07:20.457 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_bottom": 13967, 2023-07-21T19:07:20.458 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_pin_tail": 10, 2023-07-21T19:07:20.458 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_pinned": 12157, 2023-07-21T19:07:20.458 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_expired": 13605, 2023-07-21T19:07:20.459 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_with_caps": 0, 2023-07-21T19:07:20.459 INFO:teuthology.orchestra.run.smithi155.stdout: "caps": 0, 2023-07-21T19:07:20.459 INFO:teuthology.orchestra.run.smithi155.stdout: "subtrees": 1, 2023-07-21T19:07:20.459 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse": 0, 2023-07-21T19:07:20.460 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_hit": 0, 2023-07-21T19:07:20.460 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_forward": 0, 2023-07-21T19:07:20.460 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_discover": 0, 2023-07-21T19:07:20.460 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_dir_fetch": 0, 2023-07-21T19:07:20.461 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_remote_ino": 0, 2023-07-21T19:07:20.461 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_lock": 0, 2023-07-21T19:07:20.461 INFO:teuthology.orchestra.run.smithi155.stdout: "load_cent": 0, 2023-07-21T19:07:20.461 INFO:teuthology.orchestra.run.smithi155.stdout: "q": 0, 2023-07-21T19:07:20.462 INFO:teuthology.orchestra.run.smithi155.stdout: "exported": 0, 2023-07-21T19:07:20.462 INFO:teuthology.orchestra.run.smithi155.stdout: "exported_inodes": 0, 2023-07-21T19:07:20.462 INFO:teuthology.orchestra.run.smithi155.stdout: "imported": 0, 2023-07-21T19:07:20.462 INFO:teuthology.orchestra.run.smithi155.stdout: "imported_inodes": 0, 2023-07-21T19:07:20.462 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_dir_fetch": 0, 2023-07-21T19:07:20.462 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_backtrace_fetch": 0, 2023-07-21T19:07:20.463 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_peer_discover": 0, 2023-07-21T19:07:20.463 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rfiles": 0, 2023-07-21T19:07:20.463 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rbytes": 0, 2023-07-21T19:07:20.463 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rsnaps": 0, 2023-07-21T19:07:20.463 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_backtrace_fetch": 0, 2023-07-21T19:07:20.463 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_set_tag": 0, 2023-07-21T19:07:20.463 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_backtrace_repaired": 0, 2023-07-21T19:07:20.463 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_inotable_repaired": 0, 2023-07-21T19:07:20.464 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dir_inodes": 0, 2023-07-21T19:07:20.464 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dir_base_inodes": 0, 2023-07-21T19:07:20.464 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dirfrag_rstats": 0, 2023-07-21T19:07:20.464 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_file_inodes": 0, 2023-07-21T19:07:20.464 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_inode_file_caps": 0, 2023-07-21T19:07:20.464 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_revoke": 0, 2023-07-21T19:07:20.464 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_grant": 0, 2023-07-21T19:07:20.465 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_trunc": 0, 2023-07-21T19:07:20.465 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-21T19:07:20.465 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-21T19:07:20.465 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_caps": 0, 2023-07-21T19:07:20.465 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_caps_dirty": 0, 2023-07-21T19:07:20.465 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_cap_release": 0, 2023-07-21T19:07:20.465 INFO:teuthology.orchestra.run.smithi155.stdout: "process_request_cap_release": 0 2023-07-21T19:07:20.465 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:20.466 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_cache": { 2023-07-21T19:07:20.466 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_update": 0, 2023-07-21T19:07:20.466 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_update_receipt": 0, 2023-07-21T19:07:20.466 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_try_discover": 0, 2023-07-21T19:07:20.466 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_send_discover": 0, 2023-07-21T19:07:20.466 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_handle_discover": 0, 2023-07-21T19:07:20.466 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays": 0, 2023-07-21T19:07:20.466 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays_delayed": 0, 2023-07-21T19:07:20.467 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays_enqueuing": 0, 2023-07-21T19:07:20.467 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_created": 0, 2023-07-21T19:07:20.467 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_enqueued": 0, 2023-07-21T19:07:20.467 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_reintegrated": 0, 2023-07-21T19:07:20.467 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_migrated": 0, 2023-07-21T19:07:20.467 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_processing": 0, 2023-07-21T19:07:20.467 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_enqueued": 0, 2023-07-21T19:07:20.468 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_prioritized": 0, 2023-07-21T19:07:20.468 INFO:teuthology.orchestra.run.smithi155.stdout: "recovery_started": 0, 2023-07-21T19:07:20.468 INFO:teuthology.orchestra.run.smithi155.stdout: "recovery_completed": 0, 2023-07-21T19:07:20.469 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_enqueue_scrub": 0, 2023-07-21T19:07:20.469 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_exportdir": 0, 2023-07-21T19:07:20.469 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_flush": 0, 2023-07-21T19:07:20.469 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_fragmentdir": 0, 2023-07-21T19:07:20.470 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_fragstats": 0, 2023-07-21T19:07:20.470 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_inodestats": 0 2023-07-21T19:07:20.470 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:20.470 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_log": { 2023-07-21T19:07:20.471 INFO:teuthology.orchestra.run.smithi155.stdout: "evadd": 0, 2023-07-21T19:07:20.471 INFO:teuthology.orchestra.run.smithi155.stdout: "evex": 0, 2023-07-21T19:07:20.471 INFO:teuthology.orchestra.run.smithi155.stdout: "evtrm": 0, 2023-07-21T19:07:20.471 INFO:teuthology.orchestra.run.smithi155.stdout: "ev": 0, 2023-07-21T19:07:20.472 INFO:teuthology.orchestra.run.smithi155.stdout: "evexg": 0, 2023-07-21T19:07:20.472 INFO:teuthology.orchestra.run.smithi155.stdout: "evexd": 0, 2023-07-21T19:07:20.472 INFO:teuthology.orchestra.run.smithi155.stdout: "segadd": 0, 2023-07-21T19:07:20.472 INFO:teuthology.orchestra.run.smithi155.stdout: "segex": 0, 2023-07-21T19:07:20.473 INFO:teuthology.orchestra.run.smithi155.stdout: "segtrm": 0, 2023-07-21T19:07:20.473 INFO:teuthology.orchestra.run.smithi155.stdout: "seg": 128, 2023-07-21T19:07:20.473 INFO:teuthology.orchestra.run.smithi155.stdout: "segexg": 0, 2023-07-21T19:07:20.474 INFO:teuthology.orchestra.run.smithi155.stdout: "segexd": 0, 2023-07-21T19:07:20.474 INFO:teuthology.orchestra.run.smithi155.stdout: "expos": 377498254, 2023-07-21T19:07:20.474 INFO:teuthology.orchestra.run.smithi155.stdout: "wrpos": 0, 2023-07-21T19:07:20.475 INFO:teuthology.orchestra.run.smithi155.stdout: "rdpos": 913753181, 2023-07-21T19:07:20.475 INFO:teuthology.orchestra.run.smithi155.stdout: "jlat": { 2023-07-21T19:07:20.475 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:20.475 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:20.476 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:20.476 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:20.476 INFO:teuthology.orchestra.run.smithi155.stdout: "replayed": 148303 2023-07-21T19:07:20.477 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:20.477 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_mem": { 2023-07-21T19:07:20.477 INFO:teuthology.orchestra.run.smithi155.stdout: "ino": 1250, 2023-07-21T19:07:20.477 INFO:teuthology.orchestra.run.smithi155.stdout: "ino+": 31707, 2023-07-21T19:07:20.477 INFO:teuthology.orchestra.run.smithi155.stdout: "ino-": 30457, 2023-07-21T19:07:20.478 INFO:teuthology.orchestra.run.smithi155.stdout: "dir": 52, 2023-07-21T19:07:20.478 INFO:teuthology.orchestra.run.smithi155.stdout: "dir+": 110, 2023-07-21T19:07:20.478 INFO:teuthology.orchestra.run.smithi155.stdout: "dir-": 58, 2023-07-21T19:07:20.478 INFO:teuthology.orchestra.run.smithi155.stdout: "dn": 18222, 2023-07-21T19:07:20.478 INFO:teuthology.orchestra.run.smithi155.stdout: "dn+": 32102, 2023-07-21T19:07:20.479 INFO:teuthology.orchestra.run.smithi155.stdout: "dn-": 13880, 2023-07-21T19:07:20.479 INFO:teuthology.orchestra.run.smithi155.stdout: "cap": 0, 2023-07-21T19:07:20.479 INFO:teuthology.orchestra.run.smithi155.stdout: "cap+": 0, 2023-07-21T19:07:20.480 INFO:teuthology.orchestra.run.smithi155.stdout: "cap-": 0, 2023-07-21T19:07:20.480 INFO:teuthology.orchestra.run.smithi155.stdout: "rss": 76056, 2023-07-21T19:07:20.480 INFO:teuthology.orchestra.run.smithi155.stdout: "heap": 207156 2023-07-21T19:07:20.480 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:20.480 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_server": { 2023-07-21T19:07:20.480 INFO:teuthology.orchestra.run.smithi155.stdout: "dispatch_client_request": 0, 2023-07-21T19:07:20.481 INFO:teuthology.orchestra.run.smithi155.stdout: "dispatch_server_request": 0, 2023-07-21T19:07:20.481 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_request": 0, 2023-07-21T19:07:20.481 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_session": 0, 2023-07-21T19:07:20.481 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_peer_request": 0, 2023-07-21T19:07:20.481 INFO:teuthology.orchestra.run.smithi155.stdout: "req_create_latency": { 2023-07-21T19:07:20.481 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:20.481 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:20.482 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:20.482 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:20.482 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getattr_latency": { 2023-07-21T19:07:20.482 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:20.482 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:20.482 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:20.482 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:20.482 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getfilelock_latency": { 2023-07-21T19:07:20.483 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:20.483 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:20.483 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:20.483 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:20.483 INFO:teuthology.orchestra.run.smithi155.stdout: "req_link_latency": { 2023-07-21T19:07:20.483 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:20.484 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:20.484 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:20.484 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:20.484 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookup_latency": { 2023-07-21T19:07:20.484 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:20.484 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:20.484 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:20.485 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:20.485 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookuphash_latency": { 2023-07-21T19:07:20.485 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:20.485 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:20.485 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:20.485 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:20.485 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupino_latency": { 2023-07-21T19:07:20.485 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:20.486 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:20.486 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:20.486 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:20.486 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupname_latency": { 2023-07-21T19:07:20.486 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:20.486 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:20.486 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:20.487 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:20.487 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupparent_latency": { 2023-07-21T19:07:20.487 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:20.487 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:20.487 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:20.487 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:20.487 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupsnap_latency": { 2023-07-21T19:07:20.487 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:20.488 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:20.488 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:20.488 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:20.488 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lssnap_latency": { 2023-07-21T19:07:20.488 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:20.489 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:20.489 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:20.489 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:20.489 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mkdir_latency": { 2023-07-21T19:07:20.490 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:20.490 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:20.490 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:20.491 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:20.491 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mknod_latency": { 2023-07-21T19:07:20.491 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:20.492 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:20.492 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:20.492 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:20.492 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mksnap_latency": { 2023-07-21T19:07:20.493 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:20.493 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:20.493 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:20.494 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:20.494 INFO:teuthology.orchestra.run.smithi155.stdout: "req_open_latency": { 2023-07-21T19:07:20.494 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:20.495 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:20.495 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:20.495 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:20.496 INFO:teuthology.orchestra.run.smithi155.stdout: "req_readdir_latency": { 2023-07-21T19:07:20.496 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:20.496 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:20.496 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:20.497 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:20.497 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rename_latency": { 2023-07-21T19:07:20.497 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:20.497 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:20.498 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:20.498 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:20.498 INFO:teuthology.orchestra.run.smithi155.stdout: "req_renamesnap_latency": { 2023-07-21T19:07:20.499 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:20.499 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:20.499 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:20.499 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:20.499 INFO:teuthology.orchestra.run.smithi155.stdout: "req_snapdiff_latency": { 2023-07-21T19:07:20.500 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:20.500 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:20.500 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:20.500 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:20.501 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmdir_latency": { 2023-07-21T19:07:20.501 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:20.501 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:20.501 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:20.502 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:20.502 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmsnap_latency": { 2023-07-21T19:07:20.502 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:20.502 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:20.502 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:20.502 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:20.503 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmxattr_latency": { 2023-07-21T19:07:20.503 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:20.503 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:20.503 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:20.503 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:20.503 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setattr_latency": { 2023-07-21T19:07:20.503 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:20.504 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:20.504 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:20.504 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:20.504 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setdirlayout_latency": { 2023-07-21T19:07:20.504 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:20.505 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:20.505 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:20.505 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:20.505 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setfilelock_latency": { 2023-07-21T19:07:20.505 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:20.505 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:20.505 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:20.505 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:20.506 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setlayout_latency": { 2023-07-21T19:07:20.506 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:20.506 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:20.506 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:20.506 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:20.506 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setxattr_latency": { 2023-07-21T19:07:20.506 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:20.507 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:20.507 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:20.507 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:20.507 INFO:teuthology.orchestra.run.smithi155.stdout: "req_symlink_latency": { 2023-07-21T19:07:20.507 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:20.507 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:20.507 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:20.507 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:20.508 INFO:teuthology.orchestra.run.smithi155.stdout: "req_unlink_latency": { 2023-07-21T19:07:20.508 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:20.508 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:20.508 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:20.508 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:20.508 INFO:teuthology.orchestra.run.smithi155.stdout: "cap_revoke_eviction": 0, 2023-07-21T19:07:20.508 INFO:teuthology.orchestra.run.smithi155.stdout: "cap_acquisition_throttle": 0, 2023-07-21T19:07:20.508 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getvxattr_latency": { 2023-07-21T19:07:20.509 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:20.509 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:20.509 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:20.509 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-07-21T19:07:20.509 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:20.509 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_sessions": { 2023-07-21T19:07:20.509 INFO:teuthology.orchestra.run.smithi155.stdout: "session_count": 1, 2023-07-21T19:07:20.510 INFO:teuthology.orchestra.run.smithi155.stdout: "session_add": 2, 2023-07-21T19:07:20.510 INFO:teuthology.orchestra.run.smithi155.stdout: "session_remove": 1, 2023-07-21T19:07:20.510 INFO:teuthology.orchestra.run.smithi155.stdout: "sessions_open": 1, 2023-07-21T19:07:20.510 INFO:teuthology.orchestra.run.smithi155.stdout: "sessions_stale": 0, 2023-07-21T19:07:20.510 INFO:teuthology.orchestra.run.smithi155.stdout: "total_load": 0, 2023-07-21T19:07:20.510 INFO:teuthology.orchestra.run.smithi155.stdout: "average_load": 0, 2023-07-21T19:07:20.511 INFO:teuthology.orchestra.run.smithi155.stdout: "avg_session_uptime": 3009 2023-07-21T19:07:20.511 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:20.511 INFO:teuthology.orchestra.run.smithi155.stdout: "mempool": { 2023-07-21T19:07:20.511 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_bytes": 0, 2023-07-21T19:07:20.511 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_items": 0, 2023-07-21T19:07:20.511 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_bytes": 0, 2023-07-21T19:07:20.511 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_items": 0, 2023-07-21T19:07:20.511 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_bytes": 0, 2023-07-21T19:07:20.512 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_items": 0, 2023-07-21T19:07:20.512 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-21T19:07:20.512 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_items": 0, 2023-07-21T19:07:20.512 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-21T19:07:20.512 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_items": 0, 2023-07-21T19:07:20.512 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_bytes": 0, 2023-07-21T19:07:20.512 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_items": 0, 2023-07-21T19:07:20.513 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-21T19:07:20.513 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_items": 0, 2023-07-21T19:07:20.513 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_bytes": 0, 2023-07-21T19:07:20.513 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_items": 0, 2023-07-21T19:07:20.513 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_bytes": 0, 2023-07-21T19:07:20.513 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_items": 0, 2023-07-21T19:07:20.513 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-21T19:07:20.513 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_items": 0, 2023-07-21T19:07:20.514 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-21T19:07:20.514 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_items": 0, 2023-07-21T19:07:20.514 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_bytes": 0, 2023-07-21T19:07:20.514 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_items": 0, 2023-07-21T19:07:20.514 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_bytes": 0, 2023-07-21T19:07:20.514 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_items": 0, 2023-07-21T19:07:20.514 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-21T19:07:20.515 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_items": 0, 2023-07-21T19:07:20.516 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_bytes": 0, 2023-07-21T19:07:20.516 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_items": 0, 2023-07-21T19:07:20.516 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_bytes": 0, 2023-07-21T19:07:20.516 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_items": 0, 2023-07-21T19:07:20.517 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_bytes": 0, 2023-07-21T19:07:20.517 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_items": 0, 2023-07-21T19:07:20.517 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_bytes": 0, 2023-07-21T19:07:20.517 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_items": 0, 2023-07-21T19:07:20.517 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_bytes": 283708, 2023-07-21T19:07:20.517 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_items": 1573, 2023-07-21T19:07:20.518 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_bytes": 0, 2023-07-21T19:07:20.518 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_items": 0, 2023-07-21T19:07:20.518 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_bytes": 0, 2023-07-21T19:07:20.518 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_items": 0, 2023-07-21T19:07:20.518 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_bytes": 0, 2023-07-21T19:07:20.518 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_items": 0, 2023-07-21T19:07:20.518 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_bytes": 0, 2023-07-21T19:07:20.518 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_items": 0, 2023-07-21T19:07:20.519 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_bytes": 8160, 2023-07-21T19:07:20.519 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_items": 186, 2023-07-21T19:07:20.519 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_bytes": 0, 2023-07-21T19:07:20.519 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_items": 0, 2023-07-21T19:07:20.520 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_bytes": 0, 2023-07-21T19:07:20.520 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_items": 0, 2023-07-21T19:07:20.520 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_bytes": 16129411, 2023-07-21T19:07:20.520 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_items": 108217, 2023-07-21T19:07:20.521 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_bytes": 0, 2023-07-21T19:07:20.521 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_items": 0, 2023-07-21T19:07:20.521 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_bytes": 0, 2023-07-21T19:07:20.522 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_items": 0 2023-07-21T19:07:20.522 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:20.522 INFO:teuthology.orchestra.run.smithi155.stdout: "objecter": { 2023-07-21T19:07:20.522 INFO:teuthology.orchestra.run.smithi155.stdout: "op_active": 0, 2023-07-21T19:07:20.523 INFO:teuthology.orchestra.run.smithi155.stdout: "op_laggy": 0, 2023-07-21T19:07:20.523 INFO:teuthology.orchestra.run.smithi155.stdout: "op_send": 6356, 2023-07-21T19:07:20.523 INFO:teuthology.orchestra.run.smithi155.stdout: "op_send_bytes": 21, 2023-07-21T19:07:20.523 INFO:teuthology.orchestra.run.smithi155.stdout: "op_resend": 3, 2023-07-21T19:07:20.524 INFO:teuthology.orchestra.run.smithi155.stdout: "op_reply": 6356, 2023-07-21T19:07:20.524 INFO:teuthology.orchestra.run.smithi155.stdout: "op_latency": { 2023-07-21T19:07:20.524 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 6356, 2023-07-21T19:07:20.525 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 22.380574214, 2023-07-21T19:07:20.525 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.003521172 2023-07-21T19:07:20.525 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:20.525 INFO:teuthology.orchestra.run.smithi155.stdout: "op_inflight": 0, 2023-07-21T19:07:20.525 INFO:teuthology.orchestra.run.smithi155.stdout: "oplen_avg": { 2023-07-21T19:07:20.526 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 6356, 2023-07-21T19:07:20.526 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 6357 2023-07-21T19:07:20.526 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:20.526 INFO:teuthology.orchestra.run.smithi155.stdout: "op": 6356, 2023-07-21T19:07:20.526 INFO:teuthology.orchestra.run.smithi155.stdout: "op_r": 6356, 2023-07-21T19:07:20.526 INFO:teuthology.orchestra.run.smithi155.stdout: "op_w": 0, 2023-07-21T19:07:20.527 INFO:teuthology.orchestra.run.smithi155.stdout: "op_rmw": 0, 2023-07-21T19:07:20.527 INFO:teuthology.orchestra.run.smithi155.stdout: "op_pg": 0, 2023-07-21T19:07:20.527 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_stat": 3431, 2023-07-21T19:07:20.527 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_create": 0, 2023-07-21T19:07:20.527 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_read": 2923, 2023-07-21T19:07:20.527 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_write": 0, 2023-07-21T19:07:20.527 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_writefull": 0, 2023-07-21T19:07:20.527 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_writesame": 0, 2023-07-21T19:07:20.528 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_append": 0, 2023-07-21T19:07:20.528 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_zero": 0, 2023-07-21T19:07:20.528 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_truncate": 0, 2023-07-21T19:07:20.528 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_delete": 0, 2023-07-21T19:07:20.528 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_mapext": 0, 2023-07-21T19:07:20.528 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_sparse_read": 0, 2023-07-21T19:07:20.528 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_clonerange": 0, 2023-07-21T19:07:20.529 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_getxattr": 1, 2023-07-21T19:07:20.529 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_setxattr": 0, 2023-07-21T19:07:20.529 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_cmpxattr": 0, 2023-07-21T19:07:20.529 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_rmxattr": 0, 2023-07-21T19:07:20.529 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_resetxattrs": 0, 2023-07-21T19:07:20.529 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_call": 0, 2023-07-21T19:07:20.529 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_watch": 0, 2023-07-21T19:07:20.530 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_notify": 0, 2023-07-21T19:07:20.530 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_src_cmpxattr": 0, 2023-07-21T19:07:20.530 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_pgls": 0, 2023-07-21T19:07:20.530 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_pgls_filter": 0, 2023-07-21T19:07:20.530 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_other": 0, 2023-07-21T19:07:20.530 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_active": 0, 2023-07-21T19:07:20.531 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_send": 0, 2023-07-21T19:07:20.531 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_resend": 0, 2023-07-21T19:07:20.531 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_ping": 0, 2023-07-21T19:07:20.531 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_active": 0, 2023-07-21T19:07:20.531 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_send": 0, 2023-07-21T19:07:20.532 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_resend": 0, 2023-07-21T19:07:20.532 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_active": 0, 2023-07-21T19:07:20.532 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_send": 0, 2023-07-21T19:07:20.532 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_resend": 0, 2023-07-21T19:07:20.532 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_active": 0, 2023-07-21T19:07:20.532 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_send": 0, 2023-07-21T19:07:20.532 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_resend": 0, 2023-07-21T19:07:20.533 INFO:teuthology.orchestra.run.smithi155.stdout: "command_active": 0, 2023-07-21T19:07:20.533 INFO:teuthology.orchestra.run.smithi155.stdout: "command_send": 0, 2023-07-21T19:07:20.533 INFO:teuthology.orchestra.run.smithi155.stdout: "command_resend": 0, 2023-07-21T19:07:20.533 INFO:teuthology.orchestra.run.smithi155.stdout: "map_epoch": 124, 2023-07-21T19:07:20.533 INFO:teuthology.orchestra.run.smithi155.stdout: "map_full": 0, 2023-07-21T19:07:20.533 INFO:teuthology.orchestra.run.smithi155.stdout: "map_inc": 45, 2023-07-21T19:07:20.533 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_sessions": 10, 2023-07-21T19:07:20.533 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_session_open": 10, 2023-07-21T19:07:20.534 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_session_close": 0, 2023-07-21T19:07:20.534 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_laggy": 0, 2023-07-21T19:07:20.534 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_wr": 0, 2023-07-21T19:07:20.534 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_rd": 2, 2023-07-21T19:07:20.534 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_del": 0 2023-07-21T19:07:20.534 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:20.534 INFO:teuthology.orchestra.run.smithi155.stdout: "oft": { 2023-07-21T19:07:20.535 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_objs": 0, 2023-07-21T19:07:20.535 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_kv_pairs": 0, 2023-07-21T19:07:20.535 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_updates": 0, 2023-07-21T19:07:20.535 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_removes": 0 2023-07-21T19:07:20.535 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:20.535 INFO:teuthology.orchestra.run.smithi155.stdout: "osd-slow-ops": { 2023-07-21T19:07:20.535 INFO:teuthology.orchestra.run.smithi155.stdout: "slow_ops_count": 0 2023-07-21T19:07:20.535 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:20.536 INFO:teuthology.orchestra.run.smithi155.stdout: "purge_queue": { 2023-07-21T19:07:20.536 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_ops": 0, 2023-07-21T19:07:20.536 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_ops_high_water": 0, 2023-07-21T19:07:20.536 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing": 0, 2023-07-21T19:07:20.536 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_high_water": 0, 2023-07-21T19:07:20.536 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executed": 0, 2023-07-21T19:07:20.536 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_item_in_journal": 0 2023-07-21T19:07:20.536 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:20.537 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-21T19:07:20.537 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2023-07-21T19:07:20.537 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 104857600, 2023-07-21T19:07:20.537 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2023-07-21T19:07:20.537 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 6945, 2023-07-21T19:07:20.537 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 912668744, 2023-07-21T19:07:20.537 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2023-07-21T19:07:20.537 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 6945, 2023-07-21T19:07:20.538 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2023-07-21T19:07:20.538 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2023-07-21T19:07:20.538 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 6945, 2023-07-21T19:07:20.538 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 912668744, 2023-07-21T19:07:20.538 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2023-07-21T19:07:20.538 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:20.538 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:20.538 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:20.539 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-07-21T19:07:20.539 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:20.539 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-objecter_bytes": { 2023-07-21T19:07:20.539 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2023-07-21T19:07:20.539 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 104857600, 2023-07-21T19:07:20.539 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2023-07-21T19:07:20.539 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2023-07-21T19:07:20.540 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2023-07-21T19:07:20.540 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2023-07-21T19:07:20.540 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2023-07-21T19:07:20.540 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 6356, 2023-07-21T19:07:20.540 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 909649768, 2023-07-21T19:07:20.540 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 6356, 2023-07-21T19:07:20.540 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 909649768, 2023-07-21T19:07:20.540 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2023-07-21T19:07:20.541 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:20.541 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:20.541 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:20.541 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-07-21T19:07:20.541 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:20.541 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-objecter_ops": { 2023-07-21T19:07:20.541 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2023-07-21T19:07:20.541 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 1024, 2023-07-21T19:07:20.542 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2023-07-21T19:07:20.542 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2023-07-21T19:07:20.542 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2023-07-21T19:07:20.542 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2023-07-21T19:07:20.542 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2023-07-21T19:07:20.542 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 6356, 2023-07-21T19:07:20.542 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 6356, 2023-07-21T19:07:20.542 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 6356, 2023-07-21T19:07:20.543 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 6356, 2023-07-21T19:07:20.543 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2023-07-21T19:07:20.543 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:20.543 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:20.543 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:20.543 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-07-21T19:07:20.543 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:20.544 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-write_buf_throttle": { 2023-07-21T19:07:20.544 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2023-07-21T19:07:20.544 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 3758096384, 2023-07-21T19:07:20.544 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2023-07-21T19:07:20.544 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2023-07-21T19:07:20.544 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2023-07-21T19:07:20.544 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2023-07-21T19:07:20.544 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2023-07-21T19:07:20.545 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2023-07-21T19:07:20.545 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2023-07-21T19:07:20.545 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 0, 2023-07-21T19:07:20.545 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 0, 2023-07-21T19:07:20.545 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2023-07-21T19:07:20.545 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:20.545 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:20.545 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:20.546 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-07-21T19:07:20.546 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-21T19:07:20.546 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-write_buf_throttle-0x5596d9192460": { 2023-07-21T19:07:20.546 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2023-07-21T19:07:20.546 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 3758096384, 2023-07-21T19:07:20.546 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2023-07-21T19:07:20.546 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2023-07-21T19:07:20.546 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2023-07-21T19:07:20.547 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2023-07-21T19:07:20.547 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2023-07-21T19:07:20.547 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2023-07-21T19:07:20.547 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2023-07-21T19:07:20.547 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 0, 2023-07-21T19:07:20.547 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 0, 2023-07-21T19:07:20.547 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2023-07-21T19:07:20.547 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-21T19:07:20.548 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-21T19:07:20.548 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-21T19:07:20.548 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-07-21T19:07:20.548 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-07-21T19:07:20.548 INFO:teuthology.orchestra.run.smithi155.stdout:} 2023-07-21T19:07:20.793 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:20 smithi155 ceph-mon[97934]: pgmap v812: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 236 KiB/s wr, 32 op/s 2023-07-21T19:07:20.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:20 smithi193 ceph-mon[98204]: pgmap v812: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 236 KiB/s wr, 32 op/s 2023-07-21T19:07:20.879 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2023-07-21T19:07:20.879 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2023-07-21T19:07:20.880 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2023-07-21T19:07:20.880 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2023-07-21T19:07:20.880 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2023-07-21T19:07:20.880 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2023-07-21T19:07:20.880 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2023-07-21T19:07:20.880 DEBUG:tasks.check_counter:Getting stats from c 2023-07-21T19:07:20.881 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph daemon mds.c perf dump 2023-07-21T19:07:20.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:20 smithi039 ceph-mon[96489]: pgmap v812: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 236 KiB/s wr, 32 op/s 2023-07-21T19:07:21.579 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T19:07:22.705 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:22 smithi193 ceph-mon[98204]: pgmap v813: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 236 KiB/s wr, 31 op/s 2023-07-21T19:07:22.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:22 smithi155 ceph-mon[97934]: pgmap v813: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 236 KiB/s wr, 31 op/s 2023-07-21T19:07:22.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:22 smithi039 ceph-mon[96489]: pgmap v813: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 236 KiB/s wr, 31 op/s 2023-07-21T19:07:23.091 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2023-07-21T19:07:23.092 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-0": { 2023-07-21T19:07:23.092 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 681, 2023-07-21T19:07:23.092 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 681, 2023-07-21T19:07:23.092 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 50542699, 2023-07-21T19:07:23.092 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 211005, 2023-07-21T19:07:23.092 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 8, 2023-07-21T19:07:23.092 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 4, 2023-07-21T19:07:23.093 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.080369542, 2023-07-21T19:07:23.093 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.013846231, 2023-07-21T19:07:23.093 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.040389523, 2023-07-21T19:07:23.093 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.020184430, 2023-07-21T19:07:23.093 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:07:23.093 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 680, 2023-07-21T19:07:23.093 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.077623674, 2023-07-21T19:07:23.093 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000114152 2023-07-21T19:07:23.094 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:23.094 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:07:23.094 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:23.094 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:23.094 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:23.094 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:23.095 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-07-21T19:07:23.095 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 448 2023-07-21T19:07:23.095 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:23.095 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-1": { 2023-07-21T19:07:23.095 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 506, 2023-07-21T19:07:23.095 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 589, 2023-07-21T19:07:23.095 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 71598690, 2023-07-21T19:07:23.095 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 380472, 2023-07-21T19:07:23.096 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 6, 2023-07-21T19:07:23.096 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 5, 2023-07-21T19:07:23.096 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.077155786, 2023-07-21T19:07:23.099 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.011957878, 2023-07-21T19:07:23.099 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.048385479, 2023-07-21T19:07:23.099 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.013275244, 2023-07-21T19:07:23.099 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:07:23.099 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 587, 2023-07-21T19:07:23.099 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.045101639, 2023-07-21T19:07:23.099 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000076834 2023-07-21T19:07:23.100 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:23.100 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:07:23.100 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:23.100 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:23.100 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:23.100 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:23.100 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 160, 2023-07-21T19:07:23.101 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 223360 2023-07-21T19:07:23.101 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:23.101 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-2": { 2023-07-21T19:07:23.101 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 589, 2023-07-21T19:07:23.101 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 578, 2023-07-21T19:07:23.101 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 67480601, 2023-07-21T19:07:23.101 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 205121, 2023-07-21T19:07:23.101 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 5, 2023-07-21T19:07:23.102 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 4, 2023-07-21T19:07:23.102 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.085009415, 2023-07-21T19:07:23.102 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.013434405, 2023-07-21T19:07:23.102 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.054959609, 2023-07-21T19:07:23.102 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.014529253, 2023-07-21T19:07:23.102 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:07:23.102 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 577, 2023-07-21T19:07:23.102 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.077218037, 2023-07-21T19:07:23.103 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000133826 2023-07-21T19:07:23.103 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:23.103 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:07:23.103 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:23.103 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:23.103 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:23.103 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:23.103 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 256208, 2023-07-21T19:07:23.104 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 65472 2023-07-21T19:07:23.104 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:23.104 INFO:teuthology.orchestra.run.smithi193.stdout: "cct": { 2023-07-21T19:07:23.104 INFO:teuthology.orchestra.run.smithi193.stdout: "total_workers": 1, 2023-07-21T19:07:23.104 INFO:teuthology.orchestra.run.smithi193.stdout: "unhealthy_workers": 0 2023-07-21T19:07:23.104 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:23.104 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-MDSRank": { 2023-07-21T19:07:23.104 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2023-07-21T19:07:23.105 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2023-07-21T19:07:23.105 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 2757, 2023-07-21T19:07:23.105 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.188579469, 2023-07-21T19:07:23.105 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000068400 2023-07-21T19:07:23.105 INFO:teuthology.orchestra.run.smithi193.stdout: } 2023-07-21T19:07:23.105 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:23.105 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-PurgeQueue": { 2023-07-21T19:07:23.105 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2023-07-21T19:07:23.106 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2023-07-21T19:07:23.106 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:23.106 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:23.106 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:23.106 INFO:teuthology.orchestra.run.smithi193.stdout: } 2023-07-21T19:07:23.106 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:23.106 INFO:teuthology.orchestra.run.smithi193.stdout: "mds": { 2023-07-21T19:07:23.107 INFO:teuthology.orchestra.run.smithi193.stdout: "request": 0, 2023-07-21T19:07:23.107 INFO:teuthology.orchestra.run.smithi193.stdout: "reply": 0, 2023-07-21T19:07:23.107 INFO:teuthology.orchestra.run.smithi193.stdout: "reply_latency": { 2023-07-21T19:07:23.107 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:23.107 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:23.107 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:23.107 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:23.107 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_reply": 0, 2023-07-21T19:07:23.108 INFO:teuthology.orchestra.run.smithi193.stdout: "forward": 0, 2023-07-21T19:07:23.108 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_complete": 0, 2023-07-21T19:07:23.108 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_keys": 0, 2023-07-21T19:07:23.108 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_commit": 0, 2023-07-21T19:07:23.108 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_split": 0, 2023-07-21T19:07:23.108 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_merge": 0, 2023-07-21T19:07:23.109 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes": 10407, 2023-07-21T19:07:23.109 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_top": 6937, 2023-07-21T19:07:23.109 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_bottom": 3309, 2023-07-21T19:07:23.109 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pin_tail": 161, 2023-07-21T19:07:23.109 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pinned": 497, 2023-07-21T19:07:23.109 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_expired": 395, 2023-07-21T19:07:23.109 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_with_caps": 0, 2023-07-21T19:07:23.109 INFO:teuthology.orchestra.run.smithi193.stdout: "caps": 0, 2023-07-21T19:07:23.110 INFO:teuthology.orchestra.run.smithi193.stdout: "subtrees": 2, 2023-07-21T19:07:23.110 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse": 0, 2023-07-21T19:07:23.110 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_hit": 0, 2023-07-21T19:07:23.110 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_forward": 0, 2023-07-21T19:07:23.110 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_discover": 0, 2023-07-21T19:07:23.110 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_dir_fetch": 0, 2023-07-21T19:07:23.110 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_remote_ino": 0, 2023-07-21T19:07:23.110 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_lock": 0, 2023-07-21T19:07:23.111 INFO:teuthology.orchestra.run.smithi193.stdout: "load_cent": 0, 2023-07-21T19:07:23.111 INFO:teuthology.orchestra.run.smithi193.stdout: "q": 0, 2023-07-21T19:07:23.111 INFO:teuthology.orchestra.run.smithi193.stdout: "exported": 0, 2023-07-21T19:07:23.111 INFO:teuthology.orchestra.run.smithi193.stdout: "exported_inodes": 0, 2023-07-21T19:07:23.111 INFO:teuthology.orchestra.run.smithi193.stdout: "imported": 0, 2023-07-21T19:07:23.111 INFO:teuthology.orchestra.run.smithi193.stdout: "imported_inodes": 0, 2023-07-21T19:07:23.111 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_dir_fetch": 0, 2023-07-21T19:07:23.111 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_backtrace_fetch": 0, 2023-07-21T19:07:23.112 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_peer_discover": 0, 2023-07-21T19:07:23.112 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rfiles": 131, 2023-07-21T19:07:23.112 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rbytes": 42134725, 2023-07-21T19:07:23.112 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rsnaps": 7, 2023-07-21T19:07:23.112 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_fetch": 0, 2023-07-21T19:07:23.112 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_set_tag": 0, 2023-07-21T19:07:23.112 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_repaired": 0, 2023-07-21T19:07:23.112 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_inotable_repaired": 0, 2023-07-21T19:07:23.113 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_inodes": 0, 2023-07-21T19:07:23.113 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_base_inodes": 0, 2023-07-21T19:07:23.113 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dirfrag_rstats": 0, 2023-07-21T19:07:23.113 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_file_inodes": 0, 2023-07-21T19:07:23.113 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_inode_file_caps": 0, 2023-07-21T19:07:23.113 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_revoke": 0, 2023-07-21T19:07:23.113 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_grant": 0, 2023-07-21T19:07:23.113 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_trunc": 0, 2023-07-21T19:07:23.114 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-21T19:07:23.114 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-21T19:07:23.114 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps": 0, 2023-07-21T19:07:23.114 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps_dirty": 0, 2023-07-21T19:07:23.114 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_cap_release": 0, 2023-07-21T19:07:23.114 INFO:teuthology.orchestra.run.smithi193.stdout: "process_request_cap_release": 0 2023-07-21T19:07:23.114 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:23.115 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_cache": { 2023-07-21T19:07:23.115 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update": 0, 2023-07-21T19:07:23.115 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update_receipt": 0, 2023-07-21T19:07:23.115 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_try_discover": 0, 2023-07-21T19:07:23.115 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_send_discover": 0, 2023-07-21T19:07:23.115 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_handle_discover": 0, 2023-07-21T19:07:23.115 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays": 0, 2023-07-21T19:07:23.115 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_delayed": 0, 2023-07-21T19:07:23.116 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_enqueuing": 0, 2023-07-21T19:07:23.116 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_created": 0, 2023-07-21T19:07:23.116 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_enqueued": 0, 2023-07-21T19:07:23.116 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_reintegrated": 0, 2023-07-21T19:07:23.116 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_migrated": 0, 2023-07-21T19:07:23.116 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_processing": 0, 2023-07-21T19:07:23.116 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_enqueued": 0, 2023-07-21T19:07:23.116 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_prioritized": 0, 2023-07-21T19:07:23.117 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_started": 0, 2023-07-21T19:07:23.117 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_completed": 0, 2023-07-21T19:07:23.117 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_enqueue_scrub": 0, 2023-07-21T19:07:23.117 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_exportdir": 0, 2023-07-21T19:07:23.117 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_flush": 0, 2023-07-21T19:07:23.117 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragmentdir": 0, 2023-07-21T19:07:23.117 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragstats": 0, 2023-07-21T19:07:23.117 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_inodestats": 0 2023-07-21T19:07:23.118 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:23.118 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_log": { 2023-07-21T19:07:23.118 INFO:teuthology.orchestra.run.smithi193.stdout: "evadd": 0, 2023-07-21T19:07:23.118 INFO:teuthology.orchestra.run.smithi193.stdout: "evex": 0, 2023-07-21T19:07:23.118 INFO:teuthology.orchestra.run.smithi193.stdout: "evtrm": 0, 2023-07-21T19:07:23.118 INFO:teuthology.orchestra.run.smithi193.stdout: "ev": 0, 2023-07-21T19:07:23.118 INFO:teuthology.orchestra.run.smithi193.stdout: "evexg": 0, 2023-07-21T19:07:23.118 INFO:teuthology.orchestra.run.smithi193.stdout: "evexd": 0, 2023-07-21T19:07:23.119 INFO:teuthology.orchestra.run.smithi193.stdout: "segadd": 0, 2023-07-21T19:07:23.119 INFO:teuthology.orchestra.run.smithi193.stdout: "segex": 0, 2023-07-21T19:07:23.119 INFO:teuthology.orchestra.run.smithi193.stdout: "segtrm": 0, 2023-07-21T19:07:23.119 INFO:teuthology.orchestra.run.smithi193.stdout: "seg": 2, 2023-07-21T19:07:23.119 INFO:teuthology.orchestra.run.smithi193.stdout: "segexg": 0, 2023-07-21T19:07:23.119 INFO:teuthology.orchestra.run.smithi193.stdout: "segexd": 0, 2023-07-21T19:07:23.119 INFO:teuthology.orchestra.run.smithi193.stdout: "expos": 703277767, 2023-07-21T19:07:23.119 INFO:teuthology.orchestra.run.smithi193.stdout: "wrpos": 0, 2023-07-21T19:07:23.120 INFO:teuthology.orchestra.run.smithi193.stdout: "rdpos": 705166400, 2023-07-21T19:07:23.120 INFO:teuthology.orchestra.run.smithi193.stdout: "jlat": { 2023-07-21T19:07:23.120 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:23.120 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:23.120 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:23.121 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:23.121 INFO:teuthology.orchestra.run.smithi193.stdout: "replayed": 62119 2023-07-21T19:07:23.121 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:23.121 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_mem": { 2023-07-21T19:07:23.121 INFO:teuthology.orchestra.run.smithi193.stdout: "ino": 411, 2023-07-21T19:07:23.121 INFO:teuthology.orchestra.run.smithi193.stdout: "ino+": 10579, 2023-07-21T19:07:23.121 INFO:teuthology.orchestra.run.smithi193.stdout: "ino-": 10168, 2023-07-21T19:07:23.121 INFO:teuthology.orchestra.run.smithi193.stdout: "dir": 53, 2023-07-21T19:07:23.122 INFO:teuthology.orchestra.run.smithi193.stdout: "dir+": 78, 2023-07-21T19:07:23.122 INFO:teuthology.orchestra.run.smithi193.stdout: "dir-": 25, 2023-07-21T19:07:23.122 INFO:teuthology.orchestra.run.smithi193.stdout: "dn": 10407, 2023-07-21T19:07:23.122 INFO:teuthology.orchestra.run.smithi193.stdout: "dn+": 11105, 2023-07-21T19:07:23.122 INFO:teuthology.orchestra.run.smithi193.stdout: "dn-": 698, 2023-07-21T19:07:23.122 INFO:teuthology.orchestra.run.smithi193.stdout: "cap": 0, 2023-07-21T19:07:23.122 INFO:teuthology.orchestra.run.smithi193.stdout: "cap+": 0, 2023-07-21T19:07:23.122 INFO:teuthology.orchestra.run.smithi193.stdout: "cap-": 0, 2023-07-21T19:07:23.123 INFO:teuthology.orchestra.run.smithi193.stdout: "rss": 61180, 2023-07-21T19:07:23.123 INFO:teuthology.orchestra.run.smithi193.stdout: "heap": 207132 2023-07-21T19:07:23.123 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:23.123 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_server": { 2023-07-21T19:07:23.123 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_client_request": 0, 2023-07-21T19:07:23.123 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_server_request": 0, 2023-07-21T19:07:23.123 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_request": 0, 2023-07-21T19:07:23.123 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_session": 0, 2023-07-21T19:07:23.124 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_peer_request": 0, 2023-07-21T19:07:23.124 INFO:teuthology.orchestra.run.smithi193.stdout: "req_create_latency": { 2023-07-21T19:07:23.124 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:23.124 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:23.124 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:23.124 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:23.124 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getattr_latency": { 2023-07-21T19:07:23.125 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:23.125 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:23.125 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:23.125 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:23.125 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getfilelock_latency": { 2023-07-21T19:07:23.125 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:23.125 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:23.125 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:23.126 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:23.126 INFO:teuthology.orchestra.run.smithi193.stdout: "req_link_latency": { 2023-07-21T19:07:23.126 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:23.126 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:23.126 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:23.126 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:23.126 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookup_latency": { 2023-07-21T19:07:23.126 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:23.127 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:23.127 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:23.127 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:23.127 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookuphash_latency": { 2023-07-21T19:07:23.127 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:23.127 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:23.127 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:23.127 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:23.128 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupino_latency": { 2023-07-21T19:07:23.128 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:23.128 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:23.128 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:23.128 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:23.128 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupname_latency": { 2023-07-21T19:07:23.128 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:23.128 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:23.129 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:23.129 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:23.129 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupparent_latency": { 2023-07-21T19:07:23.129 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:23.129 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:23.129 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:23.129 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:23.130 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupsnap_latency": { 2023-07-21T19:07:23.130 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:23.130 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:23.130 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:23.130 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:23.130 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lssnap_latency": { 2023-07-21T19:07:23.130 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:23.130 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:23.131 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:23.131 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:23.131 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mkdir_latency": { 2023-07-21T19:07:23.131 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:23.131 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:23.131 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:23.131 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:23.131 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mknod_latency": { 2023-07-21T19:07:23.132 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:23.132 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:23.132 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:23.132 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:23.132 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mksnap_latency": { 2023-07-21T19:07:23.132 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:23.132 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:23.133 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:23.133 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:23.133 INFO:teuthology.orchestra.run.smithi193.stdout: "req_open_latency": { 2023-07-21T19:07:23.133 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:23.133 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:23.133 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:23.133 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:23.133 INFO:teuthology.orchestra.run.smithi193.stdout: "req_readdir_latency": { 2023-07-21T19:07:23.134 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:23.134 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:23.134 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:23.134 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:23.134 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rename_latency": { 2023-07-21T19:07:23.134 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:23.134 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:23.134 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:23.135 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:23.135 INFO:teuthology.orchestra.run.smithi193.stdout: "req_renamesnap_latency": { 2023-07-21T19:07:23.135 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:23.136 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:23.136 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:23.136 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:23.136 INFO:teuthology.orchestra.run.smithi193.stdout: "req_snapdiff_latency": { 2023-07-21T19:07:23.137 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:23.137 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:23.137 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:23.137 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:23.137 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmdir_latency": { 2023-07-21T19:07:23.137 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:23.137 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:23.138 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:23.138 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:23.138 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmsnap_latency": { 2023-07-21T19:07:23.138 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:23.138 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:23.138 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:23.138 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:23.139 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmxattr_latency": { 2023-07-21T19:07:23.139 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:23.139 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:23.139 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:23.139 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:23.139 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setattr_latency": { 2023-07-21T19:07:23.139 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:23.139 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:23.140 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:23.140 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:23.140 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setdirlayout_latency": { 2023-07-21T19:07:23.140 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:23.140 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:23.140 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:23.141 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:23.141 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setfilelock_latency": { 2023-07-21T19:07:23.141 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:23.141 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:23.141 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:23.141 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:23.142 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setlayout_latency": { 2023-07-21T19:07:23.142 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:23.142 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:23.142 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:23.142 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:23.142 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setxattr_latency": { 2023-07-21T19:07:23.142 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:23.142 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:23.143 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:23.143 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:23.143 INFO:teuthology.orchestra.run.smithi193.stdout: "req_symlink_latency": { 2023-07-21T19:07:23.143 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:23.143 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:23.143 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:23.143 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:23.144 INFO:teuthology.orchestra.run.smithi193.stdout: "req_unlink_latency": { 2023-07-21T19:07:23.144 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:23.144 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:23.144 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:23.144 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:23.144 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_revoke_eviction": 0, 2023-07-21T19:07:23.144 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_acquisition_throttle": 0, 2023-07-21T19:07:23.145 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getvxattr_latency": { 2023-07-21T19:07:23.145 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:23.145 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:23.145 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:23.145 INFO:teuthology.orchestra.run.smithi193.stdout: } 2023-07-21T19:07:23.145 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:23.145 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_sessions": { 2023-07-21T19:07:23.145 INFO:teuthology.orchestra.run.smithi193.stdout: "session_count": 2, 2023-07-21T19:07:23.146 INFO:teuthology.orchestra.run.smithi193.stdout: "session_add": 2, 2023-07-21T19:07:23.146 INFO:teuthology.orchestra.run.smithi193.stdout: "session_remove": 0, 2023-07-21T19:07:23.146 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_open": 0, 2023-07-21T19:07:23.146 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_stale": 0, 2023-07-21T19:07:23.146 INFO:teuthology.orchestra.run.smithi193.stdout: "total_load": 0, 2023-07-21T19:07:23.146 INFO:teuthology.orchestra.run.smithi193.stdout: "average_load": 0, 2023-07-21T19:07:23.146 INFO:teuthology.orchestra.run.smithi193.stdout: "avg_session_uptime": 3014 2023-07-21T19:07:23.147 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:23.147 INFO:teuthology.orchestra.run.smithi193.stdout: "mempool": { 2023-07-21T19:07:23.147 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_bytes": 0, 2023-07-21T19:07:23.147 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_items": 0, 2023-07-21T19:07:23.147 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_bytes": 0, 2023-07-21T19:07:23.147 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_items": 0, 2023-07-21T19:07:23.147 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_bytes": 0, 2023-07-21T19:07:23.147 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_items": 0, 2023-07-21T19:07:23.148 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-21T19:07:23.148 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_items": 0, 2023-07-21T19:07:23.148 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-21T19:07:23.148 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_items": 0, 2023-07-21T19:07:23.148 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_bytes": 0, 2023-07-21T19:07:23.148 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_items": 0, 2023-07-21T19:07:23.148 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-21T19:07:23.149 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_items": 0, 2023-07-21T19:07:23.149 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_bytes": 0, 2023-07-21T19:07:23.149 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_items": 0, 2023-07-21T19:07:23.149 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_bytes": 0, 2023-07-21T19:07:23.149 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_items": 0, 2023-07-21T19:07:23.149 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-21T19:07:23.149 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_items": 0, 2023-07-21T19:07:23.149 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-21T19:07:23.150 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_items": 0, 2023-07-21T19:07:23.150 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_bytes": 0, 2023-07-21T19:07:23.150 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_items": 0, 2023-07-21T19:07:23.150 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_bytes": 0, 2023-07-21T19:07:23.150 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_items": 0, 2023-07-21T19:07:23.150 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-21T19:07:23.150 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_items": 0, 2023-07-21T19:07:23.150 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_bytes": 0, 2023-07-21T19:07:23.151 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_items": 0, 2023-07-21T19:07:23.151 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_bytes": 0, 2023-07-21T19:07:23.151 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_items": 0, 2023-07-21T19:07:23.151 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_bytes": 0, 2023-07-21T19:07:23.151 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_items": 0, 2023-07-21T19:07:23.151 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_bytes": 0, 2023-07-21T19:07:23.151 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_items": 0, 2023-07-21T19:07:23.151 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_bytes": 249295, 2023-07-21T19:07:23.152 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_items": 625, 2023-07-21T19:07:23.152 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_bytes": 0, 2023-07-21T19:07:23.152 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_items": 0, 2023-07-21T19:07:23.152 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_bytes": 0, 2023-07-21T19:07:23.152 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_items": 0, 2023-07-21T19:07:23.152 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_bytes": 0, 2023-07-21T19:07:23.152 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_items": 0, 2023-07-21T19:07:23.153 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_bytes": 0, 2023-07-21T19:07:23.153 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_items": 0, 2023-07-21T19:07:23.153 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_bytes": 8160, 2023-07-21T19:07:23.153 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_items": 186, 2023-07-21T19:07:23.153 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_bytes": 0, 2023-07-21T19:07:23.153 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_items": 0, 2023-07-21T19:07:23.153 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_bytes": 0, 2023-07-21T19:07:23.153 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_items": 0, 2023-07-21T19:07:23.154 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_bytes": 8499770, 2023-07-21T19:07:23.154 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_items": 56119, 2023-07-21T19:07:23.154 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_bytes": 0, 2023-07-21T19:07:23.154 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_items": 0, 2023-07-21T19:07:23.154 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_bytes": 0, 2023-07-21T19:07:23.154 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_items": 0 2023-07-21T19:07:23.154 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:23.155 INFO:teuthology.orchestra.run.smithi193.stdout: "objecter": { 2023-07-21T19:07:23.155 INFO:teuthology.orchestra.run.smithi193.stdout: "op_active": 0, 2023-07-21T19:07:23.155 INFO:teuthology.orchestra.run.smithi193.stdout: "op_laggy": 0, 2023-07-21T19:07:23.155 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send": 1631, 2023-07-21T19:07:23.155 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send_bytes": 26, 2023-07-21T19:07:23.155 INFO:teuthology.orchestra.run.smithi193.stdout: "op_resend": 4, 2023-07-21T19:07:23.155 INFO:teuthology.orchestra.run.smithi193.stdout: "op_reply": 1631, 2023-07-21T19:07:23.155 INFO:teuthology.orchestra.run.smithi193.stdout: "op_latency": { 2023-07-21T19:07:23.156 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 1631, 2023-07-21T19:07:23.156 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 9.690022139, 2023-07-21T19:07:23.156 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.005941153 2023-07-21T19:07:23.156 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:23.156 INFO:teuthology.orchestra.run.smithi193.stdout: "op_inflight": 0, 2023-07-21T19:07:23.156 INFO:teuthology.orchestra.run.smithi193.stdout: "oplen_avg": { 2023-07-21T19:07:23.157 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 1631, 2023-07-21T19:07:23.157 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 1632 2023-07-21T19:07:23.157 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:23.157 INFO:teuthology.orchestra.run.smithi193.stdout: "op": 1631, 2023-07-21T19:07:23.157 INFO:teuthology.orchestra.run.smithi193.stdout: "op_r": 1631, 2023-07-21T19:07:23.157 INFO:teuthology.orchestra.run.smithi193.stdout: "op_w": 0, 2023-07-21T19:07:23.158 INFO:teuthology.orchestra.run.smithi193.stdout: "op_rmw": 0, 2023-07-21T19:07:23.158 INFO:teuthology.orchestra.run.smithi193.stdout: "op_pg": 0, 2023-07-21T19:07:23.158 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_stat": 824, 2023-07-21T19:07:23.158 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_create": 0, 2023-07-21T19:07:23.158 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_read": 804, 2023-07-21T19:07:23.159 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_write": 0, 2023-07-21T19:07:23.159 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writefull": 0, 2023-07-21T19:07:23.159 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writesame": 0, 2023-07-21T19:07:23.159 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_append": 0, 2023-07-21T19:07:23.159 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_zero": 0, 2023-07-21T19:07:23.160 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_truncate": 0, 2023-07-21T19:07:23.160 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_delete": 0, 2023-07-21T19:07:23.160 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_mapext": 0, 2023-07-21T19:07:23.160 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_sparse_read": 0, 2023-07-21T19:07:23.160 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_clonerange": 0, 2023-07-21T19:07:23.160 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_getxattr": 2, 2023-07-21T19:07:23.160 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_setxattr": 0, 2023-07-21T19:07:23.160 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_cmpxattr": 0, 2023-07-21T19:07:23.161 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_rmxattr": 0, 2023-07-21T19:07:23.161 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_resetxattrs": 0, 2023-07-21T19:07:23.161 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_call": 0, 2023-07-21T19:07:23.161 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_watch": 0, 2023-07-21T19:07:23.161 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_notify": 0, 2023-07-21T19:07:23.161 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_src_cmpxattr": 0, 2023-07-21T19:07:23.161 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls": 0, 2023-07-21T19:07:23.161 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls_filter": 0, 2023-07-21T19:07:23.162 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_other": 0, 2023-07-21T19:07:23.162 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_active": 0, 2023-07-21T19:07:23.162 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_send": 0, 2023-07-21T19:07:23.162 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_resend": 0, 2023-07-21T19:07:23.162 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_ping": 0, 2023-07-21T19:07:23.162 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_active": 0, 2023-07-21T19:07:23.162 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_send": 0, 2023-07-21T19:07:23.162 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_resend": 0, 2023-07-21T19:07:23.163 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_active": 0, 2023-07-21T19:07:23.163 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_send": 0, 2023-07-21T19:07:23.163 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_resend": 0, 2023-07-21T19:07:23.163 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_active": 0, 2023-07-21T19:07:23.163 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_send": 0, 2023-07-21T19:07:23.163 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_resend": 0, 2023-07-21T19:07:23.163 INFO:teuthology.orchestra.run.smithi193.stdout: "command_active": 0, 2023-07-21T19:07:23.163 INFO:teuthology.orchestra.run.smithi193.stdout: "command_send": 0, 2023-07-21T19:07:23.164 INFO:teuthology.orchestra.run.smithi193.stdout: "command_resend": 0, 2023-07-21T19:07:23.164 INFO:teuthology.orchestra.run.smithi193.stdout: "map_epoch": 124, 2023-07-21T19:07:23.164 INFO:teuthology.orchestra.run.smithi193.stdout: "map_full": 0, 2023-07-21T19:07:23.164 INFO:teuthology.orchestra.run.smithi193.stdout: "map_inc": 12, 2023-07-21T19:07:23.164 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_sessions": 10, 2023-07-21T19:07:23.164 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_open": 10, 2023-07-21T19:07:23.164 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_close": 0, 2023-07-21T19:07:23.165 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_laggy": 0, 2023-07-21T19:07:23.165 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_wr": 0, 2023-07-21T19:07:23.165 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_rd": 2, 2023-07-21T19:07:23.165 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_del": 0 2023-07-21T19:07:23.165 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:23.165 INFO:teuthology.orchestra.run.smithi193.stdout: "oft": { 2023-07-21T19:07:23.165 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_objs": 0, 2023-07-21T19:07:23.165 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_kv_pairs": 0, 2023-07-21T19:07:23.166 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_updates": 0, 2023-07-21T19:07:23.166 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_removes": 0 2023-07-21T19:07:23.166 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:23.166 INFO:teuthology.orchestra.run.smithi193.stdout: "osd-slow-ops": { 2023-07-21T19:07:23.166 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_ops_count": 0 2023-07-21T19:07:23.166 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:23.166 INFO:teuthology.orchestra.run.smithi193.stdout: "purge_queue": { 2023-07-21T19:07:23.167 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops": 0, 2023-07-21T19:07:23.167 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops_high_water": 0, 2023-07-21T19:07:23.167 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing": 0, 2023-07-21T19:07:23.167 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_high_water": 0, 2023-07-21T19:07:23.167 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed": 0, 2023-07-21T19:07:23.167 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_item_in_journal": 0 2023-07-21T19:07:23.167 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:23.167 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-21T19:07:23.168 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2023-07-21T19:07:23.168 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2023-07-21T19:07:23.168 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2023-07-21T19:07:23.168 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 1772, 2023-07-21T19:07:23.168 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 189452457, 2023-07-21T19:07:23.169 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2023-07-21T19:07:23.169 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 1772, 2023-07-21T19:07:23.169 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2023-07-21T19:07:23.169 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2023-07-21T19:07:23.169 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 1772, 2023-07-21T19:07:23.169 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 189452457, 2023-07-21T19:07:23.169 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2023-07-21T19:07:23.169 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:23.170 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:23.170 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:23.170 INFO:teuthology.orchestra.run.smithi193.stdout: } 2023-07-21T19:07:23.170 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:23.170 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_bytes": { 2023-07-21T19:07:23.170 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2023-07-21T19:07:23.170 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2023-07-21T19:07:23.170 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2023-07-21T19:07:23.171 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2023-07-21T19:07:23.171 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2023-07-21T19:07:23.171 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2023-07-21T19:07:23.171 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2023-07-21T19:07:23.171 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 1631, 2023-07-21T19:07:23.171 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 188909672, 2023-07-21T19:07:23.171 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 1631, 2023-07-21T19:07:23.172 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 188909672, 2023-07-21T19:07:23.172 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2023-07-21T19:07:23.172 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:23.172 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:23.172 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:23.172 INFO:teuthology.orchestra.run.smithi193.stdout: } 2023-07-21T19:07:23.172 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:23.172 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_ops": { 2023-07-21T19:07:23.173 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2023-07-21T19:07:23.173 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 1024, 2023-07-21T19:07:23.173 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2023-07-21T19:07:23.173 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2023-07-21T19:07:23.173 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2023-07-21T19:07:23.173 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2023-07-21T19:07:23.173 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2023-07-21T19:07:23.173 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 1631, 2023-07-21T19:07:23.174 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 1631, 2023-07-21T19:07:23.174 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 1631, 2023-07-21T19:07:23.174 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 1631, 2023-07-21T19:07:23.174 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2023-07-21T19:07:23.174 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:23.174 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:23.174 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:23.174 INFO:teuthology.orchestra.run.smithi193.stdout: } 2023-07-21T19:07:23.175 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:23.175 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle": { 2023-07-21T19:07:23.175 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2023-07-21T19:07:23.175 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2023-07-21T19:07:23.175 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2023-07-21T19:07:23.175 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2023-07-21T19:07:23.175 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2023-07-21T19:07:23.175 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2023-07-21T19:07:23.176 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2023-07-21T19:07:23.176 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2023-07-21T19:07:23.176 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2023-07-21T19:07:23.176 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 0, 2023-07-21T19:07:23.176 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 0, 2023-07-21T19:07:23.176 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2023-07-21T19:07:23.176 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:23.177 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:23.177 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:23.177 INFO:teuthology.orchestra.run.smithi193.stdout: } 2023-07-21T19:07:23.177 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:23.177 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle-0x55a21a124460": { 2023-07-21T19:07:23.177 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2023-07-21T19:07:23.177 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2023-07-21T19:07:23.178 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2023-07-21T19:07:23.178 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2023-07-21T19:07:23.178 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2023-07-21T19:07:23.178 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2023-07-21T19:07:23.178 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2023-07-21T19:07:23.178 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2023-07-21T19:07:23.178 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2023-07-21T19:07:23.178 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 0, 2023-07-21T19:07:23.179 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 0, 2023-07-21T19:07:23.179 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2023-07-21T19:07:23.179 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:23.179 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:23.179 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:23.179 INFO:teuthology.orchestra.run.smithi193.stdout: } 2023-07-21T19:07:23.179 INFO:teuthology.orchestra.run.smithi193.stdout: } 2023-07-21T19:07:23.179 INFO:teuthology.orchestra.run.smithi193.stdout:} 2023-07-21T19:07:23.433 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2023-07-21T19:07:23.434 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2023-07-21T19:07:23.434 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2023-07-21T19:07:23.434 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2023-07-21T19:07:23.434 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=7 2023-07-21T19:07:23.434 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2023-07-21T19:07:23.434 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2023-07-21T19:07:23.435 DEBUG:tasks.check_counter:Getting stats from f 2023-07-21T19:07:23.435 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph daemon mds.f perf dump 2023-07-21T19:07:24.026 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T19:07:24.577 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:24 smithi193 ceph-mon[98204]: pgmap v814: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 9.8 KiB/s wr, 14 op/s 2023-07-21T19:07:24.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:24 smithi155 ceph-mon[97934]: pgmap v814: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 9.8 KiB/s wr, 14 op/s 2023-07-21T19:07:24.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:24 smithi039 ceph-mon[96489]: pgmap v814: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 9.8 KiB/s wr, 14 op/s 2023-07-21T19:07:25.281 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2023-07-21T19:07:25.282 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-0": { 2023-07-21T19:07:25.282 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 3300, 2023-07-21T19:07:25.282 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 3202, 2023-07-21T19:07:25.283 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 336896613, 2023-07-21T19:07:25.283 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 1104611, 2023-07-21T19:07:25.283 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 7, 2023-07-21T19:07:25.283 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 3, 2023-07-21T19:07:25.283 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.454482506, 2023-07-21T19:07:25.283 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.083463391, 2023-07-21T19:07:25.284 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.262144050, 2023-07-21T19:07:25.284 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.095773744, 2023-07-21T19:07:25.284 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:07:25.284 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 3202, 2023-07-21T19:07:25.284 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.216622367, 2023-07-21T19:07:25.284 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000067652 2023-07-21T19:07:25.285 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:25.285 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:07:25.285 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:25.285 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:25.285 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:25.285 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:25.285 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 1928304, 2023-07-21T19:07:25.285 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 262112 2023-07-21T19:07:25.286 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:25.286 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-1": { 2023-07-21T19:07:25.286 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 1193, 2023-07-21T19:07:25.286 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 1527, 2023-07-21T19:07:25.286 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 289745417, 2023-07-21T19:07:25.287 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 1103007, 2023-07-21T19:07:25.287 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 8, 2023-07-21T19:07:25.287 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 6, 2023-07-21T19:07:25.287 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.262157501, 2023-07-21T19:07:25.287 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.031016752, 2023-07-21T19:07:25.288 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.187067798, 2023-07-21T19:07:25.288 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.032161697, 2023-07-21T19:07:25.288 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:07:25.288 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 1524, 2023-07-21T19:07:25.289 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.090036416, 2023-07-21T19:07:25.289 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000059079 2023-07-21T19:07:25.289 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:25.289 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:07:25.289 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:25.289 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:25.289 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:25.289 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:25.290 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 320, 2023-07-21T19:07:25.290 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 732544 2023-07-21T19:07:25.290 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:25.290 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-2": { 2023-07-21T19:07:25.290 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 2748, 2023-07-21T19:07:25.290 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 2750, 2023-07-21T19:07:25.290 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 646664525, 2023-07-21T19:07:25.290 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 854635, 2023-07-21T19:07:25.291 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 7, 2023-07-21T19:07:25.291 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 5, 2023-07-21T19:07:25.291 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.479955940, 2023-07-21T19:07:25.291 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.054997315, 2023-07-21T19:07:25.291 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.335474709, 2023-07-21T19:07:25.291 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.072689481, 2023-07-21T19:07:25.292 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:07:25.292 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 2748, 2023-07-21T19:07:25.292 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.159291694, 2023-07-21T19:07:25.292 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000057966 2023-07-21T19:07:25.292 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:25.293 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:07:25.293 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:25.293 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:25.294 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:25.294 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:25.294 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-21T19:07:25.294 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 0 2023-07-21T19:07:25.294 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:25.294 INFO:teuthology.orchestra.run.smithi193.stdout: "cct": { 2023-07-21T19:07:25.295 INFO:teuthology.orchestra.run.smithi193.stdout: "total_workers": 1, 2023-07-21T19:07:25.295 INFO:teuthology.orchestra.run.smithi193.stdout: "unhealthy_workers": 0 2023-07-21T19:07:25.295 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:25.295 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-MDSRank": { 2023-07-21T19:07:25.295 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2023-07-21T19:07:25.295 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2023-07-21T19:07:25.295 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 10957, 2023-07-21T19:07:25.295 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.511666660, 2023-07-21T19:07:25.296 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000046697 2023-07-21T19:07:25.296 INFO:teuthology.orchestra.run.smithi193.stdout: } 2023-07-21T19:07:25.296 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:25.296 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-PurgeQueue": { 2023-07-21T19:07:25.296 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2023-07-21T19:07:25.296 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2023-07-21T19:07:25.296 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:25.297 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:25.297 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:25.297 INFO:teuthology.orchestra.run.smithi193.stdout: } 2023-07-21T19:07:25.297 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:25.297 INFO:teuthology.orchestra.run.smithi193.stdout: "mds": { 2023-07-21T19:07:25.297 INFO:teuthology.orchestra.run.smithi193.stdout: "request": 0, 2023-07-21T19:07:25.297 INFO:teuthology.orchestra.run.smithi193.stdout: "reply": 0, 2023-07-21T19:07:25.298 INFO:teuthology.orchestra.run.smithi193.stdout: "reply_latency": { 2023-07-21T19:07:25.298 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:25.298 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:25.298 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:25.298 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:25.298 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_reply": 0, 2023-07-21T19:07:25.298 INFO:teuthology.orchestra.run.smithi193.stdout: "forward": 0, 2023-07-21T19:07:25.298 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_complete": 0, 2023-07-21T19:07:25.299 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_keys": 0, 2023-07-21T19:07:25.299 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_commit": 0, 2023-07-21T19:07:25.299 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_split": 0, 2023-07-21T19:07:25.299 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_merge": 0, 2023-07-21T19:07:25.299 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes": 21845, 2023-07-21T19:07:25.299 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_top": 8168, 2023-07-21T19:07:25.299 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_bottom": 13667, 2023-07-21T19:07:25.299 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pin_tail": 10, 2023-07-21T19:07:25.300 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pinned": 10176, 2023-07-21T19:07:25.300 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_expired": 17066, 2023-07-21T19:07:25.300 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_with_caps": 0, 2023-07-21T19:07:25.300 INFO:teuthology.orchestra.run.smithi193.stdout: "caps": 0, 2023-07-21T19:07:25.300 INFO:teuthology.orchestra.run.smithi193.stdout: "subtrees": 1, 2023-07-21T19:07:25.300 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse": 0, 2023-07-21T19:07:25.300 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_hit": 0, 2023-07-21T19:07:25.300 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_forward": 0, 2023-07-21T19:07:25.301 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_discover": 0, 2023-07-21T19:07:25.301 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_dir_fetch": 0, 2023-07-21T19:07:25.301 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_remote_ino": 0, 2023-07-21T19:07:25.301 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_lock": 0, 2023-07-21T19:07:25.301 INFO:teuthology.orchestra.run.smithi193.stdout: "load_cent": 0, 2023-07-21T19:07:25.301 INFO:teuthology.orchestra.run.smithi193.stdout: "q": 0, 2023-07-21T19:07:25.301 INFO:teuthology.orchestra.run.smithi193.stdout: "exported": 0, 2023-07-21T19:07:25.302 INFO:teuthology.orchestra.run.smithi193.stdout: "exported_inodes": 0, 2023-07-21T19:07:25.302 INFO:teuthology.orchestra.run.smithi193.stdout: "imported": 0, 2023-07-21T19:07:25.302 INFO:teuthology.orchestra.run.smithi193.stdout: "imported_inodes": 0, 2023-07-21T19:07:25.302 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_dir_fetch": 0, 2023-07-21T19:07:25.302 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_backtrace_fetch": 0, 2023-07-21T19:07:25.302 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_peer_discover": 0, 2023-07-21T19:07:25.302 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rfiles": 0, 2023-07-21T19:07:25.302 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rbytes": 0, 2023-07-21T19:07:25.303 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rsnaps": 0, 2023-07-21T19:07:25.303 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_fetch": 0, 2023-07-21T19:07:25.303 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_set_tag": 0, 2023-07-21T19:07:25.303 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_repaired": 0, 2023-07-21T19:07:25.303 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_inotable_repaired": 0, 2023-07-21T19:07:25.303 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_inodes": 0, 2023-07-21T19:07:25.303 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_base_inodes": 0, 2023-07-21T19:07:25.303 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dirfrag_rstats": 0, 2023-07-21T19:07:25.304 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_file_inodes": 0, 2023-07-21T19:07:25.304 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_inode_file_caps": 0, 2023-07-21T19:07:25.304 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_revoke": 0, 2023-07-21T19:07:25.304 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_grant": 0, 2023-07-21T19:07:25.304 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_trunc": 0, 2023-07-21T19:07:25.304 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-21T19:07:25.304 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-21T19:07:25.305 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps": 0, 2023-07-21T19:07:25.305 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps_dirty": 0, 2023-07-21T19:07:25.305 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_cap_release": 0, 2023-07-21T19:07:25.305 INFO:teuthology.orchestra.run.smithi193.stdout: "process_request_cap_release": 0 2023-07-21T19:07:25.305 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:25.305 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_cache": { 2023-07-21T19:07:25.305 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update": 0, 2023-07-21T19:07:25.305 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update_receipt": 0, 2023-07-21T19:07:25.306 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_try_discover": 0, 2023-07-21T19:07:25.306 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_send_discover": 0, 2023-07-21T19:07:25.306 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_handle_discover": 0, 2023-07-21T19:07:25.306 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays": 0, 2023-07-21T19:07:25.306 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_delayed": 0, 2023-07-21T19:07:25.306 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_enqueuing": 0, 2023-07-21T19:07:25.306 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_created": 0, 2023-07-21T19:07:25.306 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_enqueued": 0, 2023-07-21T19:07:25.307 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_reintegrated": 0, 2023-07-21T19:07:25.307 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_migrated": 0, 2023-07-21T19:07:25.307 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_processing": 0, 2023-07-21T19:07:25.307 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_enqueued": 0, 2023-07-21T19:07:25.307 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_prioritized": 0, 2023-07-21T19:07:25.307 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_started": 0, 2023-07-21T19:07:25.307 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_completed": 0, 2023-07-21T19:07:25.307 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_enqueue_scrub": 0, 2023-07-21T19:07:25.308 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_exportdir": 0, 2023-07-21T19:07:25.308 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_flush": 0, 2023-07-21T19:07:25.308 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragmentdir": 0, 2023-07-21T19:07:25.308 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragstats": 0, 2023-07-21T19:07:25.308 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_inodestats": 0 2023-07-21T19:07:25.308 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:25.308 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_log": { 2023-07-21T19:07:25.309 INFO:teuthology.orchestra.run.smithi193.stdout: "evadd": 0, 2023-07-21T19:07:25.309 INFO:teuthology.orchestra.run.smithi193.stdout: "evex": 0, 2023-07-21T19:07:25.309 INFO:teuthology.orchestra.run.smithi193.stdout: "evtrm": 0, 2023-07-21T19:07:25.309 INFO:teuthology.orchestra.run.smithi193.stdout: "ev": 0, 2023-07-21T19:07:25.309 INFO:teuthology.orchestra.run.smithi193.stdout: "evexg": 0, 2023-07-21T19:07:25.309 INFO:teuthology.orchestra.run.smithi193.stdout: "evexd": 0, 2023-07-21T19:07:25.309 INFO:teuthology.orchestra.run.smithi193.stdout: "segadd": 0, 2023-07-21T19:07:25.309 INFO:teuthology.orchestra.run.smithi193.stdout: "segex": 0, 2023-07-21T19:07:25.310 INFO:teuthology.orchestra.run.smithi193.stdout: "segtrm": 0, 2023-07-21T19:07:25.310 INFO:teuthology.orchestra.run.smithi193.stdout: "seg": 128, 2023-07-21T19:07:25.310 INFO:teuthology.orchestra.run.smithi193.stdout: "segexg": 0, 2023-07-21T19:07:25.310 INFO:teuthology.orchestra.run.smithi193.stdout: "segexd": 0, 2023-07-21T19:07:25.310 INFO:teuthology.orchestra.run.smithi193.stdout: "expos": 738218953, 2023-07-21T19:07:25.310 INFO:teuthology.orchestra.run.smithi193.stdout: "wrpos": 0, 2023-07-21T19:07:25.310 INFO:teuthology.orchestra.run.smithi193.stdout: "rdpos": 1273660069, 2023-07-21T19:07:25.310 INFO:teuthology.orchestra.run.smithi193.stdout: "jlat": { 2023-07-21T19:07:25.311 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:25.311 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:25.311 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:25.311 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:25.311 INFO:teuthology.orchestra.run.smithi193.stdout: "replayed": 186843 2023-07-21T19:07:25.311 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:25.311 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_mem": { 2023-07-21T19:07:25.311 INFO:teuthology.orchestra.run.smithi193.stdout: "ino": 1708, 2023-07-21T19:07:25.312 INFO:teuthology.orchestra.run.smithi193.stdout: "ino+": 38841, 2023-07-21T19:07:25.312 INFO:teuthology.orchestra.run.smithi193.stdout: "ino-": 37133, 2023-07-21T19:07:25.312 INFO:teuthology.orchestra.run.smithi193.stdout: "dir": 62, 2023-07-21T19:07:25.312 INFO:teuthology.orchestra.run.smithi193.stdout: "dir+": 137, 2023-07-21T19:07:25.312 INFO:teuthology.orchestra.run.smithi193.stdout: "dir-": 75, 2023-07-21T19:07:25.312 INFO:teuthology.orchestra.run.smithi193.stdout: "dn": 21845, 2023-07-21T19:07:25.312 INFO:teuthology.orchestra.run.smithi193.stdout: "dn+": 39589, 2023-07-21T19:07:25.312 INFO:teuthology.orchestra.run.smithi193.stdout: "dn-": 17744, 2023-07-21T19:07:25.313 INFO:teuthology.orchestra.run.smithi193.stdout: "cap": 0, 2023-07-21T19:07:25.313 INFO:teuthology.orchestra.run.smithi193.stdout: "cap+": 0, 2023-07-21T19:07:25.313 INFO:teuthology.orchestra.run.smithi193.stdout: "cap-": 0, 2023-07-21T19:07:25.313 INFO:teuthology.orchestra.run.smithi193.stdout: "rss": 79500, 2023-07-21T19:07:25.313 INFO:teuthology.orchestra.run.smithi193.stdout: "heap": 207156 2023-07-21T19:07:25.313 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:25.313 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_server": { 2023-07-21T19:07:25.313 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_client_request": 0, 2023-07-21T19:07:25.314 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_server_request": 0, 2023-07-21T19:07:25.314 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_request": 0, 2023-07-21T19:07:25.314 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_session": 0, 2023-07-21T19:07:25.314 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_peer_request": 0, 2023-07-21T19:07:25.314 INFO:teuthology.orchestra.run.smithi193.stdout: "req_create_latency": { 2023-07-21T19:07:25.314 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:25.314 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:25.314 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:25.315 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:25.315 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getattr_latency": { 2023-07-21T19:07:25.315 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:25.315 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:25.315 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:25.315 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:25.315 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getfilelock_latency": { 2023-07-21T19:07:25.316 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:25.316 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:25.316 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:25.316 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:25.316 INFO:teuthology.orchestra.run.smithi193.stdout: "req_link_latency": { 2023-07-21T19:07:25.316 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:25.316 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:25.316 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:25.317 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:25.317 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookup_latency": { 2023-07-21T19:07:25.317 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:25.317 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:25.317 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:25.317 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:25.317 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookuphash_latency": { 2023-07-21T19:07:25.317 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:25.318 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:25.318 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:25.318 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:25.318 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupino_latency": { 2023-07-21T19:07:25.318 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:25.318 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:25.318 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:25.318 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:25.319 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupname_latency": { 2023-07-21T19:07:25.319 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:25.319 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:25.319 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:25.319 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:25.319 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupparent_latency": { 2023-07-21T19:07:25.319 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:25.319 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:25.320 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:25.320 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:25.320 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupsnap_latency": { 2023-07-21T19:07:25.320 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:25.320 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:25.320 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:25.320 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:25.321 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lssnap_latency": { 2023-07-21T19:07:25.321 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:25.321 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:25.321 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:25.321 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:25.321 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mkdir_latency": { 2023-07-21T19:07:25.321 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:25.321 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:25.322 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:25.322 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:25.322 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mknod_latency": { 2023-07-21T19:07:25.322 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:25.322 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:25.322 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:25.322 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:25.322 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mksnap_latency": { 2023-07-21T19:07:25.323 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:25.323 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:25.323 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:25.323 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:25.323 INFO:teuthology.orchestra.run.smithi193.stdout: "req_open_latency": { 2023-07-21T19:07:25.323 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:25.323 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:25.323 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:25.324 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:25.324 INFO:teuthology.orchestra.run.smithi193.stdout: "req_readdir_latency": { 2023-07-21T19:07:25.324 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:25.324 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:25.324 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:25.324 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:25.324 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rename_latency": { 2023-07-21T19:07:25.324 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:25.325 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:25.325 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:25.325 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:25.325 INFO:teuthology.orchestra.run.smithi193.stdout: "req_renamesnap_latency": { 2023-07-21T19:07:25.325 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:25.325 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:25.325 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:25.325 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:25.326 INFO:teuthology.orchestra.run.smithi193.stdout: "req_snapdiff_latency": { 2023-07-21T19:07:25.326 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:25.326 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:25.326 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:25.326 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:25.326 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmdir_latency": { 2023-07-21T19:07:25.326 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:25.327 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:25.327 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:25.327 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:25.327 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmsnap_latency": { 2023-07-21T19:07:25.327 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:25.327 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:25.327 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:25.327 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:25.328 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmxattr_latency": { 2023-07-21T19:07:25.328 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:25.328 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:25.328 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:25.328 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:25.328 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setattr_latency": { 2023-07-21T19:07:25.328 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:25.328 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:25.329 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:25.329 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:25.329 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setdirlayout_latency": { 2023-07-21T19:07:25.329 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:25.329 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:25.329 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:25.329 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:25.329 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setfilelock_latency": { 2023-07-21T19:07:25.330 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:25.330 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:25.330 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:25.330 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:25.330 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setlayout_latency": { 2023-07-21T19:07:25.330 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:25.330 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:25.330 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:25.331 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:25.331 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setxattr_latency": { 2023-07-21T19:07:25.331 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:25.331 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:25.331 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:25.331 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:25.331 INFO:teuthology.orchestra.run.smithi193.stdout: "req_symlink_latency": { 2023-07-21T19:07:25.331 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:25.332 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:25.332 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:25.332 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:25.332 INFO:teuthology.orchestra.run.smithi193.stdout: "req_unlink_latency": { 2023-07-21T19:07:25.332 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:25.332 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:25.332 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:25.333 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:25.333 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_revoke_eviction": 0, 2023-07-21T19:07:25.333 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_acquisition_throttle": 0, 2023-07-21T19:07:25.333 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getvxattr_latency": { 2023-07-21T19:07:25.333 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:25.333 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:25.333 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:25.333 INFO:teuthology.orchestra.run.smithi193.stdout: } 2023-07-21T19:07:25.334 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:25.334 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_sessions": { 2023-07-21T19:07:25.334 INFO:teuthology.orchestra.run.smithi193.stdout: "session_count": 2, 2023-07-21T19:07:25.334 INFO:teuthology.orchestra.run.smithi193.stdout: "session_add": 2, 2023-07-21T19:07:25.334 INFO:teuthology.orchestra.run.smithi193.stdout: "session_remove": 0, 2023-07-21T19:07:25.334 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_open": 2, 2023-07-21T19:07:25.334 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_stale": 0, 2023-07-21T19:07:25.334 INFO:teuthology.orchestra.run.smithi193.stdout: "total_load": 0, 2023-07-21T19:07:25.335 INFO:teuthology.orchestra.run.smithi193.stdout: "average_load": 0, 2023-07-21T19:07:25.335 INFO:teuthology.orchestra.run.smithi193.stdout: "avg_session_uptime": 3019 2023-07-21T19:07:25.335 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:25.335 INFO:teuthology.orchestra.run.smithi193.stdout: "mempool": { 2023-07-21T19:07:25.335 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_bytes": 0, 2023-07-21T19:07:25.335 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_items": 0, 2023-07-21T19:07:25.335 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_bytes": 0, 2023-07-21T19:07:25.335 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_items": 0, 2023-07-21T19:07:25.336 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_bytes": 0, 2023-07-21T19:07:25.336 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_items": 0, 2023-07-21T19:07:25.336 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-21T19:07:25.336 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_items": 0, 2023-07-21T19:07:25.336 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-21T19:07:25.336 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_items": 0, 2023-07-21T19:07:25.336 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_bytes": 0, 2023-07-21T19:07:25.336 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_items": 0, 2023-07-21T19:07:25.337 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-21T19:07:25.337 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_items": 0, 2023-07-21T19:07:25.337 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_bytes": 0, 2023-07-21T19:07:25.337 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_items": 0, 2023-07-21T19:07:25.337 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_bytes": 0, 2023-07-21T19:07:25.337 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_items": 0, 2023-07-21T19:07:25.337 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-21T19:07:25.337 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_items": 0, 2023-07-21T19:07:25.338 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-21T19:07:25.338 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_items": 0, 2023-07-21T19:07:25.338 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_bytes": 0, 2023-07-21T19:07:25.338 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_items": 0, 2023-07-21T19:07:25.338 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_bytes": 0, 2023-07-21T19:07:25.338 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_items": 0, 2023-07-21T19:07:25.338 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-21T19:07:25.338 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_items": 0, 2023-07-21T19:07:25.339 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_bytes": 0, 2023-07-21T19:07:25.339 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_items": 0, 2023-07-21T19:07:25.339 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_bytes": 0, 2023-07-21T19:07:25.339 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_items": 0, 2023-07-21T19:07:25.339 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_bytes": 0, 2023-07-21T19:07:25.339 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_items": 0, 2023-07-21T19:07:25.339 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_bytes": 0, 2023-07-21T19:07:25.339 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_items": 0, 2023-07-21T19:07:25.340 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_bytes": 305755, 2023-07-21T19:07:25.340 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_items": 2199, 2023-07-21T19:07:25.340 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_bytes": 0, 2023-07-21T19:07:25.340 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_items": 0, 2023-07-21T19:07:25.340 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_bytes": 0, 2023-07-21T19:07:25.340 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_items": 0, 2023-07-21T19:07:25.340 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_bytes": 0, 2023-07-21T19:07:25.341 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_items": 0, 2023-07-21T19:07:25.341 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_bytes": 0, 2023-07-21T19:07:25.341 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_items": 0, 2023-07-21T19:07:25.341 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_bytes": 8160, 2023-07-21T19:07:25.341 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_items": 186, 2023-07-21T19:07:25.341 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_bytes": 0, 2023-07-21T19:07:25.341 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_items": 0, 2023-07-21T19:07:25.341 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_bytes": 0, 2023-07-21T19:07:25.342 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_items": 0, 2023-07-21T19:07:25.342 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_bytes": 19868395, 2023-07-21T19:07:25.342 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_items": 135888, 2023-07-21T19:07:25.342 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_bytes": 0, 2023-07-21T19:07:25.342 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_items": 0, 2023-07-21T19:07:25.342 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_bytes": 0, 2023-07-21T19:07:25.342 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_items": 0 2023-07-21T19:07:25.342 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:25.343 INFO:teuthology.orchestra.run.smithi193.stdout: "objecter": { 2023-07-21T19:07:25.343 INFO:teuthology.orchestra.run.smithi193.stdout: "op_active": 0, 2023-07-21T19:07:25.343 INFO:teuthology.orchestra.run.smithi193.stdout: "op_laggy": 0, 2023-07-21T19:07:25.343 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send": 6648, 2023-07-21T19:07:25.343 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send_bytes": 21, 2023-07-21T19:07:25.343 INFO:teuthology.orchestra.run.smithi193.stdout: "op_resend": 3, 2023-07-21T19:07:25.343 INFO:teuthology.orchestra.run.smithi193.stdout: "op_reply": 6648, 2023-07-21T19:07:25.343 INFO:teuthology.orchestra.run.smithi193.stdout: "op_latency": { 2023-07-21T19:07:25.344 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 6648, 2023-07-21T19:07:25.344 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 21.985050075, 2023-07-21T19:07:25.344 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.003307017 2023-07-21T19:07:25.344 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:25.344 INFO:teuthology.orchestra.run.smithi193.stdout: "op_inflight": 0, 2023-07-21T19:07:25.344 INFO:teuthology.orchestra.run.smithi193.stdout: "oplen_avg": { 2023-07-21T19:07:25.344 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 6648, 2023-07-21T19:07:25.344 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 6649 2023-07-21T19:07:25.345 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:25.345 INFO:teuthology.orchestra.run.smithi193.stdout: "op": 6648, 2023-07-21T19:07:25.345 INFO:teuthology.orchestra.run.smithi193.stdout: "op_r": 6648, 2023-07-21T19:07:25.345 INFO:teuthology.orchestra.run.smithi193.stdout: "op_w": 0, 2023-07-21T19:07:25.345 INFO:teuthology.orchestra.run.smithi193.stdout: "op_rmw": 0, 2023-07-21T19:07:25.345 INFO:teuthology.orchestra.run.smithi193.stdout: "op_pg": 0, 2023-07-21T19:07:25.345 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_stat": 3558, 2023-07-21T19:07:25.345 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_create": 0, 2023-07-21T19:07:25.346 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_read": 3088, 2023-07-21T19:07:25.346 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_write": 0, 2023-07-21T19:07:25.346 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writefull": 0, 2023-07-21T19:07:25.346 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writesame": 0, 2023-07-21T19:07:25.346 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_append": 0, 2023-07-21T19:07:25.346 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_zero": 0, 2023-07-21T19:07:25.346 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_truncate": 0, 2023-07-21T19:07:25.346 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_delete": 0, 2023-07-21T19:07:25.347 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_mapext": 0, 2023-07-21T19:07:25.347 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_sparse_read": 0, 2023-07-21T19:07:25.347 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_clonerange": 0, 2023-07-21T19:07:25.347 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_getxattr": 1, 2023-07-21T19:07:25.347 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_setxattr": 0, 2023-07-21T19:07:25.347 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_cmpxattr": 0, 2023-07-21T19:07:25.347 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_rmxattr": 0, 2023-07-21T19:07:25.347 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_resetxattrs": 0, 2023-07-21T19:07:25.348 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_call": 0, 2023-07-21T19:07:25.348 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_watch": 0, 2023-07-21T19:07:25.348 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_notify": 0, 2023-07-21T19:07:25.348 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_src_cmpxattr": 0, 2023-07-21T19:07:25.348 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls": 0, 2023-07-21T19:07:25.348 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls_filter": 0, 2023-07-21T19:07:25.348 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_other": 0, 2023-07-21T19:07:25.348 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_active": 0, 2023-07-21T19:07:25.349 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_send": 0, 2023-07-21T19:07:25.349 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_resend": 0, 2023-07-21T19:07:25.349 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_ping": 0, 2023-07-21T19:07:25.349 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_active": 0, 2023-07-21T19:07:25.349 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_send": 0, 2023-07-21T19:07:25.349 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_resend": 0, 2023-07-21T19:07:25.349 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_active": 0, 2023-07-21T19:07:25.349 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_send": 0, 2023-07-21T19:07:25.350 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_resend": 0, 2023-07-21T19:07:25.350 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_active": 0, 2023-07-21T19:07:25.350 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_send": 0, 2023-07-21T19:07:25.350 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_resend": 0, 2023-07-21T19:07:25.350 INFO:teuthology.orchestra.run.smithi193.stdout: "command_active": 0, 2023-07-21T19:07:25.350 INFO:teuthology.orchestra.run.smithi193.stdout: "command_send": 0, 2023-07-21T19:07:25.350 INFO:teuthology.orchestra.run.smithi193.stdout: "command_resend": 0, 2023-07-21T19:07:25.350 INFO:teuthology.orchestra.run.smithi193.stdout: "map_epoch": 124, 2023-07-21T19:07:25.351 INFO:teuthology.orchestra.run.smithi193.stdout: "map_full": 0, 2023-07-21T19:07:25.351 INFO:teuthology.orchestra.run.smithi193.stdout: "map_inc": 45, 2023-07-21T19:07:25.351 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_sessions": 10, 2023-07-21T19:07:25.351 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_open": 11, 2023-07-21T19:07:25.351 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_close": 1, 2023-07-21T19:07:25.351 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_laggy": 0, 2023-07-21T19:07:25.351 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_wr": 0, 2023-07-21T19:07:25.352 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_rd": 2, 2023-07-21T19:07:25.352 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_del": 0 2023-07-21T19:07:25.352 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:25.352 INFO:teuthology.orchestra.run.smithi193.stdout: "oft": { 2023-07-21T19:07:25.352 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_objs": 0, 2023-07-21T19:07:25.352 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_kv_pairs": 0, 2023-07-21T19:07:25.352 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_updates": 0, 2023-07-21T19:07:25.352 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_removes": 0 2023-07-21T19:07:25.353 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:25.353 INFO:teuthology.orchestra.run.smithi193.stdout: "osd-slow-ops": { 2023-07-21T19:07:25.353 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_ops_count": 0 2023-07-21T19:07:25.353 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:25.353 INFO:teuthology.orchestra.run.smithi193.stdout: "purge_queue": { 2023-07-21T19:07:25.353 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops": 0, 2023-07-21T19:07:25.353 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops_high_water": 0, 2023-07-21T19:07:25.353 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing": 0, 2023-07-21T19:07:25.354 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_high_water": 0, 2023-07-21T19:07:25.354 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed": 0, 2023-07-21T19:07:25.354 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_item_in_journal": 0 2023-07-21T19:07:25.354 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:25.354 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-21T19:07:25.354 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2023-07-21T19:07:25.354 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2023-07-21T19:07:25.354 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2023-07-21T19:07:25.355 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 7237, 2023-07-21T19:07:25.355 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 1272622723, 2023-07-21T19:07:25.355 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2023-07-21T19:07:25.355 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 7237, 2023-07-21T19:07:25.355 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2023-07-21T19:07:25.355 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2023-07-21T19:07:25.355 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 7237, 2023-07-21T19:07:25.355 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 1272622723, 2023-07-21T19:07:25.356 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2023-07-21T19:07:25.356 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:25.356 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:25.356 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:25.356 INFO:teuthology.orchestra.run.smithi193.stdout: } 2023-07-21T19:07:25.356 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:25.356 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_bytes": { 2023-07-21T19:07:25.356 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2023-07-21T19:07:25.357 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2023-07-21T19:07:25.357 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2023-07-21T19:07:25.357 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2023-07-21T19:07:25.357 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2023-07-21T19:07:25.357 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2023-07-21T19:07:25.357 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2023-07-21T19:07:25.357 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 6648, 2023-07-21T19:07:25.358 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 1269554533, 2023-07-21T19:07:25.358 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 6648, 2023-07-21T19:07:25.358 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 1269554533, 2023-07-21T19:07:25.358 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2023-07-21T19:07:25.358 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:25.358 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:25.358 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:25.358 INFO:teuthology.orchestra.run.smithi193.stdout: } 2023-07-21T19:07:25.359 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:25.359 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_ops": { 2023-07-21T19:07:25.359 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2023-07-21T19:07:25.359 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 1024, 2023-07-21T19:07:25.359 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2023-07-21T19:07:25.359 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2023-07-21T19:07:25.359 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2023-07-21T19:07:25.359 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2023-07-21T19:07:25.360 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2023-07-21T19:07:25.360 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 6648, 2023-07-21T19:07:25.360 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 6648, 2023-07-21T19:07:25.360 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 6648, 2023-07-21T19:07:25.360 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 6648, 2023-07-21T19:07:25.360 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2023-07-21T19:07:25.360 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:25.360 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:25.361 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:25.361 INFO:teuthology.orchestra.run.smithi193.stdout: } 2023-07-21T19:07:25.361 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:25.361 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle": { 2023-07-21T19:07:25.361 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2023-07-21T19:07:25.361 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2023-07-21T19:07:25.361 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2023-07-21T19:07:25.361 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2023-07-21T19:07:25.362 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2023-07-21T19:07:25.362 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2023-07-21T19:07:25.362 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2023-07-21T19:07:25.362 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2023-07-21T19:07:25.362 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2023-07-21T19:07:25.362 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 0, 2023-07-21T19:07:25.362 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 0, 2023-07-21T19:07:25.363 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2023-07-21T19:07:25.363 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:25.363 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:25.363 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:25.363 INFO:teuthology.orchestra.run.smithi193.stdout: } 2023-07-21T19:07:25.363 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:25.363 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle-0x562a41fce460": { 2023-07-21T19:07:25.363 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2023-07-21T19:07:25.364 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2023-07-21T19:07:25.364 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2023-07-21T19:07:25.364 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2023-07-21T19:07:25.364 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2023-07-21T19:07:25.364 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2023-07-21T19:07:25.364 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2023-07-21T19:07:25.364 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2023-07-21T19:07:25.364 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2023-07-21T19:07:25.365 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 0, 2023-07-21T19:07:25.365 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 0, 2023-07-21T19:07:25.365 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2023-07-21T19:07:25.365 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:25.365 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:25.365 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:25.365 INFO:teuthology.orchestra.run.smithi193.stdout: } 2023-07-21T19:07:25.365 INFO:teuthology.orchestra.run.smithi193.stdout: } 2023-07-21T19:07:25.366 INFO:teuthology.orchestra.run.smithi193.stdout:} 2023-07-21T19:07:25.681 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2023-07-21T19:07:25.682 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2023-07-21T19:07:25.682 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2023-07-21T19:07:25.682 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2023-07-21T19:07:25.682 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2023-07-21T19:07:25.682 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2023-07-21T19:07:25.683 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2023-07-21T19:07:25.683 DEBUG:tasks.check_counter:Getting stats from i 2023-07-21T19:07:25.683 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph daemon mds.i perf dump 2023-07-21T19:07:26.236 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T19:07:26.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:26 smithi193 ceph-mon[98204]: pgmap v815: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 9.8 KiB/s wr, 14 op/s 2023-07-21T19:07:26.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:26 smithi155 ceph-mon[97934]: pgmap v815: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 9.8 KiB/s wr, 14 op/s 2023-07-21T19:07:26.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:26 smithi039 ceph-mon[96489]: pgmap v815: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 9.8 KiB/s wr, 14 op/s 2023-07-21T19:07:27.535 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2023-07-21T19:07:27.535 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-0": { 2023-07-21T19:07:27.535 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 720586, 2023-07-21T19:07:27.536 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 712618, 2023-07-21T19:07:27.536 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 207319527, 2023-07-21T19:07:27.536 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 493531156, 2023-07-21T19:07:27.536 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 10, 2023-07-21T19:07:27.536 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 3, 2023-07-21T19:07:27.537 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 31.666925289, 2023-07-21T19:07:27.537 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 12.337754314, 2023-07-21T19:07:27.537 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 23.316685330, 2023-07-21T19:07:27.537 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.885535878, 2023-07-21T19:07:27.537 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:07:27.537 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 712620, 2023-07-21T19:07:27.538 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 42.196390989, 2023-07-21T19:07:27.538 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000059213 2023-07-21T19:07:27.538 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:27.538 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:07:27.538 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 1352150, 2023-07-21T19:07:27.538 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.167435279, 2023-07-21T19:07:27.538 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000123 2023-07-21T19:07:27.539 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:27.539 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-21T19:07:27.539 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 0 2023-07-21T19:07:27.539 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:27.539 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-1": { 2023-07-21T19:07:27.539 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 232343, 2023-07-21T19:07:27.540 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 237279, 2023-07-21T19:07:27.540 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 64239883, 2023-07-21T19:07:27.540 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 418690343, 2023-07-21T19:07:27.540 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 9, 2023-07-21T19:07:27.540 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 7, 2023-07-21T19:07:27.540 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 12.556340552, 2023-07-21T19:07:27.540 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 4.920440911, 2023-07-21T19:07:27.541 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 7.494793682, 2023-07-21T19:07:27.541 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 1.328268243, 2023-07-21T19:07:27.541 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:07:27.541 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 237278, 2023-07-21T19:07:27.541 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 19.230607461, 2023-07-21T19:07:27.541 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000081046 2023-07-21T19:07:27.541 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:27.542 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:07:27.542 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 383356, 2023-07-21T19:07:27.542 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.056126633, 2023-07-21T19:07:27.542 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000146 2023-07-21T19:07:27.542 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:27.542 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 1936864, 2023-07-21T19:07:27.542 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 286368 2023-07-21T19:07:27.542 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:27.543 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-2": { 2023-07-21T19:07:27.543 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 412289, 2023-07-21T19:07:27.543 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 606922, 2023-07-21T19:07:27.543 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 121348282, 2023-07-21T19:07:27.543 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 698943260, 2023-07-21T19:07:27.543 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 26, 2023-07-21T19:07:27.544 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 7, 2023-07-21T19:07:27.544 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 20.451147653, 2023-07-21T19:07:27.544 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 9.893037141, 2023-07-21T19:07:27.544 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 13.273504489, 2023-07-21T19:07:27.544 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 1.402975223, 2023-07-21T19:07:27.544 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:07:27.544 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 606916, 2023-07-21T19:07:27.545 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 48.307766925, 2023-07-21T19:07:27.545 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000079595 2023-07-21T19:07:27.545 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:27.545 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:07:27.545 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 499196, 2023-07-21T19:07:27.545 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.107293669, 2023-07-21T19:07:27.545 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000214 2023-07-21T19:07:27.545 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:27.546 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 5024, 2023-07-21T19:07:27.546 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 735840 2023-07-21T19:07:27.546 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:27.546 INFO:teuthology.orchestra.run.smithi193.stdout: "cct": { 2023-07-21T19:07:27.546 INFO:teuthology.orchestra.run.smithi193.stdout: "total_workers": 1, 2023-07-21T19:07:27.546 INFO:teuthology.orchestra.run.smithi193.stdout: "unhealthy_workers": 0 2023-07-21T19:07:27.547 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:27.547 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-MDSRank": { 2023-07-21T19:07:27.547 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2023-07-21T19:07:27.547 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2023-07-21T19:07:27.547 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 90878, 2023-07-21T19:07:27.547 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 78.407931310, 2023-07-21T19:07:27.548 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000862782 2023-07-21T19:07:27.548 INFO:teuthology.orchestra.run.smithi193.stdout: } 2023-07-21T19:07:27.548 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:27.549 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-PurgeQueue": { 2023-07-21T19:07:27.549 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2023-07-21T19:07:27.549 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2023-07-21T19:07:27.549 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 56528, 2023-07-21T19:07:27.549 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 4.979992493, 2023-07-21T19:07:27.550 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000088097 2023-07-21T19:07:27.550 INFO:teuthology.orchestra.run.smithi193.stdout: } 2023-07-21T19:07:27.550 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:27.550 INFO:teuthology.orchestra.run.smithi193.stdout: "mds": { 2023-07-21T19:07:27.551 INFO:teuthology.orchestra.run.smithi193.stdout: "request": 268427, 2023-07-21T19:07:27.552 INFO:teuthology.orchestra.run.smithi193.stdout: "reply": 222386, 2023-07-21T19:07:27.552 INFO:teuthology.orchestra.run.smithi193.stdout: "reply_latency": { 2023-07-21T19:07:27.552 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 222386, 2023-07-21T19:07:27.552 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 865.855233046, 2023-07-21T19:07:27.552 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.003893479 2023-07-21T19:07:27.552 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:27.553 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_reply": 0, 2023-07-21T19:07:27.553 INFO:teuthology.orchestra.run.smithi193.stdout: "forward": 46041, 2023-07-21T19:07:27.553 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_complete": 2, 2023-07-21T19:07:27.553 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_keys": 0, 2023-07-21T19:07:27.553 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_commit": 91, 2023-07-21T19:07:27.553 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_split": 6, 2023-07-21T19:07:27.553 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_merge": 3, 2023-07-21T19:07:27.553 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes": 1959, 2023-07-21T19:07:27.554 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_top": 945, 2023-07-21T19:07:27.554 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_bottom": 984, 2023-07-21T19:07:27.554 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pin_tail": 30, 2023-07-21T19:07:27.554 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pinned": 597, 2023-07-21T19:07:27.554 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_expired": 914, 2023-07-21T19:07:27.554 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_with_caps": 0, 2023-07-21T19:07:27.554 INFO:teuthology.orchestra.run.smithi193.stdout: "caps": 0, 2023-07-21T19:07:27.554 INFO:teuthology.orchestra.run.smithi193.stdout: "subtrees": 6, 2023-07-21T19:07:27.555 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse": 417065, 2023-07-21T19:07:27.555 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_hit": 215464, 2023-07-21T19:07:27.555 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_forward": 46034, 2023-07-21T19:07:27.555 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_discover": 292, 2023-07-21T19:07:27.555 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_dir_fetch": 0, 2023-07-21T19:07:27.555 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_remote_ino": 0, 2023-07-21T19:07:27.555 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_lock": 4644, 2023-07-21T19:07:27.555 INFO:teuthology.orchestra.run.smithi193.stdout: "load_cent": 0, 2023-07-21T19:07:27.556 INFO:teuthology.orchestra.run.smithi193.stdout: "q": 0, 2023-07-21T19:07:27.556 INFO:teuthology.orchestra.run.smithi193.stdout: "exported": 11, 2023-07-21T19:07:27.556 INFO:teuthology.orchestra.run.smithi193.stdout: "exported_inodes": 280, 2023-07-21T19:07:27.556 INFO:teuthology.orchestra.run.smithi193.stdout: "imported": 11, 2023-07-21T19:07:27.556 INFO:teuthology.orchestra.run.smithi193.stdout: "imported_inodes": 660, 2023-07-21T19:07:27.556 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_dir_fetch": 0, 2023-07-21T19:07:27.556 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_backtrace_fetch": 23, 2023-07-21T19:07:27.557 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_peer_discover": 0, 2023-07-21T19:07:27.557 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rfiles": 131, 2023-07-21T19:07:27.557 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rbytes": 42134725, 2023-07-21T19:07:27.557 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rsnaps": 7, 2023-07-21T19:07:27.557 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_fetch": 23, 2023-07-21T19:07:27.557 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_set_tag": 0, 2023-07-21T19:07:27.557 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_repaired": 0, 2023-07-21T19:07:27.558 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_inotable_repaired": 0, 2023-07-21T19:07:27.558 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_inodes": 0, 2023-07-21T19:07:27.558 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_base_inodes": 0, 2023-07-21T19:07:27.558 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dirfrag_rstats": 0, 2023-07-21T19:07:27.558 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_file_inodes": 23, 2023-07-21T19:07:27.558 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_inode_file_caps": 26, 2023-07-21T19:07:27.558 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_revoke": 60932, 2023-07-21T19:07:27.559 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_grant": 149291, 2023-07-21T19:07:27.559 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_trunc": 0, 2023-07-21T19:07:27.559 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flushsnap_ack": 1224, 2023-07-21T19:07:27.559 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-21T19:07:27.559 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps": 95354, 2023-07-21T19:07:27.559 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps_dirty": 20750, 2023-07-21T19:07:27.559 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_cap_release": 4449, 2023-07-21T19:07:27.559 INFO:teuthology.orchestra.run.smithi193.stdout: "process_request_cap_release": 183605 2023-07-21T19:07:27.560 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:27.560 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_cache": { 2023-07-21T19:07:27.560 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update": 603, 2023-07-21T19:07:27.560 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update_receipt": 6156, 2023-07-21T19:07:27.560 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_try_discover": 3163, 2023-07-21T19:07:27.560 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_send_discover": 6126, 2023-07-21T19:07:27.560 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_handle_discover": 1439, 2023-07-21T19:07:27.561 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays": 20, 2023-07-21T19:07:27.561 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_delayed": 0, 2023-07-21T19:07:27.561 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_enqueuing": 0, 2023-07-21T19:07:27.561 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_created": 30250, 2023-07-21T19:07:27.561 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_enqueued": 30230, 2023-07-21T19:07:27.562 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_reintegrated": 0, 2023-07-21T19:07:27.562 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_migrated": 0, 2023-07-21T19:07:27.562 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_processing": 0, 2023-07-21T19:07:27.562 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_enqueued": 0, 2023-07-21T19:07:27.562 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_prioritized": 0, 2023-07-21T19:07:27.562 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_started": 0, 2023-07-21T19:07:27.563 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_completed": 0, 2023-07-21T19:07:27.563 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_enqueue_scrub": 0, 2023-07-21T19:07:27.563 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_exportdir": 11, 2023-07-21T19:07:27.563 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_flush": 0, 2023-07-21T19:07:27.563 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragmentdir": 9, 2023-07-21T19:07:27.563 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragstats": 0, 2023-07-21T19:07:27.564 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_inodestats": 0 2023-07-21T19:07:27.564 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:27.564 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_log": { 2023-07-21T19:07:27.564 INFO:teuthology.orchestra.run.smithi193.stdout: "evadd": 148303, 2023-07-21T19:07:27.564 INFO:teuthology.orchestra.run.smithi193.stdout: "evex": 93296, 2023-07-21T19:07:27.564 INFO:teuthology.orchestra.run.smithi193.stdout: "evtrm": 93296, 2023-07-21T19:07:27.564 INFO:teuthology.orchestra.run.smithi193.stdout: "ev": 55007, 2023-07-21T19:07:27.565 INFO:teuthology.orchestra.run.smithi193.stdout: "evexg": 0, 2023-07-21T19:07:27.565 INFO:teuthology.orchestra.run.smithi193.stdout: "evexd": 541, 2023-07-21T19:07:27.565 INFO:teuthology.orchestra.run.smithi193.stdout: "segadd": 254, 2023-07-21T19:07:27.565 INFO:teuthology.orchestra.run.smithi193.stdout: "segex": 126, 2023-07-21T19:07:27.565 INFO:teuthology.orchestra.run.smithi193.stdout: "segtrm": 126, 2023-07-21T19:07:27.565 INFO:teuthology.orchestra.run.smithi193.stdout: "seg": 129, 2023-07-21T19:07:27.565 INFO:teuthology.orchestra.run.smithi193.stdout: "segexg": 0, 2023-07-21T19:07:27.565 INFO:teuthology.orchestra.run.smithi193.stdout: "segexd": 1, 2023-07-21T19:07:27.566 INFO:teuthology.orchestra.run.smithi193.stdout: "expos": 377498254, 2023-07-21T19:07:27.566 INFO:teuthology.orchestra.run.smithi193.stdout: "wrpos": 913818354, 2023-07-21T19:07:27.566 INFO:teuthology.orchestra.run.smithi193.stdout: "rdpos": 0, 2023-07-21T19:07:27.566 INFO:teuthology.orchestra.run.smithi193.stdout: "jlat": { 2023-07-21T19:07:27.566 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 48319, 2023-07-21T19:07:27.566 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 370.411817704, 2023-07-21T19:07:27.566 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.007665966 2023-07-21T19:07:27.567 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:27.567 INFO:teuthology.orchestra.run.smithi193.stdout: "replayed": 0 2023-07-21T19:07:27.567 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:27.567 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_mem": { 2023-07-21T19:07:27.567 INFO:teuthology.orchestra.run.smithi193.stdout: "ino": 1268, 2023-07-21T19:07:27.567 INFO:teuthology.orchestra.run.smithi193.stdout: "ino+": 32049, 2023-07-21T19:07:27.568 INFO:teuthology.orchestra.run.smithi193.stdout: "ino-": 30781, 2023-07-21T19:07:27.568 INFO:teuthology.orchestra.run.smithi193.stdout: "dir": 71, 2023-07-21T19:07:27.568 INFO:teuthology.orchestra.run.smithi193.stdout: "dir+": 3215, 2023-07-21T19:07:27.568 INFO:teuthology.orchestra.run.smithi193.stdout: "dir-": 3144, 2023-07-21T19:07:27.568 INFO:teuthology.orchestra.run.smithi193.stdout: "dn": 1959, 2023-07-21T19:07:27.568 INFO:teuthology.orchestra.run.smithi193.stdout: "dn+": 56165, 2023-07-21T19:07:27.568 INFO:teuthology.orchestra.run.smithi193.stdout: "dn-": 54206, 2023-07-21T19:07:27.568 INFO:teuthology.orchestra.run.smithi193.stdout: "cap": 0, 2023-07-21T19:07:27.569 INFO:teuthology.orchestra.run.smithi193.stdout: "cap+": 33474, 2023-07-21T19:07:27.569 INFO:teuthology.orchestra.run.smithi193.stdout: "cap-": 33474, 2023-07-21T19:07:27.569 INFO:teuthology.orchestra.run.smithi193.stdout: "rss": 82184, 2023-07-21T19:07:27.569 INFO:teuthology.orchestra.run.smithi193.stdout: "heap": 198964 2023-07-21T19:07:27.569 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:27.569 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_server": { 2023-07-21T19:07:27.569 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_client_request": 392563, 2023-07-21T19:07:27.569 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_server_request": 20739, 2023-07-21T19:07:27.570 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_request": 268427, 2023-07-21T19:07:27.570 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_session": 3184, 2023-07-21T19:07:27.570 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_peer_request": 52158, 2023-07-21T19:07:27.570 INFO:teuthology.orchestra.run.smithi193.stdout: "req_create_latency": { 2023-07-21T19:07:27.570 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 30179, 2023-07-21T19:07:27.570 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 139.086607098, 2023-07-21T19:07:27.570 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.004608721 2023-07-21T19:07:27.571 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:27.571 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getattr_latency": { 2023-07-21T19:07:27.571 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 8224, 2023-07-21T19:07:27.571 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 152.825495349, 2023-07-21T19:07:27.571 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.018582866 2023-07-21T19:07:27.571 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:27.571 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getfilelock_latency": { 2023-07-21T19:07:27.572 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:27.572 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:27.572 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:27.572 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:27.572 INFO:teuthology.orchestra.run.smithi193.stdout: "req_link_latency": { 2023-07-21T19:07:27.572 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:27.573 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:27.573 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:27.573 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:27.573 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookup_latency": { 2023-07-21T19:07:27.573 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 56573, 2023-07-21T19:07:27.573 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 155.577460790, 2023-07-21T19:07:27.573 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.002750030 2023-07-21T19:07:27.573 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:27.574 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookuphash_latency": { 2023-07-21T19:07:27.574 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:27.574 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:27.574 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:27.574 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:27.574 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupino_latency": { 2023-07-21T19:07:27.574 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:27.574 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:27.575 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:27.575 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:27.575 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupname_latency": { 2023-07-21T19:07:27.575 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:27.575 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:27.575 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:27.575 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:27.575 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupparent_latency": { 2023-07-21T19:07:27.576 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:27.576 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:27.576 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:27.576 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:27.576 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupsnap_latency": { 2023-07-21T19:07:27.576 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:27.577 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:27.577 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:27.577 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:27.577 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lssnap_latency": { 2023-07-21T19:07:27.577 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:27.577 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:27.578 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:27.578 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:27.578 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mkdir_latency": { 2023-07-21T19:07:27.578 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:27.578 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:27.578 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:27.578 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:27.578 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mknod_latency": { 2023-07-21T19:07:27.579 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:27.579 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:27.579 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:27.579 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:27.579 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mksnap_latency": { 2023-07-21T19:07:27.579 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:27.579 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:27.579 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:27.580 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:27.580 INFO:teuthology.orchestra.run.smithi193.stdout: "req_open_latency": { 2023-07-21T19:07:27.580 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:27.580 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:27.580 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:27.580 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:27.580 INFO:teuthology.orchestra.run.smithi193.stdout: "req_readdir_latency": { 2023-07-21T19:07:27.580 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 90869, 2023-07-21T19:07:27.581 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 291.186269539, 2023-07-21T19:07:27.581 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.003204462 2023-07-21T19:07:27.581 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:27.581 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rename_latency": { 2023-07-21T19:07:27.581 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 5378, 2023-07-21T19:07:27.581 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 34.104285415, 2023-07-21T19:07:27.581 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.006341443 2023-07-21T19:07:27.582 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:27.582 INFO:teuthology.orchestra.run.smithi193.stdout: "req_renamesnap_latency": { 2023-07-21T19:07:27.582 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:27.582 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:27.582 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:27.582 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:27.582 INFO:teuthology.orchestra.run.smithi193.stdout: "req_snapdiff_latency": { 2023-07-21T19:07:27.582 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:27.583 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:27.583 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:27.583 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:27.583 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmdir_latency": { 2023-07-21T19:07:27.583 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 20, 2023-07-21T19:07:27.583 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.222838458, 2023-07-21T19:07:27.583 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.011141922 2023-07-21T19:07:27.583 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:27.584 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmsnap_latency": { 2023-07-21T19:07:27.584 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:27.584 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:27.584 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:27.584 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:27.584 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmxattr_latency": { 2023-07-21T19:07:27.584 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:27.584 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:27.585 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:27.585 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:27.585 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setattr_latency": { 2023-07-21T19:07:27.585 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 41, 2023-07-21T19:07:27.585 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.737286814, 2023-07-21T19:07:27.585 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.017982605 2023-07-21T19:07:27.585 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:27.585 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setdirlayout_latency": { 2023-07-21T19:07:27.586 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:27.586 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:27.586 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:27.586 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:27.586 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setfilelock_latency": { 2023-07-21T19:07:27.586 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 872, 2023-07-21T19:07:27.586 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.943602301, 2023-07-21T19:07:27.587 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.001082112 2023-07-21T19:07:27.587 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:27.587 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setlayout_latency": { 2023-07-21T19:07:27.587 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:27.587 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:27.587 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:27.587 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:27.587 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setxattr_latency": { 2023-07-21T19:07:27.588 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:27.588 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:27.588 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:27.588 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:27.588 INFO:teuthology.orchestra.run.smithi193.stdout: "req_symlink_latency": { 2023-07-21T19:07:27.588 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:27.588 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:27.588 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:27.589 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:27.589 INFO:teuthology.orchestra.run.smithi193.stdout: "req_unlink_latency": { 2023-07-21T19:07:27.589 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 30230, 2023-07-21T19:07:27.589 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 91.171387282, 2023-07-21T19:07:27.589 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.003015924 2023-07-21T19:07:27.589 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:27.589 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_revoke_eviction": 0, 2023-07-21T19:07:27.589 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_acquisition_throttle": 0, 2023-07-21T19:07:27.590 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getvxattr_latency": { 2023-07-21T19:07:27.590 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:27.590 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:27.590 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:27.590 INFO:teuthology.orchestra.run.smithi193.stdout: } 2023-07-21T19:07:27.590 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:27.590 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_sessions": { 2023-07-21T19:07:27.591 INFO:teuthology.orchestra.run.smithi193.stdout: "session_count": 1, 2023-07-21T19:07:27.591 INFO:teuthology.orchestra.run.smithi193.stdout: "session_add": 2, 2023-07-21T19:07:27.591 INFO:teuthology.orchestra.run.smithi193.stdout: "session_remove": 1, 2023-07-21T19:07:27.591 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_open": 1, 2023-07-21T19:07:27.591 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_stale": 0, 2023-07-21T19:07:27.591 INFO:teuthology.orchestra.run.smithi193.stdout: "total_load": 19695, 2023-07-21T19:07:27.591 INFO:teuthology.orchestra.run.smithi193.stdout: "average_load": 19695, 2023-07-21T19:07:27.591 INFO:teuthology.orchestra.run.smithi193.stdout: "avg_session_uptime": 1479 2023-07-21T19:07:27.592 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:27.592 INFO:teuthology.orchestra.run.smithi193.stdout: "mempool": { 2023-07-21T19:07:27.592 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_bytes": 0, 2023-07-21T19:07:27.592 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_items": 0, 2023-07-21T19:07:27.592 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_bytes": 0, 2023-07-21T19:07:27.592 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_items": 0, 2023-07-21T19:07:27.592 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_bytes": 0, 2023-07-21T19:07:27.593 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_items": 0, 2023-07-21T19:07:27.593 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-21T19:07:27.593 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_items": 0, 2023-07-21T19:07:27.593 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-21T19:07:27.593 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_items": 0, 2023-07-21T19:07:27.593 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_bytes": 0, 2023-07-21T19:07:27.593 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_items": 0, 2023-07-21T19:07:27.594 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-21T19:07:27.594 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_items": 0, 2023-07-21T19:07:27.594 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_bytes": 0, 2023-07-21T19:07:27.594 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_items": 0, 2023-07-21T19:07:27.594 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_bytes": 0, 2023-07-21T19:07:27.594 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_items": 0, 2023-07-21T19:07:27.594 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-21T19:07:27.594 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_items": 0, 2023-07-21T19:07:27.595 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-21T19:07:27.595 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_items": 0, 2023-07-21T19:07:27.595 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_bytes": 0, 2023-07-21T19:07:27.595 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_items": 0, 2023-07-21T19:07:27.595 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_bytes": 0, 2023-07-21T19:07:27.595 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_items": 0, 2023-07-21T19:07:27.595 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-21T19:07:27.596 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_items": 0, 2023-07-21T19:07:27.596 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_bytes": 0, 2023-07-21T19:07:27.596 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_items": 0, 2023-07-21T19:07:27.596 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_bytes": 0, 2023-07-21T19:07:27.596 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_items": 0, 2023-07-21T19:07:27.597 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_bytes": 0, 2023-07-21T19:07:27.597 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_items": 0, 2023-07-21T19:07:27.597 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_bytes": 0, 2023-07-21T19:07:27.597 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_items": 0, 2023-07-21T19:07:27.597 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_bytes": 476268, 2023-07-21T19:07:27.597 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_items": 664, 2023-07-21T19:07:27.597 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_bytes": 88, 2023-07-21T19:07:27.598 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_items": 1, 2023-07-21T19:07:27.598 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_bytes": 0, 2023-07-21T19:07:27.598 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_items": 0, 2023-07-21T19:07:27.598 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_bytes": 0, 2023-07-21T19:07:27.598 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_items": 0, 2023-07-21T19:07:27.598 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_bytes": 0, 2023-07-21T19:07:27.598 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_items": 0, 2023-07-21T19:07:27.598 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_bytes": 8160, 2023-07-21T19:07:27.599 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_items": 186, 2023-07-21T19:07:27.599 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_bytes": 0, 2023-07-21T19:07:27.599 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_items": 0, 2023-07-21T19:07:27.599 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_bytes": 0, 2023-07-21T19:07:27.599 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_items": 0, 2023-07-21T19:07:27.599 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_bytes": 4725089, 2023-07-21T19:07:27.599 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_items": 37378, 2023-07-21T19:07:27.599 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_bytes": 0, 2023-07-21T19:07:27.600 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_items": 0, 2023-07-21T19:07:27.600 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_bytes": 0, 2023-07-21T19:07:27.600 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_items": 0 2023-07-21T19:07:27.600 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:27.600 INFO:teuthology.orchestra.run.smithi193.stdout: "objecter": { 2023-07-21T19:07:27.600 INFO:teuthology.orchestra.run.smithi193.stdout: "op_active": 0, 2023-07-21T19:07:27.600 INFO:teuthology.orchestra.run.smithi193.stdout: "op_laggy": 0, 2023-07-21T19:07:27.600 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send": 82465, 2023-07-21T19:07:27.601 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send_bytes": 915563521, 2023-07-21T19:07:27.601 INFO:teuthology.orchestra.run.smithi193.stdout: "op_resend": 28, 2023-07-21T19:07:27.601 INFO:teuthology.orchestra.run.smithi193.stdout: "op_reply": 82465, 2023-07-21T19:07:27.601 INFO:teuthology.orchestra.run.smithi193.stdout: "op_latency": { 2023-07-21T19:07:27.601 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 82465, 2023-07-21T19:07:27.601 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 679.137540042, 2023-07-21T19:07:27.601 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.008235464 2023-07-21T19:07:27.601 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:27.602 INFO:teuthology.orchestra.run.smithi193.stdout: "op_inflight": 0, 2023-07-21T19:07:27.602 INFO:teuthology.orchestra.run.smithi193.stdout: "oplen_avg": { 2023-07-21T19:07:27.602 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 82465, 2023-07-21T19:07:27.602 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 83213 2023-07-21T19:07:27.602 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:27.602 INFO:teuthology.orchestra.run.smithi193.stdout: "op": 82465, 2023-07-21T19:07:27.602 INFO:teuthology.orchestra.run.smithi193.stdout: "op_r": 1227, 2023-07-21T19:07:27.603 INFO:teuthology.orchestra.run.smithi193.stdout: "op_w": 81238, 2023-07-21T19:07:27.603 INFO:teuthology.orchestra.run.smithi193.stdout: "op_rmw": 0, 2023-07-21T19:07:27.603 INFO:teuthology.orchestra.run.smithi193.stdout: "op_pg": 0, 2023-07-21T19:07:27.603 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_stat": 29, 2023-07-21T19:07:27.603 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_create": 21, 2023-07-21T19:07:27.603 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_read": 1201, 2023-07-21T19:07:27.603 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_write": 49521, 2023-07-21T19:07:27.603 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writefull": 740, 2023-07-21T19:07:27.604 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writesame": 0, 2023-07-21T19:07:27.604 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_append": 0, 2023-07-21T19:07:27.604 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_zero": 0, 2023-07-21T19:07:27.604 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_truncate": 6, 2023-07-21T19:07:27.604 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_delete": 30571, 2023-07-21T19:07:27.604 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_mapext": 0, 2023-07-21T19:07:27.604 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_sparse_read": 0, 2023-07-21T19:07:27.605 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_clonerange": 0, 2023-07-21T19:07:27.605 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_getxattr": 25, 2023-07-21T19:07:27.605 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_setxattr": 42, 2023-07-21T19:07:27.605 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_cmpxattr": 0, 2023-07-21T19:07:27.605 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_rmxattr": 0, 2023-07-21T19:07:27.605 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_resetxattrs": 0, 2023-07-21T19:07:27.605 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_call": 0, 2023-07-21T19:07:27.605 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_watch": 0, 2023-07-21T19:07:27.606 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_notify": 0, 2023-07-21T19:07:27.606 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_src_cmpxattr": 0, 2023-07-21T19:07:27.606 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls": 0, 2023-07-21T19:07:27.606 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls_filter": 0, 2023-07-21T19:07:27.606 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_other": 0, 2023-07-21T19:07:27.606 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_active": 0, 2023-07-21T19:07:27.606 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_send": 0, 2023-07-21T19:07:27.606 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_resend": 0, 2023-07-21T19:07:27.607 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_ping": 0, 2023-07-21T19:07:27.607 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_active": 0, 2023-07-21T19:07:27.607 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_send": 0, 2023-07-21T19:07:27.607 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_resend": 0, 2023-07-21T19:07:27.607 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_active": 0, 2023-07-21T19:07:27.607 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_send": 0, 2023-07-21T19:07:27.608 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_resend": 0, 2023-07-21T19:07:27.608 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_active": 0, 2023-07-21T19:07:27.608 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_send": 0, 2023-07-21T19:07:27.608 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_resend": 0, 2023-07-21T19:07:27.608 INFO:teuthology.orchestra.run.smithi193.stdout: "command_active": 0, 2023-07-21T19:07:27.608 INFO:teuthology.orchestra.run.smithi193.stdout: "command_send": 0, 2023-07-21T19:07:27.608 INFO:teuthology.orchestra.run.smithi193.stdout: "command_resend": 0, 2023-07-21T19:07:27.608 INFO:teuthology.orchestra.run.smithi193.stdout: "map_epoch": 124, 2023-07-21T19:07:27.609 INFO:teuthology.orchestra.run.smithi193.stdout: "map_full": 0, 2023-07-21T19:07:27.609 INFO:teuthology.orchestra.run.smithi193.stdout: "map_inc": 45, 2023-07-21T19:07:27.609 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_sessions": 12, 2023-07-21T19:07:27.609 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_open": 12, 2023-07-21T19:07:27.609 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_close": 0, 2023-07-21T19:07:27.609 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_laggy": 0, 2023-07-21T19:07:27.609 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_wr": 737, 2023-07-21T19:07:27.610 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_rd": 5, 2023-07-21T19:07:27.610 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_del": 315 2023-07-21T19:07:27.610 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:27.610 INFO:teuthology.orchestra.run.smithi193.stdout: "oft": { 2023-07-21T19:07:27.610 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_objs": 1, 2023-07-21T19:07:27.610 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_kv_pairs": 0, 2023-07-21T19:07:27.610 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_updates": 0, 2023-07-21T19:07:27.610 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_removes": 0 2023-07-21T19:07:27.611 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:27.611 INFO:teuthology.orchestra.run.smithi193.stdout: "osd-slow-ops": { 2023-07-21T19:07:27.611 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_ops_count": 288318 2023-07-21T19:07:27.611 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:27.611 INFO:teuthology.orchestra.run.smithi193.stdout: "purge_queue": { 2023-07-21T19:07:27.611 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops": 0, 2023-07-21T19:07:27.611 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops_high_water": 6, 2023-07-21T19:07:27.612 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing": 0, 2023-07-21T19:07:27.612 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_high_water": 6, 2023-07-21T19:07:27.612 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed": 30230, 2023-07-21T19:07:27.612 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_item_in_journal": 0 2023-07-21T19:07:27.612 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:27.612 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-21T19:07:27.612 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2023-07-21T19:07:27.612 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2023-07-21T19:07:27.613 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2023-07-21T19:07:27.613 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 1365214, 2023-07-21T19:07:27.613 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 276153774, 2023-07-21T19:07:27.613 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2023-07-21T19:07:27.613 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 1365214, 2023-07-21T19:07:27.613 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2023-07-21T19:07:27.613 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2023-07-21T19:07:27.614 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 1365214, 2023-07-21T19:07:27.614 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 276153774, 2023-07-21T19:07:27.614 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2023-07-21T19:07:27.614 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:27.614 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:27.614 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:27.614 INFO:teuthology.orchestra.run.smithi193.stdout: } 2023-07-21T19:07:27.614 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:27.615 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_bytes": { 2023-07-21T19:07:27.615 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2023-07-21T19:07:27.615 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2023-07-21T19:07:27.615 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2023-07-21T19:07:27.615 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2023-07-21T19:07:27.615 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2023-07-21T19:07:27.615 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2023-07-21T19:07:27.615 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2023-07-21T19:07:27.616 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 82465, 2023-07-21T19:07:27.616 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 918624665, 2023-07-21T19:07:27.616 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 82465, 2023-07-21T19:07:27.616 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 918624665, 2023-07-21T19:07:27.616 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2023-07-21T19:07:27.616 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:27.616 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:27.617 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:27.617 INFO:teuthology.orchestra.run.smithi193.stdout: } 2023-07-21T19:07:27.617 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:27.617 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_ops": { 2023-07-21T19:07:27.617 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2023-07-21T19:07:27.617 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 1024, 2023-07-21T19:07:27.617 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2023-07-21T19:07:27.617 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2023-07-21T19:07:27.618 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2023-07-21T19:07:27.618 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2023-07-21T19:07:27.618 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2023-07-21T19:07:27.618 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 82465, 2023-07-21T19:07:27.618 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 82465, 2023-07-21T19:07:27.618 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 82465, 2023-07-21T19:07:27.618 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 82465, 2023-07-21T19:07:27.618 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2023-07-21T19:07:27.619 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:27.619 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:27.619 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:27.619 INFO:teuthology.orchestra.run.smithi193.stdout: } 2023-07-21T19:07:27.619 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:27.619 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle": { 2023-07-21T19:07:27.619 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2023-07-21T19:07:27.619 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2023-07-21T19:07:27.620 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2023-07-21T19:07:27.620 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 30230, 2023-07-21T19:07:27.620 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 3061206, 2023-07-21T19:07:27.620 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2023-07-21T19:07:27.620 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 30230, 2023-07-21T19:07:27.620 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2023-07-21T19:07:27.620 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2023-07-21T19:07:27.620 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 1202, 2023-07-21T19:07:27.621 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 3061206, 2023-07-21T19:07:27.621 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2023-07-21T19:07:27.621 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:27.621 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:27.621 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:27.621 INFO:teuthology.orchestra.run.smithi193.stdout: } 2023-07-21T19:07:27.621 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:27.622 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle-0x55b5653e20a0": { 2023-07-21T19:07:27.622 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2023-07-21T19:07:27.622 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2023-07-21T19:07:27.622 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2023-07-21T19:07:27.622 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 148303, 2023-07-21T19:07:27.622 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 909624050, 2023-07-21T19:07:27.622 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2023-07-21T19:07:27.622 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 148303, 2023-07-21T19:07:27.623 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2023-07-21T19:07:27.623 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2023-07-21T19:07:27.623 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 48319, 2023-07-21T19:07:27.623 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 909624050, 2023-07-21T19:07:27.623 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2023-07-21T19:07:27.623 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:27.623 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:27.623 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:27.624 INFO:teuthology.orchestra.run.smithi193.stdout: } 2023-07-21T19:07:27.624 INFO:teuthology.orchestra.run.smithi193.stdout: } 2023-07-21T19:07:27.624 INFO:teuthology.orchestra.run.smithi193.stdout:} 2023-07-21T19:07:27.893 INFO:tasks.check_counter:Daemon mds.i mds.exported=11 2023-07-21T19:07:27.893 INFO:tasks.check_counter:Daemon mds.i mds.imported=11 2023-07-21T19:07:27.894 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=603 2023-07-21T19:07:27.894 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=6156 2023-07-21T19:07:27.894 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=7 2023-07-21T19:07:27.894 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2023-07-21T19:07:27.894 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2023-07-21T19:07:27.894 DEBUG:tasks.check_counter:Getting stats from l 2023-07-21T19:07:27.895 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c -- ceph daemon mds.l perf dump 2023-07-21T19:07:28.449 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T19:07:28.829 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:28 smithi193 ceph-mon[98204]: pgmap v816: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.5 KiB/s wr, 16 op/s 2023-07-21T19:07:28.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:28 smithi155 ceph-mon[97934]: pgmap v816: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.5 KiB/s wr, 16 op/s 2023-07-21T19:07:28.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:28 smithi039 ceph-mon[96489]: pgmap v816: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.5 KiB/s wr, 16 op/s 2023-07-21T19:07:29.661 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2023-07-21T19:07:29.662 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-0": { 2023-07-21T19:07:29.662 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 3215, 2023-07-21T19:07:29.662 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 3215, 2023-07-21T19:07:29.662 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 357485844, 2023-07-21T19:07:29.662 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 999070, 2023-07-21T19:07:29.662 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 8, 2023-07-21T19:07:29.663 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 4, 2023-07-21T19:07:29.663 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.430708649, 2023-07-21T19:07:29.663 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.073441642, 2023-07-21T19:07:29.663 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.233974206, 2023-07-21T19:07:29.663 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.098234747, 2023-07-21T19:07:29.663 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:07:29.663 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 3214, 2023-07-21T19:07:29.663 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.250442646, 2023-07-21T19:07:29.664 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000077922 2023-07-21T19:07:29.664 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:29.664 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:07:29.664 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:29.664 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:29.664 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:29.664 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:29.665 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-07-21T19:07:29.665 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 448 2023-07-21T19:07:29.665 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:29.665 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-1": { 2023-07-21T19:07:29.665 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 2283, 2023-07-21T19:07:29.666 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 2620, 2023-07-21T19:07:29.666 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 380983875, 2023-07-21T19:07:29.666 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 1444083, 2023-07-21T19:07:29.666 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 8, 2023-07-21T19:07:29.666 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 6, 2023-07-21T19:07:29.666 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.325191236, 2023-07-21T19:07:29.666 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.061056176, 2023-07-21T19:07:29.666 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.189419220, 2023-07-21T19:07:29.667 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.058784912, 2023-07-21T19:07:29.667 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:07:29.667 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 2617, 2023-07-21T19:07:29.667 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.152068633, 2023-07-21T19:07:29.667 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000058107 2023-07-21T19:07:29.667 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:29.667 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:07:29.667 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:29.668 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:29.668 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:29.668 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:29.668 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 320, 2023-07-21T19:07:29.668 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 734624 2023-07-21T19:07:29.668 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:29.668 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-2": { 2023-07-21T19:07:29.668 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 1545, 2023-07-21T19:07:29.669 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 1449, 2023-07-21T19:07:29.669 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 266426454, 2023-07-21T19:07:29.669 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 560445, 2023-07-21T19:07:29.669 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 5, 2023-07-21T19:07:29.669 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 4, 2023-07-21T19:07:29.669 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.279788283, 2023-07-21T19:07:29.669 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.040763269, 2023-07-21T19:07:29.669 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.204608355, 2023-07-21T19:07:29.670 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.035330939, 2023-07-21T19:07:29.670 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:07:29.670 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 1448, 2023-07-21T19:07:29.670 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.255595369, 2023-07-21T19:07:29.670 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000176516 2023-07-21T19:07:29.670 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:29.670 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:07:29.671 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:29.671 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:29.671 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:29.671 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:29.671 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 1926128, 2023-07-21T19:07:29.671 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 263440 2023-07-21T19:07:29.671 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:29.671 INFO:teuthology.orchestra.run.smithi193.stdout: "cct": { 2023-07-21T19:07:29.672 INFO:teuthology.orchestra.run.smithi193.stdout: "total_workers": 1, 2023-07-21T19:07:29.672 INFO:teuthology.orchestra.run.smithi193.stdout: "unhealthy_workers": 0 2023-07-21T19:07:29.672 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:29.672 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-MDSRank": { 2023-07-21T19:07:29.672 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2023-07-21T19:07:29.672 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2023-07-21T19:07:29.672 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 10676, 2023-07-21T19:07:29.672 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.426917064, 2023-07-21T19:07:29.673 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000039988 2023-07-21T19:07:29.673 INFO:teuthology.orchestra.run.smithi193.stdout: } 2023-07-21T19:07:29.673 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:29.673 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-PurgeQueue": { 2023-07-21T19:07:29.673 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2023-07-21T19:07:29.673 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2023-07-21T19:07:29.673 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:29.673 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:29.674 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:29.674 INFO:teuthology.orchestra.run.smithi193.stdout: } 2023-07-21T19:07:29.674 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:29.674 INFO:teuthology.orchestra.run.smithi193.stdout: "mds": { 2023-07-21T19:07:29.674 INFO:teuthology.orchestra.run.smithi193.stdout: "request": 0, 2023-07-21T19:07:29.674 INFO:teuthology.orchestra.run.smithi193.stdout: "reply": 0, 2023-07-21T19:07:29.674 INFO:teuthology.orchestra.run.smithi193.stdout: "reply_latency": { 2023-07-21T19:07:29.674 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:29.675 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:29.675 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:29.675 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:29.675 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_reply": 0, 2023-07-21T19:07:29.675 INFO:teuthology.orchestra.run.smithi193.stdout: "forward": 0, 2023-07-21T19:07:29.675 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_complete": 0, 2023-07-21T19:07:29.675 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_keys": 0, 2023-07-21T19:07:29.675 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_commit": 0, 2023-07-21T19:07:29.676 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_split": 0, 2023-07-21T19:07:29.676 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_merge": 0, 2023-07-21T19:07:29.676 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes": 22019, 2023-07-21T19:07:29.676 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_top": 6402, 2023-07-21T19:07:29.676 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_bottom": 15607, 2023-07-21T19:07:29.676 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pin_tail": 10, 2023-07-21T19:07:29.676 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pinned": 12873, 2023-07-21T19:07:29.676 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_expired": 11785, 2023-07-21T19:07:29.677 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_with_caps": 0, 2023-07-21T19:07:29.677 INFO:teuthology.orchestra.run.smithi193.stdout: "caps": 0, 2023-07-21T19:07:29.677 INFO:teuthology.orchestra.run.smithi193.stdout: "subtrees": 1, 2023-07-21T19:07:29.677 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse": 0, 2023-07-21T19:07:29.677 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_hit": 0, 2023-07-21T19:07:29.677 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_forward": 0, 2023-07-21T19:07:29.677 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_discover": 0, 2023-07-21T19:07:29.677 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_dir_fetch": 0, 2023-07-21T19:07:29.678 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_remote_ino": 0, 2023-07-21T19:07:29.678 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_lock": 0, 2023-07-21T19:07:29.678 INFO:teuthology.orchestra.run.smithi193.stdout: "load_cent": 0, 2023-07-21T19:07:29.678 INFO:teuthology.orchestra.run.smithi193.stdout: "q": 0, 2023-07-21T19:07:29.678 INFO:teuthology.orchestra.run.smithi193.stdout: "exported": 0, 2023-07-21T19:07:29.678 INFO:teuthology.orchestra.run.smithi193.stdout: "exported_inodes": 0, 2023-07-21T19:07:29.678 INFO:teuthology.orchestra.run.smithi193.stdout: "imported": 0, 2023-07-21T19:07:29.678 INFO:teuthology.orchestra.run.smithi193.stdout: "imported_inodes": 0, 2023-07-21T19:07:29.679 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_dir_fetch": 0, 2023-07-21T19:07:29.679 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_backtrace_fetch": 0, 2023-07-21T19:07:29.679 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_peer_discover": 0, 2023-07-21T19:07:29.679 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rfiles": 0, 2023-07-21T19:07:29.679 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rbytes": 0, 2023-07-21T19:07:29.679 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rsnaps": 0, 2023-07-21T19:07:29.679 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_fetch": 0, 2023-07-21T19:07:29.679 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_set_tag": 0, 2023-07-21T19:07:29.680 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_repaired": 0, 2023-07-21T19:07:29.680 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_inotable_repaired": 0, 2023-07-21T19:07:29.680 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_inodes": 0, 2023-07-21T19:07:29.680 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_base_inodes": 0, 2023-07-21T19:07:29.680 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dirfrag_rstats": 0, 2023-07-21T19:07:29.680 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_file_inodes": 0, 2023-07-21T19:07:29.680 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_inode_file_caps": 0, 2023-07-21T19:07:29.681 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_revoke": 0, 2023-07-21T19:07:29.681 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_grant": 0, 2023-07-21T19:07:29.681 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_trunc": 0, 2023-07-21T19:07:29.681 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-21T19:07:29.681 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-21T19:07:29.681 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps": 0, 2023-07-21T19:07:29.681 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps_dirty": 0, 2023-07-21T19:07:29.681 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_cap_release": 0, 2023-07-21T19:07:29.682 INFO:teuthology.orchestra.run.smithi193.stdout: "process_request_cap_release": 0 2023-07-21T19:07:29.682 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:29.682 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_cache": { 2023-07-21T19:07:29.682 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update": 0, 2023-07-21T19:07:29.682 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update_receipt": 0, 2023-07-21T19:07:29.682 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_try_discover": 0, 2023-07-21T19:07:29.682 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_send_discover": 0, 2023-07-21T19:07:29.682 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_handle_discover": 0, 2023-07-21T19:07:29.683 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays": 0, 2023-07-21T19:07:29.683 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_delayed": 0, 2023-07-21T19:07:29.683 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_enqueuing": 0, 2023-07-21T19:07:29.683 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_created": 0, 2023-07-21T19:07:29.683 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_enqueued": 0, 2023-07-21T19:07:29.683 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_reintegrated": 0, 2023-07-21T19:07:29.683 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_migrated": 0, 2023-07-21T19:07:29.683 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_processing": 0, 2023-07-21T19:07:29.684 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_enqueued": 0, 2023-07-21T19:07:29.684 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_prioritized": 0, 2023-07-21T19:07:29.684 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_started": 0, 2023-07-21T19:07:29.684 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_completed": 0, 2023-07-21T19:07:29.684 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_enqueue_scrub": 0, 2023-07-21T19:07:29.684 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_exportdir": 0, 2023-07-21T19:07:29.684 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_flush": 0, 2023-07-21T19:07:29.684 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragmentdir": 0, 2023-07-21T19:07:29.685 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragstats": 0, 2023-07-21T19:07:29.685 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_inodestats": 0 2023-07-21T19:07:29.685 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:29.685 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_log": { 2023-07-21T19:07:29.685 INFO:teuthology.orchestra.run.smithi193.stdout: "evadd": 0, 2023-07-21T19:07:29.685 INFO:teuthology.orchestra.run.smithi193.stdout: "evex": 0, 2023-07-21T19:07:29.685 INFO:teuthology.orchestra.run.smithi193.stdout: "evtrm": 0, 2023-07-21T19:07:29.685 INFO:teuthology.orchestra.run.smithi193.stdout: "ev": 0, 2023-07-21T19:07:29.686 INFO:teuthology.orchestra.run.smithi193.stdout: "evexg": 0, 2023-07-21T19:07:29.686 INFO:teuthology.orchestra.run.smithi193.stdout: "evexd": 0, 2023-07-21T19:07:29.686 INFO:teuthology.orchestra.run.smithi193.stdout: "segadd": 0, 2023-07-21T19:07:29.686 INFO:teuthology.orchestra.run.smithi193.stdout: "segex": 0, 2023-07-21T19:07:29.686 INFO:teuthology.orchestra.run.smithi193.stdout: "segtrm": 0, 2023-07-21T19:07:29.686 INFO:teuthology.orchestra.run.smithi193.stdout: "seg": 128, 2023-07-21T19:07:29.686 INFO:teuthology.orchestra.run.smithi193.stdout: "segexg": 0, 2023-07-21T19:07:29.686 INFO:teuthology.orchestra.run.smithi193.stdout: "segexd": 0, 2023-07-21T19:07:29.687 INFO:teuthology.orchestra.run.smithi193.stdout: "expos": 473973044, 2023-07-21T19:07:29.687 INFO:teuthology.orchestra.run.smithi193.stdout: "wrpos": 0, 2023-07-21T19:07:29.687 INFO:teuthology.orchestra.run.smithi193.stdout: "rdpos": 1005303249, 2023-07-21T19:07:29.687 INFO:teuthology.orchestra.run.smithi193.stdout: "jlat": { 2023-07-21T19:07:29.687 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:29.687 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:29.687 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:29.687 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:29.688 INFO:teuthology.orchestra.run.smithi193.stdout: "replayed": 167601 2023-07-21T19:07:29.688 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:29.692 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_mem": { 2023-07-21T19:07:29.692 INFO:teuthology.orchestra.run.smithi193.stdout: "ino": 1236, 2023-07-21T19:07:29.693 INFO:teuthology.orchestra.run.smithi193.stdout: "ino+": 34104, 2023-07-21T19:07:29.693 INFO:teuthology.orchestra.run.smithi193.stdout: "ino-": 32868, 2023-07-21T19:07:29.693 INFO:teuthology.orchestra.run.smithi193.stdout: "dir": 45, 2023-07-21T19:07:29.694 INFO:teuthology.orchestra.run.smithi193.stdout: "dir+": 140, 2023-07-21T19:07:29.694 INFO:teuthology.orchestra.run.smithi193.stdout: "dir-": 95, 2023-07-21T19:07:29.694 INFO:teuthology.orchestra.run.smithi193.stdout: "dn": 22019, 2023-07-21T19:07:29.694 INFO:teuthology.orchestra.run.smithi193.stdout: "dn+": 34470, 2023-07-21T19:07:29.695 INFO:teuthology.orchestra.run.smithi193.stdout: "dn-": 12451, 2023-07-21T19:07:29.695 INFO:teuthology.orchestra.run.smithi193.stdout: "cap": 0, 2023-07-21T19:07:29.695 INFO:teuthology.orchestra.run.smithi193.stdout: "cap+": 0, 2023-07-21T19:07:29.696 INFO:teuthology.orchestra.run.smithi193.stdout: "cap-": 0, 2023-07-21T19:07:29.696 INFO:teuthology.orchestra.run.smithi193.stdout: "rss": 79404, 2023-07-21T19:07:29.696 INFO:teuthology.orchestra.run.smithi193.stdout: "heap": 207156 2023-07-21T19:07:29.696 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:29.697 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_server": { 2023-07-21T19:07:29.697 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_client_request": 0, 2023-07-21T19:07:29.697 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_server_request": 0, 2023-07-21T19:07:29.697 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_request": 0, 2023-07-21T19:07:29.697 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_session": 0, 2023-07-21T19:07:29.698 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_peer_request": 0, 2023-07-21T19:07:29.698 INFO:teuthology.orchestra.run.smithi193.stdout: "req_create_latency": { 2023-07-21T19:07:29.698 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:29.698 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:29.698 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:29.699 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:29.699 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getattr_latency": { 2023-07-21T19:07:29.699 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:29.699 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:29.700 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:29.700 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:29.700 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getfilelock_latency": { 2023-07-21T19:07:29.700 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:29.701 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:29.701 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:29.701 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:29.701 INFO:teuthology.orchestra.run.smithi193.stdout: "req_link_latency": { 2023-07-21T19:07:29.702 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:29.702 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:29.702 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:29.702 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:29.703 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookup_latency": { 2023-07-21T19:07:29.703 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:29.703 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:29.703 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:29.704 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:29.704 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookuphash_latency": { 2023-07-21T19:07:29.704 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:29.704 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:29.705 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:29.705 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:29.705 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupino_latency": { 2023-07-21T19:07:29.706 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:29.706 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:29.706 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:29.706 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:29.707 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupname_latency": { 2023-07-21T19:07:29.707 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:29.707 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:29.707 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:29.708 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:29.708 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupparent_latency": { 2023-07-21T19:07:29.708 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:29.709 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:29.709 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:29.709 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:29.710 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupsnap_latency": { 2023-07-21T19:07:29.710 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:29.710 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:29.710 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:29.711 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:29.711 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lssnap_latency": { 2023-07-21T19:07:29.711 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:29.712 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:29.712 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:29.712 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:29.713 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mkdir_latency": { 2023-07-21T19:07:29.713 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:29.713 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:29.713 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:29.713 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:29.713 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mknod_latency": { 2023-07-21T19:07:29.713 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:29.714 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:29.714 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:29.714 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:29.714 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mksnap_latency": { 2023-07-21T19:07:29.714 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:29.714 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:29.714 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:29.714 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:29.715 INFO:teuthology.orchestra.run.smithi193.stdout: "req_open_latency": { 2023-07-21T19:07:29.715 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:29.715 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:29.715 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:29.715 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:29.715 INFO:teuthology.orchestra.run.smithi193.stdout: "req_readdir_latency": { 2023-07-21T19:07:29.715 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:29.715 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:29.716 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:29.716 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:29.716 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rename_latency": { 2023-07-21T19:07:29.716 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:29.716 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:29.716 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:29.716 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:29.717 INFO:teuthology.orchestra.run.smithi193.stdout: "req_renamesnap_latency": { 2023-07-21T19:07:29.717 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:29.717 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:29.717 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:29.717 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:29.718 INFO:teuthology.orchestra.run.smithi193.stdout: "req_snapdiff_latency": { 2023-07-21T19:07:29.718 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:29.718 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:29.718 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:29.718 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:29.718 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmdir_latency": { 2023-07-21T19:07:29.718 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:29.718 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:29.719 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:29.719 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:29.719 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmsnap_latency": { 2023-07-21T19:07:29.719 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:29.719 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:29.719 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:29.719 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:29.720 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmxattr_latency": { 2023-07-21T19:07:29.720 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:29.720 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:29.720 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:29.720 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:29.720 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setattr_latency": { 2023-07-21T19:07:29.720 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:29.720 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:29.721 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:29.721 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:29.721 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setdirlayout_latency": { 2023-07-21T19:07:29.721 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:29.721 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:29.721 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:29.721 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:29.721 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setfilelock_latency": { 2023-07-21T19:07:29.722 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:29.722 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:29.722 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:29.722 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:29.722 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setlayout_latency": { 2023-07-21T19:07:29.722 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:29.722 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:29.722 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:29.723 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:29.723 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setxattr_latency": { 2023-07-21T19:07:29.723 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:29.723 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:29.724 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:29.724 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:29.724 INFO:teuthology.orchestra.run.smithi193.stdout: "req_symlink_latency": { 2023-07-21T19:07:29.725 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:29.725 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:29.725 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:29.725 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:29.726 INFO:teuthology.orchestra.run.smithi193.stdout: "req_unlink_latency": { 2023-07-21T19:07:29.726 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:29.726 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:29.726 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:29.727 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:29.727 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_revoke_eviction": 0, 2023-07-21T19:07:29.727 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_acquisition_throttle": 0, 2023-07-21T19:07:29.727 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getvxattr_latency": { 2023-07-21T19:07:29.728 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:29.728 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:29.728 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:29.729 INFO:teuthology.orchestra.run.smithi193.stdout: } 2023-07-21T19:07:29.729 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:29.729 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_sessions": { 2023-07-21T19:07:29.729 INFO:teuthology.orchestra.run.smithi193.stdout: "session_count": 2, 2023-07-21T19:07:29.730 INFO:teuthology.orchestra.run.smithi193.stdout: "session_add": 3, 2023-07-21T19:07:29.730 INFO:teuthology.orchestra.run.smithi193.stdout: "session_remove": 1, 2023-07-21T19:07:29.730 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_open": 3, 2023-07-21T19:07:29.730 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_stale": 0, 2023-07-21T19:07:29.731 INFO:teuthology.orchestra.run.smithi193.stdout: "total_load": 0, 2023-07-21T19:07:29.731 INFO:teuthology.orchestra.run.smithi193.stdout: "average_load": 0, 2023-07-21T19:07:29.731 INFO:teuthology.orchestra.run.smithi193.stdout: "avg_session_uptime": 3763 2023-07-21T19:07:29.732 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:29.732 INFO:teuthology.orchestra.run.smithi193.stdout: "mempool": { 2023-07-21T19:07:29.732 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_bytes": 0, 2023-07-21T19:07:29.732 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_items": 0, 2023-07-21T19:07:29.732 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_bytes": 0, 2023-07-21T19:07:29.733 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_items": 0, 2023-07-21T19:07:29.733 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_bytes": 0, 2023-07-21T19:07:29.733 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_items": 0, 2023-07-21T19:07:29.733 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-21T19:07:29.734 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_items": 0, 2023-07-21T19:07:29.734 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-21T19:07:29.734 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_items": 0, 2023-07-21T19:07:29.734 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_bytes": 0, 2023-07-21T19:07:29.735 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_items": 0, 2023-07-21T19:07:29.735 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-21T19:07:29.735 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_items": 0, 2023-07-21T19:07:29.735 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_bytes": 0, 2023-07-21T19:07:29.735 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_items": 0, 2023-07-21T19:07:29.736 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_bytes": 0, 2023-07-21T19:07:29.736 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_items": 0, 2023-07-21T19:07:29.736 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-21T19:07:29.736 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_items": 0, 2023-07-21T19:07:29.737 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-21T19:07:29.737 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_items": 0, 2023-07-21T19:07:29.737 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_bytes": 0, 2023-07-21T19:07:29.737 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_items": 0, 2023-07-21T19:07:29.737 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_bytes": 0, 2023-07-21T19:07:29.738 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_items": 0, 2023-07-21T19:07:29.738 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-21T19:07:29.738 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_items": 0, 2023-07-21T19:07:29.738 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_bytes": 0, 2023-07-21T19:07:29.739 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_items": 0, 2023-07-21T19:07:29.739 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_bytes": 0, 2023-07-21T19:07:29.739 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_items": 0, 2023-07-21T19:07:29.739 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_bytes": 0, 2023-07-21T19:07:29.740 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_items": 0, 2023-07-21T19:07:29.740 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_bytes": 0, 2023-07-21T19:07:29.740 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_items": 0, 2023-07-21T19:07:29.740 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_bytes": 283833, 2023-07-21T19:07:29.741 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_items": 1576, 2023-07-21T19:07:29.741 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_bytes": 0, 2023-07-21T19:07:29.741 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_items": 0, 2023-07-21T19:07:29.741 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_bytes": 0, 2023-07-21T19:07:29.742 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_items": 0, 2023-07-21T19:07:29.742 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_bytes": 0, 2023-07-21T19:07:29.742 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_items": 0, 2023-07-21T19:07:29.742 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_bytes": 0, 2023-07-21T19:07:29.743 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_items": 0, 2023-07-21T19:07:29.743 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_bytes": 8160, 2023-07-21T19:07:29.743 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_items": 186, 2023-07-21T19:07:29.743 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_bytes": 0, 2023-07-21T19:07:29.743 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_items": 0, 2023-07-21T19:07:29.743 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_bytes": 0, 2023-07-21T19:07:29.744 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_items": 0, 2023-07-21T19:07:29.744 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_bytes": 18772382, 2023-07-21T19:07:29.744 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_items": 123346, 2023-07-21T19:07:29.744 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_bytes": 0, 2023-07-21T19:07:29.744 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_items": 0, 2023-07-21T19:07:29.744 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_bytes": 0, 2023-07-21T19:07:29.744 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_items": 0 2023-07-21T19:07:29.744 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:29.745 INFO:teuthology.orchestra.run.smithi193.stdout: "objecter": { 2023-07-21T19:07:29.745 INFO:teuthology.orchestra.run.smithi193.stdout: "op_active": 0, 2023-07-21T19:07:29.747 INFO:teuthology.orchestra.run.smithi193.stdout: "op_laggy": 0, 2023-07-21T19:07:29.747 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send": 6447, 2023-07-21T19:07:29.748 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send_bytes": 21, 2023-07-21T19:07:29.748 INFO:teuthology.orchestra.run.smithi193.stdout: "op_resend": 3, 2023-07-21T19:07:29.748 INFO:teuthology.orchestra.run.smithi193.stdout: "op_reply": 6447, 2023-07-21T19:07:29.748 INFO:teuthology.orchestra.run.smithi193.stdout: "op_latency": { 2023-07-21T19:07:29.748 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 6447, 2023-07-21T19:07:29.748 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 25.512058354, 2023-07-21T19:07:29.748 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.003957198 2023-07-21T19:07:29.749 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:29.749 INFO:teuthology.orchestra.run.smithi193.stdout: "op_inflight": 0, 2023-07-21T19:07:29.749 INFO:teuthology.orchestra.run.smithi193.stdout: "oplen_avg": { 2023-07-21T19:07:29.749 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 6447, 2023-07-21T19:07:29.749 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 6448 2023-07-21T19:07:29.749 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:29.749 INFO:teuthology.orchestra.run.smithi193.stdout: "op": 6447, 2023-07-21T19:07:29.749 INFO:teuthology.orchestra.run.smithi193.stdout: "op_r": 6447, 2023-07-21T19:07:29.750 INFO:teuthology.orchestra.run.smithi193.stdout: "op_w": 0, 2023-07-21T19:07:29.750 INFO:teuthology.orchestra.run.smithi193.stdout: "op_rmw": 0, 2023-07-21T19:07:29.750 INFO:teuthology.orchestra.run.smithi193.stdout: "op_pg": 0, 2023-07-21T19:07:29.750 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_stat": 3512, 2023-07-21T19:07:29.750 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_create": 0, 2023-07-21T19:07:29.750 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_read": 2933, 2023-07-21T19:07:29.750 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_write": 0, 2023-07-21T19:07:29.750 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writefull": 0, 2023-07-21T19:07:29.751 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writesame": 0, 2023-07-21T19:07:29.751 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_append": 0, 2023-07-21T19:07:29.751 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_zero": 0, 2023-07-21T19:07:29.751 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_truncate": 0, 2023-07-21T19:07:29.751 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_delete": 0, 2023-07-21T19:07:29.751 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_mapext": 0, 2023-07-21T19:07:29.751 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_sparse_read": 0, 2023-07-21T19:07:29.751 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_clonerange": 0, 2023-07-21T19:07:29.752 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_getxattr": 1, 2023-07-21T19:07:29.752 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_setxattr": 0, 2023-07-21T19:07:29.752 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_cmpxattr": 0, 2023-07-21T19:07:29.752 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_rmxattr": 0, 2023-07-21T19:07:29.752 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_resetxattrs": 0, 2023-07-21T19:07:29.752 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_call": 0, 2023-07-21T19:07:29.752 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_watch": 0, 2023-07-21T19:07:29.752 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_notify": 0, 2023-07-21T19:07:29.753 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_src_cmpxattr": 0, 2023-07-21T19:07:29.753 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls": 0, 2023-07-21T19:07:29.753 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls_filter": 0, 2023-07-21T19:07:29.753 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_other": 0, 2023-07-21T19:07:29.753 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_active": 0, 2023-07-21T19:07:29.753 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_send": 0, 2023-07-21T19:07:29.753 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_resend": 0, 2023-07-21T19:07:29.753 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_ping": 0, 2023-07-21T19:07:29.754 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_active": 0, 2023-07-21T19:07:29.754 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_send": 0, 2023-07-21T19:07:29.754 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_resend": 0, 2023-07-21T19:07:29.754 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_active": 0, 2023-07-21T19:07:29.754 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_send": 0, 2023-07-21T19:07:29.754 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_resend": 0, 2023-07-21T19:07:29.754 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_active": 0, 2023-07-21T19:07:29.754 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_send": 0, 2023-07-21T19:07:29.755 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_resend": 0, 2023-07-21T19:07:29.755 INFO:teuthology.orchestra.run.smithi193.stdout: "command_active": 0, 2023-07-21T19:07:29.755 INFO:teuthology.orchestra.run.smithi193.stdout: "command_send": 0, 2023-07-21T19:07:29.755 INFO:teuthology.orchestra.run.smithi193.stdout: "command_resend": 0, 2023-07-21T19:07:29.755 INFO:teuthology.orchestra.run.smithi193.stdout: "map_epoch": 124, 2023-07-21T19:07:29.755 INFO:teuthology.orchestra.run.smithi193.stdout: "map_full": 0, 2023-07-21T19:07:29.755 INFO:teuthology.orchestra.run.smithi193.stdout: "map_inc": 45, 2023-07-21T19:07:29.756 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_sessions": 10, 2023-07-21T19:07:29.756 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_open": 10, 2023-07-21T19:07:29.756 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_close": 0, 2023-07-21T19:07:29.756 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_laggy": 0, 2023-07-21T19:07:29.756 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_wr": 0, 2023-07-21T19:07:29.756 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_rd": 2, 2023-07-21T19:07:29.756 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_del": 0 2023-07-21T19:07:29.756 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:29.757 INFO:teuthology.orchestra.run.smithi193.stdout: "oft": { 2023-07-21T19:07:29.757 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_objs": 0, 2023-07-21T19:07:29.757 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_kv_pairs": 0, 2023-07-21T19:07:29.757 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_updates": 0, 2023-07-21T19:07:29.757 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_removes": 0 2023-07-21T19:07:29.757 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:29.757 INFO:teuthology.orchestra.run.smithi193.stdout: "osd-slow-ops": { 2023-07-21T19:07:29.757 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_ops_count": 0 2023-07-21T19:07:29.758 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:29.758 INFO:teuthology.orchestra.run.smithi193.stdout: "purge_queue": { 2023-07-21T19:07:29.758 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops": 0, 2023-07-21T19:07:29.758 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops_high_water": 0, 2023-07-21T19:07:29.758 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing": 0, 2023-07-21T19:07:29.758 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_high_water": 0, 2023-07-21T19:07:29.758 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed": 0, 2023-07-21T19:07:29.758 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_item_in_journal": 0 2023-07-21T19:07:29.759 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:29.759 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-21T19:07:29.759 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2023-07-21T19:07:29.759 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2023-07-21T19:07:29.759 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2023-07-21T19:07:29.759 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 7039, 2023-07-21T19:07:29.759 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 1004230005, 2023-07-21T19:07:29.759 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2023-07-21T19:07:29.760 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 7039, 2023-07-21T19:07:29.760 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2023-07-21T19:07:29.760 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2023-07-21T19:07:29.760 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 7039, 2023-07-21T19:07:29.760 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 1004230005, 2023-07-21T19:07:29.760 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2023-07-21T19:07:29.760 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:29.761 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:29.761 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:29.761 INFO:teuthology.orchestra.run.smithi193.stdout: } 2023-07-21T19:07:29.761 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:29.761 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_bytes": { 2023-07-21T19:07:29.761 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2023-07-21T19:07:29.761 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2023-07-21T19:07:29.761 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2023-07-21T19:07:29.762 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2023-07-21T19:07:29.762 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2023-07-21T19:07:29.762 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2023-07-21T19:07:29.762 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2023-07-21T19:07:29.762 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 6447, 2023-07-21T19:07:29.762 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 1001193889, 2023-07-21T19:07:29.762 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 6447, 2023-07-21T19:07:29.762 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 1001193889, 2023-07-21T19:07:29.763 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2023-07-21T19:07:29.763 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:29.763 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:29.763 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:29.763 INFO:teuthology.orchestra.run.smithi193.stdout: } 2023-07-21T19:07:29.763 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:29.763 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_ops": { 2023-07-21T19:07:29.763 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2023-07-21T19:07:29.764 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 1024, 2023-07-21T19:07:29.764 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2023-07-21T19:07:29.764 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2023-07-21T19:07:29.764 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2023-07-21T19:07:29.764 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2023-07-21T19:07:29.764 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2023-07-21T19:07:29.764 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 6447, 2023-07-21T19:07:29.764 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 6447, 2023-07-21T19:07:29.765 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 6447, 2023-07-21T19:07:29.765 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 6447, 2023-07-21T19:07:29.765 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2023-07-21T19:07:29.765 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:29.765 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:29.765 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:29.765 INFO:teuthology.orchestra.run.smithi193.stdout: } 2023-07-21T19:07:29.765 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:29.766 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle": { 2023-07-21T19:07:29.766 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2023-07-21T19:07:29.766 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2023-07-21T19:07:29.766 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2023-07-21T19:07:29.766 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2023-07-21T19:07:29.766 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2023-07-21T19:07:29.766 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2023-07-21T19:07:29.766 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2023-07-21T19:07:29.767 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2023-07-21T19:07:29.767 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2023-07-21T19:07:29.767 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 0, 2023-07-21T19:07:29.767 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 0, 2023-07-21T19:07:29.767 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2023-07-21T19:07:29.767 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:29.767 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:29.767 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:29.768 INFO:teuthology.orchestra.run.smithi193.stdout: } 2023-07-21T19:07:29.768 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2023-07-21T19:07:29.768 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle-0x55cde7d82460": { 2023-07-21T19:07:29.768 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2023-07-21T19:07:29.768 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2023-07-21T19:07:29.768 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2023-07-21T19:07:29.768 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2023-07-21T19:07:29.768 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2023-07-21T19:07:29.769 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2023-07-21T19:07:29.769 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2023-07-21T19:07:29.769 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2023-07-21T19:07:29.769 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2023-07-21T19:07:29.769 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 0, 2023-07-21T19:07:29.769 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 0, 2023-07-21T19:07:29.769 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2023-07-21T19:07:29.769 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2023-07-21T19:07:29.770 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2023-07-21T19:07:29.770 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2023-07-21T19:07:29.770 INFO:teuthology.orchestra.run.smithi193.stdout: } 2023-07-21T19:07:29.770 INFO:teuthology.orchestra.run.smithi193.stdout: } 2023-07-21T19:07:29.770 INFO:teuthology.orchestra.run.smithi193.stdout:} 2023-07-21T19:07:30.006 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2023-07-21T19:07:30.007 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2023-07-21T19:07:30.007 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2023-07-21T19:07:30.007 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2023-07-21T19:07:30.007 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2023-07-21T19:07:30.007 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2023-07-21T19:07:30.008 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2023-07-21T19:07:30.008 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-07-21T19:07:30.020 INFO:tasks.kclient:Unmounting kernel clients... 2023-07-21T19:07:30.020 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-07-21T19:07:30.021 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-21T19:07:30.057 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-07-21T19:07:30.057 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-21T19:07:30.118 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T19:07:30.302 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.290+0000 7f0fe0996700 1 -- 172.21.15.39:0/2125578903 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0fdc105b00 msgr2=0x7f0fdc105ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:30.302 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.290+0000 7f0fe0996700 1 --2- 172.21.15.39:0/2125578903 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0fdc105b00 0x7f0fdc105ed0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f0fd0009990 tx=0x7f0fd0009ca0 comp rx=0 tx=0).stop 2023-07-21T19:07:30.302 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.290+0000 7f0fe0996700 1 -- 172.21.15.39:0/2125578903 shutdown_connections 2023-07-21T19:07:30.303 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.290+0000 7f0fe0996700 1 --2- 172.21.15.39:0/2125578903 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0fdc1004a0 0x7f0fdc10c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:30.303 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.290+0000 7f0fe0996700 1 --2- 172.21.15.39:0/2125578903 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0fdc105b00 0x7f0fdc105ed0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:30.303 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.290+0000 7f0fe0996700 1 --2- 172.21.15.39:0/2125578903 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fdc0ffaf0 0x7f0fdc0fff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:30.304 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.290+0000 7f0fe0996700 1 -- 172.21.15.39:0/2125578903 >> 172.21.15.39:0/2125578903 conn(0x7f0fdc0fb5d0 msgr2=0x7f0fdc0fd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:30.304 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.291+0000 7f0fe0996700 1 -- 172.21.15.39:0/2125578903 shutdown_connections 2023-07-21T19:07:30.304 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.291+0000 7f0fe0996700 1 -- 172.21.15.39:0/2125578903 wait complete. 2023-07-21T19:07:30.308 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.292+0000 7f0fe0996700 1 Processor -- start 2023-07-21T19:07:30.309 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.292+0000 7f0fe0996700 1 -- start start 2023-07-21T19:07:30.309 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.292+0000 7f0fe0996700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fdc0ffaf0 0x7f0fdc197d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:30.309 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.292+0000 7f0fe0996700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0fdc1004a0 0x7f0fdc1982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:30.310 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.292+0000 7f0fe0996700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0fdc105b00 0x7f0fdc19c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:30.310 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.292+0000 7f0fe0996700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fdc19cd70 con 0x7f0fdc105b00 2023-07-21T19:07:30.310 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.292+0000 7f0fe0996700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fdc19cee0 con 0x7f0fdc0ffaf0 2023-07-21T19:07:30.311 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.292+0000 7f0fe0996700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fdc19d0a0 con 0x7f0fdc1004a0 2023-07-21T19:07:30.312 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.292+0000 7f0fdad9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0fdc105b00 0x7f0fdc19c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:30.312 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.292+0000 7f0fdad9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0fdc105b00 0x7f0fdc19c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35268/0 (socket says 172.21.15.39:35268) 2023-07-21T19:07:30.312 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.292+0000 7f0fdad9d700 1 -- 172.21.15.39:0/1257275051 learned_addr learned my addr 172.21.15.39:0/1257275051 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:07:30.312 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.292+0000 7f0fd9d9b700 1 --2- 172.21.15.39:0/1257275051 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0fdc1004a0 0x7f0fdc1982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:30.312 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.292+0000 7f0fda59c700 1 --2- 172.21.15.39:0/1257275051 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fdc0ffaf0 0x7f0fdc197d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:30.312 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.292+0000 7f0fdad9d700 1 -- 172.21.15.39:0/1257275051 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0fdc1004a0 msgr2=0x7f0fdc1982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:30.313 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.292+0000 7f0fdad9d700 1 --2- 172.21.15.39:0/1257275051 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0fdc1004a0 0x7f0fdc1982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:30.313 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.292+0000 7f0fdad9d700 1 -- 172.21.15.39:0/1257275051 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fdc0ffaf0 msgr2=0x7f0fdc197d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:30.313 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.292+0000 7f0fdad9d700 1 --2- 172.21.15.39:0/1257275051 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fdc0ffaf0 0x7f0fdc197d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:30.313 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.292+0000 7f0fdad9d700 1 -- 172.21.15.39:0/1257275051 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fd00095f0 con 0x7f0fdc105b00 2023-07-21T19:07:30.313 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.293+0000 7f0fdad9d700 1 --2- 172.21.15.39:0/1257275051 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0fdc105b00 0x7f0fdc19c730 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f0fd4007ee0 tx=0x7f0fd400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:30.313 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.293+0000 7f0fcb7fe700 1 -- 172.21.15.39:0/1257275051 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd4017070 con 0x7f0fdc105b00 2023-07-21T19:07:30.313 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.293+0000 7f0fcb7fe700 1 -- 172.21.15.39:0/1257275051 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0fd400f040 con 0x7f0fdc105b00 2023-07-21T19:07:30.314 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.293+0000 7f0fe0996700 1 -- 172.21.15.39:0/1257275051 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fdc19d380 con 0x7f0fdc105b00 2023-07-21T19:07:30.314 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.293+0000 7f0fcb7fe700 1 -- 172.21.15.39:0/1257275051 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd4005740 con 0x7f0fdc105b00 2023-07-21T19:07:30.314 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.293+0000 7f0fe0996700 1 -- 172.21.15.39:0/1257275051 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fdc1a4bc0 con 0x7f0fdc105b00 2023-07-21T19:07:30.314 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.294+0000 7f0fe0996700 1 -- 172.21.15.39:0/1257275051 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fdc066a70 con 0x7f0fdc105b00 2023-07-21T19:07:30.314 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.295+0000 7f0fcb7fe700 1 -- 172.21.15.39:0/1257275051 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f0fd400dc60 con 0x7f0fdc105b00 2023-07-21T19:07:30.314 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.296+0000 7f0fcb7fe700 1 --2- 172.21.15.39:0/1257275051 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f0fc40a1300 0x7f0fc40a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:30.314 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.296+0000 7f0fda59c700 1 --2- 172.21.15.39:0/1257275051 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f0fc40a1300 0x7f0fc40a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:30.314 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.296+0000 7f0fcb7fe700 1 -- 172.21.15.39:0/1257275051 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f0fd40ccb70 con 0x7f0fdc105b00 2023-07-21T19:07:30.315 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.296+0000 7f0fda59c700 1 --2- 172.21.15.39:0/1257275051 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f0fc40a1300 0x7f0fc40a37b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0fd000b5c0 tx=0x7f0fd0005fd0 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:30.315 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.299+0000 7f0fcb7fe700 1 -- 172.21.15.39:0/1257275051 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f0fd4095f20 con 0x7f0fdc105b00 2023-07-21T19:07:30.515 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.504+0000 7f0fe0996700 1 -- 172.21.15.39:0/1257275051 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0fdc1a4e10 con 0x7f0fdc105b00 2023-07-21T19:07:30.517 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.506+0000 7f0fcb7fe700 1 -- 172.21.15.39:0/1257275051 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) v1 ==== 74+0+22980 (secure 0 0 0) 0x7f0fd4099540 con 0x7f0fdc105b00 2023-07-21T19:07:30.517 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T19:07:30.518 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":124,"fsid":"eaf8e25a-27f4-11ee-9b34-001a4aab830c","created":"2023-07-21T18:33:06.504979+0000","modified":"2023-07-21T19:07:16.405558+0000","last_up_change":"2023-07-21T18:39:23.531009+0000","last_in_change":"2023-07-21T18:39:05.993024+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T18:36:13.572583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T18:40:02.726735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T18:40:03.990112+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":46,"snap_epoch":123,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03502c07-2d0f-4723-8d0b-d34c9f5b91bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6803","nonce":860193857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6805","nonce":860193857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6809","nonce":860193857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6807","nonce":860193857}]},"public_addr":"172.21.15.39:6803/860193857","cluster_addr":"172.21.15.39:6805/860193857","heartbeat_back_addr":"172.21.15.39:6809/860193857","heartbeat_front_addr":"172.21.15.39:6807/860193857","state":["exists","up"]},{"osd":1,"uuid":"0831a0c0-a1fa-4d30-9a28-513223aa69eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6801","nonce":3246721764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6803","nonce":3246721764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6807","nonce":3246721764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6805","nonce":3246721764}]},"public_addr":"172.21.15.155:6801/3246721764","cluster_addr":"172.21.15.155:6803/3246721764","heartbeat_back_addr":"172.21.15.155:6807/3246721764","heartbeat_front_addr":"172.21.15.155:6805/3246721764","state":["exists","up"]},{"osd":2,"uuid":"22df0003-bde9-48e4-b47a-cb8eafbab0db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6801","nonce":2563594449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6803","nonce":2563594449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6807","nonce":2563594449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6805","nonce":2563594449}]},"public_addr":"172.21.15.193:6801/2563594449","cluster_addr":"172.21.15.193:6803/2563594449","heartbeat_back_addr":"172.21.15.193:6807/2563594449","heartbeat_front_addr":"172.21.15.193:6805/2563594449","state":["exists","up"]},{"osd":3,"uuid":"54562558-a2e8-45dd-bc95-561569b9ae6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6811","nonce":863708386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6813","nonce":863708386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6817","nonce":863708386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6815","nonce":863708386}]},"public_addr":"172.21.15.39:6811/863708386","cluster_addr":"172.21.15.39:6813/863708386","heartbeat_back_addr":"172.21.15.39:6817/863708386","heartbeat_front_addr":"172.21.15.39:6815/863708386","state":["exists","up"]},{"osd":4,"uuid":"491118e7-ae1e-495c-9eac-ef1cdcae9600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6809","nonce":3361780673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6811","nonce":3361780673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6815","nonce":3361780673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6813","nonce":3361780673}]},"public_addr":"172.21.15.155:6809/3361780673","cluster_addr":"172.21.15.155:6811/3361780673","heartbeat_back_addr":"172.21.15.155:6815/3361780673","heartbeat_front_addr":"172.21.15.155:6813/3361780673","state":["exists","up"]},{"osd":5,"uuid":"d267bc2c-42b3-4d96-b2d1-b186dc7e6b18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6809","nonce":2376867721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6811","nonce":2376867721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6815","nonce":2376867721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6813","nonce":2376867721}]},"public_addr":"172.21.15.193:6809/2376867721","cluster_addr":"172.21.15.193:6811/2376867721","heartbeat_back_addr":"172.21.15.193:6815/2376867721","heartbeat_front_addr":"172.21.15.193:6813/2376867721","state":["exists","up"]},{"osd":6,"uuid":"beec8088-970a-473e-bd67-6eee7cbdef24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6819","nonce":2417770529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6821","nonce":2417770529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6825","nonce":2417770529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6823","nonce":2417770529}]},"public_addr":"172.21.15.39:6819/2417770529","cluster_addr":"172.21.15.39:6821/2417770529","heartbeat_back_addr":"172.21.15.39:6825/2417770529","heartbeat_front_addr":"172.21.15.39:6823/2417770529","state":["exists","up"]},{"osd":7,"uuid":"5a88ce30-1835-4254-90cc-804ed5cf1066","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6817","nonce":2542002111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6819","nonce":2542002111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6823","nonce":2542002111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6821","nonce":2542002111}]},"public_addr":"172.21.15.155:6817/2542002111","cluster_addr":"172.21.15.155:6819/2542002111","heartbeat_back_addr":"172.21.15.155:6823/2542002111","heartbeat_front_addr":"172.21.15.155:6821/2542002111","state":["exists","up"]},{"osd":8,"uuid":"d964a183-3021-4ace-9a0c-aad00f2570e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6817","nonce":2985835085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6819","nonce":2985835085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6823","nonce":2985835085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6821","nonce":2985835085}]},"public_addr":"172.21.15.193:6817/2985835085","cluster_addr":"172.21.15.193:6819/2985835085","heartbeat_back_addr":"172.21.15.193:6823/2985835085","heartbeat_front_addr":"172.21.15.193:6821/2985835085","state":["exists","up"]},{"osd":9,"uuid":"9725ea5c-49a3-4c99-bac4-e95069100b1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6827","nonce":1946493651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6829","nonce":1946493651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6833","nonce":1946493651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6831","nonce":1946493651}]},"public_addr":"172.21.15.39:6827/1946493651","cluster_addr":"172.21.15.39:6829/1946493651","heartbeat_back_addr":"172.21.15.39:6833/1946493651","heartbeat_front_addr":"172.21.15.39:6831/1946493651","state":["exists","up"]},{"osd":10,"uuid":"1fd60ccb-92db-45f0-a580-2003db0976e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6825","nonce":1234615022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6827","nonce":1234615022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6831","nonce":1234615022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6829","nonce":1234615022}]},"public_addr":"172.21.15.155:6825/1234615022","cluster_addr":"172.21.15.155:6827/1234615022","heartbeat_back_addr":"172.21.15.155:6831/1234615022","heartbeat_front_addr":"172.21.15.155:6829/1234615022","state":["exists","up"]},{"osd":11,"uuid":"efb31ea1-4775-4e5f-8740-728d6f4aeea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6825","nonce":215104663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6827","nonce":215104663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6831","nonce":215104663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6829","nonce":215104663}]},"public_addr":"172.21.15.193:6825/215104663","cluster_addr":"172.21.15.193:6827/215104663","heartbeat_back_addr":"172.21.15.193:6831/215104663","heartbeat_front_addr":"172.21.15.193:6829/215104663","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:35:30.211698+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:35:50.883846+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:36:11.124168+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:36:32.711905+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:36:54.442007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:37:14.843526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:37:36.587354+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:37:57.882374+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:38:18.729407+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:38:40.139231+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:39:01.290862+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:39:21.767537+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:0/1762298871":"2023-07-22T18:41:44.230582+0000","172.21.15.39:0/1127767082":"2023-07-22T18:34:07.039823+0000","172.21.15.39:6800/2116291712":"2023-07-22T18:41:44.230582+0000","172.21.15.39:0/3819152602":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/4249110368":"2023-07-22T18:41:44.230582+0000","172.21.15.39:6801/2116291712":"2023-07-22T18:41:44.230582+0000","172.21.15.39:0/1275748831":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/545627125":"2023-07-22T18:41:44.230582+0000","172.21.15.39:6800/889516631":"2023-07-22T18:33:40.852706+0000","172.21.15.39:6800/2893866425":"2023-07-22T18:34:07.039823+0000","172.21.15.39:0/3636510379":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/19717002":"2023-07-22T18:34:07.039823+0000","172.21.15.39:6801/2893866425":"2023-07-22T18:34:07.039823+0000","172.21.15.39:0/2902600414":"2023-07-22T18:34:07.039823+0000","172.21.15.39:0/1870486291":"2023-07-22T18:41:44.230582+0000","172.21.15.39:6801/889516631":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/3797141824":"2023-07-22T18:41:44.230582+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":34,"length":1},{"begin":46,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T19:07:30.519 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.508+0000 7f0fe0996700 1 -- 172.21.15.39:0/1257275051 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f0fc40a1300 msgr2=0x7f0fc40a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:30.519 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.508+0000 7f0fe0996700 1 --2- 172.21.15.39:0/1257275051 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f0fc40a1300 0x7f0fc40a37b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0fd000b5c0 tx=0x7f0fd0005fd0 comp rx=0 tx=0).stop 2023-07-21T19:07:30.519 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.508+0000 7f0fe0996700 1 -- 172.21.15.39:0/1257275051 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0fdc105b00 msgr2=0x7f0fdc19c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:30.520 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.508+0000 7f0fe0996700 1 --2- 172.21.15.39:0/1257275051 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0fdc105b00 0x7f0fdc19c730 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f0fd4007ee0 tx=0x7f0fd400d3b0 comp rx=0 tx=0).stop 2023-07-21T19:07:30.521 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.508+0000 7f0fe0996700 1 -- 172.21.15.39:0/1257275051 shutdown_connections 2023-07-21T19:07:30.521 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.508+0000 7f0fe0996700 1 --2- 172.21.15.39:0/1257275051 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f0fc40a1300 0x7f0fc40a37b0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:30.521 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.508+0000 7f0fe0996700 1 --2- 172.21.15.39:0/1257275051 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fdc0ffaf0 0x7f0fdc197d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:30.521 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.508+0000 7f0fe0996700 1 --2- 172.21.15.39:0/1257275051 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0fdc105b00 0x7f0fdc19c730 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:30.522 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.508+0000 7f0fe0996700 1 --2- 172.21.15.39:0/1257275051 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0fdc1004a0 0x7f0fdc1982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:30.522 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.508+0000 7f0fe0996700 1 -- 172.21.15.39:0/1257275051 >> 172.21.15.39:0/1257275051 conn(0x7f0fdc0fb5d0 msgr2=0x7f0fdc0fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:30.522 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.508+0000 7f0fe0996700 1 -- 172.21.15.39:0/1257275051 shutdown_connections 2023-07-21T19:07:30.522 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.508+0000 7f0fe0996700 1 -- 172.21.15.39:0/1257275051 wait complete. 2023-07-21T19:07:30.534 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-21T18:36:13.572583+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-21T18:40:02.726735+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-21T18:40:03.990112+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 46, 'snap_epoch': 123, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-21T19:07:30.534 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-21T19:07:30.663 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.652+0000 7f8d64dfa700 1 -- 172.21.15.39:0/3090234406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d60105b00 msgr2=0x7f8d60105ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:30.663 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.652+0000 7f8d64dfa700 1 --2- 172.21.15.39:0/3090234406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d60105b00 0x7f8d60105ed0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f8d50009990 tx=0x7f8d50009ca0 comp rx=0 tx=0).stop 2023-07-21T19:07:30.664 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.653+0000 7f8d64dfa700 1 -- 172.21.15.39:0/3090234406 shutdown_connections 2023-07-21T19:07:30.664 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.653+0000 7f8d64dfa700 1 --2- 172.21.15.39:0/3090234406 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d601004a0 0x7f8d6010c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:30.664 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.653+0000 7f8d64dfa700 1 --2- 172.21.15.39:0/3090234406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d60105b00 0x7f8d60105ed0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:30.664 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.653+0000 7f8d64dfa700 1 --2- 172.21.15.39:0/3090234406 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d600ffaf0 0x7f8d600fff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:30.664 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.653+0000 7f8d64dfa700 1 -- 172.21.15.39:0/3090234406 >> 172.21.15.39:0/3090234406 conn(0x7f8d600fb650 msgr2=0x7f8d600fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:30.664 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.653+0000 7f8d64dfa700 1 -- 172.21.15.39:0/3090234406 shutdown_connections 2023-07-21T19:07:30.665 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.653+0000 7f8d64dfa700 1 -- 172.21.15.39:0/3090234406 wait complete. 2023-07-21T19:07:30.667 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.653+0000 7f8d64dfa700 1 Processor -- start 2023-07-21T19:07:30.667 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.653+0000 7f8d64dfa700 1 -- start start 2023-07-21T19:07:30.667 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.654+0000 7f8d64dfa700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d600ffaf0 0x7f8d6019ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:30.667 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.654+0000 7f8d64dfa700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d601004a0 0x7f8d6019bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:30.667 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.654+0000 7f8d64dfa700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d60105b00 0x7f8d60195a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:30.667 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.654+0000 7f8d64dfa700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d601960e0 con 0x7f8d600ffaf0 2023-07-21T19:07:30.668 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.654+0000 7f8d64dfa700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d60196250 con 0x7f8d601004a0 2023-07-21T19:07:30.668 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.654+0000 7f8d64dfa700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d60196410 con 0x7f8d60105b00 2023-07-21T19:07:30.668 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.654+0000 7f8d5e59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d600ffaf0 0x7f8d6019ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:30.668 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.654+0000 7f8d5e59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d600ffaf0 0x7f8d6019ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35278/0 (socket says 172.21.15.39:35278) 2023-07-21T19:07:30.668 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.654+0000 7f8d5e59c700 1 -- 172.21.15.39:0/1486377752 learned_addr learned my addr 172.21.15.39:0/1486377752 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:07:30.668 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.654+0000 7f8d5ed9d700 1 --2- 172.21.15.39:0/1486377752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d60105b00 0x7f8d60195a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:30.668 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.654+0000 7f8d5dd9b700 1 --2- 172.21.15.39:0/1486377752 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d601004a0 0x7f8d6019bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:30.669 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.654+0000 7f8d5e59c700 1 -- 172.21.15.39:0/1486377752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d60105b00 msgr2=0x7f8d60195a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:30.669 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.654+0000 7f8d5e59c700 1 --2- 172.21.15.39:0/1486377752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d60105b00 0x7f8d60195a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:30.669 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.654+0000 7f8d5e59c700 1 -- 172.21.15.39:0/1486377752 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d601004a0 msgr2=0x7f8d6019bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:30.669 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.654+0000 7f8d5e59c700 1 --2- 172.21.15.39:0/1486377752 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d601004a0 0x7f8d6019bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:30.669 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.654+0000 7f8d5e59c700 1 -- 172.21.15.39:0/1486377752 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d500095f0 con 0x7f8d600ffaf0 2023-07-21T19:07:30.669 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.654+0000 7f8d5dd9b700 1 --2- 172.21.15.39:0/1486377752 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d601004a0 0x7f8d6019bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:07:30.669 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.654+0000 7f8d5ed9d700 1 --2- 172.21.15.39:0/1486377752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d60105b00 0x7f8d60195a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:07:30.669 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.654+0000 7f8d5e59c700 1 --2- 172.21.15.39:0/1486377752 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d600ffaf0 0x7f8d6019ba00 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f8d500095c0 tx=0x7f8d5000f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:30.670 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.654+0000 7f8d4f7fe700 1 -- 172.21.15.39:0/1486377752 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d5001d070 con 0x7f8d600ffaf0 2023-07-21T19:07:30.670 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.654+0000 7f8d4f7fe700 1 -- 172.21.15.39:0/1486377752 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d5000fc00 con 0x7f8d600ffaf0 2023-07-21T19:07:30.670 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.654+0000 7f8d64dfa700 1 -- 172.21.15.39:0/1486377752 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d60196690 con 0x7f8d600ffaf0 2023-07-21T19:07:30.670 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.654+0000 7f8d4f7fe700 1 -- 172.21.15.39:0/1486377752 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d500173f0 con 0x7f8d600ffaf0 2023-07-21T19:07:30.670 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.654+0000 7f8d64dfa700 1 -- 172.21.15.39:0/1486377752 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d601a29f0 con 0x7f8d600ffaf0 2023-07-21T19:07:30.670 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.655+0000 7f8d64dfa700 1 -- 172.21.15.39:0/1486377752 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d60066a00 con 0x7f8d600ffaf0 2023-07-21T19:07:30.671 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.656+0000 7f8d4f7fe700 1 -- 172.21.15.39:0/1486377752 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f8d50022470 con 0x7f8d600ffaf0 2023-07-21T19:07:30.671 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.657+0000 7f8d4f7fe700 1 --2- 172.21.15.39:0/1486377752 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f8d480a1340 0x7f8d480a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:30.671 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.657+0000 7f8d4f7fe700 1 -- 172.21.15.39:0/1486377752 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f8d500ce330 con 0x7f8d600ffaf0 2023-07-21T19:07:30.671 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.657+0000 7f8d5dd9b700 1 --2- 172.21.15.39:0/1486377752 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f8d480a1340 0x7f8d480a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:30.672 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.658+0000 7f8d5dd9b700 1 --2- 172.21.15.39:0/1486377752 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f8d480a1340 0x7f8d480a37f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8d54005950 tx=0x7f8d540058e0 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:30.672 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.660+0000 7f8d4f7fe700 1 -- 172.21.15.39:0/1486377752 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f8d500977d0 con 0x7f8d600ffaf0 2023-07-21T19:07:30.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:30 smithi193 ceph-mon[98204]: pgmap v817: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.5 KiB/s wr, 11 op/s 2023-07-21T19:07:30.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:30 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/1257275051' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:07:30.870 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.860+0000 7f8d64dfa700 1 -- 172.21.15.39:0/1486377752 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8d601a2b80 con 0x7f8d600ffaf0 2023-07-21T19:07:30.871 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.860+0000 7f8d4f7fe700 1 -- 172.21.15.39:0/1486377752 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v124) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8d5009adf0 con 0x7f8d600ffaf0 2023-07-21T19:07:30.871 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2023-07-21T19:07:30.873 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.862+0000 7f8d64dfa700 1 -- 172.21.15.39:0/1486377752 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f8d480a1340 msgr2=0x7f8d480a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:30.873 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.862+0000 7f8d64dfa700 1 --2- 172.21.15.39:0/1486377752 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f8d480a1340 0x7f8d480a37f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8d54005950 tx=0x7f8d540058e0 comp rx=0 tx=0).stop 2023-07-21T19:07:30.874 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.862+0000 7f8d64dfa700 1 -- 172.21.15.39:0/1486377752 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d600ffaf0 msgr2=0x7f8d6019ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:30.874 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.862+0000 7f8d64dfa700 1 --2- 172.21.15.39:0/1486377752 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d600ffaf0 0x7f8d6019ba00 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f8d500095c0 tx=0x7f8d5000f710 comp rx=0 tx=0).stop 2023-07-21T19:07:30.874 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.862+0000 7f8d64dfa700 1 -- 172.21.15.39:0/1486377752 shutdown_connections 2023-07-21T19:07:30.874 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.862+0000 7f8d64dfa700 1 --2- 172.21.15.39:0/1486377752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d60105b00 0x7f8d60195a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:30.875 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.862+0000 7f8d64dfa700 1 --2- 172.21.15.39:0/1486377752 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d600ffaf0 0x7f8d6019ba00 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:30.875 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.862+0000 7f8d64dfa700 1 --2- 172.21.15.39:0/1486377752 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f8d480a1340 0x7f8d480a37f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:30.875 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.862+0000 7f8d64dfa700 1 --2- 172.21.15.39:0/1486377752 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d601004a0 0x7f8d6019bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:30.875 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.862+0000 7f8d64dfa700 1 -- 172.21.15.39:0/1486377752 >> 172.21.15.39:0/1486377752 conn(0x7f8d600fb650 msgr2=0x7f8d60103600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:30.876 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.863+0000 7f8d64dfa700 1 -- 172.21.15.39:0/1486377752 shutdown_connections 2023-07-21T19:07:30.876 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:30.863+0000 7f8d64dfa700 1 -- 172.21.15.39:0/1486377752 wait complete. 2023-07-21T19:07:30.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:30 smithi155 ceph-mon[97934]: pgmap v817: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.5 KiB/s wr, 11 op/s 2023-07-21T19:07:30.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:30 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/1257275051' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:07:30.885 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:30 smithi039 ceph-mon[96489]: pgmap v817: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.5 KiB/s wr, 11 op/s 2023-07-21T19:07:30.885 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:30 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/1257275051' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:07:30.886 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-21T19:07:31.015 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.004+0000 7f5f1709f700 1 -- 172.21.15.39:0/130430552 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f101004e0 msgr2=0x7f5f1010c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:31.015 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.004+0000 7f5f1709f700 1 --2- 172.21.15.39:0/130430552 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f101004e0 0x7f5f1010c920 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f5f0800b0a0 tx=0x7f5f0800b3b0 comp rx=0 tx=0).stop 2023-07-21T19:07:31.017 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.004+0000 7f5f1709f700 1 -- 172.21.15.39:0/130430552 shutdown_connections 2023-07-21T19:07:31.017 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.004+0000 7f5f1709f700 1 --2- 172.21.15.39:0/130430552 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f10105b60 0x7f5f10105f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:31.018 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.004+0000 7f5f1709f700 1 --2- 172.21.15.39:0/130430552 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f101004e0 0x7f5f1010c920 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:31.018 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.004+0000 7f5f1709f700 1 --2- 172.21.15.39:0/130430552 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5f100ffb30 0x7f5f100fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:31.018 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.004+0000 7f5f1709f700 1 -- 172.21.15.39:0/130430552 >> 172.21.15.39:0/130430552 conn(0x7f5f100fb650 msgr2=0x7f5f100fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:31.018 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.004+0000 7f5f1709f700 1 -- 172.21.15.39:0/130430552 shutdown_connections 2023-07-21T19:07:31.018 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.005+0000 7f5f1709f700 1 -- 172.21.15.39:0/130430552 wait complete. 2023-07-21T19:07:31.018 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.005+0000 7f5f1709f700 1 Processor -- start 2023-07-21T19:07:31.018 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.005+0000 7f5f1709f700 1 -- start start 2023-07-21T19:07:31.019 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.005+0000 7f5f1709f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f100ffb30 0x7f5f1019b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:31.019 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.005+0000 7f5f1709f700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5f101004e0 0x7f5f1019bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:31.019 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.005+0000 7f5f1709f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f10105b60 0x7f5f10195a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:31.019 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.005+0000 7f5f1709f700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f101960d0 con 0x7f5f100ffb30 2023-07-21T19:07:31.019 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.005+0000 7f5f1709f700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f10196240 con 0x7f5f10105b60 2023-07-21T19:07:31.019 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.005+0000 7f5f1709f700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f10196400 con 0x7f5f101004e0 2023-07-21T19:07:31.020 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.005+0000 7f5f14e3b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f100ffb30 0x7f5f1019b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:31.020 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.005+0000 7f5f14e3b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f100ffb30 0x7f5f1019b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35292/0 (socket says 172.21.15.39:35292) 2023-07-21T19:07:31.020 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.005+0000 7f5f14e3b700 1 -- 172.21.15.39:0/1625270160 learned_addr learned my addr 172.21.15.39:0/1625270160 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:07:31.021 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.006+0000 7f5f1563c700 1 --2- 172.21.15.39:0/1625270160 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f10105b60 0x7f5f10195a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:31.021 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.006+0000 7f5f0ffff700 1 --2- 172.21.15.39:0/1625270160 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5f101004e0 0x7f5f1019bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:31.021 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.006+0000 7f5f14e3b700 1 -- 172.21.15.39:0/1625270160 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5f101004e0 msgr2=0x7f5f1019bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:31.021 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.006+0000 7f5f14e3b700 1 --2- 172.21.15.39:0/1625270160 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5f101004e0 0x7f5f1019bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:31.021 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.006+0000 7f5f14e3b700 1 -- 172.21.15.39:0/1625270160 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f10105b60 msgr2=0x7f5f10195a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:31.021 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.006+0000 7f5f14e3b700 1 --2- 172.21.15.39:0/1625270160 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f10105b60 0x7f5f10195a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:31.021 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.006+0000 7f5f14e3b700 1 -- 172.21.15.39:0/1625270160 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f08009d00 con 0x7f5f100ffb30 2023-07-21T19:07:31.022 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.006+0000 7f5f0ffff700 1 --2- 172.21.15.39:0/1625270160 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5f101004e0 0x7f5f1019bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:07:31.022 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.006+0000 7f5f1563c700 1 --2- 172.21.15.39:0/1625270160 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f10105b60 0x7f5f10195a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:07:31.022 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.006+0000 7f5f14e3b700 1 --2- 172.21.15.39:0/1625270160 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f100ffb30 0x7f5f1019b9f0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f5f0000d950 tx=0x7f5f0000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:31.024 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.006+0000 7f5f0dffb700 1 -- 172.21.15.39:0/1625270160 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f00009b40 con 0x7f5f100ffb30 2023-07-21T19:07:31.024 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.006+0000 7f5f0dffb700 1 -- 172.21.15.39:0/1625270160 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f00009ca0 con 0x7f5f100ffb30 2023-07-21T19:07:31.024 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.006+0000 7f5f1709f700 1 -- 172.21.15.39:0/1625270160 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f101966e0 con 0x7f5f100ffb30 2023-07-21T19:07:31.025 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.006+0000 7f5f0dffb700 1 -- 172.21.15.39:0/1625270160 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0000f620 con 0x7f5f100ffb30 2023-07-21T19:07:31.025 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.006+0000 7f5f1709f700 1 -- 172.21.15.39:0/1625270160 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f101a29e0 con 0x7f5f100ffb30 2023-07-21T19:07:31.025 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.007+0000 7f5f1709f700 1 -- 172.21.15.39:0/1625270160 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f100676e0 con 0x7f5f100ffb30 2023-07-21T19:07:31.026 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.008+0000 7f5f0dffb700 1 -- 172.21.15.39:0/1625270160 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f5f00010460 con 0x7f5f100ffb30 2023-07-21T19:07:31.026 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.009+0000 7f5f0dffb700 1 --2- 172.21.15.39:0/1625270160 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f5efc0a1340 0x7f5efc0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:31.026 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.009+0000 7f5f0ffff700 1 --2- 172.21.15.39:0/1625270160 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f5efc0a1340 0x7f5efc0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:31.026 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.009+0000 7f5f0dffb700 1 -- 172.21.15.39:0/1625270160 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f5f000cd020 con 0x7f5f100ffb30 2023-07-21T19:07:31.026 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.009+0000 7f5f0ffff700 1 --2- 172.21.15.39:0/1625270160 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f5efc0a1340 0x7f5efc0a37f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5ef8005950 tx=0x7f5ef80058e0 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:31.026 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.013+0000 7f5f0dffb700 1 -- 172.21.15.39:0/1625270160 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f5f00096430 con 0x7f5f100ffb30 2023-07-21T19:07:31.223 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.212+0000 7f5f1709f700 1 -- 172.21.15.39:0/1625270160 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5f10196f70 con 0x7f5f100ffb30 2023-07-21T19:07:31.224 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.213+0000 7f5f0dffb700 1 -- 172.21.15.39:0/1625270160 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v124) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5f00099a50 con 0x7f5f100ffb30 2023-07-21T19:07:31.224 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2023-07-21T19:07:31.226 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.215+0000 7f5f1709f700 1 -- 172.21.15.39:0/1625270160 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f5efc0a1340 msgr2=0x7f5efc0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:31.226 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.215+0000 7f5f1709f700 1 --2- 172.21.15.39:0/1625270160 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f5efc0a1340 0x7f5efc0a37f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5ef8005950 tx=0x7f5ef80058e0 comp rx=0 tx=0).stop 2023-07-21T19:07:31.226 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.215+0000 7f5f1709f700 1 -- 172.21.15.39:0/1625270160 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f100ffb30 msgr2=0x7f5f1019b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:31.227 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.215+0000 7f5f1709f700 1 --2- 172.21.15.39:0/1625270160 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f100ffb30 0x7f5f1019b9f0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f5f0000d950 tx=0x7f5f0000dd10 comp rx=0 tx=0).stop 2023-07-21T19:07:31.227 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.215+0000 7f5f1709f700 1 -- 172.21.15.39:0/1625270160 shutdown_connections 2023-07-21T19:07:31.227 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.215+0000 7f5f1709f700 1 --2- 172.21.15.39:0/1625270160 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f5efc0a1340 0x7f5efc0a37f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:31.227 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.215+0000 7f5f1709f700 1 --2- 172.21.15.39:0/1625270160 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f10105b60 0x7f5f10195a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:31.227 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.215+0000 7f5f1709f700 1 --2- 172.21.15.39:0/1625270160 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f100ffb30 0x7f5f1019b9f0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:31.228 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.215+0000 7f5f1709f700 1 --2- 172.21.15.39:0/1625270160 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5f101004e0 0x7f5f1019bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:31.228 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.215+0000 7f5f1709f700 1 -- 172.21.15.39:0/1625270160 >> 172.21.15.39:0/1625270160 conn(0x7f5f100fb650 msgr2=0x7f5f10103730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:31.228 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.215+0000 7f5f1709f700 1 -- 172.21.15.39:0/1625270160 shutdown_connections 2023-07-21T19:07:31.228 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.215+0000 7f5f1709f700 1 -- 172.21.15.39:0/1625270160 wait complete. 2023-07-21T19:07:31.239 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-21T19:07:31.368 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.358+0000 7f7a64549700 1 -- 172.21.15.39:0/4093550079 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a5c105c00 msgr2=0x7f7a5c105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:31.369 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.358+0000 7f7a64549700 1 --2- 172.21.15.39:0/4093550079 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a5c105c00 0x7f7a5c105fd0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f7a58009990 tx=0x7f7a58009ca0 comp rx=0 tx=0).stop 2023-07-21T19:07:31.369 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.358+0000 7f7a64549700 1 -- 172.21.15.39:0/4093550079 shutdown_connections 2023-07-21T19:07:31.369 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.358+0000 7f7a64549700 1 --2- 172.21.15.39:0/4093550079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a5c0feb90 0x7f7a5c10c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:31.369 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.358+0000 7f7a64549700 1 --2- 172.21.15.39:0/4093550079 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a5c105c00 0x7f7a5c105fd0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:31.370 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.358+0000 7f7a64549700 1 --2- 172.21.15.39:0/4093550079 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a5c0fe200 0x7f7a5c0fe650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:31.370 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.358+0000 7f7a64549700 1 -- 172.21.15.39:0/4093550079 >> 172.21.15.39:0/4093550079 conn(0x7f7a5c076200 msgr2=0x7f7a5c076600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:31.370 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.358+0000 7f7a64549700 1 -- 172.21.15.39:0/4093550079 shutdown_connections 2023-07-21T19:07:31.370 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.358+0000 7f7a64549700 1 -- 172.21.15.39:0/4093550079 wait complete. 2023-07-21T19:07:31.372 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.359+0000 7f7a64549700 1 Processor -- start 2023-07-21T19:07:31.372 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.359+0000 7f7a64549700 1 -- start start 2023-07-21T19:07:31.372 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.359+0000 7f7a64549700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a5c0fe200 0x7f7a5c195d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:31.372 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.359+0000 7f7a64549700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a5c0feb90 0x7f7a5c196260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:31.373 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.359+0000 7f7a64549700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a5c105c00 0x7f7a5c19a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:31.373 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.359+0000 7f7a64549700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5c19ad00 con 0x7f7a5c105c00 2023-07-21T19:07:31.373 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.359+0000 7f7a64549700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5c19ae70 con 0x7f7a5c0fe200 2023-07-21T19:07:31.373 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.359+0000 7f7a64549700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5c19b030 con 0x7f7a5c0feb90 2023-07-21T19:07:31.373 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.359+0000 7f7a62ae6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a5c105c00 0x7f7a5c19a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:31.373 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.359+0000 7f7a62ae6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a5c105c00 0x7f7a5c19a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35308/0 (socket says 172.21.15.39:35308) 2023-07-21T19:07:31.373 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.359+0000 7f7a62ae6700 1 -- 172.21.15.39:0/2260527571 learned_addr learned my addr 172.21.15.39:0/2260527571 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:07:31.373 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.359+0000 7f7a61ae4700 1 --2- 172.21.15.39:0/2260527571 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a5c0feb90 0x7f7a5c196260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:31.374 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.359+0000 7f7a622e5700 1 --2- 172.21.15.39:0/2260527571 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a5c0fe200 0x7f7a5c195d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:31.374 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.359+0000 7f7a62ae6700 1 -- 172.21.15.39:0/2260527571 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a5c0feb90 msgr2=0x7f7a5c196260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:31.374 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.359+0000 7f7a62ae6700 1 --2- 172.21.15.39:0/2260527571 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a5c0feb90 0x7f7a5c196260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:31.374 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.359+0000 7f7a62ae6700 1 -- 172.21.15.39:0/2260527571 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a5c0fe200 msgr2=0x7f7a5c195d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:31.374 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.359+0000 7f7a62ae6700 1 --2- 172.21.15.39:0/2260527571 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a5c0fe200 0x7f7a5c195d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:31.374 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.359+0000 7f7a62ae6700 1 -- 172.21.15.39:0/2260527571 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a580095f0 con 0x7f7a5c105c00 2023-07-21T19:07:31.374 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.360+0000 7f7a62ae6700 1 --2- 172.21.15.39:0/2260527571 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a5c105c00 0x7f7a5c19a6c0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f7a54007ee0 tx=0x7f7a5400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:31.375 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.360+0000 7f7a537fe700 1 -- 172.21.15.39:0/2260527571 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a54017070 con 0x7f7a5c105c00 2023-07-21T19:07:31.375 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.360+0000 7f7a537fe700 1 -- 172.21.15.39:0/2260527571 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a5400f040 con 0x7f7a5c105c00 2023-07-21T19:07:31.375 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.360+0000 7f7a64549700 1 -- 172.21.15.39:0/2260527571 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a5c19b310 con 0x7f7a5c105c00 2023-07-21T19:07:31.375 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.360+0000 7f7a537fe700 1 -- 172.21.15.39:0/2260527571 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a54005580 con 0x7f7a5c105c00 2023-07-21T19:07:31.375 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.360+0000 7f7a64549700 1 -- 172.21.15.39:0/2260527571 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a5c1a2b50 con 0x7f7a5c105c00 2023-07-21T19:07:31.375 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.361+0000 7f7a64549700 1 -- 172.21.15.39:0/2260527571 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a5c066a00 con 0x7f7a5c105c00 2023-07-21T19:07:31.376 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.362+0000 7f7a537fe700 1 -- 172.21.15.39:0/2260527571 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f7a5400db20 con 0x7f7a5c105c00 2023-07-21T19:07:31.376 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.363+0000 7f7a537fe700 1 --2- 172.21.15.39:0/2260527571 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f7a480a12c0 0x7f7a480a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:31.376 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.363+0000 7f7a537fe700 1 -- 172.21.15.39:0/2260527571 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f7a540cca20 con 0x7f7a5c105c00 2023-07-21T19:07:31.376 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.363+0000 7f7a622e5700 1 --2- 172.21.15.39:0/2260527571 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f7a480a12c0 0x7f7a480a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:31.377 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.363+0000 7f7a622e5700 1 --2- 172.21.15.39:0/2260527571 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f7a480a12c0 0x7f7a480a3770 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7a5800b5c0 tx=0x7f7a58005fb0 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:31.377 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.366+0000 7f7a537fe700 1 -- 172.21.15.39:0/2260527571 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f7a54095e30 con 0x7f7a5c105c00 2023-07-21T19:07:31.575 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.565+0000 7f7a64549700 1 -- 172.21.15.39:0/2260527571 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7a5c1a2ce0 con 0x7f7a5c105c00 2023-07-21T19:07:31.576 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.565+0000 7f7a537fe700 1 -- 172.21.15.39:0/2260527571 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v124) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7a54099450 con 0x7f7a5c105c00 2023-07-21T19:07:31.577 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 64 2023-07-21T19:07:31.578 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.568+0000 7f7a64549700 1 -- 172.21.15.39:0/2260527571 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f7a480a12c0 msgr2=0x7f7a480a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:31.578 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.568+0000 7f7a64549700 1 --2- 172.21.15.39:0/2260527571 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f7a480a12c0 0x7f7a480a3770 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7a5800b5c0 tx=0x7f7a58005fb0 comp rx=0 tx=0).stop 2023-07-21T19:07:31.579 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.568+0000 7f7a64549700 1 -- 172.21.15.39:0/2260527571 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a5c105c00 msgr2=0x7f7a5c19a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:31.579 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.568+0000 7f7a64549700 1 --2- 172.21.15.39:0/2260527571 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a5c105c00 0x7f7a5c19a6c0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f7a54007ee0 tx=0x7f7a5400d3b0 comp rx=0 tx=0).stop 2023-07-21T19:07:31.579 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.568+0000 7f7a64549700 1 -- 172.21.15.39:0/2260527571 shutdown_connections 2023-07-21T19:07:31.579 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.568+0000 7f7a64549700 1 --2- 172.21.15.39:0/2260527571 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f7a480a12c0 0x7f7a480a3770 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:31.579 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.568+0000 7f7a64549700 1 --2- 172.21.15.39:0/2260527571 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a5c0fe200 0x7f7a5c195d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:31.579 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.568+0000 7f7a64549700 1 --2- 172.21.15.39:0/2260527571 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a5c105c00 0x7f7a5c19a6c0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:31.579 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.568+0000 7f7a64549700 1 --2- 172.21.15.39:0/2260527571 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a5c0feb90 0x7f7a5c196260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:31.580 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.568+0000 7f7a64549700 1 -- 172.21.15.39:0/2260527571 >> 172.21.15.39:0/2260527571 conn(0x7f7a5c076200 msgr2=0x7f7a5c0fbb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:31.580 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.568+0000 7f7a64549700 1 -- 172.21.15.39:0/2260527571 shutdown_connections 2023-07-21T19:07:31.580 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.568+0000 7f7a64549700 1 -- 172.21.15.39:0/2260527571 wait complete. 2023-07-21T19:07:31.591 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-21T19:07:31.719 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.708+0000 7f2258950700 1 -- 172.21.15.39:0/275980124 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22540feb90 msgr2=0x7f225410c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:31.719 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.708+0000 7f2258950700 1 --2- 172.21.15.39:0/275980124 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22540feb90 0x7f225410c9c0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f224c00b0a0 tx=0x7f224c00b3b0 comp rx=0 tx=0).stop 2023-07-21T19:07:31.719 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.708+0000 7f2258950700 1 -- 172.21.15.39:0/275980124 shutdown_connections 2023-07-21T19:07:31.720 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.708+0000 7f2258950700 1 --2- 172.21.15.39:0/275980124 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2254105c00 0x7f2254105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:31.720 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.708+0000 7f2258950700 1 --2- 172.21.15.39:0/275980124 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22540feb90 0x7f225410c9c0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:31.720 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.708+0000 7f2258950700 1 --2- 172.21.15.39:0/275980124 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22540fe200 0x7f22540fe650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:31.720 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.708+0000 7f2258950700 1 -- 172.21.15.39:0/275980124 >> 172.21.15.39:0/275980124 conn(0x7f2254076200 msgr2=0x7f2254076600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:31.720 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.708+0000 7f2258950700 1 -- 172.21.15.39:0/275980124 shutdown_connections 2023-07-21T19:07:31.720 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.708+0000 7f2258950700 1 -- 172.21.15.39:0/275980124 wait complete. 2023-07-21T19:07:31.722 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.709+0000 7f2258950700 1 Processor -- start 2023-07-21T19:07:31.723 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.709+0000 7f2258950700 1 -- start start 2023-07-21T19:07:31.723 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.709+0000 7f2258950700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22540fe200 0x7f2254195d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:31.723 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.709+0000 7f2258950700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22540feb90 0x7f22541962c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:31.723 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.709+0000 7f2258950700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2254105c00 0x7f225419a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:31.723 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.709+0000 7f2258950700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f225419ad60 con 0x7f22540fe200 2023-07-21T19:07:31.723 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.709+0000 7f2258950700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f225419aed0 con 0x7f2254105c00 2023-07-21T19:07:31.724 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.709+0000 7f2258950700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f225419b090 con 0x7f22540feb90 2023-07-21T19:07:31.724 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.710+0000 7f225259c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22540fe200 0x7f2254195d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:31.724 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.710+0000 7f225259c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22540fe200 0x7f2254195d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35318/0 (socket says 172.21.15.39:35318) 2023-07-21T19:07:31.724 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.710+0000 7f225259c700 1 -- 172.21.15.39:0/4253764157 learned_addr learned my addr 172.21.15.39:0/4253764157 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:07:31.724 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.710+0000 7f224bfff700 1 --2- 172.21.15.39:0/4253764157 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22540feb90 0x7f22541962c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:31.724 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.710+0000 7f225259c700 1 -- 172.21.15.39:0/4253764157 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22540feb90 msgr2=0x7f22541962c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:31.724 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.710+0000 7f225259c700 1 --2- 172.21.15.39:0/4253764157 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22540feb90 0x7f22541962c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:31.724 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.710+0000 7f2252d9d700 1 --2- 172.21.15.39:0/4253764157 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2254105c00 0x7f225419a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:31.725 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.710+0000 7f225259c700 1 -- 172.21.15.39:0/4253764157 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2254105c00 msgr2=0x7f225419a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:31.725 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.710+0000 7f225259c700 1 --2- 172.21.15.39:0/4253764157 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2254105c00 0x7f225419a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:31.725 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.710+0000 7f225259c700 1 -- 172.21.15.39:0/4253764157 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f224c009d00 con 0x7f22540fe200 2023-07-21T19:07:31.725 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.710+0000 7f225259c700 1 --2- 172.21.15.39:0/4253764157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22540fe200 0x7f2254195d80 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f224400e9a0 tx=0x7f224400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:31.725 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.710+0000 7f224b7fe700 1 -- 172.21.15.39:0/4253764157 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2244009790 con 0x7f22540fe200 2023-07-21T19:07:31.725 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.710+0000 7f224b7fe700 1 -- 172.21.15.39:0/4253764157 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f224400cbd0 con 0x7f22540fe200 2023-07-21T19:07:31.725 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.710+0000 7f2258950700 1 -- 172.21.15.39:0/4253764157 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f225419b370 con 0x7f22540fe200 2023-07-21T19:07:31.726 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.710+0000 7f224b7fe700 1 -- 172.21.15.39:0/4253764157 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2244018680 con 0x7f22540fe200 2023-07-21T19:07:31.726 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.710+0000 7f2258950700 1 -- 172.21.15.39:0/4253764157 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22541a2bb0 con 0x7f22540fe200 2023-07-21T19:07:31.726 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.711+0000 7f2258950700 1 -- 172.21.15.39:0/4253764157 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2254066a00 con 0x7f22540fe200 2023-07-21T19:07:31.726 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.712+0000 7f224b7fe700 1 -- 172.21.15.39:0/4253764157 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f224400cd40 con 0x7f22540fe200 2023-07-21T19:07:31.727 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.713+0000 7f224b7fe700 1 --2- 172.21.15.39:0/4253764157 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f22400a12c0 0x7f22400a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:31.727 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.713+0000 7f224bfff700 1 --2- 172.21.15.39:0/4253764157 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f22400a12c0 0x7f22400a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:31.727 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.713+0000 7f224b7fe700 1 -- 172.21.15.39:0/4253764157 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f2244014070 con 0x7f22540fe200 2023-07-21T19:07:31.727 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.713+0000 7f224bfff700 1 --2- 172.21.15.39:0/4253764157 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f22400a12c0 0x7f22400a3770 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2254197390 tx=0x7f223c009260 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:31.727 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.716+0000 7f224b7fe700 1 -- 172.21.15.39:0/4253764157 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f2244096870 con 0x7f22540fe200 2023-07-21T19:07:31.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:31 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/1486377752' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T19:07:31.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:31 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/1625270160' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T19:07:31.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:31 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/1486377752' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T19:07:31.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:31 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/1625270160' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T19:07:31.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:31 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/1486377752' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T19:07:31.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:31 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/1625270160' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T19:07:31.940 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.930+0000 7f2258950700 1 -- 172.21.15.39:0/4253764157 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f22541971c0 con 0x7f22540fe200 2023-07-21T19:07:31.941 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.930+0000 7f224b7fe700 1 -- 172.21.15.39:0/4253764157 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 17 entries v124) v1 ==== 81+0+999 (secure 0 0 0) 0x7f2244099e90 con 0x7f22540fe200 2023-07-21T19:07:31.941 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/1762298871 2023-07-22T18:41:44.230582+0000 2023-07-21T19:07:31.941 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/1127767082 2023-07-22T18:34:07.039823+0000 2023-07-21T19:07:31.941 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6800/2116291712 2023-07-22T18:41:44.230582+0000 2023-07-21T19:07:31.941 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/3819152602 2023-07-22T18:33:40.852706+0000 2023-07-21T19:07:31.942 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/4249110368 2023-07-22T18:41:44.230582+0000 2023-07-21T19:07:31.942 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6801/2116291712 2023-07-22T18:41:44.230582+0000 2023-07-21T19:07:31.942 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/1275748831 2023-07-22T18:33:40.852706+0000 2023-07-21T19:07:31.942 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/545627125 2023-07-22T18:41:44.230582+0000 2023-07-21T19:07:31.942 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6800/889516631 2023-07-22T18:33:40.852706+0000 2023-07-21T19:07:31.942 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6800/2893866425 2023-07-22T18:34:07.039823+0000 2023-07-21T19:07:31.942 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/3636510379 2023-07-22T18:33:40.852706+0000 2023-07-21T19:07:31.943 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/19717002 2023-07-22T18:34:07.039823+0000 2023-07-21T19:07:31.943 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6801/2893866425 2023-07-22T18:34:07.039823+0000 2023-07-21T19:07:31.943 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/2902600414 2023-07-22T18:34:07.039823+0000 2023-07-21T19:07:31.943 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/1870486291 2023-07-22T18:41:44.230582+0000 2023-07-21T19:07:31.943 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6801/889516631 2023-07-22T18:33:40.852706+0000 2023-07-21T19:07:31.943 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/3797141824 2023-07-22T18:41:44.230582+0000 2023-07-21T19:07:31.944 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.932+0000 7f2258950700 1 -- 172.21.15.39:0/4253764157 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f22400a12c0 msgr2=0x7f22400a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:31.945 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.932+0000 7f2258950700 1 --2- 172.21.15.39:0/4253764157 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f22400a12c0 0x7f22400a3770 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2254197390 tx=0x7f223c009260 comp rx=0 tx=0).stop 2023-07-21T19:07:31.945 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.932+0000 7f2258950700 1 -- 172.21.15.39:0/4253764157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22540fe200 msgr2=0x7f2254195d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:31.945 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.932+0000 7f2258950700 1 --2- 172.21.15.39:0/4253764157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22540fe200 0x7f2254195d80 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f224400e9a0 tx=0x7f224400ed60 comp rx=0 tx=0).stop 2023-07-21T19:07:31.946 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.932+0000 7f2258950700 1 -- 172.21.15.39:0/4253764157 shutdown_connections 2023-07-21T19:07:31.946 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.932+0000 7f2258950700 1 --2- 172.21.15.39:0/4253764157 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f22400a12c0 0x7f22400a3770 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:31.946 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.932+0000 7f2258950700 1 --2- 172.21.15.39:0/4253764157 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2254105c00 0x7f225419a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:31.946 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.932+0000 7f2258950700 1 --2- 172.21.15.39:0/4253764157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22540fe200 0x7f2254195d80 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:31.946 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.932+0000 7f2258950700 1 --2- 172.21.15.39:0/4253764157 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22540feb90 0x7f22541962c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:31.946 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.932+0000 7f2258950700 1 -- 172.21.15.39:0/4253764157 >> 172.21.15.39:0/4253764157 conn(0x7f2254076200 msgr2=0x7f22540fc7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:31.946 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.932+0000 7f2258950700 1 -- 172.21.15.39:0/4253764157 shutdown_connections 2023-07-21T19:07:31.947 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:31.932+0000 7f2258950700 1 -- 172.21.15.39:0/4253764157 wait complete. 2023-07-21T19:07:31.947 INFO:teuthology.orchestra.run.smithi039.stderr:listed 17 entries 2023-07-21T19:07:31.955 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-07-21T19:07:31.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T19:07:31.956 DEBUG:teuthology.orchestra.run.smithi039:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-07-21T19:07:32.159 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi039.front.sepia.ceph.com 2023-07-21T19:07:32.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T19:07:32.159 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-07-21T19:07:32.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T19:07:32.354 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list 2023-07-21T19:07:32.532 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-07-21T19:07:32.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T19:07:32.533 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-21T19:07:32.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T19:07:32.716 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link delete ceph-brx 2023-07-21T19:07:32.843 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:32 smithi039 ceph-mon[96489]: pgmap v818: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.5 KiB/s wr, 12 op/s 2023-07-21T19:07:32.844 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:32 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/2260527571' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T19:07:32.844 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:32 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/4253764157' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-21T19:07:32.859 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:32 smithi193 ceph-mon[98204]: pgmap v818: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.5 KiB/s wr, 12 op/s 2023-07-21T19:07:32.859 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:32 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/2260527571' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T19:07:32.859 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:32 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/4253764157' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-21T19:07:32.878 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:32 smithi155 ceph-mon[97934]: pgmap v818: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.5 KiB/s wr, 12 op/s 2023-07-21T19:07:32.878 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:32 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/2260527571' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T19:07:32.878 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:32 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/4253764157' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-21T19:07:33.015 INFO:tasks.kclient:Unmounting kernel clients... 2023-07-21T19:07:33.016 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-07-21T19:07:33.016 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-21T19:07:33.152 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=32,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15663 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/045d3f91276f78a764bed228b0cbbdb2b0f8f9b326edaef40eb7101801650c20/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/GGT6VB746GUDEICMPC5J37OJT7:/var/lib/containers/storage/overlay/l/KLNMNARWHJYP6SBEMVNBHKBOAE,upperdir=/var/lib/containers/storage/overlay/045d3f91276f78a764bed228b0cbbdb2b0f8f9b326edaef40eb7101801650c20/diff,workdir=/var/lib/containers/storage/overlay/045d3f91276f78a764bed228b0cbbdb2b0f8f9b326edaef40eb7101801650c20/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/29a1d91ccf47de6efccf8e5a532c9e24046153b80b515b8578efe15e2b94b5f1/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/GGT6VB746GUDEICMPC5J37OJT7:/var/lib/containers/storage/overlay/l/KLNMNARWHJYP6SBEMVNBHKBOAE,upperdir=/var/lib/containers/storage/overlay/29a1d91ccf47de6efccf8e5a532c9e24046153b80b515b8578efe15e2b94b5f1/diff,workdir=/var/lib/containers/storage/overlay/29a1d91ccf47de6efccf8e5a532c9e24046153b80b515b8578efe15e2b94b5f1/work,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,relatime 0 0 overlay /var/lib/containers/storage/overlay/834a27dd7426bec279c39d860a38bfddc64fde183dbbc5649dad297685609a31/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/GGT6VB746GUDEICMPC5J37OJT7:/var/lib/containers/storage/overlay/l/KLNMNARWHJYP6SBEMVNBHKBOAE,upperdir=/var/lib/containers/storage/overlay/834a27dd7426bec279c39d860a38bfddc64fde183dbbc5649dad297685609a31/diff,workdir=/var/lib/containers/storage/overlay/834a27dd7426bec279c39d860a38bfddc64fde183dbbc5649dad297685609a31/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/a5de08269cb82b48d1ba5cef30fb941be7b12dec40659f347b67b4deeb8d3967/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/GGT6VB746GUDEICMPC5J37OJT7:/var/lib/containers/storage/overlay/l/KLNMNARWHJYP6SBEMVNBHKBOAE,upperdir=/var/lib/containers/storage/overlay/a5de08269cb82b48d1ba5cef30fb941be7b12dec40659f347b67b4deeb8d3967/diff,workdir=/var/lib/containers/storage/overlay/a5de08269cb82b48d1ba5cef30fb941be7b12dec40659f347b67b4deeb8d3967/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/9250affb0ec7ecc333ea4f96f88ebbc5695fad8b97590d58fb3d4a6ffdfef9ec/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/GGT6VB746GUDEICMPC5J37OJT7:/var/lib/containers/storage/overlay/l/KLNMNARWHJYP6SBEMVNBHKBOAE,upperdir=/var/lib/containers/storage/overlay/9250affb0ec7ecc333ea4f96f88ebbc5695fad8b97590d58fb3d4a6ffdfef9ec/diff,workdir=/var/lib/containers/storage/overlay/9250affb0ec7ecc333ea4f96f88ebbc5695fad8b97590d58fb3d4a6ffdfef9ec/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/8070146e71f824c4015d485153d03eb574051581f257498f34f3085272dec213/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/GGT6VB746GUDEICMPC5J37OJT7:/var/lib/containers/storage/overlay/l/KLNMNARWHJYP6SBEMVNBHKBOAE,upperdir=/var/lib/containers/storage/overlay/8070146e71f824c4015d485153d03eb574051581f257498f34f3085272dec213/diff,workdir=/var/lib/containers/storage/overlay/8070146e71f824c4015d485153d03eb574051581f257498f34f3085272dec213/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/18f2de2c57b0f262f376f2c0997b4375d32b4725def8c1e1a0d0beec0c85b82e/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/GGT6VB746GUDEICMPC5J37OJT7:/var/lib/containers/storage/overlay/l/KLNMNARWHJYP6SBEMVNBHKBOAE,upperdir=/var/lib/containers/storage/overlay/18f2de2c57b0f262f376f2c0997b4375d32b4725def8c1e1a0d0beec0c85b82e/diff,workdir=/var/lib/containers/storage/overlay/18f2de2c57b0f262f376f2c0997b4375d32b4725def8c1e1a0d0beec0c85b82e/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/ec41a7c6f323f223cc7b9f3fde1d80f7d9c7058e79112127d79bbc83d8bd831f/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/GGT6VB746GUDEICMPC5J37OJT7:/var/lib/containers/storage/overlay/l/KLNMNARWHJYP6SBEMVNBHKBOAE,upperdir=/var/lib/containers/storage/overlay/ec41a7c6f323f223cc7b9f3fde1d80f7d9c7058e79112127d79bbc83d8bd831f/diff,workdir=/var/lib/containers/storage/overlay/ec41a7c6f323f223cc7b9f3fde1d80f7d9c7058e79112127d79bbc83d8bd831f/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/60c2c804b786ca4e5173fe8d7fb3615d0c44af7e88c2604b263f2195c2ea2584/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/GGT6VB746GUDEICMPC5J37OJT7:/var/lib/containers/storage/overlay/l/KLNMNARWHJYP6SBEMVNBHKBOAE,upperdir=/var/lib/containers/storage/overlay/60c2c804b786ca4e5173fe8d7fb3615d0c44af7e88c2604b263f2195c2ea2584/diff,workdir=/var/lib/containers/storage/overlay/60c2c804b786ca4e5173fe8d7fb3615d0c44af7e88c2604b263f2195c2ea2584/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/2579199dab4e144d89cd8bfc3b29c7b320be3a99ff522addac4b85a1bea0943d/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/GGT6VB746GUDEICMPC5J37OJT7:/var/lib/containers/storage/overlay/l/KLNMNARWHJYP6SBEMVNBHKBOAE,upperdir=/var/lib/containers/storage/overlay/2579199dab4e144d89cd8bfc3b29c7b320be3a99ff522addac4b85a1bea0943d/diff,workdir=/var/lib/containers/storage/overlay/2579199dab4e144d89cd8bfc3b29c7b320be3a99ff522addac4b85a1bea0943d/work,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 2023-07-21T19:07:33.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T19:07:33.153 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list 2023-07-21T19:07:33.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T19:07:33.205 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link delete ceph-brx 2023-07-21T19:07:33.319 INFO:teuthology.orchestra.run.smithi039.stderr:Cannot find device "ceph-brx" 2023-07-21T19:07:33.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T19:07:33.321 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2023-07-21T19:07:33.337 INFO:tasks.cephadm:Teardown begin 2023-07-21T19:07:33.339 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-07-21T19:07:33.339 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T19:07:33.467 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.456+0000 7f95d9fca700 1 -- 172.21.15.39:0/1701984698 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95d40fe9d0 msgr2=0x7f95d40fee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:33.467 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.456+0000 7f95d9fca700 1 --2- 172.21.15.39:0/1701984698 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95d40fe9d0 0x7f95d40fee40 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f95c8009990 tx=0x7f95c8009ca0 comp rx=0 tx=0).stop 2023-07-21T19:07:33.468 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.456+0000 7f95d9fca700 1 -- 172.21.15.39:0/1701984698 shutdown_connections 2023-07-21T19:07:33.468 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.456+0000 7f95d9fca700 1 --2- 172.21.15.39:0/1701984698 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95d41049e0 0x7f95d4104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:33.468 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.456+0000 7f95d9fca700 1 --2- 172.21.15.39:0/1701984698 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95d40ff380 0x7f95d410b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:33.468 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.456+0000 7f95d9fca700 1 --2- 172.21.15.39:0/1701984698 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95d40fe9d0 0x7f95d40fee40 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:33.468 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.456+0000 7f95d9fca700 1 -- 172.21.15.39:0/1701984698 >> 172.21.15.39:0/1701984698 conn(0x7f95d40fa490 msgr2=0x7f95d40fc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:33.469 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.456+0000 7f95d9fca700 1 -- 172.21.15.39:0/1701984698 shutdown_connections 2023-07-21T19:07:33.469 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.457+0000 7f95d9fca700 1 -- 172.21.15.39:0/1701984698 wait complete. 2023-07-21T19:07:33.472 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.457+0000 7f95d9fca700 1 Processor -- start 2023-07-21T19:07:33.472 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.458+0000 7f95d9fca700 1 -- start start 2023-07-21T19:07:33.472 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.458+0000 7f95d9fca700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95d40fe9d0 0x7f95d419a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:33.473 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.458+0000 7f95d9fca700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95d40ff380 0x7f95d419acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:33.473 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.458+0000 7f95d9fca700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95d41049e0 0x7f95d4194820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:33.473 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.458+0000 7f95d9fca700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d4194e90 con 0x7f95d40ff380 2023-07-21T19:07:33.473 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.458+0000 7f95d9fca700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d4195000 con 0x7f95d41049e0 2023-07-21T19:07:33.473 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.458+0000 7f95d9fca700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d41951c0 con 0x7f95d40fe9d0 2023-07-21T19:07:33.473 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.458+0000 7f95d2ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95d40ff380 0x7f95d419acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:33.473 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.458+0000 7f95d2ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95d40ff380 0x7f95d419acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35334/0 (socket says 172.21.15.39:35334) 2023-07-21T19:07:33.474 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.458+0000 7f95d37fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95d40fe9d0 0x7f95d419a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:33.474 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.458+0000 7f95d2ffd700 1 -- 172.21.15.39:0/966135522 learned_addr learned my addr 172.21.15.39:0/966135522 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:07:33.474 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.458+0000 7f95d3fff700 1 --2- 172.21.15.39:0/966135522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95d41049e0 0x7f95d4194820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:33.474 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.458+0000 7f95d2ffd700 1 -- 172.21.15.39:0/966135522 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95d40fe9d0 msgr2=0x7f95d419a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:33.474 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.458+0000 7f95d2ffd700 1 --2- 172.21.15.39:0/966135522 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95d40fe9d0 0x7f95d419a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:33.474 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.458+0000 7f95d2ffd700 1 -- 172.21.15.39:0/966135522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95d41049e0 msgr2=0x7f95d4194820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:33.474 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.458+0000 7f95d2ffd700 1 --2- 172.21.15.39:0/966135522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95d41049e0 0x7f95d4194820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:33.475 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.458+0000 7f95d2ffd700 1 -- 172.21.15.39:0/966135522 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c80095f0 con 0x7f95d40ff380 2023-07-21T19:07:33.475 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.458+0000 7f95d3fff700 1 --2- 172.21.15.39:0/966135522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95d41049e0 0x7f95d4194820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:07:33.475 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.459+0000 7f95d2ffd700 1 --2- 172.21.15.39:0/966135522 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95d40ff380 0x7f95d419acf0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f95c80095c0 tx=0x7f95c801a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:33.475 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.459+0000 7f95d0ff9700 1 -- 172.21.15.39:0/966135522 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c801f070 con 0x7f95d40ff380 2023-07-21T19:07:33.475 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.459+0000 7f95d0ff9700 1 -- 172.21.15.39:0/966135522 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f95c800f460 con 0x7f95d40ff380 2023-07-21T19:07:33.475 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.459+0000 7f95d9fca700 1 -- 172.21.15.39:0/966135522 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95d4195440 con 0x7f95d40ff380 2023-07-21T19:07:33.475 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.459+0000 7f95d0ff9700 1 -- 172.21.15.39:0/966135522 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c8025680 con 0x7f95d40ff380 2023-07-21T19:07:33.475 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.459+0000 7f95d9fca700 1 -- 172.21.15.39:0/966135522 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95d4067850 con 0x7f95d40ff380 2023-07-21T19:07:33.476 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.460+0000 7f95d9fca700 1 -- 172.21.15.39:0/966135522 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95d4067300 con 0x7f95d40ff380 2023-07-21T19:07:33.476 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.461+0000 7f95d0ff9700 1 -- 172.21.15.39:0/966135522 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f95c8023070 con 0x7f95d40ff380 2023-07-21T19:07:33.476 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.461+0000 7f95d0ff9700 1 --2- 172.21.15.39:0/966135522 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f95bc0a1300 0x7f95bc0a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:33.476 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.462+0000 7f95d37fe700 1 --2- 172.21.15.39:0/966135522 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f95bc0a1300 0x7f95bc0a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:33.476 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.462+0000 7f95d0ff9700 1 -- 172.21.15.39:0/966135522 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f95c80d0210 con 0x7f95d40ff380 2023-07-21T19:07:33.477 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.462+0000 7f95d37fe700 1 --2- 172.21.15.39:0/966135522 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f95bc0a1300 0x7f95bc0a37b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f95d40ffb10 tx=0x7f95c4007400 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:33.477 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.467+0000 7f95d0ff9700 1 -- 172.21.15.39:0/966135522 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f95c8099650 con 0x7f95d40ff380 2023-07-21T19:07:33.725 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.714+0000 7f95d9fca700 1 -- 172.21.15.39:0/966135522 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f95d4195d30 con 0x7f95d40ff380 2023-07-21T19:07:33.726 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.715+0000 7f95d0ff9700 1 -- 172.21.15.39:0/966135522 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) v1 ==== 95+0+10911 (secure 0 0 0) 0x7f95c809cc70 con 0x7f95d40ff380 2023-07-21T19:07:33.726 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T19:07:33.727 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":110,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/147318681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":147318681},{"type":"v1","addr":"172.21.15.39:6835","nonce":147318681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/1062891760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1062891760},{"type":"v1","addr":"172.21.15.39:6837","nonce":1062891760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":110,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T18:40:04.594553+0000","modified":"2023-07-21T19:06:58.580946+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24536,"mds_1":24500,"mds_2":24487,"mds_3":24464,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/3184273349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":3184273349},{"type":"v1","addr":"172.21.15.39:6839","nonce":3184273349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":52,"addr":"172.21.15.39:6841/640981220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":640981220},{"type":"v1","addr":"172.21.15.39:6841","nonce":640981220}]},"join_fscid":-1,"export_targets":[0,1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3106166804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3106166804},{"type":"v1","addr":"172.21.15.193:6833","nonce":3106166804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24437":{"gid":24437,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/1034755277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1034755277},{"type":"v1","addr":"172.21.15.155:6833","nonce":1034755277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/381560657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":381560657},{"type":"v1","addr":"172.21.15.193:6835","nonce":381560657}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24464":{"gid":24464,"name":"e","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.155:6835/3854856286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3854856286},{"type":"v1","addr":"172.21.15.155:6835","nonce":3854856286}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":53,"addr":"172.21.15.193:6839/3148488639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3148488639},{"type":"v1","addr":"172.21.15.193:6839","nonce":3148488639}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":54,"addr":"172.21.15.155:6837/947670056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":947670056},{"type":"v1","addr":"172.21.15.155:6837","nonce":947670056}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6839/431690342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":431690342},{"type":"v1","addr":"172.21.15.155:6839","nonce":431690342}]},"join_fscid":-1,"export_targets":[1,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_27040":{"gid":27040,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6837/2158259657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2158259657},{"type":"v1","addr":"172.21.15.193:6837","nonce":2158259657}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T19:07:33.728 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.717+0000 7f95d9fca700 1 -- 172.21.15.39:0/966135522 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f95bc0a1300 msgr2=0x7f95bc0a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:33.728 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.717+0000 7f95d9fca700 1 --2- 172.21.15.39:0/966135522 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f95bc0a1300 0x7f95bc0a37b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f95d40ffb10 tx=0x7f95c4007400 comp rx=0 tx=0).stop 2023-07-21T19:07:33.728 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.717+0000 7f95d9fca700 1 -- 172.21.15.39:0/966135522 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95d40ff380 msgr2=0x7f95d419acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:33.728 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.717+0000 7f95d9fca700 1 --2- 172.21.15.39:0/966135522 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95d40ff380 0x7f95d419acf0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f95c80095c0 tx=0x7f95c801a8c0 comp rx=0 tx=0).stop 2023-07-21T19:07:33.728 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.717+0000 7f95d9fca700 1 -- 172.21.15.39:0/966135522 shutdown_connections 2023-07-21T19:07:33.729 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.717+0000 7f95d9fca700 1 --2- 172.21.15.39:0/966135522 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f95bc0a1300 0x7f95bc0a37b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:33.729 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.717+0000 7f95d9fca700 1 --2- 172.21.15.39:0/966135522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95d41049e0 0x7f95d4194820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:33.729 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.717+0000 7f95d9fca700 1 --2- 172.21.15.39:0/966135522 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95d40fe9d0 0x7f95d419a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:33.729 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.717+0000 7f95d9fca700 1 --2- 172.21.15.39:0/966135522 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95d40ff380 0x7f95d419acf0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:33.729 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.717+0000 7f95d9fca700 1 -- 172.21.15.39:0/966135522 >> 172.21.15.39:0/966135522 conn(0x7f95d40fa490 msgr2=0x7f95d4105850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:33.730 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.717+0000 7f95d9fca700 1 -- 172.21.15.39:0/966135522 shutdown_connections 2023-07-21T19:07:33.730 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.717+0000 7f95d9fca700 1 -- 172.21.15.39:0/966135522 wait complete. 2023-07-21T19:07:33.730 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 110 2023-07-21T19:07:33.741 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T19:07:33.866 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.855+0000 7f89db1c9700 1 -- 172.21.15.39:0/529717046 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89d41004a0 msgr2=0x7f89d410c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:33.866 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.855+0000 7f89db1c9700 1 --2- 172.21.15.39:0/529717046 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89d41004a0 0x7f89d410c930 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f89cc00b0a0 tx=0x7f89cc00b3b0 comp rx=0 tx=0).stop 2023-07-21T19:07:33.867 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.855+0000 7f89db1c9700 1 -- 172.21.15.39:0/529717046 shutdown_connections 2023-07-21T19:07:33.867 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.855+0000 7f89db1c9700 1 --2- 172.21.15.39:0/529717046 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89d4105b00 0x7f89d4105ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:33.867 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.855+0000 7f89db1c9700 1 --2- 172.21.15.39:0/529717046 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89d41004a0 0x7f89d410c930 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:33.867 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.855+0000 7f89db1c9700 1 --2- 172.21.15.39:0/529717046 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89d40ffaf0 0x7f89d40fff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:33.867 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.856+0000 7f89db1c9700 1 -- 172.21.15.39:0/529717046 >> 172.21.15.39:0/529717046 conn(0x7f89d40fb5d0 msgr2=0x7f89d40fd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:33.867 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.856+0000 7f89db1c9700 1 -- 172.21.15.39:0/529717046 shutdown_connections 2023-07-21T19:07:33.867 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.856+0000 7f89db1c9700 1 -- 172.21.15.39:0/529717046 wait complete. 2023-07-21T19:07:33.869 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.856+0000 7f89db1c9700 1 Processor -- start 2023-07-21T19:07:33.870 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.856+0000 7f89db1c9700 1 -- start start 2023-07-21T19:07:33.870 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.856+0000 7f89db1c9700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89d40ffaf0 0x7f89d4072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:33.870 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.856+0000 7f89db1c9700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89d41004a0 0x7f89d4072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:33.870 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.856+0000 7f89db1c9700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89d4105b00 0x7f89d41a3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:33.870 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.856+0000 7f89db1c9700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89d4073580 con 0x7f89d4105b00 2023-07-21T19:07:33.870 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.856+0000 7f89db1c9700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89d41a42f0 con 0x7f89d41004a0 2023-07-21T19:07:33.870 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.856+0000 7f89db1c9700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89d41a4430 con 0x7f89d40ffaf0 2023-07-21T19:07:33.871 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.857+0000 7f89d9766700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89d4105b00 0x7f89d41a3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:33.871 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.857+0000 7f89d9766700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89d4105b00 0x7f89d41a3db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35350/0 (socket says 172.21.15.39:35350) 2023-07-21T19:07:33.871 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.857+0000 7f89d9766700 1 -- 172.21.15.39:0/3037794595 learned_addr learned my addr 172.21.15.39:0/3037794595 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:07:33.871 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.857+0000 7f89d8f65700 1 --2- 172.21.15.39:0/3037794595 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89d40ffaf0 0x7f89d4072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:33.871 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.857+0000 7f89d3fff700 1 --2- 172.21.15.39:0/3037794595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89d41004a0 0x7f89d4072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:33.871 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.857+0000 7f89d9766700 1 -- 172.21.15.39:0/3037794595 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89d40ffaf0 msgr2=0x7f89d4072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:33.871 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.857+0000 7f89d9766700 1 --2- 172.21.15.39:0/3037794595 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89d40ffaf0 0x7f89d4072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:33.871 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.857+0000 7f89d9766700 1 -- 172.21.15.39:0/3037794595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89d41004a0 msgr2=0x7f89d4072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:33.872 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.857+0000 7f89d9766700 1 --2- 172.21.15.39:0/3037794595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89d41004a0 0x7f89d4072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:33.872 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.857+0000 7f89d9766700 1 -- 172.21.15.39:0/3037794595 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89cc009d00 con 0x7f89d4105b00 2023-07-21T19:07:33.872 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.857+0000 7f89d8f65700 1 --2- 172.21.15.39:0/3037794595 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89d40ffaf0 0x7f89d4072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:07:33.872 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.857+0000 7f89d9766700 1 --2- 172.21.15.39:0/3037794595 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89d4105b00 0x7f89d41a3db0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f89cc00b830 tx=0x7f89cc007a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:33.872 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.857+0000 7f89d1ffb700 1 -- 172.21.15.39:0/3037794595 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89cc00e040 con 0x7f89d4105b00 2023-07-21T19:07:33.872 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.857+0000 7f89d1ffb700 1 -- 172.21.15.39:0/3037794595 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89cc004e10 con 0x7f89d4105b00 2023-07-21T19:07:33.872 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.857+0000 7f89db1c9700 1 -- 172.21.15.39:0/3037794595 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89d41a4630 con 0x7f89d4105b00 2023-07-21T19:07:33.873 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.857+0000 7f89d1ffb700 1 -- 172.21.15.39:0/3037794595 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89cc01d480 con 0x7f89d4105b00 2023-07-21T19:07:33.873 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.857+0000 7f89db1c9700 1 -- 172.21.15.39:0/3037794595 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89d41a4ab0 con 0x7f89d4105b00 2023-07-21T19:07:33.873 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.858+0000 7f89db1c9700 1 -- 172.21.15.39:0/3037794595 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89d4066a70 con 0x7f89d4105b00 2023-07-21T19:07:33.874 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.860+0000 7f89d1ffb700 1 -- 172.21.15.39:0/3037794595 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f89cc003a00 con 0x7f89d4105b00 2023-07-21T19:07:33.875 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.860+0000 7f89d1ffb700 1 --2- 172.21.15.39:0/3037794595 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f89c40a1300 0x7f89c40a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:33.875 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.861+0000 7f89d8f65700 1 --2- 172.21.15.39:0/3037794595 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f89c40a1300 0x7f89c40a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:33.875 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.861+0000 7f89d1ffb700 1 -- 172.21.15.39:0/3037794595 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f89cc0cebc0 con 0x7f89d4105b00 2023-07-21T19:07:33.875 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.861+0000 7f89d8f65700 1 --2- 172.21.15.39:0/3037794595 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f89c40a1300 0x7f89c40a37b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f89c8005fd0 tx=0x7f89c800a5e0 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:33.875 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:33.864+0000 7f89d1ffb700 1 -- 172.21.15.39:0/3037794595 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f89cc098060 con 0x7f89d4105b00 2023-07-21T19:07:34.119 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.108+0000 7f89db1c9700 1 -- 172.21.15.39:0/3037794595 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f89d4073d40 con 0x7f89d4105b00 2023-07-21T19:07:34.119 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.109+0000 7f89d1ffb700 1 -- 172.21.15.39:0/3037794595 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) v1 ==== 95+0+10911 (secure 0 0 0) 0x7f89cc09b680 con 0x7f89d4105b00 2023-07-21T19:07:34.120 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T19:07:34.120 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":110,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/147318681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":147318681},{"type":"v1","addr":"172.21.15.39:6835","nonce":147318681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/1062891760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1062891760},{"type":"v1","addr":"172.21.15.39:6837","nonce":1062891760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":110,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T18:40:04.594553+0000","modified":"2023-07-21T19:06:58.580946+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24536,"mds_1":24500,"mds_2":24487,"mds_3":24464,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/3184273349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":3184273349},{"type":"v1","addr":"172.21.15.39:6839","nonce":3184273349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":52,"addr":"172.21.15.39:6841/640981220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":640981220},{"type":"v1","addr":"172.21.15.39:6841","nonce":640981220}]},"join_fscid":-1,"export_targets":[0,1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3106166804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3106166804},{"type":"v1","addr":"172.21.15.193:6833","nonce":3106166804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24437":{"gid":24437,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/1034755277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1034755277},{"type":"v1","addr":"172.21.15.155:6833","nonce":1034755277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/381560657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":381560657},{"type":"v1","addr":"172.21.15.193:6835","nonce":381560657}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24464":{"gid":24464,"name":"e","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.155:6835/3854856286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3854856286},{"type":"v1","addr":"172.21.15.155:6835","nonce":3854856286}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":53,"addr":"172.21.15.193:6839/3148488639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3148488639},{"type":"v1","addr":"172.21.15.193:6839","nonce":3148488639}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":54,"addr":"172.21.15.155:6837/947670056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":947670056},{"type":"v1","addr":"172.21.15.155:6837","nonce":947670056}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6839/431690342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":431690342},{"type":"v1","addr":"172.21.15.155:6839","nonce":431690342}]},"join_fscid":-1,"export_targets":[1,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_27040":{"gid":27040,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6837/2158259657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2158259657},{"type":"v1","addr":"172.21.15.193:6837","nonce":2158259657}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T19:07:34.122 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.111+0000 7f89db1c9700 1 -- 172.21.15.39:0/3037794595 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f89c40a1300 msgr2=0x7f89c40a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:34.122 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.111+0000 7f89db1c9700 1 --2- 172.21.15.39:0/3037794595 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f89c40a1300 0x7f89c40a37b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f89c8005fd0 tx=0x7f89c800a5e0 comp rx=0 tx=0).stop 2023-07-21T19:07:34.122 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.111+0000 7f89db1c9700 1 -- 172.21.15.39:0/3037794595 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89d4105b00 msgr2=0x7f89d41a3db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:34.123 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.111+0000 7f89db1c9700 1 --2- 172.21.15.39:0/3037794595 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89d4105b00 0x7f89d41a3db0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f89cc00b830 tx=0x7f89cc007a60 comp rx=0 tx=0).stop 2023-07-21T19:07:34.123 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.111+0000 7f89db1c9700 1 -- 172.21.15.39:0/3037794595 shutdown_connections 2023-07-21T19:07:34.123 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.111+0000 7f89db1c9700 1 --2- 172.21.15.39:0/3037794595 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89d4105b00 0x7f89d41a3db0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:34.123 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.111+0000 7f89db1c9700 1 --2- 172.21.15.39:0/3037794595 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89d40ffaf0 0x7f89d4072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:34.123 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.111+0000 7f89db1c9700 1 --2- 172.21.15.39:0/3037794595 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f89c40a1300 0x7f89c40a37b0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:34.123 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.111+0000 7f89db1c9700 1 --2- 172.21.15.39:0/3037794595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89d41004a0 0x7f89d4072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:34.123 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.111+0000 7f89db1c9700 1 -- 172.21.15.39:0/3037794595 >> 172.21.15.39:0/3037794595 conn(0x7f89d40fb5d0 msgr2=0x7f89d4103600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:34.124 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.111+0000 7f89db1c9700 1 -- 172.21.15.39:0/3037794595 shutdown_connections 2023-07-21T19:07:34.124 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.111+0000 7f89db1c9700 1 -- 172.21.15.39:0/3037794595 wait complete. 2023-07-21T19:07:34.125 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 110 2023-07-21T19:07:34.135 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T19:07:34.265 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.252+0000 7f7d9b65a700 1 -- 172.21.15.39:0/2517861134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d940ff140 msgr2=0x7f7d940ff5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:34.266 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.252+0000 7f7d9b65a700 1 --2- 172.21.15.39:0/2517861134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d940ff140 0x7f7d940ff5b0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f7d900099c0 tx=0x7f7d90009cd0 comp rx=0 tx=0).stop 2023-07-21T19:07:34.266 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.252+0000 7f7d9b65a700 1 -- 172.21.15.39:0/2517861134 shutdown_connections 2023-07-21T19:07:34.266 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.252+0000 7f7d9b65a700 1 --2- 172.21.15.39:0/2517861134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d94103990 0x7f7d94103d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:34.266 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.252+0000 7f7d9b65a700 1 --2- 172.21.15.39:0/2517861134 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d940ffaf0 0x7f7d9410b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:34.266 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.252+0000 7f7d9b65a700 1 --2- 172.21.15.39:0/2517861134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d940ff140 0x7f7d940ff5b0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:34.266 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.252+0000 7f7d9b65a700 1 -- 172.21.15.39:0/2517861134 >> 172.21.15.39:0/2517861134 conn(0x7f7d94074e00 msgr2=0x7f7d94075200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:34.266 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.252+0000 7f7d9b65a700 1 -- 172.21.15.39:0/2517861134 shutdown_connections 2023-07-21T19:07:34.267 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.252+0000 7f7d9b65a700 1 -- 172.21.15.39:0/2517861134 wait complete. 2023-07-21T19:07:34.267 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.253+0000 7f7d9b65a700 1 Processor -- start 2023-07-21T19:07:34.267 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.253+0000 7f7d9b65a700 1 -- start start 2023-07-21T19:07:34.267 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.253+0000 7f7d9b65a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d940ff140 0x7f7d94100680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:34.267 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.253+0000 7f7d9b65a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d940ffaf0 0x7f7d94100bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:34.267 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.253+0000 7f7d9b65a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d94103990 0x7f7d941a3d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:34.268 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.253+0000 7f7d9b65a700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d941013f0 con 0x7f7d94103990 2023-07-21T19:07:34.268 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.253+0000 7f7d9b65a700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d941a4250 con 0x7f7d940ffaf0 2023-07-21T19:07:34.268 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.253+0000 7f7d9b65a700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d941a4390 con 0x7f7d940ff140 2023-07-21T19:07:34.268 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.253+0000 7f7d99bf7700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d94103990 0x7f7d941a3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:34.268 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.253+0000 7f7d99bf7700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d94103990 0x7f7d941a3d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35364/0 (socket says 172.21.15.39:35364) 2023-07-21T19:07:34.268 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.254+0000 7f7d99bf7700 1 -- 172.21.15.39:0/3908306900 learned_addr learned my addr 172.21.15.39:0/3908306900 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:07:34.268 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.254+0000 7f7d993f6700 1 --2- 172.21.15.39:0/3908306900 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d940ff140 0x7f7d94100680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:34.269 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.254+0000 7f7d98bf5700 1 --2- 172.21.15.39:0/3908306900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d940ffaf0 0x7f7d94100bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:34.269 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.254+0000 7f7d99bf7700 1 -- 172.21.15.39:0/3908306900 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d940ff140 msgr2=0x7f7d94100680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:34.269 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.254+0000 7f7d99bf7700 1 --2- 172.21.15.39:0/3908306900 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d940ff140 0x7f7d94100680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:34.269 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.254+0000 7f7d99bf7700 1 -- 172.21.15.39:0/3908306900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d940ffaf0 msgr2=0x7f7d94100bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:34.269 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.254+0000 7f7d99bf7700 1 --2- 172.21.15.39:0/3908306900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d940ffaf0 0x7f7d94100bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:34.269 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.254+0000 7f7d99bf7700 1 -- 172.21.15.39:0/3908306900 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d900095f0 con 0x7f7d94103990 2023-07-21T19:07:34.269 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.254+0000 7f7d99bf7700 1 --2- 172.21.15.39:0/3908306900 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d94103990 0x7f7d941a3d10 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c00ef90 tx=0x7f7d8c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:34.269 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.254+0000 7f7d867fc700 1 -- 172.21.15.39:0/3908306900 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c018070 con 0x7f7d94103990 2023-07-21T19:07:34.270 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.254+0000 7f7d867fc700 1 -- 172.21.15.39:0/3908306900 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d8c00f040 con 0x7f7d94103990 2023-07-21T19:07:34.270 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.254+0000 7f7d9b65a700 1 -- 172.21.15.39:0/3908306900 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d941a4590 con 0x7f7d94103990 2023-07-21T19:07:34.270 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.254+0000 7f7d867fc700 1 -- 172.21.15.39:0/3908306900 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c00cda0 con 0x7f7d94103990 2023-07-21T19:07:34.270 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.254+0000 7f7d9b65a700 1 -- 172.21.15.39:0/3908306900 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d941a4a20 con 0x7f7d94103990 2023-07-21T19:07:34.270 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.255+0000 7f7d9b65a700 1 -- 172.21.15.39:0/3908306900 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d94066a70 con 0x7f7d94103990 2023-07-21T19:07:34.271 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.256+0000 7f7d867fc700 1 -- 172.21.15.39:0/3908306900 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f7d8c003a00 con 0x7f7d94103990 2023-07-21T19:07:34.271 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.257+0000 7f7d867fc700 1 --2- 172.21.15.39:0/3908306900 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f7d800a1300 0x7f7d800a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:34.271 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.257+0000 7f7d993f6700 1 --2- 172.21.15.39:0/3908306900 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f7d800a1300 0x7f7d800a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:34.272 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.257+0000 7f7d867fc700 1 -- 172.21.15.39:0/3908306900 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f7d8c0cd2d0 con 0x7f7d94103990 2023-07-21T19:07:34.272 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.257+0000 7f7d993f6700 1 --2- 172.21.15.39:0/3908306900 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f7d800a1300 0x7f7d800a37b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7d88009c60 tx=0x7f7d88009260 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:34.272 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.260+0000 7f7d867fc700 1 -- 172.21.15.39:0/3908306900 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f7d8c096710 con 0x7f7d94103990 2023-07-21T19:07:34.473 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.463+0000 7f7d9b65a700 1 -- 172.21.15.39:0/3908306900 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7d94101bb0 con 0x7f7d94103990 2023-07-21T19:07:34.475 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.464+0000 7f7d867fc700 1 -- 172.21.15.39:0/3908306900 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) v1 ==== 74+0+22980 (secure 0 0 0) 0x7f7d8c099d30 con 0x7f7d94103990 2023-07-21T19:07:34.476 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-07-21T19:07:34.477 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":124,"fsid":"eaf8e25a-27f4-11ee-9b34-001a4aab830c","created":"2023-07-21T18:33:06.504979+0000","modified":"2023-07-21T19:07:16.405558+0000","last_up_change":"2023-07-21T18:39:23.531009+0000","last_in_change":"2023-07-21T18:39:05.993024+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T18:36:13.572583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T18:40:02.726735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T18:40:03.990112+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":46,"snap_epoch":123,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03502c07-2d0f-4723-8d0b-d34c9f5b91bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6803","nonce":860193857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6805","nonce":860193857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6809","nonce":860193857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":860193857},{"type":"v1","addr":"172.21.15.39:6807","nonce":860193857}]},"public_addr":"172.21.15.39:6803/860193857","cluster_addr":"172.21.15.39:6805/860193857","heartbeat_back_addr":"172.21.15.39:6809/860193857","heartbeat_front_addr":"172.21.15.39:6807/860193857","state":["exists","up"]},{"osd":1,"uuid":"0831a0c0-a1fa-4d30-9a28-513223aa69eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6801","nonce":3246721764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6803","nonce":3246721764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6807","nonce":3246721764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":3246721764},{"type":"v1","addr":"172.21.15.155:6805","nonce":3246721764}]},"public_addr":"172.21.15.155:6801/3246721764","cluster_addr":"172.21.15.155:6803/3246721764","heartbeat_back_addr":"172.21.15.155:6807/3246721764","heartbeat_front_addr":"172.21.15.155:6805/3246721764","state":["exists","up"]},{"osd":2,"uuid":"22df0003-bde9-48e4-b47a-cb8eafbab0db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6801","nonce":2563594449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6803","nonce":2563594449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6807","nonce":2563594449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":2563594449},{"type":"v1","addr":"172.21.15.193:6805","nonce":2563594449}]},"public_addr":"172.21.15.193:6801/2563594449","cluster_addr":"172.21.15.193:6803/2563594449","heartbeat_back_addr":"172.21.15.193:6807/2563594449","heartbeat_front_addr":"172.21.15.193:6805/2563594449","state":["exists","up"]},{"osd":3,"uuid":"54562558-a2e8-45dd-bc95-561569b9ae6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6811","nonce":863708386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6813","nonce":863708386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6817","nonce":863708386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":863708386},{"type":"v1","addr":"172.21.15.39:6815","nonce":863708386}]},"public_addr":"172.21.15.39:6811/863708386","cluster_addr":"172.21.15.39:6813/863708386","heartbeat_back_addr":"172.21.15.39:6817/863708386","heartbeat_front_addr":"172.21.15.39:6815/863708386","state":["exists","up"]},{"osd":4,"uuid":"491118e7-ae1e-495c-9eac-ef1cdcae9600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6809","nonce":3361780673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6811","nonce":3361780673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6815","nonce":3361780673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":3361780673},{"type":"v1","addr":"172.21.15.155:6813","nonce":3361780673}]},"public_addr":"172.21.15.155:6809/3361780673","cluster_addr":"172.21.15.155:6811/3361780673","heartbeat_back_addr":"172.21.15.155:6815/3361780673","heartbeat_front_addr":"172.21.15.155:6813/3361780673","state":["exists","up"]},{"osd":5,"uuid":"d267bc2c-42b3-4d96-b2d1-b186dc7e6b18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6809","nonce":2376867721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6811","nonce":2376867721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6815","nonce":2376867721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2376867721},{"type":"v1","addr":"172.21.15.193:6813","nonce":2376867721}]},"public_addr":"172.21.15.193:6809/2376867721","cluster_addr":"172.21.15.193:6811/2376867721","heartbeat_back_addr":"172.21.15.193:6815/2376867721","heartbeat_front_addr":"172.21.15.193:6813/2376867721","state":["exists","up"]},{"osd":6,"uuid":"beec8088-970a-473e-bd67-6eee7cbdef24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6819","nonce":2417770529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6821","nonce":2417770529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6825","nonce":2417770529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":2417770529},{"type":"v1","addr":"172.21.15.39:6823","nonce":2417770529}]},"public_addr":"172.21.15.39:6819/2417770529","cluster_addr":"172.21.15.39:6821/2417770529","heartbeat_back_addr":"172.21.15.39:6825/2417770529","heartbeat_front_addr":"172.21.15.39:6823/2417770529","state":["exists","up"]},{"osd":7,"uuid":"5a88ce30-1835-4254-90cc-804ed5cf1066","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6817","nonce":2542002111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6819","nonce":2542002111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6823","nonce":2542002111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2542002111},{"type":"v1","addr":"172.21.15.155:6821","nonce":2542002111}]},"public_addr":"172.21.15.155:6817/2542002111","cluster_addr":"172.21.15.155:6819/2542002111","heartbeat_back_addr":"172.21.15.155:6823/2542002111","heartbeat_front_addr":"172.21.15.155:6821/2542002111","state":["exists","up"]},{"osd":8,"uuid":"d964a183-3021-4ace-9a0c-aad00f2570e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6817","nonce":2985835085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6819","nonce":2985835085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6823","nonce":2985835085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2985835085},{"type":"v1","addr":"172.21.15.193:6821","nonce":2985835085}]},"public_addr":"172.21.15.193:6817/2985835085","cluster_addr":"172.21.15.193:6819/2985835085","heartbeat_back_addr":"172.21.15.193:6823/2985835085","heartbeat_front_addr":"172.21.15.193:6821/2985835085","state":["exists","up"]},{"osd":9,"uuid":"9725ea5c-49a3-4c99-bac4-e95069100b1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6827","nonce":1946493651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6829","nonce":1946493651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6833","nonce":1946493651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1946493651},{"type":"v1","addr":"172.21.15.39:6831","nonce":1946493651}]},"public_addr":"172.21.15.39:6827/1946493651","cluster_addr":"172.21.15.39:6829/1946493651","heartbeat_back_addr":"172.21.15.39:6833/1946493651","heartbeat_front_addr":"172.21.15.39:6831/1946493651","state":["exists","up"]},{"osd":10,"uuid":"1fd60ccb-92db-45f0-a580-2003db0976e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6825","nonce":1234615022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6827","nonce":1234615022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6831","nonce":1234615022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":1234615022},{"type":"v1","addr":"172.21.15.155:6829","nonce":1234615022}]},"public_addr":"172.21.15.155:6825/1234615022","cluster_addr":"172.21.15.155:6827/1234615022","heartbeat_back_addr":"172.21.15.155:6831/1234615022","heartbeat_front_addr":"172.21.15.155:6829/1234615022","state":["exists","up"]},{"osd":11,"uuid":"efb31ea1-4775-4e5f-8740-728d6f4aeea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6825","nonce":215104663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6827","nonce":215104663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6831","nonce":215104663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":215104663},{"type":"v1","addr":"172.21.15.193:6829","nonce":215104663}]},"public_addr":"172.21.15.193:6825/215104663","cluster_addr":"172.21.15.193:6827/215104663","heartbeat_back_addr":"172.21.15.193:6831/215104663","heartbeat_front_addr":"172.21.15.193:6829/215104663","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:35:30.211698+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:35:50.883846+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:36:11.124168+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:36:32.711905+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:36:54.442007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:37:14.843526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:37:36.587354+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:37:57.882374+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:38:18.729407+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:38:40.139231+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:39:01.290862+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:39:21.767537+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:0/1762298871":"2023-07-22T18:41:44.230582+0000","172.21.15.39:0/1127767082":"2023-07-22T18:34:07.039823+0000","172.21.15.39:6800/2116291712":"2023-07-22T18:41:44.230582+0000","172.21.15.39:0/3819152602":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/4249110368":"2023-07-22T18:41:44.230582+0000","172.21.15.39:6801/2116291712":"2023-07-22T18:41:44.230582+0000","172.21.15.39:0/1275748831":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/545627125":"2023-07-22T18:41:44.230582+0000","172.21.15.39:6800/889516631":"2023-07-22T18:33:40.852706+0000","172.21.15.39:6800/2893866425":"2023-07-22T18:34:07.039823+0000","172.21.15.39:0/3636510379":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/19717002":"2023-07-22T18:34:07.039823+0000","172.21.15.39:6801/2893866425":"2023-07-22T18:34:07.039823+0000","172.21.15.39:0/2902600414":"2023-07-22T18:34:07.039823+0000","172.21.15.39:0/1870486291":"2023-07-22T18:41:44.230582+0000","172.21.15.39:6801/889516631":"2023-07-22T18:33:40.852706+0000","172.21.15.39:0/3797141824":"2023-07-22T18:41:44.230582+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":34,"length":1},{"begin":46,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T19:07:34.478 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.467+0000 7f7d9b65a700 1 -- 172.21.15.39:0/3908306900 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f7d800a1300 msgr2=0x7f7d800a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:34.479 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.467+0000 7f7d9b65a700 1 --2- 172.21.15.39:0/3908306900 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f7d800a1300 0x7f7d800a37b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7d88009c60 tx=0x7f7d88009260 comp rx=0 tx=0).stop 2023-07-21T19:07:34.479 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.467+0000 7f7d9b65a700 1 -- 172.21.15.39:0/3908306900 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d94103990 msgr2=0x7f7d941a3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:34.479 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.467+0000 7f7d9b65a700 1 --2- 172.21.15.39:0/3908306900 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d94103990 0x7f7d941a3d10 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c00ef90 tx=0x7f7d8c00c3a0 comp rx=0 tx=0).stop 2023-07-21T19:07:34.480 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.467+0000 7f7d9b65a700 1 -- 172.21.15.39:0/3908306900 shutdown_connections 2023-07-21T19:07:34.480 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.467+0000 7f7d9b65a700 1 --2- 172.21.15.39:0/3908306900 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d94103990 0x7f7d941a3d10 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:34.480 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.467+0000 7f7d9b65a700 1 --2- 172.21.15.39:0/3908306900 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d940ff140 0x7f7d94100680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:34.481 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.467+0000 7f7d9b65a700 1 --2- 172.21.15.39:0/3908306900 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f7d800a1300 0x7f7d800a37b0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:34.481 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.467+0000 7f7d9b65a700 1 --2- 172.21.15.39:0/3908306900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d940ffaf0 0x7f7d94100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:34.481 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.467+0000 7f7d9b65a700 1 -- 172.21.15.39:0/3908306900 >> 172.21.15.39:0/3908306900 conn(0x7f7d94074e00 msgr2=0x7f7d940fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:34.481 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.467+0000 7f7d9b65a700 1 -- 172.21.15.39:0/3908306900 shutdown_connections 2023-07-21T19:07:34.482 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.467+0000 7f7d9b65a700 1 -- 172.21.15.39:0/3908306900 wait complete. 2023-07-21T19:07:34.490 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-07-21T19:07:34.618 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.608+0000 7f6188fb6700 1 -- 172.21.15.39:0/4170467075 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61840fe9d0 msgr2=0x7f61840fee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:34.618 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.608+0000 7f6188fb6700 1 --2- 172.21.15.39:0/4170467075 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61840fe9d0 0x7f61840fee40 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f6178009990 tx=0x7f6178009ca0 comp rx=0 tx=0).stop 2023-07-21T19:07:34.619 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.608+0000 7f6188fb6700 1 -- 172.21.15.39:0/4170467075 shutdown_connections 2023-07-21T19:07:34.619 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.608+0000 7f6188fb6700 1 --2- 172.21.15.39:0/4170467075 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f61841049e0 0x7f6184104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:34.619 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.608+0000 7f6188fb6700 1 --2- 172.21.15.39:0/4170467075 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f61840ff380 0x7f618410b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:34.619 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.608+0000 7f6188fb6700 1 --2- 172.21.15.39:0/4170467075 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61840fe9d0 0x7f61840fee40 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:34.619 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.608+0000 7f6188fb6700 1 -- 172.21.15.39:0/4170467075 >> 172.21.15.39:0/4170467075 conn(0x7f61840fa4b0 msgr2=0x7f61840fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:34.619 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.608+0000 7f6188fb6700 1 -- 172.21.15.39:0/4170467075 shutdown_connections 2023-07-21T19:07:34.619 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.608+0000 7f6188fb6700 1 -- 172.21.15.39:0/4170467075 wait complete. 2023-07-21T19:07:34.621 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.609+0000 7f6188fb6700 1 Processor -- start 2023-07-21T19:07:34.621 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.609+0000 7f6188fb6700 1 -- start start 2023-07-21T19:07:34.621 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.609+0000 7f6188fb6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61840fe9d0 0x7f61841005e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:34.622 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.609+0000 7f6188fb6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f61840ff380 0x7f6184100b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:34.622 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.609+0000 7f6188fb6700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f61841049e0 0x7f61841a3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:34.622 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.609+0000 7f6188fb6700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6184101350 con 0x7f61840fe9d0 2023-07-21T19:07:34.622 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.610+0000 7f6188fb6700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61841a42b0 con 0x7f61840ff380 2023-07-21T19:07:34.622 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.610+0000 7f6188fb6700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61841a43f0 con 0x7f61841049e0 2023-07-21T19:07:34.622 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.610+0000 7f618259c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61840fe9d0 0x7f61841005e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:34.622 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.610+0000 7f618259c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61840fe9d0 0x7f61841005e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35374/0 (socket says 172.21.15.39:35374) 2023-07-21T19:07:34.623 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.610+0000 7f618259c700 1 -- 172.21.15.39:0/3541658420 learned_addr learned my addr 172.21.15.39:0/3541658420 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:07:34.623 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.610+0000 7f6181d9b700 1 --2- 172.21.15.39:0/3541658420 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f61840ff380 0x7f6184100b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:34.623 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.610+0000 7f6182d9d700 1 --2- 172.21.15.39:0/3541658420 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f61841049e0 0x7f61841a3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:34.623 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.610+0000 7f618259c700 1 -- 172.21.15.39:0/3541658420 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f61841049e0 msgr2=0x7f61841a3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:34.623 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.610+0000 7f618259c700 1 --2- 172.21.15.39:0/3541658420 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f61841049e0 0x7f61841a3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:34.623 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.610+0000 7f618259c700 1 -- 172.21.15.39:0/3541658420 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f61840ff380 msgr2=0x7f6184100b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:34.623 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.610+0000 7f618259c700 1 --2- 172.21.15.39:0/3541658420 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f61840ff380 0x7f6184100b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:34.624 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.610+0000 7f618259c700 1 -- 172.21.15.39:0/3541658420 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61780095f0 con 0x7f61840fe9d0 2023-07-21T19:07:34.624 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.610+0000 7f6182d9d700 1 --2- 172.21.15.39:0/3541658420 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f61841049e0 0x7f61841a3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:07:34.624 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.610+0000 7f618259c700 1 --2- 172.21.15.39:0/3541658420 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61840fe9d0 0x7f61841005e0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f617400d980 tx=0x7f617400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:34.624 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.610+0000 7f61737fe700 1 -- 172.21.15.39:0/3541658420 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6174004020 con 0x7f61840fe9d0 2023-07-21T19:07:34.624 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.610+0000 7f61737fe700 1 -- 172.21.15.39:0/3541658420 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6174009730 con 0x7f61840fe9d0 2023-07-21T19:07:34.624 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.610+0000 7f6188fb6700 1 -- 172.21.15.39:0/3541658420 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61841a4650 con 0x7f61840fe9d0 2023-07-21T19:07:34.624 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.610+0000 7f61737fe700 1 -- 172.21.15.39:0/3541658420 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6174003a80 con 0x7f61840fe9d0 2023-07-21T19:07:34.625 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.610+0000 7f6188fb6700 1 -- 172.21.15.39:0/3541658420 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61841a4b80 con 0x7f61840fe9d0 2023-07-21T19:07:34.625 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.611+0000 7f6188fb6700 1 -- 172.21.15.39:0/3541658420 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6184067290 con 0x7f61840fe9d0 2023-07-21T19:07:34.625 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.612+0000 7f61737fe700 1 -- 172.21.15.39:0/3541658420 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f61740098a0 con 0x7f61840fe9d0 2023-07-21T19:07:34.626 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.613+0000 7f61737fe700 1 --2- 172.21.15.39:0/3541658420 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f616c0a13c0 0x7f616c0a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:34.626 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.613+0000 7f61737fe700 1 -- 172.21.15.39:0/3541658420 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f61740cc990 con 0x7f61840fe9d0 2023-07-21T19:07:34.626 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.613+0000 7f6181d9b700 1 --2- 172.21.15.39:0/3541658420 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f616c0a13c0 0x7f616c0a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:34.626 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.613+0000 7f6181d9b700 1 --2- 172.21.15.39:0/3541658420 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f616c0a13c0 0x7f616c0a3870 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6184101dd0 tx=0x7f6178005fd0 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:34.626 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.617+0000 7f61737fe700 1 -- 172.21.15.39:0/3541658420 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6174095d40 con 0x7f61840fe9d0 2023-07-21T19:07:34.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:34 smithi193 ceph-mon[98204]: pgmap v819: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.2 KiB/s wr, 17 op/s 2023-07-21T19:07:34.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:34 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/966135522' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:07:34.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:34 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/3037794595' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:07:34.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:34 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/3908306900' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:07:34.874 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.865+0000 7f6188fb6700 1 -- 172.21.15.39:0/3541658420 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f61841a4f60 con 0x7f61840fe9d0 2023-07-21T19:07:34.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:34 smithi155 ceph-mon[97934]: pgmap v819: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.2 KiB/s wr, 17 op/s 2023-07-21T19:07:34.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:34 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/966135522' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:07:34.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:34 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/3037794595' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:07:34.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:34 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/3908306900' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:07:34.877 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:34 smithi039 ceph-mon[96489]: pgmap v819: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.2 KiB/s wr, 17 op/s 2023-07-21T19:07:34.877 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:34 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/966135522' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:07:34.877 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:34 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/3037794595' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:07:34.877 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:34 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/3908306900' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:07:34.882 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.873+0000 7f61737fe700 1 -- 172.21.15.39:0/3541658420 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v111) v1 ==== 161+0+0 (secure 0 0 0) 0x7f6174099360 con 0x7f61840fe9d0 2023-07-21T19:07:34.885 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.875+0000 7f6188fb6700 1 -- 172.21.15.39:0/3541658420 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f616c0a13c0 msgr2=0x7f616c0a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:34.885 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.875+0000 7f6188fb6700 1 --2- 172.21.15.39:0/3541658420 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f616c0a13c0 0x7f616c0a3870 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6184101dd0 tx=0x7f6178005fd0 comp rx=0 tx=0).stop 2023-07-21T19:07:34.885 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.875+0000 7f6188fb6700 1 -- 172.21.15.39:0/3541658420 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61840fe9d0 msgr2=0x7f61841005e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:34.885 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.875+0000 7f6188fb6700 1 --2- 172.21.15.39:0/3541658420 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61840fe9d0 0x7f61841005e0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f617400d980 tx=0x7f617400dc90 comp rx=0 tx=0).stop 2023-07-21T19:07:34.885 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.875+0000 7f6188fb6700 1 -- 172.21.15.39:0/3541658420 shutdown_connections 2023-07-21T19:07:34.885 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.875+0000 7f6188fb6700 1 --2- 172.21.15.39:0/3541658420 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f61841049e0 0x7f61841a3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:34.886 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.875+0000 7f6188fb6700 1 --2- 172.21.15.39:0/3541658420 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61840fe9d0 0x7f61841005e0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:34.886 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.875+0000 7f6188fb6700 1 --2- 172.21.15.39:0/3541658420 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f616c0a13c0 0x7f616c0a3870 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:34.886 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.875+0000 7f6188fb6700 1 --2- 172.21.15.39:0/3541658420 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f61840ff380 0x7f6184100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:34.886 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.875+0000 7f6188fb6700 1 -- 172.21.15.39:0/3541658420 >> 172.21.15.39:0/3541658420 conn(0x7f61840fa4b0 msgr2=0x7f61840676b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:34.886 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.875+0000 7f6188fb6700 1 -- 172.21.15.39:0/3541658420 shutdown_connections 2023-07-21T19:07:34.886 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:34.875+0000 7f6188fb6700 1 -- 172.21.15.39:0/3541658420 wait complete. 2023-07-21T19:07:34.887 INFO:teuthology.orchestra.run.smithi039.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-07-21T19:07:34.898 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-07-21T19:07:35.036 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:35.026+0000 7f8be2eaf700 1 -- 172.21.15.39:0/3461072204 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bdc0fe9d0 msgr2=0x7f8bdc0fee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:35.036 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:35.026+0000 7f8be2eaf700 1 --2- 172.21.15.39:0/3461072204 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bdc0fe9d0 0x7f8bdc0fee40 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f8bc8009990 tx=0x7f8bc8009ca0 comp rx=0 tx=0).stop 2023-07-21T19:07:35.036 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:35.026+0000 7f8be2eaf700 1 -- 172.21.15.39:0/3461072204 shutdown_connections 2023-07-21T19:07:35.037 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:35.026+0000 7f8be2eaf700 1 --2- 172.21.15.39:0/3461072204 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8bdc0ff380 0x7f8bdc10b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:35.037 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:35.026+0000 7f8be2eaf700 1 --2- 172.21.15.39:0/3461072204 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bdc1049e0 0x7f8bdc104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:35.037 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:35.026+0000 7f8be2eaf700 1 --2- 172.21.15.39:0/3461072204 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bdc0fe9d0 0x7f8bdc0fee40 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:35.037 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:35.026+0000 7f8be2eaf700 1 -- 172.21.15.39:0/3461072204 >> 172.21.15.39:0/3461072204 conn(0x7f8bdc0fa4b0 msgr2=0x7f8bdc0fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:35.038 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:35.026+0000 7f8be2eaf700 1 -- 172.21.15.39:0/3461072204 shutdown_connections 2023-07-21T19:07:35.038 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:35.026+0000 7f8be2eaf700 1 -- 172.21.15.39:0/3461072204 wait complete. 2023-07-21T19:07:35.038 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:35.027+0000 7f8be2eaf700 1 Processor -- start 2023-07-21T19:07:35.038 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:35.027+0000 7f8be2eaf700 1 -- start start 2023-07-21T19:07:35.040 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:35.027+0000 7f8be2eaf700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bdc0fe9d0 0x7f8bdc19b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:35.040 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:35.027+0000 7f8be2eaf700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bdc0ff380 0x7f8bdc19beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:35.040 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:35.027+0000 7f8be2eaf700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8bdc1049e0 0x7f8bdc1959e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:35.041 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:35.027+0000 7f8be2eaf700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bdc196050 con 0x7f8bdc0fe9d0 2023-07-21T19:07:35.041 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:35.027+0000 7f8be2eaf700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bdc1961c0 con 0x7f8bdc1049e0 2023-07-21T19:07:35.041 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:35.027+0000 7f8be2eaf700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bdc196380 con 0x7f8bdc0ff380 2023-07-21T19:07:35.041 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:35.027+0000 7f8be0c4b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bdc0fe9d0 0x7f8bdc19b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:35.041 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:35.027+0000 7f8be0c4b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bdc0fe9d0 0x7f8bdc19b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35412/0 (socket says 172.21.15.39:35412) 2023-07-21T19:07:35.041 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:35.027+0000 7f8be0c4b700 1 -- 172.21.15.39:0/182218254 learned_addr learned my addr 172.21.15.39:0/182218254 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:07:35.041 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:35.027+0000 7f8bdbfff700 1 --2- 172.21.15.39:0/182218254 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bdc0ff380 0x7f8bdc19beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:35.042 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:35.027+0000 7f8be144c700 1 --2- 172.21.15.39:0/182218254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8bdc1049e0 0x7f8bdc1959e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:35.042 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:35.027+0000 7f8bdbfff700 1 -- 172.21.15.39:0/182218254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8bdc1049e0 msgr2=0x7f8bdc1959e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:35.042 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:35.027+0000 7f8bdbfff700 1 --2- 172.21.15.39:0/182218254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8bdc1049e0 0x7f8bdc1959e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:35.042 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:35.027+0000 7f8bdbfff700 1 -- 172.21.15.39:0/182218254 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bdc0fe9d0 msgr2=0x7f8bdc19b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:35.043 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:35.027+0000 7f8bdbfff700 1 --2- 172.21.15.39:0/182218254 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bdc0fe9d0 0x7f8bdc19b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:35.043 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:35.027+0000 7f8bdbfff700 1 -- 172.21.15.39:0/182218254 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc80095f0 con 0x7f8bdc0ff380 2023-07-21T19:07:35.043 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:35.027+0000 7f8be0c4b700 1 --2- 172.21.15.39:0/182218254 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bdc0fe9d0 0x7f8bdc19b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:07:35.044 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:35.028+0000 7f8bdbfff700 1 --2- 172.21.15.39:0/182218254 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bdc0ff380 0x7f8bdc19beb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8bc80098e0 tx=0x7f8bc8004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:35.044 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:35.028+0000 7f8bd9ffb700 1 -- 172.21.15.39:0/182218254 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc801d070 con 0x7f8bdc0ff380 2023-07-21T19:07:35.044 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:35.028+0000 7f8be2eaf700 1 -- 172.21.15.39:0/182218254 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bdc196600 con 0x7f8bdc0ff380 2023-07-21T19:07:35.044 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:35.028+0000 7f8bd9ffb700 1 -- 172.21.15.39:0/182218254 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8bc8005030 con 0x7f8bdc0ff380 2023-07-21T19:07:35.045 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:35.028+0000 7f8bd9ffb700 1 -- 172.21.15.39:0/182218254 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc8004ab0 con 0x7f8bdc0ff380 2023-07-21T19:07:35.045 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:35.028+0000 7f8be2eaf700 1 -- 172.21.15.39:0/182218254 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bdc1a2960 con 0x7f8bdc0ff380 2023-07-21T19:07:35.045 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:35.029+0000 7f8be2eaf700 1 -- 172.21.15.39:0/182218254 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bdc067290 con 0x7f8bdc0ff380 2023-07-21T19:07:35.045 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:35.030+0000 7f8bd9ffb700 1 -- 172.21.15.39:0/182218254 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f8bc8005410 con 0x7f8bdc0ff380 2023-07-21T19:07:35.046 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:35.031+0000 7f8bd9ffb700 1 --2- 172.21.15.39:0/182218254 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f8bcc0a12c0 0x7f8bcc0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:35.047 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:35.031+0000 7f8bd9ffb700 1 -- 172.21.15.39:0/182218254 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 8913+0+0 (secure 0 0 0) 0x7f8bc80ce6f0 con 0x7f8bdc0ff380 2023-07-21T19:07:35.047 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:35.031+0000 7f8be0c4b700 1 --2- 172.21.15.39:0/182218254 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f8bcc0a12c0 0x7f8bcc0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:35.047 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:35.031+0000 7f8be0c4b700 1 --2- 172.21.15.39:0/182218254 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f8bcc0a12c0 0x7f8bcc0a3770 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0005950 tx=0x7f8bd0009260 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:35.048 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:35.034+0000 7f8bd9ffb700 1 -- 172.21.15.39:0/182218254 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f8bc80979a0 con 0x7f8bdc0ff380 2023-07-21T19:07:35.175 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:34 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mon-a[96465]: 2023-07-21T19:07:34.866+0000 7f094b519700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-21T19:07:35.176 INFO:journalctl@ceph.mds.g.smithi039.stdout:Jul 21 19:07:34 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-g[130792]: ignoring --setuser ceph since I am not root 2023-07-21T19:07:35.176 INFO:journalctl@ceph.mds.g.smithi039.stdout:Jul 21 19:07:34 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-g[130792]: ignoring --setgroup ceph since I am not root 2023-07-21T19:07:35.176 INFO:journalctl@ceph.mds.g.smithi039.stdout:Jul 21 19:07:34 smithi039 ceph-mds[130817]: -- 172.21.15.39:0/4071868209 <== mon.0 v2:172.21.15.39:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558b462731e0 con 0x558b46248c00 2023-07-21T19:07:35.177 INFO:journalctl@ceph.mds.g.smithi039.stdout:Jul 21 19:07:34 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-g[130792]: starting mds.g at 2023-07-21T19:07:35.177 INFO:journalctl@ceph.mds.a.smithi039.stdout:Jul 21 19:07:34 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-a[131175]: ignoring --setuser ceph since I am not root 2023-07-21T19:07:35.177 INFO:journalctl@ceph.mds.a.smithi039.stdout:Jul 21 19:07:34 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-a[131175]: ignoring --setgroup ceph since I am not root 2023-07-21T19:07:35.178 INFO:journalctl@ceph.mds.a.smithi039.stdout:Jul 21 19:07:34 smithi039 ceph-mds[131201]: -- 172.21.15.39:0/858996266 <== mon.0 v2:172.21.15.39:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5619cd7e11e0 con 0x5619cd7b7000 2023-07-21T19:07:35.178 INFO:journalctl@ceph.mds.a.smithi039.stdout:Jul 21 19:07:34 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-a[131175]: starting mds.a at 2023-07-21T19:07:35.293 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:35.283+0000 7f8be2eaf700 1 -- 172.21.15.39:0/182218254 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f8bdc1a2af0 con 0x7f8bdc0ff380 2023-07-21T19:07:35.357 INFO:journalctl@ceph.mds.c.smithi193.stdout:Jul 21 19:07:34 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-c[124252]: ignoring --setuser ceph since I am not root 2023-07-21T19:07:35.358 INFO:journalctl@ceph.mds.c.smithi193.stdout:Jul 21 19:07:34 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-c[124252]: ignoring --setgroup ceph since I am not root 2023-07-21T19:07:35.358 INFO:journalctl@ceph.mds.c.smithi193.stdout:Jul 21 19:07:34 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-c[124252]: starting mds.c at 2023-07-21T19:07:35.359 INFO:journalctl@ceph.mds.f.smithi193.stdout:Jul 21 19:07:34 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-f[123905]: ignoring --setuser ceph since I am not root 2023-07-21T19:07:35.359 INFO:journalctl@ceph.mds.f.smithi193.stdout:Jul 21 19:07:34 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-f[123905]: ignoring --setgroup ceph since I am not root 2023-07-21T19:07:35.359 INFO:journalctl@ceph.mds.f.smithi193.stdout:Jul 21 19:07:34 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-f[123905]: starting mds.f at 2023-07-21T19:07:35.360 INFO:journalctl@ceph.mds.i.smithi193.stdout:Jul 21 19:07:34 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-i[124588]: ignoring --setuser ceph since I am not root 2023-07-21T19:07:35.360 INFO:journalctl@ceph.mds.i.smithi193.stdout:Jul 21 19:07:34 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-i[124588]: ignoring --setgroup ceph since I am not root 2023-07-21T19:07:35.360 INFO:journalctl@ceph.mds.i.smithi193.stdout:Jul 21 19:07:34 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-i[124588]: starting mds.i at 2023-07-21T19:07:35.361 INFO:journalctl@ceph.mds.l.smithi193.stdout:Jul 21 19:07:34 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-l[123564]: ignoring --setuser ceph since I am not root 2023-07-21T19:07:35.361 INFO:journalctl@ceph.mds.l.smithi193.stdout:Jul 21 19:07:34 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-l[123564]: ignoring --setgroup ceph since I am not root 2023-07-21T19:07:35.361 INFO:journalctl@ceph.mds.l.smithi193.stdout:Jul 21 19:07:34 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-l[123564]: starting mds.l at 2023-07-21T19:07:35.376 INFO:journalctl@ceph.mds.b.smithi155.stdout:Jul 21 19:07:34 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-b[123270]: ignoring --setuser ceph since I am not root 2023-07-21T19:07:35.376 INFO:journalctl@ceph.mds.b.smithi155.stdout:Jul 21 19:07:34 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-b[123270]: ignoring --setgroup ceph since I am not root 2023-07-21T19:07:35.376 INFO:journalctl@ceph.mds.b.smithi155.stdout:Jul 21 19:07:34 smithi155 ceph-mds[123294]: -- 172.21.15.155:0/1731315724 <== mon.2 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564c842f71e0 con 0x564c842cc800 2023-07-21T19:07:35.376 INFO:journalctl@ceph.mds.b.smithi155.stdout:Jul 21 19:07:34 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-b[123270]: starting mds.b at 2023-07-21T19:07:35.377 INFO:journalctl@ceph.mds.e.smithi155.stdout:Jul 21 19:07:34 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-e[122591]: ignoring --setuser ceph since I am not root 2023-07-21T19:07:35.377 INFO:journalctl@ceph.mds.e.smithi155.stdout:Jul 21 19:07:34 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-e[122591]: ignoring --setgroup ceph since I am not root 2023-07-21T19:07:35.377 INFO:journalctl@ceph.mds.e.smithi155.stdout:Jul 21 19:07:34 smithi155 ceph-mds[122615]: -- 172.21.15.155:0/2499578413 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5560b23651e0 con 0x5560b233ac00 2023-07-21T19:07:35.377 INFO:journalctl@ceph.mds.e.smithi155.stdout:Jul 21 19:07:34 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-e[122591]: starting mds.e at 2023-07-21T19:07:35.378 INFO:journalctl@ceph.mds.h.smithi155.stdout:Jul 21 19:07:34 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-h[122931]: ignoring --setuser ceph since I am not root 2023-07-21T19:07:35.378 INFO:journalctl@ceph.mds.h.smithi155.stdout:Jul 21 19:07:34 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-h[122931]: ignoring --setgroup ceph since I am not root 2023-07-21T19:07:35.378 INFO:journalctl@ceph.mds.h.smithi155.stdout:Jul 21 19:07:34 smithi155 ceph-mds[122955]: -- 172.21.15.155:0/2571515250 <== mon.2 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5578fe1251e0 con 0x5578fe0fa800 2023-07-21T19:07:35.378 INFO:journalctl@ceph.mds.h.smithi155.stdout:Jul 21 19:07:34 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-h[122931]: starting mds.h at 2023-07-21T19:07:35.378 INFO:journalctl@ceph.mds.k.smithi155.stdout:Jul 21 19:07:34 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-k[122248]: ignoring --setuser ceph since I am not root 2023-07-21T19:07:35.379 INFO:journalctl@ceph.mds.k.smithi155.stdout:Jul 21 19:07:34 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-k[122248]: ignoring --setgroup ceph since I am not root 2023-07-21T19:07:35.379 INFO:journalctl@ceph.mds.k.smithi155.stdout:Jul 21 19:07:34 smithi155 ceph-mds[122272]: -- 172.21.15.155:0/3659674553 <== mon.2 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556bd31231e0 con 0x556bd30f8800 2023-07-21T19:07:35.379 INFO:journalctl@ceph.mds.k.smithi155.stdout:Jul 21 19:07:34 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-k[122248]: starting mds.k at 2023-07-21T19:07:35.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:35 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/3541658420' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-07-21T19:07:35.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:35 smithi193 ceph-mon[98204]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-07-21T19:07:35.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:35 smithi193 ceph-mon[98204]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-21T19:07:35.859 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:35 smithi193 ceph-mon[98204]: Health check cleared: MDS_DAMAGE (was: 3 MDSs report damaged metadata) 2023-07-21T19:07:35.859 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:35 smithi193 ceph-mon[98204]: osdmap e125: 12 total, 12 up, 12 in 2023-07-21T19:07:35.859 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:35 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/3541658420' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-07-21T19:07:35.859 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:35 smithi193 ceph-mon[98204]: fsmap cephfs:0/5 2 up:standby, 5 failed 2023-07-21T19:07:35.859 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:35 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/182218254' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-07-21T19:07:35.859 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:35 smithi193 ceph-mon[98204]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-07-21T19:07:35.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:35 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/3541658420' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-07-21T19:07:35.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:35 smithi155 ceph-mon[97934]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-07-21T19:07:35.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:35 smithi155 ceph-mon[97934]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-21T19:07:35.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:35 smithi155 ceph-mon[97934]: Health check cleared: MDS_DAMAGE (was: 3 MDSs report damaged metadata) 2023-07-21T19:07:35.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:35 smithi155 ceph-mon[97934]: osdmap e125: 12 total, 12 up, 12 in 2023-07-21T19:07:35.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:35 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/3541658420' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-07-21T19:07:35.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:35 smithi155 ceph-mon[97934]: fsmap cephfs:0/5 2 up:standby, 5 failed 2023-07-21T19:07:35.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:35 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/182218254' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-07-21T19:07:35.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:35 smithi155 ceph-mon[97934]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-07-21T19:07:35.884 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:35.874+0000 7f8bd9ffb700 1 -- 172.21.15.39:0/182218254 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v112) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8bc8084e60 con 0x7f8bdc0ff380 2023-07-21T19:07:35.887 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:35.877+0000 7f8be2eaf700 1 -- 172.21.15.39:0/182218254 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f8bcc0a12c0 msgr2=0x7f8bcc0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:35.888 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:35.877+0000 7f8be2eaf700 1 --2- 172.21.15.39:0/182218254 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f8bcc0a12c0 0x7f8bcc0a3770 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0005950 tx=0x7f8bd0009260 comp rx=0 tx=0).stop 2023-07-21T19:07:35.888 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:35.877+0000 7f8be2eaf700 1 -- 172.21.15.39:0/182218254 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bdc0ff380 msgr2=0x7f8bdc19beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:35.888 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:35.877+0000 7f8be2eaf700 1 --2- 172.21.15.39:0/182218254 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bdc0ff380 0x7f8bdc19beb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8bc80098e0 tx=0x7f8bc8004730 comp rx=0 tx=0).stop 2023-07-21T19:07:35.888 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:35.877+0000 7f8be2eaf700 1 -- 172.21.15.39:0/182218254 shutdown_connections 2023-07-21T19:07:35.888 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:35.877+0000 7f8be2eaf700 1 --2- 172.21.15.39:0/182218254 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f8bcc0a12c0 0x7f8bcc0a3770 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:35.889 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:35.877+0000 7f8be2eaf700 1 --2- 172.21.15.39:0/182218254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8bdc1049e0 0x7f8bdc1959e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:35.889 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:35.877+0000 7f8be2eaf700 1 --2- 172.21.15.39:0/182218254 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bdc0fe9d0 0x7f8bdc19b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:35.889 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:35.877+0000 7f8be2eaf700 1 --2- 172.21.15.39:0/182218254 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bdc0ff380 0x7f8bdc19beb0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:35.889 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:35.877+0000 7f8be2eaf700 1 -- 172.21.15.39:0/182218254 >> 172.21.15.39:0/182218254 conn(0x7f8bdc0fa4b0 msgr2=0x7f8bdc0fbfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:35.889 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:35.878+0000 7f8be2eaf700 1 -- 172.21.15.39:0/182218254 shutdown_connections 2023-07-21T19:07:35.889 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:35.878+0000 7f8be2eaf700 1 -- 172.21.15.39:0/182218254 wait complete. 2023-07-21T19:07:35.904 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:35 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/3541658420' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-07-21T19:07:35.904 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:35 smithi039 ceph-mon[96489]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-07-21T19:07:35.904 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:35 smithi039 ceph-mon[96489]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-21T19:07:35.904 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:35 smithi039 ceph-mon[96489]: Health check cleared: MDS_DAMAGE (was: 3 MDSs report damaged metadata) 2023-07-21T19:07:35.904 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:35 smithi039 ceph-mon[96489]: osdmap e125: 12 total, 12 up, 12 in 2023-07-21T19:07:35.905 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:35 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/3541658420' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-07-21T19:07:35.905 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:35 smithi039 ceph-mon[96489]: fsmap cephfs:0/5 2 up:standby, 5 failed 2023-07-21T19:07:35.905 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:35 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/182218254' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-07-21T19:07:35.905 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:35 smithi039 ceph-mon[96489]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-07-21T19:07:35.906 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-07-21T19:07:36.046 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:36.036+0000 7ff7de512700 1 -- 172.21.15.39:0/3286383214 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7d8103ce0 msgr2=0x7ff7d810b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:36.046 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:36.036+0000 7ff7de512700 1 --2- 172.21.15.39:0/3286383214 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7d8103ce0 0x7ff7d810b8f0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7ff7d000b0d0 tx=0x7ff7d000b3e0 comp rx=0 tx=0).stop 2023-07-21T19:07:36.046 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:36.036+0000 7ff7de512700 1 -- 172.21.15.39:0/3286383214 shutdown_connections 2023-07-21T19:07:36.046 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:36.036+0000 7ff7de512700 1 --2- 172.21.15.39:0/3286383214 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7d80ff580 0x7ff7d80ff950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:36.046 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:36.036+0000 7ff7de512700 1 --2- 172.21.15.39:0/3286383214 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7d8103ce0 0x7ff7d810b8f0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:36.046 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:36.036+0000 7ff7de512700 1 --2- 172.21.15.39:0/3286383214 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7d80fff20 0x7ff7d8103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:36.047 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:36.036+0000 7ff7de512700 1 -- 172.21.15.39:0/3286383214 >> 172.21.15.39:0/3286383214 conn(0x7ff7d8076200 msgr2=0x7ff7d8076600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:36.047 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:36.036+0000 7ff7de512700 1 -- 172.21.15.39:0/3286383214 shutdown_connections 2023-07-21T19:07:36.047 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:36.036+0000 7ff7de512700 1 -- 172.21.15.39:0/3286383214 wait complete. 2023-07-21T19:07:36.049 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:36.037+0000 7ff7de512700 1 Processor -- start 2023-07-21T19:07:36.049 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:36.037+0000 7ff7de512700 1 -- start start 2023-07-21T19:07:36.049 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:36.037+0000 7ff7de512700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7d80ff580 0x7ff7d8196b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:36.049 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:36.037+0000 7ff7de512700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7d80fff20 0x7ff7d81970d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:36.049 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:36.037+0000 7ff7de512700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7d8103ce0 0x7ff7d81a1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:36.049 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:36.037+0000 7ff7de512700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d8197a10 con 0x7ff7d80fff20 2023-07-21T19:07:36.050 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:36.037+0000 7ff7de512700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d81a2150 con 0x7ff7d80ff580 2023-07-21T19:07:36.050 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:36.037+0000 7ff7de512700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d81a2310 con 0x7ff7d8103ce0 2023-07-21T19:07:36.050 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:36.037+0000 7ff7d77fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7d80fff20 0x7ff7d81970d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:36.050 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:36.037+0000 7ff7d77fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7d80fff20 0x7ff7d81970d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35432/0 (socket says 172.21.15.39:35432) 2023-07-21T19:07:36.050 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:36.037+0000 7ff7d77fe700 1 -- 172.21.15.39:0/3564828223 learned_addr learned my addr 172.21.15.39:0/3564828223 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:07:36.050 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:36.037+0000 7ff7dcaaf700 1 --2- 172.21.15.39:0/3564828223 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7d8103ce0 0x7ff7d81a1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:36.050 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:36.037+0000 7ff7d7fff700 1 --2- 172.21.15.39:0/3564828223 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7d80ff580 0x7ff7d8196b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:36.051 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:36.037+0000 7ff7d77fe700 1 -- 172.21.15.39:0/3564828223 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7d8103ce0 msgr2=0x7ff7d81a1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:36.051 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:36.037+0000 7ff7d77fe700 1 --2- 172.21.15.39:0/3564828223 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7d8103ce0 0x7ff7d81a1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:36.051 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:36.037+0000 7ff7d77fe700 1 -- 172.21.15.39:0/3564828223 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7d80ff580 msgr2=0x7ff7d8196b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:36.051 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:36.037+0000 7ff7d77fe700 1 --2- 172.21.15.39:0/3564828223 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7d80ff580 0x7ff7d8196b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:36.051 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:36.038+0000 7ff7d77fe700 1 -- 172.21.15.39:0/3564828223 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7d0009d00 con 0x7ff7d80fff20 2023-07-21T19:07:36.051 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:36.038+0000 7ff7d77fe700 1 --2- 172.21.15.39:0/3564828223 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7d80fff20 0x7ff7d81970d0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7ff7c000d950 tx=0x7ff7c000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:36.051 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:36.038+0000 7ff7d57fa700 1 -- 172.21.15.39:0/3564828223 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c0009930 con 0x7ff7d80fff20 2023-07-21T19:07:36.051 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:36.038+0000 7ff7d57fa700 1 -- 172.21.15.39:0/3564828223 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7c0010460 con 0x7ff7d80fff20 2023-07-21T19:07:36.052 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:36.038+0000 7ff7d57fa700 1 -- 172.21.15.39:0/3564828223 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c000f620 con 0x7ff7d80fff20 2023-07-21T19:07:36.052 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:36.038+0000 7ff7de512700 1 -- 172.21.15.39:0/3564828223 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7d81a25f0 con 0x7ff7d80fff20 2023-07-21T19:07:36.052 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:36.038+0000 7ff7de512700 1 -- 172.21.15.39:0/3564828223 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7d81a2b00 con 0x7ff7d80fff20 2023-07-21T19:07:36.052 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:36.039+0000 7ff7de512700 1 -- 172.21.15.39:0/3564828223 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7d8066a00 con 0x7ff7d80fff20 2023-07-21T19:07:36.053 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:36.040+0000 7ff7d57fa700 1 -- 172.21.15.39:0/3564828223 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7ff7c0010860 con 0x7ff7d80fff20 2023-07-21T19:07:36.053 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:36.040+0000 7ff7d57fa700 1 --2- 172.21.15.39:0/3564828223 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7ff7c40a1340 0x7ff7c40a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:36.053 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:36.041+0000 7ff7d7fff700 1 --2- 172.21.15.39:0/3564828223 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7ff7c40a1340 0x7ff7c40a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:36.053 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:36.041+0000 7ff7d57fa700 1 -- 172.21.15.39:0/3564828223 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 8913+0+0 (secure 0 0 0) 0x7ff7c00cc930 con 0x7ff7d80fff20 2023-07-21T19:07:36.053 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:36.041+0000 7ff7d7fff700 1 --2- 172.21.15.39:0/3564828223 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7ff7c40a1340 0x7ff7c40a37f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff7c8005950 tx=0x7ff7c80058e0 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:36.054 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:36.045+0000 7ff7d57fa700 1 -- 172.21.15.39:0/3564828223 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7ff7c0095bb0 con 0x7ff7d80fff20 2023-07-21T19:07:36.256 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:36.246+0000 7ff7de512700 1 -- 172.21.15.39:0/3564828223 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff7d80fe810 con 0x7ff7d80fff20 2023-07-21T19:07:36.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:36 smithi193 ceph-mon[98204]: pgmap v820: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2023-07-21T19:07:36.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:36 smithi193 ceph-mon[98204]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-07-21T19:07:36.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:36 smithi193 ceph-mon[98204]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-07-21T19:07:36.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:36 smithi193 ceph-mon[98204]: Cluster is now healthy 2023-07-21T19:07:36.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:36 smithi193 ceph-mon[98204]: mds.? [v2:172.21.15.39:6838/2045081578,v1:172.21.15.39:6840/2045081578] up:boot 2023-07-21T19:07:36.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:36 smithi193 ceph-mon[98204]: mds.? [v2:172.21.15.39:6839/1673334824,v1:172.21.15.39:6841/1673334824] up:boot 2023-07-21T19:07:36.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:36 smithi193 ceph-mon[98204]: mds.? [v2:172.21.15.155:6836/1977221026,v1:172.21.15.155:6837/1977221026] up:boot 2023-07-21T19:07:36.859 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:36 smithi193 ceph-mon[98204]: mds.? [v2:172.21.15.155:6838/4141775609,v1:172.21.15.155:6839/4141775609] up:boot 2023-07-21T19:07:36.859 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:36 smithi193 ceph-mon[98204]: mds.? [v2:172.21.15.155:6832/938172601,v1:172.21.15.155:6834/938172601] up:boot 2023-07-21T19:07:36.859 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:36 smithi193 ceph-mon[98204]: mds.? [v2:172.21.15.155:6833/2015257836,v1:172.21.15.155:6835/2015257836] up:boot 2023-07-21T19:07:36.859 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:36 smithi193 ceph-mon[98204]: mds.? [v2:172.21.15.193:6834/2423783306,v1:172.21.15.193:6838/2423783306] up:boot 2023-07-21T19:07:36.859 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:36 smithi193 ceph-mon[98204]: mds.? [v2:172.21.15.193:6832/2439903460,v1:172.21.15.193:6836/2439903460] up:boot 2023-07-21T19:07:36.859 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:36 smithi193 ceph-mon[98204]: mds.? [v2:172.21.15.193:6835/1797758312,v1:172.21.15.193:6839/1797758312] up:boot 2023-07-21T19:07:36.859 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:36 smithi193 ceph-mon[98204]: mds.? [v2:172.21.15.193:6833/1854558969,v1:172.21.15.193:6837/1854558969] up:boot 2023-07-21T19:07:36.859 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:36 smithi193 ceph-mon[98204]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-07-21T19:07:36.860 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:36 smithi193 ceph-mon[98204]: fsmap 12 up:standby 2023-07-21T19:07:36.860 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:36 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-21T19:07:36.860 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:36 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-21T19:07:36.860 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:36 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-21T19:07:36.860 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:36 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-21T19:07:36.860 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:36 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-21T19:07:36.860 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:36 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-21T19:07:36.861 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:36 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-21T19:07:36.861 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:36 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-21T19:07:36.861 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:36 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-21T19:07:36.861 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:36 smithi193 ceph-mon[98204]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-21T19:07:36.861 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:36 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/3564828223' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-21T19:07:36.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:36 smithi155 ceph-mon[97934]: pgmap v820: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2023-07-21T19:07:36.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:36 smithi155 ceph-mon[97934]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-07-21T19:07:36.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:36 smithi155 ceph-mon[97934]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-07-21T19:07:36.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:36 smithi155 ceph-mon[97934]: Cluster is now healthy 2023-07-21T19:07:36.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:36 smithi155 ceph-mon[97934]: mds.? [v2:172.21.15.39:6838/2045081578,v1:172.21.15.39:6840/2045081578] up:boot 2023-07-21T19:07:36.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:36 smithi155 ceph-mon[97934]: mds.? [v2:172.21.15.39:6839/1673334824,v1:172.21.15.39:6841/1673334824] up:boot 2023-07-21T19:07:36.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:36 smithi155 ceph-mon[97934]: mds.? [v2:172.21.15.155:6836/1977221026,v1:172.21.15.155:6837/1977221026] up:boot 2023-07-21T19:07:36.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:36 smithi155 ceph-mon[97934]: mds.? [v2:172.21.15.155:6838/4141775609,v1:172.21.15.155:6839/4141775609] up:boot 2023-07-21T19:07:36.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:36 smithi155 ceph-mon[97934]: mds.? [v2:172.21.15.155:6832/938172601,v1:172.21.15.155:6834/938172601] up:boot 2023-07-21T19:07:36.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:36 smithi155 ceph-mon[97934]: mds.? [v2:172.21.15.155:6833/2015257836,v1:172.21.15.155:6835/2015257836] up:boot 2023-07-21T19:07:36.878 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:36 smithi155 ceph-mon[97934]: mds.? [v2:172.21.15.193:6834/2423783306,v1:172.21.15.193:6838/2423783306] up:boot 2023-07-21T19:07:36.878 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:36 smithi155 ceph-mon[97934]: mds.? [v2:172.21.15.193:6832/2439903460,v1:172.21.15.193:6836/2439903460] up:boot 2023-07-21T19:07:36.878 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:36 smithi155 ceph-mon[97934]: mds.? [v2:172.21.15.193:6835/1797758312,v1:172.21.15.193:6839/1797758312] up:boot 2023-07-21T19:07:36.878 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:36 smithi155 ceph-mon[97934]: mds.? [v2:172.21.15.193:6833/1854558969,v1:172.21.15.193:6837/1854558969] up:boot 2023-07-21T19:07:36.878 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:36 smithi155 ceph-mon[97934]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-07-21T19:07:36.878 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:36 smithi155 ceph-mon[97934]: fsmap 12 up:standby 2023-07-21T19:07:36.878 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:36 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-21T19:07:36.878 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:36 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-21T19:07:36.879 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:36 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-21T19:07:36.879 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:36 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-21T19:07:36.879 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:36 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-21T19:07:36.879 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:36 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-21T19:07:36.879 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:36 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-21T19:07:36.879 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:36 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-21T19:07:36.879 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:36 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-21T19:07:36.880 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:36 smithi155 ceph-mon[97934]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-21T19:07:36.880 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:36 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/3564828223' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-21T19:07:36.891 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:36.881+0000 7ff7d57fa700 1 -- 172.21.15.39:0/3564828223 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v126) v1 ==== 181+0+0 (secure 0 0 0) 0x7ff7c00991d0 con 0x7ff7d80fff20 2023-07-21T19:07:36.895 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:36.885+0000 7ff7de512700 1 -- 172.21.15.39:0/3564828223 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7ff7c40a1340 msgr2=0x7ff7c40a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:36.895 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:36.885+0000 7ff7de512700 1 --2- 172.21.15.39:0/3564828223 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7ff7c40a1340 0x7ff7c40a37f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff7c8005950 tx=0x7ff7c80058e0 comp rx=0 tx=0).stop 2023-07-21T19:07:36.895 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:36.885+0000 7ff7de512700 1 -- 172.21.15.39:0/3564828223 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7d80fff20 msgr2=0x7ff7d81970d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:36.895 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:36.885+0000 7ff7de512700 1 --2- 172.21.15.39:0/3564828223 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7d80fff20 0x7ff7d81970d0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7ff7c000d950 tx=0x7ff7c000dc60 comp rx=0 tx=0).stop 2023-07-21T19:07:36.896 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:36.886+0000 7ff7de512700 1 -- 172.21.15.39:0/3564828223 shutdown_connections 2023-07-21T19:07:36.896 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:36.886+0000 7ff7de512700 1 --2- 172.21.15.39:0/3564828223 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7ff7c40a1340 0x7ff7c40a37f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:36.896 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:36.886+0000 7ff7de512700 1 --2- 172.21.15.39:0/3564828223 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7d80ff580 0x7ff7d8196b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:36.896 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:36.886+0000 7ff7de512700 1 --2- 172.21.15.39:0/3564828223 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7d8103ce0 0x7ff7d81a1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:36.896 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:36.886+0000 7ff7de512700 1 --2- 172.21.15.39:0/3564828223 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7d80fff20 0x7ff7d81970d0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:36.896 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:36.886+0000 7ff7de512700 1 -- 172.21.15.39:0/3564828223 >> 172.21.15.39:0/3564828223 conn(0x7ff7d8076200 msgr2=0x7ff7d80fbc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:36.897 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:36.886+0000 7ff7de512700 1 -- 172.21.15.39:0/3564828223 shutdown_connections 2023-07-21T19:07:36.897 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:36.886+0000 7ff7de512700 1 -- 172.21.15.39:0/3564828223 wait complete. 2023-07-21T19:07:36.898 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_metadata' removed 2023-07-21T19:07:36.918 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:36 smithi039 ceph-mon[96489]: pgmap v820: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2023-07-21T19:07:36.918 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:36 smithi039 ceph-mon[96489]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-07-21T19:07:36.918 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:36 smithi039 ceph-mon[96489]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-07-21T19:07:36.918 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:36 smithi039 ceph-mon[96489]: Cluster is now healthy 2023-07-21T19:07:36.918 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:36 smithi039 ceph-mon[96489]: mds.? [v2:172.21.15.39:6838/2045081578,v1:172.21.15.39:6840/2045081578] up:boot 2023-07-21T19:07:36.919 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:36 smithi039 ceph-mon[96489]: mds.? [v2:172.21.15.39:6839/1673334824,v1:172.21.15.39:6841/1673334824] up:boot 2023-07-21T19:07:36.919 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:36 smithi039 ceph-mon[96489]: mds.? [v2:172.21.15.155:6836/1977221026,v1:172.21.15.155:6837/1977221026] up:boot 2023-07-21T19:07:36.919 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:36 smithi039 ceph-mon[96489]: mds.? [v2:172.21.15.155:6838/4141775609,v1:172.21.15.155:6839/4141775609] up:boot 2023-07-21T19:07:36.919 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:36 smithi039 ceph-mon[96489]: mds.? [v2:172.21.15.155:6832/938172601,v1:172.21.15.155:6834/938172601] up:boot 2023-07-21T19:07:36.919 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:36 smithi039 ceph-mon[96489]: mds.? [v2:172.21.15.155:6833/2015257836,v1:172.21.15.155:6835/2015257836] up:boot 2023-07-21T19:07:36.919 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:36 smithi039 ceph-mon[96489]: mds.? [v2:172.21.15.193:6834/2423783306,v1:172.21.15.193:6838/2423783306] up:boot 2023-07-21T19:07:36.919 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:36 smithi039 ceph-mon[96489]: mds.? [v2:172.21.15.193:6832/2439903460,v1:172.21.15.193:6836/2439903460] up:boot 2023-07-21T19:07:36.920 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:36 smithi039 ceph-mon[96489]: mds.? [v2:172.21.15.193:6835/1797758312,v1:172.21.15.193:6839/1797758312] up:boot 2023-07-21T19:07:36.920 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:36 smithi039 ceph-mon[96489]: mds.? [v2:172.21.15.193:6833/1854558969,v1:172.21.15.193:6837/1854558969] up:boot 2023-07-21T19:07:36.920 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:36 smithi039 ceph-mon[96489]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-07-21T19:07:36.920 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:36 smithi039 ceph-mon[96489]: fsmap 12 up:standby 2023-07-21T19:07:36.920 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:36 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-21T19:07:36.920 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:36 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-21T19:07:36.920 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:36 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-21T19:07:36.920 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:36 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-21T19:07:36.921 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:36 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-21T19:07:36.921 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:36 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-21T19:07:36.921 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:36 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-21T19:07:36.921 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:36 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-21T19:07:36.921 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:36 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-21T19:07:36.921 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:36 smithi039 ceph-mon[96489]: from='mgr.15471 172.21.15.39:0/3091307769' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-21T19:07:36.921 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:36 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/3564828223' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-21T19:07:36.922 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-07-21T19:07:37.084 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:37.074+0000 7f5f35b27700 1 -- 172.21.15.39:0/3858208719 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f301049e0 msgr2=0x7f5f30104db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:37.084 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:37.074+0000 7f5f35b27700 1 --2- 172.21.15.39:0/3858208719 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f301049e0 0x7f5f30104db0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f5f20009990 tx=0x7f5f20009ca0 comp rx=0 tx=0).stop 2023-07-21T19:07:37.084 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:37.074+0000 7f5f35b27700 1 -- 172.21.15.39:0/3858208719 shutdown_connections 2023-07-21T19:07:37.084 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:37.074+0000 7f5f35b27700 1 --2- 172.21.15.39:0/3858208719 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5f300ff380 0x7f5f3010b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:37.085 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:37.074+0000 7f5f35b27700 1 --2- 172.21.15.39:0/3858208719 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f301049e0 0x7f5f30104db0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:37.085 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:37.074+0000 7f5f35b27700 1 --2- 172.21.15.39:0/3858208719 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f300fe9d0 0x7f5f300fee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:37.085 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:37.074+0000 7f5f35b27700 1 -- 172.21.15.39:0/3858208719 >> 172.21.15.39:0/3858208719 conn(0x7f5f300fa4b0 msgr2=0x7f5f300fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:37.085 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:37.074+0000 7f5f35b27700 1 -- 172.21.15.39:0/3858208719 shutdown_connections 2023-07-21T19:07:37.085 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:37.074+0000 7f5f35b27700 1 -- 172.21.15.39:0/3858208719 wait complete. 2023-07-21T19:07:37.087 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:37.075+0000 7f5f35b27700 1 Processor -- start 2023-07-21T19:07:37.087 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:37.075+0000 7f5f35b27700 1 -- start start 2023-07-21T19:07:37.087 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:37.075+0000 7f5f35b27700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f300fe9d0 0x7f5f301969f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:37.087 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:37.075+0000 7f5f35b27700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f300ff380 0x7f5f30196f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:37.088 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:37.075+0000 7f5f35b27700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5f301049e0 0x7f5f301a1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:37.088 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:37.075+0000 7f5f35b27700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f30197840 con 0x7f5f300fe9d0 2023-07-21T19:07:37.088 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:37.075+0000 7f5f35b27700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f301a1fb0 con 0x7f5f300ff380 2023-07-21T19:07:37.088 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:37.075+0000 7f5f35b27700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f301a2120 con 0x7f5f301049e0 2023-07-21T19:07:37.088 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:37.075+0000 7f5f2f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f300fe9d0 0x7f5f301969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:37.088 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:37.075+0000 7f5f2f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f300fe9d0 0x7f5f301969f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35448/0 (socket says 172.21.15.39:35448) 2023-07-21T19:07:37.088 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:37.075+0000 7f5f2f7fe700 1 -- 172.21.15.39:0/4137007346 learned_addr learned my addr 172.21.15.39:0/4137007346 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-07-21T19:07:37.089 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:37.075+0000 7f5f2effd700 1 --2- 172.21.15.39:0/4137007346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f300ff380 0x7f5f30196f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:37.089 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:37.075+0000 7f5f2ffff700 1 --2- 172.21.15.39:0/4137007346 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5f301049e0 0x7f5f301a1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:37.089 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:37.075+0000 7f5f2f7fe700 1 -- 172.21.15.39:0/4137007346 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5f301049e0 msgr2=0x7f5f301a1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:37.089 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:37.075+0000 7f5f2f7fe700 1 --2- 172.21.15.39:0/4137007346 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5f301049e0 0x7f5f301a1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:37.089 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:37.075+0000 7f5f2f7fe700 1 -- 172.21.15.39:0/4137007346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f300ff380 msgr2=0x7f5f30196f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:37.089 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:37.075+0000 7f5f2f7fe700 1 --2- 172.21.15.39:0/4137007346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f300ff380 0x7f5f30196f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:37.089 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:37.075+0000 7f5f2f7fe700 1 -- 172.21.15.39:0/4137007346 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f200095f0 con 0x7f5f300fe9d0 2023-07-21T19:07:37.089 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:37.075+0000 7f5f2ffff700 1 --2- 172.21.15.39:0/4137007346 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5f301049e0 0x7f5f301a1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:07:37.090 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:37.075+0000 7f5f2effd700 1 --2- 172.21.15.39:0/4137007346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f300ff380 0x7f5f30196f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:07:37.090 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:37.076+0000 7f5f2f7fe700 1 --2- 172.21.15.39:0/4137007346 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f300fe9d0 0x7f5f301969f0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f5f20000c00 tx=0x7f5f20009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:37.090 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:37.076+0000 7f5f2cff9700 1 -- 172.21.15.39:0/4137007346 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2001d070 con 0x7f5f300fe9d0 2023-07-21T19:07:37.090 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:37.076+0000 7f5f2cff9700 1 -- 172.21.15.39:0/4137007346 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f20022470 con 0x7f5f300fe9d0 2023-07-21T19:07:37.090 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:37.076+0000 7f5f35b27700 1 -- 172.21.15.39:0/4137007346 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f301a23a0 con 0x7f5f300fe9d0 2023-07-21T19:07:37.090 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:37.076+0000 7f5f2cff9700 1 -- 172.21.15.39:0/4137007346 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f20017650 con 0x7f5f300fe9d0 2023-07-21T19:07:37.090 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:37.076+0000 7f5f35b27700 1 -- 172.21.15.39:0/4137007346 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f301a27c0 con 0x7f5f300fe9d0 2023-07-21T19:07:37.091 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:37.077+0000 7f5f35b27700 1 -- 172.21.15.39:0/4137007346 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f30067290 con 0x7f5f300fe9d0 2023-07-21T19:07:37.091 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:37.078+0000 7f5f2cff9700 1 -- 172.21.15.39:0/4137007346 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f5f20022870 con 0x7f5f300fe9d0 2023-07-21T19:07:37.091 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:37.079+0000 7f5f2cff9700 1 --2- 172.21.15.39:0/4137007346 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f5f180a12c0 0x7f5f180a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:37.092 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:37.079+0000 7f5f2effd700 1 --2- 172.21.15.39:0/4137007346 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f5f180a12c0 0x7f5f180a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:37.092 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:37.079+0000 7f5f2cff9700 1 -- 172.21.15.39:0/4137007346 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 8417+0+0 (secure 0 0 0) 0x7f5f200cdfc0 con 0x7f5f300fe9d0 2023-07-21T19:07:37.092 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:37.079+0000 7f5f2effd700 1 --2- 172.21.15.39:0/4137007346 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f5f180a12c0 0x7f5f180a3770 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5f30067f20 tx=0x7f5f24009260 comp rx=0 tx=0).ready entity=mgr.15471 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:37.092 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:37.083+0000 7f5f2cff9700 1 -- 172.21.15.39:0/4137007346 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f5f2005c790 con 0x7f5f300fe9d0 2023-07-21T19:07:37.350 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:37.340+0000 7f5f35b27700 1 -- 172.21.15.39:0/4137007346 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5f301a29e0 con 0x7f5f300fe9d0 2023-07-21T19:07:37.889 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:37.880+0000 7f5f2cff9700 1 -- 172.21.15.39:0/4137007346 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v127) v1 ==== 169+0+0 (secure 0 0 0) 0x7f5f200974c0 con 0x7f5f300fe9d0 2023-07-21T19:07:37.894 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:37.885+0000 7f5f35b27700 1 -- 172.21.15.39:0/4137007346 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f5f180a12c0 msgr2=0x7f5f180a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:37.894 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:37.885+0000 7f5f35b27700 1 --2- 172.21.15.39:0/4137007346 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f5f180a12c0 0x7f5f180a3770 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5f30067f20 tx=0x7f5f24009260 comp rx=0 tx=0).stop 2023-07-21T19:07:37.895 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:37.885+0000 7f5f35b27700 1 -- 172.21.15.39:0/4137007346 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f300fe9d0 msgr2=0x7f5f301969f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:37.895 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:37.885+0000 7f5f35b27700 1 --2- 172.21.15.39:0/4137007346 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f300fe9d0 0x7f5f301969f0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f5f20000c00 tx=0x7f5f20009580 comp rx=0 tx=0).stop 2023-07-21T19:07:37.899 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:37.887+0000 7f5f35b27700 1 -- 172.21.15.39:0/4137007346 shutdown_connections 2023-07-21T19:07:37.899 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:37.887+0000 7f5f35b27700 1 --2- 172.21.15.39:0/4137007346 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5f301049e0 0x7f5f301a1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:37.900 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:37.887+0000 7f5f35b27700 1 --2- 172.21.15.39:0/4137007346 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f300fe9d0 0x7f5f301969f0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:37.900 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:37.887+0000 7f5f35b27700 1 --2- 172.21.15.39:0/4137007346 >> [v2:172.21.15.39:6800/13616514,v1:172.21.15.39:6801/13616514] conn(0x7f5f180a12c0 0x7f5f180a3770 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:37.900 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:37.887+0000 7f5f35b27700 1 --2- 172.21.15.39:0/4137007346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f300ff380 0x7f5f30196f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:37.901 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:37.887+0000 7f5f35b27700 1 -- 172.21.15.39:0/4137007346 >> 172.21.15.39:0/4137007346 conn(0x7f5f300fa4b0 msgr2=0x7f5f300fbaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:37.901 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:37.890+0000 7f5f35b27700 1 -- 172.21.15.39:0/4137007346 shutdown_connections 2023-07-21T19:07:37.901 INFO:teuthology.orchestra.run.smithi039.stderr:2023-07-21T19:07:37.890+0000 7f5f35b27700 1 -- 172.21.15.39:0/4137007346 wait complete. 2023-07-21T19:07:37.901 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_data' removed 2023-07-21T19:07:37.917 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-21T19:07:37.966 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-21T19:07:38.045 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-21T19:07:38.113 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2023-07-21T19:07:38.113 DEBUG:teuthology.orchestra.run.smithi039:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-07-21T19:07:38.130 DEBUG:teuthology.orchestra.run.smithi155:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-07-21T19:07:38.146 DEBUG:teuthology.orchestra.run.smithi193:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-07-21T19:07:38.162 INFO:tasks.cephadm:Stopping all daemons... 2023-07-21T19:07:38.163 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2023-07-21T19:07:38.163 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@mon.a 2023-07-21T19:07:38.175 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:37 smithi039 ceph-mon[96489]: pgmap v822: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.5 KiB/s wr, 13 op/s 2023-07-21T19:07:38.175 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:37 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/3564828223' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-07-21T19:07:38.175 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:37 smithi039 ceph-mon[96489]: osdmap e126: 12 total, 12 up, 12 in 2023-07-21T19:07:38.175 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:37 smithi039 ceph-mon[96489]: from='client.? 172.21.15.39:0/4137007346' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-21T19:07:38.357 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:37 smithi193 ceph-mon[98204]: pgmap v822: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.5 KiB/s wr, 13 op/s 2023-07-21T19:07:38.358 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:37 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/3564828223' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-07-21T19:07:38.358 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:37 smithi193 ceph-mon[98204]: osdmap e126: 12 total, 12 up, 12 in 2023-07-21T19:07:38.358 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:37 smithi193 ceph-mon[98204]: from='client.? 172.21.15.39:0/4137007346' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-21T19:07:38.376 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:37 smithi155 ceph-mon[97934]: pgmap v822: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.5 KiB/s wr, 13 op/s 2023-07-21T19:07:38.377 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:37 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/3564828223' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-07-21T19:07:38.377 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:37 smithi155 ceph-mon[97934]: osdmap e126: 12 total, 12 up, 12 in 2023-07-21T19:07:38.377 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:37 smithi155 ceph-mon[97934]: from='client.? 172.21.15.39:0/4137007346' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-21T19:07:38.597 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:38 smithi039 systemd[1]: Stopping Ceph mon.a for eaf8e25a-27f4-11ee-9b34-001a4aab830c... 2023-07-21T19:07:38.597 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:38 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mon-a[96465]: 2023-07-21T19:07:38.586+0000 7f0950d24700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T19:07:38.597 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:38 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mon-a[96465]: 2023-07-21T19:07:38.586+0000 7f0950d24700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2023-07-21T19:07:39.175 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 21 19:07:38 smithi039 bash[162959]: ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mon-a 2023-07-21T19:07:39.264 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@mon.a.service' 2023-07-21T19:07:39.361 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T19:07:39.361 INFO:tasks.cephadm.mon.a:Stopped mon.a 2023-07-21T19:07:39.362 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2023-07-21T19:07:39.362 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@mon.b 2023-07-21T19:07:39.771 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:39 smithi155 systemd[1]: Stopping Ceph mon.b for eaf8e25a-27f4-11ee-9b34-001a4aab830c... 2023-07-21T19:07:40.126 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:39 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mon-b[97910]: 2023-07-21T19:07:39.760+0000 7f65ba61c700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T19:07:40.126 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:39 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mon-b[97910]: 2023-07-21T19:07:39.760+0000 7f65ba61c700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2023-07-21T19:07:40.594 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 21 19:07:40 smithi155 bash[132531]: ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mon-b 2023-07-21T19:07:40.600 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@mon.b.service' 2023-07-21T19:07:40.649 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T19:07:40.650 INFO:tasks.cephadm.mon.c:Stopped mon.b 2023-07-21T19:07:40.650 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2023-07-21T19:07:40.651 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@mon.c 2023-07-21T19:07:41.058 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:40 smithi193 systemd[1]: Stopping Ceph mon.c for eaf8e25a-27f4-11ee-9b34-001a4aab830c... 2023-07-21T19:07:41.357 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:41 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mon-c[98180]: 2023-07-21T19:07:41.048+0000 7f2ebdcd8700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T19:07:41.357 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:41 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mon-c[98180]: 2023-07-21T19:07:41.048+0000 7f2ebdcd8700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2023-07-21T19:07:41.822 INFO:journalctl@ceph.mon.c.smithi193.stdout:Jul 21 19:07:41 smithi193 bash[136525]: ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mon-c 2023-07-21T19:07:41.829 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@mon.c.service' 2023-07-21T19:07:41.884 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T19:07:41.885 INFO:tasks.cephadm.mon.c:Stopped mon.c 2023-07-21T19:07:41.885 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2023-07-21T19:07:41.886 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@mgr.x 2023-07-21T19:07:42.175 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 19:07:41 smithi039 systemd[1]: Stopping Ceph mgr.x for eaf8e25a-27f4-11ee-9b34-001a4aab830c... 2023-07-21T19:07:42.545 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 19:07:42 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T19:07:42.227+0000 7f93c491d700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T19:07:42.545 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 19:07:42 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x[96735]: 2023-07-21T19:07:42.227+0000 7f93c491d700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2023-07-21T19:07:42.791 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@mgr.x.service' 2023-07-21T19:07:42.816 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 19:07:42 smithi039 bash[163103]: ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-x 2023-07-21T19:07:42.816 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 19:07:42 smithi039 systemd[1]: ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@mgr.x.service: Succeeded. 2023-07-21T19:07:42.817 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 21 19:07:42 smithi039 systemd[1]: Stopped Ceph mgr.x for eaf8e25a-27f4-11ee-9b34-001a4aab830c. 2023-07-21T19:07:42.829 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T19:07:42.829 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2023-07-21T19:07:42.830 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2023-07-21T19:07:42.830 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@mgr.y 2023-07-21T19:07:43.126 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 21 19:07:42 smithi155 systemd[1]: Stopping Ceph mgr.y for eaf8e25a-27f4-11ee-9b34-001a4aab830c... 2023-07-21T19:07:43.725 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 21 19:07:43 smithi155 bash[132678]: ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-y 2023-07-21T19:07:43.725 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 21 19:07:43 smithi155 systemd[1]: ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@mgr.y.service: Main process exited, code=exited, status=143/n/a 2023-07-21T19:07:43.730 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@mgr.y.service' 2023-07-21T19:07:43.769 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T19:07:43.769 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2023-07-21T19:07:43.770 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2023-07-21T19:07:43.770 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@mgr.z 2023-07-21T19:07:44.107 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Jul 21 19:07:43 smithi193 systemd[1]: Stopping Ceph mgr.z for eaf8e25a-27f4-11ee-9b34-001a4aab830c... 2023-07-21T19:07:44.575 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Jul 21 19:07:44 smithi193 bash[136673]: ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mgr-z 2023-07-21T19:07:44.576 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Jul 21 19:07:44 smithi193 systemd[1]: ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@mgr.z.service: Main process exited, code=exited, status=143/n/a 2023-07-21T19:07:44.582 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@mgr.z.service' 2023-07-21T19:07:44.619 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T19:07:44.620 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2023-07-21T19:07:44.620 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2023-07-21T19:07:44.621 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@osd.0 2023-07-21T19:07:44.925 INFO:journalctl@ceph.osd.0.smithi039.stdout:Jul 21 19:07:44 smithi039 systemd[1]: Stopping Ceph osd.0 for eaf8e25a-27f4-11ee-9b34-001a4aab830c... 2023-07-21T19:07:45.425 INFO:journalctl@ceph.osd.0.smithi039.stdout:Jul 21 19:07:44 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-0[109474]: 2023-07-21T19:07:44.962+0000 7f668fd12700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T19:07:45.425 INFO:journalctl@ceph.osd.0.smithi039.stdout:Jul 21 19:07:44 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-0[109474]: 2023-07-21T19:07:44.962+0000 7f668fd12700 -1 osd.0 127 *** Got signal Terminated *** 2023-07-21T19:07:45.425 INFO:journalctl@ceph.osd.0.smithi039.stdout:Jul 21 19:07:44 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-0[109474]: 2023-07-21T19:07:44.962+0000 7f668fd12700 -1 osd.0 127 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-21T19:07:51.175 INFO:journalctl@ceph.osd.0.smithi039.stdout:Jul 21 19:07:50 smithi039 bash[163244]: ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-0 2023-07-21T19:07:52.220 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@osd.0.service' 2023-07-21T19:07:52.264 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T19:07:52.265 INFO:tasks.cephadm.osd.0:Stopped osd.0 2023-07-21T19:07:52.265 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2023-07-21T19:07:52.266 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@osd.1 2023-07-21T19:07:52.626 INFO:journalctl@ceph.osd.1.smithi155.stdout:Jul 21 19:07:52 smithi155 systemd[1]: Stopping Ceph osd.1 for eaf8e25a-27f4-11ee-9b34-001a4aab830c... 2023-07-21T19:07:53.126 INFO:journalctl@ceph.osd.1.smithi155.stdout:Jul 21 19:07:52 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-1[103027]: 2023-07-21T19:07:52.616+0000 7f54ffb96700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T19:07:53.126 INFO:journalctl@ceph.osd.1.smithi155.stdout:Jul 21 19:07:52 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-1[103027]: 2023-07-21T19:07:52.616+0000 7f54ffb96700 -1 osd.1 127 *** Got signal Terminated *** 2023-07-21T19:07:53.126 INFO:journalctl@ceph.osd.1.smithi155.stdout:Jul 21 19:07:52 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-1[103027]: 2023-07-21T19:07:52.616+0000 7f54ffb96700 -1 osd.1 127 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-21T19:07:58.876 INFO:journalctl@ceph.osd.1.smithi155.stdout:Jul 21 19:07:58 smithi155 bash[132826]: ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-1 2023-07-21T19:07:59.977 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@osd.1.service' 2023-07-21T19:08:00.017 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T19:08:00.018 INFO:tasks.cephadm.osd.1:Stopped osd.1 2023-07-21T19:08:00.018 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2023-07-21T19:08:00.018 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@osd.2 2023-07-21T19:08:00.357 INFO:journalctl@ceph.osd.2.smithi193.stdout:Jul 21 19:08:00 smithi193 systemd[1]: Stopping Ceph osd.2 for eaf8e25a-27f4-11ee-9b34-001a4aab830c... 2023-07-21T19:08:00.857 INFO:journalctl@ceph.osd.2.smithi193.stdout:Jul 21 19:08:00 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-2[104065]: 2023-07-21T19:08:00.362+0000 7f6798169700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T19:08:00.857 INFO:journalctl@ceph.osd.2.smithi193.stdout:Jul 21 19:08:00 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-2[104065]: 2023-07-21T19:08:00.362+0000 7f6798169700 -1 osd.2 127 *** Got signal Terminated *** 2023-07-21T19:08:00.858 INFO:journalctl@ceph.osd.2.smithi193.stdout:Jul 21 19:08:00 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-2[104065]: 2023-07-21T19:08:00.362+0000 7f6798169700 -1 osd.2 127 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-21T19:08:06.358 INFO:journalctl@ceph.osd.2.smithi193.stdout:Jul 21 19:08:06 smithi193 bash[136816]: ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-2 2023-07-21T19:08:07.749 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@osd.2.service' 2023-07-21T19:08:07.792 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T19:08:07.793 INFO:tasks.cephadm.osd.2:Stopped osd.2 2023-07-21T19:08:07.793 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2023-07-21T19:08:07.793 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@osd.3 2023-07-21T19:08:08.160 INFO:journalctl@ceph.osd.3.smithi039.stdout:Jul 21 19:08:07 smithi039 systemd[1]: Stopping Ceph osd.3 for eaf8e25a-27f4-11ee-9b34-001a4aab830c... 2023-07-21T19:08:08.425 INFO:journalctl@ceph.osd.3.smithi039.stdout:Jul 21 19:08:08 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-3[114860]: 2023-07-21T19:08:08.149+0000 7f79dd7ec700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T19:08:08.425 INFO:journalctl@ceph.osd.3.smithi039.stdout:Jul 21 19:08:08 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-3[114860]: 2023-07-21T19:08:08.149+0000 7f79dd7ec700 -1 osd.3 127 *** Got signal Terminated *** 2023-07-21T19:08:08.425 INFO:journalctl@ceph.osd.3.smithi039.stdout:Jul 21 19:08:08 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-3[114860]: 2023-07-21T19:08:08.149+0000 7f79dd7ec700 -1 osd.3 127 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-21T19:08:11.675 INFO:journalctl@ceph.osd.3.smithi039.stdout:Jul 21 19:08:11 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-3[114860]: 2023-07-21T19:08:11.238+0000 7f79d9de2700 -1 osd.3 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:47.779015+0000 front 2023-07-21T19:07:47.779054+0000 (oldest deadline 2023-07-21T19:08:10.579922+0000) 2023-07-21T19:08:11.857 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:11 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:11.405+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:47.629329+0000 front 2023-07-21T19:07:47.629176+0000 (oldest deadline 2023-07-21T19:08:10.529277+0000) 2023-07-21T19:08:11.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:11 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:11.468+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:45.542174+0000 front 2023-07-21T19:07:45.542192+0000 (oldest deadline 2023-07-21T19:08:11.442323+0000) 2023-07-21T19:08:12.526 INFO:journalctl@ceph.osd.3.smithi039.stdout:Jul 21 19:08:12 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-3[114860]: 2023-07-21T19:08:12.255+0000 7f79d9de2700 -1 osd.3 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:47.779015+0000 front 2023-07-21T19:07:47.779054+0000 (oldest deadline 2023-07-21T19:08:10.579922+0000) 2023-07-21T19:08:12.526 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:12 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:12.232+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:46.109399+0000 front 2023-07-21T19:07:46.109431+0000 (oldest deadline 2023-07-21T19:08:11.410380+0000) 2023-07-21T19:08:12.857 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:12 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:12.389+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:47.629329+0000 front 2023-07-21T19:07:47.629176+0000 (oldest deadline 2023-07-21T19:08:10.529277+0000) 2023-07-21T19:08:12.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:12 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:12.443+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:45.542174+0000 front 2023-07-21T19:07:45.542192+0000 (oldest deadline 2023-07-21T19:08:11.442323+0000) 2023-07-21T19:08:12.925 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 21 19:08:12 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-6[120362]: 2023-07-21T19:08:12.516+0000 7fe22b5f5700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.093785+0000 front 2023-07-21T19:07:48.093754+0000 (oldest deadline 2023-07-21T19:08:11.593863+0000) 2023-07-21T19:08:13.514 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:13 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:13.226+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:46.109399+0000 front 2023-07-21T19:07:46.109431+0000 (oldest deadline 2023-07-21T19:08:11.410380+0000) 2023-07-21T19:08:13.626 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:13 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:13.260+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.769616+0000 front 2023-07-21T19:07:48.769555+0000 (oldest deadline 2023-07-21T19:08:12.770222+0000) 2023-07-21T19:08:13.855 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 21 19:08:13 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-6[120362]: 2023-07-21T19:08:13.504+0000 7fe22b5f5700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.093785+0000 front 2023-07-21T19:07:48.093754+0000 (oldest deadline 2023-07-21T19:08:11.593863+0000) 2023-07-21T19:08:13.857 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:13 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:13.367+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:47.629329+0000 front 2023-07-21T19:07:47.629176+0000 (oldest deadline 2023-07-21T19:08:10.529277+0000) 2023-07-21T19:08:13.857 INFO:journalctl@ceph.osd.5.smithi193.stdout:Jul 21 19:08:13 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-5[109570]: 2023-07-21T19:08:13.356+0000 7fdc63162700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:49.767864+0000 front 2023-07-21T19:07:49.767907+0000 (oldest deadline 2023-07-21T19:08:12.667755+0000) 2023-07-21T19:08:13.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:13 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:13.449+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:45.542174+0000 front 2023-07-21T19:07:45.542192+0000 (oldest deadline 2023-07-21T19:08:11.442323+0000) 2023-07-21T19:08:14.142 INFO:journalctl@ceph.osd.3.smithi039.stdout:Jul 21 19:08:14 smithi039 bash[163518]: ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-3 2023-07-21T19:08:14.626 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:14 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:14.260+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.769616+0000 front 2023-07-21T19:07:48.769555+0000 (oldest deadline 2023-07-21T19:08:12.770222+0000) 2023-07-21T19:08:14.675 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 21 19:08:14 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-6[120362]: 2023-07-21T19:08:14.470+0000 7fe22b5f5700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.093785+0000 front 2023-07-21T19:07:48.093754+0000 (oldest deadline 2023-07-21T19:08:11.593863+0000) 2023-07-21T19:08:14.675 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:14 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:14.231+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:46.109399+0000 front 2023-07-21T19:07:46.109431+0000 (oldest deadline 2023-07-21T19:08:11.410380+0000) 2023-07-21T19:08:14.857 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:14 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:14.395+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:47.629329+0000 front 2023-07-21T19:07:47.629176+0000 (oldest deadline 2023-07-21T19:08:10.529277+0000) 2023-07-21T19:08:14.857 INFO:journalctl@ceph.osd.5.smithi193.stdout:Jul 21 19:08:14 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-5[109570]: 2023-07-21T19:08:14.378+0000 7fdc63162700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:49.767864+0000 front 2023-07-21T19:07:49.767907+0000 (oldest deadline 2023-07-21T19:08:12.667755+0000) 2023-07-21T19:08:14.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:14 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:14.421+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:45.542174+0000 front 2023-07-21T19:07:45.542192+0000 (oldest deadline 2023-07-21T19:08:11.442323+0000) 2023-07-21T19:08:15.407 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:15 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:15.215+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:46.109399+0000 front 2023-07-21T19:07:46.109431+0000 (oldest deadline 2023-07-21T19:08:11.410380+0000) 2023-07-21T19:08:15.416 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@osd.3.service' 2023-07-21T19:08:15.458 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T19:08:15.460 INFO:tasks.cephadm.osd.3:Stopped osd.3 2023-07-21T19:08:15.460 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2023-07-21T19:08:15.460 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@osd.4 2023-07-21T19:08:15.626 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:15 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:15.262+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.769616+0000 front 2023-07-21T19:07:48.769555+0000 (oldest deadline 2023-07-21T19:08:12.770222+0000) 2023-07-21T19:08:15.626 INFO:journalctl@ceph.osd.4.smithi155.stdout:Jul 21 19:08:15 smithi155 systemd[1]: Stopping Ceph osd.4 for eaf8e25a-27f4-11ee-9b34-001a4aab830c... 2023-07-21T19:08:15.675 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 21 19:08:15 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-6[120362]: 2023-07-21T19:08:15.429+0000 7fe22b5f5700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.093785+0000 front 2023-07-21T19:07:48.093754+0000 (oldest deadline 2023-07-21T19:08:11.593863+0000) 2023-07-21T19:08:15.857 INFO:journalctl@ceph.osd.5.smithi193.stdout:Jul 21 19:08:15 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-5[109570]: 2023-07-21T19:08:15.382+0000 7fdc63162700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:49.767864+0000 front 2023-07-21T19:07:49.767907+0000 (oldest deadline 2023-07-21T19:08:12.667755+0000) 2023-07-21T19:08:15.857 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:15 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:15.375+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:47.629329+0000 front 2023-07-21T19:07:47.629176+0000 (oldest deadline 2023-07-21T19:08:10.529277+0000) 2023-07-21T19:08:15.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:15 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:15.426+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:45.542174+0000 front 2023-07-21T19:07:45.542192+0000 (oldest deadline 2023-07-21T19:08:11.442323+0000) 2023-07-21T19:08:16.376 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:16 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:16.214+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.769616+0000 front 2023-07-21T19:07:48.769555+0000 (oldest deadline 2023-07-21T19:08:12.770222+0000) 2023-07-21T19:08:16.376 INFO:journalctl@ceph.osd.4.smithi155.stdout:Jul 21 19:08:16 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-4[108462]: 2023-07-21T19:08:16.042+0000 7f62d19c4700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T19:08:16.376 INFO:journalctl@ceph.osd.4.smithi155.stdout:Jul 21 19:08:16 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-4[108462]: 2023-07-21T19:08:16.042+0000 7f62d19c4700 -1 osd.4 127 *** Got signal Terminated *** 2023-07-21T19:08:16.377 INFO:journalctl@ceph.osd.4.smithi155.stdout:Jul 21 19:08:16 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-4[108462]: 2023-07-21T19:08:16.042+0000 7f62d19c4700 -1 osd.4 127 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-21T19:08:16.466 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:16 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:16.178+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:46.109399+0000 front 2023-07-21T19:07:46.109431+0000 (oldest deadline 2023-07-21T19:08:11.410380+0000) 2023-07-21T19:08:16.857 INFO:journalctl@ceph.osd.5.smithi193.stdout:Jul 21 19:08:16 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-5[109570]: 2023-07-21T19:08:16.391+0000 7fdc63162700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:49.767864+0000 front 2023-07-21T19:07:49.767907+0000 (oldest deadline 2023-07-21T19:08:12.667755+0000) 2023-07-21T19:08:16.858 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:16 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:16.368+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:47.629329+0000 front 2023-07-21T19:07:47.629176+0000 (oldest deadline 2023-07-21T19:08:10.529277+0000) 2023-07-21T19:08:16.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:16 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:16.389+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:45.542174+0000 front 2023-07-21T19:07:45.542192+0000 (oldest deadline 2023-07-21T19:08:11.442323+0000) 2023-07-21T19:08:16.925 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 21 19:08:16 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-6[120362]: 2023-07-21T19:08:16.456+0000 7fe22b5f5700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.093785+0000 front 2023-07-21T19:07:48.093754+0000 (oldest deadline 2023-07-21T19:08:11.593863+0000) 2023-07-21T19:08:17.425 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:17 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:17.151+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:46.109399+0000 front 2023-07-21T19:07:46.109431+0000 (oldest deadline 2023-07-21T19:08:11.410380+0000) 2023-07-21T19:08:17.607 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:17 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:17.328+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:47.629329+0000 front 2023-07-21T19:07:47.629176+0000 (oldest deadline 2023-07-21T19:08:10.529277+0000) 2023-07-21T19:08:17.608 INFO:journalctl@ceph.osd.5.smithi193.stdout:Jul 21 19:08:17 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-5[109570]: 2023-07-21T19:08:17.436+0000 7fdc63162700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:49.767864+0000 front 2023-07-21T19:07:49.767907+0000 (oldest deadline 2023-07-21T19:08:12.667755+0000) 2023-07-21T19:08:17.608 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:17 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:17.403+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:45.542174+0000 front 2023-07-21T19:07:45.542192+0000 (oldest deadline 2023-07-21T19:08:11.442323+0000) 2023-07-21T19:08:17.626 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:17 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:17.218+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.769616+0000 front 2023-07-21T19:07:48.769555+0000 (oldest deadline 2023-07-21T19:08:12.770222+0000) 2023-07-21T19:08:17.925 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 21 19:08:17 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-6[120362]: 2023-07-21T19:08:17.494+0000 7fe22b5f5700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.093785+0000 front 2023-07-21T19:07:48.093754+0000 (oldest deadline 2023-07-21T19:08:11.593863+0000) 2023-07-21T19:08:18.425 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:18 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:18.116+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:46.109399+0000 front 2023-07-21T19:07:46.109431+0000 (oldest deadline 2023-07-21T19:08:11.410380+0000) 2023-07-21T19:08:18.607 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:18 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:18.286+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:47.629329+0000 front 2023-07-21T19:07:47.629176+0000 (oldest deadline 2023-07-21T19:08:10.529277+0000) 2023-07-21T19:08:18.608 INFO:journalctl@ceph.osd.5.smithi193.stdout:Jul 21 19:08:18 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-5[109570]: 2023-07-21T19:08:18.419+0000 7fdc63162700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:49.767864+0000 front 2023-07-21T19:07:49.767907+0000 (oldest deadline 2023-07-21T19:08:12.667755+0000) 2023-07-21T19:08:18.608 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:18 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:18.419+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:45.542174+0000 front 2023-07-21T19:07:45.542192+0000 (oldest deadline 2023-07-21T19:08:11.442323+0000) 2023-07-21T19:08:18.626 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:18 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:18.213+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.769616+0000 front 2023-07-21T19:07:48.769555+0000 (oldest deadline 2023-07-21T19:08:12.770222+0000) 2023-07-21T19:08:18.925 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 21 19:08:18 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-6[120362]: 2023-07-21T19:08:18.496+0000 7fe22b5f5700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.093785+0000 front 2023-07-21T19:07:48.093754+0000 (oldest deadline 2023-07-21T19:08:11.593863+0000) 2023-07-21T19:08:19.425 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:19 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:19.085+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:46.109399+0000 front 2023-07-21T19:07:46.109431+0000 (oldest deadline 2023-07-21T19:08:11.410380+0000) 2023-07-21T19:08:19.425 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:19 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:19.085+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.711568+0000 front 2023-07-21T19:07:56.711252+0000 (oldest deadline 2023-07-21T19:08:19.011261+0000) 2023-07-21T19:08:19.607 INFO:journalctl@ceph.osd.5.smithi193.stdout:Jul 21 19:08:19 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-5[109570]: 2023-07-21T19:08:19.421+0000 7fdc63162700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:49.767864+0000 front 2023-07-21T19:07:49.767907+0000 (oldest deadline 2023-07-21T19:08:12.667755+0000) 2023-07-21T19:08:19.607 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:19 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:19.283+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:47.629329+0000 front 2023-07-21T19:07:47.629176+0000 (oldest deadline 2023-07-21T19:08:10.529277+0000) 2023-07-21T19:08:19.608 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:19 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:19.462+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:45.542174+0000 front 2023-07-21T19:07:45.542192+0000 (oldest deadline 2023-07-21T19:08:11.442323+0000) 2023-07-21T19:08:19.608 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:19 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:19.462+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.044189+0000 front 2023-07-21T19:07:56.044156+0000 (oldest deadline 2023-07-21T19:08:18.944205+0000) 2023-07-21T19:08:19.626 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:19 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:19.213+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.769616+0000 front 2023-07-21T19:07:48.769555+0000 (oldest deadline 2023-07-21T19:08:12.770222+0000) 2023-07-21T19:08:19.925 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 21 19:08:19 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-6[120362]: 2023-07-21T19:08:19.543+0000 7fe22b5f5700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.093785+0000 front 2023-07-21T19:07:48.093754+0000 (oldest deadline 2023-07-21T19:08:11.593863+0000) 2023-07-21T19:08:20.376 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:20 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:20.165+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.769616+0000 front 2023-07-21T19:07:48.769555+0000 (oldest deadline 2023-07-21T19:08:12.770222+0000) 2023-07-21T19:08:20.376 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 21 19:08:20 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-7[114007]: 2023-07-21T19:08:20.074+0000 7fceded29700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:54.504220+0000 front 2023-07-21T19:07:54.504248+0000 (oldest deadline 2023-07-21T19:08:19.804390+0000) 2023-07-21T19:08:20.425 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:20 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:20.103+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:46.109399+0000 front 2023-07-21T19:07:46.109431+0000 (oldest deadline 2023-07-21T19:08:11.410380+0000) 2023-07-21T19:08:20.425 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:20 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:20.103+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.711568+0000 front 2023-07-21T19:07:56.711252+0000 (oldest deadline 2023-07-21T19:08:19.011261+0000) 2023-07-21T19:08:20.607 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:20 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:20.277+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:47.629329+0000 front 2023-07-21T19:07:47.629176+0000 (oldest deadline 2023-07-21T19:08:10.529277+0000) 2023-07-21T19:08:20.607 INFO:journalctl@ceph.osd.5.smithi193.stdout:Jul 21 19:08:20 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-5[109570]: 2023-07-21T19:08:20.388+0000 7fdc63162700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:49.767864+0000 front 2023-07-21T19:07:49.767907+0000 (oldest deadline 2023-07-21T19:08:12.667755+0000) 2023-07-21T19:08:20.608 INFO:journalctl@ceph.osd.5.smithi193.stdout:Jul 21 19:08:20 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-5[109570]: 2023-07-21T19:08:20.388+0000 7fdc63162700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.168989+0000 front 2023-07-21T19:07:56.169028+0000 (oldest deadline 2023-07-21T19:08:20.269093+0000) 2023-07-21T19:08:20.608 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:20 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:20.495+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:45.542174+0000 front 2023-07-21T19:07:45.542192+0000 (oldest deadline 2023-07-21T19:08:11.442323+0000) 2023-07-21T19:08:20.608 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:20 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:20.495+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.044189+0000 front 2023-07-21T19:07:56.044156+0000 (oldest deadline 2023-07-21T19:08:18.944205+0000) 2023-07-21T19:08:20.876 INFO:journalctl@ceph.osd.4.smithi155.stdout:Jul 21 19:08:20 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-4[108462]: 2023-07-21T19:08:20.429+0000 7f62cbfb6700 -1 osd.4 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.616245+0000 front 2023-07-21T19:07:56.616309+0000 (oldest deadline 2023-07-21T19:08:20.116410+0000) 2023-07-21T19:08:20.925 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 21 19:08:20 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-6[120362]: 2023-07-21T19:08:20.529+0000 7fe22b5f5700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.093785+0000 front 2023-07-21T19:07:48.093754+0000 (oldest deadline 2023-07-21T19:08:11.593863+0000) 2023-07-21T19:08:20.925 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 21 19:08:20 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-6[120362]: 2023-07-21T19:08:20.529+0000 7fe22b5f5700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:57.494878+0000 front 2023-07-21T19:07:57.495039+0000 (oldest deadline 2023-07-21T19:08:19.795061+0000) 2023-07-21T19:08:21.376 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:21 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:21.155+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.769616+0000 front 2023-07-21T19:07:48.769555+0000 (oldest deadline 2023-07-21T19:08:12.770222+0000) 2023-07-21T19:08:21.376 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 21 19:08:21 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-7[114007]: 2023-07-21T19:08:21.028+0000 7fceded29700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:54.504220+0000 front 2023-07-21T19:07:54.504248+0000 (oldest deadline 2023-07-21T19:08:19.804390+0000) 2023-07-21T19:08:21.425 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:21 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:21.118+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:46.109399+0000 front 2023-07-21T19:07:46.109431+0000 (oldest deadline 2023-07-21T19:08:11.410380+0000) 2023-07-21T19:08:21.425 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:21 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:21.118+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.711568+0000 front 2023-07-21T19:07:56.711252+0000 (oldest deadline 2023-07-21T19:08:19.011261+0000) 2023-07-21T19:08:21.607 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:21 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:21.254+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:47.629329+0000 front 2023-07-21T19:07:47.629176+0000 (oldest deadline 2023-07-21T19:08:10.529277+0000) 2023-07-21T19:08:21.608 INFO:journalctl@ceph.osd.5.smithi193.stdout:Jul 21 19:08:21 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-5[109570]: 2023-07-21T19:08:21.353+0000 7fdc63162700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:49.767864+0000 front 2023-07-21T19:07:49.767907+0000 (oldest deadline 2023-07-21T19:08:12.667755+0000) 2023-07-21T19:08:21.608 INFO:journalctl@ceph.osd.5.smithi193.stdout:Jul 21 19:08:21 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-5[109570]: 2023-07-21T19:08:21.353+0000 7fdc63162700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.168989+0000 front 2023-07-21T19:07:56.169028+0000 (oldest deadline 2023-07-21T19:08:20.269093+0000) 2023-07-21T19:08:21.608 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:21 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:21.465+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:45.542174+0000 front 2023-07-21T19:07:45.542192+0000 (oldest deadline 2023-07-21T19:08:11.442323+0000) 2023-07-21T19:08:21.608 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:21 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:21.465+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.044189+0000 front 2023-07-21T19:07:56.044156+0000 (oldest deadline 2023-07-21T19:08:18.944205+0000) 2023-07-21T19:08:21.912 INFO:journalctl@ceph.osd.4.smithi155.stdout:Jul 21 19:08:21 smithi155 bash[133092]: ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-4 2023-07-21T19:08:21.925 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 21 19:08:21 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-6[120362]: 2023-07-21T19:08:21.513+0000 7fe22b5f5700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.093785+0000 front 2023-07-21T19:07:48.093754+0000 (oldest deadline 2023-07-21T19:08:11.593863+0000) 2023-07-21T19:08:21.925 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 21 19:08:21 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-6[120362]: 2023-07-21T19:08:21.513+0000 7fe22b5f5700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:57.494878+0000 front 2023-07-21T19:07:57.495039+0000 (oldest deadline 2023-07-21T19:08:19.795061+0000) 2023-07-21T19:08:22.376 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 21 19:08:21 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-7[114007]: 2023-07-21T19:08:21.980+0000 7fceded29700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:54.504220+0000 front 2023-07-21T19:07:54.504248+0000 (oldest deadline 2023-07-21T19:08:19.804390+0000) 2023-07-21T19:08:22.376 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:22 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:22.138+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.769616+0000 front 2023-07-21T19:07:48.769555+0000 (oldest deadline 2023-07-21T19:08:12.770222+0000) 2023-07-21T19:08:22.425 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:22 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:22.138+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:46.109399+0000 front 2023-07-21T19:07:46.109431+0000 (oldest deadline 2023-07-21T19:08:11.410380+0000) 2023-07-21T19:08:22.425 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:22 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:22.138+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.711568+0000 front 2023-07-21T19:07:56.711252+0000 (oldest deadline 2023-07-21T19:08:19.011261+0000) 2023-07-21T19:08:22.517 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:22 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:22.224+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:47.629329+0000 front 2023-07-21T19:07:47.629176+0000 (oldest deadline 2023-07-21T19:08:10.529277+0000) 2023-07-21T19:08:22.517 INFO:journalctl@ceph.osd.5.smithi193.stdout:Jul 21 19:08:22 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-5[109570]: 2023-07-21T19:08:22.313+0000 7fdc63162700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:49.767864+0000 front 2023-07-21T19:07:49.767907+0000 (oldest deadline 2023-07-21T19:08:12.667755+0000) 2023-07-21T19:08:22.517 INFO:journalctl@ceph.osd.5.smithi193.stdout:Jul 21 19:08:22 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-5[109570]: 2023-07-21T19:08:22.313+0000 7fdc63162700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.168989+0000 front 2023-07-21T19:07:56.169028+0000 (oldest deadline 2023-07-21T19:08:20.269093+0000) 2023-07-21T19:08:22.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:22 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:22.507+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:45.542174+0000 front 2023-07-21T19:07:45.542192+0000 (oldest deadline 2023-07-21T19:08:11.442323+0000) 2023-07-21T19:08:22.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:22 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:22.507+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.044189+0000 front 2023-07-21T19:07:56.044156+0000 (oldest deadline 2023-07-21T19:08:18.944205+0000) 2023-07-21T19:08:22.925 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 21 19:08:22 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-6[120362]: 2023-07-21T19:08:22.509+0000 7fe22b5f5700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.093785+0000 front 2023-07-21T19:07:48.093754+0000 (oldest deadline 2023-07-21T19:08:11.593863+0000) 2023-07-21T19:08:22.925 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 21 19:08:22 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-6[120362]: 2023-07-21T19:08:22.509+0000 7fe22b5f5700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:57.494878+0000 front 2023-07-21T19:07:57.495039+0000 (oldest deadline 2023-07-21T19:08:19.795061+0000) 2023-07-21T19:08:23.126 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 21 19:08:22 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-7[114007]: 2023-07-21T19:08:22.989+0000 7fceded29700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:54.504220+0000 front 2023-07-21T19:07:54.504248+0000 (oldest deadline 2023-07-21T19:08:19.804390+0000) 2023-07-21T19:08:23.173 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@osd.4.service' 2023-07-21T19:08:23.214 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T19:08:23.214 INFO:tasks.cephadm.osd.4:Stopped osd.4 2023-07-21T19:08:23.214 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2023-07-21T19:08:23.215 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@osd.5 2023-07-21T19:08:23.425 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:23 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:23.142+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:46.109399+0000 front 2023-07-21T19:07:46.109431+0000 (oldest deadline 2023-07-21T19:08:11.410380+0000) 2023-07-21T19:08:23.425 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:23 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:23.142+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.711568+0000 front 2023-07-21T19:07:56.711252+0000 (oldest deadline 2023-07-21T19:08:19.011261+0000) 2023-07-21T19:08:23.499 INFO:journalctl@ceph.osd.5.smithi193.stdout:Jul 21 19:08:23 smithi193 systemd[1]: Stopping Ceph osd.5 for eaf8e25a-27f4-11ee-9b34-001a4aab830c... 2023-07-21T19:08:23.500 INFO:journalctl@ceph.osd.5.smithi193.stdout:Jul 21 19:08:23 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-5[109570]: 2023-07-21T19:08:23.317+0000 7fdc63162700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:49.767864+0000 front 2023-07-21T19:07:49.767907+0000 (oldest deadline 2023-07-21T19:08:12.667755+0000) 2023-07-21T19:08:23.500 INFO:journalctl@ceph.osd.5.smithi193.stdout:Jul 21 19:08:23 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-5[109570]: 2023-07-21T19:08:23.317+0000 7fdc63162700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.168989+0000 front 2023-07-21T19:07:56.169028+0000 (oldest deadline 2023-07-21T19:08:20.269093+0000) 2023-07-21T19:08:23.500 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:23 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:23.261+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:47.629329+0000 front 2023-07-21T19:07:47.629176+0000 (oldest deadline 2023-07-21T19:08:10.529277+0000) 2023-07-21T19:08:23.626 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:23 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:23.128+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.769616+0000 front 2023-07-21T19:07:48.769555+0000 (oldest deadline 2023-07-21T19:08:12.770222+0000) 2023-07-21T19:08:23.857 INFO:journalctl@ceph.osd.5.smithi193.stdout:Jul 21 19:08:23 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-5[109570]: 2023-07-21T19:08:23.549+0000 7fdc6836f700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T19:08:23.858 INFO:journalctl@ceph.osd.5.smithi193.stdout:Jul 21 19:08:23 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-5[109570]: 2023-07-21T19:08:23.549+0000 7fdc6836f700 -1 osd.5 127 *** Got signal Terminated *** 2023-07-21T19:08:23.858 INFO:journalctl@ceph.osd.5.smithi193.stdout:Jul 21 19:08:23 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-5[109570]: 2023-07-21T19:08:23.549+0000 7fdc6836f700 -1 osd.5 127 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-21T19:08:23.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:23 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:23.489+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:45.542174+0000 front 2023-07-21T19:07:45.542192+0000 (oldest deadline 2023-07-21T19:08:11.442323+0000) 2023-07-21T19:08:23.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:23 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:23.489+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.044189+0000 front 2023-07-21T19:07:56.044156+0000 (oldest deadline 2023-07-21T19:08:18.944205+0000) 2023-07-21T19:08:23.925 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 21 19:08:23 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-6[120362]: 2023-07-21T19:08:23.543+0000 7fe22b5f5700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.093785+0000 front 2023-07-21T19:07:48.093754+0000 (oldest deadline 2023-07-21T19:08:11.593863+0000) 2023-07-21T19:08:23.925 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 21 19:08:23 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-6[120362]: 2023-07-21T19:08:23.543+0000 7fe22b5f5700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:57.494878+0000 front 2023-07-21T19:07:57.495039+0000 (oldest deadline 2023-07-21T19:08:19.795061+0000) 2023-07-21T19:08:24.376 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 21 19:08:23 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-7[114007]: 2023-07-21T19:08:23.956+0000 7fceded29700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:54.504220+0000 front 2023-07-21T19:07:54.504248+0000 (oldest deadline 2023-07-21T19:08:19.804390+0000) 2023-07-21T19:08:24.376 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:24 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:24.139+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.769616+0000 front 2023-07-21T19:07:48.769555+0000 (oldest deadline 2023-07-21T19:08:12.770222+0000) 2023-07-21T19:08:24.376 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:24 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:24.139+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.271039+0000 front 2023-07-21T19:07:56.271067+0000 (oldest deadline 2023-07-21T19:08:23.871770+0000) 2023-07-21T19:08:24.504 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:24 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:24.231+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:47.629329+0000 front 2023-07-21T19:07:47.629176+0000 (oldest deadline 2023-07-21T19:08:10.529277+0000) 2023-07-21T19:08:24.505 INFO:journalctl@ceph.osd.5.smithi193.stdout:Jul 21 19:08:24 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-5[109570]: 2023-07-21T19:08:24.335+0000 7fdc63162700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:49.767864+0000 front 2023-07-21T19:07:49.767907+0000 (oldest deadline 2023-07-21T19:08:12.667755+0000) 2023-07-21T19:08:24.505 INFO:journalctl@ceph.osd.5.smithi193.stdout:Jul 21 19:08:24 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-5[109570]: 2023-07-21T19:08:24.335+0000 7fdc63162700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.168989+0000 front 2023-07-21T19:07:56.169028+0000 (oldest deadline 2023-07-21T19:08:20.269093+0000) 2023-07-21T19:08:24.569 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:24 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:24.168+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:46.109399+0000 front 2023-07-21T19:07:46.109431+0000 (oldest deadline 2023-07-21T19:08:11.410380+0000) 2023-07-21T19:08:24.569 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:24 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:24.168+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.711568+0000 front 2023-07-21T19:07:56.711252+0000 (oldest deadline 2023-07-21T19:08:19.011261+0000) 2023-07-21T19:08:24.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:24 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:24.494+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:45.542174+0000 front 2023-07-21T19:07:45.542192+0000 (oldest deadline 2023-07-21T19:08:11.442323+0000) 2023-07-21T19:08:24.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:24 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:24.494+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.044189+0000 front 2023-07-21T19:07:56.044156+0000 (oldest deadline 2023-07-21T19:08:18.944205+0000) 2023-07-21T19:08:24.925 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 21 19:08:24 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-6[120362]: 2023-07-21T19:08:24.559+0000 7fe22b5f5700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.093785+0000 front 2023-07-21T19:07:48.093754+0000 (oldest deadline 2023-07-21T19:08:11.593863+0000) 2023-07-21T19:08:24.925 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 21 19:08:24 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-6[120362]: 2023-07-21T19:08:24.559+0000 7fe22b5f5700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:57.494878+0000 front 2023-07-21T19:07:57.495039+0000 (oldest deadline 2023-07-21T19:08:19.795061+0000) 2023-07-21T19:08:25.376 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 21 19:08:24 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-7[114007]: 2023-07-21T19:08:24.926+0000 7fceded29700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:54.504220+0000 front 2023-07-21T19:07:54.504248+0000 (oldest deadline 2023-07-21T19:08:19.804390+0000) 2023-07-21T19:08:25.376 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:25 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:25.151+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.769616+0000 front 2023-07-21T19:07:48.769555+0000 (oldest deadline 2023-07-21T19:08:12.770222+0000) 2023-07-21T19:08:25.376 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:25 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:25.151+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.271039+0000 front 2023-07-21T19:07:56.271067+0000 (oldest deadline 2023-07-21T19:08:23.871770+0000) 2023-07-21T19:08:25.425 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:25 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:25.118+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:46.109399+0000 front 2023-07-21T19:07:46.109431+0000 (oldest deadline 2023-07-21T19:08:11.410380+0000) 2023-07-21T19:08:25.425 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:25 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:25.118+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.711568+0000 front 2023-07-21T19:07:56.711252+0000 (oldest deadline 2023-07-21T19:08:19.011261+0000) 2023-07-21T19:08:25.534 INFO:journalctl@ceph.osd.5.smithi193.stdout:Jul 21 19:08:25 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-5[109570]: 2023-07-21T19:08:25.355+0000 7fdc63162700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:49.767864+0000 front 2023-07-21T19:07:49.767907+0000 (oldest deadline 2023-07-21T19:08:12.667755+0000) 2023-07-21T19:08:25.535 INFO:journalctl@ceph.osd.5.smithi193.stdout:Jul 21 19:08:25 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-5[109570]: 2023-07-21T19:08:25.355+0000 7fdc63162700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.168989+0000 front 2023-07-21T19:07:56.169028+0000 (oldest deadline 2023-07-21T19:08:20.269093+0000) 2023-07-21T19:08:25.535 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:25 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:25.202+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:47.629329+0000 front 2023-07-21T19:07:47.629176+0000 (oldest deadline 2023-07-21T19:08:10.529277+0000) 2023-07-21T19:08:25.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:25 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:25.525+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:45.542174+0000 front 2023-07-21T19:07:45.542192+0000 (oldest deadline 2023-07-21T19:08:11.442323+0000) 2023-07-21T19:08:25.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:25 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:25.525+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.044189+0000 front 2023-07-21T19:07:56.044156+0000 (oldest deadline 2023-07-21T19:08:18.944205+0000) 2023-07-21T19:08:25.925 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 21 19:08:25 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-6[120362]: 2023-07-21T19:08:25.543+0000 7fe22b5f5700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.093785+0000 front 2023-07-21T19:07:48.093754+0000 (oldest deadline 2023-07-21T19:08:11.593863+0000) 2023-07-21T19:08:25.925 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 21 19:08:25 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-6[120362]: 2023-07-21T19:08:25.543+0000 7fe22b5f5700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:57.494878+0000 front 2023-07-21T19:07:57.495039+0000 (oldest deadline 2023-07-21T19:08:19.795061+0000) 2023-07-21T19:08:26.376 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 21 19:08:25 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-7[114007]: 2023-07-21T19:08:25.944+0000 7fceded29700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:54.504220+0000 front 2023-07-21T19:07:54.504248+0000 (oldest deadline 2023-07-21T19:08:19.804390+0000) 2023-07-21T19:08:26.376 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:26 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:26.138+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.769616+0000 front 2023-07-21T19:07:48.769555+0000 (oldest deadline 2023-07-21T19:08:12.770222+0000) 2023-07-21T19:08:26.376 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:26 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:26.138+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.271039+0000 front 2023-07-21T19:07:56.271067+0000 (oldest deadline 2023-07-21T19:08:23.871770+0000) 2023-07-21T19:08:26.425 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:26 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:26.113+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:46.109399+0000 front 2023-07-21T19:07:46.109431+0000 (oldest deadline 2023-07-21T19:08:11.410380+0000) 2023-07-21T19:08:26.425 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:26 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:26.113+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.711568+0000 front 2023-07-21T19:07:56.711252+0000 (oldest deadline 2023-07-21T19:08:19.011261+0000) 2023-07-21T19:08:26.527 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:26 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:26.184+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:47.629329+0000 front 2023-07-21T19:07:47.629176+0000 (oldest deadline 2023-07-21T19:08:10.529277+0000) 2023-07-21T19:08:26.528 INFO:journalctl@ceph.osd.5.smithi193.stdout:Jul 21 19:08:26 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-5[109570]: 2023-07-21T19:08:26.319+0000 7fdc63162700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:49.767864+0000 front 2023-07-21T19:07:49.767907+0000 (oldest deadline 2023-07-21T19:08:12.667755+0000) 2023-07-21T19:08:26.528 INFO:journalctl@ceph.osd.5.smithi193.stdout:Jul 21 19:08:26 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-5[109570]: 2023-07-21T19:08:26.319+0000 7fdc63162700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.168989+0000 front 2023-07-21T19:07:56.169028+0000 (oldest deadline 2023-07-21T19:08:20.269093+0000) 2023-07-21T19:08:26.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:26 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:26.518+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:45.542174+0000 front 2023-07-21T19:07:45.542192+0000 (oldest deadline 2023-07-21T19:08:11.442323+0000) 2023-07-21T19:08:26.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:26 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:26.518+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.044189+0000 front 2023-07-21T19:07:56.044156+0000 (oldest deadline 2023-07-21T19:08:18.944205+0000) 2023-07-21T19:08:26.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:26 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:26.518+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:02.345832+0000 front 2023-07-21T19:08:02.345919+0000 (oldest deadline 2023-07-21T19:08:26.445973+0000) 2023-07-21T19:08:26.925 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 21 19:08:26 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-6[120362]: 2023-07-21T19:08:26.560+0000 7fe22b5f5700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.093785+0000 front 2023-07-21T19:07:48.093754+0000 (oldest deadline 2023-07-21T19:08:11.593863+0000) 2023-07-21T19:08:26.925 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 21 19:08:26 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-6[120362]: 2023-07-21T19:08:26.560+0000 7fe22b5f5700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:57.494878+0000 front 2023-07-21T19:07:57.495039+0000 (oldest deadline 2023-07-21T19:08:19.795061+0000) 2023-07-21T19:08:27.376 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 21 19:08:26 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-7[114007]: 2023-07-21T19:08:26.973+0000 7fceded29700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:54.504220+0000 front 2023-07-21T19:07:54.504248+0000 (oldest deadline 2023-07-21T19:08:19.804390+0000) 2023-07-21T19:08:27.376 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:27 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:27.093+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.769616+0000 front 2023-07-21T19:07:48.769555+0000 (oldest deadline 2023-07-21T19:08:12.770222+0000) 2023-07-21T19:08:27.376 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:27 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:27.093+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.271039+0000 front 2023-07-21T19:07:56.271067+0000 (oldest deadline 2023-07-21T19:08:23.871770+0000) 2023-07-21T19:08:27.377 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:27 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:27.093+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:03.872288+0000 front 2023-07-21T19:08:03.872364+0000 (oldest deadline 2023-07-21T19:08:26.172487+0000) 2023-07-21T19:08:27.425 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:27 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:27.111+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:46.109399+0000 front 2023-07-21T19:07:46.109431+0000 (oldest deadline 2023-07-21T19:08:11.410380+0000) 2023-07-21T19:08:27.426 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:27 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:27.111+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.711568+0000 front 2023-07-21T19:07:56.711252+0000 (oldest deadline 2023-07-21T19:08:19.011261+0000) 2023-07-21T19:08:27.516 INFO:journalctl@ceph.osd.5.smithi193.stdout:Jul 21 19:08:27 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-5[109570]: 2023-07-21T19:08:27.313+0000 7fdc63162700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:49.767864+0000 front 2023-07-21T19:07:49.767907+0000 (oldest deadline 2023-07-21T19:08:12.667755+0000) 2023-07-21T19:08:27.516 INFO:journalctl@ceph.osd.5.smithi193.stdout:Jul 21 19:08:27 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-5[109570]: 2023-07-21T19:08:27.313+0000 7fdc63162700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.168989+0000 front 2023-07-21T19:07:56.169028+0000 (oldest deadline 2023-07-21T19:08:20.269093+0000) 2023-07-21T19:08:27.517 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:27 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:27.160+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:47.629329+0000 front 2023-07-21T19:07:47.629176+0000 (oldest deadline 2023-07-21T19:08:10.529277+0000) 2023-07-21T19:08:27.517 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:27 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:27.160+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:04.631984+0000 front 2023-07-21T19:08:04.631918+0000 (oldest deadline 2023-07-21T19:08:26.332176+0000) 2023-07-21T19:08:27.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:27 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:27.507+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:45.542174+0000 front 2023-07-21T19:07:45.542192+0000 (oldest deadline 2023-07-21T19:08:11.442323+0000) 2023-07-21T19:08:27.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:27 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:27.507+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.044189+0000 front 2023-07-21T19:07:56.044156+0000 (oldest deadline 2023-07-21T19:08:18.944205+0000) 2023-07-21T19:08:27.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:27 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:27.507+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:02.345832+0000 front 2023-07-21T19:08:02.345919+0000 (oldest deadline 2023-07-21T19:08:26.445973+0000) 2023-07-21T19:08:27.925 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 21 19:08:27 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-6[120362]: 2023-07-21T19:08:27.536+0000 7fe22b5f5700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.093785+0000 front 2023-07-21T19:07:48.093754+0000 (oldest deadline 2023-07-21T19:08:11.593863+0000) 2023-07-21T19:08:27.925 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 21 19:08:27 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-6[120362]: 2023-07-21T19:08:27.536+0000 7fe22b5f5700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:57.494878+0000 front 2023-07-21T19:07:57.495039+0000 (oldest deadline 2023-07-21T19:08:19.795061+0000) 2023-07-21T19:08:27.925 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 21 19:08:27 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-6[120362]: 2023-07-21T19:08:27.536+0000 7fe22b5f5700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:04.496384+0000 front 2023-07-21T19:08:04.496400+0000 (oldest deadline 2023-07-21T19:08:26.697105+0000) 2023-07-21T19:08:28.376 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 21 19:08:27 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-7[114007]: 2023-07-21T19:08:27.934+0000 7fceded29700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:54.504220+0000 front 2023-07-21T19:07:54.504248+0000 (oldest deadline 2023-07-21T19:08:19.804390+0000) 2023-07-21T19:08:28.376 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:28 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:28.139+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.769616+0000 front 2023-07-21T19:07:48.769555+0000 (oldest deadline 2023-07-21T19:08:12.770222+0000) 2023-07-21T19:08:28.376 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:28 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:28.139+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.271039+0000 front 2023-07-21T19:07:56.271067+0000 (oldest deadline 2023-07-21T19:08:23.871770+0000) 2023-07-21T19:08:28.377 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:28 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:28.139+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:03.872288+0000 front 2023-07-21T19:08:03.872364+0000 (oldest deadline 2023-07-21T19:08:26.172487+0000) 2023-07-21T19:08:28.425 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:28 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:28.089+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:46.109399+0000 front 2023-07-21T19:07:46.109431+0000 (oldest deadline 2023-07-21T19:08:11.410380+0000) 2023-07-21T19:08:28.425 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:28 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:28.089+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.711568+0000 front 2023-07-21T19:07:56.711252+0000 (oldest deadline 2023-07-21T19:08:19.011261+0000) 2023-07-21T19:08:28.534 INFO:journalctl@ceph.osd.5.smithi193.stdout:Jul 21 19:08:28 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-5[109570]: 2023-07-21T19:08:28.319+0000 7fdc63162700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:49.767864+0000 front 2023-07-21T19:07:49.767907+0000 (oldest deadline 2023-07-21T19:08:12.667755+0000) 2023-07-21T19:08:28.535 INFO:journalctl@ceph.osd.5.smithi193.stdout:Jul 21 19:08:28 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-5[109570]: 2023-07-21T19:08:28.320+0000 7fdc63162700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.168989+0000 front 2023-07-21T19:07:56.169028+0000 (oldest deadline 2023-07-21T19:08:20.269093+0000) 2023-07-21T19:08:28.535 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:28 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:28.154+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:47.629329+0000 front 2023-07-21T19:07:47.629176+0000 (oldest deadline 2023-07-21T19:08:10.529277+0000) 2023-07-21T19:08:28.535 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:28 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:28.154+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:04.631984+0000 front 2023-07-21T19:08:04.631918+0000 (oldest deadline 2023-07-21T19:08:26.332176+0000) 2023-07-21T19:08:28.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:28 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:28.525+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:45.542174+0000 front 2023-07-21T19:07:45.542192+0000 (oldest deadline 2023-07-21T19:08:11.442323+0000) 2023-07-21T19:08:28.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:28 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:28.525+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.044189+0000 front 2023-07-21T19:07:56.044156+0000 (oldest deadline 2023-07-21T19:08:18.944205+0000) 2023-07-21T19:08:28.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:28 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:28.525+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:02.345832+0000 front 2023-07-21T19:08:02.345919+0000 (oldest deadline 2023-07-21T19:08:26.445973+0000) 2023-07-21T19:08:28.925 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 21 19:08:28 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-6[120362]: 2023-07-21T19:08:28.545+0000 7fe22b5f5700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.093785+0000 front 2023-07-21T19:07:48.093754+0000 (oldest deadline 2023-07-21T19:08:11.593863+0000) 2023-07-21T19:08:28.925 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 21 19:08:28 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-6[120362]: 2023-07-21T19:08:28.545+0000 7fe22b5f5700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:57.494878+0000 front 2023-07-21T19:07:57.495039+0000 (oldest deadline 2023-07-21T19:08:19.795061+0000) 2023-07-21T19:08:28.926 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 21 19:08:28 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-6[120362]: 2023-07-21T19:08:28.545+0000 7fe22b5f5700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:04.496384+0000 front 2023-07-21T19:08:04.496400+0000 (oldest deadline 2023-07-21T19:08:26.697105+0000) 2023-07-21T19:08:29.376 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 21 19:08:28 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-7[114007]: 2023-07-21T19:08:28.885+0000 7fceded29700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:54.504220+0000 front 2023-07-21T19:07:54.504248+0000 (oldest deadline 2023-07-21T19:08:19.804390+0000) 2023-07-21T19:08:29.376 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:29 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:29.108+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.769616+0000 front 2023-07-21T19:07:48.769555+0000 (oldest deadline 2023-07-21T19:08:12.770222+0000) 2023-07-21T19:08:29.376 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:29 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:29.108+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.271039+0000 front 2023-07-21T19:07:56.271067+0000 (oldest deadline 2023-07-21T19:08:23.871770+0000) 2023-07-21T19:08:29.377 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:29 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:29.108+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:03.872288+0000 front 2023-07-21T19:08:03.872364+0000 (oldest deadline 2023-07-21T19:08:26.172487+0000) 2023-07-21T19:08:29.425 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:29 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:29.090+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:46.109399+0000 front 2023-07-21T19:07:46.109431+0000 (oldest deadline 2023-07-21T19:08:11.410380+0000) 2023-07-21T19:08:29.426 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:29 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:29.090+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.711568+0000 front 2023-07-21T19:07:56.711252+0000 (oldest deadline 2023-07-21T19:08:19.011261+0000) 2023-07-21T19:08:29.426 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:29 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:29.090+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:04.912868+0000 front 2023-07-21T19:08:04.912667+0000 (oldest deadline 2023-07-21T19:08:29.012921+0000) 2023-07-21T19:08:29.567 INFO:journalctl@ceph.osd.5.smithi193.stdout:Jul 21 19:08:29 smithi193 bash[137101]: ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-5 2023-07-21T19:08:29.568 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:29 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:29.116+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:47.629329+0000 front 2023-07-21T19:07:47.629176+0000 (oldest deadline 2023-07-21T19:08:10.529277+0000) 2023-07-21T19:08:29.568 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:29 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:29.116+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:04.631984+0000 front 2023-07-21T19:08:04.631918+0000 (oldest deadline 2023-07-21T19:08:26.332176+0000) 2023-07-21T19:08:29.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:29 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:29.557+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:45.542174+0000 front 2023-07-21T19:07:45.542192+0000 (oldest deadline 2023-07-21T19:08:11.442323+0000) 2023-07-21T19:08:29.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:29 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:29.557+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.044189+0000 front 2023-07-21T19:07:56.044156+0000 (oldest deadline 2023-07-21T19:08:18.944205+0000) 2023-07-21T19:08:29.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:29 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:29.557+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:02.345832+0000 front 2023-07-21T19:08:02.345919+0000 (oldest deadline 2023-07-21T19:08:26.445973+0000) 2023-07-21T19:08:29.925 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 21 19:08:29 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-6[120362]: 2023-07-21T19:08:29.568+0000 7fe22b5f5700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.093785+0000 front 2023-07-21T19:07:48.093754+0000 (oldest deadline 2023-07-21T19:08:11.593863+0000) 2023-07-21T19:08:29.926 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 21 19:08:29 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-6[120362]: 2023-07-21T19:08:29.568+0000 7fe22b5f5700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:57.494878+0000 front 2023-07-21T19:07:57.495039+0000 (oldest deadline 2023-07-21T19:08:19.795061+0000) 2023-07-21T19:08:29.926 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 21 19:08:29 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-6[120362]: 2023-07-21T19:08:29.568+0000 7fe22b5f5700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:04.496384+0000 front 2023-07-21T19:08:04.496400+0000 (oldest deadline 2023-07-21T19:08:26.697105+0000) 2023-07-21T19:08:30.333 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:30 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:30.160+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:47.629329+0000 front 2023-07-21T19:07:47.629176+0000 (oldest deadline 2023-07-21T19:08:10.529277+0000) 2023-07-21T19:08:30.334 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:30 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:30.160+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:04.631984+0000 front 2023-07-21T19:08:04.631918+0000 (oldest deadline 2023-07-21T19:08:26.332176+0000) 2023-07-21T19:08:30.376 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 21 19:08:29 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-7[114007]: 2023-07-21T19:08:29.895+0000 7fceded29700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:54.504220+0000 front 2023-07-21T19:07:54.504248+0000 (oldest deadline 2023-07-21T19:08:19.804390+0000) 2023-07-21T19:08:30.376 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 21 19:08:29 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-7[114007]: 2023-07-21T19:08:29.895+0000 7fceded29700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:03.905635+0000 front 2023-07-21T19:08:03.905615+0000 (oldest deadline 2023-07-21T19:08:29.805636+0000) 2023-07-21T19:08:30.376 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:30 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:30.118+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.769616+0000 front 2023-07-21T19:07:48.769555+0000 (oldest deadline 2023-07-21T19:08:12.770222+0000) 2023-07-21T19:08:30.376 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:30 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:30.118+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.271039+0000 front 2023-07-21T19:07:56.271067+0000 (oldest deadline 2023-07-21T19:08:23.871770+0000) 2023-07-21T19:08:30.377 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:30 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:30.118+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:03.872288+0000 front 2023-07-21T19:08:03.872364+0000 (oldest deadline 2023-07-21T19:08:26.172487+0000) 2023-07-21T19:08:30.425 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:30 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:30.060+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:46.109399+0000 front 2023-07-21T19:07:46.109431+0000 (oldest deadline 2023-07-21T19:08:11.410380+0000) 2023-07-21T19:08:30.425 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:30 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:30.060+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.711568+0000 front 2023-07-21T19:07:56.711252+0000 (oldest deadline 2023-07-21T19:08:19.011261+0000) 2023-07-21T19:08:30.425 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:30 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:30.060+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:04.912868+0000 front 2023-07-21T19:08:04.912667+0000 (oldest deadline 2023-07-21T19:08:29.012921+0000) 2023-07-21T19:08:30.662 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@osd.5.service' 2023-07-21T19:08:30.703 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T19:08:30.703 INFO:tasks.cephadm.osd.5:Stopped osd.5 2023-07-21T19:08:30.703 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2023-07-21T19:08:30.704 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@osd.6 2023-07-21T19:08:30.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:30 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:30.580+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:45.542174+0000 front 2023-07-21T19:07:45.542192+0000 (oldest deadline 2023-07-21T19:08:11.442323+0000) 2023-07-21T19:08:30.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:30 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:30.580+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.044189+0000 front 2023-07-21T19:07:56.044156+0000 (oldest deadline 2023-07-21T19:08:18.944205+0000) 2023-07-21T19:08:30.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:30 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:30.580+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:02.345832+0000 front 2023-07-21T19:08:02.345919+0000 (oldest deadline 2023-07-21T19:08:26.445973+0000) 2023-07-21T19:08:30.925 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 21 19:08:30 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-6[120362]: 2023-07-21T19:08:30.575+0000 7fe22b5f5700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.093785+0000 front 2023-07-21T19:07:48.093754+0000 (oldest deadline 2023-07-21T19:08:11.593863+0000) 2023-07-21T19:08:30.925 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 21 19:08:30 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-6[120362]: 2023-07-21T19:08:30.575+0000 7fe22b5f5700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:57.494878+0000 front 2023-07-21T19:07:57.495039+0000 (oldest deadline 2023-07-21T19:08:19.795061+0000) 2023-07-21T19:08:30.925 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 21 19:08:30 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-6[120362]: 2023-07-21T19:08:30.575+0000 7fe22b5f5700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:04.496384+0000 front 2023-07-21T19:08:04.496400+0000 (oldest deadline 2023-07-21T19:08:26.697105+0000) 2023-07-21T19:08:30.925 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 21 19:08:30 smithi039 systemd[1]: Stopping Ceph osd.6 for eaf8e25a-27f4-11ee-9b34-001a4aab830c... 2023-07-21T19:08:31.169 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 21 19:08:30 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-7[114007]: 2023-07-21T19:08:30.891+0000 7fceded29700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:54.504220+0000 front 2023-07-21T19:07:54.504248+0000 (oldest deadline 2023-07-21T19:08:19.804390+0000) 2023-07-21T19:08:31.169 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 21 19:08:30 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-7[114007]: 2023-07-21T19:08:30.891+0000 7fceded29700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:03.905635+0000 front 2023-07-21T19:08:03.905615+0000 (oldest deadline 2023-07-21T19:08:29.805636+0000) 2023-07-21T19:08:31.425 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 21 19:08:31 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-6[120362]: 2023-07-21T19:08:31.071+0000 7fe230802700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T19:08:31.426 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 21 19:08:31 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-6[120362]: 2023-07-21T19:08:31.071+0000 7fe230802700 -1 osd.6 127 *** Got signal Terminated *** 2023-07-21T19:08:31.426 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 21 19:08:31 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-6[120362]: 2023-07-21T19:08:31.071+0000 7fe230802700 -1 osd.6 127 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-21T19:08:31.426 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:31 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:31.092+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:46.109399+0000 front 2023-07-21T19:07:46.109431+0000 (oldest deadline 2023-07-21T19:08:11.410380+0000) 2023-07-21T19:08:31.426 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:31 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:31.092+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.711568+0000 front 2023-07-21T19:07:56.711252+0000 (oldest deadline 2023-07-21T19:08:19.011261+0000) 2023-07-21T19:08:31.426 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:31 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:31.092+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:04.912868+0000 front 2023-07-21T19:08:04.912667+0000 (oldest deadline 2023-07-21T19:08:29.012921+0000) 2023-07-21T19:08:31.554 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:31 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:31.209+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:47.629329+0000 front 2023-07-21T19:07:47.629176+0000 (oldest deadline 2023-07-21T19:08:10.529277+0000) 2023-07-21T19:08:31.554 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:31 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:31.209+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:04.631984+0000 front 2023-07-21T19:08:04.631918+0000 (oldest deadline 2023-07-21T19:08:26.332176+0000) 2023-07-21T19:08:31.626 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:31 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:31.159+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.769616+0000 front 2023-07-21T19:07:48.769555+0000 (oldest deadline 2023-07-21T19:08:12.770222+0000) 2023-07-21T19:08:31.626 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:31 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:31.159+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.271039+0000 front 2023-07-21T19:07:56.271067+0000 (oldest deadline 2023-07-21T19:08:23.871770+0000) 2023-07-21T19:08:31.626 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:31 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:31.159+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:03.872288+0000 front 2023-07-21T19:08:03.872364+0000 (oldest deadline 2023-07-21T19:08:26.172487+0000) 2023-07-21T19:08:31.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:31 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:31.545+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:45.542174+0000 front 2023-07-21T19:07:45.542192+0000 (oldest deadline 2023-07-21T19:08:11.442323+0000) 2023-07-21T19:08:31.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:31 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:31.545+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.044189+0000 front 2023-07-21T19:07:56.044156+0000 (oldest deadline 2023-07-21T19:08:18.944205+0000) 2023-07-21T19:08:31.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:31 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:31.545+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:02.345832+0000 front 2023-07-21T19:08:02.345919+0000 (oldest deadline 2023-07-21T19:08:26.445973+0000) 2023-07-21T19:08:31.925 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 21 19:08:31 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-6[120362]: 2023-07-21T19:08:31.569+0000 7fe22b5f5700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.093785+0000 front 2023-07-21T19:07:48.093754+0000 (oldest deadline 2023-07-21T19:08:11.593863+0000) 2023-07-21T19:08:31.925 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 21 19:08:31 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-6[120362]: 2023-07-21T19:08:31.569+0000 7fe22b5f5700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:57.494878+0000 front 2023-07-21T19:07:57.495039+0000 (oldest deadline 2023-07-21T19:08:19.795061+0000) 2023-07-21T19:08:31.925 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 21 19:08:31 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-6[120362]: 2023-07-21T19:08:31.569+0000 7fe22b5f5700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:04.496384+0000 front 2023-07-21T19:08:04.496400+0000 (oldest deadline 2023-07-21T19:08:26.697105+0000) 2023-07-21T19:08:32.126 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 21 19:08:31 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-7[114007]: 2023-07-21T19:08:31.859+0000 7fceded29700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:54.504220+0000 front 2023-07-21T19:07:54.504248+0000 (oldest deadline 2023-07-21T19:08:19.804390+0000) 2023-07-21T19:08:32.126 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 21 19:08:31 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-7[114007]: 2023-07-21T19:08:31.859+0000 7fceded29700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:03.905635+0000 front 2023-07-21T19:08:03.905615+0000 (oldest deadline 2023-07-21T19:08:29.805636+0000) 2023-07-21T19:08:32.425 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:32 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:32.047+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:46.109399+0000 front 2023-07-21T19:07:46.109431+0000 (oldest deadline 2023-07-21T19:08:11.410380+0000) 2023-07-21T19:08:32.425 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:32 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:32.047+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.711568+0000 front 2023-07-21T19:07:56.711252+0000 (oldest deadline 2023-07-21T19:08:19.011261+0000) 2023-07-21T19:08:32.425 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:32 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:32.047+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:04.912868+0000 front 2023-07-21T19:08:04.912667+0000 (oldest deadline 2023-07-21T19:08:29.012921+0000) 2023-07-21T19:08:32.584 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:32 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:32.184+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:47.629329+0000 front 2023-07-21T19:07:47.629176+0000 (oldest deadline 2023-07-21T19:08:10.529277+0000) 2023-07-21T19:08:32.584 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:32 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:32.184+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:04.631984+0000 front 2023-07-21T19:08:04.631918+0000 (oldest deadline 2023-07-21T19:08:26.332176+0000) 2023-07-21T19:08:32.626 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:32 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:32.182+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.769616+0000 front 2023-07-21T19:07:48.769555+0000 (oldest deadline 2023-07-21T19:08:12.770222+0000) 2023-07-21T19:08:32.626 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:32 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:32.182+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.271039+0000 front 2023-07-21T19:07:56.271067+0000 (oldest deadline 2023-07-21T19:08:23.871770+0000) 2023-07-21T19:08:32.627 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:32 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:32.182+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:03.872288+0000 front 2023-07-21T19:08:03.872364+0000 (oldest deadline 2023-07-21T19:08:26.172487+0000) 2023-07-21T19:08:32.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:32 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:32.575+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:45.542174+0000 front 2023-07-21T19:07:45.542192+0000 (oldest deadline 2023-07-21T19:08:11.442323+0000) 2023-07-21T19:08:32.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:32 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:32.575+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.044189+0000 front 2023-07-21T19:07:56.044156+0000 (oldest deadline 2023-07-21T19:08:18.944205+0000) 2023-07-21T19:08:32.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:32 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:32.575+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:02.345832+0000 front 2023-07-21T19:08:02.345919+0000 (oldest deadline 2023-07-21T19:08:26.445973+0000) 2023-07-21T19:08:32.925 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 21 19:08:32 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-6[120362]: 2023-07-21T19:08:32.593+0000 7fe22b5f5700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.093785+0000 front 2023-07-21T19:07:48.093754+0000 (oldest deadline 2023-07-21T19:08:11.593863+0000) 2023-07-21T19:08:32.925 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 21 19:08:32 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-6[120362]: 2023-07-21T19:08:32.593+0000 7fe22b5f5700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:57.494878+0000 front 2023-07-21T19:07:57.495039+0000 (oldest deadline 2023-07-21T19:08:19.795061+0000) 2023-07-21T19:08:32.926 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 21 19:08:32 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-6[120362]: 2023-07-21T19:08:32.593+0000 7fe22b5f5700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:04.496384+0000 front 2023-07-21T19:08:04.496400+0000 (oldest deadline 2023-07-21T19:08:26.697105+0000) 2023-07-21T19:08:33.206 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 21 19:08:32 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-7[114007]: 2023-07-21T19:08:32.904+0000 7fceded29700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:54.504220+0000 front 2023-07-21T19:07:54.504248+0000 (oldest deadline 2023-07-21T19:08:19.804390+0000) 2023-07-21T19:08:33.207 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 21 19:08:32 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-7[114007]: 2023-07-21T19:08:32.904+0000 7fceded29700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:03.905635+0000 front 2023-07-21T19:08:03.905615+0000 (oldest deadline 2023-07-21T19:08:29.805636+0000) 2023-07-21T19:08:33.425 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:33 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:33.085+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:46.109399+0000 front 2023-07-21T19:07:46.109431+0000 (oldest deadline 2023-07-21T19:08:11.410380+0000) 2023-07-21T19:08:33.425 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:33 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:33.085+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.711568+0000 front 2023-07-21T19:07:56.711252+0000 (oldest deadline 2023-07-21T19:08:19.011261+0000) 2023-07-21T19:08:33.425 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:33 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:33.085+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:04.912868+0000 front 2023-07-21T19:08:04.912667+0000 (oldest deadline 2023-07-21T19:08:29.012921+0000) 2023-07-21T19:08:33.605 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:33 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:33.185+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:47.629329+0000 front 2023-07-21T19:07:47.629176+0000 (oldest deadline 2023-07-21T19:08:10.529277+0000) 2023-07-21T19:08:33.606 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:33 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:33.185+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:04.631984+0000 front 2023-07-21T19:08:04.631918+0000 (oldest deadline 2023-07-21T19:08:26.332176+0000) 2023-07-21T19:08:33.626 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:33 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:33.195+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.769616+0000 front 2023-07-21T19:07:48.769555+0000 (oldest deadline 2023-07-21T19:08:12.770222+0000) 2023-07-21T19:08:33.626 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:33 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:33.195+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.271039+0000 front 2023-07-21T19:07:56.271067+0000 (oldest deadline 2023-07-21T19:08:23.871770+0000) 2023-07-21T19:08:33.626 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:33 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:33.195+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:03.872288+0000 front 2023-07-21T19:08:03.872364+0000 (oldest deadline 2023-07-21T19:08:26.172487+0000) 2023-07-21T19:08:33.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:33 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:33.596+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:45.542174+0000 front 2023-07-21T19:07:45.542192+0000 (oldest deadline 2023-07-21T19:08:11.442323+0000) 2023-07-21T19:08:33.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:33 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:33.596+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.044189+0000 front 2023-07-21T19:07:56.044156+0000 (oldest deadline 2023-07-21T19:08:18.944205+0000) 2023-07-21T19:08:33.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:33 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:33.596+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:02.345832+0000 front 2023-07-21T19:08:02.345919+0000 (oldest deadline 2023-07-21T19:08:26.445973+0000) 2023-07-21T19:08:33.925 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 21 19:08:33 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-6[120362]: 2023-07-21T19:08:33.568+0000 7fe22b5f5700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.093785+0000 front 2023-07-21T19:07:48.093754+0000 (oldest deadline 2023-07-21T19:08:11.593863+0000) 2023-07-21T19:08:33.926 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 21 19:08:33 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-6[120362]: 2023-07-21T19:08:33.568+0000 7fe22b5f5700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:57.494878+0000 front 2023-07-21T19:07:57.495039+0000 (oldest deadline 2023-07-21T19:08:19.795061+0000) 2023-07-21T19:08:33.926 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 21 19:08:33 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-6[120362]: 2023-07-21T19:08:33.568+0000 7fe22b5f5700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:04.496384+0000 front 2023-07-21T19:08:04.496400+0000 (oldest deadline 2023-07-21T19:08:26.697105+0000) 2023-07-21T19:08:34.242 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 21 19:08:33 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-7[114007]: 2023-07-21T19:08:33.901+0000 7fceded29700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:54.504220+0000 front 2023-07-21T19:07:54.504248+0000 (oldest deadline 2023-07-21T19:08:19.804390+0000) 2023-07-21T19:08:34.243 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 21 19:08:33 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-7[114007]: 2023-07-21T19:08:33.901+0000 7fceded29700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:03.905635+0000 front 2023-07-21T19:08:03.905615+0000 (oldest deadline 2023-07-21T19:08:29.805636+0000) 2023-07-21T19:08:34.425 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:34 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:34.123+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:46.109399+0000 front 2023-07-21T19:07:46.109431+0000 (oldest deadline 2023-07-21T19:08:11.410380+0000) 2023-07-21T19:08:34.425 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:34 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:34.123+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.711568+0000 front 2023-07-21T19:07:56.711252+0000 (oldest deadline 2023-07-21T19:08:19.011261+0000) 2023-07-21T19:08:34.426 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:34 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:34.123+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:04.912868+0000 front 2023-07-21T19:08:04.912667+0000 (oldest deadline 2023-07-21T19:08:29.012921+0000) 2023-07-21T19:08:34.580 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:34 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:34.189+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:47.629329+0000 front 2023-07-21T19:07:47.629176+0000 (oldest deadline 2023-07-21T19:08:10.529277+0000) 2023-07-21T19:08:34.580 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:34 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:34.189+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:04.631984+0000 front 2023-07-21T19:08:04.631918+0000 (oldest deadline 2023-07-21T19:08:26.332176+0000) 2023-07-21T19:08:34.626 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:34 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:34.233+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.769616+0000 front 2023-07-21T19:07:48.769555+0000 (oldest deadline 2023-07-21T19:08:12.770222+0000) 2023-07-21T19:08:34.626 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:34 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:34.233+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.271039+0000 front 2023-07-21T19:07:56.271067+0000 (oldest deadline 2023-07-21T19:08:23.871770+0000) 2023-07-21T19:08:34.626 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:34 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:34.233+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:03.872288+0000 front 2023-07-21T19:08:03.872364+0000 (oldest deadline 2023-07-21T19:08:26.172487+0000) 2023-07-21T19:08:34.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:34 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:34.570+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:45.542174+0000 front 2023-07-21T19:07:45.542192+0000 (oldest deadline 2023-07-21T19:08:11.442323+0000) 2023-07-21T19:08:34.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:34 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:34.570+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.044189+0000 front 2023-07-21T19:07:56.044156+0000 (oldest deadline 2023-07-21T19:08:18.944205+0000) 2023-07-21T19:08:34.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:34 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:34.570+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:02.345832+0000 front 2023-07-21T19:08:02.345919+0000 (oldest deadline 2023-07-21T19:08:26.445973+0000) 2023-07-21T19:08:34.925 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 21 19:08:34 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-6[120362]: 2023-07-21T19:08:34.535+0000 7fe22b5f5700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.093785+0000 front 2023-07-21T19:07:48.093754+0000 (oldest deadline 2023-07-21T19:08:11.593863+0000) 2023-07-21T19:08:34.925 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 21 19:08:34 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-6[120362]: 2023-07-21T19:08:34.535+0000 7fe22b5f5700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:57.494878+0000 front 2023-07-21T19:07:57.495039+0000 (oldest deadline 2023-07-21T19:08:19.795061+0000) 2023-07-21T19:08:34.926 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 21 19:08:34 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-6[120362]: 2023-07-21T19:08:34.535+0000 7fe22b5f5700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:04.496384+0000 front 2023-07-21T19:08:04.496400+0000 (oldest deadline 2023-07-21T19:08:26.697105+0000) 2023-07-21T19:08:35.225 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 21 19:08:34 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-7[114007]: 2023-07-21T19:08:34.912+0000 7fceded29700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:54.504220+0000 front 2023-07-21T19:07:54.504248+0000 (oldest deadline 2023-07-21T19:08:19.804390+0000) 2023-07-21T19:08:35.226 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 21 19:08:34 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-7[114007]: 2023-07-21T19:08:34.912+0000 7fceded29700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:03.905635+0000 front 2023-07-21T19:08:03.905615+0000 (oldest deadline 2023-07-21T19:08:29.805636+0000) 2023-07-21T19:08:35.562 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:35 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:35.176+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:47.629329+0000 front 2023-07-21T19:07:47.629176+0000 (oldest deadline 2023-07-21T19:08:10.529277+0000) 2023-07-21T19:08:35.562 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:35 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:35.176+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:04.631984+0000 front 2023-07-21T19:08:04.631918+0000 (oldest deadline 2023-07-21T19:08:26.332176+0000) 2023-07-21T19:08:35.563 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:35 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:35.176+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:11.534192+0000 front 2023-07-21T19:08:11.534232+0000 (oldest deadline 2023-07-21T19:08:35.034206+0000) 2023-07-21T19:08:35.583 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:35 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:35.172+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:46.109399+0000 front 2023-07-21T19:07:46.109431+0000 (oldest deadline 2023-07-21T19:08:11.410380+0000) 2023-07-21T19:08:35.584 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:35 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:35.172+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.711568+0000 front 2023-07-21T19:07:56.711252+0000 (oldest deadline 2023-07-21T19:08:19.011261+0000) 2023-07-21T19:08:35.584 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:35 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:35.172+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:04.912868+0000 front 2023-07-21T19:08:04.912667+0000 (oldest deadline 2023-07-21T19:08:29.012921+0000) 2023-07-21T19:08:35.626 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:35 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:35.216+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.769616+0000 front 2023-07-21T19:07:48.769555+0000 (oldest deadline 2023-07-21T19:08:12.770222+0000) 2023-07-21T19:08:35.626 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:35 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:35.216+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.271039+0000 front 2023-07-21T19:07:56.271067+0000 (oldest deadline 2023-07-21T19:08:23.871770+0000) 2023-07-21T19:08:35.626 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:35 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:35.216+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:03.872288+0000 front 2023-07-21T19:08:03.872364+0000 (oldest deadline 2023-07-21T19:08:26.172487+0000) 2023-07-21T19:08:35.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:35 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:35.552+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:45.542174+0000 front 2023-07-21T19:07:45.542192+0000 (oldest deadline 2023-07-21T19:08:11.442323+0000) 2023-07-21T19:08:35.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:35 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:35.552+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.044189+0000 front 2023-07-21T19:07:56.044156+0000 (oldest deadline 2023-07-21T19:08:18.944205+0000) 2023-07-21T19:08:35.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:35 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:35.552+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:02.345832+0000 front 2023-07-21T19:08:02.345919+0000 (oldest deadline 2023-07-21T19:08:26.445973+0000) 2023-07-21T19:08:35.925 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 21 19:08:35 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-6[120362]: 2023-07-21T19:08:35.574+0000 7fe22b5f5700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.093785+0000 front 2023-07-21T19:07:48.093754+0000 (oldest deadline 2023-07-21T19:08:11.593863+0000) 2023-07-21T19:08:35.925 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 21 19:08:35 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-6[120362]: 2023-07-21T19:08:35.574+0000 7fe22b5f5700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:57.494878+0000 front 2023-07-21T19:07:57.495039+0000 (oldest deadline 2023-07-21T19:08:19.795061+0000) 2023-07-21T19:08:35.925 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 21 19:08:35 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-6[120362]: 2023-07-21T19:08:35.574+0000 7fe22b5f5700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:04.496384+0000 front 2023-07-21T19:08:04.496400+0000 (oldest deadline 2023-07-21T19:08:26.697105+0000) 2023-07-21T19:08:36.255 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 21 19:08:35 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-7[114007]: 2023-07-21T19:08:35.939+0000 7fceded29700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:54.504220+0000 front 2023-07-21T19:07:54.504248+0000 (oldest deadline 2023-07-21T19:08:19.804390+0000) 2023-07-21T19:08:36.256 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 21 19:08:35 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-7[114007]: 2023-07-21T19:08:35.939+0000 7fceded29700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:03.905635+0000 front 2023-07-21T19:08:03.905615+0000 (oldest deadline 2023-07-21T19:08:29.805636+0000) 2023-07-21T19:08:36.256 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 21 19:08:35 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-7[114007]: 2023-07-21T19:08:35.939+0000 7fceded29700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:09.806283+0000 front 2023-07-21T19:08:09.806373+0000 (oldest deadline 2023-07-21T19:08:35.106431+0000) 2023-07-21T19:08:36.597 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:36 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:36.219+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:47.629329+0000 front 2023-07-21T19:07:47.629176+0000 (oldest deadline 2023-07-21T19:08:10.529277+0000) 2023-07-21T19:08:36.597 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:36 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:36.219+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:04.631984+0000 front 2023-07-21T19:08:04.631918+0000 (oldest deadline 2023-07-21T19:08:26.332176+0000) 2023-07-21T19:08:36.597 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:36 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:36.219+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:11.534192+0000 front 2023-07-21T19:08:11.534232+0000 (oldest deadline 2023-07-21T19:08:35.034206+0000) 2023-07-21T19:08:36.626 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:36 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:36.245+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.769616+0000 front 2023-07-21T19:07:48.769555+0000 (oldest deadline 2023-07-21T19:08:12.770222+0000) 2023-07-21T19:08:36.627 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:36 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:36.245+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.271039+0000 front 2023-07-21T19:07:56.271067+0000 (oldest deadline 2023-07-21T19:08:23.871770+0000) 2023-07-21T19:08:36.627 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:36 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:36.245+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:03.872288+0000 front 2023-07-21T19:08:03.872364+0000 (oldest deadline 2023-07-21T19:08:26.172487+0000) 2023-07-21T19:08:36.627 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:36 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:36.245+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:10.873569+0000 front 2023-07-21T19:08:10.873733+0000 (oldest deadline 2023-07-21T19:08:35.573593+0000) 2023-07-21T19:08:36.667 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:36 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:36.194+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:46.109399+0000 front 2023-07-21T19:07:46.109431+0000 (oldest deadline 2023-07-21T19:08:11.410380+0000) 2023-07-21T19:08:36.668 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:36 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:36.194+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.711568+0000 front 2023-07-21T19:07:56.711252+0000 (oldest deadline 2023-07-21T19:08:19.011261+0000) 2023-07-21T19:08:36.668 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:36 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:36.194+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:04.912868+0000 front 2023-07-21T19:08:04.912667+0000 (oldest deadline 2023-07-21T19:08:29.012921+0000) 2023-07-21T19:08:36.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:36 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:36.587+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:45.542174+0000 front 2023-07-21T19:07:45.542192+0000 (oldest deadline 2023-07-21T19:08:11.442323+0000) 2023-07-21T19:08:36.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:36 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:36.587+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.044189+0000 front 2023-07-21T19:07:56.044156+0000 (oldest deadline 2023-07-21T19:08:18.944205+0000) 2023-07-21T19:08:36.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:36 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:36.587+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:02.345832+0000 front 2023-07-21T19:08:02.345919+0000 (oldest deadline 2023-07-21T19:08:26.445973+0000) 2023-07-21T19:08:36.925 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 21 19:08:36 smithi039 bash[163781]: ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-6 2023-07-21T19:08:37.254 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 21 19:08:36 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-7[114007]: 2023-07-21T19:08:36.931+0000 7fceded29700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:54.504220+0000 front 2023-07-21T19:07:54.504248+0000 (oldest deadline 2023-07-21T19:08:19.804390+0000) 2023-07-21T19:08:37.254 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 21 19:08:36 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-7[114007]: 2023-07-21T19:08:36.931+0000 7fceded29700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:03.905635+0000 front 2023-07-21T19:08:03.905615+0000 (oldest deadline 2023-07-21T19:08:29.805636+0000) 2023-07-21T19:08:37.255 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 21 19:08:36 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-7[114007]: 2023-07-21T19:08:36.931+0000 7fceded29700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:09.806283+0000 front 2023-07-21T19:08:09.806373+0000 (oldest deadline 2023-07-21T19:08:35.106431+0000) 2023-07-21T19:08:37.525 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:37 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:37.200+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:46.109399+0000 front 2023-07-21T19:07:46.109431+0000 (oldest deadline 2023-07-21T19:08:11.410380+0000) 2023-07-21T19:08:37.526 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:37 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:37.200+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.711568+0000 front 2023-07-21T19:07:56.711252+0000 (oldest deadline 2023-07-21T19:08:19.011261+0000) 2023-07-21T19:08:37.526 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:37 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:37.200+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:04.912868+0000 front 2023-07-21T19:08:04.912667+0000 (oldest deadline 2023-07-21T19:08:29.012921+0000) 2023-07-21T19:08:37.578 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:37 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:37.229+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:47.629329+0000 front 2023-07-21T19:07:47.629176+0000 (oldest deadline 2023-07-21T19:08:10.529277+0000) 2023-07-21T19:08:37.578 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:37 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:37.229+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:04.631984+0000 front 2023-07-21T19:08:04.631918+0000 (oldest deadline 2023-07-21T19:08:26.332176+0000) 2023-07-21T19:08:37.578 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:37 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:37.229+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:11.534192+0000 front 2023-07-21T19:08:11.534232+0000 (oldest deadline 2023-07-21T19:08:35.034206+0000) 2023-07-21T19:08:37.626 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:37 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:37.244+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.769616+0000 front 2023-07-21T19:07:48.769555+0000 (oldest deadline 2023-07-21T19:08:12.770222+0000) 2023-07-21T19:08:37.626 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:37 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:37.244+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.271039+0000 front 2023-07-21T19:07:56.271067+0000 (oldest deadline 2023-07-21T19:08:23.871770+0000) 2023-07-21T19:08:37.626 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:37 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:37.244+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:03.872288+0000 front 2023-07-21T19:08:03.872364+0000 (oldest deadline 2023-07-21T19:08:26.172487+0000) 2023-07-21T19:08:37.626 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:37 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:37.244+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:10.873569+0000 front 2023-07-21T19:08:10.873733+0000 (oldest deadline 2023-07-21T19:08:35.573593+0000) 2023-07-21T19:08:37.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:37 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:37.568+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:45.542174+0000 front 2023-07-21T19:07:45.542192+0000 (oldest deadline 2023-07-21T19:08:11.442323+0000) 2023-07-21T19:08:37.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:37 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:37.568+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.044189+0000 front 2023-07-21T19:07:56.044156+0000 (oldest deadline 2023-07-21T19:08:18.944205+0000) 2023-07-21T19:08:37.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:37 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:37.568+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:02.345832+0000 front 2023-07-21T19:08:02.345919+0000 (oldest deadline 2023-07-21T19:08:26.445973+0000) 2023-07-21T19:08:38.298 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 21 19:08:37 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-7[114007]: 2023-07-21T19:08:37.958+0000 7fceded29700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:54.504220+0000 front 2023-07-21T19:07:54.504248+0000 (oldest deadline 2023-07-21T19:08:19.804390+0000) 2023-07-21T19:08:38.298 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 21 19:08:37 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-7[114007]: 2023-07-21T19:08:37.958+0000 7fceded29700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:03.905635+0000 front 2023-07-21T19:08:03.905615+0000 (oldest deadline 2023-07-21T19:08:29.805636+0000) 2023-07-21T19:08:38.298 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 21 19:08:37 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-7[114007]: 2023-07-21T19:08:37.958+0000 7fceded29700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:09.806283+0000 front 2023-07-21T19:08:09.806373+0000 (oldest deadline 2023-07-21T19:08:35.106431+0000) 2023-07-21T19:08:38.422 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@osd.6.service' 2023-07-21T19:08:38.464 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T19:08:38.465 INFO:tasks.cephadm.osd.6:Stopped osd.6 2023-07-21T19:08:38.465 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2023-07-21T19:08:38.465 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@osd.7 2023-07-21T19:08:38.539 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:38 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:38.245+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:47.629329+0000 front 2023-07-21T19:07:47.629176+0000 (oldest deadline 2023-07-21T19:08:10.529277+0000) 2023-07-21T19:08:38.539 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:38 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:38.245+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:04.631984+0000 front 2023-07-21T19:08:04.631918+0000 (oldest deadline 2023-07-21T19:08:26.332176+0000) 2023-07-21T19:08:38.540 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:38 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:38.246+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:11.534192+0000 front 2023-07-21T19:08:11.534232+0000 (oldest deadline 2023-07-21T19:08:35.034206+0000) 2023-07-21T19:08:38.626 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 21 19:08:38 smithi155 systemd[1]: Stopping Ceph osd.7 for eaf8e25a-27f4-11ee-9b34-001a4aab830c... 2023-07-21T19:08:38.626 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:38 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:38.288+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.769616+0000 front 2023-07-21T19:07:48.769555+0000 (oldest deadline 2023-07-21T19:08:12.770222+0000) 2023-07-21T19:08:38.627 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:38 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:38.288+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.271039+0000 front 2023-07-21T19:07:56.271067+0000 (oldest deadline 2023-07-21T19:08:23.871770+0000) 2023-07-21T19:08:38.627 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:38 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:38.288+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:03.872288+0000 front 2023-07-21T19:08:03.872364+0000 (oldest deadline 2023-07-21T19:08:26.172487+0000) 2023-07-21T19:08:38.627 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:38 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:38.288+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:10.873569+0000 front 2023-07-21T19:08:10.873733+0000 (oldest deadline 2023-07-21T19:08:35.573593+0000) 2023-07-21T19:08:38.675 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:38 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:38.224+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:46.109399+0000 front 2023-07-21T19:07:46.109431+0000 (oldest deadline 2023-07-21T19:08:11.410380+0000) 2023-07-21T19:08:38.675 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:38 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:38.224+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.711568+0000 front 2023-07-21T19:07:56.711252+0000 (oldest deadline 2023-07-21T19:08:19.011261+0000) 2023-07-21T19:08:38.676 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:38 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:38.224+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:04.912868+0000 front 2023-07-21T19:08:04.912667+0000 (oldest deadline 2023-07-21T19:08:29.012921+0000) 2023-07-21T19:08:38.676 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:38 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:38.224+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:13.114021+0000 front 2023-07-21T19:08:13.114060+0000 (oldest deadline 2023-07-21T19:08:37.214254+0000) 2023-07-21T19:08:38.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:38 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:38.529+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:45.542174+0000 front 2023-07-21T19:07:45.542192+0000 (oldest deadline 2023-07-21T19:08:11.442323+0000) 2023-07-21T19:08:38.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:38 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:38.529+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.044189+0000 front 2023-07-21T19:07:56.044156+0000 (oldest deadline 2023-07-21T19:08:18.944205+0000) 2023-07-21T19:08:38.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:38 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:38.529+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:02.345832+0000 front 2023-07-21T19:08:02.345919+0000 (oldest deadline 2023-07-21T19:08:26.445973+0000) 2023-07-21T19:08:39.126 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 21 19:08:38 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-7[114007]: 2023-07-21T19:08:38.818+0000 7fcee3f36700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T19:08:39.126 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 21 19:08:38 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-7[114007]: 2023-07-21T19:08:38.818+0000 7fcee3f36700 -1 osd.7 127 *** Got signal Terminated *** 2023-07-21T19:08:39.127 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 21 19:08:38 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-7[114007]: 2023-07-21T19:08:38.818+0000 7fcee3f36700 -1 osd.7 127 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-21T19:08:39.127 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 21 19:08:39 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-7[114007]: 2023-07-21T19:08:39.002+0000 7fceded29700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:54.504220+0000 front 2023-07-21T19:07:54.504248+0000 (oldest deadline 2023-07-21T19:08:19.804390+0000) 2023-07-21T19:08:39.127 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 21 19:08:39 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-7[114007]: 2023-07-21T19:08:39.002+0000 7fceded29700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:03.905635+0000 front 2023-07-21T19:08:03.905615+0000 (oldest deadline 2023-07-21T19:08:29.805636+0000) 2023-07-21T19:08:39.128 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 21 19:08:39 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-7[114007]: 2023-07-21T19:08:39.002+0000 7fceded29700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:09.806283+0000 front 2023-07-21T19:08:09.806373+0000 (oldest deadline 2023-07-21T19:08:35.106431+0000) 2023-07-21T19:08:39.560 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:39 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:39.247+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:47.629329+0000 front 2023-07-21T19:07:47.629176+0000 (oldest deadline 2023-07-21T19:08:10.529277+0000) 2023-07-21T19:08:39.560 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:39 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:39.247+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:04.631984+0000 front 2023-07-21T19:08:04.631918+0000 (oldest deadline 2023-07-21T19:08:26.332176+0000) 2023-07-21T19:08:39.561 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:39 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:39.247+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:11.534192+0000 front 2023-07-21T19:08:11.534232+0000 (oldest deadline 2023-07-21T19:08:35.034206+0000) 2023-07-21T19:08:39.626 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:39 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:39.254+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.769616+0000 front 2023-07-21T19:07:48.769555+0000 (oldest deadline 2023-07-21T19:08:12.770222+0000) 2023-07-21T19:08:39.626 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:39 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:39.254+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.271039+0000 front 2023-07-21T19:07:56.271067+0000 (oldest deadline 2023-07-21T19:08:23.871770+0000) 2023-07-21T19:08:39.627 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:39 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:39.254+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:03.872288+0000 front 2023-07-21T19:08:03.872364+0000 (oldest deadline 2023-07-21T19:08:26.172487+0000) 2023-07-21T19:08:39.627 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:39 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:39.254+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:10.873569+0000 front 2023-07-21T19:08:10.873733+0000 (oldest deadline 2023-07-21T19:08:35.573593+0000) 2023-07-21T19:08:39.675 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:39 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:39.218+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:46.109399+0000 front 2023-07-21T19:07:46.109431+0000 (oldest deadline 2023-07-21T19:08:11.410380+0000) 2023-07-21T19:08:39.675 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:39 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:39.218+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.711568+0000 front 2023-07-21T19:07:56.711252+0000 (oldest deadline 2023-07-21T19:08:19.011261+0000) 2023-07-21T19:08:39.675 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:39 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:39.218+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:04.912868+0000 front 2023-07-21T19:08:04.912667+0000 (oldest deadline 2023-07-21T19:08:29.012921+0000) 2023-07-21T19:08:39.675 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:39 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:39.218+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:13.114021+0000 front 2023-07-21T19:08:13.114060+0000 (oldest deadline 2023-07-21T19:08:37.214254+0000) 2023-07-21T19:08:39.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:39 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:39.550+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:45.542174+0000 front 2023-07-21T19:07:45.542192+0000 (oldest deadline 2023-07-21T19:08:11.442323+0000) 2023-07-21T19:08:39.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:39 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:39.550+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.044189+0000 front 2023-07-21T19:07:56.044156+0000 (oldest deadline 2023-07-21T19:08:18.944205+0000) 2023-07-21T19:08:39.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:39 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:39.550+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:02.345832+0000 front 2023-07-21T19:08:02.345919+0000 (oldest deadline 2023-07-21T19:08:26.445973+0000) 2023-07-21T19:08:40.275 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 21 19:08:40 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-7[114007]: 2023-07-21T19:08:40.004+0000 7fceded29700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:54.504220+0000 front 2023-07-21T19:07:54.504248+0000 (oldest deadline 2023-07-21T19:08:19.804390+0000) 2023-07-21T19:08:40.276 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 21 19:08:40 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-7[114007]: 2023-07-21T19:08:40.004+0000 7fceded29700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:03.905635+0000 front 2023-07-21T19:08:03.905615+0000 (oldest deadline 2023-07-21T19:08:29.805636+0000) 2023-07-21T19:08:40.276 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 21 19:08:40 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-7[114007]: 2023-07-21T19:08:40.004+0000 7fceded29700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:09.806283+0000 front 2023-07-21T19:08:09.806373+0000 (oldest deadline 2023-07-21T19:08:35.106431+0000) 2023-07-21T19:08:40.599 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:40 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:40.262+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:47.629329+0000 front 2023-07-21T19:07:47.629176+0000 (oldest deadline 2023-07-21T19:08:10.529277+0000) 2023-07-21T19:08:40.599 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:40 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:40.262+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:04.631984+0000 front 2023-07-21T19:08:04.631918+0000 (oldest deadline 2023-07-21T19:08:26.332176+0000) 2023-07-21T19:08:40.599 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:40 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:40.262+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:11.534192+0000 front 2023-07-21T19:08:11.534232+0000 (oldest deadline 2023-07-21T19:08:35.034206+0000) 2023-07-21T19:08:40.626 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:40 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:40.265+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.769616+0000 front 2023-07-21T19:07:48.769555+0000 (oldest deadline 2023-07-21T19:08:12.770222+0000) 2023-07-21T19:08:40.626 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:40 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:40.265+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.271039+0000 front 2023-07-21T19:07:56.271067+0000 (oldest deadline 2023-07-21T19:08:23.871770+0000) 2023-07-21T19:08:40.627 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:40 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:40.265+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:03.872288+0000 front 2023-07-21T19:08:03.872364+0000 (oldest deadline 2023-07-21T19:08:26.172487+0000) 2023-07-21T19:08:40.627 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:40 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:40.265+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:10.873569+0000 front 2023-07-21T19:08:10.873733+0000 (oldest deadline 2023-07-21T19:08:35.573593+0000) 2023-07-21T19:08:40.675 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:40 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:40.221+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:46.109399+0000 front 2023-07-21T19:07:46.109431+0000 (oldest deadline 2023-07-21T19:08:11.410380+0000) 2023-07-21T19:08:40.676 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:40 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:40.221+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.711568+0000 front 2023-07-21T19:07:56.711252+0000 (oldest deadline 2023-07-21T19:08:19.011261+0000) 2023-07-21T19:08:40.676 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:40 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:40.221+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:04.912868+0000 front 2023-07-21T19:08:04.912667+0000 (oldest deadline 2023-07-21T19:08:29.012921+0000) 2023-07-21T19:08:40.676 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:40 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:40.221+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:13.114021+0000 front 2023-07-21T19:08:13.114060+0000 (oldest deadline 2023-07-21T19:08:37.214254+0000) 2023-07-21T19:08:40.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:40 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:40.589+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:45.542174+0000 front 2023-07-21T19:07:45.542192+0000 (oldest deadline 2023-07-21T19:08:11.442323+0000) 2023-07-21T19:08:40.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:40 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:40.589+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.044189+0000 front 2023-07-21T19:07:56.044156+0000 (oldest deadline 2023-07-21T19:08:18.944205+0000) 2023-07-21T19:08:40.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:40 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:40.589+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:02.345832+0000 front 2023-07-21T19:08:02.345919+0000 (oldest deadline 2023-07-21T19:08:26.445973+0000) 2023-07-21T19:08:41.307 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 21 19:08:41 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-7[114007]: 2023-07-21T19:08:41.007+0000 7fceded29700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:54.504220+0000 front 2023-07-21T19:07:54.504248+0000 (oldest deadline 2023-07-21T19:08:19.804390+0000) 2023-07-21T19:08:41.307 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 21 19:08:41 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-7[114007]: 2023-07-21T19:08:41.007+0000 7fceded29700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:03.905635+0000 front 2023-07-21T19:08:03.905615+0000 (oldest deadline 2023-07-21T19:08:29.805636+0000) 2023-07-21T19:08:41.307 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 21 19:08:41 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-7[114007]: 2023-07-21T19:08:41.007+0000 7fceded29700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:09.806283+0000 front 2023-07-21T19:08:09.806373+0000 (oldest deadline 2023-07-21T19:08:35.106431+0000) 2023-07-21T19:08:41.590 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:41 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:41.243+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:47.629329+0000 front 2023-07-21T19:07:47.629176+0000 (oldest deadline 2023-07-21T19:08:10.529277+0000) 2023-07-21T19:08:41.590 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:41 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:41.243+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:04.631984+0000 front 2023-07-21T19:08:04.631918+0000 (oldest deadline 2023-07-21T19:08:26.332176+0000) 2023-07-21T19:08:41.590 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:41 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:41.243+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:11.534192+0000 front 2023-07-21T19:08:11.534232+0000 (oldest deadline 2023-07-21T19:08:35.034206+0000) 2023-07-21T19:08:41.626 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:41 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:41.297+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.769616+0000 front 2023-07-21T19:07:48.769555+0000 (oldest deadline 2023-07-21T19:08:12.770222+0000) 2023-07-21T19:08:41.626 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:41 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:41.297+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.271039+0000 front 2023-07-21T19:07:56.271067+0000 (oldest deadline 2023-07-21T19:08:23.871770+0000) 2023-07-21T19:08:41.626 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:41 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:41.297+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:03.872288+0000 front 2023-07-21T19:08:03.872364+0000 (oldest deadline 2023-07-21T19:08:26.172487+0000) 2023-07-21T19:08:41.626 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:41 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:41.297+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:10.873569+0000 front 2023-07-21T19:08:10.873733+0000 (oldest deadline 2023-07-21T19:08:35.573593+0000) 2023-07-21T19:08:41.675 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:41 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:41.213+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:46.109399+0000 front 2023-07-21T19:07:46.109431+0000 (oldest deadline 2023-07-21T19:08:11.410380+0000) 2023-07-21T19:08:41.675 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:41 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:41.213+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.711568+0000 front 2023-07-21T19:07:56.711252+0000 (oldest deadline 2023-07-21T19:08:19.011261+0000) 2023-07-21T19:08:41.676 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:41 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:41.213+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:04.912868+0000 front 2023-07-21T19:08:04.912667+0000 (oldest deadline 2023-07-21T19:08:29.012921+0000) 2023-07-21T19:08:41.676 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:41 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:41.213+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:13.114021+0000 front 2023-07-21T19:08:13.114060+0000 (oldest deadline 2023-07-21T19:08:37.214254+0000) 2023-07-21T19:08:41.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:41 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:41.580+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:45.542174+0000 front 2023-07-21T19:07:45.542192+0000 (oldest deadline 2023-07-21T19:08:11.442323+0000) 2023-07-21T19:08:41.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:41 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:41.580+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.044189+0000 front 2023-07-21T19:07:56.044156+0000 (oldest deadline 2023-07-21T19:08:18.944205+0000) 2023-07-21T19:08:41.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:41 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:41.580+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:02.345832+0000 front 2023-07-21T19:08:02.345919+0000 (oldest deadline 2023-07-21T19:08:26.445973+0000) 2023-07-21T19:08:42.328 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 21 19:08:41 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-7[114007]: 2023-07-21T19:08:41.989+0000 7fceded29700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:54.504220+0000 front 2023-07-21T19:07:54.504248+0000 (oldest deadline 2023-07-21T19:08:19.804390+0000) 2023-07-21T19:08:42.328 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 21 19:08:41 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-7[114007]: 2023-07-21T19:08:41.989+0000 7fceded29700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:03.905635+0000 front 2023-07-21T19:08:03.905615+0000 (oldest deadline 2023-07-21T19:08:29.805636+0000) 2023-07-21T19:08:42.329 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 21 19:08:41 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-7[114007]: 2023-07-21T19:08:41.989+0000 7fceded29700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:09.806283+0000 front 2023-07-21T19:08:09.806373+0000 (oldest deadline 2023-07-21T19:08:35.106431+0000) 2023-07-21T19:08:42.607 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:42 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:42.260+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:47.629329+0000 front 2023-07-21T19:07:47.629176+0000 (oldest deadline 2023-07-21T19:08:10.529277+0000) 2023-07-21T19:08:42.607 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:42 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:42.260+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:04.631984+0000 front 2023-07-21T19:08:04.631918+0000 (oldest deadline 2023-07-21T19:08:26.332176+0000) 2023-07-21T19:08:42.608 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:42 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:42.260+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:11.534192+0000 front 2023-07-21T19:08:11.534232+0000 (oldest deadline 2023-07-21T19:08:35.034206+0000) 2023-07-21T19:08:42.626 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:42 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:42.318+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.769616+0000 front 2023-07-21T19:07:48.769555+0000 (oldest deadline 2023-07-21T19:08:12.770222+0000) 2023-07-21T19:08:42.626 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:42 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:42.318+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.271039+0000 front 2023-07-21T19:07:56.271067+0000 (oldest deadline 2023-07-21T19:08:23.871770+0000) 2023-07-21T19:08:42.626 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:42 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:42.318+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:03.872288+0000 front 2023-07-21T19:08:03.872364+0000 (oldest deadline 2023-07-21T19:08:26.172487+0000) 2023-07-21T19:08:42.627 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:42 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:42.318+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:10.873569+0000 front 2023-07-21T19:08:10.873733+0000 (oldest deadline 2023-07-21T19:08:35.573593+0000) 2023-07-21T19:08:42.675 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:42 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:42.229+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:46.109399+0000 front 2023-07-21T19:07:46.109431+0000 (oldest deadline 2023-07-21T19:08:11.410380+0000) 2023-07-21T19:08:42.675 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:42 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:42.229+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.711568+0000 front 2023-07-21T19:07:56.711252+0000 (oldest deadline 2023-07-21T19:08:19.011261+0000) 2023-07-21T19:08:42.676 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:42 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:42.229+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:04.912868+0000 front 2023-07-21T19:08:04.912667+0000 (oldest deadline 2023-07-21T19:08:29.012921+0000) 2023-07-21T19:08:42.676 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:42 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:42.229+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:13.114021+0000 front 2023-07-21T19:08:13.114060+0000 (oldest deadline 2023-07-21T19:08:37.214254+0000) 2023-07-21T19:08:42.676 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:42 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:42.229+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:17.214857+0000 front 2023-07-21T19:08:17.214903+0000 (oldest deadline 2023-07-21T19:08:41.915044+0000) 2023-07-21T19:08:43.107 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:42 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:42.618+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:45.542174+0000 front 2023-07-21T19:07:45.542192+0000 (oldest deadline 2023-07-21T19:08:11.442323+0000) 2023-07-21T19:08:43.108 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:42 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:42.618+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.044189+0000 front 2023-07-21T19:07:56.044156+0000 (oldest deadline 2023-07-21T19:08:18.944205+0000) 2023-07-21T19:08:43.108 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:42 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:42.618+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:02.345832+0000 front 2023-07-21T19:08:02.345919+0000 (oldest deadline 2023-07-21T19:08:26.445973+0000) 2023-07-21T19:08:43.360 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 21 19:08:42 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-7[114007]: 2023-07-21T19:08:42.954+0000 7fceded29700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:54.504220+0000 front 2023-07-21T19:07:54.504248+0000 (oldest deadline 2023-07-21T19:08:19.804390+0000) 2023-07-21T19:08:43.361 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 21 19:08:42 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-7[114007]: 2023-07-21T19:08:42.954+0000 7fceded29700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:03.905635+0000 front 2023-07-21T19:08:03.905615+0000 (oldest deadline 2023-07-21T19:08:29.805636+0000) 2023-07-21T19:08:43.361 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 21 19:08:42 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-7[114007]: 2023-07-21T19:08:42.954+0000 7fceded29700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:09.806283+0000 front 2023-07-21T19:08:09.806373+0000 (oldest deadline 2023-07-21T19:08:35.106431+0000) 2023-07-21T19:08:43.361 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 21 19:08:42 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-7[114007]: 2023-07-21T19:08:42.954+0000 7fceded29700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:16.807689+0000 front 2023-07-21T19:08:16.807666+0000 (oldest deadline 2023-07-21T19:08:42.707811+0000) 2023-07-21T19:08:43.590 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:43 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:43.273+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:47.629329+0000 front 2023-07-21T19:07:47.629176+0000 (oldest deadline 2023-07-21T19:08:10.529277+0000) 2023-07-21T19:08:43.590 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:43 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:43.273+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:04.631984+0000 front 2023-07-21T19:08:04.631918+0000 (oldest deadline 2023-07-21T19:08:26.332176+0000) 2023-07-21T19:08:43.591 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:43 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:43.273+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:11.534192+0000 front 2023-07-21T19:08:11.534232+0000 (oldest deadline 2023-07-21T19:08:35.034206+0000) 2023-07-21T19:08:43.591 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:43 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:43.273+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:18.535636+0000 front 2023-07-21T19:08:18.535608+0000 (oldest deadline 2023-07-21T19:08:42.635555+0000) 2023-07-21T19:08:43.626 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:43 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:43.351+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.769616+0000 front 2023-07-21T19:07:48.769555+0000 (oldest deadline 2023-07-21T19:08:12.770222+0000) 2023-07-21T19:08:43.626 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:43 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:43.351+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.271039+0000 front 2023-07-21T19:07:56.271067+0000 (oldest deadline 2023-07-21T19:08:23.871770+0000) 2023-07-21T19:08:43.627 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:43 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:43.351+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:03.872288+0000 front 2023-07-21T19:08:03.872364+0000 (oldest deadline 2023-07-21T19:08:26.172487+0000) 2023-07-21T19:08:43.627 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:43 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:43.351+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:10.873569+0000 front 2023-07-21T19:08:10.873733+0000 (oldest deadline 2023-07-21T19:08:35.573593+0000) 2023-07-21T19:08:43.675 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:43 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:43.227+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:46.109399+0000 front 2023-07-21T19:07:46.109431+0000 (oldest deadline 2023-07-21T19:08:11.410380+0000) 2023-07-21T19:08:43.676 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:43 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:43.227+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.711568+0000 front 2023-07-21T19:07:56.711252+0000 (oldest deadline 2023-07-21T19:08:19.011261+0000) 2023-07-21T19:08:43.676 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:43 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:43.227+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:04.912868+0000 front 2023-07-21T19:08:04.912667+0000 (oldest deadline 2023-07-21T19:08:29.012921+0000) 2023-07-21T19:08:43.676 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:43 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:43.227+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:13.114021+0000 front 2023-07-21T19:08:13.114060+0000 (oldest deadline 2023-07-21T19:08:37.214254+0000) 2023-07-21T19:08:43.677 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:43 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:43.227+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:17.214857+0000 front 2023-07-21T19:08:17.214903+0000 (oldest deadline 2023-07-21T19:08:41.915044+0000) 2023-07-21T19:08:43.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:43 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:43.579+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:45.542174+0000 front 2023-07-21T19:07:45.542192+0000 (oldest deadline 2023-07-21T19:08:11.442323+0000) 2023-07-21T19:08:43.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:43 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:43.579+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.044189+0000 front 2023-07-21T19:07:56.044156+0000 (oldest deadline 2023-07-21T19:08:18.944205+0000) 2023-07-21T19:08:43.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:43 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:43.579+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:02.345832+0000 front 2023-07-21T19:08:02.345919+0000 (oldest deadline 2023-07-21T19:08:26.445973+0000) 2023-07-21T19:08:43.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:43 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:43.579+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:17.547994+0000 front 2023-07-21T19:08:17.548134+0000 (oldest deadline 2023-07-21T19:08:43.448059+0000) 2023-07-21T19:08:44.604 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 21 19:08:44 smithi155 bash[133373]: ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-7 2023-07-21T19:08:44.604 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:44 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:44.301+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.769616+0000 front 2023-07-21T19:07:48.769555+0000 (oldest deadline 2023-07-21T19:08:12.770222+0000) 2023-07-21T19:08:44.604 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:44 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:44.301+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.271039+0000 front 2023-07-21T19:07:56.271067+0000 (oldest deadline 2023-07-21T19:08:23.871770+0000) 2023-07-21T19:08:44.604 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:44 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:44.301+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:03.872288+0000 front 2023-07-21T19:08:03.872364+0000 (oldest deadline 2023-07-21T19:08:26.172487+0000) 2023-07-21T19:08:44.605 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:44 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:44.301+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:10.873569+0000 front 2023-07-21T19:08:10.873733+0000 (oldest deadline 2023-07-21T19:08:35.573593+0000) 2023-07-21T19:08:44.607 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:44 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:44.228+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:47.629329+0000 front 2023-07-21T19:07:47.629176+0000 (oldest deadline 2023-07-21T19:08:10.529277+0000) 2023-07-21T19:08:44.607 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:44 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:44.228+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:04.631984+0000 front 2023-07-21T19:08:04.631918+0000 (oldest deadline 2023-07-21T19:08:26.332176+0000) 2023-07-21T19:08:44.607 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:44 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:44.228+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:11.534192+0000 front 2023-07-21T19:08:11.534232+0000 (oldest deadline 2023-07-21T19:08:35.034206+0000) 2023-07-21T19:08:44.607 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:44 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:44.228+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:18.535636+0000 front 2023-07-21T19:08:18.535608+0000 (oldest deadline 2023-07-21T19:08:42.635555+0000) 2023-07-21T19:08:44.675 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:44 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:44.208+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:46.109399+0000 front 2023-07-21T19:07:46.109431+0000 (oldest deadline 2023-07-21T19:08:11.410380+0000) 2023-07-21T19:08:44.675 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:44 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:44.208+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.711568+0000 front 2023-07-21T19:07:56.711252+0000 (oldest deadline 2023-07-21T19:08:19.011261+0000) 2023-07-21T19:08:44.675 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:44 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:44.208+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:04.912868+0000 front 2023-07-21T19:08:04.912667+0000 (oldest deadline 2023-07-21T19:08:29.012921+0000) 2023-07-21T19:08:44.675 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:44 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:44.208+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:13.114021+0000 front 2023-07-21T19:08:13.114060+0000 (oldest deadline 2023-07-21T19:08:37.214254+0000) 2023-07-21T19:08:44.676 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:44 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:44.208+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:17.214857+0000 front 2023-07-21T19:08:17.214903+0000 (oldest deadline 2023-07-21T19:08:41.915044+0000) 2023-07-21T19:08:45.107 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:44 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:44.602+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:45.542174+0000 front 2023-07-21T19:07:45.542192+0000 (oldest deadline 2023-07-21T19:08:11.442323+0000) 2023-07-21T19:08:45.108 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:44 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:44.602+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.044189+0000 front 2023-07-21T19:07:56.044156+0000 (oldest deadline 2023-07-21T19:08:18.944205+0000) 2023-07-21T19:08:45.108 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:44 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:44.603+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:02.345832+0000 front 2023-07-21T19:08:02.345919+0000 (oldest deadline 2023-07-21T19:08:26.445973+0000) 2023-07-21T19:08:45.108 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:44 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:44.603+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:17.547994+0000 front 2023-07-21T19:08:17.548134+0000 (oldest deadline 2023-07-21T19:08:43.448059+0000) 2023-07-21T19:08:45.537 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:45 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:45.287+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.769616+0000 front 2023-07-21T19:07:48.769555+0000 (oldest deadline 2023-07-21T19:08:12.770222+0000) 2023-07-21T19:08:45.538 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:45 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:45.287+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.271039+0000 front 2023-07-21T19:07:56.271067+0000 (oldest deadline 2023-07-21T19:08:23.871770+0000) 2023-07-21T19:08:45.538 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:45 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:45.287+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:03.872288+0000 front 2023-07-21T19:08:03.872364+0000 (oldest deadline 2023-07-21T19:08:26.172487+0000) 2023-07-21T19:08:45.538 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:45 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:45.287+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:10.873569+0000 front 2023-07-21T19:08:10.873733+0000 (oldest deadline 2023-07-21T19:08:35.573593+0000) 2023-07-21T19:08:45.538 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:45 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:45.287+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:19.074521+0000 front 2023-07-21T19:08:19.074500+0000 (oldest deadline 2023-07-21T19:08:44.374675+0000) 2023-07-21T19:08:45.588 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:45 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:45.195+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:47.629329+0000 front 2023-07-21T19:07:47.629176+0000 (oldest deadline 2023-07-21T19:08:10.529277+0000) 2023-07-21T19:08:45.588 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:45 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:45.195+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:04.631984+0000 front 2023-07-21T19:08:04.631918+0000 (oldest deadline 2023-07-21T19:08:26.332176+0000) 2023-07-21T19:08:45.589 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:45 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:45.195+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:11.534192+0000 front 2023-07-21T19:08:11.534232+0000 (oldest deadline 2023-07-21T19:08:35.034206+0000) 2023-07-21T19:08:45.589 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:45 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:45.195+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:18.535636+0000 front 2023-07-21T19:08:18.535608+0000 (oldest deadline 2023-07-21T19:08:42.635555+0000) 2023-07-21T19:08:45.675 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:45 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:45.167+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:46.109399+0000 front 2023-07-21T19:07:46.109431+0000 (oldest deadline 2023-07-21T19:08:11.410380+0000) 2023-07-21T19:08:45.675 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:45 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:45.167+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.711568+0000 front 2023-07-21T19:07:56.711252+0000 (oldest deadline 2023-07-21T19:08:19.011261+0000) 2023-07-21T19:08:45.675 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:45 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:45.167+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:04.912868+0000 front 2023-07-21T19:08:04.912667+0000 (oldest deadline 2023-07-21T19:08:29.012921+0000) 2023-07-21T19:08:45.676 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:45 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:45.167+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:13.114021+0000 front 2023-07-21T19:08:13.114060+0000 (oldest deadline 2023-07-21T19:08:37.214254+0000) 2023-07-21T19:08:45.676 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:45 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:45.167+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:17.214857+0000 front 2023-07-21T19:08:17.214903+0000 (oldest deadline 2023-07-21T19:08:41.915044+0000) 2023-07-21T19:08:45.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:45 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:45.579+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:45.542174+0000 front 2023-07-21T19:07:45.542192+0000 (oldest deadline 2023-07-21T19:08:11.442323+0000) 2023-07-21T19:08:45.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:45 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:45.579+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.044189+0000 front 2023-07-21T19:07:56.044156+0000 (oldest deadline 2023-07-21T19:08:18.944205+0000) 2023-07-21T19:08:45.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:45 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:45.579+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:02.345832+0000 front 2023-07-21T19:08:02.345919+0000 (oldest deadline 2023-07-21T19:08:26.445973+0000) 2023-07-21T19:08:45.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:45 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:45.579+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:17.547994+0000 front 2023-07-21T19:08:17.548134+0000 (oldest deadline 2023-07-21T19:08:43.448059+0000) 2023-07-21T19:08:45.864 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@osd.7.service' 2023-07-21T19:08:45.905 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T19:08:45.905 INFO:tasks.cephadm.osd.7:Stopped osd.7 2023-07-21T19:08:45.906 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2023-07-21T19:08:45.906 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@osd.8 2023-07-21T19:08:46.240 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:45 smithi193 systemd[1]: Stopping Ceph osd.8 for eaf8e25a-27f4-11ee-9b34-001a4aab830c... 2023-07-21T19:08:46.552 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:46 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:46.231+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:47.629329+0000 front 2023-07-21T19:07:47.629176+0000 (oldest deadline 2023-07-21T19:08:10.529277+0000) 2023-07-21T19:08:46.552 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:46 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:46.231+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:04.631984+0000 front 2023-07-21T19:08:04.631918+0000 (oldest deadline 2023-07-21T19:08:26.332176+0000) 2023-07-21T19:08:46.552 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:46 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:46.231+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:11.534192+0000 front 2023-07-21T19:08:11.534232+0000 (oldest deadline 2023-07-21T19:08:35.034206+0000) 2023-07-21T19:08:46.552 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:46 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:46.231+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:18.535636+0000 front 2023-07-21T19:08:18.535608+0000 (oldest deadline 2023-07-21T19:08:42.635555+0000) 2023-07-21T19:08:46.552 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:46 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:46.254+0000 7feabb4dc700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T19:08:46.553 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:46 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:46.254+0000 7feabb4dc700 -1 osd.8 127 *** Got signal Terminated *** 2023-07-21T19:08:46.553 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:46 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:46.254+0000 7feabb4dc700 -1 osd.8 127 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-21T19:08:46.626 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:46 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:46.316+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.769616+0000 front 2023-07-21T19:07:48.769555+0000 (oldest deadline 2023-07-21T19:08:12.770222+0000) 2023-07-21T19:08:46.626 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:46 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:46.316+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.271039+0000 front 2023-07-21T19:07:56.271067+0000 (oldest deadline 2023-07-21T19:08:23.871770+0000) 2023-07-21T19:08:46.626 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:46 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:46.316+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:03.872288+0000 front 2023-07-21T19:08:03.872364+0000 (oldest deadline 2023-07-21T19:08:26.172487+0000) 2023-07-21T19:08:46.627 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:46 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:46.316+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:10.873569+0000 front 2023-07-21T19:08:10.873733+0000 (oldest deadline 2023-07-21T19:08:35.573593+0000) 2023-07-21T19:08:46.627 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:46 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:46.316+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:19.074521+0000 front 2023-07-21T19:08:19.074500+0000 (oldest deadline 2023-07-21T19:08:44.374675+0000) 2023-07-21T19:08:46.675 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:46 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:46.179+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:46.109399+0000 front 2023-07-21T19:07:46.109431+0000 (oldest deadline 2023-07-21T19:08:11.410380+0000) 2023-07-21T19:08:46.676 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:46 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:46.179+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.711568+0000 front 2023-07-21T19:07:56.711252+0000 (oldest deadline 2023-07-21T19:08:19.011261+0000) 2023-07-21T19:08:46.676 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:46 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:46.179+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:04.912868+0000 front 2023-07-21T19:08:04.912667+0000 (oldest deadline 2023-07-21T19:08:29.012921+0000) 2023-07-21T19:08:46.676 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:46 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:46.179+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:13.114021+0000 front 2023-07-21T19:08:13.114060+0000 (oldest deadline 2023-07-21T19:08:37.214254+0000) 2023-07-21T19:08:46.677 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:46 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:46.179+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:17.214857+0000 front 2023-07-21T19:08:17.214903+0000 (oldest deadline 2023-07-21T19:08:41.915044+0000) 2023-07-21T19:08:46.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:46 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:46.542+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:45.542174+0000 front 2023-07-21T19:07:45.542192+0000 (oldest deadline 2023-07-21T19:08:11.442323+0000) 2023-07-21T19:08:46.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:46 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:46.542+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.044189+0000 front 2023-07-21T19:07:56.044156+0000 (oldest deadline 2023-07-21T19:08:18.944205+0000) 2023-07-21T19:08:46.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:46 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:46.542+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:02.345832+0000 front 2023-07-21T19:08:02.345919+0000 (oldest deadline 2023-07-21T19:08:26.445973+0000) 2023-07-21T19:08:46.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:46 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:46.542+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:17.547994+0000 front 2023-07-21T19:08:17.548134+0000 (oldest deadline 2023-07-21T19:08:43.448059+0000) 2023-07-21T19:08:47.511 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:47 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:47.189+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:47.629329+0000 front 2023-07-21T19:07:47.629176+0000 (oldest deadline 2023-07-21T19:08:10.529277+0000) 2023-07-21T19:08:47.511 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:47 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:47.189+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:04.631984+0000 front 2023-07-21T19:08:04.631918+0000 (oldest deadline 2023-07-21T19:08:26.332176+0000) 2023-07-21T19:08:47.511 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:47 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:47.189+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:11.534192+0000 front 2023-07-21T19:08:11.534232+0000 (oldest deadline 2023-07-21T19:08:35.034206+0000) 2023-07-21T19:08:47.512 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:47 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:47.189+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:18.535636+0000 front 2023-07-21T19:08:18.535608+0000 (oldest deadline 2023-07-21T19:08:42.635555+0000) 2023-07-21T19:08:47.626 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:47 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:47.316+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.769616+0000 front 2023-07-21T19:07:48.769555+0000 (oldest deadline 2023-07-21T19:08:12.770222+0000) 2023-07-21T19:08:47.626 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:47 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:47.316+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.271039+0000 front 2023-07-21T19:07:56.271067+0000 (oldest deadline 2023-07-21T19:08:23.871770+0000) 2023-07-21T19:08:47.626 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:47 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:47.316+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:03.872288+0000 front 2023-07-21T19:08:03.872364+0000 (oldest deadline 2023-07-21T19:08:26.172487+0000) 2023-07-21T19:08:47.627 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:47 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:47.316+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:10.873569+0000 front 2023-07-21T19:08:10.873733+0000 (oldest deadline 2023-07-21T19:08:35.573593+0000) 2023-07-21T19:08:47.627 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:47 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:47.316+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:19.074521+0000 front 2023-07-21T19:08:19.074500+0000 (oldest deadline 2023-07-21T19:08:44.374675+0000) 2023-07-21T19:08:47.675 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:47 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:47.176+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:46.109399+0000 front 2023-07-21T19:07:46.109431+0000 (oldest deadline 2023-07-21T19:08:11.410380+0000) 2023-07-21T19:08:47.676 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:47 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:47.177+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.711568+0000 front 2023-07-21T19:07:56.711252+0000 (oldest deadline 2023-07-21T19:08:19.011261+0000) 2023-07-21T19:08:47.676 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:47 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:47.177+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:04.912868+0000 front 2023-07-21T19:08:04.912667+0000 (oldest deadline 2023-07-21T19:08:29.012921+0000) 2023-07-21T19:08:47.676 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:47 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:47.177+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:13.114021+0000 front 2023-07-21T19:08:13.114060+0000 (oldest deadline 2023-07-21T19:08:37.214254+0000) 2023-07-21T19:08:47.676 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:47 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:47.177+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:17.214857+0000 front 2023-07-21T19:08:17.214903+0000 (oldest deadline 2023-07-21T19:08:41.915044+0000) 2023-07-21T19:08:47.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:47 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:47.501+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:45.542174+0000 front 2023-07-21T19:07:45.542192+0000 (oldest deadline 2023-07-21T19:08:11.442323+0000) 2023-07-21T19:08:47.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:47 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:47.501+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.044189+0000 front 2023-07-21T19:07:56.044156+0000 (oldest deadline 2023-07-21T19:08:18.944205+0000) 2023-07-21T19:08:47.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:47 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:47.501+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:02.345832+0000 front 2023-07-21T19:08:02.345919+0000 (oldest deadline 2023-07-21T19:08:26.445973+0000) 2023-07-21T19:08:47.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:47 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:47.501+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:17.547994+0000 front 2023-07-21T19:08:17.548134+0000 (oldest deadline 2023-07-21T19:08:43.448059+0000) 2023-07-21T19:08:48.425 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:48 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:48.159+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:46.109399+0000 front 2023-07-21T19:07:46.109431+0000 (oldest deadline 2023-07-21T19:08:11.410380+0000) 2023-07-21T19:08:48.425 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:48 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:48.159+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.711568+0000 front 2023-07-21T19:07:56.711252+0000 (oldest deadline 2023-07-21T19:08:19.011261+0000) 2023-07-21T19:08:48.425 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:48 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:48.159+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:04.912868+0000 front 2023-07-21T19:08:04.912667+0000 (oldest deadline 2023-07-21T19:08:29.012921+0000) 2023-07-21T19:08:48.426 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:48 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:48.159+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:13.114021+0000 front 2023-07-21T19:08:13.114060+0000 (oldest deadline 2023-07-21T19:08:37.214254+0000) 2023-07-21T19:08:48.426 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:48 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:48.159+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:17.214857+0000 front 2023-07-21T19:08:17.214903+0000 (oldest deadline 2023-07-21T19:08:41.915044+0000) 2023-07-21T19:08:48.464 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:48 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:48.171+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:47.629329+0000 front 2023-07-21T19:07:47.629176+0000 (oldest deadline 2023-07-21T19:08:10.529277+0000) 2023-07-21T19:08:48.464 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:48 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:48.171+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:04.631984+0000 front 2023-07-21T19:08:04.631918+0000 (oldest deadline 2023-07-21T19:08:26.332176+0000) 2023-07-21T19:08:48.464 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:48 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:48.171+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:11.534192+0000 front 2023-07-21T19:08:11.534232+0000 (oldest deadline 2023-07-21T19:08:35.034206+0000) 2023-07-21T19:08:48.464 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:48 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:48.171+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:18.535636+0000 front 2023-07-21T19:08:18.535608+0000 (oldest deadline 2023-07-21T19:08:42.635555+0000) 2023-07-21T19:08:48.626 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:48 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:48.331+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.769616+0000 front 2023-07-21T19:07:48.769555+0000 (oldest deadline 2023-07-21T19:08:12.770222+0000) 2023-07-21T19:08:48.626 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:48 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:48.331+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.271039+0000 front 2023-07-21T19:07:56.271067+0000 (oldest deadline 2023-07-21T19:08:23.871770+0000) 2023-07-21T19:08:48.626 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:48 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:48.331+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:03.872288+0000 front 2023-07-21T19:08:03.872364+0000 (oldest deadline 2023-07-21T19:08:26.172487+0000) 2023-07-21T19:08:48.626 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:48 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:48.331+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:10.873569+0000 front 2023-07-21T19:08:10.873733+0000 (oldest deadline 2023-07-21T19:08:35.573593+0000) 2023-07-21T19:08:48.626 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:48 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:48.331+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:19.074521+0000 front 2023-07-21T19:08:19.074500+0000 (oldest deadline 2023-07-21T19:08:44.374675+0000) 2023-07-21T19:08:48.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:48 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:48.454+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:45.542174+0000 front 2023-07-21T19:07:45.542192+0000 (oldest deadline 2023-07-21T19:08:11.442323+0000) 2023-07-21T19:08:48.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:48 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:48.454+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.044189+0000 front 2023-07-21T19:07:56.044156+0000 (oldest deadline 2023-07-21T19:08:18.944205+0000) 2023-07-21T19:08:48.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:48 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:48.454+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:02.345832+0000 front 2023-07-21T19:08:02.345919+0000 (oldest deadline 2023-07-21T19:08:26.445973+0000) 2023-07-21T19:08:48.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:48 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:48.454+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:17.547994+0000 front 2023-07-21T19:08:17.548134+0000 (oldest deadline 2023-07-21T19:08:43.448059+0000) 2023-07-21T19:08:49.425 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:49 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:49.139+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:46.109399+0000 front 2023-07-21T19:07:46.109431+0000 (oldest deadline 2023-07-21T19:08:11.410380+0000) 2023-07-21T19:08:49.425 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:49 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:49.139+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.711568+0000 front 2023-07-21T19:07:56.711252+0000 (oldest deadline 2023-07-21T19:08:19.011261+0000) 2023-07-21T19:08:49.425 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:49 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:49.139+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:04.912868+0000 front 2023-07-21T19:08:04.912667+0000 (oldest deadline 2023-07-21T19:08:29.012921+0000) 2023-07-21T19:08:49.425 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:49 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:49.139+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:13.114021+0000 front 2023-07-21T19:08:13.114060+0000 (oldest deadline 2023-07-21T19:08:37.214254+0000) 2023-07-21T19:08:49.426 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:49 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:49.139+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:17.214857+0000 front 2023-07-21T19:08:17.214903+0000 (oldest deadline 2023-07-21T19:08:41.915044+0000) 2023-07-21T19:08:49.442 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:49 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:49.164+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:47.629329+0000 front 2023-07-21T19:07:47.629176+0000 (oldest deadline 2023-07-21T19:08:10.529277+0000) 2023-07-21T19:08:49.443 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:49 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:49.164+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:04.631984+0000 front 2023-07-21T19:08:04.631918+0000 (oldest deadline 2023-07-21T19:08:26.332176+0000) 2023-07-21T19:08:49.443 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:49 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:49.164+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:11.534192+0000 front 2023-07-21T19:08:11.534232+0000 (oldest deadline 2023-07-21T19:08:35.034206+0000) 2023-07-21T19:08:49.443 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:49 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:49.164+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:18.535636+0000 front 2023-07-21T19:08:18.535608+0000 (oldest deadline 2023-07-21T19:08:42.635555+0000) 2023-07-21T19:08:49.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:49 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:49.433+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:45.542174+0000 front 2023-07-21T19:07:45.542192+0000 (oldest deadline 2023-07-21T19:08:11.442323+0000) 2023-07-21T19:08:49.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:49 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:49.433+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.044189+0000 front 2023-07-21T19:07:56.044156+0000 (oldest deadline 2023-07-21T19:08:18.944205+0000) 2023-07-21T19:08:49.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:49 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:49.433+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:02.345832+0000 front 2023-07-21T19:08:02.345919+0000 (oldest deadline 2023-07-21T19:08:26.445973+0000) 2023-07-21T19:08:49.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:49 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:49.433+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:17.547994+0000 front 2023-07-21T19:08:17.548134+0000 (oldest deadline 2023-07-21T19:08:43.448059+0000) 2023-07-21T19:08:49.876 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:49 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:49.374+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.769616+0000 front 2023-07-21T19:07:48.769555+0000 (oldest deadline 2023-07-21T19:08:12.770222+0000) 2023-07-21T19:08:49.876 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:49 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:49.374+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.271039+0000 front 2023-07-21T19:07:56.271067+0000 (oldest deadline 2023-07-21T19:08:23.871770+0000) 2023-07-21T19:08:49.876 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:49 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:49.374+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:03.872288+0000 front 2023-07-21T19:08:03.872364+0000 (oldest deadline 2023-07-21T19:08:26.172487+0000) 2023-07-21T19:08:49.876 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:49 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:49.374+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:10.873569+0000 front 2023-07-21T19:08:10.873733+0000 (oldest deadline 2023-07-21T19:08:35.573593+0000) 2023-07-21T19:08:49.877 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:49 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:49.374+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:19.074521+0000 front 2023-07-21T19:08:19.074500+0000 (oldest deadline 2023-07-21T19:08:44.374675+0000) 2023-07-21T19:08:49.877 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:49 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:49.374+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2023-07-21T19:08:24.375386+0000 front 2023-07-21T19:08:24.375361+0000 (oldest deadline 2023-07-21T19:08:49.075435+0000) 2023-07-21T19:08:50.425 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:50 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:50.150+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:46.109399+0000 front 2023-07-21T19:07:46.109431+0000 (oldest deadline 2023-07-21T19:08:11.410380+0000) 2023-07-21T19:08:50.425 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:50 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:50.150+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.711568+0000 front 2023-07-21T19:07:56.711252+0000 (oldest deadline 2023-07-21T19:08:19.011261+0000) 2023-07-21T19:08:50.425 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:50 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:50.150+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:04.912868+0000 front 2023-07-21T19:08:04.912667+0000 (oldest deadline 2023-07-21T19:08:29.012921+0000) 2023-07-21T19:08:50.426 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:50 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:50.150+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:13.114021+0000 front 2023-07-21T19:08:13.114060+0000 (oldest deadline 2023-07-21T19:08:37.214254+0000) 2023-07-21T19:08:50.426 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:50 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:50.150+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:17.214857+0000 front 2023-07-21T19:08:17.214903+0000 (oldest deadline 2023-07-21T19:08:41.915044+0000) 2023-07-21T19:08:50.457 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:50 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:50.179+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:47.629329+0000 front 2023-07-21T19:07:47.629176+0000 (oldest deadline 2023-07-21T19:08:10.529277+0000) 2023-07-21T19:08:50.458 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:50 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:50.179+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:04.631984+0000 front 2023-07-21T19:08:04.631918+0000 (oldest deadline 2023-07-21T19:08:26.332176+0000) 2023-07-21T19:08:50.458 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:50 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:50.179+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:11.534192+0000 front 2023-07-21T19:08:11.534232+0000 (oldest deadline 2023-07-21T19:08:35.034206+0000) 2023-07-21T19:08:50.458 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:50 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:50.179+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:18.535636+0000 front 2023-07-21T19:08:18.535608+0000 (oldest deadline 2023-07-21T19:08:42.635555+0000) 2023-07-21T19:08:50.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:50 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:50.447+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:45.542174+0000 front 2023-07-21T19:07:45.542192+0000 (oldest deadline 2023-07-21T19:08:11.442323+0000) 2023-07-21T19:08:50.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:50 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:50.447+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.044189+0000 front 2023-07-21T19:07:56.044156+0000 (oldest deadline 2023-07-21T19:08:18.944205+0000) 2023-07-21T19:08:50.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:50 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:50.447+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:02.345832+0000 front 2023-07-21T19:08:02.345919+0000 (oldest deadline 2023-07-21T19:08:26.445973+0000) 2023-07-21T19:08:50.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:50 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:50.447+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:17.547994+0000 front 2023-07-21T19:08:17.548134+0000 (oldest deadline 2023-07-21T19:08:43.448059+0000) 2023-07-21T19:08:50.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:50 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:50.447+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2023-07-21T19:08:28.149016+0000 front 2023-07-21T19:08:28.149051+0000 (oldest deadline 2023-07-21T19:08:49.849039+0000) 2023-07-21T19:08:50.876 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:50 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:50.415+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.769616+0000 front 2023-07-21T19:07:48.769555+0000 (oldest deadline 2023-07-21T19:08:12.770222+0000) 2023-07-21T19:08:50.876 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:50 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:50.415+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.271039+0000 front 2023-07-21T19:07:56.271067+0000 (oldest deadline 2023-07-21T19:08:23.871770+0000) 2023-07-21T19:08:50.876 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:50 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:50.415+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:03.872288+0000 front 2023-07-21T19:08:03.872364+0000 (oldest deadline 2023-07-21T19:08:26.172487+0000) 2023-07-21T19:08:50.876 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:50 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:50.415+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:10.873569+0000 front 2023-07-21T19:08:10.873733+0000 (oldest deadline 2023-07-21T19:08:35.573593+0000) 2023-07-21T19:08:50.876 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:50 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:50.415+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:19.074521+0000 front 2023-07-21T19:08:19.074500+0000 (oldest deadline 2023-07-21T19:08:44.374675+0000) 2023-07-21T19:08:50.877 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:50 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:50.415+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2023-07-21T19:08:24.375386+0000 front 2023-07-21T19:08:24.375361+0000 (oldest deadline 2023-07-21T19:08:49.075435+0000) 2023-07-21T19:08:51.425 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:51 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:51.115+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:46.109399+0000 front 2023-07-21T19:07:46.109431+0000 (oldest deadline 2023-07-21T19:08:11.410380+0000) 2023-07-21T19:08:51.425 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:51 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:51.115+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.711568+0000 front 2023-07-21T19:07:56.711252+0000 (oldest deadline 2023-07-21T19:08:19.011261+0000) 2023-07-21T19:08:51.425 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:51 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:51.115+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:04.912868+0000 front 2023-07-21T19:08:04.912667+0000 (oldest deadline 2023-07-21T19:08:29.012921+0000) 2023-07-21T19:08:51.425 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:51 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:51.115+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:13.114021+0000 front 2023-07-21T19:08:13.114060+0000 (oldest deadline 2023-07-21T19:08:37.214254+0000) 2023-07-21T19:08:51.426 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:51 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:51.115+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:17.214857+0000 front 2023-07-21T19:08:17.214903+0000 (oldest deadline 2023-07-21T19:08:41.915044+0000) 2023-07-21T19:08:51.607 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:51 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:51.409+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:45.542174+0000 front 2023-07-21T19:07:45.542192+0000 (oldest deadline 2023-07-21T19:08:11.442323+0000) 2023-07-21T19:08:51.608 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:51 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:51.409+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.044189+0000 front 2023-07-21T19:07:56.044156+0000 (oldest deadline 2023-07-21T19:08:18.944205+0000) 2023-07-21T19:08:51.608 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:51 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:51.409+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:02.345832+0000 front 2023-07-21T19:08:02.345919+0000 (oldest deadline 2023-07-21T19:08:26.445973+0000) 2023-07-21T19:08:51.608 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:51 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:51.409+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:17.547994+0000 front 2023-07-21T19:08:17.548134+0000 (oldest deadline 2023-07-21T19:08:43.448059+0000) 2023-07-21T19:08:51.608 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:51 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:51.409+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2023-07-21T19:08:28.149016+0000 front 2023-07-21T19:08:28.149051+0000 (oldest deadline 2023-07-21T19:08:49.849039+0000) 2023-07-21T19:08:51.608 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:51 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:51.207+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:47.629329+0000 front 2023-07-21T19:07:47.629176+0000 (oldest deadline 2023-07-21T19:08:10.529277+0000) 2023-07-21T19:08:51.609 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:51 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:51.207+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:04.631984+0000 front 2023-07-21T19:08:04.631918+0000 (oldest deadline 2023-07-21T19:08:26.332176+0000) 2023-07-21T19:08:51.609 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:51 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:51.207+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:11.534192+0000 front 2023-07-21T19:08:11.534232+0000 (oldest deadline 2023-07-21T19:08:35.034206+0000) 2023-07-21T19:08:51.609 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:51 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8[115132]: 2023-07-21T19:08:51.207+0000 7feab72d1700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:18.535636+0000 front 2023-07-21T19:08:18.535608+0000 (oldest deadline 2023-07-21T19:08:42.635555+0000) 2023-07-21T19:08:51.876 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:51 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:51.396+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.769616+0000 front 2023-07-21T19:07:48.769555+0000 (oldest deadline 2023-07-21T19:08:12.770222+0000) 2023-07-21T19:08:51.876 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:51 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:51.396+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.271039+0000 front 2023-07-21T19:07:56.271067+0000 (oldest deadline 2023-07-21T19:08:23.871770+0000) 2023-07-21T19:08:51.876 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:51 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:51.396+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:03.872288+0000 front 2023-07-21T19:08:03.872364+0000 (oldest deadline 2023-07-21T19:08:26.172487+0000) 2023-07-21T19:08:51.877 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:51 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:51.396+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:10.873569+0000 front 2023-07-21T19:08:10.873733+0000 (oldest deadline 2023-07-21T19:08:35.573593+0000) 2023-07-21T19:08:51.877 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:51 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:51.396+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:19.074521+0000 front 2023-07-21T19:08:19.074500+0000 (oldest deadline 2023-07-21T19:08:44.374675+0000) 2023-07-21T19:08:51.877 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:51 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:51.396+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2023-07-21T19:08:24.375386+0000 front 2023-07-21T19:08:24.375361+0000 (oldest deadline 2023-07-21T19:08:49.075435+0000) 2023-07-21T19:08:52.357 INFO:journalctl@ceph.osd.8.smithi193.stdout:Jul 21 19:08:51 smithi193 bash[137371]: ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-8 2023-07-21T19:08:52.425 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:52 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:52.125+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:46.109399+0000 front 2023-07-21T19:07:46.109431+0000 (oldest deadline 2023-07-21T19:08:11.410380+0000) 2023-07-21T19:08:52.425 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:52 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:52.125+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.711568+0000 front 2023-07-21T19:07:56.711252+0000 (oldest deadline 2023-07-21T19:08:19.011261+0000) 2023-07-21T19:08:52.425 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:52 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:52.125+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:04.912868+0000 front 2023-07-21T19:08:04.912667+0000 (oldest deadline 2023-07-21T19:08:29.012921+0000) 2023-07-21T19:08:52.425 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:52 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:52.125+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:13.114021+0000 front 2023-07-21T19:08:13.114060+0000 (oldest deadline 2023-07-21T19:08:37.214254+0000) 2023-07-21T19:08:52.426 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:52 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:52.125+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:17.214857+0000 front 2023-07-21T19:08:17.214903+0000 (oldest deadline 2023-07-21T19:08:41.915044+0000) 2023-07-21T19:08:52.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:52 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:52.431+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:45.542174+0000 front 2023-07-21T19:07:45.542192+0000 (oldest deadline 2023-07-21T19:08:11.442323+0000) 2023-07-21T19:08:52.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:52 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:52.431+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.044189+0000 front 2023-07-21T19:07:56.044156+0000 (oldest deadline 2023-07-21T19:08:18.944205+0000) 2023-07-21T19:08:52.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:52 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:52.431+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:02.345832+0000 front 2023-07-21T19:08:02.345919+0000 (oldest deadline 2023-07-21T19:08:26.445973+0000) 2023-07-21T19:08:52.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:52 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:52.431+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:17.547994+0000 front 2023-07-21T19:08:17.548134+0000 (oldest deadline 2023-07-21T19:08:43.448059+0000) 2023-07-21T19:08:52.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:52 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:52.431+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2023-07-21T19:08:28.149016+0000 front 2023-07-21T19:08:28.149051+0000 (oldest deadline 2023-07-21T19:08:49.849039+0000) 2023-07-21T19:08:52.876 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:52 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:52.411+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.769616+0000 front 2023-07-21T19:07:48.769555+0000 (oldest deadline 2023-07-21T19:08:12.770222+0000) 2023-07-21T19:08:52.876 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:52 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:52.411+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.271039+0000 front 2023-07-21T19:07:56.271067+0000 (oldest deadline 2023-07-21T19:08:23.871770+0000) 2023-07-21T19:08:52.876 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:52 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:52.411+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:03.872288+0000 front 2023-07-21T19:08:03.872364+0000 (oldest deadline 2023-07-21T19:08:26.172487+0000) 2023-07-21T19:08:52.877 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:52 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:52.411+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:10.873569+0000 front 2023-07-21T19:08:10.873733+0000 (oldest deadline 2023-07-21T19:08:35.573593+0000) 2023-07-21T19:08:52.877 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:52 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:52.411+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:19.074521+0000 front 2023-07-21T19:08:19.074500+0000 (oldest deadline 2023-07-21T19:08:44.374675+0000) 2023-07-21T19:08:52.877 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:52 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:52.411+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2023-07-21T19:08:24.375386+0000 front 2023-07-21T19:08:24.375361+0000 (oldest deadline 2023-07-21T19:08:49.075435+0000) 2023-07-21T19:08:53.343 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@osd.8.service' 2023-07-21T19:08:53.383 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T19:08:53.383 INFO:tasks.cephadm.osd.8:Stopped osd.8 2023-07-21T19:08:53.383 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2023-07-21T19:08:53.384 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@osd.9 2023-07-21T19:08:53.415 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:53 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:53.112+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:46.109399+0000 front 2023-07-21T19:07:46.109431+0000 (oldest deadline 2023-07-21T19:08:11.410380+0000) 2023-07-21T19:08:53.415 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:53 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:53.112+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.711568+0000 front 2023-07-21T19:07:56.711252+0000 (oldest deadline 2023-07-21T19:08:19.011261+0000) 2023-07-21T19:08:53.416 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:53 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:53.112+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:04.912868+0000 front 2023-07-21T19:08:04.912667+0000 (oldest deadline 2023-07-21T19:08:29.012921+0000) 2023-07-21T19:08:53.416 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:53 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:53.112+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:13.114021+0000 front 2023-07-21T19:08:13.114060+0000 (oldest deadline 2023-07-21T19:08:37.214254+0000) 2023-07-21T19:08:53.416 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:53 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:53.112+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:17.214857+0000 front 2023-07-21T19:08:17.214903+0000 (oldest deadline 2023-07-21T19:08:41.915044+0000) 2023-07-21T19:08:53.417 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:53 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:53.112+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2023-07-21T19:08:27.617572+0000 front 2023-07-21T19:08:27.617376+0000 (oldest deadline 2023-07-21T19:08:52.317340+0000) 2023-07-21T19:08:53.607 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:53 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:53.462+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:45.542174+0000 front 2023-07-21T19:07:45.542192+0000 (oldest deadline 2023-07-21T19:08:11.442323+0000) 2023-07-21T19:08:53.607 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:53 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:53.462+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.044189+0000 front 2023-07-21T19:07:56.044156+0000 (oldest deadline 2023-07-21T19:08:18.944205+0000) 2023-07-21T19:08:53.608 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:53 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:53.462+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:02.345832+0000 front 2023-07-21T19:08:02.345919+0000 (oldest deadline 2023-07-21T19:08:26.445973+0000) 2023-07-21T19:08:53.608 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:53 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:53.462+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:17.547994+0000 front 2023-07-21T19:08:17.548134+0000 (oldest deadline 2023-07-21T19:08:43.448059+0000) 2023-07-21T19:08:53.608 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:53 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:53.462+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2023-07-21T19:08:28.149016+0000 front 2023-07-21T19:08:28.149051+0000 (oldest deadline 2023-07-21T19:08:49.849039+0000) 2023-07-21T19:08:53.675 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:53 smithi039 systemd[1]: Stopping Ceph osd.9 for eaf8e25a-27f4-11ee-9b34-001a4aab830c... 2023-07-21T19:08:53.876 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:53 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:53.395+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.769616+0000 front 2023-07-21T19:07:48.769555+0000 (oldest deadline 2023-07-21T19:08:12.770222+0000) 2023-07-21T19:08:53.877 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:53 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:53.395+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.271039+0000 front 2023-07-21T19:07:56.271067+0000 (oldest deadline 2023-07-21T19:08:23.871770+0000) 2023-07-21T19:08:53.877 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:53 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:53.395+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:03.872288+0000 front 2023-07-21T19:08:03.872364+0000 (oldest deadline 2023-07-21T19:08:26.172487+0000) 2023-07-21T19:08:53.877 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:53 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:53.395+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:10.873569+0000 front 2023-07-21T19:08:10.873733+0000 (oldest deadline 2023-07-21T19:08:35.573593+0000) 2023-07-21T19:08:53.877 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:53 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:53.395+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:19.074521+0000 front 2023-07-21T19:08:19.074500+0000 (oldest deadline 2023-07-21T19:08:44.374675+0000) 2023-07-21T19:08:53.877 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:53 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:53.395+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2023-07-21T19:08:24.375386+0000 front 2023-07-21T19:08:24.375361+0000 (oldest deadline 2023-07-21T19:08:49.075435+0000) 2023-07-21T19:08:54.146 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:53 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:53.734+0000 7f1c07a96700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T19:08:54.147 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:53 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:53.734+0000 7f1c07a96700 -1 osd.9 127 *** Got signal Terminated *** 2023-07-21T19:08:54.147 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:53 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:53.734+0000 7f1c07a96700 -1 osd.9 127 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-21T19:08:54.147 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:54 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:54.137+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:46.109399+0000 front 2023-07-21T19:07:46.109431+0000 (oldest deadline 2023-07-21T19:08:11.410380+0000) 2023-07-21T19:08:54.425 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:54 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:54.137+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.711568+0000 front 2023-07-21T19:07:56.711252+0000 (oldest deadline 2023-07-21T19:08:19.011261+0000) 2023-07-21T19:08:54.425 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:54 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:54.137+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:04.912868+0000 front 2023-07-21T19:08:04.912667+0000 (oldest deadline 2023-07-21T19:08:29.012921+0000) 2023-07-21T19:08:54.425 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:54 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:54.137+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:13.114021+0000 front 2023-07-21T19:08:13.114060+0000 (oldest deadline 2023-07-21T19:08:37.214254+0000) 2023-07-21T19:08:54.425 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:54 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:54.137+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:17.214857+0000 front 2023-07-21T19:08:17.214903+0000 (oldest deadline 2023-07-21T19:08:41.915044+0000) 2023-07-21T19:08:54.425 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:54 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:54.137+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2023-07-21T19:08:27.617572+0000 front 2023-07-21T19:08:27.617376+0000 (oldest deadline 2023-07-21T19:08:52.317340+0000) 2023-07-21T19:08:54.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:54 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:54.480+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:45.542174+0000 front 2023-07-21T19:07:45.542192+0000 (oldest deadline 2023-07-21T19:08:11.442323+0000) 2023-07-21T19:08:54.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:54 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:54.480+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.044189+0000 front 2023-07-21T19:07:56.044156+0000 (oldest deadline 2023-07-21T19:08:18.944205+0000) 2023-07-21T19:08:54.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:54 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:54.480+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:02.345832+0000 front 2023-07-21T19:08:02.345919+0000 (oldest deadline 2023-07-21T19:08:26.445973+0000) 2023-07-21T19:08:54.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:54 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:54.480+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:17.547994+0000 front 2023-07-21T19:08:17.548134+0000 (oldest deadline 2023-07-21T19:08:43.448059+0000) 2023-07-21T19:08:54.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:54 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:54.480+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2023-07-21T19:08:28.149016+0000 front 2023-07-21T19:08:28.149051+0000 (oldest deadline 2023-07-21T19:08:49.849039+0000) 2023-07-21T19:08:54.876 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:54 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:54.443+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.769616+0000 front 2023-07-21T19:07:48.769555+0000 (oldest deadline 2023-07-21T19:08:12.770222+0000) 2023-07-21T19:08:54.876 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:54 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:54.443+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.271039+0000 front 2023-07-21T19:07:56.271067+0000 (oldest deadline 2023-07-21T19:08:23.871770+0000) 2023-07-21T19:08:54.877 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:54 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:54.443+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:03.872288+0000 front 2023-07-21T19:08:03.872364+0000 (oldest deadline 2023-07-21T19:08:26.172487+0000) 2023-07-21T19:08:54.877 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:54 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:54.443+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:10.873569+0000 front 2023-07-21T19:08:10.873733+0000 (oldest deadline 2023-07-21T19:08:35.573593+0000) 2023-07-21T19:08:54.877 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:54 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:54.443+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:19.074521+0000 front 2023-07-21T19:08:19.074500+0000 (oldest deadline 2023-07-21T19:08:44.374675+0000) 2023-07-21T19:08:54.877 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:54 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:54.443+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2023-07-21T19:08:24.375386+0000 front 2023-07-21T19:08:24.375361+0000 (oldest deadline 2023-07-21T19:08:49.075435+0000) 2023-07-21T19:08:55.425 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:55 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:55.123+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:46.109399+0000 front 2023-07-21T19:07:46.109431+0000 (oldest deadline 2023-07-21T19:08:11.410380+0000) 2023-07-21T19:08:55.425 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:55 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:55.123+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.711568+0000 front 2023-07-21T19:07:56.711252+0000 (oldest deadline 2023-07-21T19:08:19.011261+0000) 2023-07-21T19:08:55.426 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:55 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:55.123+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:04.912868+0000 front 2023-07-21T19:08:04.912667+0000 (oldest deadline 2023-07-21T19:08:29.012921+0000) 2023-07-21T19:08:55.426 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:55 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:55.123+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:13.114021+0000 front 2023-07-21T19:08:13.114060+0000 (oldest deadline 2023-07-21T19:08:37.214254+0000) 2023-07-21T19:08:55.426 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:55 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:55.123+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:17.214857+0000 front 2023-07-21T19:08:17.214903+0000 (oldest deadline 2023-07-21T19:08:41.915044+0000) 2023-07-21T19:08:55.426 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:55 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:55.123+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2023-07-21T19:08:27.617572+0000 front 2023-07-21T19:08:27.617376+0000 (oldest deadline 2023-07-21T19:08:52.317340+0000) 2023-07-21T19:08:55.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:55 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:55.522+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:45.542174+0000 front 2023-07-21T19:07:45.542192+0000 (oldest deadline 2023-07-21T19:08:11.442323+0000) 2023-07-21T19:08:55.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:55 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:55.522+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.044189+0000 front 2023-07-21T19:07:56.044156+0000 (oldest deadline 2023-07-21T19:08:18.944205+0000) 2023-07-21T19:08:55.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:55 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:55.522+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:02.345832+0000 front 2023-07-21T19:08:02.345919+0000 (oldest deadline 2023-07-21T19:08:26.445973+0000) 2023-07-21T19:08:55.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:55 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:55.522+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:17.547994+0000 front 2023-07-21T19:08:17.548134+0000 (oldest deadline 2023-07-21T19:08:43.448059+0000) 2023-07-21T19:08:55.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:55 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:55.522+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2023-07-21T19:08:28.149016+0000 front 2023-07-21T19:08:28.149051+0000 (oldest deadline 2023-07-21T19:08:49.849039+0000) 2023-07-21T19:08:55.876 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:55 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:55.404+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.769616+0000 front 2023-07-21T19:07:48.769555+0000 (oldest deadline 2023-07-21T19:08:12.770222+0000) 2023-07-21T19:08:55.876 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:55 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:55.404+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.271039+0000 front 2023-07-21T19:07:56.271067+0000 (oldest deadline 2023-07-21T19:08:23.871770+0000) 2023-07-21T19:08:55.877 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:55 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:55.404+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:03.872288+0000 front 2023-07-21T19:08:03.872364+0000 (oldest deadline 2023-07-21T19:08:26.172487+0000) 2023-07-21T19:08:55.877 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:55 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:55.404+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:10.873569+0000 front 2023-07-21T19:08:10.873733+0000 (oldest deadline 2023-07-21T19:08:35.573593+0000) 2023-07-21T19:08:55.877 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:55 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:55.404+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:19.074521+0000 front 2023-07-21T19:08:19.074500+0000 (oldest deadline 2023-07-21T19:08:44.374675+0000) 2023-07-21T19:08:55.877 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:55 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:55.404+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2023-07-21T19:08:24.375386+0000 front 2023-07-21T19:08:24.375361+0000 (oldest deadline 2023-07-21T19:08:49.075435+0000) 2023-07-21T19:08:56.425 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:56 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:56.097+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:46.109399+0000 front 2023-07-21T19:07:46.109431+0000 (oldest deadline 2023-07-21T19:08:11.410380+0000) 2023-07-21T19:08:56.425 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:56 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:56.097+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.711568+0000 front 2023-07-21T19:07:56.711252+0000 (oldest deadline 2023-07-21T19:08:19.011261+0000) 2023-07-21T19:08:56.425 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:56 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:56.097+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:04.912868+0000 front 2023-07-21T19:08:04.912667+0000 (oldest deadline 2023-07-21T19:08:29.012921+0000) 2023-07-21T19:08:56.425 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:56 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:56.097+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:13.114021+0000 front 2023-07-21T19:08:13.114060+0000 (oldest deadline 2023-07-21T19:08:37.214254+0000) 2023-07-21T19:08:56.426 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:56 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:56.097+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:17.214857+0000 front 2023-07-21T19:08:17.214903+0000 (oldest deadline 2023-07-21T19:08:41.915044+0000) 2023-07-21T19:08:56.426 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:56 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:56.097+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2023-07-21T19:08:27.617572+0000 front 2023-07-21T19:08:27.617376+0000 (oldest deadline 2023-07-21T19:08:52.317340+0000) 2023-07-21T19:08:56.860 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:56 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:56.489+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:45.542174+0000 front 2023-07-21T19:07:45.542192+0000 (oldest deadline 2023-07-21T19:08:11.442323+0000) 2023-07-21T19:08:56.861 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:56 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:56.489+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.044189+0000 front 2023-07-21T19:07:56.044156+0000 (oldest deadline 2023-07-21T19:08:18.944205+0000) 2023-07-21T19:08:56.861 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:56 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:56.489+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:02.345832+0000 front 2023-07-21T19:08:02.345919+0000 (oldest deadline 2023-07-21T19:08:26.445973+0000) 2023-07-21T19:08:56.861 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:56 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:56.489+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:17.547994+0000 front 2023-07-21T19:08:17.548134+0000 (oldest deadline 2023-07-21T19:08:43.448059+0000) 2023-07-21T19:08:56.861 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:56 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:56.489+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2023-07-21T19:08:28.149016+0000 front 2023-07-21T19:08:28.149051+0000 (oldest deadline 2023-07-21T19:08:49.849039+0000) 2023-07-21T19:08:56.876 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:56 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:56.446+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.769616+0000 front 2023-07-21T19:07:48.769555+0000 (oldest deadline 2023-07-21T19:08:12.770222+0000) 2023-07-21T19:08:56.876 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:56 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:56.446+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.271039+0000 front 2023-07-21T19:07:56.271067+0000 (oldest deadline 2023-07-21T19:08:23.871770+0000) 2023-07-21T19:08:56.876 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:56 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:56.446+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:03.872288+0000 front 2023-07-21T19:08:03.872364+0000 (oldest deadline 2023-07-21T19:08:26.172487+0000) 2023-07-21T19:08:56.876 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:56 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:56.446+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:10.873569+0000 front 2023-07-21T19:08:10.873733+0000 (oldest deadline 2023-07-21T19:08:35.573593+0000) 2023-07-21T19:08:56.877 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:56 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:56.446+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:19.074521+0000 front 2023-07-21T19:08:19.074500+0000 (oldest deadline 2023-07-21T19:08:44.374675+0000) 2023-07-21T19:08:56.877 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:56 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:56.446+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2023-07-21T19:08:24.375386+0000 front 2023-07-21T19:08:24.375361+0000 (oldest deadline 2023-07-21T19:08:49.075435+0000) 2023-07-21T19:08:57.425 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:57 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:57.072+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:46.109399+0000 front 2023-07-21T19:07:46.109431+0000 (oldest deadline 2023-07-21T19:08:11.410380+0000) 2023-07-21T19:08:57.425 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:57 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:57.072+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.711568+0000 front 2023-07-21T19:07:56.711252+0000 (oldest deadline 2023-07-21T19:08:19.011261+0000) 2023-07-21T19:08:57.425 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:57 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:57.072+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:04.912868+0000 front 2023-07-21T19:08:04.912667+0000 (oldest deadline 2023-07-21T19:08:29.012921+0000) 2023-07-21T19:08:57.426 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:57 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:57.072+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:13.114021+0000 front 2023-07-21T19:08:13.114060+0000 (oldest deadline 2023-07-21T19:08:37.214254+0000) 2023-07-21T19:08:57.426 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:57 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:57.072+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:17.214857+0000 front 2023-07-21T19:08:17.214903+0000 (oldest deadline 2023-07-21T19:08:41.915044+0000) 2023-07-21T19:08:57.426 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:57 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:57.072+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2023-07-21T19:08:27.617572+0000 front 2023-07-21T19:08:27.617376+0000 (oldest deadline 2023-07-21T19:08:52.317340+0000) 2023-07-21T19:08:57.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:57 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:57.468+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:45.542174+0000 front 2023-07-21T19:07:45.542192+0000 (oldest deadline 2023-07-21T19:08:11.442323+0000) 2023-07-21T19:08:57.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:57 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:57.468+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.044189+0000 front 2023-07-21T19:07:56.044156+0000 (oldest deadline 2023-07-21T19:08:18.944205+0000) 2023-07-21T19:08:57.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:57 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:57.468+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:02.345832+0000 front 2023-07-21T19:08:02.345919+0000 (oldest deadline 2023-07-21T19:08:26.445973+0000) 2023-07-21T19:08:57.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:57 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:57.468+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:17.547994+0000 front 2023-07-21T19:08:17.548134+0000 (oldest deadline 2023-07-21T19:08:43.448059+0000) 2023-07-21T19:08:57.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:57 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:57.468+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2023-07-21T19:08:28.149016+0000 front 2023-07-21T19:08:28.149051+0000 (oldest deadline 2023-07-21T19:08:49.849039+0000) 2023-07-21T19:08:57.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:57 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:57.468+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2023-07-21T19:08:35.050771+0000 front 2023-07-21T19:08:35.050572+0000 (oldest deadline 2023-07-21T19:08:57.350446+0000) 2023-07-21T19:08:57.876 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:57 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:57.438+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.769616+0000 front 2023-07-21T19:07:48.769555+0000 (oldest deadline 2023-07-21T19:08:12.770222+0000) 2023-07-21T19:08:57.876 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:57 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:57.438+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.271039+0000 front 2023-07-21T19:07:56.271067+0000 (oldest deadline 2023-07-21T19:08:23.871770+0000) 2023-07-21T19:08:57.877 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:57 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:57.438+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:03.872288+0000 front 2023-07-21T19:08:03.872364+0000 (oldest deadline 2023-07-21T19:08:26.172487+0000) 2023-07-21T19:08:57.877 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:57 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:57.438+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:10.873569+0000 front 2023-07-21T19:08:10.873733+0000 (oldest deadline 2023-07-21T19:08:35.573593+0000) 2023-07-21T19:08:57.877 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:57 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:57.438+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:19.074521+0000 front 2023-07-21T19:08:19.074500+0000 (oldest deadline 2023-07-21T19:08:44.374675+0000) 2023-07-21T19:08:57.878 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:57 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:57.438+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2023-07-21T19:08:24.375386+0000 front 2023-07-21T19:08:24.375361+0000 (oldest deadline 2023-07-21T19:08:49.075435+0000) 2023-07-21T19:08:58.425 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:58 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:58.111+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:46.109399+0000 front 2023-07-21T19:07:46.109431+0000 (oldest deadline 2023-07-21T19:08:11.410380+0000) 2023-07-21T19:08:58.425 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:58 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:58.112+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.711568+0000 front 2023-07-21T19:07:56.711252+0000 (oldest deadline 2023-07-21T19:08:19.011261+0000) 2023-07-21T19:08:58.426 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:58 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:58.112+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:04.912868+0000 front 2023-07-21T19:08:04.912667+0000 (oldest deadline 2023-07-21T19:08:29.012921+0000) 2023-07-21T19:08:58.426 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:58 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:58.112+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:13.114021+0000 front 2023-07-21T19:08:13.114060+0000 (oldest deadline 2023-07-21T19:08:37.214254+0000) 2023-07-21T19:08:58.426 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:58 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:58.112+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:17.214857+0000 front 2023-07-21T19:08:17.214903+0000 (oldest deadline 2023-07-21T19:08:41.915044+0000) 2023-07-21T19:08:58.426 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:58 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9[125825]: 2023-07-21T19:08:58.112+0000 7f1c02088700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2023-07-21T19:08:27.617572+0000 front 2023-07-21T19:08:27.617376+0000 (oldest deadline 2023-07-21T19:08:52.317340+0000) 2023-07-21T19:08:58.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:58 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:58.463+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:45.542174+0000 front 2023-07-21T19:07:45.542192+0000 (oldest deadline 2023-07-21T19:08:11.442323+0000) 2023-07-21T19:08:58.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:58 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:58.463+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.044189+0000 front 2023-07-21T19:07:56.044156+0000 (oldest deadline 2023-07-21T19:08:18.944205+0000) 2023-07-21T19:08:58.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:58 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:58.463+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:02.345832+0000 front 2023-07-21T19:08:02.345919+0000 (oldest deadline 2023-07-21T19:08:26.445973+0000) 2023-07-21T19:08:58.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:58 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:58.463+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:17.547994+0000 front 2023-07-21T19:08:17.548134+0000 (oldest deadline 2023-07-21T19:08:43.448059+0000) 2023-07-21T19:08:58.859 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:58 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:58.463+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2023-07-21T19:08:28.149016+0000 front 2023-07-21T19:08:28.149051+0000 (oldest deadline 2023-07-21T19:08:49.849039+0000) 2023-07-21T19:08:58.859 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:58 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:58.463+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2023-07-21T19:08:35.050771+0000 front 2023-07-21T19:08:35.050572+0000 (oldest deadline 2023-07-21T19:08:57.350446+0000) 2023-07-21T19:08:58.876 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:58 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:58.475+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.769616+0000 front 2023-07-21T19:07:48.769555+0000 (oldest deadline 2023-07-21T19:08:12.770222+0000) 2023-07-21T19:08:58.876 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:58 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:58.475+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.271039+0000 front 2023-07-21T19:07:56.271067+0000 (oldest deadline 2023-07-21T19:08:23.871770+0000) 2023-07-21T19:08:58.877 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:58 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:58.475+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:03.872288+0000 front 2023-07-21T19:08:03.872364+0000 (oldest deadline 2023-07-21T19:08:26.172487+0000) 2023-07-21T19:08:58.877 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:58 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:58.475+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:10.873569+0000 front 2023-07-21T19:08:10.873733+0000 (oldest deadline 2023-07-21T19:08:35.573593+0000) 2023-07-21T19:08:58.877 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:58 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:58.475+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:19.074521+0000 front 2023-07-21T19:08:19.074500+0000 (oldest deadline 2023-07-21T19:08:44.374675+0000) 2023-07-21T19:08:58.878 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:58 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:58.475+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2023-07-21T19:08:24.375386+0000 front 2023-07-21T19:08:24.375361+0000 (oldest deadline 2023-07-21T19:08:49.075435+0000) 2023-07-21T19:08:59.675 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 21 19:08:59 smithi039 bash[164040]: ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-9 2023-07-21T19:08:59.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:59 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:59.486+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:45.542174+0000 front 2023-07-21T19:07:45.542192+0000 (oldest deadline 2023-07-21T19:08:11.442323+0000) 2023-07-21T19:08:59.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:59 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:59.486+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.044189+0000 front 2023-07-21T19:07:56.044156+0000 (oldest deadline 2023-07-21T19:08:18.944205+0000) 2023-07-21T19:08:59.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:59 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:59.486+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:02.345832+0000 front 2023-07-21T19:08:02.345919+0000 (oldest deadline 2023-07-21T19:08:26.445973+0000) 2023-07-21T19:08:59.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:59 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:59.486+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:17.547994+0000 front 2023-07-21T19:08:17.548134+0000 (oldest deadline 2023-07-21T19:08:43.448059+0000) 2023-07-21T19:08:59.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:59 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:59.486+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2023-07-21T19:08:28.149016+0000 front 2023-07-21T19:08:28.149051+0000 (oldest deadline 2023-07-21T19:08:49.849039+0000) 2023-07-21T19:08:59.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:08:59 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:08:59.486+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2023-07-21T19:08:35.050771+0000 front 2023-07-21T19:08:35.050572+0000 (oldest deadline 2023-07-21T19:08:57.350446+0000) 2023-07-21T19:08:59.876 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:59 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:59.440+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.769616+0000 front 2023-07-21T19:07:48.769555+0000 (oldest deadline 2023-07-21T19:08:12.770222+0000) 2023-07-21T19:08:59.876 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:59 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:59.440+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.271039+0000 front 2023-07-21T19:07:56.271067+0000 (oldest deadline 2023-07-21T19:08:23.871770+0000) 2023-07-21T19:08:59.876 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:59 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:59.440+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:03.872288+0000 front 2023-07-21T19:08:03.872364+0000 (oldest deadline 2023-07-21T19:08:26.172487+0000) 2023-07-21T19:08:59.877 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:59 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:59.440+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:10.873569+0000 front 2023-07-21T19:08:10.873733+0000 (oldest deadline 2023-07-21T19:08:35.573593+0000) 2023-07-21T19:08:59.877 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:59 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:59.440+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:19.074521+0000 front 2023-07-21T19:08:19.074500+0000 (oldest deadline 2023-07-21T19:08:44.374675+0000) 2023-07-21T19:08:59.877 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:08:59 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:08:59.440+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2023-07-21T19:08:24.375386+0000 front 2023-07-21T19:08:24.375361+0000 (oldest deadline 2023-07-21T19:08:49.075435+0000) 2023-07-21T19:09:00.855 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@osd.9.service' 2023-07-21T19:09:00.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:00 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:00.466+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:45.542174+0000 front 2023-07-21T19:07:45.542192+0000 (oldest deadline 2023-07-21T19:08:11.442323+0000) 2023-07-21T19:09:00.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:00 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:00.466+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.044189+0000 front 2023-07-21T19:07:56.044156+0000 (oldest deadline 2023-07-21T19:08:18.944205+0000) 2023-07-21T19:09:00.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:00 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:00.466+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:02.345832+0000 front 2023-07-21T19:08:02.345919+0000 (oldest deadline 2023-07-21T19:08:26.445973+0000) 2023-07-21T19:09:00.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:00 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:00.466+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:17.547994+0000 front 2023-07-21T19:08:17.548134+0000 (oldest deadline 2023-07-21T19:08:43.448059+0000) 2023-07-21T19:09:00.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:00 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:00.466+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2023-07-21T19:08:28.149016+0000 front 2023-07-21T19:08:28.149051+0000 (oldest deadline 2023-07-21T19:08:49.849039+0000) 2023-07-21T19:09:00.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:00 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:00.466+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2023-07-21T19:08:35.050771+0000 front 2023-07-21T19:08:35.050572+0000 (oldest deadline 2023-07-21T19:08:57.350446+0000) 2023-07-21T19:09:00.876 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:09:00 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:09:00.487+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.769616+0000 front 2023-07-21T19:07:48.769555+0000 (oldest deadline 2023-07-21T19:08:12.770222+0000) 2023-07-21T19:09:00.876 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:09:00 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:09:00.487+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.271039+0000 front 2023-07-21T19:07:56.271067+0000 (oldest deadline 2023-07-21T19:08:23.871770+0000) 2023-07-21T19:09:00.877 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:09:00 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:09:00.487+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:03.872288+0000 front 2023-07-21T19:08:03.872364+0000 (oldest deadline 2023-07-21T19:08:26.172487+0000) 2023-07-21T19:09:00.877 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:09:00 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:09:00.487+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:10.873569+0000 front 2023-07-21T19:08:10.873733+0000 (oldest deadline 2023-07-21T19:08:35.573593+0000) 2023-07-21T19:09:00.877 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:09:00 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:09:00.487+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:19.074521+0000 front 2023-07-21T19:08:19.074500+0000 (oldest deadline 2023-07-21T19:08:44.374675+0000) 2023-07-21T19:09:00.877 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:09:00 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:09:00.487+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2023-07-21T19:08:24.375386+0000 front 2023-07-21T19:08:24.375361+0000 (oldest deadline 2023-07-21T19:08:49.075435+0000) 2023-07-21T19:09:00.877 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:09:00 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:09:00.487+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2023-07-21T19:08:33.577726+0000 front 2023-07-21T19:08:33.577687+0000 (oldest deadline 2023-07-21T19:08:59.477398+0000) 2023-07-21T19:09:00.937 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T19:09:00.938 INFO:tasks.cephadm.osd.9:Stopped osd.9 2023-07-21T19:09:00.938 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2023-07-21T19:09:00.938 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@osd.10 2023-07-21T19:09:01.291 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:09:00 smithi155 systemd[1]: Stopping Ceph osd.10 for eaf8e25a-27f4-11ee-9b34-001a4aab830c... 2023-07-21T19:09:01.626 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:09:01 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:09:01.282+0000 7f80beed7700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T19:09:01.626 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:09:01 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:09:01.282+0000 7f80beed7700 -1 osd.10 127 *** Got signal Terminated *** 2023-07-21T19:09:01.626 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:09:01 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:09:01.282+0000 7f80beed7700 -1 osd.10 127 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-21T19:09:01.626 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:09:01 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:09:01.486+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.769616+0000 front 2023-07-21T19:07:48.769555+0000 (oldest deadline 2023-07-21T19:08:12.770222+0000) 2023-07-21T19:09:01.626 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:09:01 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:09:01.486+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.271039+0000 front 2023-07-21T19:07:56.271067+0000 (oldest deadline 2023-07-21T19:08:23.871770+0000) 2023-07-21T19:09:01.627 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:09:01 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:09:01.486+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:03.872288+0000 front 2023-07-21T19:08:03.872364+0000 (oldest deadline 2023-07-21T19:08:26.172487+0000) 2023-07-21T19:09:01.627 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:09:01 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:09:01.486+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:10.873569+0000 front 2023-07-21T19:08:10.873733+0000 (oldest deadline 2023-07-21T19:08:35.573593+0000) 2023-07-21T19:09:01.627 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:09:01 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:09:01.486+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:19.074521+0000 front 2023-07-21T19:08:19.074500+0000 (oldest deadline 2023-07-21T19:08:44.374675+0000) 2023-07-21T19:09:01.627 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:09:01 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:09:01.486+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2023-07-21T19:08:24.375386+0000 front 2023-07-21T19:08:24.375361+0000 (oldest deadline 2023-07-21T19:08:49.075435+0000) 2023-07-21T19:09:01.627 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:09:01 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:09:01.486+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2023-07-21T19:08:33.577726+0000 front 2023-07-21T19:08:33.577687+0000 (oldest deadline 2023-07-21T19:08:59.477398+0000) 2023-07-21T19:09:01.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:01 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:01.416+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:45.542174+0000 front 2023-07-21T19:07:45.542192+0000 (oldest deadline 2023-07-21T19:08:11.442323+0000) 2023-07-21T19:09:01.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:01 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:01.416+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.044189+0000 front 2023-07-21T19:07:56.044156+0000 (oldest deadline 2023-07-21T19:08:18.944205+0000) 2023-07-21T19:09:01.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:01 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:01.416+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:02.345832+0000 front 2023-07-21T19:08:02.345919+0000 (oldest deadline 2023-07-21T19:08:26.445973+0000) 2023-07-21T19:09:01.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:01 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:01.416+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:17.547994+0000 front 2023-07-21T19:08:17.548134+0000 (oldest deadline 2023-07-21T19:08:43.448059+0000) 2023-07-21T19:09:01.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:01 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:01.416+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2023-07-21T19:08:28.149016+0000 front 2023-07-21T19:08:28.149051+0000 (oldest deadline 2023-07-21T19:08:49.849039+0000) 2023-07-21T19:09:01.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:01 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:01.416+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2023-07-21T19:08:35.050771+0000 front 2023-07-21T19:08:35.050572+0000 (oldest deadline 2023-07-21T19:08:57.350446+0000) 2023-07-21T19:09:02.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:02 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:02.387+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:45.542174+0000 front 2023-07-21T19:07:45.542192+0000 (oldest deadline 2023-07-21T19:08:11.442323+0000) 2023-07-21T19:09:02.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:02 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:02.387+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.044189+0000 front 2023-07-21T19:07:56.044156+0000 (oldest deadline 2023-07-21T19:08:18.944205+0000) 2023-07-21T19:09:02.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:02 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:02.387+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:02.345832+0000 front 2023-07-21T19:08:02.345919+0000 (oldest deadline 2023-07-21T19:08:26.445973+0000) 2023-07-21T19:09:02.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:02 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:02.387+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:17.547994+0000 front 2023-07-21T19:08:17.548134+0000 (oldest deadline 2023-07-21T19:08:43.448059+0000) 2023-07-21T19:09:02.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:02 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:02.387+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2023-07-21T19:08:28.149016+0000 front 2023-07-21T19:08:28.149051+0000 (oldest deadline 2023-07-21T19:08:49.849039+0000) 2023-07-21T19:09:02.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:02 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:02.387+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2023-07-21T19:08:35.050771+0000 front 2023-07-21T19:08:35.050572+0000 (oldest deadline 2023-07-21T19:08:57.350446+0000) 2023-07-21T19:09:02.876 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:09:02 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:09:02.471+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.769616+0000 front 2023-07-21T19:07:48.769555+0000 (oldest deadline 2023-07-21T19:08:12.770222+0000) 2023-07-21T19:09:02.876 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:09:02 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:09:02.471+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.271039+0000 front 2023-07-21T19:07:56.271067+0000 (oldest deadline 2023-07-21T19:08:23.871770+0000) 2023-07-21T19:09:02.876 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:09:02 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:09:02.472+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:03.872288+0000 front 2023-07-21T19:08:03.872364+0000 (oldest deadline 2023-07-21T19:08:26.172487+0000) 2023-07-21T19:09:02.876 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:09:02 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:09:02.472+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:10.873569+0000 front 2023-07-21T19:08:10.873733+0000 (oldest deadline 2023-07-21T19:08:35.573593+0000) 2023-07-21T19:09:02.877 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:09:02 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:09:02.472+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:19.074521+0000 front 2023-07-21T19:08:19.074500+0000 (oldest deadline 2023-07-21T19:08:44.374675+0000) 2023-07-21T19:09:02.877 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:09:02 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:09:02.472+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2023-07-21T19:08:24.375386+0000 front 2023-07-21T19:08:24.375361+0000 (oldest deadline 2023-07-21T19:08:49.075435+0000) 2023-07-21T19:09:02.877 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:09:02 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:09:02.472+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2023-07-21T19:08:33.577726+0000 front 2023-07-21T19:08:33.577687+0000 (oldest deadline 2023-07-21T19:08:59.477398+0000) 2023-07-21T19:09:03.607 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:03 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:03.340+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:45.542174+0000 front 2023-07-21T19:07:45.542192+0000 (oldest deadline 2023-07-21T19:08:11.442323+0000) 2023-07-21T19:09:03.607 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:03 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:03.340+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.044189+0000 front 2023-07-21T19:07:56.044156+0000 (oldest deadline 2023-07-21T19:08:18.944205+0000) 2023-07-21T19:09:03.607 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:03 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:03.340+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:02.345832+0000 front 2023-07-21T19:08:02.345919+0000 (oldest deadline 2023-07-21T19:08:26.445973+0000) 2023-07-21T19:09:03.608 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:03 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:03.340+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:17.547994+0000 front 2023-07-21T19:08:17.548134+0000 (oldest deadline 2023-07-21T19:08:43.448059+0000) 2023-07-21T19:09:03.608 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:03 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:03.340+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2023-07-21T19:08:28.149016+0000 front 2023-07-21T19:08:28.149051+0000 (oldest deadline 2023-07-21T19:08:49.849039+0000) 2023-07-21T19:09:03.608 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:03 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:03.340+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2023-07-21T19:08:35.050771+0000 front 2023-07-21T19:08:35.050572+0000 (oldest deadline 2023-07-21T19:08:57.350446+0000) 2023-07-21T19:09:03.876 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:09:03 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:09:03.520+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.769616+0000 front 2023-07-21T19:07:48.769555+0000 (oldest deadline 2023-07-21T19:08:12.770222+0000) 2023-07-21T19:09:03.876 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:09:03 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:09:03.520+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.271039+0000 front 2023-07-21T19:07:56.271067+0000 (oldest deadline 2023-07-21T19:08:23.871770+0000) 2023-07-21T19:09:03.876 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:09:03 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:09:03.520+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:03.872288+0000 front 2023-07-21T19:08:03.872364+0000 (oldest deadline 2023-07-21T19:08:26.172487+0000) 2023-07-21T19:09:03.876 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:09:03 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:09:03.520+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:10.873569+0000 front 2023-07-21T19:08:10.873733+0000 (oldest deadline 2023-07-21T19:08:35.573593+0000) 2023-07-21T19:09:03.877 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:09:03 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:09:03.520+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:19.074521+0000 front 2023-07-21T19:08:19.074500+0000 (oldest deadline 2023-07-21T19:08:44.374675+0000) 2023-07-21T19:09:03.877 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:09:03 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:09:03.520+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2023-07-21T19:08:24.375386+0000 front 2023-07-21T19:08:24.375361+0000 (oldest deadline 2023-07-21T19:08:49.075435+0000) 2023-07-21T19:09:03.877 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:09:03 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:09:03.520+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2023-07-21T19:08:33.577726+0000 front 2023-07-21T19:08:33.577687+0000 (oldest deadline 2023-07-21T19:08:59.477398+0000) 2023-07-21T19:09:04.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:04 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:04.390+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:45.542174+0000 front 2023-07-21T19:07:45.542192+0000 (oldest deadline 2023-07-21T19:08:11.442323+0000) 2023-07-21T19:09:04.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:04 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:04.390+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.044189+0000 front 2023-07-21T19:07:56.044156+0000 (oldest deadline 2023-07-21T19:08:18.944205+0000) 2023-07-21T19:09:04.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:04 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:04.390+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:02.345832+0000 front 2023-07-21T19:08:02.345919+0000 (oldest deadline 2023-07-21T19:08:26.445973+0000) 2023-07-21T19:09:04.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:04 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:04.390+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:17.547994+0000 front 2023-07-21T19:08:17.548134+0000 (oldest deadline 2023-07-21T19:08:43.448059+0000) 2023-07-21T19:09:04.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:04 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:04.390+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2023-07-21T19:08:28.149016+0000 front 2023-07-21T19:08:28.149051+0000 (oldest deadline 2023-07-21T19:08:49.849039+0000) 2023-07-21T19:09:04.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:04 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:04.390+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2023-07-21T19:08:35.050771+0000 front 2023-07-21T19:08:35.050572+0000 (oldest deadline 2023-07-21T19:08:57.350446+0000) 2023-07-21T19:09:04.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:04 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:04.390+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6820 osd.7 since back 2023-07-21T19:08:39.051318+0000 front 2023-07-21T19:08:39.051358+0000 (oldest deadline 2023-07-21T19:09:04.351321+0000) 2023-07-21T19:09:04.876 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:09:04 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:09:04.533+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.769616+0000 front 2023-07-21T19:07:48.769555+0000 (oldest deadline 2023-07-21T19:08:12.770222+0000) 2023-07-21T19:09:04.876 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:09:04 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:09:04.533+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.271039+0000 front 2023-07-21T19:07:56.271067+0000 (oldest deadline 2023-07-21T19:08:23.871770+0000) 2023-07-21T19:09:04.876 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:09:04 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:09:04.533+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:03.872288+0000 front 2023-07-21T19:08:03.872364+0000 (oldest deadline 2023-07-21T19:08:26.172487+0000) 2023-07-21T19:09:04.876 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:09:04 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:09:04.533+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:10.873569+0000 front 2023-07-21T19:08:10.873733+0000 (oldest deadline 2023-07-21T19:08:35.573593+0000) 2023-07-21T19:09:04.877 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:09:04 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:09:04.533+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:19.074521+0000 front 2023-07-21T19:08:19.074500+0000 (oldest deadline 2023-07-21T19:08:44.374675+0000) 2023-07-21T19:09:04.877 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:09:04 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:09:04.533+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2023-07-21T19:08:24.375386+0000 front 2023-07-21T19:08:24.375361+0000 (oldest deadline 2023-07-21T19:08:49.075435+0000) 2023-07-21T19:09:04.877 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:09:04 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:09:04.533+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2023-07-21T19:08:33.577726+0000 front 2023-07-21T19:08:33.577687+0000 (oldest deadline 2023-07-21T19:08:59.477398+0000) 2023-07-21T19:09:05.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:05 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:05.431+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:45.542174+0000 front 2023-07-21T19:07:45.542192+0000 (oldest deadline 2023-07-21T19:08:11.442323+0000) 2023-07-21T19:09:05.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:05 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:05.431+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.044189+0000 front 2023-07-21T19:07:56.044156+0000 (oldest deadline 2023-07-21T19:08:18.944205+0000) 2023-07-21T19:09:05.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:05 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:05.431+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:02.345832+0000 front 2023-07-21T19:08:02.345919+0000 (oldest deadline 2023-07-21T19:08:26.445973+0000) 2023-07-21T19:09:05.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:05 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:05.431+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:17.547994+0000 front 2023-07-21T19:08:17.548134+0000 (oldest deadline 2023-07-21T19:08:43.448059+0000) 2023-07-21T19:09:05.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:05 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:05.431+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2023-07-21T19:08:28.149016+0000 front 2023-07-21T19:08:28.149051+0000 (oldest deadline 2023-07-21T19:08:49.849039+0000) 2023-07-21T19:09:05.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:05 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:05.431+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2023-07-21T19:08:35.050771+0000 front 2023-07-21T19:08:35.050572+0000 (oldest deadline 2023-07-21T19:08:57.350446+0000) 2023-07-21T19:09:05.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:05 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:05.431+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6820 osd.7 since back 2023-07-21T19:08:39.051318+0000 front 2023-07-21T19:08:39.051358+0000 (oldest deadline 2023-07-21T19:09:04.351321+0000) 2023-07-21T19:09:05.876 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:09:05 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:09:05.535+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:48.769616+0000 front 2023-07-21T19:07:48.769555+0000 (oldest deadline 2023-07-21T19:08:12.770222+0000) 2023-07-21T19:09:05.876 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:09:05 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:09:05.535+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.271039+0000 front 2023-07-21T19:07:56.271067+0000 (oldest deadline 2023-07-21T19:08:23.871770+0000) 2023-07-21T19:09:05.876 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:09:05 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:09:05.535+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:03.872288+0000 front 2023-07-21T19:08:03.872364+0000 (oldest deadline 2023-07-21T19:08:26.172487+0000) 2023-07-21T19:09:05.876 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:09:05 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:09:05.535+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-07-21T19:08:10.873569+0000 front 2023-07-21T19:08:10.873733+0000 (oldest deadline 2023-07-21T19:08:35.573593+0000) 2023-07-21T19:09:05.876 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:09:05 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:09:05.535+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:19.074521+0000 front 2023-07-21T19:08:19.074500+0000 (oldest deadline 2023-07-21T19:08:44.374675+0000) 2023-07-21T19:09:05.877 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:09:05 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:09:05.535+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2023-07-21T19:08:24.375386+0000 front 2023-07-21T19:08:24.375361+0000 (oldest deadline 2023-07-21T19:08:49.075435+0000) 2023-07-21T19:09:05.877 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:09:05 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10[119578]: 2023-07-21T19:09:05.535+0000 7f80ba4cb700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2023-07-21T19:08:33.577726+0000 front 2023-07-21T19:08:33.577687+0000 (oldest deadline 2023-07-21T19:08:59.477398+0000) 2023-07-21T19:09:06.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:06 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:06.406+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:45.542174+0000 front 2023-07-21T19:07:45.542192+0000 (oldest deadline 2023-07-21T19:08:11.442323+0000) 2023-07-21T19:09:06.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:06 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:06.406+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.044189+0000 front 2023-07-21T19:07:56.044156+0000 (oldest deadline 2023-07-21T19:08:18.944205+0000) 2023-07-21T19:09:06.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:06 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:06.406+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:02.345832+0000 front 2023-07-21T19:08:02.345919+0000 (oldest deadline 2023-07-21T19:08:26.445973+0000) 2023-07-21T19:09:06.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:06 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:06.406+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:17.547994+0000 front 2023-07-21T19:08:17.548134+0000 (oldest deadline 2023-07-21T19:08:43.448059+0000) 2023-07-21T19:09:06.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:06 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:06.406+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2023-07-21T19:08:28.149016+0000 front 2023-07-21T19:08:28.149051+0000 (oldest deadline 2023-07-21T19:08:49.849039+0000) 2023-07-21T19:09:06.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:06 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:06.406+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2023-07-21T19:08:35.050771+0000 front 2023-07-21T19:08:35.050572+0000 (oldest deadline 2023-07-21T19:08:57.350446+0000) 2023-07-21T19:09:06.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:06 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:06.406+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6820 osd.7 since back 2023-07-21T19:08:39.051318+0000 front 2023-07-21T19:08:39.051358+0000 (oldest deadline 2023-07-21T19:09:04.351321+0000) 2023-07-21T19:09:07.376 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 21 19:09:07 smithi155 bash[133633]: ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-10 2023-07-21T19:09:07.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:07 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:07.429+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:45.542174+0000 front 2023-07-21T19:07:45.542192+0000 (oldest deadline 2023-07-21T19:08:11.442323+0000) 2023-07-21T19:09:07.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:07 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:07.429+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.044189+0000 front 2023-07-21T19:07:56.044156+0000 (oldest deadline 2023-07-21T19:08:18.944205+0000) 2023-07-21T19:09:07.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:07 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:07.429+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:02.345832+0000 front 2023-07-21T19:08:02.345919+0000 (oldest deadline 2023-07-21T19:08:26.445973+0000) 2023-07-21T19:09:07.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:07 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:07.429+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:17.547994+0000 front 2023-07-21T19:08:17.548134+0000 (oldest deadline 2023-07-21T19:08:43.448059+0000) 2023-07-21T19:09:07.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:07 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:07.429+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2023-07-21T19:08:28.149016+0000 front 2023-07-21T19:08:28.149051+0000 (oldest deadline 2023-07-21T19:08:49.849039+0000) 2023-07-21T19:09:07.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:07 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:07.429+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2023-07-21T19:08:35.050771+0000 front 2023-07-21T19:08:35.050572+0000 (oldest deadline 2023-07-21T19:08:57.350446+0000) 2023-07-21T19:09:07.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:07 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:07.429+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6820 osd.7 since back 2023-07-21T19:08:39.051318+0000 front 2023-07-21T19:08:39.051358+0000 (oldest deadline 2023-07-21T19:09:04.351321+0000) 2023-07-21T19:09:08.437 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@osd.10.service' 2023-07-21T19:09:08.477 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T19:09:08.478 INFO:tasks.cephadm.osd.10:Stopped osd.10 2023-07-21T19:09:08.478 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2023-07-21T19:09:08.478 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@osd.11 2023-07-21T19:09:08.818 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:08 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:08.460+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:45.542174+0000 front 2023-07-21T19:07:45.542192+0000 (oldest deadline 2023-07-21T19:08:11.442323+0000) 2023-07-21T19:09:08.818 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:08 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:08.460+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.044189+0000 front 2023-07-21T19:07:56.044156+0000 (oldest deadline 2023-07-21T19:08:18.944205+0000) 2023-07-21T19:09:08.819 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:08 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:08.460+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:02.345832+0000 front 2023-07-21T19:08:02.345919+0000 (oldest deadline 2023-07-21T19:08:26.445973+0000) 2023-07-21T19:09:08.819 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:08 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:08.460+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:17.547994+0000 front 2023-07-21T19:08:17.548134+0000 (oldest deadline 2023-07-21T19:08:43.448059+0000) 2023-07-21T19:09:08.819 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:08 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:08.460+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2023-07-21T19:08:28.149016+0000 front 2023-07-21T19:08:28.149051+0000 (oldest deadline 2023-07-21T19:08:49.849039+0000) 2023-07-21T19:09:08.819 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:08 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:08.460+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2023-07-21T19:08:35.050771+0000 front 2023-07-21T19:08:35.050572+0000 (oldest deadline 2023-07-21T19:08:57.350446+0000) 2023-07-21T19:09:08.819 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:08 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:08.460+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6820 osd.7 since back 2023-07-21T19:08:39.051318+0000 front 2023-07-21T19:08:39.051358+0000 (oldest deadline 2023-07-21T19:09:04.351321+0000) 2023-07-21T19:09:08.819 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:08 smithi193 systemd[1]: Stopping Ceph osd.11 for eaf8e25a-27f4-11ee-9b34-001a4aab830c... 2023-07-21T19:09:08.819 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:08 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:08.808+0000 7fc95b24b700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T19:09:08.820 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:08 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:08.808+0000 7fc95b24b700 -1 osd.11 127 *** Got signal Terminated *** 2023-07-21T19:09:09.107 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:08 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:08.808+0000 7fc95b24b700 -1 osd.11 127 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-21T19:09:09.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:09 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:09.485+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:45.542174+0000 front 2023-07-21T19:07:45.542192+0000 (oldest deadline 2023-07-21T19:08:11.442323+0000) 2023-07-21T19:09:09.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:09 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:09.485+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.044189+0000 front 2023-07-21T19:07:56.044156+0000 (oldest deadline 2023-07-21T19:08:18.944205+0000) 2023-07-21T19:09:09.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:09 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:09.485+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:02.345832+0000 front 2023-07-21T19:08:02.345919+0000 (oldest deadline 2023-07-21T19:08:26.445973+0000) 2023-07-21T19:09:09.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:09 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:09.485+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:17.547994+0000 front 2023-07-21T19:08:17.548134+0000 (oldest deadline 2023-07-21T19:08:43.448059+0000) 2023-07-21T19:09:09.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:09 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:09.485+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2023-07-21T19:08:28.149016+0000 front 2023-07-21T19:08:28.149051+0000 (oldest deadline 2023-07-21T19:08:49.849039+0000) 2023-07-21T19:09:09.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:09 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:09.485+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2023-07-21T19:08:35.050771+0000 front 2023-07-21T19:08:35.050572+0000 (oldest deadline 2023-07-21T19:08:57.350446+0000) 2023-07-21T19:09:09.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:09 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:09.485+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6820 osd.7 since back 2023-07-21T19:08:39.051318+0000 front 2023-07-21T19:08:39.051358+0000 (oldest deadline 2023-07-21T19:09:04.351321+0000) 2023-07-21T19:09:10.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:10 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:10.465+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:45.542174+0000 front 2023-07-21T19:07:45.542192+0000 (oldest deadline 2023-07-21T19:08:11.442323+0000) 2023-07-21T19:09:10.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:10 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:10.465+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.044189+0000 front 2023-07-21T19:07:56.044156+0000 (oldest deadline 2023-07-21T19:08:18.944205+0000) 2023-07-21T19:09:10.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:10 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:10.465+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:02.345832+0000 front 2023-07-21T19:08:02.345919+0000 (oldest deadline 2023-07-21T19:08:26.445973+0000) 2023-07-21T19:09:10.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:10 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:10.465+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:17.547994+0000 front 2023-07-21T19:08:17.548134+0000 (oldest deadline 2023-07-21T19:08:43.448059+0000) 2023-07-21T19:09:10.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:10 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:10.465+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2023-07-21T19:08:28.149016+0000 front 2023-07-21T19:08:28.149051+0000 (oldest deadline 2023-07-21T19:08:49.849039+0000) 2023-07-21T19:09:10.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:10 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:10.465+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2023-07-21T19:08:35.050771+0000 front 2023-07-21T19:08:35.050572+0000 (oldest deadline 2023-07-21T19:08:57.350446+0000) 2023-07-21T19:09:10.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:10 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:10.465+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6820 osd.7 since back 2023-07-21T19:08:39.051318+0000 front 2023-07-21T19:08:39.051358+0000 (oldest deadline 2023-07-21T19:09:04.351321+0000) 2023-07-21T19:09:11.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:11 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:11.481+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:45.542174+0000 front 2023-07-21T19:07:45.542192+0000 (oldest deadline 2023-07-21T19:08:11.442323+0000) 2023-07-21T19:09:11.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:11 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:11.481+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.044189+0000 front 2023-07-21T19:07:56.044156+0000 (oldest deadline 2023-07-21T19:08:18.944205+0000) 2023-07-21T19:09:11.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:11 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:11.481+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:02.345832+0000 front 2023-07-21T19:08:02.345919+0000 (oldest deadline 2023-07-21T19:08:26.445973+0000) 2023-07-21T19:09:11.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:11 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:11.481+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:17.547994+0000 front 2023-07-21T19:08:17.548134+0000 (oldest deadline 2023-07-21T19:08:43.448059+0000) 2023-07-21T19:09:11.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:11 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:11.481+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2023-07-21T19:08:28.149016+0000 front 2023-07-21T19:08:28.149051+0000 (oldest deadline 2023-07-21T19:08:49.849039+0000) 2023-07-21T19:09:11.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:11 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:11.481+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2023-07-21T19:08:35.050771+0000 front 2023-07-21T19:08:35.050572+0000 (oldest deadline 2023-07-21T19:08:57.350446+0000) 2023-07-21T19:09:11.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:11 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:11.481+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6820 osd.7 since back 2023-07-21T19:08:39.051318+0000 front 2023-07-21T19:08:39.051358+0000 (oldest deadline 2023-07-21T19:09:04.351321+0000) 2023-07-21T19:09:12.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:12 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:12.509+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:45.542174+0000 front 2023-07-21T19:07:45.542192+0000 (oldest deadline 2023-07-21T19:08:11.442323+0000) 2023-07-21T19:09:12.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:12 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:12.509+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.044189+0000 front 2023-07-21T19:07:56.044156+0000 (oldest deadline 2023-07-21T19:08:18.944205+0000) 2023-07-21T19:09:12.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:12 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:12.509+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:02.345832+0000 front 2023-07-21T19:08:02.345919+0000 (oldest deadline 2023-07-21T19:08:26.445973+0000) 2023-07-21T19:09:12.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:12 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:12.509+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:17.547994+0000 front 2023-07-21T19:08:17.548134+0000 (oldest deadline 2023-07-21T19:08:43.448059+0000) 2023-07-21T19:09:12.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:12 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:12.509+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2023-07-21T19:08:28.149016+0000 front 2023-07-21T19:08:28.149051+0000 (oldest deadline 2023-07-21T19:08:49.849039+0000) 2023-07-21T19:09:12.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:12 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:12.509+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2023-07-21T19:08:35.050771+0000 front 2023-07-21T19:08:35.050572+0000 (oldest deadline 2023-07-21T19:08:57.350446+0000) 2023-07-21T19:09:12.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:12 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:12.509+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6820 osd.7 since back 2023-07-21T19:08:39.051318+0000 front 2023-07-21T19:08:39.051358+0000 (oldest deadline 2023-07-21T19:09:04.351321+0000) 2023-07-21T19:09:13.857 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:13 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:13.543+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-07-21T19:07:45.542174+0000 front 2023-07-21T19:07:45.542192+0000 (oldest deadline 2023-07-21T19:08:11.442323+0000) 2023-07-21T19:09:13.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:13 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:13.543+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-21T19:07:56.044189+0000 front 2023-07-21T19:07:56.044156+0000 (oldest deadline 2023-07-21T19:08:18.944205+0000) 2023-07-21T19:09:13.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:13 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:13.543+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2023-07-21T19:08:02.345832+0000 front 2023-07-21T19:08:02.345919+0000 (oldest deadline 2023-07-21T19:08:26.445973+0000) 2023-07-21T19:09:13.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:13 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:13.543+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-21T19:08:17.547994+0000 front 2023-07-21T19:08:17.548134+0000 (oldest deadline 2023-07-21T19:08:43.448059+0000) 2023-07-21T19:09:13.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:13 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:13.543+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2023-07-21T19:08:28.149016+0000 front 2023-07-21T19:08:28.149051+0000 (oldest deadline 2023-07-21T19:08:49.849039+0000) 2023-07-21T19:09:13.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:13 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:13.543+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2023-07-21T19:08:35.050771+0000 front 2023-07-21T19:08:35.050572+0000 (oldest deadline 2023-07-21T19:08:57.350446+0000) 2023-07-21T19:09:13.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:13 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:13.543+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.155:6820 osd.7 since back 2023-07-21T19:08:39.051318+0000 front 2023-07-21T19:08:39.051358+0000 (oldest deadline 2023-07-21T19:09:04.351321+0000) 2023-07-21T19:09:13.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:13 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11[120632]: 2023-07-21T19:09:13.543+0000 7fc95603e700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.193:6820 osd.8 since back 2023-07-21T19:08:47.753023+0000 front 2023-07-21T19:08:47.752980+0000 (oldest deadline 2023-07-21T19:09:13.053021+0000) 2023-07-21T19:09:14.858 INFO:journalctl@ceph.osd.11.smithi193.stdout:Jul 21 19:09:14 smithi193 bash[137640]: ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-osd-11 2023-07-21T19:09:15.896 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@osd.11.service' 2023-07-21T19:09:15.935 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T19:09:15.936 INFO:tasks.cephadm.osd.11:Stopped osd.11 2023-07-21T19:09:15.936 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2023-07-21T19:09:15.937 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@mds.a 2023-07-21T19:09:16.299 INFO:journalctl@ceph.mds.a.smithi039.stdout:Jul 21 19:09:15 smithi039 systemd[1]: Stopping Ceph mds.a for eaf8e25a-27f4-11ee-9b34-001a4aab830c... 2023-07-21T19:09:16.675 INFO:journalctl@ceph.mds.a.smithi039.stdout:Jul 21 19:09:16 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-a[131175]: 2023-07-21T19:09:16.288+0000 7fb0635c6700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T19:09:16.675 INFO:journalctl@ceph.mds.a.smithi039.stdout:Jul 21 19:09:16 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-a[131175]: 2023-07-21T19:09:16.288+0000 7fb0635c6700 -1 mds.a *** got signal Terminated *** 2023-07-21T19:09:19.270 INFO:journalctl@ceph.mds.a.smithi039.stdout:Jul 21 19:09:19 smithi039 bash[164323]: ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-a 2023-07-21T19:09:19.466 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@mds.a.service' 2023-07-21T19:09:19.504 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T19:09:19.505 INFO:tasks.cephadm.mds.a:Stopped mds.a 2023-07-21T19:09:19.505 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2023-07-21T19:09:19.506 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@mds.d 2023-07-21T19:09:19.882 INFO:journalctl@ceph.mds.d.smithi039.stdout:Jul 21 19:09:19 smithi039 systemd[1]: Stopping Ceph mds.d for eaf8e25a-27f4-11ee-9b34-001a4aab830c... 2023-07-21T19:09:20.175 INFO:journalctl@ceph.mds.d.smithi039.stdout:Jul 21 19:09:19 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-d[130187]: 2023-07-21T19:09:19.871+0000 7f952718d700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T19:09:20.175 INFO:journalctl@ceph.mds.d.smithi039.stdout:Jul 21 19:09:19 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-d[130187]: 2023-07-21T19:09:19.871+0000 7f952718d700 -1 mds.d *** got signal Terminated *** 2023-07-21T19:09:24.408 INFO:journalctl@ceph.mds.d.smithi039.stdout:Jul 21 19:09:24 smithi039 bash[164463]: ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-d 2023-07-21T19:09:24.413 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@mds.d.service' 2023-07-21T19:09:24.449 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T19:09:24.450 INFO:tasks.cephadm.mds.d:Stopped mds.d 2023-07-21T19:09:24.450 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2023-07-21T19:09:24.450 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@mds.g 2023-07-21T19:09:24.675 INFO:journalctl@ceph.mds.g.smithi039.stdout:Jul 21 19:09:24 smithi039 systemd[1]: Stopping Ceph mds.g for eaf8e25a-27f4-11ee-9b34-001a4aab830c... 2023-07-21T19:09:25.175 INFO:journalctl@ceph.mds.g.smithi039.stdout:Jul 21 19:09:24 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-g[130792]: 2023-07-21T19:09:24.820+0000 7f5a02ff4700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T19:09:25.175 INFO:journalctl@ceph.mds.g.smithi039.stdout:Jul 21 19:09:24 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-g[130792]: 2023-07-21T19:09:24.820+0000 7f5a02ff4700 -1 mds.g *** got signal Terminated *** 2023-07-21T19:09:27.294 INFO:journalctl@ceph.mds.g.smithi039.stdout:Jul 21 19:09:27 smithi039 bash[164607]: ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-g 2023-07-21T19:09:27.488 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@mds.g.service' 2023-07-21T19:09:27.522 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T19:09:27.523 INFO:tasks.cephadm.mds.g:Stopped mds.g 2023-07-21T19:09:27.523 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2023-07-21T19:09:27.523 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@mds.j 2023-07-21T19:09:27.905 INFO:journalctl@ceph.mds.j.smithi039.stdout:Jul 21 19:09:27 smithi039 systemd[1]: Stopping Ceph mds.j for eaf8e25a-27f4-11ee-9b34-001a4aab830c... 2023-07-21T19:09:28.175 INFO:journalctl@ceph.mds.j.smithi039.stdout:Jul 21 19:09:27 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-j[129799]: 2023-07-21T19:09:27.895+0000 7fb622fa6700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T19:09:28.175 INFO:journalctl@ceph.mds.j.smithi039.stdout:Jul 21 19:09:27 smithi039 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-j[129799]: 2023-07-21T19:09:27.895+0000 7fb622fa6700 -1 mds.j *** got signal Terminated *** 2023-07-21T19:09:31.284 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@mds.j.service' 2023-07-21T19:09:31.309 INFO:journalctl@ceph.mds.j.smithi039.stdout:Jul 21 19:09:31 smithi039 bash[164744]: ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-j 2023-07-21T19:09:31.309 INFO:journalctl@ceph.mds.j.smithi039.stdout:Jul 21 19:09:31 smithi039 systemd[1]: ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@mds.j.service: Succeeded. 2023-07-21T19:09:31.309 INFO:journalctl@ceph.mds.j.smithi039.stdout:Jul 21 19:09:31 smithi039 systemd[1]: Stopped Ceph mds.j for eaf8e25a-27f4-11ee-9b34-001a4aab830c. 2023-07-21T19:09:31.319 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T19:09:31.319 INFO:tasks.cephadm.mds.j:Stopped mds.j 2023-07-21T19:09:31.320 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2023-07-21T19:09:31.320 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@mds.b 2023-07-21T19:09:31.626 INFO:journalctl@ceph.mds.b.smithi155.stdout:Jul 21 19:09:31 smithi155 systemd[1]: Stopping Ceph mds.b for eaf8e25a-27f4-11ee-9b34-001a4aab830c... 2023-07-21T19:09:32.126 INFO:journalctl@ceph.mds.b.smithi155.stdout:Jul 21 19:09:31 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-b[123270]: 2023-07-21T19:09:31.676+0000 7f1ea408d700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T19:09:32.126 INFO:journalctl@ceph.mds.b.smithi155.stdout:Jul 21 19:09:31 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-b[123270]: 2023-07-21T19:09:31.676+0000 7f1ea408d700 -1 mds.b *** got signal Terminated *** 2023-07-21T19:09:35.517 INFO:journalctl@ceph.mds.b.smithi155.stdout:Jul 21 19:09:35 smithi155 bash[133904]: ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-b 2023-07-21T19:09:35.522 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@mds.b.service' 2023-07-21T19:09:35.559 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T19:09:35.560 INFO:tasks.cephadm.mds.b:Stopped mds.b 2023-07-21T19:09:35.560 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2023-07-21T19:09:35.560 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@mds.e 2023-07-21T19:09:35.876 INFO:journalctl@ceph.mds.e.smithi155.stdout:Jul 21 19:09:35 smithi155 systemd[1]: Stopping Ceph mds.e for eaf8e25a-27f4-11ee-9b34-001a4aab830c... 2023-07-21T19:09:36.376 INFO:journalctl@ceph.mds.e.smithi155.stdout:Jul 21 19:09:35 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-e[122591]: 2023-07-21T19:09:35.917+0000 7f50bb7f5700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T19:09:36.376 INFO:journalctl@ceph.mds.e.smithi155.stdout:Jul 21 19:09:35 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-e[122591]: 2023-07-21T19:09:35.917+0000 7f50bb7f5700 -1 mds.e *** got signal Terminated *** 2023-07-21T19:09:39.307 INFO:journalctl@ceph.mds.e.smithi155.stdout:Jul 21 19:09:39 smithi155 bash[134060]: ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-e 2023-07-21T19:09:39.499 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@mds.e.service' 2023-07-21T19:09:39.534 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T19:09:39.535 INFO:tasks.cephadm.mds.e:Stopped mds.e 2023-07-21T19:09:39.535 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2023-07-21T19:09:39.535 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@mds.h 2023-07-21T19:09:39.876 INFO:journalctl@ceph.mds.h.smithi155.stdout:Jul 21 19:09:39 smithi155 systemd[1]: Stopping Ceph mds.h for eaf8e25a-27f4-11ee-9b34-001a4aab830c... 2023-07-21T19:09:40.376 INFO:journalctl@ceph.mds.h.smithi155.stdout:Jul 21 19:09:39 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-h[122931]: 2023-07-21T19:09:39.891+0000 7fa8a285a700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T19:09:40.376 INFO:journalctl@ceph.mds.h.smithi155.stdout:Jul 21 19:09:39 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-h[122931]: 2023-07-21T19:09:39.891+0000 7fa8a285a700 -1 mds.h *** got signal Terminated *** 2023-07-21T19:09:43.535 INFO:journalctl@ceph.mds.h.smithi155.stdout:Jul 21 19:09:43 smithi155 bash[134201]: ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-h 2023-07-21T19:09:43.539 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@mds.h.service' 2023-07-21T19:09:43.573 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T19:09:43.574 INFO:tasks.cephadm.mds.h:Stopped mds.h 2023-07-21T19:09:43.574 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2023-07-21T19:09:43.575 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@mds.k 2023-07-21T19:09:43.876 INFO:journalctl@ceph.mds.k.smithi155.stdout:Jul 21 19:09:43 smithi155 systemd[1]: Stopping Ceph mds.k for eaf8e25a-27f4-11ee-9b34-001a4aab830c... 2023-07-21T19:09:44.376 INFO:journalctl@ceph.mds.k.smithi155.stdout:Jul 21 19:09:43 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-k[122248]: 2023-07-21T19:09:43.923+0000 7fd228830700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T19:09:44.376 INFO:journalctl@ceph.mds.k.smithi155.stdout:Jul 21 19:09:43 smithi155 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-k[122248]: 2023-07-21T19:09:43.923+0000 7fd228830700 -1 mds.k *** got signal Terminated *** 2023-07-21T19:09:47.376 INFO:journalctl@ceph.mds.k.smithi155.stdout:Jul 21 19:09:47 smithi155 bash[134348]: ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-k 2023-07-21T19:09:47.439 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@mds.k.service' 2023-07-21T19:09:47.472 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T19:09:47.472 INFO:tasks.cephadm.mds.k:Stopped mds.k 2023-07-21T19:09:47.472 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2023-07-21T19:09:47.473 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@mds.c 2023-07-21T19:09:47.828 INFO:journalctl@ceph.mds.c.smithi193.stdout:Jul 21 19:09:47 smithi193 systemd[1]: Stopping Ceph mds.c for eaf8e25a-27f4-11ee-9b34-001a4aab830c... 2023-07-21T19:09:48.107 INFO:journalctl@ceph.mds.c.smithi193.stdout:Jul 21 19:09:47 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-c[124252]: 2023-07-21T19:09:47.818+0000 7f0f1b8d1700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T19:09:48.107 INFO:journalctl@ceph.mds.c.smithi193.stdout:Jul 21 19:09:47 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-c[124252]: 2023-07-21T19:09:47.818+0000 7f0f1b8d1700 -1 mds.c *** got signal Terminated *** 2023-07-21T19:09:51.517 INFO:journalctl@ceph.mds.c.smithi193.stdout:Jul 21 19:09:51 smithi193 bash[137924]: ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-c 2023-07-21T19:09:51.523 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@mds.c.service' 2023-07-21T19:09:51.560 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T19:09:51.561 INFO:tasks.cephadm.mds.c:Stopped mds.c 2023-07-21T19:09:51.561 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2023-07-21T19:09:51.561 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@mds.f 2023-07-21T19:09:51.857 INFO:journalctl@ceph.mds.f.smithi193.stdout:Jul 21 19:09:51 smithi193 systemd[1]: Stopping Ceph mds.f for eaf8e25a-27f4-11ee-9b34-001a4aab830c... 2023-07-21T19:09:52.357 INFO:journalctl@ceph.mds.f.smithi193.stdout:Jul 21 19:09:51 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-f[123905]: 2023-07-21T19:09:51.905+0000 7f115d6b4700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T19:09:52.357 INFO:journalctl@ceph.mds.f.smithi193.stdout:Jul 21 19:09:51 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-f[123905]: 2023-07-21T19:09:51.905+0000 7f115d6b4700 -1 mds.f *** got signal Terminated *** 2023-07-21T19:09:55.357 INFO:journalctl@ceph.mds.f.smithi193.stdout:Jul 21 19:09:55 smithi193 bash[138067]: ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-f 2023-07-21T19:09:55.498 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@mds.f.service' 2023-07-21T19:09:55.532 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T19:09:55.533 INFO:tasks.cephadm.mds.f:Stopped mds.f 2023-07-21T19:09:55.533 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2023-07-21T19:09:55.533 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@mds.i 2023-07-21T19:09:55.857 INFO:journalctl@ceph.mds.i.smithi193.stdout:Jul 21 19:09:55 smithi193 systemd[1]: Stopping Ceph mds.i for eaf8e25a-27f4-11ee-9b34-001a4aab830c... 2023-07-21T19:09:56.357 INFO:journalctl@ceph.mds.i.smithi193.stdout:Jul 21 19:09:55 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-i[124588]: 2023-07-21T19:09:55.908+0000 7fd5413fd700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T19:09:56.357 INFO:journalctl@ceph.mds.i.smithi193.stdout:Jul 21 19:09:55 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-i[124588]: 2023-07-21T19:09:55.908+0000 7fd5413fd700 -1 mds.i *** got signal Terminated *** 2023-07-21T19:09:59.559 INFO:journalctl@ceph.mds.i.smithi193.stdout:Jul 21 19:09:59 smithi193 bash[138209]: ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-i 2023-07-21T19:09:59.565 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@mds.i.service' 2023-07-21T19:09:59.598 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T19:09:59.598 INFO:tasks.cephadm.mds.i:Stopped mds.i 2023-07-21T19:09:59.599 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2023-07-21T19:09:59.599 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@mds.l 2023-07-21T19:09:59.857 INFO:journalctl@ceph.mds.l.smithi193.stdout:Jul 21 19:09:59 smithi193 systemd[1]: Stopping Ceph mds.l for eaf8e25a-27f4-11ee-9b34-001a4aab830c... 2023-07-21T19:10:00.357 INFO:journalctl@ceph.mds.l.smithi193.stdout:Jul 21 19:09:59 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-l[123564]: 2023-07-21T19:09:59.983+0000 7f2e46a71700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T19:10:00.357 INFO:journalctl@ceph.mds.l.smithi193.stdout:Jul 21 19:09:59 smithi193 ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-l[123564]: 2023-07-21T19:09:59.983+0000 7f2e46a71700 -1 mds.l *** got signal Terminated *** 2023-07-21T19:10:03.538 INFO:journalctl@ceph.mds.l.smithi193.stdout:Jul 21 19:10:03 smithi193 bash[138354]: ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c-mds-l 2023-07-21T19:10:03.545 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eaf8e25a-27f4-11ee-9b34-001a4aab830c@mds.l.service' 2023-07-21T19:10:03.581 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T19:10:03.582 INFO:tasks.cephadm.mds.l:Stopped mds.l 2023-07-21T19:10:03.582 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c --force --keep-logs 2023-07-21T19:10:05.598 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c --force --keep-logs 2023-07-21T19:10:07.514 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c --force --keep-logs 2023-07-21T19:10:09.523 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-21T19:10:09.608 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-21T19:10:09.645 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-21T19:10:09.674 INFO:tasks.cephadm:Archiving crash dumps... 2023-07-21T19:10:09.688 DEBUG:teuthology.misc:Transferring archived files from smithi039:/var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/crash to /home/teuthworker/archive/rishabh-2023-07-21_17:58:19-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7346865/remote/smithi039/crash 2023-07-21T19:10:09.689 DEBUG:teuthology.orchestra.run.smithi039:> sudo tar c -f - -C /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/crash -- . 2023-07-21T19:10:09.742 INFO:teuthology.orchestra.run.smithi039.stderr:tar: /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/crash: Cannot open: No such file or directory 2023-07-21T19:10:09.743 INFO:teuthology.orchestra.run.smithi039.stderr:tar: Error is not recoverable: exiting now 2023-07-21T19:10:09.744 DEBUG:teuthology.misc:Transferring archived files from smithi155:/var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/crash to /home/teuthworker/archive/rishabh-2023-07-21_17:58:19-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7346865/remote/smithi155/crash 2023-07-21T19:10:09.746 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar c -f - -C /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/crash -- . 2023-07-21T19:10:09.800 INFO:teuthology.orchestra.run.smithi155.stderr:tar: /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/crash: Cannot open: No such file or directory 2023-07-21T19:10:09.800 INFO:teuthology.orchestra.run.smithi155.stderr:tar: Error is not recoverable: exiting now 2023-07-21T19:10:09.802 DEBUG:teuthology.misc:Transferring archived files from smithi193:/var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/crash to /home/teuthworker/archive/rishabh-2023-07-21_17:58:19-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7346865/remote/smithi193/crash 2023-07-21T19:10:09.803 DEBUG:teuthology.orchestra.run.smithi193:> sudo tar c -f - -C /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/crash -- . 2023-07-21T19:10:09.878 INFO:teuthology.orchestra.run.smithi193.stderr:tar: /var/lib/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/crash: Cannot open: No such file or directory 2023-07-21T19:10:09.879 INFO:teuthology.orchestra.run.smithi193.stderr:tar: Error is not recoverable: exiting now 2023-07-21T19:10:09.880 INFO:tasks.cephadm:Checking cluster log for badness... 2023-07-21T19:10:09.880 DEBUG:teuthology.orchestra.run.smithi039:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2023-07-21T19:10:09.920 INFO:teuthology.orchestra.run.smithi039.stderr:grep: /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph.log: No such file or directory 2023-07-21T19:10:09.926 INFO:tasks.cephadm:Compressing logs... 2023-07-21T19:10:09.927 DEBUG:teuthology.orchestra.run.smithi039:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-07-21T19:10:09.968 DEBUG:teuthology.orchestra.run.smithi155:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-07-21T19:10:09.971 DEBUG:teuthology.orchestra.run.smithi193:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-07-21T19:10:10.001 INFO:teuthology.orchestra.run.smithi039.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-07-21T19:10:10.001 INFO:teuthology.orchestra.run.smithi039.stderr:: No such file or directory 2023-07-21T19:10:10.004 INFO:teuthology.orchestra.run.smithi193.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-07-21T19:10:10.004 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-07-21T19:10:10.005 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-osd.0.log 2023-07-21T19:10:10.006 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-mds.g.log 2023-07-21T19:10:10.007 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-mds.a.log 2023-07-21T19:10:10.007 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-osd.0.log: /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-osd.6.log 2023-07-21T19:10:10.008 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-osd.3.log 2023-07-21T19:10:10.008 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-mds.d.log 2023-07-21T19:10:10.009 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-mgr.x.log 2023-07-21T19:10:10.009 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-osd.9.log 2023-07-21T19:10:10.009 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-volume.log 2023-07-21T19:10:10.009 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-mds.j.log 2023-07-21T19:10:10.011 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-mon.a.log 2023-07-21T19:10:10.011 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-osd.5.log 2023-07-21T19:10:10.012 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-osd.2.log 2023-07-21T19:10:10.012 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-osd.11.log 2023-07-21T19:10:10.012 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-mgr.z.log 2023-07-21T19:10:10.012 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-osd.8.log 2023-07-21T19:10:10.014 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-mds.f.log 2023-07-21T19:10:10.014 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-mds.c.log 2023-07-21T19:10:10.014 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-mds.i.log 2023-07-21T19:10:10.014 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-mon.c.log 2023-07-21T19:10:10.014 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-mds.c.log: /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-volume.log 2023-07-21T19:10:10.015 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-mds.l.log 2023-07-21T19:10:10.015 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-volume.log: /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-mds.j.log: /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-mon.a.log: 94.5% -- replaced with /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-mds.d.log.gz 2023-07-21T19:10:10.015 INFO:teuthology.orchestra.run.smithi039.stderr: 94.6% -- replaced with /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-mds.j.log.gz 2023-07-21T19:10:10.017 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-volume.log: /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-mds.l.log: 92.6% -- replaced with /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-mgr.z.log.gz 2023-07-21T19:10:10.027 INFO:teuthology.orchestra.run.smithi155.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-07-21T19:10:10.027 INFO:teuthology.orchestra.run.smithi155.stderr:: No such file or directory 2023-07-21T19:10:10.035 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-osd.10.log 2023-07-21T19:10:10.036 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-mds.h.log 2023-07-21T19:10:10.037 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-mds.e.log 2023-07-21T19:10:10.037 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-osd.4.log 2023-07-21T19:10:10.037 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-osd.1.log 2023-07-21T19:10:10.038 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-mds.k.log 2023-07-21T19:10:10.038 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-volume.log 2023-07-21T19:10:10.038 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-osd.7.log 2023-07-21T19:10:10.039 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-mon.b.log 2023-07-21T19:10:10.039 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-mds.b.log 2023-07-21T19:10:10.039 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-mgr.y.log 2023-07-21T19:10:10.042 INFO:teuthology.orchestra.run.smithi039.stderr: 93.4% -- replaced with /var/log/ceph/cephadm.log.gz 2023-07-21T19:10:10.046 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-mds.b.log: /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-mgr.y.log: 92.5% -- replaced with /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-mgr.y.log.gz 2023-07-21T19:10:10.435 INFO:teuthology.orchestra.run.smithi039.stderr: 91.6% -- replaced with /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-volume.log.gz 2023-07-21T19:10:10.466 INFO:teuthology.orchestra.run.smithi155.stderr: 91.9% -- replaced with /var/log/ceph/cephadm.log.gz 2023-07-21T19:10:10.552 INFO:teuthology.orchestra.run.smithi039.stderr: 89.6% -- replaced with /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-mgr.x.log.gz 2023-07-21T19:10:10.852 INFO:teuthology.orchestra.run.smithi155.stderr: 91.5% -- replaced with /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-volume.log.gz 2023-07-21T19:10:11.467 INFO:teuthology.orchestra.run.smithi193.stderr: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2023-07-21T19:10:11.518 INFO:teuthology.orchestra.run.smithi193.stderr: 91.5% -- replaced with /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-volume.log.gz 2023-07-21T19:10:13.114 INFO:teuthology.orchestra.run.smithi193.stderr: 91.9% -- replaced with /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-mon.c.log.gz 2023-07-21T19:10:14.935 INFO:teuthology.orchestra.run.smithi155.stderr: 91.9% -- replaced with /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-mon.b.log.gz 2023-07-21T19:10:18.232 INFO:teuthology.orchestra.run.smithi039.stderr: 92.4% -- replaced with /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-mon.a.log.gz 2023-07-21T19:10:29.664 INFO:teuthology.orchestra.run.smithi039.stderr: 94.9% -- replaced with /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-mds.g.log.gz 2023-07-21T19:10:35.288 INFO:teuthology.orchestra.run.smithi193.stderr: 95.1% -- replaced with /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-mds.l.log.gz 2023-07-21T19:10:38.049 INFO:teuthology.orchestra.run.smithi193.stderr: 95.1% -- replaced with /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-mds.c.log.gz 2023-07-21T19:10:38.869 INFO:teuthology.orchestra.run.smithi193.stderr: 95.0% -- replaced with /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-mds.f.log.gz 2023-07-21T19:10:49.405 INFO:teuthology.orchestra.run.smithi039.stderr: 93.8% -- replaced with /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-osd.0.log.gz 2023-07-21T19:10:50.939 INFO:teuthology.orchestra.run.smithi039.stderr: 93.9% -- replaced with /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-osd.9.log.gz 2023-07-21T19:10:55.252 INFO:teuthology.orchestra.run.smithi039.stderr: 93.8% -- replaced with /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-osd.6.log.gz 2023-07-21T19:10:55.755 INFO:teuthology.orchestra.run.smithi039.stderr: 93.7% -- replaced with /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-osd.3.log.gz 2023-07-21T19:10:56.923 INFO:teuthology.orchestra.run.smithi193.stderr: 93.7% -- replaced with /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-osd.11.log.gz 2023-07-21T19:11:01.790 INFO:teuthology.orchestra.run.smithi155.stderr: 95.3% -- replaced with /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-mds.k.log.gz 2023-07-21T19:11:07.337 INFO:teuthology.orchestra.run.smithi193.stderr: 93.8% -- replaced with /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-osd.2.log.gz 2023-07-21T19:11:09.290 INFO:teuthology.orchestra.run.smithi193.stderr: 93.9% -- replaced with /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-osd.5.log.gz 2023-07-21T19:11:10.145 INFO:teuthology.orchestra.run.smithi193.stderr: 93.9% -- replaced with /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-osd.8.log.gz 2023-07-21T19:11:32.137 INFO:teuthology.orchestra.run.smithi155.stderr: 94.0% -- replaced with /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-osd.4.log.gz 2023-07-21T19:11:40.948 INFO:teuthology.orchestra.run.smithi155.stderr: 93.7% -- replaced with /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-osd.1.log.gz 2023-07-21T19:11:50.706 INFO:teuthology.orchestra.run.smithi155.stderr: 93.8% -- replaced with /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-osd.10.log.gz 2023-07-21T19:12:03.110 INFO:teuthology.orchestra.run.smithi155.stderr: 93.8% -- replaced with /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-osd.7.log.gz 2023-07-21T19:13:40.059 INFO:teuthology.orchestra.run.smithi039.stderr: 92.9% -- replaced with /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-mds.a.log.gz 2023-07-21T19:13:40.061 INFO:teuthology.orchestra.run.smithi039.stderr: 2023-07-21T19:13:40.061 INFO:teuthology.orchestra.run.smithi039.stderr:real 3m30.083s 2023-07-21T19:13:40.062 INFO:teuthology.orchestra.run.smithi039.stderr:user 4m37.822s 2023-07-21T19:13:40.062 INFO:teuthology.orchestra.run.smithi039.stderr:sys 0m9.898s 2023-07-21T19:13:50.870 INFO:teuthology.orchestra.run.smithi193.stderr: 93.1% -- replaced with /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-mds.i.log.gz 2023-07-21T19:13:50.873 INFO:teuthology.orchestra.run.smithi193.stderr: 2023-07-21T19:13:50.873 INFO:teuthology.orchestra.run.smithi193.stderr:real 3m40.893s 2023-07-21T19:13:50.874 INFO:teuthology.orchestra.run.smithi193.stderr:user 4m46.953s 2023-07-21T19:13:50.874 INFO:teuthology.orchestra.run.smithi193.stderr:sys 0m11.884s 2023-07-21T19:18:19.786 INFO:teuthology.orchestra.run.smithi155.stderr: 92.9% -- replaced with /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-mds.e.log.gz 2023-07-21T19:18:43.817 INFO:teuthology.orchestra.run.smithi155.stderr: 93.0% -- replaced with /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-mds.h.log.gz 2023-07-21T19:19:55.109 INFO:teuthology.orchestra.run.smithi155.stderr: 93.2% -- replaced with /var/log/ceph/eaf8e25a-27f4-11ee-9b34-001a4aab830c/ceph-mds.b.log.gz 2023-07-21T19:19:55.113 INFO:teuthology.orchestra.run.smithi155.stderr: 2023-07-21T19:19:55.113 INFO:teuthology.orchestra.run.smithi155.stderr:real 9m45.141s 2023-07-21T19:19:55.114 INFO:teuthology.orchestra.run.smithi155.stderr:user 11m38.362s 2023-07-21T19:19:55.114 INFO:teuthology.orchestra.run.smithi155.stderr:sys 0m28.702s 2023-07-21T19:19:55.114 INFO:tasks.cephadm:Archiving logs... 2023-07-21T19:19:55.115 DEBUG:teuthology.misc:Transferring archived files from smithi039:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-07-21_17:58:19-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7346865/remote/smithi039/log 2023-07-21T19:19:55.116 DEBUG:teuthology.orchestra.run.smithi039:> sudo tar c -f - -C /var/log/ceph -- . 2023-07-21T19:20:26.598 DEBUG:teuthology.misc:Transferring archived files from smithi155:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-07-21_17:58:19-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7346865/remote/smithi155/log 2023-07-21T19:20:26.599 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar c -f - -C /var/log/ceph -- . 2023-07-21T19:21:53.450 DEBUG:teuthology.misc:Transferring archived files from smithi193:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-07-21_17:58:19-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7346865/remote/smithi193/log 2023-07-21T19:21:53.452 DEBUG:teuthology.orchestra.run.smithi193:> sudo tar c -f - -C /var/log/ceph -- . 2023-07-21T19:22:28.950 INFO:tasks.cephadm:Removing cluster... 2023-07-21T19:22:28.951 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c --force 2023-07-21T19:22:30.059 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c --force 2023-07-21T19:22:31.735 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid eaf8e25a-27f4-11ee-9b34-001a4aab830c --force 2023-07-21T19:22:32.843 INFO:tasks.cephadm:Removing cephadm ... 2023-07-21T19:22:32.843 DEBUG:teuthology.orchestra.run.smithi039:> rm -rf /home/ubuntu/cephtest/cephadm 2023-07-21T19:22:32.865 DEBUG:teuthology.orchestra.run.smithi155:> rm -rf /home/ubuntu/cephtest/cephadm 2023-07-21T19:22:32.895 DEBUG:teuthology.orchestra.run.smithi193:> rm -rf /home/ubuntu/cephtest/cephadm 2023-07-21T19:22:32.918 INFO:tasks.cephadm:Teardown complete 2023-07-21T19:22:32.919 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-07-21T19:22:32.930 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-07-21T19:22:32.931 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-07-21T19:22:32.933 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-07-21T19:22:32.938 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-07-21T19:22:33.093 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-07-21T19:22:33.104 INFO:teuthology.task.clock:Checking final clock skew... 2023-07-21T19:22:33.104 DEBUG:teuthology.orchestra.run.smithi039:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-21T19:22:33.107 DEBUG:teuthology.orchestra.run.smithi155:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-21T19:22:33.110 DEBUG:teuthology.orchestra.run.smithi193:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-21T19:22:33.124 INFO:teuthology.orchestra.run.smithi039.stderr:bash: ntpq: command not found 2023-07-21T19:22:33.125 INFO:teuthology.orchestra.run.smithi193.stderr:bash: ntpq: command not found 2023-07-21T19:22:33.126 INFO:teuthology.orchestra.run.smithi155.stderr:bash: ntpq: command not found 2023-07-21T19:22:33.129 INFO:teuthology.orchestra.run.smithi039.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-21T19:22:33.129 INFO:teuthology.orchestra.run.smithi039.stdout:=============================================================================== 2023-07-21T19:22:33.129 INFO:teuthology.orchestra.run.smithi039.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 28 +3226us[+3207us] +/- 89ms 2023-07-21T19:22:33.129 INFO:teuthology.orchestra.run.smithi039.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 26 -1400us[-1400us] +/- 75ms 2023-07-21T19:22:33.129 INFO:teuthology.orchestra.run.smithi039.stdout:^* hv03.front.sepia.ceph.com 3 6 377 28 -1132us[-1151us] +/- 70ms 2023-07-21T19:22:33.129 INFO:teuthology.orchestra.run.smithi039.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T19:22:33.139 INFO:teuthology.orchestra.run.smithi155.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-21T19:22:33.139 INFO:teuthology.orchestra.run.smithi155.stdout:=============================================================================== 2023-07-21T19:22:33.139 INFO:teuthology.orchestra.run.smithi155.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 30 +3149us[+3176us] +/- 89ms 2023-07-21T19:22:33.140 INFO:teuthology.orchestra.run.smithi155.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 22 -1414us[-1414us] +/- 75ms 2023-07-21T19:22:33.140 INFO:teuthology.orchestra.run.smithi155.stdout:^* hv03.front.sepia.ceph.com 3 6 377 28 -1140us[-1112us] +/- 70ms 2023-07-21T19:22:33.140 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T19:22:33.185 INFO:teuthology.orchestra.run.smithi193.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-21T19:22:33.185 INFO:teuthology.orchestra.run.smithi193.stdout:=============================================================================== 2023-07-21T19:22:33.185 INFO:teuthology.orchestra.run.smithi193.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 28 +3195us[+3195us] +/- 89ms 2023-07-21T19:22:33.186 INFO:teuthology.orchestra.run.smithi193.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 30 -1443us[-1472us] +/- 75ms 2023-07-21T19:22:33.186 INFO:teuthology.orchestra.run.smithi193.stdout:^* hv03.front.sepia.ceph.com 3 6 377 28 -1158us[-1187us] +/- 70ms 2023-07-21T19:22:33.186 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T19:22:33.187 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-07-21T19:22:33.197 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-07-21T19:22:33.198 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-07-21T19:22:33.221 DEBUG:teuthology.orchestra.run.smithi039:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-07-21T19:22:33.393 DEBUG:teuthology.orchestra.run.smithi155:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-07-21T19:22:33.615 DEBUG:teuthology.orchestra.run.smithi193:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-07-21T19:22:33.753 DEBUG:teuthology.orchestra.run.smithi039:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-21T19:22:33.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T19:22:33.791 DEBUG:teuthology.orchestra.run.smithi155:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-21T19:22:33.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T19:22:33.829 DEBUG:teuthology.orchestra.run.smithi193:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-21T19:22:33.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T19:22:33.862 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-07-21T19:22:33.873 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-07-21T19:22:33.905 INFO:teuthology.task.internal:Duration was 3777.443857 seconds 2023-07-21T19:22:33.906 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-07-21T19:22:33.939 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-07-21T19:22:33.939 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-07-21T19:22:33.942 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-07-21T19:22:33.944 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-07-21T19:22:34.015 INFO:teuthology.orchestra.run.smithi039.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-21T19:22:34.018 INFO:teuthology.orchestra.run.smithi193.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-21T19:22:34.022 INFO:teuthology.orchestra.run.smithi155.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-21T19:22:34.378 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-07-21T19:22:34.379 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi039.front.sepia.ceph.com 2023-07-21T19:22:34.379 DEBUG:teuthology.orchestra.run.smithi039:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-07-21T19:22:34.406 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi155.front.sepia.ceph.com 2023-07-21T19:22:34.407 DEBUG:teuthology.orchestra.run.smithi155:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-07-21T19:22:34.443 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi193.front.sepia.ceph.com 2023-07-21T19:22:34.443 DEBUG:teuthology.orchestra.run.smithi193:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-07-21T19:22:34.467 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-07-21T19:22:34.467 DEBUG:teuthology.orchestra.run.smithi039:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-07-21T19:22:34.470 DEBUG:teuthology.orchestra.run.smithi155:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-07-21T19:22:34.486 DEBUG:teuthology.orchestra.run.smithi193:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-07-21T19:22:34.563 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-07-21T19:22:34.564 DEBUG:teuthology.orchestra.run.smithi039:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-07-21T19:22:34.566 DEBUG:teuthology.orchestra.run.smithi155:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-07-21T19:22:34.568 DEBUG:teuthology.orchestra.run.smithi193:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-07-21T19:22:34.699 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-07-21T19:22:34.711 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-07-21T19:22:34.712 DEBUG:teuthology.orchestra.run.smithi039:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-07-21T19:22:34.751 DEBUG:teuthology.orchestra.run.smithi155:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-07-21T19:22:34.780 DEBUG:teuthology.orchestra.run.smithi193:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-07-21T19:22:34.812 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-07-21T19:22:34.827 DEBUG:teuthology.orchestra.run.smithi039:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-07-21T19:22:34.830 DEBUG:teuthology.orchestra.run.smithi155:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-07-21T19:22:34.832 DEBUG:teuthology.orchestra.run.smithi193:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-07-21T19:22:34.855 INFO:teuthology.orchestra.run.smithi039.stdout:kernel.core_pattern = core 2023-07-21T19:22:34.868 INFO:teuthology.orchestra.run.smithi155.stdout:kernel.core_pattern = core 2023-07-21T19:22:34.900 INFO:teuthology.orchestra.run.smithi193.stdout:kernel.core_pattern = core 2023-07-21T19:22:34.935 DEBUG:teuthology.orchestra.run.smithi039:> test -e /home/ubuntu/cephtest/archive/coredump 2023-07-21T19:22:34.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T19:22:34.956 DEBUG:teuthology.orchestra.run.smithi155:> test -e /home/ubuntu/cephtest/archive/coredump 2023-07-21T19:22:34.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T19:22:34.972 DEBUG:teuthology.orchestra.run.smithi193:> test -e /home/ubuntu/cephtest/archive/coredump 2023-07-21T19:22:34.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T19:22:34.989 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-07-21T19:22:35.001 INFO:teuthology.task.internal:Transferring archived files... 2023-07-21T19:22:35.001 DEBUG:teuthology.misc:Transferring archived files from smithi039:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-07-21_17:58:19-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7346865/remote/smithi039 2023-07-21T19:22:35.002 DEBUG:teuthology.orchestra.run.smithi039:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-07-21T19:22:35.041 DEBUG:teuthology.misc:Transferring archived files from smithi155:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-07-21_17:58:19-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7346865/remote/smithi155 2023-07-21T19:22:35.041 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-07-21T19:22:35.080 DEBUG:teuthology.misc:Transferring archived files from smithi193:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-07-21_17:58:19-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7346865/remote/smithi193 2023-07-21T19:22:35.081 DEBUG:teuthology.orchestra.run.smithi193:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-07-21T19:22:35.122 INFO:teuthology.task.internal:Removing archive directory... 2023-07-21T19:22:35.122 DEBUG:teuthology.orchestra.run.smithi039:> rm -rf -- /home/ubuntu/cephtest/archive 2023-07-21T19:22:35.124 DEBUG:teuthology.orchestra.run.smithi155:> rm -rf -- /home/ubuntu/cephtest/archive 2023-07-21T19:22:35.127 DEBUG:teuthology.orchestra.run.smithi193:> rm -rf -- /home/ubuntu/cephtest/archive 2023-07-21T19:22:35.172 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-07-21T19:22:35.185 INFO:teuthology.task.internal:Not uploading archives. 2023-07-21T19:22:35.186 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-07-21T19:22:35.212 INFO:teuthology.task.internal:Tidying up after the test... 2023-07-21T19:22:35.212 DEBUG:teuthology.orchestra.run.smithi039:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-07-21T19:22:35.214 DEBUG:teuthology.orchestra.run.smithi155:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-07-21T19:22:35.217 DEBUG:teuthology.orchestra.run.smithi193:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-07-21T19:22:35.229 INFO:teuthology.orchestra.run.smithi039.stdout: 262307 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 21 19:22 /home/ubuntu/cephtest 2023-07-21T19:22:35.231 INFO:teuthology.orchestra.run.smithi155.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 21 19:22 /home/ubuntu/cephtest 2023-07-21T19:22:35.233 INFO:teuthology.orchestra.run.smithi193.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 21 19:22 /home/ubuntu/cephtest 2023-07-21T19:22:35.247 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-07-21T19:22:35.259 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-07-21T19:22:35.332 INFO:teuthology.nuke:Checking targets against current locks 2023-07-21T19:22:35.362 DEBUG:teuthology.nuke:shortname: smithi039 2023-07-21T19:22:35.362 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-21T19:22:35.388 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi039.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-21_17:58:19-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7346865', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-21 18:12:12.109513', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='} 2023-07-21T19:22:35.393 DEBUG:teuthology.nuke:shortname: smithi155 2023-07-21T19:22:35.393 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-21T19:22:35.420 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi155.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-21_17:58:19-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7346865', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-21 18:12:12.107384', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-21T19:22:35.421 DEBUG:teuthology.nuke:shortname: smithi193 2023-07-21T19:22:35.421 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-21T19:22:35.444 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi193.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-21_17:58:19-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7346865', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-21 18:12:12.110764', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='} 2023-07-21T19:22:35.455 INFO:teuthology.orchestra.console:Power off smithi039 2023-07-21T19:22:35.456 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-21T19:22:35.487 INFO:teuthology.orchestra.console:Power off smithi155 2023-07-21T19:22:35.488 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-21T19:22:35.497 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T19:22:35.514 INFO:teuthology.orchestra.console:Power off smithi193 2023-07-21T19:22:35.515 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi193.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-21T19:22:35.527 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T19:22:35.540 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi193.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T19:22:39.538 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T19:22:39.554 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T19:22:39.566 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi193.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T19:22:43.652 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T19:22:43.667 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T19:22:43.678 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi193.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T19:22:47.767 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T19:22:47.779 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T19:22:47.789 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi193.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T19:22:47.979 INFO:teuthology.orchestra.console:Power off for smithi039 completed 2023-07-21T19:22:47.992 INFO:teuthology.orchestra.console:Power off for smithi155 completed 2023-07-21T19:22:48.001 INFO:teuthology.orchestra.console:Power off for smithi193 completed 2023-07-21T19:22:48.107 INFO:teuthology.run:Summary data: description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} duration: 3777.443857192993 failure_reason: 'error during scrub thrashing: rank damage found: {''backtrace''}' flavor: default owner: scheduled_rishabh@teuthology status: fail success: false 2023-07-21T19:22:48.107 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-21T19:22:48.192 INFO:teuthology.run:FAIL